branchrust_reboot/main/parellel_validatemovecancel
20 Commits over 0 Days - ∞cph!
Update: DemoServer - Isolated all client demo logic into it's own player
This is prep for full server demo support.
Tests: played back a short craggy demo - it went through the entire thing without issues. Tried without demo - it started as expected
Buildfix: removing unused variable
Tests: editor compile
Update: DemoServer - ripping out fixed timestep logic
After experimenting with slowing down playback to bellow play speed, it did reduce the number of violations, but timing inconsistency between demo playback and server simulation leads to more issues.
Tests: none, simple change
Update: DemoServer - implement fixed step playback
- Should keep demo stream consumption stable
Right now we're streaming too much data(at 30hz with ~200hz editor sim), which trips up a number of violation checks. Going to try tweaking the number to see if it helps with reproducable results.
Tests: played the demo twice - the step count was the same, but the result numbers were different.
▍▉▇▍▇█: ▅▄▉▋▋▇▉██▆ - ▅▋▍▆▌ ▆▌▉▍▋▆█ ▌█▋▆▌▄▋▄▋ ▉▌▅▇█▋▇▇▆
- █▌▌▌ ▍█▌▄▍▋▊▋▄▊▆ ▋▆▄▍██▉▊ █▉▊▍▋▍███ ▆▊▆▇▉▍▉▇ ▇▋▄ ▍▇▍▉ ▅▌▊▆▊▇▌▌▍▅.
- █▆▇█ ▄▅▉▇▆█▇ ▇▍▅▆▉▊▅▊▉ ▋▅▉▅▋▌-▆▅▋▄ ▇▌▇▆ ▅▋▆▌▆▊█▊▅█▅▌▍
█▄▅▊▋▅█ ▇▆▋▆ ▊▆▌█ ▉▅▅▉ ▍▊▄█▇▌▍▄ ▄▄▄▄▍▍▍▌▅ ▊▇▊-▋▍▌▌▍▍▅▄▋▄▄▄ ▇▉▍▇█▉▋(▌▋▄▇▆▆ ▊▆▋ ▇▄ ▆▍▋█▌▄▋ ▅▉▇▊▌▇▄▄ ▄▊▇▄▉█ ▊▌▉▌▊▊&▉▊▇▋█▋▋ ▋▅▆▍) - ▋▍▊▆ ▄█ ▆▍▋▆▆▆ ▍▌▋ ▋▊▊ ▅▅ ▉▅▇▅▉▅ ▊▆ ▅▊▆▍▋▋ ▉▄▍▆▊█▇ ▍▍ ▇▆▇▇ ▌▊▍▄▅▍ ▄▇▊▉█.
▊▌▋▄▋: ▅▍▇▉▆▆ ▊▌█▍ ▋ ▌▉▅ ▄▌▆▌ ▋▉▅▉▅ ▍▊▇▍-▉▉-▉▋▊▇ - ▅▌▇ ▍▌▅▊▄▅▊▅▆ ▅▋▉▉▉▇▄ (▄▉▊▉ ▌█ ▌▌▇▄ ▇▌▆▅▉█▆▊▍█)
Update: DemoServer - bypas failed validation
- We can't always reconstruct correct tick history in some situations, so instead we'll use them as data to compare against
Tests: played the opriginal long demo - 1711 total violations across ~18 players.
Update: DemoServer - rudimentary tick visualization using gizmos
- Temporrary while working on server demo reconstruction - will rip out once done with feature
Tests: used on a demo recording on Craggy + the original demo that started it all.
Update: DemoServer - spawn entities with the right initial flags
Turns out I had doors in a base to be closed on spawn, leading to tick violations - this fixes it. There's more violations to go.
Tests: Played the demo, checked that the relevant door is now open.
Update: DemoServer - hook up metabolism and make every player invincible
I thought metabolism would fix the drowning of main player, but the recording info contains empty oxygen. Instead, we treat every player as invincible unless there's a replication message to destroy them.
Tests: Played the demo till the end - no more logs on main player drowning
Update: DemoServer handles a number of RPC messages
- Only propertly implementing model flags for now
- Adding a bunch of RPCs to ignore to avoid heavy spam during playback
- Also renaming player game objects during playback to make it easier to track and inspect their state
This revelas that during playback we're triggering a bunch of tick violations, which prevents position updates. Need to figure out how to deal with them.
Tests: ran the same demo, this time with warnings not filtered out - once map loaded, the rate of warnings was decreased substantially.
Clean: removing no longer relevant comment
Update: DemoServer improvements and fixes
- All ticks are now accepted
- exposed an editor only API to inject ticks (avoid serialization roundtrip)
- cleaned away tick logging - it generated too many logs
Ticks are now caught, which is nice, but it looks like it's not validating them all outside of demo playback (saw only 2 players doing it on a perf capture). That'll be next.
Tests: added temp debug assertions that would catch any discarded tick - played the new demo, and there were no more assertions.
Update: DemoServer - split ticks by distance instead of time
- Splitting by time didn't guarantee that they were in valid distance ranges
- Also handle case where we get positional data while the player is still initializing
Getting closer, according to logs most ticks get accepted, but there are still a bunch that get filtered out - investigating.
Tests: played the same demo, observed the logs.
Update: Server-Editor tries to synthesize position ticks for other players in client-demos
- Also supports movement of other, non-player entities
- Only handling positions for now
Doing this to allow for more thorough testing. Some ticks get rejected despite being in the same position - need to investigate why.
Tests: played the same demo as before - checked logs to see the injection and acceptance of ticks.
Bugfix: more NRE reductions in server-demo
- Skip VoiceData and other messages that we can't support in editor environment (or don't want to)
- Properly "disconnect" player when entity is being destroyed
- "shutdown" the demo server when at the end of the demo to avoid unnecessary replication attempts/NREs
This brings down NRE count during playback and shutdown from 40+ down to 4. Next up need to figure out if Tick processing works correctly (it ticks, but main player doesn't move).
Tests: played back the same client demo, saaw the reduction in errors
Bugfix: no more duplicate players when playing a client demo on server-editor
Now there's an issue with disconnecting/destryoed players - about 8 NREs about acessing something dead during BasePlayer.ServerCycle
Tests: played the same demo - max players was 20 instead of 1k
Update: properly initialize players when playing a client-demo in server-editor
- Also log when creating a main player
- Report kick reasons as errors
No more unexpected kicks for players. But, looks like we're duplicating players - by the end of playback we had 1k players, which is much more than I expected
Tests: played back the same demo to completion, accumulated errors are only related to some invalid packets that we don't care about (like Voice)
Update: Server-editor is able to see ticks from the player when playing a client-demo
- now also handling flag messages
- skip server demos and warn user that it's not supported for now
There are a couple things left to investigate and validate - why the kicks happening for being under terrain, whether I can restore full initialization flow
Tests: ran the same demo, was able to verify that main player is identified and it's tick history is being stepped through
Update: Server demo playback now creates entities on palyers as it first encounters them
- Added demo progress logging
- Avoided a number of reasons for kicking (as we don't fully setup entity simulation)
I can see more activity now - next up is making sure the important history is also replicated/present.
Tests: played the same demo from before - logs confirmed players were present.
New: Editor-Server can playback a server demo
Mimics how client demos are played back - streams commands to the server for execution. Currently doesn't spawn players/has some entities missing - that's next to investigate
Tests: Took an old 5 min demo and played it back until it stopped the editor play session.
