branchrust_reboot/main/parellel_validatemovecancel
102 Commits over 31 Days - 0.14cph!
Merge: from main
Tests: none, no conflicts
Merge: from main
Tests: none, no conflicts
Merge: from main
Tests: none
Clean: ServerDemo - fix up names of entities that get reported via errors
Tests: played back procgen demo
Bugfix: ServerDemoPlayer - invert the error reporting logic for DestroyEntity
Previously, we considered it an error if we couldn't find an entity that demo stream tried to destroy. But, if the server behaves correctly, then it should be already destroyed - so now we report an error when we do find an entity.
Tests: played back the complex demo from procgen - errors still present, but harmless
Update: FullServerDemo - wrap transient entity snapshotting in try-catch
- Also correctly clean up NetWrite
I've checked a bunch of code related to saving, and I'm 100% certain there are landmines that I haven't spotted. This way I've neutered the danger of recording the demo, and I'll be fixing up those cases as we discover them in the wild.
Tests: recorded a demo on Craggy
Bugfix: FullServerDemo - use save for disk instead of network when snapshotting transient entities
- Also couple safety checks for BasePlayer.Save, since now I'm triggering a path that usually isn't triggered for a class of entities (NPCs)
Tests: while recording a server demo, on a procgen map in a local server did all basic activities (looting, harvesting, crafting and building) + drove a bike and attacked a ScientistNPC at outpost. Then was able to play back both demo chunks.
Tests: on local standalone server and procgen world, went to pier monument and descended to find aggressive NPCs - no more exceptions in server log
Update: ServerDemoPlayer - report errors if failing to find entities
Need to verify if my previous changes are working in more complex scenarios.
Tests: none, trivial change
Update: FullServerDemo - detect full network ID range used in demo
Replaces old invalid system of just reserving a constant chunk
Tests: played back all demos that I have - all valid.
Update: FullServerDemo - support client RPCs for transient entities
Implemented via an extra parse of demo stream, but should be okay since it's an editor-only process.
Tests: Played the same demo where I collected wood - previously it quietly failed, now player got attributed resources.
Update: FullServerDemo - try to handle transient(not in the savefile) Entities
- Record an entity snapshot once per demo chunk if sending specific messages - Positions, Flags, RPC(both server-sending and recieving)
- DemoTransientEntities are not sent if we've sent an entity snapshot via Entities message during active chunk
- Fixed not handling demo messages that can have 0 connections associated with it
- Fixed incorrectly aborting on "unknown" message type when types are related to demo-specific messages.
In theory this should allow us to avoid missing entities scenarios when playing back, which helps with determinism. But the server-recieving-RPC scenario needs more work - we record the transient snapshot after we recorded an RPC from the client, so I need to implement look-ahead in demo playback (or do what client demo playback does and index the demo).
Tests: on Craggy in editor recorded a full server demo where I'm looting a wood-collectible. During playback it spawns, but doesn't get removed
Buildfix: if-def out editor only variable
Tests: build standalone server locally
Update: adding missing comment about layerMask in batched CheckCapsule
Tests: none, trivial change
Bugfix: found another invalid scatter
Tests: none, will come next
Bugfix: Fixing incorrect scatter logic in batched GetIgnore
Really need to cover this path with tests
Tests: none, will come next
Clean: removing -Batch suffix from APIs
Doesn't really add any additional clarity, and makes the code a smidge shorter
Tests: editor compile
Merge: from main
Tests: none
Update: WaterLevel - rewrite WaterSystem GetIgnore serial chunk in batch form
This eliminates the other "jobify/batch" todo. Think I'll do a bit of cleanup, add more tests and get ready to merge to staging for stewing.
Tests: ran unit tests. Ran araound on craggy and checked a couple results of water checks.
Update: WaterCollision - add batch version for GetIgnore that does an overlap capsule test
Needed to get replace last serial chunk in batched WaterLevel.GetWaterInfos
Tests: none, compiled in editor
Update: GamePhisycs - added OverlapCapsule and CheckCapsure that use Unity jobs
Turns out I'll need it to support WaterCollision.GetIgnore(start, end...)
Tests: none, just compile
Bugfix: When spreading WaterInfo results, prevent trying to do a sawp between same indices
- This leads to invalidating WaterInfo, which returns invalid info
I need to extend tests to do checks with Entities, as this case/branch isn't covered currently
Tests: enabled parallel player updates and logged cached water checks - now it returns valid water where expected.
Bugfix: replace Temp allocators with TempJob allocators for physics jobs
- Added misisng resource cleanup
This partially fixes test failures.
Tests: ran the WaterLevel test checks
Update: WaterLevel.GetWaterInfos - replaced a serial GetWaterInfoFromVolumes call with a batched one
- should save us time by parallelizing physics checks, if they do get used
This removes one of 2 "batch" TODOs in the method.
Tests: ran unit tests, some are failing - will fix next.
Update: added batched WaterLevel.GetWaterInfoFromVolumes
- there's a number of inefficiencies - left TODOs to address later
Continuing to propagate batching support upwards, only got 1 level left to swap out a serial query with batched.
Tests: none, editor compile
Update: Added batched version of Vis.Components
Still building up to engage it as part of main parallel logic. Also still need to cover with tests.
Tests: compiles in editor
Clean: removing hanging meta files that I've commited earlier
Tests: open editor
Merge: from main
Tests: none
Update: Impllement GamePhysics.HandleIgnoreCollisions as a batch
- extended TerrainCollision to support batching
- extended WaterColllision to support batching
There's still more improvements that can be done (translating into burst jobs and better job-graph building), but currently the goal is to translate more of code into batch form away from singular.
Tests: none, it's not hooked up yet - will explore writing unit tests next-ish
Bugfix: read back from the right array of heights after sampling ocean
- Reported by new unit tests
Tests: ran water-related unit tests - now they pass
Bugfix: Make sure WaterLevel tests run full simulation path during unit testing
Previously it was earlying out during GetWaterLevel
Tests: ran the new tests, they report an issue that I need to fix
Update: missing meta files from merge
Submitting for now, but should nuke once back on main
Tests: none, trivial change
Merge: from main
- Skipping a couple meta files for empty folders, will submit them separately
Tests: booted editor for import
Clean: removing unused WaterFactorForPlayer that didn't return WaterInfo
Tests: compiles in editor in SERVER+CLIENT mode
Bugfix: ensure we cache WaterInfo for players that use various vehicles
Some vehicles have custom logic to check how submerged the player is, and we would miss the water info taht was used to calculate it.
Tests: tested via DPV - submerged factor was 1 and waterinfo was valid (before it would report factor of 1, but invalid waterinfo).
Update: BaseMountable.WaterfactorForPlayer can now return WaterInfo
- allows further reusing of the same water info down the line
I actually need it to make sure all our player's cached values are correct - will fix in next submit.
Tests: none (as it's not in use in this CL)
Clean: fixing profiling scope name
Tests: none, trivial change
Update: FinalzieTickParallel now uses batched WaterFactors query
- had to add more fork-join-style code to deal with batching
This allows us to sample water information for all relevant players as a burst job (and there's further untapped potential). Next up is trying to batch and jobify physics checks.
Tests: played in a solo session with parallel processing enabled - running, driving, swimming worked normal. Played back all of my server demos with analysis enabled - no new violations detected.
Bugfix: use the right allocator for a waterlevels batch
- This revealed that we have an Application.isPlaying check that prevented this from being detected via unit tests
Tests: ran unit tests (but they're useless in this case), ran all demos that use the new code(yet to submit) - no more exceptions
Update: able to get WaterFactors for players as a batch operation
- internally splits players into sequential queries(vehicles, parenting) and a batch query
Another building block covered, I should be able to rewrite caching of water factors step of FinalizeMove(next submit).
Tests: just editor compilation, I need to adapt rest of code to work with it.
Update: BufferList - new utility methods
- Resize - to explicitly control internal capacity
- ContentSpan, ContentReadOnlySpan - shortcuts to create a ro-/span over the active part of the buffer
Need both as I'm using it further in batched player processing operations
Tests: none, trivial changes.
Bugfix: setup water query params fully
0s are valid, but it caused the sampling positions to lean towards origin
Tests: rerun the WaterLevel tests
Clean: removing unnecessary logging
Tests: none, trivial change
Update: Able to grab WaterInfos as a batch
- Added a test to validate it returns same results as non-batched
- Made sure WaterLevel tests work in an initialized scene
It's not fully converted to a batch - there are a couple calls internally that are yet to be converted (game physics and the like).
Tests: ran the WaterLevel tests
Update: WaterLevel supports getting water heights for a batch of positions
- added a test to make sure batched produces same results as the singular version
Tests: ran the new unit tests
Update: Putting in a skeleton of logic for Parallel update
- Duplicated BasePlayer.ServerUpdate and FinalizeMove - rewrote to work on a list of players
- Added `server.useplayerupdatejobs` to control whether to use parallel or serial processing of players, default to off
New code doesn't really do much, just organized in a way to start working on parallel bits. I feel like I'm about to depart on a year-long journey into the jungle.
Tests: compiled in editor in SERVER-CLIENT mode
Update: ServerDemoPlayer - use time as progress indicator, instead of bytes of stream consumed
- Also removed a couple TODOs that are either done or decided to drop
In the previous version there would be no progress report until the next timestamped messages was reached - this is confusing UX.
Tests: played back the demo where I connect 15 seconds in, causing a jump from 0% to 18% progress. Now it's gradual.
Update: DemoServer - don't try to launch it in SERVER+CLIENT mode
That's a can of worms I don't want to deal with right now
Tests: none, trivial change
Clean: DemoServer - allow configuring the server for demo playback
Tests: none, trivial change
Update: ServerDemoPlayer - added error logging on hierarchy desync
Since some messages are handled by hand, it's possible things can go wrong in the future - this should catch it.
Tests: played all demos I have, no errors reported
Clean: removing no longer needed method
- Was previously part of ClientDemoPlayer
Tests: builds in editor in SERVER mode
