992 Commits over 274 Days - 0.15cph!
Bugfix: Use valid index in WaterTestFromVolumes
Tests: detected during staging demo playback with useparallelupdatejobs - reran the demo, no more NREs
Merge: from parallel_validatemove
- Fixers a couple rare bugs leading to missing data from FullServerDemo recordings
- More work on BasePlayer.SErverUpdateParallel, still disabled
- Editor-only: Added a couple unit tests
- Editor-only: ServerDemoPlayer - disable error spam during demo playback, improve log format
- Editor-only: ServerDemoPlayer - automatically authenticate connections during demo playback
Tests: played back demo from staging server, recorded a couple new demos in local editor
Merge: from main
Tests: none, no conflicts
Bugfix: Initialize WaterSystem coarse grid on clients
- Also make it safe for scenes that don't have a water setup (like playground)
Tests: tested with a separate client connecting to server - saw that Client was initializing everything correctly in the right order
Bugfix: ensure WaterCollision grid is setup with right terrain dimensions
Tests: In editor, started on craggy - size matched. Started on procgen - matched. Played demo - matched. In all cases validated initialization order to confirm no colliders/volumes were added too early. Exported PNG of the grid.
Bugfix: FullServerDemos - transient entity recording fixes
- Use DemoCount instead of ChunkIndex when determing to send snapshots - ChunkIndices reset, so they can rarely overlap for an entity and cause it to not send a snapshot
- Reset counter on prefab pooling - previously it could cause skipping of transient entity snapshotting
Tests: recorded a bunch of demos in editor
Bugfix: ServerDemoPlayer - skip the auth flow when playing the demo
- also index packet logging messages - makes it easier to sort out order of things in editor logs
Seems the simpler thing to do for now.
Tests: played back staging demo - much less error spam
Update: ServerDemoPlayer - add Ready message logging
Tests: none, trivial change
Update: ServerDemoPlayer - log more message types
Tests: ran demo from staging server
Update: ServerDemoPlayer - switch to control error reporting
Tried the demo from staging, the error spam is too much and not super helpful, so adding a toggle to disable it.
Tests: Played back the staging demo
Optim: Avoid handling null cases in a batch with only non-null values
- Also updated the test to spawn entities, since now it's a requirement for the func.
This removes the need to juggle data to setup batch operations. Should save a bit of time.
Tests: Ran the updated unit tests
▄▇▅▆▇▊: ▇▆▆▊▉▍ ▄▋ ▊▉▆▇▅▉▋▇▉▄▅▋█▉▆ ▋▇▆▍█ ▅▊ ▅▅▋▌▅▅▄▍▄▆▍█▅▆▋█▇▉▆▋ ▌▅▄▊ ▅▌▍▉▆▇▍▊▉▄▇▄
▉▆▇▊▆: ▉▄▌▌▄▇ ▍▄▌▌ ▇ ▅▇▋▄ ▅▅▊▇ ▅▉▋▊▇▇▌▅▌▋▌▄▅▊▉▆▄▆▋ ▅█▄ █▇ █
Merge: from main
Tests: none, no conflicts
Merge: from profiling_improvements
- makes the Linux binaries compatible with more distros (Ubuntu 20.04, Debian 12)
Tests: ran in Ubunti 20.04 and 24.04 via WSL. Took snapshots and opened them in perfetto.
Update: ServerProfiler now usable on older Linux distros
- Binaries built using revision 45d79338
Should work on Debian 12.
Tests: Ran on WSL Ubuntu 20.04 and Ubuntu 24.04 and took snapshots - all worked.
Merge: from main
Tests: none, no conflicts
Merge: from profiling_improvements
- Fixes ProfilerBinViewer to display all available threads and fix invalid callstack depth calculation
- Fixes a bug that would prevent json from being generated on busy servers
- Fixes a bug with timelines being very-slightly out of sync
- Optim/Bugfix to filter out all constructors from being profiled
Tests: a lot of exports in the edittor and a bit of forced "bad" cases
Bugfix: ServerProfiler - properly filter out constructors
- Due to a typo (missing .) it would never match constructors and never filter them out
Using release libs built from ec8c5522
Tests: snapshot in editor on Craggy - confirmed no contructors were recorded.
Bugfix: ServerProfiler - sync up non-main-thread timelines to main thread
Previously it was possible to have a small gap at the very start of the snapshot on non-main thread views.
Tests: exported a snapshot in the editor.
Bugfix: ServerProfiler - properly handle sheared callstack at the start of worker threads during export
This revealed that I have a bug with non-main thread timing (things are slightly offset). Will fix next.
Tests: injected data that was tripping up the export originally. Validated that it does trip up before fix, and now exports correctly after fix. Exported 10 snapshots in the editor - no failures, and all look correct.
Bugfix: ProfileBinViewer - fix callstack depth calculation
- Also replace RadioBoxGroup for threads with a Dropdown - turns out 50 threads radio buttons don't vertically fit my monitor. Who knew.
Hoping it's the same issue in the json exporter.
Tests: Opened a debug snapshot from official server.
Merge: from main
Tests: none, no conflicts
Bugfix: CollectionUtil.SortInplace now correctly sorts
- Also renamed these utils as previous names were confusing
Tests: ran the old + new unit tests - all pass
Update: Adding unit tests for CollectionUtil
Discovered that my SortInplace is borked, so I'll fix it in the next update.
Tests: ran the new unit tests
Update: consolidate various scatter code used in batched WaterLevel and GamePhysics checks
Will cover them with tests just to be extra safe.
Tests: ran WaterLevel and GamePhysics unit tests
Update: Name a couple magic constants
Tests: none, trivial change
Clean: rogue newline
Tests: trivial change
Update: TerrainCollision.GetIgnore(Span...) now uses the ignore grid broadphase
Tests: ran new unit tests that go through this path
Update: new unit tests for GamePhysics.HandleIgnoreCollisions
- Also removed an unnecessary branch
Tests: ran the new unit tests
Merge: from main
Tests: none, no conflicts
▍▌█▇█: ▅▅▇▍ ▋▅▊▄▊█_▌▋▌▊█▌▊▋▌▉▋_▄▅▆▇▆▉▉
▄▍▅▇▅▌ ▆▇▇ ▆▌▉█▅▊ ▊▋▅▌▌▉ ▍▇▅▊▌▊▅
▍▅▊█▉: ▅▄▋▇ █▉▍█ ▅▉▇▌▆▄ ▉▇▉ ▉ ▅▇▋▍▄, ▊▍▊ ▇▅▄█ ▅▊▇▍▇▅▍ ▇▆▉▍▇█ ▇▍▆█▋▅ ▌▇▊█ ▅▌ ▅▄▄▇▌▆▌ █▊▇ ▊▉▇▋█▇.
█▉▋▅▍▆: ▊▌▌ ▇█▌▇▅▋▅ ▇▆▊▄▋▇▉██▄▅ ▉▍▉▌▇▅▍ ▉▄█ █▊▊ ▊▄▋▋▄▉▌▊
- ▉▉▍█▅ ▇▋▄▄▇ ▋▌ ▅▄▍▇█▅ ▋█▉ ▊▊▌▉▄█▊ ▊▌█▌▆ █▄ ▇▉█ ▅▋▆▉ ▄▅▋▌▉▉▅ █▆ ▌▌█▅▌ ▌▅▍▅▌▄▇▅ ▌▌▊ ▊▄▉▋, ▄▍▍▅▍▉▉ ▄▋ ▉▍ ▉▆▉▉▄▌ ▇▅█▇▉.
▄▍▅▅ ▌▆▌▅▌█ ▉▊▅▍ ▌▆▅▇▇▅ ▅▊▌█▅▊ ▇▌▌▆▍▄█ ▉▌▄▍ ▌▊▆▅▉▊ ▉▍▊▇▊▅▉▆▋▅ ▆▇█▄ ▋▄▆▍▄▇▊█ ▉▅▊▄ ▋▆ ▇▊▅+ ▍▉▍▇.
██▆▋▊: ▆▍▅▍▆▆ █▍▆▋ █▇▅▆ ▄▅██ - ▋▉█ ▆▍▅▍ ▆▋ █▋▆▊▄▆▇▅ ▊▆▉ ▍█▊▌ ▍▉▅▋▌. ▇▍▅ ▄▍█▅ ▆▍ ▍▄▍▍█▄▅ ▄▄ █▅▋ ▄▇▍▍▉▇▍ ▉▇█▄▅█ ▇█▉ ▆▊▉ ▄▅▅▊▆▊. ▍▍▊▊ ▍█ ▋ ▊▅▇▊▆▄▇▅ ▆▊▊▉▄▋ ▉▄▉ ▉▌ █▌▄▉▍▊▊▆▊ ▇▇▍ ▇▍▆ █ ▇▄▌▇ ▅▆▇▋ ▉▉ ▅▅▌▇▍▄▄▅▄▌ ▊▉▇▍▉▌.
Merge: from parallel_validatemove
* Modifies Full Server Demo recording to grab more data + timestamps per packet
* Adds editor-only "DemoServer" server backend that we can use in editor to play back full server demos. Switch editor to ServerMode and put path in GameSetup object
* New server-side batch-update of players routine - disabled by default as it's not validated yet. Controlled by `server.UsePlayerUpdateJobs`
* Added GamePhysics batch versions of OverlapCapsule and OverlapSphere
* Couple unit tests to check Water's batch queries against non-batched versions
Tests: A lot of server demos recorded and played back in the editor doing core activities (looting, harvesting, interacting). Made sure standalone server and client builds locally.
Merge: from main
Tests: none, no conflicts
Merge: from main
Tests: none
Clean: ServerDemo - fix up names of entities that get reported via errors
Tests: played back procgen demo
Bugfix: ServerDemoPlayer - invert the error reporting logic for DestroyEntity
Previously, we considered it an error if we couldn't find an entity that demo stream tried to destroy. But, if the server behaves correctly, then it should be already destroyed - so now we report an error when we do find an entity.
Tests: played back the complex demo from procgen - errors still present, but harmless
Update: FullServerDemo - wrap transient entity snapshotting in try-catch
- Also correctly clean up NetWrite
I've checked a bunch of code related to saving, and I'm 100% certain there are landmines that I haven't spotted. This way I've neutered the danger of recording the demo, and I'll be fixing up those cases as we discover them in the wild.
Tests: recorded a demo on Craggy
Bugfix: FullServerDemo - use save for disk instead of network when snapshotting transient entities
- Also couple safety checks for BasePlayer.Save, since now I'm triggering a path that usually isn't triggered for a class of entities (NPCs)
Tests: while recording a server demo, on a procgen map in a local server did all basic activities (looting, harvesting, crafting and building) + drove a bike and attacked a ScientistNPC at outpost. Then was able to play back both demo chunks.
Tests: on local standalone server and procgen world, went to pier monument and descended to find aggressive NPCs - no more exceptions in server log
Update: ServerDemoPlayer - report errors if failing to find entities
Need to verify if my previous changes are working in more complex scenarios.
Tests: none, trivial change
Update: FullServerDemo - detect full network ID range used in demo
Replaces old invalid system of just reserving a constant chunk
Tests: played back all demos that I have - all valid.
Update: FullServerDemo - support client RPCs for transient entities
Implemented via an extra parse of demo stream, but should be okay since it's an editor-only process.
Tests: Played the same demo where I collected wood - previously it quietly failed, now player got attributed resources.
Update: FullServerDemo - try to handle transient(not in the savefile) Entities
- Record an entity snapshot once per demo chunk if sending specific messages - Positions, Flags, RPC(both server-sending and recieving)
- DemoTransientEntities are not sent if we've sent an entity snapshot via Entities message during active chunk
- Fixed not handling demo messages that can have 0 connections associated with it
- Fixed incorrectly aborting on "unknown" message type when types are related to demo-specific messages.
In theory this should allow us to avoid missing entities scenarios when playing back, which helps with determinism. But the server-recieving-RPC scenario needs more work - we record the transient snapshot after we recorded an RPC from the client, so I need to implement look-ahead in demo playback (or do what client demo playback does and index the demo).
Tests: on Craggy in editor recorded a full server demo where I'm looting a wood-collectible. During playback it spawns, but doesn't get removed
Buildfix: if-def out editor only variable
Tests: build standalone server locally
Update: adding missing comment about layerMask in batched CheckCapsule
Tests: none, trivial change
Bugfix: found another invalid scatter
Tests: none, will come next
Bugfix: Fixing incorrect scatter logic in batched GetIgnore
Really need to cover this path with tests
Tests: none, will come next
Clean: removing -Batch suffix from APIs
Doesn't really add any additional clarity, and makes the code a smidge shorter
Tests: editor compile
Merge: from main
Tests: none
Merge: from profiling_improvements
Further reducing overhead by recording 25% less data overall (based on 350p release server snapshot)
Tests: Took a snapshot on Craggy in Editor
