495 Commits over 123 Days - 0.17cph!
Bugfix: patch sheared frames when exporting json snapshot
- In Editor EndOfFrame is called as part of nested GUI, which our profiler shears apart. For now we inject additional marks to maintain callstack structure
- left a TODO to properly reconstruct a sheared frame
- For now expanding frame scope to cover sheared period
Tests: exported craggy snapshot and opened in perfetto - no more randomly trashed frames
Update: Factor out profielr exporting logic to a separate script
- Also cleaned up a couple log outputs, as the collection seems sensible
- Cleaned a couple already-done TODOs
Tests: Did an export from editor and standalone server (6k size, 0 pop), loaded in perfetto
Undo: removing ignore.conf accidental submit
Tests: none
Update: ServerProfiler snapshot now contains full names for scopes
Tests: opened the new profile in perfetto
Update: Emit all 10 frames of snapshot
- Also emitting UnityFrame as a CompleteEvent, so it's always visible in the profiler
- UnityFrames are now numbered
This produces a 160mb snapshot on Craggy (taken immediately after spawning), will need testing on larger worlds (and this is before proper labels)
Tests: opened snapshot in perfetto - saw all 10 frames
Update: Synthesize missing OnEnter profiling marks
This restore the profile's structure
Test: opened snapshot in perfetto
Update: first export of profiling snapshot
- Also filtering out System and System.Core from annotation (otherwise we produce too much data, even for 1 frame)
- Also added timestamp utility functions (as trace format requires micros)
Not super reliable - need to post-process gathered data to get rid of torn marks
Tests: opened trace in perfetto - although wonky, it's there
Update: minor improvements to ServerProfiler
- Added a smidge more debug to track the growth of per-frame lazy storage
- Added output of how long each frame took
Tests: ran the perfsnapshot command
Update: add profile.perfsnapshot command
Currently just outputs gathered marks telemetry for sanity checking - will expand more once happy with results.
Tests: ran the command and confirmed output
Update: more improvements and debug to ServerProfiler
- Added tracking of thread IDs
- Only gather main thread marks (have thread races in my lock-free approach, so delaying this till later)
- Add tracking of assemblies to which the method belongs
- Filter our mscorlib assembly from instrumenting
- rewrote code to use ref semantics
Still chasing why consecurtive snapshots seem to double-up in size on frames 2-10.
Tests: ran a bunch of profile snapshots (next frame) and looked at the output. It works but has above weird behavior.
Bugfix: various ServerProfiler fixes and improvements
- Account that native strings are not null terminated when doing unmanaged string comparison
- Filter out native profiler funcs from annotation
- Unregister profiler callbacks when stopping play in editor (this would cause mono crash on repeat plays due to GCed callbacks)
- Reduce scope of unsafe keyword (as some funcs are safe)
- Replace test-run annotation with fully enabled one
- Hook in on frame end to count frames
Tests: Started, stopped and started the game in the editor. Wrote some simple test code to validate StrEq. Validated in logs that native serverprofiler functionality is ignored.
Update: Filter instrumenting anotations based on class and namespaces
- Added reporting of class and namespace into annotation log
Currently only skipping ServerProfiler itself to avoid a recursion when invoking callbacks (will be tested in next submit).
Tests: pressed play in editor - saw the instrumentation logs
Update: Invoking of profiler method annotation filter works
- Instead of hooking in we rely on Unity's assembly initialization to invoke us first - seems to be early enough, but we'll see
- Store profiler callbacks to avoid GC cleaning them up and crashes in mono
- Add compile guards (for now allowing to run editor to test)
Tests: launched in editor, no crashes and a lot(112k) of debug logs indicating what gets annotated.
Update: skeleton of a server profiler
- Most of the native imports in place
- Implemented the general flow of per-frame tracking
In theory should be able to track function entry/exits, but can't test until a decide on an early enough method of initialization.
Tests: none, too early
Merge: from main
Tests: none (trivial merge)
Merge: from remove_editor_update
- Removes last editor-only update invoke case with many invocations(saves a measly 0.1ms).
- `demo.play` accepts absolute paths
Tests: setup an industrial chain with a chest provider and a crafter and linked those up - debug vizualization worked. Loaded a demo outside of demos folder.
Update: demo.play respects absolute paths
Tests: Tried to play a demo outside of demos folder
Optim: Remove IO pipe debug update invoke
Turns out to save much less than expected (just 0.1ms), but this eliminates last editor-only update invoke with many instances.
Tests: On craggy setup a chest->producer industrial circuit, was able to visualize debug
Merge: from main
Tests: none
Merge: from main
Tests: none(trivial merge)
Merge: from remove_editor_update
Removes around 1.5ms from 6k world in the editor and fixes longstanding bugs with the decal editing.
Tests: Created a bunch of decals, modified their properties(changes visible), deleted them(no NREs), ran in game - still visible.
Merge: from main
Tests: none(trivial merge)
Bugfix: avoid exceptions when modifying decals in editor
Tests: on craggy spawned a bunch of decals, modified them in various ways, deleted them - no exceptions
Optim: Remove editor-only per-decal Update callback
- They've all been consolidated to DeferredDecalRenrerer by tracking Undo history changes
This saves about 1.5ms during Update stage (in reality it's a bit less because of profiling overhead).
Tests: On Craggy ran around and validated I could still see footprints. Spawned them manually and tweaked settings and moved it's placement(ok) - trying to delete placed decal starts an exception loop(pre-existing bug, will fix next)
Merge: from main
Tests: none (trivial merge)
Merge: from vehicle_optims
Saves 0.5ms on a 6k world by avoiding iterating over unmounted mountables
Tests: On craggy drove a kayak. On 6k world confirmed that the profiling scope shrunk to 0
Merge: from main
Tests: none (trivial merge)
Optim: Avoid trying to sync non-mounted mountables
- Implemented via a mounted-queue that gets populated when a vehicle/animal is mounted
This saves 0.5ms on a 6k world map with no players riding vehicles.
Tests: on Craggy rode a kayak and validated player transform sync was happening via profiler. Loaded 6k world and validated that the original 1.3k item queue was gone.
Update: Reducing access to BaseMountable internal state
- Mods can workaround by using existing API
Future optim depends on internal state being tightly controlled, so this makes it easier to prove that it'll be safe.
Tests: rode the zipline on craggy
Merge: from profiling_improvements
Removes boxing allocations in Pool.Get and Pool.Free in Editor context.
Tests: in editor on Craggy started a normal session, couldn't see allocations in the profiler. Overrode the startup params and was able to see the overhead table printed out when invoking cmd.
Merge: from main
Tests: none
Update: Make Pool capacity waste tracking off by default
- Added an editor-only command to enable it, "pool.memory_overhead_tracking_enabled"
- Cleaned up implementation a smidge, left a comment explaining where the box is coming from
- "pool.print_memory_overhead" skips entries with 0 overhead
I couldn't find a way to avoid boxing in a generic call, so I decided to disable this telemetry gathering by default to avoid muddying profiler data.
Tests: in editor on Craggy started a normal session, couldn't see allocations in the profiler. Overrode the startup params and was able to see the overhead table printed out when invoking cmd.
Merge: from main
Tests: none
Update: changing starting scene to be Bootstrap when running a perf test
- Also removed direct world setup settings modification, instead pushing them to global World static params
We have to go via Bootstrap because that's what initializes global systems. Will need to massage platform-specific transition logic to account for perf testing workflow (need to support CLIENT+SERVER flow).
Tests: Test player fails because of CWD, but if I run TestPlayer manually it boots and loads to main menu, so I count it as pass-ish
Update: Symlink assetbundles on test player build
Tests: ran the test, validated that they got symlinked to expected path (but there are other issues, investigating)
Update: Sneak in console cmds via command line
Need a better way in the future, but for now this should allow to control test conditions.
Tests: none (need to fix assetbundle copying first), though validated it sohuld work by reading through code
Merge: from main
Tests: none
Merge: from main
Tests: none
Merge: from profiling_improvements
Just additional profiling annotations to fill in voids on FixedUpdate and UpdateLOD. Also editor microoptim in GamePhysics.
Tests: Rode a horse on Craggy and checked profiler for changes
Merge: from main
Tests: none
Update: Gamephysics now uses TryGetComponent instead of GetComponent
- Same perf in standalone builds, better perf in editor builds (avoids string format alloc for null obj description)
Tests: Rode a horse on craggy and check profiler to confirm GamePhysics routines were triggered
Update: Adding missing ChangeLOD profiling annotation
Tests: loaded 6k world and found it in profile data
Update: Annotating all VehicleFixedUpdate calls
- Replacing existing Profiler.BeginSample with TimeWarning
- Moved some samplers around to build a nested flamegraph (consolidates vehicles in the profile)
Tests: loaded 6k world and looked through the profiling info
Merge: from remove_editor_update
This shaves off another 5ms from editor update times on 6k world map
Tests: built all modes in editor, tested debug rendering still works on craggy and loaded a 6k proc map and checked profiler
Merge: from main
Tests: built all modes locally
Buildfix: Hide DDraw calls behind CLIENT
Tests: checked all build modes
Update: DDrawAIDataPoints is an editor server var (instead of client)
- Also fixed formatting and borked whitespace on one line
This makes it a bit more consistent with how it's implemented (since ServerMgr executes the drawing).
Tests: On craggy spawned junkpile_j, enabled the flag - debug rendering worked.
Optim: Reimplement AI-Cover/Move-Point debug drawing as a utility function instead of FixedUpdate
- Also now requires both CLIENT+SERVER being active in the editor (as it's part of ServerMgr's FixedUpdate step)
- Will reconcile the client cmd line arg but server logic in next CL
This wins us 5.5ms on 6k procedural map in editor
Tests: loaded a 6k procgen map on 1337 seed and checked the profiled. Tried enabling the debug flag, but it nukes the perf. Tested flag on craggy by spawning junkpile_j - saw the debug graphics.
Merge: from remove_treeentity_update
This eliminates the many calls of TreeEntity.Update in editor play mode, saves ~16ms on 6k procgen map
Tests: booted editor with swamp_a scene and shut down, booted into craggy then switched to swamp_a, added and moved trees to swamp_a, changed tree prefabs - all cases could see expected visuals.
Bugfix: TreeToolRenderer can render on editor start
- Replaced sceneOpened with activeSceneChangedInEditMode, as former didn't react to boot process changing scenes
Tests: closed editor and reopened, trees started to render
