574 Commits over 153 Days - 0.16cph!
Buildfix: define symbol on Mac Server
Tests: compiled editor, then compiled linux DGS
Merge: from profiler_improvements
- Adds linux support (tested on Ubuntu
22404 via WSL)
- Optimizations for JSON export
- Added debug utility to export binary snapshot - run `perfsnapshot <delay> <name> <frames> <shouldBinExport>`
- Added Tools/Profiler Bin Viewer, an editor only tool to inspect binary snapshots
- Reduced default frames captured to 4 from 10
- Profiler now skips annotating UnityEngine.CoreModule methods (reduces capture overhead)
- Works around Perfetto visualization issue with Complete events (https://github.com/google/perfetto/issues/970)
Tests:
- Exported a number of editor snapshots with binary snapshots to test bin viewer
- Using WSL, tested exporting a snapshot on Ubuntu - 3k procgen world
Merge: from main
Tests: editor compiles
Bugfix: Workaround Perfetto's "Complete" event hierarchy bug
- Reported issue on their repo: https://github.com/google/perfetto/issues/970
Tests: exported snapshot from a linux server (running on WSL Ubuntu), 3k procgen world. Exported from editor as well.
Update: Binary export no longer pre-processes the stream
- Saves time on the export
- Also added if-deffed out extra checks, disabled by default
My previous checks were wrong and produced false positives. Also, think I got an idea what jumbles the json vizualization - will fix in next CL.
Tests: used the extra-debug version to export linux snapshot - it succeded
Update: ProfileBinViewer - report found exceptions in thread stream
Still looking for why things are wrong with linux snapshot
Tests: opened a borked linux snapshot
Buildfix: Disable ProfileBinViewer if we're not in Server mode
Tests: switched to Client in editor
Update: ProfileBinViewer now shows thread summary
Tests: opened a snapshot from editor
WIP: rewriting stream processing to gather frame data during pre-process step
- Should fix invalid make placement in a frame + be more efficient to generate, as we only do 2 stream scans per thread stream instead of 4.
Not complete, need to track down why alloc offsets are invalid for last frame.
Merge: from amvienceemitter_recycle
Fixes an inconsistent bug on client disconnect from a server trying to reactivate a gameobject.
Tests: validated it doesn't affect entity pool warmup sequence (as we create->retire there). Using `log.level Audio 2` and a bit of 100% code-forcing the issue, disconnected 3 times:
- without the fix, it was 100% generating an error on disconnect
- with the fix, had 0 error reports
Bugfix: don't try to reactivate a destroyed spawnpoing in SERVER+CLIENT
- Reimplemented the spawn point status based on new internal state - this retains original behavior
Editor-only bug caused by my previous change of pooling behavior (destroy object instead of trying to pool it on scene unload).
Tests: on Craggy spawned and disconnected - no errors. Checked BaseSpawnPoint inheritance hierarchy to ensure there's no other places trying to activate gameobjects
Merge: from main
Tests: compiled in editor
Update: added search support to bin snapshot viewer
I think I have all I need to explore the broken profile
Tests: opened the borked profile snapshot
Bugfix: fixed reading string f rom the binary snapshot
- Forgot that they're not null terminated - this fixes random characters at the ends
Tests: Opened borked editor snapshot
Update: added ability to display sub range of thread track in Bin viewer
- also supports rudimentary [N] input to resolve syncpoint indices
- added mark index to view as well
Tests: vizualized borked editor snapshot
Update: display call depth for marks in bin viewer
Makes it easier to track callstack consistency at sync points.
Tests: opened borked snapshot from editor
Update: reworked the bin vizualizer to have a different layout
- Able to jump to sync points in the list
- Able to view specific thread's stream
Couldn't figure out how to do nested dynamic scrollviews, so went for a different approach. Already revealed a question mark about some names having invalid characters at the end, though doubt it's the contributing factor
Tests: loaded up a borked binary snapshot, was able to inspect it
Bugfix: don't double up threads in bin viewer
Also got lucky and captured a snapshot in editor where frames weren't properly aligned.
Tests: opened an existing snapshot, saw no duplicate threads
New: Editor viewer for binary profile snapshots
- Very rudimentary, needs more work
- Also added how many marks there are in a thread profile for binary snapshot export
Reveals that I have a bug in binary exporter - looks like I double up the threads somewhere.
Tests: opened a snapshot from the editor in the tool
Update: Exposing binary exporter
- Added missing features from ServerProfiler.Core
- Can be triggered via profile.perfsnapshot last argument
I need it to be able to investigate and fix hard-to-reproduce issues - hopefully it'll speed up the workflow.
Tests: in Editor on craggy took a binary snapshot - it exported succesfully
Bugfix: don't access invalid memory when exporting Linux snapshot
Same GCC vs MSVC issue in the native libs, but this time on Managed side (since I have a copy of these structs for name resolving purposes).
Tests: Built a linux server, ran it on WSL and triggered a snapshot - it generated. But periodically select frames export incorrectly, investigating further
Bugfix: new ServerProfiler.Core dynamic libs
- Fixes Linux exporting symbols with name mangling
- Fixes GCC vs MSVC struct packing inconsistency, causing Linux server to crash when instrumenting functions
Tests: DLL tested in editor on Craggy, SO tested in WSL standalone server (snapshot export fails though)
Merge: from mapuploader_retry_exc
Fixes an exception when map uploading fails and it tries to incorrectly retry.
Tests: in editor's procgen server - hardcoded a failure point on first attempt, 2nd attempt managed to send it off properly.
Bugfix: Recache map uploading stream to avoid using a disposed one
In the rare event that the first request was sent & failed, it would consume and dispose the stream linked to it, preventing us from using it in the next retry.
Tests: in editor's procgen server - hardcoded a failure point on first attempt, 2nd attempt managed to send it off properly.
Update: borked linux ServerProfiler.Core
It gets loaded, but fails to find an entry point, so it just falls over. I'll continue tomorrow.
Tests: built a standalone linux server and tried launching in via WSL - my profiler script throws an exception
Update: ServerProfiler.Core separates platform specific code
Required for Linux support - still figuring out how to organize msbuild projects
Tests: took spanshot in Editor's Craggy, and did a couple start-stops of playing to ensure no errors
Clean: fixing typo in "ServerProfiler::AppendNameTo"
Noticed when working on now-discarded assembly name skipping
Tests: scripts build in editor
Clean: remove unused "Profile.cs" script
Tests: made sure ditor compiles
Update: don't annotate UnityEngine.CoreModule funcs
It has a lot of small functions, so it inflates the profile by quite a bit as well as adds overhead to small loops.
Tests: tested in editor on Craggy
Update: Reduce default frame count for perf snapshot to 4
Going to max(10) by default is too much - on a 150 pop release server that tried to produce 3.5GB json snapshot. I'm expecting 4 to give us a 1 GB profile
Tests: took a snapshot on Craggy in editor - it took 4 frames
Update: Reduce main buffer relocations when processing generating json snapshot
- Also report total marks instead of max to allow predicting how big the snapshot is in case of failure
Will help with reducing GC activity on the server, hopefully also reducing the generation time.
Tests: locally on Craggy in editor took a couple snapshots - no issues
Merge: from main
Tests: none
Bugfix: Prevent game-object pooling if done mid-scene switch
- When MonoBehaviours are being destroyed due to scene switch, it can cause an invalid transform hierarchy modification, causing an error
MonoBehaviors can clean up their resources in OnDisable, which can cause them to be pooled - example: AmbienceWaveSounds.
Tests: validated it doesn't affect entity pool warmup sequence (as we create->retire there). Using `log.level Audio 2` and a bit of 100% code-forcing the issue, disconnected 3 times:
- without the fix, it was 100% generating an error on disconnected
- with the fix, had 0 error reports
Merge: from main
Tests: none
Clean: reduce code duplication for BasePlayer.WaterFactor
Tests: none, trivial change
▅█▆▋▊: ▊▌▌▍ █▆▉▍█▊▍▊▌▌▉▅_▋▆█▊▇
▄▉▌▆▇▆ ▆▇▄█▉▆ ▄ ▅▅▄▌▊ ▇▍ ▆▍█▄▍▌▋ █▍ ▇▅▋▄▆▊▆▄▋▅ ▄▆▊█▆█▇ - ▋▋▇▅▍▊ ▅▋ ▄▄ ▄▋▆█▉▋ █▊▆▋▆▌▉▍::▍▄█▇▊▊█▆▊▊▅▍ ▅▌ █▉%, ▊▇▆ ▉▆▇▉▋█▉▅▌▄▊▅ ▍▆ █ ▆▌▅▌▄█ ▊▌▄▊ (▇▉▊▊▆▋ ▅▌▊▍ ▋▉ ▉▋▌▅▉▊▆▄ █▅ ▋ ▋▊▊▇ ▌▇▋▍▇▆)
▌█▍▌▉: ▉▇▉ ▇▇ ▆▊▊▆▇▍, ▌▍▍▊▄▅▅▌ ▌▍▊▇▇▉▌▄▇▇ ▋▇▍ ▆▍▄▌ █ ▅▋█▄▄▇ ▍▌ ▉▌▍▋ ▅▊▆▄▄▉▍▍▆ - ▌▉▆▇▆▉▄▄▉ ▍▌▋▊▄▇█▌ ▍▆▇█▍ ▍▄▊▅▄▅▌▆▇▄▍ ▆▇▆ ▅▋▋▋▍▋ ▍▍▅▍▉▉▌ ▊▅ ▉▋▊▋ ▌▊▅▋ ▄▉▍▆
▉▉▅: `▅▆▄█▇▉▇▄.▅▍▍▉▄▇▍_▌▇▋▄▋▇▇██▊▍▌▅█` ▄▇▍▆▄▆▉ █▌▅▅▇▋ ▇▆ ▌▅▊▊▉▉▌ ▇▉▋▋▋ ▌▉ ▋█▋ ▅▉▅▊▍▆▄▅::█▇█▉▌█▊▄
- ▅▉█ ▇▅ ▊▍▅ ▍██▇▉▋▌ ▅▋▇▅▅▅▊, ▍▍▊█▅▅ ▇▋ ▊ ▇▄▊ ▊▄ ▆▄▊█. ▊▍▊▉▌▊▊▋ ▉▇ ▌▉▇▉▌ ▉▊▉▊▋▉ ▋▋▅▌▅▆▌▄▌▌▊.
▊▊█ ▆ ▍█▄▍▌▇ ▊▄▍ ▄▍▉▋▅▇▊█ ▍▍▍ ▅█ ▋▊█▋ ▊▋▉ ▅▌▊ ▄▊▉ ▅▉ ▋▌▅▅▋▅▋.
▅▋▄▌█: ▅▆▋▋▉▄█▋ █▉ ▄▋▌█▌▅, ▇█▅ ▊▆▅▋▄▋, ▋█▇▍█ ▋ ▋▌▆█▌▄▍█▊▄ █▄ ▆▌█▆▆▇▉▌ ▆▍▆▆ - ▍▉▋▄▋▉▅▋▍ ▌▆ ▍▉▄▄▄█ ▍▉ ▉▋▅█▅▆▉▅
Update: comments
- fixed spelling mistake
- added an explanation for invalidation of cached tick state
Tests: none, trivial change
▄▊▇▇▇▇: ▅▉▋▄▊▄ ▆▌▊▉▍ ▆▄▌▆ ▉▌▉▄▉ ▆▄▄▆ ▌▌▌▌▅▊█▊ ▆▍ ▄▄▌▆██▄ ▅▍▆▄▋▍
- ▌▍█▋ ▌▆▋▄▅ ▉█▆▌▅▇▊▄▄ ▇▍ ▅▌▆▋▉▅▉▉ ▅▅▌▅▅▍▄, ▍▊ ▌▉ ▇█▌▌▉▇ ▄▉▇▊█▌▅▊▋ ▌▍▊▋ ▅▊▋▆▍▄ ▊▆▅ ▌▄█▍▆▉▄ ▉▆▍, ▋▇ ▉ ▉▆▌▊▆, ▊▉▇ ▅▄▆▍▊▍ ▊▆▍▌▌ ▅▍▋▆▆█
▉▊▊▆▉▄▍▇▊▅ █▇▆ ▋▄▋▋ ▋▄▅▄▉▊ █▆ ▇▅▌▆▊▌, ▄▊▆ ▊▍▌ ▅▉'▄ ▇▉▄▋ ▅▌ ▋▄▋▇▅ ▍▆▅█ ▄▄▅▍▆▍ ▄▅▄ ▇█▋▍▄▄▌▊▆▍(▌▉▉, ▌▆█▊▄▄▌▉▌▋)
▉▌▊▍▊: ▅▇▅▆, ▇▄█▋▅▋▄ ▋▉█▋▆▆▌
▌▊▇▅▌: ▉▊▉ █▇▇ ▅▇▊▊ ▆▋▄▋▍▍ ▅▅▄▄▅▊▌▇▆▅
- ▆▌▊▊ ▅▅▋▉▆▇▆▅▅▅ ▊▊▍▍▍ ▌▆▊▋▍▋ ▉▇▅▋▍▌▆▆▆ ▄▇▋▍ ▉▅▍▉▉▊▊▄ ▄▍█▉▋▌▇ ▇▍▉ ▄▋▍▊▌▊▌▇ - █▍▌▍▄▇▌ ▄█▌▍, ▋▅▅▆ ▊▉█▋▅ ▆▍▌▍▇
- ▉▆▄▄ ▌▇▇▊▆ ▅▊▅▅▌▅▊ ▋█▉▇▅█▍█ ▆▅▇▍▊▄▉▅▌▉.▉▄▌▄▉█▉▄▍▍▉ ▉▇▉▊▉, ▉█▇▉▅▄▇▉ ▋▋▍█▄▅▊▉▆▉▇▄▋▊ ▍▄▌▅ █▍ ▍▋ ▋▌ ▇▅▉▄▌▇
██▊▆▆: ▌▉▅▄▌▇ ▍▆▌▆▉▋▌▉▆▋▄▊█ ▊▄ ▍▄▄▍▉▊ ▋▍▄▌▇▉, █▄▉ ▊▄▊▉▍ █▍▊▋▇▋ █▊▉ █▌▆▊ ▆▍█▄▅, ▆▋▆▇ ▋▇▅▆▇▉▊▅ ▍▍▌▊▊▄█▇▉ ▋▅▅ ▇▊▄▅ ▌ ▆▅▄▉▌▇▌ ▋▄▅▆▊ ▄▇▍▍██▌ - █▍▉▌▄▋▉█▉ ▄▍▄▇ ▄▇▅ ▇▉▊▅▉▍
▊▇▌█▆: ▊▄▉███▍▌ █▄ ▆█▄▉▌▌ ▅▌▆▆▄▋▌▅▋▆▅ ▊█▌▌ ▉▇▅▊▅▄▉▆ ▍▊▍▌▅▌▍ ▊▌▅▇█▉▉▆
- ▆▌▆█▋▆ ▆▇█▉▆▋▄▍▌▊▌▊▉ ▇▊▄▍ ▋▆▋▉▆█▅ ▇▆▍ ▅▍▌▍▉▌▍▊▆ █▆▌▅▌▋
▄██▌▍: ▅▉▅▌, ▉▆▍▊▄▌▇ ▆▍▋▍▍▍
█▄▆▋▊: ▊▉▉█ █▉▋▇▆█▆▄::▅▇▋▋▆███ ▌▍▍▍▇ ▆▄█▆▄ █▌▌▌▅ ▄▋▍ ▇▋▉▅▋▇▌ ▄▄▄▉▋▆▉ ▋▆▋▊ ▊▊▅▆▉
- ▆▅▆▍ ▇▅▇▅▋▄▆ ▅▋▍▊▆▉▄ ▆▆▍▍▉▋▍▅ ▉▄██ ▅▋ ▄▅ ▆▄█▇█ ▌▋▌▌ ▋▄ █▍▇▅█▄▉ (▌▄▉▇▊█▄▆ ▉▆▋▆ ▌█▄▉ ▊▊▋▅ ▊▇ ▉▅ ▅▋▋▉▅▆▍▍)
▇▉▍▄▊▌█▄▇▅▇█▅ ▉▌▉▇▇▊ - █▋ ▌▍█▉█▆ ▌▊▉ ▄▇▇▋▌▆▉▍ ▍▋▆▊▌█▄▅ ▍▉▍▅▅▋▉█ ▅▉ ▊█▉▆▉ ▉█▋▅, ▍▄▇▊▉▄▇ █▍ ▆▅▄████▋ ▍█▇▆▋▌▊ ▇▊▅█-▊▅▊▆ ▄▇▄ ▄▉▍▇▅▌ ▆▊▆▄█▅ ▍▅ ▌▌▊▍█▌ ▆▅ ▋▆▍▍ - ▍▌▋▉ ▍▍ ▅▇▇▌▋▉▇ ▉▍▅▊ ▉▌█▅.
▆▌▇▄▆: ▊▉▌▌▋▊▇▅ ▌█▍▊▄ ▉▄▊▌▍▄ ▄▉▍ ▌▄▉▅ ▇ ▉▋▋▄█▉▊▇ ▉█▉▄▅ ▆▅▊▉▆▄▉ - ▇▇█ █ ▍▆▍▋▊▋▌▇▉ ▆▋ █▅██▄█▆█ ▆▆▌▍▌▇▆ ▆▌ █▊▄▅ ▇▆▄▆▊▍▊.
▊▋▆▋▅: ▊▇▌▋▉ ▊▆▉▋▇ ▍▋▆▉▌ ▅▍▄██▊▇ ▄█▇ ▆▌▍▊▅▄▉▊::▅▊▍▅▄▊▇▌▅▄
▇▆▄▊ ▅▉▄▋▋▋▇ ▉ ▆▆█▇▆▋▇▍▉ ▇▇▄▅▌, ▌▍▍▇▄▉█▆ ▋▆▆▊▋█▉▇▆▌ ▆▊█▊ ▋▊▇▊▉▆▉▍ ▌▄ ▉▇▊▅▌▇▊ ▌▄ ▍█▅▄▊▄▌(▇▆▄▉▋██▆ ▅▆▍█▍ ▆▋▌█ ▆▅▇▌▇▉▍▆█ ▉█▆▉▉▋▉▋)
▌▆▌▆▆: ▄▌█▍▉▇▅▌ ▌▆▊▋▌ ▉▊▍▍▊▅ ▇▇▌ ▄▍▉ ▊▆▌▆▇█ ▍▅ ▄▄▉██▍. ▊▄▉▋ ▆ ▉▌▋▅ ▍▋▌▉▋▍▍ - ▆▌█▇▍▄▇▊ █▋▊ ▊▋█▅▉ ▉▋▉.
Clean: further comments
Tests: none, trivial change
Clean: fixed comments
They got out of date from my initial experiments.
Tests: none, trivial change
Optim: reimplementing List.Compare extension via hash->sort approach
- Only implemented for ref types for now, but can be extended to value-types as well
- Has a limitation of requiring no duiplicates in queries lists
- Dependent on the hash quality - we have a slow, quadratic path when a hash collision occurs
- Measured performance - largest case (2048, 512) is an order faster than previous impl (0.75ms vs 12.5ms)
Goal was to reduce the number of hash lookups from the original algorithm and to reduce it's quadratic nature.
Tests: Ran the unit tests - all pass
Update: rewrote ListExtension test to soon-to-chan
- Also added a perf test
- Measured initial implementation's timings - (2048,512) takes 12.5ms on average
I've been working on an optimization that requires me to use ref-types rather than value-types
Tests: run the test in editor, all cases pass
Optim: remove dead bots from global tracking
- Removed now unnecessary null check when simulating bots
Tests: on craggy spawned 15 bots then killed them - counter went up and down accordingly. Flew around 3.5k procmap, gave it a soak test. Both cases no NREs.
New: console command to report bot count on the server.
Tests: loaded into 3.5k procgen world and used the command
Merge: from soundmodulator_leak
Fixes an audio-related NRE when performing generic interactions (zipline, bike, etc).
Tests: rode the zipline on Craggy a bunch of times(the whole way and with quick jump-off). Couldn't repro the NRE (though couldn't do it originally anyway)
