3,110 Commits over 730 Days - 0.18cph!
Merge: from mt_snapshot_saving
- Bugfix for invalid-api exception when trying to save attackheli turret with ammo/gun
Tests: spawned one, loaded it up and shot all while in Jobs 4 mode
Bugfix: cache ammo for heli turret, avoid TryGetComponent in Save which is not safe in MT
Tests: spawend a heli, setup an ak turret with 100 bullets, shot in Jobs 4 mode - no errors
Merge: from fix_skipinit_error
- Buildfix(editor) - replace unsupported intrinsic
Tests: no errors in editor on script compile, checked asm
Buildfix: get rid of SkipInit, LLVM optimizes out variable initialization from final asm
Tests: triggered compile - no errors in editor. Checked asm against externally stack allocated storage - same asm
Merge: from pool_leak_experiments
- Optim: replace FuzzyPool with LPRQ segment - it can be up to 20% faster and doesn't leak/spill sporadically
Tests: unit + perf tests, loaded 2.5k procgen map, checked print_memory - no spills
Clean: remove editor-only add/remove fail counters
- also cache cell buffer length - jit might already do it automatically, so might be moot
Tests: compiles
Merge: from lprq
Prepping LPRQ segment implementation to use instead of current FuzzyPool
Tests: ran unit tests
Update(tests): expand TestAllocWhenEmpty test slightly to sanity check leaking behavior
Tests: ran unit test
Clean: Add general explanation on the pool implementation
- remove TODOs
- buildfix for non-editor builds
- bugfix of setting the wrong epoch for a slot for stale cells
Tests: ran unit and perf tests
Optim: inline LPRQ segment logic, get rid of the overall queue
Perf test is averaging 2.8ms, improvement over FuzzyPool by 20%
Tests: unit tests + AllocDeallocMTShortLived(32,False) perf test
Optim: use same trick of oversized segment to avoid segment churn
Looks faster than FuzzyPool, noice
Tests: ran unit tests + select perf tests
Optim: inline segment storage and strip all ConcurrentQueue logic
My previous estimate of 10% degradation was off - it actually was 6ms vs 3.5ms. With these changes I consistently get 4ms for AllocDeallocMTShortLived(32,False) perf test - 15% loss, still acceptable
Tests: unit tests + AllocDeallocMTShortLived(32,False) perf test
Update(tests): port unit test fix from child branch
Tests: ran PoolTests batch of unit tests
Merge: from main
Random Editor crash seems to be gone on latest
Optim: use an over capacity segment to avoid segment churn
This keeps GC activity at 0. Perf seems 10% worse than Fuzzy, but it's acceptable. Though there's a bug somewhere.
Tests: AllocDeallocMTShortLived(32,False) perf test
Update: reimpleemnt whole of CoreCLR's ConcurrentQueue (same as in shipped mscorlib)
- added extra telemetry to track segment alloc/retire, general tries, gc activity
This time didn't rip out the segment growing logic, and looks like if it safely fits in one segment, it can be consistently used. Going to confirm if extra large segment is enough, then it'll be a viable candidate
Tests: ran unit tests and AllocDeallocMTShortLived(32,False) perf test
Update: minor simplifications
- get rid of retry limiters
- fixup unsafe increments
Couldn't figure out how to do add segment recycling, so going to change focus for a bit - will see how ConcurrentQueue performs
Tests: ran unit tests
Optim: reduce GC impact by reducing how many segments we trash
This forces serialization at the point of segment exhaustion. Need to see if I can setup some sort of recycling to reduce busy spinning
Tests: ran AllocDeallocMTShortLived(32, false), no more GC reports
Bugfix(tests): fix 2 pooling tests having bad setup after Spillage tests
These tests rely on explicit sizes/counters, so were randomly failing
Tests: ran all PoolTests unit tests
Update: add a bunch of perf counters to try to track down the spike source
- slightly amend Enqueue to help progress swap before creating a new segment if it failed
The spikes are from GC collection events, likely triggered by excessive trashing of segments(after 10 runs - 106 created, 97 trashed)
Tests: ran AllocDeallocMTShortLived(32, false) perf test
Update: reimplement pool internals on top of LPRQ
Based on paper by Raed Romanov & Nikita Koval: https://dl.acm.org/doi/epdf/10.1145/3572848.3577485
Shows same or a bit better times than FuzzyPool, but periodically has a very spiky test (from 2ms to 40ms). Also needs further tweaking and documenting
Tests: ran unit & perf tests
Update: try using CoreCLR's ConcurrentQueueSegment as a Pool implementation
Leaks a bit in busy MT perf tests, but breaks down at 32 threads (can leak up to 50% of the buffer). Not good enough on it's own
Tests: ran perf and unit tests
Update(editor): SaveViewer - add a size check when loading a save
- also emit what was the last prefab loaded (if there was one) to help track down protobuf/save without save protocol update
Tests: used to load a borked save, didn't get stuck in an infinite loop
Merge: from fix_app_turretcontrol
- No changes, just plastic history graph cleanup
Merge: from fix_app_turretcontrol
- Bugfix for frozen turret view in Rust+ app
Tests: controlled turret in Rust+ app
Bugfix: feed controlled entities transformation matrices to Rust+ camera instead of transforms
AutoTurret has a virtualized hierarchy, so the eye transform was never moved, and the camera rendered same view angles
Tests: used Rust+ app to control the turret
Merge: from codeanalyzer_getwrapperattrib
- Update Rust.CodeAnalyzer for PoolAnalyzerGetWrapperAttribute support
- Bugfix 4 pool leaks from above attrib
Tests: compiler simulator passes
Bugfix: fixup pooling errors in 4 places - 2 in WireTool, BaseAIBrain and CommunityEntity
- added a bunch of PoolAnalyzerNonCachingAttribute to BaseAIBrain calls working on proto types
Noticed 2 cases where analysis escape didn't trigger like it should've
Tests: none, trivial changes
Update: update CodeAnalyzer with support for PoolAnalyzerGetWrapperAttribute
- tag NetRead.Proto and ProtoDelta with it
Tests: editor compile shows 4 pooling errors
Merge: from fix_unitask_taskpool
- Bugfix for not-thread-safe taskpool
Tests: unit tests + booted into craggy with Jobs 4
Update: merge up to UnitTask/d239195
- Bugfix for buggy task pool returning dupes
Tests: booted into craggy, unit tests
Bugfix(editor): WhatUsesThis - don't require a whitespace after m_EditorClassIdentifier
- wrap the serach into a try-catch-throw block so we can quickly check if there's anything wrong with the prefab
Whitespace got lost in the prefab asset, causing the tool to trip up
Tests: ran it, got results
Merge: from mt_snapshot_saving
- Bugfix for duplicate field serialization
- Bugfix for unsafe Time access on worker thread
Tests: Booted into craggy
Bugfix: remove duplicate transformHandle property
Prevents error in editor about duplicate serialization. Though the member is private and shouldn't serialize anyway?
Tests: tried opening & saving the rpefab, but couldn't cause original error
Bugfix: use cached time snapshot for server occlusion snapshot sending
Prevents Time usage in worker threads
Tests: none, trivial change
Merge: from mt_snapshot_saving
- Bugfix for ExactArrayPool not being thread-safe
Tests: TestMTSave
Bugfix: rewrite ExactArrayPool to be a ConcurrentDictionary
Was not MT safe for Jobs 4 parallel serialization as it could lazy initialize
Tests: TestMTSave
Update(tests): make SendEntityUpdates perf test spawn as many recievers as threads available
Allows to fully saturate the CPU in a test, giving a bit fairer comparison between Jobs 3 and 4
Tests: ran new test
Merge: from mt_snapshot_saving
- Optim: in UsePlayerUpdateJobs 4 entitiy saving for network is done in parallel. Use server.ParallelNetworkQueueBatchSize to tweak number of tasks
Tests: 2k procgen, built a base and a boat, rode vehicle, heli, zipline, boat, horse, fly across the world. ran all TestMTSave unit tests
Clean: fixed whitespacing
Tests: none, trivial change
Buildfix: add missing server guard
Tests: compile sim
Update: add an default TransformHandle check to TransformEx.Unsafe.ExtractTranformAccess
Better we lose a smidge of perf than have a native crash
Tests: ran all TestMTSave
Update: add GetPosMT and GetRotMT to TransformEx
- bugfix Elevator Save accessing transform
chcked internals, should be safe
Tests: fly across all of 2.5k procgen, no errors triggered
Bugfix: avoid resolving transformHandle on an optional BatteringRam.doorPhysicsHinge
Checked other cases I've modified, rest are required
Tests: none, trivial change
Bugfix: resolve game controlelr at init of BaseCardGameEntity
- use cached controller instead of trying to lazy create it in Save
Avoids invoking non MT-safe APIs in save
Tests: visited bandit camp on procgen, no errors. Ran TestMTSave tests
