3,093 Commits over 730 Days - 0.18cph!
Update: reimpleemnt whole of CoreCLR's ConcurrentQueue (same as in shipped mscorlib)
- added extra telemetry to track segment alloc/retire, general tries, gc activity
This time didn't rip out the segment growing logic, and looks like if it safely fits in one segment, it can be consistently used. Going to confirm if extra large segment is enough, then it'll be a viable candidate
Tests: ran unit tests and AllocDeallocMTShortLived(32,False) perf test
Update: minor simplifications
- get rid of retry limiters
- fixup unsafe increments
Couldn't figure out how to do add segment recycling, so going to change focus for a bit - will see how ConcurrentQueue performs
Tests: ran unit tests
Optim: reduce GC impact by reducing how many segments we trash
This forces serialization at the point of segment exhaustion. Need to see if I can setup some sort of recycling to reduce busy spinning
Tests: ran AllocDeallocMTShortLived(32, false), no more GC reports
Bugfix(tests): fix 2 pooling tests having bad setup after Spillage tests
These tests rely on explicit sizes/counters, so were randomly failing
Tests: ran all PoolTests unit tests
Update: add a bunch of perf counters to try to track down the spike source
- slightly amend Enqueue to help progress swap before creating a new segment if it failed
The spikes are from GC collection events, likely triggered by excessive trashing of segments(after 10 runs - 106 created, 97 trashed)
Tests: ran AllocDeallocMTShortLived(32, false) perf test
Update: reimplement pool internals on top of LPRQ
Based on paper by Raed Romanov & Nikita Koval: https://dl.acm.org/doi/epdf/10.1145/3572848.3577485
Shows same or a bit better times than FuzzyPool, but periodically has a very spiky test (from 2ms to 40ms). Also needs further tweaking and documenting
Tests: ran unit & perf tests
Update: try using CoreCLR's ConcurrentQueueSegment as a Pool implementation
Leaks a bit in busy MT perf tests, but breaks down at 32 threads (can leak up to 50% of the buffer). Not good enough on it's own
Tests: ran perf and unit tests
Update(editor): SaveViewer - add a size check when loading a save
- also emit what was the last prefab loaded (if there was one) to help track down protobuf/save without save protocol update
Tests: used to load a borked save, didn't get stuck in an infinite loop
Merge: from fix_app_turretcontrol
- No changes, just plastic history graph cleanup
Merge: from fix_app_turretcontrol
- Bugfix for frozen turret view in Rust+ app
Tests: controlled turret in Rust+ app
Bugfix: feed controlled entities transformation matrices to Rust+ camera instead of transforms
AutoTurret has a virtualized hierarchy, so the eye transform was never moved, and the camera rendered same view angles
Tests: used Rust+ app to control the turret
Merge: from codeanalyzer_getwrapperattrib
- Update Rust.CodeAnalyzer for PoolAnalyzerGetWrapperAttribute support
- Bugfix 4 pool leaks from above attrib
Tests: compiler simulator passes
Bugfix: fixup pooling errors in 4 places - 2 in WireTool, BaseAIBrain and CommunityEntity
- added a bunch of PoolAnalyzerNonCachingAttribute to BaseAIBrain calls working on proto types
Noticed 2 cases where analysis escape didn't trigger like it should've
Tests: none, trivial changes
Update: update CodeAnalyzer with support for PoolAnalyzerGetWrapperAttribute
- tag NetRead.Proto and ProtoDelta with it
Tests: editor compile shows 4 pooling errors
Merge: from fix_unitask_taskpool
- Bugfix for not-thread-safe taskpool
Tests: unit tests + booted into craggy with Jobs 4
Update: merge up to UnitTask/d239195
- Bugfix for buggy task pool returning dupes
Tests: booted into craggy, unit tests
Bugfix(editor): WhatUsesThis - don't require a whitespace after m_EditorClassIdentifier
- wrap the serach into a try-catch-throw block so we can quickly check if there's anything wrong with the prefab
Whitespace got lost in the prefab asset, causing the tool to trip up
Tests: ran it, got results
Merge: from mt_snapshot_saving
- Bugfix for duplicate field serialization
- Bugfix for unsafe Time access on worker thread
Tests: Booted into craggy
Bugfix: remove duplicate transformHandle property
Prevents error in editor about duplicate serialization. Though the member is private and shouldn't serialize anyway?
Tests: tried opening & saving the rpefab, but couldn't cause original error
Bugfix: use cached time snapshot for server occlusion snapshot sending
Prevents Time usage in worker threads
Tests: none, trivial change
Merge: from mt_snapshot_saving
- Bugfix for ExactArrayPool not being thread-safe
Tests: TestMTSave
Bugfix: rewrite ExactArrayPool to be a ConcurrentDictionary
Was not MT safe for Jobs 4 parallel serialization as it could lazy initialize
Tests: TestMTSave
Update(tests): make SendEntityUpdates perf test spawn as many recievers as threads available
Allows to fully saturate the CPU in a test, giving a bit fairer comparison between Jobs 3 and 4
Tests: ran new test
Merge: from mt_snapshot_saving
- Optim: in UsePlayerUpdateJobs 4 entitiy saving for network is done in parallel. Use server.ParallelNetworkQueueBatchSize to tweak number of tasks
Tests: 2k procgen, built a base and a boat, rode vehicle, heli, zipline, boat, horse, fly across the world. ran all TestMTSave unit tests
Clean: fixed whitespacing
Tests: none, trivial change
Buildfix: add missing server guard
Tests: compile sim
Update: add an default TransformHandle check to TransformEx.Unsafe.ExtractTranformAccess
Better we lose a smidge of perf than have a native crash
Tests: ran all TestMTSave
Update: add GetPosMT and GetRotMT to TransformEx
- bugfix Elevator Save accessing transform
chcked internals, should be safe
Tests: fly across all of 2.5k procgen, no errors triggered
Bugfix: avoid resolving transformHandle on an optional BatteringRam.doorPhysicsHinge
Checked other cases I've modified, rest are required
Tests: none, trivial change
Bugfix: resolve game controlelr at init of BaseCardGameEntity
- use cached controller instead of trying to lazy create it in Save
Avoids invoking non MT-safe APIs in save
Tests: visited bandit camp on procgen, no errors. Ran TestMTSave tests
Bugfix: avoid NRE if bike doesn't have a sidecar
Tests: spawned a motorbike
Update(tests): TestServer now also creates a dummy BiomeMap
Not sure why pre-merge TestMTSave(GrowableEntity) didn't trip over, but sorted now
Tests: all TestMTSave
Clean: add comments for new perf test
Tests: none, trivial change
New(tests): SendEntityUpdates perf test, creates a bunch(100, 1000, 10000) of players that get sent to 10 players
Runs Jobs 3(parallelize dependency chains) vs Jobs 4(parallelize queues) serialization modes and whether to use network cache. Doesn't create any entity hierarchy (best case for Jobs 3). With network cache, Jobs 4 beats by ~2x (10k: 16ms vs 9ms), without cache Jobs 4 scales much better, up to 3x faster (10k: 62ms vs 14ms).
Tests: ran SendEntityUpdates
Update(tests): assign prefabIDs to entities created in TestServer
Avoids tests derping out trying to save global managers/players
Tests: SendEntityUpdates perf test passes
Bugfix: fix double-free on a pooled buffer
This was causing a buffer to be shared between 2 tasks when it was fetched from the pool, leading to chaos
Tests: ran SendEntityUpdates perf test (gonna submit in a bit)
Update: Jobs 4 - intercept exceptions from SendAsSnapshot/ShouldNetworkTo and log once per task
- Also if ToStream threw, don't save the resulting buffer as a network cache, as that would poison everything
Instead of bubling them all up to the top like usual, handle locally by just skipping sending the data, and report error once to avoid filling console
Tests: booted into craggy with CCTV fix reverted, no more spam
Optim: Implement parallel serialization in Jobs 4 mode
- fix CCTV_RC doing GetComponentInParent on an MT path, cache on init
- add server.ParallelNetworkQueueBatchSize, controls how many players end up being batched in a task. Higher number - less tasks
Offloading ShouldNetworkTo, serialization and sending all to worker threads. Main thread steals last task to run on main thread before spinning for other task completion.
Tests: booted craggy, gathered resources, build a base, rode ahorse, shot a gun, drove a car, heli, zipline, built a boat, used codelock
Update: rewrite ToStreamForNetwork to be thread safe
- EntityMemoryStreamPool is now a concurrent queue
Now, theoretically, we just throw it at threads and drink champagne
Tests: craggy, built a base and checked decay timer
Update: initialize every SaveInfo with ThreadSafeTime.TakeSnapshot
- also removed some jobs 2 code I missed previously
Tests: on craggy rode a horse, built a base, checked decay timer was updating as I added/removed resources
Bugfix: avoid multiple BuildingPrivilege.CalculateProtectedMinutes calls in a frame for 1 entity
Atomic latch wasn't correctly checking if it was updating from an old value only, so we could CompareExchange(1,1,1) and pass
Tests: none
Update: avoid Time calls in BasePlayer.GetUnderwearSkin
- UnderweatManifest now initialized in ServerMgr.StartServer
Tests: switched to jobs 4 and toggled underwear
Update: in UseParallelSaves mode, Ragdol component samples transform handles
Tests: on playground enabled Jobs 4 and ragdolled a bunch of times
Update: use handles to get pitch and yaw rotations for AttackHelicopterTurret in UseParallelSaves mode
- amended test to cover turret saving path
Tests: TestMTSave pass
Update: move OrientableLight interpolation invoke from Save to rpc + ondeply
Scheduling invokes won't be safe in MT
Tests: none
Update: avoid transform call in TravellingVendor.Save when it's following a spline
Tests: none
