495 Commits over 123 Days - 0.17cph!
Merge: from soundmodulator_leak
Workarounds the AudioAmbience NREs by disabling pooling (audio.enablesoundpooling).
Tests: on Craggy shot up 20 scientists, rode a horse, cut down trees, placed 100 barricades - no uptick in pooling of sound modulators in telemetry. No old NREs triggered.
Merge: from main
Tests: none
Update: Add console switch to enable sound pooling
- Disabled by default while I'm investigating ongoing NREs in the area
Tests: spawned 20 scientists and killed them. Ran around placing 100 barricades. Cut down 2 trees. Telemetry showed no pooled sound modulators.
Merge: from main
Tests: none
Merge: from runtime_profiling_pooling
Fixes pool telemetry flood.
Tests: hacked code to force run pooling telemetry - confirmed reduced telemetry.
Bugfix: Respect profiling pool flush interval for pool telemetry
This caused to send ~100kb every update
Tests: hacked code to run analytics
Merge: from main
Tests: none
merge: from soundmodulator_leak
Tests: Spawned and killed scientists in batches of 10. Previously I would get an NRE on 2nd batch, now doesn't happen after 6.
Bugfix: Don't update disabled sound players
I haven't seen it happen, but it is possible that someone could add a SoundPlayer to the Manager's queue, retire it, and next frame Manager would try to update it triggering an NRE.
Tests: spawned 10 scientists and killed them - no nres, audio was good
Bugfix: recycle sounds used by SoundPlayer when owner is returned to pool
Tests: Killed 60 scientistnpc_patrol in batches of 10 - NRE didn't reproduce.
Merge: from main
Tests: none
Merge: from main
Tests: built all modes, tried to repro 2 previous known bugs - didn't happen.
Merge: from main
Tests: none
Optim: Send trees in batches over multiple frames to avoid spike on player connection
This is experimental, but it seems stable initially. On 4.5k procgen world from a save it took 90ms(very rough numbers, including client-side spawning overhead) in total to send all trees, but each frame 1 player took on average 0.2ms to gather & send data. Also need to fix a bunch of bugs
Tests: Ran craggy in local editor session - no exceptions.
Merge: from main
Tests: built all modes in editor
Merge: from pool_container_telem
Tests: took telem info from craggy, procgen 4.5k and saved procgen map. Built all modes in editor. Built standalone server exe.
Buildfix: properly exclude command from non-editor builds
Tests: built all modes in editor. Built standalone server.
Update: added editor-only print_memory_overhead cmd
- Marked that it gives estimates, not accurate results
Tests: ran on craggy - got simlar results to print_memory
Update: print extra mem held by containers in Pool
Tests: took a couple snapshots on craggy, fresh 4.5k procgen + 4.5k procgen save
Update: Pool can track container capacity memory overhead
- Implemented as an editor-only feature to avoid adding overhead in release
I needed to confirm that we don't keep around giant lists/buffers in the pool.
Tests: entered play mode in editor
Clean: replacing obsolete Pool api usage
Tests: none, trivial change
Merge: from /main/soundmodulator_leak
Tests: spawned car_2mod_01, drove around - pool telemetry stayed stable, audio was good.
Merge: from main
Tests: built all modes in editor
Optim: lazy-allocate Sound Modulator lists
- Added an assert to trigger when we try to remove a modulator we don't own
Noticed that we have 3 modulators on average per 4 lists of modulators, so we can save 1-3 lists per sound and recycle better.
Tests: When driving around, got max 42 SoundModulation instances with 66 lists (instead of 168 previously)
Update: don't expose modulators outside of SoundModulation
- Changed printEngineSounds cmd to update more detialed info while removing redundant info
- Simplified checks since we can now guarantee that our object is safe by construction.
Tests: drove around in a car and checked the output in console.
Bugfix: Cleanup leaking Sound modulators
- Sound now gets notified when it gets retired
- Moved cleanup from Init to Retire and added safety asserts to Init
- Recycle more of SoundModulation state (revealed that we have more storage than modulators)
Tests: Ran around on Craggy, gathered resources, smashed a tree, crushed myself with a car by accident, drove around, swam - sound was good.
Merge: from /main/clean_dead_code
Tests: upgraded half wall - it worked
Clean: removing code marked for deletion since 2023
Tests: upgraded half-walls - it worked
Merge: from main
Tests: none, no conflicts
Optim: Recycle a list when paying for placement
Tests: none, trivial change
Optim: Avoid transient allocations for GetReloadMenu
This is more like a janitorial fixup rather than anything significant. Not recycling GameMenu.Options list because it's a struct, and we don't have a good strategy of shrinking containers in the pool yet.
Tests: Changed ammo for AK via radial menu
Update: reset radioactivity when recycling Items
Although Item pooling is not live on this branch, instead of going through merge overhead I'm just patching it directly here.
Tests: none, trivial change
Optim: Reduce allocations when running reloading logic on the server
- Added non-allocating singular IAmmoContainer.FindX methods
We've got a bunch of places where we are only interested in result but try to gather all.
Tests: Built all modes in editor. In local session, shot a couple mags with different ammo types. Explicitly switched ammo type via radial menu, and implicitly via reload-on-empty.
Update: Getting rid of allocating PlayerInventory.AllItems
- Previous AllItemsNoAlloc is now new GetAllItems
- Added PlayerInventory.Contains
Tests: none, trivial changes
Update: List<Item> now use Free instead of FreeUnmanaged.
- Amortized a couple consecutive push calls
Need to do the same for ItemContainer.
Tests: Built all modes in editor. No runtime tests as it's trivial changes.
Merge: from main
Tests: built all modes in editor
Merge: from main
Tests: built all modes in editor
Update: Disable ItemPooling on Client and make server on by default
- Controlling variable is no longer replicated to client.
Client-side item handling is very lax and would need substantial changes. I don't want to hold up serverside release to get the full package out, I'd rather relase it bit by bit (and less risk of volatility).
Tests: Did all tests with both enabled and disabled server-side Item pooling
- Built all modes in editor.
- Tried reproducing 2 original bugs - didn't work.
- Loaded up a save that used to lead to a bug - didn't happen.
- Tested in local 2p session on Craggy - could kill, loot, equip and see changes on characters.
- Logs enabled through entire process - client didn't try to explicitly cleanup items (since disabled, as intended).
- Validated that it's possible to launch with command line disabling item pooling.
- Validated that it's possible to override item pooling with executable args
Update: added ItemManager logging of Item's lifecycle
- level 1 reports creation, loading and removal
- level 2 reports scheduling for removal
Tests: ran locally in the editor, switched items, dropped gear, looted existing containers. Checked that we get logging related allocs, but they're small (and I can remove them a bit later)
Bugfix: Rcon no longer waits for first command to start broadcasting log messages
Tests: Connected to server via test RCon page - started seeing output immediately
New: RustLog utility - consistently formatted logs with conditional output
- Extracted from BaseMonoBehavior, with forwarding of it's logs to RustLog
- Added Item category
We have scenarios where we have categorized logs in cases where we don't have BaseMonoBehavior available(Items) - this fills that niche.
Tests: Built all modes in editor. Ran with Network logging enabled - saw expected output. RCon page was good.
Update: regenerated ItemManager.EnablePooling command
Forgot to do it in previous submit
Tests: none, trivial change
Update: ItemRemove now schedules self recycling on Client
- subject if ItemManager.EnablePooling is set or not
This can automatically handle cases where we have client-side WorldItem & ItemContainers storing client-side items(amortizes synchronization) - but needs code fixing for that.
Tests: Tested dropping items(with pooling disabled it's ok, with enabled it's borked), equipping armor(same) - will fix in next CLs
Merge: from main
Tests: Build Server mode in editor, all good
Clean: comment explaining Client-side exclusive ItemManager use
Tests: none, trivial change
Merge: from main
Tests: editor build for all modes(Server borked, but fix coming)
Bugfix: Don't leak pooled lists in RunInRadius<T>
Thanks for the report!
Tests: ran admin commands that would generate a leak - no more.
Update: ItemManager is available on the client
- Like server, ticks in the background and cleans up items
- when creating/loading items can optionally specify whether it's client-side or server-side item (checked in exclusive modes)
- Made EnablePooling a replicated var, it disables clientside effects
We have a bunch of client-side item allocations that we don't correctly clean up - this will help deal with that.
Tests: built all modes in editor
Bugfix: don't leak an item if creation request was invalid
Tests: none, trivial change
Bugfix: don't create and access a dangling Item when Pooling is enabled
- at the same time it also optimizes inventory processing by batching the update as a result of multiple equipment events.
Tests: Tested by loadinga a save that borked the inventory by spawning NPCs - passed. Also validated that in multiplayer, wearables update when the player sleeps if they got changed by a different player.
