Fixed a bug where the parent network object could be destroyed when doing a network refresh if there were children to prune (Fixes Facepunch/sbox-issues#7812)