Tried moving NetworkSimulator away its slightly awkward Prefab + NetworkEntity existence into a simpler one that just used networkServer.Send. In the end it jut made things complicated in a different way, since initial data and changes couldn't be propagated through Serialize anymore. So just putting this into a branch. The only thing not functional here is an initial send when new clients connect.