Attempt at global building privilege cache, using a simple grid holding all the building blocks + a collider test on the winning block, because triangles and roofs have bounds larger than their colliders
Thats enough to remove the Vis Entities call and keep the same precision. Works on the client and the server
In a big base, 575 blocks around me:
4.60ms -> ~0.50ms