Update the hit vector when passing through to a ball, or else it's always just the up normal of the table