Remove neighbors that aren't reflected in our buffer, compile optimized shaders