I am trying to do the following
Pass 1. Render meshes to frame buffer object and set gl_FragDepth to 0 based on a certain condition. I only set COLOR_ATTACHMENT0, so depth is written to the common z buffer.
This is done so all the shader used in the next pass will not execute on pixels that I don’t want them to.
Pass 2. Render meshes to FBO. This is an expensive shader, so i’d like to limit the number of pixels. This pass also only has COLOR_ATTACHMENT0 so I assume I am still using the common depth buffer (which I haven’t cleared between the passes).
Render some other objects with tex generated in pass 2.
The problem: How do I use the depth buffer values generated in Pass 1 to ‘cull’ pixels in pass2? If I add a GL_DEPTH attachment, how can I use this texture as preset depth values in the next pass? My current setup does not seem to be doing the job. Is there a better way to cull pixels? I know texkill etc don’t really stop the pixel shader from executing.
I only set COLOR_ATTACHMENT0, so depth is written to the common z buffer.
First problem right there. If your FBO has no depth attachment, depth is not written.
FBOs are completely independent of the default framebuffer. So if you’re using an FBO, nothing you do will affect the default framebuffer. And since there is no “common z buffer” unless you create it, you must create a “common z buffer”.
How do I use the depth buffer values generated in Pass 1 to ‘cull’ pixels in pass2?
If you really want to, it’s simply a matter of outputting a depth that will always fail. What depth that is depends on how your depth comparison is set up. If you’re using GL_LESS, then 1.0 will cause any fragment rendered later to fail. If you’re using GL_GREATER, 0.0 will do it.
However, if you can determine what needs to cause the culling of the second pass based on primitives (glDraw* calls), if some draw calls always kill and all the rest always don’t, I would suggest using the stencil buffer. Clear the stencil buffer, then set the stencil mask based on whether you want to cull or not. On the second pass, set the stencil test to check for the masked value.