Line Z-fighting workaround

Like I may have mentioned in my previous questions, I’m working on rendering a relatively high poly “model”.

Functionally, it’s generated terrain from a square heightmap image, a few hundred pixels to a side. Since it’s the only 3D object in the scene (everything else is 2D GUI elements that ignore the depth buffer) I decided to use glPolygonOffset to draw its wireframe.

Now, I want to draw another set of lines on top of the wireframe. Each member of this set of lines (they make up a contour line, if you’re wondering) is guaranteed to be coplanar to one of the triangles that make up the terrain; however, this also means that the lines can interfere (z-fighting) with the wireframe. As a workaround, I draw these extra lines into the stencil buffer, then, after drawing the wireframe, painting the lines back on via stencil.

Now, I’m wondering if it’s possible to utilize the 8 bits of the stencil buffer for more features. As it stands, I don’t get any antialiasing from the stencil buffer – the test seems to only be on or off, so the lines appear thick (where the partial alpha gets drawn as solid).

My main purpose is to be able to draw multiple contours of different colors, so I was wondering if it was possible to do some bitwise tests. If it’s not possible, can I use other methods (textures perhaps?) to replicate this masking behavior? Antialiasing would be preferred, but not an absolute necessity.

EDIT: I seem to have left out that I’m currently handling the multiple color issue by simply overwriting the stencil buffer with a different value, then checking for equality later. This method works “well enough”, but I feel it’s inelegant and could use improvement, hence the thread.

You do if you enable MSAA. Yes, the stencil value for a particular sample is a specific value, but you get multiple samples per pixel, which (if use to modify the color buffer) are downsampled and thus nicely antialiased.

My main purpose is to be able to draw multiple contours of different colors, so I was wondering if it was possible to do some bitwise tests.

Yes. See glStencilFunc – in particular the ref and mask fields, and glStencilMask.

Thanks for this. It was there all along, but I was so focused on looking for “bitwise” keywords that it slipped right by me.

I’ll try MSAA out, it sounds like it’s just what I need.

I might just be being a little greedy here, but can I bitwise shift the stencil buffer somehow?

Possibly – never tried. OpenGL 4.3 and/or ARB_stencil_texturing along with ARB_shader_stencil_export would probably make it easy. But you can probably do it another way as well (possibly copying through PBOs to retype the data in case your GPU doesn’t support ARB_shader_stencil_export. And there’s always CPU readback, but would avoid that.

However, keep in mind that you can get similar results as (for example) bit-shifting the whole stencil buffer by just adjusting your stencil test mask and reference value to pick out (or set) the right bits.

Another “d’oh” moment for me here… it’s so obvious, now that you mention it, but I wouldn’t have seen it otherwise.

Thank you for your help.