TNT xor stencil buffer bug

Hi,

I’ve been working on a CAD program for lego bricks and I have a
problem with the stencil buffer on the TNT2. I don’t really know
what I’m doing, so it could be me, but it does work with MESA and
the Microsoft Generic drivers.

I’m trying to XOR a moving wireframe lego brick sprite on top of a
static rendered model. The problem is: when viewed head on, the front
and back edges cancel each other out and there isn’t much to look at.
I thought I solved this problem by setting a bit in the stencil buffer
the first time a bit is XORed, so future XORed lines can’t cancel it
out. This works in Mesa and Microsoft Generic OpenGL but maybe that’s
just dumb luck? On the TNT the I get a sloppy looking wireframe
sprite that leaves bits behind when I XOR it a 2nd time to move the
sprite. Are there some other fns that affect this?

You can see a picture here of the XORed sprite before I added the
stencil buffer test. The sprite used to disappear when viewed head
on.
http://www.brickshelf.com/cgi-bin/gallery.cgi?i=44638

Here’s the code if that helps any. I call this once to draw the sprite, and then again to erase it when the user indicates a move with the arrow keys.

/***************************************************************/
int XORcurPiece()
{
int retval = 0;
int drawmode = ldraw_commandline_opts.F;
int savewire = zWire;
int saveshade = zShading;

//ldraw_commandline_opts.F |= STUDLESS_MODE;
//ldraw_commandline_opts.F = (WIREFRAME_MODE | STUDLESS_MODE);
ldraw_commandline_opts.F = WIREFRAME_MODE;
zWire = 1;
zShading = 0;

glMatrixMode(GL_MODELVIEW);
glPushMatrix();

rendersetup();

#ifdef USE_OPENGL_STENCIL
if (use_stencil_for_XOR)
{
// Too many lines cancel out in wireframe mode when viewed head on.
// I could make sure I only XOR each pixel once by setting
// the stencil buffer at each write but only passing when it’s zero.
glStencilMask(GL_TRUE); // Enable stencil buffer writes.
glClearStencil(0x0); // Set stencil clear color
glClear(GL_STENCIL_BUFFER_BIT); // Perhaps just clear the bbox?
glEnable(GL_STENCIL_TEST); // Enable the stencil Test.
glStencilFunc(GL_EQUAL, 0x0, 0x1); // Stencil test(fn, refbits, bitmask)
glStencilOp(GL_KEEP, GL_KEEP, GL_INVERT); // S-Buf write fns(sf, zf, zp)
}
#endif

glDisable(GL_LIGHTING); // No need for lighting
if (movingpiece == curpiece)
{
z_line_offset += 1.0;
glDepthMask(GL_FALSE); // keep depth test, just disable depth writes

}
else
glDisable( GL_DEPTH_TEST ); // don’t test for depth – just put in front
glEnable( GL_COLOR_LOGIC_OP );
glLogicOp(GL_XOR);
glColor3f(1.0, 1.0, 1.0); // white
glCurColorIndex = -2;
retval = Draw1Part(curpiece, 15);
glLogicOp(GL_COPY);
if (movingpiece == curpiece)
{
z_line_offset -= 1.0;
}
#ifdef USE_OPENGL_STENCIL
glDisable(GL_STENCIL_TEST);
glStencilMask(GL_FALSE);
#endif
glDepthMask(GL_TRUE); // keep depth test, just disable depth writes
glEnable( GL_DEPTH_TEST );
glDisable( GL_COLOR_LOGIC_OP );

glCurColorIndex = -1;

glFlush();

glPopMatrix();

zShading = saveshade;
zWire = savewire;
ldraw_commandline_opts.F = drawmode;

return retval;
}

Sounds like path invariance. LogicOp is a SW fallback on TNT, so you’re going through a different path and hit different pixels.

There is no obvious workaround, if that’s what it is.

• Matt

Path invariance eh? I’m not clear what the different paths are.
Could you explain this a bit more? Or just point me somewhere.

A search for “invariance” in this group leads me to the last few
pages of the red book, but that didn’t clear it up for me.
What I get from that is that perhaps I should try switching
glDisable(STENCIL_TEST) to a null StencilOp or something like
that.

No, SW rendering vs. HW rendering will touch different pixels, and with different Z values, colors, texcoords, rendering precision, etc.

What that means is that, say, if you render some geometry on a black background, and then render it again with logic op XOR, expecting to erase it, you won’t erase it. You’ll erase some pixels and draw some new ones, too.

There is no good workaround. You can’t even use a LogicOp of GL_COPY for the first pass, because that also will be accelerated.

The only real options are to (1) find another way to accomplish an XOR-like effect, (2) use HW that supports LogicOp in HW (we support this on Quadro2, but not on Quadro1), or (3) force SW rendering on both passes; you may need to use some logic op other than GL_COPY.

I doubt this has anything to do with stencil specifically. However, if you’re relying on multipass rendering to produce a specific result in the stencil buffer, invariance can sometimes hit you.

• Matt

But as far as I can tell it should use the same rendering path HW or
SW for both passes. I already use the same function (XORcurPiece()
listed above) for both passes and it sets up the same state both
times. On the TNT2, the XOR logicOp works fine when I don’t use the
stencil buffer. The 2nd call to XORcurPiece() erases the pixels drawn
by the 1st call to XORcurPiece. When I run the code with the stencil
buffer enabled (use_stencil_for_XOR is set to 1) I get leftover pixels
after the 2nd pass. Either way, I use the same function and identical
states for both passes. So it still seems like a bug to me.

This isn’t a huge problem because I have another method which uses the
ktx_buffer_region extension (supported on the TNT2) to draw an opaque
sprite instead of XORing. However this is still disappointing because
I’d like to make both methods available on all hardware/driver
combinations and let the user decide which one they prefer.

All right, so maybe that’s yet another invariance issue. We will not document all the cases where we have path invariance issues, because (1) it would require far too much work to list them all, (2) it would change between driver versions, and (3) it would expose too much proprietary information about our hardware.

If you are seeing stencil causing invariance problems, perhaps you could enable stencil test but put it into a NOP mode (accept all pixels, never write to the stencil buffer).

• Matt

Would this be a stencil noop?

glEnable(GL_STENCIL_TEST);
glStencilFunc(GL_ALWAYS, 0x0, 0x0);
glStencilOp(GL_KEEP, GL_KEEP, GL_KEEP);

If so, it doesn’t help any. Oh well, I guess
I’ll just avoid the stencil buffer with the
nvidia drivers.

So it’s probably a logic op problem.

• Matt

Yeah, can you think of another way to achieve
the same effect without the stencil buffer?
The XOR logic op works fine when stencil is
disabled. Could I use the depth buffer for
the stencil effect? This would be OK on the
TNT since I can back up the original zbuf
with the ktx extension.

I thought about it a bit and I think I can
use the depth buffer instead of the stencil.
I have to draw the sprite an extra time to
get the “stencil” in the depth buffer but
that shouldn’t be too much of a penalty.

Backup depth buffer with ktx extension.

While (1) {
draw sprite:
restore depth buffer.
Depth func = GL_LESS,
Mask off color writes.
XOR sprite into depth buffer.
Set Depth func to GL_EQUAL
Allow color writes.
XOR sprite into color buffer.

Get new sprite position from arrow keys.

erase old sprite:
Set Depth func to GL_EQUAL
Allow color writes.
XOR sprite into color buffer.

Set sprite to new position.
}