I’m trying to do z-fail stencil shadows using separate stencil when OpenGL 2.0 is available, and otherwise GL_EXT_stencil_two_side when that is available. The latter works, the former doesn’t, and it’s driving me crazy because there’s not much difference between the two code paths.
When I start the stencil marking pass, I do this:
glClearStencil( 128 );
glClear( GL_STENCIL_BUFFER_BIT );
glEnable( GL_STENCIL_TEST );
if (separateStencil)
{
glStencilOpSeparate( GL_BACK, GL_KEEP, GL_INCR_WRAP_EXT, GL_KEEP );
glStencilOpSeparate( GL_FRONT, GL_KEEP, GL_DECR_WRAP_EXT, GL_KEEP );
glStencilMask( ~0 );
glStencilFunc( GL_ALWAYS, 0, ~0 );
}
else if (stencilTwoSide)
{
glEnable( GL_STENCIL_TEST_TWO_SIDE_EXT );
glActiveStencilFace( GL_BACK );
glStencilOp( GL_KEEP, GL_INCR_WRAP_EXT, GL_KEEP );
glStencilMask( ~0 );
glStencilFunc( GL_ALWAYS, 0, ~0 );
glActiveStencilFace( GL_FRONT );
glStencilOp( GL_KEEP, GL_DECR_WRAP_EXT, GL_KEEP );
glStencilMask( ~0 );
glStencilFunc( GL_ALWAYS, 0, ~0 );
}
(and of course other code that does not depend on which kind of stencil extension I use.) When I start the lighting pass, the critical code is:
if (separateStencil)
{
glStencilFunc( GL_EQUAL, 128, ~0 );
glStencilOp( GL_KEEP, GL_KEEP, GL_KEEP );
glStencilMask( 0 );
}
else if (stencilTwoSide)
{
glActiveStencilFace( GL_BACK );
glStencilFunc( GL_EQUAL, 128, ~0 );
glStencilOp( GL_KEEP, GL_KEEP, GL_KEEP );
glStencilMask( 0 );
glActiveStencilFace( GL_FRONT );
glStencilFunc( GL_EQUAL, 128, ~0 );
glStencilOp( GL_KEEP, GL_KEEP, GL_KEEP );
glStencilMask( 0 );
}
Any idea what could be wrong? I’ve tried it on both a Mac with an ATI card and a PC with an NVidia card, so as much as I’d like to blame the hardware, I don’t think I can.