Stencil buffer

Hi all

I have rendered a scene with the stencil test enabled. Now I want to render in white color all the places where the value of the stencil test is 2 or more (for instance).

What I do is to render a GL_QUAD for all the screen with the stencil test activated and just paint those pixels who passes the stencil test. That works fine.

I define the stencil function as this:

glStencilFunc(GL_EQUAL, popularPlaceValue-1, popularPlaceValue-1);

Where popularPlaceValue is an uint of my class. That renders in white color all the pixels where the stencil value is exactly the value popularPlaceValue.

What I want is to render in white all the pixels where stencil value <= popularPlaceValue.

Does anyone can help me?

2 potential errors here :

  • the third parameter is an AND bitmask, if you don’t understand, keep it to all bit sets to 1 : 0xFFFFFFFF.
  • EQUAL with (popularPlaceValue-1) as ref value should never show ‘popularPlaceValue’, but rather (popularPlaceValue-1).

http://www.opengl.org/sdk/docs/man/xhtml/glStencilFunc.xml
From the doc it does not sound so complex :
glStencilFunc(GL_LEQUAL, popularPlaceValue, 0xFFFFFFFF);

That was I tried at the beginning but I don’t know why it doesn’t work. Maybe the values of the stencil are wrong.

Thank you for your reply.

It’s most likely that your comparisons are the wrong way round. I had that problem when I first used the stencil - took ages to figure out!

The stencil test will pass if your reference value is {less,equal,greater} than the value in the stencil buffer.
You are probably assuming that the stencil buffer is compared to your ref value - it’s not and the comparision works the otherway round.

glStencilFunc (GL_EQUAL, myRefValue, $FF); //pass stencil test if the ref value (equal) to stencil buffer

glStencilFunc (GL_LEQUAL, myRefValue, $FF); //pass stencil test if the ref value (Less or equal) to stencil buffer

That’s exactly what’s happening. I was assuming the comparision in the wrong way. Really amazing error because I read the glStencilFunc documentacion several times and I didn’t realize that.

Thank you for your reply.

I read the spec over and over again as well…still could not figure it out. At one point I thought it was a driver bug. The only thing which stopped me reporting it was that nVidia are good with drivers and it’s an unlikely bug to have persisted around for so long. I had access to other h/w with much older drivers and the same was happening there - therefore it had to be me (which it was).

Glad to help.