HLSL stencil function to OpenGL

[mindbreaker]

Hi,

I'm in the middle of porting some HLSL shaders to opengl (with CG) and I'm struggling a bit with the correct opengl stencil statements...
I have to states that need to be set for the stencil buffer :


//first one
StencilEnable = true;
StencilPass = REPLACE;
StencilRef = 1;

//second
StencilEnable = true;
StencilPass = KEEP;
StencilFunc = EQUAL;
StencilRef = 1;

I thought it would be this but it looks wrong sometimes. Withing my directX project, I'm clearing the backbuffer with stencil value 0

//first one
glClearStencil(0);
glEnable(GL_STENCIL_TEST);
glStencilFunc(GL_ALWAYS, 1, 1);
glStencilOp(GL_KEEP, GL_KEEP, GL_REPLACE);

//second one
glClearStencil(0);
glEnable(GL_STENCIL_TEST);
glStencilFunc(GL_EQUAL, 1, 1);
glStencilOp(GL_KEEP, GL_KEEP, GL_KEEP);

So, can anybody verify if this is correct (and what would needed to be changed)?

Thx!!

[_arts_]

The last 2 arguments of StencilOp treat about when stencil test passed, the first of them when depth buffer test failed, the second one when depth buffer test passed. So, I think you should keep the same values for the last 2 arguments, regarding what you provided.

[Dan Bartlett]

glClearStencil doesn't actually clear the stencil buffer, it just sets the clear value. You need to to call glClear with GL_STENCIL_BUFFER_BIT included to clear it.

[BionicBytes]

What are you trying to do here:


//first one
glClearStencil(0);
glEnable(GL_STENCIL_TEST);
glStencilFunc(GL_ALWAYS, 1, 1);
glStencilOp(GL_KEEP, GL_KEEP, GL_REPLACE);

Are you trying to initially set the stencil to 1 for every pixel of the depth/stencil buffer?
In which case all you need is:
glClearStencil (1);
glclear (GL_DEPTH_BUFFER_BIT | GL_STENCIL_BUFFER_BIT);


and with this:


//second one
glClearStencil(0);
glEnable(GL_STENCIL_TEST);
glStencilFunc(GL_EQUAL, 1, 1);
glStencilOp(GL_KEEP, GL_KEEP, GL_KEEP);

You are trying to find areas of the stencil buffer which have been tagged with a value of 1 (which initially they all are)?
In which case, you just need:
glEnable(GL_STENCIL_TEST);
glStencilFunc(GL_EQUAL, 1, 1);
glStencilOp(GL_KEEP, GL_KEEP, GL_KEEP);