PDA

View Full Version : Stencil buffer question



PollyPocket4eva
03-14-2011, 07:27 AM
Hi,
I've been looking for good stencil buffer tutorials online but can't get a handle on the whole thing.
I'm trying to create a circular matte for a viewport.
Basically I'm trying to achieve something like glScissor but with a circle rather than a rectangle.
Could anyone provide a simple example?
Thanks!

V-man
03-14-2011, 06:04 PM
You should call glClearStencil(0) somewhere in your code so that the stencil buffer gets zero when glClear is called.



//Clear
glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT|GL _STENCIL_BUFFER_BIT);

//Allow writing to all bits in the stencil buffer
glStencilMask(0xFFFFFFFF);
//Always pass the stencil test
glStencilFunc(GL_ALWAYS);
glStencilOp(GL_KEEP, GL_KEEP, GL_INCR);
glEnable(GL_STENCIL_TEST);

//Now render whatever shape you want. Example : A circle.
//The stencil will be 1 where the circle is drawn
RenderMyCircle();

//Now, we want only the framebuffer to be updated if stencil value is 1
glStencilMask(0xFFFFFFFF);
glStencilFunc(GL_EQUAL);
//Do not change stencil
glStencilOp(GL_KEEP, GL_KEEP, GL_KEEP);
glEnable(GL_STENCIL_TEST);

//Render your other stuff
RenderMyStuff();

PollyPocket4eva
03-14-2011, 07:34 PM
Hey,
Thanks for your help. This is not working for me. Here is the code I'm using:



glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT|GL _STENCIL_BUFFER_BIT);

//Allow writing to all bits in the stencil buffer
glStencilMask(0xFFFFFFFF);
//Always pass the stencil test
glStencilFunc (GL_ALWAYS, 0x1, 0x1); //glStencilFunc(GL_ALWAYS);
glStencilOp(GL_KEEP, GL_KEEP, GL_INCR);
glEnable(GL_STENCIL_TEST);

//Now render whatever shape you want. Example : A circle.
//The stencil will be 1 where the circle is drawn
glColor4f(1.0,1.0,1.0,1.0);
gl::drawSolidCircle(Vec2f(0.0,0.0),50.0);

//Now, we want only the framebuffer to be updated if stencil value is 1
glStencilMask(0xFFFFFFFF);
glStencilFunc (GL_EQUAL, 0x0, 0x1); //glStencilFunc(GL_EQUAL);
//Do not change stencil
glStencilOp(GL_KEEP, GL_KEEP, GL_KEEP);
glEnable(GL_STENCIL_TEST);

//Render your other stuff
glColor4f(1.0,0.0,0.0,1.0);
gl::drawSolidRect(Rectf(0.0,0.0,25.0,25.0));


The glStencilFunc's you provided wouldn't compile for me, so I replaced them... Is that correct?

Here is what the above code looks like rendered:
http://tinypic.com/r/2jakq6g/7

What am I doing wrong?

Thanks.

V-man
03-15-2011, 08:54 AM
You can set the mask and reference to 0xFFFFFFFF
glStencilFunc(GL_ALWAYS, 0xFFFFFFFF, 0xFFFFFFFF);
glStencilFunc(GL_EQUAL, 0xFFFFFFFF, 0xFFFFFFFF);

PollyPocket4eva
03-15-2011, 10:00 AM
Hey,
Thanks for your help.
That seems to make no difference. I'm getting the same results as before. Any ideas?
Thanks!

Dan Bartlett
03-15-2011, 11:28 AM
You probably want to disable writing to the color buffer when drawing your stencil shape:

glColorMask(GL_FALSE, GL_FALSE, GL_FALSE, GL_FALSE);
And then re-enable drawing to the color buffer to draw the rest:

glColorMask(GL_TRUE, GL_TRUE, GL_TRUE, GL_TRUE);
If you want to write inside the stencil shape, then your second glStencilFunc should probably be:

glStencilFunc (GL_EQUAL, 0x1, 0x1);
And if you are going to be forming a complex stencil shape with multiple draw calls, then it may be better to use

glStencilOp(GL_KEEP, GL_KEEP, GL_REPLACE); because otherwise you may have values >1 in the stencil buffer if you draw to the same pixel of the stencil buffer multiple times.

Oh, and have you remembered to request a stencil buffer at context creation time?

A couple of articles/tutorials that might be helpful, although quite advanced:
http://nehe.gamedev.net/data/lessons/lesson.asp?lesson=26
http://www.slideshare.net/Mark_Kilgard (improving shadows + reflections via the stencil buffer)

PollyPocket4eva
03-19-2011, 02:30 PM
Hey, thanks.
Got it!