I am really confused with the use of the stencil buffer. I was trying to modify the stencil example in the red book to just draw
a square in the center of the screen and then try to see what would happen if I then tried to draw in there again. I got some
weird results that I was not able to interpret. Can someone please help me with this. I would appreciate a simple example. I was trying to define a square stencil area in the middle of the screen and then see what happens when I drew other objects that
overlaped it partially. I guess I don’t understand exactly what glStencilFunc and glStencilOp are doing. I don’t understand
what the argument mask means in glStencilFunc.
A good example is hard to find but here some explanations.
You have to think that the stencil buffer bits are property bits.
if you use an 8bits stencil buffer that means that you can have 8 properties. So imagine those properties are for surfaces and you want to render objects on surfaces with different kind of properties.
Just to be less abstract, let’s say that bit 0 says that the surfaces is wet and that bit 1 says if surface is grass and that bit 2 says that surface is asphalt.
grass is 0000 0010
wet grass is 0000 0011
wet asphalt is 0000 0101
etc.
the bits i have shown you are the reference values that you are going to use.
Before practice, some explanations of the functions.
glStencilfunc(comp, refvalue, mask)
comp is the comparaison function used for the test.
The test is simply that :
(refvalue) && mask compared to (stencil buffer value) && mask)
so you can set the comparaison to equal, not equal, less, greater, always passes, never passes, etc.
if you set the comparaison function to GL_ALWAYS OR GL_NEVER, the refvalue and the mask doesn’t matter because the stencil will always pass or always fail.
As you can see the mask is used to “select” properties from the stencil buffer. but more on that later.
glStencilOp(failstencil, zfail, zpass)
the first param says what happens to the stencil buffer value if the stencil test fails. The two other values are not used.
if the stencil passes than zfail or zpass will be used.
zFail is for what happens when the zbuffer test fails and zpass is for what happen for the zbuffer passes.
Now we have everything to draw our surfaces and our objects.
First, we will draw our surfaces.
glStencilOp(GL_REPLACE, GL_KEEP, GL_REPLACE);
//the first param is actually useless because we will set the stencil to pass all the time.
the second term is GL_KEEP because some surfaces might hide others and so if the zbuffer test fails, we want to keep the upper surface properties on top. the last one, we just set the value to the reference value.
while (we have surfaces to draw)
{
glStencilfunc(GL_ALWAYS, refvalue, 0);
//GL_ALWAYS because we want the surfaces to be drawn no matter what. the revalue are the set of properties that the surface has.
drawsurface();
}
Now will draw our objects.
glStencilOp(GL_KEEP, GL_KEEP, GL_KEEP);
//we put keep to everything because we don’t want the stencil buffer to be updated now.
Our objects also have propertie bits just like the objects that says over wich surfaces they to be drawn.
so the code is
while (still objects)
{
glStencilfunc(GL_EQUAL, 111111111, objectproperties);
drawobject();
}
I’ve set the referenve value to 111111111, but I could also have used the object properties because they are “ANDed” together.
So the comparison is :
(111111111 && objectproperties) == (objectpropertes && surfaces properties)
so if the surfaces has the properties needed to have the object drawn over it, then the comparison will be equal and the object will be drawn.
example: our objects can be draw over wet asphalt and the surface is grass.
mask object mask surface
(11111111 && 00000101) == (11111111 && 0000010)
this fails so the object is not draw.
You can change how the properties react by switching the refvalue and the mask(in the example) and the setting this new mask to something else then 11111111
They are many other options with the stencil buffer. like you can set the glStencilOp parameters to increment or decrement the stencilbuffer bits. This can be used to do shadows and other cool stuff.
Experiment, there is really a lot you can do with the different functions(less, greater, etc).
I hoped this helped! (wow that was long!)
Gorg
[This message has been edited by Gorg (edited 04-12-2000).]
Hi Gorg,
Thanks for taking the time to give me that deep explanation. It will take some playing around to sink in. Thanks again…
Thanks, it helped me alot 