Thanks everyone for your latest responses,
jwatte: I was mistaken when I said
GL_STENCIL_BITS = 0. It is infact 8.
rgpc: Not all of my polygons appear to be drawn as backface. I depends on the orientation of my model. All polygons draw correctly when I switch back to drawing my target without subtracting out the cylinder. Remember, I am using csg.c which allows my to render my target as “A or B” and “B - A”. “A or B” renders both models correctly because no stencil planes are used. “B - A” has problems.
V-man: The csg.c does not use wrapping. I would really like to learn more about wrapping put I cannot find any documentation of this technique. Could someone point me in the right direction?
Below I have copied the “inside” function used to “cut out” model “a” from “b”. It apears to me that the stecil function is always resetting the reference back to zero. Is this what is wrong? I would just define the reference once during initialization and then do the increments or decrements as I draw more polygons. Maybe csg.c has a bug?
void inside(void(*a)(void), void(b)(void), GLenum face, GLenum test)
{
/ draw A into depth buffer, but not into color buffer */
glEnable(GL_DEPTH_TEST);
glColorMask(GL_FALSE, GL_FALSE, GL_FALSE, GL_FALSE);
glCullFace(face);
a();
/* use stencil buffer to find the parts of A that are inside of B
* by first incrementing the stencil buffer wherever B's front faces
* are...
*/
glDepthMask(GL_FALSE);
glEnable(GL_STENCIL_TEST);
glStencilFunc(GL_ALWAYS, 0, 0); // ref and mask do not matter because the polys are always passed
//glStencilOp(failstencil, zfail, zpass); zbuffer fail/pass
//the first param says what happens to the stencil buffer value if the stencil test fails. The two other values are not used
//if the stencil passes than zfail or zpass will be used.
glStencilOp(GL_KEEP, GL_KEEP, GL_INCR);
glCullFace(GL_BACK);
b();
/* ...then decrement the stencil buffer wherever B's back faces are */
glStencilOp(GL_KEEP, GL_KEEP, GL_DECR);
glCullFace(GL_FRONT);
b();
/* now draw the part of A that is inside of B */
glDepthMask(GL_TRUE);
glColorMask(GL_TRUE, GL_TRUE, GL_TRUE, GL_TRUE);
glStencilFunc(test, 0, 1); // ref is 0, test is either equal or not equal, 1 is the mask
glDisable(GL_DEPTH_TEST);
glCullFace(face);
a();
/* reset stencil test */
glDisable(GL_STENCIL_TEST);
}
Please feel free to download the csg.c and view the entire example at: http://www.opengl.org/resources/code/basics/more_samples/index.html
Thanks again,
Tim