View Full Version : Problem With Stencil Operations

11-26-2008, 03:11 PM
I am basic problems adding stencil operations to my apps. Have tried it with several apps. They work fine UNTIL I try to configure the graphics window for stencil operations. My app uses fltk to initialize an OpenGL window. What happens (to all of my fltk-based apps) is that all shaded polys disappear when I enable stenciling. I have a nice glut-based CSG demo done with stenciling. It works fine. I've done a lot of experimenting with it to try to find out what's going wrong with my fltk apps (to no avail). Unfortunately, fltk/fluid based apps involve lots of files (even simple ones). So I can't post the whole thing. However, I've posted a file with the main routine and most of the OpenGL graphics at (http://pastebin.com/m5d4f562). This is the fltk CubeView demo which I've altered, simplifying it as much as possible. It works fine in the form posted. A window is opened inside of which a cube appears that I can rotate, translate, and scale. BUT - if I replace line 102 with 103 and line 139 with line 140 (to enable stenciling), nothing appears in the graphics window. Anyone have a clue why? Thanks.

11-26-2008, 04:05 PM
If you replace the line 139 with Fl::visual (FL_DOUBLE | FL_RGB | FL_STENCIL | FL_DEPTH); and comment the line 102 and take the line 103 to make the parameters of both functions the same. In general (in all case maybe), the stencil buffer and the depth buffer are packed together (stencil is 8 bits and depth is 24 bits so they fit in 32 bits). And I think that stenciling don't work (I never verify this, I will test soon) when there is not a depth buffer enabled.

Edit: No depth buffer enabled, stenciling don't work at all which is logical because often the stencil test depend on depth buffer.

Reedit: Did you forget to call glEnable(GL_STENCIL_TEST); and glEnable(GL_DEPTH_TEST) somewhere?

11-26-2008, 05:47 PM
Tried your suggestion. It didn't work. Still got a blank window. Yes - I have tried enabling and disabling STENCIL_TEST and DEPTH_TEST. My first goal is to get the code to run in a window where FL_STENCIL is turned on, but without doing any stencil testing or operations. So it seems to me that I want to disable GL_STENCIL_TEST. Thanks for your response.

11-27-2008, 12:26 PM
I do not have experience with the FLTK. What happens if you remove both


lines and move all those parameters to the call to Fl::visual(...). Or if you remove the FL::visual(...) call and use only that mode(...) call?

11-28-2008, 05:13 PM
You da man Komat! When I comment out both 'mode' calls, it works! Also, the call to FL::visual seems to be unnecessary. When I comment that out too, the stencil operations still work! I am amazed by that - and sort of pissed off. I spent a LOT of time trying to get that to work. You made my weekend Komat. Where do I send the check :)