PDA

View Full Version : Stencil works with GL_FLOAT but not with GL_BYTE



Koen
11-20-2007, 11:17 AM
I'm trying to get some stencil operations to work. I'm displaying a 1024x1024 pixels image to the screen, but I would like to not show some parts.

I tried to set up a stencil with GLbytes like this:


GLbyte m_stencildata = new GLbyte[1024*1024];
GLbyte *p=m_stencildata;
for ( int y=0; y<1024; y++ )
for ( int x=0; x<1024; x++, p++ )
*p=((y % 20)>10)?0:0x7F;

glEnable(GL_STENCIL_TEST);
glPixelTransferf(GL_MAP_STENCIL, GL_FALSE);
glClear(GL_STENCIL_BUFFER_BIT);
glStencilFunc(GL_ALWAYS, 1, 1);
glColorMask(GL_FALSE, GL_FALSE, GL_FALSE, GL_FALSE);
glRasterPos2i(0, 0);
glDrawPixels(1024,1024,GL_STENCIL_INDEX,GL_BYTE,m_ stencildata);
glStencilFunc(GL_EQUAL, 1, 1);
glColorMask(GL_TRUE, GL_TRUE, GL_TRUE, GL_TRUE);

delete [] m_stencildata;
Unfortunately, this does not work. If I do the same with GL_FLOAT:


GLfloat m_stencildata = new GLfloat[1024*1024];
GLfloat *p=m_stencildata;
for ( int y=0; y<1024; y++ )
for ( int x=0; x<1024; x++, p++ )
*p=((y % 20)>10)?0.0:1.0;
[...]
glDrawPixels(1024,1024,GL_STENCIL_INDEX,GL_FLOAT,m _stencildata);
[...]

everything works as expected. However, I expect this to be slower than the GL_BYTE approach. I appreciate any pointers on how to get this to work with GL_BYTE!

Koen

Humus
11-20-2007, 02:17 PM
Try using GL_UNSIGNED_BYTE instead. Stencil is unsigned anyway, so your two code snippets are not equivalent. The first writes 0x7F, the latter 0xFF.

Koen
11-21-2007, 02:47 AM
OK, I changed the code to:


GLubyte m_stencildata = new GLubyte[1024*1024];
GLubyte *p=m_stencildata;
for ( int y=0; y<1024; y++ )
for ( int x=0; x<1024; x++, p++ )
*p=((y % 20)>10)?0:0xFF;
[...]
glDrawPixels(1024,1024,GL_STENCIL_INDEX,GL_UNSIGNE D_BYTE,m_stencildata);
[...]

but it doesn't work either.

As a sidenote: I tried GL_BITMAP as well, which worked quite well. I guess that rules out problems in other parts of the code.

I also tried GL_UNSIGNED_SHORT, with no luck.