ATI Driver bug!

Seems that alpha test + write to stencil doesn’t work properly on Vista + ATI x1600 + Cat 7.3. Same code works on XP and NVidia hw.

Here is a short explanation what Im trying to do.

// render background
glBegin(GL_QUADS);
{
 glColor3f(1,0,0); glVertex3f(0,0,0);
 glColor3f(1,1,1); glVertex3f(0,m_ViewPortSize.y,0);
 glColor3f(0,0,1); glVertex3f(m_ViewPortSize.x,m_ViewPortSize.y,0);
 glColor3f(0,1,0); glVertex3f(m_ViewPortSize.x,0,0);
}
glEnd();




// write to stencil
glColor4f(1,1,1,1); 
glColorMask(0,0,0,0);	

/* Enable stencil test */ 
glEnable (GL_STENCIL_TEST); 
/* Prepare to write a single bit into the stencil buffer in the area outside the viewport */ 
glStencilFunc (GL_ALWAYS, 0x1, 0x1); 
glStencilOp(GL_KEEP, GL_KEEP, GL_REPLACE);		

glEnable(GL_ALPHA_TEST);
glAlphaFunc(GL_GREATER, 0.45f);

glEnable(GL_TEXTURE_2D);
glBindTexture(GL_TEXTURE_2D, m_rgbaTexture);

glBegin(GL_QUADS);
{
 glTexCoord2f(0,0); glVertex3f(0,0,0);
 glTexCoord2f(0,1); glVertex3f(0,m_ViewPortSize.y,0);
 glTexCoord2f(1,1); glVertex3f(m_ViewPortSize.x,m_ViewPortSize.y,0);
 glTexCoord2f(1,0); glVertex3f(m_ViewPortSize.x,0,0);
}
glEnd();

/* The stencil buffer now has a single bit painted in the area outside the viewport */ 

glDisable(GL_TEXTURE_2D);
glDisable(GL_ALPHA_TEST);



// now, render something over masked area
glStencilOp(GL_KEEP, GL_KEEP, GL_KEEP);    //Stencil buffer read only
glStencilFunc(GL_EQUAL, 1, 1);
glColorMask(1,1,1,1);
glBegin(GL_TRIANGLES);
{
...
}
glEnd();

This code use alpha channel in texture to make mask in stencil buffer and then render something in that mashed area.

On NVidia hardware (XP and Vista) everything work ok.
On ATI hardware rendering is correct only in XP, but on Vista it seems that alpha testing doesn’t work with stencil write. Just like alpha test is ignored…

Have you verified that your pixel format have any stencil bits?

Yes. Pixelformat have 32bpp color with 8 alpha bits, 24bit for depth and 8bit for stencil.

Even more… Realtech’s glView rendering tests for GL2 fail.

If you could make a small sample showing the problem and send to me at emil.persson ‘at’ amd.com I’ll get this to the driver guys.