Initializing multisample stencil buffer in glsl?

[Verdagon]

Hi, I'm implementing stencil-routed a-buffer for some good transparency for my game, as described in http://staffwww.itn.liu.se/~andyn/courses/tncg08/sketches/content/sketches/0518.pdf, and I've got a quick question.

In another stencil-routing algorithm (not multisample) at http://www.tatwood.net/articles/25/stencil-routing, they initialize the entire incrementing stencil pattern themselves, with the following code:


void createStencilPattern()
{
glBindBuffer(GL_PIXEL_UNPACK_BUFFER_ARB, mStencilUnpackBuffer);
unsigned* unpackData = static_cast<unsigned*>(glMapBuffer(GL_PIXEL_UNPACK_BUFFER_ARB, GL_WRITE_ONLY));

for(int tr = 0; tr < TILE_ROWS; ++tr)
{
for(int tc = 0; tc < TILE_COLS; ++tc)
{
for(int gr = 0; gr < GRID_ROWS; ++gr)
{
for(int gc = 0; gc < GRID_COLS; ++gc)
{
int x = gc*TILE_COLS + tc;
int y = gr*TILE_ROWS + tr;
unpackData[y*BUFFER_WIDTH + x] = tr*TILE_COLS + tc;
}
}
}
}

glUnmapBuffer(GL_PIXEL_UNPACK_BUFFER_ARB);
glBindBuffer(GL_PIXEL_UNPACK_BUFFER_ARB, 0);
}


And they do this every time we render the screen. Questions:

1. Isn't this inefficient, because the stencil buffer lives on the graphics card, and we're setting the data manually from the CPU?
2. Would it be more efficient to initialize the stencil pattern inside the graphics card with a shader?
3. If so, how would I go about doing that?
4. Even if I could do that for a regular stencil buffer, I still need to be able to do it with a multisample buffer. Is that possible?

If this is the best way to do it, awesome. I just have an itching feeling that there's a better way out there. Any ideas?

Thank you so much!

- Evan Ovadia

[mbentrup]

Well, technically speaking that code only initializes a (pixel) buffer object that could be used to initialize the stencil buffer resp. texture.

Unfortunately multisample textures cannot be initialized in this fashion, the glTexImage2DMultisample function does not accept a data parameter.

The multisample stencil buffer can be initialized by rendering one fullscreen quad per sample and setting up the StencilFunc, StencilOp and SampleMask to update one sample the stencil buffer for all pixels per pass.

e.g. you have to draw 8 quads to initialize an 8 sample buffer to the stencil values 1 - 8:



glEnable(GL_STENCIL_TEST);
for(i = 0; i < 8; i++) {
glStencilFunc(GL_ALWAYS, i + 1, 0xff);
glStencilOp(GL_REPLACE, GL_REPLACE, GL_REPLACE);
glSampleMaski(0, 0x1 << i);
renderFullScreenQuad();
}


(If you have the GL_ARB_shader_stencil_export extension, you can theoretically do this in one pass by writing gl_SampleID + 1 into the gl_FragStencilRefARB output variable.)

[tatwood]

Hi, I'm the author of the stencil routing example at tatwood.net, and just ran across this post. To clarify a couple of things:



And they do this every time we render the screen. Questions:

1. Isn't this inefficient, because the stencil buffer lives on the graphics card, and we're setting the data manually from the CPU?
The function referenced above is called once on initialization to seed a renderbuffer object, which is then transferred to the stencil buffer every frame using glDrawpixels. This still requires a copy, but it should be a GPU->GPU transfer. It is not ideal, as glDrawPixels is deprecated, but I went this way because I ran into a driver bug when I was using a depth stencil texture instead. The example includes code for both methods, but the depth stencil texture method is disabled by default. Hope that helps.