Again stencil shadows.

To build shadows with stencil buffer

First pass:

  • clear depth and stencil buffers;
  • render solid geometry to fill depth buffer;

Next, for each light source:

  • clear stencil;
  • build stencil shadow mask;
  • process illumination pass;

Ok, but stencil and depth buffer store together and clearing stencil without depth ( or depth without stencil ) -
inefficient. But we can’t clear depth buffer on shadows building.
So, how clear stecil with necessary values before lighting pass without glClear( GL_STENCIL_BUFFER_BIT_CALL )?

There is correct stencil before first light source processing ( we clear it on depth buffer pass ).

Usually after stecil mask building use next stencil test settings:

stencil ref : 0 ( suppose stencil inited with zero before building of stencil shadows mask );
stencil pass : keep
stencil fail : keep
stencil depth fail : keep
stecil func : equal

But if we set DepthFunc to EQUAL, then exactly only one fragment pass depth test, and in that case we can replace
fragment’s stencil value with some_other_value say - ONE ( but there is no such stecil operation like GL_ZERO ),
and for each fragment whose fail on stecil test we set same value - ONE.

Now, after lighting pass whole stencil buffer filled with our ONE value and for next illumination pass
we can set “stencil ref” to ONE for shadwing test, but now for each passed/discarded fragment we set stencil to ZERO
and again stencil buffer filled with ZERO values. And so on.

In that way we work without stencil clearing. But how fill stencil with homogeneous values ?
For unshadowed framents we can use INVERT operation ( we always know that value ), but stencil for shadowed fragment
maybe either. May be clever manipulation with stencil mask ?

Originally posted by Nikolai Timofeew:
Ok, but stencil and depth buffer store together and clearing stencil without depth ( or depth without stencil ) -
inefficient.

This is not necessarily true.

[b]Usually after stecil mask building use next stencil test settings:

stencil ref : 0 ( suppose stencil inited with zero before building of stencil shadows mask );
stencil pass : keep
stencil fail : keep
stencil depth fail : keep
stecil func : equal

But if we set DepthFunc to EQUAL, then exactly only one fragment pass depth test, and in that case we can replace
fragment’s stencil value with some_other_value say - ONE ( but there is no such stecil operation like GL_ZERO ),
and for each fragment whose fail on stecil test we set same value - ONE.[/b]
Assuming that the initial stencil value means the pixel is lit, you can do the following:

stencil reference value: 1
stencil function: less
stencil fail: replace
depth fail: keep
depth pass: replace

Stencil values >0 will fail the stencil test and thus be set to 1. Stencil 0 will stay 0 until you draw the one fragment that passes the depth test. After this, your stencil buffer should be all ones.

Next pass:
stencil reference value: 1
stencil function: equal
stencil fail: zero
depth fail: keep
depth pass: zero

However, this method will probably be slower than just using a stencil clear.

Thank.