Stencil shadows, ambient pass and transparent objects

[RealtimeSlave]

Hello,

as explained in this article...
http://www.gamasutra.com/view/featur...t_stencil_.php
...I would like to split my lighting into ambient pass and a pass for diffuse+specular lighting.

For opaque objects this should be straight forward:

1. Render all objects with global scene ambient
2. For each LightSource
a. Render all affected objects with LightSource.ambient (could be skipped if LS does not contribute to ambient)
b. Calculate shadow volumes
c. Calculate stencil mask
d. Render all affected objects with LightSource.diffuse/specular (only areas not in the shadow are lit thanks to the stencil mask)

Now after that I would like to repeat the same thing in order to render transparent objects.
And the transparent objects should be rendered in back-to-front order (for correct blending).
But how can I do this with separate ambient and diffuse+specular pass?

I cannot just render all affected objects for each LightSource as done in (a.)
and then render all objects with diffuse+specular values as in (d.) without
violating the back-to-front order.

I already thought about something like this

For each transparent Object
For each LightSource
a. Render with LightSource.ambient
b. Reuse shadow volumes from opaque pass
c. Calculate stencil mask
d. Render with LightSource.diffuse/specular

But this sounds like an overkill and maybe is also not 100% correct when it comes to blending.

Help is really appreciated!

[Dark Photon]

Yeah, isn't translucency fun? And it isn't one of shadow volumes strong points either. It takes an already heavyweight algorithm, makes it even more expensive, and adds more limitations.

Let's assume your translucent objects are solid and convex (with backface culling, no translucent overlap within the object), and let's assume they do not themselves cast shadows (in our virtual world that is... ). Then seems something along the lines of what you propose might work. Just before step b. you'll need to seed the depth buffer with those samples visible in the translucent object. This is because step c. counts the shadow volume entries and exits either from the eye to the nearest sample, or from the nearest sample to infinity. So you need the depth value in there. Could be done in step a. for light source 1. But when you go onto light source 2, in step a. the depth buffer will already be set. So you may need to flip to EQUAL depth testing for that. Also, to limit the royal boatload of fill rendering these translucents, you may also want to use stencil to limit the fill to only the visible samples within the object.

If you were really interested in pushing this, you could probably optimize the outer loop from "each translucent object" to "each group of non-overlapping translucent objects".

A random thought for how to possibly speed this up even further if your "groups of non-overlapping translucent objects" can be bounded to a relatively small number and you've got a ton of casters you'd rather not send down the pipe a bunch of times is to just have a deep framebuffer (multiple color/depth buffers), one for each level of alpha, blast them all at the same time from different translucent layers, and composite the result. And even then if many layers of translucency, batching like this might greatly reduce the number of times those shadow volumes are pushed down the pipe.

[RealtimeSlave]

Many thanks for sharing your thoughts. I wonder how the game industry is
doing it (crytek, id & Co) when it comes to stencil shadows :/

Ok would be that the transparent objects don't cast shadows themselves.
But I would like to support back and front face transparency.

[Alfonse Reinheart]

I wonder how the game industry is
doing it (crytek, id & Co) when it comes to stencil shadows

They don't. The game industry pretty much abandoned stencil shadows after a couple of years in favor of shadow maps.

[RealtimeSlave]

Hm at least Doom 3 was using stencil shadows but I have no clue how they deal with transparent objects.

I also find it strange that I find NOTHING useful when googling for "stencil shadows" + "transparent" keywords.

Makes me somehow feel as if I am the first person stumbling over this issue or if I am just too stupid to see the ultra easy solution (apart from shadow maps?!




Regarding shadow maps, I wonder how that can work nicely with omni lights (one of the reasons I was looking into stencil shadows)?

[Alfonse Reinheart]

I also find it strange that I find NOTHING useful when googling for "stencil shadows" + "transparent" keywords.

Makes me somehow feel as if I am the first person stumbling over this issue or if I am just too stupid to see the ultra easy solution (apart from shadow maps?!

Has it occurred to you that maybe the reason "NOTHING useful" comes up is because... there is no solution? And maybe that's why people abandoned shadow volumes in the first place, so that they could have a more general-purpose shadow algorithm rather than one with the limitations of stencil shadows?

Regarding shadow maps, I wonder how that can work nicely with omni lights

The same way they work with directional lights; you just have to render to a cubemap instead of a 2D texture. The basic algorithm is essentially unchanged otherwise.

[Nowhere-01]

the reason i abandoned volume shadows algorithm was the fact it requires manifold and welded geometry. only manifold geometry is a big pain in the ass already, but in combination with inability to use unwelded meshes, i'd certainly need to have a separate version of mesh for volume shadows, which is unsuitable for every other pass.

[Dark Photon]

Has it occurred to you that maybe the reason "NOTHING useful" comes up is because... there is no solution?

No need to be so harsh. Thinking down lines others have not (or refusing to make the assumptions they've made) is how innovation happens. And sometimes web searches don't work as well as you'd like.

Regarding shadow maps, I wonder how that can work nicely with omni lights (one of the reasons I was looking into stencil shadows)?

For an example of point light source shadows, see this example program:

* GLSL - cube shadows - projecting

[RealtimeSlave]

Thank you all for your comments and suggestions.

I'll skip further looks into stencil shadows in this case and instead try to implement omnidirectional shadow mapping:
http://http.developer.nvidia.com/GPU...gems_ch12.html
http://www.altdevblogaday.com/2011/0...hadow-mapping/

@Dark Photon: Thanks for the code sample!