Weired stencil shadows

Hi,

I’m trying to render stencil shadows using the z-fail approach. I already got z-pass shadows to work but the z-fail shadows do not work for some reason. To me it looks like as if there’s something wrong with the back cap of the volume but i can’t figure out what it is. Here’s some rendering code:

vxglColor4f(1.0f, 0.0f, 0.0f, 1.0f);
vxglDepthFunc(GL_LEQUAL);
vxglDepthMask(GL_FALSE);
vxglColorMask(GL_FALSE, GL_FALSE, GL_FALSE, GL_FALSE);
vxglDisableClientState(GL_TEXTURE_COORD_ARRAY);
vxglEnable(GL_STENCIL_TEST);
vxglStencilFunc(GL_ALWAYS, 1, 255);

// Z-Fail shadows (not working )
vxglCullFace(GL_FRONT);
vxglStencilOp(GL_KEEP, GL_INCR, GL_KEEP);
vxglDrawElements(…);

vxglCullFace(GL_BACK);
vxglStencilOp(GL_KEEP, GL_DECR, GL_KEEP);
vxglDrawElements(…);

/*// Z-Pass shadows (working)
vxglCullFace(GL_BACK);
vxglStencilOp(GL_KEEP, GL_KEEP, GL_INCR);
vxglDrawElements(…);

vxglCullFace(GL_FRONT);
vxglStencilOp(GL_KEEP, GL_KEEP, GL_DECR);
vxglDrawElements(…);
//*/

I’m drawing the silhouette as triangles which form a quad. I also draw the projected, light facing triangles in reversed order (CCW as CW) as the back cap of the shadow volume. As what i heared i don’t need a front cap for the shadows to work.

I properly load the infinite projection matrix. My projected vertices are defined as (vertex - lightPos) with a w value of 0.0. a unprojected vertex simply has a w value of 1.0. I assume this is correct because z-pass shadows work

When drawing the shadows with the z-pass method, everything is fine except the usual camera in shadow problem. Even when drawing the back caps. But when it comes to the z-fail approach, most of the scene is shadowed and occassional polygonal shapes where the geometry is lighted are moving with the camera. strange eh?

I’f be happy if someone could help me with this problem. There seems to be a lot of practice with this shadow tech around here…

Silly me, didn’t know how to write “wierd”. doh!

[This message has been edited by Jens Scheddin (edited 01-11-2004).]

True, for z fail, you don’t need to calculate a front cap when the viewer is inside the shadow volume. You still must have a closed shadow volume though. This means you have to process the light facing triangles as well - not just project them at infinity.

JotDot:

Ok, but what do you mean by “you have to process the light facing triangles as well”? How should I process them?
I do have a closed shadow volume from the light facing triangles which I also render, if that wasn’t clear. It has a constant winding and all. I know this because z-pass shadows do work with exactly the same shadow volume as i use for z-fail shadows.
Even when I add the far cap to the z-pass approach, the result is still correct. But if i use the z-fail, then the result is totally wrong.

Another thing I noticed while searching the forums about my problem:
Some guy named Dtag posted this thread: http://www.opengl.org/discussion_boards/ubb/Forum3/HTML/010097.html

I think that he had exactly the same problem because I’m also using quake like geometry. My world geometry is divided into “faces”. each face is a planar surface consisting of one or more triangles. Each face has it’s own facing (plane) and so on.
As he finally posted it was some sort of bug with generating the shadow volume. It would be more helpfull if he had posted his sollution too

Maybe i’ll have to e-mail him about this. But this would be my last step. I hope to get this solved via this forum…

I may have misunderstood what you were saying but from what I read, it seemed like you didn’t treat the light facing triangles as part of the shadow volume. If you don’t process those triangles as part of your shadow volume then the volume will not be closed.

“A picture is worth a thousand words…” (Aka a screen shot would probably make it much easier to understand what your problem is)

JotDot:

My shadow volume consists ONLY of triangles facing the light source. The back caps are the projected triangles facing the light with a reversed winding. As far as i understand this is all i need. or am i missing something?

I use light facing triangles only, because the way my geometry is arranged (as previously stated). My surfaces don’t know which surface might share an edge with it. For z-pass this doesn’t seem to be a problem.

rgpc:
i’ll have to build a simple scene and take a few screenshots tomorrow. do you know where to get free webspace fast?

Are you using nVidia hardware? And are you utilizing the extension NV_depth_clamp?

I’m haven’t used the Depth Pass method so I don’t know if that can operate without worrying about depth clipping but if you are extruding to infinity then your far cap is being clipped by the far plane - unless you are using NV_depth_clamp. Which is only available (AFAIK) on NV hardware.

Ps. It’s actually “weird”

rgpc:

I’m developing on a machine with radeon 9700. So, no i don’t use NV_depth_clamp
But as far as i understand, the infinite projection matrix should deal with the far cap not beeing clipped. Also, could this really explain why most of the scene will get shadowed? i tried to render without capping at all but of course i get artifacts. but not as bad as with rendering the far cap. This is odd but i think it must have something to do with the way i’m creating the shadow volume and/or cap.

Here’s some code of this:

for (surface=light->mp_visibleSurfaces;surface;surface=surface->mp_lightChain)
{
if (surface->m_plane.VxClassifyPoint(light->m_center) != VxPlane::VX_POSITIVE_HALFSPACE)
continue;

// Shadow volume back cap
verts = mp_vertices + surface->m_firstVertex;
for (i=surface->m_numElements-1;i>=0;–i)
{
*iBuffer = numVerts + mp_elements[surface->m_firstElement+i];
++iBuffer;
++numElems;
}
for (i=0;im_numVertices;++i)
{
*vBuffer = VxVector4(
verts[i].m_position[0] - light->m_center[0],
verts[i].m_position[1] - light->m_center[1],
verts[i].m_position[2] - light->m_center[2],
0.0f);
++vBuffer;
++numVerts;
}

// Shadow volume
for (i=0;im_numEdges;++i)
{
VxEdge& edge=surface->mp_edges[i];

  *iBuffer = numVerts + 0;
  ++iBuffer;
  *iBuffer = numVerts + 1;
  ++iBuffer;
  *iBuffer = numVerts + 2;
  ++iBuffer;
  *iBuffer = numVerts + 2;
  ++iBuffer;
  *iBuffer = numVerts + 1;
  ++iBuffer;
  *iBuffer = numVerts + 3;
  ++iBuffer;
  numElems += 6;

  vert1 = &(mp_vertices[edge.m_vertexIndex[0]+surface->m_firstVertex].m_position);
  vert2 = &(mp_vertices[edge.m_vertexIndex[1]+surface->m_firstVertex].m_position);
  *vBuffer = VxVector4((*vert2)[0], (*vert2)[1], (*vert2)[2], 1.0f);
  ++vBuffer;
  *vBuffer = VxVector4((*vert1)[0], (*vert1)[1], (*vert1)[2], 1.0f);
  ++vBuffer;

  vert3 = (*vert2) - light->m_center;
  vert4 = (*vert1) - light->m_center;
  *vBuffer = VxVector4(vert3[0], vert3[1], vert3[2], 0.0f);
  ++vBuffer;
  *vBuffer = VxVector4(vert4[0], vert4[1], vert4[2], 0.0f);
  ++vBuffer;

  numVerts += 4;

}
}

hmm, so i wasn’t that wrong with my “weired”? lol

Originally posted by Jens Scheddin:
But as far as i understand, the infinite projection matrix should deal with the far cap not beeing clipped.

Der, of course it does… (I don’t use an infinite proj matrix - and I glossed over that bit…)

If most of your scene is being shadowed (doesn’t look like your shadow is turned inside out does it?) then perhaps there’s something in your stencil setup that may be wrong. There used to be a paper kicking around the nVidia web site on “Practical and robust stencil shadows” or the like, that had everything you needed to know (including the code) for creating z-fail shadow volumes (that’s what I used).

It’s not on the site (and hasn’t been for a while) I’ll have a look tonight and see if I can find it for you (plus I can use it to check your code).

FYI – the mother of all stencil shadow articles:

http://www.gamasutra.com/features/20021011/lengyel_01.htm

Jens, are you inverting the winding of your far cap faces? If not, try putting in a glFrontface(GL_CW) before rendering your far caps (and set it back to GL_CCW afterards). Sounds like that might be it.

I see where I wasn’t clear before. I have never bothered with z pass method. I do know of the problem when the shadow volume intersects the near plane in certain circumstances. From what I know, in order to correct this problem with z pass, you have to calculate a front cap in that situation. This is what I meant by “calculate a front cap” in my first post. This is not needed with z fail.

Using the infinite projection matrix with the z fail approach, you just have to present a closed volume. From your code, I see you have a “back cap”. Using that terminology, (which most people use) you are missing the “front cap” which is the actual triangle that caused that shadow volume. Without it, the volume is not closed.

In other words, your shadow volume for a single triangle would have five sides: The actual triangle, the extruded triangle (backfacing) and the three extruded sides.

using a C-like pseudocode:

float * v[3]; // Points to the three verts of your triangle

glBegin(GL_TRIANGLES);
// Front facing triangle
glVertex4fv(v[0]); // w=1
glVertex4fv(v[1]); // w=1
glVertex4fv(v[2]); // w=1
// Backfacing
glVertex4fv(v[2]-LightPos); // w=0
glVertex4fv(v[1]-LightPos); // w=0
glVertex4fv(v[0]-LightPos); // w=0
glEnd();

glBegin(GL_QUAD_STRIP);
// Extruded sides
glVertex4fv(v[0]); // w=1
glVertex4fv(v[0]-LightPos); // w=0
glVertex4fv(v[1]); // w=1
glVertex4fv(v[1]-LightPos); // w=0
glVertex4fv(v[2]); // w=1
glVertex4fv(v[2]-LightPos); // w=0
glVertex4fv(v[0]); // w=1
glVertex4fv(v[0]-LightPos); // w=0
glEnd();

I hope I am a bit clearer this time. I am quite ill today and I just hope I haven’t confused the issue more. I have successfully used the z fail approach with an infinite far plane matrix.

rgpc:
thanks, i know this paper. it gave me a basic understanding of how the shadows work and such. it’s really good. so i don’t think there is something wrong with my stencil setup.

But while taking some screenshots (i’ll upload in an hour or so) i noticed that when the camera is inside the shadow volume, the shadow is well for this light if i render z-fail with front and back caps (it doesn’t matter if back cap uses inverted winding). Moving out of the shadow makes everything shadowed by that light. it seems as if the other geometry that doesn’t actually cast a shadow is shadowing everything outside the shadow volume.

Eric Lengyel:
thanks, i also know this article. it’s really good, especially in explaining optimisation methods. The only thing that is not so clear after reading it is how to build the caps. But i heared there is the second edition of your “mathematics for 3D game programming & computer graphics” with a reworked version of it. right?

Madoc:
yes, i’m using the reversed winding for the far cap.

JotDot:
Well, now i know what you ment. But i don’t wand to use z-pass shadows when the camera is inside the shadow. i heared of this beeing hard to make waterproof and it’s way too expansive.
If i render the shadow volume like you described, then i get the result i mentioned in this post.

Here are the images. i hope it’ll help…

UPDATE: Please right click and select “save as…” if the link doesn’t work. sorry

ZPass rendering without any caps (looks like as supposed be):
http://projectnemesis.netfirms.com/zpass_no_capping.jpg

ZPass with back caps renderd (no difference at all):
http://projectnemesis.netfirms.com/zpass_back_capping.jpg

ZFail without any caps (looks ok so far, doesn’t it?):
http://projectnemesis.netfirms.com/zfail_no_capping.jpg

ZFail with front and back caps but no reversed winding for both of them:
[http://projectnemesis.netfirms.com/zfail_front_back(no rev. winding)_capping.jpg](http://projectnemesis.netfirms.com/zfail_front_back(no rev. winding)_capping.jpg)

ZFail with back capping (reversed winding). As you see you don’t see any lighting at all :
http://projectnemesis.netfirms.com/zfail_back_capping.jpg

And these two are from another test map:

ZPass without capping:
http://projectnemesis.netfirms.com/2zpass_no_capping.jpg

ZFail when viewed from inside the shadow volume (otherwise everything is shadowed):
http://projectnemesis.netfirms.com/2zfail_front_back_capping_(inside_shadow).jpg

i can post a link to a demo for you if this might help more. but the renderer currently only supports two render paths: R200 (ARB_vertex_program, ATI_fragment_shader) and an ARB2-path (ARB_fragment_program, ARB_vertex_program). the images are from the R200 path. so just ask for it if you want to try it out.

Ohh, and sorry for using those tenebrae textures. i needed something for testing and they just look so cool

[This message has been edited by Jens Scheddin (edited 01-12-2004).]

Eric Lengyel:
thanks, i also know this article. it’s really good, especially in explaining optimisation methods. The only thing that is not so clear after reading it is how to build the caps. But i heared there is the second edition of your “mathematics for 3D game programming & computer graphics” with a reworked version of it. right?

Yes, the second edition has an updated version of the stencil shadow article in it, but the changes are mostly just to utilize newer extensions.

The front and back caps are really easy to calculate. Assuming you have already identified which polygons face the light source and which face away (to determine the silhouette), the front cap is just made up of those facing the light without any modification to their vertices. The back cap consists of the polygons facing away from the light, but with their vertices extruded away from it to infinity. See Equation 15 in the article on Gamasutra and the two paragraphics above it.

I see, but rendering the shadow volume caps as you described doesn’t change anything.

Something i fergot to mention: I’m calculating the edges per surface. so if we have a quadratic surface there will be 4 edges, whether or not the surface has a neighbouring surface or not and if the surface is facing the light, i use these 4 edges to calculate the shadow volume. for z-pass this works fine but now i think my problems are because of this.
let’s assume there is a surface next to another. they are poth sharing the same plane and so are both facing the light source. at the point where these two surfaces meet there are two edges with a different winding. with z-pass shadows these edged are eleminating each other when i render them. isn’t this happening for the z-fail approach, too?

Hell, what a mass. I finally got it:

I had to use polygon offset of (0.0, 600.0). otherwise the front cap was completely clipped away which caused the problems i had.
I now render the extruded silhouette edges with the front cap (unprojected vertices) and back cap (extruded vertices w reversed winding).

Thanks to everyone for your help