View Full Version : stencil shadow problem

08-25-2003, 12:37 PM
After studying Nehe lesson #27, i implemented stencil shadows in my flight simulator. But if you take a look at this screenshot ( http://www.web-discovery.net/self_cast.jpg ), you can clearly see that due to the aircraft mesh, the shadow is very jerky and not smooth. What kind of solutions can i use to get a smoother shadow ? Should i increase the mesh quality ? But how much ? How many million triangles do i need to get a smooth shadow ? Or there are other tweaks ?

08-25-2003, 12:51 PM
looks more like zfighting between the frontcap and your real geometry. try to push your stencil shadow geometry slightly away from the viewer using polygonoffset:

08-25-2003, 03:20 PM
Perhaps not, this could be saddle triangle pairs from non planar quads since it looks related to model tesselation. Or it could be slightly dodgy normals on the polys of the silhouette detection algorithm. You may want to add a duad midpoint in problem aread adjusted for overall curvature while it would double your tris in places it would stop the concavity, alternatively you could try to tesselate the quads along the non saddle diagonal to prevent the concavity.

In general this problem will be mitigated if you correctly illuminate your model.

You have shadows adjacent to almost fully illuminated surfaces. The problem here is that the shadows need to modulate the direct illumination term not the ambient.

When this happens you will see shading fade to almost black before it enters shadow and your problems will seem much less pronounced.

So, try turning ambient to zero and place the illumination where the shadow is being projected from. That should begin to look OK. Next add ambient using another pass, and you'll have a great looking illumination model.

Then you can rearrange the passes to suit your requirements, perhaps drawing ambient in the z fill pass and adding direct later. The main advantage of this is you can easily support multiple lights.

There are problems that remain with coarse tesselation on models that are gouraud or phong shaded but you're not ready to tackle those issues yet, and you may find it acceptable by comparrison to your current results.

[This message has been edited by dorbie (edited 08-25-2003).]

08-25-2003, 03:34 PM
P.S. to picture what I mean by saddle tesselation imagine a quad on a flat surface.

Now lift two vertices of the quad that are diagonally opposite. Now there are two ways to tesselats this quad into two triangles. One way is to draw a diagonal across the lower vertices, the other is to draw a diagonal across the higher vertices. Your shadow problem is caused in part because relative to your model volume the tesselation has been performed across the lower vertices causing a concave region across that quad. Short of some subdivision scheme tesselating across the higher vertices to form a convex quad will produce better behaved shadows.

That's my theory anyway.

All that stuff about the lighting still holds, you only want the shadow to modulate the direct illumination term. That means in general that because the stencil test is a framebuffer op you must split the direct illumination calculation out to a different pass and test the lighting *result* where the entire pass has been modulated by the direct illumination term.

[This message has been edited by dorbie (edited 08-25-2003).]

08-25-2003, 03:41 PM

Hmm... your shadows are cast from above and your model is illuminated from below (or at least from the eye), that's a problem for your application and really undermines what you are trying to achive. Get the basics right and it will improve the rest.

[This message has been edited by dorbie (edited 08-25-2003).]

08-26-2003, 04:01 AM
Thanks to your advices i fixed it ! First of all i corrected lighting (it was rotating with the object so it was all messed up!) and that already improved the smoothness. Then i also used the following instructions:
glPolygonOffset(0.0, -200);

and now it seems to be ok. Take a look here:

Thx again for helping !

08-26-2003, 06:30 AM
You may find your choice of polygon offset values is extremely non portable.

You may want to try (-1, -1) or (-2, -2). Keep it low to minimize adverse effects and test at different depths in the frustum, the downside is that your shadows may cast incorrectly with "punchthrough" etc. But be warned I've found this incorrectly implemented on some implementations.

IMHO offset is not the right fix for your problem anyway, the polygon geometry edges match the edges of your model exactly so it's not z fighting in the clasisc sense (invariance is not guaranteed with different OpenGL state but you can rely on it unless you extrude with a vertex program, and even then you can get it with a shader operation). One of the advantages of stencil shadows is they don't require offsets like this with the adverse effects they cause.

[This message has been edited by dorbie (edited 08-26-2003).]

08-26-2003, 07:03 AM
Thx dorbie, so probably i should try to fix it in the other ways you suggested in your early replies. What do u mean "You may find your choice of polygon offset values is extremely non portable" ? You mean that behaviour can be very different on other vga cards ?