Shadow mapping - another approach?

My idea is to use the shadow map to cast a shadow over the scene after its opaque part has been fully rendered (kind of deferred shadowing). So what I am doing is to render a scene from a light’s view (depth only), render the (opaque) scene from the viewer’s eye position, then render a fullscreen quad over the scene, comparing each depth value of the scene with the corresponding depth value from the shadow map:

enable client states for GL_TEXTURE0
bind scene color buffer to GL_TEXTURE0
enable client states for GL_TEXTURE1
bind scene depth buffer to GL_TEXTURE1
bind shadow map to GL_TEXTURE2
load shadowmap transformation matrix to texture matrix of GL_TEXTURE2
render fullscreen quad, using shader as below

The shadow map’s texture matrix has been setup as follows after having set the camera to the light’s position and view direction:

float lightProjection [16];
float lightModelview [16];
float shadowMapTransformation [16];
glGetFloatv(GL_PROJECTION_MATRIX, lightProjection);
glGetFloatv(GL_MODELVIEW_MATRIX, lightModelview);
glMatrixMode(GL_TEXTURE);
glLoadIdentity();
glTranslatef(0.5f, 0.5f, 0.5f);
glScalef(0.5f, 0.5f, 0.5f);
glMultMatrixf(lightProjection);
glMultMatrixf(lightModelview);
glGetFloatv(GL_TEXTURE_MATRIX, shadowMapTransformation);

Vertex shader:

varying vec4 vertex;
void main() 
{
gl_TexCoord[0] = gl_MultiTexCoord0;
gl_TexCoord[1] = gl_TextureMatrix[2] * gl_Vertex;
gl_Position = ftransform();
gl_FrontColor = gl_Color;
}

Fragment shader:

uniform sampler2D frameBuffer;
uniform sampler2D depthBuffer;
uniform sampler2D shadowMap;
void main() 
{
float sceneDepth = texture2D (depthBuffer, gl_TexCoord [0]).r;
float shadowDepth = texture2DProj (shadowMap, gl_TexCoord [1]).r;
float light = 0.25 + ((sceneDepth < shadowDepth + 0.0005) ? 0.75 : 0.0);
vec3 sceneColor = texture2D (frameBuffer, gl_TexCoord [0]).rgb;
gl_FragColor = vec4 (sceneColor * light, 1.0);
}

Problem is: It doesn’t work.

Is the texture2DProj call with gl_TexCoord [1] wrong (i.e. is gl_TexCoord[1] setup wrong)?
Do I need to multiply the shadow map transformation matrix with the inverse of the viewer’s camera modelview matrix?

You’re multiplying ‘vertex’ by the texture matrix, which has already been multiplied by the modelview matrix. Try multiplying gl_Vertex by the texture matrix instead.

Did that, but still no shadows …

Edit: I think I know what the problem is. Since I am simply trying to patch the shadow map over the otherwise finalized frame buffer, gl_Vertex doesn’t actually contain any world coordinates at the point of rendering where I try to apply the shadow map.

What I need is to compute the shadow map clip space position of a camera clip space position (or vice versa) to make this work. How would I do that?

Is there a way to compute the eye space coordinate from a clip space coordinate and depth value in GLSL (gluUnproject in GLSL, so to speak)? How?

Sorry, I really shouldn’t answer questions when recovering from the flu

You need a depth texture of the main scene in order to convert the screen-space pixel coordinate of the pixel you are shading to world space and from there into light space. If you’re rendering your main scene to an FBO with a depth texture attached, then you don’t have to do much more work. Otherwise, you need to either:

• read the depth buffer to a pixel buffer object via glReadPixels() and then load that into a depth texture via glTexImage2D() using the FBO (haven’t tried that myself)
• re-render your scene to a depth texture
• restructure your main scene so that it renders to an FBO

Once you have the depth texture, bind it and the shadow map texture. Create texture coordinates for your fullscreen quad from [0…1] and do this in your fragment shader:

uniform mat4 modelviewProjInverse;
uniform mat4 shadowTransform;
uniform sampler2D depthTex;
uniform sampler2D shadowTex;

// ....

vec4 ndcPos, wsPos, lsPos;

ndcPos = vec4(gl_TexCoord[0].s - 0.5,
	      gl_TexCoord[0].t - 0.5,
	      texture(depthTex, gl_TexCoord[0].xy) - 0.5,
	      0.5);
wsPos = modelviewProjInverse * ndcPos;
lsPos = shadowTransform * wsPos ;
shadow = shadowProj(shadowTex, lsPos).z;

The modelViewProjInverse is the inverse of the (modelview*proj). The shadowTransform is the light’s modelview * the light’s projection.

Alternatively, you can use gl_FragCoord.xy instead of using texture coordinates, you just need a vec2 uniform representing the viewport size to map it to [-1,1].

Hope that helps.

Thank you for your hints.

The shadowProj call doesn’t work though, since that function doesn’t exist. shadow2DProj cannot be called this way since shadowTex is a sampler2D and not a sampler2DShadow. I guess I need to set the texture compare mode to GL_COMPARE_R_TO_TEXTURE before rendering the shadow map and passing shadowTex as sampler2DShadow, right?

So what I did is this:

uniform sampler2D sceneColor;
uniform sampler2D sceneDepth;
uniform sampler2DShadow shadowMap;
uniform mat4 modelviewProjInverse;
void main() 
{
float z = texture2D (sceneDepth, gl_TexCoord [0]).r;
vec4 ndcPos = vec4 (gl_TexCoord [0].s - 0.5, gl_TexCoord [0].t - 0.5, z - 0.5, 0.5);
vec4 wsPos = modelviewProjInverse * ndcPos;
vec4 lsPos = gl_TextureMatrix [2] * wsPos;
float light = 0.25 + shadow2DProj (shadowMap, lsPos) * 0.75;
gl_FragColor = vec4 (texture2D (sceneColor, gl_TexCoord [0].xy).rgb * light, 1.0);
}

To compute modelviewProjInverse, I multiplied OpenGL’s projection matrix with the modelview matrix after having setup camera and projection:

float modelview [16];
float projection [16];
float modelviewProjInv [16];
GLint matrixMode;
glGetIntegerv (GL_MATRIX_MODE, &matrixMode);
glGetfloatv (GL_MODELVIEW_MATRIX, modelview);
glMatrixMode (GL_PROJECTION_MATRIX);
glPushMatrix ();
glMultMatrixf (modelview);
glGetfloatv (GL_PROJECTION_MATRIX, projection);

modelviewProjInv [0]  = projection [0]; 
modelviewProjInv [1]  = projection [4]; 
modelviewProjInv [2]  = projection [8];
modelviewProjInv [4]  = projection [1]; 
modelviewProjInv [5]  = projection [5]; 
modelviewProjInv [6]  = projection [9];
modelviewProjInv [8]  = projection [2]; 
modelviewProjInv [9]  = projection [6]; 
modelviewProjInv [10] = projection [10];
modelviewProjInv [3]  =
modelviewProjInv [7]  = 
modelviewProjInv [11] = 0.0f;
modelviewProjInv [15] = 1.0f;

modelviewProjInv [12] = -(projection [12] * projection [0]) 
                        -(projection [13] * projection [1]) 
                        -(projection [14] * projection [2]);
modelviewProjInv [13] = -(projection [12] * projection [4]) 
                        -(projection [13] * projection [5]) 
                        -(projection [14] * projection [6]);
modelviewProjInv [14] = -(projection [12] * projection [8]) 
                        -(projection [13] * projection [9]) 
                        -(projection [14] * projection [10]);

Before rendering the shadow map, I have set the proper texture parameters for shadow2DProj:

	glTexParameteri (GL_TEXTURE_2D, GL_TEXTURE_COMPARE_MODE, GL_COMPARE_R_TO_TEXTURE);
	glTexParameteri (GL_TEXTURE_2D, GL_TEXTURE_COMPARE_FUNC_ARB, GL_LEQUAL);
	glTexParameteri (GL_TEXTURE_2D, GL_DEPTH_TEXTURE_MODE, GL_INTENSITY); 				

gl_TextureMatrix [2] has been setup in similar fashion using modelview and projection from rendering the scene from the light’s view.

This doesn’t work though: The entire scene always is in shadow.

Here is something that works half for me:

uniform sampler2D sceneColor;
uniform sampler2D sceneDepth;
uniform sampler2D shadowMap;
#define ZNEAR 1.0
#define ZFAR 5000.0
#define ZRANGE (ZFAR - ZNEAR)
#define ZSCALE (ZFAR / ZRANGE)
#define EyeZ(_z) -ZSCALE / ((ZSCALE - (_z))
void main() 
{
float colorDepth = texture2D (sceneDepth, gl_TexCoord [0]).r;
float z = EyeZ (colorDepth);
vec4 ndc = vec4 ((gl_TexCoord [0].xy - vec2 (0.5, 0.5)) * 2.0 * -z, z, 1.0);
// gl_TextureMatrix [2] = light projection * light modelview * inverse (camera modelview)
vec4 ls = gl_TextureMatrix [2] * ndc;
float shadowDepth = texture2DProj (shadowMap, ls).r;
float light = 0.25 + ((colorDepth < shadowDepth + 0.0005) ? 0.75 : 0.0);
gl_FragColor = vec4 (texture2D (sceneColor, gl_TexCoord [0].xy).rgb * light, 1.0);
}

Problem is, the shadow is moving in the same direction as the viewer does, and with a magnification factor. I reckon the projection should move the shadow to the opposite direction relative to the viewer when the viewer moves, so that the shadow stays at the same place in relation to the light that causes it. I just cannot figure how to fix that.

Here’s a screen shot (the light source is right above the ship in the middle of the scene, so the shadow should be below that ship, and not below the viewer):

Fixed that moving shadow, but there are still problems remaining (see here: http://www.opengl.org/discussion_boards/ubbthreads.php?ubb=showflat&Number=295658#Post295658).