Help with shadow map projection.

[DeathCarrot]

So, here's what I'm doing:

Set projection matrix to light's gluPerspective
Set modelview matrix to light's gluLookAt

Render the scene geometry into a depth-only FBO

Set projection matrix to players's gluPerspective
Set modelview matrix to players's gluLookAt
Make texture matrix using:
(bias matrix) * (light's projection) * (light's modelview) * (camera's modelview inverse)

Render scene properly using the following shadow coordinates specified in the vert shader:

shadowCoord = gl_TextureMatrix[3] * gl_ModelViewMatrix * gl_Vertex + vec4(0.0,0.0,-0.004,0.0);

the offset is there to prevent z-fighting.
What is supposed to happen is the camera's modelview inverse in the texture matrix should cancel out the camera transformation in the gl_ModelViewMatrix in the vert shader leaving only the object transformation (I think).

This is what happens:
http://img165.imageshack.us/img165/6050/shadowbroken1pe5.jpg http://img521.imageshack.us/img521/1803/shadowbroken2aj6.jpg

However, if I place the gluLookAt for both the light and scene camera into the projection matrix, shadows are rendered correctly (but lighting breaks):
http://img521.imageshack.us/img521/5075/shadowworkingeh8.jpg

Am I missing out on a step? The drawing of the geometry includes calling glLightfv with GL_POSITION, but taking it out didn't help.

I'm not sure what other details might be requried, but here's the main bits.

Note: I also posted this on GameDev (http://www.gamedev.net/community/forums/topic.asp?topic_id=482156) last week.

Thanks.

[-NiCo-]

First you should check whether camera's modelview inverse is correct by pre-/postmultiplying it with gl_ModelViewMatrix and checking if it returns the identity matrix.

I assume you're using a projective texture lookup instead of a 2D texture lookup?

N.

[zeoverlord]

I think you confuse you matrices here, it's never good to try and cancel one thing out when it's not nessecary.
what i do is that i combine only the light's projection and modelview matrices and put it in the texture matrix.

then i just combine it with the coordinates in the vertex shader
gl_TexCoord[2] = gl_TextureMatrix[0]*gl_Vertex;

then in the fragment shader i transform it to fit the shadow textures space

gl_TexCoord[2] = gl_TexCoord[2]/gl_TexCoord[2].w;
gl_TexCoord[2]=(gl_TexCoord[2]+ 1.0) * 0.5;

and now i only have to compare it with the shadow texture like this
if((shadow.z+0.005) < gl_TexCoord[2].z)

read more here (http://www.flashbang.se/postbreak.php?id=35)

[DeathCarrot]

First you should check whether camera's modelview inverse is correct by pre-/postmultiplying it with gl_ModelViewMatrix and checking if it returns the identity matrix.

I assume you're using a projective texture lookup instead of a 2D texture lookup?

N.
Yup, inverse modelview * modelview has been checked to return the identity. (I transpose the rotation and negatise the translation for the inverse from my recollection)

shadow2DProj(shadowMap,shadowCoord)
is what I use in the frag shader.


I think you confuse you matrices here, it's never good to try and cancel one thing out when it's not nessecary.
what i do is that i combine only the light's projection and modelview matrices and put it in the texture matrix.

then i just combine it with the coordinates in the vertex shader
gl_TexCoord[2] = gl_TextureMatrix[0]*gl_Vertex;

then in the fragment shader i transform it to fit the shadow textures space

gl_TexCoord[2] = gl_TexCoord[2]/gl_TexCoord[2].w;
gl_TexCoord[2]=(gl_TexCoord[2]+ 1.0) * 0.5;

and now i only have to compare it with the shadow texture like this
if((shadow.z+0.005) < gl_TexCoord[2].z)

read more here (http://www.flashbang.se/postbreak.php?id=35)

How do you get the vertex's world-space coordinates, though? gl_Vertex contains the values from the vert buffer/array/glVertex, which are relative to the object's position. I can see this approach working when all object are drawn at the origin, but not otherwise.

Thanks for the replies guys :)

[-NiCo-]

I transpose the rotation and negatise the translation for the inverse from my recollection

That's only valid for the identity rotation matrix.




Modelview = | R3x3 t3x1 |
| 0 1 |

Inverse = | R3X3_transpose -R3X3_transpose*t3x1 |
| 0 1 |


N.

[DeathCarrot]

Wow, thanks, I guess when I checked that the inverse was working I only looked at the rotation matrix (the numbers had very low orders of magnitude), I kind of assumed it would work given the vector maths library I'm using isn't written by me.

Again, thanks, this problem has been a serious pain in the ass for the past week :D

[zeoverlord]

How do you get the vertex's world-space coordinates, though? gl_Vertex contains the values from the vert buffer/array/glVertex, which are relative to the object's position. I can see this approach working when all object are drawn at the origin, but not otherwise.

Thanks for the replies guys :)

Yea sorry about that, i code to fast for my own good sometimes, though it's a quick fix to add that.
1. add the inverse matrix to the texture matrix
2. instead of this
gl_TexCoord[2] = gl_TextureMatrix[0]*gl_Vertex;
i now have
gl_TexCoord[2] = gl_TextureMatrix[0]*gl_ModelViewMatrix*gl_Vertex;

[koushik]

thanks Zeoverlord.

Your comments were helpful to me