Shadow Mapping depth comparison issues

[notoriousRob]

Hello. I am having some issues with a simple implemenation of variance shadows in OpenGL 3.2, GLSL 1.2. I have tried to solve the problem by reading these forums and searching the net at large but, I finally have to post and ask for direct help.

I'm linking a zip of the c++ file and shaders for anyone wishing to bypass my probably muddled explanation. It uses GLM for the matrix and vector math. The lighting is an ambient and diffuse point light.

http://dl.dropbox.com/u/7187892/vsm.zip

The image in the right top corner is of the first pass.
http://i.imgur.com/ysaaY.png (http://imgur.com/ysaaY)

The problem seems to be in the comparison of depths from the shadow map texture lookup and and the depth calculated in the fragment shader of the 2nd pass. I have tried various approaches with the matrix used to get the coordinates.


mat4 shadowMatrix = bias * lightPMVMatrix;
//mat4 shadowMatrix = bias * lightPMVMatrix * inverse(vsmView);
//via the orange book
//mat4 shadowMatrix = bias * modelMatrix * lightViewMatrix * lightPerspective;

Below is the way I am calculating the depth for the 2nd pass which, I think may be too simplistic.

Vertex Shader


varying vec4 worldDepth;
...
worldDepth = LightModel * vPos;

Fragment Shader


float d = length(lightPos - worldDepth.xyz)/far;

The strangest issue is that no matter what I do I can not get the box to cast any shadow. If so, I would at least know I am on the right track and probably be able to solve my other issues. I would appreciate anyone that can point me in the right direction to get over this hump.

[Dark Photon]

The strangest issue is that no matter what I do I can not get the box to cast any shadow.
Well, there are some basic shadow mapping problems here. I'd suggest you put all VSM aside for now and get a simple shadow mapping demo going with straight unfiltered shadow lookups and comparisons. That is, render to DEPTH_COMPONENT texture, bind that back to the shader as a sampler2DShadow, and let the hardware do the shadow compare. Then it's a small step to change that to sampler2D shadow and do the compare yourself. Then go on to adding VSM filtering.

One problem I see is that your shadow FBO has 2 color attachments but no depth attachment (?). Yet you are clearing the depth buffer and rendering with depth test as if there was one.

Really, it'll pay you bigtime to drop back and get a basic shadow mapping test case running, understand it, and then go for fancy filtering. Then if problems occur, suspect the changes between Fabian's original code and yours (link (http://www.fabiensanglard.net/shadowmappingVSM/index.php), link (http://www.fabiensanglard.net/shadowmapping/index.php)).

[notoriousRob]

Thanks for your feedback Dark Photon. It was a good idea. I added the more basic shadow mapping approach to the existing code so that I could switch between them. I quickly saw that my depth textures were different. It became clear that the box was being rendered behind the floor and that was messing up the depths being saved in the texture. Hence, no shadow. So, I learned that manually adding a depth buffer is necessary, even if you are not using it directly. You mentioned the missing depth buffer in your post and indeed, it was the main problem.

It's still not working 100% percent but at least the occluder is casting a shadow now. This link has the latest and I will update it again when I get the rest figured out.
http://dl.dropbox.com/u/7187892/vsm.zip

http://i.imgur.com/eC1Ou.png (http://imgur.com/eC1Ou)

[Dark Photon]

So, I learned that manually adding a depth buffer is necessary, even if you are not using it directly. You mentioned the missing depth buffer in your post and indeed, it was the main problem.

It's still not working 100% percent but at least the occluder is casting a shadow now.
Good deal. Yeah, anytime you want to do DEPTH_TESTing, write depth, do occlusion queries, etc. you need to have a depth buffer out there for those features to work.