Spotlight and shadowmapping issues

[TheKaiser]

I'm trying to implement shadowmapping for spotlights in an OpenGL 4.3 deferred renderer. The problem is described below.

Here is the scene from the cameras POV, with shadow calculations commented-out:



Here is the depth texture generated during the shadow pass.



The two images describe the problem: in the first image, you can see small spots of light along the edges of the ground plane, and second image shows this up-close.




Here is the fragment shader for the spotlight lighting pass:

    #version 430      
    layout(std140) uniform; 
 
    uniform UnifDirLight  
    {       
        mat4 mWVPMatrix;     // camera world-view-projection matrix
        mat4 mVPMatrix;      // the spotlights view-projection matrix
        vec4 mLightColor; 
        vec4 mLightDir;   
        vec4 mGamma;     
        vec2 mScreenSize;  
    } UnifDirLightPass;  
 
    layout (binding = 2) uniform sampler2D unifPositionTexture; 
    layout (binding = 3) uniform sampler2D unifNormalTexture;   
    layout (binding = 4) uniform sampler2D unifDiffuseTexture;  
    layout (binding = 5) uniform sampler2D unifShadowTexture; 
 
    out vec4 fragColor;                                       
 
    void main()    
    {    
        vec2 texcoord = gl_FragCoord.xy / UnifDirLightPass.mScreenSize;
 
        vec3 worldPos = texture(unifPositionTexture, texcoord).xyz;    
        vec3 normal   = normalize(texture(unifNormalTexture, texcoord).xyz); 
        vec3 diffuse  = texture(unifDiffuseTexture, texcoord).xyz;     
 
        vec4 lightClipPos = UnifDirLightPass.mVPMatrix * vec4(worldPos, 1.0);
        vec3 projCoords   = lightClipPos.xyz / lightClipPos.w;              
 
        projCoords.x = 0.5 * projCoords.x + 0.5;                         
        projCoords.y = 0.5 * projCoords.y + 0.5;       
        projCoords.z = 0.5 * projCoords.z + 0.5;       
 
        float depthValue   = texture(unifShadowTexture, projCoords.xy).x;  
        float shadowFactor = 0.0;                                         
        if (depthValue >= (projCoords.z + 0.0001))               
             shadowFactor = 1.0;  
 
        float angleNormal = clamp(dot(normal, UnifDirLightPass.mLightDir.xyz), 0, 1);
 
        fragColor = vec4(diffuse, 1.0) * shadowFactor * angleNormal * 
            UnifDirLightPass.mLightColor;
    }

I don't see where the problem is, or what might be causing the strange white spots on the edges of the ground plane. Any pointers?

[Dark Photon]

Maybe I'm missing something, but I don't see how your last image maps to the previous ones. And the immediately previous one seems blank. Except for some kind of blur effect that's probably part of your task bar.

[TheKaiser]

In the third image, you can see some distortion on the far end of the ground plane; and the last image shows that distortion close-up, and shows that the distortion is actually that only the edges are lit. Why are only the ground edges lit and all else in shadow?

[Dark Photon]

I don't see it in the 3rd image. But I'll take your word for it.

Does that edge correspond to the edge of your GBuffer textures or the edges of your shadow map? If so, look at your texture filtering wrap modes. What do you have them set to?

[TheKaiser]

What do you mean 'as the edge' of the shadow map/gbuffer texture? Dosn't that depend on where the camera is and looking?

I use the following sampler properties to sample the depth map:

        GLCALL(glSamplerParameteri(mTextureSampler, GL_TEXTURE_MAG_FILTER, GL_LINEAR));
        GLCALL(glSamplerParameteri(mTextureSampler, GL_TEXTURE_MIN_FILTER, GL_LINEAR));
        GLCALL(glSamplerParameteri(mTextureSampler, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE));
        GLCALL(glSamplerParameteri(mTextureSampler, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE));

But even if the edges are lit due to wrong sampler parameters, it dosn't explain the lack of light in the scene. What is so strange is that as you can see from the depth map, since it is from the lights POV, it should light the whole scene - not just along the edges of the ground plane?

[Dark Photon]

What do you mean 'as the edge' of the shadow map/gbuffer texture?

I'm talking about where the texture coordinates you compute have you lookup up into the edge of the shadow map, or the edge of the GBuffer texture. Of course the latter would be near the edges of your screen.

[quote]I use the following sampler properties to sample the depth map:

        GLCALL(glSamplerParameteri(mTextureSampler, GL_TEXTURE_MAG_FILTER, GL_LINEAR));
        GLCALL(glSamplerParameteri(mTextureSampler, GL_TEXTURE_MIN_FILTER, GL_LINEAR));
        GLCALL(glSamplerParameteri(mTextureSampler, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE));
        GLCALL(glSamplerParameteri(mTextureSampler, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE));

You're using CLAMP_TO_EDGE which is good, so scratch the idea I had.

But even if the edges are lit due to wrong sampler parameters, it dosn't explain the lack of light in the scene. What is so strange is that as you can see from the depth map, since it is from the lights POV, it should light the whole scene - not just along the edges of the ground plane?

Have you used the shader lighting code before? If not, is it possible there's a bug there? To work your way through the problem, I'd cook a debug mode where you can selectively render each GBuffer channel to the screen so you can visually verify its correct. That'll help you verify that at least the inputs you are feeding into lighting are correct. Also, simplify the problem gradually any way you can think of to try and whittle down the problem space (for instance, disable application of the shadow map).

[TheKaiser]

I have done that, as I wrote it in the OP the first image you see is the lighting shader I posted *except* the "shadowFactor" variable is commented-out in the return statement. Look at the cube in the 1st screenshot; you see the light looking down on it, and it corresponds to the shadowmap (2nd picture) aswell. I'm positive the input textures are correct, but I don't see why the "shadowFactor" will always (except at the edges) be 0, making the whole scene dark.

There is no issues you see concerning shadowfactor calculation in the lighting shader?

[Dark Photon]

Sorry, it's not jumping out at me. It's a bit difficult since I'm not seeing the artifacts you mention in the original image. Could be shadow buffer coverage. Could be errors in your math (for instance, it concerns me a bit that you're just dividing by w before you ensure that you're in-frustum for the face of the spotlight you're looking up into). You might compare your code against this working spotlight shadow example. Alternatively you could try posting a short standalone test program for folks to work with.

[TheKaiser]

UnifDirLightPass.mVPMatrix * vec4(worldPos, 1.0)

There is nothing wrong with the order of the following matrix multiplication? It gives the same result as P * V * W?

Could be errors in your math (for instance, it concerns me a bit that you're just dividing by w before you ensure that you're in-frustum for the face of the spotlight you're looking up into).

Could you elaborate on that?

[Dark Photon]

UnifDirLightPass.mVPMatrix * vec4(worldPos, 1.0)

There is nothing wrong with the order of the following matrix multiplication? It gives the same result as P * V * W?

Looks OK.

Could you elaborate on that?

Just that before you divide by w, you typically want to make sure you're inside the frustum first. -w < x,y,z < w. This to avoid divide-by-zeros (e.g. if w=0) and out-of-frustum values passing down through the math.