Spotlight and shadowmapping issues

I’m trying to implement shadowmapping for spotlights in an OpenGL 4.3 deferred renderer. The problem is described below.

Here is the scene from the cameras POV, with shadow calculations commented-out:

Here is the depth texture generated during the shadow pass.

The two images describe the problem: in the first image, you can see small spots of light along the edges of the ground plane, and second image shows this up-close.


Here is the fragment shader for the spotlight lighting pass:

    #version 430      
    layout(std140) uniform; 
                           
    uniform UnifDirLight  
    {       
        mat4 mWVPMatrix;     // camera world-view-projection matrix
        mat4 mVPMatrix;      // the spotlights view-projection matrix
        vec4 mLightColor; 
        vec4 mLightDir;   
        vec4 mGamma;     
        vec2 mScreenSize;  
    } UnifDirLightPass;  
        
    layout (binding = 2) uniform sampler2D unifPositionTexture; 
    layout (binding = 3) uniform sampler2D unifNormalTexture;   
    layout (binding = 4) uniform sampler2D unifDiffuseTexture;  
    layout (binding = 5) uniform sampler2D unifShadowTexture; 
                                                               
    out vec4 fragColor;                                       
                                                          
    void main()    
    {    
        vec2 texcoord = gl_FragCoord.xy / UnifDirLightPass.mScreenSize;
                                                                        
        vec3 worldPos = texture(unifPositionTexture, texcoord).xyz;    
        vec3 normal   = normalize(texture(unifNormalTexture, texcoord).xyz); 
        vec3 diffuse  = texture(unifDiffuseTexture, texcoord).xyz;     
               
        vec4 lightClipPos = UnifDirLightPass.mVPMatrix * vec4(worldPos, 1.0);
        vec3 projCoords   = lightClipPos.xyz / lightClipPos.w;              
                                                                               
        projCoords.x = 0.5 * projCoords.x + 0.5;                         
        projCoords.y = 0.5 * projCoords.y + 0.5;       
        projCoords.z = 0.5 * projCoords.z + 0.5;       
                     
        float depthValue   = texture(unifShadowTexture, projCoords.xy).x;  
        float shadowFactor = 0.0;                                         
        if (depthValue >= (projCoords.z + 0.0001))               
             shadowFactor = 1.0;  
                                 
        float angleNormal = clamp(dot(normal, UnifDirLightPass.mLightDir.xyz), 0, 1);
         
        fragColor = vec4(diffuse, 1.0) * shadowFactor * angleNormal * 
            UnifDirLightPass.mLightColor;
    }

I don’t see where the problem is, or what might be causing the strange white spots on the edges of the ground plane. Any pointers?

Maybe I’m missing something, but I don’t see how your last image maps to the previous ones. And the immediately previous one seems blank. Except for some kind of blur effect that’s probably part of your task bar.

In the third image, you can see some distortion on the far end of the ground plane; and the last image shows that distortion close-up, and shows that the distortion is actually that only the edges are lit. Why are only the ground edges lit and all else in shadow?

I don’t see it in the 3rd image. But I’ll take your word for it.

Does that edge correspond to the edge of your GBuffer textures or the edges of your shadow map? If so, look at your texture filtering wrap modes. What do you have them set to?

What do you mean ‘as the edge’ of the shadow map/gbuffer texture? Dosn’t that depend on where the camera is and looking?

I use the following sampler properties to sample the depth map:

        GLCALL(glSamplerParameteri(mTextureSampler, GL_TEXTURE_MAG_FILTER, GL_LINEAR));
        GLCALL(glSamplerParameteri(mTextureSampler, GL_TEXTURE_MIN_FILTER, GL_LINEAR));
        GLCALL(glSamplerParameteri(mTextureSampler, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE));
        GLCALL(glSamplerParameteri(mTextureSampler, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE));

But even if the edges are lit due to wrong sampler parameters, it dosn’t explain the lack of light in the scene. What is so strange is that as you can see from the depth map, since it is from the lights POV, it should light the whole scene - not just along the edges of the ground plane?

I’m talking about where the texture coordinates you compute have you lookup up into the edge of the shadow map, or the edge of the GBuffer texture. Of course the latter would be near the edges of your screen.

I use the following sampler properties to sample the depth map:

        GLCALL(glSamplerParameteri(mTextureSampler, GL_TEXTURE_MAG_FILTER, GL_LINEAR));
        GLCALL(glSamplerParameteri(mTextureSampler, GL_TEXTURE_MIN_FILTER, GL_LINEAR));
        GLCALL(glSamplerParameteri(mTextureSampler, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE));
        GLCALL(glSamplerParameteri(mTextureSampler, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE));

You’re using CLAMP_TO_EDGE which is good, so scratch the idea I had.

[quote]But even if the edges are lit due to wrong sampler parameters, it dosn’t explain the lack of light in the scene. What is so strange is that as you can see from the depth map, since it is from the lights POV, it should light the whole scene - not just along the edges of the ground plane?

Have you used the shader lighting code before? If not, is it possible there’s a bug there? To work your way through the problem, I’d cook a debug mode where you can selectively render each GBuffer channel to the screen so you can visually verify its correct. That’ll help you verify that at least the inputs you are feeding into lighting are correct. Also, simplify the problem gradually any way you can think of to try and whittle down the problem space (for instance, disable application of the shadow map).

I have done that, as I wrote it in the OP :stuck_out_tongue: the first image you see is the lighting shader I posted except the “shadowFactor” variable is commented-out in the return statement. Look at the cube in the 1st screenshot; you see the light looking down on it, and it corresponds to the shadowmap (2nd picture) aswell. I’m positive the input textures are correct, but I don’t see why the “shadowFactor” will always (except at the edges) be 0, making the whole scene dark.

There is no issues you see concerning shadowfactor calculation in the lighting shader?

Sorry, it’s not jumping out at me. It’s a bit difficult since I’m not seeing the artifacts you mention in the original image. Could be shadow buffer coverage. Could be errors in your math (for instance, it concerns me a bit that you’re just dividing by w before you ensure that you’re in-frustum for the face of the spotlight you’re looking up into). You might compare your code against this working spotlight shadow example. Alternatively you could try posting a short standalone test program for folks to work with.

UnifDirLightPass.mVPMatrix * vec4(worldPos, 1.0)

There is nothing wrong with the order of the following matrix multiplication? It gives the same result as P * V * W?

Could you elaborate on that?

[QUOTE=TheKaiser;1257326]

UnifDirLightPass.mVPMatrix * vec4(worldPos, 1.0)

There is nothing wrong with the order of the following matrix multiplication? It gives the same result as P * V * W?[/quote]
Looks OK.

Could you elaborate on that?

Just that before you divide by w, you typically want to make sure you’re inside the frustum first. -w < x,y,z < w. This to avoid divide-by-zeros (e.g. if w=0) and out-of-frustum values passing down through the math.

Alright, will check that.
I’d like to add all depth values sampled from the shadowmap appear to be in 0.97+; must be inconsistent with the 2nd picture?

[QUOTE=TheKaiser;1257342]Alright, will check that.
I’d like to add all depth values sampled from the shadowmap appear to be in 0.97+; must be inconsistent with the 2nd picture?[/QUOTE]

For a depth buffer rendered with a perspective projection (e.g. that from a point light source shadow map), the values are non-linear and do tend to move toward 1.0 fairly quickly past the near clip plane. You can use something like this to get back eye.z from the depth buffer and then render that as intensity (i.e. map -n…-f to 0…1).

I made the matrix bias multiplication with the light view-projection matrix before the shader execution and removed the added shadow bias constant, and here’s the result:

http://s18.postimg.org/9t6sft7s9/shadowmap.png

The white artifacts are due to lack of shadow bias constant, but it shows there is actual shadowing; the strange thing is, you can see the models are lit, but the ground is not. Why would the shadowFactor cause the ground plane to be dark?

    const std::string gDirLightFragmentShader =
    "#version 430                                                                                                                   
 \
                                                                                                                                    
 \
    layout(std140) uniform;                                                                                                         
 \
                                                                                                                                    
 \
    uniform UnifDirLight                                                                                                            
 \
    {                                                                                                                               
 \
        mat4 mWVPMatrix;                                                                                                            
 \
        mat4 mVPMatrix;                                                                                                             
 \
        vec4 mLightColor;                                                                                                           
 \
        vec4 mLightDir;                                                                                                             
 \
        vec4 mGamma;                                                                                                                
 \
        vec2 mScreenSize;                                                                                                           
 \
    } UnifDirLightPass;                                                                                                             
 \
                                                                                                                                    
 \
    layout (binding = 2) uniform sampler2D unifPositionTexture;                                                                     
 \
    layout (binding = 3) uniform sampler2D unifNormalTexture;                                                                       
 \
    layout (binding = 4) uniform sampler2D unifDiffuseTexture;                                                                      
 \
    layout (binding = 5) uniform sampler2D unifShadowTexture;                                                                       
 \
                                                                                                                                    
 \
    out vec4 fragColor;                                                                                                             
 \
                                                                                                                                    
 \
    void main()                                                                                                                     
 \
    {                                                                                                                               
 \
        vec2 texcoord = gl_FragCoord.xy / UnifDirLightPass.mScreenSize;                                                             
 \
                                                                                                                                    
 \
        vec3 worldPos = texture(unifPositionTexture, texcoord).xyz;                                                                 
 \
        vec3 normal   = normalize(texture(unifNormalTexture, texcoord).xyz);                                                        
 \
        vec3 diffuse  = texture(unifDiffuseTexture, texcoord).xyz;                                                                  
 \
                                                                                                                                    
 \
        vec4 lightClipPos  = UnifDirLightPass.mVPMatrix * vec4(worldPos, 1.0);                                                      
 \
        vec3 projCoords    = lightClipPos.xyz / lightClipPos.w;                                                                   
 \
                                                                                                                                    
 \
        float depthValue   = texture(unifShadowTexture, projCoords.xy).x;                                                           
 \
        float shadowFactor = 0.0;                                                                                                   
 \
        if (depthValue >= (projCoords.z))                                                                                  
 \
             shadowFactor = 1.0;                                                                                                    
 \
                                                                                                                                    
 \
        float angleNormal = clamp(dot(normal, UnifDirLightPass.mLightDir.xyz), 0, 1);                                               
 \
                                                                                                                                    
 \
        fragColor = vec4(diffuse, 1.0) * shadowFactor * angleNormal * UnifDirLightPass.mLightColor;                                 
 \
    }                                                                                                                               
";

In your code it’s always 0 or 1, and you’re multiplying it directly into the output color. If it’s 0, so’s the output color.

I managed to get it working using the sampler2Dshadow and the following adjustments:

    const std::string gDirLightFragmentShader =
    "#version 430                                                                                                                   
 \
                                                                                                                                    
 \
    layout(std140) uniform;                                                                                                         
 \
                                                                                                                                    
 \
    uniform UnifDirLight                                                                                                            
 \
    {                                                                                                                               
 \
        mat4 mWVPMatrix;                                                                                                            
 \
        mat4 mVPMatrix;                                                                                                             
 \
        vec4 mLightColor;                                                                                                           
 \
        vec4 mLightDir;                                                                                                             
 \
        vec4 mGamma;                                                                                                                
 \
        vec2 mScreenSize;                                                                                                           
 \
    } UnifDirLightPass;                                                                                                             
 \
                                                                                                                                    
 \
    layout (binding = 2) uniform sampler2D unifPositionTexture;                                                                     
 \
    layout (binding = 3) uniform sampler2D unifNormalTexture;                                                                       
 \
    layout (binding = 4) uniform sampler2D unifDiffuseTexture;                                                                      
 \
    layout (binding = 5) uniform sampler2DShadow unifShadowTexture;                                                                 
 \
                                                                                                                                    
 \
    out vec4 fragColor;                                                                                                             
 \
                                                                                                                                    
 \
    void main()                                                                                                                     
 \
    {                                                                                                                               
 \
        vec2 texcoord = gl_FragCoord.xy / UnifDirLightPass.mScreenSize;                                                             
 \
                                                                                                                                    
 \
        vec3 worldPos = texture(unifPositionTexture, texcoord).xyz;                                                                 
 \
        vec3 normal   = normalize(texture(unifNormalTexture, texcoord).xyz);                                                        
 \
        vec3 diffuse  = texture(unifDiffuseTexture, texcoord).xyz;                                                                  
 \
                                                                                                                                    
 \
        vec4 lightClipPos = UnifDirLightPass.mVPMatrix * vec4(worldPos, 1.0);                                                       
 \
        vec3 projCoords   = lightClipPos.xyz / lightClipPos.w;                                                                      
 \
        projCoords.z      = projCoords.z - 0.00001;                                                                                 
 \
                                                                                                                                    
 \
        float visibilty  = texture(unifShadowTexture, projCoords); //1.0;                                                           
 \
                                                                                                                                    
 \
        float angleNormal = clamp(dot(normal, UnifDirLightPass.mLightDir.xyz), 0, 1);                                               
 \
                                                                                                                                    
 \
        fragColor = vec4(diffuse, 1.0) * visibilty * angleNormal * UnifDirLightPass.mLightColor;                                    
 \
    }                                                                                                                               
";

As for why I couldnt get it to work without the sampler2Dshadow, I do not know - from what I understand, the two should perform the same function.
Nevertheless, thanks the information!

In order to use a sampler2DShadow, you need to provide a texture which is of base type GL_DEPTH_COMPONENT and have the depth compare mode (GL_TEXTURE_COMPARE_MODE) enabled (e.g. set to GL_COMPARE_R_TO_TEXTURE). If you do so, then the GPU will do the depth comparison behind-the-scenes, possibly with PCF filting (if you enable it with GL_LINEAR and if your GPU supports it).

If you just use a sampler2D, then you shouldn’t enable the compare mode and you of course have to do the depth comparisons yourself.

Sorry, I missed seeing that you were just using a sampler2D (and thus getting 0…1 depth components back from your lookup) and not doing the depth compare logic in your shader.