Shadow Mapping Problem

[LiveAndFight]

Hey,

I'm trying to implement shadow mapping using Dual Paraboloid shadow mapping or basic cubemap shadow mapping and I get the same error using both techniques. The shadows are more or less being drawn properly but they draw as if depth testing was turned off.

http://img840.imageshack.us/img840/6755/shadowerror.th.png (http://img840.imageshack.us/i/shadowerror.png/)

First I draw the scene with the depth texture using BACK FACE culling(sense my engine is based off of idTech3) and than with the Shadow Map program I draw using FRONT FACE culling. But the shadows seem to be bleeding through floor/walls from surfaces that are below or behind the shadow caster.

For the cubemapped shadows I've been looking at a example that was originally coded in GPU ASM, which I ported over to CG and with my CG->GLSL converter it converts over to GLSL.



// Shadow_Fragment.cg
//

uniform samplerCUBE depthLookup : TEXUNIT0;
uniform sampler2DArrayShadow shadowTexture : TEXUNIT1;

//
// R_ReadFromShadowDepth
//
float R_ReadFromShadowDepth( float4 coords ) {
float shadowOut = 0;

shadowOut = texture( shadowTexture, coords ).r;

return shadowOut;
}

//
// fragment_main
//
void fragment_main ( in float2 texCoord0 : TEXCOORD0,
in float4 worldLightVec : TEXCOORD0,
in float4 EyeXyz : TEXCOORD1,
in float4 xyz : TEXCOORD2,
in float4 LightPos : TEXCOORD3,

out float4 colorOUT : COLOR0 )
{
float4 shadow_tc = texCUBE( depthLookup, -worldLightVec.xyz );

float temp;
temp = shadow_tc.z - 0.75;
if( temp >= 0 ) {
shadow_tc.z = -worldLightVec.z;
}
else {
shadow_tc.z = -worldLightVec.y;
}

temp = shadow_tc.z - 0.25;

if( temp < 0 ) {
shadow_tc.z = -worldLightVec.x;
}

float temp2 = shadow_tc.w * 2 + -1;
shadow_tc.z = shadow_tc.z * temp2;

const float proj_scale = -1.002002;
const float proj_bias = -2.002002;

shadow_tc.w = -shadow_tc.z;
shadow_tc.z = shadow_tc.z * proj_scale + proj_bias;
shadow_tc.w = 1 / shadow_tc.w;
shadow_tc.z = shadow_tc.z * shadow_tc.w;

shadow_tc.z = shadow_tc.z * 0.5 + 0.5;
shadow_tc.w = shadow_tc.z;
//shadow_tc.w += 3; // wtf??

float shadowFinal = 1;
for( int i = 0; i < 1; i++ ) {
shadow_tc.z = 0;

float shadowOut = R_ReadFromShadowDepth( shadow_tc );

/*
float d = clamp(distance(xyz, LightPos) / (800), 0.0, 1.0);

if( d >= 0.1 ) {
shadowOut *= 1.0 - d;
}
else {
shadowOut = 1;
}
*/
shadowFinal += shadowOut;
}

colorOUT = float4( 1, 1, 1, 1 ) * shadowFinal;
colorOUT.a = 1;
}


Any ideas?

[ravage]

Maybe the fbo has no depth buffer to render to.

I had to create a renderbuffer and attach it to gl_depth_attachment

[...]
glFramebufferRenderbuffer(GL_FRAMEBUFFER, GL_DEPTH_ATTACHMENT, GL_RENDERBUFFER, depthRenderBuffer);

that solved my problem with the depth when I used cubemapping to do shadows with an omni-light.

[LiveAndFight]

Thats a interesting idea that it has to do with my FBO setup, but I attach a depth buffer for all my FBO's.

Here is my FBO code:



/*
================
R_CreateShadowLightTexture
================
*/
image_t *jmvImageManager::CreateShadowLightTexture( const char *name, int shadowMapSize, int maxShadowLights ) {
image_t *lightShadowMap;

// Allocate our light shadow map inside of the engine.
lightShadowMap = R_AllocImage( name, shadowMapSize * 4, shadowMapSize * 3 );

// Generate our FBO texture.
qglGenFramebuffersEXT(1, &amp;lightShadowMap->fboHandle);

// Bind our FBO.
qglBindFramebufferEXT(GL_FRAMEBUFFER_EXT, lightShadowMap->fboHandle);

// Create and setup our lightShadowMap texture.
qglBindTexture( GL_TEXTURE_2D_ARRAY_EXT, lightShadowMap->texnum );
qglTexParameteri(GL_TEXTURE_2D_ARRAY_EXT, GL_TEXTURE_MIN_FILTER, GL_LINEAR); // Use PCF if available; otherwise, this falls
qglTexParameteri(GL_TEXTURE_2D_ARRAY_EXT, GL_TEXTURE_MAG_FILTER, GL_LINEAR); // back to nearest neighbor
qglTexParameteri(GL_TEXTURE_2D_ARRAY_EXT, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE );
qglTexParameteri(GL_TEXTURE_2D_ARRAY_EXT, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE );
//qglTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_COMPARE_FAIL_VALUE_ARB, 0.5f);
qglTexParameteri(GL_TEXTURE_2D_ARRAY_EXT, GL_TEXTURE_COMPARE_MODE_ARB, GL_COMPARE_R_TO_TEXTURE);
qglTexParameteri(GL_TEXTURE_2D_ARRAY_EXT, GL_TEXTURE_COMPARE_FUNC_ARB, GL_LEQUAL);
qglTexParameteri(GL_TEXTURE_2D_ARRAY_EXT, GL_DEPTH_TEXTURE_MODE, GL_LUMINANCE);
qglTexImage3D(GL_TEXTURE_2D_ARRAY_EXT, 0, GL_DEPTH_COMPONENT32, shadowMapSize * 4, shadowMapSize * 2, maxShadowLights, 0, GL_DEPTH_COMPONENT, GL_FLOAT, NULL);

// Attach this texture as a DEPTH ONLY texture.
GLuint depthbuffer;
qglGenRenderbuffersEXT(1, &amp;depthbuffer);
qglBindRenderbufferEXT(GL_RENDERBUFFER_EXT, depthbuffer);
qglFramebufferRenderbufferEXT(GL_FRAMEBUFFER, GL_DEPTH_ATTACHMENT, GL_RENDERBUFFER, depthbuffer);
qglFramebufferTexture2DEXT(GL_FRAMEBUFFER_EXT, GL_DEPTH_ATTACHMENT_EXT,GL_TEXTURE_2D, lightShadowMap->texnum, 0);
qglDrawBuffer(GL_NONE);
qglReadBuffer(GL_NONE);

// Get the status of our framebuffer object.
GLenum status = qglCheckFramebufferStatusEXT(GL_FRAMEBUFFER_EXT);
if(status != GL_FRAMEBUFFER_COMPLETE_EXT) {
return NULL;
}

// Unbind our FBO
qglBindFramebufferEXT(GL_FRAMEBUFFER_EXT, 0);

return lightShadowMap;
}




/*
================
R_CreateFrameBufferObject
================
*/
image_t *jmvImageManager::CreateFrameBufferObject( const char *name, int width, int height, int numColorAttachments, bool depthCompare ) {
GLuint depthBuffer;
GLuint stencil_rb;
GLuint diffuseBuffers[ 16 ];
image_t *images[ 16 ];

// Generate our FBO.
images[ 0 ] = R_AllocImage( name, width, height );
qglGenFramebuffersEXT(1, &amp;images[ 0 ]->fboHandle);
qglGenFramebuffersEXT(1, &amp;images[ 0 ]->multiSampleFBO);

// Bind our FBO.
qglBindFramebufferEXT(GL_FRAMEBUFFER_EXT, images[ 0 ]->fboHandle);

if( numColorAttachments == -1 ) {
qglBindTexture(GL_TEXTURE_2D, images[ 0 ]->texnum);
glTexParameterf(GL_TEXTURE_2D,GL_TEXTURE_MIN_FILTE R,GL_LINEAR);
qglTexParameterf(GL_TEXTURE_2D,GL_TEXTURE_MAG_FILT ER,GL_LINEAR);
qglTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
qglTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);

//qglTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_COMPARE_FAIL_VALUE_ARB, 0.5f);
qglTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_COMPARE_FUNC, GL_GEQUAL);
qglTexParameteri(GL_TEXTURE_2D, GL_DEPTH_TEXTURE_MODE, GL_INTENSITY);
//qglTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_COMPARE_MODE, GL_COMPARE_R_TO_TEXTURE);
qglTexImage2D(GL_TEXTURE_2D, 0, GL_DEPTH_COMPONENT24, width, height, 0, GL_DEPTH_COMPONENT, GL_FLOAT, NULL);

GLuint depthbuffer;
qglGenRenderbuffersEXT(1, &amp;depthbuffer);
qglBindRenderbufferEXT(GL_RENDERBUFFER_EXT, depthbuffer);
qglFramebufferRenderbufferEXT(GL_FRAMEBUFFER, GL_DEPTH_ATTACHMENT, GL_RENDERBUFFER, depthbuffer);

qglFramebufferTexture2DEXT(GL_FRAMEBUFFER_EXT, GL_DEPTH_ATTACHMENT_EXT,GL_TEXTURE_2D, images[ 0 ]->texnum, 0);

qglDrawBuffer(GL_NONE);
qglReadBuffer(GL_NONE);
}
else {
// Create our texture attach FBO.
for( int i = 0; i < numColorAttachments; i++ ) {
if( i != 0 ) {
images[ i ] = R_AllocImage( name, width, height );
}

qglBindTexture(GL_TEXTURE_2D, images[ i ]->texnum );
if( i == 4 ) {
qglTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA32F_ARB, width, height, 0, GL_RGBA, GL_FLOAT, NULL);
}
else {
qglTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA16F_ARB, width, height, 0, GL_RGBA, GL_FLOAT, NULL);
}
qglTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
qglTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
qglTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
qglTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);

if( depthCompare ) {
qglTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_COMPARE_MODE, GL_COMPARE_R_TO_TEXTURE);
}
qglFramebufferTexture2DEXT(GL_FRAMEBUFFER_EXT, GL_COLOR_ATTACHMENT0_EXT + i, GL_TEXTURE_2D, images[ i ]->texnum, 0);
}
GLenum status = qglCheckFramebufferStatusEXT(GL_FRAMEBUFFER_EXT);
if(status != GL_FRAMEBUFFER_COMPLETE_EXT) {
return NULL;
}


// Create our multi sample FBO.
qglBindFramebufferEXT(GL_FRAMEBUFFER_EXT, images[ 0 ]->multiSampleFBO);

// Generate our frame and render buffers.
qglGenRenderbuffersEXT(1 , &amp;depthBuffer);
qglGenRenderbuffersEXT(1 , &amp;stencil_rb);
qglGenFramebuffersEXT(numColorAttachments , diffuseBuffers);

// Create our render buffers.
for( int i = 0; i < numColorAttachments; i++ ) {
qglBindRenderbufferEXT(GL_RENDERBUFFER_EXT, diffuseBuffers[ i ]);
// This isn't a ugly hack....:/
if( i == 4 ) {
qglRenderbufferStorageMultisampleEXT(GL_RENDERBUFF ER_EXT, 4, GL_RGB32F_ARB, width, height);
}
else {
qglRenderbufferStorageMultisampleEXT(GL_RENDERBUFF ER_EXT, 4, GL_RGB16F_ARB, width, height);
}
qglFramebufferRenderbufferEXT(GL_FRAMEBUFFER_EXT, GL_COLOR_ATTACHMENT0_EXT + i, GL_RENDERBUFFER_EXT, diffuseBuffers[ i ] );
}

qglBindRenderbufferEXT(GL_RENDERBUFFER_EXT, depthBuffer);
qglRenderbufferStorageMultisampleEXT(GL_RENDERBUFF ER_EXT, 4,GL_DEPTH_COMPONENT24, width, height);
//qglFramebufferRenderbufferEXT(GL_FRAMEBUFFER_EXT, GL_DEPTH_ATTACHMENT_EXT, GL_RENDERBUFFER_EXT, depthBuffer);
qglFramebufferRenderbufferEXT(GL_FRAMEBUFFER_EXT,G L_DEPTH_ATTACHMENT_EXT, GL_RENDERBUFFER_EXT, depthBuffer);
}

// Get the status of our framebuffer object.
GLenum status = qglCheckFramebufferStatusEXT(GL_FRAMEBUFFER_EXT);
if(status != GL_FRAMEBUFFER_COMPLETE_EXT) {
char errorMessage[ 1024 ];
switch(status)
{
case GL_FRAMEBUFFER_UNSUPPORTED_EXT:
//Choose different formats
strcpy(errorMessage, "Framebuffer object format is unsupported by the video hardware. (GL_FRAMEBUFFER_UNSUPPORTED_EXT)(FBO - 820)");
break;
case GL_FRAMEBUFFER_INCOMPLETE_ATTACHMENT_EXT:
strcpy(errorMessage, "Incomplete attachment. (GL_FRAMEBUFFER_INCOMPLETE_ATTACHMENT_EXT)(FBO - 820)");
break;
case GL_FRAMEBUFFER_INCOMPLETE_MISSING_ATTACHMENT_EXT:
strcpy(errorMessage, "Incomplete missing attachment. (GL_FRAMEBUFFER_INCOMPLETE_MISSING_ATTACHMENT_EXT) (FBO - 820)");
break;
case GL_FRAMEBUFFER_INCOMPLETE_DIMENSIONS_EXT:
strcpy(errorMessage, "Incomplete dimensions. (GL_FRAMEBUFFER_INCOMPLETE_DIMENSIONS_EXT)(FBO - 820)");
break;
case GL_FRAMEBUFFER_INCOMPLETE_FORMATS_EXT:
strcpy(errorMessage, "Incomplete formats. (GL_FRAMEBUFFER_INCOMPLETE_FORMATS_EXT)(FBO - 820)");
break;
case GL_FRAMEBUFFER_INCOMPLETE_DRAW_BUFFER_EXT:
strcpy(errorMessage, "Incomplete draw buffer. (GL_FRAMEBUFFER_INCOMPLETE_DRAW_BUFFER_EXT)(FBO - 820)");
break;
case GL_FRAMEBUFFER_INCOMPLETE_READ_BUFFER_EXT:
strcpy(errorMessage, "Incomplete read buffer. (GL_FRAMEBUFFER_INCOMPLETE_READ_BUFFER_EXT)(FBO - 820)");
break;
case GL_FRAMEBUFFER_INCOMPLETE_MULTISAMPLE_EXT:
strcpy(errorMessage, "Incomplete multisample buffer. (GL_FRAMEBUFFER_INCOMPLETE_MULTISAMPLE_EXT)(FBO - 820)");
break;
default:
//Programming error; will fail on all hardware
strcpy(errorMessage, "Some video driver error or programming error occured. Framebuffer object status is invalid. (FBO - 823)");
break;
}
common->Error( errorMessage );
}

// Unbind our FBO
qglBindFramebufferEXT(GL_FRAMEBUFFER_EXT, 0);

return images[ 0 ];
}


The same thing happens with multisample FBO's and regular FBO's.

Here is the actual shadow code as well:




/*
=======================
jmvShadowMap::RunPass
=======================
*/
void jmvShadowMap::RunPass( drawSurf_t *drawSurfs, int entityNum ) {
GL_State(GLS_DEFAULT);

/*
=================================

Tess.isShadowPass is valid when we are rendering the shadows to the forward rendering,
shadow FBO. When it is NOT true we are rendering the depth pass using GL_LUMINANCE,
technically this should be done via a GLSL program, but for now it's not.

For proper VSM were going to need to fix this....

FIXME!!! Ugly code.
=================================
*/
if( tess.isShadowPass == qfalse ) {
// Setup our light matrix.
SetupLightMatrix();

// FIXME: This is ugly, but we are using fixed function to render the depth map.
for( int i = 8; i >= 0; i-- ) {
GL_SelectTexture( i );
qglDisable( GL_TEXTURE_2D );
}

GL_SelectTexture( 0 );
qglEnable( GL_TEXTURE_2D );
GL_Bind( tr.whiteImage );

// Fixme!
qglUseProgramObjectARB( 0 );

qglEnable(GL_CULL_FACE);
qglCullFace(GL_BACK);

qglDisable( GL_BLEND );
qglEnable( GL_DEPTH_TEST );
qglDepthFunc(GL_LEQUAL);
qglDepthMask( true );
qglColorMask( true, true, true, true);
RB_RenderSurfaceVBO();

qglMatrixMode( GL_MODELVIEW );
qglPopMatrix();
}
else {
jmvCgProgram *shadowProgram = &amp;interactionManager.programs[ shadowProgHandle ];

qglUseProgramObjectARB(shadowProgram->linkedProgram);

qglUniform1iARB(qglGetUniformLocationARB(shadowPro gram->linkedProgram, "depthLookup"), 0); // depth-maps
qglUniform1iARB(qglGetUniformLocationARB(shadowPro gram->linkedProgram, "shadowTexture"), 1);

qglUniform4fARB(qglGetUniformLocationARB(shadowPro gram->linkedProgram, "eyeXyz"), backEnd.or.viewOrigin[ 0 ], backEnd.or.viewOrigin[ 1 ], backEnd.or.viewOrigin[ 2 ], 0.0 );
qglUniform4fARB( qglGetUniformLocationARB(shadowProgram->linkedProgram, "entityXyz"), backEnd.currentEntity->e.origin[ 0 ], backEnd.currentEntity->e.origin[ 1 ], backEnd.currentEntity->e.origin[ 2 ], 0.0 );

int shadowLightNum = 0;
for( int c = 0; c < tr.refdef.num_dlights; c++ ) {
if(tr.refdef.dlights[ c ].castsShadow) {
float lightRadiusParem = 1.0f / tr.refdef.dlights[ c ].radius;

tess.lightXyzRadius[ shadowLightNum ][ 0 ] = tr.refdef.dlights[ c ].transformed[ 0 ];
tess.lightXyzRadius[ shadowLightNum ][ 1 ] = tr.refdef.dlights[ c ].transformed[ 1 ];
tess.lightXyzRadius[ shadowLightNum ][ 2 ] = tr.refdef.dlights[ c ].transformed[ 2 ];
tess.lightXyzRadius[ shadowLightNum ][ 3 ] = lightRadiusParem;

shadowLightNum++;
}
}

qglUniform4fv(qglGetUniformLocationARB(shadowProgr am->linkedProgram, "lightXyz"), shadowLightNum, &amp;tess.lightXyzRadius[ 0 ][ 0 ] );

GL_SelectTexture( 0 );
qglEnable( GL_TEXTURE_CUBE_MAP );
qglBindTexture( GL_TEXTURE_CUBE_MAP, shadowFBO[ 1 ]->texnum );

// Texture 1 is the shadow depth texture.
GL_SelectTexture( 1 );
qglBindTexture( GL_TEXTURE_2D_ARRAY_EXT, shadowFBO[ 0 ]->texnum );

qglEnable(GL_CULL_FACE);
qglCullFace(GL_FRONT);

qglDisable( GL_BLEND );
qglEnable( GL_DEPTH_TEST );
qglDepthFunc(GL_LEQUAL);
qglDepthMask( true );
qglColorMask(true, true, true, true);

RB_RenderSurfaceVBO();

qglUseProgramObjectARB( 0 );
}
}

[ravage]

I keep looking at your image but I don't really see anything wrong. If there is it's kinda hard to see.

I looked and I didn't see anything wrong. It's possible that the multisampling has something to do with it(I'm not sure and take it as a last resort in which case try doing this without multisampling).

The only other thing I can think of is your depth comparison may be off.

in the depth shader you should have stored the length of the vector from the light to the position of the vertex (everything is world space)



Done with the front faces culled in the 1st pass
--------
1st pass
--------
varying vec3 lightvec;
lightvec = lightposition - position;

calculate in the frag

gl_FragColor.r = length(lightvec);

--------
2nd pass
--------

varying vec3 lightvec;
lightvec = lightposition - position;

then in the frag

float dist = length(lightvec);
float depth = texture(cubeshadowmap, -lightvec).r;

float fshadow = dist<depth ? 0.0 : 1.0;


This might be okay in cg, hlsl but not glsl since the values would be clamped [0,1] and I think cg, hlsl let you get away with it or have some kinda internal thing going on that I don't know about.

[LiveAndFight]

Do I still need to compute the depth values my self? Cause I'm using a depth texture thats attached to the FBO depth buffer.

http://img405.imageshack.us/img405/7121/lighterror2.th.png (http://img405.imageshack.us/i/lighterror2.png/)

Thats a better screenshot if you look at the floor, the shadows that are white behind the pillar are a grate thats bellow the floor on the other side of the pillar.

Here is a vid that will better illustrate the problem:

http://www.youtube.com/watch?v=_C8_weYROV8

[ravage]

Do I still need to compute the depth values my self

I'm assuming an omni-light because I see you trying to use a cubemap. You would use an fbo with an attached cubemap in one of the color attachments like I see your able to do. In the 1st pass I calculate the direction of the light into the r channel (GL_R32F cubemap texture) and render in 6 directions according the cubemap using 6 different viewMatrices/glulookats. that creates my depth needed for comparison. The depth renderbuffer just helps to make sure that the objects are occluded behind others correctly. You don't get the bilinear filtering with this method but you probably could if you use opengl 3.0 since cubemaps can accept depth values now though I'm not really sure about that (I did it before and didn't pay attention before I changed it so I don't remember).

The other lights don't require you to have use a red channel texture but just a a 2d depth texture.
---------------

here's a link that explains it better:
link (http://www.gamedev.net/reference/articles/article2457.asp)

Another thing that came to mind could be blending. It looks like the grill blends into almost everything but the other objects don't do that. Are you doing any additive blending?

In the end, I can only think that some blending is going on when it shouldn't, something weird is going on with the renderbuffer that is supposed to be meant for holding depth values, or the there's something weird with the comparison.

One last idea:

try changing

qglTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_COMPARE_FUNC, GL_GEQUAL);
to
qglTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_COMPARE_FUNC, GL_LEQUAL);