help with Shadow mapping with cube textures

[MrManlyMan]

I just finished getting the splot light shadowing mapping to work and I would like to extend it to a point light. I'm targeting OpenGL 3.3 .
I need help understanding how to set up the camera matrices for the cube faces and the how to sample the cube map

To setup the shadow map I'm doing this:

GLuint			shadowMap;
GLuint			fbo[6];
 
glBindTexture	( GL_TEXTURE_CUBE_MAP, &shadowMap );
 
glTexParameteri( GL_TEXTURE_CUBE_MAP, GL_TEXTURE_MIN_FILTER, GL_NEAREST );
glTexParameteri( GL_TEXTURE_CUBE_MAP, GL_TEXTURE_MAG_FILTER, GL_NEAREST );
 
glTexParameterf( GL_TEXTURE_CUBE_MAP, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE );
glTexParameterf( GL_TEXTURE_CUBE_MAP, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE );
glTexParameterf( GL_TEXTURE_CUBE_MAP, GL_TEXTURE_WRAP_R, GL_CLAMP_TO_EDGE );
 
glTexParameterf( GL_TEXTURE_CUBE_MAP, GL_TEXTURE_COMPARE_MODE, GL_COMPARE_R_TO_TEXTURE );
glTexParameterf( GL_TEXTURE_CUBE_MAP, GL_TEXTURE_COMPARE_FUNC, GL_LEQUAL);
 
 
glGenFramebuffers( 6, fbo );
 
 
for ( i=0; i<6; i++ )
{
	glTexImage2D( GL_TEXTURE_CUBE_MAP_POSITIVE_X+i, 0, GL_DEPTH_COMPONENT32F, 1024, 1024, 0, GL_DEPTH_COMPONENT, GL_FLOAT, 0 );
 
	glBindFramebuffer( GL_FRAMEBUFFER, fbo[i] );
 
	// disable color buffer, only need depth.  Is this the right way to do this?
	glDrawBuffer( GL_NONE );
	glReadBuffer( GL_NONE );
 
	// attach the texture to FBO depth attachment point
	glFramebufferTexture2D( GL_FRAMEBUFFER, GL_DEPTH_ATTACHMENT, GL_TEXTURE_CUBE_MAP_POSITIVE_X+i, shadowMap, 0 );
}

To update the shadow map, I'm trying something like ( at some point I intend to do this in a single pass, but one step at a time ):

int f;
 
MatrixPerspective( &lightProjection, &identity, 90.0f, 1.0f, 2.0f, 200.0f );   // fov, aspect, near, far
 
glEnable	( GL_DEPTH_TEST );		// yes, depth test
glDisable	( GL_BLEND );			// no blending
glDisable	( GL_CULL_FACE );		// no culling
glDepthMask	( GL_TRUE );			// yes, write to depth buffer
glDisable	( GL_TEXTURE_2D );		// no texturing
glDisable	( GL_ALPHA_TEST );		// no alpha testing
 
glPolygonOffset( 4.0f, 32.0f );
glEnable(GL_POLYGON_OFFSET_FILL);
 
for ( f=0; f<6; f++ )
{
	// create a matrix in the space of the light that faces each of the six directions  
	MatrixModelView( &amp;lightModelview, f, &amp;light->position, &amp;light->orientation );
 
 
	glBindFramebuffer( GL_FRAMEBUFFER, fbo[f] );	// switch rending to shadow map
	glViewport	( 0,0, 1024, 1024 );
	glClear		( GL_DEPTH_BUFFER_BIT );
	glColorMask	( GL_FALSE, GL_FALSE, GL_FALSE, GL_FALSE );
 
	RenderScene( &amp;lightProjection, &amp;lightModelview, scene, NULL, NULL );
}
 
glDisable(GL_POLYGON_OFFSET_FILL);

So my questions are:

How do I sample the cube map? So far I have this fragment shader:

uniform sampler2D		diffuseMap; 
uniform sampler2D		normalMap;
uniform sampler2D		specularMap;
uniform samplerCubeShadow	shadowMap; 
 
varying vec2    TexCoord;
varying vec4    ShadowCoord;    // interpolated position in light space
 
varying vec3	tbnDirToLight;	// direction from fragment to light in tangent space
varying vec3	tbnDirToEye;
 
 
void main() 
{ 
	float shadow = texture( shadowMap, -ShadowCoord ); 
 
	vec3 n = normalize( texture2D( normalMap, TexCoord.st ).rgb * 2.0 - 1.0 );
	vec3 L = normalize( tbnDirToLight );
	vec3 E = normalize( tbnDirToEye );
 
	float diffuse	= max(dot(n,L), 0.0) * 2.0;
	float specular	= max( pow( dot( reflect( -L, n ), E ), 32.0 ) * 8.0, 0.0 ); 
 
 
	vec4 d = texture2D( diffuseMap, TexCoord );
	vec4 s = texture2D( specularMap, TexCoord );
 
	gl_FragColor =	(shadow * (d*diffuse + s*specular) * gl_LightSource[0].diffuse * att);
}

Also, I'd like to using nothing but forward compatible stuff. I've having a hard time distinguishing but I'm pretty sure there's old stuff in this code.

[Geometrian]

From my old library (GL 2 compatible):

float cubemap_depthtest(samplerCube cubetex, vec3 center, float near, float far) {
    vec3 lightdir = vec4(transform_matrix*vertex).xyz - center;
 
    float distance = max(max(abs(lightdir.x),abs(lightdir.y)),abs(lightdir.z));
    distance = ((far+near)/(far-near)) + (1.0/distance)*((-2.0*far*near)/(far-near));
    distance = (distance+1.0)/2.0;
 
    float shaddepth = textureCube(cubetex,normalize(lightdir)).r;
 
    if (distance>shaddepth) { return 0.0; }
    else                    { return 1.0; }
}

There may be a more efficient method of doing this (and certainly this function can be improved) but this should at least get you started. "transform_matrix" is the model matrix, "center" is the center of projection for the cubemap, and "near" and "far" are the clipping planes used when making the cubemap.

[MrManlyMan]

Do you think I shouldn't bother with samplerCubeShadow? I can't seem to find a definitive answer on how to actually use that sampler.

[ravage]

You can use it but then to make it work you have to custom define the depth by using gl_FragDepth = <whatever linear equation>. This has the problem of not having hw depth or not being particular fast though. I've actually done this and got it to work. I didn't pay attention but I'm curious to try it again to see if you possibly get the free bi-linear filtering(nvidia) though I doubt it. Since I had figured out how it worked I switched to use a 1 channel(Red) render target and use samplerCube instead.

[malexander]

A while back I noticed that using a samplerCube on a depth texture on GL2 Nvidia hardware worked, but only in software (ie very, very slowly on an Nvidia 7900). I had to split the cube map into 6 2D textures and use sampler2DShadow in order to get acceptable performance. On GL3+ hardware, it's HW-accelerated.