shadows

Having trouble with shadows here. Pulling hair out. Getting that blind feeling. My cg fragment program always returns a 0 -- making black screen of death. cg tutorial book is clear on vert and fragment programs and theory but not how to set up the shadow map in opengl. Been looking at redbook example. Anyhow, time for sanity check. See any obvious problems?

//step 1 create texture map
glActiveTexture(GL_TEXTURE4);
glGenTextures(1,&mTexID);
glBindTexture(GL_TEXTURE_2D, mTexID);
glEnable(GL_TEXTURE_2D);
glTexImage2D( GL_TEXTURE_2D, 0, GL_DEPTH_COMPONENT, mmapwidth, mmapheight, 0, GL_DEPTH_COMPONENT, GL_UNSIGNED_BYTE, NULL);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);
glDisable(GL_TEXTURE_2D);

//do other stuff texture state should be saved..

//step 2 move camera to light and draw.

//step 3 calculate texture matrix

static float bias_matrix[16] = {
0.5,0,0,0.5,
0,0.5,0,0.5,
0,0,0.5,0.5,
0,0,0,0.5
};

float proj_matrix[16],lookat_matrix[16];

glMatrixMode(GL_MODELVIEW);
glPushMatrix();
glLoadIdentity();
gluPerspective(cfg_cam.fov,cfg_cam.aspect_ratio, cfg_cam.near_plane, cfg_cam.far_plane);
glGetFloatv( GL_MODELVIEW_MATRIX, proj_matrix );

glLoadIdentity();
gluLookAt(pos.x, pos.y, pos.z, look_at.x, look_at.y, look_at.z, 0.0, 1.0, 0.0);
glGetFloatv( GL_MODELVIEW_MATRIX, lookat_matrix );

//throw it away...still in model view
glPopMatrix();

//calculate texture matrix
float temp[16];
matrix_mult(temp, proj_matrix, lookat_matrix);
matrix_mult(tex_matrix, bias_matrix, temp);


//step 4 save tex_matrix to CG vert program. save model matrix to cg vert program. enable texture unit4 use tex2dproj in frag program. multiply by model matrix then texmatrix in vert program.

//step 5 Draw into depth buffer.
//capture image
glActiveTexture(GL_TEXTURE4);
glEnable(GL_TEXTURE_2D);
glBindTexture(GL_TEXTURE_2D, mTexID);
glCopyTexImage2D(GL_TEXTURE_2D, 0, GL_DEPTH_COMPONENT, 0, 0, mmapwidth, mmapheight,0);

glTexGeni(GL_S, GL_TEXTURE_GEN_MODE, GL_EYE_LINEAR);
glTexGenfv(GL_S, GL_EYE_PLANE, &tex_matrix[0]);
glEnable(GL_TEXTURE_GEN_S);

glTexGeni(GL_T, GL_TEXTURE_GEN_MODE, GL_EYE_LINEAR);
glTexGenfv(GL_T, GL_EYE_PLANE, &tex_matrix[4]);
glEnable(GL_TEXTURE_GEN_T);

glTexGeni(GL_R, GL_TEXTURE_GEN_MODE, GL_EYE_LINEAR);
glTexGenfv(GL_R, GL_EYE_PLANE, &tex_matrix[8]);
glEnable(GL_TEXTURE_GEN_R);

glTexGeni(GL_Q, GL_TEXTURE_GEN_MODE, GL_EYE_LINEAR);
glTexGenfv(GL_Q, GL_EYE_PLANE, &tex_matrix[12]);
glEnable(GL_TEXTURE_GEN_Q);


glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_COMPARE_FUNC, GL_LEQUAL);
glTexParameteri(GL_TEXTURE_2D, GL_DEPTH_TEXTURE_MODE, GL_LUMINANCE);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_COMPARE_MODE, GL_COMPARE_R_TO_TEXTURE);
glDisable(GL_TEXTURE_2D);


//step 6 later on when I want to use the shadow during drawing...

glActiveTexture(GL_TEXTURE4);
glEnable(GL_TEXTURE_2D);
glBindTexture (GL_TEXTURE_2D, mTexID );
glEnable(GL_TEXTURE_GEN_S);
glEnable(GL_TEXTURE_GEN_T);
glEnable(GL_TEXTURE_GEN_R);
glEnable(GL_TEXTURE_GEN_Q);

//draw stuff

glDisable(GL_TEXTURE_GEN_S);
glDisable(GL_TEXTURE_GEN_T);
glDisable(GL_TEXTURE_GEN_R);
glDisable(GL_TEXTURE_GEN_Q);
glDisable(GL_TEXTURE_2D);

Couple more notes.

In the apple opengl profiler program, I can see the depth grayscale image has been copied correctly to a texture -- under resources when I hit a break point. Its a GL_DEPTH_COMPONENT..

Aside from the shadow stuff, If I lauch the program and emulate another card in Opengl profiler the cg will blow up the program. If I disable CG stuff then it emulates other cards fine.

[arekkusu]

What hardware are you using?

And, start from something that is known to work:
http://www.paulsprojects.net/tutorials/smt/smt.html

I've got a Radeon 9800 XT. 256 megs of ram. Should handle the situation. I have bump mapping working with a tbn matrix etc.

Ya, I've been looking at the Pauls Project stuff. Its been helpful. Also, the Red Book has been helpful -- though they calculate the matrix stuff a little differently.

I suppose the difference is, I'm attempting to do the shadowing through a projective texture with cg. ( open gl 2 was not on my mac when I started the project ) So, it would be two passes to draw the scene rather than three. The key goals are to get the depth map copied to a texture, get the texture matrix into the vert program along with the model matrix and finally use tex2Dproj in the fragement program. tex2Dproj divides by r...

I think one problem in my code ( I initially pasted ) is I was using glEnable and glDisable TEXTURE_2D everywhere when all I needed to do was bind -- when enabled some garbage got drawn to the screen. So, took that out.

I think I'm getting the TexGen. If I understand it right, the TexGen is going to operate on the current glActiveTexture. So, I activate and bind to the texture with the appropriate texture_matrix then call the tex gen stuff. But I don't enable TEXTURE_2D -- the fragment program will handle it. (?)

I also found that cg wants cgGLSetTextureParameter on the texture name. Its needs to associate the Texture unit. Though, I'm a little confused if tex2Dproj wants cgGLSetTextureParameter or cgGLSetupSampler. I suspect it wants cgGLSetTextureParameter. Though, the CG reference manual says tex2Dproj expects the shadow map to be set up correctly or it basically just passes back a zero.

Still getting black screen of doom because tex2Dproj is returning 0. It returns c,c,c,w. So, c must be 0. Argh...

I notice Red Book does a transpose. I suppose this is because the matricies were multiplied backwards?

[execom_rt]

I shouldn't use this 'old' approach using the ARB_shadow (unless you are targetting 7-year old hardware like the Geforce 3 or Geforce 4).

Instead, you should consider the 'GLSL' approach, with one pass that render the depth into a floating point buffer (using EXT_fbo or PBuffer), then an other shader to do the depth comparision.

The code is much simpler, and easier to debug (you can do the ARB_shadow version later of course )

This will ensure that the result will works both on nVidia and ATI and will run as fast (also you will be able to perform some effect like filtering etc ..)

Finally, you should be aware that Cg is not optimized for ATI video cards : Basically a Cg program running on a Radeon X1800 for example, is using the same assembler code than if it was running on Radeon 9500. This is true on MacOSX and Windows.

I'd like to move to glsl but there is no open gl 2 on my computer. I'm running the latest mac os and using the latest dev tools. glVersion says:

opengl version = '1.5 ATI-1.4.18'
vendor = 'ATI Technologies Inc.'
renderer = 'ATI Radeon 9800 OpenGL Engine'

ATI says goto apple to get the driver.

If it runs on windows, mac and its not an extension then I like it. Unless, the circumstances are warranted I might consider a extension -- minimize incompatibilities.

I wonder if intel macs will have a newer opengl version.

[arekkusu]

Use GLSL via these extensions:

GL_ARB_shader_objects
GL_ARB_shading_language_100
GL_ARB_vertex_shader
GL_ARB_fragment_shader

These are exported on ATI Radeon 9800 since Mac OS X 10.4.3.

[OneSadCookie]

OpenGL version is *completely* irrelevant. There is no difference between an extension and a part of a base specification.

Hmm, there is a GLSL Showpiece under the /Developer/Examples/OpenGL/Cocoa/GLSLShowpiece . Looks like its come a *long* way since the last time I looked at it. Seems waay faster now. The GLSL migration will be a futer project. Fortunately, in my little program, all this shader stuff is encapsulated in a wrapper class. One problem with cg is I have to use the arb arbvp1/arbfp1 profiles -- my registers and calls are limited. Sort of a pain if I want many lights because I have to pass texture coordinates to my fragment program. There is an GLSLEditorSample program too. Looks like there are some busy bees at Apple.

Makes sense nvidia is focusing on the cgfx stuff. Its the only draw they'll have.

Meanwhile, back to cg. Can't switch pacemaker vendors in the middle of an operation. Just want to get shadows to work then I can convert over.

I notice I have a choice between eyespace and object space. Though, object space requires I grab the 'model' matrix and recompute the texture matrix everytime it changes. So, eyespace seems like the better alternative but I have to be aware of the camera matrix inversion.

Another question. I notice in the Pauls project example his bias matrix is:

0.5f, 0.0f, 0.0f, 0.0f,
0.0f, 0.5f, 0.0f, 0.0f,
0.0f, 0.0f, 0.5f, 0.0f,
0.5f, 0.5f, 0.5f, 1.0f

Versus other examples the bias matrix is:


0.5f, 0.0f, 0.0f, 0.5f,
0.0f, 0.5f, 0.0f, 0.5f,
0.0f, 0.0f, 0.5f, 0.5f,
0.0f, 0.0f, 0.0f, 1.0f

A litte confused on the difference...

Also, I notice there is an Alpha check. They init RGB and not RGBA. So, I figure this is going to mess up transparent things. Thus, I really have to do the check in the fragment program.(?)

[execom_rt]

You still implement the Cg path, but not using the ARB_shadow (I'm doing it for my DX9 render path, which is using HLSL (which is using Cg syntax)

Correct shadow mapping requires a step by step verification. Doing the full implementation at once, without understanding what you are doing is quite a 'risky business' : There is 3 important steps:

- Check your depth map generation, render it into a quad in a corner of the screen for example. Tune the FOV of from the light and clipping plane accordingly.

- Check your projection mapping code : render the depth map as normal texture on the scene for example. Use the 'border' texture wrap mode (not 'clamp' texture wrap mode).

- Enable the shadow map depth test. Remember, that you can use a fragment program for the depth comparision (even with Cg, if you don't want to use GLSL).

Hope it will help.