gluProject

[HellKnight]

Ok, so I'm trying to implement a basic depth bounds test to offload my pixel shaders a bit, however I'm stuck when trying to compute the needed znear and zfar arguments.

As far as I know, the default perspective projection matrix takes the z coordinate in eye space and transforms it into clipspace, according to the specified znear and zfar clip planes. After the perspective divide (w), we end up with a device space z coordinate in the interval [-1; 1], which is then scaled and biased to fit in the [0; 1] interval (the actual z-buffer value is then obtained by multipliyng by 2^precision_bits - 1).

That's what gluProject should do - I checked the Mesa implementation; so, if I haven't totally misunderstood the above, if a point in eye space is between the znear and zfar planes, the obtained device z coordinate should be in the specified interval [0; 1]; anything in front of znear should yield z < 0, and obviously, anything behind zfar > 1.

What amazes me, however, is that obviously that's not true... I have tested it a dozen times; so what I'm doing is, I calculate two bounding points of a given light (a "near" and a "far" one) by subtracting/adding a scaled "forward" vector from/to the light position. The forward vector is obtained either from the modelview matrix (elements 2, 6 and 10) or by my own routines, with the same results. Furthermore, it's scaled by the light radius.

So far so good, but I get totally strange results. The device space z coordinate of a point *in front* of the znear plane is greater than 1, while points between znear/zfar are little below 1. I am *sure* there isn't a bug in my code (i.e. the world space "near" and "far" points are correct).

Any help is appreciated, thanks!

[HellKnight]

Come on guys, where are the OGL gurus?

[Relic]

Hard to tell without seeing the code.
You got the projection and value range expected by the viewport scaling right in theory.
You're sure your direction vector is in the correct handedness?
Model and eye-space are right handed. Projection flips it to left handed.

[HellKnight]

Originally posted by Relic:
You're sure your direction vector is in the correct handedness?
Model and eye-space are right handed. Projection flips it to left handed.

Actually it's not a direction vector, it's just a point in world space - it is in the same coordinate system as the rest of the wolrd, so I assume the handedness is right.
I'm aware of the right-to-left hand transformation, however, how does this affect the ndc z coordinate?

Anywyays, I'm at work right now and I don't have my real code at hand, but the pseudecode below should resemble it closely enough:

//lightPos - position of light
//lightRadius - radius
//modelview - modelview matrix
//projection - projection matrix
 
vec3f znear, zfar, zbounds, forward;
float near, far;
 
//one way to get the forward vector; using my own math routines yields the same results
forward[0] = modelview[2];
forward[1] = modelview[6];
forward[2] = modelview[10];
 
forward *= lightRadius;
 
//forward is inverted, so we add it to the light position to find the "near" z bounds point
znear = lightPos + forward;
zfar = lightPos - forward;
 
gluProject(znear[0], znear[1], znear[2], modelview, ..., &near);
gluProject(zfar[0], zfar[1], zfar[2], modelview, ..., &far);
 
//now, near and far should contain the ndc z coordinates
 
glDepthBoundsEXT(near, far);

[Relic]

>>I'm aware of the right-to-left hand transformation, however, how does this affect the ndc z coordinate?<<

I was just shooting in the dark.

Put in the code you're actually running. Above mixes float and double requirements and won't work.

[HellKnight]

Originally posted by Relic:
Put in the code you're actually running. Above mixes float and double requirements and won't work.

Sorry, as I said it was just pseudocode, the actual implementation uses doubles for near and far, as well as for the modelview and projection matrices.

[Relic]

Yes, I read that, that's why I was asking for the real code which compiles.

What do you use as local forward vector? The object's z-axis (0, 0, 1)?
Matrices are left-multiplied in OpenGL which makes vectors columns, and matrices are filled column-major.
That would make matrix elements (8, 9, 10) the forward vector.

[HellKnight]

Originally posted by Relic:
Yes, I read that, that's why I was asking for the real code which compiles.

I'll upload it today, but it's actually almost the same thing.

Originally posted by Relic:
What do you use as local forward vector? The object's z-axis (0, 0, 1)?
Matrices are left-multiplied in OpenGL which makes vectors columns, and matrices are filled column-major.
That would make matrix elements (8, 9, 10) the forward vector.

Hehe, I was shocked for a moment But usually there's a difference between the memory representation and matrices "on paper". I was curious myself, so here's a part of the actual Mesa implementation of gluLookAt:

    forward[0] = centerx - eyex;
    forward[1] = centery - eyey;
    forward[2] = centerz - eyez;
 
    //...
 
    m[0][2] = -forward[0];
    m[1][2] = -forward[1];
    m[2][2] = -forward[2];

As you can clearly see, the inverted forward vector is stored in the elements 2, 6 and 10.

[Relic]

I interpreted your modelview[i] coming from the usual GLdouble[16] you use in glGetDoublev for matrices. Then my indexing would be the right one. Another reason to post real code.

I looked in to the gluLookAt implementation and what side, up and forward do, is to create a basis transformation which is post-multiplied to the current matrix.

Mesa is throwing in the float[4][4] matrix via glMultMatrixf(&m[0][0]) which effectively transposes the matrix because OpenGL interprets that as float[16] and loads column-major.

A misunderstanding on my side. I was thinking about model space forward vector z-axis transformed into eye-space.
You want the eye-space z-direction in modelspace which is the inverse transformations, which with no scaling is the transpose. There you go, both right.

[HellKnight]

Yeah, I'm a little confused now

All OGL matrices are column-major (vectors are columns), which means that the translation occupies the 12th, 13th and 14th element (counting from 0), which is what glGetDoublev returns.

gluLookAt computes a column-major matrix as well and, as you said, post-multiplies it with the current one - I don't see a reason why it should be effectively transposed (it is and is interpreted as column-major), right?

Anyways, the forward vector is right, so the question remains why gluProject doesn't return the expected ndc z values...