gluUnProject: How to use it in a proper way?

Hello,
I am using gluUnproject to get world coordinate from mouse coordinate. I do not know why it is producing weird result. I have used the following command:

gluUnProject( x, y, -1, modelview, projection, viewport, &nx, &ny, &nz);
gluUnProject( x, y, 1, modelview, projection, viewport, &fx, &fy, &fz);

I here -1, and 1 relate to near and far plane. In my program near plane is around 40.0 and far plane around 360.0. From the above equation I want to derive a ray originated at nx, ny, nz and directing towards (fx-nx), (fy-ny) and (fz-nz). I want to test when I place the mouse on a sphere, whether it intersects or not. But I am not getting correct result. Could any one give me suggestion about how to use it properly?

Change -1 to 0 in the first call, might do the trick.
The z parameter is [0;1] , corresponds to ((z/w)*0.5+0.5)

Thank you very much. I am still not getting the correct answer. As when I place the mouse on the top of the sphere, it should intersect it. I am checking with ray tracing whether the ray intersects the sphere or not. I have put down the result as follows:

radius of sphere: 30;
center of sphere: 1.241047 0.371793 -1.207528
x, y are: 519, 562.

(near) nx, ny and nz are: -2.629506 229.396622 0.00000
(far) fx, fy and fz are: -21.036052 -13.049021 -83.861326

So my ray is originated at(nx, ny,nz) and ray direction is (fx-nx, fy-ny, fz-nz). Then I use the function whether a ray originated from (nx, ny, nz) intersects my sphere. But the determinant is always negative.

I do not know whether I need any world to object space conversion in this case. I am too much worried with this problem. Any suggestion will be highly appreciated.

Thanks in advance.

gluUnproject will return the coordinates in the space corresponding to the supplied modelview-matrix parameter.

If you supply the “complete” modelview of the sphere including all transformations of the scenegraph you might use, the returned position will be in the object space of the sphere.

If you supply the “render” modelview (like the one generated after “gluLookAt()”), the returned positions will be in that space(which you might call world space).

I suggest you use your own matrix, even if only for checking what glu is doing, invert it (4x4 is not so hard to invert) and multiply with the vector you want to unproject, then check what is happening and where the error lies.

I agree with Ilian that supplying -1 for the near plane is a bit odd. Even 0 for the near plane is a bit odd, unless you are using an orthogonal projection.

0 is perfectly fine and shouldn’t have anything to do with an orthogonal projection. I’ve used 0 and 1 or .3 and .6 for the two z points to build a ray in world space without any problems. You’ll get different points back, but since you’re going to normalize the distance anyway, it doesn’t matter. Just don’t make them too close.

As Nighthawk mentioned, you have to make sure you’re passing the correct space to gluUnproject. Make sure you normalize the ray direction you’re computing. For the ray origin, I don’t use (nx,ny,nz). I use your camera origin, of course, I don’t think that should matter.

I am totally in this area. I need some expalnation about “complete modelview” and “render model view” and how they are used. As I am just using the following function:

glGetDoublev(GL_MODELVIEW_MATRIX, modelview);
glGetDoublev(GL_PROJECTION_MATRIX, projection);
glGetIntegerv(GL_VIEWPORT, viewport);

I know how to invert matrix but I do not know how to check by myself. Could anyone explain it. If I take some screen value and then do the transformation, how can I check where the error lies?

Thanks in advance.

I want to construct a ray with its origin and direction. So obviously when I click the mouse on my object the originated ray should intersect it, but I always find it does not intersect. Could any one help me with some hints how it works. Or with some example.

I do it like this:


float depth;
vec4 color; // unused
ilReadFrameBuffer(g_MAIN_FBO,1,&depth,&color.x,int(ILXMousePos.x),int(ILXMousePos.y),1,1); // uses glReadPixels(......, GL_DEPTH_COMPONENT, GL_FLOAT,&depth);

if(depth!=1.0f){
	vec4 screenPos = vec4((2.0f*ILXMousePos.x)/SCREEN_WID - 1.0f,(2.0f*ILXMousePos.y)/SCREEN_HEI - 1.0f,2*depth-1,1);
	
	Mat4 invMVP; 
	invert(invMVP,CAMERA_MVP_ORIGINAL); // this is the model_view_projection matrix,  converts from WORLD-space to clipspace

	vec4 projPos = invMVP * screenPos;
	projPos*=1.0/projPos.w; // divide by W!!
	
	vec3 worldPos = projPos.xyz; // this is the worldspace coordinate of the pixel you clicked in
	
	
	
	vec3 ray = normalize(worldPos-cameraPos); // the cameraPos is in worldspace, too
	
	// done. "ray" is what you need
}

Sorry I need some clarification about the code:

So I can use the following lines for “ilReadFrameBuffer”

glReadPixels(x,y,1,1,GL_DEPTH_COMPONENT,GL_FLOAT,&depth);

I think depth returns some value less than 1.0, that’s why this checking: // I am newbie in this area, could you explain

I also don’t understand about the screen pos, is not it just the mouse position, could you explain.

You have inverted the multiplication of modelview and projection matrix, is not it?

So in the end I got a ray originated at camera position with a direction towards world position.

Thanks in advance for your explanation.

Yes, the “depth” var will receive values [0;1]

ILXMousePos.x, ILXMousePos.y : integer coordinates of the mouse i.e [0…1279] [0…719] , where the bottom-left is [0][0] and top-right is [1279][719]

SCREEN_WID and SCREEN_HEI are i.e 1280 and 720 respectively

The CAMERA_MVP_ORIGINAL is (projection * view) effectively

Yes, a ray from the camera-position towards the world-pos of the pixel you clicked on.

Thank you very much for your clarification. I shall try it right now. Could you tell me why thr following code did not work with me? What is wrong with it? Also why the ray should always originate from the camera position?

/**************************************************/
ray_t getRayFromScreenCoords(int sx, int sy, int sw, int sh) {

ray_t ray;
GLdouble modelview[16], projection[16];
GLdouble x, y, z;
GLint viewport[4];

glGetDoublev(GL_MODELVIEW_MATRIX, modelview);
glGetDoublev(GL_PROJECTION_MATRIX, projection);
glGetIntegerv(GL_VIEWPORT, viewport);

gluUnProject(sx, sh - sy, 0.0, modelview, projection, viewport,
&x, &y, &z);

ray.origin = vvector( (float)x, (float)y, (float)z );

gluUnProject(sx, sh - sy, 1.0, modelview, projection, viewport,
&x, &y, &z);

ray.direction.x = x - ray.origin.x;
ray.direction.y = y - ray.origin.y;
ray.direction.z = z - ray.origin.z;

vnormalize(&ray.direction);

return ray;
}
/*********************************/
Again, thanks in advance for your cooperation.

Hello Llian,
I am getting some weird result.
My x, y positions are: 623, 649

Z value from read pixels: 0.993965;

world coordinates are: 0.00000 0.007431 0.00000.
Always the x and z coordinates are showing 0 values after converting to world coordinates.

Also sometime I am getting z value which is equal to 1.

Please help me with some explanation and guidance.

Z values from ReadPixels in 0…1 is to be expected as they are window-space Zs:

The Z range used for window space is given by glDepthRange (default 0…1).

Dunno what you’re saying about world-space Zs, because gluUnProject has nothing at all to do with world space. It takes window-coordinates in, and accepts viewport transform (inverse of this gets it from window-to-NDC space, projection transform (inverse gets it from NDC/clip-to-eye space), and modelview transform (inverse gets it from eye-to-object space). No world space involved here…

World space is actually between the MODELING and VIEWING transforms, but gluUnProject (like OpenGL in general) accepts the full composite (product) of these transforms, MODELVIEW.

Thnk you all very much for your cooperation. I have fixed the problem. gluUnProject with z value 0 and 1 is working fine for me for constructing a ray with a direction and origin.

But as I rotate and translate the object, I had to give that transformation to the point I got from gluUnProject as gluUnProject gives us position in world coordinate.

Again thank you all for your cooperation.