gluUnproject

Hi,
I've got a problem with the gluUnproject function.
When I use translation, it works fine but when I add rotation, the coordinates back to me are wrong.
I don't understand, coz when you use gluUnproject you recover ModelView and Projection matrices ?!
Thanx

please post some code. can't do much without it, except explain the universe of possibilities. but i've got plans for the rest of the week :-).

[beebop]

void picking(int mouseX, int mouseY)
{
GLint viewport[4];
GLdouble mvmatrix[16], projmatrix[16];
GLint realy; /* OpenGL y coordinate position */
GLint winZ;
GLdouble wx, wy, wz; /* returned world x, y, z coords */

glGetIntegerv (GL_VIEWPORT, viewport);
glGetDoublev (GL_MODELVIEW_MATRIX, mvmatrix);
glGetDoublev (GL_PROJECTION_MATRIX, projmatrix);

realy = viewport[3] - (GLint) mouseY - 1;
printf ("Coordinates at cursor are (%4d, %4d)\n", mouseX, realy);

// First Try
gluUnProject ((GLdouble) mouseX, (GLdouble) realy, 0,
mvmatrix, projmatrix, viewport, &wx, &wy, &wz);
// Second
gluUnProject ((GLdouble) mouseX, (GLdouble) realy, 1,
mvmatrix, projmatrix, viewport, &wx, &wy, &wz);
// Third
glReadPixels( mouseX, mouseY, 1, 1, GL_DEPTH_COMPONENT, GL_FLOAT, &winZ );
gluUnProject ((GLdouble) mouseX, (GLdouble) realy, winZ,
mvmatrix, projmatrix, viewport, &wx, &wy, &wz);
printf ("World coords at z=0.0 are (%f, %f, %f)\n",
wx, wy, wz);
}

void glDraw()
{
//
// ############# PROJECTION #############
//
glMatrixMode( GL_PROJECTION );
glLoadIdentity();
gluPerspective( 45.0, (GLfloat)_viewW/(GLfloat)_viewH,
_camera.getDistance()*0.1f, _camera.getDistance()*20.0f );

glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);

//
// ############# MODELVIEW #############
//
glMatrixMode( GL_MODELVIEW );
glLoadIdentity();

glRotatef(90, -1, 0, 0);
glTranslatef(0, _distance, 0);

glRotatef(_r[2], 0, -1, 0);
glRotatef(_r[1], -1, 0, 0);
glRotatef(_r[0], 0, 0, 1);

glTranslatef(- _t[0], - _t[1], - _t[2]);

// DRAW
}

When I've got rotation (float _r[3]), it's done,
gluUnproject isn't working any more.

[plasmonster]

hmmm, looky here

GLint winZ; // should be float?

glReadPixels( mouseX, mouseY, 1, 1, GL_DEPTH_COMPONENT, GL_FLOAT, &winZ );

gluUnProject ((GLdouble) mouseX, (GLdouble) realy, winZ,
mvmatrix, projmatrix, viewport, &wx, &wy, &wz);

[plasmonster]

you see, the depth component will give you a value betweem 0 and 1, so you need to supply a float for winZ in the glReadPixels() and glUnProject() calls.

it is right but my true problem is that the x and y coordinates are totally wrong.

[plasmonster]

beebop, which x and y coordinates, and what's wrong with them?

remember that you want only the view matrix at the top of the modelview stack when you grab the matrices for UnProject(). this can cause problems if you're not careful.

[plasmonster]

hmmm, just noticed a problem with this

glReadPixels( mouseX, mouseY, 1, 1, GL_DEPTH_COMPONENT, GL_FLOAT, &winZ );

mouseY should be your realy instead. you need to be consistent in your treatment of the screen y. i missed this earlier, due to my chronic stupor.