gluUnProject

Hello, is gluUnProject is the only method to find the x, y, z coordinate of the mouse or there is other method. Thank you.

gluUnproject() is just:

GLint GLAPIENTRY
gluUnProject(GLdouble winx, GLdouble winy, GLdouble winz,
		const GLdouble modelMatrix[16], 
		const GLdouble projMatrix[16],
                const GLint viewport[4],
	        GLdouble *objx, GLdouble *objy, GLdouble *objz)
{
    double finalMatrix[16];
    double in[4];
    double out[4];

    __gluMultMatricesd(modelMatrix, projMatrix, finalMatrix);
    if (!__gluInvertMatrixd(finalMatrix, finalMatrix)) return(GL_FALSE);

    in[0]=winx;
    in[1]=winy;
    in[2]=winz;
    in[3]=1.0;

    /* Map x and y from window coordinates */
    in[0] = (in[0] - viewport[0]) / viewport[2];
    in[1] = (in[1] - viewport[1]) / viewport[3];

    /* Map to range -1 to 1 */
    in[0] = in[0] * 2 - 1;
    in[1] = in[1] * 2 - 1;
    in[2] = in[2] * 2 - 1;

    __gluMultMatrixVecd(finalMatrix, in, out);
    if (out[3] == 0.0) return(GL_FALSE);
    out[0] /= out[3];
    out[1] /= out[3];
    out[2] /= out[3];
    *objx = out[0];
    *objy = out[1];
    *objz = out[2];
    return(GL_TRUE);
}

You could do exactly the same thing using e.g. GLM or whichever matrix functions are convenient.

Depends on what information you have at your disposal. Assuming you can mouse pick your window for int mouse x and y and you can access float depth (distance from eye), you can use the following as a template:

int sx= mouse_x; //position in your rendering window not screen mouse x
int sy= mouse_y; //position in your rendering window not screen mouse y

float coordX = ( float(sx)/float(height) * 2.0 - float(width) / float(height) ); //height and width
float coordY = ( float(height - sy) / float(height) * 2.0 - 1.0 ); // are your window height and width

Vector_3D dir = normalize (target-eye); //center= target= center… eye is viewing position
Vector_3D up = normalize (up - dot(up, dir)*dir); // normalize these 3d vectors…
Vector_3D right = normalize (cross(dir, up));

dir = (coordXright + coordYup )*fov+dir; // fov… is FOV

Vector_3D mouse_XYZ = eye + dir * DFE; // DFE = distance from eye… or depth… you might have saved it to gl_fragDepth?

Thank you! But I don’t understand the following:

dir = (coordXright + coordYup )*fov+dir;

Why do I need to multiply by FOV? Also it is considering right and up only?

I’m also confused with the following:

Vector_3D mouse_XYZ = eye + dir * DFE;

How can I calculate DFE in my application?
It’d be great if you clarify a bit. Thank you!

[QUOTE=Lee_Jennifer_82;1281180]
Why do I need to multiply by FOV?[/quote]
In the case I mentioned, that’s part of how ray direction is set by window coordinations. Use what I said as a template to figure out what you have to do- if it doesn’t match up to your code (you don’t have the required information available), you either can’t use it or you have to modify it to fit your code.

Also it is considering right and up only?

Not only: right and up are 2 of the 3 direction vectors the template uses to determine ray direction- right and up from the Eye to –> Target (Center) direction (eye to target is the other direction vector). In the case I mention:

NDC.x = normalized device coordinates x component
NDC.y = normalized device coordinates y component

ray_direction = right * NDC.x * FOV + up * NDC.y *FOV + [Eye to Target (Center) direction]

give you your ray-direction.

How can I calculate DFE in my application?

You need that- if you can’t extract the GL depth component with glReadPixel or have the depth component stored somewhere you’re going to have to figure out another method of mouse picking your world coordinates.