trouble using gluUnproject for 3D Mouse Coords

Hello all

I am trying to create a simple ray trace function whereby I can draw a 3D line that goes from the character to the point in space that the mouse is over. I am using the following code taken off this Nehe’s tutorial :

Vector3 World::projectedMouse(float mx, float my){

    GLdouble model_view[16];
    GLint viewport[4];
    GLdouble projection[16];
        
    GLfloat winX, winY, winZ;
    GLdouble dx, dy, dz;

    glGetDoublev(GL_MODELVIEW_MATRIX, model_view);
    glGetDoublev(GL_PROJECTION_MATRIX, projection);
    glGetIntegerv(GL_VIEWPORT, viewport);
    
    winX = (float)mx;
    winY = (float)viewport[3] - (float)my;



    glReadPixels ((int)mx, (int)winY, 1, 1, GL_DEPTH_COMPONENT, GL_FLOAT, &winZ); 
    gluUnProject(winX, winY, winZ, model_view, projection, viewport, &dx, &dy, &dz);

    Vector3 pr = Vector3(dy, dy, dz);


    
    glColor3f(1,0,0);
    glBegin (GL_LINE_LOOP); 
    glVertex3f(player->getPosition().x, player->getPosition().y + 100, player->getPosition().z); // 0
    glVertex3f(pr.x,pr.y,pr.z); // 1
    glVertex3f(player->getPosition().x, player->getPosition().y, player->getPosition().z); // 0
    glEnd();

    return pr;
}

And I draw a line from the player position to the unprojected point but the results I get are really not accurate, the point in space rarely corresponds to where the mouse lies, most of the time it is some random spot in space. Here’s a screen with the grey line that goes from the character to the sky when the mouse was simply over the character’s head:

Am I doing something wrong in that function? Am I not doing something? Or is there a specific location in the Opengl code where that should be placed?

Thank you in advance!

Right seems like it’s fixed now! I get a properly looking line when using:

gluUnProject(winX, winY, 0, model_view, projection, viewport, &bx, &by, &bz);
Vector3 pr2 = Vector3(bx, by, bz);

instead of plain:

gluUnProject(winX, winY, winZ, model_view, projection, viewport, &dx, &dy, &dz);
Vector3 pr = Vector3(dx, dy, dz);

with unspecified Z… Plus i realised that there was a dy where there should have been a dx… Maybe it was simply that…

am I right in thinking that using 0 as third parameter casts a point on the far plane whereas a 1 does on the near one? Or was it the other way round?

Anyhow, would anyone know where I can find some tutorial material on how to calculate intersection between a ray and a 3d object/its bounding box?

Thanks all

You need a good book on ray tracing. Have several, but I especially like Peter Shirley’s Realistic Ray Tracing. I actually like the first edition a little better for its beginning material on how to do exactly what you’re asking:

Excellent at getting you from a beginning stage of math/graphics knowledge up to writing your own ray tracer.

Once you get your “ray tracing legs”, if you want to go further, Physically-Based Rendering is a great next step (see http://www.pbrt.org for the book web site, where you can download the PBRT ray tracer for free).

If you’re looking for totally-free options, google “ray-sphere intersection” and “ray-triangle intersection”. That’ll get you a bunch of hits related to your request. Ray-sphere is cheap and a good first test for small scenes. Once you get scenes of larger size though, you’re going to want some kind of spatial acceleration data structure to make searching space for hits fast. Often used in ray-tracing: k-D trees, a spatial subdivision technique (for static objects) and bounding volume hierarchies, aka BVHs (for dynamic objects).