Retrieve frstum corners with gluUnProject?

OpenGL OpenGL: Basic Coding
1

Which coordinates do I have to pass to gluUnProject to retrieve the frustum corners as world space coordinates? Whatever I tried doesn’t seem to work:


void CFrustum::Compute (void)
{
   static CFloatVector corners [8] = {
#if 0
      {{0.0, 0.0, 0.0}}, {{0.0, 1.0, 0.0}}, {{1.0, 1.0, 0.0}}, {{1.0, 0.0, 0.0}},
      {{0.0, 0.0, 1.0}}, {{0.0, 1.0, 1.0}}, {{1.0, 1.0, 1.0}}, {{1.0, 0.0, 1.0}}
#else
      {{-1.0f, -1.0f, 0.0}}, {{-1.0f, 1.0f, 0.0}}, {{1.0f, 1.0f, 0.0}}, {{1.0f, -1.0f, 0.0}},
      {{-1.0f, -1.0f, 1.0}}, {{-1.0f, 1.0f, 1.0}}, {{1.0f, 1.0f, 1.0}}, {{1.0f, -1.0f, 1.0}},
#endif
      };

transformation.SystemMatrix (-1).Get (GL_MODELVIEW_MATRIX, false); 
transformation.SystemMatrix (-2).Get (GL_PROJECTION_MATRIX, false); 

GLint viewport [4];
glGetIntegerv (GL_VIEWPORT, viewport);

for (int i = 0; i < 8; i++) {
   GLdouble x, y, z;
   gluUnProject (corners [i].v.coord.x, corners [i].v.coord.y, corners [i].v.coord.z, 
                 &transformation.SystemMatrix (-1) [0],
                 &transformation.SystemMatrix (-2) [0],
                 viewport,
                 &x, &y, &z);
   // ...
   }

The modelview matrix is the identity matrix.

2

You would have to pass your window coordinates.

{0.0, 0.0, 0.0} //depth is 0.0
{0.0, 0.0, 1.0} //depth is 1.0
{windowWidth, 0.0, 0.0} //depth is 0.0
{windowWidth, 0.0, 1.0} //depth is 0.0
and so on.

With the coordinates you are using in your example, you can’t use glUnProject but you can do this


memcpy(ProjectionModelviewMatrix, ProjectionMatrix, sizeof(float)*16);
glhMultMatrixf2(ProjectionModelviewMatrix, ModelviewMatrix);

//Let's invert PM matrix
glhInvertMatrixf2(ProjectionModelviewMatrix, PMInvert);

//Let's transform a single 4D point
corner = -1.0, -1.0, 0.0, 1.0;   //W is 1.0
glhMultiplyVectorByMatrix4by4f_1(4, corner, PMInvert, result4D);

//and result4D has the values in object space that you want.
3

While you can do that (and V-man’s already given you a good pointer there), you may not need to go to all that trouble.

If you’re setting your PERSPECTIVE projection matrix via glFrustum( l,r,b,t,n,f ), you know your frustum corners in EYE-SPACE are:

l,b,-n
r,b,-n
r,t,-n
l,t,-n
ls,bs,-f
rs,bs,-f
rs,ts,-f
ls,ts,-f

where s = f/n.

Then just multiply these EYE-SPACE points by the VIEWING^-1 transform to get to WORLD-SPACE.

If your projection matrix is ORTHOGRAPHIC instead – i.e. glOrtho( l,r,b,t,n,f ) – then it’s all the same, except s = 1.