Help! Can't get picking to work properly

I am trying to get picking to work, but despite reading about it in the OpenGL Super Bible and studying the NeHe examples for C++Builder 6, I can’t get it to work properly.

I am able to pick some objects but not all, and only in certain views (typically perspective and isometric, not cross-section, elevation, or plan). Some objects are picked when I click near them (noy on them)!

Can anyone see what is wong wih my code please? I would be happy to supply ALL the code (written for C++Builder 6) if that helps.

Many thanks in anticipation

Andrew

Here are (what I think are) the most significant functions:

void OnResize(int width, int height)
{
glViewport(0, 0, width, height);
}

void OnPaint()
{
glLoadIdentity();
glClearColor(1.0f, 1.0f, 1.0f, 1.0f);
glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);

glMatrixMode(GL_PROJECTION);
glLoadIdentity();
if(projection == Perspective) {
    double aspect(double(width) / height);
double near_plane(size * width / 10);
    double far_plane(near_plane * 1000);
gluPerspective(45, aspect, near_plane, far_plane);
}
else {
    double h(size * height);    // in cm
    double w(size * width);     // in cm
    glOrtho(-w, w, -h, h, -w, w);
}
glMatrixMode(GL_MODELVIEW);

LocateCamera();  // see below
RenderScene();   // see below
SwapBuffers();
::ValidateRect(hwnd, NULL);

}

void LocateCamera()
{
switch(projection) {
case CrossSection:
gluLookAt(1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, -1.0, 0.0);
break;
case Plan:
gluLookAt(0.0, -1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0);
break;
case Elevation:
gluLookAt(0.0, 0.0, -1.0, 0.0, 0.0, 0.0, 0.0, -1.0, 0.0);
break;
case Isometric:
gluLookAt(-1.0, -1.0, -1.0, 0.0, 0.0, 0.0, 0.0, -1.0, 0.0);
break;
case Perspective:
gluLookAt(-20.0, -20.0, -20.0, 0.0, 0.0, 0.0, 0.0, -1.0, 0.0);
break;
default:
throw std::invalid_argument(FUNC);
}
}

void RenderScene()
{
glMatrixMode(GL_MODELVIEW);
glPushMatrix();

glInitNames();
glPushName(0);

glScaled(scale, scale, scale);
glScaled(100, 100, 100);    // we are working in cm per pixel

assert(Draw);
Draw();  // draws the obejcts and pushes names onto the name stack

glPopMatrix();

}

bool HitTest(int x, int y)
{
glSelectBuffer(name_stack.size(), &name_stack[0]);

int viewport[4];
glGetIntegerv(GL_VIEWPORT, viewport);

glRenderMode(GL_SELECT);

glMatrixMode(GL_PROJECTION);
glPushMatrix();
glLoadIdentity();		

gluPickMatrix((double)x, viewport[3] - (double)y, 20, 20, viewport);

double aspect(double(width) / height);
double near_plane(size * width / 10.0);
double far_plane(near_plane * 100);
gluPerspective(45, aspect, near_plane, far_plane);

glMatrixMode(GL_MODELVIEW);
RenderScene();
glMatrixMode(GL_PROJECTION);

unsigned int hits = glRenderMode(GL_RENDER);

// pop the projection matrix
glPopMatrix();

return hits;

}
// end of code

Can anyone help me with this please? I am banging my head against a brick wall and had hoped this newsgroup would be able to help.

Here’s hoping…

Andrew

Of course this doesn’t work because your HitTest routine is rendering something completely different than the OnPaint routine!

1. Because you only handled the Perspective case in the picking matrix setup, not the orthographic cases!
2. You didn’t do LocateCamera in the picking.
3. Your y-picking coordinate is one unit off. viewport[3] is height, mouse y-coordinates are from 0 to height - 1.
   (4. glRenderMode returns GLint. Negative values say that the selection buffer had an overflow and the last hit record is incomplete.)
   (5. The pick location is 20*20 pixels which is pretty big.)

What you should do instead:

• Code only one single rendering function for both render and selction to always use the exact same matrices for both pathes.
• Only premultiply the gluPickMatrix if you’re picking in the function (needs a single boolean flag in your app).

A good way to debug such errors is to omit the glRenderMode call, use a bigger picking region and just look at what is rendered in that path.

Originally posted by Relic:
Of course this doesn’t work because your HitTest routine is rendering something completely different than the OnPaint routine!

Thanks for the reply. I looked at several online examples and they didn’t seem to have identical drawing during rendering and hit testing (but I did think that strange).

Are you sure about the off-by-one error in the screen’s y-position? None of the books/online examples I have looked at correct for that.

I have now amended my code to use identical drawing during render and hit-test, but the problems persists. Any other thoughts?

Andrew

Originally posted by Andrew Bond:

Are you sure about the off-by-one error in the screen’s y-position? None of the books/online examples I have looked at correct for that.

Just insert some values into your equation and see for yourself.

You have: viewport[3] - (double)y

Let’s say viewport[3] is 100, meaning the window is 100 pixels high, where the Y-coordinate ranges from 0 to 99 inclusive. You click on the bottom row, wich corresponds to Y-coordinate 99 in window coordinates, and, according to your formula, corresponds to Y-coordinate 1 in OpenGLs coordinate space. Since coordinates ranges from 0 to 99, 1 means the second bottom row, not bottom one.

The correct formula is: viewport[3] - (double)y - 1

But honestly, this isn’t usually much of a problem, as there’s just a one pixel offset. It is, however, not the correct formula for transforming the coordinate from window to OpenGL coordinates. And it’s a surprisingly common error too.

As I said: “A good way to debug such errors is to omit the glRenderMode call, use a bigger picking region and just look at what is rendered in that path.”

There are perfectly working examples in the GLUT examples and the RedBook.
Here is one. http://www.cs.umu.se/kurser/TDBC07/HT05/projects/doc/openglbk/select.c
(Yes, the rendering is different than the selection in this code, because it’s showing how the selection volume surrounds the triangles for educational purposes. Just concentrate on the selection.)

Show the new code.

Thanks to those of you who offered me help on this - I have now solved the problem and it was a fairly basic error (mea culpa)…

I was passing screen x, y coordinates to the hit testing function instead of window coordinates. My hit testing therefore almost worked when I had my window full size and never worked when normalized.

Anyway, thanks for the helpful feedback.

Andrew