PDA

View Full Version : Insane picking problem



Johan Torp
08-02-2004, 12:29 AM
Hi!

I'm going insane over this...

I have a really strange picking problem. glRenderMode(GL_RENDER) returns -1 if I draw anything while having anything else than zero on the name stack.

First I do picking, then I update the time and move things around and then I do the framebufffer rendering and after that I swap back and front buffer.

Here is the actual code:


glGetIntegerv(GL_VIEWPORT, viewPort);
glSelectBuffer(PICK_BUFFER_LENGTH, mPickBuffer);

glRenderMode(GL_SELECT);

glInitNames();
glPushName(0);

glMatrixMode(GL_PROJECTION);
glPushMatrix();
glLoadIdentity();
gluPickMatrix(screenX, (GLdouble)(viewPort[3]-screenY),
1.0f, 1.0f, viewPort);
TheGame->SetPerspective(); // set proper perspective go to modelview and call gluLookAt

TheGame->RenderScene();
glMatrixMode(GL_PROJECTION);
glPopMatrix();
glMatrixMode(GL_MODELVIEW);
hits = glRenderMode(GL_RENDER);glRenderMode is never ever supposed to return -1 according to any spec. No GL errors are reported.

Thanks for your help!

/Johan Torp

Relic
08-02-2004, 01:20 AM
Read the glSelectBuffer manual:
"If the hit record is larger than the number of remaining locations in buffer, as much data as can fit is copied, and the overflow flag is set."
"If the overflow bit was set when glRenderMode was called, a negative hit record count is returned."

If glRenderMode returns 0 it means an error, -1 means one hit record and your buffer had an overflow. Increase the buffer size.

Usual picking errors include wrong matrices.
Make sure you matrices are set correctly, e.g. if TheGame->RenderScene() needs the modelview matrix to be active your code doesn't work.

To validate the picking matrices I normally render things with a bigger selection matrix applied into the front buffer, step through the debugger and look at what is rendered.