Object Selection Using the Back Buffer

OpenGL OpenGL: Basic Coding
1

Hi,

I’m trying to implement a 3D editor program. I’m having trouble getting the selection mechanism to work.

I’m using the approach outlined in Chapter 14 of the Red Book (“Object Selection Using the Back Buffer”). According to this approach I need to redraw the scene into the back buffer with the objects encoded by colour. Then I read the colour of the pixel under the cursor.

I’ve implemented this, however I’ve run into two problems.

  1. I don’t understand how the viewing position is included in this approach. This obviously makes a difference (e.g. compare a scene containing a rectangle drawn in the x-z plane when viewed from above as opposed to when viewed from the front). Also, if I translate the object then things don’t work properly.

  2. The second question follows on from the previous one. My application uses two subwindows which show different views of the object. Is there a way to detect which subwindow the user clicked in? If not, how do I know which view to use?

Here’s the selection code that I’m using:

void mouse(int button, int state, int x, int y)
{
	GLfloat pixel[4];											// An RGBA pixel.
	GLint value;

	if(state == GLUT_DOWN)
	{
		glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);
		glDrawBuffer(GL_BACK);
			
		glPushMatrix();
		glColor3f(1.0, 1.0, 1.0);
		glTranslatef(0.0, -1.0, 0.0);
		glRotatef(gRotate, 0.0f, 1.0f, 0.0f);
		glutSolidSphere(1.0, 32, 32);
		glPopMatrix();

		glColor3f(0.0, 0.0, 0.0);
		glBegin(GL_QUADS);
			glVertex3i(-10, 0, 10);
			glVertex3i(10, 0, 10);
			glVertex3i(10, 0, -10);
			glVertex3i(-10, 0, -10);
		glEnd();

		glReadBuffer(GL_BACK);
		glReadPixels(x, y, 1, 1, GL_RGBA, GL_FLOAT, pixel);
		printf("R: %f	 G: %f	 B: %f	
", pixel[0], pixel[1], pixel[2]);
		value = (GLint)pixel[0];

		if(value == 1)
			printf("Hit!
");
		else
			printf("Miss!
");
	}

}

I’ve searched for an example of this approach to selection but haven’t been able to find anything. Is there any sample code available that demonstrates selection using the back buffer?

Cheers,

Chris

2

I’ve figured out how to detect the different subwindows–use a different mouse function for each subwindow (obvious really!).

I’m still not clear about how to integrate the viewing position into the code.

Any ideas?

Cheers,

Chris

3

you have to draw the scene into the back buffer using exactly the same projection and modelview matrices, otherwise it will not work.

using glColor3f (float) could result in rounding errors. if you draw something with glColor3f(0.3, 0.4, 0.5) and read the color values from the buffer, you might get something close to (0.3, 0.4, 0.5), but not the exact values. better use glColor3ub (unsigned byte).

int object_id = 123456;
char r = object_id&255, g = (object_id/256)&255, b = (object_id/65536)&255;
glColor3ub(r,g,b);
// draw object

note that this does only work in the above way if you have 24-bit colours.

4

Thanks,

The thing that I’m not clear about is how I set the view of the back buffer so that it matches the view in the main part of the program. Here’s my code for drawing the scene. Note that the sphere is translated 1.0 in the y direction:

void drawScene(int window)
{
	int i;
	
	glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);
	
	glPushMatrix();
	glColor3f(0.5, 1.0, 1.0);
	glTranslatef(0.0, 1.0, 0.0);
	glRotatef(gRotate, 0.0f, 1.0f, 0.0f);
	glutWireSphere(1.0, 32, 32);
	glPopMatrix();

	glColor3f(0.5, 0.5, 0.5);
	glBegin(GL_QUADS);
		glVertex3i(-10, 0, 10);
		glVertex3i(10, 0, 10);
		glVertex3i(10, 0, -10);
		glVertex3i(-10, 0, -10);
	glEnd();

	glColor3f(0.7f, 0.7f, 0.7f);
	glBegin(GL_LINES);
	for(i = -10; i < 11; i+=2)
	{
		glVertex3f((GLfloat)i, 0.01f, -10.0);
		glVertex3f((GLfloat)i, 0.01f, 10.0);

		glVertex3f(-10.0, 0.01f, (GLfloat)i);
		glVertex3f(10.0, 0.01f, (GLfloat)i);
	}
	glEnd();
	
	glutSwapBuffers();
	
	gRotate += 1.0;
}

I’ve used:

gluLookAt(0.0, 0.0, 10.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0);

and

gluLookAt(0.0, 10.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0);

to set the two views.

I’m not clear how I account for the translation and the two different viewing positions when rendering to the back buffer.

Cheers,

Chris

5

if i understand your question, the answer is: it doesn’t make a difference if you’re in front or back buffer. you can choose a buffer to render to, but that will not change the projection or modelview matrix.

actually, you don’t even need to select glDrawBuffer(GL_BACK), because in a double-buffered configuration, the back buffer is default for rendering.

so you just have to draw your scene as usual, but without lighting, blending etc. and with the appropriate colours and without swapping buffers.

6

Hi,

The problem I’m having is not with drawing into the back buffer–it’s with setting the viewing position. Do I have to reverse the transformations that I perform on the model and the view?

In the code below, the sphere is translated one unit in the positive y direction. Do I need to undo this transformation when drawing into the back buffer?

In the code below, the hit test doesn’t work properly for the left-hand subwindow. If I reverse the translation that takes place in the drawScene funtion then it works properly.

I’ve included the full code below.

Cheers,

Chris

#include <GL/openglut.h>
#include <GL/glu.h>
#include <stdlib.h>
#include <stdio.h>

//********************************************************************\\

#define WINDOW_WIDTH	1000
#define WINDOW_HEIGHT	500
#define BORDER			5

//********************************************************************\\

void init(void);
void renderMain(void);
void renderIdle(void);
void renderSW01(void);
void renderSW02(void);
void drawScene(int window);
void reshape(int w, int h);
void resetProjMat(int w1, int h1);
void mouseSW01(int button, int state, int x, int y);
void mouseSW02(int button, int state, int x, int y);
int	 main(int argc, char** argv);

//********************************************************************\\

GLint	gMainWindow;
GLint	gSubWindow01;
GLint	gSubWindow02;
static	GLfloat	gRotate = 0.0;
GLint	w;
GLint	h;
GLint	gHit = FALSE;

//********************************************************************\\

void init(void)
{
	glClearColor(0.0, 0.0, 0.0, 0.0);
	glShadeModel(GL_FLAT);
	glClearDepth(1.0f);									// Depth Buffer Setup
	glEnable(GL_DEPTH_TEST);							// Enables Depth Testing
	glDepthFunc(GL_LEQUAL);								// The Type Of Depth Testing To Do
	glEnable(GL_COLOR_MATERIAL );
	glHint(GL_PERSPECTIVE_CORRECTION_HINT, GL_NICEST);
}

//********************************************************************\\

void renderMain(void)
{
	glutSetWindow(gMainWindow);
	glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);
	glutSwapBuffers();
}

//********************************************************************\\

void renderIdle(void)
{
	renderSW01();
	renderSW02();
}

//********************************************************************\\

void renderSW01(void)
{
	glutSetWindow(gSubWindow01);
	glLoadIdentity();
	gluLookAt(0.0, 0.0, 10.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0);
	drawScene(gSubWindow01);
}

//********************************************************************\\

void renderSW02(void)
{
	glutSetWindow(gSubWindow02);
	glLoadIdentity();
	gluLookAt(0.0, 10.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0);
	drawScene(gSubWindow02);
}

//********************************************************************\\

void drawScene(int window)
{
	int i;
	
	glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);
	
	if(gHit == TRUE)
		glColor3f(1.0, 0.0, 0.0);
	else
		glColor3f(0.5, 1.0, 1.0);
	
	glPushMatrix();
	glTranslatef(0.0, 1.0, 0.0);
	glRotatef(gRotate, 0.0f, 1.0f, 0.0f);
	glutWireSphere(1.0, 32, 32);
	glPopMatrix();

	glColor3f(0.5, 0.5, 0.5);
	glBegin(GL_QUADS);
		glVertex3i(-10, 0, 10);
		glVertex3i(10, 0, 10);
		glVertex3i(10, 0, -10);
		glVertex3i(-10, 0, -10);
	glEnd();

	glColor3f(0.7f, 0.7f, 0.7f);
	glBegin(GL_LINES);
	for(i = -10; i < 11; i+=2)
	{
		glVertex3f((GLfloat)i, 0.01f, -10.0);
		glVertex3f((GLfloat)i, 0.01f, 10.0);

		glVertex3f(-10.0, 0.01f, (GLfloat)i);
		glVertex3f(10.0, 0.01f, (GLfloat)i);
	}
	glEnd();
	
	glutSwapBuffers();
	
	gRotate += 1.0;
}

//********************************************************************\\

void reshape(int w1, int h1)
{
	if(h1 == 0)
		h1 = 1;

	w = w1;
	h = h1;

	glutSetWindow(gSubWindow01);
	glutPositionWindow(BORDER, BORDER);
	glutReshapeWindow((w - (3 * BORDER)) / 2, h - (2 * BORDER));
	resetProjMat((w - (3 * BORDER)) / 2, h - (2 * BORDER));

	glutSetWindow(gSubWindow02);
	glutPositionWindow((2 * BORDER) + ((w - (3 * BORDER)) / 2), BORDER);
	glutReshapeWindow((w - (3 * BORDER)) / 2, h - (2 * BORDER));
	resetProjMat((w - (3 * BORDER)) / 2, h - (2 * BORDER));
}

//********************************************************************\\

void resetProjMat(int w1, int h1)
{
	GLfloat ratio;
	
	ratio = 1.0f * w1 / h1;
	
	// Reset the coordinate system before modifying.
	glMatrixMode(GL_PROJECTION);
	glLoadIdentity();
	
	// Set the viewport to be the entire window.
    glViewport(0, 0, w1, h1);

	// Set the clipping volume.
	gluPerspective(45, ratio, 0.1, 1000);
	glMatrixMode(GL_MODELVIEW);
}

//********************************************************************\\

void mouseSW01(int button, int state, int x, int y)
{
	GLfloat pixel[4];											// An RGBA pixel.
	GLint value;
	
	if(state == GLUT_DOWN)
	{
		printf("In subwindow 01
");
		
		glLoadIdentity();
		gluLookAt(0.0, 0.0, 10.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0);
		
		glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);
		glDrawBuffer(GL_BACK);
			
		glPushMatrix();
		glColor3f(1.0, 1.0, 1.0);
		glTranslatef(0.0, 1.0, 0.0);							
		glRotatef(gRotate, 0.0f, 1.0f, 0.0f);
		glutSolidSphere(1.0, 32, 32);
		glPopMatrix();

		glColor3f(0.0, 0.0, 0.0);
		glBegin(GL_QUADS);
			glVertex3i(-10, 0, 10);
			glVertex3i(10, 0, 10);
			glVertex3i(10, 0, -10);
			glVertex3i(-10, 0, -10);
		glEnd();

		glReadBuffer(GL_BACK);
		glReadPixels(x, y, 1, 1, GL_RGBA, GL_FLOAT, pixel);
		printf("R: %f	 G: %f	 B: %f	
", pixel[0], pixel[1], pixel[2]);
		value = (GLint)pixel[0];

		if(value == 1)
		{
			printf("Hit!
");
			gHit = TRUE;
		}
		else
		{
			printf("Miss!
");
			gHit = FALSE;
		}
	}
}

//********************************************************************\\

void mouseSW02(int button, int state, int x, int y)
{
	GLfloat pixel[4];											// An RGBA pixel.
	GLint value;

	if(state == GLUT_DOWN)
	{
		printf("In subwindow 02
");

		glLoadIdentity();
		gluLookAt(0.0, 10.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0);
		
		glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);
		glDrawBuffer(GL_BACK);
			
		glPushMatrix();
		glColor3f(1.0, 1.0, 1.0);
		glTranslatef(0.0, 1.0, 0.0);
		glRotatef(gRotate, 0.0f, 1.0f, 0.0f);
		glutSolidSphere(1.0, 32, 32);
		glPopMatrix();

		glColor3f(0.0, 0.0, 0.0);
		glBegin(GL_QUADS);
			glVertex3i(-10, 0, 10);
			glVertex3i(10, 0, 10);
			glVertex3i(10, 0, -10);
			glVertex3i(-10, 0, -10);
		glEnd();

		glReadBuffer(GL_BACK);
		glReadPixels(x, y, 1, 1, GL_RGBA, GL_FLOAT, pixel);
		printf("R: %f	 G: %f	 B: %f	
", pixel[0], pixel[1], pixel[2]);
		value = (GLint)pixel[0];

		if(value == 1)
		{
			printf("Hit!
");
			gHit = TRUE;
		}
		else
		{
			printf("Miss!
");
			gHit = FALSE;
		}
	}
}

//********************************************************************\\

int main(int argc, char** argv)
{
	int subWindowWidth = (WINDOW_WIDTH - (3 * BORDER)) / 2;
	int subWindowHeight = WINDOW_HEIGHT - (2 * BORDER);
	
	glutInit(&argc, argv);
	glutInitDisplayMode(GLUT_DOUBLE | GLUT_RGBA);
	glutInitWindowSize(WINDOW_WIDTH, WINDOW_HEIGHT);
	glutInitWindowPosition(100, 100);
	
	gMainWindow = glutCreateWindow("Test");
	init();
	glutDisplayFunc(renderMain);
	glutIdleFunc(renderIdle);
	glutReshapeFunc(reshape);

	gSubWindow01 = glutCreateSubWindow(gMainWindow, BORDER, BORDER, subWindowWidth, subWindowHeight);
	init();
	glutDisplayFunc(renderSW01);
	glutMouseFunc(mouseSW01);

	gSubWindow02 = glutCreateSubWindow(gMainWindow, (2 * BORDER) + subWindowWidth, BORDER, subWindowWidth, subWindowHeight);
	init();
	glutDisplayFunc(renderSW02);
	glutMouseFunc(mouseSW02);

	glutMainLoop();
	return 0;
}

//********************************************************************\\
7

to say it again: you have to draw the scene exactly the same way as you would do if it were exposed to the user. using the same projection and modelview matrices, and using the same transformations. you only have to turn off light, blending, textures or anything else that could affect the unique colour that you assign to each object (or each triangle).

so if you draw the scene and swap buffers- which you usually don’t do for picking- it should look exactly like the scene which is shown to the user. only with different colors.

8

I’m still having problems getting this to work properly. At the moment, selection in the left subwindow is not working properly. In this window, a hit is being registered when the user clicks in the region below where the object is drawn. The selection area seems to be off by the amount that the sphere is translated. That is, if I translate by -sy (rather than sy) in the mouseSW01(…) then the selection works properly.

I’m obviously missing something here although I’m not sure what. If someone could explain this it would be very much appreciated.

The code for drawing the original scene, setting the viewing positions and drawing into the back buffer for selection testing is below.

Cheers,

Chris

void drawScene(int window)
{
	int i;
	
	glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);
	
	if(gHit == TRUE)
		glColor3f(1.0, 0.0, 0.0);
	else
		glColor3f(0.5, 1.0, 1.0);
	
	glPushMatrix();
	glTranslatef(sx, sy, sz);
	glRotatef(gRotate, 0.0f, 1.0f, 0.0f);
	glutWireSphere(0.5, 32, 32);
	glPopMatrix();

	glColor3f(0.5, 0.5, 0.5);
	glBegin(GL_QUADS);
		glVertex3i(-10, 0, 10);
		glVertex3i(10, 0, 10);
		glVertex3i(10, 0, -10);
		glVertex3i(-10, 0, -10);
	glEnd();

	glColor3f(0.75f, 0.75f, 0.75f);
	glBegin(GL_LINES);
		for(i = -10; i < 11; i++)
		{
			glVertex3f((GLfloat)i, 0.0001f, -10.0);
			glVertex3f((GLfloat)i, 0.0001f, 10.0);

			glVertex3f(-10.0, 0.0001f, (GLfloat)i);
			glVertex3f(10.0, 0.0001f, (GLfloat)i);
		}
	glEnd();
	
	glutSwapBuffers();
	
	gRotate += 1.0;
}  

//********************************************************************\\

void mouseSW01(int button, int state, int x, int y)
{
	GLfloat pixel[4];											// An RGBA pixel.
	GLint value;
	
	if(state == GLUT_DOWN)
	{
		glLoadIdentity();
		gluLookAt(0.0, 0.0, 10.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0);
		
		glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);
		glDrawBuffer(GL_BACK);
			
		glPushMatrix();
		glColor3f(1.0, 1.0, 1.0);
		glTranslatef(sx, -sy, sz);	// Problem here!						
		glRotatef(gRotate, 0.0f, 1.0f, 0.0f);
		glutSolidSphere(0.5, 32, 32);
		glPopMatrix();

		glColor3f(0.0, 0.0, 0.0);
		glBegin(GL_QUADS);
			glVertex3i(-10, 0, 10);
			glVertex3i(10, 0, 10);
			glVertex3i(10, 0, -10);
			glVertex3i(-10, 0, -10);
		glEnd();

		glReadBuffer(GL_BACK);
		glReadPixels(x, y, 1, 1, GL_RGBA, GL_FLOAT, pixel);
		value = (GLint)pixel[0];

		if(value == 1)
		{
			printf("Hit!
");
			gHit = TRUE;
		}
		else
		{
			printf("Miss!
");
			gHit = FALSE;
		}
	}
}

//********************************************************************\\

void mouseSW02(int button, int state, int x, int y)
{
	GLfloat pixel[4];											// An RGBA pixel.
	GLint value;

	if(state == GLUT_DOWN)
	{
		glLoadIdentity();
		gluLookAt(0.0, 10.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0);
		
		glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);
		glDrawBuffer(GL_BACK);
			
		glPushMatrix();
		glColor3f(1.0, 1.0, 1.0);
		glTranslatef(sx, sy, sz);
		glRotatef(gRotate, 0.0f, 1.0f, 0.0f);
		glutSolidSphere(0.5, 32, 32);
		glPopMatrix();

		glColor3f(0.0, 0.0, 0.0);
		glBegin(GL_QUADS);
			glVertex3i(-10, 0, 10);
			glVertex3i(10, 0, 10);
			glVertex3i(10, 0, -10);
			glVertex3i(-10, 0, -10);
		glEnd();

		glReadBuffer(GL_BACK);
		glReadPixels(x, y, 1, 1, GL_RGBA, GL_FLOAT, pixel);
		value = (GLint)pixel[0];

		if(value == 1)
		{
			printf("Hit!
");
			gHit = TRUE;
		}
		else
		{
			printf("Miss!
");
			gHit = FALSE;
		}
	}
}

//********************************************************************\\

void renderSW01(void)
{
	glutSetWindow(gSubWindow01);
	glLoadIdentity();
	gluLookAt(0.0, 0.0, 10.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0);
	drawScene(gSubWindow01);
}

//********************************************************************\\

void renderSW02(void)
{
	glutSetWindow(gSubWindow02);
	glLoadIdentity();
	gluLookAt(0.0, 10.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0);
	drawScene(gSubWindow02);
}