How to drag a 2D selection rectangle over 3D view?

In my app, I want to let the user drag a 2D “crop rectangle” over a static 3D view. (This is so they can save or print this selected area.)

I can take care of the mouse tracking and calculating the coordinates for the rectangle.

What I’m wondering is what is a good way to re-draw the 3D view as the rectangle covers and uncovers the 3D area underneath?

My app creates a shaded polygon mesh with directional lighting and potentially MILLIONS of vertexes. I have the mesh set up as a display list. However, when the mesh object gets big, it gets pretty slow to redraw.

I’m thinking that I should render my 3D view into the color buffer, then copy it off to an auxiliary color buffer or an FBO the size of the screen. Then, as the user drags the selection rectangle I can quickly copy the newly exposed pixels back to the front buffer.

However, I want my app to run on a wide variety of different machines (any Mac that has OS 10.4 (Tiger) or later on it.) From what I’ve read, the version of OpenGL running on macs varies widely, so I can’t be sure of FBO support or auxiliary color buffers.

Duncan C

You may use the oldish way of glCopyTexSubImage to copy once the framebuffer to a texture when the user starts the crop rectangle, then draw it as a fullscreen quad for fast interaction.
http://www.opengl.org/sdk/docs/man/xhtml/glCopyTexSubImage2D.xml

You probably should take a look at rubber banding. Rubber banding basically draws a rectangle on the screen while the pixels in the area covered by the rectangle are XOR-ed with some value (usually white), by drawing the rectangle twice at the exact same position you get the original image back. Here’s some code to get you started


void drawRect(int x1, int y1, int x2, int y2)
{
    glEnable(GL_COLOR_LOGIC_OP);
    glLogicOp(GL_XOR);
    glColor4f(1.0f, 1.0f, 1.0f, 1.0f);
    glPolygonMode(GL_FRONT_AND_BACK, GL_LINE);
    glRecti(x1, windowheight - y1, x2,windowheight - y2);
    glPolygonMode(GL_FRONT_AND_BACK, GL_FILL);
    glDisable(GL_COLOR_LOGIC_OP);
}

So first you draw the rectangle to show the rubber band. If you want to update the rectangle to a new position/size, draw the first rectangle again to restore the original image and then draw the new rectangle.

It’s fast because it only needs to draw the rectangle without any texture accesses. I believe color logic ops are supported as of OpenGL 1.1 so you shouldn’t run into any problems there.

Another advantage of rubber banding is that it is always visible, independent of the underlying scene color. Of the scene is black you will see a white rectangle, if it’s white you will see a black rectangle,etc.

NiCo,

That would certainly work, even if it would create some really odd-looking colors. I was actually thinking of showing the crop area in a more elaborate way, (darkening the area outside the crop the way Photoshop does), and that would require buffering the current contents of the viewport somehow.

By the way, can you draw 2D rectangles onto a viewport that’s set up for 3D with perspective projection? Wouldn’t I need to switch to orthographic projection at the beginning of the routine, draw the rect, then switch back to perspective projection?

Duncan C

ZbuffeR,

Don’t textures need to be square, and a power of 2 in size, prior to very recent versions of OpenGL? Would you suggest creating a square power-of-two texture large enough to cover the whole view?

Duncan C

Of course :slight_smile: Even though it’s possible to draw it in 3D the math is simpler when using ortho projection.

Regarding the power of 2 size. That’s true for older graphics cards, but in your case texture rectangles are also an option. Texture rectangles are not constrained to have power of 2 sizes. Your hardware needs to support GL_NV_texture_rectangle/GL_ARB_texture_rectangle.

Indeed, if the hardware does not support NPOT nor rectangular textures, you may create a power of two and only use the rectangular part you need, with appropriate coordinates so the unused parts are not shown.

NiCo,

Based on your recommendation I coded my app to use XOR drawing. It sounded like a simple solution to the problem.

I set up for ortho drawing and get my coordinates set up to match the system’s coordinates for the “view” the OpenGL content is drawn into.

I’m getting rectangles drawn in inverse colors, but for some reason they aren’t being erased when I re-draw them.

My app uses double buffered rendering, and for the duration of my mouse handling while drawing the selection rectangle, I temporarily set the target for drawing to the front color buffer with code like this:

	//Save the current buffer target
	glGetIntegerv(GL_DRAW_BUFFER, &last_write_buffer);
	glDrawBuffer(GL_FRONT);

I found i also had to turn off lighting in order to get my xor based drawing to work correctly.

Anyway, the rectangles draw with bits flipped in the color buffer, as expected. They just don’t un-draw when I repeat the operation.

Interestingly, If I put two draw calls in a row, e.g:

	glRecti(x1,  y1, x2, y2);
	glRecti(x1,  y1, x2, y2);

They DO cancel each other out, and nothing changes in my view. If I draw my rectangle, do a glFlush() and go back to my “while mouse is down” loop before trying to re-draw the rect to erase it, it doesn’t work.

Any thoughts?

P.S.: My whole setup routine to get ready to draw looks like this. (This is mostly straight C code, with a little Objective C syntax. The function definitions are in Objective C, and object method calls are in the form
[object method]

  • (void) ortho_setup
    {
    GLsizei w,h;
    NSRect rectView = [self bounds]; //Get bounds of OpenGL view

    w = rectView.size.width;
    h = rectView.size.height;
    [[self openGLContext] makeCurrentContext];
    glViewport(0, 0, w, h);
    glMatrixMode (GL_PROJECTION);
    glLoadIdentity();
    glOrtho(0, w, 0, h, 1, -1);
    glMatrixMode (GL_MODELVIEW);
    glLoadIdentity();

    glDisable(GL_LIGHTING); //turn off lighting effects

    glEnable(GL_COLOR_LOGIC_OP);
    glLogicOp(GL_XOR);
    glColor4f(1.0f, 1.0f, 1.0f, 1.0f);
    glPolygonMode(GL_FRONT_AND_BACK, GL_LINE);
    }

And my rectangle drawing routine looks like this:

-(void) drawSelectionRect: (NSRect) theSelectionRect
{
if (!NSEqualRects(theSelectionRect, NSZeroRect) )
{
GLint x1 = theSelectionRect.origin.x;
GLint y1 = theSelectionRect.origin.y;
GLint x2 = theSelectionRect.origin.x + theSelectionRect.size.width;
GLint y2 = theSelectionRect.origin.y + theSelectionRect.size.height;
glRecti(x1, y1, x2, y2);
glFlush ();
}
}

Maybe there’s something wrong with the number of times the drawSelectionRect func is called on every mouse movement. Try printing it’s 4 parameter values to stderr each time the func is called and see if they come in pairs.

Here’s some code I wrote to double check if it still works :slight_smile:


#include <stdlib.h>
#include <stdio.h>
#include "window.hpp"
#include <math.h>

ncglWindow glw;

int  rect[4];
bool active = false;

void drawRect(int x1, int y1, int x2, int y2)
{
    glDrawBuffer(GL_FRONT);

	glMatrixMode(GL_PROJECTION);
	glLoadIdentity();
	glOrtho(0,glw.getWidth(),0,glw.getHeight(),-1,1);

	glMatrixMode(GL_MODELVIEW);
	glLoadIdentity();

	glEnable(GL_COLOR_LOGIC_OP);
    glLogicOp(GL_XOR);
    glColor4f(1.0f, 1.0f, 1.0f, 1.0f);
    glPolygonMode(GL_FRONT_AND_BACK, GL_LINE);
    glRecti(x1, y1, x2, y2);
    glPolygonMode(GL_FRONT_AND_BACK, GL_FILL);
    glDisable(GL_COLOR_LOGIC_OP);
    glFlush();

	glDrawBuffer(GL_BACK);
}

void display() {

	if (!active)
	{

		glViewport(0,0,glw.getWidth(),glw.getHeight());

		glClearColor(0.0f,0.0f,0.0f,0.0f);
		glClear(GL_COLOR_BUFFER_BIT);

		glMatrixMode(GL_PROJECTION);
		glLoadIdentity();

		glMatrixMode(GL_MODELVIEW);
		glLoadIdentity();

		glBegin(GL_TRIANGLES);
		glColor3f(1.0f,0.0f,0.0f);
		glVertex3f(-0.5f,-0.5f,0.0f);
		glColor3f(0.0f,1.0f,0.0f);
		glVertex3f( 0.5f,-0.5f,0.0f);
		glColor3f(0.0f,0.0f,1.0f);
		glVertex3f( 0.0f, 0.5f,0.0f);
		glEnd();


		glw.swapBuffers();
	}

}

void cleanExit() {

}

void key (unsigned char key, int x, int y)  {

	switch (key) {

	case '\033':
		cleanExit();
		exit(0);
		break;
	}
	glw.postRedisplay();
}

void mouse (int button, int state, int x, int y)  {

	switch (button) {

	case GLUT_LEFT_BUTTON:
		if (state==GLUT_DOWN)
		{
			rect[0]=rect[2]=x;
			rect[1]=rect[3]=glw.getHeight()-1-y;
			drawRect(rect[0],rect[1],rect[2],rect[3]);
			active=true;
		}
		if (state==GLUT_UP)
		{
			drawRect(rect[0],rect[1],rect[2],rect[3]);
			active=false;
		}
		break;
	}
	glw.postRedisplay();
}

void movedMouse (int x, int y)  {
	drawRect(rect[0],rect[1],rect[2],rect[3]);
	rect[2]=x;
	rect[3]=glw.getHeight()-1-y;
	drawRect(rect[0],rect[1],rect[2],rect[3]);
	glw.postRedisplay();
}

int main(int argc, char** argv) {

	glw.init("Template",512,512);

	glw.setDisplayFunc(display);
	glw.setKeyboardFunc(key);
	glw.setMouseFunc(mouse);
	glw.setMotionFunc(movedMouse);
	glw.start();

	return 0;
}

NiCo,

I did use a message to the log to make sure the code is doing what it should.

It draws each rectangle once to show it, and once again to erase it. No extra drawRect calls.

I suspect something platform-specific. I’m on a Mac, where EVERYTHING is done using OpenGL, and the system plays games with the way your front and back buffers work (glFlush swaps the buffers)

Auch… in that case I would try ZbuffeRs solution using glCopyTexSubImage. It’s also cleaner code as it doesn’t require you to mess with the drawbuffers. That way you can also implement the rectangle the way you want it like darkening the area outside the crop area the way Photoshop does.

Folks,

I sent a private message to NiCo with a sample Glut project that had the same problem. It turns out that I had to disable depth testing in my 2D orthographic drawing. With depth testing turned off, the second time I draw a rectangle, it has the same depth as the first rectangle, so it is not shown.

When I turn off depth testing as part of the setup for my 2D selection rectangles in the test project, it works!

For other newbies like me, the call is

glDisable(GL_DEPTH_TEST
//2D drawing goes here.
glEnable(GL_DEPTH_TEST);

My full setup routine for 2D drawing (on top of a viewport set up for 3D perspective drawing) is:

  • (void) ortho_setup
    {
    GLsizei w,h;
    NSRect rectView = [self bounds]; //Get the bounds of the OpenGL view

    w = rectView.size.width;
    h = rectView.size.height;
    [[self openGLContext] makeCurrentContext];
    glViewport(0, 0, w, h);
    glMatrixMode (GL_PROJECTION);
    glLoadIdentity();
    glOrtho(0, w, 0, h, 1, -1);
    glMatrixMode (GL_MODELVIEW);
    glLoadIdentity();

    glDisable(GL_LIGHTING); //turn off lighting effects
    glEnable(GL_COLOR_LOGIC_OP);
    glLogicOp(GL_XOR);
    glDisable(GL_DEPTH_TEST);
    glColor4f(1.0f, 1.0f, 1.0f, 1.0f);
    glPolygonMode(GL_FRONT_AND_BACK, GL_LINE);

    //Save the current buffer target
    glGetIntegerv(GL_DRAW_BUFFER, &last_write_buffer);
    glDrawBuffer(GL_FRONT); //set the drawbuffer to the front readbuffer.
    }

Thanks again NiCo for your help.

You’re welcome :slight_smile: