glOrtho and gluUnproject inversion problem

Hi guys!

I’ve got a scene setup that is using a glOrtho projection, and it’s looking great so far. I’m inverting my bottom and top coordinates though because I want 0,0 to be at the top left…so after calculating my left, right, bottom, and top, I’m passing it like so:

glOrtho(left, right, top, bottom, 1.0, 100.0);

This gives me a projection where 0,0 is at the top left and, for example, 10,10 is at the bottom right. To do it any other way will require me to completely rewrite my scene logic and that would suck.

As I said though, all of this is working fine, except for one problem. I’ve added gluUnProject to the mix, and when I grab the mouse cursor position in world coordinates, I’m getting an inverted Y axis. It’s like it’s ignoring my glOrtho inversion. In the 0,0 top left - 10,10 bottom right example…if I click in the top left corner, gluUnProject gives me about 0,10.

Do I have to invert any of the parameters I’m sending to gluUnProject to match my inverted glOrtho?

Figured it out. I have to subtract the mouse cursor.y from viewport[3] to give me the real y value.

In other news, I have this code working excellently both ways (Project and UnProject). Only one problem…it’s only accurate in full screen mode.

If I’m running my GL instance in a window not full screen, the point that I draw representing the UnProjected mouse cursor position is offset…it appears to be offset by however much the window is moved from instantiation. for example, if I move the window down, the difference increases vertically. Same for if I resize the window. On initialization, the offset appears to push the drawn vertex down about the same distance as the titlebar is tall, and right about the same distance as the left window border. Odd.

Any way to cancel that out?