Position of pixel in 3D world space?

OpenGL OpenGL: Basic Coding
1

Hi,

Can anyone tell me how to compute the 3D world position of a 2D pixel (x, y)?

The 2D pixel lies in the image. The image is on a rectangle in 3D space. What are the 3D coordinates of this pixel?

Thanks,
Slammo

2

hi,

After you do your transformation (it must always start from glLoadIdentity(); and before glPopMatrix() , use
glGetFloatv(GL_MODELVIEW_MATRIX,cvm) to get the tranfromed matrix;where cvm is used to hold it which is defined as: float cvm[16];
Then you call something like below;
convert_m(oldx,oldy,oldz,& newx,&newy,&newz);(cvm of course is the input to this function too)
where old is something you know before transformation and new is the output which is what you want to know after transforamtion.
Hope you get it

Originally posted by slammo:
[b]Hi,

Can anyone tell me how to compute the 3D world position of a 2D pixel (x, y)?

The 2D pixel lies in the image. The image is on a rectangle in 3D space. What are the 3D coordinates of this pixel?

Thanks,
Slammo[/b]

3

Hi, thanks for your post to my question.

If I understand what you’re saying, the transformation between a pixel’s screen coordinate and its world space coordinate depends on the modelview matrix. I agree, in fact, I think it also depends on the viewport and the projection matrix.

My question is, given a pixel’s coordinate, (x, y) on the screen, and the modelview matrix, projection matrix, and viewport, how do I compute the pixel’s location in 3D space?

Best regards,
Slammo

4

read up on gluUnProject(). it does what you need

jebus

5

Thanks for your suggestion, jebus.

gluUnproject requires an image-space z-coordinate. What z-coordinate should be used? I tried z = 0 but this didn’t work – I assume because z = 0 is the near clipping plane, which is in a different location than the image plane. Any suggestions?

Thanks,
Slammo

6

read it back from the depth buffer using glReadPixels

7

Thanks for the suggestion, chowe6685.

Reading the z-value from the depth buffer would give the position of the 3D point that projects onto the image pixel(x, y).

Instead, I am looking for the 3D position of the image pixel(x, y).

Thanks,
Slammo

8

so what you want to do is convert from image/rectangle space to world space, right? how about something like the following:

//rectangle points in world space. assume they're initialized.
Vector3 lowerLeft, lowerRight, upperLeft, upperRight;

//assume that the image covers the entire rectangle.
int xMin = 0, xMax = imageWidth, yMin = 0, yMax = imageHeight;

//image space coords. assume they're initialized.
int x, y;

float xFactor = x/(xMax-xMin);
float yFactor = y/(yMax-yMin);

Vector3 worldPt = (1.0f-xFactor)*(1.0f-yFactor)*lowerLeft + (xFactor)*(1.0f-yFactor)*lowerRight + (xFactor)*(yFactor)*upperRight + (1.0f-xFactor)*(yFactor)*upperLeft;

there might be something wrong with the code there, but it illustrates the general idea.

9

There is none. You cannot compute it.
If you are familiar with 3d space, you’ll know that viewport is a plane. If you take a point on this plane(your pixel), you cannot know from where it was projected, decause you need z value. You can find a line, which projectzion onto the line is your pixel:

   |
   |

__. plane with pixel
|

10

slammo, what are you asking for here?

do you want the world coordinate corresponding to a particular screen coordinate? or do you have a textured rectangle in world space and you want the world coordinate of an image coordinate on the textured rectangle? if it’s the latter, my previously posted code should work. if it’s the former, you have to read back the projected z value from the depth buffer and call gluUnProject() with the appropriate transforms, as everyone’s been saying.

11

Thanks for the posts, Zengar and SThomas.

Think of it this way – all unclipped points in 3D space project to the image, which is a rectangle lying somewhere in 3D space.

Each pixel in the image also has a 3D location, since it lies in the image rectangle. Note the location of a pixel is not the same as the location of a 3D point that projects to the pixel – that is why reading from the z-buffer will not work here.

SThomas, I believe your code would work if the 3D coordinates of the image rectangle were known. How would one compute these?

Thanks,
Slammo

12

I think you’re misunderstanding what the depth buffer is, slammo. It is not the z-coordinate of the point in world coordinates. It is a value between 0-1 that specifies where it is proportionately between the near and far clip plane. So, you use the x and y from the screen coordinates, and the z from the depth buffer value at that point, then use gluUnproject and you should get what you need.

What everyone here has been trying to tell you is essentially correct if I’m understanding what you want correctly.

13

Hi, thanks for the message, Deiussum.

I do understand the depth buffer – using it to get the z-value at the pixel (x, y), then using (x, y, z) with gluUnproject, will give the point in 3D space that projects the pixel.

However, I’m not looking for the point in 3D space that projects to the pixel, rather, I’m looking for the 3D coordinate of the pixel itself. The pixel exists in 3D space on a rectangle in 3D space – in this rectangle OpenGL creates the image.

Sorry if that point hasn’t been clear. Can you suggest a way to compute the pixel location?

Thanks,
Slammo

14

ok I went back and re-read the first post of yours. That seems to be saying that lets say you have a square with a point x, y.then this square is projected into 3d, say to make a side of a cube. You now want to find the 3d position of the original x, y point?

15

I’m confused. Are you saying you have an image of a 3d scene that you are using as a texture and you want to get a the world space for a pixel of that texture? If so, there’s no way you can do what you want. A 2D image does not contain any sort of depth data. You can’t just take a BMP and say that this point at x,y has a depth of z, because the BMP has no concept of z.

There are exceptions to this with something like a height map. Heightmaps view the color data as specifying the height. With images that aren’t heightmaps, color seldom corresponds to height/depth. I’m sure you could come up with algorithms that could find edges and approximately extrude depths, but they would be just that, approximations.

16

Hi Gavin and Deiussum,

Thanks for the responses… unfortunately, I guess I’m not making myself clear, since we’re not on the same page.

I thought that maybe someone on this board knew of a quick solution to this problem but I suppose that is not the case. I will figure it out on my own and post the solution when I get it.

Thanks to all who tried to help.

Slammo

17

originally posted by slammo:
SThomas, I believe your code would work if the 3D coordinates of the image rectangle were known. How would one compute these?

how are you drawing the rectangle? with glVertex() calls? if so, you can take your object space coordinates (the coordinates you supply to glVertex()) and transform them by the modelling transformation to get the world space coordinates. i’m fundamentally confused about what you’re asking for though. if we knew exactly what you were asking for any of us would be able to answer your question i think. “Think of it this way – all unclipped points in 3D space project to the image, which is a rectangle lying somewhere in 3D space.” i have no idea what you mean by that.

-steve

18

hmmmmm … i still think gluUnProject will do what you need. i can’t see the difference between:

  1. “…the point in 3D space that projects to the pixel…”
  2. “…the 3D coordinate of the pixel itself…”

using the pixel’s z-buffer value in gluUnProject would give you #1. i’m not sure what #2 is, cause the pixel only exists in 2D :stuck_out_tongue:

jebus

19

Perhaps a figure will help…

Consider this image (it’s the best I could find after a quick search).
http://www.esat.kuleuven.ac.be/~pollefey/tutorial/img306.gif

This is the pinhole camera model. In the figure, C is the camera center of projection, through which all imaged rays pass. R is a rectangle on the image plane where the image is formed. This is a 2D rectangle in 3D space.

In the figure, m is a pixel, that lies on the 2D image, which is on a plane in 3D space. One can refer to m using 2D pixel coordinates (x, y) in a coordinate system in the plane. What I want to know is m’s location in 3D space.

Note that I’m not looking for M, the point in 3D space the projects to m. I could easily compute M by reading the z-buffer at (x, y) and calling gluUnproject, as many of you have suggested.

I believe determining the 3D position of m is a function of the camera location, view direction, viewport, projection, and modelview matrix. I can grind through the equations but I thought maybe someone here knew a quick solution.

Regards,
Slammo

20

so … is R the near clipping plane? or a representation of the final rasterized image? or some other rectangle that happens to be in the scene and is also parallel to the screen?

jebus