View Full Version : Position of pixel in 3D world space?
slammo
05282003, 05:50 AM
Hi,
Can anyone tell me how to compute the 3D world position of a 2D pixel (x, y)?
The 2D pixel lies in the image. The image is on a rectangle in 3D space. What are the 3D coordinates of this pixel?
Thanks,
Slammo
beginner620824
05282003, 05:01 PM
hi,
After you do your transformation (it must always start from glLoadIdentity(); and before glPopMatrix() http://192.48.159.181/discussion_boards/ubb/wink.gif, use
glGetFloatv(GL_MODELVIEW_MATRIX,cvm) to get the tranfromed matrix;where cvm is used to hold it which is defined as: float cvm[16];
Then you call something like below;
convert_m(oldx,oldy,oldz,& newx,&newy,&newz);(cvm of course is the input to this function too)
where old is something you know before transformation and new is the output which is what you want to know after transforamtion.
Hope you get it
Originally posted by slammo:
Hi,
Can anyone tell me how to compute the 3D world position of a 2D pixel (x, y)?
The 2D pixel lies in the image. The image is on a rectangle in 3D space. What are the 3D coordinates of this pixel?
Thanks,
Slammo
slammo
05292003, 04:51 AM
Hi, thanks for your post to my question.
If I understand what you're saying, the transformation between a pixel's screen coordinate and its world space coordinate depends on the modelview matrix. I agree, in fact, I think it also depends on the viewport and the projection matrix.
My question is, given a pixel's coordinate, (x, y) on the screen, and the modelview matrix, projection matrix, and viewport, how do I compute the pixel's location in 3D space?
Best regards,
Slammo
jebus
05292003, 08:28 AM
read up on gluUnProject(). it does what you need
jebus
slammo
05292003, 09:26 AM
Thanks for your suggestion, jebus.
gluUnproject requires an imagespace zcoordinate. What zcoordinate should be used? I tried z = 0 but this didn't work  I assume because z = 0 is the near clipping plane, which is in a different location than the image plane. Any suggestions?
Thanks,
Slammo
chowe6685
05292003, 09:34 AM
read it back from the depth buffer using glReadPixels
slammo
05292003, 09:38 AM
Thanks for the suggestion, chowe6685.
Reading the zvalue from the depth buffer would give the position of the 3D point that projects onto the image pixel(x, y).
Instead, I am looking for the 3D position of the image pixel(x, y).
Thanks,
Slammo
SThomas
05292003, 01:21 PM
so what you want to do is convert from image/rectangle space to world space, right? how about something like the following:
//rectangle points in world space. assume they're initialized.
Vector3 lowerLeft, lowerRight, upperLeft, upperRight;
//assume that the image covers the entire rectangle.
int xMin = 0, xMax = imageWidth, yMin = 0, yMax = imageHeight;
//image space coords. assume they're initialized.
int x, y;
float xFactor = x/(xMaxxMin);
float yFactor = y/(yMaxyMin);
Vector3 worldPt = (1.0fxFactor)*(1.0fyFactor)*lowerLeft + (xFactor)*(1.0fyFactor)*lowerRight + (xFactor)*(yFactor)*upperRight + (1.0fxFactor)*(yFactor)*upperLeft;
there might be something wrong with the code there, but it illustrates the general idea.
Zengar
05292003, 01:32 PM
There is none. You cannot compute it.
If you are familiar with 3d space, you'll know that viewport is a plane. If you take a point on this plane(your pixel), you cannot know from where it was projected, decause you need z value. You can find a line, which projectzion onto the line is your pixel:


_____.___ plane with pixel

SThomas
05292003, 07:58 PM
slammo, what are you asking for here?
do you want the world coordinate corresponding to a particular screen coordinate? or do you have a textured rectangle in world space and you want the world coordinate of an image coordinate on the textured rectangle? if it's the latter, my previously posted code should work. if it's the former, you have to read back the projected z value from the depth buffer and call gluUnProject() with the appropriate transforms, as everyone's been saying.
slammo
05302003, 03:36 AM
Thanks for the posts, Zengar and SThomas.
Think of it this way  all unclipped points in 3D space project to the image, which is a rectangle lying somewhere in 3D space.
Each pixel in the image also has a 3D location, since it lies in the image rectangle. Note the location of a pixel is *not* the same as the location of a 3D point that projects to the pixel  that is why reading from the zbuffer will not work here.
SThomas, I believe your code would work if the 3D coordinates of the image rectangle were known. How would one compute these?
Thanks,
Slammo
Deiussum
05302003, 04:40 AM
I think you're misunderstanding what the depth buffer is, slammo. It is not the zcoordinate of the point in world coordinates. It is a value between 01 that specifies where it is proportionately between the near and far clip plane. So, you use the x and y from the screen coordinates, and the z from the depth buffer value at that point, then use gluUnproject and you should get what you need.
What everyone here has been trying to tell you is essentially correct if I'm understanding what you want correctly.
slammo
05302003, 05:38 AM
Hi, thanks for the message, Deiussum.
I do understand the depth buffer  using it to get the zvalue at the pixel (x, y), then using (x, y, z) with gluUnproject, will give the point in 3D space that projects the pixel.
However, I'm not looking for the point in 3D space that *projects* to the pixel, rather, I'm looking for the 3D coordinate of the pixel itself. The pixel exists in 3D space on a rectangle in 3D space  in this rectangle OpenGL creates the image.
Sorry if that point hasn't been clear. Can you suggest a way to compute the pixel location?
Thanks,
Slammo
Gavin
05302003, 07:08 AM
ok I went back and reread the first post of yours. That seems to be saying that lets say you have a square with a point x, y.then this square is projected into 3d, say to make a side of a cube. You now want to find the 3d position of the original x, y point?
Deiussum
05302003, 07:22 AM
I'm confused. Are you saying you have an image of a 3d scene that you are using as a texture and you want to get a the world space for a pixel of that texture? If so, there's no way you can do what you want. A 2D image does not contain any sort of depth data. You can't just take a BMP and say that this point at x,y has a depth of z, because the BMP has no concept of z.
There are exceptions to this with something like a height map. Heightmaps view the color data as specifying the height. With images that aren't heightmaps, color seldom corresponds to height/depth. I'm sure you could come up with algorithms that could find edges and approximately extrude depths, but they would be just that, approximations.
slammo
05302003, 08:06 AM
Hi Gavin and Deiussum,
Thanks for the responses... unfortunately, I guess I'm not making myself clear, since we're not on the same page. http://www.opengl.org/discussion_boards/ubb/frown.gif
I thought that maybe someone on this board knew of a quick solution to this problem but I suppose that is not the case. I will figure it out on my own and post the solution when I get it.
Thanks to all who tried to help.
Slammo
SThomas
05302003, 08:38 AM
originally posted by slammo:
SThomas, I believe your code would work if the 3D coordinates of the image rectangle were known. How would one compute these?
how are you drawing the rectangle? with glVertex() calls? if so, you can take your object space coordinates (the coordinates you supply to glVertex()) and transform them by the modelling transformation to get the world space coordinates. i'm fundamentally confused about what you're asking for though. if we knew exactly what you were asking for any of us would be able to answer your question i think. "Think of it this way  all unclipped points in 3D space project to the image, which is a rectangle lying somewhere in 3D space." i have no idea what you mean by that.
steve
jebus
05302003, 08:42 AM
hmmmmm ... i still think gluUnProject will do what you need. i can't see the difference between:
1. "...the point in 3D space that *projects* to the pixel..."
2. "...the 3D coordinate of the pixel itself..."
using the pixel's zbuffer value in gluUnProject would give you #1. i'm not sure what #2 is, cause the pixel only exists in 2D :p
jebus
slammo
05302003, 09:06 AM
Perhaps a figure will help...
Consider this image (it's the best I could find after a quick search).
http://www.esat.kuleuven.ac.be/~pollefey/tutorial/img306.gif
This is the pinhole camera model. In the figure, C is the camera center of projection, through which all imaged rays pass. R is a rectangle on the image plane where the image is formed. This is a 2D rectangle in 3D space.
In the figure, m is a pixel, that lies on the 2D image, which is on a plane in 3D space. One can refer to m using 2D pixel coordinates (x, y) in a coordinate system in the plane. What I want to know is m's location in 3D space.
Note that I'm *not* looking for M, the point in 3D space the projects to m. I could easily compute M by reading the zbuffer at (x, y) and calling gluUnproject, as many of you have suggested.
I believe determining the 3D position of m is a function of the camera location, view direction, viewport, projection, and modelview matrix. I can grind through the equations but I thought maybe someone here knew a quick solution.
Regards,
Slammo
jebus
05302003, 09:15 AM
so ... is R the near clipping plane? or a representation of the final rasterized image? or some other rectangle that happens to be in the scene and is also parallel to the screen?
jebus
Deiussum
05302003, 09:27 AM
Ok, I'm assuming you're using a perspective projection here...
Using the parameters you used in glFrustum/gluPerspective, you should easily be able to find the location of what m was before applying the MODELVIEW matrix. R is essentially the near clip plane, I think. (If I'm understanding correctly) So the z is easy. x and y would be found by knowing what your left, right, top, and bottom clip planes are. (If you use gluPerspective you can calculate those using the FOV, aspect and near clip plane. Think of those as defining a pyramid, then you should be able to figure out the trig involved.)
Once you have that location, just multiply it by the MODELVIEW matrix after any "camera transformations" you may have done.
[This message has been edited by Deiussum (edited 05302003).]
slammo
05302003, 09:27 AM
Hi Jebus,
R is a representation of the final rasterized image. Its the rectangle that represents the image being synthesized. Its position is independent of the near clipping plane.
Regards,
Slammo
slammo
05302003, 09:42 AM
Hi Deiussum,
Thanks for the message. I should have started this thread with an image, but now I think you're starting to understand me.
R is not the near clipping plane though, it is the plane on which the image is formed. R is located a distance f (where f is the focal length) away from the center of projection C. So for a given focal length, the location of R is fixed. If I change the position of the clipping planes, R does not move.
Regards,
Slammo
SThomas
05302003, 10:27 AM
here's something that i think would work. ignore everything i had previously said though... i just didn't understand what you were asking.
first you need to get a definition of the plane that you want to intersect in world space. you have the distance from the center of projection (which i assume is the eye point in standard opengl terms), and the normalized view direction. multiply the distance by the view direction and add it to the eye location. this will get you a point in the plane. the normal to the plane is the vector from the point you just computed to the eye location. now you have a complete specification of a plane.
next you can find a line segment to intersect against the plane. take your (x,y) screen coordinates and supply them to gluUnProject(), specifying a z value of 0.0 and the appropriate transformations. this'll get you the world space location of the point projected onto the near clip plane. now do the same thing, except specify a z value of 1.0. this'll get you the world space location of the screen point projected onto the back clipping plane. these two points specify your line segment. one thing to watch out for is that you don't include the modeling transformation when you call gluUnProject(), or else you'll get coordinates in object space instead of world space. just specify the viewing transformation and the unprojected points should be in world space.
now interset this line segment with the plane to get the world coordinates. code for line segmentplane intersection can be found almost anywhere, including here (http://astronomy.swin.edu.au/~pbourke/geometry/planeline/) .
eh... may as well do up some pseudo code.
Vector3 eyeLoc; //initialize to your eye location in world coordinates
Vector3 viewDir; //make sure this is normalized
float focalPtDistanceFromEye; //initialize as appropriate
//first form the plane
Vector3 ptOnPlane = eyeLoc + focalPtDistanceFromEye*viewDir;
Vector3 planeNormal = eyeLoc  ptOnPlane;
//now compute the line segment in world space
int x, y; //screen coordinates
Matrix modelView, projection;
Rect viewport;
//initialize transformations as appropriate
//make sure to only include the viewing transformation in the modelview matrix
Vector3 worldPtOnNearPlane = gluUnProject(x,y,0.0,modelView,projection,viewport );
Vector3 worldPtOnFarPlane = gluUnProject(x,y,1.0,modelView,projection,viewport );
Vector3 intersectionLocation = SegmentPlaneIntersection(worldPtOnNearPlane,worldP tOnFarPlane,planeNormal,ptOnPlane);
as before, that code hasn't been tested or anything, so be careful. hope i have the right idea here.
steve
slammo
05302003, 11:03 AM
Hi Steve,
Thanks for the idea. I think your solution may work. Perhaps I will give it a try.
Regards,
Slammo
slammo
07112003, 12:17 PM
Thanks everyone for your help. I finally figured this out  many of you were close to the right solution.
The easiest way to do it is to use gluUnproject(), but use the negative of the near clipping plane for the z coordinate.
Thanks to all who helped.
Regards,
Slammo
wimute
06162012, 07:57 AM
I found it entertaining to read this thread because of the repeated misunderstanding of the OP's question.
I'm also trying to figure out the 3d coordinates of points on the image plane, but as a relative newbie to OpenGL, to me at least the whole concept of the image plane seems elusive in the OpenGL framework!
Slammo: from your last post, can I deduce that the image plane is at the same distance from the focal / view point as the near clipping plane?
In fact, would it be correct to say that that is just a convention for gluUnproject to take the z coordinate of the pixel as (near clipping distance), but that in general, choosing the "focal length" is also to choose: the actual size of objects relative to the actual size of the pixels.
Dark Photon
06162012, 03:39 PM
Why did you resurrect a 9 year old thread? Wonder if we could consider locking threads after 6 months or so of inactivity...
wimute
07202012, 02:12 AM
Why did you resurrect a 9 year old thread? Wonder if we could consider locking threads after 6 months or so of inactivity...
Because OpenGL's API hasn't changed much in 9 years and this particular issue is discussed practically nowhere else on the net...
thokra
07202012, 05:20 AM
Because OpenGL's API hasn't changed much in 9 years[..]
:doh: (*random characters here to reach minimum*)
wimute
07212012, 03:29 AM
Hehe, ok i'll control my forum policy transgressions in future. My contributions can only improve from here!
Powered by vBulletin® Version 4.2.3 Copyright © 2016 vBulletin Solutions, Inc. All rights reserved.