OpenGL pipeline

[Vieinco]

I try to reconstruct some 3d scene from real world.
From camera calibration by chessboard i had: Matrix(R|T) - transformation matrix (rotation|translation), and internal camera paramenters (focal, cx,cy) - matrix K. And now i'll try to reconstruct chessboard in OpenGL.

Step #1: i perfomed projection calculation by matrix operation (using pure C++)
1. M = K*(R|T)
2. V = M*ObjectModelPoint
3. ScreenX = v.x/v.z; ScreenY = v.y/v.z;
4. Draw point on ScreenX, ScreenY coordinates.
And i had correct recunstruction (visual prove)...

Step #2: I try to translate visualization process via OpenGL.
Step #2.1 If i try to perform some calculation sequence like OpenGL pipeline (according with documentation), i would have code like this
1. (R|T)*ObjectModelPoint = eyePoint
2. ProjectionMatrix * eyePoint = clipPoint
// <= glFrustum( cx, cx - w, cy, cy - h, focal, 3*focal);
// glGetFloatv(GL_PROJECTION_MATRIX, ProjectionMatrix); to get ProjectionMatrix
3. clip all points out of -1,1
4. ndcPoint = clipPoint / clipPoint.w
5. ViewPoint = ViewportMatrix * ndcPoint
// ViewportMatrix - matrix equevalent to glViewport operation
6. draw point at ViewPoint.x, ViewPoint.y
And I have correct visualization (and check all transformation formula manualy)

Step #2.2(!!!) I try to use OpenGL pipeline
1. glViewport(0, 0, w, h);

2. glMatrixMode(GL_PROJECTION);
glLoadIdentity();
glFrustum( cx , cx - w, cy, cy - h, z, 3*z);
// glGetFloatv return the same matrix like ProjectionMatrix in step 2.1

3. glMatrixMode(GL_MODELVIEW);
glLoadIdentity();
gluLookAt( 0.0, 0.0, -focal, 0.0, 0.0, 0.0, 0.0, -1.0, 0.0);
glMultMatrixf(Matrix(R|T));

4. set ObjectModelPoints // glBegin(GL_QUADS); glVertex3d(....)....

BUT in this step i have incorrect visualization :sick:
Position and orientation of model looks like a correct (if a zoom it), but scale factor is toooooo smaaalll (i have aproximatly 5 on 5 pixels object =)) )

Where I fail?! or may be OpenGL pipeline do some also?....
// I can give sources of test project

[Vieinco]

Sorry, it's my fail... when i user matrix projection - ti's back projection...
but in openGL i use forward projection, therefore i had very small visualisation! =)

But i have new question... how can i perform back projection in OpenGL?

[david_f_knight]

I have a question:

In step #2.2 part 3, you have:

glMultMatrixf(Matrix(R|T));

How are you performing the R|T matrix operation?

OpenGL by default stores rectangular arrays in column major order, as opposed to C++ storing them in row major order. So, the R|T operation needs to be different depending on whether you are doing things in C++ like you did in step #1, or doing things in OpenGL like you are in step #2.2.

If you are implementing R|T by storing the T vector in the R matrix with C++ code, then to use R|T with OpenGL matrix functions you need to compute R|T by doing something like:

R[12] = T[0];
R[13] = T[1];
R[14] = T[2];

and not something like:

R[3] = T[0];
R[7] = T[1];
R[11] = T[2];

Assuming R is holding a 4x4 matrix and T is holding a 3 or 4 elements vector.

If you have it wrong, rather than T translating your vertices, it will be changing the W coordinate, which amounts to an inverse scale factor when the homogeneous coordinates are converted to Euclidean coordinates by dividing X, Y, and Z by W in the OpenGL pipeline right before rasterization (the "perspective division"). That could make your scale factor seem "toooooo smaaalll!"