LookAt and Up to Euler/AxisAngle/Quat

[Talion]

This is a pretty basic issue so sorry if this has been answered before. For objects in my world I save orientation with a lookat and up vector. I'm having trouble finding the right euler angle rotations for it, though. I can get the yaw and pitch rotations fine, but I just can't get the object's roll to match the up vector in all (or even most) cases.

Unfortunately most stuff online dealing with this sort of maneuver are talking about cameras.

Basically the technique I'm using now is to get the yaw and then pitch rotations using the usual algebra. Then I take the up and do the opposite of the rotations I just found and compare it to the local up, i.e. 0,1,0 to find the roll. This is a bit of a special case nightmare but the results seem to come out about right when I eyeball them. It just doesn't work at all when all this goes into 3 sequential glRotatef calls. The order I'm using is y axis (yaw), x axis (pitch), z axis (roll).

Any suggestions?

[shinpaughp]

Use the vector dot product to find cos where

cos theta = ( X dot Y ) / ( mag(X) * mag(Y) )

but the up angle should be perpindicular to the vector formed by the eye and center.

here is some code I wrote to get angles from a matrix... it may be helpful... or not...

good luck

Important about following code: positive x axis is to left, positive y axis is into screen, positive z axis is up.

dVec3 dMatrix::getHPR()
{
    dMatrix mat( *this );
    mat( 3, 0 ) = 0.0f;
    mat( 3, 1 ) = 0.0f;
    mat( 3, 2 ) = 0.0f;
    dVec3 hpr;
    dVec3 diry = mat * dVec3( 0.0f, 1.0f, 0.0f ), dirz = mat * dVec3( 0.0f, 0.0f, 1.0f );
    diry.normalize();
    dirz.normalize();
    if ( ABS( diry.x() ) < 1.0e-6 &amp;&amp; ABS( diry.y() ) < 1.0e-6 )
    {
        hpr.h() = 0.0f;
        hpr.p() = 90.0f * diry.z();
    }
    else
    {
        hpr.h() = RAD2DEG( atan2( diry.x(), diry.y() ) );
        float length = sqrt( diry.x() * diry.x() + diry.y() * diry.y() );
        if ( length < 1.0e-6 )
            hpr.p() = RAD2DEG( atan2( diry.z(), length ) );
        else
            hpr.p() = 90.0f * diry.z();
    }
    dVec3 temp = diry ^ dVec3( 0.0f, 0.0f, 1.0f );//x'' = y' X z ( cross product )
    temp.normalize();
    dVec3 temp2 = temp ^ diry;//z'' = x'' X y' ( cross product )
    temp.normalize();
    float dotprod = temp2 * dirz;//dot product
    if ( dotprod > 1.0f )
        dotprod = 1.0f;
    else if ( dotprod < -1.0f )
        dotprod = -1.0f;
 
    hpr.r() = RAD2DEG( acos( dotprod ) );
 
    return hpr;
}

[This message has been edited by shinpaughp (edited 02-17-2004).]