Part of the Khronos Group
OpenGL.org

The Industry's Foundation for High Performance Graphics

from games to virtual reality, mobile phones to supercomputers

Results 1 to 3 of 3

Thread: Skeleton Animation Trouble: Small error in rotation causes child bone deformation?

  1. #1
    Newbie Newbie
    Join Date
    Jul 2013
    Posts
    2

    Unhappy Skeleton Animation Trouble: Small error in rotation causes child bone deformation?

    Hello everyone. I have been trying to get a model into a pose other than its bind pose. Currently I'm using the first frame out of an animation file and trying to set the model into that initial pose.

    I believe my math equations are now correct. Transforming a single bone works perfectly (the children follow like it should). However, with Compound Transformations (where a child is transformed, and its parent has also been transformed), very young children seem to be highly deformed. (For example, the fingers of the model when the wrist, elbow, and shoulder bones have also been transformed.)

    Code :
    int targetFrame = CONST_TEST_FRAME_NUMBER;
     
        // root bone
        PMXBone   *b  = pmxInfo.bones[0];
        BoneFrame *bf = getBoneFrame(targetFrame, b->name);
     
        b->absoluteForm = b->relativeForm;
        if(bf!=NULL)
        {
            b->finalRotation = bf->quaternion;
     
            glm::vec4 homoPosition=glm::vec4(b->position + bf->translation,1.0); //position in homogeneous coordinates
            glm::vec4 localPosition=glm::rotate(b->finalRotation,homoPosition);
     
            b->relativeForm[3][0]=localPosition[0];
            b->relativeForm[3][1]=localPosition[1];
            b->relativeForm[3][2]=localPosition[2];
            b->relativeForm[3][3]=localPosition[3];
        }
        Bone[0] = b->absoluteForm * invBindPose[0];
     
     
     
        // other bones
        for (size_t i = 1; i < pmxInfo.bone_continuing_datasets; i++)
        {
            b  = pmxInfo.bones[i];
            PMXBone *parent = pmxInfo.bones[b->parentBoneIndex];
            bf = getBoneFrame(targetFrame, b->name);
     
            if(bf!=NULL)
            {            
                b->finalRotation = glm::normalize(bf->quaternion * parent->finalRotation);
     
     
                glm::vec4 homoPosition=glm::vec4(b->position + bf->translation,1.0); //position in homogeneous coordinates
                glm::vec4 localPosition=glm::rotate(glm::normalize(parent->finalRotation),homoPosition);
     
                b->relativeForm[3][0]=localPosition[0];
                b->relativeForm[3][1]=localPosition[1];
                b->relativeForm[3][2]=localPosition[2];
                b->relativeForm[3][3]=localPosition[3];
     
                b->absoluteForm = b->relativeForm * glm::toMat4(bf->quaternion) * parent->absoluteForm;
     
                Bone[i] = b->absoluteForm * invBindPose[i];
            }
            else
            {        
                b->finalRotation = glm::normalize(parent->finalRotation);
     
                glm::vec4 homoPosition=glm::vec4(b->translation,1.0); //position in homogeneous coordinates
                glm::vec4 localPosition=glm::rotate(b->finalRotation,homoPosition);
     
                b->relativeForm[3][0]=localPosition[0];
                b->relativeForm[3][1]=localPosition[1];
                b->relativeForm[3][2]=localPosition[2];
                b->relativeForm[3][3]=localPosition[3];
     
                b->absoluteForm = b->relativeForm * parent->absoluteForm;
     
                Bone[i] = b->absoluteForm * invBindPose[i];
            }
     
        }

    To help clarify the code some:

    • b->position is a glm::vec3 containing the position of the Bone in its Local/Bone-space, relative to the Parent Bone.
    • bf contains the transformation information for one bone in one frame. In other words, you must get multiple Bone Frames to get the transformation information for all the bones in one frame.
    • bf->quaternion is a 4-float glm::quat containing the rotational information for bone-frame in the animation. In other words, it contains information on how the bone must be rotated to get the model from its bind pose to the current pose.
    • Similarly, bf->translation, a glm::vec3, contains the translational information for the bone-frame. Because human skeletons are rigid, most of the values for bf->translation are set to (0,0,0).
    • relativeForm and absoluteForm refer to the Local and Global matrices of the transformed bones, respectively. Before this code snippet is run, relativeForm is simply b->position converted to a matrix, and absoluteForm is the Bind Pose matrix for the bone.
    • Bone[] is the Final Skinning Matrix, which is sent to the shader.


    Here is an image of how the model appears with this code: imgur.com/tbur5Lf

    As an extra, here is an image of a single bone in the model being transformed using this code (Instead of bf, a keyboard-controlled quaternion was used): t.co/wf38ibGoyc

    Here is a video that demonstrates my problem: youtu.be/8Cv3jMYcz64

    It took me a good two weeks to get this far, so I greatly appreciate any help. Thanks, and let me know if there is any other information I need to provide.
    Last edited by Dark Photon; 07-31-2013 at 02:33 PM.

  2. #2
    Senior Member OpenGL Guru Dark Photon's Avatar
    Join Date
    Oct 2004
    Location
    Druidia
    Posts
    3,209
    Quote Originally Posted by sn0w75 View Post
    ...Transforming a single bone works perfectly (the children follow like it should). However, with Compound Transformations ... very young children seem to be highly deformed.
    I've looked at your code, and while what you're doing bears some resemblance to what it needs to do, there are some problems. So let me make some comments. Please use this post for formula and terminology reference: Re: Need help with skeletal animation as in there I pretty well spell out what you need to do.

    First, what I infer from what you have. PMXBone.position is a joint orientation transform translation (O.trans). BoneFrame.quaternion and BoneFrame.translation are the the joint animation transform (A). PMXBone.relativeForm is the joint local transform (L), and PMXBone.absoluteForm is a joint global transform (G).

    Ok, now the overall comments/problems. I do not see anything here that functions as a joint orientation transform rotation (O.rot). This represents the orientation difference between joint spaces for a joint and its parent joint in the bind pose. You need this. Also, I think maybe the slickness of trying to keep your accumulated transforms separated into rotate and translate components is tripping you up. I'd suggest you accumulate transforms in Mat4x4 form and then later once you get it working (and lock in how it works!) consider maintaining and accumulating transforms in rotation/translation form (quat-trans).

    Now let's look at the root bone computation. First, this assumes there is no rotation in the root bone orientation transform (O.rot). This may be very reasonable for your use case, but as I said, there is no O.rot to be found in your code (...that I see). Also, it appears that you snapshot your G transform too early (i.e. b->absoluteForm in your code), as right below it you augment L (b->relativeForm) with the effects of the root animation transform (bf).

    As for the child bone logic, considering the bf != NULL case, it looks like you've stacked your L transform (b->relativeForm) and parent's G transform (parent->absoluteForm) on in the wrong order. Compare with the order of transforms in your joint final transform (F) computation right below that (Bone[i]). Also, again, you're missing the joint orientation transform's rotation component in here (O.rot), as there's no O.rot to be found anywhere in your code (again, that I see). Finally, you're only cooking the translation for the joint local transform L (b->relativeForm). While your root bone's joint local transform may very well not have a rotation component, many of your internal bone's joint local transforms will. So again I suggest you just use a Mat4x4 to composite transforms and get that right first.

    On the child bone logic for the bf == NULL case, I didn't look too much at that as it shouldn't even compile (there isn't a b->translation AFAICT; I'm guessing you meant b->position here).

    Hope this helps.

    For reference, here's your code with some of the expressions collapsed and identifiers changed so that they match the transform identifiers (e.g. O, A, G, etc.) that I use in that post I suggested you use as a reference. Note that I didn't fix anything -- just collapsed and renamed things.

    Code cpp:
    struct PMXBone
    {
      const char *name           ;
      int         parentBoneIndex;
     
      Vec3        O.trans        ; // Bone pos in local bone space (parent relative) -- Was "position"
      Quat        finalRotation  ;
     
      Mat4x4      L   ; // Joint local transform  (starts as b->O.trans) -- Was "relativeForm"
      Mat4x4      G   ; // Joint global transform (starts as bind pose) -- Was "absoluteForm"
    }
     
    struct BoneFrame
    {
      Quat A.rot ; // - Was "quaternion"
      vec3 A.trans;  // Most are 0,0,0 (because joints don't translate); Was "translation"
    };
     
     
    // root bone
    PMXBone   *b  = pmxInfo.bones[0];
    BoneFrame *bf = getBoneFrame( targetFrame, b->name );
    Mat4x4     Bone[];        // Joint final transforms
     
    b->G = b->L;   // O*A
    if ( bf )
    {
      b->finalRotation = bf->A.rot;
      b->L[3].xyz      = glm::rotate( b->finalRotation, b->O.trans + bf->A.trans );
    }
    Bone[0] = b->G * invB[0];  // O*A * O^-1
     
     
    // other bones
    for ( size_t i = 1; i < pmxInfo.bone_continuing_datasets; i++ )
    {
      b               = pmxInfo.bones[ i ];
      PMXBone *parent = pmxInfo.bones[ b->parentBoneIndex ];
      bf              = getBoneFrame( targetFrame, b->name );
     
      if( bf )
      {            
        // A = bf
     
        b->finalRotation = glm::normalize( bf->A.rot * parent->finalRotation );
     
        b->L[3].xyz      = glm::rotate( glm::normalize( parent->finalRotation ), 
                                        b->O.trans + bf->A.trans )
     
        b->G             = b->L * glm::toMat4( bf->A.rot ) * parent->G;
      }
      else
      {        
        b->finalRotation = glm::normalize( parent->finalRotation );
     
        b->L[3].xyz      = glm::rotate( b->finalRotation, 
                                        b->A.trans );
     
        b->G             = b->L * parent->G;            // G = L * G_parent
      }
      Bone[i] = b->G * invB[i];
     
    }
    Last edited by Dark Photon; 07-31-2013 at 05:25 PM.

  3. #3
    Newbie Newbie
    Join Date
    Jul 2013
    Posts
    2
    Hello, I apologize for the late response. Thanks to your advice, I was able to FINALLY get FK bone transformations working. (I had actually read through that old post of yours on skeletal animation when I was trying to get it to work before, but didn't have as much luck >.<"").

    I got to work right away on getting animation working. For the moment I'm just using simple Slerp interpolation, but it's working great: youtu.be/Qis6Ns5dgFw

    However, I'm stumped once again trying to get IK bone transformations working. I believe I've avoided getting the multiplication order mixed up like last time, but something is clearly not right.

    Obviously there are several methods for solving IK systems; The type of model I'm loading was designed with the CCD algorithm in mind, so I am trying to use CCD here.

    For starters, here is the code. It has gone under major revisions after receiving your advice, and FK bone transformation isn't completely un-related to IK bone transformation, so I've included the updated FK transformation code as well.

    Please note that bones under the effect of an IK Bone in this particular model type are rotational only (no translation).

    Code :
    void VMDMotionController::updateBoneMatrix()
    {
    	// Root bone
    	PMXBone   *b  = pmxInfo.bones[0];
     
    	// Other bones
    	for (unsigned i = 1; i < pmxInfo.bone_continuing_datasets; i++)
    	{
    		b  = pmxInfo.bones[i];
    		PMXBone *parent = pmxInfo.bones[b->parentBoneIndex];
     
    		unsigned long t0,t1;
    		glm::quat q0,q1;
    		glm::vec3 p0,p1;
     
    		if(ite_keyFrames[i] != keyFrames[i].end())
    		{
    			t0=(*ite_keyFrames[i]).frame*2; //MMD=30fps, this program=60fps, 60/30=2
    			boneRot[i] = q0 = (*ite_keyFrames[i]).rot;
    			bonePos[i] = p0 = (*ite_keyFrames[i]).trans;
     
    			if(++ite_keyFrames[i] != keyFrames[i].end())
    			{
    				t1 = (*ite_keyFrames[i]).frame*2;
    				q1 = (*ite_keyFrames[i]).rot;
    				p1 = (*ite_keyFrames[i]).trans;
     
    				float s = (float)(time - t0)/(float)(t1 - t0);	//Linear Interpolation
     
    				boneRot[i]=Slerp(q0,q1,s);
    				bonePos[i] = p0 + (p1 - p0)*s;
    				if (time != t1) --ite_keyFrames[i];
    			}
    		}
     
    		b->L = glm::translate( bonePos[i] + b->position - parent->position ) * glm::toMat4(boneRot[i]);
    		b->G = parent->G * b->L;
    		skinMatrix[i] = b->G * invBindPose[i];
    	}
     
     
     
     
    	updateIK();
     
    	glUniformMatrix4fv(Bones_loc, pmxInfo.bone_continuing_datasets, GL_FALSE, (const GLfloat*)skinMatrix);
    }
     
    void VMDMotionController::updateIK()
    {
    	// IK Bones	
    	for(unsigned b=0; b<pmxInfo.bone_continuing_datasets; ++b)
    	{
    		PMXBone *IKBone=pmxInfo.bones[b];
    		if(IKBone->IK)
    		{
    			PMXBone *targetBone=pmxInfo.bones[IKBone->IKTargetBoneIndex];
     
    			for(unsigned iterations=0; iterations<IKBone->IKLoopCount; ++iterations)
    			{
    				for(int ik=IKBone->IKLinkNum-1; ik>=0; --ik)
    				{
    					PMXIKLink *IKLink=IKBone->IKLinks[ik];
    					PMXBone *linkBone=pmxInfo.bones[IKLink->linkBoneIndex]; //Bone under the effect of the IK Bone
     
    					glm::vec3 targetVector=glm::vec3(glm::normalize(targetBone->G[3] - linkBone->G[3]));
    					glm::vec3 IKVector=glm::vec3(glm::normalize(IKBone->G[3] - linkBone->G[3]));
     
    					float cosAngle=glm::dot(targetVector,IKVector);
    					float angle=acos(cosAngle);
     
    					glm::vec3 axis=glm::cross(targetVector,IKVector);
     
    					glm::mat4 rotation=glm::rotate((float)angle*180.0f/(float)M_PI,axis);
     
    					/*if(IKLink->angleLimit)
    					{
    						glm::vec3 &lowerLimit=IKLink->lowerLimit;
    						glm::vec3 &upperLimit=IKLink->upperLimit;
    					}*/
     
    					for(int j=ik; j<IKBone->IKLinkNum; ++j)
    					{
    						PMXBone *b=pmxInfo.bones[IKBone->IKLinks[j]->linkBoneIndex]; //current link bone
    						PMXBone *parent=pmxInfo.bones[b->parentBoneIndex];
     
    						b->L = b->L * rotation;
    						b->G = parent->G * b->L;
     
    						skinMatrix[IKBone->IKLinks[j]->linkBoneIndex] = b->G * invBindPose[IKBone->IKLinks[j]->linkBoneIndex];
    					}
     
    						//NOTE: Left out error-tolerance check
    				}
    			}
    		}
    	}
    }

    The bone transformation functions are now encapsulated in a Motion Controller class. The Motion Controller holds and maintains a set of local translation and rotation variables in the boneRot and bonePos glm::quat and glm::vec3 vectors.

    These are then calculated by interpolating between two BoneFrame's quats and translations, which are chosen based on the current time in the animation. (The direct use of BoneFrame/bf in my previous code has been replaced with vector<list<BoneFrame>> keyframes, which holds a list of BoneFrames for each bone, and vector<list<BoneFrame>::iterator> ite_keyFrames, an iterator for the keyframes.) Once calculated, boneRot[i] and bonePos[i] are used for calculating the Local and Global (L and G) matrices of the bone.)

    (b->finalRotation has also been removed in favor holding the local rot/pos transformations in boneRot[]).

    ^^^This is all just an explanation for updateBoneMatrix(), my updated FK transformation function. The trouble is in updateIK(), my IK Bone transformation function.

    The result of un-commenting updateIK() in updateBoneMatrix() is along these lines:
    i.imgur.com/Z0dWPpp.png

    As the animation starts, the legs begin moving all over the place...Which is clearly not what we want. My suspicion is that the way targetVector and IKVector are being created is incorrect, but again, I am new to this, so I'm not very sure.

    To explain the code a bit...linkBone, which I believe may be a non-typical naming, is the name of a given Child Bone under the influence of the current IKBone. For some reason these linkBones are given in normal parent-to-child order (ex: hip -> foot), so I am traversing through the IKLinks in reverse (foot -> hip).

    targetVector is the vector going from the targetBone to the linkBone, and IKVector is the vector going from the IKBone to the linkBone.

    cosAngle is the dot product of the two vectors. I believe there is some kind of check you need to add with this dot product to prevent close-to-zero situations, but I'm at a bit of a loss how to go about it.

    I'm also unsure how to make proper use of the IKLink's lower and upperLimit, because these are given as a 3-float vector, not a quaternion. I'm afraid there may be complications if I convert my resulting 4x4 rotation matrix or quaternion to Euler angles to check the lower and upper rotation limits.

    In general, I feel much less familiar with the mathematics and algorithms involved with IK than I did with FK. Perhaps it's because I haven't spent two full weeks on this problem yet, but I'm struggling to grasp the concepts of solving IK systems and implementing the CCD algorithm.

    Again, I really thank you for your previous help. I was finally able to get out of a full 2-week slump, which was absolutely miserable. I'm hoping you can perhaps give me a few more pointers here so that I can get a firm grasp on bone transformations. I look forward to hearing your response.
    Last edited by sn0w75; 08-03-2013 at 03:43 PM.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •