Skeleton Animation Trouble: Small error in rotation causes child bone deformation?

Hello everyone. I have been trying to get a model into a pose other than its bind pose. Currently I’m using the first frame out of an animation file and trying to set the model into that initial pose.

I believe my math equations are now correct. Transforming a single bone works perfectly (the children follow like it should). However, with Compound Transformations (where a child is transformed, and its parent has also been transformed), very young children seem to be highly deformed. (For example, the fingers of the model when the wrist, elbow, and shoulder bones have also been transformed.)

int targetFrame = CONST_TEST_FRAME_NUMBER;
    
    // root bone
    PMXBone   *b  = pmxInfo.bones[0];
    BoneFrame *bf = getBoneFrame(targetFrame, b->name);

    b->absoluteForm = b->relativeForm;
    if(bf!=NULL)
    {
        b->finalRotation = bf->quaternion;
        
        glm::vec4 homoPosition=glm::vec4(b->position + bf->translation,1.0); //position in homogeneous coordinates
        glm::vec4 localPosition=glm::rotate(b->finalRotation,homoPosition);
        
        b->relativeForm[3][0]=localPosition[0];
        b->relativeForm[3][1]=localPosition[1];
        b->relativeForm[3][2]=localPosition[2];
        b->relativeForm[3][3]=localPosition[3];
    }
    Bone[0] = b->absoluteForm * invBindPose[0];
    
    
    
    // other bones
    for (size_t i = 1; i < pmxInfo.bone_continuing_datasets; i++)
    {
        b  = pmxInfo.bones[i];
        PMXBone *parent = pmxInfo.bones[b->parentBoneIndex];
        bf = getBoneFrame(targetFrame, b->name);

        if(bf!=NULL)
        {            
            b->finalRotation = glm::normalize(bf->quaternion * parent->finalRotation);
            
            
            glm::vec4 homoPosition=glm::vec4(b->position + bf->translation,1.0); //position in homogeneous coordinates
            glm::vec4 localPosition=glm::rotate(glm::normalize(parent->finalRotation),homoPosition);
            
            b->relativeForm[3][0]=localPosition[0];
            b->relativeForm[3][1]=localPosition[1];
            b->relativeForm[3][2]=localPosition[2];
            b->relativeForm[3][3]=localPosition[3];
            
            b->absoluteForm = b->relativeForm * glm::toMat4(bf->quaternion) * parent->absoluteForm;
            
            Bone[i] = b->absoluteForm * invBindPose[i];
        }
        else
        {        
            b->finalRotation = glm::normalize(parent->finalRotation);
            
            glm::vec4 homoPosition=glm::vec4(b->translation,1.0); //position in homogeneous coordinates
            glm::vec4 localPosition=glm::rotate(b->finalRotation,homoPosition);
            
            b->relativeForm[3][0]=localPosition[0];
            b->relativeForm[3][1]=localPosition[1];
            b->relativeForm[3][2]=localPosition[2];
            b->relativeForm[3][3]=localPosition[3];
            
            b->absoluteForm = b->relativeForm * parent->absoluteForm;
            
            Bone[i] = b->absoluteForm * invBindPose[i];
        }
            
    }

To help clarify the code some:

[ul]
[li]b->position is a glm::vec3 containing the position of the Bone in its Local/Bone-space, relative to the Parent Bone. [/li][li]bf contains the transformation information for one bone in one frame. In other words, you must get multiple Bone Frames to get the transformation information for all the bones in one frame. [/li][li]bf->quaternion is a 4-float glm::quat containing the rotational information for bone-frame in the animation. In other words, it contains information on how the bone must be rotated to get the model from its bind pose to the current pose. [/li][li]Similarly, bf->translation, a glm::vec3, contains the translational information for the bone-frame. Because human skeletons are rigid, most of the values for bf->translation are set to (0,0,0). [/li][li]relativeForm and absoluteForm refer to the Local and Global matrices of the transformed bones, respectively. Before this code snippet is run, relativeForm is simply b->position converted to a matrix, and absoluteForm is the Bind Pose matrix for the bone. [/li][li]Bone[] is the Final Skinning Matrix, which is sent to the shader. [/li][/ul]

Here is an image of how the model appears with this code: imgur.com/tbur5Lf

As an extra, here is an image of a single bone in the model being transformed using this code (Instead of bf, a keyboard-controlled quaternion was used): t.co/wf38ibGoyc

Here is a video that demonstrates my problem: youtu.be/8Cv3jMYcz64

It took me a good two weeks to get this far, so I greatly appreciate any help. Thanks, and let me know if there is any other information I need to provide.

I’ve looked at your code, and while what you’re doing bears some resemblance to what it needs to do, there are some problems. So let me make some comments. Please use this post for formula and terminology reference: Re: Need help with skeletal animation as in there I pretty well spell out what you need to do.

First, what I infer from what you have. PMXBone.position is a joint orientation transform translation (O.trans). BoneFrame.quaternion and BoneFrame.translation are the the joint animation transform (A). PMXBone.relativeForm is the joint local transform (L), and PMXBone.absoluteForm is a joint global transform (G).

Ok, now the overall comments/problems. I do not see anything here that functions as a joint orientation transform rotation (O.rot). This represents the orientation difference between joint spaces for a joint and its parent joint in the bind pose. You need this. Also, I think maybe the slickness of trying to keep your accumulated transforms separated into rotate and translate components is tripping you up. I’d suggest you accumulate transforms in Mat4x4 form and then later once you get it working (and lock in how it works!) consider maintaining and accumulating transforms in rotation/translation form (quat-trans).

Now let’s look at the root bone computation. First, this assumes there is no rotation in the root bone orientation transform (O.rot). This may be very reasonable for your use case, but as I said, there is no O.rot to be found in your code (…that I see). Also, it appears that you snapshot your G transform too early (i.e. b->absoluteForm in your code), as right below it you augment L (b->relativeForm) with the effects of the root animation transform (bf).

As for the child bone logic, considering the bf != NULL case, it looks like you’ve stacked your L transform (b->relativeForm) and parent’s G transform (parent->absoluteForm) on in the wrong order. Compare with the order of transforms in your joint final transform (F) computation right below that (Bone[i]). Also, again, you’re missing the joint orientation transform’s rotation component in here (O.rot), as there’s no O.rot to be found anywhere in your code (again, that I see). Finally, you’re only cooking the translation for the joint local transform L (b->relativeForm). While your root bone’s joint local transform may very well not have a rotation component, many of your internal bone’s joint local transforms will. So again I suggest you just use a Mat4x4 to composite transforms and get that right first.

On the child bone logic for the bf == NULL case, I didn’t look too much at that as it shouldn’t even compile (there isn’t a b->translation AFAICT; I’m guessing you meant b->position here).

Hope this helps.

For reference, here’s your code with some of the expressions collapsed and identifiers changed so that they match the transform identifiers (e.g. O, A, G, etc.) that I use in that post I suggested you use as a reference. Note that I didn’t fix anything – just collapsed and renamed things.


struct PMXBone
{
  const char *name           ;
  int         parentBoneIndex;

  Vec3        O.trans        ; // Bone pos in local bone space (parent relative) -- Was "position"
  Quat        finalRotation  ;

  Mat4x4      L   ; // Joint local transform  (starts as b->O.trans) -- Was "relativeForm"
  Mat4x4      G   ; // Joint global transform (starts as bind pose) -- Was "absoluteForm"
}

struct BoneFrame
{
  Quat A.rot ; // - Was "quaternion"
  vec3 A.trans;  // Most are 0,0,0 (because joints don't translate); Was "translation"
};
  
 
// root bone
PMXBone   *b  = pmxInfo.bones[0];
BoneFrame *bf = getBoneFrame( targetFrame, b->name );
Mat4x4     Bone[];        // Joint final transforms

b->G = b->L;   // O*A
if ( bf )
{
  b->finalRotation = bf->A.rot;
  b->L[3].xyz      = glm::rotate( b->finalRotation, b->O.trans + bf->A.trans );
}
Bone[0] = b->G * invB[0];  // O*A * O^-1


// other bones
for ( size_t i = 1; i < pmxInfo.bone_continuing_datasets; i++ )
{
  b               = pmxInfo.bones[ i ];
  PMXBone *parent = pmxInfo.bones[ b->parentBoneIndex ];
  bf              = getBoneFrame( targetFrame, b->name );

  if( bf )
  {            
    // A = bf

    b->finalRotation = glm::normalize( bf->A.rot * parent->finalRotation );

    b->L[3].xyz      = glm::rotate( glm::normalize( parent->finalRotation ), 
                                    b->O.trans + bf->A.trans )

    b->G             = b->L * glm::toMat4( bf->A.rot ) * parent->G;
  }
  else
  {        
    b->finalRotation = glm::normalize( parent->finalRotation );

    b->L[3].xyz      = glm::rotate( b->finalRotation, 
                                    b->A.trans );

    b->G             = b->L * parent->G;            // G = L * G_parent
  }
  Bone[i] = b->G * invB[i];

}

Hello, I apologize for the late response. Thanks to your advice, I was able to FINALLY get FK bone transformations working. (I had actually read through that old post of yours on skeletal animation when I was trying to get it to work before, but didn’t have as much luck >.<“”).

I got to work right away on getting animation working. For the moment I’m just using simple Slerp interpolation, but it’s working great: youtu.be/Qis6Ns5dgFw

However, I’m stumped once again trying to get IK bone transformations working. I believe I’ve avoided getting the multiplication order mixed up like last time, but something is clearly not right.

Obviously there are several methods for solving IK systems; The type of model I’m loading was designed with the CCD algorithm in mind, so I am trying to use CCD here.

For starters, here is the code. It has gone under major revisions after receiving your advice, and FK bone transformation isn’t completely un-related to IK bone transformation, so I’ve included the updated FK transformation code as well.

Please note that bones under the effect of an IK Bone in this particular model type are rotational only (no translation).


void VMDMotionController::updateBoneMatrix()
{
	// Root bone
	PMXBone   *b  = pmxInfo.bones[0];
	
	// Other bones
	for (unsigned i = 1; i < pmxInfo.bone_continuing_datasets; i++)
	{
		b  = pmxInfo.bones[i];
		PMXBone *parent = pmxInfo.bones[b->parentBoneIndex];
		
		unsigned long t0,t1;
		glm::quat q0,q1;
		glm::vec3 p0,p1;
		
		if(ite_keyFrames[i] != keyFrames[i].end())
		{
			t0=(*ite_keyFrames[i]).frame*2; //MMD=30fps, this program=60fps, 60/30=2
			boneRot[i] = q0 = (*ite_keyFrames[i]).rot;
			bonePos[i] = p0 = (*ite_keyFrames[i]).trans;
			
			if(++ite_keyFrames[i] != keyFrames[i].end())
			{
				t1 = (*ite_keyFrames[i]).frame*2;
				q1 = (*ite_keyFrames[i]).rot;
				p1 = (*ite_keyFrames[i]).trans;
				
				float s = (float)(time - t0)/(float)(t1 - t0);	//Linear Interpolation
				
				boneRot[i]=Slerp(q0,q1,s);
				bonePos[i] = p0 + (p1 - p0)*s;
				if (time != t1) --ite_keyFrames[i];
			}
		}

		b->L = glm::translate( bonePos[i] + b->position - parent->position ) * glm::toMat4(boneRot[i]);
		b->G = parent->G * b->L;
		skinMatrix[i] = b->G * invBindPose[i];
	}
			

	
	
	updateIK();
	
	glUniformMatrix4fv(Bones_loc, pmxInfo.bone_continuing_datasets, GL_FALSE, (const GLfloat*)skinMatrix);
}

void VMDMotionController::updateIK()
{
	// IK Bones	
	for(unsigned b=0; b<pmxInfo.bone_continuing_datasets; ++b)
	{
		PMXBone *IKBone=pmxInfo.bones[b];
		if(IKBone->IK)
		{
			PMXBone *targetBone=pmxInfo.bones[IKBone->IKTargetBoneIndex];
			
			for(unsigned iterations=0; iterations<IKBone->IKLoopCount; ++iterations)
			{
				for(int ik=IKBone->IKLinkNum-1; ik>=0; --ik)
				{
					PMXIKLink *IKLink=IKBone->IKLinks[ik];
					PMXBone *linkBone=pmxInfo.bones[IKLink->linkBoneIndex]; //Bone under the effect of the IK Bone
					
					glm::vec3 targetVector=glm::vec3(glm::normalize(targetBone->G[3] - linkBone->G[3]));
					glm::vec3 IKVector=glm::vec3(glm::normalize(IKBone->G[3] - linkBone->G[3]));
					
					float cosAngle=glm::dot(targetVector,IKVector);
					float angle=acos(cosAngle);
					
					glm::vec3 axis=glm::cross(targetVector,IKVector);
						
					glm::mat4 rotation=glm::rotate((float)angle*180.0f/(float)M_PI,axis);
					
					/*if(IKLink->angleLimit)
					{
						glm::vec3 &lowerLimit=IKLink->lowerLimit;
						glm::vec3 &upperLimit=IKLink->upperLimit;
					}*/
						
					for(int j=ik; j<IKBone->IKLinkNum; ++j)
					{
						PMXBone *b=pmxInfo.bones[IKBone->IKLinks[j]->linkBoneIndex]; //current link bone
						PMXBone *parent=pmxInfo.bones[b->parentBoneIndex];

						b->L = b->L * rotation;
						b->G = parent->G * b->L;
						
						skinMatrix[IKBone->IKLinks[j]->linkBoneIndex] = b->G * invBindPose[IKBone->IKLinks[j]->linkBoneIndex];
					}
				
						//NOTE: Left out error-tolerance check
				}
			}
		}
	}
}

The bone transformation functions are now encapsulated in a Motion Controller class. The Motion Controller holds and maintains a set of local translation and rotation variables in the boneRot and bonePos glm::quat and glm::vec3 vectors.

These are then calculated by interpolating between two BoneFrame’s quats and translations, which are chosen based on the current time in the animation. (The direct use of BoneFrame/bf in my previous code has been replaced with vector<list<BoneFrame>> keyframes, which holds a list of BoneFrames for each bone, and vector<list<BoneFrame>::iterator> ite_keyFrames, an iterator for the keyframes.) Once calculated, boneRot[i] and bonePos[i] are used for calculating the Local and Global (L and G) matrices of the bone.)

(b->finalRotation has also been removed in favor holding the local rot/pos transformations in boneRot[]).

^^^This is all just an explanation for updateBoneMatrix(), my updated FK transformation function. The trouble is in updateIK(), my IK Bone transformation function.

The result of un-commenting updateIK() in updateBoneMatrix() is along these lines:
i.imgur.com/Z0dWPpp.png

As the animation starts, the legs begin moving all over the place…Which is clearly not what we want. My suspicion is that the way targetVector and IKVector are being created is incorrect, but again, I am new to this, so I’m not very sure.

To explain the code a bit…linkBone, which I believe may be a non-typical naming, is the name of a given Child Bone under the influence of the current IKBone. For some reason these linkBones are given in normal parent-to-child order (ex: hip -> foot), so I am traversing through the IKLinks in reverse (foot -> hip).

targetVector is the vector going from the targetBone to the linkBone, and IKVector is the vector going from the IKBone to the linkBone.

cosAngle is the dot product of the two vectors. I believe there is some kind of check you need to add with this dot product to prevent close-to-zero situations, but I’m at a bit of a loss how to go about it.

I’m also unsure how to make proper use of the IKLink’s lower and upperLimit, because these are given as a 3-float vector, not a quaternion. I’m afraid there may be complications if I convert my resulting 4x4 rotation matrix or quaternion to Euler angles to check the lower and upper rotation limits.

In general, I feel much less familiar with the mathematics and algorithms involved with IK than I did with FK. Perhaps it’s because I haven’t spent two full weeks on this problem yet, but I’m struggling to grasp the concepts of solving IK systems and implementing the CCD algorithm.

Again, I really thank you for your previous help. I was finally able to get out of a full 2-week slump, which was absolutely miserable. I’m hoping you can perhaps give me a few more pointers here so that I can get a firm grasp on bone transformations. I look forward to hearing your response.