Confusions with skeletal animations

Hi guys, I’m making my engine and now it’s time for animations. I learned a lot of things on the internet and on this site too, I think I have understood the overall principles, but I can’t figure out some things.
I made an fbx loader, no problem with the meshes, but if I think in terms of animations, apparently I don’t have problems to extract data, but I don’t know what kind of information to extract from the file.
Every tutorial or disussion, speaks about a bind pose. Many of these use the concept of local matrix,local transform, global transform, parent transform, …
I have some problems with this local matrix. I have a number of joints, every joint, except for the root joint, has a parent joint. For what I understood, the parent of the root joint is the world space. Now I have to think about joint spaces, I have to transform the joints in relation with their parents with something like:


    Joint.GlobalTransform = Joint.Parent.GlobalTransform * Joint.LocalTransform;

I have to do this at the beginning for make the bind pose inverse matrices and on the engine cycles for the keyframes interpolation, is this correct?
If this is correct, my problem is calculate the LocalTransform, because for every joint in the file(fbx), I can evaluate the global and the local transformation matrices at a given time for the animations and a “pose matrix” if any, for the bind pose.
If I have a normal object in my engine, this has a position and rotations, I put this object in the “scene” by rotating it and then translating it, so I think I have to do with the joints and my questions are: is this transformation the LocalTrasform I need?
If not, what positions and rotations need I, for relate the joint to its parent?
Because in the file, the global transformations are the same as the world reference in the modeler I use for models, and the locals are transformations relative to the parents, where the origin of the joint is the parent.
I miss something in the way because I tried with both globals and locals to get the correct joints positions and orientations, ignoring the skinning part for now, but are always wrong, only the root joint seems to move correct, the quaternion interpolations works and the timing too.
Maybe you can help me.
Thank you in advance!!!

[QUOTE=Duilio;1283537]
Now I have to think about joint spaces, I have to transform the joints in relation with their parents with something like:


    Joint.GlobalTransform = Joint.Parent.GlobalTransform * Joint.LocalTransform;

I have to do this at the beginning for make the bind pose inverse matrices and on the engine cycles for the keyframes interpolation, is this correct?
Yes.

The object’s transformation is the parent of the root node’s transformation. For the root node, the expression


    Joint.GlobalTransform = Joint.Parent.GlobalTransform * Joint.LocalTransform;

isn’t valid, because Joint.Parent won’t exist for the root node. So you’d use the object’s transformation instead of Joint.Parent.GlobalTransform.

Each vertex needs to be transformed from bind-pose space to animated-pose world space. If you aren’t using any kind of blending, you’d transform by the inverse of the bind-pose global transformation for the vertex’ joint then by the animated global transformation. Because the first step doesn’t change with the animation, you can do that part on load.

If you’re using linear-blend skinning, you can either perform that process for each joint to which a vertex is attached and blend the results based upon the weights, or blend the transformations (from bind pose to animated pose) for each joint and transform the vertex by the blended transformation. The former requires less computation (blending vectors is cheaper than blending transformations), but if you’re transforming from bind-pose space to joint space on load, you need to store multiple joint-space positions for each vertex.

If you’re using dual-quaternion skinning, you have to blend the transformations (from bind pose to animated pose) and transform each vertex by the blended transformation.

If you aren’t getting correct results, the first thing to do is to add an option to draw the animated skeleton based upon the computed global transformations for each joint. If the skeleton is incorrect, you aren’t calculating the transformations correctly. If the skeleton is correct but the mesh isn’t following it, you aren’t using the transformations correctly.

GClements has already given you some good tips. Here are a few more.

Or, more generally, the parent of the root joint is the OBJECT-SPACE in which the skeletally-animated model instance is defined. That instance has a MODELING transform which will take its OBJECT-SPACE to WORLD-SPACE. This is typically stored completely separately from the “skinning matrices”, for instancing and other reasons.

If I have a normal object in my engine, this has a position and rotations, I put this object in the “scene” by rotating it and then translating it,…

Yes. You apply its MODELING transform to transform it from it’s local OBJECT-SPACE to WORLD-SPACE (which is the space in-which all scene objects are positioned).

…so I think I have to do with the joints and my questions are: is this transformation the LocalTrasform I need?

It’s hard to tell, but it sounds like you think that positioning the skeletal character instance in the world is somehow different from any other object. In the simplest case, it’s not – it’s exactly the same.

Where the skeletal magic comes is in computing the positions of the vertices rendered within that character’s OBJECT-SPACE. One the skeletal model’s verticies are positioned (“skinned”) within its OBJECT-SPACE, you just multiply on the instance’s MODELING transform as usual to take it to WORLD-SPACE.

Definitely follow GClements tip to just render the joint skeletons for starters using your computed skinning matrices. I’d drop any kinds of animation blending/interpolation you might be doing (keyframe interpolation, track blending, etc.) and just render a single keyframe of single animation track per frame for each skeletal instance. With this rendering approach, you’d precompute your skinning transforms (aka joint final transforms) for all joints for all keyframes for all animation tracks, upload them to the GPU once, and use them to position your skeletal instance based on the animation time base. Then just take the skinned result (in posed OBJECT-SPACE) and transform to WORLD-SPACE by applying the whole instance’s MODELING transform as you would for any model.

If you’re semi-confused about the various skeletal transforms and spaces (been there; I feel your pain!), you might find these two posts useful: POST #1, POST #2.

Thank you guys for the quick answers, so I’m not so messed up, also because if a take some cubes in my scene and treat them like joints, it works, if I rotate the parent the childs move correctly, but not with the skeletal informations got in the file, only with a simple code I made for rotate the joints with keyboard buttons.
Maybe my problem is the data extracted from the file. I’ll try again rewriting all the skeletal part.

If you’re using linear-blend skinning, you can either perform that process for each joint to which a vertex is attached and blend the results based upon the weights, or blend the transformations (from bind pose to animated pose) for each joint and transform the vertex by the blended transformation. The former requires less computation (blending vectors is cheaper than blending transformations), but if you’re transforming from bind-pose space to joint space on load, you need to store multiple joint-space positions for each vertex.

I didn’t understand everything(because my english), but I have tried to interpolate keyframes with slerp for quaternion rotations but only the root seems to work correctly, a rotation from 0° to 45° and back to 0°.
For the skinning, from the file I exctract 4 weights and 4 joints per vertex that goes into the vertex buffer, then I send matrices as uniforms to the shader for transformations.
I made another vertex buffer for rendering the skeleton with only the joints global positions, parent-child coupled and rendered as lines.

Ok, debugging the engine, rotations and translations seem to work, I have the joints in the correct position, anyway I don’t think is correct because I have problems with the inverses.
I have a simple arm with 3 joints, straight on the x+ direction, something like:
root

0,0,0

joint1

25,0,0

joint2

50,0,0

The arm is stored in a file like this:

-1 root 0.0 0.0 0.0 / 0.0 90.0 0.0
0 joint1 0.0 0.0 25.0 / 0.0 0.0 0.0
1 joint2 0.0 0.0 25.0 / 0.0 0.0 0.0

where the first number is the parent joint index, then the name, the first triplet is the local position and the second triplet the rotations.
I thought that a joint with no rotations, points to the z+ direction, so for an arm straight on the x+ direction I rotate only the root by 90°.
For the global transforms I used something like:


if(!Joint.Parent)
    Joint.GlobalTransform = Joint.LocalTransform;
else
    Joint.GlobalTransform = Joint.Parent.GlobalTransform * Joint.LocalTransform;

In the loading part I get the inverses by inverting the global transformations of the bind pose with this:


Inverses[Joint.Index] = inverse(Joint.GlobalTransform);

Now I have some vertices scattered around this arm, stored in model space coordinates. Suppose a vertex with coordinates:

30.0, 2.5, 3.4

skinned to the second joint, the middle one, if I multiply the inverse of the second joint with the vertex, shouldn’t I get this coordinates:

5.0, 2.5, 3.4

?
I’m so sorry because I feel a little bit stupid now and I don’t want to annoy anyone, specially with my english!:o

Ok, let’s make sure we’re on the same page…

  1. Is this a description of the positioning of joints in the “bind pose”?
  2. And each line provides the joint’s “orientation transform” (O), positioning it within its parent joint’s space in the bind pose?
  3. And rotation is applied first, then translate ( (T*R)*v1 = v2, to use OpenGL’s vector-on-the-right convention )?
  4. And we’re using the math convention of positive rotation around an axis follows the right-hand rule?
  5. And we’re using a right-handed coordinate frame?

Now I have some vertices scattered around this arm, stored in model space coordinates. Suppose a vertex with coordinates:

30.0,  2.5, 3.4

skinned to the second joint, the middle one, if I multiply the inverse of the second joint with the vertex, shouldn’t I get this coordinates:

5.0, 2.5, 3.4

?

Let’s call “root” “joint 0”, and use “R” = rotation, “T” = translation.

Ok, so for joint orientation (O) transforms we have:

joint 0: O0 = (R = +90Y, T = 0)
joint 1: O1 = (R = 0, T = +25Z )

Given this, let’s compute the bind pose transform (B) for joint 1: B1. Again, using OpenGL’s vector-on-the-right convention, that’s:

B1 = O0O1 = (rot = +90Y, trans = 0) * (R = 0, T = +25Z) = (R = +90Y, T = +25X)

Stated more simply, we’re just rotating (0,0,25) around the +Y axis by 90 degrees to get the new translation (25,0,0).

So what’s the inverse bind pose transform (B-1) for joint 1: B1-1?

B1-1 = (O0O1)-1 = (R = -90Y, T = -25Z)

Checking this intuitively, we’re just taking B’s original translate (25,0,0), negating it, and running it through the inverse rotation of -90Y, and out pops (0,0,-25).

So now let’s take those OBJECT-space (aka model space) coordinates you provided and run them through the inverse bind pose for joint 1 to see what coordinates we get in joint 1’s local space in the bind pose:

B1-1*v = (R = -90Y, T = -25Z) * [30.0, 2.5, 3.4] = [-3.4, 2.5, 5]

…assuming I didn’t make any mistakes above or mis-infer your assumptions.

Let me address the rest of your message in another post.

I thought that a joint with no rotations, points to the z+ direction, …

Joints are just coordinate frames (aka frames-of-reference). Like all frames, they have an orientation for the axis vectors (a rotation) and an origin (a translation) defined relative to some parent coordinate frame.

There’s no firm convention in skeletal animation for how “joint space” (a joint’s local space) is oriented by default, or how “bones” (connections between this joint and other connected joints) are oriented within that local joint’s space by default. Different 3D modeling packages have different conventions here. However, as much as possible you don’t want your software to care about these conventions.

For the global transforms I used something like:


if(!Joint.Parent)
    Joint.GlobalTransform = Joint.LocalTransform;
else
    Joint.GlobalTransform = Joint.Parent.GlobalTransform * Joint.LocalTransform;

Math-wise this looks fine. Though since we’re talking about the bind pose, I would suggest different terminology.

Often times,

  • LOCAL (L) transform means the joint-to-parent joint transform in the CURRENT POSE, and
  • GLOBAL (G) transform means the joint-to-root joint transform in the CURRENT POSE.

Since you haven’t even described transforms for the CURRENT POSE, we’re implicitly talking about the BIND POSE here.

In that case, I might suggest alternate terminology:

  • ORIENTATION (O) transform means the joint-to-parent joint transform in the BIND POSE, and
  • BIND POSE (B) transform means the joint-to-root joint transform in the BIND POSE.

With distinct terminology for transforms in the BIND POSE vs. the CURRENT POSE, it’s easier to talk about and harder to get confused. It also completely explains what the “BIND POSE” and “INVERSE BIND POSE” transforms are. The “INVERSE BIND POSE” transform is one of the few constants in terminology across skeletal animation literature. So that’s useful.

Ultimately though, it’s up to you what you call it. That’s just my suggestion based on the reading I’ve done.

  1. Is this a description of the positioning of joints in the “bind pose”?
  2. And each line provides the joint’s “orientation transform” (O), positioning it within its parent joint’s space in the bind pose?
  3. And rotation is applied first, then translate ( (T*R)*v1 = v2, to use OpenGL’s vector-on-the-right convention )?
  4. And we’re using the math convention of positive rotation around an axis follows the right-hand rule?
  5. And we’re using a right-handed coordinate frame?

Yes, your suppositions are right, I’m sorry I didn’t specify!
I wrote again the animation part, as said above I created a vertex buffer populated of vertices rappresenting the joints and rendered as lines.
I created an array of 6 vertices skinned to the skeleton, with only a position, a color and a joint index, the weight is 1 so I don’t need to memorize it.
vertex 0: pos:

0,0,0

joint

0

vertex 1: pos:

25,0,0

joint

0

vertex 2: pos:

25,0,0

joint

1

vertex 3: pos:

50,0,0

joint

1

vertex 4: pos:

50,0,0

joint

2

vertex 5: pos:

75,0,0

joint

2

I also changed the above bind pose removing rotations:


-1 joint0 0.0 0.0 0.0 / 0.0 0.0 0.0
0 joint1 25.0 0.0 0.0 / 0.0 0.0 0.0
1 joint2 25.0 0.0 .0 / 0.0 0.0 0.0

again where the first is the parent index, then the name, the translations and the rotations.
I kept the array after the VB and VAO creation and tried to transform the vertices the old way(I mean without vertex buffers), it works!!!
I have actually the vertices in the correct position.
Unfortunately doesn’t work with the vertex shader even if I pass to it the same matrices and exept for the projection and view transformation the calculation is the same.:confused:

Clearly something’s different. I suspect you just need to look carefully at your shader inputs and computations to determine what is different.

Hi guys thanks for the support, I tried to understand what is the problem without success, so I decided to write a simple C test program, render only the skeleton as lines and nothing else.
I made a file with the bind pose joint positions and rotations:


3
-1 Bone01 0.0000 0.0000 0.0000 / -90.0000 0.0000 0.0000
0 Bone02 25.0000 0.0000 0.0000 / 0.0000 0.0000 0.0000
1 Bone03 25.0000 0.0000 0.0000 / 0.0000 0.0000 0.0000

the number in the first line is the number of joints and like before, from the second line we have the parent index, the name, positions and rotations.
I filled a global array of joints(I use glm and glew):


struct JOINT {
	GLchar	sName[30];
	GLint	nParent;
	vec3	vPosition;
	quat	quatRotations;
	mat4	matLocal,	//to parent
		matGlobal;	//to root
};
[...]
JOINT    *g_pBindPose;
GLuint    g_uiJointsNum;
[...]
int BuildBindPose()
{
    [...]
    g_pBindPose = new JOINT[g_uiJointsNum];
    [...]
}

For the bind pose calculation I do:


int BuildBindPose()
{
    [...]
    for (GLuint j = 0; j < g_uiJointsNum; j++) {
        g_pBindPose[j].matLocal = translate(mat4(), g_pBindPose[j].vPosition);
        g_pBindPose[j].matLocal *= mat4(g_pBindPose[j].quatRotations);
        if (g_pBindPose[j].nParent == -1)
            g_pBindPose[j].matGlobal = g_pBindPose[j].matLocal;
        else
            g_pBindPose[j].matGlobal = g_pBindPose[g_pBindPose[j].nParent].matGlobal * g_pBindPose[j].matLocal;
    [...]
}
[...]

Now, I have also an animation file with 3 keyframes, is a simple animation where the skeleton starts from the bind pose, then rotates every joint by 30°, then back again to the bind pose, I show you only the second keyframe:


1666.0000 -90.0000 0.0000 30.0000 / 0.0000 -30.0000 -0.0000  / 0.0000 -30.0000 0.0000 

the first is the key time in milliseconds, then we have the rotations of the 3 joints.
I just want to render the skeleton in that pose, so in the file loading part I added:


[...]
JOINT *g_pKeyframe;
mat4  *g_pmatTransformations; //final joints transformation
[...]
int LoadAnimations()
{
    [...]
    g_pKeyframe = new JOINT[g_uiJointsNum];

    for (GLuint j = 0; j < g_uiJointsNum; j++) {
        g_pKeyframe[j].matLocal = translate(mat4(), g_pBindPose[j].vPosition);
        g_pKeyframe[j].matLocal *= mat4(g_pKeyframe[j].quatRotations);
        if (g_pKeyframe[j].nParent == -1)
            g_pKeyframe[j].matGlobal = g_pKeyframe[j].matLocal;
        else
            g_pKeyframe[j].matGlobal = g_pKeyframe[g_pKeyframe[j].nParent].matGlobal * g_pKeyframe[j].matLocal;
				
        g_pmatTransformations[j] = g_pKeyframe[j].matGlobal * inverse(g_pBindPose[j].matGlobal);
    [...]
}

In the animation there are only rotations so I use the position of the bind pose, is this correct?
I get the quaternion for the rotations in the loading part, with this:


quat GetQuaternionRotation(vec3 vAxis, GLfloat fAngle)
{
	quat quatRotation;
	
	quatRotation.x = sin(radians(fAngle / 2.0f)) * vAxis.x;
	quatRotation.y = sin(radians(fAngle / 2.0f)) * vAxis.y;
	quatRotation.z = sin(radians(fAngle / 2.0f)) * vAxis.z;
	quatRotation.w = cos(radians(fAngle / 2.0f));

	return quatRotation;
}

for the 3 axis (I know there is already a function for that, but I was learning quaternions and I wrote one):


[...]
g_pKeyframe[j].quatRotations = GetQuaternionRotation(vec3(0.0f, 0.0f, 1.0f), fZAngle) *
					   GetQuaternionRotation(vec3(0.0f, 1.0f, 0.0f), fYAngle) *
					   GetQuaternionRotation(vec3(1.0f, 0.0f, 0.0f), fXAngle);
[...]

For rendering the skeleton I use the same type of vertex I use for the meshes:


struct VERTEX {
	GLfloat x, y, z,	//position
		nx, ny, nz,	//normal
		tu, tv,		//texture coord.
		tx, ty, tz,	//tangent
		bx, by, bz,	//binormal
		w[4];		//weights
	GLint	j[4];		//joints indices	
};

here my simple vertex shader:


#version 330 core
layout (location = 0) in vec3	vVertexPos;
layout (location = 1) in vec3	vNormal;
layout (location = 2) in vec2	vTexCoords;
layout (location = 3) in vec3	vTangent;
layout (location = 4) in vec3	vBitangent;
layout (location = 5) in vec4	vWeight;
layout (location = 6) in ivec4	vJoint;

uniform mat4 matTransforms[30]; //final transformations
uniform mat4 matProjection;
uniform mat4 matView;
uniform mat4 matModel;

out vec3 vColor;

void main()
{
    vColor = vec3(1.0, 0.5, 0.7);

    vec4 vNewVertex = (matTransforms[vJoint[0]] * vec4(vVertexPos, 1.0)) * vWeight[0]; //only one joint
	    
    gl_Position = matProjection * matView * matModel * vec4(vNewVertex.xyz, 1.0);
	
}

The idea, is that I have six vertices, 2 for joint, skinned to the skeleton, like my above post, the vertices are the global joint positions in bind pose, I couple a joint with his parent to form a line (bone).
The same formula I use in the shader, works if I calculate the position outside of it, everything is correct even if I don’t understand the rotations of the root node (I extracted these from the fbx file) it works like in the modelling software I use.
I took the formula from here.
The vertex array creation:


glGenVertexArrays(1, &g_uiVAO);
glGenBuffers(1, &g_uiVBO);
glGenBuffers(1, &g_uiEBO);
glBindVertexArray(g_uiVAO);

glBindBuffer(GL_ELEMENT_ARRAY_BUFFER, g_uiEBO);
glBufferData(GL_ELEMENT_ARRAY_BUFFER, sizeof(GLuint)*g_uiIndicesNum, g_puiIndices, GL_STATIC_DRAW);

glBindBuffer(GL_ARRAY_BUFFER, g_uiVBO);
glBufferData(GL_ARRAY_BUFFER, g_uiVerticesNum * sizeof(VERTEX), &g_pVertices[0], GL_STATIC_DRAW);

glVertexAttribPointer(0, 3, GL_FLOAT, GL_FALSE, sizeof(VERTEX), (GLvoid*)0);//vertices
glEnableVertexAttribArray(0);
glVertexAttribPointer(1, 3, GL_FLOAT, GL_FALSE, sizeof(VERTEX), (GLvoid*)(3 * sizeof(GLfloat)));//normal
glEnableVertexAttribArray(1);
glVertexAttribPointer(2, 2, GL_FLOAT, GL_FALSE, sizeof(VERTEX), (GLvoid*)(6 * sizeof(GLfloat)));//texture coord.
glEnableVertexAttribArray(2);
glVertexAttribPointer(3, 3, GL_FLOAT, GL_FALSE, sizeof(VERTEX), (GLvoid*)(8 * sizeof(GLfloat)));//tangent
glEnableVertexAttribArray(3);
glVertexAttribPointer(4, 3, GL_FLOAT, GL_FALSE, sizeof(VERTEX), (GLvoid*)(11 * sizeof(GLfloat)));//binormal
glEnableVertexAttribArray(4);
glVertexAttribPointer(5, 4, GL_FLOAT, GL_FALSE, sizeof(VERTEX), (GLvoid*)(14 * sizeof(GLfloat)));//weights
glEnableVertexAttribArray(5);
glVertexAttribPointer(6, 4, GL_INT, GL_FALSE, sizeof(VERTEX), (GLvoid*)(18 * sizeof(GLfloat)));//joints indices
glEnableVertexAttribArray(6);

glBindBuffer(GL_ARRAY_BUFFER, 0);
glBindVertexArray(0);

g_puiIndices is an array of 6 unsigned int ordered from 0 to 5.
I send the final matrices with this:


[...]
for (GLuint m = 0; m < g_uiJointsNum; m++)
		glUniformMatrix4fv(glGetUniformLocation(g_uiShaderProgram, ("matTransforms[" + to_string(m) + "]").c_str()), 1, GL_FALSE,   
                                                                      value_ptr(g_pmatTransformations[m]));
[...]

and this is the render pass:


glBindVertexArray(g_uiVAO);
glBindBuffer(GL_ELEMENT_ARRAY_BUFFER, g_uiEBO);
glDrawElements(GL_LINES, g_uiIndicesNum, GL_UNSIGNED_INT, 0);
glBindVertexArray(0);

I tried all the week reading many times and changing stuff, but I can’t get out of it.
Maybe you can see something I can’t, or advise me.:o
(How can I upload images?)
Thank you in advance.

You need to use glVertexAttribIPointer() (note the extra I) for integer attributes.

I love you!
From now I will put more attention to these vertex attribute functions and on gl functions in general.
Thank you very much!!!