PDA

View Full Version : Question about FBX/ASSIMP - skeletal animation



NoobCoder
09-20-2017, 11:59 AM
I am not sure if this is the right place to ask this question but this forum has been very helpful to me in the past so I am hoping that some of you more experienced with skeletal animation can help me figure this out.

I followed several tutorials to implement my animation engine with assimp and opengl. I downloaded a mixamo fbx model + walking animation. I have been able to load the static model and texture it without issues.

But when I go to animate it the model "explodes". I can see the "cadence" of the walk and can even make out some semblance of human structure walking ... but the polygons are stretched out in every direction and basically it's unrecognizable garbage from the first frame.

I don't think the problem is in my code though. I re-implemented this animation algorithm in 3 different ways following 3 different tutorials with completely different approaches. I even swapped the assignment of the bones from model to meshes back to model ... in other words, i gutted my logic (including the shader logic) several times and the model explodes in the exact same way every single time no matter what i change.

I tracked back the issue (I think! I am not sure ... ) to this:

1.000000, -0.000000, 0.000000, 0.000000,
0.000000, 1.000000, 0.000000, -150.028214,
-0.000000, -0.000000, 1.000000, 2.682794,
0.000000, 0.000000, 0.000000, 1.000000

The above is the one of the bone offset transforms coming straight from assimp mapped into neck bone.

Take notice of the "y" component of the translation section of the matrix. It's equal to -150. The biggest vertex y value I have in the model is +15. If my math is right this would shift the neck upwards by about 10 times the height of the model ...

That doesn't even end there of course since all the offsets are off: if the y translation is not out of whack, the z translation or the x translation are. In other words I feel that the offset matrix of the bone array coming from assimp has disproportionately big translation section.

Which means that the final transformation (which gets compounded as you get lower and lower into the bone hierarchy) is absurdly big ... in the thousands (the translation section at least). The rest of the transforms seem to be ok.

Just so you know ... if I take that exact same FBX file and play it either in Maya or in Blender ... they work just fine and the model walks without issues.

So ... assuming that the code from my custom bone structure to the shader is correct and that the FBX model is ok ... the thing in between must be the issue: either assimp is doing something wacky or I am not getting the offset matrix correctly from it.

As anyone experience anything like this before?

Any help would be appreciated ... I've been struggling with this for the past week.

Thanks!!

Dark Photon
09-21-2017, 06:24 AM
...skeletal animation ...fbx model + walking animation. I have been able to load the static model and texture it without issues.

But when I go to animate it the model "explodes". I can see the "cadence" of the walk and can even make out some semblance of human structure walking ... but the polygons are stretched out in every direction and basically it's unrecognizable garbage from the first frame.

Yeah, that's classic behavior for when your skin mesh is perfectly valid (when you ignore the joint influences and weights on the vertices anyway), but there is something wrong with your skinning. In other words, you're getting bogus skinning transforms in your shader or blending them improperly.

Because the way that you are fetching or blending them is deterministic, not random (you're iterating across the keyframes of your animation track), you end up replaying the same bogus transforms every cycle allowing you to see the same period in the "trash" that you'd see in the animation.

Possible problems:
1) You're not computing the base skinning transforms for each keyframe of the animation track properly.
2) You're not interpolating those base skinning transforms properly (if you're doing keyframe interpolation).
3) You're not binding/uploading your skinning transform palette to the GPU shader properly.
4) Your shader is not sampling the correct joints in your skinning transform palette.
5) Your shader is not blending the skinning transforms correctly when multiple joint influences are involved.
6) You are not applying the resultant skinning transform to your vertex positions properly.

My bet is it's #1, #2, or #3, particularly given your symptoms, but it could be anywhere. Don't assume, and don't rule anything out without strong evidence.


Which means that the final transformation (which gets compounded as you get lower and lower into the bone hierarchy) is absurdly big ... in the thousands (the translation section at least). The rest of the transforms seem to be ok.

This is a good clue. You should be able to walk this back into the component joint orientation transforms (from the bind pose) and joint animation transforms (for a new pose) to see where these big numbers are coming from.

Sometimes when you have skinning problems like this it helps to ignore the skin mesh and go back and render an animated "stick figure" with just bones build directly from skeleton and the animation data so you can just see where the joints are in space at each keyframe in the animation. This helps verify that the base animation looks legit. Once it does, then add skinned mesh rendering which makes use of this but also adds in how the skin mesh was rigged to the skeleton (joint weights/indices, blending between joints, etc.)

NoobCoder
09-21-2017, 01:18 PM
Ok so following your advice(s) I started "removing" things and in fact the first thing I removed was the offset matrix. What I got is a walking skeleton with a mesh that "adhering" to it (it looks like a skeleton with colored plastic shrinkwrap around it).

Given that, I would rule out the bone transforms as being the issue ... or the way they are transmitted to the shader since the bones themselves are all there, they appear to be lined up properly and they are moving as I would expect them to.

So ... the base animation looks legit. I am now looking at the skinning and weights/indices.

I thought those were fine but ... I guess not. The offset matrix I guess may not be the problem after all ...

I think you set me in the right direction! Thank you!

PS: I feel like a fumbling idiot with this skeletal animation stuff ... :P

Dark Photon
09-21-2017, 05:39 PM
PS: I feel like a fumbling idiot with this skeletal animation stuff ... :P

My suggestion: don't feel like that! It sounds like you're doing great!

It's tricky stuff for sure, especially the first time around, particularly with the plethora of really poor to so-so skeletal animation tutorials out there on the net to trip over, written by folks that (in hindsight) probably don't understand how skeletal transforms work on a nuts-and-bolts level. I've been where you are, and the trick for me was getting a line on a few really solid skeletal animation articles and book chapters. With that, I could "prune away" the bogus and incomplete info I'd slogged through and really internalize this stuff.

In case you're interested, here are a very, very few good "core reads" on this topic:


Chapter 4.15: Filling the Gaps: Advanced Animation Using Stitching and Skinning from Game Programming Gems (http://www.amazon.com/Game-Programming-Gems-CD/dp/1584500492) (also in Best of Game Programming Gems (http://www.amazon.com/Best-Game-Programming-Gems-DeLoura/dp/1584505710/ref=sr_1_1?s=books&ie=UTF8&qid=1341514816&sr=1-1&keywords=best+of+game+programming+gems)).
Chapter 11 of Jason Gregory's Game Engine Architecture (http://www.amazon.com/Game-Engine-Architecture-Jason-Gregory/dp/1568814135/ref=sr_1_1?ie=UTF8&qid=1341514685), 1st edition (prob 2nd ed. too, but I haven't read it in that edition)

The latter is the best comprehensive "graphics software guy" foundation on skeletal animation that I know of. Whenever I talk about skeletal, I use Gregory's terminology.

Once you've got that down, then you're ready to branch out into newer techniques that didn't make his book.

stephenmwyatt
09-22-2017, 06:33 AM
I am using tubular legs, torso, and arms, and an sphere for a head.

I have a guy that can walk, but it is very complicated to understand the locomotion of walking.

I am more than happy to show how I do this if you email me at Stephenmwyatt@gmail.com

As soon as I get it cleaned up a bit, I will put it on youtube. My version does not use GLUT, GLU, GLFW, or any of that, just pure OpenGL 4.5 and I use Xcode with Objective-C, Objective-C++, and regular C and C++.

The timing of the neck, knee, hip, ankle, shoulder, elbow, and wrist movements are the critical part after you get the basics of drawing the limbs from a central point (perhaps 0,0,0). My end goal is to be able to create cartoon movies by coding things like WALK_TO(0,10,4) or other functions that allow the cartoon locomotion to be easily coded and changed. What I have seen using Maya and the other commercial tools looks too robotic (a.k.a. CRAPPY), for my tastes.

If anyone wants to team up on doing this, I am all ears

NoobCoder
09-23-2017, 03:19 PM
Ok ... now I really feel dumb. I found out what the problem was. The problem is that when I converted the matrices loaded by assimp into my own custom system I forgot one small detail: the vector math I am using in my custom objects is all based on glm. Glm is column-major. Assimp treats matrices the standard C++ way: row-major. So all my matrices were translated from the onset.

Every piece of math after loading assimp was wrong from the get go and basically my animation was exploding.

I fixed the loading process and now the model is walking PERFECTLY!

On to animation blending ... :D

PS: the way I found out is as follows: I pulled all the logic out of the vertex shader.

So instead of doing this




void main() {
mat4 BoneTransform = gBones[aBoneIDs[0]] * aWeights[0];
BoneTransform += gBones[aBoneIDs[1]] * aWeights[1];
BoneTransform += gBones[aBoneIDs[2]] * aWeights[2];
BoneTransform += gBones[aBoneIDs[3]] * aWeights[3];

vec4 PosL = BoneTransform * vec4(aPos, 1.0);
gl_Position = MVP * PosL;
TexCoords = aTexCoords;
vec4 NormalL = BoneTransform * vec4(aNormal, 0.0);
Normal = (model * NormalL).xyz;
FragPos = (model * PosL).xyz;

}



in the shader ... I moved it into the host code and did it in C++. My frame rate dropped from 60 to 3 but it doesn't matter because I needed to see the values of the vertices and verify that the problem was not in the way the data was transfered to the shaders (which it wasn't ... the problem was waaaaaay more upstream than that).

So debugging and stepping through the code line by line I noticed that if I took one bone and applied all the appropriate math to it .. my math (I reproduced all that math in excel ... literally) was ok. In C++ it was horrendous. and then I realized that the matrix was translated.

Doh.

2486

Thank you for your help and by the way, that book you suggested (Game programming gems) ... I bought it and it's fantastic!

By the time I got it I had already solved the problem but it has a great section on animation blending (which is my next challenge!)

Thank you all!