Part of the Khronos Group
OpenGL.org

The Industry's Foundation for High Performance Graphics

from games to virtual reality, mobile phones to supercomputers

Results 1 to 10 of 10

Thread: Skeletal Animation: Bones for Models or Bones for Meshes?

  1. #1
    Junior Member Newbie
    Join Date
    Nov 2016
    Posts
    10

    Skeletal Animation: Bones for Models or Bones for Meshes?

    Hello everyone,

    this question is, as you can probably see, more a question of data structure than one about OpenGL itself. Yet I am stuck at a topic, that is probably quite familiar to most OpenGL users, therefore I hope it is alright for me to have posted this question in this board.

    Basically I am trying to implement skeletal Animation in my program. The basic concept of using vertex attributes for refering to bones and their weight is pretty clear to me, yet I struggle with the data structure:
    Since models usually can be composed of multiple meshes, I dont know, where to place the bones. Giving each mesh a separate set of bones has the advantage that bones stay in place in "mesh space", when one mesh is rotated relative to another mesh. Yet, if one of the meshs is some kind of attachment, for example a gun in the hand of a human character, the gun needs to have access to the rotation of the hand as well, since it should move with the hand.

    There are probably many ways to solve this, and they probably have different advantages and disadvantages. Yet I feel like everything I come up with is outright horrible as well in code maintenance as in performance. So IŽd be very glad, if someone could give me a short outline, of how this topic is usually, or at least "effectively" solved.

    Thanks in advance!

  2. #2
    Senior Member OpenGL Guru Dark Photon's Avatar
    Join Date
    Oct 2004
    Location
    Druidia
    Posts
    4,171
    Quote Originally Posted by Patsch View Post
    ... implement skeletal Animation ... if one of the meshs is some kind of attachment, for example a gun in the hand of a human character, the gun needs to have access to the rotation of the hand as well, since it should move with the hand. ... IŽd be very glad, if someone could give me a short outline, of how this topic is usually, or at least "effectively" solved.
    Often times, the attachment (gun) itself doesn't have a joint skeleton. It's just attached to a joint skeleton of another model at a special attachment joint in the skeleton designated for "gluing on" other models.

    To implement this, on frames where your skeletal animation is updated, just sample the attachment joint's local transform (joint-space -to- mesh-space), and use the product of that and the mesh's modeling transform as the modeling transform for the attachment.

    That said, there's nothing stopping you from attaching a model that itself has a joint skeleton too. A joint on the model to attach would be attached to a joint on the target model. The last part is the same as above, so you just need to add an additional transform to account for the first part.

    ...more a question of data structure ... Since models usually can be composed of multiple meshes, I dont know, where to place the bones.

    • A skeleton is composed of heirarchy of joints (bones being the spaces between the joints).
    • One or more meshes are rigged to a skeleton (rigged meaning the vertices reference one or more joints in the skeleton, with their influences often weighted).
    • One or more animation tracks animate a joint skeleton.


    Don't be confused by attachment models. They are typically not rigged to the joint skeleton of the target model (i.e. their modeled vertices do not contain joint IDs and weights for the target model's skeleton). These attachment models are just placed in space using transforms sampled from animation tracks for the target model's skeleton to make it look like they were built into the model.
    Last edited by Dark Photon; 08-08-2017 at 06:42 PM.

  3. #3
    Senior Member OpenGL Guru Dark Photon's Avatar
    Join Date
    Oct 2004
    Location
    Druidia
    Posts
    4,171
    It also occurs to me to mention that the above presumes you may want to attach an attachment model to one of multiple target models which may themselves be rigged to different joint skeletons.

    If on the other hand you have an attachment model that you know will always be attached to the same target skeleton, you could still do the above. Or for this special case, you could just rig the attachment directly to the same joint skeleton in the same bind pose space (i.e. store the joint index of the attachment joint in its vertices, weighted 100%, with the appropriate position/rotation in bind-pose space encoded into the positions) and then just render the attachment as a normal skeletal model, animated by the same animation track and animation state as the target model rigged to the same skeleton. Pros: That would avoid needing to do anything special to position the attachment in the world. Cons: That would prevent you from attaching it to other models easily.

    For this special case, you could just combine this attachment model into the target model and have one unified model. Reason why you wouldn't though include: 1) you want to be able to selectively draw/hide the attachment model separate from the target model, or 2) you want to render the attachment with different render state (e.g. shader) than the target model.
    Last edited by Dark Photon; 08-09-2017 at 06:26 PM.

  4. #4
    Junior Member Newbie
    Join Date
    Nov 2016
    Posts
    10
    A skeleton is composed of heirarchy of joints (bones being the spaces between the joints).
    One or more meshes are rigged to a skeleton (rigged meaning the vertices reference one or more joints in the skeleton, with their influences often weighted).
    One or more animation tracks animate a joint skeleton.



    Don't be confused by attachment models. They are typically not rigged to the joint skeleton of the target model (i.e. their modeled vertices do not contain joint IDs and weights for the target model's skeleton). These attachment models are just placed in space using transforms sampled from animation tracks for the target model's skeleton to make it look like they were built into the model.
    Alright, sorry, I am having trouble following you, so a really basic question: Given, that I want to use models composed of different meshes (no attachments involved right now, just like a .blend file would maybe contain multiple meshes). My structure would be:

    Model (has a model matrix, storing the rotation and position of the "model", so basically the models origin)
    That Model has an array of Meshes. Those Meshes .... when I import them from blender, they are already in "model space", not in "mesh space", but in order to be able to rotate them relative to the rest of the model, I would need to define a Mesh Matrix, which converts in model space, right? Are mesh matrices a common thing, or are interactions between different meshes handled via animation anyways? (Thinking about a plane having a propeller or something the like)

    Yet all meshes are transformed by a single skeleton, which is an array of joints for the whole model, therefore it should be managed in the model class. Did I understand that correctly?



    Quote Originally Posted by Dark Photon View Post
    For this special case, you could just combine this attachment model into the target model and have one unified model. Reason why you wouldn't though include: 1) you want to be able to selectively draw/hide the attachment model separate from the target model, or 2) you want to render the attachment with different render state (e.g. shader) than the target model.
    Exactly, I was planning to use different shaders, for example for some kind of magic orbs or something, which could even be light sources or such.

  5. #5
    Member Regular Contributor
    Join Date
    May 2016
    Posts
    445
    what you want to have is a "hierarchical structured model", something like a tree, but each "node" can have multiple children (not just 1, or 2). take a look at "scene graph" structures, example:
    https://research.ncl.ac.uk/game/mast...e%20Graphs.pdf

    a "node" has 1 transformation matrix, a reference (possibly an array index) to its parent node if present, and references to possible child nodes. each node can have several references to meshes

    a model can have several animations, 1 animation is a process in which several nodes are moved around / rotated. 1 animation therefore has several "channels", 1 "channel" describes how a certain node moves over time.

    if you take a look at "assimp" library, that's exactly the structure of a model's "scene" (not to be confused with your openGL "world" which is also often called "scene")

    1 "joint" in DarkPhoton's terms would be 1 "node"

  6. #6
    Junior Member Newbie
    Join Date
    Nov 2016
    Posts
    10
    Thanks for the link and the explanation. While the Scene Graph or Node approach is probably a good solution for the behaviour of isolated meshes. Yet your answer looks a bit oversimplified as far as I understand.
    Because on the one hand, files imported via assimp, even though having a hierarchy, all meshes still share a single origin. So transforming them would basically transform them _in relation_ to the origin and the other parts of the model. For the Joints: Skeletal animation, which I want to include as well, is done on a per vertex basis, and is not the same like transforming nodes. So my problem is not solved by reducing joints to nodes, because I would miss some functionality.

  7. #7
    Member Regular Contributor
    Join Date
    May 2016
    Posts
    445
    not oversimplified, but exactly what you need

    consider my arm, my shoulder is a "joint" or "node", my ellbow is another "node", and my hand would be yet another.

    if i rotate around shoulder, the whole arm moves
    if i rotate around ellbow, my upper arm doesnt move, but my lower arm does

    here another site:
    https://www.khronos.org/opengl/wiki/Skeletal_Animation

    each "uniform mat4 Bone" is the transformation of 1 node, each vertex has 2 additional attributes you have to compute:
    vec4 vertexweight and
    ivec4 boneindices

    these 2 attributes allow the vertex to be "influenced" by up to 4 bones

  8. #8
    Senior Member OpenGL Guru Dark Photon's Avatar
    Join Date
    Oct 2004
    Location
    Druidia
    Posts
    4,171
    Quote Originally Posted by Patsch View Post
    Given, that I want to use models composed of different meshes .... My structure would be:

    Model (has a model matrix, storing the rotation and position of the "model", so basically the models origin)
    That Model has an array of Meshes.
    Those Meshes .... ... are already in "model space", not in "mesh space",
    but in order to be able to rotate them relative to the rest of the model, I would need to define a Mesh Matrix, which converts in model space, right?
    If all the meshes in a model are defined in a shared object-space, then it doesn't sound like there's a need for a mesh matrix (i.e. mesh-space-to-model-space transform), because it's the identity.

    If you did need for one part of the model (particularly a skeletal model) to rotate or translate relative to another, then you could just use a skeletal animation to do that, avoiding the need to bring in this mesh matrix concept.

    Are mesh matrices a common thing, or are interactions between different meshes handled via animation anyways?
    No, AFAIK a "mesh matrix" ("mesh-space-to-model-space" transform) is not pervasive. However, do whatever makes the most sense for your app. Just keep in mind that if your app is going to launch a lot of draw calls per frame, one con to this "multiple meshes per model" idea is that it's probably going to increase the number of draw calls you'll be using, which can reduce your performance. I'd prefer a single mesh per model where it makes sense.

    Yet all meshes are transformed by a single skeleton, which is an array of joints for the whole model, therefore it should be managed in the model class. Did I understand that correctly?
    Yes, you could structure it that way.

    However, in general there's no reason why you can't have "multiple models" that all share the same joint skeleton.

    Why do this? This would allow you to have a single humanoid skeleton, and then define 5-10 different appearance models ("skins") all of which are rigged to that same joint skeleton. Then any animations you define for that skeleton can be trivially applied to any/all of the 5-10 appearance models -- all because they're associated with the same joint skeleton. Big savings in animator time, and a memory and potential perf savings in your app. Make sense?

    In other words, a more general solution would be to have the skeleton not managed by the model class. But rather have it be a shared entity "referenced" by a model class.

    Quote Originally Posted by Model-Loading/Assimp (LearnOpenGL.com)
    Mesh
    When modelling objects in modelling toolkits, artists generally do not create an entire model out of a single shape. Usually each model has several sub-models/shapes that it consists of. Each of those single shapes that a model is composed of is called a mesh. Think of a human-like character: artists usually model the head, limbs, clothes, weapons all as separate components and the combined result of all these meshes represents the final model. A single mesh is the minimal representation of what we need to draw an object in OpenGL (vertex data, indices and material properties). A model (usually) consists of several meshes. In the next tutorials we'll create our own Model and Mesh class that load and store the imported models using the structure we've just described. If we then want to draw a model we do not render the model as a whole but we render all of the individual meshes that the model is composed of. ...
    LINK

    Given this description, and mixing it with good performance recommendations for OpenGL, you probably want to (if not now then later) consider merging separate meshes for a model into shared draw calls, if there isn't a good reason not to (e.g. separate shader, renderstate, visibility, etc.).
    Last edited by Dark Photon; 08-09-2017 at 07:08 PM.

  9. #9
    Member Regular Contributor
    Join Date
    Jul 2012
    Posts
    429
    I've ready quickly threw this thread, and probably this had been said, but since it still looks unclear to the OP, I would like to emphasize that the most important thing here is the notion of parent and children. Children inherit the coordinate system of their parent, recursively.

  10. #10
    Member Regular Contributor
    Join Date
    May 2016
    Posts
    445
    Quote Originally Posted by Silence View Post
    I would like to emphasize that the most important thing here is the notion of parent and children. Children inherit the coordinate system of their parent, recursively.
    exactly, thats where i had problems understanding at first, too


    Quote Originally Posted by Patsch
    Because on the one hand, files imported via assimp, even though having a hierarchy, all meshes still share a single origin.
    thats not necessarily true
    a mesh's origin doesnt have to be the origin of the models scene
    consider a spherical mesh as "head", a cylidrical mesh as (a kind of ..) "flesh"

    you can model a human by reusing the cylindrical mesh for:
    --> LEFT:
    ----> upper arm
    ----> lower arm
    ----> upper leg
    ----> lower leg
    --> RIGHT:
    ... the same, maybe mirrored at the symmetry level

    that means you have the mesh 8x available in your model, OK maybe a cylinder + sphere it a bit too simple, but if you have a detailled leg (left), you dont mave to copy it for the other side (right), you can just draw it twice with a "mirrored" transformation

    that all non-animated ...

    take a look at nvidia's sample code:
    https://github.com/NVIDIAGameWorks/G...l/NvSkeleton.h
    https://github.com/NVIDIAGameWorks/G...NvSkeleton.cpp

    Code :
    struct NvSkeletonNode
    {
            std::string m_name;
            int32_t m_parentNode;
            nv::matrix4f m_parentRelTransform;
            std::vector<int32_t> m_childNodes;
            std::vector<uint32_t> m_meshes;
    };

    it is inevitable, you have to keep the transformation chain parent-->child-->child-->... intact. otherwise you hand wont move if you rotate aruond your ellbow / shoulder / etc.
    how you do that on the cpp side of your app is not really "predefined". but later when you look at the gpu side, you need an array ("uniform mat4 Bones[10];" in the previous linked site), that's why you need an array of node transformations, to be able to correctly reference a node's matrix in the vertexshader AND to build correctly your mesh buffer data.


    "skinning a mesh" is what you mean by "on a per-vertex base", for example if you rotate around the ellbow, you may want the mesh there to "bend" a little so that the wont be a visible gap between upper and lower arm.


    consider the node struct above:
    you can reference a node in the skeleton in 2 ways, either by its string name or by its int arrayindex, on the gpu-side you cant do that, you can only reference by array index (there is no std::string in glsl). once you've layed down all nodes in an array, you are able to give each vertex the correct reference (= array index) to a influencing node and its weight factor (the bone information)
    Last edited by john_connor; 08-10-2017 at 04:16 AM.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •