picking algorithm for skeletal animation
I have some skeletal animation that is driven in the vertex shader, each vertex has a selection of bones that influence it and a bone matrix is passed representing all the bones that can influence and the eventual result is a perfect skeleton. This is a simple matrix multiplication in the shader.
I'd like to know when my mouse cursor is selecting any of the bone meshes that are drawn as part of the skeleton. Because all the grunt work is done in the shader I don't know the model matrix that is used to work out if a ray intersected the mouse to the mesh.
Can anyone suggest a way to successfully know if my mouse is overlapping a bone mesh?
Do I have to result to colour based picking or is there a clever way to do this using the ray picking based method?
If your selection needs to precisely match the drawn mesh you will need to transformed vertices. You could get these with transform feedback (see wiki). If a rougher selection is sufficient (e.g. because you only need to select the mesh, not parts of it) you could use a proxy geometry (in the extreme case just the bounding box) of the mesh to test the ray against.