Skinning tutorial does not detail joint matrix calculation

[kevinmoran]

The Skins Tutorial focuses mainly on the vertex shader calculations involved in skinning, and gives almost no useful information on producing the joint matrices to pass to the shader:

So the pseudocode for computing the joint matrix of joint j may look as follows:

jointMatrix(j) =
globalTransformOfJointNode(j) *
inverseBindMatrixForJoint(j);

There is no other mention of globalTransformOfJointNode, and this transform is not immediately obvious - most animation file formats produce animation in local space, whereas GLTF animation data also converts a joint into its parent's local space. I had a lot of trouble deducing this (see here) and so did the writer of this very useful blog post, without which I would have been even more lost.

The actual way you compute joint matrices is:

mat4 animMatrix[i] = /*calculate interpolated pose from animation keyframe data*/ 
int parentIndex = joints[i].parentIndex;
jointMatrix[i] =  animMatrix[parentIndex] * animMatrix[i] * invBindPoses[i];

Assuming right-to-left matrix multiplication order. Additionally there are plenty of gotchas due to the (in my opinion) mistake of specifying animation data in this coordinate space, for example if an animation does not specify any keyframe data for a joint you must transform that joint into its parent space manually using:

mat4 animMatrix[i] = invBindPose[parentIndex] * bindPose[i]

I think the tutorial should be expanded to explain how to calculate joint matrices, as it stands that blog post is far more useful than what's there currently. Thanks!

[javagl]

The skinning section refers to "globalTransformOfJointNode", but does not reiterate what the global transform of a node is.

The snippets that you posted (also) lack some context. Is invBindPoses supposed to be the inverse bind matrix? What is the parentIndex for a joint?

Although it should not be considered to be any sort of "reference", the implementation for obtaining the joint matrices in JglTF pretty much matches the description in the tutorial.

One aspect that may be noteworthy is that the "Bind Shape Matrix" is no longer present in glTF 2.0. (In the code, this is still present, because it also handles glTF 1.0). This might explain some confusion from KhronosGroup/glTF#2489 (comment), where you mentioned COLLADA (which also had a Bind Shape Matrix), but I have not yet fully processed all information from that other issue.

It may very well be that the skinning section can be improved. But I'm not entirely sure which kind of update could be expected here.

[kevinmoran]

Thanks for the response!

The skinning section refers to "globalTransformOfJointNode", but does not reiterate what the global transform of a node is.

Thanks, that's a useful link, it makes the rationale behind the spec much clearer in that the animation data is meant to replace this transform for the relevant nodes - It would be no harm imo to reference back to that section in the Skins tutorial.

The snippets that you posted (also) lack some context. Is invBindPoses supposed to be the inverse bind matrix? What is the parentIndex for a joint?

For game development it's common to remove the tree structure and represent a skeleton as a linear array such that each node's parent is placed at an earlier index - this improves data locality for better cache utilisation and allows for fast iteration instead of recursion. Instead of a recursing through a tree you just process each item in sequence and fetch the parent data using an index stored in the joint. So invBindPoses is an array of matrices such that invBindPose[i] is the matrix for the i-th joint.

One aspect that may be noteworthy is that the "Bind Shape Matrix" is no longer present in glTF 2.0. (In the code, this is still present, because it also handles glTF 1.0). This might explain some confusion from KhronosGroup/glTF#2489 (comment), where you mentioned COLLADA (which also had a Bind Shape Matrix), but I have not yet fully processed all information from that other issue.

I've never heard of a bind shape matrix, the bind pose (or bind matrix) is the inverse of the inverse bind pose.

It may very well be that the skinning section can be improved. But I'm not entirely sure which kind of update could be expected here.

I think now I have a better understanding of the GLTF's object-oriented way of doing things, which makes it tricky to use in data-oriented settings. As I mention above I think a reference back to the section where global transforms are described would help for a reader who's jumped straight to the Skinning section and doesn't know what globalTransformOfJointNode means exactly. As for getting into the nitty-gritty maths of skinning or how to collapse the node structure into a flat array, they're probably subjects beyond the scope of the tutorials. Cheers!