title: Controlling a virtual character's joints rotation with OpenNI + Kinect
body: <p>I'm starting a project where I need to control a virtual character. The character is being rendered in multiple 3D engines, such as Three.JS and iOS SceneKit.</p>
  
  <p>I'm getting the Quaternions of every joint of the skeleton with OpenNI, and it looks kind of like this:</p>
  
  <p>The code that saves and pass the quaternion, looks like this:</p>
  
  <pre><code>float confidence = context.getJointOrientationSkeleton(userId, jointName, 
  joint);
  joints[jointIndex]=joint.m00;
  joints[jointIndex+1]=joint.m01;
  joints[jointIndex+2]=joint.m02;
  joints[jointIndex+3]=joint.m10;
  joints[jointIndex+4]=joint.m11;
  joints[jointIndex+5]=joint.m12;
  joints[jointIndex+6]=joint.m20;
  joints[jointIndex+7]=joint.m21;
  joints[jointIndex+8]=joint.m22;
  jointIndex+=9;
  </code></pre>
  
  <p>This repeats for every joint of the skeleton.</p>
  
  <p>The last row and last column is always <code>[0 0 0 1] [0, 0, 0, 1]</code>, so I just append that on the client once I receive it and build a <code>4x4 matrix</code>.</p>
  
  <p>I want to be able to make the right rotations with this data, but the rotations I'm getting are definitely wrong.</p>
  
  <p>This is how I'm processing the matrix: (pseudo-code)</p>
  
  <pre><code>row1 = [m00 m01 m02 0]
  row2 = [m10 m11 m12 0]
  row3 = [m20 m21 m22 0]
  row4 = [0 0 0 1]
  
  matrix4by4 = matrix4x4(rows:[row1,row2,row3,row4])
  </code></pre>
  
  <p>And then I got the quaternion with two methods, and both methods were showing bad rotations, I cannot find what's wrong or what I'm missing.</p>
  
  <p><strong>First method</strong></p>
  
  <p>There's an iOS function that gets a 3x3 or 4x4 matrix, and transforms it into quaternion:</p>
  
  <pre><code>boneRotation = simd_quatf.init(matrix4by4).vector //X,Y,Z,W
  </code></pre>
  
  <p><strong>Second method</strong></p>
  
  <p>I found the following code on the web:</p>
  
  <pre><code>let tr = m00 + m11 + m22
  
  var qw = 0
  var qx = 0
  var qy = 0
  var qz = 0
  if (tr &gt; 0) {
      var S = sqrt(tr+1.0) * 2 // S=4*qw
      qw = 0.25 * S
      qx = (m21 - m12) / S
      qy = (m02 - m20) / S
      qz = (m10 - m01) / S
  } else if ((m00 &gt; m11) &amp;&amp; (m00 &gt; m22)) {
      var S = sqrt(1.0 + m00 - m11 - m22) * 2 // S=4*qx
      qw = (m21 - m12) / S
      qx = 0.25 * S
      qy = (m01 + m10) / S
      qz = (m02 + m20) / S
  } else if (m11 &gt; m22) {
      var S = sqrt(1.0 + m11 - m00 - m22) * 2 // S=4*qy
      qw = (m02 - m20) / S
      qx = (m01 + m10) / S
      qy = 0.25 * S
      qz = (m12 + m21) / S
  } else {
      let S = sqrt(1.0 + m22 - m00 - m11) * 2 // S=4*qz
      var = (m10 - m01) / S
      qx = (m02 + m20) / S
      qy = (m12 + m21) / S
      qz = 0.25 * S
  }
  
  boneRotation = vector4(qx, qy, qw, qz)
  </code></pre>
  
  <p>I started testing only with shoulder and elbow rotation to help me visualize what could be wrong or missing, and made a video.</p>
  
  <p>Here's how it's behaving: <a href="https://www.youtube.com/watch?v=xUtNiwH_AGk" rel="nofollow noreferrer">https://www.youtube.com/watch?v=xUtNiwH_AGk</a></p>
  
  <p>What could I be missing? For example, the axis of rotation of the shoulder are like this:</p>
  
  <p>X-Axis
  <a href="https://i.stack.imgur.com/Jocun.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/Jocun.png" alt="X-Axis"></a></p>
  
  <p>Y-Axis
  <a href="https://i.stack.imgur.com/hQCd1.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/hQCd1.png" alt="Y-Axis"></a></p>
  
  <p>Z-Axis
  <a href="https://i.stack.imgur.com/b2OSV.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/b2OSV.png" alt="Z-Axis"></a></p>
  
  <p>Thank you in advance :-)</p>
  
  <p>YouTuve Video: <a href="https://www.youtube.com/watch?v=xUtNiwH_AGk" rel="nofollow noreferrer">https://www.youtube.com/watch?v=xUtNiwH_AGk</a>
  <a href="https://i.stack.imgur.com/87S82.jpg" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/87S82.jpg" alt="YouTube Video"></a></p>