title: OpenCV>> Structure from motion, triangulation
body: <p><strong>Use-case</strong></p>
  
  <hr>
  
  <ol>
  <li>Generate a synthetic 3D scene using random points</li>
  <li>Generate two synthetic cameras</li>
  <li>Get the 2 camera 2D projections</li>
  <li>Derive the Fundamental &amp; Essential matrices</li>
  <li>Derive rotation &amp; translation using the Essential matrix</li>
  <li>Triangulate the 2 2D projections to result the initial 3D scene</li>
  </ol>
  
  <p><strong>Implementation</strong></p>
  
  <hr>
  
  <ol>
  <li>Random 3D points ( x, y, z ) are generated</li>
  <li>Camera intrinsic matrix is statically defined</li>
  <li>Rotation matrix is statically defined ( 25 deg rotation on the Z-axis )</li>
  <li>Identity Translation Matrix ( no translation )</li>
  <li>Two projections are synthetically generated ( K*R*T )</li>
  <li>Fundamental matrix is resolved using cv::findFundamentalMat ( F )</li>
  <li>Essential matrix E is computed using 'K.t() * F * K'</li>
  <li>Camera extrinsic is extracted using SVD resulting in 4 possible solutions ( in accordance to 'Hartley &amp; Zisserman Multiple View Geometry chapter 9.2.6 )</li>
  <li>Triangulation is done using cv::triangulatePoints in the following manner:
  cv::triangulatePoints(K * matRotIdentity, K * R * T, v1, v2, points);</li>
  <li>'points' is a 4-rows N-columns matrix with homogeneous  coordinates ( x, y, z, w )</li>
  <li>'points' is converted to un-homogeneous  ( local ) coordinates by dividing 'x, y, z' with 'w'</li>
  </ol>
  
  <p><strong>The result</strong></p>
  
  <hr>
  
  <p>The resulting 3D points match to the original points up to a scale ( ~144 in my case ).</p>
  
  <h2>Questions</h2>
  
  <ol>
  <li>Camera Translation is derived up to a scale ( at #8 ), having that in mind, would it be right to assume that the triangulation result is also up to a scale? </li>
  <li>Is it possible to derive scale w/o having any prior knowledge of the camera position or absolute size of the points ?</li>
  </ol>
  
  <p>Any help would be appreciated.</p>
  
  <p><br />
  <br /></p>
  
  <h2><em>EDIT:</em></h2>
  
  <p>I was trying to use the exact same projection matrices used for the 3D -> 2D projection to convert back from 2D to 3D ( using cv::tirangulatePoints ), surprisingly, this has resulted a null vector ( all 3D points have x,y,z,w == 0 ), this has ended up to be because the two cameras differed only by rotation and not by transation, and thus, the two projection lines were orthogonal ( in 3D Space ) resulting a zero length baseline ( an epipolar line rather than a plane ), and thus, minimizing the distance at x,y,z == 0, resulting the <em>null</em> vector.</p>
  
  <p>Adding a translation between the two cameras resulted in properly recovering the original coordinates, this however, given that I was using the exact same projection matrices for the 3D to 2D transfer, AND, then, 3D back to 2D triangulation.</p>
  
  <p>When doing camera pose estimation ( extracting the projection matrix from points correspondence ) the translation is derived up to a scale, and thus, the triangulation result is also up to a scale.</p>
  
  <h2><strong>Question</strong></h2>
  
  <p>Is it possible to derive the difference in translation ( how much the camera has moved ) in metric/pixel, ... units rather than up to a scale? what prior knowledge is needed? </p>