title: OpenCV undistortPoints and triangulatePoint give odd results (stereo)
body: <p>I'm trying to get 3D coordinates of several points in space, but I'm getting odd results from both <code>undistortPoints()</code> and <code>triangulatePoints()</code>. </p>
  
  <p>Since both cameras have different resolution, I've calibrated them separately, got RMS errors of <code>0,34</code> and <code>0,43</code>, then used <code>stereoCalibrate()</code> to get more matrices, got an RMS of <code>0,708</code>, and then used <code>stereoRectify()</code> to get remaining matrices. With that in hand I've started the work on gathered coordinates, but I get weird results.</p>
  
  <p>For example, input is: <code>(935, 262)</code>, and the <code>undistortPoints()</code> output is <code>(1228.709125, 342.79841)</code> for one point, while for another it's <code>(934, 176)</code> and <code>(1227.9016, 292.4686)</code> respectively. Which is weird, because both of these points are very close to the middle of the frame, where distortions are the smallest. I didn't expect it to move them by 300 pixels. </p>
  
  <p>When passed to <code>traingulatePoints()</code>, the results get even stranger - I've measured the distance between three points in real life (with a ruler), and calculated the distance between pixels on each picture. Because this time the points were on a pretty flat plane, these two lengths (pixel and real) matched, as in |AB|/|BC| in both cases was around 4/9. However, <code>triangulatePoints()</code> gives me results off the rails, with |AB|/|BC| being 3/2 or 4/2. </p>
  
  <p>This is my code:</p>
  
  <pre><code>double pointsBok[2] = { bokList[j].toFloat()+xBok/2, bokList[j+1].toFloat()+yBok/2 };
  cv::Mat imgPointsBokProper = cv::Mat(1,1, CV_64FC2, pointsBok);
  
  double pointsTyl[2] = { tylList[j].toFloat()+xTyl/2, tylList[j+1].toFloat()+yTyl/2 };
  //cv::Mat imgPointsTyl = cv::Mat(2,1, CV_64FC1, pointsTyl);
  cv::Mat imgPointsTylProper = cv::Mat(1,1, CV_64FC2, pointsTyl);
  
  cv::undistortPoints(imgPointsBokProper, imgPointsBokProper, 
        intrinsicOne, distCoeffsOne, R1, P1);
  cv::undistortPoints(imgPointsTylProper, imgPointsTylProper, 
        intrinsicTwo, distCoeffsTwo, R2, P2);
  
  cv::triangulatePoints(P1, P2, imgWutBok, imgWutTyl, point4D);
  
  double wResult = point4D.at&lt;double&gt;(3,0);
  double realX = point4D.at&lt;double&gt;(0,0)/wResult;
  double realY = point4D.at&lt;double&gt;(1,0)/wResult;
  double realZ = point4D.at&lt;double&gt;(2,0)/wResult;
  </code></pre>
  
  <p>The angles between points are kinda sorta good but usually not:</p>
  
  <pre><code>`7,16816    168,389 4,44275` vs `5,85232    170,422 3,72561` (degrees)
  `8,44743    166,835 4,71715` vs `12,4064    158,132 9,46158`
  `9,34182    165,388 5,26994` vs `19,0785    150,883 10,0389`
  </code></pre>
  
  <p>I've tried to use <code>undistort()</code> on the entire frame, but got results just as odd. The distance between B and C points should be pretty much unchanged at all times, and yet this is what I get: </p>
  
  <pre><code>7502,42     
  4876,46 
  3230,13 
  2740,67 
  2239,95 
  </code></pre>
  
  <p>Frame by frame. </p>
  
  <p>Pixel distance (bottom) vs real distance (top) - should be very similar:
  <a href="https://i.stack.imgur.com/gJQvc.png" rel="noreferrer"><img src="https://i.stack.imgur.com/gJQvc.png" alt="|BC| distance"></a></p>
  
  <p>Angle:</p>
  
  <p><a href="https://i.stack.imgur.com/1Ad1X.png" rel="noreferrer"><img src="https://i.stack.imgur.com/1Ad1X.png" alt="ABC angle"></a></p>
  
  <p>Also, shouldn't both <code>undistortPoints()</code> and <code>undistort()</code> give the same results (another set of videos here)?<br>
  <a href="https://i.stack.imgur.com/0jazw.png" rel="noreferrer"><img src="https://i.stack.imgur.com/0jazw.png" alt="enter image description here"></a></p>