title: How to improve accuracy of estimateAffine2D (or estimageRigidTransform) in OpenCV?
body: <p>I have two sets of points, one from time t-1 and current time t. The first set was generated using <code>goodFeaturesToTrack</code>, and the latter from using <code>calcOpticalFlowPyrLK()</code>. Using these two sets of points, I then estimate a transformation matrix via <code>estimateAffine2DPartial()</code> in order to keep track of its scale &amp; rotation. Code snippet is listed below:</p>
  
  <pre><code>// Precompute image pyramids
  maxLvl = cv::buildOpticalFlowPyramid(_imgPrev, imPyr1, _winSize, maxLvl, true);
  maxLvl = cv::buildOpticalFlowPyramid(tmpImg, imPyr2, _winSize, maxLvl, true);
  
  // Optical flow call for tracking pixels
  cv::calcOpticalFlowPyrLK(imPyr1, imPyr2, _currentPoints, nextPts, status, err, _winSize, maxLvl, _terminationCriteria, 0, 0.000001);
  
  // Get transformation matrix between the two data sets
  cv::Mat H = cv::estimateAffinePartial2D(_currentPoints, nextPts, inlier_mask, cv::RANSAC, 10.0, 2000, 0.99);
  
  </code></pre>
  
  <p>Using H, I then map my masking points using <code>perspectiveTransform()</code>. The result seems accurate for the first few dozen frames until I notice some <em>drift</em> (in terms of rotation) occurring when the object I am tracking continues to rotate (usually when rotation becomes > M_PI). I'm honestly stumped on where the culprit is, but my main suspicion is perhaps my window size for optical flow might be too small, or too big. However, tweaking the window size did not seem to help, the position of my object is still accurate, but the estimated rotation (and scale) got worse. Can anyone hope to shed a light on this? </p>
  
  <p>Warm regards and thanks.</p>
  
  <p>EDIT: Images attached to show drift issue</p>
  
  <p><a href="https://i.stack.imgur.com/oOFV5.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/oOFV5.png" alt="Starting Frame"></a></p>
  
  <p>Starting Frame</p>
  
  <p><a href="https://i.stack.imgur.com/ZEsiy.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/ZEsiy.png" alt="First few frames -- Rotation OK"></a></p>
  
  <p>First few frames -- Rotation OK</p>
  
  <p><a href="https://i.stack.imgur.com/skJDE.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/skJDE.png" alt="Z-Rotation Drift occurs -- See Anchor Line"></a></p>
  
  <p>Z-Rotation Drift occurs -- see anchor line has drifted towards the red rectangle. </p>