title: Better estimation of Homography using Kalman filter?
body: <p>I am creating an AR application that tracks feature , calculates homography and then obtains the object's pose from 3D-2D point correspondences and use that to render any 3D Object.</p>
  
  <p>I am selecting a specific area for detecting features on my source image (by masking). and then matching it with features detected on subsequent frames.Then I filter those matches and estimate Homography of the unmasked region. </p>
  
  <p>The problem is lies in Homography estimation. It differs every time(very slightly, but nonetheless, differs). The effect is : Even on keeping my camera still,  I get a vibrating rectangle around my tracked region, which i draw using the estimated homography. </p>
  
  <p>I have already posted a question titled        <a href="https://stackoverflow.com/questions/17789474/unstable-homography-estimation-using-orb">Unstable homography estimation using ORB</a> and got  a reassurance of a fact i was considering (not recalculating my homography if the position of the region is similar to its last position).</p>
  
  <p>However, I recently came to know of the <strong>Kalman filter</strong>, that it gives a better estimate of the position by combining our prior knowledge with our measurement observation.</p>
  
  <p>So ,after looking at various examples (one in particular, <a href="http://www.youtube.com/watch?v=GBYW1j9lC1I" rel="nofollow noreferrer">http://www.youtube.com/watch?v=GBYW1j9lC1I</a>), I modeled a Kalman filter(rather 4, one for every point of the rectanglular region) for my scenario:</p>
  
  <pre><code>m_KF1.init(4, 2, 1); 
  setIdentity(m_KF2.transitionMatrix);
  m_measurement1 = Mat::zeros(2,1,cv::DataType&lt;float&gt;::type);
  m_KF1.statePre.setTo(0);
  m_KF1.controlMatrix.setTo(0);
  
  //initialzing filter 
  m_KF1.statePre.at&lt;float&gt;(0) = m_scene_corners[1].x; //the first reading
  m_KF1.statePre.at&lt;float&gt;(1) = m_scene_corners[1].y;
  m_KF1.statePre.at&lt;float&gt;(2) = 0;
  m_KF1.statePre.at&lt;float&gt;(3) = 0;
  
  setIdentity(m_KF1.measurementMatrix);
  setIdentity(m_KF1.processNoiseCov,Scalar::all(.1)) //updated at every step
  setIdentity(m_KF1.measurementNoiseCov, Scalar::all(4)); //assuming measurement error of      
                                                          //not more than 2 pixels
  setIdentity(m_KF1.errorCovPost, Scalar::all(.1)); 
  </code></pre>
  
  <p>4 state variables (position in x, y and velocity in x,y).</p>
  
  <p>2 measurement variables (position in x,y)</p>
  
  <p>1 control variable (acceleration)</p>
  
  <p><strong>following steps taken at every iteration</strong></p>
  
  <pre><code>//---First,the predicion phase , to update the internal variables-------//
  
  // 'dt' is the time taken between the measurements
  
  //Updating the transitionMatrix
  m_KF1.transitionMatrix.at&lt;float&gt;(0,2) = dt;
  m_KF1.transitionMatrix.at&lt;float&gt;(1,3) = dt;
  
  //Updating the Control matrix
  m_KF1.controlMatrix.at&lt;float&gt;(0,1) = (dt*dt)/2;
  m_KF1.controlMatrix.at&lt;float&gt;(1,1) = (dt*dt)/2;
  m_KF1.controlMatrix.at&lt;float&gt;(2,1) = dt;
  m_KF1.controlMatrix.at&lt;float&gt;(3,1) = dt;
  
  //Updating the processNoiseCovmatrix
  m_KF1.processNoiseCov.at&lt;float&gt;(0,0) = (dt*dt*dt*dt)/4;
  m_KF1.processNoiseCov.at&lt;float&gt;(0,2) = (dt*dt*dt)/2;
  m_KF1.processNoiseCov.at&lt;float&gt;(1,1) = (dt*dt*dt*dt)/4;
  m_KF1.processNoiseCov.at&lt;float&gt;(1,3) = (dt*dt*dt)/2;
  
  m_KF1.processNoiseCov.at&lt;float&gt;(2,0) = (dt*dt*dt)/2;
  m_KF1.processNoiseCov.at&lt;float&gt;(2,2) = dt*dt;
  
  m_KF1.processNoiseCov.at&lt;float&gt;(3,1) = (dt*dt*dt)/2;
  m_KF1.processNoiseCov.at&lt;float&gt;(3,3) = dt*dt;
  
  Mat prediction1 = m_KF1.predict();
  Point2f predictPt1(prediction1.at&lt;float&gt;(0),prediction1.at&lt;float&gt;(1));
  
  // Get the measured corner
  m_measurement1.at&lt;float&gt;(0,0) = scene_corners[0].x;
  m_measurement1.at&lt;float&gt;(0,1) = scene_corners[0].y;
  
  //----Then, the correction phase which uses the predicted value and our measured value
  
  Mat estimated = m_KF1.correct(m_measurement1);
  Point2f statePt1(estimated.at&lt;float&gt;(0),estimated.at&lt;float&gt;(1));
  </code></pre>
  
  <p>This model hardly corrects my measured value </p>
  
  <p><strong>Now My Questions are:</strong></p>
  
  <ol>
  <li>Is Kalman filter suited for my scenario? Will it give me any better results?</li>
  <li>If it is, then what's missing? am I modelling it right? Instead creating 4 filters for four points of the rectangle, should I model it in some other manner (for instance, take the 10 strongest matches based on the distance and use those as input to the filter)</li>
  <li>If Kalman filter isn't suited, what else can i do to provide more stability to the estimated homography?</li>
  </ol>
  
  <p>Any help would be highly appreciated.
  Thanks.</p>