 <p>There are several techiniques for removing outliers or refining matches:</p> <ul> <li><p>Cross check</p></li> <li><p>Ratio test</p></li> <li><p>RANSAC+Homography for planar shapes</p></li> </ul> <p>I usually apply the last two ones.</p> <p>RATIO TEST:</p> <pre><code>BFMatchermatcher(CV_L2); vector&lt;vector&lt;DMatch&gt;&gt;nearest_neighbors; matcher.radiusMatch( right_points_to_find_flat, right_features_flat, nearest_neighbors, 2.0f); // Check that the found neighbors are unique (throw away neighbors // that are too close together, as they may be confusing) std::set&lt;int&gt;found_in_right_points; // for duplicate prevention for(inti=0;i&lt;nearest_neighbors.size();i++) { DMatch _m; if(nearest_neighbors[i].size()==1) { _m = nearest_neighbors[i][0]; // only one neighbor } else if(nearest_neighbors[i].size()&gt;1) { // 2 neighbors – check how close they are double ratio = nearest_neighbors[i][0].distance / nearest_neighbors[i][1].distance; if(ratio &lt; 0.7) { // not too close // take the closest (first) one _m = nearest_neighbors[i][0]; } else { // too close – we cannot tell which is better continue; // did not pass ratio test – throw away } } else { continue; // no neighbors... :( } </code></pre> <p>RANSAC+Homography:</p> <pre><code>std::vector&lt;Point2f&gt; object; std::vector&lt;Point2f&gt; scene; for( int i = 0; i &lt; good_matches.size(); i++ ) { //-- Get the keypoints from the good matches object.push_back( keypoints_object[ good_matches[i].queryIdx ].pt ); scene.push_back( keypoints_scene[ good_matches[i].trainIdx ].pt ); } Mat H = findHomography( object, scene, CV_RANSAC, status ); //draw only the matches considered inliers by RANSAC =&gt; status=1 for(i=0;i=object.size();i++){ if(status[i]) { } } </code></pre>