<p>I am not 100% sure about what you are really asking, because what you call a "match" is vague. But since you said you already matched your SURF points and mentionned pattern recognition and the use of a template, I am assuming that, ultimately, you want to localize the template in your image and you are asking about a localization score to decide whether you found the template in the image or not.</p>

<p>This is a challenging problem and I am not aware that a good and always-appropriate solution has been found yet. </p>

<p>However, given your approach, what you could do is analyze the density of matched points in your image: consider local or global maximas as possible locations for your template (global if you know your template appears only once in the image, local if it can appear multiple times) and use a threshold on the density to decide whether or not the template appears. A sketch of the algorithm could be something like this:</p>

<ol>
<li>Allocate a floating point density map of the size of your image</li>
<li>Compute the density map, by increasing by a fixed amount the density map in the neighborhood of each matched point (for instance, for each matched point, add a fixed value <em>epsilon</em> in the rectangle your are displaying in your question)</li>
<li>Find the global or local maximas of the density map (global can be found using opencv function MinMaxLoc, and local maximas can be found using morpho maths, e.g. <a href="https://stackoverflow.com/questions/1856197/how-can-i-find-local-maxima-in-an-image-in-matlab">How can I find local maxima in an image in MATLAB?</a>)</li>
<li>For each maxima obtained, compare the corresponding density value to a threshold <em>tau</em>, to decide whether your template is there or not</li>
</ol>

<p>If you are into resarch articles, you can check the following ones for improvement of this basic algorithm:</p>

<ul>
<li><p><a href="http://arxiv.org/pdf/0910.1273" rel="nofollow noreferrer">"ADABOOST WITH KEYPOINT PRESENCE FEATURES FOR REAL-TIME VEHICLE VISUAL DETECTION", by T.Bdiri, F.Moutarde, N.Bourdis and B.Steux, 2009.</a></p></li>
<li><p><a href="ftp://ftp.vision.ee.ethz.ch/publications/bookchapters/eth_biwi_00422.pdf" rel="nofollow noreferrer">"Interleaving Object Categorization and Segmentation", by B.Leibe and B.Schiele, 2006.</a></p></li>
</ul>

<p><strong>EDIT</strong>: another way to address your problem is to try and remove accidently-matched points in order to keep only those truly corresponding to your template image. This can be done by enforcing a constraint of consistancy between close matched points. The following research article presents an approach like this: <a href="http://perso.telecom-paristech.fr/~sahbi/publication-193.pdf" rel="nofollow noreferrer">"Context-dependent logo matching and retrieval", by H.Sahbi, L.Ballan, G.Serra, A.Del Bimbo, 2010</a> (however, this may require some background knowledge...).</p>

<p>Hope this helps.</p>
Measure of accuracy in pattern recognition using SURF in OpenCV
<p>I’m currently working on pattern recognition using SURF in OpenCV. What do I have so far: I’ve written a program in C# where I can select a source-image and a template which I want to find. After that I transfer both pictures into a C++-dll where I’ve implemented a program using the OpenCV-SURFdetector, which returns all the keypoints and matches back to my C#-program where I try to draw a rectangle around my matches.</p>

<p><img src="https://i.stack.imgur.com/RUhbQ.jpg" alt="This pictures shows a source-image and a template with it&#39;s keypoints and matches. Also I&#39;ve tried to calculate a rectangle around my matches"></p>

<p>Now my question: Is there a common measure of accuracy in pattern recognition? Like for example number of matches in proportion to the number of keypoints in the template? Or maybe the size-difference between my match-rectangle and the original size of the template-image? What are common parameters that are used to say if a match is a “real” and “good” match?</p>

<p><strong>Edit:</strong> To make my question clearer. I have a bunch of matchpoints, that are already thresholded by minHessian and distance value. After that I draw something like a rectangle around my matchpoints as you can see in my picture. This is my MATCH. How can I tell now how good this match is? I'm already calculating angle, size and color differences between my now found match and my template. But I think that is much too vague.</p>