title: Classify a program from a screenshot
body: <p>I need to write and algorithm that can detect which state an application (used to fill out forms) is in based on screenshots.<br>
  It has 2 inputs:<br>
  <strong>A:</strong> Approximately 2-10 screenshots from an application with different tabs selected. These are made by the user, so i can instruct him to things like "select the upper area of the program" or "select the whole window", but i can not expect pixel-perfect precision.<br>
  <strong>B:</strong> a screenshot of one of these states. The forms are filled with different data.</p>
  
  <p>The goal is to determine which screenshot from "A" is from the same state as "B".</p>
  
  <p>An example screenshot:
  <img src="https://i.stack.imgur.com/F4qqN.jpg" alt="example screenshot"></p>
  
  <p>An example based on this screenshot:<br>
  <strong>A input:</strong> 10 screenshots from this program with "Menu","Sale Order","Purchase order",... tabs selected<br>
  <strong>B input:</strong> the screenshot above.</p>
  
  <p>The task is to determine which of the 10 screenshots matches this image.</p>
  
  <p>I have tried to use an image descriptor algorithm, (<a href="http://code.google.com/p/in-spirit/wiki/ASSURF" rel="nofollow noreferrer">SURF</a>) but it has a really high error ratio, since its not made for such tasks.</p>
  
  <p>Anyone has an idea how to make such classification? Should i use some filter (e.g median or blur) on the screenshots, and then run trought some classification algorithm? Or extract some other feature to classify (FFT,histogram,..)?</p>