title: Numerical regression testing
body: <p>I'm working on a scientific computing code (written in C++), and in addition to performing unit tests for the smaller components, I'd like to do regression testing on some of the numerical output by comparing to a "known-good" answer from previous revisions. There are a few features I'd like:</p>
  
  <ul>
  <li>Allow comparing numbers to a specified tolerance (for both roundoff error and looser expectations)</li>
  <li>Ability to distinguish between ints, doubles, etc, and to ignore text if necessary</li>
  <li>Well-formatted output to tell what went wrong and where: in a multi-column table of data, only show the column entry that differs</li>
  <li>Return <code>EXIT_SUCCESS</code> or <code>EXIT_FAILURE</code> depending on whether the files match</li>
  </ul>
  
  <p>Are there any good scripts or applications out there that do this, or will I have to roll my own in Python to read and compare output files? Surely I'm not the first person with these kind of requirements.</p>
  
  <p><i>[The following is not strictly relevant, but it may factor into the decision of what to do. I use CMake and its embedded CTest functionality to drive unit tests that use the Google Test framework. I imagine that it shouldn't be hard to add a few <code>add_custom_command</code> statements in my <code>CMakeLists.txt</code> to call whatever regression software I need.]</i></p>