title: How do I measure variability of a benchmark comprised of many sub-benchmarks?
body: <p>(Not strictly programming, but a question that programmers need answered.)</p>
  
  <p>I have a benchmark, <strong>X</strong>, which is made up of a lot of sub-benchmarks <strong>x<sub>1</sub></strong>..<strong>x<sub>n</sub></strong>. Its quite a noisy test, with the results being quite variable. To accurately benchmark, I must reduce that "variability", which requires that I first <strong>measure</strong> the variability.</p>
  
  <p>I can easily calculate the variability of each sub-benchmark, using perhaps standard deviation or variance. However, I'd like to get a single number which represents the overall variability as a single number.</p>
  
  <p>My own attempt at the problem is:</p>
  
  <pre><code>sum = 0
  foreach i in 1..n
     calculate mean across the 60 runs of x_i
     foreach j in 1..60
         sum += abs(mean[i] - x_i[j])
  variability = sum / 60
  </code></pre>