title: Is it possible to dynamically run benchmark tests?
body: <p>I have a few different implementations of an interface, and a variety of factors I want to test them under. End goal being to make a grid of results for different implementations in different situations.</p>
  
  <p>I could write a test for each possible combination, but that gets exhausting:</p>
  
  <pre><code>func Benchmark_ImplA_N100_X300(b *testing.B){
     impl := newImplA(100,300)
     runBenchmark(b,impl)
  }
  </code></pre>
  
  <p>The more combinations I add, the more I have to copy/paste. This gets cumbersome fast.</p>
  
  <p>I would love to do something like:</p>
  
  <pre><code>tests := []testing.InternalBenchmark{}
  for _, n := range []int{50,100,500,10000}{
     for _,x := range []int{300,200}{
        for name, gen := range implementations{
           impl = gen(n,x)
           bm := testing.InternalBenchmark{Name: fmt.Sprint(name,x,n)}
           bm.F = func(b *testing.B){
              runBench(b,impl)
           }
           tests = append(tests,bm)
        }
     }
  }
  testing.RunBenchmarks(anything, tests)
  </code></pre>
  
  <p>so that I can update the list(s) of combinations, and benchmarks will magically run in all combinations. I have tries something like this in main and in TestMain, and nothing ever outputs. I am not sure if I am using it wrong, or if the testing package is just doing something funny.</p>
  
  <p>I really don't care if the <code>go test</code> tool can handle it or if there is some other way.</p>