<p>You could use the <a href="http://cran.r-project.org/web/packages/SPOT/index.html" rel="nofollow">SPOT</a> packet (R programming language). It allows you to find (near-)optimal parameter settings using significantly less runs than brute force. You can use any programming language for your fitness function code, SPOT has an adapter for that, and offers an automatic mode with default setup (You don't have to worry about the design types and prediction models). It has a steep learning curve, but once you understood the basics, it is a powerful tool. <a href="http://arxiv.org/pdf/1006.4645v1.pdf" rel="nofollow">Here</a> is a quick guide; chapter 2.6 offers a concrete example. The SPOT package comes with several examples.</p>
Automatic parameter tuning
<p>I've got a an audio processing app that takes an input audio file, processes it, and spits out a modified output audio file. This audio processing app has 10-15 parameters that affect how it processes the audio, and thus affects the content of the output audio file (it might have, say, a different frequency response, be louder, quieter, etc.). All these parameters have constrained ranges (x0 must be &lt; 1 and > -1 for example).</p>

<p>The output audio file is evaluated by a tool that gives it a score. This tool knows what the "ideal" output should sound like, and scores the output file accordingly. A score of 1.0 means the output is ideal, i.e. the input file was processed with the best possible parameter set. A score of 0 means the output is completely wrong.</p>

<p>So with 10-15 parameters with their valid ranges, the combinations are endless! I'd be sitting here manually tweaking these parameters forever until I got the best solution. I've checked out some LP/MIP solvers (CBC, MS Solver Foundation, GKLP) but these use a mathematical equation as an objective function... you don't "plug in" an external evaluation function as far as I can see.</p>

<p>Is a LP/MIP solver the right tool to aid in the parameter tuning? Any ideas?</p>

<p>Thanks,</p>

<p>akevan</p>