title: Algorithm or formula to find good tuning values for searching
body: <p>I have a programming search problem and I am wondering if there are any algorithm, class, formula or procedure that can produce good search locations based on past results.   (I’m guessing there is somewhere.) Or, would the solution I threw out there be good?</p>
  
  <p>Let me try to explain with a simple example: Say there is a pond that that is 2 X 2 meters and 3 meters deep.  I can basically put my fishing lure at any of the x,y,z locations (2 X 2 X 3 = 27 locations).  Say I fish at each location for one hour (testing out the pond) and I ketch a different amount of fish at each of the 27 locations. Now, after I do that, the best place to logically fish is the location I caught the most fish BUT just because I caught the most fish there it does not mean it’s the best spot.  I could have just been lucky.  It would probably might be better to spend a big chunk of my time at that location but still adventure out a percentage of the time to confirm that is the best place.  </p>
  
  <p>One simple (and bad?) solution is just to fish 10 hour in every location and wherever the most fish are caught would probably be a good location but that would be a lot of wasted time(270 hours). Chances are if I ketch 15 finish at some x,y,z and none at x2,y2,z2 then I should not spend much time at the x2,y2,z2.</p>
  
  <p>A second solution I was thinking about was to keep a tally of the hours spent and total fish caught at each location. And then do something like: (simple example)</p>
  
  <pre><code>float catchesByLocation[2,2,3] = {1}; //init all to 1
  float totalTimeSpentByLocation[2,2,3] = {1}; //init all to 1
  
  While(true) //never really ends
  {
    Do x = 0 to 2 
      Do y = 0 to 2 
        Do z = 0 to 3  //depth
        {
          float timeToSpendAtThisLoc = catchesByLocation[x,y,z] / totalTimeSpentByLocation[x,y,z];
          float catches = GoFishing(x,y,z);
          catchesByLocation[x,y,z] = catchesByLocation[x,y,z] + catches;
          totalTimeSpentByLocation[x,y,z] = totalTimeSpentByLocation[x,y,z] + timeToSpendAtThisLoc;
        }
  }
  </code></pre>
  
  <p>With this solution, some amount of time will always be spend on the bad locations but as time goes on the bad locations will get a very small fraction of the total time.</p>
  
  <p>So the question I have - is there some logical approach to do this?  Maybe there is even an exact correct way to solve this using math? Any thoughts on ways to attack this problem?  Sorry for the bad title, I cannot think of how to title it and am open to suggestions. Thank you for reading my question.</p>