title: OpenCL, how do I generalize code for large computations?
body: <p>I am confused about how OpenCL divides a massive computation task. I am unsure about how to ask the questions correctly, so please let me start with how I think OpenCL works, and then I will ask a specific question based on the example. Here's how I think it works: </p>
  
  <ul>
  <li>load in the kernel</li>
  <li>set the arguments</li>
  <li>choose the number of work items   <strong>&lt;- my question will be about this step</strong></li>
  <li>and then run the kernel.</li>
  </ul>
  
  <hr>
  
  <p>Suppose I want to use a kernel (whatever it is) to solve a given number of problems. I manually set <code>local_item_size</code>, and then set <code>global_item_size</code> so that it is one multiple of <code>local_item_size</code> larger than <code>number of problems</code>. Like this:</p>
  
  <pre><code>// Set work item population 
  local_item_size = 8;
  global_item_size = (number_of_problems / local_item_size + 1) * local_item_size;
  
  // Run
  clEnqueueNDRangeKernel(queue, kernel, 1, NULL, &amp;global_item_size, &amp;local_item_size, 0, NULL, NULL);
  </code></pre>
  
  <p>So, if there are <code>number_of_problems = 25</code> then <code>global_item_size = 32</code>. This is because the smallest multiple of <code>8</code> that is greater than <code>25</code> is <code>32</code>. Okay, simple enough, the kernel runs and we ignore the results of some items, job done.</p>
  
  <hr>
  
  <p>Now let's suppose I want my to do the same, but for massive numbers of work items. As an example, say I want to run the kernel for every combination of indices in some large array:</p>
  
  <pre><code>number_of_problems = getCombinations(input);
  
  // Set work item population 
  local_item_size = _____;
  global_item_size = (number_of_problems / local_item_size + 1) * local_item_size;
  </code></pre>
  
  <p>If <code>input</code> is large, the <code>number_of_problems</code> will be very large - you get the idea I think. I want to understand <strong>how</strong> to go about choosing <code>local_item_size</code> wisely.</p>
  
  <hr>
  
  <p>Finally, my two-part question:</p>
  
  <p>1) When using a GPU, does OpenCL:</p>
  
  <ul>
  <li>a) run some set of work-groups, then another set, and so on until all work items have finished, or</li>
  <li>b) simply try to run all of the work-groups at once?</li>
  </ul>
  
  <p>If the answer is <strong>a</strong>, then how can I set <code>local_item_size</code> <strong>wisely</strong> so as use as much resource as possible (without having work groups that are too large)?</p>
  
  <p>If the answer is <strong>b</strong>, then I'm confused (I thought OpenCL has some magic to split the work items up automatically)! Should I re-build the kernel and run it a number of times, changing the input accordingly? If that's the best approach, then how do I choose the number of times to re-build the kernel <strong>wisely</strong>?</p>
  
  <p>Thanks for reading, I look forward to figuring this out!</p>