title: curl_multi_getcontent returns random NULL value from 1000+ urls
body: <p>Im having a problem retrieving json data using multi <code>curl</code> from several url generated from database, If I limit the query from 100 to 500 links the issue will not occur, but when the links reaches 1000+, im starting to get random NULL return from <code>curl_multi_getcontent</code>.</p>
  
  <p>The multi curl function:</p>
  
  <pre><code>function curlMultiExec($nodes)
  {     
    $node_count = count($nodes);     
    $ch_arr = array();
  
    $master = curl_multi_init();
  
    for($i = 0; $i &lt; $node_count; $i++)
    {
        $url        = $nodes[$i]['url'];
        $ch_arr[$i] = curl_init($url);
  
        curl_setopt($ch_arr[$i], CURLOPT_RETURNTRANSFER, TRUE);  
        curl_setopt($ch_arr[$i], CURLOPT_BINARYTRANSFER, TRUE);
        curl_setopt($ch_arr[$i], CURLOPT_FOLLOWLOCATION, TRUE);  
        curl_setopt($ch_arr[$i], CURLOPT_AUTOREFERER,    TRUE); 
        curl_setopt($ch_arr[$i], CURLOPT_HEADER,         FALSE);
        curl_multi_add_handle($master, $ch_arr[$i]);
    }
  
    $running = null;
    do 
    {
      curl_multi_exec($master,$running);
    } while( $running &gt; 0 );
  
  
    $obj = array();
    for($i = 0; $i &lt; $node_count; $i++ )
    {
         $item = array(              
             'url'        =&gt; $nodes[$i]['url'],
             'content'    =&gt; curl_multi_getcontent($ch_arr[$i])              
          ); 
          array_push($obj, $item);       
  }        
  curl_multi_close($master);  
  return $obj;
  }
  </code></pre>
  
  <p>Currently the <code>$nodes[$i]['url']</code> contains 1,912 url. </p>
  
  <p>The output using <code>print_r</code></p>
  
  <pre><code>Array
      (
         [0] =&gt; Array
         (
          [url] =&gt; http://api.worldbank.org/countries/AFG/indicators/NY.GDP.MKTP.CD?per_page=100&amp;date=1960:2014&amp;format=json
          [content] =&gt; [{ /* json data */ }]
      )
  
  [1] =&gt; Array
    (
          [url] =&gt; http://api.worldbank.org/countries/ALB/indicators/NY.GDP.MKTP.CD?per_page=100&amp;date=1960:2014&amp;format=json
          [content] =&gt;   // -&gt; here's the sample null value
    )
  
    .
    .  //-&gt; and some random [content] here also contains null value 
    .
    .
    [1191] =&gt; Array
    (
          [url] =&gt; http://api.worldbank.org/countries/ZWE/indicators/NY.GDP.MKTP.CD?per_page=100&amp;date=1960:2014&amp;format=json
          [content] =&gt; [{ /* json data */ }]
    )
  )
  </code></pre>
  
  <p>Kindly enlightened me why it returns random null value, Or what causes this behavior or is there a better approach than this?</p>
  
  <hr>
  
  <p>UPDATE (2014-02-20);
   I found the solution here : <a href="http://www.onlineaspect.com/2009/01/26/how-to-use-curl_multi-without-blocking" rel="nofollow">curl_multi() without blocking</a>  </p>
  
  <p>The problem is that most implementations of curl_multi wait for each set of requests to complete before processing them. If there are too many requests to process at once, they usually get broken into groups that are then processed one at a time.</p>
  
  <p>The solution is to process each request as soon as it completes. This eliminates the wasted CPU cycles from busy waiting. </p>
  
  <p>Based on his approach, I successfully solve this. Maybe this post can help, just in case someone stumble the same issue.</p>
  
  <p>Cheers!</p>