title: Non Blocking Curl requests in PHP
body: <p>So I have this function for making non-blocking curl requests. It works fine on what I've tested so far (small amounts of requests). But I need this to scale up to thousands of requests (maybe max 10,000). My issue is that I don't want to run into issues with too many parallel requests running at once.</p>
  
  <p>What would you suggest to rate-limit the requests? Usleep? Requests in batches? The function is below:</p>
  
  <pre><code>function poly_curl($requests){
  
              $queue = curl_multi_init(); 
              $curl_array = array(); 
              $count = 0;
              foreach($requests as $request) 
              { 
                  $curl_array[$count] = curl_init($request); 
                  curl_setopt($curl_array[$count], CURLOPT_RETURNTRANSFER, true); 
                  curl_multi_add_handle($queue, $curl_array[$count]); 
                  $count++;
              } 
  
              $running = NULL; 
              do { 
  
                 curl_multi_exec($queue,$running); 
  
              } while($running &gt; 0); 
  
              $res = array(); 
              $count = 0;
              foreach($requests as $request) 
              { 
                  $res[$count] = curl_multi_getcontent($curl_array[$count]); 
                  $count++;
              } 
  
              $count = 0;
              foreach($requests as $request){ 
                  curl_multi_remove_handle($queue, $curl_array[$count]); 
                  $count++;
              } 
              curl_multi_close($queue);        
              return $res; 
      }
  </code></pre>