title: Max number of concurrent HttpWebRequests
body: <p>I'm stress testing a web app and have set up a windows test program that spins up a number of threads and sends out a web request on each one.</p>
  
  <p>Problem is I get the following output:</p>
  
  <pre><code>01/09/09 11:34:04 Starting new HTTP request on 10
  01/09/09 11:34:04 Starting new HTTP request on 11
  01/09/09 11:34:04 Starting new HTTP request on 13
  01/09/09 11:34:05 Starting new HTTP request on 14
  01/09/09 11:34:05 Starting new HTTP request on 11
  01/09/09 11:34:05 11 has finished!
  01/09/09 11:34:05 Starting new HTTP request on 13
  01/09/09 11:34:05 13 has finished!
  01/09/09 11:34:05 Starting new HTTP request on 14
  01/09/09 11:34:05 14 has finished!
  01/09/09 11:34:05 Starting new HTTP request on 11
  01/09/09 11:34:05 11 has finished!
  01/09/09 11:34:05 Starting new HTTP request on 14
  01/09/09 11:34:05 14 has finished!
  01/09/09 11:34:05 Starting new HTTP request on 13
  01/09/09 11:34:05 13 has finished!
  01/09/09 11:34:05 Starting new HTTP request on 15
  01/09/09 11:34:06 Starting new HTTP request on 11
  01/09/09 11:34:06 11 has finished!
  01/09/09 11:34:06 Starting new HTTP request on 14
  01/09/09 11:34:06 14 has finished!
  </code></pre>
  
  <p>which sort of looks like there's a maximum of 5 threads, even if I create 100 as so:</p>
  
  <pre><code>int numberOfThreads = Convert.ToInt32(txtConcurrentThreads.Text);
  
      List&lt;BackgroundWorker&gt; workers = new List&lt;BackgroundWorker&gt;();
  
      for (int N = 0; N &lt; numberOfThreads; N++)
      {
  
          BackgroundWorker worker = new BackgroundWorker();
          worker.DoWork += new DoWorkEventHandler(worker_DoWork);
          worker.RunWorkerCompleted += new RunWorkerCompletedEventHandler(worker_RunWorkerCompleted);
          workers.Add(worker);
      }
  
  
      foreach(BackgroundWorker worker in workers)
      {
          worker.RunWorkerAsync();
      }
  </code></pre>
  
  <p>Can anyone enlighten me as to what is going on?</p>
  
  <p>Thanks</p>
  
  <p>EDIT: If as suggested I sleep for 5 seconds, instead of httpwebrequest, then I do get more threads firing but not as many as I would have expected:</p>
  
  <pre><code>01/09/09 11:56:14 Starting new HTTP request on 7
  01/09/09 11:56:14 Starting new HTTP request on 11
  01/09/09 11:56:15 Starting new HTTP request on 12
  01/09/09 11:56:15 Starting new HTTP request on 13
  01/09/09 11:56:16 Starting new HTTP request on 14
  01/09/09 11:56:16 Starting new HTTP request on 15
  01/09/09 11:56:17 Starting new HTTP request on 16
  01/09/09 11:56:17 Starting new HTTP request on 17
  01/09/09 11:56:18 Starting new HTTP request on 18
  01/09/09 11:56:19 Starting new HTTP request on 7
  01/09/09 11:56:19 7 has finished!
  01/09/09 11:56:19 Starting new HTTP request on 11
  01/09/09 11:56:19 11 has finished!
  01/09/09 11:56:19 Starting new HTTP request on 19
  01/09/09 11:56:20 Starting new HTTP request on 20
  01/09/09 11:56:20 Starting new HTTP request on 12
  01/09/09 11:56:20 12 has finished!
  </code></pre>
  
  <p>It still looks like I'm only getting 2 threads starting every second, which seems mighty slow to me.  I suppose the Console.WriteLine could be a problem?</p>
  
  <p>EDIT: I set </p>
  
  <pre><code>ThreadPool.SetMinThreads(100, 4); 
  </code></pre>
  
  <p>and </p>
  
  <pre><code>System.Net.ServicePointManager.DefaultConnectionLimit = 100;
  </code></pre>
  
  <p>and got the following results:</p>
  
  <pre><code>01/09/09 14:00:07 Starting new HTTP request on 11
  01/09/09 14:00:07 Starting new HTTP request on 81
  01/09/09 14:00:07 Starting new HTTP request on 82
  01/09/09 14:00:07 Starting new HTTP request on 79
  01/09/09 14:00:07 Starting new HTTP request on 83
  01/09/09 14:00:07 Starting new HTTP request on 84
  01/09/09 14:00:07 Starting new HTTP request on 85
  01/09/09 14:00:07 Starting new HTTP request on 87
  01/09/09 14:00:07 Starting new HTTP request on 88
  ...
  01/09/09 14:00:07 84 has finished! Took 323.0323 milliseconds
  01/09/09 14:00:08 88 has finished! Took 808.0808 milliseconds
  01/09/09 14:00:08 96 has finished! Took 806.0806 milliseconds
  01/09/09 14:00:08 94 has finished! Took 806.0806 milliseconds
  01/09/09 14:00:08 98 has finished! Took 801.0801 milliseconds
  01/09/09 14:00:08 80 has finished! Took 799.0799 milliseconds
  01/09/09 14:00:08 86 has finished! Took 799.0799 milliseconds
  01/09/09 14:00:08 92 has finished! Took 799.0799 milliseconds
  01/09/09 14:00:08 100 has finished! Took 812.0812 milliseconds
  01/09/09 14:00:08 82 has finished! Took 1010.101 milliseconds
  </code></pre>
  
  <p>so was able to push out a whole lot of web requests concurrently.  Which seemed to queue (calling out to an STA COM+ server) so that's what I expected.</p>
  
  <p>Thanks for your help</p>