title: python-requests with multithreading
body: <p>I am working on creating a HTTP client which can generate hundreds of connections each second and send up to 10 requests on each of those connections. I am using threading so concurrency can be achieved.
  Here is my code:</p>
  
  <pre><code>def generate_req(reqSession):
      requestCounter = 0
      while requestCounter &lt; requestRate:
          try:
              response1 = reqSession.get('http://20.20.1.2/tempurl.html')
              if response1.status_code == 200:
                  client_notify('r')
          except(exceptions.ConnectionError, exceptions.HTTPError, exceptions.Timeout) as Err:
              client_notify('F')
              break
          requestCounter += 1
  
  def main():
      for q in range(connectionPerSec):
          s1 = requests.session()
          t1 = threading.Thread(target=generate_req, args=(s1,))
          t1.start()
  </code></pre>
  
  <p>Issues:</p>
  
  <ol>
  <li><p>It is not scaling above 200 connections/sec with requestRate = 1. I ran other available HTTP clients on the same client machine and against the server, test runs fine and it is able to scale.</p></li>
  <li><p>When requestRate = 10, connections/sec drops to 30.
  Reason: Not able to create targeted number of threads every second.</p></li>
  </ol>
  
  <p>For issue #2, client machine is not able to create enough request sessions and start new threads. As soon as requestRate is set to more than 1, things start to fall apart.
  I am suspecting it has something to do with HTTP connection pooling which requests uses.</p>
  
  <p>Please suggest what am I doing wrong here.</p>