title: accessing a list of urls via Multi-processing in Python2.7
body: <p>I've been playing with multiprocessing and my code works when looking at smaller numbers but when I want to run a larger sample 2 things happen: Either the code locks up or I get the following error message: "urlopen error [Errno 10054] An existing connection was forcibly closed by the remote host" . I can't figure out how to get it to work. Thanks.</p>
  
  <pre><code>    from multiprocessing import cpu_count
      import urllib2
      from bs4 import BeautifulSoup
      import json
      import timeit
      import socket
      import errno
  
      def parseWeb(id):
          url = 'https://carhood.com.au/rent/car_detail/'+str(id)+'/'
          hdr = {'Accept': 'text/html,application/xhtml+xml,*/*',"user-agent":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/48.0.2564.116 Safari/537.36"}
  
          html = urllib2.urlopen(url).read()
          soup=BeautifulSoup(html,"lxml")
          car=soup.find("h1",{"class":"intro-title intro-title-tertiary"}).text
  
          return car
  
      if __name__ == '__main__':
          start = timeit.default_timer()
          pool = Pool(cpu_count()*100)
  #This works for me when xrange(1,70)
          results=pool.map(parseWeb,xrange(1,400))
          print results
  ##I've tried this as a solution but it didn't work
          ##    startNum=1
  ##    endNum=470
  ##    for x in range(startNum,endNum,70):
  ##        print x
  ##        results=pool.map(parseWeb,xrange(startNum,x))
  ##        print results
  ##        startNum=x  
          stop = timeit.default_timer()
          print stop - start
  </code></pre>