title: Using gevent with pyramid
body: <p>I'm building a website using pyramid, and I want to fetch some data from other websites.  Because there may be 50+ calls of <code>urlopen</code>, I wanted to use gevent to speed things up.</p>
  <p>Here's what I've got so far using gevent:</p>
  <pre><code>import urllib2    
  from gevent import monkey; monkey.patch_all()
  from gevent import pool
  
  gpool = gevent.pool.Pool()
  
  def load_page(url):
      response = urllib2.urlopen(url)
      html = response.read()
      response.close()
      return html
  
  def load_pages(urls):
      return gpool.map(load_page, urls)
  </code></pre>
  <p>Running <code>pserve development.ini --reload</code> gives:</p>
  <p><code>NotImplementedError: gevent is only usable from a single thread</code>.</p>
  <p>I've read that I need to monkey patch before anything else, but I'm not sure where the right place is for that.  Also, is this a pserve-specific issue?  Will I need to re-solve this problem when I move to <a href="http://docs.pylonsproject.org/projects/pyramid/en/latest/tutorials/modwsgi/index.html" rel="nofollow noreferrer">mod_wsgi</a>?  Or is there a way to handle this use-case (just urlopen) without gevent?  I've seen suggestions for <a href="http://docs.python-requests.org/en/latest/" rel="nofollow noreferrer">requests</a> but I couldn't find an example of fetching multiple pages in the docs.</p>
  <h3>Update 1:</h3>
  <p>I also tried eventlet from <a href="https://stackoverflow.com/a/2361129/312364">this SO question</a> (almost directly copied from this eventlet <a href="http://eventlet.net/doc/design_patterns.html#client-pattern" rel="nofollow noreferrer">example</a>):</p>
  <pre><code>import eventlet
  from eventlet.green import urllib2
  
  def fetch(url):
      return urllib2.urlopen(url).read()
  
  def fetch_multiple(urls):
      pool = eventlet.GreenPool()
      return pool.imap(fetch, urls)
  </code></pre>
  <p>However when I call <code>fetch_multiple</code>, I'm getting <code>TypeError: request() got an unexpected keyword argument 'return_response'</code></p>
  <h3>Update 2:</h3>
  <p>The <code>TypeError</code> from the previous update was likely from earlier attempts to monkeypatch with gevent and not properly restarting pserve.  Once I restarted everything, it works properly.  Lesson learned.</p>