title: urllib2 HTTP error 429
body: <p>So I have a list of sub-reddits and I'm using urllib to open them. As I go through them eventually urllib fails with:</p>
  
  <pre><code>urllib2.HTTPError: HTTP Error 429: Unknown
  </code></pre>
  
  <p>Doing some research I found that reddit limits the ammount of requests to their servers by IP:</p>
  
  <blockquote>
    <p>Make no more than one request every two seconds. There's some allowance for bursts of requests, but keep it sane. In general, keep it to no more than 30 requests in a minute.</p>
  </blockquote>
  
  <p>So I figured I'd use <code>time.sleep()</code> to limit my requests to one page each 10 seconds. This ends up failing just as well.</p>
  
  <p>The quote above is grabbed from the <a href="https://github.com/reddit/reddit/wiki/API" rel="noreferrer">reddit API</a> page. I am not using the reddit API. At this point I'm thinking two things. Either that limit applies only to the reddit API or urllib also has a limit. </p>
  
  <p>Does anyone know which one of these two things it is? Or how I could go around this issue?</p>