title: Python follow redirects and then download the page?
body: <p>I have the following python script and it works beautifully. </p>
  
  <pre><code>import urllib2
  
  url = 'http://abc.com' # write the url here
  
  usock = urllib2.urlopen(url)
  data = usock.read()
  usock.close()
  
  print data
  </code></pre>
  
  <p>however, some of the URL's I give it may redirect it 2 or more times. How can I have python wait for redirects to complete before loading the data. 
  For instance when using the above code with </p>
  
  <pre><code>http://www.google.com/search?hl=en&amp;q=KEYWORD&amp;btnI=1
  </code></pre>
  
  <p>which is the equvilant of hitting the im lucky button on a google search, I get:</p>
  
  <pre><code>&gt;&gt;&gt; url = 'http://www.google.com/search?hl=en&amp;q=KEYWORD&amp;btnI=1'
  &gt;&gt;&gt; usick = urllib2.urlopen(url)
  Traceback (most recent call last):
    File "&lt;stdin&gt;", line 1, in &lt;module&gt;
    File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 126, in urlopen
      return _opener.open(url, data, timeout)
    File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 400, in open
      response = meth(req, response)
    File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 513, in http_response
      'http', request, response, code, msg, hdrs)
    File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 438, in error
      return self._call_chain(*args)
    File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 372, in _call_chain
      result = func(*args)
    File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 521, in http_error_default
      raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
  urllib2.HTTPError: HTTP Error 403: Forbidden
  &gt;&gt;&gt; 
  </code></pre>
  
  <p>Ive tried the (url, data, timeout) however, I am unsure what to put there.</p>
  
  <p>EDIT:
  I actually found out if I dont redirect and just used the header of the first link, I can grab the location of the next redirect and use that as my final link</p>