title: How can I retrieve files with User-Agent headers in Python 3?
body: <p>I'm trying to write a (simple) piece of code to download files off the internet. The problem is, some of these files are on websites that block the default python User-Agent headers. For example:</p>
  
  <pre><code>import urllib.request as html
  html.urlretrieve('http://stackoverflow.com', 'index.html')
  </code></pre>
  
  <p>returns</p>
  
  <pre><code>urllib.error.HTTPError: HTTP Error 403: Forbidden`
  </code></pre>
  
  <p>Normally, I would set the headers in the request, such as:</p>
  
  <pre><code>import urllib.request as html
  request = html.Request('http://stackoverflow.com', headers={"User-Agent":"Firefox"})
  response = html.urlopen(request)
  </code></pre>
  
  <p>however, as <code>urlretrieve</code> doesn't work with requests for some reason, this isn't an option.</p>
  
  <p>Are there any simple-ish solutions to this (that don't include importing a library such as requests)? I've noticed that urlretrieve is part of the legacy interface posted over from Python 2, is there anything I should be using instead?</p>
  
  <p>I tried creating a custom FancyURLopener class to handle retrieving files, but that caused more problems than it solved, such as creating empty files for links that 404.</p>