title: urlretrieve seems to corrupt image file
body: <p>I use urlretrieve to scrape images from a web site. That works well except for one, not so minor detail. The files are not readable. I tried several websites but the result is the same. I wondered whether I should indicate it is a binary download but could not find any hint in the documentation. Searched the web, found some alternative with the requests library but again same results. The Windows Photo Viewer, Paint and Gimp all report that the file is corrupted or unreadable. I am pretty sure I am making some dumb mistake. Any help will be greatly appreciated!</p>
  
  <pre><code>def get_images(url, soup):
      #this makes a list of bs4 element tags
      print 'URL: ', url
      n = 0
      images = [img for img in soup.findAll('img')]
  
      #compile our unicode list of image links
      image_links = [each.get('src') for each in images]
      for each in image_links:
          n = n + 1
          path = urlparse.urlparse(each).path
          fn = (os.path.split(path)[1]).strip()
          ext = (os.path.splitext(fn)[1]).strip().lower()
          if (fn == '' or ext == ''):
              continue
  
          fn = os.path.join ("images", fn)
  
  #        print 'From: ', url
          print 'Each&gt; ', each
  #        print 'File&lt; ', fn
  #        avatar = open(fn, 'wb')
  #        avatar.write(requests.get(url).content)
  #        avatar.close()
          result = urllib.urlretrieve(url, fn)
          print result
  
      return n
  </code></pre>
  
  <p><strong>Update</strong></p>
  
  <p>Jephron pointed me to the right direction, I did not correctly combine the url with the image path. His solution works by using <code>urlparse.urljoin(url, each)</code> while using <code>os.path.join</code> as I did initially, may result in suddenly having backslashed in a url on a windows system. Quite annoying. I added a test for relative and absolute url paths and the final code is shown below.</p>
  
  <pre><code>def get_images(url, soup):
      #this makes a list of bs4 element tags
      print ' '
      print 'URL: ', url
      n = 0
      images = [img for img in soup.findAll('img')]
  
      #compile our unicode list of image links
      image_links = [each.get('src') for each in images]
  
      for each in image_links:
          path = urlparse.urlparse(each).path
          fn = (os.path.split(path)[1]).strip()
          ext = (os.path.splitext(fn)[1]).strip().lower()
          if (fn == '' or ext == ''):
              continue
  
          fn = os.path.join ("images", fn)
          if (not (each.startswith ('http:') or each.startswith('https:'))):
              image_link = urlparse.urljoin(url, each)
          else:
              image_link = each
  
          print 'Found: ', fn
  
          try:
              urllib.urlretrieve(image_link, fn)
              n = n + 1
          except:
              continue
  
      return n
  </code></pre>
  
  <p>Be aware however, that 3/4 of the .png's are still unreadable. I have to find out why, but there might still be a hidden snag. </p>