title: Downloading all links on a webpage using Mechanize in Python
body: <p>I was trying to follow the following thread which seemed to answer my question. It serves as a great example that shows how to download all links on a webpage using Mechanize:</p>
  
  <p><a href="https://stackoverflow.com/questions/5974595/download-all-the-linksrelated-documents-on-a-webpage-using-python">Download all the links(related documents) on a webpage using Python</a></p>
  
  <p>I followed the code that was posted (i.e.):</p>
  
  <pre><code>import mechanize
  from time import sleep
  #Make a Browser (think of this as chrome or firefox etc)
  br = mechanize.Browser()
  
  #visit http://stockrt.github.com/p/emulating-a-browser-in-python-with-mechanize/
  #for more ways to set up your br browser object e.g. so it look like mozilla
  #and if you need to fill out forms with passwords.
  
  # Open your site
  br.open('http://pypi.python.org/pypi/xlwt')
  
  f=open("source.html","w")
  f.write(br.response().read()) #can be helpful for debugging maybe
  
  filetypes=[".zip",".exe",".tar.gz"] #you will need to do some kind of pattern matching on your files
  myfiles=[]
  for l in br.links(): #you can also iterate through br.forms() to print forms on the page!
      for t in filetypes:
          if t in str(l): #check if this link has the file extension we want (you may choose to use reg expressions or something)
              myfiles.append(l)
  
  
  def downloadlink(l):
      f=open(l.text,"w") #perhaps you should ensure that file doesn't already exist.
  
      br.click_link(l)
      f.write(br.response().read())
      print l.text," has been downloaded"
      #br.back()
  
  for l in myfiles:
      sleep(1) #throttle so you dont hammer the site
      downloadlink(l)
  </code></pre>
  
  <p>i only changed:</p>
  
  <pre><code>f=open(l.text,"w") #perhaps you should open in a better way &amp; ensure that file doesn't already exist.
  </code></pre>
  
  <p>To:</p>
  
  <pre><code>f=open('C:\\l.text',"w") #perhaps you should open in a better way &amp; ensure that file doesn't already exist.
  </code></pre>
  
  <p>That made the code work for me, else it was giving me an error. When i run the code, i get the following output:</p>
  
  <pre><code>Download&gt; xlwt-0.7.5.tar.gz has been downloaded 
  xlwt-0.7.5.tar.gz has been downloaded
  </code></pre>
  
  <p>So it worked. But i have no idea where this file was downloaded to? Any ideas? I have searched my C drive, and could not find it.</p>
  
  <p>If the code is run as:</p>
  
  <pre><code>f=open(l.text,"w")
  </code></pre>
  
  <p>It raises the following exception:</p>
  
  <pre><code>Traceback (most recent call last):
    File "C:\Python27\mech.py", line 33, in &lt;module&gt;
  downloadlink(l)
    File "C:\Python27\mech.py", line 25, in downloadlink
  f=open(l.text,"w") #perhaps you should ensure that file doesn't already exist.
  IOError: [Errno 22] invalid mode ('w') or filename: 'Download&gt; &lt;span style="font-size: 75%"&gt;xlwt-0.7.5.tar.gz&lt;span&gt;'
  </code></pre>