title: python urlib2 fails when calling variable
body: <p>I am scrapping data using beautiful soup. I have a list of urls I want to loop my code through, so I need to include a variable in the <code>urllib2.Request</code> command. When I add a variable to <code>urllib2.Request</code> I get this error (line 1240 of urllib2.py):</p>
  
  <pre><code>raise URLError('unknown url type: %s' % type)
  </code></pre>
  
  <p>Here is my code:</p>
  
  <pre><code>from bs4 import BeautifulSoup
  import urllib2
  
  webstring = "/DIRECTORY/"+"'"  
  webfull = "urllib2.Request('http://www.caao.org"+webstring+", None, headers)"
  
  print webfull
  
  #webfull prints: urllib2.Request('http://www.caao.org/DIRECTORY/', None, headers)
  
  headers = { 'User-Agent' : 'Mozilla/5.0' }
  html = urllib2.urlopen(webfull).read()
  soup = BeautifulSoup(html)
  
  print soup
  </code></pre>
  
  <p>The variable <code>webfull</code> prints out the correct code. I can cut and paste it into urlopen and it will work. Just like this:</p>
  
  <pre><code>from bs4 import BeautifulSoup
  import urllib2
  
  headers = { 'User-Agent' : 'Mozilla/5.0' }
  html = urllib2.urlopen(urllib2.Request('http://www.caao.org/DIRECTORY/', None, headers)).read()
  soup = BeautifulSoup(html)
  
  print soup
  </code></pre>
  
  <p>I've tried using multiple websites as tests, and I have tried using triple-double quotes on certain strings (like below), but I always get the same <code>unknown url type</code> error. </p>
  
  <pre><code>webstring = "/DIRECTORY/"+"'"
  web1 = """'http://www.caao.org"""+webstring+", None, headers)"
  </code></pre>
  
  <hr>
  
  <p>As a side note: </p>
  
  <p>I'm new to python and I trying to scrape data from multiple pages within the same website. The code above is to let me run down my list of URLs to run my beautiful soup code on each page. If there is an easier way to loop through a list of URLs and use <code>urllib2.urlopen</code> to open each page so I can run my scraping code, let me know. </p>