title: How to modify Pandas's Read_html user-agent?
body: <p>I'm trying to scrape English football stats from various html tables via the <a href="http://www.transfermarkt.co.uk/en/premier-league/gegentorminuten/wettbewerb_GB1.html" rel="nofollow">Transfetmarkt</a> website using the pandas.read_html() function. </p>
  
  <p>Example:</p>
  
  <pre><code>import pandas as pd
  url = r'http://www.transfermarkt.co.uk/en/premier-league/gegentorminuten/wettbewerb_GB1.html'
  df = pd.read_html(url)
  </code></pre>
  
  <p>However this code generates a <strong>"ValueError: Invalid URL"</strong> error. </p>
  
  <p>I then attempted to parse the same website using the urllib2.urlopen() function. This time i got a "HTTPError: HTTP Error 404: Not Found". After the usual trial and error fault finding, it turns that the urllib2 header presents a python like agent to the webserver, which i presumed it doesn't recognize.  </p>
  
  <p>Now if I modify urllib2's agent and read its contents using beautifulsoup, i'm able to read the table without a problem.</p>
  
  <p>Example:</p>
  
  <pre><code>from BeautifulSoup import BeautifulSoup
  import urllib2
  opener = urllib2.build_opener()
  opener.addheaders = [('User-agent', 'Mozilla/5.0')]
  url = r'http://www.transfermarkt.co.uk/en/premier-league/gegentorminuten/wettbewerb_GB1.html'
  response = opener.open(url)
  html = response.read()
  soup = BeautifulSoup(html)
  table = soup.find("table")
  </code></pre>
  
  <p>How do I modify pandas's urllib2 header to allow python to scrape this website? </p>
  
  <p>Thanks </p>