title: Webscraping with Xpath in Python
body: <p>From what I've seen the method to derive a path for Xpath to scrape a page  is not totally clear to me. I'm trying to use Xpath in python to scrape the wikipedia article for states and capitals to get a list of states and a list of capitals, but so far I've had no luck when trying to figure out the correct path to use. I've tried inspecting the element and copying the Xpath there but I still have had no luck. I'm looking for someone to explain a method to figure out the correct xpath to use to grab certain elements in a page.</p>
  
  <pre><code>from lxml import html
  import requests
  
  page = requests.get('https://en.wikipedia.org/wiki/List_of_capitals_in_the_United_States')
  tree = html.fromstring(page.text)
  
  #creating list of states
  state = tree.xpath('xpath')
  #list of capitals
  capital = tree.xpath('xpath')
  
  print 'State: ', state
  print 'Capital: ', capital
  </code></pre>
  
  <p>Two of the xpaths I've tried so far have been:</p>
  
  <pre><code>//*[@id="mw-content-text"]/table[1]/tbody/tr[1]/td[1]/a
  
  //*[@id="mw-content-text"]/table[1]/tbody/tr[1]/td[2]
  </code></pre>