title: Scraping using Mechanize and BS4
body: <p>I am trying to scrape articles from the Wall Street Journal. This involves logging in using mechanize and scraping using BeautifulSoup. I was hoping someone could take a look at my code and explain to me why it's not working. </p>
  
  <p>I am using python 2.7 on a 2012 MacBook Pro running the latest software. I'm new to python so explain to me like I'm 5. Any advice would be deeply appreciated. Thanks in advance.</p>
  
  <pre><code>from bs4 import BeautifulSoup
  import cookielib
  import mechanize
  
  #Browser
  br = mechanize.Browser()
  
  #Cookie Jar
  cj = cookielib.LWPCookieJar()
  br.set_cookiejar(cj) 
  
  # Browser options
  br.set_handle_equiv(True)
  br.set_handle_gzip(True)
  br.set_handle_redirect(True)
  br.set_handle_referer(True)
  br.set_handle_robots(False) 
  
  # User-Agent 
  br.addheaders = [('User-agent', 'Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.1) Gecko/2008071615 Fedora/3.0.1-1.fc9 Firefox/3.0.1')]
  
  # Follows refresh 0 but not hangs on refresh &gt; 0
  br.set_handle_refresh(mechanize._http.HTTPRefreshProcessor(), max_time=1)
  
  br.set_debug_http(True) # Print HTTP headers.
  # Want more debugging messages?
  #br.set_debug_redirects(True)
  #br.set_debug_responses(True)
  
  # The site we will navigate into, handling it's session
  br.open('https://id.wsj.com/access/pages/wsj/us/login_standalone.html?mg=id-wsj')
  
  # Select the first (index zero) form
  br.select_form(nr=0)
  
  # User credentials
  br.form['username'] = 'username'
  br.form['password'] = 'password' 
  
  # Login
  br.submit()
  
  #br.open("http://online.wsj.com/home-page")
  br.open("http://online.wsj.com/news/articles/SB10001424052702304626304579506924089231470?mod=WSJ_hp_LEFTTopStories&amp;mg=reno64-wsj&amp;url=http%3A%2F%2Fonline.wsj.com%2Farticle%2FSB10001424052702304626304579506924089231470.html%3Fmod%3DWSJ_hp_LEFTTopStories&amp;cb=logged0.9458705162058179")
  soup = BeautifulSoup(br.response().read())
  title = soup.find('h1')
  print title
  </code></pre>