title: nltk in python to extract information from web page
body: <p>how can I use nltk in python to extract information from web page (job offers in my case)</p>
  
  <p>I use this code to extract a part of text , </p>
  
  <pre><code>import nltk
  import time
  import urllib2 
  from urllib2 import urlopen 
  from cookielib import CookieJar 
  import datetime
  
  
  website = "http://tanitjobs.com/search-results-jobs/"
  topSplit = "&lt;div class=\"offre\"&gt;"
  ButtomSplit = "&lt;div class=\"offre-emploi&amp;nbsp;\"&gt;"
  cj = CookieJar()
  opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(cj))
  opener.addheaders = [('Uer-agent', 'Mozilla/5.0')]
  
  def main():
  
      try:
  
          ss =opener.open(website).read()
          sourceCodeSplit = ss.split(topSplit)[1].split(ButtomSplit)[0]
          texte = nltk.clean_html(sourceCodeSplit)
          print texte
      except Exception,e:
          print 'fail in the main loop'
          print str(e)
  
  
  main()
  </code></pre>
  
  <p>but I don't know how to do it if I would  extract specific paragraph (job offers) from a web page in general</p>