title: Extracting data from HTML-files with BeautifulSoup and Python
body: <p>I need to extract data from HTML-files. The files in question are, most likely, automatically generated. I have uploaded the code of one of these files to Pastebin: <a href="http://pastebin.com/9Nj2Edfv">http://pastebin.com/9Nj2Edfv</a>. This is the link to the actual page: <a href="http://eur-lex.europa.eu/Notice.do?checktexts=checkbox&amp;val=60504%3Acs&amp;pos=1&amp;page=1&amp;lang=en&amp;pgs=10&amp;nbl=1&amp;list=60504%3Acs%2C&amp;hwords=&amp;action=GO&amp;visu=%23texte">http://eur-lex.europa.eu/Notice.do?checktexts=checkbox&amp;val=60504%3Acs&amp;pos=1&amp;page=1&amp;lang=en&amp;pgs=10&amp;nbl=1&amp;list=60504%3Acs%2C&amp;hwords=&amp;action=GO&amp;visu=%23texte</a></p>
  
  <p>The data I need to extract is found under the different headings. </p>
  
  <p>This is what I have so far:</p>
  
  <pre><code>from BeautifulSoup import BeautifulSoup
  ecj_data = open("data\ecj_1.html",'r').read()
  
  soup = BeautifulSoup(ecj_data)
  
  celex = soup.find('h1')
  auth_lang = soup('ul', limit=14)[13].li
  procedure = soup('ul', limit=20)[17].li
  
  print "Celex number:", celex.renderContents(),
  print "Authentic language:", auth_lang
  print "Type of procedure:", procedure
  </code></pre>
  
  <p>I have all the data stored locally which is the reason it opens the file ecj_1.html. </p>
  
  <p>The Celex number and the Authentic language works somewhat good. </p>
  
  <p>celex returns </p>
  
  <pre><code>"Celex number: 
  61977J0059"
  </code></pre>
  
  <p>auth_lang returns <code>"Authentic language: &lt;li&gt;French&lt;/li&gt;"</code></p>
  
  <p>I need just the contents of the h1 tag (not the break at the end). </p>
  
  <p>[Also, I need auth_lang to return just "French", and not the <code>&lt;li&gt;</code>-tags.]
  <strong>This is not a problem anymore. I realized I could just add ".text" to the end of "auth_lang".</strong></p>
  
  <p>Procedure on the other hand returns this:</p>
  
  <pre><code>    Type of procedure: &lt;li&gt;
      &lt;strong&gt;Type of procedure:&lt;/strong&gt;
      &lt;br /&gt;
      Reference for a preliminary ruling
      &lt;/li&gt;
  </code></pre>
  
  <p>which is quite wrong as I just need it to return "Reference for a preliminary ruling".</p>
  
  <p>Is there any way I can achieve this?</p>
  
  <p><strong>Second edit:
  I replaced <code>celex = soup.find('h1')</code> with <code>celex = soup('h1', limit=2)[0]</code> and added <code>.text</code> to the print celex.</strong> </p>