title: Parsing an html table
body: <p>To start off here's my current code in its entirety:</p>
  
  <pre><code>import urllib
  from BeautifulSoup import BeautifulSoup
  import sgmllib
  import re
  
  page = 'http://www.sec.gov/Archives/edgar/data/\
  8177/000114036111018563/form10k.htm'
  
  sock = urllib.urlopen(page)
  raw = sock.read()
  soup = BeautifulSoup(raw)
  
  tablelist = soup.findAll('table')
  
  class MyParser(sgmllib.SGMLParser):
  
  def parse(self, segment):
      self.feed(segment)
      self.close()
  
  def __init__(self, verbose=0):
      sgmllib.SGMLParser.__init__(self, verbose)
      self.descriptions = []
      self.inside_td_element = 0
      self.starting_description = 0
  
  def start_td(self, attributes):
      for name, value in attributes:
          if name == "valign":
              self.inside_td_element = 1
              self.starting_description = 1
          else:
              self.inside_td_element = 1
              self.starting_description = 1
  
  def end_td(self):
      self.inside_td_element = 0
  
  def handle_data(self, data):
      if self.inside_td_element:
          if self.starting_description:
              self.descriptions.append(data)
              self.starting_description = 0
          else:
              self.descriptions[-1] += data
  
  def get_descriptions(self):
      return self.descriptions
  
  counter = 0
  trlist = []
  dtablelist = []
  
  while counter &lt; len(tablelist):
      trsegment = tablelist[counter].findAll('tr')
      trlist.append(trsegment)
      strsegment = str(trsegment)
      myparser = MyParser()
      myparser.parse(strsegment)
      sub = myparser.get_descriptions()
      dtablelist.append(sub)
      counter = counter + 1
  
  ex = []
  
  dtablelist = [s for s in dtablelist if s != ex]
  </code></pre>
  
  <p>So what I want to accomplish is take all the tables from an html document, then reprint them onto an Excel spreadsheet. So when I create trlist the output looks like this:</p>
  
  <pre><code>print trlist[1]
  [&lt;tr&gt;
  &lt;td valign="top" width="25%"&gt;&lt;font style="DISPLAY: inline; FONT-WEIGHT: bold; FONT-    SIZE: 10pt; FONT-FAMILY: times new roman; TEXT-DECORATION: underline"&gt;&amp;#160;&lt;/font&gt;&lt;/td&gt;
  &lt;td valign="top" width="25%"&gt;
  &lt;div style="DISPLAY: block; MARGIN-LEFT: 0pt; TEXT-INDENT: 0pt; MARGIN-RIGHT: 0pt" align="center"&gt;&lt;font style="DISPLAY: inline; FONT-WEIGHT: bold; FONT-SIZE: 10pt; FONT-FAMILY: times new roman; TEXT-DECORATION: underline"&gt;Title of each class&lt;/font&gt;&lt;/div&gt;
  &lt;/td&gt;
  &lt;td valign="top" width="25%" style="TEXT-ALIGN: center"&gt;&lt;font style="DISPLAY: inline; FONT-WEIGHT: bold; FONT-SIZE: 10pt; FONT-FAMILY: times new roman; TEXT-DECORATION: underline"&gt;Name of exchange&lt;/font&gt;&lt;/td&gt;
  &lt;td valign="top" width="25%" style="TEXT-ALIGN: center"&gt;&lt;font style="DISPLAY: inline; FONT-WEIGHT: bold; FONT-SIZE: 10pt; FONT-FAMILY: times new roman; TEXT-DECORATION: underline"&gt;&amp;#160;&lt;/font&gt;&lt;/td&gt;
  &lt;/tr&gt;, &lt;tr&gt;
  &lt;td valign="top" width="25%"&gt;&lt;font style="DISPLAY: inline; FONT-SIZE: 10pt; FONT-FAMILY: times new roman"&gt;&amp;#160;&lt;/font&gt;&lt;/td&gt;
  &lt;td valign="top" width="25%"&gt;
  &lt;div style="DISPLAY: block; MARGIN-LEFT: 0pt; TEXT-INDENT: 0pt; MARGIN-RIGHT: 0pt" align="center"&gt;&lt;font style="DISPLAY: inline; FONT-SIZE: 10pt; FONT-FAMILY: times new roman"&gt;&lt;font style="DISPLAY: inline; FONT-WEIGHT: bold"&gt;Common Stock, par value&lt;/font&gt;    &lt;/font&gt;&lt;/div&gt;
  &lt;/td&gt;
  &lt;td valign="top" width="25%"&gt;
  &lt;div style="DISPLAY: block; MARGIN-LEFT: 0pt; TEXT-INDENT: 0pt; MARGIN-RIGHT: 0pt" align="center"&gt;
  &lt;div style="DISPLAY: block; MARGIN-LEFT: 0pt; TEXT-INDENT: 0pt; MARGIN-RIGHT: 0pt" align="center"&gt;&lt;font style="DISPLAY: inline; FONT-WEIGHT: bold; FONT-SIZE: 10pt; FONT-FAMILY: times new roman"&gt;&lt;font style="FONT-WEIGHT: bold"&gt;&lt;font style="FONT-WEIGHT: bold"&gt;&lt;     &lt;font style="FONT-WEIGHT: bold"&gt;NASDAQ Global Market&lt;/font&gt;&lt;/font&gt;&lt;/font&gt;&lt;/font&gt;&lt;/div&gt;
  &lt;/div&gt;
  &lt;/td&gt;
  &lt;td valign="top" width="25%"&gt;&lt;font style="DISPLAY: inline; FONT-WEIGHT: bold; FONT-SIZE: 10pt; FONT-FAMILY: times new roman"&gt;&amp;#160;&lt;/font&gt;&lt;/td&gt;
  &lt;/tr&gt;,...
  </code></pre>
  
  <p>As you can see each item in trlist is each individual row ( . . . ) of the table which is what I want. But when I run each trlist item through my sgmllib parser to retrieve the contents between the tags I get this output:</p>
  
  <pre><code>print dtablelist[1]
  ['\nTitle of each class\n', 'Name of exchange', '\nCommon Stock, par value\n', '\n\nNASDAQ Global Market\n\n', '\n$1.00 per share\n']
  </code></pre>
  
  <p>As you can see, the output is each of the contents as their own individual string, instead of a list of the contents of each table row (). So essentially I want the output:</p>
  
  <pre><code>[['\nTitle of each class\n', 'Name of exchange'], ['\nCommon Stock, par value\n', '\n\nNASDAQ Global Market\n\n'], ['\n$1.00 per share\n']]
  </code></pre>
  
  <p>Is it because I have to turn trlist into a string before I parse it with MyParser? Does anyone know any way around this, allowing me to parse lists within lists (aka Inception shit)?</p>