title: Python HTMLParser dividing data at &
body: <p>I'm using a simple HTMLParser to parse a webpage with code that is always well-formed (it's automatically generated). It works well, until it hits a piece of data with an '&amp;' sign in it - it seems to think that that makes it two separate pieces of data and processes them separately. (That is, it calls "handle_data" twice.) I at first thought that unescaping the '&amp;'would solve the issue, but I don't think it does. Does anyone have any suggestion for how I can get my parser to treat, for instance "Paradise Bakery and Cafe" (that is, "Paradise Bakery &amp; Café") as a single data item rather than as two?</p>
  
  <p>Thanks a lot, 
    bsg</p>
  
  <p>P.S. Please don't tell me that I really should be using BeautifulSoup. I know. But in this case, I knew the markup was guaranteed to be well-formed every time, and I found HTMLParser easier to work with than BeautifulSoup. Thanks.</p>
  
  <p>I'm adding my code - thanks!</p>
  
  <pre><code>#this class, extending HTMLParser, is written to process HTML within a &lt;ul&gt;. 
  #There are 6 &lt;a&gt; elements nested within each &lt;li&gt;, and I need the data from the second 
  #one. Whenever it encounters an &lt;li&gt; tag, it sets the 'is_li' flag to true and resets 
  #the count of a's seen to 0; whenever it encounters an &lt;a&gt; tag, it increments the count
  #by 1.   When handle_data is called, it checks to make sure that the data is within
  #1)an li element and 2) an a element, and that the a element is the second one in that
  #li (num_as == 2). If so, it adds the data to the list. 
  
  class MyHTMLParser(HTMLParser):
  pages = []
  is_li = 'false'
  #is_li 
  num_as = 0
  
  def _init_(self):
      HTMLParser._init_(self)
      self.pages = []
      self.is_li = 'false'
      self.num_as = 0
      self.close_a = 'false'
      sel.close_li = 'false'
      print "initialized"
  
  
  def handle_starttag(self, tag, attrs):
        if tag == 'li':
            self.is_li = 'true'
            self.close_a = 'false'
            self.close_li = 'false'
  
  
        if tag == 'a' and self.is_li == 'true':
            if self.num_as &lt; 7:
                self.num_as += 1
                self.close_a = 'false'
  
            else:
                self.num_as = 0
                self.is_li = 'false'
  
  def handle_endtag(self, tag):
       if tag == 'a':
           self.close_a = 'true'
  
       if tag == 'li':
           self.close_li = 'true'
           self.num_as = 0
  
  def handle_data(self, data):
      if self.is_li == 'true':
          if self.num_as == 2 and self.close_li == 'false' and self.close_a == 'false':
              print "found data",  data
              self.pages.append(data)
  
  def get_pages(self):
      return self.pages
  </code></pre>