title: Using python to edit html, but lxml converts nice html entities to strange encoding
body: <p>I'm trying to use python (with pyquery and lxml) to alter and clean up some html.</p>
  
  <pre><code>Eg. html = "&lt;div&gt;&lt;!-- word style&gt;&lt;bleep&gt;&lt;omgz 1,000 tags&gt;&lt;--&gt;&lt;p&gt;It&amp;#146;s a spicy meatball!&lt;/div&gt;"
  </code></pre>
  
  <p>The lxml.html.clean function, clean_html(), works well, except that it replaces the nice html entities like</p>
  
  <pre><code>&amp;#146; 
  </code></pre>
  
  <p>with some unicode string</p>
  
  <pre><code>\xc2\x92
  </code></pre>
  
  <p>The unicode looks strange in different browsers (firefox and opera using auto encoding, utf8, latin-1, etc), like an empty box.  How can I stop lxml converting the entities?  How can I get it all in latin-1 encoding?  Seems strange that a module built specifically for html would do this.</p>
  
  <p>I can't be sure of which characters are there, so I can't just use </p>
  
  <pre><code>replace("\xc2\x92","&amp;#146;").
  </code></pre>
  
  <p>I've tried using</p>
  
  <pre><code>clean_html(html).encode('latin-1')
  </code></pre>
  
  <p>but the unicode persists.</p>
  
  <p>And yes, I'd tell people to stop using word to write html, but then I'd hear the whole </p>
  
  <blockquote>
    <p>"iz th wayz i liks it u cant mak me chang hitlr".</p>
  </blockquote>
  
  <p>Edit: a beautifulsoup solution:</p>
  
  <pre><code>from BeautifulSoup import BeautifulSoup, Comment
  soup = BeautifulSoup(str(desc[desc_type]))
                      comments = soup.findAll(text=lambda text:isinstance(text, Comment))
                      [comment.extract() for comment in comments]
                      print soup
  </code></pre>