title: How to remove insignificant whitespace in lxml.html?
body: <p>I'm rather surprised that lxml.html leaves insignificant whitespace when parsing HTML by default. I'm also surprised that I can't find any obvious way to make it not do that.</p>
  
  <pre><code>Python 2.7.3 (default, Apr 10 2013, 06:20:15) 
  [GCC 4.6.3] on linux2
  Type "help", "copyright", "credits" or "license" for more information.
  &gt;&gt;&gt; import lxml.etree
  &gt;&gt;&gt; parser = lxml.etree.HTMLParser(remove_blank_text=True)
  &gt;&gt;&gt; html = lxml.etree.HTML("&lt;p&gt;      Hello     World     &lt;/p&gt;", parser=parser)
  &gt;&gt;&gt; print lxml.etree.tostring(html)
  &lt;html&gt;&lt;body&gt;&lt;p&gt;      Hello     World     &lt;/p&gt;&lt;/body&gt;&lt;/html&gt;
  </code></pre>
  
  <p>I expect the result would be something like:</p>
  
  <pre><code>&gt;&gt;&gt; print lxml.etree.tostring(html)
  &lt;html&gt;&lt;body&gt;&lt;p&gt;Hello World&lt;/p&gt;&lt;/body&gt;&lt;/html&gt;
  </code></pre>
  
  <p>BeautifulSoup4 does the same thing with the html5lib parser:</p>
  
  <pre><code>&gt;&gt;&gt; from bs4 import BeautifulSoup
  &gt;&gt;&gt; soup = BeautifulSoup("&lt;p&gt;      Hello     World     &lt;/p&gt;", "html5lib")
  &gt;&gt;&gt; soup.p
  &lt;p&gt;      Hello     World     &lt;/p&gt;
  </code></pre>
  
  <p>After doing some research, I found that the HTML5 parsing specification does not specify to remove consecutive whitespace; that is done at render time instead. So I understand that's it technically not the responsibility of any of these libraries to perform the same behavior, but it seems useful enough that I'm surprised none of them have it anyway.</p>
  
  <p>Can somebody prove me wrong?</p>
  
  <p>Edit: </p>
  
  <p>I know how to remove whitespace using a regex — that was not my question. (I also know how to search SO for questions about regex.) </p>
  
  <p>My question has to do with the <em>insignificant</em> whitespace, where <em>significance</em> is defined by the standards for rendering HTML. I doubt that a 1-liner regex can correctly implement this standard. And let's not even delve into the regex vs CFG debate again, please? </p>
  
  <p><a href="https://stackoverflow.com/questions/1732348/regex-match-open-tags-except-xhtml-self-contained-tags">RegEx match open tags except XHTML self-contained tags</a></p>
  
  <p>Edit 2:</p>
  
  <p>In case it's not clear from the context, I am interested in HTML, not XHTML/XML. Whitespace does have some non-trivial rules of significance in HTML, however those rules are implemented in the renderer, not the parser. I understand that, as evidenced in my initial post. My question is whether anybody has implemented the white space logic of an HTML renderer in a library that operates at the DOM level rather than at the rendering level?</p>