title: Incorrectly formatted html inconsistencies between DOM and what's displayed in firefox plugin
body: <p>I'm currently developing a firefox plugin. This plugin has to handle very crappy website that is really incorrectly formatted. I cannot modify these websites, so I have to handle them.</p>
  
  <p>I reduced the bug I'm facing to a short sample of html (if this appellation is appropriate for an horror like this) :</p>
  
  <pre><code>&lt;!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"&gt;
  &lt;html&gt;
  &lt;head&gt;
  &lt;title&gt;Some title.&lt;/title&gt;
      &lt;!-- Oh yes ! --&gt;
      &lt;div style="visability:hidden;"&gt;
          &lt;a href="//example.com"&gt; &lt;/a&gt;
      &lt;/div&gt;
  
  &lt;!-- If meta are reduced, then the bug disapears ! --&gt;
  &lt;meta name="description" content="Homepage of Company.com, Company's corporate Web site" /&gt;
  &lt;meta name="keywords" content="Company, Company &amp; Co., Inc., blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla" /&gt;
  &lt;meta http-equiv="Content-Language" content="en-US" /&gt;
  &lt;meta http-equiv="content-type" content="text/html; charset=utf-8"/&gt;
  
  &lt;/head&gt;
  &lt;body class="homePage"&gt;
  
  &lt;div class="globalWrapper"&gt;&lt;a href="/page.html"&gt;My gorgeous link !&lt;/a&gt;&lt;/div&gt;
  
  &lt;/body&gt;
  
  &lt;/html&gt;
  </code></pre>
  
  <p>When opening the webpage, « My gorgeous link ! » if displayed and clickable. However, when I'm exploring the DOM with Javascript into my plugin, everything behaves (DOM exploration and innerHTML property) like the code was this one :</p>
  
  <pre><code>&lt;html&gt;
  &lt;head&gt;
  &lt;title&gt;Some title.&lt;/title&gt;
      &lt;!-- Oh yes ! --&gt;
      &lt;/head&gt;&lt;body&gt;&lt;div style="visability:hidden;"&gt;
          &lt;a href="//example.com"&gt; &lt;/a&gt;
      &lt;/div&gt;
  
  &lt;!-- If meta are reduced, then the bug disapears ! --&gt;
  &lt;meta name="description" content="Homepage of Company.com, Company's corporate Web site"&gt;
  &lt;meta name="keywords" content="Company, Company &amp;amp; Co., Inc., blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla, blablabla"&gt;
  &lt;meta http-equiv="Content-Language" content="en-US"&gt;
  &lt;/body&gt;
  &lt;/html&gt;
  </code></pre>
  
  <p>So, when exploring the DOM within the plugin, the document is somehow fixed by firefox. But this fixed DOM is inconsistent with what is in the webpage. Thus, my plugin doesn't behave as expected.</p>
  
  <p>I'm really puzzled with that issue. The problem exists in both firefox 3.6 and firefox 4 (didn't tested firefox 5 yet). For example, reducing the meta, will fix the issue.</p>
  
  <p>Where does this discrepancy come from ? How can I handle it ?</p>
  
  <p>EDIT: With the answer I get, I think I should be a little more precise. I do know what firefow is doing when modifying the webpage in the second code snippet. The problem is the following one : « In the fixed DOM that I get into my plugin, the gorgeous link doesn't appear anywhere, but this link is actually visible on the webpage, and works. So the DOM I'm manipulating, and the DOM in the webpage are different - they are fixed in a different manner. » . So where does the difference come in the fixing behaviour, and how can I handle that, or, in other terms, how can I be aware, in my plugin, of the existance of the gorgeous link ?</p>
  
  <p>NB: Exploring the DOM with firebug show a different DOM from what I'm getting in my module. Both DOM are fixed by firefox, but in a different manner. I do get the DOM like this :</p>
  
  <pre><code> var html = browser.contentDocument.documentElement;
   // Then, for example :
   html.getElementsByTagName('a'); // Returns only the a element in the header. On the webpage, only the a in the body appears.
  </code></pre>
  
  <p>DOM exploration with firebug show that the div and a within the header are removed, which is a different behaviour.</p>
  
  <p>EDIT² : The code in my plugin is ran after the page loading end, by this mecanism :</p>
  
  <pre><code>gBrowser.addTabsProgressListener({
      onStateChange: function(aBrowser, aWebProgress, aRequest, aStateFlags, aStatus) {
          if( (aStateFlags &amp; Components.interfaces.nsIWebProgressListener.STATE_STOP) ) {
              // Some operations including the DOM parsing here
          }
      }
  });
  </code></pre>