title: Use python to a open web browser (on windows), trigger javascript actions, and get the html contents?
body: <p>Yes that sounds overly complicated.</p>
  
  <p>I am trying to mine data from pages on our intranet. The pages are secure. The connection is refused when I try to get the contents with urllib.urlopen().</p>
  
  <p>So I would like to use python to open a web browser to open the site then click some links that trigger javascript pop ups containing tables of info that I want to collect.</p>
  
  <p>Any suggestions on where to begin?</p>
  
  <p>I know the format of the page. It is something like this:</p>
  
  <pre><code>&lt;div id="list"&gt;
      &lt;ul id="list item"&gt;
          &lt;li&gt;&lt;a onclict="Openpopup('1');"&gt;blah&lt;/a&gt;&lt;/li&gt;
      &lt;/ul&gt;
      &lt;ul&gt;&lt;/ul&gt;
      etc
  </code></pre>
  
  <p>Then a hidden frame becomes visible and the fields in the table within are filled.</p>
  
  <pre><code>&lt;div&gt;
      &lt;table&gt;
         &lt;tr&gt;&lt;td&gt;&lt;span id="info_i_want"&gt;...
  </code></pre>