title: Crawling links on a page, then visiting and inspecting each link with node and zombie.js
body: <p>I'm trying to build a simple utility in Node with zombie.js to visit a page, find and open all links on the page, and ensure that each child page returns a 200 successfully.</p>
  
  <p>Here's an example of this code (written in CoffeeScript), crawling the home page of stackoverflow.com</p>
  
  <pre><code>Browser = require('zombie')
  
  browserOpts =
    runScripts: false
    site: 'http://www.stackoverflow.com'
  
  home = new Browser browserOpts
  
  home.visit '/', (e, browser) -&gt;
    questions = browser.queryAll '#question-mini-list .summary h3 a'
    for q in questions
      qUrl = q.getAttribute 'href'
      page = new Browser browserOpts
      page.visit qUrl, (e, browser, statusCode, errors) -&gt;
        console.log "Arrived at page #{browser.window.location} and found " + browser.html().length + " bytes"
        console.log statusCode
        browser.dump()
        return
    return
  </code></pre>
  
  <p>If you attempt to run this code, you'll notice the first handful of links are loaded correctly, and the number of bytes in the page is displayed.</p>
  
  <p>However, after the first batch of successful page loads (the size of which seems random), all subsequent page loads seem to execute the callback to <code>visit</code> prematurely. The document is empty (it's just <code>&lt;html&gt;&lt;head&gt;&lt;/head&gt;&lt;body&gt;&lt;/body&gt;&lt;/html&gt;</code>) and the <code>statusCode</code> argument to the callback is <code>undefined</code>.</p>
  
  <p>I can't explain or figure out why this is happening. Any tips would be greatly appreciated.</p>