 <p>I've found a couple of Java crawlers that are supposed to be pretty close to Mercator:</p> <ul> <li><a href="http://nutch.apache.org/" rel="nofollow">Nutch</a> is multithreaded and distributed.</li> <li><a href="http://crawler.archive.org/index.html" rel="nofollow">Heritrix</a> is only multithreaded.</li> </ul> <p>Other references are welcome.</p>