title: Crawlers other than Nutch that work with Elasticsearch
body: <p>I'm trying to get some suggestions as I setup my data system. I'd like to setup a system for web crawling. It'll crawl probably a few hundred/thousand sites on a regular basis.</p>
  
  <p>I'm aware of Nutch and have used Nutch, however I'd like to know if others know of a better crawler than Nutch.</p>
  
  <p>I'm also using Elasticsearch as the indexer and its quite hard to get Nutch to work with newer versions of ES.</p>