title: Sharing crawled nutch data between multiple solr indexes
body: <p>We have thousands of solr indexes/collections that share pages being crawled by nutch. </p>
  
  <p>Currently these pages are being crawled multiple times, once for each solr index that contains them.</p>
  
  <p>It is possible to crawl these sites once, and share the crawl data between indexes?</p>
  
  <p>Maybe by checking existing crawldbs if a site has been crawled and get the data from there for parsing and indexing.</p>
  
  <p>Or crawl all sites in one go, and then selectively submit crawl data to each index. (eg: one site per segment, but not sure how to identify which segment  belongs to what site due to segment names are numeric)</p>
  
  <p>Any ideas or help appreciated :)</p>