title: Need suggestion on using Map/Reduce to create solr index
body: <p>I'm pretty new to Map/Reduce world and trying to evaluate the best option to figure if I can leverage it to create index in Solr. Currently, I'm using a regular crawl to fetch data and index it in Solr directly. This is working without any issues. </p>
  
  <p>But going forward, we need to access a sizable data residing in Amazon S3. There are around 5 million data presently stored in S3 , which needs to be indexed. I'm thinking of using Amazon Elastic Map/Reduce (EMR) to directly access the content from S3 and subsequently create the index in Solr. The data structure is simple, the url (which is unique) is the S3 key, the value is a XML file. The url will be used as the doc id in Solr while relevant portion of the XML data will be stored as fields in Solr index.</p>
  
  <p>My question is whether EMR is the right approach? The task is to access the data from S3, extract certain elements from XML, do some processing and then call Solr API to generate the index. The processing part requires few classes, possibly a chain of command pattern, before indexing the data. Is it something achievable? Doo I require a reducer or can use a mapper to do the process? If reducer is need, what will the scope of it? Currently, I've a single index which is storing the data.</p>
  
  <p>Any pointers on this will be highly appreciated.</p>
  
  <p>Thanks</p>