title: Using AWS for parallel processing with R
body: <p>I want to take a shot at the <a href="http://www.kaggle.com/c/dunnhumbychallenge" rel="noreferrer">Kaggle Dunnhumby challenge</a> by building a model for each customer. I want to split the data into ten groups and use Amazon web-services (AWS) to build models using R on the ten groups in parallel. Some relevant links I have come across are:</p>
  
  <ul>
  <li>The <a href="http://code.google.com/p/segue/" rel="noreferrer">segue package</a>;</li>
  <li>A <a href="http://www.econsteve.com/r/barr-parallelPresoFeb2011.pdf" rel="noreferrer">presentation</a> on parallel web-services using Amazon.</li>
  </ul>
  
  <p>What I don't understand is:</p>
  
  <ul>
  <li>How do I get the data into the ten nodes? </li>
  <li>How do I send and execute the R functions on the nodes?</li>
  </ul>
  
  <p>I would be very grateful if you could share suggestions and hints to point me in the right direction.</p>
  
  <p>PS I am using the free usage account on AWS but it was very difficult to install R from source on the Amazon Linux AMIs (lots of errors due to missing headers, libraries and other dependencies).</p>