title: Stack to handle large computation and data explosion
body: <p>I recently started working with a team that has been building a solution that involves parallel calculations &amp; data explosion</p>
  
  <p>The input to the system is provided in a set of excel files. Says there are 5 sets of data A, B, C, D and E the calculated output is a multiple of A, B, C, D and E. This output also grows over years - i.e. if the data is spread across 5 yrs - the output for yr1 is the smallest and output of yr5 is the largest (~3 billion rows)</p>
  
  <p>We currently use Microsoft SQL Server to store the input, Microsoft Orleans for computation and store the calculated output in Hadoop. Some concerns I have here are - what we are doing seems to be opposite of map reduce and we have limited big data skills on the team.</p>
  
  <p>I wanted to see if someone has experience working on similar systems and what kind of solution stack was used</p>
  
  <p>Thanks</p>