title: GemfireXD - How to parallelize data processing for bigger data size
body: <p>I am using JDBC to process some data where I am using gemfireXD as DB which is an InMemory data grid and a NewSQL solution.</p>
  
  <p>Step1 - I am getting all distinct keys(field1) from a table (Table1)</p>
  
  <pre><code>             Statement stmt = conn.createStatement();
               stmt.execute("select distinct field1 from Table1");
               ResultSet rs = stmt.getResultSet();
               List&lt;String&gt; keyList = new ArrayList&lt;String&gt; ();
               while (rs.next()) {
                     keyList.add(rs.getString("field1"));
               }
  </code></pre>
  
  <p>Step2 - I am querying the same table for each key from the above list. This should return me 2 records for each key.</p>
  
  <p>I need to process these 2 records for each key and create one final processed record out of these 2 rows for the key, which finally need to be stored in a separate table (table2).</p>
  
  <pre><code>    for (int i = 0; i&lt; fieldList.size(); i++) {
          stmt.execute("select * from Table1 where field1=" + keyList.get(i));
          ResultSet rs = stmt.getResultSet();
  
          // the resultset should contain 2(or more) rows for the key which need to be processed based on some logic.
          // Finally need to create 1 processed record for the given key.
          // Insert the processed record to Table2
      }
  </code></pre>
  
  <p>Since I have millions of records in Table1, The above way of processing data is time consuming.</p>
  
  <p>I need to parallelize the processing in gemfireXD.</p>
  
  <p>I can do this same processing in PIG or SPARK within few hours using their parallel processing features(creating Tuples/bags of records and using MR programming). </p>
  
  <p>I am hoping gemfireXD must have some facilities to do parallel processing of data, though I have tried DATAAWARE procedure in gemfireXD and used it to parallelize the procedure call, but it did not work for me. </p>
  
  <p>I am hoping gemfire XD must have some other approach for this purpose</p>
  
  <p>Could any one please suggest any implementation approach in gemfireXD to achieve the desired result?</p>