title: Spring,Hibernate - Batch processing of large amounts of data with good performance
body: <p>Imagine you have large amount of data in database approx. ~100Mb. We need to process all data somehow (update or export to somewhere else). How to implement this task with good performance ? How to setup transaction propagation ?</p>
  
  <p>Example 1# (with bad performance) :</p>
  
  <pre><code>@Singleton
  public ServiceBean {
  
   procesAllData(){
  
     List&lt;Entity&gt; entityList = dao.findAll();
  
     for(...){
       process(entity);
     }
  
   }
  
   private void process(Entity ent){
    //data processing    
    //saves data back (UPDATE operation) or exports to somewhere else (just READs from DB)
   }
  </code></pre>
  
  <p>}</p>
  
  <p>What could be improved here ? </p>
  
  <p>In my opinion :</p>
  
  <ol>
  <li>I would set hibernate batch size (see hibernate documentation for batch processing).</li>
  <li>I would separated ServiceBean into two Spring beans with different transactions settings. Method processAllData() should run out of transaction, because it operates with large amounts of data and potentional rollback wouldnt be 'quick' (i guess). Method process(Entity entity) would run in transaction - no big thing to make rollback in the case of one data entity.</li>
  </ol>
  
  <p>Do you agree ? Any tips ?</p>