title: Fastest way to process a file & DB insert - Java Multi-Threading
body: <p>I am completely new in Multi-threading coding.</p>
  
  <p>This is my requirement:
  I have a file with 50 000 - 300 000 records.</p>
  
  <p>It’s column based data (4 columns), with space as a separator. I need to split the line using space and save the records in DB in 4 columns.</p>
  
  <p>I want to develop a multi-threaded application, which inserts the data to H2 DB with 4 columns (using JDBC/anything else ?) with in 2 seconds approximately.
  I need to change the thread pool size dynamically as per the number of records I receive.</p>
  
  <p>I am developing a desktop application using Java Swings. (not a web-based app)</p>
  
  <p>I am not aware if there are any better Concurrent classes which does this task much faster.</p>
  
  <p>If it's not multi-threading, is there any other approach? or any other framework ?</p>
  
  <p>After adding batch processing, it's taking 5 seconds approx, for 250,000 records:</p>
  
  <pre><code>    BufferedReader in = new BufferedReader(new FileReader(file));
      java.util.List&lt;String[]&gt; allLines = new ArrayList&lt;String[]&gt;(); // used for something else
  
      String sql = "insert into test (a, b, c, d)” +
              " values (?,?,?,?)";
  
      PreparedStatement pstmt = conn.prepareStatement(sql);
      int i=0;
      while ((line = in.readLine()) != null) {
  
          line = line.trim().replaceAll(" +", " ");
          String[] sp = line.split(" ");
          String msg = line.substring(line.indexOf(sp[5]));
          allLines.add(new String[]{sp[0] + " " + sp[1], sp[4], sp[5], msg});
  
          pstmt.setString(1, sp[0] + " " + sp[1]);
          pstmt.setString(2, sp[4]);
          pstmt.setString(3, sp[5]);
          pstmt.setString(4, msg);
  
          pstmt.addBatch();
  
          i++;
  
          if (i % 1000 == 0){
              pstmt.executeBatch();
              conn.commit();
          }
      }
  
      pstmt.executeBatch();
  </code></pre>