title: Efficient batch processing with Stanford CoreNLP
body: <p><strong>Is it possible to speed up batch processing of documents with CoreNLP from command line so that models load only one time? I would like to trim any unnecessarily repeated steps from the process.</strong></p>
  
  <p>I have 320,000 text files and I am trying to process them with CoreNLP. The desired result is 320,000 finished XML file results. </p>
  
  <p>To get from one text file to one XML file, I use the CoreNLP jar file from command line:</p>
  
  <pre><code>java -cp "*" -Xmx2g edu.stanford.nlp.pipeline.StanfordCoreNLP -props config.properties 
  -file %%~f -outputDirectory MyOutput -outputExtension .xml -replaceExtension`
  </code></pre>
  
  <p>This loads models and does a variety of machine learning magic. The problem I face is when I try to loop for every text in a directory, I create a process that by my estimation will complete in 44 days. I literally have had a command prompt looping on my desktop for the last 7 days and I'm nowhere near finished. The loop I run from batch script:</p>
  
  <pre><code>for %%f in (Data\*.txt) do (
      java -cp "*" -Xmx2g edu.stanford.nlp.pipeline.StanfordCoreNLP -props config.properties
      -file %%~f -outputDirectory Output -outputExtension .xml -replaceExtension
  )
  </code></pre>
  
  <p>I am using these annotators, specified in config.properties:
  <br>
  <code>annotators = tokenize, ssplit, pos, lemma, ner, parse, dcoref, sentiment</code></p>