title: quicker way to detect n-grams in a string?
body: <p>I found this solution on SO to detect n-grams in a string:
  (here: <a href="https://stackoverflow.com/questions/3656762/n-gram-generation-from-a-sentence">N-gram generation from a sentence</a>)</p>
  
  <pre><code>import java.util.*;
  
  public class Test {
  
      public static List&lt;String&gt; ngrams(int n, String str) {
          List&lt;String&gt; ngrams = new ArrayList&lt;String&gt;();
          String[] words = str.split(" ");
          for (int i = 0; i &lt; words.length - n + 1; i++)
              ngrams.add(concat(words, i, i+n));
          return ngrams;
      }
  
      public static String concat(String[] words, int start, int end) {
          StringBuilder sb = new StringBuilder();
          for (int i = start; i &lt; end; i++)
              sb.append((i &gt; start ? " " : "") + words[i]);
          return sb.toString();
      }
  
      public static void main(String[] args) {
          for (int n = 1; n &lt;= 3; n++) {
              for (String ngram : ngrams(n, "This is my car."))
                  System.out.println(ngram);
              System.out.println();
          }
      }
  }
  </code></pre>
  
  <p>=> this bit of code takes by far the longest processing time (28 seconds for the detection of 1-grams, 2-grams, 3-grams and 4grams for my corpus: 4Mb of raw text), as compared to milliseconds for other operations (removal of stopwords etc.)</p>
  
  <p>Does anybody know of solutions in Java which would go faster than the looping solution presented above? (I was thinking multithreading, use of collections, or maybe creative ways to split a string...?) Thanks!</p>