title: n-gram modeling with java hashmap
body: <p>I need to model a collection of n-grams (sequences of n words) and their contexts (words that appear near the n-gram along with their frequency). My idea of was this:</p>
  
  <pre><code>public class Ngram {
  
      private String[] words;
      private HashMap&lt;String, Integer&gt; contextCount = new HashMap&lt;String, Integer&gt;();
  }
  </code></pre>
  
  <p>Then, for the count of all the different n-grams, I use another Hashmap, like</p>
  
  <pre><code>HashMap&lt;String, Ngram&gt; ngrams = new HashMap&lt;String, Ngram&gt;();
  </code></pre>
  
  <p>and I add to it while receiving text. The problem is, when the number of n-grams surpasses 10,000 or so, the JVM Heap fills up (it's set to a max of 1.5GB), and everything slows down really badly. </p>
  
  <p>Is there a better way to do this, so to avoid such memory consumption? Also, the contexts should be easily comparable between the n-grams, which I'm not sure is possible with my solution.</p>