title: Distribution of words per topic p(w|t) in Mallet
body: <p>I need to get the distribution of words for each topic found by <a href="http://mallet.cs.umass.edu/" rel="nofollow noreferrer">Mallet</a> in Java (not in the CLI as asked in <a href="https://stackoverflow.com/questions/33251703/how-to-get-a-probability-distribution-for-a-topic-in-mallet">how to get a probability distribution for a topic in mallet?</a>). For an example of what I mean: <a href="http://blog.echen.me/2011/08/22/introduction-to-latent-dirichlet-allocation/" rel="nofollow noreferrer">Introduction to Latent Dirichlet Allocation</a>:</p>
  
  <pre><code>Topic A: 30% broccoli, 15% bananas, 10% breakfast, 10% munching, … (at which point, you could interpret topic A to be about food)
  Topic B: 20% chinchillas, 20% kittens, 20% cute, 15% hamster, … (at which point, you could interpret topic B to be about cute animals)
  </code></pre>
  
  <p>Mallet provides token "weights" per topic, and in <a href="http://comments.gmane.org/gmane.comp.ai.mallet.devel/2064" rel="nofollow noreferrer">http://comments.gmane.org/gmane.comp.ai.mallet.devel/2064</a> somebody attempted to write a method for getting the distribution of words per topic for Mallet. </p>
  
  <p>I modified the method, so that all weights are divided by their sum as discussed in the mailing list above. </p>
  
  <p>Does the following method (when added to ParallelTopicModel.java) correctly calculate the distribution of words per topic p(w|t) in Mallet?</p>
  
  <pre><code>/**
   * Get the normalized topic word weights (weights sum up to 1.0)
   * @param topic the topic
   * @return the normalized topic word weights (weights sum up to 1.0)
   */
  public ArrayList&lt;double[]&gt; getNormalizedTopicWordWeights(int topic) {
      ArrayList&lt;double[]&gt; tokenWeights = new ArrayList&lt;double[]&gt;();
      for (int type = 0; type &lt; numTypes; type++) {
          int[] topicCounts = typeTopicCounts[type];
          double weight = beta;
          int index = 0;
          while (index &lt; topicCounts.length &amp;&amp; topicCounts[index] &gt; 0) {
              int currentTopic = topicCounts[index] &amp; topicMask;
              if (currentTopic == topic) {
                  weight += topicCounts[index] &gt;&gt; topicBits;
                  break;
              }
              index++;
          }
          double[] tokenAndWeight = { (double) type, weight };
          tokenWeights.add(tokenAndWeight);
      }
      // normalize
      double sum = 0;
      // get the sum
      for (double[] tokenAndWeight : tokenWeights) {
          sum += tokenAndWeight[1];
      }
      // divide each element by the sum
      ArrayList&lt;double[]&gt; normalizedTokenWeights = new ArrayList&lt;double[]&gt;();
      for (double[] tokenAndWeight : tokenWeights) {
          tokenAndWeight[1] = tokenAndWeight[1]/sum;
          normalizedTokenWeights.add(tokenAndWeight);
      }
      return normalizedTokenWeights;
  }
  </code></pre>