title: Efficient implementation of mutual information in Java
body: <p>I'm looking to calculate mutual information between two features, using Java.</p>
  
  <p>I've read <a href="https://stackoverflow.com/questions/2002521/calculating-mutual-information-for-selecting-a-training-set-in-java">Calculating Mutual Information For Selecting a Training Set in Java</a> already, but that was a discussion of if mutual information was appropriate for the poster, with only some light pseudo-code as to the implementation.</p>
  
  <p>My current code is below, but I'm hoping there is a way to optimise it, as I have large quantities of information to process. I'm aware that calling out to another language/framework may improve speed, but would like to focus on solving this in Java for now.</p>
  
  <p>Any help much appreciated.</p>
  
  <pre><code>public static double calculateNewMutualInformation(double frequencyOfBoth, double frequencyOfLeft,
                                                     double frequencyOfRight, int noOfTransactions) {
      if (frequencyOfBoth == 0 || frequencyOfLeft == 0 || frequencyOfRight == 0)
          return 0;
      // supp = f11
      double supp = frequencyOfBoth / noOfTransactions; // P(x,y)
      double suppLeft = frequencyOfLeft / noOfTransactions; // P(x)
      double suppRight = frequencyOfRight / noOfTransactions; // P(y)
      double f10 = (suppLeft - supp); // P(x) - P(x,y)
      double f00 = (1 - suppRight) - f10; // (1-P(y)) - P(x,y)
      double f01 = (suppRight - supp); // P(y) - P(x,y)
  
      // -1 * ((P(x) * log(Px)) + ((1 - P(x)) * log(1-p(x)))
      double HX = -1 * ((suppLeft * MathUtils.logWithoutNaN(suppLeft)) + ((1 - suppLeft) * MathUtils.logWithoutNaN(1 - suppLeft)));
      // -1 * ((P(y) * log(Py)) + ((1 - P(y)) * log(1-p(y)))
      double HY = -1 * ((suppRight * MathUtils.logWithoutNaN(suppRight)) + ((1 - suppRight) * MathUtils.logWithoutNaN(1 - suppRight)));
  
      double one = (supp * MathUtils.logWithoutNaN(supp)); // P(x,y) * log(P(x,y))
      double two = (f10 * MathUtils.logWithoutNaN(f10)); 
      double three = (f01 * MathUtils.logWithoutNaN(f01));
      double four = (f00 * MathUtils.logWithoutNaN(f00));
      double HXY = -1 * (one + two + three + four);
      return (HX + HY - HXY) / (HX == 0 ? MathUtils.EPSILON : HX);
  }        
  
  public class MathUtils {
  public static final double EPSILON = 0.000001;
  
  public static double logWithoutNaN(double value) {
      if (value == 0) {
          return Math.log(EPSILON);
      } else if (value &lt; 0) {
          return 0;
      }
      return Math.log(value);
  }  
  </code></pre>