title: Implementation of Logistic regression with Gradient Descent in Java
body: <p>I have implemented Logistic Regression with Gradient Descent in Java. It doesn't seem to work well (It does not classify records properly; the probability of y=1 is a lot.)   I don't know whether my implementation is correct.I have gone through the code several times and i am unable to find any bug. I have been following Andrew Ng's tutorials on Machine learning on Course Era. My Java implementation has 3 classes. namely :</p>
  
  <ol>
  <li>DataSet.java : To read the data set</li>
  <li>Instance.java : Has two members : 1. double[ ] x  and 2. double label  </li>
  <li>Logistic.java : This is the main class that implements Logistic Regression with Gradient Descent.            </li>
  </ol>
  
  <p>This is my cost function:
  <h2>J(Θ) = (- 1/m ) [Σ<sup>m</sup><sub>i=1</sub> y<sup>(i)</sup> log( h<sub>Θ</sub>( x<sup>(i)</sup> ) ) + (1 - y<sup>(i) </sup>) log(1 - h<sub>Θ</sub> (x<sup>(i)</sup>) )]
  </h2>For the above Cost function, this is my Gradient Descent algorithm:
  <h1>Repeat (</h1><h1> Θ<sub>j</sub> := Θ<sub>j</sub> - α Σ<sup>m</sup><sub>i=1</sub> ( h<sub>Θ</sub>( x<sup>(i)</sup>) - y<sup>(i)</sup> ) x<sup>(i)</sup><sub>j</sub></h1>
  (Simultaneously update all Θ<sub>j</sub> )<h1>
  )
  </h1></p>
  
  <pre><code>import java.io.FileNotFoundException;
  import java.util.Arrays;
  import java.util.Collections;
  import java.util.List;
  
  public class Logistic {
  
      /** the learning rate */
      private double alpha;
  
      /** the weight to learn */
      private double[] theta;
  
      /** the number of iterations */
      private int ITERATIONS = 3000;
  
      public Logistic(int n) {
          this.alpha = 0.0001;
          theta = new double[n];
      }
  
      private double sigmoid(double z) {
          return (1 / (1 + Math.exp(-z)));
      }
  
      public void train(List&lt;Instance&gt; instances) {
  
      double[] temp = new double[3];
  
      //Gradient Descent algorithm for minimizing theta
      for(int i=1;i&lt;=ITERATIONS;i++)
      {
         for(int j=0;j&lt;3;j++)
         {      
          temp[j]=theta[j] - (alpha * sum(j,instances));
         }
  
         //simulataneous updates of theta  
         for(int j=0;j&lt;3;j++)
         {
           theta[j] = temp[j];
         }
          System.out.println(Arrays.toString(theta));
      }
  
      }
  
      private double sum(int j,List&lt;Instance&gt; instances)
      {
          double[] x;
          double prediction,sum=0,y;
  
  
         for(int i=0;i&lt;instances.size();i++)
         {
            x = instances.get(i).getX();
            y = instances.get(i).getLabel();
            prediction = classify(x);
            sum+=((prediction - y) * x[j]);
         }
           return (sum/instances.size());
  
      }
  
      private double classify(double[] x) {
          double logit = .0;
          for (int i=0; i&lt;theta.length;i++)  {
              logit += (theta[i] * x[i]);
          }
          return sigmoid(logit);
      }
  
  
      public static void main(String... args) throws FileNotFoundException {
  
        //DataSet is a class with a static method readDataSet which reads the dataset
        // Instance is a class with two members: double[] x, double label y
        // x contains the features and y is the label.
  
          List&lt;Instance&gt; instances = DataSet.readDataSet("data.txt");
        // 3 : number of theta parameters corresponding to the features x 
        // x0 is always 1   
          Logistic logistic = new Logistic(3);
          logistic.train(instances);
  
          //Test data
          double[]x = new double[3];
          x[0]=1;
          x[1]=45;
          x[2] = 85;
  
          System.out.println("Prob: "+logistic.classify(x));
  
  
      }
  }
  </code></pre>
  
  <p>Can anyone tell me what am I doing wrong? 
  Thanks in advance! :)</p>