title: Java Back-propagation ANN output values
body: <p>I'm trying to write a simple implementation of a back-propagation ANN in Java and I'm getting very odd output. The ANN has an input layer with two nodes (one for each value in input vector), a single hidden layer with 4 nodes (I have experimented with more to no avail) and an output layer with 3 nodes. The three output nodes represent the three possible classifications of the data, with "one-hot" encoding.</p>
  
  <p>Some input to the ANN is formatted as follows:</p>
  
  <pre><code>0.020055 0.40759 2
  0.020117 0.14934 3
  0.020117 0.25128 3
  0.020262 1.6068 1
  </code></pre>
  
  <p>with the 2 decimals as the 2 inputs, and the integer being the desired classification (1-3). </p>
  
  <p>After training the ANN with every possible data combination in the list, and All of the outputs I get look like this, with the decimals as the output from the three output nodes</p>
  
  <pre><code>0.11237534579646044   0.15172262242962917   0.7906017313009316   2
  0.13686775670201043   0.1774606939461421   0.7656339988150088   3
  0.13554918638846133   0.1761024282314506   0.766924262279491   3
  0.06185317503169881   0.09410559150503017   0.8516964148498476   1
  </code></pre>
  
  <p>The way I have written it, each line should display a "high" value (close to 0.95 due to the sigmoidal function) and the rest should be "low" (close to 0.05).</p>
  
  <p>Below is the method I have written to calculate the final output values:</p>
  
  <pre><code>public static double [] testANN(double [] input, List&lt;BPNode&gt; hiddenLayer, List &lt;BPNode&gt; outputLayer){
      double [] outInputs = new double[hiddenLayer.size()];
      double [] results = new double[outputLayer.size()];
      for(int i = 0; i&lt;hiddenLayer.size(); i++){
          BPNode node = hiddenLayer.get(i);
          node.inputs = input.clone();
          outInputs[i] = node.getOutput();
      }
      for(int i = 0; i&lt;outputLayer.size(); i++){
          BPNode node = outputLayer.get(i);
          node.inputs = outInputs.clone();
          results[i] = node.getOutput();
      }
  
      return results;
  }
  </code></pre>
  
  <p>And here is the method to train the ANN with the back-propagation algorithm:</p>
  
  <pre><code>public static void trainANN(double [] input, int desired, List&lt;BPNode&gt; hiddenLayer, List &lt;BPNode&gt; outputLayer){
     double [] d = new double[outputLayer.size()];
     d[0] = desired==1 ? 0.95 :0.05;
     d[1] = desired==2 ? 0.95 :0.05;
     d[2] = desired==3 ? 0.95 :0.05;
     double [] [] weights = new double[outputLayer.size()][hiddenLayer.size()];
     double [] output = testANN(input,hiddenLayer,outputLayer);
     double [] del = new double[outputLayer.size()];
     for(int i = 0; i&lt;outputLayer.size(); i++){
         del[i] = (d[i]-output[i])*output[i]*(1-output[i]);
         for (int j = 0; j&lt;outputLayer.get(i).weights.length; j++){
             weights[i][j] = outputLayer.get(i).weights[j];
             outputLayer.get(i).weights[j]+=0.2*del[i]*outputLayer.get(i).inputs[j];
         }
     }
     for(int i = 0; i&lt;hiddenLayer.size(); i++){
         double hiddenDel = 0.0;
         for(int j = 0; j&lt;outputLayer.size(); j++){
             hiddenDel+=(del[j]*weights[j][i]*hiddenLayer.get(i).getOutput()*(1-hiddenLayer.get(i).getOutput()));
         }
         for(int j = 0; j&lt;hiddenLayer.get(i).weights.length; j++){
             hiddenLayer.get(i).weights[j]+=0.2*hiddenDel*input[j];
         }
     }
  </code></pre>
  
  <p>}</p>
  
  <p>And finally, here's the Node class I have used to implement the ANN:</p>
  
  <pre><code>public class BPNode {
  
  public double [] inputs = new double[10];
  public double [] weights = new double[10];
  
  public BPNode(double [] w){
      weights = w;
  }
  
  public double getOutput() {
      double a = 0;
      for(int j = 0; j&lt;inputs.length; j++){
          a += (inputs[j] * weights[j]);
      }
      return sigmoid(a,10.0);
  }
  
  private static double sigmoid(double x, double m)
  {
      return 1 / (1 + Math.exp(-x*m));
  }
  
  }
  </code></pre>
  
  <p>All of the weights have been initialized to 0.1, and the nodes have been placed in the Array Lists. Thank you so much for your help.</p>