title: Multilayered neural network converging to infinity after barely 10 epochs with 0.1 learning rate
body: <p>So I am trying to design a multilayered neural network with 3 input neurons 3 hidden neurons and 1 output neuron. </p>
  
  <p>I plan on making it learn the 3 bit xor pattern.  b1 xor b2 xor b3 kind of a table.
  Right now I am trying to teach it  the following table.</p>
  
  <pre><code>0d,0d,0d    =&gt;0d
  0d,0d,1d    =&gt;1d    
  0d,1d,0d    =&gt;1d
  0d,1d,1d    =&gt;0d
  1d,1d,0d    =&gt;0d
  1d,1d,1d    =&gt;1d
  </code></pre>
  
  <p><img src="https://i.stack.imgur.com/yNlnF.png" alt="enter image description here"></p>
  
  <p>hidden[] is the weights between hidden layer and the output layer
  input[] is the weights between input and hidden layer.</p>
  
  <p>Ignore the D's</p>
  
  <p>Here's the code and it converges after barely 10 epochs with 0.1 learning rate and the value of error is down to infinity :|</p>
  
  <pre><code>class neuron{
      double hidden[] = {1d,1d,1d};
      double input[][] = { {1d,1d,1d},{1d,1d,1d},{1d,1d,1d}};
      double learning = 0.1;
      double bias = 1d;
  
      public double[] react(double a,double b,double c){
          double h1 = a*input[0][0] + b*input[0][1] + c*input[0][2];
          double h2 = a*input[1][0] + b*input[1][1] + c*input[1][2];
          double h3 = a*input[2][0] + b*input[2][1] + c*input[2][2];
          //System.out.println(input[0][0]);
          double total[] = new double[4];
          total[0] = h1*hidden[0] + h2*hidden[1] + h3*hidden[2] + bias;
          total[1] = h1; total[2] = h2; total[3] = h3;
          return total;
      }
  
      public void learn(double a,double b, double c, double ideal){
          double actual[] = react(a,b,c);
  
          double error = ideal - actual[0];
          System.out.println(error);
          error *= learning;
  
          for( int i = 0; i &lt; 3; i++ )
              hidden[i] += error * actual[i+1];
          bias += error;
          for( int i = 0; i &lt; 3; i++ ){
              input[i][0] +=  error * actual[i+1] * a;
              input[i][1] +=  error * actual[i+1] * b;
              input[i][2] +=  error * actual[i+1] * c;
          }
  
          }
      }
  
  
  
  public class multilayer{
  
          public static void main(String argz[]){
          neuron a = new neuron();
          for( int i = 0; i &lt; 20; i++){
              a.learn(0d,0d,0d,0d);
              a.learn(0d,0d,1d,0d);   
              a.learn(0d,1d,0d,0d);
              a.learn(0d,1d,1d,0d);
              a.learn(1d,1d,0d,0d);
              a.learn(1d,1d,1d,1d);
  
          }
          System.out.println(a.react(0d,0d,0d)[0] &gt;=0.5 ? 1 : 0);
          System.out.println(a.react(0d,0d,1d)[0]&gt;=0.5 ? 1 : 0);
          System.out.println(a.react(0d,1d,0d)[0]&gt;=0.5 ? 1 : 0);
          System.out.println(a.react(1d,1d,0d)[0]&gt;=0.5 ? 1 : 0);
          System.out.println(a.react(0d,1d,1d)[0]&gt;=0.5 ? 1 : 0);
          System.out.println(a.react(1d,1d,1d)[0]&gt;=0.5 ? 1 : 0);
          }
  }
  </code></pre>
  
  <p>I knocked off a hidden neuron and it still converges to infinity all the weights !! </p>