title: With bias, ANN not converge anymore
body: <p>I'm learning ANN, I did two script (in fortran90 and python) for simple binary classification problem. </p>
  
  <p>I first did without bias, and I get a good convergence. But adding a bias for each node it does not converge anymore (or everything is going near to 0 or everything near to 1) </p>
  
  <p>The bias is 1 and has a specific weight for each node. It is randomly initialized and then update adding delta such as others weights. I have tried to change gradient step size but it still doing the same thing. </p>
  
  <p>Someone has any clues ?....</p>
  
  <p>EDIT :</p>
  
  <p>The network :</p>
  
  <pre><code>    IN                HIDDEN             OUTPUT node
  (each column is a     LAYERS          (each column is
  training data)   (2layers of 3node)   the wanted result)
  
  
            W1      .___W2__.    W3
  |0|0|1|1|--------&gt;|___|___|______
  |0|1|0|1|--\/_-\-&gt;|___|___|______\_--&gt; |1|1|0|0|
  |1|0|1|1|--/\__/-&gt;|___|___|______/
  </code></pre>
  
  <p>The activation function is a sigmoid (1/(1+exp(-x)))</p>
  
  <p>The weight are initialized with a normal distribution in range of [-1, 1]</p>