title: Why the BIAS is necessary in ANN? Should we have separate BIAS for each layer?
body: <p>I want to make a model which predicts the future response of the input signal, the architecture of my network is [3, 5, 1]:</p>
  
  <ul>
  <li>3 inputs, </li>
  <li>5 neurons in the hidden layer, and </li>
  <li>1 neuron in output layer. </li>
  </ul>
  
  <p>My questions are:</p>
  
  <ol>
  <li>Should we have separate BIAS for each hidden and output layer?</li>
  <li>Should we assign weight to BIAS at each layer (as BIAS becomes extra value to our network and cause the over burden the network)?</li>
  <li>Why BIAS is always set to one? If eta has different values, why we don't set the BIAS with different values?</li>
  <li>Why we always use log sigmoid function for non linear functions, can we use tanh ?</li>
  </ol>