 <p>Generally: yes, the bias weights should be updated and included in training just as any other weight in the NN (also in backpropagation).</p> <p>In the example you've posted, the bias <code>b1</code> is added to both neurons the hidden layer, and bias <code>b2</code> to both neutrons in the output layer</p> <pre><code>Hidden layer: h1 = i1*w1 + i2*w2 + 1*b1 h2 = i1*w3 + i2*w4 + 1*b1 Output layer: o1 = h1*w5 + h2*w6 + 1*b2 o2 = i2*w7 + h2*w8 + 1*b2 </code></pre> <p>With initial, and in this example, fixed, biases</p> <pre><code>b1 = 0.35 b2 = 0.60 </code></pre> <p>This means that the biases for hidden neutrons is always exactly 0.35, and for output neutrons exactly 0.60. This, however, is not usual practice, as you want to train your NN to find "good" biases just as much you want it to train to find good weights.</p> <p>Note also, in the comments of the link you provided, that another user has asked why biases are not changed, and the author has replied, quote:</p> <blockquote> <p>"Hey, in the tutorials I went through they didn’t update the bias which is why I didn’t include it here."</p> </blockquote> <p>This lack of a specific "why" possibly implies that the author of this example/tutorial is, however well-versed, no expert at the subject of NN, so you shouldn't out to much weight (no pun intended...) into the fast that the biases are not changed in this specific example.</p> <hr> <p>If you really want to dig into a sound and thorough covering of NN in the context of back propagation, I would rather recommend you Michael Nielsen excellent book on NN and deep learning, specifically for this subject, Chapter 2. Note that the bias weights, here, are treated just as weights for neuron-neuron data transfer.</p> <ul> <li><a href="http://neuralnetworksanddeeplearning.com/chap2.html" rel="nofollow">http://neuralnetworksanddeeplearning.com/chap2.html</a></li> </ul> <p>Michael is a Google Researcher with numerous publicised articles in the subject of advanced NN and deep learning.</p> <blockquote> <p>At the heart of backpropagation is an expression for the partial derivative ∂C/∂w of the cost function C with respect to any weight w (or bias b) in the network. The expression tells us <strong>how quickly the cost changes when we change the weights and biases.</strong></p> </blockquote>