A cost function is defined for the network. The loss function tries to estimate how well the neural network is doing at its assigned task. Finally, an optimization technique is applied to minimize the output of the cost function by changing the weights and biases of the network. This process is called training. Training is done one small step at a time. After thousands of steps, the network is typically able to do its assigned task pretty well.