<p>Conjugate gradient and quasi-Newton algorithms are still gradient descent algorithms. Backpropagation (or backprop) is <a href="https://idontgetoutmuch.wordpress.com/2013/10/13/backpropogation-is-just-steepest-descent-with-automatic-differentiation-2/" rel="noreferrer">nothing more than a fancy name</a> to a gradient computation. </p>

<p>However, the original question of alternatives to backprop is very important. One of the recent alternatives, for example, is <a href="http://arxiv.org/abs/1602.05179" rel="noreferrer">equilibrium propagation</a> (or shortly eqprop).</p>
Are there alternatives to backpropagation?
<p>I know a neural network can be trained using gradient descent and I understand how it works.</p>

<p>Recently, I stumbled upon other training algorithms: conjugate gradient and quasi-Newton algorithms.
I tried to understand how they work but the only good intuition I could get is that they use higher order derivative.</p>

<p>My questions are the following: are those alternative algorithms I mentioned fundamentally different from a backpropagation process where weights are adjusted by using the gradient of the loss function? 
If not, is there an algorithm to train a neural network that is fundamentally different from the mechanism of backpropagation?</p>

<p>Thanks</p>