<p>I think this may help： </p>

<blockquote>
  <p>In [18], two approaches for back-propagating gradients through a
  computational graph are described. The first, which the authors refer
  to as symbol-to-number differentiation, re- ceives a set of input
  values and then computes the numerical values of the gradients at
  those input values. It does so by explicitly traversing the graph
  first in the forward order (forward-propagation) to compute the cost,
  then in reverse order (back-propagation) to compute the gradients via
  the chain rule. Another approach, more relevant to TensorFlow, is what
  [18] calls symbol-to-symbol derivatives and [8] terms automatic
  gradient computation. In this case, gradients are not computed by an
  explicit implementation of the back- propagation algorithm. Rather,
  special nodes are added to the computational graph that calculate the
  gradient of each operation and thus ultimately the chain rule. To
  perform back- propagation, these nodes must then simply be executed
  like any other nodes by the graph evaluation engine. As such, this
  approach does not produce the desired derivatives as a numeric value,
  but only as a symbolic handle to compute those values.</p>
</blockquote>

<p>Reference: <a href="http://arxiv.org/abs/1610.0117" rel="nofollow noreferrer">http://arxiv.org/abs/1610.0117</a></p>
How tensorflow use back propagation in training the model?
<p>I am a beginner with tensorflow and I want to implement MLP and train it based on the back propagation algorithm but when I read tutorials I found that it uses optmizers like “Stochastic Gradient Descent” and called it back propagation without implementing the algorithm phases. How is this back propagation?</p>