title: Is it correct backpropogation for NN
body: <p>I have 3 layer NN. 3 neurons in input, then 3 with RELU activation, then 3 with Sigmoig activation, then 3 with Softmax activation(its output).
  And there are weights:</p>
  <pre><code>W = np.array([[[0.1, 0.2, 0.3],
               [0.3, 0.2, 0.7],
               [0.1, 0.2, 0.3]],
              [[0.2, 0.3, 0.5],
               [0.3, 0.5, 0.7],
               [0.6, 0.4, 0.8]],
              [[0.1, 0.4, 0.8],
               [0.3, 0.7, 0.2],
               [0.5, 0.2, 0.9]]])
  </code></pre>
  <p>Input and result:</p>
  <pre><code>X = np.array([0.1, 0.2, 0.7])
  Y = np.array([1.0, 0.0, 0.0])
  </code></pre>
  <p>Feed forward:</p>
  <pre><code>Z1 = W[0] @ X
  H1 = RELU(Z1)
  Z2 = W[1] @ H1
  H2 = Sigmoid(Z2)
  Z3 = W[2] @ H2
  H3 = Softmax(Z3)
  Loss = CrossEntropy(Ypred,Y)
  </code></pre>
  <p>Back propagation:</p>
  <pre><code>dLoss = dCrossEntropy(Ypred, Y)
  o_error = dLoss
  o_delta = dLoss*(dSoftmax(o_error))
  
  z2_error = o_delta*(W[2].T)
  z2_delta = z2_error * dSigmoid(Z2)
  
  z1_error = z2_error*(W[1].T)
  z1_delta = z1_error * dRELU(Z1)
  
  W[0] += X.T.dot(z1_delta)
  W[1] += (Z1.T).dot(z1_delta)
  W[2] += (Z2.T).dot(o_delta)
  </code></pre>
  <p>Am I correct in backpropagation?</p>