at the rest of the positions. Clearly, we can not use such partial derivative for gradient descent algorithm. Therefore, an approximation of this curve (shown in Fig. \[curve\] (b)) is used to calculate the $\dfrac{\partial y'}{\partial x_k'}$. The new curve is a strait line, go through the point $(-1, l-\dfrac{1}{n})$ and $(1, l+\dfrac{1}{n})$. Here, we assume that $w_k'>0$. If $w_k'<0$, the slope of the approximation line will be reversed. Consequently, by using the approximation, we obtain $$\dfrac{\partial y'}{\partial x_k'}=\dfrac{2w_k'}{n\left|w_k'\right|}.
\label{dx}$$ Similarly, since in function $f'$, the $x_k'$ and $w_k'$ are symmetric, so that $$\dfrac{\partial y'}{\partial w_k'}=\dfrac{2x_k'}{n\left|x_k'\right|}.
\label{dw}$$ Here, in and , the $\frac{ 2 }{ n }$ is the value of the slope; the $\dfrac{w_k'}{\left|w_k'\right|}$ or $\dfrac{x_k'}{\left|x_k'\right|}$ determines the sign of the slope. Please note that the slope value can be set to not only $\frac{2}{n}$ but also other proper values.

With and , the gradient descent algorithm can be applied to train the BDNN. Although an approximation was used, the following experimental results showed that, the BDNN could be trained well with such method.

The training of transition part of hybrid-BDNN
----------------------------------------------

As shown in Fig. \[HybridBDNN\], now the backpropagation can be applied to the normal neural network part 