{#training}
========================================

Usually, the gradient descent algorithm is seen as the training method for neural networks. For the training of BDNN and hybrid-BDNN, we also used this algorithm. However, compared with the conventional neural network, the BDNN has obviously different properties, thus we can not directly use this algorithm on BDNN training. In order to solve this problem, some approximation and conversion are applied to BDNN to make it suitable for gradient descent training.

Gradient descent training for BDNN
----------------------------------

In the conventional gradient descent training, in each iteration, the weights are adjusted by a small value (depends on the error propagation and learning rate). Nevertheless, since the weights in BDNN are binarized, thus it is hard to adjust the weights in the same way. Hence, in order to apply the gradient descent algorithm, in the training of BDNN, the real numbers are used instead of the binary values. A conversion function $C$ is defined to convert the real numbers to corresponding binary values, which is $$C(x) = \begin{cases} 1,& if \quad x \geq 0 \\ -1,& if \quad x < 0.
\label{C}
\end{cases}$$ With function , we can convert the trained weights of real number to binary value.

![image](curve.eps){width="85.00000%"}

