Now we can train the hybrid-BDNN by using the gradient descent algorithm.

Special training technique for BDNN
-----------------------------------

Clearly, the training method for BDNN proposed in this paper is very close to the conventional training method. Thus it is possible to borrow the training techniques of conventional neural networks to BDNN, such as dropout and dropconnect. Besides, there are several special training techniques, which are necessary for BDNN.

First, in the training, we must keep the $n$ of an odd number all the time. This is because the output $0$ of $f'$ is not defined. The result of $f'$ should be either a positive number or a negative number. Consequently, in the following experiments, in the BDNN, the neuron number $n$ was always an odd number.

Second, in the training of conventional neural network, the error of the output is determined by the difference between the output neuron and its ground truth. However, in the training of BDNN, the error is determined by not only the difference but also the sign of each value. Assume the error of a certain output neuron $y$ is $e$ and the corresponding ground truth is $y^T$, then the error calculation is given by $$e 