= \begin{cases} 0,& if \quad yy^T > 0 \\ \frac{1}{2}(y-y^T),& if \quad yy^T < 0.
\end{cases}
\label{e}$$ The means that if the output neuron and its ground truth are both positive or negative, then the error of this output is $0$; otherwise, the error is calculated as conventional neural network. Actually, according to , if the output neuron has the same sign with its ground truth, then the output is seen as correct. This technique is essential for the BDNN training, without which the training can not converge.

Experiments
===========

In the experiments, two different network structures were studied — the classical three-layer network and the convolutional neural network (CNN). Besides the BDNN and hybrid-BDNN, the conventional neural networks of the same structures were also tested for comparison. Since now we only have the basic training method for BDNN, so in order to make fair comparison, most of the training optimization techniques of the conventional neural network were not used. For example, in the experiments, the learning rate was fixed and the simple hyperbolic tangent function was used as activation function.

Moreover, two datasets, MNIST and CIFAR10 were used for the experiments. As we know, MNIST is a dataset of 