handwritten digits, thus it is suitable for binary data classification test. In contrast, the CIFAR10 was used as non-binary data, on which the hybrid-BDNN was tested. In each dataset, the training data was used to train the networks; the test data was used as validation data. After the training was finished, the iteration with the lowest error rate on test data was seen as the final result.

The classical three-layer network
---------------------------------

In the early stages of ANN, before the DNN was introduced, the three-layer network was widely used for classification tasks. As shown in Fig. \[3layer\], in such network, there are three different layers: the input layer, the hidden layer and the output layer. The input layer contains the input data while the output layer generates the classification results. Besides, all the layers are fully connected. Since this network structure was very classic, so the BDNN of this structure was first tested.

![The structure of three-layer network. \[3layer\]](3layer.eps){width="30.00000%"}

![image](cnn.eps){width="85.00000%"}

For the experiments of three-layer BDNN, the binary data of MNIST was used. Because the original images of MNIST were grayscale, so we used a simple binarization method (fixed threshold) to convert the grayscale images into binary. All the training 