follows: $$x_1=t(\bm{\vec g},\bm{\vec w}) = \begin{cases} 1,& if \quad A(\sum_{i=1}^{n}g_i w_i) \geq T \\ -1,& if \quad A(\sum_{i=1}^{n}g_i w_i) < T,
\label{t}
\end{cases}$$ where $A$ is activation function and $T$ is a fixed threshold. In fact, the calculation $A(\sum_{i=1}^{n}g_i w_i)$ is just the basic function of the normal neural network part. The means that we use a threshold $T$ to convert the output real number of the normal neural network part into binary value.

With , the forward propagation of the whole hybrid-BDNN can be realized. This network can be used for any input data. By the way, if the range of activation function $A$ is $(-1,1)$, then we can set the threshold $T=0$. This is convenient for the training.

In summary, the BDNN is very suitable for binary input data classification, such as binary image of characters. If the input data is non-binary, the hybrid-BDNN can be used. Please note that it is better not to use too many layers in normal neural network part, otherwise the computation speed of hybrid-BDNN may be slowed much. In the experiments, we only used one layer (including the input data) as the normal neural network part.

Training method for BDNN and hybrid-BDNN 