and the BDNN part. However, in order to apply the backpropagation on hybrid-BDNN, we still need to consider about the backpropagation of the transition part.

First, the same with the BDNN training, in the training of the transition part, the real numbers are used instead of the binary values. Assume that $x_1'$ is the corresponding real number of $x_1$ and function $t'$ is the real number version of function $t$. Then, the same with $f'$, in order to keep the same forward propagation result, $t'$ should satisfy the following equation $$C( t'(\bm{\vec g},\bm{\vec w}) ) = C(\bm{\vec g},\bm{\vec w}).$$ Therefore, we define $t'$ as $$t'(\bm{\vec g}, \bm{\vec w})=A(\sum_{i=1}^{n}g_i w_i)-T.
\label{tp}$$ Obviously, the $t'$ of satisfies the request. If we set $T=0$, then $t'$ is just the activation function. Naturally, for a certain neuron $g_k$ and its weight $w_k$, the two derivatives of the transition part are given by $$\dfrac{\partial x_1'}{\partial g_k}=\dfrac{\partial A}{\partial g_k}, \quad \dfrac{\partial x_1'}{\partial w_k}=\dfrac{\partial A}{\partial w_k}.
\label{dt}$$ In , the two partial derivatives are just the same with the normal neural network. With , the backpropagation of the transition part becomes possible.

Consequently, by using the above method, the backpropagation can be applied to the whole hybrid-BDNN. 