and choose ${\bf K}^i$ to be random and completely independent of ${\bf M}{^i}$, the network failed to learn and error rates stayed at near-chance level. We also experimented with training based on feedback alignment where errors from the top layer are backpropagated using random fixed weights. The network’s performance using feedback alignment is worse than learning using local errors (using either symmetric or sign-concordant weights) as shown in Table \[tab:mnist\].

It is important to note that in feedback alignment, the feedforward weights eventually ’align’ with the random weights used to backpropagate errors [@Lillicrap_etal16] enabling the network to learn. When learning using random fixed local classifiers, and if we choose random error feedback weights, the classifier weights are fixed and thus can not align with the random weights used in the one-step backpropagation. Reliable error information, however, can still reach the layer being trained if the signs of the random backpropagation weights, ${\bf K}^i$, match the signs of the fixed local classifier weights ${\bf M}^i$. This is in-line with previous investigations into the importance of weight symmetry in backpropagation that argue for the importance of sign-concordance between forward and backward weights [@Liao_etal16].

CIFAR10
-------

We trained a convolutional network with three 