hyper-parameters. We use a network with 3 fully connected hidden layers with $1000$ neurons per layer and train the weights in the entire network using local errors. As a baseline, we also train a 2-hidden layers network and a 3-hidden layers network using standard backpropagation where each hidden layer also has $1000$ neurons. Dropout was used in all networks to reduce overfitting. We first used fixed symmetric random weights in the forward and backward pathways in the local error loops, i.e, ${\bf K}^i = {\bf M}{^i}^T$ in all layers. The results are shown in Fig. \[fig:mnist\_symmetric\]. The local classifier errors improve for the second and third hidden layers compared to the first hidden layer implying that the network is able to make use of depth to obtain better accuracy. The local classifier errors in the second and third layers are similar implying that the network is unable to make use of the increased depth beyond two hidden layers for this simple dataset. This observation is also valid for standard backpropagation where accuracy does not improve when going from two hidden layers to three hidden layers. When training using local errors, we also ran experiments where the local classifier weights were 