the weight and bias updates, $\Delta {\bf W}^i$ and $\Delta {\bf b}^i$: $$\begin{aligned}
  {\bf e_s}^i &= \frac{dE}{d{\bf s}^i} \\
  {\bf e_y}^i &= {\bf K^i}{\bf e_s^i} \odot {\bf y}{^i}'  \\
  \Delta{\bf W}^i &= -\eta {\bf e_y}^i {\times} {\bf x}^i \\
  \Delta{\bf b}^i &= -\eta {\bf e_y}^i\end{aligned}$$ where $\odot$ is the element-wise multiplication operator, $\times$ is the outer product operator, and $\eta$ is the learning rate. ${\bf K}^i$ is the $N \times C$ matrix used to backpropagate the classifier error to the layer being trained. If we set ${\bf K}^i = {\bf M}{^i}^T$, then the weight and bias updates are executing exact gradient descent to minimize the random classifier error, $E$. In that case, training of each layer is equivalent to training a network with one hidden layer where only the hidden layer’s input weights and biases are trainable, while the output weights, ${\bf M}^i$ are fixed. The learning scheme is illustrated in Fig. \[fig:local\]. For convolutional layers, the learning scheme remains unchanged. The post-activation feature maps tensor is simply flattened to yield a 1D vector before multiplying by the random classifier matrix ${\bf M}$.

![Supervised learning in a multi-layer network using local errors. Biases are omitted for clarity. Red arrows indicate 