Feed-Forward Optimization With Delayed Feedback for Neural Networks
Backpropagation has long been criticized for being biologically implausible,
relying on concepts that are not viable in natural learning processes. This
paper proposes an alternative approach to solve two core issues, i.e., weight
transport and update locking, for biological plausibility and computational
efficiency. We introduce Feed-Forward with delayed Feedback (F$^3$), which
improves upon prior work by utilizing delayed error information as a
sample-wise scaling factor to approximate gradients more accurately. We find
that F$^3$ reduces the gap in predictive performance between biologically
plausible training algorithms and backpropagation by up to 96%. This
demonstrates the applicability of biologically plausible training and opens up
promising new avenues for low-energy training and parallelization.
Katharina Flügel, Daniel Coquelin, Marie Weiel, Charlotte Debus, Achim Streit, Markus Götz
# Feed-Forward Optimization With Delayed Feedback for Neural Networks

###### Abstract

Backpropagation has long been criticized for being biologically implausible, relying on concepts that are not viable in natural learning processes. This paper proposes an alternative approach to solve two core issues, i.e., weight transport and update locking, for biological plausibility and computational efficiency. We introduce **F**eed-**F**orward with delayed **F**eedback (F\({}^{3}\)), which improves upon prior work by utilizing delayed error information as a sample-wise scaling factor to approximate gradients more accurately. We find that F\({}^{3}\) reduces the gap in predictive performance between biologically plausible training algorithms and backpropagation by up to 96%. This demonstrates the applicability of biologically plausible training and opens up promising new avenues for low-energy training and parallelization.

## 1 Introduction

Today, nearly all artificial neural networks are trained with gradient-descent-based optimization methods [1]. Typically, backpropagation [2] is used to efficiently compute the network's gradients. However, backpropagation relies on multiple biologically implausible factors, making it highly unlikely that the human brain functions in a similar fashion [3]. Besides increasing biological plausibility, solving these issues could significantly improve neural algorithms by increasing generalization performance, requiring fewer computational operations, and consuming less energy [4; 5]. As backpropagation makes up a significant portion of the computational and memory cost of neural network training, replacing it can yield significant time and energy savings. Diminishing the energy requirements is especially important in embedded and edge computing. Neuromorphic devices, for example, are novel brain-inspired hardware architectures [6] designed to accelerate neural networks and reduce the required energy. However, they are often unable to implement backpropagation and are thus in particular demand for biologically plausible alternatives [7].

Several training algorithms have been proposed to increase biological plausibility and reduce computational requirements. One option is removing the backward pass entirely, thus training an artificial neural network using only a forward pass. This can also lower the computational cost and open up new possibilities for parallelization but typically results in reduced predictive performance.

This paper presents _Feed-**F**orward with delayed **F**eedback (F\({}^{3}\)), our novel, biologically-inspired, backpropagation-free training algorithm for deep neural networks 1. F\({}^{3}\) yields superior predictive performance on a variety of tasks compared to prior approaches by using delayed error information
from previous epochs as feedback signals. Using fixed feedback weights, F\({}^{3}\) passes these feedback signals directly to each layer without layer-wise propagation. Together, this resolves the dependence on downstream layers, making it possible to update the network parameters during the forward pass. As a result, F\({}^{3}\) eliminates the need for the computationally expensive backward pass. This reduces both the computational cost and the required communication between layers, opening up promising possibilities for parallel and embedded training. F\({}^{3}\) also enables in-the-loop training on neuromorphic and edge devices, making it a promising candidate for application in low-powered computational settings.

Our main contributions are as follows:

* F\({}^{3}\), a novel, biologically-inspired training algorithm for deep neural networks.
* Using delayed error information as feedback signal for improved predictive performance of backpropagation-free training.
* Empirical evaluation of F\({}^{3}\) on various datasets and neural network architectures.
* Theoretical analysis of the computational complexity of F\({}^{3}\) compared to backpropagation.

## 2 Related Work

While biological plausibility is not a necessary requirement for training artificial neural networks, there has been great interest in methods that combine biological plausibility with effective training [11; 5; 3; 12; 13; 14]. Two of the most substantial obstacles in that regard are the weight transport and the update locking problem [5; 10].

_Weight Transport_[15] Backpropagation reuses the forward weights symmetrically in the backward pass to propagate the gradients. However, synapses are unidirectional; and synchronizing separate forward and backward pathways precludes biological plausibility [16; 6]. The weights' non-locality additionally constrains the memory access pattern, which can severely impact the computational efficiency [4].

Figure 1: **F\({}^{3}\) (a)** solves both the weight transport and the update locking problem. In contrast to prior approaches, it uses delayed error information in the updates, improving the predictive performance. The current error signal \(e_{t}\) in epoch \(t\) is stored (green) and used in the forward pass (blue) of the next epoch \(t+1\). This eliminates the backward pass (red) for all hidden layers.

**Previous approaches (b) to (e):** Backpropagation (BP) (b) is not biologically plausible due to the weight transport and update locking problems caused by the backward pass. FA [8] (c) and DFA [9] (d) solve the weight transport problem by replacing the backward paths with random feedback weights \(B_{i}\) independent of the forward weights \(W_{i}\). However, they are still update-locked as they depend on the current error \(e_{t}\). DRTP [10] (e) releases update locking by using the target \(y^{*}\) instead of the error \(e_{t}\), but this comes at the cost of reduced accuracy.
_Update Locking_[17, 18] Artificial neural networks are typically trained with a clear separation between forward and backward passes. Every layer in a neural network must wait for all downstream layers to complete their forward and backward pass before completing its own backward pass. This introduces a considerable delay between a layer's forward pass and the calculation of the gradients used to update its weights. This can significantly increase training times as results of the forward pass need to be buffered or recomputed [19, 10] and restricts parallelization. Moreover, it is not biologically plausible, as a neuron's activation may have already changed during the delay, causing a desynchronization with the error signal [20].

Several approaches have been suggested to address both weight transport and update locking. Figure 1 illustrates some of these approaches compared to our approach \(\mathrm{F}^{3}\).

To solve the weight transport problem, _target propagation_[13] replaces the loss gradients with layer-wise target values computed by auto-encoders. Using an additional linear correction, the so-called difference target propagation, this approach achieves results comparable to backpropagation. Ororohia and Mali [5] take a similar approach to target propagation but employ the current layer's pre-activation and the next layer's post-activation instead of generating the layer-wise targets with auto-encoders. _Feedback alignment (FA)_[8], illustrated in Figure 1 (c), replaces the forward weights with fixed random feedback weights in the backward pass, thereby demonstrating that symmetric feedback weights are not required for effective training. FA solves the weight transport problem; however, the training signals remain non-local, meaning the gradient estimates are propagated backward through the network's hidden layers. To alleviate this, _direct feedback alignment (DFA)_[9] passes the error directly to each hidden layer, see Figure 1 (d).

For solving update locking, Mostafa et al. [19] use auxiliary classifiers to generate _local errors_ from each layer's output. With this strategy, they outperform FA and approach backpropagation-like accuracy. Improving upon this, Nokland and Eidnes [21] demonstrate that combining local classifiers with a so-called local similarity matching loss can close the gap to backpropagation. _Decoupled greedy learning (DGL)_[22] follows a similar approach that is based on greedy objectives, which even scales to large datasets such as ImageNet [23].

Another approach to address update locking are _synthetic gradients_[18, 17], which model a subgraph of the network and predict its future output based on only local information. Replacing backpropagation with these synthetic gradients decouples the layers, resulting in so-called _decoupled neural interfaces (DNIs)_. The same approach can also be used to predict synthetic input signals and thus solve the _forward locking problem_[18]. _Direct random target projection (DRTP)_[10], illustrated in Figure 1 (e), builds upon DFA to solve weight transport and update locking simultaneously. By substituting the error with the targets, DFA's direct backward paths can be replaced with direct forward paths to the hidden layers. While DRTP cannot reach the same accuracy as FA and DFA, it significantly outperforms training only the last layer of the network, demonstrating its ability to train hidden layers.

The _forward gradient_[24] is another approach to gradient approximation without backpropagation. It is based on forward mode automatic differentiation, which requires multiple runs to compute the exact gradient. To reduce this overhead, the gradient is estimated with the forward gradient based on the directional derivative.

The recently proposed _Forward-Forward (FF)_ algorithm [25] replaces the backward pass with another forward pass on so-called negative data. Each layer is trained separately to distinguish the real data in the first forward pass from the negative data in the second forward pass. _PEPITA_[26] takes a similar approach by executing two forward passes on different input data, where the input to the second forward pass is modulated based on the error of the first one.

_Multilayer SoftHebb_[27] combines Hebbian learning with a softmax-based plasticity rule and a corresponding soft anti-Hebbian plasticity to enable training deep networks of a specific architecture. Using greedy layer-wise training, multilayer SoftHebb can train the hidden layers in a single unsupervised epoch, followed by up to 200 epochs of supervised training for the linear classifier head.

_Predictive coding_[28, 29] is another approach to biological plausibility based on the predicting processing model, an influential theory in cognitive neuroscience. It compares predictions of the expected input signal to a layer to the actual observations and aims to minimize this prediction error. In contrast to the more traditional feed-forward model, only the residual, the prediction error, is
passed to the next level [30]. Predictive coding has been successfully applied to artificial neural networks; for example, Whittington and Bogacz [31] show that predictive coding networks can approximate backpropagation on multilayer-perceptrons. This has since been extended to arbitrary computation graphs, including convolutional and recurrent neural networks [32].

## 3 Feed-Forward-Only Training

### Training Neural Networks with Backpropagation

Let us consider a multi-layer fully-connected neural network like the one illustrated in Figure 1. It consists of \(K\) fully-connected layers where each layer \(i\) has weights \(W_{i}\) and biases \(b_{i}\) and passes its output \(h_{i}\) to the next layer. To obtain the network's output \(y=h_{K}\), the forward pass computes the output of layer \(i\) as

\[z_{i} =W_{i}h_{i-1}+b_{i} \tag{1}\] \[h_{i} =f(z_{i}), \tag{2}\]

with \(f\) being a non-linear activation function. A loss function \(L\) measures how close the last layer's output \(y\) is to the target value \(y^{*}\). When training a neural network, we aim to decrease this loss by adjusting the network's parameters \(W_{i}\) and \(b_{i}\). The _credit assignment problem_[33] states how much of the loss can be attributed to each parameter and how they should be changed accordingly. A common solution to the credit assignment problem is updating each parameter proportional to the gradient of the loss with respect to that parameter.

Backpropagation can compute this gradient efficiently by applying the chain rule in a backward pass over the network. For example, it computes the gradient of the loss with respect to \(h_{i}\) as

\[\delta h_{i}=\frac{\partial L}{\partial h_{i}}=W_{i+1}^{T}\delta z_{i+1}, \tag{3}\]

which can also be expressed as

\[\delta h_{i} =W_{i+1}^{T}\left(\delta h_{i+1}\odot f^{\prime}\left(z_{i+1} \right)\right) \tag{4}\] \[=\boxed{\frac{\partial L}{\partial y}\boxed{W_{i+1}^{T}\left( \frac{\partial y}{\partial h_{i+1}}\odot f^{\prime}\left(z_{i+1}\right) \right)}}. \tag{5}\]

The gradient \(\delta h_{i}\) thus depends on an error term (green), represented by \(\partial L/\partial y\), and the feedback path (blue). This feedback path relies on both the transpose of the forward weights \(W_{j}^{T}\) and the derivative of the activation \(f\) at the input \(z_{j}\) for all downstream layers \(j\in\{i+1,\dots,K\}\). The gradients of the weights \(W_{i}\) and biases \(b_{i}\) can be derived directly from \(\delta h_{i}\).

### Approximating Gradients without Backpropagation

Not all of the information necessary to compute \(\delta h_{i}\) as described in Equation (5) is available to a biologically plausible training algorithm, as a) the algorithm cannot rely on symmetric feedback weights (weight transport) and thus cannot utilize the forward weights \(W\) or their transpose, and b) the update step cannot wait for the downstream layers \(j>i\) (update locking). Thus, neither the intermediate values \(z_{j}\), the final output \(y\), nor the loss \(L\) is available. This means that biologically plausible training algorithms cannot compute the true gradient \(\delta h_{i}\). Instead, they typically approximate the gradients based on the information that is available.

F\({}^{3}\) is inspired by DRTP [10], which approximates the gradients \(\delta h_{i}\) using the targets \(y^{*}\) instead of the error signal and fixed random feedback weights for the feedback path. However, the targets are not a practically good approximation of the true error signal as they contain no information on the error magnitude and only limited information on its direction, which can severely impact the training. Therefore, we propose _Feed-Forward with delayed _Feedback (F\({}^{3}\))_, which uses delayed error information \(e_{t-1}\) to approximate the error signal more precisely while remaining update locking free.
### Delayed Error Information as Feedback Signal

\(\mathrm{F}^{3}\), illustrated in Figure 1 (a), approximates the gradient \(\delta h_{i}\) as

\[\delta^{\mathrm{F}^{3}}h_{i}=\boxed{B_{i}^{T}}\boxed{e_{t-1}}. \tag{6}\]

We reuse the idea of fixed feedback weights \(B_{i}\) to replace the feedback path (blue) but use delayed error information \(e_{t-1}\) as error signal (green). Computing \(\delta^{\mathrm{F}^{3}}h_{i}\) is thus independent of other layers. The remaining gradients within the layer, e.g., for the weights and biases, rely only on the incoming gradient \(\delta h_{i}\) and information within the layer itself. They can be computed with the chain rule in the same way as with backpropagation using the approximate gradient \(\delta^{\mathrm{F}^{3}}h_{i}\). This makes all gradient approximations independent of downstream layers and allows updating a layer immediately during the forward pass, eliminating the need for a separate backward pass. Since the final layer \(K\) has access to the model's output \(y=y_{K}\), it can use the current error signal and compute the true gradients. As a result, \(\mathrm{F}^{3}\) only affects hidden layers; the output layer is updated as it would be with backpropagation.

Algorithm 1 outlines how \(\mathrm{F}^{3}\) is used to train a fully-connected neural network. The layers are processed in a forward pass which computes the layer's output \(h_{i}\), followed immediately by computing the approximate gradients \(\delta^{\mathrm{F}^{3}}W_{i}\) and \(\delta^{\mathrm{F}^{3}}b_{i}\) and updating the layer's parameters \(W_{i}\) and \(b_{i}\). The approximate gradients can be used to update the network's parameters like the true gradients. For example, using stochastic gradient descent with a learning rate of \(\eta\) would result in the update

\[W_{i} \gets W_{i}-\eta\delta^{\mathrm{F}^{3}}W_{i} \tag{7}\] \[b_{i} \gets b_{i}-\eta\delta^{\mathrm{F}^{3}}b_{i}. \tag{8}\]

\(\mathrm{F}^{3}\) is thus independent of the optimizer and can be used with various gradient-based optimizers from standard SGD to more advanced methods like Adam [34].

\(\mathrm{F}^{3}\) utilizes sample-wise delayed error information

\[e_{t-1}=\left(\frac{\partial L}{\partial y}\right)_{t-1} \tag{9}\]

from the previous epoch \(t-1\) to approximate the current error signal

\[e_{t}=\left(\frac{\partial L}{\partial y}\right)_{t}. \tag{10}\]
The feedback is thus always delayed by one epoch. The targets \(y^{*}\) are used to initialize the error information in the first epoch. Alternatively, the error information can be initialized from an additional inference-only epoch \(t=0\), leaving the model parameters unchanged. On a technical level, F\({}^{3}\) stores each training sample's error information at the end of the forward pass. As the loss is already used to train the last layer, this requires no additional computation.

Delayed error information more closely resembles the actual error signal than, for example, the targets \(y^{*}\) used in prior, similarly plausible approaches [10]. Using delayed error information includes both the magnitude and direction of the error in the update, which helps to differentiate between samples requiring more attention and those not. F\({}^{3}\) thus offers a more accurate gradient approximation.

### Different Variants of Delayed Error Information

The sample-wise gradient of the loss with respect to the model's output \(y_{t-1}\) is the most natural choice for modeling the error signal as it corresponds to the information used in backpropagation delayed by one epoch. However, F\({}^{3}\) can also use other types of delayed error information. For example, it can use the delayed error

\[y^{*}-y_{t-1}. \tag{11}\]

We refer to these two variants as F\({}^{3}\)-Loss and F\({}^{3}\)-Error.

Additional transformations can be applied to the chosen error information. For classification tasks, one could consider only the feedback signal of the target class \(c^{*}\). We refer to this as _one-hot error information_ as it corresponds to multiplying the error information with the one-hot encoded targets. One-hot error information reduces the computational cost of the gradient approximation by reducing the matrix-vector multiplication \(B_{i}^{T}e_{t-1}\) to the multiplication of a single column of \(B_{i}\) with the scalar \(e_{t-1}[c^{*}]\). Another option is applying an additional softmax to the output \(y_{t-1}\) before computing the error information.

Instead of sample-wise error information, one could also aggregate the error over all samples in the last epoch. This would reduce the already small memory overhead to just a single value for the whole training set, which could benefit applications with severe hardware constraints. While losing the ability to differentiate between low and high error samples, this still adjusts the update step size as the error decreases throughout the training.

### How F\({}^{3}\) Solves the Weight Transport Problem

Prior work [8; 9; 10] has shown that neural networks can be trained using fixed random feedback paths. We therefore replace the layer-wise propagation of gradients (blue) in Equation (5) with a fixed feedback weight matrix \(B_{i}\) to directly pass the error signal to layer \(i\) in Equation (6). All gradient information other than the error signal, e.g., the forward weights and the derivatives of the activation functions for all downstream layers, is encoded in this feedback weight matrix. The forward weights \(W_{j}\) are thus no longer required to approximate the gradient \(\delta^{\text{F}^{3}}h_{i}\), making F\({}^{3}\) defacto weight transport free.

### How F\({}^{3}\) Solves the Update Locking Problem

As discussed in Section 3.2, avoiding update locking means that the update step in layer \(i\) cannot rely on results or gradients in downstream layers \(j>i\). Using fixed direct feedback paths already eliminates the dependence on downstream layers in the feedback path. The error signal \(\partial L/\partial y\) (green) in Equation (5) is not available either, as the last layer's output \(y\) has not yet been computed for the current epoch \(t\). Since training typically requires multiple epochs, F\({}^{3}\) instead uses delayed error information from the previous epoch \(t-1\) to approximate the current error signal in Equation (6). As discussed in Section 3.3, this makes it independent of downstream layers.

This introduces a dependency between consecutive epochs, as processing a sample requires its delayed error information from the previous epoch. However, this is generally not a problem since training epochs are typically clearly separated. Often, there is even additional auxiliary computation in between epochs, such as intermediate evaluation or check-pointing. One could also relax this requirement at the cost of a potentially higher delay of the error information by using the most
current loss available for a sample, be it from the previous or any earlier epoch. In summary, F3 can immediately update the weights in every layer without waiting for downstream layers and is thus free of update locking.

### Computational Considerations

By eliminating layer-wise error propagation, F3 generally requires fewer operations than backpropagation. The exact reduction depends on the input data and selected network architecture. For a fully-connected neural network with \(K\) linear layers of equal width \(w\), input size \(n_{0}\), and output size \(C\), the number of fused multiply-add operations to perform the forward, backward, and update step for a single training sample is

\[3w\left(n_{0}+(K-2)w+C\right)+\underbrace{\left(K-2\right)w^{2}+Cw}_{\text{ computing }\delta y_{i}} \tag{12}\]

with backpropagation and

\[3w\left(n_{0}+(K-2)\,w+C\right)+\underbrace{C(K-1)w}_{\text{computing }\delta^{y}y_{i}}. \tag{13}\]

with F3. The detailed derivation of these equations is given in Appendix A. The number of operations saved increases with deeper and wider networks. For sufficiently large widths, F3 approaches a 100% reduction for computing the \(\delta y_{i}\). This results in a 25% theoretical maximum reduction in time for the whole training step, i.e., forward and backward pass followed by an update step.

As an example, we consider a five layer network with input size \(n_{0}=100\) and output size \(C=10\). With a hidden layer width \(w=200\), F3 can save more than 90% of operations computing the \(\delta y_{i}\) and more than 20% in total throughout the whole batch. With a width of \(w=1024\), this increases to more than 98% and 24%, respectively. In experimental evaluations, we find that the practically achievable reduction in training time is closer to 15%.

## 4 Evaluation

### Methods

We evaluate F3 on different datasets and learning tasks. For image classification, we use the well-known image classification datasets MNIST [35] and CIFAR-10 [36]. For general classification, we use the Census Income dataset [37] to predict whether a person's income exceeds a threshold based on their census data. For regression, we use the SGEMM dataset [38; 39] to predict the run time of sparse matrix-matrix multiplications on GPU using different kernel parameters. The MNIST, CIFAR-10, and SGEMM datasets have been standardized.

We use F3 to train different fully-connected neural network architectures with \(1\) to \(50\) hidden layers at a width of \(500\) neurons. We use the hyperbolic tangent activation function in hidden layers and a sigmoid in the output layer for classification tasks. For classification tasks, one-hot encoding with binary cross-entropy (BCE) loss is used, while mean squared error (MSE) loss is used for regression tasks. We set a fixed learning rate determined by a grid search, as shown in Tables 2 and 3, and use the Adam optimizer [34] with default parameters.

We compare multiple variants of F3 using the different options for delayed error information described in Section 3.4. Results are shown for both the delayed loss gradient and the delayed error as feedback signal. Unless otherwise mentioned, we use the one-hot error information for classification tasks, i.e., using only the feedback coming from the target class. For regression tasks, we always use the delayed feedback signal directly without further modifications.

We compare F3 to multiple other training algorithms: backpropagation and DRTP [10] were chosen as baselines for traditional and biologically plausible training approaches, respectively. To demonstrate the benefits of biologically plausible approaches, we include the results of training only the last layer while keeping the hidden layers constant at their random initialization. We refer to this as last-layer-only (LLO) training in the following. We additionally include DFA [9], which corresponds to F3 using current instead of delayed error information, thus not solving update locking.
### Experimental Environment

All experiments were conducted on a single compute node with two Intel Xeon 8386 "Ice Lake" processors and four NVIDIA A100-40 tensor core GPUs. Models were implemented in Python 3.8.6 using the PyTorch framework [40] version 1.12.1 with CUDA version 11.6. The source code is publicly available at github.com/Helmholtz-AI-Energy/f3.

### Delayed Error Information

Figure 3 illustrates the performance of F\({}^{3}\) using the different variants of delayed error information introduced in Section 3.4. For classification, error-based updates (F\({}^{3}\)-Error, green) tend to outperform updates based on the gradient of the loss (F\({}^{3}\)-Loss, blue). This result is unexpected since F\({}^{3}\)-Loss is more similar to backpropagation than F\({}^{3}\)-Error from a theoretical point of view. When using MSE loss, e.g., for the regression task SGEMM, the error is equivalent to the loss gradient except for a scalar factor. In this case, the utilized error information affects only the step size but not the general direction of the update step. Both effects can also be observed in Figure 2.

For classification, we test additional transformations of the error information, applying either a softmax to the output or considering only the error information of the target class, which corresponds to multiplication with the one-hot encoded target. In Figure 3, these variants are highlighted with different markers. We find that both the raw error information and the one-hot version are valid options for effective training, whereas applying an additional softmax tends to decrease the resulting quality as measured by an increased test loss.

### Comparison to Other Training Algorithms

Figure 2 illustrates the behavior of different training algorithms for multiple datasets and tasks. The exact results are also given in Tables 4 and 5. As expected, backpropagation outperforms biologically plausible training on most datasets; however, F\({}^{3}\) reduces this gap in performance by a significant fraction. On MNIST, F\({}^{3}\)-Error noticeably outperforms DRTP, increasing the test accuracy from 95.8% to 97.2%, thus reducing the gap to backpropagation by 54.9%. None of the algorithms

Figure 2: Test loss and accuracy using different training approaches for a fully-connected neural network with one hidden layer consisting of \(500\) neurons. We evaluate two variants of F\({}^{3}\), using either the error or the loss gradient as delayed error information, and compare them to backpropagation, last-layer-only (LLO) training, DFA, and DRTP on the classification tasks MNIST, CIFAR-10, Census Income, and the regression task SGEMM. By using delayed error information as feedback signal, F\({}^{3}\) improves upon the performance of the similarly plausible DRTP, thus closing the gap of update locking free training to backpropagation.
solve the CIFAR-10 image classification task to an acceptable level, with even backpropagation only reaching a test accuracy of 51.2%. This is likely due to a sub-optimal network architecture. Counterintuitively, F\({}^{3}\) and DRTP slightly outperform DFA on CIFAR-10. Comparing to Frenkel et al. [10]'s results, we suspect this is caused by DFA requiring a significantly lower learning rate for CIFAR-10 than other biologically plausible approaches and might thus require a more fine-grained hyper-parameter optimization differentiating more between the different algorithms.

All tested algorithms achieve good performance of about 80% on the Census Income dataset within less than ten epochs. On the regression dataset SGEMM, F\({}^{3}\) reduces the test loss from 0.364 to 0.014, thus reducing the gap to backpropagation by 96.9%. F\({}^{3}\) performs significantly better than LLO on all datasets except Census Income, where the difference between all methods is minor. This demonstrates the effectiveness of the biologically plausible update rule for hidden layers, proving that F\({}^{3}\) is more than just a sophisticated scheme to train the output layer.

In summary, F\({}^{3}\) notably improves upon the similarly plausible DRTP for both MNIST and the SGEMM regression task. On CIFAR-10, none of the algorithms shown solved the task to a satisfying level for the tested network architecture. All algorithms solved the Census Income classification problem to a similar degree, indicating that current biologically plausible algorithms are already applicable to some tasks without significant performance declines compared to backpropagation.

### Impact of the Feedback Weight Initialization

We further explore different approaches to initializing the feedback weights \(B_{i}\) and their impact on the training. As a baseline, we use the Kaiming-He uniform distribution [41] with a gain of \(\sqrt{2}\) and the fan-in mode. To test the impact of the feedback weights' magnitude and sign, we consider two discrete uniform distributions: trinomial with the values \(\{-1,0,1\}\) and binomial with the values \(\{0,1\}\). Finally, we test repeating the identity matrix \(I\) along the larger dimension with alternating signs, referred to as \(\pm I\). Figure 4 illustrates these initialization methods. Note that the bounds of the Kaiming-He distribution can become very small for large matrices.

Figure 5 shows the test loss for the different initialization methods for DRTP and F\({}^{3}\)-Error. With both training algorithms, we observe a small but notable improvement using trinomial initialization over Kaiming uniform, indicating that continuous magnitudes are not necessary for effective learning. In contrast to trinomial initialization, binomial initialization results in a massive increase in loss. This is a clear indication that the sign of the values plays a significant role and is necessary for successfully communicating the feedback signals. Repeating the identity matrix is similarly in

Figure 3: Test loss for F\({}^{3}\) using different types of delayed error information when training a fully-connected neural network with one hidden layer consisting of 500 neurons each on MNIST. The error tends to give better training results than the gradient of the loss. The one-hot transformed error signal provides similar results to the raw error information, whereas softmax offers no advantage but rather increases the resulting test loss.
effective, suggesting that the arrangement of feedback weights matters even with both positive and negative values.

These results illustrate that feed-forward training algorithms do not depend on a specific initialization of the feedback weights, but certain characteristics are necessary to promote effective training. Namely, they require a combination of different signs in the rows of the two-dimensional feedback weight matrix. The feedback weights \(B_{i}\) of layer \(i\) are used to approximate the gradient \(\delta h_{i}\) according to Equation (6). Each column in the matrix \(B_{i}\) is used to compute one value in \(\delta^{\mathrm{F}^{3}}h_{i}\) via a dot product with the feedback signal \(e_{t-1}\). The feedback signal consists of one value per model output, e.g., one value per class. For classification, a column in \(B_{i}\) thus prescribes the class mixture of the feedback for each value in \(\delta^{\mathrm{F}^{3}}h_{i}\). Only columns with more than one non-zero value allow multi-class mixtures, explaining the ineffectiveness of \(\pm I\). Furthermore, different signs help separate classes from each other, as shown by the improved performance of trinomial over binomial initialization.

### Scaling with Network Depth

Figure 6 shows the scaling behavior of the different training approaches with increasing network depth. Biologically plausible algorithms appear more robust to problems typically arising with very deep networks. While training smaller networks very effectively, we find that backpropagation fails to train networks with more than 25 layers, offering virtually no improvement over random initialization. In contrast, \(\mathrm{F}^{3}\) can train shallow networks to nearly the same degree of predictive performance as backpropagation while diminishing considerably less sharply with increasing depth. This is a result of the direct feedback pathways [9], which are independent of the network's depth and thus immune to the typical problems arising when training very deep networks with backprop

Figure 4: The different feedback weight initialization schemes. Kaiming uses the Kaiming-He uniform distribution with a gain of \(\sqrt{2}\) and the fan-in mode. Trinomial and binomial are discrete uniform distributions over the values \(\{-1,0,1\}\) and \(\{0,1\}\), respectively. \(\pm I\) repeats the identity matrix \(I\) along the larger dimension, alternating the sign.

Figure 5: Best test loss for DRTP and \(\mathrm{F}^{3}\) with error scaling using different feedback weight initialization methods, shown for a fully-connected neural network with one hidden layer of 500 neurons trained on MNIST for 100 epochs. Trinomial initialization offers a slight improvement over the baseline Kaiming initialization, while neither binomial nor \(\pm I\) allow effective training.
agation [42]. This immunity is highlighted by DFA, which is unaffected by the network depth as expected from Nokland [9]'s results.

The scaling behavior for DRTP is similar to F\({}^{3}\), i.e., the performance decreases much more slowly with network depth than when training with backpropagation. As both DRTP and F\({}^{3}\) are based on direct random feedback pathways, this result was expected. However, DRTP performs much worse for regression problems and cannot train even more shallow networks on the SGEMM task to a similar predictive performance as F\({}^{3}\). In summary, both biologically plausible algorithms can train even deep, 100-layer networks without further remedies, such as specific activation functions, yet F\({}^{3}\) offers better predictive performance throughout all tested network depths.

## 5 Conclusion

Backpropagation relies on concepts infeasible in natural learning, rendering it an effective yet biologically implausible and computationally expensive approach to training artificial neural networks. In this paper, we introduce Feed-Forward with delayed Feedback (F\({}^{3}\)), a novel algorithm for backpropagation-free training of multi-layer neural networks using a feed-forward approximation of intermediate gradients based on delayed error information. F\({}^{3}\) solves the weight transport problem by implementing direct random feedback connections and avoids update locking by using the delayed error information from the previous epoch as an additional sample-wise scaling factor. This makes it more biologically plausible than backpropagation while reducing training time and energy consumption per epoch. F\({}^{3}\) significantly improves the predictive performance compared to previous update locking free algorithms, reducing the gap to backpropagation by over 50% for classification and more than 95% for regression tasks.

By addressing the weight transport and update locking problems, F\({}^{3}\) is more biologically plausible than backpropagation. However, some implausibilities remain, such as the spiking problem, Dale's Law, and the general implausibility of supervised learning [3; 43]. A further limitation is the remaining accuracy gap to backpropagation and F\({}^{3}\) frequently requiring more epochs to reach a similar accuracy. This also limits the applicability to large-scale tasks and often outweighs the benefit of reduced compute requirements per epoch. Furthermore, there is currently no natural way to apply F\({}^{3}\) to convolutional networks while retaining the advantages of weight sharing for the feedback weights. The most promising next steps for more biological plausibility are combining F\({}^{3}\) with approaches like spiking neural networks [44] and semi- or self-supervised learning. To increase the

Figure 6: Test loss on the SGEMM dataset for different network depths and a width of 500 neurons per layer. Feed-forward-only training algorithms like F\({}^{3}\), DFA, and DRTP are more robust to very deep networks, while backpropagation fails to train networks above a depth of 25, achieving no improvement over the random initialization.
predictive performance, improving the feedback pathways and applying F\({}^{3}\) more effectively to other architectures are essential to scale F\({}^{3}\) to larger tasks.

By releasing update locking and thus the inter-layer dependencies when computing the gradients, F\({}^{3}\) can update the network's parameters during the forward pass. As a result, F\({}^{3}\) eliminates the need for the computationally expensive backward pass, thus requiring fewer operations and removing the need to buffer or recompute activations. Beyond that, it opens up promising possibilities for parallel training setups like pipeline parallelism by reducing the amount of communication and synchronization between different layers. Furthermore, F\({}^{3}\) enables on-device training on highly promising neuromorphic devices, which would simplify the training process significantly and has the potential to economize compute resources and energy tremendously.

## Acknowledgments and Disclosure of Funding

This work is supported by the Helmholtz Association Initiative and Networking Fund under the Helmholtz AI platform and the HAICORE@KIT grant.
