A Study of Biologically Plausible Neural Network: The Role and
  Interactions of Brain-Inspired Mechanisms in Continual Learning
Humans excel at continually acquiring, consolidating, and retaining
information from an ever-changing environment, whereas artificial neural
networks (ANNs) exhibit catastrophic forgetting. There are considerable
differences in the complexity of synapses, the processing of information, and
the learning mechanisms in biological neural networks and their artificial
counterparts, which may explain the mismatch in performance. We consider a
biologically plausible framework that constitutes separate populations of
exclusively excitatory and inhibitory neurons that adhere to Dale's principle,
and the excitatory pyramidal neurons are augmented with dendritic-like
structures for context-dependent processing of stimuli. We then conduct a
comprehensive study on the role and interactions of different mechanisms
inspired by the brain, including sparse non-overlapping representations,
Hebbian learning, synaptic consolidation, and replay of past activations that
accompanied the learning event. Our study suggests that the employing of
multiple complementary mechanisms in a biologically plausible architecture,
similar to the brain, may be effective in enabling continual learning in ANNs.
Fahad Sarfraz, Elahe Arani, Bahram Zonooz
# A Study of Biologically Plausible Neural Network:

###### Abstract

Humans excel at continually acquiring, consolidating, and retaining information from an ever-changing environment, whereas artificial neural networks (ANNs) exhibit catastrophic forgetting. There are considerable differences in the complexity of synapses, the processing of information, and the learning mechanisms in biological neural networks and their artificial counterparts, which may explain the mismatch in performance. We consider a biologically plausible framework that constitutes separate populations of exclusively excitatory and inhibitory neurons that adhere to Dale's principle, and the excitatory pyramidal neurons are augmented with dendritic-like structures for context-dependent processing of stimuli. We then conduct a comprehensive study on the role and interactions of different mechanisms inspired by the brain, including sparse non-overlapping representations, Hebbian learning, synaptic consolidation, and replay of past activations that accompanied the learning event. Our study suggests that the employing of multiple complementary mechanisms in a biologically plausible architecture, similar to the brain, may be effective in enabling continual learning in ANNs. 1

Footnote 1: Our code is publicly available at [https://github.com/NeurAI-Lab/Bio-ANN](https://github.com/NeurAI-Lab/Bio-ANN).

## 1 Introduction

The human brain excels at continually learning from a dynamically changing environment, whereas standard artificial neural networks (ANNs) are inherently designed for training from stationary i.i.d. data. Sequential learning of tasks in continual learning (CL) violates this strong assumption, resulting in catastrophic forgetting. Although ANNs are inspired by biological neurons (Fukushima, 1980), they omit numerous details of the design principles and learning mechanisms in the brain. These fundamental differences may account for the mismatch in performance and behavior.

Biological neural networks are characterized by considerably more complex synapses and dynamic context-dependent processing of information. In addition, individual neurons have a specific role. Each presynaptic neuron has an exclusive excitatory or inhibitory impact on its postsynaptic partners, as postulated by Dale's principle (Strata et al., 1999). Furthermore, distal dendritic segments in pyramidal neurons, which comprise the majority of excitatory cells in the neocortex, receive additional context information and enable context-dependent processing of information. This, in conjunction with inhibition, allows the network to learn task-specific patterns and avoid catastrophic forgetting (Yang et al., 2014; Iyer et al., 2022; Barron et al., 2017). Furthermore, replay of non-overlapping and sparse neural activities of previous experiences in the neocortex and hippocampus is considered to play a critical role in memory formation, consolidation, and retrieval (Walker & Stickgold, 2004; McClelland et al., 1995). To protect information from erasure,
the brain employs synaptic consolidation, in which plasticity rates are selectively reduced in proportion to strengthened synapses (Cichon and Gan, 2015).

Thus, we study the role and interactions of different mechanisms inspired by the brain in a biologically plausible framework in a CL setup. The underlying model constitutes separate populations of exclusively excitatory and inhibitory neurons in each layer, which adheres to Dale's principle (Cornford et al., 2020) and excitatory neurons (mimicking pyramidal cells) are augmented with dendrite-like structures for context-dependent processing of information (Iyer et al., 2022). Dendritic segments process an additional context signal encoding task information and subsequently modulate the feedforward activity of the excitatory neuron (Figure 1). We then systematically study the effect of controlling the overlap in representations, employing the "fire together" learning paradigm and employing experience replay and synaptic consolidation. Our study shows that:

1. An ANN architecture equipped with context-dependent processing of information by dendrites and adhering to Dale's principle can learn effectively in CL setup. Importantly, accounting for the discrepancy in the effect of weight changes in excitatory and inhibitory neurons further reduces forgetting in CL.
2. Enforcing different levels of activation sparsity in the hidden layers using k-winner-take-all activations and employing a complementary dropout mechanism (Heterogeneous Dropout) that encourages the model to use a different set of active neurons for each task can effectively control the overlap in representations, and hence reduce interference while allowing for resusability.

Figure 1: Architecture of one hidden layer in the biologically plausible framework. Each layer consists of separate populations of exclusively excitatory pyramidal cells and inhibitory neurons, which adhere to Dale’s principle. The shade indicates the strength of weights or activations, with a darker shade indicating a higher value. (a) The pyramidal cells are augmented with dendritic segments, which receive an additional context signal \(c\) and the dendritic segment whose weights are most aligned with the context vector (bottom row) is selected to modulate the output activity of the feedforward neurons for context-dependent processing of information. (b) The Hebbian update step further strengthens the association between the context and the winning dendritic segment with maximum absolute value (indicated with a darker shade in the bottom row). Finally, Heterogeneous dropout keeps the activation count of each pyramidal cell (indicated with the gray shade) and drops the neurons that were most active for the previous task (the darkest shade dropped) to enforce non-overlapping representations. The top-k remaining cells then project to the next layer (increased shade). This provides us with a more biologically plausible framework within which we can study the role of different brain-inspired mechanisms and provide insights for designing new CL methods.
3. Task similarities need to be considered when enforcing such constraints to allow for a balance between forwarding transfer and interference.
4. Mimicking the ubiquitous "fire together, wire together" learning rule in the brain through a Hebbian update step on the connection between context signal and dendritic segments, which further strengthens context gating and facilitates the formation of task-specific subnetworks.
5. We show that employing both synaptic consolidation with importance measures adjusted to take into account the discrepancy in the effect of weight changes and a replay mechanism in a context-specific manner is critical for consolidating information across different tasks, especially for challenging CL settings.

Our study suggests that employing multiple complementary mechanisms in a biologically plausible architecture, similar to what is believed to exist in the brain, can be effective in enabling CL in ANNs. To the best of our knowledge, we are the first to provide a comprehensive study of the integration of different brain-inspired mechanisms in a biologically plausible architecture in a CL setting.

## 2 Biologically Plausible Framework for CL

We provide details of the biologically plausible framework within which we conduct our study.

### Dale's Principle

Biological neural networks differ from their artificial counterparts in the complexity of synapses and the role of individual units. In particular, the majority of neurons in the brain adhere to Dale's principle, which posits that presynaptic neurons can only have an exclusive excitatory or inhibitory impact on their postsynaptic partners (Strata et al., 1999). Several studies show that the balanced dynamics (Murphy and Miller, 2009; Van Vreeswijk and Sompolinsky, 1996) of excitatory and inhibitory populations provide functional advantages, including efficient predictive coding (Boerlin et al., 2013) and pattern learning (Ingrosso and Abbott, 2019). Furthermore, inhibition is hypothesized to play a role in alleviating catastrophic forgetting (Barron et al., 2017). Standard ANNs, however, lack adherence to Dale's principle, as neurons contain both positive and negative output weights, and signs can change while learning.

Cornford et al. (2020) incorporate Dale's principle into ANNs (referred to as DANNs), which take into account the distinct connectivity patterns of excitatory and inhibitory neurons (Tremblay et al., 2016) and perform comparable to standard ANNs in the benchmark object recognition task. Each layer \(l\) comprises of a separate population of excitatory, \(h_{e}^{l}\in\mathbb{R}_{+}^{n_{e}}\), and inhibitory \(h_{i}^{l}\in\mathbb{R}_{+}^{n_{i}}\) neurons, where \(n_{e}\gg n_{i}\) and synaptic weights are strictly non-negative. Similar to biological networks, while both populations receive excitatory projections from the previous layer (\(h_{e}^{l-1}\)), only excitatory neurons project between layers, whereas inhibitory neurons inhibit the activity of excitatory units of the same layer. Cornford et al. (2020) characterized these properties by three sets of strictly positive weights: excitatory connections between layers \(W_{ee}^{l}\in\mathbb{R}_{+}^{n_{e}\times n_{e}}\), excitatory projection to inhibitory units \(W_{ie}^{l}\in\mathbb{R}_{+}^{n_{i}\times n_{e}}\), and inhibitory projections within the layer \(W_{ei}^{l}\in\mathbb{R}_{+}^{n_{e}\times n_{i}}\). The output of the excitatory units is impacted by the subtractive inhibition from the inhibitory units:

\[z^{l}=(W_{ee}^{l}-W_{ei}^{l}W_{ie}^{l})h_{e}^{l-1}+b^{l} \tag{1}\]

where \(b^{l}\in\mathbb{R}^{n_{e}}\) is the bias term. Figure 1 shows the interactions and connectivity between excitatory pyramidal cells (triangle symbol) and inhibitory neurons (denoted by \(i\)).

We aim to employ DANNs as feedforward neurons to show that they can also learn in a challenging CL setting and performance comparable to standard ANNs and provide a biologically plausible framework for further studying the role of inhibition in alleviating catastrophic forgetting.

### Active Dendrites

The brain employs specific structures and mechanisms for context-dependent processing and routing of information. The prefrontal cortex, which plays an important role in cognitive control (Miller and Cohen,
2001), receives sensory inputs as well as contextual information, which allows it to choose the most relevant sensory features for the present task to guide actions (Mante et al., 2013; Fuster, 2015; Siegel et al., 2015; Zeng et al., 2019). Of particular interest are pyramidal cells, which represent the most populous members of the excitatory family of neurons in the brain (Bekkers, 2011). The dendritic spines in pyramid cells exhibit highly non-linear integrative properties that are considered important for learning task-specific patterns (Yang et al., 2014). Pyramidal cells integrate a range of diverse inputs into multiple independent dendritic segments, allowing contextual inputs in active dendrites to modulate the response of a neuron, making it more likely to fire. However, standard ANNs are based on a point neuron model (Lapique, 1907) which is an oversimplified model of biological computations and lacks the sophisticated non-linear and context-dependent behavior of pyramidal cells.

Iyer et al. (2022) model these integrative properties of dendrites by augmenting each neuron with a set of dendritic segments. Multiple dendritic segments receive additional contextual information, which is processed using a separate set of weights. The resultant dendritic output modulates the feedforward activation, which is computed by a linearly weighted sum of the feedforward inputs. This computation results in a neuron where the magnitude of the response to a given stimulus is highly context-dependent. To enable task-specific processing of information, the prototype vector for task \(\tau\) is evaluated by taking the element-wise mean of the tasks samples, \(\mathcal{D}_{\tau}\) at the beginning of the task and then subsequently provided as context during training.

\[c_{\tau}=\frac{1}{\left|\mathcal{D}_{\tau}\right|}\sum_{x\in\mathcal{D}_{\tau }}x \tag{2}\]

During inference, the closest prototype vector to each test sample, \(x^{\prime}\), is selected as the context using the Euclidean distance among all task prototypes, \(\mathbf{C}\), stored in memory.

\[c^{\prime}=\operatorname*{arg\,min}_{c_{\tau}}\left\|\mathbf{x}^{\prime}-\mathbf{C}_{ \tau}\right\|_{2} \tag{3}\]

Following Iyer et al. (2022), we augment the excitatory units in each layer with dendritic segments (Figure 1 (a)). The feedforward activity of excitatory units is modulated by dendritic segments, which receive a context vector. Given the context vector, each dendritic segment \(j\) computes \(u_{j}^{T}c\), given weight \(u_{j}\in\mathbb{R}^{d}\) and the context vector \(c\in\mathbb{R}^{d}\) where \(d\) is the dimensions of the input image. For excitatory neurons, the dendritic segment with the highest response to the context (maximum absolute value with the sign retained) is selected to modulate output activity.

\[h_{e}^{l}=\text{\emph{k-WTA}}(z_{l}\times\sigma(u_{\kappa}^{T}c)),\qquad \text{ where }\kappa=\operatorname*{arg\,max}_{j}|u_{j}^{T}c| \tag{4}\]

where \(\sigma\) is the sigmoid function (Han & Moraga, 1995) and _k-WTA_(.) is the k-Winner-Take-All activation function (Ahmad & Scheinkman, 2019) which propagates only the top \(k\) neurons and sets the rest at zero. This provides us with a biologically plausible framework where, similar to biological networks, the feedforward neurons adhere to Dale's principle, and the excitatory neurons mimic the integrative properties of active dendrites for context-dependent processing of stimuli.

## 3 Continual Learning Settings

To study the role of different components inspired by the brain in a biologically plausible NN for CL and gauge their roles in the performance and characteristics of the model, we conduct all our experiments under uniform settings. Implementation details and experimental setup are provided in Appendix. We evaluate the models on two CL scenarios. **Domain incremental learning (Domain-IL)** refers to the CL scenario in which the classes remain the same in subsequent tasks but the input distribution changes. We consider Rot-MNIST which involves classifying the 10 digits in each task with each digit rotated by an angle between 0 and 180 degrees, and Perm-MNIST which applies a fixed random permutation to the pixels for each task. Importantly, there are different variants of Rot-MNIST with varying difficulties. We incrementally rotate the digits to a fixed degree, i.e. {0, 8,..., (N-1)*8} for task \(\{\tau_{1},\tau_{2},..,\tau_{N}\}\) which is substantially more challenging than random sampling rotations. Importantly, the Rot-MNIST dataset captures the notion of
similarity in subsequent tasks, where the similarity between two tasks is defined by the difference in their degree of rotation, whereas each task in Perm-MNIST is independent. We also consider the challenging **Class incremental learning (Class-IL)** scenario where new classes are added with each subsequent task and the agent must learn to distinguish not only amongst the classes within the current task but also across all learned tasks. Seq-MNIST divides the MNIST classification into 5 tasks with 2 classes for each task.

## 4 Empirical Evaluation

To investigate the impact of the different components inspired by the brain, we use the aforementioned biologically plausible framework and study the effect on the performance and characteristics of the model.

### Effect of Inhibitory Neurons

We first study whether feedforward networks with separate populations of excitatory and inhibitory units can work well in the CL setting. Importantly, we note that when learning a sequence of tasks with inhibitory neurons, it is beneficial to take into account the disparities in the degree to which updates to different parameters affect the layer's output distribution (Cornford et al., 2020) and hence forgetting. Specifically, since \(W^{l}_{ie}\) and \(W^{l}_{ci}\) affect the output distribution to a higher degree than \(W^{l}_{ee}\), we reduce the learning rate for these weights after the first task (see Appendix).

Table 1 shows that models with feedforward neurons adhering to Dale's principle perform on par with standard neurons and can also further mitigate forgetting in conjunction with Active Dendrites when the quality of context signal is high (in case of Permuted-MNIST). Note that this gain comes with considerably fewer parameters and context-dependent processing, as we keep the number of neurons in each layer the same, and only excitatory neurons (\(\sim\)90%) are augmented with dendritic segments. For 20 tasks, Active Dendrite with Dale's principle reduces the parameters from \(\sim\)70M to less than \(\sim\)64M parameters. We hypothesize that having separate populations within a layer enables them to play a specialized role. In particular, inhibitory neurons can selectively inhibit certain excitatory neurons based on stimulus, which can further facilitate the formation of task-specific subnetworks and complement the context-dependent processing of information by dendritic segments.

\begin{table}
\begin{tabular}{l|c c c|c c c|c} \hline \hline \multirow{2}{*}{Method} & \multicolumn{3}{c|}{Rot-MNIST} & \multicolumn{3}{c|}{Perm-MNIST} & \multirow{2}{*}{Seq-MNIST} \\ \cline{2-2} \cline{5-8}  & 5 Tasks & 10 Tasks & 20 Tasks & 5 Tasks & 10 Tasks & 20 tasks \\ \hline Joint & 98.27\({}_{\texttt{a0.07}}\) & 98.25\({}_{\texttt{a0.06}}\) & 98.17\({}_{\texttt{a0.11}}\) & 97.64\({}_{\texttt{a0.19}}\) & 97.61\({}_{\texttt{a0.15}}\) & 97.59\({}_{\texttt{a0.08}}\) & 94.53\({}_{\texttt{a0.50}}\) \\ SGD & 93.79\({}_{\texttt{a0.32}}\) & 74.13\({}_{\texttt{a0.36}}\) & 51.89\({}_{\texttt{a0.36}}\) & 77.96\({}_{\texttt{a3.84}}\) & 76.42\({}_{\texttt{a2.54}}\) & 65.18\({}_{\texttt{a1.21}}\) & 19.83\({}_{\texttt{a0.04}}\) \\ \hline Active Dendrites & 92.58\({}_{\texttt{a0.26}}\) & 71.06\({}_{\texttt{a0.32}}\) & 48.18\({}_{\texttt{a0.52}}\) & 95.71\({}_{\texttt{a0.27}}\) & 94.41\({}_{\texttt{a0.21}}\) & 91.74\({}_{\texttt{a0.34}}\) & 19.97\({}_{\texttt{a0.29}}\) \\ + Dale’s Principle & 92.28\({}_{\texttt{a0.27}}\) & 70.78\({}_{\texttt{a0.23}}\) & 48.79\({}_{\texttt{a0.27}}\) & 96.18\({}_{\texttt{a0.14}}\) & 95.20\({}_{\texttt{a0.11}}\) & 92.44\({}_{\texttt{a0.20}}\) & 19.79\({}_{\texttt{a0.08}}\) \\ \hline + Hebbian Update & 92.58\({}_{\texttt{a0.35}}\) & 71.22\({}_{\texttt{a0.86}}\) & 48.88\({}_{\texttt{a0.90}}\) & 95.90\({}_{\texttt{a0.24}}\) & 94.72\({}_{\texttt{a0.29}}\) & 92.55\({}_{\texttt{a0.47}}\) & 19.88\({}_{\texttt{a0.02}}\) \\ + HD & 93.24\({}_{\texttt{a0.25}}\) & 75.50\({}_{\texttt{a0.74}}\) & 51.11\({}_{\texttt{a0.76}}\) & 96.58\({}_{\texttt{a0.17}}\) & 95.94\({}_{\texttt{a0.24}}\) & 93.20\({}_{\texttt{a0.32}}\) & 38.45\({}_{\texttt{a0.27}}\) \\ + SC & 93.34\({}_{\texttt{a0.57}}\) & 75.94\({}_{\texttt{a1.15}}\) & 64.99\({}_{\texttt{a2.19}}\) & 96.54\({}_{\texttt{a0.29}}\) & 96.14\({}_{\texttt{a0.47}}\) & 95.43\({}_{\texttt{a0.49}}\) & 27.31\({}_{\texttt{a2.20}}\) \\ + ER & 95.21\({}_{\texttt{a0.28}}\) & 90.99\({}_{\texttt{a0.51}}\) & 83.45\({}_{\texttt{a0.44}}\) & 96.72\({}_{\texttt{a0.13}}\) & 96.04\({}_{\texttt{a0.17}}\) & 94.26\({}_{\texttt{a0.78}}\) & 86.93\({}_{\texttt{a0.82}}\) \\ + ER + CR & 96.48\({}_{\texttt{a0.35}}\) & 93.87\({}_{\texttt{a0.25}}\) & 89.39\({}_{\texttt{a0.23}}\) & 97.23\({}_{\texttt{a0.30}}\) & 96.93\({}_{\texttt{a0.31}}\) & 96.13\({}_{\texttt{a0.05}}\) & 89.60\({}_{\texttt{a0.73}}\) \\ \hline Bio-ANN & **96.82\({}_{\texttt{a0.14}}\)** & **94.64\({}_{\texttt{a0.23}}\)** & **91.32\({}_{\texttt{a0.26}}\)** & **97.33\({}_{\texttt{a0.04}}\)** & **97.07\({}_{\texttt{a0.05}}\)** & **96.51\({}_{\texttt{a0.03}}\)** & **89.90\({}_{\texttt{a0.24}}\)** \\ \hline \hline \end{tabular}
\end{table}
Table 1: Effect of each component of the biologically plausible framework on different datasets with varying number of tasks. We first show the effect of utilizing feedforward neurons adhering to Dale’s principle in conjunction with _Active Dendrites_ to form the framework within which we evaluate the individual effect of brain-inspired mechanisms (Hebbian Update, Heterogeneous Dropout (HD), Synaptic Consolidation (SC), Experience Replay (ER) and Consistency Regularization (CR)) before combining them all together to forge Bio-ANN. For all experiments, we set the percentage of active neurons at 5. We provide the average task performance and 1 std of five runs. We also demonstrate performance gains with the brain-inspired mechanisms on top of standard ANNs in Table 6 in Appendix
### Sparse Activations Facilitate the Formation of Subnetworks

Neocortical circuits are characterized by high levels of sparsity in neural activations (Barth & Poulet, 2012; Graham & Field, 2006). There is further evidence suggesting that neuronal coding of natural sensory stimuli should be sparse (Barth & Poulet, 2012; Tolhurst et al., 2009). This is in stark contrast to the dense and highly entangled connectivity in standard ANNs. Particularly for CL, sparsity provides several advantages: sparse non-overlapping representations can reduce interference between tasks (Abbasi et al., 2022; Iyer et al., 2022; Aljundi et al., 2019), can lead to the natural emergence of task-specific modules (Hadsell et al., 2020).

We study the effect of different levels of activation sparsity by varying the ratio of active neurons in k-winner-take-all (k-WTA) activations (Ahmad & Scheinkman, 2019). Each hidden layer of our model has a constant sparsity in its connections (randomly 50% of weights are set to 0 at initialization) and propagates only the top-k activations (in Figure 1, k-WTA layer). Table 2 shows that sparsity plays a critical role in enabling CL in DNNs. Sparsity in activations effectively reduces interference by reducing the overlap in representations. Interestingly, the stark difference in the effect of different levels of sparse activations on Rot-MNIST and Perm-MNIST highlights the importance of considering task similarity in the design of CL methods. As the tasks in Perm-MNIST are independent of each other, having fewer active neurons (5%) enables the network to learn non-overlapping representations for each task, while the high task similarity in Rot-MNIST can benefit from overlapping representations, which allows for the reusability of features across the tasks. The number of tasks the agent has to learn also has an effect on the optimal sparsity level. In Appendix, we show that having different levels of sparsity in different layers can further improve performance. As the earlier layers learn general features, having a higher ratio of active neurons can enable higher reusability and forward transfer. For the later layers, a smaller ratio of active neurons can reduce interference between task-specific features.

### Heterogeneous Dropout for Non-overlapping Activations and Subnetworks

Information in the brain is encoded by the strong activation of a relatively small set of neurons that form a sparse coding. A different subset of neurons is utilized to represent different types of stimuli Graham

\begin{table}
\begin{tabular}{r|c c c c|c c c c} \hline \multirow{2}{*}{\#Tasks} & \multicolumn{5}{c|}{Rot-MNIST} & \multicolumn{5}{c}{Perm-MNIST} \\ \cline{2-9}  & 0.05 & 0.10 & 0.20 & 0.50 & 0.05 & 0.10 & 0.20 & 0.50 \\ \hline
5 & \(92.28_{\pm 0.27}\) & \(92.26_{\pm 0.31}\) & \(\mathbf{92.79}_{\pm 0.44}\) & \(92.26_{\pm 0.65}\) & \(95.77_{\pm 0.33}\) & \(\mathbf{96.32}_{\pm 0.20}\) & \(90.29_{\pm 0.67}\) & \(74.51_{\pm 13.55}\) \\
10 & \(70.78_{\pm 0.23}\) & \(71.95_{\pm 1.54}\) & \(\mathbf{73.32}_{\pm 0.69}\) & \(71.61_{\pm 0.76}\) & \(\mathbf{95.06}_{\pm 0.29}\) & \(93.45_{\pm 0.92}\) & \(72.68_{\pm 12.83}\) & \(41.33_{\pm 0.72}\) \\
20 & \(\mathbf{48.79}_{\pm 0.27}\) & \(47.96_{\pm 1.84}\) & \(48.65_{\pm 0.91}\) & \(47.71_{\pm 0.91}\) & \(\mathbf{92.40}_{\pm 0.38}\) & \(84.28_{\pm 1.33}\) & \(63.84_{\pm 3.45}\) & \(20.80_{\pm 0.90}\) \\ \hline \end{tabular}
\end{table}
Table 2: Effect of different levels of sparsity in activations on the performance of the model. Columns show the ratio of active neurons (\(k\) in k-WTA activation), and rows provide the number of tasks.

Figure 2: Total activation counts for the test set of each task (y-axis) for a random set of 25 units in the second hidden layer of the model. Heterogeneous dropout reduces the overlap in activations and facilitates the formation of task-specific subnetworks.
& Field (2006). Furthermore, there is evidence of non-overlapping representations in the brain. To mimic this, we employ Heterogeneous dropout (Abbasi et al., 2022) which in conjunction with context-dependent processing of information, can effectively reduce the overlap of representations, leading to less interference between tasks and, thereby, less forgetting. During training, we track the frequency of activations for each neuron in a layer for a given task, and in the subsequent tasks, the probability of a neuron being dropped is inversely proportional to its activation counts. This encourages the model to learn the new task using neurons that have been less active for previous tasks. Figure 1 shows that neurons that have been more active (darker shade) are more likely to be dropped before k-WTA is applied. Specifically, let \([a_{t}^{l}]_{j}\) denote the activation counter of the neuron \(j\) in the layer \(l\) after learning \(t\) tasks. For the learning task \(t\) + 1, the probability that this neuron is retained is given by:

\[[p_{t+1}^{l}]_{j}=exp(\frac{-[a_{t}^{l}]_{j}}{\max_{j}{[a_{t}^{l}]_{j}}\rho)} \tag{5}\]

where \(\rho\) controls the strength of enforcement of non-overlapping representations, with larger values leading to less overlap. This provides us with an efficient mechanism for controlling the degree of overlap between the representations of different tasks and, hence, the degree of forward transfer and interference based on the task similarities.

Table 3 shows that employing Heterogeneous dropout can further improve the performance of the model. We also analyze the effect of the \(\rho\) parameter on the activation counts and the overlap in the representations. Figure 2 shows that Heterogeneous dropout can facilitate the formation of task-specific subnetworks and Figure 3 shows the symmetric KL-divergence between the distribution of activation counts on the test set of Task 1 and Task 2 on the model trained with different \(\rho\) values on Perm-MNIST with two tasks. As we increase the \(\rho\) parameter, the activations in each layer become increasingly dissimilar. Heterogeneous dropout provides a simple mechanism for balancing the reusability and interference of features depending on the similarity of tasks.

### Layerwise Heterogeneous Dropout and Task Similarity

For an effective CL agent, it is important to maintain a balance between forward transfer and interference across tasks. As the earlier layers learn general features, a higher portion of the features can be reused to learn the new task, which can facilitate forward transfer, whereas the later layers learn more task-specific features, which can cause interference. Heterogeneous dropout provides us with an efficient mechanism for controlling the degree of overlap between the activations, and hence the features of each layer. Here, we investigate whether having different levels of sparsity (controlled with the \(\rho\) parameter) in different layers can further improve performance. As the earlier layers learn general features, having higher overlap (smaller \(\rho\)) between the set of active neurons can enable higher reusability and forward transfer. For the later layers, a lesser overlap between the activations (higher \(\rho\)) can reduce interference between task-specific features.

To study the effect of Heterogeneous dropout in relation to task similarity, we vary the incremental rotation, \(\theta_{inc}\), in each subsequent task for Rot-MNIST setting with 5 tasks. The rotation of task \(\tau\) is given by

\begin{table}
\begin{tabular}{l|c|c|c c c c c} \hline \multirow{2}{*}{Dataset} & \multirow{2}{*}{\# Tasks} & \multirow{2}{*}{w/o} & \multicolumn{5}{c}{Dropout parameter (\(\rho\))} \\ \cline{3-8}  & & & Dropout & 0.1 & 0.3 & 0.5 & 0.7 & 1.0 \\ \hline \multirow{3}{*}{Rot-MNIST} & 5 & 92.28\({}_{\pm 0.20}\) & 91.79\({}_{\pm 0.53}\) & 92.53\({}_{\pm 0.11}\) & 92.74\({}_{\pm 0.38}\) & 93.19\({}_{\pm 0.32}\) & **93.42\({}_{\pm 0.25}\)** \\  & 10 & 70.78\({}_{\pm 0.23}\) & 71.53\({}_{\pm 1.07}\) & 72.38\({}_{\pm 1.44}\) & 73.63\({}_{\pm 1.00}\) & 74.20\({}_{\pm 0.78}\) & **75.50\({}_{\pm 0.74}\)** \\  & 20 & 48.79\({}_{\pm 0.27}\) & 48.57\({}_{\pm 0.90}\) & 48.91\({}_{\pm 0.65}\) & 49.84\({}_{\pm 0.59}\) & 51.03\({}_{\pm 0.31}\) & **51.11\({}_{\pm 0.76}\)** \\ \hline \multirow{3}{*}{Perm-MNIST} & 5 & 95.77\({}_{\pm 0.33}\) & 95.70\({}_{\pm 0.29}\) & 95.97\({}_{\pm 0.44}\) & 96.40\({}_{\pm 0.28}\) & **96.58\({}_{\pm 0.17}\)** & 96.48\({}_{\pm 0.26}\) \\  & 10 & 95.06\({}_{\pm 0.29}\) & 95.23\({}_{\pm 0.04}\) & 95.65\({}_{\pm 0.20}\) & 95.54\({}_{\pm 0.26}\) & 95.74\({}_{\pm 0.22}\) & **95.94\({}_{\pm 0.24}\)** \\ \cline{1-1}  & 20 & 92.40\({}_{\pm 0.38}\) & 92.83\({}_{\pm 0.42}\) & **93.20\({}_{\pm 0.32}\)** & 92.82\({}_{\pm 0.06}\) & 93.09\({}_{\pm 0.47}\) & 91.77\({}_{\pm 0.30}\) \\ \hline \end{tabular}
\end{table}
Table 3: Effect of Heterogeneous dropout with increasing \(\rho\) values on different datasets with a varying number of tasks.
\(1)\theta_{inc}\). Table 4 shows the performance of the model for different layerwise \(\rho\) values. Generally, heterogeneous dropout consistently improves the performance of the model, especially when the task similarity is low. For \(\theta_{inc}=32\), it provides \(\sim\)25% improvement. As task similarity decreases (\(\theta_{inc}\) increases), higher values of \(\rho\) are more effective. Furthermore, we see that having different \(\rho\) values for each layer can provide additional gains in performance.

### Hebbian Learning Strengthens Context Gating

For a biologically plausible ANN, it is important to incorporate not only the design elements of biological neurons but also the learning mechanisms it employs. Lifetime plasticity in the brain generally follows the Hebbian principle: a neuron that consistently contributes to the firing of another neuron will build a stronger connection to that neuron (Hebb, 2005).

Therefore, we follow the approach in Flesch et al. (2023) to complement error-based learning with the Hebbian update to strengthen the connections between contextual information and dendritic segments (Figure 1(b)).
Each supervised parameter update with backpropagation is followed by a Hebbian update step on the dendritic segments to strengthen the connections between the context input and the corresponding dendritic segment, which is activated. To constrain the parameters, we use Oja's rule, which adds weight decay to Hebbian learning (Oja, 1982),

\[u_{\kappa}\gets u_{\kappa}+\eta_{h}d(c-du_{\kappa}) \tag{6}\]

where \(\eta_{h}\) is the learning rate, \(\kappa\) is the index of the winning dendrite with weight \(u_{\kappa}\) and the modulating signal \(d=u_{\kappa}^{T}c\) for the context signal \(c\). Figure 4 shows that the Hebbian update step increases the magnitude of the modulating signal from the dendrites on the feedforward activity, which can further strengthen context-dependent gating and facilitate the formation of task-specific subnetworks. Table 1 shows that this can consequently lead to improvement in results. Though the gains are not considerable in these settings, Table 5 shows the gains with Hebbian Learning are more pronounced in challenging CL settings with higher task and dataset complexity.

### Synaptic Consolidation Further Mitigates Forgetting

In addition to their integrative properties, dendrites also play a key role in retaining information and providing protection against erasure (Cichon and Gan, 2015; Yang et al., 2009). New spines that are formed on different sets of dendritic branches in response to learning different tasks are protected from being eliminated through mediation of synaptic plasticity and structural changes that persist when learning a new task (Yang et al., 2009).

We employ synaptic consolidation by incorporating _Synaptic Intelligence_(Zenke et al., 2017) (details in Appendix) which maintains an importance estimate of each synapse in an online manner during training and subsequently reduces the plasticity of synapses which are considered important for learned tasks. In particular, we adjust the importance estimate to account for the disparity in the degree to which updates to different parameters affect the output of the layer due to the inhibitory interneuron architecture in the DANN layers (Cornford et al., 2020). The importance estimates of the excitatory connections to the inhibitory units and the intra-layer inhibitory connections are upscaled to further penalize changes to these weights. Table 1 shows that employing Synaptic Intelligence (+SC) in this manner further mitigates forgetting. Particularly for Rot-MNIST with 20 tasks, it provides a considerable performance improvement.

### Experience Replay is Essential for Enabling CL in Challenging Scenarios

Replay of past neural activation patterns in the brain is considered to play a critical role in memory formation, consolidation, and retrieval (Walker and Stickgold, 2004; McClelland et al., 1995). The replay mechanism in the hippocampus (Kumaran et al., 2016) has inspired a series of rehearsal-based approaches (Li and Hoiem, 2017; Chaudhry et al., 2019; Lopez-Paz and Ranzato, 2017; Arani et al., 2022) that have been proven to be effective in challenging continual learning scenarios (Farquhar and Gal, 2018; Hadsell et al., 2020). Therefore, to replay samples from previous tasks, we utilize a small episodic memory buffer that is maintained through _Reservoir sampling_(Vitter, 1985). It attempts to approximately match the distribution of the incoming stream by assigning equal probabilities to each new sample to be represented in the buffer. During training, samples from the current task, \((x_{b},y_{b})\sim\mathcal{D}_{\tau}\), are interleaved with memory buffer samples, \((x_{m},y_{m})\sim\mathcal{M}\) to approximate the joint distribution of tasks seen so far. Furthermore, to mimic the replay of the activation patterns that accompanied the learning event in the brain, we also save the output logits, \(z_{m}\), across the training trajectory and enforce consistency loss when replaying samples from episodic memory. Concretely, the loss is given by:

\[\mathcal{L}=\mathcal{L}_{cls}(f(x_{b};\theta),y_{b})+\alpha\mathcal{L}_{cls}(f (x_{m};\theta),y_{m})+\beta(f(x_{m};\theta)-z_{m})^{2} \tag{7}\]

where \(f(.;\theta)\) is the model parameterized by \(\theta\), \(\mathcal{L}_{cls}\) is the standard cross-entropy loss, and \(\alpha\) and \(\beta\) controls the strength of interleaved training and the consistency constraint, respectively.

Table 1 shows that experience replay (+ER) complements context-dependent information processing and enables the model to learn the joint distribution well in varying challenging settings. In particular, the failure of the model to avoid forgetting in the Class-IL setting (Seq-MNIST) without experience replay
suggests that context-dependent processing of information alone does not suffice, and experience replay might be essential. Adding consistency regularization (+CR) further improves performance as the model receives additional relational information about the structural similarity of classes, which facilitates the consolidation of information.

### Combining the individual components

Having shown the individual effect of each of the brain-inspired components in the biologically plausible framework, here we look at their combined effect. The resultant model is referred to as Bio-ANN. Table 1 shows that the different components complement each other and consistently improve the performance of the model. Our empirical results suggest that employing multiple complementary components and learning mechanisms, similar to the brain, may be an effective approach to enable continual learning in ANNs.

### Additional Results on Challenging CL settings

To further evaluate the versatility of the biological components on more challenging settings, we conducted experiments on Fashion-MNIST and grayscale version of CIFAR10. We considered both the Class-IL and Domain-IL settings. Seq-FMNIST, Seq-MNIST, and Seq-GCIFAR10 divide the classification into 5 tasks with 2 classes each, while Rot-FMNIST involves 20 tasks with each task involving classifying the 10 classes in each task with the samples rotated by increments of 8 degrees.

For brevity, we refer to Active Dendrites + Dale's principle as ActiveDANN. To show the effect of different components better (ActiveDANN without ER fails in the class-IL setting), we consider ActiveDann + ER as the baseline upon which we add the other components. Empirical results in Table 5 show that the findings on MNIST settings also translate to more challenging datasets and each component leads to performance improvement. In particular, we observe that for more complex datasets, Hebbian learning provides a significant performance improvement. The preliminary results suggest that the effect of biological mechanisms and architecture might be more pronounced on more complex datasets and CL settings.

## 5 Discussion

Continual learning is a hallmark of intelligence, and the human brain constitutes the most efficient learning agent capable of CL. Therefore, incorporating the different components and mechanisms employed in the brain and studying their interactions can provide valuable insights for the design of ANNs suitable for CL. While there are several studies that are inspired by the brain, they focus primarily on one aspect. Since the brain employs all these different components in tandem, it stands to reason that their interactions, or complementary nature, are what enables effective learning instead of one component alone. Furthermore, the underlying framework within which these components are employed and the learning mechanisms might

\begin{table}
\begin{tabular}{l|c|c c c} \hline \hline \multirow{2}{*}{Method} & Domain-IL & \multicolumn{3}{c}{Class-IL} \\ \cline{2-5}  & Rot-FMNIST & Seq-FMNIST & Seq-MNIST & Seq-GCIFAR10 \\ \hline Joint & 98.15\({}_{\texttt{a}0.09}\) & 94.33\({}_{\texttt{a}0.51}\) & 94.53\({}_{\texttt{a}0.53}\) & 36.10\({}_{\texttt{a}0.86}\) \\ SGD & 51.89\({}_{\texttt{a}0.27}\) & 19.83\({}_{\texttt{a}0.04}\) & 19.83\({}_{\texttt{a}0.04}\) & 14.86\({}_{\texttt{a}1.06}\) \\ \hline ActiveDANN & 49.42\({}_{\texttt{a}0.83}\) & 21.46\({}_{\texttt{a}1.95}\) & 19.97\({}_{\texttt{a}0.29}\) & 16.12\({}_{\texttt{a}0.31}\) \\ ActiveDANN + ER & 80.99\({}_{\texttt{a}0.53}\) & 77.56\({}_{\texttt{a}0.27}\) & 86.88\({}_{\texttt{a}0.83}\) & 28.74\({}_{\texttt{a}0.42}\) \\ \hline + Hebbian Update & 82.16\({}_{\texttt{a}0.26}\) & 78.02\({}_{\texttt{a}0.38}\) & 88.39\({}_{\texttt{a}0.78}\) & 30.12\({}_{\texttt{a}0.49}\) \\ + SC & 82.55\({}_{\texttt{a}0.37}\) & 78.05\({}_{\texttt{a}0.61}\) & 88.79\({}_{\texttt{a}0.49}\) & 30.45\({}_{\texttt{a}0.61}\) \\ + HD & 83.97\({}_{\texttt{a}0.46}\) & 78.74\({}_{\texttt{a}0.38}\) & 89.60\({}_{\texttt{a}0.73}\) & 30.29\({}_{\texttt{a}0.51}\) \\ \hline Bio-ANN & **89.22\({}_{\texttt{a}0.21}\)** & **79.28\({}_{\texttt{a}0.42}\)** & **89.90\({}_{\texttt{a}0.24}\)** & **30.95\({}_{\texttt{a}0.17}\)** \\ \hline \hline \end{tabular}
\end{table}
Table 5: Effect of each component of the biologically plausible framework on different Domain-IL and Class-IL settings. For all experiments, we use a memory budget of 500 samples. We provide the average task performance and 1 std of 5 runs.
also be critical. The effort to close the gap between current AI and human intelligence could benefit from our enhanced understanding of the brain and incorporating similar mechanisms in ANNs. This is the fundamental question we aimed to study and bring to the attention of the research community at large.

We conducted a study on the effect of different brain-inspired mechanisms under a biologically plausible framework in the CL setting. The underlying model incorporates several key components of the design principles and learning mechanisms in the brain: each layer constitutes separate populations of exclusively excitatory and inhibitory units, which adheres to Dale's principle, and the excitatory pyramidal neurons are augmented with dendritic segments for context-dependent processing of information. We first showed that equipped with the integrative properties of dendrites, the feedforward network adhering to Dale's principle perform on par with standard ANNs, and provide considerable performance gains in cases where the the quality of context signal in high. Then we studied the individual role of different components. We showed that controlling the sparsity in activations using k-WTA activations and Heterogeneous dropout mechanism that encourages the model to use a different set of neurons for each task is an effective approach for maintaining a balance between reusability of features and interference, which is critical for enabling CL. We further showed that complementing error-based learning with the "fire together, wire together" learning paradigm can further strengthen the association between the context signal and dendritic segments that process them and facilitate context-dependent gating. To further mitigate forgetting, we incorporated synaptic consolidation in conjunction with experience replay and showed their effectiveness in challenging CL settings. Finally, the combined effect of these components suggests that similar to the brain, employing multiple complementary mechanisms in a biologically plausible architecture is an effective approach to enable CL in ANN. It also provides a framework for further study of the role of inhibition in mitigating catastrophic forgetting.

However, there are several limitations and potential avenues for future research. In particular, as dendritic segments provide an effective mechanism for studying the effect of encoding different information in the context signal, they provide an interesting research avenue as to what information is useful for the sequential learning of tasks and the effect of different context signals. Neuroscience studies suggest that multiple brain regions are involved in processing a stimulus and, while there is evidence that active dendritic segments receive contextual information that is different from the input received by the proximal segments, it is unclear what information is encoded in the contextual information and how it is extracted. Here, we used the context signal as in (Iyer et al., 2022), which aims to encode the identity of the task by taking the average input image of all the samples in the task. Although this approach empirically works well in the Perm-MNIST setting, it is important to consider its utility and limitations under different CL settings. Given the specific design of Perm-MNIST, binary-centered digits, and the independent nature of the permutations in each task, the average input image can provide a good approximation of the applied permutation, and hence efficiently encode the task identity. However, this is not straightforward for Rot-MNIST where the task similarities are higher and even more challenging for natural images, where averaging the input image does not provide a meaningful signal. More importantly, it does not seem biologically plausible to encode task information alone as the context signal and ignore the similarity of classes occurring in different tasks. For instance, it seems more reasonable to process slight rotations of the same digits similarly (as in Rot-MNIST) rather than processing them through different subnetworks. This argument is supported by the performance degradation on the Rotated-MNIST setting with active dendrites over standard ANNs. Ideally, we would like the context signal for different rotations of a digit to be highly similar. It is, however, quite challenging to design context signals that can capture a wide range of complexities in the sequential learning of tasks. Furthermore, instead of hand engineering the context signal to bias learning towards certain types of tasks, an effective approach for learning the context signal in an end-to-end training framework is an interesting direction for future search. Another limitation of our framework would be the use of backpropagation which is widely criticized for being biologically implausible as it requires symmetric weight matrices in the feedforward and feedback pathways Xiao et al. and requires each neuron to have access to and contribute towards optimizing a single global objective function. The synapses in the brain, on the other hand, are unidirectional, and feedforward and feedback connections are physically distinct. The brain is also believed to perform localized learning. An interesting avenue for future research could be to extend our framework with more biologically plausible learning rules and study how the learning rule affects the interactions between the biologically plausible
mechanisms. Future studies can also study the role of more plausible organization of neurons instead of fully connected, for instance, incorporation of population coding.

In general, our study presents a compelling case for incorporating the design principles and learning machinery of the brain into ANNs and provides credence to the argument that distilling the details of the learning machinery of the brain can bring us closer to human intelligence (Hassabis et al., 2017; Hayes et al., 2021). Furthermore, deep learning is increasingly being used in neuroscience research to model and analyze brain data (Richards et al., 2019). The utility of the model for such research depends on two critical aspects: the performance of the model and how close the architecture is to the brain (Cornford et al., 2020; Schrimpf et al., 2020). The biologically plausible framework in our study incorporates several design components and learning mechanisms of the brain and performs well in a (continual learning) task that is closer to human learning. Therefore, we believe that this work may also be useful for the neuroscience community in evaluating and guiding computational neuroscience. Studying the properties of ANNs with higher similarity to the brain may provide insight into the mechanisms of brain functions. We believe that the fields of artificial intelligence and neuroscience are intricately intertwined, and progress in one can drive the other as well.
