Meta-Learning in Spiking Neural Networks with Reward-Modulated STDP
The human brain constantly learns and rapidly adapts to new situations by
integrating acquired knowledge and experiences into memory. Developing this
capability in machine learning models is considered an important goal of AI
research since deep neural networks perform poorly when there is limited data
or when they need to adapt quickly to new unseen tasks. Meta-learning models
are proposed to facilitate quick learning in low-data regimes by employing
absorbed information from the past. Although some models have recently been
introduced that reached high-performance levels, they are not biologically
plausible. We have proposed a bio-plausible meta-learning model inspired by the
hippocampus and the prefrontal cortex using spiking neural networks with a
reward-based learning system. Our proposed model includes a memory designed to
prevent catastrophic forgetting, a phenomenon that occurs when meta-learning
models forget what they have learned as soon as the new task begins. Also, our
new model can easily be applied to spike-based neuromorphic devices and enables
fast learning in neuromorphic hardware. The final analysis will discuss the
implications and predictions of the model for solving few-shot classification
tasks. In solving these tasks, our model has demonstrated the ability to
compete with the existing state-of-the-art meta-learning techniques.
Arsham Gholamzadeh Khoee, Alireza Javaheri, Saeed Reza Kheradpisheh, Mohammad Ganjtabesh
# Meta-Learning in Spiking Neural Networks with Reward-Modulated STDP

###### Abstract

The human brain constantly learns and rapidly adapts to new situations by integrating acquired knowledge and experiences into memory. Developing this capability in machine learning models is considered an important goal of AI research since deep neural networks perform poorly when there is limited data or when they need to adapt quickly to new unseen tasks. Meta-learning models are proposed to facilitate quick learning in low-data regimes by employing absorbed information from the past. Although some models have recently been introduced that reached high-performance levels, they are not biologically plausible. We have proposed a bio-plausible meta-learning model inspired by the hippocampus and the prefrontal cortex using spiking neural networks with a reward-based learning system. Our proposed model includes a memory designed to prevent catastrophic forgetting, a phenomenon that occurs when meta-learning models forget what they have learned as soon as the new task begins. Also, our new model can easily be applied to spike-based neuromorphic devices and enables fast learning in neuromorphic hardware. The final analysis will discuss the implications and predictions of the model for solving few-shot classification tasks. In solving these tasks, our model has demonstrated the ability to compete with the existing state-of-the-art meta-learning techniques.

 Meta-Learning, Few-Shot Learning, Learning to Learn, Spiking Neurons, STDP, Reward-Modulated STDP, PFC, Hippocamp.

## I Introduction

Today's machine learning and deep learning models excel at solving single tasks; however, they struggle when training data is insufficient or they have to adapt to changing tasks [1]. Meta-learning is a desirable solution to remedy this problem by leveraging previous experiences [2]. A meta-learning model should be able to generalize a learning strategy to different tasks derived from a common distribution [3]. In contrast, traditional machine learning models are limited in adapting to a single task by finding patterns that generalize across data points.

Recently, several meta-learning methods have been proposed and demonstrated to be effective. The main objective is to provide a model for efficient learning of new tasks to help us get closer to Artificial General Intelligence (AGI). The primary idea behind meta-learning is to employ the acquired knowledge in solving previous tasks to generalize the learning to new tasks. In general, meta-learning algorithms can be classified into three types: metric-based, optimization-based, and model-based.

We can point to Matching Networks [4], Prototypical Networks [5], and Relation Networks [6] as notable methods belonging to metric-based algorithms. They can allow unseen tasks to be efficiently learned while not experiencing catastrophic forgetting. However, they cannot be directly applied to other methods associated with reinforcement learning. Optimization-based algorithms are prevalent since they are model agnostic besides being task agnostic [7]. However, they are computationally inefficient since they are required to compute second-order gradients. MAML [7], Meta-SGD [8], and AVID [9] are practical methods in this category. We can refer to MANN [10] and SNAIL [11] as noteworthy examples of model-based techniques. Most of these models include memory modules such as Recurrent Neural Networks (RNNs) or Long Short-Term Memory (LSTM) units, allowing them to store their experiences. Nevertheless, using these modules would increase the number of model parameters.

Despite being influenced by human brain behavior, most successful meta-learning approaches lack bio-plausibility. The human brain can quickly learn new skills by utilizing prior knowledge and experiences, which are mostly encoded in the temporal lobe of the human cortex [12]. The temporal lobe is the second largest lobe, responsible for processing auditory information, language comprehension, and the formation of long-term memories [13]. The hippocampus is one of the main regions in the medial temporal lobe, which plays a crucial role in episodic memory [14]. As a type of declarative memory, episodic memory refers to the human ability to recall specific events and experiences from the past.

Wang et al. [15] have simulated the Prefrontal Cortex (PFC)
behavior for meta-reinforcement learning using the assumption that the PFC also encodes the recent history of actions and rewards besides representing the expected values of actions, objects, and states. It has been conceived that PFC, along with the thalamic nuclei and basal ganglia, form a recurrent neural network [16, 17]. Through its inputs, this network is provided with information about the performed actions and received rewards. On the output side, actions and estimation of the state value are generated by the network [18]. By incorporating PFC with dopamine (DA), Wang et al. have created a model that includes two full-fledged reinforcement learning systems, one based on activity representations and the other based on synaptic learning to develop a unique meta-learning algorithm [15]. The model was further enhanced by Ritter et al. [19] with embedding the episodic recall mechanism to overcome catastrophic forgetting using differentiable-neural-dictionary (DND) [20, 21] as long-term memory.

Although Wang et al. [15] and Ritter et al. [19] have proposed models inspired by biological procedures, they use LSTM, making them to be less biologically plausible. However, using Spiking Neural Networks (SNNs) can provide a better solution to design biologically plausible models, as SNNs are more similar to the behavior of the human brain neural network [22]. In a sense, SNNs can be viewed as a particular case of RNNs with internal states similar to the LSTM [23]. Also, SNNs are more energy-efficient than conventional artificial neural networks and can be easily incorporated into neuromorphic electronic systems [24]. Neuromorphic devices aim to capture the brain's fundamental properties to enable low-power, versatile, and fast processing of information [25, 26, 27]. Stewart and Neftci [28] have utilized SNNs for meta-learning using surrogate gradient descent to take advantage of the MAML paradigm, which has achieved remarkable results. Subramoney et al. [29] have investigated how synaptic plasticity and network dynamics contribute to fast learning in SNNs by allowing synaptic weights to store prior information via backpropagation through time (BPTT) to mimic the brain's ability to learn from very few samples. Scherr et al. [30] have proposed a model for one-shot learning using SNNs by simulating the ventral tegmental area (VTA) using an additional model to produce some learning signals as dopamine that enables fast learning via local synaptic plasticity in their network. However, all these models rely on gradient computations and backpropagation algorithms, which undermines the bio-plausibility of the model.

In this paper, we have utilized SNNs to simulate memory and decision-making systems using Spike-Timing-Dependent Plasticity (STDP) [31, 32] and reward-modulated STDP (R-STDP) [33, 34] for synaptic plasticity. Using these methods, we propose a novel bio-plausible meta-learner based on the role of the hippocampus and the PFC to effectively solve new unseen tasks and prevent catastrophic forgetting by encoding past experiences into a memory layer. The required information for solving new tasks that share a similar structure with previously learned tasks can then be retrieved from this memory layer.

## II Materials and Methods

We consider a few-shot learning task \(\mathcal{T}\) to be sampled from an unknown task distribution \(p(\mathcal{T})\). Each task is characterized by a distinct dataset \(\mathcal{D}=\{(x_{i},y_{i})\,|\,x_{i}\in X,\,y_{i}\in Y\}_{i=1}^{n}\), consisting of \(n\) independent and identically distributed input-output pairs \((x_{i},y_{i})\). The corresponding dataset of task \(\mathcal{T}\) is divided into two sets: a support set \(\mathcal{D}^{\mathcal{S}}\) and a query set \(\mathcal{D}^{\mathcal{Q}}\), such that \(\mathcal{D}=\mathcal{D}^{\mathcal{S}}\cup\mathcal{D}^{\mathcal{Q}}\).

Here, we present a model to solve unseen tasks sampled from \(p(\mathcal{T})\). The proposed model relies on the interactions between the hippocampus, PFC, and VTA, as shown in Figure 1. Together, they create a memory system where the hippocampus memorizes long-term events for the future while the PFC receives signals from the hippocampus and other regions to make decisions. The PFC also includes short-term memory, or working memory, which can store recent events and accelerate decision-making [35].

Also, the ventral tegmental area (VTA) plays a crucial role in the brain's reward system. It contains dopaminergic neurons that release dopamine, which interacts with the prefrontal cortex (PFC) and hippocampus [35].

This work provides a new perspective on reward-based learning computations by considering bio-plausibility and simplicity as fundamental principles. With this in mind, we have utilized SNNs in order to enhance computational power in designing a highly dynamic meta-learner. Figure 2 depicts the overall architecture of the model, composed of three major components: the convolutional layer, the memory layer, and the decision layer. In this model, input data arrives at the

Fig. 1: The interactions between PFC, hippocampus, and VTA in the brain.
convolutional layer, where it is processed and its features are extracted. The resulting features are then encoded in the memory layer, which serves as an episodic memory system for the model. Finally, the decision layer classifies the input data using the retrieved information from the memory layer, which provides the necessary context for accurate classification.

Meta-learning includes two phases: the meta-training phase and the meta-testing phase. The meta-training phase involves performing both the memory and the decision adaptation stages. Accordingly, the memory layer is updated with the support set, whereas the decision layer is updated with the query set. During the meta-testing phase, the model is given a support set of few data points for each unseen task in order to update the memory layer through the memory adaptation stage. The updated memory layer is then utilized in conjunction with the decision layer to predict the labels of the corresponding query set.

### _Convolutional Layer_

Humans can recognize surrounding objects easily [36] despite changes in object position, size, pose, illumination conditions, and background context [37, 38]. Apparently, this invariant recognition is handled through a hierarchical process in the ventral pathway. It begins with the V1 area, which extracts simple features like bars and edges [39]. It continues through intermediate areas such as V2 and V4, which are responsive to more complex features [40], and finally, in the inferior temporal cortex (IT), in which the neurons selectively respond to objects or parts of the objects [41].

In the same way as the ventral path, the convolutional layer extracts the features of the input image and feeds them to the next layer, i.e., the memory layer. The convolutional layer is composed of leaky integrate-and-fire (LIF) neurons, which receive Poisson-encoded images as inputs.

The LIF neuron model is a mathematical model used to describe the behavior of biological neurons, where the membrane potential depends on incoming excitatory and inhibitory inputs and passive ion leakage across the membrane. When the membrane potential reaches a certain threshold \(u_{\theta}\), the neuron fires an action potential or spike, and the membrane potential is then reset to a lower value. The LIF model assumes that the membrane potential can be described by a single variable, \(u(t)\), which represents the voltage across the neuron's membrane at time \(t\). The equation governing the dynamics of \(u(t)\) is as follows [42]:

\[\tau.\frac{du}{dt}=-(u-u_{rest})+R.I(t), \tag{1}\]

where \(u_{rest}\) denotes the voltage across the neuron's membrane in the rest mode, \(R\) is the resistance of the neuron's membrane, and \(I(t)\) represents its input current at time \(t\). Also, \(\tau\) is a time constant that corresponds to the leakage rate.

The training process of the convolutional layer is unsupervised and is performed through a particular form of synaptic plasticity known as spike-timing-dependent plasticity (STDP), which has also been observed to occur in the human visual cortex [43]. In general, STDP potentiates the afferent connections involved in making a neuron fire while depressing the others [44, 45].

We have utilized the standard STDP, where the weights are updated as follows:

\[\Delta w_{ij}=\left\{\begin{aligned} & A_{+}\exp\frac{-\Delta t}{\tau_{+}}, \ \ if\ \ \Delta t\geq 0,\\ &-A_{-}\exp\frac{\Delta t}{\tau_{-}},\ \ if\ \ \Delta t<0,\end{aligned}\right. \tag{2}\]

where \(i\) and \(j\) respectively refer to the index of post- and pre-synaptic neurons, \(\Delta t=t_{i}-t_{j}\), in which \(t_{i}\) and \(t_{j}\) are the corresponding spike times, and \(\Delta w_{ij}\) is the synaptic weight change. Also, positive constants \(A_{+}\) and \(A_{-}\) scale the strength of potentiation and depression of weight changes, respectively, and \(\tau_{+}\) and \(\tau_{-}\) are positive time constants defining the width of the positive and negative learning window.

In addition, we have utilized the soft bounding by multiplying the term \(w_{ij}(1-w_{ij})\) by \(\Delta w_{ij}\) in order to keep the weights within the range \([0,1]\) and stabilize the weight changes as it converges [46].

### _Memory Layer_

Typically, human decisions are based on memories of previous events. It seems that we learn by storing summaries of individual episodes for long periods and then retrieving them when similar situations arise. Past experiences are recalled through episodic memory [14].

As part of meta-learning, the goal is to encode the information of past decisions into memories, which can be used to make future decisions more effectively. Specifically, the memory layer aims to mimic the hippocampus role and equip the model with an episodic memory system.

Memories are made by changes in collections of neurons and the synaptic connections between them [47]. A memory may be encoded in one group of neural circuits and can be recalled in another. Every time a memory is recalled, it may change depending on the active neural circuits at that time [48]. When a memory is constantly recalled, its active connections become stronger.

Consequently, this layer encodes the information received from the convolutional layer within a portion of LIF neurons with stochastic thresholds, in which spikes of neuron \(i\) are generated stochastically with stochastic intensity \(\rho_{i}=g(u_{i})\)
in which \(g(u_{i})\) is an exponential function of the membrane potential [49]:

\[\rho_{i}=g(u_{i})=\rho_{\theta}exp\left(\frac{u_{i}-u_{\theta}}{\Delta u}\right), \tag{3}\]

where \(\rho_{\theta}\) indicates the stochastic intensity at threshold, \(u_{\theta}\) is the default firing threshold and \(\Delta u\) defines the width of threshold region.

In the memory layer, LIF neurons with stochastic thresholds form an episodic memory system that can recall prior information to solve unseen tasks more effectively. The LIF neuron with stochastic threshold has been utilized to make the model robust for intrinsic or synaptic noises generated by pre-synaptic neurons to improve generalization.

The weights of the memory layer are updated during the memory adaptation stage, as indicated in Figure 2. During this stage, the reward-modulated STDP (R-STDP) is used to ensure that large amounts of information can be encoded efficiently. Using R-STDP, each sample is forced to be encoded with a small subset of neurons. As a result, when previous experiences are recalled frequently, these neurons become more selective towards similar features extracted from the convolutional layer. This process enables the memory layer to adaptively learn and store relevant information.

R-STDP is a type of synaptic plasticity that is thought to underlie learning and memory in the brain [33]. More precisely, it is a process by which the strength of synapses between neurons can be modified based on the relative timing of their activities, as well as reward signals provided by neuromodulators such as dopamine (DA) [50]. The reward signal typically comes from dopaminergic neurons in the midbrain, which release dopamine in response to rewarding or aversive stimuli [34]. R-STDP is a form of reinforcement learning that allows for a more nuanced and context-specific form of synaptic plasticity that takes into account the rewards and punishments associated with different patterns of neural activity [46]. This process can refine and optimize neural circuits over time in response to environmental feedback, leading to enhance learning and adaptation. The R-STDP learning rule can be formulated by the following equations:

\[\begin{split}\frac{dc}{dt}&=-\frac{c}{\tau_{c}}+ STDP(\Delta t)\delta(t-t_{pre/post}),\\ \frac{dw}{dt}&=cd,\\ \frac{dd}{dt}&=-\frac{d}{\tau_{d}}+DA(t),\end{split} \tag{4}\]

where \(c\) represents the eligibility traces that act as synaptic tags, while \(\Delta t=t_{post}-t_{pre}\) measures the time difference between post- and pre-synaptic activities. The variable \(d\) describes the concentration of extracellular dopamine, and \(\delta(t)\) is the Dirac delta function. Also, \(\tau_{c}\) and \(\tau_{d}\) are time constants of eligibility traces and DA uptake, respectively. Lastly, the function \(DA(t)\) models the source of dopamine resulting from the activity of dopaminergic neurons in the midbrain.

In the memory adaptation stage, we aim to encode the extracted information of each sample in a portion of neurons of this layer using the R-STDP learning rule to maintain memory over time. It allows the network to adapt itself to new situations while retaining existing knowledge which improves its generalization capabilities and prevents catastrophic forgetting. Adjusting reward/punishment intervals allows only a certain percentage of neurons to fire; therefore, learning rarely occurs in synaptic weights. The network becomes more flexible and efficient when appropriately considering

Fig. 2: The proposed model architecture overview.
the reward/punishment intervals. The intervals along with the corresponding reward/punishment values are determined using the following piecewise function:

\[r(n_{s})=\left\{\begin{array}{ll}-2,&if~{}~{}n_{s}<c-4s,\\ -1,&if~{}~{}c-4s\leq n_{s}<c-2s,\\ +1,&if~{}~{}c-2s\leq n_{s}<c-s,\\ +2,&if~{}~{}c-s\leq n_{s}\leq c+s,\\ +1,&if~{}~{}c+s<n_{s}\leq c+2s,\\ -1,&if~{}~{}c+2s<n_{s}\leq c+4s,\\ -2,&if~{}~{}c+4s<n_{s},\end{array}\right. \tag{5}\]

where, \(n_{s}\) is the percentage of activated neurons in this layer. Also, the sparsity level is specified by \(c\pm s\), where \(c\) is a constant that represents the average percentage of neurons that can fire, and \(s\) is a spread percentage used to enhance the flexibility of the memory layer.

### _Decision Layer_

The prefrontal cortex (PFC) plays a crucial role in the brain, with decision-making being one of its most important functions [51, 52]. It receives input from multiple regions of the brain to appropriately process information as well as to make decisions, where the history of choices and rewards are dynamically encoded in PFC [53, 54, 55, 56, 57]. A reinforcement learning mechanism is evident in the neural activity of PFC [58].

The principal objective of this layer in our model is to simulate the behavior of the PFC for decision-making using the reward-modulated STDP, introduced in Section II-B as a reinforcement learning rule. This layer receives recalled information from its preceding layer, i.e., the memory layer, and processes it for decision-making. This layer consists of several groups of LIF neurons that are categorized based on the task. Each group of neurons includes \(M\) neurons that can represent the input's class. Typically, for solving an \(N\)-way \(K\)-shot classification problem, this layer would have \(N\) groups of neurons, each containing \(M\) neurons, resulting in a total of \(N\times M\) neurons. Each class could be represented by \(M\) different neurons in a group, enhancing the generalization and versatility of the model. Finally, the input's class can be determined by the group of neurons that fire most frequently.

During the decision adaptation stage, this layer is trained by rewarding accurate predictions and punishing inaccurate ones. To update the weights more effectively, we have implemented adaptive R-STDP in this layer, in which the reward signal value is updated after solving each task to maintain a balance between rewards and punishments in the network. Initially, the reward and punishment values are \(\nicefrac{{1}}{{2}}\) and \(\nicefrac{{-1}}{{2}}\), respectively. These values are updated in each task containing \(N\) samples by \(\nicefrac{{N_{incorrect}}}{{N}}\) and \(\nicefrac{{N_{correct}}}{{N}}\), respectively. Here, \(N_{correct}\) and \(N_{incorrect}\) represent the number of samples that are classified correctly and incorrectly over all samples of a task. Additionally, lateral inhibition mechanisms are utilized between different groups of neurons to make them compete. As a result, when the activity of one group of neurons is increased, the activities of other groups are suppressed, allowing the model to converge faster. Algorithm 1 provides an outline of the complete algorithm in its general form.

```
1:Task distribution \(p(\mathcal{T})\) and corresponding dataset \(\mathcal{D}\)
2:for samples in \(\mathcal{D}\)do
3: Compute \(w_{c}\) for convolutional layer by Eq (2)
4:endfor
5:Randomly initialize \(w_{m}\) and \(w_{d}\) for memory and decision layers, respectively
6:for each epoch do
7: Sample batch of tasks \(\mathcal{T}_{i}=(\mathcal{D}_{i}^{\mathcal{S}},\mathcal{D}_{i}^{\mathcal{Q}})\) from \(p(\mathcal{T})\)
8:for samples in \(\mathcal{D}_{i}^{\mathcal{S}}\)do
9: Compute the percentage of activated neurons \((n_{s})\)
10: Determine the corresponding reward/punishment to \(n_{s}\), as described in Section II-B
11: Update \(w_{m}\) by Eq (4)
12:endfor
13:for samples in \(\mathcal{D}_{i}^{\mathcal{Q}}\)do
14: Determine the corresponding reward/punishment based on the decision layer's prediction, as described in Section II-C
15: Update \(w_{d}\) by Eq (4)
16:endfor
```

**Algorithm 1** Learning procedure for few-shot learning

## III Experiments

The performance of the proposed meta-learner is evaluated by testing its ability to solve various few-shot classification tasks. The primary purpose of this study was to determine whether this simple model could gain comparative results on few-shot learning benchmarks and whether it is capable of learning to learn.

### _The Few-shot Learning Setting_

Meta-learning comprises two phases for training and evaluating a few-shot learning model: meta-training and meta-testing.

During the meta-training phase, we sample a batch of tasks \(\mathcal{T}_{i}\) represented by \(\mathcal{D}_{i}\), and the meta-learner is trained on the support set \(\mathcal{D}_{i}^{\mathcal{S}}\) in the memory adaptation stage (task-level),
followed by the decision adaptation stage (meta-level) on a query set \(\mathcal{D}_{i}^{\mathcal{Q}}\) for each task.

In the meta-testing phase, the trained meta-learner is evaluated on a set of held-out unseen tasks \(\mathcal{T}_{u}\sim p(\mathcal{T})\) that were not used for training. The model is given a support set \(\mathcal{D}_{u}^{\mathcal{S}}\) corresponding to the new task \(\mathcal{T}_{u}\) for the memory adaptation stage and then used to predict labels of the corresponding query set \(\mathcal{D}_{u}^{\mathcal{Q}}\).

### _Model Configuration_

To set up the model for experiments, we first pre-trained the convolutional layer discussed in Section II-A using random samples in the given dataset, where each sample is exposed to this network for \(50ms\) through Poisson encoding. This layer contains \(30\) convolutional filters with a kernel size of \(8\times 8\) and a stride size of \(2\) for downsampling. To enhance the edge perception of the model, we also employed the lateral inhibition mechanism in this layer, where each neuron inhibits the activity of neighboring neurons of an individual filter using lateral connections.

In the memory layer, we used \(N=100\) neurons with \(u_{rest}=-70mV\), \(\rho_{\theta}=1/ms\), \(u_{\theta}=-50mV\), and \(\Delta u=5mV\) in Equation (3). Furthermore, we have included reward/punishment intervals from Section II-B to regulate the sparsity of neuronal activity in the memory layer to around \(15\%\) by using \(c=15\%\) and \(s=3\%\) in Equation (5) to boost the model's generalization.

To make the decision layer scalable, we consider \(M=10\) neurons for each group representing a class in the decision layer, as discussed in Section II-C.

### _Few-Shot Classification Tasks_

The \(N\)-way \(K\)-shot classification problem involves \(N\) different classes, each containing \(K\) samples. A powerful meta-learner should be capable of recognizing inputs by comparing them rather than memorizing a definite mapping between those inputs and the desired classes. We scale up our approach to few-shot classification tasks using the Omniglot and Double MNIST datasets.

The Omniglot dataset is a well-known benchmark for few-shot learning introduced by Lake et al. [59]. This dataset includes handwritten characters from \(50\) different languages, representing 1632 different classes, with \(20\) samples in each class as shown in Figure 3. According to the baseline models, 1200 characters are randomly selected for training, and the remaining ones are used for testing. We also applied data augmentation by rotating each instance of a class by a multiple of 90 degrees, following the approach suggested by Santoro et al. [10].

The Double MNIST dataset consists of 100 distinct classes of two-digit numbers, each containing 1000 unique handwritten samples as depicted in Figure 4. For training purposes, 80 classes are randomly chosen, while the remaining 20 classes are preserved for testing.

In order to evaluate the model on the \(N\)-way \(K\)-shot classification problem, we feed \(N\times K\) samples to the meta-learner in a random order to perform the memory adaptation stage explained in Section II-B. We then present a new, unlabeled sample from one of the \(N\) classes and report the average accuracy on this last \((N\times K+1)\)-th pass based on the decision layer discussed in Section II-C in a similar fashion to SNAIL [11].

Using these standard datasets, we were able to demonstrate the efficiency and generalization capabilities of our proposed model.

### _Results_

We present an analysis of the results obtained from solving few-shot classification tasks. Firstly, we examine the performance of our proposed model in solving 5-way 5-shot and 5-way 1-shot classification tasks on the Omniglot dataset, as shown in Table I.

Our proposed model outperformed other models such as Siamese Networks [60], Matching Networks [4], Prototypical Networks [5], and Meta Networks [61] in solving 5-way 5-shot tasks on Omniglot. It also performed similarly to the state-of-the-art meta-learning models, namely MAML [62] and

Fig. 4: Illustration of a few of the two-digit numbers provided in Double MNIST.

Fig. 3: Illustration of a few of the alphabets provided in Omniglot.
SNAIL [11]. Moreover, our proposed model achieved superior results in solving 5-way 1-shot tasks on the same dataset compared to the other competitors.

By comparing the results of our proposed model with those of comparable simple and generic methods such as MANN [10], it can be concluded that our proposed model is highly effective in solving problems with a small number of samples.

MANN [10] is a regular method for meta-learning that utilizes Neural Turing Machine (NTM) [63] as the embedded external memory. This model stores the information of each task in its memory so that when it encounters a new one, it recovers the information needed to solve that task by measuring its cosine similarity with the information stored in the memory. The disadvantage of this method is that it requires more memory than our model since our proposed model uses different combinations of neurons of the episodic memory layer to simulate memory. Moreover, the episodic memory layer in our model is more flexible than the embedded memory in MANN [10]. While MANN retrieves information using cosine similarity, which returns a row of information related to a task that has the highest degree of similarity, some tasks in practice are composed of a combination of several tasks and cannot be solved by retrieving information from one previously solved task. Our proposed model overcomes this limitation by providing an episodic memory system that resembles the human brain.

In another experiment, we evaluated the performance of our model in solving \(5\)-way \(1\)-shot tasks on the Double MNIST dataset, with results presented in Table II. Our primary objective was to compare our model's performance with a previously introduced meta-learning model, which is essentially the same as MAML and its first-order variation (FOMAML) but uses spiking neurons as its building blocks with surrogate gradient descent as the learning rule [28]. In spite of their important work embedding SNNs into MAML, it was expected to have the same or weaker performance than the original MAML model, as confirmed by the results. However, our proposed model is entirely different in terms of network design and learning rules, and we attempted to provide a new meta-learning approach that is consistent with biological observations.

The results show that our model can compete with other meta-learning models, despite being based on biologically plausible learning rules that are less computationally complex than the surrogate gradient descent [24]. It should be noted that previous models were all based on gradient computations, which are computationally expensive and not biologically plausible. Our model takes advantage of the potential of SNNs in meta-learning, which is difficult to implement using conventional artificial neural networks.

We further analyze the effect of the sparsity level in the memory layer to understand its impact when we encode the information of each sample in either a smaller or larger portion of neurons by considering the appropriate reward/ punishment intervals. Table III presents the results corresponding to different sparsity levels in solving 5-way 1-shot tasks using the Omniglot and Double MNIST datasets. It shows that encoding information in too few neurons may not yield the best performance as it lacks sufficient information for decision-making. Conversely, encoding information in a large portion of neurons can also decrease the model's performance as it cannot efficiently preserve previously learned information. By comparing the results of different sparsity levels for Omniglot (5-way 1-shot) and Double MNIST (5-way 1-shot), we can observe that decreasing the sparsity level of the memory layer reduces the performance of the model in both tasks. However, this effect is more pronounced for Omniglot, which contains more classes than Double MNIST. Due to this, it is more susceptible to catastrophic forgetting when compared to Double MNIST. Therefore, selecting an appropriate sparsity level for the memory layer is crucial to achieve high performance and maintain the knowledge learned from previous tasks to prevent
## IV Discussion

Meta-learning algorithms have recently attracted the attention of many researchers due to their ability to learn new tasks with just a few samples. Despite the remarkable performance of existing meta-learning models, designing a lightweight and efficient bio-plausible meta-learning model is a critical challenge. Bio-plausible neural networks must have three essential characteristics [64]: (i) the ability to integrate temporal input and generate spikes, (ii) the use of spike-based computation for both training and inference, (iii) and the ability to use learning rules based on findings from biological experiments. As mentioned in Section I, some biologically inspired models have been proposed. Wang et al. [15] have discovered the role of PFC as a meta-reinforcement learning system. Later on, Ritter et al. [19] improved the prior work and utilized the differentiable neural dictionary (DND) to store some information to solve new tasks. Although both models are the basis for our work, they use LSTMs, DND, gradient descent, and backpropagation, hindering their bio-plausibility. Stewart and Neftci have embedded SNNs into MAML, which is highly advantageous. Still, they use surrogate gradient and backpropagation, restricting their algorithm's bio-plausibility. Furthermore, Subramoney et al. [29], and Scherr et al. [30] have proposed a novel approach to enable quick learning in SNNs; however, their models do not meet bio-plausibility requirements. Here, we used LIF neurons along with their stochastic threshold variation incorporating a combination of STDP and R-STDP learning rules. Accordingly, the proposed model is highly bio-plausible as it meets all the aforementioned requirements.

On the other hand, the efficiency and hardware friendliness of SNNs makes them an appropriate choice for neuromorphic hardware deployment. Neuromorphic hardware is particularly well suited for online learning at the edge. In spite of this, they face several challenges, such as learning from scratch on data-hungry models due to robustness and time-to-convergence issues. Here, we have presented a lightweight and simple SNN model with a pre-trained convolutional layer that can alleviate these issues. According to the results, our model can learn new tasks in a few-shot setting. As a result, this enables learning in low-data regimes since accessing sufficient labeled data is demanding. In addition, our proposed model features a highly efficient episodic memory that significantly mitigates the issue of catastrophic forgetting. This is achieved through the memory's ability to store a vast amount of information and link similar underlying patterns, received from the convolutional layer. Consequently, it can effectively and partially recall previously stored data when presented with new, similar data. As a result of this functionality, the proposed model exhibits a high degree of generalization, enabling it to adapt to novel tasks with few examples.

We conducted an analysis to assess the efficacy of the memory layer, which involved the memory representation of a subset of samples from the Omniglot dataset, as depicted in Figure 5. The memory representation of each sample is a binary vector that indicates which neurons were activated for that sample. To evaluate the correlation between pairs of samples, we calculated their Pearson correlation coefficient and generated a corresponding heatmap, which is also presented in Figure 5. Higher values on the heatmap indicate a stronger correlation between the memory representations of the respective samples. Our results demonstrate that the sample with index 0 shares similar features with sample indices 1 and 2, resulting in a high correlation with sample index 1 due to their high degree of similarity, as well as a correlation with sample index 2 given the presence of some shared patterns. In contrast, sample index 1 has a low correlation with sample index 2 since they do not share any analogous patterns. These findings suggest that the memory layer functions appropriately by partially selecting neurons for underlying similarities.

## V Conclusion and Future Work

We presented a bio-plausible meta-learner with spiking neural networks (SNNs), motivated by the need for a meta-learner to be capable of mimicking the human brain's behavior when encountering a new unseen task. We have appropriately implemented the episodic memory and decision-making system by studying the role of the Hippocamp and the Prefrontal Cortex (PFC) to enable quick learning by incorporating past experiences. Our proposed model can be considered as an adaptive lifelong learning algorithm because it can attend to experiences over a lifetime and overcome catastrophic forgetting. As a result, our model can learn faster and generalize better due to its lifelong memory. Furthermore,
it is computationally very efficient as it relies on spike-timing-dependent plasticity (STDP) that empowers the memory layer to decide what experiences are worth remembering.

Our work represents an initial step toward developing a bio-plausible meta-learning model using SNNs, and future works could build on this foundation to address more complex tasks. Moreover, by scaling these ideas, we could facilitate the implementation of meta-learning models on spike-based neuromorphic devices, enabling fast learning on neuromorphic hardware.
