Learning Spiking Neural Network from Easy to Hard task
Starting with small and simple concepts, and gradually introducing complex
and difficult concepts is the natural process of human learning. Spiking Neural
Networks (SNNs) aim to mimic the way humans process information, but current
SNNs models treat all samples equally, which does not align with the principles
of human learning and overlooks the biological plausibility of SNNs. To address
this, we propose a CL-SNN model that introduces Curriculum Learning(CL) into
SNNs, making SNNs learn more like humans and providing higher biological
interpretability. CL is a training strategy that advocates presenting easier
data to models before gradually introducing more challenging data, mimicking
the human learning process. We use a confidence-aware loss to measure and
process the samples with different difficulty levels. By learning the
confidence of different samples, the model reduces the contribution of
difficult samples to parameter optimization automatically. We conducted
experiments on static image datasets MNIST, Fashion-MNIST, CIFAR10, and
neuromorphic datasets N-MNIST, CIFAR10-DVS, DVS-Gesture. The results are
promising. To our best knowledge, this is the first proposal to enhance the
biologically plausibility of SNNs by introducing CL.
Lingling Tang, Jiangtao Hu, Hua Yu, Surui Liu, Jielei Chu
# Learning Spiking Neural Network from Easy to Hard task

###### Abstract

Starting with small and simple concepts, and gradually introducing complex and difficult concepts is the natural process of human learning. Spiking Neural Networks (SNNs) aim to mimic the way humans process information, but current SNNs models treat all samples equally, which does not align with the principles of human learning and overlooks the biological plausibility of SNNs. To address this, we propose a CL-SNN model that introduces Curriculum Learning(CL) into SNNs, making SNNs learn more like humans and providing higher biological interpretability. CL is a training strategy that advocates presenting easier data to models before gradually introducing more challenging data, mimicking the human learning process. We use a confidence-aware loss to measure and process the samples with different difficulty levels. By learning the confidence of different samples, the model reduces the contribution of difficult samples to parameter optimization automatically. We conducted experiments on static image datasets MNIST, Fashion-MNIST, CIFAR10, and neuromorphic datasets N-MNIST, CIFAR10-DVS, DVS-Gesture. The results are promising. To our best knowledge, this is the first proposal to enhance the biologically plausibility of SNNs by introducing CL.

 Spiking Neural Network, Curriculum Learning, Biologically Plausibility, Deep Learning.

## I Introduction

Efficient and low-power earning and information processing, similar to the human brain has always been a goal pursued in the field of deep learning. The design of Spiking Neural Networks (SNNs) is inspired by the neuroscience research on human brain. It attempts to simulate the behavior and information processing of neurons to better approximate the functioning principles of the biological brain [1, 2]. Unlike traditional Artificial Neural Networks (ANNs) [3, 4, 5], which use continuous-valued activations, SNNs use discrete, time-based representations spikes or action potentials to transmit and process information [6]. SNNs have the potential for efficient and low-energy information processing, as they only transmit spikes when necessary. This makes SNNs attractive for implementing neuromorphic hardware and energy-efficient computing systems [7]. SNNs can capture the temporal dynamics and precise timing information of neural computations, which are crucial for areas such as speech recognition [8], event-based processing [7], and sensory integration [9].

Unsupervised SNNs [15, 16, 17] are primarily based on biological principles, with the most common being the Spiking Time Dependency Plasticity (STDP) rule [18]. It adjusts the synaptic weights based on the timing relationship between pre-synaptic and post-synaptic neurons' spike emissions, demonstrating high biological plausibility. However, it is challenging to achieve good performance on deep networks.

Supervised learning methods [19, 20], on the other hand, train SNNs using labeled data by minimizing the error between predicted outputs and true labels to adjust network parameters [21]. Since spiking neural networks transmit discrete spike signals and lack differentiability [22], surrogate gradient [23] methods are commonly used for backpropagation to optimize the parameters. This approach has shown promising results and provides a relatively simple solution for handling the non-differentiability of SNNs.Another learning approach is ANN to SNN conversion, where an ANNs is first trained, and then a structurally equivalent SNNs is constructed, initializing the SNN's weights with those of the ANNs. This conversion allows SNNs to achieve performance close to that of ANNs [24, 25, 26]. However, it requires longer time steps [27] and is not suitable for event-based datasets.

When we learn new knowledge, we often start with small and simple concepts and gradually introduce complex and challenging knowledge. This is a natural learning process for us. Just as we receive education based on a curriculum designed by schools to continuously enhance our knowledge. SNNs is proposed to mimic the way humans process information [7]. However, the current SNNs models treat all samples equally during the training process without considering whether the model's learning capacity can effectively acquire relatively complex and challenging knowledge [10, 11, 12]. This approach is not reasonable and does not align with the natural laws of human learning, resulting in lower biological interpretability. Therefore, to improve the biological plausibility of SNNs, we propose introducing Curriculum Learning (CL) into SNNs to make them learn more like humans.

CL is inspired by the process of human learning new knowledge. Its main idea is to gradually increase the training difficulty, allowing the model to learn useful knowledge more quickly and better generalize to new data in later stages of training [13]. Specifically, CL provides training data to the model in a certain order called a "curriculum" [14]. Initially, the model is exposed to simple training samples, such as easily classifiable samples or correctly labeled ones. Once the model performs well, the difficulty is gradually increased by providing more complex and challenging samples. This process aligns closely with the process of human learning new knowledge.

The sequential learning process of CL mimics the natural way humans acquire new knowledge. In the [28], CL is summarized as a combination of a difficulty estimator and a training scheduler. The difficulty estimator evaluates the difficulty of samples based on selected evaluation features such as complexity [29], noise [30, 31], or suggestions from
a mature teacher network [29]. The training scheduler then feeds the samples to the model in the order of difficulty according to the designed scheduling rules, by which can improve the model convergence speed. Pre-defined CL, as described in [28], relies on manually defining methods for evaluating sample difficulty and scheduling strategies. While simpler to implement, this approach tends to overlook the model's feedback during the learning process. Moreover, it is challenging to determine the most suitable training schedule, such as when and how much to introduce more challenging samples, based solely on prior knowledge, especially for specific tasks and datasets. It is more suitable for small datasets. In contrast to pre-defined CL, automatic CL introduces the concept of a teacher network. The teacher network can be the student network itself, which evaluates its own learning progress based on the training loss and adjusts the training samples accordingly. Alternatively, it can be a well-trained, more advanced network model that assesses sample difficulty based on the teacher model's performance on those samples and dynamically adjusts the sample inputs based on feedback from the student network model.

Confidence-aware loss has been proposed as a type of loss function that considers the model's confidence or certainty in its predictions [32, 33]. During the training process, it aims to assign higher importance to samples with higher confidence and lower importance to samples with lower confidence. This is achieved by introducing sample weights, namely confidence, and adjusting the weights of samples during the training process to modulate their contribution to the model parameter updates during backpropagation. The confidence-aware loss achieves curriculum learning without the need for predefined curriculum design.

The paper's main contributions can be summarized as follows:

* We propose a CL-SNN model that has higher biological interpretability by introducing CL into SNNs. The model dynamically evaluates the difficulty of samples, assigns higher confidence to simple samples, and amplifies their contribution in backpropagation. It automatically reduces the impact of more difficult samples on parameter updates. This approach exhibits high biological plausibility and effectively simulates the process of human learning new knowledge. To our best knowledge, this is the first attempt to enhance the biologically plausibility of SNNs by introducing CL.
* We evaluate our CL-SNN model on three static image datasets, MNIST, CIFAR10 and Fashion-MNIST, as well as three neuromorphic datasets, including DVS-Gesture, N-MNIST, and CIFAR10-DVS for classification tasks. The results surpass the current state-of-the-art experimental results for all six datasets.

The rest of the paper is organized as follows. Section II is the proposed methods. Section III introduces the experiment results. And in section IV we make a summary and discussion.

## II Methods

### _Neuron model_

The most commonly used spiking neuron model for SNNs is Leaky Integrate and Fire (LIF) neuron model [34]. It features a simple computation process while preserving essential biological characteristics. The dynamics of LIF can be described as follows:

\[\tau_{m}\frac{\mathrm{d}V\left(t\right)}{\mathrm{d}t}=-\left(V\left(t\right)- V_{r}\right)+I\left(t\right), \tag{1}\]

in which \(V_{r}\) is the fixed resting potential, \(V(t)\) represents the membrane potential at time-step \(t\). \(V(t)\) decays continuously and, in the absence of input, it will decay until it reaches \(V_{r}\). \(\tau_{m}\) represents the membrane time constant. The LIF neuron model combines biological plausibility with computational convenience. Since SNNs transmit discrete spike signals, the dynamical mechanism of SNNs can be described as follows:

\[Q_{t}=f(V_{t-1},I_{t}), \tag{2}\]

\[O_{t}=\theta(Q_{t}-V_{th}), \tag{3}\]

\[V_{t}=Q_{t}\left(1-O_{t}\right)+V_{r}O_{t}. \tag{4}\]

where \(Q_{t}\) represents the membrane potential after receiving input, and depend on the membrane potential at time-step \(t-1\) and the input \(I_{t}\). \(I_{t}=\mathbf{w}_{ji}*O_{t}\) and \(\mathbf{w}_{ji}\) denotes the weight for spiking neuron \(i\) and \(j\). \(Q_{t}\) is calculated by the function \(f(x)\). \(O_{t}\) represents the output of the pre-neuron,in which

\[\theta(x)=\left\{\begin{array}{ll}1,&x\geqslant 0\\ 0,&x<0\end{array}\right. \tag{5}\]

\(O_{t}=1\) means the potential after receiving input reaches the threshold \(V_{th}\) and generate a spike, and \(O_{t}=0\) means neuron keep silent. \(V_{t}\) represents the membrane potential after fire process. Eqs.(2)(3)(4) describe the charging, discharging, and resetting processes of a spiking neuron, respectively. The differences between different spiking neuron models mainly lie in the function \(f(x)\). The \(f(x)\) for LIF neuron is defined as:

\[Q_{t}=f(V_{t-1},I_{t})=V_{t-1}+\frac{1}{\tau_{m}}\left(-\left(V_{t-1}-V_{r} \right)+I_{t}\right). \tag{6}\]

Most SNNs models that utilize LIF neurons typically set the membrane time constant \(\tau_{m}\) as a fixed constant. However, in [10] proposed the importance and necessity of a learnable \(\tau_{m}\) on model performance. They introduced the PLIF model, which simultaneously learns \(\tau_{m}\) and synaptic weights during training, resulting in improved fitting capabilities. To avoid errors caused by \(\tau_{m}\) in the denominator during the learning process, Eq.(6) is rewritten as follows:

\[Q_{t}=\left(1-g(a)\right)V_{t-1}+g(a)\left(V_{r}+I_{t}\right) \tag{7}\]

where \(\tau_{m}=\frac{1}{g(a)}\in\left(1,+\infty\right)\), a is a learnable parameter. In this paper, \(g(a)\) is taken as the sigmoid function, \(g(a)=\frac{1}{1+e^{\left(-a\right)}}\).
### _Surrogate gradient_

Because SNNs transmit discrete spike signals and exhibit non-continuous neuron responses. And \(\theta(x)\) is non-differentiable,the derivative of \(\theta(x)\) is:

\[\theta^{\prime}(x)=\left\{\begin{array}{ll}+\infty,&x=0\\ 0,&x\neq 0\end{array}\right. \tag{8}\]

it becomes challenging to use traditional backpropagation for gradient updates and parameter optimization in SNNs. Previous research [23] has proposed the surrogate gradient method to enable backpropagation in SNNs. The principle of surrogate gradient is to use \(y=\theta(x)\) during forward propagation and \(\frac{\mathrm{d}y}{\mathrm{d}x}=\phi^{\prime}(x)\) during backward propagation, instead of \(\frac{\mathrm{d}y}{\mathrm{d}x}=\theta^{\prime}(x)\), where \(\phi(x)\) represents the surrogate function. \(\phi(x)\) is typically a smooth and continuous function that has a similar shape to \(\theta(x)\). The training process of SNNs using the surrogate gradient method is as follows:

1. Forward Propagation: The input is passed through the network, simulating the spiking behavior of neurons, and generating spike outputs.
2. Surrogate Function Calculation: Based on the spike outputs, the value of the surrogate function is computed, and the gradient of the surrogate function is calculated.
3. Backpropagation: The gradients of the surrogate function are used for backpropagation to update the network parameters.

The surrogate function used in this paper is:

\[\phi(x)=\frac{1}{\pi}arctan(\frac{\pi}{2}\alpha x)+\frac{1}{2}. \tag{9}\]

And we get:

\[\frac{\mathrm{d}}{\mathrm{d}x}\phi(x)=\frac{\alpha}{2\left(1+\left(\frac{\pi} {2}\alpha x\right)^{2}\right)}. \tag{10}\]

Gradient-based surrogate simplifies the computation of SNNs and achieves promising results.

### _Curriculum learning with confidence-aware loss_

The main idea of CL is to accelerate the learning of useful knowledge by gradually increasing the training difficulty, enabling the model to generalize better to new data in later stages of training. Common methods in CL involve presenting the model with simple data first and gradually introducing more challenging data in a predefined order of difficulty. This has led to the development of various difficulty measures and training schedulers [28]. However, most of these methods are suit for small datasets or focus on curriculum design at the data scheduling level. Moreover, it always requires additional resources for sample difficulty evaluation and sample scheduling.

Previous research [32, 33] has introduced the concept of confidence-aware loss, which is a type of loss function that takes into account the confidence or certainty of model predictions. During training, it aims to assign higher importance to samples with higher confidence and lower importance to samples with lower confidence. A confidence-aware loss function, denoted as \(l(\hat{y},y,\omega)\), introduces a learnable parameter \(\omega\) as an additional input compared to the traditional loss function \(l(\hat{y},y)\), \(\omega\) is the confidence or reliability of the current prediction result \(\hat{y}\).

In [35], a general and lightweight approach to implementing the critical purpose of curriculum learning was proposed. It is based on confidence-aware loss. And its mathematical definition is as follows:

\[CAL_{\lambda}\left(l_{i},\omega_{i}\right)=\left(l_{i}-\epsilon\right)\omega_ {i}+\lambda\left(log(\omega_{i})\right)^{2}, \tag{11}\]

Fig. 1: The structure of proposed CL-SNN.
where \(l_{i}\) represents the initial training loss, which can be calculated using common loss functions such as cross-entropy loss or mean squared error (MSE). \(\omega_{i}\) is the confidence or certainty associated with sample \(i\) and is a learnable parameter. \(\epsilon\) is the threshold used to differentiate between easy and difficult samples and can be set as the average of the batch initial training loss \(l_{i}\) or a predetermined constant. \(\lambda\) is a hyper parameter that controls the regularization term. By dynamically learning the confidence \(\omega\) for each sample, aiming to reduce the impact of difficult samples on parameter updates while amplifying the confidence of simple and reliable samples, thereby expanding their contribution to the model. As training progresses, the fitting capacity of model continuously improves, leading to high confidence for all samples in the end. For the confidence \(\omega_{i}\), it scales the learning level of the samples. To simplify the computation, the confidence \(\omega\) directly obtained from the training loss \(l_{i}\) of the sample is defined as follows [35]:

\[\omega_{i}(l_{i})=exp\left(-W\left(\frac{1}{2}max\left(-\frac{2}{e},\eta \right)\right)\right), \tag{12}\]

with \(\eta=\frac{l_{i}-\epsilon}{\lambda}\), and W is the Lambert W function. The final loss used for backpropagation is computed by combining the confidence \(\omega\) obtained from the initial training loss \(l_{i}\) and \(l_{i}\) itself. In CL-SNN, we directly calculate the initial loss \(l_{i}\) using the cross-entropy loss as follows:

\[L=\frac{1}{N}\sum_{i}L_{i}=-\frac{1}{N}\sum_{i}\sum_{c=1}^{M}y_{ic}log(p_{ic}), \tag{13}\]

Combining Eqs.(11)(12)(13), we obtain the confidence-aware loss function for CL-SNN. For samples with a large initial training loss, they can be considered as more challenging for the current model. Hence, we assign them lower confidence values to reduce their impact during parameter updates. Conversely, for samples with smaller \(l_{i}\) values, we can regard them as simpler and more reliable, this just like a dynamic CL.

## III Experiments

We evaluated the proposed CL-SNN model on classification tasks using six datasets, including three static datasets (MNIST, CIFAR10, Fashion-MNIST) and three neuromorphic datasets (N-MNIST, CIFAR10-DVS, DVS-Gesture).

### _Experiment setting_

The experiments were conducted using a network architecture similar to that described in [10]. The SpikingJelly framework [36] was utilized, without employing operations similar to Poisson encoding on the input, but instead directly feeding it into the network. For the MNIST, Fashion-MNIST, and N-MNIST datasets, we employed the same network architecture: 128c3-BN-MP2-128c3-BN-MP2-DP-FC2048-DP-FC100-AP10. Here, "128c3" refers to a convolutional layer with 128 channels and kernel size 3. "MP2" represents max-pooling layer with kernel size 2. "BN" represents batch normalization. "DP" refers to a dropout layer. We use the PLIF neuron described in Section 3.1. The detailed network architecture for other datasets are described in the code.

In the surrogate function, we set \(\alpha=2\), so

\[\phi(x)=\frac{1}{\pi}arctan(\pi x)+\frac{1}{2}, \tag{14}\]

and \(\phi^{\prime}\left(x\right)=\frac{1}{1+(\pi x)^{\eta}}\).

For the threshold \(\epsilon\) used to distinguish between easy and hard samples in the confidence-aware loss, there are two methods to define. One approach is to set \(\epsilon\) as a fixed constant, while the other approach involves using a dynamic threshold. We conducted experiments for both cases. In the first case, we set \(\epsilon=log(c)\), where \(c\) represents the number of classes. For datasets such as MNIST, CIFAR10, Fashion-MNIST, N-MNIST, and CIFAR10-DVS, \(c\) is equal to \(10\). For the DVS-Gesture dataset, \(c\) is \(11\), and we set \(\lambda=0\). In the second case, we set \(\epsilon\) as the average initial loss of each batch, resulting in a dynamic threshold, and in this case we set \(\lambda=1\).

### _Experiment results_

The experimental results of our proposed CL-SNN model on static datasets for classification tasks, along with the comparison with state-of-the-art methods, are shown in the Table(I). Our method outperforms the comparison methods on all six datasets, and our model exhibits higher biological plausibility, aligning with the principles of human knowledge acquisition. The results on neuromorphic datasets are show in Table(II), all show a better performance. The accuracy change curve of the model for different datasets is shown in Figure(2).

In terms of the difficulty differentiation threshold \(\epsilon\), experimental results show that dynamic \(\epsilon\) performs better on CIFAR10, N-MNIST and DVS-Gesture datasets. And for MNIST, Fashion-MNIST and CIFAR10-DVS, the fixed \(\epsilon\) achieves better performance.

The confidence levels \(\omega\) vary for samples of different difficulties, as shown in Figure(3a). Easier samples reach the maximum confidence earlier, while more difficult samples take longer to reach maximum confidence. The level of confidence-aware loss for different samples is illustrated in Figure(3b).

### _Other metrics_

We recorded other evaluation metrics of the model on six datasets, including macro Precision, macro Recall, and macro F1-score, hoping to provide some assistance for future research.

Macro Precision is a metric used in multi-class classification tasks to measure the average precision across all classes and provides an overall measure of the precision performance across the different classes in the classification task. Similar to macro Precision, macro Recall is a metric used in multi-class classification tasks to measure the average recall across all classes. It calculates the average of the recall values for each class. Macro F1-score provides an overall measure of the F1-score performance across the different classes in the classification task.The macro Precision, macro Recall, and macro F1-score for used datasets are show in Table(III).
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**Dataset** & **model** & **method** & **accuracy** \\ \hline \multirow{8}{*}{N-MNIST} & BackEISNN [38] & directed trained & 99.57 \\  & Ling et al. [39] & spike-based BP & 99.49 \\  & Wu et al. [44] & STBP with NeuNorm & 99.53 \\  & Zhu et al. [42] & Time-based BP & 99.39 \\  & CL-SNN with fixed \(\epsilon\)(**ours**) & Spike-based BP & **99.58** \\  & CL-SNN with dynamic \(\epsilon\)(**ours**) & Spike-based BP & **99.63** \\ \hline \multirow{8}{*}{CIFAR10-DVS} & Ling et al. [39] & spike-based BP & 64.6 \\  & Hanle et al. [45] & STBP-tdBN & 67.8 \\  & Wu et al. [44] & STBP with NeuNorm & 60.5 \\  & CL-SNN with fixed \(\epsilon\)(**ours**) & Spike-based BP & **69.4** \\  & CL-SNN with dynamic \(\epsilon\)(**ours**) & Spike-based BP & **68.6** \\ \hline \multirow{2}{*}{DVS-Gesture} & Ling et al. [39] & spike-based BP & 91.32 \\  & BRP-SNN [40] & spike-based BP & 80.9 \\  & CL-SNN with fixed \(\epsilon\)(**ours**) & Spike-based BP & **94.44** \\  & CL-SNN with dynamic \(\epsilon\)(**ours**) & Spike-based BP & **94.72** \\ \hline \hline \end{tabular}
\end{table} TABLE III: The proposed CL-SNN model’s macroPrecision, macroRecall, macroF1-score on different datasets
## IV Conclusion and discussion

The previous SNNs model always processed all samples indiscriminately, which was not in line with the natural process of human learning new knowledge from easy to difficult. We propose to introduce CL based on confidence perception loss function into SNNs. CL is a training strategy that makes the model learn knowledge from easy to difficult. It is inspired by the process of human learning new knowledge. And based on this, a CL-SNN model was proposed, which has a high degree of biological rationality. By scaling the contributions of samples with different confidence levels in parameter updates, the core principles of course learning are achieved. To our knowledge, this is the first proposal to enhance the biological rationality of SNNs by introducing CL.

However, determining the most suitable curriculum learning strategy for a specific task still requires further exploration. Confidence-aware loss is a convenient method, but the choice of difficulty threshold and the measurement of difficulty are not unique. Nonetheless, this paper provides insights into how to design SNNs that align with human cognitive processes.

## Acknowledgments

This work is supported by the National Natural Science Foundation of China (No. 62276218), Sichuan Science and Technology Program (No. 2022YFG0031) and Chengdu International Science and Technology Cooperation (No. 2023-GH02-00029-HZ).
