Paired Competing Neurons Improving STDP Supervised Local Learning In
  Spiking Neural Networks
Direct training of Spiking Neural Networks (SNNs) on neuromorphic hardware
has the potential to significantly reduce the energy consumption of artificial
neural network training. SNNs trained with Spike Timing-Dependent Plasticity
(STDP) benefit from gradient-free and unsupervised local learning, which can be
easily implemented on ultra-low-power neuromorphic hardware. However,
classification tasks cannot be performed solely with unsupervised STDP. In this
paper, we propose Stabilized Supervised STDP (S2-STDP), a supervised STDP
learning rule to train the classification layer of an SNN equipped with
unsupervised STDP for feature extraction. S2-STDP integrates error-modulated
weight updates that align neuron spikes with desired timestamps derived from
the average firing time within the layer. Then, we introduce a training
architecture called Paired Competing Neurons (PCN) to further enhance the
learning capabilities of our classification layer trained with S2-STDP. PCN
associates each class with paired neurons and encourages neuron specialization
toward target or non-target samples through intra-class competition. We
evaluate our methods on image recognition datasets, including MNIST,
Fashion-MNIST, and CIFAR-10. Results show that our methods outperform
state-of-the-art supervised STDP learning rules, for comparable architectures
and numbers of neurons. Further analysis demonstrates that the use of PCN
enhances the performance of S2-STDP, regardless of the hyperparameter set and
without introducing any additional hyperparameters.
Gaspard Goupy, Pierre Tirilly, Ioan Marius Bilasco
# Paired Competing Neurons Improving STDP Supervised Local Learning In Spiking Neural Networks

###### Abstract

Direct training of Spiking Neural Networks (SNNs) on neuromorphic hardware has the potential to significantly reduce the high energy consumption of Artificial Neural Networks (ANNs) training on modern computers. The biological plausibility of SNNs allows them to benefit from bio-inspired plasticity rules, such as Spike Timing-Dependent Plasticity (STDP). STDP offers gradient-free and unsupervised local learning, which can be easily implemented on neuromorphic hardware. However, relying solely on unsupervised STDP to perform classification tasks is not enough. In this paper, we propose Stabilized Supervised STDP (S2-STDP), a supervised STDP learning rule to train the classification layer of an SNN equipped with unsupervised STDP. S2-STDP integrates error-modulated weight updates that align neuron spikes with desired timestamps derived from the average firing time within the layer. Then, we introduce a training architecture called Paired Competing Neurons (PCN) to further enhance the learning capabilities of our classification layer trained with S2-STDP. PCN associates each class with paired neurons and encourages neuron specialization through intra-class competition. We evaluated our proposed methods on image recognition datasets, including MNIST, Fashion-MNIST, and CIFAR-10. Results showed that our methods outperform current supervised STDP-based state of the art, for comparable architectures and numbers of neurons. Also, the use of PCN enhances the performance of S2-STDP, regardless of the configuration, and without introducing any hyperparameters. Further analysis demonstrated that our methods exhibited improved hyperparameter robustness, which reduces the need for tuning.

 Spiking Neural Networks, Image Recognition, Supervised Local Learning, STDP, Intra-class competition.

## I Introduction

Spiking Neural Networks (SNNs), often considered as the third generation of artificial neural networks, are alternatives that more closely mimic brain behavior [1], compared to second-generation Artificial Neural Networks (ANNs). Spiking neurons communicate through discrete binary signals, called spikes, to transmit information, allowing asynchronous processing with spatiotemporal dynamics. Recently, SNNs have gained growing attention due to their energy-efficient computing ability, when implemented on dedicated neuromorphic hardware [2, 3, 4]. Especially, on-chip training is of particular interest since it is known that the training phase consumes the most energy.

However, training SNNs directly on neuromorphic hardware remains a significant challenge. Due to the non-differentiable nature of the spike generation function, the use of the gradient-based Backpropagation (BP) algorithm is not as straightforward as it is for traditional ANNs. Many approaches were proposed in the literature [5, 6] to adapt the BP algorithm for SNNs, mainly relying on gradient approximation. Nevertheless, although these methods have achieved state-of-the-art results, they are challenging to implement on neuromorphic hardware as they require global communication during backpropagation. Consequently, other approaches attempt to make gradient computation local, notably by utilizing feedback connections [7, 8], or by employing a layer-wise cost function [9, 10]. These approaches still do not solve the gradient approximation problem.

On the other hand, the Spike Timing-Dependent Plasticity (STDP) [11] is a bio-inspired unsupervised learning rule based on Hebb's theory [12]. In this rule, the synaptic weights are updated locally, i.e., only with the information from pre and postsynaptic neurons, which greatly facilitates implementation on neuromorphic hardware. Convolutional SNNs (CSNNs) trained with STDP have demonstrated the ability to extract relevant features from images [13, 14, 15, 16]. In addition, they benefit from unsupervised learning, which reduces the dependency on labels and improves robustness to changing data. However, these SNNs still require an additional supervised module to perform the classification from the extracted features. To leverage the potential of unsupervised STDP in neuromorphic hardware and enable end-to-end SNN solutions, spike-based classifiers with supervised learning must be designed.

In particular, combining unsupervised STDP training for feature extraction and supervised STDP training for classification, as shown in Figure 1, is very appealing [17, 18]. First, both training processes rely on the same rule, which facilitates hardware implementation. Second, unsupervised training can be used to reduce the number of supervised layers, and hence, the reliance on labeled data. Third, unlike BP-based methods, supervised training with STDP may eliminate the need for global communication and gradient approximation. Several supervised adaptations of STDP are reported in the literature [18, 19, 20, 21, 22, 23]. Among them, Reward-modulated STDP (R-STDP) [18] is a reinforcement learning rule based on Winner-Takes-All (WTA) competition that modulates the polarity of the STDP update to apply
rewards and punishments. R-STDP has shown effectiveness on simple image recognition datasets (such as MNIST [24]) with SNNs combining unsupervised STDP and supervised R-STDP, but has not been evaluated on complex datasets such as CIFAR-10 [25]. More recently, Supervised STDP (SSTDP) [22] proposes a method to modulate both the polarity and intensity of the STDP update based on temporal neuron error. The rule enables effective learning in deep networks (using global communication) and has demonstrated very good performance on various image recognition datasets, including CIFAR-10. However, it has not yet been investigated in SNNs incorporating unsupervised STDP.

In this paper, we focus on the supervised training of a spike-based classification layer with temporal decision-making in a two-layer SNN. The SNN, illustrated in Figure 1, comprises a feature extraction layer trained with unsupervised STDP and a classification layer trained with supervised STDP. The main contributions of this paper include the following:

1. To train the classification layer, we propose Stabilized Supervised STDP (S2-STDP), a supervised STDP learning rule where the polarity and the intensity of the updates are modulated by an error signal. In this rule, neurons update their weights to align their spikes with desired timestamps derived from the average firing time within the layer. S2-STDP is adapted from SSTDP.
2. To further enhance the learning capabilities of our S2-STDP classification layer, we introduce a training architecture called Paired Competing Neurons (PCN). This architecture associates each class with paired neurons and encourages neuron specialization on target or non-target samples through intra-class competition.
3. We evaluate the performance of S2-STDP and PCN on three image recognition datasets of growing complexity: MNIST, Fashion-MNIST, and CIFAR-10.

The remainder of this paper is organized as follows. Section II provides the necessary background information about the SNN employed in this study. Section III outlines some issues encountered with SSTDP with our SNN architecture, which serve as the basis for our contribution. Section IV presents a detailed description of our spike-based classification layer and our proposed training methods. Section V covers our results on image recognition datasets and provides an in-depth investigation of the key characteristics of our methods. Section VI concludes the paper. The source code is publicly available at: [https://gitlab.univ-lille.fr/gaspard.goupy/snn-pcn](https://gitlab.univ-lille.fr/gaspard.goupy/snn-pcn).

## II Background

### _Neural coding_

Since spiking neurons communicate through spikes, encoding the image is a necessary step before the SNN can process it. Two main coding schemes exist in the literature [26]: rate coding and temporal coding. Rate coding represents the information in the neuron firing rate, whereas temporal coding represents the information through the precise timing of the spike. Hence, temporal coding is more efficient in terms of energy consumption, making it particularly suitable for hardware implementation on low-power devices. In this work, we use a temporal scheme called latency coding [27], formulated as:

\[t\left(x\right)=T_{\max}-x \tag{1}\]

where \(x\) is the normalized pixel value, \(t(x)\) the encoded timestamp, and \(T_{\max}\) is the maximum firing time, set to 1. Here, each pixel is represented by a single spike, which constrains the number of generated spikes.

### _Neuron model_

In our SNN, we use the Single-Spike Integrate-and-Fire (SSIF) model [28], describing IF neurons that can fire at most

Fig. 1: Architecture of the SNN employed in this paper. First, the image is preprocessed and encoded into timestamps using the latency coding scheme. Then, a Convolutional SNN (CSNN) trained with unsupervised STDP is used to extract relevant features from the image. The resulting feature maps are compressed through a max-pooling layer to reduce their size and provide invariance to translation on the input. Lastly, they are flattened and fed to a fully-connected SNN trained with a supervised adaptation of STDP for classification. Each output neuron is associated with a class and the first one to fire predicts the label. The SNN training is done in a layer-wise fashion.
once per sample. The neurons integrate input spikes to their membrane potential \(V\) as follows:

\[V_{j}(t)=V_{j}(t-1)+\sum_{i}w_{ij}\cdot S_{i}(t-1) \tag{2}\]

where \(t\) is the timestamp, \(V_{j}\) is the membrane potential of the output neuron \(j\), \(S_{i}\) is the spike of the input neuron \(i\) (0 or 1), and \(w_{ij}\) is the weight of the synapse. When the membrane potential of a neuron exceeds a defined threshold \(V_{\text{th}}\), the neuron emits a spike, its membrane potential is reset, and the neuron is deactivated until the next sample is shown.

### _Unsupervised feature extraction layer_

Our SNN comprises a feature extraction layer trained with STDP to improve image representation before classification. This layer, illustrated in Figure 1, is based on the CSNN described in [13]. It consists of a trainable convolutional layer to extract spatial features from input images. Then, the feature maps are compressed with a max-pooling layer to reduce their size and provide translation invariance. The convolutional layer is entirely trained before the training of the classification layer starts.

Synaptic weights of the convolutional layer are updated with the multiplicative STDP rule [29], which can be described as:

\[\Delta w_{ij}=\begin{cases}A^{+}\times\exp\left(-\beta\,\frac{w_{ij}-w_{\min} }{w_{\max}-w_{\min}}\right),&\text{if }t_{j}\geq t_{i}\\ A^{-}\times\exp\left(-\beta\,\frac{w_{\max}-w_{ij}}{w_{\max}-w_{\min}}\right),&\text{if }t_{j}<t_{i}\end{cases} \tag{3}\]

where \(w_{ij}\) represents the weight of the synapse connecting input neuron \(i\) and output neuron \(j\), \(\Delta w_{ij}\) the weight change, \(t_{i}\) the firing timestamp of the neuron \(i\), \(\beta\) the saturation factor, \(w_{\min}\) and \(w_{\max}\) the minimum and maximum weight values, \(A^{+}\) and \(A^{-}\) the learning rates. In addition, a WTA mechanism is employed among neurons to promote the learning of distinct patterns, where only the first neuron to fire updates its weights with STDP.

Neuron thresholds in the convolutional layer are updated through two adaptation rules [13]. The first one is applied to control the timestamp at which neurons should fire:

\[V_{\text{th}_{i}}=\max\left(\text{Th}_{\min},V_{\text{th}_{i}}-\eta_{\text{th} }\left(t_{i}-t_{\text{target}}\right)\right) \tag{4}\]

where \(V_{\text{th}_{i}}\) represents the threshold of neuron \(i\), \(\text{Th}_{\min}\) the minimum threshold, \(\eta_{\text{th}}\) the learning rate, \(t_{i}\) and \(t_{\text{target}}\) the actual and target firing timestamp of the neuron. The second one is applied to ensure homeostasis (i.e. fair competition among neurons) in the layer:

\[\Delta V_{\text{th}_{i}} =\begin{cases}\ \ \ \eta_{\text{th}}\text{ if }t_{i}=\min\left\{t_{0}, \cdots,t_{N}\right\}\\ -\frac{\eta_{\text{th}}}{N}\text{ o.w.}\end{cases} \tag{5}\] \[V_{\text{th}_{i}} =\max\left(\text{Th}_{\min},V_{\text{th}_{i}}+\Delta V_{\text{ th}_{i}}\right)\]

where \(N\) is the number of neurons in competition.

## III Problem statement

In the introduction, we highlight some of the recent supervised adaptations of STDP that apply to spike-based classification. Among them, SSTDP [22] is a rule with state-of-the-art performance in diverse visual tasks. SSTDP incorporates neuron error to modulate the STDP updates in temporally-encoded SNNs. In the output layer and for each sample, desired time ranges are computed for target and non-target neurons based on the average firing time \(T_{\text{mean}}\) in the layer. For target neurons, the time range corresponds to \([0,T_{\text{mean}}-t_{1}]\) and for non-target neurons, it corresponds to \([T_{\text{mean}}+t_{2},1]\) (with \(t_{1}\), \(t_{2}\) defined time gaps). The error of a neuron is measured by the temporal difference to reach its time range. Although SSTDP is designed for multi-layer SNNs, we consider the method as a good starting point (given its reported performance) for training the classification (or output) layer of our SNN. To this end, we conducted preliminary experiments with SSTDP on the MNIST [24] and the Fashion-MNIST [30] datasets. It led us to identify two primary issues that we aim to resolve in this work:

1. the limited number of STDP updates;
2. the saturation of firing timestamps toward the maximum firing time.

First, SSTDP incorporates the concept of temporal error and desired time ranges. During the training process and for each sample, if a neuron fires in its desired range, the error is clipped and the neuron weights are not updated. We believe that the desired time ranges defined by SSTDP are too broad and lack sufficient constraints, resulting in a limited number of updates. Figure 2 illustrates the average update ratio per epoch in the classification layer, calculated by dividing the number of updates by the possible number of updates (in an epoch).

For both datasets, this update ratio hardly exceeds \(30\)% in the initial epoch, implying that neurons do not update their weights for approximately \(70\)% of the training samples. As the number of epochs increases, the ratio progressively decreases to \(7\)% for MNIST and \(19\)% for Fashion-MNIST, but the accuracy continues to improve, indicating the ongoing necessity of training. This results in a highly inefficient training process as a significant portion of the samples undergoes computational processing within the SNN without producing weight updates.

Second, because negative updates are more frequent than positive updates, neurons are continually pushed to fire later. It creates a saturation effect where their firing timestamps

Fig. 2: Average update ratio and validation accuracy per epoch in the classification layer trained with SSTDP. The training process of SSTDP is inefficient as it results in a very limited number of updates per epoch.
rapidly approach the maximum firing time. This behavior is illustrated in Figure 3.

As observed, the average firing time grows rapidly and then stabilizes within a range close to the maximum firing time. During the last training epoch, we compute an average firing time of \(0.98\pm 0.008\) for MNIST and \(0.99\pm 0.009\) for Fashion-MNIST. As a result, we believe that the SSTDP update method affects the ability of the SNN to separate classes since the firing timestamps saturate toward the maximum time and show limited variance.

## IV Methods

In this paper, we first propose S2-STDP for training a spike-based classification layer with temporal decision-making. S2-STDP employs error-modulated weight updates to reach desired timestamps derived from the average firing time within the layer. Then, we introduce the PCN training architecture, which further enhances the learning capabilities of our classification layer trained with S2-STDP. PCN is based on intra-class competition and does not introduce any additional hyperparameters.

### _Supervised classification layer_

The classification layer is the output layer of our SNN. In this work, we employ a fully-connected architecture composed of \(N\) output neurons, receiving input spikes from the flattened feature maps of the CSNN, as illustrated in Figure 1. The purpose of the classification layer is to predict the class of the input image. To do so, each output neuron \(j\) is associated with a specific class label \(c_{j}\) and the prediction of the SNN \(\hat{y}\) is defined as:

\[\hat{y} =c_{j^{*}} \tag{6}\] \[j^{*} =\operatorname{argmin}\left\{t_{0},\cdots,t_{N}\right\}\]

where \(t_{j}\) denotes the firing timestamp of the neuron \(j\). In essence, the neuron with the earliest firing timestamp predicts the class label. This method is appealing as it eliminates the need to propagate the entire encoded input for inference, which reduces computation time and the number of generated spikes.

### _Supervised STDP training_

To optimize the synaptic weights of the classification layer, we propose an error-modulated supervised STDP learning rule named S2-STDP. This rule trains target neurons to fire at early timestamps and non-target neurons to fire at late timestamps. During the training process, each sample triggers the weight update of output neurons according to Equation 7:

\[\Delta w_{ij}=\begin{cases}e_{j}\times A^{+}\times\exp\left(-\beta\frac{w_{ij }-w_{\min}}{w_{\max}-w_{\min}}\right),&\text{if }t_{j}\geq t_{i}\\ e_{j}\times A^{-}\times\exp\left(-\beta\frac{w_{\max}-w_{ij}}{w_{\max}-w_{ \min}}\right),&\text{if }t_{j}<t_{j}\end{cases} \tag{7}\]

where \(w_{ij}\) represents the weight of the synapse connecting input neuron \(i\) and output neuron \(j\), \(\Delta w_{ij}\) the weight change, \(t_{i}\) the firing timestamp of the neuron \(i\), \(e_{j}\) the error of the neuron \(j\), \(\beta\) the saturation factor, \(w_{\min}\) and \(w_{\max}\) the minimum and maximum weight values, \(A^{+}\) and \(A^{-}\) the learning rates.

The error plays a crucial role in guiding the learning process by modulating the polarity and intensity of the STDP update. In the context of temporal learning, the error of an output neuron \(j\) is defined as:

\[e_{j}=\frac{t_{j}-T_{j}}{T_{\max}} \tag{8}\]

where \(t_{j}\) and \(T_{j}\) respectively represent the actual and desired firing timestamps, and \(T_{\max}\) denotes the maximum firing time. To compute the desired firing timestamps, we introduce a method described in Equation 9, adapted from SSTDP [22]:

\[T_{j}=\begin{cases}T_{\mathrm{mean}}-\frac{n-1}{n}g,&\text{if }c_{j}=y\\ T_{\mathrm{mean}}+\frac{1}{n}g,&\text{if }c_{j}\neq y\end{cases} \tag{9}\]

where \(T_{\mathrm{mean}}\) represents the average firing time in the layer, \(n\) is the number of neurons, \(c_{j}\) is the neuron label, \(y\) is the sample label, \(g\) is the time gap hyperparameter that determines the desired distance from \(T_{\mathrm{mean}}\). Specifically, in our adaptation, we remove the error clipping from the original rule, which prevents neurons from updating their weights if their firing timestamp is lower (when they match the input sample class, otherwise higher) than their computed desired timestamp. In other words, SSTDP defines desired time ranges \(\left[0,T_{\mathrm{mean}}-\frac{n-1}{n}g\right]\) and \(\left[T_{\mathrm{mean}}+\frac{1}{n}g,1\right]\) whereas our adaption defines desired timestamps. Therefore, neurons can undergo weight updates in both directions, regardless of their associated class. For instance, if a neuron that does not correspond to the input sample class fires after its desired timestamp, it will receive a positive weight update to promote earlier firing. Conversely, if a neuron that does match the input sample class fires before its desired timestamp, it will receive a negative weight update to promote later firing. With our adaptation, we aim to drastically increase the number of updates per epoch (to nearly \(100\)%) and reduce the saturation of firing timestamps toward the maximum firing time. This is achieved by forcing

Fig. 3: Average firing time for some classes against the number of training samples, in the classification layer trained with SSTDP. The firing timestamps tend to saturate toward the maximum firing time.
neurons to fire around the average firing time, thereby ensuring its stabilization.

In addition to STDP, we propose a heterosynaptic plasticity model [14, 31, 32] to regulate changes in synaptic weights. After each update of an output neuron \(j\), its weights are normalized as follows:

\[w_{ij}=w_{ij}\cdot\frac{f_{\mathrm{norm}}}{\sum_{k}w_{kj}} \tag{10}\]

where \(w_{ij}\) represents the weight of the synapse with input neuron \(i\), \(f_{\mathrm{norm}}\) is the normalization factor, computed as the sum of weights of neuron \(j\) at initialization. This normalization ensures that all neurons maintain a constant and similar weight average throughout the learning process, allowing them equal chances of activation regardless of the number of weight updates they have undergone.

### _Paired Competing Neurons_

S2-STDP involves training each neuron to alternate its firing timestamp between two separate objectives, depending on the class of the input sample (see Equation 9). If the input sample class matches the neuron class, the neuron receives a target weight update, teaching it to spike just before \(T_{\mathrm{mean}}\). If the input sample class does not match the neuron class, the neuron receives a non-target weight update, teaching it to spike right after \(T_{\mathrm{mean}}\). Hence, the neurons must adapt their weights to satisfy both requirements, which can prevent them from learning too specific patterns. However, S2-STDP also has the advantage of stabilizing the average firing time of the neurons in the layer. To further enhance the learning capabilities of the layer trained with S2-STDP, we propose the PCN training architecture, described in Figure 4.

In this method, each class is associated with a pair of output neurons (instead of a single neuron) that are connected with lateral inhibition to promote intra-class competition. Within each pair and for every sample, the first neuron to spike, called the winner, inhibits the other one, called the loser, and undergoes the weight update process. The purpose of PCN is to encourage neuron specialization on one of the two firing objectives: target or non-target. In other words, for a pair of neurons \(n_{1}\) and \(n_{2}\) associated with the class \(c\), when we present samples of class \(y=c\), we want \(n_{1}\) to learn to spike at the target desired timestamp (\(T_{\mathrm{mean}}-\frac{n-1}{n}g\)) and win the competition against \(n_{2}\) (\(t_{n_{1}}\leq t_{n_{2}}\)). Conversely, when we present samples of class \(y\neq c\), we want \(n_{2}\) to learn to spike at the non-target desired timestamp (\(T_{\mathrm{mean}}+\frac{1}{n}g\)) and win the competition against \(n_{1}\) (\(t_{n_{2}}\leq t_{n_{1}}\)). Note that the neuron order \(n_{1}\) and \(n_{2}\) is arbitrary as we do not assign an objective to each neuron at initialization: this behavior emerges through intra-class competition. By encouraging specialization on one

Fig. 4: Classification layer equipped with Paired Competing Neurons (PCN) and trained with supervised STDP. Each class is represented by paired neurons, where the first neuron to spike (the winner) inhibits the other one (the loser). Only the winners undergo STDP updates. The difference \(\Delta t\) between the desired and actual firing timestamps is used to compute the neuron error, which modulates the intensity and the polarity of the STDP update. The purpose of PCN is to reduce training constraints arising from desired firing timestamps by encouraging specialization on one type of sample, such that one neuron learns to win the competition for samples of its class, whereas the other neuron learns to win the competition for samples of other classes.
of the two objectives, we reduce training constraints on the neuron weights.

It is important to mention that the use of PCN offers notable other advantages. First, thanks to lateral inhibition, it does not create more S2-STDP updates per sample, as only the winners are updated. Second, it is straightforward to implement in an existing SNN as it only requires attaching a new neuron to each output neuron. Third, it does not introduce any additional hyperparameters. Last, it can be used with any other learning rule involving two objective timestamps.

## V Results

### _Experimental setup_

#### V-A1 Datasets

We select three image recognition datasets of ten classes, with growing complexity: MNIST [24], Fashion-MNIST [30], and CIFAR-10 [25]. Both MNIST and Fashion-MNIST consist of \(28\times 28\) grayscale images. They contain 60,000 samples for training and 10,000 samples for testing. We preprocess the images with on-center/off-center coding to extract edge information [13]. CIFAR-10 is composed of \(32\times 32\) RGB images, 50,000 for the training set and 10,000 for the test set. We preprocess the images with a hardware-friendly whitening method presented in [33]. Note that CIFAR-10 is challenging for STDP-based SNNs and only a limited number of studies have considered this dataset thus far [14, 16, 34]. All the preprocessing methods are used with their original hyperparameters, provided in Supplementary Material (Section SuppM-I).

#### V-A2 SNN model

The SNN model used in this work is presented in Figure 1. The feature extraction layer is trained with unsupervised STDP whereas the classification layer is trained with a supervised adaptation of STDP. For each dataset, we compare S2-STDP with two existing methods, R-STDP [18] and SSTDP [22], both of which we have implemented. We consider three configurations of the feature extraction layer with increasing numbers of filters: CCNN-16, CCNN-64, and CCNN-128. Within a given dataset, these configurations share the same hyperparameters, except for the number of filters, allowing us to analyze the performance of the learning rule across various input sizes of the classification layer. Note that unless specified, the experiments are conducted on the CCNN-128 configuration.

#### V-A3 Protocol

We divide our experimental protocol into two phases: hyperparameter optimization and evaluation.

During the hyperparameter optimization phase, a subset of the training set is used as a validation set, which is created by randomly selecting, for each class, a percentage \(p_{\mathrm{val}}\) of its samples. We then apply the gridsearch algorithm to optimize the hyperparameters of the SNN based on the validation accuracy obtained. Hence, we ensure that the hyperparameters are not optimized for the test set. In this work, only the hyperparameters of the classification layer are optimized with gridsearch. The hyperparameters of the extraction layer are manually chosen based on preliminary experiments. For each dataset, all the rules are optimized on the CCNN-128 configuration. Then, the optimal hyperparameters are used with CCNN-64 and CCNN-16, except for the firing threshold that is divided by 2 and 4, respectively (as the input size decreases). All the hyperparameters are provided in Supplementary Material (Section SuppM-I).

During the evaluation phase, we use the K-fold cross-validation strategy (for the training set, as the test set is fixed) and change the seed for each fold. This allows us to assess the performance of the SNN with varying training and validation data, as well as different parameter initialization. In both phases, we employ an early stopping mechanism (with a patience \(p_{\mathrm{stop}}\)) during training to prevent overfitting. In all our following experiments, we choose \(p_{\mathrm{stop}}=10\), \(K=10\) and \(p_{\mathrm{val}}=\frac{1}{K}\) (i.e. \(10\)% of the training sets are used for validation).

### _Accuracy comparison_

In this section, we present a comparative analysis of the accuracy performance of two existing methods, R-STDP and SSTDP, and our methods, S2-STDP and S2-STDP+PCN. The average test accuracy achieved by each method on the MNIST, Fashion-MNIST, and CIFAR-10 datasets is presented in Table I.

Across all datasets and CSNN configurations, S2-STDP outperforms SSTDP. This demonstrates that our adaptation for computing the desired firing timestamps is more effective. Also, SSTDP exhibits poor and unstable results when operating with small input sizes (i.e. output sizes of the CSNN, cf. columns CCNN-16 in Table I). On the Fashion-MNIST dataset, it even leads to training divergence in certain runs, as evidenced by the high standard deviation of \(9.16\) ppt. This behavior suggests that SSTDP is unstable when confronted with varying input sizes, which can be undesirable, considering the current limitations of neuromorphic hardware in terms of network size. Note that this behavior is influenced by the

\begin{table}
\begin{tabular}{|c|c|c|c|} \hline Rule & CSNN-16 & CSNN-64 & CSNN-128 \\ \hline R-STDP (N=200) & \(96.05\pm 0.51\) & \(97.47\pm 0.16\) & \(97.88\pm 0.13\) \\ \hline R-STDP (N=20) & \(61.23\pm 27.74\) & \(93.34\pm 0.66\) & \(93.28\pm 0.87\) \\ \hline SSTDP & \(81.12\pm 2.87\) & \(95.84\pm 0.18\) & \(96.37\pm 0.09\) \\ \hline S2-STDP & \(95.40\pm 0.21\) & \(97.19\pm 0.09\) & \(97.81\pm 0.05\) \\ \hline
**S2-STDP+PCN** & \(\mathbf{97.02\pm 0.14}\) & \(\mathbf{95.21\pm 0.07}\) & \(\mathbf{97.89\pm 0.06}\) \\ \hline \hline SVM & \(98.4\pm 0.06\) & \(98.84\pm 0.05\) & \(98.93\pm 0.04\) \\ \hline \end{tabular} 
\begin{tabular}{|c|c|c|c|} \hline Rule & CSNN-16 & CSNN-64 & CSNN-128 \\ \hline R-STDP (N=200) & \(82.30\pm 0.26\) & \(82.30\pm 0.92\) & \(83.26\pm 0.22\) \\ \hline R-STDP (N=20) & \(76.50\pm 0.52\) & \(76.73\pm 0.13\) & \(77.01\pm 0.22\) \\ \hline SSTDP & \(51.22\pm 9.16\) & \(79.68\pm 1.67\) & \(84.12\pm 1.11\) \\ \hline S2-STDP & \(82.23\pm 0.40\) & \(84.92\pm 0.24\) & \(85.88\pm 0.22\) \\ \hline
**S2-STDP+PCN** & \(\mathbf{83.89\pm 0.40}\) & \(\mathbf{85.84\pm 0.19}\) & \(\mathbf{87.12\pm 0.21}\) \\ \hline \hline SVM & \(87.63\pm 0.17\) & \(89.02\pm 0.23\) & \(89.3\pm 0.21\) \\ \hline \end{tabular} 
\begin{tabular}{|c|c|c|c|} \hline Rule & CSNN-16 & CSNN-64 & CSNN-128 \\ \hline
**R-STDP (N=200)** & \(\mathbf{51.55\pm 1.23}\) & \(\mathbf{61.85\pm 0.61}\) & \(\mathbf{65.56\pm 0.38}\) \\ \hline R-STDP (N=20) & \(38.51\pm 4.41\) & \(51.74\pm 0.93\) & \(54.02\pm 0.80\) \\ \hline SSTDP & \(42.57\pm 1.38\) & \(57.89\pm 0.24\) & \(61.34\pm 0.14\) \\ \hline S2-STDP & \(47.94\pm 0.49\) & \(58.24\pm 0.27\) & \(61.53\pm 0.16\) \\ \hline S2-STDP+PCN** & \(49.23\pm 0.56\) & \(59.58\pm 0.16\) & \(62.81\pm 0.
hyperparameters. When tuning SSTDP specifically for the Fashion-MNIST CCNN-16 configuration, we can reach an accuracy of \(80.10\)% instead of \(51.22\)%, indicating that the rule can still deal with small input sizes but does not provide great robustness against hyperparameters. The use of PCN further improves S2-STDP effectiveness without requiring additional hyperparameters. In comparison to the other considered STDP-based methods, S2-STDP+PCN achieves the highest accuracy on the MNIST and Fashion-MNIST datasets. Specifically, compared to SSTDP on CCNN-128, it exhibits an accuracy improvement of \(2.22\) ppt on MNIST, \(3\) ppt on Fashion-MNIST, and \(1.47\) ppt on CIFAR-10.

It is worth noting that R-STDP significantly outperforms all the STDP-based methods on CIFAR-10. However, R-STDP requires 200 output neurons to achieve this performance whereas S2-STDP+PCN only uses 20 neurons. When R-STDP is used with 20 output neurons, it performs significantly worse than all other methods on all datasets and CCNN configurations. This observation highlights the importance of error-modulated weight updates in enabling effective supervised training with STDP. In our case, it leads to a substantial reduction in the number of trainable parameters by a factor of 10. Also, on Fashion-MNIST, R-STDP obtains relatively low performance and fails to extract more relevant features when the number of feature maps increases. The difference in performance improvement between CCNN-16 and CCNN-128 with R-STDP is only about \(0.96\) ppt, whereas it is about \(3.23\) ppt with S2-STDP+PCN. Lastly, although the Support Vector Machine (SVM) gets the highest accuracy, it is not compatible with neuromorphic hardware.

### _Stabilization of the output firing timestamps_

In Section III, we elaborated on an issue of SSTDP: the saturation of firing timestamps toward the maximum firing time. Our proposed S2-STDP is specifically designed to address this issue by encouraging neurons to fire at exact desired timestamps instead of time ranges. In this section, we evaluate its impact on the output firing timestamps of our SNN.

First, in Figure 5, we compare the average firing time in the classification layer obtained on training samples at each epoch by the different SSTDP-based methods. As previously illustrated in Figure 3, the average firing time of SSTDP is very close to the maximum firing time. On the contrary, our proposed methods successfully reduce the saturation of firing timestamps. In addition, they obtain a significantly higher average firing time variance between the samples over an epoch. For instance, at epoch 20, we measure a standard deviation of \(0.008\) for SSTDP and \(0.049\) for S2-STDP. This indicates that the average firing time tends to vary more between samples with S2-STDP, which can enable better class separation.

Then, we study the impact of integrating a PCN architecture with S2-STDP. Figure 6 illustrates the average firing time of output neurons trained with and without PCN, on MNIST test samples. We notice that neurons trained with a PCN architecture are better at reaching the desired timestamps, particularly for the non-target neurons. This is because, through intra-class competition, PCN successfully creates neuron specialization on one type of sample (target or non-target). For instance, when we provide samples from class 7 to the pair associated with this class, neuron 2 is used to win the competition against neuron 1. Conversely, when we provide samples from class 8 to the pair associated with class 7, neuron 1 is used to win against neuron 2. Furthermore, intra-class competition permits to reduce training constraints, as the non-target losers (i.e. losers when we present samples of class different from theirs) do not have to reach an exact desired timestamp. As observed, their firing timestamps are much higher compared to the desired timestamp for non-target winning neurons (red line in the figure). Consequently, the synaptic weights of the neurons trained with a PCN architecture are more specialized on samples linked to the assigned objective.

### _Robustness against the time gap hyperparameter_

During training, the time gap hyperparameter \(g\) is used to define the distance between the desired timestamps and the average firing time. Selecting an appropriate value for this hyperparameter is crucial to ensure accurate class separation. However, hyperparameter tuning can be a time-consuming task, and achieving the optimal value may not always be feasible. In this section, we investigate the influence of the time gap value on the accuracy.

Figure 7 compares SSTDP and our proposed methods across various values of \(g\) on the three datasets. S2-STDP demonstrates greater robustness compared to SSTDP regarding the choice of \(g\), as it significantly expands the range of values that can achieve near-optimal accuracy. The accuracy curve of S2-STDP exhibits a more pronounced bell-shaped pattern with a larger plateau near the maximum. This implies that tuning \(g\) can be easier with S2-STDP. When considering a suitable range for \(g\), S2-STDP always achieves higher accuracy than SSTDP on MNIST and Fashion-MNIST datasets. On CIFAR-10, the behavior observed differs slightly, but the low accuracy arising from the complexity of the task may hinder accurate interpretation. The use of PCN as a training architecture for S2-STDP almost always improves its performance. This suggests that the efficacy of PCN is not dependent on a specific value of \(g\). Note that all the methods use their respective gridsearch-optimized hyperparameters.

Fig. 5: Average firing time in the classification layer against the epoch. The standard deviation measures the variance over the samples during an epoch. Our methods using S2-STDP significantly reduce the saturation of firing timestamps toward the maximum firing time.
### _Robustness against the hyperparameter set_

Our PCN training architecture improves the accuracy of S2-STDP across all evaluated datasets, as indicated in Section V-B. However, the hyperparameters of the classification layer are individually optimized for S2-STDP and S2-STDP+PCN, suggesting that PCN might only enhance performance in specific hyperparameter sets. In this section, we present evidence that PCN can effectively improve the accuracy of S2-STDP irrespective of the hyperparameters used. Specifically, we aim to demonstrate that implementing PCN on an existing SNN trained with S2-STDP is likely to always yield improved results.

Figure 8 illustrates the comparison of S2-STDP accuracy with and without PCN across different hyperparameter sets on the Fashion-MNIST dataset. The varying hyperparameters include the firing threshold (\(\mathrm{thr}\)), time gap (\(g\)), and learning rates (\(\mathrm{ap}\) and \(\mathrm{am}\)). Their values are selected within a suitable range to achieve satisfactory performance. The results show that integrating PCN improves S2-STDP accuracy consistently, with an average improvement of \(0.94\) ppt and a maximum improvement of \(1.57\) ppt. For comparison, when independently optimizing the hyperparameters for both methods, the measured accuracy improvement is \(1.24\) ppt. Although the observed improvement may not be substantial, the use of PCN as a training architecture enhances the accuracy of S2-STDP regardless of the hyperparameter set and does not introduce any additional hyperparameters. In the Supplementary Material (Section SuppM-II-B), we present similar plots for all the datasets, accompanied by a comparison with SSTDP. The results exhibit consistency with the analysis conducted on the Fashion-MNIST dataset. Furthermore, SSTDP yields considerably lower accuracy for various configurations, once again showing its poor robustness against hyperparameters.

Fig. 6: Neuron average firing time in the classification layer on MNIST test samples. Each point represents a neuron, and the row denotes its associated class. The points with lower alpha are the inhibited neurons (i.e. losers). The green and red lines correspond to the average desired timestamps of the target neuron and the non-target neurons, respectively. Through intra-class competition, the use of PCN architecture creates neuron specialization which helps them reach their desired firing timestamp.

Fig. 7: Accuracy of the different SSTDP-based learning rules against the time gap hyperparameter. Our proposed methods using S2-STDP enable better robustness against the time gap value.
### _Comparison with the literature_

In Table II, we present an accuracy comparison between our partially supervised SNN, trained with STDP + S2-STDP+PCN, and other state-of-the-art algorithms employed for training SNNs. For this comparison, we focused on supervised methods, primarily STDP-based and BP-based with local updates.

On the MNIST dataset, our proposed SNN outperforms the STDP-based approaches and achieves results that are close to the non-local BP-based state of the art. Similarly, for the Fashion-MNIST dataset, our SNN surpasses the STDP-based approaches with similar architectures and demonstrates competitive performance compared to most of the other methods. It is important to note that only the output layer of our SNN is trained with supervision, whereas all the other methods in Fashion-MNIST are fully supervised. For instance, in [23], they employ a 6-layer SNN trained with global feedback + STDP, achieving an accuracy of \(89.05\)%. In contrast, we employ a 2-layer partially supervised CSNN, resulting in an accuracy loss of only \(1.93\) ppt. Additionally, our results are highly competitive with a 2-layer SNN trained using a BP-based algorithm, with a loss of \(0.88\) ppt.

However, when it comes to the CIFAR-10 dataset, the performance of our method remains significantly low compared to BP-based state-of-the-art SNNs. This dataset is particularly challenging for shallow architectures with only one supervised layer. We believe that the features extracted by our unsupervised layer are not distinguishable enough to allow

\begin{table}
\begin{tabular}{c c c c} \hline \hline Dataset & Architecture & Learning rule & Local learning & Accuracy \\ \hline MNIST & **20CS-P2-50CS-P3-200FC-10FC**[35] & **STDP pretraining + BP** & **no** & **99.28** \\  & 300FC-10FC [22] & SSTDP & no & \(98.10\) \\  & 500FC-150FC-10FC [20] & BP-STDP & no & \(97.20\pm 0.07\) \\  & C5-1000FC-10FC [36] & BRP & yes & \(99.01\) \\  & 500FC-500FC-10FC [37] & EMSTD & yes & \(97.30\) \\  & 30C5-P2-2250CS-P3-200C5-P5 [18] & STDP + R-STDP & yes & \(97.20\) \\  & _128C5-P4-20FC (Ours)_ & _STDP + S2-STDP+PCN_ & yes & \(98.59\pm 0.06\) \\ \hline Fashion-MNIST & 1000FC-10FC [38] & BP & no & \(88.00\) \\  & **20CS-P2-4005-P2-100FC-10FC**[39] & **STIDi-BP** & **yes** & **92.80** \\  & 200FC-200FC-200FC-200FC-200FC-10FC**[23] & Global feedback + STDP & yes & \(89.05\) \\  & 500FC-500FC-10FC [37] & EMSTDP & yes & \(86.10\) \\  & 6400FC-10FC [21] & Sym-STDP & yes & \(85.31\pm 0.16\) \\  & _128C5-P4-20FC (Ours)_ & _STDP + S2-STDP+PCN_ & yes & \(87.12\pm 0.21\) \\ \hline CIFAR-10 & **VEGF-7**[22] & **SSTDP** & **no** & **91.31** \\  & 64C7-P8-512FC-512FC-10FC [14] & STDP + ANN-based BP & no & \(71.20\) \\  & 256C3-P2-1024FC-10FC [16] & Probalistic STDP + ANN-based BP & no & \(66.23\) \\  & 16C5-P2-8C3-P2-100FC-10FC [34] & EMSTDP & yes & \(64.40\) \\  & C5-1000FC-10FC [36] & BRP & yes & \(57.08\) \\  & _128C5-P4-20FC (Ours)_ & _STDP + S2-STDP+PCN_ & yes & _62.81 \(\pm\) 0.15_ \\ \hline \hline \end{tabular}
\end{table} TABLE II: Accuracy comparison of our proposed SNN with the literature. Accuracies come from the original papers.

Fig. 8: Accuracy of S2-STDP with and without the use of PCN architecture across different hyperparameter sets, on Fashion-MNIST test samples. PCN always improves S2-STDP performance, without introducing any hyperparameters.
for an accurate analysis. The SNNs employed in [14, 16] share similarities with ours, as they consist of an unsupervised feature extraction layer trained with STDP and a supervised module. However, they employ a two and three-layer ANN for classification, while we employ a single-layer SNN, explaining the accuracy gap measured. When compared to another shallow SNN as in [36], our approach still demonstrates significantly superior performance.

As a concluding remark, it is important to mention that the fairness of comparisons may vary, as some authors report top-1 accuracy, and some do not provide details on their evaluation methodology nor how hyperparameters have been tuned.

## VI Conclusion

In this paper, we proposed Stabilized Supervised STDP (S2-STDP), a supervised STDP learning rule for training a spike-based classification layer with temporal decision-making. This layer is the output of a convolutional SNN (CSNN) equipped with unsupervised STDP for feature extraction. Our learning rule integrates error-modulated weight updates that align neuron spikes with desired timestamps derived from the average firing time within the layer. S2-STDP has been specifically developed to address the issues observed with SSTDP. To further enhance the learning capabilities of the classification layer trained with S2-STDP, we introduced a training architecture called Paired Competing Neurons (PCN). PCN associates each class with paired neurons connected via lateral inhibition and encourages neuron specialization through intra-class competition. We evaluated our S2-STDP and PCN on three image recognition datasets of growing complexity: MNIST, Fashion-MNIST, and CIFAR-10. Our experiments showed that:

1. Our methods outperform current supervised STDP-based state of the art, for comparable architectures and numbers of neurons.
2. S2-STDP successfully addressed the issues of SSTDP concerning the limited number of STDP updates and the saturation of firing timestamps toward the maximum firing time.
3. Our methods exhibited improved hyperparameter robustness as compared to SSTDP.
4. The PCN architecture enhances the performance of S2-STDP, regardless of the hyperparameter set.

In the future, we plan to expand S2-STDP to multi-layer architectures, while maintaining local computation required for on-chip learning. This includes exploring both feedback connexions [23] and local losses [10].

## Acknowledgements

This work is funded by Chaire Luxant-ANVI (Metropole de Lille) and supported by IRCICA (CNRS UAR 3380). We would like to thank Benoit Miramond for the helpful exchange.
