Improving Stability and Performance of Spiking Neural Networks through
  Enhancing Temporal Consistency
Spiking neural networks have gained significant attention due to their
brain-like information processing capabilities. The use of surrogate gradients
has made it possible to train spiking neural networks with backpropagation,
leading to impressive performance in various tasks. However, spiking neural
networks trained with backpropagation typically approximate actual labels using
the average output, often necessitating a larger simulation timestep to enhance
the network's performance. This delay constraint poses a challenge to the
further advancement of SNNs. Current training algorithms tend to overlook the
differences in output distribution at various timesteps. Particularly for
neuromorphic datasets, inputs at different timesteps can cause inconsistencies
in output distribution, leading to a significant deviation from the optimal
direction when combining optimization directions from different moments. To
tackle this issue, we have designed a method to enhance the temporal
consistency of outputs at different timesteps. We have conducted experiments on
static datasets such as CIFAR10, CIFAR100, and ImageNet. The results
demonstrate that our algorithm can achieve comparable performance to other
optimal SNN algorithms. Notably, our algorithm has achieved state-of-the-art
performance on neuromorphic datasets DVS-CIFAR10 and N-Caltech101, and can
achieve superior performance in the test phase with timestep T=1.
Dongcheng Zhao, Guobin Shen, Yiting Dong, Yang Li, Yi Zeng
Improving Stability and Performance of Spiking Neural Networks through Enhancing Temporal Consistency

###### Abstract

Spiking neural networks have gained significant attention due to their brain-like information processing capabilities. The use of surrogate gradients has made it possible to train spiking neural networks with backpropagation, leading to impressive performance in various tasks. However, spiking neural networks trained with backpropagation typically approximate actual labels using the average output, often necessitating a larger simulation timestep to enhance the network's performance. This delay constraint poses a challenge to the further advancement of SNNs. Current training algorithms tend to overlook the differences in output distribution at various timesteps. Particularly for neuromorphic datasets, inputs at different timesteps can cause inconsistencies in output distribution, leading to a significant deviation from the optimal direction when combining optimization directions from different moments. To tackle this issue, we have designed a method to enhance the temporal consistency of outputs at different timesteps. We have conducted experiments on static datasets such as CIFAR10, CIFAR100, and ImageNet. The results demonstrate that our algorithm can achieve comparable performance to other optimal SNN algorithms. Notably, our algorithm has achieved state-of-the-art performance on neuromorphic datasets DVS-CIFAR10 and N-Caltech101, and can achieve superior performance in the test phase with timestep T=1.

## 1 Introduction

Spike neural networks (SNNs), inspired by biological neural systems, exhibit unique advantages in processing spatiotemporal data. SNNs represent and transmit information through sparse, discrete spike sequences, which not only improves energy efficiency but also allows for better integration with neuromorphic chips [1], attracting the attention of researchers from various fields [2]. However, the binary method of information transmission, which is similar to that of the human brain, possesses non-differentiable characteristics. This makes it challenging to directly apply the backpropagation algorithm to the training of Spiking Neural Networks (SNNs), consequently posing a significant obstacle in training deep SNNs.

Some researchers have attempted to incorporate brain-inspired learning principles into the modeling process of spiking neural networks, such as spike timing dependent plasticity (STDP) [3] and short
term plasticity [4]. Although these methods have addressed the training issues of SNNs to some extent, they still have limitations in dealing with more complex structures and tasks. Currently, there are two main strategies for achieving high-performance deep SNNs: one is the conversion-based approach [5; 6; 7; 8], and the other is the backpropagation (BP)-based approach [9; 10; 11; 12]. The conversion-based method uses the activations of a pre-trained artificial neural network (ANN) as the firing rate of the SNN, allowing for a nearly lossless conversion from an ANN to an SNN. However, since this method requires firing rates to simulate activation values in ANNs accurately, it necessitates longer simulation steps and cannot fully exploit the spatiotemporal information processing capabilities of SNNs. On the other hand, the backpropagation-based training algorithm uses surrogate functions to approximate the gradients of the spike firing function, making it possible to train deep SNNs. Nevertheless, this approach also has limitations, such as gradient vanishing or exploding when dealing with more complex structures.

In contrast to ANNs that output results directly, SNNs need to accumulate outputs over multiple timesteps to make accurate judgments. However, this requires SNNs to use longer simulation steps to model more precise information, resulting in greater computational resources when deploying on edge devices. The latency caused by large simulation length has also become an important factor hindering the development of SNNs. When SNNs receive inputs at different timesteps, especially for neuromorphic data with rich spatiotemporal characteristics, different output patterns are generated at different timesteps. Currently, the training of spiking neural networks based on the backpropagation algorithm mainly optimizes the difference between the average membrane potential or spike firing rate of the output layer and the accurate labels. Then, it uses the output at different timesteps to guide network training in the backward process, which does not consider the difference in output distribution under different timesteps. During training, the combined optimization direction of inconsistent outputs at different moments conflicts with the optimization direction of the average output value, thus preventing the network from being effectively optimized. During testing, the variance in the distribution at different moments can result in the network's test results being offset by incorrect inference results at certain moments, thereby reducing overall performance. Therefore, this paper proposes a new method to enhance the temporal consistency (ETC) at different timesteps during the training process, making the training of spiking neural networks more stable. At the same time, we have verified the method on multiple datasets, and the results show that the method has achieved outstanding performance on all datasets. The main contributions of this paper are as follows:

* Through theoretical analysis, this paper reveals the limitations of existing backpropagation methods in the training of SNNs, especially in dealing with the differences in output distribution between different moments.
* We propose a novel method that enhances the temporal consistency across different moments, which improves the stability of SNN training and significantly reduces the required timesteps.
* To validate the superiority of our proposed method, we have conducted experiments on the static datasets CIFAR10, CIFAR100, and ImageNet and the neuromorphic datasets DVS-CIFAR10 and N-Caltech101. The results show that our algorithm achieves the best performance on neuromorphic datasets and delivers competitive performance compared to other state-of-the-art algorithms on static datasets.

## 2 Related Work

Researchers mainly improve the BP-based SNNs from the following two aspects. First, they tried to enhance the information transmission capability of spiking neural networks. A series of studies [13; 14; 15; 16; 17] focused on introducing learnable parameters, such as membrane potential constants and thresholds, into spiking neurons to construct different types of spiking neurons. This approach enhances the adaptability of neural networks to various information types. NeuNorm [18], tdBN [19], and TEBN [12] incorporated various normalization techniques to enhance information transmission in deep spiking neural networks. These approaches effectively mitigate gradient vanishing and explosion issues, improving network performance and stability. In the works by [20; 21; 22], various attention mechanisms were introduced across different dimensions to guide SNNs in transmitting more task-relevant information, as a result of this enhancing their overall performance and adaptability.

Another group of researchers attempted to improve the structure of SNNs. Drawing inspiration from brain-inspired structures, LISNN [23] introduced lateral connections, while BackEISNN [24]
incorporated self-feedback connections to enhance SNN's information processing capabilities. Furthermore, SewResNet [25] designed a more suitable residual module for SNNs, providing a simple method for training deep SNNs. Spikformer [26] combined self attention with SNNs, constructing a high-performance SNN structure. AutoSNN [27] and NASSNN [28] employed the neural architecture search technique to design more optimal structures for SNNs.

However, the works above all used the average membrane potential or spike firing rate at the output layer for prediction without considering the impact of the output distribution of SNNs at different timesteps on performance. Reducing the differences in SNN output distribution at various timesteps is crucial for constructing a stable and high-performance SNNs.

## 3 Method

In this section, we first introduce the spiking neurons used and theoretically analyze the issue of uneven distribution at different timesteps. Ultimately, we introduce the enhancing temporal consistency constraint to standardize the output distribution at different timesteps.

### Spiking Neuron Model

The leaky integrate-and-fire (LIF) neuron, as the most commonly used neuron model in deep spiking neural networks, describes the complex dynamics of biological neurons with a relatively simple differential equation as shown in Eq. 1:

\[\begin{split}\tau_{m}\frac{dV}{dt}&=-V+RI\quad V \leq V_{th}\\ S&=H(V-V_{th})\end{split} \tag{1}\]

\(V\) represents the membrane potential, \(\tau_{m}=RC\) is the membrane potential time constant, \(I=\sum_{j}W_{ij}S_{j}\) denotes the input current obtained by aggregating presynaptic spikes, \(W\) denotes the connection weight from pre-synaptic to post-synaptic neurons, and \(H\) refers to the step function for spike emission. When the membrane potential surpasses the threshold \(V_{th}\), the neuron emits a spike \(S\) and resets to the resting potential \(V_{r}\). In this study, we set \(V_{th}\) to 0.5, \(\tau_{m}\) to 2, \(R\) to 1, and the resting potential \(V_{r}\) to 0. By using the first-order Euler method, we obtain the discretized representation of the above differential equation as shown in Eq. 2:

\[V_{t+1}=(1-\frac{1}{\tau_{m}})V_{t}+\frac{1}{\tau_{m}}I_{t} \tag{2}\]

Figure 1: The whole training pipeline of our model. SNNs receive input data at various timesteps and generate corresponding outputs at each timestep. The ETC constraint helps ensure output distribution consistency across different timesteps.
In order to use the backpropagation algorithm for network training, we employ surrogate gradients to approximate the gradients of the spike firing function, as follows:

\[\frac{\partial H}{\partial V_{t}}=\left\{\begin{aligned} 0,& |V_{t}-V_{th}|>\frac{1}{a}\\ -a^{2}|V_{t}-V_{th}|+a,&|V_{t}-V_{th}|\leq\frac{1}{a} \end{aligned}\right. \tag{3}\]

\(a\) is a hyperparameter used to control the shape of the surrogate gradient. In this study, we set \(a\) to 2.

### The Training and Inference Procedures compared SNNs with ANNs

Unlike traditional artificial neural networks, SNNs adjust their weights during training based on outputs at multiple timesteps. During the inference of the network, the final output is calculated by weighting the outputs at various timesteps. For deep SNNs, the most widely used approach is to employ the average membrane potential of the neurons in the last layer as the final output, resulting in more efficient and accurate inference.

We represent the average membrane potential of the last layer as \(O_{\text{mean}}=\frac{1}{T}\sum_{t=1}^{T}V_{t}\), where T is the total number of timesteps and \(V_{t}\) is the membrane potential at timestep \(t\). The softmax function is applied to convert the average membrane potential into output probabilities \(P_{\text{mean}}=\text{softmax}(O_{\text{mean}})\). During the network training process, we use the cross-entropy loss to minimize the difference between the actual outputs \(P_{\text{mean}}\) and expected outputs \(y\) as shown in Eq. 4.

\[L_{CE}=-\sum_{i=1}^{C}y_{i}\cdot\log(P_{\text{mean}}) \tag{4}\]

C is the number of categories. By applying the chain rule, we can compute the gradient of the loss function concerning the network parameters \(W\).

\[\frac{\partial L_{CE}}{\partial W}=\frac{\partial L_{CE}}{\partial O_{\text{ mean}}}\frac{\partial O_{\text{mean}}}{\partial V_{t}}\frac{\partial V_{t}}{ \partial W}=\frac{1}{T}\sum_{t=1}^{T}[P_{\text{mean}}-y]\frac{\partial V_{t}} {\partial W} \tag{5}\]

For ANNs, we use \(O\) to denote the final output and \(P\) to represent the probability distribution after the softmax operation. Using the same loss function, we can also obtain the gradient of the loss concerning the network parameters:

\[\frac{\partial L_{CE}}{\partial W}=\frac{\partial L_{CE}}{\partial O}\frac{ \partial O}{\partial W}=[P_{\text{mean}}-y]\frac{\partial O}{\partial W} \tag{6}\]

As shown in Eq. 5 and Eq. 6, the final output determines the partial derivative of the weight. In minimizing the loss, the output \(O\) of the ANN will gradually approach the actual label. At this time, using the partial derivative of \(O\) concerning W will more accurately control the optimization direction of the ANN. However, the optimization direction of the weights in SNN depends on the direction

Figure 2: The training and inference procedure compared our method with the traditional methods. For traditional methods, consistency in the distribution across different timesteps can cause discrepancies in the optimization direction and lead to misjudgments during inference.
of the partial derivative of the output \(V_{t}\) concerning the weight at each moment. SNN optimizes the distance between the average membrane potential and the actual label. As shown in Fig. 2, when the optimization directions at different moments are inconsistent or even significantly different, there will be a severe mismatch between \(\sum_{t=1}^{T}\frac{\partial V_{t}}{\partial W}\) and \(\frac{\partial\sum_{t=1}^{T}V_{t}}{\partial W}\), which significantly interferes with the optimization direction of the spiking neural network. Simultaneously, as shown on the right side of Fig. 2, during the inference phase of the network, inconsistencies in the distribution at different timesteps can lead to overall network results being skewed by erroneous results, thereby leading to a decline in performance.

### Reducing Temporal Diversity Procedure

As discussed above, the performance degradation of SNNs is due to the mismatch of output distributions at different timesteps. In order to enhance the consistency between different timesteps, we propose an enhancing temporal consistency constraint, aiming to make the distributions at each timestep as similar as possible. First, we define the output probability distribution at each timestep \(P_{t}^{i}\) as shown in Eq. 7:

\[P_{t}^{i}(V_{t};\tau)=softmax(V_{t}^{i};\tau)=\frac{exp(V_{t}^{i}/\tau)}{\sum _{j}exp(V_{t}^{j}/\tau)} \tag{7}\]

The temperature parameter \(\tau\) controls the smoothness of the model's output distribution, which is more conducive to learning the relationships between different categories [29]. Here we set \(\tau=4\). After obtaining the output distributions at different timesteps, we aim to minimize the distribution gap between the output \(P_{t}\) at time t and the outputs at other timesteps, here we use the Kullback-Leibler (KL) divergence. The loss function is shown in Eq. 8.

\[\begin{split} L_{ETC}^{t}&=\frac{1}{T-1}\sum_{m=1, m\neq t}^{T}KL(P_{m}||P_{t})=\frac{1}{T-1}\sum_{m=1,m\neq t}^{T}\sum_{i=1}^{C}P_{m}^{ i}log\frac{P_{m}^{i}}{P_{t}^{i}}\\ &=\frac{1}{T-1}\sum_{m=1,m\neq t}^{T}\sum_{i=1}^{C}(P_{m}^{i}logP _{m}^{i}-P_{m}^{i}logP_{t}^{i})\end{split} \tag{8}\]

To avoid model collapse, we do not propagate the gradient through \(P_{m}\). As a result, the final loss can be represented as shown in Eq. 9:

\[L_{ETC}^{t}=-\frac{1}{T-1}\sum_{m=1,m\neq t}^{T}\sum_{i=1}^{C}P_{m}^{i}logP_{ t}^{i}\Rightarrow L_{RTD}=-\frac{1}{T}\frac{1}{T-1}\sum_{t=1}^{T}\sum_{m=1,m \neq t}^{T}\sum_{i=1}^{C}P_{m}^{i}logP_{t}^{i} \tag{9}\]

Thus, the final loss function can be written as a dynamic combination of cross-entropy loss and ETC loss, as shown in Eq. 10.

\[L_{all}=L_{CE}+\lambda\tau^{2}L_{RTD} \tag{10}\]

\(\lambda\) is the weight constraint term to control the influence of ETC loss on overall loss. Here we set \(\lambda=1\). We can obtain the partial derivative of the total loss concerning the weights \(\frac{\partial L_{all}}{\partial W}\):

\[\begin{split}\frac{\partial L_{all}}{\partial W}&= \frac{\partial L_{CE}}{\partial W}+\lambda\tau^{2}\frac{\partial L_{ETC}}{ \partial W}\\ &=\frac{1}{T}\sum_{t=1}^{T}[P_{\text{mean}}-y]\frac{\partial V_{ t}}{\partial W}+\lambda\tau^{2}\frac{1}{T}\frac{1}{T-1}\sum_{t=1}^{T}\sum_{m=1,m \neq t}^{T}(P_{t}-P_{m})\frac{\partial V_{t}}{\partial W}\end{split} \tag{11}\]

As shown in Eq. 11, the first term ensures that the average output is close to the actual target, while the second term ensures the consistency of the output distribution at each timestep. Combining these two loss terms can make the output at each moment as accurate as possible, thus better guiding the optimization direction of the output for the weights at each moment. By using the output of different timesteps to approximate each other and correcting erroneous prediction moments, this combination of loss functions can also prevent overconfident predictions, thereby further improving the model's
generalization ability on the test dataset. Moreover, this constraint can be considered a self-distillation process of the model. It uses the dark knowledge of the model's output at other timesteps to provide soft labels for each timestep, optimizing the output at each timestep and fully utilizing the temporal information of SNNs.

## 4 Experiments

In order to demonstrate the superiority of our algorithm, we conduct experiments on multiple datasets, including static datasets such as CIFAR-10 [30], CIFAR-100 [31], and ImageNet [32], as well as neuromorphic datasets like DVS-CIFAR10 [33] and N-Caltech101 [34]. We develop the SNN code based on the open-source framework BrainCog with NVIDIA A100 graphic processing unit (GPU), employing the AdamW optimizer with a weight decay setting of 0.0001. The learning rate is set to 0.001 with the cosine annealing strategy. The batch size is set to 128. In our experiments, we set the total training epochs to 600. The experiments are repeated five times randomly. We have reported the mean and standard deviation of the corresponding performance.

As shown in Tab. 1, we conduct experiments on static datasets based on SEW-ResNet and compare them with other state-of-the-art algorithms. We use the SEW-ResNet18 structure and perform experiments on the CIFAR10 and CIFAR100 datasets under simulation step lengths of 2, 4, 6, and 8, respectively. On the CIFAR10 dataset, our model achieves the accuracy of 95.73% at the simulation step length of 6, surpassing all other algorithms and achieving the current best performance. For the CIFAR100 dataset, even with a relatively small network, we achieve performance comparable to the current best algorithm, only 0.4% lower than the performance of TEBN. For the more complex ImageNet dataset, we conduct experiments based on SEW-ResNet-18 and SEW-ResNet-34. The results show that compared with the GLIF algorithm, the performance of our model has improved by 0.6%. What's more, compared with the original SEW algorithm, our performance has improved by about 2%. This fully demonstrates the superiority of our algorithm.

### Results on Neuromorphic Datasets

Compared to static datasets, neuromorphic datasets reveal richer spatiotemporal features, thereby better highlighting the advantages of spiking neural networks. The DVS-CIFAR10 and N-Caltech101 datasets convert original image information into event information through dynamic vision sensors. In our study, we first resize the input samples to a fixed size of 48\(\times\)48 and adopt the VGGNN structure used in TET. As shown in Tab. 2, our algorithm has achieved the best performance on both datasets. On the DVS-CIFAR10 dataset, our performance improved by 2% compared to TET and by 0.8% compared to TEBN. On the N-Caltech101 dataset, we has achieved the accuracy of 85.53%, a 1.4% improvement compared to TKS. Meanwhile, our performance has significantly surpassed that of the EventMix and NDA algorithms, which use data augmentation techniques. Due to the characteristic of neuromorphic datasets where inputs at different times vary, the output distribution also changes. In such cases, introducing the method we propose in this paper can better standardize the membrane potential distribution at different timesteps, thereby significantly enhancing the network's performance.

### Ablation Studies

To verify the effectiveness of our algorithm, we conduct ablation experiments on the DVS-CIFAR10 and N-Caltech101 datasets. As shown in Tab. 3, without adding the ETC module, the accuracy of the N-Caltech101 dataset is only 78.28%. However, after adding the ETC module with \(\lambda=1\) and \(\tau=4\), the network performance improves to 83.3%, an increase of 5%. Under the same settings, the performance of the DVS-CIFAR10 dataset is also improved by 2.5%. It is worth noting that the hyperparameter \(\tau\) controls the smoothness of the output at different moments. If \(\tau\) is too large, the output will be too smooth to show significant differences. Conversely, if \(\tau\) is too small, the output will be overly confident and unable to reflect the relationships between different categories. Furthermore, the \(\lambda\) parameter controls the influence of the ETC module on the final loss. If \(\lambda\) is too large or too small, it cannot effectively guide the loss function to update the weights. To better illustrate the impact of these hyperparameters, we test the performance of the ETC algorithm under different hyperparameter settings.
### Temporal Consistency Verification

In addition to performance, latency is a crucial factor that constrains the development of SNNs. When the output distribution at each moment becomes more consistent, we can achieve higher accuracy in the testing phase using only a shorter simulation length. We have validated the results on the DVS-CIFAR10 and N-Caltech101 datasets. During the training phase, we use the simulation timestep T=10, while in the testing phase, we conduct separate experiments for different timesteps. As shown in Fig. 3, with a simulation step size of 1, the accuracy of the conventional algorithm is only 43.3% for the DVS-CIFAR10 dataset. In contrast, our ETC algorithm achieves an accuracy of 63.1%, an improvement of about 20%. At a step size of 5, the ETC algorithm already surpasses the performance of the traditional algorithm at a step size of 10, significantly reducing the network's latency. For the N-Caltech101 dataset, when the step size is 1, our algorithm has improved by 16% compared to the

\begin{table}
\begin{tabular}{c c c c c} \hline
**Dataset** & **Model** & **Architecture** & **Simulation Step** & **Accuracy** \\ \hline \multirow{10}{*}{CIFAR10} & Opt [35] & ResNet-18 & 4 & 90.43 \\  & CSTDB [36] & ResNet-20 & 250 & 92.22 \\  & Diet-SNN [37] & ResNet-20 & 10 & 92.54 \\  & NeuNorm [18] & CIFARNet & 12 & 90.53 \\  & TSSL-BP [38] & CIFARNet & 5 & 91.41 \\  & BPSTA [39] & 7-layer-CNN & 8 & 92.15 \\  & NASNN [28] & NAS & 5 & 92.73 \\  & AutoSNN [27] & NAS & 16 & 93.15 \\  & tdBN [19] & ResNet-19 & 6 & 93.16 \\  & PLIF [13] & PLIFNet & 8 & 93.5 \\  & TET [11] & ResNet-19 & 6 & 94.50 \\  & GLIF [15] & ResNet-19 & 6 & 95.03 \\  & TKS [40] & ResNet-19 & 4 & 95.3 \\  & Rec-Dis [41] & ResNet-19 & 6 & 95.55 \\  & TEBN [12] & ResNet-19 & 6 & 95.60 \\ \cline{2-5}  & \multirow{4}{*}{**Our Method**} & SEW-ResNet-18 & 2 & 94.65 \(\pm\) 0.08 \\  & & SEW-ResNet-18 & 4 & 95.4 \(\pm\) 0.07 \\  & & SEW-ResNet-18 & 6 & 95.73 \(\pm\) 0.02 \\  & & SEW-ResNet-18 & 8 & 95.84 \(\pm\) 0.03 \\ \hline \multirow{10}{*}{CIFAR100} & Diet-SNN[37] & ResNet-20 & 5 & 64.07 \\  & BPSTA [39] & ResNet34 & 8 & 69.32 \\  & AutoSNN [27] & NAS & 16 & 69.16 \\  & NASNN [28] & NAS & 5 & 73.04 \\  & TET [11] & ResNet-19 & 6 & 74.72 \\  & Rec-Dis [41] & ResNet-19 & 4 & 74.10 \\  & TKS [40] & ResNet-19 & 4 & 76.2 \\  & GLIF [15] & ResNet-19 & 6 & 77.35 \\  & TEBN [12] & ResNet-19 & 6 & 78.76 \\ \cline{2-5}  & \multirow{4}{*}{**Our Method**} & SEW-ResNet-18 & 2 & 75.96 \(\pm\) 0.24 \\  & & SEW-ResNet-18 & 4 & 77.65 \(\pm\) 0.13 \\  & & SEW-ResNet-18 & 6 & 78.25 \(\pm\) 0.11 \\  & & SEW-ResNet-18 & 8 & 78.32 \(\pm\) 0.07 \\ \hline \multirow{10}{*}{ImageNet} & tdBN [19] & Spiking-ResNet-34 & 6 & 63.72 \\  & SEW [25] & SEW-ResNet-34 & 4 & 67.04 \\ \cline{1-1}  & Rec-Dis [41] & ResNet-34 & 6 & 67.33 \\ \cline{1-1}  & TET [11] & SEW-ResNet-34 & 4 & 68.00 \\ \cline{1-1}  & TEBN [12] & SEW-ResNet-34 & 4 & 68.28 \\ \cline{1-1}  & GLIF [15] & ResNet-34 & 6 & 69.09 \\ \cline{1-1}  & TKS [40] & SEW-ResNet-34 & 4 & 69.6 \\ \cline{1-1} \cline{2-5}  & \multirow{4}{*}{**Our Method**} & SEW-ResNet-18 & 4 & 63.70 \\ \cline{1-1}  & SEW-ResNet-34 & 4 & 68.54 \\ \cline{1-1}  & SEW-ResNet-34 & 6 & 69.64 \\ \hline \end{tabular}
\end{table}
Table 1: Compare with existing works on static image datasets.
baseline. At a step size of 5, the accuracy of the ETC algorithm has reached 80.57%, which is 2% higher than the baseline at a step size of 10.

Meanwhile, we have visualized the outputs at different timesteps in the final layer. As shown in Fig. 4, we visualize the distribution of the outputs at different timesteps for the samples in the DVS-CIFAR10 dataset. It can be observed that the output distributions of the traditional algorithm vary significantly at different timesteps. For instance, the lsample's output distribution is accurate in the early moments, but from the sixth moment onwards, the distribution begins to fluctuate. This led to a sample that should have been categorized as 8 being incorrectly predicted as 6, 8, 1, 8, 6. In contrast, the output distribution of the ETC algorithm is much more consistent. By enhancing the consistency of the temporal distribution, the ETC algorithm has dramatically improved the accuracy of the overall distribution of outputs at each moment. This allows us to achieve high-precision predictions in the testing phase with fewer simulation steps. This characteristic greatly facilitates the deployment of SNNs on various edge devices.

## 5 Conclusion

Unlike artificial neural networks, spiking neural networks can receive and generate inputs and outputs at multiple moments. However, the inconsistency of output distributions at different moments often requires longer simulation steps to produce stable and precise outputs. This seriously affects the

\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Dataset** & **Model** & **Architecture** & **Simulation Step** & **Accuracy** \\ \hline \multirow{8}{*}{DVS-CIFAR10} & tdBN [19] & ResNet-19 & 10 & 67.8 \\  & LIAF [42] & LIAF-Net & 10 & 71.70 \\  & IAF [42] & LIAF-Net & 10 & 70.40 \\  & AutoSNN [27] & NAS & 16 & 72.50 \\  & BPSTA [39] & 5-layer-CNN & 16 & 78.95 \\  & Rec-Dis [41] & ResNet-19 & 10 & 72.42 \\  & TET [11] & VGGSNN & 10 & 83.17 \\  & TEBN [12] & VGGSNN & 10 & 84.90 \\  & TKS [40] & VGGSNN\({}^{*}\) & 10 & 85.3 \\ \cline{2-5}  & **Our Method** & VGGSNN & 10 & 85.35 \(\pm\) 0.40 \\ \hline \multirow{8}{*}{N-Caltech101} & ConvertSNN [43] & VGG11 & 20 & 55.0 \\  & Dart [44] & N/A & N/A & 66.8 \\ \cline{1-1}  & TCJA [21] & TCJAnet & 14 & 78.5 \\ \cline{1-1}  & NDA [45] & ResNet-19 & 10 & 78.6 \\ \cline{1-1}  & EventMix [46] & ResNet-18 & 10 & 79.5 \\ \cline{1-1}  & TKS [40] & VGGSNN\({}^{*}\) & 10 & 84.1 \\ \cline{1-1} \cline{2-5}  & **Our Method** & VGGSNN & 10 & 83.33 \(\pm\) 0.41 \\ \cline{1-1} \cline{2-5}  & **Our Method** & VGGSNN\({}^{*}\) & 10 & 85.53 \(\pm\) 0.09 \\ \hline \hline \end{tabular}
\end{table}
Table 2: Compare with existing works on neuromorphic datasets. \({}^{*}\) indicates using a model with tdBN [19].

\begin{table}
\begin{tabular}{c c c c c c c c} \multicolumn{8}{c}{DVS-CIFAR10} & \multicolumn{3}{c}{N-Caltech101} \\ \hline \hline \(\lambda=0\). & \multicolumn{3}{c}{82.9} & \multicolumn{3}{c}{78.28} \\ \hline \hline \(\tau\)\(\lambda\)\(\lambda\) & 0.1 & 1 & 2 & 8 & 0.1 & 1 & 2 & 8 \\ \hline
1 & 82.20 & 85.10 & 84.50 & 84.50 & 80.11 & 80.92 & 80.00 & 80.69 \\
2 & 85.70 & 85.90 & 84.60 & 84.40 & 81.84 & 82.13 & 80.57 & 80.00 \\
4 & 85.30 & 85.40 & 85.60 & 85.30 & 81.72 & 83.33 & 81.72 & 81.60 \\
8 & 85.60 & 85.00 & 85.10 & 84.40 & 81.95 & 82.03 & 80.92 & 80.34 \\
16 & 85.40 & 85.00 & 85.30 & 84.60 & 81.38 & 81.95 & 81.95 & 81.03 \\ \hline \hline \end{tabular}
\end{table}
Table 3: Ablation analysis on ETC loss on DVS-CIFAR10 and N-Caltech101 datasets, as well as the sensitivity analysis of different hyperparameters
performance of spiking neural networks and increases their latency. In this study, we propose a strategy to enhance temporal consistency, aiming to reduce the inconsistency in the output distributions at different timesteps. The approach significantly improves the performance of spiking neural networks on multiple datasets and effectively reduces latency. We can achieve a high accuracy in the testing phase with only short simulation steps. In biological neurons, there exist efficient coding methods that adaptively encode diverse inputs, along with decoding methods that accurately interpret the output information from different moments [47; 48; 49]. In the future, we can take more inspiration from the biologically plausible encoding and decoding method to process the information more effectively.

Figure 4: The comparison between the base method and our method on output distribution at different timesteps for the sample in DVS-CIAFR10

Figure 3: Test Accuracy in different timesteps on DVS-CIFAR10 and N-Caltech101, the model is trained at timestep 10.
## Acknowledgement

This work was supported by the National Key Research and Development Program (Grant No. 2020AAA0104305), and the Strategic Priority Research Program of the Chinese Academy of Sciences (Grant No. XDB32070100).
