Temporal Knowledge Sharing enable Spiking Neural Network Learning from
  Past and Future
Spiking Neural Networks (SNNs) have attracted significant attention from
researchers across various domains due to their brain-like information
processing mechanism. However, SNNs typically grapple with challenges such as
extended time steps, low temporal information utilization, and the requirement
for consistent time step between testing and training. These challenges render
SNNs with high latency. Moreover, the constraint on time steps necessitates the
retraining of the model for new deployments, reducing adaptability. To address
these issues, this paper proposes a novel perspective, viewing the SNN as a
temporal aggregation model. We introduce the Temporal Knowledge Sharing (TKS)
method, facilitating information interact between different time points. TKS
can be perceived as a form of temporal self-distillation. To validate the
efficacy of TKS in information processing, we tested it on static datasets like
CIFAR10, CIFAR100, ImageNet-1k, and neuromorphic datasets such as DVS-CIFAR10
and NCALTECH101. Experimental results demonstrate that our method achieves
state-of-the-art performance compared to other algorithms. Furthermore, TKS
addresses the temporal consistency challenge, endowing the model with superior
temporal generalization capabilities. This allows the network to train with
longer time steps and maintain high performance during testing with shorter
time steps. Such an approach considerably accelerates the deployment of SNNs on
edge devices. Finally, we conducted ablation experiments and tested TKS on
fine-grained tasks, with results showcasing TKS's enhanced capability to
process information efficiently.
Yiting Dong, Dongcheng Zhao, Yi Zeng
# Temporal Knowledge Sharing enable Spiking Neural Network

###### Abstract

Spiking neural networks have attracted extensive attention from researchers in many fields due to their brain-like information processing mechanism. The proposal of surrogate gradient enables the spiking neural networks to migrate to more complex tasks, and gradually close the gap with the conventional artificial neural networks. Current spiking neural networks utilize the output of all moments to produce the final prediction, which compromises their temporal characteristics and causes a reduction in performance and efficiency. We propose a temporal knowledge sharing approach (TKS) that enables the interaction of information between different moments, by selecting the output of specific moments to compose teacher signals to guide the training of the network along with the real labels. We have validated TKS on both static datasets CIFAR10, CIFAR100, ImageNet-1k and neuromorphic datasets DVS-CIFAR10, NCALTECH101. Our experimental results indicate that we have achieved the current optimal performance in comparison with other algorithms. Experiments on Fine-grained classification datasets further demonstrate our algorithm's superiority with CUB-200-2011, Stanford-Dogs, and StanfordCars. TKS algorithm helps the model to have stronger temporal generalization capability, allowing the network to guarantee performance with large time steps in the training phase and with small time steps in the testing phase. This greatly facilitates the deployment of SNNs on edge devices.

## 1 Introduction

The spiking neural networks (SNNs) are known as the third generation of artificial neural networks (ANNs) because of their brain-like information processing method. Due to their non-differential characteristics, the training of spiking neural networks has been extensively studied by researchers. Inspired by the learning mechanism of the brain, some researchers have introduced biologically plausible learning rules into the training of SNNs [14, 15]. Furthermore, the proposal of surrogate gradient enables the SNNs to be extended to more complex structures and tasks [21, 15], and has shown comparable performance to ANNs in many fields.

Researchers have attempted to enhance the performance of SNNs trained based on backpropagation from several perspectives. Some researchers improve the performance of the SNNs by improving the structure. For example, inspired by biological structures, LISNN [1] introduces lateral interactions in the convolutional SNNs, which greatly enhances the performance and robustness of the SNNs. Back-EISNN [14] is motivated by the autapse, and introduces self-feedback connections to facilitate and accelerate the training of SNNs. In addition, some works have been inspired by the classic structures in the field of ANNs, such as the ResNet-based [13], Transformer-based [14]. The representation ability of SNNs may also be improved by improving the dynamic characteristics of spiking neurons, such as the learnable mem

Figure 1: The inference process of spiking neural networks over time, the correct prediction during the process is not reasonably used.
brane time constant [14], adaptive thresholds [15]. However, the temporal properties of SNNs are not fully exploited in these works.

Spiking neural networks are mostly trained in long simulation time. After that, firing rates or average membrane potentials are used to determine the final prediction. This results in a substantial latency of SNNs and restricts the development of SNNs on edge hardware. In addition, the process of combining information from all of the moments in order to make the final prediction means that each of the moments contributes equally to the final result. During this process, the SNN's temporal information is not fully utilized. As shown in Fig. 1, where the SNN runs through six time steps and performs a five-class classification. The correct prediction is achieved at timestep 1 and 3 during the process. However, after merging all the timesteps, the final prediction is wrong. The information at the correct timestep does not assist the model in making a correct prediction, nor does it provide a positive contribution to subsequent model training.

With the aim of better leveraging the temporal information of spiking neural networks, we view the way that SNNs aggregate the results of multiple moments together to jointly determine prediction as an aggregation of models. The membrane potential distribution at the previous moment will be part of the model parameters and states at the next moment, and we expect each sub-model of the aggregated model to be sufficiently accurate. Considering that knowledge distillation can guide the training of low-performance student networks with the knowledge of high-performance teachers [13, 14], we train the network with guidance by knowledge transfer of the models to each other between different moments. The information from different moments is merged and represented as soft labels. Also, to prevent incorrect information transfer, we only integrate the output at the correct moment as the teacher signal. We expect that models at different moments can continuously modify their output through mutual interaction. Not only can the aggregated model obtain the true label information, but it can also receive knowledge information from other models at each moment, which is called temporal knowledge sharing (TKS). The temporal knowledge sharing greatly utilizes the temporal information during the training of SNNs, and has improved the performance of SNNs. Our contribution can be summarized as follows:

1. We propose a temporal knowledge sharing training strategy, which will extract the temporal information at the correct moment and combine them into the teacher signal to guide the learning process during training.
2. We have conducted experiments on several datasets to verify the superiority of our model. We have achieved state-of-the-art performance on not only the static datasets CIFAR10, CIFAR100 and ImageNet-1K, but also on the neuromorphic datasets DVS-CIFAR10 and NCALTECH101. We also conducted experiments on the fine-grained classification datasets CUB-200-2011, StanfordDogs, and StanfordCars, all of which show significant performance improvements compared to other state-of-the-art algorithms.
3. The temporal knowledge sharing enables to improve the accuracy of the model at each moment, which greatly enhances the stability of SNN training. Furthermore, it allows for high performance inference using only a few number of simulation steps, which facilitates deployment on edge devices in a convenient manner.

## 2 Related Work

Several works have done research on exploiting the temporal information of spiking neural networks. TCJA-SNN [15] assigns attention mechanisms to temporal channels by introducing additional networks. AttentionSNN [21] exploits temporal information by optimizing membrane potentials with attention weights. TET [1] supervises each moment individually to exploit temporal information. TEBN [13] assigns different weights to each moment by extending the batch normalization layer to the temporal dimension. BPSTA [2] designs the temporal residual connection to help the error propagate across the spikes in the temporal dimension. These efforts do not take full advantage of the information available in the network's output at the correct moment.

Knowledge distillation helps to transfer knowledge from a high performing model to a low performing model to obtain better results than the original [13]. Some SNNs guide the training of SNNs by introducing additional network knowledge from pre-trained ANNs [15] or pre-trained large SNNs [16]. They all require additional training of a larger network, which adds a lot of computational cost. Self-knowledge distillation [15, 14, 13] generates the teacher signal itself without additional network for computation. Inspired by self-knowledge distillation, we design to use the results generated by the network at different moments to guide the training of the network, and do not train an additional network, which greatly reduces the computational effort.

## 3 Method

### Spiking Neuron Model

The spiking neuron is the basic computing unit of the spiking neural network, which uses a differential equations to describe the dynamic behavior of biological neurons, as shown in Equation 1.

\[\begin{split}\tau_{m}\frac{dV_{t}}{dt}&=-V_{t}+I_{t }\quad V_{t}\leq V_{th}\\ S_{t}&=1,\quad V_{t}=V_{rest}\quad V_{t}\geq V_{th }\end{split} \tag{1}\]

By accumulating membrane potential over time, the neuron releases a spike after reaching the threshold \(V_{th}\), and re-sets to the resting potential \(V_{rest}\), which we set to 0 here. In order to facilitate the simulation calculation, we discretize Equation 1 to get Equation 2.

\[V[t]=(1-\frac{1}{\tau})V[t-1](1-S[t-1])+\frac{1}{\tau}I[t] \tag{2}\]
\(V[t]\) denotes the membrane potential at time \(t\), \(tau_{m}\) denotes the membrane potential constant, \(I[t]=\sum_{j}w_{ij}S_{j}\) denotes the input current obtained by collecting pre-synaptic neuron spikes, and \(S[t]\) denotes the spike at time \(t\).

### Temporal Knowledge Sharing

For the training of SNNs, the commonly used training method is to use the information of all moments as the final prediction, and construct a loss function with the true label, then perform error backpropagation. Here, we use the average membrane potential of the output layer as shown in Equation 3:

\[O=\frac{1}{T}\sum_{t=1}^{T}V_{t}^{out} \tag{3}\]

And we use the cross entropy loss:

\[L_{CE}=-\sum_{i=1}^{n}y_{i}log(p_{i}) \tag{4}\]

where \(p_{i}=\frac{e^{O_{i}}}{\sum_{j=1}^{n}e^{O_{j}}}\), and \(y_{i}\) is the label.

As shown in Equation 4, the model needs to accumulate the output of all moments to make the final prediction, which ignores the temporal characteristic of SNNs. For SNNs, the output at each moment contains rich information. Inspired by knowledge distillation, we consider the output at different moments in the model can be used to guide the learning of the network. As shown in the Figure 2, we collect the predictions generated at the correct moment of classification and use the mean value of these signals as the teacher signal \(Z\). If there is no correct signal in the prediction result of the model, we will adjust the prediction probability of the teacher signal to \(1/num_{category}\).

\[Z=\frac{1}{m}\sum_{t=1}^{m}V_{t}^{out} \tag{5}\]

We use the cross-entropy function to measure the gap between the student and the teacher output as shown in Equation 6:

\[L_{TKS}=-\sum_{i=1}^{n}Z_{i}log(q_{i}) \tag{6}\]

As the common operation in knowledge distillation, \(q_{i}\) is obtained by calculating the output through softmax with temperature, adding the temperature parameter to make the prediction smoother:

```
0: Training dataset D = \(\{(x_{i},y_{i})\}_{i=1}^{N}\), SNN model, Simulation Length,
0: A high-performance, low-latency and stable spiking neural network model;
1:for each mini-batch training data \(D_{i}=\{x_{i},y_{i}\}\)do
2: Compute the SNN output O as shown in Eq. 3
3: Collect the output membrane potential at the correct moment to make up the teacher signal \(Z\) as shown in Eq. 5
4: Calculate the final loss using Eq. 8
5: Adjust the synaptic weights through the spatial temporal backpropagation with surrogate gradients;
6:endfor
```

**Algorithm 1** Temporal Knowledge Sharing For Training Deep Spiking Neural Networks

\[q_{i}=\frac{exp(O_{i}/\tau)}{\sum_{j}exp(O_{j}/\tau)} \tag{7}\]

\(\tau\) denotes the temperature. Then the final loss function can be written as:

\[L_{all}=(1-\alpha)L_{CE}+\alpha\tau^{2}L_{TKS} \tag{8}\]

Figure 2: The whole training process of our temporal knowledge sharing. TKS will collect information at the right moment and compose additional teacher signals to guide the training of the network
\(\alpha\) is the balance coefficient, which adjusts the balance between loss of the predicted value and the true label, as well as the degree of the teacher signal. A larger \(\alpha\) indicates that the teacher signal is more critical for the weight adjustment of the model. Because the model has not been able to provide a more accurate supervised signal in the early stage of training, true labels are needed for the main guidance. Here we adopt a gradual ascent strategy that will initialize \(\alpha\) to a small value and increase linearly with training. Empirically, the initial value of \(\alpha\) is set to 0 and grows gradually with the training process, reaching its maximum value in the last epoch of the training process.

## 4 Experiment

To illustrate the superiority of our algorithm, we validated it on commonly used static datasets: CIFAR10, CIFAR100, ImageNet-1K as well as neuromorphic datasets: DVS-CIFAR10, NCALTECH101. To demonstrate the effectiveness of the algorithm, we further conducted experiments on fine-grained object recognition tasks: CUB-200-2011, Stanford Dogs, Stanford Cars. In the above experiments, we applied the surrogate function from [11].except for ImageNet-1K, AdamW trainer is applied for model optimization, with the weight decay set to 0.01. For ImageNet-1K, we used Adam optimizer, with the weight decay set to 0. The batch size for all experiments is set to 128. And the cosine strategy was used to control the learning rate.

### Evaluation metrics

**Accuracy:** Top1 accuracy and Top5 accuracy reflect the classification accuracy of the model on the corresponding dataset. Top1 represents the accuracy of the category corresponding to the highest probability, while Top5 represents the accuracy of the label category belonging to the top five probability categories of output.

**AURC (the Area Under the Risk-coverage Curve)**[1]: The AURC measures the area under the risk-coverage curve (RC) during the convergence process. It

\begin{table}
\begin{tabular}{l l l l|r} \hline \hline Dataset & Model & Method & Architecture & T & Accuracy(\%) \\ \hline \multirow{8}{*}{CIFAR10} & [10] & Hybrid training & ResNet-20 & 250 & 92.22 \\  & [10] & ANN2SNN & ResNet-20 & 2048 & 91.36 \\  & [11] & ANN2SNN & ResNet-20 & 128 & 93.56 \\  & [11] & STBP & CIFARNet & 12 & 89.83 \\  & [12] & DSpike & ResNet-18 & 4 & 93.66 \\  & [12] & TSSL-BP & CIFARNet & 5 & 91.41 \\  & [12] & STBP-tdBN & ResNet-19 & 4 & 92.92 \\  & [13] & TET* & ResNet-19 & 4 & 94.44 \\  & [14] & TEBN* & ResNet-19 & 4 & 95.58 \\ \cline{2-5}  & Ours & TKS & ResNet-19 & 4 & **95.30** \\  & Ours & TKS* & ResNet-19 & 4 & **96.35** \\ \cline{2-5}  & Ours & TKS* & SEW-ResNet-19 & 4 & **96.76** \\ \hline \multirow{8}{*}{CIFAR100} & [10] & Hybrid training & VGG11 & 125 & 67.87 \\  & [10] & Diet-SNN & ResNet-20 & 5 & 64.07 \\  & [15] & ANN2SNN & ResNet-20 & 2048 & 67.82 \\  & [16] & DSpike & ResNet-18 & 4 & 73.35 \\  & [13] & STBP-tdBN & ResNet-19 & 4 & 70.86 \\  & [13] & TET* & ResNet-19 & 4 & 74.47 \\  & [13] & TEBN* & ResNet-19 & 4 & 78.71 \\ \cline{2-5}  & Ours & TKS & ResNet-19 & 4 & **76.20** \\  & Ours & TKS* & SEW-ResNet-19 & 4 & **79.89** \\ \hline \multirow{8}{*}{ImageNet-1K} & [10] & Hybrid training & ResNet-34 & 250 & 61.48 \\  & [13] & SPIKE-NORM & ResNet-34 & 2500 & 69.96 \\ \cline{1-1}  & [13] & STBP-tdBN & Spiking-ResNet-34 & 6 & 63.72 \\ \cline{1-1}  & [13] & SEW ResNet & SEW-ResNet-34 & 4 & 67.04 \\ \cline{1-1}  & [13] & SEW ResNet & SEW-ResNet-50 & 4 & 67.78 \\ \cline{1-1}  & [13] & TET & SEW-ResNet-34 & 4 & 68.00 \\ \cline{1-1}  & [13] & TEBN & SEW-ResNet-34 & 4 & 68.28 \\ \cline{1-1} \cline{2-5}  & Ours & TKS & SEW-ResNet-34 & 4 & **69.60** \\ \cline{1-1}  & Ours & TKS & SEW-ResNet-50 & 4 & **70.70** \\ \hline \hline \end{tabular}
\end{table}
Table 1: Accuracy of static datasets: CIFAR10, CIFAR100, ImageNet-1K. Where * denotes data augmentation, The best performing model is indicated as boldface.
describes the quality of probabilistic predictions from the perspective of confidence estimation. It assesses whether correct and incorrect predictions can be separated well.

### Static Datasets

We validate our TKS algorithm on CIFAR10, CIFAR100 and ImageNet-1k, and compare it with the current state-of-the-art algorithms. For CIFAR10 and CIFAR100 [11], we use the ResNet-19 [22] structure as well as the SEW-ResNet-19 structure [4]. SEW-ResNet-19 improves the original ResNet, allowing residual learning to be performed more effectively in SNNs. As a fair comparison, we use SEW-ResNet-34 and SEW-ResNet-50 structures for the ImageNet-1k dataset. For CIFAR10 and CIFAR100, the temperature parameter \(\tau\) is set to 3. And for the ImageNet-1k dataset, \(\tau\) is set to 1. For all the experiments above, \(\alpha\) is increased from the initial value 0 to 0.7.

As shown in Table 1, for all datasets, our TSK method has achieved the best performance compared with other famous deep SNNs algorithms. compared with TET and TEBN, our TKS algorithm has a significant improvement on CIFAR10 and CIFAR100. Especially, for the ImageNet-1k dataset, we outperform them by 1.3% and 1.6% on the same network structure SEW-ResNet-34.

### Neuromorphic Datasets

Neuromorphic datasets are usually recorded by event cameras and the data consists of events. We conduct experiments on the DVS-CIFAR10 [11] as well as the NCAL-TECH101 [1] datasets, respectively. The VGG-SNN [4] model is adopted as the basic architecture. The temperature is set at 5 and \(\alpha\) grow from the initial value 0 to 0.3. As shown in Table 2, compared with other current best algorithms, TKS obtains the highest accuracy, achieving 85.3% on DVS-CIFAR10 and 84.1% on NCALTECH101, respectively.

### Fine-grained Classification Task

Generally, fine-grained datasets have similar features between categories [23], which are difficult to distinguish. In addition, it has a limited number of samples in each category. We conduct experiments on the CUB-200-2011[21], StanfordDogs [24], and StanfordCars [11] datasets and compare our

\begin{table}
\begin{tabular}{c l l l l r} \hline \hline Dataset & Method & Model & Architecture & T & Accuracy(\%) \\ \hline \multirow{8}{*}{DVS-CIFAR10} & [22] & STBP-tdBN & ResNet-19 & 10 & 67.8 \\  & [20] & Streaming Rollout & DenseNet & 10 & 66.8 \\  & [24] & Conv3D & LIAF-Net & 10 & 71.7 \\  & [21] & LIAF & LIAF-Net & 10 & 70.4 \\  & [21] & DSpike & ResNet-18 & 10 & 75.4 \\  & [21] & PLIF & CNN6 & 20 & 74.8 \\  & [23] & TCJA-SNN & VGGSNN & 10 & 80.7 \\  & [4] & TET & VGGSNN & 10 & 83.2 \\  & [4] & TEBN & VGGSNN & 10 & 84.9 \\ \cline{2-5}  & Ours & TKS & VGG-SNN & 10 & **85.3** \\ \hline \multirow{8}{*}{NCALTECH101} & [23] & TCJA-SNN & TCJAnet & 14 & 78.5 \\  & [23] & EventMixer & ResNet-18 & 10 & 79.5 \\ \cline{1-1}  & [21] & NDA & ResNet-19 & 10 & 78.6 \\ \cline{1-1}  & [24] & HATS & - & - & 64.2 \\ \cline{1-1}  & [24] & DART & - & - & 66.8 \\ \cline{1-1} \cline{2-5}  & Ours & TKS & VGG-SNN & 10 & **84.1** \\ \hline \hline \end{tabular}
\end{table}
Table 2: Accuracy of neuromorphic datasets: DVS-CIFAR10, NCALTECH101. The best result are shown in boldface.

\begin{table}
\begin{tabular}{c l l l|l l|l l} \hline \hline \multirow{2}{*}{Model} & \multirow{2}{*}{Method} & \multicolumn{2}{c}{CUB-200-2011} & \multicolumn{2}{c}{StanfordDogs} & \multicolumn{2}{c}{StanfordCars} \\ \cline{3-8}  & & top1 & top5 & top1 & top5 & top1 & top5 \\ \hline \multirow{2}{*}{
\begin{tabular}{c} SEW- \\ ResNet-18 \\ \end{tabular} } & Baseline & 37.94 & 63.55 & 51.54 & 78.10 & 68.59 & 88.96 \\  & TET & 40.49 & 65.53 & 52.14 & 78.69 & 68.52 & 88.56 \\  & TKS & **42.82** & **66.59** & **54.03** & **79.76** & **71.22** & **89.70** \\  & Baseline & 46.77 & 72.35 & 56.79 & **83.57** & 76.04 & 92.71 \\  & TET & 47.38 & 71.06 & 57.06 & 81.98 & 72.94 & 92.81 \\  & TKS & **51.71** & **75.28** & **59.06** & 82.82 & **76.94** & **92.95** \\ \hline \hline \end{tabular}
\end{table}
Table 3: Accuracy of fine-grained task datasets: CUB-200-2011, Stanford Dogs, Stanford Cars. The best result are shown in boldface. It demonstrates both top1 Acc and top5 Acc.
TKS with the baseline model and TET model. The temperature parameter \(\tau\) for the fine-grained task is 3, and \(\alpha\) grows from initial value 0 to 0.7. As shown in the Table 3, our TKS outperforms the baseline on all datasets and also exceeds the TET algorithm. Fine-grained tasks are usually used to measure the feature extraction ability of the model. And TKS uses the information of the distribution of the output category as the teacher signal, which will help the model to improve the classification ability of the categories with similar features.

## 5 Discussion

In this section, we will conduct the ablation analysis to show the superiority of our TKS. Then, we will conduct experiments to verify the generalization of TKS under different timesteps. Finally, we will visualize the feature embedding.

### Ablation analysis

To illustrate the effectiveness of the TKS algorithm, we conducted ablation experiments on the CIFAR10, CIFAR100, ImageNet-1k, DVS-CIFAR10, and NCALTECH101 datasets. Also, we will include LS [21] as well as TET [10] methods on the same baseline for fairness of comparison with existing methods. The LS method is a regularization method, which can also be viewed as being passed teacher information that follows a uniform distribution [12, 23]. Hence, we include the LS method in the comparison to ensure that the teacher signal delivered by TKS does indeed provide useful information and is not simply a regularization factor. The TET algorithm supervises each moment individually to exploit temporal information. As shown in Table 4, TKS has achieved the highest performance compared with the baseline and TET. Meanwhile, in order to illustrate the superiority of TKS, we evaluate the model from the perspective of confidence estimation. We calculate the AURC for each model. As shown in the Table 4, a lower AURC represents a higher quality of probabilistic prediction. TKS helps the model to improve in most cases.

### Robustness with Different Time Steps

In addition to the performance of the SNNs, another problem that restricts the development of SNNs is latency. SNNs often

\begin{table}
\begin{tabular}{c|c c c|c} \hline \hline Dataset & Model & Method & Top1 Acc(\%) & Top5 Acc(\%) & AURC (x\(10^{3}\)) \\ \hline \multirow{6}{*}{CIFAR10} & \multirow{6}{*}{ResNet19} & Baseline & 95.76 & **99.94** & 4.10 \\  & & LS & 95.99 & 99.63 & 6.60 \\  & & TET & 96.14 & 99.93 & 3.64 \\  & & TKS & **96.35** & 99.85 & **2.76** \\  & \multirow{6}{*}{SEW-ResNet-19} & Baseline & 96.54 & **99.94** & 3.50 \\  & & LS & 96.54 & 99.70 & 8.44 \\  & & TET & 96.41 & 99.89 & 3.59 \\  & & TKS & **96.76** & 99.86 & **2.75** \\ \hline \multirow{6}{*}{CIFAR100} & \multirow{6}{*}{ResNet19} & Baseline & 76.78 & 94.13 & 78.27 \\  & & LS & 76.77 & 92.04 & 71.96 \\  & & TET & 78.90 & 94.57 & 65.28 \\  & & TKS & **79.89** & **94.77** & **54.07** \\  & & Baseline & 78.35 & 94.38 & 67.55 \\  & & LS & 79.39 & 93.40 & 63.43 \\  & & TET & 79.53 & 94.70 & 58.93 \\  & & TKS & **80.67** & **94.71** & **54.56** \\ \hline \multirow{6}{*}{ImageNet-1K} & SEW-ResNet-34 & Baseline & 68.61 & 87.67 & - \\  & ResNet-34 & TET & 68.68 & 88.13 & - \\ \cline{1-1}  & & TKS & 69.60 & 88.40 & - \\ \cline{1-1}  & & TKS & **70.70** & **88.77** & - \\ \hline \hline \multirow{6}{*}{DVS-CIFAR10} & \multirow{6}{*}{VGG-SNN} & Baseline & 83.20 & 98.60 & 46.14 \\  & & LS & 84.00 & **98.70** & 40.20 \\ \cline{1-1}  & & TET & 84.70 & 97.80 & 43.42 \\ \cline{1-1}  & & TKS & **85.30** & 98.00 & **40.18** \\ \hline \multirow{6}{*}{NCAL-TECH101} & \multirow{6}{*}{VGG-SNN} & Baseline & 78.26 & 91.26 & 52.03 \\ \cline{1-1}  & & LS & 82.76 & 92.18 & 35.21 \\ \cline{1-1}  & & TET & 81.72 & 92.29 & 36.89 \\ \cline{1-1}  & & TKS & **84.10** & **93.33** & **30.00** \\ \hline \hline \end{tabular}
\end{table}
Table 4: Comparison of the multiple methods implemented on our Baseline. The best performing model is indicated as boldface.
require long simulation timesteps to produce accurate results. Moreover, latency may often vary when dealing with different edge devices. This means that when changing devices, the model needs to be retrained with different timesteps, which greatly reduces efficiency. We hope that SNNs can achieve higher performance with large timesteps during training and achieve almost lossless performance with smaller simulation steps during the inference phase. The performance of conventional SNN models drops sharply when the test timestep is inconsistent with the training timestep. We demonstrate the performance of the VGG-SNN model trained on the DVS-CIFAR10 dataset. For the training phase, we set the timestep to 10, and for the test phase, we verify the performance at different timesteps. As shown in Figure 3, for the baseline SNN model, when the timestep is set to 1 in the test phase, the network performance drops by nearly 33%. Our TKS can still achieve accuracy of about 75%. Further, we train and test with the same timestep for the SNN model. As can be seen in Figure 3, our TKS model outperforms them at different timesteps. The TKS model has strong generalization in simulation timesteps, which provides strong convenience for the development of SNNs.

Also, we verify the performance on DVS-CIFAR10 as well as NCALTECH101 using the output of SNN at each single moment. The models are trained in timestep 10. As shown in Fig. 4, for TKS, the performance of the output at each moment is higher than that of the baseline model. The teacher signal helps the model to achieve better performance at each moment. Thus the model at each moment acquires the ability to process the data separately.

### Feature Visualization

We visualize the feature embedding of the penultimate layer of the model using the t-SNE [22] method. The results of TKS and the baseline model are shown in the following graphs on CIFAR100 of ResNet-19 and DVS-CIFAR10 of VGG-SNN, respectively. As shown in Figure 5, TKS can well separate the data for both static images and neuromorphic images compared with the baseline model. The teacher signal of TKS expands the distance between similar categories by adjusting the distribution of sample features, making it easier for the classifier to classify.

## 6 Conclusion

The performance and latency of SNNs have been a significant constraint to the development of SNNs. In this paper, we demonstrate that SNNs do not make effective use of the information generated from different moments. Based on this, we propose a temporal knowledge sharing (TKS) approach, where we select the output of correctly classified moments in SNNs as teacher signals to further guide model training. We validate the effectiveness of TKS both on static datasets CIFAR10, CIFAR100, ImageNet-1k, and neuromorphic datasets DVS-CIFAR10, NCALTECH101. TKS achieves the current optimal performance. Also, on fine-grained classification datasets, our algorithm outperforms the baseline model and other excellent algorithms. In addition to performance, TKS can support SNNs with different simulation steps in the training and inference phases. This allows our algorithm to be deployed with a few simulation steps after obtaining performance guarantees with larger simulation steps in the training phase. This greatly facilitates the deployment of SNNs on edge devices.

Figure 4: Accuracy in each timestep on DVS-CIFAR10(left) NCALTECH101(right). The model is trained at timestep 10. We calculate the accuracy of the output in each step.

Figure 5: The visualization results for t-SNE, (a) Baseline results for CIFAR100. (b) TKS results for CIFAR100. (c) Baseline result for DVS-CIFAR10. (d) TKS results for DVS-CIFAR10

Figure 3: Accuracy in different timestep on DVS-CIFAR10, the model is trained at timestep 10.