SEENN: Towards Temporal Spiking Early-Exit Neural Networks
Spiking Neural Networks (SNNs) have recently become more popular as a
biologically plausible substitute for traditional Artificial Neural Networks
(ANNs). SNNs are cost-efficient and deployment-friendly because they process
input in both spatial and temporal manner using binary spikes. However, we
observe that the information capacity in SNNs is affected by the number of
timesteps, leading to an accuracy-efficiency tradeoff. In this work, we study a
fine-grained adjustment of the number of timesteps in SNNs. Specifically, we
treat the number of timesteps as a variable conditioned on different input
samples to reduce redundant timesteps for certain data. We call our method
Spiking Early-Exit Neural Networks (SEENNs). To determine the appropriate
number of timesteps, we propose SEENN-I which uses a confidence score
thresholding to filter out the uncertain predictions, and SEENN-II which
determines the number of timesteps by reinforcement learning. Moreover, we
demonstrate that SEENN is compatible with both the directly trained SNN and the
ANN-SNN conversion. By dynamically adjusting the number of timesteps, our SEENN
achieves a remarkable reduction in the average number of timesteps during
inference. For example, our SEENN-II ResNet-19 can achieve 96.1% accuracy with
an average of 1.08 timesteps on the CIFAR-10 test dataset. Code is shared at
https://github.com/Intelligent-Computing-Lab-Yale/SEENN.
Yuhang Li, Tamar Geller, Youngeun Kim, Priyadarshini Panda
# SEENN: Towards Temporal Spiking Early-Exit Neural Networks

###### Abstract

Spiking Neural Networks (SNNs) have recently become more popular as a biologically plausible substitute for traditional Artificial Neural Networks (ANNs). SNNs are cost-efficient and deployment-friendly because they process input in both spatial and temporal manners using binary spikes. However, we observe that the information capacity in SNNs is affected by the number of timesteps, leading to an accuracy-efficiency tradeoff. In this work, we study a fine-grained adjustment of the number of timesteps in SNNs. Specifically, we treat the number of timesteps as a variable conditioned on different input samples to reduce redundant timesteps for certain data. We call our method **S**piking **E**arly-**E**xit **N**eural **N**etworks (**SEENNs**). To determine the appropriate number of timesteps, we propose SEENN-I which uses a confidence score thresholding to filter out the uncertain predictions, and SEENN-II which determines the number of timesteps by reinforcement learning. Moreover, we demonstrate that SEENN is compatible with both the directly trained SNN and the ANN-SNN conversion. By dynamically adjusting the number of timesteps, our SEENN achieves a remarkable reduction in the average number of timesteps during inference. For example, our SEENN-II ResNet-19 can achieve **96.1**% accuracy with an average of **1.08** timesteps on the CIFAR-10 test dataset.

Machine Learning, ICML

## 1 Introduction

Deep learning has revolutionized a range of computational tasks such as computer vision and natural language processing (LeCun et al., 2015) using Artificial Neural Networks (ANNs). These successes, however, have come at the cost of tremendous computational demands and high latency (Han et al., 2015). In recent years, Spiking Neural Networks (SNNs) have gained traction as an energy-efficient alternative to ANNs (Roy et al., 2019; Tavanaei et al., 2019). SNNs infer inputs across a number of timesteps as opposed to ANNs, which infer over what is essentially a single timestep. Moreover, during each timestep, the neuron in an SNN either fires a spike or remains silent, thus making the output of the SNN neuron binary and sparse. Such spike-based computing produces calculations that substitute multiplications with additions.

In the field of SNN research, there are two main approaches to getting an SNN: (1) direct training SNNs from scratch and (2) converting ANNs to SNNs. Direct training seeks to optimize an SNN using methods such as spiking timing-based plasticity (Iyer and Chua, 2020) or surrogate gradient-based optimization (Wu et al., 2018; Shrestha and Orchard, 2018). In contrast, the ANN-SNN conversion approach (Diehl et al., 2015; Rueckauer et al., 2016; Sengupta et al., 2018; Han and Roy, 2020; Deng and Gu, 2021; Li et al., 2021) uses the feature representation of a pre-trained ANN and aims to replicate it in the corresponding SNN. Both methods have the potential to achieve high-performance SNNs when implemented correctly.

Despite the different approaches, both training-based and conversion-based SNNs are limited by binary activations. As a result, the key factor that affects their information processing capacity is the **number of timesteps**. Expanding the number of timesteps enables SNNs to capture more features in the temporal dimension, which can improve their accuracy in conversion and training. However, a larger number of timesteps increases latency and computational requirements, resulting in a lower acceleration ratio, resulting in a tradeoff between accuracy and time. Therefore, current efforts to enhance SNN accuracy often involve finding ways to achieve it within the same number of timesteps.

In this paper, we propose a novel approach to improve the tradeoff between accuracy and time in SNNs. Specifically, our method allows each input sample to have a varying number of timesteps during inference, increasing the number of timesteps only when the current sample is hard to classify, resulting in an early exit in the time dimension. We refer to this approach as **S**piking **E**arly-**E**xit **N**eural **N**etworks (**SEENNs**). To determine the optimal number of timesteps for each sample, we propose two methods: SEENN-I, which
uses confidence score thresholding to output a confident prediction as fast as possible; and SEENN-II, which employs reinforcement learning to find the optimal policy for the number of timesteps. Our results show that SEENNs can be applied to both conversion-based and direct training-based approaches, achieving a new state-of-the-art performance for SNNs. In summary, our contributions are threefold:

1. We introduce a new direction to optimize SNN performance by treating the number of timesteps as a variable conditioned to input samples.
2. We propose Spiking Early-Exit Neural Networks (SEENNs) and use two methods to determine which timestep to exit: confidence score thresholding and reinforcement learning optimized with policy gradients.
3. We evaluate our SEENNs on both conversion-based and training-based models with large-scale datasets like CIFAR and ImageNet. For example, our SEENNs can use \(\sim\)**1.1** timesteps to achieve similar performance with a 6-timestep model on the CIFAR-10 dataset.

## 2 Related Work

### Spiking Neural Networks

#### 2.1.1 ANN-SNN Conversion

Converting ANNs to SNNs utilizes the knowledge from pre-trained ANNs and replaces the ReLU activation in ANNs with a spike activation in SNNs (Rueckauer et al., 2016, 2017; Sengupta et al., 2019; Han and Roy, 2020). The conversion-based method, therefore, seeks to match the features in two different models. For example, Rueckauer et al. (2016, 2017) studies how to select the firing threshold to cover all the features in an ANN. (Han et al., 2020) studies using a smaller threshold and (Deng and Gu, 2021) proposes to use a bias shift to better match the activation. Based on the error analysis, (Li et al., 2021) utilizes a parameter calibration technique and (Bu et al., 2022) further changes the training scheme in ANN.

#### 2.1.2 Direct Training of SNN

Direct training from scratch allows SNNs to operate within extremely few timesteps. In recent years, the number of timesteps used to train SNNs has been reduced from more than 100 (Rathi et al., 2019) to less than 5 (Zheng et al., 2020). The major success is based on the spatial-temporal backpropagation (Wu et al., 2018; 2019) and surrogate gradient estimation of the firing function (Dong et al., 2017). Through gradient-based learning, recent works (Fang et al., 2021; Rathi and Roy, 2021; Kim and Panda, 2021; \(\mathbf{\mathbf{\mathbf{\mathbf{\mathbf{\mathbf{\mathbf{\mathbf{\mathbf{ \mathbf{\mathbf{\mathbf{\mathbf{\mathbf{\mathbf{\mathbf{\mathbf{ \mathbf{      \mathbf{    }}}}}}}}}}}}\)\) propose to optimize not only parameters but also firing threshold and leaky factor. Moreover, loss function (Deng et al., 2022), surrogate gradient estimation (Li et al., 2021), batch normalization (Duan et al., 2022), activation distribution (Guo et al., 2022) are also factors that affect the learning behavior in direct training and have been investigated properly. Our method, instead, focuses on the time dimension, which is fully complementary to both conversion and training.

### Conditional Computing

Conditional computing models can boost the representation power by adapting the model architectures, parameters, or activations to different input samples (Han et al., 2021). BranchyNet (Teerapittayanon et al., 2016) and Conditional Deep Learning (Panda et al., 2016) add multiple classifiers in different layers to apply spatial early exit to ANNs. SkipNet (Wang et al., 2018) and BlockDrop (Wu et al., 2018) use a dynamic computation graph by skipping different blocks based on different samples. CondConv (Yang et al., 2019) and Dynamic Convolution (Han et al., 2021) use the attention mechanism to change the weight in convolutional layers according to input features. Squeeze-and-Excitation Networks (Hu et al., 2018) proposes to reweight different activation channels based on the global context of the input samples. To the best of our knowledge, our work is the first to incorporate conditional computing into SNNs.

## 3 Methodology

In this section, we describe our overall methodology and algorithm details. We start by introducing background on fixed-timestep spiking neural networks and then, based on a strong rationale, we introduce our SEENN method.

### Spiking Neural Networks

SNNs simulate biological neurons with Leaky Integrate-and-Fire (LIF) layers (Burkitt, 2006). In each timestep, the input current in the \(\ell\)-th layer charges the membrane potential \(\mathbf{u}\) in the LIF neurons. When the membrane potential exceeds a threshold, a spike \(\mathbf{s}\) will fire to the next layer, as given by:

\[\mathbf{u}^{\ell}[t+1] =\tau\mathbf{u}^{\ell}[t]+\mathbf{W}^{\ell}\mathbf{s}^{\ell-1}[t], \tag{1}\] \[\mathbf{s}^{\ell}[t+1] =H(\mathbf{u}^{\ell}[t+1]-V), \tag{2}\]

where \(\tau\in(0,1]\) is the leaky factor, mimicking natural potential decay. \(H(\cdot)\) is the Heaviside step function and \(V\) is the firing threshold. If a spike fires, the membrane potential will be reset to 0: \((\mathbf{u}[t+1]=\mathbf{u}[t+1]*(1-\mathbf{s}[t+1]))\). Following existing baselines, in direct training, we use \(\tau=0.5\) while in conversion we use \(\tau=1.0\), transforming to the Integrate-and-Fire (IF) model. Moreover, the reset in conversion is done by subtraction: \((\mathbf{u}[t+1]=\mathbf{u}[t+1]-V_{th}\mathbf{s}[t+1])\) as suggested by (Han and Roy, 2020). Now, denote the overall spiking neural network as a function \(f_{T}(\mathbf{x})\), its forward propagation can be formulated as

\[f_{T}(\mathbf{x})=\frac{1}{T}\sum_{t=1}^{T}h\circ g^{L}\circ g^{L-1}\circ g^{L -2}\circ\cdots g^{1}(\mathbf{x}), \tag{3}\]
where \(h(\cdot)\) denotes the final linear classifier, and \(g^{t}(\cdot)\) denotes the \(\ell\)-th block of backbone networks. \(L\) represents the total number of blocks in the network. A block contains a convolutional layer to compute input current (\(\mathbf{W}^{\ell}\mathbf{s}^{\ell-1}\)), a normalization layer (Zheng et al., 2020), and a LIF layer. In this work, we use a direct encoding method, \(i.e.\), using \(g^{1}(\mathbf{x})\) to encode the input tensor into spike trains, as done in recent SNN works (Fang et al., 2021; Deng et al., 2022; Kim et al., 2022). In SNNs, we repeat the inference process for \(T\) times and average the output from the classifier to produce the final result.

### Introducing Early-Exit to SNNs

Conventionally, the number of timesteps \(T\) is set as a fixed hyper-parameter, causing a tradeoff between accuracy and efficiency. Here, we also provide several \(accuracy-T\) curves of spiking ResNets in Fig. 1. These models are trained from scratch or converted to 4, 6, or 8 timesteps and we evaluate the performance with the available numbers of timesteps. For the CIFAR-10 dataset, increasing the number of timesteps from 2 to 6 only brings 0.34% top-1 accuracy gain, at the cost of 300% more latency. _In other words, the majority of correct predictions can be inferred with much fewer timesteps._

The observation in Fig. 1 motivates us to explore a more fine-grained adjustment of \(T\). We are interested in an SNN that can adjust \(T\) based on the characteristics of different input images. Hypothetically, each image is linked to an _difficulty_ factor, and this SNN can identify this difficulty to decide how many timesteps should be used, thus eliminating unnecessary timesteps for easy images. We refer to such a model as a spiking early-exit neural network (SEENN).

To demonstrate the potential of SEENN, we propose a metric that calculates the minimum timesteps needed to perform correct prediction averaged on the test dataset, \(i.e.\), the lowest timesteps we can achieve without comprising the accuracy, and it is based on the following assumption:

**Assumption 3.1**.: Given a spiking neural network \(f_{T}\), if it can correctly predict \(\mathbf{x}\) with \(t\) timesteps, then it always outputs correct prediction for any \(t^{\prime}\) such that \(t\leq t^{\prime}\leq T\).

This assumption indicates the inclusive property of different timesteps. In Appendix A, we provide empirical evidence to support this assumption. Let \(\mathbb{C}_{t}\) be the set of correct predicted input samples for timestep \(t\), we have \(\mathbb{C}_{1}\subseteq\mathbb{C}_{2}\subseteq\cdots\subseteq\mathbb{C}_{T}\) based on Assumption 3.1. Also denote \(\mathbb{W}=\overline{\mathbb{C}_{T}}\) as the wrong prediction set, we propose the _averaged earliest timestep (AET)_ metric, given by

\[\text{AET}=\frac{1}{N}\left(|\mathbb{C}_{1}|+T|\mathbb{W}|+\sum_{t=2}^{T}t(| \mathbb{C}_{t}|-|\mathbb{C}_{t-1}|)\right), \tag{4}\]

where \(|\cdot|\) returns the cardinal number of the set, and \(|\mathbb{C}_{t}|-|\mathbb{C}_{t-1}|\) returns the number of samples that belong to \(\mathbb{C}_{t}\) yet not to \(\mathbb{C}_{t-1}\). \(N=|\mathbb{C}_{T}|+|\mathbb{W}|\) is the total number of samples in the validation dataset. The AET metric describes an ideal scenario where correct predictions are always inferred using the minimum number of timesteps required, while still preserving the original accuracy. It's worth noting that incorrect samples are inferred using the maximum number of timesteps, as it is usually not possible to determine if a sample cannot be correctly classified before inference.

In Fig. 1, we report the AET in each case. For models directly trained on CIFAR10 or CIFAR100, the AET remains slightly higher than \(2\) (note that the minimum number of timesteps is 2). With merely 11% more latency added to the 2-timestep SNN on CIFAR10 (\(T=2.228\)), we can achieve an accuracy equal to a 6-timestep SNN. The converted SNNs also only need a few extra time steps. This suggests the huge potential for using early exit in SNNs.

Despite the potential performance boost from the early exit, it is impossible to achieve the AET effect in practice since we cannot access the label in the test set. Therefore, the question of how to design an efficient and effective predictor that determines the number of timesteps for each input is

Figure 1: The accuracy of spiking ResNet under different numbers of timesteps on CIFAR10, CIFAR100, and ImageNet datasets, either by direct training (Deng et al., 2022) or by conversion (Bu et al., 2022).
non-trivial. In the following sections, we propose two methods, SEENN-I and SEENN-II, to address this challenge.

### Seenn-I

In SEENN-I, we adopt a proxy signal to determine the difficulty of the input sample--confidence score. Formally, let the network prediction probability distribution be \(\mathbf{p}=\mathrm{softmax}(f_{t}(\mathbf{x}))=[p_{1},p_{2},\ldots,p_{M}]\), where \(M\) is the number of object classes, the confidence score (CS) is defined as

\[\text{CS}=\mathrm{max}(\mathbf{p}), \tag{5}\]

which means the maximum probability in \(\mathbf{p}\). The CS is a signal that measures the level of uncertainty. If the CS is high enough (, CS\(=0.99\)), the prediction distribution will be highly deterministic; otherwise (, CS\(=\frac{1}{M}\)), the prediction distribution is uniform and extremely uncertain.

A line of work (Teerapittayanon et al., 2016; Guo et al., 2017) has shown that the level of uncertainty is highly correlated with the accuracy of the neural networks. For input samples with deterministic output prediction, the neural network typically achieves high accuracy, which corresponds to relatively _easy_ samples. Hence, we can utilize this property as an indicator of how many time steps should be assigned to this sample. We adopt a simple thresholding mechanism,, given a preset threshold \(\alpha\), we iterate each timestep and once the confidence score is higher than \(\alpha\), the accumulated output is used for prediction. A diagram describing this method can be found in Fig. 2.

### Seenn-Ii

Our SEENN-I is a post-training method that can be applied to an off-the-shelf SNN. It is easy to use, but the SNN is not explicitly trained with the early exit, so the full potential of early exit is not exploited through SEENN-I. In this section, we propose an "early-exit-aware training" method for SNNs called SEENN-II.

SEENN-II is built from reinforcement learning to directly predict the difficulty of the image. Specifically, we define an action space \(\mathcal{T}=\{t_{1},t_{2},\cdots,t_{n}\}\), which contains the candidates for the number of timesteps that can be applied to an input sample \(\mathbf{x}\). To determine the optimal timestep candidates, we develop a policy network that generates an \(n\)-dimensional _policy vector_ to sample actions. During training, a reward function is calculated based on the policy and the prediction result, which is generated by running the SNN with the number of timesteps suggested by the policy vector. Unlike traditional reinforcement learning (Sutton and Barto, 2018), our problem does not consider state transition as the policy can predict all actions at once. We also provide a diagram describing the overall method in Fig. 3.

Formally, consider an input sample \(\mathbf{x}\) and a policy network \(f_{p}\) with parameter \(\theta\), we define the policy of selecting the timestep candidates as an \(n\)-dim categorical distribution:

\[\mathbf{v}=\mathrm{softmax}(f_{p}(\mathbf{x};\theta)), \tag{6}\]

\[\pi_{\theta}(\mathbf{z}|\mathbf{x})=\prod_{k=1}^{n}\mathbf{v}_{k}^{\mathbf{z }_{k}}, \tag{7}\]

where \(\mathbf{v}\) is the probability of the categorical distribution, obtained by inferring the policy networks with a \(\mathrm{softmax}\) function. Thus \(\mathbf{v}_{k}\) represents the probability of choosing \(t_{k}\) as the number of timesteps in the SNN. An action \(\mathbf{z}\in\{0,1\}^{n}\) is sampled based on the policy \(\mathbf{v}\). Here, \(\mathbf{z}\) is a one-hot vector since only one timestep can be selected. Note that the policy network architecture is made sufficiently small such that the cost of inferring the policy is negligible compared to SNN (see architecture details in Sec. 4.1).

Once we obtain an action vector \(\mathbf{z}\), we can evaluate the prediction using the target number of timesteps,, \(f_{t}(\mathbf{x})\). Our objective is to minimize the number of timesteps we used while not sacrificing accuracy. Therefore, we associate the actions taken with the following reward function:

\[R(\mathbf{z})=\begin{cases}\frac{1}{2^{t_{k}}|\mathbf{x}_{k}=1}&\text{if correct prediction}\\ -\beta&\text{if incorrect prediction}\end{cases}. \tag{8}\]

Here, \(t_{k}|_{\mathbf{z}_{k}=1}\) represents the number of timesteps selected by \(\mathbf{z}\). Here, the reward function is determined by whether the prediction is correct or incorrect. If the prediction is correct, then we incentivize early exit by assigning a larger reward to a policy that uses fewer timesteps. However, if the prediction is wrong, we penalize the reward with \(\beta\), which serves the role to balance the accuracy and the efficiency. As an example, a large \(\beta\) leads to more correct predictions but also more timesteps.

**Gradient Calculation in the Policy Network** To this end, our objective for training the policy network is to maximize

Figure 2: The overview of SEENN-I, which computes the confidence score to select the optimal number of timesteps.
the expected reward function, given by:

\[\max_{\theta}\ \mathbb{E}_{\mathbf{z}\sim\pi_{\theta}}[R(\mathbf{z})]. \tag{9}\]

In order to calculate the gradient of the above objective, we utilize the policy gradient method (Sutton and Barto, 2018) to compute the derivative of the reward function w.r.t. \(\theta\), given by

\[\nabla_{\theta}\mathbb{E}[R(\mathbf{z})] =\mathbb{E}[R(\mathbf{z})\nabla_{\theta}\log\pi_{\theta}(\mathbf{ z}|\mathbf{x})]\] \[=\mathbb{E}[R(\mathbf{z})\nabla_{\theta}\log\prod_{k=1}^{n} \mathbf{v}_{k}^{\mathbf{z}_{k}}]\] \[=\mathbb{E}[R(\mathbf{z})\nabla_{\theta}\sum_{k=1}^{n}\mathbf{z} _{k}\log\mathbf{v}_{k}]. \tag{10}\]

Moreover, unlike other reinforcement learning which relies on Monte-Carlo sampling to compute the expectation, our method can compute the exact expectation. During the forward propagation of the SNN, we can store the intermediate accumulated output at each \(t_{k}\), and calculate the reward function using the stored accumulated output \(f_{t_{k}}(\mathbf{x})\). Since \(\pi_{\theta}(\mathbf{z}|\mathbf{x})\)is a categorical distribution, we can rewrite Eq. (10) as

\[\nabla_{\theta}\mathbb{E}[R(\mathbf{z})]=\sum_{k=1}^{n}R(\mathbf{z}|_{\mathbf{ z}_{k}=1})\mathbf{v}_{k}\nabla_{\theta}\log\mathbf{v}_{k}, \tag{11}\]

where \(R(\mathbf{z}|_{\mathbf{z}_{k}=1})\) is the reward function evaluated with the output prediction using \(t_{k}\) timesteps.

### Training SEENN

In this section, we describe the training methods for our SEENN. To train the model for SEENN-I, we explicitly add a cross-entropy function to each timestep, thus making the prediction in early timesteps better, given by

\[\min_{\mathbf{W}}\frac{1}{n}\sum_{k=1}^{n}L_{CE}(f_{t_{k}}(\mathbf{x}), \mathbf{W},\mathbf{y}), \tag{12}\]

where \(L_{CE}\) denotes the cross-entropy loss function and \(\mathbf{y}\) is the label vector. This training objective is essentially Temporal Efficient Training (TET) loss proposed in Deng et al. (2022). We find this function can enhance the performance from every \(t_{k}\) compared to \(L_{CE}\) applied to only the maximum number of timesteps. As for conversion-based SEENN-I, we do not modify any training function. Instead, we directly apply the confidence score thresholding to the converted model.

To train the SEENN-II, we first employ TET loss to train the model for several epochs without involving the policy network. This can avoid the low training accuracy in the early stage of the training which may damage the optimization of the policy network. Then, we jointly optimize the SNN and the policy network by

\[\min_{\mathbf{W},\theta}\mathbb{E}_{\mathbf{z}\sim\pi_{\theta}}[-R(\mathbf{z })+L_{CE}(f_{t_{k}|_{\mathbf{z}_{k}=1}}(\mathbf{x}),\mathbf{W},\mathbf{y})]. \tag{13}\]

Note that we do not train SEENN-II for the converted model since in conversion, a pre-trained ANN is used to obtain the converted SNN and does not warrant any training.

## 4 Experiments

To demonstrate the efficacy and the efficiency of our SEENN, we conduct experiments on popular image recognition datasets, including CIFAR10, CIFAR100 (Krizhevsky et al., 2010), ImageNet (Deng et al., 2009), and an event-stream dataset CIFAR10-DVS (Li et al., 2017). Moreover, to show the compatibility of our method, we compare SEENN with both training-based and conversion-based state-of-the-art methods. Finally, we also provide some hardware and qualitative evaluation on SEENN.

### Implementation Details

In order to implement SEENN-I, we adopt the code framework of TET (Deng et al., 2022) and QCFS (Bu et al., 2022), both of which provide open-source implementation. All models are trained with a stochastic gradient descent optimizer with a momentum of 0.9 for 300 epochs. The learning rate is 0.1 and decayed following a cosine annealing schedule (Loshchilov and Hutter, 2016). The weight decay is set to \(5e-4\). For ANN pre-training in QCFS, we set the step \(l\) to 4. We use Cutout (DeVries and Taylor, 2017) and AutoAugment (Cubuk et al., 2019) for better accuracy as adopted in (Li et al., 2021; Duan et al., 2022).

To implement SEENN-II, we first take the checkpoint of pre-trained SNN in SEENN-I and initialize the policy network. Then, we jointly train the policy network as well as the SNN. The policy network we take for the CIFAR dataset is ResNet-8, which contains only 0.547% computation of ResNet-19. We also provide the details of architecture in Appendix B and measure the latency/energy of the policy

Figure 3: The overview of SEENN-II, which computes the confidence score to select the optimal number of timesteps.
network in Sec. 4.3. For the ImageNet dataset, we downsample the image resolution to 112\(\times\)112 for the policy network. Both the policy network and the SNN are finetuned for 75 epochs using a learning rate of 0.01. Other training hyper-parameters are kept the same as SEENN-I.

### Comparison to SOTA work

CifarWe first provide the results on the CIFAR-10 and CIFAR-100 datasets (Krizhevsky et al., 2010). We test the architectures in the ResNet family (He et al., 2016). We summarize both the direct training comparison as well as the ANN-SNN conversion comparison in Table 1 and Table 2. Since the \(T\) in our SEENN can be variable based on the input, we report the average number of timesteps in the test dataset. As we can see from Table 1, existing direct training methods usually use 2, 4, and 6 timesteps in inference. However, increasing the number of timesteps from 2 to 6 brings marginal improvement in accuracy, for example, 95.45% to 95.60% in TEBN (Duan et al., 2022). Our SEENN-I can achieve **96.07**% with only **1.09** average timesteps, which is **5.5\(\times\)** lower than the state of the art. SEENN-II gets a similar performance with SEENN-I in the case of CIFAR-10, but it can achieve 0.7% higher accuracy than SEENN-I on the CIFAR-100 dataset as shown in Table 2.

We also include the results of ANN-SNN conversion using SEENN-I. Here, the best existing work is QCFS (Bu et al., 2022b), which can convert the SNN in lower than 8 timesteps. We directly run SEENN-I with different choices of confidence score threshold \(\alpha\). Surprisingly, on the CIFAR-10 dataset, our SEENN-I can convert the model with **1.4** timesteps, and get **93.63**% accuracy. Instead, the QCFS only gets 75.44% accuracy using 2 timesteps for all input images. By selecting a higher threshold, we can obtain **95.08**% accuracy with **2.01** timesteps, which uses **4\(\times\)** lower number of timesteps than QCFS under the same accuracy level.

ImageNetWe compare the direct training and conversion methods on the ImageNet dataset. The results are

\begin{table}
\begin{tabular}{l l c c} \hline
**Model** & **Method** & **T** & **Acc.** \\ \hline \multicolumn{4}{c}{_Direct Training of SNNs_} \\ \hline \multirow{3}{*}{ResNet-19} & \multirow{3}{*}{tdBN (Zheng et al., 2020)} & 2 & 92.34 \\  & & 4 & 92.92 \\  & & 6 & 93.16 \\ \hline \multirow{3}{*}{ResNet-20} & \multirow{3}{*}{Dspike (Li et al., 2021b)} & 2 & 93.13 \\  & & 4 & 93.66 \\  & & 6 & 94.25 \\ \hline \multirow{6}{*}{ResNet-19} & \multirow{3}{*}{TET (Deng et al., 2022)} & 2 & 94.16 \\  & & 4 & 94.44 \\  & & 6 & 94.50 \\ \cline{2-4}  & & 2 & 95.45 \\ \cline{2-4}  & TEBN (Duan et al., 2022) & 4 & 95.58 \\  & & 6 & 95.60 \\ \cline{2-4}  & & **1.09** & **96.07** \\  & & **1.20** & **96.38** \\ \cline{2-4}  & **SEENN-II (Ours)** & **1.08** & **96.01** \\ \hline \multicolumn{4}{c}{_ANN-SNN Conversion_} \\ \hline \multirow{3}{*}{ResNet-20} & \multirow{3}{*}{Opt (Deng \& Gu, 2021)} & 16 & 92.41 \\  & & 32 & 93.30 \\  & Calibration (Li et al., 2021a) & 32 & 94.78 \\ \hline \multirow{6}{*}{ResNet-18} & \multirow{3}{*}{QCFS (Bu et al., 2022b)} & 8 & 75.44 \\  & & 16 & 90.43 \\ \cline{1-1}  & & 32 & 94.82 \\ \cline{1-1} \cline{2-4}  & & 2 & 75.44 \\ \cline{1-1}  & & 4 & 90.43 \\ \cline{1-1}  & & 8 & 94.82 \\ \cline{1-1} \cline{2-4}  & & **1.40** & **93.63** \\ \cline{1-1}  & & **2.01** & **95.08** \\ \hline \end{tabular}
\end{table}
Table 1: Accuracy-\(T\) comparison on CIFAR-10 dataset.

\begin{table}
\begin{tabular}{l l c c} \hline
**Model** & **Method** & **T** & **Acc.** \\ \hline \multicolumn{4}{c}{_Direct Training of SNNs_} \\ \hline \multirow{3}{*}{ResNet-20} & \multirow{3}{*}{Dspike (Li et al., 2021b)} & 2 & 71.68 \\  & & 4 & 73.35 \\  & & 6 & 74.24 \\ \hline \multirow{6}{*}{ResNet-19} & \multirow{3}{*}{TET (Deng et al., 2022)} & 2 & 72.87 \\  & & 4 & 74.47 \\  & & 6 & 74.72 \\ \cline{2-4}  & & 2 & 78.07 \\ \cline{2-4}  & TEBN (Duan et al., 2022) & 4 & 78.71 \\  & & 6 & 78.76 \\ \cline{2-4}  & & **1.19** & **79.56** \\  & & **1.55** & **81.42** \\ \cline{2-4}  & **SEENN-II (Ours)** & **1.21** & **80.23** \\ \cline{2-4}  & & _ANN-SNN Conversion_ & \\ \hline \multirow{3}{*}{ResNet-20} & \multirow{3}{*}{Opt (Deng \& Gu, 2021)} & 16 & 63.73 \\  & & 32 & 68.40 \\ \cline{1-1}  & & 32 & 75.53 \\ \hline \multirow{6}{*}{ResNet-20} & \multirow{3}{*}{OPI (Bu et al., 2022a)} & 8 & 23.09 \\  & & 16 & 52.34 \\ \cline{1-1}  & & 32 & 67.18 \\ \cline{1-1} \cline{2-4}  & & 2 & 19.96 \\ \cline{1-1}  & QCFS (Bu et al., 2022b) & 4 & 34.14 \\ \cline{1-1}  & & 8 & 55.37 \\ \cline{1-1} \cline{2-4}  & **SEENN-I (Ours)** & **2.57** & **39.33** \\ \cline{1-1}  & & **4.41** & **56.99** \\ \hline \end{tabular}
\end{table}
Table 2: Accuracy-\(T\) comparison on CIFAR-100 dataset.
sorted Table 3. For direct training, we use two baseline networks: vanilla ResNet-34 (He et al., 2016) and SEW-ResNet-34 (Fang et al., 2021). For SEENN-I, we directly use the pre-trained checkpoint from TET (Deng et al., 2022). It can be observed that our SEENN-I can achieve the same accuracy using only 50% of original number of timesteps. For example, SEENN-I ResNet-34 reaches the maximum accuracy in **2.35** timesteps. SEENN-II can obtain a similar performance by using a smaller \(T\), _i.e_., 1.79. For conversion experiments, we again compare against QCFS using ResNet-34. Our SEENN-I obtains **70.2%** accuracy with **23.5** timesteps, higher than a 32-timestep QCFS model.

**CIFAR10-DVS** Here, we compare our SEENN-I on an event-stream dataset, CIFAR10-DVS (Li et al., 2017). Following existing baselines, we train the SNN with a fixed number of 10 timesteps. From Table 4 we can find that SEENN-I can surpass all existing work except TET with only **2.53** timesteps, amounting to nearly **4\(\times\)** faster inference. Moreover, SEENN-II can get an accuracy of **82.6** using 4.5 timesteps.

### Hardware Efficiency

In this section, we analyze the hardware efficiency of our method, including latency and energy consumption. Due to the sequential processing nature of SNNs, the latency is generally proportional to \(T\) on most hardware devices. Therefore, we directly use a GPU (NVIDIA Tesla V100) to evaluate the latency (or throughput) of SEENN. For energy estimation, we follow a rough measure to count only the energy of operations that is adopted in previous work (Rathi and Roy, 2021; Li et al., 2021; Bu et al., 2022), as SNNs are usually deployed on memory-cheap devices. Fig. 4 plots the comparison of inference throughput and energy. It can be found that our SEENN-I simultaneously improves inference speed while reducing energy costs. Meanwhile, the policy network in SEENN-II only brings marginal effect and does not impact the overall inference speed and energy cost, demonstrating the efficiency of our proposed method.

### Ablation Study

In this section, we conduct the ablation study on our SEENN. In particular, we allow SEENN to flexibly change their balance between the accuracy and the number of timesteps. For example, SEENN-I can adjust the confidence score threshold \(\alpha\) and SEENN-II can adjust the penalty value \(\beta\) defined in the reward function. To demonstrate the impact of differ

\begin{table}
\begin{tabular}{l l c c} \hline \hline
**Model** & **Method** & **T** & **Acc.** \\ \hline \multicolumn{4}{c}{_Direct Training of SNNs_} \\ \hline ResNet-19 & tdBN (Zheng et al., 2020) & 10 & 67.8 \\ VGGSNN & PLIF (Fang et al., 2021) & 20 & 74.8 \\ ResNet-18 & Dspike (Li et al., 2021) & 10 & 75.4 \\ ResNet-19 & RecDis-SNN (Guo et al., 2022) & 10 & 72.4 \\ VGGSNN & TET (Deng et al., 2022) & 10 & **83.1** \\ \hline \multirow{4}{*}{VGG-SNN} & **SEENN-I (Ours)** & **2.53** & **77.6** \\  & **5.17** & **82.7** \\ \cline{2-4}  & **SEENN-II (Ours)** & **4.49** & **82.6** \\ \hline \hline \end{tabular}
\end{table}
Table 4: Accuracy-\(T\) comparison on CIFAR10-DVS dataset.

Figure 4: Comparison of latency (inference throughput) and energy consumption between SNN and SEENN.

\begin{table}
\begin{tabular}{l l c c} \hline \hline
**Model** & **Method** & **T** & **Acc.** \\ \hline \multicolumn{4}{c}{_Direct Training of SNNs_} \\ \hline ResNet-34 & tdBN (Zheng et al., 2020) & 6 & 63.72 \\  & TET (Deng et al., 2022) & 6 & **64.79** \\ ResNet-34 & TEBN (Duan et al., 2022) & 6 & 64.29 \\ \cline{2-4}  & **SEENN-I (Ours)** & **2.28** & **63.65** \\  & **3.38** & **64.66** \\ \cline{2-4}  & **SEENN-II (Ours)** & **2.40** & **64.18** \\ \hline \multirow{4}{*}{SEW-ResNet-34} & SEW (Fang et al., 2021) & 4 & 67.04 \\  & TET (Deng et al., 2022) & 4 & 68.00 \\ \cline{1-1}  & TEBN (Duan et al., 2022) & 4 & **68.28** \\ \cline{1-1}  & **SEENN-I (Ours)** & **1.66** & **66.21** \\  & **2.35** & **67.99** \\ \cline{1-1} \cline{2-4}  & **SEENN-II (Ours)** & **1.79** & **67.48** \\ \hline \multicolumn{4}{c}{_ANN-SNN Conversion_} \\ \hline ResNet-34 & Opt (Deng and Gu, 2021) & 32 & 33.01 \\  & 64 & 59.52 \\ \cline{1-1} \cline{2-4}  & Calibration (Li et al., 2021) & 32 & 64.54 \\  & 64 & 71.12 \\ \cline{1-1} \cline{2-4}  & & 16 & 59.35 \\ QCFS (Bu et al., 2022) & 32 & 69.37 \\  & 64 & 72.35 \\ \cline{1-1} \cline{2-4}  & **SEENN-I (Ours)** & **23.47** & **70.18** \\  & **29.53** & **71.84** \\ \hline \hline \end{tabular}
\end{table}
Table 3: Accuracy-\(T\) comparison on ImageNet dataset.
ent hyper-parameters selection, we utilize SEENN-I evaluated with different \(\alpha\). Fig. 5 shows the comparison. The yellow line denotes the trained SNN with a fixed number of timesteps while the blue line denotes the corresponding SEENN-I with 6 different thresholds. We test the SEW-ResNet-34 on the ImageNet dataset and the ResNet-19 on the CIFAR10 dataset. For both datasets, our SEENN-I has a higher \(accuracy-T\) curve than the vanilla SNN, which confirms that **our SEENN improves the accuracy-efficiency tradeoff**. Moreover, our SEENN-I largely reduces the distance between the AET coordinate and the \(accuracy-T\) curve, meaning that our method is approaching the upper limit of the early exit. On the right side of the Fig. 5, we additionally draw the composition pie charts of SEENN-I, which shows how many percentages of inputs are using 1, 2, 3, or 4 timesteps, respectively. It can be shown that, as we gradually adjust \(\alpha\) from 0.4 to 0.9 for the SEW-ResNet-34, the percentage of inputs using 1 timestep decreases (73.4% to 30.7%). For the CIFAR10 dataset, we find images have a higher priority in using the first timestep, ranging from 95.4% to 56.2%.

### Qualitative Assessment

In this section, we conduct a qualitative assessment of SEENN by visualizing the input images that are separated by our SEENN-II, or the policy network. Specifically, we take the policy network and let it output the number of timesteps for each image in the ImageNet validation dataset. In principle, the policy network can differentiate whether the image is _easy_ or _hard_ so that easy images can be inferred with less number of timesteps and hard images can be inferred with more number of timesteps.

Fig. 6 provides some examples of this experiment, where images are chosen from orange, cucumber, bubble, broccoli, aircraft carrier, and torch classes in the ImageNet validation dataset. We can find that \(T=1\) (easy) images and \(T=4\) (hard) images have huge visual discrepancies. As an example, the orange, cucumber, and broccoli images from \(T=1\) row are indeed easier to be identified, as they contain single objects in a clean background. However, in the case of \(T=4\), there are many irrelevant objects overlapped with the target object, or there could be many small samples of target objects which makes it harder to identify. For instance, in the cucumber case, there are other vegetables that increase the difficulty of identifying them as cucumbers. These results confirm our hypothesis that visually simpler images are indeed easier and can be correctly predicted using a fewer number of timesteps.

## 5 Conclusion

In this paper, we introduce SEENN, a novel attempt to allow a varying number of timesteps on an input-dependent basis. Our SEENN includes both a post-training approach (confidence score thresholding) and an early-exit-aware training approach (reinforcement learning for selecting the appropriate number of timesteps). Our experimental results show that SEENN is able to find a sweet spot that maintains accuracy while improving efficiency. Moreover, we show that the number of timesteps selected by SEENNs is related to

Figure 5: Comparison between SEENN-I and SNN. _Left:_ Accuracy vs. the number of timesteps curve. _Right:_ Pie charts indicating the composition of input images using different numbers of timesteps for inference.

Figure 6: Qualitative assessment using input images from ImageNet dataset. We select images from _orange, cucumber, bubble, broccoli, aircraft carrier, and torch_ classes and separate them according to SEENN.
the visual difficulty of the image. By taking an input-by-input approach during inference, SEENN is able to achieve state-of-the-art accuracy with less computational resources.
