MINT: Multiplier-less INTeger Quantization for Energy Efficient Spiking
  Neural Networks
We propose Multiplier-less INTeger (MINT) quantization, a uniform
quantization scheme that efficiently compresses weights and membrane potentials
in spiking neural networks (SNNs). Unlike previous SNN quantization methods,
MINT quantizes memory-intensive membrane potentials to an extremely low
precision (2-bit), significantly reducing the memory footprint. MINT also
shares the quantization scaling factor between weights and membrane potentials,
eliminating the need for multipliers required in conventional uniform
quantization. Experimental results show that our method matches the accuracy of
full-precision models and other state-of-the-art SNN quantization techniques
while surpassing them in memory footprint reduction and hardware cost
efficiency at deployment. For example, 2-bit MINT VGG-16 achieves 90.6%
accuracy on CIFAR-10, with roughly 93.8% reduction in memory footprint from the
full-precision model and 90% reduction in computation energy compared to
vanilla uniform quantization at deployment. The code is available at
https://github.com/Intelligent-Computing-Lab-Yale/MINT-Quantization.
Ruokai Yin, Yuhang Li, Abhishek Moitra, Priyadarshini Panda
# MINT: Multiplier-less INTeger Quantization for Spiking Neural Networks

###### Abstract

We propose Multiplier-less INTeger (MINT) quantization, an efficient uniform quantization scheme for the weights and membrane potentials in spiking neural networks (SNNs). Unlike prior SNN quantization works, MINT quantizes the memory-hungry membrane potentials to extremely low bit-width (2-bit) to significantly reduce the total memory footprint. Additionally, MINT quantization shares the quantization scale between the weights and membrane potentials, eliminating the need for multipliers and floating arithmetic units, which are required by the standard uniform quantization. Experimental results demonstrate that our proposed method achieves accuracy that matches other state-of-the-art SNN quantization works while outperforming them on total memory footprint and hardware cost at deployment time. For instance, 2-bit MINT VGG-16 achieves 48.6% accuracy on TinyImageNet (+0.28% from the full-precision baseline) with approximately 93.8% reduction in total memory footprint from the full-precision model; meanwhile, our model reduces area by 93% and dynamic power by 98% compared to other SNN quantization counterparts.

 Neuromorphic computing, Spiking neural networks, Neural network compression, Computer architecture

## I Introduction

Spiking Neural Networks (SNNs) [1, 2, 3] have emerged as a promising alternative to Artificial Neural Networks (ANNs) due to their energy efficiency in processing highly sparse unary spike trains. This feature makes them attractive candidates for low-power edge devices, as they can run with much-simplified arithmetic units.

As SNNs continue to gain popularity, numerous prior studies have aimed to enhance their performance for edge deployment. As shown in Fig. 1, one group of previous works [2, 4, 5, 6, 7] has a discernible trend of improving accuracy and diminishing timesteps of SNNs on the CIFAR10 image classification task. These efforts help SNNs achieve higher task accuracy with lower computation energy. Another group of research efforts has focused on the limited on-chip memory for edge deployment of SNNs. These prior works have sought to compress the SNN model size by either pruning [8] or quantizing the weights of SNNs [9, 10, 11]. While prior optimizations have improved the performance of SNNs towards more efficient edge deployment, one significant issue has been largely overlooked in previous research - the memory footprint of the membrane potential.

As shown in Fig. 1, the memory footprint of the membrane potential, often remains several times larger than the memory size of weights. The membrane potential is used to store temporal information and is related to the generation of output spikes inside the Leaky-Integrate-and-Fire (LIF) neuron. A dedicated memory is required for each hidden neuron to store its unique membrane potential for each timestep and input image, in order to ensure the proper functionality of SNNs. Thus, the memory footprint of the membrane potential grows significantly as the size of SNNs, the number of timesteps, or the number of mini-batches scales up.

While weight quantization has been extensively explored by the prior works [9, 10, 11, 12] and brings significant SNN model size reduction, the storage size of membrane potential becomes dominant when the size of weights decreases. For instance, as shown in Fig. 2 (a), on the VGG9 network, as the bit-precision of weights decreases from 32-bit to 4-bit, the proportion of the membrane potential in the total memory footprint grows from less than 20% to more than 60%. Moreover, as batch size increases from 1 to 32, the membrane potential takes up over 98% of the total memory footprint. Unfortunately, the compression of membrane potential has been consistently overlooked by existing SNN research efforts.

To address the problem of the expanding size of membrane potential, this work studies the quantization of membrane potential together with weight quantization. We explore the membrane potential quantization based on standard uniform quantization (UQ), an integer-based technique extensively used in prior quantization works [10, 13, 14]. However, as shown on the left side of Fig. 2 (b), naively applying UQ to SNNs brings two hardware-related problems.

Fig. 1: Visualization of the selected prior works including, STQ [9], BNIT [4], Direct SNN [2], TSSL [5], Event-BP [6], TDBN [7], and LTH [8], on improving the performance of SNNs on the CIFAR10 dataset.
**Firstly**, in the standard UQ, after the convolution between the quantized weights (int) and the inputs (unary spikes), the convolution result (int) is multiplied by a full-precision floating-point (float) scaling factor to ensure high inference accuracy. Consequently, full-precision floating-point multipliers are physically required, leading to high hardware costs in the system that deploys the SNNs. **Secondly**, the LIF operations right after the scaling become floating-point-based due to the floating-point multiplications. The float LIFs require extra floating-point hardware units, increasing hardware overheads in SNN deployment.

To address the issues mentioned above, we propose Multiplier-less Integer-based (MINT) quantization for both the weights and membrane potentials of SNNs, which is illustrated on the right-hand side of Fig. 2 (b). MINT effectively reduces the size of weights and membrane potentials, while also addressing the two hardware issues associated with Uniform Quantization (UQ) by avoiding multipliers and floating-point arithmetic units of LIF.

We summarize our key contributions as follows:

1. We introduce the MINT quantization scheme for the weights and membrane potentials of SNNs. MINT is a uniform quantization scheme that uses fully integer-based operations, eliminating the need for multipliers to perform quantization scaling during inference time.
2. We evaluate MINT with two representative deep SNN architectures (VGG-9 [15] and VGG-16 [15]) on two benchmark datasets (CIFAR10 [16] and TinyImageNet [17]). The experimental results show that our quantized models achieve similar accuracy to the full-precision baselines while significantly reducing memory cost. For example, our 2-bit quantized VGG-16 leads to approximately 93.8% memory footprint reduction with a negligible accuracy degradation (-0.6%) on CIFAR-10.
3. We further deploy the SNN models quantized by MINT and other state-of-the-art SNN quantization schemes on three existing specialized SNN hardware accelerators (SpinalFlow [18], PTB [19], and SATA [20]). While achieving competitive accuracy results with other methods, MINT saves significant hardware resources at deployment. For instance, MINT saves 93% area and 96% dynamic power at the PE-array level on SpinalFlow compared to ADMM-Quant [10]- a state-of-the-art SNN quantization scheme that reports the best accuracy on CIFAR-10 among all other SNN quantization works.
4. We report the speedup of the int8 model quantized by MINT from the fp32 baseline on RTX-2080 Ti and NVIDIA A100. Additionally, we carry out sparsity-related ablation studies on MINT.

## II Related Work

Spiking neural networks (SNNs) have recently been extensively explored as a new generation of low-power deep neural networks for efficient edge device deployment [1, 3]. There are two main types of SNNs: conversion-based SNNs and BPTT-based SNNs.

Conversion-based SNNs are converted from pre-trained artificial neural network (ANN) models, as seen in [21, 22, 23]. Although conversion-based SNNs offer good accuracy, they require a large number of timesteps (50-2000 timesteps), leading to prohibitive energy costs [20], unsuitable for edge devices. BPTT-based SNNs use surrogate gradients to circumvent the non-differentiability problem of LIF and backpropagate the gradients through both spatial and temporal dimensions to train SNNs from scratch [2, 4, 5, 6, 7]. Specifically, we call this training method as backpropagation-through-time (BPTT) [24]. These BPTT-based SNNs can achieve iso-accuracy performance compared to the conversion-based SNNs while running at a much lower number of timesteps (\(<30\)). Our work focuses on the quantization of BPTT-based SNNs.

Many prior works focus on improving the accuracy of BPTT-based SNNs, such as [2, 4, 5, 6, 7]. Some works also aim to reduce the number of timesteps required in SNNs. For example, in [4], the early exit algorithm is explored to reduce the number of timesteps to below 20. In [2], a more efficient input encoding is utilized, further reducing the number of timesteps below 10 while improving accuracy. Other works, such as [8] and [25], explore the lottery-ticket-hypothesis-based pruning method for SNNs, which retains high accuracy while removing more than \(98\%\) of the weight connections.

Weight quantization in SNNs has also been explored in [9, 10, 11]. In [9] and [10], weight quantization together with pruning has been studied. In [10], the ADMM method optimizes a pre-trained full-precision network to quantize the weights into low precision. K-means clustering quantization is used in [9] to achieve reasonable accuracy with 5-bit weight SNNs. However, all of these prior works used full-precision floating-point representation to store the membrane potential. As a result, the total memory footprint of SNNs remains large. Additionally, they require either multipliers for quantization scaling or specialized arithmetic circuits to convert the weights into the format of K-means clustering. In our work, we explore the simultaneous quantization of weights and membrane potentials using fully-integer operations without the need for multipliers to scale convolution results or any extra hardware circuits to support the quantization-aware inference.

One very recent work [11] explored fully integer-based SNNs with both weights and membrane potentials quantized in

Fig. 2: (a) The proportion of membrane potential (denoted as \(U\)) in the overall memory footprint of SNNs when reducing weight precision and increasing mini-batches. (b) An overview of the comparison between our MINT quantization and the standard uniform quantization (UQ). The color blue denotes integer operations while green represents floating-point operations.
the fixed-point format without quantization scaling. However, due to the limited representation range of the fixed-point format, they did not achieve reasonable inference accuracy at low precision. For example, they only achieved 33.5% accuracy on CIFAR10 with 2-bit weights and 8-bit membrane potentials, whereas our quantization scheme achieved 88.39% with both weights and membrane potentials quantized to 2 bits. We summarize the methodology differences between our MINT quantization scheme and other prior SNN quantization methods in Table. I. Detailed quantitative comparisons with prior works are presented in the experimental section.

It is important to note that while several other prior works have explored integer quantization of SNNs [26, 27], they have primarily focused on very shallow networks with a local learning rule that is not suitable for large-scale image classification tasks. Furthermore, they did not consider the quantization of the membrane potential, making their work orthogonal to ours.

## III Preliminaries

### _Spiking Neural Networks_

SNNs process unary spike trains over multiple discrete timesteps through Leaky-Integrate-and-Fire (LIF) neurons which introduce non-linearity and capture temporal information in the network.

We describe the behavior of LIF neurons in formal mathematical equations. At timestep \(t\), the LIF neurons in layer \(l\) first convolve the input spikes with the weights, and the resulting values are stored in a dedicated memory called the membrane potential. Since the input spikes are unary, the convolution operation in the LIF neurons simplifies to a simple accumulation instead of a multiply-and-accumulate (MAC) operation, which can be implemented without multipliers. The accumulated potential at the current timestep is added to the residual memory potential from the previous timestep \(t-1\). We refer to this step as the "membrane potential update" stage, which we define as follows:

\[\mathbf{H}_{l}^{(t)}=\ \mathbf{W}_{l}\mathbf{S}_{l-1}^{(t)}+\mathbf{U}_{l}^{(t-1)}. \tag{1}\]

The \(\mathbf{W}_{l}\) denotes the weight matrix at layer \(l\) and \(\mathbf{S}_{l-1}^{(t)}\) denotes the input spike matrix from layer \(l-1\) at timestep \(t\). The residual membrane potential matrix from the previous timestep \(t-1\) at layer \(l\) is denoted as \(\mathbf{U}_{l}^{(t-1)}\) and the matrix of membrane potential before reset is defined as \(\mathbf{H}_{l}^{(t)}\).

The membrane potentials before reset are then compared to a pre-set threshold \(v_{th}\). If the potential exceeds the threshold, the neuron will generate an output spike, otherwise, no output spike will be produced. The output spike matrix for layer \(l\) at timestep \(t\) is denoted as \(\mathbf{S}_{l}^{(t)}\). We refer to this step as the "firing" stage and formulate it as follows:

\[\mathbf{S}_{l}^{(t)}=\ \Theta(\mathbf{H}_{l}^{(t)},v_{th}), \tag{2}\]

where

\[\Theta(x,y)=\left\{\begin{array}{ll}1&\quad x>y\\ 0&\quad\text{else.}\end{array}\right.\]

Finally, we calculate the residual membrane potential matrix \(\mathbf{U}_{l}^{(t)}\) at layer \(l\) and timestep \(t\). First, we reset the membrane potential matrix \(\mathbf{H}_{l}^{(t)}\) according to the output spikes generated at this timestep and the selected reset mode. In the hard reset mode, the membrane potential is reset to \(0\) upon output spike generation. In contrast, the soft reset mode reduces the membrane potential by \(v_{th}\). We then multiply the membrane potential after reset by a leakage factor \(\tau\) and store it back in memory to be used in the next timestep. We refer to this stage as the "reset" stage:

\[\mathbf{U}_{l}^{(t)}=\left\{\begin{array}{ll}\tau\mathbf{H}_{l}^{(t)}\left(1-\mathbf{S}_ {l}^{(t)}\right)&\quad\text{hard reset}\\ &\\ \tau\left(\mathbf{H}_{l}^{(t)}-\mathbf{S}_{l}^{(t)}v_{th}\right)&\quad\text{soft reset} \end{array}\right. \tag{3}\]

We further illustrate the spatial and temporal dynamics of one LIF neuron in Fig. 3.

### _Uniform Quantization_

Uniform fixed-point quantization methods have been extensively studied by prior works [13, 14, 28]. The uniform fixed-point quantization works as an affine mapping between the fixed-point number vector \(\mathbf{q}\) and the floating-point number vector \(\mathbf{r}\) of the form:

\[\mathbf{r}=\alpha\left(\mathbf{q}-Z\right), \tag{4}\]

where both the \(\alpha\) (scaling factor) and \(Z\) (zero-point) are the quantization parameters.

For \(b\)-bit quantization, \(\mathbf{q}\) consists of integers representing one of the \(2^{\text{b}}\) quantized levels in the range of \([0,2^{b}]\). The scaling factor \(\alpha\) is an arbitrary 32-bit floating number that

Fig. 3: Illustration of the temporal and spatial dynamics of the LIF neuron, where the temporal dynamics refer to the LIF operations across different timesteps, and the spatial dynamics refer to the convolution operation between input spikes and weights.
scales the quantized levels to best fit the distribution of the original values in \(\mathbf{r}\). The zero-point \(Z\) is an integer that ensures \(\mathbf{r}=0\) can be exactly represented by one of the quantized levels. We empirically find that zero-point is not necessary for quantization in SNNs, thus we do not include the zero-point \(Z\) in the rest of this work.

For instance, assume that we have a full precision weight matrix \(W^{l}\) for layer \(l\), then the \(b\)-bit uniform quantization of the weight is defined as:

\[\mathbf{W}_{l}=\alpha\mathbf{\hat{W}}_{l}, \tag{5}\]

where the \(\mathbf{\hat{W}}_{l}\) is the quantized weight matrix with each of its elements represented by a \(b\)-bit integer. The value of scaling factor \(\alpha\) is calculated from

\[\alpha=\frac{\max\left(|\tanh(\mathbf{W}_{l})|\right)}{2^{b-1}-1}, \tag{6}\]

and the quantized weight matrix \(\mathbf{\hat{W}}_{l}\) is then represented as

\[\mathbf{\hat{W}}_{l}=\left|\left(\frac{\mathbf{W}_{l}}{\alpha}\right)\right| \tag{7}\]

The \(\lfloor\cdot\rceil\) denotes the rounding function that rounds the input into the nearest integer.

## IV Analysis of Quantization in SNNs

### _Naive Uniform Quantization of SNNs_

We begin our analysis by applying a naive approach of uniform quantization to the \(l\)-th layer of SNNs. We assume a scaling factor of \(\alpha_{1}\) for the weight matrix \(\mathbf{W}_{l}\), \(\alpha_{2}\) for the residual membrane potential matrix at the previous timestep \(\mathbf{U}_{l}^{(t-1)}\), and \(\alpha_{3}\) for the membrane potential matrix at the current timestep \(\mathbf{U}_{l}^{(t)}\). It is important to note that we do not consider the use of a zero-point and assume the reset type to be hard. The LIF update equations for the naive uniform quantized SNNs are given below:

\[\mathbf{H}_{l}^{(t)}= \ \alpha_{1}\mathbf{\hat{W}}_{l}\mathbf{S}_{l-1}^{(t)}+\alpha_{2}\mathbf{\hat {U}}_{l}^{(t-1)}, \tag{8}\] \[\mathbf{S}_{l}^{(t)}= \ \Theta(\mathbf{H}_{l}^{(t)},v_{th}),\] (9) \[\alpha_{3}\mathbf{\hat{U}}_{l}^{(t)}= \ \tau\mathbf{H}_{l}^{(t)}(1-\mathbf{S}_{l}^{(t)}). \tag{10}\]

We further illustrate the dataflow of the naive quantized LIF neuron in Fig. 4. In the naive uniform quantized LIF, we observe that although the convolution operations between the weights and input spikes are in the integer domain (colored in blue), all other computations are in the floating-point domain (colored in green). Moreover, there are three 32-bit floating-point multiplications due to the scaling factors, \(\alpha\), along the datapath, which undermines the benefits of the simplified arithmetic units of SNNs. We do not consider the operation of leakage as a multiplication since \(0.5\) is a very common leakage factor used by prior literature [4, 8] that can be efficiently implemented with a shift operation. We observe that quantizing the membrane potentials \(\mathbf{U}_{l}^{(t)}\) and \(\mathbf{U}_{l}^{(t-1)}\) into integers does not help accelerate any operations along the datapath. As a result, it is better to keep them at floating-point precision, thereby eliminating the need for floating-point multiplications for \(\alpha_{2},\alpha_{3}\). Still, there will be one 32-bit floating-point multiplication for \(\alpha_{1}\) and other floating-point operations remaining on the datapath. These operations will incur additional energy and area overhead for the SNN hardware. We present a comparison of the relative energy and area costs between integers and floats for several arithmetic units in Table II.

### _Role of Scaling in Quantized SNNs_

In this section, we reflect on the role of scaling in SNNs and check whether it is possible to remove it, thus eliminating the multiplication involved with scaling.

We start the examination by substituting Eq. 8 into Eq. 10. Assume there is no output spike generated at timestep \(t\). Then we will have the full equation for calculating the residual membrane potential

\[\alpha_{3}\mathbf{\hat{U}}_{l}^{(t)}= \ \tau\left(\alpha_{1}\mathbf{\hat{W}}_{l}\mathbf{S}_{l-1}^{(t)}+\alpha_{2} \mathbf{\hat{U}}_{l}^{(t-1)}\right), \tag{11}\]

which can be rewritten as

\[\mathbf{\hat{U}}_{l}^{(t)}= \ \tau\left(\frac{\alpha_{1}}{\alpha_{3}}\mathbf{\hat{W}}_{l}\mathbf{S}_{l -1}^{(t)}+\frac{\alpha_{2}}{\alpha_{3}}\mathbf{\hat{U}}_{l}^{(t-1)}\right) \tag{12}\]

In Eq. 12, the only two non-integers left are \(\frac{\alpha_{1}}{\alpha_{3}}\) and \(\frac{\alpha_{2}}{\alpha_{3}}\). Assume \(\tau=0.5\). In order to remove those two full-precision floating-point multiplications, one straightforward method is to have \(\alpha_{1}=\alpha_{2}=\alpha_{3}\). By having all the scaling factors equal to each other, we manage to transform Eq. 12 into:

\[\mathbf{\hat{U}}_{l}^{(t)}= \ \tau\left(\mathbf{\hat{W}}_{l}\mathbf{S}_{l-1}^{(t)}+\mathbf{\hat{U}}_{l}^{( t-1)}\right). \tag{13}\]

The Eq. 13 now only consists of integer values and integer operations. Moreover, there is no multiplication required for scaling.

Fig. 4: Illustration of naive uniform quantization applied to the LIF neuron. The blue color indicates integer values and operations, while the green color represents floating-point operations and values.
### _Review the Firing in Quantized SNNs_

In Eq. 13, we manage to eliminate the multiplication involved in the datapath for calculating the residual membrane potential. We then examine the other datapath that generates the output spike.

We again substitute Eq. 8 into Eq. 9 and find that the firing function will only generate an output spike if the following condition is valid:

\[\alpha_{1}\hat{\mathbf{W}}_{l}\mathbf{S}_{l-1}^{(t)}+\alpha_{2}\hat{\mathbf{U}}_{l}^{(t-1) }>v_{th}. \tag{14}\]

We can apply the previous idea of maintaining the same scaling value \(\alpha=\alpha_{1}=\alpha_{2}\). Then we divide \(\alpha\) into both sides of the inequality. The firing condition in Eq. 15 is thus of the form:

\[\hat{\mathbf{W}}_{l}\mathbf{S}_{l-1}^{(t)}+\hat{\mathbf{U}}_{l}^{(t-1)}>\frac{v_{th}}{ \alpha}. \tag{15}\]

We have empirically observed that \(\alpha\) is always greater than zero, which means the direction of the inequality in Eq. 15 is always preserved. Therefore, we can compute \(\frac{v_{th}}{\alpha}\) offline, which eliminates the need for any multiplication operations along the datapath. Additionally, since the left-hand side of Eq. 15 is always an integer, we can transform the equation into the following form and still achieve the same firing results:

\[\hat{\mathbf{W}}_{l}\mathbf{S}_{l-1}^{(t)}+\hat{\mathbf{U}}_{l}^{(t-1)}\geq\left\lceil \frac{v_{th}}{\alpha}\right\rceil, \tag{16}\]

where \(\lceil\cdot\rceil\) is the ceil operation. We define this new integer firing threshold as \(\theta\).

## V Multiplier-less Integer SNN Quantization

### _Inference Datapath_

As discussed in Sec. IV-B and IV-C, by sharing the quantization scaling factor between weights and membrane potentials across timesteps for each layer, we can implement our quantization scheme using integer-only arithmetic that requires no multiplications during inference.

The inference algorithm is described in Alg. 1. We first perform a convolution operation between the input spikes and weights. As the input spikes are unary and the weights are quantized to integers, the convolution operation is simplified to an integer-based accumulation. Subsequently, the membrane potential before reset \(\mathbf{H}_{l}^{(t)}\) is computed by adding the integer convolution result \(\mathbf{X}_{l}^{(l)}\) and the quantized integer residual membrane potential \(\mathbf{U}_{l}^{(t-1)}\).

Next, the integer membrane potential before reset \(\mathbf{H}_{l}^{(t)}\) is compared with the precomputed integer firing threshold \(\theta\) discussed in Sec. IV-C. Based on the comparison results, the unary output spikes are generated. If no output spike is generated, the integer \(\mathbf{H}_{l}^{(t)}\) is left shifted by 1, which is the same as multiplying by a leak factor with the value of 0.5. The result is then cast down to the target bitwidth and stored as the residual membrane potential \(\mathbf{U}_{l}^{(t)}\). Alternatively, if an output spike is generated, a zero is stored as the residual membrane potential.

This inference path is repeated for all other layers and timesteps in SNNs, utilizing integer arithmetic and eliminating the need for multiplications. Fig. 5 illustrates the dataflow for MINT quantized LIF neurons.

### _Training with Simulated Quantization_

During the forward path of training, we apply the following simulated quantization function \(Q\) to weights and membrane potentials:

\[Q(r,n)=\frac{\left\lfloor\mathrm{clamp}\left(\frac{r}{\mathrm{s}(r)},-1,1 \right)\mathrm{s}(n)\right\rceil\mathrm{a}(r)}{\mathrm{s}(n)}, \tag{17}\]

where

\[\mathrm{s}(n) =2^{n-1}-1,\] \[\mathrm{a}(r) =\max(|\tanh(r)|),\] \[\mathrm{clamp}(r,a,b) =\min(\max(r,a),b).\]

Here, \(r\) represents a full-precision floating-point number to be quantized, and \(n\) represents the number of bits assigned to the quantized integer. As we focus on uniform quantization in this work, we fix \(n_{w}\) for all layers' weights and \(n_{u}\) for the membrane potentials across all layers and timesteps in our experiments. We illustrate the position where the fake quantization is inserted in the forward path of training in Fig. 6.

During backward propagation, we use full-precision floating-point gradients for all quantities. We employ the straight-through estimator [13] to approximate the derivative

Fig. 5: Illustration of our MINT quantization of the LIF neuron. The blue color indicates the integer values or operations.
for the non-differentiable rounding function \(\lfloor\cdot\rceil\). Hence, we can simulate the quantization errors during inference for our models at training time.

### _Shareable Quantization Scale_

As discussed in Sec. IV-B and IV-C, our method requires sharing one quantization scaling factor or scale between weights and membrane potentials. This means that we will have a single quantization scale \(\alpha\) for each layer that can be shared between these two quantities. During inference, we replace the firing threshold \(v_{th}\) with the integer value \(\left\lceil\frac{v_{th}}{\alpha}\right\rceil\). During training, we apply the fake quantization \(Q\) to weights and membrane potentials along the forward path, and the \(\text{a}(r)\) term in Eq. 17 is replaced with \(\alpha\).

Further, instead of directly sharing the quantization scales between weights and membrane potentials, we make the shared scale \(\alpha\) a learnable parameter [29]. We update \(\alpha\) with full-precision gradients during backward propagation in the training process.

## VI Evaluation under Existing SNN Systems

To better understand the potential benefits of multiplier-less integer quantization on hardware, we evaluated MINT on several existing ASIC SNN hardware systems. Specifically, we focused our evaluation on systolic-array-based ASIC accelerators for SNNs [18, 19, 20]. These accelerators utilize a dataflow architecture with high data reuse between processing elements (PEs), which is commonly found in ANN accelerators.

### _Exisiting SNN Accelerators_

**SpinalFlow**[18] is a systolic-array-based SNN inference accelerator that utilizes the extremely sparse temporal encoded SNNs through unique tick-batch dataflow. Inside each PE, several small scratchpad memories hold the membrane potential and weights. Additionally, an adder and a comparator support the LIF operation inside every PE. Each PE generates the output for only one neuron across the entire timestep.

**PTB**[19] is another state-of-the-art systolic-array-based SNN inference accelerator that supports highly parallel processing of rate-coded SNNs. Inside each PE of the PTB, there are small scratchpads for holding the weights, input spikes, and membrane potentials. Each PE equips LIF arithmetic units and an accumulator for carrying out the convolution. Although several PEs divide the workload across timesteps, each PE still needs to complete a full convolution and LIF cycle (Eq. 1 - 3) for at least one timestep.

**SATA**[20] is a systolic-array-based training accelerator for BPTT-based SNNs. To minimize the data movement of multi-bit operands across timesteps, SATA combines the weight stationary dataflow with the tick-batch dataflow of SpinalFlow. SATA has a much larger size of both the input scratch pad and the weight scratch pad. We will use SATA's energy estimation tool to get an estimation of the energy reduction from our MINT-quantized models.

Although these prior SNN accelerator designs vary in their architectural details, they share a common feature: to minimize the data movements of multi-bit partial sums and membrane potentials, they adopt an output-stationary-like dataflow. In these designs, one LIF neuron is mapped to one dedicated PE for the entire input interval, as shown in Figure 7. This design feature implies that the hardware units inside each PE must support the entire LIF operation as shown in Eq. 1-Eq. 3, which means that all prior works [18, 19, 20] have embedded LIF units inside each PE.

### _Impact of Quantization on SNN Accelerator Design_

As discussed in Sec. IV-A, supporting naive uniform quantization on SNNs requires at least one 32-bit floating-point multiplier and several floating-point arithmetic units along the LIF computation datapath. This means that to support uniform quantization on output-stationary-oriented SNN accelerators, we need to add quantization support to all PEs, resulting in each PE having its own full-precision multipliers and other floating-point arithmetic units. This would cause significant overheads at the PE level, especially since the array size in prior designs is at least 128. Although larger PE array size increases throughput, it brings additional hardware cost for uniform quantization of SNNs. Fig. 8 illustrates the difference in area breakdown of the 4-bit version of SpinalFlow [18] before and after adding uniform quantization support to the PEs.

Fortunately, our MINT quantization scheme eliminates the need for multipliers and all floating-point arithmetic units

Fig. 6: (a) The inference path that shows the use of MINT arithmetic to compute the membrane potential and generate output spikes. (b) The training path illustrates the fake quantization inserted for both weights and membrane potentials to simulate the quantization error that will occur during the inference time. The blue color indicates the integer values or operations and the green indicates the float values or operations.

Fig. 7: Illustration of the output-stationary dataflow that is commonly adopted by the prior SNN accelerators
inside PEs, saving significant hardware resources for the entire system.

## VII Experiments

### _Experimental Settings_

#### Vii-A1 Software Configuration

**Models and datasets.** We choose two representative deep network architectures: VGG-9 [15] and VGG-16 [15]. We evaluate our quantization schemes on two public image-classification datasets: CIFAR-10 [16] and TinyImageNet, which contains 200 classes from the ImageNet dataset [17].

**Baselines.** We use the model trained with 32-bit floating-point (fp32) weights and membrane potentials as our baseline. To validate our quantization scheme, we compare our MINT quantized model of varying precisions to the baseline. Additionally, we compare our quantization scheme with three prior SNN quantization works: ST-Quant [9], ADMM-Quant [10], and STBP-Quant [11].

**Training.** We use the state-of-the-art direct encoding technique [2] to train our SNNs. This method has demonstrated remarkable effectiveness in training BPTT-based SNNs, enabling them to achieve high accuracy with only a few timesteps. Unless otherwise specified, all experiments in the following sections use timesteps \(T\) = 4. We include all the training configurations in Table. III. Additionally, we initialize our shareable quantization scaling \(\alpha\) and scale its gradients with the hyperparameters in [29].

**Implementation.** We implement the SNN training framework for our MINT quantization in PyTorch. We also implement VGG-16 using the CUDA source code to test memory cost and speedup. 2D convolutions and the max-pooling operations are implemented using CuDNN for achieving accurate GPU statistics. To compare the 8-bit quantized model with the full precision baseline, we implement the network with \(\text{CUDNN\_DATA\_INT8}\) and \(\text{CUDNN\_DATA\_FLOAT}\).

#### Vii-A2 Hardware Configuration

We synthesize all the arithmetic units using Synopsys Design Compiler at 400MHz using 32nm CMOS technology to obtain area, dynamic power, and static power. We use CACTI [30] to simulate on-chip SRAM and off-chip DRAM to obtain memory statistics. We also build a PyTorch-based simulator to estimate latency and energy based on standard modeling strategies [19, 20, 31].

To evaluate our quantization scheme and other quantization methods on existing SNN hardware systems, we synthesize [18, 19] from scratch, all architecture configurations identical to the original work. The only changes we make are adjusting the operand sizes of the arithmetic units according to the bit-width of the weights and membrane potentials. Additionally, we synthesize another version with the PE arrays supporting the naive uniform quantization (adding 32-bit multipliers and using floating-point LIF units). We perform energy estimation using SATA tool [20].

### _Experimental Results_

#### Vii-B1 Accuracy and Convergence

Table. IV summarizes the accuracy of MINT quantization scheme for various combinations of weight and membrane potential precision for different SNN models and datasets. For instance, w8u8 represents a network with both weights and membrane potentials quantized to 8 bits using MINT quantization. The fp32 represents the full-precision baseline. Although the fp32 baseline achieves the highest accuracy for most of the datasets and networks, MINT achieves a similar level of accuracy while significantly reducing the memory footprint of the models.

On the CIFAR10 dataset, the accuracy of the relatively shallow VGG-9 network does not drop significantly even when both weights and membrane potentials are compressed to 2-bit precision. This result suggests that very low bit-width integer operations can achieve similar accuracy as the full-precision floating-point baseline on small SNN networks. On the other hand, the accuracy of the VGG-16 network drops

Fig. 8: Area breakup of (a) 4-bit Spinalflow without naive uniform quantization supports and (b) with naive uniform quantization supports
monotonically with the decreasing precision of weights and membrane potentials. However, even at w2u2 precision, the accuracy of the quantized network remains very close to the fp32 baseline, with less than 0.6% top-1 accuracy drop.

On the more challenging TinyImageNet dataset, we observe an accuracy drop at the precision of w2u2 for the VGG-9 network. However, the accuracy drop remains within an acceptable range (less than 1.5% compared to fp32 baseline). Remarkably, MINT achieves higher accuracy than the full-precision baseline on the VGG-16 network, suggesting that our method may serve the purpose of regularization.

#### Iv-B2 Memory Saving and Speedup

Memory Saving.In this section, we first present the memory footprint reduction of our quantization scheme with different precision combinations of weights and membrane potentials, as shown in Fig. 9, using a batch size of one. We observe a reduction in memory usage proportional to the reduction in weight precision compared to the fp32 baseline. For instance, our w2u2 VGG-16 model compresses approximately 93.8% of the memory footprint from the fp32 baseline. Therefore, with a batch size of one, the model size reduction primarily comes from weight quantization.

However, to improve inference speed, prior works [2, 4, 5, 6, 7] commonly use mini-batch sizes larger than 1. In such cases, membrane potential quantization becomes important. We illustrate the importance of membrane potential quantization in Fig. 10. When a single batch size is used, we observe that solely quantizing the weight to 4 bits and leaving the membrane potential at full precision can still achieve approximately 70% memory reduction from the fp32 VGG-16 baseline on TinyImageNet. However, with a batch size of 4, the 4-bit weight quantization without the membrane potential quantization can only reduce around 40% of the total memory footprint of the baseline. Moreover, with a batch size of 16, the 4-bit weight quantization can only compress the total memory footprint by 15%, while quantizing the membrane potential to 4 bits can further reduce 72.4% of the total memory footprint. Please note that, for visualization purposes, we only illustrate the importance of membrane potential using a relatively small batch size (16). However, a batch size that is larger than 50 is very commonly used in prior SNN works [2, 4, 5, 6, 7] while keeping the membrane potentials at full precision. Thus, we emphasize the importance of membrane potential quantization in this work.

Speedup.In Fig. 11, we present the layerwise inference latency of our quantization scheme at w8u8 precision compared to the fp32 baseline for VGG-16 on CIFAR-10 and TinyImageNet, tested on two different CUDA devices: RTX-2080Ti and NVIDIA A100.

Overall, we observe a trend of faster inference with the 8-bit quantized model across all devices. Specifically, we find that the int8 model consistently runs faster than the fp32 baseline on the first two layers, while also demonstrating a significant acceleration on the last three layers in most cases. MINT achieves an overall acceleration of 4.5%(9.1%) for CIFAR-10(TinyImageNet) on RTX-2080Ti and 6.2%(17.4%) on A100.

Overall, we observe that the larger the input image, the more significant the acceleration, and that A100 provides better support for int8 inference, leading to greater benefits from int8 operations.
(\(T=8\) for all methods) when making the comparison. We used the same number of mini-batches as the prior works when comparing the total memory footprint (50 for STBP-Quant and ADMM-Quant, and 32 for ST-Quant). The prior works used a high precision of membrane potential, which incurs more memory cost when mini-batches are used. We quantized the membrane potential to the same precision as the weights to adopt our multiplier-less integer quantization scheme and save more memory.

We show the comparison results in Table. V. In general, our quantization scheme is the first to consider compressing both the weights and membrane potentials to a very low precision (less than 8 bits) while preserving very similar accuracy with the state-of-the-art baseline. At 8-bit, 4-bit, and 2-bit weight quantization, we consistently outperformed STBP-Quant, another prior SNN quantization work that considered membrane potential quantization. STBP-Quant achieved low accuracy of 33.5% at w2u8 precision, while our work retained accuracy at 88.4% with both membrane potential and weight quantized to 2 bits.

When compared with the full-precision membrane potential baseline, our quantization scheme encounters an acceptable level of accuracy degradation (-0.84% to -1.28%) across 8-bit, 5-bit, 4-bit, and 2-bit weight quantization. This is because the full-precision membrane potential can to some extent help alleviate the information loss brought by weight quantization. However, we achieved significant memory footprint reduction by quantizing the weights and membrane potential simultaneously. On average, our quantized SNN models only require 2.8% of the total memory footprint of ADMM-Quant and 8% of ST-Quant.

Furthermore, for the baseline with the highest accuracy, ADMM-Quant, quantization scaling (Eq. 5) has been used to compensate for the quantization error. As we discussed in Sec. IV-A, quantization scaling would require high precision multipliers and floating-point arithmetic units, which incur high hardware costs on SNN systems (Sec. VI-B). Our method, on the other hand, does not require any multipliers or floating-point units for scaling and thus will receive more hardware benefits at deployment time. We will discuss more hardware benefits of our method in the next section.

### _Evaluation on Existing Hardware Systems_

In this section, we demonstrate the hardware benefits of our MINT quantization scheme. We evaluate our method on SpinalFlow [18] and PTB [19], with a focus on the PE-array level analysis. The PE-array level evaluation measures the PE-array area, dynamic power, and static power.

We divide this evaluation process into two categories: **scale vs. non-scale** and **high-precision vs. low-precision**.

In the **scale vs. non-scale** evaluation, we compare the PE-array level evaluation results at the same operand bit-width, between our method (high accuracy without multipliers and fully integer) and ADMM-Quant [10] (highest accuracy but requires multipliers for quantization scaling and not fully integer).

For the **high-precision vs. low-precision**, we compare the PE-array level evaluation results at different operand sizes between our method (fully integer and multiplier-less that achieves high accuracy with small operand size) and STBP-Quant [11] (fully integer and does not require multipliers, but achieves reasonable accuracy only with large operand size).

Additionally, we evaluate our quantization scheme on SATA [20] to estimate the total forward energy consumption from the system level.

Fig. 11: Layer-wise speedup results for VGG-16 using int8 weight and membrane potential precision compared to fp32 precisions on CIFAR10 and TinyImageNet. Results are shown for RTX 2080Ti (a) and A100 (b) GPUs.
#### Vii-C1 Evaluation on SpinalFlow and PTB

**Scale vs. non-scale:** Although our quantization scheme has slightly lower accuracy performance than ADMM-Quant (on average \(-1.1\%\)), our hardware deployment cost is much lower compared to ADMM-Quant on both SpinalFlow and PTB. In Fig. 12, we show the PE-array level comparison results between ADMM-Quant and our quantization scheme on both SpinalFlow and PTB. We first evaluate the area, dynamic power, and static power of the PE array between our methods and ADMM-Quant with full precision membrane potential (admm-Ufp). We find that the PE array implemented with our quantization scheme saves 76% (93%), 86% (95%), and 93% (96%) total PE-array area with weights of 8 bits, 4 bits, and 2 bits on SpinalFlow (PTB), respectively. Furthermore, our method reduces the dynamic power cost of ADMM-Quant by 84%, 96%, and 98% on SpinalFlow and 94%, 94%, and 95% on PTB across different weight precisions. The static power cost reduction follows a similar trend as the dynamic power on both hardware systems.

We further assume that ADMM-Quant will work with membrane potential quantized to the same precision as the weights (admm-Uq) and re-compare it with our quantization scheme. It turns out that the trend of reduction still holds. This observation suggests that inserting full-precision multipliers and floating-point arithmetic units into each PE will hurt the overall PE-array level performance significantly.

**High-Precision vs. Low-Precision** We further compare our quantization scheme with STBP-Quant, which produces a fully integer quantized network without the use of scaling in the quantizer. We evaluate the CIFAR-10 accuracy trends of both methods as we reduce the operand size of the processing element (PE) array from w8 to w5, w4, and w2.

In Fig. 13, we present the accuracy vs. PE-array area results obtained with both methods on SpinalFlow. Our method achieves high accuracy while scaling down the total PE-array area by approximately 80%. On the other hand, STBP-Quant exhibits accuracy degradation of more than 50% during the PE-array scaling down. We observe similar trends on PTB.

#### Vii-C2 Evaluation on SATA

We use SATA [20] to study the total inference energy difference across different precisions of weights and membrane potentials. The result is presented in Fig. 14. As our MINT quantization does not introduce any hardware overheads to the processing element (PE), the computation energy and memory movement energy scale down linearly with the operands' precisions.

We observe that our w2u2 VGG-9 network achieves an approximately 87.3% total inference energy reduction on a single image of CIFAR-10 when compared to the one with 16-bit weights and membrane potentials.

## VIII Ablation Studies

#### Vi-1 Study on Spike Sparsity

We investigate the spike activity of our MINT-quantized model by measuring the average spike

Fig. 14: Normalized computation and memory energy cost for our MINT-quantized VGG-9 SNN models with different weight and membrane potential precision on CIFAR-10.

Fig. 12: Comparisons of area, dynamic power, and static power between our method and two baseline methods: ADMM-Quant with full precision membrane potential (admm-Ufp), and ADMM-Quant with quantized membrane potential (admm-Uq).

Fig. 13: CIFAR-10 accuracy vs. PE-array area on SpinalFlow and PTB for STBP-Quant and our MINT quantization.
sparsity for the network. This is done by computing the average spike sparsity across all inputs and all timesteps. For example, for a 4-timestep SNN, a 92% sparsity indicates that each LIF neuron is expected to spike only \((1-0.92)*4=0.32\) times throughout the inference.

We compare the average spike sparsity of our MINT-quantized VGG-9 models on CIFAR-10 with other state-of-the-art full-precision SNN works. The results show that our MINT quantization does not incur any sparsity reduction. By maintaining the same level of high spike sparsity as other SNN works, our quantized models can enjoy the same amount of computation energy reduction [4, 20, 32, 33].

#### V-C2 Study on Reset Mode

We further investigate the effects of different LIF reset modes on our MINT quantization scheme by comparing the hard reset mode, which is used in MINT, with the soft reset mode, a popular reset mode used in many prior SNN works [9, 35]. In Fig. 16, we observe that the w2u2 VGG-16 model trained with the hard reset mode has better training accuracy and convergence speed than the one trained with the soft reset mode on CIFAR-10.

Although our MINT quantization scheme currently uses the hard reset mode, we note that it also supports the soft reset mode. To implement the soft reset LIF in MINT, we simply replace the \(0\) in line 5 of Alg. 1 with \((H_{l}^{t}-v_{th})\).

## IX Conclusion

In this paper, we have introduced multiplier-less integer-based (MINT) quantization for both the weights and the membrane potentials of SNNs. By sharing the quantization scale between the weights and membrane potentials and transforming the LIF update equations, we are able to significantly compress the memory footprint of SNNs while dodging all the hardware overheads brought by the original uniform quantization. Our method achieves comparable accuracy to full-precision baselines and other state-of-the-art SNN quantization methods. At the same time, our method achieves a smaller memory footprint by quantizing the membrane potential, which is a memory-hungry quantity that has been mostly neglected in prior works. Our MINT method also achieves huge hardware resource reduction at deployment time on several existing SNN hardware platforms due to the elimination of multipliers and floating-point units.

In conclusion, our proposed MINT quantization method offers a practical and effective solution for reducing the memory footprint and hardware resource requirements of SNNs without sacrificing accuracy. Our method has the potential to enable the deployment of large-scale SNNs on resource-constrained devices, opening up new possibilities for SNN-based edge computing.

## X Acknowledgements

We thank our reviewers for their valuable feedback. This work was supported in part by CoCoSys, a JUMP2.0 center sponsored by DARPA and SRC, Google Research Scholar Award, the National Science Foundation CAREER Award, TII (Abu Dhabi), the DARPA AI Exploration (AIE) program, and the DoE MMICC center SEA-CROGS (Award #DE-SC0023198).
