A Survey of Spiking Neural Network Accelerator on FPGA
Due to the ability to implement customized topology, FPGA is increasingly
used to deploy SNNs in both embedded and high-performance applications. In this
paper, we survey state-of-the-art SNN implementations and their applications on
FPGA. We collect the recent widely-used spiking neuron models, network
structures, and signal encoding formats, followed by the enumeration of related
hardware design schemes for FPGA-based SNN implementations. Compared with the
previous surveys, this manuscript enumerates the application instances that
applied the above-mentioned technical schemes in recent research. Based on
that, we discuss the actual acceleration potential of implementing SNN on FPGA.
According to our above discussion, the upcoming trends are discussed in this
paper and give a guideline for further advancement in related subjects.
Murat Isik
# A Survey of Spiking Neural Network Accelerator on FPGA

###### Abstract

Due to the ability to implement customized topology, FPGA is increasingly used to deploy SNNs in both embedded and high-performance applications. In this paper, we survey state-of-the-art SNN implementations and their applications on FPGA. We collect the recent widely-used spiking neuron models, network structures, and signal encoding formats, followed by the enumeration of related hardware design schemes for FPGA-based SNN implementations. Compared with the previous surveys, this manuscript enumerates the application instances that applied the above-mentioned technical schemes in recent research. Based on that, we discuss the actual acceleration potential of implementing SNN on FPGA. According to our above discussion, the upcoming trends are discussed in this paper and give a guideline for further advancement in related subjects.

 Neural Networks, Spiking Neural Network, FPGA Implementation, Hardware Accelerator, Survey.

## I Introduction

Spiking Neural Network (SNN) has been one of the most extensive research subjects in recent decades. Researchers successfully deployed the related instances in various application scenarios, such as speech recognition, biomedical analysis, and self-driving cars. SNNs are inspired by the biological neural system with the understanding of brain functionalities. The related works are more biologically plausible in both information transmission across neurons and internal neuron signal processing. Therefore, this technique causes a paradigm shift in the field of neural network research. The goal of SNNs is to develop a computational system modeling the behavior of real neurons. However, the complexities of the network model and corresponding computation requirements in SNN inference are rapidly increasing. The trade-off between hardware and power consumption and acceleration performance has become a research topic of importance. This leads to an actual requirement for customized hardware accelerators to achieve higher computing/power efficiency, especially for embedded and lightweight applications. Recent research shows that SNNs have the following excellent hardware implementation features: SNNs communicate across neurons using spikes, which are equivalent to a single bit in terms of logic resources and decrease the logic occupation. Therefore, _Field Programmable Gate Arrays_ (FPGAs) become a potential hardware platform for SNN acceleration, based on its features of logic reconfigurability, high power efficiency, and support of computing parallelization and bit-level operations.

Recent research has deployed SNNs on different platforms, such as _Central Processing Units_ (CPUs), _Graphics Processing Units_ (GPUs) and _Application-Specific Integrated Circuits_ (ASICs). However, due to the restricted memory bandwidth, SNNs on CPU/GPU consume high power overhead with limited throughput [1, 2]. To achieve the best performance and energy efficiency, many researchers have focused on building custom ASICs for accelerating network inference workloads. Despite being an attractive solution, ASICs cannot offer sufficient flexibility to accommodate the rapid evolution of _Neural Network_ (NN) models. The emergence of new types of layers within NN, including branching, elementwise addition, and batch normalization layers, has been seen in more recent models that require flexibility. Further, the high _Non-Recurring Engineering_ (NRE) cost and time for design, verification, and fabrication of a large ASIC chip make it difficult to keep pace with the rapid model improvements in this space [3]. Consequently, FPGAs serve as configurable tools, facilitating the creation of unique logic, which may mitigate the limitations on the execution of neural networks. As an outcome, due to the apparent benefits of the SNN on hardware implementation, one of the current research hotspots is the development of hardware systems supporting SNN inference based on FPGA to achieve high throughput and power efficiency [4].

### _Contributions of Our Survey_

In this manuscript, we survey the upcoming techniques of FPGA-based SNN accelerators and its application instance in various scenarios. The major contributions of our survey are as follows:

* We categorize the widely-used spiking neuron models, spiking signal format, currently explored SNNs, SNN training tools, etc.
* Based on the above-mentioned background, we survey the feasible techniques for hardware design of SNN accelerator on FPGA and analyze the difficulties in the implementation, including accelerator architectures, optimization, and upcoming solutions.
* We enumerated the related application-based SNN acceleration on FPGA.
* We examine the research trends and optimization directions in developing FPGA-based SNN accelerators based on the above works.
### _Organization_

This manuscript is structured as following **Section II** introduces the technology background of _Spiking Neural Networks_. **Section III** discusses the popular implementations and solutions for SNN accelerators on FPGA in recent works. **Section IV** briefly emulates the previous FPGA applications in different scenarios based on SNN. **Section V** analyzes the trend of further work and upcoming research for the SNN accelerator on FPGA. **Section VI** concludes the survey.

## II Background

### _Coding Formats of Spiking Signals_

In recent artificial intelligence research, spiking neural networks have become a state-of-the-art topic for further modeling the working principle of bio-neural systems. Therefore, the input data need to be encoded into a spiking signal format. For instance, Figure 1 shows five most popular spiking coding formats in recent research [5, 6]:

* _Rate Coding Format:_ Rate coding is the most widely-used format in related works. This format converts the input value to a Poisson spiking train with the corresponding fire rate. As shown in Figure 1a, larger input can be represented as the spiking train with a higher fire rate.
* _Time-to-first Spiking Coding Format: Time-to-first Spiking Coding_ is a kind of temporal coding method. Instead of using fire rate to represent different inputs value, Time-to-first coding applies the fire latency in a stimulus duration to convert data as a spike. As shown in Figure 1b, the spike of larger input will be fired earlier. Compared with the rate coding format, time-to-first coding only requires a single spike, data information will be encoded in the temporal latency.
* _Phase Coding Format:_ Above two coding methods need a conversion between data value and fire rate/latency. Different conversion solutions will influence the encoded spike trains. As shown in Figure 1c, work [7] explored another spiking encoding method that directly converts the binary format of input value as the spike train. The weight spike in this work will represent the significance of each input bit.
* _Burst Coding Format:_ To reduce the transmission duration of spiking signal, work [8][9] explored a burst transmission encoding method as shown in Figure 1d. This method encodes the input value as the number of spikes and the inter-spike interval (ISI) in the burst. More intensive spikes in burst represent a larger input.
* _Rank Order Coding Format:_ Figure 1d shows the scheme of _Rank Order Coding_[10]. Instead of applying the spikes latency in stimulus to encode the inputs like _Time-to-first Spiking Coding_ scheme, this encoding method encodes the input information as the spikes fire order ignoring the fire latency. Therefore, for \(N\) pre-neurons, their outputs can be encoded as \(2^{N}\) different data.

### _Spiking Neuron Modelling and the Differences with Classic ANN Neuron_

#### Ii-B1 Spiking Neuron Modelling

A bio-inspired artificial intelligence technology called SNN has become a state-of-the-art technology in recent years. Therefore, the implementations of spiking neurons also mimic the actual natural neural cells. Based on the cell structure modelling, there are three widely-explored spiking neuron models on FPGA accelerator implementations: i) _Hodgkin-Huxley (HH) Model_[11], ii) _Izhikevich Model_[12], iii) _Leaky Integrate-and-Fire Model_ (LIF) [13][14].

* _Hodgkin-Huxley (HH) Model:_ This model simplified the neural cell structure as a RC circuit model as shown in Figure 2.

In this model, \(C\) represents the capacity of the cell membrane. \(R_{K}\) and \(R_{Na}\) mimic the sodium and potassium ion channels in neurons. \(R_{Leaky}\) simulates the leaky channel to lose the charge in the membrane. Therefore, Equation 1

\[\begin{split} I\left(t\right)=I_{C}\left(t\right)+I_{K}\left(t \right)+I_{Na}\left(t\right)+I_{Leaky}\left(t\right)\\ \Rightarrow C\frac{\mathrm{d}U}{\mathrm{d}t}=I\left(t\right)-I_{K} \left(t\right)-I_{Na}\left(t\right)-I_{Leaky}\left(t\right)\end{split} \tag{1}\]

Because the conductance of ion channels in this model is simulated as the function of time and voltage, Equation 1 will be rewritten as:

Fig. 1: Five popular spiking coding formats

Fig. 2: RC Circuit Modelling of Hodgkin-Huxley Spiking Neuron Model
\[I\left(t\right) =C\frac{\mathrm{d}U}{\mathrm{d}t}+g_{K}n\left(t,U\right)^{4}\cdot \left(U-U_{K}\right) \tag{2}\] \[\quad+g_{Na}m\left(t,U\right)^{3}h\left(t,U\right)\cdot\left(U-U_{ Na}\right)\] \[\quad+g_{Leaky}\left(t\right)\cdot\left(U-U_{K}\right)\] \[\begin{cases}\frac{\mathrm{d}n\left(t,U\right)}{\mathrm{d}t}= \alpha_{n}\left(U\right)\cdot\left(1-n\left(t,U\right)\right)-\beta_{n}\left( U\right)\cdot n\left(t,U\right)\\ \frac{\mathrm{d}m\left(t,U\right)}{\mathrm{d}t}=\alpha_{m}\left(U\right) \cdot\left(1-m\left(t,U\right)\right)-\beta_{m}\left(U\right)\cdot m\left(t,U \right)\\ \frac{\mathrm{d}h\left(t,U\right)}{\mathrm{d}t}=\alpha_{h}\left(U\right) \cdot\left(1-h\left(t,U\right)\right)-\beta_{h}\left(U\right)\cdot h\left(t,U \right)\\ \end{cases}\] \[\begin{cases}\alpha_{i}\left(U\right)=p_{\infty}\left(U\right)/\tau_{p} \\ \beta_{i}\left(U\right)=(1-p_{\infty}\left(U\right))/\tau_{p}\end{cases}\]

In Equation 2, \(n\), \(m\), and \(h\) are the conductance parameters for sodium and potassium ion channels, \(p\) represent the steady state values for activation.
* _Izhikevich Model:_ Considering the parameters and complex computing in Equation 2, work [12] simplified the HH model based on bifurcation methodologies as Equation 3 to reduce the number of parameters. \[v^{\prime} =0.04v^{2}+5v+140-u+I\] (3) \[u^{\prime} =a(bv-u)\] \[if\quad v\geq 30mV,\quad then\quad\begin{cases}v\gets c \\ u\gets u+d\end{cases}\] In Equation 3, \(v\) implies the membrane potential (input). \(u\) means recover parameter. \(a\) means how fast the membrane potential recovers, the typical value is 0.02. \(b\) means the responsibility between \(u\) and input \(v\), typical value is 0.2. \(c\) means the reset potential after spiking, the typical value is \(-65mV\). \(d\) means the increase value of \(u\) after reset, typical value is \(2mV\)[15].
* _Leaky Integrate-and-Five Model:_ To further simplify the spiking neuron model, authors in [13, 14] discussed the simplification of the HH model as _Leaky Integrate-and-Fire_ (LIF) model, which has been the most widely-used model in recent research. As shown in Figure 3, LIF further simplified the HH model, the RC modeling formula is Equation 4: \[C\frac{\mathrm{d}U}{\mathrm{d}t}=I\left(t\right)-\frac{V-V_{rest}}{R}\] (4)

#### Ii-B2 Brief introduction of SNN Neurons hardware implementation

Figure 4 shows the structure of neuron processing elements in SNNs hardware implementation. For SNN, when networks apply different neuron models and signal encoding schemes, the structures of neuron processing elements will be different. For instance, in the implementations based on LIF model and rate coding schemes, the neurons can consist of the following submodules: 1) _Multiplier_, 2) _Accumulator_, 3) _Thresholds_, and 4) _Spiking Encoder_:

* _Multiplier_ and _Accumulator_ compute the product sum of weighted spikes based on different data formats. However, storing network weights consumes a large number of on-chip memory resource on FPGA. Considering the limitation of hardware resources, in addition to the common floating-point and fixed-point formats, some low-precision data formats are widely explored in recent research, such as i) _Quantization_, ii) _Binarization_, iii) _Approximation Computation_, and iv) _Posit-floating Computation_: The authors of _Q-SpiNN_[16] and work [17] implemented a _Quantized Spiking Neural Network_ (QSNN) framework to reduce the memory consumption of network weights based on the low-precision integer scheme. Moreover, work _BS4NN_[18] and work [19] further reduced the weight precision to the _Binarized Spiking Neural Network_ (BSNN). Work _AxSNN_[20] explored the application of approximation computation on SNN. The authors of [21] researched the possibility of applying _Posit_-floating computation [22, 23] on FPGA-based SNN accelerators.
* _Thresholds and Spiking Encoding_: As shown in Figure 4, the parameters of thresholds will be the leaky rate, refractory time, fire thresholds, etc. For some coding schemes, such as _Phase Coding_, the encoding module will be necessary. More technique details about the spiking neuron implementation will be discussed in Section III.

### _SNN Model Training and Conversion Tools_

The simulation and evaluation of SNN accelerator implementations on FPGA require trained models. However,

Fig. 4: Neuron Structure of Spiking Neural Networks

Fig. 3: RC Circuit Modelling of Leaky Integrate-and-Fire Spiking Neuron Model
because of the difference between SNN and classic ANN, SNN training is still a challenge in recent research and limits the application of SNNs. Some upcoming toolkits support the SNN model generation-based conversion from trained ANN models or directly SNN training.

#### Ii-A1 ANN Model Conversion

Because of the difficulties in synaptic weight learning on SNN training, some works explored the solution that converts an ANN model to SNN. _Spiking Neural Network Conversion Toolbox_[24] implemented one python-based conversion tool from ANN to SNN supporting various ANN training libraries, such as _Keras_, _PyTorch_, _Caffe_, etc. It also supports the SNN model export for evaluation based on SNN simulation tools, like _pyNN_[25], _Brain2_[26], _sPyNNaker_[27], etc.

#### Ii-A2 SNN Model Training

Compared with the ANN conversion, direct training of SNN can achieve higher accuracy. Some researchers explored the related SNN training algorithms [28, 29, 30]. Based on previous works, snnTorch [31] implemented a PyTorch-based SNN training acceleration library. This library supports the training of _Spiking Convolution Neural Network_ (SCNN) and _Spiking Long-Short Term Memory Network_ (SLSTM) based on the LIF neuron model and rate/latency coding. The training based on this work can also be accelerated by applying the GPU platform through PyTorch.

## III Techniques

### _Topology Conversion_

Spiking neural networks are biologically inspired _Artificial Neural Networks_ (ANN), which are more power-efficient, but the discrete nature of the spikes makes it difficult to train an SNN. Thus, much work has been done to convert the topology of a trained network from ANN or CNN to SNN while copying the trained weights and adjusting the spiking threshold potential in SNN. In one such work [32], Deng et al. analyze the ANN to SNN conversion error and propose a method to transfer weights with no loss of accuracy. The authors modify the ReLU function on the source ANN by thresholding the maximum activation and reducing the simulation time to 1/10th of a typical SNN simulation time. Followed by a shifting operation to balance the output frequency is termed as the threshold balancing mechanism [33].

The ANN-SNN conversion techniques have managed to achieve deeper architectures but most of these techniques are based on the assumption that spike patterns of SNN a time-sampled ANN. Thus, the temporal properties of SNN are ignored. Some training algorithms like _Backpropagation Through Time_ (BPTT) could solve this by training through time but it introduces vanishing gradients. Samadzadeh et al. in [34] instead initialize the weights with non-spiking training data. For the first few epochs, the SNN is trained like an ANN where the output activation function is changed to leaky ReLU. After which the activation function is modeled as a step function with the LIF neuron. By utilizing the skip connections in _ResNet_ architecture a _Spiking ResNet_ (STS-ResNet) architecture is proposed for testing the conversion accuracy and temporal feature extraction on _CIFAR10-DVS_, _NMNIST_ and _DVS-Gesture_.

The spatial and temporal properties of the SNN can be preserved to a greater extent with the use of recurrent convolution SNN. A sampling window in recurrent architecture captures the temporal correlations in event-based sequences. In [35], Xing et al. use a supervised _Spike Layer Error Reassignment_ (SLAYER) training mechanism for ANN-SNN conversion. The SCRNN architecture is a combination of single SCRNN cells that process input sequences separately to maintain the temporal dimension. The spatial complexity is handled with decomposed input processed in cells at every time step. Each SCRNN cell accepts an input feature map, a fused feature map of previous states, and an output feature map of the current input. Thus, the network was made recurrent and evaluated on the _Dynamic Vision Sensor_ (DVS) gesture dataset, achieving an accuracy of \(96.59\%\) for a 10-class gesture recognition task.

The elimination of all matrix multiplications in SNN makes them more energy-efficient than CNN. In [36], Wu et al. develop a unified framework that supports weight normalization, threshold balancing and an _Explicit Current Control_ (ECC) method that controls the number of spikes passed to the SNN and reduces the residual membrane potential. The residual current in the neurons causes accuracy loss for shorter inference. The ECC also enables the conversion of _Batch Normalization_ (BN) layer that many of the recent works fail to include. The role of the BN layer is to normalize the previous layer's output, which accelerates the CNN's convergence. This is implemented in the conversion by a numerical constant dependent on the training platform that updates the weight and bias to a normalized version of the same.

### _Neuron models on FPGA_

A biological neuron can be implemented in either of the neuron models as discussed in section II. In [37], Juneeth et al. develop a spiking neuron model of _Hodgkin-Huxley_ on an FPGA. The hardware architecture is based on a series of adder and multiplier blocks to implement the first-order differential equations in Verilog on _Xilinx Virtex-5_. Each of the neuron parameters is represented in a fixed point with chosen bit lengths. However, the physical justification for the bit optimization strategies is inconsistent. The use of multiplier blocks increases the number of LUTs and eventually, power consumption.

On the contrary, in [38], Farzin et al. propose a multiplier-less _Hodgkin-Huxley_ model. The model approximates hyperbolic functions as piece-wise linear terms implement all multiply operations as logical shifts and adds. This is made possible with equations modified to power-2-based functions. Considering the high accuracy of a HH neuron, a multiplier-less design furthermore reduces the operational cost and increases the frequency.

Two simplified two-dimensional versions of the HH neuron model are the _FitzHugh-Nagumo_ and _Morris-Lecar_ neuron models [39]. In [40], Nouri et al. investigate the _FitzHugh-Nagumo_ neuron model in terms of its digital implementation feasibility and computation overhead. The model is a cubical approximation with no auxiliary reset equations. The area and power numbers of the proposed design with _Euler_ discititization are lower than a HH model apparatus. In [41], Mellal et
al. study the behavior of the Morris-Lecar neuronal model on a _Xilinx Zynq UltraScale+_ board. The simulated and hardware results of the original and discrete implementations have a high correlation.

One of the basic building blocks of the human body is the _Central Nervous System_ comprised of neurons, synapses, and glial cells. In [42], Abdulaziz et al. evaluate a model based on its complexity and hardware resources required to realize it on an FPGA. The two-neuron coupled _Izhikevich_ model while satisfying the above parameters regenerates all the different dynamics of the human brain. The authors use a LUT-based approach to the quadratic equations instead of an approximation technique to replace the non-linear terms. As the size of the LUT increases to 1000 points the model becomes closer to the mathematical equations.

Another unique technique involving the _Izhikevich_ neuron model is discussed in [43]. According to the authors, the _Izhikevich_ neuron is practically more suitable for electrical realization due to its chaotic behavior. A modified version of the model is compared with the original coupled neuron dynamics through numerical simulations and FPGA device demonstration. The modified model has a longer processing time (2 clock cycles) but with no multiplier usage. This design characteristic has significantly reduced the utilization number while maintaining the accuracy of the model. Along with the spinal network these neuron models have been altered for use. In [44], Niu et al. the motoneurons in the motor nervous system are emulated with the _Izhikevich_ model on an FPGA to analyze pediatric neurological diseases.

In [45], Panchapakesan et al. proposed a novel synchronous SNN execution divided into layers for each input, hidden, and output layers. The input is executed layer-by-layer as shown in Figure 5 synchronously to avoid frequent off-chip memory communication. Further, the internal parameters like membrane potential can be immediately used, reducing the on-chip resources utilized. Each layer is composed of modules that implement the functionality of integrating and fire neuron structure as discussed in section II. Such layer network depiction can also be observed in [46]. Khodamoradi et al. propose a streaming SNN accelerator architecture that utilizes the concept of layer modules for generating output spikes. However, this architecture is made more specific to edge devices with sparse input data. Highly sparse events can still utilize significant memory to store null information. This is mitigated with the use of binary tensors as input buffers.

The hardware layer network is also observed in [47] where Fang focuses their work to retain the temporal information in a sequence of spikes in the architecture. The leaky integrate and fire neuron-based hardware avails a series of _Infinite Impulse Response_ (IIR) filters to represent SNN. The time-encoded spikes provide the time information to interpret neurons as filters. Further, the suggested model incorporates synapse dynamics adding flexibility to the system. Each layer has a processing element that updates the neuron and synapse state variables. An optimized framework of end-to-end neuromorphic systems is described with reduced inference latency. The results compared to a CPU has better performance metrics.

Additionally, the LIF model has been holistically modified to several other forms. In [48], Heidarpour et al present a _CORDIC_ implementation of an _Adaptive exponential integrate and fire_ (AdEx) neuron model. The primary difference to Equation 4 is that the potential difference integrates exponentially in the current equation. In a biological neuron, the increase in action potential with an input spike is non-linear. Thus, non-linear LIF neuron models depict the biological behavior rather closely although the implementation cost of non-linear behavior on hardware is always higher. The authors in [48] discretized the differential equations explicitly with the Euler method and implement it on a _Xilinx Spartan 6_ using simple add and shift operations only. Similarly, in [49], Basham et al demonstrate a _quadratic integrate and fire_ (QIF) neuron model where the current equation integrates the square of potential difference. The design utilizes a fixed-point multiplier to evaluate the square of the voltage.

Lastly, in [50], Nallathambi et al instrument a probabilistic integrate and fire neuron model. In a deterministic approach when a neuron i spikes, the applied weight of the postsynaptic neuron is \(w_{ij}\).For a set of \(N_{i}\) spikes with a probability of \(p_{ij}\) the new weight updates to \(N_{i}\times p_{ij}\times w_{ij}\).In stochastic processing, the number of memory fetches can be reduced significantly. By sorting the synaptic weights the authors report a reduction of \(90\%\) off-chip memory accesses.

### _Optimization techniques_

_Spiking Neural Networks_ can be divided in two different categories of optimization strategies. One, to make more biologically plausible neural models with better plasticity and second, to devise tuning algorithms that can transcend performance. More plasticity is accomplished with a trainable spiking network. Backpropagation is one such learning mechanism that has shown significant accuracy in ANN but it is not realizable for a biological spiking network. Thus, much work has been done to create the equivalent spiking backpropagation.

#### Iii-C1 Training SNN

Training deep spiking Neural networks have shown a great requirement for optimized gradient-based approaches. One of the reasons for facing difficulty in achieving deeper DNN is its complex _Spatio-Temporal Dynamics_. _Converting ANN to SNN_ and _Backpropagation with Surrogate Gradient_ are two ways to get deep SNN. Converted SNNs need a longer training time for similar precision as ANN. Most popularly, _Backpropagation Through Time_ (BPTT) is implemented by unfolding the gradient over a simulation time.

Fig. 5: Layer-by-layer architecture with LIF neuron in Processing Unit (PU).
In [51], Fang et al propose _Spike-Element-wise_ (SEW) _ResNet_ to resolve exploding gradient with identity mapping. In _Spiking ResNet_, the adaptation of residual learning from _ResNet_, the ReLU activation block is replaced by spiking neurons which is incapable to replicate the identity mapping for all neuron models. However, in _SEW ResNet_ element-wise function "g" facilitates identity mapping. When "g" is chosen to be an ADD function it avoids the infinite outputs problem by restricting the output.

Apart from a purely algorithmic perspective of improvement in spiking realizable Backpropagation, some work has been done by Shrestha et al to develop a biologically plausible algorithm that can fit the constraints of neuromorphic hardware. In [52], the authors have shown in-hardware supervised learning demonstrated on _Intel's Loihi_ chip. The supervised learning is based on _Error modulated Spike-Timing Dependent Plasticity_ (EMSTDP) with _Direct Feedback Alignment_ (DFA) that reduces the number of neuron updates in the feedback path. The hardware realization of the algorithm is designed by creating two replicas of the same neuron with each of them functional in feed-forward and feedback paths respectively.

Further in [53], Heidarpur et al propose on-FPGA online STDP. Based on _CORDIC_ an iterative algorithm all hyperbolic and exponential functions can be implemented with shift and addition operations. A _Izhikevich_ neuron behaviour was replicated with qualitative error analysis between the _CORDIC_ model compared to the original model of operation to achieve maximum precision.The proposed algorithm is tested for a two-layer network of 21 neurons with one output neuron.To demonstrate STDP learning on _Xilinx Spartan-6_ FPGA, all the exponential terms were approximated with a negative power of 2 terms which can be synthesized in fast and low-cost hardware.

In STDP the weight update depends on the timing difference between the presynaptic and postsynaptic neuron pair. On the other hand, _Triplet STDP_ (TSTDP) considers three consecutive spikes, one presynaptic and two post synaptic spikes for potentiation and depression respectively. In [54], Gomar et al could reproduce the learning curve by approximating the two variable TSTDP equations to one variable piecewise linear term where exponentials are converted to base-2 functions. The discrete values of the lines are stored in LUT memory whose size is optimized as per the design. The design is driven through a _Finite State Machine_ (FSM) of 4 states with 2 states representing the Learning unit of the system. Compared to the LUT model this model is less accurate but consumes less than \(1\%\) on-chip resources.

So far we have discussed _Hebbian_ learning applications, however, in [55], Liu et al explored non-Hebbian on-chip learning IP on recurrent SNN. The _Liquid State Machine_ (LSM) based recurrent SNN consists of a reservoir which maps the input pattern to a multi-dimensional response in the reservoir. The output of the reservoir is then passed to the readout layer. The focus of this work is to develop learning rules with _SpiKL-IP_ algorithm based on neural plasticity. _SpiKL_ is a _Intrinsic Plasticity_ (IP) rule whose key idea is to maximize the input to output information transfer. The algorithm is modified to reduce the complexity of hardware implementation with LUTs and series expansion techniques. Moreover, the LIF neuron model is discretized to surpass differential equations. Compared to baseline LSM the current model has outperformed by \(8\%\) for speech recognition _TIMIT_ dataset. Although a hardware resource utilization trade-off with extra energy overhead could be observed.

#### Iii-D2 Tuning SNN

Since SNN architectures are resource-constrained, compression methods like pruning are an essential optimization to tune the performance. Nonetheless, most of the pruning approaches are ANN-specific and not compatible with SNN. This factor lingers the performance of a pruned SNN. In [56], Chen et al formulate a learning algorithm of connectivity and weight that defines gradient as a different parameter called _Gradient Rewiring_ (Grad R). The combined learning and pruning algorithm change the synaptic connections based on the synaptic gradient parameter introduced. The algorithm is evaluated on _MNIST_ and _CIFAR-10_ datasets with a maximum of \(73\%\) connectivity. Although the techniques discussed in this section are the new frontiers of SNN optimization innovation, most of them have not made their way to a physical hardware implementation.

### _Technique Discussion_

The design and implementation of deep learning algorithms, particularly SNN hardware implementation, has been a hot topic during the previous decade [57]. SNN is hungry for computing power. Because of the rapid improvements in integrated circuit technology, it is now feasible to build high-performance chips at a low cost and with efficient energy consumption. This allowed for the deployment of inference at the edge. Along with, SNN can be executed on any device with adequate capability. There are some examples: i) CPUs, ii) GPUs, iii) FPGAs, iv) ASICs. Diverse technologies have a different throughput, performance, design flexibility, and power efficiency.

Firstly, we partially consider traditional computing platforms to provide a comparison overview of the latest SNN accelerators targeting low-power and high-performance. As depicted in Table I, after developing circuits for edge computing, energy and performance efficiency are the most important criteria to consider. CPUs [47][58] are the most adaptable gadgets, and as such, have the lowest performance and energy efficiency.

Furthermore, ASICs [59, 60, 61, 62, 63] provide the highest performance and energy efficiency, but with very little flexibility attainable primarily by additional hardware logic. It has reached a peak performance of \(100Kimg/sec\) and \(7.89TOPS/W\) mean that are very energy efficient. ASIC circuits are specifically built for neural networks to improve performance and power efficiency.

Modern GPUs incorporate single-precision computation modules capable of performing several half-precision floating-point calculations. Thus, they seem to be the perfect basis for both learning and prediction. However, due to their multi-core structure, GPUs present a challenge with high power usage. Studies [47, 58] have shown power consumption figures markedly higher than those for FPGAs.
FPGAs combine hardware and software (processing) in a single device, where software ensures programmability and hardware is utilized to implement specialized accelerators. GPUs have the higher performance among programmable or configurable devices and are less expensive than FPGAs.

Researchers have demonstrated that FPGAs may efficiently implement Spiking Neural Networks (SNN) in both hardware and software/hardware contexts [64, 65, 66, 67, 58, 47, 68, 69, 70]. They showed that their FPGA-based design had the highest performance of 2124 frame/sec and the lowest power of \(0.40W\). This corresponds to an energy efficiency of \(16.80mOPs/W\). The same platform may then be upgraded with an SNN without any board modifications just by reconfiguring the device.

In this section, key ideas for SNN implementation in equipment are shown, along with a comparison to different techniques in Table II. The table combines the most important information received from the available sources from among the numerous structures specified from now on. Without being exhaustive, this chart allows for a quick representation of the show status of the advanced systems.

This work [76], the usage of the _NeuroFlow_ which FPGA architecture for SNN that was suggested approach aims for a real-time execution time of \(0.1ms\). _NeuroFlow_ may be further reduced by adjusting the size of the arrangement or guessing on the network within this architecture. Moreover, the huge quantity of simulated neurons made available by _NeuroFlow_ may be achieved using six FPGAs in a toroidal network design. The configuration restricts the number of synapses to a range of 1,000-10,000, where the chance of connection is based on a Gaussian probability of synaptic disconnection, with a standard deviation varying from 32 to 512. In this spectrum, a direct comparison might be challenging and potentially unfair to other SNN models. Another approach in this direction has been implemented by Han [66], which highlights crossover upgrade algorithms that integrate the points of interest of current algorithms to improve equipment planning and execution. Thus, this design, capacitively supports 16 384 neurons and 16.8 million synapses but uses fewer hardware resources and has a very low power consumption (\(0.477\)W) and the performance for processing neuron activation events is \(6.72ms\). For a modest to large scale SNNs on sophisticated equipment should be computationally simple and simultaneously capable of communicating to the large range of termination patterns displayed by various organic neurons. For this reason, several designs employ simpler custom neural models [71], and much more with the LIF model [73, 72].

Fig. 6: SNN on Different Heterogeneous Devices
Spikes collection is performed at \(32bits\) to maintain as much accuracy as conceivable. In [75], the authors demonstrate a comparable technique capable of simulating up to 167 neurons. In comparison to the one suggested, due to increased information exactness and synaptic current preparation which is more sophisticated to a certain degree such a task employs a greater number of assets. The FPGA architectures by [74] and [65] simulate a fully-connected network of 1024 neurons, 1440 neurons, based on the biologically plausible _Izhikevich_ spiking model. The reported time resolutions are 10.0 \(\upmu\)s and \(0.1ms\). These implementations have low latency, despite the fact that simplified resource utilization is employed for it. An alternate notable design is that by Gupta et al. [77], which offers a simulation of a simpler and more computationally efficient model using FPGA infrastructure. This was engineered to leverage the sparsity of the network and generate each time unit corresponding to network activities. A network comprising 784 input and 16 output neurons, along with 12,544 synapses, was realized on hardware, thereby facilitating the creation of a hardware accelerator with minimal resource consumption.

In another study by Liu et al. [67], a neuron computing module was used, designed to replicate both _LIF_ and _Izhikevich_ neurons using the concurrent spike caching and scheduling strategy while simulating 16,384 neurons and 16.8 million synapses. They structured two distinct three-layer SNN networks applied for recognition tasks on the suggested platform. The employment of Slice LUTs, Slice Registers, and DSP of the SNN acceleration unit stands at \(6.0\%\), \(2.5\%\), and \(7.6\%\) respectively.

It is noteworthy that their design exhibits the highest occupancy ratio in BRAM. As a consequence, the parameters of the LIF and IZH neurons, along with the event buffer and synaptic delays, are produced using the on-chip storage resource BRAM. Hence, the BRAM is significantly employed, leading to increased power usage. Ma et al. [68] established an embedded auxiliary processor to hasten the inference mechanism of SNNs. By time-sharing the physical neuron components, and developing a reconfigurable memory subsystem, the design realizes high hardware efficiency, fitting for resource-limited embedded applications. It also permits a customizable quantity of neurons, synapses, and synaptic delays, yielding significant flexibility.

Moreover, work by Fang H et al [79], Asgari et al [78], and Li et al [69] proposed to utilize their design on the identical neuron model (_LIF_) and targeting the same dataset (_MNIST_). Recently, Carpegna et al [70] presented Spiker, which employs a clock-driven neuron design wherein the membrane potential is modified at each clock cycle absent any spikes. Nevertheless, inputs are only processed when there's at least one spike present at the input of a layer. This approach uses more power than purely event-driven designs but allows for the utilization of fewer hardware resources. The FPGA usage is of 1384 neurons and 313600 synapses are around \(55\%\) for the LUTs, and \(25\%\) for the FFs. Considering the number of instantiated neurons, this is a major finding.

## IV Applications

### _Image and Audio Processing_

In [80], authors presented an application of animal behavior recognition using an SNN-based sound recognition system associated with animal movements. The spiking neural network was built on an FPGA device, and the neuromorphic auditory system employed in this study creates a representation similar to the spike outputs of the biological cochlea. Even when the sound was accompanied by the white noise of the same strength, the detection system based on SNN achieved an accuracy of over 91%.

In [81], the authors proposed a continuous voice recognition system that employs a _Delta Recurrent Neural Network_ (DeltaRNN) on a _Xilinx Zynq-7100_ FPGA to provide low
latency _Recurrent Neural Network_ (RNN) processing. This system is suitable for _Internet of Thing_ (IoT) applications since FPGA uses just \(70mW\) and can process each feature frame in microseconds.

Edge detection is a prominent machine learning job that necessitates the use of a large number of neurons as well as software simulations. As a result, scalability suffers, and computational time increases. This research [82] proposed a scalable FPGA implementation strategy with several innovations to shorten calculation times. Within this scenario, given spatial constraints, the time-division multiplexing structure proposes a balance between acceleration performance and the dimensions of SNN simulations. Furthermore, a study by Louis-Charles Caron et al. [83] accomplished image segmentation and monophonic audio source separation utilizing _Oscillatory Dynamic Link Matcher_ (ODLM) protocols for motif recognition.

The authors in [84] developed a character identification model for multicore architectures relying on two SNN models. The selection of the _Izhikevich_ and _Hodgkin-Huxley_ neuron models over the integrate and fire models were motivated by their biological accuracy. All 48 visuals in the learning datasets were accurately distinguished within \(14ms\) and 3.75ms, respectively. Moreover, the authors of [85] employed SNN for image clustering using FPGA. The _Gaussian Receptive Field_ (GRF) is the most commonly used method for encoding data, with Hebbian learning being used for neural models. Not only did [85] use the _Receptive Field_ (RF) for data encoding, but [86] also applied the T.Iakymchuk model. To examine an image in its spatial domain form, the authors also utilized the Gabor filter, a bandpass filter.

A biologically driven, rotationally invariant visual identification system, executed using a pixel camera on FPGA, was deployed in [87]. The structure merged the _Ripple Pond Network_ (RPN), a neural network proficient in fundamental 2D to 1D image conversion, rotationally invariant _Temporal Patterns_ (TPs), and the _Synaptic Kernel Adaptation Network_ (SKAN). By using the event-driven _Spike-timing dependent plasticity_ (STDP) rule, Zaibo Kuang and Jiang Wang [88] trained the MNIST dataset for digital categorization on a _Stratix-3_ FPGA processor with \(93\%\) precision. As a result, the cost-effective network demonstrated great efficiency and can readily be scaled to a larger size.

### _Biomedical Applications_

The manuscript [89] presents a freely available FPGA-centric emulation environment for delving into the neuromorphic computation. The authors implemented the _MNIST_[90] dataset and_Vector-Matrix Multiplication_ (VMM), comparing their accuracy with analogous architectures generated by IBM's Compass simulation infrastructure. Considering the substantial computational intricacy of spiking neural networks, it's arduous to actualize them on hardware necessitating proficiency. Thus, QingXiang Wu [91] unveiled a straightforward, effective, and swift strategy to instantiate SNN employing a customized toolkit. Thus, participants in this investigation seamlessly engineered and modeled various spiking neural networks on FPGA, enhancing the execution speed.

In a different study, Junxiu Liu [92] proposed a novel biologically-derived gas identification technique with minimal hardware footprints. This method utilized spiking neural pathways and neurons to recognize the abnormal frequency of the input spike generation by adjusting the discharge likelihood of the inhibitory neural linkage under diverse input situations.

In [93], a neuromorphic system on a _Virtex-2 pro_ FPGA was shown how neuronal elements from this construction kit can be deployed in parallel. The neuromorphic system mirroring the known architecture of the early olfactory pathway when coupled spontaneously produces rhythmic, limit-cycle dynamics resembling biology. Thus, The findings of this work show how these attractors may be read from the network to recognize previously experienced learned fragrances without confusion.

The authors [94] present a neuromorphic system that combines a neural recording headstage with an SNN processing core on the same die for processing _iElectroencephalography_ (iEEG), and demonstrate how it can reliably detect _High-Frequency Oscillations_ (HFO), achieving state-of-the-art accuracy, sensitivity, and specificity. This is the first feasibility research aimed at finding significant characteristics in iEEG in real-time utilizing mixed-signal neuromorphic computing Technologies on a custom board _XEM7360_ FPGA.

In [95], The researchers wanted to construct an artificial intelligence signal identification system in a _PYNQ-Z_ FPGA

Fig. 8: Block diagram of the neuromorphic platform (a) and _Functional magnetic resonance imaging_ (fMRI) brain image (b)

Fig. 7: System architecture of microphone
board that can detect patterns of bio-signals such as _Electrocardiography_ (ECG) in battery-powered edge devices. Despite the increase in classification accuracy, deep learning models need expensive processing resources and power. Therefore, it makes deep neural network mapping a time-consuming process and deployment on wearable devices difficult to achieve. SNNs have been used to overcome these constraints.

### _Control Systems_

In [96], proposed a procedure that permits the usage of analog-like spike-based _Proportional-Integral-Derivative_ (PID) controllers on low-cost _Virtex-3_ FPGA. Spike-based PID controllers have been logically analyzed and characterized, demonstrating that they are exceptionally near to nonstop controller models unless they are actualized in a totally computerized gadget as FPGAs.

In [97], biological neuron model-generated spiking waveforms were utilized as substitutes for sawtooth waveforms to effectuate _Pulse-Width Modulation_ (PWM), a modulation strategy that produces variable-width pulses to symbolize the amplitude of an analog input signal. This study proposes a signal generator on the FPGA platform, grounded in the _Izhikevich_ neuron model, prized for its simplicity and precision.

An innovative stride in the realm of prosthetics is the bioelectric upper limb prosthesis serving as an actuator's control. The authors of [98] adopted the _Izhikevich_ neuron model and an FPGA-oriented LIF to mimic the behavior of biological neurons. Moreover, _PWM_ has surged in its acceptance as a governing technique in both analog and digital circuits.

In [99], a spike-oriented proportional-integral-derivative engine speed controller was modified to govern the position of the 4 joints of a lightweight and _safe physical human-robot interaction_ (pHRI) robotic arm, referred to as _event-driven BioRob_ (ED-BioRob). These _spiking PID_ (sPID) controllers were deployed on two _Spartan-3_ FPGA platforms, that is, the _address-event representation Robot_ (AER-Robot). The robot furnishes address-event-representation interfacing for spiking systems and is capable of driving DC motors with Pulse Frequency Modulation signals, reflecting the motor neurons of mammals.

### _Autonomous Systems_

FPGAs are gaining popularity because of their reconfigurability and hardware efficiency, and have been proposed for robotic vision. The publication [100] introduced the _Efficient Large-scale Stereo_ (ELAS) based stereo vision system that is completely implemented on FPGA and is intended for real-time and energy-efficient applications. The _ELAS_ which is shown in Figure 9 calculation is reformulated as a standard design with a back focus introduction for hardware execution.

Within the examination at [101] the bound-together compute bottleneck of different localization frameworks is distinguished. Therefore, an _Oriented-Fast and Rotated-BRIEF_ (ORB) based visual frontend design is displayed for real-time and energy-efficient localization and assessed on the FPGA platform.

Authors illustrated [102] an end-to-end usage on a genuine, high-_Degrees of Freedom_ (DOF) mechanical arm, and the quickening agent was able to unravel energetic pick-and-place scenarios with a high rate of success. This sub-millisecond speed is adequate with a few limitations to empower already the infeasible automated applications, such as real-time arranging in energetic situations.

### _Robotics_

Due to their handy and beneficial assistance, robots are increasingly being integrated into our society. The authors of [103] aimed to leverage FPGA to create a cerebellum model capable of learning and adjusting conversational robot timing. The _Central Pattern Generator_ (CPG), a locomotion mechanism for multi-legged robots, is an SNN application as well.

The research conducted in [104] applied advanced hippocampal pyramidal neuron models to construct hardware-spiking neural networks for navigation purposes. J.Parker Mitchell [105] utilized a _Dynamic Adaptive Neural Network Array_ (DANNA) framework for Autonomous Robotic Navigation. The DANNA framework comprises a grid of adaptable neuromorphic computing components, each capable of behaving like a neuron and connected to its adjacent counterpart in the grid.

A modular hardware execution of a spiking neural network with real-time adjustable connections is presented in [106], framed within an autonomous robot impediment evasion schema. The structure is situated in a conventional 2D matrix of cells, each operating as a spiking neuron with unique functions. Another study [107] likewise tackled the robot

Fig. 9: Overview of ELAS
obstacle avoidance dilemma. Inspired by the robustness and adaptability of biological systems, this research offered a uniquely flexible neural network model. The network observed a reduction of up to \(75\%\) of the initial synaptic inputs to a cell.

In [108], a neuromorphic framework was established on a _Spartan-6_ FPGA to enable locomotion for three types of robots: bipedal, quadrupedal, and hexapods. In this exploration, the researcher conceived a fast, compact, and configurable FPGA architecture that hinged on the _CPG_, the movement mechanism for multi-legged robots.

### _Event vision sensors_

A fault injection experiment and fault resilience analysis for an SNN hardware accelerator were given in the study at [109]. They created a fault injection framework that builds and maps SNN faulty instances into the hardware. The framework is allowing the researchers to expedite fault injection and analyze fault criticality on _ZCU104_ FPGA development board.

This research described [110] one of the first end-to-end neuromorphic frameworks for real-time object tracking and categorization shown utilizing a low-power hardware implementation on _Trenz TE0720_ FPGA. To take use of the low latency and asynchronous nature of _Neuromorphic Vision Sensors_ (NVS) as an optional paradigm. The framework's integral operation includes gathering occurrences from the camera, handling these occurrences to isolate surveilled entities, forwarding these observed coordinates to the neuromorphic chip for sorting, and the entity detection illustration, as demonstrated in Figure 10.

### _Communication_

The authors of the work [111] describe a hybrid-mode router architecture for large-scale neuromorphic simulation by merging two types of router schemes. These router schemes are proposed to allow chip-to-chip transmission of spike and non-spike data. This work is being tested on a neuromorphic platform constructed using an chip _Artix-7_ FPGA.

In Table IV displays the comparison of data transmission exists similar works. The proposed routers separate themselves from prior efforts by supporting both source-driven and destination-driven packet representations. The FPGA implemented method has a spike processing rate of \(200MSpikes/s\). The research found that their algorithm, routing strategies, and router design improved communication efficiency in the specified multichip network.

## V Trend

Based on the above surveyed related works, we can conclude the following four potential topics in further FPGA-based SNN accelerator research:

* _Approximation Computing in SNN:_ Considering the hardware resource limitation on FPGA platforms, further compressing the memory consumption of weights and parameters in SNN models is necessary. The approximation computing methods and _Posit_ computing methods applied in work [114][115] are the upcoming techniques that can be applied in SNN accelerators on FPGA. Besides, the application of quantization and binarization techniques on SNN accelerators can also be potential.
* _Implementation Toolchain/Framework for SNN on FPGA:_ Considering the complexity of FPGA development, as the python-based hardware generation framework shown in HLS4ML [116][117][118] and FINN [119][120][121], an automatic hardware design generation framework/library/toolchain, will highly benefit the implementation and deployment of SNN acceleration on FPGA platforms. These frameworks/libraries/toolchains should be able to cooperate with the training/accelerating software libraries and convert the trained SNN model to the suitable hardware design for targeting platforms.
* _Automatic Network Generation of SNN on FPGA:_ The automatic network model generation techniques, such as _Network Architecture Search_ (NAS) are the upcoming topic in recent research. Work [122] explored the application of NAS on FPGA, which could also be one potential research and application idea for SNN acceleration on FPGA.
* _Upcoming Network Models of SNN on FPGA:_ Recent work of FPGA-based SNN acceleration focus on the classic MLP, CNN, and LSTM models. Some state-of-the-art works, such as [123][124][125], explored the application of SNN on _Transformer_ and _Graph Neural Networks_ (GNNs). The implementation of FPGA-based accelerators for the above networks can extend the application scenarios of related research.

Fig. 10: System flow diagram of the neuromorphic vision sensor
## VI Conclusions

We have provided a survey on the hardware implementation of SNN that, in particular, covers recent trends in deploying FPGAs in different applications. The field of FPGA has proven that, despite the already apparent exceptional results, various researchers have illustrated the advantages of reconfigurable hardware implementations over other heterogeneous devices. In addition, we explained how different neural networks perform on FPGAs. With this survey, we proposed to summarize state-of-the-art work and highlight critical directions of inquiry about that such as image/audio processing, robotics, autonomous systems, event vision sensor, and biomedical applications.
