SPENSER: Towards a NeuroEvolutionary Approach for Convolutional Spiking
  Neural Networks
Spiking Neural Networks (SNNs) have attracted recent interest due to their
energy efficiency and biological plausibility. However, the performance of SNNs
still lags behind traditional Artificial Neural Networks (ANNs), as there is no
consensus on the best learning algorithm for SNNs. Best-performing SNNs are
based on ANN to SNN conversion or learning with spike-based backpropagation
through surrogate gradients. The focus of recent research has been on
developing and testing different learning strategies, with hand-tailored
architectures and parameter tuning. Neuroevolution (NE), has proven successful
as a way to automatically design ANNs and tune parameters, but its applications
to SNNs are still at an early stage. DENSER is a NE framework for the automatic
design and parametrization of ANNs, based on the principles of Genetic
Algorithms (GA) and Structured Grammatical Evolution (SGE). In this paper, we
propose SPENSER, a NE framework for SNN generation based on DENSER, for image
classification on the MNIST and Fashion-MNIST datasets. SPENSER generates
competitive performing networks with a test accuracy of 99.42% and 91.65%
respectively.
Henrique Branquinho, Nuno Lourenço, Ernesto Costa
# SPENSER: Towards a NeuroEvolutionary Approach for Convolutional Spiking Neural Networks

###### Abstract.

Spiking Neural Networks (SNNs) have attracted recent interest due to their energy efficiency and biological plausibility. However, the performance of SNNs still lags behind traditional Artificial Neural Networks (ANNs), as there is no consensus on the best learning algorithm for SNNs. Best-performing SNNs are based on ANN to SNN conversion or learning with spike-based backpropagation through surrogate gradients. The focus of recent research has been on developing and testing different learning strategies, with hand-tailored architectures and parameter tuning. Neuroevolution (NE), has proven successful as a way to automatically design ANNs and tune parameters, but its applications to SNNs are still at an early stage. DENSER is a NE framework for the automatic design and parametrization of ANNs, based on the principles of Genetic Algorithms (GA) and Structured Grammatical Evolution (SGE). In this paper, we propose SPENSER, a NE framework for SNN generation based on DENSER, for image classification on the MNIST and Fashion-MNIST datasets. SPENSER generates competitive performing networks with a test accuracy of 99.42% and 91.65% respectively.

spiking neural networks, neuroevolution, DENSER, computer vision +
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision

+
Footnote †: journal: Computer Vision
To the best of our knowledge, this is the first work focusing on evolving SNNs trained with BPTT for image classification, including not only different architectures but different neuronal dynamics and optimizers in the search space. The main contribution of this paper is the preliminary validation of neuroevolution through SPENSER in the automatic generation of competitively performing CSNNs. The main focus of the paper is on the performance of the generated networks in terms of accuracy.

The remainder of this paper is structured as follows: Section 2 provides a review of important concepts regarding SNNs; Section 3 covers related work regarding evolutionary approaches for SNNs; Section 4 describes SPENSER; Section 5 describes the experimental setup; Section 6 analyses the experimental results, covering the evolutionary search and the testing performance of the generated models; Section 7 provides some final remarks and suggested guidelines for future research.

## 2. Spiking Neural Networks

Spiking Neural Networks (SNNs) are a class of neural network models built with spiking neurons where information is encoded in the timing and frequency of discrete events called spikes (or action potentials) over time (Srivastava et al., 2017). Spiking neurons can be characterized by a membrane potential \(V(t)\) and activation threshold \(V_{thresh}\). The weighted sum of inputs of the neuron increases the membrane potential over time. When the membrane potential reaches its activation threshold, a spike is generated (fired) and propagated to subsequent connections. In a feed-forward network, inputs are presented to the network in the form of spike trains (timed sequences of spikes) over \(T\) time steps, during which time spikes are accumulated and propagated throughout the network up to the output neurons.

There are a number of spiking neuron models that vary in biological plausibility and computational cost, such as the more realistic and computationally expensive Hodgkin-Huxley (Hodgkin and Huxley, 1983), to the more simplistic and computationally lighter models such as the Izhikevich (Izhikevich, 1983), Integrate-and-Fire (IF) (Ling and Fang, 2017) and Leaky Integrate-and-Fire (LIF) (Ling and Fang, 2017). We refer to Long and Fang (Long and Fang, 2017) for an in-depth review of existing spiking neuron models and their behaviour.

The LIF neuron is the most commonly used in the literature due to its simplicity and low computational cost. The LIF neuron can be modulated as a simple parallel Resistor-Capacitor (RC) circuit with a "leaky" resistor:

\[C\frac{dV}{dt}=-g_{L}(V(t)-E_{L})+I(t) \tag{1}\]

In Eq. 1, \(C\) is a capacitor, \(g_{L}\) is the "leaky" resistor (conductor), \(E_{L}\) is the resting potential and \(I(t)\) is the current source (synaptic input) that charges up the capacitor to increase the membrane potential \(V(t)\). Solving this differential equation through Euler method (demonstration in (Kirkpatrick, 1983)), we can calculate a neuron's membrane potential at a given timestep \(t\) as:

\[V[t]=\beta V[t-1]+WX[t]-Act[t-1]V_{thresh} \tag{2}\]

In Eq. 2, \(\beta\) is the decay rate of the membrane potential, \(X[t]\) is the input vector (corresponding to \(I(t)\)), \(W\) is the vector of input weights, and \(Act[t]\) is the activation function. The activation function can be defined as follows:

\[Act[t]=\left\{\begin{array}{ll}1,&\text{if }V[t]>V_{thresh}\\ 0,&otherwise\end{array}\right\} \tag{3}\]

A LIF neuron's membrane potential naturally decays to its resting state over time if no input is received (\(\beta V[t-1]\)). The potential increases when a spike is received from incoming connections, proportionally to the connection's weight (\(WX[t]\)). When the membrane potential \(V(t)\) surpasses the activation threshold \(V_{thresh}\) a spike is emitted and propagated to outgoing connections and the membrane's potential resets (\(-Act[t-1]V_{thresh}\)). Resetting the membrane's potential can be done either by subtraction, as is done in the presented example, where \(V_{thresh}\) is subtracted at the onset of a spike; or to zero, where the membrane potential is set to \(0\) after a spike. A refractory period is usually taken into account where a neuron's potential remains at rest after spiking in spite of incoming spikes. The decay rate and threshold can be static or trainable.

Existing frameworks such as _snntorch_(Kirkpatrick, 1983) allow for the development of SNNs by integration of spiking neuron layers in standard ANN architectures such as Convolutional Neural Networks, by simply replacing the activation layer with a spiking neuron layer.

### Information Coding

Spiking systems rely on discrete events to propagate information, so the question arises as to how this information is encoded. We focus on two encoding strategies: rate coding and temporal coding. In **rate coding**, information is encoded in the frequency of firing rates. This is the case in the communication between photoreceptor cells and the visual cortex, where brighter inputs generate higher frequency firing rates as opposed to darker inputs and respectively lower frequency firing rates (Kirkpatrick, 1983). ANNs rely on rate coding of information, as each neuron's output is meant to represent an average firing rate. In **temporal coding**, information is encoded in the precise timing of spikes. A photoreceptor system with temporal coding would encode a bright input as an early spike and a dark input as a last spike. When considering the output of an SNN for a classification task, the predicted class would either be: the one with the highest firing frequency, using rate coding; the one that fires first, using temporal coding.

Temporal coding is advantageous in terms of speed and power consumption, as fewer spikes are needed to convey information, resulting in more sparse events which translate to fewer memory accesses and computation. On the other hand, rate coding is advantageous in terms of error tolerance, as the timing constraint is relaxed to the overall firing rate, and promoting learning, as the absence of spikes can lead to the "dead neuron" problem, where no learning takes place as there is no spike in the forward pass. Increased spiking activity prevents the "dead neuron" problem.

### Learning

Learning in SNNs remains one of the biggest challenges in the community due to the non-differentiability of the activation function of spiking neurons (Eq. 3), which does not allow for the direct transposition of the error backpropagation algorithm.

Commonly used learning strategies include unsupervised learning through Spike-Timing-Dependent Plasticity (STDP) (Ling and Fang, 2017), offline conversion from trained ANNs to SNNs (also known as shadow
training) (Krizhevsky et al., 2014; Krizhevsky et al., 2014), and supervised learning through backpropagation either using spike times (Beng et al., 2015) or adaptations of the activation function to a continuous-valued function (Krizhevsky et al., 2014; Krizhevsky et al., 2014; Krizhevsky et al., 2014; Krizhevsky et al., 2014; Krizhevsky et al., 2014). In this work, we focus on the latter, by training SNNs using backpropagation through time (BPTT) and surrogate gradients.

BPTT is an application of the backpropagation algorithm to the unrolled computational graph over time, usually applied to Recurrent Neural Networks (RNNs) (Krizhevsky et al., 2014). In order to bypass the non-differentiability of the spiking neuron's activation function, one can use surrogate gradients, by approximating the activation function with continuous functions centered at the activation threshold during the backward pass of backpropagation (Srivastava et al., 2014).

In this experimental study, we considered two surrogate gradient functions available in _sntorch_(Krizhevsky et al., 2014):

* **Fast-Sigmoid** \[Act\approx\frac{V}{1+k|V|}\] (4)
* Shifted arc-tan function \[Act\approx\frac{1}{\pi}arctan(\pi V\frac{\alpha}{2})\] (5)

Regarding the loss function, there are a number of choices available depending on the output encoding of the network (rate vs temporal), that calculate the loss based on spikes or on membrane potential. For this experimental study, we considered rate encoding for inputs and outputs, and as such, chose the **Mean Square Error Spike Count Loss** (adapted from (Srivastava et al., 2014)). The spike counts of both correct and incorrect classes are specified as targets as a proportion of the total number of time steps (for example, the correct class should fire 80% of the time and the incorrect classes should only fire 10%). The target firing rates are not required to sum to 100%. After a complete forward pass, the mean square error between the actual (\(\sum_{t=0}^{T}Act[t]\)) and target (\(\hat{Act}\)) spike counts of each class \(C\) is calculated and summed together (Eq.6).

\[\mathcal{L}=\frac{1}{T}\sum_{j=0}^{C-1}(\sum_{t=0}^{T}Act_{j}[t]-\hat{Act}_{j })^{2} \tag{6}\]

## 3. Related Work

Recent works blending EC and SNNs are mostly focused on evolving a network's weights, using evolutionary approaches as a learning strategy (Krizhevsky et al., 2014; Krizhevsky et al., 2014; Krizhevsky et al., 2014).

Schuman et al. (Schuman et al., 2015) proposed Evolutionary Optimization for Neuromorphic Systems, aiming to train spiking neural networks for classification and control tasks, to train under hardware constraints, to evolve a reservoir for a liquid state machine, and to evolve smaller networks using multi-objective optimization. However, they focus on simple machine learning classification tasks and scalability is unclear. Elbrecht and Schuman (Elbrecht and Schuman, 2015) used HyperNeat (Krizhevsky et al., 2014) to evolve SNNs focusing on the same classification tasks. Grammatical Evolution (GE) has also been used previously by Lopez-Vazquez et al. (Lopez-Vazquez et al., 2015) to evolve SNNs for simple classification tasks.

The current state of the art in the automatic design of CSNN architectures are the works of Kim et al. (Kim et al., 2017) and AutoSNN by Na et al. (Na et al., 2019). Both works focus on Neural Architecture Search (NAS), with an evolutionary search component implemented in AutoNN, and attain state-of-the-art performances in the CIFAR-10, CIFAR-100 (Krizhevsky et al., 2014), and TinyImageNet datasets. However, both works fix the generated networks' hyperparameters such as LIF neuron parameters and learning optimizer. Our work differs from these works by incorporating these properties in the search space.

## 4. Spenser

SPENSER (SPiking Evolutionary Network StructurEd Representation) is a general-purpose evolutionary-based framework for the automatic design of SNNs, based on DENSER (Beng et al., 2015; Beng et al., 2015), combining the principles of Genetic Algorithms (GA) (Krizhevsky et al., 2014) and Dynamical Structured Grammatical Evolution (DSGE) (Krizhevsky et al., 2014; Krizhevsky et al., 2014). SPENSER works on a two-level basis, separating the GA and the DSGE level, which allows for the modeling of the overall network structure at the GA level while leaving the network layer's specifications for the DSGE (Figure 1). The use of a grammar is what makes SPENSER a general-purpose framework, as one solely needs to change the grammar to handle different network and layer types, problems and parameters range.

The GA level encodes the macrostructure representing the sequence of evolutionary units that form the network. Each unit corresponds to a nonterminal from the grammar that is later expanded through DSGE. With this representation, we can encode not only the network's layers as evolutionary units but also the optimizer and data augmentation. Furthermore, by assigning each evolutionary unit to a grammar nonterminal, we can encode prior knowledge and bound the overall network architecture.

The DSGE level is responsible for the specification of each layer's type and parameters, working independently from the GA level. DSGE represents an individual's genotype as a set of expansion choices for each expansion rule in the grammar. Starting from a nonterminal unit from the GA level, DSGE follows the expansions set in the individual's genotype until all symbols in the phenotype are nonterminals. Rules for the layer types and parameters are represented as a Context-Free Grammar (CFG), making it easier to adapt the framework to different types of networks, layers and problem domains.

An example encoding to build CSNNs could be defined by Grammar 1 and the following GA macro structure:

\[[(features,1,10),(classification,1,3),\] \[(output,1,1),(learning,1,1)]\]

The numbers in each macro unit represent the minimum and maximum number of units that can be incorporated into the network. With this example, the \(features\) block encodes layers for feature extraction, and therefore we can generate networks with convolutional and pooling layers, followed by 1 to 3 fully connected layers from the \(classification\) units. The activation layers are restricted to LIF nodes with different surrogate gradient options. The \(learning\) unit represents the optimizer used for learning and its parameters. The \(output\) unit encodes the network's output layer. Numeric parameters are defined by their type, the number of parameters to generate, and the range of possible values.
Regarding variation operators, SPENSER relies on mutations on both levels. At the GA level, individuals can be mutated by adding, replicating, or removing genes i.e. layers. At the DSGE level, mutation changes the layers' parameters by grammatical mutation (replacing grammatical expansions), integer mutation (replacing an integer parameter with a uniformly generated random one), and float mutation (modifying a float parameter through Gaussian perturbation). SPENSER follows a \((1+\lambda)\) evolutionary strategy where the parent individual for the next generation is chosen by highest fitness and mutated to generate the offspring. This evolutionary strategy was chosen due to the computational demands of the network training process, which limits the population size in regard to execution time.

## 5. Experimental Setup

For this experimental study, we evolved and tested networks on the MNIST (Krizhevsky et al., 2017) and Fashion-MNIST (Zhu et al., 2017) datasets, available through the Torchvision library of Pytorch. All images were converted to grayscale and their original size was kept (28x28). In order to apply SNNs to these datasets, the images were converted to spike trains using rate coding. The pixel values are normalized between 0 and 1 and each pixel value is used as a probability in a Binomial distribution, which is then sampled from to generate spike trains of length \(T\) time steps. No data augmentation was used. We considered different time steps for each dataset according to their complexity.

Datasets were split in three subsets: EvoTrain, Fitness and Test. The Test split is the one provided by Torchvision. The EvoTrain and Fitness splits are a 70/30 split of the original Train split. Each independent run generates different EvoTrain and Fitness splits. Table 1 summarises the chosen time steps and the number of samples per split for each dataset.

As this is a preliminary study to validate SPENSER, we settled on one-pass training of individuals as a trade-off between speed and accuracy. During the evolutionary search, individuals are trained on the EvoTrain split for 1 epoch and tested against the Fitness split for fitness assignment. After the evolutionary search is complete, the best individual is further trained for 50 epochs on the entire Train set, and tested against the Test set for accuracy assessment.

We used _smtorch_(Krizhevsky et al., 2017) to assemble, train and evaluate SNNs based on rate coding. Individuals are trained using BPTT and the chosen loss function was the Mean Square Error Spike Count described in Section 2.2, with a target spiking proportion of 100% for the correct class and 0% for the incorrect class. The predicted class for a given instance is calculated based on the highest spike count of the output neurons. Accuracy is used as the fitness metric during the evolutionary search and as the final performance assessment of the best found individuals.

The macro structure of individuals for the GA level was set as:

\[[(features,1,6),(classification,1,4),\] \[(output,1,1),(learning,1,1)]\]

Because we are dealing with an image recognition problem, we defined a grammar that contains primitives allowing for the construction of CSNNs, as shown in Grammar 2. Following is a brief description of the grammar.

\(features\) units can be expanded to either Convolutional + Activation, Convolutional + Pooling + Activation, or Dropout layers. Convolutional layers are defined by the number of filters, filter shape, stride, padding and bias. Pooling layers are defined by the pooling type (max or average) and the kernel size. \(classification\) units can be expanded to either Fully-Connected + Activation or Dropout layers. Fully-Connected layers are defined by the number of units. Dropout layers are defined by the dropout rate. The \(output\) unit is set as a Fully-Connected + Activation where the number of units is fixed to the number of classes. Activation layers are currently limited to LIF neurons. LIF neurons are defined by the decay rate \(\beta\), the activation threshold \(V_{\textit{thresh}}\), and the reset mechanism

\begin{table}
\begin{tabular}{|l|c|c|c|c|} \hline  & \multicolumn{2}{|c|}{**Train**} & \\ \cline{2-5}  & **Time Steps (T)** & EvoTrain & Fitness & **Test** \\ \hline
**MNIST** & 10 & \multirow{2}{*}{42000} & \multirow{2}{*}{18000} & \multirow{2}{*}{10000} \\
**F-MNIST** & 25 & & & \\ \hline \end{tabular}
\end{table}
Table 1. Time steps and number of samples per split for each dataset (MNIST and Fashion-MNIST).

Figure 1. Individual generation by SPENSER. The first line represents the GA level where the macrostructure of the network is defined (this individual has 2 _features_ units and 2 _classification_ units). The second line represents the specification of a _classification_ unit through DSGE. Each number in the DSGE level represents the index of the chosen expansion rule for the current non-terminal. The last line is the resulting phenotype of the layer in question (Branquinho et al., 2017).
(subtraction or zero). Furthermore, they are also defined by the surrogate gradient function, which in this case can be either the ATan or the Fast-Sigmoid functions described in Section 2.2. The _learning_ unit encodes the optimizer and can be expanded to either Stochastic Gradient Descent, Adam, or RMSProp. We increased the probability of choosing feature extraction layers over dropout for \(features\) units (Grammar 2, line 1).

Regarding SPENSER's main hyper-parameters, we followed the recommendations of (B
different have small variations, showcasing SPENSER's robustness in generating high-performing networks.

We compared the best attained test accuracy with other works that also trained hand-tailored networks through spike based back-propagation. A comparison of test results is presented in Tab. 4. Albeit not surpassing the state-of-the-art, networks generated by SPENSER are head-to-head with the best-performing networks in the literature.

In order to validate our choice of one epoch training for fitness assessment, we also trained the best networks found in the first generation of each run for another 50 epochs and tested their performance on the Test set. Fig. 5 displays violin plots for the test accuracy of the best individuals from generation 1 and generation 2. The results are shown in Tab. 4. The results are shown in Tab. 5. The results are shown in Tab. 6. The results are shown in Tab. 7. The results are shown in Tab. 8. The results are shown in Tab. 9. The results are shown in Tab. 10. The results are shown in Tab. 11. The results are shown in Tab. 12. The results are shown in Tab. 13. The results are shown in Tab. 14. The results are shown in Tab. 15. The results are shown in Tab. 16. The results are shown in Tab. 17. The results are shown in Tab. 18. The results are shown in Tab. 19. The results are shown in Tab. 10. The results are shown in Tab. 11. The results are shown in Tab. 12. The results are shown in Tab. 13. The results are shown in Tab. 14. The results are shown in Tab. 15. The results are shown in Tab. 16. The results are shown in Tab. 17. The results are shown in Tab. 18. The results are shown in Tab. 19. The results are shown in Tab. 10. The results are shown in Tab. 11. The results are shown in Tab. 12. The results are shown in Tab. 13. The results are shown in Tab. 14. The results are shown in Tab. 15. The results are shown in Tab. 16. The results are shown in Tab. 17. The results are shown in Tab. 18. The results are shown in Tab. 19. The results are shown in Tab. 10. The results are shown in Tab. 11. The results are shown in Tab. 12. The results are shown in Tab. 13. The results are shown in Tab. 14. The results are shown in Tab. 15. The results are shown in Tab. 16. The results are shown in Tab. 17. The results are shown in Tab. 18. The results are shown in Tab. 19. The results are shown in Tab. 10. The results are shown in Tab. 11. The results are shown in Tab. 12. The results are shown in Tab. 13. The results are shown in Tab. 14. The results are shown in Tab. 15. The results are shown in Tab. 16. The results are shown in Tab. 17. The results are shown in Tab. 18. The results are shown in Tab. 19. The results are shown in Tab. 10. The results are shown in Tab. 11. The results are shown in Tab. 12. The results are shown in Tab. 13. The results are shown in Tab. 14. The results are shown in Tab. 15. The results are shown in Tab. 16. The results are shown in Tab. 17. The results are shown in Tab. 18. The results are shown in Tab. 19. The results are shown in Tab. 10. The results are shown in Tab. 11. The results are shown in Tab. 12. The results are shown in Tab. 13. The results are shown in Tab. 14. The results are shown in Tab. 15. The results are shown in Tab. 16. The results are shown in Tab. 17. The results are shown in Tab. 18. The results are shown in Tab. 19. The results are shown in Tab. 10. The results are shown in Tab. 11. The results are shown in Tab. 12. The results are shown in Tab. 13. The results are shown in Tab. 14. The results are shown in Tab. 15. The results are shown in Tab. 16. The results are shown in Tab. 17. The results are shown in Tab. 18. The results are shown in Tab. 19. The results are shown in Tab. 10. The results are shown in Tab. 11. The results are shown in Tab. 12. The results are shown in Tab. 13. The results are shown in Tab. 14. The results are shown in Tab. 15. The results are shown in Tab. 16. The results are shown in Tab. 17. The results are shown in Tab. 18. The results are shown in Tab. 19. The results are shown in Tab. 10. The results are shown in Tab. 11. The results are shown in Tab. 12. The results are shown in Tab. 13. The results are shown in Tab. 14. The results are shown in Tab. 15. The results are shown in Tab. 16. The results are shown in Tab. 17. The results are shown in Tab. 18. The results are shown in Tab. 19. The results are shown in Tab. 19. The results are shown in Tab. 10. The results are shown in Tab. 11. The results are shown in Tab. 12. The results are shown in Tab. 13. The results are shown in Tab. 14. The results are shown in Tab. 15. The results are shown in Tab. 16. The results are shown in Tab. 17. The results are shown in Tab. 18. The results are shown in Tab. 19. The results are shown in Tab. 10. The results are shown in Tab. 11. The results are shown in Tab. 12. The results are shown in Tab. 13. The results are shown in Tab. 14. The results are shown in Tab. 15. The results are shown in Tab. 16. The results are shown in Tab. 17. The results are shown in Tab. 18. The results are shown in Tab. 19. The results are shown in Tab. 10. The results are shown in Tab. 11. The results are shown in Tab. 12. The results are shown in Tab. 13. The results are shown in Tab. 14. The results are shown in Tab. 15. The results are shown in Tab. 16. The results are shown in Tab. 17. The results are shown in Tab. 18. The results are shown in Tab. 19. The results are shown in Tab. 10. The results are shown in Tab. 11. The results are shown in Tab. 12. The results are shown in Tab. 13. The results are shown in Tab. 14. The results are shown in Tab. 15. The results are shown in Tab. 16. The results are shown in Tab. 17. The results are shown in Tab. 18. The results are shown in Tab. 19. The results are shown in Tab. 19. The results are shown in Tab. 10. The results are shown in Tab. 11. The results are shown in Tab. 12. The results are shown in Tab. 13. The results are shown in Tab. 14. The results are shown in Tab. 15. The results are shown in Tab. 16. The results are shown in Tab. 17. The results are shown in Tab. 18. The results are shown in Tab.
200. It is clear that the networks' performance is dependent on the architecture rather than training epochs and that the networks evolved by SPENSER perform better than random initialization.

We hypothesize that a big limitation in this experimental study was the choice of the loss function's parameters, as it does not follow the literature's recommendations (Kim et al., 2019). By setting the target firing rate of incorrect classes to 0%, we might be suppressing output activity which is important to distinguish between closely distanced inputs. Furthermore, this experimental setup is sluggish, as training with BPTT is slower than in traditional ANNs and highly memory intensive. Kim et al. (Kim et al., 2019) have achieved impressive results without training the generated networks during the search phase, by estimating their future performance based on spike activation patterns across different data samples, and we believe this might be an important improvement to our framework. With faster experiments, we can focus on increasing diversity and coverage of the search space, so that SPENSER can yield better individuals.

## 7. Final Remarks

In this paper we propose SPENSER, a NE framework to automatically design CSNNs. SPENSER is able to generate competitive performing networks for image classification at the level of the state of the art, without human parametrization of the network's architecture and parameters. SPENSER generated networks with competitive results, attaining 99.42% accuracy on the MNIST (Kim et al., 2019) and 91.65% accuracy on the Fashion-MNIST (Kim et al., 2019) datasets. Current limitations rely on the execution time, due to the computationally intensive BPTT learning algorithm and the memory requirements. Furthermore, we believe the configuration of the loss function played a role in suppressing output activity and potentially decreasing accuracy.

### Future Work

In the future, we plan on:

* Experiment with different loss functions / encode the loss function as an evolvable macro parameter;
* Perform a more in-depth study of the preferred choices during evolution and observable patterns in the best-performing individuals. This could be relevant in uncovering novel optimal architectures and parameters;
* Experiment with different learning algorithms.
* Implement skip connections and back connections.
* Apply regularisation methods to prevent vanishing and exploding gradients.

###### Acknowledgements.

 This research was supported by the Portuguese Recovery and Resilience Plan (PRR) through project C645008882-00000055, Center for Responsible AI, by the FCT - Foundation for Science and Technology, IP./MCTES through national funds (PIDDAC), within the scope of CISUC R&D Unit - UIDB/00326/2020 or project code UIDP/00326/2020. The first author is partially funded by FCT - Foundation for Science and Technology, Portugal, under the grant 2022.11314.BD.
