Beyond Weights: Deep learning in Spiking Neural Networks with pure
  synaptic-delay training
Biological evidence suggests that adaptation of synaptic delays on short to
medium timescales plays an important role in learning in the brain. Inspired by
biology, we explore the feasibility and power of using synaptic delays to solve
challenging tasks even when the synaptic weights are not trained but kept at
randomly chosen fixed values. We show that training ONLY the delays in
feed-forward spiking networks using backpropagation can achieve performance
comparable to the more conventional weight training. Moreover, further
constraining the weights to ternary values does not significantly affect the
networks' ability to solve the tasks using only the synaptic delays. We
demonstrate the task performance of delay-only training on MNIST and
Fashion-MNIST datasets in preliminary experiments. This demonstrates a new
paradigm for training spiking neural networks and sets the stage for models
that can be more efficient than the ones that use weights for computation.
Edoardo W. Grappolini, Anand Subramoney
# Beyond Weights: Deep learning in Spiking Neural Networks with pure synaptic-delay training

###### Abstract.

Biological evidence suggests that adaptation of synaptic delays on short to medium timescales plays an important role in learning in the brain. Inspired by biology, we explore the feasibility and power of using synaptic delays to solve challenging tasks even when the synaptic weights are not trained but kept at randomly chosen fixed values. We show that training ONLY the delays in feed-forward spiking networks using backpropagation can achieve performance comparable to the more conventional weight training. Moreover, further constraining the weights to ternary values does not significantly affect the networks' ability to solve the tasks using only the synaptic delays. We demonstrate the task performance of delay-only training on MNIST and Fashion-MNIST datasets in preliminary experiments. This demonstrates a new paradigm for training spiking neural networks and sets the stage for models that can be more efficient than the ones that use weights for computation.

spiking neural networks, synaptic delays, neuromorphic computing

## 1. Introduction

Spiking neural networks (SNNs) (Grover et al., 2016) are biologically inspired neuron models that have recently become increasingly popular for deep learning use cases due to their potential for extreme energy efficiency on neuromorphic hardware. In these models, the complex electrochemical dynamics of a biological synapse are modelled in the connection between neurons and (typically) weights parameters. Neuroscientific literature classically focuses on the neuron and synaptic strength as the only factor of learning; this was because of several factors, the most trivial of which is that electrical activity is a relatively easy measurement of cellular and brain activity. Recent literature (Berg et al., 2016) suggests, however, that other types of cells could contribute to computations and learning in the brain. Glia cells, especially Oligo-dendrocytes, have been shown to be activated through learning processes (Grover et al., 2016; Grover et al., 2016). At a very high level, they work by wrapping the axon in a myelin sheath which can control the electrical signal speed through synapses and as a consequence, the time of the spike reception.

The role and importance of exact spike times are also considered in many studies (Berg et al., 2016; Grover et al., 2016; Grover et al., 2016) and the hypothesis is also reinforced by simulative works (Grover et al., 2016).

In this work, we explore the potential of computations being a direct consequence of synaptic delays in a network of spiking neurons. Although the idea of computations through delays comes from nature, our techniques come from the current technical literature in spiking neural networks, and we train our models through backpropagation. More precisely, we use an SNN based on SLAYER (Krause et al., 2015) and compare training only the synaptic delays with the conventional procedure of training weights of the network. We show that training just the synaptic delays can lead to competitive task performance on deep learning benchmarks - specifically MNIST and Fashion-MNIST.

## 2. Related Work

Many previous works explore the possibility of using time coding schemes and precise spike time learning in spiking neural networks as well as training the delays in an SNN. Examples include (Grover et al., 2016; Grover et al., 2016; Grover et al., 2016) that combines time coding and backpropagation for different types of neurons (IF, LIF), DL-ReSuMe (Krause et al., 2015), as an extension of ReSuMe (Krause et al., 2015) introduces the concept of delay training to improve performance and reduce weight adjustments, utilizing a supervised learning rule that is not backpropagation based. Hazan et al. (Hazan et al., 2015) is perhaps the most closely related work to ours, where only the delays of a weightless spiking neural network are trained using an STDP unsupervised learning rule to create latent representations of the MNIST dataset, which are then classified using a linear classifier. SLAYER (Krause et al., 2015) uses pseudo-derivatives to train axonal delays and weights using backpropagation with a spike response model (SRM) of spiking neurons. Our work was heavily influenced by SLAYER, although we train synaptic rather than axonal delays and explore the case where weights aren't trained. To our knowledge, we are the first to show that pure delay training using backpropagation with surrogate gradient method can achieve comparable performance to weight training.

## 3. Methods

### Spike response model (SRM)

We utilize the spike response model (SRM) and all the parameters were trained using surrogate gradients as in (Krause et al., 2015). This includes the synaptic weights and delays. For the convenience of the reader, we describe the notation necessary for the comprehension of our work.

Let \(s_{i}(t)=\sum_{f}\delta(t-t_{i}^{(f)})\) be one of a series of spike trains that reaches a neuron; \(t_{i}^{(f)}\) is the time of the \(f^{th}\) spike of the \(i^{th}\) input. Let \(e(\cdot)_{d}\) be a spike response kernel that also takes into consideration the _axonal_ delay. Then the membrane potential of the neuron that
we are taking into consideration is:

\[u(t) = \sum_{i}w_{i}(\epsilon_{d}*s_{i})(t)+(v*s)=w^{\top}a(t)+(v*s)(t)\] \[= \sum_{i}w_{i}(\epsilon(t-d)*s_{i})(t)+(v*s)=w^{\top}a(t)+(v*s)(t),\]

where \(*\) represents the convolution operation and \(\epsilon(\cdot)\) is a spike response kernel that does not take into account delays. Let \(\vartheta\) be the spiking threshold: an output spike is generated when the membrane potential \(u(t)\) reaches \(\vartheta\), more formally:

\[f_{s}(u):u\to s,s(t):=s(t)+\delta(t-t^{(f+1)})\] \[\text{where }\ t^{(f+1)}=\min\{t:u(t)=\vartheta,t>t^{(f)}\}.\]

### Synaptic delays

To achieve pure delay training, axonal delays are not sufficient since they lack expressivity. Training an axonal delay means in practice that, starting from a neuron in a layer \(i\), the delay applied to all the neurons of the following layer \(i+1\), is exactly the same, i.e. we have a fraction of the trainable parameters that we have when training the synaptic delays. Our implementation is prototypical and has not been optimized for a CUDA execution. Notice however that the number of _trainable_ parameters in a weight-based and _synaptic delay-based_ network is exactly the same; we expect therefore that a lower-level implementation would perform similarly to (Han et al., 2017) in terms of training time. In the simulated environment, the inference is badly influenced by the double operation needed in a synapse (application of delay + multiplication), but we expect advantages in a hardware context.

Taking into consideration the synaptic delays, the formulation is simply extended as:

\[u(t)=\sum_{i}w_{i}(\epsilon_{d_{i}}*s_{i})(t)+(v*s)=\sum_{i}w_{i}(\epsilon(t-d _{i})*s_{i})(t)+(v*s).\]

In all our experiments, the spike response kernels were:

\[\epsilon(t)=\frac{t}{\tau_{s}}\exp(1-\frac{t}{\tau_{s}})\Theta(t),v(t)=2 \vartheta\exp(1-\frac{t}{\tau_{r}})\Theta(t),\]

where \(\Theta(t)\) is the Heaviside step function, although the formulation is independent of the chosen kernel. Real-valued delays were stored for each synapse during training, but only the quantized values were used during inference. Quantization was obtained by a simple round _down_ to the nearest allowed number. With a simulation timestep of 1ms, this meant that if we had a synaptic delay of, say, \(4.421ms\), the spike is delayed by \(4ms\). A stochastic rounding (Kraus et al., 2017) method was tried, but did not lead to improvements and led to an increase to compute time; it was considered therefore not worthy at this stage.

### Loss and spike target

For all the experiments, a spike time-based loss was used. For a target spike train \(\hat{s}(t)\), for a time interval [0,T], the loss function was defined as:

\[E=\int_{0}^{T}L(s^{(m)},\hat{s}(t))dt=\frac{1}{2}\int_{0}^{T}(e^{m_{t}}(s^{(m )}(t),\hat{s}(t)))^{2}dt,\]

where \(L(s^{(m_{t})}(t),\hat{s}(t))\) is the loss at time instance \(t\) and \(e^{(m_{t})}(s^{(m_{t})},\hat{s}(t))\) is the error signal at the final layer.

Finally, the error signal was:

\[e^{(n_{I})}(s^{(n_{I})},\hat{s}(t))=e*(s^{(n_{I})}(t)-\hat{s}(t))=a^{(n_{I})} (t)-\hat{a}(t).\]

For the classification tasks, the target spike train was specified as the neuron corresponding to the correct class, spiking for the whole simulation period. The class was inferred by looking at the first neuron that spikes; in the case where more than one neuron spikes, we infer the class of the neuron that, at parity of arrival time, has the most spikes.

### Data encoding

For the image-based classification task we converted the non-temporal deep learning datasets to spiking encodings. In this case, because the trained parameters (the delays) work intrinsically in a temporal domain, we opt for temporal encoding. Although a temporal coding similar to what is done in (Han et al., 2017) was tried (i.e. for each pixel, a spike temporally placed in a proportional way to the pixel intensity), we found that a simpler encoding was more effective in terms of accuracy performance for our setup. The strategy that we used in practice is one where, for each pixel, if the greyscale value is higher than 127 we get a spike, and otherwise we don't get a spike.

### Training delays

Training delays is a fundamentally different operation, semantically, than training weights. Here we attempt to give an intuitive explanation at a high level, of what the neural dynamics can be. The explanation applies generally to all neuronal models. The spikes cause an increase of the membrane potential \(u(t)\) in the receiving neuron - see Figure 1. As the membrane potential reaches a threshold \(\vartheta\), the receiving neuron will output a spike. Training synaptic weights act on how much a spike will impact the magnitude of the membrane potential: this is a direct action on the amplitude of the receiving spike (Bahdan et al., 2017). Training delays, on the other hand, act on when (or if) a spike is generated, following different dynamics. Remembering that generation of a spike is the consequence of a change in amplitude, we consider the latter, as the rest follows naturally. Considering Figure 2: (A) shows a possible dynamics of a neuron receiving two spikes. Consider the magnitude \(\hat{u}\) as the maximum value of the membrane potential \(u(t)\) as the consequence of the spikes in this case, we will show how we can obtain a lower or higher maximum membrane potential value compared to \(\hat{u}\). If we want to use delays to change the amplitude of the membrane potential, in the case depicted, we have two possibilities, delay the first spike, or the second. We can see that, delaying the second spike leads to the silhouette of the membrane

Figure 1. Spiking neuron dynamics. As a neuron receives a spike, a change in the membrane potential is obtained. As the membrane potential reaches a threshold \(\vartheta\) a spike is generated
potential never reaching \(\hat{u}\) (Figure 2.B). Delaying the first spike, leads to a \(u(t)\) function that surpasses \(\hat{u}\) (Figure 2.A).

### Training procedure

All our experiments were on a fully connected network with one hidden layer of 800 neurons (784-800-10). The optimization was done with the Adam (Kingmare et al., 2014), with an initial learning rate of 0.01 and a batch size of 32. The training set was split to train and validation with an 80%, 20% split, and we report the test accuracy in the tables. Weights were initialized following a normal distributions: \(N(0.0571,0.5458)\) for the first layer and \(N(-0.5244,1.0490)\) for the second layer and scaled by a factor of \(x10\) as in (Kolmogorov, 2015), based on analysing the weight distributions of weight-trained networks

The weights are then scaled by a factor of \(x10\), following the specifications of SLAYER. For the initialization of the _constrained weights_, the same strategy was used, with the addition that, before applying the multiplicative factor, we apply the following, simple rule: let \(w\in\mathbb{R}\) be a weight of the network. Let \(\hat{w}\) be the quantized version of the weight. Then, \(\hat{w}=round(w)\) where \(round(\cdot)\) rounds \(w\) to the nearest integer in the set \(\{-1,0,1\}\).

For the delays initialisation, we kept the same strategy used in (Kolmogorov, 2015): A uniform random initialisation between 0 and 1. In practice, this means that initially, the applied delay will be 0 in the forward pass, but non-zero initialization allows the avoidance of gradient problems in the backward pass. In some trials, a random delays initialisation in \([0,n]\) with different values for \(n\in\mathbb{R}\) was also tried, but it did not lead to any performance improvements.

_Parameters of the simulation._ All simulations have a duration of 10 ms, and time is quantized with 1 ms precision. The spiking threshold was set to \(\hat{\vartheta}=10\,\mathrm{mV}\), the time constant to \(\tau_{s}=1\,\mathrm{ms}\) and the refractory time constant to \(\tau_{r}=1\,\mathrm{ms}\).

## 4. Results

We test pure delay training on two datasets, the MNIST (Krizhevsky et al., 2014) handwritten digits dataset and the Fashion-MNIST (Krizhevsky et al., 2014) dataset. We compare the results with weight training, and other similar methods in literature.

### Mnist

We show in Figure 3 the learning curves for our experiments. In the case of MNIST, we can see how the weights baseline fits the training dataset quickly, and our two methods do not achieve the same training accuracy. However, viewing the validation accuracy curve, we can see how, without additional regularizers, weights training overfits, in contrast to our method. With both free and constrained weights initialisation, we achieve an accuracy improvement over (Kolmogorov, 2015), which is the only work that attempts a similar approach to ours (albeit not fully supervised). Our goal here is to demonstrate a proof-of-concept that pure delay training is competitive rather than achieve state-of-the-art accuracy.

### Fashion-MNIST

In Figure 4, we present the learning curves for the experiments on the Fashion-MNIST dataset. In this case, we observe how the overfitting on the training set for the weight training is even more evident than the previous case; weight training achieves over 97% training accuracy on the training set, delay training stops at 92%. We can see from the validation curves how, as epochs pass, our delay training method is more robust to overfitting, indeed, at the 100th epoch mark the weights training baseline has lower accuracy values than both our delay training methods. We also notice that our methods get very close to the weights training baseline for the best validation model, and are not far behind the non-spiking and basic ANN baseline.

\begin{table}
\begin{tabular}{l c c c c} \hline \hline  & **Coding** & **Training Method** & **Neuron model** & **Accuracy (\%)** \\ \hline \hline SNNN (Krizhevsky et al., 2014) & Temporal & Shcriptor & IF & 97.4 \\ Memoriess with delays (Kolmogorov, 2015) & Temporal & STIP & LIF & 93.5 \\ \hline ANN-\(\hat{\epsilon}\) (\(\cdot\)) & \multirow{2}{*}{\begin{tabular}{} \end{tabular} } & \multirow{2}{*}{\begin{tabular}{} \end{tabular} } & \multirow{2}{*}{\begin{tabular}{} \end{tabular} } & \multirow{2}{*}{\begin{tabular}{} \end{tabular} } \\ SLAYER weights baseline (\(\cdot\)) & Temporal & Shcriptor & SNM & 96.1\(\pm\)0.1 \\ \hline Delay training & \multirow{2}{*}{\begin{tabular}{} \end{tabular} } & \multirow{2}{*}{\begin{tabular}{} \end{tabular} } & \multirow{2}{*}{\begin{tabular}{} \end{tabular} } & \multirow{2}{*}{
\begin{tabular}{} \end{tabular} } \\ \cline{1-1} \cline{5-5} \cline{5-
## 5. Discussion

We have demonstrated a proof of concept that training just the delays in a spiking neural network can work, as well as training weights. We showed that on both MNIST and Fashion-MNIST datasets, pure delay training achieves task performance comparable to conventional weight training, with delay training having a slight advantage in not overfitting the dataset. Moreover, we demonstrated that even when the weights are randomly initialised to ternary values ({+x,0,-x}), the task performance of the networks remains good.

This is the first step toward understanding the computational power of delays in spiking neural networks for biology and machine learning. Input and output encodes that use better temporal information and more precise delay training methods, and using pure delay-training for more powerful event-based models such as the EGRU (Kang et al., 2021) are potential ways to extend this in future work.

A forward pass in a spiking network using only delays with these ternary weights can also be implemented significantly more efficiently than a network that uses floating point weights. In software, such a forward pass can use just matrix roll operations combined with addition/subtraction instead of multiply-accumulate. In neuromorphic computing, our work suggests new ways of configuring the hardware to achieve extremely efficient inference. It might especially be relevant to analog and photonic (Krizhevsky et al., 2014) neuromorphic devices, although significant savings can also be realised in other devices. Overall, this work demonstrates a new paradigm of using time in and training spiking neural networks.

###### Acknowledgements.

 This work was co-funded by the European Union within the Erasmus+ traineeship program.

AS was funded by the Ministry of Culture and Science of the State of North Rhine-Westphalia, Germany during part of this work. We would like to thank Andrew D. Bagdanov, Laurenz Wiskott and the Institute for Neural Computation at Ruhr University Bochum for institutional and infrastructural support.
