[1]

[1]By using STDP and homeostasis on spiking neural networks simulated on
SpiNNaker, Davies et al. demonstrate that it is possible for such a network to
learn and recognise patterns of spikes by presenting to the network only once
the desired pattern. The pattern is presented as a set of simultaneous spikes
at the input layer, and the output is produced after a short delay. In
addition, the same network is trained with more than one pattern, and the
accuracy and other performance metrics are computed.

[type=author, prefix=Dr, orcid=0000-0001-5330-5527]

[1]

[1]

url]https://www.mmu.ac.uk/staff/profile/dr-sergio-davies

Conceptualisation of this study, Design of the methodology, Software
development, Interpretation of the results, Preparation of the article

[1]Corresponding author

1]organization=Department of Computing and Mathematics, Manchester
Metropolitan University, addressline=John Dalton Building, All Saints Campus,
Oxford Road, city=Manchester, postcode=M15 6BH, state=United Kingdom,
country=U.K.

[type=author, prefix=Dr]

Support on the definition of the methodology and on the interpretation of the
results, Software development, Preparation of the article

[type=collab, prefix=Dr]

Software development, Review of the article

2]organization=APT group, School of Computer Science, The University of
Manchester, addressline=IT Building, Oxford Road, city=Manchester,
postcode=M13 9PL, state=United Kingdom, country=U.K.

[type=collab, prefix=Prof.]

Review of the article

3]organization=Department of Computing, Sheffield Hallam University,
addressline=Cantor Building, 153 Arundel Street, city=Sheffield, postcode=S1
2NU, state=United Kingdom, country=U.K.

# Supervised learning of spatial features with STDP and homeostasis using
Spiking Neural Networks on SpiNNaker

Sergio Davies<EMAIL_ADDRESS>[  Andrew Gait  Andrew Rowley  [
Alessandro Di Nuovo  [

###### Abstract

Artificial Neural Networks (ANN) have gained large popularity thanks to their
ability to learn using the well-known backpropagation algorithm. On the other
hand, Spiking Neural Networks (SNNs), despite having wider abilities than
ANNs, have always presented a challenge in the training phase. This paper
shows a new method to perform supervised learning on SNNs, using Spike Timing
Dependent Plasticity (STDP) and homeostasis, aiming at training the network to
identify spatial patterns. The method is tested using the SpiNNaker digital
architecture. A SNN is trained to recognise one or multiple patterns and
performance metrics are extracted to measure the performance of the network.
Some considerations are drawn from the results showing that, in the case of a
single trained pattern, the network behaves as the ideal detector, with
$100\%$ accuracy in detecting the trained pattern. However, as the number of
trained patterns on a single network increases, the accuracy of the
identification is linked to the similarities between these patterns. This
method of training an SNN to detect spatial patterns may be applied on pattern
recognition in static images or traffic analysis in computer networks, where
each network packet represents a spatial pattern. It will be stipulated that
the homeostatic factor may enable the network to detect patterns with some
degree of similarities, rather than only perfectly matching patterns.

###### keywords:

Spiking Neural Networks SNN Spatial Pattern STDP Spike Timing Dependent
Plasticity Supervised Learning

##  1 Introduction

The rising popularity of neural networks can be attributed to their
information processing capabilities, despite being regarded as “black-box”
systems due to their emulation of the behaviour of biological neural networks,
rather than relying on established biological structures (Prieto et al.,
2016).

Artificial neural network models draw inspiration from their biological
counterparts trying to mimic how the human brain performs specific tasks
(Haykin, 1999). On the basis of the computational unit (neurons) in use in
these networks, we can classify three main categories of neural networks
(Maass, 1997). Each of these categories is referred to as a “generation” of
neural network. Each generation simulates biological processes with an
increasing degree of accuracy.

The first generation of neural networks were dominated by the McCulloch-Pitts
neuron model (McCulloch and Pitts, 1943) which allows discrete inputs and
outputs (only “0”s or “1”s). The next generation (second generation of neural
networks, more commonly known as Artificial Neural Networks) evolved this
model to allow input and output values to be continuous in a specified range,
either $[0;1]$ or $[-1;1]$.

In both these generations of neural networks the output of a neuron is
transferred to the subsequent neuron(s) through weighted connections. This
weight is altered during the training phase by presenting to the network an
input for the training session, analysing the network output, and determining
the error between the current network output and the desired output. This
error is then used to compute the changes in synaptic weights throughout the
network using the backpropagation algorithm (Linnainmaa, 1976; Schmidhuber,
2015).

It is suggested that the popularity of Artificial Neural Networks (ANNs) can
be attributed to the use of the backpropagation training algorithm (LeCun et
al., 2015). This algorithm has played a pivotal role in enabling the training
of significant ANN models (e.g.: Sermanet et al., 2013; Mohamed et al., 2012;
Dahl et al., 2012). Indeed, backpropagation has provided an advantage to this
type of networks with a well-known and robust method of training which has now
been embedded in most, if not all, ANN simulation platforms.

However, both the first and the second generations of neural networks do not
consider one fundamental aspect: biological networks evolve following
biological time. Artificial neural networks perform their operations in an
abstract time that does not relate with biological time. Even advanced models
such as Continuous Timescale Recurrent Neural Networks (CTRNN, Funahashi and
Nakamura, 1993) and Multi-Timescale Recurrent Neural Networks (MTRNN,
Yamashita and Tani, 2008) use the recurrent structure of the neural network to
keep track of the state and its evolution, even though this does not relate to
biological real-time, but rather follows the number of iterations in the
network.

The third generation of neural networks (Maass, 1997), also known as Spiking
Neural Networks (SNNs), improves the biological realism of previous
generations of neural and synaptic models by introducing the time variable in
the models (Gerstner et al., 2014; Kasabov, 2019). Indeed, the models proposed
for this generation of neural networks are directly inspired from biology: the
most realistic model is the Hodgkin-Huxley neuron (Hodgkin and Huxley, 1952),
which is also the most complex to simulate numerically on a computer. Other
models, instead, limit their biological plausibility to reduce their numerical
complexity (Izhikevich, 2004). All of these neuron models are described using
differential equations that depict the evolution of a neuron’s state over time
(e.g.: Izhikevich, 2003; Lapicque, 1907; Dayan and Abbott, 2001; Kasabov,
2019).

The communication between neurons is also inspired from biology: it is known
that neurons interact by means of action potentials, also known as spikes,
(Dayan and Abbott, 2001). Such communications use a wide array of mechanisms
to encode information, as described by Auge et al. (2021), and even more
methods could be envisioned.

Similarly, also synapses (the interconnection between neurons) in second
generation neural networks are represented by a single number that represents
its “strength”. This value is altered during training using the
backpropagation algorithm. However, from a biological perspective, this
algorithm has raised some skepticism on its plausibility (Tavanaei et al.,
2019). This is also supported by the fact that biological findings have shown
that signals transmitted through synapses have a time evolution that may be
described through a differential equation (Kasabov, 2019).

In addition, biology has described a number of mechanisms that allow
biological neural networks to adapt to input stimuli. Among these we can
mention STP – short-term plasticity (Zucker and Regehr, 2002), STDP – Spike
Timing Dependent Plasticity (Markram et al., 1997; Bi and Poo, 2001),
homeostasis (Turrigiano, 1999, 2011), structural plasticity (Zecevic and
Rakic, 1991) and evolutionary learning (Schmidhuber, 2015). All these
mechanisms, following different processes, alter the architecture of the
neural network by altering the synaptic strength of existing synapses, by
creating new synapses (synaptogenesis), or by removing existing synapses
(synaptic pruning). Despite all the changes imposed by these mechanisms,
neurons within the network need to maintain their functional stability, and
the network itself needs to keep a stable behaviour. This happens through the
homeostatic process at two levels: on a neuron scale it helps to keep a
healthy neural activity, while on the network scale homeostasis helps keeping
the network stability (Maffei and Fontanini, 2009).

The learning process underlying mechanism was proposed by Hebb (1949), and
commonly summarised as _“Cells that fire together wire together”_. More
details of this biological process have emerged in the last few decades,
leading to a number of learning rules which can affect synapses on a time
interval spanning a few milliseconds to a lifetime, or more, through
generations of individuals (Zeraati et al., 2021; Nolfi et al., 1994).

As a general rule, the longer the learning period, the more permanent the
effects are on the neural networks: short-term plasticity affects quickly the
stability of the network, but the effects do not last very long (Tsodyks and
Wu, 2013). On the other hand, long-term plasticity has a stronger impact on
the network, so that it allows the network to self-organise towards a stable
critical regime (Zeraati et al., 2021). Evolutionary learning has an even
stronger impact that allows generation of individuals to behave in a specific
way innately (Tierney, 1986).

Such learning rules have been replicated in computer simulations, and showed
their characteristics in applied tasks. In particular, it is relevant to
mention that STDP was successfully applied in many applications related to the
identification of spatio-temporal spike patterns (e.g.: Davies, 2012;
Guyonneau et al., 2005; Masquelier et al., 2008; Davies et al., 2012).

A neural network is trained within an environment. On the basis of this it is
possible to classify four learning paradigms (Kasabov, 2019) depending on the
presence and the structure of the teaching signal: supervised learning, semi-
supervised learning, unsupervised learning and reinforcement learning. The
learning rules introduced before (STP, STDP, etc.) refer to an unsupervised
learning paradigm, where the teaching signal is absent and the network aims to
identify autonomously a pattern in the input signal.

In this paper we present a new method of training a spiking neural network to
identify spatial patterns (patterns of spikes presented at the same time as
input to the network) using STDP and homeostasis. The network is trained
initially to identify a single pattern and the accuracy is then evaluated by
testing exhaustively all the possible input patterns to the network. In a
second step, the network is trained to identify two patterns, and we will show
that the accuracy of the identification depends strictly on the degree of
similarity between the two patterns on which the network has been trained.
This similarity will be measured by the Hamming distance between input
patterns. Finally, a more thorough experiment includes training the network on
three patterns and measuring again the detection accuracy, among other
classification metrics.

The experiments are performed on the SpiNNaker digital architecture (Furber
and Bogdan, 2020), using the sPyNNaker implementation of the PyNN neural
network language (Rhodes et al., 2018; Rowley et al., 2019). SpiNNaker is a
system designed at the University of Manchester: each SpiNNaker chip comprises
eighteen very-low-power ARM986 processors (cores); the main SpiNNaker server,
housed at the University of Manchester, consists of a million cores built of
multiple boards containing multiple chips. Computations in the brain are
inherently parallel and the architecture is designed to mimic this
parallelism. SNNs may be simulated on the machine by submitting scripts based
on the PyNN neural network language. These scripts are then converted by the
software stack into executable files which run on as many cores as required by
the neural network.

The remaining sections of this article encompass a detailed account of the
experiments outlined in the methodology section (Section 2). This includes an
in-depth exploration of the STDP learning rule (Section 2.1), training
procedures (Section 2.2), and testing methods (Section 2.3). Subsequently, the
results section (Section 3) will shed light on the research outcomes,
involving training the network on a single pattern (Section 3.1), two patterns
(Section 3.2), and multiple patterns (Section 3.3). In each case, various
classification metrics will be employed to assess the network’s performance in
pattern identification following the training process. Finally, the conclusion
section (Section 4) will summarise the key findings of this research.

##  2 Methodology

In this paper we refer to spatial patterns of spikes as a set of spikes that
are presented to the network from different source neurons at the same time,
and whose source neuron is meaningful for the pattern.

As spatial patterns relate only to the presence or absence of a spike from a
specific source, this type of patterns can be identified and encoded with the
use of binary numbers, where “1”s represent the presence of a spike, while
“0”s reflect its absence. These numbers represented either in their binary or
decimal format will also be referred to as “code words”.

The spikes used to transfer information follow two of the possible encodings
suggested by Auge et al. (2021), namely:

  * •

Time To First Spike: This is utilised during the network training phase to
ensure that the supervised learning paradigm correctly triggers the relevant
side of the STDP learning rule. Depending on whether the input spike
represents “0” or “1”, the generated spike occurs slightly before or after the
training signal.

  * •

Parallel binary encoding: Since spatial patterns only correspond to the
presence or absence of spikes from each source, we can represent patterns with
binary numbers that are presented to the network. A binary number “1”
indicates the presence of a spike from that source, while “0” represents its
absence.

Two related neural networks are designed for this exercise: the first one is
used to train the relevant synapses, while the second network, which is a
simplified version of the training network, is used to test and validate the
model obtained in the first step.

###  2.1 STDP on SpiNNaker

Figure 1: An example of how weight change (dW) is calculated based on the time
difference between pre- and post-synaptic spikes (dt). In green on the left is
the Long Term Potentiation (LTP) generated by a pre-synaptic and post-synaptic
spike sequence. In blue on the right is the Long Term Depression (LTD)
generated by a post-synaptic and pre-synaptic spike sequence.

STDP is a form of learning whereby the weight of a synapse between two neurons
is either potentiated (LTP) or depressed (LTD) dependent upon whether a post-
synaptic spike follows or precedes a pre-synaptic spike. The size of this
change, in general, drops off exponentially as the time difference between the
pre- and post-synaptic spikes gets larger (Sjöström and Gerstner, 2010). This
is shown graphically in fig.1. In PyNN, STDP is defined in a modular fashion
such that the user may specify which timing rule (for example, to determine
the shape of the exponential decay) and weight update rule (for example, to
indicate whether the weight update is additive or multiplicative) they wish to
use.

Figure 2: The sequence of spikes in the network used for training. Figure 3:
The network used for training. In blue the STDP-enabled synapses.

This is how the rules are also implemented on SpiNNaker, with one proviso: due
to local memory restrictions on how much data can be held for parameters,
multiple STDP projections to the same target population must use the same rule
with the same parameters.

On SpiNNaker, the plasticity mechanism for STDP is also only activated when
the post-synaptic neuron receives the second (pre-synaptic) spike: at least
two pre-synaptic spikes are, therefore, required for the calculations to take
place. This is because the conventional method for calculating STDP at every
pre-synaptic spike and every post-synaptic spike is difficult on SpiNNaker due
to the synaptic weights being held in external memory and only copied into
local memory when a pre-synaptic spike arrives. Thus, a deferred event-driven
model is used to postpone the STDP calculation until future spike timings
determine how the pre-synaptic sensitive scheme is applied (Jin et al., 2010).
Because of this deferred event-driven model, STDP weight changes can only be
computed when a pre-synaptic spike is received. Therefore, to detect the
effects of the sequence of spikes to the output neuron at the end of the
training phase, a “save neuron” (see fig.2 and fig.3) emits a final spike
whose only effect is to trigger the execution of the STDP learning rule on
plastic synapses.

###  2.2 Training phase

The training phase relies on the network shown in fig.3. This network consists
of two sections: one focused on the “0”s on the left, and, symmetrically,
another section dedicated to the “1”s on the right.

The “Spike Source Populations” inject spikes according to specific patterns to
train the network. The “Spike injector” populations comprise leaky integrate-
and-fire neurons with delta synapses. These synapses have the characteristic
that the current transferred to the post-synaptic neuron is applied within a
single-millisecond time slot, where it receives all the current. The neuron
parameters are set to the default values provided by the PyNN (Davison et al.,
2009) interface to the SpiNNaker backend simulator. Synapses are initialized
with a weight of zero, indicating that the output spike is generated
exclusively by the teacher neuron’s contribution. Additionally, the plastic
synapses employ the nearest-neighbor spike pair rule, which is a custom
extension for the SpiNNaker backend simulator in PyNN. The parameters used for
the weight update rule are as follows: $\tau_{+}=5$, $\tau_{-}=5$, $A_{+}=1$
and $A_{-}=-1$. However, later in this article it will be discussed that the
specific values of these parameters have little relevance on the whole set of
experiments.

The post-synaptic neurons in the spike injector populations emit a spike for
every spike they receive. These spikes are then transmitted to the output
neuron through synapses enabled with STDP (Spike Timing-Dependent Plasticity).

Through these synapses, in fact, the output neuron receives a specific set of
spikes from each injector population, as indicated in fig.2.

Figure 4: The precise timing of the spikes for potentiation and depression.
The timing for both Long Term Potentiation (LTP) and Long Term Depression
(LTD) is considered on the timestep immediately following the outgoing spike
value, so the values concerned here are 5 ms when the "1" is potentiated and
the "0" is depressed, and 25 ms when the "0" is potentiated and the "1" is
depressed.

The encoding of the spikes follows the “time to first spike” method. The
spikes allow for the synapses propagating “1”s to be potentiated and for the
“0”s synapses to be depressed by an equal amount. Details of the precise spike
timing effective on plastic synapses can be seen in fig.4.

The synaptic weights so obtained are then used in the testing network in
fig.5. The input pattern is injected in this network through the “Spike
Injectors 0” and “Spike Injectors 1” populations. In the first population a
neuron fires if the corresponding bit of the input pattern is a “0”. On the
contrary, if the bit is a “1”, then the corresponding neuron of the “Spike
Injectors 1” population fires. In addition, all synapses are fixed, and the
excitatory weights originating from “Spike injector 0” or “Spike injector 1”
to the output neuron mirror the patterns learned by the synapses in the
corresponding locations of the preceding network. In contrast, the inhibitory
weights stemming from “Spike injector 0” to the output neuron are guided by
the weights learned by “Spike injector 1”, and conversely, the inhibitory
weights originating from “Spike injector 1” to the output neuron are
influenced by the weights learned by “Spike injector 0”. This reciprocal
relationship between excitatory and inhibitory weights ensures that the output
neuron is able to select the pattern to respond to.

In the cases where the network needs training on multiple pattern, each
pattern will be trained separately, and always starting from a network with
plastic synaptic weights equal to “0”, and the final value of the synaptic
weights will be obtained by summing the weights resulting from training
synapse-by-synapse. The weights obtained from the previous step are used as
the basis for the homeostatic process. To complete this step, the testing
network is used and each of the training pattern is injected. All the weights
are re-scaled during multiple iterations so to obtain the minimum scaling
value for each training pattern that allows the output neuron to fire exactly
once for each of the learned pattern. Then the maximum among these factors is
selected as the homeostatic factor.

The search for the re-scaling factor is performed initially through a binary
search in the interval between the values of $0.0001$ and $1000$, reducing the
interval until the values between the two extremes $x$ and $y$ is less than or
equal to $0.0001$. Then the search becomes linear in the interval
$[x-0.00001;y+0.00001]$ with a step equal to $0.00001$, one order of magnitude
smaller than the interval. The precision of this step descends from the
precision of the synaptic weights on SpiNNaker. This precision is determined
at runtime based upon the maximum weight value possible within the network
(van Albada et al., 2018). For the network described here the minimum weight
that can be represented is $2^{-11}\approx 0.0005$, so a linear search with a
step size equal to $0.00001$ does not reduce the precision of the network.
Once the homeostatic factor is determined to allow the network to spike once
for every pattern presented, the process moves to the testing and validation
phase. Because the homeostatic process re-scales all weights to obtain a
specific required result, the original weights obtained through the STDP is of
little relevance to the whole training process.

Name | Variable | Description
---|---|---
|  |
| Positives
---
| $P_{i}$
---
Equals to $1$ in case the network emits a spike associated with the i-th input
pattern
| Negatives
---
| $N_{i}$
---
Equals to 1 in case the network does not emit a spike associated with the i-th
input pattern
| True
---
Positives
| $t_{p}$
---
Number of spikes emitted associated with patterns learned by the network
| True
---
Negatives
| $t_{n}$
---
Number of patterns correctly not identified by the network
| False
---
Positives
| $f_{p}$
---
Number of spikes emitted, but not associated with patterns learned by the
network
| False
---
Negatives
| $f_{n}$
---
Number of patterns on which the network was trained but that the network
failed to identify
Name | Formula | Description
|  |
| Accuracy
---
$\frac{t_{p}+t_{n}}{t_{p}+t_{n}+f_{p}+f_{n}}$ |  Proximity of the identification task to the training. It evaluates the overall performance of classification
| Precision
---
$\frac{t_{p}}{t_{p}+f_{p}}$ |  Positive predicted value. This indicates the reliability of identification
| Negative
---
Prediction
$\frac{t_{n}}{t_{n}+f_{n}}$ |  Reliability of classification of distractions
| Sensitivity
---
$\frac{t_{p}}{t_{p}+f_{n}}$ |  Focuses on how good is the performance in classifying attention
| Specificity
---
$\frac{t_{n}}{t_{n}+f_{p}}$ |  Evaluates the performance in classifying distractions
Table 1: Classification metrics used for the evaluation of the performance of
trained networks. Figure 5: The network used for testing.

###  2.3 Test and Validation phase

During this phase, the objective is to validate the methodology for the
training of a spatial feature classifier. To this end, we calculate the
following classification metrics: Accuracy, Precision, Negative Prediction,
Sensitivity, and Specificity (as presented in Davies et al., 2021). These
metrics are employed for class identification, as detailed in Table 1.

The network utilised is depicted in Figure 5. All neurons operate as leaky
integrate-and-fire units with delta synapses, as previously described.

##  3 Results

###  3.1 Single pattern training

To create a sufficiently broad testing space, the network in fig.5 undergoes
testing and validation using 10-bit patterns. These patterns are represented
by numbers in the range $[0;1023]$, where their binary representation
effectively reflects the combination of spikes in the pattern. The initial
test focuses on the pattern expressed by the number $992_{10}=1111100000_{2}$
(the subscript numbers represent the base in which the code word is
expressed). Since the network comprises only 10 synapses per injector
population, the details of the weights generated by the training step are
documented fully in table 2 to provide context for the discussion.

| Neuron ID
---|---
Population | 9 | 8 | 7 | 6 | 5 | 4 | 3 | 2 | 1 | 0
Injector “0” | $0$ | $0$ | $0$ | $0$ | $0$ | $0.367$ | $0.367$ | $0.367$ | $0.367$ | $0.367$
Injector “1” | $0.367$ | $0.367$ | $0.367$ | $0.367$ | $0.367$ | $0$ | $0$ | $0$ | $0$ | $0$

Table 2: Trained synaptic weights for a single pattern expressed by the number
$992_{10}=1111100000_{2}$

It is evident that the weights precisely mirror the pattern of “0”s and “1”s
in the pattern. Subsequently, homeostasis is applied to the group of synapses
to ensure that the output neuron fires once when the pattern is presented to
the network. The resulting homeostasis factor is computed as $4.18817$, which
re-scales the weight values to those presented in table 3.

| Neuron ID
---|---
Population | 9 | 8 | 7 | 6 | 5 | 4 | 3 | 2 | 1 | 0
Injector “0” | $0$ | $0$ | $0$ | $0$ | $0$ | $1.538$ | $1.538$ | $1.538$ | $1.538$ | $1.538$
Injector “1” | $1.538$ | $1.538$ | $1.538$ | $1.538$ | $1.538$ | $0$ | $0$ | $0$ | $0$ | $0$

Table 3: Re-scaled synaptic weights (after homeostasis)

With the weights outlined in the latter table, the network is validated using
all possible combinations of spikes, showing that it produces only one output
spike in response to the input $992_{10}=1111100000_{2}$, in accordance with
the training provided, demonstrating perfect pattern recognition, as shown in
table 4.

###  3.2 Dual pattern training

In addition to the single-pattern testing, a set of two-pattern training
experiments has been conducted to investigate how training a single network on
multiple patterns influences the recognition process. Building upon the
previous experiment, this set of experiments aims to elucidate how the
disparity between the two learned patterns impacts the recognition process.
The first experiment aims to train the network to identify the patterns
$992_{10}=1111100000_{2}$ and $960_{10}=1111000000_{2}$. Between the two
patterns there is only one bit difference in position $5$. Network training
takes this difference into account both at STDP training stage and at the
homeostasis adjustment stage. Indeed, the synaptic weights for the plastic
synapses to the output neuron achieve the values presented in table 5.

Metric | Formula | Value
---|---|---
Positives | $P_{i}$ | $1$
Negatives | $N_{i}$ | $1023$
True Positives | $t_{p}$ | $1$
True Negatives | $t_{n}$ | $1023$
False Positives | $f_{p}$ | $0$
False Negatives | $f_{n}$ | $0$
Accuracy |  $\frac{t_{p}+t_{n}}{t_{p}+t_{n}+f_{p}+f_{n}}$ | $\frac{1024}{1024}=1$
Precision |  $\frac{t_{p}}{t_{p}+f_{p}}$ | $\frac{1}{1+0}=1$
| Negative
---
Prediction
$\frac{t_{n}}{t_{n}+f_{n}}$ | $\frac{1023}{1023+0}=1$
Sensitivity |  $\frac{t_{p}}{t_{p}+f_{n}}$ | $\frac{1}{1}=1$
Specificity |  $\frac{t_{n}}{t_{n}+f_{p}}$ | $\frac{1023}{1023}=1$
Table 4: Classification metrics for the network trained with a single pattern.

| Neuron ID
---|---
Population | 9 | 8 | 7 | 6 | 5 | 4 | 3 | 2 | 1 | 0
Injector “0” | $0$ | $0$ | $0$ | $0$ | $0.367$ | $0.734$ | $0.734$ | $0.734$ | $0.734$ | $0.734$
Injector “1” | $0.734$ | $0.734$ | $0.734$ | $0.734$ | $0.367$ | $0$ | $0$ | $0$ | $0$ | $0$

Table 5: Trained synaptic weights for two patterns expressed by numbers
$992_{10}=1111100000_{2}$ and $960_{10}=1111000000_{2}$

At a first glance, two main differences become evident when comparing these
values with those related to the single-pattern experiment: in the first
instance, weights of neurons $0$ to $4$ and $6$ to $9$ are doubled. This is
because the two-pattern experiment sums the corresponding synapses trained
independently on the two patterns. Therefore these synapses are reinforced
twice, and their weights are doubled.

Pattern 1 | $992_{10}$ | $992_{10}$ | $992_{10}$ | $992_{10}$ | $992_{10}$ | $992_{10}$ | $992_{10}$ | $992_{10}$ | $992_{10}$ | $992_{10}$ | $992_{10}$ | $992_{10}$ | $992_{10}$ | $992_{10}$
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---
Pattern 2 | $1008_{10}$ | $1016_{10}$ | $1020_{10}$ | $1022_{10}$ | $1023_{10}$ | $960_{10}$ | $896_{10}$ | $768_{10}$ | $512_{10}$ | $0_{10}$ | $16_{10}$ | $24_{10}$ | $28_{10}$ | $30_{10}$
| Hamming
---
distance
$1$ | $2$ | $3$ | $4$ | $5$ | $1$ | $2$ | $3$ | $4$ | $5$ | $6$ | $7$ | $8$ | $9$
| Homeostasis
---
factor
$2.326$ | $2.617$ | $2.991$ | $3.490$ | $4.187$ | $2.326$ | $2.617$ | $2.991$ | $3.490$ | $4.187$ | $5.235$ | $6.981$ | $10.470$ | $20.941$
Positives | $2$ | $4$ | $8$ | $16$ | $32$ | $2$ | $4$ | $8$ | $16$ | $32$ | $64$ | $128$ | $256$ | $512$
Negatives | $1022$ | $1020$ | $1016$ | $1008$ | $992$ | $1022$ | $1020$ | $1016$ | $1008$ | $992$ | $960$ | $896$ | $768$ | $512$
| True
---
Positives
$2$ | $2$ | $2$ | $2$ | $2$ | $2$ | $2$ | $2$ | $2$ | $2$ | $2$ | $2$ | $2$ | $2$
| True
---
Negatives
$1022$ | $1020$ | $1016$ | $1008$ | $992$ | $1022$ | $1020$ | $1016$ | $1008$ | $992$ | $960$ | $896$ | $768$ | $512$
| False
---
Positives
$0$ | $2$ | $6$ | $14$ | $30$ | $0$ | $2$ | $6$ | $14$ | $30$ | $62$ | $126$ | $254$ | $510$
| False
---
Negatives
$0$ | $0$ | $0$ | $0$ | $0$ | $0$ | $0$ | $0$ | $0$ | $0$ | $0$ | $0$ | $0$ | $0$
Accuracy | $1$ | $0.998$ | $0.994$ | $0.986$ | $0.97$ | $1$ | $0.998$ | $0.994$ | $0.986$ | $0.97$ | $0.939$ | $0.877$ | $0.752$ | $0.502$
Precision | $1$ | $0.5$ | $0.25$ | $0.125$ | $0.0625$ | $1$ | $0.5$ | $0.25$ | $0.125$ | $0.0625$ | $0.0312$ | $0.0156$ | $0.00781$ | $0.00391$
| Negative
---
Prediction
$1$ | $1$ | $1$ | $1$ | $1$ | $1$ | $1$ | $1$ | $1$ | $1$ | $1$ | $1$ | $1$ | $1$
Sensitivity | $1$ | $1$ | $1$ | $1$ | $1$ | $1$ | $1$ | $1$ | $1$ | $1$ | $1$ | $1$ | $1$ | $1$
Specificity | $1$ | $0.998$ | $0.994$ | $0.986$ | $0.971$ | $1$ | $0.998$ | $0.994$ | $0.986$ | $0.971$ | $0.939$ | $0.877$ | $0.751$ | $0.501$

Table 6: Classification metrics for the network trained with two patterns

In the second instance, it is possible to notice that the weights of the
synapses from neuron $5$ is evenly distributed between the two injector
populations. This distribution arises from the training process: while the
pattern $992_{10}=1111100000_{2}$ trained the network to detect a $1$ in
position $5$, the pattern $960_{10}=1111000000_{2}$ trained the network to
detect a $0$ in the same position.

This means that neither of the neurons with ID $5$ contributes to the
generation of the output spike. In the testing network (fig.5) both the
inhibitory and excitatory synapses from both injector populations have the
same weight:

$I_{5}=W_{\text{exc5}|\text{Inj0}}-W_{\text{inh5}|\text{Inj0}}+\\\
W_{\text{exc5}|\text{Inj1}}-W_{\text{inh5}|\text{Inj1}}=$ (1)

$=\underbrace{0.367-0.367}_{\text{\normalsize\begin{tabular}[x]{@{}c@{}}for a
``0''\\\
spike\end{tabular}}}+\underbrace{0.367-0.367}_{\text{\normalsize\begin{tabular}[x]{@{}c@{}}for
a ``1''\\\ spike\end{tabular}}}=0$ (2)

Therefore, the input current to the output neuron contributed by neuron $5$ in
either spike injector population is null. The homeostasis process takes this
into account by increasing the overall value of the other contributing neurons
by the same amount of the missing synapses. In fact, the homeostatic parameter
in this instance is

$2.32647\approx\frac{4.18817}{2}\times\frac{10}{9}$ (3)

Where the value $4.18817$ is the homeostatic value from the single pattern
experiment, and $\nicefrac{{10}}{{9}}$ represents the fact that one of the
synapses is not contributing to the identification, and therefore all the
other synapses need to be stronger.

In these conditions, it is possible to evaluate the network performance
metrics in the detection of the patterns by testing all the possible
combinations. In this case the network positively identifies only the two
trained patterns ($992_{10}=1111100000_{2}$ and $960_{10}=1111000000_{2}$)
behaving as the perfect classifier.

As we test the network with patterns increasingly divergent from the original
$992_{10}=1111100000_{2}$ pattern, several general trends become evident:

  * •

The synapses related to the bits that are different among the two patterns
have weights evenly distributed between the two injector populations. In this
way, the output neuron does not depend on these inputs, which can be
considered “don’t care” synapses or bits.

  * •

The homeostatic factor increases with the number of “don’t care” bits, to
account for the fewer synapses that contribute to the detection.

  * •

The network’s performance metrics demonstrate a noticeable deterioration as
the number of dissimilarities between the patterns learned by the network
increases. This degradation in performance underscores the sensitivity of the
network to discrepancies among the learned patterns.

  * •

The deterioration in the performance of the pattern recognition task is
related to the Hamming distance (Hamming, 1950; Tomlinson et al., 2017)
between the two learned patterns: the number of “don’t care” bits in the
network and therefore the number of patterns that the network is able to
identify.

These results can be seen in table 6. The position of the “don’t care” bits or
synapses in the pattern is irrelevant for the purpose of this task. This can
be seen in the cases where the Hamming distance is 1 to 5: even though the
patterns to learn are different, the classification metrics are equal and
dependent only on the Hamming distance between them.

A special mention should be made for the case of two patterns that are
completely opposite. In our case $992_{10}=1111100000_{2}$ and
$31_{10}=0000011111_{2}$ have a Hamming distance of $10$. In this scenario,
none of the synapses in the network would contribute to the identification of
the pattern, and therefore, the output neuron would never spike. Consequently,
the homeostatic factor becomes infinite ($+\infty$). As a result, all the
classification metrics become incalculable since no output spike is generated
under any circumstances.

###  3.3 Multiple pattern training

| Population “0”
---|---
Neuron ID | 10 | 9 | 8 | 7 | 6 | 5 | 4 | 3 | 2 | 1
| Code Word “$0$”
---
Unit Synaptic Contributions
1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1
| Code word “$1$”
---
Unit Synaptic Contribution
1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 0
| Code word “$2$”
---
Unit Synaptic Contribution
1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 0 | 1
Final Unit Synaptic Weights | 3 | 3 | 3 | 3 | 3 | 3 | 3 | 3 | 2 | 2
| Population “1”
Neuron ID | 10 | 9 | 8 | 7 | 6 | 5 | 4 | 3 | 2 | 1
| Code Word “$0$”
---
Unit Synaptic Contributions
0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0
| Code word “$1$”
---
Unit Synaptic Contribution
0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1
| Code word “$2$”
---
Unit Synaptic Contribution
0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0
Final Unit Synaptic Weights | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 1

Table 7: Trained unit synaptic weights for the three code words “$0$”, “$1$”
and “$2$”

| Code
---
Word
| Synaptic
---
Contribution
| Unit
---
Weight
“$0$” |  | ${\color[rgb]{0,0,1}3}+{\color[rgb]{0,0,1}3}+{\color[rgb]{0,0,1}3}+{\color[rgb]{0,0,1}3}+{\color[rgb]{0,0,1}3}+{\color[rgb]{0,0,1}3}+{\color[rgb]{0,0,1}3}+{\color[rgb]{0,0,1}3}+{\color[rgb]{0,0,1}2}+{\color[rgb]{0,0,1}2}-$
---
$({\color[rgb]{1,0,0}0}+{\color[rgb]{1,0,0}0}+{\color[rgb]{1,0,0}0}+{\color[rgb]{1,0,0}0}+{\color[rgb]{1,0,0}0}+{\color[rgb]{1,0,0}0}+{\color[rgb]{1,0,0}0}+{\color[rgb]{1,0,0}0}+{\color[rgb]{1,0,0}1}+{\color[rgb]{1,0,0}1})$
$26$
“$1$” |  | ${\color[rgb]{0,0,1}3}+{\color[rgb]{0,0,1}3}+{\color[rgb]{0,0,1}3}+{\color[rgb]{0,0,1}3}+{\color[rgb]{0,0,1}3}+{\color[rgb]{0,0,1}3}+{\color[rgb]{0,0,1}3}+{\color[rgb]{0,0,1}3}+{\color[rgb]{0,0,1}2}+{\color[rgb]{1,0,0}1}-$
---
$({\color[rgb]{1,0,0}0}+{\color[rgb]{1,0,0}0}+{\color[rgb]{1,0,0}0}+{\color[rgb]{1,0,0}0}+{\color[rgb]{1,0,0}0}+{\color[rgb]{1,0,0}0}+{\color[rgb]{1,0,0}0}+{\color[rgb]{1,0,0}0}+{\color[rgb]{1,0,0}1}+{\color[rgb]{0,0,1}2})$
$24$
“$2$” |  | ${\color[rgb]{0,0,1}3}+{\color[rgb]{0,0,1}3}+{\color[rgb]{0,0,1}3}+{\color[rgb]{0,0,1}3}+{\color[rgb]{0,0,1}3}+{\color[rgb]{0,0,1}3}+{\color[rgb]{0,0,1}3}+{\color[rgb]{0,0,1}3}+{\color[rgb]{1,0,0}1}+{\color[rgb]{0,0,1}2}-$
---
$({\color[rgb]{1,0,0}0}+{\color[rgb]{1,0,0}0}+{\color[rgb]{1,0,0}0}+{\color[rgb]{1,0,0}0}+{\color[rgb]{1,0,0}0}+{\color[rgb]{1,0,0}0}+{\color[rgb]{1,0,0}0}+{\color[rgb]{1,0,0}0}+{\color[rgb]{0,0,1}2}+{\color[rgb]{1,0,0}1})$
$24$

Table 8: Example of synaptic unit contributions for the three code words
“$0$”, “$1$” and “$2$”, colored numbers indicate the source of the
corresponding weight from table 7

CW1 | CW2 | CW3 |  | HD
---
(1,2)
| HD
---
(1,3)
| HD
---
(2,3)
| Code Word 1
---
Unit Weight
| Code Word 2
---
Unit Weight
| Code Word 3
---
Unit Weight
| Homeostatic
---
factor
0 | 1 | 2 | 1 | 1 | 2 | 26 | 24 | 24 | 1.74513
0 | 1 | 6 | 1 | 2 | 3 | 24 | 22 | 20 | 2.09442
0 | 1 | 14 | 1 | 3 | 4 | 22 | 20 | 16 | 2.61791
0 | 1 | 30 | 1 | 4 | 5 | 20 | 18 | 12 | 3.49203
0 | 1 | 62 | 1 | 5 | 6 | 18 | 16 | 8 | 5.23937
0 | 1 | 126 | 1 | 6 | 7 | 16 | 14 | 4 | 10.47873
0 | 1 | 254 | 1 | 7 | 8 | 14 | 12 | 0 | 3.4907
0 | 1 | 510 | 1 | 8 | 9 | 12 | 10 | -4 | 4.19016
0 | 1 | 1022 | 1 | 9 | 10 | 10 | 8 | -8 | 5.23804
0 | 3 | 5 | 2 | 2 | 2 | 22 | 22 | 22 | 1.90382
0 | 3 | 12 | 2 | 2 | 4 | 22 | 18 | 18 | 2.32669
0 | 3 | 13 | 2 | 3 | 3 | 20 | 20 | 18 | 2.32713
0 | 3 | 28 | 2 | 3 | 5 | 20 | 16 | 14 | 2.99203
0 | 3 | 29 | 2 | 4 | 4 | 18 | 18 | 14 | 2.99203
0 | 3 | 60 | 2 | 4 | 6 | 18 | 14 | 10 | 4.18883
0 | 3 | 61 | 2 | 5 | 5 | 16 | 16 | 10 | 4.18618
0 | 3 | 124 | 2 | 5 | 7 | 16 | 12 | 6 | 6.98405
0 | 3 | 125 | 2 | 6 | 6 | 14 | 14 | 6 | 6.98405
0 | 3 | 252 | 2 | 6 | 8 | 14 | 10 | 2 | 21
0 | 3 | 253 | 2 | 7 | 7 | 12 | 12 | 2 | 21
0 | 3 | 508 | 2 | 7 | 9 | 12 | 8 | -2 | 5.23671
0 | 3 | 509 | 2 | 8 | 8 | 10 | 10 | -2 | 4.18883
0 | 3 | 1020 | 2 | 8 | 10 | 10 | 6 | -6 | 6.98139
0 | 3 | 1021 | 2 | 9 | 9 | 8 | 8 | -6 | 5.23671
0 | 7 | 25 | 3 | 3 | 4 | 18 | 16 | 16 | 2.61791
0 | 7 | 56 | 3 | 3 | 6 | 18 | 12 | 12 | 3.49025
0 | 7 | 57 | 3 | 4 | 5 | 16 | 14 | 12 | 3.49025
0 | 7 | 120 | 3 | 4 | 7 | 16 | 10 | 8 | 5.23582
0 | 7 | 121 | 3 | 5 | 6 | 14 | 12 | 8 | 5.23582
0 | 7 | 248 | 3 | 5 | 8 | 14 | 8 | 4 | 10.47873
0 | 7 | 249 | 3 | 6 | 7 | 12 | 10 | 4 | 10.47873
0 | 7 | 504 | 3 | 6 | 9 | 12 | 6 | 0 | 6.98139
0 | 7 | 505 | 3 | 7 | 8 | 10 | 8 | 0 | 5.23671
0 | 7 | 1016 | 3 | 7 | 10 | 10 | 4 | -4 | 10.47607
0 | 7 | 1017 | 3 | 8 | 9 | 8 | 6 | -4 | 6.98139
0 | 15 | 51 | 4 | 4 | 4 | 14 | 14 | 14 | 2.99203
0 | 15 | 113 | 4 | 4 | 6 | 14 | 10 | 10 | 4.18972
0 | 15 | 115 | 4 | 5 | 5 | 12 | 12 | 10 | 4.18972
0 | 15 | 240 | 4 | 4 | 8 | 14 | 6 | 6 | 6.9805
0 | 15 | 241 | 4 | 5 | 7 | 12 | 8 | 6 | 6.9805
0 | 15 | 243 | 4 | 6 | 6 | 10 | 10 | 6 | 6.9805
0 | 15 | 496 | 4 | 5 | 9 | 12 | 4 | 2 | 20.95745
0 | 15 | 497 | 4 | 6 | 8 | 10 | 6 | 2 | 20.95745
0 | 15 | 499 | 4 | 7 | 7 | 8 | 8 | 2 | 20.95745
0 | 15 | 1008 | 4 | 6 | 10 | 10 | 2 | -2 | 20.95213
0 | 15 | 1009 | 4 | 7 | 9 | 8 | 4 | -2 | 10.47341
0 | 15 | 1011 | 4 | 8 | 8 | 6 | 6 | -2 | 6.98139
0 | 31 | 227 | 5 | 5 | 6 | 10 | 8 | 8 | 5.23582
0 | 31 | 481 | 5 | 5 | 8 | 10 | 4 | 4 | 10.47164
0 | 31 | 483 | 5 | 6 | 7 | 8 | 6 | 4 | 10.47164
0 | 31 | 992 | 5 | 5 | 10 | 10 | 0 | 0 | 4.19016
0 | 31 | 993 | 5 | 6 | 9 | 8 | 2 | 0 | 20.94681
0 | 31 | 995 | 5 | 7 | 8 | 6 | 4 | 0 | 10.47341
0 | 63 | 455 | 6 | 6 | 6 | 6 | 6 | 6 | 6.98139
0 | 63 | 963 | 6 | 6 | 8 | 6 | 2 | 2 | 20.94681
0 | 63 | 967 | 6 | 7 | 7 | 4 | 4 | 2 | 20.94681

Table 9: Combinations of code words (CW1, CW2 and CW3) trained on the network,
their Hamming distance HD(x,y), the synaptic unit weight, computed as
described in the text, and the resulting homeostatic factor.

CW1 | CW2 | CW3 |  | True
---
Positives
| True
---
Negatives
| False
---
Positives
| False
---
Negatives
Accuracy | Precision |  | Negative
---
Prediction
Sensitivity | Specificity
0 | 1 | 2 | 3 | 1021 | 0 | 0 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000
0 | 1 | 6 | 3 | 1017 | 4 | 0 | 0.996 | 0.429 | 1.000 | 1.000 | 0.996
0 | 1 | 14 | 3 | 1003 | 18 | 0 | 0.982 | 0.143 | 1.000 | 1.000 | 0.982
0 | 1 | 30 | 3 | 963 | 58 | 0 | 0.943 | 0.049 | 1.000 | 1.000 | 0.943
0 | 1 | 62 | 3 | 873 | 148 | 0 | 0.855 | 0.020 | 1.000 | 1.000 | 0.855
0 | 1 | 126 | 3 | 705 | 316 | 0 | 0.691 | 0.009 | 1.000 | 1.000 | 0.690
0 | 1 | 254 | 2 | 1014 | 7 | 1 | 0.992 | 0.222 | 0.999 | 0.667 | 0.993
0 | 1 | 510 | 2 | 1013 | 8 | 1 | 0.991 | 0.200 | 0.999 | 0.667 | 0.992
0 | 1 | 1022 | 2 | 1012 | 9 | 1 | 0.990 | 0.182 | 0.999 | 0.667 | 0.991
0 | 3 | 5 | 3 | 1020 | 1 | 0 | 0.999 | 0.750 | 1.000 | 1.000 | 0.999
0 | 3 | 12 | 3 | 1013 | 8 | 0 | 0.992 | 0.273 | 1.000 | 1.000 | 0.992
0 | 3 | 13 | 3 | 1013 | 8 | 0 | 0.992 | 0.273 | 1.000 | 1.000 | 0.992
0 | 3 | 28 | 3 | 998 | 23 | 0 | 0.978 | 0.115 | 1.000 | 1.000 | 0.977
0 | 3 | 29 | 3 | 998 | 23 | 0 | 0.978 | 0.115 | 1.000 | 1.000 | 0.977
0 | 3 | 60 | 3 | 963 | 58 | 0 | 0.943 | 0.049 | 1.000 | 1.000 | 0.943
0 | 3 | 61 | 3 | 963 | 58 | 0 | 0.943 | 0.049 | 1.000 | 1.000 | 0.943
0 | 3 | 124 | 3 | 817 | 204 | 0 | 0.801 | 0.014 | 1.000 | 1.000 | 0.800
0 | 3 | 125 | 3 | 817 | 204 | 0 | 0.801 | 0.014 | 1.000 | 1.000 | 0.800
0 | 3 | 252 | 3 | 591 | 430 | 0 | 0.580 | 0.007 | 1.000 | 1.000 | 0.579
0 | 3 | 253 | 3 | 591 | 430 | 0 | 0.580 | 0.007 | 1.000 | 1.000 | 0.579
0 | 3 | 508 | 2 | 977 | 44 | 1 | 0.956 | 0.043 | 0.999 | 0.667 | 0.957
0 | 3 | 509 | 2 | 1013 | 8 | 1 | 0.991 | 0.200 | 0.999 | 0.667 | 0.992
0 | 3 | 1020 | 2 | 967 | 54 | 1 | 0.946 | 0.036 | 0.999 | 0.667 | 0.947
0 | 3 | 1021 | 2 | 1012 | 9 | 1 | 0.990 | 0.182 | 0.999 | 0.667 | 0.991
0 | 7 | 25 | 3 | 1008 | 13 | 0 | 0.987 | 0.188 | 1.000 | 1.000 | 0.987
0 | 7 | 56 | 3 | 982 | 39 | 0 | 0.962 | 0.071 | 1.000 | 1.000 | 0.962
0 | 7 | 57 | 3 | 982 | 39 | 0 | 0.962 | 0.071 | 1.000 | 1.000 | 0.962
0 | 7 | 120 | 3 | 922 | 99 | 0 | 0.903 | 0.029 | 1.000 | 1.000 | 0.903
0 | 7 | 121 | 3 | 922 | 99 | 0 | 0.903 | 0.029 | 1.000 | 1.000 | 0.903
0 | 7 | 248 | 3 | 787 | 234 | 0 | 0.771 | 0.013 | 1.000 | 1.000 | 0.771
0 | 7 | 249 | 3 | 787 | 234 | 0 | 0.771 | 0.013 | 1.000 | 1.000 | 0.771
0 | 7 | 504 | 2 | 892 | 129 | 1 | 0.873 | 0.015 | 0.999 | 0.667 | 0.874
0 | 7 | 505 | 2 | 977 | 44 | 1 | 0.956 | 0.043 | 0.999 | 0.667 | 0.957
0 | 7 | 1016 | 2 | 847 | 174 | 1 | 0.829 | 0.011 | 0.999 | 0.667 | 0.830
0 | 7 | 1017 | 2 | 967 | 54 | 1 | 0.946 | 0.036 | 0.999 | 0.667 | 0.947
0 | 15 | 51 | 3 | 1002 | 19 | 0 | 0.981 | 0.136 | 1.000 | 1.000 | 0.981
0 | 15 | 113 | 3 | 957 | 64 | 0 | 0.938 | 0.045 | 1.000 | 1.000 | 0.937
0 | 15 | 115 | 3 | 957 | 64 | 0 | 0.938 | 0.045 | 1.000 | 1.000 | 0.937
0 | 15 | 240 | 3 | 859 | 162 | 0 | 0.842 | 0.018 | 1.000 | 1.000 | 0.841
0 | 15 | 241 | 3 | 859 | 162 | 0 | 0.842 | 0.018 | 1.000 | 1.000 | 0.841
0 | 15 | 243 | 3 | 859 | 162 | 0 | 0.842 | 0.018 | 1.000 | 1.000 | 0.841
0 | 15 | 496 | 3 | 632 | 389 | 0 | 0.620 | 0.008 | 1.000 | 1.000 | 0.619
0 | 15 | 497 | 3 | 632 | 389 | 0 | 0.620 | 0.008 | 1.000 | 1.000 | 0.619
0 | 15 | 499 | 3 | 632 | 389 | 0 | 0.620 | 0.008 | 1.000 | 1.000 | 0.619
0 | 15 | 1008 | 2 | 637 | 384 | 1 | 0.624 | 0.005 | 0.998 | 0.667 | 0.624
0 | 15 | 1009 | 2 | 847 | 174 | 1 | 0.829 | 0.011 | 0.999 | 0.667 | 0.830
0 | 15 | 1011 | 2 | 967 | 54 | 1 | 0.946 | 0.036 | 0.999 | 0.667 | 0.947
0 | 31 | 227 | 3 | 931 | 90 | 0 | 0.912 | 0.032 | 1.000 | 1.000 | 0.912
0 | 31 | 481 | 3 | 767 | 254 | 0 | 0.752 | 0.012 | 1.000 | 1.000 | 0.751
0 | 31 | 483 | 3 | 767 | 254 | 0 | 0.752 | 0.012 | 1.000 | 1.000 | 0.751
0 | 31 | 992 | 1 | 1021 | 0 | 2 | 0.998 | 1.000 | 0.998 | 0.333 | 1.000
0 | 31 | 993 | 2 | 637 | 384 | 1 | 0.624 | 0.005 | 0.998 | 0.667 | 0.624
0 | 31 | 995 | 2 | 847 | 174 | 1 | 0.829 | 0.011 | 0.999 | 0.667 | 0.830
0 | 63 | 455 | 3 | 893 | 128 | 0 | 0.875 | 0.023 | 1.000 | 1.000 | 0.875
0 | 63 | 963 | 3 | 638 | 383 | 0 | 0.626 | 0.008 | 1.000 | 1.000 | 0.625
0 | 63 | 967 | 3 | 638 | 383 | 0 | 0.626 | 0.008 | 1.000 | 1.000 | 0.625

Table 10: Combinations of code words (CW1, CW2 and CW3) trained on the
network, and the corresponding test classification metrics obtained from
simulations.

Finally, the network was tested with three code words. As discussed in the
previous case, the accuracy of the network relies on the similarities between
the various code words, and in particular on the Hamming distance across all
code words trained.

To ensure the test was conducted with the largest possible set comprising all
10-bit code words, the Hamming distance between all possible combinations of
10-bit numbers was computed, and only the first occurrence of the code words
for each Hamming distance was recorded, where the code words were all
different. The resulting combinations of code words is described in table 9.

The network was trained using the same protocol as in previous experiments,
this time using three code words. The weights obtained in this way are then
re-scaled simulating an homeostatic process so that, where possible, the
network would spike for all three trained patterns. However, as shown in table
9, not in all cases this is possible: with specific combinations of code words
it is not possible to have the positive identification of one or even two code
words.

To explain this behaviour we can start considering each training iteration as
providing a synaptic weight unit contribution to the “0” population or to the
“1” population. When the code word is then re-applied during homeostasis,
these unit weights contribute to the final weight applied to the output
neuron. If the sum of excitatory and inhibitory unit weights is positive, then
the homeostasis finds the smallest factor for which the network fires for all
the code words. In alternative, one or two code words are “discarded” during
this process and will appear in the “False Negative” count.

For example, in the case of code words “$0$”, “$1$” and “$2$”, the final
weight unit contribution to the output neuron is described in table 7:

As for the code word $0$ all the neurons in population “0” are spiking, these
neuron contribute to the output neuron by exciting it through the excitatory
“0” synapses and inhibit it through the inhibitory “1” synapses (see fig.5).
In this case the final contribution is positive and includes $26$ unit
weights. If we repeat this process for all the code words in the example, the
synaptic weights contributions are as described in table 8, where colored
numbers indicate the source of the corresponding weight from table 7.

As all the contributions are positive, in this case applying the appropriate
homeostatic factor to the weights will lead to the output neuron firing (at
least) for the trained code words. Considering the STDP parameters applied to
the network, one synaptic weight unit is equal to $0.3671875$ and the
homeostatic factor required for the network is estimated in this case equal to
$1.74513$.

The table describing all the combinations of code words, their Hamming
distance, the synaptic unit weights and homeostasis factors computed for this
case is presented in table 9.

Table 10 introduces the performance metrics of the test and validation
network, and clearly illustrates two trends: maintaining two code words
constant while progressively increasing the Hamming distance of the remaining
one from the others results in the deterioration of synaptic unit weights.
These weights reach zero or become negative, rendering one or more code
word(s) no longer positively identifiable by the network.

Simultaneously, as the synaptic unit weight decreases, the homeostatic factor
increases to compensate for the limited efficacy of the incoming excitation.
This holds true until the network is no longer able to detect one of the code
words, which is then automatically excluded from the identification task
during the search for a valid homeostatic factor: indeed the neural network
cannot emit a spike in case the output neuron excitation results in a “$0$” or
even a negative number. The unit weights for which one or two code words are
no longer identifiable are highlighted in red in table 9.

(a) True positive + False Positives

(b) False negatives

(c) True negatives

(d) Homeostatic factor

(e) Accuracy

(f) Precision

(g) Negative prediction

(h) Sensitivity

(i) Specificity

Figure 6: Statistical classification metrics for the multiple code words
training. The axes represent the Hamming distance between the various code
words as indicated in each graph.

Comparing table 9 with table 10, it is possible to notice that the false
negatives appear in correspondence to the $0$ or negative unit weights. These
classification metrics show that the accuracy is linked to the Hamming
distance between code words. However, when one code word is dropped from
identification, the negative prediction and specificity parameters receive
lower values, but the overall accuracy improves, as the number of false
positive identifications drastically reduces. However, the precision parameter
decreases rapidly as the number of false positives increases.

The classification metrics presented in table 10 are also displayed
graphically in fig.6. In these graphs it is possible to notice how the closer
the experiments are to the bottom left corner, the better the classification
metrics that represent the outcome. Indeed, the bottom left corner represents
experiments using three code words whose Hamming distance among them is
minimum. In these graphs the three axis HD(x,y) indicate the hamming distance
between the “x” and “y” code words.

In particular it is possible to notice also how there is an inverse relation
between the homeostatic factor and the overall accuracy of the identification
task. As the distance between code words increases, the number of synapses
with overall contribution “0” (“don’t care” synapses) or small (“care little”
synapses) increases, and these require a higher homeostatic factor to allow
the remaining synapses to trigger an output spike. However, this also causes
the number of false positive identifications to increase, which in turn
reduces the overall accuracy of the identification task of the network.

##  4 Conclusions

This paper presented a method for training a spiking neural network to
identify spatial patterns. Validation results show that a spiking network
trained this way is able to successfully complete this task. The testing
involved training on a single pattern, two patterns and three patterns,
covering all meaningful combinations of code words. In these experiments, it
became evident that specific parameter values for the STDP learning rule hold
little significance. This is because the homeostatic process plays a crucial
role in re-scaling synaptic weights to elicit spikes from the output neuron
under specific conditions.

The results of the experiments show that a network trained with a single
pattern acts as a perfect classifier, which only identifies spatial patterns
that are identical to the trained one. The ability of the network to select a
specific pattern comes from the structure of the network: while excitatory
synapses trigger the output neuron, inhibitory connections perform the
selection of the pattern during the testing and validation phase.

When two or more patterns are trained on a single network, the accuracy of the
identification task depends only on the Hamming distance between the code
words imprinted in the network.

From the analysis of the classification metrics, it is possible to see that
the negative prediction in the case of one or two trained patterns is always
$1$, which highlights how the absence of an output spike always correctly
identifies that the trained pattern is not present.

On the other hand, positive identification spikes depend on the number of
“don’t care” and “care little” synapses or bits imprinted in the network. As
the number of “don’t care” or “care little” synapses increases, the number of
false positives increases, and this reduces the overall accuracy of the
identification task.

In the multiple code word experiments, the classification metrics extracted
show that the accuracy is linked to the Hamming distance between code words.
However, when one code word is dropped from identification because its Hamming
distance is too high, the negative prediction and specificity parameters
receive lower values, but the overall accuracy improves because the number of
false positive identifications drastically reduces. Overall, precision and
sensitivity decrease rapidly as the number of false positives increases when
the Hamming distance between code words increases.

This article has presented a method to train a spiking neural network to
detect spatial patterns which do not have a temporal component. These kind of
patterns may be found in the analysis of computer network traffic packets,
where each single packet does not have a temporal component, but holds enough
information for a neural network to determine the type of traffic that is
carries (e.g.: Rasteh et al., 2022; Aceto et al., 2021). An additional example
of spatial pattern is represented by static images which can be encoded into
spikes to allow a spiking neural network to perform pattern matching tasks on
trained patterns (e.g.: Davies et al., 2021).

In these applications, it is conceivable to adjust the homeostatic factor,
calculated during the training phase, to also serve as a similarity factor in
matching the input pattern with the trained pattern. Even in the case of a
single trained pattern, increasing the homeostatic factor appropriately may
enable the network to identify patterns with a certain degree of similarity,
rather than requiring an exact match between the input and the trained
pattern.

## Acknowledgement

This work has been supported by the Department of Computing and Mathematics at
the Manchester Metropolitan University and authorised through ethical review
59273. Development of SpiNNaker software was supported by the EU ICT Flagship
Human Brain Project which has received funding from the European Union’s FP7
programme under Grant Agreement no. 604102, and from the European Union’s
Horizon 2020 research and innovation programme under FPA No 650003
(HBP-785907). Prof. Di Nuovo acknowledges the support of the UK Engineering
and Physical Sciences Research Council (grant number EP/X018733/1 for the
project ALDENS), and Innovate UK (grant number 10089807 for the Horizon Europe
project PRIMI Grant agreement n. 101120727). The authors extend their
gratitude to Prof. Steve Furber and all members, current and past, of the APT
group at the University of Manchester for their invaluable support.

## Conflicts of interest

The authors declare that they have no known competing financial interests or
personal relationships that could have appeared to influence the work reported
in this paper.

## Declaration of Generative AI and AI-assisted technologies in the writing
process

During the preparation of this work the authors used ChatGPT in order to
improve readability and language. After using this tool, the authors reviewed
and edited the content as needed and take full responsibility for the content
of the publication.

## References

  * Aceto et al. (2021) Aceto, G., Ciuonzo, D., Montieri, A., Pescapé, A., 2021.  DISTILLER: Encrypted traffic classification via multimodal multitask deep learning.  Journal of Network and Computer Applications 183-184, 102985.  URL: https://linkinghub.elsevier.com/retrieve/pii/S1084804521000126, doi:10.1016/j.jnca.2021.102985.
  * van Albada et al. (2018) van Albada, S.J., Rowley, A.G., Senk, J., Hopkins, M., Schmidt, M., Stokes, A.B., Lester, D.R., Diesmann, M., Furber, S.B., 2018.  Performance Comparison of the Digital Neuromorphic Hardware SpiNNaker and the Neural Network Simulation Software NEST for a Full-Scale Cortical Microcircuit Model.  Frontiers in Neuroscience 12.  URL: https://www.frontiersin.org/article/10.3389/fnins.2018.00291/full, doi:10.3389/fnins.2018.00291.
  * Auge et al. (2021) Auge, D., Hille, J., Mueller, E., Knoll, A., 2021.  A Survey of Encoding Techniques for Signal Processing in Spiking Neural Networks.  Neural Processing Letters 53, 4693–4710.  URL: https://link.springer.com/article/10.1007/s11063-021-10562-2, doi:10.1007/S11063-021-10562-2/TABLES/1.
  * Bi and Poo (2001) Bi, G.q., Poo, M.m., 2001.  Synaptic Modification by Correlated Activity: Hebb’s Postulate Revisited.  Annual Review of Neuroscience 24, 139–166.  URL: http://www.annualreviews.org/doi/10.1146/annurev.neuro.24.1.139, doi:10.1146/annurev.neuro.24.1.139.
  * Dahl et al. (2012) Dahl, G.E., Dong Yu, Li Deng, Acero, A., 2012.  Context-Dependent Pre-Trained Deep Neural Networks for Large-Vocabulary Speech Recognition.  IEEE Transactions on Audio, Speech, and Language Processing 20, 30–42.  URL: http://ieeexplore.ieee.org/document/5740583/, doi:10.1109/TASL.2011.2134090.
  * Davies (2012) Davies, S., 2012.  Learning in Spiking Neural Networks.  Ph.D. thesis. The University of Manchester. Kilburn Building, Oxford road, M13 9PL.
  * Davies et al. (2012) Davies, S., Galluppi, F., Rast, A., Furber, S., 2012.  A forecast-based STDP rule suitable for neuromorphic implementation.  Neural Networks 32, 3–14.  URL: https://linkinghub.elsevier.com/retrieve/pii/S0893608012000470, doi:10.1016/j.neunet.2012.02.018.
  * Davies et al. (2021) Davies, S., Lucas, A., Ricolfe-Viala, C., Di Nuovo, A., 2021.  A Database for Learning Numbers by Visual Finger Recognition in Developmental Neuro-Robotics.  Frontiers in Neurorobotics 15, 12.  URL: https://www.frontiersin.org/articles/10.3389/fnbot.2021.619504/full, doi:10.3389/fnbot.2021.619504.
  * Davison et al. (2009) Davison, A.P., Brüderle, D., Eppler, J., Kremkow, J., Muller, E., Pecevski, D., Perrinet, L., Yger, P., 2009.  PyNN: a common interface for neuronal network simulators.  Frontiers in Neuroinformatics 2.  URL: http://journal.frontiersin.org/article/10.3389/neuro.11.011.2008/abstract, doi:10.3389/neuro.11.011.2008.
  * Dayan and Abbott (2001) Dayan, P., Abbott, L.F., 2001.  Theoretical Neuroscience: Computational and Mathematical Modeling of Neural Systems.  1st ed., The MIT Press.  URL: https://mitpress.mit.edu/9780262041997/theoretical-neuroscience/.
  * Funahashi and Nakamura (1993) Funahashi, K.i., Nakamura, Y., 1993.  Approximation of dynamical systems by continuous time recurrent neural networks.  Neural Networks 6, 801–806.  URL: https://linkinghub.elsevier.com/retrieve/pii/S089360800580125X, doi:10.1016/S0893-6080(05)80125-X.
  * Furber and Bogdan (2020) Furber, S., Bogdan, P. (Eds.), 2020.  SpiNNaker: A Spiking Neural Network Architecture.  now publishers, Inc., Boston-Delft.  URL: https://nowpublishers.com/article/BookDetails/9781680836523, doi:10.1561/9781680836523.
  * Gerstner et al. (2014) Gerstner, W., Kistler, W.M., Naud, R., Paninski, L., 2014.  Neuronal Dynamics.  Cambridge University Press.  URL: https://www.cambridge.org/core/product/identifier/9781107447615/type/book, doi:10.1017/CBO9781107447615.
  * Guyonneau et al. (2005) Guyonneau, R., VanRullen, R., Thorpe, S.J., 2005.  Neurons Tune to the Earliest Spikes Through STDP.  Neural Computation 17, 859–879.  URL: https://direct.mit.edu/neco/article/17/4/859-879/6942, doi:10.1162/0899766053429390.
  * Hamming (1950) Hamming, R.W., 1950.  Error Detecting and Error Correcting Codes.  Bell System Technical Journal 29, 147–160.  doi:10.1002/J.1538-7305.1950.TB00463.X.
  * Haykin (1999) Haykin, S.S., 1999.  Neural networks: a comprehensive foundation.  2 ed., Prentice Hall.  URL: https://cdn.preterhuman.net/texts/science_and_technology/artificial_intelligence/NeuralNetworks-AComprehensiveFoundation-SimonHaykin.pdf.
  * Hebb (1949) Hebb, D.O., 1949.  The Organization of Behavior: A Neuropsychological Theory.  Wiley.
  * Hodgkin and Huxley (1952) Hodgkin, A.L., Huxley, A.F., 1952.  A quantitative description of membrane current and its application to conduction and excitation in nerve.  The Journal of physiology 117, 500–544.
  * Izhikevich (2003) Izhikevich, E., 2003.  Simple model of spiking neurons.  IEEE Transactions on Neural Networks 14, 1569–1572.  URL: http://ieeexplore.ieee.org/document/1257420/, doi:10.1109/TNN.2003.820440.
  * Izhikevich (2004) Izhikevich, E., 2004.  Which Model to Use for Cortical Spiking Neurons?  IEEE Transactions on Neural Networks 15, 1063–1070.  URL: http://ieeexplore.ieee.org/document/1333071/, doi:10.1109/TNN.2004.832719.
  * Jin et al. (2010) Jin, X., Rast, A., Galluppi, F., Davies, S., Furber, S., 2010.  Implementing spike-timing-dependent plasticity on SpiNNaker neuromorphic hardware, in: The 2010 International Joint Conference on Neural Networks (IJCNN), Sch. of Comput. Sci., Univ. of Manchester, Manchester, UK. IEEE. pp. 1–8.  URL: http://ieeexplore.ieee.org/document/5596372/, doi:10.1109/IJCNN.2010.5596372.
  * Kasabov (2019) Kasabov, N.K., 2019.  Time-Space, Spiking Neural Networks and Brain-Inspired Artificial Intelligence. volume 7 of Springer Series on Bio- and Neurosystems.  Springer Berlin Heidelberg, Berlin, Heidelberg.  URL: http://link.springer.com/10.1007/978-3-662-57715-8, doi:10.1007/978-3-662-57715-8.
  * Lapicque (1907) Lapicque, L., 1907.  Recherches quantitatives sur l’excitation electrique des nerfs traitee comme une polarization.  Journal of Physiol Pathol Générale 9, 620–635.  URL: https://fr.wikisource.org/wiki/Recherches_quantitatives_sur_l%27excitation_%C3%A9lectrique_des_nerfs_trait%C3%A9e_comme_une_polarisation.
  * LeCun et al. (2015) LeCun, Y., Bengio, Y., Hinton, G., 2015.  Deep learning.  Nature 521, 436–44.  URL: http://www.ncbi.nlm.nih.gov/pubmed/26017442, doi:10.1038/nature14539.
  * Linnainmaa (1976) Linnainmaa, S., 1976.  Taylor expansion of the accumulated rounding error.  BIT 16, 146–160.  URL: https://link.springer.com/article/10.1007/BF01931367, doi:10.1007/BF01931367/METRICS.
  * Maass (1997) Maass, W., 1997.  Networks of spiking neurons: The third generation of neural network models.  Neural Networks 10, 1659–1671.  URL: https://linkinghub.elsevier.com/retrieve/pii/S0893608097000117, doi:10.1016/S0893-6080(97)00011-7.
  * Maffei and Fontanini (2009) Maffei, A., Fontanini, A., 2009.  Network homeostasis: a matter of coordination.  Current opinion in neurobiology 19, 168–73.  URL: http://www.ncbi.nlm.nih.gov/pubmed/19540746http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=PMC3427905, doi:10.1016/j.conb.2009.05.012.
  * Markram et al. (1997) Markram, H., Lübke, J., Frotscher, M., Sakmann, B., 1997.  Regulation of Synaptic Efficacy by Coincidence of Postsynaptic APs and EPSPs.  Science 275, 213–215.  URL: https://www.science.org/doi/10.1126/science.275.5297.213, doi:10.1126/science.275.5297.213.
  * Masquelier et al. (2008) Masquelier, T., Guyonneau, R., Thorpe, S.J., 2008.  Spike Timing Dependent Plasticity Finds the Start of Repeating Patterns in Continuous Spike Trains.  PLoS ONE 3, e1377.  URL: https://dx.plos.org/10.1371/journal.pone.0001377, doi:10.1371/journal.pone.0001377.
  * McCulloch and Pitts (1943) McCulloch, W.S., Pitts, W., 1943.  A logical calculus of the ideas immanent in nervous activity.  The Bulletin of Mathematical Biophysics 5, 115–133.  URL: http://link.springer.com/10.1007/BF02478259, doi:10.1007/BF02478259.
  * Mohamed et al. (2012) Mohamed, A.r., Dahl, G.E., Hinton, G., 2012.  Acoustic Modeling Using Deep Belief Networks.  IEEE Transactions on Audio, Speech, and Language Processing 20, 14–22.  URL: http://ieeexplore.ieee.org/document/5704567/, doi:10.1109/TASL.2011.2109382.
  * Nolfi et al. (1994) Nolfi, S., Parisi, D., Elman, J.L., 1994.  Learning and Evolution in Neural Networks.  Adaptive Behavior 3, 5–28.  URL: http://journals.sagepub.com/doi/10.1177/105971239400300102, doi:10.1177/105971239400300102.
  * Prieto et al. (2016) Prieto, A., Prieto, B., Ortigosa, E.M., Ros, E., Pelayo, F., Ortega, J., Rojas, I., 2016.  Neural networks: An overview of early research, current frameworks and new challenges.  Neurocomputing 214, 242–268.  doi:10.1016/J.NEUCOM.2016.06.014.
  * Rasteh et al. (2022) Rasteh, A., Delpech, F., Aguilar-Melchor, C., Zimmer, R., Shouraki, S.B., Masquelier, T., 2022.  Encrypted internet traffic classification using a supervised spiking neural network.  Neurocomputing 503, 272–282.  URL: https://linkinghub.elsevier.com/retrieve/pii/S0925231222007895, doi:10.1016/j.neucom.2022.06.055.
  * Rhodes et al. (2018) Rhodes, O., Bogdan, P.A., Brenninkmeijer, C., Davidson, S., Fellows, D., Gait, A., Lester, D.R., Mikaitis, M., Plana, L.A., Rowley, A.G.D., Stokes, A.B., Furber, S.B., 2018.  sPyNNaker: A Software Package for Running PyNN Simulations on SpiNNaker.  Frontiers in Neuroscience 12, 816.  URL: https://www.frontiersin.org/article/10.3389/fnins.2018.00816/full, doi:10.3389/fnins.2018.00816.
  * Rowley et al. (2019) Rowley, A.G.D., Brenninkmeijer, C., Davidson, S., Fellows, D., Gait, A., Lester, D.R., Plana, L.A., Rhodes, O., Stokes, A.B., Furber, S.B., 2019.  SpiNNTools: The Execution Engine for the SpiNNaker Platform.  Frontiers in Neuroscience 13.  URL: https://www.frontiersin.org/article/10.3389/fnins.2019.00231/full, doi:10.3389/fnins.2019.00231.
  * Schmidhuber (2015) Schmidhuber, J., 2015.  Deep Learning in neural networks: An overview.  doi:10.1016/j.neunet.2014.09.003.
  * Sermanet et al. (2013) Sermanet, P., Kavukcuoglu, K., Chintala, S., Lecun, Y., 2013.  Pedestrian Detection with Unsupervised Multi-stage Feature Learning, in: 2013 IEEE Conference on Computer Vision and Pattern Recognition, IEEE. pp. 3626–3633.  URL: http://ieeexplore.ieee.org/document/6619309/, doi:10.1109/CVPR.2013.465.
  * Sjöström and Gerstner (2010) Sjöström, J., Gerstner, W., 2010.  Spike-timing dependent plasticity.  Scholarpedia 5, 1362.
  * Tavanaei et al. (2019) Tavanaei, A., Ghodrati, M., Kheradpisheh, S.R., Masquelier, T., Maida, A., 2019.  Deep learning in spiking neural networks.  Neural Networks 111, 47–63.  URL: https://linkinghub.elsevier.com/retrieve/pii/S0893608018303332, doi:10.1016/j.neunet.2018.12.002.
  * Tierney (1986) Tierney, A.J., 1986.  The evolution of learned and innate behavior: Contributions from genetics and neurobiology to a theory of behavioral evolution.  Animal Learning & Behavior 14, 339–348.  URL: https://link.springer.com/article/10.3758/BF03200077, doi:10.3758/BF03200077/METRICS.
  * Tomlinson et al. (2017) Tomlinson, M., Tjhai, C.J., Ambroze, M.A., Ahmed, M., Jibril, M., 2017.  Error-Correction Coding and Decoding.  Springer International Publishing, Cham.  doi:10.1007/978-3-319-51103-0.
  * Tsodyks and Wu (2013) Tsodyks, M., Wu, S., 2013.  Short-term synaptic plasticity.  Scholarpedia 8, 3153.  URL: http://www.scholarpedia.org/article/Short-term_synaptic_plasticity, doi:10.4249/scholarpedia.3153.
  * Turrigiano (2011) Turrigiano, G., 2011.  Too Many Cooks? Intrinsic and Synaptic Homeostatic Mechanisms in Cortical Circuit Refinement.  Annual Review of Neuroscience 34, 89–103.  doi:10.1146/annurev-neuro-060909-153238.
  * Turrigiano (1999) Turrigiano, G.G., 1999.  Homeostatic plasticity in neuronal networks: the more things change, the more they stay the same.  Trends in Neurosciences 22, 221–227.
  * Yamashita and Tani (2008) Yamashita, Y., Tani, J., 2008.  Emergence of Functional Hierarchy in a Multiple Timescale Neural Network Model: A Humanoid Robot Experiment.  PLOS Computational Biology 4, e1000220.  URL: https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1000220, doi:10.1371/JOURNAL.PCBI.1000220.
  * Zecevic and Rakic (1991) Zecevic, N., Rakic, P., 1991.  Synaptogenesis in Monkey Somatosensory Cortex.  Cerebral Cortex 1, 510–523.  URL: https://academic.oup.com/cercor/article-lookup/doi/10.1093/cercor/1.6.510, doi:10.1093/cercor/1.6.510.
  * Zeraati et al. (2021) Zeraati, R., Priesemann, V., Levina, A., 2021.  Self-Organization Toward Criticality by Synaptic Plasticity.  Frontiers in Physics 9.  doi:10.3389/fphy.2021.619661.
  * Zucker and Regehr (2002) Zucker, R.S., Regehr, W.G., 2002.  Short-Term Synaptic Plasticity.  Annual Review of Physiology 64, 355–405.  URL: https://www.annualreviews.org/doi/10.1146/annurev.physiol.64.092501.114547, doi:10.1146/annurev.physiol.64.092501.114547.

a1.jpg Sergio Davies is a senior lecturer in the Department of Computing and
Mathematics at Manchester Metropolitan University. He received his Laurea
(equivalent to MSc Eng) in Telecommunication Engineering from the University
“Federico II” in Napoli, Italy in 2006. Following his Ph.D. in Computer
Science at the University of Manchester in 2012, Sergio continued his research
on spiking neural networks as a postdoc in the Human Brain Project until 2016.
Transitioning to industry, he took leadership roles in strategic advising as a
consultant, and then managed various research projects covering computer
hardware design, embedded systems architecture, software development and
system networking. In 2019, Sergio returned to academia, initially at
Sheffield Hallam University before joining Manchester Metropolitan University.
His current research focuses on practical application of spiking neural
networks in real-world scenarios.

a2.jpg Andrew Gait is a Research Software Engineer at the University of
Manchester within the Research IT department. Following his PhD completed at
the University of Leeds in 2007, he has worked in multiple different
departments and institutes across the University of Manchester on various
software projects and within multiple cross-disciplinary teams. This has
included work on a multi-physics software library, the design of software for
use in segmenting medical images, and the development and user support of
spiking neural network software as part of the Human Brain Project in the APT
group in Computer Science.

a3.jpg Andrew Rowley is a Senior Research Software Engineer at the University
of Manchester within the Research IT department. After completing his PhD in
Artificial Intelligence at the University of St. Andrews in 2004, he joined
the University of Manchester as a Research Software Engineer. There he worked
on various projects related to Access Grid video conferencing before becoming
a Senior Research Software Engineer for NaCTeM working on text mining
projects. He then joined the Human Brain Project team in Manchester and was
working on the SpiNNaker project designing, building and supporting the
sPyNNaker software since 2014.

a4.jpg Alessandro Di Nuovo is Professor of Machine Intelligence at Sheffield
Hallam University. He received the Laurea (MSc Eng) and the PhD in Informatics
Engineering from the University of Catania, Italy, in 2005 and 2009,
respectively. He is the leader of the Smart Interactive Technologies research
laboratory of the Department of Computing. He has published over 120 articles
in computational intelligence and its application to cognitive modelling,
human-robot interaction, computer-aided assessment of intellectual
disabilities, and embedded computer systems. Prof. Di Nuovo has an extensive
track record of leading interdisciplinary research and innovation in
fundamental and applied topics in AI and Robotics, for which he has received
several grants from prestigious funders (EPSRC, European Union) and companies.
Currently, Prof. Di Nuovo is editor-in-chief (topics AI in Robotics; Human
Robot/Machine Interaction) of the International Journal of Advanced Robotic
Systems (SAGE). He is serving as Associate Editor for IEEE Journal of
Translational Engineering in Health and Medicine.