Neuromorphic analog circuits for robust on-chip always-on learning in
  spiking neural networks
Mixed-signal neuromorphic systems represent a promising solution for solving
extreme-edge computing tasks without relying on external computing resources.
Their spiking neural network circuits are optimized for processing sensory data
on-line in continuous-time. However, their low precision and high variability
can severely limit their performance. To address this issue and improve their
robustness to inhomogeneities and noise in both their internal state variables
and external input signals, we designed on-chip learning circuits with
short-term analog dynamics and long-term tristate discretization mechanisms. An
additional hysteretic stop-learning mechanism is included to improve stability
and automatically disable weight updates when necessary, to enable continuous
always-on learning. We designed a spiking neural network with these learning
circuits in a prototype chip using a 180 nm CMOS technology. Simulation and
silicon measurement results from the prototype chip are presented. These
circuits enable the construction of large-scale spiking neural networks with
online learning capabilities for real-world edge computing tasks.
Arianna Rubino, Matteo Cartiglia, Melika Payvand, Giacomo Indiveri
# Neuromorphic analog circuits for robust on-chip always-on learning in spiking neural networks

###### Abstract

Mixed-signal neuromorphic systems represent a promising solution for solving extreme-edge computing tasks without relying on external computing resources. Their spiking neural network circuits are optimized for processing sensory data on-line in continuous-time. However, their low precision and high variability can severely limit their performance. To address this issue and improve their robustness to inhomogeneities and noise in both their internal state variables and external input signals, we designed on-chip learning circuits with short-term analog dynamics and long-term tristate discretization mechanisms. An additional hysteretic stop-learning mechanism is included to improve stability and automatically disable weight updates when necessary, to enable continuous always-on learning. We designed a spiking neural network with these learning circuits in a prototype chip using a \(180\,\mathrm{nm}\) CMOS technology. Simulation and silicon measurement results from the prototype chip are presented. These circuits enable the construction of large-scale spiking neural networks with online learning capabilities for real-world edge computing tasks.

 always-on learning, edge-computing, on-chip learning online, SNN, hysteresis, tristability.

## I Introduction

The requirements of artificial intelligence (AI) systems operating at the edge are similar to those that living organisms face to function in daily life. They need to measure sensory signals in real-time, perform closed-loop interactions with their surroundings, be energy-efficient, and continuously adapt to changes in the environment and in their own internal state. These requisites are well supported by neuromorphic systems and emerging memory technologies that implement brain-inspired mixed-signal spiking neural network (SNN) architectures [1, 2, 3, 4, 5]. These types of SNNs operate in a data-driven manner, with an event-based representation that is typically sparse in both space and time. Since they compute only when data is present, they are very power efficient. Similar to the biological neural systems they model, these SNNs are particularly well-suited to processing real-world signals. They can be designed to operate at the same data-rate of the input streams in real-time by matching the time constants of neural computation with those of the incoming signal dynamics. However, similar to their biological counterparts, these systems are affected by a high degree of variability and sensitivity to noise. One of the most effective strategies that biology uses to cope with noise and variability is to utilize adaptation and plasticity. This strategy has also been adopted by the neuromorphic community: several on-chip implementations of spike-based learning circuits have been proposed in the past [6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17]. However, few have addressed the problem of being able to operate robustly and autonomously in continuous time, with the ability to switch automatically and reliably between learning and inference modes. Following the original neuromorphic engineering approach [18], we propose a set of analog circuits that faithfully emulate synaptic plasticity mechanisms observed in pyramidal cells of cortical circuits and implement complex spike-based learning and state-dependent mechanisms that support this functionality. In addition, we extend the concept of long-term bi-stability of synaptic weights, proposed to increase robustness to noise and variability in the input signals [14, 19], to a tristate stability and weight discretization circuit that increases the resolution of the (stable and crystallized) synaptic weights. The synaptic plasticity circuits update an internal state variable of the synapse on every pre-synaptic input spike. The change in this state variable is computed in continuous time by the soma block of the neuron. In parallel, depending on its value, the internal variable is driven to one of three possible stable states and converted to a discrete three-state synaptic weight current value. The post-synaptic learning circuits comprise an additional mechanism that gates

Fig. 1: Micrograph of the prototype chip comprising of the learning circuits in a network of 4 neurons with 64 synapses each (see highlighted area). The chip, comprising of additional test structures, measures \(3\times 5\,\mathrm{mm}^{2}\).
the weight changes, to stop the learning process when the neuron's mean firing rate is outside a defined learning window. The circuits were fabricated on a prototype SNN chip designed in a 180 nm 6M1P CMOS technology and tested within a network of 4 neurons with 64 synapses each (see Fig. 1). In the following sections we describe the main building blocks used at both the synapse and neuron level, demonstrate their expected behavior with circuit simulations, and provide experimental results measured from the chip.

## II Network architecture

The block diagram of each neuron in the network is shown in Fig. 2. Input digital events (\(\mathsf{x_{0-N}}\)) arrive at the individual synapses via asynchronous logic [2] and trigger local weight update circuits to induce a change in the voltage stored on a local capacitor by an amount determined by the post-synaptic learning circuits. In parallel, a tristate stability circuit drives this internal voltage to one of three possible stable states. This local internal voltage is then discretized and converted to a low, intermediate or high current value. All currents produced by all synapses are summed spatially and conveyed to a differential pair integrator (DPI), which integrates the weighted sum over time [20]. A parallel and analogous pathway receives input events representing a desired target signal (\(\mathsf{x_{t}}\)), and produces a corresponding current from its dedicated DPI circuit. The target and input currents are both summed to drive the neuron's postsynaptic Integrate & Fire (I&F) circuit [21], and subtracted to drive the soma's Delta rule circuit [22]. The Delta rule circuit produces either positive or negative weight update signals proportional to the difference between the target input and the weighted synaptic input. These signals are broadcast to all the neuron's input synapses in continuous time if learning is enabled. Learning is enabled (or disabled) by means of two hysteretic Winner-Take-All (hWTA) circuits [23] that compare the neuron's mean output firing rate to a low and a high threshold (see Section III-B for details).

## III Circuits

As the details of the Delta rule and I&F circuits have already been presented [11, 21, 22], we describe the synapse learning circuits and the soma hWTA circuit.

### _Plastic synapse circuit_

Figure 3 presents all of the learning circuits used at the synaptic level. With every pre-synaptic spike, the weight update circuit (Fig. 2(a)) increases or decreases the internal analog weight variable \(\mathsf{V_{w}}\) by an amount determined by the voltages \(\mathsf{V_{UP}}\) and \(\mathsf{V_{DN}}\), produced by the post-synaptic Delta rule circuits. The tri-stability supply voltage circuit (Fig. 2(b)), produces the biases that power either of the positive feedback amplifiers of Fig. 2(c), depending on the state of \(\mathsf{V_{w}}\) with respect to \(\mathsf{V_{dd}}/2\). The tri-stability circuit (Fig. 2(c)) consists of two slew-rate limited positive feedback amplifiers which slowly drive \(\mathsf{V_{w}}\) towards ground, \(\mathsf{V_{dd}}/2\), or \(\mathsf{V_{dd}}\) depending on the value of \(\mathsf{V_{w}}\) relative

Fig. 3: Synapse learning circuits. (a) The weight update circuit increases or decreases the internal analog weight variable \(\mathsf{V_{w}}\) with every input spike \(\mathsf{V_{PRE}}\); (b) The tri-stability supply voltage circuit determines which of the two amplifiers in (c) to power depending on the value of \(\mathsf{V_{w}}\) with respect to \(\mathsf{V_{dd}}/2\) by activating either \(\mathsf{V_{DH}}\) or \(\mathsf{V_{DL}}\). (c) The tristability circuit drives the \(\mathsf{V_{w}}\) voltage towards ground, \(\mathsf{V_{dd}}/2\) or \(\mathsf{V_{dd}}\) depending on its value relative to \(\mathsf{V_{THH}}\) and \(\mathsf{V_{THL}}\). (d) The current discretization circuit converts \(\mathsf{V_{w}}\) into a low (the leakage current \(\mathsf{I_{0}}\)), intermediate, or high current.

Fig. 2: Block diagram of a single neuron row. The plastic synapses (in red) consist of input logic, a weight update, tristate stability, and a current ADC block. Input spikes arriving at a synapse update the internal weight variable \(\mathsf{V_{w}}\), and change it by an amount that is determined by the post-synaptic learning circuits at the soma (in green). The weight voltage is slowly driven to one of three possible stable states, and converted into a synaptic current by thresholding circuits. A parallel pathway provides a target current (in blue). Both input and target currents are integrated over time by differential pair integrator (DPI) circuits. The soma (in green), comprises of an Integrate & Fire (I&F) block which integrates the sum of target and input currents, a Delta rule block that calculates the difference of the two DPI currents to determine the amplitude of the weight change, and a hysteretic Winner-Take-All (hWTA) block used to determine if and when to “stop-learning”.
to \(\mathsf{V_{THH}}\) and \(\mathsf{V_{THL}}\). The weight discretization circuit (Fig. (d)d) sets the value of the effective synaptic current \(\mathsf{l_{w}}\) to \(\mathsf{l_{0}}\), \(\mathsf{l_{wb}}\), or \(2\mathsf{l_{wb}}\) depending on the state of \(\mathsf{V_{w}}\) with respect to \(\mathsf{V_{THL}}\) and \(\mathsf{V_{THH}}\).

### _Hysteretic WTA for "stop-learning"_

Figure 4 shows an instance of a hWTA circuit: it consists of two identical cells, (\(M_{2}\)-\(M_{6}\)) and (\(M_{7}\)-\(M_{11}\)) that compete with each other. As soon as one cell wins (e.g., the left one), the bias current \(\mathsf{l_{bn}}\) is copied and added to the input current of the winning branch (e.g., \(\mathsf{l_{L}}\)). This creates a hysteresis window, such that for the winning (left) cell to lose the competition, its input current has to decrease below the input current of the opposite branch by an additional factor equal to the bias current (\(\mathsf{l_{L}}<\mathsf{l_{R}}-\mathsf{l_{bn}}\)). The output voltage of this circuit \(\mathsf{V_{OUT}}\) switches to "high" when the left cell wins, and to "low" when the right cell becomes the winner.

To implement the "stop-learning" mechanism [14], we produce a current \(\mathsf{l_{Ca}}\) (a surrogate of the neuron's calcium concentration) by integrating the post-synaptic neuron spikes with a DPI circuit [20]. We then compare this current to two thresholds with the two hWTA circuits. The digital output nodes of the two hWTA circuits were connected to logic gates to produce an active high \(\mathsf{Learn}\) signal when the \(\mathsf{l_{Ca}}\) current is within the set bounds (i.e., within the learning region) and a low when it is outside this region. This \(\mathsf{Learn}\) signal is then used as a "third factor" to enable or disable the Delta rule weight circuit, and switch on or off the weight updates. The hysteresis windows of the hWTA circuits are used to distinguish between cases in which the target input is present (to enable learning) or absent (to disable learning and automatically switch to an "inference" mode). The effect of this window is described in Section IV-A.

## IV Results

We validate the learning circuits with both circuit simulations and with experimental results measured from the fabricated chip.

### _Circuit simulation results_

Here, we show simulations of a single neuron and 40 plastic synapses during a learning task and show how the hWTA enables automatic switching from learning to inference.

After initializing all synaptic weights to zero, we started a training phase by stimulating each plastic synapse with a \(25\,\mathrm{Hz}\) input spike train, and by sending a spike train with a \(1\,\mathrm{kHz}\) frequency to the target synapse. As expected, during this training phase, the weights of the synapses potentiated and the total weighted synaptic input current increased (see the red trace of Fig. 5). During the inference phase, we removed the target input spike train while keeping on stimulating the input synapses. As expected, without this extra input, the average

Fig. 4: hysteresis WTA circuit used to determine the “stop-learning” signals. The digital output voltage \(\mathsf{V_{OUT}}\) switches from low to high only if the left input current \(\mathsf{l_{L}}\) increases above \(\mathsf{l_{R}}\) by an amount at least equal to \(\mathsf{l_{bh}}\).

Fig. 5: Row simulation results. In the top plots of both sub-figures, the DPI synapse current (in red) follows the DPI target current (in blue) when the target input is high. The neuron’s membrane activity (in grey) is scaled for visibility. In the lower plots of both sub-figures the calcium current (in black) enables the \(\mathsf{Learn}\) signal when it lies between the learning region’s low (green line) and high (red line) thresholds. (a) Small hysteresis bias (\(\mathsf{l_{bh}}=100\,\mathrm{pA}\)): after the target current is removed the calcium current drops back into the learning region, the weights are decreased, and the neuron forgets its tuning (b) Large hysteresis bias (\(\mathsf{l_{bh}}=800\,\mathrm{pA}\)): the large hysteresis region around the highest threshold (shaded pink) keeps the learning disabled, despite the fact that the calcium current fell below the high threshold. The weights of the plastic synapses do not change and the neuron maintains its proper tuning to the trained pattern.
mean firing rate of the neuron decreased, and the calcium concentration current fell below the upper bound of the learning region. Figure 5 shows this task performed with two values for \(\mathsf{b_{th}}\), which governs the width of the hysteresis window. Without a proper hysteresis window (Fig. 4(a)), when the neuron falls back into a learning region it "forgets" its training (i.e., the learning circuits decrease the weights). On the other hand, by properly tuning the hysteresis window (Fig. 4(b)), the network remains in a "stop-learning" mode, and the neuron retains a high output firing rate response to the trained pattern, even in absence of a target signal. In the larger hysteresis window case, the total estimated power consumption is 1.07 uW, and a maximum (mean) energy of 740 pJ (680 pJ) is required to update the weights.

### _Chip measurement results_

#### Iv-B1 Tristability

The results from the measurements of the plastic synapse circuits are shown in Fig. 6. Initially, the neuron is presented with a high target activity, triggering large positive weight updates and causing a rapid increase in the synapse weight internal variable. Upon removal of the target, the weight is decreased. By increasing the power to the tristate stability amplifiers (Fig. 4(c)), i.e., by increasing \(\mathsf{b_{bs}}\) of Fig. 4(b), the circuit opposes the weight changes more strongly and drives \(\mathsf{V_{w}}\) to one of the three stable states more quickly. Stability at \(\mathsf{V_{dd}}\) and ground is shown in Fig. 5(a) and at \(\mathsf{V_{dd}}\)/2 in Fig. 5(b). Once the stimulation ends, the tristability circuit crystalizes the weight to one of the three stable states depending on the value of \(\mathsf{b_{bs}}\).

#### Iv-B2 Hysteresis for "stop-learning"

Figure 7 shows the results of the characterization of the hysteretic calcium-based stop-learning mechanism. Similarly to the previous experiment, the neuron is initially stimulated with a high target activity, bringing it to the learning region. The plastic synapse weight rapidly increases, pushing the neuron into the "stop-learning" region. Once the target activity is removed, the neuron returns to the learning region, and, for small hysteresis window settings (top blue plot in Fig. 7), the plastic synapse decreases its weight as it is stimulated. For higher values of \(\mathsf{b_{th}}\) the hysteresis window increases (orange plot in Fig. 7) and when the target is removed, the neuron's return to the learning mode is delayed. As this delay increases, even though the plastic synapse keeps on being stimulated, the neuron remains in the "stop-learning" region and the weight remains unchanged (purple plot in Fig. 7).

## V Conclusions

We presented a set of analog circuits that enable learning in mixed-signal neuromorphic SNNs, with tristate stability and weight discretization circuits. By comparing the neuron's calcium concentration to lower and upper bounds, and by using hysteresis, we demonstrate effective always-on learning features, that automatically switch from learning mode to inference mode, without having to manually disable or enable learning. Comparisons to previous efforts are provided in Table I.

## Acknowledgment

The authors thank Shyam Narayanan, Charlotte Frenkel, and Junren Chen for fruitful discussions and contributions.

\begin{table}
\begin{tabular}{l l l l l l} \hline \hline  & [24] & [25] & [17] & [11] & [This work] \\ \hline \begin{tabular}{l} **Technology** \\ **Design** \\ **Learning** \\ **Stop** \\ **learning** \\ **Weight** \\ **resolution** \\ **Energy/SOP** \\ **Power** \\ **supply** \\ \end{tabular} & \begin{tabular}{l} 28 nm \\ digital \\ semi super- \\ vised \\ yes \\ yes \\ **Weight** \\ **22.7 pJ** \\ 0.55 V \\ \end{tabular} & \begin{tabular}{l} 14 nm \\ digital \\ program- \\ mobile \\ no \\ 120.75 V \\ \end{tabular} & \begin{tabular}{l} 180 nm \\ mixed- \\ signal super- \\ error based \\ yes \\ no \\ 120.75 V \\ \end{tabular} & 
\begin{tabular}{l} 180 nm \\ mixed- \\ signal \\ error based \\ yes \\ yes \\ \end{tabular} \\ \hline \hline \end{tabular}
\end{table} TABLE I: Comparison to the state-of-the-art

Fig. 6: Tristability chip measurements. (a) Stability at \(\mathsf{V_{dd}}\) and ground: When the target is presented, the plastic synapse weight is increased by the post-synaptic learning circuits (\(\overline{\mathsf{V_{up}}}\) and \(\mathsf{V_{dn}}\) scaled for visibility). As the tristability bias increases, the circuit opposes the weight update more strongly. When the target is removed, the tristability maintains the weight value around \(\mathsf{V_{dd}}\) for larger values of \(\mathsf{b_{bs}}\) (in orange and purple). (b) Stability at \(\mathsf{V_{dd}}\)/2: as \(\mathsf{b_{bs}}\) increases the tristability opposes the learning with more strength.

Fig. 7: Hysteresis chip measurements. The hysteresis window is shown for three different values of \(\mathsf{b_{th}}\). Increases in \(\mathsf{b_{th}}\) produce larger hysteresis windows which are useful for tuning the “stop-learning” properties of the network.