Event-Enhanced Multi-Modal Spiking Neural Network for Dynamic Obstacle
  Avoidance
Autonomous obstacle avoidance is of vital importance for an intelligent agent
such as a mobile robot to navigate in its environment. Existing
state-of-the-art methods train a spiking neural network (SNN) with deep
reinforcement learning (DRL) to achieve energy-efficient and fast inference
speed in complex/unknown scenes. These methods typically assume that the
environment is static while the obstacles in real-world scenes are often
dynamic. The movement of obstacles increases the complexity of the environment
and poses a great challenge to the existing methods. In this work, we approach
robust dynamic obstacle avoidance twofold. First, we introduce the neuromorphic
vision sensor (i.e., event camera) to provide motion cues complementary to the
traditional Laser depth data for handling dynamic obstacles. Second, we develop
an DRL-based event-enhanced multimodal spiking actor network (EEM-SAN) that
extracts information from motion events data via unsupervised representation
learning and fuses Laser and event camera data with learnable thresholding.
Experiments demonstrate that our EEM-SAN outperforms state-of-the-art obstacle
avoidance methods by a significant margin, especially for dynamic obstacle
avoidance.
Yang Wang, Bo Dong, Yuji Zhang, Yunduo Zhou, Haiyang Mei, Ziqi Wei, Xin Yang
# Event-Enhanced Multi-Modal Spiking Neural Network for Dynamic Obstacle Avoidance

###### Abstract.

Autonomous obstacle avoidance is of vital importance for an intelligent agent such as a mobile robot to navigate in its environment. Existing state-of-the-art methods train a spiking neural network (SNN) with deep reinforcement learning (DRL) to achieve energy-efficient and fast inference speed in complex/unknown scenes. These methods typically assume that the environment is static while the obstacles in real-world scenes are often _dynamic_. The movement of obstacles increases the complexity of the environment and poses a great challenge to the existing methods. In this work, we approach robust dynamic obstacle avoidance twofold. First, we introduce the neuromorphic vision sensor (_i.e._, event camera) to provide _motion cues_ complementary to the traditional Laser depth data for handling dynamic obstacles. Second, we develop an DRL-based event-enhanced multimodal spiking actor network (EEM-SAN) that extracts information from motion events data via _unsupervised representation learning_ and fuses Laser and event camera data with _learnable thresholding_. Experiments demonstrate that our EEM-SAN outperforms state-of-the-art obstacle avoidance methods by a significant margin, especially for dynamic obstacle avoidance.

Dynamic Vision Sensor (DVS); Spiking Neural Network (SNN); Deep Reinforcement Learning (DRL); dynamic obstacle avoidance. +
Footnote †: ccs: Computing methodologies Vision for robotics; Reinforcement learning: Spiking neural networks

+
power consumption and low latency obstacle avoidance (Beng et al., 2017; Chen et al., 2018; Wang et al., 2019). With recent advances in deep learning, training SNN with RL for obstacle avoidance becomes the mainstream method in the field and achieves promising results. However, existing methods typically assume that the environment is static (Beng et al., 2017; Wang et al., 2019) while in real-world scenes such as malls and streets, the obstacles (_e.g._, pedestrians and vehicles) are not always static but often _dynamic_. The movement of obstacles makes the scene become more complex and leads to a faster relative speed between the agent and the obstacle, which requires the agent to make decisions and take avoidance actions in a shorter time, posing a great challenge to the existing methods.

In this work, we strive to embrace challenges toward robust _dynamic obstacle avoidance_. We approach this twofold. First, based on that the neuromorphic event camera called Dynamic Vision Sensor (DVS) (Shen et al., 2016; Wang et al., 2019) can record a _high-frequency_ stream of asynchronous brightness change events with extremely _low latency_ (in the order of \(\mu\)s) and _low power consumption_, we for the first time introduce the two-dimensional DVS event modality into the SNN-based obstacle avoidance framework to provide motion cues complementary to the traditional Laser depth data for handling dynamic obstacles, building an omni perception of scenes, similar to human stereo vision (Shen et al., 2016). Second, we develop an event-enhanced multimodal spiking actor network (EEM-SAN) that achieves dynamic obstacle avoidance in a deep reinforcement learning manner. EEM-SAN is built on three key modules: (i) a hybrid spiking variational autoencoder (HSVAE) that extracts information from DVS event data via unsupervised representation learning; (ii) a population coding (PC) module that combines population coding (Shen et al., 2016) and Poisson coding (Shen et al., 2016) to decode information from the activity of neurons; and (iii) a middle fuse decision module with learnable thresholding (MFDM-LT) designed for multimodal data fusion.

We perform extensive experiments to demonstrate the efficacy of our method and show that DVS events provide a powerful and complementary cue for dynamic obstacle avoidance (Figure 1). In summary, our contributions are:

* the first solution to solve the challenging dynamic obstacle avoidance problem using a deep-reinforcement-learning-based spiking neural network with robot state and both Laser and DVS data as input, action decision as output;
* a novel hybrid spiking variational autoencoder that decouples the representation learning of DVS event data from the whole reinforcement learning and greatly facilitates the training process;
* a new middle fuse decision module with learnable thresholding to robustly integrate Laser and DVS data.

## 2. Background and Related Work

_Leaky Integrate-and-Fire (LIF) spiking neuron model._ A spiking neural network (SNN) is a bio-plausible neural network that simulates biological information processing: neurons exchange information through spikes (or action potentials). Many different mathematical spiking neuron models have been developed, spanning from the simplest Integrate-and-Fire (IF) (Wang et al., 2019) to sophisticated Spike Response Model (SRM) (Shen et al., 2016). Our approach leverages the _Leaky Integrated-and-Fire (LIF) model_(Shen et al., 2016), a simplified variant of SRM model. We can define an \(n_{l}\)-layer feedforward SNN architecture with LIF neurons. Given \(N^{l}\) incoming spike trains at layer \(l\), \(s_{l}^{l}(t)\), the SNN forward propagation is mathematically defined as:

\[v_{l}^{l+1}(t)=\sum_{j=1}^{N^{l}}w_{ij}s_{j}^{l}(t)+\] \[v_{i}^{l+1}(t-1)f_{d}\left(s_{i}^{l+1}(t-1)\right)+b_{i}^{l+1},\] \[s_{l}^{l+1}(t)=f_{s}\left(b_{i}^{l+1}(t)\right),\] \[f_{d}(s(t))=\begin{cases}D&s(t)=0\\ 0&s(t)=1\end{cases}, \tag{1}\]

where \(w_{ij}\) is the synaptic weight between the \(j\)-th neuron on the \(l\)-th layer and the \(i\)-th neuron on the layer \(l+1\); \(b_{i}^{l+1}\) is an adjustable bias, and \(D\) is a constant. The operator \(f_{s}(\cdot)\) is a spike function defined as:

\[f_{s}(v):v\to s,s(t):=s(t)+\delta(t-t^{(f+1)}), \tag{2}\] \[t^{f+1}=\min\{t:v(t)=\Theta,t>t^{(f)}\}, \tag{3}\]

where \(s(t)\) is a sequence of spikes called a spike train, \(\delta\) is a mathematical function whose value is zero everywhere except at zero and whose integral over the entire real line is equal to one, and \(\Theta\) is the membrane potential threshold which is static and the same for all neurons in the network (Shen et al., 2016).

_Dynamic Vision Sensor._ DVS is a bio-inspired sensor that reports per-pixel brightness changes in log scale as a stream of asynchronous events (Shen et al., 2016; Wang et al., 2019). Compared to conventional frame-based cameras, DVS offers a very high dynamic range (140 dB versus 60 dB)

Figure 1. Illustration of obstacle avoidance using different sensors: (a) Laser can help detect large static objects but fails to perceive fast-moving objects; (b) event camera DVS excels to capture moving objects but cannot provide depth information to avoid large textureless obstacles; and (c) combining Laser and DVS achieves robust obstacle avoidance.
and high temporal resolution (in the order of \(\mu\)s). An event, \(e\), encodes three pieces of information: the pixel location, \((x,y)\), of an event, the timestamp, \(t\), records the time when the event is triggered, and the polarity, \(p\in\{-1,1\}\), of an event, which reflects the direction of the changes. Formally, a set of events can be defined as

\[\mathcal{E}=\{e_{k}\}_{k=1}^{N}=\{[x_{k},y_{k},t_{k},p_{k}]\}_{k=1}^{N}. \tag{4}\]

In constant lighting conditions, events are triggered by moving edges (_e.g._, object contour and texture boundaries), making the DVS a natural edge extractor. However, this attractive feature also poses a unique challenge since events predominantly stem from edges, making the measured events inherently sparse. Asynchronous and sparse events cannot be effectively handled by CNN-based approaches designed for conventional frames.

DVS-based Robot ControlMost robotics applications use traditional frame-based cameras as their perception devices. However, frame-based cameras have inherent characteristics such as high data volume, low temporal resolution, and high latency, which weakens their ability to perceive fast-moving objects and greatly limit the robot's manipulation capabilities. In this context, event cameras have attracted the attention of scholars in the field of robotics. By combining event cameras with robot perception and control, a series of breakthroughs in robot control has emerged, overcoming the limitations of traditional frame-based cameras. (Zang et al., 2017) is the first work to use DVS to avoid high-speed moving objects. To further explore the advantages of DVS, Falanga _et al.(Falanga et al., 2018)_ conducted experiments on UAVs equipped with three sensors: monocular, stereo frame-based cameras, and DVS. The results showed that the DVS (\(2\sim 4\)_ms_) had significantly lower delay than the monocular (\(26\sim 40\)_ms_) and stereo frame-based cameras (\(17\sim 70\)_ms_) when operating within the perception range of \(2\)_m_. Sanket _et al.(Sanket et al., 2018)_ used a artificial neural network (ANN) to segment independent moving objects from event streams, and reasoned about their 3D motion to perform evasion tasks. Most of these methods rely on hand-crafted features or priors (_e.g._, Kalman filter (Zang et al., 2017), optical flow (Falanga et al., 2018), and the obstacles (Sanket et al., 2018)) to perform obstacle reasoning and avoidance. Our method differs from the above works in that we combine DRL and SNN with DVS to achieve robot control, enabling continuous autonomous robot navigation and robust yet efficient dynamic obstacle avoidance.

SNNs for Multimodal-Based SensingSNNs have not been widely explored for multimodal-based sensing (Sanket et al., 2018; Sanket et al., 2018). A few attempts have been made to combine image and audio modality. Liu _et al.(Liu et al., 2018)_ developed a weighted-sum-based attention scheme to fuse image and audio modalities. The weights for each modality are dynamically decided. In the same vein, Jia _et al.(Jia et al., 2018)_ proposed an MR-SNN algorithm to fuse the same two modalities using a fusion mask. MR-SNN learns a Motif mask for each modality and generates a fusion mask by averaging the two learned masks. Instead of learning in a supervised manner, Rathi _et al.(Rathi et al., 2018)_ proposed an unsupervised multimodal learning method, which combines the image and audio modality by learning cross-modal connections enabled by the Spike Timing Dependent Plasticity (STDP) algorithm. All these methods assess their effectivenesses with a simple MNIST classification task. We validate our multimodal-based SNN approach with a more complex practical problem (_i.e._, obstacle avoidance), under both static and dynamic conditions.

SNNs_for Robot ControlSNNs get substantial attention from the robot control community due to their high energy efficiency when deployed on neuromorphic hardware. A thread of work leverages SNNs for robot obstacle avoidance tasks (Tang et al., 2018; Sanket et al., 2018; Sanket et al., 2018). Tang _et al.(Tang et al., 2018)_ proposed a hybrid framework SDDPG for mapless navigation tasks. The SDDPG framework consists of an SNN-based actor network and a CNN-based critic network, where the two networks are contained together. Ding _et al.(Ding et al., 2018)_ proposed a bio-inspired dynamic spiking threshold scheme to enhance SDDPG's homeostasis in obstacle avoidance tasks under normal and degraded conditions. Later, Tang _et al.(Tang et al., 2018)_ extended their approach to continuous control tasks and proposed the PopSAN method. An essential contribution introduced by PopSAN is the population-coding scheme, which effectively addresses the high-dimensional state problem presented in continuous control tasks. Recently, Zhang _et al.(Zhang et al., 2018)_ combined multiscale dynamic neurons coding and population coding to improve the performance of a spiking actor network. A critical difference between our approach and these methods is that we leverage two modalities for robot control instead of one.

## 3. Methodology

Figure 2(a) illustrates the diagram of our proposed event-enhanced multimodal spiking actor network (EEM-SAN) that is trained with reinforcement learning. First, the DVS sensor mounted on the agent records a high-frequency stream of brightness change events which is then encoded by a hybrid spiking variational autoencoder (HSVAE) (Figure 2(b)) into a one-dimensional vector (_i.e._, DVS State). Then, the obtained DVS state, together with the robot state and Laser state, is processed by a population coding (PC) module (Figure 2(c)) that can decode information from the activity of neurons. Finally, a middle fuse decision module with learnable thresholding (MFDM-LT) (Figure 2(d)) is used to fuse multimodal data and output obstacle avoidance decision which will be decoded into actual control action by Algorithm 1.

### Hybrid Spiking Variational Autoencoder

To exploit useful motion cues embedded in DVS event data, a key problem is how to effectively and efficiently learn its feature representations. Variational Autoencoder (VAE) (Goodfellow et al., 2016) is a powerful tool to extract the high-level embeddings of the input data. It typically consists of an encoder and decoder networks and is trained by:

\[Loss=\frac{1}{\mathsf{w}\times h}\sum_{i,j}^{w,h}(x_{i,j}-\overline{x_{i,j}})^ {2}+\int p(x)log\frac{p(x)}{q(x)}, \tag{5}\]

where the first item is the mean squared error (MSE) between the original input image \(x\) and the reconstruction \(\overline{x}\), and the second item measures how the latent vector \(p(x)\) matches a unit Gaussian \(q(x)\) by Kullback-Leibler (KL) divergence. \(W\) and \(1\) represent the width and height of the event frame, respectively. VAEs have stable learning abilities in generative models and can be applied to various tasks. Recently, a series of works have attempted to use SNNs to create VAEs and make validation on classification datasets (Sanket et al., 2018; Sanket et al., 2018; Sanket et al., 2018; Sanket et al., 2018). To take full advantage of the event-based nature of the continuous DVS stream and its rich temporal features, we design a novel hybrid spiking variational autoencoder (HSVAE) that contains
an SNN encoder and ANN decoder. The architecture details of HSVAE are shown in Table 1. Unlike traditional spiking neural networks that reset the membrane potential, our method records all SNN-related states during one episode of simulated environment interaction. We record the current value, membrane potential and spike value of every neuron during one episode for the complete backpropagation chain. During deployment, we only need to record the relevant states at the last moment. We verified that SNNs trained in this way can integrate more temporal information to make better decisions. A single SNN layer of our HSVAE is shown in Figure 2(b) where refractory states are omitted for clarity. As illustrated, a stream of events recorded by the event camera mounted on a mobile robot is fed into SNN that can encode the spatio-temporal features of the input data into a latent state \(z\). In Figure 2(b), \(P\) is pre-synaptic potential, \(C\) is current value and \(V\) is the membrane potential of the spiking neuron. The training process of HSVAE is presented in Algorithm 2.

\begin{table}
\begin{tabular}{|c|c|c||c|} \hline
**Layer** & **Kernel** & **Output** & **Layer Type** \\ \hline input & & 128\(\times\)128\(\times\)1 & DVS128 \\
1 & 16\(\times\)4p1s2 & 64\(\times\)64\(\times\)16 & \\
2 & 32\(\times\)4p1s2 & 32\(\times\)32\(\times\)32 & SNN LIF Encoder \\
3 & 16\(\times\)4p1s2 & 16\(\times\)16\(\times\)16 & \\
4 & 1e1p1s2 & 8\(\times\)8\(\times\)1 & \\
5 & - & 64 & Flatten() \\
6 & - & 64 & \(\mu=\) FC(\(V^{t}\)) \\
7 & - & 64 & \(\sigma=\) FC(\(V^{t}\)) \\
8 & - & 8\(\times\)8\(\times\)1 & UnFlatten() \\
9 & 16\(\times\)4p1s2 & 16\(\times\)16\(\times\)16 & \\
10 & 32\(\times\)4p1s2 & 32\(\times\)32\(\times\)32 & ANN Decoder \\
11 & 16\(\times\)4p1s2 & 64\(\times\)64\(\times\)16 & \\
12 & 1e4p1s2 & 128\(\times\)128\(\times\)1 & Event Frame \\ \hline \end{tabular}

* Notation: XcIPzS represents channel X convolution filters (YxY) with padding \(Z\) and stride \(S\).

\end{table}
Table 1. Architecture of our Hybrid Spiking VAE (HSVAE).

Figure 2. Overview of EEM-SAN (a) and its three main components: (b) a Hybrid Spiking Variational Autoencoder (HSVAE) module, (c) a Population Coding (PC) module, and (d) a Middle Fuse Decision Module with Learnable Thresholding (MFDM-LT).
```
0: Left and right wheel speeds \(v_{L}\), \(v_{R}\)
0: Min and max wheel speeds \(v_{min}\), \(v_{max}\)
0: Robot State, Laser State, Event frame Deque: \(D_{E}=(E_{1},\cdots,E_{5})\)
0: Learnable encoding means \(\mathbf{\mu}\) and standard deviations \(\mathbf{\sigma}\) for all populations
0: Middle Fuse Module: \(MF\) and total timestep T DVS state generated by the Spiking Encoder of HSVAE: DVS State = HSVAE (\(D_{E}\)) Input Multimodal Observation \(O\) = [Robot State, Laser State, DVS State] Spikes generated by the population coding module: \(\mathbf{X}\) = PC (\(\mathbf{o}\), \(\mathbf{\mu}\), \(\mathbf{\sigma}\)) for\(t=1,\cdots,T\)do  Laser spikes from populations at timestep \(t\): \(\mathbf{S}_{L}^{(t)(0)}=\mathbf{X}^{(t)(O_{0}\cdots O_{L})}\)  DVS spikes from populations at timestep \(t\): \(\mathbf{S}_{L}^{(t)(0)}=\mathbf{X}^{(t)(O_{D}\cdots O_{cmd})}\)  Fuse spikes from Middle Fuse Module: \(\mathbf{S}_{M}^{(t)(0)}=MF(\mathbf{S}_{L}^{(t)(0)},\mathbf{S}_{D}^{(t)(0)})\) for\(k=1,\cdots,K\)do  Update LIF neurons in layer \(K\) at timestep \(t\) based \(\mathbf{e}^{(t)(k)}=d_{c}\cdot\mathbf{e}^{(t-1)(k)}+\mathbf{W}^{(k)}\mathbf{s}^{(t)(k-1)}+ \mathbf{b}^{(k)}\); \(\mathbf{g}^{(t)(k)}=d_{o}\cdot\mathbf{g}^{(t-1)(k)}\cdot(1-\mathbf{s}^{(t-1)(k)})+\mathbf{e}^ {(t)(k)}\); \(\mathbf{s}^{(t)(k)}=Threshold(\mathbf{a}^{(t)(k)})\); endfor SpikeCount\({}^{t}\) = SpikeCount\({}^{t-1}+\mathbf{s}^{(t)(t)}\) endfor Action = SpikeCount\({}^{(T)}/T\) \(v_{L}\) = Action[0] * \((v_{max}-v_{min})+v_{min}\) \(v_{R}\) = Action[1] * \((v_{max}-v_{min})+v_{min}\)
```

**Algorithm 1** Forward propagation through EEM-SAN

### Population Coding

Neurons in the brain often use population coding and it isn't easy to decode correct information from the activity of a single neuron (Sang et al., 2017). Hence, some researchers have begun to use populations of neurons to encode information into spike trains fed into SNNs instead of simple frequency encoding (_e.g._, Poisson coding) (Sang et al., 2017; Wang et al., 2018). To encode each dimension of the state, we created a population with 10 neurons, where each neuron has a Gaussian receptive field with two parameters: mean and standard deviation. These parameters were learned with surrogate backpropagation. Since SAN adopts the Poisson encoding, we apply the group encoding method combined with Poisson coding as PopSAN method in our experimental comparison. For a fair comparison, our method takes the same population and Poisson coding. Formally, the population coding function can be formulated as:

\[\begin{cases}A_{P_{t,j}}=EXP(-1/2\cdot((s_{i}-\mu_{i,j})/\sigma_{i,j})^{2}) \\ \mathbf{A}_{P}=[A_{P_{t,1}},\cdots,A_{P_{t,j}},\cdots,A_{P_{N,J}}]\end{cases}, \tag{6}\]

\[P(O_{k,t}=1)=C_{R}^{\sigma}A_{P_{k}}^{\sigma}(1-A_{P_{k}})^{R-r}, \tag{7}\]

where \(i\) is the index of the input state (\(i=1,...,N\)), \(j\) is the index of neurons in a population (j = 1,..., \(J\)), and \(A_{P}\) is the stimulation strength after population coding, used for drawing the binary random number (Wang et al., 2018).

### Middle Fuse Decision Module with Learnable Thresholding

The middle fuse decision module consists of a middle fuse (MF) module and a decision module (DM). In MF, two modalities are first transformed to a one-dimensional vector with a length of 20 by two fully connected layers composed of LIF neurons, respectively, and the two obtained one-dimensional vectors are then fused via the element-wise addition. DM contains four fully connected SNN layers and its forward propagation process is presented in Algorithm 1. Since the threshold function of SNNs is non-differentiable, many methods focus on how to learn and tune it (Wang et al., 2018; Wang et al., 2018; Wang et al., 2018). Among them, the approximate backpropagation method is widely due to its efficiency and flexibility. Here, we adopt the STBP (Sang et al., 2017) algorithm which uses the rectangular function to approximate the gradient of a spike as follows:

\[z(V)=\begin{cases}1&\text{if }|V-V_{th}|<0.5\\ 0&\text{otherwise}\end{cases}, \tag{8}\]

where \(z\) is the pseudo-gradient, \(V\) is membrane potential, and \(V_{th}\) is the threshold.

Several dynamic thresholding schemes have been developed in recent years to make neuronal thresholds learnable and varied throughout the network. However, most existing methods require strict prerequisites or are not based on the gradient descent method, which results in a higher computational load or makes it difficult to migrate to other SNNs (Wang et al., 2018; Wang et al., 2018). A few attempts have been made to integrate with the existing back-propagation-based training algorithms on classification tasks (Wang et al., 2018; Wang et al., 2018). Following the path, we further explore the performance of learnable thresholds under multimodal reinforcement learning tasks. We use the learnable thresholding
mechanism in our MFDM. This mechanism is endowed with parameter optimization capability through STBP. By doing so, all neurons have different thresholds in the same network. This means the neuron's response depends not only on its internal state but also on the threshold level. The key idea of the learnable thresholding is to find the comprehensive gradient of the loss function, and then the weight \(W\) and the neuron's threshold \(H\) are simultaneously updated until convergence. From Eq. 1 and 8, the threshold's partial derivatives of the loss function can be calculated as follows:

\[\frac{\partial L}{\partial H^{n}}=\sum_{t=1}^{T}\frac{\partial L}{\partial s^{ t,n}}\frac{\partial s^{t,n}}{\partial H^{n}}\frac{\partial t^{n}}{\partial H^{n}}. \tag{9}\]

To ensure that the threshold remains within the appropriate area, we create a new parameter \(r\) to define \(H\) using hyperbolic tangent relation, formulated as \(H=\tanh\left(r\right)\). With this, Eq. 9 can be expressed as:

\[\begin{split}\frac{\partial L}{\partial r^{n}}&= \sum_{t=1}^{T}\frac{\partial L}{\partial s^{t,n}}\frac{\partial s^{t,n}}{ \partial H^{n}}\frac{\partial H^{n}}{\partial r^{n}}\\ &=-\sum_{t=1}^{T}\frac{\partial L}{\partial s^{t,n}}f^{{}^{\prime }}\left({{v}^{t,n}}-tanh(r^{n})\right)\left(1-tanh^{2}(r^{n})\right).\end{split} \tag{10}\]

Improved in this way, the threshold \(H\) of the neurons can be iteratively trained using the backpropagation method and will be in the range (-1, 1). In this context, we apply the learnable thresholding scheme to MFDM and let neurons in the same layer share the same threshold to reduce learnable parameters.

## 4. Experiments and Evaluation

We evaluate the obstacle avoidance capabilities of the proposed EEM-SAN using success rate (SR) as a metric. SR is the percentage of successful passes among 200 trials. A successful pass is a trial in which a robot can reach the destination without touching any static or dynamic obstacles within 1000 steps. Our evaluation baseline model and test environment are modified variants of the SAN (Song et al., 2018) and its original simulated test environment, respectively.

### Experimental Settings

**Implementation Details.** We integrated the EEM-SAN with the deep reinforcement learning algorithm DDPG (Krizhevsky et al., 2014). We repeated each experiment three times to obtain the mean success rate. The frequency of the Laser sensor and DVS are set to 20 Hz and 100 Hz, respectively. The timestep of MFDM-LT is set as 5. The number of groups in the population coding is set as 10. The neuron current decay constant and the voltage decay constant are set as 0.5 and 0.75, respectively. During the training process, we set the batch size to 256 and the learning rate to 0.0001.

**Simulator Setup.** Our experiments are based on the Gazebo simulator (Gazebo, 2019). Both the training and testing use the robot operating system (ROS) as a middleware. The testing environment was set to be different from the training environment for better validating the generalization capability of a method. Following existing methods SAN (Song et al., 2018), PopSAN (Song et al., 2018), and BDETT (Bahdan et al., 2018), we developed and tested our method in the same environment and with the same random seeds. Our testing environments are set to be very challenging to better validate the robustness of an obstacle avoidance method. The challenges of our testing environments are in the following aspects: (i) **densely and highly dynamic obstacles** (eleven dynamic obstacles, all set to higher than the maximum speed of the robot); (ii) **faster robot speed** (the maximum speed of the robot is twice that in SAN); and (iii) **narrow traversal passages and more densely organized static obstacles**. We have experimentally demonstrated in Table 2 that the existing SOTA method SAN (Song et al., 2018) has a significant accuracy drop (_i.e._, from 97.8% to 58.0%) when transferred from common scenes to our challenging scenes. More details about training and testing environment can be found in Appendix and SAN (Song et al., 2018).

### Evaluation

We extensively compare the effectiveness of our proposed EEM-SAN to three state-of-the-art methods including SAN (Song et al., 2018), PopSAN (Song et al., 2018), and BDETT (Bahdan et al., 2018) across two different test maps (Bahdan et al., 2018; Song et al., 2018) under both dynamic and static conditions. The experimental results are reported in Table 2 and Figure 3.

_Dynamic Conditions._ To ensure the diversity and challenge of the scene, we set up eleven dynamic obstacles that reciprocate linearly along different trajectories at different speeds, and the speed of all moving obstacles is set to slightly higher than the maximum speed of the robot. From the results in Table 2, we can see that our EEM-SAN outperforms all competing methods by a significant margin. For example, compared with the state-of-the-art method BDETT (Bahdan et al., 2018), our method improves _SR_ by 10.8% and 11.8% on the two testing maps, respectively. This clearly demonstrates the effectiveness of our method for dynamic obstacle avoidance.

_Static Conditions._ Although our method was originally designed for dynamic obstacle avoidance, we also tested its performance in static conditions to see its robustness. From Table 2, we observe that: (1) our method still performs the best among all the compared methods for static obstacle avoidance; (2) when varying the maximum robot speed from 1.0 m/s to 0.5 m/s, all the methods get significant performance improvement. This is because a robot with a slow-moving speed has more time to make a decision and take action, which greatly decreases the difficulty of obstacle avoidance. Under such a condition, our method achieves very robust avoidance, _i.e._, the success rate _SR_ is up to 1.000; (3) compared to SAN (Song et al., 2018), the PopSAN (Song et al., 2018) which is based on SAN (Song et al., 2018) but equipped with population coding method instead of frequency coding performs better, especially for the robot with a maximum speed of 1.0 m/s. This indicates that the population coding method is more suitable for handling rapidly changing scenes than the frequency coding method; and (4) by comparing the results of BDETT (Bahdan et al., 2018) and PopSAN (Song et al., 2018), the obvious performance improvement can be found under dynamic conditions but not under static conditions. This reveals that the dynamic thresholding scheme developed in BDETT (Bahdan et al., 2018) benefits much from the dynamic scenes but little from static conditions. By contrast, our method can significantly improve obstacle avoidance performance in both dynamic and static conditions. Furthermore, Figure 3 visualizes an example that clearly shows that the existing methods often fail around the right-angle turn while our method can achieve robust obstacle avoidance.
### Ablation Study

To analyze our EEM-SAN, we investigate (a) the importance of DVS event cues; (b) the effectiveness of HSVAE; (c) the effectiveness of MFDM-LT; and (d) the efficiency advantage of SNN over ANN. Table 3, 4, 5, and Figure 4, 5 summarize our findings.
settings. The reason behind this may be (i) the DVS modality generates larger amount of data and thus a higher threshold is needed to filter out the useless spikes and (ii) lower threshold in Laser layer enables more depth information to pass through to dominate the action decision at the startup phase of the robot when lots of DVS events noise would be generated to interfere with decision making. Another interesting finding is that the threshold values in the decision layers increase as the network goes deeper. This phenomenon can be explained as the modality-fused data can fire neurons in shallow layers easily to maintain sufficient information which would be filtered by deep-layer neurons strictly to extract most informative data for the final action decision. Furthermore, we fix the learnable thresholds in MFDM-LT during the training process (Figure 5 MFDM) and find that the performance degrades dramatically. And this can be a strong evidence to demonstrate the effectiveness of the learnable threshold scheme in our MFDM-LT.

_Efficiency Advantage of SNN over ANN._ Both HSVAE and MFDM-LT modules in our EEM-SAN are implemented by SNN, but not ANN. To show the advantage of such a choice, we compare the superior computational efficiency of our fully-SNN-based EEM-SAN against its ANN variants in terms of addition and multiplication FLOPs in Table 5. We can observe that, compared with the fully-SNN-based architecture (Table 5 (i)), much more computations are required when adopting ANN-based HSVAE (ii), ANN-based MFDM-LT (iii), or fully-ANN-based network (iv). For example, the ANN version of EEM-SAN (iv) is 4.63 and 11.66 times more expensive than the SNN version (i) in terms of addition and multiplication operations, respectively. Prior works [14; 17; 48] have demonstrated that the computing complexity of the network is positively correlated with inference speed and energy consumption, especially when the network is implemented in neuromorphic device [3; 35; 43]. Therefore, our fully-SNN-based EEM-SAN can achieve much faster inference with much lower energy consumption than its ANN counterpart.

### Limitations

In this work, we make the first attempt to connect multi-sensor representation and fully-SNN-based DRL towards robust and efficient robot control in extreme navigation scenarios with both static and fast-moving dynamic obstacles. Testing our method on different robot platforms (_e.g._, unmanned aerial vehicles) with more real challenging scenes (_e.g._, subterranean) would be a promising future work but out of focus of this work. Besides, our EEM-SAN has only been tested in a realistic simulator and has not been implemented on a real robot due to the unavailability of neuromorphic hardware. To conduct such an engineering verification, we are actively seeking permission from the Intel Neuromorphic Research Community (INRC) to use the neuromorphic chip Lohli. We will perform such an interesting verification once the hardware becomes available.

## 5. Conclusion

In this paper, we presented an event-enhanced multimodal spiking actor network (EEM-SAN) for autonomous navigation and dependable obstacle avoidance. Our solution is the first to introduce the Dynamic Vision Sensor (DVS) to provide motion cues that complement the traditional Laser depth data for handling dynamic obstacles. EEM-SAN consists of two main modules: a hybrid spiking variational autoencoder (HSVAE) which encodes the DVS event data through unsupervised representation learning, and a middle fuse decision module with learnable thresholding (MFDM-LT) designed for multimodal data fusion. Through extensive validation and ablation studies, we demonstrate the value of DVS event cues, as well as the effectiveness and robustness of our EEM-SAN. In the future, we plan to deploy our method on neuromorphic devices to maximize its advantages in terms of computational efficiency and energy consumption.

###### Acknowledgements.

 This work was supported in part by National Key Research and Development Program of China (2022ZD0210500), the National Natural Science Foundation of China under Grants 61972067/U21A20491, and the Distinguished Young Scholars Funding of Dalian (No. 2022RJ01). Ziqi Wei was supported by the open funding of State Key Laboratory of Structural Analysis for Industrial Equipment.

Figure 4. Comparison between learned thresholds in different MFDM-LT layers under different initial training thresholds.

Figure 5. Performance comparison between MFDM variants with and without learnable threshold scheme under different initial training thresholds settings.