Brain-inspired Evolutionary Architectures for Spiking Neural Networks
The complex and unique neural network topology of the human brain formed
through natural evolution enables it to perform multiple cognitive functions
simultaneously. Automated evolutionary mechanisms of biological network
structure inspire us to explore efficient architectural optimization for
Spiking Neural Networks (SNNs). Instead of manually designed fixed
architectures or hierarchical Network Architecture Search (NAS), this paper
evolves SNNs architecture by incorporating brain-inspired local modular
structure and global cross-module connectivity. Locally, the brain
region-inspired module consists of multiple neural motifs with excitatory and
inhibitory connections; Globally, we evolve free connections among modules,
including long-term cross-module feedforward and feedback connections. We
further introduce an efficient multi-objective evolutionary algorithm based on
a few-shot performance predictor, endowing SNNs with high performance,
efficiency and low energy consumption. Extensive experiments on static datasets
(CIFAR10, CIFAR100) and neuromorphic datasets (CIFAR10-DVS, DVS128-Gesture)
demonstrate that our proposed model boosts energy efficiency, archiving
consistent and remarkable performance. This work explores brain-inspired neural
architectures suitable for SNNs and also provides preliminary insights into the
evolutionary mechanisms of biological neural networks in the human brain.
Wenxuan Pan, Feifei Zhao, Zhuoya Zhao, Yi Zeng
# Brain-inspired Evolutionary Architectures for

###### Abstract

The complex and unique neural network topology of the human brain formed through natural evolution enables it to perform multiple cognitive functions simultaneously. Automated evolutionary mechanisms of biological network structure inspire us to explore efficient architectural optimization for Spiking Neural Networks (SNNs). Instead of manually designed fixed architectures or hierarchical Network Architecture Search (NAS), this paper evolves SNNs architecture by incorporating brain-inspired local modular structure and global cross-module connectivity. Locally, the brain region-inspired module consists of multiple neural motifs with excitatory and inhibitory connections; Globally, we evoke free connections among modules, including long-term cross-module feedforward and feedback connections. We further introduce an efficient multi-objective evolutionary algorithm based on a few-shot performance predictor, endowing SNNs with high performance, efficiency and low energy consumption. Extensive experiments on static datasets (CIFAR10, CIFAR100) and neuromorphic datasets (CIFAR10-DVS, DVS128-Gesture) demonstrate that our proposed model boosts energy efficiency, archiving consistent and remarkable performance. This work explores brain-inspired neural architectures suitable for SNNs and also provides preliminary insights into the evolutionary mechanisms of biological neural networks in the human brain.

 Evolutionary Neural Architecture Search, Spiking Neural Networks, Brain-inspired Module and Long-term Connection, Neural Circuit Motifs, Efficient Multi-objective Evolution

## I Introduction

After millions of years of evolution, neurons with complex information processing capabilities and microcircuits with specific functions have emerged in the human brain, empowering it to be a powerful device. As the third generation of neural network, Spiking Neural Network (SNN) [1] realizes a low energy consumption and high-efficiency computing paradigm by simulating the characteristics of neuron communication in the brain. However, existing SNN research employs fixed architectures that lack references to brain-inspired topological properties, limiting their performance on multiple tasks.

Neural Architecture Search (NAS) attempts to replace the experience-based manual design of architectures with an automated searching approach. Studies have shown that it even outperforms architectures designed with the expertise of human experts on some tasks [2, 3]. As a kind of optimization scheme in NAS, Evolutionary Neural Architecture Search (ENAS) is promising due to its insensitivity to local optimal values and without a large number of computing resources compared to reinforcement learning (RL)-based algorithms [4, 5, 6]. Based on the current understanding of the evolutionary structure of the brain, this paper is committed to exploring the evolution of brain-inspired local and global neural circuits for optimizing SNN architecture and function.

In ENAS, when the population and the number of iterations are large, it is very time-consuming to train SNN models one by one to evaluate the performance of individuals. Thus efficient evaluation methods are particularly important, and research ideas are usually inseparable from weight-sharing [7, 8], performance predictor [9, 10] or zero-shot methods [11, 12]. The weight-sharing method is also called the one-shot method, and only one supernet needs to be trained in the entire algorithm process. However, training to convergence takes a long time, and it may encounter the problem of performance collapse, which could not reflect real network fitness. Zero-cost evaluation methods require no training and have extremely low time costs, but only roughly measuring attributes positively related to architectural performance may also lead to inaccurate rankings of fitness. As a few-shot method, performance predictor trains a regression model for directly predicting the performance of unknown architectures based on the information of some trained architectures [9, 10, 13, 14, 15] and can be divided into online and offline according to whether the predictor is updated during the NAS process. Offline predictors only sample limited architectures once, and no new information is considered after construction. Therefore, the online predictor is more practical and flexible, able to continuously improve the evaluation accuracy by leveraging the existing knowledge [15, 16, 17].

A large amount of ENAS research is based on deep neural networks (DNNs), and existing works on evolving SNNs architecture are still very limited. [18] proposes a search model named AutoSNN to evolve more energy-efficient architecture. It defines an artificially formulated weight factor to optimize the number of spikes and classification accuracy, which is still essentially a single-objective optimization without manifesting the coordination of the two contradictory