Paper Title: SHA-CNN: Scalable Hierarchical Aware Convolutional Neural Network for Edge AI.







SHA-CNN: Scalable Hierarchical Aware Convolutional Neural Network for Edge AI
    Narendra Singh Dhakad, Graduate Student Member, IEEE, Yuvnish Malhotra, Santosh Kumar Vishvakarma, Senior Member, IEEE, Kaushik Roy, Fellow, IEEE

Narendra Singh Dhakad, Yuvnish Malhotra and Santosh Kumar Vishvakarma are with Nanoscale
Devices, VLSI Circuit and System Design (NSDCS), Department of Electrical Engineering, Indian Institute of Technology Indore, MP 453552, India.

Kaushik Roy is with the School of Electrical and Computer Engineering, Purdue University, West Lafayette, IN 47907 USA.

    August 5, 2024
=========================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================



§ ABSTRACT 
This paper introduces a Scalable Hierarchical Aware Convolutional Neural Network (SHA-CNN) model architecture for Edge AI applications. The proposed hierarchical CNN model is meticulously crafted to strike a balance between computational efficiency and accuracy, addressing the challenges posed by resource-constrained edge devices. SHA-CNN demonstrates its efficacy by achieving accuracy comparable to state-of-the-art hierarchical models while outperforming baseline models in accuracy metrics. The key innovation lies in the model's hierarchical awareness, enabling it to discern and prioritize relevant features at multiple levels of abstraction. The proposed architecture classifies data in a hierarchical manner, facilitating a nuanced understanding of complex features within the datasets. Moreover, SHA-CNN exhibits a remarkable capacity for scalability, allowing for the seamless incorporation of new classes. This flexibility is particularly advantageous in dynamic environments where the model needs to adapt to evolving datasets and accommodate additional classes without the need for extensive retraining. Testing has been conducted on the PYNQ Z2 FPGA board to validate the proposed model. The results achieved an accuracy of 99.34%, 83.35%, and 63.66% for MNIST, CIFAR-10, and CIFAR-100 datasets, respectively. For CIFAR-100 our proposed architecture perform hierarchical classification with 10% reduced computation while compromising only 0.7% of accuracy with the state-of-the-art. The adaptability of SHA-CNN to FPGA architecture underscores its potential for deployment in edge devices, where computational resources are limited. The SHA-CNN framework thus emerges as a promising advancement in the intersection of hierarchical CNNs, scalability, and FPGA-based Edge AI.



Convolutional neural networks, hierarchical CNN, incremental learning, scalable neural networks, edge AI




§ INTRODUCTION

Convolutional Neural Networks (CNNs) have been widely used in computer vision applications such as image classification, object detection, and segmentation <cit.>. However, as the size and complexity of datasets continue to grow, there is a need for more scalable and efficient CNN architectures that can handle large amounts of data while maintaining high accuracy. One approach to addressing this challenge is to use hierarchical CNN architectures consisting of multiple feature extraction and abstraction levels. Hierarchical CNNs have been shown to be effective in capturing complex visual patterns and improving the generalization performance of CNNs. However, existing hierarchical CNN architectures <cit.> often suffer from scalability issues, such as high computational and memory requirements, which limit their applicability to large-scale datasets. Therefore, there is a need for a scalable hierarchical CNN architecture that can handle large amounts of data while maintaining high accuracy. Similar work has been proposed <cit.>, where they addressed the catastrophic forgetting concept to learn the new classes.



Conventional CNNs are trained as a flat N-way classifier, meaning all the dataset classes are considered equally difficult to distinguish from each other and exclusive. However, some classes may be more difficult to distinguish than others as they may depend on higher hierarchy levels. For example, an apple and an orange belong to a common superclass of fruits and have some similarities. An apple and a zebra are easily distinguishable, but an apple and an orange are relatively difficult to distinguish. From a human perspective, if a human is given an image of an apple and an orange, the brain will first classify both as fruits and then the classification will go deeper to specific fruit for each image. More examples of such class dependencies from ImageNet dataset <cit.> are shown in Fig. <ref>. An example of a hierarchical tree based on class dependencies for different modes of transportation is shown in Fig. <ref>. Such an approach of classification following the tree structure going from superclasses to subclasses reduces the region of interest of classification in the final step, indeed improving the accuracy. NNs fail to encode such hierarchy or dependencies in classes into their architecture in their conventional form. This is where the concept of hierarchical classification comes into the picture. 



Scalability is a significant challenge in CNNs as the size and complexity of the input images and the number of parameters in the model can quickly become very large, making training and inference computationally expensive and memory-intensive. One main factor contributing to scalability issues in CNNs is the number of convolutional layers and filters in the model. Increasing the number of layers and filters can improve the model's accuracy but also increase the computational cost and memory requirements. This can limit the size of the input images or the batch size during training and make it challenging to deploy the model on resource-constrained devices or in real-time applications. For the N level of hierarchical classification, we need to add N branches, which improves the computational parameters linearly. 

We proposed a modified CNN model for scalable and hierarchical-aware CNN to address the issue of learning new classes and classification in a hierarchical approach. The motivation behind scalable hierarchical-aware CNN is to address the scalability challenge in CNNs by designing efficient, compact, and adaptable models to different input image sizes and resolutions. Traditional CNNs use fixed-size convolutional filters and feature maps, which can limit their ability to generalize to images with different sizes or aspect ratios. This can be especially problematic for object detection and semantic segmentation applications, where the input images can vary in size and aspect ratio. Scalable hierarchical-aware CNNs aim to overcome these limitations by incorporating multi-scale and hierarchical representations in the model architecture. This allows the model to process images at multiple resolutions and scales, capturing both fine-grained and coarse-grained visual patterns in the input images. 

The contribution of the paper is as follows: 

    
  * Modified a hierarchical CNN architecture that can effectively capture complex visual patterns while minimizing computations, memory requirements and delay for inference.
    
  * The proposed model architecture supports the scalability if the number of classes increases. 
    
  * Evaluate the scalability and performance of the proposed approach on large-scale datasets and compare it with existing state-of-the-art methods.
    
  * The proposed model architecture has been deployed to the PYNQ Z2 FPGA board and performance has been evaluated for MNIST, CIFAR-10, CIFAR-100 datasets.
    


The rest of the paper is organized as follows. The related work on hierarchical and incremental learning is discussed in Section II. In Section III, we present our proposed network architecture. In Section IV, the experimental performance and hardware demonstration of the model. Finally, Section V concludes the work.








§ BACKGROUND AND RELATED WORK

Conventional neural networks are considered an N-way classifier with independent final categories. However, a few classes may be more difficult to distinguish from one another than the others as they may depend on a higher level of hierarchy. There are numerous works have been proposed to address hierarchical learning <cit.>, while only a few works address the issue of scalability <cit.>. 

B-CNN <cit.> is designed to leverage the hierarchical structure of the classification problem. It consists of two branches of convolutional neural networks: one for extracting fine-grained features and the other for extracting coarse-grained features. The fine-grained branch is designed to capture subtle details and local features, while the coarse-grained branch captures more general and global features. The two branches are combined to form a joint feature representation fed into a hierarchical classification classifier. B-CNN uses the fine-grained branch to refine the feature representation based on the subclass label and the coarse-grained branch to refine the feature representation based on the superclass label. This approach allows the network to learn a discriminative feature representation for fine-grained and coarse-grained categories, improving the accuracy of hierarchical classification tasks. However, the work bottleneck is that if the number of levels increases in the hierarchy, the computation cost increases linearly, which is inefficient for edge devices.

HD-CNN <cit.> presents a hierarchical deep convolutional neural network architecture for large-scale visual recognition tasks. The proposed architecture consists of multiple levels of feature extraction, where each level refines the input data representation. This approach improves the accuracy of recognition tasks, allowing for more complex and discriminative features to be extracted.

The article <cit.> suggests that encoding hierarchical information in neural networks can help deal with subpopulation shift. This problem arises when the training and test data come from different subpopulations. The work proposes a new neural network architecture called HiGCN (Hierarchical Graph Convolutional Network) that utilizes the hierarchical structure of the data to improve performance on subpopulation shift tasks. The HiGCN model uses a graph-based approach to represent the hierarchical structure of the data, with nodes representing different levels of abstraction in the hierarchy. They demonstrate the effectiveness of the hierarchical graph structure in capturing and utilizing the hierarchical information in the data. 



Condition-CNN <cit.> proposed a hierarchical multi-label image classification model specifically designed for fashion images. It aims to classify images into multiple labels, such as the type of clothing, its style, color, and pattern. This utilizes a hierarchical approach, where the coarse label classifier acts as a gating mechanism for the fine label classifier. The fine label classifier is only activated if the coarse label classifier predicts the correct general category for the clothing item. This approach reduces the search space for the fine-label classifier, resulting in a more efficient and accurate classification process. 

CF-CNN <cit.> proposed a coarse-to-fine convolutional neural network (CF-CNN) for learning multilabel classes, which is a hierarchical learning method that creates a class group with the hierarchical association and assigns a new label to each class belonging to the group so that each class acquires multiple labels. CF-CNN is a three-layer network with a main coarse layer, a main fine layer, and a refined layer. 



Tree-CNN <cit.> explores the challenges of incorporating new data into fixed Deep CNNs due to catastrophic forgetting and sensitivity to hyperparameter tuning. The paper proposes an adaptive hierarchical network structure composed of DCNNs that can grow and learn as new data becomes available, organizing the incrementally available data into feature-driven superclasses. The paper compares this hierarchical model against fine-tuning a deep network and demonstrates a significant reduction of training effort while maintaining competitive accuracy. This classifier is composed of multiple nodes connected in a tree-like manner, where each node has a DCNN trained to classify the input into one of its children. The network adds branch nodes as new classes get grouped under the same output nodes. The work discusses the constraints applied to the algorithm, including the maximum depth of the tree and the maximum number of child nodes for a branch node. The hierarchical node-based learning model of the Tree-CNN attempts to confine the model change to only a few nodes, limiting the computation costs of retraining. 

The work in <cit.> discusses the impact of hierarchical structure on CNNs and how it affects their accuracy in image classification. They proposed a visual analytics method, which allows users to analyze the similarities between classes and the structure of the feature detectors learned by the CNNs in relation to the class hierarchy. The tool offers a hierarchy viewer and a seriation algorithm to reveal block patterns in a confusion matrix. Additionally, the tool helps diagnose various quality issues in the data and improve the curation of training datasets. They propose hierarchy-aware CNN architectures that yield significant gains in classification accuracy and convergence speed. 

An approach to hierarchical multi-label classification that uses a neural network to predict all categorizations of an input source, exploiting multiple layers of a neural model is presented in <cit.>. The approach is an architectural extension that can be adapted to a generic neural network using deep stack layers. The experiments demonstrate the ability of the hierarchical approach to recognize the correct categories of a class hierarchy and when a classification model fails to recognize the correct class. 

HierarchyNet <cit.> proposes an approach to a multi-label hierarchical classification of buildings that uses fewer parameters than existing hierarchical networks and achieves more accurate classification. The proposed model can extract features and classify buildings based on functional purpose and architectural style using a coarse-to-fine hierarchical structure with a custom multiplicative layer. The model is based on the B-CNN <cit.>, which is scalable for two-level hierarchical classification but not for more levels. 

















§ PROPOSED MODEL DESCRIPTION

In our proposed work, we considered the model presented in B-CNN <cit.> as a reference. The model architecture B-CNN is built upon one important characteristic of CNNs and their architecture - lower layers of CNNs capture low-level features like edges and corners. In comparison, deeper layers extract higher-level features like faces, eyes etc. These deeper layers are built on top of lower levels, meaning higher-level features are extracted from lower-level features. This completely aligns with the hierarchical classification required. B-CNN model leverages this property of CNN by adding branches along the architecture flow, each branch making predictions at each level of the label tree. On top of each branch, fully connected layers and a softmax layer are used to produce the output in a one-hot encoding fashion. The branches may contain convolutional and pooling layers as well. However, in the experimentation of B-CNN models, branches only contained fully connected layers. As the number of levels increases in the label tree, the number of branches increases, increasing the number of computational parameters in the network. This makes scaling difficult, especially for edge devices with limited computational resources. Our proposed model architecture and training method will make the final model scalable at inference. The procedure includes two steps: (i) B-CNN training and trained weights extraction, and (ii) combining fully connected layers of each branch and using a single fully connected set for prediction at each level. The details of each step are mentioned in the following sub-sections. 



 §.§ B-CNN Training and Weights Extraction

CNNs are made of two components: the feature extraction component, which consists of convolutional and pooling layers, and the classification component, which consists of fully connected layers for prediction based on features extracted by the feature extraction component. B-CNN models are trained using the Branch Training Strategy - the loss weights are changed after specific epochs so that the training focus stays at the specific branch for some epochs and then the focus changes to the next branch. This helps the Feature Extraction component of CNN to learn weights capable of extracting features specific to a specific level of the label tree. So in our procedure, firstly, the B-CNN model in trained on the dataset and the label tree. One change from the B-CNN model is adding a dense layer to each branch. This additional dense layer will have same number of hidden units in all the branches. Weights of convolutional layers and the additional dense layer of each branch are saved. These weights will be used in the second step. Model architecture for weight extraction is shown in Fig. <ref>.



 §.§ Shared Fully Connected Layers at all levels

Fully connected layers can be considered a mathematical function followed by an activation function mapping features to final predictions. The idea behind shared fully connected layers at each level prediction is to have a mathematical function with weights and parameters learned to predict at all levels of the label tree. The model architecture for this step is shown in Fig. 5. As can be seen, the softmax layer for each branch is kept separate, as there may be a different number of labels at each level of the label tree. While training the model, the layers for which the weights were extracted from the model from the previous step are kept frozen, meaning the weights of these layers are not trained in this step. Only the shared fully connected layers and the softmax layer of each branch are trained at a low learning rate. This model will be used at inference once trained. Having shared fully connected layers in the network will reduce the number of parameters in the model at inference which in turn will make the model more scalable but also, less storage space will be required for storing the model weights. 















§ EXPERIMENTAL RESULTS

Three datasets were used to experiment and validate the concepts MNIST, CIFAR-10 and CIFAR-100. The label trees for the MNIST and CIFAR-10 were taken from B-CNN <cit.> while for CIFAR-100, the label tree was created manually. The label trees for MNIST and CIFAR-10 are shown in Fig. <ref>. Test Set accuracy and catastrophic distance were used to compare model performances as shown in Table <ref>. The accuracy mentioned here is the top-1 accuracy of the model during inference. Catastrophic distance is the graphical distance between the true label and the predicted label of the data sample in the label tree as shown in Fig. <ref>. Graphical distance between the true and predicted class is calculated for each data sample in the test set and the average distance is reported. Fig. <ref> shows some sample images and their prediction at different levels. All the models were trained using Google Colab, and Python and Keras were used to write the codes. The main aim of this experiment is not to outperform the state-of-the-art models but to achieve scalability at inference, keeping in mind the applications of these models in edge devices. To achieve scalability, a compromise has to be made with accuracy. However, the proposed model performance is still comparable with state-of-the-art models while being the most scalable among all with respect to computational cost. The hardware implementation for the proposed model is performed using a PYNQ Z2 board, a camera is interfaced with the board, and the model is deployed on the board as shown in Fig. <ref>. Fig. <ref> compares the computational parameters count for B-CNN <cit.> and our proposed SHA-CNN model architecture. Here, it can be observed that as the number of classes increases, our proposed architecture uses fewer computations to achieve comparable accuracy. Fig. <ref>(a) shows the overall MAC count for the baseline VGG16 model, B-CNN model, and our proposed SHA-CNN model science. For hierarchical classification (B-CNN and SHA-CNN), the MAC counts are higher than the baseline; however, the proposed SHA-CNN has 10% fewer MACs than B-CNN, while the accuracy of our proposed architecture is compromised with only 0.74% for CIFAR-100 dataset. The number it requires fewer resources when deployed on edge devices. Fig. <ref>(b) shows the MACs count for level wise classification of CIFAR-100 dataset. Here we have classified the data into three level. In the first level it has 8 classes, in second level it has 20 classes and at the third level it has 100 classes. As can be seen from the Fig. <ref>(a) for each level of classification the MACs count is less for our proposed model as compared to the B-CNN. As the hierarchical level increases the MACs count decreases significantly.   







§ CONCLUSION

This paper introduces a Scalable Hierarchical Aware Convolutional Neural Network (SHA-CNN) tailored for Edge AI applications. The model strikes a delicate balance between computational efficiency and accuracy, showcasing its superiority over baseline models and comparability with state-of-the-art hierarchical architectures. The innovation lies in the model's hierarchical awareness, enabling nuanced feature extraction at multiple abstraction levels. The hierarchical classification approach enhances the model's interpretability, allowing for a more granular understanding of complex datasets. Notably, SHA-CNN exhibits scalability, effortlessly accommodating new classes without extensive retraining. This adaptability is advantageous in dynamic edge environments where datasets evolve. Validation on the PYNQ Z2 FPGA board demonstrates the practical feasibility of deploying SHA-CNN in resource-constrained edge devices. SHA-CNN shows 99.34%, 83.35%, and 63.66% accuracy for MNIST, CIFAR-10, and CIFAR-100 datasets, respectively. 







IEEEtran
 

 



 Paper Title: How quantum and evolutionary algorithms can help each other: two examples.




[
    M. Amato^*
    August 5, 2024
==================

empty






§ INTRODUCTION


Quantum computing utilizes principles of quantum mechanics to perform certain computations faster and on/and larger scales than classical computers are capable of <cit.>. 
Quantum bits, or qubits, are key building blocks of quantum computers that enable the latter to be so advantageous in solving complex problems  <cit.>. 
The intersection of quantum computing and artificial intelligence (AI) is currently an area of active research that has the potential to revolutionize various fields and industry sectors. 
Quantum Machine Learning (ML) algorithms and, in particular, evolutionary algorithms, belong to one of the research directions that are actively explored now <cit.>. It is expected that the new results will help to speed up the optimization process further and improve the overall performance of computational algorithms. In particular, Evolutionary Quantum Computation (EQC) is an emerging approach that blends evolutionary algorithms and quantum computing to solve complex optimization problems <cit.>. 


The aim of this work is twofold. First, we would like to show how quantum algorithms can be used to solve specific problems related to the engineering of evolutionary algorithms. Next, in a complimentary manner, we want to illustrate how evolutionary algorithms can be used to design quantum circuits fulfilling specific requirements.
In particular, we will address the problem of designing quantum circuits that
process initial product states into
highly entangled states.

 
Entanglement, one of the key concepts of quantum physics, also plays a crucial role in quantum information processing <cit.>. The term 'entangled state' refers to a state of several quantum objects that are intrinsically correlated with each other, in a way different from all possible classical correlations <cit.>. F.e.,  when two qubits are entangled, changes in the state of one qubit affect the state of the other one. There are several measures proposed to quantify the strength of these essentially quantum correlations, and so a quantitative definition of the word 'highly,' strictly speaking, depends on the chosen measure. Highly entangled quantum systems are usually difficult to simulate on classical computers <cit.>, and this fact has significant implications for quantum information processing technologies. Metaphorically speaking, highly entangled qubit states serve as fuel to quantum information processing and without them, the latter loses its superiority with respect to the classical processing <cit.>.
To create a highly entangled state out of a non-entangled one is not an easy task. And even if we have managed to prepare a maximally entangled state, it can become less entangled just because the quantum system prepared in this state is interacting with the environment <cit.>. All real quantum systems suffer from this interaction whose action is conventionally described with the term "decoherence". Therefore, it is important to keep the time during which decoherence strikes the system as short as possible. Practically, this means to minimize the computation time. In addition, every time a quantum gate (that is a transformation of the state of a system of qubits) is implemented, an additional error occurs due to the non-ideal character of the implementation. After passing through many gates, the state
is no longer reliable in the sense that it is very different from what it has to be according to the corresponding quantum program. Thus, the more complicated the quantum circuit is (in terms of the number of gates it consists of), the stronger the joint action of decoherence and accumulated implementation errors <cit.>. As has been mentioned above, not all entangled states are equal in their 'strength'. This means that we need to quantify and measure the amount of entanglement between different qubits. This brings another challenge, because, in a generic quantum system, the quantification of entanglement is a non-trivial task  <cit.>.


Our study addresses two specific problems, which are discussed in Section <ref>. The first problem, addressed in  Subsection <ref>, involves realizing classical stochastic cellular automata using quantum circuits. This approach is innovative as it leverages quantum mechanics to enhance the modeling of complex systems, a topic of extensive interest 

in prior research <cit.>. The second problem, discussed in Subsection <ref>, focuses on designing quantum circuits that generate highly entangled multi-qubit states. This is, up to our knowledge, a new application of evolutionary algorithms: to optimize entanglement in quantum circuits. To this end we use the Meyer-Wallach entanglement measure as the fitness function. 

In Section <ref> we summarize the results and discuss further perspective. Finally, in Section <ref>, we describe the methods used and highlight the relevant concepts.











§ RESULTS



 §.§ Realizing stochastic cellular automata with quantum circuits






In this part, we use an evolutionary algorithm to realize cellular automata (CA) for specific rules, deterministic and stochastic.
The CAs are realized with quantum 
circuits and measurements. The circuits are constructed by assembling gates from a fixed set of five gates.  Our approach involves a mutation-based evolutionary algorithm, enabling the optimization of gate types and their placements.

Initially, a predetermined number of chromosomes is generated, corresponding to the number of quantum gates per circuit. Through iterative refinement, the genetic algorithm evolves the existing chromosomes into new variations, singling out the fittest chromosome within each generation to serve as the parent for the subsequent generation. The assessment of the performance of the derived quantum circuits revolves around the measurement of the first qubit,  q_0. This assessment entails comparing the measured probability of an initial state, computed from the population, with the corresponding target probability. To quantify the difference between two sets of probability distributions,  we use the Kullback-Leibler divergence (KL) as a fitness function. The KL-fitness score measures the discrepancy between the distributions of the output states and the target states. In our case, these distributions are discrete, each corresponding to the eight different initial states of the three qubits. If the distributions completely match, the fitness function value is zero. A decreasing fitness value close to zero indicates an increase in the efficiency of the circuits.




The resulting quantum circuits are evaluated through measurements, thereby demonstrating the algorithm's efficacy. For more details, see Supplementary Note I.

    







We consider three different types of quantum CAs, starting with a stochastic critical cellular automaton, followed by more general stochastic CA (random updates) and a few deterministic rules, namely, rule 90 and 110  <cit.>. The updates for each type of CA are shown in Table <ref>. To enhance the robustness of our method, we limit the number of chromosomes by  N_c=20 chromosomes and the number of generations by N_g=500. Simulations are then performed for N_ic=50 randomly chosen initial conditions. This approach builds upon our previous experiment, where we set N_c=20 chromosomes, N_g=150 generations, and N_ic=20 initial conditions were used <cit.>. 













The authors in <cit.> have evolved a stochastic cellular automata model to reach criticality, a property of dynamical systems that allows them to do robust computations.
For each triad pattern, a probability has been calculated through a genetic algorithm for the central cell to have state 1. These probabilities are shown in Table <ref>, second column. 

We start by considering the number of gates used, evolving quantum CAs with 3, 5, 10, 15, and 20 gates and 50 runs. The mutation probability is fixed at 10 percent. The results are presented in Figure <ref>. They show that the best fitness score per run improves with increasing the number of gates up to 15 gates, after which a gradual increase is observed until the limit of 20 gates. Therefore, for experiments 2 and 3, 15 gates are used. 
Furthermore, the results of experiments indicate that, while fitness scores initially improve with an increasing number of gates, the rate of progress diminishes as the number of gates continues to increase. 


Next, we explore how the fitness changes with the value of the mutation probability. The goal here is to test the impact of the mutation on the fitness function of the different generations. We fixed the number of gates to 15 and used different mutation probabilities,  5%, 10%, 20%, 30%, and 50%.

Figure <ref> illustrates the impact of different mutation rates on the KL-fitness score. The left panel of figure <ref> shows that all mutation rates result in a decrease in fitness scores over generations, indicating improvement. However, mutation rates of 5%, 10%, and 20% stabilize more consistently compared to higher rates. The right panel box plots show that while higher mutation rates (30%, 50%) lead to decreased fitness scores, they introduce greater variability and less stability. These results highlight that while higher mutation rates can still improve circuit efficiency, the optimal balance between exploration and stability is achieved at a 20% mutation rate.



We now test the above approach with generic deterministic and stochastic cellular CAs. We fixed parameters, by utilizing 15 gates and a 10% mutation probability. Specifically, we examine the application of our methodology to Rule 90 and Rule 110 (deterministic rules with fixed probabilities of 0 or 1 for the eight neighborhoods), as well as to eight randomly generated probabilities, each repeated three times, serving as target probabilities. The corresponding target probabilities for each experimental condition are detailed in   Table <ref>, while the ensuing outcomes are visually represented in Figure <ref>. This comprehensive analysis showcases the versatility and effectiveness of our approach across a spectrum of rule types and probabilistic settings.


The fitness scores obtained for the deterministic CAs, Rule 90 and Rule 110, are remarkably good, as shown in Figure <ref> (left). This highlights the efficiency of our approach to the synthesis of quantum circuits for deterministic CAs. Notably, the outcomes stemming from the stochastic CA approach, characterized by the incorporation of randomly generated probabilities over three separate iterations, exhibit some promise. The optimal fitness scores attained for Random1, Random2, and Random3 are 0.3364, 0.1681, and 0.0014, respectively. It is worth mentioning that while Random1 achieves a commendable fitness score of 0.3364, indicating its promising performance, there is potential for further refinement of the evolutionary process to reduce any existing disparities. A comprehensive evaluation of the fitness scores across different probability sets, as presented in Figure <ref> (right), to the number of generations, provides valuable insights into the performance trends for the deterministic CA models (Rule 90 and Rule 110), as well as the stochastic CA model utilizing randomly generated probabilities over three iterations (Random1, Random2, and Random3), all conducted with an initial set of 50 run conditions.

Figure <ref> (Left) illustrates the fitness scores corresponding to the diverse CA rules. The best fitness score of 0.294 was achieved by the critical CA. Notably, for two deterministic CA rules, namely Rule 90 and Rule 110, the fitness scores approximate zero, reflecting a deliberate pursuit of minimizing the absolute disparity between initial and final states. This trend aligns logically with the deterministic nature of these CA rules, where probability values exclusively assume binary states of 0 or 1. In contrast, the scenario changes when dealing with randomly generated stochastic CA rules. In this context, the fitness scores demonstrate variability across three distinct randomly assigned probabilities. Intriguingly, among these variations, Random 3 emerges as the front-runner, boasting the most favorable fitness scores. Remarkably, all three sets of randomly generated probabilities exhibit fitness values superior to that of the critical stochastic CA value. Nevertheless, to unravel the underlying mechanisms driving these observed phenomena, a comprehensive exploration beckons. This entails conducting a more extensive array of experiments, encompassing increased generations and a higher replication count. The intricate interplay of factors contributing to fitness score disparities between distinct randomly generated probabilities necessitates a more exhaustive investigation.
 
In Figure <ref> (Right), we show the circuit 
that significantly outperforms the average outcomes of runs associated with the critical stochastic cellular automaton. Impressively, this particular realization achieved a fitness score of 0.294, as evaluated by the KL fitness function outlined in Equation (<ref>). 





 §.§ Designing entangling quantum circuits



In this section, we present the results obtained with an evolutionary algorithm as a tool to create a circuit 
that entangles qubits. The strength of the entanglement is quantified with the Mayer-Wallach measure which plays the role of a fitness function. We consider systems of 3, 4, and 5 qubits, and the performance of the algorithm is explored by varying mutation rates and numbers of gates. We ran experiments for 500 generations, repeating it 50 times. The outcomes are shown as the average fitness along with the standard error, and the best fitness achieved, also with its standard error. 






  §.§.§ Three-qubit circuits

We start with a three-qubit case. 

Figure <ref> presents the evolutionary optimization of a three-qubit quantum circuit across 500 generations, highlighting both average and peak fitness scores. We use three different mutation rates, 5%, 10%, and 15%, and limit the number of gates in the circuits to  3, 5, 8, 10, and 12. The best fitness scores obtained for all mutation rates approached unity, indicating that the resulting circuits are indeed highly entangling. 

Since the best fitness scores for the quantum circuits approached unity, evolutionary algorithms indeed serve as a toolbox of methods to design highly entangling three-qubit circuits. For a three-qubit system (three lists of integer gates for each qubit), a mutation probability of 1/9 (11.11%) is needed to replace at least one individual population consisting of nine integer lists. Therefore, a 10% mutation rate, which can at least replace one individual population in each generation can be considered an optimal probability for the mutation rate. Experimentally, the optimal mutation rate was found to be 10%, which struck a balance between exploring different solutions in the search space and exploiting the best solutions as shown in Figure <ref> (Top: right). In addition, a lower mutation rate is capable of striking a balance between exploration and exploitation, leading to a higher average fitness score across all runs. However, the mutation rate of 10% is more effective in building and generating the circuit that performs best in each run due to the higher degree of exploration it allows for the entanglement.

We also varied the of gates in the quantum circuit in the range from 3 to 12 keeping the optimal mutation rate at 10%. The result is the fitness score decreased as the number of gates in the circuit increased as shown in Figure <ref> (Bottom: Left). These results suggest that as the quantum circuit becomes more complex with an increasing number of gates, the entanglement capability of the circuit is reduced, indicating that the circuit is less efficient at performing the desired task. This suggests that there may be a trade-off between circuit complexity and performance. Therefore choosing the optimal number of gates can provide the best balance between the complexity of the quantum circuit and the performance.

The best three-qubit entangling circuit with three gates is shown in Figure  <ref> (Bottom: right). It corresponds to the fitness score of 0.999. Similarly, a three-qubit quantum circuit with 12 gates also achieved a fitness value of 0.999. In Supplementary Note II, the detailed analyses of the state vector and reduced density matrices from optimal circuits in a 500-generation are presented. Table 1 in Supplementary Note shows these details for the state vector probability of the respective density matrices. The methodologies used for calculating the probabilities of quantum states and the reduced density matrices for the three-qubit circuits, along with an assessment of their entanglement using the Mayer-Wallach entanglement measure, are given in Supplementary Note III.





  §.§.§ Four and five-qubit circuits




In this section, we extended the designing of quantum circuits to advance our findings to include both four and five-qubit systems. The findings are shown in Figures <ref> and <ref> for four and five qubit quantum circuits, respectively. Notably, Figures (Top: Left) <ref> and <ref> show the mean fitness (illustrated by a green line) alongside its standard error (denoted by a shaded area), in addition to the average of the best fitness (represented by a red line) and its standard error (also shaded). A third-order polynomial fit applied to the mean fitness is shown by a blue dashed line. These figures representations demonstrate that the average fitness score approximates 0.6, while the mean of the highest fitness score approaches 0.8.

Further experiment with an optimal mutation probability of 10% and 5% for four and five qubit circuits, respectively, was conducted focusing on circuits with varying numbers of gates as shown in Figures (Top: right) <ref> and <ref>. Through an analysis of mutation rates of 3%, 5%, and 15%, we found that increasing the mutation rate beyond 5% results in a decrease in the average fitness value. This indicates that excessive randomness in the evolutionary process can have a negative impact on circuit performance. However, it was also observed that the average of the best fitness value was not affected by higher mutation rates. This highlights the importance of selecting an appropriate mutation rate for obtaining the quantum circuits of maximal entanglement.

The visualization of the quantum circuit with four and five qubits incorporating four and five gates, respectively, achieving a fitness score of 0.999, is shown in Figures (Bottom) <ref> and <ref>. Additionally, the five-qubit quantum circuit with 12 gates yielded a fitness score of 0.199, signifying a decrement in entanglement capabilities with an increase in the number of gates. Detailed outcomes for the most entangled four and five-qubit quantum circuits, including the state vector and reduced density matrices, are elaborated in Section II of the Supplementary Note. These results pertain to the optimal five-qubit circuit identified across 500 generations, encompassing circuits with both five and twelve gates, as illustrated in Tables 2 and 3 (Section II of the Supplementary Note).











Figure <ref> (Top: left) presents the results obtained. The plot shows the mean fitness (green line) and its shaded standard error, as well as the mean of the best fitness (red line) and its shaded standard error, against the number of generations. The blue dashed line represents the third-order polynomial fit to the mean fitness. From the figure it is clear that the mean fitness score is near 0.6, while the mean of the best fitness score is close to 0.8. The results indicates that circuits generated with the best fitness score have a very high degree of entanglement among the qubits in each generation of the evolutionary process.

The experiment with an optimal muttation probability of 10% for four-qubit circuits was repeated for circuits with varying numbers of gates as shown in Figure <ref> (Top: right).  It clear from the figure that with increasing number of gates in quantum circuits, the fitness score decreases. It is because increasing the number of gates in quantum circuits increases complexity of the circuit and hence drop of entanglement capability of the circuits. Figure <ref> (Bottom) presents a four-qubit circuit with four gates and a fitness score 0.9996.

The reduced density matrix (ρ_k) of a generated quantum circuit with state vector and a specific number of qubits (n-qubits) is computed by tracing out all qubits except for the k^th qubit. For four qubit system, k ranges over 0, 1, 2, 3. The resulting output is four 2 × 2 reduced density matrices. The calculated reduced density matrix and the respective state vector of four qubit quantum circuit generated bu the evolutionary algorithms are shown in Section II of the Supplementary Note, specifically to Table 2. In additional, the details of the mathematical formulation of the reduced density matrix and its role as an analytical tool in evaluating the entangled states is provided in Section III of the Supplementary Note.






  §.§.§ Five-qubit circuits




Figure <ref> (Top: Left) shows the results obtained for five-qubit circuits with five gates. The plot shows the average fitness, third-order polynomial fit to the average fitness, and the best fitness values. For five qubit system also, we explored the impact of different mutation rates (3%, 5%, and 15%) on the generated quantum circuits. The best fitness scores obtained for the optimal mutation rate of 3% were close to 0.8, with an target fitness value of one. This indicate a high degree of entanglement between the qubits for a system of five qubit quantum circuits. Also, the third-order polynomial fit to the average fitness shows a smooth curve that approximates the general trend of the average fitness scores.

Among the explored mutation rate of 3%, 5%, and 15%, we found that increasing the mutation rate beyond 5% results in a decrease in the average fitness value. This  indicates that excessive randomness in the evolutionary process can have a negative impact on circuit performance. However, it was also observed that the average of the best fitness value was not affected by higher mutation rates. This highlight the importance of selecting an appropriate mutation rate for obtaining the quantum circuits of maximal entanglement. Therefore, to determine the optimal mutation rate, we calculated the probability of replacing at least one individual population, consisting of 15 integer lists (three lists of integer gates for each of the 5 qubits). The probability of this occurring is 6.67%. So, mutation rate of 5% (close to 6.67%) would be an optimal and it would be sufficient to replace at least one individual population in each generation.

Next, we keep the mutation rate constant at 5% (optimal to replace at least one individual in each generation) and vary the number of gates. The results, as shown in Figure <ref> (Top: right), indicate that the fitness score is maximal or close to 1 for the 3-gate 5-qubit circuit. However, as the number of gates increases, the fitness scores decrease continuously resulting in a reduction of the entangling power of the circuits. This means that  a simpler quantum circuit with fewer gates can have better entanglement capability than a more complex circuit with higher number of gates.

The visualization of five-qubit quantum circuit consisting of five gates with a fitness score of 0.9996 is shown in Figure <ref> (Bottom). Also, the five-qubit quantum circuit with 12 gates yielded a fitness score of 0.199, indicating that increasing gates in a circuit decreases in entanglement capability. Section II of the Supplementary Note presents detailed results for the best entangled 5-qubit quantum circuit, including state vector and reduced density matrices. This results are from the best or optimal five qubit circuit among 500 generations, featuring circuits with both five and twelve gates (see Table 3).






  §.§.§ The von Neumann Entropy as a Fitness Function


In our experimental setup, we also incorporated von Neumann entropy within an evolutionary algorithm framework. The algorithm was configured for a three-qubit system equipped with ten gates, operating at a mutation rate of 10%. This procedure was iteratively performed across 50 cycles, spanning 500 generations, with each generation consisting of 20 chromosomes. The results of this implementation, illustrating the effectiveness of von Neumann entropy in the optimization process, are graphically represented in Figure <ref>.

The theoretical maximum entropy for such a system is 2.999. Nevertheless, when implementing von Neumann entropy entanglement as a fitness function in the evolutionary algorithm, we obtained a maximum entropy value of 1.05557. This result has significant implications for the entanglement within the system. The quantification of entanglement can be achieved through the computation of von Neumann entropy for the reduced density matrix of any given subsystem. 

Our findings indicate that the implementation of von Neumann entropy in the three-qubit system successfully detected the presence of entanglement within the evolved three-qubit quantum circuit. This is consistent with the understanding that the entropy of entanglement is the von Neumann entropy of the reduced density matrix for any of the subsystems.







§ DISCUSSION AND CONCLUSIONS




In this study, we explore the two examples of evolutionary algorithms in two distinct yet interconnected areas of quantum computing. Initially, we introduce a framework for generating quantum circuits that replicate specific cellular automata rules using genetic algorithms <cit.>. The implementation code is available at GitHub repository <https://github.com/Overskott/Quevo>. This framework effectively evolves quantum circuits for various CA rules, from deterministic to stochastic types. We improved our original framework by increasing the number of generations and runs, enhancing performance and more consistent fitness function outcomes. 

This initial investigation set the stage for more nuanced applications of evolutionary algorithms in quantum circuit design, suggesting future research paths.

To conclude,
building upon this groundwork, we have demonstrated the effectiveness of evolutionary algorithms in generating highly entangled quantum circuits, emphasizing their role in calculating reduced density matrices for three-, four-, and five-qubit systems. These circuits displayed a significant degree of entanglement, measured by the Mayer-Wallach entanglement measure. By tracing out all other qubits, the study illustrated how reduced density matrices could reveal detailed information about the circuit's entanglement. In this context, the algorithm's role was to optimize quantum gates and their connectivity, leading to the creation of quantum circuits. 

This contribution is, as far as we have checked, novel in its approach to optimizing quantum circuits for entanglement.
Moreover, the simulator provided by Qiskit enabled the evaluation of these circuits, gates, and connections, showcasing the potential of this approach in solving complex entanglement optimization problems. Furthermore, we highlighted the importance of circuit depth in determining the entanglement capability of a quantum circuit, and we found that increasing the circuit depth decreases its entanglement capability. 

Another important conclusion is that, an optimal mutation rate was identified for quantum circuits with different qubits and gates, marking a significant stride in optimizing quantum circuits for entanglement generation.
To note that the KL-fitness score measures the discrepancy between the distributions of the output states and the target states. In our case, these distributions are discrete, each corresponding to the eight different initial states of the three qubits. If the distributions completely match the fitness function, the KL-value is zero. A decreasing fitness value close to zero indicates an increase in the efficiency/entanglement of the circuits.
Our findings show that with mutation rates up to 20%, the optimization process effectively explores the solution space, leading to better circuit designs with lower fitness scores. However, mutation rates beyond 20% introduce excessive randomness, which disrupts the optimization process and results in higher fitness scores, indicating less efficient/entangled circuits.

All mutation rates result in a decrease in fitness scores over generations, indicating improvement. However, mutation rates of 5%, 10%, and 20% stabilize more consistently compared to higher rates. The right panel box plots show that while higher mutation rates (30%, 50%) lead to decreased fitness scores, they introduce greater variability and less stability. These results highlight that while higher mutation rates can still improve circuit efficiency, the optimal balance between exploration and stability during the "evolution" of the algorithm is achieved at a 20% mutation rate.

Finally, some last points for discussion. This paper opens avenues for future investigations into advanced evolutionary algorithms and alternative fitness functions, such as Von Neumann entropy <cit.> and Schmidt measures <cit.>, to maximize entanglement in more complex quantum circuits. 
Future work could explore the scalability of these methods to larger quantum circuits, composed by more than five qubits. 
The fitness function here introduced, measuring the entanglement between qubits could be extended to multi-dimensional optimization scheme, incorporating also measures of error and noise in quantum circuits. 
Moreover, it would also be of interest to adapt the method here proposed and integrate it in hybrid optimization approaches.
Future research could explore the maximization of entanglement in quantum circuits, drawing inspiration from recent advancements in benchmarking highly entangled states on a 60-atom analog quantum simulator <cit.> and leveraging non-locality as a resource in quantum computing demonstrated by loophole-free Bell inequality violations with superconducting circuits <cit.>.
All in all, applying optimized quantum circuits to real-world problems in quantum cryptography <cit.>, quantum simulation <cit.>, and quantum machine learning <cit.>, as well as testing them on actual quantum hardware, could further demonstrate their practical utility and effectiveness <cit.>.







§ METHODS



Our study implements two distinct methods for designing quantum circuits. The first method utilizes cellular automata rules combined with the KL fitness function. Here we advanced the original framework from Ref. <cit.> for generating quantum circuits that replicate quantum circuits with evolutionary algorithms. Upon this groundwork the second method focuses on designing quantum circuits for maximum entanglement, using the Mayer-Wallach measure and von Neumann entropy for entanglement evaluation. The comprehensive details regarding the implementation of evolutionary quantum algorithms for both methodologies are thoroughly delineated in Supplementary Note I.




 §.§ Evolutionary algorithms: Mutations and fitness function
 

The evolutionary algorithm initiates with a specific assigned number of chromosomes, each populated with a set number of quantum gates within the circuit. This initial population is then subject to evolution, where the fittest chromosomes are identified, iteratively refining the existing ones, and introducing the best-fit chromosome at the forefront of each new generation. This evolutionary cycle continued for a predetermined number of generations, with each generation's best chromosome becoming the parent for the next. Throughout this process, the chromosomes are subjected to mutations to enhance the optimization. The mutation process is introduced in two different forms: either by swapping existing gates in the chromosome with randomly selected ones from the gate pool, or by entirely replacing the chromosome to identify the best four parental candidates suitable for the next generation. The mutation process is controlled by a set of probabilities. The four best chromosomes are preserved as "elites" without changes, whereas the remaining chromosomes are selected for evolution based on their fitness, with selection changes directly linked to their performance.

In this context, we considered one-dimensional CAs where each of the cells can be in one of two possible states, 0 or 1 (Boolean CA), The evolution of these CA is governed by a set of rules, precisely 256 possible ones, that determine how the state of a cell updates at each step based on the states of its immediate neighbors. The rules correspond to each of the eight possible configurations of a cell and its two immediate neighbors, namely [0, 0, 0], [0, 0, 1], [0, 1, 0], [0, 1, 1], [1, 0, 0], [1, 0, 1], [1, 1, 0], and [1, 1, 1]. The rule dictates the next state of the central cell in these configurations. Table <ref> shows the matching in all cases considered in this paper. 

To evaluate the performance of the quantum circuits, we measure only the first qubit (q0) and consider
the fitness function as the Kullback-Leibler (KL) divergence, which measures the difference between two probability distributions and has been used in other works <cit.> as a fitness function:

    D_KL(P||Q)= ∑_ω∈Ω P(ω)log(P(ω)/Q(ω)).


In our case, the distributions are discrete, and each one has eight different values related to the eight different initial states of the three qubits. If the distributions are a perfect match, then the fitness function is zero. 



 §.§ Assessing entanglement in basic quantum states of qubits


Consider, for the start, the state of a system of two-qubit configurations of a quantum system. The most general form of a state for such a system can be written as: 

    |ψ⟩ = α_00|00⟩+α_01|01⟩+α_10|10⟩+α_11|11⟩   ,

where α_00, α_01, α_10, and α_11 are complex amplitudes of the corresponding basis states. The normalization of the basis state is

    |α_00|^2 + |α_01|^2+ |α_10|^2 + |α_11|^2 = ∑_X∈{0,1}^2|α_X|^2 = 1  ,

where X∈{0,1}^2 means all binary combinations of length 2.
For a two-qubit state, there is one fully entangled state given by <cit.>

    |ψ⟩ = 1/√(2) ( |00⟩+|11⟩ )   .

This state exemplifies quantum entanglement where the measurement outcome of one qubit directly determines the outcome of the other, showcasing the non-classical correlation inherent in quantum systems <cit.>.
To contrast, consider the following non-entangled state:

    |ψ⟩ = 1/2|00⟩+1/2|01⟩+1/2|10⟩+1/2|11⟩.

In this state, when measuring one qubit, there is no certainty about the state of the other qubit. Namely, there is a probability of 1/2 that the other qubit is in state 0 and probability 1/2 that it is in state 1.

In general, suppose we have two quantum systems Q_1 and Q_2. These systems are said to be entangled if the values of certain properties of system Q_1 (accessed with measurement) are correlated with the values that those properties will assume for system Q_2. The simplest forms of entangled state are known as the Bell states <cit.>, which enable tasks such as quantum cryptography<cit.>, super-dense coding<cit.>, teleportation<cit.>, and entanglement swapping <cit.>. 

The descriptors, expressibility, and entangling capability have been used to study the capabilities of the parametrized quantum circuit by quantifying its deviation from random circuits to approach the research question of how much generalization is effective enough in a quantum circuit for a given task <cit.>. The expressibility of a quantum circuit is the ability to generate pure states that are well representative of the Hilbert space <cit.>. In a single qubit, the expressibility corresponds to the circuit's ability to explore the Bloch sphere. Sim et al. <cit.> propose to quantify the ability of the quantum circuit to generate a pure state as a representative of Hilbert space by comparing the true distribution of the fidelities corresponding to the parameterized quantum circuit (PQC),  to the distribution of fidelities from the ensemble of Haar random states. 

For example, the measure of expressibility can be calculated by taking the Kullback-Leibner divergence between the estimated fidelity distribution and that of the Haar-distributed ensemble <cit.>. 
In this paper, we considered two measures, the Meyer-Wallach (MW)<cit.> and the entanglement entropy (EE)<cit.>.
The Meyer-Wallach entanglement measure is popular because of its scalability and ease of computation.
It considers a pure state of n qubits if the form <cit.>:


    Q(|ψ⟩) = 4/n∑_j=1^n D(|û^k⟩, |v̂^k⟩) ,

where |û^k⟩ and |v̂^k⟩ denote non-normalized vectors belonging to the complex vector space 𝐂^2𝐧-2, where n denoting the number of qubits in the quantum system. These vectors are obtained by projecting the pure state |ψ⟩ of the n-qubit system onto the local basis states of the k^th qubit, where k ∈1, 2, ..., n. The symbol "" above the vectors signifies their non-normalized status.
The function D(|û^k⟩, |v̂^k⟩) measures a distance between the two vectors |û^k⟩ and |v̂^k⟩, measured through the generalized cross-product: 

    D(|û^k⟩, |v̂^k⟩) = ∑_i<j|û_i^kv̂_j^k-û_j^kv̂_i^k|^2   .


Since the purity of the state of qubit k is given by Tr[ρ_k^2]=⟨x̂^k|x̂^k|^⟩2+⟨ŷ^k|ŷ^k|^⟩2, we obtain <cit.>  


    D(|x̂^k⟩, |ŷ^k⟩) = ∑_i<j|û_i^kv̂_j^k-û_j^kv̂_i^k|^2

It is because generalized cross product under logical unitaries, D(|û^k⟩, |v̂^k⟩)= D(|x̂^k⟩, |ŷ^k⟩) in relation to the norm of an anti-symmetric tensor M_k = |x̂^k⟩⟨ŷ^*k| - |ŷ^k⟩⟨x̂^*k|. Therefore, one arrives at an entanglement measure given by:

    Q(|ψ⟩)= 2  ( 
                         1-1/n∑_k=0^n-1Tr[ρ_k^2]
                          ),

which is the usual definition of the Meyer-Wallach measure. A detailed derivation and discussion of this measure, especially its implications for both pure and mixed states, are provided in Supplementary Note IV.


The entanglement entropy quantifies the quantum entanglement among two subsystems within a larger quantum system. Specifically, for a pure bipartite quantum state of the mixed system, it is feasible to derive a reduced density matrix that characterizes the state of one of the subsystems <cit.>. The entropy of entanglement is defined as the von Neumann entropy of the reduced density matrix for either of the subsystems <cit.>.

The von Neumann entropy quantifies the uncertainty or randomness in a quantum state and measures the degree of entanglement between the subsystems <cit.>. If the entropy of entanglement is non-zero, it implies that the subsystem is in a mixed state, and consequently, the two subsystems are entangled <cit.>. Therefore, the entropy of entanglement offers a useful method for quantifying the degree of entanglement between the subsystems of a composite quantum system.

The von Neumann entropy of entanglement is mathematically defined as the von Neumann entropy of the reduced density matrix for a subsystem within a composite quantum system <cit.>, namely

    S(ρ_A) = -Tr(ρ_A log_2 ρ_A)   ,

where Tr represents the trace operation, and log_2 denotes the base-2 logarithm. The von Neumann entropy of entanglement provides a measure of the uncertainty or randomness in the state of subsystem A. Thus, it serves as an indicator of the degree of entanglement between the subsystems <cit.>. When dealing with a composite quantum system that's in a mixed state, the von Neumann entropy of entanglement measures the least amount of entanglement present across all possible pure-state decompositions of the mixed state. This method allows for measuring entanglement in a mixed state by considering the representation of the composite system that has minimal entanglement.

In what concerns implementation, we have used the mathematical formulations presented in Equations (<ref>) and (<ref>) in a framework previously developed (<https://github.com/Overskott/Quevo>). It is further aligned with the methodologies outlined in Ref.<cit.>. The evolutionary algorithms (EAs) have emerged as a promising tool for quantum circuit design <cit.>. So our implementation of EAs for quantum circuit design aims to maximize entanglement using two specified methods. The framework employs an evolutionary algorithm to automatically generate quantum circuits that satisfy defined properties specified through a fitness function. The evolution properties can be controlled by tuning parameters such as initial population, number of generations, probability of mutation operator, number of gates in the circuit, and the chosen fitness function.  Details of the implementation process are available in Supplementary Note III.



§ DATA AVAILABILITY

The datasets used and/or analyzed during the current study are available in the QUEVO1 repository, <https://github.com/shailendrabhandari/QUEVO1>. 








§ ACKNOWLEDGEMENTS


We extend our gratitude to Sebastian T. Overskott for making the https://github.com/Overskott/QuevoQuevo framework available, which we have adapted for the entanglement assessment in our study. 



§ AUTHOR CONTRIBUTIONS STATEMENT


S.B. conducted the experiments and performed the simulations. All authors have analyzed the results and written the manuscript. S.N., S.D., and P.G.L. have revised the text and P.G.L. coordinated the overall project.



§ SUPPLEMENTARY NOTE I: IMPLEMENTATION OF AN EVOLUTIONARY QUANTUM ALGORITHM



Our approach was implemented in Python and resulted in a Python module called https://github.com/Overskott/QuevoQUEVO. This module consists of three classes: , , and . The  and the  classes are part of the genetic algorithm, while the  class is responsible for generating and simulating quantum circuits.





The  class is the core of the genetic algorithm used for generating quantum circuits. It handles the integer representation of the gates and their connections. The class contains a list of integers and functions to generate and mutate the list. It also handles the initialization of the population and the evolution of the population into a new one. Mutations in the  class can occur in two different ways: replacing gates from the pool of gates in the chromosome with a randomly generated new one or replacing the chromosome to generate the four best parents. The class is also responsible for checking the chromosome for gates that connect multiple qubits. If the gate has an invalid connection, meaning that it is connected to itself through the randomly generated integers, the class generates a valid configuration randomly.

The  class is another important class in the QUEVO module, responsible for managing the population of chromosomes. It is a collection of chromosomes that undergo evolution for a specified number of generations. After each evolution step, changes in the chromosomes in the generation occur by selecting a fixed number of chromosomes as elite and allowing the rest of them to evolve further. In each evolution step, the chromosomes are evaluated with the fitness function, and the fittest chromosomes become the parents for the next generation. The rest of the chromosomes are reset for the initial chromosomes. The  class stores a generation of chromosomes, the fitness associated with each chromosome, methods for running and retrieving fitness for two different fitness functions, and functions for printing. It provides methods for performing selection, crossover, and mutation to generate a new population of chromosomes. 

The  class within the QUEVO module, which utilizes IBM's Qiskit (<https://qiskit.org/>), plays a crucial role in generating and managing quantum circuits. It is designed to create Qiskit quantum circuits from string representations, simulate these circuits, perform measurements, and visualize the results. Operated on the Qiskit AER simulator, this class yields results in a dictionary format. It can also emulate an IBMQ backend through the AER simulator, allowing the simulator to use custom quantum gates, basis gates, and coupling maps specific to that backend.

Qiskit, a Python package provided by IBM, is integral to creating and simulating quantum circuits in QUEVO. It offers tools for writing quantum programs and can simulate quantum circuits locally or run them on actual quantum computers via the cloud. To ensure seamless integration and operation of Qiskit within the QUEVO framework, a dedicated Python virtual environment is recommended, preferably set up using Anaconda. 

The  method implements the Ma­yer-­Wa­llach measure of entanglement, as a fitness function of the EA. This method processes a state vector representing a quantum circuit's state, converting it into a 2^n x 2^n tensor, where n is the circuit's qubit count. The entanglement for each qubit k is determined by tracing out all other qubits, summing the squared eigenvalues of their respective reduced density matrices. The results are then normalized by 1 - 1/n, ensuring values fall between 0 and 1. More information and method usage are detailed in the QUEVO1 GitHub repository, accessible at <https://github.com/shailendrabhandari/QUEVO1>. Besides the well-known Mayer-Wallach measure, we evaluate the entanglement properties of quantum states using the von Neumann entropy as fitness functions. EA creates random density matrices from complex state vectors and calculates the von Neumann entropy based on their eigenvalues. This dual approach in measuring entanglement with Mayer-Wallach and von Neumann entropy enhances the quantum circuits' entanglement characteristics, optimized via quantum evolutionary algorithms.




§ SUPPLEMENTARY NOTE II: STATE VECTOR PROBABILITIES AND DENSITY MATRICES FOR MULTI-QUBIT CIRCUITS



In quantum systems, probabilities of states are derived from state vector coefficients within the wave function, typically complex numbers of the form a + bi. The probability P for a state with coefficient c = a + bi is given by:

    P = |c|^2 = a^2 + b^2,

where a represents the real part and b the imaginary part of the coefficient. Following the computation of reduced density matrices for all qubits, the entanglement of the circuit is assessed using the Meyer-Wallach entanglement measure.

This appendix explicitly presents maximally entangled states for three, four, and five qubits. Results are systematically organized in tables, each corresponding to distinct quantum circuit configurations. Table <ref> illustrates the outcomes for a three-qubit system, with reduced density matrices for circuits comprising 3 and 12 gates (Left), and their associated quantum state probabilities (Right). Table <ref> explores a four-qubit system, detailing both reduced density matrices and state probabilities for four-gate circuits. Finally, Table <ref> showcases results for a five-qubit system, encompassing reduced density matrices for circuits with five and twelve gates (Top), alongside the state probabilities (Bottom). These tables collectively provide a comprehensive view of quantum state entanglement across varying circuit complexities and gate counts.
















§ SUPPLEMENTARY NOTE III: REDUCED DENSITY MATRIX AS A TOOL FOR UNDERSTANDING THE PURITY OF A STATE



The density matrix is a representation of a quantum state that can describe both pure states and mixed states. While the state-vector representation is only suitable for describing pure states, the density matrix can also be used to represent mixed states, which are probabilistic mixtures of pure states. A mixed state can be expressed as a sum of outer products of pure states, each term weighted by a probability. In the density matrix formalism, a mixed state is represented by a Hermitian matrix with non-negative eigenvalues that sum up to one.
The density operator representation is an alternative way to express pure quantum states using a matrix formalism. It is defined as:

    ρ≡ | ψ⟩⟨ψ | ,








where |ψ⟩ is the state vector representing the pure quantum state. The expression |ψ⟩⟨ψ| represents the outer product of the state |ψ⟩ with itself, 

    ρ = 
    [ α_0; α_1;   ⋮; α_N ][ α_0^* α_1^*     … α_N^* ] 
    =
    
    
    
    
    
    [   |α_0|^2 α_0 α_1^*         … α_0 α_N^*; α_1 α_0^*   |α_1|^2         … α_1 α_N^*;         ⋮         ⋮         ⋱         ⋮; α_N α_0^* α_N α_1^*         …   |α_N|^2 ].

The density operator has several important properties. It is Hermitian, which means that it is equal to its conjugate transpose. It has a trace equal to one, reflecting the fact that it represents a pure state. Furthermore, it also satisfies the property, ρ^2 = ρ, indicating the state is pure. In other words, the density matrix characterizes the pure state.
To illustrate this, let us consider an example of a two-qubit, maximally-entangled pure state |ψ_AB⟩, which can be expressed as:

    |ψ_AB⟩ = 1/√(2) ( |00⟩ + |11⟩ ) = 1/√(2)[ 1; 0; 0; 1 ].



Using the density operator representation, we can express this state as:

    ρ_AB  = 
    
    
    
     ( 1/√(2)[ 1; 0; 0; 1 ] )  ( 1/√(2)[ 1 0 0 1 ] )  = 
    
    
    
    1/2[ 1 0 0 1; 0 0 0 0; 0 0 0 0; 1 0 0 1;   ]  .

This matrix satisfies the properties of a density operator, namely, it is Hermitian, positive semi-definite, and has a trace equal to one. The density operator representation provides a compact and convenient way to represent pure states in a matrix formalism, which can be useful for calculations and quantum information processing tasks.

The reduced density matrix ρ_A describes the state of subsystem A after tracing out the degrees of freedom of subsystem B. It is obtained by performing a partial trace of the density matrix ρ_AB over subsystem B <cit.>:

    ρ_A = Tr_B(ρ_AB) = ∑_i=1^d_B⟨ i|ρ_AB|i⟩ ,

where d_B is the dimension of the Hilbert space of subsystem B, and |i⟩ denotes a basis state of subsystem B.




The reduced density matrix ρ_A inherits some of the properties of the original density matrix ρ_AB. For example, it is Hermitian, positive semi-definite, and has a trace equal to one. However, it is generally not pure, even if the original state ρ_AB is pure. This is because entanglement between subsystems cannot be removed by taking a partial trace.

Tr_B in Equation (<ref>) is an operation known as the partial trace and is used to extract the state of a subsystem of a composite system from its overall density matrix. The partial trace over subsystem B of a tensor product of two operators |ξ_u⟩⟨ξ_v| and |χ_u⟩⟨χ_v| is given by:

    Tr_B  (| ξ_u ⟩⟨ξ_v | ⊗ | χ_u ⟩⟨χ_v |  ) ≡ | ξ_u ⟩⟨ξ_v |  Tr ( | χ_u ⟩⟨χ_v |  ) ,

where | ξ_u ⟩ and | ξ_v ⟩ are arbitrary states in the subspace of A, and | χ_u ⟩ and | χ_v ⟩ arbitrary states in the subspace of B. Tr is the standard trace operation, which for two arbitrary states Tr ( | χ_u ⟩⟨χ_v |  ) = ⟨χ_v |χ_u ⟩. Similarly, the reduced density matrix of subsystem B can be obtained by taking the partial trace over subsystem A of the tensor product of two operators:

    Tr_A  (| ξ_u ⟩⟨ξ_v | ⊗ | χ_u ⟩⟨χ_v |  ) ≡Tr ( | ξ_u ⟩⟨ξ_v |  ) | χ_u ⟩⟨χ_v | .




As an example, let's reconsider the pure entangled state:
    | ψ_AB⟩ = 1/√(2) ( | 0_A 0_B ⟩ + | 1_A 1_B ⟩ ).

 

This system is then composed of single-qubit subsystem A with basis vectors { |ξ_1 ⟩, |ξ_2 ⟩} = { | 0_A ⟩, | 1_A ⟩}, and single-qubit subsystem B with basis vectors { |χ_1 ⟩, |χ_2 ⟩} = { | 0_B ⟩, | 1_B ⟩}. We know that this system is not separable (i.e., | χ_AB⟩≠ |χ_A⟩⊗ |χ_B⟩); however, by using the reduced density matrix, we can find a full description for subsystems A and B as follows.

The density matrix of our state | ψ_AB⟩ can be expressed in terms of outer products of the basis vectors as
ρ_AB = 
1/2 [ | 0_A 0_B ⟩⟨ 0_A 0_B | + | 0_A 0_B ⟩⟨ 1_A 1_B |  .
. +| 1_A 1_B ⟩⟨ 0_A 0_B | + | 1_A 1_B ⟩⟨ 1_A 1_B |  ].
To calculate the reduced density matrix for, let's say, subsystem B, we have:

    ρ_B   = Tr_A(ρ_AB)
       = 1/2 [ Tr_A(| 0_A 0_B ⟩⟨ 0_A 0_B |) + Tr_A(| 0_A 0_B ⟩⟨ 1_A 1_B |) + Tr_A(| 1_A 1_B ⟩⟨ 0_A 0_B |) + Tr_A(| 1_A 1_B ⟩⟨ 1_A 1_B |)  ] 
       = 1/2 [ Tr(| 0_A ⟩⟨ 0_A |)| 0_B ⟩⟨ 0_B | + Tr(| 0_A ⟩⟨ 1_A |)| 0_B ⟩⟨ 1_B | + Tr(| 1_A ⟩⟨ 0_A |) | 1_B ⟩⟨ 0_B | + Tr(| 1_A ⟩⟨ 1_A |) | 1_B ⟩⟨ 1_B |  ] 
       = 1/2 [ ⟨ 0_A | 0_A ⟩ | 0_B ⟩⟨ 0_B | + ⟨ 1_A | 0_A ⟩ | 0_B ⟩⟨ 1_B | + ⟨ 0_A | 1_A ⟩ | 1_B ⟩⟨ 0_B | + ⟨ 1_A | 1_A ⟩ | 1_B ⟩⟨ 1_B |  ] 
       = 1/2 [ | 0_B ⟩⟨ 0_B | + | 1_B ⟩⟨ 1_B |  ].



Starting with an entangled pure state |ψ_AB⟩, the reduced density matrix ρ_B of subsystem B emerges as a mixed state. This matrix ρ_B statistically represents subsystem B by averaging out measurements from subsystem A. Tracing out A yields ρ_B, reflecting the probability distribution of B. This approach is crucial for analyzing entangled states in complex systems, allowing insight into individual subsystems within entanglements. Key concepts and details about density matrices are sourced from <cit.>, especially the section "The density matrix and mixed states" (<https://qiskit.org/textbook/ch-quantum-hardware/density-matrix.html##4.-The-Reduced-Density-Matrix–>) 







§ SUPPLEMENTARY NOTE IV: EVALUATION OF QUANTUM PURITY AND THE MEYER-WALLACH ENTANGLEMENT MEASURE



In this appendix, a comprehensive derivation and analysis of the quantum purity of a state is provided, particularly focusing on its relation to the Meyer-Wallach measure. 
If the state |ψ⟩ is pure, then its density matrix is given by ρ = |ψ⟩⟨ψ|, and its purity yields:

    Tr[ρ^2] = Tr[(|ψ⟩⟨ψ|)(|ψ⟩⟨ψ|)] 
    = Tr[|ψ⟩⟨ψ|ψ|⟨%s|⟩ψ] = Tr[|ψ⟩⟨ψ|] = 1.


The given equation is an expression for the quantum purity of a state |ψ⟩, denoted by Q(|ψ⟩). The purity is a measure of how pure or mixed a quantum state is, and it is defined as the trace of the square of the density matrix ρ that represents the state.
For a pure state, the expression 1 - 1/n∑_k=0^n-1Tr[ρ_k^2] is equal to zero.
For a mixed state, the density matrix can be written as a convex combination of pure states, ρ = ∑_i p_i |ψ_i⟩⟨ψ_i|, where p_i are probabilities and ∑_i p_i = 1. The purity of such a mixed state can not exceed 1, as one can conclude from: 

    Tr[ρ^2]    = Tr[(∑_i p_i |ψ_i⟩⟨ψ_i|)(∑_j p_j |ψ_j⟩⟨ψ_j|)] 
       = ∑_i p_i^2 Tr[|ψ_i⟩⟨ψ_i|] + ∑_i≠ j p_ip_j Tr[|ψ_i⟩⟨ψ_j||ψ_j⟩⟨ψ_i|] 
       = ∑_i p_i^2 + ∑_i≠ j p_ip_j |⟨ψ_i|ψ_j||⟩^2 
       ≤∑_i p_i^2 + ∑_i≠ j p_ip_j = (∑_i p_i)^2 = 1   .


Therefore, the purity of a mixed state is always less than or equal to one. Using this result, we can see that the expression 1 - 1/n∑_k=0^n-1Tr[ρ_k^2] is a measure of how mixed the state is. If the state is pure, then this expression is zero, and if the state is entangled/mixed, then this expression is positive.




 Paper Title: Lexicase-based Selection Methods with Down-sampling for Symbolic Regression Problems: Overview and Benchmark.



Lexicase-based Selection Methods with Down-sampling for Symbolic Regression Problems]Lexicase-based Selection Methods with Down-sampling for Symbolic Regression Problems: Overview and Benchmark



geiger@uni-mainz.de
0009-0002-3413-283X

  Johannes Gutenberg University Mainz
  Mainz
  Germany



dsobania@uni-mainz.de
0000-0001-8873-7143

  Johannes Gutenberg University Mainz
  Mainz
  Germany



rothlauf@uni-mainz.de
0000-0003-3376-427X

  Johannes Gutenberg University Mainz
  Mainz
  Germany





Research









§ ABSTRACT 

   In recent years, several new lexicase-based selection variants have emerged due to the success of standard lexicase selection in various application domains. For symbolic regression problems, variants that use an ϵ-threshold or batches of training cases, among others, have led to performance improvements.  
   Lately, especially variants that combine lexicase selection and down-sampling strategies have received a lot of attention. This paper evaluates random as well as informed down-sampling in combination with the relevant lexicase-based selection methods on a wide range of symbolic regression problems. In contrast to most work, we not only compare the methods over a given evaluation budget, but also over a given time as time is usually limited in practice.
   We find that for a given evaluation budget, ϵ-lexicase selection in combination with random or informed down-sampling outperforms all other methods. Only for a rather long running time of 24h, the best performing method is tournament selection in combination with informed down-sampling. If the given running time is very short, lexicase variants using batches of training cases perform best. 



[
    Franz Rothlauf
    
==================





§ INTRODUCTION


Symbolic regression searches for a  mathematical expression that best describes a given problem <cit.>. Applications of symbolic regression range from finance <cit.>, and medicine <cit.> to materials science <cit.>. Genetic Programming (GP) <cit.> is an evolutionary computation technique that has often been used successfully to solve even complex symbolic regression tasks <cit.>. In GP, a population of individuals evolves in an evolutionary process guided by selection and modified through variation operators. Recent work has shown that especially the choice of the selection method has a major impact on the solution quality <cit.>. 

Traditional selection methods like tournament selection evaluate the quality of individuals based on an aggregated fitness score leading to a loss of information about the structure of the training data <cit.>. Therefore, lexicase selection <cit.> has been proposed that uses the error values of individuals on each training case. 
This allows lexicase selection to select specialists that perform particularly well for certain parts of the problem, which has been found to be beneficial for search towards better solutions <cit.>. 

Due to the successful application of lexicase selection in many problem domains <cit.>, several variants of lexicase selection have been proposed. In the domain of symbolic regression, ϵ-lexicase selection outperformed standard lexicase selection and tournament selection on the considered problems <cit.>.  Others suggested to use batches of training cases instead of single training cases to evaluate the quality of individuals <cit.>. 

Recently, especially variants combining lexicase selection with down-sampling strategies received much  attention because it led to higher problem-solving success in different applications <cit.>. Down-sampling uses only a subset of the training cases  in each generation to evaluate the quality of the individuals. For a fixed evaluation budget, the saved evaluations can be allocated to a longer search which allows the exploration of more individuals. The simplest down-sampling strategy is to randomly sample a percentage of the training cases in each generation <cit.>. However, this might cause the exclusion of important training cases for several generations <cit.>. Therefore,  boldi2024informed proposed informed down-sampling as a strategy to create more diverse subsets. 

In preliminary work <cit.>, we analyzed and compared several relevant lexicase-based selection methods in combination with random down-sampling. We used tournament selection as baseline and also studied the combination of lexicase variants with batches of training cases. 
The paper at hand builds upon this work and presents a more comprehensive and more detailed comparison. <cit.>  analyzed selection methods only in combination with random down-sampling. The paper at hand also adds selection variants using informed down-sampling.
For the comparison, we use a wide range of symbolic regression problems to provide researchers as well as practitioners with a comprehensive guide for choosing the appropriate selection method.
 
We find that ϵ-lexicase selection in combination with random or informed down-sampling performs best for a given evaluation budget. For a time budget of 24h, the best performing method is tournament selection in combination with informed down-sampling, followed by ϵ-lexicase selection in combination with random down-sampling. For almost all studied selection methods, the combination with informed down-sampling outperformed random down-sampling for a given time budget of 24h. We observe that the relative performance of each selection method depends on the given running time. For example, if the running time  is very short (15 minutes), batch-tournament selection and batch-ϵ-lexicase selection perform best. Additionally, the detailed analysis for each problem shows that while there are general trends, the relative performance of the methods varies between problems. 

Sect. <ref> provides a brief overview of the related work. In Sect. <ref>, we describe the selection methods analyzed in this study, followed by a description of the applied down-sampling strategies in Sect. <ref>. Section <ref> presents our experimental setting and the benchmark problems. In Sect. <ref> we present our results, followed by the conclusions in Sect. <ref>.



§ RELATED WORK


We briefly discuss prior work on lexicase selection. More detailed descriptions of the lexicase-based selection methods and down-sampling strategies are given in Sec. <ref> and <ref>.

Traditional selection methods (e.g., tournament selection) evaluate the quality of individuals based on an aggregated fitness value. This  leads to a loss of information about the structure of the training data <cit.>. Lexicase selection <cit.> 
has been proposed as an alterantive considering the errors of individuals on each training case separately. Lexicase selection is able to select specialists as parents, meaning individuals that solve some part of a problem better than others while performing worse on average <cit.>. In addition, prior work has found that lexicase selection maintains a higher population diversity compared to tournament selection <cit.>. 

For continuous problems, standard lexicase selection is not able to achieve the same performance improvements due to the selection of strictly elite individuals on training cases <cit.>. 
Therefore, lexicase variants with a relaxed pass condition have been proposed <cit.>, with ϵ-lexicase selection <cit.> being the best known. 
Additionally, variants using batches of training cases have been introduced <cit.>. It has been found that the use of batches can increase the generalizability of the found solution <cit.>. Batch-tournament selection  <cit.> uses batches in combination with tournament selection to improve the efficiency of lexicase selection. 
Another idea to improve the runtime of lexicase selection is to combine lexicase selection and weighted shuffle with partial evaluation  <cit.>. Plexicase selection <cit.> improves the runtime of lexicase selection by sampling individuals from a probability distribution instead of doing the actual selection procedure. ni2024dalex proposed DALex to improve the runtime by performing all calculations as matrix multiplications. In other studies, the combination of lexicase selection and novelty search has been explored to prevent premature convergence  <cit.>.

Recently, the combination of lexicase variants with down-sampling strategies has been found to lead to higher performance <cit.>. Therefore, the influence of down-sampling has been analyzed in several studies <cit.>. Down-sampling reduces the number of evaluations per generations by sampling a subset of training cases from the training set allowing the evaluation of more individuals with the same evaluation budget <cit.>. While random down-sampling <cit.> randomly creates subsets, informed down-sampling <cit.> creates subsets that includes distinct training cases. It has been found that informed down-sampling outperforms random down-sampling in combination with lexicase selection on program synthesis problems <cit.>. Therefore, informed down-sampling has been further studied in combination with tournament and fitness-proportionate selection for program synthesis problems and synthetic regression problems with an exact solution <cit.>. However, to our knowledge, there is no study that analyzes the influence of informed down-sampling for continuous symbolic regression problems.

Lexicase selection and its variants have been successfully applied and analyzed in several application domains, like program synthesis <cit.>, symbolic regression <cit.>, evolutionary robotics <cit.>, rule-based learning systems <cit.>, and deep learning <cit.>.

Prior work analyzed and compared several lexicase variants for program synthesis problems <cit.>. However, to our knowledge, a study that compares lexicase-based selection methods in combination with different down-sampling strategies, including informed down-sampling, on a wide range of symbolic regression problems is missing so far. 



§ SELECTION METHODS FOR SYMBOLIC REGRESSION PROBLEMS


We provide a detailed description of tournament selection, lexicase selection, as well as the lexicase-based variants studied in this work in the context of symbolic regression. 



 §.§ Tournament Selection


With tournament selection, k individuals are randomly chosen to participate in a tournament. The participant with the best fitness wins the tournament and is selected as a parent <cit.>. Fitness is measured as an aggregated value, for example the mean squared error (MSE)

    MSE(T) = 1/|T|∑_t ∈ T (y_t - ŷ_t)^2,

where ŷ_t is the predicted output of an individual and y_t the desired output for all training cases t ∈ T. 



 §.§ Lexicase Selection


In contrast to tournament selection, lexicase selection <cit.> considers the error values e_t on each training case t ∈ T separately rather than comparing individuals based on an aggregated fitness value. Algorithm <ref> shows the procedure of a single selection event with lexicase selection with population P, the set of training cases T, the error e_t of an individual i ∈ P for training case t∈ T, and the minimal error e^*_t on cases t∈ T.
 



With lexicase selection, all individuals i ∈ P are considered as candidates for selection (line <ref>) and all training cases t ∈ T are randomly ordered (line <ref>). Lexicase selection iterates through all training cases always keeping only the individuals in the candidate pool C with the lowest error e^*_t on the current case t (line <ref>). This is repeated until there is only one candidate left or there are no more training cases. In this case, one individual is randomly chosen from the remaining candidate pool C (line <ref>) <cit.>.




 §.§ ϵ-Lexicase Selection

For continuous problems, the pass condition of standard lexicase selection that only candidates with the lowest error remain in the candidate pool (Algorithm <ref>, line <ref>) is too strict. Therefore, ϵ-lexicase selection has been proposed by LaCava.2016, LaCava.2019 for solving symbolic regression problems. 

ϵ-lexicase selection modifies the pass condition of standard lexicase selection in such a way that all candidates with an error e_t <= e^*_t + ϵ_t remain in the candidate pool.
ϵ_t is calculated for a training case t using the median absolute deviation <cit.> 

    ϵ_t = median(|e_t - median(e_t)|),

where e_t is a vector of all errors on the current case t across the candidate pool C <cit.>.  




 §.§ ϵ-Plexicase Selection

ding2023probabilistic proposed plexicase selection as a more efficient alternative to lexicase selection. The idea is to reduce the runtime of lexicase selection by sampling individuals from a probability distribution instead of doing the actual selections. However, dolson2023calculating has shown that the problem of calculating the actual selection probabilities using lexicase selection is 𝒩𝒫-hard. Therefore, ding2023probabilistic only approximate the probability of individuals being selected with lexicase selection by finding the Pareto set boundaries through pairwise comparisons of individuals and assigning selection probabilities to each individual based on these comparisons. In order to further improve the solution quality, ding2023probabilistic introduce a hyperparameter α to manipulate the generated probability distribution.
For symbolic regression problems, an ϵ-threshold like in ϵ-lexicase is used to calculate ϵ-relaxed Pareto set boundaries. They call this variant ϵ-plexicase selection.




 §.§ Batch-Tournament Selection

The problem-solving performance of lexicase selection is superior to tournament selection <cit.>. However, lexicase selection is computationally expensive compared to tournament selection <cit.>. Therefore, de2019batch proposed batch-tournament selection as well as the variant Batch Tournament Selection Shuffled (BTSS) which shuffles the cases in addition to grouping them in batches. Both approaches combine the good performance of lexicase selection with the low computational effort of tournament selection. 

For BTSS, the training cases are randomly combined to batches of size b and the fitness values of the individuals are derived from their performance on each batch (e.g., in terms of the MSE). For each selection event, k individuals participate in a tournament. The individual with the lowest MSE on the current batch is selected as a parent. For each selection event, another batch is used to compare the quality of the individuals. If there are less batches than parent selection events, the batches are used multiple times.




 §.§ Batch-ϵ-Lexicase Selection

Using lexicase selection, the outcome of a selection event highly depends on the ordering of training cases <cit.>. Therefore, lexicase variants using batches of training cases instead of single training cases to compare the performance of individuals in the lexicase selection process have been proposed <cit.>. It has been found that the generalization ability of the found solutions is better using batch-lexicase selection compared to standard lexicase selection <cit.>.
For continuous-valued problems, geiger2024comprehensive proposed Batch-ϵ-lexicase selection to combine batches with ϵ-lexicase selection. In each generation, the training cases are randomly combined to batches of size b and the fitness values of the individuals are calculated on each batch (e.g., in terms of the MSE). The selection is then performed with ϵ-lexicase selection. 



§ DOWN-SAMPLING STRATEGIES

We describe the down-sampling strategies random down-sampling and informed down-sampling. 



 §.§ Random Down-sampling

In  application domains, like program synthesis <cit.> and symbolic regression <cit.>, it has been found that the combination of lexicase-based selection methods and random down-sampling improves the problem-solving success. Random down-sampling means that in each generation a random subset of training cases is used to evaluate the performance of the individuals. Therefore, the saved fitness evaluations in each generation can be used to search for more generations or to increase the population size. This is beneficial as more individuals can be explored with the same evaluation budget <cit.>. Since a different random subset of training cases is used in each generation, the population is likely to be evaluated on a large proportion of the training cases over a few generations <cit.>. 

The down-sampling rate is defined by a parameter d. For example, if d = 0.1, only 10% of the training cases are used in each generation, meaning the search can be performed 10 times longer or the population size can be increased by a factor of 10. 




 §.§ Informed Down-sampling

A problem of randomly creating subsets of training cases is that important cases might be excluded for several generations while training cases with redundant information might end up in the same subset <cit.>. Therefore, boldi2024informed proposed informed down-sampling to create subsets consisting of more diverse training cases. The authors define synonymous cases as cases that are solved by the same individuals in a population. If two cases are solved by different individuals, this indicates that the cases measure a different functionality. Hence, it would be beneficial to have both cases in a subset.

boldi2024informed define the distance between two cases as the Hamming distance between their two solve vectors (a solve vector contains the output of each individual for that case). Then, they use the Farthest First Traversal Algorithm <cit.> to select a subset of training cases with a large pairwise distance. 
However, evaluating all individuals on all training cases in each generation to create informed subsets would mean that we lose the benefit of down-sampling as we do not save any fitness evaluations. Therefore, boldi2024informed propose to sample a fraction s of the parents that are evaluated on all training cases in order to calculate the distance between training cases. In addition, the distance calculation is only performed every g generations to further reduce the number of fitness evaluations required to create the informed subsets. 

Since boldi2024informed study informed down-sampling for program synthesis problems, solve vectors are binary. In the domain of symbolic regression, the solve vectors contain continuous errors values. Therefore, we calculate the distances between two cases in this work using Euclidean distances instead of  Hamming distances.  




§ EXPERIMENTAL SETTING


We present our experimental setup, followed by a description of the used benchmark problems. 




 §.§ Setup

All experiments were implemented within the DEAP framework <cit.> (version 1.4.1). Table <ref> summarizes the parameter values of our GP setting which is in line with previous work <cit.>.  



The primitive set consists of all input features x, an ephemeral random constant (ERC) where ERC∈{-1,0,1}, and the arithmetic functions addition, subtraction, multiplication, analytic quotient (AQ) <cit.>, sine, cosine, and negative. 
The population is initialized using ramped half-and-half with tree depths between 0 and 4 and a population size of 500. In each generation, crossover is applied with a probability of 80% and mutation with a probability of 5%. We restrict the maximum tree depth to 17 <cit.>. Each configuration is repeated 30 times. 

Usually, selection methods are compared for a given evaluation budget that depends on the number of training cases, the population size and the number of generations <cit.>. If down-sampling is applied, we allocate the saved evaluations to a longer search. Table <ref> shows the number of generations given to each run depending on the down-sampling strategy applied. 



If there is no down-sampling used (d = 1.0), we set the number of generations to G = 100. For random down-sampling we set a down-sampling rate of d = 0.1 as suggested by geiger.2023, meaning we can search for Ĝ = 1,000 generations with the same evaluation budget. For runs with informed down-sampling, the down-sampling rate is set to d = 0.1 as well. The parent sampling rate is set to s = 0.01 and the distance calculation scheduling parameter is set to g = 10 as suggested by boldi2024informed. This means that every 10 generations, 5 individuals are evaluated on all training cases to calculate the distance between the cases. The generational limit for informed down-sampling is calculated as 

    Ĝ = G/d+s(1-d)/g

where G is the number of generations for runs without down-sampling, d is the down-sampling rate, s the parent sampling rate and g the distance calculation scheduling parameter <cit.>. In our case, this leads to limit of Ĝ = 991 for runs using informed down-sampling. 

For each problem, we randomly split all cases into 70% training cases, 15% validation cases, and 15% test cases. All individuals are evaluated on the training cases to perform the selection process. In each generation, the individual with the lowest MSE on the validation cases is stored as the current best solution to avoid solutions that are overfitted on the training cases. After a run, this individual is evaluated on the unseen test cases in terms of the MSE. 

The calculation of the fitness of individuals during the search depends on the selection method. As tournament selection evaluates individuals based on an aggregated value, we use the MSE as the fitness measure. For ϵ-lexicase selection and ϵ-plexicase selection it is necessary to evaluate the performance of individuals on each training case separately. Therefore, we calculate for each individual the squared error on each training case. For batch-tournament selection and batch-ϵ-lexicase selection, the performance of individuals is measured in terms of the MSE on each batch of training cases. 

The parameter settings of the selection methods are shown in Table <ref>. They are carefully chosen according to the recommendations from the literature. 



For tournament selection, we set a tournament size of k = 5 <cit.>. The parameter of ϵ-plexicase selection is set to α = 1.0 as suggested by ding2023probabilistic. 
For batch-ϵ-lexicase selection, we study batch sizes b ∈{0.05, 0.075, 0.1} <cit.>, which means that a single batch consists of 5%, 7.5% or 10% of the training cases. 
For batch-tournament selection, we set a tournament size of k = 64 <cit.> and we study batch sizes b ∈{0.05, 0.075, 0.1} as well.


In addition to runs given a fixed evaluation budget, we also study the performance over a running time of 24h. Here, we measure time in terms of wall clock time. In each generation, we track the performance of the current best individual on the test cases, the median tree size across the population and the time. We conducted the experiments on a high performance computing cluster using Intel 2630v4 2,20GHz CPUs. All processes were executed single threaded. 




 §.§ Benchmark Problems


The selection methods are evaluated on 20 symbolic regression problems that are sampled without replacement from SR bench <cit.>. Due to limited computational resources, we want to ensure that all experiments finish within 24h. Therefore, we only considered problems with less than or equal to 1,000 instances. Table <ref> shows all considered problems with the number of instances and the number of features. 



The number of instances range from 100 to 1,000 and the number of features range from 2 to 50 for the studied problems.




§ RESULTS AND DISCUSSIONS

To develop a guideline for choosing the right selection method for the problem at hand, we  compare state-of-the-art selection methods on a wide range of symbolic regression problems under various conditions. In detail, we compare tournament selection (denoted as tourn), ϵ-lexicase selection (ϵ-lex), ϵ-plexicase selection (ϵ-plex), batch-tournament selection (batch-tourn), and batch-ϵ-lexicase selection (batch-ϵ-lex) in combination with no down-sampling (nds), with random down-sampling (rds) and with informed down-sampling (ids).

For the sake of simplicity, only the results for the best batch size found are shown for batch-tournament selection and batch-ϵ-lexicase selection, meaning the batch size with the lowest average MSE on the validation cases for each problem.
 
First, we show and discuss the results achieved by using a given evaluation budget. Then, we present the results comparing the selection methods over different running times. A statistical analysis of the results is given in Appendix <ref>.



 §.§ Results for runs with a given maximum number of evaluations


Usually, selection methods are compared over a fixed evaluation budget because fitness evaluations are considered the most expensive operation <cit.>. This means for runs without down-sampling that each selection method searches for 100 generations to find a solution. For runs with random down-sampling (d = 0.1) the search is performed for 1,000 generations, and for runs with informed down-sampling (d = 0.1, s = 0.01, g = 10) for 991 generations (see Sect. <ref> for further explanations). 

Figure <ref> plots the performance rankings of the selection methods for all 20 problems following the approach of Orzechowski.07022018 for comparing multiple algorithms over many datasets. 
For each method, we measured the relative performance on a problem by comparing the median MSE on the test cases over all 30 runs.
The down-sampling strategy is indicated by a prefix (nds for no down-sampling, rds for random down-sampling and ids for informed down-sampling). 

We observe that batch-tourn, tourn, ϵ-lex, and ϵ-plex perform better in combination with down-sampling. For batch-tourn and tourn, the variant with informed down-sampling performed best according to the median ranking. The overall best result is achieved by ϵ-lex in combination with random down-sampling and the second best by ϵ-lex in combination with informed down-sampling. Interestingly, we observe that ids-tourn performs significantly better than nds-tourn and rds-tourn, with ids-tourn being the third best method in terms of the median ranking. 











Since studying the ranking of each method only provides a summarized overview of their performance, we also show their actual performance in terms of the median MSE on the test cases for each problem in Table <ref>. The problems are abbreviated by their prefix number (see Table <ref> for the full names).  Boxplots of the distribution of the results for each problem can be found in Appendix <ref>.


According to the median MSE on the test cases, rds-ϵ-lex is the best performing method on 10 out of 20 problems and ids-ϵ-lex performs best on 6 out of 20 problems (on three problems, rds-ϵ-lex and ids-ϵ-lex perform equally well).

la2023flexible argue that for symbolic regression tasks one should not only take into account the performance of a model but also its simplicity, which is typically approximated by the size of a model. Therefore, we measured the median tree size of the final population for each run. Then, we compared the median population tree size of each method to derive their ranking for each problem (smaller median tree size is better), which is shown in Figure <ref>. 




Tournament selection generates populations with a large median tree size, especially rds-tourn. Apart from that, we found that the trees generated by the best performing methods rds-ϵ-lex and ids-ϵ-lex are relatively large while the ones generated by the worst performing method nds-ϵ-plex are small. We observed that rds-batch-ϵ-lex, ids-batch-ϵ-lex, and ids-batch-tourn generate populations with relatively small individuals. 

Furthermore, we observed that the tree size usually increases over time for all selection methods <cit.>. However, evaluating the performance of individuals is more expensive if their tree size is large. As trees grow over time, generations get more expensive. This is not taken into account when comparing methods only for a given evaluation budget. 


Additionally, the run time of the selection methods varies greatly. In Figure <ref>, we can see the ranking of the median time each method required to perform the runs for each problem (less time is better). 




For all methods, the variants without down-sampling are faster. We can see that the fastest method is nds-ϵ-plex, followed by nds-batch-tourn and nds-tourn. The slowest methods are the ϵ-lex variants. This means that some methods could perform more generations in the same time than others. Therefore, we are not only comparing the selection methods over a given evaluation budget but also over a given time period. 

To sum up, we found that ϵ-lexicase selection in combination with random and informed down-sampling perform best for a given evaluation budget. For four out of five selection methods the combination with down-sampling improved the performance. 
We observed that the median tree size of the generated populations as well as the run time varies greatly between methods. Therefore, we are not only comparing the methods for a given maximum number of evaluations but also for a given amount of time.  




 §.§ Results for runs with a given maximum running time


Now, we compare all methods for a given running time instead of a given maximum number of evaluations. This means that some methods run for more generations than others. We analyze the results for different run times to observe if there are differences in the relative performance of each method depending on the given time. 
First, we compare the best solution found within a running time of 24h. In Figure <ref>, the performance ranking of each method is shown after a running time of 24h. To get a more detailed view at the results, we show the median MSE of all methods on all problems in Table <ref>. The distributions of the results for each problem are shown in Appendix <ref> (Figures <ref>- <ref>).









For all methods, the combination with down-sampling improved the performance. For 4 out of 5 methods the combination with informed down-sampling performed best. Overall, the best result according to the median ranking was achieved by ids-tourn, followed by rds-ϵ-lex, ids-batch-tourn, and ids-batch-ϵ-lex. The worst performing methods are nds-tourn and rds-tourn.

It is interesting to note that informed down-sampling has the greatest influence on the performance of tournament selection, with ids-tourn performing significantly better than nds-tourn and rds-tourn. Tournament selection without down-sampling or with random down-sampling performed worst while the combination with informed down-sampling outperformed all other methods in this setting. We assume that this is because informed down-sampling includes only distinct cases in a subset. As a result, the  average error on a informed subset weights the performance for different problem parts more equally than for the full training set, since training cases that test the same behavior are excluded. This leads to the observed higher performance of informed down-sampling. 

Although ids-tourn performed best according to the median ranking, it was only the best performing method on 6 out of 20 problems. Rds-ϵ-lex performed best on 3 problems, ids-batch-tourn on 7 problems, and ids-batch-ϵ-lex on 5 problems. All in all, variants using informed down-sampling achieved the best results on 12 out of 20 problems.

As before, we are not only analyzing the performance but also at the tree sizes generated by each method. Figure <ref> shows the ranking of the median population tree size. 



The populations with the smallest individuals are generated by rds-batch-ϵ-lex and ids-batch-ϵ-lex, followed by ids-batch-tourn, rds-ϵ-plex, and ids-ϵ-plex. 
Of those methods, ids-batch-ϵ-lex and ids-batch-tourn performed best, suggesting that they offer a good compromise between size and performance. 
Interestingly, the variants using down-sampling always generate populations with smaller individuals than their counterpart without down-sampling. 

For practitioners, the time to find a solution for a problem is often strictly limited. Therefore, we also analyze the results found after a shorter running time. Figures <ref> and  <ref> show the performance ranking of each method for solutions found after 1h and 15 minutes, respectively. For the detailed results we point to the figures and tables in Appendix <ref>.





The rankings of the results found within 1h differ only slightly from the ones for 24h. For example, the relative performance of batch-tourn and batch-ϵ-lex without down-sampling is better now. However, the best performing methods are still ids-tourn and rds-ϵ-lex, followed now by rds-batch-tourn and ids-batch-tourn.

We observed further changes in the performance rankings when we limited the time even more. Looking at the results after 15 minutes, we can see that nds-batch-tourn is the best performing method now, followed by ids-batch-tourn and nds-batch-ϵ-lex. This indicates that the use of batches of training cases is especially useful if the given time to find a solution is very short. We assume that this is because the batch variants evaluate individuals from the beginning on several training cases, making the selection process less dependent on the order of the training cases <cit.>. 

The rankings of the tree sizes for the populations given 1h or 15 minutes are shown in Appendix <ref>. Again, tournament selection generates larger trees compared to the other methods. The populations with the smallest median tree size are generated by nds-ϵ-lex. 

To sum up, we observed that the relative performance of the selection methods as well as the influence of the down-sampling strategies depends on the given running time. Given 24h or 1h, the best performing methods are tournament selection in combination with informed down-sampling and ϵ-lexicase selection in combination with random down-sampling. However, if the available running time is even more limited the variants using batches of training cases outperform the other selection methods. For a given time of 15 minutes we observed that batch-tournament selection and batch-ϵ-lexicase selection perform best. 

For a time budget of 24h, down-sampling improves the performance of all selection methods, with informed down-sampling outperforming random down-sampling in 4 out of 5 cases. For less time, the advantage of down-sampling is lower. 
For example, given 15 minutes, only tournament selection, ϵ-lexicase selection and ϵ-plexicase selection increase their performance in combination with a down-sampling strategy. 
 
Furthermore, we analyzed the trade-off between performance and size. Given 24h, we found that informed down-sampled batch-tournament selection and informed down-sampled batch-ϵ-lexicase selection generate populations with relatively small individuals while still performing reasonably well. 




§ CONCLUSIONS


We compared relevant lexicase variants in combination with different down-sampling strategies on a wide range of symbolic regression problems. We signifcantly extended a previous conference paper by including random as well as informed down-sampling and by providing detailed results for all studied problems. To our knowledge, this is the first work studying the influence of informed down-sampling on various lexicase-based selection methods on a wide range of symbolic regression problems. We not only compared the selection method over a fixed number of evaluations, as usually done in the literature, but also over different running times  as users usually have limited time to solve their problems. Therefore, this work provides users with a comprehensive guide for choosing the best selection method for their individual problem. Furthermore, we analyzed the influence of the selection methods on the solution size.

We found that the relative performance of each selection method depends on the given setting. For a fixed number of evaluations, ϵ-lexicase selection in combination with random or informed down-sampling performed best. For almost all selection methods, the combination with a down-sampling strategy increased the performance. Furthermore, we observed that the median population tree size as well as the median runtime differs between methods. This highlights the importance of comparing the selection methods also for a given running time and not just for a given evaluation budget, as it is usually the case in literature. 

Therefore, we studied the solutions found after 24h, as well as after 1h and 15 minutes. For a given running time of 24h or 1h, the best performing methods are tournament selection in combination with informed down-sampling and ϵ-lexicase selection in combination with random down-sampling. Given 24h, the variants with informed down-sampling outperformed the variants with no down-sampling or with random down-sampling most of the times. If running time is limited to 15 minutes lexicase variants using batches of training cases outperform all other methods. 

To sum up, choosing the right selection method depends on the given setting. For a given evaluation budget, we recommend using ϵ-lexicase selection in combination with a down-sampling strategy. If the search can be performed for a few hours, tournament selection in combination with informed down-sampling should be the preferred choice. If there is a need for a quick solution, batch-ϵ-lexicase selection or batch-tournament selection lead to the best solution. However, the detailed results shows that while there are general trends, the performance of each method depends on the problem at hand. 

In future work, we will extend this study by including more problems and by analyzing other population dynamics next to tree size like diversity or specialist selection. Additionally, we will study the influence of informed down-sampling on tournament selection in the domain of symbolic regression further. 

ACM-Reference-Format




§ RESULTS FOR A GIVEN MAXIMUM NUMBER OF EVALUATIONS













§ RESULTS FOR A GIVEN RUN TIME





































§ TREE SIZES FOR A GIVEN RUNNING TIME OF 1H OR 15 MINUTES








§ RESULTS OF THE STATISTICAL ANALYSIS


For our statistical analysis, we followed the recommendations of demvsar2006statistical for comparing multiple methods over many datasets. For each configuration, we first use the Friedman test to test the null-hypothesis that all methods perform equivalent. We reject the null-hypothesis if p < 0.05. If the null-hypothesis is rejected, we use the Nemenyi post-hoc test for pairwise comparisons. The performance of two methods is significantly different if p < 0.05.  

The p-values of the Friedman test for each configuration are reported in Table <ref>. We reject the null-hypothesis in all cases. Therefore, we conducted the Nemenyi test for all configurations. Figures <ref>- <ref> report the corresponding p-values. 


















 Paper Title: Back to the Continuous Attractor.







The Aldous–Lyons Conjecture I: Subgroup Tests
    [
    August 5, 2024
=============================================



§ ABSTRACT 
Continuous attractors offer a unique class of solutions for storing continuous-valued variables in recurrent system states for indefinitely long time intervals.
Unfortunately, continuous attractors suffer from severe structural instability in general—they are destroyed by most infinitesimal changes of the dynamical law that defines them.
This fragility limits their utility especially in biological systems as their recurrent dynamics are subject to constant perturbations.
We observe that the bifurcations from continuous attractors in theoretical neuroscience models display various structurally stable forms.
Although their asymptotic behaviors to maintain memory are categorically distinct, their finite-time behaviors are similar.
We build on the persistent manifold theory to explain the commonalities between bifurcations from and approximations of continuous attractors.
Fast-slow decomposition analysis uncovers the persistent manifold that survives the seemingly destructive bifurcation.
Moreover, recurrent neural networks trained on analog memory tasks display approximate continuous attractors with predicted slow manifold structures.
Therefore, continuous attractors are functionally robust and remain useful as a universal analogy for understanding analog memory.





§ INTRODUCTION

Biological systems exhibit robust behaviors that require neural information processing of analog variables such as intensity, direction, and distance.
Virtually all neural models of working memory for continuous-valued information rely on persistent internal representations through recurrent dynamics.
The continuous attractor structure in their recurrent dynamics has been a pivotal theoretical tool due to their ability to maintain activity patterns indefinitely through neural population states <cit.>.
They are hypothesized to be the neural mechanism for the maintenance of eye positions, heading direction, self-location, target location, sensory evidence, working memory, and decision variables, to name a few <cit.>.
Observations of persistent neural activity across many brain areas, organisms, and tasks have corroborated the existence of continuous attractors <cit.>.









Despite their widespread adoption as models of analog memory, continuous attractors are brittle mathematical objects, casting significant doubts on their ontological value and hence suitability in accurately representing biological functions.
Even the smallest arbitrary change in recurrent dynamics can be problematic, destroying the continuum of fixed points essential for continuous-valued working memory.
In neuroscience, this vulnerability is well-known and often referred to as the “fine-tuning problem” <cit.>.
There are two primary sources of perturbations in the recurrent network dynamics:
(1) the stochastic nature of online learning signals that act via synaptic plasticity, and
(2) spontaneous fluctuations in synaptic weights <cit.>.
Thus, additional mechanisms are necessary to compensate for the degradation in particular implementations, by bringing the short-term behavior closer to that of a continuous attractor <cit.>.
However, we lack the theoretical basis for relying on the brittle concept of continuous attractors for understanding biological analog working memory.


In this study, we explore perturbations and approximations of continuous attractors in the space of dynamical models.
We first report on the differences and similarities between the various structurally stable dynamics in the vicinity of continuous attractors in these models.
The distinct topology of these dynamics implies qualitatively different behaviors in the asymptotic limit of time, and hence differences in the pattern of errors in temporal generalization.
By assuming normal hyperbolicity we will separate the time scales to obtain a decomposition of the dynamics by separating out the fast flow normal towards the “ghost” continuous attractor (a.k.a. slow manifold) and the slow flow within it.
The proposed decomposition applied to theoretical models and task-trained recurrent neural networks (RNNs) reveals a “universal motif” of analog memory mechanism with various potential topologies.
We derive a theory describing the persistent manifold that survives the seemingly destructive bifurcations and connects resulting systems with different topologies as approximate continuous attractors.
Our theory guarantees that systems close to a continuous attractor (in the space of vector fields) will have similar behavior to it.





§ A CRITIQUE OF PURE CONTINUOUS ATTRACTORS

We will first lay out a number of observations about the dynamics of bifurcations and approximations of continuous attractors used in theoretical neuroscience.

Ordinary differential equations (ODEs) are commonly used to describe the dynamical laws governing the temporal evolution of firing rates or latent population states<cit.>.
In this framework, neural systems are viewed as implementing the continuous time evolution of neural states to perform computations.
We will consider a continuous attractor as a mechanism that implements analog memory computation: carrying a particular memory representation over time.
To define it formally, let (t) ∈^d denote the neural state, and = () represent its dynamics.
Let ⊂^d be a manifold.
We say  is a continuous attractor, if (1) every state on the manifold is a fixed point, ∀∈, () = 0, and (2) the fixed points are marginally stable tangent to the manifold and stable normal to the manifold.
In other words, the continuous attractor is a continuum of equilibrium points such that the neural state near the manifold is attracted to it, and on the manifold, the state does not move.





 §.§ Motivating example: bounded line attractor





As an example, we can construct a line attractor (a continuous attractor with a line manifold) as follows:

    = - + [  + ]_+

where = [0, -1; -1, 0] and = [1; 1], and [·]_+ = max(0,·) is the threshold nonlinearity per unit.
We get = 0 on the x_1 = -x_2 + 1 line segment in the first quadrant as the manifold (Fig. <ref>A, left; black line).
We refer to it as the bounded line attractor (BLA).
Linearization of the fixed points on the manifold exhibits two eigenvalues, 0 and -2;
the 0 eigenvalue allows the continuum of fixed points, while -2 makes the flow normal to the manifold attractive (Fig. <ref>A, left; flow field).

In general, continuous attractors are not only structurally unstable <cit.>, they bifurcate almost certainly for an arbitrary perturbation.
Small changes to the parameters (,) perturb the eigenvalues.
Specifically, any perturbation to the 0 eigenvalue destroys the continuous attractor as it bifurcates to either
a single stable fixed point (Fig. <ref>A, top) or two stable fixed points separated with a saddle-node in between (Fig. <ref>A, bottom).
However, interestingly, after bifurcation, continuous attractors seemingly tend to leave a “ghost” manifold topologically equivalent to the original continuous attractor (note the slow speed).
Furthermore, the flow after the bifurcation is contained in the ghost manifold, i.e., it is an invariant manifold.
This phenomenon, wherein a continuous attractor is approximated by a manifold within the neural space along which the drift occurs at a very slow pace, has previously been commented on <cit.>.















 §.§ Theoretical models of ring attractors



For circular variables such as the goal-direction (e.g. for home navigation and working memory for communication	 in bees <cit.>) or head-direction, the temporal integration, and working memory functions are naturally solved by a ring attractor (continuous attractor with a ring topology) <cit.>.
Other examples include
integration of evidence for continuous perceptual judgments, e.g. a hunting predator that needs to compute the net direction of motion of a large group of prey <cit.>.
In this section, we investigate the bifurcations of various implementations of continuous attractors.
Continuous attractor network models of the head direction are based on the interactions of neurons that (anatomically) form a ring-like overlapping neighbor connectivity <cit.>.
Similarly, as with the line attractor, the ring attractor bifurcates for virtually every perturbation.
Nevertheless, the bifurcated dynamics again follow a similar pattern: the dynamics are restricted to a ghost manifold very close to the original continuous attractor.


Piecewise-linear ring attractor model of the central complex
Firstly, we discuss perturbations of a continuous ring attractor proposed as a model for the head direction representation in fruit flies <cit.>.
This model is composed of N heading-tuned neurons with preferred headings θ_j ∈{2π i/N}_i=1… N radians (Sec. <ref>).
For sufficiently strong local excitation (given by the parameter J_E) and broad inhibition (J_I), this network will generate a stable bump of activity corresponding to the head direction.
This continuum of fixed points forms a one-dimensional manifold homeomorphic to the circle.



We evaluate the effect of parametric perturbations of the form ← + with _i,j𝒩(0,1/100) on a network of size N = 6 with J_E= 4 and J_I=-2.4 by identifying all bifurcations (Sec. <ref>).
We found that the ring (consisting of infinite fixed points) can be perturbed into systems with between 2 and 12 fixed points (Fig. <ref>A).
As far as we know, this bifurcation from a ring of equilibria to a saddle and node has not been described previously in the literature.
The probability of each type of bifurcation was numerically estimated (Sec. <ref>).
There are several additional co-dim 1 bifurcations with measure zero (Fig. <ref>).
Surprisingly, the number of fixed points is maintained throughout a range of perturbation sizes and hence depends only on the direction of the perturbation (Fig. <ref>).





Bump attractor model
A well-established approach, consistent with the principles observed in the fly head direction network <cit.>, involves utilizing a circulant matrix as the connectivity matrix<cit.>.
This type of ring attractor network can support a stable “activity bump” that can move around the ring in correspondence with changes in head direction.
This can be accomplished with a connection matrix W with entries that follow a circular Gaussian function of i-j <cit.> (see for more details Sec. <ref>).
For finite-sized networks, the dynamics are constrained to an attractive invariant ring, covered with N stable fixed points for a network of size N (Fig. <ref>B).
For such networks the number of fixed points does change with the size of the perturbation (Fig. <ref>).


Low-rank ring attractor model Low-rank networks can be used to approximate ring attractors <cit.>.
Using dynamical mean field theory, in the limit of infinite-size networks, one can construct a ring attractor through a rank 2 network by constraining overlap of the right- and left-connectivity vectors (see for more details  Sec. <ref>).
However, in simulations of finite-size networks with this constraint, the dynamics instead always converge to a small number of stable fixed points arranged on a ring (Fig. <ref>).

Embedding Manifolds with Population-level Jacobians Approximate ring attractors can be constructed by constraining the connectivity so that the networks Jacobian satisfies certain requirements for a ring attractor to exist <cit.> (see for more details Sec. <ref>)
The models optimized using this method also can contain an invariant ring manifold on which the dynamics contain stable and saddle fixed points (Fig. <ref>C).

Similarity between all bifurcations and approximations of continuous attractors
In all discussed models of ring attractors, we verify that they suffer from the fine-tuning problem.
However, we observe ghosts of the continuous attractor in many different systems (either through bifurcation or from finite-size effects) that is an attractive invariant manifold.
Is this a universal phenomenon?
Moreover, while they were not a continuous attractor in the mathematical sense, they were approximate ring attractors in the sense that the fixed points and connecting orbits still form a circle.
Are continuous attractors somehow still useful as a model of how animals might represent continuous variables?
We will answer these questions in an implementation-agnostic manner.












§ THEORY OF APPROXIMATE CONTINUOUS ATTRACTORS

In this section, we apply invariant manifold theory to continuous attractor models and translate the results for the theoretical neuroscience audience.


 §.§ Persistent Manifold Theorem

Let l be the intrinsic dimension of the manifold of equilibria that defines the continuous attractor.
Given a perturbation () to the ODE that induces a bifurcation,

    = () + ϵ ()

where (·) = 1 and ϵ > 0 is the bifurcation parameter,
we can reparameterize the dynamics around the manifold with coordinates ∈^l and the remaining ambient space with ∈^d-l.
To describe an arbitrary bifurcation of interest, we introduce a sufficiently smooth function  and , such that the following system is equivalent to the original ODE:

    =           ϵ(, , ϵ)     (tangent)
       = (, , ϵ)     (normal)

where ϵ = 0 gives the condition for the continuous attractor = 0.
We denote the corresponding manifold of l dimensions _0 = {(,) |(,,0) = 0}.

We say the flow around the manifold is normally hyperbolic, if the flow normal to the manifold is hyperbolic, meaning that the Jacobians ∇_ evaluated at any point on the _0 has d-l eigenvalues with their real part uniformly bounded away from zero, and ∇_ has l eigenvalues with zero real part.
More specifically, for continuous attractors, the real part of the eigenvalues of ∇_ will be negative, representing sufficiently strong attractive flow toward the manifold.
Equivalently, for the ODE, = (), the variational system is of constant rank and has exactly (d-l) eigenvalues with negative real parts uniformly away from zero and l eigenvalues with zero real parts everywhere along the continuous attractor.
Informally, normal hyperbolicity guarantees that the perturbations change the structure of the manifold before they can destroy the attractiveness of the manifold.


[10][bthp]
  
    < g r a p h i c s >

  
    Persistent manifold theorem applied to compact continuous attractor guarantees the flow on the slow manifold _ϵ is locally invariant and continues to be attractive.
    The dashed line is a trajectory “trapped” in the slow manifold (locally invariant). 


For any parameterization , ϵ > 0 induces a bifurcation of the continuous attractor.
What can we say about the fate of the perturbed system?
The continuous dependence theorem <cit.> says that the trajectories will change continuously as a function of ϵ without a guarantee on how quickly they change.
However, the topological structure and the asymptotic behavior of trajectories change discontinuously due to the bifurcation.
Yet, surprisingly, there is a strong connection in the geometry due to Fenichel's theorem <cit.>.
We informally present a special case due to <cit.>:




Let _0 be a connected, compact, normally hyperbolic manifold of equilibria originating from a sufficiently smooth ODE.
For a sufficiently small perturbation ϵ > 0, there exists a manifold _ϵ diffeomorphic to _0 and locally invariant under the flow of Eq. (<ref>)-(<ref>).
Moreover, _ϵ has 𝒪(ϵ) Hausdoff distance to _0 and has the same smoothness as g and h. 

The manifold _ϵ is called the slow manifold which is no longer necessarily a continuum of equilibria.
However, the local invariance implies that trajectories remain within the manifold except potentially at the boundary.
Furthermore, the non-zero flow on the slow manifold is slow and given in the ϵ→ 0 limit as τ = (c^ϵ(), , 0) where τ = ϵ t is a rescaled time and c^ϵ(·) parameterizes the l dimensional slow manifold.
In addition, the stable manifold of _0 is similarly persistent <cit.>, implying the manifold _ϵ to remain attractive.


These conditions are met for the examples in Fig. <ref> (see Sec. <ref> for the rerparametrization of the BLA[As a technical note, for the theory to apply to a continuous piecewise-linear system, it is required that the invariant manifold is globally attracting <cit.>, which is also the case for the BLA (see also <cit.> for a discussion of geometric singular perturbation theory for piecewise linear dynamical systems).]).
As the theory predicts, BLA bifurcates into a 1-dimensional slow manifold (Fig. <ref>, dark-colored regions) that contains fixed points and connecting orbits and is overall still attractive.
Furthermore, Fenichel's Persistent Manifold theorem explains the bifurcation structure of the theoretical models discussed in Sec. <ref>.
Because continuous ring attractors are bounded, their invariant manifold persists and it remains attractive under small perturbations <cit.>.










 §.§ Fast-slow decomposition and the revival of continuous attractors

Consider a behaviorally relevant timescale for working memory, for example, roughly up to a few tens of seconds.
If the dynamical system is orders of magnitude slower, for example, 1000 sec or longer, its effect is too slow to have a practical impact on the behavior that relies on the working memory mechanism.
This clear gap in the fast and slow time scales can be recast as normal hyperbolicity of the slow manifold by relaxing the zero real part to a separation of time scales (reciprocal of eigenvalues or Lyapunov exponents).
In other words, the attractive flow normal to the manifold needs to be uniformly faster than the flow on the slow manifold.

By taking the limit of the slow flow on the manifold to arbitrarily long time constant (i.e., to zero flow), we achieve the reversal of the persistent manifold theorem.

Let _ϵ be a connected, compact, normally hyperbolic slow manifold  (as parametrized by Eq. (<ref>)-(<ref>)).
Further, assume that the real part of the eigenvalues of ∇_ are negative. Let the uniform norm of the flow tangent to the manifold be _∞=η.
There exists a perturbation with uniform norm at most η that induces a bifurcation to a continuous attractor manifold. 

An explicit perturbation is derived in Sec. <ref>.
This makes the uniform norm of the vector field on a (slow) manifold a useful measure to express the distance of an approximation to a continuous attractor.
It can also be used to bound the memory performance on the short-time scale.
Let _0 ∈, and φ = (·)|_ be the flow restricted to the manifold.
The average deviation from initial memory _0 over time is bounded linearly (for a derivation, see Sec. <ref>):

    1/vol∫_(t, _0) - _0 _0≤ tφ    (error bound)

Note that this bound is the worst case and tighter for sufficiently small t ≥ 0.
Furthermore, for compact invariant manifolds the error is bounded by the diameter of the manifold and hence this bound becomes irrelevant for t large.


Prop. <ref>  can be extended to the case where the invariant manifold has other dynamics than just slow, as long as that is happening inside submanifolds to which the output mapping is invariant (see Theorem <ref>).
In this case, we can guarantee that the system can be perturbed onto a decomposable system where one of the subsystems has a slow flow.



 §.§ Relevance of dynamics on the slow manifold in the asymptotic time scale

While the uniform norm gives insight on the short-time scale behavior of the perturbed ODE, we expect that working memory tasks generalize to longer durations <cit.>.
The long-time scale behavior on the slow manifold is dominated by the stability structure, i.e., the topology of the dynamics.
Although we have seen numerous topologies in Sec. <ref>, dynamical systems theory says that they are fundamentally limited, especially in low dimensions (see for more details Sec. <ref>).
This is especially relevant as previous works have identified a low-dimensional organization of neural activity to explain the brain's ability to adapt behavioral responses to changing stimuli and environments <cit.>.
For a ring attractor, this implies that the stability structure of the invariant manifold is either
(1) composed of an equal number of stable fixed points and saddle nodes, placed alternatingly and with connecting orbits, or (2) a limit cycle.
These different strability structures have different generalization properties (see Sec. <ref>).
In more complex scenarios, such as a two-dimensional attractor, stable fixed points and saddle nodes can coexist with limit cycles, creating a rich tapestry of possible neural states.









§ NUMERICAL EXPERIMENTS ON TASK-OPTIMIZED RECURRENT NETWORKS


While our theory describes the abundance of approximate continuous attractors in the vicinity of a continuous attractor, it does not imply that there are no approximate solutions away from continuous attractors.
We use task-optimized RNNs as a means to search for plausible solutions for analog memory for a circular variable.
We train a diverse set of RNNs, and then identify the solution type of trained RNNs to gain insights into its performance, error patterns, generalization capabilities, and, ultimately, proximity to a continuous attractor.

Understanding the implemented computation in neural systems in terms of dynamical systems is a well-established practice <cit.>.
Researchers have analyzed task-optimized RNNs through nonlinear dynamical systems analysis <cit.> and to compare those artificial networks to biological circuits <cit.>.
Previously, systematic analysis of the variability in network dynamics has been surveyed in vanilla RNNs, and variations in dynamical solutions over architecture and nonlinearity have been quantified <cit.>.
Furthermore, working memory mechanisms in RNNs had tendencies to find sequential or persistent representations through training depending on the task specification <cit.>.
We therefore investigated to what extent training RNNs on a task uniquely determines the low-dimensional dynamics, independent of neural architectures.



 §.§ Model Architectures and Training Procedure

We examined vanilla continuous-time RNNs for this work.
The time-discretized version of RNN network activity is given by

    _t    = f(_t + _t-1 + ) + ζ_t 
    _t    = _t +

where _t ∈^d is the hidden state,
_t is the readout,
_t ∈^K is the input,
f→ an activation function that acts on each of the hidden dimensions,
state noise variable ζ_t𝒩(0,σ^2 = 1/100), and , , , , are parameters.
Assuming Euler-Maruyama integration with unit time step, the discrete-time RNN of (<ref>) corresponds to the stochastic differential equation:

    = - t + f((t) +  + ) t + σ W.

where W is a Wiener process that models the intrinsic state noise in the brain.

Building upon prior work, which has shown their capabilities on such tasks, we trained RNNs to either
(1) estimate head direction through integration of angular velocity <cit.>
and (2) perform a memory-guided saccade task for a ring variable <cit.> (details in Sec. <ref>).
We numerically minimized the loss L which was the mean squared error (MSE) between the network output y(t) and the target output ŷ(t):
L_MSE⟨ m_i,t(y_i,t-ŷ_i,t)^2⟩_i,t,
with a mask m_i,t with i the index of the output units and t  the index for time.
We implemented a mask, m_i,t, for modulating the loss with respect to certain time intervals.
We trained 10 networks for every combination of the following parameters:
 ReLU, tanh, and rectified tanh activation functions and
 number of units/neurons (64, 128, 256).








 §.§ Numerical Fast-Slow Decomposition

For each trained network, we find the slow manifold by integrating the autonomous dynamics then selecting the parts of the trajectory that have a speed slower than a threshold.
We identify the points on the invariant manifold from the simulated trajectories that are projected closest to a set of points in the output space relevant to the task after convergence, i.e. on the target ring.
We parametrize the one-dimensional invariant manifold  by fitting a cubic spline with periodic boundary constraints to these points (black line in Fig. <ref>A and B).
Normal hyperbolicity is measured by a gap in the timescales of the system (measured from the eigenvalue spectrum of the linearization along points on the invariant manifold (Fig. <ref>E and F).



We find the fixed points on the invariant ring by identifying regions where the direction of the flow flips for a dense set of points on the found invariant manifold.
We assess the direction through projection onto the output mapping and calculate the angular flow.
Stable fixed points are identified where the flow directions are both pointing towards this flip point,
while saddle nodes are identified where they are pointing away (filled and hollow circles Fig. <ref>A and B for stable and saddle fixed points respectively).
We find that long integrated trajectories of the network converge to the found stable fixed points through this independent method.




 §.§ Variations in the Topologies of the Networks

To understand the what solutions the RNNs found to solve the task, we investigate their memory mechanism.
For this, we dissect the dynamics of RNNs by segregating time scales to delineate the rapid flow normal to the slow manifold, and the flow on the manifold (Sec. <ref>).
All solutions involve a slow manifold with the same topology as the relevant variable in the task.
The different solutions are different in their asymptotic dynamics (Fig. <ref>).
The most often found solution is of the type fixed point ring manifold (Fig. <ref>A and B).
Less commonly found topologies includes
the slow torus around a repulsive ring invariant manifold (Fig. <ref>D).
All stability structures (fixed points and limit cycles) are mapped close to the target output circle (Figs. <ref>, <ref>, <ref>).
These solutions are consistent with both observations of the possibility of using non-constant dynamics for memory storage <cit.> and neuronal circuits underlying persistent representations despite time-varying activity <cit.>.

We verify that this gap exists in the trained networks shown in Fig. <ref>B and C.
The largest  eigenvalue of the Jacobian is non-zero (the invariant manifold is not a continuous attractor), but it is removed from the second largest, see Fig. <ref>E and F.
This implies that normal hyperbolicity for these trained networks holds.





 §.§ Universality amongst Good Solutions


The fixed point topologies show a lot of variation across networks (Fig. <ref>A and B),
 much like the systems next to continuous attractors (Fig. <ref> and Fig. <ref>).
Previously, it has been observed that fixed point analysis has a major limitation, namely, that the number of fixed points must be equal across compared networks  <cit.>.
Our methodology effectively addresses and overcomes this limitation.
The universal structure of continuous attractor approximations as slow invariant manifolds allows us to connect different topologies as approximate continuous attractors (Sec. <ref>).










§ GENERALIZATION ANALYSIS


When neuroscientists study neural computations in animals, the animals are trained on tasks with finite durations. As a result, it is unclear whether the animals learn the intended computation or merely a finite-time approximation of it. The same issue arises with trained neural networks.
We will investigate to what extent the networks have the intended memory necessary for perfect memory or whether they implement the task only on the timescale on which they have been trained.
We show that the theoretical upper bound for the memory performance based on the distance to a continuous attractor (Prop. <ref>) holds in trained RNNs (Fig. <ref>A).


The two possible approximations of a ring attractor discussed in Sec. <ref> exhibit markedly distinct generalization characteristics.
Approximating the system as a limit cycle results in a memory trace that gradually diminishes over time.
Conversely, the alternative approximation's memory states are contingent upon the quantity and positioning of stable fixed points within the system.
We describe in detail the generalization properties of the trained networks on the angular velocity integration task at two different time scales:  asymptotic and finite time.
We tested all networks with a validation set and took a cutoff for the normalized MSE for the networks we consider for the analysis at -20 dB (Fig. <ref>C).





Finite time
Aside from the angular velocity integration component of the task, the trained networks learn to store a memory of an angular variable.
We assess the performance of the network to store the memory of the angle over time.
The networks typically perform well on the timescale on which they have been trained T_1=256 time steps (Fig. <ref>B).
This loss is bounded by the uniform norm of the vector field on the invariant manifold (Fig. <ref>A,  see Sec. <ref> and <ref>).



Asymptotic time
Looking beyond the finite timescale provides valuable insights into the network's ability to store information.
For the asymptotic time scale, the asymptotic behavior of the system dominates and we identify the network behavior through its ω-limit sets.
For a one-dimensional system, this will either be fixed points or a limit cycle.
For the fixed-point type solution, the maximal error is given by the maximal distance to the next fixed point, while for a limit cycle, this will always be π.
We calculate the average fixed point distance by takeing the average of the inter-fixed-point interval for each neighbouring pair of fixed points.
We identify the location of the fixed points as described in Sec. <ref>.

Besides the maximal error that the network will make in the asymptotic limit, we can also characterize how many different angles the network would confuse in the asymptotic limit.
We construct a probability distribution of what part of state space we end up in an infinite time through the calculation of the size of the basins of attraction of stable fixed points as a proportion of the ring.
Finally, we characterize the memory capacity of the network by calculating the entropy of this probability distribution (this gives the memory capacity of the network, see Sec. <ref>).


Error Accumulation in Neural Networks
The mean accumulated error at the time at which the task was trained has an exponential relationship with the number of fixed points (Fig. <ref>A).
Furthermore, this error is bounded by the mean distance between stable and unstable fixed points (red dots in Fig. <ref>D).
This is another indication that the networks rely on a ring invariant manifold to implement the task.
Networks with different numbers of fixed points might have the same performance on the finite time scale (bounded by T_1φ for the trial time T_1 and the uniform norm of the vector field on the invariant slow manifold φ) but have vastly different generalization properties because they differ in the number of fixed points (Fig. <ref>C).




§ APPROXIMATE SLOW MANIFOLDS ARE NEAR CONTINUOUS ATTRACTORS

In Sec. <ref>, we presented a theory of approximate solutions in the neighborhood of continuous attractors.
When are approximate solutions to the analog working memory problem near a continuous attractor?
We posit that there are four conditions (See for more detail Sec. <ref>):


  * sufficiently smooth approximate bijection between neural activity and memory content,

  * the speed of drift of memory content is bounded,

  * robustness against state (S-type) noise, and

  * robustness against dynamical (D-type) noise.

The correspondence implied by <ref> translates to the existence of a manifold in the neural activity space with the same topology as the memory content.[Note that effectively feedforward solutions <cit.> do not satisfy <ref>.]
The persistence <ref> requires that the flow on the manifold is slow and bounded.
The S-type robustness <ref> implies non-expansive flow, that is, non-positive Lyapunov exponents.
Along with the D-type robustness <ref>, it implies the manifold is “attractive”, and normally hyperbolic (see also Sec. <ref>).

If these four conditions hold, for example for task-trained RNNs, there exists a smooth function with a uniform norm matching the slowness on the manifold such that when added, the slow manifold becomes a continuous attractor (Prop. <ref> and Theorem <ref>, see also Sec. <ref>).
For the RNN experiments, we added state-noise while training using stochastic gradient descent, satisfying <ref> and <ref>.
We have also verified that <ref> holds (Fig. <ref>A).
Although the stochastic optimization cannot lead to the continuous attractor solution, it gets to the neighborhood where all approximate solutions share the same main feature: having a subsystem that has a slow flow.






§ DISCUSSION

A manifold of equilibria in a continuous attractor network provides analog memory, making it a compelling model in theoretical neuroscience.
However, continuous attractors almost certainly bifurcate under arbitrary perturbations except in special paramtric forms.
This makes them unsuitable as models for neural computation in noisy biological systems according to the traditional perspective on robustness.
We will therefore never see we reach through traditional RNN training techniques and never observe a biological one.
Nevertheless, we have shown that continuous attractors are functionally robust and therefore are an important concept to describe the neural computation underlying analog memory.

We further show that under some assumptions, approximations of analog memory must have slow manifold dynamics and hence be close to a continuous attractor in the space of dynamical systems.
So for biological system or artificial neural networks it is sufficient to be near a continuous attractor to be effectively solving the problem similar enough to how the perfect theoretical model would solve it on a behaviorally relevant timescale.


We investigated the neighborhood of the continuous attractor networks and analyzed diverse bifurcations in various example systems.
Although the configurations corresponding to a continuous attractor are measure-zero in parameter space, we showed that when the persistent manifold theorem holds, the finite time behavior of the trajectories only slowly breaks down under perturbations.
For all perturbations and approximations there a diverse set of topologies.
However, all systems share an essential element: an attractive invariant slow manifold.
This empirical evidence strongly corroborates our proposed theory of analog memory.

Training RNNs on tasks involving continuous variables resulted similarly in a great variety of solutions.
As the theory predicted, trained networks on a circular analog memory task all have an attractive ring invariant manifold, which allows for a more useful comparison between trained networks than existing fixed point analyses.
This supports our theory that theoretical models that satisfy conditions <ref>-<ref> will be near continuous attractors (in parameter space).
Nevertheless, the topological structure of the found solutions still determines how well a network will perform generalization for longer time scales.

Although, we only explicitly describe the topology and dimensionality of the identified invariant manifolds for a representative set, the results indicate that most solutions have a ring invariant manifold with a slow flow on it.
This separation of timescale necessarily exists for well-trained networks, however, the analysis is not guaranteed to work for systems without a fast-slow decomposition.


Previous psychophysical, theoretical, and neurophysiological work has shown that noise in neural activity can cause memories to diffuse away from their original representation, leading to errors in working memory.
Discrete attractor dynamics can counteract this noise by pulling memories towards a few stable representations <cit.>.
The found solutions  reflect the trade-off between robustness to noise and the ability to store continuous variables.
A future line of research could verify whether the size of homeostatic mechanisms to maintain analog working memory as proposed in e.g. <cit.> is in line with our theory.



We would like to extend our heartfelt thanks to Sukbin Lim, Srdjan Ostojic and Daniel Durstewitz for their invaluable feedback and thoughtful suggestions which significantly refined the work.
This work was supported by NIH RF1-DA056404 and the Portuguese Recovery and Resilience Plan (PPR), through project number 62, Center for Responsible AI, and the Portuguese national funds, through FCT - Fundação para a Ciência e a Tecnologia - in the context of the project UIDB/04443/2020.









89


urlstyle

[Ajabi et al.(2023)Ajabi, Keinath, Wei, and Brandon]ajabi2023
Z. Ajabi, A. T. Keinath, X.-X. Wei, and M. P. Brandon.
Population dynamics of head-direction neurons during drift and
  reorientation.
Nature, 6150 (7954):0 892–899, 2023.

[Altan et al.(2023)Altan, Ma, Miller, Perreault, and
  Solla]altan2023low
E. Altan, X. Ma, L. E. Miller, E. J. Perreault, and S. A. Solla.
Low-dimensional neural manifolds for the control of constrained and
  unconstrained movements.
bioRxiv, pages 2023–05, 2023.

[Barak et al.(2013)Barak, Sussillo, Romo, Tsodyks, and
  Abbott]barak2013
O. Barak, D. Sussillo, R. Romo, M. Tsodyks, and L. Abbott.
From fixed points to chaos: Three models of delayed discrimination.
Progress in neurobiology, 103:0 214–222, 2013.

[Barron(2017)]barron2017
J. T. Barron.
Continuously differentiable exponential linear units.
arXiv preprint arXiv:1704.07483, 2017.

[Beiran et al.(2021)Beiran, Dubreuil, Valente, Mastrogiuseppe, and
  Ostojic]beiran2021
M. Beiran, A. Dubreuil, A. Valente, F. Mastrogiuseppe, and S. Ostojic.
Shaping dynamics with multiple populations in low-rank recurrent
  networks.
Neural Computation, 330 (6):0 1572–1615,
  2021.

[Beiran et al.(2023)Beiran, Meirhaeghe, Sohn, Jazayeri, and
  Ostojic]beiran2023parametric
M. Beiran, N. Meirhaeghe, H. Sohn, M. Jazayeri, and S. Ostojic.
Parametric control of flexible timing through low-dimensional neural
  manifolds.
Neuron, 1110 (5):0 739–753, 2023.

[Ben-Yishai et al.(1995)Ben-Yishai, Bar-Or, and
  Sompolinsky]benyishai1995theory
R. Ben-Yishai, R. L. Bar-Or, and H. Sompolinsky.
Theory of orientation tuning in visual cortex.
Proceedings of the National Academy of Sciences, 920
  (9):0 3844–3848, 1995.

[Biswas and Fitzgerald(2022)]biswas2022geometric
T. Biswas and J. E. Fitzgerald.
Geometric framework to predict structure from function in neural
  networks.
Physical review research, 40 (2):0 023255,
  2022.

[Boerlin et al.(2013)Boerlin, Machens, and Denève]Boerlin2013
M. Boerlin, C. K. Machens, and S. Denève.
Predictive coding of dynamical variables in balanced spiking
  networks.
PLoS computational biology, 90 (11):0
  e1003258, Nov. 2013.
ISSN 1553-734X, 1553-7358.
10.1371/journal.pcbi.1003258.

[Boucheny et al.(2005)Boucheny, Brunel, and
  Arleo]boucheny2005continuous
C. Boucheny, N. Brunel, and A. Arleo.
A continuous attractor network model without recurrent excitation:
  Maintenance and integration in the head direction cell system.
Journal of computational neuroscience, 180
  (2):0 205–227, 2005.

[Chaudhuri and Fiete(2016)]chaudhuri2016
R. Chaudhuri and I. Fiete.
Computational principles of memory.
Nature neuroscience, 190 (3):0 394, 2016.

[Chicone(2006)]Chicone2006
C. Chicone.
Ordinary Differential Equations with Applications.
Springer Science & Business Media, Sept. 2006.
ISBN 9780387357942.

[Clevert et al.(2015)Clevert, Unterthiner, and Hochreiter]clevert2015
D.-A. Clevert, T. Unterthiner, and S. Hochreiter.
Fast and Accurate Deep Network Learning by Exponential Linear
  Units (ELUs), 2015.

[Compte et al.(2000)Compte, Brunel, Goldman-Rakic, and
  Wang]compte2000synaptic
A. Compte, N. Brunel, P. S. Goldman-Rakic, and X.-J. Wang.
Synaptic mechanisms and network dynamics underlying spatial working
  memory in a cortical network model.
Cerebral cortex, 100 (9):0 910–923, 2000.

[Cueva et al.(2019)Cueva, Wang, Chin, and Wei]cueva2019headdirection
C. J. Cueva, P. Y. Wang, M. Chin, and X. X. Wei.
Emergence of functional and structural properties of the head
  direction system by optimization of recurrent neural networks.
arXiv preprint, 2019.

[Cueva et al.(2021)Cueva, Ardalan, Tsodyks, and
  Qian]cueva2021continuous
C. J. Cueva, A. Ardalan, M. Tsodyks, and N. Qian.
Recurrent neural network models for working memory of continuous
  variables: Activity manifolds, connectivity patterns, and dynamic codes.
arXiv preprint arXiv:2111.01275, 2021.

[Dayan and Abbott(2001)]dayan2001
P. Dayan and L. F. Abbott.
Theoretical neuroscience: Computational and mathematical modeling
  of neural systems.
2001.

[Driscoll et al.(2022)Driscoll, Shenoy, and Sussillo]driscoll2022
L. Driscoll, K. Shenoy, and D. Sussillo.
Flexible multitask computation in recurrent networks utilizes shared
  dynamical motifs.
bioRxiv, pages 2022–08, 2022.

[Druckmann and Chklovskii(2012)]druckmann2012neuronal
S. Druckmann and D. B. Chklovskii.
Neuronal circuits underlying persistent representations despite time
  varying activity.
Current Biology, 220 (22):0 2095–2103, 2012.

[Ehresmann(1950)]ehresmann1950connexions
C. Ehresmann.
Les connexions infinitésimales dans un espace fibré
  différentiable.
In Colloque de topologie, Bruxelles, volume 29, pages 55–75,
  1950.

[Esnaola-Acebes et al.(2022)Esnaola-Acebes, Roxin, and
  Wimmer]esnaola2022flexible
J. M. Esnaola-Acebes, A. Roxin, and K. Wimmer.
Flexible integration of continuous sensory evidence in perceptual
  estimation tasks.
Proceedings of the National Academy of Sciences, 1190
  (45):0 e2214441119, 2022.

[Fanthomme and Monasson(2021)]fanthomme2021low
A. Fanthomme and R. Monasson.
Low-dimensional manifolds support multiplexed integrations in
  recurrent neural networks.
Neural Computation, 330 (4):0 1063–1112,
  2021.

[Fenichel and Moser(1971)]fenichel1971
N. Fenichel and J. Moser.
Persistence and smoothness of invariant manifolds for flows.
Indiana University Mathematics Journal, 210
  (3):0 193–226, 1971.

[Folland(1999)]folland1999real
G. B. Folland.
Real analysis: Modern techniques and their applications,
  volume 40.
John Wiley & Sons, 1999.

[Frisch(1993)]frisch1993dance
K. v. Frisch.
The dance language and orientation of bees.
Harvard University Press, 1993.

[Fusi and Abbott(2007)]Fusi2007-yg
S. Fusi and L. F. Abbott.
Limits on the memory storage capacity of bounded synapses.
Nature neuroscience, 100 (4):0 485–493, Apr.
  2007.
ISSN 1097-6256,1546-1726.
10.1038/nn1859.

[Ghazizadeh and Ching(2021)]ghazizadeh2021slow
E. Ghazizadeh and S. Ching.
Slow manifolds within network dynamics encode working memory
  efficiently and robustly.
PLoS computational biology, 170 (9):0
  e1009366, 2021.

[Glorot and Bengio(2010)]Glorot2010
X. Glorot and Y. Bengio.
Understanding the difficulty of training deep feedforward neural
  networks.
In Proceedings of the thirteenth international conference on
  artificial intelligence and statistics, pages 249–256. jmlr.org, 2010.

[Goldman(2009)]Goldman2009
M. S. Goldman.
Memory without feedback in a neural network.
Neuron, 610 (4):0 621–634, Feb. 2009.
ISSN 0896-6273, 1097-4199.
10.1016/j.neuron.2008.12.012.

[Goldman et al.(2003)Goldman, Levine, Major, Tank, and
  Seung]Goldman2003-cz
M. S. Goldman, J. H. Levine, G. Major, D. W. Tank, and H. S. Seung.
Robust persistent neural activity in a model integrator with multiple
  hysteretic dendrites per neuron.
Cerebral cortex, 130 (11):0 1185–1195, Nov.
  2003.
ISSN 1047-3211.
10.1093/cercor/bhg095.

[Golubitsky and Stewart()]golubitsky2002symmetry
M. Golubitsky and I. Stewart.
The Symmetry Perspective: From Equilibrium to Chaos in Phase
  Space and Physical Space.
Number 200 in Progress in Mathematics. Birkhäuser.
ISBN 978-3-7643-6609-4.

[Goodridge and Touretzky(2000)]goodridge2000
J. P. Goodridge and D. S. Touretzky.
Modeling attractor deformation in the rodent head-direction system.
Journal of neurophysiology, 830 (6):0
  3402–3410, 2000.

[Gu and Lim(2022)]gu2022
J. Gu and S. Lim.
Unsupervised learning for robust working memory.
PLoS Computational Biology, 180 (5):0
  e1009083, 2022.

[Guillemin and Pollack(2010)]guillemin2010differential
V. Guillemin and A. Pollack.
Differential topology, volume 370.
American Mathematical Soc., 2010.

[Hayou et al.(2019)Hayou, Doucet, and Rousseau]hayou2019
S. Hayou, A. Doucet, and J. Rousseau.
On the impact of the activation function on deep neural networks
  training.
In International conference on machine learning, pages
  2672–2680. PMLR, 2019.

[Hirsch and Baird(1995)]hirsch1995computing
M. W. Hirsch and B. Baird.
Computing with dynamic attractors in neural networks.
Biosystems, 340 (1-3):0 173–195, 1995.

[Hirsch et al.(2013)Hirsch, Smale, and Devaney]hirsch2013differential
M. W. Hirsch, S. Smale, and R. L. Devaney.
Differential equations, dynamical systems, and an introduction
  to chaos.
Academic press, 2013.

[Hulse and Jayaraman(2020)]hulse2020mechanisms
B. K. Hulse and V. Jayaraman.
Mechanisms underlying the neural computation of head direction.
Annual Review of Neuroscience, 43:0 31–54, 2020.

[Jagtap and Karniadakis(2023)]jagtap2023
A. D. Jagtap and G. E. Karniadakis.
How important are activation functions in regression and
  classification? a survey, performance comparison, and future directions.
Journal of Machine Learning for Modeling and Computing,
  40 (1), 2023.

[Jones(1995)]Jones1995
C. K. R. T. Jones.
Geometric singular perturbation theory.
In L. Arnold, C. K. R. T. Jones, K. Mischaikow, G. Raugel, and
  R. Johnson, editors, Dynamical Systems: Lectures Given at the 2nd
  Session of the Centro Internazionale Matematico Estivo (C.I.M.E.) held in
  Montecatini Terme, Italy, June 13–22, 1994, pages 44–118. Springer Berlin
  Heidelberg, Berlin, Heidelberg, 1995.
ISBN 9783540494157.
10.1007/BFb0095239.

[Kakaria and de Bivort(2017)]kakaria2017
K. S. Kakaria and B. L. de Bivort.
Ring attractor dynamics emerge from a spiking model of the entire
  protocerebral bridge.
Frontiers in behavioral neuroscience, 11:0 8, 2017.

[Khona and Fiete(2022)]Khona2022
M. Khona and I. R. Fiete.
Attractor and integrator networks in the brain.
Nature reviews. Neuroscience, 230 (12):0
  744–766, Dec. 2022.
ISSN 1471-003X, 1471-0048.
10.1038/s41583-022-00642-0.

[Kim et al.(2019)Kim, Hermundstad, Romani, Abbott, and
  Jayaraman]kim2019generation
S. S. Kim, A. M. Hermundstad, S. Romani, L. Abbott, and V. Jayaraman.
Generation of stable heading representations in diverse visual
  scenes.
Nature, 5760 (7785):0 126–131, 2019.

[Knierim and Zhang(2012)]knierim2012
J. J. Knierim and K. Zhang.
Attractor dynamics of spatially correlated neural activity in the
  limbic system.
Annual review of neuroscience, 35:0 267–285, 2012.

[Koulakov et al.(2002)Koulakov, Raghavachari, Kepecs, and
  Lisman]Koulakov2002
A. A. Koulakov, S. Raghavachari, A. Kepecs, and J. E. Lisman.
Model for a robust neural integrator.
Nature neuroscience, 50 (8):0 775–782, Aug.
  2002.
ISSN 1097-6256.
10.1038/nn893.

[Lim and Goldman(2012)]Lim2012
S. Lim and M. S. Goldman.
Noise tolerance of attractor and feedforward memory models.
Neural computation, 240 (2):0 332–390, Feb.
  2012.
ISSN 0899-7667, 1530-888X.
10.1162/NECO_a_00234.

[Lim and Goldman(2013)]Lim2013
S. Lim and M. S. Goldman.
Balanced cortical microcircuitry for maintaining information in
  working memory.
Nature neuroscience, 160 (9):0 1306–1314,
  Sept. 2013.
ISSN 1097-6256, 1546-1726.
10.1038/nn.3492.

[Mañé(1978)]mane1978persistent
R. Mañé.
Persistent manifolds are normally hyperbolic.
Transactions of the American Mathematical Society,
  246:0 261–283, 1978.

[Maheswaranathan et al.(2019)Maheswaranathan, Williams, Golub, Ganguli,
  and Sussillo]maheswaranathan2019universality
N. Maheswaranathan, A. Williams, M. Golub, S. Ganguli, and D. Sussillo.
Universality and individuality in neural dynamics across large
  populations of recurrent networks.
Advances in neural information processing systems, 32, 2019.

[Mañé(1987)]mane1987proof
R. Mañé.
A proof of the c^1 stability conjecture.
Publications Mathématiques de l'IHÉS, 66:0
  161–210, 1987.

[Mante et al.(2013)Mante, Sussillo, Shenoy, and
  Newsome]mante2013context
V. Mante, D. Sussillo, K. V. Shenoy, and W. T. Newsome.
Context-dependent computation by recurrent dynamics in prefrontal
  cortex.
nature, 5030 (7474):0 78–84, 2013.

[Mastrogiuseppe and Ostojic(2018)]mastrogiuseppe2018
F. Mastrogiuseppe and S. Ostojic.
Linking connectivity, dynamics, and computations in low-rank
  recurrent neural networks.
Neuron, 990 (3):0 609–623, 2018.

[Morrison et al.(2024)Morrison, Degeratu, Itskov, and
  Curto]morrison2024diversity
K. Morrison, A. Degeratu, V. Itskov, and C. Curto.
Diversity of emergent dynamics in competitive threshold-linear
  networks.
SIAM Journal on Applied Dynamical Systems, 230
  (1):0 855–884, 2024.

[Nair et al.(2023)Nair, Karigo, Yang, Ganguli, Schnitzer, Linderman,
  Anderson, and Kennedy]Nair2023
A. Nair, T. Karigo, B. Yang, S. Ganguli, M. J. Schnitzer, S. W. Linderman,
  D. J. Anderson, and A. Kennedy.
An approximate line attractor in the hypothalamus encodes an
  aggressive state.
Cell, 1860 (1):0 178–193.e15, Jan. 2023.
ISSN 0092-8674.
10.1016/j.cell.2022.11.027.

[Noorman et al.(2022)Noorman, Hulse, Jayaraman, Romani, and
  Hermundstad]Noorman2022
M. Noorman, B. K. Hulse, V. Jayaraman, S. Romani, and A. M. Hermundstad.
Accurate angular integration with only a handful of neurons.
bioRxiv, 2022.
10.1101/2022.05.23.493052.

[Orhan and Ma(2019)]orhan2019diverse
A. E. Orhan and W. J. Ma.
A diverse range of factors affect the nature of neural
  representations underlying short-term memory.
Nature neuroscience, 220 (2):0 275–283, 2019.

[Panichello et al.(2019)Panichello, DePasquale, Pillow, and
  Buschman]panichello2019
M. F. Panichello, B. DePasquale, J. W. Pillow, and T. J. Buschman.
Error-correcting dynamics in visual working memory.
Nature communications, 100 (1):0 3366, 2019.

[Park et al.(2023)Park, Ságodi, and Sokół]Park2023a
I. M. Park, Á. Ságodi, and P. A. Sokół.
Persistent learning signals and working memory without continuous
  attractors.
Aug. 2023.

[Paszke et al.(2017)Paszke, Gross, Chintala, Chanan, Yang, DeVito, Lin,
  Desmaison, Antiga, and Lerer]paszke2017automatic
A. Paszke, S. Gross, S. Chintala, G. Chanan, E. Yang, Z. DeVito, Z. Lin,
  A. Desmaison, L. Antiga, and A. Lerer.
Automatic differentiation in PyTorch.
In NIPS-W, 2017.

[Petrucco et al.(2023)Petrucco, Lavian, Wu, Svara, Štih, and
  Portugues]petrucco2023neural
L. Petrucco, H. Lavian, Y. K. Wu, F. Svara, V. Štih, and R. Portugues.
Neural dynamics and architecture of the heading direction circuit in
  zebrafish.
Nature neuroscience, 260 (5):0 765–773, 2023.

[Pollock and Jazayeri(2020)]pollock2020
E. Pollock and M. Jazayeri.
Engineering recurrent neural networks from task-relevant manifolds
  and dynamics.
PLoS computational biology, 160 (8):0
  e1008128, 2020.

[Prohens and Teruel(2013)]prohens2013canard
R. Prohens and A. E. Teruel.
Canard trajectories in 3D piecewise linear systems.
Discrete Contin. Dyn. Syst, 330 (3):0
  4595–4611, 2013.

[Prohens et al.(2016)Prohens, Teruel, and Vich]prohens2016slow
R. Prohens, A. Teruel, and C. Vich.
Slow–fast n-dimensional piecewise linear differential systems.
Journal of Differential Equations, 2600 (2):0
  1865–1892, 2016.

[Ramachandran et al.(2017)Ramachandran, Zoph, and Le]ramachandran2017
P. Ramachandran, B. Zoph, and Q. V. Le.
Searching for activation functions.
arXiv preprint arXiv:1710.05941, 2017.

[Redish et al.(1996)Redish, Elga, and Touretzky]redish1996coupled
A. D. Redish, A. N. Elga, and D. S. Touretzky.
A coupled attractor model of the rodent head direction system.
Network: Computation in Neural Systems, 70
  (4):0 671–685, 1996.

[Remington et al.(2018)Remington, Narain, Hosseini, and
  Jazayeri]remington2018flexible
E. D. Remington, D. Narain, E. A. Hosseini, and M. Jazayeri.
Flexible sensorimotor computations through rapid reconfiguration of
  cortical dynamics.
Neuron, 980 (5):0 1005–1019, 2018.

[Renart et al.(2003)Renart, Song, and Wang]Renart2003
A. Renart, P. Song, and X.-J. Wang.
Robust spatial working memory through homeostatic synaptic scaling in
  heterogeneous cortical networks.
Neuron, 380 (3):0 473–485, May 2003.
ISSN 0896-6273.
10.1016/s0896-6273(03)00255-1.

[Romo et al.(1999)Romo, Brody, Hernández, and Lemus]Romo1999
R. Romo, C. D. Brody, A. Hernández, and L. Lemus.
Neuronal correlates of parametric working memory in the prefrontal
  cortex.
Nature, 3990 (6735):0 470–473, June 1999.
ISSN 0028-0836.
10.1038/20939.

[Samsonovich and McNaughton(1997)]samsonovich1997path
A. Samsonovich and B. L. McNaughton.
Path integration and cognitive mapping in a continuous attractor
  neural network model.
Journal of Neuroscience, 170 (15):0
  5900–5920, 1997.

[Seeholzer et al.(2017)Seeholzer, Deger, and
  Gerstner]seeholzer2017efficient
A. Seeholzer, M. Deger, and W. Gerstner.
Efficient low-dimensional approximation of continuous attractor
  networks.
arXiv preprint arXiv:1711.08032, 2017.

[Seung(1996)]seung1996
H. S. Seung.
How the brain keeps the eyes still.
Proceedings of the National Academy of Sciences, 930
  (23):0 13339–13344, 1996.
ISSN 0027-8424, 1091-6490.
10.1073/pnas.93.23.13339.

[Seung(1997)]seung1997learning
H. S. Seung.
Learning continuous attractors in recurrent networks.
Advances in neural information processing systems, 10, 1997.

[Seung et al.(2000)Seung, Lee, Reis, and Tank]Seung2000
H. S. Seung, D. D. Lee, B. Y. Reis, and D. W. Tank.
Stability of the memory of eye position in a recurrent network of
  conductance-based model neurons.
Neuron, 260 (1):0 259–271, Apr. 2000.
ISSN 0896-6273.
10.1016/s0896-6273(00)81155-1.

[Shimizu et al.(2021)Shimizu, Yoshida, Kasai, and
  Toyoizumi]shimizu2021
G. Shimizu, K. Yoshida, H. Kasai, and T. Toyoizumi.
Computational roles of intrinsic synaptic dynamics.
70:0 34–42, 2021.
ISSN 09594388.
10.1016/j.conb.2021.06.002.

[Simpson(2018)]simpson2018
D. J. Simpson.
Dimension reduction for slow-fast, piecewise-smooth, continuous
  systems of ODEs.
arXiv preprint arXiv:1801.04653, 2018.

[Sompolinsky et al.(1988)Sompolinsky, Crisanti, and
  Sommers]sompolinsky1988
H. Sompolinsky, A. Crisanti, and H.-J. Sommers.
Chaos in random neural networks.
Physical review letters, 610 (3):0 259, 1988.

[Sussillo(2014)]sussillo2014
D. Sussillo.
Neural circuits as computational dynamical systems.
Current opinion in neurobiology, 25:0 156–163, 2014.

[Sussillo and Barak(2013)]sussillo2013blackbox
D. Sussillo and O. Barak.
Opening the black box: Low-dimensional dynamics in high-dimensional
  recurrent neural networks.
Neural computation, 250 (3):0 626–649, 2013.

[Taube(2007)]taube2007head
J. S. Taube.
The head direction signal: Origins and sensory-motor integration.
Annu. Rev. Neurosci., 30:0 181–207, 2007.

[Tsodyks and Sejnowski(1995)]tsodyks1995associative
M. Tsodyks and T. Sejnowski.
Associative memory and hippocampal place cells.
International journal of neural systems, 6:0 81–86,
  1995.

[Turner-Evans et al.(2017)Turner-Evans, Wegener, Rouault, Franconville,
  Wolff, Seelig, Druckmann, and Jayaraman]turner2017angular
D. Turner-Evans, S. Wegener, H. Rouault, R. Franconville, T. Wolff, J. D.
  Seelig, S. Druckmann, and V. Jayaraman.
Angular velocity integration in a fly heading circuit.
Elife, 6:0 e23496, 2017.

[Turner-Evans et al.(2020)Turner-Evans, Jensen, Ali, Paterson,
  Sheridan, Ray, Wolff, Lauritzen, Rubin, Bock,
  et al.]turner2020neuroanatomical
D. B. Turner-Evans, K. T. Jensen, S. Ali, T. Paterson, A. Sheridan, R. P. Ray,
  T. Wolff, J. S. Lauritzen, G. M. Rubin, D. D. Bock, et al.
The neuroanatomical ultrastructure and function of a biological ring
  attractor.
Neuron, 1080 (1):0 145–163, 2020.

[Vafidis et al.(2022)Vafidis, Owald, D'Albis, and Kempter]vafidis2022
P. Vafidis, D. Owald, T. D'Albis, and R. Kempter.
Learning accurate path integration in ring attractor models of the
  head direction system.
Elife, 11:0 e69841, 2022.

[Vyas et al.(2020)Vyas, Golub, Sussillo, and Shenoy]vyas2020
S. Vyas, M. D. Golub, D. Sussillo, and K. V. Shenoy.
Computation through neural population dynamics.
Annual review of neuroscience, 430 (1):0
  249–275, July 2020.
ISSN 0147-006X.
10.1146/annurev-neuro-092619-094115.

[Wiggins(1994)]wiggins1994
S. Wiggins.
Normally hyperbolic invariant manifolds in dynamical systems,
  volume 105.
Springer Science & Business Media, 1994.

[Wimmer et al.(2014)Wimmer, Nykamp, Constantinidis, and
  Compte]wimmer2014
K. Wimmer, D. Q. Nykamp, C. Constantinidis, and A. Compte.
Bump attractor dynamics in prefrontal cortex explains behavioral
  precision in spatial working memory.
Nature neuroscience, 170 (3):0 431–439, 2014.

[Yang et al.(2022)Yang, Zwart, James, Rubinov, Wei, Narayan,
  Vladimirov, Mensh, Fitzgerald, and Ahrens]yang2022brainstem
E. Yang, M. F. Zwart, B. James, M. Rubinov, Z. Wei, S. Narayan, N. Vladimirov,
  B. D. Mensh, J. E. Fitzgerald, and M. B. Ahrens.
A brainstem integrator for self-location memory and positional
  homeostasis in zebrafish.
Cell, 1850 (26):0 5011–5027, 2022.

[Yang et al.(2019)Yang, Joglekar, Song, Newsome, and
  Wang]yang2019task
G. R. Yang, M. R. Joglekar, H. F. Song, W. T. Newsome, and X.-J. Wang.
Task representations in neural networks trained to perform many
  cognitive tasks.
Nature Neuroscience, 220 (2):0 297–306, 2019.
10.1038/s41593-018-0310-2.

[Zhang(1996)]zhang1996
K. Zhang.
Representation of spatial orientation by the intrinsic dynamics of
  the head-direction cell ensemble: A theory.
Journal of Neuroscience, 160 (6):0 2112–2126,
  1996.




@preprint


§ NEURIPS PAPER CHECKLIST










  * Claims
    
   Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
    
   Answer:  
   Justification: The theoretical and experimental results match and we provide various examples that indicate that they generalize to other settings.
    
   Guidelines:
    
        
  * The answer NA means that the abstract and introduction do not include the claims made in the paper.
        
  * The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
        
  * The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
        
  * It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.
    


  * Limitations
    
   Question: Does the paper discuss the limitations of the work performed by the authors?
    
   Answer:  
   Justification: We report on the limitations of the analysis in the discussion section.
    
   Guidelines:
    
        
  * The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
        
  * The authors are encouraged to create a separate “Limitations” section in their paper.
        
  * The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
        
  * The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
        
  * The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
        
  * The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
        
  * If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
        
  * While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
    


  * Theory Assumptions and Proofs
    
   Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
    
   Answer:  
   Justification: We provide proofs in the supplemental and provide assumptions about the applicability of the theoretical results.
    
   Guidelines:
    
        
  * The answer NA means that the paper does not include theoretical results.
        
  * All the theorems, formulas, and proofs in the paper should be numbered and cross-referenced.
        
  * All assumptions should be clearly stated or referenced in the statement of any theorems.
        
  * The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.
        
  * Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
        
  * Theorems and Lemmas that the proof relies upon should be properly referenced.
    

    
  * Experimental Result Reproducibility
    
   Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
    
   Answer:  
   Justification: We report on all the analysis step decisions that might affect the results.
    
   Guidelines:
    
        
  * The answer NA means that the paper does not include experiments.
        
  * If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
        
  * If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
        
  * Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
        
  * While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
        
            
  * If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.
            
  * If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
            
  * If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
            
  * We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
        
    



  * Open access to data and code
    
   Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?
    
   Answer:  
   Justification: Although we do not provide open access to the data and code, we strive to make it available as soon as possible.
    
   Guidelines:
    
        
  * The answer NA means that paper does not include experiments requiring code.
        
  * Please see the NeurIPS code and data submission guidelines (<https://nips.cc/public/guides/CodeSubmissionPolicy>) for more details.
        
  * While we encourage the release of code and data, we understand that this might not be possible, so No is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
        
  * The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (<https://nips.cc/public/guides/CodeSubmissionPolicy>) for more details.
        
  * The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
        
  * The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.
        
  * At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
        
  * Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.
    



  * Experimental Setting/Details
    
   Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
    
   Answer:  
   Justification: These are reported in detail in the Supplemental.
    
   Guidelines:
    
        
  * The answer NA means that the paper does not include experiments.
        
  * The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.
        
  * The full details can be provided either with the code, in appendix, or as supplemental material.
    


  * Experiment Statistical Significance
    
   Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
    
   Answer:  
   Justification: We do not consider error bars in the paper.
    
   Guidelines:
    
        
  * The answer NA means that the paper does not include experiments.
        
  * The authors should answer “Yes” if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
        
  * The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
        
  * The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
        
  * The assumptions made should be given (e.g., Normally distributed errors).
        
  * It should be clear whether the error bar is the standard deviation or the standard error of the mean.
        
  * It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.
        
  * For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).
        
  * If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
    


  * Experiments Compute Resources
    
   Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
    
   Answer:  
   Justification: Yes, we have reported the computer resources in the Supplemental.
    
   Guidelines:
    
        
  * The answer NA means that the paper does not include experiments.
        
  * The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
        
  * The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
        
  * The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
    


  * Code Of Ethics
    
   Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics <https://neurips.cc/public/EthicsGuidelines>?
    
   Answer:  
   Justification: We have read the NeurIPS Code of Ethics and make sure to preserve anonymity.
    
   Guidelines:
    
        
  * The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
        
  * If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.
        
  * The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).
    



  * Broader Impacts
    
   Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
    
   Answer:  
   Justification: The paper is very theoretical and therefore, we do not expect any direct societal impact in the forseeable future.
    
   Guidelines:
    
        
  * The answer NA means that there is no societal impact of the work performed.
        
  * If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
        
  * Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
        
  * The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
        
  * The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
        
  * If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
    


  * Safeguards
    
   Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
    
   Answer:  
   Justification: We do not rely on any pretrained language models, image generators, or scraped datasets.
    
   Guidelines:
    
        
  * The answer NA means that the paper poses no such risks.
        
  * Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
        
  * Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
        
  * We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.
    


  * Licenses for existing assets
    
   Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
    
   Answer:  
   Justification: Yes, we rely on PyTorch and we cite it.
    
   Guidelines:
    
        
  * The answer NA means that the paper does not use existing assets.
        
  * The authors should cite the original paper that produced the code package or dataset.
        
  * The authors should state which version of the asset is used and, if possible, include a URL.
        
  * The name of the license (e.g., CC-BY 4.0) should be included for each asset.
        
  * For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
        
  * If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, <paperswithcode.com/datasets> has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.
        
  * For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
        
  * If this information is not available online, the authors are encouraged to reach out to the asset's creators.
    


  * New Assets
    
   Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
    
   Answer:  
   Justification: The paper does not release new assets.
    
   Guidelines:
    
        
  * The answer NA means that the paper does not release new assets.
        
  * Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
        
  * The paper should discuss whether and how consent was obtained from people whose asset is used.
        
  * At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
    


  * Crowdsourcing and Research with Human Subjects
    
   Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
    
   Answer:  
   Justification: The paper does not use any data from human subjects.
    
   Guidelines:
    
        
  * The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
        
  * Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.
        
  * According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
    


  * Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
    
   Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
    
   Answer:   
   Justification: The paper does not use any data from human subjects.
    
   Guidelines:
    
        
  * The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
        
  * Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
        
  * We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
        
  * For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
    






§ SUPPLEMENTAL MATERIAL


  








§ BIFURCATION ANALYSIS OF THE BOUNDED LINE ATTRACTOR


In order to demonstrate the implications of the theory of the persistence of bounded continuous attractors, we rigorously test the predictions of the theory on the stability of the BLA.
 Our objective is to assess the practical implications of the theoretical findings of bounded continuous attractors in a small and tractable system, and second, to contribute empirical evidence that can help refine and extend existing theoretical frameworks.


Similarly as for UBLA, the BLA has a parameter that determines step size along line attractor α. Analogously as for UBLA, these parameters determine the capacity of the network.
The inputs push the input along the line attractor in two opposite directions, see below. UBLA and BLA need to be initialized at β(1,1) and β2(1,1), respectively, for correct decoding, i.e., output projection.

    = α[ -1  1;  1 -1 ],  =
    [  0 -1; -1  0 ],  = 1/2α[  1; -1 ],  = β[ 1; 1 ],  = 0.



Parameter that determines step size along line attractor α.
The size determines the maximum number of clicks as the difference between the two channels.
This pushes the input along the line “attractor” in two opposite directions, see below.


We consider all parametrized perturbations of the form ← + for a random matrix ∈ℝ^2× 2 to the BLA.
The BLA can bifurcate in the following systems, characterized by their invariant sets: a system with single stable fixed point, a system with three fixed points (one unstable and two stable) and  a system with two fixed points (one stable and the other a half-stable node) and a system with a (rotated) line attractor.
Only the first two bifurcations (Fig. <ref>A) can happen with nonzero chance for the type of random perturbations we consider.
The perturbations that leave the line attractor intact or to lead to a system with two fixed points have measure zero in the parameter space.
The perturbation that results in one fixed point happen with probability 3/4, while perturbations lead to a system with three fixed points with probability 1/4, see Sec. <ref>.
The (local) invariant manifold manifold is indeed persistent for the BLA and homeomorphic to the original (the bounded line).










  
Stabilty of the fixed point with full support
We investigate how perturbations to the bounded line affect the Lyapunov spectrum.
We calculate the eigenspectrum of the Jacobian:

    [W' -(1+λ)𝕀]    = (ϵ_11-1-λ)(ϵ_22-1-λ)-(ϵ_12+1)(ϵ_21+1)
       =λ^2 - (2+ϵ_11+ϵ_22)λ -ϵ_11-ϵ_22+ϵ_11ϵ_22 -ϵ_12 - ϵ_21 - ϵ_12ϵ_21


Let
u=- (2+ϵ_11+ϵ_22)
and
v=-ϵ_11-ϵ_22+ϵ_11ϵ_22 -ϵ_12 - ϵ_21 - ϵ_12ϵ_21

There are only two types of invariant set for the perturbations of the line attractor. Both have as invariant set a fixed point at the origin. What distinguishes them is that one type of perturbations leads to this fixed point being stable while the other one makes it unstable.














  
Stability of the fixed points
We perform the stability analysis for the part of the state space where Wx>0.
There, the Jacobian is

    J = -
    [ 1 1; 1 1 ]


We apply the perturbation

    W' =
    [  0 -1; -1  0 ]
    + ϵ

with

    ϵ =
    [ ϵ_11 ϵ_12; ϵ_21 ϵ_22 ]


The eigenvalues are computed as

    [W' -(1+λ)𝕀]    = (ϵ_11-1-λ)(ϵ_22-1-λ)-(ϵ_12-1)(ϵ_21-1)
       =λ^2 + (2-ϵ_11-ϵ_22)λ -ϵ_11-ϵ_22+ϵ_11ϵ_22 +ϵ_12 + ϵ_21 - ϵ_12ϵ_21


Let
u=2-ϵ_11-ϵ_22
and
v=-ϵ_11-ϵ_22+ϵ_11ϵ_22 + ϵ_12 + ϵ_21 - ϵ_12ϵ_21


    λ = -u ±√(u^2-4v)/2




Case 1: Re(√(u^2-4v))<-u, then
λ_1,2<0


Case 2:  Re(√(u^2-4v))>-u, then
λ_1<0 and λ_2>0


Case 3: v=0, then
λ=12(-u± u), i.e.,
λ_1=0 and  λ_2=-u


    ϵ_11   = -ϵ_22+ϵ_11ϵ_22 + ϵ_12 + ϵ_21 - ϵ_12ϵ_21




We give some examples of the different types of perturbations to the bounded line attractor.
The first type is when the invariant set is composed of a single fixed point, for example for the perturbation:

    ϵ = 1/10[ -2  1;  1 -2 ]





The second type is when the invariant set is composed of three fixed points:

    ϵ = 1/10[  1 -2; -2  1 ]


The third type is when the invariant set is composed of two fixed points, both with partial support.

    b' =  1/10[  1 -1 ]


The fourth and final type is when the line attractor is maintained but rotated:

    ϵ =  1/20[  1 10; 10  1 ]






 §.§ Bifurcation landscape


All perturbations of the bounded line attractor are of the types as listed above.





We enumerate all possibilities for the dynamics of a ReLU activation network with two units.
First of all, note that there can be no limit cycle or chaotic orbits.

Now, we look at the different possible systems with fixed points.
There can be at most three fixed points <cit.>.
There has to be at least one fixed point, because the bias is non-zero.

General form (example):

    ϵ = 1/10[ -2  1;  1 -2 ]


One fixed point with full support:

In this case we can assume W to be full rank.


    ẋ =
    [
    [ ϵ_11 ϵ_12; ϵ_21 ϵ_22 ][ x_1; x_2 ]
    +
    [ 1; 1 ]]
    -
    [ x_1; x_2 ]   =0



Note that x>0 iff z_1ϵ_11x_1 + (ϵ_12-1)x_2-1>0. Similarly for x_2>0.

So for a fixed point with full support, we have

    [ x_1; x_2 ]
    =A^-1[ -1; -1 ]

with

    A[ ϵ_11-1 ϵ_12-1; ϵ_21-1 ϵ_22-1 ].



Note that it is not possible that x_1=0=x_2.

Now define

    B A^-1 = 1/ A[ ϵ_22-1 1-ϵ_12; 1-ϵ_21 ϵ_11-1 ]

with 
    A = ϵ_11ϵ_22-ϵ_11-ϵ_22-ϵ_12ϵ_21+ϵ_12+ϵ_21.


Hence, we have that x_1,x_2>0 if B_11+B_12>0, B_21+B_22>0 and A >0
or if B_11+B_12<0, B_21+B_22<0 and A <0.

This can be satisfied in two ways,
If A >0, this is satisfied if ϵ_22>ϵ_12 and ϵ_11>ϵ_21,
while if A <0, this is satisfied if ϵ_22<ϵ_12 and ϵ_11<ϵ_21.
This gives condition 1. 



Finally, we investigate the condition that specify that there are fixed points with partial support.
If x_1=0 then (ϵ_22-1)x_2+1=0 and z_1<0.
From the equality, we get that x_2=1/1-ϵ_22.
From the inequality, we get  (ϵ_12-1)x_2+1≥ 0, i.e. 1/1-ϵ_12≥ x_2.
Hence,

    1/1-ϵ_12≥1/1-ϵ_22

and thus

    ϵ_22≤ϵ_12.


Similarly to have a fixed point x^* such that x_2^*=0, we must have that

    ϵ_11≤ϵ_21.


Equation <ref> and <ref> together form condition 2.


Then, we get the following conditions for the different types of bifurcations:


  * If condition 1 is violated, but condition 2 is satisfied with exactly one strict inequality, there are two fixed points on the boundary of the admissible quadrant.

  * If condition 1 is violated, and only one of the subconditions of condition 2 is satisfied, there is a single fixed point on one of the axes.

  * If condition 2 is violated, there is a single fixed point with full support.

  * If both conditions are satisfied, there are three fixed points.



We now look at the possibility of the line attractor being preserved.
This is the case if v=0.
It is not possible to have a line attractor with a fixed point off of it for as there cannot be disjoint fixed points that are linearly dependent <cit.>







  §.§.§ Probability of bifurcation types

We will now calculate which proportion proportion of the bifurcation parameter space is results in the different bifurcation types.
The conditions  that result in three fixed points are

    0    < ϵ_11ϵ_22-ϵ_11-ϵ_22-ϵ_12ϵ_21-ϵ_12-ϵ_21,
    ϵ_22   ≤ϵ_12,
    ϵ_11   ≤ϵ_21.

Therefore, because

    ϵ_22   ≤ϵ_12,
    ϵ_11   ≤ϵ_21.

we always have that

    0    < ϵ_11ϵ_22-ϵ_11-ϵ_22-ϵ_12ϵ_21-ϵ_12-ϵ_21.

This implies that this bifurcation happens with probability 1/4  in a ϵ-ball around the BLA neural integrator with ϵ<1.
We conclude that the single stable fixed point type perturbation happens with probability 3/4.




 §.§ Structure of the parameter space

We will present the structure of the bifurcation space through a slice in which we fix  ϵ_11 and ϵ_12.
First, we summarize which conditions result in which bifurcation in Table <ref>.
We derive that the local bifurcation in this slice  has the structure as shown in Fig. <ref>.






 §.§ Fast-slow form

We will now show how to transform the BLA network to the slow-fast form in Eq. <ref>-<ref>.
We transform the state space so that the line attractor aligns with the y-axis.
So, we apply the affine transformation R_θ(x-1/2) with the rotation matrix R_θ = [  cosθ -sinθ;  sinθ  cosθ ]= 1/√(2)[  1  1; -1  1 ] where we have set θ=-π/4.
So we perform the transformation x→ x'= R_θ(x-1/2) and so we have x=R^-1_θ x'+1/2 with R^-1_θ = R_-θ.
Then we get that

    R_θ^-1ẋ' = ReLU(W(R^-1_θ x`+1/2)+1)-R^-1_θ x'-1/2.

For a perturbed connection matrix W=[  ϵ -1; -1  0 ] we get

    R_θ^-1ẋ'    = ReLU(1/√(2)[  ϵ -1; -1  0 ]([  1 -1;  1  1 ] x`+1/2)+1)-1/√(2)[  1 -1;  1  1 ] x'-1/2
    ẋ'    =[ -1  1;  1  1 ](1/2[  ϵ-1 -ϵ-1;   -1    1 ]x' + 1/2√(2)[ ϵ-1;  -1 ]+[ 1; 1 ]-1/2[ 1; 1 ])-x'
    ẋ'    =([ -2  0;  0  0 ]+ϵ/2[  1 -1; -1  1 ])x' + 1/2√(2)[  ϵ; -ϵ ]




 §.§ Smoother activation functions

It is well-known that activation functions (σ in  Eqs. <ref> and <ref>), which can take many forms, play a critical role in propagating gradients effectively through the network and backwards in time <cit.>.
Activation functions that are C^r for r≥ 1 are the ones to which the Persistence Theorem applies.
The Persistence Theorem further specifies how the smoothness of the activation can have implications on the smoothness of the persistent invariant manifold.
For situations where smoothness of the persistent invariant manifold is of importance, smoother activation functions might be preferable, such as the Exponential Linear Unit (ELU)<cit.> or the Continuously Differentiable Exponential Linear Units (CELU) <cit.>.
































§ RING PERTURBATIONS



To computationally investigate the neighborhood of recurrent dynamical systems that implement continuous attractors, we investigate 5 RNNs that are known a priori to form 1 or 2 dimensional continuous attractors.


We define a local perturbation (i.e., a change to the ODE with compact support) through the bump function Ψ(x) = exp(1/x^2-1) for x<1 and zero outside, by multiplying it with a uniform, unidirectional vector field. All such perturbations leave at least a part of the continuous attractor intact and preserve the invariant manifold, i.e. the parts where the fixed points disappear a slow flow appears.

The parametrized perturbations are characterized as the addition of a random matrix to the ODE.







 §.§ Simple ring attractor

We further analyzed a simple (non-biological)  ring attractor, defined by the following ODE: ṙ = r(1-r),  θ̇= 0.
This system has as fixed points the origin and the ring with radius one centered around zero, i.e., (0,0)∪{(1,θ) | θ∈ [0,2π)}. We investigate bifurcations caused by parametric and bump perturbations of the ring invariant manifold (see Sec. <ref>), which is bounded and boundaryless.
All perturbations maintain the topological structure of the invariant manifold. 








 §.§ Heading direction network

The networks proposed in <cit.> are composed of N heading-tuned neurons whose preferred headings θ_j uniformly tile heading space, with an angular separation of Δθ=2π/N radians.
These neurons can be arranged topologically in a ring according to their preferred headings, with neurons locally exciting and broadly inhibiting their neighbors.
The total input activity h_j of each neuron is governed by:

    τḣ_j = -h_j + 1/N∑_k (W^sym_jk + v_in W^asym_jk)ϕ(h_k)+c_ff,     j=1,…,N,

with

    W^sym_jk = J_I + J_E cos(θ_j - θ_k)
, where J_E and J_I respectively control the strength of the tuned and untuned components of recurrent connectivity between neurons with preferred headings θ_j and θ_j
and v_in is an angular velocity input which the network receives through asymmetric, velocity-modulated weights 
    W^asym =sin(θ_j - θ_k).





  
Fixed points
In the absence of an input (v_in=0) fixed points of the system can be found analytically by considering all submatrices W^sym_σ for all subsets {σ⊂ [n]} with[n]={1,…, N}.
A fixed point x^* needs to satisfy

    x^*= -(W^sym_σ)^-1c_ff

and

    x^*_i<0    for  	  i∈σ.


We bruteforce check all possible supports to find all fixed points.
We use the eigenvalues of the Jacobian to identify the stability of the found fixed points.



  
Connecting orbits
We approximate connecting orbits through numerical integration of the ODE intialized in close to the identified saddle points along the unstable manifold.




  
Measure zero co-dimension 1 bifurcations
Measure zero co-dimension 1 bifurcations of the ring attractor network fall into two types, see Fig. <ref>.






  
Measure zero co-dimension N bifurcation
The limit cycle is the only bifurcation that we found that can be achieved on only a measure zero set of parameter values around the parameter for the continuous attractor.





  
Independence of norm of perturbation on bifurcation
As we can see in Fig. <ref>, the topology of the system is maintained through a range of bifurcation sizes when the bifurcation direction is fixed.









 §.§ Ring attractor approximation with tanh neurons



We investigated the bifurcations around the approximate ring attractor constructed with a symmetric weight matrix for a tanh network  <cit.>.
The functional form of W is the sum of a constant term plus a Gaussian centered at θ_i - θ_j =0:

    W(θ_i - θ_j) = J^- + (J^+ - J^-) exp[ -(θ_i - θ_j)^2/2σ^2],

with the dimensionless parameter J^- representing the strength of the weak crossdirectional connections, J^+ the strength of the stronger isodirectional connections,
 and σ the width of the connectivity footprint.


 Such ring attractor approximations are similar to the ones in <cit.>. However, some have other nonlinearities, e.g.,  the sigmoid is used in <cit.>.




  
Loss of function: Sensitivity of continuous attractors to perturbations
We will show that there are differences at how well approximations perform at different timescales.





We measure how performance of different models for the representation of an angular variable drop as a function of perturbation size Fig. <ref> through the memory capacity metric.
For each perturbation size, we sample a low rank (rank 1,2 or 3) random matrix with norm equal to that perturbation size.
We determine the location of the fixed points through the local flow direction criterion as described in Sec. <ref>
and determine the basin of attraction

    (x^*) {x∈ | lim_t→∞φ(t,x)={x^*}}.

through assesing the local flow direction for 1024 sample points in the found invariant manifold.
This invariant manifold was found to be consistently close the the original invariant ring attractor.
The initial ring had 2N fixed points (N stable, N saddle) on this invariant ring manifold.
The memory capacity of this initial configuration is Nlog(N) for the 2N uniformly spaced fixed points.









 §.§ Ring attractor approximation with a low-rank network

The networks consisted of N firing rate units with a sigmoid inputoutput transfer function <cit.>:

    ξ̇_i(t) = - ξ_i(t) + ∑_j=1^N J_ijϕ(x_j(t)) + I_i,

where x_i(t) is the total input current to unit i,
J_ij = gχ_ij + P_ij is the connectivity matrix,
ϕ(x) = tanh(x) is the current-to-rate transfer function, and I_i is the external, feedforward input to unit i.
gχ is considered unknown except for its statistics (mean 0, variance g^2/N).
A general structured component of rank r≪ N can be written as a superposition of r independent unit-rank terms

    P_ij = m^(1)_i n^(1)_j/N + ⋯ + m^(r)_i n^(r)_j/N,
 and is in principle characterized by 2r vectors m^(k) and n^(k).


We consider structured matrices where the two connectivity pairs m^(1) and n^(1), m^(2) and n^(2) share two different overlap directions, defined by vectors y_1 and y_2. We set:

    m^(1)   = √(Σ^2 - r_1^2)  x_1 + r_1 y_1, 
    
        m^(2)   = √(Σ^2 - r_2^2)  x_2 + r_2 y_2, 
    
        n^(1)   = √(Σ^2 - r_1^2)  x_3 + r_1 y_1, 
    
        n^(2)   = √(Σ^2 - r_2^2)  x_4 + r_2 y_2,

where Σ^2 is the variance of the connectivity vectors and r_1^2 and r_2^2 quantify the overlaps along the directions y_1 and y_2.

We keep the following parameters for the analysis:
Σ=2,
ρ=1.9 and
g∈{0, 0.1}.








 §.§ Embedding Manifolds with Population-level Jacobians

We fit 3 networks with the Embedding Manifolds with Population-level Jacobians (EMPJ) method <cit.>.

The networks are RNNs of N=10 neurons with a tanh activation function and τ = 0.05 time constant.
 To use EMPJ we need to specify our desired k fixed points and m eigenvectors and eigenvalues per fixed point.

To do this we choose k=3 points {(x_i, y_i)}_i=1^N in the 2D ring with radius r=1 that are equally spaced from 0.1 to 2 π radians. Then we specify the vectors orthogonal to the ring in those points (V_o = {(x_i, y_i)}_i=1^N, same as the points) and tangent (V_t = {(-y_i, x_i)}_i=1^N).
Finally we associate the negative eigenvalue -1/τ to the orthogonal eigenvectors and different eigenvalues to the tangent eigenvectors. We use -1, 0 or 1 depending on whether we want the points to be stable, center or unstable in the direction of the slow ring manifold.

Since these determined dynamics are only in a 2D plane, we use a random linear mapping D^2→^N to map the fixed points and the eigenvectors to a 10-dimensional space. First, we sample a random from orthogonal matrix A from the O(N) Haar distribution and then we take the first two columns of this matrix to be D=[A]_12.




EMPJ then returns a network parameters that satisfy these constraints. Furthermore, due to the particular solver used, that regularises the magnitude of the parameters, we get that all the other eigenvalues not specified are also set to -1/τ (for details see <cit.>).




  
Finding fixed points
As we remark in Sec. <ref>, EMPJ networks are not robust to S-type noise, therefore we cannot apply our analysis of identifying the invariant set through the convergence criterion of numerically integrated trajectories.
We therefore find fixed points through the Newton-Raphson method.
We iteratively solve

    (_i)_i = _i

where (_i) is the Jacobian of the system at _i and _i+1=_i-_i.
The iteration stops when ||<δ for a tolerance threshold δ.
We initialize _0 on the invariant ring uniformly.
The maximum number of iterations was set to 10.000 and the tolerance level to δ=10^-8.



  §.§.§ Lack of S-type robustness

We remark that the resulting invariant manifold is not robust to S-type perturbations.
Althought the fixed points that are constrained in the fitting procedure are attractive in all directions, some of the points along the ring might not be.
For on-manifold perturbations (in the plane in which the ring is embedded), S-type perturbations do lead to flow towards the invariant ring (Fig. <ref>A).
However, for (small) off-manifold perturbations, the trajectories typically diverge away from the invariant ring (Fig. <ref>B).
This indicates that the basin of attration is very small and hence this approximation is not robust to S-type noise.


All of the perturbations were sampled as
x(0) = x(0) + η with x(0)∈span(D).
For the on-manifold perturbations η∈span(D)  and η_2=10^-2.
For the off-manifold perturbations η∈^10 and η_2=10^-5.


























§ PERSISTENCE THEOREM

Understanding the long-term behavior of dynamical systems is a fundamental question in various fields, including mathematical biology, ecological modeling, and neuroscience.
 Fenichel's Persistence Theorem provides critical insights into the behavior of such systems, particularly in relation to the stability and persistence of invariant manifolds under perturbations.

Fenichel's Persistence Theorem extends the classical theory of invariant manifolds, offering conditions under which normally hyperbolic invariant manifolds persist despite small perturbations.
This theorem is particularly powerful in analyzing systems where perturbations are inevitable, providing a  framework for understanding how qualitative features of the system's dynamics are maintained.
In this section, we delve into the specifics of Fenichel's Persistence Theorem, outlining its key components, assumptions, and implications.




  
Invariance
One of the main concepts in the Persistence Theorem is the notion of an invariant manifold. 
Intuitively, this just means that trajectories stay inside the manifold for all time.
Local invariance is a bit more involved and allows for the possibility of leaving the manifold, but only through it's boundary.


A set  is locally invariant under the flow from Eq. <ref>-<ref> if it has neighborhood V so that no trajectory can leave  without also leaving V.
In other words, it is locally invariant if for all x ∈, φ(x, [0, t] ⊂ V implies that  φ(x,[0, t]) ⊂ M, similarly with [0, t] replaced by [t, 0] when t < 0.



A set S is said to be forward invariant under a flow φ_t if for every point x in S and for all t ≥ 0, the image of x under the flow at time t, denoted φ_t(x), remains in S. This can be written as:


    φ_t(x) ∈ S   for all   x ∈ S   and   t ≥ 0.







  
Small perturbation
In the context of Fenichel's Persistence Theorem, a "sufficiently small C^r perturbation of the vector field f" refers to perturbations that are small in the C^r norm. The C^r norm measures the size of a function and its derivatives up to order r.

Formally, let f: ℝ^n →ℝ^n be the original smooth vector field and let f̃ be a perturbed vector field. The perturbation f̃ is a sufficiently small C^r perturbation if the difference f̃ - f has a small C^r norm. Mathematically, this can be expressed as:


    f̃ - f _C^r < ϵ,

where ϵ is a small positive number, and the C^r norm is defined as:

    f̃ - f _C^r = max_0 ≤ k ≤ rsup_x ∈ℝ^n D^k (f̃(x) - f(x)) .

Which gives a constraint on how much each of the k-th derivatives D^k of the perturbed vector field can differ from the original.

In summary, a perturbation f̃ is considered sufficiently small in the C^r sense if the difference between f̃ and f, along with their derivatives up to order r, is uniformly small across the domain.




 §.§ Fenichels' Invariant Manifold Theorem


System <ref>-<ref> can be reformulated with a change of time-scale as

    ϵ x'    = g ( x , y , ϵ) 
    
    y'   = h(x, y, ϵ)

where  '=d/dτ and τ=t/ϵ.
 The time scale given by τ is said to be fast whereas that for t is slow, as long as ϵ≠ 0 the two systems are equivalent.


The functions g and hin Eq. <ref>-<ref> are both assumed to be C^r (for r>0 on a set U× I where U⊂^d is open, and I is an open interval, containing 0.

Suppose that the set _0  is a subset of the set {h(x, y, 0) = 0} and is a compact manifold, possibly with boundary, and is normally hyperbolic relative to <ref>.



If ϵ >0, but sufficiently small, there exists a manifold _ϵ that lies within 𝒪(ϵ) of _0 and is diffeomorphic to _0.
 Moreover it is locally invariant under the flow of Eq. <ref>-<ref>, and C^r, including in ϵ, for any 0<r< ∞.



If the invariant manifold _0 is attractive, then the only way trajectories can escape the invariant set _ϵ after perturbation, is in negative time through the boundaries.
This guarantees that the persistent manifold _ϵ is still attractive.




 §.§ Fundamental limitations of the topology of bifurcated continuous attractors





  §.§.§ Flow on a ring

The flow on a ring can be described by the differential equation<cit.>:

    dθ/dt = f(θ),

with θ∈[0,2π].

The simplest type of flow on a ring is the uniform circular flow, where each point moves with a constant angular velocity f(θ)=ω.
If |f(θ)|>0 for all θ, there is a circular flow with a variable speed.

There is a fixed point for each unique θ for which f(θ)=0. The nature of the flow around these fixed points can be classified into:
Stable fixed points: Points where the flow tends to as time progresses (f(θ)>0 for all θ∈[0,θ]∩ V and f(θ)<0 for all θ∈[θ,2π]∩ V for some open ball V around θ).
Unstable fixed points: Points from which the flow diverges (f(θ)<0 for all θ∈[0,θ]∩ V and f(θ)>0 for all θ∈[θ,2π]∩ V for some open ball V around θ).

For a detailed discussion of how bifurcations of a continuous attractors depend on its symmetry see for example The Equivariant Branching Lemma (Lemma 1.31 in <cit.>).
If the solutions are symmetric under rotations (a circular symmetry such as for a ring attractor).
As you change the bifurcation parameter, you find that new solutions appear that also respect this rotational symmetry.
The lemma tells you that these new solutions will align with specific symmetries (like different rotation angles), which are described by the irreducible representations of the symmetry group.



 §.§ Consequences to system identification

Fenichel's Persistence Theorem has several significant implications for modelling and system identification in dynamical systems.
Because the theorem provides a guarantee that small perturbations in the system do not lead to significant changes in the qualitative behavior of the system,
we can be (slightly) wrong for example about the exact nonlinearity of a neuron's transform function.
For example, if neurons are only approximately ReLU the theory developed in <cit.> still holds (at the behaviorally relevant timescales).






§ NEAR PERFECT ANALOG MEMORY SYSTEMS ARE CLOSE TO CONTINUOUS ATTRACTORS

We will give some clarifications and proofs of the claims on systems near perfect analog memory systems.



 §.§ Revival of the continuous attractor from a slow manifold


We  provide a proof of Prop. <ref> which is dependent on the ϵ-Neighborhood Theorem<cit.>[Not to be confused with the perturbation parameter ϵ], which we state here.


Let Y ⊆ℝ^n be a smooth, compact manifold. Then for a small enough ϵ > 0,
each w ∈ Y_ϵ := { w ∈ℝ^n |∃ y ∈ Y : y - w < ϵ}
has a unique closest point in Y, denoted π(w).
Moreover, π: Y_ϵ→ Y is a submersion with π|_Y = id.




We now prove  Prop. <ref>.
It is sufficient to show that there is a perturbation  that has zero flow off of _ϵ but for which +=0 on _ϵ for the full system   as defined in Eq. <ref>.
Define

    () = ∫__ϵ - δ(-)(),

 with the Dirac delta function δ()=δ(x_1)δ(x_2)…δ(x_d).
It is then easy to check that ()+()=0 for all ∈_ϵ and ()+()=() for all ∉_ϵ.
Hence, we have a continuous attractor at _ϵ.
If smoothness is important, we can construct the following perturbation.
From the ϵ-Neighborhood Theorem<cit.>, we get that  there exists a smooth positive function δ_ϵ→^+,
 such that if we let N_δ be the
δ-neighborhood of _ϵ,

    M_ϵ{y  ∈^n : |y - x| < δ(x)  for some   x ∈ M_ϵ},

then each y∈ N_δ possesses a unique closest point π_δ(y) in _ϵ with the map π_δ_δ→_ϵ being a submersion.

We can then define a bump function

    ψ(y) =
    exp(-1/1-(π_δ(y)-y)^2)      if   y∈(-δ-π_δ(y), δ-π_δ(y))
    
    0      otherwise


Then the perturbation

    () = - ψ()(π_δ())

is smooth and creates a continuous attractor at _ϵ.







 §.§ Output mapping
 This paper focuses on a linear output mapping for simplicity.
Errors can be minimized with a nonlinear mapping.
However, this can only be applied for error off of the output manifold.
If there is memory degradation along the output manifold, it is not possible to choose another mapping that corrects for this error.




 §.§ Approximate solutions  to an analog working memory problem

We will now dicuss the conditions for when approximate solutions to an analog working memory problem are near a continuous attractor.
We consider approximate solutions to an analog working memory problem to be systems of the form <ref> or <ref> (in both cases following a linear decoder), which have a small memory error over time in output space.






  §.§.§ Robustness

Noise, practically defined as unpredictable components of the system's behavior, comes from many sources.
The concept of S- and D-type noise is based on <cit.>.




  
S-type robustness
S-type noise encapsulates reversible changes in the neural state such that the deterministic part of the dynamics itself remains unchanged.




  
D-type robustness
D-type noise, in the space of recurrent network dynamics parameterized by the synaptic weights.
This corresponds to slight changes to the ODE, i.e. perturbations.
Previously, it robustness to D-type noise was considered to correspond to structurally stability <cit.>.

Here we shortly discuss what we consider to be a necessary and sufficient condition for D-type robustness.
We can use the concept of Lipschitz persistence to define D-type robustness.
Mañe showed that if an invariant manifold is Lipschitz persistent then it must be normally hyperbolic <cit.>.
To understand the concept of Lipschitz persistence, we need to define the Lipschitz section and Lipschitz constant.


Let M be a C^∞ boundaryless manifold and V ⊂ M a C^1 compact boundaryless submanifold.
Assume that M is a submanifold of ℝ^n. Let NV be a C^1 subbundle of TM|_V satisfying TV ⊕ NV = TM|_V.
If η is a section of NV, define the Lipschitz constant of η by

    Lip(η) = sup{η(x) - η(y)/x - y| x, y ∈ V, x ≠ y }.

We say that η is a Lipschitz section if Lip(η) < +∞.
Let Γ_ℒ(NV) be the space of Lipschitz sections of NV endowed with the norm

    η_ℒ = sup{η(x)| x ∈ V } + Lip(η).

Let Diff^1(M) be the space of C^1 diffeomorphisms with the topology of the C^1 convergence on compact subsets.



 Let f ∈Diff^1(M).
 We say that V is a Lipschitz persistent invariant manifold of f if there exists a neighborhood U of V such that for all δ > 0 there exists a neighborhood 𝒰_δ of f such that if g ∈𝒰_δ there exists η∈Γ_ℒ(NV) with η_ℒ < δ satisfying V_g = graph(η), where graph(η) = {exp_x(η(x)) | x ∈ V}, V_g = g(U).

Observe that this definition implies V_f = V, hence f(V) = V. Moreover, the Lipschitz persistence is independent of the bundle NV.



For a flow φ_t (coming from the solutions of an ODE) we can fix t=τ∈_>0 so that we get a homeomorphism φ_τ.
This allows us to apply this result to apply to our case.


In the case where V is a point, then it is a hyperbolic fixed point and the persistence follows trivially from the implicit function theorem.



It is sufficient to take a persistent manifold that is uniformly locally maximal.
If V is persistent and uniformly locally maximal, then V has to be normally hyperbolic <cit.>.



 There exist neighborhoods U of V in M and U of f in the space Diff^1(M) of C^1-diffeomorphisms of M, such that for any g ∈ U, N_g = ⋂_k ∈ℤ g^k(U) is a C^1-submanifold close to V, with N_f = N.
 The latter property implies the uniqueness of the invariant submanifold.







 §.§ Near Perfect Analog Memory Systems are close to Decomposable Systems with a Continuous Attractor


We now prove a more general statement about the kind of systems that are close to perfect analog memory systems.
The theory guarantees that a system that satisfies conditions <ref>-<ref>  will have a continuous attractor in the following sense.
For such a system there exists a decomposition such that the system can be effectively decomposed into a continuous attractor (attractive invariant manifold with zero flow) and a component on which a (possibly “fast” flow) can exist buy which get quenched by the out put projection.

For this part of the theory, we need to consider the output manifold _output, the manifold on which we determine the error over time as in Sec. <ref>.
For this section, we will consider the output mapping to be a smooth (possibly nonlinear) mapping g X→ Y between the neural state space X=^d_X and the output space Y=^d_Y.
For a circular variable this will be the ring S^1.

The construction of a perturbation relies on finding the necessary minimal structure in the invariant manifold for which we can guarantee closeness to a continuous attractor.
Therefore, first of all, we need this closeness in terms of the geometry of the manifold, which we guarantee through the notion of a fibration of the output mapping.
Second, we need to guarantee that the flow is bounded in a sense so that our perturbation is also bounded by this amount.
We will characterize this by the vector field normal to the fibers of the fiber bundle.
A fiber bundle is a mathematical structure that allows us to study spaces that are locally like a product space but globally may have a different structure.
So we first state the definition of a fiber bundle and a trivial fibration.


A fiber bundle is a structure (E, B, π, F) where:

    
  * E is the total space,
    
  * B is the base space,
    
  * π: E → B is a continuous surjection called the projection map, and
    
  * F is a topological space called the fiber.

This structure must satisfy the local triviality condition: for each b ∈ B, there exists an open neighborhood U of b such that there is a homeomorphism 

    φ: π^-1(U_b) → U_b× F_b

that commutes with the projection onto U, meaning that the following diagram commutes:

    π^-1(U_b) [rightarrow]rφ[swap]dπ    U_b× F_b[rightarrow]dpr_1
    
            U_b[rightarrow]rid   	 	U_b

where pr_1: U_b× F_b→ U_b is the projection onto the first factor.


So, around every point in the base space, you can "zoom in" and see that the bundle looks like a straightforward product of the base space and the fiber and φ is providing this trivialization.


We say that a projection map π:E →  B is locally trivial if each point b ∈ B is contained in an open set U having the property that E_Uπ^-1(U) is trivial over U.


We will rely on the concept of a submersion to charaterize how the output space needs to relate to our invarinat manifold.

Let M and N be differentiable manifolds and f M→ N be a differentiable map between them.
 The map f is a submersion at a point p∈ M if its differential Df_p T_pM→ T_f(p)N is a surjective linear map.




This allows us to characterize what kind of structure the invariant manifold needs to have, namely it can be see as the direct product of the output manifold and some other manifold which described over what part of the invariant manifold the output mapping is invariant.

If a smooth mapping f M→ N, where M and N are smooth manifolds, is
 

  * a surjective submersion, and

  * a proper map

then it is a locally trivial fibration.



If a manifold M is compact, then the above smooth map  is a proper map.
If we do not have a submersion or that the output mapping is transversal to the output manifold, we have a situation in which the flow on the invariant manifold can be arbitrarily fast  (even though memory is degrading slowly).
This happens when the flow is in a singularity of the output mapping.


For our statement we want that the invariant manifold can be decomposed into a space that is diffeomorphic to the output manifold and another compact manifold: _ϵ= _slow×_null with _slow≃_Y. We can relax this to allow for the possibility of some torsion along the invariant manifold.
For this to hold, it is sufficient that the output mapping must be a submersion because this makes it a locally trivial fibration.
So we get that g X→ Y defines a locally trivial fiber mapping.


The second assumption we need is to have a bound for the speed of trajectories along the fibers, which will correspond to a speed along the output manifold, resulting in memory degradation.
We need to assume that there is a slow flow (in the direction of _slow).
We characterize the relevant maximal size of the vector field that needs to be perturbed as the  supremum over the uniform norms of the vector field normal to the fibers of the fiber bundle.



Let _ϵ be a connected, compact, normally hyperbolic slow manifold  (as parametrized by Eq. (<ref>)-(<ref>)) with the real part of the eigenvalues of ∇_ all negative. Further, assume that this manifold can be decomposed _ϵ= _slow×_null with _slow≃_Y and that
 the uniform norm of the flow tangent to _ϵ restricted to _slow be (̇)̇_slow_∞=η.
Then, there exists a perturbation with uniform norm at most η that induces a bifurcation to a system that is decomposable into a continuous attractor and a system with a non-zero flow.


In other words, after applying this perturbation, the slow component of the perturbed system satisfies '|_slow = 0 for the system ' = () + ().
Furthermore, the trajectories of the resulting system form a fiber bundle where the output projection serves as the bundle projection.
Each fiber consists of trajectories that are mapped to the same value in _output, meaning that the fibers describe an invariance under the output projection. 



Assume that the output mapping is a submersion from the invariant manifold _ϵ to the output manifold _output.
Ehresmann's lemma implies that this implies that we have a locally trivial fibration (_ϵ, _output, g,_null).
We construct the perturbation  as the part of the vector field that us normal to the fibers g^-1(y) for each y∈_output.
By construction, this perturbation has uniform norm at most η.
This perturbation makes the vector field normal to each fiber zero.
That implies that each fiber is an invariant submanifold of _ϵ.
From the structure of the fiber bundle it follows that there is a continuum of such invariant submanifolds.
Hence, the perturbed system is decomposable into a system that is a continuous attractor and a system with a non-zero flow.





  
Remark
This perturbation can be made smoothness, along similar lines as above.
We can take aϵ-Neighborhood around the invariant manifold on which we extend the above vector field with bump functions to get a smooth vector field that still results in an attractive invariant manifold.




  
Example
An example of an approximate continous attractor solution of the form with a decomposable system of which one of the subsystems is close to a continuous attractor is the torus solution in Fig. <ref>D.
In this case, the system can be perturbed slightly such that there exists a continuum of limit cycles laid out over a ring.


















§ UPPER BOUND FOR THE MEMORY PERFORMANCE ON A SHORT TIME SCALE

We will now formalize the statement about an upper bound dependent on the uniform norm of the vector field on the slow manifold in Sec.<ref> and provide a proof.

Let  be a normally hyperbolic slow manifold as in Prop. <ref>.
Let _0 ∈, and φ = |_ be the flow restricted to the manifold.
The average deviation from initial memory _0 over time is bounded linearly

    1/vol∫_(t, _0) - _0 _0≤ tφ.




Numerical integration of the ODE gives

    x(t,_0)    = ∫_0^tφ(x(τ))dτ + _0 
       ≤∫_0^tφ_∞ dτ + _0 
       = tφ_∞+ _0

From this, we get

    1/vol∫_(t, _0) - _0 _0   ≤1/vol∫_ tφ_∞
       = tφ.





We formulate here a theory of continuous attractor approximations in terms of memory loss over time.
It can be used uniform norm of vector field on the manifold to bound the memory performance on the short-time scale. Let _0 ∈, and φ = |_ be the flow restricted to the manifold.
We will show that the average deviation from initial memory _0 over time is bounded linearly as in Eq. <ref>.



 §.§ Ring attractor

For a ring attractor we can give more tight bounds on the accumulated error for the angular memory.
Suppose we have a dynamical system x∈ℝ^N with autonomous dynamics ẋ = F_θ (x) and solutions (t, _0) uniquely defined for each _0.
Let us define the error (i.e.the  average deviation from initial memory) for an attractor as

    ℒ(T) 1/vol∫_(t, _0) - _0 _0


If we further assume that the memory is not simply the state of the network, we need to take into consideration a decoder of the memory.
Suppose that there is an invertible decoder mapping f: 𝒰→ℝ^N. For a ring variable, we can take this to be the projection onto the plane  and then applying the arctan:

    g(x) = arctan( x).



In the case of the ring attractor, the memory we would like to encode is α∈𝒰 = [0,2 π), the error is defined as |x-y|_o = o_π(|x-y|) where: 
    o_π(x) = 
        x    if  x < π
    
        2 π - x    if  x ≥π


If we call α̂_θ(α_0, t) = g(φ_θ(f(α),t)) we get the expression of the memory loss for this kind of memory as:


    ℒ(T) =  1/2 π∫_0^2 π( | α̂_θ(α_0, t) - α_0  | )  dα_0













General bounds

We can call:


    ϵ^+(t)    = sup_α_0  o_π( | α̂_θ(α_0, t) - α_0  | ) ≥1/2 π∫_0^2 π  o_π( | α̂_θ(α_0, t) - α_0  | )  
    ϵ^m(t)    = 1/2 π∫_0^2 π  o_π( | α̂_θ(α_0, t) - α_0  | ) dα_0 
    ϵ^-(t)    = inf_α_0  o_π( | α̂_θ(α_0, t) - α_0  | ) ≤1/2 π∫_0^2 π  o_π( | α̂_θ(α_0, t) - α_0  | )


Then we get the bounds for the loss ℒ(T) as:


    1/T∫_0^Tϵ^-(t)dt ≤ℒ(T) = 1/T∫_0^Tϵ^m(t) dt ≤1/T∫_0^Tϵ^+(t) dt



Speed bounds

We can define the maximum, average and minimum memory error speed as:


    v_ϵ^+   = sup_α_0 d/dt (o_π( | α̂_θ(α_0, t) - α_0  | )) |_t = 0
    
        v_ϵ^m   = 1/2 π∫_0^2 πd/dt (o_π( | α̂_θ(α_0, t) - α_0  | ))|_t = 0 dα_0 
    
        v_ϵ^-   = inf_α_0 d/dt (o_π( | α̂_θ(α_0, t) - α_0  | ))|_t = 0


Notice then that since:


    ϵ^+(t)    ≤min(t v_ϵ^+, π) 
    ϵ^-(t)    ≥ t v_ϵ^-

then,

    1/T∫_0^Tϵ^+(t)dt    ≤min( 1/T∫_0^T t v_ϵ^+ dt, π) = min( T v_ϵ^+/2, π) 
    1/T∫_0^Tϵ^-(t)dt    ≥1/T∫_0^T tv_ϵ^-dt  = T v_ϵ^-/2

and we get:


    T v_ϵ^-/2≤1/T∫_0^Tϵ^-(t)dt ≤ℒ(T) = 1/T∫_0^Tϵ^m(t) dt ≤1/T∫_0^Tϵ^+(t) dt ≤min( T v_ϵ^+/2, π)


Finally, if the error is uniform enough we can expect ϵ^m(t) ≈ tv_ϵ^m and


    ℒ(T) = 1/T∫_0^Tϵ^m(t) dt ≈1/T∫_0^T t v_ϵ^mdt = T v_ϵ^m/2




  
Within manifold case
Let's assume that we have managed the system F_θ  have a slow manifold ∈ℝ^N in bijection with 𝒰, i.e. f|_ is not a mapping but a bijective function and:


    ∀x∈  ẋ = ϵ_θ(x)∂ f/∂α(f^-1(x))/||∂ f/∂α(f^-1(x))||




Then we have a slow manifold in the form of a ring attractor, we have f(0) = f(2 π) and:


    α̂_θ(α_0, t) = (α_0 + ∫_0^t ϵ_θ(α_0, s) ds ) 2 π



Then:

    α̂_θ(α_0, t)    = o_π( | (α_0 + ∫_0^t ϵ_θ(α_0, s) ds ) 2 π - α_0 | )  
       = o_π( | (α_0 + ∫_0^t ϵ_θ(α_0, s) ds ) 2 π - α_0  2 π| ) 
       = o_π( | ∫_0^t ϵ_θ(α_0, s) ds  2 π| )

where we used that α_0 ∈ [0,2 π) ⇒α_0 = α_0  2 π and that |x  2 π - y  2 π| = |(x-y)  2 π|. 

The final equation of the loss in this case has the form:

    ℒ(T) =  1/2 π∫_0^2 π1/T∫_0^T o_π( | ∫_0^t ϵ_θ(α_0, s) ds  2 π| ) dt dα_0.


Slow manifold bounds

In this case, if we have N fixed points in the ring-like slow manifold, we know that:

    ϵ^+(t)    ≤min( 2 π/N, π),

and therefore:

    ℒ(T) ≤min( 2 π/N, π).






§ SLOW MANIFOLD IN TRAINED RNNS



 §.§ Tasks






  
Memory guided saccade task

The total time length of a trial is 512 steps.
The time delay to the output cue was sampled from

    T_delay∼𝒰(50, 400).

We applied a mask m_i,t=0 for 5 time steps (t=T_delay+j for j=0,… 4) after the go cue.





  
Angular velocity integration task
The time length of a trial is 256 steps.
The input is an angular velocity and the target output is the sine and cosine of the integrated angular velocity.
Velocity at every timestep  is sampled from as a Gaussian Process (GP) for smooth movement trajectories, consistent with the observed animal behavior in flies and rodents.

    k(x,y)=exp(-x-y/2ℓ^2),

with length scale ℓ.
 The length scale of the kernel was fixed at 1.

 




 §.§ Output projection of the invariant manifold





The output projection of the invariant manifolds and stability structures (fixed points and limit cycles) is very close to the target output circle.




 §.§ Training methods


 We trained vanilla RNNs with PyTorch <cit.> with a tanh nonlinearity for the recurrent layer and a linear output layer on the angular velocity integration task Fig. <ref>.
The parameters were initialized were initialized using the Xavier normal distribution <cit.>.
For the recurrent weights we used W_ij∼𝒩(0, g/√(N)) with a high gain g=1.5.
The initial hidden state was initialized using the output to recurrent mapping matrix W_otr^2→^N which was trained together with the other parameters.

Adam optimization with β_1=0.9 and β_2=0.999 was employed with a batch size of 512 and training was run until no decrease in the loss occurred for 1000 epochs.
The task has 100 time steps for the training samples, and the mean squared error loss was used.
The batches were generated on-line, similar to how animals are trained with a new trial instead of iterating through a dataset of trials.


The best learning rate was chosen from a set of values {10^-2,10^-3,10^-4,10^-5} by 5 initial runs for all nonlinearity and size pairings with the lowest average loss after 100 gradient steps.
Training a single network took around 10 minutes on a CPU and occupied 10 percent of an 8GB RAM.


 








 §.§ RNN analysis methods





  §.§.§ Evaluation Metric

After training, we report the normalized mean squared error (NMSE) to asses the how good a found solution is:

    NMSE = 𝔼[(y-ŷ)^2]/𝔼[y],

where y is the target and ŷ is the prediction.


 

  §.§.§ Fast-slow decomposition

 We simulated 1024 trajectories without noise with inputs from the task and let the networks evolve for 16 times the task definition lengths.
 We took the cutoff to identify the slow manifold to be 10^-3 of the highest speed along each trajectory.
 We believe that this guarantees the identification of the slow manifold in a system that has a fast-slow decomposition.
 We sampled 1024 points from these points to fit a periodic, cubic spline (black line in Fig. <ref>, <ref>, <ref>).



We use the eigenvalue spectrum as evidence for normal hyperbolicity.
Normal hyperbolicity of an attractive ring invariant manifold implies that the eigenvalue spectrum has a gap in its eigenvalue spectrum.
To measure this, we linearize at reference points on the invariant manifold (calculate the Jacobian) and calculate the eigenvalues.
The largest  eigenvalue (real part) for such a manifold needs to be much closer to zero than the second largest.



For a stable system, where the eigenvalues have negative real parts, the time constant τ is given by the negative inverse of the eigenvalue's real part: τ = -1/(λ),
where (λ) denotes the real part of the eigenvalue λ.






  §.§.§ Identified invariant manifold output projections

 The identified invariant manifolds in the trained networks (Fig. <ref> and Fig. <ref>).
 However, not all solutions can be meaningfully analyzed with this the slow-fast decomposition method.
 For example, the solution at the center of the tanh, N=256 block, the found invariant manifold is not correctly captured.
 This is true for the networks that have not learned the task correctly (networks with a NMSE higher than -20dB).

 










  §.§.§ Vector field on invariant manifold


 We assess the vector field for the ODE (Eq.  <ref> without noise and input) on the found invariant manifold  by calculating it in the state space
 and then projecting it onto the output space:
 
    α̇=   f(α̂)

for sampled points α̂∈.
These points α̂∈ on the manifold are associated with the points on the ring through the mapping α = α̂.


 

This vector field in the output space captures in what direction and how quickly angular memory will decay.
 The vector field  suggests that the system indeed has an invariant manifold (Fig. <ref>).
 Furthermore, the vector field and fixed points are consistent with each other, as the vector field flips direction around found fixed points.


 There are some inconsistencies around saddle nodes, where the vector field seems to point off of the manifold.
 This is probably just inaccuracies  coming from numerically calculating the vector field and the exact location of the invariant manifold.
For the bound discussed in Sec. <ref>, we calculate the uniform norm of the found vector field 
    f_∞ = sup_α f(α)
 (see also Sec. <ref>).














 Paper Title: High Performance Im2win and Direct Convolutions using Three Tensor Layouts on SIMD Architectures.








High Performance Im2win and Direct Convolutions using Three Tensor Layouts on SIMD Architectures
    

    Xiang Fu1
    Xinpeng Zhang1
    Jixiang Ma1
    Peng Zhao2
    Shuai Lu1
    Xu T. Liu340000-0003-3980-9803This work is not related to the author's position at Amazon.
    
    
    1Nanchang Hangkong University, China
    2Microsoft, China
    3University of Washington, USA
    4Amazon Web Services, USA
    

    August 5, 2024
==================================================================================================================================================================================================================================================================================================================================



§ ABSTRACT 
Convolution is the core component within deep neural networks and it is computationally intensive and time consuming. Tensor data layouts significantly impact convolution operations in terms of memory access and computational efficiency. Yet, there is still a lack of comprehensive performance characterization on data layouts on SIMD architectures concerning convolution methods. This paper proposes three novel data layouts for im2win convolution: NHWC, CHWN, and CHWN8, and introduces a set of general optimization techniques for both direct and im2win convolutions. 
We compare the optimized im2win convolution with the direct convolution and PyTorch's im2col-based convolution across the aforementioned layouts on SIMD machines. The experiments demonstrated that the im2win convolution with the new NHWC layout achieved up to 355% performance speedup over NCHW layout. 
Our optimizations also significantly improve the performance of both im2win and direct convolutions. 
Our optimized im2win and direct convolutions achieved up to 95% and 94% of machine's theoretical peak performance, respectively.



direct convolution, im2win convolution, NHWC layout, CHWN layout, CHWN8 layout 




§ INTRODUCTION


Convolution is the essential component of deep neural networks for computer vision tasks such as feature exaction from large-scale image data<cit.>. It not only comprises 50%-90% of computational operations including convolutional, pooling, ReLU, and fully-connected layers<cit.>, but also consumes more than 90% of the total execution time of many popular neural networks<cit.>. Hence, optimizing convolution operations is crucial for enhancing the performance of neural networks.

Convolution methods can be classified into three main categories based on how they transform the input tensor: direct, im2col-based, and im2win. Direct convolution performs convolution operations directly on the tensor without changing its format<cit.>. This approach avoids extra memory consumption compared to im2col-based and im2win convolutions, but it suffers from nonconsecutive memory access.
Im2col-based convolution transforms the convolution into general matrix-matrix multiplications (GEMM)<cit.>, leveraging optimized Basic Linear Algebra Subprograms (BLAS)<cit.> for excellent performance. It has regular memory access but a significant extra memory footprint, which greatly limits its applicability on memory-constrained devices. 
Previously, we propose a memory-efficient convolution called image-to-window (im2win), which reorganizes the input tensor into a row of dot product windows and flattens the unique elements of these windows into a row that correspond to the convolution operation's receptive fields<cit.>. It provides sequential memory access and data reuse, and thus greatly reduces memory overhead. 

A tensor memory/data layout (referred as layout henceforth) refers to how the data of a tensor is physically arranged in memory. There are commonly three layouts for tensors: NCHW, NHWC and CHWN, where N is the batch size, C is the number of channels, H is the image height, and W is the image width. It significantly impacts convolution operations in terms of memory access, computational efficiency and compatibility with deep learning frameworks<cit.>.




Tensors are stored as one-dimensional arrays in memory, although logically represented as four-dimensional arrays. An input tensor and an im2win tensor in a NCHW layout are illustrated in <Ref>. Different colors represent different channels. In the NCHW layout, the elements in the width dimension are contiguous in memory by prioritizing the width dimension first, followed by height, channel and batch. 
Assuming a stride of 1 and a filter tensor in the NCHW layout with dimensions of 1x3x2x2, the elements in the solid-lined boxes are used to compute the first output element in the output tensor, while the elements in the dashed-lined boxes are for the next. 
Further details will be provided in <Ref>.

In general, the memory efficiency and performance implications of various tensor layouts with different convolution algorithms on single-input, multiple-data (SIMD) architectures have received limited attention.
The previous im2win works optimize the NCHW layout on CPU and GPU but have not tried the NHWC and CHWN layouts<cit.>.
Li et al. reveal the performance impact of the NCHW layout with the im2col-based convolution and the CHWN layout with the direct convolution in different CNN layers, and propose a fast multi-dimension layout transformation algorithm on GPU<cit.>. 
Li et al. optimize the cache utilization of the NCHW and NHWC layouts in DNN training along with DNN pruning on CPU<cit.>.
To date, no performance characterization have been conducted on NCHW, NHWC and CHWN layouts on SIMD architectures with the three convolution methods. 
 
To address aforementioned gaps, we propose three new layouts for the im2win convolution: NHWC, CHWN, and CHWN8,
and propose a set of optimization techniques based on the roofline model for both direct and im2win convolutions.
We compare the optimized im2win convolution with the optimized direct and PyTorch's im2col-based convolutions with the above layouts on SIMD machines.
Our experiments demonstrate that the new NHWC layout on the im2win convolution achieves 11% to 355% performance speedup compared with the NCHW layout. Our proposed optimizations have been empirically validated to enhance the performance of both im2win and direct convolutions. 
Our optimized im2win convolution and our optimized direct convolution achieve up to 95% and 94% of the theoretical peak performance of the machine, respectively.
As the main contributions, this work

1) proposes three novel layouts for im2win convolution.

2) comes up with a set of general optimization techniques that not only can be applied to im2win convolution and but also to direct convolution on different layouts.

3) compares the optimized im2win convolution with the im2col-based and the optimized direct convolutions using four tensor layouts on SIMD machines.





§ PRELIMINARY AND RELATED WORKS




 §.§ Notation

The input of a convolution operation includes an input tensor (ℐ), a filter tensor (ℱ), and an output tensor (𝒪). In these tensors,
N_i is the batch size, s is the stride size, C_i and C_o are the number of input and output channels (also known as feature maps), H_i/f/o and W_i/f/o denote the height and width of a feature map.



 §.§ Tensor Layouts: NCHW, NHWC, CHWN



In the NCHW layout, the input (ℐ), the filter (ℱ), and the output tensors (𝒪) are expressed as ℐ[N_i][C_i][H_i][W_i], ℱ[C_o][C_i][H_f][W_f], and 𝒪[N_i][C_o][H_o][W_o], respectively. The convolution is defined as:

    [ 𝒪_(i, j, m, n)=∑_j=1^C_i∑_m=1^H_f∑_n=1^W_f(ℐ_(i, j, m × s+u, m × s+v). 
    .×ℱ_(j, r, u, v)), ]


In the NHWC layout, the input (ℐ), the filter (ℱ), and the output tensors (𝒪) are expressed as ℐ[N_i][H_i][W_i][C_i], ℱ[C_o][H_f][W_f][C_i], and 𝒪[N_i][H_o][W_o][C_o], respectively. The convolution is defined as:

    [ 𝒪_(i, m, n, j)=∑_j=1^C_i∑_m=1^H_f∑_n=1^W_f(ℐ_(i, m × s+u, m × s+v, j). 
    .×ℱ_(j, u, v, r)), ]


In the CHWN layout, the input (ℐ), the filter (ℱ) and the output tensors (𝒪) are expressed as ℐ[C_i][H_i][W_i][N_i], ℱ[C_i][H_f][W_f][C_o], and 𝒪[C_o][H_o][W_o][N_i] respectively. The convolution is defined as:

    [ 𝒪_(j, m, n, i)=∑_j=1^C_i∑_m=1^H_f∑_n=1^W_f(ℐ_(j, m × s+u, m × s+v, i). 
    .×ℱ_(r, u, v, j)), ]

The definitions (1), (2) and (3) above are all subject to

    j=1,2,..,C_o, m=1,2,..,H_o, n=1,2,..,W_o, 
       i=1,2,..,N_i, u=1,2,..,H_f, v=1,2,..,W_f,
       r=1,2,..,C_i.





 §.§ Convolution Algorithms and Related Works



Direct convolution performs on the original ℐ and ℱ without any tensor transformation.
It has seven nested for loops and an AXPY operation in the innermost loop.
Based on the tensor layouts that direct convolution works with, the AXPY operation needs to read at different indices of ℱ and ℐ, and writes at different indices of 𝒪.

Direct convolution can compute with the original input tensors, so they usually adopt the NCHW layout of raw images. A study shows that, for convolution instances with large C, NHWC layout outperforms NCHW layout<cit.>. Grouping a certain dimension of the input tensors by a fixed size can also enhance the performance of direct convolution, such as NC32HW32 layout<cit.>.
Several works <cit.> have shown that the performance of direct convolution can be greatly improved by designing specific layouts based on the loop ordering of the algorithm on SIMD architecture. 

The im2col-based convolution transforms a convolution operation into a GEMM operation. ℐ[N_i][C_i][H_i][W_i] is processed in N_i batches, each batch contains data ℐ'[C_i][H_i][W_i] (that is, a single image). 
The im2col algorithm flattens the elements of each dot product window of  ℐ' and copies them into a single row of a 2D matrix<cit.>.
In addition to the conventional im2col data transformation algorithm, 
the MEC algorithm compresses the matrix layout, while still enabling the utilization of high-performance BLAS algorithms to perform convolution operations<cit.>.



§ HIGH-PERFORMANCE IM2WIN AND DIRECT CONVOLUTION USING THREE TENSOR LAYOUTS


In this section, we first review three layouts in the context of the direct convolution. Then we present three new layouts for the im2win convolution, following by how to determine the loop ordering based on the layouts and a set of optimizations for the im2win and direct convolutions on SIMD architectures.








 §.§ Motivations for Different Tensor Layouts



An example of direct convolution on an original input tensor in the NHWC layout is illustrated in <Ref>. 
The NHWC layout prioritizes the storage of elements in the last logical dimension—C_i—followed by that of W_i, H_i, and N_i. 
In the NHWC layout, the elements with the same N_i, H_i, and W_i but different C_i are contiguous in memory, which have unit stride access.
Assuming a stride of 1,
the red and green elements (outlined by solid lines, representing the convolutional windows to compute a single output element) in the input tensor are multiplied with the corresponding elements in the filter tensor, and the results are summed up (i.e., an AXPY) to obtain O_00. 
Next, the green and yellow elements (outlined by dashed lines) are used to calculate O_01. This process continues, with the convolutional window moving by the stride length in H_i or W_i, until all elements of the output tensor are computed.


Recall in <Ref>, the NCHW layout prioritizes the storage of elements in the last logical dimension—W_i—followed by that of H_i, C_i, and N_i.
Note that all C_i but not all W_i are used during the AXPY operation to obtain one output element. NCHW has non-unit stride access during the tensor convolution. However, the non-unit stride access may not be harmful, depending on the effects of caching and the access patterns used (determined by the loop ordering).

As shown in <Ref>, the CHWN layout stores elements by prioritizing N_i, followed by W_i, H_i, and C_i in memory. 
Previous research on GPU recommends to use N as the lowest dimension for coalesced memory access and data reuse in registers<cit.>. It has been observed that the performance is sensitive to the value of N.
The elements within the solid-lined boxes are used to compute the first eight output elements, while the elements within the dashed-lined box are used for the next eight output elements. 
This facilitates the use of vector registers for vectorization.
Assuming s=1, the convolutional window moves by one element in both the H_i and W_i dimensions until all output elements are computed.


 













 §.§ Im2win Tensor Transformation on Three Tensor Layouts



In this subsection, we present the im2win tensor transformation on three tensor layouts. The im2win tensor transformation process for the NWHC layout is shown as <Ref>. For other layouts, slight modifications need to be made on it. The im2win transformation flattens the elements of a convolutional window, storing them contiguously in memory and prioritizing them in the C_i dimension, as outlined in <Ref> from <Ref> to <Ref>. 
An example of transforming the original input tensor into an im2win tensor in the NHWC layout is shown in <Ref>. It is important to note that the green elements can be reused between two adjacent windows, which eliminates the need for repetitive storage in memory like the im2col transformation<cit.>.
In the output im2win tensor of <Ref>, the elements involved in each convolution operation are contiguous and more compact in memory, 
improving spatial locality,  cache and SIMD efficiency.

An example of the im2win convolution using the NHWC layout is shown in <Ref>. A naive im2win convolution method is shown as <Ref>. It is similar to the direct convolution, which involves seven nested loops and an AXPY in the innermost loop, 
except prior to that, the im2win convolution performs a tensor transformation upon ℐ to get Î, and ℱ in the NHWC layout is transformed into a NWHC layout to match Î.
All output elements can be computed through the outer four loops of <Ref> from <Ref> to <Ref>.

An im2win tensor in CHWN/CHWN8 layout is shown in <Ref>. The characteristics of C_i, H_i, and W_i in this layout are similar to NCHW, but the CHWN layout prioritizes storage in N_i. 
The CHWN layout's efficiency is constrained by 256-bit vector registers, which process only 8 outputs at once. When N_i > 8, this leads to low cache utilization, as the cache holds unnecessary data for these 8 output calculations.
Motivated by the approach of dividing C_i into blocks in the direct convolution <cit.>, we propose a new CHWN8 layout.
CHWN8 lays 8 N_i in the innermost layer and remaining N_i in the outermost layer, which takes full advantage of the vector registers without sacrificing the cache utilization.
N_i can be set to a multiple of 8 (with padding if necessary), 
CHWN8 layout may provide better data continuity than the NHWC layout when C_i is relatively small.







 §.§ Loop Reordering


In this subsection, we reorder the loops for the im2win and direct convolutions based on different tensor layouts.
Ideally, we arrange the inner loops to access data closer in memory to enjoy the unit stride access as long as possible.
In both direct and im2win convolutions, for NCHW, CHWN, and CHWN8 layouts, as they have the CHW memory access pattern, we use the width of the convolution window as the innermost loop, followed by the height, and the channel (W_f, H_f, and C_i). 
Conversely, for the NHWC layout, the innermost three loops iterate over the channel, the width, and the height (C_i, W_f, and H_f) of the convolution window. 
For the im2win convolution, since the im2win transformation flattens the elements of a convolutional window, we usually coalesce the height and the width layers (W_f, H_f) into a W_f × H_f.

Next we determine the order of the outer four loops. It not only determines the order in which the elements of the output tensor are produced but also influences the order in which the element in the input tensor are accessed. 
The layout mainly affects the inner three levels of the loop order, for the outer four levels the loop order applies to all data layouts.
Recall in <Ref>, the green elements of the input tensor are shared between the convolution windows of adjacent output tensor elements. Therefore, we consider to position the width (W_o) of the traversed output tensor in the fourth layer of loops at <Ref> in <Ref>.
Because the memory access of the input tensor is expensive in a convolution operation, we set the channel (C_o) as the third layer to reduce its access.
Since the batch of the output tensor corresponds to the batch of the input tensor, we place the batch (N_i) in the first layer of the loop. 
Finally we have the order of the outer four loops as NHCW in <Ref>. 



 §.§ Optimizations for the Im2win and Direct Convolutions




In this subsection, we use the Roofline Model<cit.> to determine how to optimize the im2win and direct convolutions on SIMD systems. 
We propose a set of optimizations for both the im2win and direct convolutions and apply the optimizations to them (with a slight modification on the direct convolution). The optimizations are classified into two categories: reducing the memory bottleneck and increasing the arithmetic intensity of the convolution kernel. The former includes hoist, memory alignment, register and cache blocking. The latter includes
loop unrolling, vectorization and FMA instructions, loop coalescing, and parallelization strategies.
We have three level parallelization: Non-Uniform Memory Access (NUMA) level, thread level using OpenMP and instruction level using SIMD.

We present the optimized im2win convolution as <Ref>. 
Since each element of the output tensor in  each batch (N_i) can be computed independently, there is abundant parallelism available<cit.> and this is NUMA-friendly.
Within each batch, the shared elements of adjacent windows can be maximized across the threads.
When the dimension we choose to parallelize is small and the number of CPU cores is large, this leads to a workload imbalance in many core architecture. The solution is to coalesce multiple dimensions/loops into one parallel loop to achieve better load balance. In practice, we find that coalescing two dimensions of the output tensor yields the best load balance. 
Hence, we apply parallel strategies and coalesce N_i and H_o in a parallel loop at <Ref> in <Ref>.

In <Ref>, we hoist three things: the indices of the elements in the 1D array of the tensor, the elements of the input tensor, and the entire filter tensor at <Ref>.
We use register and cache blocking<cit.> at <Ref> to reduce cache misses because the neighboring convolutional windows share duplicate elements of the input tensor, which W_o,b is the blocking size.
Next, we unroll the loop at <Ref> to take advantage of the spatial locality,
because the innermost loop accesses consecutive elements.
Since we are using FP32 and AVX2, we set the loop unrolling size N_vec=8. Finally, we vectorize the input tensor and filter tensor in units of eight to use the FMA instruction in AVX2<cit.> at <Ref>. 

A cache-line is 512 bits on contemporary x86_64 architectures, which is the minimum data quantity that can be fetched from memory to cache. Without memory alignment of the tensor data structure, the CPU needs to issue two memory requests to access an element, because the element may be in the middle of the cache-line. This hurts the performance greatly because not only the CPU has to wait for memory access but also more caches are used. This leads to higher cache miss and lower cache utilization. We store the elements of the tensor in a memory-aligned way using posix_memalign in C during the memory allocation.













§ EXPERIMENTS



 §.§ Experimental Setup

Architectures. We use a server with two Intel® Xeon® Gold 6330 CPUs and 251 GB RAM. Each CPU has 28 physical cores, running at 2.0 GHz, with 48 KB L1d cache, 32 KB L1i cache, 1.28MB L2 cache and 43 MB L3 cache.

Benchmarks We aim to cover the majority of the convolutional layers in commonly used DNNs. 
Hence for our experimental evaluation, we select an state-of-the-art DNN benchmark<cit.> shown in <Ref>, which includes twelve unique convolution layers, conv1-conv12.



Software We compare the direct, im2win, and im2col-based convolutions with four tensor layouts: NHWC, NCHW, CHWN, CHWN8. We intend to compare with the state-of-the-art implementation of direct convolution and the layout proposed in<cit.>, but their implementation is not open-sourced. 
We use the im2col-based convolution in PyTorch 2.1<cit.> with MKL<cit.>. Note that PyTorch only supports the NHWC and NCHW layouts.
Our code is compiled with GCC 9.5.0 compiler and  compilation flags. We use OpenMP 4.0 for parallelization with guided scheduling. 





 §.§ Performance of Different Convolution Algorithms


We run each algorithm 50 times on each benchmark with N_i=128 and report the best runtime. 
<Ref> shows the performance results in TFLOPS and <Ref> shows the memory usage of our high-performance direct convolution and high-performance im2win-based convolution, and the im2col-based convolution using MKL in PyTorch. In <Ref>, the left y-axis shows the performance in TFLOPS, and the right y-axis shows the performance of the machine peak.
Overall, our optimized im2win convolution achieves eight best TFLOPS out of twelve benchmarks; our optimized direct convolution achieves three best TFLOPS out of twelve benchmarks and the im2win convolution achieves close performance on these three; the im2col-based convolution in PyTorch achieves one out of twelve best TFLOPS on conv12. 
All twelve best TFLOPS are all yielded from the NHWC layout across these three methods.
Our proposed optimization techniques are proven effective on both im2win and direct convolutions. Our im2win convolution achieves 95% and 91% of the theoretical peak performance of the architecture on conv5 and conv6 respectively. Our direct convolution achieves 91% and 94% of the theoretical peak performance on conv5 and conv6 respectively. 

With the NHWC layout and performance normalization, 
excluding conv6 and conv12, our im2win convolution achieves between 1.1× and 4.6× performance speedup, and our direct convolution achieves between 1.1× and 3.8× performance speedup against the im2col-based convolution. All the best performance of the im2win convolution on twelve benchmarks is achieved using the NHWC layout on CPU. Our im2win convolution using the NHWC layout outperforms the NCHW layout by at least 11% and up to 355% across all benchmarks, showcasing significant efficiency gains in various scenarios.
For the direct convolution, nine out of twelve best TFLOPS are achieved using the NHWC layout on CPU, the rest three are achieved using the CHWN8 layout.

With the NCHW layout,
our im2win-based convolution achieves between 1.4× and 2.4× performance speedup on all benchmarks, and PyTorch's im2col-based convolution achieves between 1.1× and 7.5× performance speedup (excluding conv7) against the direct convolution. The direct convolution performs poorly on the NCHW layout. 

With the CHWN/CHWN8 layout, im2win_CHWN8 outperforms im2win_CHWN on all benchmarks from 3.7× to 16×; direct_CHWN8 outperforms 2.3× and 8× over direct_CHWN except in conv7. 
This shows our proposed CHWN8 layout overwhelmingly beats the CHWN layout on these two methods.
The direct convolution with the CHWN8 layout 
 performs better than the CHNW, NCHW and NHWC layouts when C_i is small (C_i=3 for conv1, conv2, conv3). Similar results have also been observed in the previous GPU work<cit.>.

In <Ref>, using the same tensor layout, the memory usage of each convolution is the same hence we only annotate one number per method. In all benchmarks, the direct convolution uses the least memory, while the im2col-based convolution consumes the most memory.
On average, the im2col-based convolution has 3.9× more memory usage than direct convolution, and im2win-based convolution has 1.5× more memory usage than direct convolution. The im2win-based convolution uses on average 39% of the memory of the im2col-based method, and in conv5, it uses only 24% of the memory of the im2col-based convolution.




§ CONCLUSION


We proposed three new layouts for the im2win convolution: NHWC, CHWN and CHWN8, and a set of general optimization techniques for both direct and im2win convolutions on SIMD architectures. We applied these optimizations on the im2win and the direct convolutions, and compared with PyTorch's im2col-based convolution on the above layouts along with the NCHW layout. 
Our experiments demonstrated that im2win convolution using the new NHWC layout achieved 11% to 355% performance speedup compared to NCHW layout. 
The proposed optimizations were proven to boost the performance of the im2win convolution and direct convolution. 
Our optimized im2win and direct convolutions achieved up to 95% and 94% of the theoretical peak performance of the machine, respectively.



§ ACKNOWLEDGMENT

This work was partially supported by the NSF OAC-2402542, and the National Natural Science Foundation of China (Grant No. 42261070).

IEEEtran








We leave some information out from the main body of this paper and have them as part of an appendix. Because we consider they are not essential to the main argument but may be useful for readers who want to dive deeper into the topic. 




 §.§ Peak Performance

We use the following formula to calculate the peak-gflops of the server:



    peak_flop/s = (#processors) × (#cores_per_processor) 
    × (clock_speed[1/s]) × (2 ×#FMA_units) ×vector_size[bits]/64



Based on <Ref>, the peak GFLOPS of the server that we use in our experimental evaluation is 3584 GFLOPS.



 §.§ Batch Size Scaling on Different layouts


We perform a strong scaling on the batch size from 32, 64, 128, 256 to 512 with different layouts using the direct convolution and the im2win convolution.
The performance results of the direct convolution with the CHWN, CHWN8, NCHW, and NHWC layouts are shown in <Ref>, <Ref>, <Ref>, and <Ref>, respectively.
The performance results of the im2win convolution with the CHWN, CHWN8, NCHW, and NHWC layouts are shown in <Ref>, <Ref>, <Ref>, and <Ref>, respectively.

From the figures, we can tell that the CHWN layout is most sensitive to the batch size among four layouts. The performance of the direct and im2win convolutions is the best on twelve benchmarks when the batch size is 32 (except for the direct convolution on conv12). Recall in <Ref>, the efficiency of the CHWN layout is constrained by the number of vector registers on the SIMD machines. When N_i>8, this leads to low cache utilization.

With our proposed CHWN8 layouts, the performance of the direct convolution and the im2win convolution exhibits similar patterns on twelve benchmarks, that is, when the channel sizes of the benchmarks are small (C_i=3 for conv1, conv2, conv3), the smaller the batch size is, the better performance the convolutions have on these benchmarks; when the channel sizes of the benchmarks (conv4-conv12) are large, the larger the batch size is, the better performance the convolution have on these benchmarks.

With the NWHC and NCHW layouts, both convolution methods show no obvious evidence that they are sensitive to the batch size across all benchmarks. The performance variance is contributed by the overall dimension of the input/filter tensors, and the loop coalescing which we coalesce the N_i and H_o dimensions into one parallel loop to achieve better load balance. 
























  


 Paper Title: Using CSNNs to Perform Event-based Data Processing & Classification on ASL-DVS.









Using CSNNs to Perform Event-based Data Processing & Classification on ASL-DVS
    
    
        Ria Patel,
        Sujit Tripathy, Zachary Sublett, Seoyoung An and Riya Patel
    
        
            Department of Electrical Engineering & Computer Science,
        
    
    
        
            University of Tennessee, Knoxville, TN 37916, USA
        
   

    
===========================================================================================================================================================================================================================================================================================




Using CSNNs to Perform Event-based Data Processing & Classification on ASL-DVS
    
    
        Ria Patel,
        Sujit Tripathy, Zachary Sublett, Seoyoung An and Riya Patel
    
        
            Department of Electrical Engineering & Computer Science,
        
    
    
        
            University of Tennessee, Knoxville, TN 37916, USA
        
   

    
===========================================================================================================================================================================================================================================================================================

§ ABSTRACT 


Recent advancements in bio-inspired visual sensing and neuromorphic computing have led to the development of various highly efficient bio-inspired solutions with real-world applications. One notable application integrates event-based cameras with spiking neural networks (SNNs) to process event-based sequences that are asynchronous and sparse, making them difficult to handle. In this project, we develop a convolutional spiking neural network (CSNN) architecture that leverages convolutional operations and recurrent properties of a spiking neuron to learn the spatial and temporal relations in the ASL-DVS gesture dataset. The ASL-DVS gesture dataset is a neuromorphic dataset containing hand gestures when displaying 24 letters (A to Y, excluding J and Z due to the nature of their symbols) from the American Sign Language (ASL). We performed classification on a pre-processed subset of the full ASL-DVS dataset to identify letter signs and achieved 100% training accuracy. Specifically, this was achieved by training in the Google Cloud compute platform while using a learning rate of 0.0005, batch size of 25 (total of 20 batches), 200 iterations, and 10 epochs. 









§ INTRODUCTION


The application of deep learning has been instrumental in solving numerous computer vision problems, notably image classification <cit.>, object detection and semantic segmentation <cit.>, and gesture recognition <cit.>, etc. For some specific applications such as static image recognition, Deep Neural Networks (DNNs) have been demonstrated to outperform human performance <cit.>. Training state-of-the-art deep learning models is computationally expensive, leading to high energy usage. To reduce the energy cost of future AI systems, researchers are exploring the use of energy-efficient, brain-inspired spiking neural networks (SNNs) in various fields <cit.>. SNNs typically take inputs from neuromorphic sensors that generate spikes with changes in input signals. Next, SNNs interpret such spikes, which contain sparsely encoded spatial and temporal information. SNNs implemented on neuromorphic hardware allow the hardware to utilize sparse activations within the SNNs to gain significant power and latency improvements compared to conventional hardware performing the same tasks <cit.>.

For data acquisition, dynamic vision sensors (DVS) are used to capture individual pixel changes within each frame of a recording, making them capable of capturing motion very similar to humans. Pixels that change are considered events whereas those that don’t change are not considered events by the camera. Next, the event-based data is encoded into spikes such that the SNN can interpret the spatiotemporal features.

Conventional frame-based cameras capture information from all pixels within the frame, making the image densely rich in information and memory demanding. Event-based cameras mitigate this issue by reducing this pixel information to make them sparsely encoded. As a result, these DVS cameras consume low power enabling operation at a finer time resolution of a few milliseconds <cit.>. 

Recently, SNNs have been used for problems that involve data with strong spatiotemporal correlations such as hand gesture recognition, object tracking, action and object recognition etc. <cit.>. In this work, we contribute a CSNN that performs classification on event-based data from ASL-DVS dataset  <cit.>. The rest of the report is organized as follows: Section II includes a
literature review that provides some background on CSNN architecture as well as discusses some prior work that utilized SNNs including CSNNs. Section III presents the proposed CSNN framework for ASL-DVS data processing and classification. Section IV discusses the characteristics of the ASL-DVS dataset and its preprocessing steps in detail. Next, we present the experimental set-up and results before concluding the report.






§ LITERATURE REVIEW




In this section, we first present the SNN and CSNN model architecture. Next, we discuss some prior works that utilized SNNs including CSNNs for event-based data processing.





 §.§ SNNs

An SNN shares similarities with an artificial neural network (ANN). Both models calculate neuron outputs using the weighted sums of inputs. However, SNNs process single-bit spikes as input, which are sparse in both space and time. Also, SNNs employ different types of activations compared to ANNs. The weights contribute to the membrane potential, and when this potential reaches a certain threshold, the neuron generates a spike. Spikes introduce a temporal component to the data processed by an SNN. In the ASL-DVS dataset, this temporal aspect is measured in μs to capture subtle movements occurring within the images  <cit.>.



There are several different types of spiking neuron models, but one of the most commonly used is called the Leaky Integrate and Fire (LIF) neuron. This neuron is closely modeled after biological neurons, making it suitable for neural networks. It integrates the weights of its input over time until it reaches or exceeds a certain threshold and then, the neuron will fire a spike. The membrane potential will decay towards a lower value at each step controlled by the decay rate β, enforcing the "leaky" aspect of the neuron. Figure <ref> demonstrates this feature, which is taken from  <cit.>.




The LIF membrane is modeled with:

    τdU(t)/dt = -U(t) + RI_int


This equation models how the membrane potential changes over time. Where I_in represents the input-weight matrix, R represents the membrane resistance, and the membrane potential at a given time t is represented by U(t) = β U(t - 1) + (1 - β) I_in(t), where β is the decay rate of the membrane potential. 




 §.§ CSNNs


A convolutional spiking neural network typically comprises spiking convolution operations and spiking encoding. Assume an input sequence of S(n), for n = 0, 1, 2,..., S(n) is a tensor at every time step that has shape u,v, where u and v are the width and height of each frame. Output from event cameras can be processed using a variable batch window to achieve the desired temporal resolution. For example, a 3 s 240 × 180 resolution events data stream with 100 ms temporal resolution will generate S(n), n = 0, 1, 2, ..., 30. For each segment, the tensor shape is (240,180). Next, the input tensor is convolved with a 2-D convolutional kernel to generate a spiking feature map  <cit.>. The convolution operation can be described using the following equations  <cit.>:


    y[i,j] = ∑_j = -∞^∞∑_i = -∞^∞ = x(i + m, j + n).K(m,n)



Where, y[i,j] represents the spiking feature map, x is the input tensor, K is the kernel, (i,j) are coordinates on the feature map and (m,n) are the coordinates in the kernel. Neurons in the feature map receive spikes, which are aggregated to generate membrane potential. 

The convolutional layers extract features. The LIF neurons at the output of each layer then gather all the input spikes and output a spike when the membrane potential crosses the set threshold. Lastly, similar to the traditional convolutional neural networks (CNNs), the last layer of CSNNs is a fully connected (FC) layer with LIF neurons and has an output shape equal to the number of classes for classification.



 §.§ Prior Technical Work

In this subsection, we review prior works discussing applications of SNNs and CSNNs for event-based data processing.

Amir et al. implemented a gesture recognition system that utilized a TrueNorth neurosynaptic processor alongside a DVS for real-time hand gesture recognition with minimal power consumption. They achieved an out-of-sample accuracy of approximately 96.5% on a diverse dataset, demonstrating the robustness of the approach for various hand gestures under different lighting conditions <cit.>.

Weijie et. al used CSNN for EMG (Electromyogram)-based hand gesture recognition. They utilized the Strathclyde and CapgMyo open-source datasets for model development. The average recognition accuracy was observed to be around 98.76% based on the Strathclyde dataset and 98.21% based on the CapgMyo open source dataset <cit.>.

Xing et al. introduced a spiking convolutional recurrent neural network (SCRNN) model for event-based hand gesture recognition, using the DVS gesture dataset. They achieved an accuracy of 96.59% for 10-class gesture recognition and 90.28% for 11-class gesture recognition <cit.>.

Samadzade et al. investigated the spatiotemporal feature extraction capabilities of CSNNs for event-based data processing. They observed superior performance of CSNNs methods on N-MNIST (97.6%), DVS-CIFAR10 (69.2%), and DVS-Gesture (96.7%) datasets. Additionally, the model achieved results comparable to ANN methods on UCF-101 (42.1%) and HMDB-51 (21.5%) datasets <cit.>.

The availability of comprehensive neuromorphic datasets plays a crucial role in the increased implementation of SNN-based architecture. Yin et. al <cit.> and Orchard et. al <cit.> created neuromorphic ASL-DVS and MNIST datasets. Eshraghian et al. compiled a comprehensive list of neuromorphic vision and audio benchmark datasets, widely accessed for SNN-based model development <cit.>. Moreover, efforts have been made to convert traditional computer vision datasets to neuromorphic vision datasets. A method for converting static image datasets into spiking neuromorphic datasets using an actuated pan-tilt camera platform was presented. By simulating motion updates through sensor movement, this approach enabled the utilization of popular image datasets like MNIST and Caltech101 for spike-based recognition algorithms <cit.>.

Lastly, to bridge the gap between deep learning and SNNs, Eshraghian et al. presented principles from deep learning, gradient descent, backpropagation, and neuroscience to develop SNNs. By elucidating the relationship between encoding data as spikes and the learning process, this work helps to advance the understanding and application of SNNs in NN research and development <cit.>.





§ TECHNICAL APPROACH


 
In this section, we present the overall deep learning pipeline for ASL-DVS data processing and classification. Then, we briefly discuss data selection and processing, the architecture of the CSNN model trained for ASL-DVS classification, and the CSNN learning method.



 §.§ Deep Learning Pipeline

Figure <ref> shows the overall deep-learning pipeline. We start with acquiring neuromorphic ASL-DVS datasets. Subsequently, we pre-process the data to bring it to a format so that it can be imported into the CSNN model. Next, we have an iterative process of model training and tuning to improve the CSNN model performance. Then, we validate our model that performs ASL-DVS gesture classification.



 §.§ Data Selection and Pre-Processing

For this project, we have used the ASL-DVS dataset to perform event-based data processing and classification. ASL-DVS dataset was acquired in AEDAT format and then, IniVation DV Processing software was used to pre-process it before the data could be imported into a CSNN model for training and validation. In the next section, we discuss the characteristics of the ASL-DVS dataset and its preprocessing steps in detail.




 §.§ Network Architecture

The CSNN model used to process the neuromorphic ASL-DVS dataset is built upon a similar CSNN implemented for N-MNIST data processing and classification  <cit.>. However, we updated the network
architecture to accommodate the ASL-DVS dataset. The model architecture comprises of three convolutional layers and a FC layer with 24 outputs for the 24 classes of the ASL-DVS dataset. Figure <ref> is a diagram showing the model's architecture. Each convolutional layer includes 5x5 filters with layers 1, 2, and 3 containing 12, 32, and 45 filters respectively. Each convolutional layer outputs to a 2x2 max pool layer and then to a LIF (leaky integrate and fire) neuron. The model then feeds into a FC layer with 24 output LIF neurons, that will determine the class or letter in the ASL-DVS dataset that the input belongs to. Figure <ref> shows the details involved in calculating the size of CSNN layers.



 §.§ CSNN Learning Method

For model training, the Adam optimizer is utilized. The loss function employed is the Mean Square Error (MSE) of the spike count, calculating the MSE between the correct and incorrect spike count estimates. These estimates are typically expressed as proportions of spikes over the total number of time steps. The target proportions for the loss function can vary, but a common choice is an 80% correct rate and a 20% incorrect rate <cit.>, which is used in this model.

To perform backpropagation using spikes, a specific method known as BackPropagation Through Time (BPTT) is utilized. This algorithm computes gradients by determining the loss of all descendants in the computational graph and aggregating them. Consequently, the derivative of the loss with respect to the weights ∂ℒ/∂ W is computed for each time step. The equation below illustrates how backpropagation computes this derivative  <cit.>:


    ∂ℒ/∂ W = ∑_t∂ℒ(t)/∂ W = ∑_t∑_t≤ s∂ℒ(t)/∂ W(s)∂𝒲(s)/∂ W


This allows for the model to learn the parameters, but there is still the problem of calculating the gradients when spikes themselves are non-differentiable (also, called "dead neuron problem"). The adopted solution to this is known as the surrogate gradient method. Essentially, this swaps the non-differentiable function for one that is differentiable during the calculation of gradients.














§ DATASET & IMPLEMENTATION




 §.§ Dataset Characteristics




The ASL-DVS dataset was created using the IniVation DAVIS240C NVS event-based camera under constant lighting conditions, which captures at a 240 × 180 resolution and has a 12 µs latency <cit.>. The events are captured and stored in the AEDAT 2.0 format, but a conversion process is used to achieve AEDAT 4.0, the newest AEDAT format that allows for the usage of the IniVation DV Processing software framework. Using this software, the AEDAT files were converted to CSV files to better view the event-based information and perform data exploration. 



This dataset contains recording data from five subjects who signed each letter of the alphabet, excluding J and Z. The ASL for J and Z are excluded from the dataset due to the nature of how those letters are signed (they include hand motion along with gesture). In this project, recording data from the first four subjects is used to train the CSNN. 

Figure <ref> illustrates the components of the ASL-DVS dataset. The dataset comprises 100,800 samples, also referred to as "frames," each lasting approximately 100 microseconds (μs). The number of samples varies for each letter in each subject, and each sample contains a varying number of events detected by the DVS camera. The terminology used in the data pre-processing section is explained in Figure <ref>.

Each event has four attributes: timestamp, x, y, and polarity. The timestamp denotes the exact μs that this event took place in the specified pixel via x and y coordinates. Lastly, polarity denotes whether there is an increase or decrease in the pixel brightness or intensity, eventually to be encoded as a spike.




 §.§ Dataset Exploration


To visualize the ASL-DVS dataset, we used the DV-Processing software to load each subject and each letter in the AEDAT4.0 format. The Python module library contains classes that load in each recording as a batch of events, which renders an image as shown in Figure <ref>. This image contains 1,515 events, creating the silhouette of Subject 1's hand signing the letter 'f' as they move it in an oscillating left-to-right motion.
The DVS camera picks up all movements and stores them as the black pixels shown in Figure <ref>, however, in the AEDAT files, the polarity field denotes them as either 0 or 1. This signifies that at the specific pixel, either there was a decrease or increase in pixel brightness. For the sake of visualization and ease of use for the SNN model, we encode all events that occur as 1 (black pixels) while the rest are encoded as 0 (white space).



Figure <ref> illustrates the overall sample distribution for each class across all subjects in the original ASL-DVS dataset. It is important to note that there are fewer samples for the letters Y and Z. This discrepancy is due to Subject 5 excluding recordings for the letters J and Y, while Subjects 1-4 exclude recordings for the letters J and Z.



Figure <ref> presents the sample distribution for each subject concerning vowel letters. Vowels were chosen for visualization as they are the most frequently used letters in English. Notably, Subject 5 contributes significantly to the sample distribution, indicating that Subject 5's dataset contains the longest recordings.



Figure <ref> provides a visualization of the number of events captured from Subject 1's recording of the letter F. The recording is divided into 3-second batches, each containing a varying number of events within the time window. This figure illustrates the subject's hand movement patterns at the beginning, middle, and end of the recording, demonstrating how the oscillation path of the hand changes throughout the recording.





 §.§ Dataset Pre-processing


We have extracted a subset from the ASL-DVS dataset by randomly choosing batches of events from each subject's recordings for a letter. Each recording varies in duration (e.g. 3 minutes, 5 minutes). We divide each recording into 3-second event batches, each containing a different number of events in each sample. Figure <ref> provides an example of an event batch, although it is not from the preprocessed dataset.

Following the event batch creation, we randomly select 3 event batches from each recording to include in our subset. For each subject and class, we save the randomly selected event batches into Numpy binary files. Subsequently, we combine all event batches from each subject and organize them into 24 classes.

To create the train and validate datasets, we randomly shuffle the event batches for each class. We then allocate 70% of the event batches for training and reserve the remaining 30% for validation. Our training set contains 495 event batches and our validation set contains 214 event batches.





§ EXPERIMENTS & RESULTS





 §.§ Experimental Setup

As shown in <ref>, the input to the model is a tensor of dimension: [time steps, batch size, channels, height, width]. An input event batch of 3 sec duration has 30 time steps with bin window duration of 100 milliseconds. The batch size is set at 25 implementing mini-batch gradient descent (GD). Also, the frame height and width are 180 and 240 respectively with 2 number of channels. The training dataset that consists of 495 event batches is passed onto the CSNN model and the model is trained based on the learning method discussed in section III. For this work, the 'Adam' optimizer is utilized with a learning rate = 0.0005, Betas = (0.9, 0.999). The loss function used is 'MSE of the spike count' with the 80% correct rate and a 20% incorrect rate. With 495 training data points (event batches) and a mini-batch batch size of 25, we have 20 iterations in each epoch. For training, we have experimented with varying numbers of epochs. Similarly, for validation, we have a batch size of 214 with all 214 event batches being processed simultaneously to generate validation results.




 §.§ Hardware


Given the size of the dataset, we turned to Google Cloud Platform (GCP) for training our model using various hyperparameter combinations. When attempting to run the model locally on the dataset, we encountered issues with insufficient memory.

In terms of hardware, we employed the N2 machine series, featuring 8 Intel Cascade Lake and Ice Lake virtual CPUs (referred to as threads by GCP) and 32GB RAM. This series utilizes x86 architecture and the SCSI and NVMe disk interface type. We selected the N2 machine series due to its suitability for general-purpose workloads and batch processing, aligning perfectly with our requirements.

On GCP, we leveraged the Vertex AI Machine Learning platform to conduct our model training within a Jupyter Notebook. Before training, we set up a Bucket, a storage solution in GCP, to house our dataset. Subsequently, within the Vertex AI-hosted Jupyter Notebook, we trained the model using the dataset stored in the Bucket, which already contained the separated training and testing data.



 §.§ Results

This subsection shows the results from our model for the best-performing hyper-parameters, in the form of loss and accuracy plots as well as a table.

Figures <ref> and <ref> show the model's training accuracy and loss plots for training cases with 1 epoch and 10 epochs each. This was the best training performance the model had achieved. It starts to stabilize around 90%-100% for the last several GD batches. Also, the loss is gradually decreasing across all batches. 

Figure <ref> shows the validation loss and accuracy across models trained for 1, 3, 5, 9, and 10 epochs. The models that had been trained for longer epochs generally got a higher accuracy and a lower loss indicating that longer training will result in the model learning better and in turn performing better. 

Table <ref> shows an overview of results from the different epochs. As supported by the graphs, the training loss gradually decreases while the training accuracy increases with the number of epochs. It even hits 100% accuracy for a single training batch (GD batch) as early as 5 epochs. Similarly, the validation loss also decreases while accuracy increases for each of the epochs. However, we also noted the mismatch between training and validation accuracy. This might indicate that the model was overfitting to the training dataset although it could otherwise be attributed to noise in event batches (some event batches captured background objects behind some subjects showing ASL gestures). This leads to the proposed future work, which may aim at exploring different regularization techniques to improve validation accuracy.


 

 





§ CONCLUSION



For this project, we aimed to develop a CSNN model capable of processing and classifying the ASL-DVS dataset. To begin, we obtained the ASL-DVS dataset and pre-processed it to prepare it for training the CSNN model. Using GCP, we successfully developed and implemented the CSNN, achieving a high training accuracy of 100% and a validation accuracy of 81%. Our findings indicate that as the number of training epochs increased, the losses decreased and the accuracies improved. However, we observed a disparity between training and validation accuracies, suggesting the need for further research to explore different regularization techniques aimed at enhancing validation accuracies.







§ ACKNOWLEDGMENT

We would like to express our gratitude to Dr. Charlie Rizzo from the TENNLab for his invaluable assistance in clarifying DVS datasets. Despite recently defending his Ph.D. thesis, he generously devoted his time to aid us in understanding the intricacies of neuromorphic dataset processing.






§ APPENDICES






 §.§ Coding

Our code repository <cit.> is organized into three main directories. The  directory contains visualization GIFs showcasing the event-based data. The  directory houses our model code and dataloader scripts. The  directory contains code for data preprocessing, analysis, and visualization.

For our implementation, we utilize the PyTorch framework along with the snnTorch framework, which serves as a PyTorch extension for spiking neural networks. Additionally, we incorporate Tonic, a library that streamlines the loading of neuromorphic datasets into the PyTorch framework. Tonic enables the application of transforms specifically designed for spatiotemporal datasets, while also being compatible with native PyTorch transforms.




 §.§ Code Design




  §.§.§ Data Pre-processing

Our data pre-processing code utilizes the DV-Processing library mostly. In the `utils/` directory of our repository, we created a `helper_funcs.py` that contains utility functions to perform batching of the recording data. These functions are used in the Jupyter notebooks `data_explore.ipynb` and `data_preprocess.ipynb`, which are to be run in the respective order. The `generate_letter_datasets.py` script, which must be run before the `data_preprocess.ipynb` code, utilizes the other two scripts in the directory `export_aedat4_to_csv.py` and `process_aedat_files.py` to create the training and validation datasets. 




  §.§.§ Model

The model design code was influenced by a snnTorch tutorial demonstrating the use of Convolutional Spiking Neural Networks (CSNNs) on the Neuromorphic MNIST (N-MNIST) dataset <cit.>. This tutorial provided insights into importing the DVS dataset into Tonic. 

To adapt the CSNN architecture for the ASL-DVS dataset, which has larger and non-square input dimensions compared to the 34 × 34 dimensions of the N-MNIST dataset, we made significant changes to the network structure. These changes included modifying the layers, adjusting the number of neurons in each layer, and refining the convolutional filters.







 Paper Title: An Asynchronous Multi-core Accelerator for SNN inference.


Atmospheric characterization of the super-Jupiter HIP 99770 b with KPIC
    [
    August 5, 2024
=======================================================================


plain



§ ABSTRACT 

  Spiking Neural Networks (SNNs) are extensively utilized in brain-inspired computing and neuroscience research. To enhance the speed and energy efficiency of SNNs, several many-core accelerators have been developed. However, maintaining the accuracy of SNNs often necessitates frequent explicit synchronization among all cores, which presents a challenge to overall efficiency. In this paper, we introduce an asynchronous architecture that eliminates inter-core synchronization, thereby enabling fast and energy-efficient SNN inference with commendable scalability. The core concept is to leverage the dependency of neuromorphic cores predetermined at compile time. We design a scheduler for each core to monitor the running state of its dependencies and control the core to safely proceed to the next timestep without waiting for other cores to complete their tasks. This approach eliminates the need for global synchronization, allowing our architecture to minimize core waiting time in the face of inherent core and time imbalances in SNN workloads. Comprehensive evaluations using five SNN workloads demonstrate that the proposed architecture achieves a 1.86x speedup and 1.55x energy efficiency compared to the state-of-the-art synchronization architectures.





§ INTRODUCTION


Brain-inspired computing, or neuromorphic computing, aims to simulate brain behavior to energy-efficiently achieve artificial intelligence <cit.>. Spiking neural networks (SNNs) <cit.> are widely used models for neuromorphic computing and neuroscience research. Rather than conventional artificial neural networks (ANNs) use only rate coding, SNNs are more similar to biological neural networks, using various coding with richer spatial-temporal information. Neurons in SNNs are supposed to model simplified dynamics like biological neurons, which is mathematically equivalent to discrete-updated differential equations (Fig. <ref>).

To exploit the great potential in low power consumption of SNNs, several neuromorphic accelerators have been proposed. They can be divided into two categories. The first kind of SNN accelerators <cit.> adopts an architecture similar to existing ANN accelerators <cit.>, which composes a 2-D systolic array of processing elements (PEs) and a large shared global buffer. The ANN-accelerator-like architecture separates computation units and memory units, which is not suitable for SNNs. On the other hand, a near-memory many-core architecture can dramatically eliminate redundant data movement, which is adopted by some industrial neuromorphic chips, such as TrueNorth <cit.>, SpiNNaker <cit.>, Loihi <cit.>, Tianjic <cit.> and Darwin <cit.>. In general, they comprise of many neuromorphic cores with built-in memories for computation, and an on-chip network (NoC) for spike communication. There are many researches <cit.> improving performance, flexibility, and energy efficiency. In this work, we focus on this architecture.



Although recent multi-core SNN accelerators can reduce memory access to lower power consumption, they suffer from low utilization caused by periodic all-core synchronization. The synchronization procedure is necessary for the SNN computation mode mentioned before, where each core in an accelerator updates neuron states step-by-step. Each neuromorphic core should wait for all other cores and spike transfers at each timestep to ensure result correctness, wasting considerable time. Many solutions have been proposed to improve SNN accelerator performance by eliminating or relaxing synchronization. Unfortunately, they either require new electronic devices <cit.>, or compromise the accuracy of the results <cit.>, or reduce the synchronization frequency by speculative execution mechanism meanwhile bring rollback overheads <cit.>.




In this work, we propose DepAsync to tackle this problem without global synchronization and additional devices, meanwhile keeping exact spike results at each timestep which we called time-accuracy.
We first explain synchronization from the perspective of data dependency among neuromorphic cores in SNN accelerators. Then we introduce core dependency determined at compilation time (Fig. <ref>) as prior information into time-driven architectures and design a mechanism to dynamically trace running states of dependencies for each core. With static core dependency and dynamic running states, DepAsync can control cores to safely forward to the next timestep, reducing redundant waiting time.
Our asynchronous mechanism eliminates all-core synchronization without speculative execution and is more flexible to make fine-grained control in core-imbalance and time-imbalance workloads, achieving higher speedup and energy efficiency. Our experiments use five different SNNs to evaluate DepAsync. The results show that DepAsync improves performance and energy cost by 1.86x and 1.55x over plain time-driven accelerators, and 1.46x, 1.29x over speculative-execution architectures.




The main contributions of this work are three-fold:

    
  * We identify that the synchronization in time-driven SNN accelerators derives from data dependencies between neuromorphic cores and review time-driven architectures by analyzing core dependencies.
    
  * We introduce core dependencies as prior information to time-driven SNN hardware accelerators, and propose DepAsync to eliminate all-core synchronization as a solution to the low utilization problem.
    
  * We evaluate our architecture with five different SNN workloads and demonstrate its advantages over existing accelerators.    




§ BACKGROUND



 §.§ Time-driven SNN Inference


Neurons in SNNs, acting as biological neurons, keep updating internal states (i.e., membrane potential) and output a spike when the voltage reaches the preset threshold as illustrated in Fig. <ref>, which gives SNNs the ability to model richer spatial-temporal information than ANNs. For example, the Leaky Integrate-and-Fire (LIF) model is prevalent in nowadays SNNs because of its good balance between biological plausibility and computing complexity. In mathematics, the internal state updating procedure can be represented as a differential equation like Equation <ref>:

    τ_m dV/dt = -(V - V_rst) + I/g_L

where V is the membrane potential, I is the external input, and V_rst,τ_m,g_L are parameters, representing resetting potential, membrane time constant, and leak conductance, respectively.



However, it is hard and inefficient to calculate exact analytical solutions using digital circuits. Thus, in order to simulate an SNN on digital circuits efficiently, some numerical methods are used in solving differential equations in Equation <ref>, where the differential w.r.t. time dt is usually called timestep. Fig. <ref> shows an example of SNNs, where x_i denotes neurons and the arrows represent connections between them (i.e., synapses) and the overall SNN inference can be written as Algorithm <ref>.





 §.§ Many-core Time-driven SNN Accelerators
 

Compared with GPU-based acceleration <cit.> and systolic-array-based SNN accelerators <cit.>, a custom-designed near-memory architecture can leverage more energy-efficient advantages of SNN as it eliminates intense data movement between global and local memory, and therefore has been adopted by industrial neuromorphic chips <cit.>. This architecture is usually composed of many individual neuromorphic cores and an on-chip network, as illustrated in Fig. <ref>. The neuron unit in each neuromorphic core calculates neuron state updating step-by-step and the in-core memory is for storing neuron states and synapse weights. For simplicity, we still use the term time-driven accelerators to represent many-core architectures in the subsequent sections without specific clarification.



The primary contribution to accelerating SNN inference mainly comes from exploiting inter-core parallelism within the single timestep. Neuromorphic cores can safely parallelize neuron internal state updating since there is no data dependency at this stage. Then, all cores go through a synchronization stage to wait for spikes generated in this timestep to arrive at destination cores via the NoC. Fig. <ref> illustrates the topology of a typical SNN inference on a 4×4 time-driven accelerator with black arrows representing core dependency, and Fig. <ref> shows the corresponding SNN inference workflow: cores perform neuromorphic computing (black dots) in parallel before synchronization (in a red rectangle), then work in a timestep-by-timestep manner. The whole system can only simulate SNNs step-by-step, restricted to per-timestep synchronization.

To implement a synchronization mechanism, TrueNorth uses a 1 kHz synchronization trigger signal <cit.>. When receiving this tick arrival, the neuromorphic core will be triggered to read the spike buffer and forward to the next timestep. SpiNNaker adopts a similar strategy, defining a timer event by a fixed number of clock cycles <cit.>. It also provides a catch-up mechanism to handle time imbalance in SNN workloads. Loihi <cit.> realizes synchronization more adaptively. After neuron state updating, cores in Loihi will send special packages that will drill up spike packages remaining in NoCs. Darwin uses both synchronization methods to give users more options <cit.>.























 §.§ Under-utilization Problem

Although time-driven SNN accelerators dispatch neuron-updating computation to parallel neuromorphic cores, the sparsity nature of spiking data in SNN workloads is not adequately considered. Compared with dense multiply-and-accumulate operations in ANNs, highly sparse spikes make workloads among timesteps and cores become imbalanced, which hinders overall hardware acceleration.



  §.§.§ Imbalance in SNN Workloads

We perform an SNN inference with several workloads to demonstrate workload imbalance. The evaluation details are discussed in Section <ref>. The SNN runs 100 timesteps on a 4×4 accelerator. Fig. <ref> shows that generated spikes exhibit significant variability across timesteps; and Fig. <ref> renders 4×4 neuromorphic cores with different colors and numbers in each circle representing normalized firing rates of each core. Timestep imbalance is from temporal sparsity in SNNs, that is, neurons don't fire at all timesteps. On the other hand, the core imbalance is from spatial sparsity, which means not all neurons fire at each timestep. Thus the total number of spikes is different.





  §.§.§ Under-utilization Caused by Synchronization

The under-utilization problem in time-driven accelerators has been observed in  <cit.>. It's difficult for a rigid synchronization mechanism to simulate imbalanced workloads. Fix synchronization signals cannot be changed dynamically, which is not suitable for temporal-variant spike rates. Synchronization messages after the timestep finish make some cores always wait for the slowest core in the whole system, which cannot tackle inter-core imbalance. What makes it worse is that spike transfer time in NoC grows as the number of cores scales up, and workloads with higher firing rates generate more spikes which cause more NoC congestion. Both of these factors deteriorate the under-utilization problem.


Fig. <ref> illustrates that the utilization ratio decreases with the growth of spike transfer time caused by increasing scales. Then, we adjust the average firing rate of SNN workloads. The results in Fig. <ref> show that the under-utilization problem becomes worse as the firing rate increases, due to more generated spikes slowing down the transmission ability of the NoC.




 §.§ Related Work


There are several approaches to reduce synchronization.

Firstly, leverage novel devices. Memristors are efficient emulators of biological neurons and synapses, because of the similarity in using ion migration as the fundamental mechanism <cit.>. Analog circuits with memristors can directly simulate neuron dynamics <cit.>. Furthermore, memristors also leverage physical characteristics to store synapse weights, and even simulate learning rules to support synapse plasticity <cit.>. Thus, timesteps become unnecessary in such total neuromorphic chips. Recently, Chen et al. <cit.> use a passive electrochemical memory (ECRAM) array to improve programming accuracy for analog circuits. However, these emerging devices are still in the research stage, and not as mature as the large-scale integrated circuit technology.

Secondly, get rid of time accuracy. Synchronization guarantees that outputs from accelerators are exactly the same as we expected at all timesteps, which is referred to as time accuracy. Hence, we can eliminate synchronization when approximation is endurable. Minitaur <cit.> and PEASE <cit.> adopt a totally event-driven architecture, where the neuron states updating only occurs at income spike arrivals. No synchronization is needed, but the length of timesteps becomes a variant, leading to inaccuracy compared with numerical computations in Algorithm <ref>. NeuroEngine <cit.> uses a bi-exponential model to reduce the complexity of solving the differential equation and add synapse delay support for SNN accelerators. 
However, an input spike can have effects on the next several timesteps when the input current (I in Equation <ref>) decays over timesteps. In this case, updating neurons only at spike arrivals may make neurons generate spikes from the past timesteps, which makes inconsistent results with Algorithm <ref>. Sometimes we expect the hardware accelerators to precisely simulate SNNs we trained before, which limits applications of event-driven neuromorphic chips.



Thirdly, incorporate additional mechanisms to mitigate synchronization. NeuroSync <cit.> proposes a speculative-execution (SE) and rollback-recovery (RR) mechanism to reduce synchronization frequency. However, periodical all-core synchronization is still inevitable, and the rollback-and-recovery overheads grow with the firing rate increases. Fig. <ref> gives a 4-core example of the SE mechanism: cores run speculatively with rollback and recovery until the period synchronization. Fig. <ref> shows RR cycles of time-driven accelerators with speculative execution, identifying RR overheads become gradually inevitable when more spikes are generated. Speculative execution mechanisms cannot fully leverage their advantages without the cooperation of predictors. SpikeNC <cit.> proposes a software agent-based asynchronous scheme. However, it does not leverage core dependencies and is limited at the SNN layer level. It also lacks hardware design. In this work, a novel time-accurate architecture is proposed to eliminate synchronization. We consider inter-cores dependencies as an additional control mechanism to achieve our design goals. Then, we realize DepAsync by adding a scheduler into each core and extending the spike communication mechanism.


It's worth noting that at circuit level, many-core neuromorphic chips often adopt a global asynchronous locally synchronous (GALS) manner <cit.> in circuit design <cit.>, which means the whole chip doesn't operate under single clock signal. The term 'synchronization' in this work is at the SNN computation level.





§ DEPASYNC DESIGN



 §.§ Ignored Inter-Core Dependencies

Line 6-9 in algorithm <ref> demonstrates that data dependencies between neurons are connected by synapses. At timestep t, post-synapse neurons update their internal states requiring spikes generated by pre-synapse neurons at timestep t-1. When we dispatch neurons to multicore SNN accelerators, dependencies between neurons are inherited by core dependencies. Similarly, we call core A a pre-dependency of core B if A sends spikes to B; versus the post-dependency.

Noting that these dependencies are Read-After-Writes (RAWs), synchronization therefore becomes necessary in time-driven architectures for result correctness. In most SNN accelerators, synchronization is rigid, since it requires all cores to wait together, like a barrier primitive. However, each core is supposed to wait for only part of the cores which it depends. When all pre-dependencies of a core finish timestep t and all spikes are received, it can safely forward to the next timestep without waiting for other unfinished cores.

Fig. <ref> illustrates a 4-core example, where grey arrows denote data dependencies same as the topology at the left, meanwhile, dots in black, green, and red represent core workloads at finished, running, and unfinished timesteps, respectively. At this moment, core 0 is running at timestep t_2, core 1-2 are at t_1, and core 3 is at t_0. For core 0, no pre-dependency means it can keep forwarding to the next, while core 1 and 2 rely on spikes from core 0, thus they can only safely proceed t_2 after the current timestep finishes without waiting for the slowest core 3. Finally, core 3 only has a forward window [t_0, t_1], since both core 1 and 2 are its pre-dependencies.

On the other hand, post-dependencies limit the size of the forward window (red rectangles in Fig. <ref>), which means how many timesteps a core can forward without waiting for its post-dependencies. In neuromorphic cores, the memory size assigned for buffering input spikes is finite, which means there may be no more empty buffer for spikes generated at far future timesteps. Here, we set the size of the forward window is 2 for visualization. For example, with constraints of core 1 and 2 to buffering input spikes from 2 more timesteps, the forward window of core 0 will be [t_2, t_3]. In this case, if core 0 runs over t_3 fast enough that core 1 or 2 is still working on t_1, it has to wait for them to release some spike buffers when starting t_2.





 §.§ Tracing Dependencies and Running States

The dependency analysis above gives neuromorphic cores the ability to get rid of whole-system synchronization. To implement such a mechanism, the main challenge is that each core has to trace dependencies and their running states.


Storing dependencies. For most SNNs, once the network is defined, connections (not weights) between neurons are determined. When deployed to accelerators, SNNs are first partitioned into several groups, each of which corresponds to a logic neuromorphic core. Then these logic cores are mapped to physical cores in real hardware by mapping algorithms <cit.> designed for reducing spike transfer latency and on-chip network communication congestion. After mapping, fixed real core dependencies can be stored at in-core memories as other neuron parameters.

Tracing running states. Unlike static dependencies, running states dynamically change during whole the SNN inference, which is traced by NoCs in our proposed architecture. NoCs in most SNN accelerators are mainly used for transferring generated spikes. As the communication component of the system, they are also responsible for transferring meta information, such as input parameters <cit.>, core statistics <cit.>, and synchronization messages <cit.>. Naturally, we allow neuromorphic cores to transmit running states to their dependencies.

Generally, packages in NoCs compose package headers including meta information like package type and routing information, and package bodies containing different kinds of message payloads. As shown in Fig. <ref>, we add a new kind of package, DEP, for transmitting running states. The DEP package body has three components. The identifier entry denotes the dependency the package belongs to. The timestep entry records the running progress of dependencies. The flag entry is for distinguishing running states. In this work, flag is only 1 bit, with 0 for a timestep finish (FINISH package), and 1 for a timestep start (START package). Then, the core procedure in DepAsync is changed to Algorithm <ref>, where red lines highlight the differences.

In each neuromorphic core, two tables are prepared for running states of dependencies, which will be updated at the arrival of START/FINISH packages. Assume a core can accept spikes from m future timesteps, its safe forwarding conditions at timestep t is two-fold: 1) all its pre-dependencies finish t; 2) all its post-dependencies start t - m + 2. If any condition is unsatisfied, the core turns to a wait state and keeps receiving and monitoring new packages. Once new DEP packages arrive and carry relevant information, the core is triggered to forward to the next timestep. When m = 1, cores holding the output layer of the SNN, which have no post-dependencies, will first forward to t+1, then wake up other cores in a cascading way. In this case, the whole system falls back to the synchronization mechanism. Otherwise, if m > 1, DepAsync can provide fine-grained control of neuromorphic cores with output correctness at every timestep.







Formally, the safe forwarding condition at Line 2 in Algorithm <ref> is calculated by Equation <ref>:

    cond   = pre-cond∧post-cond
       = (⋀_i=1^N_pre t_i^pre >= t^cur) ∧ (⋀_i=1^N_post t_i^post > t^cur - m + 1)

where N_pre, N_post are the number of pre- and post-dependencies, t_i^pre,t_i^post denote the timestep from received FINISH/START packages, and t^cur represents the current timestep of each neuromorphic core.


Fig. <ref> gives an example of the DepAsync workflow. The topology is the same as Fig. <ref>, and the spike buffer can hold spikes from 2 timesteps (i.e., m=2). Table headers pre, post and cur denote tables for pre-dependencies, post-dependencies, and the current timestep, respectively. The c_i within the parentheses at the beginning represents core dependency topology corresponding to Fig. <ref>. Solid arrows represent safe forwarding and dash arrows are DEP packages sent by neuromorphic cores. At the start moment 182, we also assume running states of cores align to Fig. <ref> and timestep t_3 is the end. In detail, core c_0 has finished timestep t_2, c_1 is running at t_1, while c_2 is at the end of t_1, finally, c_3 is still working at t_0. In this case, contents in tables for pre- and post-dependencies are determined. c_0 is stopped after t_2 (filled in the red cell) because of the post-condition (illustrated as a red font), and c_2 is stuck at t_1 for the same reason. At moment 183, c_3 safely forwards to its next timestep t_1. Since c_1 and c_2 are pre-dependencies of c_3, START packages at t_1 from c_3 are generated and sent to them. After a transmit latency in the NoC, c_1 and c_2 receive the packages and update their post-tables. In particular, the update in c_2 satisfies the post-condition, thus c_2 is wakened up to work on t_2, which also sends a START@2 package to c_0. When it comes to moment 184, c_3 surpasses the progress of c_1, which means it has to wait for pre-dependency c_1 to finish t_1 first. Then at moment 185, c_1 accomplishes t_1 and successfully goes to the next timestep, therefore it sends a FINISH@1 to c_3 and a START@2 to c_0. Simultaneously, c_2 ends t_2 and waits c_3 again. The FINISH@1 from c_1 and FINISH@2 from c_2 arrive c_3 at time 186, then replace the timesteps recorded in the pre-table and trigger c_3 to proceed to t_2. Then c_3 triggers c_2 after a while. Finally at moment 187, c_3 goes to the last timestep, sending FINISH@2, which satisfies pre-cond of c_3. Overall, neuromorphic cores in DepAsync alternately proceed until the simulation end, and leverage dependencies to ensure correct results without all-core synchronization.




§ DEPASYNC IMPLEMENTATION

In this section, we implement DepAsync by three key components in neuromorphic cores. Fig. <ref> gives an overview of our architecture. We first introduce the novel DepAysnc Scheduler, which plays a core role in dependency analysis. Then, we design our spike buffers used for storing future spikes, making it possible to proceed neuromorphic core without whole-system waiting. Next, we enhance the package generator to support new DEP packages cooperating with the DepAysnc scheduler. Finally, routers in NoCs are modified, ensuring correctness during the DEP package transmission. It is worth noting that the DepAsync mechanism is suitable for a wide range of neuron and synapse models, and we adopt the computation (including neuron update and synapse accumulation) and firing control components similar to current accelerators <cit.>. For example, the LIF model needs a multiplier to implement the leak mechanism, an adder to add on the accumulation that is accumulated after spike arrivals, and a comparator to detect whether the membrane voltage exceeds the threshold (i.e. firing a spike). 





 §.§ DepAsync Scheduler


The main component for core dependency is the DepAsync Scheduler with a working mechanism illustrated as Fig. <ref>. The scheduler is responsible for tracing dependency running states and controlling core forwarding.

Tracing dependency. The scheduler keeps monitoring income packages from the network interface. When DEP packages arrive, the scheduler will store the carried timestep into the pre- or post-dependency table decided by the flag entry, with the address given by dep id.

Controlling. On the other hand, the scheduler keeps generating control signals to direct the neuromorphic core whether forwarding to next. The timesteps stored in the pre-dependency table are compared with the current timestep t, checking whether there are potential unarrived spikes required by the next timestep. Meanwhile, all timesteps stored in the post-dependency table are compared with t - m, ensuring all post-dependencies have enough space for buffering spikes generated at t + 1. These comparators and AND gates implement Equation <ref>. Once a timestep is finished or a DEP package arrives, the scheduler will update the related entry in the dependency table and update the control signal, as demonstrated in Algorithm <ref>. 












 §.§ Circular Spike Buffer

The spike buffer in DepAsync is circular, storing incoming spikes for subsequent processing. At the end of each timestep, buffers are rotated one slot. Circular spike buffers are generally adopted for supporting synapse delay <cit.>, which means spikes received at t are used at t+delay. In this work, other slots store future spikes from pre-dependencies, logically the same as spikes with some delay. DepAsync works with synapse delay as well, subject to:


    N_slot = max_delay + m - 1




Fig. <ref> shows how to store a spike in the circular buffer. The timestep entry added by the delay determines which slot to access. As constrained by the scheduler, the slot destination never conflicts with the slot being used.



 §.§ Package Generator

To work with dependency analysis performed by the scheduler, the package generator is supposed to correctly send packages of DEP type. 



As illustrated in Fig. <ref>, entry flag and timestep in the DEP package are trivial, directly determined by the DEP package type and the current timestep. On the other hand, we need an additional mechanism to generate the correct dep id. As mentioned before, dep id is for addressing dependency tables. Thus, a neuromorphic core ought to be aware of its reversed index in pre- and post-dependencies, which is static and determined along with dependency relationships. In DepAsync, we store them at dependency tables. Fig. <ref> gives an example corresponding to the topology in Fig. <ref>. The entry DEP ID stored in one core points to the index in its dependencies. Then the destination core of a DEP package can decide which line in the dependency table to write.



 §.§ Multi-VC NoC Router

The basic assumption in DepAsync is that START/FINISH packages precisely represent the start and finish of a timestep of neuromorphic cores, which implies that the START/FINISH packages must arrive at destination cores in order, and the FINISH package at timestep t must be later than all spike packages. It is naturally satisfied if the on-chip network orderly transfers packages.



Recently, prevalent NoCs adopt the virtual channel technique to improve network performance. Each router port has more than one physical channel and concurrently transmits several packages by multiple channels. Each input port in a router has an arbiter to grant a request from multiple virtual channels at each cycle in a round-robin strategy <cit.>, which makes the package transmission out of the original order and violates the assumption in DepAsync. 



To cooperate DepAsync with virtual channels, the spike packages should be prior to the FINISH packages with the same source, destination, and timestep. Fortunately, routers with virtual channels generally mask out invalid channels before arbitrating one to transmit <cit.>. Thus, an additional mask is attached as shown in Fig. <ref>. In this case, FINISH packages always wait for spikes transfer first, and START packages are orderly transmitted as well. Noticing that the out-order of spikes within the same timestep is insignificant for correctness, DepAsync is capable of benefiting from performance improvement due to virtual channels.



§ EVALUATION



 §.§ Experimental Setup





  §.§.§ Experimental Platform
 
To model performance, we develop a cycle-accurate simulator to obtain workload latency. Neuromorphic cores execute instructions in Algorithm <ref> cycle by cycle, and the NoC is modified based on HeteroGarnet <cit.>. Table <ref> lists part of the parameters.




The energy model used in this work originates from Eyeriss <cit.>, where the energy cost of a single operation (computation or data movement) is first synthesized and modeled, and then the overall energy consumption is estimated by combining it with the number of operations. This model has been validated <cit.>, and become prevalent SNN accelerators <cit.>. In this work, DepAsync is evaluated at 400MHz using 65nm CMOS technology, with memory units estimated by CACTI <cit.>. DepAsync uses 270.3 KB and 8 KB SRAM for each core to hold synapses and neurons, and an additional 4 KB for the scheduler to hold 512 core dependencies. The limitation of the number of dependencies leads to a new constraint in compilation. The spike buffer is dependent on the parameter m, consuming 4m KB SRAM. In the following experiments, m is 4 unless otherwise specified. Then we use a 2-cycle router with 4 virtual channels for communication. In a 64-core chip with m=4 and 4 virtual channels, the area of each component is listed in Table <ref>. The area overhead caused by DepAsync is 6.76%.



Overall, the evaluation pipeline for DepAysnc is as follows. Firstly, the SNN workload and configuration generated by compilers are first fed into the cycle-accuracy simulator. Then, the simulator simulates the SNN and output statistics such as the number of neuron updating operations, memory access, and NoC hops. Finally, the analytic energy model is invoked to generate overall energy consumption.




















  §.§.§ Baselines

We compare DepAsync with two time-driven baselines, Loihi <cit.> and NeuroSync <cit.>. As Loihi is a commercial SNN accelerator, we carefully implement the synchronization mechanism of Loihi in our simulator (referred to as Sync) for convenience. Simulator parameters in Table <ref> are all available for the Sync baseline mode. On the other hand, the speculative-execution and rollback-and-recovery mechanisms in NeuroSync are developed as well (referred to as SE). It is worth noting that NeuroSync has proposed a Defer Learning method to reduce rollbacks caused by post-learning, which is however not useful to compare with DepAsync because misspeculation caused by RAWs in this work cannot be avoided. We also validate the output spike results of the two baselines to guarantee the exact time accuracy same as DepAsync.

We do not compare against time-driven accelerators with fixed synchronization signals such as TrueNorth <cit.> and SpiNNaker <cit.>, because their synchronization frequency relies on user settings. It is inconvenient to choose a proper frequency for all SNN workloads. Loihi provided a more adaptive way to implement synchronization.




  §.§.§ Workloads
 
We select five different SNN workloads to evaluate DepAsync. The workloads have different SNN structures and scales, as listed in Table <ref>.



The synthetic workload is generated like Izhikevich et al. <cit.>, with both excitatory and inhibitory neurons involved. Synapses and connection weights are randomly set. We choose this workload for a wide range of workload scales and neuron parameters(Table <ref>), which are useful to evaluate the scalability of the proposed DepAsync. Then, we utilize two workloads with real data. The SNN for MNIST <cit.> is converted from a pre-trained ANN by snntoolbox <cit.>, and dense images from the dataset are encoded by a Poisson process. On the other hand, the NMNIST <cit.>, CIFAR10DVS <cit.> and DVS-Gesture dataset <cit.> contain real spike trains generated from digital images, CIFAR10 images and hand gestures recorded by dynamic vision sensor (DVS) cameras, which are used to train an SNN by the STCA learning rule <cit.> in a SNN framework SPAIC <cit.>. It is worth noting that our DepAsync focuses on the synchronization issue and is suitable for step-by-step simulated SNNs with different neuron and synapse models. In our experiments, we use the LIF model in the above workloads.

We convert different workloads to a common configuration format containing SNN structures and sample data. The SNNs are partitioned layer-by-layer where each layer takes several logic cores. Then we adopt an efficient mapping algorithm <cit.> to map high-level SNN logic cores to low-level hardware neuromorphic cores for future simulation.



 §.§ Overall Results







Compared with the two baselines, we show the overall speedup and energy efficiency of DepAsync. Fig. <ref> illustrates the improvement of speculative execution (SE) and DepAsync over synchronous accelerators (Sync) on five workloads. In summary, DepAsync achieves 1.86x and 1.55x harmonic mean speedup and energy efficiency over plain synchronous time-driven accelerators, meanwhile 1.46x, and 1.29x over speculative-execution architectures. Exploiting more core parallelism than Sync speeds up DepAsync. And DepAsync performs better than SE because it schedules cores by core dependencies, avoiding misspeculation and rollback overheads.


DepAsync leverages core dependencies to eliminate all-core synchronization. In the worst case, if a core with many post-dependencies is always the slowest one, these post-cores still wait at every timestep, which is formally equivalent to a many-core synchronization. However, this situation is quite unlikely to occur in a real data SNN workload.




Fig. <ref> shows the latency breakdown of Sync, SE, and DepAsync. The waiting time in both SE and DepAsync decreases as the level of relaxation to synchronization increases. Although the idle time of SE cores is less than DepAsync, the cost of RR operations remains and even slightly grows due to the more aggressive speculative execution. For fair comparison between SE and DepAsync, the rest experiments set the synchronization period the same as the number of spike buffer slots m. Fig. <ref> shows the DepAsync scheduler and additional spike buffers take about 5% of total energy consumption. 

Fig. <ref> captures a slice in the MNIST workload, showing the difference between DepAsync and synchronous architectures, where colored lines represent neuron computation in a single timestep. Periodic synchronization can be observed from blanks lying in two workload lines, meanwhile, DepAsync makes neuromorphic cores consecutively execute computations.




































 §.§ Sensitivity Analysis



  §.§.§ Spike Buffer Size

Fig. <ref> shows the impact of the spike buffer size on DepAysnc performance. When the spike buffer size becomes large, more future timestep spikes can be stored, leading to a larger forwarding window and better performance. Different workloads have different performance converge speeds. Up to 16x spike buffer size, DepAsync gains 92% and 58% speedup and energy efficiency than baseline, respectively. Moreover, large spike buffers occupy more memory resources. Thus, it is a trade-off in real applications.





  §.§.§ Firing Rate

Furthermore, we compare SE and DepAsync at different levels of firing rate. As mentioned before, the more spikes are generated, the more likely rollback and recovery (RR) caused by misspeculations occur, which hints at overall performance. We change neuron parameters τ_m, V_rst in Equation <ref> to control the average firing rate in SNN workloads. Fig. <ref> demonstrates that the more spikes are generated, the more DepAsync exceeds SE, as the performance of SE is limited by the massive RR procedure at a high firing rate. It is worth noting that practical high-performance SNNs either ANN-converted or trained by learning rules behave quite differently from real biological neural networks since the sparse coding and learning mechanism in the human brain is still an open problem. Thus, recent SNNs generally have a higher firing rate, in which case DepAsync provides faster SNN inference performance.





  §.§.§ Cyclic Connections

DepAsync analyzes dependencies to control core behavior. When there is a cyclic connection, for example, two cores point at each other, both cores are pre-dependencies to one another. In this case, the post-condition is always satisfied if m > 1, however, the pre-condition is broken until all cores finish the current timestep. Thus, the slowest core will trigger others at its end. Finally, the whole system is fallback to a synchronous architecture. We exchange part of neurons between different neuromorphic cores and core C_0 to simulate the cyclic connections because C_0 holds neurons in the input layer and has no pre-dependencies in the default partition. As shown in Fig. <ref>, C_0 has to wait for other cores when there are cyclic connections. Fig. <ref> shows that the performance decreases by 30% at worst with such cyclic connections. Fortunately, some cyclic connections can be reduced in software compilers with an additional restriction. 







  §.§.§ Mapping Algorithms

The mapping algorithm in SNN compilers is responsible for mapping the logic cores to real physical cores, which affects the distance between a core and its dependencies. Three mapping algorithms are involved in our experiments: Plain means logical cores are orderly mapped to hardware cores; HSC leverages the spatial locality of Hilbert space-filling curves; HSC+FD finetunes HSCs to exploit more locality <cit.>. We calculate the average physical distance (measured by NoC hops) between a core
and its dependencies. The results are 2.37, 2.54, and 2.40 hops when using HSC+FD, HSC, and plain mapping algorithms, respectively. The difference in physical distance affects the overall performance. Fig. <ref> shows the normalized cycle cost of Sync, SE, and DepAsync with three mapping algorithms. The results demonstrate that DepAsync performs worse than two baselines with poor mapping, which means it is more sensitive to the mapping result. Thus, a better algorithm is necessary for DepAsync. Nevertheless, DepAsync with poor mapping strategies still runs faster and costs less energy than the baseline synchronization architecture.



  §.§.§ NoC Virtual Channels

Fig. <ref> shows the average cycles of spike packages blocked in NoC routers. The block cycle decreases when there are more channels which means the larger NoC bandwidth. The block cycle grows when we increase m in both SE and DepAsync. That is because the more timesteps we allow neuromorphic cores to process, the more spikes will be in-flight at the same time. It is worth noting that the block cycle in DepAsync are smaller than that in Sync when m=2. This is because additional packages in DepAsync(m=2) are from only one future timestep, thus are less than additional packages for global synchronization in Sync. Moreover, block occurs less in DepAsync than in SE with the same m. That is because when cores are allowed to process future timesteps, the NoC in DepAsync is only supposed to transfer future spikes, while the NoC in SE has to transfer packages generated by Rollback-and-Recovery operations as well. As for overall performance, Fig. <ref> demonstrates speedup due to additional virtual channels, where DepAsync overwhelms Sync and SE in all settings.





 §.§ Scalability of DepAsync

Finally, we evaluate the scalability of DepAsync. We gradually increase the number of neurons and synapses in the synthetic workload from 16 cores to 256 cores as shown in Table <ref>, then evaluate speedup and energy efficiency of DepAsync. The results shown in Fig. <ref> identify the excellent scalability of DepAsync compared to Sync. As the system scales up, the synchronization cost becomes more expensive, thus DepAsync accelerates SNN inference faster. The energy efficiency grows as static power consumption decreases due to higher speed. The fact that speedup grows faster than energy efficiency is mainly because spike packages hop in the NoC grows as O(√(N_core)) while computation and memory units in neuromorphic cores grow as O(N_core). DepAsync gains 4.99x speedup and 3.62x energy efficiency at the 256-core scale, which is better than SE.



Moreover, we consider the bandwidth hierarchy in real-world many-core systems. Generally, the whole system consists of multiple chips, and each chip contains several neuromorphic cores. The intra-chip NoC is multiple times faster than inter-chip (about 4 times in Darwin <cit.>). To simulate the bandwidth hierarchy in a large-scale system, we divide 16×16 cores into 8×8 4-core clusters and then decrease the bandwidth between adjacent clusters. Fig. <ref> gives an example topology of 4 clusters. Fig. <ref> shows a linear relationship between bandwidth difference and the overall performance.









 §.§ Comparison to ANN-Accelerator-Like Counterparts

The ANN-accelerator-like architectures differentiate from our many-core architectures in SNN inference manners. In DepAsync, layers in SNN networks are deployed to different neuromorphic cores and computed simultaneously as mentioned in Section <ref>. However, the ANN-accelerator-like architectures act as a layer-sequential mode where a layer will be processed after computing all timesteps of the previous layer, which is not suitable for SNN networks with no clear layer structure ('Synthetic' in Table <ref>). Fig. <ref> shows latency and energy cost (normalized to DepAsync) in our real-data SNN workloads of the state-of-the-art ANN-accelerator-like architecture SATA <cit.> which optimized SNN dataflow to reduce data movement energy overhead. Our DepAsync achieves 50.98x speedup and 16.39x energy efficiency on average. The lower energy efficiency is caused by large SRAM overheads in DepAsync.


































§ CONCLUSIONS AND DISCUSSIONS

In this paper, we propose a novel asynchronous architecture to time-accurately accelerate SNN inference on hardware, inspired by dependency relationships on neuromorphic cores. We first observe the imbalance in SNN workloads deployed on multi-core accelerators and identify under-utilization caused by all-core synchronization. Then, we leverage core dependencies determined at compilation time to control the timestep proceeding behavior of neuromorphic cores, implemented by adding a scheduler in each core worked with a new type NoC package. Finally, we use three SNN workloads to evaluate our DepAsync architecture. The experiment results demonstrate that DepAsync brings higher performance and energy efficiency over synchronous accelerators with or without speculative execution, and excellent scalability as scale up.

DepAsync introduces core dependencies as prior information into SNN accelerators. However, other information such as distance between dependencies, synapse numbers of two dependent cores, and dynamic network congestion influences SNN inference performance as well. In future work, more information involved makes more precise schedules which are useful to improve performance. Furthermore, although waiting time of neuromorphic cores is not totally eliminated in DepAsync due to hardware resource limitation, wasted cycles can be utilized by processing another SNN inference if we can deploy multiple SNNs to the same core.





IEEEtranS





 Paper Title: Interactive embodied evolution for socially adept Artificial General Creatures.



On Approximating Volumes of Clipped Hypercubes
    Costandin Marius
    August 5, 2024
==============================================



§ ABSTRACT 
 We introduce here the concept of Artificial General Creatures (AGC) which encompasses "robotic or virtual agents with a wide enough range of capabilities to ensure their continued survival".
 With this in mind, we propose a research line aimed at incrementally building both the technology and the trustworthiness of AGC.
 The core element in this approach is that trust can only be built over time, through demonstrably mutually beneficial interactions.
 
 To this end, we advocate starting from unobtrusive, nonthreatening artificial agents that would explicitly collaborate with humans, similarly to what domestic animals do.
 By combining multiple research fields, from Evolutionary Robotics to Neuroscience, from Ethics to Human-Machine Interaction, we aim at creating embodied, self-sustaining Artificial General Creatures that would form social and emotional connections with humans.
 Although they would not be able to play competitive online games or generate poems, we argue that creatures akin to artificial pets would be invaluable stepping stones toward symbiotic Artificial General Intelligence.
 

 
As far back as the mid-1950s, researchers were foreseeing a future where Artificial General Intelligence (AGI) would be commonplace.
For example, in <cit.> the authors state that “within ten years a digital computer will write music [with] considerable aesthetic value".
However, more than half a century later, AGI is still a much-investigated theoretical problem <cit.>.
While some of the recent advances in Large Language Models (LLM) have sparked worldwide interest and renewed confidence in the near-future capabilities of Artificial Intelligence (AI), practical feasibility remains an open question.
Indeed, on one side of the spectrum, researchers have argued that the breadth and depth of capabilities exhibited by GPT-4 make it an early version of an AGI <cit.>.
On the other side, there is extensive evidence that these capabilities may, in fact, not demonstrate any underlying human-level reasoning <cit.>.

Furthermore, some argue that the consequences of realizing AGI could lead to drastic societal changes, not all of which would be beneficial to mankind <cit.>.
This led to entire research fields working to identify and prevent any “catastrophic risk" that AGI <cit.> could pose.
At the same time, there are growing, opposite, concerns that we humans should not be afraid of AGIs, but instead that they should be afraid of us.
Authors from both within <cit.> and without <cit.> the scientific community are bringing attention to the need for a moral framework around the use of artificial intelligence.
In the end, though, both sides of the argument may be moot as, according to some authors, “general artificial intelligence will not be realized" <cit.> citing critical gaps such as embodiment or social skills <cit.>.





One could argue that most contemporary Artificial Intelligence agents are, indeed, disembodied puzzle solvers: 's () Atari players are just an eye and a hand while Chat-GPT is a browser-based chatbot.
The line of research that we advocate here explicitly considers these limitations (embodiment, mutual trust, and being socially adept) as primary goals, in addition to the agent's ability to self-sustain.
Expert skills, as shown by current AIs, would be an emerging longer-term property, just as they have been in biological evolution.

With this in mind, we suggest that future research in AI should place less emphasis on an agent's puzzle-solving performance, but should instead focus on two intertwined aspects of intelligence:

 
 
  Embodiment Interacting within a physical, potentially simulated <cit.>, world ensures a diversity of experience promoting general behavior and adaptivity.
 In fact, we argue that, at its basic level, AI could not be called general without any sensorimotor skills <cit.>.
 
  Social and emotional skills Most AIs in modern culture are depicted with cold and logical behavior, a position that is reinforced by current implementations.
 In contrast, showing empathy or performing social tasks would not only make artificial agents more relatable, it would also be very likely to improve their own efficiency <cit.>.


We argue that combining a capacity for (a) autonomous and self-sufficient behavior with (b) human agency, through social directives, would give AGI robust foundations both practically and theoretically.
Furthermore, a very desirable side effect of such an incremental undertaking is that of trust-building, which would start at the individual level.
In essence, the point here is to aim for relatable creatures that humans may, at least want to, understand and interact with.



§ ARTIFICIAL COMPANIONS


In a more practical and short-term view, we propose to take advantage of current works in Evolutionary Robotics to thin the barrier between virtual and real and bring such Artificial General Creatures closer to realization.
Thanks to extensive research in that field, there now exist frameworks in which everything that can be simulated can also be 3D printed <cit.>, and vice versa.
Reusing inexpensive and readily available components makes it possible to generate complex robot morphologies in virtual space, evaluate performance a-priori, and establish them physically with minimal overhead (Figure <ref>).

To leverage such “thinning of the barrier", we propose an interactive approach with four nested loops, in an extension of the Triangle of Life <cit.>. Each corresponds to different levels of control, as described in Figure <ref>.
Starting with a traditional evolutionary loop, individuals are mutated, mated, and evaluated in a virtual environment.
The a-priori performance is computed following some additional optimization, for example, through reinforcement learning, until some criteria A is met.
Regularly, a pre-selection (B) of the most promising individuals in the current population is performed based on task-specific criteria (raw performance), but also on the agent's resilience (self-sustainability) and capacity to learn (learning delta, <cit.>).
This subset is further evaluated by a “end-user" human agent, and a refined selection is extracted (C).
The use of an “end user" is essential to guarantee adequacy between the agents' theoretical performances, as exhibited during simulation, and the expected behavior once deployed in the field.

The result of this second evaluation can then be used in one of two ways: either to provide additional directions to the evolutionary algorithm (e.g., tweaking the fitness, budgets, or population) or to proceed with the assembly of promising agent(s).
Once in the physical world, the agent(s) would interact with humans according to the specifics of the task, while once again, undergoing some form of reinforcement.
Through this, it would adapt to this new environment on two fronts: sensorimotor discrepancies (reality gap) and human behavior (personalizing).
Following this interaction, the results could be directly fed back to the evolutionary algorithm as a final layer of control or, once the performance is deemed adequate, lead to direct deployment (D).



§ EXPECTED IMPACTS


On an implementation level, this line of research is expected to foster strong interactions between a host of disciplines ranging from Evolutionary Robotics to Neurosciences.

 
  NeuroEvolution and Reinforcement Learning Due to the emerging morphology of these artificial companions, it is not possible to devise an adequate controller beforehand.
 In contrast, observing the impact of such an interactive multilayered evolution on ANNs' topological features could, in itself, be the subject of dedicated inquiries, e.g. through VfMRI analysis <cit.>.
 Additionally, as individuals are expected to learn both in the simulation and in the physical world, they should be endowed with an efficient mechanism to do so, one that is tailored to non-regular topologies.
 As such, further cooperation between NeuroEvolution and Reinforcement Learning researchers could prove most fruitful as advocated in <cit.>.
 
 
  Neurosciences and Evolutionary Robotics Similarly, the challenges of embodied Artificial Intelligence bring about the opportunity to investigate the evolution and development of (primitive) artificial life forms on a larger scale.
 For instance, it would be possible to study sensorimotor processing from the elementary mapping of actions to consequences to more complex patterns when facing social or emotional situations both throughout evolution and during an agent's lifetime.
 In fact, this methodology is reminiscent of a domestication process, although at a much faster pace, enabling the exploration of the evolutionary interactions between mind and body in the face of complex environmental constraints.
 
 
  Ethics, Legal disciplines, Philosophy, etc. Naturally, the scope also includes social sciences, as AI touches all aspects of society.
 Thus, it would also fall under the scope of this research line to foster debate around the legal status of the artificial creatures that would result from such work.
 Unlike expert systems, the incremental nature of the agents envisioned here would allow a more progressive integration into society with the overarching objective of promoting harmonious, mutually beneficial partnerships <cit.>.




§ ACKNOWLEDGEMENTS


This research was funded by the Hybrid Intelligence Center, a 10-year programme funded by the Dutch Ministry of Education, Culture and Science through the Netherlands Organization for Scientific Research, <https://hybrid-intelligence-centre.nl>, grant number 024.004.022.

apalike




 Paper Title: Hilbert curves for efficient exploratory landscape analysis neighbourhood sampling.











Hilbert curves for efficient exploratory landscape analysis neighbourhood sampling
     Johannes J. Pienaar

	Department of Computer Science

	University of Pretoria 

        Pretoria, South Africa 

	 

	
	https://orcid.org/0000-0003-3546-1467
    < g r a p h i c s >
Anna S. Bosman 

	Department of Computer Science

	University of Pretoria 

        Pretoria, South Africa 

	 

 https://orcid.org/0000-0002-6070-2632
    < g r a p h i c s >
Katherine M. Malan 

	Department of Decision Sciences

	University of South Africa 

        Pretoria, South Africa 

	 



    August 5, 2024
======================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================



§ ABSTRACT 

Landscape analysis aims to characterise optimisation problems based on their objective (or fitness) function landscape properties. 
The problem search space is typically sampled, and various landscape features are estimated based on the samples.
One particularly salient set of features is information content, which requires the samples to be sequences of neighbouring solutions, such that the local relationships between consecutive sample points are preserved. Generating such spatially correlated samples that also provide good search space coverage is challenging. It is therefore common to first obtain an unordered sample with good search space coverage, and then apply an ordering algorithm such as the nearest neighbour to minimise the distance between consecutive points in the sample. However, the nearest neighbour algorithm becomes computationally prohibitive in higher dimensions, thus there is a need for more efficient alternatives.
In this study, Hilbert space-filling curves are proposed as a method to efficiently obtain high-quality ordered samples. Hilbert curves are a special case of fractal curves, and guarantee uniform coverage of a bounded search space while providing a spatially correlated sample. We study the effectiveness of Hilbert curves as samplers, and discover that they are capable of extracting salient features at a fraction of the computational cost compared to Latin hypercube sampling with post-factum ordering. Further, we investigate the use of Hilbert curves as an ordering strategy, and find that they order the sample significantly faster than the nearest neighbour ordering, without sacrificing the saliency of the extracted features.














§ INTRODUCTION

Search landscape analysis has established itself as a useful approach for understanding complex optimisation problems and analysing evolutionary algorithm behaviour <cit.>. Many landscape analysis techniques have been developed over the years, with the most widely used techniques including fitness distance correlation <cit.>, local optima networks <cit.>, and exploratory landscape analysis (ELA) <cit.>. When landscape analysis produces numeric outputs, the resulting feature vectors can be used as abstract representations of problem instances, where instances with similar feature vectors are assumed to fall into similar problem classes. If these feature vectors effectively capture the important characteristics of problems, they can be used as the feature component for automated algorithm design (AAD) – specifically, automated algorithm configuration and selection. A number of recent studies have achieved different aspects of AAD using landscape analysis in specific contexts <cit.>. 

Landscape analysis approaches differ in terms of what they measure or predict (e.g. ruggedness, modality, presence of funnels, and so on), and also on what they produce (e.g. numerical results or a visualisation of a phenomenon) <cit.>. They can also be distinguished based on the scale of the analysis <cit.> – a global approach attempts to characterise the features of the search space as a whole, while a local approach will consider the features of the landscape in the neighbourhood of solutions. Fitness distance correlation <cit.> and local optima networks <cit.> are both examples of global approaches, whereas the average length of an adaptive walk <cit.> is an example of a local landscape feature. 

Many landscape analysis techniques that measure local features are based on samples that are spatially correlated, i.e. sequences of neighbouring solutions, as opposed to a sample of independent solutions from the whole search space.  Techniques that require such sequences of sampled solutions include correlation length for measuring ruggedness <cit.>, entropic profiles of ruggedness and smoothness with respect to neutrality <cit.> – adapted as a single measure of ruggedness for continuous spaces <cit.>, approximations of gradient <cit.>, information content features <cit.>, and measures of neutrality <cit.>. 

In the context of numerical optimisation, a wide range of landscape analysis approaches have been implemented in the R package  <cit.>, which has also been re-implemented in Python ([<https://pypi.org/project/pflacco/>]). Over 300 landscape metrics are included in , organised into 17 sets, which include the original ELA metrics <cit.> covering six of the feature sets, and the set of information content features <cit.> consisting of five metrics. In a study of the subset of the “cheap" feature sets in , Renau et al. <cit.> found that the two most salient (in terms of distinguishing between problems) and robust feature metrics were from the information content and ELA meta-model feature sets. Unlike the other feature sets in , the information content metrics require spatially correlated samples.
 The saliency of the information content feature metrics and the additional requirement of neighbourhood ordering is the motivation for this study.  

To date, the two main strategies proposed for generating spatially correlated samples in continuous search spaces are random walks <cit.> and post-factum ordering of global samples <cit.>. Desirable properties of a sampling strategy for estimating local features are that the solutions provide good coverage of the search space <cit.>, successive points are positioned close to each other (compared to other solutions in the sample) to capture landscape changes in the neighbourhood, and that the process has low computational expense. The most commonly used approaches to neighbourhood sampling for numerical optimisation are uniform random sampling or Latin hypercube sampling (LHS) <cit.>, followed by either random or nearest-neighbour ordering. These are the approaches implemented in  and .

In this paper, we investigate the use of Hilbert curves <cit.> as a new sampling strategy. Hilbert curves are fractal space-filling curves with two desirable properties: (1) they guarantee uniform search space coverage, and (2) neighbouring points on the curve are located close to one another. We show that Hilbert curves are comparable to ordered LHS in terms of the saliency of the extracted features, but are significantly cheaper to compute. Additionally, we show that a Hilbert curve can be used to spatially order an LHS sample, resulting in significant computational gains compared to the commonly used nearest neighbour ordering method.   







§ HILBERT CURVES

A space-filling curve is a surjective continuous function from the unit interval [0,1] to a unit hypercube [0,1]^d. Surjectivity implies that every point in the hypercube maps to at least one point in the interval, and continuity ensures that no areas in [0,1]^d are missed <cit.>. Space-filling curves are a special case of fractal curves, and are guaranteed to fill a continuous space in the limit. 

Our interest in space-filling curves derives from two useful properties that they offer, namely (1) uniform coverage of a bounded d-dimensional space, and (2) the ability to provide a unique mapping between points in the d-dimensional space and points on the  1-dimensional curve <cit.>. Specifically, this study considers the Hilbert space-filling curve, first proposed by D. Hilbert in 1891 <cit.>. The Hilbert curve is defined recursively, and can be constructed through a limit process of iteration. Each successive iteration creates an approximation of the true Hilbert curve that passes through more points in the d-dimensional unit hypercube. For practical purposes, the number of iterations is chosen to be finite, and is further referred to as the order of the Hilbert curve. 

Consider the construction of a Hilbert curve in 2D. For the first iteration, a line on the closed interval [0,1] and a square [0,1]^2 are taken. Four equidistant points are selected on the line, where the starting point is at 0 and the end point is at 1. The square is subdivided into four equal parts. Intervals of the line connecting each pair of points are then mapped onto the square such that the intervals adjacent on the line share a common edge on the square. This results in a simple U-shape, illustrated in Figure <ref>(hc:p1). At each subsequent iteration, the curve from the previous iteration is divided into four equal parts. Each part is then shrunk by a factor of 1/2, rotated, and repositioned such that the four curves connect at their endpoints in a U-shaped or reverse U-shaped pattern. Figure <ref> shows 2D Hilbert curves of order 1 to 3. 





The Hilbert curve is bijective, i.e. the mapping between points on the curve and a d-dimensional space is reversible. Bijectivity of space-filling curves has been useful in organising multi-dimensional data storage and retrieval systems, since it allows a linear index to the data to be constructed and searched <cit.>. Ideally, closely related data in the d-dimensional space would also be close together on the 1-dimensional curve. Faloutsos and Roseman <cit.> investigated the clustering properties of Hilbert curves compared to other space-filling curves, and found Hilbert curves to be superior when measuring the average distance (along the curve) to the nearest neighbours of a given point. This finding suggests that Hilbert curves may serve as a viable alternative to nearest neighbour sorting of a sample. 

For the Hilbert curve implementation in this study, we use the  Python package (<https://pypi.org/project/hilbertcurve/>), based on the algorithm presented by Skilling <cit.>. All code used for the experiments conducted in this study can be found at <https://github.com/jpienaar-tuks/MIT807>.



§ HILBERT CURVES AS SAMPLERS


Given a Hilbert curve of a particular order, the vertices can be used as a basis for a sample in the associated multidimensional space. Table <ref> shows how the number of vertices grows with the order of the curve and the dimension of the search space. When sampling for ELA, it is common practice to use sample sizes of 10^2× d to 10^3× d <cit.>. 

Table <ref> highlights in bold the order and dimension combinations where the number of vertices exceeds a common sample budget of 10^3 × d. To remain within a sampling budget, we sub-sample randomly from the set of vertices on a Hilbert curve with a minimum order of 3. 



 §.§ Stochastic sampling using Hilbert curves



Since the generation of a Hilbert curve is deterministic, the following two strategies were evaluated to introduce stochasticity:

    
  * Selecting random points along the edges of the Hilbert curve, illustrated in Figure <ref>(fig:hilbert_along_edges): Given any two sequential vertices, P_i and P_i+1, a new point is selected P_j = rP_i+(1-r)P_i+1, where r ∼ U(0,1). 
    
  * Selecting points near vertices of the curve, illustrated in Figure <ref>(fig:hilbert_near_vertexes): For each vertex P_i, generate a new point drawn from a normal distribution centred on P_i with a standard deviation σ (constrained by the bounds of the search space). The step size of the Hilbert curve before scaling is 1, so σ was empirically chosen to be 0.3 to prevent excessive potential overlap with points generated by neighbouring vertices.




Neighbouring vertices on a Hilbert curve are equidistant, so the sequence of vertices has a constant step size. The two randomisation techniques introduce variation into the step size. 
Figure <ref> shows that 

the randomisation near vertices strategy produces step sizes with a more normal distribution. For all further experiments we have used the randomisation around vertices strategy.



 §.§ Search space coverage


We now investigate the extent to which Hilbert curve sampling covers the search space compared to competing strategies such as Latin hypercube sampling (LHS). To investigate the search space coverage, the following three strategies were used to draw samples from search space [-5,5]^d of sizes n ∈{100d, 316d, 1000d} for dimensions d ∈{5, 10, 20, 30}:

    
  * Hilbert curve: randomly select the required number of points from the curve (without replacement).
    
  * LHS: generate the required sample using a Latin hypercube design <cit.>.
    
  * Random walk: use a simple random walk <cit.> with maximum step size of 1 to generate the sample.

For each sample size and dimension combination, a random uniform sample was drawn as a reference set. Thirty independent runs of each sampling strategy were implemented, and for each run, the Hausdorff distance <cit.> to the reference set was calculated. The Hausdorff distance as a measure of search space coverage was first proposed and investigated by Lang and Engelbrecht <cit.>, and their methodology is followed in this study. Statistical significance tests were performed for each sampling strategy as proposed by Derrac et al. <cit.> and described by Lang and Engelbrecht <cit.>.





Table <ref> shows that the Hilbert curve and LHS have similar Hausdorff distances, implying a similar coverage of the search space, whereas the random walk provides the worst coverage. Lower values were achieved in most cases by the Hilbert curve sampler, confirmed by the statistical significance tests presented in Table <ref>. The null hypothesis is that there is no significant difference between the Hausdorff distances for each of the sampling strategies. Results show that the null hypothesis can be rejected, given the p-values and a significance level of 0.05. We therefore conclude that the Hilbert curve sampler achieves a more uniform search space coverage than LHS.

    
    




 §.§ Computational cost

We now compare the cost of the Hilbert curve as a sampler to alternative sampling strategies. 
Generating the Hilbert curve is not computationally cheap, but we expect that this upfront investment will be justified if the sample is subsequently used to calculate landscape metrics that require ordered samples. 


To evaluate the computational cost, a performance counter[Python's ] was used to keep track of the time to a) generate the sample and b) calculate information content metrics for both the Hilbert curve and LHS. Using the  library, the information content () metrics were calculated for each of the 24 BBOB functions defined as part of the COCO platform <cit.>. This was done for each dimension and sample size listed in Table <ref>. Note that for LHS to be used for information content metrics, the sample needs to be ordered. We evaluated both the random ordering and the nearest neighbour ordering strategies. Also note that the information content function provided by the  library was modified to accept  as an ordering argument to use the pre-ordered Hilbert curve samples.

Figure <ref>(fig:timing:generation) 
shows that the Hilbert curve sampler is slower than LHS to generate a sample of the same size. While the computational cost of the Hilbert curve sampler seems to grow at least quadratically, it remains acceptable even for large sample sizes (just over 1.2 seconds to generate a sample of 30 000 points). 

Figure <ref>(fig:timing:total) 
shows that for the information content metrics, Hilbert curve sampling strategy is substantially faster (especially at larger sample sizes) than LHS with nearest neighbour ordering.























    























 §.§ Predictive performance of Hilbert curve samples



We have shown that Hilbert curve sampling provides an efficient method for generating spatially correlated sequences of solutions from continuous search spaces, and that these sequences provide good coverage of the search space. We now investigate whether the landscape features extracted from these samples provide good predictive performance for algorithm selection. 

Following the approach by Muñoz et al. <cit.>, we use the task of predicting the class of each BBOB function as a proxy for algorithm selection: if the landscape features of function instances can be used to discriminate between problem classes, then they should also be effective predictor variables for algorithm selection. Five target classes corresponding to the BBOB function groupings as outlined in <cit.> were used, namely (1) separable functions: {f_1… f_5}, (2) functions with low or moderate conditioning: {f_6… f_9}, (3) unimodal functions and functions with high conditioning: {f_10… f_14}, (4) multimodal functions with adequate global structure: {f_15… f_19}, (5) functions with low or moderate conditioning: {f_20… f_24}.

The experimental setup is as follows: 24 BBOB functions in dimensions d ∈{2,5,10,20} are used, with an evaluation budget of 1 000 × d. Each BBOB function has 15 predefined instances, and three random instances per function are held out as testing data[This leave-one-instance out approach is used instead of the leave-one-problem out approach due to the small number of classes.]. For each combination of instance and dimension, 1 000d samples are drawn from [-5,5]^d, using three sampling strategies: Hilbert curve (HC), Latin hypercube (LH) and random walk (RW). Four feature sets from the  package were selected that did not require further function evaluations, namely: dispersion (), ELA y-distribution (), ELA meta model (), and information content ().=-1

Table <ref> gives the testing accuracy of a decision tree, k-nearest neighbour, and random forest classifier 

for the task of predicting the function class from the landscape metrics. The  version 1.1.3 default settings were used in all instances. Results show that a random forest model was the most effective at predicting the function class across all sampling strategies. Overall, the RW strategy emerges as the least competitive performer, while the Hilbert curve and Latin hypercube sampling strategies were comparable for all three classifiers.  

    

Table <ref> gives the performance for each of the three sampling strategies based on the four feature sets separately. An observable trend is that both the  and  features are very salient features and allow the classifiers to accurately discriminate between the function classes, confirming results of Renau et al. <cit.>. 

    
 











    




§ HILBERT CURVES AS AN ORDERING TOOL

In this section, we investigate the use of Hilbert curves as an ordering aid for samples generated from other sampling methodologies. 
A Hilbert curve of order p maps each point on a 1-dimensional curve [0,…,2^dp] to a point in [0,…,2^p-1]^d and vice versa, as illustrated earlier in Figure <ref>. The mapping is bijective, and allows for an efficient mapping from a point in the d-dimensional space to a point on the 1-dimensional Hilbert curve. 

This d-D to 1-D mapping can be exploited, given that a Latin hypercube sample of n points divides each axis into n intervals. By selecting the appropriate curve order p, i.e. 
    p=⌈log_2(n+1)⌉,
we can ensure that the resulting Hilbert curve is of sufficient length to accurately map all LHS points to points on the Hilbert curve. The ordering of the points on the Hilbert curve can then be used to provide a spatially correlated ordering of the Latin hypercube sample.

Given that the current best practice of using a greedy nearest neighbour strategy to order LHS requires calculating at least half of the pairwise distance matrix between all points in the sample, the Hilbert curve ordering strategy has the potential to be significantly faster. Furthermore, the nearest neighbour strategy tends to start with relatively short step sizes, which increase as the number of unvisited points decrease. Hilbert curve ordering is likely to provide more consistent step sizes.



 §.§ Step size consistency

Figure <ref> shows a visual comparison of the Hilbert curve and nearest neighbour ordering strategies on a 2D Sphere function sample. The underlying sample X for both strategies is identical, and was obtained using LHS. In the figures the orderings produced by the Hilbert curve (HC) and the nearest neighbour (NN) approach are depicted as a red line. Colour scale is used to indicate the fitness values of the sampled points. It is evident from Figure <ref> that the maximum step size is smaller for HC than for NN. The step size distributions shown in Figures <ref>(fig:step_size_viz:hc) and <ref>(fig:step_size_viz:nn) confirm that the maximum step size of HC is lower, and show a greater skew in the step size distribution for NN.







 §.§ Computational cost

To evaluate the computational cost of the Hilbert curve ordering, we compare it to the nearest neighbour and random (RND) ordering strategies applied to a sample generated using LHS. The Latin hypercube samples of sizes n ∈{100d, 316d, 1000d} were drawn from search space [-5,5]^d for dimensions d ∈{5, 10, 20, 30}. For each configuration, 24 independent samples were drawn, and timed with a performance counter (Python's ). 

Figure <ref> shows the performance comparison between the various ordering strategies. Hilbert curve sampling as discussed in Section <ref> is included for completeness, as it does not require additional ordering. From Figure <ref>, NN ordering performs the worst, especially for large sample sizes. Furthermore, RND sorting and Hilbert curve sampling are equally fast (red and blue curves overlap). Hilbert curve ordering produces an intermediate result, slightly slower than random ordering, but significantly faster than the NN ordering. Since the underlying sample is generated by LHS, Hilbert curve ordering retains all the benefits of LHS for ELA features insensitive to order (such as , or ).






 §.§ Evaluation of features generated using Hilbert curve ordering


We now compare the information content features generated from a Latin hypercube sample using the HC, NN, and RND ordering. Since the feature values are evaluated rather than the computational cost, this experiment was restricted to dimensions d∈{2,5} and sample sizes n∈{100d, 1000d}. A single sample set X was generated and evaluated on each of the BBOB functions to generate corresponding y = f(X) results. Information content metrics were then calculated on these (X, y) pairs using each of the ordering strategies. This procedure was repeated 30 times.

Figures <ref> and <ref> present initial partial information (M_0) and maximum information content (H_max) landscape features based on the entropy of the sample. We consider BBOB problems in 5D (problem indices listed on the x-axis), and calculate feature values based on NN, RND, and HC ordering for n=1000d.

It is evident from Figure <ref> that the HC ordering produces values that lie between RND and NN, but follow a pattern similar to that of NN. It is clear that M_0^RND converges to the same value regardless of the underlying function (confirming the results of  <cit.>). M_0^NN and M_0^HC, on the other hand, produce values that allow for discrimination between functions. Since larger values for M_0 indicate a more rugged landscape (more changes in concavity over the length of the walk), the HC ordering would on average indicate a more rugged landscape than the NN ordering.

According to Figure <ref>, H_max^RND varies over a wider range than M_0^RND. However, the H_max^RND values still tend to cluster around the upper edge of the graph, closer to the maximum value of 1. In comparison, H_max^NN and H_max^HC vary over a larger part of the [0,1] range. Larger values for H_max indicate a larger variety of objects in the landscape (pits, peaks, plateaus). Thus, HC ordering again indicates a more rugged landscape than the NN ordering.















To evaluate the relative saliency of the features produced by the various ordering strategies, permutation-based feature importance technique <cit.> is employed. We train a random forest (RF) classifier using information content features as input ({ϵ_s, 
ϵ_max, ϵ_0.5, H_max and  M_0}, as defined in <cit.>), and the function class as output (as defined in <cit.> and used in Section <ref>). Feature saliency is gauged by randomly permuting each input feature in turn across the dataset, and observing the corresponding reduction in RF accuracy. For this purpose, the features are grouped by the ordering strategy, and a RF is trained on 2/3 of the data, with the remaining 1/3 held out for testing.  

After training, the base accuracy of the RF classifier is evaluated on the held out data. The accuracy of the RF classifier is then re-evaluated as each feature in the held-out dataset is randomly permuted in turn, and the difference to the base accuracy is recorded. This procedure is repeated 10 times for each sampling strategy. The results are shown in Table <ref>. 

Table <ref> shows that  ϵ_s was the most salient feature across all sample orderings, with the RF trained on the HC ordered features relying on ϵ_s more heavily than the RF trained on the NN or RND ordered features. RND sample ordering made the M_0 feature near useless for the classifier, in line with the observations made in Figure <ref>. Conversely, RF trained on the NN ordering relied on M_0 more than RF trained on the HC ordered features. Overall, HC ordering yielded marginally better RF performance compared to the NN and RND ordering approaches.






    






§ CONCLUSION

This paper proposed the use of Hilbert space-filling curves in the context of optimisation problem landscape analysis for the purpose of (1) sampling the search space in a spatially correlated manner that also guarantees uniform coverage, and (2) spatially ordering samples generated using other  sampling algorithms such as the Latin hypercube. Experiments were conducted to evaluate the relative computational efficiency of the Hilbert curves, as well as the saliency of the landscape features extracted using Hilbert curve sampling and Hilbert curve ordering. In the context of sampling, Hilbert curves were significantly faster than Latin hypercube sampling for the purpose of generating order-sensitive features such as the information content metrics. Features extracted by the Hilbert curves were informative, and allowed for successful discrimination between problem classes. As an ordering tool, Hilbert curves performed significantly faster than the commonly used nearest neighbour ordering, and also yielded salient landscape features. Thus, Hilbert curves present a viable computationally efficient alternative to both Latin hypercube sampling, and nearest neighbour ordering of a sample for the purpose of landscape analysis.
 


    



Displayed equations are centered and set on a separate
line.

    x + y = z

Please try to avoid rasterized images for line-art diagrams and
schemas. Whenever possible, use vector graphics instead (see
Fig. <ref>).



<cit.>








splncs04




 Paper Title: Approximating Rayleigh Scattering in Exoplanetary Atmospheres using Physics-informed Neural Networks (PINNs).



firstpage–lastpage
The extremes of AGN variability: outbursts, deep fades, changing looks, exceptional spectral states, and semi-periodicities
    [
    August 5, 2024
===========================================================================================================================




§ ABSTRACT 
This research introduces an innovative application of physics-informed neural networks (PINNs) to tackle the intricate challenges of radiative transfer (RT) modeling in exoplanetary atmospheres, with a special focus on efficiently handling scattering phenomena. Traditional RT models often simplify scattering as absorption, leading to inaccuracies. Our approach utilizes PINNs, noted for their ability to incorporate the governing differential equations of RT directly into their loss function, thus offering a more precise yet potentially fast modeling technique.
The core of our method involves the development of a parameterized PINN tailored for a modified RT equation, enhancing its adaptability to various atmospheric scenarios. We focus on RT in transiting exoplanet atmospheres using a simplified 1D isothermal model with pressure-dependent coefficients for absorption and Rayleigh scattering. In scenarios of pure absorption, the PINN demonstrates its effectiveness in predicting transmission spectra for diverse absorption profiles. For Rayleigh scattering, the network successfully computes the RT equation, addressing both direct and diffuse stellar light components.
While our preliminary results with simplified models are promising, indicating the potential of PINNs in improving RT calculations, we acknowledge the errors stemming from our approximations as well as the challenges in applying this technique to more complex atmospheric conditions. Specifically, extending our approach to atmospheres with intricate temperature-pressure profiles and varying scattering properties, such as those introduced by clouds and hazes, remains a significant area for future development.





radiative transfer – scattering – planets and satellites: atmospheres – methods: numerical








§ INTRODUCTION


The use of Machine Learning (ML) in exoplanetary science has grown increasingly popular during past years <cit.>. Hereby, it is mostly used to accelerate the time-consuming process of finding the posterior distributions of atmospheric parameters given some observed spectrum. Usually, this inference is done using a Bayesian scheme, like nested sampling <cit.> where many so-called forward models simulate data which can then be compared to observations.

Pioneered by <cit.>, who used a simple multi-layer perceptron (MLP) to predict the existence of molecules in exoplanetary atmospheres from their transit spectra, the approaches have diversified since then, from Random Forests <cit.> to generative adversarial networks like ExoGAN <cit.> and convolutional neural networks, often in combination with Monte Carlo dropout <cit.>.

More recent approaches achieve generating more accurate and versatile posterior distributions by employing normalizing flows in combination with variational inference <cit.>, neural posterior estimation <cit.>, or flow matching and importance sampling <cit.>.

The approaches most similar to our own are by <cit.> and <cit.>, who both try to speed up the forward model while leaving the inference scheme untouched. While <cit.> use a neural network to emulate radiative transfer by predicting the spectrum given some atmospheric parameters, <cit.> focus on accelerating computationally expensive disequilibrium chemistry calculations using an autoencoder and an LSTM-like neural network.

Most of these models rely on traditional numerical methods to generate the data that the ML model uses for training. Since all ML models cannot contain more information than their training data, they cannot generalize well beyond the used forward model.
Including the time to train a model, the respective method only offers a speed-up if enough exoplanets can be analyzed using the same model trained on the same data. This can be particularly challenging for exoplanetary atmospheres, given the diversity of exoplanets that have been observed. Additionally, as soon as a forward model is updated to include some new insight, the ML model might be completely retrained. It can also prove difficult to use one model for observations with varying instruments, that, e.g., have a different number of data points or an incompatible spectral range.

In the future, as data become more abundant and precise and the models more detailed, the computational burden of running a retrieval will further increase. Therefore, in this work, we will assess the feasibility of using pre-trained physics-informed neural networks (PINNs) to cheaply model complex phenomena in exoplanetary atmospheres. We focus on the simple example of Rayleigh scattering in the isothermal atmosphere of a transiting exoplanet.

Building upon the work of <cit.>, who used PINNs to solve the radiative transfer (RT) equation, we first model RT considering only absorption. Since for transiting exoplanets, the traditional numerical method treats scattering the same as absorption in that the light scattered out of the beam is analogous to absorbed light <cit.>, we can compare our PINN to existing models to establish its speed and accuracy.

Secondly, we include Rayleigh scattering in the RT equation. As has been shown by <cit.>, scattered light can have a significant effect on the transit spectrum at short wavelengths. Using PINNs could help us study this effect in more detail and potentially improve our estimates of the hydrogen-to-helium ratio or the existence and properties of clouds and hazes.

In Section 2, after briefly introducing PINNs, we discuss how the respective problem is presented to both the absorption and scattering PINN, such that they are able to solve it efficiently. This involves generating and normalizing the inputs, the formulation of the loss function, and the exact training procedure.
The results of both models are assessed in Section 3, while drawbacks are summarized in Section 4. Section 5 discusses key insights and possible enhancements.




§ METHODS




 §.§ PINNs


While there are many different ways of incorporating physical laws into the training of a neural network, PINNs, as introduced by <cit.>, achieve this through their loss function, which is directly based on a differential equation.

To see how a PINN is able to solve a nonlinear partial differential equation (PDE), as explained in <cit.>, let us first describe a PDE in its most general form defined on the domain Ω∈ℝ^d with its boundary ∂Ω:


    ℱ(u(z);γ) = f(z),    z ∈Ω
    ℬ(u(z)) = g(z),    z ∈∂Ω ,



where u describes the unknown solution to the differential equation, dependent on the space-time coordinate vector z := [x_1, ⋯, x_d-1, t]. ℱ refers to the nonlinear differential operator dependent on physics-related parameters γ, while f is the function identifying the data of the problem (if available). Eq. (<ref>) defines the boundary and initial conditions, where ℬ is again an operator and g is the boundary function.


While the PINN can also be used for an inverse problem of finding the parameters γ, we use it for approximating the solution u for every z, dependent on the (given) parameters γ. Hereby, the PINN itself functions as the approximation to this solution. The PINN therefore has inputs z and output û_θ(z) ≈ u(z) dependent on the network parameters θ. The corresponding loss function that should be minimized is made up of different components ℒ_i with respective weights ω_i:


    ℒ = ω_ℱℒ_ℱ + ω_ℬℒ_ℬ .



The first term ℒ is the differential equation (<ref>) itself, with all terms on one side of the equation such that it is equal to zero. Often, the following form analogous to the mean squared error is used:


    ℒ_ℱ = ∑_i=1^n ( ℱ (û_θ(z_i);γ) - f(z_i) )^2,    z_i ∈Ω ,



where the square guarantees both positive values, such that the minimum lies at zero, and differentiability (compared to the mean absolute error).

Now we calculate the value of the differential equation ℱ(u(z);γ) with automatic differentiation. We can not only use it to calculate the gradient of the loss function with respect to the weights of the network, but also calculate the gradient of the output û_θ(z) of the network with respect to the inputs z, the space and time coordinates. Hence, we can insert all the values of the variables and derivatives into the differential equation to calculate the loss.

The second term can be calculated accordingly to make sure that the boundary conditions are fulfilled:


    ℒ_ℬ = ∑_i=1^n ( ℬ (û_θ(z_i)) - g(z_i) )^2,    z_i ∈∂Ω .



In this work, our approach to employing PINNs deviates from their conventional application. Typically, PINNs are tasked with solving specific instances of differential equations, where their advantage over traditional numerical methods is most pronounced in scenarios involving complex geometries or boundary conditions. However, we leverage PINNs in a novel way by training them on a parameterized form of the radiative transfer equation. This means that the solution approximation provided by the PINN, denoted as û_θ(z), is influenced not only by the space-time coordinates z but also by a set of physical parameters γ. This unique application allows the parameterized PINN to generate solutions, û_θ(z, γ), for varying physical conditions without the need for retraining, showcasing a flexible and powerful tool for modeling radiative transfer processes in exoplanetary atmospheres.





 §.§ Setup
 

In this study, our objective extends beyond solving a PDE; we aim to tackle an integro-differential equation, specifically the stationary radiative transfer equation, which operates in two spatial dimensions:


    cos(ϕ)  u/ x   + sin(ϕ)  u/ y + (α_a + α_s) · u
    
        - α_s/4π   ·∫Φ(θ, ϕ, θ', ϕ') u(θ', ϕ') Ω' = 0 ,



where the angle ϕ = 0 is defined as the positive x-direction and lies
in the range ϕ∈ [-π, π]. α_a and α_s are the absorption and scattering coefficient, respectively and Φ the scattering phase function, depending on the incoming (θ', ϕ') and outgoing angles (θ, ϕ) of the light. In the case of Rayleigh scattering, the scattering phase function depends only on the angular distance Δ:


    Φ(Δ) = 3/4( 1 + cos^2Δ) ,



where cosΔ = cosθ cosθ' + sinθ sinθ'  cos(ϕ - ϕ') according to the law of cosines. For both the absorbed and scattered light component sensible symmetries for small angular extents of the star are exploited such that the integral can be evaluated numerically over only one angle (see Appendix <ref> for the explicit integral).

The function of interest that we want to solve for is u(x,y,ϕ), the transmission through the atmosphere. For this, we also need to define our geometry, as shown in Fig. <ref>. More specifically, we want to find the transmission u(x=x_max,y,ϕ=0), which we can then integrate over the y or radial direction to find the effective area of the planet.


This depends on a number of other parameters. So, to reduce the dimension of the problem, we assume an isothermal atmosphere and constant conditions throughout the day-night-terminator region of the planet. The models are trained on an atmosphere spanning pressures from 10^-6 to 10^2 bar and consisting of 100 log-spaced layers, where the absorption and scattering coefficient is interpolated logarithmically between them.

Furthermore, for the PINN with scattering, we assume that the planet transits its star with an orbital inclination i = 90^∘, simplifying the boundary condition. The scattering PINN also only calculates the solution for pure scattering without any absorption (α_a = 0). The total transmission function, when simulating a spectrum, is then calculated by multiplying the solution for purely absorbing species with the solution for purely scattering species, which should hold as long as one of the two is dominating.



 §.§ Data Generation
 

Before we are able to train a neural network on this problem, we have to generate and normalize the necessary input data.

For both the absorption and the scattering model, this includes the spacial coordinates x and y, which are normalized to be within the interval [-1,1] by scaling them with factors l_x and l_y, respectively. These are defined such that the innermost atmospheric layer touches the lower boundary at (x,y) = (0, -1) and the outermost layer touches the upper boundary at (x,y) = (0, 1) as well as going through the lower corners (x,y) = (-1, -1) and (x,y) = (1, -1):


    l_x = √(r_0^2 - r_99^2),    l_y = r_0 - r_99/2 ,


where r_0 refers to the outermost and r_99 to the innermost layer. Note that we have to account for these scaling factors inside of the differential equation by dividing the respective gradients by this factor.

The other two input parameters which are given to both models include the radius of the planet R_0 at the reference pressure P_0 = 10^-2 bar, spanning from 0.2 R_J to 2.0 R_J, normalized to also be within [-1,1]. Secondly, the scale height H := k_ B T / (μ  g), where T refers to the temperature, μ to the mean molecular weight, and g is the gravity at the reference pressure P_0. log_10(H[ cm]) is sampled from a normal distribution with a mean of 7.1 and a standard deviation of 0.35 and is normalized accordingly. Extreme values of the scale height can change the geometry of the problem significantly and hence, make it harder for the PINN to learn a solution. The normal distribution ensures that this does not hinder the training of the PINN, while maintaining a broad range of values in the training data, allowing the neural network to approximate regions of higher data density more quickly and accurately <cit.>. Together, H and R_0 allow us to calculate the radius profile r(P) assuming a constant temperature but variable gravity:


    r(P) = R_0 ·[ 1 + H/R_0ln(P/P_0) ]^-1 .


Although isothermality is often an inadequate approximation for exoplanetary atmospheres <cit.>, we use it to assess if the PINN can manage the simplest scenario. That is why we further assume constant mass fractions (or mixing ratios) of all species throughout the atmosphere.

For the pure-absorption PINN, we also need to generate the α_a(P)-profiles, where the absorption coefficient α_a is extended to encompass both absorption and scattering sources. In this approach, scattering is effectively treated as a form of absorption, consistent with the traditional model. Ideally, these profiles should not just consist of a database of absorption coefficients from certain molecules at certain wavelengths, but should be more general by generating them randomly on the fly during training. For this, we first investigated example α_a(P)-profiles using petitRADTRANS <cit.>, which calculates the absorption coefficient α_a = κ X ρ for every species at every layer as the product of opacity κ, mass fraction X and density ρ. Our analysis included the correlated-k opacities of H_2O and CO <cit.>, CH_4 and CO_2 <cit.> and Na and K <cit.>, as well as Rayleigh scattering from H_2 <cit.> and He <cit.> and collisionally-induced absorption from H_2-H_2 <cit.> and H_2-He <cit.>. We found that most profiles follow either α_a ∼ρ∼ P or α_a ∼ P^2 (if κ∼ P) and hence ln(α_a)/ln(P) either equates to 1 or 2. Therefore, a truncated exponential profile around 1 and 2 (≤ 2) was used to create the profiles from their logarithmic gradient. During training, we observed that the PINN improves when using more accurate but diverse profiles. Therefore, we include 6 different types of randomly generated profiles in total:


    
  * ln(α_a)/ln(P)≈ 1
    
  * ln(α_a)/ln(P)≈ 2
    
  * random transition from ln(α_a)/ln(P)≈ 1 to 2
    
  * random transition from ln(α_a)/ln(P)≈ 2 to 1
    
  * randomly switching between ln(α_a)/ln(P)≈ 1 and ln(α_a)/ln(P)≈ 2
    
  * random negative values (≥ -0.5) for ln(α_a)/ln(P) in some profiles.




In Fig. <ref>, we compare the distribution of the generated logarithmic gradients with actual ones from all opacity sources and layers of the example atmosphere shown in Fig. <ref>. The more diverse set of gradient-profiles improved the error of a spectrum from initially around 3-5% to 1-2%.

The final α_a(P)-profiles are then created by cumulatively summing up the gradients in both directions from a middle layer (with index 49), which gets assigned a log_10(α_a)-value by a uniform distribution and hence acts as a random shift of the profiles.

To inform the PINN about the profile, we give it all the absorption coefficients α_i, i ∈ [0,99] from the 100 atmospheric layers, such that it has a total of 104 inputs.

In the case of Rayleigh scattering from hydrogen and helium, the scattering cross-section does not depend on the pressure. Hence, the scattering coefficient is simply proportional to the density α∼ρ∼ P, an example of which can be seen in Fig. <ref>, and can be parameterized with just one input variable.

The scattering PINN has additional inputs, namely the angle ϕ∈ [-π, π] as well as (half) the angular extent of the star as seen from the planet Δ_* ∈ [0, 0.3π]. Both are normalized to be within [-1,1].
A summary of the parameter bounds and how they are sampled can be found in Table <ref>.






 §.§ Training


The construction of the loss function for the absorption PINN initially appears straightforward, but involves intricate considerations. The boundary condition is established by the incident stellar radiation, characterized by the function u(x=-1, y) = 1.0, representing the intensity of starlight entering the system from the left boundary. This boundary condition is coupled with the RT equation governing the system:


    u/ x + α' · u = 0 ,


where the scaling factor is taken into account inside the α'  l_x α_a.
However, utilizing this equation directly as the loss function presents significant challenges in solving the problem. To understand why this is the case, let us consider two extremes for different regions of the atmosphere: α' ≪ 1 and α' ≫ 1.

For an extremely high value of α', the RT equation effectively becomes α' · u = 0 with the simple solution u = 0. Now consider small deviations of u from 0; Because of the high α' they can result in high values of the loss function and proportionally steep gradients. This means that the PINN effectively overfits these regions to be as close to zero as possible, even though other regions might be of more interest. We can solve this by dividing the whole equation by α'. This gives us approximately u = 0 as our loss function in such regions, resulting in the PINN learning the right solution without overfitting.

In the second case where α' ≪ 1, the form of Eq. (<ref>) would be preferable, since it would effectively be u/ x = 0. Again, the neural network would learn the right solution of u = const., but would not overfit the region as would be the case if we would divide the gradient by α'.

We can combine these two forms of the same equation by calculating both and taking the (element-wise) minimum.
The RT equation, which only considers Rayleigh scattering, is normalized in a similar fashion, details of which can be found in Appendix <ref>.

Meanwhile, the boundary condition of the scattering PINN has more complexity to it. On the left side, the light from the central star enters the atmosphere, this time with an angular extent. Because of this angular extent, also light from the upper boundary could enter the atmosphere, while from the right boundary, no light should enter but only leave the considered slab:


    u(x=-1, y, |ϕ|≤π/2)    = 1  if  |ϕ|≤Δ_*, else =0
    
        u(x, y=1, ϕ≤ 0 )    = 1  if  |ϕ|≤Δ_*, else =0
    
        u(x=1, y, |ϕ|≥π/2)    = 0 .





For the lower boundary, defining an exact boundary condition poses a challenge due to the uncertainty in the amount of light that could be scattered into the observed atmospheric slab from below. However, the absence of a boundary condition can potentially lead to erroneous solutions. To mitigate this issue, we approximate the lower boundary condition by setting the scattered light component to zero. This implies that no additional scattered light enters the system from this boundary. This approximation is expected to yield an accurate solution, provided that the optical depth of the atmosphere is sufficiently high.

For the absorption component, an accurate boundary condition can be found by recognizing the spherical symmetry of the system and rotating the cord of the light ray along a path of constant radius until the point of interest reaches the x-coordinate of x=0:


    u_a(x_0 ≤ 0,y=-1,ϕ≥ 0) = u_a(x=0,y',ϕ+Δϕ),
       where  y' = y(x_0, r(x_0,y=-1)),  tan(Δϕ) = l_x x_0/r_99.


Note that this only works for x ≤ 0. Otherwise, this boundary condition is set to zero, which again should hold, if the optical depth of the atmosphere is high enough at the highest pressures. The other edge case that we have to consider is if abs(ϕ+Δϕ) > Δ_*,max (= 0.3 π), for which u_a is zero because of the inherent architecture of the PINN. As this only happens for particularly low values of r_99 and large values of |l_x x_0| (i.e., for the outer layers of the atmosphere), the boundary condition is set to u_a = 1 for these rare instances.

The final boundary condition is to ensure the periodicity of ϕ: u(x,y,ϕ=π) = u(x,y,ϕ=-π).


Because of its boundary conditions, the scattering PINN has a special architecture, as shown in Fig. <ref>. This architecture has two outputs: One (u_a) should fulfill the RTE without scattering and utilizes a hard-coded filter, such that the neural network does not have to learn the non-differentiable boundary at |ϕ| = Δ_*, where the value of u can jump from 1 to 0. The other output (u_s) should then model the diffuse component of the light such that when adding the two outputs u = (u_a + u_s), they together fulfill the complete RTE with scattering.

If we would now sample ϕ uniformly in [-π,π], most of the "absorption output" u_a would equate to zero only because of the mentioned filter. This not only wastes a lot of training points, but also effectively lowers the weight (or importance) of the corresponding residual in the loss term. We mitigate this problem by sampling half of ϕ within [-Δ_*, Δ_*].


Both the boundary conditions and the residuals enter the loss function with their mean square. To make sure that the boundary conditions are fulfilled first, a weight of 0.5 was given to the residual term (ω_ℱ = 0.5 and ω_ℬ = 1 in Eq. (<ref>)). The lower boundary condition is given a weight of 0.1, as it is reasonable to fulfill only if u_a at x=0 is already sufficiently accurate. The final loss function is then the logarithm of their weighted sum.

The used optimizer is the higher-order quasi-Newton L-BFGS. We found that initially using Adam and then switching to L-BFGS yielded no benefit, so L-BFGS was used from the start. The parameters of the optimizer differ from the default PyTorch parameters in a learning rate of 0.8, the maximal number of both function evaluations and iterations per optimization steps of 50 (or 20), the tolerance change is set to the machine epsilon and the Strong Wolfe condition is chosen for the line search. As described in many PINN papers before <cit.>, we confirmed that having a larger batch size generally improves the final model. Accordingly, we always used the maximum batch size possible on an NVIDIA A100-SXM4-40GB GPU, with double the number of points for the residual loss compared to the boundary loss. To mitigate overfitting, the batch was resampled every 50 training steps (every 20 steps for the first 8000 steps in the case of the scattering PINN). The parameters that are computationally more expensive to sample (mainly the α_a-profiles, but also the two radius profile parameters) were sampled pre-training with a size of 10^7. During training, these parameters were then sampled from this saved array and loaded onto the GPU, thereby accelerating the process.

Throughout the training of the absorption PINNs, the inputs were resampled 100 times. Considering the number of points for which the residual loss is computed (2^21 or 2^22), the total number of input combinations amounts to roughly 2.1 × 10^8 or 4.2 × 10^8, depending on the network size. For the scattering PINN, which was resampled 400 times and repeated thereafter, with 2^19 residual points, the model was similarly trained on approximately 2.1 × 10^8 unique input combinations.

For the absorption PINN, a hyperparameter grid search is used to optimize the number of layers (4 or 6) as well as the width of each layer (32, 64, or 128). A PINN with more free parameters may improve our result even more, but the 6x128 network already needed more time to calculate a spectrum than the classical numerical method, so we decided against trying out a deeper or wider network. For each configuration, 7 PINNs were trained with different random Xavier initialization. They were evaluated on 95 test spectra with the help of petitRADTRANS <cit.>.

The number of subsequent layers of the scattering PINN were not optimized, although one layer was insufficient to accurately model the correct solution.




§ RESULTS




 §.§ Absorption PINN








After trying out what maximum batch size is possible for each hyperparameter configuration, we found that the three models with the most adjustable parameters (6x64, 4x128, 6x128) are able to utilize a batch size of 3 × 2^20 while the three smaller models (4x32, 6x32, 4x64) could use double the batch size (3 × 2^21).

The distribution of the root mean squared errors with respect to each example spectrum can be seen in Fig. <ref>. It seems that the 4x64 model consistently gives us the best result, although no architecture performs significantly worse.

Given that the errors of the smallest networks with a width of 32 were not significantly higher than those of larger networks, we experimented with an even smaller architecture of 4x16 and 4x8. For the 4x16 PINN, this adjustment resulted in a slight increase in error, similar to the worst-performing 6x32 network, and a considerably larger error in the case of the 4x8 PINN, but it produced diminishing returns in terms of speed.

For one of the 4x64 models, we also show the evolution of the different loss components during training in Fig <ref>. Here we see how the weight ω_ℱ of 0.5 allows the network to quickly fulfill the boundary condition, while still giving the residual loss component sufficient weight to decrease over the epochs. The final value of this residual loss log_10(ℒ_ℱ) = -4.0 already gives us a first order of estimate for the errors of the PINN of 10^-2.

We also checked for possible correlations of the error with any atmospheric parameters. We only found that for really low radii, our formula for the radius profile r(P) breaks down for the defined pressure range, yielding inaccurate results.

Fig. <ref> presents an example spectrum, notably incorporating sodium (Na) and potassium (K). These elements serve as robust indicators for assessing the PINN's capacity to generalize beyond its initial training data, due to their potential for exhibiting abrupt increases in absorption coefficients at specific atmospheric layers. During the training phase, the α_a(P) profiles we generated typically did not showcase logarithmic gradients ln(α_a)/ln(P) > 2. However, for these atomic species, the gradient between two layers can surge to values well above 2, occasionally surpassing 10. Such dramatic shifts can result in the optical depths of subsequent layers abruptly increasing from negligible to substantial values. Although α_a(P) profiles with such non-differentiable characteristics were not included in the training due to the complexity they introduced in minimizing the residual loss, the PINN demonstrates a commendable ability to approximate accurate solutions, even though this results in slightly larger errors than for known α_a(P) profiles. This success underscores the network's adaptability and effectiveness, showcasing its capability to handle complex scenarios that extend beyond the initial training data.

While the observed accuracy gives us confidence in our method, it does not offer any improvement in performance. Using the larger models of width 128 can even significantly slow down the simulation of a transit spectrum compared to the numerical method.

This contrasts with the results of <cit.>, who could significantly speed up their forward model using a neural network. But their method directly calculates the spectrum (in a fixed spectral range and spectral resolution) from the atmospheric parameters (also fixed in their number and range). Our PINN, on the other hand, calculates the transmission function u(y) for every individual α_a(P) profile, so for every wavelength and every source of absorption. Depending on the number of wavelength points and absorbing species, this can mean that the PINN has to calculate the solution in multiple batches, significantly reducing its speed. The advantage of our model lies within its versatility, as it is not trained on, e.g., specific molecules.




 §.§ Scattering PINN


Since we cannot easily compare the spectra obtained from the scattering PINN to the ones from a traditional numerical method, which do not take scattering into account, evaluating its performance is harder to do.

In Fig. <ref>, we can see an example solution with the two components, the absorbed u_a and scattered light u_s separated, as well as the corresponding residuals. These residuals show how well the RT equation is fulfilled and give us an estimate of the error. In the example, we can see residuals on the order of 10^-2 spanning a significant fraction of the considered slab.

Additionally, we compared our method to a fixed PINN for the same example, the result of which can be seen in Fig. <ref>. By fixed PINN, we mean that all the parameters of this PINN have been fixed to certain values during training, apart from the spacial coordinates x, y, ϕ (and Δ_*, which cannot be fixed because of the boundary condition at y=-1). Leaving the batch size the same should yield a solution of higher accuracy, as the fixed PINN has to explore a significantly smaller parameter space. Similarly to the absorption PINN, we can see errors of around 1%.

Finally, we want to explore the usefulness of this method by plotting an example spectrum (see Fig. <ref>). Even for the extreme case of Δ_* = 0.15π, which would correspond to an exoplanet orbiting a sun-like star (R_* = R_⊙) at a distance of a = 0.01 AU, the Rayleigh scattering barely influences the spectrum. This is because our assumptions for calculating the scattering integral only hold for small angles. Thus, while the scattering PINN is able to find an accurate solution, the approximation of parameterizing our problem with only one angle is not sufficient to accurately model Rayleigh scattering for large angular extents of its host star, which is when it becomes relevant.









§ LIMITATIONS
 

The assumptions under which both the absorption and scattering PINN operate include isothermality of the atmosphere and a radius range of R_0 ∈ [0.2, 2.0] R_ J.

In the case of the scattering PINN we further assume an orbital inclination of i = 90^∘. It also focuses narrowly on pressure-independent scattering cross-sections, characteristic of Rayleigh scattering from hydrogen and helium, and does not consider any absorbing species simultaneously being present in the atmosphere. Additionally, it lacks an exact lower boundary condition. The most significant shortcoming is the small angle approximation for the calculation of the scattering integral, which is ultimately insufficient to accurately model Rayleigh scattering.

Lastly, the fully-connected layers of both PINNs offer no interpretability, although the special architecture of the scattering PINN at least simplifies differentiating the diffuse light component from the total solution.




§ CONCLUSIONS


In this study, we have assessed the effectiveness of PINNs in simulating complex atmospheric phenomena like scattering, which are traditionally challenging and resource-intensive to model.

The absorption PINN demonstrated in this research exhibits a high level of precision in forecasting atmospheric transmission for transiting exoplanets, achieving an error margin within a few percent, even when confronted with unfamiliar non-differentiable α_a(P) profiles. This accuracy underscores the model's robustness and reliability in complex atmospheric simulations. However, it is worth noting that, in terms of computational speed, this method does not yet surpass the efficiency of existing numerical techniques. This opens up opportunities for further optimization and development in the realm of PINN methodologies to enhance their speed performance while maintaining their high accuracy. The use of other architectures than the simple MLP chosen in this work, could aid in achieving this. A 1D-CNN for the α_a inputs could, for example, significantly reduce the number of weights in the network.

The Rayleigh scattering PINN in this study, while showcasing potential, operates within certain constraints, as summarized in Section <ref>, that currently limit its broader applicability. Having these caveats in mind, nevertheless, a parameterized PINN could prove effective in modeling scattering processes in the atmosphere with a minimal additional computational cost beyond the training phase.

In this work, we have laid out one possible path of how to present this complex problem to a neural network. Here we want to emphasize the key ingredients for successfully training a PINN:

    
  * especially when dealing with a complicated differential equation, one has to carefully consider all possible configurations of parameters, how they affect the absolute values of the respective residual and normalize all cases accordingly
    
  * similar to the insight from <cit.>, it is important to respect causality when choosing weights for different parts of the loss function
    
  * having a larger batch size generally improves the accuracy of a PINN
    
  * incorporating physical knowledge into, e.g., the architecture of the PINN can significantly simplify finding the correct solution, especially if any non-differentiable boundaries are involved


However, further enhancements to this approach are necessary to calculate the correct solution to the RTE with scattering, where it becomes relevant for transit spectra. E.g., auxiliary PINNs <cit.> could aid in expanding the problem to a second angular dimension (θ), while making it possible to evaluate the scattering integral without any numerical integration. Experimenting with alternative architectures like FNOs <cit.> could also make addressing these challenges more feasible.

In the future, the inclusion of specific pressure-temperature profiles can be achieved by simply extending the input parameters, though this might slightly impact the model's accuracy due to the increased dimensionality of the problem. Integrating other scattering sources, such as clouds with their variable α(P) profiles and complex scattering phase functions Φ(θ, θ'), presents a more significant challenge. One major concern is the possible non-differentiability of cloud decks. But, as the absorption model's predictions for Na and K show, a PINN might be able to overcome such difficulties.




§ ACKNOWLEDGEMENTS


We thank the referee for the helpful comments.
We also thank Lukas Heinrich, Nicole Hartman and the ORIGINS Data Science Lab (ODSL) for valuable discussions and hardware support during the development of the project, which was made possible through the funding of the ORIGINS Excellence Cluster by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) under Germany’s Excellence Strategy - EXC 2094 - 390783311; in particular through ORIGINS Excellence Cluster's Seed Money Program.

This research has made use of the NASA Astrophysics Data System,
and the Python packages NumPy, SciPy, Matplotlib, and PyTorch.




§ DATA AVAILABILITY


The code, based on <cit.>, is available at <https://github.com/DavidDahlbudding/AtmosphericScatteringPinn> (commit https://github.com/DavidDahlbudding/AtmosphericScatteringPinn/tree/3c805f5f4a67be9caf76c644c91144fda956d6ea3c805f5). The respective models and other data can be downloaded from <https://zenodo.org/doi/10.5281/zenodo.10727497>.





mnras
 



















§ THE MULTI-LAYER PERCEPTRON


A feedforward neural network or multilayer perceptron (MLP) is a versatile approximator that maps x to y = f(x,θ), where θ defines the free network parameters that are adjusted during training.

What makes these standard MLPs so versatile is just simple linear algebra: a linear mapping W (the weights), a displacement b (the biases) and, most importantly, a non-linear activation function σ. The output y of one such a layer f_1 can then be computed with:


    y = f_1 (x) = σ(Wx + b),

where the activation function σ is applied element-wise: σ((x_1, x_2)^T) = (σ(x_1), σ(x_2))^T. Multiple of these layers can be stacked on top of each other to be able to represent highly complex functions between the input x and the output y = f_n∘⋯∘ f_2 ∘ f_1 (x) (where ∘ denotes a function composition: f_2 ∘ f_1 (x) = f_2(f_1 (x))) <cit.>. The layers between the input and the output layer are called the hidden layers and the number of hidden layers is called the depth of the network, while the number of nodes per layer is called the width. This simple architecture is also mostly used in this paper.

To train a neural network, we need another important function, the so-called objective or loss function. This function should be minimized to achieve a desired goal (e.g., the mean squared error when fitting a function to given data points). Then, the gradient of the loss function with respect to the weights and biases can be cumulatively calculated for a batch of data using automatic differentiation, and the neural network can "move" towards a minimum by adjusting its parameters in small steps in the negative direction of this gradient.




§ THE SCATTERING INTEGRAL
 

As mentioned in Section <ref> we need to make a few approximations valid for small angular extents of the star for the problem to be parameterizable by only one angle (ϕ). Here, we choose different symmetries for the two components of the radiant flux: u_a and u_s. Because they are added together to get the total solution to the RTE, we can calculate the total integral by adding the two integrals for each component:


    ∫Φ  u  Ω = ∫Φ  u_a  Ω + ∫Φ  u_s  Ω


For the absorption component u_a we choose the conventional spherical (local) coordinate system, such that the angle ϕ∈ [-π, π] lies in the x-y-plane and θ∈ [0,π] is the angle relative to the z-axis. If we now take a fixed ϕ, let θ vary and expand this into a plain, the intersection of this plain with the sphere of an atmospheric layer is a circle. For small angular extents of the star each light ray for varying θ travels roughly the same path through this circle, which is why we assume the symmetry of u_a(ϕ, θ) ≈ u_a(ϕ, θ=π/2).

We can parameterize θ' in terms of ϕ' such that we can integrate over the complete solid angle of the star with angular extent 2 Δ_*, for which u_a ≠ 0, by using the law of cosines with θ = π/2:


    cosΔ = sinθ' cos(ϕ-ϕ').


Inserting this into the scattering phase function Φ(Δ) (<ref>) and the integration limits, yields:


    ∫_-Δ_*^Δ_*ϕ'   u_a(ϕ') ∫_-θ'(ϕ')^θ'(ϕ')θ'   |sinθ'|  Φ(Δ) =
       ∫_-Δ_*^Δ_* u_a(ϕ')  3/2√(1-a^2)( cos^2(ϕ-ϕ') ( 1-a^2/3 - 1) - 1 ) ϕ',
       where  a := cosΔ_*/cosϕ'


For exploiting a symmetry of the scattering component we have to rotate our local coordinate system. The ϕ_s, coordinate now lies in the y-z-plane and θ_s is the angle relative to the x-axis. The absorption component u_a therefore is a concentrated beam of light around θ_s = 0 for small Δ_*. Since the scattering phase function is only dependent on the angular distance Δ, relative to θ=0, we get Δ = θ. Hence we can approximate, while allowing for some slight asymmetry: u_s(θ_s, ϕ_s ≶ 0) ≈ u_s(θ_s, ϕ=±π/2). The integral can therefore be calculated using:


    ∬Φ(Δ)   u_s(θ_s'   , ϕ_s')   |sinθ_s'|  θ_s'  ϕ_s' =
    ∫_0^π |sinθ_s'|  θ_s'    [ u_s(θ_s', ϕ_s'=-π/2) ∫_-π^0 Φ(Δ) dϕ_s' + .
       .   u_s(θ_s', ϕ_s'=π/2) ∫_0^πΦ(Δ) dϕ_s' ] .


The integral over ϕ_s' can again be integrated analytically by inserting the law of cosines (assuming ϕ_s = ±π/2) into the scattering phase function Φ(Δ) (<ref>):


    ∫_0^±π   Φ(Δ) dϕ_s' =
    ±3/4   [ π( 1 + cosθ_s cosθ_s' ±1/2sinθ_s sinθ_s') .
       . ∓sgn(ϕ_s) sin(2θ_s) sin(2θ_s') ] .


The numerical integration employs the Trapezoidal rule with a total of 19 sample points in the interval [-π,π], where 5 of these points are equally spaced in the interval [-Δ_*, Δ_*] and the remaining intervals [-π, -Δ_*] and [Δ_*, π] each have another 7 equidistant sample points.




§ RESIDUAL LOSS FOR SCATTERING PINN
 

As mentioned in Section <ref>, we have to divide the gradients in the RTE (Eq. <ref>) with the respective scale lengths l_x and l_y. Given in cm, these parameters are quite large and can therefore diminish the value of the residual loss, especially for low values of α. Hence, we multiply the whole equation with l_x:


    cos(ϕ)  u/ x   + sin(ϕ) l_x/l_y u/ y + l_x α_s · u
    
        - l_x α_s/4π   ·∫Φ(θ, ϕ, θ', ϕ') u(θ', ϕ') dΩ' = res_1.


The remaining problem is that the value of l_x/l_y can be on the order of 10, possibly inflating the value of the residual loss for a certain set of parameters. Because of this, we further normalize this formulation of the residual with norm = |cos(ϕ)| + l_x/l_y |sin(ϕ)|.

The second formulation is similar to the one for the absorption PINN, where we divide the RTE by α_s:


    cos(ϕ)/l_x α_s u/ x   + sin(ϕ)/l_y α_s u/ y + u
    
        - 1/4π   ·∫Φ(θ, ϕ, θ', ϕ') u(θ', ϕ') dΩ' = res_2.


The complete residual is then again the minimum of the two formulations:


    res = min( res_1/norm, res_2 ).











 Paper Title: Modeling stochastic eye tracking data: A comparison of quantum generative adversarial networks and Markov models.












shailendra.bhandari@oslomet.no
0000-0002-7860-4854

  Department of Computer Science, OsloMet – Oslo Metropolitan University, N-0130
  Oslo
  Norway



pedrog@oslomet.no
XXXX-XXXX-XXXX-XXXX

  Department of Computer Science, OsloMet – Oslo Metropolitan University, N-0130
  Oslo
  Norway





pedrolin@oslomet.no
0000-0002-8176-666X

  Department of Computer Science, OsloMet – Oslo Metropolitan University, N-0130 Oslo
  Norway;
  
  
  
  
  Simula Research Laboratory, Numerical Analysis and Scientific Computing, 0164
  Oslo
  Norway



§ ABSTRACT 
 We explore the use of quantum generative adversarial networks QGANs for modeling eye movement velocity data. We assess whether the advanced computational capabilities of QGANs can enhance the modeling of complex stochastic distribution beyond the traditional mathematical models, particularly the Markov model. The findings indicate that while QGANs demonstrate potential in approximating complex distributions, the Markov model consistently outperforms in accurately replicating the real data distribution. This comparison underlines the challenges and avenues for refinement in time series data generation using quantum computing techniques. It emphasizes the need for further optimization of quantum models to better align with real-world data characteristics.




Modeling stochastic eye tracking data: A comparison of quantum generative adversarial networks and Markov models
    Pedro Lind
    August 5, 2024
================================================================================================================





§ INTRODUCTION

The generative adversarial networks (GANs) <cit.> are the type of artificial intelligence algorithms that consist of two neural networks: the generator and the discriminator. The generator is trained to create realistic data, which are then used as negative instances for the discriminator to learn from. This process is also known as generative modeling, a branch of machine learning that involves training a model to produce new data similar to a given dataset. Meanwhile, the discriminator improves its ability to differentiate between authentic and generated data, providing feedback to the generator when it produces data that is not convincing. Throughout the training process, the generator continually strives to enhance its ability to create increasingly convincing forgeries. Simultaneously, the discriminator aims to improve its ability to discern between real and fake data accurately. Therefore, a GAN is a battle between two adversaries, the generator and the discriminator. The point of balance in this dynamic battle is achieved when the generator is capable of producing forgeries that are indistinguishable from the original training data, leaving the discriminator with a mere 50% certainty in distinguishing real from fake outputs.

GANs become tremendously interesting and challenging topics in Machine Learning <cit.>. It is beyond the boundary line of computational creativity, showcasing increasingly amazing examples each year <cit.>. While GANs have shown remarkable power and interest, it is also limited by various challenges: difficulties in achieving stable training of the GAN <cit.>, the vanishing gradient, and the mode collapse <cit.>. Efforts from both the academic community and the commercial sector have been made to address these problems <cit.>. The exploration of Quantum Deep Learning models has been gaining popularity recently, followed by the concept of quantum supremacy, which Google has showcased <cit.>, along with the potential for a quantum edge in machine learning. Recently, discussions around Quantum Generative Adversarial Networks (QGANs) mainly focused on applications like efficient data handling or seeking benefits over traditional classical models <cit.>. An example of a successful QGAN involves using a manageable number of quantum bits to train the GAN to recognize and replicate random patterns. The motivation behind quantum-enabled research is the potential quantum advantage that could partially alleviate the computational complexity issue and search for the best solution to the mode collapse and vanishing gradient problem. 

In the context of integrating quantum principles into neural networks, it is important to recognize foundational work that has set the stage for contemporary advancements. One of the important works is by Behera et al.  <cit.>, who developed a recurrent quantum neural network (RQNN) model to describe eye movements in tracking moving targets. Their work utilized a nonlinear Schrödinger wave equation to mediate the neural responses, providing a foundation for understanding how quantum mechanics can be integrated with neural processing. This early exploration into quantum neural networks is important. Our approach to QGANs aims to enhance the generative capabilities of neural networks through the incorporation of quantum bits, quantum gates, and the fundamental properties of quantum mechanics: superposition and entanglement.

The main goal of this paper is to use the QGAN and test its performance in reproducing stochastic trajectories and compare them with some benchmarks from stochastic modeling. Previously, Lencastre et al. <cit.> tested the classical GANs'performance, comparing with a mathematical model, namely a Markov chain. They found that GANs struggle to capture rare events and cross-feature relations and are unable to create faithful synthetic data successfully. Here, we test the QGANs performance compared with the previous classical GANs and mathematical models. One of the problems of the classical GAN is the vanishing gradient problem when generating discrete data <cit.> to overcome this issue we explore the ability of quantum computing, which naturally has advantages to equip GANs with the ability to deal with discrete data.



§ THE QUANTUM GAN MODEL




In our study, the utilized GAN framework comprises two neural networks: the generator and the discriminator, as illustrated in Figure <ref>. These networks undergo alternating training phases. Consider a classical training data set X = {x^0, ..., x^s-1} drawn from unknown time series distribution. The generator (G_θ) receives a random noise vector z and then produces the generated sample G(z). The discriminator D_ ϕ is trained to distinguish between the training data x and the generated data G(z). The parameters of D_ ϕ are updated in order to maximize <cit.>:


    𝔼_x ∼ P_d(x)[log D_ϕ(x)] + 𝔼_z ∼ P_z(z)[log(1 - D_ϕ(G_θ(z)))]  ,

where P_d(x) is the real-time series distribution and the P_z(z) is the distribution of the input noise. The output D_ϕ(x) can be explained as the probability that D_ϕ thinks the sample x is real. The goal of the discriminator here is to maximize the probability of correctly classifying to make D_ϕ  a better adversary so that G_θ has to try harder to fool the D_ϕ. Similarly, the parameters of the G_θ are updated to maximize: 

    𝔼_z ∼ P_z(z)[log (D_ϕ(G_θ(z)))]  ,

to convince D_ϕ that the generates samples G(z) are real. This process is illustrated in Figure <ref>. 

The goal of optimizing classical GANs can be approached from several perspectives. In this study, we adopt the non-saturating loss function <cit.>, which is also implemented in the original GAN publication's code <cit.>. The generator loss function is given by:

    L_G(θ) = -𝔼_z ∼ P_z(z)[log (D_ϕ(G_θ(z)))]  ,

which aims at maximizing the likelihood that the generator creates samples that are labeled as real data samples. Also, the discriminator's loss function is given by the expression:

    L_D(ϕ) = 𝔼_x ∼ P_d(x)[log D_ϕ(x)] + 𝔼_z ∼ P_z(z)[log(1 - D_ϕ(G_θ(z)))]  ,

which aims at maximizing the likelihood that the discriminator labels training data samples as training data samples and generated data samples as generated data samples. In practice, the expected values are approximated by batches of size m

    L_G(ϕ, θ) = -1/m∑_i=1^m [log D_ϕ(G_θ(z^(i)))],

and

    L_D(ϕ, θ) = 1/m∑_i=1^m [log D_ϕ(x^(i)) + log (1 - D_ϕ(G_θ(z^(i))))],

for x' in X and z' sampled from P_d(x). 













 §.§ Quantum generator






The model we designed is a hybrid architecture (quantum-classical) where the input is in classical form with a classical discriminator and generates an output by measurement of a state from the parametrized quantum circuit. Specifically, it is a quantum GAN that generates a classical discrete time series, which is composed of a parametrized quantum circuit as a generator and a classical feed-forward neural network as a discriminator. 

The parameterized quantum circuit is then expressed as G(θ), where θ = {θ_1, …, θ_k} represents the set of parameters which can be tuned as necessary. This circuit is operated by the single qubit R_Y and R_Z rotation gates, along with the two-qubit Controlled-NOT (CX) gates, all operating on an initial state <cit.>. When applying the rotation operators on each qubit, it is expressed as:

    ∏_i=1^N R_Z^i(θ^i _l,2) R_Y^i(θ^i _l,1),

where l denotes the l^th layer and i denotes the i^th qubit. R_z and R_Y are the rotation gates given by following expressions: 

    R_Z(θ) = e^-iθσ_z/2 = cosθ/2I- isinθ/2σ_z = [ e^-iθ/2       0;       0  e^iθ/2 ]  ,
    
    R_Y(θ) = e^-iθσ_y/2 = cosθ/2I- isinθ/2σ_y = [   cosθ/2 -isinθ/2;  isinθ/2   cosθ/2 ].



The Controlled-NOT gate is used to create entanglement between the qubits, and this process is expressed as:

    ∏_i=1^N CU^i_(i mod N)+1  ,

where i denote the controlled qubit and (i mod N)+1 represent the target qubit. The number of single-qubit gates represents the number of parameters in the generative quantum circuit in a circuit which is 3N per layer.

The parametrized quantum circuit for generating N-bit samples consists of N qubit and U(θ_L) layers as shown in Figure <ref>. The input quantum state |ψ_in⟩ is initialized to |0⟩^⊗ N and passed through L layers of unitary operator. All the qubits outcomes of the circuit are measured on the computational basis at the end of the circuit. The measurement outcome is then collected to form N-bit sample X. Overall, the quantum generator is trained to transform N-qubit input to N-qubit output state <cit.>:

    |g_θ⟩    =    G_θ|ψ_in⟩    =   ∏_p=1^2^N( ⊗_q=1^N (R_Y R_Z (θ^q,p)) U(θ_L) ) ⊗_q=1^N (R_Y R_Z (θ^q,0) |ψ_in⟩)    =    ∑_j=0^2^N - 1√(p_j)|j⟩  ,

where samples are drawn by measuring the output state |g_θ⟩ in computational basis with measurement outcomes |j⟩, j ∈{0,1....,2^N-1}. N is the number of qubits. In addition, the term p_j is the sampling probability associated with the basis state |x_j⟩. For m data samples (g^l) from the quantum generator and m randomly chosen training data samples (x^l), where l = 1,...m, the generator and discriminator are optimized with a respective loss function.

The generator's goal is to generate data that can fool the discriminator. The generator loss function for a data batch of size m is expressed as:
    
    L_G(ϕ, θ) = -1/m∑_i=1^m [log D_ϕ(g^l)],

or equivalently,

    L_G(ϕ, θ) = ∑_j=0^2^N-1 p_θ ^j log (D_ϕ(g^j)),

where p_θ ^j = |⟨j|g_θ⟩|^2.
The loss function of the discriminator is given by
    
    L_D(ϕ,θ) = 1/m∑_l=1^m [log D_ϕ(x^l)+log(1-D_ϕ(g^l))].


Gradient-based optimization techniques can enhance convergence speed, particularly near local optima in a convex region, when compared to methods that do not utilize gradient information <cit.>. This method for calculating analytical gradients <cit.> for the variational circuit is as discussed below. The parameters θ can be updated with gradient-based methods that require the evaluation of

    ∂ℒ_G (ϕ, θ)/∂θ^i,l = - ∑_j=1^m∂ p^j_θ/∂θ_i,llog (D_ϕ (g^j)) .


Eq. (<ref>) can be further evaluated based on Ref. <cit.>

    ∂ p_j^θ/∂θ^i,l = 1/2( p_θ^i,j_+^j - p_θ^i,j_-^j ) ,

with θ^i,l_± = θ^i,l±π/2 e_i,l and e_i,l denoting the (i, l)-unit vector of the respective parameter space.





The selection of parameters θ is important, especially when l > 1. A key factor to consider is the circuit depth, as increasing it enhances the complexity of the quantum circuit. Utilizing a circuit depth greater than 1 is advantageous for training on complex datasets, as deeper circuits can capture more intricate structural patterns. Thus, it is essential to design a circuit that is both deep and rich in parameters. This choice is driven by the need to effectively capture and represent the complex distribution characteristics of the data. By increasing both the number of parameters and the circuit's depth, we can significantly enhance the generator's ability to accurately model the data. Such a strategy is pivotal for quantum generators tasked with processing and generating data distributions of considerable complexity.





 §.§ The discriminator

The discriminator model, a classical feed-forward neural network, is constructed and implemented using PyTorch's <cit.> neural network module. It comprises a three-layer LSTM <cit.> (Long Short-Term Memory) network with 128 hidden units and bidirectional processing. This LSTM feeds into a series of linear layers that further process the data. A dropout of 0.3 is applied after the first linear transformation to prevent overfitting. The output of the discriminator is a single scalar value representing the probability that the input data is from the real distribution, obtained through a sigmoid activation function.

The discriminator is trained using AMSGRAD <cit.> with a learning rate of 0.002 and a momentum of 0.999. Using the first and the second momentum terms, AMASGRAD is a robust optimization technique for non-stationary objective functions and noisy gradients <cit.>. The loss function employed is binary cross-entropy (BCE) and is suitable for binary classification tasks such as distinguishing between real and fake data sequences. During training, the discriminator evaluates both real and generated data sequences, updating its weights to minimize the loss function. The training stability is maintained using a gradient penalty on a discriminator's loss function <cit.>. The analytic computation of the quantum generator loss function gradients is implemented based on Equations. (<ref>) and (<ref>). The training procedure is iteratively conducted for a specified number of epochs, with both generator and discriminator losses monitored to ensure the model's convergence.

The adversarial training of the quantum GAN described above is illustrated in Algorithm 1. The training of the GAN iterates for the specified number of epochs, until the convergence of the loss function. At each epoch, the loss function optimizes alternatively to parameters θ and ϕ. 





    



    






§ MARKOV-CHAIN MODELS OF TIME-SERIES


We generate a two-dimensional time series based on a Markov transition matrix T, where each element T_ij denotes the transition probability from state i to state j. Mathematically, a time series X_t is said to follow a Markov process if it fulfills a Markov property:

    Pr(X_t=j = x̂_j | X_t=j-1 = x̂_j-1, …,     X_t=0 = x̂_0) = 
        Pr(X_t=j = x̂_j | X_t=j-1 = x̂_j-1),

for all positive integers j, where capital letters mean stochastic
variables at different time steps and lowercase letters are the respective values of those variables.

In a Markov model, time-series generation is based on computing the conditional probability Pr(X_t=n+1 = x_n+1 |  X_t=n = x_n). This is estimated empirically with the Gaussian estimation kernel K as follows:

    Pr(X_t=n+1 = x_n+1 |  X_t=n = x_n) = Pr(X_t=n+1 = x_n+1, X_t=n = x_n)/Pr(X_t= n = x_n),

with

    Pr(X_t = n+1 = x_n+1, X_t=n = x_n) =    1/(N̂ - 1)h^2∑_i=1^N̂-1 K(x_n+1 - x̂_i+1/h) 
       × K(x_n - x̂_i/h),

and the kernel K defined as

    K(x_n - x̂_i/h) = 1/√(2π)exp(-1/2(x_n - x̂_i/h)^2 ).

The bandwidth h of the Gaussian estimation kernel is computed following Silverman's rule <cit.>:

    h = (4σ̂^5/3N̂)^1/5≈ 1.06 σ̂ (N̂ - 1)^-1/5,

where σ̂ is the standard deviation of the sample and N is the number of data points in our sample.

For empirical data analysis, the conditional probability can be represented by a transition matrix T of dimension N_s × N_s with entries

    T_ij = Pr(X_t=n+1∈ [k_i, k_i+1) | X_n ∈ [k_j, k_j+1)),

with i, j ∈ N, i, j ∈ [0, N_s) and k_m > k_n for m > n.

The efficacy of the proposed method is contingent upon three critical factors: (i) the robustness of the Markov assumption <cit.>; (ii) the number of states N_s, which directly influences computational efficiency and scales with N_s^4 for bi-dimensional processes; and (iii) the sample size N, which affects the bandwidth h and thereby determines the spatial resolution of the model. We implemented this process using the open source code accessible via GitHub <cit.>.





§ DATA AND THE STATISTICAL MEASURE

We used the Eye-tracking data[All data collected was anonymized and follows the ethical requirements from the Norwegian Agency for Shared Services in Education and Research (SIKT), under the application with Ref. 129768.] which was gathered at Oslo Metropolitan University utilizing the advanced eye-link Duo device, capable of reaching up to 2000 Hz but was adjusted to 200 Hz for this study. The measurements, recorded in screen pixels, were taken as participants searched for specific targets in images from the book “Where's Waldo?” <cit.>. Each of the eight selected images was viewed for two minutes, a duration not expected to suffice for finding all targets but intended to keep participants focused. The data from eye-tracking measurements were preprocessed and utilized to train a QGAN. Initially, the velocity data for both left and right eyes are calculated by finding the Euclidean distance between consecutive position points and then dividing by the time interval, set at 1/200 seconds, to convert this distance into velocity. This velocity data is then aggregated into a structured format suitable for feeding into the discriminator. The data undergoes resampling to a fixed interval of 10 seconds, aggregating measurements by their mean within these intervals to reduce noise and temporal variability. The resultant dataset is then normalized using a MinMaxScaler to ensure the data fits within the required operational range of  0 and 1 <cit.>. The normalized data is transformed into sequences of a specified length (100 data points in this case), which are then used to create training batches. These sequences were fed into the discriminator, where it assessed them and learned to discern features that distinguish real data from the outputs generated by the quantum generator. This process aided the discriminator in enhancing its accuracy in identifying authentic samples.

The performance of the QGAN model is measured by the Jensen-Shannon (JS) divergence <cit.>. This metric, a symmetrized variant of the Kullback-Leibler (KL) divergence <cit.>, offers a symmetric distance measure between probability distributions. The JS divergence <cit.> is defined by 

    D_JS(P | Q) = 1/2 D_KL(P | M) + 1/2 D_KL(Q | M),

where P and Q are distributions, and M = 1/2(P + Q). The KL divergence is standard for assessing distributional similarity which enhances maximum likelihood estimates. The JS divergence, while preserving these properties, is more intuitive as it assesses the approximation of synthetic distributions to empirical ones. Therefore JS divergence in this case can be relevant for discriminators within QGANs to distinguish synthetic data from the generator.





§ RESULTS




    















The implementation of QGAN was done using PyTorch <cit.> and IBM's Qiskit <cit.> simulator 0.45. The discriminator's LSTM network is built with linear layers and a dropout mechanism distinguished between real and generated data sequences. The quantum generator is modeled by a parametrized quantum circuit and was optimized to replicate the complex distribution of the eye movement velocities.  For the QGAN applied to eye-tracking data, the discretization and representation in quantum states was done using 3 and 4 qubits, each offering a different resolution of data representation. With 3 qubits, the velocity data is mapped onto 2^3 = 8 discrete levels, resulting in the quantum state:

    G_θ|ψ_in⟩ = ∑_i=0^7√(p_θ^i) |i⟩,

where |i⟩ represents the i-th discretized state of the velocity data. p_θ^i are the associated probabilities, learned through the training of the quantum generator as elaborated in Eq. (<ref>). The training of our QGAN model was carried out over 300 epochs, with data batches of 500 samples each. 

Figures <ref> illustrate the histograms of log-transformed real versus generated data for three and four-qubit configurations across five layers. The use of log transformation ensures a focus on the distribution's dynamics rather than its absolute scale to facilitate a clearer understanding of the model's performance across varying complexities.

The evaluation of configurations of 3 and 4 qubits presents a nuanced understanding of the models' capabilities in reproducing real eye movement velocity data. Tab. <ref> shows the statistical analysis for the generated data across different circuit layers including the Markovs model and the real data. The data generated by the QGANs exhibits a trend of increased skewness and kurtosis with increasing layers in quantum circuits, indicating a divergence from the real data's distribution especially in distribution tails. The generated distributions to the actual data are further quantified through the JSD values, as shown in Tab. <ref>. For 3 qubits QGAN performance is relatively stable across layers. For 4 qubits the second and third layers show optimal JSD, indicating minimal divergence points. Notably the Markov model exhibits superior performance with the lowest JSD, highlighting its efficiency in achieving minimal divergence compared to the QGAN-generated data. 


The comparison to a mathematical Markov model, with its significantly lower JSD value, highlights the distinct challenges faced in quantum data generation. Figure <ref> illustrates the comparative histograms of log-transformed eye movement velocities between real data and the generated data by a Markov model. The visual comparison and the lower JSD value clearly emphasize the effectiveness of Markov models in simulating complex data distributions closely resembling real data.

















§ DISCUSSION AND CONCLUSIONS


The comprehensive analysis of QGANs alongside Markov models, particularly in modeling stochastic eye movement velocity, underscores the promising aspects of quantum computing in handling complex datasets. However, when we put side by side with the mathematical Markov model, our analysis paints a comprehensive picture of the present state of synthetic data generation via quantum methods. 

We have shown that by increasing the number of qubits as well as the number of layers, QGANs improve their performance in reproducing eye-gaze trajectories.
However, 
despite QGANs' innovative nature and their capability to discern intricate eye gaze data patterns, Markov models outshine them in closely mirroring the actual data distributions.

This observation not only reinforces the previously identified challenges faced by AI in creating synthetic data—a topic already explored within classical GANs <cit.> research, but it also broadens this understanding to include the emerging domain of quantum computing. Such findings underscore the critical need for ongoing advancements in quantum algorithm development. The goal is clear: to overcome these obstacles and fully harness quantum computing's potential to produce synthetic datasets that closely resemble their real-world analogs.






§ ACKNOWLEDGMENTS

We extend our sincere gratitude to Mr. Ramesh Uprety for his invaluable contributions and insightful discussions on the architecture of GANs, hyperparameter tuning strategies, and careful data-cleaning processes. We are deeply appreciative of his collaborative spirit and expert advice, which have significantly enriched this work. This work was funded by the Research Council of Norway under grant number 335940 for the project `Virtual-Eye'.




ACM-Reference-Format







