[1]Marco Roth

1]Fraunhofer Institute for Manufacturing Engineering and Automation (IPA),
Nobelstrasse 12, Stuttgart, 70569, Germany

2]Institute of Industrial Manufacturing and Management IFF, University of
Stuttgart, Allmandring 35, Stuttgart, 70569, Germany

# Reinforcement learning-based architecture search for quantum machine
learning

Frederic Rapp<EMAIL_ADDRESS>David A. Kreplin
<EMAIL_ADDRESS>[  [

###### Abstract

Quantum machine learning (QML) leverages the large Hilbert space provided by
quantum computing for data encoding, typically realized by parameterized
quantum circuits (PQCs). While classical machine learning deals extensively
with problem-specific model design, QML models mainly use hardware-efficient
and heuristic circuit designs for PQCs. This work presents a novel approach
employing the reinforcement learning algorithm MuZero to generate problem-
specific PQCs to improve the QML performance. Diverging from previous search
algorithms, we adopt a layered circuit design to significantly reduce the
search space. Furthermore, we utilize cross-validation scoring to train the
reinforcement learning algorithm, rewarding the discovery of high-performing
circuits. In benchmarks we compare our tailored circuits with reference
circuits from the literature, randomly generated circuits, and circuits
generated by genetic algorithms. Our findings underscore the efficacy of
problem-tailored encoding circuits in enhancing QML model performance.

###### keywords:

quantum computing, quantum machine learning, reinforcement learning,
architecture search

##  1 Introduction

Quantum machine learning (QML) is a promising application of today’s noisy
intermediate-scale quantum (NISQ) computing [1], and has gained significant
attention in recent years [2]. Most QML methods rely on encoding data into the
high-dimensional Hilbert space accessible through quantum computing [3, 4].
Parameterized quantum circuits (PQCs) [5], which use parameterized gates that
depend on input data and trainable weights, have become a focus of recent
research to encode data. The weights are often optimized through an
optimization loop based on traditional routines [6]. However, the high
dimensionality of Hilbert spaces can lead to exponentially vanishing gradients
if the quantum circuit is not chosen properly [7, 8]. PQCs, in the following
also denoted as encoding circuits, are central to many methods used in QML,
such as quantum neural networks (QNNs) [9, 5] and quantum kernel methods [3,
10, 11]. Various encoding strategies, such as data re-uploading or non-linear
encoding, have been introduced in the literature [12, 13, 14, 15]. In contrast
to other applications of PQCs in variational algorithms, such as optimization
with the quantum approximate optimization algorithm (QAOA) [16] or quantum
simulations using the variational quantum eigensolver (VQE) [17, 18], the
circuits used in QML offer a high degree of flexibility, allowing for
adaptation to specific hardware configurations, such as connectivity or native
gates. Commonly used architectures often share properties like a layered
structure where each qubit is acted upon by the same operation simultaneously
[19]. Current designs are often based on these design principles only, and are
not necessarily tailored for optimal model performance.

This heuristic design of QML models contrasts with the no free lunch theorem
[20], which asserts that optimal performance is achieved by a problem specific
model. Studies have shown that generic QML models often fail to outperform
classical ML approaches [21, 22]. As a result, recent research has focused on
incorporating geometric properties, such as symmetries, into QML architectures
to provide models with a informed inductive bias [23, 24, 25, 26].
Nonetheless, many of these approaches require expert knowledge of problem
domains and quantum circuit design, a prerequisite shared by other informed
design choices [27]. Despite existing works discussing the necessity of
considering structure data structure in PQCs [22], there are no established
guidelines for constructing suitable PQCs for specific QML problems. This gap
motivates the development of automated methods that can create and evaluate
different encoding circuits, capable of generating problem-specific encoding
circuits from input data. Current research primarily employs genetic
algorithms for this architecture search [28, 29, 30].

In this work, we introduce a novel approach for QML circuit generation using
model-based reinforcement learning (RL) with the MuZero algorithm [31]. In RL,
an agent develops a policy by interacting with an environment to maximize
cumulative rewards. For circuit generation, the observation space includes the
current circuit, actions determine the next layer of quantum gates, and
rewards are based on the QML model’s training performance measured by the
cross-validation score. MuZero uses deep neural networks to map the
interaction between circuit architecture and QML model performance into a
hidden state representation, significantly reducing the need for detailed
environmental information. Actions are selected in a Monte Carlo Tree Search
(MCTS) [32] fashion, which relies on internal actions and rewards, reducing
the number of evaluations of the QML model. Figure 1 illustrates our method’s
conceptual layout. We demonstrate through benchmarks that the MuZero circuit
search algorithm can generate problem-specific encoding circuits that surpass
both reference circuits from the literature and those derived from random
sampling or genetic search algorithms [28]. The results are consistently
favorable across three different datasets: two involving regression tasks and
one classification task. The resulting QML models, which utilize projected
quantum kernels (PQKs) [11], exhibit performance that is either comparable to
or slightly superior to various classical ML models.

The remainder of this work is structured as follows. In Section 1.1, we
provide an overview of prior work in the field of quantum architecture search.
Section 2 defines the general problem setting and elaborates the MuZero
circuit search algorithm. In Section 3, we demonstrate the versatility and
circuit-creating capabilities of our algorithm by benchmarking it on several
datasets. Finally, we discuss the results of the MuZero circuit search
algorithm in Section 3.2.

###  1.1 Prior work

The integration of RL techniques into quantum circuit design has been explored
in various approaches. One key aspect focuses on optimizing quantum circuits
to meet quantum hardware requirements, such as reducing gate count or
enhancing qubit routing efficiency [33]. A notable contribution in the area of
general quantum circuit optimization is the methodology proposed by Ruiz et
al. [34], which employs tensor networks to reduce the number of gates in
quantum circuits. This framework focuses on minimizing the gate overhead in
deep circuit architectures, thereby facilitating practical implementations on
quantum hardware.

The integration of RL techniques into quantum circuit design has been explored
by [35] and [36]. The first study [35] applies RL techniques to create and
optimize quantum circuits for VQE, aiming to enhance accuracy while
maintaining a low circuit depth. Ref. [36] addresses the challenges associated
with using RL for quantum circuit design, introducing the concept of using
Markov decision processes to guide the application of continuously
parameterized quantum gates to optimize quantum circuits, specifically in the
context of quantum state preparation. While these studies utilize RL for
quantum circuit architecture search, their primary applications lie outside of
QML. Additionally, in contrast to our work, they rely on model-free RL
approaches.

In the context of QML, quantum circuit design has been investigated in the
following two references. Incudini et al. [29] propose an architecture search
for quantum kernels, wherein they employ techniques from neural architecture
search and AutoML. They demonstrate the effectiveness of automated methods
over manual design approaches, as illustrated by their experiments with high-
energy physics datasets. Among other techniques, such as Bayesian
optimization, they also employ RL optimization based on a model-free
state–action–reward–state–action (SARSA) algorithm. Ref. [28] introduces a
multiobjective genetic algorithm for the automatic generation of quantum
feature maps for fidelity-based quantum kernels. We include a slightly
modified version of their implementation in our benchmarks in Sec. 3.1.

Notably, our research incorporates and extends upon these methodologies. In
contrast to previous works, we focus on a layered approach that considerably
reduces the search space. Furthermore, we employ model-based RL to decrease
the number of circuit evaluations by incorporating the MuZero algorithm [31].

##  2 Methods

Figure 1: Automatic PQC generation layout. The MuZero model observes the
current PQC configuration, and transforms it into a hidden state
representation $s_{t}^{n}$. By performing an MCTS for each time step $t$ the
model obtains a policy $\Pi_{t}$ from which a action $a_{t}$ is sampled [31].
The action represent a choice of unitary operator (e.g., $H,CX,...$) which is
appended on the current encoding circuit in a layered fashion (one gate for
each qubit). Each circuit observation is used as basis for a QML model of the
form presented in Eq. (2). Based on the cross-validation score the MuZero
agent receives a reward $r_{t}$ at each time step.

This study focuses on supervised learning where a model is trained on a data
set $\mathcal{D}=(X,y)$, with $X=\\{\mathbf{x}_{1},\dots,\mathbf{x}_{N}\\}$
comprising $N$ data points
$\mathbf{x}_{i}\in\mathcal{X}\subset\mathcal{R}^{d}$ with corresponding labels
$y_{i}\in\mathcal{Y}$. We address regression problems with
$\mathcal{Y}\subset{\mathbb{R}}$, and binary classification tasks with
$\mathcal{Y}=\\{0,1\\}$. Quantum machine learning models such as QNNs or
quantum kernel methods rely on embedding data $\mathbf{x}\in\mathcal{X}$ into
quantum states

$\rho(\mathbf{x},\boldsymbol{\theta})=U(\mathbf{x},\boldsymbol{\theta})\rho_{0}U(\mathbf{x},\boldsymbol{\theta})^{\dagger}\,,$
(1)

with $\rho_{0}=\outerproduct{0}{0}$. From this definition one can derive QML
models with respect to some observable $O$ as

$f(\mathbf{x},\boldsymbol{\theta})=\tr\left(\rho(\mathbf{x},\boldsymbol{\theta})O\right)\,.$
(2)

The operator $U(\mathbf{x},\boldsymbol{\theta})$ is typically realized by a
quantum circuit, and can be expressed as a product of unitary operators

$U(\mathbf{x},\boldsymbol{\theta})=\prod_{l=1}V_{l}(\mathbf{x},\boldsymbol{\theta})\,,$
(3)

where
$V_{l}(\mathbf{x},\boldsymbol{\theta})=\exp(-i\phi(\mathbf{x},\boldsymbol{\theta})H_{l})$.
Here $H_{l}$ is a traceless Hermitian operator and $\phi$ is a (potentially)
non-linear function of the parameters and the data. Note that this include the
simple cases $\phi=x^{j}$ and $\phi=\theta^{j}$, where $x^{j}$ ($\theta^{j}$)
is the $j$-th component of a datapoint $\mathbf{x}$ (parameter vector
$\boldsymbol{\theta}$).

A model of the form of Eq. (2) can be trained directly by minimizing an
appropriate loss function or used as a variant of quantum pre-processing, as
for example in projected quantum kernels [37, 11]. From Eqs. (2)–(3) it
becomes clear that the choice of PQCs plays a defining role in the model’s
general architecture. To account for the current hardware restrictions in the
NISQ regime, the set of operators $\mathcal{V}$ from which the operators
$V_{l}$ in Eq. (3) are chosen is often restricted to the native gate set of a
specific hardware, resulting in hardware-efficient circuits which require a
minimal amount of transpilation.

The choice of circuit architecture profoundly impacts the properties of the
resulting model. For example, the expressivity of QML models is tightly
connected to the circuit structure [38]. Also, certain encoding circuits in
quantum kernel methods can cause the quantum kernel to resemble a classical
kernel, losing potential quantum advantage [39]. Therefore, the choice of
operators and their placement in the encoding circuit is crucially important
to the overall performance of the QML model. Despite its importance, this
aspect is often neglected. A notable exception where informed architecture
design is used is geometric QML. Here symmetries in the data are reflected in
the design of the circuit, resulting in models with desirable properties such
as good generalization and trainability [23, 24, 25, 26, 40]. However,
tailoring the encoding requires deep knowledge about the underlying properties
of the dataset, which is often intractable. For general data sets and ML
problems, the choice of circuit architecture is therefore an open question
which is usually not addressed beyond heuristics. It is desirable to create
encoding circuits that are not solely reliant on hardware efficiency but also
consider the model’s performance on the specific problem.

In this work, we employ the RL algorithm MuZero to build encoding circuits.
The algorithm creates encoding circuits iteratively by choosing and placing
unitary operators such that the performance of a resulting QML model is
optimized. The individual components of the framework will be discussed in the
following, an overview of the algorithm can be found in Fig. 1.

###  2.1 Model-based reinforcement learning algorithm

Generally, a RL agent interacts with its environment by taking actions $a_{t}$
at each time step $t$ and receiving a reward $r_{t}$ in return. The agent’s
objective is to optimize its policy
$\Pi_{t}(a_{t+1}|o_{1},...,o_{t},a_{1},...,a_{t})$, which describes the
probability of taking an action $a_{t+1}$ given previous observations
$o_{1},...,o_{t}$ and actions $a_{1},...,a_{t}$, in order to maximize
cumulative rewards. RL encompasses a variety of approaches [41] which can be
categorized into model-free approaches like proximal policy optimization [42]
and model-based techniques such as model predictive control [43] or the MuZero
algorithm [31].

The general workflow of the MuZero algorithm transforms the observation, in
our case the current encoding circuit, into a hidden state representation,
which comprises a dynamics function, a prediction function, and a
representation function—all based on deep neural networks. At each time step
$t$, the model processes this hidden state for $k=1,\dots,K$ steps and
computes an internal reward, a value function, and a policy based on the
network’s predictions in an MCTS fashion. At the end of each MCTS step, an
action is sampled from the policy
$p^{k}_{t}=\Pi_{t}(a_{t+k+1}|o_{1},...,o_{t},a_{t}+1,...,a_{t}+k)$, and the
actual reward and value function are computed. The weights of the three
networks are optimized during the training process to minimize the difference
between the internal hidden state predictions and real outcomes. For more
detail on the algorithm, we refer to [31].

The transformation of the observation space into a hidden state does not
impose strong restrictions on capturing the dynamics of the environment,
essentially interpreting the abstract process as a hidden layer of a deep
neural network. This characteristic makes the algorithm suitable for QML,
where full information about the complex dynamics of the interaction between
PQCs and data problems is not fully known. Another critical aspect is its
scalability in computational resources. The algorithm approximates all
necessary functions using classical neural networks, thereby minimizing calls
to the QML environment, typically the primary computational bottleneck in the
NISQ era. In the next section, we provide a detailed explanation on how MuZero
can be utilized to generate encoding circuits that achieve good performance.

###  2.2 Encoding circuit generation

The overall workflow of the circuit generation process based on MuZero,
denoted in the following as MuZero circuit search, is shown in Fig. 1. The RL
agent builds quantum circuits in a layered structure, where each gate operator
is applied to every accessible qubit in a layered fashion. Two-qubit gates are
considered in a linear nearest neighbour entangling configuration between the
qubits. An exemplary layout of the layered structure is shown in Fig. 2.
Although a more flexible structure is generally possible, it comes with the
caveat of a rapidly growing search space, in which good solutions might be
rare to find. In addition, we have found the layered structure to perform very
well in our benchmark calculations compared to a flexible structure as is
discussed in more detail in Section 3.2.

The action set of the RL framework, i.e., the set of gates available to the
agent, is listed in Table 1. While additional one and multi-qubit gates could
be considered, we limit the selection to those primarily used in PQCs for QML
to reduce the search space. Rotational gates are employed to encode input
features or manipulate the quantum state. Besides linear data encoding, we
also included gates with nonlinear transformations of the data using
$\tan^{-1}(x)$. In our approach, we avoid trainable weights and instead use
fixed angles. This allows the MuZero circuit search to optimize the angles
implicitly by selecting gates and angles that lead to an optimal reward. The
fixed angles are given by $\frac{\pi}{n}$ with $n\in\\{1,2,3,4,8\\}$. The
total number of qubits must be predefined and, in our implementation, is
always the number of features of the ML problem, ensuring that each feature is
represented once in every layer. While it is conceptually reasonable to allow
the agent to choose the number of qubits, we do not consider this to reduce
the complexity of the actions.

The environment of the RL agent is initialized by an empty circuit with all
qubits initialized in the zero state. The agent then iteratively adds gates
until a specified number $n_{\text{maxdepth}}$ of layers has been reached
which triggers a resets of the environment. Furthermore, we found it
beneficial to incorporate random initial architectures to improve the agent’s
exploration capabilities. These random restarts are executed with a
probability of $20\%$, and they are achieved by randomly picking
$n_{\text{maxdepth}}/2$ actions and ensuring that at least one feature
encoding layer is included.

The reward for the RL agent is determined based on the cross-validation score
of the QML model utilizing the generated circuit. This approach aims to avoid
creating circuits that could lead to overfitting. The specific metric for the
cross-validation value may vary depending on the type of problem. The agent is
granted the highest reward for surpassing the overall highest found score.
Similarly, a substantial reward is provided for reproducing the best-known
result. A lower reward is given for improving the result within the limits of
the current environment, with the reference value being reset in each new
environment. A minor penalty is applied if the score does not improve,
encouraging the development of shorter circuits. A major penalty is imposed if
the encoding circuit does not contain any gates dependent on $\mathbf{x}$. In
such cases, the cross-validation evaluation is also skipped.

In principle, the presented MuZero circuit search algorithm is independent of
the QML model. For practical reasons, in the reaming work we focus exclusively
on QML models based on projected quantum kernels (PQKs), as they provide a
robust QML framework that is quick to evaluate. PQKs use quantum states as an
intermediate step and project back to classical representations using
observables before evaluating the kernel. This process can be viewed as
preprocessing the input data by the quantum model before applying a classical
kernel method. We employ a Gaussian outer kernel with 1-reduced-density
matrices as proposed in Ref. [11]:

$K^{\rm
PQK}(\mathbf{x},\mathbf{x}^{\prime})=\exp\Big(-\gamma\sum_{O,k}\big{[}f_{O,k}(\mathbf{x})-f_{O,k}(\mathbf{x}^{\prime})\big{]}^{2}\Big{missing})\,.$
(4)

Here, $\gamma>0$ is a hyperparameter of the kernel, and
$f_{O,k}=\tr(\rho(\mathbf{x})O_{k})$, where the observable $O_{k}$ is taken
from the set of single-qubit Pauli matrices $\\{X,Y,Z\\}$ and applied to qubit
$k$. We primarily chose PQKs for two reasons: they are efficient to evaluate
and have shown good results for various ML tasks. Unlike fidelity quantum
kernels that are based on the overlap of the two quantum states [10], the
evaluation of quantum circuits with PQKs scales linearly with the data.
Additionally, caching of intermediate results enables a fast evaluation of the
cross-validation score. In contrast to PQKs, QNNs require an expensive
training routine that involves numerous evaluations of quantum circuits for
each training data point, making the search for optimal architectures
prohibitive. The implementation of the circuits, the PQK, and the QML models
is achieved using the QML library _sQUlearn_ [44].

Figure 2: Example of the layered circuit structure with $q=4$ qubits that has been created after 5 actions of the RL agent. The operators are chosen out of the action set described in Tab. 1. Different configurations of layered approach are used for the results in this paper.  $X$ |  $CX$ |  $R_{x}(\pi x)$ |  $R_{x}(\frac{\pi}{n})$
---|---|---|---
$Y$ |  $CY$ |  $R_{y}(\pi x)$ |  $R_{y}(\frac{\pi}{n})$
$Z$ |  $CZ$ |  $R_{z}(\pi x)$ |  $R_{z}(\frac{\pi}{n})$
$H$ |  $CR_{x}(\frac{n\pi}{8})$ |  $CR_{y}(\frac{\pi}{n})$ |  $CR_{z}(\frac{\pi}{n})$
$R_{x}(\tan^{-1}(x))$ |  $R_{y}(\tan^{-1}(x))$ |  $R_{z}(\tan^{-1}(x))$ |
Table 1: Set of gates which can be chosen as actions by the RL agent. Gates
are added as layers to all qubits at the end of the circuit, two-qubit gates
are considered in a linear nearest neighbor entangling configuration.
Parameterized gates are available for $n\in\\{1,2,3,4,8\\}$

##  3 Results

To benchmark the capabilities of the MuZero circuit search, we compare the
performance of the generated circuits to reference encoding circuits used in
the literature [12, 13, 14, 15]. Additionally, we compare the results to those
obtained by generating encoding circuits with a genetic algorithm [28], as
well as circuits with randomly chosen gate sequences. Furthermore, we compare
these results to those from various classical machine learning models. Our
approach is evaluated across three distinct datasets, including two regression
tasks and one classification task.

###  3.1 Benchmark set-up

This section details the setup of our benchmark calculations, which is
consistent across all compared methods and circuits. First, the dataset is
split into two parts: a training set and a test set. The training set is used
to determine suitable encoding circuits and optimize the hyperparameters of
the underlying (Q)ML model. The test data is reserved solely for calculating
the final score, which is used for comparing the different approaches. The
architecture search algorithms, including the MuZero circuit search algorithm,
random searches, and the genetic algorithm, follow a three-step approach:

  1. 1.

Generation and selection: We generate various circuits and select the five
best circuits based on the cross-validation score on the training data.

  2. 2.

Hyperparameter optimization: We optimize the hyperparameters of the underlying
(Q)ML model using the cross-validation score of the training data.

  3. 3.

Final evaluation: We train the final model on the full training data and
evaluate the final score using the test data.

The cross-validation in steps (1) and (2) utilizes the $\mathrm{R}^{2}$ score
for regression tasks and accuracy for the classification example, based on
five fixed folds for every dataset. The hyperparameters in step (1) are
consistent across all architecture search algorithms. The hyperparameter
optimization in step (2) is performed using the Optuna library [45],
optimizing both the hyperparameters of the (Q)ML model and the hyperparameter
$\gamma$ of the PQK. The number of executed cross-validations is comparable
for all architecture search algorithms and is in the order of $10^{5}$
evaluations. For reference circuits from the literature, step (1) is replaced
by optimizing the variational parameters of the PQCs with target alignment
(cf. Sec. 3.1.4). For classical ML models, step (1) is not required. All
results in this work are obtained using ideal statevector simulations. In the
following, we discuss details specific to each method.

####  3.1.1 MuZero circuit search

The MuZero circuit search algorithm employs a maximum of
$n_{\text{maxdepth}}=10$ layers to maintain hardware efficiency. The final
encoding circuits are obtained from those generated during both the training
and inference phases of the MuZero search algorithm, in which the algorithm’s
weights remaining fixed during inference.

####  3.1.2 Random encoding circuits

Random circuits are generated using two distinct methodologies: the layered
approach of the MuZero circuit search and a fully flexible structure. We
utilize the same gate set as employed in the RL algorithm (cf. Tab. 1) and
construct random layered structures as illustrated in Fig. 2 and detailed in
Section 2.2, by randomly selecting between 2 and $n_{\text{maxdepth}}=10$
actions. Moreover, we generate circuits based on a flexible structure that do
not adhere to a layer-wise application of gates. In this fully random
approach, one- and two-qubit gates are placed at every possible position
within the circuit. The rotation angles of individual gates are randomly
sampled from a uniform distribution between $[0,2\pi]$. The maximum number of
placed gates mirrors the total number of gates in the layered approach.
Additionally, we ensure that all features in $\mathbf{x}$ are encoded at least
once in the circuits, a prerequisite for the MuZero circuit search algorithm
to learn. For both random generation methods, we sample 10,000 circuits and
select the top 5 circuits based on the cross-validation score of the training
data.

####  3.1.3 Genetic algorithms

We compare our approach with the genetic algorithm proposed in [28],
incorporating a slight modification in the selection of the quantum kernel.
While [28] employs fidelity-based quantum kernels, we integrate PQKs into the
algorithm. Furthermore, we adjust the training procedure to prioritize the
cross-validation score, departing from the original methodology that relies on
training and testing scores, in order to align with the RL and random search
methods. The genetic algorithm is not bound to the layered circuit structure.
It has a high level of flexibility when building circuits, e.g. it can encode
multiple features on one qubit wire, as well as having full freedom in placing
entangling gates, as described in detail in Ref. [28].

####  3.1.4 Reference circuits

Additionally, we benchmark our findings against popular circuits found in the
literature [12, 13, 14, 15]. We use four different circuits including a data-
reuploading scheme which has been successful in previous QML studies [12, 46].
With the exception of one circuit, all others circuits feature variational
parameters $\theta$ that we optimize by maximizing the kernel-target-alignment
(TA) metric [12, 47] over the training set:

$\mathrm{TA}(K)=\frac{\sum_{i,j}K(\mathbf{x}_{i},\mathbf{x}_{j})y_{i}y_{j}}{\sqrt{\sum_{i,j}K(\mathbf{x}_{i},\mathbf{x}_{j})^{2}\sum_{i,j}y_{i}^{2}y_{j}^{2}}}\,,$

where $K$ represents the data-dependent Kernel matrix (cf. Eq. (4)). The
chosen reference circuits provide a overview of prevalent strategies found in
the literature, encompassing design principles such as hardware-efficient
entanglement and the placement of trainable variational parameters within the
circuits.

####  3.1.5 Classical models

We employ various classical ML models, including SVMs, Artificial Neural
Networks, Random Forests, Gradient Boosting, and Kernel Ridge Regression for
regression tasks. Specifically, Gradient Boosting is implemented utilizing the
XGBoost library [48], while neural networks are constructed using the
TensorFlow library [49]. The remaining models leverage the scikit-learn
library [50]. Hyperparameter optimization for all classical models is
conducted via the Optuna library [45] on the training dataset, employing
cross-validation techniques.

####  3.1.6 Data sets

We evaluate all models on three data sets, two regression problems and one
classification problem. The number of qubits in the QML algorithms is equal to
the number of features.

The _California housing_ data set from scikit-learn [50] is a regression
problem that aims to predict the prices of houses in California. The data is
scaled in a range of $[0,1]$, and we choose a fixed sub set of
$n_{\text{data}}=1,000$ randomly chosen data points. These data points are
partitioned into $n_{\text{training}}=700$ training points and
$n_{\text{test}}=300$ test points. The dataset has $8$ features.

The _Quantum fashion MNIST_ (QFMNIST) data set is based on the fashion-MNIST
data set [51], and prepared in a quantum state as described in [11]. This
essentially creates a regression problem for an artificial quantum data set.
The data is re-scaled to zero mean and unit variance, and the original
features are reduced using scikit-learn’s PCA implementation [50] using the
first $4$ principal components as features. We use $n_{\text{training}}=100$
training points, and $n_{\text{test}}=100$ test points, that are scaled in a
range from $[-1,1]$.

The _Two-curves (diff)_ data set [46] is a classification task which models
the curvature and the minimum distance of two curves in a $d$-dimensional
space. Here, we follow the data creation procedure given in [46]. The
problems’ difficulty can vary depending on the choice of degree and the number
of features. We use the most difficult version of the data set described in
[46] using $10$ features and a degree of $20$. The data is re-scaled to zero
mean and unit variance. We use $n_{\text{data}}=300$ samples and split them
into $n_{\text{training}}=240$ training points for cross-validation, and
$n_{\text{test}}=60$ test points.

###  3.2 Experiments

Figure 3: Performance of the regression (Q)ML models; (a) California housing
test data, and (b) QFMNIST test data. The first category marks the results of
different classical ML models, as described in Sec. 3.1.5. The second category
shows the results of a QSVR based on a variety of reference encoding circuits
(Sec. 3.1.4). The third shows the results of the QSVR based on encoding
circuits built by the MuZero circuit search (Sec. 2.2). The fourth and fifth
columns mark the results of a QSVR based on random encoding circuits (Sec.
3.1.2). The sixth category shows the results of an encoding circuit built by a
genetic algorithm (Sec. 3.1.3).

Fig. 3 displays the $\mathrm{R}^{2}$ scores of the two regression datasets.
For the California housing data (Fig. 3(a)), the best overall performance is
achieved by the QSVR approach utilizing the encoding circuits generated by the
MuZero circuit search algorithm. This approach outperforms the encoding
circuits generated by random search and the genetic algorithm. The best QML
models utilizing the reference circuits perform about as well as the best
classical model for this task. Classical ML models, shown in the first column
and labeled as CML in Fig. 3(a), have the largest performance range. The best
CML model is the SVR, with an $\mathrm{R}^{2}_{\text{SVR}}=0.752$ test score.

We observe a similar behavior for the QFMNIST regression problem. As can be
seen in Fig. 3(b), the best performance is obtained with the circuits found
with the MuZero circuit search algorithm, followed by circuits constructed
with the genetic algorithm. The use of reference circuits yield good
solutions, however, some of the chosen reference circuits lead to
significantly underperforming models. In contrast to Fig. 3(a), in which the
best reference has no trainable parameters [13], the same circuit in Fig. 3(b)
shows the worst results among all encoding circuits considered. For the
QFMNIST data, QML models perform better than the best classical ML model. This
is because this data set is a synthetic data set specifically designed to be
difficult to simulate classically. [11]

Figure 4: Performance of the classification (Q)ML models on the two-curves
(diff) test dataset. The columns follow the description in Fig. 3, with
regression models replaced by classification models.

The third benchmark is a classification task on the two-curves (diff) dataset
from [46], with the settings described in Sec. 3.1.6. Figure 4 showcases the
models’ performance measured by the accuracy score on the test data. For this
problem, the QSVC model, which is based on encoding circuits generated with
the MuZero circuit search algorithm, and the classical SVM perform equally
well with a test accuracy of $0.933$. The reference circuits perform similarly
to random circuits, with the best architecture for this problem being the
optimized YZ-CX approach from Ref. [14]. The results of the genetic algorithm
are below the two random search approaches.

Another important insight arises from the comparison with the random search
routines and the genetic algorithm. The results from the QSVMs utilizing
random encoding circuits are consistently positive across all cases and, on
average, slightly outperform the reference encoding circuits. Encoding
circuits build by genetic algorithms lag behind in Fig. 4). We found the
layered structure to be beneficial over the flexible structure, not only when
comparing the genetic algorithm to the MuZero circuit search, but also when
comparing the two random search approaches with and without the layered
structure.

We observed that in all cases there is a performance threshold that can be
reached to a certain degree. We noted for both classical and QML models that a
high training score does not consistently predict a high test score.
Consequently, it is essential to design multiple circuits and evaluate them,
as minor variations in cross-validated training scores can result in
substantial differences in test scores. Conversely, poor cross-validation
scores typically indicate significant generalization errors.

Due to the limited size of the data sets, SVR performed better than the other
classical ML methods in two scenarios. This is advantageous for the comparison
of QML and classical ML, since the QML models investigated in this study are
essentially kernel methods. However, it is difficult to make predictions about
the scaling behavior for larger future problems. Especially in the area of big
data, neural network approaches are starting to outperform their kernel-based
counterparts.

These results underscore the main finding of this work: contrary to common
practice in QML research, where reference circuit architectures are used and
their parameters optimized, better results can be achieved with custom
circuits. Notably, circuits created by a random search can yield already good
outcomes, but the best possible performance was consistently achieved using
the MuZero circuit search algorithm. This approach excels in learning from the
problem structure, enabling the creation of tailored circuits that outperform
other methods. Therefore, the MuZero circuit search algorithm is a reasonable
choice for designing problem-specific quantum circuits, as it consistently
produces the most effective solutions.

##  4 Discussion

In this study, we demonstrated the automatic generation of PQCs for QML using
the MuZero circuit search algorithm. Our experiments illustrated the
effectiveness of this RL framework in creating tailored circuits that
considerably improve the performance of the QML model. The results
consistently showed that these circuits outperformed the reference
architectures from the literature. Interestingly, random search strategies
also produce good results, often matching or even surpassing the performance
of reference circuits from the literature. This is particularly notable given
that reference circuits use optimized parameters, while the random search
chooses these parameters randomly.

Although the presented QML models slightly outperform or match their classical
counterparts, we do not claim an overall quantum advantage of the presented
methodology, since all discussed datasets are relatively small compared to
those that can be handled classically. The inherent complexity and variability
of QML models make it difficult to establish consistent claims of performance
advantages over classical ML models, especially when moving to larger
datasets.

Looking forward, there are several opportunities to extend the workflow
presented in this study. The MuZero circuit search algorithm can be adapted to
generate circuits for different QML models, such as QNNs, and could serve as a
general circuit optimization tool for other circuit approaches outside of QML.
Furthermore, incorporating operators that reflect the symmetries of the data
set could introduce an informed inductive bias, leading to more practical
circuit designs. Moreover, the MuZero circuit search is promising for
transferring the learned RL model from one dataset to another, since the
hidden state representation of the circuit has already been learned. Further
research could evaluate the ability to generalize to new datasets without
requiring intensive retraining of the RL algorithm.

In conclusion, our study highlights the significant potential of using
reinforcement learning, in particular the MuZero circuit search algorithm, to
generate problem-specific PQCs for QML. The results show that tailored
circuits can significantly improve the performance of QML models and provide a
solid foundation for future research and development in the field of QML.

Acknowledgments We thank Jan Schnabel, Tobias Nagel and Philipp Wagner for
insightful discussions. This work was supported by the Ministry of Economic
Affairs, Labour, and Tourism Baden-Württemberg in the frame of the Competence
Center Quantum Computing Baden-Württemberg (project SEQUOIA End-to-End).

## References

  *   * [1] Preskill, J.  Quantum computing in the NISQ era and beyond.  _Quantum_ 2, 79 (2018).  URL https://doi.org/10.22331%2Fq-2018-08-06-79.
  * [2] Cerezo, M., Verdon, G., Huang, H.-Y., Cincio, L. & Coles, P.  Challenges and opportunities in quantum machine learning.  _Nature Computational Science_ 2 (2022).
  * [3] Schuld, M. & Killoran, N.  Quantum machine learning in feature hilbert spaces.  _Physical Review Letters_ 122 (2019).  URL https://doi.org/10.1103%2Fphysrevlett.122.040504.
  * [4] Lloyd, S., Schuld, M., Ijaz, A., Izaac, J. & Killoran, N.  Quantum embeddings for machine learning (2020).  URL https://arxiv.org/abs/2001.03622.
  * [5] Benedetti, M., Lloyd, E., Sack, S. & Fiorentini, M.  Parameterized quantum circuits as machine learning models.  _Quantum Science and Technology_ 4, 043001 (2019).  URL https://doi.org/10.1088%2F2058-9565%2Fab4eb5.
  * [6] Cerezo, M. _et al._ Variational quantum algorithms.  _Nature Reviews Physics_ 3, 625–644 (2021).  URL http://arxiv.org/pdf/2012.09265v2.
  * [7] Ragone, M. _et al._ A unified theory of barren plateaus for deep parametrized quantum circuits (2023).  2309.09342.
  * [8] Larocca, M. _et al._ A review of barren plateaus in variational quantum computing (2024).  2405.00781.
  * [9] Mitarai, K., Negoro, M., Kitagawa, M. & Fujii, K.  Quantum circuit learning.  _Phys. Rev. A_ 98, 032309 (2018).  URL https://link.aps.org/doi/10.1103/PhysRevA.98.032309.
  * [10] Havlíček, V. _et al._ Supervised learning with quantum-enhanced feature spaces.  _Nature_ 567, 209–212 (2019).
  * [11] Huang, H.-Y. _et al._ Power of data in quantum machine learning.  _Nature Communications_ 12 (2021).  URL https://doi.org/10.1038%2Fs41467-021-22539-9.
  * [12] Hubregtsen, T. _et al._ Training quantum embedding kernels on near-term quantum computers (2021).  URL https://arxiv.org/abs/2105.02276.
  * [13] Peters, E. _et al._ Machine learning of high dimensional data on a noisy quantum processor (2021).
  * [14] Haug, T., Self, C. N. & Kim, M. S.  Quantum machine learning of large datasets using randomized measurements.  _Machine Learning: Science and Technology_ 4, 015005 (2023).  URL https://dx.doi.org/10.1088/2632-2153/acb0b4.
  * [15] Havlíček, V. _et al._ Supervised learning with quantum-enhanced feature spaces.  _Nature_ 567, 209–212 (2019).  URL https://doi.org/10.1038%2Fs41586-019-0980-2.
  * [16] Farhi, E., Goldstone, J. & Gutmann, S.  A quantum approximate optimization algorithm (2014).  1411.4028.
  * [17] Tilly, J. _et al._ The variational quantum eigensolver: A review of methods and best practices.  _Physics Reports_ 986, 1–128 (2022).  URL https://www.sciencedirect.com/science/article/pii/S0370157322003118.  The Variational Quantum Eigensolver: a review of methods and best practices.
  * [18] Peruzzo, A. _et al._ A variational eigenvalue solver on a photonic quantum processor.  _Nature Communications_ 5, 4213 (2014).  URL https://doi.org/10.1038/ncomms5213.
  * [19] Skolik, A., McClean, J. R., Mohseni, M., van der Smagt, P. & Leib, M.  Layerwise learning for quantum neural networks.  _Quantum Mach. Intell._ 3, 5 (2021).
  * [20] Wolpert, D. & Macready, W.  No free lunch theorems for optimization.  _IEEE Transactions on Evolutionary Computation_ 1, 67–82 (1997).
  * [21] Kübler, J. M., Buchholz, S. & Schölkopf, B.  The inductive bias of quantum kernels (2021).  2106.03747.
  * [22] Bowles, J., Wright, V. J., Farkas, M., Killoran, N. & Schuld, M.  Contextuality and inductive bias in quantum machine learning (2023).  2302.01365.
  * [23] Ragone, M. _et al._ Representation theory for geometric quantum machine learning (2023).  2210.07980.
  * [24] Larocca, M. _et al._ Group-invariant quantum machine learning.  _PRX Quantum_ 3, 030341 (2022).  URL https://link.aps.org/doi/10.1103/PRXQuantum.3.030341.
  * [25] Meyer, J. J. _et al._ Exploiting symmetry in variational quantum machine learning.  _PRX Quantum_ 4, 010328 (2023).  URL https://link.aps.org/doi/10.1103/PRXQuantum.4.010328.
  * [26] Schatzki, L., Larocca, M., Nguyen, Q. T., Sauvage, F. & Cerezo, M.  Theoretical guarantees for permutation-equivariant quantum neural networks.  _npj Quantum Inf._ 10, 12 (2024).
  * [27] Gili, K., Alonso, G. & Schuld, M.  An inductive bias from quantum mechanics: learning order effects with non-commuting measurements (2023).  2312.03862.
  * [28] Altares-López, S., Ribeiro, A. & García-Ripoll, J. J.  Automatic design of quantum feature maps.  _Quantum Science and Technology_ 6, 045015 (2021).  URL https://dx.doi.org/10.1088/2058-9565/ac1ab1.
  * [29] Incudini, M. _et al._ Automatic and effective discovery of quantum kernels (2023).  2209.11144.
  * [30] Sünkel, L., Martyniuk, D., Mattern, D., Jung, J. & Paschke, A.  Ga4qco: Genetic algorithm for quantum circuit optimization (2023).  2302.01303.
  * [31] Schrittwieser, J. _et al._ Mastering atari, go, chess and shogi by planning with a learned model.  _Nature_ 588, 604–609 (2020).  URL http://dx.doi.org/10.1038/s41586-020-03051-4.
  * [32] Świechowski, M., Godlewski, K., Sawicki, B. & Mańdziuk, J.  Monte carlo tree search: a review of recent modifications and applications.  _Artificial Intelligence Review_ 56, 2497–2562 (2023).  URL https://doi.org/10.1007/s10462-022-10228-y.
  * [33] Pozzi, M. G., Herbert, S. J., Sengupta, A. & Mullins, R. D.  Using reinforcement learning to perform qubit routing in quantum compilers.  _ACM Transactions on Quantum Computing_ 3 (2022).  URL https://doi.org/10.1145/3520434.
  * [34] Ruiz, F. J. R. _et al._ Quantum circuit optimization with alphatensor (2024).  2402.14396.
  * [35] Ostaszewski, M., Trenkwalder, L. M., Masarczyk, W., Scerri, E. & Dunjko, V.  Ranzato, M., Beygelzimer, A., Dauphin, Y., Liang, P. & Vaughan, J. W. (eds) _Reinforcement learning for optimization of variational quantum circuit architectures_.  (eds Ranzato, M., Beygelzimer, A., Dauphin, Y., Liang, P. & Vaughan, J. W.) _Advances in Neural Information Processing Systems_ , Vol. 34, 18182–18194 (Curran Associates, Inc., 2021).  URL https://proceedings.neurips.cc/paper_files/paper/2021/file/9724412729185d53a2e3e7f889d9f057-Paper.pdf.
  * [36] Altmann, P. _et al._ Challenges for reinforcement learning in quantum circuit design (2024).  2312.11337.
  * [37] Gil-Fuster, E., Eisert, J. & Dunjko, V.  On the expressivity of embedding quantum kernels.  _Machine Learning: Science and Technology_ 5, 025003 (2024).  URL https://dx.doi.org/10.1088/2632-2153/ad2f51.
  * [38] Schuld, M., Sweke, R. & Meyer, J. J.  Effect of data encoding on the expressive power of variational quantum-machine-learning models.  _Physical Review A_ 103 (2021).  URL https://doi.org/10.1103%2Fphysreva.103.032430.
  * [39] Schuld, M.  Supervised quantum machine learning models are kernel methods (2021).  URL https://arxiv.org/abs/2101.11020.
  * [40] Nguyen, Q. T. _et al._ Theory for equivariant quantum neural networks.  _PRX Quantum_ 5, 020328 (2024).  URL https://link.aps.org/doi/10.1103/PRXQuantum.5.020328.
  * [41] Sutton, R. S. & Barto, A. G.  _Reinforcement learning: An introduction_ (MIT press, 2018).
  * [42] Schulman, J., Wolski, F., Dhariwal, P., Radford, A. & Klimov, O.  Proximal policy optimization algorithms (2017).  1707.06347.
  * [43] García, C. E., Prett, D. M. & Morari, M.  Model predictive control: Theory and practice—a survey.  _Automatica_ 25, 335–348 (1989).  URL https://www.sciencedirect.com/science/article/pii/0005109889900022.
  * [44] Kreplin, D. A., Willmann, M., Schnabel, J., Rapp, F. & Roth, M.  squlearn — a python library for quantum machine learning (2023).  2311.08990.
  * [45] Akiba, T., Koyama, M. _et al._ Optuna: A next-generation hyperparameter optimization framework.  _Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining_ (2019).
  * [46] Bowles, J., Ahmed, S. & Schuld, M.  Better than classical? the subtle art of benchmarking quantum machine learning models (2024).  2403.07059.
  * [47] Glick, J. R. _et al._ Covariant quantum kernels for data with group structure.  _Nature Phys._ 20, 479–483 (2024).
  * [48] Chen, T. & Guestrin, C.  Xgboost: A scalable tree boosting system.  _Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining_ (2016).  URL http://dx.doi.org/10.1145/2939672.2939785.
  * [49] Abadi, M., Zheng, X. _et al._ TensorFlow: Large-scale machine learning on heterogeneous systems (2015).  URL https://www.tensorflow.org/.  Software available from tensorflow.org.
  * [50] Pedregosa, F., Duchesnay, E. _et al._ Scikit-learn: Machine learning in Python.  _Journal of Machine Learning Research_ 12, 2825–2830 (2011).
  * [51] Xiao, H., Rasul, K. & Vollgraf, R.  Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms (2017).  1708.07747.


# User interfaces for computational science: a domain specific language for
OOMMF embedded in Python

Marijan Beg<EMAIL_ADDRESS>Ryan A. Pepper  Hans Fangohr
<EMAIL_ADDRESS>1Faculty of Engineering and the Environment, University
of Southampton, Southampton, SO17 1BJ, United Kingdom

###### Abstract

Computer simulations are used widely across the engineering and science
disciplines, including in the research and development of magnetic devices
using computational micromagnetics. In this work, we identify and review
different approaches to configuring simulation runs: (i) the re-compilation of
source code, (ii) the use of configuration files, (iii) the graphical user
interface, and (iv) embedding the simulation specification in an existing
programming language to express the computational problem. We identify the
advantages and disadvantages of different approaches and discuss their
implications on effectiveness and reproducibility of computational studies and
results. Following on from this, we design and describe a domain specific
language for micromagnetics that is embedded in the Python language, and
allows users to define the micromagnetic simulations they want to carry out in
a flexible way. We have implemented this micromagnetic simulation description
language together with a computational backend that executes the simulation
task using the Object Oriented MicroMagnetic Framework (OOMMF). We illustrate
the use of this Python interface for OOMMF by solving the micromagnetic
standard problem 4. All the code is publicly available and is open source.

##  I Introduction

Computational Science is emerging as the third pillar of research and
development in academia and in industry across all science and engineering
disciplines. Computational studies complement experimental and theoretical
studies, and are at times the only feasible way to address research
challenges, effective industrial design and engineering of various products
and systems.

In the field of magnetism, micromagnetic simulations have become well
established and are often the only possible technique for the exploration of
different magnetic phenomena. Najafi  _et al._ (2009); Venkat  _et al._
(2013); Baker  _et al._ (2017) Their use becomes more widespread and reliable
as the micromagnetic models, simulation techniques, and the processing power
of computers advance.

Computational science brings its own challenges: results based on computer
simulation should be reproducible, Baker (2016); Peng (2011) ideally by the
whole research community but at the very least by the authors of the
publication. This requires, amongst other things, tracking of all input
parameters for a simulation, and all post-processing steps, often for very
many simulation runs. Freire  _et al._ (2008)

In this work, we provide an overview of different approaches to configuring
simulations in Sec. II: code re-compilation, the use of configuration scripts,
graphical user interfaces, as well as the use of domain specific languages to
specify the computational problem. We discuss advantages and disadvantages
with particular focus on the reproducibility associated with these approaches.
In Sec. III, we use the most flexible approach identified and describe the
design of a Python van Rossum and Drake  interface for the widely used Object
Oriented MicroMagnetic Framework (OOMMF) simulation tool. Donahue and Porter
(1999) Finally, we illustrate the use of this interface by solving the
micromagnetic standard problem McMichael  _et al._ 4 in Sec. IV, before
closing with a summary.

##  II Simulation User Interfaces

Micromagnetic simulations, as with simulations in many other fields, need to
simulate the behaviour of particular materials with their specific material
constants, under particular circumstances (for example with an applied field)
and specific geometries. We refer to this parameter set as the _simulation
input parameters_. Despite changing input parameters, the computational
framework (here the micromagnetic model) stays roughly the same for all
possible parameter sets (although sometimes extra terms need to be added).

The challenges for the researcher using simulations include: (i) communicating
the input parameters to the simulation, and (ii) keeping track of the input
parameters that were used for particular simulation results (for publication,
and to be able to repeat the exact simulation in the future). We describe 4
approaches that have emerged over time.

### Approach 1: Code re-compilation.

The first approach for providing input parameters to a computer simulation is
to hard-code the simulation parameters in the source code of the simulation,
and re-compile the simulation tool source code for each set of input
parameters. While easy to implement, it makes it difficult to store the input
parameters in an efficient way (unless the full copy of the source code is
archived for every simulation run).

### Approach 2: Configuration files.

The second approach is the writing of the input parameters to a configuration
file, which is then read by the simulation tool at run time. There are several
advantages of this over the re-compilation approach. First, multiple
simulations can be run using the same simulation executable by providing
multiple configuration files, each describing material properties and geometry
for one simulation. Secondly, if simulation configuration files as well as the
simulation software version are kept, all simulations can be repeated at a
later stage. This is important for reproducibility of the results; a topic
receiving increasing (and well deserved) attention in computational science.
Peng (2011); Baker (2016) A disadvantage of the configuration-file based
simulation configuration approach is that the developers may have to define a
syntax for the configuration files and implement a parser for it. Depending on
the complexity of the syntax, the number of different operations the
simulation tool can be configured to perform is limited. Experience shows that
this syntax (and parser) is often extended as the simulation tool’s
capabilities grow and demand more flexibility. Magpar Scholz  _et al._ (2003)
is a micromagnetic simulation tool using this approach. OOMMF Donahue and
Porter (1999) also uses configuration files, but is using an existing language
and parser (Tcl).

### Approach 3: Graphical User Interface.

Simulation tools for which input parameters are set through graphical user
interfaces are often perceived as particularly user friendly. If the completed
configuration can be saved to a file and reloaded later, this is similar to
the configuration file based approaches. Where the configuration cannot be
saved, it is difficult to reliably record all configuration options as this
would have to be done manually, and tedious to re-execute a particular
simulation as all input parameters need to be entered again manually.

### Approach 4: Domain Specific Language (DSL) embedded in general purpose
language.

It is also possible to embed all input parameter definition and high-level
simulation commands in an executable file which fully defines the simulation,
using the syntax of an existing programming language. The simulation is
carried out by executing this file with the appropriate interpreter or
compiler. The one file contains all the information that needs to be preserved
for reproducibility. Examples of embedding micromagnetic simulation tools into
existing programming language include Nmag, Fischbacher  _et al._ (2007)
Micromagnum, Mic  Magnum.fe, Abert  _et al._ (2013) and Fidimag. Wang  _et
al._ (2016) An important advantages of this method over the configuration file
based approach is the increased flexibility: as the simulation script contains
a sequence of ’normal’ commands in a given programming language, these can be
used and combined as required to, for example, create complicated spatial
field distributions, fetch data from a file or a connected experimental kit,
and do parameter sweeps automatically _within the same file_. It is possible
to carry out postprocessing within the same simulation file; thus keeping
input data, simulation process and extraction of results closely together
(supporting tracking of the provenance Freire  _et al._ (2008)). If the chosen
programming language is an interpreted one (such as Python van Rossum and
Drake ), this embedded approach may also allow _interactive execution_ of
simulations (sometimes referred to as “computational steering”), interactive
analysis, and visualisation (for instance, in Jupyter notebooks Pérez and
Granger (2007)). Eventually, by using an existing language, both code
developers and users can benefit from using existing and well tested modules
for the selected programming language. A disadvantage of this approach is that
it requires more planning on the coding side to provide the described
framework that can be used flexibly.

### Discussion.

It is known from software engineering that there are significant advantages of
separating (configuration) metadata from (simulation) software code to obtain
programs that are more flexible and robust, easier to maintain and test, and
more versatile in their use.Hunt and Thomas (1999) In our context, this
separation of metadata from the simulation code is given in the configuration-
file approach and in the embedded language approach; the latter providing more
flexibility in driving the simulation and integrating other steps of the
computational work flow.

##  III Python interface for OOMMF

### Introduction.

In what follows, we describe a design for a Domain Specific Language (DSL) for
micromagnetic simulations that is embedded in the Python language. This allows
scientists from the domain of micromagnetics to express their simulation
requirements using this language, and the language is valid Python. We show an
example in Sec. IV. We have implemented a tool Beg, Pepper, and Fangohr  that
can understand this domain specific language and carry out the required
simulation using OOMMF. This allows to express the micromagnetic simulation
requirements in Python and to postprocess and analyse data with great
flexibility in Python, while the computation is done by OOMMF.

### Choice of Python to embed the micromagnetic specification language.

Here we explain the choice of Python as the implementation language for the
interface. Python van Rossum and Drake  has been gaining popularity in
computational science since soon after its inception in 1991\. Python has been
identified as a language that is easy to learn Fangohr (2004) by computational
scientists and engineers. The value of Python for computational science is its
flexibility and readability – both attributes that reduce the time required to
express algorithms (including post-processing and plotting instructions) and
debugging them. While Python code – if used natively and naively – can be
orders of magnitude slower than C or Fortran code, it is possible to develop
and drive High Performance Computing projects in Python.Gorelick and Ozsvald
(2014) Finally, Python has a rich variety of high quality and well tested
modules providing algorithms for performing common operations in computational
science and engineering, such as SciPy, Jones  _et al._ NumPy, van der Walt,
Colbert, and Varoquaux (2011) Pandas, McKinney (2011) matplotlib, Hunter
(2007) that we use in this work.

### The choice of OOMMF as the computational backend.

The Object Oriented MicroMagnetic Framework (OOMMF) Donahue and Porter (1999)
developed at the Information Technology Laboratory at the National Institute
of Standards and Technology by Michael J. Donahue and Donald G. Porter is
widely used. Its discretisation scheme is based on the finite difference
method. The computational core is written in C++, and combined with Tcl/Tk for
high level interfaces and Graphical User Interfaces; combining tools in a
clever way and using state-of-the art technology at the time of OOMMF’s
inception. OOMMF uses predefined simulation modes (for instance, hysteresis or
dynamics), and does not allow the user to carry out micromagnetic operations
in an arbitrary order. OOMMF’s configuration files use the Tcl syntax, and
thus allow a convenient way to compute spatially distributed fields from
equations within that configuration file. However, OOMMF is not embedded into
an existing programming language which means, for example, that no single
configuration file can carry out a parameter sweep, or host multiple
simulation objects, and any post-processing and visualisation must be carried
out separately. We believe that OOMMF’s computational capabilities are highly
valued by the community, and it is likely to be the most widely used
micromagnetic simulation code.

### Low level OOMMF interface implementation choice.

Here, we describe how the implementation communicates with OOMMF. This is
transparent to the scientists using the specification language and targeting a
more technical audience of simulation tool developers. The core computational
routines of OOMMF are implemented in C++ but important higher level
functionality is written in Tcl, including platform specific installation. We
have researched and considered a variety of technical solutions for
interfacing Tcl and C++ code with Python, including handcoding an interface,
using SWIG, Beazley (1996) Boost, Abrahams (2003) and Cython. Behnel  _et al._
(2011) Eventually, we have opted for communication with OOMMF via mif
configuration and output data files. The main advantage of this approach is
the robustness. For example, we do not need to adopt this interface code
depending on the platform (operating system and compiler) on which the OOMMF
code was compiled. We can deploy the Python interface code on a system where
there is no C++ compiler (but only the OOMMF executable). Furthermore, the
coupling between the tool and the OOMMF executable is relatively loose (the
OOMMF implementation internals can change without affecting the Python
interface) and transparent (developers and users can inspect mif files for
debugging purposes if required). A disadvantage of this approach is that there
are situations in which CPU cycles are needed that could have been avoided
through a more tightly coupled interface that connects more directly to the
OOMMF internals.

### Micromagnetic model description language.

Here, we summarise the design of the micromagnetic model language that is
exposed to the user at the Python level. There is not sufficient space to
detail our design and reasoning in this article. The full code and growing
documentation is available. Beg, Pepper, and Fangohr  In summary, we base the
micromagnetic model around a “system” which is defined by providing: (i) mesh,
(ii) Hamiltonian $\mathcal{H}$, (iii) optional dynamics equation
$\text{d}\mathbf{m}/\text{d}t$, and (iv) the current magnetisation
configuration $\mathbf{m}$. The mesh contains the geometry and discretisation
information, the Hamiltonian captures the relevant interactions contributing
to the energy, and for time dependent problems there is an equation of motion.
To change the magnetisation, we use “drivers” (following terminology
introduced by the OOMMF team Donahue and Porter (1999)). Drivers “drive” the
system in phase space by changing its magnetisation. This can be either energy
minimisation (e.g. conjugate gradient) or time evolution through integration
of the Landau-Lifshitz-Gilbert (LLG)Gilbert (2004) equation. We show the
schematic representation of our micromagnetic model in Fig. 1.

Figure 1:  A schematic representation of our micromagnetic model. The system
object is completely defined with: (i) mesh, (ii) Hamiltonian $\mathcal{H}$,
(iii) dynamics equation $\text{d}\mathbf{m}/\text{d}t$, and (iv) magnetisation
configuration $\mathbf{m}$. The driver is “driving” (moving) the system
through phase space by changing its magnetisation from $\mathbf{m}_{1}$ to
$\mathbf{m}_{2}$.

Based on this micromagnetic model, we have implemented the “OOMMF Calculator”
(oommfc) which can carry out the required micromagnetic computations. When
required, the OOMMF calculator writes a mif file, calls OOMMF to execute it,
extracts the required information from the output files, and makes the results
available within the Python environment.

##  IV Example

To illustrate the use of the Python interface for OOMMF, we solve Standard
problem McMichael  _et al._ 4 by computing the magnetisation evolution in the
thin film with $3\,\text{nm}$ thickness of length $L=500\,\text{nm}$ and width
$d=125\,\text{nm}$. The material is Permalloy with magnetisation saturation
$M_{\text{s}}$ and exchange energy constant $A$.

As the first step, we need to import the Python module oommfc (the name
standing for OOMMF Calculator) which provides the commands for defining the
micromagnetic system and running OOMMF simulations, as well as the
discretisedfield module that we use for defining the finite difference mesh
and fields.

⬇

import oommfc as oc

import discretisedfield as df

In our micromagnetic model, a system is defined completely by providing its
mesh, Hamiltonian, dynamics equation, and magnetisation configuration. We
specify the geometry through coordinates of two points between which the
cuboidal domain spans and the size of a discretisation cell
($d_{x},d_{y},d_{z}=2.5\,\text{nm},2.5\,\text{nm},3\,\text{nm}$).

⬇

L, d, th = 500e-9, 125e-9, 3e-9 # (m)

cellsize = (2.5e-9, 2.5e-9, 3e-9) # (m)

mesh = oc.Mesh((0, 0, 0), (L, d, th), cellsize)

The variable name in the system object labels the directory structure that
holds the OOMMF output files.

⬇

system = oc.System(name="stdprob4")

According to the standard problem 4 specification, the system’s Hamiltonian
contains ferromagnetic exchange and demagnetisation energy terms

$\mathcal{H}=\underbrace{A[(\nabla m_{x})^{2}+(\nabla m_{y})^{2}+(\nabla
m_{z})^{2}]}_{\texttt{Exchange(A)}}+\underbrace{w_{\text{d}}}_{\texttt{Demag()}},$
(1)

where $m_{x}$, $m_{y}$, and $m_{z}$ are Cartesian coordinates of unit
magnetisation vector $\mathbf{m}=\mathbf{M}/M_{\text{s}}$. We provide this
Hamiltonian to the system object.

⬇

A = 1.3e-11 # (J/m)

system.hamiltonian = oc.Exchange(A) + oc.Demag()

Next, we define the dynamics of the system, which is governed by the Landau-
Lifshitz and Gilbert Gilbert (2004) equation which consists of two (precession
and damping) terms

$\frac{\text{d}\mathbf{m}}{\text{d}t}=\underbrace{-\gamma_{0}^{*}(\mathbf{m}\times\mathbf{H}_{\text{eff}})}_{\texttt{Precession(gamma)}}+\underbrace{\alpha(\mathbf{m}\times\frac{\text{d}\mathbf{m}}{\text{d}t})}_{\texttt{Damping(alpha)}},$
(2)

where $\mathbf{H}_{\text{eff}}$ is the effective field computed from the
system’s Hamiltonian. We specify the equation of motion:

⬇

gamma = 2.211e5 # (m/As)

alpha = 0.02

system.dynamics = oc.Precession(gamma) + \

oc.Damping(alpha)

In order to complete the definition of our micromagnetic model, we specify the
initial magnetisation configuration $\mathbf{m}$, which is uniform in
direction $(1,0.25,0.1)$, normalised to the value of $M_{\text{s}}$.

⬇

Ms = 8e5 # (A/m)

system.m = df.Field(mesh, value=(1, 0.25, 0.1),

normalisedto=Ms)

In the first stage of standard problem 4, we need to relax the system, and we
use an energy minimisation driver. First we create the MinDriver object and
then pass the system object to the drive method.

⬇

md = oc.MinDriver()

md.drive(system) # updates system.m in-place

After having called the minimisation driver, the system object now contains
the (equilibrium) magnetisation configuration for which the system’s energy is
minimised.

So far, our micromagnetic system was at zero external magnetic field. In the
second stage we need to add the external magnetic field
$\mathbf{B}=(-24.6,4.3,0.0)\,\text{mT}$. Therefore, we add the Zeeman energy
density term $w_{\text{z}}=-\mu_{0}\mathbf{M}\cdot\mathbf{H}$ to the system’s
Hamiltonian, where $\mathbf{H}=\mathbf{B}/\mu_{0}$, with $\mu_{0}$ being the
magnetic constant. We carry on using the same micromagnetic system as in the
relaxation step, but modify it by adding the Zeeman term to its Hamiltonian.
Nothing else changes; in particular the magnetisation $\mathbf{m}$ is the same
as after the relaxation stage.

⬇

H = (-24.6e-3/oc.mu0, 4.3e-3/oc.mu0, 0)

system.hamiltonian += oc.Zeeman(H)

Now, we drive the system using a TimeDriver for $1\,\text{ns}$ and instruct
the system to remember its magnetisation evolution at 200 points during the
nanosecond.

⬇

td = oc.TimeDriver()

td.drive(system, t=1e-9, n=200)

Finally, we can plot and save the time evolution of the average $y$ component
of magnetisation as shown in Fig. 2.

⬇

myplot = system.dt.plot("t", "my")

myplot.figure.savefig("stdprob4-t-my.pdf")

The data analysis can be decoupled from running the simulation. In that case,
we parse the saved output files from OOMMF runs and make the data available.

For multi-material simulations, we intend to use spatially varying material
parameters.

Figure 2:  The average $y$ component of magnetisation time evolution for the
first $1\,\text{ns}$ computed with OOMMF simulation tool via our Python
interface.

##  V Summary

We summarise and discuss approaches to simulation user interface design with
computational micromagnetics as a case study. We argue that embedding both the
high level simulation commands and input parameters as a domain specific
language in an existing programming language is the approach with most
benefits. Consequently, we implement such an interface that allows to drive
OOMMF through the Python programming language, and make it available as open
source.Beg, Pepper, and Fangohr  We illustrated its use by solving the
micromagnetic standard problem 4. We hope this interface can improve the
micromagnetic simulation workflows, supporting more reproducible and effective
computational science.

The design of the Python interface for OOMMF is a prototype for a generic
(Python-based) specification language for micromagnetic simulation problems,
which in the future can be extended to use other computational backends in
addition to OOMMF.

###### Acknowledgements.

This work was financially supported by the OpenDreamKit – Horizon 2020
European Research Infrastructure project (676541), the EPSRC’s Centre for
Doctoral Training in Next Generation Computational Modelling grant
EP/L015382/1, and the EPSRC’s Programme grant on Skyrmionics (EP/N032128/1).
We thank Michael Donahue, Donald Porter, and Dmitri Chernyshenko for helpful
discussions.

## References

  * Najafi  _et al._ (2009) M. Najafi, B. Krüger, S. Bohlens, M. Franchin, H. Fangohr, A. Vanhaverbeke, R. Allenspach, M. Bolte, U. Merkt, D. Pfannkuche, D. P. F. Möller, and G. Meier, “Proposal for a standard problem for micromagnetic simulations including spin-transfer torque,” J. Appl. Phys. 105, 113914 (2009).
  * Venkat  _et al._ (2013) G. Venkat, D. Kumar, M. Franchin, O. Dmytriiev, M. Mruczkiewicz, H. Fangohr, A. Barman, M. Krawczyk, and A. Prabhakar, “Proposal for a Standard Micromagnetic Problem: Spin Wave Dispersion in a Magnonic Waveguide,” IEEE Trans. Magn. 49, 524–529 (2013).
  * Baker  _et al._ (2017) A. Baker, M. Beg, G. Ashton, M. Albert, D. Chernyshenko, W. Wang, S. Zhang, M.-A. Bisotti, M. Franchin, C. L. Hu, R. Stamps, T. Hesjedal, and H. Fangohr, “Proposal of a micromagnetic standard problem for ferromagnetic resonance simulations,” J. Magn. Magn. Mater. 421, 428–439 (2017).
  * Baker (2016) M. Baker, “1,500 scientists lift the lid on reproducibility,” Nature 533, 452–454 (2016).
  * Peng (2011) R. D. Peng, “Reproducible Research in Computational Science,” Science 334, 1226–1227 (2011).
  * Freire  _et al._ (2008) J. Freire, D. Koop, E. Santos, and C. Silva, “Provenance for Computational Tasks: A Survey,” Comput. Sci. Eng. 10, 11–21 (2008).
  * (7) G. van Rossum and F. L. Drake, “Python Reference Manual,” PythonLabs, Virginia, USA, 2001  URL https://www.python.org/ .
  * Donahue and Porter (1999) M. J. Donahue and D. G. Porter, “OOMMF User’s Guide, Version 1.0,” Interag. Rep. NISTIR 6376 Natl. Inst. Stand. Technol. Gaithersburg, MD (1999).
  * (9) B. McMichael, R. Koch, T. Schrefl, and J. Eicke, “µMAG Standard Problem #4”  URL http://www.ctcms.nist.gov/~rdm/mumag.org.html .
  * Scholz  _et al._ (2003) W. Scholz, J. Fidler, T. Schrefl, D. Suess, R. Dittrich, H. Forster, and V. Tsiantos, “Scalable parallel micromagnetic solvers for magnetic nanostructures,” Comput. Mater. Sci. 28, 366–383 (2003).
  * Fischbacher  _et al._ (2007) T. Fischbacher, M. Franchin, G. Bordignon, and H. Fangohr, “A systematic approach to multiphysics extensions of finite-element-based micromagnetic simulations: Nmag,” IEEE Trans. Magn. 43, 2896–2898 (2007) URL http://nmag.soton.ac.uk .
  * (12) “MicroMagnum”  URL http://micromagnum.informatik.uni-hamburg.de .
  * Abert  _et al._ (2013) C. Abert, L. Exl, F. Bruckner, A. Drews, and D. Suess, “Magnum.fe: A micromagnetic finite-element simulation code based on FEniCS,” J. Magn. Magn. Mater. 345, 29–35 (2013) .
  * Wang  _et al._ (2016) W. Wang, M.-A. Bisotti, D. Cortes, T. Kluyver, M. Vousden, R. Pepper, O. Laslett, R. Carey, and H. Fangohr, “Fidimag 0.1: finite difference micromagnetic simulation environment,”  (2016)  URL https://github.com/computationalmodelling/fidimag .
  * Pérez and Granger (2007) F. Pérez and B. E. Granger, “IPython: A system for interactive scientific computing,” Comput. Sci. Eng. 9, 21–29 (2007) URL https://ipython.org/ .
  * Hunt and Thomas (1999) A. Hunt and D. Thomas,  _The Pragmatic Programmer_ , 1st ed. (Addison Wesley, 1999).
  * (17) M. Beg, R. A. Pepper, and H. Fangohr, “OOMMFC”  URL https://github.com/joommf/oommfc .
  * Fangohr (2004) H. Fangohr,  _Lect. Notes Comput. Sci._, edited by M. Bubak, G. D. van Albada, P. M. A. Sloot, and J. Dongarra, Lecture Notes in Computer Science, Vol. 3039 (Springer Berlin Heidelberg, Berlin, Heidelberg, 2004) pp. 1210–1217.
  * Gorelick and Ozsvald (2014) M. Gorelick and I. Ozsvald,  _High Performance Python: Practical Performant Programming for Humans_ , 1st ed. (O’Reilly Media, 2014).
  * (20) E. Jones, T. Oliphant, P. Peterson, and Others, “SciPy: Open source scientific tools for Python”  URL https://www.scipy.org .
  * van der Walt, Colbert, and Varoquaux (2011) S. van der Walt, S. C. Colbert, and G. Varoquaux, “The NumPy Array: A Structure for Efficient Numerical Computation,” Comput. Sci. Eng. 13, 22–30 (2011).
  * McKinney (2011) W. McKinney, “pandas: a Foundational Python Library for Data Analysis and Statistics”  (2011)  URL https://pandas.pydata.org .
  * Hunter (2007) J. D. Hunter, “Matplotlib: A 2D graphics environment,” Comput. Sci. Eng. 9, 99–104 (2007).
  * Beazley (1996) D. M. Beazley, “SWIG: an easy to use tool for integrating scripting languages with C and C++” in  _TCLTK’96 Proc. 4th Conf. USENIX Tcl/Tk Work._ (1996) p. 15  URL http://www.swig.org .
  * Abrahams (2003) D. Abrahams, “Building Hybrid Systems with Boost.Python”  (2003)  URL https://www.boost.org .
  * Behnel  _et al._ (2011) S. Behnel, R. Bradshaw, C. Citro, L. Dalcin, D. S. Seljebotn, and K. Smith, “Cython: The Best of Both Worlds,” Comput. Sci. Eng. 13, 31–39 (2011).
  * Gilbert (2004) T. L. Gilbert, “A phenomenological theory of damping in ferromagnetic materials,” IEEE Trans. Magn. 40, 3443–3449 (2004).