Controlled Gate Networks Applied to Eigenvalue Estimation

Max Bee-Lindgren
School of Physics, Georgia Institute of Technology, Atlanta, GA 30332, USA
Zhengrong Qian
Facility for Rare Isotope Beams and Department of Physics and
Astronomy, Michigan State University, MI 48824, USA
Matthew DeCross
Quantinuum, 303 S. Technology Ct., Broomfield, Colorado 80021, USA
Natalie C. Brown
Quantinuum, 303 S. Technology Ct., Broomfield, Colorado 80021, USA
Christopher N. Gilbreth
Quantinuum, 303 S. Technology Ct., Broomfield, Colorado 80021, USA
Jacob Watkins
Xilin Zhang
Dean Lee
Facility for Rare Isotope Beams and Department of Physics and
Astronomy, Michigan State University, MI 48824, USA

We introduce a new scheme for quantum circuit design called controlled gate networks.  Rather than trying to reduce the complexity of individual unitary operations, the new strategy is to toggle between all of the unitary operations needed with the fewest number of gates.  We illustrate our approach using two examples.  The first example is a variational subspace calculation for a two-qubit system.  We demonstrate an approximately five-fold reduction in the number of two-qubit gates required for computing inner products and Hamiltonian matrix elements.  The second example is estimating the eigenvalues of a two-qubit Hamiltonian via the Rodeo Algorithm [Choi et al., 2021] using a specific class of controlled gate networks called controlled reversal gates.  Again, a five-fold reduction in the number of two-qubit gates is demonstrated. We use the Quantinuum H1-2 and IBM Perth devices to realize the quantum circuits. Our work demonstrates that controlled gate networks are a useful tool for reducing gate complexity in quantum algorithms for quantum many-body problems.

§ INTRODUCTION

Many quantum circuits involve combinations of unitary operators executed in sequence, with logical control directed by one or more auxiliary registers [Childs and Wiebe, 2012, Berry et al., 2014, Low and Chuang, 2019, Gilyén et al., 2019]. In order to minimize circuit complexity, many promising techniques and strategies have been developed [Khatri et al., 2019, Kamaka, 2020, Yordanov et al., 2020, Nash et al., 2020, Younis and Iancu, 2022, Dobbs et al., 2022, Weiden et al., 2023].  For circuits with controlled unitary operations, the conventional approach is to reduce the gate complexity of each controlled unitary operator as much as possible. During the transpilation stage, simplifications between circuit blocks may be found and exploited, such as cancelling neighboring CNOT gates. However, this alone does not suggest how one ought to design circuits intentionally such that cancellations are common.

Here we present a different approach in which all the unitary operations needed are viewed as a connected network, and the transformations between the operations are optimized. We define a controlled gate network to be a set of controlled operations such that any two operations can be transformed into each other by inserting or removing a small number of gates that we call transformation gates.  An example of a controlled gate network is illustrated in Fig. <ref>.  The left side shows a circuit using two ancilla qubits to toggle between four possible unitary operators, $U_1, U_2, U_3, U_4$.  The right side achieves the same results using a controlled gate network.  Each of the transformation gates $G_i$ act on a small number of qubits, and the ancilla qubits control the transformation gates.  Further details are given in the figure caption.

Controlled gate networks can be very useful for quantum algorithms that repeatedly use controlled gates.

Examples include linear combinations of unitaries for Hamiltonian simulation and quantum signal processing [Childs and Wiebe, 2012, Hao Low and Chuang, 2016], phase estimation [Kitaev, 1995, Abrams and Lloyd, 1999, Cleve et al., 1998, Svore et al., 2013], imaginary time evolution [Motta et al., 2020, Turro et al., 2022], and variational subspace methods [Parrish and McMahon, 2019, Francis et al., 2022, Yang et al., 2022, Cortes and Gray, 2022, Mejuto-Zaera and Kemper, 2023]. In particular, controlled operations that are closely related to each other are well-suited for these networks, as only a small number of gates may be used to change between them. The application to variational subspace methods is particularly noteworthy.  There a single auxiliary qubit can be used to prepare the superposition of any two variational states generated by the controlled gate network. These superpositions can then be used to determine all the inner products and Hamiltonian matrix elements needed.

Our purpose of this work is to introduce the concept of a controlled gate network and show how it can reduce the number of gates in circuits when repeated controlled operations with similar structures are required. We illustrate with two specific examples that are analyzed in detail.  The first example is a variational subspace calculation for a two-qubit system.  The task requires computing inner products and Hamiltonian matrix elements for state vectors produced using unitary transformations with different input parameters.  We show that the standard approach requires 64 CNOT gates and 82 single qubit gates, but the controlled gate network approach requires only 13 CNOT gates and 21 single qubit gates.

The second controlled gate network example we discuss is called controlled reversal gates.  Controlled reversal gates reverse the flow of time in Hamiltonian simulations using anticommuting operators and an auxiliary qubit as a switch.  This functionality makes it valuable for eigenvalue estimation methods such as the Rodeo Algorithm.  We will see that controlled reversal gates reduce the number of two-qubit entangling gates required for our system by a factor of $5$ as compared with an approach using standard controlled gates. These calculations are carried out on the IBM Perth and Quantinuum H1-2 machines. In the following, we present the implementation of the Rodeo Algorithm with controlled reversal gates on the IBM Perth and Quantinuum H1-2 machines.

Example of a controlled gate network.  The left side shows a circuit using two ancilla qubits to toggle between four possible unitary operators, $U_1, U_2, U_3, U_4$.  The filled (open) dot means that the controlled operator is actuated when the attached ancilla is in the $\ket{1}$ $(\ket{0})$ state.  The right side achieves the same results using a controlled gate network.  Each of the transformation gates $G_i$ act on a small number of qubits.  $U_1$ corresponds to no $G_i$ actuated, $U_2$ corresponds to actuating $\{G_1,G_3,G_4\}$, $U_3$ corresponds to $\{G_2,G_5\}$, and $U_4$ corresponds to $\{G_1,G_2,G_3,G_4,G_5\}$.

§ VARIATIONAL SUBSPACE CALCULATIONS USING CONTROLLED GATE NETWORKS

In the quantum approximate optimization algorithm (QAOA) [Farhi et al., 2014, Farhi and Harrow, 2016, Zhou et al., 2020], one approximates the ground state of a Hamiltonian $H$ using a variational ansatz of the form,

\begin{equation}
\ket{\vec{\epsilon},\vec{\lambda}} = \exp(-i\epsilon_N H_I)\exp(-i\lambda_N H) \cdots
\exp(-i\epsilon_1 H_I)\exp(-i\lambda_1 H)\ket{\psi_I},
\end{equation}

where $\ket{\psi_I}$ is the ground state of some trivial initial Hamiltonian, $H_I$.  The parameters $\vec{\epsilon}$ and $\vec{\lambda}$ are then optimized to give the lowest expectation value $\braket{\vec{\epsilon},\vec{\lambda}|H|\vec{\epsilon},\vec{\lambda}}$ \footnote{These parameters are typically labeled $\vec{\beta}$ and $\vec{\gamma}$ in the QAOA literature; however, these symbols have already been used elsewhere in the text so they have been relabeled here for clarity.}.  However, this single-vector ansatz for the ground state may not be sufficient.  The variational approximation can be improved if we allow for linear combinations of variationally-optimized QAOA vectors $|ϵ⃗^(i),λ⃗^(i)⟩$.  Variational optimization using a subspace rather than a single vector allows for the mixing of configurations that explicitly break symmetries of the Hamiltonian such as rotational invariance, particle number conservation, or internal flavor symmetries. The underlying symmetry is restored by allowing for linear combinations of the individual configurations.  This strategy of explicit symmetry breaking and symmetry restoration is widely used in nuclear physics
[\citenamefont{Satula et~al.}, 2010, \citenamefont{Yao et~al.}, 2020, \citenamefont{Sheikh et~al.}, 2021, \citenamefont{Frosini et~al.}, 2022, \citenamefont{Ruiz~Guzman and Lacroix}, 2022, \citenamefont{Lacroix et~al.}, 2023].  In order to perform a variational subspace calculation, one needs to be able to compute inner products, $⟨ϵ⃗^(i),λ⃗^(i)|ϵ⃗^(j),λ⃗^(j)|$⟩ and Hamiltonian matrix elements $\braket{\vec{\epsilon}^{(i)},\vec{\lambda}^{(i)}|H|\vec{\epsilon}^{(j)},\vec{\lambda}^{(j)}}$ between pairs of QAOA states.  One can then solve the generalized eigenvalue problem using classical post-processing to find the ground state of $H$ within the projected subspace [\citenamefont{Frame et~al.}, 2018, \citenamefont{Epperly et~al.}, 2022, \citenamefont{{Francis} et~al.}, 2022, \citenamefont{Mejuto-Zaera and Kemper}, 2023, \citenamefont{Sarkar et~al.}, 2023, \citenamefont{Duguet et~al.}, 2023].

To illustrate with a simple example, we consider a two-qubit system where the Hamiltonian $H$ corresponds to a Heisenberg model,
\begin{equation}
H = a X_j X_k + b Y_j Y_k + c Z_j Z_k.
\end{equation}
If we take the initial Hamiltonian to be $H_I = -∑_i X_i$, then each layer of the QAOA product will produce a product of exponentials of the form,
\begin{equation}
N(\alpha,\beta,\gamma,\delta) =\exp(i\delta X_j)\exp(i\delta X_k) \exp[i(\alpha X_j X_k + \beta Y_j Y_k + \gamma Z_j Z_k)],
\end{equation}
with real parameters $α$, $β$, $γ$, $δ$.
In order to perform the variational subspace calculations using such QAOA states, we need to construct superpositions of $|A⟩≡N(α,β,γ,δ)|ψ_I⟩$ and $|B⟩≡N(α+Δα,β+Δβ,γ+Δγ,δ+Δδ)|ψ_I⟩$, where $|ψ_I⟩$ is the initial state of the system.  Let us first show how this is done efficiently using a controlled gate network.

We can implement $N(α,β,γ,δ)$ and $N(α+Δα,β+Δβ,γ+Δγ,δ+Δδ)$ using a controlled gate network as shown in Fig.~\ref{fig:variational}.  We are using the implementation of $N(α,β,γ,δ)$ described in Ref.~[\citenamefont{Smith et~al.}, 2019].  The transformation gates are highlighted in yellow.  When the transformation gates are not included, the circuit gives $N(α,β,γ,δ)$.  When the transformation gates are included, the circuit equals $N(α+Δα,β+Δβ,γ+Δγ,δ+Δδ)$. By using one ancilla qubit to control all of the transformation gates, we can construct an arbitrary superposition of $|A⟩≡N(α,β,γ,δ)|ψ_I⟩$ and $|B⟩≡N(α+Δα,β+Δβ,γ+Δγ,δ+Δδ)|ψ_I⟩$, where $|ψ_I⟩$ is the initial state of the system.

\begin{figure}[ht]
\begin{adjustbox}{width=16cm}
\begin{quantikz}[transparent,slice style=blue]
\lstick{} & \gate{R_{x/y}(\theta)}\gategroup[1,steps=1,style={dashed,fill=cyan!60, inner xsep=2pt},background,label style={label position=below,anchor=north,yshift=-0.2cm}]{{\sc }}  & \qw & \qw & \ctrl{1} & \ctrl{2} & \qw & \qw & \ctrl{2} & \qw  & \qw &\qw & \ctrl{2}& \ctrl{1} & \gate{H} & \meter{}  \\
\qw & \qw    &  \targ{} & \gate{R_z(\pi/2 -2\gamma)} & \gate{R_z(-2\Delta\gamma)}\gategroup[1,steps=1,style={fill=yellow,inner sep=2pt},background] \qw & \qw & \ctrl{1} & \qw & \qw & \targ{} & \gate{R_z(\pi/2)} &  \gate{R_x(-2\delta)}  &\qw &\gate{R_x(-2\Delta\delta)}\gategroup[1,steps=1,style={fill=yellow,inner sep=2pt},background]  \qw & \qw &\qw\\
\qw & \gate{R_z(-\pi/2)} & \ctrl{-1} &  \gate{R_y(2\alpha-\pi/2)} & \qw & \gate{R_y(2\Delta\alpha)}\gategroup[1,steps=1,style={fill=yellow,inner sep=2pt},background]  \qw & \targ{} & \gate{R_y(\pi/2-2\beta)} & \gate{R_y(-2\Delta\beta)}\gategroup[1,steps=1,style={fill=yellow,inner sep=2pt},background]  \qw & \ctrl{-1} & \gate{R_x(-2\delta)} & \qw &\gate{R_x(-2\Delta\delta)}\gategroup[1,steps=1,style={fill=yellow,inner sep=2pt},background]  \qw & \qw & \qw & \qw
\end{quantikz}
\end{adjustbox}
\caption{Controlled gate network for our variational subspace calculation example. The transformation gates are highlighted in yellow.  When the transformation gates are not included, the circuit gives $N(\alpha,\beta,\gamma,\delta)$.  When the transformation gates are included, the circuit equals $N(\alpha+\Delta \alpha,\beta+\Delta \beta,\gamma+\Delta \gamma,\delta+\Delta\delta)$.  By using one ancilla qubit to control all of the transformation gates, we can construct an arbitrary superposition of $N(\alpha,\beta,\gamma,\delta)$ and $N(\alpha+\Delta \alpha,\beta+\Delta \beta,\gamma+\Delta \gamma,\delta + \Delta\delta)$.}
\label{fig:variational}
\end{figure}

By aligning $n̂$ with $y$ and $x$ axes in the $R_n̂(θ)$ gate and performing a measurement on the ancilla qubit, we can deduce the real and imaginary parts of the inner product $⟨A|B|$⟩ from the measurement probabilities of the $\ket{0}$ and $\ket{1}$ states.  We can apply the same technique to calculate the matrix elements for the Hamiltonian, $\langle A|H|B\rangle$, and their complex conjugates.  This is done by decomposing $H$ into a sum of unitary operators such as strings of Paul matrices, $H=\sum_k U_k$.  We then replace $\ket{B}$ in the description above with $U_k\ket{B}$ for each $k$.  If we decompose and transpile the circuit in Fig. <ref> into native gates for the IBM system, we get the circuit shown in Fig. <ref>.  A total of 13 CNOT gates and 21 single qubit gates are required.

[transparent, slice style=blue, every arrow/.append style=-Triangle, line width=0.7mm]
U_3(θ,-π/2/0,π/2/0)[1,steps=1,style=dashed,fill=cyan!60, inner xsep=2pt,background,label style=label position=below,anchor=north,yshift=-0.2cm]              [2,steps=4,style=dashed,rounded corners,fill=yellow, inner xsep=2pt,background,label style=label position=below,anchor=north,yshift=-0.2cm]      1         1     [3,steps=4,style=dashed,rounded corners,fill=yellow, inner xsep=2pt,background,label style=label position=below,anchor=north,yshift=-0.2cm]      2          2             [3,steps=4,style=dashed,rounded corners,fill=yellow, inner xsep=2pt,background,label style=label position=below,anchor=north,yshift=-0.2cm]      2         2      [r]

U_1(π/2 -2γ)     U_1(-Δγ)        U_1(Δγ)                     1                             [r]

U_1(-π/2)     -1      U_3(2α-π/2,0,0)                  U_3(Δα,0,0)       U_3(-Δα,0,0)               U_3(π/2-2β,0,0)     U_3(-Δβ,0,0)        U_3(Δβ,0,0)         [r]

[transparent, slice style=blue, every arrow/.append style=-Triangle, line width=0.7mm]
[l]                     [3,steps=5,style=dashed,rounded corners,fill=yellow, inner xsep=2pt,background,label style=label position=below,anchor=north,yshift=-0.2cm]       2         2            [2,steps=5,style=dashed,rounded corners,fill=yellow, inner xsep=2pt,background,label style=label position=below,anchor=north,yshift=-0.2cm]     1         1       U_2(0,π)

[l]             U_1(π/2)      U_3(-2δ,-π/2,π/2)                          U_1(π/2)        U_3(Δδ,0,0)        U_3(-Δδ,-π/2,0)

[l]        -1     U_3(-2δ,-π/2,π/2)        U_1(π/2)        U_3(Δδ,0,0)        U_3(-Δδ,-π/2,0)

This is the fully decomposed and transpiled version for the circuit in Fig. <ref> using gates native to the IBM system.  A total of 13 CNOT gates and 21 single qubit gates are required.

We can now compare with the standard approach for calculating $\braket{A|B}$ using a Hadamard test [\citenamefont{{Francis} et~al.}, 2022].  The corresponding circuit is shown in Fig.~\ref{fig:standard}.
If we decompose and transpile the circuit in Fig.~\ref{fig:standard} into native gates for the IBM quantum devices, we get the circuit shown in Fig.~\ref{fig:standard_transpiled}.  A total of 64 CNOT gates and 82 single qubit gates are required.  We conclude that the controlled gate network approach substantially more efficient than the standard approach.  The reduction in the number of gates is a factor of about $5$ for the CNOT gates and a factor of about $4$ for the single qubit gates.

\begin{figure}
\begin{adjustbox}{width=0.6\columnwidth}
\begin{quantikz}[transparent,slice style=blue]
& \gate{H}      & \ctrl{1}
% \gategroup[2,steps=3,style={dashed, rounded corners,fill=cyan!20, inner xsep=2pt},background,label style={label position=below,anchor= north,yshift=-0.2cm}]{{Base system}}
& \qw & \ctrl{1} & \gate{R_{x/y}(\theta)}\gategroup[1,steps=1,style={dashed,fill=cyan!60, inner xsep=2pt},background,label style={label position=below,anchor=north,yshift=-0.2cm}]{{\sc }}   & \meter{} \\
& \qw & \gate{N(\alpha,\beta,\gamma,\delta)}  & \qw & \gate{N^\dagger(\alpha+\Delta\alpha,\beta+\Delta\beta,\gamma+\Delta\gamma,\delta+\Delta\delta)}  & \qw & \qw
\end{quantikz}
\end{adjustbox}
\caption{Standard Hadamard test circuit for computing the real and imaginary parts of the inner product $\braket{A|B}$.}
\label{fig:standard}
\end{figure}

\begin{figure}
\begin{adjustbox}{scale=0.5}
\begin{quantikz}[transparent, slice style=blue, every arrow/.append style={-Triangle, line width=0.7mm}]
\qw& \gate{U_2(0,\pi)} & \ctrl{2} & \qw & \ctrl{2} & \qw& \qw& \ctrl{1} & \qw & \qw& \qw& \ctrl{1} &\qw &\ctrl{2} &\gate{U_1(\pi/4)}&\ctrl{2} &\ctrl{1} &\qw &\ctrl{1} &\ctrl{2} &\qw &\ctrl{2}&  \arrow[r]\qw&
\\
\qw&\qw & \qw & \gate{U_2(0,\pi)} & \qw  &\targ{} & \gate{U_1(-\pi/4)}&\targ{} & \gate{U_1(\pi/4)} &\targ{}& \gate{U_1(-\pi/4)}&\targ{} & \gate{U_2(0,-3\pi/4)} &\qw &\gate{U_1(\pi/4-\gamma)} &\qw &\targ{} &\gate{U_1(\gamma-\pi/4)} & \targ{} &\qw&\qw &\qw&  \arrow[r]\qw&
\\
\qw&\gate{U_1(-\pi/4)} & \targ{} & \gate{U_1(\pi/4)} & \targ{} & \ctrl{-1}& \qw& \qw& \qw& \ctrl{-1} & \gate{U_1(\pi/4)}& \qw & \qw & \targ{} &\gate{U_1(-\pi/4)} &\targ{} &\qw  &\gate{U_3(\alpha-\pi/4,0,0)} &\qw &\targ{} &\gate{U_3(\pi/4-\alpha,0,0)} &\targ{}&  \arrow[r]\qw&
\end{quantikz}
\end{adjustbox}

\vspace{0.5cm}

\begin{adjustbox}{scale=0.5}
\begin{quantikz}[transparent, slice style=blue, every arrow/.append style={-Triangle, line width=0.7mm}]
\qw & \arrow[l]& \qw & \qw& \qw& \ctrl{2}& \qw& \qw& \qw& \ctrl{2} &\ctrl{1} &\qw &\gate{U_1(\pi/4)} &\ctrl{1} &\ctrl{2} &\qw &\ctrl{2}
&\qw&\qw &\ctrl{1} &\qw &\qw &\qw &\arrow[r]\qw&
\\
\qw & \arrow[l]& \qw &\ctrl{1} &\qw &\qw&\qw &\ctrl{1}  &\gate{U_1(\pi/4)}&\qw &\targ{} &\qw &\gate{U_1(-\pi/4)} &\targ{} &\qw &\gate{U_2(0,\pi)} &\qw &\targ{} &\gate{U_1(-\pi/4)} &\targ{} &\gate{U_1(\pi/4)} &\targ{}&\gate{U_1(-\pi/4)}  &\arrow[r]\qw&
\\
\qw & \arrow[l]& \gate{U_2(0,\pi)} & \targ{} &\gate{U_1(-\pi/4)} & \targ{} &\gate{U_1(\pi/4)} & \targ{} &\gate{U_1(-\pi/4)} &\targ{} &\gate{U_2(0,-3\pi/4)} &\qw &\gate{U_3(\pi/4-\beta, 0,0)}&\qw &\targ{} &\gate{U_3(\beta-\pi/4, 0,0)} &\targ{} &\ctrl{-1} &\qw &\qw &\qw &\ctrl{-1} &\gate{U_1(\pi/4)} &\arrow[r]\qw&
\end{quantikz}
\end{adjustbox}

\vspace{0.5cm}

\begin{adjustbox}{scale=0.5}
\begin{quantikz}[transparent, slice style=blue, every arrow/.append style={-Triangle, line width=0.7mm}]
\qw & \arrow[l]& \ctrl{1} &\qw & \ctrl{2} &\gate{U_1(\frac{\pi}{4})} & \ctrl{2}& \ctrl{1} &\qw & \ctrl{1} &\ctrl{2} &\qw & \ctrl{2} & \ctrl{1} &\qw & \ctrl{1} &\qw &\qw & \ctrl{1} &\qw & \ctrl{1} &\qw &\arrow[r]\qw&
\\
\qw & \arrow[l]& \targ{} &\gate{U_2(0,-\frac{3\pi}{4})} & \qw & \gate{U_1(\frac{\pi}{4})} & \qw & \targ{} & \gate{U_1(-\frac{\pi}{4})} & \targ{} &\qw &\gate{U_1(\frac{\pi}{2})}  &\qw &\targ{} &\gate{U_3(\delta,0,0)} &\targ{} &\gate{U_3(-\delta, -\frac{\pi}{2},0)} & \gate{U_1(\frac{\pi}{2})} &\targ{} &\gate{U_3(-\Delta\delta-\delta,0,0)} &\targ{} &\gate{U_3(\Delta\delta+\delta,-\frac{\pi}{2},0)} &\arrow[r]\qw&
\\
\qw & \arrow[l]& \qw & \qw & \targ{}  &\gate{U_1(-\frac{\pi}{4})} & \targ{}& \qw &\gate{U_1(\frac{\pi}{2})} & \qw & \targ{} &\gate{U_3(\delta,0,0)} & \targ{} &\gate{U_3(-\delta, -\frac{\pi}{2},0)} &\qw &\qw &\qw &\qw &\qw &\qw  &\qw  &\qw &\arrow[r]\qw&
\end{quantikz}
\end{adjustbox}

\vspace{0.5cm}

\begin{adjustbox}{scale=0.5}
\begin{quantikz}[transparent, slice style=blue, every arrow/.append style={-Triangle, line width=0.7mm}]
\qw & \arrow[l]& \qw &\ctrl{2} & \ctrl{1}  &\ctrl{2}  & \qw & \ctrl{1} & \qw & \qw & \qw & \ctrl{1} & \qw & \qw & \qw & \ctrl{1} & \ctrl{2} & \gate{U_1(\frac{\pi}{4})} & \ctrl{2} & \qw & \ctrl{2} & \qw &\arrow[r]\qw&
\\
\qw & \arrow[l]& \gate{U_1(-\frac{\pi}{4})} & \qw & \targ{} & \qw  & \gate{U_1(\frac{\pi}{4})} & \targ{}  & \gate{U_2(0, \pi)} & \targ{}  & \gate{U_1(-\frac{\pi}{4})} & \targ{} & \gate{U_1(\frac{\pi}{4})} & \targ{} & \gate{U_1(-\frac{\pi}{4})} & \targ{} &\qw & \gate{U_2(0,-\frac{3\pi}{4})} &\qw
& \qw & \qw & \qw  &\arrow[r]\qw&
\\
\qw & \arrow[l]& \gate{U_1(\frac{\pi}{2})} & \targ{} & \gate{U_3(-\Delta\delta-\delta,0,0)}  & \targ{} & \gate{U_3(\Delta\delta+\delta,-\frac{\pi}{2},0)} &\qw &\qw & \ctrl{-1} & \qw & \qw & \qw & \ctrl{-1} & \gate{U_1(\frac{\pi}{4})} & \qw & \targ{} & \gate{U_1(-\frac{\pi}{4})} & \targ{} &\gate{U_3(\Delta\beta+\beta-\frac{\pi}{4},0,0)} & \targ{} &\gate{U_3(-\Delta\beta-\beta+\frac{\pi}{4},0,0)} &\arrow[r]\qw&
\end{quantikz}
\end{adjustbox}

\vspace{0.5cm}

\begin{adjustbox}{scale=0.5}
\begin{quantikz}[transparent, slice style=blue, every arrow/.append style={-Triangle, line width=0.7mm}]
\qw & \arrow[l]& \ctrl{2} &\qw &\qw & \qw & \ctrl{2} & \qw & \qw & \qw & \ctrl{2} &\ctrl{1} & \gate{U_1(\frac{\pi}{4})} &\ctrl{1} &\ctrl{2} &\qw &\ctrl{2} &\ctrl{1} & \qw &\ctrl{1} & \qw & \qw  &\arrow[r]\qw&
\\
\qw & \arrow[l]& \qw & \qw &\ctrl{1} & \qw & \qw & \qw &\ctrl{1} &\gate{U_1(\frac{\pi}{4})} & \qw &\targ{} & \gate{U_1(-\frac{\pi}{4})}&\targ{} &\qw &\gate{U_1(\Delta\gamma+\gamma-\frac{\pi}{4})} &\qw &\targ{} &\gate{U_1(-\Delta\gamma-\gamma+\frac{\pi}{4})} &\targ{} &\gate{U_2(0,\pi)} &\targ{}  &\arrow[r]\qw&
\\
\qw & \arrow[l]& \targ{} &\gate{U_2(0,\pi)} & \targ{} &\gate{U_1(-\frac{\pi}{4})}  & \targ{} &\gate{U_1(\frac{\pi}{4})} & \targ{} &\gate{U_1(-\frac{\pi}{4})} & \targ{} &\gate{U_2(0,-\frac{3\pi}{4})} &\gate{U_3(-\Delta\alpha-\alpha+\frac{\pi}{4},0,0)} &\qw &\targ{} &\gate{U_3(\Delta\alpha+\alpha-\frac{\pi}{4},0,0)} &\targ{} &\qw &\qw &\qw &\qw &\ctrl{-1}  &\arrow[r]\qw&
\end{quantikz}
\end{adjustbox}

\vspace{0.5cm}

\begin{adjustbox}{scale=0.5}
\begin{quantikz}[transparent, slice style=blue, every arrow/.append style={-Triangle, line width=0.7mm}]
\qw & \arrow[l]& \qw &\ctrl{1} & \qw & \qw & \qw  & \ctrl{1} & \qw & \ctrl{2} &\gate{U_1(\frac{\pi}{4})} & \ctrl{2} & \qw & \ctrl{2} & \qw & \ctrl{2} & \qw  & \qw & \gate{U_3(\theta,-\frac{\pi}{2}/0,\frac{\pi}{2}/0)}\gategroup[1,steps=1,style={dashed,fill=cyan!60, inner xsep=2pt},background,label style={label position=below,anchor=north,yshift=-0.2cm}]{{\sc }}  &\meter{}
\\
\qw & \arrow[l]& \gate{U_1(-\frac{\pi}{4})} &\targ{} & \gate{U_1(\frac{\pi}{4})} &\targ{} & \gate{U_1(-\frac{\pi}{4})} &\targ{} &\gate{U_2(0,-\frac{3\pi}{4})} & \qw & \qw& \qw & \qw & \qw & \qw & \qw & \qw & \qw  & \qw & \qw
\\
\qw & \arrow[l]& \qw & \qw & \qw & \ctrl{-1} &\gate{U_1(\frac{\pi}{4})}& \qw & \qw &\targ{}&\gate{U_1(-\frac{\pi}{4})} &\targ{} &\gate{U_1(\frac{\pi}{4})} &\targ{} &\gate{U_1(-\frac{\pi}{4})} &\targ{}  & \qw & \qw  & \qw & \qw
\end{quantikz}
\end{adjustbox}
\caption{The fully decomposed and transpiled version for the circuit in Fig.~\ref{fig:standard} using gates native to the IBM quantum devices.  A total of 64 CNOT gates and 82 single qubit gates are required.}.
\label{fig:standard_transpiled}
\end{figure}

\section{Rodeo algorithm with controlled reversal gates}
There are several well-known quantum computing algorithms that measure eigenvalues of a Hermitian operator $H$ by means of time evolution controlled by an auxiliary quantum register. An important example is the class of phase estimation algorithms, which includes standard phase estimation [\citenamefont{{Abrams} and {Lloyd}}, 1999, \citenamefont{{Cleve} et~al.}, 1998, \citenamefont{Siwach and Lacroix}, 2021] and iterative quantum phase estimation [\citenamefont{Kitaev}, 1995, \citenamefont{{Svore} et~al.}, 2013]. The Rodeo Algorithm is another recently introduced method that uses controlled time evolution [\citenamefont{Choi et~al.}, 2021]. It is an iterative algorithm that uses destructive interference to suppress eigenvectors with eigenvalues different from a desired ``target" eigenvalue. The Rodeo Algorithm was shown to have exponentially improved scaling in terms of the precision $\Delta$ for preparing eigenstates [\citenamefont{Choi et~al.}, 2021], compared to quantum phase estimation and adiabatic evolution [\citenamefont{{Farhi} et~al.}, 2000, \citenamefont{{Wiebe} and {Babcock}}, 2012].

The first implementation of the Rodeo Algorithm was performed in Ref.~[\citenamefont{Qian et~al.}, 2021] for a single qubit Hamiltonian. However, the use of controlled time evolution makes the implementation for a multi-qubit system significantly more difficult, especially on existing noisy quantum devices.  In order to reduce the number of two qubit operations, in this work we use controlled reversal gates that allow one to flip the direction of time evolution using an auxiliary qubit.

Let us consider the two-qubit Hamiltonian
\begin{align}\label{eq:system}
H_\mathrm{obj} = c_1 X_1 Z_2 + c_2 Z_1  X_2
\end{align}
where \(c_1, c_2 \in \mathbb{R}\) and \(X_i\) and \(Z_i\) are the Pauli operators on qubit $i$. Following the notation of Ref.~[\citenamefont{Choi et~al.}, 2021, \citenamefont{Qian et~al.}, 2021], we call $H_\mathrm{obj}$ the ``object Hamiltonian". We take the coefficients to be $c_1 = 2.5$ and $c_2 = 1.5$, and the Hamiltonian has four eigenvalues \(\pm(c_1 \pm c_2)\). The quantum circuit implementating the time evolution of \(H_\mathrm{obj}\) is shown in \cref{fig:system}.

Each cycle of the Rodeo Algorithm uses a controlled time evolution with a random time $t$ sampled from a normal distribution with zero mean and standard deviation \(\sigma\) set by the user. For any given rodeo cycle, the measurement of \(0\) for the ancilla corresponds to a successful measurement. Let $\ket{\psi_I}$ be the initial state and let $\ket{E_k}$ denote the energy eigenstates of $H_{\rm obj}$ with energy $E_k$. If we apply \(n\) cycles of the Rodeo Algorithm and average over many random trials, the probability of success for all $n$ cycles at target energy $E$ (a free parameter) is
\begin{equation}
P_n(E) = \sum_k  \frac{\left[1+e^{-(E-E_k)^2\sigma^2/2}\right]^n \left| \braket{E_k|\psi_I} \right|^2 }{2^n} .
\label{eq:probability1}
\end{equation}
We see that there are peaks centered at each energy eigenvalue $E_k$ where the initial state has nonzero overlap with $\ket{E_k}$. We also see that there is a constant background value of \(\frac{1}{2^n}\) for values of $E$ that are significantly further than $\sigma^{-1}$ away from all energy eigenvalues. Thus, repeated cycles will indicate peaks whenever $E$ is chosen within \(O(\sigma^{-1})\) of some \(E_k\) such that \(\left| \braket{E_k|\psi_I} \right|^2\) is not vanishingly small.

When implemented using the IBM Qiskit transpiler [\citenamefont{Anis et~al.}, 2021] under the assumption of full connectivity between all qubits, directly controlling the time evolution of the circuit in \cref{fig:system} with an ancilla qubit requires a total of 20 CNOT gates. For the Quantinuum H1-2 device at the time of this experiment, the two-qubit entangling gate was the $R_{ZZ}$ gate with angle $\pi/2$, and one needs 20 $R_{ZZ}$ gates for the analogous circuit. With only partial connectivity, the number of two-qubit entangling gates grows even larger. For three cycles of the Rodeo Algorithm, we therefore need 60 two-qubit entangling gates. For five cycles of the Rodeo Algorithm, we need 100 two-qubit entangling gates. For systems with more than two qubits, the problem is even more severe since the number of two-qubit gates will also scale with the number of Trotter-Suzuki time steps.

\begin{figure}
\begin{adjustbox}{width=0.3\columnwidth}
\begin{quantikz}[transparent,slice style=blue]
\lstick{$\ket{0}$} & \qw      & \ctrl{1}
% \gategroup[2,steps=3,style={dashed, rounded corners,fill=cyan!20, inner xsep=2pt},background,label style={label position=below,anchor= north,yshift=-0.2cm}]{{Base system}}
& \gate{R_x(2 c_1 t)} & \ctrl{1} & \qw      & \qw \\
\lstick{$\ket{0}$} & \gate{H} & \targ{}  & \gate{R_z(2 c_2 t)} & \targ{}  & \gate{H} & \qw
\end{quantikz}
\end{adjustbox}
\caption{Quantum circuit implementing the two-qubit unitary \(\exp\{-i H_\mathrm{obj} t\}\), where \(H_\mathrm{obj}\) is given by~\cref{eq:system}.}
\label{fig:system}
\end{figure}

Controlled reversal gates can reduce these circuit complexities. We define a reversal gate $R$ to be a product of single qubit gates $R = G_1 G_2 \cdots G_N$ that anticommutes with some subset of terms in the Hamiltonian, and write $H_R$ to be the part of the Hamiltonian anticommuting with $R$. If the Hamiltonian is written as a sum of products of Pauli gates, then it is straightforward to partition the terms in the Hamiltonian so that each partition has some corresponding reversal gate. A controlled reversal gate $C_R$ is the controlled version of $R$, i.e., $C_R = C_{G_1} C_{G_2} \cdots C_{G_N}$. Since $R H_R R^\dagger = -H_R$, and since $-H$ generates a backwards evolution of $H$, $C_R$ allows one to toggle the flow of time forwards and backwards, depending on the state of the auxiliary register.

The relative phase between the states entangled with the ancilla states $\ket{0}$ and $\ket{1}$ is the physically relevant quantity, while any global phase is irrelevant.  When using controlled reversal gates, it is desirable that the approximate time evolution operator $\tilde{U}(t)$ is time-reversal symmetric, in the sense of $\tilde{U}(-t) = \tilde{U}(t)^\dagger$. This ensures that $\tilde{U}(t)$ and $\tilde{U}(-t)$ have the same stationary states. For our two-qubit Hamiltonian example, the evolution is exact and thus this symmetry is satisfied. For Hamiltonian simulation using the Trotter approximation, the symmetry is satisfied if the single Trotter step $\mathcal{P}(t)$ satisfies $\mathcal{P}(-dt)=\mathcal{P}(dt)^{-1}$. This is automatically satisfied by the recursive Trotter-Suzuki formulas, particularly at second order [\citenamefont{Suzuki}, 1993, \citenamefont{{Hatano} and {Suzuki}}, 2005].

The controlled reversal gates implement forward or backward time evolution depending on the ancilla state.  Therefore, the relative phase between the two states evolves at twice the rate as the standard controlled-$\tilde{U}$ setup, and we can reduce the number of Trotter steps by a factor of two. This feature by itself provides a two-fold performance advantage.  For this reason, the computational advantage in using controlled reversal gates with the rodeo algorithm is at least a factor of two, independent of the details of the Hamiltonian.  For the Hamiltonian that we consider here, we can find a reversal gate that commutes with all of the terms in the Hamiltonian, resulting in an even greater computational advantage.  For the general case, however, we will need break up the Hamiltonian into pieces and apply the appropriate reversal gate for each piece.  If we are using the Trotter approximation to implement time evolution, then this break up of the Hamiltonian is already done, and the application of controlled reversal gates for the time evolution is straightfoward.

\section{Identifying Energy Eigenvalues}

We now use controlled reversal gates and the Rodeo Algorithm to compute the energy spectrum of $H_\mathrm{obj}$ from Eq.~\eqref{eq:system}. We note that the single-qubit Pauli \(Y_1\) anticommutes with $H_\mathrm{obj}$ and thus choose $C_{Y_1}$ as our controlled reversal gate ($C_{Y_2}$ is equally valid). One cycle of the Rodeo Algorithm for our system is shown in \cref{fig:1CycleCirc}. It consists of a pair of controlled reversal gates controlled by a single ancilla qubit followed by mid-circuit measurement. By implementing the controlled reversal gates, we are able to reduce the number of CNOT gates for each rodeo cycle from 20 to 4.

\begin{figure}
\begin{adjustbox}{width=0.5\columnwidth}

\begin{quantikz}[transparent,slice style=blue]
\lstick{$\ket{0}$} & \gate{H} & \octrl{1}
\gategroup[2,steps=1,style={dashed, rounded corners,fill=cyan!20, inner xsep=2pt},background,label style={label position=above,anchor= west,xshift=0.75cm}]{{Controlled reversal gates}}
& \qw      & \qw                 & \qw      & \octrl{1}
\gategroup[2,steps=1,style={dashed, rounded corners,fill=cyan!20, inner xsep=2pt},background]{} &
\gate{P(2E t)} & \gate{H} & \meter{} & \qw \\
\lstick{$\ket{0}$} & \qw      & \gate{Y}  & \ctrl{1} & \gate{R_x(2 c_1 t)} & \ctrl{1} & \gate{Y}  &  \qw             & \qw      & \qw      & \qw \\
\lstick{$\ket{0}$} & \qw      & \gate{H}  & \targ{}  & \gate{R_z(2 c_2 t)} & \targ{}  & \gate{H}  &  \qw             & \qw      & \qw      & \qw
\end{quantikz}
\end{adjustbox}
\caption{Circuit diagram for one cycle of the Rodeo Algorithm on the two-qubit system. The ancilla is shown on top, and the controlled reversal gates are the pair of $C_Y$ gates. The phase gate $P(2Et)$ multiplies a phase of $\exp(i2Et)$ to the $\ket{1}$ state of the ancilla and cancels the relative phase produced by the controlled time evolution for an eigenstate with energy $E$.}
\label{fig:1CycleCirc}
\end{figure}

Our search for the energy eigenvalues consists of three separate ``energy scans," referring to a collection of cycles with same resolution \(\sigma\) but varying target energy \(E\). Each subsequent scan has a finer resolution and smaller range.
\begin{comment}The energy range for the first scan is chosen to be from $-5$ to $5$. These values are determined using the Gershgorin circle theorem \cite{gershgorin1931} together with some extra padding to allow for the tails of the probability distribution.
\end{comment}
The purpose of our coarse-grained first scan ($\sigma=4$) is to locate all possible regions of the energy spectrum with energy eigenvalue peaks. We identify any region with a success probability well above the average background value of $\tfrac{1}{2^n}$ as a candidate for the more focused second scan with finer resolution ($\sigma=14$).
\begin{comment}Later in our discussion we give the expected statistical error $\epsilon_n(E)$ associated with our measurement of $P_n(E)$.
\end{comment}
The second scan is used to more finely determine the structure of the peaks associated with each energy eigenvalue. The third and final scan uses the finest resolution ($\sigma=24$) in order to accurately determine the center of each eigenvalue peak. The final scan consists of about twenty circuits spanning a narrow region around each peak identified in the second scan.

%Our search for the energy eigenvalues consists of three separate scans. Each subsequent scan has a finer resolution with \(\sigma=4, 14, 24\) and covers a smaller range. The energy range for the first scan was chosen to be from $-5$ to $5$. These values were determined using the Gershgorin circle theorem \cite{gershgorin1931} together with some extra padding to allow for the tails of probability distribution. The purpose of our coarse-grained first scan ($\sigma=4$) is to locate all possible regions of the energy spectrum with energy eigenvalue peaks. We identify any region with a success probability well above the average value as a candidate for the more focused second scan with finer resolution ($\sigma=14$). The second scan ($\sigma=14$) is used to determine shapes of the peaks produced the energy eigenvalues. The third scan uses the finest resolution ($\sigma=24$) in order to accurately determine the location of the center of each eigenvalue peaks.
%The second scan is run over the range of energies with potential energy eigenvalues, and it is fine enough to fit a Gaussian function to the success probability to find an estimate for the energy eigenvalue. We then run a narrow, high-resolution scan around each of the energy eigenvalues identified in the second scan. We fit a Gaussian to obtain our final estimate for each energy eigenvalue.
%Our first pass consisted of 11 scans each with 5 runs of the Rodeo Algorithm, and the average success probability for each scan was used. The additional runs suppress peaks not on an energy eigenvalue, but they do not reduce the background noise. The second pass consists of 11 scans each the average of two runs at all potential peaks identified in the first pass, then a final pass can consists of at least with 22 measurements closely centered around scans each per peak.

\section{IBM and Quantinuum Results}
We have performed the Rodeo Algorithm using three cycles and five cycles on the IBM Perth machine and five cycles on the Quantinuum H1-2 machine. For all cases, we follow the same procedure. The initial state is $\ket{\psi_I} = \ket{00}$.  The first energy scan is performed with $\sigma=4$, the second scan with $\sigma=14$, and third and final scan with $\sigma=24$.

Let $N_c$ be the number of distinct circuits implemented in a given scan, each with a unique set of Gaussian random values for the times. Let $N_s$ be the number of shots used for each circuit. The observed success probability $P_n(E)$ for $n$ cycles equals the number of successful measurements of $0$'s for all $n$ cycles divided by $N_sN_c$. For all the circuits performed in this study, we use $N_s=1024$. For the first scan we take $N_c = 5$, for the second scan $N_c = 2$, and for the final scan $N_c=1$. Since we are drawing from a binomial distribution, we can estimate that the one-standard-deviation statistical error for our measurement of $P_n(E)$ will be $\epsilon_n(E)=\sqrt{P_n(E)[1-P_n(E)]/(N_sN_c)}$. Since the success probability has a peak at each energy eigenvalue, $E_k$, we can fit a Gaussian function to each peak in the final scan data and determine the eigenvalues $E_k$.

The results for three cycles of the Rodeo Algorithm on IBM Perth are shown in the third column of \cref{tb:results}. The success probabilities versus energy are shown in \cref{fig:three}.  The curve with the dashed lines shows the exact analytical results for $\sigma = 4$. We also show the results for the first scan using a noiseless classical simulation. The peak success probabilities for the first scan are about $15\%$ lower than the values from the noiseless classical simulations. The average CNOT error for the portion of the IBM Perth device used is about $1\%$, and we expect that the largest source of noise is due to the 12 CNOT gates needed for each circuit. When compared with the exact results, the RMS deviation for the four eigenvalues is $0.010$, which is $0.12\%$ of the full range of the energy spectrum.  Meanwhile, the RMS value for the one-sigma uncertainties of the four eigenvalues is $0.006$ or $0.07\%$ of the full energy range.  We observe that the relative accuracy of the energy measurement is far better than the fidelity of the quantum circuits and is a striking example of the strong noise resilience of the Rodeo Algorithm.

\begin{figure}[htb]
\centering
\subfigure[]{\includegraphics[width=8cm]{Figures/3cycle_Perth.png}}
\caption{Success probabilities corresponding to three scans of the rodeo algorithm for the IBM Perth system with three cycles.  The dashed line is the expected success probability from classical calculations of the first scan, and the white boxes are noiseless simulations at the energies of first scan.}
\label{fig:three}
\end{figure}

The success probability versus energy for five cycles on IBM Perth are shown in Panel (a) of \cref{fig:five}. In this case we see that the peak success probabilities for the first scan are about $30\%$ lower than the values from the noiseless classical simulations --- a significant reduction in performance. In this case we are using 20 CNOT gates. The results of the fit for five cycles on IBM Perth are shown in the fourth column of \cref{tb:results}. When compared with the exact results, the RMS deviation for the four eigenvalues is $0.004$, which is $0.05\%$ of the full range of the energy spectrum. The RMS value for the one-sigma uncertainties of the four eigenvalues is $0.003$ or $0.04\%$ of the full energy range.  We observe that the relative accuracy of the energy measurement is many orders of magnitude better than the roughly $30\%$ error rate for the quantum circuits.

The success probability versus energy for five cycles on the Quantinuum H1-2 are shown in Panel (b) of \cref{fig:five} [46]. In contrast with the IBM Perth results, the peak success probabilities for the first scan are close to those from the noiseless classical simulations, as one might expect due to the much better gate fidelities for the Quantinuum H1-2. The error rate for the $R_{ZZ}$ gate at the time of the experiment was $0.3\%$, and the error rate for the $20$ $R_{ZZ}$ gates needed for five cycles explains the approximately $5\%$ reduction in peak heights compared with noiseless simulations. The results of the fit for five cycles on the Quantinuum H1-2 are shown in the fifth column of \cref{tb:results}. When compared with the exact results, the RMS deviation for the four eigenvalues is $0.005$, which is $0.06\%$ of the full range of the energy spectrum.  The RMS value for the one-sigma uncertainties of the four eigenvalues is $0.003$ or $0.03\%$ of the full energy range.  We again see that the accuracy of the energy measurement is several orders of magnitude better than the roughly $5\%$ error rate for the quantum circuits.

The noise robustness of the Rodeo Algorithm comes from the fact that it implements a simple strategy of reducing the spectral weight of eigenstates with the wrong energy [\citenamefont{Qian et~al.}, 2021].  While noise will reduce the spectral weight of the desired eigenstate, the associated eigenvalue peak will still be visible if the peak height is not reduced below the random background level.  Therefore, we expect the peak height to be reduced by noise, but the peak location should be unaffected.  This is clearly demonstrated in the results described above.  See the Supplemental Materials for a detailed analysis using a simple noise model.

\begin{figure}[htb]
\centering
\subfigure[]{\includegraphics[width=8cm]{Figures/5cycle_Perth.png}}
\subfigure[]{\includegraphics[width=8cm]{Figures/5cycle_H1-2.png}}
\caption{Success probabilities corresponding to three scans of the Rodeo Algorithm for (a) the IBM Perth system with five cycles and (b) the Quantinuum H1-2 system with five cycles.  The dashed line is the expected success probability from classical calculations of the first scan, and the white boxes are noiseless simulations at the energies of first scan.}
\label{fig:five}
\end{figure}

\section{Discussion and outlook}
We have introduced a new scheme for quantum circuit design called controlled gate networks.  A controlled gate network is a set of controlled operations such that any two operations can be transformed into each other by inserting or removing a small number of gates called transformation gates.  Rather than reducing the complexity of individual unitary operations, the goal is to toggle between the required unitary operations with the fewest number of gates.

We have demonstrated the method with two examples. The first is a variational subspace calculation for a two-qubit system. The second example is estimating the eigenvalues of a two-qubit Hamiltonian using the Rodeo Algorithm.  For the two examples presented here, the reduction in gate complexity using controlled gate networks is substantial.  We found an approximately five-fold reduction in the number of CNOT gates for the two-qubit variational subspace example.  We have also demonstrated a five-fold reduction in the number of CNOT gates need for the two-qubit Rodeo Algorithm implementation.  In the Supplemental Materials, we show that the generalization of the Rodeo Algorithm calculation for $N$ qubits yields an even more substantial factor of 20 reduction in the number of two-qubit entangling gates for large $N$ when using controlled reversal gates.

Controlled gate networks represent a new paradigm for quantum circuit design where a controlling qubit is used to efficiently transform from one unitary operation to another unitary operation with a similar structure.  These controlled unitary operations are common in quantum state preparation algorithms for quantum many-body systems.  We have explored two examples in this work and found substantial reductions in gate complexity.  We anticipate that controlled gate networks can play a prominent role in improving the efficiency for these applications and as well as many other quantum algorithms.

We have also demonstrated that the Rodeo Algorithm with controlled reversal gates is a practical and efficient method for eigenvalue estimation on current quantum devices.  In order to determine energy eigenvalues accurately, the Rodeo Algorithm needs only to be able to distinguish the peak associated with an energy eigenvalue above the background level of $\tfrac{1}{2^n}$.
Once this threshold criterion is reached, the accuracy of the Rodeo Algorithm for determining energy eigenvalue is set by the sharpness of the energy eigenvalue peaks in Eq.~(\ref{eq:probability1}). The width of the peak scales as $(\sqrt{n}\sigma)^{-1}$, where $n$ is the number of cycles and $\sigma$ is the standard deviation of the Gaussian random times. We have demonstrated here that with $n=3$ cycles of the Rodeo Algorithm and $\sigma = 24$, we can resolve the energy eigenvalue with an error of about $0.010$ on IBM Perth, or $0.42(\sqrt{n}\sigma)^{-1}$. With $n=5$ cycles of the Rodeo Algorithm and $\sigma = 24$, we can resolve the energy eigenvalue with an error of about $0.005$ on IBM Perth, or $0.27(\sqrt{n}\sigma)^{-1}$; on Quantinuum H1-2, we can resolve the energy eigenvalue with an error of about $0.006$, or $0.32(\sqrt{n}\sigma)^{-1}$.\\
\\
\begin{comment}
Each search gave a result that closely aligned with the exact value. All measured eigenvalues were within 1\% of their true value, and the largest absolute error 0.0141 from an initial scan width of 10. Furthermore, the errors of searches with different numbers of cycles was comparable. All errors were of order \(10^{-3}\) to \(10^{-4}\). This suggests that the error comes primarily from statistical noise as additional scans - and therefore additional suppression of unwanted states - has little effect on the error in the measurements.
\end{comment}

\begin{table}[htb]
\begin{tabular}{|c|c|c|c|c|}
\hline
& Exact & \begin{tabular}[c]{@{}l@{}}IBM Perth\\ Three Cycles\end{tabular} & \begin{tabular}[c]{@{}l@{}}IBM Perth\\ Five Cycles\end{tabular} &  \begin{tabular}[c]{@{}l@{}}Quantinuum H1-2\\ Five Cycles\end{tabular}\\ \hline
\,\(\ket{\psi_0}\)\, &\, -4\, &\, -4.0022(49)\, & \, -4.0006(42)\, &\, -3.9982(21)\, \\ \hline
\,\(\ket{\psi_1}\)\, &\, -1\, &\, -0.9829(56)\, & \,-0.9927(40)\, &\, -1.0083(39)\, \\ \hline
\,\(\ket{\psi_2}\)\, &\, 1\, &\, 1.0007(26)\, & \,1.0008(19)\, &\, 1.0028(22)\, \\ \hline
\,\(\ket{\psi_3}\)\, &\, 4\, &\, 4.0093(84)\, & \,3.9982(25)\, &\,  4.0036(17)\, \\ \hline
\end{tabular}
\caption{Energy eigenvalue estimates from the final pass of the Rodeo Algorithm with differing numbers of cycles on different devices. The exact values are provided for comparison. \label{tb:results}}
\end{table}

\paragraph*{Acknowledgements}
We are grateful for discussions with Joey Bonitati, Eli Chertkov, Michael Foss-Feig, Gabriel Given, David Hayes, Caleb Hicks, Brian Neyenhuis, and Danny Samuel. We also thank Jay Gambetta and IBM Q for the research accounts and access to hardware. We acknowledge financial support from the U.S. Department of Energy through grants DE-SC0021152, DE-SC0013365, DE-SC0023658, DE-SC0024586, DE-SC0023175 and the U.S. National Science Foundation through the grant PHY-2310620 and the Graduate Research Fellowship Program through the grant DGE-1848739.  This research used resources of the Oak Ridge Leadership Computing Facility, which is a DOE Office of Science User Facility supported under Contract DE-AC05-00OR22725.

\clearpage
\beginsupplement

\onecolumngrid
\section{Supplemental Materials}

\subsection{Success probabilities}
For each rodeo cycle, the random times $t$ are selected according to the normalized Gaussian distribution
\begin{equation}
p(t) = \tfrac{1}{\sqrt{2\pi}\sigma}e^{-\tfrac{t^2}{2\sigma^2}}.
\end{equation}
If our initial is an eigenstate with energy $E_{k}$, then the success probability for one cycle with target energy $E$ is
\begin{equation}
\frac{1}{\sqrt{2\pi}\sigma}\int dt e^{-\frac{t^2}{2\sigma^2}} \cos^2 [(E_{k}-E)\tfrac{t}{2}] =  \frac{1+e^{-(E_{k}-E)^2\sigma^2/2} }{2} \label{eq:continuous},
\end{equation}
and the probability of success for all $n$ cycles is
\begin{equation}
\left[ \frac{1+e^{-(E_{k}-E)^2\sigma^2/2} }{2} \right]^n.
\end{equation}
Let us now consider an initial state $\ket{\psi_I}$, which we can decompose into energy eigenstates
\begin{equation}
\ket{\psi_I} = \sum_k \braket{E_k|\psi_I}\ket{E_k}.
\end{equation}
With this initial state, the success probability after $n$ rodeo cycles will then equal
\begin{equation}
P_n(E) = \sum_k  \frac{\left[1+e^{-(E_k-E)^2\sigma^2/2}\right]^n \left| \braket{E_k|\psi_I} \right|^2 }{2^n}.
\label{eq:probability2}
\end{equation}

\subsection{Noisy Hamiltonian evolution}
Consider a simple noise model in which the circuit still implements Hamiltonian time evolution for each cycle, but noise in the Hamiltonian shifts the energy eigenvalues $E_k$ by some amount $\delta E_{k,i}$ for the $i$th cycle of the algorithm, where $\delta E_{k,i}$ is random shot-to-shot and is independently distributed between cycles. To first order in the noise, we assume that the eigenvectors $|E_k\rangle$ are unchanged. The noisy success probability can be obtained by averaging $P_n(E)$ over the distribution $p(\delta E_{k,n})$ of the noise:
\begin{equation}
P_{n,\text{noisy}}(E) = \sum_k \frac{|\langle E_k | \psi_I\rangle|^2 }{2^n} \prod_{i=1}^n \int d(\delta E_{k,i}) p(\delta E_{k,i}) \left[ 1 + e^{-(E_k + \delta E_{k,i} - E)^2 \sigma^2 / 2} \right] \,.
\end{equation}
In the simple case where $\delta E_{k,i}$ is Gaussian distributed with mean 0 and standard deviation $\varepsilon_k$, then $p(\delta E_{k,i}) = \frac{1}{\sqrt{2 \pi \varepsilon_k^2}} e^{-(\delta E_k)^2/(2 \varepsilon_k^2)}$ and
\begin{equation}
\int d(\delta E_{k,i}) p(\delta E_{k,i}) \left[ 1 + e^{-(E_k + \delta E_{k,i} - E)^2 \sigma^2 / 2} \right] = 1 + \frac{e^{-(E_k - E)^2 \sigma^2 / [2 (1 + \varepsilon_k^2 \sigma^2)]}}{\sqrt{1 + \varepsilon_k^2 \sigma^2}} \,.
\end{equation}
The centers of the peaks thereby remain at $E_k$. When the eigenenergies are well-separated, the peak height at $E = E_k$ is reduced by a factor of $\left[1 + \frac{1}{\sqrt{1 + \varepsilon_k^2 \sigma^2}}\right]^n / 2^n$, which gives exponential suppression in the number $n$ of cycles. When $\varepsilon_k^2 \sigma_k^2 \ll 1$, one obtains
\begin{equation}
P_{n,\text{noisy}}(E_k) \approx |\langle E_k | \psi_I\rangle|^2 \left[1 - \varepsilon_k^2 \sigma^2 / 4 \right]^n \,.
\end{equation}

One can also consider the case when $\delta E_{k,i} = \delta E_k$ is independent shot-to-shot, but the same for all cycles. This might, for instance, reflect slowly-varying coherent errors arising from calibration inaccuracies. Assuming sufficient sampling that averages over the error distribution,
\begin{equation}
P_{n,\text{noisy}}(E) = \sum_k \frac{|\langle E_k | \psi_I\rangle|^2 }{2^n} \int d(\delta E_{k}) p(\delta E_{k}) \left[ 1 + e^{-(E_k + \delta E_{k} - E)^2 \sigma^2 / 2} \right]^n \,.
\end{equation}
For a Gaussian error distribution, this will again leave the centers of the peaks unchanged. To find the suppression of the peak height, one can set $E = E_k$ and perform the binomial expansion and the integration. Assuming the peaks are well-separated, one obtains
\begin{equation}
P_{n,\text{noisy}}(E_k) \approx \frac{|\langle E_k | \psi_I\rangle|^2 }{2^n} \sum_m \binom{n}{m} \frac{1}{\sqrt{1 + m \varepsilon^2 \sigma^2}} \,.
\end{equation}
Assuming $m \varepsilon_k^2 \sigma^2 \ll 1$, then to lowest order one obtains
\begin{equation}
P_{n,\text{noisy}}(E_k) \approx |\langle E_k | \psi_I\rangle|^2 \left[ 1 - n \varepsilon_k^2 \sigma^2 / 4 \right] \,.
\end{equation}
In this case, one sees only a linear decay of the peak height in the number of rodeo cycles.

\subsection{Generalization to the $N$ qubit system}
We now consider the performance of the Rodeo Algorithm for the generalization of our Hamiltonian to a one-dimensional chain with $N$ qubits,
\begin{align}
H_{\rm obj} = \sum_{n=0}^{N-1} \left( c_1 X_n \otimes Z_{n+1} + c_2 Z_n \otimes X_{n+1} \right).
\label{Nqubits}
\end{align}
For notational convenience we assume that $N$ is even. We perform the time evolution using the second-order Trotter-Suzuki approximation with time step $dt$ [\citenamefont{Suzuki}, 1993, \citenamefont{{Hatano} and {Suzuki}}, 2005]. This involves alternating the time evolution of the even-$n$ and odd-$n$ terms in Eq.~(\ref{Nqubits}). If the $N$-qubit system is evolved for time duration $\sigma$, then the total number of two-qubit exponentials required is $N\sigma/dt+N/2$. If the controlled time evolution is implemented in a straightforward manner [\citenamefont{Anis et~al.}, 2021], the number of two-qubit entangling gates is $20N\sigma/dt+10N$. If instead we use the controlled reversal gate $\prod_{n\;{\rm even}} C_{Y_n}$, where the product of controlled-$Y$ gates is over all even qubits, then only $N\sigma/dt+2N$ two-qubit entangling gates are required.   This is a roughly twenty-fold reduction in the number of gates. If we use the values $\sigma = 6$ and $dt = 0.2$, we arrive at $32N$ two-qubit entangling gates per rodeo cycle. In order for the peak probability to rise above the background value of $\tfrac{1}{2^n}$, the error rate per two-qubit entangling gate should be less than about $\tfrac{1}{64N}$. For an error rate of $0.3\%$ we get $N \le 5$, and for an error rate of $0.1\%$ we get $N \le 16$. Without controlled reversal gates, the corresponding values of $N$ are reduced by a factor of twenty. This shows quite clearly the utility of the controlled reversal gates. We should mention that larger values of $N$ are possible if high statistics with many Gaussian random times are used to make the background signal a smooth function of energy so that smaller peaks are also visible.

In addition to these constraints, we also need that the overlap between the initial state and eigenstate of interest is not much smaller than $1$. One simple way to increase the initial state overlap is by using quantum adiabatic evolution as a preconditioner. This technique is discussed in Ref.~[\citenamefont{Choi et~al.}, 2021].

\begin{thebibliography}{46}
\expandafter\ifx\csname natexlab\endcsname\relax\def\natexlab#1{#1}\fi
\expandafter\ifx\csname bibnamefont\endcsname\relax
\def\bibnamefont#1{#1}\fi
\expandafter\ifx\csname bibfnamefont\endcsname\relax
\def\bibfnamefont#1{#1}\fi
\expandafter\ifx\csname citenamefont\endcsname\relax
\def\citenamefont#1{#1}\fi
\expandafter\ifx\csname url\endcsname\relax
\def\url#1{\texttt{#1}}\fi
\expandafter\ifx\csname urlprefix\endcsname\relax\def\urlprefix{URL }\fi
\providecommand{\bibinfo}[2]{#2}
\providecommand{\eprint}[2][]{\url{#2}}

[\citenamefont{Choi et~al.}, 2021]
\bibinfo{author}{\bibfnamefont{K.}~\bibnamefont{Choi}},
\bibinfo{author}{\bibfnamefont{D.}~\bibnamefont{Lee}},
\bibinfo{author}{\bibfnamefont{J.}~\bibnamefont{Bonitati}},
\bibinfo{author}{\bibfnamefont{Z.}~\bibnamefont{Qian}}, \bibnamefont{and}
\bibinfo{author}{\bibfnamefont{J.}~\bibnamefont{Watkins}},
\bibinfo{journal}{Phys. Rev. Lett.} \textbf{\bibinfo{volume}{127}},
\bibinfo{pages}{040505} (\bibinfo{year}{2021}), \eprint{2009.04092}.

[\citenamefont{Childs and Wiebe}, 2012]
\bibinfo{author}{\bibfnamefont{A.~M.} \bibnamefont{Childs}} \bibnamefont{and}
\bibinfo{author}{\bibfnamefont{N.}~\bibnamefont{Wiebe}},
\bibinfo{journal}{Quantum Info. Comput.} \textbf{\bibinfo{volume}{12}},
\bibinfo{pages}{901–924} (\bibinfo{year}{2012}), ISSN
\bibinfo{issn}{1533-7146}.

[\citenamefont{Berry et~al.}, 2014]
\bibinfo{author}{\bibfnamefont{D.~W.} \bibnamefont{Berry}},
\bibinfo{author}{\bibfnamefont{A.~M.} \bibnamefont{Childs}},
\bibinfo{author}{\bibfnamefont{R.}~\bibnamefont{Cleve}},
\bibinfo{author}{\bibfnamefont{R.}~\bibnamefont{Kothari}}, \bibnamefont{and}
\bibinfo{author}{\bibfnamefont{R.~D.} \bibnamefont{Somma}}, in
\emph{\bibinfo{booktitle}{Proceedings of the forty-sixth annual ACM symposium
on Theory of computing}} (\bibinfo{year}{2014}), pp.
\bibinfo{pages}{283--292}.

[\citenamefont{Low and Chuang}, 2019]
\bibinfo{author}{\bibfnamefont{G.~H.} \bibnamefont{Low}} \bibnamefont{and}
\bibinfo{author}{\bibfnamefont{I.~L.} \bibnamefont{Chuang}},
\bibinfo{journal}{Quantum} \textbf{\bibinfo{volume}{3}}, \bibinfo{pages}{163}

[\citenamefont{Gily{\'e}n et~al.}, 2019]
\bibinfo{author}{\bibfnamefont{A.}~\bibnamefont{Gily{\'e}n}},
\bibinfo{author}{\bibfnamefont{Y.}~\bibnamefont{Su}},
\bibinfo{author}{\bibfnamefont{G.~H.} \bibnamefont{Low}}, \bibnamefont{and}
\bibinfo{author}{\bibfnamefont{N.}~\bibnamefont{Wiebe}}, in
\emph{\bibinfo{booktitle}{Proceedings of the 51st Annual ACM SIGACT Symposium
on Theory of Computing}} (\bibinfo{year}{2019}), pp.
\bibinfo{pages}{193--204}.

[\citenamefont{Khatri et~al.}, 2019]
\bibinfo{author}{\bibfnamefont{S.}~\bibnamefont{Khatri}},
\bibinfo{author}{\bibfnamefont{R.}~\bibnamefont{LaRose}},
\bibinfo{author}{\bibfnamefont{A.}~\bibnamefont{Poremba}},
\bibinfo{author}{\bibfnamefont{L.}~\bibnamefont{Cincio}},
\bibinfo{author}{\bibfnamefont{A.~T.} \bibnamefont{Sornborger}},
\bibnamefont{and} \bibinfo{author}{\bibfnamefont{P.~J.} \bibnamefont{Coles}},
\bibinfo{journal}{{Quantum}} \textbf{\bibinfo{volume}{3}},
\bibinfo{pages}{140} (\bibinfo{year}{2019}), ISSN \bibinfo{issn}{2521-327X},
\urlprefix\url{https://doi.org/10.22331/q-2019-05-13-140}.

[\citenamefont{Kamaka}, 2020]
\bibinfo{author}{\bibfnamefont{B.~K.} \bibnamefont{Kamaka}}

[\citenamefont{{Yordanov} et~al.}, 2020]
\bibinfo{author}{\bibfnamefont{Y.~S.} \bibnamefont{{Yordanov}}},
\bibinfo{author}{\bibfnamefont{D.~R.~M.} \bibnamefont{{Arvidsson-Shukur}}},
\bibnamefont{and} \bibinfo{author}{\bibfnamefont{C.~H.~W.}
\bibnamefont{{Barnes}}}, \bibinfo{journal}{\pra}
\textbf{\bibinfo{volume}{102}}, \bibinfo{eid}{062612} (\bibinfo{year}{2020}),
\eprint{2005.14475}.

[\citenamefont{Nash et~al.}, 2020]
\bibinfo{author}{\bibfnamefont{B.}~\bibnamefont{Nash}},
\bibinfo{author}{\bibfnamefont{V.}~\bibnamefont{Gheorghiu}},
\bibnamefont{and} \bibinfo{author}{\bibfnamefont{M.}~\bibnamefont{Mosca}},
\bibinfo{journal}{Quantum Science and Technology}
\textbf{\bibinfo{volume}{5}}, \bibinfo{pages}{025010} (\bibinfo{year}{2020}),
\urlprefix\url{https://dx.doi.org/10.1088/2058-9565/ab79b1}.

[\citenamefont{Younis and Iancu}, 2022]
\bibinfo{author}{\bibfnamefont{E.}~\bibnamefont{Younis}} \bibnamefont{and}
\bibinfo{author}{\bibfnamefont{C.}~\bibnamefont{Iancu}}, in
\emph{\bibinfo{booktitle}{2022 IEEE International Conference on Quantum
Computing and Engineering (QCE)}} (\bibinfo{year}{2022}), pp.
\bibinfo{pages}{465--475}.

[\citenamefont{Dobbs et~al.}, 2022]
\bibinfo{author}{\bibfnamefont{E.~E.} \bibnamefont{Dobbs}},
\bibinfo{author}{\bibfnamefont{J.~S.} \bibnamefont{Friedman}},
\bibnamefont{and} \bibinfo{author}{\bibfnamefont{A.}~\bibnamefont{Paler}}
(\bibinfo{year}{2022}), \eprint{2206.04990}.

[\citenamefont{Weiden et~al.}, 2023]
\bibinfo{author}{\bibfnamefont{M.}~\bibnamefont{Weiden}},
\bibinfo{author}{\bibnamefont{Younis}},
\bibinfo{author}{\bibfnamefont{J.}~\bibnamefont{Kalloor}},
\bibinfo{author}{\bibfnamefont{J.}~\bibnamefont{Kubiatowicz}},
\bibnamefont{and} \bibinfo{author}{\bibfnamefont{C.}~\bibnamefont{Iancu}}
(\bibinfo{year}{2023}), \eprint{2306.05622}.

[\citenamefont{{Hao Low} and {Chuang}}, 2016]
\bibinfo{author}{\bibfnamefont{G.}~\bibnamefont{{Hao Low}}} \bibnamefont{and}
\bibinfo{author}{\bibfnamefont{I.~L.} \bibnamefont{{Chuang}}},
\bibinfo{journal}{arXiv e-prints} \bibinfo{eid}{arXiv:1610.06546}
(\bibinfo{year}{2016}), \eprint{1610.06546}.

[\citenamefont{Kitaev}, 1995]
\bibinfo{author}{\bibfnamefont{A.~Y.} \bibnamefont{Kitaev}}
(\bibinfo{year}{1995}), \eprint{quant-ph/9511026}.

[\citenamefont{{Abrams} and {Lloyd}}, 1999]
\bibinfo{author}{\bibfnamefont{D.~S.} \bibnamefont{{Abrams}}} \bibnamefont{and}
\bibinfo{author}{\bibfnamefont{S.}~\bibnamefont{{Lloyd}}},
\bibinfo{journal}{\prl} \textbf{\bibinfo{volume}{83}}, \bibinfo{pages}{5162}
(\bibinfo{year}{1999}), \eprint{quant-ph/9807070}.

[\citenamefont{{Cleve} et~al.}, 1998]
\bibinfo{author}{\bibfnamefont{R.}~\bibnamefont{{Cleve}}},
\bibinfo{author}{\bibfnamefont{A.}~\bibnamefont{{Ekert}}},
\bibinfo{author}{\bibfnamefont{C.}~\bibnamefont{{Macchiavello}}},
\bibnamefont{and} \bibinfo{author}{\bibfnamefont{M.}~\bibnamefont{{Mosca}}},
\bibinfo{journal}{Proceedings of the Royal Society of London Series A}
\textbf{\bibinfo{volume}{454}}, \bibinfo{pages}{339} (\bibinfo{year}{1998}),
\eprint{quant-ph/9708016}.

[\citenamefont{{Svore} et~al.}, 2013]
\bibinfo{author}{\bibfnamefont{K.~M.} \bibnamefont{{Svore}}},
\bibinfo{author}{\bibfnamefont{M.~B.} \bibnamefont{{Hastings}}},
\bibnamefont{and}
\bibinfo{author}{\bibfnamefont{M.}~\bibnamefont{{Freedman}}},
\bibinfo{journal}{Quant. Inf. Comp.} \textbf{\bibinfo{volume}{14}},
\bibinfo{pages}{306} (\bibinfo{year}{2013}), \eprint{1304.0741}.

[\citenamefont{Motta et~al.}, 2020]
\bibinfo{author}{\bibfnamefont{M.}~\bibnamefont{Motta}},
\bibinfo{author}{\bibfnamefont{C.}~\bibnamefont{Sun}},
\bibinfo{author}{\bibfnamefont{A.~T.} \bibnamefont{Tan}},
\bibinfo{author}{\bibfnamefont{M.~J.} \bibnamefont{O’Rourke}},
\bibinfo{author}{\bibfnamefont{E.}~\bibnamefont{Ye}},
\bibinfo{author}{\bibfnamefont{A.~J.} \bibnamefont{Minnich}},
\bibinfo{author}{\bibfnamefont{F.~G.} \bibnamefont{Brand{\~a}o}},
\bibnamefont{and} \bibinfo{author}{\bibfnamefont{G.~K.-L.}
\bibnamefont{Chan}}, \bibinfo{journal}{Nature Physics}
\textbf{\bibinfo{volume}{16}}, \bibinfo{pages}{205} (\bibinfo{year}{2020}).

[\citenamefont{Turro et~al.}, 2022]
\bibinfo{author}{\bibfnamefont{F.}~\bibnamefont{Turro}},
\bibinfo{author}{\bibfnamefont{A.}~\bibnamefont{Roggero}},
\bibinfo{author}{\bibfnamefont{V.}~\bibnamefont{Amitrano}},
\bibinfo{author}{\bibfnamefont{P.}~\bibnamefont{Luchi}},
\bibinfo{author}{\bibfnamefont{K.~A.} \bibnamefont{Wendt}},
\bibinfo{author}{\bibfnamefont{J.~L.} \bibnamefont{DuBois}},
\bibinfo{author}{\bibfnamefont{S.}~\bibnamefont{Quaglioni}},
\bibnamefont{and} \bibinfo{author}{\bibfnamefont{F.}~\bibnamefont{Pederiva}},
\bibinfo{journal}{Phys. Rev. A} \textbf{\bibinfo{volume}{105}},
\bibinfo{pages}{022440} (\bibinfo{year}{2022}), \eprint{2102.12260}.

[\citenamefont{{Parrish} and {McMahon}}, 2019]
\bibinfo{author}{\bibfnamefont{R.~M.} \bibnamefont{{Parrish}}}
\bibnamefont{and} \bibinfo{author}{\bibfnamefont{P.~L.}
\bibnamefont{{McMahon}}}, \bibinfo{journal}{arXiv e-prints}
\bibinfo{eid}{arXiv:1909.08925} (\bibinfo{year}{2019}), \eprint{1909.08925}.

[\citenamefont{{Francis} et~al.}, 2022]
\bibinfo{author}{\bibfnamefont{A.}~\bibnamefont{{Francis}}},
\bibinfo{author}{\bibfnamefont{A.~A.} \bibnamefont{{Agrawal}}},
\bibinfo{author}{\bibfnamefont{J.~H.} \bibnamefont{{Howard}}},
\bibinfo{author}{\bibfnamefont{E.}~\bibnamefont{{K{\"o}kc{\"u}}}},
\bibnamefont{and} \bibinfo{author}{\bibfnamefont{A.~F.}
\bibnamefont{{Kemper}}}, \bibinfo{journal}{arXiv e-prints}
\bibinfo{eid}{arXiv:2209.10571} (\bibinfo{year}{2022}), \eprint{2209.10571}.

[\citenamefont{{Yang} et~al.}, 2022]
\bibinfo{author}{\bibfnamefont{Y.}~\bibnamefont{{Yang}}},
\bibinfo{author}{\bibfnamefont{Z.}~\bibnamefont{{Zhang}}},
\bibinfo{author}{\bibfnamefont{X.}~\bibnamefont{{Xu}}},
\bibinfo{author}{\bibfnamefont{B.}~\bibnamefont{{Lu}}}, \bibnamefont{and}
\bibinfo{author}{\bibfnamefont{Y.}~\bibnamefont{{Li}}},
\bibinfo{journal}{arXiv e-prints} \bibinfo{eid}{arXiv:2211.14854}
(\bibinfo{year}{2022}), \eprint{2211.14854}.

[\citenamefont{{Cortes} and {Gray}}, 2022]
\bibinfo{author}{\bibfnamefont{C.~L.} \bibnamefont{{Cortes}}} \bibnamefont{and}
\bibinfo{author}{\bibfnamefont{S.~K.} \bibnamefont{{Gray}}},
\bibinfo{journal}{\pra} \textbf{\bibinfo{volume}{105}}, \bibinfo{eid}{022417}
(\bibinfo{year}{2022}), \eprint{2109.06868}.

[\citenamefont{Mejuto-Zaera and Kemper}, 2023]
\bibinfo{author}{\bibfnamefont{C.}~\bibnamefont{Mejuto-Zaera}}
\bibnamefont{and} \bibinfo{author}{\bibfnamefont{A.~F.} \bibnamefont{Kemper}}
(\bibinfo{year}{2023}), \eprint{2305.00060}.

[\citenamefont{Farhi et~al.}, 2014]
\bibinfo{author}{\bibfnamefont{E.}~\bibnamefont{Farhi}},
\bibinfo{author}{\bibfnamefont{J.}~\bibnamefont{Goldstone}},
\bibnamefont{and} \bibinfo{author}{\bibfnamefont{S.}~\bibnamefont{Gutmann}},
\bibinfo{journal}{arXiv preprint arXiv:1411.4028}  (\bibinfo{year}{2014}).

[\citenamefont{Farhi and Harrow}, 2016]
\bibinfo{author}{\bibfnamefont{E.}~\bibnamefont{Farhi}} \bibnamefont{and}
\bibinfo{author}{\bibfnamefont{A.~W.} \bibnamefont{Harrow}},
\bibinfo{journal}{arXiv preprint arXiv:1602.07674}  (\bibinfo{year}{2016}).

[\citenamefont{Zhou et~al.}, 2020]
\bibinfo{author}{\bibfnamefont{L.}~\bibnamefont{Zhou}},
\bibinfo{author}{\bibfnamefont{S.-T.} \bibnamefont{Wang}},
\bibinfo{author}{\bibfnamefont{S.}~\bibnamefont{Choi}},
\bibinfo{author}{\bibfnamefont{H.}~\bibnamefont{Pichler}}, \bibnamefont{and}
\bibinfo{author}{\bibfnamefont{M.~D.} \bibnamefont{Lukin}},
\bibinfo{journal}{Physical Review X} \textbf{\bibinfo{volume}{10}},
\bibinfo{pages}{021067} (\bibinfo{year}{2020}).

[\citenamefont{Satula et~al.}, 2010]
\bibinfo{author}{\bibfnamefont{W.}~\bibnamefont{Satula}},
\bibinfo{author}{\bibfnamefont{J.}~\bibnamefont{Dobaczewski}},
\bibinfo{author}{\bibfnamefont{W.}~\bibnamefont{Nazarewicz}},
\bibnamefont{and} \bibinfo{author}{\bibfnamefont{M.}~\bibnamefont{Rafalski}},
\bibinfo{journal}{Phys. Rev. C} \textbf{\bibinfo{volume}{81}},
\bibinfo{pages}{054310} (\bibinfo{year}{2010}), \eprint{0912.4381}.

[\citenamefont{Yao et~al.}, 2020]
\bibinfo{author}{\bibfnamefont{J.~M.} \bibnamefont{Yao}},
\bibinfo{author}{\bibfnamefont{B.}~\bibnamefont{Bally}},
\bibinfo{author}{\bibfnamefont{J.}~\bibnamefont{Engel}},
\bibinfo{author}{\bibfnamefont{R.}~\bibnamefont{Wirth}},
\bibinfo{author}{\bibfnamefont{T.~R.} \bibnamefont{Rodr\'\i{}guez}},
\bibnamefont{and} \bibinfo{author}{\bibfnamefont{H.}~\bibnamefont{Hergert}},
\bibinfo{journal}{Phys. Rev. Lett.} \textbf{\bibinfo{volume}{124}},
\bibinfo{pages}{232501} (\bibinfo{year}{2020}), \eprint{1908.05424}.

[\citenamefont{Sheikh et~al.}, 2021]
\bibinfo{author}{\bibfnamefont{J.~A.} \bibnamefont{Sheikh}},
\bibinfo{author}{\bibfnamefont{J.}~\bibnamefont{Dobaczewski}},
\bibinfo{author}{\bibfnamefont{P.}~\bibnamefont{Ring}},
\bibinfo{author}{\bibfnamefont{L.~M.} \bibnamefont{Robledo}},
\bibnamefont{and}
\bibinfo{author}{\bibfnamefont{C.}~\bibnamefont{Yannouleas}},
\bibinfo{journal}{J. Phys. G} \textbf{\bibinfo{volume}{48}},
\bibinfo{pages}{123001} (\bibinfo{year}{2021}), \eprint{1901.06992}.

[\citenamefont{Frosini et~al.}, 2022]
\bibinfo{author}{\bibfnamefont{M.}~\bibnamefont{Frosini}},
\bibinfo{author}{\bibfnamefont{T.}~\bibnamefont{Duguet}},
\bibinfo{author}{\bibfnamefont{J.-P.} \bibnamefont{Ebran}},
\bibinfo{author}{\bibfnamefont{B.}~\bibnamefont{Bally}},
\bibinfo{author}{\bibfnamefont{T.}~\bibnamefont{Mongelli}},
\bibinfo{author}{\bibfnamefont{T.~R.} \bibnamefont{Rodr\'\i{}guez}},
\bibinfo{author}{\bibfnamefont{R.}~\bibnamefont{Roth}}, \bibnamefont{and}
\bibinfo{author}{\bibfnamefont{V.}~\bibnamefont{Som\`a}},
\bibinfo{journal}{Eur. Phys. J. A} \textbf{\bibinfo{volume}{58}},
\bibinfo{pages}{63} (\bibinfo{year}{2022}), \eprint{2111.00797}.

[\citenamefont{Ruiz~Guzman and Lacroix}, 2022]
\bibinfo{author}{\bibfnamefont{E.~A.} \bibnamefont{Ruiz~Guzman}}
\bibnamefont{and} \bibinfo{author}{\bibfnamefont{D.}~\bibnamefont{Lacroix}},
\bibinfo{journal}{Phys. Rev. C} \textbf{\bibinfo{volume}{105}},
\bibinfo{pages}{024324} (\bibinfo{year}{2022}), \eprint{2111.13080}.

[\citenamefont{Lacroix et~al.}, 2023]
\bibinfo{author}{\bibfnamefont{D.}~\bibnamefont{Lacroix}},
\bibinfo{author}{\bibfnamefont{E.~A.} \bibnamefont{Ruiz~Guzman}},
\bibnamefont{and} \bibinfo{author}{\bibfnamefont{P.}~\bibnamefont{Siwach}},
\bibinfo{journal}{Eur. Phys. J. A} \textbf{\bibinfo{volume}{59}},
\bibinfo{pages}{3} (\bibinfo{year}{2023}), \eprint{2208.11567}.

[\citenamefont{Frame et~al.}, 2018]
\bibinfo{author}{\bibfnamefont{D.}~\bibnamefont{Frame}},
\bibinfo{author}{\bibfnamefont{R.}~\bibnamefont{He}},
\bibinfo{author}{\bibfnamefont{I.}~\bibnamefont{Ipsen}},
\bibinfo{author}{\bibfnamefont{D.}~\bibnamefont{Lee}},
\bibinfo{author}{\bibfnamefont{D.}~\bibnamefont{Lee}}, \bibnamefont{and}
\bibinfo{author}{\bibfnamefont{E.}~\bibnamefont{Rrapaj}},
\bibinfo{journal}{Phys. Rev. Lett.} \textbf{\bibinfo{volume}{121}},
\bibinfo{pages}{032501} (\bibinfo{year}{2018}), \eprint{1711.07090}.

[\citenamefont{Epperly et~al.}, 2022]
\bibinfo{author}{\bibfnamefont{E.~N.} \bibnamefont{Epperly}},
\bibinfo{author}{\bibfnamefont{L.}~\bibnamefont{Lin}}, \bibnamefont{and}
\bibinfo{author}{\bibfnamefont{Y.}~\bibnamefont{Nakatsukasa}},
\bibinfo{journal}{SIAM J. Matrix Anal. Appl.} \textbf{\bibinfo{volume}{43}},
\bibinfo{pages}{1263} (\bibinfo{year}{2022}), \eprint{2110.07492}.

[\citenamefont{Sarkar et~al.}, 2023]
\bibinfo{author}{\bibfnamefont{A.}~\bibnamefont{Sarkar}},
\bibinfo{author}{\bibfnamefont{D.}~\bibnamefont{Lee}}, \bibnamefont{and}
\bibinfo{author}{\bibfnamefont{U.-G.} \bibnamefont{Mei\ss{}ner}},
\bibinfo{journal}{Phys. Rev. Lett.} \textbf{\bibinfo{volume}{131}},
\bibinfo{pages}{242503} (\bibinfo{year}{2023}), \eprint{2306.11439}.

[\citenamefont{Duguet et~al.}, 2023]
\bibinfo{author}{\bibfnamefont{T.}~\bibnamefont{Duguet}},
\bibinfo{author}{\bibfnamefont{A.}~\bibnamefont{Ekstr\"om}},
\bibinfo{author}{\bibfnamefont{R.~J.} \bibnamefont{Furnstahl}},
\bibinfo{author}{\bibfnamefont{S.}~\bibnamefont{K\"onig}}, \bibnamefont{and}
\bibinfo{author}{\bibfnamefont{D.}~\bibnamefont{Lee}} (\bibinfo{year}{2023}),
\eprint{2310.19419}.

[\citenamefont{Smith et~al.}, 2019]
\bibinfo{author}{\bibfnamefont{A.}~\bibnamefont{Smith}},
\bibinfo{author}{\bibfnamefont{M.~S.} \bibnamefont{Kim}},
\bibinfo{author}{\bibfnamefont{F.}~\bibnamefont{Pollmann}}, \bibnamefont{and}
\bibinfo{author}{\bibfnamefont{J.}~\bibnamefont{Knolle}},
\bibinfo{journal}{npj Quantum Information} \textbf{\bibinfo{volume}{5}},
\bibinfo{pages}{106} (\bibinfo{year}{2019}), ISSN \bibinfo{issn}{2056-6387},
\urlprefix\url{https://doi.org/10.1038/s41534-019-0217-0}.

[\citenamefont{Siwach and Lacroix}, 2021]
\bibinfo{author}{\bibfnamefont{P.}~\bibnamefont{Siwach}} \bibnamefont{and}
\bibinfo{author}{\bibfnamefont{D.}~\bibnamefont{Lacroix}},
\bibinfo{journal}{Phys. Rev. A} \textbf{\bibinfo{volume}{104}},
\bibinfo{pages}{062435} (\bibinfo{year}{2021}), \eprint{2106.10867}.

[\citenamefont{{Farhi} et~al.}, 2000]
\bibinfo{author}{\bibfnamefont{E.}~\bibnamefont{{Farhi}}},
\bibinfo{author}{\bibfnamefont{J.}~\bibnamefont{{Goldstone}}},
\bibinfo{author}{\bibfnamefont{S.}~\bibnamefont{{Gutmann}}},
\bibnamefont{and} \bibinfo{author}{\bibfnamefont{M.}~\bibnamefont{{Sipser}}}
(\bibinfo{year}{2000}), \eprint{quant-ph/0001106}.

[\citenamefont{{Wiebe} and {Babcock}}, 2012]
\bibinfo{author}{\bibfnamefont{N.}~\bibnamefont{{Wiebe}}} \bibnamefont{and}
\bibinfo{author}{\bibfnamefont{N.~S.} \bibnamefont{{Babcock}}},
\bibinfo{journal}{New Journal of Physics} \textbf{\bibinfo{volume}{14}},
\bibinfo{eid}{013024} (\bibinfo{year}{2012}).

[\citenamefont{Qian et~al.}, 2021]
\bibinfo{author}{\bibfnamefont{Z.}~\bibnamefont{Qian}},
\bibinfo{author}{\bibfnamefont{J.}~\bibnamefont{Watkins}},
\bibinfo{author}{\bibfnamefont{G.}~\bibnamefont{Given}},
\bibinfo{author}{\bibfnamefont{J.}~\bibnamefont{Bonitati}},
\bibinfo{author}{\bibfnamefont{K.}~\bibnamefont{Choi}}, \bibnamefont{and}
\bibinfo{author}{\bibfnamefont{D.}~\bibnamefont{Lee}} (\bibinfo{year}{2021}),
\eprint{2110.07747}.

[\citenamefont{Anis et~al.}, 2021]
\bibinfo{author}{\bibfnamefont{M.~S.} \bibnamefont{Anis}},
\bibinfo{author}{\bibnamefont{Abby-Mitchell}},
\bibinfo{author}{\bibfnamefont{H.}~\bibnamefont{Abraham}},
\bibinfo{author}{\bibnamefont{AduOffei}},
\bibinfo{author}{\bibfnamefont{R.}~\bibnamefont{Agarwal}},
\bibinfo{author}{\bibfnamefont{G.}~\bibnamefont{Agliardi}},
\bibinfo{author}{\bibfnamefont{M.}~\bibnamefont{Aharoni}},
\bibinfo{author}{\bibfnamefont{V.}~\bibnamefont{Ajith}},
\bibinfo{author}{\bibfnamefont{I.~Y.} \bibnamefont{Akhalwaya}},
\bibinfo{author}{\bibfnamefont{G.}~\bibnamefont{Aleksandrowicz}},
\bibnamefont{et~al.}, \emph{\bibinfo{title}{Qiskit: An open-source framework
for quantum computing}} (\bibinfo{year}{2021}).

[\citenamefont{Suzuki}, 1993]
\bibinfo{author}{\bibfnamefont{M.}~\bibnamefont{Suzuki}},
\bibinfo{journal}{Proceedings of the Japan Academy, Series B}
\textbf{\bibinfo{volume}{69}}, \bibinfo{pages}{161} (\bibinfo{year}{1993}).

[\citenamefont{{Hatano} and {Suzuki}}, 2005]
\bibinfo{author}{\bibfnamefont{N.}~\bibnamefont{{Hatano}}} \bibnamefont{and}
\bibinfo{author}{\bibfnamefont{M.}~\bibnamefont{{Suzuki}}}, in
\emph{\bibinfo{booktitle}{Lecture Notes in Physics, Berlin Springer Verlag}},
edited by \bibinfo{editor}{\bibfnamefont{A.}~\bibnamefont{{Das}}}
\bibnamefont{and} \bibinfo{editor}{\bibfnamefont{B.~K.}
\bibnamefont{{Chakrabarti}}} (\bibinfo{year}{2005}), vol.
\bibinfo{volume}{679}, p.~\bibinfo{pages}{37}.

[46]
\bibinfo{author}{\bibnamefont{Quantinuum}}, \emph{\bibinfo{title}{{Quantinuum
H1-2}}} (\bibinfo{year}{March 9, 2022}),
\urlprefix\url{https://www.quantinuum.com/}.

\end{thebibliography}

\bibliographystyle{apsrev}

\end{document}


* Pascanu et al. (2013) Razvan Pascanu, Tomás Mikolov, and Yoshua Bengio. 2013.  On the difficulty of training recurrent neural networks. In _Proceedings of the 30th International Conference on Machine Learning, ICML_ , Vol. 28. JMLR.org, 1310–1318.
  * Paster et al. (2022) Keiran Paster, Sheila A. McIlraith, and Jimmy Ba. 2022\.  You Can’t Count on Luck: Why Decision Transformers and RvS Fail in Stochastic Environments.  (2022).
  * Petit et al. (2021) Olivier Petit, Nicolas Thome, Clément Rambour, Loic Themyr, Toby Collins, and Luc Soler. 2021\.  U-Net Transformer: Self and Cross Attention for Medical Image Segmentation. In _Machine Learning in Medical Imaging - 12th International Workshop, MLMI, MICCAI_ , Vol. 12966. Springer, 267–276.
  * Putterman et al. (2021) Aaron Putterman, Kevin Lu, Igor Mordatch, and P. Abbeel. 2021\.  Pretraining for Language-Conditioned Imitation with Transformers.
  * Qu et al. (2022) Jia Qu, Shotaro Miwa, and Yukiyasu Domae. 2022.  Interpretable Navigation Agents Using Attention-Augmented Memory. In _IEEE International Conference on Systems, Man, and Cybernetics, SMC 2022, Prague, Czech Republic, October 9-12, 2022_. IEEE, 2575–2582.
  * Radford et al. (2018) Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever, et al. 2018\.  Improving language understanding by generative pre-training.  (2018).
  * Rakelly et al. (2019) Kate Rakelly, Aurick Zhou, Chelsea Finn, Sergey Levine, and Deirdre Quillen. 2019.  Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variables. In _Proceedings of the 36th International Conference on Machine Learning, ICML_ , Vol. 97. PMLR, 5331–5340.
  * Ramachandram and Taylor (2017) Dhanesh Ramachandram and Graham W. Taylor. 2017.  Deep Multimodal Learning: A Survey on Recent Advances and Trends.  _IEEE Signal Process. Mag._ 34, 6 (2017), 96–108.
  * Ren et al. (2021) Hongyu Ren, Hanjun Dai, Zihang Dai, Mengjiao Yang, Jure Leskovec, Dale Schuurmans, and Bo Dai. 2021.  Combiner: Full Attention Transformer with Sparse Computation Cost. In _NeurIPS_. 22470–22482.
  * Ribeiro et al. (2020) Antônio H. Ribeiro, Koen Tiels, Luis Antonio Aguirre, and Thomas B. Schön. 2020.  Beyond exploding and vanishing gradients: analysing RNN training using attractors and smoothness. In _The 23rd International Conference on Artificial Intelligence and Statistics, AISTATS_ , Vol. 108. PMLR, 2370–2380.
  * Robine et al. (2023) Jan Robine, Marc Höftmann, Tobias Uelwer, and Stefan Harmeling. 2023.  Transformer-based World Models Are Happy With 100k Interactions.  _CoRR_ abs/2303.07109 (2023).
  * Savva et al. (2019) Manolis Savva, Jitendra Malik, Devi Parikh, Dhruv Batra, Abhishek Kadian, Oleksandr Maksymets, Yili Zhao, Erik Wijmans, Bhavana Jain, Julian Straub, Jia Liu, and Vladlen Koltun. 2019\.  Habitat: A Platform for Embodied AI Research. In _IEEE/CVF International Conference on Computer Vision, ICCV_. IEEE, 9338–9346.
  * Schmidhuber (2019) Jürgen Schmidhuber. 2019\.  Reinforcement Learning Upside Down: Don’t Predict Rewards - Just Map Them to Actions.  _CoRR_ abs/1912.02875 (2019).
  * Schulman et al. (2017) John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017.  Proximal Policy Optimization Algorithms.  _CoRR_ abs/1707.06347 (2017).
  * Seo et al. (2019) Minah Seo, Luiz Felipe Vecchietti, Sangkeum Lee, and Dongsoo Har. 2019.  Rewards Prediction-Based Credit Assignment for Reinforcement Learning With Sparse Binary Rewards.  _IEEE Access_ 7 (2019), 118776–118791.
  * Seo et al. (2022) Younggyo Seo, Danijar Hafner, Hao Liu, Fangchen Liu, Stephen James, Kimin Lee, and Pieter Abbeel. 2022.  Masked World Models for Visual Control. In _Conference on Robot Learning, CoRL_ , Vol. 205. PMLR, 1332–1344.
  * Serrano and Smith (2019) Sofia Serrano and Noah A. Smith. 2019.  Is Attention Interpretable?. In _Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL_. Association for Computational Linguistics, 2931–2951.
  * Shang et al. (2022) Jinghuan Shang, Kumara Kahatapitiya, Xiang Li, and Michael S. Ryoo. 2022.  StARformer: Transformer with State-Action-Reward Representations for Visual Reinforcement Learning. In _Computer Vision - ECCV - 17th European Conference_ , Vol. 13699. Springer, 462–479.
  * Shao et al. (2019a) Kun Shao, Zhentao Tang, Yuanheng Zhu, Nannan Li, and Dongbin Zhao. 2019a.  A Survey of Deep Reinforcement Learning in Video Games.  _CoRR_ abs/1912.10944 (2019).
  * Shao et al. (2019b) Kun Shao, Yuanheng Zhu, and Dongbin Zhao. 2019b.  StarCraft Micromanagement With Reinforcement Learning and Curriculum Transfer Learning.  _IEEE Trans. Emerg. Top. Comput. Intell._ 3, 1 (2019), 73–84.
  * Sharma et al. (2021) Piyush K. Sharma, Rolando Fernandez, Erin G. Zaroukian, Michael R. Dorothy, Anjon Basak, and Derrik E. Asher. 2021.  Survey of Recent Multi-Agent Reinforcement Learning Algorithms Utilizing Centralized Training.  _CoRR_ abs/2107.14316 (2021).
  * Shawki et al. (2021) N Shawki, R Rodriguez Nunez, I Obeid, and J Picone. 2021\.  On automating hyperparameter optimization for deep learning applications. In _2021 IEEE Signal Processing in Medicine and Biology Symposium (SPMB)_. IEEE, 1–7.
  * Shi et al. (2021) Tianyu Shi, Dong Chen, Kaian Chen, and Zhaojian Li. 2021\.  Offline Reinforcement Learning for Autonomous Driving with Safety and Exploration Enhancement.  _CoRR_ abs/2110.07067 (2021).
  * Smith (2022) Carson Smith. 2022\.  _Attention-Based Learning for Combinatorial Optimization_.  Ph. D. Dissertation. Massachusetts Institute of Technology.
  * Snoek et al. (2015) Jasper Snoek, Oren Rippel, Kevin Swersky, Ryan Kiros, Nadathur Satish, Narayanan Sundaram, Md. Mostofa Ali Patwary, Prabhat, and Ryan P. Adams. 2015.  Scalable Bayesian Optimization Using Deep Neural Networks. In _Proceedings of the 32nd ICML_ , Vol. 37. JMLR.org, 2171–2180.
  * Song et al. (2021) Hailuo Song, Ao Li, Tong Wang, and Minghui Wang. 2021\.  Multimodal Deep Reinforcement Learning with Auxiliary Task for Obstacle Avoidance of Indoor Mobile Robot.  _Sensors_ 21, 4 (2021), 1363.
  * Strnad et al. (2019) Felix M. Strnad, Wolfram Barfuss, Jonathan F. Donges, and Jobst Heitzig. 2019.  Deep reinforcement learning in World-Earth system models to discover sustainable management strategies.  _CoRR_ abs/1908.05567 (2019).
  * Sutton and Barto (1998) Richard S. Sutton and Andrew G. Barto. 1998.  _Reinforcement learning - an introduction_.  MIT Press.
  * Ta et al. (2020) Van-Dai Ta, CHUAN-MING Liu, and Direselign Addis Tadesse. 2020\.  Portfolio optimization-based stock prediction using long-short term memory network in quantitative trading.  _Applied Sciences_ 10, 2 (2020), 437.
  * Tao et al. (2022) Tianxin Tao, Daniele Reda, and Michiel van de Panne. 2022\.  Evaluating Vision Transformer Methods for Deep Reinforcement Learning from Pixels.  _CoRR_ abs/2204.04905 (2022).
  * Thakkar and Chaudhari (2021) Ankit Thakkar and Kinjal Chaudhari. 2021.  A comprehensive survey on portfolio optimization, stock price and trend prediction using particle swarm optimization.  _Archives of Computational Methods in Engineering_ 28 (2021), 2133–2164.
  * Tunstall et al. (2022) Lewis Tunstall, Leandro von Werra, and Thomas Wolf. 2022\.  _Natural language processing with transformers_.  " O’Reilly Media, Inc.".
  * Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017\.  Attention is All you Need. In _30th Annual Conference on Neural Information Processing Systems_. 5998–6008.
  * Vedantam et al. (2015) Ramakrishna Vedantam, C. Lawrence Zitnick, and Devi Parikh. 2015\.  CIDEr: Consensus-based image description evaluation. In _IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2015, Boston, MA, USA, June 7-12, 2015_. IEEE Computer Society, 4566–4575.
  * Vinyals et al. (2019) Oriol Vinyals, Igor Babuschkin, Wojciech M. Czarnecki, Michaël Mathieu, Andrew Dudzik, Junyoung Chung, David H. Choi, Richard Powell, Timo Ewalds, Petko Georgiev, Junhyuk Oh, Dan Horgan, Manuel Kroiss, Ivo Danihelka, Aja Huang, Laurent Sifre, Trevor Cai, John P. Agapiou, Max Jaderberg, Alexander Sasha Vezhnevets, Rémi Leblond, Tobias Pohlen, Valentin Dalibard, David Budden, Yury Sulsky, James Molloy, Tom Le Paine, Çaglar Gülçehre, Ziyu Wang, Tobias Pfaff, Yuhuai Wu, Roman Ring, Dani Yogatama, Dario Wünsch, Katrina McKinney, Oliver Smith, Tom Schaul, Timothy P. Lillicrap, Koray Kavukcuoglu, Demis Hassabis, Chris Apps, and David Silver. 2019.  Grandmaster level in StarCraft II using multi-agent reinforcement learning.  _Nat._ 575, 7782 (2019), 350–354.
  * Vinyals et al. (2017) Oriol Vinyals, Timo Ewalds, Sergey Bartunov, Petko Georgiev, Alexander Sasha Vezhnevets, Michelle Yeo, Alireza Makhzani, Heinrich Küttler, John P. Agapiou, Julian Schrittwieser, John Quan, Stephen Gaffney, Stig Petersen, Karen Simonyan, Tom Schaul, Hado van Hasselt, David Silver, Timothy P. Lillicrap, Kevin Calderone, Paul Keet, Anthony Brunasso, David Lawrence, Anders Ekermo, Jacob Repp, and Rodney Tsing. 2017.  StarCraft II: A New Challenge for Reinforcement Learning.  _CoRR_ abs/1708.04782 (2017).
  * Vo et al. (2017) Quan-Hoang Vo, Huy-Tien Nguyen, Bac Le, and Minh-Le Nguyen. 2017.  Multi-channel LSTM-CNN model for Vietnamese sentiment analysis. In _9th International Conference on Knowledge and Systems Engineering, KSE 2017, Hue, Vietnam, October 19-21, 2017_. IEEE, 24–29.
  * Wang et al. (2021a) Jike Wang, Chang-Yu Hsieh, Mingyang Wang, Xiaorui Wang, Zhenxing Wu, Dejun Jiang, Benben Liao, Xujun Zhang, Bo Yang, Qiaojun He, Dongsheng Cao, Xi Chen, and Tingjun Hou. 2021a.  Multi-constraint molecular generation based on conditional transformer, knowledge distillation and reinforcement learning.  _Nat. Mach. Intell._ 3, 10 (2021), 914–922.
  * Wang et al. (2021c) Jiacheng Wang, Lan Wei, Liansheng Wang, Qichao Zhou, Lei Zhu, and Jing Qin. 2021c.  Boundary-Aware Transformers for Skin Lesion Segmentation. In _Medical Image Computing and Computer Assisted Intervention - MICCAI 24th International Conference_ , Vol. 12901. Springer, 206–216.
  * Wang et al. (2022d) Jiayue Wang, Hongbo Zhao, Haoqiang Liu, Liwei Geng, and Zebin Sun. 2022d.  A Distributed Vehicle-assisted Computation Offloading Scheme based on DRL in Vehicular Networks. In _22nd IEEE International Symposium on Cluster, Cloud and Internet Computing, CCGrid 2022, Taormina, Italy, May 16-19, 2022_. IEEE, 200–209.
  * Wang et al. (2022e) Kerong Wang, Hanye Zhao, Xufang Luo, Kan Ren, Weinan Zhang, and Dongsheng Li. 2022e.  Bootstrapped Transformer for Offline Reinforcement Learning. In _NeurIPS_.
  * Wang et al. (2020a) Lingxiao Wang, Zhuoran Yang, and Zhaoran Wang. 2020a.  Breaking the Curse of Many Agents: Provable Mean Embedding Q-Iteration for Mean-Field Reinforcement Learning. In _Proceedings of the 37th International Conference on Machine Learning, ICML_ , Vol. 119. PMLR, 10092–10103.
  * Wang et al. (2022b) Minrui Wang, Mingxiao Feng, Wengang Zhou, and Houqiang Li. 2022b.  Stabilizing Voltage in Power Distribution Networks via Multi-Agent Reinforcement Learning with Transformer. In _KDD ’22: The 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Washington, DC, USA, August 14 - 18, 2022_. ACM, 1899–1909.
  * Wang et al. (2023) Qi Wang, Kenneth H. Lai, and Chunlei Tang. 2023.  Solving combinatorial optimization problems over graphs with BERT-Based Deep Reinforcement Learning.  _Inf. Sci._ 619 (2023), 930–946.
  * Wang et al. (2018) Tingwu Wang, Renjie Liao, Jimmy Ba, and Sanja Fidler. 2018\.  NerveNet: Learning Structured Policy with Graph Neural Networks. In _6th International Conference on Learning Representations, ICLR_.
  * Wang et al. (2020b) Weixun Wang, Tianpei Yang, Yong Liu, Jianye Hao, Xiaotian Hao, Yujing Hu, Yingfeng Chen, Changjie Fan, and Yang Gao. 2020b.  From Few to More: Large-Scale Dynamic Multiagent Curriculum Learning. In _The Thirty-Fourth Conference on Artificial Intelligence, AAAI_. AAAI Press, 7293–7300.
  * Wang and Chen (2022) Yang Wang and Zhibin Chen. 2022.  A Deep Reinforcement Learning Algorithm Using A New Graph Transformer Model for Routing Problems. In _Intelligent Systems and Applications - Proceedings of the Intelligent Systems Conference, IntelliSys_ , Vol. 544. Springer, 365–379.
  * Wang et al. (2021b) Yifan Wang, Zhichao Min, and Sen Jia. 2021b.  Local-Global-Aware Convolutional Transformer for Hyperspectral Image Classification. In _23rd Int Conf on High Performance Computing_. IEEE, 1188–1194.
  * Wang et al. (2022a) Zhendong Wang, Xiaodong Cun, Jianmin Bao, Wengang Zhou, Jianzhuang Liu, and Houqiang Li. 2022a.  Uformer: A General U-Shaped Transformer for Image Restoration. In _IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR_. IEEE, 17662–17672.
  * Wang et al. (2022c) Zhihai Wang, Jie Wang, Qi Zhou, Bin Li, and Houqiang Li. 2022c.  Sample-Efficient Reinforcement Learning via Conservative Model-Based Actor-Critic. In _Thirty-Sixth AAAI Conference on Artificial Intelligence_. AAAI Press, 8612–8620.
  * Wen et al. (2022) Muning Wen, Jakub Grudzien Kuba, Runji Lin, Weinan Zhang, Ying Wen, Jun Wang, and Yaodong Yang. 2022.  Multi-Agent Reinforcement Learning is a Sequence Modeling Problem. In _NeurIPS_.
  * Williams (1992) Ronald J. Williams. 1992\.  Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning.  _Mach. Learn._ 8 (1992), 229–256.
  * Woo et al. (2023) Sanghyun Woo, Shoubhik Debnath, Ronghang Hu, Xinlei Chen, Zhuang Liu, In So Kweon, and Saining Xie. 2023.  ConvNeXt V2: Co-designing and Scaling ConvNets with Masked Autoencoders.  _CoRR_ abs/2301.00808 (2023).
  * Wu et al. (2022) Yaoxin Wu, Wen Song, Zhiguang Cao, Jie Zhang, and Andrew Lim. 2022.  Learning Improvement Heuristics for Solving Routing Problems.  _IEEE Trans. Neural Networks Learn. Syst._ 33, 9 (2022), 5057–5069.
  * Xiong et al. (2019) Yuxuan Xiong, Bo Du, and Pingkun Yan. 2019.  Reinforced Transformer for Medical Image Captioning. In _Machine Learning in Medical Imaging \- 10th International Workshop, MLMI, MICCAI_ , Vol. 11861. Springer, 673–680.
  * Xu et al. (2020) Ke Xu, Yifan Zhang, Deheng Ye, Peilin Zhao, and Mingkui Tan. 2020.  Relation-Aware Transformer for Portfolio Policy Learning. In _Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, IJCAI 2020_. ijcai.org, 4647–4653.
  * Xu et al. (2022b) Mengdi Xu, Yikang Shen, Shun Zhang, Yuchen Lu, Ding Zhao, Joshua B. Tenenbaum, and Chuang Gan. 2022b.  Prompting Decision Transformer for Few-Shot Policy Generalization. In _International Conference on Machine Learning, ICML_ , Vol. 162. PMLR, 24631–24645.
  * Xu et al. (2022a) Nuo Xu, Jianlong Chang, Xing Nie, Chunlei Huo, Shiming Xiang, and Chunhong Pan. 2022a.  AME: Attention and Memory Enhancement in Hyper-Parameter Optimization. In _IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR_. IEEE, 480–489.
  * Xu et al. (2022c) Peng Xu, Xiatian Zhu, and David A. Clifton. 2022c.  Multimodal Learning with Transformers: A Survey.  _CoRR_ abs/2206.06488 (2022).
  * Yamagata et al. (2022) Taku Yamagata, Ahmed Khalil, and Raúl Santos-Rodríguez. 2022\.  Q-learning Decision Transformer: Leveraging Dynamic Programming for Conditional Sequence Modelling in Offline RL.  _CoRR_ abs/2209.03993 (2022).
  * Yang et al. (2022) Biao Yang, Jicheng Yang, Rongrong Ni, Changchun Yang, and Xiaofeng Liu. 2022.  Multi-granularity scenarios understanding network for trajectory prediction.  _Complex & Intelligent Systems_ (2022), 1–14.
  * Yang et al. (2021) Lijuan Yang, Guanghui Yang, Zhitong Bing, Yuan Tian, Yuzhen Niu, Liang Huang, and Lei Yang. 2021.  Transformer-Based Generative Model Accelerating the Development of Novel BRAF Inhibitors.  _ACS Omega_ 6 (2021), 33864 – 33873.
  * Yarats et al. (2021) Denis Yarats, Amy Zhang, Ilya Kostrikov, Brandon Amos, Joelle Pineau, and Rob Fergus. 2021\.  Improving Sample Efficiency in Model-Free Reinforcement Learning from Images. In _Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI_. AAAI Press, 10674–10681.
  * Ying et al. (2021) Chengxuan Ying, Tianle Cai, Shengjie Luo, Shuxin Zheng, Guolin Ke, Di He, Yanming Shen, and Tie-Yan Liu. 2021\.  Do Transformers Really Perform Badly for Graph Representation?. In _NeurIPS_. 28877–28888.
  * Young and Sutton (2020) Kenny Young and Richard S. Sutton. 2020.  Understanding the Pathologies of Approximate Policy Evaluation when Combined with Greedification in Reinforcement Learning.  _CoRR_ abs/2010.15268 (2020).
  * Yu et al. (2023) Chao Yu, Jiming Liu, Shamim Nemati, and Guosheng Yin. 2023\.  Reinforcement Learning in Healthcare: A Survey.  _ACM Comput. Surv._ 55, 2 (2023), 5:1–5:36.
  * Yu et al. (2022) Chao Yu, Xinyi Yang, Jiaxuan Gao, Huazhong Yang, Yu Wang, and Yi Wu. 2022\.  Learning Efficient Multi-agent Cooperative Visual Exploration. In _Computer Vision - ECCV - 17th European Conference_ , Vol. 13699. Springer, 497–515.
  * Zhan et al. (2017) Yusen Zhan, Haitham Bou-Ammar, and Matthew E. Taylor. 2017\.  Scalable lifelong reinforcement learning.  _Pattern Recognit._ 72 (2017), 407–418.
  * Zhang et al. (2022b) Fengzhuo Zhang, Boyi Liu, Kaixin Wang, Vincent Y. F. Tan, Zhuoran Yang, and Zhaoran Wang. 2022b.  Relational Reasoning via Set Transformers: Provable Efficiency and Applications to MARL. In _NeurIPS_.
  * Zhang et al. (2022c) Hao Zhang, Hao Wang, and Zhen Kan. 2022c.  Exploiting Transformer in Reinforcement Learning for Interpretable Temporal Logic Motion Planning.  _CoRR_ abs/2209.13220 (2022).
  * Zhang et al. (2018b) Jiaping Zhang, Tiancheng Zhao, and Zhou Yu. 2018b.  Multimodal Hierarchical Reinforcement Learning Policy for Task-Oriented Visual Dialog. In _Proceedings of the 19th Annual SIGdial Meeting on Discourse and Dialogue, Melbourne, Australia_. Association for Computational Linguistics, 140–150.
  * Zhang et al. (2018a) Kaiqing Zhang, Zhuoran Yang, Han Liu, Tong Zhang, and Tamer Basar. 2018a.  Fully Decentralized Multi-Agent Reinforcement Learning with Networked Agents. In _Proceedings of the 35th ICML_ , Vol. 80. PMLR, 5867–5876.
  * Zhang et al. (2022d) Li Zhang, Sixiao Zheng, Jiachen Lu, Xinxuan Zhao, Xiatian Zhu, Yanwei Fu, Tao Xiang, and Jianfeng Feng. 2022d.  Visual Representation Learning with Transformer: A Sequence-to-Sequence Perspective.  _CoRR_ abs/2207.09339 (2022).
  * Zhang et al. (2022a) Tianyao Zhang, Xiaoguang Hu, Jin Xiao, and Guofeng Zhang. 2022a.  TVENet: Transformer-Based Visual Exploration Network for Mobile Robot in Unseen Environment.  _IEEE Access_ 10 (2022), 62056–62072.
  * Zhang et al. (2023) Zhipeng Zhang, Zhimin Wei, Zhongzhen Huang, Rui Niu, and Peng Wang. 2023.  One for all: One-stage referring expression comprehension with dynamic reasoning.  _Neurocomputing_ 518 (2023), 523–532.
  * Zhong et al. (2021) Huasong Zhong, Jingyuan Chen, Chen Shen, Hanwang Zhang, Jianqiang Huang, and Xian-Sheng Hua. 2021\.  Self-Adaptive Neural Module Transformer for Visual Question Answering.  _IEEE Trans. Multim._ 23 (2021), 1264–1273.
  * Zhou et al. (2023) Ce Zhou, Qian Li, Chen Li, Jun Yu, Yixin Liu, Guangjing Wang, Kai Zhang, Cheng Ji, Qiben Yan, Lifang He, Hao Peng, Jianxin Li, Jia Wu, Ziwei Liu, Pengtao Xie, Caiming Xiong, Jian Pei, Philip S. Yu, and Lichao Sun. 2023\.  A Comprehensive Survey on Pretrained Foundation Models: A History from BERT to ChatGPT.  _CoRR_ abs/2302.09419 (2023).
  * Zhou et al. (2021) Hong-Yu Zhou, Chixiang Lu, Sibei Yang, and Yizhou Yu. 2021\.  ConvNets vs. Transformers: Whose Visual Representations are More Transferable?. In _IEEE/CVF International Conference on Computer Vision Workshops, ICCVW_. IEEE, 2230–2238.
  * Zhou et al. (2020) Meng Zhou, Ziyu Liu, Pengwei Sui, Yixuan Li, and Yuk Ying Chung. 2020.  Learning Implicit Credit Assignment for Cooperative Multi-Agent Reinforcement Learning. In _NeurIPS_.