#  Transformer Wave Function for Quantum Long-Range models

Sebastián Roca-Jerat  Instituto de Nanociencia y Materiales de Aragón (INMA),
CSIC-Universidad de Zaragoza, Zaragoza 50009, Spain  Departamento de Física de
la Materia Condensada, Universidad de Zaragoza, Zaragoza 50009, Spain  Manuel
Gallego  Instituto de Nanociencia y Materiales de Aragón (INMA), CSIC-
Universidad de Zaragoza, Zaragoza 50009, Spain  Departamento de Física
Teórica, Universidad de Zaragoza, Zaragoza 50009, Spain  Fernando Luis
Instituto de Nanociencia y Materiales de Aragón (INMA), CSIC-Universidad de
Zaragoza, Zaragoza 50009, Spain  Departamento de Física de la Materia
Condensada, Universidad de Zaragoza, Zaragoza 50009, Spain  Jesús Carrete
Instituto de Nanociencia y Materiales de Aragón (INMA), CSIC-Universidad de
Zaragoza, Zaragoza 50009, Spain  Departamento de Física de la Materia
Condensada, Universidad de Zaragoza, Zaragoza 50009, Spain  David Zueco
<EMAIL_ADDRESS>Instituto de Nanociencia y Materiales de Aragón (INMA), CSIC-
Universidad de Zaragoza, Zaragoza 50009, Spain  Departamento de Física de la
Materia Condensada, Universidad de Zaragoza, Zaragoza 50009, Spain

###### Abstract

We employ a neural-network architecture based on the Vision Transformer (ViT)
architecture to find the ground states of quantum long-range models,
specifically the transverse-field Ising model for spin-1/2 chains across
different interaction regimes. Harnessing the transformer’s capacity to
capture long-range correlations, we compute the full phase diagram and
critical properties of the model, in both the ferromagnetic and
antiferromagnetic cases. Our findings show that the ViT maintains high
accuracy across the full phase diagram. We compare these results with previous
numerical studies in the literature and, in particular, show that the ViT has
a superior performance than a restricted-Boltzmann-machine-like ansatz.

##  I Introduction

Understanding quantum many-body problems is crucial for elucidating the
properties of condensed matter systems. However, this is generally a
challenging task, as exemplified by the problem of searching for the ground
states of strongly correlated models. Beyond exactly solvable models, some
systems are tractable by introducing elements such as matrix product states
(MPS), tensor networks [1], dynamical mean field theory (DMFT) [2], quantum
Monte Carlo [3], and hybrid quantum-classical algorithms [4]. Additionally,
there are physically inspired ansätze available, like the Bethe and polaron
approaches, correlator product states [5], and generalized coherent states
[6].
In 2017, neural networks (NNs) emerged as an alternative ansatz for solving
quantum many-body systems [7]. NNs have since been recognized for their
ability to generate states with a volume law of entanglement [8, 9],
addressing problems beyond the reach of MPS. Variational optimization of
ansätze like NNs also avoids the sign problem of other quantum Monte Carlo
techniques, enabling the handling of frustrated systems and account for states
with long correlation lengths beyond those tractable with DMFT. Despite these
advantages, it is difficult to determine in advance which NN architecture will
be most suitable for a specific problem or whether it can be trained with
sufficient precision. Therefore, it is interesting to investigate generic
models of significant importance and evaluate how well they are solved using
NNs [10].

In this context, we focus on a set of models in which two-body interactions
decay according to a power law, commonly referred to as long-range systems.
They are ubiquitous in nature, with examples arising, among others, from
dipolar, Coulomb, and van der Waals interactions. Recent experimental advances
in atomic, molecular, and optical systems have renewed interest in long-range
models [11, 12, 13, 14]. In these experiments, the effective interactions are
often long ranged and tunable, underscoring the need for a comprehensive
understanding of long-range systems. Although less studied than their short-
ranged counterparts, there are already some rigorous and numerical results
available for systems with long-range interactions [15, 16, 17, 18, 19].
Various equilibrium and dynamical properties have been explored in comparison
with short-ranged systems. Notable examples include the existence (or absence)
of an area law of entanglement [20, 21, 22, 23], the algebraic decay of two-
point correlators outside criticality [24, 25, 26], the spreading of
correlations [27], the existence of Majorana modes [28], and topological
properties [29].

In this work, we test neural networks (NNs) for solving quantum long-range
models, specifically focusing on the quantum Ising model as a paradigmatic
example. Our primary motivation is to use a sufficiently generic NN ansatz
capable of covering the entire phase diagram, from short-range to strong long-
range interactions, and for both ferromagnetic and antiferromagnetic phases.
Since the introduction of Restricted Boltzmann Machines (RBMs), several NN
architectures have been developed, including multilayer perceptrons [30],
convolutional neural networks [31], and recurrent architectures [32]. This
list is not exhaustive; for a comprehensive review of the field, we refer to
Ref. [10]. More recently, the transformer has also been introduced as a
variational ansatz architecture [33, 34, 35]. In this work, we choose to work
with the Vision Transformer (ViT). We demonstrate that this architecture is
capable of computing the full phase diagram and characterizing its critical
properties. By doing so, we provide a single ansatz that encompasses previous
numerical works discussing parts of the phase diagram [36, 37, 38, 39, 40].
Additionally, we compare the performance of our ViT and RBM architectures.

The rest of the manuscript is organized as follows: in section II we lay out
the details of the family of models we are trying to solve, in section III we
discuss the methods, software and parameters that we use, in section IV we
present and contextualize our results, and in the last section we summarize
our main conclusions. Details of the finite size scaling are sent to Appendix
A.

##  II Long-range interacting quantum models

Figure 1: Dependence of the generalised harmonic numbers on the system’s size
($N$) and on the range of the interactions ($\alpha$). It can be seen that for
$\alpha<1.5$ the dependence of this factor on size intensifies. The solid grey
line indicates the asymptotic value of $H_{N}^{(\alpha)}=1$ and the dashed
black line indicates the Riemann zeta function, $\zeta(\alpha)$, to which the
generalised harmonic number tends when $N\to\infty$.

Generically, quantum long-range models on a $d$-dimensional, $N$-site lattice
contain interactions described by a Hamiltonian of the form

$\mathcal{H}_{\rm c}=-\sum_{ij}^{N}J_{ij}\mathcal{C}_{i}\mathcal{C}_{j}\,,$
(1)

where $\mathcal{C}_{i}$ is a local Hermitian operator acting on site $i$ and
the interactions decay according to a power law

$J_{ij}=J\frac{\tilde{J}_{ij}}{\tilde{N}},$ (2)

with

$\tilde{J}_{ij}=\begin{cases}b&\text{ if }\;i=j\\\ |i-j|^{-\alpha}&\text{
otherwise}.\end{cases}$ (3)

In this equation, $b$ is a parameter that can be tuned to shift the spectrum
of $J$ and is taken as $b=1$ in our simulations, and the parameter $\alpha$
sets the range of the interactions. Throughout this paper, we will assume
periodic boundary conditions (PBC) and $d=1$.

It is important to discuss the normalization used in Eq. (2). For $\alpha<d$,
the interactions decay slowly enough that the sum in the coupling term (1)
depends on $N$ superlinearly, breaking the extensive character of the model
[16, 41, 18]. Kac’s renormalization factor, $1/\tilde{N}$, restores that
feature, ensuring a well-defined thermodynamic limit. In particular,

$\tilde{N}=\sum_{j}\tilde{J}_{ij}=1+\sum_{j=1}^{N}\frac{1}{j^{\alpha}}=1+H_{N}^{(\alpha)}\;.$
(4)

where $H_{N}^{(\alpha)}$ is a generalized harmonic number. We have used the
fact PBC make the model translationally invariant and thus
$\sum_{i}\tilde{J}_{ij}$ is independent of $j$. In figure 1, we plot
$H_{N}^{(\alpha)}$ for different sizes as a function of $\alpha$. For
$\alpha>1$, $H_{N}^{(\alpha)}$ approaches the $\zeta(\alpha)$ function as
$N\to\infty$. Indeed, for $\alpha\gtrapprox 2$, in the next sections we
confirm through numerical simulations that the dependence of the Kac factor on
system size can be safely neglected. Eventually, $\tilde{N}\to 1$ as
$\alpha\to\infty$. For $\alpha<1$, the asymptotics of $H_{N}^{(\alpha)}$ are
richer, but it diverges as a function of $N$, which makes this regularization
necessary to have a well-defined thermodynamic limit.

Figure 2: Diagram of the nature of interactions as a function of $\alpha$.

The general phenomenology, both quantum and classical, can be characterized by
the value of $\alpha$. The non-additive regime, characterized by $\alpha<1$,
is termed _(strong) long-range_. This regime is often overlooked in many
analytical and numerical studies due to the challenges posed by an ill-defined
thermodynamic limit resulting from non-extensivity. However, Kac’s rescaling
overcomes this obstacle. In the quantum context, numerical studies that solve
the transverse-field Ising model in the strong long-range regime can be found
in Refs. 36, 42, confirming that in this regime, the model is within the mean-
field universality class. Furthermore, it has been proven that, in both the
classical and quantum realms, some models can be solved exactly through mean-
field-like approximations that become exact in this regime [43, 44, 45, 46,
47, 38].
Within $\alpha>1$ domain, additional subregimes are discernible: a threshold
$\alpha_{*}$ exists such that for $\alpha>\alpha_{*}$ the critical exponents
of the model align with those of the nearest-neighbour model
($\alpha\to\infty$). This is the _short-range regime_. Conversely, in the
range where $d<\alpha<\alpha_{*}$, the model exhibits critical exponents
distinct from their short-range counterparts, revealing the influence of long-
range interactions while retaining additivity. This condition defines the
_weak long-range regime_ [15, 18]. It is also possible to further subdivide
these classes by introducing $\alpha_{\mathrm{MF}}$ such that for
$1<\alpha<\alpha_{\mathrm{MF}}$ the critical exponents correspond to those
predicted by mean-field (MF) theories [48, 49]. It appears that the weak long-
range regime is the most challenging to access from an analytical standpoint,
making numerical approaches indispensable for its understanding [39, 50, 36,
20, 40]. For ease of reference, we summarize this classification in Fig. 2.

In this paper, we solve the long-range Ising model, which is of significant
interest for various experiments [11, 51, 52] and from a theoretical
perspective, offering numerous results for comparison. We consider the
Hamiltonian given by

$\mathcal{H}=\sum_{ij}^{N}J_{ij}\sigma_{i}^{z}\sigma_{j}^{z}\
-h_{x}\sum_{i}^{N}\sigma^{x}_{i},$ (5)

where the $\sigma^{x}$ represent the standard Pauli matrices and $J_{ij}$ is
defined as in Eq. (2). Throughout all computations, we set $h_{x}=1$, thereby
fixing the units of $J$. This model yields exact analytical results in the
limit $\alpha\to\infty$ (nearest neighbors) via the Jordan-Wigner
transformation. Conversely, in the limit $\alpha\to 0$, the model reduces to
the Lipkin-Meshkov-Glick (LMG) model, which is also solvable in the
thermodynamic limit. Besides for $\alpha<d$, the model admits an exact
solution in the ferromagnetic ($J<0$) case [38]. These analytical insights can
be complemented by numerical approaches that span various ranges of $\alpha$,
utilizing diverse techniques such as quantum Monte Carlo [36, 42] and ansätze
such as tensor networks [20, 39], and, more recently, restricted Boltzmann
machines (RBM) [40]. All these works together offer a comprehensive overview
for both the ferromagnetic and antiferromagnetic scenarios. The system
undergoes a second-order quantum phase transition (QPT) from ferromagnetic and
Néel respectively ordered phases to paramagnetic phases at a certain critical
point $J_{c}(\alpha)$. Within this model, $\alpha_{MF}=5/3$, below which the
critical exponents align with those predicted by Mean Field theory.
Conversely, for $\alpha>3$, the system falls within the universality class of
the 2D classical short-range Ising model. As a consequence, model (5) provides
an excellent baseline for evaluating the accuracy of the transformer
variational ansatz across the entire $\alpha$ spectrum, ranging from short-
range to flat interactions, and for both ferromagnetic and antiferromagnetic
cases.

##  III Methods

###  III.1 Variational quantum Monte Carlo

Figure 3: (a) General scheme of the parameterization of a wave-function,
$\psi_{\theta}$, through a machine learning model, $M$, characterized by a
series of variational parameters $\boldsymbol{\theta}$. (b) General structure
of a RBM. The parameter $\alpha_{\mathrm{RBM}}$ indicates the density of the
hidden layer. (c) Block diagram representing the structure of the ViT ansatz
used in this work. The color code is used: green corresponds to all those
operations that act at token level; in yellow we represent operations that
generate correlations between the channels originating from different tokens;
finally, red highlights the operations placed after the Core block that merge
the channels and post-process that result.

We use an unnormalized representation of the ground state over the canonical
basis states $\left\\{|{\mathbf{s}}\rangle\right\\}$:

$|\psi_{\boldsymbol{\theta}}\rangle=\sum_{{\mathbf{s}}}\psi_{\boldsymbol{\theta}}(\mathbf{x}_{\mathrm{in}})|{\mathbf{s}}\rangle,$
(6)

where $\psi_{\boldsymbol{\theta}}$ is a sufficiently flexible ansatz with a
collection of parameters $\boldsymbol{\theta}$, yielding the probability
amplitude of each of those states, and $\mathbf{x}_{\mathrm{in}}$ is an
appropriate encoding of $|{\mathbf{s}}\rangle$ to serve as an input to the
function. In this work we focus on spin $1/2$ chains, so
$\mathbf{x}_{\mathrm{in}}$ will be composed of a binary variable for each
spin, whose values we encode as $+1$ or $-1$ (see Figure 3a). Our choices for
$\psi_{\boldsymbol{\theta}}$ are all general machine-learning models, which we
generically represent as $M\left(\boldsymbol{\theta}\right)$.

In this framework, the solution for the ground state of the system comes in
the form of an optimal set of parameters $\boldsymbol{\theta}$. Those
parameters are thus variationally optimized by minimizing the energy of the
Hamiltonian,
$\langle\mathcal{H}\rangle_{\psi_{\boldsymbol{\theta}}}\equiv\langle\psi_{\boldsymbol{\theta}}|\mathcal{H}|\psi_{\boldsymbol{\theta}}\rangle/\langle\psi_{\boldsymbol{\theta}}|\psi_{\boldsymbol{\theta}}\rangle$.
The expected value of an observable explicitly represented on this basis as a
function
$Q\left(\mathbf{x}_{\mathrm{in}}\right)=\frac{\langle\mathbf{s}|Q|\psi_{\boldsymbol{\theta}}\rangle}{\langle\mathbf{s}|\psi_{\boldsymbol{\theta}}\rangle}$,
can be calculated as

$\langle
Q\rangle_{\psi_{\boldsymbol{\theta}}}=\sum_{{\mathbf{s}}}p_{\theta}({\mathbf{s}})Q\left(\mathbf{x}_{\mathrm{in}}\right)\
,$ (7)

where
$p_{\theta}(\mathbf{s})=|\psi_{\boldsymbol{\theta}}\left(\mathbf{x}_{\mathrm{in}}\right)|^{2}/\langle\psi_{\boldsymbol{\theta}}|\psi_{\boldsymbol{\theta}}\rangle$.

As these definitions suggest, we are forced to resort to Monte Carlo (MC)
sampling in order to evaluate the averages to carry out the minimization
process, given the impossibility of considering the totality of states.
Specifically, we use the Metropolis-Hastings algorithm with a combined
transition rule. On the one hand, a random spin is flipped according to a
uniform probability without taking into account any constraints such as the
total magnetization. On the other hand, we invert the total magnetization of a
random number of Markov chains. The probability for the former is three times
larger than the probability for the latter in our implementation. Furthermore,
among all the available optimization methods, we employ Stochastic Gradient
Descent (SGD) with custom schedules for the learning rate, $\lambda$, combined
with the Stochastic Reconfiguration (SR) method [53], characterized by a
stabilizing parameter called diagonal shift that we denote as $\Delta_{sr}$.

###  III.2 Ansätze: neural networks and restricted Boltzmann machines

In this variational framework, NNs are leveraged to provide a trainable
$\psi_{\boldsymbol{\theta}}$. In this study we employ and compare two
different architectures with very different levels of complexity: an RBM
adapted for regression and a vision transformer. Multilayer perceptrons
(MLPs), the most basic incarnation of a feed-forward neural network (FFNN),
are used as building blocks in the ViT and also serve to introduce this RBM.

The architecture of an FFNN is characterized by the unidirectional flow of
information from the input towards the output, passing through a set of hidden
layers. In the case of an MLP, each neuron in a layer computes a linear
combination of all the outputs from the previous layer and applies a nonlinear
activation function to the results to create its own output. The defining
characteristics of an MLP are the all-to-all connectivity between adjacent
layers and the fact that there is at least a single hidden layer. The
functional form of an MLP with activation function $F$ and $\ell$ hidden
layers is

$\displaystyle\operatorname{MLP}\left(\mathbf{x}_{\mathrm{in}};\mathbf{w}_{1},\mathbf{b}_{1},\mathbf{w}_{2},\mathbf{b}_{2}\ldots\mathbf{w}_{\mathrm{out}},\mathbf{b}_{\mathrm{out}}\right)=$
$\displaystyle=F\left(\mathbf{b}_{\mathrm{out}}+\mathbf{w}_{\mathrm{out}}F\left(\mathbf{b}_{\ell}+\mathbf{w}_{\ell}F\left(\ldots
F\left(\mathbf{b}_{1}+\mathbf{w}_{1}\mathbf{x}_{\mathrm{in}}\right)\right)\right)\right)$

Here, the $\mathbf{w}$ matrices and $\mathbf{b}$ vectors are the weights and
biases of each layer, respectively. Layers can have different widths, and $F$
is understood to act elementwise on its arguments. We create the functional
form of our ansatz by using an MLP for the logarithm of the wave function:

$\psi_{\boldsymbol{\theta}}^{\text{FFNN}}=\exp\left[\sum\operatorname{MLP}\left(\mathbf{x}_{\mathrm{in}};\boldsymbol{\theta}\right)\right].$
(8)

where $\boldsymbol{\theta}$ stands for the collection of weights and biases
and the sums run over elements of the output to generate a single value. This
gives rise to a real wave function, suitable for our applications. The
corresponding complex ansatz could be built by adding a second MLP
parametrizing the imaginary part of the logarithm, i.e., the phase of the wave
function.

A perceptron with a single hidden layer and some minor changes is equivalent
to a restricted Boltzmann machine (RBM; see Figure 3b), originally devised as
a generative model. Specifically, the nonlinear activation function must be
chosen as $F\left(x\right)=\log\cosh\left(x\right)$ and an additional linear
term is added to the output. The ansatz in this case is

$\displaystyle\log\psi_{\boldsymbol{\theta}}^{\text{RBM}}=\mathbf{a}\mathbf{x}_{\mathrm{in}}+$
$\displaystyle\sum_{j}\log\left[\cosh\left(\left(\operatorname{Re}\mathbf{w}\right)\mathbf{x}_{\mathrm{in}}+\operatorname{Re}\mathbf{b}\right)_{j}\right]+$
(9) $\displaystyle
i\sum_{j}\log\left[\cosh\left(\left(\operatorname{Im}\mathbf{w}\right)\mathbf{x}_{\mathrm{in}}+\operatorname{Im}\mathbf{b}\right)_{j}\right]$

In the expression above, the product runs over outputs. The width of the
output layer is usually chosen as proportional to the width of the input layer
through an integer factor known as the density [7]. This variational ansatz
results in a wave function with parameters
$\boldsymbol{\theta}=\\{\mathbf{a},\mathbf{w},\mathbf{b}\\}$. We take
$\mathbf{a}=0$ following the implementation in NetKet [54].

Although on paper MLPs can be used to approximate any sufficiently well-
behaved function according to a number of universality theorems [55, 56], the
basic MLP architecture faces important limitations in practice arising, among
other factors, from the phenomenon of vanishing gradients during training [57]
as well as the complexity of their all-to-all mapping. The former has been
alleviated through breakthroughs such as normalization layers (e.g. BatchNorm
[58] and LayerNorm [59]), residual learning by means of additive shortcuts
around groups of layers [60, 61] and new activation functions line Swish [62].
In terms of reduction in complexity, the transformer architecture, originally
developed to replace recurrent architectures in machine translation tasks
[63], has found applications in all areas of machine learning, and in
particular been very successfully adapted to computer vision tasks as the
vision transformer (ViT) [64]. A key feature of transformer architectures is
their decoupling of nonlinear processing, which is still performed by modified
MLPs acting on subsets of the data, and mixing between those pieces of data
according to the relevance of those interactions to the result, which is
handled by an attention mechanism. The details of the version of the
transformer used in this work are presented in the form of a block diagram in
Figure 3c, and the main operations are described in what follows.

First of all, _tokenization_ takes place. Here the quantum states
$|{\mathbf{s}}\rangle$, represented as their corresponding arrays
$\mathbf{x}_{\mathrm{in}}$, are divided into $n$-tokens of dimension $d$
($N=d\,n$). Next they are passed through a linear embedding layer with
trainable parameters (common to all tokens), mapping each token into a vector
of arbitrary dimension $d_{\text{emb}}$. The next stage is an attention block
where correlations between tokens are taken into account. As depicted in the
central panel of Figure 3c, we first normalize the output of the previous
embedding using a LayerNorm operation [59]. We then split each token into $h$
vectors of dimension $p=d_{\text{emb}}/h$. Each of these vectors goes into one
of the $h$ heads of the multihead attention layer, where the following
operation takes place:

$\mathbf{A}_{i}^{\mu}=\sum_{j=1}^{n}a_{ij}^{\mu}V^{\mu}\mathbf{x}^{\mu}_{j}\;.$
(10)

Here, $\mathbf{x}^{\mu}_{j}$ represents the $p$-dimensional vector processed
by the $\mu$-th head coming from token $j$. As it can be seen, tokens are
mixed within each head through the linear transformation parameterized by
$V^{\mu}$ and the attention matrix $a_{ij}^{\mu}$. Note that this is a
simplified version of the original dot-product attention mechanism [63]. The
key feature of this simplification is that the attention weights are dependent
only on the relative positions of each pair of spins. In contrast, the usual
dot-product attention is invariant with respect to permutations and therefore
does not encode positions at all, so explicit positional encodings must be
introduced to avoid losing that information [65]. Furthermore, since we deal
with spin chains with periodic boundary conditions, we enforce the
translational symmetry by constraining $a^{\mu}_{ij}$ to be a circulan matrix,
i.e., $a^{\mu}_{ij}=c^{\mu}_{(j-i)\quad{\rm mod}p}$ [33]. This is complemented
by an explicit symmetrization through an average over all the possible cyclic
permutations of the tokens. In this regard, the ViT improves upon the
complexity of a fully-connected model when applied to a translationally
symmetric system, as the latter would require a similar symmetrization over
all cyclic permutations of spins. After applying this layer, we concatenate
the $h$ $\mathbf{A}_{i}^{\mu}$ vectors to recover a single
$d_{\text{emb}}$-dimensional vector for each of the input tokens. Each of
those vectors is then fed through a modified MLP for nonlinear processing,
whose components are represented in the right-hand-side panel of Figure 3c.
The output is again a $d_{\text{emb}}$-dimensional vector for each input
token. Note that both operations (mixing by the attention block and processing
by the MLP) are also surrounded by ResNet-like bypasses [61]. We further pass
the output throug an elementwise $\log\cosh\left(x\right)$, which we have
found to greatly enhance the training performance of the model when it must
learn wave functions whose norm is concentrated over a few states. The
inspiration for our choice of transformation comes from the simpler RBM-like
architecture, where the $\operatorname{log-cosh}$ function is essential for a
satisfactory training, as opposed to more conventional nonlinear activation
functions. This completes our ViT core block and the set of operations defined
terms of tokens. A pooling operation is now applied so that the $n$ vectors
corresponding to the tokens are averaged to give rise to a single (still
$d_{\text{emb}}$-dimensional) vector. This is further processed by another MLP
where layer by layer the dimensions of the vector resulting from pooling are
reduced: $d_{\text{emb}}\to d_{1}\to\dots\to d_{n_{pp}}$. Finally, a last
linear transformation acts as an offset-and-scaling block. This complete ViT
is used to create the real parts of $\log\psi_{\boldsymbol{\theta}}$, and we
do not include an imaginary part.

###  III.3 Parameters and computational details

All calculations in the following studies were performed using GPUs. The
hyperparameters used in the ViT architecture are listed in Table 1. Our code
is written in Python and available on GitHub 111https://github.com/Theory-and-
Simulation-at-INMA/transformer_LR_WF_public/tree/v24.07.1,
doi:10.5281/zenodo.12654460, retrieved on 2024-07-04.. Our models are defined
using the Flax neural-network library [67], which runs on top of JAX [68].
Training (including the schedules for the learning rate and for the diagonal
shift in the SR preconditioner) is handled by a combination of Optax [69] and
NetKet [54], which also takes care of sampling.

Variable | Notation | Value
---|---|---
Chain length | $N$ | $50-200$
Token dimension | $d$ | $N/10$
Number of tokens | $n=N/d$ | $10$
Embedding dimension | $d_{\rm{emb}}$ | $14$
Number of heads | $h$ | $2$
Head dimension | $p=d_{\rm{emb}}/h$ | $7$
MLP layers within Core Block | $n_{\text{MLP}}$ | $3$
Post-processor MLP | $(d_{1},\dots,d_{n_{pp}})$ | $(5,)$

Table 1: Summary of parameters in the ViT architecture and their values used
for the simulations [cf. Figure 3].

Each iteration of the training is performed using a total of $4096$ MC samples
distributed over $1024$ independent Markov chains. For the training process we
chose a protocol for the learning rate, $\lambda$, consisting of a linear
warm-up followed by an exponential decay. This protocol is defined by an
initial value of the learning rate, $\lambda_{0}$, a maximum value,
$\lambda_{\mathrm{max}}$ that it attains after $n_{\mathrm{warm}}$ iterations
corresponding to the linear warm-up and a ratio $\gamma$ for the exponential
decay. To obtain the states with which we created Figure 4, a maximum of $250$
iterations for training were used, along with following parameters:
$\lambda_{0}=0.1$, $\lambda_{\mathrm{max}}=2.0$, $n_{\mathrm{warm}}=75$,
$\gamma=0.995$. The stabiliser parameter present in the SR method follows a
simple linear schedule with an initial value of $\Delta_{sr}=${10}^{-2}$$ and
a final value of $\Delta_{sr}=${10}^{-4}$$.

##  IV Results

Here we discuss the performance of the ViT ansatz for characterizing the full
phase diagram of Eq. (5). We compare these results with previous studies that
cover specific ranges of the model.

###  IV.1 Quantities to compute

We define the _generalized_ staggered magnetization through the operator
$m_{s}$,

$\hat{m}_{s}\left(q\right)=\frac{1}{N}\sum_{j}\hat{\sigma}^{z}_{j}e^{iqj}\ ,$
(11)

where $q=0$ when $J<0$ and $q=\pi$ when $J>0$. Once the network parameters
have been determined, this can be computed using Eq. (7).

To characterize the entanglement between two partitions of the system within
the ground state, we also compute the Renyi-2 entanglement entropy, choosing
the partition to be half of the system size,

$\hat{S}_{2}=-\log_{2}\operatorname{Tr}_{A}[\rho^{2}]\ ,$ (12)

where
$\rho=|\psi_{\boldsymbol{\theta}}\rangle\langle\psi_{\boldsymbol{\theta}}|$.
To compute $\hat{S}_{2}$ we follow Ref. 70, define
$|\psi_{\boldsymbol{\theta}}\rangle$ based on the states of the computational
basis corresponding to each of the partitions, $\mathbf{s}_{A}\in A$ and
$\mathbf{s}_{\bar{A}}\in\bar{A}$, such that
$|\psi_{\boldsymbol{\theta}}\rangle=\sum_{\mathbf{s}_{A},\mathbf{s}_{\bar{A}}}\psi_{\boldsymbol{\theta}}(\mathbf{s}_{A},\mathbf{s}_{\bar{A}})|\mathbf{s}_{A}\rangle|\mathbf{s}_{\bar{A}}\rangle$.
From this definition we can sample states from each of the partitions to
estimate $\langle\hat{S}_{2}\rangle$ according to

$\langle\hat{S}_{2}\rangle=\sum_{\begin{subarray}{c}\mathbf{s}_{A},\mathbf{s}_{\bar{A}}\\\
\mathbf{s}^{\prime}_{A},\mathbf{s}^{\prime}_{\bar{A}}\end{subarray}}p_{\theta}(\mathbf{s}_{A},\mathbf{s}_{\bar{A}})p_{\theta}(\mathbf{s}^{\prime}_{A},\mathbf{s}^{\prime}_{\bar{A}})S_{2}(\mathbf{s}_{A},\mathbf{s}_{\bar{A}},\mathbf{s}^{\prime}_{A},\mathbf{s}^{\prime}_{\bar{A}})\
,$ (13)

where

$S_{2}(\mathbf{s}_{A},\mathbf{s}_{\bar{A}},\mathbf{s}^{\prime}_{A},\mathbf{s}^{\prime}_{\bar{A}})=-\log\left\langle\frac{\psi_{\boldsymbol{\theta}}(\mathbf{s}_{A},\mathbf{s}^{\prime}_{\bar{A}})\psi_{\boldsymbol{\theta}}(\mathbf{s}^{\prime}_{A},\mathbf{s}_{\bar{A}})}{\psi_{\boldsymbol{\theta}}(\mathbf{s}_{A},\mathbf{s}_{\bar{A}})\psi_{\boldsymbol{\theta}}(\mathbf{s}^{\prime}_{A},\mathbf{s}^{\prime}_{\bar{A}})}\right\rangle\
.$ (14)

This procedure is already implemented in NetKet [54].

Finally, it is useful to employ a metric to quantify the quality of the
converged wave functions. One such metric is the V-score [71]. It is defined
in terms of the energy fluctuations, conveniently normalized to yield an
$N$-independent quantity:

$\mathrm{V\mathchar 45\relax
score}=N\left(\frac{\langle\mathcal{H}^{2}\rangle_{\psi_{\boldsymbol{\theta}}}-\langle\mathcal{H}\rangle_{\psi_{\boldsymbol{\theta}}}^{2}}{\langle\mathcal{H}\rangle_{\psi_{\boldsymbol{\theta}}}^{2}}\right)\
,$ (15)

###  IV.2 Phase diagram and critical exponents

Figure 4: (a) Squared magnetization, $\langle\hat{m}^{2}_{s}\rangle$, obtained
from the ground states determined by the ViT ansatz for a size of $N=50$ spins
for different values of $J$ and $\alpha$. In this diagram different transition
curves present in the literature are represented [36, 37, 39, 38]. In
addition, the points indicated with light blue circles correspond to our
estimates obtained by finite-size scaling analysis. The panels to the right of
the phase diagram indicate the behavior of different quantities along two
slices of the diagram (white lines). In panels (b) and (d) the value of
$\alpha=4.0$ has been set and the V-score (15) and squared magnetization
$\langle\hat{m}^{2}\rangle$ are shown as a function of $J$, respectively. In
panels (c) and (e) the same quantities are shown in the same order but fixing
$J=4.75$ and varying $\alpha$. The different curves in all these panels
indicate different chain sizes, with $N=20$ being the lightest color and
$N=100$ the darkest, going through $N=50$ (intermediate).

In Figure 4 we summarize our calculations for the full phase diagram of the
Ising model (5). In contour 4a) we plot the fluctuations
$\langle\hat{m}_{s}^{2}\rangle$ in order to resolve the phase transition. Here
the simulations are done with $N=50$ spins and a discretization of $\delta
J=0.375$ and $\delta\alpha=0.4$. The darker zones stand for ordered phases,
both ferro and antiferromagnetic. We also plot several critical $J_{c}$ found
with different techniques in previous works [36, 37, 38, 39]. Light blue
circles mark values where a deeper investigation has been done (see below). In
panels 4 b) and c), the V-score is plotted for different sizes $N=20,50,100$
along the cuts marked in panel a) with white lines (the darker the curve, the
larger the size). We observe that the V-score does not depend on the system
size, demonstrating that the simulations do not deteriorate as $N$ increases.
The values around $10^{-4}$ compare well with variational methods for other
quantum many-body problems [71]. Beyond this figure of merit, in panels 4 d)
and e) we plot the fluctuations $\langle\hat{m}_{s}^{2}\rangle$ for the same
sizes and along the same cuts. In addition to the finite-size behavior, the
curves are smooth, which confirms the convergence of the method. It is
important to note that all of these results are the outcome of single shot
runs. No averaging or filtering has been performed, which again highlights the
consistent convergence of the model. It is worth stressing that the inclusion
of the $\operatorname{log-cosh}$ function at the end of the core block [cf.
Figure 3], as described in Section III.2, is instrumental in obtaining
accurate states in the deep FM regime.

A key quantity that characterizes the states and the accuracy of the ansatz is
the entropy. On one hand, it reveals the limitations in the amount of
entanglement other methods such as MPS can capture, and it also characterizes
the critical points, where it is known that the behavior of the entropy is
universal and follows a volume law. Our results for the entropy are plotted in
Figure 5 as a function of $J$ for both ferromagnetic and antiferromagnetic
cases at $\alpha=2.5$. The plotted points exhibit a large dispersion, which
could be addressed by repeating the runs and selecting the best outputs.
However, our motivation here was to employ a single-shot, economical strategy.
Despite the dispersion, we observe that the entropy increases with size around
the phase transition, consistent with the volume law, and loses this
dependence outside criticality, where the area law is expected. The dispersion
prevents fitting the expected volume law accurately. To aid visualization, we
used a Savitzky-Golay filter, allowing us to follow the trend with increased
size as we approach the critical point.

Figure 5: Fluctuations in the determination of the Renyi-2 entropy. For
$\alpha=2.5$, the entropy for different chain sizes is shown in the FM regime
[panel (a)] and in the AFM regime [panel (b)] The dots represent the
simulation results [cf. Eq. (13)] while the solid lines have been obtained
with a Savitzky-Golay filter to act as a guide to the eye.

Our final characterization involves extracting the critical exponents. This
requires precision in the order parameter that characterizes the transition.
In our case, we take advantage of how well the ViT reproduces the fluctuations
$\langle\hat{m}_{s}^{2}\rangle$. This allows us to determine both the critical
point $J_{c}$ and the critical exponents associated with the phase transition,
$\nu$ and $\beta$. In order to do so, we use finite size scaling theory [72,
73]. Close the critical point, $\langle\hat{m}_{s}^{2}\rangle$ scales with the
system size, $N$, as

$\langle\hat{m}_{s}^{2}(N,J)\rangle=N^{-2\beta/\nu}f\left(N^{1/\nu}(J-J_{c})\right)\
,$ (16)

where $\nu$ is the critical exponent indicating how the correlation length
diverges and $\beta$ the exponent indicating how the magnetization diverges in
the thermodynamic limit. $f$ is a dimensionless scaling function that depends
on the ratio between system size and correlation length and controls for
finite size effects. The critical values are fitted so that different sizes
collapse with the same $f$. In our case, we use the Python fssa library [74,
75] for the fit. All the collapses obtained are plotted in Appendix A. Table 2
summarizes the critical values obtained for the most paradigmatic cases,
comparing them to analytical results, where available, and to other numerical
values reported in Refs. [36] and [40]. Note that, to enable a direct
comparison, we do not apply the Kac renormalization factor of Eq. (4). In
addition to $J_{c}$, $\nu$ and $\beta$, we also list
$\tilde{h}_{c}=1/\tilde{J}_{c}$ and $\theta_{c}=\arctan(1/\tilde{h}_{c})$.

FM
---
$\alpha$ | Method | $J_{c}$ | $\tilde{h}_{c}$ | $\theta_{c}$ | $\nu$ | $\beta$
1.5 | Ours | $-1.27(3)$ | $4.57(9)$ | $0.22(9)$ | $1.6(5)$ | $0.46(7)$
Ref. 36 | $-1.2107$ | $4.7600$ | $0.2071$ | $1.9911$ | $0.4968$
Theory | - | - | - | $2.0$ | $0.5$
2.5 | Ours | $-2.09(1)$ | $1.76(1)$ | $0.516(9)$ | $1.08(9)$ | $0.19(3)$
Ref. 36 | $-2.0878$ | $1.7631$ | $0.5159$ | $1.1084$ | $0.1880$
6.0 | Ours | $-2.963(1)$ | $1.0242(4)$ | $0.7734(2)$ | $1.00(3)$ | $0.12(3)$
Ref. 36 | $-2.9444$ | $1.0307$ | $0.7703$ | $0.9849$ | $0.1238$
Theory | - | - | - | $1.0$ | $0.125$
AFM
$\alpha$ | Method | $J_{c}$ | $\tilde{h}_{c}$ | $\theta_{c}$ | $\nu$ | $\beta$
2.5 | Ours | $4.7527(6)$ | $0.77449(9)$ | $0.91180(3)$ | $1.17(2)$ | $0.0911(9)$
Ref. 36 | $4.6638$ | $0.7893$ | $0.9026$ | $0.9103$ | $0.1153$
Ref. 40 | $4.6778$ | $0.7869$ | $0.9041$ | $1.01$ | $0.122$
6.0 | Ours | $3.143(2)$ | $0.9655(5)$ | $0.8029(2)$ | $1.04(2)$ | $0.11(2)$
Theory | - | - | - | $1.0$ | $0.125$

Table 2: List with all critical points and exponents obtained with a ViT
ansatz through finite-size scaling analysis (marked as Ours in the Method
column) for different values of $\alpha$ and in both interaction regimes, FM
($J_{c}<0$) and AFM ($J_{c}>0$). Moreover, we show the analytical results when
they exist, at $\alpha\leq\alpha_{MF}=5/3$ and $\alpha=6>\alpha^{*}$, as well
as different values reported in Refs. 36 and 40.

The values presented in Table 2 are comparable to those found in the
literature or indicated by the theory. During our investigation, we observed
that the results worsen for the smallest $\alpha$, which is unexpected since
we are approaching the MF regime where the wave function should be easier to
find. Therefore, we attribute this to finite-size effects. This was confirmed
by simulating longer chains ($N=200$) and observing an improvement in the
results.

###  IV.3 Comparison to the RBM-like solution

We conclude this section by comparing the ViT and RBM-like architectures using
the V-score (15). The results for $\alpha=2.5$ and $N=50$ as a function of $J$
are shown in Figure 6. The main observation is that the ViT architecture
achieves better quantum states across most of the spectrum, including within
the critical regions, which are marked by dashed lines. Additionally, Figure 6
shows that increasing the RBM’s density briefly improves the results, but
ultimately reaches a plateau in the V-score that cannot be surpassed by simply
adding more hidden neurons to the RBM. Notably, the RBM with
$\alpha_{\text{RBM}}=1$ already has more parameters than our ViT architecture,
with $5100$ vs. $1133$, respectively. The only region where all architectures
converge in accuracy is around $J=0$, where the model is trivial. In fact,
numerical tests indicate that in this limit it is favourable to reduce the
token dimension to a single spin, replicating the non-interacting spin
scenario, which in turn helps us understand why the simpler RBM achieves
better accuracy here. Nonetheless, the ViT provides consistent quality results
regardless of the value of $J$ and the phase, whether ordered or not.

Figure 6: Performance comparison between RBM-like models and ViT. Fixing a
maximum training time (3 minutes) for each point, the V-scores obtained by an
RBM-like ansatz with different densities (denoted as $\alpha_{\mathrm{RBM}}$
in the legend) are compared with the ones obtained by our ViT implementation.
The chain size is $N=50$ and the interaction range $\alpha=2.5$. The black
dashed lines mark the critical points obtained by FSSA [see Table 2].

##  V Conclusions

In this work, we explored the application of the Vision Transformer (ViT) to
quantum long-range models, with a focus on the transverse-field Ising model.
Our results affirm the ViT’s capability to handle all interaction regimes,
from short-range to strong long-range interactions. Figure 4 summarizes the
full phase diagram and its comparison with previous results that focus on
specific regions. Additionally, Table 2 quantitatively compares the critical
properties, $J_{c}$, and the critical exponents $\nu$ and $\beta$ with values
reported in the literature.

Throughout this work, our intention has been to employ an economical approach
in terms of the number of hyperparameters and to use single-shot results,
without further curation or cherry-picking. The attention mechanism, a
fundamental aspect of this architectural design, is responsible for
identifying the most significant correlations within a relatively small number
of iterations. Additionally, the remaining MLP layers endow the model with
sufficient expressiveness and versatility, allowing it to produce a solution
for any point within the parameter space. This is further corroborated by
Figure 6, where a comparison was made with an ansatz based on RBMs setting the
maximum training wall time. The ViT outperforms the accuracy obtained in
critical regions by up to two orders of magnitude. Moreover, the use of the
$\operatorname{log-cosh}$ function as a nonlinear activation function at the
end of the core block [cf. Figure 3] was revealed as crucial: its inclusion,
as described in Section III.2, was fundamental in obtaining consistently and
stably accurate states in the deep FM regime.

Our results illustrate the power of modern NN architectures to afford superior
accuracy and efficiency than their more traditional counterparts when used as
ansätze in variational quantum Monte Carlo workflows, and specifically
demonstrate the ability of the ViT to capture important correlations in a
cost-effective manner, opening the door to its use for a wider variety of
problems involving long-range interactions.

## Acknowledgements

The authors acknowledge funding through grants CEX2023-001286-S, from
MCIN/AEI/10.13039/501100011033 and the EU “NextGenerationEU”/PRTR, and
TED2021-131447B-C21, from MCIN/AEI/10.13039/501100011033. We also acknowledge
the Gobierno de Aragón (Grant E09-17R Q-MAD), Quantum Spain and the CSIC
Quantum Technologies Platform PTI-001. This research project was made possible
through the access granted by the Galician Supercomputing Center (CESGA) to
its supercomputing infrastructure. The supercomputer FinisTerrae III and its
permanent data storage system have been funded by the Spanish Ministry of
Science and Innovation, the Galician Government and the European Regional
Development Fund (ERDF). S. R-J. acknowledges financial support from Gobierno
de Aragón through a doctoral fellowship.

##  Appendix A Finite-size scaling analysis

Here we show all the collapses of the curves corresponding to the fluctuations
that are used to perform finite-size scaling analysis (FSSA), as detailed in
the main text, in order to obtain the critical parameters of the system. These
curves are shown in Figure 7. For each value of $\alpha$ considered, $60$
points have been simulated in different windows for $J$. Not every point has
been used to obtain the critical parameters; narrower windows centred on the
critical point and different for each $\alpha$ have been chosen in order to
obtain the best collapses. However, in Figure 7 all the $60$ points are shown.
To get these points a maximum number of $500$ iterations was set in the
training procedure. The schedule for the SR parameter, $\Delta_{\mathrm{sr}}$,
the number of warm-up iterations, $n_{\mathrm{warm}}$, as well as the decay
rate $\gamma$, are all the same as the ones described in Section III.3. The
peak value for the learning rate, $\lambda_{\mathrm{max}}$ goes from $5.0$ for
the training of the smallest chains ($N=50$) up to $12.0$ for the largest
($N=200$) taking intermediate values for the intermediate sizes.

The maximum chain size was $N=150$ spins for all interaction ranges except for
$\alpha=1.5$, where we made a further push to simulate $N=200$ spins to verify
the finite size effects in the MF regime mentioned in the main text.

Figure 7: Collapsed curves using the FSSA technique for the data obtained from
the fluctuations $\langle\hat{m}_{s}^{2}(N,J)\rangle$ for different values of
$\alpha$. In each panel the values of the critical parameters obtained through
the corresponding fit are shown.

## References

  * Orús [2014] R. Orús, A practical introduction to tensor networks: Matrix product states and projected entangled pair states, Ann. Phys. 349, 117–158 (2014).
  * Georges  _et al._ [1996] A. Georges, G. Kotliar, W. Krauth, and M. J. Rozenberg, Dynamical mean-field theory of strongly correlated fermion systems and the limit of infinite dimensions, Rev. Mod. Phys. 68, 13 (1996).
  * Becca and Sorella [2017] F. Becca and S. Sorella,  _Quantum Monte Carlo Approaches for Correlated Systems_ (Cambridge University Press, 2017).
  * Alexeev  _et al._ [2023] Y. Alexeev, M. Amsler, P. Baity, M. A. Barroca, S. Bassini, T. Battelle, D. Camps, D. Casanova, Y. jai Choi, F. T. Chong, C. Chung, C. Codella, A. D. Corcoles, J. Cruise, A. D. Meglio, J. Dubois, I. Duran, T. Eckl, S. Economou, S. Eidenbenz, B. Elmegreen, C. Fare, I. Faro, C. S. Fernández, R. N. B. Ferreira, K. Fuji, B. Fuller, L. Gagliardi, G. Galli, J. R. Glick, I. Gobbi, P. Gokhale, S. de la Puente Gonzalez, J. Greiner, B. Gropp, M. Grossi, E. Gull, B. Healy, B. Huang, T. S. Humble, N. Ito, A. F. Izmaylov, A. Javadi-Abhari, D. Jennewein, S. Jha, L. Jiang, B. Jones, W. A. de Jong, P. Jurcevic, W. Kirby, S. Kister, M. Kitagawa, J. Klassen, K. Klymko, K. Koh, M. Kondo, D. M. Kurkcuoglu, K. Kurowski, T. Laino, R. Landfield, M. Leininger, V. Leyton-Ortega, A. Li, M. Lin, J. Liu, N. Lorente, A. Luckow, S. Martiel, F. Martin-Fernandez, M. Martonosi, C. Marvinney, A. C. Medina, D. Merten, A. Mezzacapo, K. Michielsen, A. Mitra, T. Mittal, K. Moon, J. Moore, M. Motta, Y.-H. Na, Y. Nam, P. Narang, Y. ya Ohnishi, D. Ottaviani, M. Otten, S. Pakin, V. R. Pascuzzi, E. Penault, T. Piontek, J. Pitera, P. Rall, G. S. Ravi, N. Robertson, M. Rossi, P. Rydlichowski, H. Ryu, G. Samsonidze, M. Sato, N. Saurabh, V. Sharma, K. Sharma, S. Shin, G. Slessman, M. Steiner, I. Sitdikov, I.-S. Suh, E. Switzer, W. Tang, J. Thompson, S. Todo, M. Tran, D. Trenev, C. Trott, H.-H. Tseng, E. Tureci, D. G. Valinas, S. Vallecorsa, C. Wever, K. Wojciechowski, X. Wu, S. Yoo, N. Yoshioka, V. W. zhe Yu, S. Yunoki, S. Zhuk, and D. Zubarev, Quantum-centric supercomputing for materials science: A perspective on challenges and future directions (2023), arXiv:2312.09733 [quant-ph] .
  * Changlani  _et al._ [2009] H. J. Changlani, J. M. Kinder, C. J. Umrigar, and G. K.-L. Chan, Approximating strongly correlated wave functions with correlator product states, Phys. Rev. B 80, 245116 (2009).
  * Anders  _et al._ [2006] S. Anders, M. B. Plenio, W. Dür, F. Verstraete, and H.-J. Briegel, Ground-state approximation for strongly interacting spin systems in arbitrary spatial dimension, Phys. Rev. Lett. 97, 107206 (2006).
  * Carleo and Troyer [2017] G. Carleo and M. Troyer, Solving the quantum many-body problem with artificial neural networks, Science 355, 602–606 (2017).
  * Deng  _et al._ [2017] D.-L. Deng, X. Li, and S. Das Sarma, Quantum entanglement in neural network states, Phys. Rev. X 7, 021021 (2017).
  * Glasser  _et al._ [2018] I. Glasser, N. Pancotti, M. August, I. D. Rodriguez, and J. I. Cirac, Neural-network quantum states, string-bond states, and chiral topological states, Phys. Rev. X 8, 011006 (2018).
  * Medvidović and Moreno [2024] M. Medvidović and J. R. Moreno, Neural-network quantum states for many-body physics (2024), arXiv:2402.11014 [cond-mat.dis-nn] .
  * Britton  _et al._ [2012] J. W. Britton, B. C. Sawyer, A. C. Keith, C.-C. J. Wang, J. K. Freericks, H. Uys, M. J. Biercuk, and J. J. Bollinger, Engineered two-dimensional ising interactions in a trapped-ion quantum simulator with hundreds of spins, Nature 484, 489 (2012).
  * Knap  _et al._ [2013] M. Knap, A. Kantian, T. Giamarchi, I. Bloch, M. D. Lukin, and E. Demler, Probing real-space and time-resolved correlation functions with many-body ramsey interferometry, Phys. Rev. Lett. 111, 147205 (2013).
  * Monroe  _et al._ [2021] C. Monroe, W. C. Campbell, L. M. Duan, Z. X. Gong, A. V. Gorshkov, P. W. Hess, R. Islam, K. Kim, N. M. Linke, G. Pagano, P. Richerme, C. Senko, and N. Y. Yao, Programmable quantum simulations of spin systems with trapped ions, Rev. Mod. Phys. 93, 025001 (2021).
  * Browaeys and Lahaye [2020] A. Browaeys and T. Lahaye, Many-body physics with individually controlled rydberg atoms, Nat. Phys. 16, 132 (2020).
  * Mukamel [2008] D. Mukamel, Statistical mechanics of systems with long range interactions, AIP Conf. Proc. 970, 22 (2008).
  * Campa  _et al._ [2009] A. Campa, T. Dauxois, and S. Ruffo, Statistical mechanics and dynamics of solvable models with long-range interactions, Phys. Rep. 480, 57 (2009).
  * Fey and Schmidt [2016] S. Fey and K. P. Schmidt, Critical behavior of quantum magnets with long-range interactions in the thermodynamic limit, Phys. Rev. B 94, 075156 (2016).
  * Defenu  _et al._ [2023] N. Defenu, T. Donner, T. Macrì, G. Pagano, S. Ruffo, and A. Trombettoni, Long-range interacting quantum systems, Rev. Mod. Phys. 95, 035002 (2023).
  * Defenu  _et al._ [2024] N. Defenu, A. Lerose, and S. Pappalardi, Out-of-equilibrium dynamics of quantum many-body systems with long-range interactions, Phys. Rep. 1074, 1 (2024), out-of-equilibrium dynamics of quantum many-body systems with long-range interactions.
  * Koffel  _et al._ [2012] T. Koffel, M. Lewenstein, and L. Tagliacozzo, Entanglement entropy for the long-range ising chain in a transverse field, Phys. Rev. Lett. 109, 267203 (2012).
  * Kuwahara and Saito [2020] T. Kuwahara and K. Saito, Area law of noncritical ground states in 1d long-range interacting systems, Nat. Commun. 11, 4478 (2020).
  * Vodola  _et al._ [2014] D. Vodola, L. Lepori, E. Ercolessi, A. V. Gorshkov, and G. Pupillo, Kitaev chains with long-range pairing, Phys. Rev. Lett. 113, 156402 (2014).
  * Ares  _et al._ [2018] F. Ares, J. G. Esteve, F. Falceto, and A. R. de Queiroz, Entanglement entropy in the long-range kitaev chain, Phys. Rev. A 97, 062301 (2018).
  * Vodola  _et al._ [2015] D. Vodola, L. Lepori, E. Ercolessi, and G. Pupillo, Long-range ising and kitaev models: phases, correlations and edge modes, New J. Phys. 18, 015001 (2015).
  * Vanderstraeten  _et al._ [2018] L. Vanderstraeten, M. VanDamme, H. P. Büchler, and F. Verstraete, Quasiparticles in quantum spin chains with long-range interactions, Phys. Rev. Lett. 121, 090603 (2018).
  * Francica and Dell’Anna [2022] G. Francica and L. Dell’Anna, Correlations, long-range entanglement, and dynamics in long-range kitaev chains, Phys. Rev. B 106, 155126 (2022).
  * Schneider  _et al._ [2021] J. T. Schneider, J. Despres, S. J. Thomson, L. Tagliacozzo, and L. Sanchez-Palencia, Spreading of correlations and entanglement in the long-range transverse ising chain, Phys. Rev. Res. 3, L012022 (2021).
  * Jäger  _et al._ [2020] S. B. Jäger, L. Dell'Anna, and G. Morigi, Edge states of the long-range kitaev chain: An analytical study, Phys. Rev. B 102, 035152 (2020).
  * Viyuela  _et al._ [2016] O. Viyuela, D. Vodola, G. Pupillo, and M. A. Martin-Delgado, Topological massive dirac edge modes and long-range superconducting hamiltonians, Phys. Rev. B 94, 125121 (2016).
  * Saito [2017] H. Saito, Solving the bose–hubbard model with machine learning, J. Phys. Soc. Jpn. 86, 093001 (2017).
  * Choo  _et al._ [2019] K. Choo, T. Neupert, and G. Carleo, Two-dimensional frustrated ${J}_{1}\text{$-$}{J}_{2}$ model studied with neural network quantum states, Phys. Rev. B 100, 125124 (2019).
  * Sharir  _et al._ [2020] O. Sharir, Y. Levine, N. Wies, G. Carleo, and A. Shashua, Deep autoregressive models for the efficient variational simulation of many-body quantum systems, Phys. Rev. Lett. 124, 020503 (2020).
  * Viteritti  _et al._ [2023] L. L. Viteritti, R. Rende, and F. Becca, Transformer variational wave functions for frustrated quantum spin systems, Phys. Rev. Lett. 130, 236401 (2023).
  * Sprague and Czischek [2024] K. Sprague and S. Czischek, Variational monte carlo with large patched transformers, Commun. Phys. 7, 90 (2024).
  * Luo  _et al._ [2023] D. Luo, Z. Chen, K. Hu, Z. Zhao, V. M. Hur, and B. K. Clark, Gauge-invariant and anyonic-symmetric autoregressive neural network for quantum lattice models, Phys. Rev. Res. 5, 013216 (2023).
  * Koziol  _et al._ [2021] J. A. Koziol, A. Langheld, S. C. Kapfer, and K. P. Schmidt, Quantum-critical properties of the long-range transverse-field ising model from quantum monte carlo simulations, Phys. Rev. B 103, 245135 (2021).
  * Zhu  _et al._ [2018] Z. Zhu, G. Sun, W.-L. You, and D.-N. Shi, Fidelity and criticality of a quantum ising chain with long-range interactions, Phys. Rev. A 98, 023607 (2018).
  * Román-Roche  _et al._ [2023] J. Román-Roche, V. Herráiz-López, and D. Zueco, Exact solution for quantum strong long-range models via a generalized hubbard-stratonovich transformation, Phys. Rev. B 108, 165130 (2023).
  * Sun [2017] G. Sun, Fidelity susceptibility study of quantum long-range antiferromagnetic ising chain, Phys. Rev. A 96, 043621 (2017).
  * Kim  _et al._ [2024] J. Kim, D. Kim, and D.-H. Kim, Neural-network quantum-state study of the long-range antiferromagnetic ising chain, Phys. Rev. E 109, 064123 (2024).
  * Campa  _et al._ [2014] A. Campa, T. Dauxois, D. Fanelli, and S. Ruffo,  _Physics of Long-Range Interacting Systems_ (Oxford University Press, 2014).
  * Lazo  _et al._ [2021] E. G. Lazo, M. Heyl, M. Dalmonte, and A. Angelone, Finite-temperature critical behavior of long-range quantum ising models, SciPost Phys. 11, 076 (2021).
  * Campa  _et al._ [2000] A. Campa, A. Giansanti, and D. Moroni, Canonical solution of a system of long-range interacting rotators on a lattice, Phys. Rev. E 62, 303 (2000).
  * Mori [2011] T. Mori, Instability of the mean-field states and generalization of phase separation in long-range interacting systems, Phys. Rev. E 84, 031128 (2011).
  * Mori [2010] T. Mori, Analysis of the exactness of mean-field theory in long-range interacting systems, Phys. Rev. E 82, 060103(R) (2010).
  * Mori [2012a] T. Mori, Microcanonical analysis of exactness of the mean-field theory in long-range interacting systems, J. Stat. Phys. 147, 1020 (2012a).
  * Mori [2012b] T. Mori, Equilibrium properties of quantum spin systems with nonadditive long-range interactions, Phys. Rev. E 86, 021132 (2012b).
  * Dutta and Bhattacharjee [2001] A. Dutta and J. K. Bhattacharjee, Phase transitions in the quantum ising and rotor models with a long-range interaction, Phys. Rev. B 64, 184106 (2001).
  * Defenu  _et al._ [2017] N. Defenu, A. Trombettoni, and S. Ruffo, Criticality and phase diagram of quantum long-range o($n$) models, Phys. Rev. B 96, 104432 (2017).
  * Kolokolov [1990] I. Kolokolov, Functional integration for quantum magnets: New method and new results, Ann. Phys. 202, 165 (1990).
  * Schauß  _et al._ [2015] P. Schauß, J. Zeiher, T. Fukuhara, S. Hild, M. Cheneau, T. Macrì, T. Pohl, I. Bloch, and C. Gross, Crystallization in ising quantum magnets, Science 347, 1455–1458 (2015).
  * Labuhn  _et al._ [2016] H. Labuhn, D. Barredo, S. Ravets, S. de Léséleuc, T. Macrì, T. Lahaye, and A. Browaeys, Tunable two-dimensional arrays of single rydberg atoms for realizing quantum ising models, Nature 534, 667–670 (2016).
  * Sorella and Capriotti [2000] S. Sorella and L. Capriotti, Green function monte carlo with stochastic reconfiguration: An effective remedy for the sign problem, Phys. Rev. B 61, 2599 (2000).
  * Vicentini  _et al._ [2022] F. Vicentini, D. Hofmann, A. Szabó, D. Wu, C. Roth, C. Giuliani, G. Pescia, J. Nys, V. Vargas-Calderón, N. Astrakhantsev, and G. Carleo, NetKet 3: Machine Learning Toolbox for Many-Body Quantum Systems, SciPost Phys. Codebases , 7 (2022).
  * Cybenko [1989] G. Cybenko, Approximation by superpositions of a sigmoidal function, Math. Control Signal. 2, 303 (1989).
  * Hornik  _et al._ [1989] K. Hornik, M. Stinchcombe, and H. White, Multilayer feedforward networks are universal approximators, Neural Networks 2, 359 (1989).
  * Goodfellow  _et al._ [2016] I. J. Goodfellow, Y. Bengio, and A. Courville,  _Deep Learning_ (MIT Press, Cambridge, MA, USA, 2016).
  * Ioffe and Szegedy [2015] S. Ioffe and C. Szegedy, Batch normalization: Accelerating deep network training by reducing internal covariate shift (2015), arXiv:1502.03167 [cs.LG] .
  * [59] J. L. Ba, J. R. Kiros, and G. E. Hinton, Layer normalization, arXiv:1607.06450 [stat.ML], arXiv:1607.06450 [stat.ML] .
  * He  _et al._ [2016] K. He, X. Zhang, S. Ren, and J. Sun, Deep residual learning for image recognition, in  _2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ (2016) pp. 770–778.
  * Chen  _et al._ [2020] D. Chen, F. Hu, G. Nian, and T. Yang, Deep residual learning for nonlinear regression, Entropy 22, 10.3390/e22020193 (2020).
  * Ramachandran  _et al._ [2017] P. Ramachandran, B. Zoph, and Q. V. Le, Searching for activation functions (2017), arXiv:1710.05941 [cs.NE] .
  * Vaswani  _et al._ [2017] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. u. Kaiser, and I. Polosukhin, Attention is all you need, in  _Advances in Neural Information Processing Systems_, Vol. 30, edited by I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (Curran Associates, Inc., 2017).
  * Dosovitskiy  _et al._ [2021] A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, J. Uszkoreit, and N. Houlsby, An image is worth 16x16 words: Transformers for image recognition at scale (2021), arXiv:2010.11929 [cs.CV] .
  * Shaw  _et al._ [2018] P. Shaw, J. Uszkoreit, and A. Vaswani, Self-attention with relative position representations, CoRR abs/1803.02155 (2018), 1803.02155 .
  * Note [1] https://github.com/Theory-and-Simulation-at-INMA/transformer_LR_WF_public/tree/v24.07.1, doi:10.5281/zenodo.12654460, retrieved on 2024-07-04.
  * Heek  _et al._ [2020] J. Heek, A. Levskaya, A. Oliver, M. Ritter, B. Rondepierre, A. Steiner, and M. van Zee, Flax: A neural network library and ecosystem for JAX (2020), http://github.com/google/flax, accessed on 2021-12-09.
  * Bradbury  _et al._ [2018] J. Bradbury, R. Frostig, P. Hawkins, M. J. Johnson, C. Leary, D. Maclaurin, G. Necula, A. Paszke, J. VanderPlas, S. Wanderman-Milne, and Q. Zhang, JAX: composable transformations of Python+NumPy programs (2018), http://github.com/google/jax, accessed on 2021-12-09.
  * DeepMind  _et al._ [2020] DeepMind, I. Babuschkin, K. Baumli, A. Bell, S. Bhupatiraju, J. Bruce, P. Buchlovsky, D. Budden, T. Cai, A. Clark, I. Danihelka, A. Dedieu, C. Fantacci, J. Godwin, C. Jones, R. Hemsley, T. Hennigan, M. Hessel, S. Hou, S. Kapturowski, T. Keck, I. Kemaev, M. King, M. Kunesch, L. Martens, H. Merzic, V. Mikulik, T. Norman, G. Papamakarios, J. Quan, R. Ring, F. Ruiz, A. Sanchez, L. Sartran, R. Schneider, E. Sezener, S. Spencer, S. Srinivasan, M. Stanojević, W. Stokowiec, L. Wang, G. Zhou, and F. Viola, The DeepMind JAX Ecosystem (2020).
  * Hastings  _et al._ [2010] M. B. Hastings, I. González, A. B. Kallin, and R. G. Melko, Measuring renyi entanglement entropy in quantum monte carlo simulations, Phys. Rev. Lett. 104, 157201 (2010).
  * Wu  _et al._ [2023] D. Wu, R. Rossi, F. Vicentini, N. Astrakhantsev, F. Becca, X. Cao, J. Carrasquilla, F. Ferrari, A. Georges, M. Hibat-Allah, M. Imada, A. M. Läuchli, G. Mazzola, A. Mezzacapo, A. Millis, J. R. Moreno, T. Neupert, Y. Nomura, J. Nys, O. Parcollet, R. Pohle, I. Romero, M. Schmid, J. M. Silvester, S. Sorella, L. F. Tocchio, L. Wang, S. R. White, A. Wietek, Q. Yang, Y. Yang, S. Zhang, and G. Carleo, Variational benchmarks for quantum many-body problems (2023), arXiv:2302.04919 [quant-ph] .
  * Newman and Barkema [1999] M. E. Newman and G. T. Barkema,  _Monte Carlo methods in statistical physics_ (Clarendon Press, 1999).
  * Binder  _et al._ [2012] K. Binder, D. M. Ceperley, J.-P. Hansen, M. Kalos, D. Landau, D. Levesque, H. Mueller-Krumbhaar, D. Stauffer, and J.-J. Weis,  _Monte Carlo methods in statistical physics_ , Vol. 7 (Springer Science & Business Media, 2012).
  * Sorge [2015] A. Sorge, pyfssa 0.7.6 (2015).
  * Melchert [2009] O. Melchert, autoscale.py - a program for automatic finite-size scaling analyses: A user’s guide (2009), arXiv:0910.5403 [physics.comp-ph] .