# Orbital-Free Quasi-Density Functional Theory

Carlos L. Benavides-Riveros<EMAIL_ADDRESS>Pitaevskii BEC
Center, CNR-INO and Dipartimento di Fisica, Università di Trento, I-38123
Trento, Italy

###### Abstract

Wigner functions are broadly used to probe non-classical effects in the
macroscopic world. Here we develop an orbital-free functional framework to
compute the 1-body Wigner quasi-probability for both fermionic and bosonic
systems. Since the key variable is a quasi-density, this theory is
particularly well suited to circumvent the problem of finding the Pauli
potential or approximating the kinetic energy in orbital-free density
functional theory. As proof of principle, we find that the universal
functional for the building block of optical lattices results from a
translation, a contraction, and a rotation of the corresponding functional of
the 1-body reduced density matrix, indicating a strong connection between
these functional theories. Furthermore, we relate the concepts of Wigner
negativity and $v$-representability, and find a manifold of ground states with
negative Wigner functions.

Introduction.— Detecting and understanding quantum features at the macroscopic
level is one of the main theoretical and technological challenges of modern
quantum sciences. Nowadays, state-of-the-art experiments can directly observe
non-classical behavior (as quantum superposition) in systems with a truly
macroscopic number of particles, with as many as $10^{16}$ atoms Friedman  _et
al._ (2000); Fein  _et al._ (2019); Wollack  _et al._ (2022); Schrinski  _et
al._ (2023). A powerful theoretical and computational strategy to detect that
quantumness is by directly measuring the system’s corresponding Wigner
function. Although normalized to unity, Wigner functions are quasi-probability
distributions that can take negative values, a phenomenon that has no
classical counterpart. Hence, negativity in the Wigner functions has been
linked to non-classical features of quantum states and is considered a
distinctive signature of quantum entanglement Walschaers  _et al._ (2017);
Dahl  _et al._ (2006); Kenfack and Życzkowski (2004), contextuality Spekkens
(2008); Raussendorf  _et al._ (2017); Howard  _et al._ (2014), quantum
computation Veitch  _et al._ (2012), quantum steering Walschaers  _et al._
(2020); Walschaers (2022), or even quantum gravity Howl  _et al._ (2021).

Due to the exponentially large Hilbert spaces of quantum many-body systems,
finding the corresponding Wigner function is, in general, a computationally
prohibitive task. Yet for identical particles it is possible to circumvent the
Hilbert space’s exponential growth by means of a universal functional of
certain reduced, more manageable, quantities, like, e.g., the density. Based
on the important observation that electronic systems are fully determined by
the ground-state density Hohenberg and Kohn (1964), density functional theory
(DFT) is a prominent methodology in electronic structure calculations, with
applications ranging from quantum chemistry to material science Jones (2015);
Teale  _et al._ (2022). Quite remarkably, orbital-free DFT achieves a
computational linear scaling with the system size Ryczko  _et al._ (2022).
But, unfortunately, from the density alone it is not possible to reconstruct
the Wigner function, and therefore standard DFT is, in general, not suitable
for describing non-classical features of quantum many-body systems.

A recent phase-space formulation of DFT employs, as the central variable, the
1-particle Wigner quasi-density, which is in a one-to-one correspondence with
the respective ground state for interacting many-fermion/boson systems
Blanchard  _et al._ (2012). Its main feature is that the 1-body Wigner
function can be accessed directly, without pre-computing the full wave
function. This Wigner quasi-density functional theory (quasi-DFT) is a
promising theoretical tool to model many-body problems while accounting for
non-classical features, strong interactions, and quantum correlations, with
the same computational cost as standard DFT. As we will show below, the theory
has also the potential of bypassing well-known problems of orbital-free DFT.
To date, however, there are neither orbital-free nor orbital-dependent
functionals for quasi-DFT.

Here, we will obtain equations for the fermionic/bosonic 1-particle quasi-
density. This is, we will argue, the initial step to developing a full
orbital-free framework for Wigner quasi-DFT. As our main result, we will show
that $\omega({\mathbf{r}},\mathbf{p})$, the 1-particle Wigner quasi-density,
satisfies the following, effective, eigen-equation:

$\displaystyle h_{\rm
eff}\star\omega({\mathbf{r}},\mathbf{p})=\omega({\mathbf{r}},\mathbf{p})\star
h_{\rm eff}=\mu\,\omega({\mathbf{r}},\mathbf{p})\,,$ (1)

where $h_{\rm eff}=\tfrac{1}{2}\mathbf{p}^{2}+v_{\rm ext}({\mathbf{r}})+v_{\rm
eff}({\mathbf{r}},\mathbf{p})$, $v_{\rm ext}({\mathbf{r}})$ is the external
potential, $v_{\rm eff}({\mathbf{r}},\mathbf{p})$ is certain effective
potential that we introduce below, and the symbol $\star$ is the so-called
star product of phase-space quantum mechanics.

The paper is organized as follows: First, we review both the orbital-free
functional theories and the Wigner formulation of DFT. Second, we derive an
Euler-Lagrange equation for the 1-body Wigner quasi-density. Next, we derive
an equation using the Moyal product. We then employ the Hubbard model to test
our equations. We conclude with a summary and discuss some implications of our
results. In the Appendix, we provide additional technical details.

Functional theories.— The enormous success of DFT in electronic structure
calculations is mainly due to the existence of a set of self-consistent
1-particle equations that allow for the computation of the density from
1-particle orbitals Kohn and Sham (1965). Although it is much cheaper than
wave-function methods, this Kohn-Sham DFT still has an unfavorably
computational scaling with the cube of the number of electrons Engel and
Dreizler (2011). In turn, orbital-free DFT allows a much more favorable,
linear scaling with the system size Teale  _et al._ (2022); Ryczko  _et al._
(2022), but this computational advantage is counterbalanced by the fact that
the quantum mechanical kinetic energy functional is unknown, and it is written
as a classical, approximate function of the electron density. A parallel
intellectual effort is the 1-particle reduced density matrix functional theory
(1-RDMFT) that exploits the full 1-particle picture of the many-body problem
by seeking a universal functional of the 1-body reduced density matrix
(1-RDM), Gilbert (1975); Donnelly and Parr (1978); Pernal and Giesbertz
(2016), for fermionic Schilling (2018); Piris (2017); Schmidt  _et al._
(2019), bosonic Benavides-Riveros  _et al._ (2020); Liebert and Schilling
(2021); Maciażek (2021); Liebert and Schilling (2023), or relativistic
Rodríguez-Mayorga  _et al._ (2022) interacting particles. Similar to DFT,
1-RDMFT is based on a one-to-one correspondence between the ground state and
its corresponding 1-RDM. Although this theory is in a better position than DFT
to tackle strong correlations Sharma  _et al._ (2013), its broad use has been
hampered by the absence of Kohn-Sham-like equations for the natural orbitals
(i.e., the eigenvectors of the 1-RDM) Pernal and Giesbertz (2016); Piris and
Pernal (2017). It, therefore, comes as no surprise that fermionic 1-RDMFT is
computationally much more demanding than DFT Gebauer  _et al._ (2016).
Surprisingly, to the best of our knowledge, there is no orbital-free
formulation of 1-RDMFT, even though it could boost its applicability.

Phase-space quantum mechanics.— In the phase-space formulation of quantum
mechanics, observables are represented by symbols, i.e., functions of position
${\mathbf{r}}$ and momentum $\mathbf{p}$ coordinates. Out of many choices,
Wigner functions host the most natural representation of quantum mechanics
Wigner (1932). In the classical limit, it turns out to be the phase-space
distributions of statistical mechanics 111There are other phase-space
distributions (e.g., Berezin’s $Q$ or $P$ functions), but the Wigner functions
are the only ones that are real and give the correct marginal probabilities..
In this formulation, quantum operators correspond uniquely to phase-space
classical functions via the Weyl correspondence, while operator products
correspond to $\star$-products. This noncommutative star (twisted or Moyal)
product is commonly defined by the phase-space pseudo-differential operator
Moyal (1949):
$\star\equiv\exp[i\hbar(\reflectbox{$\vec{\reflectbox{$\partial$}}$}_{r}\vec{\partial}_{p}-\reflectbox{$\vec{\reflectbox{$\partial$}}$}_{p}\vec{\partial}_{r})/2]$;
the arrows denote that a given derivative acts only on a function standing on
the left/right. This product is defined by $\mathcal{Q}(f\star
g)=\mathcal{Q}(f)\mathcal{Q}(g)$ where $\mathcal{Q}(f)$ is the quantized
operator version (by the Weyl rule) of the phase-space function $f$ Weyl
(1927). The eigenvalue problem for the Hamiltonian $H$ reads $H\star\texttimes
f_{n}=E_{n}f_{n}=f_{n}\star H$ 222Another important property is that the
integral of the $\star$-multiplication reduces to a plain integral, namely:
$\int d\Omega\,(f\star g)=\int d\Omega\,(g\star f)=\int d\Omega fg$, with
$d\Omega=d^{3}rd^{3}p$..

1-body Wigner quasi-density.— By definition, the 1-body Wigner quasi-densities
are given in terms of the 1-RDM
$\gamma({\mathbf{r}},\sigma;{\mathbf{r}}^{\prime},\sigma^{\prime})$, by the
relation:

$\displaystyle\omega^{\sigma\sigma^{\prime}}({\mathbf{r}},\mathbf{p})=\frac{1}{\pi^{3}}\int\gamma({\mathbf{r}}-{\mathbf{z}},\sigma;{\mathbf{r}}+{\mathbf{z}},\sigma^{\prime})\,e^{2i\mathbf{p}\cdot{\mathbf{z}}}\,d^{3}z\,,$
(2)

where $\sigma\in\\{\uparrow,\downarrow\\}$ are the spin variables. Notice that
the marginal
$\sum_{\sigma}\int\omega^{\sigma\sigma}({\mathbf{r}},\mathbf{p})\,d^{3}p$
gives exactly the density $n({\mathbf{r}})$ (the central object of DFT) 333To
ease the notation we skip from now on the spin indices in $\omega$..

Wigner quasi-DFT.— A generalization of the Hohenberg-Kohn Hohenberg and Kohn
(1964) and Gilbert Gilbert (1975) theorems to Hamiltonians of the form
$H=h+V$, with a fixed two-particle interaction $V$, proves the existence of a
universal Wigner functional $\mathcal{F}_{V}[\omega]$ of the 1-body quasi-
density $\omega$ Blanchard  _et al._ (2012). Indeed, for any choice of the
1-particle phase-space Hamiltonian
$h({\mathbf{r}},\mathbf{p})=\frac{1}{2}\mathbf{p}^{2}+v_{\rm
ext}({\mathbf{r}},\mathbf{p})$ the energy functional:

$\displaystyle\mathcal{E}[\omega]\equiv\int
h({\mathbf{r}},\mathbf{p})\omega({\mathbf{r}},\mathbf{p})d\Omega+\mathcal{F}_{V}[\omega]\geq
E_{\rm gs}\,,$ (3)

is bounded from below by the exact ground-state energy. The equality in Eq.
(3) holds exactly when $\mathcal{E}[\omega]$ is evaluated using the ground-
state 1-body quasi-density $\omega_{\rm gs}$. As in standard DFT, the
functional $\mathcal{F}_{V}[\omega]$ is completely independent of any external
(phase-space) potential $v({\mathbf{r}},\mathbf{p})$. As in 1-RDMFT, it is
also completely independent of the kinetic energy and depends only on the
fixed two-particle interaction $V$. The (universal) functional
$\mathcal{F}_{V}[\omega]$ obeys a constrained-search formulation, by
considering only many-body wave-functions that integrate to the same $\omega$:

$\displaystyle\mathcal{F}_{V}[\omega]=\min_{\Psi\rightarrow\omega}\mbox{$\langle\Psi|$}V\mbox{$|\Psi\rangle$}\,.$
(4)

While the functional is unknown, it is known that it has some better scaling
properties than the functionals in DFT. For instance, by defining,
$\omega_{\lambda}=\omega(\lambda{\mathbf{r}},\lambda^{-1}\mathbf{p})$, one can
show that $\mathcal{F}_{V}[\omega_{\lambda}]=\lambda\mathcal{F}_{V}[\omega]$
Blanchard  _et al._ (2012), a fact that lies in the exact knowledge of the
kinetic energy functional.

Representability condition of $\omega$.— In a quite natural way, 1-body quasi-
densities inherit the representability conditions of the 1-RDM, $\gamma$. Due
to unitary invariance, those can be expressed as conditions on the eigenvalues
of $\gamma$ Coleman (1963). Therefore, it is convenient to use the spectral
representation of $\omega$ (i.e., $\omega=\sum_{i}n_{i}f_{i}$) to find its
representability conditions. In general, $n_{i}\leq 0$. In the case of
fermions:

$\displaystyle\omega\star\omega\leq\omega\,,$ (5)

which is just a consequence of the Pauli exclusion principle 444There are in
the literature other representability conditions for the spin coordinates
Benavides-Riveros and Gracia-Bondía (2013).

Equation for the 1-body quasi-density.— We now exhibit an exact equation for
the phase-space 1-body quasi-density. Let $\mathcal{E}[\omega]$ be the energy
functional of the Wigner function (3), subject to the constraint $\int
d\Omega\,\omega({\mathbf{r}},\mathbf{p})=N$. The $N$-particle phase-space
density which minimizes such a functional is found by applying a functional
derivative of the Lagrangian $\mathcal{E}[\omega]-\mu N$ with respect to
$\omega$, yielding the Euler-Lagrange equation of Wigner quasi-DFT:

$\displaystyle
h({\mathbf{r}},\mathbf{p})+\frac{\delta\mathcal{F}_{V}[\omega]}{\delta\omega({\mathbf{r}},\mathbf{p})}=\mu\,.$
(6)

There is an important consequence of this result. As is well known, one of the
central problems in orbital-free DFT is approximating the kinetic energy
functional in terms of the density Chai and Weeks (2007); Iyengar  _et al._
(2001) or, alternatively, the Pauli potential Finzel (2016); Kraisler and
Schild (2020). It is, indeed, particularly crucial that the Pauli principle be
captured precisely in the kinetic energy. As we can see in Eq. (6), this
important problem is completely absent in the phase-space formalism. First,
the kinetic energy and the external potential are exact, rather simple phase-
space functions, and no approximation is needed. Second, the representability
condition of the Wigner function (5) guarantees that the Pauli principle is
fulfilled. As a consequence, our orbital-free quasi-DFT needs only to
approximate the universal functional $\mathcal{F}_{V}[\omega]$.

$\star$-eigenequation for the phase-space density.— Inspired by the work of
Levy, Perdew and Sahni Levy  _et al._ (1984) we exhibit now an exact
$\star$-eigenequation for the phase-space density. The 1-body Wigner quasi-
density fulfills the following equation:

$\displaystyle\omega\star h_{\rm eff}=h_{\rm eff}\star\omega=\mu\,\omega\,,$
(7)

where $h_{\rm
eff}=h+\delta\mathcal{F}_{V}[\omega]/\delta\omega|_{\omega=\omega_{\rm gs}}$.
The simplicity of this formula can be compared with the one from orbital-free
DFT for $\sqrt{n({\mathbf{r}})}$ March (1986). Noticeably, the formula (7)
allows for a Wigner-Moyal expansion of the phase-space equation for the
density. Indeed, $\sum_{n}\frac{i^{n}\hbar^{n}}{2^{n}n!}h_{\rm
eff}(\reflectbox{$\vec{\reflectbox{$\partial$}}$}_{r}\vec{\partial}_{p}-\reflectbox{$\vec{\reflectbox{$\partial$}}$}_{p}\vec{\partial}_{r})^{n}\omega=\mu\omega$.

Functional realization.— To the best of our knowledge, there are no explicit
functionals of $\omega$. Although 1-RDMFT functionals could be Wigner
transformed, with few exceptions, all of them are written in terms of natural
orbitals Piris (2006); Cioslowski  _et al._ (2023); Piris (2021); Müller
(1984); Pernal (2005); Wang and Baerends (2022); Schmidt  _et al._ (2021);
Gibney  _et al._ (2022); Piris (2018), so they are not suited for our
purposes. Let us, therefore, illustrate the potential of orbital-free quasi-
DFT by discussing the Hubbard dimer, which has been broadly used to unveil
aspects of functional theories Benavides-Riveros  _et al._ (2020); Cohen and
Mori-Sánchez (2016); Schilling and Schilling (2019); Burke and Kozlowski
(2022); Di Sabatino  _et al._ (2021). For 2-fermions in the singlet state or
two bosons, the Hubbard models coincide so we take the following as our
Hamiltonian:

$H=t\left(\hat{b}^{\dagger}_{1}b_{1}-b^{\dagger}_{0}b_{0}\right)+v_{l}\hat{n}_{l}+v_{r}\hat{n}_{r}+U\sum_{j=l,r}\\!\hat{n}_{j}(\hat{n}_{j}-1),$
(8)

where the operators $b^{\dagger}_{n}$ and $b_{n}$ create and annihilate a
particle with momentum $n\in\\{0,1\\}$, and $\hat{n}_{j}$ is the corresponding
particle-number operator on the left/right sites $j\in\\{l,r\\}$. The hybrid
notation of the Hamiltonian is justified by our use of the Wigner function
(i.e., both the hopping term and the external-potential energy are diagonal in
the phase-space representation). Normalizing to 1 and assuming real-valued
matrix elements, the 1-RDM can be represented in the lattice-site basis
$|l\rangle$, $|r\rangle$ as
$\gamma=\left(\tfrac{1}{2}+\vec{\gamma}\cdot\vec{\sigma}\right)$, where
$\vec{\gamma}=(\gamma_{lr},0,\gamma_{ll}-\tfrac{1}{2})$ and
$\vec{\sigma}=(\sigma_{x},\sigma_{y},\sigma_{z})$ is the vector of Pauli
matrices.

$\gamma_{lr}$$\omega_{l,1}$$\gamma_{ll}$$\omega_{l,0}$1-RDMFTWigner quasi-
DFT$\gamma$$\omega$$U>0$

$\gamma_{lr}$$\omega_{l,1}$$\gamma_{ll}$$\omega_{l,0}$$U<0$

Figure 1: Plots and density plots of the universal functionals of 1-RDMFT and
Wigner quasi-DFT for the Hubbard dimer (20) for two particles, for both
positive and negative $U$.

To write the corresponding (discrete) Wigner transformation we follow Refs.
Feynman (1987); Wootters (1987); Schachenmayer  _et al._ (2015); Przanowski
and Tosiek (2017) where the Wigner function is represented on a grid of twice
the dimension of the underlying Hilbert space $\\{j,n\\}$. For the momentum
basis, we choose the one in which the hopping term of (20) is diagonal:
$\mbox{$|n\rangle$}=[\mbox{$|l\rangle$}+(-1)^{n}\mbox{$|r\rangle$}]/\sqrt{2}$
for $n\in\\{0,1\\}$.

The 1-body Wigner quasi-density can now be computed:

$\displaystyle\omega_{j,n}=\frac{1}{2}[\gamma_{jj}+(-1)^{n}\gamma_{lr}]\,.$
(9)

As it should be, the marginal densities are recovered by the partial sums:
$\sum_{n}\omega_{j,n}=\gamma_{jj}$ and
$\sum_{j}\omega_{j,n}=\tilde{\gamma}_{nn}$, where $\tilde{\gamma}_{nn}$ is the
momentum density. Since $\omega_{r,1}=\tfrac{1}{2}-\omega_{l,0}$ and
$\omega_{r,0}=\tfrac{1}{2}-\omega_{l,1}$, we take $\omega_{l,0}$ and
$\omega_{l,1}$ as our two degrees of freedom. It is straightforward to check
that the representability condition reads:

$\displaystyle\left(\omega_{l,0}-\frac{1}{4}\right)^{2}+\left(\omega_{l,1}-\frac{1}{4}\right)^{2}\leq\frac{1}{8}\,.$
(10)

Since this is the equation of a disk of radius $1/\sqrt{8}$ centered in
$(\tfrac{1}{4},\tfrac{1}{4})$, we parameterize the discrete Wigner function
with a radius and an angle, namely,
$\omega_{l,0}(R,\phi)=\tfrac{1}{4}[1+\sqrt{2}R\cos(\phi)]$ and
$\omega_{l,1}(R,\phi)=\tfrac{1}{4}[1+\sqrt{2}R\sin(\phi)]$. By collecting
these results we can finally write an exact phase-space functional

$\displaystyle\mathcal{F}_{\pm}(R,\phi)=\pm\left\\{2-2\left(1\pm\sqrt{1-R^{2}}\right)[\sin(\phi)-\cos(\phi)]^{2}\right\\}\,$

where $+$ indicates $U>0$ and $-$ indicates $U<0$. As sketched in Fig. 1, the
functional of quasi-DFT results from the 1-RDMFT functional after a
translation, a contraction, and a rotation of 45°. This result seems to be
general for lattice systems, as indicated in the Appendix A. After applying
Eq. (6) or (7) one can find $\omega_{\rm gs}$.

Quite remarkably, quasi-DFT can relate two important concepts in Wigner and
functional theories: Wigner negativity and $v$-representability: Which Wigner-
negative 1-body quasi-densities come from ground states? We answer this
question for 2 and 3 bosons in Fig. 2: There are 4 ground-state regions of
Wigner negativity! Relating these two important concepts seems to be new in
the literature.

$\omega_{l,0}$$\omega_{l,0}$$\omega_{l,1}$$N=2$$N=3$Non $v$-repWigner
neg$v$-repWigner neg Figure 2: Representation of the domain of Wigner 1-body
quasi-densities for the Hubbard dimer for 2 and 3 bosons. Wigner positive
$\omega>0$ are represented in black (non $v$-representable) and orange
($v$-representable). Wigner negative $\omega$ are represented in yellow
($v$-representable) and gray (non $v$-representable).

Conclusion.— Unveiling the role of quantum effects at the classical level is a
crucial problem for developing quantum technologies. The Wigner quasi-
probability is usually employed as a probe of such quantumness. This letter
showed that $\omega({\mathbf{r}},\mathbf{p})$, the (fermionic or bosonic)
1-body Wigner quasi-density, can be inserted into a functional-theoretical
framework in an orbital-free manner. By providing a Euler-Lagrange equation
and a Wigner-Moyal eigen-equation, we showed that
$\omega({\mathbf{r}},\mathbf{p})$ can be computed as a stationary point
without referring to orbital equations, circumventing some known problems of
orbital-free DFT (e.g., approximating the kinetic energy functional or finding
the Pauli potential). In this framework, the concepts of Wigner negativity and
$v$-representability can be related. There are several potential research
directions from our results: First, one could develop machine learning quasi-
DFT functionals which is now current practice for standard DFT Grisafi  _et
al._ (2022); Kirkpatrick  _et al._ (2021). Second, since Wigner negativities
carry important quantum information it will be interesting to see what
information they can unveil for electronic correlations Izsák  _et al._
(2023); Zhang and Truhlar (2021) or fermionic entanglement Ptaszyński and
Esposito (2023). Finally, it could be quite promising to tackle —within this
orbital-free framework— quantum excitations in the same spirit of state-
average calculations Benavides-Riveros  _et al._ (2022) or
$\mathbf{w}$-1-RDMFT Schilling and Pittalis (2021).

###### Acknowledgements.

I thank L. Colmenarez, J. Liebert, and J. Maki for insightful discussions and
comments on the manuscript, and gratefully acknowledge the European Union’s
Horizon Europe Research and Innovation program under the Marie Skłodowska-
Curie grant agreement n°101065295.

##  Appendix A Discrete Wigner formalism for the Hubbard model

Here we apply the discrete Wigner formalism to the Hubbard model of $L$ sites.
This is defined in the $L$-dimensional Hilbert space $\mathcal{H}^{L}$ whose
position basis is
$\mathcal{S}=\\{\mbox{$|1\rangle$},...,\mbox{$|L\rangle$}\\}$. Another
orthonormal basis for the same Hilbert space is
$\\{\mbox{$|\phi_{0}\rangle$},...,\mbox{$|\phi_{L-1}\rangle$}\\}$, defined by
the Fourier transform:

$\displaystyle\mbox{$|\phi_{m}\rangle$}=\frac{1}{\sqrt{L}}\sum^{L}_{n=1}e^{in\phi_{m}}\mbox{$|n\rangle$}\,,$
(11)

with $\phi_{m}=\frac{2\pi}{L}m$. The set of pairs $\\{n,\phi_{m}\\}_{n,m}$
constitutes a $L\times L$ grid. This is the phase space $\Gamma^{L}$
associated with the Hilbert space $\mathcal{H}^{L}$ Wootters (1987).

The operators $\hat{n}=\sum_{n}n\mbox{$|n\rangle$}\mbox{$\langle n|$}$ and
$\hat{\phi}=\sum_{m}\phi_{m}\mbox{$|\phi_{m}\rangle$}\mbox{$\langle\phi_{m}|$}$
can be used to construct the following unitary operators:

$\displaystyle\hat{V}=\exp\left(i\frac{2\pi}{L}\hat{n}\right)\qquad{\rm
and}\qquad\hat{U}=\exp(i\hat{\phi})\,,$ (12)

which satisfy the Weyl relation Przanowski and Tosiek (2017):

$\displaystyle\hat{D}(k,l)\equiv\exp\left(-i\frac{\pi
kl}{L}\right)\hat{U}^{k}\hat{V}^{l}=\exp\left(i\frac{\pi
kl}{L}\right)\hat{U}^{l}\hat{V}^{k}\,,\qquad k,l\in\mathbb{Z}\,.$ (13)

With this operator, the authors of Ref. Przanowski and Tosiek (2017) define
the phase-space point operator:

$\displaystyle\hat{\Omega}_{\kappa}(n,\phi_{m})=\frac{1}{L}\sum_{k,l}\kappa(k,l)\hat{D}(k,l)\exp\left[-i\left(k\phi_{m}+\frac{2\pi}{L}ln\right)\right]\,,$
(14)

with a kernel $\kappa(k,l)$, whose properties are determined by the properties
of $\hat{\Omega}_{\kappa}$. In particular, the operator’s hermiticity
condition:
$\hat{\Omega}_{\kappa}(n,\phi_{m})=\hat{\Omega}^{\dagger}_{\kappa}(n,\phi_{m})$,
$\forall(n,\phi_{m})\in\Gamma^{L}$, which is needed to map phase-space
functions to hermitian operators, results in the condition
$\kappa^{*}(k,l)=(-1)^{L+k+l}\kappa(L-k,L-l)$. For odd $L=2N+1$, for instance,
a kernel can be chosen to be Przanowski and Tosiek (2017):
$\kappa(k,l)=\cos(\pi kl/L)$.

The map between $f(n,\phi_{m})$, a real function in $\Gamma^{L}$, and
$\hat{f}$, an operator in $\mathcal{H}^{L}$, is realized by means of the
following relation:

$\displaystyle\hat{f}=\frac{1}{L}\sum_{m,n}f(n,\phi_{m})\hat{\Omega}_{\kappa}(n,\phi_{m})\,.$
(15)

Eq. (15) can now be used to find the Wigner quasi-distribution. Since the
average value of the observable represented by the operator $\hat{f}$ in a
state defined by the density operator $\hat{\gamma}$ reads

$\displaystyle\mbox{Tr}[\hat{\gamma}\hat{f}]=\frac{1}{L}\sum_{m,n}f(n,\phi_{m})\mbox{Tr}\left[\hat{\gamma}\hat{\Omega}_{\kappa}(n,\phi_{m})\right]\,.$
(16)

a natural definition for the Wigner quasi-probability (for the kernel
$\kappa$) arises:

$\omega(n,\phi_{m})=\mbox{Tr}\left[\hat{\gamma}\hat{\Omega}_{\kappa}(n,\phi_{m})\right].$

From this definition, one can write:

$\displaystyle\omega(n,\phi_{m})=\sum_{m^{\prime},n^{\prime}}\mathcal{D}(n,\phi_{m};n^{\prime},m^{\prime})\mbox{$\langle
n^{\prime}|$}\hat{\gamma}\mbox{$|m^{\prime}\rangle$}\,,$ (17)

where

$\displaystyle\mathcal{D}(n,\phi_{m};n^{\prime},m^{\prime})$
$\displaystyle=\frac{1}{L^{2}}\sum_{k,l,s}\kappa(k,l)e^{i\left(\frac{\pi
kl}{L}+k\phi_{s}+n^{\prime}\phi_{s+l}-m^{\prime}\phi_{s}-k\phi_{m}-\frac{2\pi}{L}ln\right)}.$
(18)

If one vectorizes both $\omega$ and $\gamma$ in the following form:

$\displaystyle\mathinner{\left|{\omega}\right\rangle\kern-2.15277pt\rangle}=\begin{pmatrix}\omega(1,\phi_{0})\\\
\omega(1,\phi_{1})\\\ \omega(1,\phi_{2})\\\ \vdots\end{pmatrix}\qquad{\rm
and}\qquad\mathinner{\left|{\gamma}\right\rangle\kern-2.15277pt\rangle}=\begin{pmatrix}\mbox{$\langle
1|$}\gamma\mbox{$|1\rangle$}\\\ \mbox{$\langle 1|$}\gamma\mbox{$|2\rangle$}\\\
\mbox{$\langle 1|$}\gamma\mbox{$|3\rangle$}\\\ \vdots\end{pmatrix}\,,$ (19)

one can formally write (17) as
$\mathinner{\left|{\omega}\right\rangle\kern-2.15277pt\rangle}=\hat{\mathcal{D}}\mathinner{\left|{\gamma}\right\rangle\kern-2.15277pt\rangle}$.

##  Appendix B The Bose-Hubbard dimer

In this section, we focus on the Bose-Hubbard dimer, whose Hamiltonian reads

$H=-t(b_{l}^{\dagger}b_{r}+b_{r}^{\dagger}b_{l})+\\!\sum_{j=l/r}\\!v_{j}\hat{n}_{j}+U\sum_{j=l/r}\\!\hat{n}_{j}(\hat{n}_{j}-1),$
(20)

where the operators $b^{\dagger}_{j}$ and $b_{j}$ create and annihilate a
particle on the sites $j=l/r$, and $\hat{n}_{j}$ is the corresponding
particle-number operator. Any $N$-body ground state of the Hamiltonian (20)
can be expressed as a linear combination of the configuration states
$|n,N-n\rangle$. Assuming real wave functions, we represent the 1-RDM
$\gamma\equiv\mbox{Tr}_{N-1}[\Gamma]$ of any pure or ensemble state with
respect to the lattice site states $\mbox{$|l\rangle$},\mbox{$|r\rangle$}$,

$\displaystyle\gamma_{ij}\equiv\frac{1}{N}\mbox{$\langle\Psi|$}b_{i}^{\dagger}b_{j}\mbox{$|\Psi\rangle$}\,,\qquad
i,j=l,r\,.$ (21)

Since $\gamma_{ll}+\gamma_{rr}=1$ (by normalization) and
$\gamma_{lr}=\gamma_{rl}$ the 1RDM is fully determined by two free parameters.
We represent the 1-RDM as:

$\displaystyle\gamma=\begin{pmatrix}\gamma_{ll}&\gamma_{lr}\\\
\gamma_{lr}&1-\gamma_{ll}\end{pmatrix}\,.$ (22)

The only two degrees of freedom of this matrix can be represented in a vector
form:
$\mathinner{\left|{\gamma}\right\rangle\kern-2.15277pt\rangle}=\begin{pmatrix}\gamma_{ll}\\\
\gamma_{lr}\end{pmatrix}$.

Our goal is to find the Wigner function associated with this matrix in the
grid $\\{(l,0),(l,1),(r,0),(r,1)\\}$, with the momentum basis:
$\mbox{$|0\rangle$}\equiv\frac{1}{\sqrt{2}}(\mbox{$|L\rangle$}+\mbox{$|R\rangle$})$
and
$\mbox{$|1\rangle$}\equiv\frac{1}{\sqrt{2}}(\mbox{$|L\rangle$}-\mbox{$|R\rangle$})$.
This grid can be seen as a two-dimensional vector space over a finite field,
in which the Wigner function is defined:

$\begin{array}[]{c@{}c@{}c}\begin{matrix}&&|&&\\\
&\omega(l,1)&|&\omega(r,1)&\\\ &&|&&\end{matrix}\\\
\hline\cr\begin{matrix}&&|&&\\\ &\omega(l,0)&|&\omega(r,0)&\\\
&&|&&\end{matrix}\end{array}$

Notice that $\gamma$ can be written as

$\displaystyle\gamma=\left(\frac{1}{2}+\vec{\gamma}\cdot\vec{\sigma}\right)\,,$
(23)

where $\vec{\gamma}=(\gamma_{lr},0,\gamma_{ll}-\tfrac{1}{2})$ and
$\vec{\sigma}=(\sigma_{x},\sigma_{y},\sigma_{z})$ are the Pauli matrices. For
a qubit a phase-space point operator is known to be Przanowski and Tosiek
(2017):

$\displaystyle\Omega(n,\phi_{m})$
$\displaystyle=\frac{1}{2}[1+(-1)^{m}(\mbox{$|0\rangle$}\mbox{$\langle
0|$})-\mbox{$|1\rangle$}\mbox{$\langle 1|$})$
$\displaystyle+(-1)^{n}(\mbox{$|0\rangle$}\mbox{$\langle
1|$})+\mbox{$|1\rangle$}\mbox{$\langle
0|$})+i(-1)^{n+m}(\mbox{$|0\rangle$}\mbox{$\langle
1|$})-\mbox{$|1\rangle$}\mbox{$\langle 0|$})]\,.$ (24)

Therefore, by computing
$\omega(n,\phi_{m})=\frac{1}{4}\mbox{Tr}[\gamma\Omega(n,\phi_{m})]$ one finds
in vectorized form the following equation:

$\displaystyle\mathinner{\left|{\omega}\right\rangle\kern-2.15277pt\rangle}=\tfrac{1}{2}\mathcal{R}\mathinner{\left|{\gamma}\right\rangle\kern-2.15277pt\rangle}\,,$
(25)

where

$\displaystyle\mathinner{\left|{\omega}\right\rangle\kern-2.15277pt\rangle}=\begin{pmatrix}\omega(l,0)\\\
\omega(l,1)\end{pmatrix}\qquad{\rm and}\qquad\mathcal{R}=\begin{pmatrix}1&1\\\
1&-1\end{pmatrix}$ (26)

with $\omega(r,1)=\tfrac{1}{2}-\omega(l,0)$ and
$\omega(r,0)=\tfrac{1}{2}-\omega(l,1)$. Notice that $\mathcal{R}$ is an
orthogonal matrix. Inverting (27) one gets

$\displaystyle\mathinner{\left|{\gamma(\omega)}\right\rangle\kern-3.75pt\rangle}=\mathcal{R}\mathinner{\left|{\omega}\right\rangle\kern-2.15277pt\rangle}\,.$
(27)

###  B.1 Representability

Since $\gamma^{2}\leq\gamma$, the domain of pure/ensemble $N$-representable
1RDMs takes the form of a disc of radius $\tfrac{1}{2}$ given by

$\left(\gamma_{ll}-\frac{1}{2}\right)^{2}+\gamma_{lr}^{2}\leq\frac{1}{4}.$
(28)

The boundary of this disk (i.e.,
$\gamma_{LR}^{2}+(\gamma_{LL}-\tfrac{1}{2})^{2}=\tfrac{1}{4}$) corresponds to
complete Bose-Einstein condensation (BEC) Benavides-Riveros  _et al._ (2020).
Plugging (27) in (28) one gets:

$\left(\omega_{l,0}-\frac{1}{4}\right)^{2}+\left(\omega_{l,1}-\frac{1}{4}\right)^{2}\leq\frac{1}{8}.$
(29)

## References

  * Friedman  _et al._ (2000) J. R. Friedman, V. Patel, W. Chen, S. K. Tolpygo, and J. E. Lukens, “Quantum superposition of distinct macroscopic states,” Nature 406, 43 (2000).
  * Fein  _et al._ (2019) Y. Y. Fein, P. Geyer, P. Zwick, F. Kiałka, S. Pedalino, M. Mayor, S. Gerlich, and M. Arndt, “Quantum superposition of molecules beyond 25 kDa,” Nature Phys. 15, 1242 (2019).
  * Wollack  _et al._ (2022) E. A. Wollack, A. Y. Cleland, R. G. Gruenke, Z. Wang, P. Arrangoiz-Arriola, and A. H. Safavi-Naeini, “Quantum state preparation and tomography of entangled mechanical resonators,” Nature 604, 463 (2022).
  * Schrinski  _et al._ (2023) B. Schrinski, Y. Yang, U. von Lüpke, M. Bild, Y. Chu, K. Hornberger, S. Nimmrichter, and M. Fadel, “Macroscopic quantum test with bulk acoustic wave resonators,” Phys. Rev. Lett. 130, 133604 (2023).
  * Walschaers  _et al._ (2017) M. Walschaers, C. Fabre, V. Parigi, and N. Treps, “Entanglement and Wigner Function Negativity of Multimode Non-Gaussian States,” Phys. Rev. Lett. 119, 183601 (2017).
  * Dahl  _et al._ (2006) J. P. Dahl, H. Mack, A. Wolf, and W. P. Schleich, “Entanglement versus negative domains of Wigner functions,” Phys. Rev. A 74, 042323 (2006).
  * Kenfack and Życzkowski (2004) A. Kenfack and K. Życzkowski, “Negativity of the Wigner function as an indicator of non-classicality,” J. Opt. B: Quantum and Semiclass. Opt. 6, 396 (2004).
  * Spekkens (2008) R. W. Spekkens, “Negativity and Contextuality are Equivalent Notions of Nonclassicality,” Phys. Rev. Lett. 101, 020401 (2008).
  * Raussendorf  _et al._ (2017) R. Raussendorf, D. E. Browne, N. Delfosse, C. Okay, and J. Bermejo-Vega, “Contextuality and Wigner-function negativity in qubit quantum computation,” Phys. Rev. A 95, 052334 (2017).
  * Howard  _et al._ (2014) M. Howard, J. Wallman, V. Veitch, and J. Emerson, “Contextuality supplies the ‘magic’ for quantum computation,” Nature 510, 351 (2014).
  * Veitch  _et al._ (2012) V. Veitch, C. Ferrie, D. Gross, and J. Emerson, “Negative quasi-probability as a resource for quantum computation,” New J. Phys. 14, 113011 (2012).
  * Walschaers  _et al._ (2020) M. Walschaers, V. Parigi, and N. Treps, “Practical Framework for Conditional Non-Gaussian Quantum State Preparation,” PRX Quantum 1, 020305 (2020).
  * Walschaers (2022) M. Walschaers, “On Quantum Steering and Wigner Negativity,” (2022), arXiv:2211.14102 [quant-ph] .
  * Howl  _et al._ (2021) R. Howl, V. Vedral, D. Naik, M. Christodoulou, C. Rovelli, and A. Iyer, “Non-Gaussianity as a Signature of a Quantum Theory of Gravity,” PRX Quantum 2, 010325 (2021).
  * Hohenberg and Kohn (1964) P. Hohenberg and W. Kohn, “Inhomogeneous electron gas,” Phys. Rev. 136, B864 (1964).
  * Jones (2015) R. O. Jones, “Density functional theory: Its origins, rise to prominence, and future,” Rev. Mod. Phys. 87, 897 (2015).
  * Teale  _et al._ (2022) A. M. Teale _et al._ , “DFT exchange: sharing perspectives on the workhorse of quantum chemistry and materials science,” Phys. Chem. Chem. Phys. 24, 28700 (2022).
  * Ryczko  _et al._ (2022) K. Ryczko, S. J. Wetzel, R. G. Melko, and I. Tamblyn, “Toward Orbital-Free Density Functional Theory with Small Data Sets and Deep Learning,” J. Chem. Theory Comput. 18, 1122 (2022).
  * Blanchard  _et al._ (2012) P. Blanchard, J. M. Gracia-Bondía, and J. C. Várilly, “Density functional theory on phase space,” Int. J. Quantum Chem. 112, 1134 (2012).
  * Kohn and Sham (1965) W. Kohn and L. J. Sham, “Self-consistent equations including exchange and correlation effects,” Phys. Rev. 140, A1133 (1965).
  * Engel and Dreizler (2011) E. Engel and R.M. Dreizler,  _Density Functional Theory: An Advanced Course_, Theoretical and Mathematical Physics (Springer Berlin Heidelberg, 2011).
  * Gilbert (1975) T. L. Gilbert, “Hohenberg-Kohn theorem for nonlocal external potentials,” Phys. Rev. B 12, 2111 (1975).
  * Donnelly and Parr (1978) R. A. Donnelly and R. G. Parr, “Elementary properties of an energy functional of the first‐order reduced density matrix,” J. Chem. Phys. 69, 4431 (1978).
  * Pernal and Giesbertz (2016) K. Pernal and K. J. H. Giesbertz, “Reduced Density Matrix Functional Theory (RDMFT) and Linear Response Time-Dependent RDMFT (TD-RDMFT),” in  _Density-Functional Methods for Excited States_, edited by Nicolas Ferré, Michael Filatov, and Miquel Huix-Rotllant (Springer International Publishing, Cham, 2016) p. 125.
  * Schilling (2018) C. Schilling, “Communication: Relating the pure and ensemble density matrix functional,” J. Chem. Phys. 149, 231102 (2018).
  * Piris (2017) M. Piris, “Global method for electron correlation,” Phys. Rev. Lett. 119, 063002 (2017).
  * Schmidt  _et al._ (2019) J. Schmidt, C. L. Benavides-Riveros, and M. A. L. Marques, “Reduced density matrix functional theory for superconductors,” Phys. Rev. B 99, 224502 (2019).
  * Benavides-Riveros  _et al._ (2020) C. L. Benavides-Riveros, J. Wolff, M. A. L. Marques, and C. Schilling, “Reduced Density Matrix Functional Theory for Bosons,” Phys. Rev. Lett. 124, 180603 (2020).
  * Liebert and Schilling (2021) J. Liebert and C. Schilling, “Functional theory for Bose-Einstein condensates,” Phys. Rev. Res. 3, 013282 (2021).
  * Maciażek (2021) T. Maciażek, “Repulsively diverging gradient of the density functional in the reduced density matrix functional theory,” New J. Phys. 23, 113006 (2021).
  * Liebert and Schilling (2023) J. Liebert and C. Schilling, “An exact one-particle theory of bosonic excitations: from a generalized Hohenberg–Kohn theorem to convexified N-representability,” New J. Phys. 25, 013009 (2023).
  * Rodríguez-Mayorga  _et al._ (2022) M. Rodríguez-Mayorga, K. J.H. Giesbertz, and L. Visscher, “Relativistic reduced density matrix functional theory.” SciPost Chem. 1, 004 (2022).
  * Sharma  _et al._ (2013) S. Sharma, J. K. Dewhurst, S. Shallcross, and E. K. U. Gross, “Spectral density and metal-insulator phase transition in mott insulators within reduced density matrix functional theory,” Phys. Rev. Lett. 110, 116403 (2013).
  * Piris and Pernal (2017) M. Piris and K. Pernal, “Comment on “Generalization of the Kohn-Sham system that can represent arbitrary one-electron density matrices”,” Phys. Rev. A 96, 046501 (2017).
  * Gebauer  _et al._ (2016) R. Gebauer, M. H. Cohen, and R. Car, “A well-scaling natural orbital theory,” Proc. Natl. Acad. Sci. U.S.A. 113, 12913 (2016).
  * Wigner (1932) E. Wigner, “On the quantum correction for thermodynamic equilibrium,” Phys. Rev. 40, 749 (1932).
  * Note (1) There are other phase-space distributions (e.g., Berezin’s $Q$ or $P$ functions), but the Wigner functions are the only ones that are real and give the correct marginal probabilities.
  * Moyal (1949) J. E. Moyal, “Quantum mechanics as a statistical theory,” Math. Proc. Camb. Philos. Soc. 45, 99 (1949).
  * Weyl (1927) H. Weyl, “Quantenmechanik und gruppentheorie,” Z. Phys. 46, 1–46 (1927).
  * Note (2) Another important property is that the integral of the $\star$-multiplication reduces to a plain integral, namely: $\intop\ilimits@d\Omega\,(f\star g)=\intop\ilimits@d\Omega\,(g\star f)=\intop\ilimits@d\Omega fg$, with $d\Omega=d^{3}rd^{3}p$.
  * Note (3) To ease the notation we skip from now on the spin indices in $\omega$.
  * Coleman (1963) A. J. Coleman, “Structure of fermion density matrices,” Rev. Mod. Phys. 35, 668–686 (1963).
  * Note (4) There are in the literature other representability conditions for the spin coordinates Benavides-Riveros and Gracia-Bondía (2013).
  * Chai and Weeks (2007) J.-D. Chai and J. D. Weeks, “Orbital-free density functional theory: Kinetic potentials and ab initio local pseudopotentials,” Phys. Rev. B 75, 205122 (2007).
  * Iyengar  _et al._ (2001) S. S. Iyengar, M. Ernzerhof, S. N. Maximoff, and G. E. Scuseria, “Challenge of creating accurate and effective kinetic-energy functionals,” Phys. Rev. A 63, 052508 (2001).
  * Finzel (2016) K. Finzel, “Local conditions for the Pauli potential in order to yield self-consistent electron densities exhibiting proper atomic shell structure,” J. Chem. Phys. 144, 034108 (2016).
  * Kraisler and Schild (2020) E. Kraisler and A. Schild, “Discontinuous behavior of the Pauli potential in density functional theory as a function of the electron number,” Phys. Rev. Res. 2, 013159 (2020).
  * Levy  _et al._ (1984) M. Levy, J. P. Perdew, and V. Sahni, “Exact differential equation for the density and ionization energy of a many-particle system,” Phys. Rev. A 30, 2745 (1984).
  * March (1986) N. H. March, “The local potential determining the square root of the ground-state electron density of atoms and molecules from the Schrödinger equation,” Phys. Lett. A 113, 476 (1986).
  * Piris (2006) M. Piris, “A new approach for the two-electron cumulant in natural orbital functional theory,” Int. J. Quantum Chem. 106, 1093–1104 (2006).
  * Cioslowski  _et al._ (2023) J. Cioslowski, C. Schilling, and R. Schilling, “1-Matrix functional for long-range interaction energy of two hydrogen atoms,” J. Chem. Phys. 158, 084106 (2023).
  * Piris (2021) M. Piris, “Global Natural Orbital Functional: Towards the Complete Description of the Electron Correlation,” Phys. Rev. Lett. 127, 233001 (2021).
  * Müller (1984) A. M. K. Müller, “Explicit approximate relation between reduced two- and one-particle density matrices,” Phys. Lett. A 105, 446 (1984).
  * Pernal (2005) K. Pernal, “Effective potential for natural spin orbitals,” Phys. Rev. Lett. 94, 233002 (2005).
  * Wang and Baerends (2022) J. Wang and E. J. Baerends, “Self-Consistent-Field Method for Correlated Many-Electron Systems with an Entropic Cumulant Energy,” Phys. Rev. Lett. 128, 013001 (2022).
  * Schmidt  _et al._ (2021) J. Schmidt, M. Fadel, and C. L. Benavides-Riveros, “Machine learning universal bosonic functionals,” Phys. Rev. Res. 3, L032063 (2021).
  * Gibney  _et al._ (2022) D. Gibney, J.-N. Boyn, and D. A. Mazziotti, “Density Functional Theory Transformed into a One-Electron Reduced-Density-Matrix Functional Theory for the Capture of Static Correlation,” J. Phys. Chem. Lett. 13, 1382 (2022).
  * Piris (2018) M. Piris, “Dynamic electron-correlation energy in the natural-orbital-functional second-order-Møller-Plesset method from the orbital-invariant perturbation theory,” Phys. Rev. A 98, 022504 (2018).
  * Cohen and Mori-Sánchez (2016) A. J. Cohen and P. Mori-Sánchez, “Landscape of an exact energy functional,” Phys. Rev. A 93, 042511 (2016).
  * Schilling and Schilling (2019) C. Schilling and R. Schilling, “Diverging Exchange Force and Form of the Exact Density Matrix Functional,” Phys. Rev. Lett. 122, 013001 (2019).
  * Burke and Kozlowski (2022) K. Burke and J. Kozlowski, “Lies My Teacher Told Me About Density Functional Theory: Seeing Through Them with the Hubbard Dimer,” (2022), arXiv:2108.11534 .
  * Di Sabatino  _et al._ (2021) S. Di Sabatino, P.-F. Loos, and P. Romaniello, “Scrutinizing GW-Based Methods Using the Hubbard Dimer,” Fron. Chem. 9, 751054 (2021).
  * Feynman (1987) R. Feynman, “Negative probabilities,” in  _Quantum Implications: Essays in Honour of David Bohm_ , edited by B. Hiley and F. Peat (Routledge, 1987) p. 235.
  * Wootters (1987) W. K. Wootters, “A Wigner-function formulation of finite-state quantum mechanics,” Ann. Phys. 176, 1 (1987).
  * Schachenmayer  _et al._ (2015) J. Schachenmayer, A. Pikovski, and A. M. Rey, “Many-Body Quantum Spin Dynamics with Monte Carlo Trajectories on a Discrete Phase Space,” Phys. Rev. X 5, 011022 (2015).
  * Przanowski and Tosiek (2017) M. Przanowski and J. Tosiek, “From the discrete Weyl–Wigner formalism for symmetric ordering to a number–phase Wigner function,” J. Math. Phys. 58, 102106 (2017).
  * Grisafi  _et al._ (2022) A. Grisafi, A. M. Lewis, M. Rossi, and M. Ceriotti, “Electronic-Structure Properties from Atom-Centered Predictions of the Electron Density,” J. Chem. Theory Comput. (2022), 10.1021/acs.jctc.2c00850.
  * Kirkpatrick  _et al._ (2021) J. Kirkpatrick _et al._ , “Pushing the frontiers of density functionals by solving the fractional electron problem,” Science 374, 1385 (2021).
  * Izsák  _et al._ (2023) R Izsák, A. V. Ivanov, N. S. Blunt, N. Holzmann, and F. Neese, “Measuring Electron Correlation: The Impact of Symmetry and Orbital Transformations,” J. Chem. Theory Comput. (2023), 10.1021/acs.jctc.3c00122.
  * Zhang and Truhlar (2021) D. Zhang and D. G. Truhlar, “Decomposition of the Electronic Energy in Terms of Density, Density Coherence, and the Connected Part of the Two-Body Reduced Density Matrix,” J. Chem. Theory Comput. 17, 5733 (2021).
  * Ptaszyński and Esposito (2023) K. Ptaszyński and M. Esposito, “Fermionic One-Body Entanglement as a Thermodynamic Resource,” Phys. Rev. Lett. 130, 150201 (2023).
  * Benavides-Riveros  _et al._ (2022) C. L. Benavides-Riveros, L. Chen, C. Schilling, S. Mantilla, and S. Pittalis, “Excitations of Quantum Many-Body Systems via Purified Ensembles: A Unitary-Coupled-Cluster-Based Approach,” Phys. Rev. Lett. 129, 066401 (2022).
  * Schilling and Pittalis (2021) C. Schilling and S. Pittalis, “Ensemble Reduced Density Matrix Functional Theory for Excited States and Hierarchical Generalization of Pauli’s Exclusion Principle,” Phys. Rev. Lett. 127, 023001 (2021).
  * Benavides-Riveros and Gracia-Bondía (2013) C. L. Benavides-Riveros and J. M. Gracia-Bondía, “Physical Wigner functions,” Phys. Rev. A 87, 022118 (2013).


# CARLS: Cross-platform Asynchronous Representation Learning System

Chun-Ta Lu∗, Yun Zeng∗, Da-Cheng Juan∗, Yicheng Fan, Zhe Li, Jan Dlabal, Yi-
Ting Chen, Arjun Gopalan, Allan Heydon, Chun-Sung Ferng, Reah Miyara, Ariel
Fuxman, Futang Peng, Zhen Li, Tom Duerig, Andrew Tomkins  Google
ResearchMountain ViewCA
<EMAIL_ADDRESS>
###### Abstract.

††footnotetext: * These authors contributed equally.

Innovation on learning systems has been a key propellant in advancing and
scaling up state-of-the-art neural models. In this work, we propose CARLS, a
novel framework for augmenting the capacity of existing deep learning
frameworks by enabling multiple components—model trainers, knowledge makers
and knowledge banks—to concertedly work together in an asynchronous fashion
across hardware platforms. The proposed CARLS is particularly suitable for
learning paradigms where model training benefits from additional knowledge
inferred or discovered during training, such as node embeddings for graph
neural networks or reliable pseudo labels from model predictions. We also
describe three learning paradigms—semi-supervised learning, curriculum
learning and multimodal learning—as examples that can be scaled up efficiently
by CARLS. One version of CARLS has been open-sourced and available for
download at:
github.com/tensorflow/neural-structured-learning/tree/master/research/carls.

##  1\. Introduction

During the past decade, several deep learning frameworks (Collobert et al.,
2002; Abadi et al., 2016; PyTorch, 2018) have been made available to public
and accelerating the advancement of many research domains in deep learning
(Goodfellow et al., 2016; LeCun et al., 2015; Esteva et al., 2019). Despite
their successes, cutting-edge technology and emerging applications are always
pushing the limits of training capacity supported by the existing frameworks.
For example, the largest T5 model (Raffel et al., 2020) with 11 billion
parameters has achieved state-of-the-art performance on multiple NLP
benchmarks, and a near-human score on the SuperGLUE natural language
understanding benchmark. GPT-3 (Brown et al., 2020), an auto-regressive
language model with 175 billion parameters, achieves strong performance on
many NLP benchmarks without any gradient updates or fine-tuning. Most
recently, the switch transformer (Fedus et al., 2021) advanced the size of the
T5 model to $7\times$ times, leading to a giant model speaking $102$ different
languages.

While most of the above efforts focus on training one monolithic model, human
brains often solve a cognition problem using multiple pathways with multi-
sensory inputs and multiple models (Kandel et al., 2000). Such a multimodal
framework has already shown its potential in boosting the performance over the
existing monolithic models (e.g., (Lu et al., 2020a; Radford et al., [n. d.];
Jia et al., 2021)) and in solving novel problems (Graves et al., 2016). For
example, for vision-language related problems such as visual question
answering (Antol et al., 2015), caption-based image retrieval (Young et al.,
2014), and visual common sense reasoning (Zellers et al., 2019), the vision
and language modalities are encoded by its own designated encoder and the two
encoders are co-trained jointly. However, implementing these multimodal
learning tasks at scale imposes unique challenges, since the computational
complexity usually grows with the number of encoders trained, and the
advancement of hardware alone cannot keep up with the fast growth of model (or
data) size and complexity. Such a scaling issue manifests itself in other
machine learning paradigms such as graph-based neural networks (Kipf and
Welling, 2016; Veličković et al., 2017; Bui et al., 2018), where each node in
a graph is processed by a kernel that can be a simple multilayer perceptron or
a complex attention model like BERT (Devlin et al., 2018b). Furthermore, the
computational cost grows linearly with the size of neighborhoods if no
sampling technique is used.

Figure 1. System Overview. CARLS employs three major components: Model
Trainer, Knowledge Maker and Knowledge Bank. Models Trainers are the main jobs
training and updating model parameters, and Knowledge Makers are running in
parallel to compute auxiliary information required for gradient calculation
and store the information in Knowledge Bank. Note that these components can be
deployed on different platforms (GPUs, TPUs, etc) and communicate with each
other in an asynchronous fashion.

In this work, we propose _CARLS_ : a Cross-platform, Asynchronous
Representation Learning System that addresses the challenges in scaling up
these “giant” models (e.g., multimodal learning or graph neural networks).
CARLS is able to (a) accommodate training of very large models with extremely
high computational complexity—e.g., graph neural networks with a graph of more
than ten billion nodes, each node encoded by a BERT kernel—and (b) accelerate
the training by shifting partial workload of inferring information required
for gradient calculation from model trainers to “Knowledge Makers” deployed in
CARLS.

CARLS contains three major components that operate in parallel: _Model
Trainer_ , _Knowledge Maker_ and _Knowledge Bank_. Figure 1 illustrates an
overview of the proposed CARLS system. Model trainers are the main training
jobs with an extended functionality to retrieve auxiliary information
generated by the knowledge makers that are usually implemented by a fleet of
servers or pipelines. The model trainers and knowledge makers share the
information via the knowledge bank, which stores the results generated by
knowledge makers and enables efficient, asynchronous training. Model trainers
and knowledge makers may be deployed using different hardware platforms (e.g.,
CPUs, GPUs or TPUs). Furthermore, model trainers and knowledge makers are
communicating in an asynchronous fashion, meaning they operate independently
without affecting the training speed.

The key observation to design CARLS for large-scale, efficient learning is
that the compute can be parallelized at model-level (or logic-level) for many
neural architectures; in other words, the interactions among multiple models
are usually limited to a few “connecting points” such as the joint of each
encoder output, or the aggregation of neighborhood embeddings in a graph
neural network. By employing the knowledge banks as the interface between
individual modules, CARLS delegates the computation cost of the individual
models from the model trainer to the knowledge makers. One potential issue of
such an asynchronous mechanism is data freshness—some knowledge makers may
generate results based on slightly outdated information. In practice, we find
the impacts of such an issue are controllable and not significant. In summary,
the main advantages of the proposed CARLS are two-fold:

  * •

Scalability: CARLS enables efficient operations to store and lookup
intermediate knowledge for training complex models. For each training step,
the intermediate knowledge can be fetched from a remote knowledge bank, rather
than being recomputed across multiple training batches.

  * •

Flexibility: CARLS allows constructing and updating the knowledge dynamically
based on the current model state at each training step, as opposed to
information preprocessed from a static input dataset or by a pretrained model.

The proposed CARLS system is validated in a number of learning tasks which are
considered to be very challenging or even infeasible before CARLS. For
example, training a graph neural network (GNN) stacked on top of a dense
encoder (e.g., ResNet (He et al., 2015) or Transformer (Vaswani et al.,
2017)), where each node features are encoded by the encoder and aggregated by
a GNN, the training time becomes prohibitively expensive especially when the
neighbor size grows large. With CARLS, we are able to train a graph-
regularized model whose neighbor size is $10$ times larger than the largest
one currently proposed by the literature (Juan et al., 2020), without
introducing any slowdown in the training speed. We also demonstrate three
learning paradigms as examples that can be scaled up by CARLS, including semi-
supervised learning, curriculum learning, and multimodal learning.

The remainder of this paper is organized as follows: after reviewing related
work in Section 2, we elaborate the design of CARLS in Section 3. In Section
4, we demonstrate the learning paradigms that can be scaled up efficiently by
CARLs. Finally, we conclude this paper in Section 5.

##  2\. Related Work

Large-scale learning systems (e.g., models with more than 10G parameters) are
often needed in language/image modeling, recommendation systems, or models
with very large sparse feature set input (e.g., click through rate prediction
models in digital advertising (Zhou et al., 2018; Zhao et al., 2019)).
Directly relying on the design of existing deep learning libraries (e.g., (Li
et al., 2014; Abadi et al., 2016)) is often limited in practice and challenges
exist both in training and serving these large models.

Existing large language/image models are characterized by their sophisticated
internal building blocks (e.g., the self-attention mechanism (Vaswani et al.,
2017) or the residual network (He et al., 2015)) that are often stacked up to
hundreds of layers (e.g., (Devlin et al., 2018a; Radford et al., 2018, 2019;
Brown et al., 2020)). Solely relying on the innovations of hardware (e.g., GPU
or TPU) can hardly keep up with the explosion of model size. To efficiently
train these models, parallelisms/sparsity among the partitions of the model
are often explored (e.g., (Narayanan et al., 2019; Lepikhin et al., 2020)).
The mixture-of-experts (MoE) model is an example model with sparsity that
allows different sub-models to be trained in parallel (Shazeer et al., 2017;
Narayanan et al., 2019). Most recently, the switch transformer (Fedus et al.,
2021) is able to handle models with 1T parameters by solving a big MoE
problem. In order to serve these large models in small devices or more
efficiently, the distillation technique (Hinton et al., 2015) or teacher-
student networks are usually applied to compress the model.

Among industrial scale recommendation systems, the number of unique items
(e.g., keywords, products) can easily reach up to billions or even trillions,
which are usually represented as the softmax/logistic output layer of a model.
Since these output layers can be regarded as nearest neighbor search in an
embedding space (one embedding for each item), a popular approach to efficient
model training and inference is to compress the output embeddings via various
hashing techniques (Medini et al., 2019; Song et al., 2020; Zhang et al.,
2018; Tagami, 2017). However, these compression/approximation approaches often
come with the price of sacrificing modeling accuracy.

There are also more recent interests in decoupling the memory part (e.g.,
sparse feature embeddings for both input and output layers) of a model from
its structural part (e.g., (Zhao et al., 2019)), therefore achieving efficient
model training/inference, automatic model growth, as well as efficient multi-
task learning. For example, the DynamicEmbedding system (Zeng et al., 2020)
employs a _memory server_ for serving the embedding data in a model with
almost unlimited sparse feature set; The Taskology system (Lu et al., 2020b)
employs an _inference server_ to facilitate multi-task training with
consistency constraints. In this work, we further extend this line of research
by proposing a highly distributed system with both memory and inference
components.

##  3\. Proposed CARLS

In this section, we provide the details of the proposed CARLS framework, which
consists of three major components that enable asynchronous representation
learning across heterogeneous platforms:

  * •

Model Trainer: responsible for both training various models and communicating
with the knowledge bank to fetch augmented information to facilitate the
training.

  * •

Knowledge Maker: responsible for generating the knowledge requested by the
model trainer. Specifically, a knowledge maker loads the latest checkpoint
(saved by the model trainer) to make inference on auxiliary data, or discover
new neighborhoods from examples with close representations learnt from the
model.

  * •

Knowledge Bank: responsible for storing and refreshing the knowledge generated
by knowledge makers. A knowledge bank also serves the fetch requests from
model trainers.

Two key characteristics of CARLS are that (a) Model Trainers, Knowledge Banks,
and Knowledge Makers can be operated asynchronously, and (b) these components
can be deployed on heterogeneous hardware platforms (CPUs, GPUs, TPUs, etc).
In the following sections, we describe the features that have been
implemented, and leave a wide range of scenarios that are not yet implemented
but accommodated by CARLS as a straight-line future work.

###  3.1. Knowledge Maker

Knowledge makers play a critical role in extending the capacity of a learning
system. Conventionally, the model trainers process the data and compute all
the information required for calculating losses. In CARLS, knowledge makers
are designed to dynamically calculate (or discover) the knowledge that can be
exploited by model trainers for calculating the losses. Therefore, part of
model trainers’ workload can be shifted to knowledge makers, which helps the
model trainers run faster or save capacity to accommodate larger models or
training with more data. Below we provide three example types of knowledge
that are suitable to be processed or discovered by knowledge makers:

  * •

Graph structure and node embedding: knowledge makers can load parts of the
model—such as the node encoder of graph neural network—to update the
knowledge. The graph structure can also be dynamically updated with the
similarity between the computed node embeddings, as opposed to a given static
graph (Section 4.1).

  * •

Augmented labels: knowledge makers can load the whole model to (a) make
inference on the missing labels for augmenting the training data, or (b) clean
up noisy labels for providing a cleaner training dataset (Section 4.2).

  * •

External Knowledge: another advanced application of CARLS is to integrate the
external knowledge/memeory for training (Graves et al., 2016; Weston et al.,
2014). In this case, the responsibility of knowledge makers is to pre-compute
and process the related knowledge for enabling fast serving by the knowledge
bank (Section 4.3)

Knowledge makers keep the same machine states as model trainers by
periodically loading the parameters from the latest checkpoints or parameter
servers. This way, the inferences made by the knowledge makers will be
consistent with model trainers. Knowledge makers can be implemented by a
MapReduce (Dean and Ghemawat, 2004) or Flume (Chambers et al., 2010) pipeline,
or by sending remote procedure calls (RPCs) to a remote inference server.

###  3.2. Knowledge Bank

The knowledge inferred by knowledge makers are stored in knowledge banks for
fast serving model-training jobs. Depending on the application type, a
knowledge bank can be an external storage system—such as Bigtable (Chang et
al., 2006) or Spanner (Corbett et al., 2012)—or a specialized embedding server
(Zeng et al., 2020) responsible for updating internal parameters during back-
propagation stage. Some applications may require knowledge banks perform
certain computations, such as retrieving top-N nearest neighbors given a
query. Note that the number of instances stored in knowledge banks can be very
large (multi-millions to multi-billions). To keep the computational latency
constant—not growing as the data size grows, the knowledge banks are sharded
and deployed in a distributed fashion (van Steen and Tanenbaum, 2017).

Below is a list of example applications enabled by knowledge banks:

  * •

Feature Lookup: an instance’s features (e.g., neighbor IDs from a graph, or
labels) are stored as a protocol buffer (pro, 2016) and keyed by the
instance’s unique ID. In this application, knowledge banks can be implemented
directly by external storage systems without special treatments to store these
feature information.

  * •

Embedding Lookup and Update: in certain applications—e.g., node embeddings in
the graph neural network or word embeddings in language modeling, the
embeddings can be stored in a knowledge bank and will be continuously updated
throughout the training process. To support back-propagate the gradients with
respect to the embeddings, we leverage the DynamicEmbedding (Zeng et al.,
2020) (as the implementation of knowledge banks) to update the embeddings with
the gradient values.

  * •

Nearest Neighbors Lookup: efficient nearest neighbor search in the embedding
space is important for solving large scale classification and retrieval tasks
(Guo et al., 2020). Unlike existing learning paradigms that either take the
pre-computed nearest neighbors as input features or search over a very limited
scope (e.g., within the same batch), CARLS enables searching over the
embeddings kept in the knowledge bank, which is essentially the entire
dataset—both labeled and unlabeled samples. Furthermore, using CARLS enables
nearest-neighbor search over the dataset that grows or is updated during
training. To facilitate efficient nearest neighbors search as the dataset
grows, the computation is distributed into multiple shards and ScaNN (Guo et
al., 2020) can be applied for search space pruning and quantization.

##### Lazy update for asynchronous gradient update

While data consistency, isolation, and durability are delegated to the design
of existing storage systems (Chang et al., 2006; Corbett et al., 2012) and
computation efficiency and parallelism can be handled by existing server
designs (Zeng et al., 2020), special treatment is still required when multiple
training jobs are attempting to update the gradients of the same embedding
entry stored in knowledge bank. In this case, simply guaranteeing atomicity
may not be sufficient since this mechanism favors the last model that updates
the gradients and ignores the contribution from other models. CARLS resolves
this issue by employing a lazy update scheme: caching the results of gradient
update until the next lookup request arrives, or an expiration time is
reached. The update is based on the average of all cached gradients with
possible outlier detection. With this lazy update mechanism, the overall
training process is more stable compared with simple stochastic gradient
descent.

###  3.3. Model Trainer

A model trainer is typically a machine-learning model equipped with a
communication module to fetch the knowledge (of inferences) from knowledge
banks. The communication module is usually implemented as customized
operations or layers, enabling the funtionalities as listed in Section 3.2.
The communication module sends the request to the knowledge bank for
retrieving additional information discovered during training. The inputs of
communication modules can come from the training data (e.g., sample IDs for
fetching neighbor IDs), or the embedding values at a hidden layer (for
retrieving the nearest neighbors). Note that model trainers can be implemented
with various platforms (e.g., TensorFlow or Pytorch) in a distributed fashion.
Special cases need to be taken for the communication module when the trainers
are running on TPU machines (Jouppi et al., 2017), which works best when the
entire model fits into the on-chip memory.

##  4\. Learning Paradigms

In the section, we provide three learning paradigms as examples that can be
scaled up efficiently by CARLS, including but not limited to semi-supervised
learning, curriculum learning, and multimodal learning.

Figure 2. Training a graph-regularized model at scale with CARLS. Flow in red
is added to enable graph regularization and required only during training. The
model trainer and knowledge makers are running in parallel with a shared
knowledge bank (KB). Knowledge makers keep loading the latest checkpoint
generated by model trainers for making inferences and update the embeddings in
KB. The input processor of the model trainer will look up training instances’
neighborhood from the KB. After retrieving the neighbor info (e.g., neighbor
IDs, edge weights), the neighbor IDs will be used to lookup neighbors’
embeddings from the KB.

###  4.1. Semi-Supervised Learning

Semi-supervised learning (SSL) is a machine learning paradigm that harnesses
both labeled and unlabeled data to improve model prediction performance, and
is particularly useful when labeled data is limited or expensive to obtain.
However, despite the advantages and successes brought by SSL, using abundant
unlabeled data also significantly increases the training computation. For
example, graph-based methods are one type of popular SSL approaches, where
each node’s information (e.g., labels) can be iteratively updated by
aggregating the information from neighboring nodes whether they are labeled or
unlabeled (Juan et al., 2020; Bui et al., 2018). This means, in addition to
computing or processing the labeled data, additional computation is required
to process these neighboring nodes. In this case, CARLS deploy a fleet of
knowledge makers to process these neighboring nodes, shifting the additional
computation conventionally done by the model trainer to knowledge makers.
Furthermore, CARLS eliminates redundant computations (e.g., one node being a
common neighbor among several other nodes only needs to be processed once) by
caching the results in the knowledge banks.

Figure 2 illustrates a graph-regularized model trained with CARLS. The
training objective is to minimize the sum of the supervised loss and the graph
regularizer, which is a pairwise distance between the embeddings of
neighboring nodes. The neighbourhood can be constructed offline using existing
signals (e.g., co-clicked image pairs (Juan et al., 2020) or images in the
same board (Ying et al., 2018)), or created online during training (e.g.,
image augmentation (Cubuk et al., 2019)), and looked up from the knowledge
bank in the trainer.

If the neighbor embeddings are directly computed in the trainer, the training
time and the trainer memory consumption will increase linearly in proportional
to the number of neighbors (Juan et al., 2020). To make the graph-regularized
model scalable, the embeddings of each node/image are computed by knowledge
makers and updated to the knowledge bank in parallel with the model trainer.

Figure 3. An illustration of the training pipeline of CARLS for a graph neural
network stacked on top of a node encoder. The node encoder encodes the node
features into embeddings and feed to the GNN. In the model trainer, each
training input node can look up its sub-graph and the embeddings of each node
in the subgraph from the KB. The knowledge makers can load the latest
checkpoint to compute and update the node embeddings as well as the graph
(based on the node embeddings) to the KB.

Figure 3 illustrates another example of applying CARLS to enable efficient
graph operations for large-scale graph learning. In this example, the model is
a stack of an encoder (e.g.ResNet or Transformer) and a graph neural network
(GNN). It can be very challenging to train such a model, especially when the
size of the subgraph is large, without the support of CARLS.

###  4.2. Curriculum Learning

Curriculum learning (Bengio et al., 2009) refers to a learning paradigm where
the challenge of a learning task gradually increases over time for improving
the model performance. The increased challenge usually forces models to focus
on learning additional information that is not well captured from the previous
iterations (Stretcu et al., 2019). This paradigm maps favorably to the CARLS
framework where the additional information can be stored in the knowledge bank
and the knowledge makers are responsible for generating this information
required to increase the training complexity. Fig. 4 illustrates the workflow
of using CARLS for curriculum learning. Specifically, we demonstrate two
examples: _online label mining_ deals with noisy labels, and the _graph
agreement model_ deals with missing labels for some inputs.

Figure 4. An illustration of the workflow of curriculum learning using CARLS.
The labels of the input examples can be improved by the knowledge maker (e.g.,
using online label mining for unreliable labels or graph agreement model for
inferring missing labels).

####  4.2.1. Online label mining

In large scale applications of image classification, the labels of each image
are usually inferred from a rather noisy input (e.g., search queries or text
captions associated to an image) that are often inconsistent with each other
(e.g., wrong or missing labels). In such cases, multiple iterations are often
needed to alternate between i) training the model based on noisy data and ii)
refining the model based on the latest trained model. Such a process can be
significantly accelerated if we can do i) and ii) in parallel. In CARLS, i)
can be achieved via a model trainer and ii) by knowledge makers, and the
knowledge bank stores the newest inferred labeled for each instance.

####  4.2.2. Graph Agreement Model

Another example of curriculum learning is the graph agreement model for semi-
supervised learning (Stretcu et al., 2019), where the labels of unknown
training examples are inferred based on the nearest neighbors from labeled
examples, where the nearest neighbors are calculated based on the embedding of
the input (from the hidden layer of the trained model). In CARLS, this nearest
neighbors search can be done by a knowledge maker and the inferred labels can
be stored in the knowledge bank.

###  4.3. Multimodal Learning

In addition to training a single deep model, CARLS can also be used for
multimodal learning such as training an image-text two-tower model (Jia et
al., 2021) and the modality nets (Kaiser et al., 2017) that has four models
for language (text data), images, audio, and categorical data.

Figure 5. An illustration of the training pipeline of CARLS for a deep image-
text two-tower model. Flow in red is added to enable lookup image/text
embeddings and is required only during training. The distributed knowledge
makers will compute the image/text embeddings and update the embeddings to the
knowledge bank. The model trainer can look up the embeddings from the KB
instead of computing the image/text embeddings through the image/text encoder.

Figure 5 illustrates an example of a image-text deep two-tower model trained
with CARLS, where the image and text encoders can be learned via a contrastive
loss (Chen et al., 2020) that pushes the embeddings of matched image-text pair
together while pushing those of non-matched image-text pair apart. The image-
text matching scores are computed as the cosine-similarity of the image
embeddings and the text embeddings from the corresponding encoders. Although
when the image/text encoder goes deeper it can often achieve better
performance (Jia et al., 2021), it is more challenging to train as the
computational complexity grows significantly. Instead of computing the
image/text embeddings directly in the model trainer, we can utilize CARLS to
lookup historical embeddings from the knowledge bank111In order to back-
propagate the gradients back to the encoders, a subset of the examples in a
training mini-batch should still be computed through the encoders in the model
trainer..

In addition to reducing the computation in the trainer, CARLS can also be used
to improve the model performance. As shown in (Jia et al., 2021) that the
model performance increases along with the number of random negatives used in
the contrastive learning. As the embeddings of the random negatives can be
looked up from the knowledge bank, we can easily scale up the number of random
negatives. Besides, because each encoder can be considered as an independent
model in the knowledge makers (by loading parts of the network from the
checkpoint of the model), the image/text embeddings can be computed and
updated to the knowledge bank independently. Thus, image/text data feed to the
knowledge makers can go beyond the training image-text pairs.

##  5\. Conclusion

In this paper, we proposed CARLS, a cross-platform, asynchronous learning
framework that significantly extends the capacity of current deep learning
frameworks by enabling dynamic knowledge sharing between model trainers and
knowledge makers. The proposed CARLS is particularly suitable for training
large-scale neural models that benefit from additional knowledge inferred or
discovered during training, such as graph neural networks and multi-modality
models. CARLS also enables existing frameworks to dynamically learn with
changes inferred or occurred during training, facilitating curriculum learning
to be more efficient. As a straight-line future work, we will extend CARLS to
train more variety of learning models, design new APIs to streamline system
setup and provide better support for heterogeneous hardware.

## References

  * (1)
  * pro (2016) 2016\.  Protocol Buffers Language Guide.
  * Abadi et al. (2016) Martín Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving, Michael Isard, et al. 2016\.  Tensorflow: a system for large-scale machine learning.. In _USENIX Symposium on Operating Systems Design and Implementation (OSDI)_ , Vol. 16. 265–283.
  * Antol et al. (2015) Stanislaw Antol, Aishwarya Agrawal, Jiasen Lu, Margaret Mitchell, Dhruv Batra, C Lawrence Zitnick, and Devi Parikh. 2015.  Vqa: Visual question answering. In _Proceedings of the IEEE international conference on computer vision_. 2425–2433.
  * Bengio et al. (2009) Yoshua Bengio, Jérôme Louradour, Ronan Collobert, and Jason Weston. 2009\.  Curriculum learning. In _Proceedings of the 26th annual international conference on machine learning_. 41–48.
  * Brown et al. (2020) Tom B Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020\.  Language models are few-shot learners.  _arXiv preprint arXiv:2005.14165_ (2020).
  * Bui et al. (2018) Thang D Bui, Sujith Ravi, and Vivek Ramavajjala. 2018.  Neural Graph Learning: Training Neural Networks Using Graphs. In _ACM International Conference on Web Search and Data Mining_.
  * Chambers et al. (2010) Craig Chambers, Ashish Raniwala, Frances Perry, Stephen Adams, Robert R Henry, Robert Bradshaw, and Nathan Weizenbaum. 2010.  FlumeJava: easy, efficient data-parallel pipelines.  _ACM Sigplan Notices_ 45, 6 (2010), 363–375.
  * Chang et al. (2006) Fay Chang, Jeffrey Dean, Sanjay Ghemawat, Wilson C. Hsieh, Deborah A. Wallach, Mike Burrows, Tushar Chandra, Andrew Fikes, and Robert E. Gruber. 2006.  Bigtable: A Distributed Storage System for Structured Data. In _7th USENIX Symposium on Operating Systems Design and Implementation (OSDI)_. 205–218.
  * Chen et al. (2020) Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. 2020.  A simple framework for contrastive learning of visual representations. In _International conference on machine learning_. PMLR, 1597–1607.
  * Collobert et al. (2002) Ronan Collobert, Samy Bengio, and Johnny Mariéthoz. 2002\.  _Torch: a modular machine learning software library_.  Technical Report. Idiap.
  * Corbett et al. (2012) James C. Corbett, Jeffrey Dean, Michael Epstein, Andrew Fikes, Christopher Frost, JJ Furman, Sanjay Ghemawat, Andrey Gubarev, Christopher Heiser, Peter Hochschild, Wilson Hsieh, Sebastian Kanthak, Eugene Kogan, Hongyi Li, Alexander Lloyd, Sergey Melnik, David Mwaura, David Nagle, Sean Quinlan, Rajesh Rao, Lindsay Rolig, Dale Woodford, Yasushi Saito, Christopher Taylor, Michal Szymaniak, and Ruth Wang. 2012\.  Spanner: Google’s Globally-Distributed Database. In _OSDI_.
  * Cubuk et al. (2019) Ekin D Cubuk, Barret Zoph, Dandelion Mane, Vijay Vasudevan, and Quoc V Le. 2019.  Autoaugment: Learning augmentation strategies from data. In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_. 113–123.
  * Dean and Ghemawat (2004) Jeffrey Dean and Sanjay Ghemawat. 2004.  MapReduce: Simplified Data Processing on Large Clusters. In _OSDI’04: Sixth Symposium on Operating System Design and Implementation_. San Francisco, CA, 137–150.
  * Devlin et al. (2018a) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018a.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding.  _CoRR_ abs/1810.04805 (2018).
  * Devlin et al. (2018b) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018b.  Bert: Pre-training of deep bidirectional transformers for language understanding.  _arXiv preprint arXiv:1810.04805_ (2018).
  * Esteva et al. (2019) Andre Esteva, Alexandre Robicquet, Bharath Ramsundar, Volodymyr Kuleshov, Mark DePristo, Katherine Chou, Claire Cui, Greg Corrado, Sebastian Thrun, and Jeff Dean. 2019\.  A guide to deep learning in healthcare.  _Nature medicine_ 25, 1 (2019), 24–29.
  * Fedus et al. (2021) William Fedus, Barret Zoph, and Noam Shazeer. 2021\.  Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity.  _arXiv preprint arXiv:2101.03961_ (2021).
  * Goodfellow et al. (2016) Ian Goodfellow, Yoshua Bengio, Aaron Courville, and Yoshua Bengio. 2016.  _Deep learning_. Vol. 1.  MIT press Cambridge.
  * Graves et al. (2016) Alex Graves, Greg Wayne, Malcolm Reynolds, Tim Harley, Ivo Danihelka, Agnieszka Grabska-Barwińska, Sergio Gómez Colmenarejo, Edward Grefenstette, Tiago Ramalho, John Agapiou, et al. 2016\.  Hybrid computing using a neural network with dynamic external memory.  _Nature_ 538, 7626 (2016), 471–476.
  * Guo et al. (2020) Ruiqi Guo, Philip Sun, Erik Lindgren, Quan Geng, David Simcha, Felix Chern, and Sanjiv Kumar. 2020.  Accelerating Large-Scale Inference with Anisotropic Vector Quantization. In _International Conference on Machine Learning_.  https://arxiv.org/abs/1908.10396
  * He et al. (2015) Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2015\.  Deep Residual Learning for Image Recognition. In _Computer Vision and Pattern Recognition (CVPR)_.
  * Hinton et al. (2015) Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. 2015\.  Distilling the knowledge in a neural network.  _arXiv preprint arXiv:1503.02531_ (2015).
  * Jia et al. (2021) Chao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoc V. Le, Yunhsuan Sung, Zhen Li, and Tom Duerig. 2021\.  Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision.  arXiv:cs.CV/2102.05918
  * Jouppi et al. (2017) Norman P Jouppi, Cliff Young, Nishant Patil, David Patterson, Gaurav Agrawal, Raminder Bajwa, Sarah Bates, Suresh Bhatia, Nan Boden, Al Borchers, et al. 2017\.  In-datacenter performance analysis of a tensor processing unit. In _Proceedings of the 44th annual international symposium on computer architecture_. 1–12.
  * Juan et al. (2020) Da-Cheng Juan, Chun-Ta Lu, Zhen Li, Futang Peng, Aleksei Timofeev, Yi-Ting Chen, Yaxi Gao, Tom Duerig, Andrew Tomkins, and Sujith Ravi. 2020\.  Ultra Fine-Grained Image Semantic Embedding. In _Proceedings of the 13th International Conference on Web Search and Data Mining_. 277–285.
  * Kaiser et al. (2017) Lukasz Kaiser, Aidan N Gomez, Noam Shazeer, Ashish Vaswani, Niki Parmar, Llion Jones, and Jakob Uszkoreit. 2017.  One model to learn them all.  _arXiv preprint arXiv:1706.05137_ (2017).
  * Kandel et al. (2000) Eric R Kandel, James H Schwartz, Thomas M Jessell, Department of Biochemistry, Molecular Biophysics Thomas Jessell, Steven Siegelbaum, and AJ Hudspeth. 2000\.  _Principles of neural science_. Vol. 4.  McGraw-hill New York.
  * Kipf and Welling (2016) Thomas N Kipf and Max Welling. 2016.  Semi-supervised classification with graph convolutional networks.  _arXiv preprint arXiv:1609.02907_ (2016).
  * LeCun et al. (2015) Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. 2015\.  Deep Learning.  _Nature_ 521 (2015), 436–444.
  * Lepikhin et al. (2020) Dmitry Lepikhin, HyoukJoong Lee, Yuanzhong Xu, Dehao Chen, Orhan Firat, Yanping Huang, Maxim Krikun, Noam Shazeer, and Zhifeng Chen. 2020\.  Gshard: Scaling giant models with conditional computation and automatic sharding.  _arXiv preprint arXiv:2006.16668_ (2020).
  * Li et al. (2014) Mu Li, David G Andersen, Jun Woo Park, Alexander J Smola, Amr Ahmed, Vanja Josifovski, James Long, Eugene J Shekita, and Bor-Yiing Su. 2014.  Scaling distributed machine learning with the parameter server. In _11th $\\{$USENIX$\\}$ Symposium on Operating Systems Design and Implementation ($\\{$OSDI$\\}$ 14)_. 583–598.
  * Lu et al. (2020a) Jiasen Lu, Vedanuj Goswami, Marcus Rohrbach, Devi Parikh, and Stefan Lee. 2020a.  12-in-1: Multi-task vision and language representation learning. In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_. 10437–10446.
  * Lu et al. (2020b) Yao Lu, Sören Pirk, Jan Dlabal, Anthony Brohan, Ankita Pasad, Zhao Chen, Vincent Casser, Anelia Angelova, and Ariel Gordon. 2020b.  Taskology: Utilizing Task Relations at Scale.  _arXiv preprint arXiv:2005.07289_ (2020).
  * Medini et al. (2019) Tharun Medini, Qixuan Huang, Yiqiu Wang, Vijai Mohan, and Anshumali Shrivastava. 2019.  Extreme classification in log memory using count-min sketch: A case study of amazon search with 50m products.  _arXiv preprint arXiv:1910.13830_ (2019).
  * Narayanan et al. (2019) Deepak Narayanan, Aaron Harlap, Amar Phanishayee, Vivek Seshadri, Nikhil R Devanur, Gregory R Ganger, Phillip B Gibbons, and Matei Zaharia. 2019.  PipeDream: generalized pipeline parallelism for DNN training. In _Proceedings of the 27th ACM Symposium on Operating Systems Principles_. 1–15.
  * PyTorch (2018) PyTorch. 2018\.  http://pytorch.org http://pytorch.org.
  * Radford et al. ([n. d.]) Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. [n. d.].  Learning Transferable Visual Models From Natural Language Supervision.  _Image_ 2 ([n. d.]), T2.
  * Radford et al. (2018) Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. 2018.  Improving language understanding by generative pre-training.  (2018).
  * Radford et al. (2019) Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019\.  Language models are unsupervised multitask learners.  _OpenAI blog_ 1, 8 (2019), 9.
  * Raffel et al. (2020) Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020.  Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer.  _Journal of Machine Learning Research_ 21, 140 (2020), 1–67.  http://jmlr.org/papers/v21/20-074.html
  * Shazeer et al. (2017) Noam Shazeer, Azalia Mirhoseini, Krzysztof Maziarz, Andy Davis, Quoc Le, Geoffrey Hinton, and Jeff Dean. 2017.  Outrageously large neural networks: The sparsely-gated mixture-of-experts layer.  _arXiv preprint arXiv:1701.06538_ (2017).
  * Song et al. (2020) Liuyihan Song, Pan Pan, Kang Zhao, Hao Yang, Yiming Chen, Yingya Zhang, Yinghui Xu, and Rong Jin. 2020\.  Large-Scale Training System for 100-Million Classification at Alibaba. In _Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining_. 2909–2930.
  * Stretcu et al. (2019) Otilia Stretcu, Krishnamurthy Viswanathan, Dana Movshovitz-Attias, Anthony Platanios, Sujith Ravi, and Andrew Tomkins. 2019\.  Graph agreement models for semi-supervised learning.  (2019).
  * Tagami (2017) Yukihiro Tagami. 2017\.  Annexml: Approximate nearest neighbor search for extreme multi-label classification. In _Proceedings of the 23rd ACM SIGKDD international conference on knowledge discovery and data mining_. 455–464.
  * van Steen and Tanenbaum (2017) Maarten van Steen and Andrew S. Tanenbaum. 2017.  _Distributed Systems_.  CreateSpace Independent Publishing Platform.
  * Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017\.  Attention is All you Need.  In _Advances in Neural Information Processing Systems (NIPS)_. 5998–6008.
  * Veličković et al. (2017) Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. 2017.  Graph attention networks.  _arXiv preprint arXiv:1710.10903_ (2017).
  * Weston et al. (2014) Jason Weston, Sumit Chopra, and Antoine Bordes. 2014\.  Memory networks.  _arXiv preprint arXiv:1410.3916_ (2014).
  * Ying et al. (2018) Rex Ying, Ruining He, Kaifeng Chen, Pong Eksombatchai, William L Hamilton, and Jure Leskovec. 2018\.  Graph convolutional neural networks for web-scale recommender systems. In _Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining_. 974–983.
  * Young et al. (2014) Peter Young, Alice Lai, Micah Hodosh, and Julia Hockenmaier. 2014\.  From image descriptions to visual denotations: New similarity metrics for semantic inference over event descriptions.  _Transactions of the Association for Computational Linguistics_ 2 (2014), 67–78.
  * Zellers et al. (2019) Rowan Zellers, Yonatan Bisk, Ali Farhadi, and Yejin Choi. 2019\.  From recognition to cognition: Visual commonsense reasoning. In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_. 6720–6731.
  * Zeng et al. (2020) Yun Zeng, Siqi Zuo, and Dongcai Shen. 2020.  DynamicEmbedding: Extending TensorFlow for Colossal-Scale Applications.  _arXiv preprint arXiv:2004.08366_ (2020).
  * Zhang et al. (2018) Xingcheng Zhang, Lei Yang, Junjie Yan, and Dahua Lin. 2018\.  Accelerated training for massive classification via dynamic class selection. In _Proceedings of the AAAI Conference on Artificial Intelligence_ , Vol. 32.
  * Zhao et al. (2019) Weijie Zhao, Jingyuan Zhang, Deping Xie, Yulei Qian, Ronglai Jia, and Ping Li. 2019\.  AIBox: CTR prediction model training on a single node. In _Proceedings of the 28th ACM International Conference on Information and Knowledge Management_. 319–328.
  * Zhou et al. (2018) Guorui Zhou, Xiaoqiang Zhu, Chenru Song, Ying Fan, Han Zhu, Xiao Ma, Yanghui Yan, Junqi Jin, Han Li, and Kun Gai. 2018\.  Deep interest network for click-through rate prediction. In _Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining_. 1059–1068.