# Tailoring the overlap distribution in driven mean-field spin models

Laura Guislain  Univ. Grenoble Alpes, CNRS, LIPhy, 38000 Grenoble, France
Eric Bertin  Univ. Grenoble Alpes, CNRS, LIPhy, 38000 Grenoble, France

###### Abstract

In a statistical physics context, inverse problems consists in determining
microscopic interactions such that a system reaches a predefined collective
state. A complex collective state may be prescribed by specifying the overlap
distribution between microscopic configurations, a notion originally
introduced in the context of disordered systems like spin-glasses. We show
that in spite of the absence of disorder, nonequilibrium spin models
exhibiting spontaneous magnetization oscillations provide a benchmark to
prescribe a non-trivial overlap distribution with continuous support,
qualitatively analogous to the ones found in disordered systems with full
replica symmetry breaking. The overlap distribution can be explicitly tailored
to take a broad range of predefined shapes by monitoring the spin dynamics.
The presence of a non-trivial overlap distribution is traced back to an
average over infinitely many pure states, a feature shared with spin-glasses.

Finding microscopic interactions or dynamical rules such that a system
composed of many interacting subunits reaches a predefined macroscopic state
is often challenging [1], as it may involve exploring a potentially large
parameter space. Such computationally demanding problems are called inverse
problems, and their interest has been renewed by the recent development of
powerful machine learning techniques [1, 2]. Inverse problems are very
relevant in particular when searching for a microscopic model able to describe
real-world data [2]. For example, interaction networks have been reconstructed
from measured data in the case of starling flocks [3, 4], neural networks [5,
6] or gene regulatory networks [7, 8, 9]. Inverse problems are also currently
explored in the context of active matter, where an emergent research trend
consists in using decentralized learning procedures to optimize individual
parameters to reach a target collective behavior [10, 11, 12, 13, 14, 15].

The collective state of an assembly of interacting entities may be
characterized by global observables, like polar or nematic order parameters in
an active matter context for instance. Yet, a finer characterization of the
collective structure may be obtained from the overlap distribution of
microscopic configurations. The overlap $q$ of two configurations is a measure
of their similarity, typically $q\approx 1$ for similar configurations, while
$q\approx 0$ when two configurations are picked up at random. The probability
distribution of overlaps has been introduced as a useful characterization of
the spin-glass phase, to deal with the absence of a visible order [16, 17,
18]. It has later proved to be an efficient characterization of the glassy
state in supercooled liquids [19, 20, 21], random lasers and disordered
optical media [22, 23], constraint satisfaction problems [24, 25], neural
networks [26, 27], population dynamics [28, 29] and directed polymer models
[30]. Quite importantly, the overlap distribution is non-trivial only for
‘complex’ phases of matter, like glassy phases that exhibit replica symmetry
breaking [16]. For ordinary phases of matter like liquid or gas phases, the
overlap distribution $P(q)$ reduces to a delta peak at $q=0$ in the
thermodynamic limit, as if configurations were picked completely at random.
For ordered phases, $P(q)$ is a sum of delta peaks related by the broken
symmetry (e.g., two symmetric delta peaks for a ferromagnetic state). Imposing
a non-trivial prescribed overlap distribution is thus a challenging task, as
one needs both to generate enough complexity in the state of the system, and
to tame it so as to impose the prescribed overlap distribution.

In this Letter, we argue that in spite of the absence of disorder, driven
mean-field spin models exhibiting spontaneous magnetization oscillations are
characterized by a non-trivial overlap distribution with a continuous support.
We show that this overlap distribution can be explicitly tailored to take a
broad range of predefined shapes by monitoring the spin dynamics, for instance
through a magnetization-dependence of the spin-flip frequency. In addition,
the physical origin of the overlap distribution is traced back to an average
over infinitely many pure states, in qualitative analogy with mean-field spin-
glass models.

We consider a generic mean-field spin model composed of $N$ spins $s_{i}=\pm
1$, endowed with a stochastic spin-flip dynamics that breaks detailed balance.
The model also typically includes auxiliary variables to allow for spontaneous
temporal oscillations of the magnetization $m=\frac{1}{N}\sum_{i=1}^{N}s_{i}$
in a far-from-equilibrium regime [31, 32] –see also [33, 34, 35, 36, 37, 38,
39] for related spin models exhibiting spontaneous magnetization oscillations.
Such oscillations lead in particular to a continuous support $[-m_{0},m_{0}]$,
with $m_{0}>0$, of the probability distribution of the magnetization, at odds
with usual paramagnetic or ferromagnetic phases. A detailed characterization
of the phase transition to an oscillating state is obtained by considering the
statistics of the overlap

$q=\frac{1}{N}\sum_{i=1}^{N}s_{i}s_{i}^{\prime}\qquad$ (1)

between two spin configurations $\\{s_{i}\\}$ and $\\{s_{i}^{\prime}\\}$ (note
that $-1\leq q\leq 1$). Identical (opposite) spin configurations have an
overlap $q=1$ ($q=-1$), while $q=0$ for uncorrelated spin configurations. To
describe the overlap statistics, we introduce the overlap probability
distribution

$P(q)=\sum_{\\{s_{i}\\},\\{s_{i}^{\prime}\\}}P_{s}(\\{s_{i}\\})P_{s}(\\{s_{i}^{\prime}\\})\,\delta\bigg{(}\frac{1}{N}\sum_{i=1}^{N}s_{i}s_{i}^{\prime}-q\bigg{)},$
(2)

obtained by averaging over two statistically independent spin configurations
$\\{s_{i}\\}$ and $\\{s_{i}^{\prime}\\}$. The overlap distribution $P(q)$ can
be evaluated for $N\to\infty$, based on the spin-configuration distribution
$P_{s}(\\{s_{i}\\})$. As the spins are exchangeable random variables in mean-
field models, de Finetti’s representation theorem [40, 41] leads for large $N$
to

$P_{s}(\\{s_{i}\\})=\int_{-1}^{1}\\!\mathrm{d}m\,\tilde{P}(m)\,\mathcal{P}(\\{s_{i}\\}|m)$
(3)

with a factorized conditional distribution $\mathcal{P}(\\{s_{i}\\}|m)$,

$\mathcal{P}(\\{s_{i}\\}|m)=\left(\frac{1-m^{2}}{4}\right)^{N/2}\prod_{i=1}^{N}\left(\frac{1+m}{1-m}\right)^{s_{i}/2}$
(4)

and where $\tilde{P}(m)$ is the probability density of the magnetization.
Taking a Fourier transform of Eq. (2) and using Eqs. (3) and (4) (see [42] for
details), the following integral expression of the overlap distribution $P(q)$
is obtained, for $|q|\leq 1$, as

$P(q)=2\int_{|q|}^{1}\frac{dm}{m}\tilde{P}(m)\tilde{P}\left(\frac{q}{m}\right).$
(5)

Hence the overlap distribution $P(q)$ is uniquely determined by the
magnetization distribution $\tilde{P}(m)$. In the presence of spontaneous
oscillations of the magnetization, $\tilde{P}(m)$ can be obtained, in the
infinite $N$ limit, from the shape of the limit cycle describing oscillations
in the plane ($m$,$\dot{m}$), where $\dot{m}=dm/dt$. The shape of the limit
cycle is described by an equation $\dot{m}=F(m)$ for $\dot{m}>0$, where the
function $F(m)$ is defined over the interval $[-m_{0},m_{0}]$. Assuming that
the reversal symmetry $(m,\dot{m})\to(-m,-\dot{m})$ holds, the limit cycle is
symmetric with respect to the origin $(m,\dot{m})=(0,0)$, and the part
corresponding to $\dot{m}<0$ is described by $\dot{m}=-F(-m)$. Using the
uniform flow of time, the magnetization distribution is obtained as

$\tilde{P}(m)=\frac{1}{\tau}\left(\frac{1}{F(m)}+\frac{1}{F(-m)}\right),$ (6)

where $\tau$ is the oscillation period. Hence $\tilde{P}(m)$ is symmetric,
$\tilde{P}(m)=\tilde{P}(-m)$. The normalization of $\tilde{P}(m)$ implies that
the period $\tau$ is given by $\tau=2\int_{-m_{0}}^{m_{0}}dm/F(m)$. The shape
of the limit cycle in the plane ($m$,$\dot{m}$) thus fully determines the
magnetization distribution $\tilde{P}(m)$, and thus the overlap distribution
$P(q)$. In other words, by monitoring the shape of the limit cycle, one may be
able to reach a prescribed overlap distribution.

There are many ways to monitor the shape of the limit cycle by tuning the
macroscopic dynamics. In the following, we explore more specifically the
generic idea of time reparameterization, which can be simply stated as
follows. As a reference model, we start from a driven mean-field spin model
that exhibits spontaneous oscillations of period $\tau_{0}$, with a limit
cycle described by $\dot{m}=F_{0}(m)$ for $\dot{m}>0$. Multiplying the
microscopic transition rates of the spin-flip dynamics by a magnetization-
dependent factor $\nu(m)$, the dynamics of the magnetization $m$ is obtained
from the original dynamics through a (generally non-linear) reparametrization
of time $t\to t^{\prime}$, with $dt^{\prime}=\nu(m)dt$, such that
$m(t)=m_{0}(t^{\prime})$, where $m_{0}(t)$ stands for the time-dependent
magnetization in the reference model. As a result, the shape of the limit
cycle is changed into $\dot{m}=\nu(m)F_{0}(m)$, leading according to Eq. (6)
to a modified magnetization distribution

$\tilde{P}(m)=\frac{1}{\tau}\left(\frac{1}{\nu(m)F_{0}(m)}+\frac{1}{\nu(-m)F_{0}(-m)}\right).$
(7)

The period $\tau$ is arbitrary, as it can be changed by a rescaling of
$\nu(m)$, and we thus set $\tau=\tau_{0}$. Under the assumption
$\nu(m)=\nu(-m)$, Eq. (7) can then be inverted to give

$\nu(m)=\frac{\tilde{P}_{0}(m)}{\tilde{P}(m)},$ (8)

where $\tilde{P}_{0}(m)$ is the magnetization distribution for the reference
model. Hence, if one is able to determine the magnetization distribution which
is a solution of Eq. (5) for a prescribed overlap distribution $P(q)$, it is
then straightforward to determine the dimensionless frequency $\nu(m)$ to be
included in the microscopic dynamics. This result provides a direct link
between the overlap distribution and the microscopic spin dynamics.

We now discuss the generic issue of inverting the integral equation (5) to
obtain $\tilde{P}(m)$ from the knowledge of $P(q)$. The magnetization
distribution $\tilde{P}(m)$ is assumed to have a bounded continuous support
$[-m_{0},m_{0}]$, leading for the overlap distribution $P(q)$ to continuous
support $[-q_{0},q_{0}]$, with $q_{0}=m_{0}^{2}$. Introducing the function
$g(u)$ defined for $u>0$ through the relation

$\tilde{P}(m)=m_{0}^{-1}g\left(\ln\left|\frac{m_{0}}{m}\right|\right)\theta(m_{0}-|m|)$
(9)

and similarly the function $\phi(v)$ defined for $v>0$ by

$P(q)=q_{0}^{-1}\phi\left(\ln\left|\frac{q_{0}}{q}\right|\right)\theta(q_{0}-|q|),$
(10)

Eq. (5) takes a convolution form,

$\phi(v)=2\int_{v}^{1}du\,g(u)g(v-u)\,.$ (11)

This equation is supplemented by the normalization constraints on $P(q)$ and
$\tilde{P}(m)$, which in terms of the functions $g(u)$ and $\phi(v)$ become

$2\int_{0}^{\infty}dv\,e^{-v}\phi(v)=1,\qquad
2\int_{0}^{\infty}du\,e^{-u}g(u)=1\,.$ (12)

The problem of finding the magnetization distribution $\tilde{P}(m)$
corresponding to a prescribed overlap distribution $P(q)$ is equivalent to
starting from a prescribed function $\phi(v)$, non-zero for $v>0$ only, and
inverting Eq. (11) to obtain the function $g(u)$. One possible technique is to
solve the convolution problem using a Laplace transform, and to invert
numerically the Laplace transform of $g(u)$. Alternatively, one may consider
classes of functions $\phi(v)$ for which the solution $g(u)$ of Eq. (11) is
known analytically. For instance, if $\phi(v)$ is proportional to an
infinitely divisible distribution [43] with support on the positive real axis
and satisfies the normalization condition (12), a solution of Eq. (11) can
easily be found. A simple example of an infinitely divisible distribution is
the class of gamma distributions. To satisfy the normalization condition (12),
we take for $\phi(v)$ a modified gamma distribution,

$\phi(v)=f(v;\alpha,\beta)\equiv\frac{\beta^{\alpha}v^{\alpha-1}e^{-(\beta-1)v}}{2\Gamma(\alpha)},$
(13)

with two parameters $\alpha>0$ and $\beta>0$. The function $g(u)$ solution of
Eq. (11) is then given by

$g(u)=f\left(u;\frac{\alpha}{2},\beta\right).$ (14)

In the original variables $q$ and $m$, the overlap distribution is given by

$P_{\alpha\beta}(q)=\frac{\beta^{\alpha}}{2q_{0}\Gamma(\alpha)}\left|\frac{q}{q_{0}}\right|^{\beta-1}\left(\ln\left|\frac{q_{0}}{q}\right|\right)^{\alpha-1}\theta(q_{0}-|q|),$
(15)

while the magnetization distribution reads as

$\tilde{P}_{\alpha\beta}(m)=\frac{\beta^{\alpha/2}}{2m_{0}\Gamma(\frac{\alpha}{2})}\left|\frac{m}{m_{0}}\right|^{\beta-1}\\!\left(\ln\left|\frac{m_{0}}{m}\right|\right)^{\alpha/2-1}\\!\\!\theta(m_{0}-|m|)$
(16)

(we recall that $q_{0}=m_{0}^{2}$). In practice, once the magnetization
distribution $\tilde{P}(m)$ associated with the prescribed overlap
distribution $P(q)$ has been obtained, one needs to find the values of the
control parameters of the reference model (i.e., the model with $\nu(m)=1$)
for which the support of $\tilde{P}_{0}(m)$ is the same as $\tilde{P}(m)$. The
dimensionless frequency $\nu(m)$ is then obtained from Eq. (8). Multiplying
the transition rates of the reference model by $\nu(m)$, one obtains a model
exhibiting the prescribed overlap distribution.

The simple parameterization (15) of the overlap distribution already exhibits
a rich behavior. At a qualitative level, $\alpha$ characterizes the behavior
of $P(q)$ close to the support boundaries, $P(q)\sim(q_{0}-|q|)^{\alpha-1}$
for $q\to\pm q_{0}$. For $\alpha=1$, $P(q)$ goes to a finite, nonzero limit
for $q\to\pm q_{0}$, similarly to the behavior observed for an elliptic limit
cycle in the plane ($m$, $\dot{m}$) [31, 42]. For $\alpha>1$, $P(q)\to 0$ for
$q\to\pm q_{0}$, while for $\alpha<1$, $P(q)$ diverges in $\pm q_{0}$. A
divergence of $P(q)$ at the boundaries of the support is reminiscent of the
low-temperature behavior of mean-field spin-glass models, although the
functional form of the divergence is quantitatively different here. The
parameter $\beta$ characterizes the behavior of $P(q)$ for $q\to 0$,
$P(q)\sim|q|^{\beta-1}$ up to logarithmic corrections. For $\beta=1$ and
$\alpha>1$, a logarithmic divergence of $P(q)$ is observed for $q\to 0$,
similarly to the behavior observed for an elliptic limit cycle [31]. For
$\beta\neq 1$, one has for $q\to 0$ either a power-law divergence of $P(q)$
[$\beta<1$], or a zero limit [$\beta>1$]. Hence, the introduction of a
reparameterization through $\nu(m)$ shows that neither the logarithmic
divergence at $q=0$ nor the finite non-zero limit at the boundaries of the
support of $P(q)$ are hallmarks of the presence of a limit cycle. These
properties are specific to elliptic-like limit cycles [31].

We now illustrate the above generic results on a specific example of a driven
mean-field spin model. We consider a generalization of the spin model studied
in [31], where spins are coupled to dynamic fields, allowing for the onset of
spontaneous collective oscillations in the parameter regime when spins tend to
align with fields while fields tend to antialign with spins. The model
involves $2N$ microscopic variables: $N$ spins $s_{i}=\pm 1$ and $N$ fields
$h_{i}=\pm 1$. Global observables are the magnetization $m$ and the average
field $h=N^{-1}\sum_{i=1}^{N}h_{i}$. The stochastic dynamics consists in
randomly flipping a single spin $s_{i}$ or a single field $h_{i}$. The
flipping rates $W_{s}$ and $W_{h}$ depend only on $m$ and $h$,

$W_{s,h}=\nu(m)[1+\exp(\beta\Delta E_{s,h})]^{-1},$ (17)

with $\beta=T^{-1}$ the inverse temperature and $\Delta E_{s,h}$ the variation
of $E_{s,h}$ when flipping a spin $s_{i}$ or a field $h_{i}$, where

$E_{s}=-N\left(\frac{J_{1}}{2}m^{2}+\frac{J_{2}}{2}h^{2}+mh\right),\quad
E_{h}=E_{s}+\mu Nhm,$ (18)

and $\nu(m)$ is a magnetization-dependent microscopic frequency scale, assumed
to be an even function of $m$. We note $T_{c}=(J_{1}+J_{2})/2$ the critical
temperature and $\mu_{c}=1+(J_{1}-J_{2})^{2}/4$. For $\mu>\mu_{c}$, the model
exhibits a transition at $T_{c}$ from a paramagnetic phase ($T>T_{c}$) to an
oscillating phase ($T<T_{c}$) [31].

We first consider the case $\nu(m)=1$ and denote as $\tilde{P}_{0}(m)$ and
$P_{0}(q)$ the corresponding magnetization and overlap distributions. We note
$\varepsilon=(T_{c}-T)/T_{c}$ and $\mu_{0}=1-(T-J_{1})(T-J_{2})$. At the
transition from a paramagnetic phase to an oscillating phase (small
$\varepsilon>0$), one finds for $\mu>\mu_{0}$ that
$q_{0}=m_{0}^{2}=\varepsilon a$ (see [42] for the expression of $a$) 111The
case $\mu=\mu_{0}$ is described in [42]. and

$\tilde{P}_{0}(m)=\frac{1}{\pi
m_{0}}\,\left[1-\left(\frac{m}{m_{0}}\right)^{2}\right]^{-1/2}\theta(m_{0}-|m|).$
(19)

The overlap distribution is given by $P_{0}(q)=q_{0}^{-1}\psi_{0}(q/q_{0})$,
with

$\psi_{0}(y)=\frac{2}{\pi^{2}}\int_{|y|}^{1}\frac{\mathrm{d}x}{\sqrt{(1-x^{2})(x^{2}-y^{2})}}\,\theta(1-|y|).$
(20)

The overlap distribution has a logarithmic divergence for $q\to 0$, and a non-
zero limit at the boundaries of its support, $P_{0}(\pm q_{0})=\frac{1}{\pi
q_{0}}$ for $|q|\to q_{0}$ [31, 42]. Now considering the case when $\nu(m)$ is
given by Eq. (8) with $\tilde{P}(m)=\tilde{P}_{\alpha\beta}(m)$ [Eq. (16)]
associated with a prescribed overlap distribution $P_{\alpha\beta}(q)$ as
given in Eq. (15), we simulate the stochastic spin dynamics to numerically
determine the overlap distribution $P(q)$ from the microscopic dynamics in a
finite size system. We find that by increasing system size, the numerically
measured overlap distributions $P(q)$ converges to the prescribed overlap
distribution $P_{\alpha\beta}(q)$ [Fig. 1(a,b)]. The corresponding limit
cycles in the plane ($m$, $\dot{m}$) are displayed in the insets of Fig.
1(a,b).

Figure 1: Overlap distribution $P(q)$ (full lines) measured in a stochastic
simulation of the spin model of size $N$, with a frequency $\nu(m)$ determined
from a prescribed distribution $P_{\alpha\beta}(q)$ (dashed lines), as given
in Eq. (15). (a) $\alpha=0.5$, $\beta=2$; (b) $\alpha=2$, $\beta=2$.
Convergence to the prescribed overlap distribution is observed by increasing
the system size $N$ (in both panels, $N=1000$, $2000$, $5000$ and $10000$,
from lighter to darker colors). Inset: limit cycle in the phase space
($m,\dot{m}$) in the deterministic limit $N\to\infty$. Parameters: $T=0.45$,
$\mu=2$, $J_{1}=0$, $J_{2}=1$.

We finally turn to the physical interpretation of the fact that the overlap
probability distribution is spread over a continuous interval, a property
reminiscent of the full replica symmetry breaking scenario in the context of
disordered systems like spin-glasses [16]. In the spin models considered here,
a replica symmetry breaking is not expected since no disorder is present. Yet,
similarities with disordered systems arise if one thinks in terms of pure
states. Indeed, the standard interpretation of replica symmetry breaking is
that configuration space is split into a large number of pure states, for
which connected correlations vanish at large distance [16]. The average value
of an observable $O$ is thus given by an average over pure states, $\langle
O\rangle=\sum_{\alpha}w_{\alpha}\langle O\rangle_{\alpha}$, where $w_{\alpha}$
is the probability weight of pure state $\alpha$, and $\langle
O\rangle_{\alpha}$ is the corresponding pure state average of the observable
$O$ [16]. Although no disorder is present, the situation is similar in spin
models with spontaneous magnetization oscillations. Pure states, labeled by
the magnetization $m$, are given by the factorized distribution
$\mathcal{P}(\\{s_{i}\\}|m)$ defined in Eq. (4), and have a probability weight
$\tilde{P}(m)$. The average of an arbitrary observable $O(\\{s_{i}\\})$ is
then obtained as

$\langle O\rangle=\int_{-1}^{1}dm\,\tilde{P}(m)\langle O\rangle_{m}$ (21)

with the pure-state average

$\langle
O\rangle_{m}=\sum_{\\{s_{i}\\}}\mathcal{P}(\\{s_{i}\\}|m)\,O(\\{s_{i}\\}).$
(22)

At odds with spin-glasses, though, pure states are not organized in a tree-
like, ultrametric structure [16]. Defining the distance $d_{ab}$ between two
pure states $a$ and $b$ as $d_{ab}=|m_{a}-m_{b}|$ the distances between three
pure states $a$, $b$ and $c$ satisfy a triangle inequality $d_{ab}\leq
d_{ac}+d_{cb}$, confirming the absence of ultrametricity 222In a spin glass
context, the Hamming distance $d_{ab}=(1-q_{ab})/2$ is often considered. Here,
$q_{ab}=m_{a}m_{b}$, and the Hamming distance would be nonzero for
$m_{a}=m_{b}$, which would contradict the fact that there is a single pure
state with a given magnetization $m$.. In addition, the self-overlap
$q_{aa}=m_{a}^{2}$ is not the same for all pure states, again at variance with
the standard mean-field spin-glass scenario [16].

To sum up, we have shown that the overlap distribution, a highly non-trivial
observable unveiling complex statistical states, can be tailored to a broad
class of prescribed distributions in driven mean-field spin models exhibiting
spontaneous oscillations of the magnetization. The overlap distribution can be
directly traced back to the shape of the limit cycle in the plane ($m$,
$\dot{m})$ of the magnetization and its time derivative, while the limit cycle
shape can be prescribed by monitoring in a simple way the microscopic spin
dynamics. This work thus provides an example of a highly non-trivial inverse
problem that can be solved explicitly. As for future work, it would be of
interest to see if such an approach may be extended beyond mean-field spin
models, by considering for instance finite-dimensional spin models or
interacting particle models.

###### Acknowledgements.

L. G. acknowledges funding from the French Ministry of Higher Education and
Research.

## References

  * Cranmer  _et al._ [2020] K. Cranmer, J. Brehmer, and G. Louppe, Proc. Nat. Acad. Sci. USA 117, 30055 (2020).
  * Nguyen  _et al._ [2017] H. Nguyen, R. Zecchina, and J. Berg, Adv. Phys. 66, 197 (2017).
  * Mora  _et al._ [2016] T. Mora, A. Walczak, L. Del Castello, F. Ginelli, S. Melillo, L. Parisi, M. Viale, A. Cavagna, and I. Giardina, Nat. Phys. 12, 1153 (2016).
  * Bialek  _et al._ [2012] W. Bialek, A. Cavagna, I. Giardina, T. Mora, E. Silvestri, M. Viale, and A. M. Walczak, Proc. Nat. Acad. Sci. USA 109, 4786 (2012).
  * Schneidman  _et al._ [2006] E. Schneidman, M. J. Berry, R. Segev, and W. Bialek, Nature 440, 1007 (2006).
  * Cocco  _et al._ [2009] S. Cocco, S. Leibler, and R. Monasson, Proc. Natl. Acad. Sci. USA 106, 14058 (2009).
  * Lezon  _et al._ [2006] T. R. Lezon, J. R. Banavar, M. Cieplak, A. Maritan, and N. V. Fedoroff, Proc. Natl. Acad. Sci. USA 103, 19033 (2006).
  * Locasale and Wolf-Yadlin [2009] J. W. Locasale and A. Wolf-Yadlin, PLoS ONE 4, e6522 (2009).
  * Molinelli  _et al._ [2013] E. J. Molinelli, A. Korkut, W. Wang, M. L. Miller, N. P. Gauthier, X. Jing, P. Kaushik, Q. He, G. Mills, D. B. Solit, C. A. Pratilas, M. Weigt, A. Braunstein, A. Pagnani, R. Zecchina, and C. Sander, PLoS Comput. Biol. 9, e1003290 (2013).
  * Durve  _et al._ [2020] M. Durve, F. Peruani, and A. Celani, Phys. Rev. E 102, 012601 (2020).
  * Falk  _et al._ [2021] M. J. Falk, V. Alizadehyazdi, H. Jaeger, and A. Murugan, Phys. Rev. Research 3, 033291 (2021).
  * VanSaders and Vitelli [2023] B. VanSaders and V. Vitelli, Informational active matter (2023), arXiv:2302.07402 .
  * Nasiri  _et al._ [2023] M. Nasiri, H. Löwen, and B. Liebchen, EPL 142, 17001 (2023).
  * Devereux and Turner [2023] H. L. Devereux and M. S. Turner, Phys. Rev. Lett. 130, 168201 (2023).
  * Ben Zion  _et al._ [2023] M. Y. Ben Zion, J. Fersula, N. Bredeche, and O. Dauchot, Science Robotics 8 (2023).
  * Mézard  _et al._ [1987] M. Mézard, G. Parisi, and M. A. Virasoro,  _Spin Glasses and Beyond_ (World Scientific, Singapore, 1987).
  * Carpentier and Orignac [2008] D. Carpentier and E. Orignac, Phys. Rev. Lett. 100, 057207 (2008).
  * Wittmann  _et al._ [2014] M. Wittmann, B. Yucesoy, H. G. Katzgraber, J. Machta, and A. P. Young, Phys. Rev. B 90, 134419 (2014).
  * Guiselin  _et al._ [2022] B. Guiselin, G. Tarjus, and L. Berthier, Phys. Chem. Glasses 63, 136 (2022).
  * Berthier  _et al._ [2016] L. Berthier, P. Charbonneau, and S. Yaida, J. Chem. Phys. 144, 024501 (2016).
  * Berthier [2013] L. Berthier, Phys. Rev. E 88, 022313 (2013).
  * Niedda  _et al._ [2023] J. Niedda, G. Gradenigo, L. Leuzzi, and G. Parisi, SciPost Phys. 14, 144 (2023).
  * Ghofraniha  _et al._ [2015] N. Ghofraniha, I. Viola, F. Di Maria, G. Barbarella, G. Gigli, L. Leuzzi, and C. Conti, Nat. Comm. 6, 6058 (2015).
  * Krabbe  _et al._ [2023] P. Krabbe, H. Schawe, and A. K. Hartmann, Phys. Rev. B 107, 064208 (2023).
  * Mézard  _et al._ [2005] M. Mézard, T. Mora, and R. Zecchina, Phys. Rev. Lett. 94, 197205 (2005).
  * Montemurro  _et al._ [2000] M. A. Montemurro, F. A. Tamarit, D. A. Stariolo, and S. A. Cannas, Phys. Rev. E 62, 5721 (2000).
  * Györgyi and Reimann [2000] G. Györgyi and P. Reimann, J. Stat. Phys. 101, 679 (2000).
  * Altieri  _et al._ [2021] A. Altieri, F. Roy, C. Cammarota, and G. Biroli, Phys. Rev. Lett. 126, 258301 (2021).
  * Manzo and Peliti [1994] F. Manzo and L. Peliti, J. Phys. A: Math. Gen. 27, 7079 (1994).
  * Hartmann [2022] A. K. Hartmann, EPL 137, 41002 (2022).
  * Guislain and Bertin [2023] L. Guislain and E. Bertin, Phys. Rev. Lett. 130, 207102 (2023).
  * [32] L. Guislain and E. Bertin, preprint arxiv:2310:13488.
  * Collet [2014] F. Collet, J. Stat. Phys. 157, 1301 (2014).
  * Collet  _et al._ [2016] F. Collet, M. Formentin, and D. Tovazzi, Phys. Rev. E. 94, 042139 (2016).
  * Collet and Formentin [2019] F. Collet and M. Formentin, J. Stat. Phys. 176, 478 (2019).
  * De Martino and Barato [2019] D. De Martino and A. C. Barato, Phys. Rev. E 100, 062123 (2019).
  * De Martino [2019] D. De Martino, J. Phys. A: Math. Theor. 52, 045002 (2019).
  * Dai Pra  _et al._ [2020] P. Dai Pra, M. Formentin, and P. Guglielmo, J. Stat. Phys. 179, 690 (2020).
  * [39] Y. Avni, M. Fruchart, D. Martin, D. Seara, and V. Vitelli, preprint arxiv:2311:05471.
  * Hewitt and Savage [1955] E. Hewitt and L. J. Savage, Trans. Amer. Math. Soc. 80, 470 (1955).
  * Aldous [1985] D. J. Aldous, in  _Ecole d’Eté de Probabilités de Saint-Flour XIII – 1983_ , Lecture Notes in Mathematics, edited by P. L. Hennequin (Springer, Berlin, Heidelberg, 1985) pp. 2–198.
  * [42] See Supplemental Material at [URL will be inserted by publisher] for technical details.
  * Sato [2013] K.-I. Sato,  _Lévy Processes and Infinitely Divisible Distributions (2nd Ed.)_ (Cambridge University Press, 2013).
  * Note [1] The case $\mu=\mu_{0}$ is described in [42].
  * Note [2] In a spin glass context, the Hamming distance $d_{ab}=(1-q_{ab})/2$ is often considered. Here, $q_{ab}=m_{a}m_{b}$, and the Hamming distance would be nonzero for $m_{a}=m_{b}$, which would contradict the fact that there is a single pure state with a given magnetization $m$.

Supplementary Information: Tailoring the overlap distribution in driven mean-
field spin model

## Derivation of the overlap distribution

We provide here a short derivation of the expression of the overlap
distribution $P(q)$ given in Eq. (5) of the main text. We introduce the
Fourier transform (or characteristic function) $\chi(\omega)$ of the overlap
distribution $P(q)$,

$\chi(\omega)=\int_{-1}^{1}dqP(q)e^{i\omega q}.$ (1)

Integrating the $\delta-$function of Eq. (2) [main text] and using the
expression of $P_{s}(\\{s_{i}\\})$ given in Eq. (3) [main text],
$\chi(\omega)$ becomes

$\begin{split}\chi(\omega)=\iint
dm_{a}dm_{b}\tilde{P}(m_{a})\tilde{P}(m_{b})e^{\frac{N}{2}\ln\left(\frac{(1-m_{a}^{2})}{4}\frac{(1-m_{b}^{2})}{4}\right)}\\\
\times\sum_{\\{s_{i}^{a}\\},\\{s_{i}^{b}\\}}\prod_{i=1}^{N}\left(\frac{1+m_{a}}{1-m_{a}}\right)^{\frac{s_{i}^{a}}{2}}\\!\left(\frac{1+m_{b}}{1-m_{b}}\right)^{\frac{s_{i}^{b}}{2}}\\!e^{\frac{i\omega}{N}s_{i}^{a}s_{i}^{b}}\,.\end{split}$
(2)

Exchanging sum and product, the term on the second line of Eq. (2) simplifies
to

$\left(\gamma_{1}e^{\frac{i\omega}{N}}+\gamma_{2}e^{-\frac{i\omega}{N}}\right)^{N}$
(3)

with

$\gamma_{1}=\sqrt{\frac{1+m_{a}}{1-m_{a}}\frac{1+m_{b}}{1-m_{b}}}+\sqrt{\frac{1-m_{a}}{1+m_{a}}\frac{1-m_{b}}{1+m_{b}}},$
(4)

and

$\gamma_{2}=\sqrt{\frac{1+m_{a}}{1-m_{a}}\frac{1-m_{b}}{1+m_{b}}}+\sqrt{\frac{1-m_{a}}{1+m_{a}}\frac{1+m_{b}}{1-m_{b}}}\,.$
(5)

In the large $N$ limit, it becomes

$\left(\gamma_{1}e^{\frac{i\omega}{N}}+\gamma_{2}e^{-\frac{i\omega}{N}}\right)^{N}=e^{\frac{N}{2}\ln(\gamma_{1}+\gamma_{2})^{2}}e^{i\omega\frac{\gamma_{1}-\gamma_{2}}{\gamma_{1}+\gamma_{2}}}\,.$
(6)

One can check the following identities,

$\frac{1}{\left(\gamma_{1}+\gamma_{2}\right)^{2}}=\frac{(1-m_{a}^{2})}{4}\frac{(1-m_{b}^{2})}{4}$
(7)

and

$\frac{\gamma_{1}-\gamma_{2}}{\gamma_{1}+\gamma_{2}}=m_{a}m_{b}\,.$ (8)

The Fourier transform of the overlap distribution $P(q)$ thus becomes

$\chi(\omega)=\iint dm_{a}dm_{b}\tilde{P}(m_{a})\tilde{P}(m_{b})e^{i\omega
m_{a}m_{b}}\,.$ (9)

Taking the inverse Fourier transform, one obtains

$P(q)=\iint dm_{a}dm_{b}\tilde{P}(m_{a})\tilde{P}(m_{b})\delta(m_{a}m_{b}-q),$
(10)

which becomes Eq. (5) of the main text after integration over $m_{b}$.

## Expression of the model coefficients

The explicit spin model defined in the main text involves the parameters
$\varepsilon$, $\mu_{0}$, $\alpha_{1}$ and $\alpha_{2}$, whose values are
given by:

$\displaystyle\varepsilon=(T_{c}-T)/T_{c},$ (11)
$\displaystyle\mu_{0}=1-T^{2}(1-J_{1}/T)(1-J_{2}/T),$
$\displaystyle\alpha_{1}=(2J_{2}+J_{2}^{3}+3J_{1})/T-2(1+J_{2}^{2})(-1+J_{1}J_{2}+\mu)/T^{2}$
$\displaystyle\qquad-2+J_{2}(-1+J_{1}J_{2}+\mu)^{2}/T^{3},$
$\displaystyle\alpha_{3}=-2/3+(2J_{2}+J_{2}^{3}+3J_{1})/3T.$

For $\mu>\mu_{0}$, one gets $m_{0}^{2}=a\varepsilon$ when $\varepsilon$ is
small, with

$a=\frac{8T_{c}T}{T^{2}\alpha_{1}+3(\mu-\mu_{0})\alpha_{3}},$ (12)

while for $\mu=\mu_{0}$, one gets $m_{0}=b\varepsilon$ with

$b=\frac{5\pi^{2}}{3\Gamma(3/4)^{2}}\frac{T_{c}}{T\alpha_{1}}.$ (13)

## Overlap distribution for $\mu=\mu_{0}$

Figure 1: Magnetization distribution $\tilde{P}_{0}(m)$ (left panel) and
overlap distribution $P_{0}(q)$ (right panel) for $\mu>\mu_{0}$ (blue/dark
color) and $\mu=\mu_{0}$ (orange/light color), upon rescaling by the size of
their support.

Close to the tricritical point, for $\mu=\mu_{0}$ and $\varepsilon>0$, the
magnetization still oscillates but its probability distribution
$\tilde{P}_{0}^{\mathrm{tc}}(m)$ takes a different form. One finds
$m_{0}=b\varepsilon$ [with $b$ given in Eq. (13)] and

$\tilde{P}_{0}^{\mathrm{tc}}(m)=\frac{d}{m_{0}}\left|1-\left(\frac{m}{m_{0}}\right)^{4}\right|^{-1/2}\theta(m_{0}-|m|),$
(14)

with $d=\sqrt{2}\Gamma(3/4)^{2}/\pi^{3/2}$. The overlap distribution
$P_{0}^{\mathrm{tc}}(q)$ is then given by

$P_{0}^{\mathrm{tc}}(q)=\frac{2d^{2}}{q_{0}}\int_{|q|/q_{0}}^{1}\frac{\mathrm{d}x\,x}{\sqrt{(1-x^{4})[x^{4}-(q/q_{0})^{4}]}}\,\theta(q_{0}-|q|),$
(15)

with $q_{0}=m_{0}^{2}$. We plot in Fig. 1 the overlap distributions $P_{0}(q)$
and $P_{0}^{\mathrm{tc}}(q)$, corresponding to the cases $\mu>\mu_{0}$ and
$\mu=\mu_{0}$ respectively, for small $\varepsilon$. Both overlap
distributions have a logarithmic divergence for $q\to 0$, and have a non-zero
limit at the boundaries of their support, in $|q|\to m_{0}^{2}$,

$P_{0}(\pm q_{0})=\frac{1}{\pi q_{0}}\,,\quad P_{0}^{\mathrm{tc}}(\pm
q_{0})=\frac{\Gamma(\frac{3}{4})^{4}}{\pi^{2}q_{0}}\,.$ (16)

## Deterministic equations with $\nu(m)$

In the limit of infinite system size, one gets the following deterministic
equations for the dynamic of $m(t)$ and $h(t)$,

$\displaystyle\frac{dm}{dt}$ $\displaystyle=\nu(m)[-m+\tanh(\beta J_{1}m+\beta
h)]$ (17) $\displaystyle\frac{dh}{dt}$ $\displaystyle=\nu(m)[-h+\tanh(\beta
J_{2}h+\beta(1-\mu)m)].$

The function $\nu(m)$ controls the local time scale of the dynamics, and it
can be reabsorbed into a non-linear reparametrization of time $t\to
t^{\prime}$ such that $dt^{\prime}=\nu(m)dt$. Rewriting Eqs. (17) in terms of
the dimensionless time variable $t^{\prime}$, the frequency $\nu(m)$ no longer
explicitly appears in the dynamics. This means in particular that the shape of
the limit cycle in the ($m$, $h$)-plane is independent of the functional form
of $\nu(m)$. However, the local speed along the limit cycle does depend on
$\nu(m)$. Reexpressing the dynamics in terms of the generic variables ($m$,
$\dot{m}$) as done in the main text, one finds that the shape of the limit
cycle in these variables does depend on $\nu(m)$.

## Overlap distribution from stochastic simulations

The overlap distribution $P_{0}(q)$ measured from stochastic simulations of
the spin model of size $N$ for $\nu(m)=1$ is plotted in Fig. 2(a). The
convergence to the theoretical limit (dashed line) is observed by increasing
the system size $N$. In Fig. 1 of the main text, two examples of overlap
distributions measured from stochastic simulations of the spin model with a
frequency $\nu(m)$ determined from a prescribed distribution
$P_{\alpha\beta}(q)$ [Eq. (15) of the main text] are displayed. Another
example is given here in Fig. 2(b) for $\alpha=1.2$ and $\beta=1$, where the
overlap distribution has a slow divergence in $q=0$ as $(-\ln|q|)^{0.2}$ and
converges to zero at the boundary of the support. The convergence to the
prescribed overlap distribution $P_{\alpha\beta}(q)$ for these parameters is
slower than for the parameters given in the main text.

Figure 2: (a) Overlap distribution $P_{0}(q)$ (full lines) measured in a
stochastic simulation of the spin model of size $N$ and theoretical prediction
in the $N\to\infty$ limit (dashed line) from Eq. (20) of the main text. (b)
$P(q)$ measured in a stochastic simulation of the spin model of size $N$ with
a frequency $\nu(m)$ determined from a prescribed distribution
$P_{\alpha\beta}(q)$ (dashed line), as given in Eq. (15) of the main text, for
$\alpha=1.2$ and $\beta=1$. A slow convergence to the prescribed overlap
distribution is observed by increasing the system size $N$. In both panels,
$N=1000$, $2000$, $5000$ and $10000$, from lighter to darker colors. Inset:
limit cycle in the phase space $(m,\dot{m})$ in the deterministic limit
$N\to\infty$. Parameters: $T=0.45$, $\mu=2$, $J_{1}=0$, $J_{2}=1$.