# Critical point representation of the mutual information in the sparse
stochastic block model

Tomas Dominguez Department of Mathematics, University of Toronto,
<EMAIL_ADDRESS>Jean-Christophe Mourrat Department of
Mathematics, ENS Lyon and CNRS<EMAIL_ADDRESS>
###### Abstract

We consider the problem of recovering the community structure in the
stochastic block model. We aim to describe the mutual information between the
observed network and the actual community structure as the number of nodes
diverges while the average degree of a given node remains bounded. Our main
contribution is a representation of the limit of this quantity, assuming it
exists, as an explicit functional evaluated at a critical point of that
functional. While we mostly focus on the two-community setting for clarity, we
expect our method to be robust to model generalizations. We also present an
example involving four communities where we show the invalidity of a plausible
candidate variational formula for this limit.

##  1 Introduction

###  1.1 Informal summary of the main results

We consider a problem of community detection in the context of the stochastic
block model. We focus on the case with two communities, where the community
structure is encoded by a vector

$\sigma^{*}:=(\sigma^{*}_{1},\ldots,\sigma^{*}_{N})\in\Sigma_{N}:=\\{-1,+1\\}^{N}.$
(1.1)

We understand that individuals $i$ and $j$ belong to the same community if and
only if $\sigma^{*}_{i}=\sigma^{*}_{j}$. The labels $(\sigma^{*}_{i})_{i\leq
N}$ are sampled independently from a Bernoulli distribution $P^{*}$ with
probability of success $p\in(0,1)$ and expectation $\overline{m}$,

$p:=P^{*}\\{1\\}=\mathbb{P}\\{\sigma^{*}_{i}=1\\}\quad\text{and}\quad\overline{m}:=\operatorname{\mathbb{E}}\sigma^{*}_{i}=2p-1.$
(1.2)

The assignment vector $\sigma^{*}$ is thus distributed according to the
product law

$\sigma^{*}\sim P_{N}^{*}:=(P^{*})^{\otimes N}.$ (1.3)

We fix parameters $c>0$ and $\Delta\in(-c,c)\setminus\\{0\\}$. The case when
$\Delta=0$ would be trivial and we prefer to exclude it. Given the community
structure $\sigma^{*}$, we sample a random undirected graph
$\mathbf{G}_{N}:=(G_{ij})_{i,j\leq N}$ with vertex set $\\{1,\ldots,N\\}$ by
stipulating that an edge between node $i$ and node $j$ is present with
conditional probability

$\mathbb{P}\\{G_{ij}=1\mid\sigma^{*}\\}:=\frac{c+\Delta\sigma_{i}^{*}\sigma_{j}^{*}}{N},$
(1.4)

and that this sampling is performed independently for each pair $i\neq
j\in\\{1,\ldots,N\\}$. In words, we create a link between two vertices with
probability $(c+\Delta)/N$ if those two vertices belong to the same community;
and we create this link with probability $(c-\Delta)/N$ otherwise. In the case
when $\Delta>0$, it is more likely for an edge to be present between nodes
belonging to the same community, and the model is called _assortative_ ;
otherwise it is called _disassortative_. The inference task that we consider
is that of recovering the community structure $\sigma^{*}$ given the
observation of the graph $\mathbf{G}_{N}$. We focus in particular on
characterizing the mutual information between the graph $\mathbf{G}_{N}$ and
the community structure $\sigma^{*}$, in the limit of large $N$.

We denote by $\mathcal{M}_{+}$ the space of non-negative measures over
$[-1,1]$ with finite total mass, and by $\mathcal{M}_{p}$ the set of
probability measures with mean $\overline{m}=2p-1$,

$\mathcal{M}_{p}:=\bigg{\\{}\nu\in\Pr[-1,1]\mid\int_{-1}^{1}x\,\mathrm{d}\nu(x)=\overline{m}\bigg{\\}}.$
(1.5)

Our main results will be stated in terms of a fixed point of the functional
$\Gamma_{t,\mu}:\mathcal{M}_{p}\to\mathcal{M}_{p}$, which is defined for each
$t\geq 0$ and $\mu\in\mathcal{M}_{+}$ according to the following procedure.
Fixing $\nu\in\mathcal{M}_{p}$, we sample $\epsilon^{*}$ according to $P^{*}$,
and conditionally on $\epsilon^{*}$, we let $\Pi_{t,\mu}(\nu)$ be a Poisson
point process with intensity measure
${(c+\Delta\epsilon^{*}x)}\,\mathrm{d}(\mu+t\nu)(x)$; we then set

$\Gamma_{t,\mu}(\nu):=\text{Law}\Bigg{(}\frac{\int_{\Sigma_{1}}\epsilon
e^{-\epsilon\overline{m}}\prod_{x\in\Pi_{t,\mu}(\nu)}(c+\Delta\epsilon
x)\,\mathrm{d}P^{*}(\epsilon)}{\int_{\Sigma_{1}}e^{-\epsilon\overline{m}}\prod_{x\in\Pi_{t,\mu}(\nu)}(c+\Delta\epsilon
x)\,\mathrm{d}P^{*}(\epsilon)}\Bigg{)}.$ (1.6)

In other words, the quantity between the large parentheses in (1.6) is a
random variable, as it depends on the realization of $\varepsilon^{*}$ and
$\Pi_{t,\mu}(\nu)$, and we define $\Gamma_{t,\mu}(\nu)$ to be the law of this
random variable.

Let $(\sigma^{\ell})_{\ell\geq 1}=((\sigma^{\ell}_{i})_{i\leq N})_{\ell\geq
1}$ be independent random variables that are sampled according to the
conditional law of $\sigma^{*}$ given the observation of $\mathbf{G}_{N}$. Our
first main result states that, up to a perturbation of the inference problem
that does not change the asymptotic behavior of the mutual information, and up
to the extraction of a subsequence in $N$, the asymptotic law of
$(\sigma^{\ell}_{i})_{i\leq N,\ell\geq 1}$ can be sampled as follows. For some
measure $\nu^{*}$ that is a fixed point of the operator $\Gamma_{1,0}$:

  1. S1

generate a sequence $(x_{i})_{i\geq 1}$ of i.i.d. random variables with law
$\nu^{*}$;

  2. S2

conditionally on $(x_{i})_{i\geq 1}$, generate $\pm 1$-valued random variables
$\sigma_{i}^{\ell}$ with mean $x_{i}$, independently over $i,\ell\geq 1$.

Our second main result states that, assuming that the limit mutual information
exists, it can be expressed in terms of an explicit functional
$\operatorname{\mathcal{P}}_{1,0}:\mathcal{M}_{+}\to\mathbb{R}$ that is
evaluated at a fixed point of the map $\Gamma_{1,0}$. The functional
$\operatorname{\mathcal{P}}_{1,0}$ is closely related to the map
$\Gamma_{1,0}$ since, at least formally, a measure is a critical point of
$\operatorname{\mathcal{P}}_{1,0}$ if and only if it is a fixed point of the
map $\Gamma_{1,0}$.

While we think of our results as being rather precise, since they
substantially narrow down the space of possible asymptotic behaviors for the
mutual information and for the conditional law of $\sigma^{*}$ given the
observation of $\mathbf{G}_{N}$, we also wish to stress that our results do
not provide a complete characterization of the limits of these objects.
Indeed, this is due to the fact that the map $\Gamma_{1,0}$ may admit several
fixed points in general. We will show that the existence of a unique fixed
point can be guaranteed in a regime of small signal-to-noise ratio, but we do
not expect this uniqueness property to extend to arbitrary choices of
parameters.

A complete characterization of the limit of the mutual information has already
been achieved in a number of cases. The disassortative case was settled in [2,
15] when $p=1/2$, and in [18] for arbitrary $p$. The technique in [18] is
based on partial differential equations (see also [19] for a presentation of
the approach), and the key point is that in the disassortative case, the non-
linearity appearing in this equation is _convex_. We expect this complete
characterization to extend to settings with more than two communities, as long
as the corresponding non-linearity remains convex. This would be analogous to
[11, Theorem 1.1] in the context of spin glasses, a result which itself builds
upon [9, 22, 42, 43, 44, 45, 46, 47, 52]. Moreover, whenever this convexity
property holds, the limit mutual information can be expressed as a variational
formula. From the point of view of partial differential equations, this
relates to the validity of a Hopf-Lax representation of the solution, see [20,
Theorem 1.3].

The assortative case presents a greater challenge. It was successfully
resolved when $p=1/2$ in [53], building upon [1, 24, 34, 36]. The approach in
[53] relies upon the study of the fixed points of an operator that relates to
the setting of broadcasting on trees, and the key step of the proof is to
establish the uniqueness of a non-trivial fixed point. This uniqueness
property does not generalize to models with more than two communities [21].
Despite similarities, we could not identify a precise correspondence between
the operator $\Gamma_{1,0}$ whose fixed points enter into our analysis and the
operators appearing in [53]. Part of the difficulty is that the first step of
the approach in these earlier works is to decompose the mutual information
into an integral involving a family of new statistical inference problems with
an additional survey mechanism. When one can guarantee that there is a unique
non-trivial fixed point to each problem in this family, there is a tight
relationship between the true community-detection problem and the problems
involving broadcasting on trees. On the other hand, when uniqueness fails,
this relationship seems less rigid to us, and we do not immediately see how
one could derive results comparable to those obtained in the present paper in
the context explored in these earlier works.

We expect the method presented here, primarily based on cavity calculations,
to be robust to model variations such as situations involving more than two
communities, and we also expect the lower bound on the mutual information
obtained in [18] to generalize similarly. The results we present here and
those in [18] are analogous to those obtained for spin glasses in [11] and in
[39, 40] respectively. We have preferred to stick to a two-community model
here to facilitate the presentation, but as an illustration of the robustness
of the approach, we will at least indicate as we proceed the few small
adaptations that need to be implemented in order to cover the case when then
measure $P^{*}$ is arbitrary with support in $[-1,1]$ in place of
$\\{-1,1\\}$.

In order to make further progress and obtain a full characterization of the
limit mutual information in general, one must first ask what a good candidate
for the limit could be. One proposal based on partial differential equations
was put forward in [18]. It would of course be convenient to identify a more
explicit formulation of the candidate limit. For dense versions of the problem
we consider, where the average degree of the graph diverges to infinity, a
sort of central limit theorem takes place at the level of each node, and one
can relate the problem to a simpler setup such as that of rank-one matrix
estimation in the presence of additive Gaussian noise, as shown in [17, 26]
(see also [19, Section 4.5]). For the latter class of problems, a saddle-point
variational formula was shown to be valid in a very wide range of settings in
[14] (see also [19, Chapter 4]), building upon [4, 5, 6, 10, 13, 12, 23, 26,
27, 28, 29, 31, 32, 37, 38, 50, 51]. From the point of view of partial
differential equations, this corresponds to the possibility to write a Hopf
formula based on the convexity of the initial condition (as opposed to the
convexity of the non-linearity for the Hopf-Lax formula, cf. [19, Theorems 3.8
and 3.13]). However, the convexity of the relevant “initial condition” in our
setting is invalid [25], and as explained around [18, (1.49)], this implies
that this tentative saddle-point Hopf formula is invalid in our setting with
sparse graph connectivity. An alternative candidate limit for the mutual
information could be considered by extending the Hopf-Lax variational formula
known to be valid in the disassortative case. While we could not rule this out
in the two-community case, we show here that this candidate variational
formula is invalid in general. To do so, we focus on a version of the two-
community problem in which the graph additionally possesses a bipartite
structure. The model can be thought of as involving four communities, and it
is inspired by the bipartite spin-glass model investigated in [39] (see in
particular [39, Section 6]). In this context, we show that a naive
generalization of the formula valid in the standard disassortative two-
community case would lead to a prediction that is in contradiction with the
main results of this paper.

As was already said, the main focus of this paper is to obtain precise
characterizations of the mutual information and conditional law of the
community structure given the observation, in the limit of large system size.
In the two-community case with $p=1/2$, it was shown in [30, 33, 35] that
there exists an efficient algorithm that allows one to recover non-trivial
information about the community structure whenever $\Delta^{2}>c$, and that it
is information-theoretically impossible to do so otherwise. To the best of our
knowledge, whether or not there exists an efficient algorithm that recovers as
much of the signal as is information-theoretically possible is not known. In
certain settings involving more than two communities, one expects the
existence of a gap between the regime for which there exists an efficient
algorithm for recovering a non-trivial fraction of the community structure,
and the regime for which it is information-theoretically possible to do so.
The results of [3] vindicate a significant part of this picture. A precise
characterization of the information-theoretic limit is not known however.
These open questions are directly related to the main purposes of the present
paper. Much of the recent work on the community detection problem has been
triggered by [16], in which the results reviewed in this paragraph and much
more had been anticipated using non-rigorous methods.

###  1.2 Precise statement of the main results

We now turn to a more precise description of our main results. As in [18], our
starting point is to embed the community-detection problem under consideration
into a richer family parametrized by a pair $(t,\mu)\in\mathbb{R}_{\geq
0}\times\mathcal{M}_{+}$. The conjecture in [18] is that, up to a simple
affine transformation, the limit $f(t,\mu)$ of the re-scaled mutual
information between the signal and the observation solves an explicit
Hamilton-Jacobi equation. As is well-known in finite dimensions, first-order
Hamilton-Jacobi equations can be solved for a short time using the method of
characteristics (see for instance [19, Exercise 3.10 and solution]). This
method allows us to uncover the value of the solution to the equation along
each characteristic line separately. At larger times though, difficulties
occur because characteristic lines may intersect, and will typically not agree
on what the value of the solution ought to be. Our main result can be
interpreted as saying that, while this might indeed occur, there is always one
characteristic line that prescribes the correct value for $f(t,\mu)$. In the
language of [19, Section 3.5], one may say that the graph of $f$ always
belongs to the wavefront of the equation.

As already stated, the central object of interest to us is the mutual
information between the community structure $\sigma^{*}$ and the observed
random graph $\mathbf{G}_{N}$, that is,

$I(\mathbf{G}_{N};\sigma^{*}):=\operatorname{\mathbb{E}}\log\frac{\mathbb{P}(\mathbf{G}_{N},\sigma^{*})}{\mathbb{P}(\mathbf{G}_{N})\mathbb{P}(\sigma^{*})}=\operatorname{\mathbb{E}}\int_{\mathbb{R}^{N}}\log\bigg{(}\frac{\,\mathrm{d}P_{\sigma^{*}\mid\mathbf{G}_{N}}}{\,\mathrm{d}P_{N}^{*}}(\sigma)\bigg{)}\,\mathrm{d}P_{\sigma^{*}\mid\mathbf{G}_{N}}(\sigma),$
(1.7)

where $P_{\sigma^{*}\mid\mathbf{G}_{N}}$ denotes the conditional law of
$\sigma^{*}$ given $\mathbf{G}_{N}$. The likelihood of the model is given by

$\mathbb{P}\big{\\{}\mathbf{G}_{N}=(G_{ij})\mid\sigma^{*}=\sigma\big{\\}}=\prod_{i<j}\Big{(}\frac{c+\Delta\sigma_{i}\sigma_{j}}{N}\Big{)}^{G_{ij}}\Big{(}1-\frac{c+\Delta\sigma_{i}\sigma_{j}}{N}\Big{)}^{1-G_{ij}},$
(1.8)

and Bayes’ formula implies that the posterior of the model is the Gibbs
measure

$\mathbb{P}\big{\\{}\sigma^{*}=\sigma\mid\mathbf{G}_{N}=(G_{ij})\big{\\}}=\frac{\exp\big{(}H_{N}^{\circ}(\sigma)\big{)}P_{N}^{*}(\sigma)}{\int_{\Sigma_{N}}\exp\big{(}H_{N}^{\circ}(\tau)\big{)}\,\mathrm{d}P_{N}^{*}(\tau)}$
(1.9)

associated with the Hamiltonian

$H_{N}^{\circ}(\sigma):=\sum_{i<j}\log\bigg{[}\big{(}c+\Delta\sigma_{i}\sigma_{j}\big{)}^{G_{ij}}\Big{(}1-\frac{c+\Delta\sigma_{i}\sigma_{j}}{N}\Big{)}^{1-G_{ij}}\bigg{]}.$
(1.10)

Up to an error vanishing with $N$ and a simple additive constant, the
normalized mutual information (1.7) coincides with the free energy

$\overline{F}_{N}^{\circ}:=\frac{1}{N}\operatorname{\mathbb{E}}\log\int_{\Sigma_{N}}\exp
H_{N}^{\circ}(\sigma)\,\mathrm{d}P_{N}^{*}(\sigma).$ (1.11)

Indeed, one can observe that

$I(\mathbf{G}_{N};\sigma^{*})=\binom{N}{2}\operatorname{\mathbb{E}}\log(c+\Delta\sigma_{1}^{*}\sigma_{2}^{*})^{G_{12}}\Big{(}1-\frac{c+\Delta\sigma_{1}^{*}\sigma_{2}^{*}}{N}\Big{)}^{1-G_{12}}-N\overline{F}_{N}^{\circ};$
(1.12)

averaging with respect to the randomness of $G_{12}$ and Taylor-expanding the
logarithm yields that

$\frac{1}{N}I(\mathbf{G}_{N};\sigma^{*})=\frac{1}{2}\operatorname{\mathbb{E}}\big{(}c+\Delta\sigma_{1}^{*}\sigma_{2}^{*}\big{)}\log\big{(}c+\Delta\sigma_{1}^{*}\sigma_{2}^{*}\big{)}-\frac{c}{2}-\frac{\Delta\overline{m}^{2}}{2}-\overline{F}_{N}^{\circ}+\operatorname{\operatorname{\mathcal{O}}}\big{(}N^{-1}\big{)}.$
(1.13)

In place of the mutual information, the main protagonist of this paper is in
fact the free energy $\overline{F}_{N}^{\circ}$ and its extension, which we
now proceed to introduce.

We begin by describing the enrichment of the free energy involving the
parameter $t\geq 0$. For each $t\geq 0$, we introduce a random variable
$\smash{\Pi_{t}\sim\operatorname{\operatorname{Poi}}t\tbinom{N}{2}}$ as well
as an independent family of i.i.d. random matrices $\smash{(G^{k})_{k\geq 1}}$
each having conditionally independent entries $\smash{(G_{i,j}^{k})_{i,j\leq
N}}$ taking values in $\\{0,1\\}$ with conditional distribution

$\mathbb{P}\big{\\{}G_{i,j}^{k}=1\mid\sigma^{*}\big{\\}}:=\frac{c+\Delta\sigma_{i}^{*}\sigma_{j}^{*}}{N}.$
(1.14)

Given a collection of random indices $\smash{(i_{k},j_{k})_{k\geq 1}}$ sampled
uniformly at random from $\smash{\\{1,\ldots,N\\}^{2}}$, independently of the
other random variables, we define the $t$-dependent Hamiltonian $H_{N}^{t}$ on
$\Sigma_{N}$ by

$H_{N}^{t}(\sigma):=\sum_{k\leq\Pi_{t}}\log\bigg{[}\big{(}c+\Delta\sigma_{i_{k}}\sigma_{j_{k}}\big{)}^{G^{k}_{i_{k},j_{k}}}\Big{(}1-\frac{c+\Delta\sigma_{i_{k}}\sigma_{j_{k}}}{N}\Big{)}^{1-G^{k}_{i_{k},j_{k}}}\bigg{]},$
(1.15)

and denote by

$\overline{F}_{N}(t):=\frac{1}{N}\operatorname{\mathbb{E}}\log\int_{\Sigma_{N}}\exp
H_{N}^{t}(\sigma)\,\mathrm{d}P_{N}^{*}(\sigma)$ (1.16)

its corresponding free energy. Notice that this is the Hamiltonian associated
with the task of inferring the signal $\sigma^{*}$ from the data

$\mathcal{D}_{N}^{t}:=\big{(}\Pi_{t},(i_{k},j_{k})_{k\leq\Pi_{t}},(G^{k}_{i_{k},j_{k}})_{k\leq\Pi_{t}}\big{)}.$
(1.17)

By this we mean that the law of $\sigma^{*}$ given the observation of
$\mathcal{D}_{N}^{t}$ is described by a formula analogous to that in (1.9),
with $H_{N}^{\circ}$ replaced by $H_{N}^{t}$ there. By [18, Appendix A], the
difference between the free energies $\overline{F}_{N}^{\circ}$ and
$\overline{F}_{N}(1)$ converges to zero as $N$ tends to infinity. The main
advantage of this construction is that it allows one to study derivatives of
the free energy with respect to the “time” parameter $t$. It is also
convenient to study derivatives of the free energy with respect to an
additional parameter denoted by $\mu\in\mathcal{M}_{+}$. Given
$\mu\in\mathcal{M}_{+}$, we let $(\Lambda_{i}(N\mu))_{i\leq N}$ be independent
Poisson point processes with intensity measure $N\mu$, and we let the
$\mu$-dependent Hamiltonian $H_{N}^{\mu}$ on $\Sigma_{N}$ be given by

$H_{N}^{\mu}(\sigma):=\sum_{i\leq
N}\sum_{x\in\Lambda_{i}(N\mu)}\log\bigg{[}\big{(}c+\Delta\sigma_{i}x\big{)}^{G^{x}_{i}}\Big{(}1-\frac{c+\Delta\sigma_{i}x}{N}\Big{)}^{1-G^{x}_{i}}\bigg{]},$
(1.18)

where $(G_{i}^{x})_{i\leq N}$ are conditionally independent random variables
taking values in $\\{0,1\\}$ with conditional distribution

$\mathbb{P}\\{G_{i}^{x}=1\mid\sigma^{*},x\\}:=\frac{c+\Delta\sigma_{i}^{*}x}{N}.$
(1.19)

For a review of Poisson point processes, we refer the reader to [19, Chapter
5]. The function $H_{N}^{\mu}$ is the Hamiltonian associated with the task of
inferring the signal $\sigma^{*}$ from the data

${\mathcal{D}}_{N}^{\mu}:=\big{(}\Lambda_{i}(N\mu),(G_{i}^{x})_{x\in\Lambda_{i}(N\mu)}\big{)}_{i\leq
N}.$ (1.20)

Finally, for each $(t,\mu)\in\mathbb{R}_{\geq 0}\times\mathcal{M}_{+}$, we
introduce the enriched Hamiltonian

$H_{N}^{t,\mu}(\sigma):=H_{N}^{t}(\sigma)+H_{N}^{\mu}(\sigma)$ (1.21)

as well as its associated free energy

$\overline{F}_{N}(t,\mu):=\frac{1}{N}\operatorname{\mathbb{E}}\log\int_{\Sigma_{N}}\exp
H_{N}^{t,\mu}(\sigma)\,\mathrm{d}P_{N}^{*}(\sigma).$ (1.22)

The function $H_{N}^{t,\mu}$ is the Hamiltonian associated with inferring the
signal $\sigma^{*}$ from the data

${\mathcal{D}}_{N}^{t,\mu}:=(\mathcal{D}_{N}^{t},\mathcal{D}_{N}^{\mu}),$
(1.23)

where the randomness in these two datasets is taken to be independent
conditionally on $\sigma^{*}$. We recall that by [18, Appendix A], the
difference between the free energy $\overline{F}_{N}^{\circ}$ in (1.11) and
$\overline{F}_{N}(1,0)$ tends to zero as $N$ tends to infinity. The problem of
finding the asymptotic value of the mutual information (1.7) is therefore a
specific instance of the task of determining the limit of the enriched free
energy (1.22).

To state the main results of this paper, we introduce additional notation. We
define the function $g:[-1,1]\to\mathbb{R}$ by

$g(z):=(c+\Delta z)\big{(}\log(c+\Delta z)-1\big{)}=(c+\Delta
z)\log(c)+c\sum_{n\geq 2}\frac{(-\Delta/c)^{n}}{n(n-1)}z^{n}-c,$ (1.24)

and for each measure $\mu\in\mathcal{M}_{+}$, we let
$G_{\mu}:[-1,1]\to\mathbb{R}$ be the function obtained by marginalizing the
kernel $(x,y)\mapsto g(xy)$,

$G_{\mu}(x):=\int_{-1}^{1}g(xy)\,\mathrm{d}\mu(y).$ (1.25)

Given $(t,\mu)\in\mathbb{R}_{\geq 0}\times\mathcal{M}_{+}$, we introduce the
functional $\operatorname{\mathcal{P}}_{t,\mu}:\mathcal{M}_{+}\to\mathbb{R}$
defined by

$\operatorname{\mathcal{P}}_{t,\mu}(\nu):=\psi(\mu+t\nu)-\frac{t}{2}\int_{-1}^{1}G_{\nu}(y)\,\mathrm{d}\nu(y),$
(1.26)

where $\psi:\mathcal{M}_{+}\to\mathbb{R}$ denotes the function

$\displaystyle\psi(\mu):=-\mu[-1,$ $\displaystyle
1]c+p\operatorname{\mathbb{E}}\log\int_{\Sigma_{1}}\exp\Big{(}-\Delta\sigma\int_{-1}^{1}x\,\mathrm{d}\mu(x)\Big{)}\prod_{x\in\Pi_{+}(\mu)}(c+\Delta\sigma
x)\,\mathrm{d}P^{*}(\sigma)$
$\displaystyle+(1-p)\operatorname{\mathbb{E}}\log\int_{\Sigma_{1}}\exp\Big{(}-\Delta\sigma\int_{-1}^{1}x\,\mathrm{d}\mu(x)\Big{)}\prod_{x\in\Pi_{-}(\mu)}(c+\Delta\sigma
x)\,\mathrm{d}P^{*}(\sigma)$ (1.27)

for Poisson point processes $\Pi_{\pm}(\mu)$ with respective intensity
measures $(c\pm\Delta x)\,\mathrm{d}\mu(x)$ on $[-1,1]$. The function $\psi$
is the initial condition of the partial differential equation that was
identified in [18], since it arises as the large-$N$ limit of
$\overline{F}_{N}(0,\mu)$. Our first main result states that the limit of the
free energy, assuming it exists, is given by the functional (1.26) evaluated
at one of its critical points. As mentioned in the previous subsection, it
will be possible to recast the critical-point condition as a fixed-point
equation for the operator $\Gamma_{t,\mu}$ defined in (1.6), so that our first
main result reads as follows.

######  Theorem 1.1.

Suppose that the sequence of enriched free energies $(\overline{F}_{N})_{N\geq
1}$ converges pointwise to some limit $f:\mathbb{R}_{\geq
0}\times\mathcal{M}_{+}\to\mathbb{R}$. For every $(t,\mu)\in\mathbb{R}_{\geq
0}\times\mathcal{M}_{+}$, the map $\nu\mapsto\Gamma_{t,\mu}(\nu)$ admits a
fixed point $\nu^{*}\in\mathcal{M}_{p}$ with the property that

$f(t,\mu)=\operatorname{\mathcal{P}}_{t,\mu}(\nu^{*}).$ (1.28)

Our second result concerns the asymptotic behavior of the conditional law of
the community structure $\sigma^{*}$ given the observation. We use the bracket
notation $\langle\cdot\rangle$ to denote the expectation with respect to the
conditional law of $\sigma^{*}$ given the observation, with $\sigma$ being the
canonical random variable under $\langle\cdot\rangle$. Explicitly, this means
that for every function $h:\Sigma_{N}\to\mathbb{R}$, we have

$\langle h(\sigma)\rangle:=\frac{\int_{\Sigma_{N}}h(\sigma)\exp
H_{N}^{t,\mu}(\sigma)\,\,\mathrm{d}P_{N}(\sigma)}{\int_{\Sigma_{N}}\exp
H_{N}^{t,\mu}(\sigma)\,\,\mathrm{d}P_{N}(\sigma)}.$ (1.29)

We stress that the expectation $\langle\cdot\rangle$ is itself random, since
it depends on the observation. We denote by $(\sigma^{\ell})_{\ell\geq
1}=((\sigma^{\ell}_{i})_{i\leq N})_{\ell\geq 1}$ independent copies, often
called replicas, of the random variable $\sigma$ under $\langle\cdot\rangle$.
We also introduce the set $\mathrm{Reg}(\mathcal{M}_{+})$ of measures in
$\mathcal{M}_{+}$ that admit a smooth and strictly positive density with
respect to the Lebesgue measure on $[-1,1]$,

$\mathrm{Reg}(\mathcal{M}_{+}):=\big{\\{}\mu\in\mathcal{M}_{+}\mid\mu\text{
admits a smooth and strictly positive density on }[-1,1]\big{\\}}.$ (1.30)

Our next result states that for most choices of $(t,\mu)\in\mathbb{R}_{\geq
0}\times\mathcal{M}_{+}$, we can identify a fixed point of the map
$\nu\mapsto\Gamma_{t,\mu}(\nu)$ for which the equality (1.28) holds, and for
which we can also identify the limit law of $(\sigma^{\ell})_{\ell\geq 1}$
after a small perturbation of the inference problem. More precisely, we will
show that this can be done at any point $(t,\mu)\in\mathbb{R}_{\geq
0}\times\mathrm{Reg}(\mathcal{M}_{+})$ at which the limit free energy is
Gateaux differentiable; we refer to Section 2 for a definition of Gateaux
differentiability and a precise discussion of why the limit free energy is
Gateaux differentiable at most points. The convergence in law of
$(\sigma^{\ell}_{i})_{i\leq N,\ell\geq 1}$ is in the sense of finite-
dimensional distributions.

######  Theorem 1.2.

Suppose that the sequence of enriched free energies $(\overline{F}_{N})_{N\geq
1}$ converges pointwise to some limit $f:\mathbb{R}_{\geq
0}\times\mathcal{M}_{+}\to\mathbb{R}$. If $(t,\mu)\in\mathbb{R}_{\geq
0}\times\mathrm{Reg}(\mathcal{M}_{+})$ is such that $f(t,\cdot)$ is Gateaux
differentiable at $\mu$, then we can identify a probability measure
$\nu^{*}\in\mathcal{M}_{p}$ such that the Gateaux derivative density of $f$ at
$(t,\mu)$ is $G_{\nu^{*}}$. Moreover, the probability measure $\nu^{*}$ is a
fixed point of the map $\Gamma_{t,\mu}$, and we have

$f(t,\mu)=\operatorname{\mathcal{P}}_{t,\mu}(\nu^{*}).$ (1.31)

Finally, there exists a sequence
$\smash{\big{(}N_{k},\lambda^{N_{k}}\big{)}_{k\geq 1}}$ such that
$\smash{(N_{k})_{k\geq 1}}$ increases to infinity, the spin array
$(\sigma^{\ell}_{i})_{i\leq N_{k},\ell\geq 1}$ converges in law under
$\operatorname{\mathbb{E}}\langle\cdot\rangle$ with the modified Hamiltonian
(4.6) with perturbation parameters $\smash{(\lambda^{N_{k}})_{k\geq 1}}$, and
the limit law is generated by $\nu^{*}$ according to (S1)-(S2). The
perturbation parameters $\lambda^{N_{k}}$ introduced in the modified
Hamiltonian (4.6) are sufficiently small that the difference between the free
energy (1.22) and the free energy with the perturbed Hamiltonian (4.6) tends
to zero as $N$ tends to infinity.

One limitation of Theorems 1.1 and 1.2 is that they assume the existence of
the limit free energy, which we do not know a priori. To a large extent, this
is a side effect of the fact that we do not know how to single out the correct
critical point in general, as was discussed in the previous subsection.
Correspondingly, this assumption can be removed whenever we can guarantee the
uniqueness of a fixed point to $\Gamma_{t,\mu}$. As shown in Lemma 5.3, the
second part of Theorem 1.2 in fact does not require that we assume that the
free energy converges. Indeed, to show this result we only need to ensure that
the free energy converges along a subsequence, and this can always be obtained
using the Arzelà-Ascoli theorem.

The next proposition certifies the uniqueness of a fixed point in a regime of
small signal-to-noise ratio.

######  Proposition 1.3.

There exists $C<+\infty$ such that for all $t<C^{-1}$ and
$\mu\in\mathcal{M}_{+}$, the map $\nu\mapsto\Gamma_{t,\mu}(\nu)$ admits a
unique fixed point $\nu^{*}$ in $\mathcal{M}_{p}$. Moreover, the sequence of
enriched free energies $\smash{(\overline{F}_{N}(t,\mu))_{N\geq 1}}$ converges
to $\operatorname{\mathcal{P}}_{t,\mu}(\nu^{*})$,

$\lim_{N\to+\infty}\overline{F}_{N}(t,\mu)=\operatorname{\mathcal{P}}_{t,\mu}(\nu^{*}).$
(1.32)

For completeness, we also explain how to recover the variational formula
already obtained in [2, 15] (for $p=1/2$) and in [18] for the limit free
energy in the disassortative case.

######  Proposition 1.4.

In the disassortative case, that is when $\Delta\leq 0$, we have for every
$(t,\mu)\in\mathbb{R}_{\geq 0}\times\mathcal{M}_{+}$ that

$\lim_{N\to+\infty}\overline{F}_{N}(t,\mu)=\sup_{\nu\in\mathcal{M}_{p}}\operatorname{\mathcal{P}}_{t,\mu}(\nu).$
(1.33)

As explained in the previous subsection, we will also show the invalidity of a
candidate variational formula for the limit free energy. Since this involves a
variant of the model with an additional bipartite structure, we prefer to
postpone a precise description of the setting and result to Section 7.

###  1.3 Organization of the paper

In Section 2, we recall the expressions for the derivatives of the free energy
obtained in [18]. We then leverage these to establish the uniform Lipschitz
continuity of the free energy, the almost everywhere differentiability of the
limit free energy, and the asymptotic behavior of the derivative of the free
energy. Section 3 is a brief overview of the main multioverlap concentration
result in [8] which implies that the spin array sampled from the asymptotic
Gibbs measure is generated according to (S1)-(S2) for some measure $\nu^{*}$
in $\mathcal{M}_{p}$. In Section 4, the cavity computations leading to the
functional $\operatorname{\mathcal{P}}_{t,\mu}$ in (1.26) are performed. In
Section 5, the cavity representation is combined with the multioverlap
concentration result in Section 3 and the regularity properties of the free
energy obtained in Section 2 to prove Theorems 1.1 and 1.2. Section 6 is
devoted to the proofs of Propositions 1.3 and 1.4. Finally, in Section 7, it
is argued that, unlike in the disassortative setting, one should not in
general expect the limit free energy to be given by evaluating the functional
$\operatorname{\mathcal{P}}_{t,\mu}$ at the measure $\nu\in\mathcal{M}_{p}$
which maximizes its value.

######  Acknowledgements.

We would like to warmly thank Dmitry Panchenko and Jean Barbier for sharing
their notes [7] on the free energy in the disassortative sparse stochastic
block model with us, which helped us with many of the computations in Section
4.

##  2 Regularity properties of the free energy

In this section we discuss three regularity properties of the enriched free
energy (1.22). In Section 2.1, we establish a Lipschitz continuity bound for
the enriched free energy which implies in particular that the free energy
restricted to the space of probability measures is Lipschitz continuous with
respect to the Wasserstein distance. In Section 2.2, we prove that a Lipschitz
continuous function on the space of cumulative distribution functions on
$[-1,1]$, viewed as a subset of $L^{2}[-1,1]$, is Gateaux differentiable on a
dense subset of its domain. Finally, in Section 2.3, we leverage a convexity
property of the free energy to prove that the sequence of derivatives of the
free energy converges to the derivative of the limit free energy at any point
of differentiability of said limit. We now state the three main results of
this section. Since the ideas developed in these proofs will not reappear
later, the reader may consider skipping these proofs on first reading.

The Lipschitz continuity result in Section 2.1 involves the Wasserstein
distance. The _Wasserstein distance_ between two probability measures
$\mu,\nu\in\Pr[-1,1]$ is defined by

$\displaystyle W(\mu,\nu)$
$\displaystyle:=\sup\bigg{\\{}\Big{\lvert}\int_{-1}^{1}h(x)\,\mathrm{d}\mu(x)-\int_{-1}^{1}h(x)\,\mathrm{d}\nu(x)\Big{\rvert}\mid\lVert
h\rVert_{\text{Lip}}\leq 1\bigg{\\}},$ (2.1)

where $\lVert\cdot\rVert_{\mathrm{Lip}}$ denotes the Lipschitz semi-norm

$\lVert h\rVert_{\mathrm{Lip}}:=\sup_{x\neq x^{\prime}\in[-1,1]}\frac{\lvert
h(x)-h(x^{\prime})\rvert}{\lvert x-x^{\prime}\rvert}$ (2.2)

on the space of Lipschitz continuous functions $h:[-1,1]\to\mathbb{R}$. The
Kantorovich-Rubinstein theorem [48, Theorem 4.15] ensures that the Wasserstein
distance admits the dual representation

$W(\mu,\nu)=\inf\big{\\{}\operatorname{\mathbb{E}}\lvert X-Y\rvert\mid
X\sim\mu\text{ and }Y\sim\nu\big{\\}}.$ (2.3)

In the one-dimensional setting, as shown in [41, Theorem 1.5.1 and Corollary
1.5.3], the Wasserstein distance admits an explicit representation in terms of
the cumulative distribution function or the quantile function. Indeed, if
$F_{\mu}:[-1,1]\to[0,1]$ and $F_{\mu}^{-1}:[0,1]\to[-1,1]$ denote the
cumulative distribution function and the quantile transform of a probability
measure $\mu\in\Pr[-1,1]$,

$F_{\mu}(x):=\mu[-1,x]\quad\text{and}\quad
F_{\mu}^{-1}(u):=\inf\\{x\in\mathbb{R}\mid F_{\mu}(x)\geq u\\},$ (2.4)

then for all probability measures $\mu,\nu\in\Pr[-1,1]$,

$W(\mu,\nu)=\int_{0}^{1}\big{\lvert}F_{\mu}^{-1}(u)-F_{\nu}^{-1}(u)\big{\rvert}\,\mathrm{d}u=\int_{-1}^{1}\big{\lvert}F_{\mu}(x)-F_{\nu}(x)\big{\rvert}\,\mathrm{d}x.$
(2.5)

In this notation, the main Lipschitz continuity result for the enriched free
energy reads as follows.

######  Proposition 2.1.

There exists $C<+\infty$ such that for any
$(t,\mu),(t,\mu^{\prime})\in\mathbb{R}_{\geq 0}\times\mathcal{M}_{+}$,

$\big{\lvert}\overline{F}_{N}(t,\mu)-\overline{F}_{N}(t^{\prime},\mu^{\prime})\big{\rvert}\leq
C\big{(}\lvert
t-t^{\prime}\rvert+\big{\lvert}\mu[-1,1]-\mu^{\prime}[-1,1]\big{\rvert}+\mu[-1,1]\lvert
F_{\overline{\mu}}-F_{\overline{\mu}^{\prime}}\rvert_{L^{2}}\big{)}.$ (2.6)

In Sections 2.2-2.3 and throughout the paper, the notion of Gateaux
differentiability that we take may differ from the most classical one, so we
proceed to define it precisely. As in [11], we fix a topological vector space
$X$ and a subset $U$ of $X$. We denote by $X^{*}$ the continuous dual of $X$,
and we write $\langle\cdot,\cdot\rangle$ for the duality pairing. We will not
necessarily take $U$ to be an open set, so, given a point $q\in U$, the set

$\mathrm{Adm}(U,q):=\big{\\{}x\in X\mid\text{ there exists }r>0\text{ such
that for all }t\in[0,r]\text{ we have }q+tx\in U\big{\\}}$ (2.7)

of admissible directions along which a small line segment starting at $q$ is
contained in $U$ will play an important part in the definition. Indeed, we say
that a function $h:U\to\mathbb{R}$ is _Gateaux differentiable_ at $q\in U$ if
the following two conditions hold.

  1. GD1

For every $x\in\mathrm{Adm}(U,q)$, the limit

$Dh(q;x):=\lim_{\epsilon\searrow 0}\frac{h(q+\epsilon x)-h(q)}{\epsilon}$
(2.8)

exists.

  2. GD2

There is a unique $y\in X^{*}$ such that, for every $x\in\mathrm{Adm}(U,q)$,
we have $Dh(q;x)=\langle y,x\rangle$.

In this case, the dual vector $y$ is called the _Gateaux derivative_ of $h$ at
$q$.

We will mainly be concerned with the case $X=\mathbb{R}_{\geq
0}\times\mathcal{M}_{+}$ endowed with the product of the Euclidean topology
and the topology of weak convergence, which is induced by the Wasserstein
distance (2.1). In this setting, given a function $f:\mathbb{R}_{\geq
0}\times\mathcal{M}_{+}\to\mathbb{R}$, a time $t\geq 0$, and measures
$\mu\in\mathcal{M}_{+}$ and $\nu\in\mathrm{Adm}(\mathcal{M}_{+},\mu)$, we
denote by $D_{\mu}f(t,\mu;\nu)$ the Gateaux derivative of the function
$f(t,\cdot)$ at the measure $\mu$ in the direction $\nu$,

$D_{\mu}f(t,\mu;\nu):=\lim_{\epsilon\to
0}\frac{f(t,\mu+\epsilon\nu)-f(t,\mu)}{\epsilon}.$ (2.9)

We say that the Gateaux derivative of $f(t,\cdot)$ admits a density at the
measure $\mu\in\mathcal{M}_{+}$ if there exists a continuous function
$x\mapsto D_{\mu}f(t,\mu,x)$ defined on the interval $[-1,1]$ such that for
all $\nu\in\mathrm{Adm}(\mathcal{M}_{+},\mu)$,

$D_{\mu}f(t,\mu;\nu)=\int_{-1}^{1}D_{\mu}f(t,\mu,x)\,\mathrm{d}\nu(x).$ (2.10)

We often abuse notation and identify the density $D_{\mu}f(t,\mu,\cdot)$ with
the Gateaux derivative $D_{\mu}f(t,\mu)$. Implicitly identifying the
continuous dual of $\mathcal{M}_{+}$ with the space of continuous functions on
$[-1,1]$, this definition of Gateaux differentiability coincides with that in
(2.8)-(GD2) but differs slightly from that in [18] where the density is only
required to be measurable and bounded as opposed to continuous. However, all
Gateaux derivative densities appearing in [18] are of the finite-volume free
energy and are continuous, so this difference is insignificant.

For technical reasons, it will be convenient for the dense subset of
$\mathbb{R}_{\geq 0}\times\mathcal{M}_{+}$ on which we establish the Gateaux
differentiability of the free energy to be composed of well-behaved measures.
Recall the definition in (1.30) of the set $\mathrm{Reg}(\mathcal{M}_{+})$ of
measures in $\mathcal{M}_{+}$ that admit a smooth and strictly positive
density with respect to Lebesgue measure on $[-1,1]$. A key property of this
set is that for any $\mu,\nu\in\mathrm{Reg}(\mathcal{M}_{+})$ and
$t\in\mathbb{R}$ with $\lvert t\rvert$ small enough, we have
$\mu+t\nu\in\mathcal{M}_{+}$. In particular, the set of admissible directions
at $\mu\in\mathrm{Reg}(\mathcal{M}_{+})$ contains
$\mathrm{Reg}(\mathcal{M}_{+})$. In this notation, the main Gateaux
differentiability results for the enriched free energy proved in Sections 2.2
and 2.3 read as follows.

######  Proposition 2.2.

Suppose that the sequence of enriched free energies $(\overline{F}_{N})_{N\geq
1}$ converges pointwise to some limit $f:\mathbb{R}_{\geq
0}\times\mathcal{M}_{+}\to\mathbb{R}$ along a subsequence $(N_{k})_{k\geq 1}$.
The subsequential limit $f$ is Gateaux differentiable jointly in its two
variables on a subset of $\mathbb{R}_{>0}\times\mathrm{Reg}(\mathcal{M}_{+})$
that is dense in $\mathbb{R}_{\geq 0}\times\mathcal{M}_{+}$.

######  Proposition 2.3.

Suppose that the sequence of enriched free energies $(\overline{F}_{N})_{N\geq
1}$ converges pointwise to some limit $f:\mathbb{R}_{\geq
0}\times\mathcal{M}_{+}\to\mathbb{R}$ along a subsequence $(N_{k})_{k\geq 1}$.
For each $t\geq 0$, if $f(t,\cdot)$ is Gateaux differentiable at
$\mu\in\mathrm{Reg}(\mathcal{M}_{+})$, then
$D_{\mu}\overline{F}_{N_{k}}(t,\mu,\cdot)$ converges weakly to
$D_{\mu}f(t,\mu,\cdot)$.

In proving Propositions 2.1 and 2.2, it will be convenient to remember from
[18, Lemmas 2.1 and 2.3] that the enriched free energy (1.22) is Gateaux
differentiable at every $(t,\mu)\in\mathbb{R}_{\geq 0}\times\mathcal{M}_{+}$
with

$\displaystyle\partial_{t}\overline{F}_{N}(t,\mu)$
$\displaystyle=\frac{1}{2}\operatorname{\mathbb{E}}\big{(}c+\Delta\langle\sigma_{1}\sigma_{2}\rangle\big{)}\log\big{(}c+\Delta\langle\sigma_{1}\sigma_{2}\rangle\big{)}-\frac{\Delta\overline{m}^{2}}{2}-\frac{c}{2}+\operatorname{\operatorname{\mathcal{O}}}(N^{-1}),$
(2.11) $\displaystyle D_{\mu}\overline{F}_{N}(t,\mu,x)$
$\displaystyle=\operatorname{\mathbb{E}}\big{(}c+\Delta\langle\sigma_{1}\rangle
x\big{)}\log\big{(}c+\Delta\langle\sigma_{1}\rangle
x\big{)}-c-\Delta\overline{m}x+\operatorname{\operatorname{\mathcal{O}}}(N^{-1}).$
(2.12)

We have used $\langle\cdot\rangle$ to denote the Gibbs average associated with
the enriched Hamiltonian (1.21). This means that for any bounded and
measurable function $f=f(\sigma^{1},\ldots,\sigma^{n})$ of finitely many
replicas,

$\langle f(\sigma^{1},\ldots,\sigma^{n})\rangle:=\langle
f\rangle:=\frac{\int_{\Sigma_{N}^{n}}f(\sigma^{1},\ldots,\sigma^{n})\prod_{\ell\leq
n}\exp
H_{N}^{t,\mu}(\sigma^{\ell})\,\mathrm{d}P_{N}^{*}(\sigma^{\ell})}{\big{(}\int_{\Sigma_{N}}\exp
H_{N}^{t,\mu}(\sigma)\,\mathrm{d}P_{N}^{*}(\sigma)\big{)}^{n}}.$ (2.13)

The computations leading to (2.11)-(2.12) are considerably simplified by the
_Nishimori identity_. This identity allows us to freely interchange one
replica $\sigma^{\ell}$ by the signal $\sigma^{*}$ when taking an average with
respect to all sources of randomness, thus avoiding a cascade of new replicas
as we differentiate the free energy. More precisely, it states that, for every
bounded and measurable function
$f=f(\sigma^{1},\ldots,\sigma^{n},\mathcal{D}_{N}^{t,\mu})$ of finitely many
replicas and the data,

$\operatorname{\mathbb{E}}\big{\langle}f\big{(}\sigma^{1},\sigma^{2},\ldots,\sigma^{n},\mathcal{D}_{N}^{t,\mu}\big{)}\big{\rangle}=\operatorname{\mathbb{E}}\big{\langle}f\big{(}\sigma^{*},\sigma^{2},\ldots,\sigma^{n},\mathcal{D}_{N}^{t,\mu}\big{)}\big{\rangle}.$
(2.14)

This can be first verified for functions of product form using that the Gibbs
average is the conditional law of the signal $\sigma^{*}$ given the data
$\mathcal{D}_{N}^{t,\mu}$, and then extended to all bounded and measurable
functions by a monotone class argument as in [19, Proposition 4.1].

The proof of Proposition 2.3 will rely on a convexity property of the free
energy that will be obtained through information-theoretic arguments taken
from [25].

###  2.1 Uniform Lipschitz continuity of the free energy

The Lipschitz continuity of the free energy stated in Proposition 2.1 will be
deduced from the Lipschitz continuity of an extension of the free energy.
Given a probability measure $\mu\in\Pr[-1,1]$, consider a sequence
$x=(x_{i,k})_{i,k\geq 1}$ of i.i.d. random variables with law $\mu$. For each
$s>0$ and $i\geq 1$, let $\Pi_{i,s}\sim\operatorname{\operatorname{Poi}}(sN)$
be independent over $i\geq 1$, and introduce the Hamiltonian on $\Sigma_{N}$
defined by

$\widetilde{H}_{N}^{s,\mu}(\sigma):=\sum_{i\leq
N}\sum_{k\leq\Pi_{i,s}}\log\bigg{[}\big{(}c+\Delta\sigma_{i}x_{i,k}\big{)}^{\widetilde{G}_{i,k}^{x}}\Big{(}1-\frac{c+\Delta\sigma_{i}x_{i,k}}{N}\Big{)}^{1-\widetilde{G}_{i,k}^{x}}\bigg{]},$
(2.15)

where the random variables $(\widetilde{G}_{i,k}^{x})_{i,k\geq 1}$ are
independent with conditional distribution

$\mathbb{P}\big{\\{}\widetilde{G}_{i,k}^{x}=1\mid\sigma^{*},x\big{\\}}:=\frac{c+\Delta\sigma_{i}^{*}x_{i,k}}{N}.$
(2.16)

Recalling the definition of the time-dependent Hamiltonian in (1.15), we
define the extended free energy functional

$\widetilde{F}_{N}(t,s,\mu):=\frac{1}{N}\operatorname{\mathbb{E}}\log\int_{\Sigma_{N}}\exp\big{(}H_{N}^{t}(\sigma)+\widetilde{H}_{N}^{s,\mu}(\sigma)\big{)}\,\mathrm{d}P_{N}^{*}(\sigma).$
(2.17)

For any $\mu\in\mathcal{M}_{+}$, we have
$\overline{F}_{N}(t,\mu)=\widetilde{F}_{N}\big{(}t,\mu[-1,1],\overline{\mu}\big{)}$.
Indeed, this is how the enriched free energy (1.22) was defined in [18]. The
Lipschitz continuity of the extended free energy (2.17) will be obtained by
combining the mean value theorem with the derivative expressions
(2.11)-(2.12). It will also be convenient to remember from [18, Lemma 2.3]
that

$\partial_{s}\widetilde{F}_{N}(t,s,\mu)=\operatorname{\mathbb{E}}\big{(}c+\Delta\langle\sigma_{1}\rangle
x_{1}\big{)}\log\big{(}c+\Delta\langle\sigma_{1}\rangle
x_{1}\big{)}-c-\Delta\overline{m}\operatorname{\mathbb{E}}x_{1}+\operatorname{\operatorname{\mathcal{O}}}(N^{-1}),$
(2.18)

where the random variable $x_{1}$ has law $\mu$.

######  Lemma 2.4.

There exists $C<+\infty$ such that for any
$(t,s,\mu),(t^{\prime},s^{\prime},\nu)\in\mathbb{R}_{\geq
0}\times\mathbb{R}_{\geq 0}\times\Pr[-1,1]$,

$\lvert\widetilde{F}_{N}(t,s,\mu)-\widetilde{F}_{N}(t^{\prime},s^{\prime},\nu)\rvert\leq
C\big{(}\lvert t-t^{\prime}\rvert+\lvert
s-s^{\prime}\rvert+s^{\prime}W(\mu,\nu)\big{)}.$ (2.19)

###### Proof.

The derivative expressions (2.11) and (2.18) reveal that
$\partial_{t}\widetilde{F}_{N}(t,s,\mu)$ and
$\partial_{s}\widetilde{F}_{N}(t,s,\mu)$ are bounded by some constant
independent of the triple $(t,s,\mu)$. By the mean value theorem and the
triangle inequality it therefore suffices to show that there exists
$C<+\infty$ such that for all $t,s\in\mathbb{R}_{>0}$ and
$\mu,\nu\in\Pr[-1,1]$,

$\lvert\widetilde{F}_{N}(t,s,\mu)-\widetilde{F}_{N}(t,s,\nu)\rvert\leq
CsW(\mu,\nu).$ (2.20)

The fundamental theorem of calculus and the definition of the Gateaux
derivative imply that

$\widetilde{F}_{N}(t,s,\mu)-\widetilde{F}_{N}(t,s,\nu)=\overline{F}_{N}(t,s\mu)-\overline{F}_{N}(t,s\nu)=s\int_{0}^{1}D_{\mu}\overline{F}_{N}\big{(}t,s\nu+us(\mu-\nu);\mu-\nu\big{)}\,\mathrm{d}u.$

For each $u\in[0,1]$, let
$f_{t,s,u}(x):=D_{\mu}\overline{F}_{N}\big{(}t,s\nu+us(\mu-\nu);x\big{)}$ in
such a way that

$\big{\lvert}\widetilde{F}_{N}(t,s,\mu)-\widetilde{F}_{N}(t,s,\nu)\big{\rvert}\leq
s\int_{0}^{1}\Big{\lvert}\int_{-1}^{1}f_{t,s,u}(x)\,\mathrm{d}(\mu-\nu)(x)\Big{\rvert}\,\mathrm{d}u.$
(2.21)

The mean value theorem and the bound

$\big{\lvert}\partial_{x}D_{\mu}\overline{F}_{N}(t,\mu,x)\big{\rvert}\leq
c\big{(}1+\lvert\log(2c)\rvert+\lvert\log(c-\lvert\Delta\rvert)\rvert\big{)}$
(2.22)

established for any $(t,\mu)\in\mathbb{R}_{\geq 0}\times\mathcal{M}_{+}$ in
[18, Lemma 3.3] imply that

$\lVert f_{t,s,u}\rVert_{\text{Lip}}\leq
c\big{(}1+\lvert\log(2c)\rvert+\lvert\log(c-\lvert\Delta\rvert)\rvert\big{)}.$

Together with the definition of the Wasserstein distance in (2.1) and the
bound (2.21), this establishes (2.20) and completes the proof. ∎

###### Proof of Proposition 2.1.

The Lipschitz bound in Lemma 2.4, the representation (2.5) for the one-
dimensional Wasserstein distance, and the Cauchy-Schwarz inequality give a
constant $C<+\infty$ such that, for all
$(t,s,\mu),(t^{\prime},s^{\prime},\nu)\in\mathbb{R}_{\geq
0}\times\mathbb{R}_{\geq 0}\times\Pr[-1,1]$,

$\lvert\widetilde{F}_{N}(t,s,\mu)-\widetilde{F}_{N}(t^{\prime},s^{\prime},\nu)\rvert\leq
C\big{(}\lvert t-t^{\prime}\rvert+\lvert s-s^{\prime}\rvert+s^{\prime}\lvert
F_{\mu}-F_{\nu}\rvert_{L^{2}}\big{)}.$

Remembering that for any $\mu\in\mathcal{M}_{+}$, we have
$\overline{F}_{N}(t,\mu)=\widetilde{F}_{N}\big{(}t,\mu[-1,1],\overline{\mu}\big{)}$
completes the proof. ∎

###  2.2 Almost everywhere differentiability of the limit free energy

The Gateaux differentiability of the free energy stated in Proposition 2.2
will be deduced from the Gateaux differentiability of the free energy thought
of as a function on the space of paths

$\displaystyle\mathcal{D}[0,1]:=\big{\\{}q:[-1,1]\to[0,1]\mid q$ is right-
continuous $\displaystyle\qquad\text{and non-decreasing with
}q(1)=1\big{\\}}\subset L^{2}[-1,1].$ (2.23)

This space of paths is in one-to-one correspondence with the set of cumulative
distribution functions via the mapping
$\mathcal{F}:\Pr[-1,1]\to\mathcal{D}[0,1]$ defined by
$\mathcal{F}(\mu):=F_{\mu}$. Through this mapping, any function
$\smash{h:\mathbb{R}_{\geq 0}\times\mathcal{M}_{+}\to\mathbb{R}}$ may be
identified with the function $\smash{\widetilde{h}^{\prime}:\mathbb{R}_{\geq
0}\times\mathbb{R}_{\geq 0}\times\mathcal{D}[0,1]\to\mathbb{R}}$ defined by

$\widetilde{h}^{\prime}(t,s,q):=h\big{(}t,s\mathcal{F}^{-1}(q)\big{)}.$ (2.24)

In particular, the free energy (1.22) may be identified with the function
$\widetilde{F}_{N}^{\prime}:\mathbb{R}_{\geq 0}\times\mathbb{R}_{\geq
0}\times\mathcal{D}[0,1]\to\mathbb{R}$ defined by

$\widetilde{F}_{N}^{\prime}(t,s,q):=\overline{F}_{N}\big{(}t,s\mathcal{F}^{-1}(q)\big{)}.$
(2.25)

Notice that the notions of Gateaux differentiability for a function
$\smash{h:\mathbb{R}_{\geq 0}\times\mathcal{M}_{+}\to\mathbb{R}}$ and its
corresponding function $\smash{\widetilde{h}^{\prime}:\mathbb{R}_{\geq
0}\times\mathbb{R}_{\geq 0}\times\mathcal{D}[0,1]\to\mathbb{R}}$ differ. On
the one hand, for each fixed $t\geq 0$, the Gateaux derivative of the function
$\smash{h(t,\cdot)}$ at the measure $\mu\in\mathcal{M}_{+}$ may be identified
with the continuous density $\smash{D_{\mu}h(t,\mu,\cdot)}$ defined according
to (2.10). On the other, for each fixed $t,s\geq 0$, the Gateaux derivative of
the function $\smash{\widetilde{h}^{\prime}(t,s,\cdot)}$ at the path
$q\in\mathcal{D}[0,1]$ may be identified with the square-integrable function
$\smash{\partial_{q}\widetilde{h}^{\prime}(t,s,q,\cdot)\in L^{2}[-1,1]}$
having the property that for all
$q^{\prime}\in\mathrm{Adm}(\mathcal{D}[0,1],q)$,

$\widetilde{h}^{\prime}(t,s,q+\epsilon
q^{\prime})=\widetilde{h}^{\prime}(t,s,q)+\epsilon\int_{-1}^{1}\partial_{q}\widetilde{h}^{\prime}(t,s,q,u)q^{\prime}(u)\,\mathrm{d}u+o(\epsilon).$
(2.26)

However, it will be important to observe that whenever
$\smash{\widetilde{h}^{\prime}}$ is Gateaux differentiable at a point
$\smash{(t,s,q)\in\mathbb{R}_{\geq 0}\times\mathbb{R}_{\geq
0}\times\mathcal{D}[0,1]}$, then $\smash{h}$ must be Gateaux differentiable at
the corresponding point $\smash{(t,s\mathcal{F}^{-1}(q))\in\mathbb{R}_{\geq
0}\times\mathcal{M}_{+}}$.

######  Lemma 2.5.

If the function $\smash{\widetilde{h}^{\prime}:\mathbb{R}_{\geq
0}\times\mathbb{R}_{\geq 0}\times\mathcal{D}[0,1]\to\mathbb{R}}$ is Gateaux
differentiable jointly in its three variables at the point
$\smash{(t,s,q)\in\mathbb{R}_{>0}\times\mathbb{R}_{>0}\times\mathcal{D}[0,1]}$,
then the function $\smash{h:\mathbb{R}_{\geq
0}\times\mathcal{M}_{+}\to\mathbb{R}}$ used to define it according to (2.24)
is Gateaux differentiable jointly in its two variables at the point
$\smash{(t,s\mathcal{F}^{-1}(q))\in\mathbb{R}_{>0}\times\mathcal{M}_{+}}.$

###### Proof.

For simplicity of notation, we will drop the time variable and prove that
whenever the function $\widetilde{h}^{\prime}:\mathbb{R}_{\geq
0}\times\mathcal{D}[0,1]\to\mathbb{R}$ is Gateaux differentiable jointly in
its two variables at the point
$(s,q)\in\mathbb{R}_{>0}\times\mathcal{D}[0,1]$, then the function
$h:\mathcal{M}_{+}\to\mathbb{R}$ used to define it according to (2.24) without
the time variable is Gateaux differentiable at the point
$s\mathcal{F}^{-1}(q)$. Generalizing the proof to incorporate the time
variable requires only minor changes in notation. Let
$\mu:=s\mathcal{F}^{-1}(q)$, and fix $\nu\in\mathrm{Adm}(\mathcal{M}_{+},\mu)$
as well as $\epsilon>0$ small enough so that
$\mu+\epsilon\nu\in\mathcal{M}_{+}$. We define $s^{\prime}:=\nu[-1,1]$, and
observe that by Gateaux differentiability of $\widetilde{h}^{\prime}$ at
$(s,q)\in\mathbb{R}_{>0}\times\mathcal{D}[0,1]$,

$\displaystyle h(\mu+\epsilon\nu)-h(\mu)$
$\displaystyle=\widetilde{h}^{\prime}\bigg{(}s+\epsilon
s^{\prime},q+\epsilon\frac{s^{\prime}F_{\overline{\nu}-\overline{\mu}}}{s+\epsilon
s^{\prime}}\bigg{)}-\widetilde{h}^{\prime}(s,q)$
$\displaystyle=\epsilon\bigg{(}s^{\prime}\partial_{s}\widetilde{h}^{\prime}(s,q)+\frac{s^{\prime}}{s}\int_{-1}^{1}\partial_{q}\widetilde{h}^{\prime}(s,q,u)F_{\overline{\nu}-\overline{\mu}}(u)\,\mathrm{d}u\bigg{)}+o(\epsilon).$

The Fubini-Tonelli theorem implies that the second term between parentheses is
given by

$\displaystyle\frac{1}{s}\int_{-1}^{1}\int_{-1}^{1}\partial_{q}\widetilde{h}^{\prime}(s,q,u)\mathbbm{1}_{[-1,u]}$
$\displaystyle(w)\,\mathrm{d}\nu(w)\,\mathrm{d}u-\frac{1}{s}\int_{-1}^{1}\int_{-1}^{1}\partial_{q}\widetilde{h}^{\prime}(s,q,u)F_{\overline{\mu}}(u)\,\mathrm{d}u\,\mathrm{d}\nu(w)$
$\displaystyle=\frac{1}{s}\int_{-1}^{1}\bigg{(}\int_{w}^{1}\partial_{q}\widetilde{h}^{\prime}(s,q,u)\,\mathrm{d}u-\int_{-1}^{1}\partial_{q}\widetilde{h}^{\prime}(s,q,u)F_{\overline{\mu}}(u)\,\mathrm{d}u\bigg{)}\,\mathrm{d}\nu(w).$

It follows that

$h(\mu+\epsilon\nu)-h(\mu)=\epsilon\int_{-1}^{1}D_{\mu}(w)\,\mathrm{d}\nu(w)+o(\epsilon)$

for the continuous function $D_{\mu}:[-1,1]\to\mathbb{R}$ defined by

$D_{\mu}(w):=\partial_{s}\widetilde{h}^{\prime}(s,q)+\frac{1}{s}\bigg{(}\int_{w}^{1}\partial_{q}\widetilde{h}^{\prime}(s,q,u)\,\mathrm{d}u-\int_{-1}^{1}\partial_{q}\widetilde{h}^{\prime}(s,q,u)F_{\overline{\mu}}(u)\,\mathrm{d}u\bigg{)}.$

We have used that $\partial_{q}\widetilde{h}^{\prime}(s,q,\cdot)\in
L^{2}[-1,1]$ to obtain the continuity of $D_{\mu}$. This establishes the
Gateaux differentiability of $h$ at $\mu\in\mathcal{M}_{+}$ and completes the
proof. ∎

To account for the fact that we would like to establish Gateaux
differentiability on a subset of
$\mathbb{R}_{>0}\times\mathrm{Reg}(\mathcal{M}_{+})$ that is dense in
$\mathbb{R}_{\geq 0}\times\mathcal{M}_{+}$, we also introduce the space of
paths

$\displaystyle\mathcal{R}[0,1]:=\big{\\{}q\in\mathcal{D}[0,1]\mid q$
$\displaystyle\text{ is smooth with strictly positive derivative}\big{\\}}.$
(2.27)

This space of paths is in one-to-one correspondence with the probability
measures in $\mathrm{Reg}(\mathcal{M}_{+})$. To establish the Gateaux
differentiability of a function defined on an infinite-dimensional Banach
space, we will follow the arguments in [11] and rely on the notion of a
Gaussian null set. A Borel subset $B$ of a separable Banach space $X$ is said
to be a _Gaussian null set_ if for every non-degenerate Gaussian measure $\mu$
on $X$, we have $\mu(B)=0$. Recall that a probability measure $\mu$ on $X$ is
said to be a _non-degenerate Gaussian measure_ if for every non-zero $y\in
X^{*}$, the measure $\mu\circ y^{-1}$ is a Gaussian measure with non-zero
variance. Gaussian null sets appear when invoking the following two lemmas.

######  Lemma 2.6.

Let $(w_{n})_{n\geq 1}$ be a sequence in a separable Banach space $X$ that has
dense linear span and satisfies $\lim_{n\to+\infty}\lvert w_{n}\rvert=0$, and
let $K$ be the closed convex hull of $\\{0\\}\cup\\{w_{n}\mid n\geq 1\\}$. For
every $x\in X$, the set $x+K$ is not a Gaussian null set.

###### Proof.

This is [11, Lemma 2.5] which itself is adapted from [49, Lemma 3]. ∎

######  Lemma 2.7.

Let $X$ be a separable Banach space. If a function $h:X\to\mathbb{R}$ is
Lipschitz continuous, then it is Gateaux differentiable outside of a Gaussian
null subset of $X$.

###### Proof.

This is [11, Theorem 2.6] which itself is a special case of [49, Theorem 6]. ∎

######  Lemma 2.8.

If $h:\mathbb{R}_{\geq 0}\times\mathbb{R}_{\geq
0}\times\mathcal{D}[0,1]\to\mathbb{R}$ is locally Lipschitz continuous in its
first two variables and uniformly Lipschitz continuous with respect to the
$L^{2}$-norm in its third variable, then it is Gateaux differentiable jointly
in its three variables (in the sense of (2.26)) on a subset of
$\mathbb{R}_{>0}\times\mathbb{R}_{>0}\times\mathcal{R}[0,1]$ that is dense in
$\mathbb{R}_{\geq 0}\times\mathbb{R}_{\geq 0}\times\mathcal{D}[0,1]$.

###### Proof.

For simplicity of notation, we will instead prove that a function
$h:\mathcal{D}[0,1]\to\mathbb{R}$ that is locally Lipschitz continuous with
respect to the $L^{2}$-norm is Gateaux differentiable on a subset of
$\mathcal{R}[0,1]$ that is dense in $\mathcal{D}[0,1]$. Our proof generalizes
with only modifications in notation to the setting of a function defined on
$\mathbb{R}_{\geq 0}\times\mathbb{R}_{\geq 0}\times\mathcal{D}[0,1]$. Up to
multiplying the function $h$ with a cutoff function that vanishes when the
total mass of the measure becomes larger than some predetermined threshold, we
will also assume without loss of generality that
$h:\mathcal{D}[0,1]\to\mathbb{R}$ is in fact uniformly Lipschitz continuous
with respect to the $L^{2}$-norm. The proof proceeds in three steps. First, we
construct a Lipschitz extension $\overline{h}:L^{2}[-1,1]\to\mathbb{R}$ of
$h$, then we show that there exists a Gaussian null set $\mathcal{N}$ such
that $h$ is Gateaux differentiable on $\mathcal{D}[0,1]\setminus\mathcal{N}$,
and finally we prove that $\mathcal{R}[0,1]\setminus\mathcal{N}$ is dense in
$\mathcal{D}[0,1]$.
_Step 1: constructing the Lipschitz extension._ We denote by
$P:L^{2}[-1,1]\to\mathcal{D}[0,1]$ the orthogonal projection onto the closed
convex set $\mathcal{D}[0,1]$. We refer the reader to [19, Exercise 2.16] and
its solution for a justification that this projection $P$ is well-defined
based on the fact that $\mathcal{D}[0,1]$ is a closed convex subset of
$L^{2}[-1,1]$. It is also shown there using the convexity of
$\mathcal{D}[0,1]$ that, for every $q\in L^{2}[-1,1]$ and
$p\in\mathcal{D}[0,1]$,

$\langle q-P(q),p-P(q)\rangle_{L^{2}}\leq 0.$

It follows that for all $q,q^{\prime}\in L^{2}[-1,1]$,

$\langle q-P(q),P(q^{\prime})-P(q)\rangle_{L^{2}}\leq 0\leq\langle
q^{\prime}-P(q^{\prime}),P(q^{\prime})-P(q)\rangle_{L^{2}}.$

Rearranging and using the Cauchy-Schwarz inequality yields

$\lvert P(q)-P(q^{\prime})\rvert_{L^{2}}^{2}\leq\langle
q^{\prime}-q,P(q^{\prime})-P(q)\rangle_{L^{2}}\leq\lvert
q^{\prime}-q\rvert_{L^{2}}\lvert P(q)-P(q^{\prime})\rvert_{L^{2}}$

from which it follows that

$\lvert P(q)-P(q^{\prime})\rvert_{L^{2}}\leq\lvert
q^{\prime}-q\rvert_{L^{2}}.$

This means that the extension $\overline{h}:L^{2}[-1,1]\to\mathbb{R}$ of $h$
defined by

$\overline{h}(q):=h(P(q))$

is Lipschitz continuous.
_Step 2: Gateaux differentiability outside a Gaussian null set._ By Lemma 2.7,
there exists a Gaussian null set $\mathcal{N}\subset L^{2}[-1,1]$ such that
$\overline{h}$ is Gateaux differentiable on $L^{2}[-1,1]\setminus\mathcal{N}$.
We now fix $q\in\mathcal{D}[0,1]\setminus\mathcal{N}$, and prove that $h$ is
Gateaux differentiable at $q$. Let $y\in(L^{2}[-1,1])^{*}\simeq L^{2}[-1,1]$
denote the Gateaux derivative of $\overline{h}$ at $q$. The limit $Dh(q;x)$
exists for every $x\in\mathrm{Adm}(\mathcal{D}[0,1],q)$, and it is equal to
$\langle y,x\rangle_{L^{2}}$. Let us now assume that there exists
$y^{\prime}\in L^{2}[-1,1]$ with $\langle y^{\prime},x\rangle_{L^{2}}=\langle
y,x\rangle_{L^{2}}$ for all $x\in\mathrm{Adm}(\mathcal{D}[0,1],q)$, and prove
that we must have $y^{\prime}=y$. Observe that
$\mathcal{D}[0,1]-q\subset\mathrm{Adm}(\mathcal{D}[0,1],q)$ by convexity of
$\mathcal{D}[0,1]$. Moreover, the orthogonal complement of
$\mathcal{D}[0,1]-q$ in $L^{2}[-1,1]$ is trivial. Indeed, suppose that $z\in
L^{2}[-1,1]$ is such that $\langle z,q^{\prime}\rangle_{L^{2}}=\langle
z,q\rangle_{L^{2}}$ for all $q^{\prime}\in\mathcal{D}[0,1]$. If
$\mu^{\prime}\in\Pr[-1,1]$ is such that $q^{\prime}=F_{\mu^{\prime}}$, then
the Fubini-Tonelli theorem implies that

$\langle
z,q^{\prime}\rangle_{L^{2}}=\int_{-1}^{1}z(u)\mu^{\prime}[-1,u]\,\mathrm{d}u=\int_{-1}^{1}\int_{-1}^{u}z(u)\,\mathrm{d}\mu^{\prime}(v)\,\mathrm{d}u=\int_{-1}^{1}\int_{v}^{1}z(u)\,\mathrm{d}u\,\mathrm{d}\mu^{\prime}(v)=0.$

Since $\mathcal{D}[0,1]$ is in one-to-one correspondence with $\Pr[-1,1]$,
this means that for all $\mu^{\prime}\in\Pr[-1,1]$,

$\int_{-1}^{1}\int_{v}^{1}z(u)\,\mathrm{d}u\,\mathrm{d}\mu^{\prime}(v)=\langle
z,q\rangle_{L^{2}}.$

Choosing $\mu^{\prime}=\delta_{v}$ for each $v\in[-1,1]$, and differentiating
the resulting expression with respect to $v$ shows that $z=0$. This means that
the closed linear span of $\mathcal{D}[0,1]-q$ is $L^{2}[-1,1]$ and that
$\langle y^{\prime},x\rangle_{L^{2}}=\langle y,x\rangle_{L^{2}}$ for all $x\in
L^{2}[-1,1]$. It follows that $y^{\prime}=y$, and therefore that $h$ is
Gateaux differentiable on $\mathcal{D}[0,1]\setminus\mathcal{N}$.
_Step 3: density of $\mathcal{R}[0,1]\setminus\mathcal{N}$ in
$\mathcal{D}[0,1]$._ The set $\mathcal{R}[0,1]$ has dense linear span in
$L^{2}[-1,1]$, so the separability of $L^{2}[-1,1]$ allows us to find a
sequence $(w_{n})_{n\geq 1}\subset\mathcal{R}[0,1]$ whose linear span is dense
in $L^{2}[-1,1]$. Re-scaling each $w_{n}$, we can assume without loss of
generality that $\lvert w_{n}\rvert_{L^{2}}\leq 1$ and that
$\lim_{n\to+\infty}\lvert w_{n}\rvert_{L^{2}}=0$. Let $K$ be as in Lemma 2.6,
and observe that $K\subset\mathcal{R}[0,1]$ by convexity of
$\mathcal{R}[0,1]$. At this point, fix $q\in\mathcal{R}[0,1]$ as well as
$\epsilon>0$. Since $q+\epsilon K$ is not a Gaussian null set, we have that
$q+\epsilon K\not\subset\mathcal{N}$. In particular, there exists $y\in
q+\epsilon K\setminus\mathcal{N}$. Notice that for any $\kappa\in K$, we have
$(q+\epsilon\kappa)^{\prime}>0$ so in fact
$y\in\mathcal{R}\setminus\mathcal{N}$ with $\lvert
y-q\rvert_{L^{2}}\leq\epsilon$. This establishes the density of
$\mathcal{R}[0,1]\setminus\mathcal{N}$ in $\mathcal{R}[0,1]$. Together with
the density of $\mathcal{R}[0,1]$ in $\mathcal{D}[0,1]$ which can be obtained
through mollifiers, this completes the proof. ∎

###### Proof of Proposition 2.2.

We denote by $\widetilde{f}^{\prime}:\mathbb{R}_{\geq 0}\times\mathbb{R}_{\geq
0}\times\mathcal{D}[0,1]\to\mathbb{R}$ the limit of the sequence of extended
free energies $\smash{(\widetilde{F}_{N}^{\prime})_{N\geq 1}}$ defined in
(2.25) along the subsequence $\smash{(N_{k})_{k\geq 0}}$. Invoking Lemma 2.8,
we can find a dense subset $\mathcal{A}$ of
$\smash{\mathbb{R}_{>0}\times\mathbb{R}_{>0}\times\mathcal{R}[0,1]}$ on which
$\smash{\widetilde{f}^{\prime}}$ is Gateaux differentiable jointly in its
three variables. By Lemma 2.5, the limit free energy $f:\mathbb{R}_{\geq
0}\times\mathcal{M}_{+}\to\mathbb{R}$ is Gateaux differentiable jointly in its
two variables on the set

$\mathcal{B}:=\big{\\{}\big{(}t,s\mathcal{F}^{-1}(q)\big{)}\mid(t,s,q)\in\mathcal{A}\big{\\}}\subset\mathbb{R}_{\geq
0}\times\mathrm{Reg}(\mathcal{M}_{+}).$ (2.28)

We now show that this set is dense in $\mathbb{R}_{\geq
0}\times\mathcal{M}_{+}$. We fix $(t,\mu)\in\mathbb{R}_{\geq
0}\times\mathcal{M}_{+}$, and aim to show that there is a sequence
$(t_{n},\mu_{n})_{n\geq 1}\subset\mathcal{B}$ such that $t_{n}\to t$ and
$\mu_{n}\to\mu$ weakly. Let $s:=\mu[-1,1]$, and consider the point
$(t,s,F_{\overline{\mu}})\in\mathbb{R}_{\geq 0}\times\mathbb{R}_{\geq
0}\times\mathcal{D}[0,1]$. There exists a sequence $(t_{n},s_{n},q_{n})_{n\geq
1}\subset\mathcal{A}$ converging to $(t,s,F_{\overline{\mu}})$, and we define
$\mu_{n}:=s_{n}\mathcal{F}^{-1}(q_{n})$. By compactness, the sequence
$\mu_{n}$ admits a subsequential limit. Since $F_{\mu_{n}}=s_{n}q_{n}\to
sF_{\overline{\mu}}=F_{\mu}$ in $L^{2}$, the only possible subsequential limit
is $\mu$ so indeed $\mu_{n}\to\mu$ weakly. ∎

###  2.3 Convergence of the derivative free energy

The convergence of the derivative free energy stated in Proposition 2.3 will
be deduced from a convexity property of the free energy. Although the enriched
free energy (1.22) is known not to be convex [25], we now use the information-
theoretic arguments in [25, Proposition 4.1] to show that it is convex in
certain directions. Together with a computation similar to that in [11,
Proposition 5.4] we then establish Proposition 2.3.

######  Proposition 2.9.

For all $(t,\mu)\in\mathbb{R}_{\geq 0}\times\mathcal{M}_{+}$ and
$\nu\in\Pr[-1,1]$, the map $r\mapsto\overline{F}_{N}(t,\mu+r\nu)$ is convex.

###### Proof.

The superposition principle for Poisson point processes implies that, jointly
in $\sigma\in\Sigma_{N}$,

$H_{N}^{\mu+r\nu}(\sigma)\stackrel{{\scriptstyle
d}}{{=}}H_{N}^{\mu}(\sigma)+H_{N}^{r\nu}(\sigma),$

where we understand that the Hamiltonians $H_{N}^{\mu}(\sigma)$ and
$H_{N}^{r\nu}(\sigma)$ are independent conditionally on the ground truth
$\sigma^{*}$. Conditionally on the ground truth $\sigma^{*}$, the Poisson
point processes $(\Lambda_{i}(N\mu))_{i\leq N}$, and the Poisson random
variable $\Pi_{t}$, we define the “prior” measure on $\Sigma_{N}$ by

$P_{t,\mu}^{*}(\sigma):=\exp\big{(}H_{N}^{t}(\sigma)+H_{N}^{\mu}(\sigma)\big{)}P_{N}^{*}(\sigma)$

in such a way that

$\overline{F}_{N}(t,\mu+r\nu)=\frac{1}{N}\operatorname{\mathbb{E}}\log\int_{\Sigma_{N}}\exp
H_{N}^{r\nu}(\sigma)\,\mathrm{d}P_{t,\mu}^{*}(\sigma).$

The measure $P_{t,\mu}^{*}$ is not necessarily a probability measure, but this
will not have any significant effect. The proof now proceeds in two steps.
First, we relate the free energy $\overline{F}_{N}(t,\mu+r\nu)$ to a mutual
information $I_{N}(r)$, and argue that $r\mapsto\overline{F}_{N}(t,\mu+r\nu)$
is convex in $r$ if and only if this mutual information is concave. Then, we
compute the second derivative of the mutual information, and we show that it
is non-positive.
_Step 1: introducing a mutual information._ Conditionally on the ground truth
$\sigma^{*}$, the Poisson point processes $(\Lambda_{i}(N\mu))_{i\leq N}$, and
the Poisson random variable $\Pi_{t}$, we sample a signal $\sigma^{\diamond}$
from the normalized prior distribution $\overline{P}_{t,\mu}^{*}$ as well as a
family $\Lambda^{r\nu}:=(\Lambda_{i}(Nr\nu))_{i\leq N}$ of Poisson point
processes. Conditionally on the family of Poisson point processes
$\Lambda^{r\nu}$, we sample the Bernoulli random variables
$\widetilde{\mathbf{G}}_{N}:=\\{\widetilde{G}_{i}^{x}\mid
x\in\Lambda_{i}(Nr\nu),i\leq N\\}$ with conditional law

$\mathbb{P}\big{\\{}\widetilde{G}_{i}^{x}=1\mid\sigma^{\diamond},x\big{\\}}:=\frac{c+\Delta\sigma^{\diamond}_{i}x}{N}.$

The likelihood of the model is given by

$\mathbb{P}\big{\\{}\widetilde{\mathbf{G}}_{N}=(\widetilde{G}_{i}^{x})\mid\sigma^{\diamond}=\sigma\big{\\}}=\prod_{i<N}\prod_{x\in\Lambda_{i}(Nr\nu)}\Big{(}\frac{c+\Delta\sigma_{i}x}{N}\Big{)}^{\widetilde{G}_{i}^{x}}\Big{(}1-\frac{c+\Delta\sigma_{i}x}{N}\Big{)}^{1-\widetilde{G}_{i}^{x}},$

and Bayes’ formula implies that the posterior of the model is the Gibbs
measure

$\mathbb{P}\big{\\{}\sigma^{\diamond}=\sigma\mid\widetilde{\mathbf{G}}_{N}=(\widetilde{G}_{i}^{x})\big{\\}}=\frac{\exp\big{(}H_{N}^{\diamond}(\sigma)\big{)}P_{t,\mu}^{*}(\sigma)}{\int_{\Sigma_{N}}\exp\big{(}H_{N}^{\diamond}(\tau)\big{)}\,\mathrm{d}P_{t,\mu}^{*}(\tau)}$

associated with the Hamiltonian

$H_{N}^{\diamond}(\sigma):=\sum_{i\leq
N}\sum_{x\in\Lambda_{i}(Nr\nu)}\log\bigg{[}\big{(}c+\Delta\sigma_{i}x\big{)}^{\widetilde{G}_{i}^{x}}\Big{(}1-\frac{c+\Delta\sigma_{i}x}{N}\Big{)}^{1-\widetilde{G}_{i}^{x}}\bigg{]}.$
(2.29)

It follows that

$I_{N}(r):=\frac{1}{N}I\big{(}\widetilde{\mathbf{G}}_{N};\sigma^{\diamond}\big{)}=\operatorname{\mathbb{E}}\sum_{x\in\Lambda_{1}(Nr\nu)}\log\bigg{[}\big{(}c+\Delta\sigma_{1}^{\diamond}x\big{)}^{\widetilde{G}_{1}^{x}}\Big{(}1-\frac{c+\Delta\sigma_{1}^{\diamond}x}{N}\Big{)}^{1-\widetilde{G}_{1}^{x}}\bigg{]}-\overline{F}_{N}^{\diamond}(r)$
(2.30)

for the free energy defined on $\Sigma_{N}$ by

$\overline{F}_{N}^{\diamond}(r):=\frac{1}{N}\operatorname{\mathbb{E}}\log\int_{\Sigma_{N}}\exp
H_{N}^{\diamond}(\sigma)\,\mathrm{d}P_{t,\mu}^{*}(\sigma).$

Together with [19, Proposition 5.12] on Poisson averages, this implies that

$I_{N}(r)=Nr\operatorname{\mathbb{E}}\int_{-1}^{1}\log\bigg{[}\big{(}c+\Delta\sigma_{1}^{\diamond}x\big{)}^{\widetilde{G}_{1}^{x}}\Big{(}1-\frac{c+\Delta\sigma_{1}^{\diamond}x}{N}\Big{)}^{1-\widetilde{G}_{1}^{x}}\bigg{]}\,\mathrm{d}\nu(x)-\overline{F}_{N}^{\diamond}(r),$

so the free energy $\overline{F}_{N}^{\diamond}(r)$ is convex if and only if
the mutual information $I_{N}(r)$ is concave. Averaging the free energy
$\overline{F}_{N}^{\diamond}(r)$ over the randomness of the ground truth
$\sigma^{*}$, the Poisson point processes $(\Lambda_{i}(N\mu))_{i\leq N}$, and
the Poisson random variable $\Pi_{t}$ yields the free energy
$\overline{F}_{N}(t,\mu+r\nu)$. It therefore suffices to prove that $r\mapsto
I_{N}(r)$ is concave.
_Step 2: showing the mutual information is concave._ The properties of Poisson
point processes imply that the Hamiltonian (2.29) may be written as

$H_{N}^{\diamond}(\sigma)=\sum_{i\leq
N}\sum_{k\leq\Pi_{i,r}}\log\bigg{[}\big{(}c+\Delta\sigma_{i}x_{i,k}\big{)}^{\widetilde{G}_{i,k}}\Big{(}1-\frac{c+\Delta\sigma_{i}x_{i,k}}{N}\Big{)}^{1-\widetilde{G}_{i,k}}\bigg{]},$

where the random variables $(x_{i,k})_{i,k\geq 1}$ are i.i.d. with law $\nu$,
the random variables $(\Pi_{i,r})_{i\leq N}$ are independent Poisson random
variables with mean $Nr$, and $(\widetilde{G}_{i,k})_{i,k\geq 1}$ are
Bernoulli random variables with conditional distribution

$\mathbb{P}\big{\\{}\widetilde{G}_{i,k}=1\mid\sigma^{\diamond},x_{i,k}\big{\\}}:=\frac{c+\Delta\sigma^{\diamond}_{i}x_{i,k}}{N}.$

This means that the mutual information (2.30) depends on $r$ only through the
Poisson random variable $\Pi_{i,r}$. With this in mind, given $i,j\leq N$ and
$L_{1},L_{2}\geq 0$, introduce the conditional mutual information functionals

$I_{N}^{i}(L_{1}):=\frac{1}{N}I\big{(}\widetilde{\mathbf{G}}_{N};\sigma^{\diamond}\mid\Pi_{i,r}=L_{1}\big{)}\quad\text{and}\quad
I_{N}^{i,j}(L_{1},L_{2}):=I\big{(}\widetilde{\mathbf{G}}_{N};\sigma^{\diamond}\mid\Pi_{i,r}=L_{1},\Pi_{j,r}=L_{2}\big{)}.$

A direct computation leveraging the product rule shows that

$\partial_{r}I_{N}(r)=\sum_{i\leq N}\sum_{L\geq
0}\pi(rN,L)\big{(}I_{N}^{i}(L+1)-I_{N}^{i}(L)\big{)}.$

Another direct computation leveraging the product rule reveals that

$\displaystyle\partial_{r}^{2}I_{N}(r)=$ $\displaystyle N\sum_{i\leq
N}\sum_{L_{1}\geq
0}\pi(rN,L_{1})\big{(}I_{N}^{i}(L_{1}+2)-2I_{N}^{i}(L_{1}+1)+I_{N}^{i}(L_{1})\big{)}$
$\displaystyle+N\sum_{i\leq N}\sum_{j\neq i}\sum_{L_{1},L_{2}\geq
0}\pi(rN,L_{1})\pi(rN,L_{2})\big{(}I_{N}^{i,j}(L_{1}+1,L_{2}+1)-I_{N}^{i,j}(L_{1}+1,L_{2})$
$\displaystyle\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\quad-
I_{N}^{i,j}(L_{1},L_{2}+1)+I_{N}^{i,j}(L_{1},L_{2})\big{)}.$

It therefore suffices to show that for fixed $i\neq j\leq N$ and
$L_{1},L_{2}\geq 0$,

$\displaystyle I_{N}^{i}(L_{1}+2)-2I_{N}^{i}(L_{1}+1)+I_{N}^{i}(L_{1})$
$\displaystyle\leq 0,$ (2.31) $\displaystyle
I_{N}^{i,j}(L_{1}+1,L_{2}+1)-I_{N}^{i,j}(L_{1}+1,L_{2})-I_{N}^{i,j}(L_{1},L_{2}+1)+I_{N}^{i,j}(L_{1},L_{2})$
$\displaystyle\leq 0.$ (2.32)

We will only prove (2.32) as the proof of (2.31) is similar. To alleviate
notation, conditionally on the events $\Pi_{i,r}=L_{1}$ and $\Pi_{j,r}=L_{2}$
introduce the random vectors

$\displaystyle\widetilde{\mathbf{G}}_{N}^{\prime}$
$\displaystyle:=\big{\\{}\widetilde{G}_{m,k}\mid m\neq i,\,\,\,m\neq j\text{
and }k\leq\Pi_{m,r}\big{\\}}\quad\text{and}\quad
Z:=\big{(}(\widetilde{G}_{i,\ell})_{\ell\leq
L_{1}},(\widetilde{G}_{j,\ell})_{\ell\leq
L_{2}},\widetilde{\mathbf{G}}_{N}^{\prime}\big{)}.$

By the chain rule for mutual information,

$\displaystyle I_{N}^{i,j}(L_{1}+1,L_{2}+1)$
$\displaystyle=I((\widetilde{G}_{i,L_{1}+1},\widetilde{G}_{j,L_{2}+1}),Z;\sigma^{\diamond})=I((\widetilde{G}_{i,L_{1}+1},\widetilde{G}_{j,L_{2}+1});\sigma^{\diamond}\mid
Z)+I(Z;\sigma^{\diamond}),$ $\displaystyle I_{N}^{i,j}(L_{1}+1,L_{2})$
$\displaystyle=I(\widetilde{G}_{i,L_{1}+1},Z;\sigma^{\diamond})=I(\widetilde{G}_{i,L_{1}+1};\sigma^{\diamond}\mid
Z)+I(Z;\sigma^{\diamond}),$ $\displaystyle I_{N}^{i,j}(L_{1},L_{2}+1)$
$\displaystyle=I(\widetilde{G}_{j,L_{2}+1},Z;\sigma^{\diamond})=I(\widetilde{G}_{j,L_{2}+1};\sigma^{\diamond}\mid
Z)+I(Z;\sigma^{\diamond}).$

It follows that (2.32) is equivalent to the inequality

$I((\widetilde{G}_{i,L_{1}+1},\widetilde{G}_{j,L_{2}+1});\sigma^{\diamond}\mid
Z)-I(\widetilde{G}_{i,L_{1}+1};\sigma^{\diamond}\mid
Z)-I(\widetilde{G}_{j,L_{2}+1};\sigma^{\diamond}\mid Z)\leq 0.$ (2.33)

Another application of the chain rule for mutual information reveals that

$I((\widetilde{G}_{i,L_{1}+1},\widetilde{G}_{j,L_{2}+1});\sigma^{\diamond}\mid
Z)=I(\widetilde{G}_{i,L_{1}+1};\sigma^{\diamond}\mid
Z)+I(\widetilde{G}_{j,L_{2}+1};\sigma^{\diamond}\mid
Z,\widetilde{G}_{i,L_{1}+1})$

and that the second term on the right side of this expression is given by

$\displaystyle I(\widetilde{G}_{j,L_{2}+1};$
$\displaystyle(\sigma^{\diamond},\widetilde{G}_{i,L_{1}+1})\mid
Z)-I(\widetilde{G}_{j,L_{2}+1};\widetilde{G}_{i,L_{1}+1}\mid Z)$
$\displaystyle=I(\widetilde{G}_{j,L_{2}+1};\sigma^{\diamond}\mid
Z)+I(\widetilde{G}_{j,L_{2}+1};\widetilde{G}_{i,L_{1}+1}\mid
Z,\sigma^{\diamond})-I(\widetilde{G}_{j,L_{2}+1};\widetilde{G}_{i,L_{1}+1}\mid
Z).$

It follows that the left side of (2.33) is

$I(\widetilde{G}_{j,L_{2}+1};\widetilde{G}_{i,L_{1}+1}\mid
Z,\sigma^{\diamond})-I(\widetilde{G}_{j,L_{2}+1};\widetilde{G}_{i,L_{1}+1}\mid
Z)=-I(\widetilde{G}_{j,L_{2}+1};\widetilde{G}_{i,L_{1}+1}\mid Z)\leq 0,$

where we have used that, conditionally on $\sigma^{\diamond}$, the random
variables $\smash{\widetilde{G}_{j,L_{2}+1}}$,
$\smash{\widetilde{G}_{i,L_{1}+1}}$, and $Z$ are independent to assert that
$\smash{I(\widetilde{G}_{j,L_{2}+1};\widetilde{G}_{i,L_{1}+1}\mid
Z,\sigma^{\diamond})=0}$. This establishes (2.33) and completes the proof. ∎

###### Proof of Proposition 2.3.

Fix $r,\eta>0$ and a probability measure
$\nu\in\Pr[-1,1]\cap\mathrm{Reg}(\mathcal{M}_{+})$. We define the measure
$\nu_{\eta}\in\mathrm{Reg}(\mathcal{M}_{+})$ by $\nu_{\eta}:=\eta\nu$. The
convexity property of the enriched free energy in Proposition 2.9 gives the
upper bound

$\overline{F}_{N}(t,\mu+r\nu_{\eta})=\overline{F}_{N}(t,\mu+(r\cdot
1+(1-r)\cdot 0)\nu_{\eta})\leq
r\overline{F}_{N}(t,\mu+\nu_{\eta})+(1-r)\overline{F}_{N}(t,\mu).$

Subtracting $\overline{F}_{N}(t,\mu)$ from both sides of this inequality,
dividing by $r$, and letting $r$ tend to zero yields

$\eta
D_{\mu}\overline{F}_{N}(t,\mu;\nu)\leq\overline{F}_{N}(t,\mu+\nu_{\eta})-\overline{F}_{N}(t,\mu).$

If $D_{t,\mu}$ denotes a weak subsequential limit of
$D_{\mu}\overline{F}_{N_{K}}(t,\mu,\cdot)$, then letting $K$ tend to infinity,
dividing by $\eta$, and letting $\eta$ tend to zero gives

$D_{t,\mu}(\nu)\leq D_{\mu}f(t,\mu;\nu).$

Repeating the argument with $\eta$ replaced by $-\eta$, which is possible as
$\mu,\nu\in\mathrm{Reg}(\mathcal{M}_{+})$, shows that for any
$\nu\in\mathrm{Reg}(\mathcal{M}_{+})$, we have
$D_{t,\mu}(\nu)=D_{\mu}f(t,\mu;\nu)$. By density of
$\mathrm{Reg}(\mathcal{M}_{+})$ in $\mathcal{M}_{+}$, this completes the
proof. ∎

##  3 Multioverlap concentration through perturbation

In the next section we will perform the cavity computation that gives rise to
the functional (1.26) and leads to the critical point representation in
Theorem 1.1. This cavity computation will require an understanding of the
asymptotic behavior of the array $(\sigma_{i}^{\ell})_{i,\ell\geq 1}$ of
i.i.d. replicas sampled from the Gibbs measure (2.13). In this section, we
leverage the main result in [8] to argue that, up to a small perturbation of
the enriched Hamiltonian (1.21) that does not affect the limit of the enriched
free energy (1.22), the asymptotic spin array $(\sigma_{i}^{\ell})_{i,\ell\geq
1}$ is generated by some probability measure $\nu\in\mathcal{M}_{p}$ according
to (S1)-(S2). In particular, as we will see in Lemma 3.4, this means that the
average of a product of finitely many replicas is asymptotically equal to a
product of moments of $\nu$. More precisely, given any finite set of $n$
replicas and collection $\\{\mathcal{C}_{\ell}\\}_{\ell\leq n}$ of finite
indices,

$\lim_{N\to+\infty}\operatorname{\mathbb{E}}\prod_{\ell\leq
n}\Big{\langle}\prod_{i\in\mathcal{C}_{\ell}}\sigma_{i}^{\ell}\Big{\rangle}=\prod_{\ell\leq
n}\operatorname{\mathbb{E}}x_{\ell}^{\lvert\mathcal{C}_{\ell}\rvert},$ (3.1)

where the $(x_{\ell})_{\ell\geq 1}$ are i.i.d. with law $\nu$. This
observation will be crucial in Lemma 4.2, where we study the asymptotic
behavior of the cavity representation and first arrive at the functional
(1.26).

The perturbation Hamiltonians that will enforce the rigid asymptotic structure
described by (S1)-(S2) depend on a perturbation parameter
$\lambda:=(\lambda_{k})_{k\geq 0}$ with $\lambda_{k}\in[2^{-k-1},2^{-k}]$ for
$k\geq 0$, and they enforce the concentration of the multioverlaps

$R_{1,\ldots,n}:=\frac{1}{N}\sum_{i\leq N}\sigma_{i}^{1}\cdots\sigma_{i}^{n}.$
(3.2)

As will be explained below, here $(\sigma_{i}^{\ell})_{i,\ell\geq 1}$ denote
i.i.d. replicas from the Gibbs measure associated with the sum of the
Hamiltonian (1.21) and the two perturbation Hamiltonians that we now
introduce. The first perturbation Hamiltonian will ensure the concentration of
the overlap $R_{1,2}$. Given a sequence $(\epsilon_{N})_{N\geq 1}$ with
$\epsilon_{N}:=N^{\gamma}$ for some $-1/8<\gamma<0$ and a standard Gaussian
vector $Z_{0}:=(Z_{0,1},\ldots,Z_{0,N})$ in $\mathbb{R}^{N}$, we introduce the
Gaussian perturbation Hamiltonian

$H_{N}^{\mathrm{gauss}}(\sigma,\lambda_{0}):=\mathcal{H}_{0}:=\sum_{i\leq
N}\big{(}\lambda_{0}\epsilon_{N}\sigma^{*}_{i}\sigma_{i}+\sqrt{\lambda_{0}\epsilon_{N}}Z_{0,i}\sigma_{i}\big{)}$
(3.3)

associated with the task of recovering the signal $\sigma^{*}$ from the data

$Y^{\mathrm{gauss}}:=\sqrt{\lambda_{0}\epsilon_{N}}\sigma^{*}+Z_{0}.$ (3.4)

Notice that

$1\geq\epsilon_{N}\to 0\quad\text{and}\quad N\epsilon_{N}\to+\infty.$ (3.5)

The second perturbation Hamiltonian will ensure the validity of the Franz-de
Sanctis identities [8, 18] which together with the concentration of the
overlap $R_{1,2}$ give the concentration of all the multioverlaps (3.2). We
consider a sequence $(s_{N})_{N\geq 1}$ with $s_{N}:=N^{\eta}$ for some
$4/5<\eta<1$, so that in particular

$\frac{s_{N}}{N}\to 0\quad\text{and}\quad\frac{s_{N}}{\sqrt{N}}\to+\infty.$
(3.6)

We fix a sequence of i.i.d. random variables $(\pi_{k})_{k\geq 1}$ with
$\operatorname{\operatorname{Poi}}(s_{N})$ distribution as well as a sequence
$e:=(e_{jk})_{j,k\geq 1}$ of random variables with
$\operatorname{\operatorname{Exp}}(1)$ distribution and a family
$(i_{jk})_{j,k\geq 1}$ of random indices uniformly sampled from the set
$\\{1,\ldots,N\\}$. We define the exponential perturbation Hamiltonian by

$H_{N}^{\mathrm{exp}}(\sigma):=\sum_{k\geq
1}\mathcal{H}_{k},\quad\text{where}\quad\mathcal{H}_{k}:=\sum_{j\leq\pi_{k}}\bigg{(}\log(1+\lambda_{k}\sigma_{i_{jk}}\big{)}-\frac{\lambda_{k}e_{jk}\sigma_{i_{jk}}}{1+\lambda_{k}\sigma_{i_{jk}}^{*}}\bigg{)}.$
(3.7)

This is the Hamiltonian associated with the task of recovering the signal
$\sigma^{*}$ from the independently generated data

$Y^{\mathrm{exp}}:=\bigg{\\{}\frac{e_{jk}}{1+\lambda_{k}\sigma^{*}_{i_{jk}}}\mid
j\leq\pi_{k}\text{ and }k\geq 1\bigg{\\}}.$ (3.8)

For each $(t,\mu)\in\mathbb{R}_{\geq 0}\times\mathcal{M}_{+}$, the perturbed
enriched Hamiltonian is the sum of the enriched Hamiltonian (1.21) and the
perturbation Hamiltonians just defined,

$H_{N}^{t,\mu}(\sigma,\lambda):=H_{N}^{t,\mu}(\sigma)+H_{N}^{\mathrm{gauss}}(\sigma,\lambda_{0})+H_{N}^{\mathrm{exp}}(\sigma,\lambda).$
(3.9)

As usual, we denote by

$F_{N}(t,\mu,\lambda):=\frac{1}{N}\log\int_{\Sigma_{N}}\exp
H_{N}^{t,\mu}(\sigma,\lambda)\,\mathrm{d}P_{N}^{*}(\sigma)\quad\text{and}\quad\overline{F}_{N}(t,\mu,\lambda):=\operatorname{\mathbb{E}}F_{N}(t,\mu,\lambda)$
(3.10)

its associated free energy functionals, and by $\langle\cdot\rangle$ its
corresponding Gibbs average. This means that for any bounded and measurable
function $f=f(\sigma^{1},\ldots,\sigma^{n})$ of finitely many replicas,

$\langle f(\sigma^{1},\ldots,\sigma^{n})\rangle:=\langle
f\rangle:=\frac{\int_{\Sigma_{N}^{n}}f(\sigma^{1},\ldots,\sigma^{n})\prod_{\ell\leq
n}\exp
H_{N}^{t,\mu}(\sigma^{\ell},\lambda)\,\mathrm{d}P_{N}^{*}(\sigma^{\ell})}{\big{(}\int_{\Sigma_{N}}\exp
H_{N}^{t,\mu}(\sigma,\lambda)\,\mathrm{d}P_{N}^{*}(\sigma)\big{)}^{n}}.$
(3.11)

Since the Gibbs measure associated with the perturbed Hamiltonian (3.9) is a
conditional expectation, it satisfies the Nishimori identity (2.14).

######  Lemma 3.1.

For all $(t,\mu)\in\mathbb{R}_{\geq 0}\times\mathcal{M}_{+}$ and perturbation
parameter $\lambda$, the enriched free energy (1.22) and the perturbed free
energy (3.10) are asymptotically equivalent,

$\lim_{N\to+\infty}\big{\lvert}\overline{F}_{N}(t,\mu,\lambda)-\overline{F}_{N}(t,\mu)\big{\rvert}=0.$
(3.12)

###### Proof.

This corresponds to [18, Lemma 4.1] but we provide a proof for completeness. A
direct computation reveals that

$\big{\lvert}\overline{F}_{N}(t,\mu,\lambda)-\overline{F}_{N}(t,\mu)\big{\rvert}\leq\frac{1}{N}\operatorname{\mathbb{E}}\max_{\sigma\in\Sigma_{N}}\big{\lvert}H_{N}^{\mathrm{gauss}}(\sigma,\lambda_{0})\big{\rvert}+\frac{1}{N}\operatorname{\mathbb{E}}\max_{\sigma\in\Sigma_{N}}\big{\lvert}H_{N}^{\mathrm{exp}}(\sigma,\lambda)\big{\rvert}.$

For any spin configuration $\sigma\in\Sigma_{N}$,

$\big{\lvert}H_{N}^{\mathrm{gauss}}(\sigma,\lambda_{0})\big{\rvert}\leq
N\epsilon_{N}+\sqrt{\epsilon_{N}}\sum_{i\leq N}\lvert Z_{0,i}\rvert$

and

$\big{\lvert}H_{N}^{\mathrm{exp}}(\sigma,\lambda)\big{\rvert}\leq\sum_{1\leq
k\leq
K^{\prime}}\sum_{j\leq\pi_{k}}\Big{(}\log(1+\lambda_{k})+\frac{\lambda_{k}e_{jk}}{1-\lambda_{k}}\Big{)}.$

Since these bounds are uniform in $\sigma$, it follows that

$\big{\lvert}\overline{F}_{N}(t,\mu,\lambda)-\overline{F}_{N}(t,\mu)\big{\rvert}\leq\epsilon_{N}+\sqrt{\epsilon_{N}}\operatorname{\mathbb{E}}Z_{0,1}+\frac{s_{N}}{N}\sum_{k\geq
1}\Big{(}\log(1+\lambda_{k})+\frac{\lambda_{k}}{1-\lambda_{k}}\Big{)}.$

The third term was obtained by taking the expectation with respect to the
randomness of $e$, and then with respect to the randomness of
$(\pi_{k})_{k\geq 1}$. Leveraging (3.5)-(3.6) to let $N$ tend to infinity
completes the proof. ∎

######  Lemma 3.2.

For every $n\geq 1$, the multioverlap $R_{1,\ldots,n}$ in (3.2) associated
with the perturbed Gibbs measure (3.11) concentrates on average over the
perturbation parameters,

$\lim_{N\to+\infty}\operatorname{\mathbb{E}}_{\lambda}\operatorname{\mathbb{E}}\big{\langle}(R_{1,\ldots,n}-\operatorname{\mathbb{E}}\langle
R_{1,\ldots,n}\rangle)^{2}\big{\rangle}=0,$ (3.13)

where $\operatorname{\mathbb{E}}_{\lambda}$ denotes the average with respect
to the uniform random variables $\lambda_{k}$ on $[2^{-k-1},2^{-k}]$.

###### Proof.

We introduce the concentration function

$v_{N}:=\frac{1}{N}\sup\Big{\\{}\operatorname{\mathbb{E}}\big{\lvert}F_{N}(t,\mu,\lambda)-\overline{F}_{N}(t,\mu,\lambda)\big{\rvert}^{2}\mid\lambda_{k}\in[2^{-k-1},2^{-k}]\text{
for all }k\geq 0\Big{\\}},$ (3.14)

and think of the perturbation parameters $\lambda:=(\lambda_{k})_{k\geq 0}$ as
being sampled independently with $\lambda_{k}$ uniform on $[2^{-k-1},2^{-k}]$
for $k\geq 0$. The concentration function $\nu_{N}$ is bounded from above by a
constant independent of $N$ by [18, Proposition B.4], so (3.13) follows from
[8, Theorem 2.2]. This completes the proof. ∎

Arguing as in [43, Lemma 3.3], it is now possible to find a deterministic
sequence $(\lambda^{N})_{N\geq 1}$ along which, for all $n\geq 1$,

$\lim_{N\to+\infty}\operatorname{\mathbb{E}}\big{\langle}(R_{1,\ldots,n}-\operatorname{\mathbb{E}}\langle
R_{1,\ldots,n}\rangle_{N})^{2}\big{\rangle}_{N}=0.$ (3.15)

Here, the Gibbs average $\langle\cdot\rangle_{N}$ for a system of size $N$ is
associated with the perturbation parameter $\lambda^{N}$, and the multioverlap
$R_{1,\ldots,n}$ is for i.i.d. replicas $(\sigma_{i}^{\ell})_{i,\ell\geq 1}$
sampled from this perturbed Gibbs measure. The multioverlap concentration
(3.15) ensures that the asymptotic spin array is generated by some probability
measure $\nu\in\mathcal{M}_{p}$ according to (S1)-(S2).

######  Proposition 3.3.

Let $(\lambda^{N})_{N\geq 1}$ be a deterministic sequence along which (3.15)
holds for all $n\geq 1$, and denote by $(\sigma_{i}^{\ell})_{i,\ell\geq 1}$
i.i.d. replicas sampled from the Gibbs measure $\langle\cdot\rangle_{N}$ in
(3.11) associated with the perturbation parameter $\lambda^{N}$. For any
subsequence $(N_{k})_{k\geq 1}$ along which the distribution of the spin array
$(\sigma_{i}^{\ell})_{i,\ell\geq 1}$ converges in the sense of finite-
dimensional distributions, there exists a probability measure
$\nu\in\mathcal{M}_{p}$ (which may depend on the subsequence) such that the
asymptotic spin array is generated by $\nu$ according to (S1)-(S2).

###### Proof.

This is immediate from [8, Corollary 2.3]. We have implicitly used that
$\operatorname{\mathbb{E}}\langle
R_{1}\rangle=\operatorname{\mathbb{E}}\sigma_{1}^{*}=\overline{m}$ by the
Nishimori identity to assert that $\nu\in\mathcal{M}_{p}$. ∎

The two-step process (S1)-(S2) to generate the asymptotic spin array from the
measure $\nu\in\mathcal{M}_{p}$ allows us to write the asymptotic spin array
$(\sigma_{i}^{\ell})_{i,\ell\geq 1}$ explicitly in terms of the i.i.d. random
variables $(x_{i})_{i\geq 1}$ sampled from $\nu$ and a family of i.i.d.
uniform random variables $(u_{i,\ell})_{i,\ell\geq 1}$ on $[0,1]$ also
independent of $(x_{i})_{i\geq 1}$,

$\sigma_{i}^{\ell}:=2\mathbbm{1}\bigg{\\{}u_{i,\ell}\leq\frac{1+x_{i}}{2}\bigg{\\}}-1.$
(3.16)

From this representation of the asymptotic spins we can readily establish
(3.1).

######  Lemma 3.4.

Let $(\lambda^{N})_{N\geq 1}$ be a deterministic sequence along which (3.15)
holds for all $n\geq 1$, denote by $(\sigma_{i}^{\ell})_{i,\ell\geq 1}$ i.i.d.
replicas sampled from the Gibbs measure $\langle\cdot\rangle_{N}$ in (3.11)
associated with the perturbation parameter $\lambda^{N}$, and suppose that the
distributional limit of the spin array $(\sigma_{i}^{\ell})_{i,\ell\geq 1}$ is
generated by a probability measure $\nu\in\mathcal{M}_{p}$ according to
(S1)-(S2). If $(x_{\ell})_{\ell\geq 1}$ denote i.i.d. samples from $\nu$, then
for any finite set of $n$ replicas and collection
$\\{\mathcal{C}_{\ell}\\}_{\ell\leq n}$ of finite indices,

$\lim_{N\to+\infty}\operatorname{\mathbb{E}}\prod_{\ell\leq
n}\Big{\langle}\prod_{i\in\mathcal{C}_{\ell}}\sigma_{i}^{\ell}\Big{\rangle}_{N}=\prod_{\ell\leq
n}\operatorname{\mathbb{E}}x_{\ell}^{\lvert\mathcal{C}_{\ell}\rvert}.$ (3.17)

###### Proof.

Let $\mathcal{C}:=\\{(i,\ell)\mid\ell\leq n\text{ and
}i\in\mathcal{C}_{\ell}\\}$, and observe that

$\operatorname{\mathbb{E}}\prod_{\ell\leq
n}\Big{\langle}\prod_{i\in\mathcal{C}_{\ell}}\sigma_{i}^{\ell}\Big{\rangle}_{N}=\operatorname{\mathbb{E}}\Big{\langle}\prod_{(i,\ell)\in\mathcal{C}}\sigma_{i}^{\ell}\Big{\rangle}_{N}.$

Recalling the representation (3.16) of an asymptotic spin in terms of a family
$(u_{i,\ell})_{i,\ell\geq 1}$ of i.i.d. uniform random variables on $[0,1]$,
and letting $N$ tend to infinity in the previous display reveals that

$\lim_{N\to+\infty}\operatorname{\mathbb{E}}\prod_{\ell\leq
n}\Big{\langle}\prod_{i\in\mathcal{C}_{\ell}}\sigma_{i}^{\ell}\Big{\rangle}_{N}=\operatorname{\mathbb{E}}\prod_{(i,\ell)\in\mathcal{C}}\bigg{(}2\mathbbm{1}\bigg{\\{}u_{i,\ell}\leq\frac{1+x_{i}}{2}\bigg{\\}}-1\bigg{)}=\operatorname{\mathbb{E}}\prod_{(i,\ell)\in\mathcal{C}}x_{i},$

where the independence of the $(u_{i,\ell})_{i,\ell\geq 1}$ has played its
part in the second equality. Leveraging the independence of the
$(x_{i})_{i\geq 1}$ completes the proof. ∎

This formula to compute the asymptotic moments of the spin array will be
applied to a slight modification of the Hamiltonian (1.21) in Section 4. The
arguments to justify the formula in this modified setting are identical to
those just given.

######  Remark 3.5.

We discuss briefly how to extend the results of the present paper and cover
the more general case when the measure $P^{*}$ is arbitrary with support in
$[-1,1]$ in place of $\\{-1,1\\}$, with the understanding that we stick with
the formula (1.4) to compute the link probabilities. In this case, the
description of the limit law of $(\sigma^{\ell}_{i})_{i,\ell\geq 1}$ must
naturally be modified from (S1)-(S2), since these explicitly refer to $\pm 1$
random variables. Instead, we would first need to sample i.i.d. random
probability measures $(\mu_{i})_{i\geq 1}$, each with law
$\Xi\in\Pr(\Pr[-1,1])$, and then conditionally on $(\mu_{i})_{i\geq 1}$, to
sample $\sigma_{i}^{\ell}$ according to the law $\mu_{i}$ independently over
$i,\ell\geq 1$. One key observation though is that Lemma 3.4 remains valid as
stated, provided that we understand that $\nu$ is the law of the mean value of
$\mu_{1}$, that is,

$\nu=\text{Law}\Big{(}\int x\,\,\mathrm{d}\mu_{1}(x)\Big{)}.$

Moreover, the cavity computations ultimately only require us to be able to
evaluate moments of the form described by Lemma 3.4. So even in this more
general setting, we do not really need to keep track of the more complex
object $\Xi\in\Pr(\Pr[-1,1])$, and can proceed as in the $\pm 1$ case by
focusing on this simpler measure $\nu\in\mathcal{M}_{p}$.

##  4 Cavity computation

In this section we perform the cavity computation that gives rise to the
functional (1.26) and leads to the critical point representation in Theorem
1.1. The idea behind the cavity computation is to determine the effect of
perturbing the Hamiltonian (3.9) for the system of size $N$ by a single
additional spin. To emphasize this additional spin, we write
$\rho:=(\epsilon,\sigma)\in\Sigma_{N+1}$ for $\sigma\in\Sigma_{N}$ and
$\epsilon\in\\{-1,+1\\}$. Similarly, we write
$\rho^{*}:=(\epsilon^{*},\sigma^{*})\in\Sigma_{N+1}$ for
$\sigma^{*}\in\Sigma_{N}$ and $\epsilon^{*}\in\Sigma_{1}$ to denote the ground
truth sampled from $P_{N+1}^{*}$. Given a pair $(t,\mu)\in\mathbb{R}_{\geq
0}\times\mathcal{M}_{+}$ and a perturbation parameter
$\lambda=(\lambda_{k})_{k\geq 0}$, we will compare the systems with
Hamiltonians $H_{N+1}^{t,\mu}(\epsilon,\sigma,\lambda)$ and
$H_{N}^{t,\mu}(\sigma,\lambda)$. This comparison will be done at the level of
the free energies, and the re-scaled difference between
$\overline{F}_{N+1}(t,\mu,\lambda)$ and $\overline{F}_{N}(t,\mu,\lambda)$ will
be expressed as the difference between the modified Gibbs averages of two
quantities that we may call the “cavity fields”. We now describe the modified
Hamiltonian associated with this modified Gibbs average as well as the two
cavity fields.

For each $t\geq 0$, we introduce a random variable
$\smash{\Pi_{t}\sim\operatorname{\operatorname{Poi}}t\tbinom{N}{2}}$ as well
as an independent family of i.i.d. random matrices
$\smash{(\widetilde{G}^{k})_{k\geq 1}}$ each having conditionally independent
entries $\smash{(\widetilde{G}_{i,j}^{k})_{i,j\leq N}}$ taking values in
$\\{0,1\\}$ with conditional distribution

$\mathbb{P}\big{\\{}\widetilde{G}_{i,j}^{k}=1\mid\sigma^{*}\big{\\}}:=\frac{c+\Delta\sigma_{i}^{*}\sigma_{j}^{*}}{N+1}.$
(4.1)

Given a collection of random indices $\smash{(i_{k},j_{k})_{k\geq 1}}$ sampled
uniformly at random from $\smash{\\{1,\ldots,N\\}^{2}}$, independently of the
other random variables, we define the modified time-dependent Hamiltonian
$\mathcal{H}_{N}^{t}$ on $\Sigma_{N}$ by

$\mathcal{H}_{N}^{t}(\sigma):=\sum_{k\leq\Pi_{t}}\log\bigg{[}\big{(}c+\Delta\sigma_{i_{k}}\sigma_{j_{k}}\big{)}^{\widetilde{G}_{i_{k},j_{k}}^{k}}\Big{(}1-\frac{c+\Delta\sigma_{i_{k}}\sigma_{j_{k}}}{N+1}\Big{)}^{1-\widetilde{G}^{k}_{i_{k},j_{k}}}\bigg{]}.$
(4.2)

Similarly, for each $\mu\in\mathcal{M}_{+}$, consider a sequence
$(\Lambda_{i}((N+1)\mu))_{i\leq N}$ of independent Poisson point processes
with mean measure $(N+1)\mu$, and define the modified measure-dependent
Hamiltonian $\mathcal{H}_{N}^{\mu}$ on $\Sigma_{N}$ by

$\mathcal{H}_{N}^{\mu}(\sigma):=\sum_{i\leq
N}\sum_{x\in\Lambda_{i}((N+1)\mu)}\log\bigg{[}\big{(}c+\Delta\sigma_{i}x\big{)}^{\widetilde{G}^{x}_{i}}\Big{(}1-\frac{c+\Delta\sigma_{i}x}{N+1}\Big{)}^{1-\widetilde{G}^{x}_{i}}\bigg{]},$
(4.3)

where $(\widetilde{G}_{i}^{x})_{i\leq N}$ are conditionally independent random
variables taking values in $\\{0,1\\}$ with conditional distribution

$\mathbb{P}\big{\\{}\widetilde{G}_{i}^{x}=1\mid\sigma^{*},x\big{\\}}:=\frac{c+\Delta\sigma_{i}^{*}x}{N+1}.$
(4.4)

For each pair $(t,\mu)\in\mathbb{R}_{\geq 0}\times\mathcal{M}_{+}$, the
modified enriched Hamiltonian is the sum of the modified time-dependent and
measure-dependent Hamiltonians,

$\mathcal{H}_{N}^{t,\mu}(\sigma):=\mathcal{H}_{N}^{t}(\sigma)+\mathcal{H}_{N}^{\mu}(\sigma),$
(4.5)

and, given a perturbation parameter $\lambda:=(\lambda_{k})_{k\geq 0}$, its
perturbed version is

$\mathcal{H}_{N}^{t,\mu}(\sigma,\lambda):=\mathcal{H}_{N}^{t,\mu}(\sigma)+H_{N}^{\mathrm{gauss}}(\sigma,\lambda_{0})+H_{N}^{\mathrm{exp}}(\sigma,\lambda)$
(4.6)

for the perturbation Hamiltonians (3.3) and (3.7). As usual, we write

$\overline{F}_{N}^{\prime}(t,\mu):=\frac{1}{N}\operatorname{\mathbb{E}}\log\int_{\Sigma_{N}}\exp\mathcal{H}_{N}^{t,\mu}(\sigma)\,\mathrm{d}P_{N}^{*}(\sigma)$
(4.7)

for the modified free energy, and for each perturbation parameter
$\lambda=(\lambda_{k})_{k\geq 0}$, we denote by

$\overline{F}_{N}^{\prime}(t,\mu,\lambda):=\frac{1}{N}\operatorname{\mathbb{E}}\log\int_{\Sigma_{N}}\exp\mathcal{H}_{N}^{t,\mu}(\sigma,\lambda)\,\mathrm{d}P_{N}^{*}(\sigma)$
(4.8)

its perturbed version. As in (3.11), we write $\langle\cdot\rangle^{\prime}$
for the Gibbs average with respect to the perturbed and modified Hamiltonian
(4.6). This means that for any bounded and measurable function
$f=f(\sigma^{1},\ldots,\sigma^{n})$ of finitely many replicas,

$\langle f(\sigma^{1},\ldots,\sigma^{n})\rangle^{\prime}:=\langle
f\rangle^{\prime}:=\frac{\int_{\Sigma_{N}^{n}}f(\sigma^{1},\ldots,\sigma^{n})\prod_{\ell\leq
n}\exp\mathcal{H}_{N}^{t,\mu}(\sigma^{\ell},\lambda)\,\mathrm{d}P_{N}^{*}(\sigma^{\ell})}{\big{(}\int_{\Sigma_{N}}\exp\mathcal{H}_{N}^{t,\mu}(\sigma,\lambda)\,\mathrm{d}P_{N}^{*}(\sigma)\big{)}^{n}}.$
(4.9)

This is the modified Gibbs measure relative to which the cavity fields will be
averaged. It is readily verified that this Gibbs measure is a conditional
expectation, and therefore satisfies the Nishimori identity (2.14).

The first cavity field will have a time-dependent and a measure-dependent
component. For each $t\geq 0$, we introduce a random variable
$\Pi_{t}^{\prime}\sim\operatorname{\operatorname{Poi}}(tN)$ as well as an
independent family of i.i.d. random vectors $\smash{({G^{\prime}}^{k})_{k\geq
1}}$ each having conditionally independent entries
$\smash{({G^{\prime}}^{k}_{i})_{i\leq N}}$ taking values in $\\{0,1\\}$ with
conditional distribution

$\mathbb{P}\big{\\{}{G^{\prime}}^{k}_{i}=1\mid\epsilon^{*},\sigma^{*}\big{\\}}:=\frac{c+\Delta\epsilon^{*}\sigma_{i}^{*}}{N+1}.$
(4.10)

Given a collection of random indices $(\ell_{k})_{k\geq 1}$ sampled uniformly
at random from $\\{1,\ldots,N\\}$, independently of the other random
variables, we define the time-dependent cavity field
$z_{N}^{t}(\epsilon,\sigma)$ on $\Sigma_{N+1}$ by

$z_{N}^{t}(\epsilon,\sigma):=\sum_{k\leq\Pi_{t}^{\prime}}\log\bigg{[}\big{(}c+\Delta\epsilon\sigma_{\ell_{k}}\big{)}^{{G^{\prime}}^{k}_{\ell_{k}}}\Big{(}1-\frac{c+\Delta\epsilon\sigma_{\ell_{k}}}{N+1}\Big{)}^{1-{G^{\prime}}^{k}_{\ell_{k}}}\bigg{]}.$
(4.11)

Similarly, for each $\mu\in\mathcal{M}_{+}$, we define the measure-dependent
cavity field $z_{N}^{\mu}(\epsilon)$ on $\Sigma_{1}$ by

$z_{N}^{\mu}(\epsilon):=\sum_{x\in\Lambda_{i}(N\mu)}\log\bigg{[}\big{(}c+\Delta\epsilon
x\big{)}^{{G^{\prime}}^{x}}\Big{(}1-\frac{c+\Delta\epsilon
x}{N+1}\Big{)}^{1-{G^{\prime}}^{x}}\bigg{]},$ (4.12)

where $G^{\prime x}$ are conditionally independent random variables taking
values in $\\{0,1\\}$ with conditional distribution

$\mathbb{P}\big{\\{}G^{\prime
x}=1\mid\epsilon^{*},x\big{\\}}:=\frac{c+\Delta\epsilon^{*}x}{N+1}.$ (4.13)

For each pair $(t,\mu)\in\mathbb{R}_{\geq 0}\times\mathcal{M}_{+}$, the cavity
field $z_{N}^{t,\mu}(\epsilon,\sigma)$ is the sum of the time-dependent and
measure-dependent cavity fields,

$z_{N}^{t,\mu}(\epsilon,\sigma):=z_{N}^{t}(\epsilon,\sigma)+z_{N}^{\mu}(\epsilon).$
(4.14)

The second cavity field will only have a time-dependent component. For each
$t\geq 0$, introduce an independent family of i.i.d. random matrices
$\smash{({G^{\prime\prime}}^{k})_{k\geq 1}}$ each having conditionally
independent entries $\smash{({G^{\prime\prime}}^{k}_{i,j})_{i,j\leq N}}$
taking values in $\\{0,1\\}$ with conditional distribution

$\mathbb{P}\big{\\{}{G^{\prime\prime}}^{k}_{i,j}=1\mid\sigma^{*}\big{\\}}:=\frac{c+\Delta\sigma^{*}_{i}\sigma^{*}_{j}}{N(N+1)}.$
(4.15)

Given a collection of random indices $\smash{(\ell_{k},m_{k})_{k\geq 1}}$
sampled uniformly at random from $\smash{\\{1,\ldots,N\\}^{2}}$, independently
of the other random variables, we define the cavity field $y_{N}(\sigma)$ on
$\Sigma_{N}$ by

$y_{N}^{t}(\sigma):=\sum_{k\leq\Pi_{t}}\log\bigg{[}\big{(}c+\Delta\sigma_{\ell_{k}}\sigma_{m_{k}}\big{)}^{{G^{\prime\prime}}^{k}_{\ell_{k},m_{k}}}\Big{(}1-\frac{c+\Delta\sigma_{\ell_{k}}\sigma_{m_{k}}}{N(N+1)}\Big{)}^{1-{G^{\prime\prime}}^{k}_{\ell_{k},m_{k}}}\bigg{]}.$
(4.16)

The cavity computation reveals that the re-scaled difference between
$\overline{F}_{N+1}(t,\mu,\lambda)$ and $\overline{F}_{N}(t,\mu,\lambda)$ is
the difference between the cavity fields (4.14) and (4.16) averaged with
respect to the Gibbs measure (4.9). More precisely, for each pair
$(t,\mu)\in\mathbb{R}_{\geq 0}\times\mathcal{M}_{+}$, the asymptotic
difference between these re-scaled free energies is given by

$A_{N}(t,\mu,\lambda):=\operatorname{\mathbb{E}}\log\int_{\Sigma_{1}}\langle\exp
z_{N}^{t,\mu}(\epsilon,\sigma)\rangle^{\prime}\,\mathrm{d}P^{*}(\epsilon)-\operatorname{\mathbb{E}}\log\big{\langle}\exp
y_{N}^{t}(\sigma)\big{\rangle}^{\prime}.$ (4.17)

To prove this, it will be convenient to introduce notation for approximate
distributional identities. Given two Hamiltonians $H_{N}^{1}$ and $H_{N}^{2}$
on $\Sigma_{N}$, we write

$H_{N}^{1}(\sigma)\stackrel{{\scriptstyle d}}{{\approx}}H_{N}^{2}(\sigma)\iff
N\overline{F}_{N}^{1}=N\overline{F}_{N}^{2}+o_{N}(1),$ (4.18)

where $\overline{F}_{N}^{1}$ and $\overline{F}_{N}^{2}$ denote the free
energies associated with the Hamiltonians $H_{N}^{1}$ and $H_{N}^{2}$,
respectively. Oftentimes we will not prove rigorously that two Hamiltonians
are approximately equal in distribution, contenting ourselves with showing
that a given realization of them differs by a quantity that is stochastically
of negative degree in $N$. However, all approximate distributional identities
that we introduce may be justified rigorously through an interpolation
argument similar to that in Lemmas 3.1 or 4.4.

######  Lemma 4.1.

For all $(t,\mu)\in\mathbb{R}_{\geq 0}\times\mathcal{M}_{+}$, perturbation
parameter $\lambda$, and $N\geq 1$, the re-scaled difference between the
perturbed free energy (3.10) for a system of size $N+1$ and a system of size
$N$ is given by

$(N+1)\overline{F}_{N+1}(t,\mu,\lambda)-N\overline{F}_{N}(t,\mu,\lambda)=A_{N}(t,\mu,\lambda)+o_{N}(1),$
(4.19)

where the error term is uniform over the perturbation parameter
$\lambda=(\lambda_{k})_{k\geq 1}$ with $\lambda_{k}\in[2^{-k-1},2^{-k}]$ for
$k\geq 0$.

###### Proof.

For simplicity, we will ignore the perturbation Hamiltonians (3.3) and (3.7);
however, the choices (3.5)-(3.6) of the sequences $(\epsilon_{N})_{N\geq 1}$
and $(s_{N})_{N\geq 1}$ can be used to show that the error incurred by
replacing the perturbation Hamiltonian
$H_{N+1}^{\mathrm{gauss}}(\sigma,\lambda_{0})+H_{N+1}^{\mathrm{exp}}(\sigma,\lambda)$
for the system of size $N+1$ by that for the system of size $N$ can be
absorbed into the error term on the right side of (4.19). The proof proceeds
in two steps. First we relate the time-dependent Hamiltonian (1.15) for the
system of size $N+1$ to that for the system of size $N$, and then we do the
same for the measure-dependent Hamiltonian (1.18).
_Step 1: time-dependent cavity computation._ The probability that the tuple
$\\{i_{k},j_{k}\\}$ in the time-dependent Hamiltonian (1.15) contains the
index $N+1$ is

$1-\Big{(}\frac{N}{N+1}\Big{)}^{2}=\frac{2N+1}{(N+1)^{2}}=\frac{2}{N+1}-\frac{1}{(N+1)^{2}}.$

It follows by the Poisson coloring theorem that

$H_{N+1}^{t}(\rho)\stackrel{{\scriptstyle
d}}{{\approx}}\mathcal{H}_{N}^{t}(\sigma)+z_{N}^{t}(\epsilon,\sigma)$ (4.20)

for the modified time-dependent Hamiltonian (4.2) and the time-dependent
cavity field (4.11). We have implicitly used that

$t\binom{N+1}{2}\Big{(}1-\frac{2}{N+1}\Big{)}=t\binom{N}{2}\quad\text{and}\quad
t\binom{N+1}{2}\cdot\frac{2}{N+1}=tN.$

Another application of the Poisson coloring theorem reveals that

$H_{N}^{t}(\sigma)\stackrel{{\scriptstyle
d}}{{\approx}}\mathcal{H}_{N}^{t}(\sigma)+y_{N}^{t}(\sigma),$ (4.21)

for the modified time-dependent Hamiltonian (4.2) and the time-dependent
cavity field (4.16). We have implicitly used that

$\frac{1}{N}-\frac{1}{N+1}=\frac{1}{N(N+1)}.$

_Step 2: measure-dependent cavity computation._ The measure-dependent
Hamiltonian (1.18) admits the approximate distributional decomposition

$H_{N+1}^{\mu}(\rho)\stackrel{{\scriptstyle
d}}{{\approx}}\mathcal{H}_{N}^{\mu}(\sigma)+z_{N}^{\mu}(\epsilon,\sigma)$
(4.22)

for the modified measure-dependent Hamiltonian (4.3) and the measure-dependent
cavity field (4.12). Similarly, the Poisson coloring theorem implies that

$H_{N}^{\mu}(\sigma)\stackrel{{\scriptstyle
d}}{{\approx}}\mathcal{H}_{N}^{\mu}(\sigma).$ (4.23)

Combining the cavity computations (4.20)-(4.23) with the definitions of the
modified enriched Hamiltonian (4.5), the Gibbs average (4.9), and the cavity
fields (4.14) and (4.16) completes the proof. ∎

The cavity representation in Lemma 4.1 suggests that to understand the limit
of the free energy, it is helpful to understand the asymptotic behavior of the
quantity $A_{N}$ in (4.17). Indeed, if both limits existed, they would
coincide. Unfortunately, it will not be possible to show that the limit of
$A_{N}$ exists in general; however, we will now argue that any subsequential
limit of $A_{N}$ along which the multioverlaps (3.2) concentrate in the sense
of (3.15) is given by the functional (1.26) evaluated at a measure
$\nu\in\mathcal{M}_{p}$ which generates the asymptotic spin array according to
(S1)-(S2).

######  Lemma 4.2.

Fix $(t,\mu)\in\mathbb{R}_{\geq 0}\times\mathcal{M}_{+}$, and suppose there is
a sequence $(N_{k},\lambda^{N_{k}})_{k\geq 1}$ such that $(N_{k})_{k\geq 1}$
increases to infinity and the multioverlap concentration (3.15) for the
Hamiltonian (4.6) with perturbation parameters $(\lambda^{N_{k}})_{k\geq 1}$
holds for every $n\geq 1$. Then there are a subsequence
$(N_{k}^{\prime},\lambda^{N_{k}^{\prime}})_{k\geq 1}$ and a measure
$\nu\in\mathcal{M}_{p}$ such that the asymptotic spin array associated with
the perturbation parameters $(\lambda^{N^{\prime}_{k}})_{k\geq 1}$ is
generated by $\nu$ according to (S1)-(S2) and

$\lim_{k\to+\infty}A_{N_{k}^{\prime}}\big{(}t,\mu,\lambda^{N_{k}^{\prime}}\big{)}=\operatorname{\mathcal{P}}_{t,\mu}(\nu).$
(4.24)

###### Proof.

Using the Prokhorov theorem, we can find a subsequence
$(N_{k}^{\prime})_{k\geq 1}$ along which the distribution of the spin array
$(\sigma_{i}^{\ell})_{i,\ell\geq 1}$ under the Gibbs measure
$\operatorname{\mathbb{E}}\langle\cdot\rangle^{\prime}$ converges in the sense
of finite-dimensional distributions. For simplicity of notation, we will
denote this subsequence $N_{k}^{\prime}$ simply by $N$. By Proposition 3.3,
there exists a probability measure $\nu\in\mathcal{M}_{p}$ such that the
asymptotic spin array is generated by $\nu$ according to (S1)-(S2). The rest
of the proof is devoted to establishing (4.24) and proceeds in two steps. Each
step determines the asymptotic behavior of one of the quantities

$A_{N}^{1}(t,\mu,\lambda^{N}):=\operatorname{\mathbb{E}}\log\int_{\Sigma_{1}}\langle\exp
z_{N}^{t,\mu}(\epsilon,\sigma)\rangle^{\prime}\,\mathrm{d}P^{*}(\epsilon)\quad\text{and}\quad
A_{N}^{2}(t,\mu,\lambda^{N}):=\operatorname{\mathbb{E}}\log\langle\exp
y_{N}^{t}(\sigma)\rangle^{\prime}$

which make up $A_{N}(t,\mu,\lambda^{N})$. The main difficulty will be that
there are two limits to account for: the convergence of the spin array
$(\sigma_{i}^{\ell})_{i,\ell\geq 1}$ and the convergence of the Poisson sums.
Nonetheless, it will be possible to show that

$\lim_{N\to+\infty}A_{N}^{1}(t,\mu,\lambda^{N})=\psi(\mu+t\nu)\quad\text{and}\quad\lim_{N\to+\infty}A_{N}^{2}(t,\mu,\lambda^{N})=\frac{t}{2}\int_{-1}^{1}G_{\nu}(y)\,\mathrm{d}\nu(y)$
(4.25)

for the initial condition $\psi:\mathcal{M}_{+}\to\mathbb{R}$ in (1.2) and the
function $G_{\nu}:[-1,1]\to\mathbb{R}$ in (1.25). To alleviate notation, the
dependence of $A_{N}^{1}$, $A_{N}^{2}$, $z_{N}$ and $y_{N}$ on $t$, $\mu$ and
$\lambda^{N}$ will be kept implicit, and we will write $s:=\mu[-1,1]$.
_Step 1: limit of $A_{N}^{1}$._ The modified Hamiltonian (4.6) corresponds to
a problem in statistical inference, so the Nishimori identity (2.14) implies
that

$A_{N}^{1}=\operatorname{\mathbb{E}}\log\int_{\Sigma_{1}}\langle\exp
z_{N}(\epsilon,\sigma^{1})\rangle^{\prime}\,\mathrm{d}P^{*}(\epsilon)$

for the cavity field

$z_{N}(\epsilon,\sigma^{1}):=\sum_{k\leq\Pi_{t}^{\prime}}\log\bigg{[}\big{(}c+\Delta\epsilon\sigma_{\ell_{k}}^{1}\big{)}^{{G^{\prime}}^{k}_{\ell_{k}}}\Big{(}1-\frac{c+\Delta\epsilon\sigma_{\ell_{k}}^{1}}{N+1}\Big{)}^{1-{G^{\prime}}^{k}_{\ell_{k}}}\bigg{]}+z_{N}^{\mu}(\epsilon),$

where, through a slight abuse of notation, the family of i.i.d. random
matrices $\smash{({G^{\prime}}^{k})_{k\geq 1}}$ each has conditionally
independent entries $\smash{({G^{\prime}}^{k}_{i})_{i\leq N}}$ taking values
in $\\{0,1\\}$ with conditional distribution

$\mathbb{P}\big{\\{}{G^{\prime}}^{k}_{i}=1\mid\epsilon^{*},\sigma^{2}\big{\\}}:=\frac{c+\Delta\epsilon^{*}\sigma_{i}^{2}}{N+1}.$

The Poisson coloring theorem implies that the number of terms in the first sum
defining $z_{N}(\epsilon,\sigma^{1})$ that have
$\smash{{G^{\prime}}^{k}_{\ell_{k}}}$ equal to zero is Poisson with mean

$tN\cdot\bigg{(}1-\frac{c+\Delta\epsilon^{*}\sigma_{\ell_{k}}^{2}}{N+1}\bigg{)}=tN+\operatorname{\operatorname{\mathcal{O}}}(1).$

Together with a Taylor expansion of the logarithm and the law of large
numbers, this implies that the contribution of the terms in the first term
defining $z_{N}(\epsilon,\sigma^{1})$ that have
$\smash{{G^{\prime}}^{k}_{\ell_{k}}}$ equal to zero is

$-\sum_{k\leq
tN}\frac{c+\Delta\epsilon\sigma^{1}_{\ell_{k}}}{N}+\operatorname{\operatorname{\mathcal{O}}}\big{(}N^{-1}\big{)}=-ct-\Delta
t\epsilon\overline{m}+\operatorname{\operatorname{\mathcal{O}}}\big{(}N^{-1}\big{)}.$

A similar argument shows that the contribution of the terms in
$z_{N}^{\mu}(\epsilon)$ that have $\smash{{G^{\prime}}^{x}}$ equal to zero is
$-cs-\Delta
s\epsilon\operatorname{\mathbb{E}}x_{1}+\operatorname{\operatorname{\mathcal{O}}}(N^{-1})$.
If we define

$z_{N}^{\prime}(\epsilon,\sigma^{1}):=\sum_{k\leq\Pi_{t}^{\prime}}\log\big{(}c+\Delta\epsilon\sigma_{\ell_{k}}^{1}\big{)}^{{G^{\prime}}^{k}_{\ell_{k}}}+\sum_{k\leq\Pi^{\prime}_{s}}\log\big{(}c+\Delta\epsilon
x_{k}\big{)}^{{G^{\prime}}_{k}^{x}},$

where $(x_{k})_{k\geq 1}$ are i.i.d. random variables with law $\mu$ and,
through a slight abuse of notation, the random variables
$\smash{({G^{\prime}}^{x}_{k})_{k\geq 1}}$ are conditionally independent with
conditional distribution

$\mathbb{P}\big{\\{}{G^{\prime}}_{k}^{x}=1\mid\epsilon^{*},x_{k}\big{\\}}:=\frac{c+\Delta\epsilon^{*}x_{k}}{N+1},$

then this implies that

$A_{N}^{1}=\operatorname{\mathbb{E}}\log\int_{\Sigma_{1}}\exp(-\Delta\epsilon(t\overline{m}+s\operatorname{\mathbb{E}}x_{1}))\langle\exp
z_{N}^{\prime}(\epsilon,\sigma^{1})\rangle^{\prime}\,\mathrm{d}P^{*}(\epsilon)-c(t+s)+\operatorname{\operatorname{\mathcal{O}}}\big{(}N^{-1}\big{)}.$
(4.26)

We now fix $\delta>0$ and focus on the first term in this expression which we
denote by $A_{N}^{1,1}$. Averaging with respect to the randomness of
$\smash{({G^{\prime}}^{k}_{\ell_{k}})_{k\geq 1}}$ and
$\smash{({G^{\prime}}^{x}_{k})_{k\geq 1}}$, and using symmetry between sites
reveals that $\smash{A_{N}^{1,1}}$ is given by averaging the sum over
$r\leq\Pi_{t}^{\prime}$ and $q\leq\Pi_{s}^{\prime}$ of

$\displaystyle\binom{\Pi_{t}^{\prime}}{r}\binom{\Pi_{s}^{\prime}}{q}\prod_{i=1}^{r}$
$\displaystyle\frac{c+\Delta\epsilon^{*}\sigma_{i}^{2}}{N+1}\prod_{i=r+1}^{\Pi_{t}^{\prime}}\bigg{(}1-\frac{c+\Delta\epsilon^{*}\sigma_{i}^{2}}{N+1}\bigg{)}\prod_{j=1}^{q}\frac{c+\Delta\epsilon^{*}x_{j}}{N+1}\prod_{j=q+1}^{\Pi_{s}^{\prime}}\bigg{(}1-\frac{c+\Delta\epsilon^{*}x_{j}}{N+1}\bigg{)}$
$\displaystyle\qquad\log\bigg{\langle}\int_{\Sigma_{1}}\exp(-\Delta\epsilon(t\overline{m}+s\operatorname{\mathbb{E}}x_{1}))\prod_{i=1}^{r}(c+\Delta\epsilon\sigma^{1}_{i})\prod_{j=1}^{q}(c+\Delta\epsilon
x_{j})\,\mathrm{d}P^{*}(\epsilon)\bigg{\rangle}^{\prime}$

with respect to the randomness of $\sigma^{2}$, $(x_{k})_{k\geq 1}$,
$\Pi_{t}^{\prime}$, $\Pi_{s}^{\prime}$, and $\epsilon^{*}$. Since the
expression in the second line of this display grows at most linearly in $r$
and $q$, the concentration of $\Pi_{t}^{\prime}$ and $\Pi_{s}^{\prime}$ about
$tN$ and $sN$, respectively, and the rapid decay of the binomial coefficients
allow us to truncate both sums at a large enough integer $M$ by incurring an
error of at most $\delta$ uniformly over $N$. Together with the approximations

$\displaystyle\prod_{i=r+1}^{\Pi_{t}^{\prime}}\bigg{(}1-\frac{c+\Delta\epsilon^{*}\sigma_{i}^{1}}{N+1}\bigg{)}$
$\displaystyle=\exp(-tc-\Delta t\epsilon^{*}\overline{m})+o_{N}(1),$
$\displaystyle\prod_{j=q+1}^{\Pi_{s}^{\prime}}\bigg{(}1-\frac{c+\Delta\epsilon^{*}x_{j}}{N+1}\bigg{)}$
$\displaystyle=\exp(-sc-\Delta
s\epsilon^{*}\operatorname{\mathbb{E}}x_{1})+o_{N}(1),$

this implies that, up to an error vanishing in $N$ and uniform in $\delta$,
the quantity $A_{N}^{1,1}$ is given by

$\displaystyle\operatorname{\mathbb{E}}\sum_{r\leq\Pi_{t}^{\prime}\wedge
M}\sum_{q\leq\Pi_{s}^{\prime}\wedge M}\binom{\Pi_{t}^{\prime}}{r}$
$\displaystyle\binom{\Pi_{s}^{\prime}}{q}\frac{e^{-c(t+s)-\Delta\epsilon^{*}(t\overline{m}+s\operatorname{\mathbb{E}}x_{1})}}{(N+1)^{r+q}}\bigg{\langle}\prod_{i=1}^{r}(c+\Delta\epsilon^{*}\sigma_{i}^{2})\prod_{j=1}^{q}(c+\Delta\epsilon^{*}x_{j})$
$\displaystyle\log\bigg{\langle}\int_{\Sigma_{1}}e^{-\Delta\epsilon(t\overline{m}+s\operatorname{\mathbb{E}}x_{1})}\prod_{i=1}^{r}(c+\Delta\epsilon\sigma^{1}_{i})\prod_{j=1}^{q}(c+\Delta\epsilon
x_{j})\,\mathrm{d}P^{*}(\epsilon)\bigg{\rangle}^{\prime}\bigg{\rangle}^{\prime}.$

Up to incurring a further error vanishing in $N$ and uniform in $\delta$, the
Weierstrass approximation theorem can be used to replace the logarithm by a
polynomial. This results in a linear combination of finitely many terms that
are functions of finitely many spins at a time. Invoking Lemma 3.4 to replace
each of the averaged spins in this expression by a sample from the measure
$\nu$, tracing back the manipulations we have done to $A_{N}^{1,1}$, and
letting $\delta$ tend to zero reveals that

$A_{N}^{1,1}=\operatorname{\mathbb{E}}\log\int_{\Sigma_{1}}\exp(-\Delta\epsilon(t\overline{m}+s\operatorname{\mathbb{E}}x_{1}))\exp
z_{N}^{\prime}(\epsilon)\,\mathrm{d}P^{*}(\epsilon)+o_{N}(1),$

for the cavity field

$z_{N}^{\prime}(\epsilon):=\sum_{k\leq\Pi_{t}^{\prime}}\log(c+\Delta\epsilon
y_{k})^{{G^{\prime}}^{y}_{k}}+\sum_{k\leq\Pi_{s}^{\prime}}\log(c+\Delta\epsilon
x_{k})^{{G^{\prime}}^{x}_{k}},$ (4.27)

where $(y_{k})_{k\geq 1}$ are independent samples from the measure
$\nu\in\mathcal{M}_{p}$ which generates the asymptotic spin array according to
(S1)-(S2). In other words, it is possible to take the limit of the spin array
while leaving the finite-volume Poisson sums unaffected. To simplify this
expression further, suppose that the probability measures $\overline{\mu}$ and
$\nu$ are discrete,

$\overline{\mu}:=\sum_{\ell\leq
K}p_{\ell}\delta_{a_{\ell}}\quad\text{and}\quad\nu:=\sum_{\ell\leq
K}q_{\ell}\delta_{a_{\ell}}$

for some integer $K\geq 1$, some weights $p_{\ell},q_{\ell}\in[0,1]$, and some
masses $a_{\ell}\in[-1,1]$. For each $\ell\leq K$, introduce the index sets

$\mathcal{I}_{t}^{\ell}:=\\{k\leq\Pi_{t}^{\prime}\mid{G^{\prime}}_{k}^{y}=1\text{
and
}y_{k}=a_{\ell}\\}\quad\text{and}\quad\mathcal{I}_{s}^{\ell}:=\\{k\leq\Pi_{s}^{\prime}\mid{G^{\prime}}_{k}^{x}=1\text{
and }x_{k}=a_{\ell}\\}$

in such a way that

$\displaystyle A_{N}^{1,1}$
$\displaystyle=\operatorname{\mathbb{E}}\log\int_{\Sigma_{1}}\exp(-\Delta\epsilon(t\overline{m}+s\operatorname{\mathbb{E}}x_{1}))\prod_{k\leq\Pi_{t}^{\prime}}(c+\Delta\epsilon
y_{k})^{{G^{\prime}}_{k}^{y}}\prod_{k\leq\Pi_{s}^{\prime}}(c+\Delta\epsilon
x_{k})^{{G^{\prime}}_{k}^{x}}\,\mathrm{d}P^{*}(\epsilon)+o_{N}(1)$
$\displaystyle=\operatorname{\mathbb{E}}\log\int_{\Sigma_{1}}\exp(-\Delta\epsilon(t\overline{m}+s\operatorname{\mathbb{E}}x_{1}))\prod_{\ell\leq
K}\prod_{k\in I_{t}^{\ell}\cup I_{s}^{\ell}}(c+\Delta\epsilon
a_{\ell})\,\mathrm{d}P^{*}(\epsilon)+o_{N}(1).$

The Poisson coloring theorem implies that $\lvert I_{t}^{\ell}\cup
I_{s}^{\ell}\rvert$ is Poisson with mean

$\displaystyle\operatorname{\mathbb{E}}\Pi_{t}^{\prime}\mathbb{P}\\{{G^{\prime}}_{1}^{y}=1,y_{1}=a_{\ell}\\}+\operatorname{\mathbb{E}}\Pi_{s}^{\prime}\mathbb{P}\\{{G^{\prime}}_{1}^{x}=1,x_{1}=a_{\ell}\\}$
$\displaystyle=sp_{\ell}(c+\Delta\epsilon^{*}a_{\ell})+tq_{\ell}(c+\Delta\epsilon^{*}a_{\ell})+o_{N}(1)$
$\displaystyle=(c+\Delta\epsilon^{*}a_{\ell})(\mu+t\nu)(a_{\ell})+o_{N}(1).$

It follows that

$\displaystyle A_{N}^{1,1}$
$\displaystyle=\operatorname{\mathbb{E}}\log\int_{\Sigma_{1}}\exp(-\Delta\epsilon(t\overline{m}+s\operatorname{\mathbb{E}}x_{1}))\prod_{x\in\Pi_{*}(\mu+t\nu)}(c+\Delta\epsilon
x)\,\mathrm{d}P^{*}(\epsilon)+o_{N}(1)$
$\displaystyle=\psi(\mu+t\nu)+c(t+s)+o_{N}(1),$

where $\Pi_{*}(\mu+t\nu)$ is a Poisson point process with mean measure
$(c+\Delta\epsilon^{*}x)\,\mathrm{d}(\mu+t\nu)(x)$. Extending this result to
the case when $\mu$ and $\nu$ are arbitrary can be done through a continuity
argument leveraging Proposition 2.1. Combining this with (4.26) establishes
the first equality in (4.25).
_Step 2: limit of $A_{N}^{2}$._ The modified Hamiltonian (4.6) corresponds to
a problem in statistical inference, so the Nishimori identity (2.14) implies
that

$A_{N}^{2}=\operatorname{\mathbb{E}}\log\langle\exp
y_{N}(\sigma^{1})\rangle^{\prime}$

for the cavity field

$\displaystyle
y_{N}(\sigma^{1}):=\sum_{k\leq\Pi_{t}}\log\bigg{[}\big{(}c+\Delta\sigma^{1}_{\ell_{k}}\sigma^{1}_{m_{k}}\big{)}^{{G^{\prime\prime}}^{k}_{\ell_{k},m_{k}}}\Big{(}1-\frac{c+\Delta\sigma^{1}_{\ell_{k}}\sigma^{1}_{m_{k}}}{N(N+1)}\Big{)}^{1-{G^{\prime\prime}}^{k}_{\ell_{k},m_{k}}}\bigg{]},$

where, through a slight abuse of notation, the family of i.i.d. random
matrices $\smash{({G^{\prime\prime}}^{k})_{k\geq 1}}$ each has conditionally
independent entries $\smash{({G^{\prime\prime}}^{k}_{i,j})_{i,j\leq N}}$
taking values in $\\{0,1\\}$ with conditional distribution

$\mathbb{P}\big{\\{}{G^{\prime\prime}}^{k}_{i,j}=1\mid\sigma^{2}\big{\\}}:=\frac{c+\sigma_{\ell_{k}}^{2}\sigma^{2}_{m_{k}}}{N(N+1)}.$

The Poisson coloring theorem implies that the number of terms in the sum
defining $y_{N}(\sigma^{1})$ that have
$\smash{{G^{\prime\prime}}^{k}_{\ell_{k},m_{k}}}$ equal to zero is Poisson
with mean

$t\binom{N}{2}\cdot\bigg{(}1-\frac{c+\Delta\sigma_{\ell_{k}}^{2}\sigma_{m_{k}}^{2}}{N(N+1)}\bigg{)}=\frac{t}{2}N^{2}+\operatorname{\operatorname{\mathcal{O}}}(1).$

Together with a Taylor expansion of the logarithm and the law of large
numbers, this implies that the contribution of the terms in the sum defining
$y_{N}(\sigma^{1})$ that have $\smash{{G^{\prime}}^{k}_{\ell_{k}}}$ equal to
zero is

$-\sum_{k\leq
tN^{2}/2}\frac{c+\Delta\sigma^{1}_{\ell_{k}}\sigma^{1}_{m_{k}}}{N(N+1)}+\operatorname{\operatorname{\mathcal{O}}}\big{(}N^{-1}\big{)}=-\frac{t}{2}\big{(}c+\Delta\overline{m}^{2}\big{)}+\operatorname{\operatorname{\mathcal{O}}}\big{(}N^{-1}\big{)}.$

If we define

$y_{N}^{\prime}(\sigma^{1}):=\sum_{k\leq\Pi_{t}}\log(c+\Delta\sigma^{1}_{\ell_{k}}\sigma^{1}_{m_{k}})^{{G^{\prime\prime}}^{k}_{\ell_{k},m_{k}}},$

then this implies that

$A_{N}^{2}=\operatorname{\mathbb{E}}\log\langle\exp
y_{N}^{\prime}(\sigma^{1})\rangle^{\prime}-\frac{t}{2}\big{(}c-\Delta\overline{m}^{2}\big{)}+\operatorname{\operatorname{\mathcal{O}}}\big{(}N^{-1}\big{)}.$
(4.28)

We now focus on the first term in this expression which we denote by
$A_{N}^{2,1}$. A similar argument as that in Step 1 shows that it is possible
to take the limit of the spin array while leaving the finite-volume Poisson
sum unaffected. More precisely, it is possible to show that

$A_{N}^{2,1}=\operatorname{\mathbb{E}}\sum_{k\leq\Pi_{t}}\log(c+\Delta
y_{1,k}y_{2,k})^{{G^{\prime\prime}}^{y}_{k}}+o_{N}(1),$

where $(y_{1,k})_{k\geq 1}$ and $(y_{2,k})_{k\geq 1}$ are independent samples
from the measure $\nu\in\mathcal{M}_{p}$ which generates the asymptotic spin
array according to (S1)-(S2), and $({G^{\prime\prime}}^{y}_{k})_{k\geq 1}$ has
conditional distribution

$\mathbb{P}\big{\\{}{G^{\prime\prime}}^{y}_{k}=1\mid
y_{1,k},y_{2,k}\big{\\}}:=\frac{c+\Delta y_{1,k}y_{2,k}}{N(N+1)}.$

Averaging with respect to the randomness of
$({G^{\prime\prime}}^{y}_{k})_{k\geq 1}$, and using the independence of
$(y_{1,k})_{k\geq 1}$ and $(y_{2,k})_{k\geq 1}$ reveals that

$\displaystyle A_{N}^{2,1}$
$\displaystyle=\frac{1}{N(N+1)}\operatorname{\mathbb{E}}\sum_{k\leq\Pi_{t}}(c+\Delta
y_{1,k}y_{2,k})\log(c+\Delta y_{1,k}y_{2,k})+o_{N}(1)$
$\displaystyle=\frac{t}{2}\operatorname{\mathbb{E}}(c+\Delta
y_{1}y_{2})\log(c+\Delta y_{1}y_{2})+o_{N}(1).$

Combining this with (4.28) and recalling the definition of the function
$G_{\nu}:[-1,1]\to\mathbb{R}$ in (1.25) gives

$\lim_{N\to+\infty}A_{N}^{2}=\frac{t}{2}\Big{(}\operatorname{\mathbb{E}}(c+\Delta
y_{1}y_{2})\log(c+\Delta
y_{1}y_{2})-\Delta\overline{m}^{2}-c\Big{)}=\frac{t}{2}\int_{-1}^{1}G_{\nu}(y)\,\mathrm{d}\nu(y).$

This completes the proof. ∎

This result can now be combined with the cavity representation in Lemma 4.1 to
bound the asymptotic free energy from above and from below by the functional
(1.26) evaluated at two possibly different measures
$\nu^{\pm}\in\mathcal{M}_{p}$ which generate the asymptotic spin array
associated with two sets of perturbation parameters according to (S1)-(S2).
This result will be improved in the next section.

######  Proposition 4.3.

For all $(t,\mu)\in\mathbb{R}_{\geq 0}\times\mathcal{M}_{+}$ there are
sequences $\smash{\big{(}N_{k}^{\pm},\lambda^{N_{k}^{\pm}}\big{)}_{k\geq 1}}$
and measures $\nu^{\pm}\in\mathcal{M}_{p}$ such that the following holds. The
sequences $\smash{(N_{k}^{\pm})_{k\geq 1}}$ increase to infinity, the
multioverlap concentration (3.15) for the Hamiltonian (4.6) with perturbation
parameters $\smash{(\lambda^{N_{k}^{\pm}})_{k\geq 1}}$ holds for $n\geq 1$,
the asymptotic spin array associated with the perturbation parameters
$\smash{(\lambda^{N_{k}^{\pm}})_{k\geq 1}}$ is generated by $\nu^{\pm}$
according to (S1)-(S2), and

$\operatorname{\mathcal{P}}_{t,\mu}(\nu^{-})\leq\liminf_{N\to+\infty}\overline{F}_{N}(t,\mu)\leq\limsup_{N\to+\infty}\overline{F}_{N}(t,\mu)\leq\operatorname{\mathcal{P}}_{t,\mu}(\nu^{+}).$
(4.29)

###### Proof.

Combining Lemma 3.2 with the arguments in [43, Lemma 3.3], it is possible to
find a sequence of perturbation parameters $(\lambda^{N})_{N\geq 1}$ such that
the multioverlap concentration (3.15) for the perturbed Hamiltonian (4.6)
holds for all $n\geq 1$. Remembering that the perturbation parameters do not
affect the asymptotic behavior of the free energy by Lemma 3.1 and that the
$\liminf$ of a Cesàro sum is bounded from below by the $\liminf$ of its
general term, we see that

$\liminf_{N\to+\infty}\overline{F}_{N}(t,\mu)\geq\liminf_{N\to+\infty}A_{N}(t,\mu,\lambda^{N}).$

Invoking Lemma 4.2 and passing to a further subsequence along which
$(A_{N}(t,\mu,\lambda^{N}))_{N\geq 1}$ converges to its $\liminf$ if
necessary, gives a sequence $\smash{(N_{k}^{-},\lambda^{N_{k}^{-}})_{k\geq
1}}$ and a measure $\nu^{-}\in\mathcal{M}_{p}$ such that the asymptotic spin
array associated with the perturbation parameters
$\smash{(\lambda^{N_{k}^{-}})_{k\geq 1}}$ is generated by $\nu^{-}$ according
to (S1)-(S2) and

$\liminf_{N\to+\infty}A_{N}\big{(}t,\mu,\lambda^{N}\big{)}=\lim_{k\to+\infty}A_{N_{k}^{-}}\big{(}t,\mu,\lambda^{N_{k}^{-}}\big{)}=\operatorname{\mathcal{P}}_{t,\mu}\big{(}\nu^{-}\big{)}.$

Together with the previous display this gives the lower bound in (4.29). The
upper bound in this inequality is proved in an identical manner by instead
using that the $\limsup$ of a Cesàro sum is bounded from above by the
$\limsup$ of its general term. This completes the proof. ∎

We close this section by showing that the asymptotic behavior of the enriched
free energy (1.22) and the modified enriched free energy (4.7) are equivalent.
This will play its part in the next section when we show that whenever the
free energy converges, we can turn (4.29) into an equality.

######  Lemma 4.4.

For all $(t,\mu)\in\mathbb{R}_{\geq 0}\times\mathcal{M}_{+}$, the enriched
free energy (1.22) and the modified enriched free energy (4.7) are
asymptotically equivalent,

$\lim_{N\to+\infty}\big{\lvert}\overline{F}_{N}(t,\mu)-\overline{F}_{N}^{\prime}(t,\mu)\big{\rvert}=0.$
(4.30)

###### Proof.

Recall from (4.21) and (4.23) that the enriched Hamiltonian (1.21) and the
modified enriched Hamiltonian (4.5) differ by the cavity field $y_{N}^{t}$ in
(4.16) in the approximate sense defined in (4.18),

$H_{N}^{t,\mu}(\sigma)\stackrel{{\scriptstyle
d}}{{\approx}}\mathcal{H}_{N}^{t,\mu}(\sigma)+y_{N}^{t}(\sigma).$ (4.31)

We now leverage an interpolation argument to establish the asymptotic
equivalence (4.30). For each $u\in[0,1]$, we define the interpolating
Hamiltonian

$\mathcal{H}_{N}^{t,\mu}(u,\sigma):=\mathcal{H}_{N}^{t,\mu}(\sigma)+uy_{N}^{t}(\sigma),$

and denote by $\overline{F}_{N}(u)$ and $\langle\cdot\rangle_{u}$ its
associated free energy and Gibbs average, respectively. The fundamental
theorem of calculus implies that

$\big{\lvert}\overline{F}_{N}(1)-\overline{F}_{N}(0)\big{\rvert}\leq\sup_{u\in[0,1]}\Big{\lvert}\frac{\,\mathrm{d}}{\,\mathrm{d}u}\overline{F}_{N}(u)\Big{\rvert}\leq\frac{1}{N}\sup_{u\in[0,1]}\operatorname{\mathbb{E}}\big{\langle}\lvert
y_{N}^{t}(\sigma)\rvert\big{\rangle}_{u}.$ (4.32)

Averaging with respect to the randomness of $({G^{\prime\prime}}^{k})_{k\geq
1}$ shows that for each $u\in[0,1]$,

$\operatorname{\mathbb{E}}\big{\langle}\lvert
y_{N}^{t}(\sigma)\rvert\big{\rangle}_{u}\leq\operatorname{\mathbb{E}}\Pi_{t}\bigg{(}\frac{(c+\lvert\Delta\rvert)(\log(c+\lvert\Delta\rvert)+1)}{N(N+1)}+\operatorname{\operatorname{\mathcal{O}}}\big{(}N^{-4}\big{)}\bigg{)}=\operatorname{\operatorname{\mathcal{O}}}(1).$

Substituting this into (4.32), recalling that
$\overline{F}_{N}(1)=\overline{F}_{N}(t,\mu)+o_{N}(1)$ by the approximate
distributional identity (4.31), and observing that
$\overline{F}_{N}(0)=\overline{F}_{N}^{\prime}(t,\mu)$ completes the proof. ∎

##  5 Critical point representation

In this section we finally prove Theorems 1.1 and 1.2. We begin by showing
that whenever the limit free energy exists and is Gateaux differentiable, it
may be identified with a measure $\nu\in\mathcal{M}_{p}$ through the function
$G_{\nu}$ in (1.25). We then show that the map $\eta\mapsto G_{\eta}$ is
injective on $\mathcal{M}_{p}$, meaning that the identified measure
$\nu\in\mathcal{M}_{p}$ is unique. We also prove that this measure generates
some subsequential limit of the spin array associated with the modified and
perturbed Hamiltonian (4.6). We then argue that under these same assumptions,
the measure $\nu$ is a valid choice for both the measure $\nu^{+}$ and the
measure $\nu^{-}$ in Proposition 4.3. Through arguments similar to those in
Lemma 4.2, at any point $(t,\mu)\in\mathbb{R}_{\geq 0}\times\mathcal{M}_{+}$
of differentiability of the limit free energy, we relate the Gateaux
derivative of the limit free energy at $(t,\mu)$ to the Gateaux derivative of
the asymptotic initial condition (1.2) at $\mu+t\nu$. This leads to the
equality $G_{\nu}=D_{\mu}\psi(\mu+t\nu,\cdot)$ which we show to be equivalent
to the fixed point equation $\nu=\Gamma_{t,\mu}(\nu)$ associated with the
operator (1.6). Finally, we combine all these insights with Proposition 2.2 on
the almost everywhere Gateaux differentiability of the limit free energy to
establish Theorems 1.1 and 1.2.

To relate the Gateaux derivative of the limit free energy to that of the
initial condition, it will be convenient to recall from [18, Remark 3.2] that
for any $\mu\in\mathcal{M}_{+}$, the density of the Gateaux derivative
$D_{\mu}\psi(\mu)$ is

$D_{\mu}\psi(\mu,x)=\operatorname{\mathbb{E}}\langle c+\Delta\epsilon
x\rangle_{*}\log\langle c+\Delta\epsilon x\rangle_{*}-c-\Delta\overline{m}x,$
(5.1)

where, conditionally on the ground truth $\sigma^{*}$, the bracket
$\langle\cdot\rangle_{*}$ denotes the Gibbs average associated with the
Hamiltonian defined on $\Sigma_{1}$ by

$H^{\mu}(\epsilon):=-\int_{-1}^{1}(c+\Delta\epsilon
x)\,\mathrm{d}\mu(x)+\sum_{x\in\Pi_{*}(\mu)}\log(c+\Delta\epsilon x).$ (5.2)

Here $\Pi_{*}(\mu)$ denotes a Poisson point process on $[-1,1]$ with mean
measure $(c+\Delta\sigma^{*}x)\,\mathrm{d}\mu(x)$. This means that for any
bounded and measurable function $f=f(\epsilon^{1},\ldots,\epsilon^{n})$ of
finitely many replicas,

$\langle f(\epsilon^{1},\ldots,\epsilon^{n})\rangle_{*}:=\langle
f\rangle_{*}:=\frac{\int_{\Sigma_{1}^{n}}f(\epsilon^{1},\ldots,\epsilon^{n})\prod_{\ell\leq
n}\exp
H^{\mu}(\epsilon^{\ell})\,\mathrm{d}P^{*}(\epsilon^{\ell})}{\big{(}\int_{\Sigma_{1}}\exp
H^{\mu}(\epsilon)\,\mathrm{d}P^{*}(\epsilon)\big{)}^{n}}.$ (5.3)

It will also be convenient to remember from the proof of [18, Lemma 2.4] that
the Gateaux derivative of the enriched free energy (1.22) at
$(t,\mu)\in\mathbb{R}_{\geq 0}\times\mathcal{M}_{+}$ in the direction of a
probability measure $\eta\in\Pr[-1,1]$ is given by

$D_{\mu}\overline{F}_{N}(t,\mu;\eta)=\sum_{i\leq
N}\operatorname{\mathbb{E}}\log\bigg{\langle}\big{(}c+\Delta\sigma_{i}w_{i}\big{)}^{\widetilde{G}_{i}^{w}}\Big{(}1-\frac{c+\Delta\sigma_{i}w_{i}}{N}\Big{)}^{1-\widetilde{G}_{i}^{w}}\bigg{\rangle},$
(5.4)

where the bracket $\langle\cdot\rangle$ denotes the Gibbs average (2.13), the
$(w_{i})_{i\geq 1}$ are i.i.d. random variables sampled from $\eta$, and
$(\widetilde{G}_{i}^{w})_{i\leq N}$ are conditionally independent random
variables taking values in $\\{0,1\\}$ with conditional distribution

$\mathbb{P}\big{\\{}\widetilde{G}_{i}^{w}=1\mid\sigma^{*},w_{i}\big{\\}}:=\frac{c+\Delta\sigma_{i}^{*}w_{i}}{N}.$
(5.5)

######  Lemma 5.1.

Suppose that the sequence of enriched free energies $(\overline{F}_{N})_{N\geq
1}$ converges pointwise to some limit $f:\mathbb{R}_{\geq
0}\times\mathcal{M}_{+}\to\mathbb{R}$ along a subsequence $(N_{k})_{k\geq 1}$,
and fix $(t,\mu)\in\mathbb{R}_{\geq 0}\times\mathrm{Reg}(\mathcal{M}_{+})$. If
$f(t,\cdot)$ is Gateaux differentiable at
$\mu\in\mathrm{Reg}(\mathcal{M}_{+})$, then there exists
$\nu\in\mathcal{M}_{p}$ with

$G_{\nu}=D_{\mu}f(t,\mu,\cdot).$ (5.6)

###### Proof.

For each integer $k\geq 1$, let
$\nu_{N_{k}}:=\mathcal{L}\big{(}\langle\sigma_{1}\rangle\big{)}$ denote the
law (under $\mathbb{P}$) of the Gibbs average of a spin variable, where the
Gibbs measure is as in (2.13). The Nishimori identity (2.14), the derivative
expression (2.12), and the definition of the kernel $g$ in (1.24) imply that

$D_{\mu}\overline{F}_{N_{k}}(t,\mu,\cdot)=G_{\nu_{N_{k}}}(\cdot)+\operatorname{\operatorname{\mathcal{O}}}(N_{k}^{-1}).$
(5.7)

By the Prokhorov theorem, the sequence $(\nu_{N_{k}})_{k\geq
1}\subset\mathcal{M}_{p}$ admits a subsequential limit
$\nu\in\mathcal{M}_{p}$. Invoking Proposition 2.3 to let $k$ tend to infinity
in (5.7) completes the proof. ∎

######  Lemma 5.2.

If $\nu,\nu^{\prime}\in\mathcal{M}_{p}$ are such that
$G_{\nu}=G_{\nu^{\prime}}$, then $\nu=\nu^{\prime}$.

###### Proof.

The equality $G_{\nu}=G_{\nu^{\prime}}$ means that for all $x\in[-1,1]$,

$\displaystyle c\log(c)-c+\Delta\overline{m}x+$ $\displaystyle c\sum_{n\geq
2}\frac{(-\Delta/c)^{n}}{n(n-1)}\int_{-1}^{1}y^{n}\,\mathrm{d}\nu(y)x^{n}$
$\displaystyle=c\log(c)-c+\Delta\overline{m}x+c\sum_{n\geq
2}\frac{(-\Delta/c)^{n}}{n(n-1)}\int_{-1}^{1}y^{n}\,\mathrm{d}\nu^{\prime}(y)x^{n}.$

It follows that $\nu$ and $\nu^{\prime}$ have all the same moments of order
$n\geq 2$, and are therefore equal by the Stone-Weierstrass theorem. ∎

######  Lemma 5.3.

Suppose that the sequence of enriched free energies $(\overline{F}_{N})_{N\geq
1}$ converges pointwise to some limit $f:\mathbb{R}_{\geq
0}\times\mathcal{M}_{+}\to\mathbb{R}$ along a subsequence $(N_{k})_{k\geq 1}$.
If $(t,\mu)\in\mathbb{R}_{\geq 0}\times\mathrm{Reg}(\mathcal{M}_{+})$ is such
that $f(t,\cdot)$ is Gateaux differentiable at $\mu$ and
$\nu\in\mathcal{M}_{p}$ is such that $G_{\nu}=D_{\mu}f(t,\mu,\cdot)$ (which
exists by Lemma 5.1), then there is a sequence
$\smash{\big{(}N_{k}^{\prime},\lambda^{N_{k}^{\prime}}\big{)}_{k\geq 1}}$ such
that the following holds. The sequence $\smash{(N_{k}^{\prime})_{k\geq
1}}\subset(N_{k})_{k\geq 1}$ increases to infinity, the multioverlap
concentration (3.15) for the Hamiltonian (4.6) with perturbation parameters
$\smash{(\lambda^{N_{k}^{\prime}})_{k\geq 1}}$ holds for every $n\geq 1$, and