# On the TAP equations via the cavity approach
in the generic mixed $p$-spin models

Wei-Kuo Chen University of Minnesota, Minneapolis, USA. Email<EMAIL_ADDRESS>Si Tang Lehigh University, Bethlehem, USA. Email<EMAIL_ADDRESS>
###### Abstract

In 1977, Thouless, Anderson, and Palmer (TAP) derived a system of consistent
equations in terms of the effective magnetization in order to study the free
energy in the Sherrington-Kirkpatrick (SK) spin glass model. The solutions to
their equations were predicted to contain vital information about the
landscapes in the SK Hamiltonian and the TAP free energy and moreover have
direct connections to Parisi’s replica ansatz. In this work, we aim to
investigate the validity of the TAP equations in the generic mixed $p$-spin
model. By utilizing the ultrametricity of the overlaps, we show that the TAP
equations are asymptotically satisfied by the conditional local magnetizations
on the asymptotic pure states.

##  1 Introduction

The study of mean-field spin glasses has been one of the central objectives in
Statistical Physics over the past decades. Based on the replica method, this
approach has attained great achievements following Parisi’s celebrated ansatz
[28, 29, 30] for the famous Sherrington-Kirkpatrick (SK) model [31] as well as
its variants, see physics literature in [19] and recent mathematical
development in [26, 36, 37]. In a different direction, Thouless-Anderson-
Palmer [38] proposed an approach to investigating the free energy in the SK
model by diagramatically expanding the free energy with respect to an
effective magnetization and arrived at a new variational expression in terms
of the TAP free energy, which involves a novel correlation energy of the spin
fluctuations in addition to maintaining the spirit of the Gibbs variational
principle and the mean-field approximation. From the first order optimality of
the TAP free energy, they deduced a system of self-consistent equations, known
as the TAP equations, where the solutions were predicted to contain crucial
information about the landscapes of the TAP free energy as well as the SK
Hamiltonian and have strong connections with Parisi’s replica ansatz, see
[19].

In this paper, we aim to present an elementary derivation for the TAP
equations in the generic mixed $p$-spin model and show that they are
asymptotically satisfied by the local magnetization associated to the
asymptotic pure states. Let $\beta=(\beta_{p})_{p\geq 2}$ be a real sequence
with $C_{\beta}:=\sum_{p=2}^{\infty}2^{p}\beta_{p}^{2}<\infty$ and let $h>0$
be fixed. For any $N\geq 1$ and $p\geq 2,$ denote by $\mathcal{I}_{N,p}$ the
collection of all index vectors $(i_{1},\ldots,i_{p})\in\\{1,\ldots,N\\}^{p}$
of distinct entries. Let $g_{i_{1},\ldots,i_{p}}$ be i.i.d. standard Gaussian
for all $(i_{1},\ldots,i_{p})\in\mathcal{I}_{N,p}$, $N\geq 1$, and $p\geq 2.$
The Hamiltonian of the mixed $p$-spin model with (inverse) temperatures
$(\beta_{p})_{p\geq 2}$ and an external field $h$ is defined as

$\displaystyle\mathcal{H}_{N}(\sigma)=H_{N}(\sigma)+h\sum_{i=1}^{N}\sigma_{i}$
(1)

for $\sigma=(\sigma_{1},\ldots,\sigma_{N})\in\Sigma_{N}$, where

$\displaystyle{H}_{N}(\sigma):=\sum_{p\geq
2}\frac{\beta_{p}}{N^{\frac{p-1}{2}}}\sum_{(i_{1},\ldots,i_{p})\in\mathcal{I}_{N,p}}g_{i_{1},\ldots,i_{p}}\sigma_{i_{1}}\cdots\sigma_{i_{p}}.$

Note that when $\beta_{p}=0$ for all $p\neq 2$ and $\beta_{2}\neq 0,$ the
mixed $p$-spin model recovers the famous SK model. Using the independence
among $g_{i_{1},\ldots,i_{p}}$, it can be computed directly that the
covariance of $H_{N}$ is essentially determined by the function
$\zeta(s):=\sum_{p\geq 2}\beta^{2}_{p}s^{p}$ and the overlap
$R(\sigma^{1},\sigma^{2}):=N^{-1}\sum_{i=1}^{N}\sigma_{i}^{1}\sigma_{i}^{2}$,
namely,

$\Bigl{|}\mathbb{E}H_{N}(\sigma^{1})H_{N}(\sigma^{2})-N\zeta\bigl{(}R(\sigma^{1},\sigma^{2})\bigr{)}\Bigr{|}\leq
C_{\beta},\qquad\forall\sigma^{1},\sigma^{2}\in\Sigma_{N}.$

The free energy and the Gibbs (probability) measure are defined respectively
by

$F_{N}=\frac{1}{N}\ln Z_{N}\ \ \mbox{and}\ \
G_{N}(\sigma)=\frac{e^{\mathcal{H}_{N}(\sigma)}}{Z_{N}},$

where $Z_{N}:=\sum_{\sigma\in\Sigma_{N}}e^{\mathcal{H}_{N}(\sigma)}.$ For any
measurable function $f$ on $\mathbb{R}^{k}$ and independent samples
$\sigma^{1},\ldots,\sigma^{k}\sim G_{N}$ (also called the “replicas” in the
physics literature), we denote by $\langle
f(\sigma^{1},\ldots,\sigma^{k})\rangle$ the average under the Gibbs measure
$G_{N}$, i.e.,

$\langle
f(\sigma^{1},\ldots,\sigma^{k})\rangle=\sum_{\sigma^{1},\ldots,\sigma^{k}}f(\sigma^{1},\ldots,\sigma^{k})G_{N}(\sigma^{1})\cdots
G_{N}(\sigma^{k}).$

It is well-known that the limiting free energy $F:=\lim_{N\to\infty}F_{N}$
exists and can be expressed as the Parisi formula (see [27, 35]), a
variational representation of a nonlinear functional defined on the space of
all probability measures on the interval $[0,1]$, in which the optimizer,
called the Parisi measure $\mu_{P}$, exists and is unique, see [1].

Throughout this paper, we shall assume that the mixed $p$-spin model is
generic in the sense that the linear span of the collection of all monomials,
$t^{p},$ for those $p$ with $\beta_{p}\neq 0$ is dense in $C[0,1]$ under the
supremum norm. Under this assumption, this model possesses two important
properties. First of all, as $N\to\infty$, the limiting distribution of the
overlap $R(\sigma^{1},\sigma^{2})$ under $\mathbb{E}\langle\cdot\rangle$ is
described by the Parisi measure, see [24]. More importantly, the Gibbs measure
is asymptotically ultrametric [25], namely, for any three replicas
$\sigma^{1},\sigma^{2},\sigma^{3}$, we have

$\displaystyle\lim_{\epsilon\downarrow
0}\limsup_{N\to\infty}\mathbb{E}\big{\langle}\mathbbm{1}_{\\{R(\sigma^{1},\sigma^{2})\geq\min(R(\sigma^{1},\sigma^{3}),R(\sigma^{2},\sigma^{3}))-\epsilon\\}}\big{\rangle}=1.$
(2)

As a consequence of ultrametricity, asymptotically, it allows one to decompose
the spin configuration space $\Sigma_{N}$ into clusters and ultimately they
induce an infinitary tree structure on $\Sigma_{N}$ under the set
containments. In particular, the clusters at the bottom of the tree are called
the pure states in the physics literature. One natural way to construct
asymptotic pure states is via the $\sqrt{2(1-q_{P}+\epsilon)}$ neighborhood of
$\alpha\sim G_{N}$, that is,

$\Sigma_{N}^{\alpha}:=\\{\sigma\in\Sigma_{N}:R(\sigma,\alpha)>q_{P}-\epsilon\\},$

where $q_{P}=q_{P}(\beta)$ is the largest point in the support of $\mu_{P}$
and $\epsilon$ may be chosen to depend on $N$ as long as
$\epsilon_{N}\downarrow 0$ slowly enough. We define the conditional Gibbs
measure $G_{N}^{\alpha}$ on the asymptotic pure state $\Sigma_{N}^{\alpha}$ by

$G_{N}^{\alpha}(\sigma):=G_{N}(\sigma|\sigma\in\Sigma_{N}^{\alpha})=\frac{G_{N}(\sigma)\mathbbm{1}_{\Sigma_{N}^{\alpha}}(\sigma)}{W_{N}(\Sigma_{N}^{\alpha})},$

where $W_{N}(A):=\sum_{\sigma\in A}G_{N}(\sigma)$ is the total Gibbs measure
for any subset $A\subseteq\Sigma_{N}$. The corresponding Gibbs average on
$\Sigma_{N}^{\alpha}$ is denoted by $\langle\cdot\rangle^{\alpha}$. Set the
local magnetization within the asymptotic pure state $\Sigma^{\alpha}_{N}$ as

$\displaystyle\langle\sigma\rangle^{\alpha}:=\bigl{(}\langle\sigma_{1}\rangle^{\alpha},\ldots,\langle\sigma_{N}\rangle^{\alpha}\bigr{)}.$

For any $1\leq i\leq N,$ let $\mathcal{I}_{N,p}(i)$ be the collection of all
$(i_{1},\ldots,i_{p})\in\mathcal{I}_{N,p}$ with $i_{r}=i$ for some $1\leq
r\leq p.$ For any $1\leq i\leq N,$ define the cavity field at site $i$ as

$\displaystyle X_{N,\beta,i}(\sigma)$ $\displaystyle=\sum_{p\geq
2}\frac{\beta_{p}}{N^{\frac{p-1}{2}}}\sum_{(i_{1},\ldots,i_{p})\in\mathcal{I}_{N,p}(i)}g_{i_{1},\ldots,i_{p}}\prod_{s=1:i_{s}\neq
i}^{p}\sigma_{i_{s}},\quad\forall\sigma\in\Sigma_{N}.$

Note that $X_{N,\beta,i}(\sigma)$ depends on all except the $i$-th spin.
Denote by $\|x\|=\bigl{(}N^{-1}\sum_{i=1}^{N}|x_{i}|^{2}\bigr{)}^{1/2}$ for
any $x\in\mathbb{R}^{N}$. Our main result is stated as follows:

######  Theorem 1.1 (TAP equations).

Assume that the model is generic. Then the conditional local magnetization
$\langle\sigma\rangle^{\alpha}$ satisfies that

$\displaystyle\begin{split}&\lim_{\epsilon\downarrow
0}\limsup_{N\to\infty}\mathbb{E}\Bigl{\langle}\Bigl{|}\langle\sigma_{N}\rangle^{\alpha}-\tanh\Bigl{(}X_{N,\beta,N}(\langle\sigma\rangle^{\alpha})+h-\zeta^{\prime\prime}\bigl{(}\|\langle\sigma\rangle^{\alpha}\|^{2}\bigr{)}\bigl{(}1-\|\langle\sigma\rangle^{\alpha}\|^{2}\bigr{)}\langle\sigma_{N}\rangle^{\alpha}\Bigr{)}\Bigr{|}^{2}\Bigr{\rangle}=0,\end{split}$
(3)

where $\epsilon\downarrow 0$ along a sequence such that $q_{P}-\epsilon$ is
always a point of continuity for $\mu_{P}$.

By symmetry, the expectation in (LABEL:eqn:TAP) is the same for any spin site.
Hence, as long as $N$ is large enough, (LABEL:eqn:TAP) implies that the local
magnetization $\langle\sigma\rangle^{\alpha}$ satisfies the following
asymptotic consistency equations on average over all spin sites $1\leq i\leq
N,$

$\displaystyle\langle\sigma_{i}\rangle^{\alpha}\approx\tanh\Bigl{(}X_{N,\beta,i}(\langle\sigma\rangle^{\alpha})+h-\zeta^{\prime\prime}\bigl{(}\|\langle\sigma\rangle^{\alpha}\|^{2}\bigr{)}\bigl{(}1-\|\langle\sigma\rangle^{\alpha}\|^{2}\bigr{)}\langle\sigma_{i}\rangle^{\alpha}\Bigr{)}.$
(4)

These self-consistent equations are the TAP equations. The term
$\zeta^{\prime\prime}\bigl{(}\|\langle\sigma\rangle^{\alpha}\|^{2}\bigr{)}\bigl{(}1-\|\langle\sigma\rangle^{\alpha}\|^{2}\bigr{)}\langle\sigma_{i}\rangle^{\alpha}$
is called the Onsager correction term, which distinguishes the disorder and
non-disorder spin systems, where the latter, taking the Curie-Weiss model for
example, does not involve an Onsager term in the mean-field equation for the
spin magnetizations, see, e.g., Equation (II.2) in [19].

The number of solutions to the TAP equations generally depends on the
temperature. In the high temperature regime, i.e., $\zeta(1)$ is sufficiently
small, it can be shown that the overlap $R(\sigma^{1},\sigma^{2})$ between two
independent samples $\sigma^{1},\sigma^{2}\sim G_{N}$ is concentrated around
$q_{P}$ (see, e.g., [36]). Consequently, the pure state $\Sigma_{N}^{\alpha}$
associated to $\alpha\sim G_{N}$ is almost $\Sigma_{N}$ and
$W_{N}(\Sigma_{N}^{\alpha})$ is approximately one. As a result, we see that
$\langle\sigma\rangle^{\alpha}\approx\langle\sigma\rangle$ and
$\langle\sigma\rangle$ satisfies the TAP equations (4) in the generic mixed
$p$-spin model at high temperature. This is similar to the one in the SK model
established by Chatterjee [10] and Talagrand [36]111Talagrand’s result asserts
that the $N$ TAP equations asymptotically hold simultaneously with high
probability, while we establish the TAP equations in the average sense..

In contrast to the high temperature case, it is expected that the TAP
equations should have exponentially many solutions in $N$ in the low
temperature regime. Although an argument for this remains missing in the
literature, it can be argued that the TAP equations could have multiple
solutions by making use of the ultrametricity (2). In fact, it is well-known
that the Parisi measure $\mu_{P}$ possesses a nontrivial distribution. Pick
$q\in[0,q_{P})$ in the support of $\mu_{P}.$ Note that in the generic mixed
$p$-spin model, the Ghirlanda-Guerra identities are valid and they ensure that
for any fixed $k\geq 1,$ with positive probability $\mathbb{E}G_{N}^{\otimes
k}$, there exist distinct $\alpha^{1},\ldots,\alpha^{k}$ with
$R(\alpha^{\ell},\alpha^{\ell^{\prime}})\approx q$ for all
$1\leq\ell<\ell^{\prime}\leq k$ (see, e.g., [26, Theorem 2.20]). From the
ultrametricity, it follows that
$R(\sigma^{\ell},\sigma^{\ell^{\prime}})\approx q$ for any
$\sigma^{\ell}\in\Sigma_{N}^{\alpha^{\ell}}$ and
$\sigma^{\ell^{\prime}}\in\Sigma_{N}^{\alpha^{\ell^{\prime}}}$ for any
$1\leq\ell<\ell^{\prime}\leq k,$ which implies that
$R(\langle\sigma\rangle^{\alpha^{\ell}},\langle\sigma\rangle^{\alpha^{\ell^{\prime}}})\approx
q$ for any $1\leq\ell<\ell^{\prime}\leq k.$ As we will also see in Remark 3.10
below that
$R(\langle\sigma\rangle^{\alpha^{\ell}},\langle\sigma\rangle^{\alpha^{\ell}})$
is approximately $q_{P}$, we arrive at
$\|\langle\sigma\rangle^{\alpha^{\ell}}-\langle\sigma\rangle^{\alpha^{\ell^{\prime}}}\|\approx\sqrt{2(q_{P}-q)}>0.$
Hence,
$\langle\sigma\rangle^{\alpha^{1}},\ldots,\langle\sigma\rangle^{\alpha^{k}}$
are $k$ distinct approximate solutions to the TAP equations.

Our approach for Theorem 1.1 is based on the cavity method, where the
computation utilizes (2), the ultrametricity of the overlaps. This property
allows us to partition the spin configuration space into random “clusters”
$\\{\mathcal{C}_{N,\alpha}\\}$, i.e., the pure states, where the distribution
of the weights of the pure states is characterized by the Ruelle Probability
Cascades, see, e.g., [26, 37]. More importantly, within each pure state, the
overlap of two spin configurations are asymptotically concentrated around
$q_{P}$ [16, 25]. When approximating the pure states $\mathcal{C}_{N,\alpha}$
via $\Sigma_{N}^{\alpha},$ this concentration enables us to establish a
central limit theorem for the cavity fields within a pure state and make our
cavity computations feasible, ultimately leading to the TAP equations in
(LABEL:eqn:TAP). Overall we believe that our approach has presented a general
receipt in deriving the TAP equations for similar local magnetizations in
related spin glass models.

###  1.1 Related works

The mathematical establishment of the TAP equations appeared firstly in
Talagrand’s book [36] and Chatterjee [10], where they showed that the spin
magnetization asymptotically satisfies the TAP equations in the SK model at
sufficiently high temperature. Later Bolthausen [7] proposed an iterative
scheme and showed that his iteration would converge to one solution of the TAP
equations, provided the temperature and external field parameters lie inside
the de Almeida-Thouless phase transition line. In a joint paper, [14], the
authors showed that Bolthausen’s iteration scheme indeed converges to the spin
magnetization whenever the overlap of two independently sampled spin
configurations from the Gibbs measure is concentrated locally uniformly in
temperature.

Beyond the SK model, some progress has also been made to establish the TAP
equations in the mixed $p$-spin model. First, Auffinger-Jagannath [2, 3]
considered the generic mixed $p$-spin model and used the ultrametric
decomposition to construct a local magnetization (not the same as our
construction) that satisfies a set of equations discovered by Mézard-Virasoro
[20], different from the TAP equations (4) we derive here. Their equations in
particular reduce to the classical TAP equation in the case of the SK model.
In [11], Chen-Panchenko established a variational principal for the free
energy in terms of the classical TAP free energy and showed that the local
magnetization $\langle\sigma\rangle^{\alpha}$ is asymptotically a TAP state,
i.e., an optimizer of the TAP free energy. Later in [12], Chen-Panchenko-Subag
derived a general framework for the TAP free energy in the mixed $p$-spin
model and from the first order optimality, it was argued that every TAP state
must satisfy the generalized TAP equations. Combining the result in [11] and
the discussion of [12, Remark 6] together, it is then expected that the
classical TAP equations, (4), should be satisfied by our local magnetization
$\langle\sigma\rangle^{\alpha}$ – Theorem 1.1 provides a justification to this
prediction.

In relation to the TAP equations, the TAP free energies have also received
great attention in recent years. By utilizing Parisi’s ansatz, the works of
Chen-Panchenko [11] and Chen-Panchenko-Subag [12, 13] established the TAP free
energy in the Ising mixed $p$-spin model as well as its generalization to the
zero temperature setting. In the spherical case, Subag established the TAP
free energy in the mixed $p$-spin model [32] and in the multi-species model
[33, 34] by introducing the multisamplable overlap property. Independently, by
means of a geometric microcanonical method, the TAP free energy involving the
Plefka condition was implemented in the spherical SK model by Belius-Kistler
[6] and an upper bound for the free energy in terms of the TAP free energy was
also obtained in the mixed $p$-spin model with general spins by Belius [5].

###  1.2 Further results related to the TAP equations

The TAP equations have many important consequences in the study of mean-field
spin glass models and related applications. First of all, based on his
iteration, Bolthausen [8] performed a conditional second moment method to
derive the replica symmetry formula for the limiting free energy in the SK
model at very high temperature. In a follow-up work, Brennecke-Yau [9]
provided a simplified argument for Bolthausen’s approach and extended the
replica symmetry formula to a larger regime. In addition, the TAP equations
have played a key role in some optimization problems in spin glasses and
statistical inference problems. Most importantly, they naturally give rise to
the so-called Approximate Message Passing (AMP) algorithms based on
Bolthausen’s iteration scheme [7]; several generalizations of the AMP
algorithms can be found in Bayati-Montanari [4] and Javanmard-Montanari [17].
By using the AMP algorithms, Montanari [21] constructed a polynomial-time
random algorithms to produce a near ground state for the SK Hamiltonian under
the assumption that the Parisi measure is full replica symmetry breaking. The
same construction was also carried out in the mixed $p$-spin model by El
Alaoui-Montanari-Sellke [15]. In the context of Bayesian inferences, various
AMPs driven by the TAP equations have also been popularly used, see, e.g.,
[18, 22, 23, 39].

###  1.3 Organization of the paper

In Section 2, we provide a sketch of our proof beginning with the settlement
of some standard notations. Section 3 will be devoted to establishing a cavity
equation for the local magnetization $\langle\sigma\rangle^{\alpha}$ based on
a univariate central limit theorem for the cavity field, see Theorem 3.7
below. In Section 4, we continue to perform some cavity computations for
$X_{N,\beta,N}(\langle\sigma\rangle^{\alpha}).$ In particular, noting that
this quantity is an infinite series, we shall establish some quantitative
bounds for its truncation and moment. Finally, the proof of Theorem 1.1 will
be presented in Section 5 by translating the cavity equation in Section 3 into
the TAP equation. This part of the argument will rely on a multivariate
central limit theorem for multiple cavity fields, a central ingredient
throughout the entire paper.

Acknowledgments. W.-K. Chen’s research is partly supported by NSF grant
(DMS-1752184). S. Tang’s research is partly supported by the Collaboration
Grant (#712728) from the Simons Foundation and the LEAPS-MPS Award
(DMS-2137614) from NSF.

##  2 Proof Sketch

Our approach adapts a similar route as [36, Theorem 1.7.7], where the TAP
equations in the SK model were established at very high temperature relying on
the fact that the overlap $R(\sigma^{1},\sigma^{2})$ between two replicas
$\sigma^{1},\sigma^{2}\sim G_{N}$ is concentrated around a deterministic
constant. However, this concentration is no longer valid in the low
temperature regime. Our proof utilizes, instead, a conditional concentration
property of the overlap, deduced from the ultrametricity (2), that the overlap
between two independently sampled spin configurations from the conditional
Gibbs measure on an approximate pure state is concentrated around $q_{P}.$ In
this section, we elaborate some key steps to summarize the main ideas of our
arguments. We believe that our approach is potentially applicable to establish
the TAP equations in many other mean-field spin glass models. First, we settle
down our standard notations.

###  2.1 Notations

######  Notation 2.1.

Recall that $\mathcal{H}_{N},$ $H_{N}$, $G_{N}$, $Z_{N},$
$\langle\cdot\rangle,$ $\langle\cdot\rangle^{\alpha},$ $W_{N}$, and
$\Sigma_{N}^{\alpha}$ are all dependent on the temperature parameter
$\beta=(\beta_{p})_{p\geq 2}$ and the external field $h$. For the rest of this
paper, we will always keep $h$ fixed and express these quantities as
$\mathcal{H}_{N,\beta},$ $H_{N,\beta},$ $G_{N,\beta},$ $Z_{N,\beta},$
$\langle\cdot\rangle_{N,\beta},$ $\langle\cdot\rangle_{N,\beta}^{\alpha}$,
$W_{N,\beta}$, and $\Sigma_{N,\beta}^{\alpha},$ respectively, to emphasize
their dependence on both $N$ and $\beta.$

######  Notation 2.2.

In this paper, we shall always work with the cavity field associated to the
last spin, $X_{N,\beta,N}.$ Due to the symmetry in the spin sites in the
Hamiltonian $\mathcal{H}_{N,\beta}$, this cavity field can be simplified as

$X_{N,\beta,N}(\sigma)=\sum_{p\geq
2}\frac{\beta_{p}\sqrt{p!}}{N^{\frac{p-2}{2}}}\sum_{1\leq
i_{1}<i_{2}<\cdots<i_{p-1}\leq
N-1}g_{i_{1},\ldots,i_{p-1},N}\sigma_{i_{1}}\cdots\sigma_{i_{p-1}}.$

Since it is indeed a function that does not depend on the last spin
coordinate, for notational simplicity, no matter $x\in[-1,1]^{N-1}$ or
$x\in[-1,1]^{N}$, we will simply denote it by $X_{N,\beta}(x)$, that is,

$\displaystyle\begin{split}X_{N,\beta}(x)&=X_{N,\beta}(x_{1},\ldots,x_{N-1})\\\
&:=\sum_{p\geq 2}\frac{\beta_{p}\sqrt{p!}}{N^{\frac{p-1}{2}}}\sum_{1\leq
i_{1}<\cdots<i_{p-1}\leq N-1}g_{i_{1},\ldots,i_{p-1},N}x_{i_{1}}\cdots
x_{i_{p-1}}\end{split}$ (5)

for $x=(x_{1},\ldots,x_{N-1},x_{N})\in[-1,1]^{N}.$

######  Notation 2.3.

For any $\alpha\sim G_{N,\beta}$, the notations $\rho$ and $\alpha_{N}$ stand
for the first $N-1$ spins and the last spin of $\alpha$, respectively. When
there is no ambiguity, we will generally use
$\tau,\tau^{1},\tau^{2},\ldots\in\Sigma_{N-1}$ to denote the spin
configurations that are sampled independently from $G_{N-1,\beta^{\prime}}$ or
$G_{N-1,\beta^{\prime}}^{\rho}$ for any
$\beta^{\prime}=(\beta_{p}^{\prime})_{p\geq 2}$ satisfying that $\sum_{p\geq
2}2^{p}{\beta_{p}^{\prime}}^{2}<\infty.$

######  Notation 2.4.

For any two vectors $x,y\in\mathbb{R}^{M}$ for some $M\geq 1,$ we define
$R(x,y)$ as the averaged inner product between $x$ and $y$ and $\|x\|$ as the
averaged $\ell_{2}$-norm of $x$ respectively by

$R(x,y)=\frac{1}{M}\sum_{i=1}^{M}x_{i}y_{i}\quad\mbox{and}\quad\|x\|=R(x,x)^{1/2}.$

For any two sequences $(a_{N})_{N\geq 1}$ and $(b_{N})_{N\geq 1}$ of real
numbers or random variables, we say that $a_{N}=b_{N}+o_{N}(1)$ (or $a_{N}\leq
b_{N}+o_{N}(1)$) if $|a_{N}-b_{N}|\leq c_{N}$ (or $a_{N}\leq b_{N}+c_{N}$) for
all $N\geq 1$, where $(c_{N})_{N\geq 1}$ is some deterministic positive
sequence that converges to $0$ as $N\to\infty$.

###  2.2 Proof sketch of Theorem 1.1

First of all, by making use of the notations in Subsection 2.1, we rewrite
(LABEL:eqn:TAP) as

$\displaystyle\lim_{\epsilon\downarrow
0}\limsup_{N\to\infty}\mathbb{E}\Bigl{\langle}\Big{[}\langle\sigma_{N}\rangle_{N,\beta}^{\alpha}-\tanh$
$\displaystyle\Big{(}X_{N,\beta}(\langle\sigma\rangle_{N,\beta}^{\alpha})+h$
$\displaystyle-\zeta^{\prime\prime}(\|\langle\sigma\rangle_{N,\beta}^{\alpha}\|^{2})(1-\|\langle\sigma\rangle_{N,\beta}^{\alpha}\|^{2})\langle\sigma_{N}\rangle_{N,\beta}^{\alpha}\Big{)}\Big{]}^{2}\Bigr{\rangle}_{N,\beta}=0.$

On the other hand, from the ultrametricity (2), it can be deduced (see Remark
3.10) that

$\displaystyle\lim_{N\to\infty}\mathbb{E}\bigl{\langle}\big{|}\|\langle\sigma\rangle_{N,\beta}^{\alpha}\|^{2}-q_{P}\bigr{|}^{2}\bigr{\rangle}_{N,\beta}=0.$

As a result, we only need to show that

$\displaystyle\lim_{\epsilon\downarrow
0}\limsup_{N\to\infty}\mathbb{E}\Bigl{\langle}\Big{[}\langle\sigma_{N}\rangle_{N,\beta}^{\alpha}-\tanh\Big{(}X_{N,\beta}(\langle\sigma\rangle_{N,\beta}^{\alpha})+h-\zeta^{\prime\prime}(q_{P})(1-q_{P})\langle\sigma_{N}\rangle_{N,\beta}^{\alpha}\Big{)}\Big{]}^{2}\Bigr{\rangle}_{N,\beta}=0.$
(6)

There are three key steps in our argument:

Step 1: Cavity computation. In Section 3, we decompose the Hamiltonian (1)
into two components. The first involves only the interactions among the first
$N-1$ coordinates and the second gathers the interactions with the $N$-th
spin. More precisely, if we fix the last spin as the cavity coordinate, then
for $\sigma=(\tau,\sigma_{N})\in\Sigma_{N-1}\times\Sigma_{1},$ we can rewrite

$\displaystyle\mathcal{H}_{N,\beta}(\sigma)=\mathcal{H}_{N-1,\beta^{\prime}}(\tau)+\sigma_{N}\bigl{(}X_{N,\beta}(\tau)+h\bigr{)},$

where $\beta^{\prime}=(\beta_{p}^{\prime})_{p\geq 2}$ is a modification of
$\beta$ defined as $\beta_{p}^{\prime}=\beta_{p}((N-1)/N)^{(p-1)/2}$ for each
$p\geq 2$. In this way, we can express
$\langle\sigma_{N}\rangle_{N,\beta}^{\alpha}$ as a fraction of averages with
respect to the Gibbs measure associated to the reduced Hamiltonian
$\mathcal{H}_{N-1,\beta^{\prime}}$ in terms of two constraints,
$\mathbbm{1}_{\Sigma_{N}^{\alpha}}\bigl{(}(\tau,1)\bigr{)}$ and
$\mathbbm{1}_{\Sigma_{N}^{\alpha}}\bigl{(}(\tau,-1)\bigr{)}$. We show that
when writing $\alpha=(\rho,\alpha_{N})$, both of them are approximately
$\mathbbm{1}_{\Sigma_{N-1}^{\rho}}(\tau)$ due to the fact that
$R(\tau,\rho)\approx R((\tau,\pm 1),\alpha)$. This will result in

$\displaystyle\langle\sigma_{N}\rangle_{N,\beta}^{\alpha}\approx\frac{\langle\sinh(X_{N,\beta}(\tau)+h)\rangle_{N-1,\beta^{\prime}}^{\rho}}{\langle\cosh(X_{N,\beta}(\tau)+h)\rangle_{N-1,\beta^{\prime}}^{\rho}}.$
(7)

Step 2: Central Limit Theorem of cavity fields. We notice that for any
$\rho\in\Sigma_{N-1}$ fixed, the disorders, $g_{i_{1},\ldots,i_{p-1},N}$, in
$X_{N,\beta}$ are independent of the conditional Gibbs measure
$G_{N-1,\beta^{\prime}}^{\rho}$ and furthermore, for independent samples
$\tau^{1},\tau^{2}\sim G_{N-1,\beta^{\prime}}^{\rho}$, the overlap
$R(\tau^{1},\tau^{2})$ is concentrated around $q_{P}$, due to ultrametricity
(2). These allow us to show (see Theorem 3.7) that the first term in the
following representation

$X_{N,\beta}(\tau)=\bigl{(}X_{N,\beta}(\tau)-\langle
X_{N,\beta}(\tau)\rangle_{N-1,\beta^{\prime}}^{\rho}\bigr{)}+\langle
X_{N,\beta}(\tau)\rangle_{N-1,\beta^{\prime}}^{\rho}$

is Gaussian distributed with some variance, approximately
$\zeta^{\prime}(1)-\zeta^{\prime}(q_{P})$, independent of $\tau$.
Consequently, we can represent

$X_{N,\beta}(\tau)\approx
z\sqrt{\zeta^{\prime}(1)-\zeta^{\prime}(q_{P})}+\langle
X_{N,\beta}(\tau)\rangle_{N-1,\beta^{\prime}}^{\rho},\quad\text{ for }z\sim
N(0,1).$

It follows that

$\displaystyle\langle\sinh(X_{N,\beta}(\tau)+h)\rangle_{N-1,\beta^{\prime}}^{\rho}$
$\displaystyle\approx\mathbb{E}_{z}\sinh(z\sqrt{\zeta^{\prime}(1)-\zeta^{\prime}(q_{P})}+\langle
X_{N,\beta}(\tau)\rangle_{N-1,\beta^{\prime}}^{\rho}+h)$
$\displaystyle=e^{(\zeta^{\prime}(1)-\zeta^{\prime}(q_{P}))/2}\sinh(\langle
X_{N,\beta}(\tau)\rangle_{N-1,\beta^{\prime}}^{\rho}+h),$

and

$\displaystyle\langle\cosh(X_{N,\beta}(\tau)+h)\rangle_{N-1,\beta^{\prime}}^{\rho}$
$\displaystyle\approx\mathbb{E}_{z}\cosh(\sqrt{\zeta^{\prime}(1)-\zeta^{\prime}(q_{P})}z+\langle
X_{N,\beta}(\tau)\rangle_{N-1,\beta^{\prime}}^{\rho}+h)$ $\displaystyle\approx
e^{(\zeta^{\prime}(1)-\zeta^{\prime}(q_{P}))/2}\cosh(\langle
X_{N,\beta}(\tau)\rangle_{N-1,\beta^{\prime}}^{\rho}+h),$

which, combining with (7), gives
$\langle\sigma_{N}\rangle_{N,\beta}^{\alpha}\approx\tanh(\langle
X_{N,\beta}(\tau)\rangle_{N-1,\beta^{\prime}}^{\rho}+h)$. In establishing this
Central Limit Theorem, there is a subtle issue that occurs from the fact that
$\alpha\in\Sigma_{N}$ is not fixed (and neither is $\rho\in\Sigma_{N-1}$) but
sampled from $G_{N,\beta}$. Since the Gaussian disorders
$g_{i_{1},\ldots,i_{p-1},N}$ appearing in $X_{N,\beta}$ are also involved in
the Gibbs measure $G_{N,\beta}$, this introduces dependence between
$X_{N,\beta}(\tau)$ and $G_{N,\beta}$. To handle this, we perform a change of
measure for $\rho$, the first $(N-1)$ coordinates of $\alpha$ and control the
associated Radon-Nykodym derivative, which allows us to prove a version of
Central Limit Theorem for the Gibbs measure conditioned on the asymptotic pure
states. This is done in Section 3.2, where the ultrametricity of the Gibbs
measure plays a key role, ensuring that the overlap between two independently
sampled spin configurations from the conditional Gibbs measure
$G_{N-1,\beta^{\prime}}^{\rho}$ on the asymptotic pure state
$\Sigma_{N-1}^{\rho}$ is approximately concentrated around $q_{P}.$ This is
analogous to the concentration of the overlap $R(\sigma^{1},\sigma^{2})$ in
the SK model at high temperature.

Step 3: Producing the Onsager correction term. In Section 5, we relate
$\langle X_{N,\beta}(\tau)\rangle_{N-1,\beta^{\prime}}^{\rho}$ with
$X_{N,\beta}(\langle\sigma\rangle_{N,\beta}^{\alpha})$ and show that their
difference is exactly the Onsager correction term
$\zeta^{\prime\prime}(q_{P})(1-q_{P})\langle\sigma_{N}\rangle_{N,\beta}^{\alpha}$.
Due to the infinite sum over $p\geq 2$ and the product
$\tau_{i_{1}}\cdots\tau_{i_{p-1}}$ in $X_{N,\beta}(\tau)$, handling the
mixed-$p$ spin model is much more involved than the SK model. Here, we will
first truncate the infinite sum over $p\geq 2$ and approximate $\langle
X_{N,\beta}(\tau)\rangle_{N-1,\beta^{\prime}}^{\rho}$ and
$X_{N,\beta}(\langle\sigma\rangle_{N,\beta}^{\alpha})$, respectively, by
finite sums of sufficiently many terms; this requires controlling the moments
of both quantities. With such truncation, we are able to estimate the
difference term-by-term for each $p\geq 2$ fixed. More precisely, by
definition (5), each term in
$X_{N,\beta}(\langle\sigma\rangle_{N,\beta}^{\alpha})$ takes the form

$\displaystyle\frac{\beta_{p}\sqrt{p!}}{N^{(p-1)/2}}\sum_{1\leq
i_{1}<\cdots<i_{p-1}\leq
N-1}g_{i_{1},\ldots,i_{p-1},N}\langle\sigma_{i_{1}}\rangle_{N,\beta}^{\alpha}\cdots\langle\sigma_{i_{p-1}}\rangle_{N,\beta}^{\alpha}.$
(8)

We will follow an argument similar to that in our first step to write each
conditioned local magnetization in (8) as

$\langle\sigma_{i}\rangle_{N,\beta}^{\alpha}\approx\frac{\langle\tau_{i}\cosh(X_{N,\beta}(\tau)+h)\rangle_{N-1,\beta^{\prime}}^{\rho}}{\langle\cosh(X_{N,\beta}(\tau)+h)\rangle_{N-1,\beta^{\prime}}^{\rho}},$

and then use replicas to represent (8) as

$\displaystyle\frac{\big{\langle}\bigl{(}\sum_{1\leq i_{1}<\ldots<i_{p-1}\leq
N-1}\frac{\beta_{p}\sqrt{p!}}{N^{(p-1)/2}}g_{i_{1},\ldots,i_{p-1},N}\tau_{i_{1}}^{1}\cdots\tau_{i_{p-1}}^{p-1}\bigr{)}\,\prod_{\ell=1}^{p-1}\cosh(X_{N,\beta}(\tau^{\ell})+h)\big{\rangle}_{N-1,\beta^{\prime}}^{\rho}}{\big{\langle}\prod_{\ell=1}^{p-1}\cosh(X_{N,\beta}(\tau^{\ell})+h)\big{\rangle}_{N-1.\beta^{\prime}}^{\rho}}.$
(9)

To proceed, a multivariate Central Limit Theorem (see Theorem 5.1) is needed
to handle the asymptotic behavior of the jointly Gaussian-distributed random
variables in (9), namely,

$\displaystyle\sum_{1\leq i_{1}<\cdots<i_{p-1}\leq
N-1}\frac{\beta_{p}\sqrt{p!}}{N^{(p-1)/2}}g_{i_{1}\cdots
i_{p-1},N}\tau_{i_{1}}^{1}\cdots\tau_{i_{p-1}}^{p-1},\ \
X_{N,\beta}(\tau^{1}),\ \ldots,\ X_{N,\beta}(\tau^{p-1}),$ (10)

which will allow us to handle the numerator and denominator in (9) separately
as in Step 2 so that (9) is essentially $\langle
X_{N,\beta}(\tau)\rangle_{N-1,\beta^{\prime}}^{\rho}$ plus an Onsager
correction term corresponding to the pure $p$-spin interaction. We emphasize
that this step is not needed for deriving the TAP equation in the SK model at
high temperature, e.g., [36, Theorem 1.7.7], because in this case, one only
needs to deal with the $p=2$ case and the collection of Gaussian random
variables in (10) reduces to a single one $X_{N,\beta}(\tau^{1})$,
consequently. A univariate central limit theorem (e.g., Theorem 3.7) would
suffice.

##  3 The Cavity equation for $\langle\sigma_{N}\rangle_{N,\beta}^{\alpha}$

We derive the cavity equation for
$\langle\sigma_{N}\rangle_{N,\beta}^{\alpha}$ in this section. As sketched in
the previous section, for any $\sigma=(\tau,\sigma_{N})\in\\{\pm
1\\}^{N-1}\times\\{\pm 1\\},$ we decompose the Hamiltonian (1) as

$\displaystyle\mathcal{H}_{N,\beta}(\sigma)=$ $\displaystyle\sum_{p\geq
2}\frac{\beta_{p}\sqrt{p!}}{N^{\frac{p-1}{2}}}\sum_{1\leq
i_{1}<\cdots<i_{p}\leq
N-1}g_{i_{1},\ldots,i_{p}}\sigma_{i_{1}}\cdots\sigma_{i_{p}}+h\sum_{i=1}^{N-1}\sigma_{i}$
$\displaystyle+\sigma_{N}\Bigl{(}\sum_{p\geq
2}\frac{\beta_{p}\sqrt{p!}}{N^{\frac{p-1}{2}}}\sum_{1\leq
i_{1}<\cdots<i_{p-1}\leq
N-1}g_{i_{1},\ldots,i_{p-1},N}\sigma_{i_{1}}\cdots\sigma_{i_{p-1}}+h\Bigr{)}$
$\displaystyle=\mathcal{H}_{N-1,\beta^{\prime}}(\tau)+\sigma_{N}\big{(}X_{N,\beta}(\tau)+h\big{)},$

where $\beta^{\prime}:=(\beta_{p}^{\prime})_{p\geq 2}$ with
$\beta^{\prime}_{p}=\beta_{p}((N-1)/N)^{(p-1)/2}$ for each $p\geq 2$ and
$X_{N,\beta}$ is defined by (5). For any $\tau\in\Sigma_{N-1}$ fixed,
$X_{N,\beta}(\tau)$ is a centered Gaussian random variable with variance

$\displaystyle\mathbb{E}X_{N,\beta}^{2}(\tau)$ $\displaystyle=\sum_{p\geq
2}\frac{\beta_{p}^{2}p!}{N^{p-1}}\sum_{1\leq i_{1}<\cdots i_{p-1}\leq
N-1}\tau_{i_{1}}^{2}\cdots\tau_{i_{p-1}}^{2}$ $\displaystyle=\sum_{p\geq
2}\beta_{p}^{2}p\prod_{l=1}^{p-1}\Bigl{(}1-\frac{l}{N}\Bigr{)}\nearrow\sum_{p\geq
2}\beta_{p}^{2}p,\ \text{ as }N\to\infty.$ (11)

We remark that the variance of $X_{N,\beta}(\tau)$ is bounded uniformly for
any $N\geq 1$ and any $\tau\in\Sigma_{N-1}$ by $C_{\beta}$ defined at the
beginning. The goal of this section is to establish the following theorem.

######  Theorem 3.1 (Cavity equation).

Let $\alpha\sim G_{N,\beta}$ and write $\alpha=(\rho,\alpha_{N})$. Then,

$\displaystyle\lim_{\epsilon\downarrow
0}\limsup_{N\to\infty}\mathbb{E}\Bigl{\langle}\Big{(}\langle\sigma_{N}\rangle_{N,\beta}^{\alpha}-\tanh\big{(}X_{N,\beta}(\langle\tau\rangle_{N-1,\beta^{\prime}}^{\rho})+h\big{)}\Big{)}^{2}\Bigr{\rangle}_{N,\beta}=0,$

where $\epsilon\downarrow 0$ along a sequence such that $q_{P}-\epsilon$ is
always a point of continuity for $\mu_{P}$.

In other words, $\langle\sigma_{N}\rangle_{N,\beta}^{\alpha}$ is
asymptotically a function of the vector
$\langle\tau\rangle_{N-1,\beta^{\prime}}^{\rho}$, the average with respect to
the conditional Gibbs measure of an $(N-1)$ system. Here, the dimension of the
system in $\langle\cdot\rangle_{N-1,\beta^{\prime}}^{\rho}$ is less than that
of $\langle\cdot\rangle_{N,\beta}^{\alpha}$ by 1. Furthermore, the last spin
$\alpha_{N}$ is absent in the constrained Gibbs measure
$\langle\cdot\rangle_{N-1,\beta^{\prime}}^{\rho}$. We establish this theorem
in three subsections.

###  3.1 Removal of the cavity spin constraint I

The conditional Gibbs average $\langle\sigma_{N}\rangle_{N,\beta}^{\alpha}$
can be written as

$\displaystyle\langle\sigma_{N}\rangle_{N,\beta}^{\alpha}$
$\displaystyle=\frac{\sum_{\sigma\in\Sigma_{N}^{\alpha}}\sigma_{N}G_{N,\beta}(\sigma)}{\sum_{\sigma\in\Sigma_{N}^{\alpha}}G_{N,\beta}(\sigma)}$
$\displaystyle=\frac{\big{\langle}e^{X_{N,\beta}(\tau)+h}\mathbbm{1}_{\Sigma_{N}^{\alpha}}\big{(}(\tau,1)\big{)}-e^{-X_{N,\beta}(\tau)-h}\mathbbm{1}_{\Sigma_{N}^{\alpha}}\big{(}(\tau,-1)\big{)}\,\big{\rangle}_{N-1,\beta^{\prime}}}{\big{\langle}e^{X_{N,\beta}(\tau)+h}\mathbbm{1}_{\Sigma_{N}^{\alpha}}\big{(}(\tau,1)\big{)}+e^{-X_{N,\beta}(\tau)-h}\mathbbm{1}_{\Sigma_{N}^{\alpha}}\big{(}(\tau,-1)\big{)}\,\big{\rangle}_{N-1,\beta^{\prime}}},$
(12)

and, for $\alpha=(\rho,\alpha_{N})$, we have

$\displaystyle\\{\tau\in\Sigma_{N-1}:(\tau,\pm
1)\in\Sigma_{N}^{\alpha}\\}=\Big{\\{}\tau\in\Sigma_{N-1}:R(\tau,\rho)\geq
q_{P}-\epsilon+\frac{1}{N-1}(q_{p}-\epsilon\mp\alpha_{N})\Big{\\}},$

Thus, by letting

$\displaystyle A_{\ominus}^{\rho}$
$\displaystyle:=\Bigl{\\{}\tau\in\Sigma_{N-1}:R(\rho,\tau)\geq
q_{P}-\Bigl{(}\epsilon-\frac{3}{N-1}\Bigr{)}\Bigr{\\}},$ $\displaystyle
A_{\oplus}^{\rho}$
$\displaystyle:=\Bigl{\\{}\tau\in\Sigma_{N-1}:R(\rho,\tau)\geq
q_{P}-\Bigl{(}\epsilon+\frac{3}{N-1}\Bigr{)}\Bigr{\\}},$

we can easily see that for all $\tau\in\Sigma_{N-1}$ and
$\alpha=(\rho,\alpha_{N})\in\Sigma_{N-1}\times\Sigma_{1},$

$\displaystyle\begin{split}&\mathbbm{1}_{A_{\ominus}^{\rho}}(\tau)\leq\mathbbm{1}_{\Sigma_{N-1}^{\rho}}(\tau)\leq\mathbbm{1}_{A_{\oplus}^{\rho}}(\tau),\\\
&\mathbbm{1}_{A_{\ominus}^{\rho}}(\tau)\leq\mathbbm{1}_{\Sigma_{N}^{\alpha}}((\tau,-1)),\mathbbm{1}_{\Sigma_{N}^{\alpha}}((\tau,+1))\leq\mathbbm{1}_{A_{\oplus}^{\rho}}(\tau).\end{split}$
(13)

We will approximate the numerator and denominator of (12) by substituting all
indicators there by $\mathbbm{1}_{\Sigma_{N-1}^{\rho}}(\tau)$. To this end, we
first prepare three lemmas.

Denote by $g_{\cdot N}$ the collection of all Gaussian disorders that appear
in $X_{N,\beta}$,

$g_{i_{1},\ldots,i_{p-1},N},\ \ \text{for }p=2,3,\ldots\text{and}\ 1\leq
i_{1}<\cdots<i_{p-1}\leq N-1$

and by $\mathbb{E}_{g_{\cdot N}}$ the expectation (only) with respect to
$g_{\cdot N}.$

######  Lemma 3.2.

Let $\alpha=(\rho,\alpha_{N})$ be sampled according to $G_{N,\beta}$ and
$(\Gamma(\rho))_{\rho\in\Sigma_{N-1}}$ be a collection of nonnegative random
variables, which are independent of the Gaussian disorders appearing in
$X_{N,\beta}.$ Then we have

$\displaystyle\mathbb{E}\bigl{\langle}\Gamma(\rho)\bigr{\rangle}_{N,\beta}$
$\displaystyle\leq e^{2^{-1}\sum_{p\geq
2}\beta_{p}^{2}p}\cosh(h)\mathbb{E}\bigl{\langle}\Gamma(\tau)\bigr{\rangle}_{N-1,\beta^{\prime}}.$

###### Proof.

Write

$\displaystyle\mathbb{E}\big{\langle}\Gamma(\rho)\big{\rangle}_{N,\beta}$
$\displaystyle=\mathbb{E}\sum_{\tau\in\Sigma_{N-1}}\Gamma(\tau)\big{[}G_{N,\beta}((\tau,1))+G_{N,\beta}((\tau,-1))\big{]}$
$\displaystyle=\mathbb{E}\sum_{\tau\in\Sigma_{N-1}}\Gamma(\tau)G_{N-1,\beta^{\prime}}(\tau)\frac{G_{N,\beta}((\tau,1))+G_{N,\beta}((\tau,-1))}{G_{N-1,\beta^{\prime}}(\tau)},$

where the last fraction is the Radon-Nykodym derivative of the two measures
$G_{N,\beta}((\cdot,1))+G_{N,\beta}((\cdot,-1))$ and
$G_{N-1,\beta^{\prime}}(\cdot)$ on $\Sigma_{N-1}$. Computing the numerator and
the denominator at $\tau$, we get

$\displaystyle G_{N,\beta}((\tau,1))+G_{N,\beta}((\tau,-1))$
$\displaystyle=\frac{e^{H_{N-1,\beta^{\prime}}(\tau)}\cosh\big{(}X_{N,\beta}(\tau)+h\big{)}}{\sum_{\tau^{\prime}\in\Sigma_{N-1}}e^{H_{N-1,\beta^{\prime}}(\tau^{\prime})}\cosh\big{(}X_{N,\beta}(\tau^{\prime})+h\big{)}},$
$\displaystyle G_{N-1,\beta^{\prime}}(\tau)$
$\displaystyle=\frac{e^{H_{N-1,\beta^{\prime}}(\tau)}}{\sum_{\tau^{\prime}\in\Sigma_{N-1}}e^{H_{N-1,\beta^{\prime}}(\tau^{\prime})}},$

and thus the ratio is bounded by

$\displaystyle\frac{G_{N,\beta}((\tau,1))+G_{N,\beta}((\tau,-1))}{G_{N-1,\beta^{\prime}}(\tau)}=\frac{\cosh\left(X_{N,\beta}(\tau)+h\right)}{\langle\cosh\left(X_{N,\beta}(\tau)+h\right)\rangle_{N-1,\beta^{\prime}}}\leq\cosh\left(X_{N,\beta}(\tau)+h\right).$

It follows from (11) that

$\displaystyle\mathbb{E}\big{\langle}\Gamma(\rho)\big{\rangle}_{N,\beta}$
$\displaystyle\leq\mathbb{E}\bigl{\langle}\Gamma(\tau)\cosh\left(X_{N,\beta}(\tau)+h\right)\bigr{\rangle}_{N-1,\beta^{\prime}}$
$\displaystyle=\mathbb{E}\bigl{\langle}\Gamma(\tau)\mathbb{E}_{g_{\cdot
N}}\bigl{[}\cosh\left(X_{N,\beta}(\tau)+h\right)\bigr{]}\bigr{\rangle}_{N-1,\beta^{\prime}}$
$\displaystyle\leq e^{2^{-1}\sum_{p\geq
2}\beta_{p}^{2}p}\cosh(h)\mathbb{E}\bigl{\langle}\Gamma(\tau)\bigr{\rangle}_{N-1,\beta^{\prime}}.$

∎

######  Lemma 3.3.

For any $\epsilon$ such that $q_{P}-\epsilon$ is a point of continuity of
$\mu_{P},$ we have that

$\displaystyle\limsup_{N\to\infty}\mathbb{E}\big{\langle}\mathbbm{1}_{\\{\left|R(\tau^{1},\tau^{2})-(q_{P}-\epsilon)\right|<\frac{3}{N-1}\\}}\big{\rangle}_{N-1,\beta^{\prime}}=0.$
(14)

In addition,

$\displaystyle\lim_{\delta\downarrow
0}\limsup_{N\to\infty}\mathbb{E}\langle\mathbbm{1}_{\\{W_{N-1,\beta^{\prime}}(A_{\ominus}^{\tau})\geq\delta\\}}\rangle_{N-1,\beta^{\prime}}=1,\,\,\forall\epsilon>0,$
(15)

and

$\displaystyle\lim_{\delta\downarrow
0}\limsup_{N\to\infty}\mathbb{E}\bigl{\langle}\mathbbm{1}_{\\{q_{P}+\delta\geq
R(\tau^{1},\tau^{2})\geq\min(R(\tau^{1},\tau^{3}),R(\tau^{2},\tau^{3}))-\delta\\}}\big{\rangle}_{N-1,\beta^{\prime}}=1.$
(16)

######  Remark 3.4.

These assertions hold if $\beta^{\prime}$ is replaced by $\beta.$ Indeed,
under this replacement, (15) was established in [11, Lemm 3], while (16) can
be concluded from [25] and the fact that $R(\tau^{1},\tau^{2})$ converges to
$\mu_{P}$ weakly, see [24]. The latter fact can also be used to deduce (14).

###### Proof.

First of all, we claim that $R(\tau^{1},\tau^{2})$ converges to $\mu_{P}$
under $\mathbb{E}G_{N-1,\beta^{\prime}}^{\otimes 2}$ as $N\to\infty.$ To see
this, note that $\beta\mapsto\mathbb{E}F_{N}(\beta)$ and
$\beta\mapsto\mathbb{E}F_{N}(\beta^{\prime})$ are convex functions and that
there exists a constant $C>0$ independent of $N$ such that
$|\mathbb{E}F_{N}(\beta)-\mathbb{E}F_{N}(\beta^{\prime})|\leq C/N$ for all
$N\geq 1.$ Hence, $\mathbb{E}F_{N-1}(\beta)$ and
$\mathbb{E}F_{N-1}(\beta^{\prime})$ converge to the same limit, say
$F(\beta),$ which can be represented using the Parisi formula. One of the
useful consequence of this representation is that $F$ is partially
differentiable with respect to any $\beta_{p}$ and from Griffith’s
lemma222Girffith’s lemma: Let $f_{N}$ be a sequence of differentiable convex
functions defined on an open interval $I$. Assume that $f_{N}$ converges $f$
pointwise on $I$. If $f$ is differentiable at some $x\in I,$ then
$\lim_{N\to\infty}f_{N}^{\prime}(x)=f^{\prime}(x).$,

$\displaystyle\lim_{N\to\infty}\frac{\beta_{p}}{2}\bigl{(}1-\mathbb{E}\bigl{\langle}R(\tau^{1},\tau^{2})^{p}\bigr{\rangle}_{N-1,\beta^{\prime}}\bigr{)}$
$\displaystyle=\lim_{N\to\infty}\partial_{p}\mathbb{E}F_{N-1}(\beta^{\prime})$
$\displaystyle=\partial_{\beta_{p}}\mathbb{E}F(\beta)=\frac{\beta_{p}}{2}\Bigl{(}1-\int_{0}^{1}q^{p}\mu_{P}(dq)\Bigr{)},$

where the last equality holds due to [24]. Consequently, whenever
$\beta_{p}\neq 0,$ we have

$\displaystyle\lim_{N\to\infty}\mathbb{E}\bigl{\langle}R(\tau^{1},\tau^{2})^{p}\bigr{\rangle}_{N-1,\beta^{\prime}}=\int_{0}^{1}q^{p}\mu_{P}(dq)$

and our claim follows since we assume that our model is generic.

Now we turn to the proof of (14). Let $\epsilon^{\prime}$ and
$\epsilon^{\prime\prime}$ be any positive numbers such that
$\epsilon^{\prime}<\epsilon<\epsilon^{\prime\prime}$. From these, as long as
$N$ is large enough, we can bound

$\displaystyle\mathbb{E}\big{\langle}\mathbbm{1}_{\\{\left|R(\tau^{1},\tau^{2})-(q_{P}-\epsilon)\right|<\frac{3}{N-1}\\}}\big{\rangle}_{N-1,\beta^{\prime}}$
$\displaystyle=\mathbb{E}\big{\langle}\mathbbm{1}_{\\{R(\tau^{1},\tau^{2})<q_{P}-\epsilon+\frac{3}{N-1}\\}}\big{\rangle}_{N-1,\beta^{\prime}}-\mathbb{E}\big{\langle}\mathbbm{1}_{\\{R(\tau^{1},\tau^{2})\leq
q_{P}-\epsilon-\frac{3}{N-1}\\}}\big{\rangle}_{N-1,\beta^{\prime}}$
$\displaystyle\leq\mathbb{E}\big{\langle}\mathbbm{1}_{\\{R(\tau^{1},\tau^{2})\leq
q_{P}-\epsilon^{\prime}\\}}\big{\rangle}_{N-1,\beta^{\prime}}-\mathbb{E}\big{\langle}\mathbbm{1}_{\\{R(\tau^{1},\tau^{2})<q_{P}-\epsilon^{\prime\prime}\\}}\big{\rangle}_{N-1,\beta^{\prime}}.$

Consequently, from the weak convergence of $R(\tau^{1},\tau^{2})$ to $\mu_{P}$
under $\mathbb{E}G_{N-1,\beta^{\prime}}^{\otimes 2},$ we see that

$\displaystyle\limsup_{N\to\infty}\mathbb{E}\big{\langle}\mathbbm{1}_{\\{\left|R(\tau^{1},\tau^{2})-(q_{P}-\epsilon)\right|<\frac{3}{N-1}\\}}\big{\rangle}_{N-1,\beta^{\prime}}\leq\mu_{P}([0,q_{P}-\epsilon^{\prime}])-\mu_{P}([0,q_{P}-\epsilon^{\prime\prime})).$

Hence, from the continuity of $\mu_{P}$ at $q_{P}-\epsilon,$ after sending
$\epsilon^{\prime}\uparrow\epsilon$ and
$\epsilon^{\prime\prime}\downarrow\epsilon,$

$\displaystyle\limsup_{N\to\infty}\mathbb{E}\big{\langle}\mathbbm{1}_{\\{\left|R(\tau^{1},\tau^{2})-(q_{P}-\epsilon)\right|<\frac{3}{N-1}\\}}\big{\rangle}_{N-1,\beta^{\prime}}=0.$

As for (15) and (16), we recall that it is already known in [11, Lemma 3] that

$\displaystyle\lim_{\delta\downarrow
0}\limsup_{N\to\infty}\mathbb{E}\langle\mathbbm{1}_{\\{W_{N-1,\beta}(A_{\ominus}^{\tau})\geq\delta\\}}\rangle_{N-1,\beta}=1.$

Since the overlap $R(\tau^{1},\tau^{2})$ under
$\mathbb{E}G_{N-1,\beta}^{\otimes 2}$ and
$\mathbb{E}G_{N-1,\beta^{\prime}}^{\otimes 2}$ converges to the same
distribution $\mu_{P}$ and both of the measures
$\mathbb{E}G_{N-1,\beta}^{\otimes\infty}$ and
$\mathbb{E}G_{N-1,\beta^{\prime}}^{\otimes\infty}$ satisfy the extended
Ghirlanda-Guerra identities, the limiting distributions of the Gram matrix
$(R(\tau^{\ell},\tau^{\ell^{\prime}}))_{1\leq\ell<\ell^{\prime}}$
corresponding to $(\tau^{\ell})_{\ell\geq 1}$ sampled from either
$G_{N-1,\beta}$ or $G_{N-1,\beta^{\prime}}$ are uniquely determined by the
overlap distribution $\mu_{P}$ and are described by the Ruelle Probability
Cascades parametrized by $\mu_{P},$ see [26]. With this, the same argument in
[11, Lemma 3] yields (15). For (16), note that the extended Ghirlanda-Guerra
identities also imply that for independent $\tau^{1},\tau^{2},\tau^{3}\sim
G_{N-1,\beta^{\prime}},$ their overlaps must be asymptotically ultrametric due
to [25]. From our claim, we also see that $R(\tau^{1},\tau^{2})$ is
asymptotically less than $q_{P}+\delta$. These together complete the proof of
(16). ∎

######  Lemma 3.5.

For $\varepsilon=0,\pm 1$ and any $\epsilon>0$ such that $q_{P}-\epsilon$ is a
point of continuity of $\mu_{P},$ we have

$\lim_{N\to\infty}\mathbb{E}\left\langle\big{(}\,\langle e^{\varepsilon
X_{N,\beta}(\tau)}\mathbbm{1}_{A_{\oplus}^{\rho}}(\tau)\rangle_{N-1,\beta^{\prime}}-\langle
e^{\varepsilon
X_{N,\beta}(\tau)}\mathbbm{1}_{A_{\ominus}^{\rho}}(\tau)\rangle_{N-1,\beta^{\prime}}\big{)}^{2}\,\right\rangle_{N,\beta}=0.$

###### Proof.

The left hand side is equal to

$\displaystyle\mathbb{E}\left\langle\langle e^{\varepsilon
X_{N,\beta}(\tau)}\mathbbm{1}_{A_{\oplus}^{\rho}\setminus
A_{\ominus}^{\rho}}(\tau)\rangle_{N-1,\beta^{\prime}}^{2}\right\rangle_{N,\beta}$

and by Cauchy-Schwarz inequality, it is no more than

$\displaystyle\mathbb{E}\bigl{\langle}\langle e^{2\varepsilon
X_{N,\beta}(\tau)}\rangle_{N-1,\beta^{\prime}}\langle\mathbbm{1}_{A_{\oplus}^{\rho}\setminus
A_{\ominus}^{\rho}}(\tau)\rangle_{N-1,\beta^{\prime}}\bigr{\rangle}_{N,\beta}$
$\displaystyle=\mathbb{E}\langle e^{2\varepsilon
X_{N,\beta}(\tau)}\rangle_{N-1,\beta^{\prime}}\bigl{\langle}\langle\mathbbm{1}_{A_{\oplus}^{\rho}\setminus
A_{\ominus}^{\rho}}(\tau)\rangle_{N-1,\beta^{\prime}}\bigr{\rangle}_{N,\beta}$
$\displaystyle\leq\Bigl{(}\mathbb{E}\langle e^{2\varepsilon
X_{N,\beta}(\tau)}\rangle_{N-1,\beta^{\prime}}^{2}\mathbb{E}\bigl{\langle}\langle\mathbbm{1}_{A_{\oplus}^{\rho}\setminus
A_{\ominus}^{\rho}}(\tau)\rangle_{N-1,\beta^{\prime}}\bigr{\rangle}_{N,\beta}^{2}\Bigr{)}^{1/2}$
$\displaystyle\leq\Bigl{(}\mathbb{E}\bigl{[}\bigl{\langle}\mathbb{E}_{g_{\cdot
N}}e^{4\varepsilon
X_{N,\beta}(\tau)}\bigr{\rangle}_{N-1,\beta^{\prime}}\bigr{]}\mathbb{E}\bigl{\langle}\langle\mathbbm{1}_{A_{\oplus}^{\rho}\setminus
A_{\ominus}^{\rho}}(\tau)\rangle_{N-1,\beta^{\prime}}\bigr{\rangle}_{N,\beta}\Bigr{)}^{1/2}.$

Here, using the independence between $g_{\cdot N}$ and the disorder appearing
in the Gibbs expectation $\langle\cdot\rangle_{N-1,\beta^{\prime}}$, one can
get from (11) that

$\displaystyle\mathbb{E}_{g_{\cdot N}}e^{4\varepsilon X_{N,\beta}(\tau)}$
$\displaystyle=e^{8\varepsilon^{2}\text{Var}(X_{N,\beta}(\tau))}\leq
e^{8\sum_{p\geq 2}p\beta_{p}^{2}}<\infty.$

Note that this upper bound holds for $\varepsilon=0$ and $\pm 1$. Thus,

$\displaystyle\ \ \mathbb{E}\left\langle\big{\langle}e^{\varepsilon
X_{N,\beta}(\tau)}\mathbbm{1}_{A_{\oplus}^{\rho}}(\tau)-e^{\varepsilon
X_{N,\beta}(\tau)}\mathbbm{1}_{A_{\ominus}^{\rho}}(\tau)\big{\rangle}^{2}_{N-1,\beta^{\prime}}\right\rangle_{N,\beta}$
$\displaystyle\leq e^{4\sum_{p\geq
2}p\beta_{p}^{2}}\Big{\\{}\mathbb{E}\Big{\langle}\big{\langle}\mathbbm{1}_{A_{\oplus}^{\rho}\setminus
A_{\ominus}^{\rho}}(\tau)\big{\rangle}_{N-1,\beta^{\prime}}\Big{\rangle}_{N,\beta}\Big{\\}}^{1/2}$
$\displaystyle\leq e^{(4+1/4)\sum_{p\geq
2}p\beta_{p}^{2}}\cosh(h)^{1/2}\Big{\\{}\mathbb{E}\left\langle\big{\langle}\mathbbm{1}_{A_{\oplus}^{\tau^{2}}\setminus
A_{\ominus}^{\tau^{2}}}(\tau^{1})\big{\rangle}_{N-1,\beta^{\prime}}\right\rangle_{N-1,\beta^{\prime}}\Big{\\}}^{1/2}$
$\displaystyle=e^{(4+1/4)\sum_{p\geq
2}p\beta_{p}^{2}}\cosh^{1/2}(h)\Big{\\{}\mathbb{E}\left\langle\mathbbm{1}_{\\{(\tau^{1},\tau^{2}):\,|R(\tau^{1},\tau^{2})-(q_{P}-\epsilon)|\leq
3/(N-1)\\}}\right\rangle_{N-1,\beta^{\prime}}\Big{\\}}^{1/2},$

where the second inequality used Lemma 3.2 and $\tau^{1}$ and $\tau^{2}$ are
sampled from the inner and outer Gibbs measures, respectively. Our proof is
then completed by applying (14). ∎

With the help of the above three lemmas, we can now replace all the indicators
in (12).

######  Proposition 3.6.

For $\epsilon>0$ such that $q_{P}-\epsilon$ is a point of continuity of
$\mu_{P},$ we have that

$\displaystyle\lim_{N\to\infty}\mathbb{E}\bigg{\langle}\Bigl{(}\langle\sigma_{N}\rangle_{N,\beta}^{\alpha}-\frac{\langle\sinh(X_{N,\beta}(\tau)+h)\rangle_{N-1,\beta^{\prime}}^{\rho}}{\langle\cosh(X_{N,\beta}(\tau)+h)\rangle_{N-1,\beta^{\prime}}^{\rho}}\Bigr{)}^{2}\bigg{\rangle}_{N,\beta}=0.$
(17)

###### Proof.

Define

$\displaystyle A_{\alpha}$
$\displaystyle:=\frac{1}{2}\big{\langle}e^{X_{N,\beta}(\tau)+h}\mathbbm{1}_{\Sigma_{N}^{\alpha}}\big{(}(\tau,1)\big{)}-e^{-X_{N,\beta}(\tau)-h}\mathbbm{1}_{\Sigma_{N}^{\alpha}}\big{(}(\tau,-1)\big{)}\,\big{\rangle}_{N-1,\beta^{\prime}},$
$\displaystyle B_{\alpha}$
$\displaystyle:=\frac{1}{2}\big{\langle}e^{X_{N,\beta}(\tau)+h}\mathbbm{1}_{\Sigma_{N}^{\alpha}}\big{(}(\tau,1)\big{)}+e^{-X_{N,\beta}(\tau)-h}\mathbbm{1}_{\Sigma_{N}^{\alpha}}\big{(}(\tau,-1)\big{)}\,\big{\rangle}_{N-1,\beta^{\prime}},$
$\displaystyle A_{\rho}$
$\displaystyle:=\big{\langle}\sinh(X_{N,\beta}(\tau)+h)\mathbbm{1}_{\Sigma_{N-1}^{\rho}}(\tau)\big{\rangle}_{N-1,\beta^{\prime}},$
$\displaystyle B_{\rho}$
$\displaystyle:=\big{\langle}\cosh(X_{N,\beta}(\tau)+h)\mathbbm{1}_{\Sigma_{N-1}^{\rho}}(\tau)\big{\rangle}_{N-1,\beta^{\prime}},$

and then we can express

$\langle\sigma_{N}\rangle_{N,\beta}^{\alpha}=\frac{A_{\alpha}}{B_{\alpha}}\quad\mbox{and}\quad\frac{\langle\sinh(X_{N,\beta}(\tau)+h)\rangle_{N-1,\beta^{\prime}}^{\rho}}{\langle\cosh(X_{N,\beta}(\tau)+h)\rangle_{N-1,\beta^{\prime}}^{\rho}}=\frac{A_{\rho}}{B_{\rho}}.$

For any $\delta>0$, the expectation in (17) is no more than

$\displaystyle\mathbb{E}\left\langle\Bigl{(}\frac{A_{\alpha}}{B_{\alpha}}-\frac{A_{\rho}}{B_{\rho}}\Bigr{)}^{2}\right\rangle_{N,\beta}$
$\displaystyle\leq
C\mathbb{E}\left\langle\Big{(}\frac{A_{\alpha}}{B_{\alpha}}-\frac{A_{\alpha}}{B_{\alpha}}\mathbbm{1}_{\\{W_{N-1,\beta^{\prime}}(A_{\ominus}^{\rho})\geq\delta\\}}\Big{)}^{2}\right\rangle_{N,\beta}$
$\displaystyle\quad+C\mathbb{E}\left\langle\Big{(}\frac{A_{\alpha}}{B_{\alpha}}-\frac{A_{\rho}}{B_{\rho}}\Big{)}^{2}\mathbbm{1}_{\\{W_{N-1,\beta^{\prime}}(A_{\ominus}^{\rho})\geq\delta\\}}\right\rangle_{N,\beta}$
$\displaystyle\quad+C\mathbb{E}\left\langle\Big{(}\frac{A_{\rho}}{B_{\rho}}-\frac{A_{\rho}}{B_{\rho}}\mathbbm{1}_{\\{W_{N-1,\beta^{\prime}}(A_{\ominus}^{\rho})\geq\delta\\}}\Big{)}^{2}\right\rangle_{N,\beta}.$

Using that $|A_{\alpha}/B_{\alpha}|\leq 1$, $|A_{\rho}/B_{\rho}|\leq 1$ and
combining the first term and the third term, we get

$\displaystyle\mathbb{E}$
$\displaystyle\left\langle\Bigl{(}\frac{A_{\alpha}}{B_{\alpha}}-\frac{A_{\rho}}{B_{\rho}}\Bigr{)}^{2}\right\rangle_{N,\beta}$
$\displaystyle\leq$
$\displaystyle\,2C\mathbb{E}\big{\langle}\mathbbm{1}_{\\{W_{N-1,\beta^{\prime}}(A_{\ominus}^{\rho})<\delta\\}}\big{\rangle}_{N,\beta}+C\mathbb{E}\left\langle\Bigl{(}\frac{A_{\alpha}}{B_{\alpha}}-\frac{A_{\rho}}{B_{\rho}}\Bigr{)}^{2}\mathbbm{1}_{\\{W_{N-1,\beta^{\prime}}(A_{\ominus}^{\rho})\geq\delta\\}}\right\rangle_{N,\beta}.$
(18)

Since $W_{N-1,\beta^{\prime}}(A_{\ominus}^{\rho})$ is independent of $g_{\cdot
N}$ for any $\rho\in\Sigma_{N-1},$ we can use (14) to bound

$\displaystyle\mathbb{E}\big{\langle}\mathbbm{1}_{\\{W_{N-1,\beta^{\prime}}(A_{\ominus}^{\rho})<\delta\\}}\big{\rangle}_{N,\beta}$
$\displaystyle\leq e^{\sum_{p\geq
2}\frac{\beta_{p}^{2}p}{2}}\cosh(h)\,\mathbb{E}\langle\mathbbm{1}_{\\{W_{N-1,\beta^{\prime}}(A_{\ominus}^{\tau})<\delta\\}}\rangle_{N-1,\beta^{\prime}}\to
0$

as $N\to\infty$ and then $\delta\downarrow 0,$ where the last limit used (15).
As for the second expectation in (18), we note that, on the event
$\\{W_{N-1,\beta^{\prime}}(A_{\ominus}^{\rho})\geq\delta\\}$,

$\displaystyle\left|\frac{A_{\alpha}}{B_{\alpha}}-\frac{A_{\rho}}{B_{\rho}}\right|$
$\displaystyle\leq\left\lvert\frac{A_{\rho}}{B_{\rho}}\right\rvert\frac{|B_{\alpha}-B_{\rho}|}{B_{\alpha}}+\frac{|A_{\rho}-A_{\alpha}|}{B_{\alpha}}$
$\displaystyle\leq\frac{|B_{\alpha}-B_{\rho}|}{W_{N-1,\beta^{\prime}}(A_{\ominus}^{\rho})}+\frac{|A_{\rho}-A_{\alpha}|}{W_{N-1,\beta^{\prime}}(A_{\ominus}^{\rho})}\leq\delta^{-1}[|B_{\alpha}-B_{\rho}|+|A_{\rho}-A_{\alpha}|],$

where we used that $|A_{\rho}/B_{\rho}|\leq 1$ and that from
(LABEL:eqn:nestedsets),

$\displaystyle B_{\alpha},B_{\rho}$
$\displaystyle\geq\big{\langle}\cosh\big{(}X_{N,\beta}(\tau)+h\big{)}\mathbbm{1}_{A_{\ominus}^{\rho}}(\tau)\big{\rangle}_{N-1,\beta^{\prime}}\geq\big{\langle}\mathbbm{1}_{A_{\ominus}^{\rho}}(\tau)\big{\rangle}_{N-1,\beta^{\prime}}=W_{N-1,\beta^{\prime}}(A_{\ominus}^{\rho}).$

Thus, for any $\delta>0$,

$\displaystyle\mathbb{E}\left\langle\Bigl{(}\frac{A_{\alpha}}{B_{\alpha}}-\frac{A_{\rho}}{B_{\rho}}\Bigr{)}^{2}\mathbbm{1}_{\\{W_{N-1,\beta^{\prime}}(A_{\ominus}^{\rho})\geq\delta\\}}\right\rangle_{N,\beta}\leq
C\delta^{-2}\Bigl{(}\mathbb{E}\big{\langle}(A_{\alpha}-A_{\rho})^{2}\big{\rangle}_{N,\beta}+\mathbb{E}\big{\langle}(B_{\alpha}-B_{\rho})^{2}\big{\rangle}_{N,\beta}\Bigr{)}.$

Here, from Lemma 3.5 and the inequality in (LABEL:eqn:nestedsets), we have

$\displaystyle\mathbb{E}\big{\langle}(A_{\alpha}-A_{\rho})^{2}\big{\rangle}_{N,\beta}$
$\displaystyle\leq
C^{\prime}\mathbb{E}\left\langle\big{(}\,\big{\langle}e^{X_{N,\beta}(\tau)+h}\mathbbm{1}_{A_{\oplus}^{\rho}}(\tau)\big{\rangle}_{N-1,\beta^{\prime}}-\big{\langle}e^{X_{N,\beta}(\tau)+h}\mathbbm{1}_{A_{\ominus}^{\rho}}(\tau)\big{\rangle}_{N-1,\beta^{\prime}}\big{)}^{2}\right\rangle_{N,\beta}$
$\displaystyle\quad+C^{\prime}\mathbb{E}\left\langle\big{(}\,\big{\langle}e^{-X_{N,\beta}(\tau)-h}\mathbbm{1}_{A_{\oplus}^{\rho}}(\tau)\big{\rangle}_{N-1,\beta^{\prime}}-\big{\langle}e^{-X_{N,\beta}(\tau)-h}\mathbbm{1}_{A_{\ominus}^{\rho}}(\tau)\big{\rangle}_{N-1,\beta^{\prime}}\big{)}^{2}\right\rangle_{N,\beta}$
$\displaystyle\quad\to 0\quad\text{as }N\to\infty.$

Similarly,
$\mathbb{E}\big{\langle}(B_{\alpha}-B_{\rho})^{2}\big{\rangle}_{N,\beta}\to
0$. These imply that the second term in (18) also vanishes for any $\delta>0$,
completing our proof. ∎

###  3.2 Central limit theorem for the cavity field

The goal of this section is to derive a central limit theorem for the centered
cavity field defined as

$\displaystyle\dot{X}_{N,\beta}^{\rho}(\tau)$
$\displaystyle:=X_{N,\beta}(\tau)-\bigl{\langle}X_{N,\beta}(\tau)\bigr{\rangle}_{N-1,\beta^{\prime}}^{\rho}$
$\displaystyle=\sum_{p\geq
2}\frac{\beta_{p}\sqrt{p!}}{N^{\frac{p-1}{2}}}\sum_{1\leq
i_{1}<\cdots<i_{p-1}\leq N-1}g_{i_{1}\ldots
i_{p-1},N}\bigl{(}\tau_{i_{1}}\cdots\tau_{i_{p-1}}-\langle\tau_{i_{1}}\cdots\tau_{i_{p-1}}\rangle_{N-1,\beta^{\prime}}^{\rho}\bigr{)}$
(19)

for any $\rho,\tau\in\Sigma_{N-1}.$

######  Theorem 3.7 (Central Limit Theorem).

Assume that $U$ is an infinitely differentiable function on $\mathbb{R}$
satisfying

$\sup_{0\leq\gamma\leq M}\mathbb{E}|U^{(d)}(\gamma z)|^{k}<\infty,\ \ \forall
d,k\in\mathbb{Z},\ 0\leq d,k,M<\infty.$

Then, for any integer $r\geq 1$, we have

$\displaystyle\lim_{\epsilon\downarrow
0}\limsup_{N\to\infty}\mathbb{E}\left\langle\Big{[}\big{\langle}U\big{(}\dot{X}_{N,\beta}^{\rho}(\tau)\big{)}\big{\rangle}^{\rho}_{N-1,\beta^{\prime}}-\mathbb{E}U\big{(}\xi\sqrt{\zeta^{\prime}(1)-\zeta^{\prime}(q_{P})}\big{)}\Big{]}^{2r}\right\rangle_{N,\beta}=0,$
(20)

where $\xi\sim N(0,1)$ is independent of all other random variables.

######  Remark 3.8.

As we will see later, this theorem is special case of the multivariate central
limit theorem stated in Theorem 5.1 below. Here we provide a standalone proof
for Theorem 3.7 to illustrate the main steps as a warm-up for the proof of
Theorem 5.1.

The proof of Theorem 3.7 relies on the concentration of the overlap within the
pure state established in the following lemma.

######  Lemma 3.9 (Concentration of the overlap).

We have that

$\displaystyle\lim_{\epsilon\downarrow
0}\limsup_{N\to\infty}\mathbb{E}\bigl{\langle}\bigl{\langle}|R(\tau^{1},\tau^{2})-q_{P}|\bigr{\rangle}_{N-1,\beta^{\prime}}^{\rho}\bigr{\rangle}_{N-1,\beta^{\prime}}$
$\displaystyle=0,$ $\displaystyle\lim_{\epsilon\downarrow
0}\limsup_{N\to\infty}\mathbb{E}\bigl{\langle}\bigl{\langle}|R(\tau,\rho)-q_{P}|\bigr{\rangle}_{N-1,\beta^{\prime}}^{\rho}\bigr{\rangle}_{N-1,\beta^{\prime}}$
$\displaystyle=0,$

where in both equations, the $\rho$’s are sampled from the outer Gibbs measure
and $\tau,\tau^{1},\tau^{2}$ are sampled from the inner one.

###### Proof.

For any $\delta>0$, letting
$\mathcal{E}_{\delta,\rho}=\\{W_{N-1,\beta^{\prime}}(\Sigma_{N-1}^{\rho})\geq\delta\\}$,
we have

$\displaystyle\mathbb{E}\big{\langle}\big{\langle}\bigl{|}R(\tau^{1},\tau^{2})-q_{P}|\big{\rangle}_{N-1,\beta^{\prime}}^{\rho}\big{\rangle}_{N-1,\beta^{\prime}}$
$\displaystyle=\mathbb{E}\big{\langle}\big{\langle}\big{|}R(\tau^{1},\tau^{2})-q_{P}\big{|}\big{\rangle}_{N-1,\beta^{\prime}}^{\rho}\mathbbm{1}_{\mathcal{E}_{\delta,\rho}^{c}}\big{\rangle}_{N-1,\beta^{\prime}}$
$\displaystyle+\mathbb{E}\left\langle\frac{\big{|}R(\tau^{1},\tau^{2})-q_{P}\big{|}\mathbbm{1}_{\Sigma_{N-1}^{\rho}}(\tau^{1})\mathbbm{1}_{\Sigma_{N-1}^{\rho}}(\tau^{2})}{W^{2}_{N-1,\beta^{\prime}}(\Sigma_{N-1}^{\rho})}\mathbbm{1}_{\mathcal{E}_{\delta,\rho}}\right\rangle_{N-1,\beta^{\prime}}$
$\displaystyle\leq
2\mathbb{E}\big{\langle}\mathbbm{1}_{\mathcal{E}_{\delta,\rho}^{c}}\big{\rangle}_{N-1,\beta^{\prime}}+\frac{1}{\delta^{2}}\mathbb{E}\big{\langle}\big{|}R(\tau^{1},\tau^{2})-q_{P}\big{|}\mathbbm{1}_{\Sigma_{N-1}^{\rho}}(\tau^{1})\mathbbm{1}_{\Sigma_{N-1}^{\rho}}(\tau^{2})\big{\rangle}_{N-1,\beta^{\prime}},$
(21)

where in the second expectations on the right-hand side,
$\rho,\tau^{1},\tau^{2}$ are understood as i.i.d. copies from
$G_{N-1,\beta^{\prime}}.$ From (15), the first term in (21) vanishes as
$N\to\infty$ and then $\delta\downarrow 0.$ Next, since both
$R(\rho,\tau^{1})$ and $R(\rho,\tau^{2})$ are at least $q_{P}-\epsilon$, it
follows from (16) that $|R(\tau^{1},\tau^{2})-q_{P}|\leq 2\epsilon$ with
probability nearly one under $\mathbb{E}G_{N-1,\beta^{\prime}}^{\otimes 2}.$
Hence, the second term in (21) also vanishes as $\epsilon\downarrow 0$, and
this completes the proof of the first assertion. As for the second one, we
have

$\displaystyle\mathbb{E}\big{\langle}\big{\langle}\bigl{|}R(\rho,\tau)-q_{P}|\big{\rangle}_{N-1,\beta^{\prime}}^{\rho}\big{\rangle}_{N-1,\beta^{\prime}}\leq
2\mathbb{E}\big{\langle}\mathbbm{1}_{\mathcal{E}_{\delta,\rho}^{c}}\big{\rangle}_{N-1,\beta^{\prime}}+\frac{1}{\delta}\mathbb{E}\big{\langle}\big{|}R(\rho,\tau)-q_{P}\big{|}\mathbbm{1}_{\Sigma_{N-1}^{\rho}}(\tau)\big{\rangle}_{N-1,\beta^{\prime}}.$

Again the first term vanishes in the limit due to (15). From (16) and the
constraint $\Sigma_{N-1}^{\rho}$, we see that $|R(\rho,\tau)-q_{P}|\leq
2\epsilon$ with probability nearly one under
$\mathbb{E}G_{N-1,\beta^{\prime}}^{\otimes 2}.$ Hence the second term also
vanishes, completing our proof. ∎

######  Remark 3.10.

Following the same proof, it can also be shown that

$\lim_{\epsilon\downarrow
0}\limsup_{N\to\infty}\mathbb{E}\bigl{\langle}\bigl{|}\|\langle\sigma\rangle_{N,\beta}^{\alpha}\|^{2}-q_{P}\bigr{|}^{2}\bigr{\rangle}_{N,\beta}=\lim_{\epsilon\downarrow
0}\limsup_{N\to\infty}\mathbb{E}\bigl{\langle}\bigl{|}\bigl{\langle}R(\sigma^{1},\sigma^{2})\bigr{\rangle}_{N,\beta}^{\alpha}-q_{P}\bigr{|}^{2}\bigr{\rangle}_{N,\beta}=0.$

###### Proof of Theorem 3.7.

We prove the central limit theorem via an interpolation argument. Fix $r\geq
1$. For every integer $1\leq m\leq 2r$ and any $\rho,\tau^{m}\in\Sigma_{N-1}$,
consider the interpolation

$\displaystyle x_{\rho}^{m}(t)$
$\displaystyle:=\sqrt{t}S_{\rho}^{m}+\sqrt{1-t}\sqrt{\zeta^{\prime}(1)-\zeta^{\prime}(q_{P})}\xi^{m},\,\,0\leq
t\leq 1,$

where the $\xi^{m}$’s are i.i.d copies of $\xi$ that are also independent of
everything else, and $S_{\rho}^{m}:=\dot{X}_{N,\beta}^{\rho}(\tau^{m}).$ Set

$\displaystyle V(x)$
$\displaystyle:=U(x)-\mathbb{E}U(\xi\sqrt{\zeta^{\prime}(1)-\zeta^{\prime}(q_{P})}),\,\,x\in\mathbb{R}.$

Recall that for any $\alpha\in\Sigma_{N},$ $\rho\in\Sigma_{N-1}$ represents
the first $N-1$ spins. For $0\leq t\leq 1,$ define

$\displaystyle\phi_{N}(t)$
$\displaystyle=\mathbb{E}\left\langle\bigg{\langle}\prod_{m=1}^{2r}V(x_{\rho}^{m}(t))\bigg{\rangle}_{N-1,\beta^{\prime}}^{\rho}\right\rangle_{N,\beta}.$

Evidently the left-hand side of (20) equals $\phi_{N}(1)$.

In order to control $\phi_{N}(1)$, we introduce

$\psi_{N}(t):=\mathbb{E}\left\langle\cosh(X_{N,\beta}(\rho)+h)\bigg{\langle}\prod_{m=1}^{2r}V(x_{\rho}^{m}(t))\bigg{\rangle}_{N-1,\beta^{\prime}}^{\rho}\right\rangle_{N-1,\beta^{\prime}}.$

where $\rho$ and $(\tau^{1},\ldots,\tau^{2r})$ are sampled from the outer and
inner Gibbs measures, $\langle\cdot\rangle_{N-1,\beta^{\prime}}$ and
$\langle\cdot\rangle_{N-1,\beta^{\prime}}^{\rho}$, respectively. Observe that
$\psi_{N}(0)=0$ and that from a change of measure argument as the one in the
proof of Lemma 3.2, we see that $\phi_{N}(1)\leq\psi_{N}(1).$ Consequently,

$\displaystyle\phi_{N}(1)\leq\psi_{N}(1)=\int_{0}^{1}\psi_{N}^{\prime}(t)dt\leq\int_{0}^{1}|\psi_{N}^{\prime}(t)|dt$

and it suffices to estimate $|\psi^{\prime}_{N}(t)|$. To this end, we
differentiate $\psi_{N}$ and use symmetry in $m$ to get

$\displaystyle\psi_{N}^{\prime}(t)$
$\displaystyle=r\mathbb{E}\big{\langle}\big{\langle}\Phi_{N,t}(\rho,\tau^{1},\ldots,\tau^{2r})\big{\rangle}_{N-1,\beta^{\prime}}^{\rho}\big{\rangle}_{N-1,\beta^{\prime}}$
$\displaystyle=r\mathbb{E}\big{\langle}\big{\langle}\mathbb{E}_{g_{\cdot
N,\xi}}\Phi_{N,t}(\rho,\tau^{1},\ldots,\tau^{2r})\big{\rangle}_{N-1,\beta^{\prime}}^{\rho}\big{\rangle}_{N-1,\beta^{\prime}},$

where

$\displaystyle\Phi_{N,t}(\rho,\tau^{1},\ldots,\tau^{2r})$
$\displaystyle:=\cosh(X_{N,\beta}(\rho)+h)V^{\prime}(x_{\rho}^{1}(t))\bigg{(}\frac{S_{\rho}^{1}}{\sqrt{t}}-\frac{\sqrt{\zeta^{\prime}(1)-\zeta^{\prime}(q_{P})}}{\sqrt{1-t}}\xi^{1}\bigg{)}\prod_{\begin{subarray}{c}s=2\end{subarray}}^{2r}V(x_{\rho}^{s}(t))$

and $\mathbb{E}_{g_{\cdot N},\xi}$ is the expectation with respect to
$g_{\cdot N}$ and $\xi^{1}.$ Using Gaussian integration by part with respect
to $g_{\cdot N}$ and $\xi^{1}$ yields that

$\displaystyle\mathbb{E}_{g_{\cdot
N,\xi}}\Phi_{N,t}(\rho,\tau^{1},\ldots,\tau^{2r})$
$\displaystyle=\mathbb{E}_{g_{\cdot
N},\xi}\Big{[}\\!\cosh(X_{N,\beta}(\rho)+h)V^{\prime\prime}(x_{\rho}^{1}(t))\prod_{m=2}^{2r}V(x_{\rho}^{m}(t))\Big{]}\Big{(}\mathbb{E}_{g_{\cdot
N}}\big{[}S_{\rho}^{1}S_{\rho}^{1}\big{]}-(\zeta^{\prime}(1)-\zeta^{\prime}(q_{P}))\Big{)}$
$\displaystyle\qquad\ +(2r-1)\mathbb{E}_{g_{\cdot
N},\xi}\Big{[}\\!\cosh(X_{N,\beta}(\rho)+h)V^{\prime}(x_{\rho}^{1}(t))V^{\prime}(x_{\rho}^{2}(t))\prod_{m=3}^{2r}V(x_{\rho}^{m}(t))\Big{]}\mathbb{E}_{g_{\cdot
N}}\big{[}S_{\rho}^{1}S_{\rho}^{2}\big{]}$ $\displaystyle\qquad\
+\frac{1}{\sqrt{t}}\mathbb{E}_{g_{\cdot
N},\xi}\Big{[}\\!\sinh(X_{N,\beta}(\rho)+h)V^{\prime}(x_{\rho}^{1}(t))\prod_{m=2}^{2r}V(x_{\rho}^{m}(t))\Big{]}\mathbb{E}_{g_{\cdot
N}}\big{[}S_{\rho}^{1}X_{N,\beta}(\rho)\big{]},$

where the term $2r-1$ arises from the symmetry in $m$. Note that
$X_{N,\beta}(\rho)+h$ is a mean-$h$ Gaussian random variable, whose variance
is uniformly bounded for all $N$ and all $\rho\in\Sigma_{N-1}$, which ensures,
for all $k\geq 0$,

$\displaystyle\max_{\rho\in\Sigma_{N-1}}\mathbb{E}_{g_{\cdot
N}}\big{[}\sinh^{2k}(X_{N,\beta}(\rho)+h)\big{]}$ $\displaystyle<\infty,$
$\displaystyle\max_{\rho\in\Sigma_{N-1}}\mathbb{E}_{g_{\cdot
N}}\big{[}\cosh^{2k}(X_{N,\beta}(\rho)+h)\big{]}$ $\displaystyle<\infty.$

Similarly, $x_{\rho}^{m}(t)$ is a mean-zero Gaussian random variable, whose
variance is uniformly bounded for all $N$, all $\rho,\tau^{m}\in\Sigma_{N-1}$,
and all $0\leq t\leq 1$:

$0\leq\text{Var}_{g_{\cdot N,\xi}}(x_{\rho}^{m}(t))=t\text{Var}_{g_{\cdot
N}}(S_{\rho}^{m})+(1-t)(\zeta^{\prime}(1)-\zeta^{\prime}(q_{P}))\leq
4\zeta^{\prime}(1)+(\zeta^{\prime}(1)-\zeta^{\prime}(q_{P}))<\infty.$

With the given assumption on $U$, we then have that for any $k,d\geq 0$ and
$1\leq m\leq 2r,$

$\displaystyle\sup_{\begin{subarray}{c}0\leq t\leq 1,\\\
\rho,\tau^{m}\in\Sigma_{N-1}\end{subarray}}\mathbb{E}_{g_{\cdot
N},\xi}\big{[}V^{(d)}(x_{\rho}^{m}(t))^{2k}\big{]}$ $\displaystyle<\infty.$

Thus, using Hölder’s inequality, the derivative $\psi_{N}^{\prime}(t)$ is
bounded by

$\displaystyle|\psi^{\prime}_{N}(t)|$ $\displaystyle\leq
C\mathbb{E}\big{\langle}\big{\langle}\,\big{|}\mathbb{E}_{g_{\cdot
N}}\big{[}S_{\rho}^{1}S_{\rho}^{1}-(\zeta^{\prime}(1)-\zeta^{\prime}(q_{P}))\big{]}\big{|}\,\big{\rangle}^{\rho}_{N-1,\beta^{\prime}}\big{\rangle}_{N-1,\beta^{\prime}}$
(22)
$\displaystyle+C\mathbb{E}\big{\langle}\big{\langle}\,\big{|}\mathbb{E}_{g_{\cdot
N}}\big{[}S_{\rho}^{1}S_{\rho}^{2}\big{]}\big{|}\,\big{\rangle}^{\rho}_{N-1,\beta^{\prime}}\big{\rangle}_{N-1,\beta^{\prime}}$
(23)
$\displaystyle+\frac{C}{\sqrt{t}}\mathbb{E}\big{\langle}\big{\langle}\big{|}\mathbb{E}_{g_{\cdot
N}}\big{[}S_{\rho}^{1}X_{N,\beta}(\rho)\big{]}\big{|}\big{\rangle}^{\rho}_{N-1,\beta^{\prime}}\big{\rangle}_{N-1,\beta^{\prime}}$
(24)

for some constant $C$ independent of $N$ and $t.$ If we can show that all
these expectations vanish as $N\to\infty$ and then $\epsilon\downarrow 0,$ our
proof is complete since $1/\sqrt{t}$ is integrable on $(0,1].$ For the
remaining of the proof, we handle the three major expectations in the above
display as follows.

Estimation of (22): Write

$\displaystyle\ \ \mathbb{E}_{g_{\cdot
N}}\big{[}S_{\rho}^{1}S_{\rho}^{1}-(\zeta^{\prime}(1)-\zeta^{\prime}(q_{P}))\big{]}$
$\displaystyle=\sum_{p\geq 2}\frac{\beta_{p}^{2}p!}{N^{p-1}}\sum_{1\leq
i_{1}<\cdots<i_{p-1}\leq
N-1}\bigl{(}\tau_{i_{1}}^{1}\cdots\tau_{i_{p-1}}^{1}-\langle\tau_{i_{1}}\cdots\tau_{i_{p-1}}\rangle_{N-1,\beta^{\prime}}^{\rho}\bigr{)}^{2}-(\zeta^{\prime}(1)-\zeta^{\prime}(q_{P}))$
$\displaystyle=\sum_{p\geq
2}\beta_{p}^{2}p\bigg{[}\prod_{l=1}^{p-1}\Bigl{(}\frac{N-l}{N}\Bigr{)}-1\bigg{]}+\bigg{[}\sum_{p\geq
2}\frac{\beta^{2}_{p}p}{N^{p-1}}\\!\sum_{\begin{subarray}{c}i_{1},\ldots,i_{p-1}=1\\\
\text{distinct}\end{subarray}}^{N-1}\langle\tau^{1}_{i_{1}}\cdots\tau^{1}_{i_{p-1}}\tau^{2}_{i_{1}}\cdots\tau^{2}_{i_{p-1}}\rangle_{N-1,\beta^{\prime}}^{\rho}-\zeta^{\prime}(q_{P})\bigg{]}$
$\displaystyle\ \ -2\bigg{[}\sum_{p\geq
2}\frac{\beta^{2}_{p}p}{N^{p-1}}\\!\sum_{\begin{subarray}{c}i_{1},\ldots,i_{p-1}=1\\\
\text{distinct}\end{subarray}}^{N-1}\tau_{i_{1}}^{1}\cdots\tau_{i_{p-1}}^{1}\langle\tau_{i_{1}}\cdots\tau_{i_{p-1}}\rangle_{N-1,\beta^{\prime}}^{\rho}-\zeta^{\prime}(q_{P})\bigg{]}.$
(25)

Note that each sum in (25) is bounded in absolute value by $C_{\beta}$,
introduced at the beginning of Section 1. By the dominated convergence
theorem, the first sum in (25) goes to 0 as $N\to\infty$. The second sum in
(25) is equal to

$\displaystyle\sum_{p\geq
2}\left(\frac{N-1}{N}\right)^{p-1}\beta^{2}_{p}p\,\big{\langle}R^{p-1}(\tau^{1},\tau^{2})\big{\rangle}_{N-1,\beta^{\prime}}^{\rho}-\zeta^{\prime}(q_{P})+o_{N}(1)=\big{\langle}\zeta^{\prime}(R_{12})\big{\rangle}_{N-1,\beta^{\prime}}^{\rho}-\zeta^{\prime}(q_{P})+o_{N}(1),$

where for any $\tau^{1},\tau^{2}\in\Sigma_{N-1},$ we abbreviate
$R_{12}=R(\tau^{1},\tau^{2})$. For the last sum in (25), we take the Gibbs
average under $\langle\cdot\rangle_{N-1,\beta^{\prime}}^{\rho}$ and use
Jensen’s inequality to get

$\displaystyle\ \ \Big{\langle}\Big{|}\sum_{p\geq
2}\frac{\beta^{2}_{p}p}{N^{p-1}}\sum_{\begin{subarray}{c}i_{1},\ldots,i_{p-1}=1\\\
\text{distinct}\end{subarray}}^{N-1}\tau_{i_{1}}^{1}\cdots\tau_{i_{p-1}}^{1}\langle\tau_{i_{1}}^{2}\cdots\tau_{i_{p-1}}^{2}\rangle_{N-1,\beta^{\prime}}^{\rho}-\zeta^{\prime}(q_{P})\Big{|}\Big{\rangle}^{\rho}_{N-1,\beta^{\prime}}$
$\displaystyle\leq\bigg{\langle}\Big{|}\sum_{p\geq
2}\frac{\beta^{2}_{p}p}{N^{p-1}}\sum_{\begin{subarray}{c}i_{1},\ldots,i_{p-1}=1\\\
\text{distinct}\end{subarray}}^{N-1}\tau_{i_{1}}^{1}\cdots\tau_{i_{p-1}}^{1}\tau_{i_{1}}^{2}\cdots\tau_{i_{p-1}}^{2}-\zeta^{\prime}(q_{P})\Big{|}\bigg{\rangle}_{N-1,\beta^{\prime}}^{\rho}$
$\displaystyle=\big{\langle}|\,\zeta^{\prime}(R_{12})-\zeta^{\prime}(q_{P})|\big{\rangle}_{N-1,\beta^{\prime}}^{\rho}+o_{N}(1).$

Thus,

$\displaystyle\ \
\mathbb{E}\left\langle\big{\langle}\,\big{|}\mathbb{E}_{g_{\cdot
N}}\big{[}S_{\rho}^{1}S_{\rho}^{1}-(\zeta^{\prime}(1)-\zeta^{\prime}(q_{P}))\big{]}\big{|}\,\big{\rangle}_{N-1,\beta^{\prime}}^{\rho}\right\rangle_{N-1,\beta^{\prime}}$
$\displaystyle\leq
3\mathbb{E}\left\langle\big{\langle}\bigl{|}\zeta^{\prime}(R_{12})-\zeta^{\prime}(q_{P})\bigr{|}\big{\rangle}_{N-1,\beta^{\prime}}^{\rho}\right\rangle_{N-1,\beta^{\prime}}+o_{N}(1).$
(26)

Therefore, from Lemma 3.9 and the fact that for all $x,y\in[0,1]$,

$\displaystyle|\zeta^{\prime}(x)-\zeta^{\prime}(y)|\leq\zeta^{\prime\prime}(1)|x-y|,$
(27)

the right hand side of (26) also vanishes in the limit $N\to\infty$ and then
$\epsilon\downarrow 0.$

Estimation of (23): With a similar argument as the above control, we see that

$\displaystyle\big{\langle}\big{|}\mathbb{E}_{g_{\cdot
N}}\big{[}S_{\rho}^{1}S_{\rho}^{2}\big{]}\big{|}\big{\rangle}^{\rho}_{N-1,\beta^{\prime}}\leq
4\big{\langle}|\,\zeta^{\prime}(R_{12})-\zeta^{\prime}(q_{P})|\big{\rangle}^{\rho}_{N-1,\beta^{\prime}}+o_{N}(1)$

and Lemma 3.9 and (27) ensure that (23) vanishes in the limit.

Estimation of (24): Note that for any $\rho\in\Sigma_{N-1},$

$\displaystyle\bigl{|}\mathbb{E}_{g_{\cdot
N}}\big{[}S_{\rho}^{1}X_{N,\beta}(\rho)\big{]}\bigr{|}$
$\displaystyle=\biggl{|}\sum_{p\geq
2}\frac{\beta_{p}^{2}p}{N^{p-1}}\sum_{\begin{subarray}{c}i_{1},\ldots,i_{p-1}=1\\\
\text{distinct}\end{subarray}}^{N-1}\bigl{(}\tau_{i_{1}}^{1}\cdots\tau_{i_{p-1}}^{1}-\langle\tau_{i_{1}}^{1}\cdots\tau_{i_{p-1}}^{1}\rangle^{\rho}_{N-1,\beta^{\prime}}\bigr{)}\rho_{i_{1}}\cdots\rho_{i_{p-1}}\bigg{|}$

Again, we can use Jensen’s inequality to obtain

$\displaystyle\mathbb{E}\bigl{\langle}\bigl{\langle}\bigl{|}\mathbb{E}_{g_{\cdot
N}}\big{[}S_{\rho}^{1}X_{N,\beta}(\rho)\big{]}\bigr{|}\bigr{\rangle}_{N-1,\beta^{\prime}}^{\rho}\big{\rangle}_{N-1,\beta^{\prime}}\leq\mathbb{E}\bigl{\langle}\bigl{\langle}\bigl{|}\zeta^{\prime}(R(\tau^{1},\rho))-\zeta^{\prime}(R(\tau^{1},\rho))\bigr{|}\bigr{\rangle}_{N-1,\beta^{\prime}}^{\rho}\big{\rangle}_{N-1,\beta^{\prime}}+o_{N}(1).$

From Lemma 3.9, this bound vanishes in the limit. ∎

###  3.3 Proof of Theorem 3.1

We begin by establishing the following proposition.

######  Proposition 3.11.

$\displaystyle\lim_{\epsilon\downarrow
0}\limsup_{N\to\infty}\mathbb{E}\left\langle\big{[}\langle\sigma_{N}\rangle_{N,\beta}^{\alpha}-\tanh\big{(}\langle
X_{N,\beta}(\tau)\rangle_{N-1,\beta^{\prime}}^{\rho}+h\big{)}\big{]}^{2}\right\rangle_{N,\beta}=0,$

where $\epsilon\downarrow 0$ along a sequence such that $q_{P}-\epsilon$ is
always a point of continuity for $\mu_{P}$.

###### Proof.

By Proposition 3.6, it is sufficient to show that

$\displaystyle\lim_{\epsilon\downarrow
0}\limsup_{N\to\infty}\mathbb{E}\bigg{\langle}\bigg{[}\frac{\langle\sinh(X_{N,\beta}(\tau)+h)\rangle_{N-1,\beta^{\prime}}^{\rho}}{\langle\cosh(X_{N,\beta}(\tau)+h)\rangle_{N-1,\beta^{\prime}}^{\rho}}-\tanh\big{(}\langle
X_{N,\beta}(\tau)\rangle_{N-1,\beta^{\prime}}^{\rho}+h\big{)}\bigg{]}^{2}\bigg{\rangle}_{N,\beta}=0.$

Notice that

$\tanh(x)=\frac{\sinh(x)}{\cosh(x)},\ \ |\sinh(x)|\leq\cosh(x),\ \text{and
}\cosh(x)\geq 1.$

From these, it suffices to show that for $\varepsilon\in\\{-1,1\\},$

$\displaystyle\lim_{\epsilon\downarrow
0}\limsup_{N\to\infty}\mathbb{E}\left\langle\Big{[}\big{\langle}e^{\varepsilon
X_{N,\beta}(\tau)}\big{\rangle}^{\rho}_{N-1,\beta^{\prime}}-\exp\Big{\\{}\varepsilon\big{\langle}X_{N,\beta}(\tau)\big{\rangle}^{\rho}_{N-1,\beta^{\prime}}+\frac{1}{2}\big{[}\zeta^{\prime}(1)-\zeta^{\prime}(q_{P})\big{]}\Big{\\}}\Big{]}^{2}\right\rangle_{N,\beta}=0.$

Note that Theorem 3.7 with $U(x)=e^{\varepsilon x}$ for
$\varepsilon\in\\{+1,-1\\}$ readily implies that

$\displaystyle\lim_{\epsilon\downarrow
0}\limsup_{N\to\infty}\mathbb{E}\left\langle\Big{[}\big{\langle}e^{\varepsilon\dot{X}_{N,\beta}^{\rho}(\tau)}\big{\rangle}_{N-1,\beta^{\prime}}^{\rho}-\exp\Big{\\{}\frac{1}{2}\big{[}\zeta^{\prime}(1)-\zeta^{\prime}(q_{P})\big{]}\Big{\\}}\Big{]}^{4}\right\rangle_{N,\beta}=0.$
(28)

For any $\rho\in\Sigma_{N-1}$, we can decompose
$X_{N,\beta}(\tau)=\dot{X}_{N,\beta}^{\rho}(\tau)+\langle
X_{N,\beta}(\tau)\rangle^{\rho}_{N-1,\beta^{\prime}}$ to get

$\displaystyle\ \ \mathbb{E}\left\langle\Big{[}\big{\langle}e^{\varepsilon
X_{N,\beta}(\tau)}\big{\rangle}^{\rho}_{N-1,\beta^{\prime}}-\exp\Big{\\{}\varepsilon\big{\langle}X_{N,\beta}(\tau)\big{\rangle}^{\rho}_{N-1,\beta^{\prime}}+\frac{1}{2}\big{[}\zeta^{\prime}(1)-\zeta^{\prime}(q_{P})\big{]}\Big{\\}}\Big{]}^{2}\right\rangle_{N,\beta}$
$\displaystyle=\mathbb{E}\left\langle e^{2\varepsilon\langle
X_{N,\beta}(\tau)\rangle^{\rho}_{N-1,\beta^{\prime}}}\Big{[}\big{\langle}e^{\varepsilon\dot{X}_{N,\beta}(\tau)}\big{\rangle}^{\rho}_{N-1,\beta^{\prime}}-\exp\Big{\\{}\frac{1}{2}\big{[}\zeta^{\prime}(1)-\zeta^{\prime}(q_{P})\big{]}\Big{\\}}\Big{]}^{2}\right\rangle_{N,\beta}$
$\displaystyle\leq\bigg{\\{}\mathbb{E}\big{\langle}e^{4\varepsilon\langle
X_{N,\beta}(\tau)\rangle^{\rho}_{N-1,\beta^{\prime}}}\big{\rangle}_{N,\beta}\mathbb{E}\left\langle\Big{[}\big{\langle}e^{\varepsilon\dot{X}_{N,\beta}(\tau)}\big{\rangle}^{\rho}_{N-1,\beta^{\prime}}-\exp\Big{\\{}\frac{1}{2}\big{[}\zeta^{\prime}(1)-\zeta^{\prime}(q_{P})\big{]}\Big{\\}}\Big{]}^{4}\right\rangle_{N,\beta}\bigg{\\}}^{1/2}.$

Here the second term vanishes as ensured by (28). By a change of measure as in
the proof of Lemma 3.2, the first expectation is bounded by

$\displaystyle\mathbb{E}\bigg{[}\sum_{\rho\in\Sigma_{N-1}}G_{N-1,\beta^{\prime}}(\rho)\mathbb{E}_{g_{\cdot
N}}\Bigl{[}\cosh(X_{N,\beta}(\rho)+h)e^{4\varepsilon\langle
X_{N,\beta}(\tau)\rangle^{\rho}_{N-1,\beta^{\prime}}}\Bigr{]}\bigg{]}$
$\displaystyle\leq\mathbb{E}\sum_{\rho\in\Sigma_{N-1}}G_{N-1,\beta^{\prime}}(\rho)\left\\{\mathbb{E}_{g_{\cdot
N}}[\cosh^{2}(X_{N,\beta}(\rho)+h)]\,\mathbb{E}_{g_{\cdot
N}}[e^{8\varepsilon\langle
X_{N,\beta}(\tau)\rangle^{\rho}_{N-1,\beta^{\prime}}}]\right\\}^{1/2}\leq C$

for some constant $C>0$ independent of $N$ and $\epsilon.$ This completes our
proof. ∎

We now turn to the proof of Theorem 3.1. From Proposition 3.11 and noting that
$|\tanh(x)-\tanh(x^{\prime})|^{2}\leq 2|x-x^{\prime}|$, it remains to show
that

$\displaystyle\lim_{\epsilon\downarrow
0}\limsup_{N\to\infty}\mathbb{E}\left\langle\Big{|}X_{N,\beta}(\langle\tau\rangle_{N-1,\beta^{\prime}}^{\rho})-\big{\langle}X_{N,\beta}(\tau)\big{\rangle}_{N-1,\beta^{\prime}}^{\rho}\Big{|}\right\rangle_{N,\beta}=0.$
(29)

As in the proof of Lemma 3.2, we apply a change of measure to deal with the
dependence of $\langle\cdot\rangle_{N,\beta}$ on the disorders $g_{\cdot N}$,

$\displaystyle\ \
\mathbb{E}\left\langle\Big{|}X_{N,\beta}\bigl{(}\big{\langle}\tau\big{\rangle}_{N-1,\beta^{\prime}}^{\rho}\bigr{)}-\big{\langle}X_{N,\beta}(\tau)\big{\rangle}_{N-1,\beta^{\prime}}^{\rho}\Big{|}\right\rangle_{N,\beta}$
$\displaystyle\leq\mathbb{E}\left\langle\cosh(X_{N,\beta}(\rho)+h)\Big{|}X_{N,\beta}\bigl{(}\big{\langle}\tau\big{\rangle}_{N-1,\beta^{\prime}}^{\rho}\bigr{)}-\big{\langle}X_{N,\beta}(\tau)\big{\rangle}_{N-1,\beta^{\prime}}^{\rho}\Big{|}\right\rangle_{N-1,\beta^{\prime}}$
$\displaystyle\leq\mathbb{E}\bigg{[}\left\langle\Big{[}\mathbb{E}_{g_{\cdot
N}}\cosh^{2}(X_{N,\beta}(\rho)+h)\Big{]}^{1/2}\Big{\\{}\mathbb{E}_{g_{\cdot
N}}\Big{[}X_{N,\beta}\bigl{(}\big{\langle}\tau\big{\rangle}_{N-1,\beta^{\prime}}^{\rho}\bigr{)}-\big{\langle}X_{N,\beta}(\tau)\big{\rangle}_{N-1,\beta^{\prime}}^{\rho}\Big{]}^{2}\Big{\\}}^{1/2}\right\rangle_{N-1,\beta^{\prime}}\bigg{]}.$

Here, uniformly in $\rho\in\Sigma_{N-1},$

$\displaystyle\mathbb{E}_{g_{\cdot N}}\cosh^{2}(X_{N,\beta}(\rho)+h)\leq
e^{2C_{\beta}}\cosh^{2}(h)$

and

$\displaystyle\mathbb{E}_{g_{\cdot
N}}\Big{[}X_{N,\beta}\bigl{(}\big{\langle}\tau\big{\rangle}_{N-1,\beta^{\prime}}^{\rho}\bigr{)}-\big{\langle}X_{N,\beta}(\tau)\big{\rangle}_{N-1,\beta^{\prime}}^{\rho}\Big{]}^{2}$
$\displaystyle\leq\sum_{p\geq
2}\frac{\beta_{p}^{2}p}{N^{p-1}}\sum_{\begin{subarray}{c}i_{1},\ldots,i_{p-1}\\\
\text{distinct}\end{subarray}}\bigg{(}\big{\langle}\tau_{i_{1}}\big{\rangle}_{N-1,\beta^{\prime}}^{\rho}\cdots\big{\langle}\tau_{i_{p-1}}\big{\rangle}_{N-1,\beta^{\prime}}^{\rho}-\big{\langle}\tau_{i_{1}}\cdots\tau_{i_{p-1}}\big{\rangle}_{N-1,\beta^{\prime}}^{\rho}\bigg{)}^{2}$
$\displaystyle=\zeta^{\prime}\bigl{(}\big{\langle}R_{1,2}\big{\rangle}_{N-1,\beta^{\prime}}^{\rho}\bigr{)}+\big{\langle}\zeta^{\prime}(R_{1,2})\big{\rangle}_{N-1,\beta^{\prime}}^{\rho}-2\sum_{p\geq
2}\beta_{p}^{2}p\big{\langle}R_{1,p}R_{2,p}\cdots
R_{p-1,p}\big{\rangle}_{N-1,\beta^{\prime}}^{\rho}+o_{N}(1).$

where we write $R_{\ell,\ell^{\prime}}:=R(\tau^{\ell},\tau^{\ell^{\prime}}).$
From Lemma 3.9 and (27), we readily see that

$\displaystyle\lim_{\epsilon\downarrow
0}\limsup_{N\to\infty}\mathbb{E}\bigl{\langle}\bigl{|}\zeta^{\prime}\bigl{(}\big{\langle}R_{1,2}\big{\rangle}_{N-1,\beta^{\prime}}^{\rho}\bigr{)}-\zeta^{\prime}(q_{P})\bigr{|}\bigr{\rangle}_{N-1,\beta^{\prime}}=0,$
$\displaystyle\lim_{\epsilon\downarrow
0}\limsup_{N\to\infty}\mathbb{E}\bigl{\langle}\bigl{|}\big{\langle}\zeta^{\prime}\bigl{(}R_{1,2})\big{\rangle}_{N-1,\beta^{\prime}}^{\rho}-\zeta^{\prime}(q_{P})\bigr{|}\bigr{\rangle}_{N-1,\beta^{\prime}}=0.$

Finally, note that

$\displaystyle\bigl{|}R_{1,p}R_{2,p}\cdots R_{p-1,p}-q_{P}^{p-1}\bigr{|}$
$\displaystyle\leq\sum_{l=1}^{p-1}|R_{l,p}-q_{P}|.$

It follows from Lemma 3.9 and the symmetry in $l$ that

$\displaystyle\mathbb{E}\Bigl{\langle}\Bigl{|}\sum_{p\geq
2}\beta_{p}^{2}p\bigl{(}\big{\langle}R_{1,p}R_{2,p}\cdots
R_{p-1,p}\big{\rangle}_{N-1,\beta^{\prime}}^{\rho}-q_{P}^{p-1}\bigr{)}\Bigr{|}\Bigr{\rangle}_{N-1,\beta^{\prime}}$
$\displaystyle\leq\bigg{(}\sum_{p\geq
2}\beta_{p}^{2}p^{2}\bigg{)}\mathbb{E}\bigl{\langle}\bigl{\langle}\bigl{|}R_{1,2}-q_{P}\bigr{|}\bigr{\rangle}_{N-1,\beta^{\prime}}^{\rho}\bigr{\rangle}_{N-1,\beta^{\prime}}\to
0,$

as we send $N\to\infty$ and $\epsilon\downarrow 0$. Combining these together
validates (29) and completes our proof.

##  4 Cavity computation for
$X_{N,\beta}(\langle\sigma\rangle_{N,\beta}^{\alpha})$

To derive the TAP equation from the cavity equation, the next step is to
represent $X_{N,\beta}(\langle\tau\rangle_{N-1,\beta^{\prime}}^{\rho})$ in
Theorem 3.1 as the difference between
$X_{N,\beta}(\langle\sigma\rangle^{\alpha}_{N,\beta})$ and the Onsager
correction term $\beta^{2}\zeta^{\prime\prime}(q_{P})(1-q_{P}).$ To this end,
in this section, we will first perform a cavity computation for
$X_{N,\beta}(\langle\sigma\rangle^{\alpha}_{N,\beta})$, like we did in
Proposition 3.6 for $\langle\sigma_{N}\rangle_{N,\beta}^{\alpha}.$

First of all, write

$\displaystyle
X_{N,\beta}(\langle\sigma\rangle_{N,\beta}^{\alpha})=\sum_{p\geq
2}\frac{\beta_{p}\sqrt{p!}}{N^{\frac{p-1}{2}}}\sum_{1\leq
i_{1}<\cdots<i_{p-1}\leq
N-1}g_{i_{1},\ldots,i_{p-1},N}\langle\sigma_{i_{1}}\rangle_{N,\beta}^{\alpha}\cdots\langle\sigma_{i_{p-1}}\rangle_{N,\beta}^{\alpha}.$

For each local magnetization on the right hand side, we can follow the same
procedure in Section 3 and write $\sigma$ as $\sigma=(\tau,\sigma_{N})$ to
express

$\langle\sigma_{j}\rangle_{N,\beta}^{\alpha}=\frac{\big{\langle}\tau_{j}e^{X_{N,\beta}(\tau)+h}\mathbbm{1}_{\Sigma_{N}^{\alpha}}((\tau,1))+e^{-X_{N,\beta}(\tau)-h}\mathbbm{1}_{\Sigma_{N}^{\alpha}}((\tau,-1))\big{\rangle}_{N-1,\beta^{\prime}}}{\big{\langle}e^{X_{N,\beta}(\tau)+h}\mathbbm{1}_{\Sigma_{N}^{\alpha}}((\tau,1))+e^{-X_{N,\beta}(\tau)-h}\mathbbm{1}_{\Sigma_{N}^{\alpha}}((\tau,-1))\big{\rangle}_{N-1,\beta^{\prime}}}$

for $1\leq j\leq N-1.$ Due to the nested structure (LABEL:eqn:nestedsets) and
Lemma 3.5, one is tempting to approximate all indicators above by
$\mathbbm{1}_{\Sigma_{N-1}^{\rho}}(\tau)$ for $\alpha=(\rho,\alpha_{N})$ and
write that

$\displaystyle\langle\sigma_{j}\rangle_{N,\beta}^{\alpha}\approx
s^{\rho}_{j}:=\frac{\big{\langle}\tau_{j}\cosh(X_{N,\beta}(\tau)+h)\big{\rangle}_{N-1,\beta^{\prime}}^{\rho}}{\big{\langle}\cosh(X_{N,\beta}(\tau)+h)\big{\rangle}_{N-1,\beta^{\prime}}^{\rho}}.$
(30)

for each $j=1,2,\ldots,N-1$. Write
$s^{\rho}=(s_{1}^{\rho},\ldots,s_{N-1}^{\rho})$. The main goal in this section
is to establish this approximation and proved the following proposition.

######  Proposition 4.1.

For any $\epsilon$ such that $q_{P}-\epsilon$ is a point of continuity of
$\mu_{P},$ we have that

$\displaystyle\lim_{N\to\infty}\mathbb{E}\big{\langle}\big{[}X_{N,\beta}(\langle\sigma\rangle_{N,\beta}^{\alpha})-X_{N,\beta}(s^{\rho})\big{]}^{2}\big{\rangle}_{N,\beta}=0.$
(31)

We will prove (31) in two steps. Firstly, we show that
$X_{N,\beta}(\langle\sigma\rangle_{N,\beta}^{\alpha})$ and
$X_{N,\beta}(s^{\rho})$ will not differ too much if we truncate the infinite
sum over $p\geq 2$ to a finite sum over $2\leq p\leq p_{0}$, provided that
$p_{0}$ is sufficiently large. This will require some moment and truncation
controls on $X_{N,\beta}(\langle\sigma\rangle_{N,\beta}^{\alpha})$ and
$X_{N,\beta}(s^{\rho})$ that we establish in the next subsection.

###  4.1 Moment and truncation controls of
$X_{N,\beta}(\langle\sigma\rangle_{N,\beta}^{\alpha})$ and
$X_{N,\beta}(s^{\rho})$

Write $X_{N,\beta}(\langle\sigma\rangle_{N}^{\alpha})=\sum_{p\geq
2}A^{\alpha}_{p}$ and $X_{N,\beta}(s^{\rho})=\sum_{p\geq 2}B^{\rho}_{p}$,
where

$\displaystyle A^{\alpha}_{p}$
$\displaystyle:=\frac{\beta_{p}\sqrt{p!}}{N^{\frac{p-1}{2}}}\sum_{1\leq
i_{1}<\cdots<i_{p-1}\leq
N-1}g_{i_{1},\ldots,i_{p-1},N}\langle\sigma_{i_{1}}\rangle_{N,\beta}^{\alpha}\cdots\langle\sigma_{i_{p-1}}\rangle_{N,\beta}^{\alpha},$
(32) $\displaystyle B^{\rho}_{p}$
$\displaystyle:=\frac{\beta_{p}\sqrt{p!}}{N^{\frac{p-1}{2}}}\sum_{1\leq
i_{1}<\cdots<i_{p-1}\leq N-1}g_{i_{1},\ldots,i_{p-1},N}s_{i_{1}}^{\rho}\cdots
s_{i_{p-1}}^{\rho}.$ (33)

The following proposition provides a control on the truncation of the above
two random series.

######  Proposition 4.2.

For any $\delta>0$, there exists $p_{0}>0$ such that for all $N\geq 1$ and
small enough $\epsilon>0,$

$\displaystyle\mathbb{E}\bigg{\langle}\Big{[}\sum_{p>p_{0}}A_{p}^{\alpha}\Big{]}^{4}\bigg{\rangle}_{N,\beta}$
$\displaystyle\leq\delta\quad\mbox{and}\quad\mathbb{E}\bigg{\langle}\Big{[}\sum_{p>p_{0}}B_{p}^{\rho}\Big{]}^{4}\bigg{\rangle}_{N,\beta}\leq\delta.$

Recall the definitions of $A_{\ominus}^{\rho}$ and
$W_{N-1,\beta^{\prime}}(A_{\ominus}^{\rho})$ from Section 3. As we have seen
from Lemma 3.3 and Remark 3.4, the Gibbs probability of the event
$W_{N-1,\beta^{\prime}}(A_{\ominus}^{\rho})<\delta$ is essentially negligible,
the next proposition further shows that the second moments of
$X_{N,\beta}(\langle\sigma\rangle_{N,\beta}^{\alpha})$ and
$X_{N,\beta}(s^{\rho})$ on this event remain negligible.

######  Proposition 4.3.

For any $\epsilon>0,$ we have that

$\displaystyle\lim_{\delta\downarrow
0}\limsup_{N\to\infty}\mathbb{E}\big{\langle}\big{[}X_{N,\beta}(\langle\sigma\rangle_{N,\beta}^{\alpha})\mathbbm{1}_{\\{W_{N-1,\beta^{\prime}}(A_{\ominus}^{\rho})<\delta\\}}\big{]}^{2}\big{\rangle}_{N,\beta}$
$\displaystyle=0,$ (34) $\displaystyle\lim_{\delta\downarrow
0}\limsup_{N\to\infty}\mathbb{E}\big{\langle}\big{[}X_{N,\beta}(s^{\rho})\mathbbm{1}_{\\{W_{N-1,\beta^{\prime}}(A_{\ominus}^{\rho})<\delta\\}}\big{]}^{2}\big{\rangle}_{N,\beta}$
$\displaystyle=0.$ (35)

The proofs of Propositions 4.2 and 4.3 are based on the following lemma:

######  Lemma 4.4.

There exists a constant $K=K(\beta,h)>0$ such that for all $N\geq 1$ and small
$\epsilon>0,$

$\displaystyle\sum_{p\geq
2}\big{(}\mathbb{E}\big{\langle}|A_{p}^{\alpha}|^{4}\big{\rangle}_{N,\beta}\big{)}^{\frac{1}{4}}\leq
K\quad\mbox{and}\quad\sum_{p\geq
2}\big{(}\mathbb{E}\big{\langle}|B_{p}^{\rho}|^{4}\big{\rangle}_{N,\beta}\big{)}^{\frac{1}{4}}\leq
K.$

###### Proof.

For notation simplicity, we suppress the superscript $\alpha$ and write
$A_{p}^{\alpha}$ as $A_{p}$. We handle the series of $A_{p}$ first. Note that
for each $p$, we only need to consider the case, $N\geq p$, otherwise
$A_{p}=0$ by the definition (32). Write

$\displaystyle\mathbb{E}\big{\langle}A_{p}^{4}\big{\rangle}_{N,\beta}$
$\displaystyle=\mathbb{E}\bigg{\langle}\Big{(}\frac{\beta_{p}^{2}p!}{N^{p-1}}\Big{)}^{2}\Big{(}\sum_{1\leq
i_{1}<\cdots<i_{p-1}\leq
N-1}g_{i_{1},\ldots,i_{p-1},N}\langle\sigma_{i_{1}}\rangle_{N,\beta}^{\alpha}\cdots\langle\sigma_{i_{p-1}}\rangle_{N,\beta}^{\alpha}\Big{)}^{4}\bigg{\rangle}_{N,\beta}$
$\displaystyle=\Big{(}\frac{\beta_{p}^{2}p!}{N^{p-1}}\Big{)}^{2}\sum_{\alpha\in\Sigma_{N}}\sum_{\mathbf{i}_{1},\ldots,\mathbf{i}_{4}}\mathbb{E}\Big{[}G_{N,\beta}(\alpha)\prod_{l=1}^{4}g_{\mathbf{i}_{l},N}\prod_{k=1}^{p-1}\langle\sigma_{i_{l,k}}\rangle_{N,\beta}^{\alpha}\Big{]},$
(36)

where the second sum is over all $\mathbf{i}_{l}=(i_{l,1},\cdots,i_{l,p-1})$,
for $l=1,2,3,4$ that are $(p-1)$-tuples with strictly increasing coordinates
from $\\{1,2,\ldots,N-1\\}^{p-1}$. We note that there are $\binom{N-1}{p-1}$
choices for each $\mathbf{i}_{l}$. Write $g_{i_{l,1},\ldots,i_{l,p-1},N}$ as
$g_{\mathbf{i}_{l},N}$ and let $\Delta_{p}:=\beta_{p}\sqrt{p!}{N^{-(p-1)/2}}.$
We have

$\displaystyle\Delta_{p}^{2}\binom{N-1}{p-1}$
$\displaystyle=\frac{\beta_{p}^{2}p!}{N^{p-1}}\,\frac{(N-1)(N-2)\cdots(N-(p-1))}{(p-1)!}$
$\displaystyle=\beta_{p}^{2}p\frac{N-1}{N}\cdot\frac{N-2}{N}\cdots\frac{N-(p-1)}{N}\leq\beta_{p}^{2}p.$

Also, there exists a constant $C>0$ independent of $p,N,\mathbf{i}$ such that
for any $0\leq k_{1},k_{2},k_{3},k_{4}\leq 4$, if we let
$d=k_{1}+k_{2}+k_{3}+k_{4},$ then

$\displaystyle\Bigl{|}\partial^{k_{1}}_{g_{\mathbf{i}_{1},N}}\partial^{k_{2}}_{g_{\mathbf{i}_{2},N}}\partial^{k_{3}}_{g_{\mathbf{i}_{3},N}}\partial^{k_{4}}_{g_{\mathbf{i}_{4},N}}G_{N,\beta}(\alpha)\Bigr{|}$
$\displaystyle\leq C\Delta_{p}^{d}G_{N,\beta}(\alpha)\leq
C\Delta_{p}^{d}(p-1)^{d}G_{N,\beta}(\alpha)$ (37)

and

$\displaystyle\Bigl{|}\partial^{k_{1}}_{g_{\mathbf{i}_{1},N}}\partial^{k_{2}}_{g_{\mathbf{i}_{2},N}}\partial^{k_{3}}_{g_{\mathbf{i}_{3},N}}\partial^{k_{4}}_{g_{\mathbf{i}_{4},N}}\prod_{l=1}^{4}\prod_{k=1}^{p-1}\langle\sigma_{i_{l,k}}\rangle_{N,\beta}^{\alpha}\Bigr{|}$
$\displaystyle\leq C\Delta_{p}^{d}(p-1)^{d}.$ (38)

These can be established by an induction argument on $d.$ Now we divide the
collection of $(\mathbf{i}_{1},\mathbf{i}_{2},\mathbf{i}_{3},\mathbf{i}_{4})$
into three cases and compute, respectively, an upper bound for the summand in
(36) under each case. In the following discussion,
$C_{1},C_{1}^{\prime},C_{2},C_{2}^{\prime},\ldots$ are absolute constants
independent of $N$ and $p.$

  * •

Case I: all 4 tuples are distinct. Applying Gaussian integration by part and
the chain rule, we get

$\displaystyle\ \ \
\Big{|}\mathbb{E}G_{N,\beta}(\alpha)g_{\mathbf{i}_{1},N}g_{\mathbf{i}_{2},N}g_{\mathbf{i}_{3},N}g_{\mathbf{i}_{4},N}\prod_{l=1}^{4}\prod_{k=1}^{p-1}\langle\sigma_{i_{l,k}}\rangle_{N,\beta}^{\alpha}\Big{|}\leq
C_{1}\Delta_{p}^{4}(p-1)^{4}\mathbb{E}G_{N,\beta}(\alpha).$

Since the number of choices for
$(\mathbf{i}_{1},\mathbf{i}_{2},\mathbf{i}_{3},\mathbf{i}_{4})$ in Case I are
no more than $\binom{N-1}{p-1}^{4}$, the summation in (36) for Case I is
bounded by

$\displaystyle\Delta_{p}^{4}\sum_{\alpha\in\Sigma_{N}}\sum_{{\tiny\rm
Case\,\,I}}\Big{|}\mathbb{E}G_{N,\beta}(\alpha)g_{\mathbf{i}_{1},N}g_{\mathbf{i}_{2},N}g_{\mathbf{i}_{3},N}g_{\mathbf{i}_{4},N}\prod_{l=1}^{4}\prod_{k=1}^{p-1}\langle\sigma_{i_{l,k}}\rangle_{N,\beta}^{\alpha}\Big{|}$
$\displaystyle\leq\Delta_{p}^{4}\cdot\binom{N-1}{p-1}^{4}\cdot
C_{1}\Delta_{p}^{4}(p-1)^{4}\leq C_{1}\beta_{p}^{8}p^{8}.$

  * •

Case II: there are three distinct tuples in
$(\mathbf{i}_{1},\mathbf{i}_{2},\mathbf{i}_{3},\mathbf{i}_{4})$. Without loss
of generality, suppose $\mathbf{i}_{1}=\mathbf{i}_{2}$ and they are both
different from distinct $\mathbf{i}_{3},\mathbf{i}_{4}$. In this case, again
using Gaussian integration by part twice and the chain rule, each summand in
(36) is bounded in absolute value by

$\Bigl{|}\mathbb{E}G_{N,\beta}(\alpha)g^{2}_{\mathbf{i}_{1},N}g_{\mathbf{i}_{3},N}g_{\mathbf{i}_{4},N}\prod_{l=1}^{4}\prod_{k=1}^{p-1}\langle\sigma_{i_{l,k}}\rangle_{N,\beta}^{\alpha}\Bigr{|}\leq
C_{2}\Delta_{p}^{2}(p-1)^{2}\mathbb{E}g_{\mathbf{i}_{1},N}^{2}G_{N,\beta}(\alpha).$

It follows that

$\displaystyle\Delta_{p}^{4}\sum_{\alpha\in\Sigma_{N}}\sum_{{\tiny\rm
Case\,\,II}}\Big{|}\mathbb{E}G_{N,\beta}(\alpha)g_{\mathbf{i}_{1},N}g_{\mathbf{i}_{2},N}g_{\mathbf{i}_{3},N}g_{\mathbf{i}_{4},N}\prod_{l=1}^{4}\prod_{k=1}^{p-1}\langle\sigma_{i_{l,k}}\rangle_{N,\beta}^{\alpha}\Big{|}$
$\displaystyle\leq
C_{2}^{\prime}\Delta_{p}^{4}\cdot\binom{N-1}{p-1}^{3}\cdot\Delta_{p}^{2}(p-1)^{2}\cdot\mathbb{E}g_{\mathbf{i}_{1},N}^{2}$
$\displaystyle\leq
C_{2}^{\prime}\Delta_{p}^{6}\binom{N-1}{p-1}^{3}(p-1)^{2}\leq
C_{2}^{\prime}\beta_{p}^{6}p^{5}$

  * •

Case III: there are no more than two distinct tuples. In this case, we have
three possibilities, each bounded in absolute value respectively as follows.

$\displaystyle\Bigl{|}\mathbb{E}G_{N,\beta}(\alpha)g^{2}_{\mathbf{i}_{1},N}g^{2}_{\mathbf{i}_{2},N}\prod_{l=1}^{4}\prod_{k=1}^{p-1}\langle\sigma_{i_{l,k}}\rangle_{N,\beta}^{\alpha}\Bigr{|}$
$\displaystyle\leq\mathbb{E}G_{N,\beta}(\alpha)g^{2}_{\mathbf{i}_{1},N}g^{2}_{\mathbf{i}_{2},N},$
$\displaystyle\Bigl{|}\mathbb{E}G_{N,\beta}(\alpha)g^{1}_{\mathbf{i}_{1},N}g^{3}_{\mathbf{i}_{2},N}\prod_{l=1}^{4}\prod_{k=1}^{p-1}\langle\sigma_{i_{l,k}}\rangle_{N,\beta}^{\alpha}\Bigr{|}$
$\displaystyle\leq\mathbb{E}G_{N,\beta}(\alpha)\left|g^{1}_{\mathbf{i}_{1},N}g^{3}_{\mathbf{i}_{2},N}\right|,$
$\displaystyle\Bigl{|}\mathbb{E}G_{N,\beta}(\alpha)g^{4}_{\mathbf{i}_{1},N}\prod_{l=1}^{4}\prod_{k=1}^{p-1}\langle\sigma_{i_{l,k}}\rangle_{N,\beta}^{\alpha}\Bigr{|}$
$\displaystyle\leq\mathbb{E}G_{N,\beta}(\alpha)g^{4}_{\mathbf{i}_{1},N}.$

Consequently,

$\displaystyle\Delta_{p}^{4}\sum_{\alpha\in\Sigma_{N}}\sum_{{\tiny\rm
Case\,\,III}}\Big{|}\mathbb{E}G_{N,\beta}(\alpha)g_{\mathbf{i}_{1},N}g_{\mathbf{i}_{2},N}g_{\mathbf{i}_{3},N}g_{\mathbf{i}_{4},N}\prod_{l=1}^{4}\prod_{k=1}^{p-1}\langle\sigma_{i_{l,k}}\rangle_{N,\beta}^{\alpha}\Big{|}$
$\displaystyle\leq
C_{3}\Delta_{p}^{4}\cdot\binom{N-1}{p-1}^{2}\cdot\left[\mathbb{E}g^{2}_{\mathbf{i}_{1},N}g^{2}_{\mathbf{i}_{2},N}+\mathbb{E}\left|g^{1}_{\mathbf{i}_{1},N}g^{3}_{\mathbf{i}_{2},N}\right|+\mathbb{E}g^{4}_{\mathbf{i}_{1},N}\right]$
$\displaystyle\leq C_{3}^{\prime}\Delta_{p}^{4}\cdot\binom{N-1}{p-1}^{2}\leq
C_{3}^{\prime}\beta_{p}^{4}p^{2}.$

Combining all three cases, we have

$\displaystyle\mathbb{E}\big{\langle}A_{p}^{4}\big{\rangle}_{N,\beta}$
$\displaystyle\leq
C_{4}\bigl{(}\beta_{p}^{4}p^{2}+\beta_{p}^{6}p^{5}+\beta_{p}^{8}p^{8}\bigr{)}.$

Since $\sum_{p\geq 2}2^{p}\beta_{p}^{2}<\infty$, we have
$\beta_{p}^{2}=o(2^{-p})$ as $p\to\infty$. Choosing $p_{0}$ large enough such
that $\beta_{p}\leq 2^{-p/2}$ and $p^{2}<2^{p/4}$ for all $p>p_{0}$, it
follows that

$\displaystyle\sum_{p>p_{0}}\bigl{(}\mathbb{E}\big{\langle}A_{p}^{4}\big{\rangle}_{N,\beta}\bigr{)}^{1/4}$
$\displaystyle\leq
C_{4}\sum_{p>p_{0}}\bigl{(}\beta_{p}^{4}p^{2}+\beta_{p}^{6}p^{5}+\beta_{p}^{8}p^{8}\bigr{)}^{1/4}\leq
3C_{4}\sum_{p>p_{0}}\beta_{p}p^{2}$ $\displaystyle\leq
3C_{4}\sum_{p>p_{0}}2^{-p/2}p^{2}\leq 3C_{4}\sum_{p>p_{0}}2^{-p/4}<\infty.$

For the summability for the series of $B_{p}^{\rho}$, the proof is essentially
the same; the only change is that in (36),
$\langle\sigma_{j}\rangle_{N,\beta}^{\alpha}$ will be replaced by
$s_{j}^{\rho}$. Notice that $|s_{j}^{\rho}|\leq 1$ and any partial derivatives
of $s_{j}^{\rho}$ of degree $d\leq 4$ with respect to the variables
$(g_{\mathbf{i},N})_{\mathbf{i}}$ are bounded by $\Delta_{p}^{d}$ up to an
absolute constant independent of $p,N$ and $\mathbf{i}$. For example,

$\displaystyle\bigg{|}\frac{\partial s_{j}^{\rho}}{\partial
g_{\mathbf{i},N}}\bigg{|}$
$\displaystyle=\bigg{|}\Delta_{p}\frac{\big{\langle}\tau_{j}\tau_{i_{1}}\cdots\tau_{i_{p-1}}\tau_{N}\sinh(X_{N,\beta}(\tau)+h)\big{\rangle}_{N-1,\beta^{\prime}}^{\rho}}{\big{\langle}\cosh(X_{N,\beta}(\tau)+h)\big{\rangle}_{N-1,\beta^{\prime}}^{\rho}}\Big{.}$
$\displaystyle\qquad\Big{.}-\Delta_{p}\frac{s_{j}^{\rho}\big{\langle}\tau_{i_{1}}\cdots\tau_{i_{p-1}}\tau_{N}\sinh(X_{N,\beta}(\tau)+h)\big{\rangle}_{N-1,\beta^{\prime}}^{\rho}}{\big{\langle}\cosh(X_{N,\beta}(\tau)+h)\big{\rangle}_{N-1,\beta^{\prime}}^{\rho}}\bigg{|}\leq
2\Delta_{p}.$

More general partial derivatives can be controlled by an induction argument on
the number of differentiations. This implies that (38) with
$\langle\sigma_{j}\rangle_{N,\beta}^{\alpha}$ replaced by $s_{j}^{\rho}$ is
also valid. We omit the rest of the details. ∎

###### Proof of Proposition 4.2.

Similar to (39), we have

$\mathbb{E}\bigg{\langle}\Big{[}\sum_{p>p_{0}}A_{p}^{\alpha}\Big{]}^{4}\bigg{\rangle}_{N,\beta}\leq\Big{(}\sum_{p>p_{0}}\bigl{(}\mathbb{E}\big{\langle}A_{p}^{4}\big{\rangle}_{N,\beta}\bigr{)}^{\frac{1}{4}}\Big{)}^{4}.$

Since
$\bigl{(}\mathbb{E}\big{\langle}A_{p}^{4}\big{\rangle}_{N,\beta}\bigr{)}^{1/4}$
is summable, as proved in Lemma 4.4, the right hand side can be made
arbitrarily small by choosing $p_{0}$ sufficiently large. The other assertion
can be treated similarly. ∎

###### Proof of Proposition 4.3.

Note that for any $p_{1},p_{2},p_{3},p_{4}\geq 2$, using Hölder’s inequality
yields

$\mathbb{E}\big{\langle}A_{p_{1}}^{\alpha}A_{p_{2}}^{\alpha}A_{p_{3}}^{\alpha}A_{p_{4}}^{\alpha}\big{\rangle}_{N,\beta}\leq\Big{(}\mathbb{E}\big{\langle}|A_{p_{1}}^{\alpha}|^{4}\big{\rangle}_{N,\beta}\mathbb{E}\big{\langle}|A_{p_{2}}^{\alpha}|^{4}\big{\rangle}_{N,\beta}\mathbb{E}\big{\langle}|A_{p_{3}}^{\alpha}|^{4}\big{\rangle}_{N,\beta}\mathbb{E}\big{\langle}|A_{p_{4}}^{\alpha}|^{4}\big{\rangle}_{N,\beta}\Big{)}^{\frac{1}{4}},$

which implies that

$\displaystyle\mathbb{E}\big{\langle}\big{[}X_{N,\beta}\bigl{(}\langle\sigma\rangle_{N,\beta}^{\alpha}\bigr{)}\big{]}^{4}\big{\rangle}_{N,\beta}=\mathbb{E}\bigg{\langle}\Big{(}\sum_{p\geq
2}A_{p}^{\alpha}\Big{)}^{4}\bigg{\rangle}_{N,\beta}\leq\Big{(}\sum_{p\geq
2}\big{(}\mathbb{E}\big{\langle}|A_{p}^{\alpha}|^{4}\big{\rangle}_{N,\beta}\big{)}^{\frac{1}{4}}\Big{)}^{4}.$
(39)

By the Cauchy-Schwarz inequality and Lemma 4.4,

$\displaystyle\mathbb{E}\big{\langle}\big{[}X_{N,\beta}(\langle\sigma\rangle_{N,\beta}^{\alpha})\mathbbm{1}_{\\{W_{N-1,\beta^{\prime}}(A_{\ominus}^{\rho})<\delta\\}}\big{]}^{2}\big{\rangle}_{N,\beta}$
$\displaystyle\leq\Big{(}\mathbb{E}\big{\langle}X^{4}_{N,\beta}(\langle\sigma\rangle_{N,\beta}^{\alpha})\big{\rangle}_{N,\beta}\mathbb{E}\big{\langle}\mathbbm{1}_{\\{W_{N-1,\beta^{\prime}}(A_{\ominus}^{\rho})<\delta\\}}\big{\rangle}_{N,\beta}\Big{)}^{1/2}$
$\displaystyle\leq\sqrt{K}\Big{(}\mathbb{E}\big{\langle}\mathbbm{1}_{\\{W_{N-1,\beta^{\prime}}(A_{\ominus}^{\rho})<\delta\\}}\big{\rangle}_{N,\beta}\Big{)}^{1/2}.$

Thus, (34) follows from (15). The proof of (35) is exactly the same. ∎

###  4.2 Removal of the cavity spin constraint II

Fix $2\leq p\leq p_{0}$ and recall the definitions of $A_{p}^{\alpha}$ and
$B_{p}^{\rho}$ in (32) and (33). Using replicas, we may rewrite
$A_{p}^{\alpha}$ and $B_{p}^{\rho}$ as follows. For
$\boldsymbol{\tau}=(\tau^{1},\ldots,\tau^{p-1})\in\Sigma_{N-1}^{p-1}$, define

$\displaystyle
Z_{N,p}(\boldsymbol{\tau})=\frac{\sqrt{p!}}{N^{(p-1)/2}}\sum_{1\leq
i_{1}<\cdots<i_{p-1}\leq
N-1}g_{i_{1},\ldots,i_{p-1},N}\tau^{1}_{i_{1}}\cdots\tau^{p-1}_{i_{p-1}}.$
(40)

Define

$\displaystyle C_{p}^{\alpha}$
$\displaystyle=\bigg{\langle}Z_{N,p}(\boldsymbol{\tau})\prod_{l=1}^{p-1}\frac{e^{X_{N,\beta}(\tau^{l})+h}\mathbbm{1}_{\Sigma_{N}^{\alpha}}((\tau^{l},1))+e^{-X_{N,\beta}(\tau^{l})-h}\mathbbm{1}_{\Sigma_{N}^{\alpha}}((\tau^{l},-1))}{2}\bigg{\rangle}_{N-1,\beta^{\prime}},$
$\displaystyle C_{p}^{\rho}$
$\displaystyle=\bigg{\langle}Z_{N,p}(\boldsymbol{\tau})\prod_{l=1}^{p-1}\cosh(X_{N,\beta}(\tau^{l})+h)\mathbbm{1}_{\Sigma_{N-1}^{\rho}}(\tau^{l})\bigg{\rangle}_{N-1,\beta^{\prime}},$
$\displaystyle D_{p}^{\alpha}$
$\displaystyle=\bigg{\langle}\prod_{l=1}^{p-1}\frac{e^{X_{N,\beta}(\tau^{l})+h}\mathbbm{1}_{\Sigma_{N}^{\alpha}}((\tau^{l},1))+e^{-X_{N,\beta}(\tau^{l})-h}\mathbbm{1}_{\Sigma_{N}^{\alpha}}((\tau^{l},-1))}{2}\bigg{\rangle}_{N-1,\beta^{\prime}},$
$\displaystyle D_{p}^{\rho}$
$\displaystyle=\bigg{\langle}\prod_{l=1}^{p-1}\cosh(X_{N,\beta}(\tau^{l})+h)\mathbbm{1}_{\Sigma_{N-1}^{\rho}}(\tau^{l})\bigg{\rangle}_{N-1,\beta^{\prime}}.$

Then we can write

$\displaystyle A_{p}^{\alpha}$
$\displaystyle=\beta_{p}\frac{C_{p}^{\alpha}}{D_{p}^{\alpha}}\quad\mbox{and}\quad
B_{p}^{\rho}=\beta_{p}\frac{C_{p}^{\rho}}{D_{p}^{\rho}}.$ (41)

Denote

$\eta_{N}=\mathbb{E}\big{\langle}\big{\langle}\mathbbm{1}_{A_{\oplus}^{\rho}\setminus
A_{\ominus}^{\rho}}(\tau)\big{\rangle}_{N-1,\beta^{\prime}}\big{\rangle}_{N-1,\beta^{\prime}}.$

As we have seen at the end of the proof of Lemma 3.5, $\eta_{N}\to 0$ as
$N\to\infty$. Our next Lemma bounds the errors when approximating
$C_{p}^{\alpha}$ with $C^{\rho}_{p}$ and $D_{p}^{\alpha}$ with $D_{p}^{\rho}$,
which are the main ingredients when we estimate the difference between
$A_{p}^{\alpha}$ and $B_{p}^{\rho}$.

######  Lemma 4.5.

For each $r\geq 1$ and $p\geq 2$, there exist positive constants
$K(r,p,\beta,h)$ and $K^{\prime}(r,p,\beta,h)$ such that for all $N\geq 1,$

$\displaystyle\mathbb{E}\big{\langle}(C_{p}^{\alpha}-C_{p}^{\rho})^{2r}\big{\rangle}_{N,\beta}$
$\displaystyle\leq\eta_{N}K(r,p,\beta,h),$
$\displaystyle\mathbb{E}\big{\langle}(D_{p}^{\alpha}-D_{p}^{\rho})^{2r}\big{\rangle}_{N,\beta}$
$\displaystyle\leq\eta_{N}K^{\prime}(r,p,\beta,h).$

###### Proof.

Fist of all, for any
$\tau\in\Sigma_{N-1},\boldsymbol{\tau}=(\tau^{1},\ldots,\tau^{p-1})\in\Sigma_{N-1}^{p-1}$,
$X_{N,\beta}$ and $Z_{N,p}$ are centered Gaussian random variables with
variances bounded by $C_{\beta}$ and $p$ respectively, which result in

$\displaystyle\mathbb{E}_{g_{\cdot
N}}\bigl{\langle}|Z_{N,p}(\boldsymbol{\tau})|^{k}\bigr{\rangle}_{N-1,\beta^{\prime}}$
$\displaystyle\leq p^{k/2}(k-1)!!,$ $\displaystyle\mathbb{E}_{g_{\cdot
N}}\bigl{\langle}\cosh^{k}(X_{N,\beta}(\tau)+h)\bigr{\rangle}_{N-1,\beta^{\prime}}$
$\displaystyle\leq e^{C_{\beta}k^{2}/2}\cosh^{k}(h),$
$\displaystyle\mathbb{E}_{g_{\cdot N}}\cosh^{k}(X_{N,\beta}(\tau)+h)$
$\displaystyle\leq e^{C_{\beta}k^{2}/2}\cosh^{k}(h).$

Using the nested structure (LABEL:eqn:nestedsets), Cauchy-Schwarz and Jensen’s
inequalities, we get

$\displaystyle|D_{p}^{\alpha}-D_{p}^{\rho}|^{2r}$
$\displaystyle\leq\bigg{\langle}\prod_{l=1}^{p-1}\cosh(X_{N,\beta}(\tau^{l})+h)\bigg{(}\prod_{l=1}^{p-1}\mathbbm{1}_{A_{\oplus}^{\rho}}(\tau^{l})-\prod_{l=1}^{p-1}\mathbbm{1}_{A_{\ominus}^{\rho}}(\tau^{l})\bigg{)}\bigg{\rangle}_{N-1,\beta^{\prime}}^{2r}$
$\displaystyle\leq(p-1)\big{\langle}\cosh^{2r(p-1)}(X_{N,\beta}(\tau)+h)\big{\rangle}_{N-1,\beta^{\prime}}\big{\langle}\mathbbm{1}_{A_{\oplus}^{\rho}\setminus
A_{\ominus}^{\rho}}(\tau)\big{\rangle}_{N-1,\beta^{\prime}}.$

Next, by a change of measure for $\alpha=(\rho,\alpha_{N})\sim G_{N,\beta}$ as
in Lemma 3.2 and the Cauchy-Schwarz inequality, we obtain the second
assertion,

$\displaystyle\mathbb{E}\big{\langle}(D_{p}^{\alpha}-D_{p}^{\rho})^{2r}\big{\rangle}_{N,\beta}$
$\displaystyle\leq\mathbb{E}\Bigl{[}\sum_{\rho}G_{N-1,\beta^{\prime}}(\rho)\big{\langle}\mathbbm{1}_{A_{\oplus}^{\rho}\setminus
A_{\ominus}^{\rho}}(\tau)\big{\rangle}_{N-1,\beta^{\prime}}$
$\displaystyle\qquad\cdot\mathbb{E}_{g_{\cdot
N}}\bigl{[}\cosh(X_{N,\beta}(\rho)+h)\big{\langle}\cosh^{2r(p-1)}(X_{N,\beta}(\tau)+h)\big{\rangle}_{N-1,\beta^{\prime}}\bigr{]}\Bigr{]}$
$\displaystyle\leq\eta_{N}pe^{(4r^{2}p^{2}+1)C_{\beta}}\cosh^{2rp}(h).$

For the first assertion, we again use the nested structure
(LABEL:eqn:nestedsets) and the Cauchy-Schwarz and Jensen inequalities to get

$\displaystyle|C_{p}^{\alpha}-C_{p}^{\rho}|^{2r}$
$\displaystyle\leq\left\langle
Z_{N,p}^{2}(\boldsymbol{\tau})\right\rangle^{r}_{N-1,\beta^{\prime}}\bigg{\langle}\prod_{l=1}^{p-1}\cosh^{2}(X_{N,\beta}(\tau^{l})+h)\bigg{(}\prod_{l=1}^{p-1}\mathbbm{1}_{A_{\oplus}^{\rho}}(\tau^{l})-\prod_{l=1}^{p-1}\mathbbm{1}_{A_{\ominus}^{\rho}}(\tau^{l})\bigg{)}^{2}\bigg{\rangle}_{N-1,\beta^{\prime}}^{r}$
$\displaystyle\leq\big{\langle}Z_{N,p}^{2r}(\boldsymbol{\tau})\big{\rangle}^{r}_{N-1,\beta^{\prime}}\big{\langle}\cosh^{2r(p-1)}(X_{N,\beta}(\tau)+h)\big{\rangle}_{N-1,\beta^{\prime}}\sqrt{p}\big{\langle}\mathbbm{1}_{A_{\oplus}^{\rho}\setminus
A_{\ominus}^{\rho}}(\tau)\big{\rangle}_{N-1,\beta^{\prime}}^{1/2}$

Via a change of measure for $G_{N,\beta}$ as above, we can then apply the
Hölder inequality in the expectation $\mathbb{E}_{g_{\cdot N}}$ with thee
conjugates exponents $3,3,3$ to get the desired bound,

$\displaystyle\begin{split}&\mathbb{E}\big{\langle}|C_{p}^{\alpha}-C_{p}^{\rho}|^{2r}\big{\rangle}_{N,\beta}\\\
&\leq\mathbb{E}\Bigl{[}\sum_{\rho}G_{N-1,\beta^{\prime}}(\rho)\big{\langle}\mathbbm{1}_{A_{\oplus}^{\rho}\setminus
A_{\ominus}^{\rho}}(\tau^{l})\big{\rangle}_{N-1,\beta^{\prime}}\\\
&\qquad\cdot\mathbb{E}_{g_{\cdot
N}}\bigl{[}\cosh(X_{N,\beta}(\rho)+h)\big{\langle}Z^{2r}_{N,p}(\boldsymbol{\tau})\big{\rangle}_{N-1,\beta^{\prime}}\big{\langle}\cosh^{2r(p-1)}(X_{N,\beta}(\tau)+h)\big{\rangle}_{N-1,\beta^{\prime}}\bigr{]}\Bigr{]}\\\
&\leq\sqrt{\eta_{N}}p^{r+1/2}\bigl{[}(6r-1)!!\bigr{]}^{1/3}e^{(3/2+6r^{2}p^{2})C_{\beta}}\cosh^{2rp}(h).\end{split}$
(42)

∎

###  4.3 Proof of Proposition 4.1

First, write

$\displaystyle\ \ \
X_{N,\beta}(\langle\sigma\rangle_{N,\beta}^{\alpha})-X_{N,\beta}(s^{\rho})$
$\displaystyle=\Big{(}X_{N,\beta}(\langle\sigma\rangle_{N,\beta}^{\alpha})-X_{N,\beta}(s^{\rho})\Big{)}\mathbbm{1}_{\\{W_{N-1,\beta^{\prime}}(A_{\ominus}^{\rho})\geq\delta\\}}+\Big{(}X_{N,\beta}(\langle\sigma\rangle_{N,\beta}^{\alpha})-X_{N,\beta}(s^{\rho})\Big{)}\mathbbm{1}_{\\{W_{N-1,\beta^{\prime}}(A_{\ominus}^{\rho})<\delta\\}}$
$\displaystyle=\Big{(}\sum_{2\leq p\leq
p_{0}}\big{(}A_{p}^{\alpha}-B_{p}^{\rho}\big{)}\Big{)}\mathbbm{1}_{\\{W_{N-1,\beta^{\prime}}(A_{\ominus}^{\rho})\geq\delta\\}}+\Big{(}\sum_{p>p_{0}}\big{(}A_{p}^{\alpha}-B_{p}^{\rho}\big{)}\Big{)}\mathbbm{1}_{\\{W_{N-1,\beta^{\prime}}(A_{\ominus}^{\rho})\geq\delta\\}}$
$\displaystyle\qquad+X_{N,\beta}(\langle\sigma\rangle_{N,\beta}^{\alpha})\mathbbm{1}_{\\{W_{N-1,\beta^{\prime}}(A_{\ominus}^{\rho})<\delta\\}}-X_{N,\beta}(s^{\rho})\mathbbm{1}_{\\{W_{N-1,\beta^{\prime}}(A_{\ominus}^{\rho})<\delta\\}}.$

It follows that

$\displaystyle\
\mathbb{E}\big{\langle}\big{(}X_{N,\beta}(\langle\sigma\rangle_{N,\beta}^{\alpha})-X_{N,\beta}(s^{\rho})\big{)}^{2}\big{\rangle}_{N,\beta}$
$\displaystyle\leq C\mathbb{E}\bigg{\langle}\Big{(}\sum_{2\leq p\leq
p_{0}}\big{(}A_{p}^{\alpha}-B_{p}^{\rho}\big{)}\Big{)}^{2}\mathbbm{1}_{\\{W_{N-1,\beta^{\prime}}(A_{\ominus}^{\rho})\geq\delta\\}}\bigg{\rangle}_{N,\beta}$
$\displaystyle\ \ \
+C\mathbb{E}\bigg{\langle}\Big{(}\sum_{p>p_{0}}A_{p}^{\alpha}\Big{)}^{2}\bigg{\rangle}_{N,\beta}+C\mathbb{E}\bigg{\langle}\Big{(}\sum_{p>p_{0}}B_{p}^{\rho}\Big{)}^{2}\bigg{\rangle}_{N,\beta}$
$\displaystyle\ \ \
+C\mathbb{E}\big{\langle}X^{2}_{N,\beta}(\langle\sigma\rangle_{N,\beta}^{\alpha})\mathbbm{1}_{\\{W_{N-1,\beta^{\prime}}(A_{\ominus}^{\rho})<\delta\\}}\big{\rangle}_{N,\beta}+C\mathbb{E}\big{\langle}X^{2}_{N,\beta}(s^{\rho})\mathbbm{1}_{\\{W_{N-1,\beta^{\prime}}(A_{\ominus}^{\rho})<\delta\\}}\big{\rangle}_{N,\beta}.$

By Propositions 4.2 and 4.3, the last four terms can be made arbitrary small
by choosing $\delta>0$ small enough and $p_{0}$ large enough. Thus, it
suffices to show that for any $0<\delta<1$ and $p_{0}\geq 2$, the first term
vanishes as $N\to\infty$, i.e.,

$\displaystyle\lim_{N\to\infty}\mathbb{E}\bigg{\langle}\bigg{[}\sum_{2\leq
p\leq
p_{0}}\big{(}A_{p}^{\alpha}-B_{p}^{\rho}\big{)}\bigg{]}^{2}\mathbbm{1}_{\\{W_{N-1,\beta^{\prime}}(A_{\ominus}^{\rho})\geq\delta\\}}\bigg{\rangle}_{N,\beta}=0.$
(43)

To do this, notice that from (41),

$\displaystyle\Big{(}\sum_{2\leq p\leq
p_{0}}\big{(}A_{p}^{\alpha}-B_{p}^{\rho}\big{)}\Big{)}^{2}$
$\displaystyle=\bigg{(}\sum_{2\leq p\leq
p_{0}}\beta_{p}\Big{(}\frac{C_{p}^{\alpha}}{D_{p}^{\alpha}}-\frac{C_{p}^{\rho}}{D_{p}^{\rho}}\Big{)}\bigg{)}^{2}$
$\displaystyle\leq\bigg{(}\sum_{2\leq p\leq
p_{0}}\beta_{p}^{2}\bigg{)}\bigg{(}\sum_{2\leq p\leq
p_{0}}\Big{(}\frac{C_{p}^{\alpha}}{D_{p}^{\alpha}}-\frac{C_{p}^{\rho}}{D_{p}^{\rho}}\Big{)}^{2}\bigg{)}$
$\displaystyle\leq 2C_{\beta}\sum_{2\leq p\leq
p_{0}}\bigg{(}\Big{(}\frac{C_{p}^{\rho}}{D_{p}^{\rho}}\Big{)}^{2}\Big{(}\frac{D_{p}^{\alpha}-D_{p}^{\rho}}{D_{p}^{\alpha}}\Big{)}^{2}+\Big{(}\frac{C_{p}^{\alpha}-C_{p}^{\rho}}{D_{p}^{\alpha}}\Big{)}^{2}\bigg{)}.$

Using (LABEL:eqn:nestedsets) and the fact $\cosh x\geq 1$, we have
$D_{p}^{\alpha},D_{p}^{\rho}\geq\delta^{p}$ on the event
$\\{W_{N-1,\beta^{\prime}}(A_{\ominus}^{\rho})\geq\delta\\}$ and it follows
that

$\displaystyle\ \ \ \mathbb{E}\bigg{\langle}\bigg{[}\sum_{2\leq p\leq
p_{0}}\big{(}A_{p}^{\alpha}-B_{p}^{\rho}\big{)}\bigg{]}^{2}\mathbbm{1}_{\\{W_{N-1,\beta^{\prime}}(A_{\ominus}^{\rho})\geq\delta\\}}\bigg{\rangle}_{N,\beta}$
$\displaystyle\leq\frac{2C_{\beta}}{\delta^{2p_{0}}}\sum_{2\leq p\leq
p_{0}}\mathbb{E}\left\langle\Big{[}\Big{(}\frac{C_{p}^{\rho}}{\delta^{p_{0}}}\Big{)}^{2}\big{(}D_{p}^{\alpha}-D_{p}^{\rho}\big{)}^{2}+\big{(}C_{p}^{\alpha}-C_{p}^{\rho}\big{)}^{2}\Big{]}\right\rangle_{N,\beta}$
$\displaystyle\leq\frac{2C_{\beta}}{\delta^{2p_{0}}}\sum_{2\leq p\leq
p_{0}}\Bigl{(}\frac{1}{\delta^{2p_{0}}}\sqrt{\mathbb{E}\big{\langle}\big{(}C_{p}^{\rho}\big{)}^{4}\big{\rangle}_{N,\beta}\mathbb{E}\big{\langle}\big{(}D_{p}^{\alpha}-D_{p}^{\rho}\big{)}^{4}\big{\rangle}_{N,\beta}}+\mathbb{E}\big{\langle}\big{(}C_{p}^{\alpha}-C_{p}^{\rho}\big{)}^{2}\big{\rangle}_{N,\beta}\Bigr{)}.$
(44)

By Lemma 4.5, we have for all $2\leq p\leq p_{0}$,

$\displaystyle\mathbb{E}\big{\langle}\big{(}D_{p}^{\rho}-D_{p}^{\rho}\big{)}^{4}\big{\rangle}_{N,\beta}$
$\displaystyle\leq K\eta_{N},$
$\displaystyle\mathbb{E}\big{\langle}\big{(}C_{p}^{\alpha}-C_{p}^{\rho}\big{)}^{2}\big{\rangle}_{N,\beta}$
$\displaystyle\leq K\sqrt{\eta_{N}},$

for all $N\geq 1,$ where $K$ is a constant depending only on $p_{0},\beta,h.$
Also, a slight modification, by dropping the indicators in (LABEL:add:eq6), we
can bound
$\mathbb{E}\langle\big{(}C_{p}^{\rho}\big{)}^{4}\rangle_{N,\beta}\leq
K^{\prime}$ for $2\leq p\leq p_{0}$, where $K^{\prime}$ is a constant
depending on $p,\beta,h.$ Plugging these into (44) and then sending
$N\to\infty$ yields (43) and thus completes the proof of (31).

##  5 From cavity to TAP equations

We will complete the proof of Theorem 1.1 in this section. At the current
stage, we have seen that
$X_{N,\beta}(\langle\sigma\rangle_{N,\beta}^{\alpha})$ asymptotically equals
$X_{N,\beta}(s^{\rho})=\sum_{p\geq 2}B^{\rho}_{p}$ following from Proposition
4.1, where $B_{p}^{\rho}$ is defined in (33). Our next step is to handle this
sum, via an argument similar to the proof of Proposition 3.11, by adapting a
multivariate central limit theorem for the Gaussian fields,
$X_{N,\beta}(\tau^{1}),\ldots,X_{N,\beta}(\tau^{p-1})$, and
$Z_{N,p}(\boldsymbol{\tau})$ appearing in $C_{p}^{\rho}$ and $D_{p}^{\rho}$,
see (41). We establish this limit theorem in the first subsection.

###  5.1 A multivariate central limit theorem for the cavity fields

Let $\boldsymbol{\xi}=(\xi_{0},\ldots,\xi_{p-1})^{T}$ be a centered Gaussian
(column) vector of length $p$ with covariance matrix

$\mathbb{E}(\boldsymbol{\xi}\boldsymbol{\xi}^{T})=\begin{bmatrix}p(1-q_{P}^{p-1})&\beta_{p}pq_{P}^{p-2}(1-q_{P})&\beta_{p}pq_{P}^{p-2}(1-q_{P})&\cdots&\beta_{p}pq_{P}^{p-2}(1-q_{P})\vspace{2mm}\\\
\beta_{p}pq_{P}^{p-2}(1-q_{P})&\zeta^{\prime}(1)-\zeta^{\prime}(q_{P})&0&\cdots&0\\\
\beta_{p}pq_{P}^{p-2}(1-q_{P})&0&\zeta^{\prime}(1)-\zeta^{\prime}(q_{P})&\cdots&0\\\
\vdots&\vdots&\vdots&\ddots&\vdots\\\
\beta_{p}pq_{P}^{p-2}(1-q_{P})&0&0&\cdots&\zeta^{\prime}(1)-\zeta^{\prime}(q_{P})\end{bmatrix},$

that is, $\xi_{1},\ldots,\xi_{p-1}$ are i.i.d.
$N(0,\zeta^{\prime}(1)-\zeta^{\prime}(q_{P}))$ and $\xi_{0}\sim
N(0,p(1-q_{P}^{p-1}))$ that has covariance $\beta_{p}pq^{p-2}(1-q)$ with each
of $\xi_{1},\ldots,\xi_{p-1}$. Note that this multivariate Gaussian
distribution is well-defined. To see this, consider i.i.d. standard normal
random variables $z_{0},z_{1},\ldots,z_{p-1}$, and $\boldsymbol{\xi}$ can be
constructed via linear combinations of $z_{0},z_{1},\ldots,z_{p-1}$ as
follows.

$\displaystyle\xi_{i}$
$\displaystyle=\sqrt{\zeta^{\prime}(1)-\zeta^{\prime}(q_{P})}z_{i},\quad
i=1,\ldots,p-1,$ $\displaystyle\xi_{0}$
$\displaystyle=\frac{\beta_{p}pq_{P}^{p-2}(1-q_{P})}{\sqrt{\zeta^{\prime}(1)-\zeta^{\prime}(q_{P})}}\sum_{i=1}^{p-1}z_{i}+z_{0}\sqrt{p(1-q_{P}^{p-1})-\frac{\beta_{p}^{2}p^{2}(p-1)q_{P}^{2p-4}(1-q_{P})^{2}}{\zeta^{\prime}(1)-\zeta^{\prime}(q_{P})}},$

where the quantity in the last square root is always nonnegative because

$\displaystyle
p(1-q_{P}^{p-1})(\zeta^{\prime}(1)-\zeta^{\prime}(q_{P}))-\beta_{p}^{2}p^{2}(p-1)q_{P}^{2p-4}(1-q_{P})^{2}$
$\displaystyle\geq
p(1-q_{P}^{p-1})\cdot\beta_{p}^{2}p(1-q_{P}^{p-1})-\beta_{p}^{2}p^{2}(p-1)q_{P}^{2p-4}(1-q_{P})^{2}$
$\displaystyle=\beta_{p}^{2}p^{2}\bigl{(}(1-q_{P}^{p-1})^{2}-(p-1)q_{P}^{2p-4}(1-q_{P})^{2}\bigr{)}$
$\displaystyle=\beta_{p}^{2}p^{2}(1-q_{P})^{2}\bigl{(}(1+q_{P}+\cdots+q_{P}^{p-2})^{2}-(p-1)q_{P}^{2p-4}\bigr{)}\geq
0.$

For $\rho\in\Sigma_{N-1}$, recall from (19) that

$\dot{X}_{N,\beta}^{\rho}(\tau^{1}),\ldots,\dot{X}_{N,\beta}^{\rho}(\tau^{p-1}),\quad\forall\tau^{1},\ldots,\tau^{p-1}\in\Sigma_{N-1}$

are defined as

$\displaystyle\dot{X}_{N,\beta}^{\rho}({\tau}^{l})$ $\displaystyle=\sum_{p\geq
2}\frac{\beta_{p}\sqrt{p!}}{N^{(p-1)/2}}\sum_{1\leq i_{1}<\cdots<i_{p-1}\leq
N-1}g_{i_{1},\ldots,i_{p-1},N}\bigl{(}\tau_{i_{1}}^{l}\cdots\tau^{l}_{i_{p-1}}-\big{\langle}\tau_{i_{1}}^{l}\cdots\tau^{l}_{i_{p-1}}\big{\rangle}_{N-1,\beta^{\prime}}^{\rho}\bigr{)}.$
(45)

Recall $Z_{N,p}$ from (40). We set the centralized
$Z_{N,p}(\boldsymbol{\tau})$ as

$\displaystyle\dot{Z}_{N,p}^{\rho}({\boldsymbol{\tau}})$
$\displaystyle=\frac{\sqrt{p!}}{N^{(p-1)/2}}\sum_{1\leq
i_{1}<\cdots<i_{p-1}\leq
N-1}g_{i_{1},\ldots,i_{p-1},N}\bigl{(}\tau_{i_{1}}^{1}\cdots\tau_{i_{p-1}}^{p-1}-\big{\langle}\tau_{i_{1}}^{1}\cdots\tau^{p-1}_{i_{p-1}}\big{\rangle}_{N-1,\beta^{\prime}}^{\rho}\bigr{)}.$
(46)

Now we are ready to state the central limit theorem.

######  Theorem 5.1 (Multivariate Central Limit Theorem).

Fix $p\geq 2$. Let $F_{0},\ldots,F_{p-1}$ be infinitely differentiable
functions on $\mathbb{R}$ such that for $z\sim N(0,1)$, any $M>0$ and integers
$m,k\geq 0$,

$\sup_{0\leq\gamma\leq M}\max\bigl{(}\mathbb{E}|F_{0}^{(m)}(\gamma
z)|^{k},\ldots,\mathbb{E}|F_{p-1}^{(m)}(\gamma z)|^{k}\bigr{)}<\infty.$

Then, for any integer $r\geq 1$ fixed, we have

$\displaystyle\begin{split}\lim_{\epsilon\downarrow
0}\limsup_{N\to\infty}\,&\mathbb{E}\Big{\langle}\Big{[}\big{\langle}U\big{(}\dot{Z}_{N,p}^{\rho}(\boldsymbol{\tau}),\dot{X}_{N,\beta}^{\rho}(\tau^{1}),\ldots,\dot{X}_{N,\beta}^{\rho}(\tau^{p-1})\big{)}\big{\rangle}^{\rho}_{N-1,\beta^{\prime}}-\mathbb{E}U(\boldsymbol{\xi})\Big{]}^{2r}\Big{\rangle}_{N,\beta}=0,\end{split}$
(47)

where $U(x_{0},\ldots,x_{p-1}):=F_{0}(x_{0})F_{1}(x_{1})\cdots
F_{p-1}(x_{p-1})$ for $x=(x_{0},\ldots,x_{p-1})\in\mathbb{R}^{p}$.

###### Proof.

The proof utilizes a similar idea as that of Theorem 3.7. The main difference
is that here we need to deal with the $2r$-th moment of a multivariate
function depending on $(p-1)$ spin replicas. Thus, we will further consider
$2r$ independent copies of these $(p-1)$ spins,
$(\tau^{1},\tau^{2},\ldots,\tau^{p-1})$, and denote them by

$\displaystyle\boldsymbol{\tau}^{m}$
$\displaystyle=(\tau^{m,1},\ldots,\tau^{m,p-1}),\quad m=1,2,\ldots,2r,$

where $(\tau^{m,l})_{1\leq m\leq 2r,1\leq p\leq p-1}$ are independently
sampled from $G_{N-1,\beta^{\prime}}^{\rho}$. For $1\leq m\leq 2r$, define

$\displaystyle{\boldsymbol{S}}_{\rho}^{m}$
$\displaystyle=(S^{m,0}_{\rho},S^{m,1}_{\rho},\ldots,S^{m,p-1}_{\rho})=\bigl{(}\dot{Z}_{N,p}^{\rho}(\boldsymbol{\tau}^{m}),\dot{X}_{N,\beta}^{\rho}({\tau}^{m,1}),\ldots,\dot{X}_{N,\beta}^{\rho}({\tau}^{m,p-1})\bigr{)}.$

Let $\boldsymbol{\xi}^{m}=(\xi_{0}^{m},\xi_{1}^{m},\ldots,\xi_{p-1}^{m})$ be
i.i.d. copies of $\boldsymbol{\xi}$ and be independent of everything else. For
$0\leq t\leq 1$, consider the interpolation

$\phi_{N}(t):=\mathbb{E}\left\langle\bigg{\langle}\prod_{m=1}^{2r}V\big{(}\boldsymbol{x}_{\rho}^{m}(t)\big{)}\bigg{\rangle}_{N-1,\beta^{\prime}}^{\rho}\right\rangle_{N,\beta}$

where

$\displaystyle\boldsymbol{x}_{\rho}^{m}(t)$
$\displaystyle=\bigl{(}x_{\rho}^{m,0}(t),\ldots,x_{\rho}^{m,p-1}(t)\bigr{)}:=\sqrt{t}{\boldsymbol{S}}_{\rho}^{m}+\sqrt{1-t}\xi^{m}$

and $V:\mathbb{R}^{p}\to\mathbb{R}$ is defined as

$\displaystyle
V(x_{0},\ldots,x_{p-1})=U(x_{0},\ldots,x_{p-1})-\mathbb{E}U(\xi_{0},\ldots,\xi_{p-1}).$

Note that the left-hand side of (47) equals $\phi_{N}(1)$ and $\phi_{N}(0)=0.$
Again, from a change of measure as in the proof of Lemma 3.2, we have
$\phi_{N}(1)\leq\psi_{N}(1)$, where

$\displaystyle\psi_{N}(t):=$
$\displaystyle\mathbb{E}\left\langle\cosh(X_{N,\beta}(\rho)+h)\bigg{\langle}\prod_{m=1}^{2r}V\big{(}\boldsymbol{x}_{\rho}^{m}(t)\big{)}\bigg{\rangle}_{N-1,\beta^{\prime}}^{\rho}\right\rangle_{N-1,\beta^{\prime}}.$

It suffices to show that $\lim_{\epsilon\downarrow
0}\limsup_{N\to\infty}\psi_{N}(1)=0.$ To justify this limit, observe that
$\psi_{N}(0)=0$ and we thus aim at showing

$\displaystyle\lim_{\epsilon\downarrow
0}\limsup_{N\to\infty}\int_{0}^{1}|\psi_{N}^{\prime}(t)|dt=0.$ (48)

First of all, by chain rule, $\psi_{N}^{\prime}(t)$ is equal to

$\displaystyle\sum_{m=1}^{2r}\sum_{l=0}^{p-1}\mathbb{E}\left\langle\cosh(X_{N,\beta}(\rho)+h)\bigg{\langle}\frac{\partial
V}{\partial
x_{l}}(\boldsymbol{x}_{\rho}^{m}(t))\frac{1}{2}\Big{(}\frac{{S}_{\rho}^{m,l}}{\sqrt{t}}-\frac{\xi_{l}^{m}}{\sqrt{1-t}}\Big{)}\prod_{\begin{subarray}{c}m^{\prime}=1\\\
m^{\prime}\neq
m\end{subarray}}^{2r}V\big{(}\boldsymbol{x}_{\rho}^{m^{\prime}}(t)\big{)}\bigg{\rangle}_{N-1,\beta^{\prime}}^{\rho}\right\rangle_{N-1,\beta^{\prime}}.$

Note that the Gibbs measures $G_{N-1,\beta^{\prime}}$ and
$G_{N-1,\beta^{\prime}}^{\rho}$ do not depend on $g_{\cdot N}$, and thus, as
before, we can compute this derivative by first taking the expectation (inside
the Gibbs averages $\langle\cdot\rangle_{N-1,\beta^{\prime}}$ and
$\langle\cdot\rangle_{N-1,\beta^{\prime}}^{\rho}$) only with respect to
$g_{\cdot N}$ and $(\xi_{l}^{m})_{0\leq l\leq p-1,1\leq m\leq 2r}$ and then
using the Gaussian integration by parts. With these, for fixed $m$ and $l$,
the relevant terms in the above display become

$\displaystyle\ \ \mathbb{E}_{g_{\cdot
N},\boldsymbol{\xi}}\bigg{[}\cosh(X_{N,\beta}(\rho)+h)\frac{\partial
V}{\partial
x_{l}}(\boldsymbol{x}_{\rho}^{m}(t))\frac{1}{2}\Big{(}\frac{{S}_{\rho}^{m,l}}{\sqrt{t}}-\frac{\xi_{l}^{m}}{\sqrt{1-t}}\Big{)}\prod_{\begin{subarray}{c}m^{\prime}=1\\\
m^{\prime}\neq
m\end{subarray}}^{2r}V\big{(}\boldsymbol{x}_{\rho}^{m^{\prime}}(t)\big{)}\bigg{]}=\text{(I)
+ (II) + (III)},$

where

$\displaystyle{\rm(I)}$ $\displaystyle:=\mathbb{E}_{g_{\cdot
N},\boldsymbol{\xi}}\bigg{[}\sinh(X_{N,\beta}(\rho)+h)\frac{\partial
V}{\partial
x_{l}}(\boldsymbol{x}_{\rho}^{m}(t))\prod_{\begin{subarray}{c}m^{\prime}=1\\\
m^{\prime}\neq
m\end{subarray}}^{2r}V\big{(}\boldsymbol{x}_{\rho}^{m}(t)\big{)}\bigg{]}\cdot\frac{1}{2\sqrt{t}}\mathbb{E}_{g_{\cdot
N}}\big{[}{S}_{\rho}^{m,l}X_{N,\beta}(\rho)\big{]},$ $\displaystyle{\rm(II)}$
$\displaystyle:=\
\sum_{\begin{subarray}{c}l^{\prime}=0\end{subarray}}^{p-1}\mathbb{E}_{g_{\cdot
N},\boldsymbol{\xi}}\bigg{[}\cosh(X_{N,\beta}(\rho)+h)\frac{\partial^{2}V}{\partial
x_{l}\partial
x_{l^{\prime}}}(\boldsymbol{x}_{\rho}^{m}(t))\prod_{\begin{subarray}{c}m^{\prime}=1\\\
m^{\prime}\neq
m\end{subarray}}^{2r}V\big{(}\boldsymbol{x}_{\rho}^{m^{\prime}}(t)\big{)}\bigg{]}$
$\displaystyle\quad\qquad\qquad\cdot\frac{1}{2}\mathbb{E}_{g_{\cdot
N},\boldsymbol{\xi}}\Big{[}\Bigl{(}\frac{S_{\rho}^{m,l}}{\sqrt{t}}-\frac{\xi_{l}^{m}}{\sqrt{1-t}}\Bigr{)}x_{\rho}^{m,l^{\prime}}(t)\Big{]},$

and

$\displaystyle{\rm(III)}$
$\displaystyle:=\sum_{\begin{subarray}{c}m^{\prime}=1\\\ m^{\prime}\neq
m\end{subarray}}^{2r}\sum_{\begin{subarray}{c}l^{\prime}=0\end{subarray}}^{p-1}\mathbb{E}_{g_{\cdot
N},\boldsymbol{\xi}}\bigg{[}\cosh(X_{N,\beta}(\rho)+h)\frac{\partial
V}{\partial x_{l}}(\boldsymbol{x}_{\rho}^{m}(t))\frac{\partial V}{\partial
x_{l^{\prime}}}(\boldsymbol{x}_{\rho}^{m^{\prime}}(t))\prod_{\begin{subarray}{c}m^{\prime\prime}=1\\\
m^{\prime\prime}\neq
m,m^{\prime}\end{subarray}}^{2r}V\big{(}\boldsymbol{x}_{\rho}^{m^{\prime\prime}}(t)\big{)}\bigg{]}$
$\displaystyle\qquad\qquad\qquad\qquad\quad\cdot\frac{1}{2}\mathbb{E}_{g_{\cdot
N},\boldsymbol{\xi}}\Big{[}\Bigl{(}\frac{S_{\rho}^{m,l}}{\sqrt{t}}-\frac{\xi_{l}^{m}}{\sqrt{1-t}}\Bigr{)}x_{\rho}^{m^{\prime},l^{\prime}}(t)\Big{]}.$

Here $\mathbb{E}_{g_{\cdot N},\boldsymbol{\xi}}$ is the expectation with
respect to $g_{\cdot N}$ and $(\xi_{l}^{m})_{0\leq l\leq p-1,1\leq m\leq 2r}.$
To handle these terms, we first note that for any $(\tau^{m,l})_{1\leq m\leq
2r,1\leq l\leq p-1}$ sampled from $G_{N-1,\beta^{\prime}}$, coordinates of
${\boldsymbol{S}}_{\rho}^{m}$ are mean-zero Gaussian random variables with
uniformly bounded variance for any $N\geq 1$, and so are the coordinates of
$\boldsymbol{x}_{\rho}^{m}(t)$ for any $0\leq t\leq 1$. Consequently, from the
given assumptions on the functions $F_{0},\ldots,F_{p-1}$ and the Hölder
inequality, the first expectations in (I), (II), and (III) are bounded by an
absolute constant independent of $N$, $t$, and $(\tau^{m,l})_{1\leq m\leq
2r,1\leq l\leq p-1}$. Furthermore, in (II) and (III), using Gaussian
integration by parts implies that for all $0<t<1,$ $1\leq m,m^{\prime}\leq
2r$, and $0\leq l,l^{\prime}\leq p-1,$

$\displaystyle\mathbb{E}_{g_{\cdot
N},\boldsymbol{\xi}}\Big{[}\Bigl{(}\frac{S_{\rho}^{m,l}}{\sqrt{t}}-\frac{\xi_{l}^{m}}{\sqrt{1-t}}\Bigr{)}x_{\rho}^{m^{\prime},l^{\prime}}(t)\Big{]}$
$\displaystyle=\mathbb{E}_{g_{\cdot
N}}\big{[}S_{\rho}^{m,l}S_{\rho}^{m^{\prime},l^{\prime}}\big{]}-\mathbb{E}\big{[}\xi_{l}^{m}\xi_{l^{\prime}}^{m^{\prime}}\big{]}.$

From these, we conclude that there exists a positive constant $K_{0}$
independent of $N$ and $t$ such that

$\displaystyle|\psi^{\prime}_{N}(t)|$
$\displaystyle\leq\frac{K_{0}}{\sqrt{t}}\sum_{m=1}^{2r}\sum_{l=0}^{p-1}\mathbb{E}\Big{\langle}\big{\langle}\big{|}\mathbb{E}_{g_{\cdot
N}}\big{[}S_{\rho}^{m,l}X_{N,\beta}(\rho)\big{]}\big{|}\big{\rangle}^{\rho}_{N-1,\beta^{\prime}}\Big{\rangle}_{N-1,\beta^{\prime}}$
(49) $\displaystyle\ \
+K_{0}\sum_{m,m^{\prime}=1}^{2r}\sum_{l,l^{\prime}=0}^{p-1}\mathbb{E}\Big{\langle}\big{\langle}\big{|}\mathbb{E}_{g_{\cdot
N}}\big{[}S_{\rho}^{m,l}S_{\rho}^{m^{\prime},l^{\prime}}\big{]}-\mathbb{E}\big{[}\xi_{l}^{m}\xi_{l^{\prime}}^{m^{\prime}}\big{]}\big{|}\big{\rangle}^{\rho}_{N-1,\beta^{\prime}}\Big{\rangle}_{N-1,\beta^{\prime}}.$
(50)

Note that $t^{-1/2}$ is integrable on $(0,1].$ It remains to show that the
expectations in (49) and (50) vanish in the limit as $N\to\infty$ and then
$\epsilon\to 0$, which implies in (48).

Estimation of (49): We claim that for any $1\leq m\leq 2r$ and $0\leq l\leq
p-1,$

$\displaystyle\lim_{\epsilon\downarrow
0}\limsup_{N\to\infty}\mathbb{E}\Big{\langle}\big{\langle}\big{|}\mathbb{E}_{g_{\cdot
N}}\big{[}S_{\rho}^{m,l}X_{N,\beta}(\rho)\big{]}\big{|}\big{\rangle}^{\rho}_{N-1,\beta^{\prime}}\Big{\rangle}_{N-1,\beta^{\prime}}=0.$
(51)

If $l\neq 0$, then recalling $S_{\rho}^{m,l}=\dot{X}_{N,\beta}(\tau^{m,l})$,
the same argument as that used in the estimation of (24) in the proof of
Theorem 3.7 yields (51). As for $l=0$, from
$S_{\rho}^{m,0}=\dot{Z}_{N,p}^{\rho}(\boldsymbol{\tau}),$ we compute

$\displaystyle\mathbb{E}_{g_{\cdot
N}}\big{[}S_{\rho}^{m,0}X_{N,\beta}(\rho)\big{]}$
$\displaystyle=\frac{\beta_{p}p}{N^{p-1}}\sum_{\begin{subarray}{c}i_{1},\ldots
i_{p-1}=1\\\
\text{distinct}\end{subarray}}^{N-1}\bigl{(}\tau_{i_{1}}^{m,1}\cdots\tau_{i_{p-1}}^{m,p-1}-\big{\langle}\tau_{i_{1}}^{m,1}\cdots\tau^{m,p-1}_{i_{p-1}}\big{\rangle}_{N-1,\beta^{\prime}}^{\rho}\bigr{)}\rho_{i_{1}}\cdots\rho_{i_{p-1}}$
$\displaystyle=\beta_{p}p\Bigl{(}R(\tau^{m,1},\rho)\cdots
R(\tau^{m,p-1},\rho)-\big{\langle}R(\tau^{m,1},\rho)\cdots
R(\tau^{m,p-1},\rho)\big{\rangle}_{N-1,\beta^{\prime}}^{\rho}\Bigr{)}+o_{N}(1).$

Since $\tau^{m,1},\ldots,\tau^{m,p-1}\sim G_{N-1,\beta^{\prime}}^{\rho}$, we
always have $R(\tau^{m,1},\rho),\ldots,R(\tau^{m,p-1},\rho)\geq
q_{P}-\epsilon$. This combining with (16) implies that

$\limsup_{N\to\infty}\mathbb{E}\Big{\langle}\big{\langle}\big{|}\mathbb{E}_{g_{\cdot
N}}\big{[}S_{\rho}^{m,0}X_{N,\beta}(\rho)\big{]}\big{|}\big{\rangle}^{\rho}_{N-1,\beta^{\prime}}\Big{\rangle}_{N-1,\beta^{\prime}}\leq\beta_{p}p\bigl{(}(q_{P}+\epsilon)^{p-1}-(q_{P}-\epsilon)^{p-1}\bigr{)}\leq
2\beta_{p}p(p-1)\epsilon.$

Sending $\epsilon\downarrow 0$ yields (51).

Estimation of (50): We claim that for any $1\leq m,m^{\prime}\leq 2r$ and
$0\leq l,l^{\prime}\leq p-1,$

$\displaystyle\lim_{\epsilon\downarrow
0}\limsup_{N\to\infty}\mathbb{E}\big{\langle}\big{\langle}\big{|}\mathbb{E}_{g_{\cdot
N}}\big{[}S_{\rho}^{m,l}S_{\rho}^{m^{\prime},l^{\prime}}\big{]}-\mathbb{E}\big{[}\xi_{l}^{m}\xi_{l^{\prime}}^{m^{\prime}}\big{]}\big{|}\big{\rangle}^{\rho}_{N-1,\beta^{\prime}}\big{\rangle}_{N-1,\beta^{\prime}}=0.$
(52)

First consider $l,l^{\prime}\neq 0$. Note that, in this case,

$\displaystyle
S_{\rho}^{m,l}=\dot{X}_{N,\beta}^{\rho}(\tau^{m,l})\quad\mbox{and}\quad
S_{\rho}^{m^{\prime},l^{\prime}}=\dot{X}_{N,\beta}^{\rho}(\tau^{m^{\prime},l^{\prime}}).$

In view of the proof of Theorem 3.7, when $m=m^{\prime}$ and $l=l^{\prime}$,
the control of $\mathbb{E}_{g_{\cdot
N}}\big{[}S_{\rho}^{m,l}S_{\rho}^{m^{\prime},l^{\prime}}\big{]}$ has been
implemented in the Estimation of (22) ; if $m\neq m^{\prime}$, or $l\neq
l^{\prime}$, or both, this expectation can also be controlled by the
Estimation of (23). As a conclusion, we readily see that (52) holds under
these two cases. It remains to consider the scenarios when at least one of the
$l,l^{\prime}$ is equal to 0. Without loss of generality, we assume $l=0$ and
divide our discussion into two cases:

Case 1: $m\neq m^{\prime}.$ Note that
$\mathbb{E}\big{[}\xi_{0}^{m}\xi_{0}^{m^{\prime}}\big{]}=0$.

  * (1a)

When $l^{\prime}=0$,

$\displaystyle\big{|}\mathbb{E}_{g_{\cdot
N}}[S_{\rho}^{m,0}S_{\rho}^{m^{\prime},0}]\big{|}$
$\displaystyle=\big{\lvert}\mathbb{E}_{g_{\cdot
N}}[\dot{Z}_{N,p}^{\rho}({\boldsymbol{\tau}^{m}})\dot{Z}_{N,p}^{\rho}({\boldsymbol{\tau}^{m^{\prime}}})]\big{\rvert}$
$\displaystyle\leq
p\Big{|}\prod_{\ell=1}^{p-1}R(\tau^{m,\ell},\tau^{m^{\prime},\ell})-q_{P}^{p-1}\Big{|}+p\Big{|}\prod_{\ell=1}^{p-1}\big{\langle}R(\tau^{m,\ell},\tau^{m^{\prime},\ell})\big{\rangle}_{N-1,\beta^{\prime}}^{\rho}-q_{P}^{p-1}\Big{|}$
$\displaystyle\ \ \
+p\Bigg{|}\sum_{\begin{subarray}{c}i_{1},\ldots,i_{p-1}=1\\\
\text{distinct}\end{subarray}}^{N-1}\tau^{m,1}_{i_{1}}\cdots\tau^{m,p-1}_{i_{p-1}}\big{\langle}\tau^{m^{\prime},1}_{i_{1}}\cdots\tau^{m^{\prime},p-1}_{i_{p-1}}\big{\rangle}_{N-1,\beta^{\prime}}^{\rho}-q_{P}^{p-1}\Bigg{|}$
$\displaystyle\ \ \
+p\Bigg{|}\sum_{\begin{subarray}{c}i_{1},\ldots,i_{p-1}=1\\\
\text{distinct}\end{subarray}}^{N-1}\tau^{m^{\prime},1}_{i_{1}}\cdots\tau^{m^{\prime},p-1}_{k_{p-1}}\big{\langle}\tau^{m,1}_{i_{1}}\cdots\tau^{m,p-1}_{i_{p-1}}\big{\rangle}_{N-1,\beta^{\prime}}^{\rho}-q_{P}^{p-1}\Bigg{|}+o_{N}(1).$

Taking Gibbs average with respect to $G_{N-1,\beta^{\prime}}^{\rho}$ and using
Jensen’s inequality, we get

$\displaystyle\big{\langle}\big{|}\mathbb{E}_{g_{\cdot
N}}[S_{\rho}^{m,0}S_{\rho}^{m^{\prime},0}]\big{|}\big{\rangle}_{N-1,\beta^{\prime}}^{\rho}\leq
4p\bigg{\langle}\bigg{|}\prod_{\ell=1}^{p-1}R(\tau^{1,\ell},\tau^{2,\ell})-q_{P}^{p-1}\bigg{|}\bigg{\rangle}_{N-1,\beta^{\prime}}^{\rho}+o_{N}(1).$

  * (1b)

If $l^{\prime}\neq 0$, then

$\displaystyle\big{|}\mathbb{E}_{g_{\cdot
N}}[S_{\rho}^{m,0}S_{\rho}^{m^{\prime},l^{\prime}}]\big{|}$
$\displaystyle=\big{\lvert}\mathbb{E}_{g_{\cdot
N}}[\dot{Z}_{N,p}^{\rho}({\boldsymbol{\tau}})\dot{X}_{N,\beta}^{\rho}({\tau}^{m^{\prime},l^{\prime}})]\big{\rvert}$
$\displaystyle\leq\beta_{p}p\Big{|}\prod_{\ell=1}^{p-1}R(\tau^{m,\ell},\tau^{m^{\prime},l^{\prime}})-q_{P}^{p-1}\Big{|}+\beta_{p}p\Big{|}\prod_{\ell=1}^{p-1}\big{\langle}R(\tau^{m,\ell},\tau^{m^{\prime},l^{\prime}})\big{\rangle}_{N-1,\beta^{\prime}}^{\rho}-q_{P}^{p-1}\Big{|}$
$\displaystyle\ \
+\beta_{p}p\Bigg{|}\sum_{\begin{subarray}{c}i_{1},\ldots,i_{p-1}=1\\\
\text{distinct}\end{subarray}}^{N-1}\tau^{m,1}_{i_{1}}\cdots\tau^{m,p-1}_{i_{p-1}}\big{\langle}\tau^{m^{\prime},l^{\prime}}_{i_{1}}\cdots\tau^{m^{\prime},l^{\prime}}_{i_{p-1}}\big{\rangle}_{N-1,\beta^{\prime}}^{\rho}-q_{P}^{p-1}\Bigg{|}$
$\displaystyle\ \
+\beta_{p}p\Bigg{|}\sum_{\begin{subarray}{c}i_{1},\ldots,i_{p-1}=1\\\
\text{distinct}\end{subarray}}^{N-1}\tau^{m^{\prime},l^{\prime}}_{i_{1}}\cdots\tau^{m^{\prime},l^{\prime}}_{i_{p-1}}\big{\langle}\tau^{m,1}_{i_{1}}\cdots\tau^{m,p-1}_{i_{p-1}}\big{\rangle}_{N-1,\beta^{\prime}}^{\rho}-q_{P}^{p-1}\Bigg{|}+o_{N}(1)$

and thus,

$\displaystyle\big{\langle}\big{|}\mathbb{E}_{g_{\cdot
N}}[S_{\rho}^{m,0}S_{\rho}^{m^{\prime},l^{\prime}}]\big{|}\big{\rangle}_{N-1,\beta^{\prime}}^{\rho}\leq
4\beta_{p}p\bigg{\langle}\bigg{|}\prod_{\ell=1}^{p-1}R(\tau^{1,\ell},\tau^{2,l^{\prime}})-q_{P}^{p-1}\bigg{|}\bigg{\rangle}_{N-1,\beta^{\prime}}^{\rho}+o_{N}(1).$

Case 2: $m=m^{\prime}$.

  * (2a)

If $l^{\prime}=0$, then $\mathbb{E}[\xi_{0}^{m}\xi_{0}^{m}]=p(1-q_{P}^{p-1})$,
and

$\displaystyle\big{\langle}\big{|}\mathbb{E}_{g_{\cdot
N}}[S_{\rho}^{m,0}S_{\rho}^{m,0}]-\mathbb{E}[\xi_{0}^{m}\xi_{0}^{m}]\big{|}\big{\rangle}_{N-1,\beta^{\prime}}^{\rho}$
$\displaystyle=\left\langle\Big{|}\mathbb{E}_{g_{\cdot
N}}\bigl{[}\dot{Z}_{N,p}^{\rho}({\boldsymbol{\tau}})^{2}\bigr{]}-p(1-q_{P}^{p-1})\Big{|}\right\rangle_{N-1,\beta^{\prime}}^{\rho}$
$\displaystyle\leq
3p\bigg{\langle}\Big{|}\prod_{\ell=1}^{p-1}R(\tau^{1,\ell},\tau^{2,\ell})-q_{P}^{p-1}\Big{|}\bigg{\rangle}_{N-1,\beta^{\prime}}^{\rho}+o_{N}(1).$

  * (2b)

If $l^{\prime}\neq 0$, then
$\mathbb{E}[\xi_{0}^{m}\xi_{l^{\prime}}^{m}]=\beta_{p}pq_{P}^{p-2}(1-q_{P})$,
and

$\displaystyle\big{\langle}\big{|}\mathbb{E}_{g_{\cdot
N}}[S_{\rho}^{m,0}S_{\rho}^{m,l^{\prime}}]-\mathbb{E}[\xi_{0}^{m}\xi_{l^{\prime}}^{m}]\big{|}\big{\rangle}_{N-1,\beta^{\prime}}^{\rho}$
$\displaystyle=\left\langle\Big{|}\mathbb{E}_{g_{\cdot
N}}\bigl{[}\dot{Z}_{N,p}^{\rho}(\boldsymbol{\tau})\dot{X}_{N,\beta}^{\rho}(\tau^{m,l^{\prime}})\bigr{]}-\beta_{p}pq_{P}^{p-2}(1-q_{P})\Big{|}\right\rangle_{N-1,\beta^{\prime}}^{\rho}$
$\displaystyle\leq\beta_{p}p\bigg{\langle}\Big{|}\prod_{\begin{subarray}{c}\ell=1\\\
\ell\neq
l^{\prime}\end{subarray}}^{p-1}R(\tau^{m,\ell},\tau^{m,l^{\prime}})-q_{P}^{p-2}\Big{|}\bigg{\rangle}_{N-1,\beta^{\prime}}^{\rho}$
$\displaystyle\ \
+3\beta_{p}p\bigg{\langle}\Big{|}\prod_{\ell=1}^{p-1}R(\tau^{1,\ell},\tau^{2,l^{\prime}})-q_{P}^{p-1}\Big{|}\bigg{\rangle}_{N-1,\beta^{\prime}}^{\rho}+o_{N}(1).$

From (1a), (1b), (2a), and (2b), we can use the concentration of the overlap
in Lemma 3.9 to conclude that (52) holds as long as one of the $l,l^{\prime}$
equals zero. This completes our proof. ∎

###  5.2 Derivation of the Onsager correction term

Recall $s^{\rho}$ from (30). We proceed to show that $X_{N,\beta}(s^{\rho})$
is asymptotically equal to the sum of the cavity field