Therefore a vector $y_{2}/\beta$ can be computed in time
$O(t\cdot(\textnormal{{nnz}}(B)+(\textnormal{{nnz}}(A)+c^{2})\log\left(\frac{\kappa^{2}\|r\|_{1}}{\varepsilon_{\textnormal{f}}\varepsilon}\right)))+O((\textnormal{{nnz}}(A)+c^{2})\log(\frac{1}{\varepsilon_{\textnormal{f}}\varepsilon}))$.

Now we compute an approximation to
$\mathcal{M}^{\prime\mathsf{T}}v=(M^{\mathsf{T}}B^{\mathsf{T}}AA^{+}/\beta)v$
for an arbitrary vector $v$. We first compute a vector $y_{1}$ such that

$\|{AA^{+}v-y_{1}}\|_{2}\leq\varepsilon_{\textnormal{reg}}\|{(I-AA^{+})v}\|_{2}\leq\varepsilon_{\textnormal{reg}}\|{v}\|_{2}.$

Then we compute $B^{\mathsf{T}}y_{1}$ exactly. Then we compute a vector
$y_{2}$ such that
$\|{MB^{\mathsf{T}}y_{1}-y_{2}}\|_{2}\leq\varepsilon_{\textnormal{r}}\|{B^{\mathsf{T}}y_{1}}\|_{2}\leq\varepsilon_{\textnormal{r}}\|{B}\|_{2}(1+\varepsilon_{\textnormal{reg}})\|{v}\|_{2}$
. We further have

$\|{MB^{\mathsf{T}}AA^{+}v-MB^{\mathsf{T}}y_{1}}\|_{2}\leq\varepsilon_{\textnormal{reg}}\|{MB^{\mathsf{T}}}\|_{2}\|{v}\|_{2}\leq\varepsilon_{\textnormal{reg}}\frac{2\|{B}\|_{2}}{\sqrt{\varepsilon}}\|{v}\|_{2}.$

Thus

$\|{y_{2}-MB^{\mathsf{T}}AA^{+}v}\|_{2}\leq
2\varepsilon_{\textnormal{r}}\|{B}\|_{2}\|{v}\|_{2}+\varepsilon_{\textnormal{reg}}\frac{2\|{B}\|_{2}}{\sqrt{\varepsilon}}\|{v}\|_{2}$

and hence

$\|{y_{2}/\beta-\mathcal{M}^{\prime\mathsf{T}}v}\|_{2}\leq
2\varepsilon_{\textnormal{r}}\kappa\|{v}\|_{2}+\varepsilon_{\textnormal{reg}}\frac{2\kappa}{\sqrt{\varepsilon}}\|{v}\|_{2}$

Now picking
$\varepsilon_{\textnormal{r}}=O(\varepsilon_{\textnormal{f}}\varepsilon/\kappa)$
and
$\varepsilon_{\textnormal{reg}}=O(\varepsilon_{\textnormal{f}}\varepsilon^{3/2}/\kappa)$,
we obtain that

$\|{(y_{2}/\beta)-\mathcal{M}^{\prime\mathsf{T}}v}\|_{2}\leq\varepsilon_{\textnormal{f}}\varepsilon\|{v}\|_{2}\leq\varepsilon_{\textnormal{f}}\|{\mathcal{M}^{\prime}}\|_{2}\|{v}\|_{2}.$

Thus, this approximation can be computed in time
$O(t\cdot(\textnormal{{nnz}}(B)+(\textnormal{{nnz}}(A)+c^{2})\log\left(\frac{\kappa^{2}\|r\|_{1}}{\varepsilon_{\textnormal{f}}\varepsilon}\right)))+O((\textnormal{{nnz}}(A)+c^{2})\log(\frac{\kappa}{\varepsilon_{\textnormal{f}}\varepsilon}))$.
It follows that given an accuracy parameter $\varepsilon_{\textnormal{f}}$, we
can compute approximate matrix-vector products with $\mathcal{M}^{\prime}$ and
$\mathcal{M}^{\prime\mathsf{T}}$ in time at most

$\displaystyle T(\varepsilon_{\textnormal{f}})$
$\displaystyle=O(t\cdot(\textnormal{{nnz}}(B)+(\textnormal{{nnz}}(A)+c^{2})\log\left({\kappa(B)^{2}\|r\|_{1}}/{(\varepsilon_{\textnormal{f}}\varepsilon})\right)))$
$\displaystyle\quad+O((\textnormal{{nnz}}(A)+c^{2})\log({\kappa(B)}/({\varepsilon_{\textnormal{f}}\varepsilon}))).$

###  D.4 Proof of Theorem 6.4

######  Proof D.13.

From Lemma 6.2,

$\sigma_{1}(\mathcal{M}^{\prime})\leq\sigma_{1}\left(\frac{AA^{+}B}{\beta}\right)\|{M}\|_{2}\leq\sigma_{1}\left(\frac{AA^{+}B}{\beta}\right)\frac{2}{\sqrt{\varepsilon}}$

and

$\sigma_{k+1}(\mathcal{M^{\prime}})\geq\sigma_{k+1}(AA^{+}B/\beta)\cdot\sigma_{\min}(M)\geq\sigma_{k+1}(AA^{+}B/\beta)(1/2).$

Therefore
$\kappa(\mathcal{M}^{\prime})\leq\sigma_{1}(AA^{+}B/\beta)(2/\sqrt{\varepsilon})/\sigma_{k+1}(AA^{+}B/\beta)/2\leq(4/\sqrt{\varepsilon})\kappa(AA^{+}B)$.
By Theorem 5.1, we can compute a matrix $Z\in\mathbb{R}^{n\times k}$ such that
$\|{(I-ZZ^{\mathsf{T}})\mathcal{M^{\prime}}}\|_{2}\leq(1+2\varepsilon)\sigma_{k+1}(\mathcal{M}^{\prime})$
in time

$T\left(\frac{\varepsilon}{\kappa(\mathcal{M}^{\prime})^{5q}k^{11}C^{q}}\right)qk+T\left(\frac{\varepsilon^{2}}{48\kappa(\mathcal{M^{\prime}}^{2}(\sqrt{qk})k)}\right)qk,$

where $q=O((1/\sqrt{\varepsilon})\log(d/\varepsilon))$. Thus the total time
required is

$O\left(tqk\cdot\left(\textnormal{{nnz}}(B)+(\textnormal{{nnz}}(A)+c^{2})\log\left(\frac{\kappa^{2}\|r\|_{1}\kappa(\mathcal{M}^{\prime})^{5q}k^{11}C^{q}}{\varepsilon^{2}}\right)\right)\right).$

As
$\|r\|_{1}=(1+\sqrt{2})^{O(1/\sqrt{\varepsilon}\log(\kappa/\varepsilon))}\log(\kappa/\varepsilon)/\varepsilon$
and $\kappa(\mathcal{M}^{\prime})=\kappa(AA^{+}B)/\sqrt{\varepsilon}$, we
obtain that the total time required is
$O(tqk\cdot\textnormal{{nnz}}(B)+tqk\cdot(\frac{1}{\sqrt{\varepsilon}}\log(\kappa/\varepsilon)+q)\log(\frac{\kappa\cdot\kappa(\mathcal{M}^{\prime})\cdot
k}{\varepsilon})\cdot(\textnormal{{nnz}}(A)+c^{2})).$ Substituting
$t=O(\sqrt{1/\varepsilon}\log(\kappa/\varepsilon))$, we obtain that the total
running time is

$O\left(\left(\frac{\textnormal{{nnz}}(B)\cdot
k}{\varepsilon}+\frac{\textnormal{{nnz}}(A)\cdot
k}{\varepsilon^{1.5}}+\frac{c^{2}k}{\varepsilon^{1.5}}\right)\cdot\text{polylog}(\kappa,\kappa(AA^{+}B),d,k,1/\varepsilon)\right),$
(11)

and there is an additional $c^{\omega}$ time for computing a preconditioner.
By Lemmas 6.1 and 6.2, we obtain that

$\|{(AA^{+}Z)(AA^{+}Z)^{+}B-B}\|_{2}=\|{ZZ^{\mathsf{T}}B-B}\|_{2}\leq(1+O(\varepsilon))\textnormal{Opt}.$

The equality is from the fact that $Z$ is spanned by the columns of matrix $A$
by Theorem 5.1, and therefore $AA^{+}Z=Z$. Thus, there exists a matrix
$X_{1}\in\mathbb{R}^{c\times k}$ such that $AX_{1}=Z$ and the matrix $X_{1}$
can be computed in time $O((\textnormal{{nnz}}(A)+c^{2})k+c^{\omega})$ using
sketching-based preconditioning techniques. Let $Y_{1}=Z^{\mathsf{T}}B$, which
can be computed in time $O(\textnormal{{nnz}}(B)\cdot k)$. Therefore,

$\|{AX_{1}Y_{1}-B}\|_{2}=\|{ZZ^{\mathsf{T}}B-B}\|_{2}\leq(1+O(\varepsilon))\textnormal{Opt}.$

Thus $X_{1}\cdot Y_{1}$ is a $(1+O(\varepsilon))$-approximation to the
regression problem. By appropriately scaling $\varepsilon$, we obtain the
proof.

###  D.5 Proof of Lemma 6.5

######  Proof D.14.

Let $G\sim N(0,1)^{n\times(k+1)}$ and
$F^{\mathsf{T}}\in\mathbb{R}^{(k+1)\times d}$ be a matrix with $k+1$
orthonormal rows. Let $\alpha$ be a parameter to be chosen later and
$\widetilde{B}:=B+\alpha GF^{\mathsf{T}}$. For all matrices $X$, by the
triangle inequality,

$\|{AX-\widetilde{B}}\|_{2}\in\|{AX-B}\|_{2}\pm\alpha\|{GF^{\mathsf{T}}}\|_{2}.$

With probability $\geq 9/10$, $\|{G}\|_{2}\leq 2\sqrt{n}$. Thus
$\|{AX-\widetilde{B}}\|_{2}\in\|{AX-B}\|_{2}\pm 2\alpha\sqrt{n}.$ Therefore,
if $\widetilde{X}$ is a $(1+\varepsilon)$-approximation to
$\min_{\text{rank-}k\ X}\|{AX-\widetilde{B}}\|_{2}$, then
$\|{A\widetilde{X}-B}\|_{2}\leq(1+\varepsilon)\textnormal{Opt}+6\alpha\sqrt{n}$.

We now have
$\sigma_{1}(AA^{+}\widetilde{B})\leq\|{\widetilde{B}}\|_{2}\leq\|{B}\|_{2}+2\alpha\sqrt{n}$
from the above discussion. We now lower bound
$\sigma_{k+1}(AA^{+}\widetilde{B})$. Let $U$ be an orthonormal basis for the
columns of $A$. Therefore $AA^{+}=UU^{\mathsf{T}}$.

$\displaystyle\sigma_{k+1}(AA^{+}\widetilde{B})$
$\displaystyle=\sigma_{k+1}(UU^{\mathsf{T}}\widetilde{B})$
$\displaystyle=\sigma_{k+1}(U^{\mathsf{T}}\widetilde{B})$
$\displaystyle=\sigma_{k+1}(U^{\mathsf{T}}B+\alpha
U^{\mathsf{T}}GF^{\mathsf{T}})$
$\displaystyle\geq\sigma_{k+1}(U^{\mathsf{T}}BFF^{\mathsf{T}}+\alpha
U^{\mathsf{T}}GF^{\mathsf{T}})$
$\displaystyle\geq\sigma_{k+1}(U^{\mathsf{T}}BF+\alpha U^{\mathsf{T}}G).$

As the rows of $U^{\mathsf{T}}$ are orthonormal, the matrix
$G^{\prime}=U^{\mathsf{T}}G$ is a matrix of i.i.d. normal random variables.
Assuming $A$ is of full rank, $G^{\prime}$ is a $c\times(k+1)$ matrix.
Assuming $c\geq k+1$, let $E$ be the top $(k+1)\times(k+1)$ submatrix of
$U^{\mathsf{T}}BF+\alpha G^{\prime}$. Then $E$ can be seen as a fixed
$(k+1)\times(k+1)$ matrix where each entry is perturbed by a Gaussian random
variable of variance $\alpha^{2}$. From Theorem 2.2 of [30], we obtain that
$\sigma_{\min}(E)\geq\alpha/(C\sqrt{k})$ for a constant $C$ with probability
$\geq 9/10$. Thus $\sigma_{k+1}(U^{\mathsf{T}}BF+\alpha
U^{\mathsf{T}}G)\geq\sigma_{\min}(E)\geq\alpha/(C\sqrt{k})$.

Thus,
$\sigma_{1}(AA^{+}\widetilde{B})/\sigma_{k+1}(AA^{+}\widetilde{B})\leq(\|{B}\|_{2}+2\alpha\sqrt{n})/(\alpha/(C\sqrt{k}))$.
For $\alpha=\frac{\varepsilon\sigma_{k+1}(B)}{(6\sqrt{n})}$, we obtain that

$\sigma_{1}(AA^{+}\widetilde{B})/\sigma_{k+1}(AA^{+}\widetilde{B})\leq\frac{Cn}{\varepsilon}\kappa$

for a constant $C$ with probability $\geq 4/5$. Also, if $\widetilde{X}$ is a
$(1+\varepsilon)$-approximation as mentioned above,
$\|{A\widetilde{X}-B}\|_{2}\leq(1+\varepsilon)\textnormal{Opt}+\varepsilon\sigma_{k+1}(B)\leq(1+2\varepsilon)\textnormal{Opt}$.
We obtain the proof by scaling $\varepsilon$ appropriately.


# On the Analytic Structure of Second-Order Non-Commutative Probability Spaces
and Functions of Bounded Fréchet Variation

MARIO DIAZ  Instituto de Investigaciones en Matemáticas Aplicadas y en
Sistemas, Universidad Nacional Autónoma de México, Mexico City, Mexico
<EMAIL_ADDRESS>and  JAMES A. MINGO  Department of Mathematics
and Statistics, Queen’s University, Jeffery Hall, Kingston, Ontario, K7L 3N6,
Canada<EMAIL_ADDRESS>
###### Abstract.

In this paper we propose a new approach to the central limit theorem (CLT),
based on functions of bounded Féchet variation for the continuously
differentiable linear statistics of random matrix ensembles which relies on: a
weaker form of a large deviation principle for the operator norm; a Poincaré-
type inequality for the linear statistics; and the existence of a second-order
limit distribution. This approach frames into a single setting many known
random matrix ensembles and, as a consequence, classical central limit
theorems for linear statistics are recovered and new ones are established,
e.g., the CLT for the continuously differentiable linear statistics of block
Gaussian matrices.

In addition, our main results contribute to the understanding of the
analytical structure of second-order non-commutative probability spaces. On
the one hand, they pinpoint the source of the unbounded nature of the bilinear
functional associated to these spaces; on the other hand, they lead to a
general archetype for the integral representation of the second-order Cauchy
transform, $G_{2}$. Furthermore, we establish that the covariance of
resolvents converges to this transform and that the limiting covariance of
analytic linear statistics can be expressed as a contour integral in $G_{2}$.

This work was supported in part by a Discovery Grant from the Natural Sciences
and Engineering Research Council of Canada, and in part by the Consejo
Nacional de Ciencia y Tecnología of Mexico under Grant A1-S-976.

##  1\. Introduction

In his seminal work [36, 37, 38], Wigner established that the empirical
spectral measure of certain random matrix ensembles converges, as the
dimension goes to infinity, to the semicircle distribution. Since then,
several other asymptotic phenomena have been discovered for a wide range of
random matrix ensembles. Examples of these phenomena include, but are not
limited to, large deviations for several spectral objects [8, 21, 20];
convergence and strong convergence of the empirical spectral distribution [23,
17, 22, 1, 10]; asymptotic non-commutative independence between random matrix
ensembles [34, 35, 2, 25], and central limit theorems for linear statistics
[12, 19, 4, 3, 31].

Another such phenomenon, which stems mainly from combinatorial considerations,
is the fact that many random matrix ensembles have a second-order limit
distribution [27, 26, 11]. We say that a random matrix ensemble
$(X_{N})_{N\in\mathbb{N}}$ has a second-order limit distribution if

  * i)

For all $m,n\in\mathbb{N}$, the following limits exist

$\alpha_{n}=\lim_{N\to\infty}\frac{1}{N}\mathbb{E}(\textnormal{Tr}(X_{N}^{n}))\quad\text{
and
}\quad\alpha_{m,n}=\lim_{N\to\infty}\textnormal{Cov}(\textnormal{Tr}(X_{N}^{m}),\textnormal{Tr}(X_{N}^{n})),$

where $\textnormal{Cov}(X,Y)=\mathbb{E}(XY)-\mathbb{E}(X)\mathbb{E}(Y)$;

  * ii)

For all $r\geq 3$ and all $n_{1},\ldots,n_{r}\in\mathbb{N}$,

$\lim_{N\to\infty}k_{r}\left(\textnormal{Tr}(X_{N}^{n_{1}}),\ldots,\textnormal{Tr}(X_{N}^{n_{r}})\right)=0,$

where $k_{r}$ denotes the classical cumulant of order $r$.

In this paper we propose a new approach to the central limit theorem (CLT) for
continuously differentiable linear statistics which is based on three other
phenomena: a weaker form of a large deviation principle for the operator norm;
a Poincaré-type inequality for the linear statistics; and the existence of a
second-order limit distribution. The first two phenomena ensure the existence
of the limiting covariance of linear statistics, while the latter leads to
their asymptotic Gaussianity. Apart from making explicit the relations between
different fundamental phenomena, this approach frames into a single setting
many known random matrix ensembles. As a consequence, classical central limit
theorems for linear statistics are recovered and new ones are established,
e.g., the CLT for the continuously differentiable linear statistics of block
Gaussian matrices.

Let $\lambda_{1},\ldots,\lambda_{N}$ be the eigenvalues of an $N\times N$
random matrix $X_{N}$. Given a function $f:\mathbb{C}\to\mathbb{C}$, the
(random) quantity $\textnormal{Tr}(f(X_{N}))=\sum_{k}f(\lambda_{k})$ is called
a linear statistic of $X_{N}$. Hence, a CLT for the linear statistics of a
random matrix ensemble is a result that establishes that, as $N\to\infty$,

$\textnormal{Tr}(f(X_{N}))-\mathbb{E}(\textnormal{Tr}(f(X_{N})))\Rightarrow\mathcal{N}_{\mathbb{R}}(0,\sigma_{f}^{2}),$

for some $\sigma_{f}^{2}>0$, where $\Rightarrow$ denotes convergence in
distribution and $\mathcal{N}_{\mathbb{R}}(\mu,\sigma^{2})$ denotes the real
Gaussian distribution with mean $\mu$ and variance $\sigma^{2}$. If the family
of functions $f$ for which such a CLT holds is a vector space, then any
$n$-tuple of (centered) linear statistics is, in the limit, jointly Gaussian.
As a result, the covariance mapping

(1) $\langle
f,g\rangle\mapsto\lim_{N\to\infty}\textnormal{Cov}(\textnormal{Tr}(f(X_{N})),\textnormal{Tr}(g(X_{N})))$

plays a privileged role in the description of the asymptotic behavior of the
linear statistics. Indeed, the so-called second-order non-commutative
probability spaces capture this covariance mapping in a bilinear functional,
which is not present in typical non-commutative probability spaces.

A second-order non-commutative probability space is a triple
$(\mathcal{A},\varphi,\allowbreak\rho)$ consisting of a unital algebra
$\mathcal{A}$, a unital linear functional $\varphi:\mathcal{A}\to\mathbb{C}$
which is tracial, and a bilinear functional
$\rho:\mathcal{A}\times\mathcal{A}\to\mathbb{C}$ which is tracial in both
arguments and $\rho(1,a)=\rho(a,1)=0$ for all $a\in\mathcal{A}$. The canonical
example in the single random matrix setting is the following. Let
$(X_{N})_{N}$ be a self-adjoint random matrix ensemble. Define
$\mathcal{A}=\mathbb{C}[x]$ and, for all $p,q\in\mathcal{A}$,

$\displaystyle\varphi(p)$
$\displaystyle=\lim_{N\to\infty}\frac{1}{N}\mathbb{E}(\textnormal{Tr}(p(X_{N})))\quad\text{
and }$ $\displaystyle\rho(p,q)$
$\displaystyle=\lim_{N\to\infty}\textnormal{Cov}(\textnormal{Tr}(p(X_{N})),\textnormal{Tr}(q(X_{N}))).$

We will assume that $\mathcal{A}=C([-M,M])$ and that the linear functional
$\varphi$ is continuous with respect to the supremum norm on $C([-M,M])$, the
space of continuous functions on $[-M,M]$, for some $M>0$. However, it was
observed that in some canonical examples, the bilinear functional $\rho$ is
not continuous (bounded) with respect to the the supremum norm on $C([-M,M])$
for any $M>0$. (Theorem 6 below shows that in fact this is always the case.)
The unboundedness of $\rho$ makes the usual analytic setting from free
probability theory unfitted for the second-order case. In addition to the CLT
for linear statistics, our main results contribute to the understanding of the
analytic structure of second-order non-commutative probability spaces. On the
one hand, they pinpoint the source of the unbounded nature of the bilinear
functional $\rho$; on the other hand, they lead to a general archetype for the
integral representation of the second-order Cauchy transform, i.e., the
generating function given by

(2) $G_{2}(z,w)=\sum_{m,n\geq 1}\frac{\alpha_{m,n}}{z^{m+1}w^{n+1}}.$

Furthermore, we establish that the covariance of resolvents converges to this
transform (see Corollary 8) and that the limiting covariance of analytic
linear statistics can be expressed as a contour integral depending on the
second order Cauchy transform, see Theorem 11. Since the second-order Cauchy
transform of block Gaussian matrices was recently found in [14], these results
provide an effective way to compute the covariance of analytic linear
statistics of block Gaussian matrices.

The organization of this paper is as follows. We present some preliminaries in
the following section. We precisely state our main results on the CLT for the
linear statistics of some random matrices in Section 3, and provide their
proofs in Section 6. In Section 4, we provide some examples of random matrix
ensembles for which our main results apply. In particular, we show that block
Gaussian matrices fall within the scope of our work. We introduce the
definition of analytic second-order non-commutative probability space and
discuss its implications in Section 5. We finish this paper with some
concluding remarks in Section 7.

##  2\. Preliminaries

######  Notation 1.

We let $C^{1}(\mathbb{R})$ denote the space of complex valued functions on
$\mathbb{R}$ which have a continuous derivative. We denote by $f|_{M}$ the
restriction of $f:\mathbb{R}\to\mathbb{C}$ to the interval $[-M,M]$ and we let
$C^{1}([-M,M])=\\{f|_{M}\mid f\in C^{1}(\mathbb{R})\\}$. For $f\in
C^{1}([-M,M])$, we let $\mathaccent 28695{f}(x)=f(x)-f(0)$ and
$C^{1}([-M,M])^{\circ}=\\{\mathaccent 28695{f}\mid f\in C^{1}([-M,M])\\}$.
Then we have a direct sum decomposition

$C^{1}([-M,M])=\mathbb{C}\oplus C^{1}([-M,M])^{\circ}$

and let $\|f\|=|f(0)|+\|f^{\prime}\|_{M}$ where $\|f\|_{M}=\sup_{|x|\leq
M}|f(x)|$. This is a norm on $C^{1}([-M,M])$ and in this norm $C^{1}([-M,M])$
is a Banach space.

###  2.1. Fréchet Representation Theorem for Bilinear Functionals

Given a function $u:[-M,M]^{2}\to\mathbb{R}$ and
$s_{0},s_{1},t_{0},t_{1}\in[-M,M]$, we let

$\Delta
u(s_{0},s_{1};t_{0},t_{1})=u(s_{1},t_{1})-u(s_{1},t_{0})-u(s_{0},t_{1})+u(s_{0},t_{0}).$

######  Definition 2.

We say $u:[-M,M]^{2}\to\mathbb{R}$ has bounded Fréchet variation if

$\sum_{i=1}^{m}\sum_{j=1}^{n}\sigma_{i}\theta_{j}\Delta
u(s_{i-1},s_{i};t_{j-1},t_{j})$

is uniformly bounded for all $m,n\in\mathbb{N}$,
$\sigma_{i},\theta_{j}\in\\{\pm 1\\}$, for all $-M=s_{0}\leq
s_{1}\leq\cdots\leq s_{m}=M$, and all $-M=t_{0}\leq t_{1}\leq\cdots\leq
t_{n}=M$. If $u:[-M,M]^{2}\to\mathbb{C}$ has real and imaginary parts which
are of bounded Fréchet variation, then we say that the complex valued function
has bounded Fréchet variation.

######  Definition 3.

Let $\phi:C([-M,M])^{2}\rightarrow\mathbb{C}$ be a bilinear function; we say
that $\Phi$ is bounded if $\exists\,K>0$ such that for all $f,g\in C([-M,M])$
we have $|\Phi(f,g)|\leq K\|f\|_{M}\,\|g\|_{M}$, where
$\|f\|_{M}=\sup\\{|f(x)|:x\in[-M,M]\\}$.

The two results of Fréchet we need can be found in [18, Ch. III §7 and §11]
and [16, §6]. See also [29] and [30]. For a partition
$s=(s_{0},s_{1},\ldots,s_{n})$ of $[-M,M]$, i.e., $-M=s_{0}\leq
s_{1}\leq\cdots\leq s_{n}=M$, we let $\lvert
s\rvert=\max\\{s_{i}-s_{i-1}:1\leq i\leq n\\}$.

######  Theorem 4 (Fréchet).

If $u:[-M,M]^{2}\to\mathbb{R}$ has bounded Fréchet variation, then for all
$f,g\in C([-M,M])$, the following limit exists

$\Phi(f,g):=\lim_{\lvert s\rvert,\lvert t\rvert\to
0}\sum_{i}\sum_{j}f(\xi_{i})g(\eta_{j})\Delta u(s_{i-1},s_{i};t_{j-1},t_{j}),$

where $\xi_{i}\in[s_{i-1},s_{i}]$ and $\eta_{j}\in[t_{j-1},t_{j}]$. Moreover
there is $K\geq 0$ such that $|\Phi(f,g)|\leq K\lVert f\rVert_{M}\lVert
g\rVert_{M}$.

Given its similarities with the Riemann-Stieltjes integral for functions of
bounded variation, the limit $\Phi(f,g)$ in the previous theorem is often
called the _Fréchet integral_ of $f$ and $g$ with respect to $u$ and it is
denoted by

$\int_{-M}^{M}\int_{-M}^{M}f(x)g(y)\mathrm{d}u(x,y).$

Observe that if $u$ has bounded variation, then it also has bounded Fréchet
variation and, for continuous functions $f$ and $g$, both the Riemann-
Stieltjes and Fréchet integrals coincide. In words, the previous theorem
establishes that the Fréchet integral exists and describes a bounded bilinear
functional. The following theorem establishes that the converse is also true:
a bounded bilinear functional is the Fréchet integral with respect to some
function of bounded Fréchet variation.

######  Theorem 5 (Fréchet).

If $\Phi:C([-M,M])^{2}\to\mathbb{R}$ is a bounded bilinear functional, then
there exists a function $u:[-M,M]^{2}\to\mathbb{R}$ of bounded Fréchet
variation such that

$\Phi(f,g)=\int_{-M}^{M}\int_{-M}^{M}f(x)g(y)\,\mathrm{d}u(x,y).$

##  3\. CLT for the Linear Statistics of Some Random Matrices

Let $(X_{N})_{N}$ be a random matrix ensemble. For all $m,n\in\mathbb{N}$, we
let

$\alpha_{n}\coloneqq\lim_{N\to\infty}\frac{1}{N}\mathbb{E}\left(\textnormal{Tr}(X_{N}^{n})\right)\text{
and
}\alpha_{m,n}\coloneqq\lim_{N\to\infty}\textnormal{Cov}\left(\textnormal{Tr}(X_{N}^{m}),\textnormal{Tr}(X_{N}^{n})\right).$

The collections $(\alpha_{n})_{n}$ and $(\alpha_{m,n})_{m,n}$ are the first
and second-order moments of $(X_{N})_{N}$, respectively. In this paper we
consider the following set of assumptions.

  * A0.

Both the first and second-order moments exist.

  * A1.

There exists $M>0$ such that $\displaystyle\lim_{N\to\infty}N^{8}\Pr(\lVert
X_{N}\rVert>M)=0$.

  * A2.

There exists $K>0$ such that, for all $f\in C^{1}(\mathbb{R})$ and
$N\in\mathbb{N}$,

$\textnormal{Var}\left(\textnormal{Tr}(f(X_{N}))\right)\leq
K\|f^{\prime}\|_{\infty}^{2}.$

Observe that A0 corresponds to Part i) in the definition of second-order limit
distribution. Also, observe that A1 is weaker than a large deviation principle
for the operator norm. Finally, note that A2 is a matricial version of the
Poincaré inequality, see, e.g., Proposition 4.1 in [17] and references
therein. In Section 4 we show that these assumptions are satisfied by some
random matrix ensembles which are common in the literature.

We define the set of smooth (test) functions $\mathcal{S}$ as

$\mathcal{S}=\left\\{f\in C^{1}(\mathbb{R}):f\text{ is polynomially
bounded}\right\\}.$

Note that assumption A$0$ gives that we can define a bi-linear map
$\rho:\mathbb{C}[x]\times\mathbb{C}[x]\rightarrow\mathbb{C}$ by linearly
extending the definition $\rho(x^{m},x^{n}):=\alpha_{m,n}$.

In the following theorem we show that under assumptions A0, A1, and A2, $\rho$
extends to a bilinear function on $C^{1}([-M,M])$, and the covariance of
smooth linear statistics converges, as $N\rightarrow\infty$, to $\rho$, and is
bounded by $K$, where the $K$ is that of assumption A2.

This shows that the two ways one might define $\rho(f,g)$, the first using
fluctuation moments $(a)$ below, the second using linear statistics $(b)$
below, agree. This is the main technical part of the paper.

######  Theorem 6.

If $(X_{N})_{N}$ is a self-adjoint random matrix ensemble satisfying A0, A1,
and A2, and $\rho$ the linear extension of the second order moments in A$0$ to
$\mathbb{C}[x]$, then,

  1. $(a)$

$\rho$ extends to a bi-linear function on $C^{1}([-M,M])\times C^{1}([-M,M])$
satisfying

(3) $|\rho(f,g)|\leq K\|f^{\prime}\|_{M}\|g^{\prime}\|_{M};$

  2. $(b)$

for $f$ and $g$ in $\mathcal{S}$

(4)
$\lim_{N\to\infty}\textnormal{Cov}(\textnormal{Tr}(f(X_{N})),\textnormal{Tr}(g(X_{N})))=\rho(f|_{M},g|_{M})$

where $f|_{M}$ denotes the restriction of $f$ to $[-M,M]$;

  3. $(c)$

there exists $u:[-M,M]^{2}\rightarrow\mathbb{R}$, of bounded Fréchet
variation, such that

$\rho(f,g)=\int_{-M}^{M}\int_{-M}^{M}f^{\prime}(x)g^{\prime}(y)\,\mathrm{d}u(x,y),$

where the double integral is in the sense of Fréchet as discussed in Section
2.1.

By $(a)$, $\rho:C^{1}([-M,M])\times C^{1}([-M,M])\to\mathbb{C}$ is a well-
defined bounded bilinear functional which satisfies an asymptotic version of
A2. Indeed, observe that A2 is equivalent to requiring that, for all $f,g\in
C^{1}(\mathbb{R})$ and $N\in\mathbb{N}$,

$\lvert\textnormal{Cov}(\textnormal{Tr}(f(X_{N})),\textnormal{Tr}(g(X_{N})))\rvert\leq
K\|f^{\prime}\|_{\infty}\|g^{\prime}\|_{\infty}.$

In the presence of A$1$ we get the stronger statement $(b)$. The boundedness
of $\rho$, as established in (3), motivates Definition 15 of an analytic
second-order NCPS in Section 5.

######  Notation 7.

For each $z\in\mathbb{C}\setminus\mathbb{R}$, let
$r_{z}:\mathbb{R}\to\mathbb{C}$ be the function defined by

$r_{z}(x)=\frac{1}{z-x}.$

Observe that $r_{z}$ is a smooth function, i.e., $r_{z}\in\mathcal{S}$.
Therefore, Theorem 6 implies that $\rho(r_{z},r_{w})$ is well-defined for
every $z,w\in\mathbb{C}\setminus\mathbb{R}$ and gives us the following
corollary.

######  Corollary 8.

For every $z,w\in\mathbb{C}\setminus\mathbb{R}$,

$\rho(r_{z},r_{w})=\lim_{N\to\infty}\textnormal{Cov}\left(\textnormal{Tr}((z-X_{N})^{-1}),\textnormal{Tr}((w-X_{N})^{-1})\right).$

Since $\alpha_{m,n}=\rho(x^{m},x^{n})$ for all $m,n\geq 1$, (3) readily
implies that

$|\alpha_{m,n}|\leq KmnM^{m+n-2}.$

Thus the power series

$\sum_{m,n\geq 1}\frac{\alpha_{m,n}}{z^{m+1}w^{n+1}}$

determines an analytic function on $\\{(z,w)\in\mathbb{C}^{2}:|z|,|w|>M\\}$.

######  Definition 9.

Let $z,w\in\mathbb{C}$ with $|z|,|w|>M$.We let

(5) $G_{2}(z,w)=\sum_{m,n\geq 1}\frac{\alpha_{m,m}}{z^{m+1}w^{n+1}}$

We call $G_{2}$ the second order Cauchy transform of the moment sequence
$\\{\alpha_{m,m}\\}_{m,n\geq 1}$. $G_{2}$ is analytic on
$\\{(z,w)\in\mathbb{C}^{2}:|z|,|w|>M\\}$.

The next theorem shows that $G_{2}$ can be analytically extended to
$(\mathbb{C}\setminus[-M,M])^{2}$ and that $G_{2}(z,w)$ coincides with
$\rho(r_{z},r_{w})$ for $z,w\in\mathbb{C}\setminus\mathbb{R}$. As a result,
the following theorem establishes a connection between the limit of the
covariance of resolvents and the second-order Cauchy transform.

######  Theorem 10.

If $(X_{N})_{N}$ is a self-adjoint random matrix ensemble satisfying A0, A1,
and A2, then $G_{2}$ can be analytically extended to
$(\mathbb{C}\setminus[-M,M])^{2}$ and, for all
$z,w\in\mathbb{C}\setminus\mathbb{R}$,

$G_{2}(z,w)=\rho(r_{z},r_{w}).$

In [14], Diaz et al. recently found a formula for the second-order Cauchy
transform of block Gaussian matrices. Specifically, they derive a formula at
the level of formal expressions and then extend it to the analytic level.
Since block Gaussian matrices satisfy A0, A1, and A2 (see Section 4.2 below),
Theorem 10 implies that their formula for the second-order Cauchy transform is
indeed equal to the limit of the covariance of resolvents.

The next theorem provides a formula for the limit of the covariance of certain
functions in terms of the second-order Cauchy transform. Given the tools
available to compute the latter, the following theorem provides an effective
way to evaluate the asymptotic covariance mapping $\rho$.

######  Theorem 11.

Assume that $f,g\in\mathcal{S}$ satisfy that $f|_{M}$ and $g|_{M}$ extend
analytically to a complex domain $\Omega\supset[-M,M]$. If $(X_{N})_{N}$ is a
self-adjoint random matrix ensemble satisfying A0, A1, and A2, then

(6) $\rho(f,g)=\frac{1}{(2\pi
i)^{2}}\int_{\mathcal{C}}\int_{\mathcal{C}}f(z)g(w)G_{2}(z,w)\mathrm{d}z\mathrm{d}w,$

where $\mathcal{C}\subset\Omega$ is a positively oriented simple closed
contour enclosing $[-M,M]$.

We end this section with a central limit theorem for the linear statistics of
random matrix ensembles having a second-order limit distribution. The
statement of following proposition is similar to Proposition 3.2.9 in [31]. As
the proof is different, we have provided for the reader’s convenience a proof
in Appendix A using the notation and techniques from §6.

######  Proposition 12.

If $(X_{N})_{N}$ is a self-adjoint random matrix ensemble having a second-
order limit distribution and satisfying A1 and A2, then, for all
$f\in\mathcal{S}$ with $f(\mathbb{R})\subset\mathbb{R}$,

$\textnormal{Tr}(f(X_{N}))-\mathbb{E}(\textnormal{Tr}(f(X_{N})))\Rightarrow{\mathcal{N}}_{\mathbb{R}}(0,\rho(f,f)),$

where $\Rightarrow$ denotes convergence in distribution.

##  4\. Examples of Random Matrix Ensembles

In this section we gather some examples of random matrix ensembles satisfying
A0, A1 and A2. In particular, Example 2 shows that block Gaussian matrices
fall under the framework of our main results.

###  4.1. Example 1: Gaussian Unitary Ensemble

Let $(X_{N})_{N}$ be the (normalized) Gaussian Unitary Ensemble (GUE), i.e.,
for each $N\in\mathbb{N}$, $X_{N}$ is an $N\times N$ self-adjoint random
matrix such that $\\{X_{N}(i,j):1\leq i\leq j\leq N\\}$ are independent random
variables with $X_{N}(i,i)\sim{\mathcal{N}}_{\mathbb{R}}(0,N^{-1})$ and
$X_{N}(i,j)\sim{\mathcal{N}}_{\mathbb{C}}(0,N^{-1})$ ($i\neq j$). In this
case:

  * 0)

$(X_{N})_{N}$ has a second-order limit distribution. See Theorem 3.1 in [27].

  * 1)

For all $\epsilon>0$, there exists $C>0$ such that

(7) $\mathbb{P}(\|X_{N}\|>2(1+\epsilon))\leq
2C\exp\left(-\frac{2\epsilon^{2}}{C}N\right).$

In particular, A1 is satisfied for every $M>2$. See (1.4) in [20].

  * 2)

If $f:\mathbb{R}\to\mathbb{C}$ is differentiable, then

(8)
$\textnormal{Var}(\textnormal{Tr}(f(X_{N})))\leq\|f^{\prime}\|_{\infty}^{2}.$

See Proposition 2.1.8 in [31]. This shows that $(X_{N})_{N}$ satisfies A2 with
$K=1$.

As an additional comment, the second-order Cauchy transform of this ensemble
is given by

(9)
$G_{2}(z,w)=\frac{G^{\prime}(z)G^{\prime}(w)}{[G(z)-G(w)]^{2}}-\frac{1}{(z-w)^{2}},$

where $\displaystyle G(z)=\frac{z-\sqrt{z^{2}-4}}{2}$. See (7) in [11]. In
Theorem 3.1.1 in [31], it was established that

(10) $\displaystyle\lim_{N\to\infty}$
$\displaystyle\textnormal{Cov}(\textnormal{Tr}((z-X_{N})^{-1}),\textnormal{Tr}((w-X_{N})^{-1}))$
$\displaystyle=\frac{1}{2(z-w)^{2}}\left(\frac{zw-4}{\sqrt{z^{2}-4}\sqrt{w^{2}-4}}-1\right).$

Theorem 10 leads to the equality of the right hand sides of (9) and (10).
Given the relative simplicity of these expressions, it can be shown directly
that they are actually equal. (Exercise for the reader.)

We would like to point out that the Wishart/Laguerre ensemble has a second-
order limit distribution and satisfies A1 and A2. See [24], [15], Theorem 3.5
in [27], Theorem 2 in [20], and Proposition 7.2.1 in [31]. As with the right
hand sides of (9) and (10), Theorem 10 establishes the non-trivial fact that
the free probability theory and the random matrix theory expressions for the
second-order Cauchy transform of this ensemble are equal.

###  4.2. Example 2: Block Gaussian Matrices

Let $A_{1},\ldots,A_{r}$ be $d\times d$ self-adjoint matrices. Assume that
$X_{N}^{(1)},\ldots,X_{N}^{(r)}$ are independent GUE matrices as in Example 1.
The $dN\times dN$ random matrix

$X_{N}=\sum_{k=1}^{r}A_{k}\otimes X_{N}^{(k)}$

is called a block Gaussian matrix. In this case:

  * 0)

Recall that $\textnormal{Tr}(A\otimes
B)=\textnormal{Tr}(A)\textnormal{Tr}(B)$. Hence

$\textnormal{Tr}(X_{N}^{n})=\sum_{k_{1},\ldots,k_{n}=1}^{r}\textnormal{Tr}(A_{k_{1}}\cdots
A_{k_{n}})\textnormal{Tr}(X_{N}^{(k_{1})}\cdots X_{N}^{(k_{n})}).$

Since the $r^{th}$ cumulant, $k_{r}$, is $r$-linear, Theorem 3.1 in [27]
readily shows that $(X_{N})_{N}$ has a second-order limit distribution.

  * 1)

A routine computation shows that, for every $M>0$,

$\mathbb{P}(\|X_{N}\|>M)\leq
r\mathbb{P}(\left\|X_{N}^{(1)}\right\|>\frac{M}{r\max_{k}\|A_{k}\|}).$

In particular, if we take $M_{0}=4r\max_{k}\|A_{k}\|$, then (7) implies that

$\mathbb{P}(\|X_{N}\|>M_{0})\leq 2rC\exp\left(-\frac{2}{C}N\right).$

Thus A1 is satisfied with $M=M_{0}$.

  * 2)

If $f:\mathbb{R}\to\mathbb{C}$ is differentiable, then

$\textnormal{Var}(\textnormal{Tr}(f(X_{N})))\leq
r^{2}\left\|\sum_{k=1}^{r}A_{k}^{2}\right\|^{2}\|f^{\prime}\|_{\infty}^{2}.$

See Proposition 4.7 in [17]. This shows that block Gaussian matrices satisfy
A2.

As mentioned in the introduction, the second-order Cauchy transform of block
Gaussian matrices was recently found in [14], see equation (5).

###  4.3. Example 3: $A+UBU^{*}$

For each $N\in\mathbb{N}$, let $U_{N}$ be an $N\times N$ Haar unitary. Assume
that $(A_{N})_{N}$ and $(B_{N})_{N}$ are self-adjoint non-random matrix
ensembles such that their eigenvalue distributions converge in distribution
and $\displaystyle T:=\sup_{N\in\mathbb{N}}\max\\{\|A_{N}\|,\|B_{N}\|\\}$ is
finite. Let $X_{N}=A_{N}+U_{N}B_{N}U_{N}^{*}$. In this case:

  * 0)

$(X_{N})_{N}$ has a second-order limit distribution. See Theorem 1 in [26].

  * 1)

For all $N\in\mathbb{N}$, $\|X_{N}\|\leq 2T$. In particular $X_{N}$ satisfies
A1 with $M=2T$.

  * 2)

If $f:\mathbb{R}\to\mathbb{C}$ is differentiable, then

$\textnormal{Var}(\textnormal{Tr}(f(X_{N})))\leq
4T^{2}\|f^{\prime}\|_{\infty}^{2}.$

Thus $X_{N}$ satisfies A2 with $K=4T^{2}$. See page 303 in [31].

For notational convenience, let $\mu_{A}$ and $\mu_{B}$ be the limiting
eigenvalue distributions of $(A_{N})_{N}$ and $(B_{N})_{N}$, respectively. By
Theorem 10,

$G_{2}(z,w)=\lim_{N\to\infty}\textnormal{Cov}(\textnormal{Tr}((z-X_{N})^{-1}),\textnormal{Tr}((w-X_{N})^{-1})).$

In the notation of Chapter 3 in [28], Theorem 10.2.1 and (10.2.30) in [31]
imply that

$G_{2}(z,w)=\frac{\partial^{2}}{\partial z\partial
w}\log\frac{\omega_{A}(z)-\omega_{A}(w)}{z-w}\frac{\omega_{B}(z)-\omega_{B}(w)}{F(z)-F(w)},$

where $\omega_{A}$ and $\omega_{B}$ are the so-called subordination functions
and $F(z)=1/G_{\mu_{A}\boxplus\mu_{B}}(z)$. Revisiting this expression for the
second-order Cauchy transform is of interest in view of the recent
developments in random matrix theory based on the subordination functions,
e.g., [6, 7, 33, 9, 5].

##  5\. Analytic Second-Order Non-Commutative Probability Spaces

In order to motivate the definition of analytic second-order NCPS below,
recall the integral representation for the asymptotic covariance $\rho$:

(11)
$\rho(f,g)\coloneqq\int_{-M}^{M}\int_{-M}^{M}f^{\prime}(x)g^{\prime}(y)\mathrm{d}u(x,y).$

Observe that the integral representation in (11) only depends on $f|_{M}$ and
$g|_{M}$. Since $f|_{M}\in C^{1}([-M,M])$ for every polynomially bounded $f\in
C^{1}(\mathbb{R})$, in the sequel we can restrict our attention to the
function space $C^{1}([-M,M])$.

We define the concept of analytic second-order NCPS motivated by the integral
representation of the asymptotic covariance mapping associated to a random
matrix ensemble in (11).

######  Definition 13.

A second-order NCPS $(\mathcal{A},\varphi,\rho)$, with
$\mathcal{A}=C^{1}([-M,\allowbreak M])$, for some $M>0$, is called _analytic_
if

  * $(a)$

there exists a probability measure $\mu$ on $[-M,M]$ such that, for all
$f\in\mathcal{A}$,

$\varphi(f)=\int f(x)\mathrm{d}\mu(x);$

  * $(b)$

there exists a bounded Fréchet variation function
$u:[-M,M]^{2}\allowbreak\to\mathbb{R}$ such that, for all $f,g\in\mathcal{A}$,

(12)
$\rho(f,g)=\int_{-M}^{M}\int_{-M}^{M}f^{\prime}(x)g^{\prime}(y)\mathrm{d}u(x,y).$

Observe that Theorem 6 $(c)$ implies that the second-order NCPS associated to
a random matrix ensemble satisfying A0, A1, and A2 is indeed analytic.
Nonetheless, it is unknown to the authors if every analytic second-order NCPS
arise in this manner, i.e., if for every $u:[-M,M]^{2}\to\mathbb{R}$ of
bounded Fréchet variation there exists a random matrix ensemble whose
asymptotic covariance mapping satisfies (11).

The Fréchet representation theorem establishes that if
$u:[-M,M]^{2}\allowbreak\to\mathbb{R}$ has bounded Fréchet variation, then the
bilinear functional $\Phi:C([-M,M])^{2}\to\mathbb{C}$ defined by

$\Phi(f,g)=\int_{-M}^{M}\int_{-M}^{M}f(x)g(y)\mathrm{d}u(x,y).$

is continuous in each argument with respect to the supremum norm. Since

$\rho=\Phi\circ\left(\frac{\mathrm{d}}{\mathrm{d}x}\times\frac{\mathrm{d}}{\mathrm{d}y}\right),$

we conclude that the unbounded nature of $\rho$ comes from the differential
operator $\frac{\mathrm{d}}{\mathrm{d}x}\times\frac{\mathrm{d}}{\mathrm{d}y}$.
This observation immediately leads to the following criterion for the exchange
of limit and $\rho$.

######  Lemma 14.

Let $(\mathcal{A},\varphi,\rho)$ be an analytic second-order NCPS with
$\mathcal{A}=C^{1}([-M,M])$. Suppose $\\{f_{n}\\}_{n=1}^{\infty}$ and
$\\{g_{n}\\}_{n=1}^{\infty}$ are in $C^{1}([-M,M])$. If
$\\{f_{n}^{\prime}\\}_{n}$ converges uniformly to $f^{\prime}$ and
$\\{g_{n}^{\prime}\\}_{n}$ converges uniformly to $g^{\prime}$, then

$\rho(f,g)=\lim_{n\to\infty}\rho(f_{n},g_{n}).$

Now we turn our attention to the analytic version of the second-order Cauchy
transform. For each $z\in\mathbb{C}\setminus[-M,M]$, let
$r_{z}:[-M,M]\to\mathbb{C}$ be defined by

$r_{z}(x)=\frac{1}{z-x}.$

Observe that $r_{z}$ is a differentiable function, i.e., $r_{z}\in
C^{1}([-M,M])$.

######  Definition 15.

Let $(\mathcal{A},\varphi,\rho)$ be an analytic second-order NCPS. We define
the (analytic) second-order Cauchy transform

$G_{2}:(\mathbb{C}\setminus[-M,M])^{2}\allowbreak\to\mathbb{C}$

as

(13) $G_{2}(z,w)\coloneqq\rho(r_{z},r_{w}).$

Observe that (13) and the integral representation of $\rho$ in (12) imply
that, for all $z,w\in\mathbb{C}\setminus[-M,M]$,

(14)
$G_{2}(z,w)=\int_{-M}^{M}\int_{-M}^{M}\frac{1}{(z-x)^{2}}\frac{1}{(w-y)^{2}}\mathrm{d}u(x,y).$

Indeed, the second-order Cauchy transform of many random matrix ensembles in
the literature have an integral representation of the form (14), see, e.g.,
[4, Eq. (1.7)] and [13, Eq. (2.9)].

In the next proposition we show that the second-order Cauchy transform is an
analytic function whose power series expansion at infinity coincides with the
generating function in (2).

######  Proposition 16.

If $(\mathcal{A},\varphi,\rho)$ is an analytic second-order NCPS with
$\mathcal{A}=C^{1}([-M,M])$, then $G_{2}$ is an analytic function on
$(\mathbb{C}\setminus[-M,M])^{2}$ such that, for all $|z|,|w|>M$,

$G_{2}(z,w)=\sum_{m,n\geq 0}\frac{\rho(x^{m},x^{n})}{z^{m+1}w^{n+1}}.$

###### Proof.

For $|z|>M$ the series, considered as a function of $x$,
$\sum_{n=1}^{\infty}\frac{nx^{n-1}}{z^{n+1}}$ converges uniformly to
$(z-x)^{-2}$ on $[-M,M]$. Likewise, for $|w|>M$, the series
$\sum_{n=1}^{\infty}\frac{ny^{n-1}}{w^{n}}$ converges uniformly to
$(w-y)^{-2}$ on $[-M,M]$. By Lemma 14 we have

$\displaystyle G_{2}(z,w)$
$\displaystyle=\int_{-M}^{M}\int_{-M}^{M}(z-x)^{-2}(w-y)^{-2}\,du(x,y)$
$\displaystyle=\sum_{m,n\geq
1}\frac{mn}{z^{m+1}w^{n+1}}\int_{-M}^{M}\int_{-M}^{M}x^{m-1}y^{n-1}\,du(x,y)$
$\displaystyle=\sum_{m,n\geq 1}\frac{\rho(x^{m},x^{n})}{z^{m+1}w^{n+1}},$

as required. ∎

Note that the previous proposition generalizes Theorem 10. Indeed, if the
second-order NCPS $(\mathcal{A},\varphi,\rho)$ is associated to a random
matrix ensemble satisfying A0, A1, and A2, then the previous proposition is a
simple consequence of Theorem 10. In a similar spirit, the following theorem
generalizes Theorem 11.

######  Theorem 17.

Assume that $(\mathcal{A},\varphi,\rho)$ is an analytic second-order NCPS. If
$f,g\in C^{1}([-M,M])$ extend analytically to a complex domain
$\Omega\supset[-M,M]$, then

(15) $\rho(f,g)=\frac{1}{(2\pi
i)^{2}}\int_{\mathcal{C}}\int_{\mathcal{C}}f(z)g(w)G_{2}(z,w)\mathrm{d}z\mathrm{d}w,$

where $\mathcal{C}\subset\Omega$ is a positively oriented simple closed
contour enclosing $[-M,M]$.

###### Proof.

On the right hand side of (15) we may interchange the integral for
$G_{2}(z,w)$ given by (14) with the two contour integrals, as all the
integrals are over compact sets. Then for $x\in[-M,M]$ we have
$f^{\prime}(x)=\frac{1}{2\pi i}\int_{C}f(z)(z-x)^{-2}\,\mathrm{d}z$; likewise
$g^{\prime}(y)=\frac{1}{2\pi i}\int_{C}g(w)(w-y)^{-2}\,\mathrm{d}w$ for
$y\in[-M,M]$. With these substitutions we have the left hand side of (15). ∎

##  6\. Proofs of Theorems 6, 10, and 11

The following notation will be used through the rest of this section. Assume
that a self-adjoint random matrix ensemble $(X_{N})_{N}$ is given. For two
Borel measurable functions $f,g:\mathbb{R}\to\mathbb{C}$, we define

$\displaystyle\varphi_{N}(f)$
$\displaystyle=\frac{1}{N}\mathbb{E}(\textnormal{Tr}(f(X_{N})))\quad\text{ and
}$ $\displaystyle\rho_{N}(f,g)$
$\displaystyle=\textnormal{Cov}(\textnormal{Tr}(f(X_{N})),\textnormal{Tr}(g(X_{N}))).$

Recall that $M$ is the constant from A1. For $f:\mathbb{R}\to\mathbb{C}$, we
define $f_{M}:\mathbb{R}\to\mathbb{C}$ by $f_{M}(x)=f(x)\mathbbm{1}_{|x|\leq
M}$. The proofs of our main results rely heavily on the following truncation
lemmas.

######  Lemma 18.

Let $(X_{N})_{N}$ be a self-adjoint random matrix ensemble satisfying A0.
Assume that $f,g:\mathbb{R}\to\mathbb{C}$ are Borel measurable functions.

  * $(a)$

If $f$ and $g$ are bounded, then, for all $N\in\mathbb{N}$,

(16) $\left|\rho_{N}(f,g)-\rho_{N}(f_{M},g_{M})\right|\leq
4\|f\|_{\infty}\|g\|_{\infty}N^{2}\mathbb{P}(\|X_{N}\|>M)^{1/4}.$

  * $(b)$

If $f$ and $g$ are polynomially bounded, then there exists $K_{f,g}>0$ such
that, for all $N\in\mathbb{N}$,

(17) $\left|\rho_{N}(f,g)-\rho_{N}(f_{M},g_{M})\right|\leq
K_{f,g}N^{2}\mathbb{P}(\|X_{N}\|>M)^{1/4}.$

###### Proof.

Let $\lambda_{1}\leq\cdots\leq\lambda_{N}$ be the eigenvalues of $X_{N}$.
Observe that, for any Borel measurable function $h:\mathbb{R}\to\mathbb{C}$,

$\textnormal{Tr}(h(X_{N}))=\textnormal{Tr}(h_{M}(X_{N}))+\sum_{i=1}^{N}h(\lambda_{i})\mathbbm{1}_{|\lambda_{i}|>M}.$

In particular, we have that
$\displaystyle\rho_{N}\left(f,g\right)-\rho_{N}(f_{M},g_{M})=\text{I}+\text{II}+\text{III}$,
where

I
$\displaystyle=\sum_{j=1}^{N}\textnormal{Cov}(\textnormal{Tr}(f_{M}(X_{N})),g(\lambda_{j})\mathbbm{1}_{|\lambda_{j}|>M}),$
II
$\displaystyle=\sum_{i=1}^{N}\textnormal{Cov}(f(\lambda_{i})\mathbbm{1}_{|\lambda_{i}|>M},\textnormal{Tr}(g_{M}(X_{N}))),$
III
$\displaystyle=\sum_{i,j=1}^{N}\textnormal{Cov}(f(\lambda_{i})\mathbbm{1}_{|\lambda_{i}|>M},g(\lambda_{j})\mathbbm{1}_{|\lambda_{j}|>M}).$

By the Cauchy-Schwarz inequality, we have that

$|\text{I}|\leq\textnormal{Var}(\textnormal{Tr}(f_{M}(X_{N})))^{1/2}\sum_{j=1}^{N}\textnormal{Var}(g(\lambda_{j})\mathbbm{1}_{|\lambda_{j}|>M})^{1/2}.$

Another application of the Cauchy-Schwarz inequality shows that, for any Borel
measurable function $h:\mathbb{R}\to\mathbb{C}$,

$|\textnormal{Tr}(h_{M}(X_{N}))|^{2}\leq
N\sum_{i=1}^{N}|h_{M}(\lambda_{i})|^{2}\leq N\textnormal{Tr}(|h(X_{N})|^{2}).$

Therefore,

$\textnormal{Var}(\textnormal{Tr}(f_{M}(X_{N}))\leq\mathbb{E}(|\textnormal{Tr}(f_{M}(X_{N}))|^{2})\leq
N^{2}\varphi_{N}(|f|^{2}),$

and hence

$|\text{I}|\leq
N\varphi_{N}(|f|^{2})^{1/2}\sum_{j=1}^{N}\textnormal{Var}(g(\lambda_{j})\mathbbm{1}_{|\lambda_{j}|>M})^{1/2}.$

Hölder’s inequality implies that, for all $j\in\\{1,\ldots,N\\}$,

$\displaystyle\textnormal{Var}(g(\lambda_{j})\mathbbm{1}_{|\lambda_{j}|>M})$
$\displaystyle\leq\mathbb{E}(|g(\lambda_{j})|^{2}\mathbbm{1}_{|\lambda_{j}|>M})$
$\displaystyle\mbox{}\leq\mathbb{E}(|g(\lambda_{j})|^{4})^{1/2}\mathbb{P}(|\lambda_{j}|>M)^{1/2}.$

Observe that $\mathbb{P}(|\lambda_{j}|>M)\leq\mathbb{P}(\|X_{N}\|>M)$. Thus,

$|\text{I}|\leq
N\,\varphi_{N}(|f|^{2})^{1/2}\mathbb{P}(\|X_{N}\|>M)^{1/4}\,\sum_{j=1}^{N}\mathbb{E}(|g(\lambda_{j})|^{4})^{1/4}.$

Another application of the generalized mean inequality shows that

$\sum_{j=1}^{N}\mathbb{E}(|g(\lambda_{j})|^{4})^{1/4}\leq
N\varphi_{N}(|g|^{4})^{1/4}.$

Therefore,

$|\text{I}|\leq\varphi_{N}(|f|^{2})^{1/2}\varphi_{N}(|g|^{4})^{1/4}N^{2}\mathbb{P}(\|X_{N}\|>M)^{1/4}.$

Mutatis mutandis, it is possible to show that

$\displaystyle|\text{II}|$
$\displaystyle\leq\varphi_{N}(|f|^{4})^{1/4}\,|\varphi_{N}(|g|^{2})^{1/2}\,N^{2}|\mathbb{P}(\|X_{N}\|>M)^{1/4},$
$\displaystyle\text{III}|$
$\displaystyle\leq|\varphi_{N}(|f|^{4})^{1/4}\,\varphi_{N}(|g|^{4})^{1/4}\,N^{2}|\mathbb{P}(\|X_{N}\|>M)^{1/2}.$

The three inequalities for $|$I$|$, $|$II$|$, and $|$III$|$ imply that

$\left|\rho_{N}(f,g)-\rho_{N}(f_{M},g_{M})\right|\leq
K_{f,g,N}\,N^{2}\mathbb{P}(\|X_{N}\|>M)^{1/4},$

where

$K_{f,g,N}=\left[\varphi_{N}(|f|^{2})^{1/2}+\varphi_{N}(|f|^{4})^{1/4}\right]\left[\varphi_{N}(|g|^{2})^{1/2}+\varphi_{N}(|g|^{4})^{1/4}\right].$

Part $(a)$ is an easy consequence of the fact that, for any Borel measurable
function $h:\mathbb{R}\to\mathbb{C}$ and $p>0$,

$\varphi_{N}(|h|^{p})^{1/p}=\left(\frac{1}{N}\sum_{i=1}^{N}\mathbb{E}(|h(\lambda_{i})|^{p})\right)^{1/p}\leq\|h\|_{\infty}.$

If $f$ and $g$ are polynomially bounded, then there exists a polynomial
$q:\mathbb{R}\to\mathbb{R}$ such that

$\max\\{|f(x)|^{2},|f(x)|^{4},|g(x)|^{2},|g(x)|^{4}\\}\leq q(x),$

for all $x\in\mathbb{R}$. In particular, for all $N\in\mathbb{N}$,

$K_{f,g,N}\leq\left[\varphi_{N}(q)^{1/2}+\varphi_{N}(q)^{1/4}\right]^{2}.$

By assumption A0, the limit $\displaystyle\lim_{N\to\infty}\varphi_{N}(q)$
exists. Taking

$K_{f,g}=\sup\left\\{\left[\varphi_{N}(q)^{1/2}+\varphi_{N}(q)^{1/4}\right]^{2}:N\in\mathbb{N}\right\\},$

Part $(b)$ now follows. ∎

We recall that $f|_{M}$ denotes the restriction of $f:\mathbb{R}\to\mathbb{C}$
to the interval $[-M,M]$. Note that $f|_{M}$ is a function on $[-M,M]$, while
$f_{M}$ is a function on $\mathbb{R}$. For a function
$f:\mathbb{R}\to\mathbb{C}$, we let

$\tilde{f}(x)=\begin{cases}f(x)&x\in[-M,M],\\\
f(M)+f^{\prime}(M)(x-M)e^{-\alpha(x-M)}&x>M,\\\
f(-M)+f^{\prime}(-M)(x+M)e^{\alpha(x+M)}&x<-M,\end{cases}$

where $\alpha=\|f^{\prime}\|_{M}/(e\|f\|_{M})$ with
$\|h\|_{M}=\sup\\{|h(x)|:x\in[-M,M]\\}$. By construction,
$\tilde{f}_{M}=(\tilde{f})_{M}=f_{M}$. It is not hard to verify that if
$f:\mathbb{R}\to\mathbb{C}$ is such that $f|_{M}\in C^{1}([-M,M])$, then

$\tilde{f}\in C^{1}(\mathbb{R}),\quad\|\tilde{f}\|_{\infty}\leq
2\|f\|_{M},\quad\text{ and
}\quad\|\tilde{f}^{\prime}\|_{\infty}=\|f^{\prime}\|_{M}.$

######  Lemma 19.

Let $(X_{N})_{N}$ be a self-adjoint random matrix ensemble satisfying A$0$ and
A$2$, and let $K$ be the constant in A$2$. Assume that
$f,g:\mathbb{R}\to\mathbb{C}$ are Borel measurable functions such that
$f|_{M},g|_{M}\in C^{1}([-M,M])$.

  * $(a)$

If $f$ and $g$ are bounded, then, for all $N\in\mathbb{N}$,

$|\rho_{N}(f,g)|\leq
20\|f\|_{\infty}\|g\|_{\infty}N^{2}\mathbb{P}(\|X_{N}\|>M)^{1/4}+K\|f^{\prime}\|_{M}\|g^{\prime}\|_{M}.$

  * $(b)$

If $f$ and $g$ are polynomially bounded, then there exists $K_{f,g}>0$ such
that, for all $N\in\mathbb{N}$,

$|\rho_{N}(f,g)|\leq
K_{f,g}N^{2}\mathbb{P}(\|X_{N}\|>M)^{1/4}+K\|f^{\prime}\|_{M}\|g^{\prime}\|_{M}.$

###### Proof.

Recall that, by construction $\tilde{h}_{M}=h_{M}$. In particular,

(18) $\displaystyle|\rho_{N}(f,g)|\leq|\rho_{N}(f,g)-\rho_{N}(f_{M},g_{M})|$
(19)
$\displaystyle\mbox{}+|\rho_{N}(\tilde{f}_{M},\tilde{g}_{M})-\rho_{N}(\tilde{f},\tilde{g})|+|\rho_{N}(\tilde{f},\tilde{g})|.$

If $f$ and $g$ are bounded, Part $(a)$ of Lemma 18 implies that

$\displaystyle|\rho_{N}(f,g)|$ $\displaystyle\leq
4(\|f\|_{\infty}\|g\|_{\infty}+\|\tilde{f}\|_{\infty}\|\tilde{g}\|_{\infty})N^{2}\mathbb{P}(\|X_{N}\|>M)^{1/4}$
$\displaystyle\qquad\mbox{}+|\rho_{N}(\tilde{f},\tilde{g})|.$

Since $\|\tilde{f}\|_{\infty}\leq 2\|f\|_{M}\leq 2\|f\|_{\infty}$, we conclude
that

$|\rho_{N}(f,g)|\leq
20\|f\|_{\infty}\|g\|_{\infty}N^{2}\mathbb{P}(\|X_{N}\|>M)^{1/4}+|\rho_{N}(\tilde{f},\tilde{g})|.$

By A2, we have that $|\rho_{N}(\tilde{f},\tilde{g})|\leq
K\|\tilde{f}^{\prime}\|_{\infty}\|\tilde{g}^{\prime}\|_{\infty}=K\|f^{\prime}\|_{M}\|g^{\prime}\|_{M}$.
Part $(a)$ follows.

If $f$ and $g$ are polynomially bounded, (18) and Part $(b)$ of Lemma 18 imply
that

$|\rho_{N}(f,g)|\leq(K_{f,g}^{\prime}+4\|\tilde{f}\|_{\infty}\|\tilde{g}\|_{\infty})N^{2}\mathbb{P}(\|X_{N}\|>M)^{1/4}+|\rho_{N}(\tilde{f},\tilde{g})|,$

for some $K_{f,g}^{\prime}>0$. Let
$K_{f,g}=K_{f,g}^{\prime}+16\|f\|_{M}\|g\|_{M}$. In particular,

$\displaystyle|\rho_{N}(f,g)|$ $\displaystyle\leq
K_{f,g}N^{2}\mathbb{P}(\|X_{N}\|>M)^{1/4}+|\rho_{N}(\tilde{f},\tilde{g})|$
$\displaystyle\leq
K_{f,g}N^{2}\mathbb{P}(\|X_{N}\|>M)^{1/4}+K\|f^{\prime}\|_{M}\|g^{\prime}\|_{M},$

where the last inequality follows from A2 and the fact that
$\|\tilde{f}^{\prime}\|_{\infty}=\|f^{\prime}\|_{M}$. ∎

###### Proof of Theorem 6.

We start by constructing the bilinear mapping $\rho$. Let
$p,q\in\mathbb{C}[x]$. We define

$\rho(p|_{M},q|_{M})=\lim_{N\to\infty}\rho_{N}(p,q).$

Note that, by A0, the limit in the previous equation exists. By Part $(b)$ of
Lemma 19, we have that

$|\rho_{N}(p,q)|\leq
K_{p,q}N^{2}\mathbb{P}(\|X_{N}\|>M)^{1/4}+K\|p^{\prime}\|_{M}\|q^{\prime}\|_{M},$

for some $K_{p,q}>0$. Taking limits, A1 implies then

$|\rho(p|_{M},q|_{M})|\leq K\|p^{\prime}\|_{M}\|q^{\prime}\|_{M}.$

In particular, we have

$|\rho(p|_{M},q|_{M})|\leq K\|p\|\|q\|,$

where $\lVert p\rVert=\lvert p(0)\rvert+\lVert p^{\prime}\rVert_{M}$ as
introduced in Notation 1. Since the polynomials without constant term are
dense in $C^{1}([-M,M])^{\circ}$ with respect to the norm $\|\cdot\|$, it is a
standard procedure to extend $\rho$ continuously (in each argument) to
$C^{1}([-M,M])^{\circ}$ with the same bound. We extend $\rho$ to
$\mathbb{C}\oplus C^{1}([-M,M])^{\circ}$ by making $\rho$ vanish on
$\mathbb{C}\oplus 0$; again without increasing the norm. Let
$\rho:C^{1}([-M,M])\times C^{1}([-M,M])\to\mathbb{C}$ denote this extension.
This proves $(a)$. Since we already had $\rho(p,1)=\rho(1,q)=0$ for
polynomials $p$ and $q$, this extension is consistent with the definition of
$\rho$.

We prove $(b)$ next. Let $f:\mathbb{R}\to\mathbb{C}$ be a polynomially bounded
function with $f|_{M}\in C^{1}([-M,M])$ and $q\in\mathbb{C}[x]$. For any
polynomial $p\in\mathbb{C}[x]$, we set
$\rho_{N}(f,q)-\rho(f|_{M},q|_{M})=\text{I}+\text{II}+\text{III}$, where

I $\displaystyle=\rho_{N}(f,q)-\rho_{N}(p,q)\text{,
}\text{II}=\rho_{N}(p,q)-\rho(p|_{M},q|_{M})\text{, }$
$\displaystyle\qquad\qquad\text{III}=\rho(p|_{M},q|_{M})-\rho(f|_{M},q|_{M}).$

By Part $(b)$ of Lemma 19, we have that

$\displaystyle|\text{I}|\leq
K_{f-p,q}N^{2}\mathbb{P}(\|X_{N}\|>M)^{1/4}+K\|(f-p)^{\prime}\|_{M}\|q^{\prime}\|_{M},$

for some $K_{f-p,q}>0$. Let $\epsilon>0$. By the density of the polynomials in
$C^{1}([-M,M])$ with respect to the $C^{1}$-norm and the continuity of $\rho$
with respect to the same norm, there exists $p_{0}\in\mathbb{C}[x]$ such that

$\|(f-p_{0})^{\prime}\|_{M}<\frac{\epsilon}{6K\|q^{\prime}\|_{M}}\quad\text{
and }|\text{III}|<\frac{\epsilon}{3}.$

By construction, $\rho_{N}(p_{0},q)\to\rho((p_{0})|_{M},q|_{M})$ as
$N\to\infty$. Combined with A1, this implies that there exists
$N_{0}\in\mathbb{N}$ such that for all $N\geq N_{0}$

$|\text{II}|<\frac{\epsilon}{3}\quad\text{ and
}K_{f-p_{0},q}N^{2}\mathbb{P}(\|X_{N}\|>M)^{1/4}<\frac{\epsilon}{6}.$

Therefore, for all $N>N_{0}$, $|\rho_{N}(f,q)-\rho(f|_{M},q|_{M})|<\epsilon$,
i.e.,

$\lim_{N\to\infty}\rho_{N}(f,q)=\rho(f|_{M},q|_{M}).$

The previous equation and a similar argument show that

$\lim_{N\to\infty}\rho_{N}(f,g)=\rho(f|_{M},g|_{M})$

for all polynomially bounded functions $f,g:\mathbb{R}\to\mathbb{C}$ with
$f|_{M},g|_{M}\in C^{1}([-M,M])$. This proves Part $(b)$.

In order to prove Part $(c)$, let $\Phi:C([-M,M])^{2}\to\mathbb{C}$ be given
by $\Phi(f,g)=\rho(F,G)$, where

$F(x)=\int_{0}^{x}f(t)\mathrm{d}t\quad\text{ and }\quad
G(x)=\int_{0}^{x}g(t)\mathrm{d}t.$

Note that $\Phi$ is bilinear and, by Part $(a)$,

$|\Phi(f,g)|=|\rho(F,G)|\leq K\|f\|_{M}\|g\|_{M}.$

By Theorem 5, there exists $u:[-M,M]^{2}\to\mathbb{R}$ of bounded Fréchet
variation such that, for all $f,g\in C([-M,M])$,

(20) $\Phi(f,g)=\int_{-M}^{M}\int_{-M}^{M}f(x)g(x)\mathrm{d}u(x,y).$

Then
$\rho(f,g)=\Phi(f^{\prime},g^{\prime})=\displaystyle\int_{-M}^{M}\int_{-M}^{M}f^{\prime}(x)g^{\prime}(y)\,\mathrm{d}u(x,y)$,
as claimed. ∎

For $z\in\mathbb{C}$, we let $r_{z}:\mathbb{C}\setminus\\{z\\}\to\mathbb{C}$
be given by $\displaystyle r_{z}(x)=\frac{1}{z-x}$ and $r_{z,M}=(r_{z})_{M}$.
Note that, for all $z\in\mathbb{C}\setminus[-M,M]$, the function
$r_{z,M}:\mathbb{R}\to\mathbb{C}$ satisfies

$\|r_{z,M}\|_{\infty}=d(z)^{-1}\quad\text{ and
}\quad\|r_{z,M}^{\prime}\|_{M}=d(z)^{-2},$

where $d(z):=\inf\\{|z-x|:x\in[-M,M]\\}$. We define
$G_{2,M}^{(N)}:(\mathbb{C}\setminus[-M,M])^{2}\to\mathbb{C}$ by

(21) $G_{2,M}^{(N)}(z,w)=\rho_{N}(r_{z,M},r_{w,M}).$

It is not hard to verify that $G_{2,M}^{(N)}$ is analytic. Indeed,

$\displaystyle\rho_{N}(r_{z,M},r_{w,M})$
$\displaystyle=\sum_{i,j,=1}^{N}\mathrm{E}(\mathbbm{1}_{|\lambda_{i}|\leq
M}(z-\lambda_{i})^{-1}\mathbbm{1}_{|\lambda_{j}|\leq M}(w-\lambda_{j})^{-1})$
$\displaystyle\qquad\quad\mbox{}-\mathrm{E}(\mathbbm{1}_{|\lambda_{i}|\leq
M}(z-\lambda_{i})^{-1})\mathrm{E}(\mathbbm{1}_{|\lambda_{j}|\leq
M}(w-\lambda_{j})^{-1}).$

Observe that the function $z\mapsto\mathrm{E}(\mathbbm{1}_{|\lambda_{i}|\leq
M}(z-\lambda_{i})^{-1})$ is the Cauchy transform of the random variable
$\lambda_{i}$ conditioned on $|\lambda_{i}|\leq M$ times the probability of
this event, and is analytic on $\mathbb{C}\setminus[-M,M]$, c.f. [28, Lemma
3.2]. If $|z^{\prime}-z|\leq\frac{1}{2}d(z)$, then for
$w\in\mathbb{C}\setminus[-M,M]$ we have

$\displaystyle\mathrm{E}(\mathbbm{1}_{|\lambda_{i}|\leq
M}(z-\lambda_{i})^{-1}\mathbbm{1}_{|\lambda_{j}|\leq M}(w-\lambda_{j})^{-1})$
$\displaystyle=-\sum_{n=0}^{\infty}\mathrm{E}\bigg{(}\mathbbm{1}_{|\lambda_{i}|\leq
M}\mathbbm{1}_{|\lambda_{j}|\leq
M}\frac{(w-\lambda_{j})^{-1}}{(\lambda_{i}-z)^{n+1}}\bigg{)}(z^{\prime}-z)^{n}$

and the convergence is uniform on
$\\{z^{\prime}\in\mathbb{C}\mid|z^{\prime}-z|<\frac{1}{2}d(z)\\}$, c.f. [28,
the proof of Lemma 3.2]. Thus $G_{2,M}^{(N)}$ is analytic on
$(\mathbb{C}\setminus[-M,M])^{2}$. Moreover, it satisfies the following
boundedness property.

######  Lemma 20.

If $(X_{N})_{N}$ is a self-adjoint random matrix ensemble satisfying $A0$,
$A1$, and $A2$, then for every $z_{0},w_{0}\in\mathbb{C}\setminus[-M,M]$ there
exists $\delta>0$ such that

$\sup\left\\{\big{|}G_{2,M}^{(N)}(z,w)\big{|}:|z-z_{0}|<\delta,|w-w_{0}|<\delta,N\in\mathbb{N}\right\\}<\infty.$

###### Proof.

By Part $(a)$ of Lemma 19, for all $z,w\in\mathbb{C}\setminus[-M,M]$,

$\displaystyle|G_{2,M}^{(N)}(z,w)|$ $\displaystyle\leq
20\|r_{z,M}\|_{\infty}\|r_{w,M}\|_{\infty}N^{2}\mathbb{P}(\|X_{N}\|>M)^{1/4}$
$\displaystyle\qquad\mbox{}+K\|r_{z,M}^{\prime}\|_{M}\|r_{w,M}^{\prime}\|_{M}$
$\displaystyle=20d(z)^{-1}d(w)^{-1}N^{2}\mathbb{P}(\|X_{N}\|>M)^{1/4}+Kd(z)^{-2}d(w)^{-2}.$

Let $\delta=\frac{1}{2}\min(d(z_{0}),d(w_{0}))$. Note that for all
$z,w\in\mathbb{C}$ such that $|z-z_{0}|<\delta$ and $|w-w_{0}|<\delta$, we
have that $d(z)>\delta$ and $d(w)>\delta$. Assumption $A1$ implies that
$\\{N^{2}\mathbb{P}(\|X_{N}\|>M)^{1/4}:N\in\mathbb{N}\\}$ is bounded. The
lemma now follows. ∎

######  Lemma 21.

If $(X_{N})_{N}$ is a self-adjoint random matrix ensemble satisfying $A0$,
$A1$, and $A2$, then the family
$\left\\{G_{2,M}^{(N)}:N\in\mathbb{N}\right\\}$ converges uniformly in compact
subsets of $(\mathbb{C}\setminus[-M,M])^{2}$ and, for all $|z|,|w|>M$,

$\lim_{N\to\infty}G_{2,M}^{(N)}(z,w)=G_{2}(z,w).$

###### Proof.

By definition,

$G_{2,M}^{(N)}(z,w)=\textnormal{Cov}\left(\sum_{i=1}^{N}\frac{\mathbbm{1}_{|\lambda_{i}|\leq
M}}{z-\lambda_{i}},\sum_{j=1}^{N}\frac{\mathbbm{1}_{|\lambda_{j}|\leq
M}}{w-\lambda_{j}}\right).$

If $|z|,|w|>M$, then we have that

$(z-\lambda_{i})^{-1}=\sum_{m\geq 0}\frac{\lambda_{i}^{m}}{z^{m+1}}\quad\text{
and }\quad(w-\lambda_{j})^{-1}=\sum_{n\geq 0}\frac{\lambda_{j}^{n}}{w^{n+1}}.$

In particular, for such $z$ and $w$, we have that

$G_{2,M}^{(N)}(z,w)=\textnormal{Cov}\left(\sum_{i=1}^{N}\sum_{m\geq
0}\frac{\lambda_{i}^{m}\mathbbm{1}_{|\lambda_{i}|\leq
M}}{z^{m+1}},\sum_{j=1}^{N}\sum_{n\geq
0}\frac{\lambda_{j}^{n}\mathbbm{1}_{|\lambda_{j}|\leq M}}{w^{n+1}}\right).$

For each $k\in\mathbb{N}$, let $\pi^{(k)}:\mathbb{R}\to\mathbb{R}$ denote the
function given by $\pi^{(k)}(x)=x^{k}$. With this notation, we can rewrite the
previous equation as

$G_{2,M}^{(N)}(z,w)=\textnormal{Cov}\left(\sum_{m\geq
0}\frac{\textnormal{Tr}(\pi^{(m)}_{M}(X_{N}))}{z^{m+1}},\sum_{n\geq
0}\frac{\textnormal{Tr}(\pi^{(n)}_{M}(X_{N}))}{w^{n+1}}\right).$

Since $|\textnormal{Tr}(\pi^{(k)}_{M}(X_{N}))|\leq NM^{k}$, a routine
application of Tonelli-Fubini theorem implies that

$G_{2,M}^{(N)}(z,w)=\sum_{m,n\geq
0}\frac{\rho_{N}(\pi^{(m)}_{M},\pi^{(n)}_{M})}{z^{m+1}w^{n+1}}.$

Note that $\pi^{(k)}_{M}$ satisfies that
$\displaystyle\|\pi^{(k)}_{M}\|_{\infty}=M^{k}$ and
$\displaystyle\|(\pi^{(k)}_{M})^{\prime}\|_{M}=kM^{k-1}$. By Part $(a)$ of
Lemma 19, we obtain

$|\rho_{N}(\pi^{(m)}_{M},\pi^{(n)}_{M})|\leq
20M^{m+n}N^{2}\mathbb{P}(\|X_{N}\|>M)^{1/4}+mnKM^{m+n-2}.$

By A1, the set
$\left\\{N^{2}\mathbb{P}(\|X_{N}\|>M)^{1/4}:N\in\mathbb{N}\right\\}$ is
bounded. Hence,

$|\rho_{N}(\pi^{(m)}_{M},\pi^{(n)}_{M})|\leq mnBM^{m+n},$

for some constant $B>0$ independent of $m$ and $n$. The previous inequality
and the dominated convergence theorem imply that

$\lim_{N\to\infty}G_{2,M}^{(N)}(z,w)=\sum_{m,n\geq
0}\lim_{N\to\infty}\frac{\rho_{N}(\pi^{(m)}_{M},\pi^{(n)}_{M})}{z^{m+1}w^{n+1}},$

for all $|z|,|w|>M$. Part $(a)$ of Lemma 18 implies that

$\lim_{N\to\infty}\rho_{N}(\pi^{(m)}_{M},\pi^{(n)}_{M})=\rho(\pi^{(m)}|_{M},\pi^{(n)}|_{M})=\alpha_{m,n}.$

In other words, for all $|z|,|w|>M$,

(22) $\lim_{N\to\infty}G_{2,M}^{(N)}(z,w)=G_{2}(z,w).$

By the previous lemma, the family $\left\\{G_{2,M}^{(N)}\right\\}_{N}$ is
locally bounded. Therefore, Montel’s theorem [32, pp. 33] and (22) imply that
$\left\\{G_{2,M}^{(N)}\right\\}_{N}$ converges uniformly on compact sets to an
analytic function on $(\mathbb{C}\setminus[-M,M])^{2}$. ∎

For notational simplicity, for each $z,w\in\mathbb{C}\setminus\mathbb{R}$, we
let

$G_{2}^{(N)}(z,w)=\rho_{N}(r_{z},r_{w})=\textnormal{Cov}(\textnormal{Tr}((z-X_{N})^{-1}),\textnormal{Tr}((w-X_{N})^{-1})).$

###### Proof of Theorem 10.

The previous lemma states that $G_{2}(z,w)=\lim_{N}G_{2,M}^{(N)}(z,w)$ for all
$|z|,|w|>M$. Also, it establishes that $\left\\{G_{2,M}^{(N)}\right\\}_{N}$
converges uniformly in compact subsets of $(\mathbb{C}\setminus[-M,M])^{2}$ to
an analytic function, which extends $G_{2}$ to the domain
$\\{(z,w)\mid|z|,|w|>M\\}$; we also call this extension $G_{2}$. By Part $(a)$
of Lemma 18, we have that for $z,w\not\in\mathbb{R}$,

$\left|G_{2}^{(N)}(z,w)-G_{2,M}^{(N)}(z,w)\right|\leq
4\|r_{z}\|_{\infty}\|r_{w}\|_{\infty}N^{2}\mathbb{P}(\|X_{N}\|>M)^{1/4}.$

Since $\|r_{z}\|_{\infty}\leq|\Im z|^{-1}$ for all
$z\in\mathbb{C}\setminus\mathbb{R}$, $A1$ implies that

$\lim_{N\to\infty}G_{2}^{(N)}(z,w)=\lim_{N\to\infty}G_{2,M}^{(N)}(z,w)=G_{2}(z,w),$

as required. ∎

###### Proof of Theorem 11.

Whenever $|\lambda_{i}|\leq M$, Cauchy integral formula implies that

$f(\lambda_{i})=\frac{1}{2\pi
i}\int_{\mathcal{C}}\frac{f(z)}{z-\lambda_{i}}\mathrm{d}z.$

In particular, we have that

$\rho_{N}(f_{M},g_{M})=\kern-3.0pt\sum_{i,j=1}^{N}\textnormal{Cov}\Big{(}\frac{1}{2\pi
i}\int_{\mathcal{C}}\frac{f(z)\mathbbm{1}_{|\lambda_{i}|\leq
M}}{z-\lambda_{i}}\mathrm{d}z,\frac{1}{2\pi
i}\int_{\mathcal{C}}\frac{g(w)\mathbbm{1}_{|\lambda_{j}|\leq
M}}{w-\lambda_{j}}\mathrm{d}w\Big{)}.$

A routine application of the Tonelli-Fubini theorem shows then that

$\displaystyle\rho_{N}(f_{M},g_{M})$
$\displaystyle=\sum_{i,j=1}^{N}\frac{1}{(2\pi
i)^{2}}\int_{\mathcal{C}}\int_{\mathcal{C}}f(z)g(w)\textnormal{Cov}\Big{(}\frac{\mathbbm{1}_{|\lambda_{i}|\leq
M}}{z-\lambda_{i}},\frac{\mathbbm{1}_{|\lambda_{j}|\leq
M}}{w-\lambda_{j}}\Big{)}\mathrm{d}z\mathrm{d}w$ $\displaystyle=\frac{1}{(2\pi
i)^{2}}\int_{\mathcal{C}}\int_{\mathcal{C}}f(z)g(w)G_{2,M}^{(N)}(z,w)\mathrm{d}z\mathrm{d}w.$

By Lemma 21, $\left\\{G_{2,M}^{(N)}\right\\}_{N}$ converges uniformly to (the
extension of) $G_{2}$ on the compact set $\mathcal{C}\times\mathcal{C}$.
Therefore, by the dominated convergence theorem,

$\lim_{N\to\infty}\rho_{N}(f_{M},g_{M})=\frac{1}{(2\pi
i)^{2}}\int_{\mathcal{C}}\int_{\mathcal{C}}f(z)g(w)G_{2}(z,w)\mathrm{d}z\mathrm{d}w.$

By Part $(b)$ of Theorem 6, we conclude that

$\lim_{N\to\infty}\rho_{N}(f,g)=\rho(f|_{M},g|_{M})=\lim_{N\to\infty}\rho_{N}(f_{M},g_{M}).$

Equation (6) follows. ∎

##  7\. Summary and Concluding Remarks

Under general assumptions, we established the existence and boundedness of the
asymptotic covariance mapping $\rho$ (Theorem 6). Also, we showed that the
second-order Cauchy transform $G_{2}$ admits an analytic extension which is
equal to the limit of the covariance of resolvents (Theorem 10). Furthermore,
we showed that the asymptotic covariance mapping $\rho$ could be recovered
from the second-order Cauchy transform (Theorem 11). For random matrix
ensembles having a second-order limit distribution, we showed that the
fluctuations of the linear statistics of a real smooth (test) function $f$ are
asymptotically Gaussian with mean zero and variance $\rho(f,f)$ (Proposition
12). In addition we proved that if $(X_{N})_{N}$ is a self-adjoint random
matrix ensemble satisfying A0, A1, and A2, then there exists
$u:\mathbb{R}^{2}\to\mathbb{R}$ of bounded Fréchet variation such that
$\textnormal{Supp}(u)$ is compact and

$G_{2}(z,w)=\int_{\mathbb{R}^{2}}\frac{1}{(z-x)^{2}}\frac{1}{(w-y)^{2}}\mathrm{d}u(x,y),$

where the integral above is in the sense of Fréchet [16, 18]. In this case, we
say that $u$ is the second-order analytic distribution of $(X_{N})_{N}$.

Let $\mathcal{F}$ be the set of functions on $\mathbb{R}^{2}$ of bounded
Fréchet variation which are the second-order analytic distribution of a random
matrix ensemble satisfying A0, A1, and A2. In this setting, the
characterization of $\mathcal{F}$ is of interest.

Q1. Given $u:\mathbb{R}^{2}\to\mathbb{R}$ of bounded Fréchet variation, does
it belong to $\mathcal{F}$?

Assume that $u_{1},u_{2}\in\mathcal{F}$ with associated random matrix
ensembles $(X_{N})_{N}$ and $(Y_{N})_{N}$, respectively. For each
$N\in\mathbb{N}$, let $U_{N}$ be an $N\times N$ Haar unitary matrix
independent of $X_{N}$ and $Y_{N}$.

Q2. The random matrix ensemble $(X_{N}+U_{N}Y_{N}U_{N}^{*})_{N}$ satisfies A0
and A1, does it satisfy also A2?

If the previous question has an affirmative answer, we can define the second-
order free additive convolution of $u_{1}$ and $u_{2}$ as the second-order
analytic distribution of $X_{N}+U_{N}Y_{N}U_{N}^{*}$. This would define a
binary operation on $\mathcal{F}$:

$\boxplus_{2}:\mathcal{F}\times\mathcal{F}\to\mathcal{F},$

similarly as the $\boxplus$-operation does in the first-order setting.

In bi-free probability theory there are Cauchy transforms of the form

$G(z,w)=\int_{\mathbb{R}^{2}}\frac{1}{z-x}\frac{1}{w-y}\mathrm{d}u(x,y),$

for some compactly supported measure $u$. Note that

(23) $\frac{\partial^{2}}{\partial z\partial
w}G(z,w)=\int_{\mathbb{R}^{2}}\frac{1}{(z-x)^{2}}\frac{1}{(w-y)^{2}}\mathrm{d}u(x,y)=G_{2}(z,w).$

In view of the expression (see [28, Eq. (5.20)]), where $F(z)=1/G(z)$,

$G_{2}(z,w)=G^{\prime}(z)G^{\prime}(w)R(G(z),G(w))+\frac{\partial^{2}}{\partial
z\partial w}\log\frac{F(z)-F(w)}{z-w},$

equation (23) suggests a potential connection between second-order free
probability theory and bi-free probability theory.

Q3. If we assume that $R(z,w)\equiv 0$, is it possible to provide new examples
in bi-free probability theory using second-order free probability ones?

In particular, the previous question provides some motivation to study the
case $R(z,w)\equiv 0$, which indeed has attracted some attention in the past.

## Acknowledgements

MD would like to thank Arturo Jaramillo and Roland Speicher for fruitful
discussions while preparing this paper. Also, MD would like to thank Malors
Espinosa for his keen comments concerning Lemma 22.

## References

  * [1] Greg Anderson.  Convergence of the largest singular value of a polynomial in independent Wigner matrices.  The Annals of Probability, pages 2103–2181, 2013.
  * [2] Greg Anderson and Brendan Farrell.  Asymptotically liberating sequences of random unitary matrices.  Advances in Mathematics, 255:381–413, 2014.
  * [3] Greg Anderson and Ofer Zeitouni.  A CLT for a band matrix model.  Probability Theory and Related Fields, 134:283–338, 2006.
  * [4] Zhidong Bai and Jack Silverstein.  CLT for linear spectral statistics of large-dimensional sample covariance matrices.  Annals of Probability, 32:533–605, 2004.
  * [5] Zhigang Bao, László Erdős, and Kevin Schnelli.  Local law of addition of random matrices on optimal scale.  Comm. Math. Phys., 349(3):947–990, 2017.
  * [6] Serban Belinschi, Hari Bercovici, Mireille Capitaine, and Maxime Fevrier.  Outliers in the spectrum of large deformed unitarily invariant models.  Annals of Probability, 45(6A):3571–3625, 2017.
  * [7] Serban Belinschi, Tobias Mai, and Roland Speicher.  Analytic subordination theory of operator-valued free additive convolution and the solution of a general random matrix problem.  Journal für die Reine und Angewandte Mathematik, 732:21–53, 2017.
  * [8] Gérard Ben Arous and Alice Guionnet.  Large deviations for Wigner’s law and Voiculescu’s non-commutative entropy.  Probab. Theory Related Fields, 108(4):517–542, 1997\.
  * [9] Mireille Capitaine and Catherine Donati-Martin.  Spectrum of deformed random matrices and free probability.  ArXiv e-prints, 2016.
  * [10] Benoît Collins and Camille Male.  The strong asymptotic freeness of Haar and deterministic matrices.  Ann. Sci. Éc. Norm. Supér. (4), 47(1):147–163, 2014\.
  * [11] Benoît Collins, James A. Mingo, Piotr Śniady, and Roland Speicher.  Second order freeness and fluctuations of random matrices. III. Higher order freeness and free cumulants.  Documenta Mathematica, 12:1–70 (electronic), 2007.
  * [12] Persi Diaconis and Mehrdad Shahshahani.  On the eigenvalues of random matrices.  Journal of Applied Probability, 31:49–62, 1994.
  * [13] Mario Diaz, Arturo Jaramillo, and Juan Carlos Pardo.  Fluctuations for matrix-valued Gaussian processes.  Ann. Inst. H. Poincaré Probab. Statist., accepted for publication.
  * [14] Mario Diaz, James A Mingo, and Serban T Belinschi.  On the global fluctuations of block Gaussian matrices.  Probability Theory and Related Fields, 176(1):599–648, 2020.
  * [15] Ioanna Dumitriu and Alan Edelman.  Global spectrum fluctuations for the $\beta$-Hermite and $\beta$-Laguerre ensembles via matrix models.  J. Math. Phy., 47(063302), 2006\.
  * [16] Maurice Fréchet.  Sur les fonctionnelles bilinéaires.  Trans. Amer. Math. Soc., 16(3):215–234, 1915.
  * [17] Uffe Haagerup and Steen Thorbjornsen.  A new application of random matrices: $\text{Ext}(\text{C}^{\ast}_{\text{red}}(f_{2}))$ is not a group.  Annals of Mathematics, 162, pages 711–775, 2005.
  * [18] Theophil Hildebrandt.  Introduction to the theory of integration.  Pure and Applied Mathematics, Vol. XIII. Academic Press, New York-London, 1963.
  * [19] Kurt Johansson.  On fluctuations of eigenvalues of random Hermitian matrices.  Duke Mathematical Journal, 91(1):151–204, 1998.
  * [20] Michel Ledoux and Brian Rider.  Small deviations for beta ensembles.  Electron. J. Probab., 15(41):1319–1343, 2010\.
  * [21] Mylène Maïda.  Large deviations for the largest eigenvalue of rank one deformations of Gaussian ensembles.  Electron. J. Probab., 12:1131–1150, 2007.
  * [22] Camille Male.  The norm of polynomials in large random and deterministic matrices.  Probab. Theory Related Fields, 154(3-4):477–532, 2012.  With an appendix by Dimitri Shlyakhtenko.
  * [23] Vladimir Marchenko and Leonid Pastur.  Distribution of eigenvalues for some sets of random matrices.  Mathematics of the USSR-Sbornik, 1(4):457–483, 1967.
  * [24] James A. Mingo and Alexandru Nica.  Annular non-crossing permutations and partitions, and second-order asymptotics for random matrices.  Int. Math. Res. Not., 28:1413–1460, 2004.
  * [25] James A. Mingo and Mihai Popa.  Freeness and the transposes of unitarily invariant random matrices.  J. Funct. Anal., 271(4):883–921, 2016.
  * [26] James A. Mingo, Piotr Śniady, and Roland Speicher.  Second order freeness and fluctuations of random matrices. II. Unitary random matrices.  Advances in Mathematics, 209(1):212–240, 2007.
  * [27] James A. Mingo and Roland Speicher.  Second order freeness and fluctuations of random matrices. I. Gaussian and Wishart matrices and cyclic Fock spaces.  Journal of Functional Analysis, 235(1):226–270, 2006\.
  * [28] James A. Mingo and Roland Speicher.  Free Probability and Random Matrices, volume 35 of Fields Institute Monographs.  Springer-Verlag New York, 2017.
  * [29] Marston Morse and William Transue.  Functionals of bounded Fréchet variation.  Canad. J. Math., 1:153–165, 1949\.
  * [30] Marston Morse and William Transue.  Integral representations of bilinear functionals.  Proceedings of the National Academy of Science U.S.A., 35:136–143, 1949.
  * [31] Leonid Pastur and Mariya Shcherbina.  Eigenvalue Distribution of Large Random Matrices, volume 171 of Mathematical Surveys and Monographs.  American Mathematical Society, Providence, RI, 2011.
  * [32] Volker Scheidemann.  Introduction to complex analysis in several variables.  Birkhäuser Verlag, Basel, 2005.
  * [33] Dimitri Shlyakhtenko.  Free probability of type B and asymptotics of finite-rank perturbations of random matrices.  Indiana University Mathematics Journal, 67:971–991, 2018\.
  * [34] Dan Voiculescu.  Limit laws for random matrices and free products.  Inventiones Mathematicae, 104(1):201–220, 1991.
  * [35] Dan Voiculescu.  A strengthened asymptotic freeness result for random matrices with applications to free entropy.  International Mathematics Research Notices, 1998(1):41–63, 1998.
  * [36] Eugene Wigner.  Characteristic vectors of bordered matrices with infinite dimensions.  Annals of Mathematics. Second Series, 62:548–564, 1955.
  * [37] Eugene Wigner.  Characteristic vectors of bordered matrices with infinite dimensions. II.  Annals of Mathematics. Second Series, 65:203–207, 1957.
  * [38] Eugene Wigner.  On the distribution of the roots of certain symmetric matrices.  Annals of Mathematics. Second Series, 67:325–327, 1958.

##  Appendix A Proof of Proposition 12

The following lemma is an easy consequence of Runge’s theorem and the Schwarz
Reflection Principle. For a set $S\subset\mathbb{C}$, we let
$S^{*}=\\{z\in\mathbb{C}:\overline{z}\in S\\}$.

######  Lemma 22.

Let $\Omega\subset\mathbb{C}$ be a domain. Assume that
$\mathcal{K}=\mathcal{K}^{*}$ is a compact subset of $\Omega$ whose complement
is connected. If $f:\Omega\to\mathbb{C}$ is an analytic function such that
$f(\mathbb{R})\subset\mathbb{R}$, then there exist polynomials
$(p_{k})_{k\in\mathbb{N}}\subset\mathbb{R}[z]$ such that

$\lim_{k\to\infty}\|p_{k}-f\|_{\mathcal{K}}=0\quad\text{ and
}\quad\lim_{k\to\infty}\|(p_{k}-f)^{\prime}\|_{\mathcal{K}}=0,$

where $\|g\|_{\mathcal{K}}=\sup\\{|g(z)|:z\in\mathcal{K}\\}$.

###### Proof of Proposition 12.

Let real valued $f\in\mathcal{S}$ be given. Let $\mathcal{K}\subset\mathbb{C}$
be a compact set that contains $[-M,M]$. Let $\Omega\subset\mathbb{C}$ be an
open disc containing $\mathcal{K}$. By Lemma 22 there exist real polynomials
$(p_{k})_{k}$ such that

$\lim_{k\to\infty}\|p_{k}-f\|_{\mathcal{K}}=0\quad\text{ and
}\quad\lim_{k\to\infty}\|(p_{k}-f)^{\prime}\|_{\mathcal{K}}=0.$

For notational simplicity, for all $k\in\mathbb{N}$ and $N\in\mathbb{N}$, we
let

$Z_{N}^{(k)}=\textnormal{Tr}(p_{k}(X_{N}))-\mathbb{E}(\textnormal{Tr}(p_{k}(X_{N}))).$

Clearly $k_{1}\left(Z_{N}^{(k)}\right)=0$. From Part ii) in the definition of
second-order limit distribution, it is immediate to see that, for all $r\geq
3$,

$\lim_{N\to\infty}k_{r}\left(Z_{N}^{(k)},\ldots,Z_{N}^{(k)}\right)=0.$

Since the Gaussian distribution on $\mathbb{R}$ is characterized by its
moments, and hence by its cumulants, we conclude from Part (b) of Theorem 6
that $\\{\displaystyle Z_{N}^{(k)}\\}_{N}$ converges in distribution (i.e.,
$Z_{N}^{(k)}\Rightarrow Z^{(k)}$) to
$Z^{(k)}\sim{\mathcal{N}}_{\mathbb{R}}(0,\sigma_{k}^{2})$, where
$\sigma_{k}^{2}=\rho(p_{k}|_{M},p_{k}|_{M})$. Let
$\sigma^{2}=\rho(f|_{M},f|_{M})$. By Part a) of the same theorem,

$\displaystyle|\sigma_{k}^{2}-\sigma^{2}|$
$\displaystyle\leq|\rho(p_{k}|_{M},p_{k}|_{M})-\rho(p_{k}|_{M},f|_{M})|$
$\displaystyle\qquad\mbox{}+|\rho(p_{k}|_{M},f|_{M})-\rho(f|_{M},f|_{M})|$
$\displaystyle\leq
K(\|p_{k}^{\prime}\|_{\mathcal{K}}+\|f^{\prime}\|_{\mathcal{K}})\|(p_{k}-f)^{\prime}\|_{\mathcal{K}}.$

In particular, $\sigma_{k}^{2}\to\sigma^{2}$ and hence $Z^{(k)}\Rightarrow
Z\sim{\mathcal{N}}_{\mathbb{R}}(0,\sigma^{2})$ as $k\to\infty$.

For all $N\in\mathbb{N}$, we let
$Z_{N}=\textnormal{Tr}(f(X_{N}))-\mathbb{E}(\textnormal{Tr}(f(X_{N})))$. Let
$g:\mathbb{R}\to\mathbb{R}$ be a bounded Lipschitz function. Note that, for
all $N\in\mathbb{N}$ and all $k\in\mathbb{N}$,

$\mathbb{E}(g(Z_{N}))-\mathbb{E}(g(Z))=\alpha_{k,N}+\beta_{k,N}+\gamma_{k},$

where

$\displaystyle\alpha_{k,N}$
$\displaystyle=\mathbb{E}\left(g(Z_{N})-g\left(Z_{N}^{(k)}\right)\right),$
$\displaystyle\beta_{k,N}$
$\displaystyle=\mathbb{E}\left(g\left(Z_{N}^{(k)}\right)-g\left(Z^{(k)}\right)\right),$
$\displaystyle\gamma_{k}$
$\displaystyle=\mathbb{E}\left(g\left(Z^{(k)}\right)-g(Z)\right).$

Let $L_{g}$ be a Lipschitz constant for $g$. A routine computation shows that

$|\alpha_{k,N}|\leq
L_{g}\,\mathbb{E}\left(\left|Z_{N}-Z_{N}^{(k)}\right|\right)\leq
L_{g}\textnormal{Var}(Z_{N}-Z_{N}^{(k)})^{1/2}.$

Observe that

$\textnormal{Var}(Z_{N}-Z_{N}^{(k)})=\textnormal{Var}(\textnormal{Tr}((p_{k}-f)(X_{N})))=\rho_{N}(p_{k}-f,p_{k}-f).$

By Part b) of Lemma 19, there exists $C_{k}>0$ such that, for all
$N\in\mathbb{N}$,

$|\rho_{N}(p_{k}-f,p_{k}-f)|\leq
C_{k}N^{2}\mathbb{P}(\|X_{N}\|>M)^{1/4}+K\|(p_{k}-f)^{\prime}\|_{\mathcal{K}}^{2}.$

In particular, we have that

$|\alpha_{k,N}|\leq
L_{g}\left(C_{k}N^{2}\mathbb{P}(\|X_{N}\|>M)^{1/4}+K\|(p_{k}-f)^{\prime}\|_{\mathcal{K}}^{2}\right)^{1/2}.$

Let $\epsilon>0$. Since $Z^{(k)}\Rightarrow Z$, we have that
$\mathbb{E}(g\left(Z^{(k)}\right))\to\mathbb{E}(g(Z))$ as $k\to\infty$. Let
$k_{0}\in\mathbb{N}$ be such that

$|\gamma_{k_{0}}|=\left|\mathbb{E}(g\left(Z^{(k_{0})}\right))-\mathbb{E}(g(Z))\right|\leq\frac{\epsilon}{3}\quad\text{
and
}\quad\|(p_{k_{0}}-f)^{\prime}\|_{\mathcal{K}}^{2}\leq\frac{\epsilon^{2}}{18KL_{g}^{2}}.$

Since $N^{8}\mathbb{P}(\|X_{N}\|>M)\to 0$ and $Z_{N}^{(k_{0})}\Rightarrow
Z^{(k_{0})}$ as $N\to\infty$, there exists $N_{0}\in\mathbb{N}$ such that for
all $N>N_{0}$

$\displaystyle
C_{k_{0}}N^{2}\mathbb{P}(\|X_{N}\|>M)^{1/4}\leq\frac{\epsilon^{2}}{18L_{g}^{2}}$
$\displaystyle\text{ and
}\quad|\beta_{k_{0},N}|=\left|\mathbb{E}\left(g\left(Z_{N}^{(k_{0})}\right)\right)-\mathbb{E}\left(g\left(Z^{(k_{0})}\right)\right)\right|\leq\frac{\epsilon}{3}.$

Therefore, $\displaystyle|\mathbb{E}(g(Z_{N}))-\mathbb{E}(g(Z))|\leq\epsilon$
for all $N>N_{0}$, i.e.,

$\lim_{N\to\infty}\mathbb{E}(g(Z_{N}))=\mathbb{E}(g(Z)).$

The Portmanteau lemma implies then that $Z_{N}\Rightarrow Z$. In other words,

$\textnormal{Tr}(f(X_{N}))-\mathbb{E}(\textnormal{Tr}(f(X_{N})))\Rightarrow{\mathcal{N}}_{\mathbb{R}}(0,\sigma^{2}),$

as we claimed. ∎