m+1}{(\frac{Mc(\epsilon_{0})}{n})^{l}\cdot\binom{m}{l-1}}=\frac{Mc(\epsilon_{0})}{n}\cdot\frac{c(\epsilon_{0})^{m}}{1-\sigma}(1+\frac{Mc(\epsilon_{0})}{n})^{m}\leq(2c(\epsilon_{0})(1+c(\epsilon_{0})))^{m+1}\cdot\frac{M}{n},$

yielding the claim $|a_{m}|\leq c(\epsilon_{0})^{m+1}\cdot\frac{M}{n}.$

Consider now the overall error given by

$\sum_{1\leq m\leq s}{S^{\prime}(m)},$
$S^{\prime}(m):=\sum{\frac{l_{k}l_{k_{1}}...l_{k_{j-m}}}{(l_{\nu}-l_{k})(l_{\nu}-l_{k_{1}})...(l_{\nu}-l_{k_{j-m}})}\cdot(\frac{l_{\nu}}{l_{k_{j_{1}}-l_{\nu}}})^{j_{2}}...(\frac{l_{\nu}}{l_{k_{j_{2l-1}}-l_{\nu}}})^{j_{2l}}(\tilde{D}_{k\nu}\tilde{D}_{kk_{1}}...\tilde{D}_{k_{j-m}\nu}-\frac{1}{n^{j-m+1}})}:$

to conclude it is $o_{p}(\frac{1}{\sqrt{n}}),$ it suffices to show

$\sum_{1\leq m\leq s}{2^{2m}\mathbb{E}[n(S^{\prime}(m))^{2}]}=o(1),$

and use the analogue of (5.44).

Fix $m$ and $k,k_{1},\hskip 1.42271pt...\hskip 1.42271pt,k_{j-m};$ then

$|\frac{l_{k}l_{k_{1}}...l_{k_{j-m}}}{(l_{\nu}-l_{k})(l_{\nu}-l_{k_{1}})...(l_{\nu}-l_{k_{j-m}})}\cdot(\frac{l_{\nu}}{l_{k_{j_{1}}-l_{\nu}}})^{j_{2}}...(\frac{l_{\nu}}{l_{k_{j_{2l-1}}-l_{\nu}}})^{j_{2l}}|\leq
c(\epsilon_{0})^{j+1},$

there are at most $\binom{j-m+1}{l}+\binom{j-m+1}{l-1}=\binom{j-m+2}{l}$ ways
of choosing $k_{j_{1}},\hskip 1.42271pt...\hskip 1.42271pt,k_{j_{2l-1}}$
(either $l$ or $l-1$ unordered positions have to be chosen depending on
$j_{1}$ being nonzero or not), and at most $\binom{m}{l-1}$ tuples
$(j_{2},\hskip 1.42271pt...\hskip 1.42271pt,j_{2l})$ for each fixed $l.$ Thus,
the number of apparitions of each such product is upper bounded by

$\sum_{l\geq 0}{\binom{j-m+2}{l}\binom{m}{m-l+1}}=\binom{j+2}{m+1},$

from which

$\mathbb{E}[(S^{\prime}(m))^{2}]\leq\sum_{m\leq j_{1},j_{2}\leq
s}{c(\epsilon_{0})^{j_{1}+1}\cdot
c(\epsilon_{0})^{j_{2}+1}\cdot\binom{j_{1}+2}{m+1}\binom{j_{2}+2}{m+1}a_{j_{1}-m+1,j_{2}-m+1}}.$

Note that

$\sum_{j_{1}+j_{2}=j}{\binom{j_{1}+2}{m+1}\binom{j_{2}+2}{m+1}}\leq\sum_{j_{1}+j_{2}=j}{2^{j_{1}+2}\cdot
2^{j_{2}+2}}=2^{j+4}(j+1)\leq 2^{2j+4},$

and for $j=j_{1}+j_{2}-2m+2$

$\mathbb{E}[(S^{\prime}(m))^{2}]\leq\sum_{2\leq j\leq
2s}{c(\epsilon_{0})^{j+2m}2^{2j+4m}\max_{l_{1}+l_{2}=j}{a_{l_{1},l_{2}}}}\leq(4c(\epsilon_{0}))^{2m}\sum_{2\leq
j\leq
2s}{c(\epsilon_{0})^{j}2^{2j}\cdot\frac{j!}{n}\cdot(\frac{M}{n})^{j/2}\cdot(2^{k_{0}}j)^{7j}}\leq$
$\leq\frac{(4c(\epsilon_{0}))^{2m}}{n}\cdot\sum_{2\leq j\leq
2s}{(c(K,\epsilon_{0})\sqrt{\frac{M}{n}}j^{8})^{j}}\leq\frac{(4c(\epsilon_{0}))^{2m}}{n}\sum_{2\leq
j\leq
2s}{(c(K,\epsilon_{0})\sqrt{\frac{M}{n}}s^{8})^{j}}\leq\frac{(4c(\epsilon_{0}))^{2m}}{n}\cdot\frac{2M}{n}s^{16}c(K,\epsilon_{0}),$

where the second inequality and the last ensue from (5.42) and an analogue of
(5.45), respectively.

Finally, this last bound implies

$\sum_{1\leq m\leq s}{2^{2m}\mathbb{E}[n(S^{\prime}(m))^{2}]}\leq
2c(K,\epsilon_{0})\cdot\frac{Ms^{16}}{n}\sum_{1\leq m\leq
s}{(8c(\epsilon_{0}))^{2m}}\leq
2c(K,\epsilon_{0})\cdot\frac{Ms^{16}}{n}\cdot(8c(\epsilon_{0})+1)^{2s}=o(1)$

from

$2s\cdot 8c(\epsilon_{0})+16\log{s}-\log{\frac{n}{M}}\leq
16c(\epsilon_{0})\cdot\frac{9\log{n}}{\log{\frac{n}{M}}}+16\cdot\frac{9\log{n}}{\log{\frac{n}{M}}}-\log{\frac{n}{M}}\leq-\frac{1}{2}\log{\frac{n}{M}}\to-\infty$

for $n$ sufficiently large.

###  5.7 Controlling Errors: A Combinatorial Lemma

In this subsection, we justify (5.42) for which the key is the following
inequality:

$a_{j,t}\leq\frac{j}{n}a_{j-1,t}+\frac{t}{n}a_{j,t-1}+\frac{M^{\frac{t+j}{2}}\cdot(\frac{t+j}{2}+1)^{t+j}}{n^{\frac{t+j}{2}+1}}\cdot(\alpha(j)\alpha(t)+\alpha(j)+\alpha(t)+1)$
(5.47)

for $2\leq j,t\leq 2s+1,$ where

$\alpha(k)=(c_{1}(K)(k+1))^{\frac{k+1}{2}}+\frac{1}{\sqrt{n}}(c_{1}(K)(k+1))^{2k+2}.$

To see why (5.47) holds, notice there are two types of pairs of tuples
$(k_{1},k_{2},\hskip 1.42271pt...\hskip
1.42271pt,k_{j}),(k^{\prime}_{1},k^{\prime}_{2},\hskip 1.42271pt...\hskip
1.42271pt,k^{\prime}_{t}):$

$1.$ There exists an index $k_{i}$ or $k^{\prime}_{i^{\prime}}$ such that its
multiplicity in the multiset $k_{1},k_{2},\hskip 1.42271pt...\hskip
1.42271pt,k_{j},k^{\prime}_{1},k^{\prime}_{2},\hskip 1.42271pt...\hskip
1.42271pt,k^{\prime}_{t}$ is one (choose $i$ or $i^{\prime}$ to be minimal).
Suppose $k_{i}$ has the property just stated: then

$\mathbb{E}[\tilde{D}_{k_{1}\nu}\tilde{D}_{k_{1}k_{2}}...\tilde{D}_{k_{j-1}k_{j}}\tilde{D}_{k_{j}\nu}]=\frac{1}{n}\mathbb{E}[\tilde{D}_{k_{1}\nu}\tilde{D}_{k_{1}k_{2}}...\tilde{D}_{k_{i-2}k_{i-1}}\tilde{D}_{k_{i-1}k_{i+1}}\tilde{D}_{k_{i+1}k_{i+2}}...\tilde{D}_{k_{j-1}k_{j}}\tilde{D}_{k_{j}\nu}]$

since each term in the product will have two factors of the form
$z_{k_{i}l},1\leq l\leq n,$ and they have to be identical to make a non-zero
contribution to the expectation (that is, $l_{1}=l_{2}$), and thus after
conditioning on $z_{k_{i}},$ the product above ensues; a similar identity
holds for the cross expectation, and so these pairs contribute at most
$\frac{j}{n}a_{j-1,t}+\frac{t}{n}a_{j,t-1}.$

$2.$ Each value in the multiset $k_{1},k_{2},\hskip 1.42271pt...\hskip
1.42271pt,k_{j},k^{\prime}_{1},k^{\prime}_{2},\hskip 1.42271pt...\hskip
1.42271pt,k^{\prime}_{t}$ has multiplicity at least two. Note that

$\mathbb{E}[\tilde{D}^{2}_{k_{1}\nu}\tilde{D}^{2}_{k_{1}k_{2}}...\tilde{D}^{2}_{k_{j-1}k_{j}}\tilde{D}^{2}_{k_{j}\nu}]\leq\frac{\alpha^{2}(j)}{n^{j+1}}$
(5.48)

because

$\mathbb{E}[\tilde{D}^{2}_{k_{1}\nu}\tilde{D}^{2}_{k_{1}k_{2}}...\tilde{D}^{2}_{k_{j-1}k_{j}}\tilde{D}^{2}_{k_{j}\nu}]\leq\frac{1}{j+1}(\mathbb{E}[\tilde{D}^{2j+2}_{k_{1}\nu}]+\mathbb{E}[\tilde{D}^{2j+2}_{k_{1}k_{2}}]+...+\mathbb{E}[\tilde{D}^{2j+2}_{k_{j}\nu}])\leq$
$\leq\frac{1}{n^{j+1}}\max{(\mathbb{E}[(\frac{1}{\sqrt{n}}z_{1}^{T}z_{2})^{2j+2}],\mathbb{E}[(\frac{1}{\sqrt{n}}||z_{1}||^{2}-\sqrt{n})^{2j+2}])}\leq\frac{1}{n^{j+1}}((c_{1}(K)(j+1))^{j+1}+\frac{1}{n}(c_{1}(K)(j+1))^{4j+4}):$
(5.49)

the products in

$\mathbb{E}[(\frac{1}{\sqrt{n}}z_{1}^{T}z_{2})^{2j+2}]$

are of the form
$(\frac{1}{\sqrt{n}}z_{1i_{1}}z_{2i_{1}})^{m_{1}}\cdot...\cdot(\frac{1}{\sqrt{n}}z_{1i_{p}}z_{2i_{p}})^{m_{p}}$
for some

$i_{1}<...<i_{p},\hskip 2.84544ptm_{1}+m_{2}+...+m_{p}=2j+2,\hskip
2.84544ptm_{1},\hskip 1.42271pt...\hskip 1.42271pt,m_{p}>0;$

if $m_{i}=1$ for some $i,$ then the expectation of this product is zero; if
some $m_{i}>2,$ then $p\leq j,$ and so the overall contribution of such terms
is at most

$\frac{1}{n^{j+1}}\sum_{1\leq p\leq
j}{\binom{n}{p}\sum_{m_{1}+m_{2}+...+m_{p}=2j+2}{\binom{2j+2}{m_{1},m_{2},\hskip
1.42271pt...\hskip
1.42271pt,m_{p}}}\mathbb{E}[z_{11}^{4j+4}]}\leq\frac{1}{n^{j+1}}\sum_{1\leq
p\leq j}{n^{j}p^{2j+2}\mathbb{E}[z_{11}^{4j+4}]}\leq$
$\leq\frac{1}{n}\mathbb{E}[z_{11}^{4j+4}]\cdot\frac{(j+1)^{2j+3}}{2j+3}\leq\frac{1}{n}(c_{0}(K)\sqrt{4(j+1)})^{4(j+1)}\cdot(j+1)^{2j+2}\leq\frac{1}{n}(c_{1}(K)(j+1))^{4j+4}$
(5.50)

where we have used there are $\binom{n}{p}$ ways of choosing
$i_{1}<i_{2}<...<i_{p},$ and the weighted arithmetic-geometric mean inequality

$x_{1}^{n_{1}}...x_{p}^{n_{p}}\leq\frac{1}{n_{1}+...+n_{p}}(n_{1}x_{1}^{n_{1}+...+n_{p}}+...+n_{p}x_{p}^{n_{1}+...+n_{p}})$
(AM-GM)

for $x_{i}\geq 0,n_{i}>0;$ lastly, when $m_{1}=m_{2}=...=m_{p}=2,p=j+1,$ the
expectation of each such term is one and their overall contribution will be

$\frac{1}{n^{j+1}}\binom{n}{j+1}\binom{2j+2}{2}\binom{2j}{2}...\binom{2}{2}=\frac{(n-1)(n-2)...(n-j)}{n^{j}}(2j+2)!!\leq(2j+2)^{j+1};$
(5.51)

a similar analysis holds for
$\mathbb{E}[(\frac{1}{\sqrt{n}}||z_{1}||^{2}-\sqrt{n})^{2j+2}]$ with the
distinction that in the analogue (5.50), $\mathbb{E}[z_{11}^{4j+4}]$ is
replaced by

$\mathbb{E}[(z_{11}^{2}-1)^{2j+2}]\leq 2^{2j+2}\mathbb{E}[z_{11}^{4j+4}]\leq
2^{2j+2}(c_{0}(K)\sqrt{4j+4})^{4j+4}\leq(c_{1}(K)(j+1))^{2j+2},$

employing $(x+y)^{2j+2}\leq 2^{2j+2}\max{(x^{2j+2},y^{2j+2})},$ and in the
surrogate of (5.51), $\mathbb{E}[z_{11}^{2}]^{j+1}=1$ is substituted by

$\mathbb{E}[(z_{11}^{2}-1)^{2}]^{j+1}\leq\mathbb{E}[z^{4}_{11}]^{j+1}\leq(\sqrt{4}c_{0}(K))^{4(j+1)}\leq(\frac{c_{1}(K)}{2})^{j+1}.$

Cauchy-Schwarz inequality yields

$|\mathbb{E}[(X-a)(Y-b)]|\leq|ab|+\sqrt{\mathbb{E}[X^{2}]\cdot\mathbb{E}[Y^{2}]}+|b|\sqrt{\mathbb{E}[X^{2}]}+|a|\sqrt{\mathbb{E}[Y^{2}]},$

and together with (5.48) it implies the contribution of each such pair is at
most

$\frac{1}{n^{t+j}}+\frac{\alpha(j)}{n^{t+\frac{j+1}{2}}}+\frac{\alpha(t)}{n^{j+\frac{t+1}{2}}}+\frac{\alpha(j)\cdot\alpha(t)}{n^{\frac{t+j}{2}+1}}\leq\frac{\alpha(j)\alpha(t)+\alpha(j)+\alpha(t)+1}{n^{\frac{t+j}{2}+1}},$

while their number is at most $M^{\frac{t+j}{2}}\cdot(\frac{t+j}{2}+1)^{t+j}$
because for each $(t+j)$-tuple of this form, let $m_{1},m_{2},\hskip
1.42271pt...\hskip 1.42271pt,m_{l}$ be the multiplicities of the distinct
values in the tuple (increasingly ordered): then $l\leq\frac{t+j}{2}$ from
$\newline m_{i}\geq 2,m_{1}+...+m_{l}=t+j,$ and thus their number is at most

$\sum_{1\leq
l\leq\frac{t+j}{2},m_{1}+...+m_{l}=t+j}{\binom{t+j}{m_{1},m_{2},\hskip
1.42271pt...\hskip 1.42271pt,m_{l}}\binom{M}{l}}\leq\sum_{1\leq
l\leq\frac{t+j}{2}}{l^{t+j}\binom{M}{l}}\leq$ $\leq\sum_{1\leq
l\leq\frac{t+j}{2}}{l^{t+j}M^{l}}\leq\sum_{1\leq
l\leq\frac{t+j}{2}}{l^{t+j}M^{(t+j)/2}}\leq
M^{(t+j)/2}\cdot\frac{(\frac{t+j}{2}+1)^{t+j+1}}{t+j+1}\leq
M^{(t+j)/2}\cdot(\frac{t+j}{2}+1)^{t+j},$ (5.52)

completing the justification of (5.47).

To finish the proof of (5.42), we get upper bounds for $a_{j,t}$ given by some
$\tilde{a}_{j+t}.$ Notice that any sequence for which the following hold

$\tilde{a}_{l}\geq\frac{l}{n}\tilde{a}_{l-1}+\frac{1}{n}\cdot(\frac{M}{n})^{l/2}\cdot(c_{1}(K)l)^{3l+4},\hskip
14.22636pt\tilde{a}_{l}\geq a_{1,l-1}$ (5.53)

for $l\geq 2$ works using (5.47), $(\frac{l}{2}+1)^{l}\leq
l^{l},\alpha(j)\alpha(t)\leq c_{1}(K)^{2j+2t+4}\cdot(j+t)^{2j+2t+4}$ from

$\alpha(k)=(c_{1}(K)(k+1))^{\frac{k+1}{2}}+\frac{1}{\sqrt{n}}(c_{1}(K)(k+1))^{2k+2}\leq(c_{1}(K)(k+1))^{2k+2}$
(5.54)

and the upper bound of $\alpha(j)\alpha(t)$ reused for $\alpha(j)+\alpha(t)+1$
inasmuch as $1+x+y\leq xy$ for $x,y\geq 3.$

Denote by

$\tilde{a}_{l}=\frac{l!}{n}\cdot(\frac{M}{n})^{l/2}\cdot\overline{a}_{l},$

and we show inductively that $\overline{a}_{l}=(2^{k_{0}}l)^{3l+4}$ satisfies
(5.53). For the base case, take $\tilde{a}_{2}=a_{1,1}$ with

$a_{1,1}=\sum_{k_{1}\neq\nu,k_{2}\neq\nu}{|\mathbb{E}[(\tilde{D}^{2}_{\nu
k_{1}}-\frac{1}{n})(\tilde{D}^{2}_{\nu
k_{2}}-\frac{1}{n})]|}\leq\frac{M^{2}(2c_{0}(K))^{4}}{n^{3}}+\frac{M\alpha^{2}(1)}{n^{2}}\leq\frac{2}{n}\cdot\frac{M}{n}\cdot
2^{k_{0}}$

since for $k_{1}\neq k_{2},$

$\mathbb{E}[\tilde{D}^{2}_{\nu k_{1}}\tilde{D}^{2}_{\nu
k_{2}}]=\frac{1}{n^{4}}\sum_{1\leq i,j\leq n}{\mathbb{E}[z_{\nu i}^{2}z_{\nu
j}^{2}]}=\frac{1}{n^{2}}+\frac{\mathbb{E}[z_{11}^{4}]-1}{n^{3}}\leq\frac{1}{n^{2}}+\frac{(2c_{0}(K))^{4}}{n^{3}}$

and for $k_{1}=k_{2},$ $\mathbb{E}[\tilde{D}^{4}_{\nu
k_{1}}]\leq\frac{\alpha^{2}(1)}{n^{2}}$ from (5.48). We proceed with the
induction step, which requires the two inequalities in (5.53): the first is
equivalent to

$\overline{a}_{l}\geq\frac{\overline{a}_{l-1}}{\sqrt{nM}}+\frac{(c_{1}(K)l)^{3l+4}}{l!},$

which holds true as $k_{0}\geq 3,$

$\frac{\overline{a}_{l-1}}{\sqrt{nM}}=\frac{(2^{k_{0}}(l-1))^{3l+1}}{\sqrt{nM}}\leq\frac{(2^{k_{0}}l)^{3l+4}}{2},$
$\frac{(c_{1}(K)l)^{3l+4}}{l!}\leq\frac{(c_{1}(K)l)^{3l+4}}{2}\leq\frac{(2^{k_{0}}l)^{3l+4}}{2}.$

For the second inequality, notice that

$a_{1,l}=\sum_{k,k_{1},\hskip 1.42271pt...\hskip
1.42271pt,k_{l}\neq\nu}{|\mathbb{E}[(\tilde{D}_{\nu
k}^{2}-\frac{1}{n})\cdot(\tilde{D}_{\nu
k_{1}}...\tilde{D}_{k_{l}\nu}-\frac{1}{n^{l}})]|}\leq$
$\leq\frac{c_{1}(K)}{n}\cdot(\frac{M}{n})^{l+1}+\frac{l}{n}a_{1,l-1}+\alpha(l+2)\cdot(\frac{l}{2}+1)^{l}\cdot\frac{M^{\frac{l}{2}+1}}{n^{\frac{l}{2}+2}}+2l\alpha(l+3)\cdot(\frac{l}{2}+1)^{l}\cdot\frac{M^{\frac{l}{2}}}{n^{\frac{l+3}{2}}}$
(5.55)

since each $(k,k_{1},\hskip 1.42271pt...\hskip 1.42271pt,k_{l})$ falls in one
of the following four categories: $\newline 1.\hskip 5.69046ptk,k_{1},\hskip
1.42271pt...\hskip 1.42271pt,k_{l}$ are pairwise distinct: by conditioning on
$z_{k},z_{k_{1}},\hskip 1.42271pt...\hskip 1.42271pt,z_{k_{l}},$

$\mathbb{E}[(\tilde{D}_{\nu k}^{2}-\frac{1}{n})\cdot(\tilde{D}_{\nu
k_{1}}...\tilde{D}_{k_{l}\nu}-\frac{1}{n^{l}})]=\mathbb{E}[\tilde{D}_{\nu
k}^{2}\tilde{D}_{\nu
k_{1}}...\tilde{D}_{k_{l}\nu}]-\frac{1}{n}\mathbb{E}[\tilde{D}_{\nu
k_{1}}...\tilde{D}_{k_{l}\nu}]=$ $=\frac{1}{n^{l+3}}\sum_{1\leq i,j\leq
n}{\mathbb{E}[z^{2}_{\nu i}z^{2}_{\nu
j}]}-\frac{1}{n^{l+1}}=\frac{1}{n^{l+2}}(\mathbb{E}[z_{11}^{4}]-1)\leq\frac{c_{1}(K)}{n^{l+2}}.$

$\newline 2.$ There exists $k_{i}$ such that its multiplicity in
$k,k_{1},\hskip 1.42271pt...\hskip 1.42271pt,k_{l}$ is one: by conditioning on
$z_{k_{i}}$ (with $k_{i}$ minimal), their overall contribution is at most
$\frac{l}{n}a_{1,l-1}.$ $\newline 3.\hskip 5.69046ptk$ has multiplicity one in
$k,k_{1},\hskip 1.42271pt...\hskip 1.42271pt,k_{l}$ whereas $k_{1},\hskip
1.42271pt...\hskip 1.42271pt,k_{l}$ have multiplicity at least two:

$\mathbb{E}[\tilde{D}_{\nu k}^{2}\tilde{D}_{\nu
k_{1}}...\tilde{D}_{k_{l}\nu}]-\frac{1}{n}\mathbb{E}[\tilde{D}_{\nu
k_{1}}...\tilde{D}_{k_{l}\nu}]=\frac{1}{n}\mathbb{E}[\tilde{D}_{\nu\nu}\tilde{D}_{\nu
k_{1}}...\tilde{D}_{k_{l}\nu}]\leq\frac{\alpha(l+2)}{n^{\frac{l}{2}+2}}$

by conditioning on $z_{k},$

$\mathbb{E}[\tilde{D}_{\nu k}^{2}\tilde{D}_{\nu
k_{1}}...\tilde{D}_{k_{l}\nu}]=\mathbb{E}[\frac{1}{n^{2}}||z_{\nu}||^{2}\tilde{D}_{\nu
k_{1}}...\tilde{D}_{k_{l}\nu}]$

in conjunction with Cauchy-Schwarz inequality and (5.49). There are at most
$M\cdot M^{l/2}\cdot(\frac{l}{2}+1)^{l}$ such tuples: $k$ can take at most $M$
values, and recall the bound (5.52) for the number of $(k_{1},\hskip
1.42271pt...\hskip 1.42271pt,k_{l}).$ $\newline 4.\hskip 5.69046ptk=k_{i}$ for
some $i,$ and $k_{1},\hskip 1.42271pt...\hskip 1.42271pt,k_{l}$ have
multiplicity at least two:

$|\mathbb{E}[\tilde{D}_{\nu k}^{2}\tilde{D}_{\nu
k_{1}}...\tilde{D}_{k_{l}\nu}]-\frac{1}{n}\mathbb{E}[\tilde{D}_{\nu
k_{1}}...\tilde{D}_{k_{l}\nu}]|\leq|\mathbb{E}[\tilde{D}_{\nu
k}^{2}\tilde{D}_{\nu
k_{1}}...\tilde{D}_{k_{l}\nu}]|+\frac{1}{n}|\mathbb{E}[\tilde{D}_{\nu
k_{1}}...\tilde{D}_{k_{l}\nu}]|\leq\frac{2\alpha(l+3)}{n^{\frac{l+3}{2}}}$

using (5.49) and $\alpha(l+3)\geq\alpha(l+1).$ The number of such tuples is at
most $l\cdot M^{l/2}\cdot(\frac{l}{2}+1)^{l}.$

Then by induction,

$a_{1,l}\leq\frac{(l+1)!}{n}\cdot(\frac{M}{n})^{\frac{l+1}{2}}\cdot(2^{k_{0}}(l+1))^{3l+4}:$

the base case $l=1$ was already covered above, and the induction step follows
from (5.55) and

$1.\hskip
5.69046pt\frac{c_{1}(K)}{n}\cdot(\frac{M}{n})^{l+1}\leq\frac{1}{4}\cdot\frac{(l+1)!}{n}\cdot(\frac{M}{n})^{\frac{l+1}{2}}\cdot(2^{k_{0}}(l+1))^{3l+4},$
$2.\hskip
5.69046pt\frac{l}{n}a_{1,l-1}\leq\frac{(l+1)!}{n^{2}}\cdot(\frac{M}{n})^{\frac{l}{2}}\cdot(2^{k_{0}}l)^{3l+1}\leq\frac{1}{4}\cdot\frac{(l+1)!}{n}\cdot(\frac{M}{n})^{\frac{l+1}{2}}\cdot(2^{k_{0}}(l+1))^{3l+4},$
$3.\hskip
5.69046pt\alpha(l+2)\cdot(\frac{l}{2}+1)^{l}\cdot\frac{M^{\frac{l}{2}+1}}{n^{\frac{l}{2}+2}}\leq\alpha(l+2)\cdot(\frac{l}{2}+1)^{l}\cdot\frac{1}{n}\cdot(\frac{M}{n})^{\frac{l+1}{2}}\leq\frac{1}{4}\cdot\frac{(l+1)!}{n}\cdot(\frac{M}{n})^{\frac{l+1}{2}}\cdot(2^{k_{0}}(l+1))^{3l+4}$

from (5.54) and $l+3\leq 2(l+1)$ which give

$\alpha(l+2)\cdot(\frac{l}{2}+1)^{l}\leq(2c_{1}(K)(l+1))^{2l+6}\cdot(\frac{l}{2}+1)^{l}\leq\frac{(l+1)!}{4}\cdot(2^{k_{0}}(l+1))^{3l+4}$

because

$\frac{(l+1)!}{4}\cdot(2^{k_{0}}(l+1))^{3l+4}\geq\frac{(l+1)^{2}}{8}\cdot(4c_{1}(K)(l+1))^{3l+4}=\frac{(l+1)^{2}}{8}\cdot(4c_{1}(K)(l+1))^{2l+6}\cdot(4c_{1}(K)(l+1))^{l-2}\geq$
$\geq\frac{(l+1)^{2}}{8}\cdot(4c_{1}(K)(l+1))^{2l+6}\cdot(l+1)^{l-2}\geq(2c_{1}(K)(l+1))^{2l+6}\cdot(l+1)^{l},$
$4.\hskip
5.69046pt2l\alpha(l+3)\cdot(\frac{l}{2}+1)^{l}\cdot\frac{M^{\frac{l}{2}}}{n^{\frac{l+3}{2}}}\leq
2l\alpha(l+3)\cdot(\frac{l}{2}+1)^{l}\cdot\frac{1}{n}\cdot(\frac{M}{n})^{\frac{l+1}{2}}\leq\frac{1}{4}\cdot\frac{(l+1)!}{n}\cdot(\frac{M}{n})^{\frac{l+1}{2}}\cdot(2^{k_{0}}(l+1))^{3l+4},$

from (5.54) and $l+4\leq 2(l+1)$ yielding

$2l\alpha(l+3)\cdot(\frac{l}{2}+1)^{l}\leq(2c_{1}(K)(l+1))^{2l+8}\cdot(l+2)^{l}\leq(2c_{1}(K)(l+1))^{3l+8}\leq\frac{(l+1)!}{4}\cdot(2^{k_{0}}(l+1))^{3l+4}$

inasmuch as

$\frac{(l+1)!}{4}\cdot(2^{k_{0}}(l+1))^{3l+4}\geq\frac{(l+1)^{2}}{8}\cdot(2c_{1}(K)(l+1))^{3l+4}\cdot(2c_{1}(K))^{3l+4}\geq$
$\geq(2c_{1}(K)(l+1))^{3l+4}\cdot(l+1)^{5}\cdot(c_{1}(K))^{3l+4}\geq(2c_{1}(K)(l+1))^{3l+8}.$

###  5.8 Second Sum: Stages I, II, and III

In this subsection, we justify (5.24): the argument is similar to the one for
(5.23) presented in 5.4-5.7, and thus we skip some of the details.

First, we truncate first the series underlying $\Sigma_{0,k}$ for $k\neq\nu:$

$\tilde{\Sigma}_{0}=\sum_{k\neq\nu}{(\sum_{0\leq j\leq
s}{((-M_{\nu})^{j}\mathcal{R}_{\nu}\mathcal{D}_{\nu}e_{\nu})_{k}}+\sum_{j>s}{((-M_{\nu})^{j}\mathcal{R}_{\nu}\mathcal{D}_{\nu}e_{\nu})_{k}})^{2}}.$

Recall that $||M_{\nu}||\xrightarrow[]{a.s.}0,$ and note that

$\sqrt{\sum_{k\neq\nu}{(\sum_{j>s}{((-M_{\nu})^{j}\mathcal{R}_{\nu}\mathcal{D}_{\nu}e_{\nu})_{k}})^{2}}}=||\sum_{j>s}{(-M_{\nu})^{j}\mathcal{R}_{\nu}\mathcal{D}_{\nu}e_{\nu}}||\leq\sum_{j>s}{||M_{\nu}||^{j}\cdot\beta_{\nu}}\leq
2||M_{\nu}||^{s+1}\cdot\beta_{\nu},$

which together with (5.2) and Lemma 2 gives almost surely for $k\neq\nu,$

$\sqrt{\sum_{k\neq\nu}{(\sum_{0\leq j\leq
s}{((-M_{\nu})^{j}\mathcal{R}_{\nu}\mathcal{D}_{\nu}e_{\nu})_{k}})^{2}}}\leq\frac{||a_{\nu}-e_{\nu}||}{|1-\frac{||a_{\nu}-e_{\nu}||^{2}}{2}|}+\sqrt{\sum_{k\neq\nu}{(\sum_{j>s}{((-M_{\nu})^{j}\mathcal{R}_{\nu}\mathcal{D}_{\nu}e_{\nu})_{k}})^{2}}}\leq$
$\leq 2||a_{\nu}-e_{\nu}||+2\beta_{\nu}\leq 6\beta_{\nu}.$ (5.56)

Cauchy-Schwarz inequality then yields

$|\tilde{\Sigma}_{0}-\sum_{k\neq\nu}{(\sum_{0\leq j\leq
s}{((-M_{\nu})^{j}\mathcal{R}_{\nu}\mathcal{D}_{\nu}e_{\nu})_{k}})^{2}}|\leq
4||M_{\nu}||^{2s+2}\cdot\beta_{\nu}^{2}+24\cdot||M_{\nu}||^{s+1}\cdot\beta_{\nu}^{2}=o_{p}(\frac{1}{\sqrt{n}})$

because $\sqrt{n}\cdot||M_{\nu}||^{s+1}\cdot\beta_{\nu}^{2}\xrightarrow[]{p}0$
from (5.18) and (5.35).

Second, we show the first type terms in

$\sum_{k\neq\nu}{(\sum_{0\leq j\leq
s}{((-M_{\nu})^{j}\mathcal{R}_{\nu}\mathcal{D}_{\nu}e_{\nu})_{k}})^{2}}$

are negligible. Recall that

$-M_{\nu}=A_{\nu}+B_{\nu},\hskip
5.69046ptA_{\nu}=-\mathcal{R}_{\nu}\Lambda^{1/2}\tilde{T}\Lambda^{1/2},\hskip
5.69046ptB_{\nu}=-\mathcal{R}_{\nu}\Lambda^{1/2}\tilde{D}\Lambda^{1/2}+(\frac{\hat{l}_{\nu}}{l_{\nu}}-1)\tilde{R},$
$\sum_{0\leq j\leq
s}{(A_{\nu}+B_{\nu})^{j}\mathcal{R}_{\nu}\mathcal{D}_{\nu}e_{\nu}}-\sum_{0\leq
j\leq s}{B_{\nu}^{j}\mathcal{R}_{\nu}\mathcal{D}_{\nu}e_{\nu}}=\sum_{1\leq
j\leq s,0\leq m\leq
j-1}{(A_{\nu}+B_{\nu})^{j-1-m}A_{\nu}B_{\nu}^{m}\mathcal{R}_{\nu}\mathcal{D}_{\nu}e_{\nu}},$
$||\sum_{0\leq j\leq
s}{B_{\nu}^{j}\mathcal{R}_{\nu}\mathcal{D}_{\nu}e_{\nu}}||\leq\sum_{0\leq
j\leq s}{||B_{\nu}||^{j}\cdot\beta_{\nu}}\leq 2\beta_{\nu},$ $||\sum_{1\leq
j\leq s,0\leq m\leq
j-1}{(A_{\nu}+B_{\nu})^{j-1-m}A_{\nu}B_{\nu}^{m}\mathcal{R}_{\nu}\mathcal{D}_{\nu}e_{\nu}}||\leq$
$\leq\sum_{1\leq j\leq s,0\leq m\leq
j-1}{(||A_{\nu}||+||B_{\nu}||)^{j-1-m}\cdot||A_{\nu}||\cdot||B_{\nu}||^{m}\cdot\beta_{\nu}}\leq||A_{\nu}||\cdot\beta_{\nu}\sum_{j\geq
1}{\frac{j}{2^{j-1}}}=4||A_{\nu}||\cdot\beta_{\nu},$

as $||B_{\nu}||,||A_{\nu}||\xrightarrow[]{a.s.}0,$ from which the desired
claim ensues:

$|\sum_{k\neq\nu}{(\sum_{0\leq j\leq
s}{(A_{\nu}+B_{\nu})^{j}\mathcal{R}_{\nu}\mathcal{D}_{\nu}e_{\nu}}})_{k}^{2}-\sum_{k\neq\nu}{(\sum_{0\leq
j\leq s}{B_{\nu}^{j}\mathcal{R}_{\nu}\mathcal{D}_{\nu}e_{\nu}})_{k}^{2}}|\leq
16\beta_{\nu}^{2}||A_{\nu}||^{2}+16\beta_{\nu}^{2}||A_{\nu}||=o_{p}(\frac{1}{\sqrt{n}})$

since $\sqrt{n}\cdot\beta_{\nu}^{2}||A_{\nu}||\xrightarrow[]{p}0$ from (5.18)
for $\delta_{n}=\frac{l_{\nu}}{M/\sqrt{n}}\to\infty,$ and
$||A_{\nu}||\leq\frac{c(K,\gamma,\epsilon_{0})}{l_{\nu}}$ almost surely.

Third, we turn to the remaining contribution in (5.24)

$\sum_{0\leq j_{1},j_{2}\leq s,k\neq\nu}{(M(j_{1}))_{k}\cdot(M(j_{2}))_{k}},$
(5.57)

where

$M(j)=(-\mathcal{R}_{\nu}\Lambda^{1/2}\tilde{D}\Lambda^{1/2}+(\frac{\hat{l}_{\nu}}{l_{\nu}}-1)\tilde{R})^{j_{1}}\mathcal{R}_{\nu}\Lambda^{1/2}\tilde{D}\Lambda^{1/2}e_{\nu},$

because

$|\sum_{k\neq\nu}{(\sum_{0\leq j\leq
s}{B_{\nu}^{j}\mathcal{R}_{\nu}\mathcal{D}_{\nu}e_{\nu}})_{k}^{2}}-\sum_{k\neq\nu}{(\sum_{0\leq
j\leq
s}{B_{\nu}^{j}\mathcal{R}_{\nu}\Lambda^{1/2}\tilde{D}\Lambda^{1/2}e_{\nu}})_{k}^{2}}|\leq||\sum_{0\leq
j\leq
s}{B_{\nu}^{j}\mathcal{R}_{\nu}\Lambda^{1/2}\tilde{T}\Lambda^{1/2}e_{\nu}}||^{2}+$
$+2\cdot||\sum_{0\leq j\leq
s}{B_{\nu}^{j}\mathcal{R}_{\nu}\Lambda^{1/2}\tilde{D}\Lambda^{1/2}e_{\nu}}||\cdot||\sum_{0\leq
j\leq
s}{B_{\nu}^{j}\mathcal{R}_{\nu}\Lambda^{1/2}\tilde{T}\Lambda^{1/2}e_{\nu}}||\leq
c(\epsilon_{0})\cdot(||\tilde{T}e_{\nu}||^{2}+2||\tilde{T}e_{\nu}||\cdot||\tilde{D}e_{\nu}||)=o_{p}(\frac{1}{\sqrt{n}})$

since (8.4) gives $\sqrt{n}\cdot||\tilde{T}e_{\nu}||^{2}\xrightarrow[]{p}0$ as
for $t>0,$ $\frac{t\sqrt{n}}{M}\cdot l_{\nu}^{2}\to\infty,$ while (5.31) and
(5.33) imply

$\sqrt{n}\cdot||\tilde{T}e_{\nu}||\cdot||\tilde{D}e_{\nu}||\xrightarrow[]{p}0.$

As before, we begin with the terms in (5.57) that have no factor of
$\frac{\hat{l}_{\nu}}{l_{\nu}}-1:$

$\sum_{0\leq j_{1},j_{2}\leq
s,k\neq\nu}{((-\mathcal{R}_{\nu}\Lambda^{1/2}\tilde{D}\Lambda^{1/2})^{j_{1}+1})_{k\nu}\cdot((-\mathcal{R}_{\nu}\Lambda^{1/2}\tilde{D}\Lambda^{1/2})^{j_{2}+1})_{k\nu}}=$
$=\sum_{0\leq j_{1},j_{2}\leq
s}{\frac{l_{k}l_{\nu}}{(l_{\nu}-l_{k})^{2}}\cdot\frac{l_{k_{1}}...l_{k_{j_{1}+j_{2}}}}{(l_{\nu}-l_{k_{1}})...(l_{\nu}-l_{k_{j_{1}+j_{2}}})}\tilde{D}_{kk_{1}}...\tilde{D}_{k_{j_{1}}\nu}\tilde{D}_{kk_{j_{1}+1}}...\tilde{D}_{k_{j_{1}+j_{2}}\nu}}=$
$=\sum_{1\leq j\leq 2s+1}{(\sum_{1\leq
m\leq\min{(s+1,j)}}{\frac{l_{\nu}}{l_{\nu}-l_{k_{m}}}})\cdot\frac{l_{k_{1}}...l_{k_{j}}}{(l_{\nu}-l_{k_{1}})...(l_{\nu}-l_{k_{j}})}\tilde{D}_{k_{1}\nu}\tilde{D}_{k_{1}k_{2}}...\tilde{D}_{k_{j}\nu}},$
(5.58)

where the last equality comes from a change of summation: instead of
$(j_{1},j_{2},k,k_{1},\hskip 1.42271pt...\hskip 1.42271pt,k_{j_{1}+j_{2}})$
with $k,k_{i}\neq\nu,0\leq j_{1},j_{2}\leq s$ use
$(j_{1}+j_{2}+1,k_{j_{1}},\hskip 1.42271pt...\hskip
1.42271pt,k_{1},k,k_{j_{1}+1},\hskip 1.42271pt...\hskip
1.42271pt,k_{j_{1}+j_{2}});$ because

$|(\sum_{1\leq
m\leq\min{(s+1,j)}}{\frac{l_{\nu}}{l_{\nu}-l_{k_{m}}}})\cdot\frac{l_{k_{1}}...l_{k_{j}}}{(l_{\nu}-l_{k_{1}})...(l_{\nu}-l_{k_{j}})}|\leq(s+1)\cdot
c(\epsilon_{0})^{j+1},$

an analogous argument to the one in subsection $6.6$ yields that the sum in
(5.58) is

$\sum_{1\leq j\leq 2s+1}{\frac{1}{n^{j}}(\sum_{1\leq
m\leq\min{(s+1,j)}}{\frac{l_{\nu}}{l_{\nu}-l_{k_{m}}}})\cdot\frac{l_{k_{1}}...l_{k_{j}}}{(l_{\nu}-l_{k_{1}})...(l_{\nu}-l_{k_{j}})}}+o_{p}(\frac{1}{\sqrt{n}}),$

with the first term being $b_{0}$ (employing (5.58) again) and

$|\sum_{1\leq j\leq 2s+1}{\frac{1}{n^{j}}(\sum_{1\leq
m\leq\min{(s+1,j)}}{\frac{l_{\nu}}{l_{\nu}-l_{k_{m}}}})\cdot\frac{l_{k_{1}}...l_{k_{j}}}{(l_{\nu}-l_{k_{1}})...(l_{\nu}-l_{k_{j}})}}|\leq$
$\leq\sum_{1\leq j\leq 2s+1}{\frac{1}{n^{j}}(\sum_{1\leq m\leq
j}{\frac{l_{\nu}}{|l_{\nu}-l_{k_{m}}|}})\cdot\frac{l_{k_{1}}...l_{k_{j}}}{|l_{\nu}-l_{k_{1}}|\cdot...\cdot|l_{\nu}-l_{k_{j}}|}}\leq\frac{1}{n}\sum_{k\neq\nu}{\frac{l_{k}l_{\nu}}{(l_{k}-l_{\nu})^{2}}}\sum_{j\geq
0}{(\frac{1}{n}\sum_{k\neq\nu}{\frac{l_{k}}{|l_{\nu}-l_{k}|}})^{j}}\leq$
$\leq\frac{2}{n}\sum_{k\neq\nu}{\frac{l_{k}l_{\nu}}{(l_{k}-l_{\nu})^{2}}}\leq
c(\epsilon_{0})\cdot\frac{M}{n}.$

We consider next the terms in (5.57) with a factor of exactly
$(\frac{\hat{l}_{\nu}}{l_{\nu}}-1)^{m}$ for $1\leq m\leq 2s:$

$\sum_{0\leq j_{1},j_{2}\leq
s}{\frac{l_{k}l_{\nu}}{(l_{k}-l_{\nu})^{2}}\cdot\frac{l_{k_{1}}...l_{k_{j_{1}+j_{2}-m}}}{(l_{\nu}-l_{k_{1}})...(l_{\nu}-l_{k_{j_{1}+j_{2}-m_{1}-m_{2}}})}\cdot(\frac{l_{\nu}}{l_{k_{j_{1}}}-l_{\nu}})^{j_{2}}...(\frac{l_{\nu}}{l_{k_{j_{2l_{1}+2l_{2}-1}}}-l_{\nu}})^{j_{2l_{1}+2l_{2}}}}\cdot$
$\cdot\tilde{D}_{kk_{1}}...\tilde{D}_{k_{j_{1}-m_{1}}\nu}\tilde{D}_{kk_{j_{1}-m_{1}+1}}...\tilde{D}_{k_{j_{1}+j_{2}-m}\nu},$

with the constraints

$m_{1}+m_{2}=m,m_{1}\leq j_{1},m_{2}\leq j_{2},1\leq
l_{1}\leq\min{(m_{1}+1,j_{1}-m_{1}+1)},1\leq
l_{2}\leq\min{(m_{2}+1,j_{2}-m_{2}+1)},$
$j_{2}+...+j_{2l_{1}}=m_{1},j_{2l_{1}+2}+...+j_{2l_{1}+2l_{2}}=m_{2},j_{2},\hskip
1.42271pt...\hskip 1.42271pt,j_{2l_{1}-2},j_{2l_{1}+2},\hskip
1.42271pt...\hskip
1.42271pt,j_{2l_{1}+2l_{2}-2}>0,j_{2l_{1}},j_{2l_{1}+2l_{2}}\geq 0.$

The analysis of these terms is completed arguing in the same vein as in
subsection $6.6$ for the case $m>0:$ using a change of summation the $m^{th}$
contribution is

$\sum_{0\leq j_{1},j_{2}\leq
s}{\frac{1}{n^{j_{1}+j_{2}-m+1}}\cdot\frac{l_{k}l_{\nu}}{(l_{k}-l_{\nu})^{2}}\cdot\frac{l_{k_{1}}...l_{k_{j_{1}+j_{2}-m}}}{(l_{\nu}-l_{k_{1}})...(l_{\nu}-l_{k_{j_{1}+j_{2}-m}})}\cdot(\frac{l_{\nu}}{l_{k_{j_{1}}}-l_{\nu}})^{j_{2}}...(\frac{l_{\nu}}{l_{k_{j_{2l_{1}+2l_{2}-1}}}-l_{\nu}})^{j_{2l_{1}+2l_{2}}}}+S^{\prime\prime}(m),$

the first term being $b_{m}$ from (5.58) with $|b_{m}|\leq
c(\epsilon_{0})^{m+1}\cdot\frac{M}{n},$ and

$\sum_{1\leq m\leq 2s}{\mathbb{E}[n(S^{\prime\prime}(m))^{2}]}=o(1).$

###  5.9 Third Sum: Stages I, II, and III

In this subsection, we justify the last identity left to complete the proof of
Theorem 1, (5.25): because the argument is in the same vein as the one
employed for (5.23) in 5.4-5.7, we omit some of the details:

$\tilde{\Sigma}_{2}=\sum_{k\neq\nu}{\frac{l_{k}}{l_{\nu}}\Sigma_{0,k}^{2}}+\frac{1}{l_{\nu}}\sum_{k_{1},k_{2}\neq\nu}{(\tilde{T}_{k_{1}k_{2}}+\tilde{D}_{k_{1}k_{2}})\sqrt{l_{k_{1}}}\Sigma_{0,k_{1}}\sqrt{l_{k_{2}}}\Sigma_{0,k_{2}}}.$

For the first component, the analysis for $\tilde{\Sigma}_{0}$ yields a
decomposition for it as well, giving the first sum in (2.6), and hence we
consider the second component. Before employing the three-step proof
previously used, note that

$|\frac{1}{l_{\nu}}\sum_{k_{1},k_{2}\neq\nu}{\tilde{T}_{k_{1}k_{2}}\sqrt{l_{k_{1}}}\Sigma_{0,k_{1}}\sqrt{l_{k_{2}}}\Sigma_{0,k_{2}}}|\leq\frac{1}{l_{\nu}}\cdot||\tilde{T}||\cdot\sum_{k\neq\nu}{l_{k}\Sigma_{0,k}^{2}}=\frac{||\Lambda^{1/2}(a_{\nu}-e_{\nu})||^{2}}{l_{\nu}(\frac{||a_{\nu}-e_{\nu}||^{2}}{2}-1)^{2}}\cdot||\tilde{T}||=o_{p}(\frac{1}{\sqrt{n}}),$

using (5.30) and $||\tilde{T}||\leq c(\epsilon_{0})\cdot\frac{1}{l_{\nu}}$
almost surely, yielding

$\frac{1}{l_{\nu}}\sum_{k_{1},k_{2}\neq\nu}{(\tilde{T}_{k_{1}k_{2}}+\tilde{D}_{k_{1}k_{2}})\sqrt{l_{k_{1}}}\Sigma_{0,k_{1}}\sqrt{l_{k_{2}}}\Sigma_{0,k_{2}}}=\frac{1}{l_{\nu}}\sum_{k_{1},k_{2}\neq\nu}{\tilde{D}_{k_{1}k_{2}}\sqrt{l_{k_{1}}}\Sigma_{0,k_{1}}\sqrt{l_{k_{2}}}\Sigma_{0,k_{2}}}+o_{p}(\frac{1}{\sqrt{n}}).$

Next, we proceed with the first stage, truncation of the series at $s:$

$|\frac{1}{l_{\nu}}\sum_{k_{1},k_{2}\neq\nu}{\tilde{D}_{k_{1}k_{2}}\sqrt{l_{k_{1}}}\Sigma_{0,k_{1}}\sqrt{l_{k_{2}}}\Sigma_{0,k_{2}}}-\frac{1}{l_{\nu}}\sum_{k_{1},k_{2}\neq\nu}{\tilde{D}_{k_{1}k_{2}}\sqrt{l_{k_{1}}}\Sigma_{s,k_{1}}\sqrt{l_{k_{2}}}\Sigma_{s,k_{2}}}|\leq$
$\leq
2||\tilde{D}||\cdot\frac{1}{l_{\nu}}\cdot\sqrt{\sum_{k\neq\nu}{l_{k}\Sigma_{s,k}^{2}}}\cdot\sqrt{\sum_{k\neq\nu}{l_{k}(\Sigma_{0,k}-\Sigma_{s,k})^{2}}}+||\tilde{D}||\cdot\frac{1}{l_{\nu}}\cdot\sum_{k\neq\nu}{l_{k}(\Sigma_{0,k}-\Sigma_{s,k})^{2}}=o_{p}(\frac{1}{\sqrt{n}}),$
(5.59)

where $\Sigma_{s,k}=\sum_{0\leq j\leq
s}{((-M_{\nu})^{j}\mathcal{R}_{\nu}\mathcal{D}_{\nu}e_{\nu})_{k}},$ using that
almost surely

$\frac{1}{\sqrt{l_{\nu}}}\cdot\sqrt{\sum_{k\neq\nu}{l_{k}\Sigma^{2}_{s,k}}}\leq
2\cdot\frac{||\Lambda^{1/2}(a_{\nu}-e_{\nu})||}{\sqrt{l_{\nu}}}+2\cdot\frac{||\Lambda^{1/2}\mathcal{R}_{\nu}\mathcal{D}_{\nu}e_{\nu}||}{\sqrt{l_{\nu}}}\leq
4\cdot\frac{||\Lambda^{1/2}\mathcal{R}_{\nu}\mathcal{D}_{\nu}e_{\nu}||}{\sqrt{l_{\nu}}}+8\beta_{\nu}^{2},$
$\sqrt{\sum_{k\neq\nu}{(\Sigma_{0,k}-\Sigma_{s,k})^{2}}}\leq
2||M_{\nu}||^{s+1}\cdot\frac{||\Lambda^{1/2}(a_{\nu}-e_{\nu})||}{\sqrt{l_{\nu}}}\leq||M_{\nu}||^{s+1}(4\cdot\frac{||\Lambda^{1/2}\mathcal{R}_{\nu}\mathcal{D}_{\nu}e_{\nu}||}{\sqrt{l_{\nu}}}+8\beta_{\nu}^{2})$

where the first inequalities in each chain are obtained arguing in the same
vein as for (5.56), the second arise from (5.29), entailing with (5.18),
(5.33), and (5.35) the bound in (5.59).

We continue with the second step, removing the first type terms in the
truncated sums:

$|\frac{1}{l_{\nu}}\sum_{k_{1},k_{2}\neq\nu}{\tilde{D}_{k_{1}k_{2}}\sqrt{l_{k_{1}}}\Sigma_{s,k_{1}}\sqrt{l_{k_{2}}}\Sigma_{s,k_{2}}}-\frac{1}{l_{\nu}}\sum_{k_{1},k_{2}\neq\nu}{\tilde{D}_{k_{1}k_{2}}\sqrt{l_{k_{1}}}\cdot\overline{\Sigma}_{s,k_{1}}\sqrt{l_{k_{2}}}\cdot\overline{\Sigma}_{s,k_{2}}}|\leq$
$\leq
2||\tilde{D}||\cdot\frac{\sqrt{\sum_{k\neq\nu}{l_{k}\overline{\Sigma}_{s,k}^{2}}}\cdot\sqrt{\sum_{k\neq\nu}{l_{k}(\Sigma_{s,k}-\overline{\Sigma}_{s,k})^{2}}}}{l_{\nu}}+||\tilde{D}||\cdot\frac{\sum_{k\neq\nu}{l_{k}(\Sigma_{s,k}-\overline{\Sigma}_{s,k})^{2}}}{l_{\nu}}=o_{p}(\frac{1}{\sqrt{n}})$

where $\overline{\Sigma}_{s,k}=\sum_{0\leq j\leq
s}{(B_{\nu}^{j}\mathcal{R}_{\nu}\mathcal{D}_{\nu}e_{\nu})_{k}},$ since

$\frac{1}{\sqrt{l_{\nu}}}\cdot\sqrt{\sum_{k\neq\nu}{l_{k}\overline{\Sigma}_{s,k}^{2}}}\leq
2\cdot\frac{||\Lambda^{1/2}\mathcal{R}_{\nu}\mathcal{D}_{\nu}e_{\nu}||}{\sqrt{l_{\nu}}},$
$\frac{1}{\sqrt{l_{\nu}}}\cdot\sqrt{\sum_{k\neq\nu}{l_{k}(\Sigma_{s,k}-\overline{\Sigma}_{s,k})^{2}}}\leq
4\cdot||A_{\nu}||\cdot\frac{||\Lambda^{1/2}\mathcal{R}_{\nu}\mathcal{D}_{\nu}e_{\nu}||}{\sqrt{l_{\nu}}}$

from

$||\sum_{0\leq j\leq
s}{\Lambda^{1/2}B_{\nu}^{j}\mathcal{R}_{\nu}\mathcal{D}_{\nu}e_{\nu}}||\leq\sum_{0\leq
j\leq
s}{||B_{\nu}||^{j}\cdot||\Lambda^{1/2}\mathcal{R}_{\nu}\mathcal{D}_{\nu}e_{\nu}||},$
$\sqrt{\sum_{k\neq\nu}{l_{k}(\Sigma_{s,k}-\overline{\Sigma}_{s,k})^{2}}}=||\sum_{1\leq
j\leq s,0\leq m\leq
j-1}{\Lambda^{1/2}(A_{\nu}+B_{\nu})^{j-1-m}A_{\nu}B_{\nu}^{m}\mathcal{R}_{\nu}\mathcal{D}_{\nu}e_{\nu}}||\leq$
$\leq\sum_{1\leq j\leq s,0\leq m\leq
j-1}{(||A_{\nu}||+||B_{\nu}||)^{j-1-m}||A_{\nu}||\cdot||B_{\nu}||^{m}\cdot||\Lambda^{1/2}\mathcal{R}_{\nu}\mathcal{D}_{\nu}e_{\nu}||}\leq$
$\leq||A_{\nu}||\cdot||\Lambda^{1/2}\mathcal{R}_{\nu}\mathcal{D}_{\nu}e_{\nu}||\sum_{1\leq
j}{\frac{j}{2^{j-1}}}=4||A_{\nu}||\cdot||\Lambda^{1/2}\mathcal{R}_{\nu}\mathcal{D}_{\nu}e_{\nu}||,$

and (5.33), $||A_{\nu}||\leq\frac{c(K,\gamma,\epsilon_{0})}{l_{\nu}}$ almost
surely. Similarly, we can replace $\mathcal{R}_{\nu}\mathcal{D}_{\nu}$ by
$\mathcal{R}_{\nu}\Lambda^{1/2}\tilde{D}\Lambda^{1/2}$ in
$\overline{\Sigma}_{s,k},$ and we are left with

$\frac{1}{l_{\nu}}\sum_{k_{1},k_{2}\neq\nu}{\tilde{D}_{k_{1}k_{2}}\sqrt{l_{k_{1}}}\sum_{0\leq
j\leq
s}{((C_{\nu}+(\frac{\hat{l}_{\nu}}{l_{\nu}}-1)\tilde{R})^{j}C_{\nu})_{k_{1}\nu}}\sqrt{l_{k_{2}}}\sum_{0\leq
j\leq
s}{(C_{\nu}+(\frac{\hat{l}_{\nu}}{l_{\nu}}-1)\tilde{R})^{j}C_{\nu})_{k_{2}\nu}}}$
(5.60)

for $C_{\nu}=-\mathcal{R}_{\nu}\Lambda^{1/2}\tilde{D}\Lambda^{1/2}.$

Consider first the terms with no factor of $\frac{\hat{l}_{\nu}}{l_{\nu}}-1:$

$\frac{1}{l_{\nu}}\sum_{k_{1},k_{2}\neq\nu}{\tilde{D}_{k_{1}k_{2}}\sqrt{l_{k_{1}}}\sum_{0\leq
j\leq s}{(C_{\nu}^{j+1})_{k_{1}\nu}}\sqrt{l_{k_{2}}}\sum_{0\leq j\leq
s}{(C_{\nu}^{j+1})_{k_{2}\nu}}}=$ $=\sum_{0\leq j_{1},j_{2}\leq
s}{\frac{l_{k_{1}}l_{k_{2}}l_{\kappa_{1}}...l_{\kappa_{j_{1}+j_{2}}}}{(l_{\nu}-l_{k_{1}})(l_{\nu}-l_{k_{2}})(l_{\nu}-l_{\kappa_{1}})...(l_{\nu}-l_{\kappa_{j_{1}+j_{2}}})}\tilde{D}_{k_{1}\kappa_{1}}...\tilde{D}_{\kappa_{j_{1}}\nu}\tilde{D}_{k_{2}\kappa_{j_{1}+1}}...\tilde{D}_{\kappa_{j_{1}+j_{2}}\nu}}=$
$=\sum_{0\leq j_{1},j_{2}\leq
s}{\frac{1}{n^{j_{1}+j_{2}+2}}\cdot\frac{l_{k_{1}}l_{k_{2}}l_{\kappa_{1}}...l_{\kappa_{j_{1}+j_{2}}}}{(l_{\nu}-l_{k_{1}})(l_{\nu}-l_{k_{2}})(l_{\nu}-l_{\kappa_{1}})...(l_{\nu}-l_{\kappa_{j_{1}+j_{2}}})}}+o_{p}(\frac{1}{\sqrt{n}}),$

where again a change of summation is used, from
$(k_{1},k_{2},\kappa_{1},\hskip 1.42271pt...\hskip
1.42271pt,\kappa_{j_{1}+j_{2}})$ to $(\kappa_{j_{1}},\hskip 1.42271pt...\hskip
1.42271pt,\kappa_{1},k_{1},k_{2},\kappa_{j_{1}+1},\hskip 1.42271pt...\hskip
1.42271pt,\kappa_{j_{1}+j_{2}}),$ and the analysis of the error for
$\tilde{\Sigma}_{0},$ with the first term coming from the second component in
(2.6) and

$\sum_{0\leq j_{1},j_{2}\leq
s}{\frac{1}{n^{j_{1}+j_{2}+2}}\cdot\frac{l_{k_{1}}l_{k_{2}}l_{\kappa_{1}}...l_{\kappa_{j_{1}+j_{2}}}}{(l_{\nu}-l_{k_{1}})(l_{\nu}-l_{k_{2}})(l_{\nu}-l_{\kappa_{1}})...(l_{\nu}-l_{\kappa_{j_{1}+j_{2}}})}}=$
$=\sum_{2\leq j\leq
2s+2}{\min{(j-1,s+1)\cdot(\frac{1}{n}\sum_{k\neq\nu}{\frac{l_{k}}{l_{k}-l_{\nu}}})^{j}}}=O(c(\epsilon_{0})\cdot\frac{M}{n}),$

as each $j=(j_{1}+j_{2}+2)$-tuple appears $\min{(j-1,s+1)}$ times (there is a
bijection between these appearances and the solutions $(j_{1},j_{2})$ of
$j=j_{1}+j_{2}+2,0\leq j_{1},j_{2}\leq s$).

Lastly, consider the terms containing a factor of
$(\frac{\hat{l}_{\nu}}{l_{\nu}}-1)^{m}$ for $1\leq m\leq 2s:$

$\sum{\frac{l_{k_{1}}l_{k_{2}}l_{\kappa_{1}}...l_{\kappa_{j_{1}+j_{2}-m_{1}-m_{2}}}}{(l_{\nu}-l_{k_{1}})(l_{\nu}-l_{k_{2}})(l_{\nu}-l_{\kappa_{1}})...(l_{\nu}-l_{\kappa_{j_{1}+j_{2}-m_{1}-m_{2}}})}\cdot(\frac{l_{\nu}}{l_{\kappa_{\overline{j}_{1}}}})^{\overline{j}_{2}}...(\frac{l_{\nu}}{l_{\kappa_{2\overline{j}_{2l_{1}+2l_{2}-1}}}})^{\overline{j}_{2l_{1}+2l_{2}}}\cdot}$
$\cdot\tilde{D}_{k_{1}\kappa_{1}}...\tilde{D}_{\kappa_{j_{1}-m_{1}}\nu}\tilde{D}_{k_{2}\kappa_{j_{1}-m_{1}+1}}...\tilde{D}_{\kappa_{j_{1}-m_{1}+j_{2}-m_{2}}\nu},$
(5.61)

where the summation is over
$(j_{1},j_{2},m_{1},m_{2},k_{1},k_{2},\kappa_{1},\hskip 1.42271pt...\hskip
1.42271pt,\kappa_{j_{1}+j_{2}-m_{1}-m_{2}},\overline{j}_{1},\hskip
1.42271pt...\hskip 1.42271pt,\overline{j}_{2l_{1}+2l_{2}})$ with

$k_{1},k_{2}\neq\nu,\kappa_{i}\neq\nu,m=m_{1}+m_{2},\overline{j}_{1}+...+\overline{j}_{2l_{1}-1}=j_{1}-m_{1},\overline{j}_{2}+...+\overline{j}_{2l_{1}}=m_{1},$
$\overline{j}_{2l_{1}+1}+...+\overline{j}_{2l_{1}+2l_{2}-1}=j_{2}-m_{2},\overline{j}_{2l_{1}+2}+...+\overline{j}_{2l_{1}+2l_{2}}=m_{2},$
$\overline{j}_{1}\geq 0,\overline{j}_{2},\hskip 1.42271pt...\hskip
1.42271pt,\overline{j}_{2l_{1}-1}>0,\overline{j}_{2l_{1}}\geq
0,\overline{j}_{2l_{1}+1}\geq 0,\overline{j}_{2l_{1}+2},\hskip
1.42271pt...\hskip
1.42271pt,\overline{j}_{2l_{1}+2l_{2}-1}>0,\overline{j}_{2l_{1}+2l_{2}}\geq
0.$

Proceeding as in the case $m>0$ in subsection 5.6, it follows from (5.60) that
(5.61) is

$\sum{\frac{1}{n^{j_{1}+j_{2}-m+2}}\cdot\frac{l_{k_{1}}l_{k_{2}}l_{\kappa_{1}}...l_{\kappa_{j_{1}+j_{2}-m_{1}-m_{2}}}}{(l_{\nu}-l_{k_{1}})(l_{\nu}-l_{k_{2}})(l_{\nu}-l_{\kappa_{1}})...(l_{\nu}-l_{\kappa_{j_{1}+j_{2}-m_{1}-m_{2}}})}\cdot(\frac{l_{\nu}}{l_{\kappa_{\overline{j}_{1}}}})^{\overline{j}_{2}}...(\frac{l_{\nu}}{l_{\kappa_{2\overline{j}_{2l_{1}+2l_{2}-1}}}})^{\overline{j}_{2l_{1}+2l_{2}}}}+\tilde{S}(m),$

with the first term $c^{\prime}_{m}$ coming from the second sum in (2.6) and

$|c^{\prime}_{m}|\leq c(\epsilon_{0})^{m+1}\cdot\frac{M}{n},\hskip
5.69046pt\sum_{1\leq m\leq 2s}{\mathbb{E}[n(\tilde{S}(m))^{2}]}=o(1).$

###  5.10 Proof of Theorem 2

Recall (5.11) and (5.10): $x=x_{n,\nu,(l_{i})_{1\leq i\leq
M}}\in\mathbb{R},|x|\leq 2c(\epsilon_{0})\cdot\frac{M}{n}$ satisfies

$x=\overline{O}+\sum_{1\leq j\leq s}{x^{j}\overline{O}_{j}},$

where

$-\tilde{\Sigma}_{3}-2(1-\tilde{\Sigma}_{3})\tilde{\Sigma}_{1}+(1-\tilde{\Sigma}_{3})\tilde{\Sigma}_{2}=\overline{O}+\sum_{1\leq
j\leq
s}{(\frac{\hat{l}_{\nu}}{l_{\nu}}-1)^{j}\overline{O}_{j}}+o_{p}(\frac{1}{\sqrt{n}}).$

(5.20) with (5.23), (5.24), (5.25) then yield

$\sum_{0\leq j\leq s}{(\sum_{0\leq i\leq
2s}{(2a_{i}+b_{i}+c_{i})z^{i}})(\sum_{0\leq i\leq
2s}{b_{i}z^{i}})^{j}}=\overline{O}+\sum_{1\leq j\leq
2s^{2}+2s}{z^{j}\overline{O}_{j}}.$

##  6 Switching between Empirical and Deterministic Centering

Subsection 6.1 presents the proof of Theorem 3, and subsection 6.2 contains
the justification of Theorem 4. Both results build on the CLT given by Theorem
1, whose description is complemented by Theorem 2, and as such their proofs
are concerned only with altering its centering.

###  6.1 Proof of Theorem 3

Theorem 2 gives

$x=2a_{0}+b_{0}+c_{0}+o(\frac{1}{\sqrt{n}})$

since $O(\frac{M^{2}}{n^{2}})=o(\frac{1}{\sqrt{n}}),$ and thus

$a_{0}=\frac{1}{n}\sum_{k\neq\nu}{\frac{l_{k}}{l_{\nu}-l_{k}}}+o(\frac{1}{\sqrt{n}}),\hskip
2.84544ptb_{0}=-\frac{1}{n}\sum_{k\neq\nu}{\frac{l_{k}l_{\nu}}{(l_{\nu}-l_{k})^{2}}}+o(\frac{1}{\sqrt{n}}),\hskip
2.84544ptc_{0}=\frac{1}{n}\sum_{k\neq\nu}{\frac{l_{k}}{l_{\nu}-l_{k}}}+\frac{1}{n}\sum_{k\neq\nu}{\frac{l_{k}^{2}}{(l_{\nu}-l_{k})^{2}}}+o(\frac{1}{\sqrt{n}}),$

from subsections $\ref{4.7},\ref{4.8},\ref{4.9},$ from which

$x=2a_{0}+b_{0}+c_{0}+o(\frac{1}{\sqrt{n}})=\frac{1}{n}\sum_{k\neq\nu}{\frac{l_{k}}{l_{\nu}-l_{k}}}+o(\frac{1}{\sqrt{n}}).$

Then by employing Theorem 1, it suffices to prove that

$\frac{1}{\sqrt{n}}\cdot(\sum_{k\neq\nu}{\frac{l_{k}}{l_{\nu}-l_{k}}}-\sum_{k\neq\nu}{\frac{\hat{l}_{k}}{\hat{l}_{\nu}-\hat{l}_{k}}})\xrightarrow[]{p}0.$
(6.1)

We show first that almost surely, $l_{k}\neq\hat{l}_{\nu}$ and
$\hat{l}_{k}\neq\hat{l}_{\nu}$ for all $k\neq\nu.$ As
$\frac{\hat{l}_{\nu}}{l_{\nu}}\xrightarrow[]{a.s.}1,$ the first claim follows
from the separability assumption. For $k<\nu,$ the second claim is clear
because $\frac{\hat{l}_{k}}{l_{k}}\xrightarrow[]{a.s.}1$ from Proposition 1.
For $k>\nu,$ if $\hat{l}_{\nu}=\hat{l}_{k},$ then

$|\frac{\hat{l}_{k}}{l_{k}}-1|\geq\frac{\epsilon_{0}}{(1+\epsilon_{0})(2+\epsilon_{0})}\cdot\frac{l_{\nu}}{l_{k}}$
(6.2)

or

$|\frac{\hat{l}_{\nu}}{l_{\nu}}-1|\geq\frac{\epsilon_{0}}{2+\epsilon_{0}}:$
(6.3)

from

$\frac{\hat{l}_{k}}{l_{k}}-1=\frac{\hat{l}_{\nu}}{l_{\nu}}\cdot(\frac{l_{\nu}}{l_{k}}-1)+(\frac{\hat{l}_{\nu}}{l_{\nu}}-1),$

if $|\frac{\hat{l}_{\nu}}{l_{\nu}}-1|<\frac{\epsilon_{0}}{2+\epsilon_{0}},$
then

$\frac{\hat{l}_{k}}{l_{k}}-1\geq\frac{2}{2+\epsilon_{0}}\cdot(\frac{l_{\nu}}{l_{k}}-1)-\frac{\epsilon_{0}}{2+\epsilon_{0}}\geq\frac{2}{2+\epsilon_{0}}\cdot\frac{\epsilon_{0}}{1+\epsilon_{0}}\cdot\frac{l_{\nu}}{l_{k}}-\frac{\epsilon_{0}}{2+\epsilon_{0}}\geq\frac{\epsilon_{0}}{(1+\epsilon_{0})(2+\epsilon_{0})}\cdot\frac{l_{\nu}}{l_{k}}.$

Proposition 1 yields that (6.3) occurs with probability zero with the same
conclusion ensuing for (6.2) from a union bound, Borel-Cantelli lemma, and the
following inequality holding for all $k>\nu,\newline
c_{1}(\epsilon_{0})=\frac{\epsilon_{0}}{(1+\epsilon_{0})(2+\epsilon_{0})},$
and $n\geq n(\epsilon_{0}),$

$\mathbb{P}(|\frac{\hat{l}_{k}}{l_{k}}-1|\geq
c_{1}(\epsilon_{0})\cdot\frac{l_{\nu}}{l_{k}})\leq 6\exp(-\sqrt{n}):$

(4.6), (4.4) give for $t>0$

$\mathbb{P}(|\frac{\hat{l}_{k}}{l_{k}}-1|\geq
t)\leq\mathbb{P}(\lambda_{1}(\mathcal{B})\geq\frac{tl_{k}}{2})+\mathbb{P}(|\frac{\lambda_{k}(\mathcal{A})}{l_{k}}-1|\geq\frac{t}{2}),$
$\mathbb{P}(|\frac{\hat{l}_{k}}{l_{k}}-1|\geq
t)\leq\mathbb{P}(\lambda_{1}(\mathcal{B})\geq\frac{tl_{k}}{2})+\mathbb{P}(|\lambda_{k}(\frac{1}{n}\sum_{1\leq
i\leq
k}{Z_{i}^{T}Z_{i}})-1|\geq\frac{t}{2})+\mathbb{P}(|\lambda_{1}(\frac{1}{n}\sum_{k\leq
i\leq M}{Z_{i}^{T}Z_{i}})-1|\geq\frac{t}{2}),$ (6.4)

which in conjunction with $\frac{l_{\nu}}{l_{k}}\geq 1$ implies

$\mathbb{P}(|\frac{\hat{l}_{k}}{l_{k}}-1|\geq
c_{1}(\epsilon_{0})\cdot\frac{l_{\nu}}{l_{k}})\leq\mathbb{P}(\lambda_{1}(\mathcal{B})\geq\frac{c_{1}(\epsilon_{0})}{2}\cdot
l_{\nu})+\mathbb{P}(|\lambda_{1}(\frac{1}{n}\sum_{1\leq i\leq
k}{Z_{i}^{T}Z_{i}})-1|\geq\frac{c_{1}(\epsilon_{0})}{2})+$
$+\mathbb{P}(|\lambda_{1}(\frac{1}{n}\sum_{k\leq i\leq
M}{Z_{i}^{T}Z_{i}})-1|\geq\frac{c_{1}(\epsilon_{0})}{2})\leq\mathbb{P}(\lambda_{1}(\mathcal{B})\geq
2c_{K,\gamma})+4\exp(-\sqrt{n})\leq 6\exp(-\sqrt{n})$

because from inequality (SM) with probability at least $1-2\exp(-\sqrt{n}),$

$(1-cK^{2}(\sqrt{\frac{k}{n}}+n^{-1/4}))^{2}\leq\lambda_{1}(\frac{1}{n}\sum_{1\leq
i\leq k}{Z_{i}^{T}Z_{i}})\leq(1+cK^{2}(\sqrt{\frac{k}{n}}+n^{-1/4}))^{2},$

from which for $n\geq n(\epsilon_{0}),$

$|\lambda_{1}(\frac{1}{n}\sum_{1\leq i\leq k}{Z_{i}^{T}Z_{i}})-1|\leq
3cK^{2}(\sqrt{\frac{k}{n}}+n^{-1/4})\leq
3cK^{2}(\sqrt{\frac{M}{n}}+n^{-1/4})\leq\frac{c_{1}(\epsilon_{0})}{4},$ (6.5)

and similarly

$\mathbb{P}(|\lambda_{1}(\frac{1}{n}\sum_{k\leq i\leq
M}{Z_{i}^{T}Z_{i}})-1|\geq\frac{c_{1}(\epsilon_{0})}{2})\leq
2\exp(-\sqrt{n}).$

Next, to obtain (6.1) (whose left-hand side term is well-defined almost surely
from the previous paragraph), we show

$\frac{1}{\sqrt{n}}\cdot(\sum_{k\neq\nu}{\frac{l_{k}}{l_{k}-l_{\nu}}}-\sum_{k\neq\nu}{\frac{l_{k}}{l_{k}-\hat{l}_{\nu}}})\xrightarrow[]{p}0,$
(6.6)
$\frac{1}{\sqrt{n}}\cdot(\sum_{k\neq\nu}{\frac{l_{k}}{l_{k}-\hat{l}_{\nu}}}-\sum_{k\neq\nu}{\frac{\hat{l}_{k}}{\hat{l}_{k}-\hat{l}_{\nu}}})\xrightarrow[]{p}0.$
(6.7)

We begin with (6.6):

$\frac{l_{k}}{l_{k}-l_{\nu}}-\frac{l_{k}}{l_{k}-\hat{l}_{\nu}}=-\frac{l_{k}(\hat{l}_{\nu}-l_{\nu})}{(l_{k}-l_{\nu})(l_{k}-\hat{l}_{\nu})}=\frac{\frac{l_{k}}{l_{\nu}}\cdot(\frac{\hat{l}_{\nu}}{l_{\nu}}-1)}{(\frac{l_{k}}{l_{\nu}}-1)(\frac{\hat{l}_{\nu}}{l_{\nu}}-\frac{l_{k}}{l_{\nu}})},$

from which Proposition 1 and the separability assumption give that almost
surely

$|\frac{l_{k}}{l_{k}-l_{\nu}}-\frac{l_{k}}{l_{k}-\hat{l}_{\nu}}|\leq
c(\epsilon_{0})\cdot|\frac{\hat{l}_{\nu}}{l_{\nu}}-1|.$

Since

$\frac{M}{\sqrt{n}}\cdot|\frac{\hat{l}_{\nu}}{l_{\nu}}-1|\leq\frac{M}{\sqrt{n}}\cdot\frac{1}{n}tr(\mathcal{M}(\hat{l}_{\nu}I-\mathcal{M})^{-1})+\frac{M}{\sqrt{n}}\cdot\frac{1}{n}|\sum_{k\neq\nu}{\frac{l_{k}}{l_{k}-l_{\nu}}}|+$
$+\frac{M}{n}\cdot\sqrt{n}\cdot|\frac{\hat{l}_{\nu}}{l_{\nu}}-1-\frac{1}{n}tr(\mathcal{M}(\hat{l}_{\nu}I-\mathcal{M})^{-1})-\frac{1}{n}\sum_{k\neq\nu}{\frac{l_{k}}{l_{\nu}-l_{k}}}|\xrightarrow[]{p}0$

from

$\frac{M}{\sqrt{n}}\cdot\frac{1}{n}|tr(\mathcal{M}(\hat{l}_{\nu}I-\mathcal{M})^{-1})|\leq
c(K,\gamma)\cdot\frac{M/\sqrt{n}}{l_{\nu}}\to 0,\hskip
5.69046pt\frac{M}{\sqrt{n}}\cdot\frac{M}{n}=\frac{M^{2}}{n^{3/2}}\to 0,\hskip
5.69046pt\frac{M}{n}\to 0,$

and Theorem 1, (6.6) follows.

We continue with (6.7):

$\frac{l_{k}}{l_{k}-\hat{l}_{\nu}}-\frac{\hat{l}_{k}}{\hat{l}_{k}-\hat{l}_{\nu}}=\frac{\hat{l}_{\nu}(\hat{l}_{k}-l_{k})}{(l_{k}-\hat{l}_{\nu})(\hat{l}_{k}-\hat{l}_{\nu})}=(\frac{\hat{l}_{k}}{l_{k}}-1)\cdot\frac{\hat{l}_{\nu}l_{k}}{(l_{k}-\hat{l}_{\nu})(\hat{l}_{k}-\hat{l}_{\nu})},$
(6.8)

and prove that with probability one

$|\frac{\hat{l}_{\nu}l_{k}}{(l_{k}-\hat{l}_{\nu})(\hat{l}_{k}-\hat{l}_{\nu})}|=|\frac{\frac{\hat{l}_{\nu}}{l_{\nu}}\cdot\frac{l_{k}}{l_{\nu}}}{(\frac{l_{k}}{l_{\nu}}-\frac{\hat{l}_{\nu}}{l_{\nu}})\cdot(\frac{l_{k}}{l_{\nu}}\cdot\frac{\hat{l}_{k}}{l_{k}}-\frac{\hat{l}_{\nu}}{l_{\nu}})}|\leq
c(\epsilon_{0},K,\gamma)\cdot\min{(\frac{l_{k}}{l_{\nu}},\frac{l_{\nu}}{l_{k}})}.$
(6.9)

If $l_{k}\geq\frac{l_{\nu}}{4(c_{K,\gamma}+1)},$ Proposition 1 yields
$\frac{\hat{l}_{k}}{l_{k}}\xrightarrow[]{a.s.}1,\frac{\hat{l}_{\nu}}{l_{\nu}}\xrightarrow[]{a.s.}1$
jointly for all such $k\neq\nu,$ and so (6.9) follows from

$|\frac{\frac{\hat{l}_{\nu}}{l_{\nu}}}{(\frac{l_{k}}{l_{\nu}}-\frac{\hat{l}_{\nu}}{l_{\nu}})\cdot(\frac{l_{k}}{l_{\nu}}\cdot\frac{\hat{l}_{k}}{l_{k}}-\frac{\hat{l}_{\nu}}{l_{\nu}})}|\leq
c(\epsilon_{0},K,\gamma),\hskip
5.69046pt|\frac{\frac{\hat{l}_{\nu}}{l_{\nu}}\cdot(\frac{l_{k}}{l_{\nu}})^{2}}{(\frac{l_{k}}{l_{\nu}}-\frac{\hat{l}_{\nu}}{l_{\nu}})\cdot(\frac{l_{k}}{l_{\nu}}\cdot\frac{\hat{l}_{k}}{l_{k}}-\frac{\hat{l}_{\nu}}{l_{\nu}})}|\leq
c(\epsilon_{0},K,\gamma).$

If $l_{k}\leq\frac{l_{\nu}}{4(c_{K,\gamma}+1)},$ then almost surely for all
such $k,$ $\frac{\hat{l}_{k}}{l_{k}}\leq 2c_{K,\gamma}+2$ from (6.4)
($\frac{(2c_{K,\gamma}+1)l_{k}}{2}\geq\frac{2c_{K,\gamma}+1}{2}>c_{K,\gamma}$).
Hence with probability one

$|\frac{\frac{\hat{l}_{\nu}}{l_{\nu}}}{(\frac{l_{k}}{l_{\nu}}-\frac{\hat{l}_{\nu}}{l_{\nu}})\cdot(\frac{l_{k}}{l_{\nu}}\cdot\frac{\hat{l}_{k}}{l_{k}}-\frac{\hat{l}_{\nu}}{l_{\nu}})}|\leq\frac{2}{(\frac{1}{1+\epsilon_{0}/2}-\frac{1}{1+\epsilon_{0}})\cdot\frac{1}{6}},$

from

$\frac{l_{k}}{l_{\nu}}\cdot\frac{\hat{l}_{k}}{l_{k}}-\frac{\hat{l}_{\nu}}{l_{\nu}}\leq\frac{1}{2}-\frac{2}{3}=-\frac{1}{6},\hskip
11.38092pt\frac{l_{k}}{l_{\nu}}-\frac{\hat{l}_{\nu}}{l_{\nu}}\leq\frac{1}{1+\epsilon_{0}}-\frac{1}{1+\epsilon_{0}/2}$

concluding the proof of (6.9).

Now we return to (6.7), and show that

$\mathbb{P}(|\frac{\hat{l}_{k}}{l_{k}}-1|\cdot\min{(\frac{l_{k}}{l_{\nu}},\frac{l_{\nu}}{l_{k}})}\geq\frac{t\sqrt{n}}{M})\leq
6\exp(-\sqrt{n}),$ (6.10)

for all $t>0,n\geq n(t):$ if $l_{k}\geq l_{\nu},$ then (6.4) gives

$\mathbb{P}(|\frac{\hat{l}_{k}}{l_{k}}-1|\geq\frac{tl_{k}\sqrt{n}}{Ml_{\nu}})\leq\mathbb{P}(|\frac{\hat{l}_{k}}{l_{k}}-1|\geq\frac{t\sqrt{n}}{M})\leq\mathbb{P}(\lambda_{1}(\mathcal{B})\geq\frac{tl_{k}\sqrt{n}}{2M})+\mathbb{P}(|\lambda_{1}(\frac{1}{n}\sum_{1\leq
i\leq k}{Z_{i}^{T}Z_{i}})-1|\geq\frac{t\sqrt{n}}{2M})+$
$+\mathbb{P}(|\lambda_{1}(\frac{1}{n}\sum_{k\leq i\leq
M}{Z_{i}^{T}Z_{i}})-1|\geq\frac{t\sqrt{n}}{2M})\leq\mathbb{P}(\lambda_{1}(\mathcal{B})\geq
2c_{K,\gamma})+4\exp(-\sqrt{n})\leq 6\exp(-\sqrt{n}),$

using (6.5) and

$(\frac{1}{n^{1/4}}+\sqrt{\frac{M}{n}})\cdot\frac{M}{\sqrt{n}}=\frac{M}{n^{3/4}}+\frac{M^{3/2}}{n}\to
0;$

if $l_{k}<l_{\nu},$ then

$\mathbb{P}(\frac{l_{k}}{l_{\nu}}\cdot|\frac{\hat{l}_{k}}{l_{k}}-1|\geq\frac{t\sqrt{n}}{M})\leq\mathbb{P}(\lambda_{1}(\mathcal{B})\geq\frac{tl_{\nu}\sqrt{n}}{2M})+\mathbb{P}(|\lambda_{1}(\frac{1}{n}\sum_{1\leq
i\leq k}{Z_{i}^{T}Z_{i}})-1|\geq\frac{t\sqrt{n}}{M})+$
$+\mathbb{P}(|\lambda_{1}(\frac{1}{n}\sum_{k\leq i\leq
M}{Z_{i}^{T}Z_{i}})-1|\geq\frac{t\sqrt{n}}{M})\leq\mathbb{P}(\lambda_{1}(\mathcal{B})\geq
2c_{K,\gamma})+4\exp(-\sqrt{n})\leq 6\exp(-\sqrt{n}).$

Finally, from (6.8), (6.9), (6.10), with probability one

$\frac{1}{\sqrt{n}}\cdot|\sum_{k\neq\nu}{\frac{l_{k}}{l_{k}-\hat{l}_{\nu}}}-\sum_{k\neq\nu}{\frac{\hat{l}_{k}}{\hat{l}_{k}-\hat{l}_{\nu}}}|\leq\frac{c(\epsilon_{0},K,\gamma)}{\sqrt{n}}\sum_{k\neq\nu}{|\frac{\hat{l}_{k}}{l_{k}}-1|\cdot\min{(\frac{l_{k}}{l_{\nu}},\frac{l_{\nu}}{l_{k}})}},$

and for any $t>0,n\geq n(t)$

$\sum_{k\neq\nu}{\mathbb{P}(|\frac{\hat{l}_{k}}{l_{k}}-1|\cdot\min{(\frac{l_{k}}{l_{\nu}},\frac{l_{\nu}}{l_{k}})}\geq\frac{t\sqrt{n}}{c(\epsilon_{0},K,\gamma)M})}\leq
6M\exp(-\sqrt{n}),$

yielding (6.7).

###  6.2 Proof of Theorem 4

Denote by

$\overline{l}_{\nu}=l_{\nu}(1+\frac{N-M}{n}\cdot\frac{1}{l_{\nu}-1}).$ (6.11)

The conclusion can be derived from the following convergence

$\sqrt{n}\cdot(\frac{1}{l_{\nu}-1}-\frac{1}{N-M}tr(\mathcal{M}(\overline{l}_{\nu}I-\mathcal{M})^{-1}))\xrightarrow[]{p}0:$
(6.12)

let us see first how it leads to the desired result and second why it is
correct.

First, suppose (6.12) holds. Theorem 1 gives

$\sqrt{\frac{n}{\mathbb{E}[(z^{(n)}_{11})^{4}]-1}}\cdot(\frac{\hat{l}_{\nu}}{l_{\nu}}-1-\frac{1}{n}tr(\mathcal{M}(\hat{l}_{\nu}I-\mathcal{M})^{-1})-x_{n,\nu,(l_{i})_{1\leq
i\leq M}})\Rightarrow N(0,1),$

and therefore, from Slutsky’s lemma, it suffices to prove that

$\sqrt{n}\cdot(\frac{1}{n}tr(\mathcal{M}(\hat{l}_{\nu}I-\mathcal{M})^{-1})-\frac{N-M}{n}\cdot\frac{1}{l_{\nu}-1})\xrightarrow[]{p}0,$

or equivalently

$\sqrt{n}\cdot(\frac{1}{N-M}tr(\mathcal{M}(\hat{l}_{\nu}I-\mathcal{M})^{-1})-\frac{1}{l_{\nu}-1})\xrightarrow[]{p}0,$

which in turn can be rewritten, employing (6.12), as

$\sqrt{n}\cdot(\frac{1}{N-M}tr(\mathcal{M}(\hat{l}_{\nu}I-\mathcal{M})^{-1})-\frac{1}{N-M}tr(\mathcal{M}(\overline{l}_{\nu}I-\mathcal{M})^{-1}))\xrightarrow[]{p}0.$
(6.13)

Take $x_{n}\in\mathbb{R}$ such that

$l_{\nu}=\frac{\hat{l}_{\nu}}{1+\frac{1}{n}tr(\mathcal{M}(\hat{l}_{\nu}I-\mathcal{M})^{-1})+x_{n,\nu,(l_{i})_{1\leq
i\leq M}}+\frac{x_{n}}{\sqrt{n}}},$ (6.14)

and from Theorem 3,

$\frac{x_{n}}{\sqrt{\mathbb{E}[z_{11}^{4}]-1}}\Rightarrow N(0,1).$

(6.12) yields

$\frac{\sqrt{n}}{l_{\nu}}\cdot(\frac{\overline{l}_{\nu}}{1+\frac{1}{n}tr(\mathcal{M}(\overline{l}_{\nu}I-\mathcal{M})^{-1})}-l_{\nu})=\frac{N-M}{n}\cdot\frac{\sqrt{n}\cdot(\frac{1}{l_{\nu}-1}-\frac{1}{N-M}tr(\mathcal{M}(\overline{l}_{\nu}I-\mathcal{M})^{-1}))}{1+\frac{1}{n}tr(\mathcal{M}(\overline{l}_{\nu}I-\mathcal{M})^{-1})}\xrightarrow[]{p}0,$
(6.15)

since
$\frac{1}{n}tr(\mathcal{M}(\overline{l}_{\nu}I-\mathcal{M})^{-1})\xrightarrow[]{a.s.}0;$
rewrite (6.15) using (6.14) as

$\frac{\sqrt{n}}{l_{\nu}}\cdot(\frac{\overline{l}_{\nu}}{1+\frac{1}{n}tr(\mathcal{M}(\overline{l}_{\nu}I-\mathcal{M})^{-1})}-\frac{\hat{l}_{\nu}}{1+\frac{1}{n}tr(\mathcal{M}(\hat{l}_{\nu}I-\mathcal{M})^{-1})+x_{n,\nu,(l_{i})_{1\leq
i\leq M}}+\frac{x_{n}}{\sqrt{n}}})\xrightarrow[]{p}0,$

from which

$y_{n}:=\frac{\sqrt{n}}{l_{\nu}}\cdot(\frac{\overline{l}_{\nu}}{1+\frac{1}{n}tr(\mathcal{M}(\overline{l}_{\nu}I-\mathcal{M})^{-1})}-\frac{\hat{l}_{\nu}}{1+\frac{1}{n}tr(\mathcal{M}(\hat{l}_{\nu}I-\mathcal{M})^{-1})})+$
$+\frac{\hat{l}_{\nu}}{l_{\nu}}\cdot\frac{\sqrt{n}x_{n,\nu,(l_{i})_{1\leq
i\leq
M}}+x_{n}}{(1+\frac{1}{n}tr(\mathcal{M}(\hat{l}_{\nu}I-\mathcal{M})^{-1}))\cdot(1+\frac{1}{n}tr(\mathcal{M}(\hat{l}_{\nu}I-\mathcal{M})^{-1})+x_{n,\nu,(l_{i})_{1\leq
i\leq M}}+\frac{x_{n}}{\sqrt{n}})}\xrightarrow[]{p}0.$

Note that for

$f(x)=\frac{x}{1+\frac{1}{n}tr(\mathcal{M}(xI-\mathcal{M})^{-1})},$

and $x>c(\gamma,K)\cdot||\mathcal{M}||,$

$f^{\prime}(x)=\frac{1+\frac{1}{n}tr(\mathcal{M}(xI-\mathcal{M})^{-1})+x\cdot\frac{1}{n}tr(\mathcal{M}(xI-\mathcal{M})^{-2})}{(1+\frac{1}{n}tr(\mathcal{M}(xI-\mathcal{M})^{-1}))^{2}}=\frac{1+\frac{1}{n}tr((2xI-\mathcal{M})\mathcal{M}(xI-\mathcal{M})^{-2})}{(1+\frac{1}{n}tr(\mathcal{M}(xI-\mathcal{M})^{-1}))^{2}}\geq\frac{1}{2},$

from which, almost surely, using Proposition 1 and the mean-value theorem,

$\frac{\sqrt{n}\cdot|\overline{l}_{\nu}-\hat{l}_{\nu}|}{l_{\nu}^{2}}\leq\frac{2|y_{n}|}{l_{\nu}}+\frac{4}{l_{\nu}}\cdot\frac{\sqrt{n}\cdot|x_{n,\nu,(l_{i})_{1\leq
i\leq
M}}|+|x_{n}|}{(1+\frac{1}{n}tr(\mathcal{M}(\hat{l}_{\nu}I-\mathcal{M})^{-1}))\cdot(1+\frac{1}{n}tr(\mathcal{M}(\hat{l}_{\nu}I-\mathcal{M})^{-1})-\frac{|x_{n}|}{\sqrt{n}}-c(\epsilon_{0})\cdot\frac{M}{n})}\xrightarrow[]{p}0$
(6.16)

because
$\frac{1}{n}tr(\mathcal{M}(\hat{l}_{\nu}I-\mathcal{M})^{-1})\xrightarrow[]{a.s.}0,$

$y_{n}\xrightarrow[]{p}0,\hskip
11.38092pt\frac{\sqrt{n}\cdot|x_{n,\nu,(l_{i})_{1\leq i\leq M}}|}{l_{\nu}}\leq
c(\epsilon_{0})\cdot\frac{M/\sqrt{n}}{l_{\nu}}\to 0,\hskip
11.38092pt\frac{x_{n}}{\sqrt{\mathbb{E}[z_{11}^{4}]-1}}\Rightarrow
N(0,1),\hskip 11.38092pt\mathbb{E}[z_{11}^{4}]-1\leq c(K),\hskip
11.38092ptl_{\nu}\to\infty,$

Finally, note (6.13) ensues from (6.16) since almost surely for $n$ large
enough, $\newline ||\mathcal{M}||\leq
c_{K,\gamma}\leq\frac{l_{\nu}}{2},\overline{l}_{\nu},\hat{l}_{\nu}\geq\frac{l_{\nu}}{2},$
and for $0\leq x\leq c_{K,\gamma}\leq\frac{l_{\nu}}{2},$

$|\frac{x}{\hat{l}_{\nu}-x}-\frac{x}{\overline{l}_{\nu}-x}|\leq\frac{4c_{K,\gamma}\cdot|\overline{l}_{\nu}-\hat{l}_{\nu}|}{l_{\nu}^{2}}.$

Second, we proceed with the proof of (6.12), which is equivalent to

$\sqrt{n}\cdot(\frac{l_{\nu}}{l_{\nu}-1}-\overline{l}_{\nu}\cdot\frac{1}{N-M}tr((\overline{l}_{\nu}I-\mathcal{M})^{-1}))\xrightarrow[]{p}0.$
(6.17)

Denote by

$m_{n}(z)=\frac{z+\gamma_{n}-1-\sqrt{(z-\gamma_{n}+1)^{2}-4z}}{2\gamma_{n}z}$

for $z\neq 0,\gamma_{n}=\frac{N-M}{n}$ (the Stieltjes transform of a
Marchenko-Pastur law), for which

$m_{n}(\overline{l}_{\nu})=\frac{l_{\nu}}{l_{\nu}-1}\cdot\frac{1}{\overline{l}_{\nu}}$

in virtue of

$(\overline{l}_{\nu}-\gamma_{n}+1)^{2}-4\overline{l}_{\nu}=(l_{\nu}+1+\frac{\gamma_{n}}{l_{\nu}-1})^{2}-4l_{\nu}(1+\frac{\gamma_{n}}{l_{\nu}-1})=(l_{\nu}-1-\frac{\gamma_{n}}{l_{\nu}-1})^{2},$

and rewrite (6.17) as

$\sqrt{n}\cdot\overline{l}_{\nu}(m_{n}(\overline{l}_{\nu})-\frac{1}{N-M}tr((\overline{l}_{\nu}I-\mathcal{M})^{-1}))\xrightarrow[]{p}0.$

Lastly, we prove that for deterministic $z=z_{n}\to\infty,$

$m_{F_{n}}(z)=\frac{1}{N-M}tr((zI-\mathcal{M})^{-1})=\frac{1}{N-M}tr((zI-
S_{BB})^{-1}),$ $\sqrt{n}\cdot z(m_{n}(z)-m_{F_{n}}(z))\xrightarrow[]{p}0,$
(6.18)

which will complete the proof of (6.17), by adopting the argument in lemma 2
from Ledoit and Péché [13].

Denote by $Z_{1},Z_{2},\hskip 1.42271pt...\hskip
1.42271pt,Z_{n}\in\mathbb{R}^{N-M}$ the columns of $Z_{B},$ and recall that
$S_{BB}=\frac{1}{n}Z_{B}Z_{B}^{T},$ from which almost surely for $n$ large
enough such that $z>c_{K,\gamma},$ $zI-S_{BB}$ is invertible and so

$-1+zm_{F_{n}}(z)=\frac{1}{n(N-M)}\sum_{1\leq i\leq n}{Z_{i}^{T}(zI-
S_{BB})^{-1}Z_{i}}$ (6.19)

from multiplying on the right by $(zI-S_{BB})^{-1}$ the identity

$-(zI-S_{BB})+zI=\frac{1}{n}\sum_{1\leq i\leq n}{Z_{i}Z_{i}^{T}},$

taking the trace of each side, and dividing by $N-M.$ Take $R_{i}=(zI-
S_{BB}+\frac{1}{n}Z_{i}Z_{i}^{T})^{-1}$ for $1\leq i\leq n,$ and recall the
resolvent identity

$R_{i}-(zI-S_{BB})^{-1}=R_{i}\frac{1}{n}Z_{i}Z_{i}^{T}(zI-S_{BB})^{-1},$

which by multiplication on the left by $Z_{i}^{T}$ and by $Z_{i}$ on the right
yields

$Z_{i}^{T}(zI-
S_{BB})^{-1}Z_{i}=\frac{Z_{i}^{T}R_{i}Z_{i}}{1-\frac{1}{n}Z_{i}^{T}R_{i}Z_{i}}=-n+\frac{n}{1-\frac{1}{n}Z_{i}^{T}R_{i}Z_{i}}$

because

$|\frac{1}{n}Z_{i}^{T}R_{i}Z_{i}|\leq\frac{1}{n}||Z_{i}||^{2}\cdot||R_{i}||\leq
2\cdot\frac{2}{z}<1$ (6.20)

almost surely (inequality (SM) yields
$||S_{BB}-\frac{1}{n}Z_{i}Z_{i}^{T}||\leq c(K,\gamma)<\frac{z}{2}$ with
probability one), from which (6.19) becomes

$-1+zm_{F_{n}}(z)=-\frac{1}{\gamma_{n}}+\frac{1}{N-M}\sum_{1\leq i\leq
n}{\frac{1}{1-\frac{1}{n}Z_{i}^{T}R_{i}Z_{i}}}=-\frac{1}{\gamma_{n}}+\frac{1}{\gamma_{n}}\cdot\frac{1}{1-\gamma_{n}m_{F_{n}}(z)}+\Delta_{1}+\Delta_{2}$
(6.21)

where

$\Delta_{1}=\frac{1}{N-M}\sum_{1\leq i\leq
n}{(\frac{1}{1-\frac{1}{n}Z_{i}^{T}R_{i}Z_{i}}-\frac{1}{1-\frac{1}{n}tr(R_{i})})},$
$\Delta_{2}=\frac{1}{N-M}\sum_{1\leq i\leq
n}{(\frac{1}{1-\frac{1}{n}tr(R_{i})}-\frac{1}{1-\frac{1}{n}tr((zI-
S_{BB})^{-1})})},$

both being small in probability as we show next.

$\Delta_{1}=o_{p}(\frac{1}{\sqrt{n}}):$ Almost surely

$|\Delta_{1}|\leq\frac{1}{N-M}\sum_{1\leq i\leq
n}{4|\frac{1}{n}Z_{i}^{T}R_{i}Z_{i}-\frac{1}{n}tr(R_{i})|}$

because from (6.20)

$\frac{1}{n}Z_{i}^{T}R_{i}Z_{i}\leq\frac{4}{z}\leq\frac{1}{2},\hskip
5.69046pt\frac{1}{n}tr(R_{i})\leq\frac{N-M}{n}\cdot||R_{i}||\leq
2\gamma\cdot\frac{2}{z}\leq\frac{1}{2}.$

It suffices to prove that

$\mathbb{E}[(\frac{1}{n}Z_{1}^{T}R_{1}Z_{1}-\frac{1}{n}tr(R_{1}))^{2}\chi_{||S_{BB}-\frac{1}{n}Z_{1}Z_{1}^{T}||\leq\frac{z}{2}}]\leq(\frac{N-M}{n})^{2}\cdot\frac{4}{z^{2}n}$

since this yields for the events
$A_{i,n}=\\{||S_{BB}-\frac{1}{n}Z_{i}Z_{i}^{T}||\leq\frac{z}{2}\\},1\leq i\leq
n,$

$n\mathbb{E}[(\frac{1}{N-M}\sum_{1\leq i\leq
n}{|\frac{1}{n}Z_{i}^{T}R_{i}Z_{i}-\frac{1}{n}tr(R_{i})|})^{2}\chi_{A_{1,n}\cap
A_{2,n}\cap...\cap A_{n,n}}]\leq$ $\leq
n\mathbb{E}[\frac{n}{(N-M)^{2}}\sum_{1\leq i\leq
n}{(\frac{1}{n}Z_{i}^{T}R_{i}Z_{i}-\frac{1}{n}tr(R_{i}))^{2}}\chi_{A_{1,n}\cap
A_{2,n}\cap...\cap A_{n,n}}]\leq$ $\leq\frac{n^{2}}{(N-M)^{2}}\cdot\sum_{1\leq
i\leq
n}{\mathbb{E}[(\frac{1}{n}Z_{i}^{T}R_{i}Z_{i}-\frac{1}{n}tr(R_{i}))^{2}\chi_{A_{i,n}}]}\leq\frac{4}{z^{2}}\to
0$

with which the following simple fact can be used:

If $|X_{n}|\leq Y_{n}$ almost surely,
$\lim_{n\to\infty}{\mathbb{E}[Y_{n}^{2}\chi_{A_{n}}]}=0,\lim_{n\to\infty}{\mathbb{P}(A_{n})}=1,$
then $X_{n}\xrightarrow[]{p}0.$

Proof: For any $t>0,$

$\mathbb{P}(|X_{n}|\geq t)\leq\mathbb{P}(A_{n}^{c})+\mathbb{P}(|X_{n}|\geq
t,A_{n})\leq\mathbb{P}(A_{n}^{c})+\mathbb{P}(|X_{n}|>Y_{n},A_{n})+\mathbb{P}(Y_{n}\geq
t,A_{n})\leq$
$\leq\mathbb{P}(A_{n}^{c})+\mathbb{P}(|X_{n}|>Y_{n})+\frac{\mathbb{E}[Y_{n}^{2}\chi_{A_{n}}]}{t^{2}}\xrightarrow[]{n\to\infty}0.$

Because $Z_{1}=[z_{(M+1)1}\hskip 2.84544ptz_{(M+2)1}\hskip 2.84544pt...\hskip
2.84544ptz_{N1}]^{T}$ and
$S_{BB}-\frac{1}{n}Z_{1}Z_{1}^{T},R_{1}:=(r_{ij})_{1\leq i,j\leq N-M}$ are
independent,

$\mathbb{E}[(\frac{1}{N-M}Z_{1}^{T}R_{1}Z_{1}-\frac{1}{N-M}tr(R_{1}))^{2}\hskip
1.42271pt|S_{BB}-\frac{1}{n}Z_{1}Z_{1}^{T}]=\frac{1}{(N-M)^{2}}\mathbb{E}[\sum_{M+1\leq
i,j\leq N}{((z_{i1}z_{j1}-\delta_{i,j})r_{ij}})^{2}|R_{1}]=$
$=\frac{1}{(N-M)^{2}}(\mathbb{E}[(z_{11}^{2}-1)^{2}]\sum_{1\leq i\leq
N-M}{r^{2}_{ii}}+4\sum_{i<j}{r_{ij}}^{2})\leq\frac{c(K)}{(N-M)^{2}}\cdot||R_{1}||_{F}^{2}\leq\frac{c(K)}{N-M}\cdot||R_{1}||^{2}$

since the cross term corresponding to $(i,j),(i^{\prime},j^{\prime})$ is zero
in expectation unless $\\{i,j\\}=\\{i^{\prime},j^{\prime}\\}$ as multisets and
$R_{1}$ is symmetric, from which

$\mathbb{E}[(\frac{1}{N-M}Z_{1}^{T}R_{1}Z_{1}-\frac{1}{N-M}tr(R_{1}))^{2}\chi_{||S_{BB}-\frac{1}{n}Z_{1}Z_{1}^{T}||\leq\frac{z}{2}}]\leq\frac{c(K)}{N-M}\cdot\frac{4}{z^{2}}$

because $||S_{BB}-\frac{1}{n}Z_{1}Z_{1}^{T}||\leq\frac{z}{2}$ implies

$||R_{1}||=\frac{1}{z-||S_{BB}-\frac{1}{n}Z_{1}Z_{1}^{T}||}\leq\frac{2}{z}.$

$\Delta_{2}=o_{p}(\frac{1}{\sqrt{n}}):$ For any $t>0,$

$\mathbb{P}(\sqrt{n}|\Delta_{2}|\geq t)\leq
n\mathbb{P}(|\frac{1}{1-\frac{1}{n}tr(R_{1})}-\frac{1}{1-\frac{1}{n}tr((zI-
S_{BB})^{-1})}|\geq\frac{t\cdot\frac{N-M}{n}}{\sqrt{n}})\leq$ $\leq
n\mathbb{P}(||R_{1}||\geq\frac{n}{2(N-M)})+n\mathbb{P}(||zI-
S_{BB}||\leq\frac{N-M}{2n})+n\mathbb{P}(|\frac{1}{n}tr(R_{1})-\frac{1}{n}tr((zI-
S_{BB})^{-1})|\geq\frac{4t\cdot\frac{N-M}{n}}{\sqrt{n}})$

which tends to zero as $n\to\infty,$ using (SM) for the first two terms while
for the third, for $B\in\mathbb{R}^{p\times p},u\in\mathbb{R}^{p}$ with
$B,B+uu^{T}$ invertible,

$u^{T}B^{-1}(B+uu^{T})=(1+u^{T}B^{-1}u)u^{T}\Longleftrightarrow
u^{T}B^{-1}=(1+u^{T}B^{-1}u)u^{T}(B+uu^{T})^{-1},$
$B^{-1}-(B+uu^{T})^{-1}=(B+uu^{T})^{-1}uu^{T}B^{-1}=(1+u^{T}B^{-1}u)\cdot(B+uu^{T})^{-1}uu^{T}(B+uu^{T})^{-1},$
$tr(B^{-1})-tr((B+uu^{T})^{-1})=(1+u^{T}B^{-1}u)\cdot u^{T}(B+uu^{T})^{-2}u,$

from which for $n$ large enough,

$|tr((zI-S_{BB})^{-1})-tr((zI-
R_{1})^{-1})|\leq(1+\frac{1}{n}||Z_{1}||^{2}\cdot||zI-
S_{BB}||^{-1})\cdot\frac{1}{n}||Z_{1}||^{2}\cdot||zI-
S_{BB}+\frac{1}{n}Z_{1}Z_{1}^{T}||^{-2}\xrightarrow[]{a.s.}0,$

yielding

$n\mathbb{P}(|\frac{1}{n}tr(R_{1})-\frac{1}{n}tr((zI-
S_{BB})^{-1})|\geq\frac{4t\cdot\frac{N-M}{n}}{\sqrt{n}})\leq
n\mathbb{P}(\frac{1}{n}||Z_{1}||^{2}\geq
2)+n\mathbb{P}(||S_{BB}||>c_{K,\gamma})\to 0.$

Finally,

$m_{n}(z)=\frac{z+\gamma_{n}-1-\sqrt{(z-\gamma_{n}+1)^{2}-4z}}{2\gamma_{n}z}$

leads to

$-1+zm_{n}(z)=-\frac{1}{\gamma_{n}}+\frac{1}{\gamma_{n}}\cdot\frac{1}{1-\gamma_{n}m_{n}(z)}$
(6.22)

from

$(z-\gamma_{n}+1)^{2}-4z=(z+\gamma_{n}-1)^{2}-4\gamma_{n}z,$

and

$\gamma_{n}zm^{2}_{n}(z)-m_{n}(z)(z+\gamma_{n}-1)+1=0$

or equivalently

$(\gamma_{n}m_{n}(z)-1)\cdot(zm_{n}(z)-1)=-m_{n}(z),$

while (6.21) and (6.22) give

$z(m_{F_{n}}(z)-m_{n}(z))=\frac{1}{\gamma_{n}}\cdot(\frac{1}{1-\gamma_{n}m_{F_{n}}(z)}-\frac{1}{1-\gamma_{n}m_{n}(z)})+\Delta_{1}+\Delta_{2},$

from which

$z(m_{F_{n}}(z)-m_{n}(z))\cdot(1-\frac{1}{z(1-\gamma_{n}m_{F_{n}}(z))\cdot(1-\gamma_{n}m_{n}(z))})=\Delta_{1}+\Delta_{2}.$

Because $\sqrt{n}\Delta_{1},\sqrt{n}\Delta_{2}\xrightarrow[]{p}0,$ and

$\frac{1}{z(1-\gamma_{n}m_{F_{n}}(z))\cdot(1-\gamma_{n}m_{n}(z))}\xrightarrow[]{a.s.}0,$

(6.18) ensues.

##  7 Consistency of Eigenvectors

The proof of Theorem 5 is covered in the first four subsections: $(a)$ in 7.1,
$(b)(i),(c)(i)$ in 7.2, $(b)(ii)$ in 7.3, and $(c)(ii)$ in 7.4. The
justification of Theorem 6, in the same spirit as the proof of Theorem 3, is
presented in 7.5.

###  7.1 Proof of Theorem 5, part (a)

Notice that

$l_{\nu}(1-<p_{\nu},u_{\nu}>^{2})=l_{\nu}(1-(1-R_{\nu}^{2})<a_{\nu},e_{\nu}>^{2})=\frac{l_{\nu}}{\hat{l}_{\nu}}\cdot(\hat{l}_{\nu}R_{\nu}^{2}-\frac{N}{n})+l_{\nu}(1-R_{\nu}^{2})(1-<a_{\nu},e_{\nu}>^{2})+\frac{N}{n}\cdot\frac{l_{\nu}}{\hat{l}_{\nu}}.$
(7.1)

Since Proposition 1, Proposition 2, and Lemma 3 yield

$\frac{l_{\nu}}{\hat{l}_{\nu}}\xrightarrow[]{a.s.}1,\hskip
14.22636ptR_{\nu}^{2}\xrightarrow[]{a.s.}0\hskip
14.22636pt\hat{l}_{\nu}R_{\nu}^{2}-\frac{N}{n}\xrightarrow[]{a.s.}0,$ (7.2)

respectively, it suffices to show that

$l_{\nu}(1-<a_{\nu},e_{\nu}>^{2})\xrightarrow[]{p}0,$ (7.3)

if $\lim_{n\to\infty}{\frac{l^{(n)}_{\nu}}{n/M(n)}}=0,$ with the convergence
holding almost surely if
$\sum_{n\in\mathbb{N}}{\exp(-\frac{an}{l^{(n)}_{\nu}})}<\infty,\forall a>0$ is
also satisfied. As

$l_{\nu}(1-<a_{\nu},e_{\nu}>^{2})\leq
2l_{\nu}(1-<a_{\nu},e_{\nu}>)=l_{\nu}||a_{\nu}-e_{\nu}||^{2},$

for $t>0,$

$\mathbb{P}(l_{\nu}(1-<a_{\nu},e_{\nu}>^{2})\geq
t)\leq\mathbb{P}(||a_{\nu}-e_{\nu}||\geq
2\beta_{\nu})+\mathbb{P}(l_{\nu}\beta_{\nu}^{2}\geq\frac{t}{4}).$

Moreover, Lemma 2 entails

$\sum_{n\to\infty}{\mathbb{P}(||a_{\nu}-e_{\nu}||\geq
2||\beta_{\nu}||)}<\infty,$

and we show next
$\lim_{n\to\infty}{\mathbb{P}(l_{\nu}\beta_{\nu}^{2}\geq\frac{t}{4})}=0,$ with
$\sum_{n\in\mathbb{N}}{\mathbb{P}(l_{\nu}\beta_{\nu}^{2}\geq\frac{t}{4})}<\infty$
if the second condition holds as well, from which (7.3) and its almost sure
variant will ensue.

With the aid of

$\beta_{\nu}^{2}\leq
c(\epsilon_{0})\sum_{k\neq\nu}{(\frac{1}{n}z_{k}^{T}z_{\nu})^{2}}+c(\epsilon_{0})\sum_{k\neq\nu}{(t_{k}^{T}\mathcal{M}(\hat{l}_{\nu}I-\mathcal{M})^{-1}t_{\nu})^{2}},$

it follows that for $n\geq n(\epsilon_{0},t)$

$\mathbb{P}(l_{\nu}\beta_{\nu}^{2}\geq\frac{t}{4})\leq\mathbb{P}(l_{\nu}\sum_{k\neq\nu}{(\frac{1}{n}z_{k}^{T}z_{\nu})^{2}}\geq\frac{t}{8c(\epsilon_{0})})+\mathbb{P}(\frac{n}{M}\sum_{k\neq\nu}{(t_{k}^{T}\mathcal{M}(\hat{l}_{\nu}I-\mathcal{M})^{-1}t_{\nu})^{2}}\geq\frac{tn}{8c(\epsilon_{0})Ml_{\nu}}).$
(7.4)

Inequality (8.4) from part $(b)$ of Lemma 1 yields the second term in (7.4) as
a function of $n\in\mathbb{N}$ is summable as:

$\frac{tn}{Ml_{\nu}}\cdot l_{\nu}^{2}\geq 2,\hskip
14.22636ptMl_{\nu}^{4}\cdot(\frac{tn}{Ml_{\nu}})^{2}\geq t^{2}n,\hskip
14.22636ptMl_{\nu}^{2}\cdot\frac{tn}{Ml_{\nu}}\geq tn.$

Regarding the first term in (7.4), for $t>0,$

$\mathbb{P}(l_{\nu}\sum_{k\neq\nu}{(\frac{1}{n}z_{k}^{T}z_{\nu})^{2}}\geq
2t|z_{\nu})\leq\mathbb{P}(l_{\nu}\cdot\frac{M}{n^{2}}||z_{\nu}||^{2}\geq
t|z_{\nu})+\mathbb{P}(l_{\nu}|\sum_{k\neq\nu}{((\frac{1}{n}z_{k}^{T}z_{\nu})^{2}-\frac{1}{n^{2}}||z_{\nu}||^{2})}|\geq
t|z_{\nu}).$

Inequality (8.3) gives

$\mathbb{P}(l_{\nu}|\sum_{k\neq\nu}{((\frac{1}{n}z_{k}^{T}z_{\nu})^{2}-\frac{1}{n^{2}}||z_{\nu}||^{2})}|\geq
t|z_{\nu})\leq
2\exp(-c\min{(\frac{t^{2}}{Ml_{\nu}^{2}\cdot(\frac{1}{n}||z_{\nu}||)^{4}},\frac{t}{l_{\nu}\cdot(\frac{1}{n}||z_{\nu}||)^{2}})}).$

Therefore, for $n$ large enough such that
$l_{\nu}\leq\frac{n}{M}\cdot\frac{t}{2},$

$\mathbb{P}(l_{\nu}\sum_{k\neq\nu}{(\frac{1}{n}z_{k}^{T}z_{\nu})^{2}}\geq
t)\leq\mathbb{P}(\frac{1}{n}||z_{\nu}||^{2}\geq
2)+2\exp(-c\min{(\frac{t^{2}n^{2}}{4Ml_{\nu}^{2}},\frac{tn}{2l_{\nu}})})\leq$
$\leq\mathbb{P}(\frac{1}{n}||z_{\nu}||^{2}\geq
2)+2\exp(-\frac{ctn}{2l_{\nu}})\leq\mathbb{P}(\frac{1}{n}||z_{\nu}||^{2}\geq
2)+2\exp(-\frac{ct}{\frac{2l_{\nu}}{n/M}}),$

from which we conclude together with
$\lim_{n\to\infty}{\frac{l^{(n)}_{\nu}}{n/M}}=0$ that the bounds in (7.4) tend
to zero, and that they are summable, if additionally
$\sum_{n\in\mathbb{N}}{\exp(-\frac{an}{l^{(n)}_{\nu}})}<\infty,\forall a>0.$

###  7.2 Proof of Theorem 5, parts (b)(i), (c)(i)

Notice that

$n(1-<p_{\nu},u_{\nu}>^{2})=nR_{\nu}^{2}+n(1-R_{\nu}^{2})(1-<a_{\nu},e_{\nu}>^{2}).$

From Lemma 3, Proposition 1, Proposition 2,

$nR_{\nu}^{2}-\frac{N}{l_{\nu}}=\frac{n}{l_{\nu}}\cdot\frac{l_{\nu}}{\hat{l}_{\nu}}\cdot(\hat{l}_{\nu}R_{\nu}^{2}-\frac{N}{n})+\frac{N}{l_{\nu}}(1-\frac{l_{\nu}}{\hat{l}_{\nu}})\xrightarrow[]{a.s.}0,\hskip
14.22636ptR_{\nu}^{2}\xrightarrow[]{a.s.}0.$

It suffices for both $(b)(i),$ and $(c)(i)$ to show

$n(1-<a_{\nu},e_{\nu}>^{2})\Rightarrow X_{(c_{k\nu})_{k\neq\nu}}$

as the claims ensue then from Slutsky’s lemma:

$n(1-<p_{\nu},u_{\nu}>^{2})-\frac{N}{l_{\nu}}\Rightarrow
X_{(c_{k\nu})_{k\neq\nu}},$

which can be rewritten as

$l_{\nu}(1-<p_{\nu},u_{\nu}>^{2})-\frac{N}{n}\Rightarrow\frac{c_{\nu}}{M}X_{(c_{k\nu})_{k\neq\nu}}.$

Note that

$n(1-<a_{\nu},e_{\nu}>^{2})=2n(1-<a_{\nu},e_{\nu}>)-n(1-<a_{\nu},e_{\nu}>)^{2}=n||a_{\nu}-e_{\nu}||^{2}-\frac{n}{4}||a_{\nu}-e_{\nu}||^{4}.$

Lemma 2 gives that almost surely $n||a_{\nu}-e_{\nu}||^{4}\leq
16n\beta_{\nu}^{4},$ and from (5.18),
$\sqrt{n}\cdot\beta_{\nu}^{2}\xrightarrow[]{p}0.$ Therefore, by employing
Slutsky’s lemma once again and Lemma 2, it suffices to show that

$n\beta_{\nu}^{2}\Rightarrow X_{(c_{k\nu})_{{k\neq\nu}}}.$

Recall that

$n\beta_{\nu}^{2}=\sum_{k\neq\nu}{\frac{1}{(l_{k}-l_{\nu})^{2}}\mathcal{D}_{\nu
k}^{2}}=\sum_{k\neq\nu}{\frac{l_{k}l_{\nu}}{(l_{k}-l_{\nu})^{2}}(\frac{1}{\sqrt{n}}z_{k}^{T}z_{\nu})^{2}}+\sum_{k\neq\nu}{\frac{l_{k}l_{\nu}}{(l_{k}-l_{\nu})^{2}}(\sqrt{n}t_{k}^{T}\mathcal{M}(\hat{l}_{\nu}I-\mathcal{M})^{-1}t_{\nu})^{2}}+$
$+2\sum_{k\neq\nu}{\frac{l_{k}l_{\nu}}{(l_{k}-l_{\nu})^{2}}\cdot\frac{1}{\sqrt{n}}z_{k}^{T}z_{\nu}}\cdot\sqrt{n}t_{k}^{T}\mathcal{M}(\hat{l}_{\nu}I-\mathcal{M})^{-1}t_{\nu}.$
(7.5)

Part $(b)$ of Lemma 1 yields the second term in (7.5) tends to zero in
probability as

$\sum_{k\neq\nu}{\frac{l_{k}l_{\nu}}{(l_{k}-l_{\nu})^{2}}(\sqrt{n}t_{k}^{T}\mathcal{M}(\hat{l}_{\nu}I-\mathcal{M})^{-1}t_{\nu})^{2}}\leq
Mc(\epsilon_{0})\cdot\frac{n}{M}\sum_{k\neq\nu}{(t_{k}^{T}\mathcal{M}(\hat{l}_{\nu}I-\mathcal{M})^{-1}t_{\nu})^{2}}\xrightarrow[]{p}0.$
(7.6)

Because

$\mathbb{E}[\sum_{k\neq\nu}{\frac{l_{k}l_{\nu}}{(l_{k}-l_{\nu})^{2}}(\frac{1}{\sqrt{n}}z_{k}^{T}z_{\nu})^{2}}]=\sum_{k\neq\nu}{\frac{l_{k}l_{\nu}}{(l_{k}-l_{\nu})^{2}}}\leq
Mc(\epsilon_{0}),$

it follows the third term in (7.5) also tends to zero in probability from
Cauchy-Schwarz inequality, (7.6), and the following simple fact:

If $X_{n},Y_{n}\geq 0$ are random variables defined on the same space and
$X_{n}\xrightarrow[]{p}0,\mathbb{E}[Y_{n}]\leq c,$ then

$X_{n}Y_{n}\xrightarrow[]{p}0.$ (7.7)

Proof: For any $t,s>0,$

$\mathbb{P}(X_{n}Y_{n}\geq t)\leq\mathbb{P}(Y_{n}\geq
s)+\mathbb{P}(X_{n}\geq\frac{t}{s})\leq\frac{c}{s}+\mathbb{P}(X_{n}\geq\frac{t}{s}),$

from which $\limsup_{n\to\infty}{\mathbb{P}(X_{n}Y_{n}\geq
t)}\leq\frac{c}{s},$ and taking $s\to\infty$ concludes the justification of
the claim.

We turn now to the main contribution in (7.5):

$\sum_{k\neq\nu}{\frac{l_{k}l_{\nu}}{(l_{k}-l_{\nu})^{2}}(\frac{1}{\sqrt{n}}z_{k}^{T}z_{\nu})^{2}}.$

Because
$\lim_{n\to\infty}{\frac{l_{k}l_{\nu}}{(l_{k}-l_{\nu})^{2}}}=c_{k\nu},$ it
suffices to prove that

$T_{n}:=\sum_{k\neq\nu}{c_{k\nu}(\frac{1}{\sqrt{n}}z_{k}^{T}z_{\nu})^{2}}\Rightarrow
X_{(c_{k\nu})_{k\neq\nu}},$ (7.8)

where

$X_{(c_{k\nu})_{k\neq\nu}}=\sum_{k\neq\nu}{c_{k\nu}y_{k\nu}^{2}},$

with $(y_{k\nu})_{k\neq\nu}$ mutually independent standard normal
distributions, because

$\sum_{k\neq\nu}{(\frac{l_{k}l_{\nu}}{(l_{k}-l_{\nu})^{2}}-c_{k\nu})(\frac{1}{\sqrt{n}}z_{k}^{T}z_{\nu})^{2}}\xrightarrow[]{p}0$

as for any $t>0,\epsilon>0,$

$\mathbb{P}(\sum_{k\neq\nu}{|\frac{l_{k}l_{\nu}}{(l_{k}-l_{\nu})^{2}}-c_{k\nu}|\cdot(\frac{1}{\sqrt{n}}z_{k}^{T}z_{\nu})^{2}}\geq
t)\leq\mathbb{P}(\sum_{k\neq\nu}{(\frac{1}{\sqrt{n}}z_{k}^{T}z_{\nu})^{2}}\geq\frac{t}{\epsilon})\leq\frac{M\epsilon}{t},$

using
$\mathbb{E}[\sum_{k\neq\nu}{(\frac{1}{\sqrt{n}}z_{k}^{T}z_{\nu})^{2}}]=M-1.$

If $c_{k\nu}=0$ for all $k\neq\nu,$ (7.8) is clear. Suppose next that
$\max_{k\neq\nu}{c_{k\nu}}>0.$ We use the moment convergence theorem and
Carleman’s condition (lemmas $B1,B3,$ respectively in Bai and Silverstein
[4]), stated below, to show (7.8) holds:

A sequence of distributions $F_{n}$ converges weakly to a limit if each
$F_{n}$ has moments of all orders; for all $k\in\mathbb{N},$ their $k^{th}$
moments converge to some finite limit $\beta_{k};$ and for any right-
continuous nondecreasing functions $F,G$ with the same moment sequence,
$F=G+c,$ for some $c\in\mathbb{R}.$

If $(\beta_{k})$ is the moment sequence of a distribution function $F$ and
$\sum_{k\geq 1}{\beta_{2k}^{-1/(2k)}}=\infty,$ then $F$ is uniquely determined
by its moment sequence.

Namely, we show that

$\lim_{n\to\infty}{\mathbb{E}[T_{n}^{m}]}=\beta_{m},m\in\mathbb{N},\hskip
14.22636pt\sum_{m\geq 1}{\beta_{2m}^{-1/(2m)}}=\infty,$ (7.9)

for

$\beta_{m}=\mathbb{E}[X_{(c_{k\nu})_{k\neq\nu}}^{m}]=\sum{\binom{m}{(q_{k})_{k\neq\nu}}{\prod_{k\neq\nu}{\frac{(2q_{k})!}{2^{q_{k}}q_{k}!}}c_{k\nu}^{q_{k}}}},$

where the sum is over non-negative integer sequences $(q_{k})_{k\neq\nu}$ with
$\sum_{k\neq\nu}{q_{k}}=m.$

Fix $m\in\mathbb{N}.$ Then

$\mathbb{E}[T_{n}^{m}]=\mathbb{E}[(\sum_{k\neq\nu}{c_{k\nu}(\frac{1}{\sqrt{n}}z_{k}^{T}z_{\nu})^{2}})^{m}].$

Consider the term in this last sum corresponding to some sequence
$(q_{k})_{k\neq\nu}$ with $\sum_{k\neq\nu}{q_{k}}=m:$

$\binom{m}{(q_{k})_{k\neq\nu}}\prod_{k\neq\nu}{c_{k\nu}^{q_{k}}}\cdot\frac{1}{n^{m}}\mathbb{E}[\prod_{k\neq\nu}{(\sum_{1\leq
j\leq n}{z_{kj}z_{\nu j})^{2q_{k}}}}].$

For the first claim in (7.9), it is enough to show that as $n\to\infty,$

$\frac{1}{n^{m}}\mathbb{E}[\prod_{k\neq\nu}{(\sum_{1\leq j\leq n}{z_{kj}z_{\nu
j})^{2q_{k}}}}]\to\prod_{k\neq\nu}{\frac{(2q_{k})!}{2^{q_{k}}q_{k}!}}.$ (7.10)

In this product, the factor corresponding to $k\neq\nu$ will contribute to the
terms that form up the expectation products of the type

$c(m_{1},m_{2},\hskip 1.42271pt...\hskip 1.42271pt,m_{t})(z_{kj_{1}}z_{\nu
j_{1}})^{m_{1}}...(z_{kj_{t}}z_{\nu j_{t}})^{m_{t}}$

for $m_{1}+...+m_{t}=2q_{k},m_{i}>0,$ and $j_{1},\hskip 1.42271pt...\hskip
1.42271pt,j_{t}$ pairwise distinct. If some $m_{i}=1,$ then the expectation of
any term in which this product appears will be zero by independence
($z_{kj_{i}}$ shows up only once in each such term, is independent of the
rest, and has mean zero). Hence we are left with the products containing
$m_{1},\hskip 1.42271pt...\hskip 1.42271pt,m_{t}\geq 2.$ For any $1\leq t\leq
m$ and fixed $k\neq\nu,$ the number of such products having some $m_{i}\geq 3$
is at most

$c(m)n^{t+\sum_{k^{\prime}\neq k}{q_{k^{\prime}}}}\leq
c(m)n^{q_{k}-1+\sum_{k^{\prime}\neq k}{q_{k}}}\leq c(m)n^{m-1},$

with an overall contribution in the left-hand side term in (7.10) upper
bounded by

$\frac{1}{n^{m}}\cdot m\cdot M\cdot c(m)n^{m-1}\mathbb{E}[z_{11}^{4m}]\to 0$

from the arithmetic-geometric mean inequality (AM-GM) and
$\mathbb{E}[|z_{11}|^{l}]\leq c(K,l)<\infty,$ for all $l\in\mathbb{N}.$

Therefore, the only such products surviving in the limit are the ones for
which $m_{1}=...=m_{t}=2.$ Moreover, the number of products for which some
$z_{\nu j}$ appears in at least two factors corresponding to some $k\neq
k^{\prime}$ is at most $M^{2}\cdot c(m)n^{m-1}:$ for each tuple
$(q_{k})_{k\neq\nu}$ with $\sum_{k\neq\nu}{q_{k}}=m,$ denote the subsets of
$\\{1,2,\hskip 1.42271pt...\hskip 1.42271pt,n\\}$ corresponding to
$(q_{k})_{k\neq\nu}$ by $(S_{q_{k}})_{k\neq\nu}$ (i.e.,
$S_{q_{k}}=\\{j_{1},j_{2},\hskip 1.42271pt...\hskip 1.42271pt,j_{t}\\}$); each
fixed $(q_{k})_{k\neq\nu}$ and pair $(k_{1},k_{2}),k_{1}\neq k_{2}$ for which
$|S_{q_{k_{1}}}\cap S_{q_{k_{2}}}|>0$ generate at most $m\cdot n^{m-1}$
products in the left-hand side term of (7.10) because the collection
$\mathcal{S}$ of ordered tuples of subsets of $\\{1,2,\hskip
1.42271pt...\hskip 1.42271pt,n\\}$ with sizes $(q_{k})_{k\neq\nu,k\neq
k_{1},k\neq k_{2}},q_{k_{1}}-1,q_{k_{2}},$ respectively has

$|\mathcal{S}|\leq n^{-1+\sum_{k\neq\nu}{q_{k}}}=n^{m-1},$

and for the function taking

$\\{(S_{q_{k}})_{k\neq\nu}:|S_{q_{k_{1}}}\cap
S_{q_{k_{2}}}|>0,|S_{q_{k}}|=q_{k},S_{q_{k}}\subset\\{1,2,\hskip
1.42271pt...\hskip 1.42271pt,n\\}\\}$

into $\mathcal{S}$ by removing from $S_{q_{k_{1}}}$ the smallest element in
$S_{q_{k_{1}}}\cap S_{q_{k_{2}}}$, the preimage of any tuple has size at most
$m$ (one element from $S_{q_{k_{2}}}$ needs to be added to $S_{q_{k_{1}}},$
and $|S_{q_{k_{2}}}|=q_{k_{2}}\leq m$). Consequently, the terms that will be
nonzero in the limit will have pairwise distinct random variables from $z_{\nu
j},1\leq j\leq n,$ and so their expectations will disappear, each being one,
from which we get that as $n\to\infty,$

$\frac{1}{n^{m}}\mathbb{E}[\prod_{k\neq\nu}{(\sum_{1\leq j\leq n}{z_{kj}z_{\nu
j})^{2q_{k}}}}]\to\frac{1}{n^{m}}\prod_{k\neq\nu}{\frac{(2q_{k})!}{2^{q_{k}}}\binom{n-q_{1}-...-q_{k-1}}{q_{k}}}\to\prod_{k\neq\nu}{\frac{(2q_{k})!}{2^{q_{k}}q_{k}!}},$

where $q_{0}=q_{\nu}=0.$ This concludes the proof of

$\lim_{n\to\infty}{\mathbb{E}[T_{n}^{m}]}=\beta_{m},m\in\mathbb{N},\hskip
14.22636pt\beta_{m}=\mathbb{E}[X_{(c_{k\nu})_{k\neq\nu}}^{m}]=\sum{\binom{m}{(q_{k})_{k\neq\nu}}{\prod_{k\neq\nu}{\frac{(2q_{k})!}{2^{q_{k}}q_{k}!}}c_{k\nu}^{q_{k}}}}.$

The second result in (7.9) follows from

$\sum_{(q_{k}),\sum{q_{k}}=m}{\prod_{k\neq\nu}{\frac{(2q_{k})!}{2^{q_{k}}q_{k}!}c_{k\nu}^{q_{k}}}}\leq\sum_{(q_{k}),\sum{q_{k}}=m}{\prod_{k\neq\nu}{2^{q_{k}}q_{k}!c_{k\nu}^{q_{k}}}}\leq(2\max_{k\neq\nu}{c_{k\nu}})^{m}\sum_{(q_{k}),\sum{q_{k}}=m}{\prod_{k\neq\nu}{q_{k}!}}\leq(2\max_{k\neq\nu}{c_{k\nu}})^{m}\cdot
m^{M}m!$

because each product is at most $m!$ and their number is at most $m^{M},$
yielding

$0<\beta_{m}^{1/m}\leq 2\max_{k\neq\nu}{c_{k\nu}}\cdot(m^{M}m!)^{1/m}\leq
2\max_{k\neq\nu}{c_{k\nu}}\cdot 2^{M}m.$

Finally, the claims about $(y_{ij})_{i\neq j}$ ensue from
$\frac{1}{\sqrt{n}}z_{i}^{T}z_{j}\Rightarrow y_{ij}.$

###  7.3 Proof of Theorem 5, part (b)(ii)

Using (7.1) and (7.2), it suffices to prove

$l_{\nu}(1-<a_{\nu},e_{\nu}>^{2})-\frac{l_{\nu}}{n}\sum_{k\neq\nu}{\frac{l_{k}l_{\nu}}{(l_{k}-l_{\nu})^{2}}}\xrightarrow[]{p}0,$

with the convergence being almost sure if additionally
$\sum_{n\in\mathbb{N}}{\exp(-a\cdot M(n))}<\infty,\forall a>0.$ Because

$l_{\nu}(1-<a_{\nu},e_{\nu}>^{2})=2l_{\nu}(1-<a_{\nu},e_{\nu}>)-l_{\nu}(1-<a_{\nu},e_{\nu}>)^{2}=l_{\nu}||a_{\nu}-e_{\nu}||^{2}-\frac{l_{\nu}}{4}||a_{\nu}-e_{\nu}||^{4},$

and from Lemma 2
$\frac{||a_{\nu}-e_{\nu}||}{\beta_{\nu}}\xrightarrow[]{a.s.}1,$ we show that

$l_{\nu}\beta_{\nu}^{2}-\frac{l_{\nu}}{n}\sum_{k\neq\nu}{\frac{l_{k}l_{\nu}}{(l_{k}-l_{\nu})^{2}}}\xrightarrow[]{p}0,$
(7.11)

which will complete the proof of $(b)(ii)$ since this will yield
$l_{\nu}\beta_{\nu}^{2}$ bounded in probability, from which

$l_{\nu}||a_{\nu}-e_{\nu}||^{2}-l_{\nu}\beta_{\nu}^{2}=l_{\nu}\beta_{\nu}^{2}\cdot(\frac{||a_{\nu}-e_{\nu}||^{2}}{\beta_{\nu}^{2}}-1)\xrightarrow[]{p}0,$

while (7.11) with Lemma 2 imply

$\beta_{\nu}^{2}=O(c(\epsilon_{0})\cdot\frac{M}{n})+o_{p}(\frac{1}{l_{\nu}}),$

and so

$\frac{l_{\nu}}{4}||a_{\nu}-e_{\nu}||^{4}\xrightarrow[]{p}0$

(these convergences hold almost surely if $\sum_{n\in\mathbb{N}}{\exp(-a\cdot
M(n))}<\infty,\forall a>0$ holds as well).

Recall that

$l_{\nu}\beta_{\nu}^{2}=l_{\nu}\sum_{k\neq\nu}{\frac{l_{k}l_{\nu}}{(l_{k}-l_{\nu})^{2}}(\frac{1}{n}z_{k}^{T}z_{\nu})^{2}}+l_{\nu}\sum_{k\neq\nu}{\frac{l_{k}l_{\nu}}{(l_{k}-l_{\nu})^{2}}(t_{k}^{T}\mathcal{M}(\hat{l}_{\nu}I-\mathcal{M})^{-1}t_{\nu})^{2}}+$
$+2l_{\nu}\sum_{k\neq\nu}{\frac{l_{k}l_{\nu}}{(l_{k}-l_{\nu})^{2}}\cdot\frac{1}{n}z_{k}^{T}z_{\nu}\cdot
t_{k}^{T}\mathcal{M}(\hat{l}_{\nu}I-\mathcal{M})^{-1}t_{\nu}}.$ (7.12)

Inequality (8.4) and $\frac{l_{k}l_{\nu}}{(l_{k}-l_{\nu})^{2}}\leq
c(\epsilon_{0}),k\neq\nu,l_{\nu}\leq 2c_{\nu}\cdot\frac{n}{M}$ give the second
term on the right-hand side of (7.12) tends to zero almost surely as the upper
bounds will be summable as functions of $n\in\mathbb{N}:Ml_{\nu}^{2}\geq
Ml_{\nu}\geq\frac{c_{\nu}}{2}\cdot n.$ We show next that

$l_{\nu}\sum_{k\neq\nu}{\frac{l_{k}l_{\nu}}{(l_{k}-l_{\nu})^{2}}(\frac{1}{n}z_{k}^{T}z_{\nu})^{2}}-\frac{l_{\nu}}{n}\sum_{k\neq\nu}{\frac{l_{k}l_{\nu}}{(l_{k}-l_{\nu})^{2}}}\xrightarrow[]{p}0,$
(7.13)

which together with Cauchy-Schwarz inequality and

$\frac{l_{\nu}}{n}\sum_{k\neq\nu}{\frac{l_{k}l_{\nu}}{(l_{k}-l_{\nu})^{2}}}\leq
c_{\nu}c(\epsilon_{0})$

will yield that the third term on the right-hand side of (7.12) converges in
probability to zero (almost surely if $\sum_{n\in\mathbb{N}}{\exp(-a\cdot
M(n))}<\infty,\forall a>0$ holds as well), concluding the justification of
(7.11).

(7.13) can be rewritten as

$\frac{1}{M}\sum_{k\neq\nu}{\frac{l_{k}l_{\nu}}{(l_{k}-l_{\nu})^{2}}\cdot((\frac{1}{\sqrt{n}}z_{k}^{T}z_{\nu})^{2}-1)}\xrightarrow[]{p}0,$

with the convergence holding almost surely if additionally
$\sum_{n\in\mathbb{N}}{\exp(-a\cdot M(n))}<\infty,\forall a>0.$ Note that

$\mathbb{P}(|\frac{1}{M}\sum_{k\neq\nu}{\frac{l_{k}l_{\nu}}{(l_{k}-l_{\nu})^{2}}\cdot((\frac{1}{\sqrt{n}}z_{k}^{T}z_{\nu})^{2}-1)}|\geq
t)\leq\mathbb{P}(\frac{1}{M}\sum_{k\neq\nu}{|(\frac{1}{\sqrt{n}}z_{k}^{T}z_{\nu})^{2}-1|}\geq\frac{t}{c(\epsilon_{0})})\leq$
$\leq\mathbb{P}(\frac{1}{M}\sum_{k\neq\nu}{|(\frac{1}{\sqrt{n}}z_{k}^{T}z_{\nu})^{2}-\frac{1}{n}||z_{\nu}||^{2}|}\geq\frac{t}{2c(\epsilon_{0})})+\mathbb{P}(|\frac{1}{n}||z_{\nu}||^{2}-1|\geq\frac{t}{2c(\epsilon_{0})}),$

and

$\mathbb{P}(\frac{1}{M}\sum_{k\neq\nu}{|(\frac{1}{\sqrt{n}}z_{k}^{T}z_{\nu})^{2}-\frac{1}{n}||z_{\nu}||^{2}|}\geq
t)\leq\mathbb{P}(\frac{1}{n}||z_{\nu}||^{2}\geq
2)+2\exp(-c\min{(\frac{Mt^{2}}{4},\frac{tM}{2})})$

from inequality (8.3)

$\mathbb{P}(\frac{1}{M}\sum_{k\neq\nu}{|(\frac{1}{\sqrt{n}}z_{k}^{T}z_{\nu})^{2}-\frac{1}{n}||z_{\nu}||^{2}|}\geq
t|z_{\nu})\leq
2\exp(-c\min{(\frac{Mt^{2}}{(\frac{1}{n}||z_{1}||^{2})^{2}},\frac{tM}{\frac{1}{n}||z_{1}||^{2}})}).$

Both conclusions then follow by employing Hanson-Wright inequality (HW) and
Borel-Cantelli lemma for the almost sure convergence.

###  7.4 Proof of Theorem 5, part (c)(ii)

From (7.1), (7.2), and Proposition 1, it suffices to show

$l_{\nu}(1-<a_{\nu},e_{\nu}>^{2})-\frac{l_{\nu}}{n}\sum_{k\neq\nu}{\frac{l_{k}l_{\nu}}{(l_{k}-l_{\nu})^{2}}}\Rightarrow
c_{\nu}N(0,2\sigma_{\nu}),$

which is a consequence of Slutsky’s lemma and the following identities proven
below:

$l_{\nu}||a_{\nu}-e_{\nu}||^{2}-\frac{l_{\nu}}{4}||a_{\nu}-e_{\nu}||^{4}=l_{\nu}\beta_{\nu}^{2}+o_{p}(1),$
(7.14)
$l_{\nu}\beta_{\nu}^{2}-\frac{l_{\nu}}{n}\sum_{k\neq\nu}{\frac{l_{k}l_{\nu}}{(l_{k}-l_{\nu})^{2}}}=\frac{l_{\nu}}{n}\sum_{k\neq\nu}{\frac{l_{k}l_{\nu}}{(l_{k}-l_{\nu})^{2}}[(\frac{1}{\sqrt{n}}z_{k}^{T}z_{\nu})^{2}-1]}+o_{p}(1),$
(7.15)
$T_{n}:=\frac{1}{\sqrt{M}}\sum_{k\neq\nu}{\frac{l_{k}l_{\nu}}{(l_{k}-l_{\nu})^{2}}[(\frac{1}{\sqrt{n}}z_{k}^{T}z_{\nu})^{2}-1]}\Rightarrow
N(0,2\sigma_{\nu}).$ (7.16)

Proof of (7.14): Recall (5.3) and (5.4),

$a_{\nu}-e_{\nu}=-\mathcal{R}_{\nu}\mathcal{D}_{\nu}e_{\nu}+r_{\nu},$
$r_{\nu}=(<a_{\nu},e_{\nu}>-1)e_{\nu}-\mathcal{R}_{\nu}\mathcal{D}_{\nu}(a_{\nu}-e_{\nu})+(\hat{l}_{\nu}-l_{\nu})\mathcal{R}_{\nu}(a_{\nu}-e_{\nu}),$

and denote by

$\alpha_{\nu}=||\mathcal{R}_{\nu}\mathcal{D}_{\nu}||+|\hat{l}_{\nu}-l_{\nu}|\cdot||\mathcal{R}_{\nu}||,\beta_{\nu}=||\mathcal{R}_{\nu}\mathcal{D}_{\nu}e_{\nu}||.$

Then

$||a_{\nu}-e_{\nu}||^{2}=\beta_{\nu}^{2}+||r_{\nu}||^{2}-2r_{\nu}\cdot\mathcal{R}_{\nu}\mathcal{D}_{\nu}e_{\nu},$
$||r_{\nu}||^{2}=\frac{||a_{\nu}-e_{\nu}||^{4}}{4}+||\mathcal{R}_{\nu}\mathcal{D}_{\nu}(a_{\nu}-e_{\nu})-(\hat{l}_{\nu}-l_{\nu})\mathcal{R}_{\nu}(a_{\nu}-e_{\nu})||^{2},$

as $e_{\nu},\mathcal{R}_{\nu}v$ are orthogonal for all $v\in\mathbb{R}^{M},$
from which

$||a_{\nu}-e_{\nu}||^{2}-\frac{||a_{\nu}-e_{\nu}||^{4}}{4}=\beta_{\nu}^{2}-2r_{\nu}\cdot\mathcal{R}_{\nu}\mathcal{D}_{\nu}e_{\nu}+||\mathcal{R}_{\nu}\mathcal{D}_{\nu}(a_{\nu}-e_{\nu})+(\hat{l}_{\nu}-l_{\nu})\mathcal{R}_{\nu}(a_{\nu}-e_{\nu})||^{2}.$

Notice that

$|2r_{\nu}\cdot\mathcal{R}_{\nu}\mathcal{D}_{\nu}e_{\nu}|=|2(\mathcal{R}_{\nu}\mathcal{D}_{\nu}(a_{\nu}-e_{\nu})-(\hat{l}_{\nu}-l_{\nu})\mathcal{R}_{\nu}(a_{\nu}-e_{\nu}))\cdot\mathcal{R}_{\nu}\mathcal{D}_{\nu}e_{\nu}|\leq
2\beta_{\nu}\alpha_{\nu}||a_{\nu}-e_{\nu}||,$
$||\mathcal{R}_{\nu}\mathcal{D}_{\nu}(a_{\nu}-e_{\nu})+(\hat{l}_{\nu}-l_{\nu})\mathcal{R}_{\nu}(a_{\nu}-e_{\nu})||^{2}\leq\alpha_{\nu}^{2}||a_{\nu}-e_{\nu}||^{2}.$

As Lemma 2 gives
$\frac{||a_{\nu}-e_{\nu}||}{\beta_{\nu}}\xrightarrow[]{a.s.}1$ and
$\alpha_{\nu}\xrightarrow[]{a.s.}0$ from Proposition 2, (7.14) will ensue from

$l_{\nu}\beta_{\nu}^{2}\alpha_{\nu}\xrightarrow[]{p}0.$

As it will be shown below, $\beta_{\nu}^{2}=\tau_{\nu}+\frac{1}{l_{\nu}}w_{n}$
for $w_{n}\Rightarrow
c_{\nu}N(0,2\sigma_{\nu}),\tau_{\nu}=\frac{1}{n}\sum_{k\neq\nu}{\frac{l_{k}l_{\nu}}{(l_{k}-l_{\nu})^{2}}}.$
Then

$l_{\nu}\beta^{2}_{\nu}\alpha_{\nu}=l_{\nu}\tau_{\nu}\alpha_{\nu}+\alpha_{\nu}w_{n}.$

The second term converges to zero in probability because
$\alpha_{\nu}\xrightarrow[]{a.s.}0,w_{n}\Rightarrow
c_{\nu}N(0,2\sigma_{\nu}),$ while for the first

$0\leq l_{\nu}\tau_{\nu}\alpha_{\nu}\leq\frac{2c_{\nu}n}{\sqrt{M}}\cdot
c(\epsilon_{0})\frac{M}{n}\cdot\alpha_{\nu}=2c_{\nu}\cdot
c(\epsilon_{0})\cdot\sqrt{M}\alpha_{\nu}.$

To conclude, we prove $\sqrt{M}\alpha_{\nu}\xrightarrow[]{p}0:$ note that from
the analysis in Proposition 2,

$\alpha_{\nu}\leq
c(\epsilon_{0})\cdot(||\frac{1}{n}Z_{A}Z_{A}^{T}-I||+||T\mathcal{M}(\hat{l}_{\nu}I-\mathcal{M})^{-1}T^{T}||+|\frac{\hat{l}_{\nu}}{l_{\nu}}-1|),$
(7.17)

and under multiplication by $\sqrt{M}$ each of the three terms on the right
tend to zero in probability:

$1.$ First term: inequality (SM) yields

$\frac{1}{n}(\sqrt{n}-CK^{2}(\sqrt{M}+(n/M)^{1/4}))^{2}\leq\lambda_{\min}(\frac{1}{n}Z_{A}Z_{A}^{T})\leq\lambda_{\max}(\frac{1}{n}Z_{A}Z_{A}^{T})\leq\frac{1}{n}(\sqrt{n}+CK^{2}(\sqrt{M}+(n/M)^{1/4}))^{2}$

with probability at least $1-4\exp(-\sqrt{n/M}).$ Since

$|(1\pm CK^{2}(\sqrt{M/n}+(nM)^{-1/4}))^{2}-1|\leq
3CK^{2}(\sqrt{M/n}+(nM)^{-1/4})\leq\frac{t}{\sqrt{M}},$

for all $t>0,$ and $n\geq n(t),$ it follows that
$\sqrt{M}\cdot||\frac{1}{n}Z_{A}Z_{A}^{T}-I||\xrightarrow[]{p}0.$

$2.$ Second term: for $n$ sufficiently large, almost surely

$\sqrt{M}\cdot||T\mathcal{M}(\hat{l}_{\nu}I-\mathcal{M})^{-1}T^{T}||\leq\sqrt{M}\cdot||\frac{1}{n}Z_{A}Z_{A}^{T}||\cdot||\mathcal{M}(\hat{l}_{\nu}I-\mathcal{M})^{-1}||\leq\sqrt{M}\cdot\frac{4c_{K,\gamma}}{l_{\nu}}\leq\frac{2c_{K,\gamma}}{c_{\nu}}\cdot\frac{M}{n},$

and thus
$\sqrt{M}\cdot||T\mathcal{M}(\hat{l}_{\nu}I-\mathcal{M})^{-1}T^{T}||\xrightarrow[]{p}0.$

$3.$ Third term: from (4.6),

$|\frac{\hat{l}_{\nu}}{l_{\nu}}-1|\leq|\frac{\lambda_{\nu}(\mathcal{A})}{l_{\nu}}-1|+\frac{\lambda_{1}(\mathcal{B})}{l_{\nu}}.$

As $||\mathcal{B}||\leq c_{K,\gamma}$ almost surely,
$\sqrt{M}\cdot\frac{\lambda_{1}(\mathcal{B})}{l_{\nu}}\leq\frac{2c_{K,\gamma}}{c_{\nu}}\cdot\frac{M}{n},$
while (4.4) gives

$|\frac{\lambda_{\nu}(\mathcal{A})}{l_{\nu}}-1|\leq|\lambda_{\nu}(\frac{1}{n}\sum_{1\leq
i\leq\nu}{Z_{i}^{T}Z_{i}})-1|+|\lambda_{1}(\frac{1}{n}\sum_{\nu\leq i\leq
M}{Z_{i}^{T}Z_{i}})-1|;$

reasoning as above for the first term, it follows that upon multiplication
with $\sqrt{M}$ these upper bounds tend to zero in probability, and so the
third term does as well.

Proof of (7.15): Recall that

$l_{\nu}\beta_{\nu}^{2}-\frac{l_{\nu}}{n}\sum_{k\neq\nu}{\frac{l_{k}l_{\nu}}{(l_{k}-l_{\nu})^{2}}}=\frac{l_{\nu}}{n}\sum_{k\neq\nu}{\frac{l_{k}l_{\nu}}{(l_{k}-l_{\nu})^{2}}[(\frac{1}{\sqrt{n}}z_{k}^{T}z_{\nu})^{2}-1]}+l_{\nu}\sum_{k\neq\nu}{\frac{l_{k}l_{\nu}}{(l_{k}-l_{\nu})^{2}}(t_{k}^{T}\mathcal{M}(\hat{l}_{\nu}I-\mathcal{M})^{-1}t_{\nu})^{2}}+$
$+2l_{\nu}\sum_{k\neq\nu}{\frac{l_{k}l_{\nu}}{(l_{k}-l_{\nu})^{2}}\cdot\frac{1}{n}z_{k}^{T}z_{\nu}\cdot
t_{k}^{T}\mathcal{M}(\hat{l}_{\nu}I-\mathcal{M})^{-1}t_{\nu}}.$

Second term: Inequality (8.4) yields this sum tends to zero in probability as
for $n$ sufficiently large,

$l_{\nu}\sum_{k\neq\nu}{\frac{l_{k}l_{\nu}}{(l_{k}-l_{\nu})^{2}}(t_{k}^{T}\mathcal{M}(\hat{l}_{\nu}I-\mathcal{M})^{-1}t_{\nu})^{2}}\leq
2c_{\nu}\cdot
c(\epsilon_{0})\cdot\frac{n}{\sqrt{M}}\sum_{k\neq\nu}{(t_{k}^{T}\mathcal{M}(\hat{l}_{\nu}I-\mathcal{M})^{-1}t_{\nu})^{2}}\xrightarrow[]{p}0,$

and

$\frac{t}{\sqrt{M}}\cdot l_{\nu}^{2}\geq\frac{tc_{\nu}^{2}n^{2}}{4M^{3/2}}\geq
2,\hskip
14.22636ptMl_{\nu}^{4}\cdot(\frac{t}{\sqrt{M}})^{2}=t^{2}l_{\nu}^{4},\hskip
14.22636ptMl_{\nu}^{2}\cdot\frac{t}{\sqrt{M}}\geq tl_{\nu}^{2}.$

Third term: Cauchy-Schwarz inequality implies the square of this sum is upper
bounded by

$4l^{2}_{\nu}\sum_{k\neq\nu}{\frac{l_{k}l_{\nu}}{(l_{k}-l_{\nu})^{2}}(t_{k}^{T}\mathcal{M}(\hat{l}_{\nu}I-\mathcal{M})^{-1}t_{\nu})^{2}}\cdot\sum_{k\neq\nu}{\frac{l_{k}l_{\nu}}{(l_{k}-l_{\nu})^{2}}\cdot(\frac{1}{n}z_{k}^{T}z_{\nu})^{2}}.$

Again using (8.4), it can be shown that

$\sqrt{M}l_{\nu}\sum_{k\neq\nu}{\frac{l_{k}l_{\nu}}{(l_{k}-l_{\nu})^{2}}(t_{k}^{T}\mathcal{M}(\hat{l}_{\nu}I-\mathcal{M})^{-1}t_{\nu})^{2}}\xrightarrow[]{p}0,$

because for $n$ sufficiently large,

$Ml_{\nu}^{4}\cdot(\frac{t}{M})^{2}=t^{2}\cdot\frac{l_{\nu}^{4}}{M}\geq\frac{t^{2}c_{\nu}^{2}n^{4}}{4M^{3}}\geq
t^{2}n,\hskip 14.22636ptMl_{\nu}^{2}\cdot\frac{t}{M}\geq tl_{\nu}^{2},\hskip
14.22636pt\frac{t}{M}\cdot l_{\nu}^{2}\geq\frac{tc_{\nu}^{2}n^{2}}{4M^{2}}\geq
2,$

and since

$\mathbb{E}[\frac{l_{\nu}}{\sqrt{M}}\sum_{k\neq\nu}{\frac{l_{k}l_{\nu}}{(l_{k}-l_{\nu})^{2}}\cdot(\frac{1}{n}z_{k}^{T}z_{\nu})^{2}}]=\frac{l_{\nu}}{n\sqrt{M}}\sum_{k\neq\nu}{\frac{l_{k}l_{\nu}}{(l_{k}-l_{\nu})^{2}}\leq
c(\epsilon_{0})\cdot\frac{\sqrt{M}l_{\nu}}{n}}\leq 2c_{\nu}\cdot
c(\epsilon_{0}),$

the third term in the decomposition above tends to zero in probability
employing the simple fact (7.7).

Proof of (7.16): We use a CLT for martingales from Billingsley [7] (theorem
$35.12$) stated below:

Let for each $n\in\mathbb{N},$ $(Y_{ni})_{1\leq i\leq k_{n}}$ be a family of
martingale differences for the filtration
$\mathcal{F}_{n1}\subset\mathcal{F}_{n2}\subset...\subset\mathcal{F}_{nk_{n}}$
that have finite second moments. Suppose that for some $\sigma>0,$ as
$n\to\infty,$

$\sum_{1\leq i\leq
k_{n}}{\mathbb{E}[Y_{ni}^{2}|\mathcal{F}_{n(i-1)}]}\xrightarrow[]{p}\sigma^{2},$

and for any $\epsilon>0,$

$\sum_{1\leq i\leq
k_{n}}{\mathbb{E}[Y_{ni}^{2}\cdot\chi_{|Y_{ni}|\geq\epsilon}]}\to 0.$

Then

$\sum_{1\leq i\leq k_{n}}{Y_{ni}}\Rightarrow N(0,\sigma^{2}).$

For simplicity, denote by

$c_{k}=\frac{l_{k}l_{\nu}}{(l_{k}-l_{\nu})^{2}},k\neq\nu;$

consider the nested sigma algebras

$\sigma(z_{\nu}),\sigma(z_{\nu},z_{l},l\neq\nu,l\leq k),k\neq\nu,1\leq k\leq
M,$

denote them by

$\mathcal{F}_{n1}\subset\mathcal{F}_{n2}\subset...\subset\mathcal{F}_{nM},$

and take the martingale differences

$Y_{n1}=\mathbb{E}[T_{n}|\mathcal{F}_{n1}],Y_{ni}=\mathbb{E}[T_{n}|\mathcal{F}_{ni}]-\mathbb{E}[T_{n}|\mathcal{F}_{n(i-1)}],2\leq
i\leq M.$

Then

$\sum_{1\leq i\leq M}{\mathbb{E}[Y^{2}_{ni}]}=\mathbb{E}[(\sum_{1\leq i\leq
M}{Y_{ni}})^{2}]=\mathbb{E}[T_{n}^{2}]\to 2\sigma_{\nu}>0$

since

$\mathbb{E}[T_{n}^{2}]=\frac{1}{M}\sum_{k\neq\nu}{c_{k}^{2}(\mathbb{E}[(\frac{1}{\sqrt{n}}z_{k}^{T}z_{\nu})^{2}-1)^{2}]-2)}+\frac{2}{M}\sum_{k\neq\nu}{c_{k}^{2}}+\frac{1}{M}\sum_{k\neq
l,k\neq\nu,l\neq\nu}{c_{k}c_{l}\cdot\frac{1}{n}(\mathbb{E}[z_{11}^{4}]-1)},$
$|\mathbb{E}[T_{n}^{2}]-\frac{2}{M}\sum_{k\neq\nu}{c_{k}^{2}}|\leq\frac{c(K)}{Mn}\sum_{k\neq\nu}{c_{k}^{2}}+\frac{1}{M}\cdot
M^{2}c(\epsilon_{0})\cdot\frac{c(K)}{n}\leq
2c(K)c(\epsilon_{0})\cdot\frac{M}{n}\to 0,$

as by conditioning on $z_{k},z_{l}$ for $k\neq l$

$\mathbb{E}[((\frac{1}{\sqrt{n}}z_{k}^{T}z_{\nu})^{2}-1)\cdot((\frac{1}{\sqrt{n}}z_{l}^{T}z_{\nu})^{2}-1)]=\mathbb{E}[(\frac{1}{n}||z_{\nu}||^{2}-1)^{2}]=\frac{1}{n^{2}}\mathbb{E}[||z_{\nu}||^{4}]-1=\frac{1}{n}(\mathbb{E}[z_{11}^{4}]-1),$

and

$\mathbb{E}[((\frac{1}{\sqrt{n}}z_{k}^{T}z_{\nu})^{2}-1)^{2}]=\mathbb{E}[\frac{1}{n^{2}}(z_{k}^{T}z_{\nu})^{4}]-1=\frac{1}{n}\mathbb{E}([z_{11}^{4}])^{2}+\frac{1}{n^{2}}\cdot
6\binom{n}{2}-1\leq 2+\frac{c(K)}{n}.$

Thus, it suffices to show that

$\sum_{1\leq i\leq
M}{(\mathbb{E}[Y_{ni}^{2}|\mathcal{F}_{n(i-1)}]-\mathbb{E}[Y_{ni}^{2}])}\xrightarrow[]{p}0,$
(7.18) $\sum_{1\leq i\leq
M}{\mathbb{E}[Y_{ni}^{2}\cdot\chi_{|Y_{ni}|\geq\epsilon}]}\to 0.$ (7.19)

Notice that

$\mathbb{E}[T_{n}|\mathcal{F}_{n1}]=\frac{1}{\sqrt{M}}\sum_{k\neq\nu}{c_{k}}\cdot(\frac{1}{n}||z_{\nu}||^{2}-1),$
$\mathbb{E}[T_{n}|\mathcal{F}_{ni}]=\frac{1}{\sqrt{M}}\sum_{k\in
S_{i}}{c_{k}((\frac{1}{\sqrt{n}}z_{\nu}^{T}z_{k})^{2}-1)}+\frac{1}{\sqrt{M}}\sum_{k\neq\nu,k\not\in
S_{i}}{c_{i}(\frac{1}{n}||z_{\nu}||^{2}-1)},\hskip 2.84544pt2\leq i\leq M,$

for $S_{i}$ the set of the smallest $i-1$ elements of $\\{k,k\neq\nu\\},$ from
which

$Y_{n1}=\frac{1}{\sqrt{M}}\sum_{k\neq\nu}{c_{k}}(\frac{1}{n}||z_{\nu}||^{2}-1),Y_{ni}=\frac{c_{\alpha(i)}}{\sqrt{M}}[((\frac{1}{\sqrt{n}}z_{\nu}^{T}z_{\alpha(i)})^{2}-1)-(\frac{1}{n}||z_{\nu}||^{2}-1)],\hskip
2.84544pt2\leq i\leq M,$ (7.20)

for $\alpha(i)=i-1,\hskip 2.84544pt2\leq i\leq\nu,$ and $\alpha(i)=i,\hskip
2.84544pt\nu<i\leq M.$

We proceed with (7.18). Using that $z_{l},l\neq\nu$ are i.i.d. and independent
of $z_{\nu},$ it follows that for $2\leq k\leq M,$

$\mathbb{E}[Y_{nk}^{2}|\mathcal{F}_{n(k-1)}]=\frac{c_{\alpha(k)}^{2}}{M}\cdot(\mathbb{E}[((\frac{1}{\sqrt{n}}z_{\nu}^{T}z_{\nu-1})^{2}-1)^{2}|z_{\nu}]-(\frac{1}{n}||z_{\nu}||^{2}-1)^{2})=$
$=\frac{c_{\alpha(k)}^{2}}{M}\cdot(\mathbb{E}[((\frac{1}{\sqrt{n}}z_{\nu}^{T}z_{\nu-1})^{2})^{2}|z_{\nu}]+1-2\cdot\frac{1}{n}||z_{\nu}||^{2}-(\frac{1}{n}||z_{\nu}||^{2}-1)^{2})=$
$=\frac{c_{\alpha(k)}^{2}}{M}(\frac{\mathbb{E}[z_{11}^{4}]}{n^{2}}\sum_{1\leq
i\leq n}{z_{\nu i}^{4}}+\frac{6}{n^{2}}\sum_{1\leq i<j\leq n}{z_{\nu
i}^{2}z_{\nu j}^{2}}-(\frac{1}{n}||z_{\nu}||^{2})^{2}),$

which gives

$|\sum_{2\leq k\leq
M}{(\mathbb{E}[Y_{nk}^{2}|\mathcal{F}_{n(k-1)}]-\mathbb{E}[Y_{nk}^{2}])}|\leq
c(\epsilon_{0})\cdot\frac{\mathbb{E}[z_{11}^{4}]}{n^{2}}|\sum_{1\leq i\leq
n}{(z_{\nu i}^{4}-\mathbb{E}[z_{11}^{4}])}|+$
$+c(\epsilon_{0})\cdot\frac{6}{n^{2}}|\sum_{1\leq i<j\leq n}{(z_{\nu
i}^{2}z_{\nu
j}^{2}-1)}|+c(\epsilon_{0})\cdot|\frac{1}{n^{2}}||z_{\nu}||^{4}-\mathbb{E}[\frac{1}{n^{2}}||z_{\nu}||^{4}]|.$

Consider each term in this sum:

$1.\hskip 5.69046pt\frac{\mathbb{E}[z_{11}^{4}]}{n^{2}}\sum_{1\leq i\leq
n}{(z_{\nu i}^{4}-\mathbb{E}[z_{11}^{4}])}\xrightarrow[]{a.s.}0,$

as its second moment is bounded from above by $\frac{c(K)}{n^{3}}$ since
$\mathbb{E}[z_{11}^{4}],\mathbb{E}[z_{11}^{8}]\leq c(K),$ and the independence
of $z_{\nu i},1\leq i\leq n;$

$2.\hskip 5.69046pt\frac{6}{n^{2}}\sum_{1\leq i<j\leq n}{(z_{\nu i}^{2}z_{\nu
j}^{2}-1)}=3(\frac{1}{n}\sum_{1\leq i\leq n}{z_{\nu
i}^{2}})^{2}-3+\frac{3}{n}-\frac{3}{n^{2}}\sum_{1\leq i\leq n}{z_{\nu
i}^{4}}\xrightarrow[]{p}0,$

using that the expectation of the last term is bounded from above by
$\frac{c(K)}{n}$ and

$3(\frac{1}{n}\sum_{1\leq i\leq n}{z_{\nu i}^{2}})^{2}-3\xrightarrow[]{a.s.}0$

from Hanson-Wright inequality (HW);

$3.\hskip
5.69046pt\frac{1}{n^{2}}||z_{\nu}||^{4}-\mathbb{E}[\frac{1}{n^{2}}||z_{\nu}||^{4}]=(\frac{1}{n}||z_{\nu}||^{2})^{2}-1-\frac{1}{n}(\mathbb{E}[z_{11}^{4}]-1)\xrightarrow[]{a.s.}0.$

In what regards (7.19), notice that for $2\leq k\leq M,$

$Y_{nk}^{2}\leq\frac{c(\epsilon_{0})}{M}\cdot(((\frac{1}{\sqrt{n}}z_{\nu}^{T}z_{\alpha(k)})^{2}-1)-(\frac{1}{n}||z_{\nu}||^{2}-1))^{2},$

from which

$\sum_{2\leq k\leq M}{\mathbb{E}[Y_{nk}^{2}\chi_{|Y_{nk}|\geq\epsilon}]}\leq
c(\epsilon_{0})\mathbb{E}[H_{n}^{2}\chi_{|H_{n}|\geq\epsilon\sqrt{\frac{M}{c(\epsilon_{0})}}}],$
(7.21)

for

$H_{n}=(\frac{1}{\sqrt{n}}z_{1}^{T}z_{2})^{2}-\frac{1}{n}||z_{1}||^{2}.$

Using $(a+b)^{4}\leq 8(a^{4}+b^{4}),$

$H_{n}^{4}\leq
8(\frac{1}{\sqrt{n}}z_{1}^{T}z_{2})^{8}+8(\frac{1}{n}||z_{1}||^{2})^{4},$

which together with $\frac{1}{\sqrt{n}}z_{1}^{T}z_{2}\Rightarrow N(0,1)$
(employing Lindeberg’s CLT as in 5.1) and (4.2) yields for $n$ large enough

$\mathbb{E}[H_{n}^{4}]\leq 8\cdot 2\cdot
8!!+8\cdot\mathbb{E}[(\frac{1}{n}||z_{1}||^{2})^{4}]\leq 8\cdot 2\cdot
8!!+8\cdot(8+8\mathbb{E}[(\frac{1}{n}||z_{1}||^{2}-1)^{4}])\leq 20\cdot 8!!,$

implying

$\mathbb{E}[H_{n}^{2}\chi_{|H_{n}|\geq\epsilon\sqrt{\frac{M}{c(\epsilon_{0})}}}]\leq\frac{\mathbb{E}[H_{n}^{4}]}{\frac{M\epsilon^{2}}{c(\epsilon_{0})}}\leq\frac{c(\epsilon_{0})}{\epsilon^{2}}\cdot\frac{20\cdot
8!!}{M}\to 0.$ (7.22)

Lastly, $|Y_{n1}|\leq
c(\epsilon_{0})\cdot\sqrt{M}\cdot|\frac{1}{n}||z_{1}||^{2}-1|,$ (7.20) and
(4.2) give

$\mathbb{E}[Y_{n1}^{2}\chi_{|Y_{n1}|\geq\epsilon}]\leq\frac{\mathbb{E}[Y_{n1}^{4}]}{\epsilon^{2}}\leq\frac{(c(\epsilon_{0}))^{4}\cdot
M^{2}}{\epsilon^{2}}\cdot\frac{c(K)}{n^{2}}\to 0.$ (7.23)

Hence (7.19) ensues from (7.21), (7.22), and (7.23).

###  7.5 Proof of Theorem 6

Proposition 1 yields the claims of the theorem for $(a),(b)(i)$ because if
$a_{n},b_{n},c_{n}$ are random variables defined on the same space with

$a_{n}\xrightarrow[]{a.s}1,\hskip 5.69046ptb_{n}\Rightarrow b,\hskip
5.69046ptc_{n}\xrightarrow[]{a.s.}c\in\mathbb{R},$

then $a_{n}b_{n}\Rightarrow b$ (from Slutsky’s lemma), and
$a_{n}c_{n}\xrightarrow[]{a.s.}c.$ Hence it suffices to consider $(b)(ii)$ and
$(c)(ii).$ As the former is simpler than the latter, we prove them in reverse
order.

$(c)(ii)$ Theorem 5, Proposition 1, and Slutsky’s lemma give

$\hat{l}_{\nu}(1-<p_{\nu},u_{\nu}>^{2})-\frac{\hat{l}_{\nu}}{n}\sum_{k\neq\nu}{\frac{l_{k}l_{\nu}}{(l_{k}-l_{\nu})^{2}}}\Rightarrow\gamma+c_{\nu}N(0,2\sigma_{\nu}),$

and by applying Slutsky’s lemma anew, it suffices to show

$\frac{\hat{l}_{\nu}}{n}\sum_{k\neq\nu}{\frac{l_{k}l_{\nu}}{(l_{k}-l_{\nu})^{2}}}-\frac{\hat{l}_{\nu}}{n}\sum_{k\neq\nu}{\frac{\hat{l}_{k}\hat{l}_{\nu}}{(\hat{l}_{k}-\hat{l}_{\nu})^{2}}}\xrightarrow[]{p}0$

(the second sum is defined almost surely reasoning as in the proof of Theorem
3), which in turn is equivalent, due to Proposition 1, to

$\frac{1}{\sqrt{M}}\cdot(\sum_{k\neq\nu}{\frac{l_{k}l_{\nu}}{(l_{k}-l_{\nu})^{2}}}-\sum_{k\neq\nu}{\frac{\hat{l}_{k}\hat{l}_{\nu}}{(\hat{l}_{k}-\hat{l}_{\nu})^{2}}})\xrightarrow[]{p}0.$
(7.24)

Similarly to the argument for Theorem 3, we use two intermediary steps to
obtain (7.24):

$\frac{1}{\sqrt{M}}\cdot(\sum_{k\neq\nu}{\frac{l_{k}l_{\nu}}{(l_{k}-l_{\nu})^{2}}}-\sum_{k\neq\nu}{\frac{l_{k}\hat{l}_{\nu}}{(l_{k}-\hat{l}_{\nu})^{2}}})\xrightarrow[]{p}0,$
(7.25)
$\frac{1}{\sqrt{M}}\cdot(\sum_{k\neq\nu}{\frac{l_{k}\hat{l}_{\nu}}{(l_{k}-\hat{l}_{\nu})^{2}}}-\sum_{k\neq\nu}{\frac{\hat{l}_{k}\hat{l}_{\nu}}{(\hat{l}_{k}-\hat{l}_{\nu})^{2}}})\xrightarrow[]{p}0,$
(7.26)

with these new sums almost surely well-defined.

We begin with the proof of (7.25). Since

$\frac{ax}{(a-x)^{2}}-\frac{ay}{(a-y)^{2}}=\frac{a(x-y)(a^{2}-xy)}{(a-x)^{2}\cdot(a-y)^{2}},$
(7.27)

it follows that for $k\neq\nu,$

$\frac{l_{k}l_{\nu}}{(l_{k}-l_{\nu})^{2}}-\frac{l_{k}\hat{l}_{\nu}}{(l_{k}-\hat{l}_{\nu})^{2}}=\frac{l_{k}(l_{\nu}-\hat{l}_{\nu})(l_{k}^{2}-l_{\nu}\hat{l}_{\nu})}{(l_{k}-l_{\nu})^{2}\cdot(l_{k}-\hat{l}_{\nu})^{2}}=(1-\frac{\hat{l}_{\nu}}{l_{\nu}})\cdot\frac{\frac{l_{k}}{l_{\nu}}\cdot((\frac{l_{k}}{l_{\nu}})^{2}-\frac{\hat{l}_{\nu}}{l_{\nu}})}{(\frac{l_{k}}{l_{\nu}}-1)^{2}\cdot(\frac{l_{k}}{l_{\nu}}-\frac{\hat{l}_{\nu}}{l_{\nu}})^{2}},$

from which Proposition 1 and
$\frac{l_{k}}{l_{\nu}}\in[0,\frac{1}{1+\epsilon_{0}}]\cup[1+\epsilon_{0},\infty)$
yield with probability one

$|\frac{l_{k}l_{\nu}}{(l_{k}-l_{\nu})^{2}}-\frac{l_{k}\hat{l}_{\nu}}{(l_{k}-\hat{l}_{\nu})^{2}}|\leq
c(\epsilon_{0})\cdot|\frac{\hat{l}_{\nu}}{l_{\nu}}-1|,$ (7.28)
$|\frac{1}{\sqrt{M}}\cdot(\sum_{k\neq\nu}{\frac{l_{k}l_{\nu}}{(l_{k}-l_{\nu})^{2}}}-\sum_{k\neq\nu}{\frac{l_{k}\hat{l}_{\nu}}{(l_{k}-\hat{l}_{\nu})^{2}}})|\leq
c(\epsilon_{0})\cdot\sqrt{M}\cdot|\frac{\hat{l}_{\nu}}{l_{\nu}}-1|\xrightarrow[]{p}0,$

where we have used the proof of (7.17) for the last convergence.

We continue with (7.26): (7.27) gives for $k\neq\nu,$

$\frac{l_{k}\hat{l}_{\nu}}{(l_{k}-\hat{l}_{\nu})^{2}}-\frac{\hat{l}_{k}\hat{l}_{\nu}}{(\hat{l}_{k}-\hat{l}_{\nu})^{2}}=\frac{\hat{l}_{\nu}(l_{k}-\hat{l}_{k})(\hat{l}_{\nu}^{2}-l_{k}\hat{l}_{k})}{(l_{k}-\hat{l}_{\nu})^{2}\cdot(\hat{l}_{k}-\hat{l}_{\nu})^{2}}=(1-\frac{\hat{l}_{k}}{l_{k}})\cdot\frac{\hat{l}_{\nu}l_{k}(\hat{l}_{\nu}^{2}-l_{k}\hat{l}_{k})}{(l_{k}-\hat{l}_{\nu})^{2}\cdot(\hat{l}_{k}-\hat{l}_{\nu})^{2}}.$
(7.29)

Arguing as for (6.9), with probability one,

$|\frac{\hat{l}_{\nu}l_{k}(\hat{l}_{\nu}^{2}-l_{k}\hat{l}_{k})}{(l_{k}-\hat{l}_{\nu})^{2}\cdot(\hat{l}_{k}-\hat{l}_{\nu})^{2}}|=|\frac{\frac{\hat{l}_{\nu}}{l_{\nu}}\cdot\frac{l_{k}}{l_{\nu}}\cdot((\frac{\hat{l}_{\nu}}{l_{\nu}})^{2}-\frac{\hat{l}_{k}}{l_{k}}\cdot(\frac{l_{k}}{l_{\nu}})^{2})}{(\frac{l_{k}}{l_{\nu}}-\frac{\hat{l}_{\nu}}{l_{\nu}})^{2}\cdot(\frac{l_{k}}{l_{\nu}}\cdot\frac{\hat{l}_{k}}{l_{k}}-\frac{\hat{l}_{\nu}}{l_{\nu}})^{2}}|\leq
c(\epsilon_{0},K,\gamma)\cdot\min{(\frac{l_{k}}{l_{\nu}},\frac{l_{\nu}}{l_{k}})},$
(7.30)

while reasoning as for (6.10) yields that for any $t>0,n\geq n(t),$

$\mathbb{P}(|\frac{\hat{l}_{k}}{l_{k}}-1|\cdot\min{(\frac{l_{k}}{l_{\nu}},\frac{l_{\nu}}{l_{k}})}\geq\frac{t}{\sqrt{M}})\leq
6\exp(-\sqrt{n}),$ (7.31)

(using

$\frac{l_{\nu}}{\sqrt{M}}\geq\frac{c_{\nu}}{2}\cdot\frac{n}{M}\to\infty,\hskip
5.69046pt\sqrt{M}\cdot(\frac{1}{n^{1/4}}+\sqrt{\frac{M}{n}})=\frac{M^{1/2}}{n^{1/4}}+\frac{M}{n^{1/2}}\to
0.)$

Finally, (7.29), (7.30), and (7.31) give for any $t>0,n\geq n(t),$

$\mathbb{P}(\frac{1}{\sqrt{M}}\cdot|\sum_{k\neq\nu}{\frac{l_{k}\hat{l}_{\nu}}{(l_{k}-\hat{l}_{\nu})^{2}}}-\sum_{k\neq\nu}{\frac{\hat{l}_{k}\hat{l}_{\nu}}{(\hat{l}_{k}-\hat{l}_{\nu})^{2}}}|\geq
t)\leq\sum_{k\neq\nu}{\mathbb{P}(|\frac{\hat{l}_{k}}{l_{k}}-1|\cdot\min{(\frac{l_{k}}{l_{\nu}},\frac{l_{\nu}}{l_{k}})}\geq\frac{t}{c(\epsilon_{0},K,\gamma)\sqrt{M}})},$

upper bounded by $6M\exp(-\sqrt{n}),$ from which (7.26) follows.

$(b)(ii)$ The arguments are analogous to the ones employed in $(c)(ii):$ it
suffices to show

$\frac{\hat{l}_{\nu}}{n}\sum_{k\neq\nu}{\frac{l_{k}l_{\nu}}{(l_{k}-l_{\nu})^{2}}}-\frac{\hat{l}_{\nu}}{n}\sum_{k\neq\nu}{\frac{\hat{l}_{k}\hat{l}_{\nu}}{(\hat{l}_{k}-\hat{l}_{\nu})^{2}}}\xrightarrow[]{p}0,$

which we split into

$\frac{1}{M}\cdot(\sum_{k\neq\nu}{\frac{l_{k}l_{\nu}}{(l_{k}-l_{\nu})^{2}}}-\sum_{k\neq\nu}{\frac{l_{k}\hat{l}_{\nu}}{(l_{k}-\hat{l}_{\nu})^{2}}})\xrightarrow[]{p}0,$
(7.32)
$\frac{1}{M}\cdot(\sum_{k\neq\nu}{\frac{l_{k}\hat{l}_{\nu}}{(l_{k}-\hat{l}_{\nu})^{2}}}-\sum_{k\neq\nu}{\frac{\hat{l}_{k}\hat{l}_{\nu}}{(\hat{l}_{k}-\hat{l}_{\nu})^{2}}})\xrightarrow[]{p}0.$
(7.33)

(7.32) is immediate from (7.28) and Proposition 1, while (7.33) is an easy
consequence of the analogue of (7.31):

$\mathbb{P}(|\frac{\hat{l}_{k}}{l_{k}}-1|\cdot\min{(\frac{l_{k}}{l_{\nu}},\frac{l_{\nu}}{l_{k}})}\geq
t)\leq 6\exp(-\sqrt{n}),$ (7.34)

for any $t>0,n\geq n(t).$

##  8 Auxiliary Lemmas

In virtue of the approach taken in our proofs, the random quantities

$R_{\nu}^{2},\hskip
2.84544pt||\mathcal{R}_{\nu}\mathcal{D}_{\nu}e_{\nu}||,||a_{\nu}-e_{\nu}||,\hskip
2.84544pt\sum_{k\neq\nu}{(\frac{1}{n}z_{k}^{T}z_{\nu})^{2}},\hskip
2.84544pt\sum_{k\neq\nu}{(t_{k}^{T}\mathcal{M}(\hat{l}_{\nu}I-\mathcal{M})^{-1}t_{\nu})^{2}},$

where $z_{1},\hskip 1.42271pt...\hskip 1.42271pt,z_{M}\in\mathbb{R}^{n}$
denote the rows of $Z_{A},$ and $t_{1},\hskip 1.42271pt...\hskip
1.42271pt,t_{M}\in\mathbb{R}^{N-M}$ the columns of $T,$ govern both the
fluctuations of the eigenvalues and the consistency rates of eigenvectors.
Consequently, we collect some results regarding their behavior that play a
central role in our analysis.

Lemma 1 is primarily concerned with
$\tilde{T}=T^{T}\mathcal{M}(\hat{l}_{\nu}I-\mathcal{M})^{-1}T,$ an essential
matrix for Theorem 1 (see subsection (3.1)) as well as for Theorem 5: its
justification relies on switching from $(\hat{l}_{\nu}I-\mathcal{M})^{-1}$ to
$(l_{\nu}I-\mathcal{M})^{-1}$ since this allows us to employ concentration
inequalities for sums depending on the entries of

$T^{T}\mathcal{M}(l_{\nu}I-\mathcal{M})^{-1}T=\frac{1}{n}Z_{A}H\mathcal{M}(l_{\nu}I-\mathcal{M})^{-1}H^{T}Z_{A}^{T},$

the key observation being that $Z_{A}$ and
$H\mathcal{M}(l_{\nu}I-\mathcal{M})^{-1}H^{T}$ are independent. Lemma 2
relates $||a_{\nu}-e_{\nu}||$ to
$||\mathcal{R}_{\nu}\mathcal{D}_{\nu}e_{\nu}||,$ an object more amenable to
scrutiny (anew $\tilde{T}$ comes into play as it directly influences
$||\mathcal{R}_{\nu}\mathcal{D}_{\nu}e_{\nu}||$): its proof follows from a
decomposition of $a_{\nu}-e_{\nu}$ from Paul [14]. Lemma 3, used repeatedly in
Theorem 5, renders the rate of $R_{\nu}^{2}\to 0$ explicit: its justification
is a careful analysis of (1.6).

We state next these results and present their proofs in subsections 8.1, 8.2,
and 8.3, respectively.

###### Lemma 1

Under the assumptions of Proposition 1, as $n\to\infty,$

$(a)\hskip
5.69046ptl_{\nu}^{4}\sum_{k\neq\nu}{(t_{k}^{T}\mathcal{M}(\hat{l}_{\nu}I-\mathcal{M})^{-2}t_{\nu})^{2}}\xrightarrow[]{a.s.}0,\hskip
14.22636pt(b)\hskip
5.69046pt\frac{n}{M}\sum_{k\neq\nu}{(t_{k}^{T}\mathcal{M}(\hat{l}_{\nu}I-\mathcal{M})^{-1}t_{\nu})^{2}}\xrightarrow[]{p}0.$

###### Lemma 2

Under the assumptions of Proposition 2, and $<p_{A,\nu},u_{A,\nu}>\geq 0,$ for
$\beta_{\nu}=||\mathcal{R}_{\nu}\mathcal{D}_{\nu}e_{\nu}||,$ as $n\to\infty,$

$\frac{||a_{\nu}-e_{\nu}||}{\beta_{\nu}}\xrightarrow[]{a.s.}1,$

where if $\beta_{\nu}=0$ for $n$ sufficiently large, then
$||a_{\nu}-e_{\nu}||=0$ as well. Moreover,

$\sum_{n\in\mathbb{N}}{\mathbb{P}(||a_{\nu}-e_{\nu}||\geq
2\beta_{\nu})}<\infty.$

###### Lemma 3

Under the assumptions of Proposition 2, as $n\to\infty,$

$\hat{l}_{\nu}R_{\nu}^{2}-\frac{N}{n}\xrightarrow[]{a.s.}0.$

###  8.1 Proof of Lemma 1

$(a)$ Recall the identity

$\frac{1}{(\hat{l}_{\nu}-x)^{2}}=\frac{1}{(l_{\nu}-x)^{2}}\cdot\frac{1}{(1-\frac{l_{\nu}-\hat{l}_{\nu}}{l_{\nu}-x})^{2}}=\sum_{m\geq
0}{\frac{(m+1)(l_{\nu}-\hat{l}_{\nu})^{m}}{(l_{\nu}-x)^{m+2}}}$

for
$x\in[0,\frac{l_{\nu}}{2}],|\hat{l}_{\nu}-l_{\nu}|<\frac{l_{\nu}}{2}\leq|x-l_{\nu}|,$
and denote by

$s_{k}(i)=t_{k}^{T}\mathcal{M}(l_{\nu}I-\mathcal{M})^{-i-2}t_{\nu}$

for $k\neq\nu,i\geq 0.$ Then as long as
$||\mathcal{M}||\leq\frac{l_{\nu}}{2},$ the sum under consideration can be
rewritten as

$l_{\nu}^{4}\sum_{k\neq\nu}{(t_{k}^{T}\mathcal{M}(\hat{l}_{\nu}I-\mathcal{M})^{-2}t_{\nu})^{2}}=l_{\nu}^{4}\sum_{m\geq
0}{(l_{\nu}-\hat{l}_{\nu})^{m}\sum_{k\neq\nu}{\sum_{0\leq i\leq
m}{(i+1)(m-i+1)s_{k}(i)s_{k}(m-i)}}};$

hence, for any $t>0,$ and $n$ large enough such that $l_{\nu}>2c_{K,\gamma},$

$\mathbb{P}(l_{\nu}^{4}\sum_{k\neq\nu}{(t_{k}^{T}\mathcal{M}(\hat{l}_{\nu}I-\mathcal{M})^{-2}t_{\nu})^{2}}\geq
t)\leq\mathbb{P}(|\hat{l}_{\nu}-l_{\nu}|>\frac{l_{\nu}}{32})+\mathbb{P}(||\mathcal{M}||>c_{K,\gamma})+\mathbb{P}(\frac{1}{n}||z_{\nu}||^{2}>2)+$
$+\sum_{m\geq 0,0\leq i\leq
m}{\mathbb{P}(|\sum_{k\neq\nu}{s_{k}(i)s_{k}(m-i)}|\geq\frac{2^{m-1}t}{l_{\nu}^{m+4}}\hskip
1.42271pt|\hskip 1.42271pt||\mathcal{M}||\leq
c_{K,\gamma},\frac{1}{n}||z_{\nu}||^{2}\leq 2)},$ (8.1)

because if none of the events on the right-hand side happens, then

$l_{\nu}^{4}\sum_{k\neq\nu}{(t_{k}^{T}\mathcal{M}(\hat{l}_{\nu}I-\mathcal{M})^{-2}t_{\nu})^{2}}=l_{\nu}^{4}\sum_{m\geq
0}{(l_{\nu}-\hat{l}_{\nu})^{m}\sum_{k\neq\nu}{\sum_{0\leq i\leq
m}{(i+1)(m-i+1)s_{k}(i)s_{k}(m-i)}}}\leq$ $\leq\sum_{m\geq
0}{\frac{l_{\nu}^{m+4}}{8^{m}}\sum_{0\leq i\leq
m}|{\sum_{k\neq\nu}{s_{k}(i)s_{k}(m-i)}}}|\leq\sum_{m\geq
0}{\frac{l_{\nu}^{m+4}}{8^{m}}\cdot(m+1)\frac{2^{m-1}t}{l_{\nu}^{m+4}}}=t\sum_{m\geq
0}{\frac{m+1}{2\cdot 4^{m}}}=\frac{t}{2}\cdot\frac{1}{(1-\frac{1}{4})^{2}}<t.$

To conclude the proof, it suffices to show the four terms on the right-hand
side in (8.1) are summable as functions of $n\in\mathbb{N}$ from Borel-
Cantelli lemma.

For the first three components in (8.1), their exponential decay ensues from
(6.4) and inequality (SM) as for $n$ large enough,
$\frac{l_{\nu}}{64}>c_{K,\gamma},$ while for the third, Cauchy-Schwarz
inequality yields

$\sum_{m\geq 0,0\leq i\leq
m}{\mathbb{P}(|\sum_{k\neq\nu}{s_{k}(i)s_{k}(m-i)}|\geq\frac{2^{m-1}t}{l_{\nu}^{m+4}}\hskip
1.42271pt|\hskip 1.42271pt||\mathcal{M}||\leq
c_{K,\gamma},\frac{1}{n}||z_{\nu}||^{2}\leq 2)}\leq$ $\leq 2\sum_{m\geq
0,0\leq i\leq
m}{\mathbb{P}(\sum_{k\neq\nu}{s^{2}_{k}(i)}\geq\frac{2^{m-1}t}{l_{\nu}^{2i+4}}\hskip
1.42271pt|\hskip 1.42271pt||\mathcal{M}||\leq
c_{K,\gamma},\frac{1}{n}||z_{\nu}||^{2}\leq 2)}.$ (8.2)

We derive an upper bound for the terms in this new sum using Bernstein’s
inequality (theorem $2.8.2$ in Vershynin [17]): for $t\geq 0$ and
$A\in\mathbb{R}^{n\times n}$ independent of $(z_{k})_{k\neq\nu},$

$\mathbb{P}(|\sum_{k\neq\nu}{((z_{k}^{t}Az_{\nu})^{2}-||Az_{\nu}||^{2})}|\geq
t|A,z_{\nu})\leq
2\exp(-c\min{(\frac{t^{2}}{M\cdot||Az_{\nu}||^{4}},\frac{t}{||Az_{\nu}||^{2}})}):$

from proposition $2.6.1$ in Vershynin [17], conditional on $A,z_{\nu},$

$||z_{k}^{t}Az_{\nu}||_{\psi_{2}}\leq C||Az_{\nu}||,$

which implies using lemma $2.7.6$ that $(z_{k}^{t}Az_{\nu})^{2}$ is
subexponential with

$||(z_{k}^{t}Az_{\nu})^{2}||_{\psi_{1}}\leq C^{2}||Az_{\nu}||^{2},$

where for a subexponential random variable $x,$

$||x||_{\psi_{1}}=\inf{\\{t>0:\mathbb{E}[\exp(|x|/t)]\leq 2\\}}.$

Since $||Az_{\nu}||\leq||A||\cdot||z_{\nu}||,$ this inequality yields

$\mathbb{P}(|\sum_{k\neq\nu}{((z_{k}^{t}Az_{\nu})^{2}-||Az_{\nu}||^{2})}|\geq
t|A,z_{\nu})\leq
2\exp(-c\min{(\frac{t^{2}}{M\cdot||A||^{4}\cdot||z_{\nu}||^{4}},\frac{t}{||A||^{2}\cdot||z_{\nu}||^{2}})}).$
(8.3)

Thus, for
$\mathcal{M}_{i,\nu}=\frac{l_{\nu}^{i+2}}{n}H\mathcal{M}(l_{\nu}I-\mathcal{M})^{-i-2}H^{T},$

$\mathbb{P}(\sum_{k\neq\nu}{s^{2}_{k}(i)}\geq\frac{2^{m-1}t}{l_{\nu}^{2i+4}}|Z_{B},z_{\nu})\leq\mathbb{P}(|\sum_{k\neq\nu}{((z_{k}^{T}\mathcal{M}_{i,\nu}z_{\nu})^{2}-||\mathcal{M}_{i,\nu}z_{\nu}||^{2})}|\geq
2^{m-2}t|Z_{B},z_{\nu})+$
$+\mathbb{P}(M\cdot||\mathcal{M}_{i,\nu}z_{\nu}||^{2}\geq
2^{m-2}t|Z_{B},z_{\nu})\leq
2\exp(-c\min{(\frac{2^{2m-4}t^{2}}{M\cdot||\mathcal{M}_{i,\nu}||^{4}\cdot||z_{\nu}||^{4}},\frac{2^{m-2}t}{||\mathcal{M}_{i,\nu}||^{2}\cdot||z_{\nu}||^{2}}}))+$
$+\mathbb{P}(M\cdot||\mathcal{M}_{i,\nu}z_{\nu}||^{2}\geq
2^{m-2}t|Z_{B},z_{\nu}).$

Furthermore, if $||\mathcal{M}||\leq c_{K,\gamma}\leq
l_{\nu}(1-2^{-\frac{1}{4}}),$ then

$M\cdot||\mathcal{M}_{i,\nu}z_{\nu}||^{2}\leq\frac{M}{n^{2}}\cdot||l_{\nu}^{i+2}\mathcal{M}(l_{\nu}I-\mathcal{M})^{-i-2}||^{2}\cdot||z_{\nu}||^{2}\leq\frac{M}{n^{2}}\cdot
c_{K,\gamma}^{2}\cdot 2^{\frac{i+2}{2}}||z_{\nu}||^{2}$

as $\frac{xl^{i+2}}{(l-x)^{i+2}}\leq c_{K,\gamma}\cdot 2^{\frac{i+2}{4}}$ for
$x\in[0,c_{K,\gamma}].$

Hence for $n$ large enough such that $c_{K,\gamma}\leq
l_{\nu}(1-2^{-\frac{1}{4}}),$

$\sum_{m\geq 0,0\leq i\leq
m}{\mathbb{P}(\sum_{k\neq\nu}{s^{2}_{k}(i)}\geq\frac{2^{m-1}t}{l_{\nu}^{2i+4}}|Z_{B},z_{\nu})}\leq\mathbb{P}(||\mathcal{M}||>c_{K,\gamma}|Z_{B},z_{\nu})+\mathbb{P}(\frac{M}{n^{2}}\cdot||z_{\nu}||^{2}\geq\frac{t}{8c_{K,\gamma}^{2}}|Z_{B},z_{\nu})+$
$+\sum_{m\geq 0,0\leq i\leq
m}{2\exp(-c\min{(\frac{2^{m-6}t^{2}n^{2}}{Mc_{K,\gamma}^{4}\cdot(\frac{1}{n}||z_{\nu}||^{2})^{2}},\frac{2^{\frac{m-6}{2}}tn}{c_{K,\gamma}^{2}\cdot\frac{1}{n}||z_{\nu}||^{2}})})},$

from which for $n$ sufficiently large so that
$\frac{n}{M}\cdot\frac{t}{8c^{2}_{K,\gamma}}>2,$ the sum in (8.2) is at most

$2\sum_{m\geq 0,0\leq i\leq
m}{2\exp(-c\min{(\frac{2^{m-10}t^{2}n^{2}}{Mc_{K,\gamma}^{4}},\frac{2^{\frac{m-7}{2}}tn}{c_{K,\gamma}^{2}})})}.$

Note that this is summable as a function of $n\in\mathbb{N}$ from the
following elementary inequality:

$\sum_{m\geq 0,0\leq i\leq
m}{\exp(-\min(2^{m}a,2^{\frac{m}{2}}b))}\leq\sum_{m\geq
0}{(m+1)(\exp(-2^{m}a)+\exp(-2^{\frac{m}{2}}b))}\leq$ $\leq\sum_{m\geq
0}{(m+1)(\exp(-(m+1)a)+\exp(-(m+1)\frac{b}{2}))}=\frac{e^{-a}}{(1-e^{-a})^{2}}+\frac{e^{-b/2}}{(1-e^{-b/2})^{2}}\leq
4e^{-a}+4e^{-b/2},$

if $e^{-a},e^{-b/2}\leq\frac{1}{2}$ together with $M\leq N\leq 2\gamma n$ for
$n$ sufficiently large.

$(b)$ A similar approach to the one in $(a)$ can be used here:

$\frac{1}{\hat{l}_{\nu}-x}=\frac{1}{l_{\nu}-x}\cdot\frac{1}{1-\frac{l_{\nu}-\hat{l}_{\nu}}{l_{\nu}-x}}=\sum_{m\geq
0}{\frac{(l_{\nu}-\hat{l}_{\nu})^{m}}{(l_{\nu}-x)^{m+1}}}$

for $x\in[0,\frac{l_{\nu}}{2}]$ and
$|\hat{l}_{\nu}-l_{\nu}|<\frac{l_{\nu}}{2}\leq|x-l_{\nu}|;$ if
$||\mathcal{M}||<\frac{l_{\nu}}{2},|\frac{\hat{l}_{\nu}}{l_{\nu}}-1|<\frac{1}{2},$
then

$\frac{n}{M}\sum_{k\neq\nu}{(t_{k}^{T}\mathcal{M}(\hat{l}_{\nu}I-\mathcal{M})^{-1}t_{\nu})^{2}}=\frac{1}{M}\sum_{m\geq
0}{(l_{\nu}-\hat{l}_{\nu})^{m}\sum_{k\neq\nu}{\sum_{0\leq i\leq
m}{s^{\prime}_{k}(i)s^{\prime}_{k}(m-i)}}},$

where
$s^{\prime}_{k}(i)=\sqrt{n}t_{k}^{T}\mathcal{M}(l_{\nu}I-\mathcal{M})^{-i-1}t_{\nu}$
for $k\neq\nu,i\geq 0;$

$\mathbb{P}(\frac{n}{M}\sum_{k\neq\nu}{(t_{k}^{T}\mathcal{M}(\hat{l}_{\nu}I-\mathcal{M})^{-1}t_{\nu})^{2}}\geq
t)\leq\mathbb{P}(|\hat{l}_{\nu}-l_{\nu}|>\frac{l_{\nu}}{8})+\mathbb{P}(||\mathcal{M}||>c_{K,\gamma})+\mathbb{P}(\frac{1}{n}||z_{\nu}||^{2}>2)+$
$+\sum_{m\geq 0,0\leq i\leq
m}{\mathbb{P}(|\sum_{k\neq\nu}{s^{\prime}_{k}(i)s^{\prime}_{k}(m-i)}|\geq\frac{2^{m-1}Mt}{l_{\nu}^{m}}\hskip
1.42271pt|\hskip 1.42271pt||\mathcal{M}||\leq
c_{K,\gamma},\frac{1}{n}||z_{\nu}||^{2}\leq 2)};$

for $0\leq i\leq m,$

$\mathbb{P}(\sum_{k\neq\nu}{(s^{\prime}_{k}(i))^{2}}\geq\frac{2^{m-1}Mt}{l_{\nu}^{2i}}|Z_{B},z_{\nu})\leq\mathbb{P}(||\mathcal{M}^{\prime}_{i,\nu}||^{2}\cdot||z_{\nu}||^{2}\geq\frac{2^{m-2}t}{l_{\nu}^{2i}}|Z_{B},z_{\nu})+$
$+\mathbb{P}(\sum_{k\neq\nu}{((s^{\prime}_{k}(i))^{2}-||\mathcal{M}^{\prime}_{i,\nu}||^{2})}\geq\frac{2^{m-2}Mt}{l_{\nu}^{2i}}|Z_{B},z_{\nu}),$

where
$\mathcal{M}^{\prime}_{i,\nu}=\frac{1}{\sqrt{n}}H\mathcal{M}(l_{\nu}I-\mathcal{M})^{-i-1}H^{T}.$

For the first term in this last inequality, whenever $||\mathcal{M}||\leq
c_{K,\gamma}\leq l_{\nu}(1-2^{-1/4}),$

$||\mathcal{M}^{\prime}_{i,\nu}||^{2}\leq\frac{1}{n}\cdot\frac{c^{2}_{K,\gamma}}{(\frac{l_{\nu}}{2^{1/4}})^{2(i+1)}},$

from which

$\mathbb{P}(||\mathcal{M}^{\prime}_{i,\nu}||^{2}\cdot||z_{\nu}||^{2}\geq\frac{2^{m-2}t}{l_{\nu}^{2i}})\leq\mathbb{P}(\frac{1}{n}||z_{\nu}||^{2}\geq\frac{tl_{\nu}^{2}}{8c^{2}_{K,\gamma}}),$

while for the second, (8.3) yields the following upper bound

$2\exp(-c\min{(\frac{2^{2m-4}M^{2}t^{2}}{Ml_{\nu}^{4i}\cdot||\mathcal{M}^{\prime}_{i,\nu}||^{4}\cdot||z_{\nu}||^{4}},\frac{2^{m-2}Mt}{l_{\nu}^{2i}\cdot||\mathcal{M}^{\prime}_{i,\nu}||^{2}\cdot||z_{\nu}||^{2})})})\leq$
$\leq
2\exp(-c\min{(\frac{2^{m-5}Mt^{2}l_{\nu}^{4}}{c_{K,\gamma}^{4}\cdot(\frac{1}{n}||z_{\nu}||^{2})^{2}},\frac{2^{\frac{m}{2}-3}Mtl_{\nu}^{2}}{c_{K,\gamma}^{2}\cdot\frac{1}{n}||z_{\nu}||^{2}})}).$

Hence for $\frac{tl_{\nu}^{2}}{8c^{2}_{K,\gamma}}>2,$

$\mathbb{P}(\frac{n}{M}\sum_{k\neq\nu}{(t_{k}^{T}\mathcal{M}(\hat{l}_{\nu}I-\mathcal{M})^{-1}t_{\nu})^{2}}\geq
t)\leq\mathbb{P}(|\hat{l}_{\nu}-l_{\nu}|>\frac{l_{\nu}}{8})+\mathbb{P}(||\mathcal{M}||>c_{K,\gamma})+\mathbb{P}(\frac{1}{n}||z_{\nu}||^{2}>2)+$
$+c_{1}\exp(-c_{2}Mt^{2}l_{v}^{4})+c_{1}\exp(-c_{2}Mtl_{\nu}^{2})$ (8.4)

for some $c_{1}>0,c_{2}=c_{2}(K,\gamma)>0,$ and the conclusion then ensues
since $l_{\nu}\to\infty.$

###  8.2 Proof of Lemma 2

Recall (5.3) and (5.4):

$a_{\nu}-e_{\nu}=-\mathcal{R}_{\nu}\mathcal{D}_{\nu}e_{\nu}+r_{\nu},$
$r_{\nu}=(<a_{\nu},e_{\nu}>-1)e_{\nu}-\mathcal{R}_{\nu}\mathcal{D}_{\nu}(a_{\nu}-e_{\nu})+(\hat{l}_{\nu}-l_{\nu})\mathcal{R}_{\nu}(a_{\nu}-e_{\nu}),$

(as in Proposition 2, both (1.5) and (1.6) hold almost surely). (5.3) provides

$\beta_{\nu}-||r_{\nu}||\leq||a_{\nu}-e_{\nu}||\leq\beta_{\nu}+||r_{\nu}||,$

while from Proposition 1,
$||\mathcal{R}_{\nu}D_{\nu}||+|\hat{l}_{\nu}-l_{\nu}|\cdot||\mathcal{R}_{\nu}||\xrightarrow[]{a.s.}0,$
and $||a_{\nu}-e_{\nu}||\xrightarrow[]{a.s.}0$ because

$<a_{\nu},e_{\nu}>^{2}\xrightarrow[]{a.s.}1,$

and

$<a_{\nu},e_{\nu}>=\frac{<p_{A,\nu},u_{A,\nu}>}{\sqrt{1-R_{\nu}^{2}}}\geq 0.$

Since $2(1-<a_{\nu},e_{\nu}>)=||a_{\nu}-e_{\nu}||^{2},$ (5.4) gives that for
all $\epsilon\in(0,\frac{1}{2}),$ almost surely for $n$ large enough,

$||r_{\nu}||\leq 2||a_{\nu}-e_{\nu}||^{2}+\epsilon||a_{\nu}-e_{\nu}||\leq
2\epsilon||a_{\nu}-e_{\nu}||,$

from which

$\frac{\beta_{\nu}}{1+2\epsilon}\leq||a_{\nu}-e_{\nu}||\leq\frac{\beta_{\nu}}{1-2\epsilon},$

yielding the first claim. Furthermore, these inequalities for
$\epsilon=\frac{1}{4}$ imply

$\mathbb{P}(||a_{\nu}-u_{\nu}||\geq
2||\beta_{\nu}||)\leq\mathbb{P}(||a_{\nu}-e_{\nu}||\geq\frac{1}{4})+\mathbb{P}(||\mathcal{R}_{\nu}\mathcal{D}_{\nu}||+|\hat{l}_{\nu}-l_{\nu}|\cdot||\mathcal{R}_{\nu}||\geq\frac{1}{4}),$

and using Proposition 2,

$||a_{\nu}-e_{\nu}||^{2}=2-2\sqrt{1-(\mathcal{P}^{\perp}_{\nu}a_{\nu})^{2}},\hskip
14.22636pt||\mathcal{P}^{\perp}_{\nu}a_{\nu}||\leq||\mathcal{R}_{\nu}\mathcal{D}_{\nu}||+|\hat{l}_{\nu}-l_{\nu}|\cdot||\mathcal{R}_{\nu}||,$

from which

$\mathbb{P}(||a_{\nu}-e_{\nu}||\geq\frac{1}{4})\leq\mathbb{P}(||\mathcal{R}_{\nu}\mathcal{D}_{\nu}||+|\hat{l}_{\nu}-l_{\nu}|\cdot||\mathcal{R}_{\nu}||\geq\frac{1}{8})$

since $2-2\sqrt{1-x^{2}}\leq 2x^{2},$ for $x^{2}\leq 1.$

Moreover, Proposition 2 entails that for $t>0$

$\mathbb{P}(||\mathcal{R}_{\nu}\mathcal{D}_{\nu}||+|\hat{l}_{\nu}-l_{\nu}|\cdot||\mathcal{R}_{\nu}||\geq
t)\leq\mathbb{P}(||\mathcal{R}_{\nu}\mathcal{D}_{\nu}||\geq
t/2)+\mathbb{P}(|\hat{l}_{\nu}-l_{\nu}|\cdot||\mathcal{R}_{\nu}||\geq
t/2)\leq$
$\leq\mathbb{P}(||\frac{1}{n}Z_{A}Z_{A}^{T}-I+T^{T}\mathcal{M}(\hat{l}_{\nu}I-\mathcal{M})^{-1}T||\geq
tc(\epsilon_{0},\gamma))+\mathbb{P}(|\frac{\hat{l}_{\nu}}{l_{\nu}}-1|\geq
tc(\epsilon_{0},\gamma)),$

with the second terms are summable as functions of $n\in\mathbb{N}$ from
(6.4), and

$\mathbb{P}(||\frac{1}{n}Z_{A}Z_{A}^{T}-I+T^{T}\mathcal{M}(\hat{l}_{\nu}I-\mathcal{M})^{-1}T||\geq
t)\leq\mathbb{P}(||\frac{1}{n}Z_{A}Z_{A}^{T}-I||\geq\frac{t}{2})+\mathbb{P}(||\frac{1}{n}Z_{A}Z_{A}^{T}||\geq
2)+$
$+\mathbb{P}(||\mathcal{M}(\hat{l}_{\nu}I-\mathcal{M})^{-1}||\geq\frac{t}{4})$

because

$||T^{T}\mathcal{M}(\hat{l}_{\nu}I-\mathcal{M})^{-1}T||\leq||\frac{1}{n}Z_{A}Z_{A}^{T}||\cdot||\mathcal{M}(\hat{l}_{\nu}I-\mathcal{M})^{-1}||,$

which in conjunction with inequality (SM), Proposition 1, and
$l_{\nu}\to\infty$ renders the first terms summable as functions of
$n\in\mathbb{N}$ as well.

###  8.3 Proof of Lemma 3

As in Proposition 2, equation (1.6) holds almost surely:

$a^{T}_{\nu}\Lambda^{1/2}T^{T}\mathcal{M}(\hat{l}_{\nu}I-\mathcal{M})^{-2}T\Lambda^{1/2}a_{\nu}=\frac{R_{\nu}^{2}}{1-R_{\nu}^{2}}.$

We show
$y_{\nu}-\frac{N}{n}=\frac{\hat{l}_{\nu}R_{\nu}^{2}}{1-R_{\nu}^{2}}-\frac{N}{n}\xrightarrow[]{a.s.}0,$
which implies

$\hat{l}_{\nu}R_{\nu}^{2}-\frac{N}{n}=y_{\nu}-R_{\nu}^{2}y_{\nu}-\frac{N}{n}\xrightarrow[]{a.s.}0$

because Proposition 2 entails $R_{\nu}^{2}\xrightarrow[]{a.s.}0.$ Denote by

$E_{\nu}=\hat{l}_{\nu}\Lambda^{1/2}T^{T}\mathcal{M}(\hat{l}_{\nu}I-\mathcal{M})^{-2}T\Lambda^{1/2}.$

Since

$y_{\nu}=a_{\nu}^{T}E_{\nu}a_{\nu},$

it suffices in turn to prove

$e_{\nu}^{T}E_{\nu}e_{\nu}-\frac{N}{n}\xrightarrow[]{a.s.}0,\hskip
14.22636pte_{\nu}^{T}E_{\nu}(a_{\nu}-e_{\nu})\xrightarrow[]{a.s.}0,\hskip
14.22636pt(a_{\nu}-e_{\nu})^{T}E_{\nu}(a_{\nu}-e_{\nu})\xrightarrow[]{a.s.}0.$

$1.$ Consider the first term:

$e_{\nu}^{T}E_{\nu}e_{\nu}=\hat{l}_{\nu}l_{\nu}\cdot
t_{\nu}^{T}\mathcal{M}(\hat{l}_{\nu}I-\mathcal{M})^{-2}t_{\nu}=\frac{l_{\nu}}{\hat{l}_{\nu}}\cdot
t_{\nu}^{T}\mathcal{M}(I-\frac{1}{\hat{l}_{\nu}}\mathcal{M})^{-2}t_{\nu}.$

Hanson-Wright inequality (HW) yields for any $t\geq 0,$

$\mathbb{P}(|t_{\nu}^{T}\mathcal{M}t_{\nu}-\frac{1}{n}tr(\mathcal{M})|\geq
t|Z_{B})\leq
2\exp(-c\min{(\frac{nt^{2}}{K^{4}\cdot||\mathcal{M}||^{2}},\frac{nt}{K^{2}\cdot||\mathcal{M}||})})$

as
$t_{\nu}^{T}\mathcal{M}t_{\nu}=\frac{1}{n}z_{\nu}^{T}H\mathcal{M}H^{T}z_{\nu},tr(H\mathcal{M}H^{T})=tr(\mathcal{M}),||H\mathcal{M}H^{T}||=||\mathcal{M}||$
from which

$\mathbb{P}(|t_{\nu}^{T}\mathcal{M}t_{\nu}-\frac{1}{n}tr(\mathcal{M})|\geq
t)\leq\mathbb{P}(||\mathcal{M}||>c_{K,\gamma})+2\exp(-c\min{(\frac{nt^{2}}{K^{4}c_{K,\gamma}^{2}},\frac{nt}{K^{2}c_{K,\gamma}})}),$

implying with Borel-Cantelli lemma that

$t_{\nu}^{T}\mathcal{M}t_{\nu}-\frac{1}{n}tr(\mathcal{M})\xrightarrow[]{a.s.}0,$

and consequently

$t_{\nu}^{T}\mathcal{M}t_{\nu}-\frac{N}{n}\xrightarrow[]{a.s.}0$ (8.5)

because (4.2) gives

$\frac{1}{n}tr(\mathcal{M})-\frac{N}{n}=\frac{1}{n}tr(\frac{1}{n}Z_{B}Z_{B}^{T})-\frac{N}{n}=-\frac{M}{n}+\frac{1}{n^{2}}\sum_{M+1\leq
i\leq N,1\leq j\leq n}{(z^{2}_{ij}-1)}\xrightarrow[]{a.s.}0:$
$\mathbb{E}[(\frac{1}{n}tr(\mathcal{M})-\frac{N-M}{n})^{2}]\leq\sqrt{\mathbb{E}[(\frac{1}{n}tr(\mathcal{M})-\frac{N-M}{n})^{4}]}\leq\frac{\sqrt{c(K)}}{n(N-M)}\leq\frac{c(K,\gamma)}{n^{2}}.$

Lastly, since almost surely

$0\leq
t_{\nu}^{T}\mathcal{M}(I-\frac{1}{\hat{l}_{\nu}}\mathcal{M})^{-2}t_{\nu}-t_{\nu}^{T}\mathcal{M}t_{\nu}=t_{\nu}^{T}\mathcal{M}((I-\frac{1}{\hat{l}_{\nu}}\mathcal{M})^{-2}-I)t_{\nu}\leq$
$\leq||(I-\frac{1}{\hat{l}_{\nu}}\mathcal{M})^{-2}-I||\cdot
t_{\nu}^{T}\mathcal{M}t_{\nu}\leq\frac{8c_{K,\gamma}}{\hat{l}_{\nu}}\cdot
2\gamma\to 0,$

from

$\frac{1}{(1-x)^{2}}-1=\frac{x(2-x)}{(1-x)^{2}}\leq\frac{2x}{(1/2)^{2}}=8x$

for $0\leq x\leq 1/2,$ and Proposition 2, it follows using Proposition 1 that

$e_{\nu}^{T}E_{\nu}e_{\nu}-\frac{N}{n}=\frac{l_{\nu}}{\hat{l}_{\nu}}\cdot(t_{\nu}^{T}\mathcal{M}(I-\frac{1}{\hat{l}_{\nu}}\mathcal{M})^{-2}t_{\nu}-t_{\nu}^{T}\mathcal{M}t_{\nu}+t_{\nu}^{T}\mathcal{M}t_{\nu}-\frac{N}{n})+(\frac{l_{\nu}}{\hat{l}_{\nu}}-1)\cdot\frac{N}{n}\xrightarrow[]{a.s.}0.$

$2.$ Consider the second term:

$2e_{\nu}^{T}E_{\nu}(a_{\nu}-e_{\nu})=2\hat{l}_{\nu}\sqrt{l_{\nu}}\sum_{k}{t_{k}^{T}\mathcal{M}(\hat{l}_{\nu}I-\mathcal{M})^{-2}t_{\nu}\cdot\sqrt{l_{k}}(a_{\nu}-e_{\nu})_{k}}.$

We have

$\frac{1}{\hat{l}_{\nu}}a_{\nu}^{T}\Lambda a_{\nu}-1\xrightarrow[]{a.s.}0:$
(8.6)

(1.5) implies

$1-\frac{1}{\hat{l}_{\nu}}a_{\nu}^{T}S_{AA}a_{\nu}=\frac{1}{\hat{l}_{\nu}}a^{T}_{\nu}\Lambda^{1/2}T^{T}\mathcal{M}(\hat{l}_{\nu}I-\mathcal{M})^{-1}T\Lambda^{1/2}a_{\nu},$

from which

$1-\frac{1}{\hat{l}_{\nu}}a_{\nu}^{T}S_{AA}a_{\nu}\leq
a^{T}_{\nu}\Lambda^{1/2}T^{T}\mathcal{M}(\hat{l}_{\nu}I-\mathcal{M})^{-2}T\Lambda^{1/2}a_{\nu}=\frac{R_{\nu}^{2}}{1-R_{\nu}^{2}}\xrightarrow[]{a.s.}0,$
$1-\frac{1}{\hat{l}_{\nu}}a_{\nu}^{T}S_{AA}a_{\nu}\geq\frac{1}{2}\cdot
a^{T}_{\nu}\Lambda^{1/2}T^{T}\mathcal{M}(\hat{l}_{\nu}I-\mathcal{M})^{-2}T\Lambda^{1/2}a_{\nu}=\frac{R_{\nu}^{2}}{2(1-R_{\nu}^{2})}\xrightarrow[]{a.s.}0,$

because

$\frac{x}{2(l-x)^{2}}\leq\frac{x}{l(l-x)}\leq\frac{x}{(l-x)^{2}},$

for $l>0,x\in[0,\frac{l}{2}],$ yielding

$1-\frac{1}{\hat{l}_{\nu}}a_{\nu}^{T}S_{AA}a_{\nu}\xrightarrow[]{a.s.}0;$

lastly, notice that

$\frac{1}{\lambda_{\max}(\frac{1}{n}Z_{A}Z_{A}^{T})}\cdot\frac{1}{\hat{l}_{\nu}}a_{\nu}^{T}S_{AA}a_{\nu}\leq\frac{1}{\hat{l}_{\nu}}a_{\nu}^{T}\Lambda
a_{\nu}\leq\frac{1}{\lambda_{\min}(\frac{1}{n}Z_{A}Z_{A}^{T})}\cdot\frac{1}{\hat{l}_{\nu}}a_{\nu}^{T}S_{AA}a_{\nu},$

and
$\lambda_{\min}(\frac{1}{n}Z_{A}Z_{A}^{T}),\lambda_{\max}(\frac{1}{n}Z_{A}Z_{A}^{T})\xrightarrow[]{a.s.}1$
from (SM).

(8.6) can be rewritten as

$\frac{1}{\hat{l}_{\nu}}\sum_{k}{l_{k}((a_{\nu}-e_{\nu})_{k})^{2}}\xrightarrow[]{a.s.}0$
(8.7)

because

$\frac{l_{\nu}}{\hat{l}_{\nu}}(<a_{\nu},e_{\nu}>^{2}-1)-\frac{l_{\nu}}{\hat{l}_{\nu}}(<a_{\nu},e_{\nu}>-1)^{2}=2\cdot\frac{l_{\nu}}{\hat{l}_{\nu}}(<a_{\nu},e_{\nu}>-1)\xrightarrow[]{a.s.}0$

from Lemma 2 and Proposition 1. Returning to our term, Cauchy-Schwarz
inequality entails

$2|e_{\nu}^{T}E_{\nu}(a_{\nu}-e_{\nu})|\leq
2\hat{l}_{\nu}\sqrt{l_{\nu}}\cdot\sqrt{\sum_{k}{(t_{k}^{T}\mathcal{M}(\hat{l}_{\nu}I-\mathcal{M})^{-2}t_{\nu})^{2}}\cdot\sum_{k}{l_{k}((a_{\nu}-e_{\nu})_{k})^{2}}},$

and in light of (8.7) and Proposition 1, it suffices to show that

$l_{\nu}^{4}\sum_{k}{(t_{k}^{T}\mathcal{M}(\hat{l}_{\nu}I-\mathcal{M})^{-2}t_{\nu})^{2}}$

is almost surely bounded to conclude that
$e_{\nu}^{T}E_{\nu}(a_{\nu}-e_{\nu})\xrightarrow[]{a.s.}0:$ from part $(a)$ of
Lemma 1,

$l_{\nu}^{4}\sum_{k\neq\nu}{(t_{k}^{T}\mathcal{M}(\hat{l}_{\nu}I-\mathcal{M})^{-2}t_{\nu})^{2}}\xrightarrow[]{a.s.}0,$

and with probability one for $n$ sufficiently large, from Proposition 1 and
(8.5),

$0\leq
l_{\nu}^{2}t_{\nu}^{T}\mathcal{M}(\hat{l}_{\nu}I-\mathcal{M})^{-2}t_{\nu}=(\frac{l_{\nu}}{\hat{l}_{\nu}})^{2}\cdot\hat{l}_{\nu}^{2}t_{\nu}^{T}\mathcal{M}(\hat{l}_{\nu}I-\mathcal{M})^{-2}t_{\nu}\leq
2\cdot 4c_{K,\gamma}\cdot t_{\nu}^{T}\mathcal{M}t_{\nu}\leq 8c_{K,\gamma}\cdot
2\gamma,$

as $\frac{xl^{2}}{(l-x)^{2}}\leq 4c_{K,\gamma}\cdot x$ for
$x\in[0,\frac{l}{2}],x\leq c_{K,\gamma},$ and $l_{\nu}\to\infty.$

$3.$ Consider the third term:

$0\leq(a_{\nu}-e_{\nu})^{T}E_{\nu}(a_{\nu}-e_{\nu})\leq\hat{l}_{\nu}||\mathcal{M}(\hat{l}_{\nu}I-\mathcal{M})^{-2}||\cdot||\Lambda^{1/2}(a_{\nu}-e_{\nu})||^{2}\leq\frac{4c_{K,\gamma}||\Lambda^{1/2}(a_{\nu}-e_{\nu})||^{2}}{\hat{l}_{\nu}}$

almost surely as $\frac{x}{(l-x)^{2}}\leq\frac{4c_{K,\gamma}}{l^{2}}$ for
$x\in[0,\frac{l}{2}],x\leq c_{K,\gamma}.$ Hence this term tends to zero almost
surely from (8.7) and Proposition 1.

The following concentration inequalities will be used repeatedly throughout
the proofs ($c>0$ is a universal constant independent of any of the
parameters):

Two-Sided Bound on Subgaussian Matrices (Vershynin [17], theorem $4.6.1$)

For any random matrix $A\in\mathbb{R}^{p\times q}$ whose entries are
independent, of mean zero, subgaussian with $\max_{1\leq i\leq p,1\leq j\leq
q}{||A_{ij}||_{\psi_{2}}}\leq K,$ and all $t\geq 0,$

$\sqrt{p}-cK^{2}(\sqrt{q}+t)\leq s_{q}(A)\leq
s_{1}(A)\leq\sqrt{p}+cK^{2}(\sqrt{q}+t),$ (SM)

with probability at least $1-2\exp(-t^{2}),$ where $s_{i}(\cdot)$ is the
$i^{th}$ largest singular value.

Hanson-Wright Inequality (Vershynin [17], theorem $6.2.1$):

For $y\in\mathbb{R}^{p}$ a random vector with independent entries of mean
zero, $\max_{1\leq i\leq p}{||y_{i}||_{\psi_{2}}}\leq
K,C\in\mathbb{R}^{p\times p},$ and all $t\geq 0,$

$\mathbb{P}(|y^{T}Cy-\mathbb{E}[y^{T}Cy]|\geq t)\leq
2\exp(-c\min{(\frac{t^{2}}{pK^{4}||C||^{2}},\frac{t}{K^{2}||C||})}).$ (HW)

Analogue of Hanson-Wright Inequality:

For $y,y^{\prime}\in\mathbb{R}^{p}$ random vectors with entries
$y_{1},y_{2},\hskip 1.42271pt...\hskip
1.42271pt,y_{p},y^{\prime}_{1},y^{\prime}_{2},\hskip 1.42271pt...\hskip
1.42271pt,y^{\prime}_{p}$ independent, of mean zero, $\newline \max_{1\leq
i\leq p}{||y_{i}||_{\psi_{2}}},\max_{1\leq i\leq
p}{||y^{\prime}_{i}||_{\psi_{2}}}\leq K,$ and all $t\geq 0,$

$\mathbb{P}(|y^{T}Cy^{\prime}|\geq t)\leq
2\exp(-c\min{(\frac{t^{2}}{pK^{4}||C||^{2}},\frac{t}{K^{2}||C||})}).$ (aHW)

Proof: This result can be derived from the proof of its counterpart as
follows: lemma $6.2.3$ from Vershynin [17] gives that for $y,y^{\prime}$ as
above, $g,g^{\prime}\in\mathbb{R}^{p}$ independent Gaussian random vectors of
mean zero, covariance the identity matrix, and all $\lambda\in\mathbb{R},$

$\mathbb{E}[\exp(\lambda
y^{T}Cy^{\prime})]\leq\mathbb{E}[\exp(c_{1}K^{2}\lambda g^{T}Cg^{\prime})].$

Take $\sigma_{1},\sigma_{2},\hskip 1.42271pt...\hskip 1.42271pt,\sigma_{p}$ to
be the eigenvalues of $C.$ Then by rotational invariance,

$\mathbb{E}[\exp(c_{1}K^{2}\lambda
g^{T}Cg^{\prime})]=\mathbb{E}[\exp(c_{1}K^{2}\lambda
g^{T}diag(\sigma_{1},\sigma_{2},\hskip 1.42271pt...\hskip
1.42271pt,\sigma_{p})g^{\prime})]=$ $=\prod_{1\leq i\leq
p}{\mathbb{E}[\exp(c_{1}K^{2}\lambda\sigma_{i}g_{i}g_{i}^{\prime})]}=\prod_{1\leq
i\leq
p}{\mathbb{E}[\mathbb{E}[\exp(c_{1}K^{2}\lambda\sigma_{i}g_{i}g_{i}^{\prime})|g_{i}]]}=$
$=\prod_{1\leq i\leq
p}{\mathbb{E}[\exp(c_{1}^{2}K^{4}\lambda^{2}\sigma_{i}^{2}g_{i}^{2})]}\leq\exp(pc_{2}||C||^{2}K^{4}\lambda^{2})$

for $|\lambda|\leq\frac{c_{3}}{c_{1}K^{2}||C||},$ from
$\mathbb{E}[\exp(\lambda
g_{i})]=\exp(\lambda^{2}),\mathbb{E}[\exp(\tau^{2}X^{2})]\leq\exp(c_{4}\tau^{2})$
for $||X||_{\psi_{2}}\leq||g_{1}||_{\psi_{2}}$ and $|\tau|\leq c_{3}$ (this is
property $(iii)$ of proposition $2.5.2$ in Vershynin [17]).

Hence, for $|\lambda|\leq\frac{c_{3}}{c_{1}K^{2}||C||},$

$\mathbb{E}[\exp(\lambda
y^{T}Cy^{\prime})]\leq\exp(pc_{2}||C||^{2}K^{4}\lambda^{2}),$

which implies (arguing as for Bernstein’s inequality) that for some universal
constant $c>0$ and all $t\geq 0,$

$\mathbb{P}(|y^{T}Cy^{\prime}|\geq t)\leq
2\exp(-c\min{(\frac{t^{2}}{pK^{4}||C||^{2}},\frac{t}{K^{2}||C||})}).$

## References

  * [1] Ahn, J. Marron, J. Muller, K. Chin, Y. (2007). The high-dimension, low-sample-size geometric representation holds under mild conditions. Biometrika, 94, 3. pp. 760-766.
  * [2] Anderson, T. (1963). Asymptotic Theory for Principal Component Analysis, Annals of Mathematical Statistics, 34(1), 122-148.
  * [3] Bai, Z. and Silverstein, J. (1955). On the Empirical Distribution of Eigenvalues of a Class of Large Dimensional Random Matrices, Journal of Multivariate Analysis, 54, 175-192.
  * [4] Bai, Z. and Silverstein, J. (2010). Spectral Analysis of Large Dimensional Random Matrices, Springer Series in Mathematics, Second Edition.
  * [5] Bai, Z. and Yin, Y. (1993). Limit of the Smallest Eigenvalue of a Large Dimensional Sample Covariance Matrix, The Annals of Probability, Volume 21, No. 3, 1275-1294.
  * [6] Baik, J. and Silverstein, J. (2006). Eigenvalues of large sample covariance matrices of spiked population models, Journal of Multivariate Analysis, Volume 97, Issue 6, 1382-1408.
  * [7] Billingsley, P. (1955). Probability and Measure, Third Edition, Wiley Series in Probability and Mathematical Statistics.
  * [8] Cai, T. Han, X. Pan, G. (2020). Limiting Laws for Divergent Spiked Eigenvalues and Largest Nonspiked Eigenvalue of Sample Covariance Matrices, The Annals of Statistics, Volume 48, No. 3, 1255–1280.
  * [9] Hall, P. Marron, J. Neeman, A. (2005). Geometric representation of high-dimension, low sample size data, Journal of the Royal Statistical Society, Series B (Statistical Methodology), Vol. 67, No. 3, pp. 427-444.
  * [10] Johnstone, I. (2001). On the Distribution of the Largest Eigenvalue in Principal Components Analysis, The Annals of Statistics, 29(2):295–327.
  * [11] Jung, S. Marron, J. (2009). PCA Consistency in High Dimension, Low Sample Size Context, The Annals of Statistics, Vol. 37, No. 6B, 4104–4130.
  * [12] Marchenko, V. and Pastur, L. (1967). Distribution of Eigenvalues For Some Sets Of Random Matrices, Math. USSR-Sbornik, Vol. 1, no. 94.
  * [13] Ledoit, O. and Péché, S. (2011). Eigenvectors of some large sample covariance matrix ensembles, Probability Theory Related Fields, 151:233–264.
  * [14] Paul, D. (2007). Asymptotics of Sample Eigenstructure for a Large Dimensional Spiked Covariance Model, Statistica Sinica 17, 1617-1642.
  * [15] Silverstein, J. (1995). Strong Convergence of the Empirical Distribution of Eigenvalues of Large Dimensional Random Matrices, Journal of Multivariate Analysis, 55, 331-339.
  * [16] Shen, D. Shen, H. Marron, J. (2016). A General Framework for Consistency of Principal Component Analysis, Journal of Machine Learning Research 17: 1-34.
  * [17] Vershynin, R. High-Dimensional Probability: An Introduction with Applications in Data Science, https://www.math.uci.edu/~rvershyn/papers/HDP-book/HDP-book.pdf.
  * [18] Wang, W. and Fan, J. (2017). Asymptotics of empirical eigenstructure for high dimensional spiked covariance, The Annals of Statistics, 45(3): 1342–1374.