# Consistently recovering the signal from noisy functional data

Siegfried Hörmann
Institute of Statistics, Graz University of Technology
and
Fatima Jammoul
Institute of Statistics, Graz University of Technology Corresponding author.
Email<EMAIL_ADDRESS>
###### Abstract

In practice most functional data cannot be recorded on a continuum, but rather
at discrete time points. It is also quite common that these measurements come
with an additive error, which one would like eliminate for the statistical
analysis. When the measurements for each functional datum are taken on the
same grid, the underlying signal-plus-noise model can be viewed as a factor
model. The signals refer to the common components of the factor model, the
noise is related to the idiosyncratic components. We formulate a framework
which allows to consistently recover the signal by a PCA based factor model
estimation scheme. Our theoretical results hold under rather mild conditions,
in particular we don’t require specific smoothness assumptions for the
underlying curves and allow for a certain degree of autocorrelation in the
noise.

Keywords: factor models, functional data, PCA, preprocessing, signal-plus-
noise.

##  1 Introduction

As a consequence of the explosive increase of available data in practically
all areas of life, data science (as a generic term for the diverse data
processing disciplines) has been steadily gaining popularity over the past
decades. Handling massive data not only requests larger human and
computational resources, but also demands and entails the development of new
scientific approaches. Within the field of statistics, functional data
analysis (FDA) is one such area which has experienced a massive surge in
interest over the last couple of years. Its basic objective is to handle data,
where each observation $X_{t}$ can be viewed as a process $X_{t}(s)$ defined
on some continuous domain $\mathcal{U}$. To date a number of comprehensive
text books (e.g., Ramsay and Silverman (2005), Ferraty and Vieu (2006),
Horváth and Kokoszka (2012), Hsing and Eubank (2015)) can be consulted for a
general overview of the numerous tools and methods which have been developed.
The most active domains of research arguably are related to functional
regression and dimension reduction techniques (see e.g., Cuevas (2014) for a
survey). The latest developments are compactly summarized in Goia and Vieu
(2016) and Aneiros et al. (2019).

Our contribution concerns the preprocessing step which is needed in most
functional data analyses. Typically $X_{t}(s)$ cannot be sampled over the
entire continuum, but rather we record a discretized version. Moreover, it is
not uncommon that measuring devices become more error-prone with an increase
in sampling frequency, so that often we actually observe a discretized and
noisy version of $X_{t}(s)$. The question how to deal with the discrepancy
between the theoretical model and the actual measurements is often the
starting point in theoretical and practical FDA problems. While existing
literature has been focusing on different variants of smoothing techniques, we
want to explore this problem now from a different perspective.

Throughout this paper we are dealing with a stationary sequence of functional
data $(X_{t}\colon t\geq 1)$, where each data point is of the form
$(X_{t}(s)\colon s\in[0,1])$. In the common case where $\mathcal{U}$ is an
interval, the restriction to $[0,1]$ comes with no loss of generality. We
assume that $X_{t}(s)$ is recorded at discrete time points $0\leq
s_{1}<s_{2}<\cdots<s_{p}\leq 1$. We focus on data where these sampling points
are identical for each $t$ and where the number of sampling points $p$ is
large. Such a setting is very common when we have machine recorded data.
Irrespective of whether we have a sparse or a dense observation grid, most
papers in FDA literature assume that measurements might come with an
additional error, yielding actual observations of the form

$y_{t}=(X_{t}(s_{1}),\ldots,X_{t}(s_{p}))^{\top}+(u_{t1},\ldots,u_{tp})^{\top}=:X_{t}(\boldsymbol{s})+u_{t}.$
(1)

Here and in the sequel $\boldsymbol{s}=(s_{1},\ldots,s_{p})$ and
$g(\boldsymbol{s})=(g(s_{1}),\ldots,g(s_{p}))^{\top}$. When we want to process
such data, we need to eliminate the noise $u_{t}$ in order to guarantee a
valid inference for $X_{t}$. Most available approaches towards this end assume
$X_{t}(s)$ to be a smooth curve and then propose to estimate the latent signal
with diverse smoothing techniques. The smoothing is mostly done curve-by-
curve, for example via spline smoothing (e.g. Ramsay and Silverman (2005)),
kernel smoothing (e.g. Wand and Jones (1995)) or local linear regression (e.g.
Fan (1993)). Alternatively, Staniswalis and Lee (1998) suggest to smooth the
empirical covariance of the raw data in order to get an estimate of
$\Gamma^{X}(s,s^{\prime})=\mathrm{Cov}(X_{t}(s),X_{t}(s^{\prime}))$. The
eigenfunctions of the smoothed covariance are then used to get a proxy for the
Karhunen-Loève expansion of $X_{t}$.

In this paper we investigate an alternative method, which doesn’t make use of
the common smoothing techniques and which in turn doesn’t rely on potentially
unverifiable smoothness of the curves. Rather we use that data $y_{t}$ given
in (1) can be shown to follow some (approximate) factor model, where
$X_{t}(\boldsymbol{s})$ und $u_{t}$ correspond to the common components and
idiosyncratic components of $y_{t}$, respectively. This means that the
components $X_{t}(\boldsymbol{s})$ are driven by some lower dimensional
process (the common factors) and hence are strongly correlated, whereas the
components of $u_{t}$ are uncorrelated or only mildly correlated and represent
unsystematic fluctuations (see e.g., Mardia et al. (1979) for a basic
introduction).

Recovering $X_{t}(\boldsymbol{s})$ can thus be seen as estimating the common
components in a factor model. In a companion paper Hörmann and Jammoul (2021)
we have pointed and worked out the connection to factor models and thoroughly
explored the practical performance of existing MLE or PCA based estimation
schemes on real and simulated data. We found that these approaches lead to
very convincing and nicely interpretable results. In a variety of considered
settings the factor model estimates outperform competing smoothing techniques
in terms of the resulting mean square approximation error. This advantage is
particularly remarkable for moderately sized $p$ and large $T$ or when the
underlying signal $X(s)$ is not smooth.

The primary purpose of this article is to provide the theoretical validation
of the factor approach. Following Fan et al. (2013) we will explore a
principal components based estimator for the common components. In Hörmann and
Jammoul (2021) we have experienced that it leads to slightly better practical
performance when compared to a likelihood based approach (e.g., Bai and Li
(2012)). Moreover, it is very simple to implement. When $EX_{t}=0$, then the
estimator is of the form

$(\hat{X}_{1}(\boldsymbol{s}),\ldots,\hat{X}_{T}(\boldsymbol{s}))=Y\hat{E}\hat{E}^{\top},$
(2)

where $Y=(y_{1},\ldots,y_{T})$ and where
$\hat{E}=(\hat{e}_{1},\ldots,\hat{e}_{L})$ are the eigenvectors of
$\frac{1}{T}Y^{\top}Y$ belonging to the $L$ largest eigenvalues
$\hat{\gamma}_{1}\geq\cdots\geq\hat{\gamma}_{L}$. In our theorems below we
will prove that

$\sup_{1\leq t\leq T}\sup_{1\leq i\leq
p}|\hat{X}_{t}(s_{i})-X_{t}(s_{i})|\stackrel{{\scriptstyle P}}{{\to}}0.$

Here $\stackrel{{\scriptstyle P}}{{\to}}$ denotes convergence in probability.
Our results are based on $T\to\infty$ and $p=p(T)\to\infty$. While our
reasoning is similar to Fan et al. (2013), we are working with assumptions
specific for functional data. In particular, we are going to allow the number
of factors $L$ to diverge, i.e. $L=L(T)\to\infty$. Still, we will be able to
work under much less restrictive moment and dependence assumptions compared to
current factor model literature. They are also much less restrictive than
assumptions in comparable FDA smoothing results. We will expand on this in
more detail in Section 2.3.

In the next section we formulate our assumptions and the main theorems. In
Section 3 we give the core steps of the proofs. We conclude in Section 4.
Technical lemmas and detailed proofs are given in the Appendix.

##  2 Results

In this section we formulate and discuss our assumptions and present our large
sample results. It will be assumed throughout that $(X_{t}\colon t\geq 1)$ is
stationary and that the curves $(X_{t}(s)\colon s\in[0,1])$ are second-order
stochastic processes. We then can define $\mu(s)=EX_{t}(s)$ and
$\Gamma^{X}(s,s^{\prime})=\mathrm{Cov}(X_{t}(s),X_{t}(s^{\prime}))$,
respectively. We will be considering the setting where the sample size $T$ and
the number of sampling points $p=p(T)$ tend to infinity. The data are sampled
as in (1).

###  2.1 Assumptions

Let us now list the detailed assumptions on the processes $(u_{t})$ and
$(X_{t})$.

######  Assumption 1.

The noise process $(u_{t})$ is i.i.d. zero mean and independent of the signals
$(X_{t})$. The processes $(u_{ti}\colon 1\leq i\leq p)$ are Gaussian with
covariance function $\gamma^{u}(h)=\text{Cov}(u_{t(i+h)},u_{ti})$, such that
$\sum_{h\in\mathbb{Z}}|\gamma^{u}(h)|\leq C_{u}<\infty.$

Assuming Gaussianity of the noise processes simplifies our proofs and
assumptions and seems reasonably justified in the context of modelling
measurement errors. It is important to note that we don’t impose that $u_{ti}$
and $u_{tj}$ are uncorrelated, as it is often required for factor models. In
many real data situations it is likely that errors of consecutive measurements
are at least mildly correlated. Our main theorems below can be extended under
suitable moment and dependence assumptions for the $(u_{ti}\colon 1\leq i\leq
p)$. See B.

Summability of the autocovariances implies that the correlation between the
errors decays quickly with increasing lag. This is in line with the paradigm
of approximate factor models. Such models have been first discussed in
Chamberlain and Rothschild (1983) in context of macroeconomic applications and
are now quite commonly used in high dimensional factor model settings. Several
works have further investigated approximate factor models since then, see
e.g., Bai (2003), Bai and Liao (2016), Choi (2012), Bai and Li (2016), Bai and
Ng (2019).

For the signal process $(X_{t})$ we will allow for temporal dependence. A
concept that proved to be rather useful in this context, is the so-called
$L^{p}$-$m$-approximability, which has been proposed in Hörmann and Kokoszka
(2010). A sequence is said to be $L^{p}$-$m$-approximable, if it has an
ergodic representation of the form $X_{t}=g(\delta_{t},\delta_{t-1},\ldots)$
with some measurable functional $g$ and i.i.d. elements $(\delta_{t})$ in some
measurable space and satisfies

$\sum_{m\geq
1}(E\|X_{t}-g(\delta_{t},\ldots,\delta_{t-m},\delta_{t-m-1}^{\prime},\delta_{t-m-2}^{\prime},\ldots)\|^{p})^{1/p}<\infty.$
(3)

Here, $\|\cdot\|$ denotes the $L^{2}$-norm for square integrable functions on
$[0,1]$ and $(\delta_{t}^{\prime})$ is an independent copy of the innovations
$(\delta_{t})$. This means that we can approximate $X_{t}$ by coupled
$m$-dependent sequences sufficiently well. Several popular functional time
series models (e.g. functional ARMA or functional GARCH) fall into this
framework. Unlike diverse mixing conditions, $L^{p}$-$m$-approximability is
often much easier to establish. For the purpose of illustration consider a
functional AR(1) model $X_{t}=\varrho(X_{t-1})+\delta_{t}$. Under suitable
assumptions on the linear operator $\varrho$ (see e.g. Bosq (2000)) we get by
iterative application of the recursion that $X_{t}=\sum_{k\geq
0}\varrho^{k}(\delta_{t-k})$, and hence the required representation holds. A
sufficient assumption is $\|\varrho\|<1$. Here the coupled version is
$X_{t}^{(m)}=\sum_{k=0}^{m}\varrho^{k}(\delta_{t-k})+\sum_{k>m}\varrho^{k}(\delta_{t-k}^{\prime})$
and thus if the $\delta_{t}$ have $p$ moments we get

$\sum_{m\geq
1}(E\|X_{t}-X_{t}^{(m)}\|^{p})^{1/p}\leq(E\|\delta_{0}-\delta_{0}^{\prime}\|^{p})^{1/p}\sum_{m\geq
1}\sum_{k>m}\|\varrho\|^{k}<\infty.$

This shows that with increasing $m$, the $L^{p}$ error between the original
data and the coupled $m$-dependent sequence goes to zero fast enough to
guarantee summability in (3). More details can be found in Hörmann and
Kokoszka (2010).

######  Assumption 2.

(i) The process $(X_{t}\colon t\geq 1)$ is zero mean and
$L^{4}$-$m$-approximable. (ii) The curves $X_{t}=(X_{t}(s)\colon s\in[0,1])$
define fourth order random processes (i.e. $\sup_{s\in[0,1]}EX_{1}^{4}(s)\leq
C_{X}<\infty$) with a continuous covariance kernel. (iii) It holds that
$E\sup_{s\in[0,1]}X^{2}_{1}(s)<\infty$. (iv) Observations $X_{t}$ lie in some
$L$-dimensional function space, where $L=L(T)$ may diverge with $T\to\infty$.

The zero mean assumption is not necessary, but it simplifies the presentation.
In practice $\mu(\boldsymbol{s})$ will be estimated via
$\hat{\mu}(\boldsymbol{s})=T^{-1}(y_{1}+\cdots+y_{T})$ and data will be
centered. It is implicit in the definition of $L^{4}$-$m$-approximability that
the process $(X_{t})$ is strictly stationary and ergodic, which is why the
conditions (ii) and (iii) can be restricted to $X_{1}$. The main theoretical
restriction is Assumption 2 (iv). It is motivated by Mercer’s theorem which
yields under the assumption of a continuous covariance kernel that for a
functional variable $X_{t}$ it holds that

$\sup_{s\in[0,1]}E\left|X_{t}(s)-\mu(s)-\sum_{\ell=1}^{L}x_{t\ell}\varphi_{\ell}(s)\right|^{2}\to
0,\quad L\to\infty,$ (4)

where $\varphi_{\ell}$ denotes the eigenfunction of the covariance operator
$\Gamma^{X}$ belonging to the $\ell$-th eigenvalue $\lambda_{\ell}$ and the
$x_{t\ell}=\int_{0}^{1}(X_{t}(s)-\mu(s))\varphi_{\ell}(s)ds=:\langle
X_{t}-\mu,\varphi_{\ell}\rangle$ are functional principal component scores. In
most real data problems the convergence in (4) is very fast, so that even for
relatively small $L$ the variables
$X_{t}^{L}(s)=\mu(s)+\sum_{\ell=1}^{L}x_{t\ell}\varphi_{\ell}(s)$ define only
a slight perturbations of $X_{t}(s)$, lying in the $L$-dimensional space
spanned by the functional principal components
$\varphi_{1},\ldots,\varphi_{L}$. Hence this Assumption 2 (iv) holds
asymptotically for a very broad class of functional data.

Now the key observation that we are going to exploit in the context of this
paper is that under Assumptions 1 and 2 the $y_{t}$ given in (1) follow some
$L$-factor model, where $X_{t}(\boldsymbol{s})$ is the common component and
$u_{t}$ the idiosyncratic component. See Hörmann and Jammoul (2021).

######  Assumption 3.

For the eigenfunctions $\varphi_{\ell}$ it holds that $\max_{1\leq k,\ell\leq
L}\left|p^{-1}\sum_{i=1}^{p}\varphi_{k}(s_{i})\varphi_{\ell}(s_{i})\right|=O(1)$
as $T\to\infty$.

At this stage we stress once more that $p=p(T)$ and $L=L(T)$, which explains
the asymptotic statement for $T\to\infty$. If $s_{i}$ are dense in $[0,1]$
then
$p^{-1}\sum_{i=1}^{p}\varphi_{k}(s_{i})\varphi_{\ell}(s_{i})\approx\int_{0}^{1}\varphi_{k}(s)\varphi_{\ell}(s)ds=\delta_{k,\ell}$
(Kronecker delta). Hence, this assumption is very mild.

###  2.2 Theorems

We are now ready to formulate our asymptotic results. We distinguish the
scenarios where $L$ is fixed (Theorem 1) and where $L=L(T)\to\infty$ (Theorem
2). Our main results are formulated with $L$ known. At the end of this section
we comment on the estimation of $L$. All following asymptotic formulations are
understood as $T\to\infty$.

######  Theorem 1.

Let Assumptions 1–3 hold. Assume that $p=p(T)$ diverges at a subexponential
rate and let $L$ be arbitrary but fixed. Furthermore, suppose that
$p/\hat{\gamma}_{L}=O_{P}(1)$. Then for any $t\leq T$

$\max_{1\leq j\leq
p}|\hat{X}_{t}(s_{j})-X_{t}(s_{j})|=O_{P}\Big{(}\frac{1}{T^{1/4}}+\frac{T^{1/4}}{\sqrt{p}}\Big{)}.$

Hence the estimator is consistent if $\sqrt{T}/p\to 0$. Moreover,

$\max_{1\leq j\leq p}\max_{1\leq t\leq
T}|\hat{X}_{t}(s_{j})-X_{t}(s_{j})|=O_{P}\left(\max_{1\leq t\leq
T}\|f_{t}\|\left(\frac{1}{T^{1/4}}+\frac{T^{1/4}}{\sqrt{p}}\right)\right),$

where
$f_{t}=(x_{t1}/\sqrt{\lambda_{1}},\ldots,x_{tL}/\sqrt{\lambda_{L}})^{\top}$.

Theorem 1 needs further explanation. First, we comment on
$p/\hat{\gamma}_{L}=O_{P}(1)$. The following lemma provides a simple condition
when this assumption holds.

######  Lemma 1.

Assume that the sampling points $s_{i}$ are equidistant and that

$\sup_{s\in[0,1]}E|X_{t}(s+h)-X_{t}(s)|^{2}=O(h),\quad h\to 0.$ (5)

Then, under the conditions of Theorem 1, we have $\max_{i\geq
1}|\lambda_{i}-\hat{\gamma}_{i}/p|=O_{P}\left(1/\sqrt{p}+1/\sqrt{T}\right)$.

The lemma shows, among others, that $p/\hat{\gamma}_{L}$ is a consistent
estimator for $\lambda_{L}^{-1}$ and hence is $O_{P}(1)$. Condition (5) holds
if the covariance kernel $\Gamma^{X}$ is uniformly continuous. We would like
to point out that the conditions in Lemma 1 may be adapted and weakened. We
decided to disentangle this problem from the theorem and to present conditions
which are neat and not very technical.

Concerning the uniform consistency in Theorem 1 we need in addition a bound
for $\max_{1\leq t\leq T}\|f_{t}\|$. Such bounds can easily be obtained under
higher order moment conditions. We formulate the following lemma.

######  Lemma 2.

Suppose that Assumption 2 holds and that $L$ is fixed.
_(i)_ If $E\|X_{1}\|^{q}<\infty$ for some $q>0$, then $\max_{1\leq t\leq
T}\|f_{t}\|=o_{P}\left(T^{1/q^{\prime}}\right)$ for all $0<q^{\prime}<q$.
_(ii)_ If $E\exp(\alpha\|X_{1}\|)<\infty$ for some $\alpha>0$, then
$\max_{1\leq t\leq T}\|f_{t}\|=O_{P}\left(\log(T)\right).$

The lemma implies that if we request moments of order $>4$ for the signal
process $X_{t}$, then our Theorem 1 provides uniform consistency over time and
cross-section.

Next we consider the setting where the dimension of the function space is
allowed to diverge. Not surprisingly, this requires extra conditions, e.g. on
the eigenvalues of $\Gamma^{X}$ and on the growth rate of $L$. Instead of
listing several different sets of conditions and results, we have decided to
present here one specific setup which allows for relatively neat conditions.
However, we transparently outline the main steps and bounds needed for the
proofs in Section 3. From there it is not very complicated to see where the
respective conditions enter and how changing the assumptions will change the
results. Here we refer to the decomposition of the approximation error in (8)
and (9) in particular. This leaves some flexibility and allows the analyst to
adapt the theorems to the setup under investigation.

######  Theorem 2.

Let Assumptions 1–3 hold. Assume that $p=p(T)$ diverges at a subexponetial
rate and that $L=L(T)$ diverges at a subpolynomial rate. Furthermore, assume
that for some $\nu>0$ and some $\rho>0$ we have $\lambda_{j}\geq\rho j^{-\nu}$
and that there is some $\beta>0$ such that
$p/\hat{\gamma}_{L}=O_{P}(L^{\beta})$. Then, for any $1\leq t\leq T$ it holds
that

$\max_{1\leq j\leq
p}|\hat{X}_{t}(s_{j})-X_{t}(s_{j})|=O_{P}\left(L^{2\beta+5/2+\nu/2}\bigg{(}\frac{1}{T^{1/4}}+\frac{T^{1/4}}{\sqrt{p}}\bigg{)}\right).$

Moreover, if $E\|X_{1}\|^{q}<\infty$ for some $q>4$, then for all
$q^{\prime}<q$ we have

$\max_{1\leq t\leq T}\max_{1\leq j\leq
p}|\hat{X}_{t}(s_{j})-X_{t}(s_{j})|=o_{P}\left(L^{2\beta+5/2+\nu}T^{1/q^{\prime}}\bigg{(}\frac{1}{T^{1/4}}+\frac{T^{1/4}}{\sqrt{p}}\bigg{)}\right).$

As one referee pointed out, it may appear counter-intuitive that the
reconstruction error vanishes more slowly when the eigenvalue sequence
decreases faster. The problem here is that the smaller the eigenvalue, the
weaker the signal associated with the corresponding factor and the more
difficult it is to estimate. A similar problem occurs in functional
regression. While a fast decay of the eigenvalues associated to the functional
covariate indicates that we are “almost” in a finite dimensional setting,
convergence rates in fact become slower. See, for example, the results in Hall
and Horowitz (2007).

The technical conditions in Theorem 2 are mild. But as we noted before, they
can easily be generalized. For example, it is possible to assume a polynomial
rate for $L(T)$, but then we will get weaker and more complicated error terms.
One part of our assumption which is less transparent is
$p/\hat{\gamma}_{L}=O_{P}(L^{\beta})$. The following lemma provides a
sufficient condition.

######  Lemma 3.

Suppose that we have positive parameters $\rho,\rho^{\prime},\nu$ and
$\nu^{\prime}$ such that
$\rho^{\prime}j^{-\nu^{\prime}}\geq\lambda_{j}\geq\rho j^{-\nu}$, for all
$j\geq 1$. Moreover, assume that Lemma 1 applies. Then if
$L^{2\nu}/\min\\{p,T\\}\to 0$, we have
$p/\hat{\gamma}_{L}=o_{P}(L^{\nu^{\prime}})$.

In Theorems 1 and 2 the required number of factors $L$ is assumed to be known,
even though it may grow with $T$ within the setting of Theorem 2. Naturally,
in practice $L$ is not known and must be estimated before applying the above
mentioned methods. Determining the number of factors is a separate problem,
which goes beyond the scope of our article. We refer e.g., to Bai and Ng
(2002) or Onatski (2010). In our practical implementations we found the bi-
cross validation method suggested by Owen and Wang (2016) to be most
effective. However, we show that the use of an appropriate estimator for $L$
does not interfere with our results. Let us assume here that an estimator
$\hat{L}$ exists, such that

$\Pr(|\hat{L}-L|=0)\longrightarrow 1.$ (6)

######  Corollary 1.

Suppose that the number of factors is estimated by $\hat{L}$, and that this
estimator satisfies (6). Then the statements of Theorems 1 and 2 remain valid.

We conclude this section by commenting on the dependence structure of the
estimated signals $(\hat{X}_{t}(\boldsymbol{s}))$. As one of the reviewers
pointed out, our procedure to some extent modifies the autocorrelation
structure of the functional series. For example, if the $X_{t}$ were
independent, then the $\hat{X}_{t}$ may nevertheless be dependent, since each
$\hat{X}_{t}$ is computed from the entire sample. Intuitively, this effect
will be asymptotically negligible by our main results. Let us illustrate that
this intuition is correct on the basis of the empirical autocovariance. To
this end we note that

$\displaystyle\left|\frac{1}{T}\sum_{t=1}^{T-|h|}\left(\hat{X}_{t+|h|}(s_{i})\hat{X}_{t}(s_{j})-X_{t+|h|}(s_{i})X_{t}(s_{j})\right)\right|$
$\displaystyle\quad\leq\frac{1}{T}\sum_{t=1}^{T-|h|}\left|X_{t}(s_{j})\right|\left|\hat{X}_{t+|h|}(s_{i})-X_{t+|h|}(s_{i})\right|+\frac{1}{T}\sum_{t=1}^{T-|h|}\left|\hat{X}_{t+|h|}(s_{i})\right|\left|\hat{X}_{t}(s_{j})-X_{t}(s_{j})\right|.$

The first term on the right hand side can be bounded by

$\frac{1}{T}\sum_{t=1}^{T-|h|}\left|X_{t}(s_{j})\right|\left|\hat{X}_{t+|h|}(s_{i})-X_{t+|h|}(s_{i})\right|\leq\max_{1\leq
i\leq p}\max_{1\leq t\leq
T}\left|\hat{X}_{t}(s_{i})-X_{t}(s_{i})\right|\times\frac{1}{T}\sum_{t=1}^{T}\max_{1\leq
j\leq p}\left|X_{t}(s_{j})\right|.$

The second term can be treated similiarly. The factor
$T^{-1}\sum_{j=1}^{T}\max_{1\leq j\leq p}\left|X_{t}(s_{j})\right|$ is
$O_{P}(1)$ by Assumption 2 (iii) and the ergodic theorem, while the first
factor tends to zero at the rate specified by our theorems. On the other hand,
it follows from the ergodic theorem that the empirical acf from the latent
signals $X_{t}(s_{i})$ is consistent for
$\mathrm{Cov}(X_{t+|h|}(s_{i}),X_{t}(s_{j}))$. The imposed weak dependence
assumption can be used to obtain rates.

###  2.3 Comparison to existing results

Let us begin with noting that the common curve-by-curve fitting techniques,
such as spline smoothing or non-linear regression techniques, don’t profit
from $T\to\infty$. The estimation error will only decrease with growing $p$
and not with a growing sample size. Hence our results do not directly compare
to these methods.

A method taking into account the entire sample is Staniswalis and Lee (1998).
Their approach, in contrast to ours, returns a full curve and doesn’t require
a fixed sampling design. Of course, we can also obtain a full curve by using a
sensible interpolation scheme, e.g. $\hat{X}_{t}(s):=X_{t}(s_{i})$ for all
$s\in[s_{i},s_{i+1})$ is a simple solution. We think, however, that such an
extension of $\hat{X}_{t}(\boldsymbol{s})$ to a curve $X_{t}(s)$ is of little
practical relevance, since for the processing of real data we will most often
use the discretised curves anyway. For example, in a functional regression
model $Y_{t}=\int\beta(s)X_{t}(s)ds+\epsilon_{t}$, the integral typically can
only be computed numerically, e.g. by applying a composite trapezoidal rule.
It is then natural to use $\boldsymbol{s}$ as the nodes.

Extending our results to obtain a theoretical bound for
$\sup_{s}|\hat{X}_{t}(s)-X_{t}(s)|$ is conceptually not very complicated. Set
$h=\max_{1\leq i\leq p}|s_{i+1}-s_{i}|$ and define the modulus of continuity
of $X_{t}$ as $\delta^{X}(h)=\sup_{s,s^{\prime}\in[0,1],|s-s^{\prime}|\leq
h}|X_{t}(s)-X_{t}(s^{\prime})|$, then

$\sup_{s\in[0,1]}|\hat{X}_{t}(s)-X_{t}(s)|\leq\max_{1\leq i\leq
p}|\hat{X}_{t}(s_{i})-X_{t}(s_{i})|+\delta^{X}(h).$

For the first term on the right we have provided bounds. By imposing suitable
assumptions on the path properties of the signal $X_{t}$, the errors
$\delta^{X}(h)$ can be controlled, provided the sampling grid $\boldsymbol{s}$
is dense. Let us remark at this point that for our theorems we don’t require a
dense grid $\boldsymbol{s}$, i.e., we can also deal with a situation where
$h\not\to 0$ as $p\to\infty$.

Corresponding asymptotic results for the method of Staniswalis and Lee (1998)
have been established in Müller et al. (2006). These authors also assume a
fixed and dense sampling design but indicate that such assumptions can be
relaxed to a certain extent. Their framework requires smoothness of the
underlying signal in terms of higher order derivatives. This is an assumption
which we particularly seek to avoid. In Hörmann and Jammoul (2021) we have
seen that smoothness of the signal is not just a technical assumption needed
for their proof, but also that the practical performance suffers severely for
signals which are not very smooth. An important technical restriction in
Müller et al. (2006) is that signals are assumed to be bounded, which then
also excludes the important case of Gaussian processes. In our paper we only
request $>4$ order moments.

The bounds in Müller et al. (2006) are quite involved and of the form
$\sup_{s}|\hat{X}_{t}(s)-X_{t}(s)|$. To the best of our knowledge, we are the
first paper to provide uniform bounds over $s$ and $1\leq t\leq T$.

Basically all papers considering the setting (1) assume that the noise
components are i.i.d. or at least white noise. We think that this is a severe
restriction. In real data it is rather plausible that errors of subsequent
measurements are likely to be correlated.

From the factor model perspective, we have improved on the conditions used in
Fan et al. (2013), who require, among others, exponential moments for the
involved processes. Moreover, we allow $L\to\infty$, which is also novel.

##  3 Main parts of the proofs

We define $U=(u_{1},\ldots,u_{T})$ and $F^{\top}=(f_{1},\ldots,f_{T})$.
Moreover, let $b_{j}^{\top}$ be the $j$-th row of

$B:=(\sqrt{\lambda_{1}}\varphi_{1}(\boldsymbol{s}),\ldots,\sqrt{\lambda_{L}}\varphi_{L}(\boldsymbol{s})).$

The $j$-th column of $Y^{\top}$ and $U^{\top}$ are denoted $Y_{j}$ and
$U_{j}$, respectively. The $j$-th row of $F^{\top}$ is denoted by $F_{j}$.
Then we have by (1), (4) and Assumption 2 (iv) that

$Y=BF^{\top}+U.$ (7)

In this notation, the objective is to estimate $BF^{\top}$ through some
estimator $\hat{B}\hat{F}^{\top}$ and to show that
$BF^{\top}-\hat{B}\hat{F}^{\top}$ tends to zero, uniformly over each
component. In particular we let $\hat{F}=\sqrt{T}\hat{E}$ and
$\hat{B}=T^{-1}Y\hat{F}$, where $\hat{E}=(\hat{e}_{1},\ldots,\hat{e}_{L})$
denotes the eigenvectors associated to the $L$ largest eigenvalues of
$T^{-1}Y^{\top}Y$. Then $\hat{X}=\hat{B}\hat{F}^{\top}$ as in (2). We
introduce the matrices
$\hat{\Lambda}=\mathrm{diag}(\hat{\gamma}_{1},\ldots,\hat{\gamma}_{L})$ and
$H:=T^{-1}\hat{\Lambda}^{-1}\hat{F}^{\top}FB^{\top}B.$ We will see in Lemma 8
that $H$ is asymptotically orthogonal, which means
$BF^{\top}-\hat{B}\hat{F}^{\top}\approx
BH^{\top}HF^{\top}-\hat{B}\hat{F}^{\top}.$ The basic idea is then to show that
$\hat{B}-BH^{\top}$ and $\hat{F}^{\top}-HF^{\top}$ become small. The matrix
$H$ guarantees that the estimated and the empirical scores share the same
orientation.

More specifically we have
$\hat{X}_{t}(s_{j})-X_{t}(s_{j})=\hat{b}_{j}^{\top}\hat{f}_{t}-b_{j}^{\top}f_{t}$
and we write

$\displaystyle\hat{b}_{j}^{\top}\hat{f}_{t}-b_{j}^{\top}f_{t}=(\hat{b}_{j}-Hb_{j})^{\top}(\hat{f}_{t}-Hf_{t})+(\hat{b}_{j}-Hb_{j})^{\top}Hf_{t}+(Hb_{j})^{\top}(\hat{f}_{t}-Hf_{t})+b_{j}^{\top}(H^{\top}H-I_{L})f_{t}.$

Thus we have

$\displaystyle\max_{j}|\hat{X}_{t}(s_{j})-X_{t}(s_{j})|$ $\displaystyle\leq
R^{(1)}R^{(2)}+R^{(2)}R^{(5)}\|f_{t}\|+R^{(5)}R^{(3)}R^{(1)}+R^{(3)}R^{(6)}\|f_{t}\|,$
(8) $\displaystyle\max_{j,t}|\hat{X}_{t}(s_{j})-X_{t}(s_{j})|$
$\displaystyle\leq
R^{(1)}R^{(2)}+R^{(2)}R^{(5)}R^{(4)}+R^{(5)}R^{(3)}R^{(1)}+R^{(3)}R^{(6)}R^{(4)},$
(9)

where $R^{(1)}:=\max_{1\leq t\leq T}R^{(1)}_{t}$ with
$R^{(1)}_{t}:=\|\hat{f}_{t}-Hf_{t}\|;$ $R^{(2)}:=\max_{1\leq j\leq
p}R^{(2)}_{j}$ with $R^{(2)}_{j}:=\|\hat{b}_{j}-Hb_{j}\|;$
$R^{(3)}=\max_{1\leq j\leq p}\|b_{j}\|;$ $R^{(4)}:=\max_{1\leq t\leq
T}\|f_{t}\|;$ $R^{(5)}:=\|H\|;$ and finally $R^{(6)}:=\|H^{\top}H-I_{L}\|.$

Here—with a slight abuse of notation—$\|\cdot\|$ denotes next to the
$L^{2}$-norm the Euclidean norm and the spectral norm of a matrix. The
concrete meaning will be clear from the context. Bounds for $R^{(i)}$ will be
obtained below in a sequence of lemmas. The proofs of these technical lemmas
are given in A.

We begin with the term $R^{(1)}$. By simple algebraic manipulations it follows
from (7) that

$\displaystyle\hat{F}^{\top}-HF^{\top}$
$\displaystyle=\hat{F}^{\top}-\hat{\Lambda}^{-1}\hat{F}^{\top}\frac{1}{T}FB^{\top}BF^{\top}=\hat{F}^{\top}-\hat{\Lambda}^{-1}\hat{F}^{\top}\frac{1}{T}(Y^{\top}Y-U^{\top}U-FB^{\top}U-U^{\top}BF^{\top})$
$\displaystyle=\hat{F}^{\top}-\hat{\Lambda}^{-1}\hat{F}^{\top}\frac{1}{T}Y^{\top}Y+\frac{1}{T}\hat{\Lambda}^{-1}\hat{F}^{\top}(U^{\top}U+FB^{\top}U+U^{\top}BF^{\top}).$

Using the spectral theorem we get the representation
$T^{-1}Y^{\top}Y=\hat{E}\hat{\Lambda}\hat{E}^{\top}$ $+\hat{M}^{c}$, with
$\hat{E}^{\top}\hat{M}^{c}=0$. Hence
$\hat{F}^{\top}-T^{-1}\hat{\Lambda}^{-1}\hat{F}^{\top}Y^{\top}Y=0.$ We thus
obtain $\hat{F}^{\top}-HF^{\top}=A_{1}+A_{2}+A_{3},$ where

$\displaystyle
A_{1}=\frac{1}{T}(\hat{\Lambda}/p)^{-1}\hat{F}^{\top}\frac{U^{\top}U}{p},\quad
A_{2}=\frac{1}{T}(\hat{\Lambda}/p)^{-1}\hat{F}^{\top}\frac{U^{\top}BF^{\top}}{p},\quad
A_{3}=\frac{1}{T}(\hat{\Lambda}/p)^{-1}\hat{F}^{\top}\frac{FB^{\top}U}{p}.$

Notice that $\hat{F}^{\top}-HF^{\top}$ is an $L\times T$ random matrix. The
$t$-th column can be written as

$\hat{f}_{t}-Hf_{t}=\frac{1}{T}(\hat{\Lambda}/p)^{-1}\hat{F}^{\top}\left(\frac{U^{\top}u_{t}}{p}+\frac{U^{\top}Bf_{t}}{p}+\frac{FB^{\top}u_{t}}{p}\right)=:A_{1t}+A_{2t}+A_{3t}.$
(10)

The following lemma provides the order of magnitude of $\max_{1\leq t\leq
T}\|A_{mt}\|$, $m\in\\{1,2,3\\}$.

######  Lemma 4.

Under Assumptions 1 and 2 we have

$\displaystyle\frac{\hat{\gamma}_{L}}{p}\max_{t=1}^{T}\|A_{1t}\|$
$\displaystyle=O_{P}\left(\sqrt{L}\left(1/T^{1/4}+T^{1/4}/\sqrt{p}\right)\right),$
(11) $\displaystyle\frac{\hat{\gamma}_{L}}{p}\max_{t=1}^{T}\|A_{2t}\|$
$\displaystyle=O_{P}\left(\sqrt{L}T^{1/4}/\sqrt{p}\right),$ (12)
$\displaystyle\frac{\hat{\gamma}_{L}}{p}\max_{t=1}^{T}\|A_{3t}\|$
$\displaystyle=O_{P}\left(\sqrt{L}T^{1/4}/\sqrt{p}\right),$ (13)

as $T\to\infty$. Hence,
$R^{(1)}=O_{P}\left((p/\hat{\gamma}_{L})\sqrt{L}\left(1/T^{1/4}+T^{1/4}/\sqrt{p}\right)\right)$.

Next we study the order of magnitude of $R^{(2)}$. By the definition of
$\hat{B}$ and $H$ we get

$\hat{b}_{j}-Hb_{j}=\frac{1}{T}\hat{F}^{\top}Y_{j}-Hb_{j}=\frac{1}{T}\hat{F}^{\top}Y_{j}-\frac{1}{T}HF^{\top}(Y_{j}-U_{j})+\left(\frac{1}{T}HF^{\top}(Y_{j}-U_{j})-Hb_{j}\right).$

Rearranging the terms we thus get

$\hat{b}_{j}-Hb_{j}=\frac{1}{T}\left(\hat{F}^{\top}-HF^{\top}\right)Y_{j}+\frac{1}{T}HF^{\top}U_{j}+H\left(\frac{F^{\top}F}{T}-I_{L}\right)b_{j}=:B_{1j}+B_{2j}+B_{3j}.$

The following lemma provides the order of magnitude of
$\max_{j=1}^{p}\|B_{mj}\|$, $m\in\\{1,2,3\\}$.

######  Lemma 5.

Consider Assumptions 1, 2 and 3. Suppose that $L/(T\lambda_{L}^{2})\to 0$ and
that $p$ diverges at a subexponential rate. Then, as $T\to\infty$,

$\displaystyle\frac{\hat{\gamma}_{L}}{p}\max_{j=1}^{p}\|B_{1j}\|$
$\displaystyle=O_{P}\left(L\left(1/\sqrt{T}+1/\sqrt{p}\right)\right),$ (14)
$\displaystyle\frac{\hat{\gamma}_{L}}{p}\max_{j=1}^{p}\|B_{2j}\|$
$\displaystyle=O_{P}\left(\frac{L^{3/2}}{\sqrt{\lambda_{L}}}\sqrt{\frac{\log(pL)}{T}}\right),$
(15) $\displaystyle\frac{\hat{\gamma}_{L}}{p}\max_{j=1}^{p}\|B_{3j}\|$
$\displaystyle=O_{P}\left(\frac{L^{2}}{\lambda_{L}^{3/2}\sqrt{T}}\right).$
(16)

Thus, if $m_{T}$ denotes the maximum of the rates in (14)–(16), then
$R^{(2)}=O_{P}\left(m_{T}(p/\hat{\gamma}_{L}\right)).$

######  Lemma 6.

Under Assumption 2 (ii), $R^{(3)}\leq C_{X}^{1/4}.$

######  Lemma 7.

Under Assumptions 2 and 3

$R^{(5)}=O_{P}\left(\frac{p}{\hat{\gamma}_{L}}\frac{L}{\sqrt{\lambda_{L}}}\right).$

######  Lemma 8.

Under Assumptions 1, 2 and 3, as $T\to\infty$,

$\displaystyle R^{(6)}$
$\displaystyle=O_{P}\left(\left(\frac{p}{\hat{\gamma}_{L}}\right)^{4}\frac{L^{5}}{\lambda_{L}^{3}}\left(\frac{1}{\sqrt{T}}+\frac{1}{\sqrt{p}}\right)\right).$
(17)

###### Proof of Theorem 1:.

The proof is immediate from Lemmas 4–8. The assumption
$p/\hat{\gamma}_{L}=O_{P}(1)$ implies that $R^{(1)}$ and $R^{(2)}$ are both
$O_{P}\left(1/T^{1/4}+T^{1/4}/\sqrt{p}\right)$. The terms $R^{(3)}$, $R^{(5)}$
and $\|f_{t}\|$ are $O_{P}(1)$ and
$R^{(6)}=O_{P}\left(1/\sqrt{T}+1/\sqrt{p}\right)$ . Now use (8) and (9). ∎

###### Proof of Theorem 2:.

We use again Lemmas 4–8. With our assumption
$p/\hat{\gamma}_{L}=O_{P}(L^{\beta})$ and the lower bound for the eigenvalues
we get that
$R^{(1)}=O_{P}\left(L^{\beta+1/2}(1/T^{1/4}+T^{1/4}/\sqrt{p})\right)$ and
$R^{(2)}=O_{P}\left(L^{\beta+1}(1/T^{1/4}+T^{1/4}/\sqrt{p})\right)$. The term
$R^{(3)}$ is bounded and by the Markov inequality $\|f_{t}\|=O_{P}(\sqrt{L})$.
Finally, $R^{(5)}=O_{P}(L^{\beta+\nu/2+1})$ and
$R^{(6)}=O_{P}\big{(}L^{4\beta+3\nu+5}(1/\sqrt{T}+1/\sqrt{p})\big{)}$. Now use
(8) for the non-uniform bound. Since we assume a subpolynomial rate for $L$,
we see that the dominating term is $R^{(2)}R^{(5)}\|f_{t}\|$. For the uniform
bound we use (9). Here the dominating term is $R^{(2)}R^{(5)}R^{(4)}$. A
simple generalisation of Lemma 2 yields that for any $q^{\prime}<q$ we have
$R^{(4)}=o_{P}(L^{1/2+\nu/2}T^{1/q^{\prime}})$. ∎

###### Proof of Corollary 1:.

We let $\hat{X}_{t}(s_{i}|K)$ be the estimated components in (2) with
$\hat{E}=(\hat{e}_{1},\ldots,\hat{e}_{K})$. When $L$ is unknown our estimator
is of the form $\hat{X}_{t}(s_{i}|\hat{L})$. Moreover,
$\hat{X}_{t}(s_{i}|L)=\hat{X}_{t}(s_{i})$ as it was used in Theorems 1 and 2.
From those theorems we have statements of the form
$\Pr\left(\max_{1\leq t\leq T}\max_{1\leq j\leq
p}|\hat{X}_{t}(s_{i}|L)-X_{t}(s_{i})|>a_{T}\right)\to 0.$ This remains true if
we replace $L$ by $\hat{L}$, since

$\displaystyle\Pr\left(\max_{1\leq t\leq T}\max_{1\leq j\leq
p}|\hat{X}_{t}(s_{i}|\hat{L})-X_{t}(s_{i})|>a_{T}\right)\leq\Pr\left(\max_{1\leq
t\leq T}\max_{1\leq j\leq
p}|\hat{X}_{t}(s_{i}|L)-X_{t}(s_{i})|>a_{T}\right)+\Pr\left(\hat{L}\neq
L\right)\to 0.$

∎

##  4 Conclusion

In this paper we develop the theoretical foundation to a factor model based
approach for preprocessing functional data. We show that the recovery of the
underlying signal from noisy functional data observations can be done
consistently using a simple and straight-forward application of a principal
component factor model. We relax many assumptions on approximate factor models
as well as comparable FDA smoothing techniques and provide a general and neat
setting which appears realistic in real life applications. We extend common
factor analysis approaches by considering the case where the number of factors
may grow with the dimensions of the dataset. Furthermore, we expand upon
comparable FDA smoothing techniques by including the whole dataset in the
signal recovery instead of applying common curve-by-curve techniques.
Practical applications as well as extensive simulation studies are presented
in the companion paper Hörmann and Jammoul (2021).

##  Appendix A Proofs of technical lemmas

In this appendix we prove the lemmas stated in Section 3 and some further
technical lemmas. We start by reviewing some further background and notation.

All random variables below are defined on a common probability space
$(\Omega,\mathcal{A},P)$. We note that by our assumption the processes $X_{t}$
are elements in $H=L^{2}([0,1])$—the space of square integrable functions on
the interval $[0,1]$. This space is equipped with scalar product $\langle
x,y\rangle=\int_{0}^{1}x(t)y(t)dt$ and norm $\|x\|=\sqrt{\langle x,x\rangle}$.
For the sake of a light notation we will use $\|\cdot\|$ also for the
Euclidean norm and the spectral norm of a matrix. It will be clear from the
context what type of norm it indicates. We use $\|\cdot\|_{F}$ for the
Frobenius norm.

The signals $X_{t}$ can also be viewed as elements in
$L^{2}_{H}=L^{2}_{H}(\Omega,\mathcal{A},P)$, i.e. the space of $H$-valued
random elements $X$ with $E\|X\|^{2}<\infty$. The space $L^{2}_{H}$ is also a
Hilbert space with inner product $E\langle X,Y\rangle$.

The outer product $\otimes$ between two functions $x,y\in H$ defines a
Hilbert-Schmidt (HS) operator through $x\otimes y(z)=x\langle z,y\rangle$. The
set of Hilbert-Schmidt (HS) operators on $H$ will be denoted by $\mathcal{S}$.
Let $(e_{i})$ be an orthonormal basis (ONB) of $H$. Then the space
$\mathcal{S}$ equipped with the scalar product $\langle
W,V\rangle_{\mathcal{S}}:=\sum_{i\geq 1}\langle W(e_{i}),V(e_{i})\rangle$
(which can be shown to be independent of the ONB) is another separable Hilbert
space. It can be easily seen that $E\|X\|^{4}<\infty$, implies $E\|X\otimes
X\|_{\mathcal{S}}^{2}<\infty$ and thus $X\otimes X\in
L^{2}_{\mathcal{S}}(\Omega,\mathcal{A},P)$. Then we may define
$\Omega^{X}:=\mathrm{Var}(X\otimes X)$. For the definition of covariance
operators on Hilbert spaces we refer to Bosq (2000).

We use $\stackrel{{\scriptstyle d}}{{\to}}$ to indicate convergence in
distribution. A normal random element in some separable Hilbert space $H$ with
mean $\mu$ and covariance operator $\Omega$ is denoted by
$\mathcal{N}_{H}(\mu,\Omega)$.

### Preliminary lemmas

######  Lemma 9.

Define $W_{T}:=\sum_{t=1}^{T}X_{t}\otimes X_{t}$. Under Assumption 2 (i)

  1. (i)

$\frac{1}{\sqrt{T}}\big{(}W_{T}-T\,\Gamma^{X}\big{)}\stackrel{{\scriptstyle
d}}{{\to}}\mathcal{N}_{\mathcal{S}}(0,\Omega^{X}),$

  2. (ii)

$E\big{\|}W_{T}-T\,\Gamma^{X}\big{\|}_{\mathcal{S}}^{2}\leq c_{1}\,T$, where
$c_{1}$ is independent of $T$.

###### Proof:.

Assumption 2 (i) implies that the process $(X_{t}\otimes X_{t}\colon t\geq 1)$
is $L^{2}$-$m$-approximable in $\mathcal{S}$. This follows by a simple
modification of the proof of Lemma 2.1 in Hörmann and Kokoszka (2010). Thus
Theorem 8 in Hörmann and Cerovecki (2017) applies and this in turn yields the
central limit theorem (see Theorem 5 in Hörmann and Cerovecki (2017)). Part
(ii) follows directly from Theorem 3.1 in Hörmann and Kokoszka (2010). ∎

######  Lemma 10.

Under Assumption 1 we have a constant $c_{2}$ which is independent of $p$ such
that for $k\in\\{1,2\\}$

  1. (i)

$E\big{(}u_{2}^{\top}u_{1}\big{)}^{2k}\leq c_{2}\,p^{k}$,

  2. (ii)

$E\|u_{1}\|^{4k}\leq c_{2}\,p^{2k}$.

###### Proof:.

We show (i). The proof of (ii) is along the same lines. The main ingredient is
Isserlis’ theorem, which states that for a zero-mean multivariate normal
vector $(Z_{1},\ldots,Z_{2k})^{\top}$ the product moments are given by

$EZ_{1}\cdots
Z_{2k}=\sum_{\mathfrak{p}\in\mathcal{P}[1,\ldots,2k]}\prod_{\\{i,j\\}\in\mathfrak{p}}\mathrm{Cov}(Z_{i},Z_{j}),$
(18)

where $\mathcal{P}[I]$ denotes the set of all pairings of the components of
some index vector $I$. Denote
$u_{1}=(\varepsilon_{1},\ldots,\varepsilon_{p})^{\top}$. Since $u_{1}$ and
$u_{2}$ are i.i.d. we get
$E\big{(}u_{2}^{\top}u_{1}\big{)}^{2k}=\sum_{i_{1},\ldots,i_{2k}=1}^{p}(E\varepsilon_{i_{1}}\ldots\varepsilon_{i_{2k}})^{2},$
and thus

$\displaystyle E\big{(}u_{2}^{\top}u_{1}\big{)}^{2k}$
$\displaystyle=\sum_{i_{1},\ldots,i_{2k}=1}^{p}\left(\sum_{\mathfrak{p}\in\mathcal{P}[(i_{1},\ldots,i_{2k})]}\prod_{\\{\ell,j\\}\in\mathfrak{p}}\gamma^{u}(\ell-j)\right)^{2}$
$\displaystyle\leq\sum_{i_{1},\ldots,i_{2k}=1}^{p}\\#\mathcal{P}[(i_{1},\ldots,i_{2k})]\sum_{\mathfrak{p}\in\mathcal{P}[(i_{1},\ldots,i_{2k})]}\prod_{\\{\ell,j\\}\in\mathfrak{p}}\big{(}\gamma^{u}(\ell-j)\big{)}^{2}$
$\displaystyle=(\\#\mathcal{P}[(1,\ldots,2k)])^{2}\left(\sum_{\ell,j=1}^{p}\big{(}\gamma^{u}(\ell-j)\big{)}^{2}\right)^{k}\leq\big{(}(2k-1)!!\big{)}^{2}\big{(}C_{u}\gamma^{u}(0)\big{)}^{k}p^{k}.$

The last inequality was deduced from Assumption 1. ∎

######  Lemma 11.

Under Assumption 2 (i)

$\left\|\frac{1}{T}F^{\top}F-I_{L}\right\|_{F}=O_{P}\left(\frac{L}{\lambda_{L}\sqrt{T}}\right).$

###### Proof:.

Denoting by $\delta_{ij}$ the Kronecker delta, we notice that the $i,j$-th
entry of the $(L\times L)$-matrix $T^{-1}F^{\top}F-I_{L}$ can be written as

$\frac{1}{T}\sum_{t=1}^{T}f_{it}f_{jt}-\delta_{ij}=\frac{1}{\sqrt{T}}\frac{1}{\sqrt{\lambda_{i}\lambda_{j}}}\bigg{\langle}\frac{1}{\sqrt{T}}\sum_{t=1}^{T}(X_{t}\otimes
X_{t}-\Gamma^{X})(\varphi_{i}),\varphi_{j}\bigg{\rangle}.$

This shows that

$\left\|\frac{1}{T}F^{\top}F-I_{L}\right\|_{F}\leq\frac{1}{\sqrt{T}}\left\|\frac{1}{\sqrt{T}}\big{(}W_{T}-T\,\Gamma^{X}\big{)}\right\|_{\mathcal{S}}\left(\sum_{i,j=1}^{L}\frac{1}{\lambda_{i}\lambda_{j}}\right)^{1/2}.$

By Lemma 9 (i) we infer that
$\|T^{-1/2}\big{(}W_{T}-T\,\Gamma^{X}\big{)}\|_{\mathcal{S}}=O_{P}(1)$ and
hence the result follows. ∎

For the next result we note that the $k$-th row of $\hat{f}_{t}-Hf_{t}$ is
given by

$\frac{1}{T}\frac{p}{\hat{\gamma}_{k}}\hat{F}_{k}\left(\frac{U^{\top}u_{t}}{p}+\frac{U^{\top}Bf_{t}}{p}+\frac{FB^{\top}u_{t}}{p}\right)=:A_{1tk}+A_{2tk}+A_{3tk},$
(19)

where $\hat{F}_{k}$ is the $k$-th row of $\hat{F}^{\top}$.

######  Lemma 12.

Under Assumptions 1 and 2

$\displaystyle\sum_{k=1}^{L}\frac{1}{T}\sum_{t=1}^{T}A_{1tk}^{2}$
$\displaystyle=O_{P}\left(L^{2}(\hat{\gamma}_{L}/p)^{-2}\big{(}1/T+1/p\big{)}\right),$
(20) $\displaystyle\sum_{k=1}^{L}\frac{1}{T}\sum_{t=1}^{T}A_{2tk}^{2}$
$\displaystyle=O_{P}\left(L^{2}(\hat{\gamma}_{L}/p)^{-2}/p\right),$ (21)
$\displaystyle\sum_{k=1}^{L}\frac{1}{T}\sum_{t=1}^{T}A_{3tk}^{2}$
$\displaystyle=O_{P}\left(L^{2}(\hat{\gamma}_{L}/p)^{-2}/p\right).$ (22)

###### Proof:.

We show (20) and (21). For (20) we use that

$\displaystyle(\hat{\gamma}_{k}/p)^{2}\frac{1}{T}\sum_{t=1}^{T}A_{1tk}^{2}=\frac{1}{T}\sum_{t=1}^{T}\frac{1}{T^{2}p^{2}}\left(\hat{F}_{k}U^{\top}u_{t}\right)^{2}=\frac{1}{T}\sum_{t=1}^{T}\frac{1}{T^{2}p^{2}}\left(\sum_{\ell=1}^{T}\hat{f}_{k\ell}u_{\ell}^{\top}u_{t}\right)^{2}\leq\frac{1}{T^{2}}\sum_{t=1}^{T}\sum_{\ell=1}^{T}\left(\frac{u_{\ell}^{\top}u_{t}}{p}\right)^{2}.$

In the last step we used the Cauchy-Schwarz inequality and the fact that
$\hat{f}_{k\ell}$ ($\ell$-th column and $k$-th row of $\hat{F}^{\top}$)
satisfies $\sum_{\ell=1}^{T}\hat{f}_{k\ell}^{2}=T$. Using the Markov
inequality with Assumption 1 and Lemma 10 we get for any $\kappa>0$

$\displaystyle\Pr\left(\frac{1}{T^{2}}\sum_{t=1}^{T}\sum_{\ell=1}^{T}\left(\frac{u_{\ell}^{\top}u_{t}}{p}\right)^{2}>\kappa\big{(}1/T+1/p\big{)}\right)$
$\displaystyle\leq\Pr\left(\frac{1}{T}\sum_{t=1}^{T}\left(\frac{u_{t}^{\top}u_{t}}{p}\right)^{2}>\kappa\right)+\Pr\left(\frac{1}{T^{2}}\sum_{t=1}^{T}\sum_{\ell=1\atop\ell\neq
t}^{T}\left(\frac{u_{\ell}^{\top}u_{t}}{p}\right)^{2}>\kappa/p\right)$
$\displaystyle\leq
E\left(\frac{u_{1}^{\top}u_{1}}{p}\right)^{2}\Big{/}\kappa+\frac{T-1}{T}E\left(\frac{u_{1}^{\top}u_{2}}{p}\right)^{2}\frac{p}{\kappa}\leq\frac{c_{2}\left(1+\frac{T-1}{T}\right)}{\kappa}.$

Thus

$\displaystyle\Pr\left(\sum_{k=1}^{L}\frac{1}{T}\sum_{t=1}^{T}A_{1tk}^{2}>L^{2}(\hat{\gamma}_{L}/p)^{-2}\big{(}1/T+1/p\big{)}\kappa\right)$
$\displaystyle\leq\Pr\left(\sum_{k=1}^{L}(\hat{\gamma}_{k}/p)^{2}\frac{1}{T}\sum_{t=1}^{T}A_{1tk}^{2}>L^{2}\big{(}1/T+1/p\big{)}\kappa\right)$
$\displaystyle\leq\sum_{k=1}^{L}\Pr\left((\hat{\gamma}_{k}/p)^{2}\frac{1}{T}\sum_{t=1}^{T}A_{1tk}^{2}>L\big{(}1/T+1/p\big{)}\kappa\right)=O(1/\kappa).$

The first statement of the lemma is shown. By similar arguments as above we
obtain, using $Bf_{t}=X_{t}(\boldsymbol{s})$,

$\displaystyle(\hat{\gamma}_{k}/p)^{2}\frac{1}{T}\sum_{t=1}^{T}A_{2tk}^{2}$
$\displaystyle\leq\frac{1}{T^{2}}\sum_{t=1}^{T}\sum_{\ell=1}^{T}\left(\frac{u_{\ell}^{\top}X_{t}(\boldsymbol{s})}{p}\right)^{2}.$

Then, using the independence between $X_{t}(\boldsymbol{s})$ and $u_{\ell}$
(Assumption 1), we get that

$\displaystyle
E\left(\frac{u_{\ell}^{\top}X_{t}(\boldsymbol{s})}{p}\right)^{2}$
$\displaystyle=\frac{1}{p^{2}}E\left(u_{\ell}^{\top}\Gamma^{X}(\boldsymbol{s},\boldsymbol{s})u_{\ell}\right)=\frac{1}{p^{2}}\sum_{i=1}^{p}\sum_{j=1}^{p}\gamma^{u}(i-j)\Gamma^{X}(s_{i},s_{j})$
$\displaystyle\leq\frac{1}{p^{2}}\sum_{i=1}^{p}\sum_{j=1}^{p}|\gamma^{u}(i-j)|\max_{i,j}|\Gamma^{X}(s_{i},s_{j})|\leq
C_{u}\frac{\sup_{s\in[0,1]}EX^{2}(s)}{p}.$

By Assumption 2 (ii) we have $\sup_{s\in[0,1]}EX^{2}(s)<\infty$ and so (21)
follows with similar arguments as before using the Markov inequality. The
bound for (22) can be derived in complete analogy to (21). ∎

######  Lemma 13.

Under Assumptions 1 and 2 and if $L/(T\lambda_{L}^{2})\to 0$ we have

$\max_{j\leq
p}\left\|\frac{1}{T}F^{\top}U_{j}\right\|=O_{P}\left(\sqrt{L}\big{(}\log(pL)/T\big{)}^{1/2}\right).$

###### Proof:.

We have

$\max_{j\leq p}\left\|\frac{1}{T}F^{\top}U_{j}\right\|=\max_{j\leq
p}\left\|\frac{1}{T}\sum_{t=1}^{T}f_{t}u_{tj}\right\|\leq\sqrt{L}\max_{j\leq
p\atop i\leq
L}\left|\frac{1}{T}\sum_{t=1}^{T}\frac{1}{\sqrt{\lambda_{i}}}\langle
X_{t},\varphi_{i}\rangle u_{tj}\right|.$

Then, we note that for any $\kappa>0$

$\displaystyle\Pr\left(\max_{j\leq p\atop i\leq
L}\left|\frac{1}{T}\sum_{t=1}^{T}\frac{1}{\sqrt{\lambda_{i}}}\langle
X_{t},\varphi_{i}\rangle u_{tj}\right|>\kappa\right)$
$\displaystyle\quad\leq\Pr\left(\max_{j\leq p\atop i\leq
L}\left|\frac{1}{T}\sum_{t=1}^{T}\frac{1}{\sqrt{\lambda_{i}}}\langle
X_{t},\varphi_{i}\rangle u_{tj}>\kappa\right|\cap\max_{k\leq
L}\frac{1}{T}\sum_{t=1}^{T}\frac{1}{\lambda_{k}}\langle
X_{t},\varphi_{k}\rangle^{2}\leq 2\right)+\Pr\left(\max_{k\leq
L}\frac{1}{T}\sum_{t=1}^{T}\frac{1}{\lambda_{k}}\langle
X_{t},\varphi_{k}\rangle^{2}>2\right)$
$\displaystyle\quad=:p_{1}(\kappa)+p_{2}.$

For $p_{2}$ we get with Lemma 9 (ii) and the Markov inequality

$\displaystyle p_{2}$
$\displaystyle\leq\sum_{k=1}^{L}\Pr\left(\bigg{\langle}\Big{(}\sum_{t=1}^{T}\Big{(}X_{t}\otimes
X_{t}-EX_{t}\otimes
X_{t}\Big{)}\Big{)}(\varphi_{k}),\varphi_{k}\bigg{\rangle}>\lambda_{k}T\right)$
$\displaystyle\leq\sum_{k=1}^{L}\Pr\left(\bigg{\|}\sum_{t=1}^{T}\Big{(}X_{t}\otimes
X_{t}-EX_{t}\otimes X_{t}\Big{)}\bigg{\|}_{\mathcal{S}}>\lambda_{k}T\right)$
$\displaystyle\leq\sum_{k=1}^{L}E\bigg{\|}\sum_{t=1}^{T}W_{T}-T\Gamma^{X}\bigg{\|}_{\mathcal{S}}^{2}\bigg{/}\Big{(}\lambda_{k}^{2}T^{2}\Big{)}\leq\frac{c_{1}L}{\lambda_{L}^{2}T}.$

To derive the probability $p_{1}(\kappa)$ we first condition on
$X_{1},\ldots,X_{T}$. Then

$\displaystyle\Pr\left(\max_{j\leq p\atop i\leq
L}\Big{|}\frac{1}{T}\sum_{t=1}^{T}\frac{1}{\sqrt{\lambda_{i}}}\langle
X_{t},\varphi_{i}\rangle u_{tj}\Big{|}>\kappa\cap\max_{k\leq
L}\frac{1}{T}\sum_{t=1}^{T}\frac{1}{\lambda_{k}}\langle
X_{t},\varphi_{k}\rangle^{2}\leq 2\bigg{|}X_{1},\ldots,X_{T}\right)$
$\displaystyle\quad\leq\sum_{j\leq p\atop i\leq
L}\Pr\left(\Big{|}\frac{1}{T}\sum_{t=1}^{T}\frac{1}{\sqrt{\lambda_{i}}}\langle
X_{t},\varphi_{i}\rangle u_{tj}\Big{|}>\kappa\cap\max_{k\leq
L}\frac{1}{T}\sum_{t=1}^{T}\frac{1}{\lambda_{k}}\langle
X_{t},\varphi_{k}\rangle^{2}\leq 2\bigg{|}X_{1},\ldots,X_{T}\right).$

Conditional on $X_{1},\ldots,X_{T}$ the variable

$\frac{1}{T}\sum_{t=1}^{T}\frac{1}{\sqrt{\lambda_{i}}}\langle
X_{t},\varphi_{i}\rangle u_{tj}\sim
N\left(0,\frac{\big{(}\gamma^{u}(0)\big{)}^{2}}{T}\times\gamma_{i}^{2}(X_{1},\ldots,X_{T})\right),$

where
$\gamma_{i}^{2}(X_{1},\ldots,X_{T})=T^{-1}\sum_{t=1}^{T}\lambda_{i}^{-1}\langle
X_{t},\varphi_{i}\rangle^{2}$. By an elementary bound for the normal
distribution function (see e.g., Petrov (1995, p. 177)) we get

$\Pr(|N(0,\sigma^{2})|>x)<\frac{\sigma}{x}e^{-\frac{x^{2}}{2\sigma^{2}}},\quad
x>0.$

Thus for any $i$ and $j$

$\Pr\left(\Big{|}\frac{1}{T}\sum_{t=1}^{T}\frac{1}{\sqrt{\lambda_{i}}}\langle
X_{t},\varphi_{i}\rangle u_{tj}\Big{|}>\kappa\cap\max_{k\leq
L}\frac{1}{T}\sum_{t=1}^{T}\frac{1}{\lambda_{k}}\langle
X_{t},\varphi_{k}\rangle^{2}\leq
2\Bigg{|}X_{1},\ldots,X_{T}\right)\leq\frac{\sqrt{2}\gamma^{u}(0)}{\kappa\sqrt{T}}e^{-\frac{T\kappa^{2}}{4(\gamma^{u}(0))^{2}}}.$

By taking the expectation of the conditional probability we obtain a bound for
the unconditional probability. Then, summing over $i$ and $j$ yields

$p_{1}(\kappa)\leq\frac{\sqrt{2}\gamma^{u}(0)\times
Lp}{\kappa\sqrt{T}}e^{-\frac{T\kappa^{2}}{4(\gamma^{u}(0))^{2}}}.$

Now the proposition of the lemma follows immediately. ∎

###### Proof of Lemma 1:.

Without loss of generality we will assume that the sampling points are
$\\{s_{j}=j/p\colon 0\leq j\leq p-1\\}$. By the triangular inequality we have

$|\lambda_{i}-\hat{\gamma}_{i}/p|\leq|\lambda_{i}-\hat{\lambda}_{i}|+|\hat{\lambda}_{i}-\hat{\lambda}_{i}^{*}|+|\hat{\lambda}_{i}^{*}-\hat{\gamma}^{X}_{i}/p|+|\hat{\gamma}^{X}_{i}/p-\hat{\gamma}_{i}/p|,$
(23)

where the terms on the right hand side will now be defined and bounded. All
the variables appearing in (23) are eigenvalues of some covariance matrix or
some covariance operator. We consider the covariance matrices

$\hat{\Sigma}^{y}=\frac{1}{T}YY^{\top}=\frac{1}{T}\sum_{t=1}^{T}y_{t}y_{t}^{\top}\quad\text{and}\quad\hat{\Sigma}^{X}:=\frac{1}{T}XX^{\top}=\frac{1}{T}\sum_{t=1}^{T}X_{t}(\boldsymbol{s})X_{t}(\boldsymbol{s})^{\top},$

with eigenvalues $(\hat{\gamma}_{i})$ and $(\hat{\gamma}^{X}_{i})$,
respectively. Thereby we note that the non-zero eigenvalues of
$T^{-1}Y^{\top}Y$ and $\hat{\Sigma}^{y}$ coincide. Moreover, we consider the
covariance operators $\Gamma^{X}$, $\hat{\Gamma}^{X}$ and
$\hat{\Gamma}^{X^{*}}$, with eigenvalues $(\lambda_{i})$,
$(\hat{\lambda}_{i})$ and $(\hat{\lambda}_{i}^{*})$, respectively. Here
$\hat{\Gamma}^{X}$ is the empirical covariance operator of the fully observed
functional data $X_{1}(v),\ldots,X_{T}(v)$ and $\hat{\Gamma}^{X^{*}}$ is the
empirical covariance operator of the discretised data
$X_{t}^{*}(v):=X_{t}(s_{j})$ for $v\in[s_{j},s_{j+1})$. Note that
$\hat{\Gamma}^{X^{*}}(s_{k},s_{j})=\hat{\Gamma}^{X}(s_{k},s_{j})=\hat{\Sigma}^{X}_{kj}$.

We start with the first term on the right in (23). It follows from Weyl’s
inequality that
$|\lambda_{i}-\hat{\lambda}_{i}|\leq\|\Gamma^{X}-\hat{\Gamma}^{X}\|$. Hence,
by $L^{4}$-$m$-approximability and Theorem 3.1 in Hörmann and Kokoszka (2010)
we get

$|\lambda_{i}-\hat{\lambda}_{i}|\leq\|\Gamma^{X}-\hat{\Gamma}^{X}\|=O_{P}\left(\frac{1}{\sqrt{T}}\right).$
(24)

Again with Weyl’s inequality and then the Cauchy-Schwarz inequality we obtain

$\displaystyle|\hat{\lambda}_{i}-\hat{\lambda}_{i}^{*}|^{2}$
$\displaystyle\leq\int\int\left(\frac{1}{T}\sum_{t=1}^{T}\big{(}X_{t}(r)X_{t}(s)-X^{*}_{t}(r)X^{*}_{t}(s)\big{)}\right)^{2}dsdr\leq\frac{2}{T}\sum_{t=1}^{T}\left(\big{\|}X_{t}\big{\|}^{2}+\big{\|}X_{t}^{*}\big{\|}^{2}\right)\frac{1}{T}\sum_{t=1}^{T}\big{\|}X_{t}-X^{*}_{t}\big{\|}^{2}$
$\displaystyle=O_{P}(1)\sum_{j=0}^{p-1}\int_{0}^{1/p}\big{(}X_{t}(j/p+h)-X_{t}(j/p)\big{)}^{2}dh.$

With (5) it is then immediate that
$|\hat{\lambda}_{i}-\hat{\lambda}_{i}^{*}|=O_{P}\left(1/\sqrt{p}\right)$.

In the next step one needs to study the relationship between the eigenvalues
of an operator and a matrix. Henceforth we write $[a]_{k}$ for the $k$-th
component of some vector $a$. Let $\hat{e}_{i}^{X}$ be the eigenvector
associated to $\hat{\gamma}_{i}^{X}$ and let
$\hat{\phi}_{i}(s)=[\hat{e}_{i}^{X}]_{k}$ for $s\in[(k-1)/p,k/p)$,
$k\in\\{1,\ldots,p\\}$. Then

$\displaystyle\hat{\gamma}_{i}^{X}[\hat{e}_{i}^{X}]_{k}=[\hat{\Sigma}^{X}\hat{e}_{i}^{X}]_{k}$
$\displaystyle=\sum_{j=1}^{p}\hat{\Gamma}^{X}(s_{k},s_{j})[e_{i}^{X}]_{j}=\sum_{j=1}^{p}p\int_{s_{j}}^{s_{j+1}}\hat{\Gamma}^{X^{*}}(s_{k},s)\hat{\phi}_{i}(s)ds.$

Thus, for any $v\in[0,1]$

$\int_{0}^{1}\hat{\Gamma}^{X^{*}}(v,s)\hat{\phi}_{i}(s)ds=\frac{\hat{\gamma}_{i}^{X}}{p}\hat{\phi}_{i}(v),$

showing that $\hat{\gamma}_{i}^{X}/p=\hat{\lambda}_{i}^{*}$.

Finally,

$\displaystyle|\hat{\gamma}_{i}-\hat{\gamma}^{X}_{i}|$
$\displaystyle\leq\|\hat{\Sigma}^{y}-\hat{\Sigma}^{X}\|=\frac{1}{T}\left\|UU^{\top}+XU^{\top}+UX^{\top}\right\|\leq\frac{1}{T}\left\|UU^{\top}\right\|+\frac{2}{T}\left\|XU^{\top}\right\|.$

Denote $\Gamma^{u}=\mathrm{Var}(u_{t})$ and $Z=[Z_{1},\ldots,Z_{T}]$ with
$Z_{t}\stackrel{{\scriptstyle\text{iid}}}{{\sim}}N_{p}(0,I_{p})$. Then

$\frac{1}{T}\left\|UU^{\top}\right\|\stackrel{{\scriptstyle
d}}{{=}}\frac{1}{T}\left\|(\Gamma^{u})^{1/2}ZZ^{\top}(\Gamma^{u})^{1/2}\right\|\leq\|\Gamma^{u}\|\frac{1}{T}\left\|ZZ^{\top}\right\|.$

By a basic inequality for matrix norms, we have
$\|\Gamma^{u}\|\leq\sqrt{\|\Gamma^{u}\|_{\text{c}}\|\Gamma^{u}\|_{\text{r}}}$,
where $\|A\|_{\text{c}}$ and $\|A\|_{\text{r}}$ denote the maximal absolute
column sum and the maximal absolute row sum of some matrix $A$, respectively.
In particular, this implies that
$\|\Gamma^{u}\|\leq\sum_{h\in\mathbb{Z}}|\gamma^{u}(h)|\leq C_{u}$ (Assumption
1). By results on the distribution of the largest eigenvalue of random
covariance matrices in Johnstone (2001) and Karoui (2005), one can deduce that

$\frac{1}{T}\left\|ZZ^{\top}\right\|=\begin{cases}O_{P}(1),&\text{
}p/T\rightarrow\gamma\in[0,\infty),\\\ O_{P}(p/T),&\text{
}p/T\rightarrow\infty.\end{cases}$

Next we observe that

$\displaystyle\frac{1}{T}\left\|XU^{\top}\right\|\leq\frac{1}{T}\|X\|\|U^{\top}\|\stackrel{{\scriptstyle
d}}{{=}}\frac{1}{T}\|X\|\|Z^{\top}(\Gamma^{u})^{1/2}\|\leq\left(\frac{1}{T}\|XX^{\top}\|\|\Gamma^{u}\|\frac{1}{T}\|ZZ^{\top}\|\right)^{1/2}.$

By our previous arguments we see that this last term is
$O_{P}\left(T^{-1}\|XX^{\top}\|\max\\{p/T,1\\}\right)^{1/2}$. Moreover,

$\frac{1}{T}\|XX^{\top}\|\leq\mathrm{tr}\left(\hat{\Sigma}^{X}\right)=\frac{1}{T}\sum_{t=1}^{T}\sum_{i=1}^{p}X_{t}^{2}(s_{i}),$

and hence $T^{-1}\|XX^{\top}\|=O_{P}(p)$. Thus, we can conclude

$|\hat{\gamma}_{i}/p-\hat{\gamma}^{X}_{i}/p|=\begin{cases}O_{P}\left(\frac{1}{\sqrt{p}}\right),&\text{
}p/T\rightarrow\gamma\in[0,\infty);\\\
O_{P}\left(\frac{1}{\sqrt{T}}\right),&\text{
}p/T\rightarrow\infty.\end{cases}$

The proof follows. ∎

###### Proof of Lemma 2:.

It holds by the Bonferroni and the Markov inequality that for any $\epsilon>0$

$\Pr\left(\max_{1\leq t\leq T}\|f_{t}\|>\epsilon
T^{1/q^{\prime}}\right)\leq\sum_{t=1}^{T}\max_{1\leq t\leq
T}\frac{E\|f_{t}\|^{q}}{T^{q/q^{\prime}}\epsilon^{q}}.$

By the stationarity of $(X_{t})$ and $\langle
X_{1},\varphi_{i}\rangle^{2}\leq\|X_{1}\|^{2}$ we have

$\displaystyle E\|f_{t}\|^{q}=E\left(\sum_{i=1}^{L}\frac{\langle
X_{1},\varphi_{i}\rangle^{2}}{\lambda_{i}}\right)^{q/2}\leq
E\|X_{1}\|^{q}\left(L/\lambda_{L}\right)^{q/2}.$

The statement (i) follows. For statement (ii) we use

$\Pr(\max_{1\leq t\leq T}\|f_{t}\|>\epsilon\log
T)\leq\sum_{t=1}^{T}\Pr(\exp(\alpha\|f_{t}\|)>T^{\alpha\epsilon}).$

Now using again the Markov inequality and choosing $\epsilon>1/\alpha$, this
last term tends to zero. ∎

###### Proof of Lemma 3:.

We are going to show that

$\Pr\left(\left|\frac{1}{\lambda_{L}}-\frac{p}{\hat{\gamma}_{L}}\right|>\frac{1}{\lambda_{L}}\right)\leq
2\Pr\left(\left|\lambda_{L}-\frac{\hat{\gamma}_{L}}{p}\right|>\frac{\lambda_{L}}{2}\right).$
(25)

Then, by the lower bound on the eigenvalues we conclude that the right-hand
term in (25) is bounded by
$2\Pr\left(\left|\lambda_{L}-\hat{\gamma}_{L}/p\right|>\rho
L^{-\nu}/2\right).$ Since we assume that Lemma 1 applies, and since we assume
$L^{2\nu}/\max\\{p,T\\}\to 0$, this term goes to zero.

As for (25) we remark that for a random variable $Y>0$ and some $x>0$, it
holds that

$\displaystyle\Pr\left(\left|\frac{1}{x}-\frac{1}{Y}\right|>\frac{1}{x}\right)=\Pr\left(|Y-x|>Y\right)\leq\Pr\left(|Y-x|>x/2\right)+\Pr(Y\leq
x/2),$

while $\Pr(Y\leq x/2)=\Pr(x-Y\geq x/2)\leq\Pr(|Y-x|\geq x/2)$.

∎

######  Proof of Lemma 4:.

Recall that

$A_{1t}=\frac{1}{T}(\hat{\Lambda}/p)^{-1}\hat{F}^{\top}\frac{U^{\top}u_{t}}{p}=\frac{1}{T}(\hat{\Lambda}/p)^{-1}\sum_{\ell=1}^{T}\frac{1}{p}\hat{f}_{\ell}u_{\ell}^{\top}u_{t}$

and that $\sum_{\ell=1}^{T}\|\hat{f}_{\ell}\|^{2}=TL$. Thus, by the triangular
inequality and the Cauchy-Schwarz inequality it follows that

$\|A_{1t}\|\leq\frac{p}{\hat{\gamma}_{L}}\frac{1}{T}\left\|\sum_{\ell=1}^{T}\frac{1}{p}\hat{f}_{\ell}u_{\ell}^{\top}u_{t}\right\|\leq\frac{p}{\hat{\gamma}_{L}}\frac{\sqrt{TL}}{T}\sqrt{\sum_{\ell=1}^{T}\frac{(u_{\ell}^{\top}u_{t})^{2}}{p^{2}}}.$

Hence

$\displaystyle\frac{\hat{\gamma}_{L}}{p}\max_{1\leq t\leq
T}\|A_{1t}\|\leq\sqrt{L/T}\left(\max_{1\leq t\leq
T}\|u_{t}\|^{2}/p+\max_{1\leq t\leq T}\sqrt{\sum_{\ell=1\atop\ell\neq
t}^{T}\frac{(u_{\ell}^{\top}u_{t})^{2}}{p^{2}}}\right)=:\sqrt{L/T}\left(\max_{1\leq
t\leq T}\|A_{1t}^{*}\|+\max_{1\leq t\leq T}\|A_{1t}^{**}\|.\right)$

The Markov inequality yields

$\displaystyle P\left(\sqrt{L/T}\max_{1\leq t\leq
T}\|A_{1t}^{*}\|>\kappa\right)\leq\kappa^{-4}\frac{L^{2}}{T^{2}p^{4}}E\left[\max_{1\leq
t\leq
T}\|u_{t}\|^{8}\right]\leq\kappa^{-4}\frac{L^{2}}{Tp^{4}}E\left[\|u_{1}\|^{8}\right].$

From Lemma 10 (ii) we infer that

$\sqrt{L/T}\max_{1\leq t\leq
T}\|A_{1t}^{*}\|=O_{P}\Big{(}\sqrt{L}/T^{1/4}\Big{)}.$

Yet another application of the Markov inequality and Lemma 10 gives

$\displaystyle\Pr\left(\sqrt{L/T}\max_{1\leq t\leq
T}\|A_{1t}^{**}\|>\kappa\right)\leq\kappa^{-4}\frac{L^{2}}{T^{2}p^{4}}E\left[\max_{1\leq
t\leq T}\left(\sum_{\ell=1\atop\ell\neq
t}^{T}(u_{\ell}^{\top}u_{t})^{2}\right)^{2}\right]$
$\displaystyle\qquad\leq\kappa^{-4}\frac{L^{2}}{Tp^{4}}E\left[\left(\sum_{\ell=2}^{T}(u_{\ell}^{\top}u_{1})^{2}\right)^{2}\right]\leq\kappa^{-4}\frac{(T-1)L^{2}}{Tp^{4}}E\left[\sum_{\ell=2}^{T}(u_{\ell}^{\top}u_{1})^{4}\right]\leq\kappa^{-4}\frac{TL^{2}}{p^{4}}E\left[(u_{2}^{\top}u_{1})^{4}\right]\leq
c_{2}\kappa^{-4}\frac{TL^{2}}{p^{2}}.$

Hence

$\sqrt{L/T}\max_{1\leq t\leq
T}\|A_{1t}^{**}\|=O_{P}\Big{(}\sqrt{L}\frac{T^{1/4}}{\sqrt{p}}\Big{)}.$

Combining these estimates yields (11).

Next we verify (12). We recall that $Bf_{t}=X_{t}(\boldsymbol{s})$ and hence

$A_{2t}=\frac{1}{T}(\hat{\Lambda}/p)^{-1}\hat{F}^{\top}\frac{U^{\top}Bf_{t}}{p}=\frac{1}{T}(\hat{\Lambda}/p)^{-1}\frac{\sum_{\ell=1}^{T}\hat{f}_{\ell}u_{\ell}^{\top}X_{t}(\boldsymbol{s})}{p}.$

Thus

$\frac{\hat{\gamma}_{L}}{p}\max_{1\leq t\leq
T}\|A_{2t}\|\leq\frac{1}{T}\max_{1\leq t\leq
T}\left\|\sum_{\ell=1}^{T}\hat{f}_{\ell}\frac{u_{\ell}^{\top}X_{t}(\boldsymbol{s})}{p}\right\|\leq\sqrt{\frac{L}{T}\frac{1}{p^{2}}\max_{1\leq
t\leq T}\sum_{\ell=1}^{T}(u_{\ell}^{\top}X_{t}(\boldsymbol{s}))^{2}}.$

Consequently, by the Markov inequality

$\displaystyle\Pr\left(\frac{\hat{\gamma}_{L}}{p}\max_{1\leq t\leq
T}\|A_{2t}\|>\kappa\right)\leq\kappa^{-4}\frac{L^{2}}{T^{2}p^{4}}E\left[\max_{1\leq
t\leq
T}\left(\sum_{\ell=1}^{T}(u_{\ell}^{\top}X_{t}(\boldsymbol{s}))^{2}\right)^{2}\right].$

Moreover,

$\displaystyle E\max_{1\leq t\leq
T}\left(\sum_{\ell=1}^{T}(u_{\ell}^{\top}X_{t}(\boldsymbol{s}))^{2}\right)^{2}\leq
T\max_{1\leq t\leq
T}E\left(\sum_{\ell=1}^{T}(u_{\ell}^{\top}X_{t}(\boldsymbol{s}))^{2}\right)^{2}=T\max_{1\leq
t\leq
T}E\left(\sum_{\ell=1}^{T}\sum_{j=1}^{T}(u_{\ell}^{\top}X_{t}(\boldsymbol{s})X_{t}^{\top}(\boldsymbol{s})u_{j})^{2}\right)$
$\displaystyle\qquad=T(T^{2}-T)\sum_{i_{1}=1}^{p}\sum_{i_{2}=1}^{p}\sum_{j_{1}=1}^{p}\sum_{j_{2}=1}^{p}\gamma^{u}(i_{2}-i_{1})\gamma^{u}(j_{2}-j_{1})EX_{1}(s_{i_{1}})X_{1}(s_{i_{2}})X_{1}(s_{j_{1}})X_{1}(s_{j_{2}})$
$\displaystyle\qquad\quad+T^{2}\sum_{i_{1}=1}^{p}\sum_{i_{2}=1}^{p}\sum_{j_{1}=1}^{p}\sum_{j_{2}=1}^{p}Eu_{1,i_{1}}u_{1,i_{2}}u_{1,j_{1}}u_{1,j_{2}}EX_{1}(s_{i_{1}})X_{1}(s_{i_{2}})X_{1}(s_{j_{1}})X_{1}(s_{j_{2}}).$

By (18)

$Eu_{1,i_{1}}u_{1,i_{2}}u_{1,j_{1}}u_{1,j_{2}}=\gamma^{u}(i_{1}-i_{2})\gamma^{u}(j_{1}-j_{2})+\gamma^{u}(i_{1}-j_{1})\gamma^{u}(i_{2}-j_{2})+\gamma^{u}(i_{1}-j_{2})\gamma^{u}(i_{2}-j_{1}).$

The Cauchy-Schwarz inequality applied twice gives

$|EX_{1}(s_{i_{1}})X_{1}(s_{i_{2}})X_{1}(s_{j_{1}})X_{1}(s_{j_{2}})|\leq\left(EX_{1}^{4}(s_{i_{1}})EX_{1}^{4}(s_{i_{2}})EX_{1}^{4}(s_{j_{1}})EX_{1}^{4}(s_{j_{2}})\right)^{1/4}.$

By Assumption 2 (ii) this term is bounded by $C_{X}$. Combing these results
with Assumption 1 it is easy to derive that

$E\max_{1\leq t\leq T}\left(\sum_{\ell=1\atop\ell\neq
t}^{T}(u_{\ell}^{\top}X_{t}(\boldsymbol{s}))^{2}\right)^{2}\leq C_{X}\times
C_{u}^{2}\times\big{(}T^{3}+2T^{2})p^{2}.$

Thus

$\frac{\hat{\gamma}_{L}}{p}\max_{1\leq t\leq
T}\|A_{2t}\|=O_{P}\Big{(}\sqrt{L}\frac{T^{1/4}}{\sqrt{p}}\Big{)}.$

This yields (12). Relation (13) can be derived analogously. ∎

######  Proof of Lemma 5:.

We have

$\displaystyle\|B_{1j}\|=\left\|\frac{1}{T}\sum_{t=1}^{T}y_{tj}\left(\hat{f}_{t}^{\top}-Hf_{t}^{\top}\right)\right\|\leq\left(\frac{1}{T}\sum_{t=1}^{T}y_{tj}^{2}\frac{1}{T}\sum_{t=1}^{T}\left\|\hat{f}_{t}^{\top}-Hf_{t}^{\top}\right\|^{2}\right)^{1/2},$

where $y_{tj}=X_{t}(s_{j})+u_{tj}$. Lemma 12 implies
$T^{-1}\sum_{t=1}^{T}\big{\|}\hat{f}_{t}^{\top}-Hf_{t}^{\top}\big{\|}^{2}=O_{P}\big{(}L^{2}(\hat{\gamma}_{L}/p)^{-2}(1/T+1/p)\big{)}$.
Moreover,

$\frac{1}{T}\sum_{t=1}^{T}y_{tj}^{2}\leq
2\sup_{s\in[0,1]}\frac{1}{T}\sum_{t=1}^{T}X_{t}^{2}(s)+2\max_{k=1,\ldots,p}\frac{1}{T}\sum_{t=1}^{T}u_{tk}^{2}.$

The required $L^{4}$-$m$-approximability implies that the sequence
$(\sup_{s\in[0,1]}X_{t}^{2}(s)\colon t\geq 1)$ is ergodic and hence by the
ergodic theorem and Assumption 2 (iii)

$\sup_{s\in[0,1]}\frac{1}{T}\sum_{t=1}^{T}X_{t}^{2}(s)\leq\frac{1}{T}\sum_{t=1}^{T}\sup_{s\in[0,1]}X_{t}^{2}(s)=O_{P}(1).$

Note that by Assumption 1 the variables $u_{tj}^{2}/(\gamma^{u}(0))^{2}$ are
$\chi^{2}$-distributed with 1 degree of freedom and are independent across
$t$. Without loss of generality let us assume that $\gamma^{u}(0)=1$. Then

$\displaystyle\Pr\left(\max_{k=1,\ldots,p}\frac{1}{T}\sum_{t=1}^{T}u_{tk}^{2}>3\right)$
$\displaystyle\leq
p\Pr\left(e^{\frac{1}{3}\sum_{t=1}^{T}u_{tk}^{2}}>e^{T}\right)\leq
pE\left(e^{\frac{1}{3}\sum_{t=1}^{T}u_{t1}^{2}}\right)e^{-T}\leq
p\Big{(}Ee^{\frac{1}{3}u_{11}^{2}}\Big{)}^{T}e^{-T}$
$\displaystyle=p\left(e/\sqrt{3}\right)^{-T}\leq pe^{-0.4T}.$

Since we require that $p\to\infty$ at a sub-exponential rate, this tends to
zero. Thus we conclude that (14) holds.

As for (15), we have

$\max_{1\leq j\leq p}\|\frac{1}{T}HF^{\top}U_{j}\|\leq\|H\|\max_{1\leq j\leq
p}\|\frac{1}{T}F^{\top}U_{j}\|.$

Using Lemma 7 and Lemma 13, the claim follows. For $B_{3j}$, we note that with
Assumption 2 it follows that

$\max_{1\leq j\leq p}\|b_{j}\|=\max_{1\leq j\leq
p}\Big{(}\sum_{\ell=1}^{L}\lambda_{\ell}\varphi_{\ell}^{2}(s_{j})\Big{)}^{1/2}=\max_{1\leq
j\leq p}\sqrt{EX^{2}_{t}(s_{j})}=O(1).$

The claim then follows directly from Lemma 7 and Lemma 11. This finishes the
proof. ∎

######  Proof of Lemma 6:.

We just have to note that $\|b_{j}\|^{2}=\mathrm{Var}(X_{1}(s_{j}))$. ∎

######  Proof of Lemma 7:.

It holds that

$\|H\|\leq\|(\hat{\Lambda}/p)^{-1}\|\|\hat{F}/\sqrt{T}\|\|F/\sqrt{T}\|\|B^{\top}B/p\|.$

Thus, the following easily verifiable assertions imply the proof:

$\displaystyle\|(\hat{\Lambda}/p)^{-1}\|$
$\displaystyle=p/\hat{\gamma}_{L};\qquad\|\hat{F}/\sqrt{T}\|\leq\|\hat{F}/\sqrt{T}\|_{F}=\sqrt{L};$
$\displaystyle\|F/\sqrt{T}\|^{2}$
$\displaystyle\leq\|F/\sqrt{T}\|_{F}^{2}=\left(\sum_{\ell=1}^{L}\frac{1}{T}\sum_{t=1}^{T}\frac{1}{\lambda_{\ell}}\langle
X_{t},\varphi_{\ell}\rangle^{2}\right)=\sum_{\ell=1}^{L}\frac{1}{\lambda_{\ell}}\bigg{\langle}\Big{(}\frac{1}{T}\sum_{t=1}^{T}X_{t}\otimes
X_{t}\Big{)}(\varphi_{\ell}),\varphi_{\ell}\bigg{\rangle}$
$\displaystyle\leq\frac{L}{\lambda_{L}}\Big{\|}\frac{1}{T}\sum_{t=1}^{T}X_{t}\otimes
X_{t}\Big{\|}=O_{P}(L/\lambda_{L});$ (26) $\displaystyle\|B^{\top}B/p\|^{2}$
$\displaystyle\leq\|B^{\top}B/p\|_{F}^{2}=\sum_{k=1}^{L}\sum_{\ell=1}^{L}\lambda_{k}\lambda_{\ell}\left(\frac{1}{p}\sum_{i=1}^{p}\varphi_{k}(s_{i})\varphi_{\ell}(s_{i})\right)^{2}=O(1).$
(27)

In (26) we used Lemma 9, while (27) follows from $\sum_{k\geq
1}\lambda_{k}<\infty$ and Assumption 3. ∎

######  Proof of Lemma 8:.

Let us first derive a bound for $\|HH^{\top}-I_{L}\|$. We have

$\displaystyle\|HH^{\top}-I_{L}\|$
$\displaystyle\leq\|HH^{\top}-\frac{1}{T}HF^{\top}FH^{\top}\|+\|\frac{1}{T}HF^{\top}FH^{\top}-I_{L}\|$
$\displaystyle\leq\|H\|^{2}\|I_{L}-\frac{1}{T}F^{\top}F\|_{F}+\|\frac{1}{T}HF^{\top}FH^{\top}-I_{L}\|.$
(28)

For the first term in (28) by Lemmas 7 and 11

$\|HH^{\top}-\frac{1}{T}HF^{\top}FH^{\top}\|=O_{P}\left(\frac{L}{\sqrt{T}}\left(\frac{L}{\lambda_{L}}\frac{p}{\hat{\gamma}_{L}}\right)^{2}\right)$

and the second term is equal to

$\displaystyle\|\frac{1}{T}HF^{\top}FH^{\top}-\frac{1}{T}\hat{F}^{\top}\hat{F}\|\leq\frac{1}{T}\|HF^{\top}-\hat{F}^{\top}\|_{F}\|H\|\|F\|_{F}+\frac{1}{T}\|\hat{F}\|_{F}\|\hat{F}-FH^{\top}\|_{F}.$

We have $\|H\|=O_{P}\Big{(}pL/\hat{\gamma}_{L}\sqrt{\lambda_{L}}\Big{)}$
(Lemma 7) and
$\|F\|_{F}=O_{P}\Big{(}\big{(}TL/\lambda_{L}\big{)}^{1/2}\Big{)}$ (see proof
of Lemma 7) and $\|\hat{F}\|_{F}=\sqrt{LT}$. From Lemma 12 we deduce that

$\frac{1}{T}\|HF^{\top}-\hat{F}^{\top}\|_{F}=O_{P}\left(\frac{1}{\sqrt{T}}\Big{(}L^{2}(\hat{\gamma}_{L}/p)^{-2}\big{(}1/T+1/p\big{)}\Big{)}^{1/2}\right).$

Combined, this shows that the second term in (28) is

$O_{P}\left(\left(\frac{p}{\hat{\gamma}_{L}}\right)^{2}\frac{L^{5/2}}{\lambda_{L}}\left(\frac{1}{\sqrt{T}}+\frac{1}{\sqrt{p}}\right)\right).$

Thus

$\|HH^{\top}-I_{L}\|=O_{P}\left(\left(\frac{p}{\hat{\gamma}_{L}}\right)^{2}\frac{L^{3}}{\lambda_{L}^{2}}\left(\frac{1}{\sqrt{T}}+\frac{1}{\sqrt{p}}\right)\right).$
(29)

Define $D_{L}:=HH^{\top}-I_{L}$ and let us denote by
$S=S_{T}=\\{\omega\in\Omega\colon\|D_{L}\|<1\\}$. If the bound in (29) tends
to zero, then $\Pr(S)\to 1$. On $S$ we have that
$W_{L}:=I_{L}-D_{L}+D_{L}^{2}-D_{L}^{3}\pm\ldots$ converges and it is easily
seen that $W_{L}$ is the inverse of $I_{L}+D_{L}=HH^{\top}$. But then $H$ is
invertible and $H^{-1}=H^{\top}W_{L}$.

Hence we get

$\Pr(S\cap\\{\|H^{-1}\|\leq 2\|H\|\\})\geq\Pr(S\cap\|W_{L}\|\leq
2)\geq\Pr(\|D_{L}\|\leq 1/2)\to 1.$ (30)

Then, using $H^{\top}H-I_{L}=H^{-1}D_{L}H$, we deduce that on $S$

$\|H^{\top}H-I_{L}\|\leq\|D_{L}\|\|H^{-1}\|\|H\|.$

So the statement of the lemma follows from (29), Lemma 7 and (30). ∎

##  Appendix B Weakening the assumption on Gaussian noise

As mentioned in Section 2.1, one may weaken the assumed Gaussianity of the
error process in Assumption 1. We discuss a rough outline of the consequences
of avoiding this specific restriction. In the following we instead assume 8
moments for $u_{t}$ and the validity of some moment inequalities which are
well known in the iid case and which have been verified under Assumption 1.
Additionally, we assume $p=O(T^{2})$ in order to guarantee the same
convergence rates in our theorems. Previously, subexponential growth of
$p=p(T)$ was possible. Below we discuss the necessary modifications.

  1. (I)

In Lemma 10 we prove (i) $E(u_{2}^{\top}u_{1})^{2k}\leq c_{2}p^{k}$ and (ii)
$E\|u_{1}\|^{4k}\leq c_{2}p^{2k}$, $k\in\\{1,2\\}$. In the case of iid noise
this follows immediately from Rosenthal’s inequality (see Petrov (1975)). To
apply this inequality we need for (i) $2k$ moments, and for (ii) $4k$ moments,
which ultimately means that we need 8 moments. Extensions to stationary
processes exist (see e.g. Liu et al. (2013)). In order to provide a more
general setting, we now state (i) and (ii) as assumptions.

  2. (II)

In Lemma 13 we need a bound for

$\sum_{j\leq p\atop i\leq
L}\Pr\left(\Big{|}\frac{1}{T}\sum_{t=1}^{T}\frac{1}{\sqrt{\lambda_{i}}}\langle
X_{t},\varphi_{i}\rangle u_{tj}\Big{|}>\kappa\cap\max_{k\leq
L}\frac{1}{T}\sum_{t=1}^{T}\frac{1}{\lambda_{k}}\langle
X_{t},\varphi_{k}\rangle^{2}\leq 2\right).$ (31)

We will use the following version of the Markov inequality:

######  Lemma 14.

Suppose that $(A_{t})$ and $(V_{t})$ are random sequences which are
independent of each other. Assume that $(V_{t})$ is iid zero mean and has a
finite fourth moment. Then

$p_{1}(\kappa)=\Pr\left(\Big{|}\frac{1}{T}\sum_{t=1}^{T}A_{t}V_{t}\Big{|}>\kappa\cap\frac{1}{T}\sum_{t=1}^{T}A_{t}^{2}\leq
2\right)\leq\frac{\mathrm{16EV_{1}^{4}}}{\kappa^{4}T^{2}}.$

###### Proof:.

We have

$\displaystyle\Pr\left(\Big{|}\frac{1}{T}\sum_{t=1}^{T}A_{t}V_{t}\Big{|}>\kappa\cap\frac{1}{T}\sum_{t=1}^{T}A_{t}^{2}\leq
2\right)\leq\frac{1}{T^{4}\kappa^{4}}E\left(\Big{|}\sum_{t=1}^{T}A_{t}V_{t}\Big{|}^{4}I\\{\frac{1}{T}\sum_{t=1}^{T}A_{t}^{2}\leq
2\\}\right)$
$\displaystyle\quad=\frac{1}{T^{4}\kappa^{4}}E\left(\sum_{r=1}^{T}\sum_{s=1}^{T}\sum_{t=1}^{T}\sum_{u=1}^{T}A_{r}A_{s}A_{t}A_{u}V_{r}V_{s}V_{t}V_{u}I\\{\frac{1}{T}\sum_{t=1}^{T}A_{t}^{2}\leq
2\\}\right)$
$\displaystyle\quad=\frac{1}{T^{4}\kappa^{4}}E\left(\sum_{r=1}^{T}\sum_{s=1}^{T}\sum_{t=1}^{T}\sum_{u=1}^{T}E\left[A_{r}A_{s}A_{t}A_{u}V_{r}V_{s}V_{t}V_{u}I\\{\frac{1}{T}\sum_{t=1}^{T}A_{t}^{2}\leq
2\\}\bigg{|}A_{1},\ldots,A_{T}\right]\right)$
$\displaystyle\quad=\frac{1}{T^{4}\kappa^{4}}E\left(\sum_{r=1}^{T}\sum_{s=1}^{T}\sum_{t=1}^{T}\sum_{u=1}^{T}E(V_{r}V_{s}V_{t}V_{u})E\left[A_{r}A_{s}A_{t}A_{u}I\\{\frac{1}{T}\sum_{t=1}^{T}A_{t}^{2}\leq
2\\}\bigg{|}A_{1},\ldots,A_{T}\right]\right)$ $\displaystyle\quad\leq
EV_{1}^{4}\frac{1}{T^{4}\kappa^{4}}E\left(\sum_{t=1}^{T}A_{t}^{4}I\\{\frac{1}{T}\sum_{t=1}^{T}A_{t}^{2}\leq
2\\}\right)+3(EV_{1}^{2})^{2}\frac{1}{T^{4}\kappa^{4}}E\left(\sum_{s=1}^{T}A_{s}^{2}\sum_{t=1}^{T}A_{t}^{2}I\\{\frac{1}{T}\sum_{t=1}^{T}A_{t}^{2}\leq
2\\}\right).$

Using $(EV_{1}^{2})^{2}\leq EV_{1}^{4}$ and
$\sum_{t=1}^{T}A_{t}^{4}\leq\left(\sum_{t=1}^{T}A_{t}^{2}\right)^{2}$, the
result is immediate. ∎

The lemma is now applied to (31) with $\kappa=c(Lp)^{1/4}T^{-1/2}$. Then for
any $\epsilon>0$ there is a $c$ independent of $T$, $p$ and $L$ such that
$p_{1}(\kappa)<\epsilon$. Using this in Lemma 13 implies that

$\max_{j\leq
p}\left\|\frac{1}{T}F^{\top}U_{j}\right\|=O_{P}\left(\frac{(Lp)^{1/4}}{T^{1/2}}\right).$

If again we assume 8 moments for $u_{t}$, the analogue approach yields

$\max_{j\leq
p}\left\|\frac{1}{T}F^{\top}U_{j}\right\|=O_{P}\left(\frac{(Lp)^{1/8}}{T^{1/2}}\right).$

  3. (III)

The bound in (II) is used in the proof of Lemma 5. In (15) we obtain the
modification

$\frac{\hat{\gamma}_{L}}{p}\max_{j=1}^{p}\|B_{2j}\|=O_{P}\left(\frac{L^{5/8}p^{1/8}}{\sqrt{\lambda_{L}}\sqrt{T}}\right).$
(32)

This implies that the bound for $R^{(2)}$ needs to be modified. In the proof
of Theorem 1 (where $L$ is fixed) this needs to be replaced by

$O_{P}\left(1/T^{1/4}+T^{1/4}/\sqrt{p}+p^{1/8}/\sqrt{T}\right).$

Since we require $p=O(T^{2})$ the rate remains unchanged. Moreover, the factor
$L$ in (32) is included with a smaller power than in (15), and hence this
modification also does not affect the rate in Theorem 2, where $L$ is allowed
to grow.

  4. (IV)

In the proof of Lemma 4 we need

$E\left(\sum_{j=1}^{p}u_{1j}\right)^{4}=O(p^{2}).$

In the iid case this can again be deduced by the Rosenthal’s inequality.
Likewise we may use some extension thereof in the dependent case.

  5. (V)

In the proof of Lemma 5 we bound

$\displaystyle\Pr\left(\max_{k=1,\ldots,p}\frac{1}{T}\sum_{t=1}^{T}u_{tk}^{2}>2Eu_{11}^{2}\right)$
$\displaystyle\leq
p\Pr\left(\sum_{t=1}^{T}(u_{t1}^{2}-Eu_{11}^{2})>Eu_{11}^{2}T\right)=O\left(\frac{p}{T^{2}}\right),$

by the Rosenthal inequality. Note that with respect to $t$ the $u_{t}$ are
assumed to be independent. For our proof this term is required to be $O(1)$,
which is the case due to the assumption of $p=O(T^{2})$.

## Acknowledgements

We would like to thank the editors and the anonymous reviewers for their
interesting questions and very constructive remarks which helped improve the
paper significantly.

## References

  * Aneiros et al. [2019] G. Aneiros, R. Cao, R. Fraiman, C. Genest, and P. Vieu.  Recent advances in functional data analysis and high-dimensional statistics.  _Journal of Multivariate Analysis_ , 170:3–9, 2019.  Special Issue on Functional Data Analysis and Related Topics.
  * Bai [2003] J. Bai.  Inferential theory for factor models of large dimensions.  _Econometrica_ , 71:135–171, 2003.
  * Bai and Li [2012] J. Bai and K. Li.  Statistical analysis of factor models of high dimension.  _The Annals of Statistics_ , 40:436–465, 2012.  doi: 10.1214/11-AOS966.
  * Bai and Li [2016] J. Bai and K. Li.  Maximum likelihood estimation and inference for approximate factor models of high dimension.  _The Review of Economics and Statistics_ , 98:298–309, 2016\.
  * Bai and Liao [2016] J. Bai and Y. Liao.  Efficient estimation of approximate factor models via regularized maximum likelihood.  _Journal of Econometrics_ , 191:1–18, 2016.
  * Bai and Ng [2002] J. Bai and S. Ng.  Determining the number of factors in approximate factor models.  _Econometrica_ , 70:191–221, 2002.
  * Bai and Ng [2019] J. Bai and S. Ng.  Rank regularized estimation of approximate factor models.  _Journal of Econometrics_ , 212(1):78 – 96, 2019\.
  * Bosq [2000] D. Bosq.  _Linear processes in function spaces: theory and applications_.  Lecture Notes in Statistics. Springer, New York, 2000.
  * Chamberlain and Rothschild [1983] G. Chamberlain and M. Rothschild.  Arbitrage, factor structure, and mean-variance analysis on large asset markets.  _Econometrica_ , 51:1281–1304, 1983.
  * Choi [2012] I. Choi.  Efficient estimation of factor models.  _Econometric Theory_ , 28:274–308, 2012.
  * Cuevas [2014] A. Cuevas.  A partial overview of the theory of statistics with functional data.  _Journal of Statistical Planning and Inference_ , 147:1–23, 2014.
  * Fan [1993] J. Fan.  Local linear regression smoothers and their minimax efficiencies.  _Ann. Statist._ , 21(1):196–216, 03 1993.
  * Fan et al. [2013] J. Fan, Y. Liao, and M. Mincheva.  Large covariance estimation by thresholding principal orthogonal complements.  _Journal of the Royal Statistical Society. Series B._ , 75, 09 2013\.
  * Ferraty and Vieu [2006] F. Ferraty and P. Vieu.  _Nonparametric Functional Data Analysis: Theory and Practice (Springer Series in Statistics)_.  Springer-Verlag, Berlin, Heidelberg, 2006.
  * Goia and Vieu [2016] A. Goia and P. Vieu.  An introduction to recent advances in high/infinite dimensional statistics.  _Journal of Multivariate Analysis_ , 146:1–6, 2016.  Special Issue on Statistical Models and Methods for High or Infinite Dimensional Spaces.
  * Hall and Horowitz [2007] P. Hall and J. Horowitz.  Methodology and convergence rates for functional linear regression.  _Ann. Statist._ , 35(1):70–91, 02 2007.
  * Hörmann and Cerovecki [2017] S. Hörmann and C. Cerovecki.  On the clt for discrete fourier transforms of functional time series.  _Journal of Multivariate Analysis_ , 154:282–295, 2017\.
  * Hörmann and Jammoul [2021] S. Hörmann and F. Jammoul.  Preprocessing noisy functional data: a multivariate perspective.  _https://arxiv.org/abs/2012.05824_ , 2021.
  * Hörmann and Kokoszka [2010] S. Hörmann and P. Kokoszka.  Weakly dependent functional data.  _The Annals of Statistics_ , 38:1845–1884, 2010.
  * Horváth and Kokoszka [2012] L. Horváth and P. Kokoszka.  _Inference for Functional Data with Applications_.  Springer, 2012.
  * Hsing and Eubank [2015] T. Hsing and R. Eubank.  _Theoretical Foundations of Functional Data Analysis, with an Introduction to Linear Operators_.  Wiley, 2015.
  * Johnstone [2001] I. M. Johnstone.  On the distribution of the largest eigenvalue in principal components analysis.  _The Annals of Statistics_ , 29:295–327, 2001.
  * Karoui [2005] N. Karoui.  Recent results about the largest eigenvalue of random covariance matrices and statistical application.  _Acta Phys. Polon. B_ , 36:2681–2697, 2005.
  * Liu et al. [2013] W. Liu, H. Xiao, and W.B. Wu.  Probability and moment inequalities under dependence.  _Statistica Sinica_ , 23:1257–1272, 2013.
  * Mardia et al. [1979] K.V. Mardia, J.T. Kent, and J.M Bibby.  _Multivariate Analysis_.  Academic Press, London, 1979.
  * Müller et al. [2006] H.-G. Müller, U. Stadtmüller, and F. Yao.  Functional variance processes.  _Journal of the American Statistical Association_ , 101:1007–1018, 2006.
  * Onatski [2010] A Onatski.  Determining the number of factors from empirical distribution of eigenvalues.  _The Review of Economics and Statistics_ , 92:1004–1016, 2010.
  * Owen and Wang [2016] A. Owen and J. Wang.  Bi-cross-validation for factor analysis.  _Statistical Science_ , 31:119–139, 2016.
  * Petrov [1975] V.V Petrov.  Sums of Independent Random Variables.  1975\.
  * Petrov [1995] V.V. Petrov.  _Limit Theorems of Probability Theory_.  Oxford Science Publications, New York, 1995.
  * Ramsay and Silverman [2005] J.O. Ramsay and B.W. Silverman.  _Functional Data Analysis_.  Springer, New York, 2005.
  * Staniswalis and Lee [1998] J. Staniswalis and J. Lee.  Nonparametric regression analysis of longitudinal data.  _Journal of the American Statistical Association_ , 93:1403–1418, 1998.
  * Wand and Jones [1995] M.P. Wand and M. C. Jones.  _Kernel Smoothing_.  Chapman & Hall, London, 1995.