Time-Varying Multivariate Causal Processes

∗Jiti Gao and ∗Bin Peng and †Wei Biao Wu and ∗Yayi Yan

∗Department of Econometrics and Business Statistics, Monash University,

and †Department of Statistics, University of Chicago

###### Abstract

In this paper, we consider a wide class of time-varying multivariate causal
processes which nests many classic and new examples as special cases. We first
prove the existence of a weakly dependent stationary approximation for our
model which is the foundation to initiate the theoretical development.
Afterwards, we consider the QMLE estimation approach, and provide both point-
wise and simultaneous inferences on the coefficient functions. In addition, we
demonstrate the theoretical findings through both simulated and real data
examples. In particular, we show the empirical relevance of our study using an
application to evaluate the conditional correlations between the stock markets
of China and U.S. We find that the interdependence between the two stock
markets is increasing over time.

Keywords: Local Linear Quasi-Maximum Likelihood Estimation; Multivariate
Causal Process; Simultaneous Confidence Interval.

JEL Classification: C14, C32, G15.

##  1 Introduction

The family of vector autoregressive (VAR) models and the family of
multivariate (G)ARCH models are among some of the most popular frameworks for
modelling dynamic interactions of multiple variables. The VAR family usually
captures the dynamic by imposing structures on the time series itself, while
the (G)ARCH family imposes restrictions on the conditional second moments. We
acknowledge the vast literature of both families, and have no intention to
exhaust all relevant studies in this paper for the sake of space. We refer
interested readers to Stock & Watson (2001) and Bauwens et al. (2006) for
excellent review on both families.

Although both families have rich literature on their own, to the best of the
authors’ knowledge not many works have been done to bridge them. Among limited
attempts (e.g., Ling & McAleer 2003, Bardet & Wintenberger 2009), most (if not
all) of these studies rely on the stationarity assumption. While the
stationarity assumption comes in handy when deriving asymptotic properties, it
may not be very realistic in practice (Preuss et al. 2015, Chen et al. 2021).
For example, economic and financial data always include different macro
shocks, as a consequence the behaviour can be quite volatile; the climate data
may contain certain time trend which recently has attracted lots of attention
due to greenhouse emission; etc. Anyway, certain nonstationarity may always
occur.

To account for nonstationarity, locally stationary processes have received
considerable attention since the seminal work of Dahlhaus (1996), Dette et al.
(2011), Zhang & Wu (2012), Truquet (2017), Dahlhaus et al. (2019), among
others. In contrast to the unit root process, the locally stationary process
nicely balances stationarity and nonstationarity by allowing for the
simultaneous presence of both types of behaviours in one time series process.
In a very recent paper, Karmakar et al. (2022) consider simultaneous inference
for a general class of univariate $p$-Markov processes with time-varying
coefficients, which covers several time-varying versions of the classical
univariate models (e.g., AR, ARCH, AR-ARCH) as special cases. Despite its
generality, their study still rules out the time-varying versions of some
widely used models (e.g., ARMA, GARCH, ARMA-GARCH). Also, it is worth
mentioning this line of research heavily focuses on univariate time series,
which somewhat limits the popularity of locally stationary processes.

That said, it is reasonable to call for a framework which can marry the VAR
family and the (G)ARCH family while allowing for nonstationarity. To provide a
concrete example, consider a time-varying multivariate GARCH model, which can
model the co-movements of financial returns. Detailed investigation on such a
model can help answer research questions like (i). Is the volatility of a
market leading the volatility of other markets? (ii) Whether the correlations
between asset returns change over time? (iii). Are they increasing in the long
run, perhaps because of the globalization of financial markets? These are of
great practical importance for both investors and policymakers (Bauwens et al.
2006, Diebold & Yilmaz 2009).

To allow for flexibility as much as possible from the modelling perspective,
we consider a class of multivariate causal processes as follows:

$\displaystyle\mathbf{x}_{t}=\left\\{\begin{array}[]{ll}\bm{\mu}\left(\mathbf{x}_{t-1},\mathbf{x}_{t-2},\ldots;\bm{\theta}(\tau_{t})\right)+\mathbf{H}\left(\mathbf{x}_{t-1},\mathbf{x}_{t-2},\ldots;\bm{\theta}(\tau_{t})\right)\bm{\varepsilon}_{t},&\text{for}\quad
t=1,\ldots,T\\\
\bm{\mu}\left(\mathbf{x}_{t-1},\mathbf{x}_{t-2},\ldots;\bm{\theta}(0)\right)+\mathbf{H}\left(\mathbf{x}_{t-1},\mathbf{x}_{t-2},\ldots;\bm{\theta}(0)\right)\bm{\varepsilon}_{t}&\text{for}\quad
t\leq 0\end{array}\right.,$ (1.3)

where $\tau_{t}=t/T$, $\bm{\mu}\left(\cdot\right)$ is an $m$-dimensional
random vector, $\mathbf{H}\left(\cdot\right)$ is an $m\times m$-dimensional
random matrix, $\bm{\theta}(\tau)$ is a $d\times 1$ time-varying parameter of
interest with each element belonging to $C^{3}[0,1]$, and
$\\{\bm{\varepsilon}_{t}\\}$ is a sequence of independent and identically
distributed (i.i.d.) random vectors. Note that the value of $d$ usually
depends on the value of $m$, and the connection becomes clear once a specific
model is considered. As far as we are concerned, both of $m$ and $d$ are fixed
throughout the paper. Notably, both $\bm{\mu}(\cdot)$ and $\mathbf{H}(\cdot)$
are known, and share the same unknown parameter $\bm{\theta}(\cdot)$. The
setting for $t\leq 0$ regulates the time series for the periods that we do not
observe, which is commonly adopted when certain nonstationarity gets involved
(e.g., Vogt 2012). Essentially, it requires the initial time period does not
have a diverging behaviour.

Before proceeding further, we provide two examples to briefly illustrate the
rationality behind (1.3), and leave the detailed investigation on these
examples to Section 2.4. We refer interested readers to Ling (2003), Ling &
McAleer (2003) and Bardet & Wintenberger (2009) for extensive investigation on
the parametric counterparts of these examples.

Example 1: Consider the time-varying VARMA($p,q$) model

$\mathbf{x}_{t}=\mathbf{a}(\tau_{t})+\sum_{j=1}^{p}\mathbf{A}_{j}(\tau_{t})\mathbf{x}_{t-j}+\bm{\eta}_{t}+\sum_{j=1}^{q}\mathbf{B}_{j}(\tau_{t})\bm{\eta}_{t-j}\quad\text{with}\quad\bm{\eta}_{t}=\bm{\omega}(\tau_{t})\bm{\varepsilon}_{t}.$
(1.4)

It is not hard to show that (1.4) admits a presentation in the form of (1.3),
and

$\displaystyle\bm{\theta}(\tau)=\mathrm{vec}(\mathbf{a}(\tau),\mathbf{A}_{1}(\tau),\ldots,\mathbf{A}_{p}(\tau),\mathbf{B}_{1}(\tau),\ldots,\mathbf{B}_{q}(\tau),\bm{\Omega}(\tau)),$
(1.5)

where $\bm{\Omega}(\cdot):=\bm{\omega}(\cdot)\bm{\omega}^{\top}(\cdot)$.

Example 2: Consider the time-varying multivariate GARCH($p,q$) model

$\displaystyle\mathbf{x}_{t}$ $\displaystyle=$
$\displaystyle\mathrm{diag}(h_{1,t}^{1/2},\ldots,h_{m,t}^{1/2})\bm{\eta}_{t},$
$\displaystyle\mathbf{h}_{t}$ $\displaystyle=$
$\displaystyle\mathbf{c}_{0}(\tau_{t})+\sum_{j=1}^{p}\mathbf{C}_{j}(\tau_{t})\left(\mathbf{x}_{t-j}\odot\mathbf{x}_{t-j}\right)+\sum_{j=1}^{q}\mathbf{D}_{j}(\tau_{t})\mathbf{h}_{t-j},$
(1.6)

where $h_{j,t}$ stands for the $j^{th}$ element of $\mathbf{h}_{t}$, and
$\bm{\eta}_{t}=\bm{\Omega}^{1/2}(\tau_{t})\bm{\varepsilon}_{t}$. The model (1)
generalizes the models of Bollerslev (1990) and Jeantheau (1998). Similar to
Example 1, we show that (1) admits a representation in the form of (1.3), and

$\displaystyle\bm{\theta}(\tau)=\mathrm{vec}(\mathbf{c}_{0}(\tau),\mathbf{C}_{1}(\tau),\ldots,\mathbf{C}_{p}(\tau),\mathbf{D}_{1}(\tau),\ldots,\mathbf{D}_{q}(\tau),\bm{\Omega}(\tau)).$
(1.7)

In view of the development of Example 1 and Example 2 in Section 2.4, one may
further show the time-varying counterparts of the parametric models mentioned
in Bardet & Wintenberger (2009) are also covered by (1.3). To this end, we
argue that (1.3) does not only allows for nonstationarity and conditional
heteroskedasticity, but also provides sufficient flexibility to cover many
well adopted models in the literature.

In this paper, our contributions are in the following four-fold: (1). we
consider a wide class of time-varying multivariate causal processes which
nests many classic and new examples as special cases; (2). we prove the
existence of a weakly dependent stationary approximation for the model (1.3)
at any given time of interest (i.e., $\forall\tau\in[0,1]$), which is the
foundation in order to establish asymptotic properties associated with the
model; (3). we establish the estimation theory, and provide both point-wise
and simultaneous inferences on the coefficient functions of which both are
important for practical works (Zhou & Wu 2010); (4). we demonstrate the
theoretical findings through both simulated and real data examples.

The paper is organized as follows. Section 2 presents the theoretical findings
associated with the stationary approximation, estimation and inferences. In
Section 3, we conduct extensive simulation studies to examine the theoretical
findings, and further investigate the time-varying conditional correlations
between the Chinese and U.S. Stock market. Section 4 concludes. Due to space
limit, we give the proofs of the main results to the online appendices of the
paper.

Before proceeding further, it is convenient to introduce some notation: the
symbol $|\cdot|$ denotes the Euclidean norm of a vector or the spectral norm
for a matrix; $\|\mathbf{v}\|_{q}:=\left(E|\mathbf{v}|^{q}\right)^{1/q}$ and
$\|\cdot\|:=\|\cdot\|_{2}$ for short; $\otimes$ denotes the Kronecker product;
$\odot$ denotes the Hadamard product; $\mathbf{I}_{a}$ stands for an $a\times
a$ identity matrix; $\mathbf{0}_{a\times b}$ stands for an $a\times b$ matrix
of zeros, and we write $\mathbf{0}_{a}$ for short when $a=b$; for a function
$g(w)$, let $g^{(j)}(w)$ be the $j^{th}$ derivative of $g(w)$, where $j\geq 0$
and $g^{(0)}(w)\equiv g(w)$; $K_{h}(\cdot)=K(\cdot/h)/h$, where $K(\cdot)$ and
$h$ stand for a nonparametric kernel function and a bandwidth respectively;
let $\tilde{c}_{k}=\int_{-1}^{1}u^{k}K(u)\mathrm{d}u$ and
$\tilde{v}_{k}=\int_{-1}^{1}u^{k}K^{2}(u)\mathrm{d}u$ for integer $k\geq 0$;
$\mathrm{diag}(\mathbf{a})$ is a diagonal matrix with the vector $\mathbf{a}$
on its main diagonal, while $\mathrm{diag}(\mathbf{A})$ creates a vector from
the diagonal of matrix $\mathbf{A}$; finally, let $\to_{P}$ and $\to_{D}$
denote convergence in probability and convergence in distribution,
respectively.

##  2 Estimation and Asymptotics

In this section, we first prove the existence of a weakly dependent stationary
approximation for the model (1.3) in Section 2.1; we then provide the
estimation approach using the local linear quasi-maximum-likelihood estimation
and establish the asymptotic properties of the proposed estimator in Section
2.2; Section 2.3 provides results on both point-wise and simultaneous
inferences; Section 2.4 gives some detailed examples to justify the usefulness
of our study.

###  2.1 Stationary Approximation

To study (1.3), the first challenge lies in the fact that the model may not be
stationary. Therefore, for $\forall\tau\in[0,1]$, we initial our analysis by
finding a stationary approximation for each $\mathbf{x}_{t}$ with $t\geq 1$.
By doing so, we are able to measure the weak dependence of
$\\{\mathbf{x}_{t}\\}$ using the nonlinear system theory in Wu (2005), which
then provides us a framework to derive the asymptotic properties accordingly.

To be clear on the dependence measure, consider an example in which
$\mathbf{e}_{t}$ is a stationary process, and admits a causal representation
$\mathbf{e}_{t}=\mathbf{J}(\bm{\varepsilon}_{t},\bm{\varepsilon}_{t-1},\ldots)$
with $\mathbf{J}(\cdot)$ being a measurable function. See Tong (1990) for
discussion on nonlinear time series of this kind. For $k\geq 0$, we define the
following dependence measure:

$\displaystyle\delta_{r}^{\mathbf{e}}(k)=\left\|\mathbf{J}(\bm{\varepsilon}_{k},\bm{\varepsilon}_{k-1},\ldots\bm{\varepsilon}_{1},\bm{\varepsilon}_{0},\bm{\varepsilon}_{-1},\ldots)-\mathbf{J}(\bm{\varepsilon}_{k},\ldots,\bm{\varepsilon}_{1},\bm{\varepsilon}_{0}^{*},\bm{\varepsilon}_{-1},\ldots)\right\|_{r},$
(2.1)

where $\bm{\varepsilon}_{0}^{*}$ is an independent copy of
$\\{\bm{\varepsilon}_{j}\\}$. Being able to measure the time series dependence
such as (2.1) is the starting point for time series analyses.

We now introduce some basic assumptions.

######  Assumption 1.

  1. 1.

$\\{\bm{\varepsilon}_{t}\\}$ is a sequence of i.i.d. random vectors with
$E(\bm{\varepsilon}_{1})=\mathbf{0}$,
$E(\bm{\varepsilon}_{1}\bm{\varepsilon}_{1}^{\top})=\mathbf{I}_{m}$, and
$\|\bm{\varepsilon}_{1}\|_{r}<\infty$ for some $r\geq 2$.

  2. 2.

For $\forall\mathbf{z},\mathbf{z}^{\prime}\in(\mathbb{R}^{m})^{\infty}$ and
$\forall\bm{\vartheta}\in\mathbb{R}^{d}$, there exist nonnegative sequences
$\\{\alpha_{j}(\bm{\bm{\vartheta}})\\}_{j=1}^{\infty}$ and
$\\{\beta_{j}(\bm{\bm{\vartheta}})\\}_{j=1}^{\infty}$ such that

$\displaystyle|\bm{\mu}(\mathbf{z};\bm{\bm{\vartheta}})-\bm{\mu}(\mathbf{z}^{\prime};\bm{\bm{\vartheta}})|\leq\sum_{j=1}^{\infty}\alpha_{j}(\bm{\bm{\vartheta}})|\mathbf{z}_{j}-\mathbf{z}_{j}^{\prime}|,$
$\displaystyle|\mathbf{H}(\mathbf{z};\bm{\bm{\vartheta}})-\mathbf{H}(\mathbf{z}^{\prime};\bm{\bm{\vartheta}})|\leq\sum_{j=1}^{\infty}\beta_{j}(\bm{\bm{\vartheta}})|\mathbf{z}_{j}-\mathbf{z}_{j}^{\prime}|,$

where $\mathbf{z}_{j}$ and $\mathbf{z}_{j}^{\prime}$ are the $j^{th}$ columns
of $\mathbf{z}$ and $\mathbf{z}^{\prime}$ respectively.

  3. 3.

For $\forall\tau\in[0,1]$, $\bm{\theta}(\tau)$ lies in the interior of
$\bm{\Theta}_{r}$, where

$\displaystyle\bm{\Theta}_{r}:=\left\\{\bm{\vartheta}\in\bm{\Theta}\mid\sum_{j=1}^{\infty}\alpha_{j}(\bm{\vartheta})+\left\|\bm{\varepsilon}_{1}\right\|_{r}\sum_{j=1}^{\infty}\beta_{j}(\bm{\vartheta})<1\right\\}$

and $\bm{\Theta}$ is a compact set of $\mathbb{R}^{d}$.

Assumption 1.1 is standard when studying dynamic time series model (Lütkepohl
2005). In Assumption 1.2, $\bm{\vartheta}$ is a generic $d\times 1$ vector,
and has the same length as $\bm{\theta}(\cdot)$. This assumption imposes
Lipschitz-type conditions on $\bm{\mu}(\cdot)$ and $\mathbf{H}(\cdot)$, which
are rather minor, and can be easily fulfilled by a variety of models such as
those mentioned in Section 1. See Propositions 2.3-2.4 below for details.
Assumption 1.3 does not only guarantee a stationary approximation for each
$\mathbf{x}_{t}$, but also ensures the approximated process has some proper
moments. Similar conditions have also been adopted in Bardet & Wintenberger
(2009).

With these conditions in hand, we present the following proposition which
facilitates the development in what follows.

######  Proposition 2.1.

Let Assumption 1 hold. For any $\tau\in[0,1]$, there exists a stationary
process

$\displaystyle\widetilde{\mathbf{x}}_{t}(\tau)=\bm{\mu}\left(\widetilde{\mathbf{x}}_{t-1}(\tau),\widetilde{\mathbf{x}}_{t-2}(\tau),\ldots;\bm{\theta}(\tau)\right)+\mathbf{H}\left(\widetilde{\mathbf{x}}_{t-1}(\tau),\widetilde{\mathbf{x}}_{t-2}(\tau),\ldots;\bm{\theta}(\tau)\right)\bm{\varepsilon}_{t}$

such that

  1. 1.

$\sup_{\tau\in[0,1]}\left\|\widetilde{\mathbf{x}}_{t}(\tau)\right\|_{r}<\infty$,

  2. 2.

$\delta_{r}^{\widetilde{\mathbf{x}}(\tau)}(k)\leq O(1)\inf_{1\leq p\leq
k}\\{\rho(\tau)^{k/p}+\sum_{j=p+1}^{\infty}\left[\alpha_{j}(\bm{\theta}(\tau))+\beta_{j}(\bm{\theta}(\tau))\right]\\}\to
0$ as $k\to\infty$,

where
$\rho(\tau):=\sum_{j=1}^{\infty}\alpha_{j}(\bm{\theta}(\tau))+\left\|\bm{\varepsilon}_{1}\right\|_{r}\sum_{j=1}^{\infty}\beta_{j}(\bm{\theta}(\tau))$.

It is worth mentioning that for a univariate $p$-Markov process

$\displaystyle\widetilde{x}_{p,t}(\tau)=\mu\left(\widetilde{x}_{t-1}(\tau),\ldots,\widetilde{x}_{t-p}(\tau);\bm{\theta}(\tau)\right)+H\left(\widetilde{x}_{t-1}(\tau),\ldots,\widetilde{x}_{t-p}(\tau);\bm{\theta}(\tau)\right)\varepsilon_{t},$

Karmakar et al. (2022) show that there exists $0<\rho<1$ such that
$\sup_{\tau\in[0,1]}\delta_{r}^{\widetilde{x}_{p}(\tau)}(k)=O(\rho^{k})$ based
on the development of Wu & Shao (2004). From a methodological viewpoint, we
give a set of new proofs which allow us to measure the dependence of
multivariate causal processes with infinity memory. The term
$\sum_{j=p+1}^{\infty}\left[\alpha_{j}(\bm{\theta}(\tau))+\beta_{j}(\bm{\theta}(\tau))\right]$
in the second result of Proposition 2.1 arises due to the infinity memory
structure of $\widetilde{\mathbf{x}}_{t}(\tau)$. Thus, the dependence
$\delta_{r}^{\widetilde{\mathbf{x}}(\tau)}(k)$ relies on the choice of $p$ and
the decay rates of the coefficients $\alpha_{j}(\bm{\theta}(\tau))$ and
$\beta_{j}(\bm{\theta}(\tau))$.

To ensure $\widetilde{\mathbf{x}}_{t}(\tau)$ can approximate $\mathbf{x}_{t}$
reasonably well, we impose more structure below.

######  Assumption 2.

  1. 1.

There exists a nonnegative sequence $\\{\chi_{j}\\}$ with
$\sum_{j=1}^{\infty}\chi_{j}<\infty$ such that for
$\forall\mathbf{z}\in(\mathbb{R}^{m})^{\infty}$ and
$\forall\bm{\vartheta},\bm{\vartheta}^{\prime}\in\bm{\Theta}_{r}$

$\displaystyle|\bm{\mu}(\mathbf{z};\bm{\vartheta})-\bm{\mu}(\mathbf{z};\bm{\vartheta}^{\prime})|+|\mathbf{H}(\mathbf{z};\bm{\vartheta})-\mathbf{H}(\mathbf{z};\bm{\vartheta}^{\prime})|\leq|\bm{\vartheta}-\bm{\vartheta}^{\prime}|\sum_{j=1}^{\infty}\chi_{j}|\mathbf{z}_{j}|.$

  2. 2.

Let $\sup_{\tau\in[0,1]}\alpha_{j}(\bm{\theta}(\tau))=O(j^{-(2+s)})$ and
$\sup_{\tau\in[0,1]}\beta_{j}(\bm{\theta}(\tau))=O(j^{-(2+s)})$ for some
$s>0$.

Assumption 2.1 imposes another Lipschitz-type condition with respect to the
parameter space. Assumption 2.2 further restricts the decay rates of
$\alpha_{j}(\bm{\theta}(\tau))$ and $\beta_{j}(\bm{\theta}(\tau))$.

Using Assumptions 1–2, we can measure the distance between
$\widetilde{\mathbf{x}}_{t}(\tau)$ and $\mathbf{x}_{t}$ as follows.

######  Proposition 2.2.

Suppose Assumptions 1–2 hold. Then

  1. 1.

$\left\|\widetilde{\mathbf{x}}_{1}(\tau)-\widetilde{\mathbf{x}}_{1}(\tau^{\prime})\right\|_{r}=O(|\tau-\tau^{\prime}|)$
for $\forall\tau,\tau^{\prime}\in[0,1]$,

  2. 2.

$\max_{t\geq
1}\left\|\mathbf{x}_{t}-\widetilde{\mathbf{x}}_{t}(\tau_{t})\right\|_{r}=O(T^{-1})$.

We can consider Proposition 2.2 as the stochastic version of the Hölder
continuity. Having established the stationary approximation in Proposition
2.2, we move on to investigate the estimation theory in the next subsection.

###  2.2 Estimation

We point out a few facts to facilitate the setup of the likelihood function.
First, let $\mathbf{z}_{t}=(\mathbf{x}_{t},\mathbf{x}_{t-1},\ldots)$ include
all the information of $\mathbf{x}_{t}$ up to the time period $t$. However, in
practice, our observation on $\mathbf{x}_{t}$ only starting from $t=1$, so we
have to work with the truncated version of $\mathbf{z}_{t}$ for each $t\geq
1$:

$\displaystyle\mathbf{z}_{t}^{c}=(\mathbf{x}_{t},\ldots,\mathbf{x}_{1},\mathbf{0},\ldots).$
(2.2)

Second, we note that when $\tau_{t}$ is sufficiently close to $\tau$,

$\displaystyle\bm{\theta}(\tau_{t})\approx\bm{\theta}(\tau)+h\bm{\theta}^{(1)}(\tau)\cdot\frac{\tau_{t}-\tau}{h}.$
(2.3)

Therefore, we are able to parametrize $\bm{\theta}(\cdot)$, and consider the
maximum-likelihood estimation for each given $\tau$. Finally, since
$\bm{\varepsilon}_{t}$ may not be normally distributed, we consider the local
linear quasi-maximum-likelihood estimation (QMLE) method.

Thus, our likelihood function is specified as follows:

$\mathcal{L}_{\tau}(\bm{\eta}_{1},\bm{\eta}_{2})=\frac{1}{T}\sum_{t=1}^{T}\mathcal{l}(\mathbf{x}_{t},\mathbf{z}_{t-1}^{c};\bm{\eta}_{1}+\bm{\eta}_{2}\cdot(\tau_{t}-\tau)/h)K_{h}(\tau_{t}-\tau),$
(2.4)

where

$\displaystyle\mathcal{l}(\mathbf{x}_{t},\mathbf{z}_{t-1}^{c};\bm{\vartheta})$
$\displaystyle=$
$\displaystyle-\frac{1}{2}(\mathbf{x}_{t}-\bm{\mu}(\mathbf{z}_{t-1}^{c};\bm{\vartheta}))^{\top}\left(\mathbf{H}(\mathbf{z}_{t-1}^{c};\bm{\vartheta})\mathbf{H}(\mathbf{z}_{t-1}^{c};\bm{\vartheta})^{\top}\right)^{-1}(\mathbf{x}_{t}-\bm{\mu}(\mathbf{z}_{t-1}^{c};\bm{\vartheta}))$
$\displaystyle-\frac{1}{2}\log\det\left(\mathbf{H}(\mathbf{z}_{t-1}^{c};\bm{\vartheta})\mathbf{H}(\mathbf{z}_{t-1}^{c};\bm{\vartheta})^{\top}\right).$

Accordingly, for $\forall\tau$, $(\bm{\theta}(\tau),h\bm{\theta}^{(1)}(\tau))$
is estimated by

$(\widehat{\bm{\theta}}(\tau),\widehat{\bm{\theta}}^{\star}(\tau))=\operatorname*{\arg\\!\max}_{(\bm{\eta}_{1},\bm{\eta}_{2})\in\mathbf{E}_{T}(r)}\mathcal{L}_{\tau}(\bm{\eta}_{1},\bm{\eta}_{2}),$
(2.5)

where $\mathbf{E}_{T}(r)=\bm{\Theta}_{r}\times(h\cdot\bm{\Theta}^{(1)})$ and
$\bm{\Theta}^{(1)}$ is a compact set.

We impose more structures in order to derive the asymptotic distribution.

######  Assumption 3.

  1. 1.

$\inf_{\bm{\vartheta}\in\bm{\Theta}_{r},\mathbf{z}\in(\mathbb{R}^{m})^{\infty}}\lambda_{\min}\left(\mathbf{H}(\mathbf{z};\bm{\vartheta})\mathbf{H}(\mathbf{z};\bm{\vartheta})^{\top}\right)\geq\underline{c}$
for some $\underline{c}>0$.

  2. 2.

For any $\bm{\vartheta}\in\bm{\Theta}_{r}$,
$\bm{\mu}(\widetilde{\mathbf{z}}_{t}(\tau);\bm{\theta}(\tau))=\bm{\mu}(\widetilde{\mathbf{z}}_{t}(\tau);\bm{\vartheta})$
and
$\mathbf{H}(\widetilde{\mathbf{z}}_{t}(\tau);\bm{\theta}(\tau))=\mathbf{H}(\widetilde{\mathbf{z}}_{t}(\tau);\bm{\vartheta})$
a.s. imply $\bm{\vartheta}=\bm{\theta}(\tau)$ for some $t$, where
$\widetilde{\mathbf{z}}_{t}(\tau)=\left(\widetilde{\mathbf{x}}_{t}(\tau),\widetilde{\mathbf{x}}_{t-1}(\tau),\ldots\right)$.

######  Assumption 4.

  1. 1.

$\bm{\mu}(\cdot;\bm{\vartheta})$ and $\mathbf{H}(\cdot;\bm{\vartheta})$ are
twice continuously differentiable with respect to $\bm{\vartheta}$.

  2. 2.

There exists a nonnegative sequence $\\{\chi_{j}\\}_{j=1}^{\infty}$ with
$\chi_{j}=O(j^{-(2+s)})$ and some $s>0$ such that for any
$\mathbf{z},\mathbf{z}^{\prime}\in(\mathbb{R}^{m})^{\infty}$ and any
$\bm{\vartheta},\bm{\vartheta}^{\prime}\in\bm{\Theta}_{r}$:

$\displaystyle|\gradient_{\bm{\vartheta}}^{k}\bm{\mu}(\mathbf{z};\bm{\vartheta})-\gradient_{\bm{\vartheta}}^{k}\bm{\mu}(\mathbf{z};\bm{\vartheta}^{\prime})|+|\gradient_{\bm{\vartheta}}^{k}\mathbf{H}(\mathbf{z};\bm{\vartheta})-\gradient_{\bm{\vartheta}}^{k}\mathbf{H}(\mathbf{z};\bm{\vartheta}^{\prime})|\leq|\bm{\vartheta}-\bm{\vartheta}^{\prime}|\sum_{j=1}^{\infty}\chi_{j}|\mathbf{z}_{j}|,$
$\displaystyle|\gradient_{\bm{\vartheta}}^{k}\bm{\mu}(\mathbf{z};\bm{\vartheta})-\gradient_{\bm{\vartheta}}^{k}\bm{\mu}(\mathbf{z}^{\prime};\bm{\vartheta})|+|\gradient_{\bm{\vartheta}}^{k}\mathbf{H}(\mathbf{z};\bm{\vartheta})-\gradient_{\bm{\vartheta}}^{k}\mathbf{H}(\mathbf{z}^{\prime};\bm{\vartheta})|\leq\sum_{j=1}^{\infty}\chi_{j}|\mathbf{z}_{j}-\mathbf{z}_{j}^{\prime}|,$

where
$\gradient_{\bm{\vartheta}}=\left(\frac{\partial}{\partial\vartheta_{1}},\ldots,\frac{\partial}{\partial\vartheta_{d}}\right)^{\top}$,
and $k=1,2$.

######  Assumption 5.

Let $K(\cdot)$ be a symmetric and positive kernel function defined on $[-1,1]$
with $\int_{-1}^{1}K(u)\mathrm{d}u=1$. Moreover, $K(\cdot)$ is Lipschitz
continuous on $[-1,1]$. As $(T,h)\to(\infty,0)$, $Th\to\infty$.

Assumption 3.1 ensures the positive definiteness of the covariance matrix of
the likelihood function, and is widely adopted when studying the multivariate
time series (e.g., page 2736 of Bardet & Wintenberger 2009). In fact, the
validity of this assumption is easy to justify in view of (2.4) and (2.16) for
Example 1 and Example 2 below. Assumption 3.2 imposes an standard
identification condition in the literature of M-estimation (e.g., Proposition
3.4 of Jeantheau 1998). It is noteworthy that the current form of Assumption 3
accommodates the flexibility of the model (1.3), which is in fact unnecessary
if we have a detailed model in practice. See Section 2.4 for example.

Assumption 4 imposes the Lipschitz-type conditions on the first and second
order derivatives of $\bm{\mu}(\cdot)$ and $\mathbf{H}(\cdot)$ to ensure the
smoothness of their functional components.

Assumption 5 is a set of regular conditions on the kernel function and the
bandwidth.

With these conditions in hand, we summarize the first theorem of this paper
below.

######  Theorem 2.1.

Suppose Assumptions 1–5 hold with $r\geq 6$.

(1). If $Th^{7}\to 0$, then for any $\tau\in(0,1)$

$\displaystyle\sqrt{Th}\left(\widehat{\bm{\theta}}(\tau)-\bm{\theta}(\tau)-\frac{1}{2}h^{2}\widetilde{c}_{2}\bm{\theta}^{(2)}(\tau)\right)\to_{D}N\left(\mathbf{0},\widetilde{v}_{0}\bm{\Sigma}_{\bm{\theta}}(\tau)\right),$

where
$\bm{\Sigma}_{\bm{\theta}}(\tau)=\bm{\Sigma}^{-1}(\tau)\bm{\Omega}(\tau)\bm{\Sigma}^{-1}(\tau)$,
$\bm{\Sigma}(\tau)=E\left(\gradient_{\bm{\vartheta}}^{2}\mathcal{l}(\widetilde{\mathbf{x}}_{1}(\tau),\widetilde{\mathbf{z}}_{0}(\tau);\bm{\theta}(\tau))\right)$
and

$\displaystyle\bm{\Omega}(\tau)=E\left(\gradient_{\bm{\vartheta}}\mathcal{l}(\widetilde{\mathbf{x}}_{1}(\tau),\widetilde{\mathbf{z}}_{0}(\tau);\bm{\theta}(\tau))\cdot\gradient_{\bm{\vartheta}}\mathcal{l}(\widetilde{\mathbf{x}}_{1}(\tau),\widetilde{\mathbf{z}}_{0}(\tau);\bm{\theta}(\tau))^{\top}\right).$

(2). In addition, if $\bm{\varepsilon}_{t}$ is normally distributed, we have
$\bm{\Omega}(\tau)=-\bm{\Sigma}(\tau)$ and thus
$\bm{\Sigma}_{\bm{\theta}}(\tau)=\bm{\Omega}^{-1}(\tau)$.

After deriving the asymptotic distribution, we will establish both the point-
wise inference and the simultaneous inference in the following.

###  2.3 Inference

In this section, we first discuss how to conduct point-wise inference, and
then move on to derive the asymptotic results associated with the simultaneous
inference. Specifically, for some preassigned significance level
$\alpha\in(0,1)$, we shall construct a $100(1-\alpha)\%$ asymptotic
simultaneous confidence band (SCB) $\\{\Upsilon(\tau),0\leq\tau\leq 1\\}$ for
$\bm{\theta}(\cdot)$ in the sense that

$\lim_{T\to\infty}\Pr\left(\bm{\theta}(\tau)\in\Upsilon(\tau),0\leq\tau\leq
1\right)=1-\alpha.$

Notably, the simultaneous inference nests the traditional constancy test as a
special case. It does not only allow one to examine whether a time-varying
model should be preferred to its parametric counterpart, but also allows one
to test any particular functional form of interest. For example, if a
horizontal line can be embedded in the SCB $\\{\Upsilon(\tau)\\}$, then we
accept the hypothesis that some elements of $\bm{\theta}(\tau)$ are constant.

Point-wise Inference: First, we construct a bias-corrected estimator in order
to remove the asymptotic bias of Theorem 2.1. Specifically, we let

$\widetilde{\bm{\theta}}(\tau)=2\widehat{\bm{\theta}}_{h/\sqrt{2}}(\tau)-\widehat{\bm{\theta}}(\tau),$
(2.6)

where $\widehat{\bm{\theta}}_{h/\sqrt{2}}(\tau)$ is defined in the same way as
$\widehat{\bm{\theta}}(\tau)$ but using the bandwidth $h/\sqrt{2}$.

After tedious development (Lemma B.7 of Appendix B), we have uniformly over
$\tau\in[h,1-h]$

$\displaystyle\widetilde{\bm{\theta}}(\tau)-\bm{\theta}(\tau)$
$\displaystyle=$
$\displaystyle-\bm{\Sigma}^{-1}(\tau)\frac{1}{Th}\sum_{t=1}^{T}\widetilde{K}((\tau_{t}-\tau)/h)\gradient_{\bm{\vartheta}}\mathcal{l}(\widetilde{\bm{x}}_{t}(\tau_{t}),\widetilde{\bm{z}}_{t-1}(\tau_{t});\bm{\theta}(\tau_{t}))$
$\displaystyle+O_{P}((Th)^{-1/2}h^{3/2}(\log T)^{1/2})+o(h^{3}),$

where $\widetilde{K}(x)=2\sqrt{2}K(\sqrt{2}x)-K(x)$ that is essentially a
fourth-order kernel. It then infers that under the conditions of Theorem 2.1,

$\displaystyle\sqrt{Th}(\widetilde{\bm{\theta}}(\tau)-\bm{\theta}(\tau))\to_{D}N\left(\mathbf{0},{v}_{0}\bm{\Sigma}_{\bm{\theta}}(\tau)\right),$
(2.7)

where $v_{0}=\int_{-1}^{1}\widetilde{K}^{2}(u)\mathrm{d}u$.

It is noteworthy that the construction of (2.6) is different from directly
using the fourth-order kernel in the regression. In terms of bandwidth
selection, the traditional methods (e.g., cross-validation) still remain valid
for (2.6) (Richter et al. 2019). However, if one directly employs the fourth-
order kernel in the regression, it remains unclear how to select the optimal
bandwidth in practice.

Now we discuss how to estimate $\bm{\Sigma}_{\bm{\theta}}(\tau)$ which is
constructed by $\bm{\Sigma}(\tau)$ and $\bm{\Omega}(\tau)$. Intuitively, we
consider the following estimator

$\widehat{\bm{\Sigma}}_{\bm{\theta}}(\tau)=\widehat{\bm{\Sigma}}^{-1}(\tau)\widehat{\bm{\Omega}}(\tau)\widehat{\bm{\Sigma}}^{-1}(\tau),$
(2.8)

where

$\displaystyle\widehat{\bm{\Sigma}}(\tau)$ $\displaystyle=$ $\displaystyle
A_{T}(\tau)^{-1}\sum_{t=1}^{T}\gradient_{\bm{\vartheta}}^{2}\mathcal{l}(\mathbf{x}_{t},\mathbf{z}_{t-1}^{c};\widehat{\bm{\theta}}(\tau))K_{h}(\tau_{t}-\tau),$
$\displaystyle\widehat{\bm{\Omega}}(\tau)$ $\displaystyle=$ $\displaystyle
A_{T}(\tau)^{-1}\sum_{t=1}^{T}\gradient_{\bm{\vartheta}}\mathcal{l}(\mathbf{x}_{t},\mathbf{z}_{t-1}^{c};\widehat{\bm{\theta}}(\tau))\cdot\gradient_{\bm{\vartheta}}\mathcal{l}(\mathbf{x}_{t},\mathbf{z}_{t-1}^{c};\widehat{\bm{\theta}}(\tau))^{\top}K_{h}(\tau_{t}-\tau),$
$\displaystyle A_{T}(\tau)$ $\displaystyle=$
$\displaystyle\sum_{t=1}^{T}K_{h}(\tau_{t}-\tau).$

Note that we consider a local constant estimator in (2.8) rather than a local
linear one, that is to avoid an implementation issue for finite sample studies
(i.e., nonpositive definite covariance may occur when the local linear
approach is employed). Such a numerical problem has been well explained and
investigated in the literature. See Chen & Leng (2015) for example.

The following corollary summarizes the asymptotic property of (2.8).

######  Corollary 2.1.

Under the conditions of Theorem 2.1.1, suppose further that

$\displaystyle\sup_{\tau\in[0,1]}[\alpha_{j}(\bm{\theta}(\tau))+\beta_{j}(\bm{\theta}(\tau))]=O(j^{-(5/2+s)})$

for some $s>0$. In addition, let $h(\log T)^{2}\to 0$ and
$T^{1-6/r}h\to\infty$. Then

$\displaystyle\sup_{\tau\in[0,1]}|\widehat{\bm{\Sigma}}_{\bm{\theta}}(\tau)-\bm{\Sigma}_{\bm{\theta}}(\tau)|=o_{P}(1).$

Simultaneous Inference: We now consider the simultaneous inference. To allow
for flexibility, we first introduce a selection matrix $\mathbf{C}$ with full
row rank, which selects the parameters of interest as follows:

$\displaystyle\bm{\theta}_{\mathbf{C}}(\tau):=\mathbf{C}\bm{\theta}(\tau).$
(2.9)

Accordingly, the estimator and the corresponding asymptotic covariance matrix
become

$\displaystyle\widehat{\bm{\theta}}_{\mathbf{C}}(\tau):=\mathbf{C}\widehat{\bm{\theta}}(\tau)\quad\text{and}\quad\bm{\Sigma}_{\mathbf{C}}(\tau)=\mathbf{C}\bm{\Sigma}_{\bm{\theta}}(\tau)\mathbf{C}^{\top}.$
(2.10)

######  Theorem 2.2.

Under the conditions of Theorem 2.1.1, suppose further that

$\displaystyle\sup_{\tau\in[0,1]}[\alpha_{j}(\bm{\theta}(\tau))+\beta_{j}(\bm{\theta}(\tau))]=O(j^{-(3+s)})$

for some $s>0$. In addition, let $(\log T)^{4}/(T^{\nu}h)\to 0$ with
$\nu=\frac{1}{2}-\frac{r-6}{4rs/3+2r-4}$ and $Th^{7}\log T\to 0$. Then

$\displaystyle\lim_{T\to\infty}\mathrm{Pr}$
$\displaystyle\left(\sqrt{\frac{Th}{\widetilde{v}_{0}}}\sup_{\tau\in[h,1-h]}\left|\bm{\Sigma}_{\mathbf{C}}^{-1/2}(\tau)\left\\{\widehat{\bm{\theta}}_{\mathbf{C}}(\tau)-\bm{\theta}_{\mathbf{C}}(\tau)-\frac{1}{2}h^{2}\widetilde{c}_{2}\bm{\theta}_{\mathbf{C}}^{(2)}(\tau)\right\\}\right|\right.$
$\displaystyle\left.-B(1/h)\leq\frac{u}{\sqrt{2\log(1/h)}}\right)=\exp(-2\exp(-u)),$

where

$\displaystyle B(1/h)$ $\displaystyle=$
$\displaystyle\sqrt{2\log(1/h)}+\frac{\log(C_{K})+(k/2-1/2)\log(\log(1/h))-\log(2)}{\sqrt{2\log(1/h)}},$
$\displaystyle C_{K}$ $\displaystyle=$
$\displaystyle\frac{\\{\int_{-1}^{1}|K^{(1)}(u)|^{2}\mathrm{d}u/\widetilde{v}_{0}\pi\\}^{1/2}}{\Gamma(k/2)},$

and $\Gamma(\cdot)$ is the Gamma function.

In Theorem 2.2, $\nu$ is slightly smaller than $1/2$ as we only require $r$ to
be slightly larger than $6$. Hence, the usual optimal bandwidth
$h_{opt}=O(T^{-1/5})$ satisfies the conditions $(\log T)^{4}/(T^{\nu}h)\to 0$
and $Th^{7}\log T\to 0$.

As shown in Theorem 2.2, the convergence rate of the simultaneous confidence
intervals for $\bm{\theta}_{\mathbf{C}}(\cdot)$ is of logarithmic rate and is
therefore slow. In order to improve the rate, we consider a bootstrap method
which shows a much better finite sample performance. We summarize the result
in the following corollary.

######  Corollary 2.2.

Under the conditions of Theorem 2.2. Suppose that $h=O(T^{-\kappa})$ with
$1/7<\kappa<\nu$. Then, on a richer probability space, there exists i.i.d.
$k$-dimensional standard normal variables
$\mathbf{v}_{1},\ldots,\mathbf{v}_{T}$ such that

$\sup_{\tau\in[0,1]}|\widehat{\bm{\theta}}_{\mathbf{C}}(\tau)-\bm{\theta}_{\mathbf{C}}(\tau)-\frac{1}{2}h^{2}b_{h}(\tau)\bm{\theta}_{\mathbf{C}}^{(2)}(\tau)-\bm{\Sigma}_{\mathbf{C}}^{1/2}(\tau)\mathbf{V}_{h}^{*}(\tau)|=O_{P}\left(\frac{T^{-\alpha}}{\sqrt{Th\log
T}}\right),$

where $\alpha=\min\\{(\nu-\kappa)/2,(7\kappa-1)/2,\kappa/2\\}$,
$\widetilde{c}_{k,h}(\tau)=\int_{-\tau/h}^{(1-\tau)/h}u^{k}K(u)\mathrm{d}u$,
$\mathbf{V}_{h}^{*}(\tau)=T^{-1}\sum_{t=1}^{T}\mathbf{v}_{t}\omega_{t,h}(\tau)$,

$b_{h}(\tau)=\frac{\widetilde{c}_{2,h}^{2}(\tau)-\widetilde{c}_{1,h}(\tau)\widetilde{c}_{3,h}(\tau)}{\widetilde{c}_{0,h}(\tau)\widetilde{c}_{2,h}(\tau)-\widetilde{c}_{1,h}^{2}(\tau)}\quad\text{and}\quad\omega_{t,h}(\tau)=K_{h}(\tau_{t}-\tau)\frac{\widetilde{c}_{2,h}(\tau)-\frac{\tau_{t}-\tau}{h}\widetilde{c}_{1,h}(\tau)}{\widetilde{c}_{0,h}(\tau)\widetilde{c}_{2,h}(\tau)-\widetilde{c}_{1,h}^{2}(\tau)}.$

By Corollary 2.2, we propose the following numerical procedure to construct
the SCB of $\bm{\theta}_{\mathbf{C}}(\tau)$:

  * Step 1

Use the sample $\\{\mathbf{x}_{t}\\}_{t=1}^{T}$ to estimate
$\widehat{\bm{\theta}}_{\mathbf{C}}(\tau)$ by (2.5), and compute
$\widetilde{\bm{\theta}}_{\mathbf{C}}(\tau)$ based on (2.6).

  * Step 2

Generate i.i.d. $k$-dimensional standard normal variables
$\\{\mathbf{v}_{t}^{*}\\}$ and calculate the quantity
$\sup_{\tau\in[0,1]}|\mathbf{V}_{h}^{*}(\tau)|$, in which
$\mathbf{V}_{h}^{*}(\tau)=T^{-1}\sum_{t=1}^{T}\mathbf{v}_{t}^{*}(2\omega_{t,h/\sqrt{2}}(\tau)-\omega_{t,h}(\tau))$.

  * Step 3

Repeat Step 2 $R$ times to obtain the empirical $(1-\alpha)^{th}$ quantile
$\widehat{q}_{1-\alpha}$ of $\sup_{\tau\in[0,1]}|\mathbf{V}_{h}^{*}(\tau)|$.

  * Step 4

Calculate $\widehat{\bm{\Sigma}}_{\mathbf{C}}(\tau)$ using (2.8), and
construct the SCB of $\bm{\theta}_{\mathbf{C}}(\tau)$ by
$\widetilde{\bm{\theta}}_{\mathbf{C}}(\tau)+\widehat{\bm{\Sigma}}_{\mathbf{C}}^{1/2}(\tau)\widehat{q}_{1-\alpha}\mathbb{B}_{k}$,
where $\mathbb{B}_{k}=\\{\mathbf{u}\in\mathbb{R}^{k}:|\mathbf{u}|\leq 1\\}$ is
the unit ball, and $k$ is the rank of $\mathbf{C}$.

###  2.4 Examples

Below, we demonstrate the usefulness of the aforementioned results by
considering Example 1 and Example 2 of Section 1.

Example 1 (Cont.) — For $\forall\tau\in[0,1]$, simple algebra shows that the
approximated stationary process is defined by

$\displaystyle\widetilde{\bm{x}}_{t}(\tau)=\bm{\mu}\left(\widetilde{\mathbf{x}}_{t-1}(\tau),\widetilde{\mathbf{x}}_{t-2}(\tau),\ldots;\bm{\theta}(\tau)\right)+\mathbf{H}\left(\widetilde{\mathbf{x}}_{t-1}(\tau),\widetilde{\mathbf{x}}_{t-2}(\tau),\ldots;\bm{\theta}(\tau)\right)\bm{\varepsilon}_{t},$
(2.11)

where $\bm{\theta}(\tau)$ has been defined in (1.5), and

$\displaystyle\bm{\mu}\left(\widetilde{\mathbf{x}}_{t-1}(\tau),\widetilde{\mathbf{x}}_{t-2}(\tau),\ldots;\bm{\theta}(\tau)\right)$
$\displaystyle=$
$\displaystyle\mathbf{B}_{\tau}^{-1}(1)\mathbf{a}(\tau)+\sum_{j=1}^{\infty}\bm{\Gamma}_{j}(\tau)\widetilde{\bm{x}}_{t-j}(\tau),$
$\displaystyle\mathbf{H}\left(\widetilde{\mathbf{x}}_{t-1}(\tau),\widetilde{\mathbf{x}}_{t-2}(\tau),\ldots;\bm{\theta}(\tau)\right)$
$\displaystyle=$ $\displaystyle\bm{\omega}(\tau).$ (2.12)

Additionally, in (2.4), $\bm{\Gamma}_{j}(\tau)$ is yielded as follows:

$\displaystyle\mathbf{I}_{m}-\sum_{j=1}^{\infty}\bm{\Gamma}_{j}(\tau)L^{j}=\mathbf{B}_{\tau}^{-1}(L)\mathbf{A}_{\tau}(L),$
(2.13)

where
$\mathbf{A}_{\tau}(L):=\mathbf{I}_{m}-\mathbf{A}_{1}(\tau)L-\cdots-\mathbf{A}_{p}(\tau)L^{p}$
and
$\mathbf{B}_{\tau}(L):=\mathbf{I}_{m}+\mathbf{B}_{1}(\tau)L+\cdots+\mathbf{B}_{q}(\tau)L^{q}$.

Then we are able to present the following proposition.

######  Proposition 2.3.

Let $\|\bm{\varepsilon}_{t}\|_{r}<\infty$ for some $r>4$. Suppose that there
is a compact set

$\displaystyle\bm{\Theta}=\\{\bm{\vartheta}=\mathrm{vec}(\mathbf{a},\mathbf{A}_{1},\ldots,\mathbf{A}_{p},\mathbf{B}_{1},\ldots,\mathbf{B}_{q},\bm{\Omega})\mid\bm{\vartheta}\in\mathbb{R}^{d}\\}$

such that (1). for $\forall\tau\in[0,1]$, $\bm{\theta}(\tau)$ lies in the
interior of $\bm{\Theta}$, (2). $\mathrm{det}(\mathbf{A}(L)\mathbf{B}(L))\neq
0$ for all $|L|\leq 1$, (3). $\bm{\Omega}>0$, where
$\mathbf{A}(L):=\mathbf{I}_{m}-\mathbf{A}_{1}L-\cdots-\mathbf{A}_{p}L^{p}$ and
$\mathbf{B}(L):=\mathbf{I}_{m}+\mathbf{B}_{1}L+\cdots+\mathbf{B}_{q}L^{q}$ are
coprime and satisfy some necessary identification conditions. Then, the
results of Theorems 2.1 and 2.2 hold for model (1.4).

We note that the detailed identification conditions required for VARMA
processes (e.g., the final equations form or echelon form) can be found in
Lütkepohl (2005). We no longer discuss them here in order not to derivative
from our main goal.

Example 2 (Cont.) — We further let

$\displaystyle\bm{\Omega}(\tau)=\left[\begin{matrix}1&\rho_{1,2}(\tau)&\cdots&\rho_{1,m}(\tau)\\\
\rho_{1,2}(\tau)&1&\ddots&\vdots\\\ \vdots&\ddots&\ddots&\rho_{m-1,m}(\tau)\\\
\rho_{1,m}(\tau)&\rho_{m-1,m}(\tau)&\ddots&1\\\ \end{matrix}\right].$ (2.14)

For $\forall\tau\in[0,1]$, the corresponding approximated stationary process
is defined as

$\displaystyle\widetilde{\bm{x}}_{t}(\tau)=\mathbf{H}\left(\widetilde{\mathbf{x}}_{t-1}(\tau),\widetilde{\mathbf{x}}_{t-2}(\tau),\ldots;\bm{\theta}(\tau)\right)\bm{\varepsilon}_{t},$
(2.15)

where

$\displaystyle\mathbf{H}\left(\widetilde{\mathbf{x}}_{t-1}(\tau),\widetilde{\mathbf{x}}_{t-2}(\tau),\ldots;\bm{\theta}(\tau)\right)$
(2.16) $\displaystyle=$
$\displaystyle\mathrm{diag}^{1/2}\left(\mathbf{D}_{\tau}^{-1}(1)\mathbf{c}_{0}(\tau)+\sum_{j=1}^{\infty}\mathbf{\Psi}_{j}(\tau)\left(\widetilde{\mathbf{x}}_{t-j}(\tau)\odot\widetilde{\mathbf{x}}_{t-j}(\tau)\right)\right).$

Note that $\mathbf{\Psi}_{j}(\tau)$ is generated as follows:

$\displaystyle\bm{\Psi}_{\tau}(L):=\mathbf{I}_{m}-\sum_{j=1}^{\infty}\bm{\Psi}_{j}(\tau)L^{j}=\mathbf{D}_{\tau}^{-1}(L)\mathbf{C}_{\tau}(L),$
(2.17)

where
$\mathbf{C}_{\tau}(L):=\mathbf{C}_{1}(\tau)L+\cdots+\mathbf{C}_{p}(\tau)L^{p}$
and
$\mathbf{D}_{\tau}(L):=\mathbf{I}_{m}-\mathbf{D}_{1}(\tau)L-\cdots-\mathbf{D}_{q}(\tau)L^{q}$.

Consequently, we can present the following proposition.

######  Proposition 2.4.

Suppose that there is a compact set

$\displaystyle\bm{\Theta}=\\{\bm{\vartheta}=\mathrm{vec}(\mathbf{c}_{0},\mathbf{C}_{1},\ldots,\mathbf{C}_{p},\mathbf{D}_{1},\ldots,\mathbf{D}_{q},\bm{\Omega})\mid\bm{\vartheta}\in\mathbb{R}^{d}\\}$

such that (1). for $\forall\tau\in[0,1]$, $\bm{\theta}(\tau)$ lies in the
interior of $\bm{\Theta}$, (2).
$\|\bm{\Omega}^{1/2}\bm{\varepsilon}_{t}\|_{r}^{2}\sum_{j=1}^{\infty}|\mathbf{\Psi}_{j}|<1$
for some $r>6$, (3). all the roots of
$|\mathbf{I}_{m}-\sum_{j=1}^{p}\mathbf{C}_{j}-\sum_{j=1}^{q}\mathbf{D}_{j}|$
are outside the unit circle with $\mathbf{C}_{j}$’s and $\mathbf{D}_{j}$’s
being squared matrices of nonnegative elements, (4). $\mathbf{c}_{0}$ is a
vector of positive elements, (5). $\mathbf{C}(L)$ and $\mathbf{D}(L)$ are
coprime and the formulation of the GARCH part is minimal, where
$\mathbf{C}(L):=\mathbf{C}_{1}L+\cdots+\mathbf{C}_{p}L^{p}$ and
$\mathbf{D}(L):=\mathbf{I}_{m}-\mathbf{D}_{1}L-\cdots-\mathbf{D}_{q}L^{q}$.
Then the results Theorems 2.1 and 2.2 hold for model (1).

For the identification conditions of the GARCH process, we refer readers to
Proposition 3.4 of Jeantheau (1998), who proves that assuming the minimal
representation is enough for ensuring Assumption 3 holds.

In the following section, we conduct numerical studies using both simulated
and real data to evaluate the finite-sample performance of the proposed
estimation and inferential methods.

##  3 Numerical Studies

In this section, we first present the details of the numerical implementations
in Section 3.1, and then conduct extensive simulations in Section 3.2. Section
3.3 presents a real data example on the conditional correlations between the
Chinese and U.S. stock markets.

###  3.1 Numerical Implementation

Throughout the numerical studies, the Epanechnikov kernel
$K(u)=0.75(1-u^{2})I(|u|\leq 1)$ is adopted. Following Zhou & Wu (2010), we
use $\widetilde{h}=2\widehat{h}$ for the biased corrected estimator, where
$\widehat{h}$ is the bandwidth selected by the cross-validation method of
Richter et al. (2019).

Specifically, define the leave-one-out local linear QMLE

$(\widehat{\bm{\theta}}_{h,-t}(\tau),h\widehat{\bm{\theta}}_{h,-t}^{(1)}(\tau))=\operatorname*{\arg\\!\max}_{(\bm{\eta}_{1},\bm{\eta}_{2})\in\mathbf{E}_{T}(r)}\mathcal{L}_{T,-t}^{c}(\tau,\bm{\eta}_{1},\bm{\eta}_{2}),$
(3.1)

where

$\mathcal{L}_{T,-t}^{c}(\tau,\bm{\eta}_{1},\bm{\eta}_{2})=\frac{1}{T}\sum_{s=1,\neq
t}^{T}\mathcal{l}(\mathbf{x}_{s},\mathbf{z}_{s-1}^{c};\bm{\eta}_{1}+\bm{\eta}_{2}\cdot(\tau_{s}-\tau)/h)K_{h}(\tau_{s}-\tau).$

Then, the bandwidth is chosen by

$\widehat{h}=\operatorname*{\arg\\!\max}_{h}T^{-1}\sum_{t=1}^{T}\mathcal{l}(\mathbf{x}_{t},\mathbf{z}_{t-1}^{c};\widehat{\bm{\theta}}_{h,-t}(\tau_{t})).$
(3.2)

As shown in Richter et al. (2019), this cross validation method works well as
long as $\gradient\mathcal{l}$ is uncorrelated, which implies that this
desirable property should hold in our case.

Notably, when considering some specific models, the implementation may be
further simplified. We provide more discussions along this line in Appendix
B.4.

###  3.2 Simulation Results

In the simulation studies, we examine the empirical coverage probabilities of
simultaneous confidence intervals for nominal levels $\alpha=90\%,\ 95\%$. We
consider the time-varying VARMA($2,1$) and multivariate GARCH($1,1$) model as
follows:

  1. 1.

$\text{DGP
1}:\mathbf{x}_{t}=a_{1}(\tau_{t})\mathbf{x}_{t-1}+a_{2}(\tau_{t})\mathbf{x}_{t-2}+\bm{\eta}_{t}+\mathbf{B}_{1}(\tau_{t})\bm{\eta}_{t-1},\quad\bm{\eta}_{t}=\bm{\omega}(\tau_{t})\bm{\varepsilon}_{t}$,
where $\\{\bm{\varepsilon}_{t}\\}$ are i.i.d. draws from
$N(\mathbf{0}_{2\times 1},\mathbf{I}_{2})$, $a_{1}(\tau)=0.6\exp(\tau-1)$,
$a_{2}(\tau)=-0.3\exp(\tau-1)$,

$\displaystyle\mathbf{B}_{1}(\tau)$ $\displaystyle=$
$\displaystyle\left[\begin{matrix}0.5\exp{\tau-0.5}&-0.8(\tau-0.5)^{2}\\\
-0.8(\tau-0.5)^{2}&0.5+0.3\sin(\pi\tau)\end{matrix}\right],$
$\displaystyle\bm{\omega}(\tau)$ $\displaystyle=$
$\displaystyle\left[\begin{matrix}1.5+0.2\exp{0.5-\tau}&0\\\
0.2\exp{0.5-\tau}&1.5+0.5(\tau-0.5)^{2}\end{matrix}\right].$

Here we use final equations form to ensure the uniqueness of the VARMA
representation.

  2. 2.

$\text{DGP
2}:\mathbf{x}_{t}=\mathrm{diag}(h_{1,t}^{1/2},\ldots,h_{m,t}^{1/2})\bm{\eta}_{t}$,
where $\bm{\eta}_{t}=\bm{\Omega}^{1/2}(\tau_{t})\bm{\varepsilon}_{t}$,
$\mathbf{h}_{t}=\mathbf{c}_{0}(\tau_{t})+\mathbf{C}_{1}(\tau_{t})\left(\mathbf{x}_{t-1}\odot\mathbf{x}_{t-1}\right)+\mathbf{D}_{1}(\tau_{t})\mathbf{h}_{t-1}$,
$\\{\bm{\varepsilon}_{t}\\}$ are i.i.d. draws from $N(\mathbf{0}_{2\times
1},\mathbf{I}_{2})$,
$\mathbf{c}_{0}(\tau)=[2\exp{0.5\tau-0.5},3+0.2\cos(\tau)]^{\top}$,

$\displaystyle\mathbf{C}_{1}(\tau)$ $\displaystyle=$
$\displaystyle\left[\begin{matrix}0.4+0.05\cos(\tau)&0.05(\tau-0.5)^{2}\\\
0.05(\tau-0.5)^{2}&0.4+0.05\sin(\tau)\end{matrix}\right],$
$\displaystyle\mathbf{D}_{1}(\tau)$ $\displaystyle=$
$\displaystyle\left[\begin{matrix}0.4-0.1\cos(\tau)&0\\\
0&0.3-0.1\sin(\tau)\end{matrix}\right],$ $\displaystyle\bm{\Omega}(\tau)$
$\displaystyle=$ $\displaystyle\left[\begin{matrix}1&0.3\sin(\tau)\\\
0.3\sin(\tau)&1\end{matrix}\right].$

Let the sample size be $T\in\\{500,1000\\}$ ($T\in\\{1000,2000,4000\\}$) for
the VARMA model (the GARCH model). We conduct $1000$ replications for each
choice of $T$. Several different bandwidths close to $\widetilde{h}$ are
reported to check the sensitivity of bandwidth selection.

We present the empirical coverage probabilities associated with the SCB in
Tables 1–2. For the vector- or matrix-valued unknown coefficients, we take an
average across the elements. A few facts emerge from the tables. First, the
finite sample coverage probabilities are smaller than their nominal level when
$T=500$ ($T=1000,2000$) for the VARMA model (the GARCH model), but are fairly
close to their nominal level as $T=1000$ ($T=4000$) for the VARMA model (the
GARCH model). Second, the behaviour of the estimated simultaneous confidence
intervals is not sensitive to the choices of bandwidths. Third, the GARCH
model requires more data to reach a reasonable finite sample performance.

Table 1: Empirical Coverage Probabilities of the SCB for DGP 1 |  |  | $90\%$ |  | $95\%$
---|---|---|---|---|---
| $\widetilde{h}$ |  | $\alpha_{1}(\cdot)$ | $\alpha_{2}(\cdot)$ | $\mathbf{B}_{1}(\cdot)$ | $\bm{\Omega}(\cdot)$ |  | $\alpha_{1}(\cdot)$ | $\alpha_{2}(\cdot)$ | $\mathbf{B}_{1}(\cdot)$ | $\bm{\Omega}(\cdot)$
$T=500$ | $0.35$ |  | 0.845 | 0.877 | 0.821 | 0.847 |  | 0.905 | 0.915 | 0.889 | 0.905
$0.4$ |  | 0.865 | 0.875 | 0.847 | 0.876 |  | 0.912 | 0.930 | 0.897 | 0.909
$0.45$ |  | 0.862 | 0.895 | 0.847 | 0.878 |  | 0.915 | 0.930 | 0.898 | 0.919
$0.5$ |  | 0.875 | 0.895 | 0.847 | 0.876 |  | 0.905 | 0.945 | 0.901 | 0.920
$T=1000$ | $0.3$ |  | 0.895 | 0.925 | 0.887 | 0.884 |  | 0.960 | 0.960 | 0.947 | 0.947
$0.35$ |  | 0.910 | 0.927 | 0.886 | 0.890 |  | 0.940 | 0.967 | 0.940 | 0.930
$0.4$ |  | 0.917 | 0.939 | 0.901 | 0.899 |  | 0.947 | 0.959 | 0.948 | 0.939
$0.45$ |  | 0.937 | 0.932 | 0.908 | 0.895 |  | 0.957 | 0.957 | 0.947 | 0.937
Table 2: Empirical Coverage Probabilities of the SCB for DGP 2 |  |  | $90\%$ |  | $95\%$
---|---|---|---|---|---
| $\widetilde{h}$ |  | $\mathbf{c}_{0}(\cdot)$ | $\mathbf{C}_{1}(\cdot)$ | $\mathbf{D}_{1}(\cdot)$ | $\bm{\Omega}(\cdot)$ |  | $\mathbf{c}_{0}(\cdot)$ | $\mathbf{C}_{1}(\cdot)$ | $\mathbf{D}_{1}(\cdot)$ | $\bm{\Omega}(\cdot)$
$T=1000$ | $0.55$ |  | 0.802 | 0.810 | 0.784 | 0.889 |  | 0.869 | 0.876 | 0.838 | 0.945
$0.60$ |  | 0.824 | 0.820 | 0.791 | 0.879 |  | 0.882 | 0.866 | 0.843 | 0.945
$0.65$ |  | 0.832 | 0.820 | 0.796 | 0.874 |  | 0.889 | 0.872 | 0.859 | 0.945
$0.70$ |  | 0.820 | 0.823 | 0.792 | 0.879 |  | 0.892 | 0.881 | 0.871 | 0.950
$T=2000$ | $0.50$ |  | 0.827 | 0.835 | 0.841 | 0.889 |  | 0.897 | 0.881 | 0.901 | 0.950
$0.55$ |  | 0.829 | 0.825 | 0.843 | 0.884 |  | 0.892 | 0.881 | 0.903 | 0.940
$0.60$ |  | 0.849 | 0.833 | 0.871 | 0.900 |  | 0.900 | 0.888 | 0.910 | 0.950
$0.65$ |  | 0.852 | 0.835 | 0.873 | 0.910 |  | 0.907 | 0.889 | 0.910 | 0.950
$T=4000$ | $0.35$ |  | 0.879 | 0.879 | 0.882 | 0.869 |  | 0.929 | 0.932 | 0.943 | 0.920
$0.4$ |  | 0.899 | 0.879 | 0.882 | 0.859 |  | 0.950 | 0.944 | 0.943 | 0.919
$0.45$ |  | 0.904 | 0.899 | 0.879 | 0.838 |  | 0.950 | 0.947 | 0.946 | 0.950
$0.50$ |  | 0.867 | 0.857 | 0.884 | 0.898 |  | 0.929 | 0.944 | 0.946 | 0.960

###  3.3 A Real Data Example

In this subsection, we investigate the time-varying conditional correlations
between the Chinese and U.S. stock markets using the time-varying multivariate
GARCH model. Recently, there is a growing literature to study the relationship
of the two stock markets (e.g., Zhang & Li 2014, Pan et al. 2022), as the
Chinese stock market has become the world’s second largest stock market after
2009. Understanding the interactions among different financial markets is
important for investors and policymakers Diebold & Yilmaz (2009), BenSaïda
(2019). For example, high equity market interdependence implies poor
diversification benefits from portfolios, but highlights the possibility of
better hedging benefits.

Previous research documents a strong positive link between the degree of
globalization and equity market interdependence Baele (2005). Along this line
of research, one important question is that whether the interdependence
between the Chinese and U.S. stock markets has increased over time due to
globalization so that estimates from historical data are unreliable for modern
policy analysis, asset pricing and risk management. The existing results
present many discrepancies, which may be due to the fact that the relationship
evolves with time. Apparently, the results also indicate that one should use
time-varying GARCH model to accommodate potential nonstationarity inherited in
these financial variables. In addition, as pointed out by Caporin & McAleer
(2013), dynamic conditional correlation (DCC) GARCH model represents the
dynamic conditional covariances of the standardized residuals, and hence does
not yield dynamic conditional correlations; DCC yields inconsistent two step
estimators; DCC has no asymptotic properties. In what follows, we address
these issues using the newly proposed approach. The estimation is conducted in
exactly the same way as in Section 3.1, so we no longer repeat the details.

We calculate the Chinese and U.S. stock returns based on weekly Shanghai Stock
Exchange (SSE) Composite Index and S&P 500 Index as they are the most
comprehensive and diversified stock indices. The sample employed in this study
spanning from January 2000 to February 2022 provides $1119$ observations111The
data are collected from Yahoo Finance at https://finance.yahoo.com/.. Figure 1
plots the two weekly returns as well as sample autocorrelation functions of
squared data, which shows the typical “volatility clustering” phenomenon.

Figure 1: S&P 500 and SSE Index returns as well as sample autocorrelation
functions of squared data

We next fit the data to a time-varying multivariate GARCH(1,1) model and are
particularly interested in the estimates of time-varying conditional
correlations, i.e.,

$E\left(x_{1,t}x_{2,t}\mid\mathcal{F}_{t-1}\right)/\sqrt{E\left(x_{1,t}^{2}\mid\mathcal{F}_{t-1}\right)E\left(x_{2,t}^{2}\mid\mathcal{F}_{t-1}\right)}=\rho_{1,2}(\tau_{t}),$

where $\rho_{1,2}(\cdot)$ is defined in (2.14). Figure 2 plots the estimates
(black solid line) of time-varying conditional correlations between the two
stock markets as well as 95% simultaneous confidence intervals (red dashed
line) and 95% pointwise confidence intervals (black dashed line). Based on the
simultaneous confidence intervals, apparently, the conditional correlations
vary with respect to time. Moreover, as clearly presented in Figure 2, the
interdependence between the two stock markets is increasing over time. By
examining the pointwise confidence intervals, we can conclude that the two
stock markets are not significantly correlated before 2005, but the
relationship has been greatly enhanced in recent years. These results have
important implications for investment and risk management. For example, it
implies that the Chinese and U.S. investors who use cross-country portfolio
strategies to eliminate country specific risks may be benefit from hedging.
However, all types of investors should be cautious since the relations between
the Chinese and U.S. stock markets are time-varying.

Figure 2: Time-varying conditional correlations between the Chinese and U.S.
stock markets

##  4 Conclusions

In this paper, we consider a wide class of time-varying multivariate causal
processes which nests many classic and new examples as special cases. We first
prove the existence of a weakly dependent stationary approximation for the
model (1.3) which is the foundation to establish the corresponding asymptotic
properties. Afterwards, we consider the QMLE estimation approach, and provide
both point-wise and simultaneous inferences on the coefficient functions. In
addition, we demonstrate the theoretical findings through both simulated and
real data examples. In particular, we show the empirical relevance of our
study using an application to evaluate the conditional correlations between
the stock markets of China and U.S. We find that the interdependence between
the two stock markets is increasing over time.

There are several directions for possible extensions. The first one is to
consider quantile regression methods for such locally stationary multivariate
causal processes. The second one is to propose a more powerful $L_{2}$ test
based on the weighted integrated squared errors for testing whether some
coefficients are time-invariant. We wish to leave such issues for future
study.

##  5 Acknowledgements

The authors of this paper would like to thank George Athanasopoulos, David
Frazier and Gael Martin for their constructive comments on earlier versions of
this paper. Thanks also go to seminar participants for their insightful
suggestions. Gao and Peng would also like to acknowledge the Australian
Research Council Discovery Projects Program for its financial support under
Grant Numbers: DP170104421 & DP210100476.

## References

  * (1)
  * Baele (2005) Baele, L. (2005), ‘Volatility spillover effects in european equity markets’, Journal of Financial and Quantitative Analysis 40(2), 373–401.
  * Bardet & Wintenberger (2009) Bardet, J.-M. & Wintenberger, O. (2009), ‘Asymptotic normality of the quasi-maximum likelihood estimator for multidimensional causal processes’, Annals of Statistics 37(5B), 2730–2759.
  * Bauwens et al. (2006) Bauwens, L., Laurent, S. & Rombouts, J. V. (2006), ‘Multivariate garch models: a survey’, Journal of Applied Econometrics 21(1), 79–109.
  * BenSaïda (2019) BenSaïda, A. (2019), ‘Good and bad volatility spillovers: An asymmetric connectedness’, Journal of Financial Markets 43, 78–95.
  * Bollerslev (1990) Bollerslev, T. (1990), ‘Modelling the coherence in short-run nominal exchange rates: a multivariate generalized ARCH model’, Review of Economics and Statistics 72(3), 498–505.
  * Caporin & McAleer (2013) Caporin, M. & McAleer, M. (2013), ‘Ten things you should know about the dynamic conditional correlation representation’, Econometrics 1(1), 115–126.
  * Chen et al. (2021) Chen, L., Wang, W. & Wu, W. B. (2021), ‘Inference of breakpoints in high-dimensional time series’, Journal of the American Statistical Association 0(0), 1–13.
  * Chen & Leng (2015) Chen, Z. & Leng, C. (2015), ‘Local linear estimation of covariance matrices via cholesky decomposition’, Statistica Sinica 25(3), 1249–1263.
  * Dahlhaus (1996) Dahlhaus, R. (1996), ‘On the kullback-leibler information divergence of locally stationary processes’, Stochastic Processes and Their Applications 62(1), 139–168.
  * Dahlhaus et al. (2019) Dahlhaus, R., Richter, S. & Wu, W. B. (2019), ‘Towards a general theory for nonlinear locally stationary processes’, Bernoulli 25(2), 1013–1044.
  * Dette et al. (2011) Dette, H., Preuß, P. & Vetter, M. (2011), ‘A measure of stationarity in locally stationary processes with applications to testing’, Journal of the American Statistical Association 106(495), 1113–1124.
  * Diebold & Yilmaz (2009) Diebold, F. X. & Yilmaz, K. (2009), ‘Measuring financial asset return and volatility spillovers, with application to global equity markets’, The Economic Journal 119(534), 158–171.
  * Hall & Heyde (1980) Hall, P. & Heyde, C. C. (1980), Martingale Limit Theory and Its Application, Academic Press.
  * Jeantheau (1998) Jeantheau, T. (1998), ‘Strong consistency of estimators for multivariate arch models’, Econometric theory 14(1), 70–86.
  * Karmakar et al. (2022) Karmakar, S., Richter, S. & Wu, W. B. (2022), ‘Simultaneous inference for time-varying models’, Journal of Econometrics 227(2), 408–428.
  * Ling (2003) Ling, S. (2003), ‘Adaptive estimators and tests of stationary and nonstationary short-and long-memory arfima–garch models’, Journal of the American Statistical Association 98(464), 955–967.
  * Ling & McAleer (2003) Ling, S. & McAleer, M. (2003), ‘Asymptotic theory for a vector ARMA-GARCH model’, Econometric theory pp. 280–310.
  * Lütkepohl (2005) Lütkepohl, H. (2005), New Introduction to Multiple Time Series Analysis, Springer Science & Business Media.
  * Pan et al. (2022) Pan, Q., Mei, X. & Gao, T. (2022), ‘Modeling dynamic conditional correlations with leverage effects and volatility spillover effects: Evidence from the chinese and us stock markets affected by the recent trade friction’, The North American Journal of Economics and Finance 59, 101591.
  * Preuss et al. (2015) Preuss, P., Puchstein, R. & Dette, H. (2015), ‘Detection of multiple structural breaks in multivariate time series’, Journal of the American Statistical Association 110(510), 654–668.
  * Richter et al. (2019) Richter, S., Dahlhaus, R. et al. (2019), ‘Cross validation for locally stationary processes’, Annals of Statistics 47(4), 2145–2173.
  * Stock & Watson (2001) Stock, J. H. & Watson, M. W. (2001), ‘Vector autoregressions’, Journal of Economic Perspectives 15(4), 101–115.
  * Tong (1990) Tong, H. (1990), Non-linear Time Series: a Dynamical Systems Approach, Oxford University Press.
  * Truquet (2017) Truquet, L. (2017), ‘Parameter stability and semiparametric inference in time varying auto-regressive conditional heteroscedasticity models’, Journal of the Royal Statistical Society: Series B 79(5), 1391–1414.
  * Vogt (2012) Vogt, M. (2012), ‘Nonparametric regression for locally stationary time series’, Annals of Statistics 40(5), 2601–2633.
  * Wu (2005) Wu, W. B. (2005), ‘Nonlinear system theory: Another look at dependence’, Proceedings of the National Academy of Sciences 102(40), 14150–14154.
  * Wu & Shao (2004) Wu, W. B. & Shao, X. (2004), ‘Limit theorems for iterated random functions’, Journal of Applied Probability 41(2), 425–436.
  * Wu & Zhou (2011) Wu, W. B. & Zhou, Z. (2011), ‘Gaussian approximations for non-stationary multiple time series’, Statistica Sinica pp. 1397–1413.
  * Zhang & Li (2014) Zhang, B. & Li, X.-M. (2014), ‘Has there been any change in the comovement between the chinese and us stock markets?’, International Review of Economics & Finance 29, 525–536.
  * Zhang & Wu (2017) Zhang, D. & Wu, W. B. (2017), ‘Gaussian approximation for high dimensional time series’, Annals of Statistics 45(5), 1895–1919.
  * Zhang & Wu (2012) Zhang, T. & Wu, W. B. (2012), ‘Inference of time-varying regression models’, Annals of Statistics 40(3), 1376–1402.
  * Zhou & Wu (2010) Zhou, Z. & Wu, W. B. (2010), ‘Simultaneous inference of linear models with time varying coefficients’, Journal of the Royal Statistical Society: Series B 72(4), 513–531.

Online Supplementary Appendices to
“Time-Varying Multivariate Causal Processes”

Jiti Gao∗ and Bin Peng∗ and Wei Biao Wu† and Yayi Yan∗

∗Monash University and †University of Chicago

The file includes Appendix A and Appendix B. We first present some technical
tools in Appendix A.1, which will be repeatedly used in the development. We
then provide the proofs of main results in Appendix A.2. We provide several
preliminary lemmas in Appendix B.1 as well as some secondary lemmas in
Appendix B.2, and then present the proofs of preliminary lemmas in Appendix
B.3. Appendix B.4 discusses several computational issues of the local linear
ML estimation.

In what follows, $M$ and $O(1)$ always stand for some bounded constants, and
may be different at each appearance.

## Appendix A

##  A.1 Technical Tools

Projection Operator: Define the projection operator

$\displaystyle\mathcal{P}_{t}(\cdot)=E[\cdot\mid\mathcal{F}_{t}]-E[\cdot\mid\mathcal{F}_{t-1}],$

where
$\mathcal{F}_{t}=\sigma(\bm{\varepsilon}_{t},\bm{\varepsilon}_{t-1},\ldots)$.
By the Jensen’s inequality and the stationarity of
$\widetilde{\mathbf{x}}_{t}(\tau)$, for $l\geq 0$, we have

$\displaystyle\|\mathcal{P}_{t-l}(\widetilde{\mathbf{x}}_{t}(\tau))\|_{r}$
$\displaystyle=$
$\displaystyle\|E[\widetilde{\mathbf{x}}_{t}(\tau)\mid\mathcal{F}_{t-l}]-E[\widetilde{\mathbf{x}}_{t}(\tau)\mid\mathcal{F}_{t-l-1}]\|_{r}$
$\displaystyle=$
$\displaystyle\|E[\widetilde{\mathbf{x}}_{t}(\tau)\mid\mathcal{F}_{t-l}]-E[\widetilde{\mathbf{x}}_{t}^{(t-l,*)}(\tau)\mid\mathcal{F}_{t-l-1}]\|_{r}$
$\displaystyle=$
$\displaystyle\|E[\widetilde{\mathbf{x}}_{t}(\tau)-\widetilde{\mathbf{x}}_{t}^{(t-l,*)}(\tau)\mid\mathcal{F}_{t-l}]\|_{r}$
$\displaystyle\leq$
$\displaystyle\|\widetilde{\mathbf{x}}_{t}(\tau)-\widetilde{\mathbf{x}}_{t}^{(t-l,*)}(\tau)\|_{r}=\delta_{r}^{\mathbf{x}(\tau)}(l),$

where $\widetilde{\mathbf{x}}_{t}^{(t-l,*)}(\tau)$ is a coupled version of
$\widetilde{\mathbf{x}}_{t}(\tau)$ with $\varepsilon_{t-l}$ replaced by
$\varepsilon_{t-l}^{*}$.

The Class $\mathcal{H}(C,\bm{\chi},M)$:

Recall that we have defined $\bm{\Theta}_{r}$ in Assumption1. Let
$\bm{\chi}=\\{\chi_{j}\\}_{j=1}^{\infty}$ be a sequence of nonnegative real
numbers with $|\bm{\chi}|_{1}:=\sum_{j=1}^{\infty}\chi_{j}<\infty$ and $M>0$
be some finite constant. Let
$|\mathbf{z}|_{\bm{\chi}}:=\sum_{j=1}^{\infty}\chi_{j}|\mathbf{z}_{j}|$ for
any $\mathbf{z}\in(\mathbb{R}^{m})^{\infty}$ and $C\geq 1$, where
$\mathbf{z}_{j}$ is the $j^{th}$ column of $\mathbf{z}$. A function
$g(\mathbf{z},\bm{\vartheta}):(\mathbb{R}^{m})^{\infty}\times\bm{\Theta}_{r}\to\mathbb{R}$
is in class $\mathcal{H}(C,\bm{\chi},M)$ if

$\displaystyle\sup_{\bm{\vartheta}\in\bm{\Theta}_{r}}|g(\mathbf{0},\bm{\vartheta})|\leq
M,$
$\displaystyle\sup_{\mathbf{z}}\sup_{\bm{\vartheta}\neq\bm{\vartheta}^{\prime}}\frac{|g(\mathbf{z},\bm{\vartheta})-g(\mathbf{z},\bm{\vartheta}^{\prime})|}{|\bm{\vartheta}-\bm{\vartheta}^{\prime}|(1+|\mathbf{z}|_{\bm{\chi}}^{C})}\leq
M,$
$\displaystyle\sup_{\bm{\vartheta}}\sup_{\mathbf{z}\neq\mathbf{z}^{\prime}}\frac{|g(\mathbf{z},\bm{\vartheta})-g(\mathbf{z}^{\prime},\bm{\vartheta})|}{|\mathbf{z}-\mathbf{z}^{\prime}|_{\bm{\chi}}(1+|\mathbf{z}|_{\bm{\chi}}^{C-1}+|\mathbf{z}^{\prime}|_{\bm{\chi}}^{C-1})}\leq
M.$

If $g$ is vector- or matrix-valued, $g\in\mathcal{H}(C,\bm{\chi},M)$ means
that every component of $g$ is in $\mathcal{H}(C,\bm{\chi},M)$.

Analytical Gradient:

Let

$\displaystyle\mathcal{l}(\mathbf{x},\mathbf{z};\bm{\vartheta})=-\frac{1}{2}(\mathbf{x}-\bm{\mu}(\mathbf{z};\bm{\vartheta}))^{\top}\mathbf{M}^{-1}(\mathbf{z};\bm{\vartheta})(\mathbf{x}-\bm{\mu}(\mathbf{z};\bm{\vartheta}))-\frac{1}{2}\log\det\left(\mathbf{M}(\mathbf{z};\bm{\vartheta})\right),$

where
$\mathbf{M}(\mathbf{z};\bm{\vartheta})=\mathbf{H}(\mathbf{z};\bm{\vartheta})\mathbf{H}(\mathbf{z};\bm{\vartheta})^{\top}$.
Then the first partial derivative is as follows:

$\displaystyle\frac{\partial\mathcal{l}(\mathbf{x},\mathbf{z};\bm{\vartheta})}{\partial\vartheta_{i}}$
$\displaystyle=$
$\displaystyle(\mathbf{x}-\bm{\mu}(\mathbf{z};\bm{\vartheta}))^{\top}\mathbf{M}^{-1}(\mathbf{z};\bm{\vartheta})\frac{\partial\bm{\mu}(\mathbf{z};\bm{\vartheta})}{\partial\vartheta_{i}}$
(A.1.1)
$\displaystyle-\frac{1}{2}(\mathbf{x}-\bm{\mu}(\mathbf{z};\bm{\vartheta}))^{\top}\frac{\partial\mathbf{M}^{-1}(\mathbf{z};\bm{\vartheta})}{\partial\vartheta_{i}}(\mathbf{x}-\bm{\mu}(\mathbf{z};\bm{\vartheta}))$
$\displaystyle-\frac{1}{2}\mathrm{tr}\left(\mathbf{M}^{-1}(\mathbf{z};\bm{\vartheta})\frac{\partial\mathbf{M}(\mathbf{z};\bm{\vartheta})}{\partial\vartheta_{i}}\right),$

where $\vartheta_{i}$ is the $i^{th}$ element of $\bm{\vartheta}$.

By (A.1.1), the second partial derivative of
$\mathcal{l}(\mathbf{x},\mathbf{z};\bm{\vartheta})$ is given by

$\displaystyle\frac{\partial^{2}\mathcal{l}(\mathbf{x},\mathbf{z};\bm{\vartheta})}{\partial\vartheta_{i}\partial\vartheta_{j}}$
$\displaystyle=$
$\displaystyle(\mathbf{x}-\bm{\mu}(\mathbf{z};\bm{\vartheta}))^{\top}\mathbf{M}^{-1}(\mathbf{z};\bm{\vartheta})\frac{\partial^{2}\bm{\mu}(\mathbf{z};\bm{\vartheta})}{\partial\vartheta_{i}\partial\vartheta_{j}}$
(A.1.2)
$\displaystyle-\frac{1}{2}(\mathbf{x}-\bm{\mu}(\mathbf{z};\bm{\vartheta}))^{\top}\frac{\partial^{2}\mathbf{M}^{-1}(\mathbf{z};\bm{\vartheta})}{\partial\vartheta_{i}\partial\vartheta_{j}}(\mathbf{x}-\bm{\mu}(\mathbf{z};\bm{\vartheta}))$
$\displaystyle+(\mathbf{x}-\bm{\mu}(\mathbf{z};\bm{\vartheta}))^{\top}\left(\frac{\partial\mathbf{M}^{-1}(\mathbf{z};\bm{\vartheta})}{\partial\vartheta_{i}}\frac{\partial\bm{\mu}(\mathbf{z};\bm{\vartheta})}{\partial\vartheta_{j}}+\frac{\partial\mathbf{M}^{-1}(\mathbf{z};\bm{\vartheta})}{\partial\vartheta_{j}}\frac{\partial\bm{\mu}(\mathbf{z};\bm{\vartheta})}{\partial\vartheta_{i}}\right)$
$\displaystyle-\left(\frac{\partial\bm{\mu}(\mathbf{z};\bm{\vartheta})}{\partial\vartheta_{j}}\right)^{\top}\mathbf{M}^{-1}(\mathbf{z};\bm{\vartheta})\frac{\partial\bm{\mu}(\mathbf{z};\bm{\vartheta})}{\partial\vartheta_{i}}$
$\displaystyle-\frac{1}{2}\mathrm{tr}\left(\mathbf{M}^{-1}(\mathbf{z};\bm{\vartheta})\frac{\partial^{2}\mathbf{M}(\mathbf{z};\bm{\vartheta})}{\partial\vartheta_{i}\partial\vartheta_{j}}\right)-\frac{1}{2}\mathrm{tr}\left(\frac{\partial\mathbf{M}^{-1}(\mathbf{z};\bm{\vartheta})}{\partial\vartheta_{j}}\frac{\partial\mathbf{M}(\mathbf{z};\bm{\vartheta})}{\partial\vartheta_{i}}\right).$

##  A.2 Proofs of the Main Results

###### Proof of Proposition 2.1.

(1). To prove the first result, we consider an approximated $p$-Markov process
defined by

$\displaystyle\widetilde{\mathbf{x}}_{p,t}(\tau)$ $\displaystyle=$
$\displaystyle\bm{\mu}\left(\widetilde{\mathbf{x}}_{p,t-1}(\tau),\ldots,\widetilde{\mathbf{x}}_{p,t-p}(\tau),\mathbf{0},\ldots;\bm{\theta}(\tau)\right)$
(A.2.1)
$\displaystyle+\mathbf{H}\left(\widetilde{\mathbf{x}}_{p,t-1}(\tau),\ldots,\widetilde{\mathbf{x}}_{p,t-p}(\tau),\mathbf{0},\ldots;\bm{\theta}(\tau)\right)\bm{\varepsilon}_{t}$

for $p\geq 1$, and

$\displaystyle\widetilde{\mathbf{x}}_{0,t}(\tau)=\bm{\mu}\left(\mathbf{0},\ldots;\bm{\theta}(\tau)\right)+\mathbf{H}\left(\mathbf{0},\ldots;\bm{\theta}(\tau)\right)\bm{\varepsilon}_{t}.$
(A.2.2)

Let $\mu_{p,r}(\tau)=\left\|\widetilde{\mathbf{x}}_{p,t}(\tau)\right\|_{r}$
and
$\Delta_{p,r}(\tau)=\left\|\widetilde{\mathbf{x}}_{p+1,t}(\tau)-\widetilde{\mathbf{x}}_{p,t}(\tau)\right\|_{r}$.

By construction, we immediately obtain

$\displaystyle\mu_{p,r}(\tau)$ $\displaystyle\leq$
$\displaystyle\|\widetilde{\mathbf{x}}_{p,t}(\tau)-\widetilde{\mathbf{x}}_{0,t}(\tau)\|_{r}+\mu_{0,r}(\tau)$
$\displaystyle\leq$
$\displaystyle\left(\sum_{j=1}^{p}\alpha_{j}(\bm{\theta}(\tau))+\|\bm{\varepsilon}_{1}\|_{r}\sum_{j=1}^{p}\beta_{j}(\bm{\theta}(\tau))\right)\mu_{p,r}(\tau)+\mu_{0,r}(\tau),$

where the second inequality follows from Assumption 1.

Recall that we have defined
$\rho(\tau):=\sum_{j=1}^{\infty}\alpha_{j}(\bm{\theta}(\tau))+\|\bm{\varepsilon}_{t}\|_{r}\sum_{j=1}^{\infty}\beta_{j}(\bm{\theta}(\tau))$
in the body of this proposition. As $0\leq\rho(\tau)<1$ by Assumption 1, we
have

$\displaystyle\sup_{p\geq
0}\mu_{p,r}(\tau)\leq(1-\rho(\tau))^{-1}\mu_{0,r}(\tau)<\infty.$

Similarly, we have

$\displaystyle\Delta_{p,r}(\tau)$ $\displaystyle\leq$
$\displaystyle\left(\sum_{j=1}^{p}\alpha_{j}(\bm{\theta}(\tau))+\|\bm{\varepsilon}_{1}\|_{r}\sum_{j=1}^{p}\beta_{j}(\bm{\theta}(\tau))\right)\Delta_{p,r}(\tau)$
$\displaystyle+\left(\alpha_{p+1}(\bm{\theta}(\tau))+\|\bm{\varepsilon}_{1}\|_{r}\beta_{p+1}(\bm{\theta}(\tau))\right)\left\|\widetilde{\mathbf{x}}_{p+1,t-p-1}(\tau)\right\|_{r}.$

Hence,

$\displaystyle\Delta_{p,r}(\tau)\leq\left(\alpha_{p+1}(\bm{\theta}(\tau))+\|\bm{\varepsilon}_{1}\|_{r}\beta_{p+1}(\bm{\theta}(\tau))\right)\left(1-\rho(\tau)\right)^{-2}\mu_{0,r}(\tau)\to
0$

as $p\to\infty$.

According to the above development, we are readily to conclude that
$\widetilde{\mathbf{x}}_{p,t}(\tau)\to\widetilde{\mathbf{x}}_{t}(\tau)$ as
$p\to\infty$ in the space of
$\mathbb{L}^{r}=\\{\mathbf{x}\mid\|\mathbf{x}\|_{r}<\infty\\}$. As a limit of
strictly stationary process in $\mathbb{L}^{r}$,
$\widetilde{\mathbf{x}}_{t}(\tau)$ is a stationary process and
$\sup_{\tau\in[0,1]}\left\|\widetilde{\mathbf{x}}_{t}(\tau)\right\|_{r}<\infty$.

(2). Let $\\{\bm{\varepsilon}_{t}^{*}\\}$ be an independent copy of
$\\{\bm{\varepsilon}_{t}\\}$. Similar to (A.2.1), we define the process
$\\{\widetilde{\mathbf{x}}_{p,t}^{*}(\tau)\\}$, in which the difference is
that we use $\bm{\varepsilon}_{t}$ when $t\neq 0$, and use
$\bm{\varepsilon}_{t}^{*}$ when $t=0$. In addition, define the process
$\\{\widetilde{\mathbf{x}}_{t}^{*}(\tau)\\}$ as
$\\{\widetilde{\mathbf{x}}_{t}(\tau)\\}$, in which again the difference is
that we use $\bm{\varepsilon}_{t}$ when $t\neq 0$, and use
$\bm{\varepsilon}_{t}^{*}$ when $t=0$. Further define
$u_{t}=\left\|\widetilde{\mathbf{x}}_{p,t}^{*}(\tau)-\widetilde{\mathbf{x}}_{p,t}(\tau)\right\|_{r}$.

By construction, $u_{t}=0$ for $t<0$, and
$u_{0}=\left\|\widetilde{\mathbf{x}}_{p,0}^{*}(\tau)-\widetilde{\mathbf{x}}_{p,0}(\tau)\right\|_{r}=O\left(\left\|\bm{\varepsilon}_{0}^{*}-\bm{\varepsilon}_{0}\right\|_{r}\right)=O(1)$.
For $t>0$, Assumption 1 gives that

$\displaystyle\left\|\widetilde{\mathbf{x}}_{p,t}(\tau)-\widetilde{\mathbf{x}}_{p,t}^{*}(\tau)\right\|_{r}\leq\sum_{j=1}^{p}\left(\alpha_{j}(\bm{\theta}(\tau))+\|\bm{\varepsilon}_{t}\|_{r}\beta_{j}(\bm{\theta}(\tau))\right)\left\|\widetilde{\mathbf{x}}_{p,t-j}(\tau)-\widetilde{\mathbf{x}}_{p,t-j}^{*}(\tau)\right\|_{r}.$
(A.2.3)

Since
$\sum_{j=1}^{p}\left(\alpha_{j}(\bm{\theta}(\tau))+\|\bm{\varepsilon}_{t}\|_{r}\beta_{j}(\bm{\theta}(\tau))\right)\leq\rho(\tau)<1$,
by a recursion argument, we have $u_{t}\leq u_{0}$ for all $t$.

Now, let $v_{t}=\max_{k\geq t}u_{k}$. Using (A.2.3) and the fact that $v_{t}$
is a nonincreasing sequence, we have $v_{t}\leq\rho(\tau)v_{t-p}$ for all
$t\geq 1$. Then recursively
$v_{t}\leq\rho(\tau)^{-\lfloor-t/p\rfloor}v_{t+p\lfloor-t/p\rfloor}$. Since
$v_{t+p\lfloor-t/p\rfloor}\leq u_{0}$ and $-\lfloor-t/p\rfloor\geq t/p$, we
have $u_{t}\leq v_{t}\leq\rho(\tau)^{t/p}u_{0}$, i.e.,
$\left\|\widetilde{\mathbf{x}}_{p,t}(\tau)-\widetilde{\mathbf{x}}_{p,t}^{*}(\tau)\right\|_{r}=O(\rho(\tau)^{t/p})$.

The proof of the first result gives

$\displaystyle\left\|\widetilde{\mathbf{x}}_{t}(\tau)-\widetilde{\mathbf{x}}_{p,t}(\tau)\right\|_{r}\leq\sum_{j=p}^{\infty}\Delta_{p,r}\leq\frac{\mu_{r}(\tau)}{(1-\rho(\tau))^{2}}\sum_{j=p}^{\infty}\left(\alpha_{p+1}(\bm{\theta}(\tau))+\|\bm{\varepsilon}_{t}\|_{r}\beta_{p+1}(\bm{\theta}(\tau))\right).$

The same bound holds for the quantity
$\|\widetilde{\mathbf{x}}_{t}^{*}(\tau)-\widetilde{\mathbf{x}}_{p,t}^{*}(\tau)\|_{r}$.
Thus,

$\displaystyle\|\widetilde{\mathbf{x}}_{t}(\tau)-\widetilde{\mathbf{x}}_{t}^{*}(\tau)\|_{r}$
$\displaystyle\leq$
$\displaystyle\|\widetilde{\mathbf{x}}_{t}(\tau)-\widetilde{\mathbf{x}}_{p,t}(\tau)\|_{r}+\|\widetilde{\mathbf{x}}_{p,t}(\tau)-\widetilde{\mathbf{x}}_{p,t}^{*}(\tau)\|_{r}+\|\widetilde{\mathbf{x}}_{t}^{*}(\tau)-\widetilde{\mathbf{x}}_{p,t}^{*}(\tau)\|_{r}$
$\displaystyle=$ $\displaystyle
O\left(\rho(\tau)^{t/p}+\sum_{j=p+1}^{\infty}\left(\alpha_{j}(\bm{\theta}(\tau))+\|\bm{\varepsilon}_{t}\|_{r}\beta_{j}(\bm{\theta}(\tau))\right)\right),$

which completes the proof. ∎

###### Proof of Proposition 2.2.

(1). Write

$\displaystyle\|\widetilde{\mathbf{x}}_{t}(\tau)-\widetilde{\mathbf{x}}_{t}(\tau^{\prime})\|_{r}$
$\displaystyle\leq$
$\displaystyle\|\bm{\mu}\left(\widetilde{\mathbf{x}}_{t-1}(\tau),\ldots;\bm{\theta}(\tau)\right)-\bm{\mu}\left(\widetilde{\mathbf{x}}_{t-1}(\tau^{\prime}),\ldots;\bm{\theta}(\tau^{\prime})\right)\|_{r}$
$\displaystyle+\|\bm{\varepsilon}_{t}\|_{r}\|\mathbf{H}\left(\widetilde{\mathbf{x}}_{t-1}(\tau),\ldots;\bm{\theta}(\tau)\right)-\mathbf{H}\left(\widetilde{\mathbf{x}}_{t-1}(\tau^{\prime}),\ldots;\bm{\theta}(\tau^{\prime})\right)\|_{r}$
$\displaystyle\leq$
$\displaystyle\sum_{j=1}^{\infty}\left(\alpha_{j}(\tau)+\|\bm{\varepsilon}_{t}\|_{r}\beta_{j}(\tau)\right)\|\widetilde{\mathbf{x}}_{t-j}(\tau)-\widetilde{\mathbf{x}}_{t-j}(\tau^{\prime})\|_{r}$
$\displaystyle+M|\tau-\tau^{\prime}|\sum_{j=1}^{\infty}\chi_{j}\|\widetilde{\mathbf{x}}_{t-j}(\tau^{\prime})\|_{r},$

where the second inequality follows from Assumption 1 and Assumption 2. In
view of the stationarity of $\widetilde{\mathbf{x}}_{t}(\tau)$, rearranging
the terms in the above inequality yields that

$\displaystyle\|\widetilde{\mathbf{x}}_{t}(\tau)-\widetilde{\mathbf{x}}_{t}(\tau^{\prime})\|_{r}\leq
M(1-\rho(\tau))^{-1}|\tau-\tau^{\prime}|\sum_{j=1}^{\infty}\chi_{j}\|\widetilde{\mathbf{x}}_{t-j}(\tau^{\prime})\|_{r}=O(|\tau-\tau^{\prime}|).$

(2). Write

$\displaystyle\|\mathbf{x}_{t}-\widetilde{\mathbf{x}}_{t}(\tau_{t})\|_{r}$
$\displaystyle\leq$
$\displaystyle\|\bm{\mu}\left(\mathbf{x}_{t-1},\mathbf{x}_{t-2},\ldots;\bm{\theta}(\tau_{t})\right)-\bm{\mu}\left(\widetilde{\mathbf{x}}_{t-1}(\tau_{t}),\widetilde{\mathbf{x}}_{t-2}(\tau_{t}),\ldots;\bm{\theta}(\tau_{t})\right)\|_{r}$
$\displaystyle+\|\bm{\varepsilon}_{t}\|_{r}\|\mathbf{H}\left(\mathbf{x}_{t-1},\mathbf{x}_{t-2},\ldots;\bm{\theta}(\tau_{t})\right)-\mathbf{H}\left(\widetilde{\mathbf{x}}_{t-1}(\tau_{t}),\widetilde{\mathbf{x}}_{t-2}(\tau_{t}),\ldots;\bm{\theta}(\tau_{t})\right)\|_{r}$
$\displaystyle\leq$
$\displaystyle\sum_{j=1}^{\infty}\left(\alpha_{j}(\tau_{t})+\|\bm{\varepsilon}_{t}\|_{r}\beta_{j}(\tau_{t})\right)\left\|\mathbf{x}_{t-j}-\widetilde{\mathbf{x}}_{t-j}(\tau_{t})\right\|_{r}$
$\displaystyle\leq$
$\displaystyle\sum_{j=1}^{\infty}\left(\alpha_{j}(\tau_{t})+\|\bm{\varepsilon}_{t}\|_{r}\beta_{j}(\tau_{t})\right)\left\|\mathbf{x}_{t-j}-\widetilde{\mathbf{x}}_{t-j}(\tau_{t-j}\lor
0)\right\|_{r}$
$\displaystyle+\sum_{j=1}^{\infty}\left(\alpha_{j}(\tau_{t})+\|\bm{\varepsilon}_{t}\|_{r}\beta_{j}(\tau_{t})\right)\left\|\widetilde{\mathbf{x}}_{t-j}(\tau_{t-j}\lor
0)-\widetilde{\mathbf{x}}_{t-j}(\tau_{t})\right\|_{r}.$

As $\left\|\mathbf{x}_{t-j}-\widetilde{\mathbf{x}}_{t-j}(\tau_{t-j}\lor
0)\right\|_{r}=0$ for $j\geq t$, by the first result of this proposition, we
have

$\displaystyle\|\mathbf{x}_{t}-\widetilde{\mathbf{x}}_{t}(\tau_{t})\|_{r}$
$\displaystyle\leq$
$\displaystyle\sum_{j=1}^{t-1}\left(\alpha_{j}(\tau_{t})+\|\bm{\varepsilon}_{t}\|_{r}\beta_{j}(\tau_{t})\right)\left\|\mathbf{x}_{t-j}-\widetilde{\mathbf{x}}_{t-j}(\tau_{t-j})\right\|_{r}$
$\displaystyle+M\cdot
T^{-1}\sum_{j=1}^{\infty}j\left(\alpha_{j}(\tau_{t})+\|\bm{\varepsilon}_{t}\|_{r}\beta_{j}(\tau_{t})\right).$

In addition, as
$\left\|\mathbf{x}_{1}-\widetilde{\mathbf{x}}_{1}(\tau_{1})\right\|_{r}=O(T^{-1})$
and $\sup_{t\geq
2}\sum_{j=1}^{t-1}\left(\alpha_{j}(\tau_{t})+\|\bm{\varepsilon}_{t}\|_{r}\beta_{j}(\tau_{t})\right)<1$,
we have

$\displaystyle\|\mathbf{x}_{t}-\widetilde{\mathbf{x}}_{t}(\tau_{t})\|_{r}$
$\displaystyle\leq$
$\displaystyle\sum_{j=1}^{t-1}\left(\alpha_{j}(\tau_{t})+\|\bm{\varepsilon}_{t}\|_{r}\beta_{j}(\tau_{t})\right)O(T^{-1})$
$\displaystyle+M\cdot
T^{-1}\sum_{j=1}^{\infty}j\left(\alpha_{j}(\tau_{t})+\|\bm{\varepsilon}_{t}\|_{r}\beta_{j}(\tau_{t})\right)=O(T^{-1}).$

The proof is now complete. ∎

###### Proof of Theorem 2.1.

(1). First, we introduce a few notations to facilitate the development. Let
$\widehat{\bm{\eta}}(\tau):=[\widehat{\bm{\theta}}(\tau)^{\top},\widehat{\bm{\theta}}^{\star}(\tau)^{\top}]^{\top}$,
$\bm{\eta}(\tau):=[\bm{\theta}(\tau)^{\top},h\bm{\theta}^{(1)}(\tau)^{\top}]^{\top}$,
and
$\mathcal{L}_{\tau}(\bm{\eta}):=\mathcal{L}_{\tau}(\bm{\eta}_{1},\bm{\eta}_{2})$
for $\bm{\eta}=[\bm{\eta}_{1}^{\top},\bm{\eta}_{2}^{\top}]^{\top}$. Recall
that we have defined $\gradient_{\bm{\vartheta}}$, and let
$\gradient_{\bm{\eta}}$ be defined similarly with respect to the elements of
$\bm{\eta}$.

By the Taylor expansion, we have

$\displaystyle\widehat{\bm{\eta}}(\tau)-\bm{\eta}(\tau)$ $\displaystyle=$
$\displaystyle-(\gradient_{\bm{\eta}}^{2}\mathcal{L}_{\tau}(\overline{\bm{\eta}}))^{-1}\gradient_{\bm{\eta}}\mathcal{L}_{\tau}(\bm{\eta}(\tau)),$

with $\overline{\bm{\eta}}$ between $\widehat{\bm{\eta}}(\tau)$ and
$\bm{\eta}(\tau)$. By Lemma B.3.4, we have

$\displaystyle|\gradient_{\bm{\eta}}\mathcal{L}_{\tau}(\bm{\eta}(\tau))-\gradient_{\bm{\eta}}\widetilde{\mathcal{L}}_{\tau}(\bm{\eta}(\tau))|=O_{P}((Th)^{-1}),$

where
$\widetilde{\mathcal{L}}_{\tau}(\bm{\eta}(\tau))=T^{-1}\sum_{t=1}^{T}\mathcal{l}(\widetilde{\bm{x}}_{t}(\tau_{t}),\widetilde{\bm{z}}_{t-1}(\tau_{t});\bm{\theta}(\tau)+\bm{\theta}^{(1)}(\tau)(\tau_{t}-\tau))K_{h}(\tau_{t}-\tau)$.

Then we consider
$\gradient_{\bm{\eta}}\widetilde{\mathcal{L}}_{\tau}(\bm{\eta}(\tau))$. Since
each element of $\bm{\theta}(\tau)$ is in $C^{3}[0,1]$, we have
$\bm{\theta}(\tau_{t})=\bm{\theta}(\tau)+\bm{\theta}^{(1)}(\tau)(\tau_{t}-\tau)+\mathbf{r}(\tau_{t})$,
where
$\mathbf{r}(\tau_{t})=\frac{1}{2}\bm{\theta}^{(2)}(\tau)(\tau_{t}-\tau)^{2}+\frac{1}{6}\bm{\theta}^{(3)}(\overline{\tau})(\tau_{t}-\tau)^{3}$
with $\overline{\tau}$ between $\tau_{t}$ and $\tau$. Let
$\widehat{\bm{K}}((\tau_{t}-\tau)/h)=[K((\tau_{t}-\tau)/h),(\tau_{t}-\tau)/hK((\tau_{t}-\tau)/h)]^{\top}$.
By the Mean Value Theorem, we have

$\displaystyle\gradient_{\bm{\eta}}\widetilde{\mathcal{L}}_{\tau}(\bm{\eta}(\tau))-\frac{1}{Th}\sum_{t=1}^{T}\widehat{\bm{K}}((\tau_{t}-\tau)/h)\otimes\gradient_{\bm{\vartheta}}\mathcal{l}(\widetilde{\bm{x}}_{t}(\tau_{t}),\widetilde{\bm{z}}_{t-1}(\tau_{t});\bm{\theta}(\tau_{t}))$
$\displaystyle=$
$\displaystyle-\frac{1}{Th}\sum_{t=1}^{T}\widehat{\bm{K}}((\tau_{t}-\tau)/h)\otimes\left[\gradient_{\bm{\vartheta}}^{2}\mathcal{l}(\widetilde{\bm{x}}_{t}(\tau_{t}),\widetilde{\bm{z}}_{t-1}(\tau_{t});\bm{\theta}(\tau_{t})-u\mathbf{r}(\tau_{t}))\mathbf{r}(\tau_{t})\right]$

with some $u\in[0,1]$. Since $\gradient_{\bm{\vartheta}}^{2}\mathcal{l}$ is in
class $\mathcal{H}(3,\bm{\chi},M)$ by Lemma B.2, using Lemma B.8 and
$|\tau_{t}-\tau|\leq h$ yields

$\displaystyle\|\gradient_{\bm{\vartheta}}^{2}\mathcal{l}(\widetilde{\bm{x}}_{t}(\tau_{t}),\widetilde{\bm{z}}_{t-1}(\tau_{t});\bm{\theta}(\tau_{t})-u\mathbf{r}(\tau_{t}))-\gradient_{\bm{\vartheta}}^{2}\mathcal{l}(\widetilde{\bm{x}}_{t}(\tau),\widetilde{\bm{z}}_{t-1}(\tau);\bm{\theta}(\tau))\|_{1}=O(h).$

The above analyses plus Lemma B.5 reveal that

$\displaystyle\gradient_{\bm{\eta}}\widetilde{\mathcal{L}}_{\tau}(\bm{\eta}(\tau))-\frac{1}{Th}\sum_{t=1}^{T}\widehat{\bm{K}}((\tau_{t}-\tau)/h)\otimes\gradient_{\bm{\vartheta}}\mathcal{l}(\widetilde{\bm{x}}_{t}(\tau_{t}),\widetilde{\bm{z}}_{t-1}(\tau_{t});\bm{\theta}(\tau_{t}))$
$\displaystyle=$
$\displaystyle-\frac{1}{2}h^{2}\frac{1}{Th}\sum_{t=1}^{T}\widehat{\bm{K}}((\tau_{t}-\tau)/h)\otimes\left[\gradient_{\bm{\vartheta}}^{2}\mathcal{l}(\widetilde{\bm{x}}_{t}(\tau),\widetilde{\bm{z}}_{t-1}(\tau);\bm{\theta}(\tau))\cdot\bm{\theta}^{(2)}(\tau)\left(\frac{\tau_{t}-\tau}{h}\right)^{2}\right]+O_{P}(h^{3})$
$\displaystyle=$
$\displaystyle\frac{1}{2}h^{2}\int_{-\tau/h}^{(1-\tau)/h}K(u)[u^{2},u^{3}]^{\top}\mathrm{d}u\otimes\left(-\bm{\Sigma}(\tau)\bm{\theta}^{(2)}(\tau)\right)+O_{P}(h^{3}).$

By Lemmas B.4 and B.5, we have

$\displaystyle\gradient_{\bm{\eta}}\widetilde{\mathcal{L}}_{\tau}(\bm{\eta}(\tau))\to_{P}0\quad\text{and}\quad\sup_{\bm{\eta}}|\gradient_{\bm{\eta}}^{2}\mathcal{L}_{\tau}(\bm{\eta})-E(\gradient_{\bm{\eta}}^{2}\widetilde{\mathcal{L}}_{\tau}(\bm{\eta}))|\to_{P}0.$

Hence, we have
$\gradient_{\bm{\eta}}^{2}\mathcal{L}_{\tau}(\overline{\bm{\eta}})\to_{P}\bm{\Sigma}(\tau)$
and thus for any $\tau\in[h,1-h]$, as $Th^{7}\to 0$, we have

$\displaystyle\sqrt{Th}\left(\widehat{\bm{\theta}}(\tau)-\bm{\theta}(\tau)-\frac{1}{2}h^{2}\widetilde{c}_{2}\bm{\theta}^{(2)}(\tau)\right)$
$\displaystyle=$
$\displaystyle-\bm{\Sigma}^{-1}(\tau)\frac{1}{\sqrt{Th}}\sum_{t=1}^{T}K((\tau_{t}-\tau)/h)\gradient_{\bm{\vartheta}}\mathcal{l}(\widetilde{\bm{x}}_{t}(\tau_{t}),\widetilde{\bm{z}}_{t-1}(\tau_{t});\bm{\theta}(\tau_{t}))+o_{P}(1).$

In addition, by Lemma B.1, we have
$E\left(\gradient_{\bm{\vartheta}}\mathcal{l}(\widetilde{\mathbf{x}}_{t}(\tau),\widetilde{\mathbf{z}}_{t-1}(\tau);\bm{\theta}(\tau))\right)=0$.
To prove this theorem, by the Cramer-Wold device, it suffices to show that for
any unit vector $\mathbf{d}$,

$\displaystyle\frac{1}{\sqrt{Th}}\sum_{t=1}^{T}K((\tau_{t}-\tau)/h)\mathbf{d}^{\top}\gradient_{\bm{\vartheta}}\mathcal{l}(\widetilde{\bm{x}}_{t}(\tau_{t}),\widetilde{\bm{z}}_{t-1}(\tau_{t});\bm{\theta}(\tau_{t}))\to_{D}N\left(\mathbf{0},\widetilde{v}_{0}\mathbf{d}^{\top}\bm{\Omega}(\tau)\mathbf{d}\right).$

Note that
$\\{\gradient_{\bm{\vartheta}}\mathcal{l}(\widetilde{\bm{x}}_{t}(\tau_{t}),\widetilde{\bm{z}}_{t-1}(\tau_{t});\bm{\theta}(\tau_{t}))\\}_{t}$
is a sequence of martingale differences, we prove the asymptotic normality by
using the martingale central limit theorem Hall & Heyde (1980). We first
consider the convergence of conditional variance. Let
$w_{t}(u)=\frac{1}{\sqrt{Th}}K((\tau_{t}-\tau)/h)\mathbf{d}^{\top}\gradient_{\bm{\vartheta}}\mathcal{l}(\widetilde{\bm{x}}_{t}(u),\widetilde{\bm{z}}_{t-1}(u);\bm{\theta}(u))$.
By Lemma B.8, we have

$\displaystyle\sum_{t=1}^{T}\|w_{t}(\tau_{t})^{2}-w_{t}(\tau)^{2}\|_{1}$
$\displaystyle\leq$
$\displaystyle\frac{1}{Th}\sum_{t=1}^{T}K((\tau_{t}-\tau)/h)^{2}\|\gradient_{\bm{\vartheta}}\mathcal{l}(\widetilde{\bm{x}}_{t}(\tau_{t}),\widetilde{\bm{z}}_{t-1}(\tau_{t});\bm{\theta}(\tau_{t}))-\gradient_{\bm{\vartheta}}\mathcal{l}(\widetilde{\bm{x}}_{t}(\tau),\widetilde{\bm{z}}_{t-1}(\tau);\bm{\theta}(\tau))\|_{2}$
$\displaystyle\times
2\sup_{u}\|\gradient_{\bm{\vartheta}}\mathcal{l}(\widetilde{\bm{x}}_{t}(u),\widetilde{\bm{z}}_{t-1}(u);\bm{\theta}(u))\|_{2}$
$\displaystyle=$ $\displaystyle O(h)=o(1).$

In addition, by Proposition 2.1,
$\\{E\left[(\mathbf{d}^{\top}\gradient_{\bm{\vartheta}}\mathcal{l}(\widetilde{\bm{x}}_{t}(u),\widetilde{\bm{z}}_{t-1}(u);\bm{\theta}(u)))^{2}\mid\mathcal{F}_{t-1}\right]\\}_{t=1}^{T}$
is a sequence of stationary variables and thus we have

$\displaystyle\sum_{t=1}^{T}E\left(w_{t}(\tau)^{2}\mid\mathcal{F}_{t-1}\right)$
$\displaystyle=$
$\displaystyle\frac{1}{Th}\sum_{t=1}^{T}K((\tau_{t}-\tau)/h)^{2}E\left[(\mathbf{d}^{\top}\gradient_{\bm{\vartheta}}\mathcal{l}(\widetilde{\bm{x}}_{t}(u),\widetilde{\bm{z}}_{t-1}(u);\bm{\theta}(u)))^{2}\mid\mathcal{F}_{t-1}\right]$
$\displaystyle\to_{P}$
$\displaystyle\widetilde{v}_{0}\mathbf{d}^{\top}\bm{\Omega}(\tau)\mathbf{d}.$

We next verify the Lindeberg condition. The sum
$\sum_{t=1}^{T}E\left(w_{t}^{2}(\tau_{t})I(|w_{t}(\tau_{t})|>v)\right)$ is
bounded by

$\displaystyle
ME\left(\sup_{\tau}|\gradient_{\bm{\vartheta}}\mathcal{l}(\widetilde{\bm{x}}_{t}(\tau),\widetilde{\bm{z}}_{t-1}(\tau);\bm{\theta}(\tau))|^{2}I(\sup_{\tau}|\gradient_{\bm{\vartheta}}\mathcal{l}(\widetilde{\bm{x}}_{t}(\tau),\widetilde{\bm{z}}_{t-1}(\tau);\bm{\theta}(\tau))|>\sqrt{Th}v)\right),$

which converges to zero since
$\|\sup_{\tau}|\gradient_{\bm{\vartheta}}\mathcal{l}(\widetilde{\bm{x}}_{t}(\tau),\widetilde{\bm{z}}_{t-1}(\tau);\bm{\theta}(\tau))|\|_{2}<\infty$
by Lemma B.8.3. The asymptotic normality is then obtained.

The proof of the first result is now complete.

(2). For notation simplicity, we abbreviate
$\mathcal{l}(\mathbf{x},\mathbf{z};\bm{\vartheta}),\bm{\mu}(\mathbf{x},\mathbf{z};\bm{\vartheta}),\mathbf{M}(\mathbf{z};\bm{\vartheta})$
to $\mathcal{l},\bm{\mu},\mathbf{M}$ in what follows. Note that

$\displaystyle\mathrm{d}\mathcal{l}$ $\displaystyle=$
$\displaystyle(\mathbf{x}-\bm{\mu})^{\top}\mathbf{M}^{-1}\mathrm{d}\bm{\mu}-\frac{1}{2}(\mathbf{x}-\bm{\mu})^{\top}\mathrm{d}\mathbf{M}^{-1}(\mathbf{x}-\bm{\mu})-\frac{1}{2}\mathrm{tr}\\{\mathbf{M}^{-1}\mathrm{d}\mathbf{M}\\}$
$\displaystyle=$
$\displaystyle(\mathbf{x}-\bm{\mu})^{\top}\mathbf{M}^{-1}\frac{\partial\bm{\mu}}{\partial\bm{\vartheta}^{\top}}\mathrm{d}\bm{\vartheta}+\frac{1}{2}((\mathbf{x}-\bm{\mu})^{\top}\mathbf{M}^{-1}\otimes(\mathbf{x}-\bm{\mu})^{\top}\mathbf{M}^{-1})\frac{\partial\mathrm{vec}(\mathbf{M})}{\partial\bm{\vartheta}^{\top}}\mathrm{d}\bm{\vartheta}$
$\displaystyle-\frac{1}{2}\mathrm{vec}(\mathbf{M}^{-1})^{\top}\frac{\partial\mathrm{vec}(\mathbf{M})}{\partial\bm{\vartheta}^{\top}}\mathrm{d}\bm{\vartheta}.$

Hence, we have

$\displaystyle\frac{\partial\mathcal{l}}{\partial\bm{\vartheta}}\frac{\partial\mathcal{l}}{\partial\bm{\vartheta}^{\top}}$
$\displaystyle=$
$\displaystyle\frac{\partial\bm{\mu}^{\top}}{\partial\bm{\vartheta}}\mathbf{M}^{-1}(\mathbf{x}-\bm{\mu})(\mathbf{x}-\bm{\mu})^{\top}\mathbf{M}^{-1}\frac{\partial\bm{\mu}}{\partial\bm{\vartheta}^{\top}}+\frac{1}{4}\frac{\partial\mathrm{vec}(\mathbf{M})^{\top}}{\partial\bm{\vartheta}}\mathrm{vec}(\mathbf{M}^{-1})\mathrm{vec}(\mathbf{M}^{-1})^{\top}\frac{\partial\mathrm{vec}(\mathbf{M})}{\partial\bm{\vartheta}^{\top}}$
(A.2.4)
$\displaystyle+\frac{1}{4}\frac{\partial\mathrm{vec}(\mathbf{M})^{\top}}{\partial\bm{\vartheta}}[\mathbf{M}^{-1}(\mathbf{x}-\bm{\mu})(\mathbf{x}-\bm{\mu})^{\top}\mathbf{M}^{-1}\otimes\mathbf{M}^{-1}(\mathbf{x}-\bm{\mu})(\mathbf{x}-\bm{\mu})^{\top}\mathbf{M}^{-1}]\frac{\partial\mathrm{vec}(\mathbf{M})}{\partial\bm{\vartheta}^{\top}}$
$\displaystyle+\frac{1}{2}\frac{\partial\bm{\mu}^{\top}}{\partial\bm{\vartheta}}[\mathbf{M}^{-1}(\mathbf{x}-\bm{\mu})(\mathbf{x}-\bm{\mu})^{\top}\mathbf{M}^{-1}\otimes(\mathbf{x}-\bm{\mu})^{\top}\mathbf{M}^{-1}]\frac{\partial\mathrm{vec}(\mathbf{M})}{\partial\bm{\vartheta}^{\top}}$
$\displaystyle-\frac{1}{2}\frac{\partial\bm{\mu}^{\top}}{\partial\bm{\vartheta}}\mathbf{M}^{-1}(\mathbf{x}-\bm{\mu})\mathrm{vec}(\mathbf{M}^{-1})^{\top}\frac{\partial\mathrm{vec}(\mathbf{M})}{\partial\bm{\vartheta}^{\top}}$
$\displaystyle+\frac{1}{2}\frac{\partial\bm{\mu}^{\top}}{\partial\bm{\vartheta}}[(\mathbf{x}-\bm{\mu})^{\top}\mathbf{M}^{-1}\otimes\mathbf{M}^{-1}(\mathbf{x}-\bm{\mu})(\mathbf{x}-\bm{\mu})^{\top}\mathbf{M}^{-1}]\frac{\partial\mathrm{vec}(\mathbf{M})}{\partial\bm{\vartheta}^{\top}}$
$\displaystyle-\frac{1}{4}\frac{\partial\mathrm{vec}(\mathbf{M})^{\top}}{\partial\bm{\vartheta}}[\mathbf{M}^{-1}(\mathbf{x}-\bm{\mu})\otimes\mathbf{M}^{-1}(\mathbf{x}-\bm{\mu})]\mathrm{vec}(\mathbf{M}^{-1})^{\top}\frac{\partial\mathrm{vec}(\mathbf{M})}{\partial\bm{\vartheta}^{\top}}$
$\displaystyle-\frac{1}{2}\frac{\partial\mathrm{vec}(\mathbf{M})^{\top}}{\partial\bm{\vartheta}}\mathrm{vec}(\mathbf{M}^{-1})(\mathbf{x}-\bm{\mu})^{\top}\mathbf{M}^{-1}\frac{\partial\bm{\mu}}{\partial\bm{\vartheta}^{\top}}$
$\displaystyle-\frac{1}{4}\frac{\partial\mathrm{vec}(\mathbf{M})^{\top}}{\partial\bm{\vartheta}}\mathrm{vec}(\mathbf{M}^{-1})[(\mathbf{x}-\bm{\mu})^{\top}\mathbf{M}^{-1}\otimes(\mathbf{x}-\bm{\mu})^{\top}\mathbf{M}^{-1}]\frac{\partial\mathrm{vec}(\mathbf{M})}{\partial\bm{\vartheta}^{\top}}.$

In addition, if $\bm{\varepsilon}_{t}$ is normal distributed, we have
$E(\bm{\varepsilon_{t}}\bm{\varepsilon_{t}}^{\top}\otimes\bm{\varepsilon_{t}}\bm{\varepsilon_{t}}^{\top})=2\mathbf{N}_{m}+\mathrm{vec}(\mathbf{I}_{m})\mathrm{vec}(\mathbf{I}_{m})^{\top}$
and
$E(\mathbf{c}\bm{\varepsilon_{t}}^{\top}\otimes\bm{\varepsilon_{t}}\bm{\varepsilon_{t}}^{\top})=\mathbf{0}$,
where $\mathbf{c}$ is independent of $\bm{\varepsilon_{t}}$,
$2\mathbf{N}_{m}=\mathbf{I}_{m^{2}}+\mathbf{K}_{mm}$ and $\mathbf{K}_{mm}$ is
a commutation matrix. By (A.2.4), if $\bm{\varepsilon_{t}}$ is normal
distributed, we have

$\displaystyle\bm{\Omega}(\tau)=E\left(\gradient_{\bm{\vartheta}}\mathcal{l}(\widetilde{\mathbf{x}}_{t}(\tau),\widetilde{\mathbf{z}}_{t-1}(\tau);\bm{\vartheta}(\tau))\cdot\gradient_{\bm{\vartheta}}\mathcal{l}(\widetilde{\mathbf{x}}_{t}(\tau),\widetilde{\mathbf{z}}_{t-1}(\tau);\bm{\vartheta}(\tau))^{\top}\right)$
$\displaystyle=$ $\displaystyle
E\left(\frac{\partial\bm{\mu}(\widetilde{\mathbf{x}}_{t}(\tau),\widetilde{\mathbf{z}}_{t-1}(\tau);\bm{\vartheta}(\tau))^{\top}}{\partial\bm{\vartheta}}\mathbf{M}^{-1}(\widetilde{\mathbf{x}}_{t}(\tau),\widetilde{\mathbf{z}}_{t-1}(\tau);\bm{\vartheta}(\tau))\frac{\partial\bm{\mu}(\widetilde{\mathbf{x}}_{t}(\tau),\widetilde{\mathbf{z}}_{t-1}(\tau);\bm{\vartheta}(\tau))}{\partial\bm{\vartheta}^{\top}}\right)$
$\displaystyle+\frac{1}{2}E\left(\frac{\partial\mathrm{vec}(\mathbf{M}(\widetilde{\mathbf{x}}_{t}(\tau),\widetilde{\mathbf{z}}_{t-1}(\tau);\bm{\vartheta}(\tau)))^{\top}}{\partial\bm{\vartheta}}[\mathbf{M}^{-1}(\widetilde{\mathbf{x}}_{t}(\tau),\widetilde{\mathbf{z}}_{t-1}(\tau);\bm{\vartheta}(\tau))\otimes\mathbf{M}^{-1}(\widetilde{\mathbf{x}}_{t}(\tau),\widetilde{\mathbf{z}}_{t-1}(\tau);\bm{\vartheta}(\tau))]\right.$
$\displaystyle\left.\times\frac{\partial\mathrm{vec}(\mathbf{M}(\widetilde{\mathbf{x}}_{t}(\tau),\widetilde{\mathbf{z}}_{t-1}(\tau);\bm{\vartheta}(\tau)))}{\partial\bm{\vartheta}^{\top}}\right).$

Next, consider the Hessian matrix.

$\displaystyle\mathrm{d}^{2}\mathcal{l}$ $\displaystyle=$
$\displaystyle-\mathrm{d}\bm{\vartheta}^{\top}\frac{\partial\bm{\mu}^{\top}}{\partial\bm{\vartheta}}\mathbf{M}^{-1}\frac{\partial\bm{\mu}}{\partial\bm{\vartheta}^{\top}}\mathrm{d}\bm{\vartheta}-\mathrm{d}\bm{\vartheta}^{\top}\frac{\partial\mathrm{vec}(\mathbf{M})^{\top}}{\partial\bm{\vartheta}}(\mathbf{M}^{-1}(\mathbf{x}-\bm{\mu})\otimes\mathbf{M}^{-1})\frac{\partial\bm{\mu}}{\partial\bm{\vartheta}^{\top}}\mathrm{d}\bm{\vartheta}$
(A.2.5)
$\displaystyle+\mathrm{d}\bm{\vartheta}^{\top}\frac{\partial\mathrm{vec}(\frac{\partial\bm{\mu}^{\top}}{\partial\bm{\vartheta}})^{\top}}{\partial\bm{\vartheta}}(\mathbf{M}^{-1}(\mathbf{x}-\bm{\mu})\otimes\mathbf{I}_{m})\frac{\partial\bm{\mu}}{\partial\bm{\vartheta}^{\top}}\mathrm{d}\bm{\vartheta}$
$\displaystyle+\frac{1}{2}\mathrm{d}\bm{\vartheta}^{\top}\frac{\partial\mathrm{vec}(\mathbf{M})^{\top}}{\partial\bm{\vartheta}}[\mathbf{M}^{-1}\otimes\mathbf{M}^{-1}]\frac{\partial\mathrm{vec}(\mathbf{M})}{\partial\bm{\vartheta}^{\top}}\mathrm{d}\bm{\vartheta}$
$\displaystyle-\frac{1}{2}\mathrm{d}\bm{\vartheta}^{\top}\frac{\partial\mathrm{vec}(\frac{\partial\mathrm{vec}(\mathbf{M})^{\top}}{\partial\bm{\vartheta}})^{\top}}{\partial\bm{\vartheta}}(\mathrm{vec}(\mathbf{M}^{-1})^{\top}\otimes\mathbf{I}_{d})\mathrm{d}\bm{\vartheta}$
$\displaystyle-\frac{1}{2}\mathrm{d}\bm{\vartheta}^{\top}\frac{\partial\mathrm{vec}(\mathbf{M})^{\top}}{\partial\bm{\vartheta}}[\mathbf{M}^{-1}(\mathbf{x}-\bm{\mu})(\mathbf{x}-\bm{\mu})^{\top}\mathbf{M}^{-1}\otimes\mathbf{M}^{-1}]\frac{\partial\mathrm{vec}(\mathbf{M})}{\partial\bm{\vartheta}^{\top}}\mathrm{d}\bm{\vartheta}$
$\displaystyle-\frac{1}{2}\mathrm{d}\bm{\vartheta}^{\top}\frac{\partial\bm{\mu}^{\top}}{\partial\bm{\vartheta}}(\mathbf{M}^{-1}(\mathbf{x}-\bm{\mu})\otimes\mathbf{M}^{-1})\frac{\partial\mathrm{vec}(\mathbf{M})}{\partial\bm{\vartheta}^{\top}}\mathrm{d}\bm{\vartheta}$
$\displaystyle-\frac{1}{2}\mathrm{d}\bm{\vartheta}^{\top}\frac{\partial\bm{\mu}^{\top}}{\partial\bm{\vartheta}}(\mathbf{M}^{-1}\otimes\mathbf{M}^{-1}(\mathbf{x}-\bm{\mu}))\frac{\partial\mathrm{vec}(\mathbf{M})}{\partial\bm{\vartheta}^{\top}}\mathrm{d}\bm{\vartheta}$
$\displaystyle-\frac{1}{2}\mathrm{d}\bm{\vartheta}^{\top}\frac{\partial\mathrm{vec}(\mathbf{M})^{\top}}{\partial\bm{\vartheta}}[\mathbf{M}^{-1}\otimes\mathbf{M}^{-1}(\mathbf{x}-\bm{\mu})(\mathbf{x}-\bm{\mu})^{\top}\mathbf{M}^{-1}]\frac{\partial\mathrm{vec}(\mathbf{M})}{\partial\bm{\vartheta}^{\top}}\mathrm{d}\bm{\vartheta}$
$\displaystyle+\frac{1}{2}\mathrm{d}\bm{\vartheta}^{\top}\frac{\partial\mathrm{vec}(\frac{\partial\mathrm{vec}(\mathbf{M})^{\top}}{\partial\bm{\vartheta}})^{\top}}{\partial\bm{\vartheta}}(\mathrm{vec}(\mathbf{M}^{-1}(\mathbf{x}-\bm{\mu})(\mathbf{x}-\bm{\mu})^{\top}\mathbf{M}^{-1})\otimes\mathbf{I}_{d})\mathrm{d}\bm{\vartheta}.$

By (A.2.5), if $\bm{\varepsilon_{t}}$ is normal distributed, we have

$\displaystyle\bm{\Sigma}(\tau)=E\left(\gradient_{\bm{\vartheta}}^{2}\mathcal{l}(\widetilde{\mathbf{x}}_{t}(\tau),\widetilde{\mathbf{z}}_{t-1}(\tau);\bm{\vartheta}(\tau))\right)$
$\displaystyle=$
$\displaystyle-E\left(\frac{\partial\bm{\mu}(\widetilde{\mathbf{x}}_{t}(\tau),\widetilde{\mathbf{z}}_{t-1}(\tau);\bm{\vartheta}(\tau))^{\top}}{\partial\bm{\vartheta}}\mathbf{M}^{-1}(\widetilde{\mathbf{x}}_{t}(\tau),\widetilde{\mathbf{z}}_{t-1}(\tau);\bm{\vartheta}(\tau))\frac{\partial\bm{\mu}(\widetilde{\mathbf{x}}_{t}(\tau),\widetilde{\mathbf{z}}_{t-1}(\tau);\bm{\vartheta}(\tau))}{\partial\bm{\vartheta}^{\top}}\right)$
$\displaystyle-\frac{1}{2}E\left(\frac{\partial\mathrm{vec}(\mathbf{M}(\widetilde{\mathbf{x}}_{t}(\tau),\widetilde{\mathbf{z}}_{t-1}(\tau);\bm{\vartheta}(\tau)))^{\top}}{\partial\bm{\vartheta}}[\mathbf{M}^{-1}(\widetilde{\mathbf{x}}_{t}(\tau),\widetilde{\mathbf{z}}_{t-1}(\tau);\bm{\vartheta}(\tau))\otimes\mathbf{M}^{-1}(\widetilde{\mathbf{x}}_{t}(\tau),\widetilde{\mathbf{z}}_{t-1}(\tau);\bm{\vartheta}(\tau))]\right.$
$\displaystyle\left.\times\frac{\partial\mathrm{vec}(\mathbf{M}(\widetilde{\mathbf{x}}_{t}(\tau),\widetilde{\mathbf{z}}_{t-1}(\tau);\bm{\vartheta}(\tau)))}{\partial\bm{\vartheta}^{\top}}\right).$

Then we have $\bm{\Omega}(\tau)=-\bm{\Sigma}(\tau)$ if $\bm{\varepsilon_{t}}$
is normal distributed. The proof is now complete. ∎

###### Proof of Corollary 2.1.

By Lemma B.5 (2) and the proof of Theorem 2.1, we have

$\sup_{\tau\in[0,1]}|\widehat{\bm{\eta}}(\tau)-\bm{\eta}(\tau)|=O_{P}((Th)^{-1/2}h^{-1/2}(\log
T)^{1/2}).$

In addition, applying Lemma B.3 (4), Lemma B.5 (2) and Lemma B.3 (2) to
$g=\gradient_{\bm{\vartheta}}^{2}\mathcal{l}$, we have

$\sup_{\tau\in[0,1]}|\widehat{\bm{\Sigma}}(\tau)-\bm{\Sigma}(\tau)|=O_{P}((Th)^{-1/2}h^{-1/2}(\log
T)^{1/2}+h)=o_{P}(1)$

as $h(\log T)^{2}\to 0$ and
$\gradient_{\bm{\vartheta}}^{2}\mathcal{l}\in\mathcal{H}(3,\bm{\chi},M)$.

For $\widehat{\bm{\Omega}}(\tau)$, as
$\gradient_{\bm{\theta}}\mathcal{l}(\gradient_{\bm{\theta}}\mathcal{l})^{\top}\in\mathcal{H}(6,\bm{\chi},M)$,
here we use a different argument to prove the result, which leads to weaker
moment conditions.

By Lemma B.3.4 and Lemma B.8.2, we have

$\displaystyle\widehat{\bm{\Omega}}(\tau)-(Th)^{-1}\sum_{t=1}^{T}\gradient_{\bm{\theta}}\mathcal{l}(\widetilde{\mathbf{x}}_{t}(\tau_{t}),\widetilde{\mathbf{z}}_{t-1}(\tau_{t});\bm{\theta}(\tau))\gradient_{\bm{\theta}}\mathcal{l}(\widetilde{\mathbf{x}}_{t}(\tau_{t}),\widetilde{\mathbf{z}}_{t-1}(\tau_{t});\bm{\theta}(\tau))^{\top}\widehat{K}\left(\frac{\tau_{t}-\tau}{h}\right)$
$\displaystyle=$ $\displaystyle O_{P}((Th)^{-1/2}h^{-1/2}(\log
T)^{1/2}+(Th)^{-1})=o_{P}(1),$

where
$\widehat{K}\left(\frac{\tau_{t}-\tau}{h}\right)=K\left(\frac{\tau_{t}-\tau}{h}\right)/\left(T^{-1}\sum_{t=1}^{T}K\left(\frac{\tau_{t}-\tau}{h}\right)\right)$.

Define
$g(\widetilde{\mathbf{y}}_{t}(\tau),\bm{\theta}(\tau)):=\gradient_{\bm{\vartheta}}\mathcal{l}(\widetilde{\mathbf{x}}_{t}(\tau),\widetilde{\mathbf{z}}_{t-1}(\tau);\bm{\theta}(\tau))\gradient_{\bm{\vartheta}}\mathcal{l}(\widetilde{\mathbf{x}}_{t}(\tau),\widetilde{\mathbf{z}}_{t-1}(\tau);\bm{\theta}(\tau))^{\top}$.
By Lemma B.4.1, we have
$\sup_{\tau\in[0,1]}\delta_{q/2}^{\sup_{\bm{\vartheta}}|g(\widetilde{\mathbf{y}}_{t}(\tau),\bm{\vartheta})|}(j)=O(j^{-(3/2+s)})$.

Define
$\mathbf{S}_{T}(\tau)=\sum_{t=1}^{T}\left[g(\widetilde{\mathbf{y}}_{t}(\tau_{t}),\bm{\theta}(\tau))-E(g(\widetilde{\mathbf{y}}_{t}(\tau_{t}),\bm{\theta}(\tau)))\right]\widehat{K}\left(\frac{\tau_{t}-\tau}{h}\right)$
and

$\mathbf{S}_{k,T}=\sum_{t=1}^{k}\left[g(\widetilde{\mathbf{y}}_{t}(\tau_{t}),\bm{\theta}(\tau))-E(g(\widetilde{\mathbf{y}}_{t}(\tau_{t}),\bm{\theta}(\tau)))\right].$

By partial summation, we have

$\mathbf{S}_{T}(\tau)=\sum_{t=1}^{T-1}\left[\widehat{K}\left(\frac{\tau_{t}-\tau}{h}\right)-\widehat{K}\left(\frac{\tau_{t+1}-\tau}{h}\right)\right]\mathbf{S}_{t,T}+\widehat{K}\left(\frac{1-\tau}{h}\right)\mathbf{S}_{T,T}.$

Hence, we have $\sup_{\tau\in[0,1]}|\mathbf{S}_{T}(\tau)|\leq
M\max_{t}|\mathbf{S}_{t,T}|$. Note that
$\\{\mathcal{P}_{t-j}g(\widetilde{\mathbf{y}}_{t}(\tau_{t}),\bm{\vartheta})\\}_{t}$
forms a sequence of martingale differences. By the Doob’s $\mathbb{L}^{q}$
maximal inequality, Burkholder’s inequality and the elementary inequality
$(\sum_{i}|a_{i}|)^{q}\leq\sum_{i}|a_{i}|^{q}$ for $0<q\leq 1$, we obtain that

$\displaystyle\|\max_{t}|\mathbf{S}_{t,T}|\|_{q/2}$ $\displaystyle\leq$
$\displaystyle\sum_{l=0}^{\infty}\|\max_{t=1,...,T}|\sum_{s=1}^{t}\mathcal{P}_{s-l}g(\widetilde{\mathbf{y}}_{s}(\tau_{s}),\bm{\theta}(\tau))|\|_{q/2}$
$\displaystyle\leq$
$\displaystyle\sum_{l=0}^{\infty}\frac{q/2}{q/2-1}\|\sum_{s=1}^{T}\mathcal{P}_{s-l}g(\widetilde{\mathbf{y}}_{s}(\tau_{s}),\bm{\theta}(\tau))\|_{q/2}$
$\displaystyle\leq$
$\displaystyle\sum_{l=0}^{\infty}\frac{q/2}{(q/2-1)^{2}}\left[E\left(\sum_{s=1}^{T}\left(\mathcal{P}_{s-l}g(\widetilde{\mathbf{y}}_{s}(\tau_{s}),\bm{\theta}(\tau))\right)^{2}\right)^{q/4}\right]$
$\displaystyle\leq$
$\displaystyle\frac{q/2}{(q/2-1)^{2}}\sum_{l=0}^{\infty}\left(\sum_{s=1}^{T}\|\mathcal{P}_{s-l}g(\widetilde{\mathbf{y}}_{s}(\tau_{s}),\bm{\theta}(\tau))\|_{q/2}^{q/2}\right)^{2/q}$
$\displaystyle\leq$
$\displaystyle\frac{q/2}{(q/2-1)^{2}}T^{2/q}\sum_{l=0}^{\infty}\sup_{\tau\in[0,1]}\delta_{q/2}^{\sup_{\bm{\vartheta}}|g(\widetilde{\mathbf{y}}_{t}(\tau),\bm{\vartheta})|}(l)$

which shows that
$\sup_{\tau\in[0,1]}|\frac{1}{Th}\mathbf{S}_{T}(\tau)|=O_{P}(T^{2/q-1}h^{-1})=o_{P}(1)$.
The result then follows directly by Lemma B.3.2. ∎

###### Proof of Theorem 2.2.

We prove this theorem by applying Lemma B.9 to the weak Bahadur representation
of $\widehat{\bm{\theta}}(\tau)$ given in Lemma B.7.

By Lemma B.7, we have

$\displaystyle\sup_{\tau\in[h,1-h]}$
$\displaystyle\left|\mathbf{C}(\widehat{\bm{\theta}}(\tau)-\bm{\theta}(\tau))-\frac{1}{2}h^{2}\widetilde{c}_{2}\mathbf{C}\bm{\theta}^{(2)}(\tau)\right.$
(A.2.6)
$\displaystyle\left.-\frac{1}{T}\sum_{t=1}^{T}(-\mathbf{C}\bm{\Sigma}^{-1}(\tau))\gradient_{\bm{\vartheta}}\mathcal{l}(\widetilde{\mathbf{x}}_{t}(\tau_{t}),\widetilde{\mathbf{z}}_{t-1}(\tau_{t});\bm{\theta}(\tau_{t}))K_{h}(\tau_{t}-\tau)\right|$
$\displaystyle=$ $\displaystyle
O_{P}(\gamma_{T}+\beta_{T}h^{2}+h^{3}+(Th)^{-1})=o_{P}((Th\log T)^{-1/2})$

as $Th^{7}\log T\to 0$ and $Th^{2}/(\log T)^{4}\to\infty$. In addition, by
Lemmas B.8, B.4.2 and B.9, we have

$\displaystyle\lim_{T\to\infty}\mathrm{Pr}$
$\displaystyle\left(\sqrt{\frac{Th}{\widetilde{v}_{0}}}\sup_{\tau\in[h,1-h]}\left|\bm{\Sigma}_{\mathbf{C}}^{-1/2}(\tau)\frac{1}{T}\sum_{t=1}^{T}(-\mathbf{C}\bm{\Sigma}^{-1}(\tau))\gradient_{\bm{\vartheta}}\mathcal{l}(\widetilde{\mathbf{x}}_{t}(\tau_{t}),\widetilde{\mathbf{z}}_{t-1}(\tau_{t});\bm{\theta}(\tau_{t}))K_{h}(\tau_{t}-\tau)\right|\right.$
(A.2.7)
$\displaystyle\left.-B(1/h)\leq\frac{u}{\sqrt{2\log(1/h)}}\right)=\exp(-2\exp(-u)).$

By (A.2.6) and (A.2.7), the proof is complete. ∎

###### Proof of Corollary 2.2.

By the proof of Lemma B.7, we have

$\displaystyle\sup_{\tau\in[0,1]}\left|\widehat{\bm{\eta}}(\tau)-\bm{\eta}(\tau)-\frac{1}{2}h^{2}\left[\begin{matrix}\widetilde{c}_{0,h}(\tau)&\widetilde{c}_{1,h}(\tau)\\\
\widetilde{c}_{1,h}(\tau)&\widetilde{c}_{2,h}(\tau)\end{matrix}\right]^{-1}\left[\begin{matrix}\widetilde{c}_{2,h}(\tau)\\\
\widetilde{c}_{3,h}(\tau)\end{matrix}\right]\otimes\bm{\theta}^{(2)}(\tau)\right.$
$\displaystyle\left.-\frac{1}{T}\sum_{t=1}^{T}K_{h}(\tau_{t}-\tau)\left[\begin{matrix}\widetilde{c}_{0,h}(\tau)&\widetilde{c}_{1,h}(\tau)\\\
\widetilde{c}_{1,h}(\tau)&\widetilde{c}_{2,h}(\tau)\end{matrix}\right]^{-1}\left[\begin{matrix}1\\\
\frac{\tau_{t}-\tau}{h}\end{matrix}\right]\otimes(-\bm{\Sigma}^{-1}(\tau))\gradient_{\bm{\vartheta}}\mathcal{l}(\widetilde{\bm{x}}_{t}(\tau_{t}),\widetilde{\bm{z}}_{t-1}(\tau_{t});\bm{\theta}(\tau_{t}))\right|$
$\displaystyle=$ $\displaystyle O_{P}((Th)^{-1/2}h^{3/2}(\log
T)^{1/2})+O(h^{3}).$

Hence, we have

$\displaystyle\sup_{\tau\in[0,1]}\left|\widehat{\bm{\theta}}(\tau)-\bm{\theta}(\tau)-\frac{1}{2}h^{2}b_{h}(\tau)\bm{\theta}^{(2)}(\tau)-\frac{1}{T}\sum_{t=1}^{T}(-\bm{\Sigma}^{-1}(\tau))\gradient_{\bm{\vartheta}}\mathcal{l}(\widetilde{\bm{x}}_{t}(\tau_{t}),\widetilde{\bm{z}}_{t-1}(\tau_{t});\bm{\theta}(\tau_{t}))\omega_{t,h}(\tau)\right|$
$\displaystyle=$ $\displaystyle O_{P}((Th)^{-1/2}h^{3/2}(\log
T)^{1/2})+O(h^{3}).$

By Lemma B.10, there exists i.i.d. $k$-dimensional standard normal variables
$\mathbf{v}_{1},...,\mathbf{v}_{T}$ such that

$\displaystyle\sup_{\tau\in[0,1]}\left|\frac{1}{Th}\sum_{t=1}^{T}\omega_{t,h}(\tau)(\gradient_{\bm{\theta}}\mathcal{l}(\widetilde{\bm{x}}_{t}(\tau_{t}),\widetilde{\bm{z}}_{t-1}(\tau_{t});\bm{\theta}(\tau_{t}))-\bm{\Omega}^{1/2}(\tau_{t})\mathbf{v}_{t})\right|$
$\displaystyle=$ $\displaystyle
O_{P}\left(\frac{T^{\frac{q(s+3)-4}{2q(2s+3)-4}}(\log
T)^{\frac{2(s+1)(q+1)}{q(2s+3)-2}}}{Th}\right)=O_{P}\left(\frac{(\log
T)^{2}(hT^{\frac{qs+2}{q(2s+3)-2}})^{-1/2}}{(Th)^{1/2}(\log T)^{1/2}}\right)$
$\displaystyle=$ $\displaystyle O_{P}\left(\frac{(\log
T)^{2}(hT^{\nu})^{-1/2}}{(Th\log T)^{1/2}}\right)$

with $\nu=\frac{qs+2}{q(2s+3)-2}$. Since $\bm{\Omega}(\tau)$ is Lipschitz
continuous and $\\{\mathbf{v}_{t}\\}_{t=1}^{T}$ is a sequence of i.i.d. normal
variables, we have

$\displaystyle\sup_{\tau\in[0,1]}\left|\frac{1}{Th}\sum_{t=1}^{T}\omega_{t,h}(\tau)(\bm{\Omega}^{1/2}(\tau)-\bm{\Omega}^{1/2}(\tau_{t}))\mathbf{v}_{t}\right|$
$\displaystyle=$ $\displaystyle O_{P}\left(\frac{h(\log
T)^{1/2}}{(Th)^{1/2}}\right)=O_{P}\left(\frac{h\log T}{(Th\log
T)^{1/2}}\right).$

Combining the above analyses, we then complete the proof.

∎

###### Proof of Proposition 2.3.

Note that in this case
$\mathcal{l},\gradient\mathcal{l},\gradient^{2}\mathcal{l}$ is in class
$\mathcal{H}(2,\bm{\chi},M)$ as
$\mathbf{H}(\mathbf{z};\bm{\theta})\equiv\mathbf{H}(\mathbf{0};\bm{\theta})$
by Lemma B.2. Hence, we only need that the innovation process has $4+s$
moments for some $s>0$ compared to $6+s$ moments needed in Theorem 2.2.

Consider Assumptions 1–2 first. For notation simplicity, we ignore the time-
varying intercept, and rewrite model (2.11) as

$\widetilde{\mathbf{y}}_{t}(\tau)=\bm{\Gamma}(\tau)\widetilde{\mathbf{y}}_{t-1}(\tau)+\widetilde{\mathbf{u}}_{t}(\tau),$

where
$\widetilde{\mathbf{y}}_{t}(\tau)=[\widetilde{\bm{\eta}}_{t}^{\top}(\tau),...,\widetilde{\bm{\eta}}_{t-q+1}^{\top}(\tau),\widetilde{\mathbf{x}}_{t}^{\top}(\tau),...,\widetilde{\mathbf{x}}_{t-p+1}^{\top}(\tau)]^{\top}$,
$\widetilde{\mathbf{u}}_{t}(\tau)=[\widetilde{\mathbf{x}}_{t}^{\top}(\tau),\mathbf{0}_{m(q-1)\times
1}^{\top},\widetilde{\mathbf{x}}_{t}^{\top}(\tau),\mathbf{0}_{m(p-1)\times
1}^{\top}]^{\top}$ and

$\bm{\Psi}(\tau)=\left[\begin{array}[]{cccc
cccc}-\mathbf{B}_{1}(\tau)&\cdots&-\mathbf{B}_{q-1}(\tau)&-\mathbf{B}_{q}(\tau)&-\mathbf{A}_{1}(\tau)&\cdots&-\mathbf{A}_{p-1}(\tau)&-\mathbf{A}_{p}(\tau)\\\
\mathbf{I}_{m}&\cdots&\mathbf{0}_{m}&\mathbf{0}_{m}&\mathbf{0}_{m}&\cdots&\mathbf{0}_{m}&\mathbf{0}_{m}\\\
\vdots&\ddots&\vdots&\vdots&\vdots&\ddots&\vdots&\vdots\\\
\mathbf{0}_{m}&\cdots&\mathbf{I}_{m}&\mathbf{0}_{m}&\mathbf{0}_{m}&\cdots&\mathbf{0}_{m}&\mathbf{0}_{m}\\\
\lx@intercol\hfil\hbox{\multirowsetup$\mathbf{0}$}\hfil\lx@intercol&\mathbf{0}_{m}&\cdots&\mathbf{0}_{m}&\mathbf{0}_{m}\\\
\lx@intercol\hfil\hfil\lx@intercol&\mathbf{I}_{m}&\cdots&\mathbf{0}_{m}&\mathbf{0}_{m}\\\
\lx@intercol\hfil\hfil\lx@intercol&\vdots&\ddots&\vdots&\vdots\\\
\lx@intercol\hfil\hfil\lx@intercol&\mathbf{0}_{m}&\cdots&\mathbf{I}_{m}&\mathbf{0}_{m}\\\
\end{array}\right].$

Let $\mathbf{J}=[\mathbf{I}_{m},\mathbf{0}_{m\times(m(p+q-1))}]$ and
$\mathbf{H}=[\mathbf{I}_{m},\mathbf{0}_{m(q-1)\times
1}^{\top},\mathbf{I}_{m},\mathbf{0}_{m(p-1)\times 1}^{\top}]^{\top}$, we have

$\displaystyle\widetilde{\bm{\eta}}_{t}(\tau)$ $\displaystyle=$
$\displaystyle\widetilde{\mathbf{x}}_{t}(\tau)+\sum_{j=1}^{\infty}(\mathbf{J}\bm{\Psi}^{j}(\tau)\mathbf{H})\widetilde{\mathbf{x}}_{t-j}(\tau)$
$\displaystyle\widetilde{\mathbf{x}}_{t}(\tau)$ $\displaystyle=$
$\displaystyle\sum_{j=1}^{\infty}(-\mathbf{J}\bm{\Psi}^{j}(\tau)\mathbf{H})\widetilde{\mathbf{x}}_{t-j}(\tau)+\widetilde{\bm{\eta}}_{t}(\tau)$

and thus $\bm{\Gamma}_{j}(\tau)=-\mathbf{J}\bm{\Psi}^{j}(\tau)\mathbf{H}$.
Then Assumption 1 is automatically met if
$\sum_{j=1}^{\infty}|\bm{\Gamma}_{j}(\tau)|<1$. By using the property of block
matrix determinants and $\mathrm{det}(\mathbf{B}_{\tau}(L))\neq 0$ for all
$|L|\leq 1$ (this implies the maximum eigenvalue of left upper $mq\times mq$
matrix in $\bm{\Gamma}(\tau)$ is less than $1$), it can be shown that the
maximum eigenvalue of $\bm{\Gamma}(\tau)$, denoted by $\rho$, is less than 1
uniformly over $\tau\in[0,1]$. Hence, we have
$\alpha_{j}(\bm{\theta}(\tau))=|\bm{\Gamma}_{j}(\tau)|=O(\rho^{j})$ and
$\beta_{j}(\bm{\theta}(\tau))=0$. In addition,
$|\bm{\Psi}^{j}-\bm{\Psi}^{\prime
j}|=|\sum_{i=1}^{j-1}\bm{\Psi}^{i}(\bm{\Psi}-\bm{\Psi}^{\prime})\bm{\Psi}^{\prime
j-1-i}|=|\bm{\Psi}-\bm{\Psi}^{\prime}|O(\rho^{j-1})$. Then Assumption 2 is
met.

However, by using techniques which are more specific to the VARMA models, the
condition $\sum_{j=1}^{\infty}|\bm{\Gamma}_{j}(\tau)|<1$ can be weakened to
$\mathrm{det}(\mathbf{A}_{\tau}(L))\neq 0$ for all $|L|\leq 1$. Similar to the
above analysis, we have
$\widetilde{\mathbf{x}}_{t}(\tau)=\sum_{j=0}^{\infty}\bm{\Phi}_{j}(\tau)\widetilde{\bm{\eta}}_{t-j}(\tau)$
with $|\bm{\Phi}_{j}(\tau)|=O(\rho^{j})$ as
$\mathrm{det}(\mathbf{A}_{\tau}(L))\neq 0$ for all $|L|\leq 1$, which implies
that $\|\widetilde{\mathbf{x}}_{t}(\tau)\|_{r}<\infty$ and
$\delta_{r}^{\widetilde{\mathbf{x}}(\tau)}(k)=O(\rho^{k})$.

For the identification conditions stated in Assumption 3, it is well known
that the final form or echelon form is enough to ensure the uniqueness of the
VARMA representation.

For verifying Assumption 4, one need the derivatives of $\bm{\Gamma}_{j}$.
Define
$\bm{\alpha}=-\mathrm{vec}(\mathbf{B}_{1},...,\mathbf{B}_{q},\mathbf{A}_{1},...,\mathbf{A}_{p})$.
Note that
$\mathrm{d}\mathrm{vec}(\bm{\Psi})=(\mathbf{I}_{m(p+q)}\otimes\mathbf{J}^{\top})\mathrm{d}\bm{\alpha}$
and
$\mathrm{d}\mathrm{vec}(\bm{\Psi}^{j})=(\bm{\Psi}^{\top}\otimes\mathbf{I}_{m(p+q)})\mathrm{d}\mathrm{vec}(\bm{\Psi}^{j-1})+(\mathbf{I}_{m(p+q)}\otimes\bm{\Psi}^{j-1}\mathbf{J}^{\top})\mathrm{d}\bm{\alpha}$,
it is easy to show that

$\frac{\partial\mathrm{vec}(\bm{\Gamma}_{j})}{\partial\bm{\alpha}^{\top}}=-\sum_{i=0}^{j-1}\mathbf{H}^{\top}(\bm{\Psi}^{\top})^{j-1-i}\otimes\mathbf{J}\bm{\Psi}^{j}(\tau)\mathbf{J}^{\top}.$

Hence, we have
$|\frac{\partial\mathrm{vec}(\bm{\Gamma}_{j})}{\partial\bm{\alpha}^{\top}}|=O(\rho^{j-1})$
and
$|\frac{\partial\mathrm{vec}(\bm{\Gamma}_{j})}{\partial\bm{\alpha}^{\top}}-\frac{\partial\mathrm{vec}(\bm{\Gamma}_{j}^{\prime})}{\partial\bm{\alpha}^{\prime\top}}|=|\bm{\Psi}-\bm{\Psi}^{\prime}|O(\rho^{j-2})$.
Similarly, we can verify the conditions imposed on second order derivatives.

The proof is now complete.

∎

###### Proof of Proposition 2.4.

Since $\mathbf{H}(\mathbf{z};\bm{\theta})$ is a positive and diagonal matrix,
we have

$\displaystyle|\mathbf{H}(\mathbf{z};\bm{\theta})-\mathbf{H}(\mathbf{z}^{\prime};\bm{\theta})|$
$\displaystyle=$
$\displaystyle|(\mathbf{H}^{2}(\mathbf{z};\bm{\theta})-\mathbf{H}^{2}(\mathbf{z}^{\prime};\bm{\theta}))\cdot(\mathbf{H}(\mathbf{z};\bm{\theta})+\mathbf{H}(\mathbf{z}^{\prime};\bm{\theta}))^{-1}|$
$\displaystyle\leq$
$\displaystyle\sum_{j=1}^{\infty}|\bm{\Psi}_{j}(\tau)|^{1/2}\cdot|\mathbf{x}_{j}-\mathbf{x}_{j}^{\prime}|.$

Then Assumption 1 is automatically met if
$\|\widetilde{\bm{\eta}}_{t}(\tau)\|_{r}\sum_{j=1}^{\infty}|\bm{\Psi}_{j}(\tau)|^{1/2}<1$.
In addition, as $|\bm{\Psi}_{j}(\tau)|$ converges to zero with exponential
rate and ${\partial
h_{i,t}^{1/2}}/{\partial\theta_{i}}=\frac{1}{2}h_{i,t}^{-1/2}{\partial
h_{i,t}}/{\partial\theta_{i}}$, similar to the proof of Proposition 2.3 we can
easily verify Assumptions 2 and 4. For the identification conditions of the
GARCH process, we refer readers to Proposition 3.4 of Jeantheau (1998), who
proves that assuming the minimal representation is enough for ensuring
Assumption 3 holds.

However, by using techniques which are more specific to the GARCH models, the
condition
$\|\widetilde{\bm{\eta}}_{t}(\tau)\|_{r}\sum_{j=1}^{\infty}|\bm{\Psi}_{j}(\tau)|^{1/2}<1$
can be weaken to
$\|\widetilde{\bm{\eta}}_{t}(\tau)\|_{r}^{2}\sum_{j=1}^{\infty}|\bm{\Psi}_{j}(\tau)|<1$.
Define
$\widetilde{\mathbf{y}}_{t}(\tau)=\widetilde{\mathbf{x}}_{t}(\tau)\odot\widetilde{\mathbf{x}}_{t}(\tau)$
and
$\widetilde{\mathbf{V}}_{t}(\tau)=\mathrm{diag}\left(\widetilde{\bm{\eta}}_{t}(\tau)\odot\widetilde{\bm{\eta}}_{t}(\tau)\right)$.
We first prove the existence of $\|\widetilde{\mathbf{y}}_{t}(\tau)\|_{r/2}$
(which implies the existence of $\|\widetilde{\mathbf{x}}_{t}(\tau)\|_{r}$) as
well as its weak dependence property by means of a chaotic expansion. Since
$\widetilde{\mathbf{y}}_{t}(\tau)=\widetilde{\mathbf{V}}_{t}(\tau)\bm{\alpha}(\tau)+\sum_{j=1}^{\infty}\widetilde{\mathbf{V}}_{t}(\tau)\bm{\Psi}_{j}(\tau)\widetilde{\mathbf{y}}_{t-j}(\tau)$,
by substitute $\widetilde{\mathbf{y}}_{t-j}(\tau)$ recursively, we have

$\widetilde{\mathbf{y}}_{t}(\tau)=\widetilde{\mathbf{V}}_{t}(\tau)\left\\{\bm{\alpha}(\tau)+\sum_{k=1}^{\infty}\sum_{j_{1},..,j_{k}=1}^{\infty}\mathbf{\Psi}_{j_{1}}(\tau)\widetilde{\mathbf{V}}_{t-j_{1}}(\tau)\cdots\mathbf{\Psi}_{j_{k}}(\tau)\widetilde{\mathbf{V}}_{t-j_{1}-\cdots-
j_{k}}(\tau)\bm{\alpha}(\tau)\right\\}.$

To prove the boundedness of
$\left\|\widetilde{\mathbf{y}}_{t}(\tau)\right\|_{r/2}$, since
$\\{\widetilde{\mathbf{V}}_{t}(\tau)\\}$ are independent random variables, it
suffices to show that

$\sum_{k=1}^{\infty}\sum_{j_{1},..,j_{k}=1}^{\infty}\left\|\bm{\Psi}_{j_{1}}(\tau)\widetilde{\mathbf{V}}_{t-j_{1}}(\tau)\cdots\bm{\Psi}_{j_{k}}(\tau)\widetilde{\mathbf{V}}_{t-j_{1}-\cdots-
j_{k}}(\tau)\bm{\alpha}(\tau)\right\|_{r/2}<\infty.$

By using $\sup_{\tau\in[0,1]}|\bm{\alpha}(\tau)|<\infty$ and
$\|\widetilde{\mathbf{V}}_{t}(\tau)\|_{r/2}\sum_{j=1}^{\infty}|\bm{\Psi}_{j}(\tau)|<1$,
we have

$\displaystyle\sum_{k=1}^{\infty}\sum_{j_{1},..,j_{k}=1}^{\infty}\left\|\bm{\Psi}_{j_{1}}(\tau)\widetilde{\mathbf{V}}_{t-j_{1}}(\tau)\cdots\bm{\Psi}_{j_{k}}(\tau)\widetilde{\mathbf{V}}_{t-j_{1}-\cdots-
j_{k}}(\tau)\bm{\alpha}(\tau)\right\|_{r/2}$ $\displaystyle\leq$
$\displaystyle\sup_{\tau\in[0,1]}|\bm{\alpha}(\tau)|\sum_{k=1}^{\infty}\sum_{j_{1},..,j_{k}=1}^{\infty}|\bm{\Psi}_{j_{1}}(\tau)|\cdots|\bm{\Psi}_{j_{k}}(\tau)|\|\widetilde{\mathbf{V}}_{t}(\tau)\|_{r/2}^{k}$
$\displaystyle\leq$
$\displaystyle\sup_{\tau\in[0,1]}|\bm{\alpha}(\tau)|\sum_{k=1}^{\infty}\left(\|\widetilde{\mathbf{V}}_{t}(\tau)\|_{r/2}\sum_{j=1}^{\infty}|\bm{\Psi}_{j}(\tau)|\right)^{k}<\infty.$

Hence, we have $\|\widetilde{\bm{x}}_{t}(\tau)\|_{r}<\infty$. Next, we show
that $\delta_{r}^{\widetilde{\mathbf{x}}(\tau)}(k)=O(\rho^{k})$ for some
$0<\rho<1$. Write

$\widetilde{\mathbf{y}}_{t}(\tau)=\mathrm{diag}\left(\bm{\alpha}(\tau)+\sum_{j=1}^{\infty}\bm{\Psi}_{j}(\tau)\widetilde{\mathbf{y}}_{t-j}(\tau)\right)(\widetilde{\bm{\eta}}_{t}(\tau)\odot\widetilde{\bm{\eta}}_{t}(\tau)).$

By using the same arguments as in the proof of Proposition 2.1, we have
$\delta_{r}^{\widetilde{\mathbf{y}}(\tau)}(k)=O(\rho^{k})$ since
$\|\widetilde{\bm{\eta}}_{t}(\tau)\|_{r}^{2}\sum_{j=1}^{\infty}|\bm{\Psi}_{j}(\tau)|<1$
and $|\bm{\Psi}_{j}(\tau)|=O(\rho^{j})$. Since $|a-b|\leq|a^{2}-b^{2}|^{1/2}$
for $a\geq 0,b\geq 0$ and $\mathbf{H}(\cdot)$ is a positive diagonal matrix,
for $t\geq 1$, we have

$\displaystyle\|\widetilde{\bm{x}}_{t}(\tau)-\widetilde{\bm{x}}_{t}^{*}(\tau)\|_{r}$
$\displaystyle\leq$
$\displaystyle\sum_{j=1}^{t}|\bm{\Psi}_{j}(\tau)|^{1/2}\cdot\|\widetilde{\mathbf{y}}_{t-j}(\tau)-\widetilde{\mathbf{y}}_{t-j}^{*}(\tau)\|_{r/2}^{1/2}\cdot\|\widetilde{\bm{\eta}}_{t}(\tau)\|_{r}$
$\displaystyle=$
$\displaystyle\sum_{j=1}^{t}O(\rho^{j/2})O(\rho^{(t-j)/2})=O(\rho^{\prime t})$

for some $0<\rho^{\prime}<1$.

The proof is now completed.

∎

## Appendix B

###  B.1 Preliminary Lemmas

First, we define a few notations for better presentation. First, let
$\bm{\eta}=(\bm{\eta}_{1}^{\top},\bm{\eta}_{2}^{\top})^{\top}$, where
$\bm{\eta}_{1}$ and $\bm{\eta}_{2}$ are the same generic vectors as in (2.4).
Let $\widehat{K}(\cdot)$ be a a kernel function being Lipschitz continuous and
bounded on $[-1,1]$.

For $\tau\in[0,1]$ and
$\bm{\eta}\in\mathbf{E}_{T}(r)=\bm{\Theta}_{r}\times(h\cdot\bm{\Theta}^{(1)})$,
define

$G_{\tau}(\bm{\eta}):=\frac{1}{Th}\sum_{t=1}^{T}\widehat{K}\left(\frac{\tau_{t}-\tau}{h}\right)\left[g(\mathbf{y}_{t},\bm{\eta}_{1}+\bm{\eta}_{2}\cdot(\tau_{t}-\tau)/h)-E(g(\mathbf{y}_{t},\bm{\eta}_{1}+\bm{\eta}_{2}\cdot(\tau_{t}-\tau)/h))\right],$
(B.1)

where $g(\cdot)\in\mathcal{H}(C,\bm{\chi},M)$ and
$\mathbf{y}_{t}=(\mathbf{x}_{t},\mathbf{z}_{t-1})$. Let
$G_{\tau}^{c}(\bm{\eta})$, $\widetilde{G}_{\tau}(\bm{\eta})$ denote the same
quantity but with $\mathbf{y}_{t}$ replaced by
$\mathbf{y}_{t}^{c}=(\mathbf{x}_{t},\mathbf{z}_{t-1}^{c})$ or
$\widetilde{\mathbf{y}}_{t}(\tau_{t})=(\widetilde{\mathbf{x}}_{t}(\tau_{t}),\widetilde{\mathbf{z}}_{t-1}(\tau_{t}))$.

In addition, let

$\displaystyle\widetilde{B}_{\tau}(\bm{\eta})$ $\displaystyle:=$
$\displaystyle\frac{1}{Th}\sum_{t=1}^{T}\widehat{K}\left(\frac{\tau_{t}-\tau}{h}\right)g(\widetilde{\mathbf{y}}_{t}(\tau_{t}),\bm{\eta}_{1}+\bm{\eta}_{2}\cdot(\tau_{t}-\tau)/h),$
$\displaystyle\mathbf{m}_{t}^{(2)}(u,\tau)$ $\displaystyle:=$
$\displaystyle\widehat{K}\left(\frac{\tau-u}{h}\right)g(\widetilde{\mathbf{y}}_{t}(u),\bm{\theta}(\tau)-v\mathbf{d}(u,\tau))\cdot\mathbf{d}(u,\tau),$
(B.2)

where
$\mathbf{d}(u,\tau):=\bm{\theta}(u)-\bm{\theta}(\tau)-(u-\tau)\bm{\theta}^{(1)}(\tau)$
and some $v\in[0,1]$.

######  Lemma B.1.

Suppose Assumptions 1 and 3 hold. Then,
$E\left(\mathcal{l}(\widetilde{\mathbf{x}}_{1}(\tau),\widetilde{\mathbf{z}}_{0}(\tau);\bm{\vartheta})\right)$
is uniquely maximized at $\bm{\theta}(\tau)$.

######  Lemma B.2.

Suppose Assumptions 3–4 hold. Then,
$\mathcal{l},\gradient\mathcal{l},\gradient^{2}\mathcal{l}\in\mathcal{H}(3,\bm{\chi},M)$
for some $M>0$ and $\bm{\chi}=\\{\chi_{j}\\}_{j=1,2,\ldots}$ with
$\chi_{j}=O(j^{-(2+s)})$ and $s>0$. In addition, if
$\mathbf{H}(\mathbf{z};\bm{\vartheta})\equiv\mathbf{H}(\mathbf{0};\bm{\vartheta})$,
$\mathcal{l},\gradient\mathcal{l},\gradient^{2}\mathcal{l}\in\mathcal{H}(2,\bm{\chi},M)$.

######  Lemma B.3.

Suppose Assumptions 1–2 hold with $r\geq C$. Then

  1. 1.

$\sup_{\tau\in[0,1]}\left\|\sup_{\bm{\eta}\neq\bm{\eta}^{\prime}}\frac{|\widetilde{G}_{\tau}(\bm{\eta})-\widetilde{G}_{\tau}(\bm{\eta}^{\prime})|}{|\bm{\eta}-\bm{\eta}^{\prime}|}\right\|_{1}\leq
M$ and
$\left\|\sup_{\tau\neq\tau^{\prime},\bm{\eta}\neq\bm{\eta}^{\prime}}\frac{|\widetilde{G}_{\tau}(\bm{\eta})-\widetilde{G}_{\tau^{\prime}}(\bm{\eta}^{\prime})|}{|\tau-\tau^{\prime}|+|\bm{\eta}-\bm{\eta}^{\prime}|}\right\|_{1}\leq
Mh^{-2}$;

  2. 2.

$\sup_{\tau\in[0,1],\bm{\eta}\in\mathbf{E}_{T}(r)}|E(\widetilde{B}_{\tau}(\bm{\eta}))-\int_{-\tau/h}^{(1-\tau)/h}\widehat{K}(u)E(g(\widetilde{\mathbf{y}}_{0}(\tau),\bm{\eta}_{1}+\bm{\eta}_{2}u))\mathrm{d}u|=O((Th)^{-1}+h)$;

  3. 3.

$\left\|\sup_{\tau\neq\tau^{\prime}}\frac{|\bm{\Pi}(\tau)-\bm{\Pi}(\tau^{\prime})|}{|\tau-\tau^{\prime}|}\right\|_{1}\leq
Mh^{-2}$ with
$\bm{\Pi}(\tau):=(Th)^{-1}\sum_{t=1}^{T}\left[\mathbf{m}_{t}^{(2)}(\tau,\tau_{t})-E(\mathbf{m}_{t}^{(2)}(\tau,\tau_{t}))\right]$.

In addition, suppose $\chi_{j}=O(j^{-(2+s)})$ for some $s>0$, then

  1. 4.

$\|\sup_{\tau\in[0,1],\bm{\eta}\in\mathbf{E}_{T}(r)}|\widetilde{G}_{\tau}(\bm{\eta})-G_{\tau}^{c}(\bm{\eta})|\|_{1}=O((Th)^{-1})$.

######  Lemma B.4.

Let $g(\cdot)\in\mathcal{H}(C,\bm{\chi},M)$, where $\chi_{j}=O(j^{-(a+s)})$
for some $s>0$ and $a\geq 1$. Suppose Assumptions 1–2 hold with $q=r/C\geq 1$,
and

$\displaystyle\sup_{\tau\in[0,1]}[\alpha_{j}(\bm{\theta}(\tau))+\beta_{j}(\bm{\theta}(\tau))]=O(j^{-(a+1+s)}).$

Then we obtain

  1. 1.

$\sup_{\tau\in[0,1]}\delta_{q}^{\sup_{\bm{\vartheta}}|g(\widetilde{\mathbf{y}}_{t}(\tau),\bm{\vartheta})|}(j)=O(j^{-(a+s)})$;

  2. 2.

$\sup_{\tau\in[0,1]}\sup_{u,\bm{\eta}}\delta_{q}^{m(\tau,\bm{\eta},u)}(j)=O(j^{-(a+s)})$
and
$\sup_{\tau\in[0,1]}\delta_{q}^{\sup_{u,\bm{\eta}}|m(\tau,\bm{\eta},u)|}(j)=O(j^{-(a+s)})$,
where
$m_{t}(\tau,\bm{\eta},u):=\widehat{K}\left(\frac{\tau-u}{h}\right)g(\widetilde{\mathbf{y}}_{t}(\tau),\bm{\eta}_{1}+\bm{\eta}_{2}\cdot(\tau-u)/h)$;

  3. 3.

$\sup_{\tau,u\in[0,1]}\delta_{q}^{\mathbf{m}_{t}^{(2)}(u,\tau)}(j)=O(h^{2}j^{-(a+s)})$
and
$\sup_{u\in[0,1]}\delta_{q}^{\sup_{\tau}|\mathbf{m}_{t}^{(2)}(u,\tau)|}(j)=O(h^{2}j^{-(a+s)})$.

######  Lemma B.5.

Under the conditions of Lemma B.4 with $q=r/C>1$, then

  1. 1.

$\|\widetilde{G}_{\tau}(\bm{\eta})\|_{q}=O\left((Th)^{-(q^{\prime}-1)/q^{\prime}}\right)$
with $q^{\prime}=\min(2,q)$,

  2. 2.

$\sup_{\bm{\eta}\in\mathbf{E}_{T}(r)}|\widetilde{G}_{\tau}(\bm{\eta})|=o_{P}(1)$;

Suppose further $q=r/C>2$ and $a\geq 3/2$. Then

  1. 3.

$\sup_{\tau\in[0,1]}\sup_{\bm{\eta}\in\mathbf{E}_{T}(r)}|\widetilde{G}_{\tau}(\bm{\eta})|=O_{P}((\log
T)^{1/2}(Th)^{-1/2}h^{-1/2})$.

######  Lemma B.6.

Suppose Assumptions 1–5 hold with $r>6$, and

$\displaystyle\sup_{\tau\in[0,1]}[\alpha_{j}(\bm{\theta}(\tau))+\beta_{j}(\bm{\theta}(\tau))]=O(j^{-(a+1+s)})$

for $a\geq 3/2$ and some $s>0$. Then

$\displaystyle\sup_{\tau\in[0,1]}$
$\displaystyle\Big{|}\gradient\widetilde{\mathcal{L}}_{\tau}(\bm{\theta}(\tau),h\bm{\theta}^{(1)}(\tau))-E[\gradient\widetilde{\mathcal{L}}_{\tau}(\bm{\theta}(\tau),h\bm{\theta}^{(1)}(\tau))]$
$\displaystyle-\frac{1}{Th}\sum_{t=1}^{T}\widehat{\bm{K}}((\tau_{t}-\tau)/h)\otimes\gradient_{\bm{\vartheta}}\mathcal{l}(\widetilde{\bm{x}}_{t}(\tau_{t}),\widetilde{\bm{z}}_{t-1}(\tau_{t});\bm{\theta}(\tau_{t}))\Big{|}=O_{P}(h^{2}\beta_{T}),$

where

$\displaystyle\beta_{T}$ $\displaystyle=$ $\displaystyle(\log
T)^{1/2}(Th)^{-1/2}h^{-1/2},$
$\displaystyle\widehat{\bm{K}}((\tau_{t}-\tau)/h)$ $\displaystyle=$
$\displaystyle K((\tau_{t}-\tau)/h)[1,(\tau_{t}-\tau)/h]^{\top},$
$\displaystyle\widetilde{\mathcal{L}}_{\tau}(\bm{\theta}(\tau),h\bm{\theta}^{(1)}(\tau))$
$\displaystyle:=$ $\displaystyle
T^{-1}\sum_{t=1}^{T}\mathcal{l}(\widetilde{\bm{x}}_{t}(\tau_{t}),\widetilde{\bm{z}}_{t-1}(\tau_{t});\bm{\theta}(\tau)+\bm{\theta}^{(1)}(\tau)(\tau_{t}-\tau))K_{h}(\tau_{t}-\tau).$

######  Lemma B.7.

Under the conditions of Theorem 2.2,

$\displaystyle(1).$
$\displaystyle\sup_{\tau\in[h,1-h]}\left|-\bm{\Sigma}(\tau)(\widehat{\bm{\theta}}(\tau)-\bm{\theta}(\tau))-\gradient_{\bm{\vartheta}}\mathcal{L}_{\tau}(\bm{\theta}(\tau),h\bm{\theta}^{(1)}(\tau))\right|=O_{P}(\gamma_{T}),$
$\displaystyle(2).$
$\displaystyle\sup_{\tau\in[h,1-h]}\left|\gradient_{\bm{\vartheta}}\mathcal{L}_{\tau}(\bm{\theta}(\tau),h\bm{\theta}^{(1)}(\tau))+\frac{1}{2}h^{2}\widetilde{c}_{2}\bm{\Sigma}(\tau)\bm{\theta}^{(2)}(\tau)\right.$
$\displaystyle\left.-\frac{1}{T}\sum_{t=1}^{T}\gradient_{\bm{\vartheta}}\mathcal{l}(\widetilde{\mathbf{x}}_{t}(\tau_{t}),\widetilde{\mathbf{z}}_{t-1}(\tau_{t});\bm{\theta}(\tau_{t}))K_{h}(\tau_{t}-\tau)\right|=O_{P}(\beta_{T}h^{2}+h^{3}+(Th)^{-1}),$

where $\beta_{T}=(Th)^{-1/2}h^{-1/2}(\log T)^{1/2}$ and
$\gamma_{T}=(\beta_{T}+h)((Th)^{-1/2}\log T+h^{2})$.

###  B.2 Secondary Lemmas

Before proceeding further, we introduce some extra notations. Assume that
there exists some measurable function $\widetilde{\mathbf{H}}(\cdot,\cdot)$
such that for $\forall\tau\in[0,1]$,
$\widetilde{\mathbf{h}}_{t}(\tau)=\widetilde{\mathbf{H}}(\tau,\mathcal{F}_{t})\in\mathbb{R}^{d}$
is well defined, where
$\mathcal{F}_{t}=\sigma(\bm{\varepsilon}_{t},\bm{\varepsilon}_{t-1},\ldots)$.
Let

$\displaystyle\widetilde{\mathbf{D}}_{\widetilde{\mathbf{h}}}(\tau):=(Th)^{-1}\sum_{t=1}^{T}\widetilde{\mathbf{h}}_{t}(\tau_{t})\widehat{K}((\tau_{t}-\tau)/h)\quad\text{and}\quad\bm{\Sigma}_{\widetilde{\mathbf{h}}}(\tau)=\sum_{j=-\infty}^{\infty}E[\widetilde{\mathbf{h}}_{0}(\tau)\widetilde{\mathbf{h}}_{j}^{\top}(\tau)].$

Assume that $\bm{\Sigma}_{\widetilde{\mathbf{h}}}(\tau)$ is Lipschitz
continuous and its smallest eigenvalue is bounded away from $0$ uniformly over
$\tau\in[0,1]$. In what follows, we let $\widetilde{h}_{0,i}(\tau)$ stand for
the $i^{th}$ component of $\widetilde{\mathbf{h}}_{t}(\tau)$.

######  Lemma B.8.

Let $q>0$. Let $g\in\mathcal{H}(C,\bm{\chi},M)$. Let
$\mathbf{y}=(\mathbf{y}_{0},\mathbf{y}_{1},\mathbf{y}_{2},\ldots)$ and
$\mathbf{y}^{\prime}=(\mathbf{y}_{0}^{\prime},\mathbf{y}_{1}^{\prime},\mathbf{y}_{2}^{\prime},\ldots)$
be two sequences of random variables. Assume that $\max_{j\geq
0}\|\mathbf{y}_{j}\|_{qC}\leq M$ and $\max_{j\geq
0}\|\mathbf{y}_{j}^{\prime}\|_{qC}\leq M$. Then, we have

  1. 1.

$\left\|\sup_{\bm{\vartheta}\in\bm{\Theta}_{r}}|g(\mathbf{y},\bm{\vartheta})-g(\mathbf{y}^{\prime},\bm{\vartheta})|\right\|_{q}\leq
M\sum_{j=0}^{\infty}\chi_{j}\|\mathbf{y}_{j}-\mathbf{y}_{j}^{\prime}\|_{qC}$;

  2. 2.

$\left\|\sup_{\bm{\vartheta}\neq\bm{\vartheta}^{\prime}}\frac{|g(\mathbf{y},\bm{\vartheta})-g(\mathbf{y},\bm{\vartheta}^{\prime})|}{|\bm{\vartheta}-\bm{\vartheta}^{\prime}|}\right\|_{q}\leq
M$;

  3. 3.

$\left\|\sup_{\bm{\vartheta}\in\bm{\Theta}_{r}}|g(\mathbf{y},\bm{\vartheta})|\right\|_{q}\leq
M$.

Lemma B.8 is Lemma D.4 of Karmakar et al. (2022).

######  Lemma B.9.

Assume that for $i=1,\ldots,d$

  1. 1.

$\sup_{\tau\in[0,1]}\|\widetilde{h}_{0,i}(\tau)\|_{q}<\infty$ with some $2\leq
q\leq 4$,

  2. 2.

$\sup_{\tau\neq\tau^{\prime}}\|\widetilde{h}_{0,i}(\tau)-\widetilde{h}_{0,i}(\tau^{\prime})\|_{2}/|\tau-\tau^{\prime}|<\infty$,

  3. 3.

$\sup_{\tau\in[0,1]}\delta_{q}^{\widetilde{h}_{0,i}(\tau)}(j)=O(j^{-(2+s)})$
for some $s\geq 0$.

In addition, assume that $h(\log T)^{3/2}\to 0$ and $\frac{(\log
T)^{4}}{T^{(sq+2)/(2sq+3q-2)}h}\to 0$. Then

$\displaystyle\lim_{T\to\infty}\Pr\left(\sqrt{\frac{Th}{\widetilde{v}_{0}}}\sup_{\tau\in[h,1-h]}\left|\bm{\Sigma}_{\widetilde{\mathbf{h}}}^{-1/2}(\tau)\widetilde{\mathbf{D}}_{\widetilde{\mathbf{h}}}(\tau)\right|-B(m^{*})\leq\frac{u}{\sqrt{2\log(m^{*})}}\right)=\exp(-2\exp(-u)),$

where

$\displaystyle B(m^{*})$ $\displaystyle=$
$\displaystyle\sqrt{2\log(m^{*})}+\frac{\log(C_{K})+(k/2-1/2)\log(\log(m^{*}))-\log(2)}{\sqrt{2\log(m^{*})}},$
$\displaystyle C_{K}$ $\displaystyle=$
$\displaystyle\frac{\\{\int_{-1}^{1}|K^{(1)}(u)|^{2}\mathrm{d}u/\widetilde{v}_{0}\pi\\}^{1/2}}{\Gamma(k/2)},\quad
m^{*}=1/h,$

and $\Gamma(\cdot)$ is the Gamma function.

Lemma B.9 is Lemma B.3 of Karmakar et al. (2022).

######  Lemma B.10.

Assume that for $i=1,\ldots,d$

  1. 1.

$\sup_{\tau\in[0,1]}\|\widetilde{h}_{0,i}(\tau)\|_{q}<\infty$ with some $2\leq
q\leq 4$,

  2. 2.

$\sup_{\tau\neq\tau^{\prime}}\|\widetilde{h}_{0,i}(\tau)-\widetilde{h}_{0,i}(\tau^{\prime})\|_{2}/|\tau-\tau^{\prime}|<\infty$,

  3. 3.

$\sup_{\tau\in[0,1]}\delta_{q}^{\widetilde{h}_{0,i}(\tau)}(j)=O(j^{-(2+s)})$
for some $s\geq 0$.

Let
$\mathbf{S}_{\widetilde{\mathbf{h}}}(t)=\sum_{s=1}^{t}\widetilde{\mathbf{h}}_{s}(\tau_{s})$.
Then on a richer probability space, there exists i.i.d. k-dimensional standard
normal variables $\mathbf{v}_{1},\mathbf{v}_{2},\ldots$ and a process
$\mathbf{S}_{\widetilde{\mathbf{h}}}^{0}(t)=\sum_{s=1}^{t}\bm{\Sigma}_{\widetilde{\mathbf{h}}}^{1/2}(\tau_{s})\mathbf{v}_{s}$
such that

$\displaystyle(\mathbf{S}_{\widetilde{\mathbf{h}}}(t))_{t=1}^{T}=_{D}(\mathbf{S}_{\widetilde{\mathbf{h}}}^{0}(t))_{t=1}^{T}\quad\text{and}\quad\max_{t\geq
1}|\mathbf{S}_{\widetilde{\mathbf{h}}}(t)-\mathbf{S}_{\widetilde{\mathbf{h}}}^{0}(t)|=O_{P}(\pi_{T}),$

where $\pi_{T}=T^{\frac{q(s+3)-4}{2q(2s+3)-4}}(\log
T)^{\frac{2(s+1)(q+1)}{q(2s+3)-2}}.$

Lemma B.10 is from Theorem 1 and Corollary 2 of Wu & Zhou (2011).

###  B.3 Proofs of Preliminary Lemmas

###### Proof of Lemma B.1.

Let

$\displaystyle\mathbf{M}_{t}(\bm{\vartheta},\bm{\theta}(\tau))$
$\displaystyle:=$
$\displaystyle(\mathbf{H}(\widetilde{\mathbf{z}}_{t-1}(\tau);\bm{\vartheta})\mathbf{H}(\widetilde{\mathbf{z}}_{t-1}(\tau);\bm{\vartheta})^{\top})^{-1/2}\mathbf{H}(\widetilde{\mathbf{z}}_{t-1}(\tau);\bm{\theta}(\tau))\mathbf{H}(\widetilde{\mathbf{z}}_{t-1}(\tau);\bm{\theta}(\tau))^{\top}$
$\displaystyle\times(\mathbf{H}(\widetilde{\mathbf{z}}_{t-1}(\tau);\bm{\vartheta})\mathbf{H}(\widetilde{\mathbf{z}}_{t-1}(\tau);\bm{\vartheta})^{\top})^{-1/2}.$

By Assumption 1.1 and the construction of $\widetilde{\mathbf{x}}_{t}(\tau)$,
we write

$\displaystyle
E\left(\mathcal{l}(\widetilde{\mathbf{x}}_{t}(\tau),\widetilde{\mathbf{z}}_{t-1}(\tau);\bm{\vartheta})\right)$
$\displaystyle=$
$\displaystyle-\frac{1}{2}E\log\det\left\\{\mathbf{H}(\widetilde{\mathbf{z}}_{t-1}(\tau);\bm{\vartheta})\mathbf{H}(\widetilde{\mathbf{z}}_{t-1}(\tau);\bm{\vartheta})^{\top}\right\\}$
$\displaystyle-\frac{1}{2}E\mathrm{tr}\left\\{(\mathbf{H}(\widetilde{\mathbf{z}}_{t-1}(\tau);\bm{\vartheta})\mathbf{H}(\widetilde{\mathbf{z}}_{t-1}(\tau);\bm{\vartheta})^{\top})^{-1}[\widetilde{\mathbf{x}}_{t}(\tau)-\bm{\mu}(\widetilde{\mathbf{z}}_{t-1}(\tau);\bm{\vartheta})][\widetilde{\mathbf{x}}_{t}(\tau)-\bm{\mu}(\widetilde{\mathbf{z}}_{t-1}(\tau);\bm{\vartheta})]^{\top}\right\\}$
$\displaystyle=$
$\displaystyle-\frac{1}{2}E\log\det\left\\{\mathbf{H}(\widetilde{\mathbf{z}}_{t-1}(\tau);\bm{\vartheta})\mathbf{H}(\widetilde{\mathbf{z}}_{t-1}(\tau);\bm{\vartheta})^{\top}\right\\}-\frac{1}{2}E\mathrm{tr}\left\\{\mathbf{M}_{t}(\bm{\vartheta},\bm{\theta}(\tau))\right\\}$
$\displaystyle-\frac{1}{2}E\left([\bm{\mu}(\widetilde{\mathbf{z}}_{t-1}(\tau);\bm{\theta}(\tau))-\bm{\mu}(\widetilde{\mathbf{z}}_{t-1}(\tau);\bm{\vartheta})]^{\top}(\mathbf{H}(\widetilde{\mathbf{z}}_{t-1}(\tau);\bm{\vartheta})\mathbf{H}(\widetilde{\mathbf{z}}_{t-1}(\tau);\bm{\vartheta})^{\top})^{-1}\right.$
$\displaystyle\left.\times[\bm{\mu}(\widetilde{\mathbf{z}}_{t-1}(\tau);\bm{\theta}(\tau))-\bm{\mu}(\widetilde{\mathbf{z}}_{t-1}(\tau);\bm{\vartheta})]\right)$
$\displaystyle=$
$\displaystyle-\frac{1}{2}\left[-E\log\det\left(\mathbf{M}_{t}(\bm{\vartheta},\bm{\theta}(\tau))\right)+E\mathrm{tr}\left\\{\mathbf{M}_{t}(\bm{\vartheta},\bm{\theta}(\tau))\right\\}\right]$
$\displaystyle-\frac{1}{2}E\log\det\left(\widetilde{\mathbf{H}}_{t}(\tau,\bm{\theta}(\tau))\widetilde{\mathbf{H}}_{t}(\tau,\bm{\theta}(\tau))^{\top}\right)$
$\displaystyle-\frac{1}{2}E\left([\bm{\mu}(\widetilde{\mathbf{z}}_{t-1}(\tau);\bm{\theta}(\tau))-\bm{\mu}(\widetilde{\mathbf{z}}_{t-1}(\tau);\bm{\vartheta})]^{\top}(\mathbf{H}(\widetilde{\mathbf{z}}_{t-1}(\tau);\bm{\vartheta})\mathbf{H}(\widetilde{\mathbf{z}}_{t-1}(\tau);\bm{\vartheta})^{\top})^{-1}\right.$
$\displaystyle\left.\times[\bm{\mu}(\widetilde{\mathbf{z}}_{t-1}(\tau);\bm{\theta}(\tau))-\bm{\mu}(\widetilde{\mathbf{z}}_{t-1}(\tau);\bm{\vartheta})]\right).$

For any positive definite matrix $\mathbf{M}$ with eigenvalues
$\lambda_{1},\ldots,\lambda_{m}>0$, we have

$\displaystyle
f(\mathbf{M}):=-\log\det\left(\mathbf{M}\right)+\mathrm{tr}\left\\{\mathbf{M}\right\\}=\sum_{i=1}^{m}(\lambda_{i}-\log\lambda_{i})\geq
m,$

where the equality holds if $\lambda_{1}=\cdots=\lambda_{m}=1$ in which case
$\mathbf{M}=\mathbf{I}_{m}$. Thus, $f(\mathbf{M})$ is uniquely minimized at
$\mathbf{M}=\mathbf{I}_{m}$, which implies that
$E[f(\mathbf{M}_{t}(\bm{\vartheta},\bm{\theta}(\tau)))]$ is uniquely minimized
at $\bm{\vartheta}=\bm{\theta}(\tau)$ by Assumption 3.2. In addition, since
$\mathbf{H}(\widetilde{\mathbf{z}}_{t-1}(\tau);\bm{\vartheta})\mathbf{H}(\widetilde{\mathbf{z}}_{t-1}(\tau);\bm{\vartheta})^{\top}$
is a positive definite matrix, then

$\displaystyle
E\left([\bm{\mu}(\widetilde{\mathbf{z}}_{t-1}(\tau);\bm{\theta}(\tau))-\bm{\mu}(\widetilde{\mathbf{z}}_{t-1}(\tau);\bm{\vartheta})]^{\top}(\mathbf{H}(\widetilde{\mathbf{z}}_{t-1}(\tau);\bm{\vartheta})\mathbf{H}(\widetilde{\mathbf{z}}_{t-1}(\tau);\bm{\vartheta})^{\top})^{-1}\right.$
$\displaystyle\left.\times[\bm{\mu}(\widetilde{\mathbf{z}}_{t-1}(\tau);\bm{\theta}(\tau))-\bm{\mu}(\widetilde{\mathbf{z}}_{t-1}(\tau);\bm{\vartheta})]\right)\geq
0$

is uniquely minimized at $\bm{\vartheta}=\bm{\theta}(\tau)$ by Assumption 3.2.
Hence,
$E\left(\mathcal{l}(\widetilde{\mathbf{x}}_{t}(\tau),\widetilde{\mathbf{z}}_{t-1}(\tau);\bm{\vartheta})\right)$
is uniquely maximized at $\bm{\theta}(\tau)$. ∎

###### Proof of Lemma B.2.

We first consider $\mathcal{l}(\cdot)$. Write

$\displaystyle\mathcal{l}(\mathbf{x},\mathbf{z};\bm{\vartheta})-\mathcal{l}(\mathbf{x}^{\prime},\mathbf{z}^{\prime};\bm{\vartheta})$
$\displaystyle=$