# Asymptotic Analysis of Higher-order Scattering Transform of Gaussian
Processes

Gi-Ren Liu  Yuan-Chung Sheu  Hau-Tieng Wu<EMAIL_ADDRESS>Department of
Mathematics, National Chen-Kung University, Tainan, Taiwan  Department of
Applied Mathematics, National Yang Ming Chiao Tung University, Hsinchu, Taiwan
Department of Mathematics and Department of Statistical Science, Duke
University, Durham, NC, USA

###### Abstract

We analyze the scattering transform with the quadratic nonlinearity (STQN) of
Gaussian processes without depth limitation. STQN is a nonlinear transform
that involves a sequential interlacing convolution and nonlinear operators,
which is motivated to model the deep convolutional neural network. We prove
that with a proper normalization, the output of STQN converges to a chi-square
process with one degree of freedom in the finite dimensional distribution
sense, and we provide a total variation distance control of this convergence
at each time that converges to zero at an exponential rate. To show these, we
derive a recursive formula to represent the intricate nonlinearity of STQN by
a linear combination of Wiener chaos, and then apply the Malliavin calculus
and Stein’s method to achieve the goal.

###### keywords:

scaling limits; wavelet transform; scattering transform; Wiener-It$\hat{{o}}$
decomposition; Stein’s method; Malliavin calculus.

###### MSC:

[2010]: Primary 60G60, 60H05, 62M15; Secondary 35K15.

##  1 Introduction

The scattering transform (ST) is motivated by establishing a mathematical
foundation of the convolutional neural network [1], and it has been applied to
various signals, for example, fetal heart rate [2], brain waves [3, 4],
respiration [5], marine bioacoustics [6], and audio [7, 8]. It provides a
variety of representations for a given function $X$ through a sequential
interlacing convolution and nonlinear operators:

$U[j_{1},j_{2},\ldots,j_{M}]X(t)=A\left(\cdots
A\left(A\left(X\star\psi_{j_{1}}\right)\star\psi_{j_{2}}\right)\cdots\star\psi_{j_{M}}(t)\right),\
t\in\mathbb{R},$

where $M\in\mathbb{N}$ is the depth of ST,
$\\{j_{1},j_{2},\ldots,j_{M}\\}\subset\\{\ldots,J-2,J-1,J\\}$ is a set of
scale parameters, $J\in\mathbb{Z}$ determines the range of interest in the
frequency domain, $\\{\psi_{j_{1}},\ldots,\psi_{j_{M}}\\}$ is a family of
wavelets generated from a selected mother wavelet $\psi$, and
$A:\mathbb{C}\to\mathbb{C}$ is the chosen activation function. This
construction is called the scattering network. The associated $M$-th order ST
coefficients are then computed through the pooling process

$S_{J}[j_{1},j_{2},\ldots,j_{M}]X(t):=U[j_{1},j_{2},\ldots,j_{M}]X\star\phi_{J}(t),\
t\in\mathbb{R},$

where $\phi_{J}$ is a low-pass filter, and it is usually chosen to be the
father wavelet associated with $\psi$ and $J$ so that the Littlewood-Paley
condition is satisfied. We mention that ST can also be defined with the Gabor
transform [9, 10] and others [11, 12], but we focus on the wavelet transform
considered in [1] in this paper. In most applications, only the first- and
second-order ST coefficients are used because extracting higher-order
information brings additional computational costs, except [7, 13], which shows
the potential benefit of considering higher order ST coefficients. There are
various choices for the activation $A$; for example, $A(\cdot)=|\cdot|$ [1,
13], $A(\cdot)=|\cdot|^{2}$ [6, 14], and more general functions like
Lipschtiz-continuous functions [15, 16]. We mention that
$A(\cdot)=|\cdot|^{2}$ is considered to speed up the computation of deeper
scattering networks [6], since the computation could be carried out without
leaving the Fourier domain. When $A(\cdot)=|\cdot|^{2}$, we call the resulting
transform the ST with the quadratic nonlinearity (STQN).

There have been several theoretical supports established for ST. If the mother
wavelet and the low-pass filter $\phi_{J}$ satisfy the Littlewood-Paley
condition and $A(\cdot)=|\cdot|$, the ST coefficients are approximately
invariant to time shifts and stable to small time-warping deformation [1].
These properties have been generalized to Lipschitz-continuous activation
functions [16]. Similar time invariance and stability to deformation results
are discussed in [15] under a different setting. The authors of [14] prove
that the STQN coefficients satisfy the Lipschitz continuous property; that is,
$\|S_{J}[j_{1},j_{2},\ldots,j_{M}]X-S_{J}[j_{1},j_{2},\ldots,j_{M}]Y\|_{L^{2}}$
can be controlled by $\|X-Y\|_{L^{2}}$ for any $X,Y\in L^{2}$, if some
conditions on the wavelets, the low-pass filter and the $L^{\infty}$ norm of
inputs hold. The authors of [17, Figure 4] showed that the decay of the STQN
coefficients depend on the number of sinusoidal components contained in the
signal and STQN has the ability to disentangle multiple factors of variability
in the spectral envelope. In addition to the above results in the
deterministic setup, ST has also been studied from the probabilistic point of
view. The authors of [18] found that the second-order ST with
$A(\cdot)=|\cdot|$ can be applied to characterize more detailed properties of
random processes with stationary increments, by computing the expectation of
the second-order ST of $X$ and observing its behavior with respect to the
scale parameters $j_{1}$ and $j_{2}$. This result has been extended to general
Lipschtiz-continuous activation functions [16].

Despite the above results, however, due to the nonlinearity of the activation
function and its subsequent convolution, the behavior of higher order ST with
a random process input is still open. Precisely, when $M>2$, since the input
$U[j_{1},j_{2},\ldots,j_{M-1}]X$ to the $M$-th layer of the scattering network
is neither a Gaussian process nor a subordinated Gaussian process even when
$X$ is a stationary Gaussian process or a Gaussian process with stationary
increments, it is challenging to study ST with random processes when $M>2$.
Motivated by its practical usefulness and theoretical challenges, in this work
we focus on exploring STQN with depth $M\geq 2$ from the probabilistic point
of view, in which the inputs are the stationary Gaussian processes, or
Gaussian processes with stationary increments.

To this end, several analysis techniques are needed. First, we derive a
recursive formula to handle the intricate nonlinearity of STQN by representing
$U[j_{1},j_{2},\ldots,j_{M}]X$ by a linear combination of Wiener chaos that
comes from Wiener-It$\hat{\textup{o}}$ integrals of functions defined
iteratively from the spectral density function of the Gaussian random process
Then, we apply the Stein’s method [19] and the product formula for Wiener-
It$\hat{\textup{o}}$ integrals [20] to prove that the total variation (TV)
distance between $\sigma_{j_{M}}^{-2}U[j_{1},j_{2},\ldots,j_{M}]X(2^{j_{M}}t)$
and the square of the standard normal random variable is $O(2^{-j_{M}/2})$ for
each fixed $t\in\mathbb{R}$, where
$\sigma_{j_{M}}^{2}=\mathbb{E}\left[U[j_{1},j_{2},\ldots,j_{M}]X(\cdot)\right]$.
Moreover, we prove that
$2^{j_{M}}U[j_{1},j_{2},\ldots,j_{M}]X(2^{j_{M}}\cdot)$ converges to a weakly
dependent chi-square process with one degree of freedom in the finite
dimensional distribution sense as $j_{M}\rightarrow\infty$. We shall mention
that the proof in this work is inspired by [21], where the Malliavin calculus
and Stein’s method are combined to derive explicit bounds in the Gaussian and
Gamma approximations of random variables in a fixed Wiener chaos of a general
Gaussian process. Although the work [21, Proposition 3.7] provides an error
bound for the Gaussian approximation of a finite sum of multiple Wiener-
It$\hat{\textup{o}}$ integrals, it cannot be directly applied to our problem.
The main obstacle comes from the intricate covariance structure between
different orders of Wiener chaos, which need new controls to ensure that the
approximation error converges to zero as $j_{M}\rightarrow\infty$.

The rest of the paper is organized as follows. In Section 2, we summarize
necessary material for ST and present some preliminaries about the Wiener-
It$\hat{\textup{o}}$ integrals and the Malliavin calculus. In Section 3, we
state our main results, including Theorems 1, 2, and 3. The proofs of our main
results and some technical lemmas are given in Section 4. Table 1 below
contains a list of frequently used symbols and abbreviations.

Symbol |  Description
---|---
$\star$ |  Convolution operator
$M$ |  Depth of the scattering network
$\pm\lambda_{1:p}$ |  Abbreviation for $\pm(\lambda_{1},\lambda_{2},\ldots,\lambda_{p})\in\mathbb{R}^{p}$
$\lambda^{+}_{1:p}$, $d\lambda_{1:p}$, |  Abbreviation for $\lambda_{1}+\lambda_{2}+\cdots+\lambda_{p}$ and $d\lambda_{1}\ d\lambda_{2}\cdots d\lambda_{p}$
$j_{1:m}$ |  Vector of scale parameters for the wavelet transform in the first $m$ layers
$\psi$, $\hat{\psi}$, $\psi_{j}(\cdot)$ |  Mother wavelet, its Fourier transform, and its scaled version $2^{-j}\psi(2^{-j}\cdot)$
$X$ |  Stationary Gaussian process
$U[j]X$ |  Modulus-squared wavelet transform of $X$ defined by $U[j]X=|\psi_{j}\star X|^{2}$
$U[j_{1:m}]$ |  The $m$-th order scattering transform defined by $U[j_{m}]U[j_{m-1}]\cdots U[j_{1}]$
$T[j_{1:m+1}]X$ |  $\left(U[j_{1:m}]X\right)\star\psi_{j_{m+1}}$
$\alpha$ |  Vanishing-moment parameter for $\psi$ defined through $\hat{\psi}(\cdot)=C_{\hat{\psi}}(\cdot)|\cdot|^{\alpha}$
$f_{Z}$ |  Spectral density of the covariance function of a random process $Z$
$\beta$ |  Singularity parameter for $f_{X}$ defined through $f_{X}(\cdot)=C_{X}(\cdot)|\cdot|^{\beta-1}$
$\overline{H}$ |  Hilber space $\\{f\in L^{2}(\mathbb{R})\mid f(-\lambda)=f(\lambda)\ \textup{for all}\ \lambda\in\mathbb{R}\\}$
$\overline{H}^{\otimes p}$ ($\overline{H}^{\odot p}$) |  The $p$-th (symmetric) tensor product of the Hilbert space $\overline{H}$
$W$ |  Complex-valued Gaussian random measure
$I_{p}$ |  $p$-fold Wiener-It$\hat{\textup{o}}$ integrals for $p\in\mathbb{N}$ ($I_{0}$: the identity map)
$\widetilde{f}$ |  Canonical symmetrization of function $f$
$f\otimes_{r}g$, $f\widetilde{\otimes}_{r}g$ |  The $r$-th contraction of $f$ and $g$, and its canonical symmetrization
$L^{2}(\Omega,\overline{H}^{\otimes p})$ |  Random functions $f$ in the Hilbert space $\overline{H}^{\otimes p}$ with $\mathbb{E}[\|f\|^{2}_{L^{2}(\mathbb{R}^{p})}]<\infty$
$M_{r}$ |  $2^{M-1}-2r$ for $r=0,1,\ldots,2^{M-2}-1$
$\\{{}^{M}F^{(M_{r})}_{t}\\}_{r=0}^{2^{M-2}-1}$ |  Integrands in the Wiener-It$\hat{\textup{o}}$ decomposition of $T[j_{1:M}]X$
$\\{{}^{M}\widetilde{G}^{(2^{M}-2\ell)}_{t}\\}_{\ell=0}^{2^{M-1}}$ |  Integrands in the Wiener-It$\hat{\textup{o}}$ decomposition of $U[j_{1:M}]X$
$a_{n}=O(b_{n})$ |  For sequences $\\{a_{n}\\}$ and $\\{b_{n}\\}$, there exists a constant $C>0$ such that $|a_{n}|\leq C|b_{n}|$ when $n$ is sufficiently large.
Table 1: List of frequently used symbols and abbreviations

##  2 Preliminaries

###  2.1 Scattering transform

Let $\psi\in L^{1}(\mathbb{R})\cap L^{2}(\mathbb{R})$ be the mother wavelet,
which satisfies $\int_{\mathbb{R}}\psi(t)dt=0$. A family of real-valued
functions $\\{\psi_{j}(t)\mid j\in\mathbb{Z},t\in\mathbb{R}\\}$ is called a
wavelet family, where

$\psi_{j}(t)=\frac{1}{2^{j}}\psi\left(\frac{t}{2^{j}}\right)\,.$

Denote the Fourier transform of $\psi$ by $\hat{\psi}$, which is defined as

$\hat{\psi}(\lambda)=\int_{\mathbb{R}}e^{-i\lambda t}\psi(t)dt.$

Because $\psi\in L^{1}(\mathbb{R})\cap L^{2}(\mathbb{R})$ and
$\int_{\mathbb{R}}\psi(t)dt=0$, $\hat{\psi}\in{C^{0}(\mathbb{R})\cap
L^{2}(\mathbb{R})}$ with $\hat{\psi}(0)=0$. We make the following assumption.

######  Assumption 1.

We assume that $\hat{\psi}\in L^{1}(\mathbb{R})$ and there exists a bounded
and continuous complex-valued function $C_{\hat{\psi}}$, which is positive at
the origin (i.e., $C_{\hat{\psi}}(0)>0$), such that $\hat{\psi}$ can be
expressed as

$\hat{\psi}(\lambda)=C_{\hat{\psi}}(\lambda)|\lambda|^{\alpha}$

for some $\alpha>0$.

######  Remark 1.

Consider the vanishing moment $N$ of $\psi$ defined as

$N=\max\left\\{n\in\mathbb{N}\mid\int_{\mathbb{R}}t^{\ell}\psi(t)dt=0\
\textup{for}\ \ell=0,1,\ldots,n-1,\ \textup{and}\
\int_{\mathbb{R}}t^{n}\psi(t)dt\neq 0\right\\}.$

The parameter $\alpha$ in Assumption 1 satisfies $\alpha\geq N$ by the Taylor
expansion. Assumption 1 with $\alpha\geq 1$ holds for commonly used wavelets,
including the real part of complex Morlet wavelet ($\alpha=1$), the Mexican
hat wavelet ($\alpha=2$), and the Daubechies-$K$ wavelet ($\alpha=K/2$), where
$K=2,4,\ldots,20.$ We shall mention that for those wavelet functions whose
Fourier transform $\hat{\psi}$ vanishes in a neighborhood of the origin, e.g.,
the Mayer wavelet, all results in this paper hold.

Given a locally bounded function $X:\mathbb{R}\rightarrow\mathbb{R}$, the
wavelet transform of $X$ is defined as

$X\star\psi_{j}(t)=\int_{\mathbb{R}}X(s)\psi_{j}(t-s)ds,\ j\in\mathbb{Z},\
t\in\mathbb{R}.$ (1)

The first-order STQN of $X$ is obtained by composing the wavelet transform of
$X$ with the nonlinear operator $|\cdot|^{2}$; that is,

$U[j_{1}]X(t)=|X\star\psi_{j_{1}}(t)|^{2},\ j_{1}\in\mathbb{Z}.$

Fix $M\geq 1$ to be the order (or depth) of ST. The $M$-th order STQN of $X$
is defined iteratively as

$U[j_{1},j_{2},\ldots,j_{M}]X(t):=U[j_{M}]U[j_{M-1}]\cdots
U[j_{2}]U[j_{1}]X(t)=:U[j_{1:M}]X(t)\,,$

where $j_{1},j_{2},\ldots,j_{M}\in\mathbb{Z}$ and
$j_{1:M}:=[j_{1},\ldots,j_{M}]\in\mathbb{Z}^{M}$ is introduced to simplify the
heavy notation for the upcoming proof. Denote the wavelet transform of the
output of the $M$-th order STQN of $X$ by

$T[j_{1:M+1}]X=U[j_{1:M}]X\star\psi_{j_{M+1}}\,.$ (2)

We define $T[j_{1}]X=X\star\psi_{j_{1}}$. By definition, the operators $U$ and
$T$ have the relationship

$U[j_{1:M}]X=\left|T[j_{1:M}]X\right|^{2}.$ (3)

###  2.2 Stationary Gaussian processes

In this section, we prepare materials about stationary Gaussian random
processes. Let $\overline{H}=\\{f\in L^{2}(\mathbb{R})\mid
f(-\lambda)=\overline{f(\lambda)}\ \textup{for all}\ \lambda\in\mathbb{R}\\}$
be a complex Hilbert space with the inner product $\langle
f,g\rangle=\int_{\mathbb{R}}f(\lambda)\overline{g(\lambda)}d\lambda,$ and let
$W$ be a complex-valued Gaussian random measure on $\mathbb{R}$ satisfying

$\displaystyle W(\Delta_{1})=\overline{W(-\Delta_{1})},\ \
\mathbb{E}[W(\Delta_{1})]=0,\ \textup{and}\ \
\mathbb{E}\left[W(\Delta_{1})\overline{W(\Delta_{2})}\right]=\textup{Leb}(\Delta_{1}\cap\Delta_{2})$
(4)

for any $\Delta_{1},\Delta_{2}\in\mathcal{B}(\mathbb{R})$, where Leb is the
Lebesgue measure on $\mathbb{R}$ and $\mathcal{B}(\mathbb{R})$ is the Borel
$\sigma$-algebra on $\mathbb{R}$. For every $f\in\overline{H}$, we define
$W(f)=\int_{\mathbb{R}}f(\lambda)W(d\lambda).$ Due to (4), $W(f)\in\mathbb{R}$
and $\mathbb{E}[W(f)W(g)]=\langle f,g\rangle.$ Hence, $W=\\{W(f)\mid
f\in\overline{H}\\}$ can be viewed as an centered isonormal Gaussian process
over $\overline{H}$ [22, Section 2.1]. Let
$(\Omega,\mathcal{F}_{W},{\mathcal{P}})$ be a probability space, where the
$\sigma$-algebra $\mathcal{F}_{W}$ is generated by $W$.

Let $X$ be a mean-square continuous and stationary real Gaussian random
process with the constant mean $\mathbb{E}[X(s)]=\mu\in\mathbb{R}$ and the
covariance function $R_{X}$:

$R_{X}(t_{1}-t_{2})=\mathbb{E}[X(t_{1})X(t_{2})]-\mathbb{E}[X(t_{1})]\mathbb{E}[X(t_{2})],\
t_{1},t_{2}\in\mathbb{R}.$

Note that $R_{X}$ is continuous by the assumption of $X$. Since
$\int_{\mathbb{R}}\psi(s)ds=0$, $X\star\psi_{j}(t)=[X-\mu]\star\psi_{j}(t)\,,$
and hence we can assume that $\mathbb{E}[X(s)]=0$ without loss of generality.
By the Bochner-Khinchin theorem, there exists a unique nonnegative measure
$F_{X}:\mathcal{B}(\mathbb{R})\rightarrow[0,\,\infty)$ such that
$F_{X}(\Delta)=F_{X}(-\Delta)$ for any $\Delta\in\mathcal{B}(\mathbb{R})$ and

$R_{X}(t)=\int_{\mathbb{R}}e^{i\lambda t}F_{X}(d\lambda),\ t\in\mathbb{R}.$
(5)

The measure $F_{X}$ is called the spectral measure of the covariance function
$R_{X}$.

######  Assumption 2.

The spectral measure $F_{X}$ has the density $f_{X}$ and

$f_{X}(\lambda)=\frac{C_{X}(\lambda)}{|\lambda|^{1-\beta}},$

where $\beta\in(0,1)$ is the Hurst index of long-range dependence and $C_{X}$
is a bounded and continuous function from $\mathbb{R}$ to $[0,\infty)$ such
that $C_{X}(\lambda)$ decays faster than $|\lambda|^{-\beta-\varepsilon}$ for
some $\varepsilon>0$ when $|\lambda|\rightarrow\infty$.

If the function $C_{X}$ in Assumption 2 satisfies $C_{X}(0)>0$, then $X$ is a
long-range dependent process [23, 24] because $f_{X}$ has a singularity at
$0$. Note that the spectral density function $f_{X}$ is even and nonnegative
with $\int_{\mathbb{R}}f_{X}(\lambda)d\lambda=R_{X}(0)$.

Under Assumption 2, (5) can be rewritten as

$R_{X}(t)=\int_{\mathbb{R}}e^{i\lambda t}f_{X}(\lambda)d\lambda,\
t\in\mathbb{R}.$

By the Karhunen theorem, the Gaussian process $X$ has the representation

$\displaystyle X(t)=\int_{\mathbb{R}}e^{i\lambda
t}\sqrt{f_{X}(\lambda)}W(d\lambda),\ t\in\mathbb{R}.$ (6)

By substituting (6) into (1), the continuous wavelet transform of $X$ can be
represented as a Wiener-It$\hat{\textup{o}}$ integral

$\displaystyle X\star\psi_{j}(t)=$
$\displaystyle\int_{\mathbb{R}}\left[\int_{\mathbb{R}}e^{i\lambda
s}\sqrt{f_{X}(\lambda)}W(d\lambda)\right]\psi_{j}(t-s)ds$ $\displaystyle=$
$\displaystyle\int_{\mathbb{R}}\left[\int_{\mathbb{R}}e^{i\lambda
s}\psi(t-s)ds\right]\sqrt{f_{X}(\lambda)}W(d\lambda)=\int_{\mathbb{R}}e^{i\lambda
t}\hat{\psi}_{j}(\lambda)\sqrt{f_{X}(\lambda)}W(d\lambda),$ (7)

where the last equality follows from the stochastic Fubini theorem [25,
Theorem 2.1] (see also A for more details). From (7), we know that
$X\star\psi_{j}$ has the spectral density

$f_{X\star\psi_{j}}(\lambda)=f_{X}(\lambda)|\hat{\psi}_{j}(\lambda)|^{2},\
\lambda\in\mathbb{R},$

and

$U[j_{1}]X(t)=\left(\int_{\mathbb{R}}e^{i\lambda
t}\hat{\psi}_{j_{1}}(\lambda)\sqrt{f_{X}(\lambda)}W(d\lambda)\right)^{2}.$ (8)

######  Remark 2.

It is known since the works of It$\hat{\textup{o}}$ [26] and Yaglom [27] that
if a Gaussian random process $X$ is increment-stationary in the sense that for
any $s$, $s^{\prime}$, $t$, $t^{\prime}$ and $h\in\mathbb{R}$,

$\mathbb{E}\left[\left(X(t+h)-X(s+h)\right)\left(X(t^{\prime}+h)-X(s^{\prime}+h)\right)\right]=\mathbb{E}\left[\left(X(t)-X(s)\right)\left(X(t^{\prime})-X(s^{\prime})\right)\right]$

and if the covariance between the increments over time intervals $I$ and
$I^{\prime}$ tends to zero when $\min\\{|a-b|\mid a\in I,\ b\in
I^{{}^{\prime}}\\}\rightarrow\infty$, then $X$ admits a spectral
representation

$\displaystyle
X(t)=c+\int_{\mathbb{R}}\left(e^{it\lambda}-1\right)Z(d\lambda),\
t\in\mathbb{R},$ (9)

where $c\in\mathbb{R}$ and $Z(d\lambda)$ is a complex-valued Gaussian random
measure on $\mathbb{R}$ with spectral measure $F_{X}$. If the spectral measure
$F_{X}$ has a density $f_{X}$, then (9) can be rewritten as

$\displaystyle
X(t)=c+\int_{\mathbb{R}}\left(e^{it\lambda}-1\right)\sqrt{f_{X}(\lambda)}W(d\lambda),\
t\in\mathbb{R}.$

By the mean-zero property of the wavelet,

$\displaystyle X\star\psi_{j}(t)=$
$\displaystyle\int_{\mathbb{R}}\left[\int_{\mathbb{R}}\left(e^{it^{\prime}\lambda}-1\right)\sqrt{f_{X}(\lambda)}W(d\lambda)\right]\psi_{j}(t-t^{\prime})dt^{\prime}$
$\displaystyle=$
$\displaystyle\int_{\mathbb{R}}e^{it\lambda}\hat{\psi}_{j}(\lambda)\sqrt{f_{X}(\lambda)}W(d\lambda).$
(10)

From the consistency between (7) and (10), we know that the all results below
hold for Gaussian random processes with stationary increments, including the
fractional Brownian motions, when the associated conditions are fulfilled.

###  2.3 Wiener-It$\hat{o}$ integrals and Winer chaos expansion

In this section, we describe how to express the STQN of stationary Gaussian
random processes in terms of the Wiener chaos expansion that comes from
Wiener-It$\hat{\textup{o}}$ integrals of functions defined iteratively from
the spectral density function of the Gaussian random process. Given an integer
$p\geq 2$, the $p$-th tensor product of the Hilbert space $\overline{H}$ is
denoted by $\overline{H}^{\otimes p}$. According to [20, p. 27],
$f\in\overline{H}^{\otimes p}$ if and only if
$f=f(\lambda_{1},\ldots,\lambda_{p})$, where
$(\lambda_{1},\ldots,\lambda_{p})\in\mathbb{R}^{p}$, is a complex valued
function satisfying
$f(-\lambda_{1},\ldots,-\lambda_{p})=\overline{f(\lambda_{1},\ldots,\lambda_{p})}$
and

$\|f\|^{2}_{\overline{H}^{\otimes
p}}=\int_{\mathbb{R}^{p}}|f(\lambda_{1:p})|^{2}d\lambda_{1:p}<\infty\,,$

where $\lambda_{p_{1}:p_{2}}$ means
$(\lambda_{p_{1}},\,\lambda_{p_{1}+1}\ldots,\lambda_{p_{2}})\in\mathbb{R}^{p_{2}-p_{1}+1}$
and $d\lambda_{p_{1}:p_{2}}$ means $d\lambda_{p_{1}}d\lambda_{p_{1}+1}\cdots
d\lambda_{p_{2}}$ when $p_{2}>p_{1}$. For any $f,g\in\overline{H}^{\otimes
p}$, their inner product is defined as

$\displaystyle\langle f,g\rangle_{\overline{H}^{\otimes
p}}=\int_{\mathbb{R}^{p}}f(\lambda_{1:p})\overline{g(\lambda_{1:p})}d\lambda_{1:p}=\int_{\mathbb{R}^{p}}f(\lambda_{1:p})g(-\lambda_{1:p})d\lambda_{1:p}\,.$

The subscripts of $\|f\|^{2}_{\overline{H}^{\otimes p}}$ and $\langle
f,g\rangle_{\overline{H}^{\otimes p}}$ will be ignored as they can be inferred
from the number of variables of $f$ and $g$. The $p$-th symmetric tensor
product of $\overline{H}$ is denoted by $\overline{H}^{\odot p}$, which
contains those functions $f\in\overline{H}^{\otimes p}$ satisfying
$f(\lambda_{\pi(1)},\ldots,\lambda_{\pi(p)})=f(\lambda_{1},\ldots,\lambda_{p})$,
where $\pi$ is any permutation of the set $\\{1,2,\ldots,p\\}$. Because
$\overline{H}^{\odot p}$ is a Hilbert subspace of $\overline{H}^{\otimes p}$,
the norm and inner product in $\overline{H}^{\odot p}$ are defined in the same
way as in $\overline{H}^{\otimes p}$. For any $f\in\overline{H}^{\otimes p}$,
where $p\in\mathbb{N}$, the $p$-fold Wiener-It$\hat{\textup{o}}$ integrals of
$f$ with respect to the random measure $W$ is defined by

$\displaystyle
I_{p}(f)=\int_{\mathbb{R}^{p}}^{{}^{\prime}}f(\lambda_{1:p})W(d\lambda_{1})\cdots
W(d\lambda_{p}),$

where $\int^{{}^{\prime}}$ means that the integral excludes the diagonal
hyperplanes $\lambda_{k}=\mp\lambda_{k^{{}^{\prime}}}$ for
$k,k^{{}^{\prime}}\in\\{1,\ldots,p\\}$ and $k\neq k^{{}^{\prime}}$. Note that
$I_{p}(f)=I_{p}(\tilde{f})$, where $\tilde{f}\in\overline{H}^{\odot p}$ is the
canonical symmetrization of $f$ defined as

$\displaystyle\tilde{f}(\lambda_{1:p})\frac{1}{p!}\underset{\pi}{\sum}f(\lambda_{\pi(1)},\ldots,\lambda_{\pi(p)}),$
(11)

where the sum runs over all permutations $\pi$ of $\\{1,\ldots,p\\}$. By
default, $I_{0}$ is the identity map, i.e., $I_{0}(f)=f$ for any function $f$.

For $p\in\mathbb{N}\cup\\{0\\}$, let $H_{p}$ be the Hermite polynomial of
degree $p$, which is defined by

${h}_{p}(y)=(-1)^{p}e^{\frac{y^{2}}{2}}\frac{d^{p}}{dy^{p}}e^{-\frac{y^{2}}{2}},\
y\in\mathbb{R}.$

We write $\mathcal{H}_{p}$ to denote the closed linear subspace of
$L^{2}(\Omega,\mathcal{F}_{W},\mathcal{P})$ generated by the random variables
of type ${h}_{p}(W(g))$, $g\in\overline{H}$, $\|g\|=1$. The space
$\mathcal{H}_{p}$ is called the $p$-th Wiener chaos of $W$. According to [20,
28, 21, 22], $I_{p}(\tilde{f})\in\mathcal{H}_{p}$ for any
$f\in\overline{H}^{\otimes p}$. The following product formula allows us to
express output of STQN in terms of a linear combination of Wiener chaos, which
is the first tool used in this paper.

######  Lemma 1 (Product Formula [20, 22]).

Let $p,q\geq 1$. If $f\in\overline{H}^{\odot p}$ and $g\in\overline{H}^{\odot
q}$, then

$\displaystyle I_{p}(f)I_{q}(g)=\overset{p\wedge
q}{\underset{r=0}{\sum}}r!\binom{p}{r}\binom{q}{r}I_{p+q-2r}\left(f\otimes_{r}g\right),$

where $f\otimes_{r}g$ is the $r$th contraction of $f$ and $g$ defined as

$\displaystyle
f\otimes_{r}g(\lambda_{1:p+q-2r})=\int_{\mathbb{R}^{r}}f(\tau_{1:r},\lambda_{1:p-r})g(-\tau_{1:r},\lambda_{p-r+1:p+q-2r})d\tau_{1:r}$

for $r=1,2,\ldots,p\wedge q$. When $r=0$, set $f\otimes_{0}g:=f\otimes g$.

Note that $f\otimes_{r}g\in\overline{H}^{\otimes p+q-2r}$, but it may not be
symmetric. Let $f\tilde{\otimes}_{r}g$ be the canonical symmetrization of
$f\otimes_{r}g$. For any $f\in\overline{H}^{\odot p}$, by noting that the
expectation of any Wiener-It$\hat{\textup{o}}$ integral equals to zero, Lemma
1 implies

$\displaystyle\mathbb{E}\left[|I_{p}(f)|^{2}\right]=\mathbb{E}\left[\overset{p}{\underset{r=0}{\sum}}r!\binom{p}{r}^{2}I_{2p-2r}\left(f\otimes_{r}f\right)\right]=p!f\otimes_{p}f=p!\|f\|_{L^{2}(\mathbb{R}^{p})}^{2},$

which is the so-called isometry property of the multiple Wiener-
It$\hat{\textup{o}}$ integrals.

We now express the ST coefficients using the product formula. By applying
Lemma 1 to (8),

$U[j_{1}]X(t)=\int^{{}^{\prime}}_{\mathbb{R}^{2}}e^{i(\lambda_{1}+\lambda_{2})t}\hat{\psi}_{j_{1}}(\lambda_{1})\sqrt{f_{X}(\lambda_{1})}\hat{\psi}_{j_{1}}(\lambda_{2})\sqrt{f_{X}(\lambda_{2})}W(d\lambda_{1})W(d\lambda_{2})+\|f_{X\star\psi_{j_{1}}}\|_{1}.$
(12)

Denote

$\displaystyle\left\\{\begin{array}[]{l}{}^{1}\widetilde{G}^{(0)}_{t}=\|f_{X\star\psi_{j_{1}}}\|_{1},\\\
{}^{1}\widetilde{G}^{(2)}_{t}(\lambda_{1},\lambda_{2})=e^{i(\lambda_{1}+\lambda_{2})t}\hat{\psi}_{j_{1}}(\lambda_{1})\sqrt{f_{X}(\lambda_{1})}\hat{\psi}_{j_{1}}(\lambda_{2})\sqrt{f_{X}(\lambda_{2})}.\end{array}\right.$
(15)

The first-order STQN of $X$ can be rewritten as

$U[j_{1}]X(t)=I_{2}\left({}^{1}\widetilde{G}^{(2)}_{t}\right)+{}^{1}\widetilde{G}^{(0)}_{t}.$

By the property
$\int_{\mathbb{R}}\psi_{j_{2}}(t-t^{{}^{\prime}})dt^{{}^{\prime}}=0$ and (12),

$\displaystyle T[j_{1:2}]X(t)=$
$\displaystyle\int_{\mathbb{R}}I_{2}\left({}^{1}\widetilde{G}^{(2)}_{t^{{}^{\prime}}}\right)\psi_{j_{2}}(t-t^{{}^{\prime}})dt^{{}^{\prime}}.$
(16)

Because (15) shows that

$\displaystyle{}^{1}\widetilde{G}^{(2)}_{t}(\lambda_{1},\lambda_{2})={}^{1}\widetilde{G}^{(2)}_{0}(\lambda_{1},\lambda_{2})\
e^{i(\lambda_{1}+\lambda_{2})t},$ (17)
${}^{2}F^{(2)}_{t}(\lambda_{1},\lambda_{2}):=\int_{\mathbb{R}}{}^{1}\widetilde{G}^{(2)}_{t^{{}^{\prime}}}(\lambda_{1},\lambda_{2})\psi_{j_{2}}(t-t^{{}^{\prime}})dt^{{}^{\prime}}={}^{1}\widetilde{G}^{(2)}_{t}(\lambda_{1},\lambda_{2})\hat{\psi}_{j_{2}}\left(\lambda_{1}+\lambda_{2}\right),$
(18)

which depends on $t$ in the same way. More concretely,

${}^{2}F^{(2)}_{t}(\lambda_{1},\lambda_{2})=e^{i(\lambda_{1}+\lambda_{2})t}\hat{\psi}_{j_{1}}(\lambda_{1})\sqrt{f_{X}(\lambda_{1})}\hat{\psi}_{j_{1}}(\lambda_{2})\sqrt{f_{X}(\lambda_{2})}\hat{\psi}_{j_{2}}\left(\lambda_{1}+\lambda_{2}\right).$

By substituting (18) into (16), we get

$\displaystyle T[j_{1:2}]X(t)=$ $\displaystyle
I_{2}\left({}^{2}F^{(2)}_{t}\right).$ (19)

By the product formula again with $p=q=2$ and $f=g={}^{2}F^{(2)}_{t}$,

$\displaystyle
U[j_{1:2}]X(t)=\left[T[j_{1:2}]X(t)\right]^{2}=\left[I_{2}\left({}^{2}F^{(2)}_{t}\right)\right]^{2}=\overset{2}{\underset{r=0}{\sum}}I_{4-2r}\left({}^{2}\widetilde{G}^{(4-2r)}_{t}\right),$
(20)

where

${}^{2}\widetilde{G}^{(4-2r)}_{t}=r!\binom{2}{r}\binom{2}{r}{}^{2}F^{(2)}_{t}\widetilde{\otimes}_{r}{}^{2}F^{(2)}_{t}\in\overline{H}^{\odot
4-2r}$ (21)

for each $t\in\mathbb{R}$. From (17) and (18), we have

$\displaystyle{}^{2}\widetilde{G}^{(4-2r)}_{t}(\lambda_{1:4-2r})=e^{i\lambda^{+}_{1:4-2r}t}\
{}^{2}\widetilde{G}^{(4-2r)}_{0}(\lambda_{1:4-2r})$ (22)

for $r=0,1$, where $\lambda^{+}_{p_{1}:p_{2}}$ means
$\lambda_{p_{1}}+\lambda_{p_{1}+1}+\ldots+\lambda_{p_{2}}\in\mathbb{R}$ for
$p_{2}>p_{1}\geq 1$. For general cases, we have the following result.

######  Lemma 2.

For every integer $M\geq 2$ and stationary Gaussian process $X$, the random
processes $T[j_{1:M}]X$ and $U[j_{1:M}]X$ arising from STQN of $X$ have the
Wiener-It$\hat{o}$ decomposition as follows

$\displaystyle
T[j_{1:M}]X(t)=\overset{2^{M-2}-1}{\underset{r=0}{\sum}}I_{2^{M-1}-2r}\left({}^{M}\\!F^{(2^{M-1}-2r)}_{t}\right)$
(23)

and

$\displaystyle
U[j_{1:M}]X(t)=\overset{2^{M-1}}{\underset{\ell=0}{\sum}}I_{2^{M}-2\ell}\left({}^{M}\widetilde{G}^{(2^{M}-2\ell)}_{t}\right),$
(24)

where the integrands, ${}^{M}F^{(2^{M-1}-2r)}_{t}$ and
${}^{M}\widetilde{G}^{(2^{M}-2\ell)}_{t}$, are defined below using the
following recursive formula starting with (15). For $\ell=0,1,\ldots,2^{M-1}$,

$\displaystyle{}^{M}\widetilde{G}^{(2^{M}-2\ell)}_{t}(\lambda_{1:2^{M}-2\ell}):=$
$\displaystyle\overset{2^{M-2}-1}{\underset{r,r^{{}^{\prime}}=0}{\sum}}(\ell-
r-r^{\prime})!\binom{2^{M-1}-2r}{\ell-
r-r^{\prime}}\binom{2^{M-1}-2r^{\prime}}{\ell-r-r^{\prime}}$
$\displaystyle\times{}^{M}F^{(2^{M-1}-2r)}_{t}\widetilde{\otimes}_{\ell-
r-r^{\prime}}{}^{M}F^{(2^{M-1}-2r^{\prime})}_{t}(\lambda_{1:2^{M}-2\ell}),$
(25)

where for $r=0,1,\ldots,2^{M-2}-1$,

${}^{M}F^{(2^{M-1}-2r)}_{t}(\lambda_{1:2^{M-1}-2r}):={}^{M-1}\widetilde{G}^{(2^{M-1}-2r)}_{t}(\lambda_{1:2^{M-1}-2r})\hat{\psi}_{j_{M}}(\lambda^{+}_{1:2^{M-1}-2r}).$
(26)

For the binomial coefficients in (25), we use the following rule: for any
$A,B,C\in\mathbb{Z}$,

$\binom{A}{C}\binom{B}{C}=0\ \textup{if}\ C<0\ \textup{or}\ \min\\{A,B\\}<C.$

The proof of Lemma 2 is based on the mathematical induction method, which can
be found in Section 4.1. For the functions generated by the recursive formula
(25) and (26) with the initial term (15), we have the following observation,
which will be used in the proof of Lemma 2 and Proposition 2. The first one is

${}^{M}\widetilde{G}^{(2^{M}-2\ell)}_{t}(\lambda_{1:2^{M}-2\ell})=\textup{exp}\left(it\lambda^{+}_{1:2^{M}-2\ell}\right)\
{}^{M}\widetilde{G}^{(2^{M}-2\ell)}_{0}(\lambda_{1:2^{M}-2\ell})$ (27)

for $\ell=0,1,\ldots,2^{M-1}-1$ and the second one is

${}^{M}F^{(2^{M-1}-2r)}_{t}(\lambda_{1:2^{M-1}-2r})=\textup{exp}\left(it\lambda^{+}_{1:2^{M-1}-2r}\right)\
{}^{M}F^{(2^{M-1}-2r)}_{0}(\lambda_{1:2^{M-1}-2r})$ (28)

for $r=0,1,\ldots,2^{M-2}-1$, which are generalization of, for instance, (18)
and (22).

${}^{1}\widetilde{G}$${}^{2}F$${}^{2}\widetilde{G}$${}^{3}F$${}^{3}\widetilde{G}$${}^{4}F$${}^{4}\widetilde{G}$$\cdots$
Figure 1: Flow chart for the recursive computation in (25) and (26).

###  2.4 Elements of Malliavin Calculus

Let $\mathcal{S}$ denote the set of all random variables $S$ of the form

$s(W(f_{1}),\ldots,W(f_{n})),$

where $n\geq 1$, $s:\mathbb{R}^{n}\rightarrow\mathbb{R}$ is a
$C^{\infty}$-function such that $s$ and its partial derivatives have at most
polynomial growth, and $f_{i}\in\overline{H}$, $i=1,\ldots,n$. We denote by
$L^{2}(\Omega)$ the set of $\mathcal{F}_{W}$-measurable random variables whose
second moments exist. The space $\mathcal{S}$ is dense in $L^{2}(\Omega)$ [22,
Lemma 2.3.1].

######  Definition 1.

For every integers $n,p\geq 1$, the $p$-th Malliavin derivative of

$S=s(W(f_{1}),\ldots,W(f_{n}))\in\mathcal{S}$

with respect to $W$ is defined by

$\displaystyle
D^{p}S=\overset{n}{\underset{i_{1},\ldots,i_{p}=1}{\sum}}\frac{\partial^{p}s}{\partial
x_{i_{1}}\cdots\partial
x_{i_{p}}}(W(f_{1}),\ldots,W(f_{n}))f_{i_{1}}\otimes\cdots\otimes f_{i_{p}}.$
(29)

Because the sum in (29) runs over all partial derivatives, $D^{p}S$ belongs to
$L^{2}(\Omega,\overline{H}^{\odot p})$.

######  Definition 2.

For every integer $p\geq 1$, we denote by Dom($\delta^{p}$) the subset of
$L^{2}(\Omega,\overline{H}^{\otimes p})$ composed of those elements $u$ such
that there exists a constant $c>0$ satisfying

$|\mathbb{E}\langle D^{p}S,u\rangle_{\overline{H}^{\otimes p}}|\leq
c\sqrt{\mathbb{E}[S^{2}]}\ \textup{for all}\ S\in\mathcal{S}.$ (30)

For each $u\in\textup{Dom}(\delta^{p})$, (30) and the Riesz representation
theorem imply that there exists a unique element in $L^{2}(\Omega)$, denoted
by $\delta^{p}(u)$, such that

$\mathbb{E}\left[S\delta^{p}(u)\right]=\mathbb{E}\langle
D^{p}S,u\rangle_{\overline{H}^{\otimes p}}\ \textup{for all}\
S\in\mathcal{S}.$ (31)

The operator $\delta^{p}:\textup{Dom}(\delta^{p})\subset
L^{2}(\Omega,\overline{H}^{\otimes p})\rightarrow L^{2}(\Omega)$ is called the
divergence operator of order $p$. By [22, Section 2.7], we know that
$\overline{H}^{\otimes p}\subset\ \textup{Dom}(\delta^{p})$ and
$\delta^{p}(u)$ coincides with the $p$-fold Wiener-It$\hat{\textup{o}}$
integrals of the function $u\in\overline{H}^{\otimes p}$ for any $p\geq 1$;
that is,

$\delta^{p}(u)=I_{p}(u)=\int_{\mathbb{R}^{p}}^{{}^{\prime}}u(\lambda_{1},\ldots,\lambda_{p})W(d\lambda_{1})\cdots
W(d\lambda_{p}).$

The Malliavin derivative and divergence of multiple Wiener-
It$\hat{\textup{o}}$ integrals have the following properties. The proof can be
found in [22, Proposition 2.7.4,Proposition 2.8.8].

######  Lemma 3 ([22]).

For every integer $p\geq 1$ and $u\in\overline{H}^{\otimes p}$,

$\delta DI_{p}(u)=pI_{p}(u)$

and

$\displaystyle DI_{p}(u)=pI_{p-1}(\tilde{u}),$ (32)

where $\tilde{u}$ is the canonical symmetrization of $u$ defined through (11)
and

$\displaystyle
I_{p-1}(\tilde{u})(\cdot)=\int^{{}^{\prime}}_{\mathbb{R}^{p-1}}\tilde{u}(\lambda_{1},\ldots,\lambda_{p-1},\cdot)W(d\lambda_{1})\cdots
W(d\lambda_{p-1}).$

On the other hand, for all $r\in\mathbb{N}$, $p\in\mathbb{N}$ and
$q\in[1,\infty]$,

$\displaystyle\|I_{p}(f)\|_{\mathbb{D}^{r,q}}\leq
c_{r,p,q}\|f\|_{\overline{H}^{\otimes p}},\ f\in\overline{H}^{\odot p},$ (33)

where $c_{r,p,q}>0$ is an universal constant and

$\displaystyle\|I_{p}(f)\|_{\mathbb{D}^{r,q}}:=\left(\mathbb{E}[|I_{p}(f)|^{q}]+\mathbb{E}[\|DI_{p}(f)\|_{\overline{H}}^{q}]+\cdots+\mathbb{E}[\|D^{r}I_{p}(f)\|_{\overline{H}^{\otimes
r}}^{q}]\right)^{1/q}.$ (34)

We note that (33) is a special case of the Meyer’s inequality [28, Proposition
1.5.7].

##  3 Main Results

As mentioned in Section 1, we will present large-scale limit theorems about
the non-Gaussian random process $U[j_{1:M}]X$ arising from STQNs of stationary
Gaussian processes.

###  3.1 Convergence Rate of STQNs with Random Inputs

Let $\mathcal{N}$ be a standard normal random variable and
$\sigma_{j_{M}}^{2}=\mathbb{E}U[j_{1:M}]X(t).$ The total variation (TV)
distance between the distributions of
$\frac{1}{\sigma^{2}_{j_{M}}}U[j_{1:M}]X(2^{j_{M}}t)$ and the square of
$\mathcal{N}$ is defined by

$\displaystyle
d_{\textup{TV}}\left(\frac{1}{\sigma_{j_{M}}^{2}}U[j_{1:M}]X(2^{j_{M}}t),\,\mathcal{N}^{2}\right)=\underset{h:\mathbb{R}\rightarrow[0,1]}{\textup{sup}}\left|\mathbb{E}\left[h\Big{(}\frac{1}{\sigma_{j_{M}}^{2}}U[j_{1:M}]X(2^{j_{M}}t)\Big{)}\right]-\mathbb{E}\left[h(\mathcal{N}^{2})\right]\right|,$

where the supremum is taken over Borel functions $h$ taking values in $[0,1]$.
Because the composition function $h(|\cdot|^{2})$ also takes values in
$[0,1]$,

$\displaystyle
d_{\textup{TV}}\left(\frac{1}{\sigma_{j_{M}}^{2}}U[j_{1:M}]X(2^{j_{M}}t),\,\mathcal{N}^{2}\right)\leq\underset{h:\mathbb{R}\rightarrow[0,1]}{\textup{sup}}\left|\mathbb{E}\left[h\Big{(}\frac{1}{\sigma_{j_{M}}}T[j_{1:M}]X(2^{j_{M}}t)\Big{)}\right]-\mathbb{E}\left[h(\mathcal{N})\right]\right|.$
(35)

In the proposition below, we apply Stein’s method and the Malliavin calculus
to the right hand side of (35) to get an upper bound for
$d_{\textup{TV}}\big{(}\frac{1}{\sigma_{j_{M}}^{2}}U[j_{1:M}]X(2^{j_{M}}t),\,\mathcal{N}^{2}\big{)}$.

######  Proposition 1.

For every stationary Gaussian process $X$, integer $M\geq 2$, and
$j_{1:M}=[j_{1},\ldots,j_{M}]\in\mathbb{Z}^{M}$, the TV distance between the
distributions of the non-Gaussian variable
$\frac{1}{\sigma^{2}_{j_{M}}}U[j_{1:M}]X(t)$ and the square of the standard
normal random variable $\mathcal{N}$ has an upper bound

$\displaystyle
d_{\textup{TV}}\left(\frac{1}{\sigma_{j_{M}}^{2}}U[j_{1:M}]X(2^{j_{M}}t),\,\mathcal{N}^{2}\right)\leq\frac{2}{\sigma^{2}_{j_{M}}}\mathcal{U}_{j_{M}}(t,t),$
(36)

where

$\displaystyle\mathcal{U}_{j_{M}}(s,t)=\overset{2^{M-2}-1}{\underset{r=0}{\sum}}(2^{M-1}-2r)\overset{2^{M-1}-2r-1}{\underset{\ell=1}{\sum}}(\ell-1)!\binom{2^{M-1}-2r-1}{\ell-1}^{2}(2^{M}-4r-2\ell)!$
$\displaystyle{\times}\|{}^{M}\\!F^{(2^{M-1}-2r)}_{2^{j_{M}}s}\otimes_{\ell}{}^{M}\\!F^{(2^{M-1}-2r)}_{2^{j_{M}}t}\|_{\overline{H}^{\otimes
2^{M}-4r-2\ell}}$ $\displaystyle+$
$\displaystyle\overset{2^{M-2}-1}{\underset{\begin{subarray}{l}r,r^{\prime}=0\\\
r\neq
r^{\prime}\end{subarray}}{\sum}}(2^{M-1}-2r^{\prime})\overset{2^{M-1}-2(r\vee
r^{\prime})}{\underset{\ell=1}{\sum}}(\ell-1)!\binom{2^{M-1}-2r-1}{\ell-1}\binom{2^{M-1}-2r^{\prime}-1}{\ell-1}(2^{M}-2r-2r^{\prime}-2\ell)!$
$\displaystyle{\times}\|{}^{M}\\!F^{(2^{M-1}-2r)}_{2^{j_{M}}s}\otimes_{\ell}{}^{M}\\!F^{(2^{M-1}-2r^{\prime})}_{2^{j_{M}}t}\|_{\overline{H}^{\otimes
2^{M}-2r-2r^{\prime}-2\ell}}$ (37)

for $s,t\in\mathbb{R}$, and the family of functions
$\\{{}^{M}\\!F^{(2^{M-1}-2r)}_{2^{j_{M}}t}\mid r=0,1,\ldots,2^{M-2}-1\\}$ is
defined through the recursive formula (25) and (26).

In practice, the scale parameters are set with $j_{m+1}\geq j_{m}$, where
$m\in\mathbb{N}$, for the continuous wavelet transform in the scattering
network in order to use the neuron $U[j_{m+1}]$ in the $(m+1)$-th layer to
extract larger-scale features from the outputs of the neuron $U[j_{m+1}]$ in
the $m$-th layer. This is supported by the fact that the spectrum of the
magnitude of the continuous wavelet transform is mainly supported on the lower
frequency area. See [1] for more discussion. For fixed
$j_{1},j_{2},\ldots,j_{M-1}\in\mathbb{Z}$, the behavior of
$\mathcal{U}_{j_{M}}$ and $\sigma^{2}_{j_{M}}$ when $j_{M}$ is sufficiently
large will be discussed in Proposition 2, whose proof relies on the following
estimate, and its proof is given in Section 4.3.

######  Lemma 4.

For every $M\in\mathbb{N}$, there exists a constant $C_{M}>0$, which depends
on $j_{1},...,j_{M-1}$ but is independent of $j_{M}$ and $t$, such that

$\left|{}^{M}\\!\widetilde{G}^{(2^{M}-2\ell)}_{t}(\lambda_{1:2^{M}-2\ell})\right|\leq
C_{M}\overset{2^{M}-2\ell}{\underset{k=1}{\prod}}\sqrt{f_{X\star\psi_{j_{1}}}(\lambda_{k})}$
(38)

for $\ell=0,1,\ldots,2^{M-1}-1$.

######  Proposition 2.

(a) If the parameter $\alpha$ in Assumption 1 and the parameter $\beta$ in
Assumption 2 satisfy $2\alpha+\beta\geq 1$, then for any
$j_{1},\ldots,j_{M-1}\in\mathbb{Z}$, integer $M\geq 2$,
$r,r^{\prime}\in\\{0,1,\ldots,2^{M-2}-1\\}$, and $s,t\in\mathbb{R}$,

$\displaystyle
2^{\frac{3}{2}j_{M}}\left\|{}^{M}\\!F^{(2^{M-1}-2r)}_{2^{j_{M}}s}\otimes_{\ell}{}^{M}\\!F^{(2^{M-1}-2r^{\prime})}_{2^{j_{M}}t}\right\|_{\overline{H}^{\otimes
2^{M}-2r-2r^{\prime}-2\ell}}$

converges when $j_{M}\rightarrow\infty$, where
$\ell\in\\{1,2,\ldots,2^{M-1}-2r-1\\}$ for the case $r=r^{\prime}$ and
$\ell\in\\{1,2,\ldots,2^{M-1}-2(r\vee r^{\prime})\\}$ for the case $r\neq
r^{\prime}$. It further implies that
$\underset{j_{M}\rightarrow\infty}{\lim}2^{\frac{3}{2}j_{M}}\mathcal{U}_{j_{M}}(s,t)$
exists and is finite, where $\mathcal{U}_{j_{M}}$ is defined in (37).

(b) Under the same condition, i.e., $2\alpha+\beta\geq 1$,

$\displaystyle\underset{j_{M}\rightarrow\infty}{\lim}2^{j_{M}}\sigma^{2}_{j_{M}}=\overset{2^{M-2}-1}{\underset{r=0}{\sum}}M_{r}!\
c^{(M_{r})}\ \|\hat{\psi}\|^{2},$

where $M_{r}=2^{M-1}-2r$ and

$\displaystyle
c^{(M_{r})}=\int_{\mathbb{R}^{M_{r}-1}}\Big{|}{}^{M-1}\widetilde{G}^{(M_{r})}_{0}(u_{1:M_{r}-1},-u^{+}_{1:M_{r}-1})\Big{|}^{2}du_{1:M_{r}-1}.$
(39)

The condition $2\alpha+\beta\geq 1$ is used for ensuring
$\|f_{X\star\psi_{j_{1}}}\|_{\infty}<\infty$, which allows us to apply the
Lebesgue dominated convergence theorem in the proof. Note that this condition
could be easily achieved for widely used wavelets, including the Daubechies
wavelets, the Morlet wavelet, the Mexican hat wavelet, etc, since
$\alpha\geq\frac{1}{2}$ holds for those wavelets. The proofs of Lemma 4 and
Proposition 2 can be found in Section 4.3 and Section 4.4, respectively. By
Propositions 1 and 2, we get the first main result as follows.

######  Theorem 1.

Let $\psi$ be a real-valued mother wavelet satisfying Assumption 1, and $X$ be
a stationary Gaussian process satisfying Assumption 2. If the parameter
$\alpha$ in Assumption 1 and the parameter $\beta$ in Assumption 2 satisfy
$2\alpha+\beta\geq 1$, then for each fixed $t\in\mathbb{R}$, integer $M\geq
2$, and $j_{1},\ldots,j_{M-1}\in\mathbb{Z}$, when $j_{M}$ is sufficiently
large,

$d_{\textup{TV}}\left(\frac{1}{\sigma^{2}_{j_{M}}}U[j_{1:M}]X(t),\,\mathcal{N}^{2}\right)=O(2^{-\frac{1}{2}j_{M}}).$

The theorem above implies that for each fixed $t\in\mathbb{R}$,

$\frac{1}{\sigma^{2}_{j_{M}}}U[j_{1:M}]X(t)\Rightarrow\mathcal{N}^{2}$

in the distribution sense when $j_{M}\rightarrow\infty$, and hence together
with Proposition 2 we have

$2^{j_{M}}U[j_{1:M}]X(t)\Rightarrow\left[\overset{2^{M-2}-1}{\underset{r=0}{\sum}}M_{r}!\
c^{(M_{r})}\ \|\hat{\psi}\|^{2}\right]\mathcal{N}^{2}$

when $j_{M}\rightarrow\infty$.

###  3.2 Multivariate Normal Approximation of
$2^{\frac{j_{M}}{2}}T[j_{1:M}]X(2^{j_{M}}t)$

In this section, we further show that $2^{j_{M}}U[j_{1:M}]X$ converges in the
finite dimensional distribution sense and derive the covariance function of
the limiting process. Our approach is based on a result in [19]. First of all,
by the Wiener-It$\hat{\textup{o}}$ decomposition [22, Definition 2.2.4], every
$F\in L^{2}(\Omega,\mathcal{F}_{W},\mathcal{P})$ admits a unique expansion of
the type

$F=\mathbb{E}[F]+\overset{\infty}{\underset{p=1}{\sum}}\
\textup{Proj}(F\mid\mathcal{H}_{p}),$

where $\textup{Proj}(F\mid\mathcal{H}_{p})\in\mathcal{H}_{p}$ and the above
series converges in $L^{2}(\Omega,\mathcal{F}_{W},\mathcal{P})$.

######  Definition 3.

For any $F\in L^{2}{(\Omega,\mathcal{F}_{W},\mathcal{P})}$, the pseudo-inverse
of the infinitesimal generator $L$ of the Ornstein-Uhlenbeck semigroup,
denoted as $L^{-1}$, is defined as

$L^{-1}F=-\overset{\infty}{\underset{p=1}{\sum}}\ \frac{1}{p}\
\textup{Proj}(F\mid\mathcal{H}_{p}).$ (40)

The details about the semigroup of the Ornstein-Uhlenbeck semigroup and its
infinitesimal generator can be found in [22, Section 2.8]. We also need the
following lemma coming from the multidimensional Stein’s method.

######  Lemma 5 ([19]).

For any integer $d\geq 2$, let $\mathbf{F}=(F_{1},\ldots,F_{d})^{\top}$ be a
$\mathcal{F}_{W}$-measurable random vector such that
$\mathbb{E}[|F_{i}|^{4}]+\mathbb{E}[\|DF_{i}\|_{\overline{H}}^{4}]<\infty$ and
$\mathbb{E}[F_{i}]=0$ for $i=1,\ldots,d$. Let $[\mathbf{C}(i,j)]_{1\leq
i,j\leq d}$ be a symmetric non-negative definite matrix in
$\mathbb{R}^{d\times d}$, and let $\mathcal{N}_{\mathbf{C}}$ be a
$d$-dimensional normal random vector with mean zero and covariance matrix
$\mathbf{C}$. Then, for any twice differentiable function
$h:\mathbb{R}^{d}\rightarrow\mathbb{R}$ with

$\|h^{{}^{\prime\prime}}\|_{\infty}:=\underset{1\leq i,j\leq d}{\sup}\
\underset{x\in\mathbb{R}^{d}}{\sup}\ \Big{|}\frac{\partial^{2}h}{\partial
x_{i}\partial x_{j}}(x_{1},\ldots,x_{d})\Big{|}<\infty,$

we have

$\displaystyle|\mathbb{E}[h(\mathbf{F})]-\mathbb{E}[h(\mathcal{N}_{\mathbf{C}})]|\leq\frac{1}{2}\|h^{{}^{\prime\prime}}\|_{\infty}\sqrt{\overset{d}{\underset{i,j=1}{\sum}}\mathbb{E}\left[\left(\mathbf{C}(i,j)-\Big{\langle}DF_{j},-DL^{-1}F_{i}\Big{\rangle}\right)^{2}\right]}.$
(41)

With Lemma 5, we proceed to study $2^{j_{M}}U[j_{1:M}]X$ in the finite
dimensional distribution sense. We start from checking conditions needed for
Lemma 5. For any $t_{1},t_{2},\ldots,t_{d}\in\mathbb{R}$, let

$F_{i}:=2^{\frac{j_{M}}{2}}T[j_{1:M}]X(2^{j_{M}}t_{i})=2^{\frac{j_{M}}{2}}\left[\overset{2^{M-2}-1}{\underset{r=0}{\sum}}I_{2^{M-1}-2r}\left({}^{M}\\!F^{(2^{M-1}-2r)}_{2^{j_{M}}t_{i}}\right)\right]$
(42)

for $i=1,2,\ldots,d.$ By (33) and (34), the condition
$\mathbb{E}[|F_{i}|^{4}]+\mathbb{E}[\|DF_{i}\|_{\overline{H}}^{4}]<\infty$
holds. On the other hand, because $0\leq r\leq 2^{M-2}-1$ in the summation
(42), $F_{i}$ is a linear combination of Wiener chaos of order greater than or
equal to 2, which implies that $\mathbb{E}[F_{i}]=0$. Set the covariance
matrix of $(F_{1},\ldots,F_{d})^{\top}$ as
$\mathbf{C}_{j_{M}}\in\mathbb{R}^{d\times d}$; that is,

$\mathbf{C}_{j_{M}}(i,j):=\mathbb{E}[F_{i}F_{j}],$ (43)

where $1\leq i,j\leq d$, which satisfies the requirement of symmetric and non-
negative definite for the matrix $\mathbf{C}$ in Lemma 5. Moreover, we claim
that it can be expressed as

$\displaystyle\mathbf{C}_{j_{M}}(i,j)=2^{j_{M}}\overset{2^{M-2}-1}{\underset{r=0}{\sum}}M_{r}!\
{}^{M}\\!F^{(M_{r})}_{2^{j_{M}}t_{i}}\otimes_{M_{r}}{}^{M}\\!F^{(M_{r})}_{2^{j_{M}}t_{j}},$
(44)

where $M_{r}=2^{M-1}-2r$. The proof of (44) can be found in Section 4.5.
Because all conditions in Lemma 5 are satisfied, the inequality (41) holds
with $\mathbf{C}=\mathbf{C}_{j_{M}}$. Next, we deal with the right hand side
of (41). To this end, we apply the Malliavin calculus, the equality (44) and
Proposition 2. The result is summarized as follows, and its proof is relegated
to Section 4.6.

######  Theorem 2.

Let $\psi$ be a real-valued mother wavelet satisfying Assumption 1, and $X$ be
a stationary Gaussian process satisfying Assumption 2. Given integers $d,M\geq
2$, $j_{1},\ldots,{j_{M}}\in\mathbb{Z}$, and
$t_{1},t_{2},\ldots,t_{d}\in\mathbb{R}$, define

$\mathbf{F}=\left(2^{\frac{j_{M}}{2}}T[j_{1:M}]X(2^{j_{M}}t_{1}),\ldots,2^{\frac{j_{M}}{2}}T[j_{1:M}]X(2^{j_{M}}t_{d})\right)^{\top}{\in\mathbb{R}^{d}}$

and let $\mathbf{C}_{j_{M}}$ be its covariance matrix. If the parameter
$\alpha$ in Assumption 1 and the parameter $\beta$ in Assumption 2 satisfy
$2\alpha+\beta\geq 1$, then for any twice differentiable function
$h:\mathbb{R}^{d}\rightarrow\mathbb{R}$ with
$\|h^{{}^{\prime\prime}}\|_{\infty}<\infty,$ we have

$\displaystyle\left|\mathbb{E}[h(\mathbf{F})]-\mathbb{E}[h(\mathcal{N}_{\mathbf{C}_{j_{M}}})]\right|\leq\frac{1}{2}\|h^{{}^{\prime\prime}}\|_{\infty}\overset{d}{\underset{i,j=1}{\sum}}2^{j_{M}}\mathcal{U}_{j_{M}}(t_{j},t_{i})=\|h^{{}^{\prime\prime}}\|_{\infty}O(2^{-\frac{1}{2}j_{M}})$
(45)

when $j_{M}$ is sufficiently large, where $\mathcal{N}_{\mathbf{C}_{j_{M}}}$
is a $d$-dimensional normal random vector with mean zero and covariance matrix
$\mathbf{C}_{j_{M}}$.

By considering the special cases
$h(\mathbf{x})=\cos(\langle\zeta,\mathbf{x}\rangle)$ and
$h(\mathbf{x})=\sin(\langle\zeta,\mathbf{x}\rangle)$, where
$\mathbf{x},\zeta\in\mathbb{R}^{d}$, (45) implies that the distance between
the characteristic function of the random vector $\mathbf{F}$ and that of
$\mathcal{N}_{\mathbf{C}_{j_{M}}}$ converges to zero as
$j_{M}\rightarrow\infty$. More precisely,

$\Big{|}\mathbb{E}\left[e^{i\langle\zeta,\,\mathbf{F}\rangle}\right]-\mathbb{E}\left[e^{i\langle\zeta,\,\mathcal{N}_{\mathbf{C}_{j_{M}}}\rangle}\right]\Big{|}\leq|\zeta|^{2}O(2^{-\frac{1}{2}j_{M}}).$
(46)

The following proposition shows that the covariance matrix
$\mathbf{C}_{j_{M}}$ converges as $j_{M}\rightarrow\infty$.

######  Proposition 3.

Let the assumptions and notation of Theorem 2 prevail. Then, for any
$i,j\in\\{1,2,\ldots,d\\}$, we have

$\displaystyle\mathbf{C}_{\infty}(i,j):=\underset{j_{M}\rightarrow\infty}{\lim}\mathbf{C}_{j_{M}}(i,j)=\left[\overset{2^{M-2}-1}{\underset{r=0}{\sum}}M_{r}!\
c^{(M_{r})}\right]\int_{\mathbb{R}}e^{i(t_{i}-t_{j})z}\left|\hat{\psi}(z)\right|^{2}\
dz,$ (47)

where $M_{r}=2^{M-1}-2r$, the constant $c^{(M_{r})}$ is defined in (39).

The proof of Proposition 3 is in Section 4.7. Let
$\mathcal{N}_{\mathbf{C}_{\infty}}$ be a $d$-dimensional normal random vector
with mean zero and covariance matrix $\mathbf{C}_{\infty}$. Because both
$\mathcal{N}_{\mathbf{C}_{j_{M}}}$ and $\mathcal{N}_{\mathbf{C}_{\infty}}$ are
normal random vectors with mean 0, Proposition 3 implies that for any
$\zeta=[\zeta_{1},\ldots,\zeta_{d}]\in\mathbb{R}^{d}$

$\mathbb{E}\left[e^{i\langle\zeta,\,\mathcal{N}_{\mathbf{C}_{j_{M}}}\rangle}\right]\rightarrow\mathbb{E}\left[e^{i\langle\zeta,\,\mathcal{N}_{\mathbf{C}_{\infty}}\rangle}\right]$
(48)

when $j_{M}\rightarrow\infty$. By combining (46) and (48), we get

$\displaystyle\Big{|}\mathbb{E}\left[e^{i\langle\zeta,\,\mathbf{F}\rangle}\right]-\mathbb{E}\left[e^{i\langle\zeta,\,\mathcal{N}_{\mathbf{C}_{\infty}}\rangle}\right]\Big{|}$
$\displaystyle\leq$
$\displaystyle\Big{|}\mathbb{E}\left[e^{i\langle\zeta,\,\mathbf{F}\rangle}\right]-\mathbb{E}\left[e^{i\langle\zeta,\,\mathcal{N}_{\mathbf{C}_{j_{M}}}\rangle}\right]\Big{|}+\Big{|}\mathbb{E}\left[e^{i\langle\zeta,\,\mathcal{N}_{\mathbf{C}_{j_{M}}}\rangle}\right]-\mathbb{E}\left[e^{i\langle\zeta,\,\mathcal{N}_{\mathbf{C}_{\infty}}\rangle}\right]\Big{|}\rightarrow
0$

when $j_{M}\rightarrow\infty$. Therefore, for any $i,j\in\\{1,2,\ldots,d\\}$,
when $j_{M}\rightarrow\infty$,

$\displaystyle\mathbb{E}\left[e^{i\langle\zeta,\,\mathbf{F}\rangle}\right]\rightarrow\mathbb{E}\left[e^{i\langle\zeta,\,\mathcal{N}_{\mathbf{C}_{\infty}}\rangle}\right]$

for all $\zeta\in\mathbb{R}^{d}$, where $\mathcal{N}_{\mathbf{C}_{\infty}}$ is
a $d$-dimensional normal random vector with mean zero and covariance matrix
$\mathbf{C}_{\infty}$ given in (47). It implies that the rescaled random
process $\\{2^{\frac{j_{M}}{2}}T[j_{1:M}]X(2^{j_{M}}t)\mid t\in\mathbb{R}\\}$
converges to a Gaussian process, denoted by $\\{G(t)\mid t\in\mathbb{R}\\}$,
in the finite dimensional distribution sense and

$\textup{Cov}\left(G(t_{1}),G(t_{2})\right)=\left[\overset{2^{M-2}-1}{\underset{r=0}{\sum}}M_{r}!\
c^{(M_{r})}\right]\int_{\mathbb{R}}e^{i(t_{1}-t_{2})z}\Big{|}\hat{\psi}(z)\Big{|}^{2}\
dz,\ \ t_{1},t_{2}\in\mathbb{R}.$

By the continuous mapping theorem, we get a corresponding result for the
rescaled random process $\\{2^{j_{M}}U[j_{1:M}]X(2^{j_{M}}t)\mid
t\in\mathbb{R}\\}$ as follows.

######  Theorem 3.

Let the assumptions and notation of Theorem 2 prevail. The rescaled random
process $\\{2^{j_{M}}U[j_{1:M}]X(2^{j_{M}}t)\mid t\in\mathbb{R}\\}$ converges
to the square of the stationary Gaussian process $\\{G(t)\mid
t\in\mathbb{R}\\}$ in the finite dimensional distribution sense. Moreover, the
covariance function of the limiting process $G^{2}$ has the following
expression

$\displaystyle\textup{Cov}\left(G^{2}(t_{1}),G^{2}(t_{2})\right)=$
$\displaystyle\,2\left[\overset{2^{M-2}-1}{\underset{r=0}{\sum}}M_{r}!\
c^{(M_{r})}\right]^{2}\left[\int_{\mathbb{R}}e^{i(t_{1}-t_{2})z}|\hat{\psi}(z)|^{2}\
dz\right]^{2}$ $\displaystyle=$
$\displaystyle\,2\left[\textup{Cov}\left(G(t_{1}),G(t_{2})\right)\right]^{2}$
(49)

for $t_{1},t_{2}\in\mathbb{R},$ where $M_{r}=2^{M-1}-2r$ and the constant
$c^{(M_{r})}$ is defined in (39).

The derivation of (49) can be found in Section 4.8. Theorem 3 implies that
when $j_{M}\rightarrow\infty$,

$2^{j_{M}}U[j_{1:M}]X(2^{j_{M}}t)\Rightarrow\left[\overset{2^{M-2}-1}{\underset{r=0}{\sum}}M_{r}!\
c^{(M_{r})}\ \|\hat{\psi}\|^{2}\right]\chi^{2}(t)$

in the finite dimensional distribution sense, where $\\{\chi^{2}(t)\mid
t\in\mathbb{R}\\}$ is a chi-square process with one degree of freedom and

$\displaystyle\textup{Cov}\left(\chi^{2}(t_{1}),\chi^{2}(t_{2})\right)=$
$\displaystyle
2\left[\int_{\mathbb{R}}e^{i(t_{1}-t_{2})z}\left(\frac{|\hat{\psi}(z)|}{\|\hat{\psi}\|_{2}}\right)^{2}\
dz\right]^{2}.$

######  Remark 3.

It is worth mentioning that in our previous work [16], we considered the
second-order ST with stationary Gaussian processes and Gaussian processes with
stationary increments as inputs, where we apply the Wiener chaos expansion and
the Feynman diagram method to derive the central and non-central limit
theorem, and the delicate interaction between mother wavelets and activation
functions are explored. Different from [16], the analysis techniques in this
work allows us to get a general form for the Wiener-It$\hat{\textup{o}}$
decomposition of the outputs of STQNs without depth limitation, and the
current approach is non-asymptotic, i.e., the Stein’s method is applied to
estimate the distance between the outputs of STQNs and their scaling limits.
For commonly used wavelet functions, the current work provides information
about the convergence speed of the rescaled random processes arising from
STQNs, which cannot be obtained from our previous work. Furthermore, the
technique used in the current work can be applied to study the same issue for
polynomial activated STs by using more complicated product formula to obtain
the Wiener-It$\hat{\textup{o}}$ decomposition of the corresponding outputs,
which is not explored in this work.

##  4 Proof

###  4.1 Proof of Lemma 2

In the following, we prove (23) and (24) by the mathematical induction method.

$\bullet$ When $M=2$, not only (25) and (26) can be simplified as (21) and
(18) respectively, but also (23) and (24) become (19) and (20). Hence, the
statement in Lemma 2 holds for $M=2$.

$\bullet$ Suppose that the statement in Lemma 2 holds for $M=m$, where $m\geq
2.$

$\bullet$ Now we start to prove the statement in Lemma 2 for $M=m+1$. By the
definition (2),

$\displaystyle T[j_{1:m+1}]X(t)=$
$\displaystyle\,U[j_{1:m}]X\star\psi_{j_{m+1}}(t)$ $\displaystyle=$
$\displaystyle\,\int_{\mathbb{R}}\overset{2^{m-1}}{\underset{\ell=0}{\sum}}I_{2^{m}-2\ell}\left({}^{m}\widetilde{G}^{(2^{m}-2\ell)}_{t^{{}^{\prime}}}\right)\psi_{j_{m+1}}(t-t^{{}^{\prime}})dt^{\prime},$
(50)

where the second equality follows from the induction hypothesis. By (27),

$\displaystyle\int_{\mathbb{R}}{}^{m}\widetilde{G}^{(2^{m}-2\ell)}_{t^{{}^{\prime}}}(\lambda_{1:2^{m}-2\ell})\psi_{j_{m+1}}(t-t^{{}^{\prime}})dt^{\prime}={}^{m}\widetilde{G}^{(2^{m}-2\ell)}_{t}(\lambda_{1:2^{m}-2\ell})\hat{\psi}_{j_{m+1}}(\lambda^{+}_{1:2^{m}-2\ell}).$
(51)

We recognize that the right hand side of (51) is just the right hand side of
(26), i.e, ${}^{m+1}F^{(2^{m}-2\ell)}_{t}(\lambda_{1:2^{m}-2\ell})$. By the
stochastic Fubini theorem [25, Theorem 2.1] with (50) and (51), we get

$\displaystyle
T[j_{1:m+1}]X(t)=\overset{2^{m-1}}{\underset{\ell=0}{\sum}}I_{2^{m}-2\ell}\left({}^{m+1}F^{(2^{m}-2\ell)}_{t}\right),$
(52)

that is, (23) with $M=m+1$. The validation of the application of the
stochastic Fubini theorem to get (52) can be found in A. Next, by the
relationship (3) and (52),

$\displaystyle
U[j_{1:m+1}]X(t)=\left[T[j_{1:m+1}]X(t)\right]^{2}=\left[\overset{2^{m-1}}{\underset{r=0}{\sum}}I_{2^{m}-2r}\left({}^{m+1}F^{(2^{m}-2r)}_{t}\right)\right]^{2}$
$\displaystyle=$
$\displaystyle\,\overset{2^{m-1}}{\underset{r,r^{\prime}=0}{\sum}}I_{2^{m}-2r}\left({}^{m+1}F^{(2^{m}-2r)}_{t}\right)I_{2^{m}-2r^{\prime}}\left({}^{m+1}F^{(2^{m}-2r^{\prime})}_{t}\right)$
$\displaystyle=$
$\displaystyle\,\overset{2^{m-1}}{\underset{r,r^{\prime}=0}{\sum}}\overset{2^{m}-2(r\vee
r^{\prime})}{\underset{p=0}{\sum}}p!\binom{2^{m}-2r}{p}\binom{2^{m}-2r^{\prime}}{p}\
I_{2^{m+1}-2(r+r^{\prime}+p)}\left({}^{m+1}F^{(2^{m}-2r)}_{t}\otimes_{p}{}^{m+1}F^{(2^{m}-2r^{\prime})}_{t}\right),$

where the last equality follows from the product formula, i.e., Lemma 1.
Denote $\ell=r+r^{\prime}+p$. Because $0\leq p\leq 2^{m}-2(r\vee r^{\prime})$,
we have $r+r^{\prime}\leq\ell\leq 2^{m}-|r-r^{{}^{\prime}}|$. Hence,

$\displaystyle U[j_{1:m+1}]X(t)=$
$\displaystyle\overset{2^{m-1}}{\underset{r,r^{\prime}=0}{\sum}}\
\overset{2^{m}}{\underset{\ell=0}{\sum}}1_{\\{r+r^{\prime}\leq\ell\leq
2^{m}-|r-r^{{}^{\prime}}|\\}}(\ell-r-r^{\prime})!\binom{2^{m}-2r}{\ell-
r-r^{\prime}}\binom{2^{m}-2r^{\prime}}{\ell-r-r^{\prime}}$
$\displaystyle\times
I_{2^{m+1}-2\ell}\left({}^{m+1}F^{(2^{m}-2r)}_{t}\otimes_{\ell-
r-r^{\prime}}{}^{m+1}F^{(2^{m}-2r^{\prime})}_{t}\right),$

where the indicator function $1_{\\{r+r^{\prime}\leq\ell\leq
2^{m}-|r-r^{{}^{\prime}}|\\}}$ can be ignored by setting

$\binom{2^{m}-2r}{\ell-r-r^{\prime}}\binom{2^{m}-2r^{\prime}}{\ell-
r-r^{\prime}}=0\ \textup{if}\ \ell-r-r^{\prime}<0\ \textup{or}\
\min\left\\{2^{m}-2r,2^{m}-2r^{\prime}\right\\}<\ell-r-r^{\prime}.$

By the linearity of the Wiener integrals,

$\displaystyle U[j_{1:m+1}]X(t)=$
$\displaystyle\,\overset{2^{m}}{\underset{\ell=0}{\sum}}I_{2^{m+1}-2\ell}\left(\overset{2^{m-1}}{\underset{r,r^{\prime}=0}{\sum}}(\ell-
r-r^{\prime})!\binom{2^{m}-2r}{\ell-
r-r^{\prime}}\binom{2^{m}-2r^{\prime}}{\ell-r-r^{\prime}}\right.$
$\displaystyle\left.\times\ \ {}^{m+1}F^{(2^{m}-2r)}_{t}\otimes_{\ell-
r-r^{\prime}}{}^{m+1}F^{(2^{m}-2r^{\prime})}_{t}\right).$

We recognize that the function inside the integrand of the
$(2^{m+1}-2\ell)$-fold Wiener integrals above coincides with the right hand
side of (25) with $M=m+1$, i.e.,
${}^{m+1}\widetilde{G}^{(2^{m+1}-2\ell)}_{t}$. Hence,

$\displaystyle U[j_{1:m+1}]X(t)=$
$\displaystyle\,\overset{2^{m}}{\underset{\ell=0}{\sum}}I_{2^{m+1}-2\ell}\left({}^{m+1}\widetilde{G}^{(2^{m+1}-2\ell)}_{t}\right),$

which is just (24) with $M=m+1$. Therefore, we finish the inductive step and
the proof.

###  4.2 Proof of Proposition 1

Because the distribution of $T[j_{1:M}]X(2^{j_{M}}t)$ does not depend on the
time variable, we will neglect the time variable of $T[j_{1:M}]X$ temporarily.
The Stein’s equation associated with $h$ is the ordinary differential equation

$f^{{}^{\prime}}(x)-xf(x)=h(x)-\mathbb{E}[h(\mathcal{N})].$ (53)

Let $f_{h}$ be the solution of (53) with
$\underset{|x|\rightarrow\infty}{\lim}f_{h}(x)e^{-\frac{x^{2}}{2}}=0$.
According to [22, Proposition 3.2.2],

$\displaystyle
f_{h}(x)=e^{\frac{x^{2}}{2}}\int_{-\infty}^{x}\left\\{h(y)-\mathbb{E}\left[h(\mathcal{N})\right]\right\\}e^{-\frac{y^{2}}{2}}dy.$

By (53),

$\displaystyle\mathbb{E}\left[h\left(\frac{1}{\sigma_{j_{M}}}T[j_{1:M}]X\right)\right]-\mathbb{E}\left[h\left(\mathcal{N}\right)\right]$
$\displaystyle=$
$\displaystyle\,\mathbb{E}\left[f_{h}^{{}^{\prime}}\left(\frac{1}{\sigma_{j_{M}}}T[j_{1:M}]X\right)\right]-\mathbb{E}\left[\frac{1}{\sigma_{j_{M}}}T[j_{1:M}]X\
f_{h}\left(\frac{1}{\sigma_{j_{M}}}T[j_{1:M}]X\right)\right].$ (54)

By (23), i.e.,
$T[j_{1:M}]X(2^{j_{M}}t)=\overset{2^{M-2}-1}{\underset{r=0}{\sum}}I_{2^{M-1}-2r}\left({}^{M}\\!F^{(2^{M-1}-2r)}_{2^{j_{M}}t}\right)$,
and Lemma 3, we have

$\displaystyle T[j_{1:M}]X(2^{j_{M}}t)=\delta
D\left(\overset{2^{M-2}-1}{\underset{r=0}{\sum}}\frac{1}{2^{M-1}-2r}I_{2^{M-1}-2r}\left({}^{M}\\!F^{(2^{M-1}-2r)}_{2^{j_{M}}t}\right)\right).$

Hence,

$\displaystyle\mathbb{E}\left[\frac{1}{\sigma_{j_{M}}}T[j_{1:M}]X\
f_{h}\left(\frac{1}{\sigma_{j_{M}}}T[j_{1:M}]X\right)\right]$ $\displaystyle=$
$\displaystyle\,\frac{1}{\sigma_{j_{M}}}\mathbb{E}\left[\delta
D\left(\overset{2^{M-2}-1}{\underset{r=0}{\sum}}\frac{1}{2^{M-1}-2r}I_{2^{M-1}-2r}\left({}^{M}\\!F^{(2^{M-1}-2r)}_{2^{j_{M}}t}\right)\right)f_{h}\left(\frac{1}{\sigma_{j_{M}}}T[j_{1:M}]X\right)\right]$
$\displaystyle=$
$\displaystyle\,\frac{1}{\sigma^{2}_{j_{M}}}\mathbb{E}\left[\Big{\langle}D\left(\overset{2^{M-2}-1}{\underset{r=0}{\sum}}\frac{1}{M_{r}}I_{M_{r}}\left({}^{M}\\!F^{(M_{r})}_{2^{j_{M}}t}\right)\right),f_{h}^{{}^{\prime}}\left(\frac{1}{\sigma_{j_{M}}}T[j_{1:M}]X\right)DT[j_{1:M}]X\Big{\rangle}\right],$
(55)

where the last equality follows from (31) and the chain rule. By substituting
(55) into (54), we get

$\displaystyle\mathbb{E}\left[h\left(\frac{1}{\sigma_{j_{M}}}T[j_{1:M}]X\right)\right]-\mathbb{E}\left[h(\mathcal{N})\right]$
$\displaystyle=$
$\displaystyle\,\mathbb{E}\left\\{f_{h}^{{}^{\prime}}\left(\frac{1}{\sigma_{j_{M}}}T[j_{1:M}]X\right)\left[1-\frac{1}{\sigma^{2}_{j_{M}}}\Big{\langle}D\left(\overset{2^{M-2}-1}{\underset{r=0}{\sum}}\frac{1}{M_{r}}I_{M_{r}}\left({}^{M}\\!F^{(M_{r})}_{2^{j_{M}}t}\right)\right),\,DT[j_{1:M}]X\Big{\rangle}\right]\right\\}.$

By [22, Theorem 3.3.1], if the Borel function $h$ is assumed to take values in
$[0,1]$, the solution of Stein’s equation associated with $h$ satisfies
$\|f_{h}\|_{\infty}\leq\sqrt{\pi/2}$ and $\|f_{h}^{{}^{\prime}}\|_{\infty}\leq
2$. Hence,

$\displaystyle
d_{\textup{TV}}\left(\frac{1}{\sigma_{j_{M}}}T[j_{1:M}]X,\,\mathcal{N}\right)\leq
2\mathbb{E}\Big{[}\Big{|}1-\frac{1}{\sigma^{2}_{j_{M}}}\Big{\langle}D\left(\overset{2^{M-2}-1}{\underset{r=0}{\sum}}\frac{1}{M_{r}}I_{M_{r}}\left({}^{M}\\!F^{(M_{r})}_{2^{j_{M}}t}\right)\right),DT[j_{1:M}]X\Big{\rangle}\Big{|}\Big{]}.$
(56)

By applying (32) in Lemma 3 to the right hand side of (56),

$\displaystyle
d_{\textup{TV}}\left(\frac{1}{\sigma_{j_{M}}}T[j_{1:M}]X,\,\mathcal{N}\right)\leq
2\mathbb{E}\left\\{\Big{|}1-\frac{1}{\sigma^{2}_{j_{M}}}\mathcal{I}\Big{|}\right\\},$
(57)

where

$\mathcal{I}:=\Big{\langle}\overset{2^{M-2}-1}{\underset{r=0}{\sum}}I_{M_{r}-1}\left({}^{M}\\!F^{(M_{r})}_{2^{j_{M}}t}\right),\overset{2^{M-2}-1}{\underset{r=0}{\sum}}M_{r}I_{M_{r}-1}\left({}^{M}\\!F^{(M_{r})}_{2^{j_{M}}t}\right)\Big{\rangle}.$

Note that

$\displaystyle\mathbb{E}\left[\mathcal{I}\right]=\overset{2^{M-2}-1}{\underset{r=0}{\sum}}M_{r}!\
\ \|{}^{M}\\!F^{(M_{r})}_{t}\|_{\overline{H}^{\otimes
M_{r}}}^{2}=\sigma_{j_{M}}^{2}.$

By applying Jensen’s inequality to (57), we get

$\displaystyle
d_{\textup{TV}}\left(\frac{1}{\sigma_{j_{M}}}T[j_{1:M}]X,\,\mathcal{N}\right)\leq
2\left\\{\mathbb{E}\left[\Big{|}1-\frac{1}{\sigma^{2}_{j_{M}}}\mathcal{I}\Big{|}^{2}\right]\right\\}^{\frac{1}{2}}=2\sqrt{\textup{Var}\left(\frac{1}{\sigma^{2}_{j_{M}}}\mathcal{I}\right)}.$
(58)

Before computing the variance in (58), we expand $\mathcal{I}$ by Lemma 1 as
follows

$\displaystyle\mathcal{I}=$
$\displaystyle\,\sigma_{j_{M}}^{2}+\overset{2^{M-2}-1}{\underset{r=0}{\sum}}M_{r}\overset{M_{r}-1}{\underset{\ell=1}{\sum}}(\ell-1)!\binom{M_{r}-1}{\ell-1}^{2}I_{2M_{r}-2\ell}\left({}^{M}\\!F^{(M_{r})}_{2^{j_{M}}t}\otimes_{\ell}{}^{M}\\!F^{(M_{r})}_{t}\right)$
$\displaystyle+\overset{2^{M-2}-1}{\underset{\begin{subarray}{l}r,r^{\prime}=0\\\
r\neq r^{\prime}\end{subarray}}{\sum}}M_{r^{\prime}}\overset{M_{r}\wedge
M_{r^{\prime}}}{\underset{\ell=1}{\sum}}(\ell-1)!\binom{M_{r}-1}{\ell-1}\binom{M_{r^{\prime}}-1}{\ell-1}I_{M_{r}+M_{r^{\prime}}-2\ell}\left({}^{M}\\!F^{(M_{r})}_{2^{j_{M}}t}\otimes_{\ell}{}^{M}\\!F^{(M_{r^{\prime}})}_{2^{j_{M}}t}\right).$

By the Minkowski inequality,

$\displaystyle\sqrt{\textup{Var}\left(\mathcal{I}\right)}=\mathbb{E}\left\\{\left[\Big{|}\mathcal{I}-\sigma_{j_{M}}^{2}\Big{|}^{2}\right]\right\\}^{\frac{1}{2}}$
$\displaystyle\leq$
$\displaystyle\overset{2^{M-2}-1}{\underset{r=0}{\sum}}M_{r}\overset{M_{r}-1}{\underset{\ell=1}{\sum}}(\ell-1)!\binom{M_{r}-1}{\ell-1}^{2}\left\\{\mathbb{E}\left[\Big{|}I_{2M_{r}-2\ell}\left({}^{M}\\!F^{(M_{r})}_{2^{j_{M}}t}\otimes_{\ell}{}^{M}\\!F^{(M_{r})}_{2^{j_{M}}t}\right)\Big{|}^{2}\right]\right\\}^{\frac{1}{2}}$
(59)
$\displaystyle+\overset{2^{M-2}-1}{\underset{\begin{subarray}{l}r,r^{\prime}=0\\\
r\neq r^{\prime}\end{subarray}}{\sum}}M_{r^{\prime}}\overset{M_{r}\wedge
M_{r^{\prime}}}{\underset{\ell=1}{\sum}}(\ell-1)!\binom{M_{r}-1}{\ell-1}\binom{M_{r^{\prime}}-1}{\ell-1}\left\\{\mathbb{E}\left[\Big{|}I_{M_{r}+M_{r^{\prime}}-2\ell}\left({}^{M}\\!F^{(M_{r})}_{2^{j_{M}}t}\otimes_{\ell}{}^{M}\\!F^{(M_{r^{\prime}})}_{2^{j_{M}}t}\right)\Big{|}^{2}\right]\right\\}^{\frac{1}{2}}.$

For the expectations above, the isometry property of the multiple Wiener-
It$\hat{\textup{o}}$ integrals implies that

$\displaystyle\mathbb{E}\left[\Big{|}I_{M_{r}+M_{r^{\prime}}-2\ell}\left({}^{M}\\!F^{(M_{r})}_{2^{j_{M}}t}\otimes_{\ell}{}^{M}\\!F^{(M_{r^{\prime}})}_{2^{j_{M}}t}\right)\Big{|}^{2}\right]=(M_{r}+M_{r^{\prime}}-2\ell)!\
\
\left\|{}^{M}\\!F^{(M_{r})}_{2^{j_{M}}t}\otimes_{\ell}{}^{M}\\!F^{(M_{r^{\prime}})}_{2^{j_{M}}t}\right\|^{2}.$

Therefore, the upper bound in (36) follows from combining (58) and (59).

###  4.3 Proof of Lemma 4

We prove this lemma by mathematical induction. For $M=1$, (15) implies that
(38) holds with $C_{1}=1$. Suppose that (38) holds for $M=m-1$, where $m$ is
an integer greater than or equal to 2. By (25),

$\displaystyle{}^{m}\widetilde{G}^{(2^{m}-2\ell)}_{t}=\overset{2^{m-2}-1}{\underset{r,r^{{}^{\prime}}=0}{\sum}}(\ell-
r-r^{\prime})!\binom{2^{m-1}-2r}{\ell-
r-r^{\prime}}\binom{2^{m-1}-2r^{\prime}}{\ell-
r-r^{\prime}}{}^{m}\\!F^{(2^{m-1}-2r)}_{t}\widetilde{\otimes}_{\ell-
r-r^{\prime}}{}^{m}\\!F^{(2^{m-1}-2r^{\prime})}_{t}.$ (60)

From (26), we know that for $r=0,1,\ldots,2^{m-2}-1$,

$\displaystyle\Big{|}{}^{m}\\!F^{(2^{m-1}-2r)}_{t}(x_{1},\ldots,x_{2^{m-1}-2r})\Big{|}=$
$\displaystyle\,\Big{|}{}^{m-1}\widetilde{G}^{(2^{m-1}-2r)}_{t}(x_{1},\ldots,x_{2^{m-1}-2r})\hat{\psi}_{j_{m}}(x_{1},\ldots,x_{2^{m-1}-2r})\Big{|}$
$\displaystyle\leq$
$\displaystyle\,\|\hat{\psi}\|_{\infty}\Big{|}{}^{m-1}\widetilde{G}^{(2^{m-1}-2r)}_{t}(x_{1},\ldots,x_{2^{m-1}-2r})\Big{|}$
$\displaystyle\leq$
$\displaystyle\,\|\hat{\psi}\|_{\infty}\left[C_{m-1}\overset{2^{m-1}-2r}{\underset{k=1}{\prod}}\sqrt{f_{X\star\psi_{j_{1}}}(x_{k})}\right],$

where the last inequality is obtained from the assumption, i.e., (60) with
$M=m-1$. Hence,

$\displaystyle\Big{|}{}^{m}\\!F^{(2^{m-1}-2r)}_{t}\widetilde{\otimes}_{\ell-
r-r^{\prime}}{}^{m}\\!F^{(2^{m-1}-2r^{\prime})}_{t}(\lambda_{1:2^{m}-2\ell})\Big{|}\leq\|\hat{\psi}\|^{2}_{\infty}C_{m-1}^{2}\|f_{X\star\psi_{j_{1}}}\|_{1}^{\ell-
r-r^{\prime}}\overset{2^{m}-2\ell}{\underset{k=1}{\prod}}\sqrt{f_{X\star\psi_{j_{1}}}(\lambda_{k})}.$

By substituting the inequality above into (60), we get

$\displaystyle\Big{|}{}^{m}\widetilde{G}^{(2^{m}-2\ell)}_{t}(\lambda_{1:2^{m}-2\ell})\Big{|}\leq$
$\displaystyle\,\left[\overset{2^{m-2}-1}{\underset{r,r^{{}^{\prime}}=0}{\sum}}(\ell-
r-r^{\prime})!\binom{2^{m-1}-2r}{\ell-
r-r^{\prime}}\binom{2^{m-1}-2r^{\prime}}{\ell-
r-r^{\prime}}\|f_{X\star\psi_{j_{1}}}\|_{1}^{\ell-r-r^{\prime}}\right]$
$\displaystyle\times\left[\|\hat{\psi}\|^{2}_{\infty}C_{m-1}^{2}\overset{2^{m}-2\ell}{\underset{k=1}{\prod}}\sqrt{f_{X\star\psi_{j_{1}}}(\lambda_{k})}\right].$

By the principle of mathematical induction, (38) holds for all
$M\in\mathbb{N}$.

###  4.4 Proof of Proposition 2

(a) For $r,r^{\prime}\in\\{0,1,\ldots,2^{M-2}-1\\}$, by the recursive
relationship (26),

$\displaystyle\left\|{}^{M}\\!F^{(M_{r})}_{2^{j_{M}}s}\otimes_{\ell}{}^{M}\\!F^{(M_{r^{\prime}})}_{2^{j_{M}}t}\right\|^{2}_{\overline{H}^{\otimes
M_{r}+M_{r^{\prime}}-2\ell}}$ $\displaystyle=$
$\displaystyle\int_{\mathbb{R}^{M_{r}+M_{r^{\prime}}-2\ell}}\Big{[}\int_{\mathbb{R}^{\ell}}{}^{M-1}\widetilde{G}^{(M_{r})}_{2^{j_{M}}s}(\tau_{1:\ell},\lambda_{1:M_{r}-\ell})\hat{\psi}_{j_{M}}(\tau^{+}_{1:\ell}+\lambda^{+}_{1:M_{r}-\ell})$
$\displaystyle\times{}^{M-1}\widetilde{G}^{(M_{r^{\prime}})}_{2^{j_{M}}t}(-\tau_{1:\ell},\lambda_{M_{r}-\ell+1:M_{r}+M_{r^{\prime}}-2\ell})\hat{\psi}_{j_{M}}(-\tau^{+}_{1:\ell}+\lambda^{+}_{M_{r}-\ell+1:M_{r}+M_{r^{\prime}}-2\ell}){d\tau_{1:\ell}}\Big{]}$
$\displaystyle\times\Big{[}\overline{\int_{\mathbb{R}^{\ell}}{}^{M-1}\widetilde{G}^{(M_{r})}_{2^{j_{M}}s}(\eta_{1:\ell},\lambda_{1:M_{r}-\ell})\hat{\psi}_{j_{M}}(\eta^{+}_{1:\ell}+\lambda^{+}_{1:M_{r}-\ell})}$
$\displaystyle\overline{\times{}^{M-1}\widetilde{G}^{(M_{r^{\prime}})}_{2^{j_{M}}t}(-\eta_{1:\ell},\lambda_{M_{r}-\ell+1:M_{r}+M_{r^{\prime}}-2\ell})\hat{\psi}_{j_{M}}(-\eta^{+}_{1:\ell}+\lambda^{+}_{M_{r}-\ell+1:M_{r}+M_{r^{\prime}}-2\ell}){d\eta_{1:\ell}}}\Big{]}d\lambda_{1:M_{r}+M_{r^{\prime}}-2\ell},$

where $\ell\in\\{1,\ldots,M_{r}-1\\}$ when $r=r^{\prime}$, and
$\ell\in\\{1,\ldots,M_{r}\wedge M_{r^{\prime}}\\}$ when $r\neq r^{\prime}$. By
considering the change of variables

$\displaystyle\left\\{\begin{array}[]{l}u_{i}=\tau_{i},\ i=1,\ldots,\ell-1\ \
\textup{for}\ \ell>1,\\\ v_{i}=\eta_{i},\ i=1,\ldots,\ell-1\ \ \textup{for}\
\ell>1,\\\ w_{i}=\lambda_{i},\ i=1,\ldots,M_{r}+M_{r^{\prime}}-2\ell-1,\\\
x=2^{j_{M}}\left(\tau^{+}_{1:\ell}+\lambda^{+}_{1:M_{r}-\ell}\right),\\\
y=2^{j_{M}}\left(-\tau^{+}_{1:\ell}+\lambda^{+}_{M_{r}-\ell+1:M_{r}+M_{r^{\prime}}-2\ell}\right),\\\
z=2^{j_{M}}\left(\eta^{+}_{1:\ell}+\lambda^{+}_{1:M_{r}-\ell}\right)\end{array}\right.$
(67)

and noting that $\hat{\psi}_{j_{M}}(\cdot)=\hat{\psi}(2^{j_{M}}\cdot)$, we
have

$\displaystyle
2^{3j_{M}}\left\|{}^{M}\\!F^{(M_{r})}_{2^{j_{M}}s}\otimes_{\ell}{}^{M}\\!F^{(M_{r^{\prime}})}_{2^{j_{M}}t}\right\|^{2}_{\overline{H}^{\otimes
M_{r}+M_{r^{\prime}}-2\ell}}$ $\displaystyle=$
$\displaystyle\,\int_{\mathbb{R}^{M_{r}+M_{r^{\prime}}}}{}^{M-1}\widetilde{G}^{(M_{r})}_{2^{j_{M}}s}(u_{1:\ell-1},u^{*}_{\ell},w_{1:M_{r}-\ell})$
$\displaystyle{\times}{}^{M-1}\widetilde{G}^{(M_{r^{\prime}})}_{2^{j_{M}}t}(-u_{1:\ell-1},-u^{*}_{\ell},w_{M_{r}-\ell+1:M_{r}+M_{r^{\prime}}-2\ell-1},w^{*}_{M_{r}+M_{r^{\prime}}-2\ell})$
$\displaystyle\times\overline{{}^{M-1}\widetilde{G}^{(M_{r})}_{2^{j_{M}}s}(v_{1:\ell-1},v_{\ell}^{*},w_{1:M_{r}-\ell})}$
$\displaystyle{\times}\overline{{}^{M-1}\widetilde{G}^{(M_{r^{\prime}})}_{2^{j_{M}}t}(-v_{1:\ell-1},-v^{*}_{\ell},w_{M_{r}-\ell+1:M_{r}+M_{r^{\prime}}-2\ell-1},w^{*}_{M_{r}+M_{r^{\prime}}-2\ell})}$
$\displaystyle\times\hat{\psi}(x)\hat{\psi}(y)\overline{\hat{\psi}(z)\hat{\psi}(x+y-z)}\
du_{1:\ell-1}\ dv_{1:\ell-1}\ dw_{1:M_{r}+M_{r^{\prime}}-2\ell-1}\ dx\ dy\
dz,$ (68)

where $u^{*}_{\ell}=2^{-j_{M}}x-u_{1}-\cdots-u_{\ell-1}-w_{1}-\cdots-
w_{M_{r}-\ell}$, $v^{*}_{\ell}=2^{-j_{M}}z-v_{1}-\cdots-
v_{\ell-1}-w_{1}-\cdots-w_{M_{r}-\ell}$, and
$w^{*}_{M_{r}+M_{r^{\prime}}-2\ell}=2^{-j_{M}}x+2^{-j_{M}}y-w_{1}-\cdots-
w_{M_{r}+M_{r^{\prime}}-2\ell-1}$. In the following, we show that the limit of
the integrand on the right hand side of the equality (68) exists when
$j_{M}\rightarrow\infty$.

  * 1.

For the second component of the integrand in (68), by (27),

$\displaystyle{}^{M-1}\widetilde{G}^{(M_{r^{\prime}})}_{2^{j_{M}}t}(-u_{1:\ell-1},-u^{*}_{\ell},w_{M_{r}-\ell+1:M_{r}+M_{r^{\prime}}-2\ell-1},w^{*}_{M_{r}+M_{r^{\prime}}-2\ell})$
$\displaystyle=$
$\displaystyle\,\textup{exp}\left\\{i2^{j_{M}}t\left[-u^{+}_{1:\ell-1}-u^{*}_{\ell}+w^{+}_{M_{r}-\ell+1:M_{r}+M_{r^{\prime}}-2\ell-1}+w^{*}_{M_{r}+M_{r^{\prime}}-2\ell}\right]\right\\}$
$\displaystyle\times{}^{M-1}\widetilde{G}^{(M_{r^{\prime}})}_{0}(-u_{1:\ell-1},-u^{*}_{\ell},w_{M_{r}-\ell+1:M_{r}+M_{r^{\prime}}-2\ell-1},w^{*}_{M_{r}+M_{r^{\prime}}-2\ell})$
$\displaystyle=$
$\displaystyle\,e^{ity}\times{}^{M-1}\widetilde{G}^{(M_{r^{\prime}})}_{0}(-u_{1:\ell-1},-u^{*}_{\ell},w_{M_{r}-\ell+1:M_{r}+M_{r^{\prime}}-2\ell-1},w^{*}_{M_{r}+M_{r^{\prime}}-2\ell})$
$\displaystyle\rightarrow$
$\displaystyle\,e^{ity}{\times}{}^{M-1}\widetilde{G}^{(M_{r^{\prime}})}_{0}(-u_{1:\ell-1},u^{+}_{1:\ell-1}+w^{+}_{1:M_{r}-\ell},w_{M_{r}-\ell+1:M_{r}+M_{r^{\prime}}-2\ell-1},-w^{+}_{1:M_{r}+M_{r^{\prime}}-2\ell-1})$

when $j_{M}\to\infty$.

  * 2.

For the fourth component of the integrand in (68),

$\displaystyle{}^{M-1}\widetilde{G}^{(M_{r^{\prime}})}_{2^{j_{M}}t}(-v_{1:\ell-1},-v^{*}_{\ell},w_{M_{r}-\ell+1:M_{r}+M_{r^{\prime}}-2\ell-1},w^{*}_{M_{r}+M_{r^{\prime}}-2\ell})$
$\displaystyle=$
$\displaystyle\,\textup{exp}\left\\{i2^{j_{M}}t\left[-v^{+}_{1:\ell-1}-v^{*}_{\ell}+w^{+}_{M_{r}-\ell+1:M_{r}+M_{r^{\prime}}-2\ell-1}+w^{*}_{M_{r}+M_{r^{\prime}}-2\ell}\right]\right\\}$
$\displaystyle\,\times{}^{M-1}\widetilde{G}^{(M_{r^{\prime}})}_{0}(-v_{1:\ell-1},-v^{*}_{\ell},w_{M_{r}-\ell+1:M_{r}+M_{r^{\prime}}-2\ell-1},w^{*}_{M_{r}+M_{r^{\prime}}-2\ell})$
$\displaystyle=$
$\displaystyle\,e^{it(x+y-z)}{\times}{}^{M-1}\widetilde{G}^{(M_{r^{\prime}})}_{0}(-v_{1:\ell-1},-v^{*}_{\ell},w_{M_{r}-\ell+1:M_{r}+M_{r^{\prime}}-2\ell-1},w^{*}_{M_{r}+M_{r^{\prime}}-2\ell})$
$\displaystyle\rightarrow$
$\displaystyle\,e^{it(x+y-z)}{\times}{}^{M-1}\widetilde{G}^{(M_{r^{\prime}})}_{0}(-v_{1:\ell-1},v^{+}_{1:\ell-1}+w^{+}_{1:M_{r}-\ell},w_{M_{r}-\ell+1:M_{r}+M_{r^{\prime}}-2\ell-1},-w^{+}_{1:M_{r}+M_{r^{\prime}}-2\ell-1})$

when $j_{M}\rightarrow\infty$.

  * 3.

Similarly, for the first and third components of the integrand in (68), we
have

${}^{M-1}\widetilde{G}^{(M_{r})}_{2^{j_{M}}s}(u_{1:\ell-1},u^{*}_{\ell},w_{1:M_{r}-\ell})\rightarrow
e^{isx}{\times}{}^{M-1}\widetilde{G}^{(M_{r})}_{0}(u_{1:\ell-1},-u^{+}_{1:\ell-1}-w^{+}_{1:M_{r}-\ell},w_{1:M_{r}-\ell})$

and

${}^{M-1}\widetilde{G}^{(M_{r})}_{2^{j_{M}}s}(v_{1:\ell-1},v_{\ell}^{*},w_{1:M_{r}-\ell})\rightarrow
e^{isz}{\times}{}^{M-1}\widetilde{G}^{(M_{r})}_{0}(v_{1:\ell-1},-v^{+}_{1:\ell-1}-w^{+}_{1:M_{r}-\ell},w_{1:M_{r}-\ell})$

respectively when $j_{M}\rightarrow\infty$.

On the other hand, by Lemma 4, for the integrand in (68), there exists a
constant $C>0$, which is independent of $j_{M}$, such that

$\displaystyle\underset{x,y,z\in\mathbb{R}}{\sup}\Big{|}{}^{M-1}\widetilde{G}^{(M_{r})}_{2^{j_{M}}s}(u_{1:\ell-1},u^{*}_{\ell},w_{1:M_{r}-\ell}){\times}{}^{M-1}\widetilde{G}^{(M_{r^{\prime}})}_{2^{j_{M}}t}(-u_{1:\ell-1},-u^{*}_{\ell},w_{M_{r}-\ell+1:M_{r}+M_{r^{\prime}}-2\ell-1},w^{*}_{M_{r}+M_{r^{\prime}}-2\ell})$
$\displaystyle\times\overline{{}^{M-1}\widetilde{G}^{(M_{r})}_{2^{j_{M}}s}(v_{1:\ell-1},v_{\ell}^{*},w_{1:M_{r}-\ell})}{\times}\overline{{}^{M-1}\widetilde{G}^{(M_{r^{\prime}})}_{2^{j_{M}}t}(-v_{1:\ell-1},-v^{*}_{\ell},w_{M_{r}-\ell+1:M_{r}+M_{r^{\prime}}-2\ell-1},w^{*}_{M_{r}+M_{r^{\prime}}-2\ell})}\Big{|}$
$\displaystyle\leq$
$\displaystyle\,C\left[\underset{x,y,z\in\mathbb{R}}{\sup}f_{X\star\psi_{j_{1}}}(u_{\ell}^{*})f_{X\star\psi_{j_{1}}}(v_{\ell}^{*})f_{X\star\psi_{j_{1}}}(w_{M_{r}+M_{r^{\prime}}-2\ell}^{*})\right]$
$\displaystyle\times\left[\overset{\ell-1}{\underset{k=1}{\prod}}f_{X\star\psi_{j_{1}}}(u_{k})f_{X\star\psi_{j_{1}}}(v_{k})\right]\left[\overset{M_{r}+M_{r^{\prime}}-2\ell-1}{\underset{k=1}{\prod}}f_{X\star\psi_{j_{1}}}(w_{k})\right].$
(69)

If $2\alpha+\beta\geq 1$, then $f_{X\star\psi_{j_{1}}}(\cdot)=2^{2\alpha
j_{1}}|C_{\hat{\psi}}(2^{j_{1}}\cdot)|^{2}C_{X}(\cdot)|\cdot|^{2\alpha+\beta-1}$
is not only integrable, but also bounded, which implies that the right hand
side of (69) is integrable with respect to
$(u_{1:\ell-1},v_{1:\ell-1},w_{1:M_{r}+M_{r^{\prime}}-2\ell-1})$. By the
Lebesgue dominated convergence theorem, we know that
$\underset{j_{M}\rightarrow\infty}{\lim}2^{3j_{M}}\big{\|}{}^{M}F^{(M_{r})}_{2^{j_{M}}s}\otimes_{\ell}{}^{M}F^{(M_{r^{\prime}})}_{2^{j_{M}}t}\big{\|}^{2}_{\overline{H}^{\otimes
M_{r}+M_{r^{\prime}}-2\ell}}$ exists and satisfies

$\displaystyle\underset{j_{M}\rightarrow\infty}{\lim}2^{3j_{M}}\left\|{}^{M}F^{(M_{r})}_{2^{j_{M}}s}\otimes_{\ell}{}^{M}F^{(M_{r^{\prime}})}_{2^{j_{M}}t}\right\|^{2}_{\overline{H}^{\otimes
M_{r}+M_{r^{\prime}}-2\ell}}$ $\displaystyle\leq$
$\displaystyle\,C\|f_{X\star\psi_{j_{1}}}\|_{1}^{M_{r}+M_{r^{\prime}}-3}\|f_{X\star\psi_{j_{1}}}\|_{\infty}^{3}\|\hat{\psi}\|_{\infty}^{3}\int_{\mathbb{R}^{3}}\Big{|}\hat{\psi}(x)\hat{\psi}(y)\overline{\hat{\psi}(z)}\overline{\hat{\psi}(x+y-z)}\Big{|}\
dx\ dy\ dz.$

The proof of the statement (a) is completed.

(b) For $r\in\\{0,1,\ldots,2^{M-2}-1\\}$, by (23) and the orthogonal property
of multiple Wiener-It$\hat{\textup{o}}$ integrals,

$\displaystyle\sigma_{j_{M}}^{2}=\mathbb{E}\Big{|}T[j_{1:M}]X(2^{j_{M}}t)\Big{|}^{2}=\mathbb{E}\Big{|}\overset{2^{M-2}-1}{\underset{r=0}{\sum}}I_{M_{r}}\left({}^{M}\\!F^{(M_{r})}_{2^{j_{M}}t}\right)\Big{|}^{2}=\overset{2^{M-2}-1}{\underset{r=0}{\sum}}M_{r}!\|{}^{M}\\!F^{(M_{r})}_{2^{j_{M}}t}\|_{\overline{H}^{\otimes
M_{r}}}^{2}.$ (70)

From (28), we know that the value of
$\|{}^{M}\\!F^{(M_{r})}_{2^{j_{M}}t}\|_{\overline{H}^{\otimes M_{r}}}^{2}$
does not change along with the time variable. Hence, we choose $t=0$. From
(26), for each $r\in\\{0,1,\ldots,2^{M-2}-1\\}$,

$\displaystyle\|{}^{M}\\!F^{(M_{r})}_{0}\|_{\overline{H}^{\otimes
M_{r}}}^{2}=\int_{\mathbb{R}^{M_{r}}}\Big{|}{}^{M-1}\widetilde{G}^{(M_{r})}_{0}(\lambda_{1:M_{r}})\hat{\psi}_{j_{M}}(\lambda^{+}_{1:M_{r}})\Big{|}^{2}\
d\lambda_{1:M_{r}}.$

By changing of variables $u_{i}=\lambda_{i}$ for $i=1,\ldots,M_{r}-1$ and
$z=2^{j_{M}}(\lambda_{1}+\cdots+\lambda_{M_{r}})$,

$\displaystyle 2^{j_{M}}\|{{}^{M}}\\!F^{(M_{r})}_{0}\|_{\overline{H}^{\otimes
M_{r}}}^{2}=\int_{\mathbb{R}^{M_{r}}}\Big{|}{}^{M-1}\widetilde{G}^{(M_{r})}_{0}(u_{1:M_{r}-1},2^{-j_{M}}z-u^{+}_{1:M_{r}-1})\Big{|}^{2}|\hat{\psi}(z)|^{2}\
du_{1:M_{r}-1}\ dz.$

For the integrand of the integral above, by Lemma 4, there exists a constant
$C_{M}$, which is independent of $j_{M}$, such that

$\displaystyle\Big{|}{}^{M-1}\widetilde{G}^{(M_{r})}_{0}(u_{1:M_{r}-1},2^{-j_{M}}z-u^{+}_{1:M_{r}-1})\Big{|}^{2}\leq
C^{2}_{M}f_{X\star\psi_{j_{1}}}(2^{-j_{M}}z-u^{+}_{1:M_{r}-1})\overset{M_{r}-1}{\underset{k=1}{\prod}}f_{X\star\psi_{j_{1}}}(u_{k}).$

Under the assumption $2\alpha+\beta\geq 1$, $f_{X\star\psi_{j_{1}}}$ is not
only integrable, but also bounded. Hence, we can again apply the Lebesgue
dominated convergence theorem to get

$\displaystyle\underset{j_{M}\rightarrow\infty}{\lim}2^{j_{M}}\|{{}^{M}}\\!F^{(M_{r})}_{0}\|_{H^{\otimes
M_{r}}}^{2}=c^{(M_{r})}\|\hat{\psi}\|^{2},$ (71)

where $c^{(M_{r})}$ is defined in (39). The proof is finished by combining
(70) and (71).

###  4.5 Proof of the equality (44)

From the definition of $\mathbf{C}_{j_{M}}(i,j)$ in (43), i.e.,
$\mathbf{C}_{j_{M}}(i,j)=\mathbb{E}[F_{i}F_{j}]$ and the definition of $F_{i}$
in (42), i.e., $F_{i}=2^{\frac{j_{M}}{2}}T[j_{1:M}]X(2^{j_{M}}t_{i})$, we have

$\displaystyle\mathbf{C}_{j_{M}}(i,j)=2^{j_{M}}\mathbb{E}\Big{[}T[j_{1:M}]X(2^{j_{M}}t_{i})T[j_{1:M}]X(2^{j_{M}}t_{j})\Big{]}.$

By replacing $T[j_{1:M}]X$ with its Wiener-It$\hat{\textup{o}}$ decomposition
in (23),

$\displaystyle\mathbf{C}_{j_{M}}(i,j)=$
$\displaystyle\,2^{j_{M}}\mathbb{E}\left\\{\left[\overset{2^{M-2}-1}{\underset{r=0}{\sum}}I_{M_{r}}\left({}^{M}\\!F^{(M_{r})}_{2^{j_{M}}t_{i}}\right)\right]\left[\overset{2^{M-2}-1}{\underset{r^{\prime}=0}{\sum}}I_{M_{r^{\prime}}}\left({}^{M}\\!F^{(M_{r^{\prime}})}_{2^{j_{M}}t_{j}}\right)\right]\right\\}$
$\displaystyle=$
$\displaystyle\,2^{j_{M}}\overset{2^{M-2}-1}{\underset{r=0}{\sum}}\mathbb{E}\left[I_{M_{r}}\left({}^{M}\\!F^{(M_{r})}_{2^{j_{M}}t_{i}}\right)I_{M_{r}}\left({}^{M}\\!F^{(M_{r})}_{2^{j_{M}}t_{j}}\right)\right],$
(72)

where the last equality follows from the orthogonal property of the multiple
Wiener integrals. For $r\in\\{0,1,\ldots,2^{M-2}-1\\}$, by Lemma 1,

$\displaystyle
I_{M_{r}}\left({}^{M}\\!F^{(M_{r})}_{2^{j_{M}}t_{i}}\right)I_{M_{r}}\left({}^{M}\\!F^{(M_{r})}_{2^{j_{M}}t_{j}}\right)=\overset{M_{r}}{\underset{\ell=0}{\sum}}\ell!\binom{M_{r}}{\ell}\binom{M_{r}}{\ell}I_{2M_{r}-2\ell}\left({}^{M}\\!F^{(M_{r})}_{2^{j_{M}}t_{i}}\otimes_{\ell}{}^{M}\\!F^{(M_{r})}_{2^{j_{M}}t_{j}}\right).$

Because the expectation of the Wiener integrals is equal to zero except the
constant term,

$\displaystyle\mathbb{E}\left[I_{M_{r}}\left({}^{M}\\!F^{(M_{r})}_{2^{j_{M}}t_{i}}\right)I_{M_{r}}\left({}^{M}\\!F^{(M_{r})}_{2^{j_{M}}t_{j}}\right)\right]=M_{r}!\
\
{}^{M}\\!F^{(M_{r})}_{2^{j_{M}}t_{i}}\otimes_{M_{r}}{}^{M}\\!F^{(M_{r})}_{2^{j_{M}}t_{j}}.$
(73)

The equality (44) follows from substituting (73) into (72).

###  4.6 Proof of Theorem 2

As mentioned before the statement of Theorem 2, all conditions in Lemma 5 are
satisfied when the random vector $\mathbf{F}$ and the matrix $\mathbf{C}$ are
given by (42) and (43). Hence, it suffices to study the behavior of
$\mathbb{E}\left[\left(\mathbf{C}_{j_{M}}(i,j)-\Big{\langle}DF_{j},-DL^{-1}F_{i}\Big{\rangle}\right)^{2}\right]$.

By (40) and (32), we have

$\displaystyle
DF_{j}=\overset{2^{M-2}-1}{\underset{r=0}{\sum}}M_{r}I_{M_{r}-1}\left({}^{M}\\!F^{(M_{r})}_{2^{j_{M}}t_{j}}\right)$

and

$\displaystyle-
DL^{-1}F_{i}=2^{\frac{j_{M}}{2}}D\left[\overset{2^{M-2}-1}{\underset{r=0}{\sum}}\frac{1}{M_{r}}I_{M_{r}}\left({}^{M}\\!F^{(M_{r})}_{2^{j_{M}}t_{i}}\right)\right]=2^{\frac{j_{M}}{2}}\left[\overset{2^{M-2}-1}{\underset{r=0}{\sum}}I_{M_{r}-1}\left({}^{M}\\!F^{(M_{r})}_{2^{j_{M}}t_{i}}\right)\right].$

Therefore,

$\displaystyle\Big{\langle}DF_{j},-DL^{-1}F_{i}\Big{\rangle}=2^{j_{M}}\Big{\langle}\overset{2^{M-2}-1}{\underset{r=0}{\sum}}M_{r}I_{M_{r}-1}\left({}^{M}\\!F^{(M_{r})}_{2^{j_{M}}t_{j}}\right),\overset{2^{M-2}-1}{\underset{r^{\prime}=0}{\sum}}I_{M_{r^{\prime}}-1}\left({}^{M}\\!F^{(M_{r^{\prime}})}_{2^{j_{M}}t_{i}}\right)\Big{\rangle}.$
(74)

By Lemma 1, (74) can be rewritten as follows

$\displaystyle\Big{\langle}DF_{j},-DL^{-1}F_{i}\Big{\rangle}=2^{j_{M}}\overset{2^{M-2}-1}{\underset{r=0}{\sum}}M_{r}!\left[{}^{M}\\!F^{(M_{r})}_{2^{j_{M}}t_{j}}\otimes_{M_{r}}{}^{M}\\!F^{(M_{r})}_{2^{j_{M}}t_{i}}\right]+\mathcal{I}_{1}+\mathcal{I}_{2},$
(75)

where

$\displaystyle\mathcal{I}_{1}=2^{j_{M}}\overset{2^{M-2}-1}{\underset{r=0}{\sum}}M_{r}\overset{M_{r}-1}{\underset{\ell=1}{\sum}}(\ell-1)!\binom{M_{r}-1}{\ell-1}^{2}I_{2M_{r}-2\ell}\left({}^{M}\\!F^{(M_{r})}_{2^{j_{M}}t_{j}}\otimes_{\ell}{}^{M}\\!F^{(M_{r})}_{2^{j_{M}}t_{i}}\right)$

and

$\displaystyle\mathcal{I}_{2}=2^{j_{M}}\overset{2^{M-2}-1}{\underset{\begin{subarray}{l}r,r^{\prime}=0\\\
r\neq r^{\prime}\end{subarray}}{\sum}}M_{r}\overset{M_{r}\wedge
M_{r^{\prime}}}{\underset{\ell=1}{\sum}}(\ell-1)!\binom{M_{r}-1}{\ell-1}\binom{M_{r^{\prime}}-1}{\ell-1}I_{M_{r}+M_{r^{\prime}}-2\ell}\left({}^{M}\\!F^{(M_{r})}_{2^{j_{M}}t_{j}}\otimes_{\ell}{}^{M}\\!F^{(M_{r^{\prime}})}_{2^{j_{M}}t_{i}}\right).$

We recognize that the first term in (75) coincides with the expression (44) of
$\mathbf{C}_{j_{M}}(i,j)$. By the Minkowski inequality,

$\displaystyle\sqrt{\mathbb{E}\left[\left(\mathbf{C}_{j_{M}}(i,j)-\Big{\langle}DF_{j},-DL^{-1}F_{i}\Big{\rangle}\right)^{2}\right]}=\sqrt{\mathbb{E}\left[|\mathcal{I}_{1}+\mathcal{I}_{2}|^{2}\right]}$
$\displaystyle\leq$
$\displaystyle\,\sqrt{\mathbb{E}\left[|\mathcal{I}_{1}|^{2}\right]}+\sqrt{\mathbb{E}\left[|\mathcal{I}_{2}|^{2}\right]}=2^{j_{M}}\mathcal{U}_{j_{M}}(t_{j},t_{i}),$

where $\mathcal{U}_{j_{M}}(\cdot,\cdot)$ is defined in (37). By Proposition 2,
we know that
$\underset{j_{M}\rightarrow\infty}{\lim}2^{\frac{3}{2}j_{M}}\mathcal{U}_{j_{M}}(\cdot,\cdot)$
exists. Hence, for $i,j\in\\{1,2,\ldots,d\\}$, we get

$\displaystyle\sqrt{\mathbb{E}\left[\left(\mathbf{C}_{j_{M}}(i,j)-\Big{\langle}DF_{j},-DL^{-1}F_{i}\Big{\rangle}\right)^{2}\right]}=O(2^{-\frac{1}{2}j_{M}})$

when $j_{M}$ is sufficiently large.

###  4.7 Proof of Proposition 3

By the equality (44), it suffices to prove that $2^{j_{M}}\
{}^{M}\\!F^{(M_{r})}_{2^{j_{M}}t_{i}}\otimes_{M_{r}}{}^{M}\\!F^{(M_{r})}_{2^{j_{M}}t_{j}}$
converges when $j_{M}\rightarrow\infty$ for each
$r\in\\{0,1,\ldots,2^{M-2}-1\\}$. By (26),

$\displaystyle{}^{M}\\!F^{(M_{r})}_{2^{j_{M}}t_{i}}\otimes_{M_{r}}{}^{M}\\!F^{(M_{r})}_{2^{j_{M}}t_{j}}=\int_{\mathbb{R}^{M_{r}}}{}^{M}\\!F^{(M_{r})}_{2^{j_{M}}t_{i}}(\lambda_{1:M_{r}})\
\overline{{}^{M}\\!F^{(M_{r})}_{2^{j_{M}}t_{j}}(\lambda_{1:M_{r}})}\
d\lambda_{1:M_{r}}$ $\displaystyle=$
$\displaystyle\int_{\mathbb{R}^{M_{r}}}\left[{}^{M-1}\widetilde{G}^{(M_{r})}_{2^{j_{M}}t_{i}}(\lambda_{1:M_{r}})\hat{\psi}_{j_{M}}(\lambda^{+}_{1:M_{r}})\right]\
\left[\
\overline{{}^{M-1}\widetilde{G}^{(M_{r})}_{2^{j_{M}}t_{j}}(\lambda_{1:M_{r}})\hat{\psi}_{j_{M}}(\lambda^{+}_{1:M_{r}})}\
\right]\ d\lambda_{1:M_{r}}$ $\displaystyle=$
$\displaystyle\int_{\mathbb{R}^{M_{r}}}\left[{}^{M-1}\widetilde{G}^{(M_{r})}_{2^{j_{M}}t_{i}}(\lambda_{1:M_{r}})\
\overline{{}^{M-1}\widetilde{G}^{(M_{r})}_{2^{j_{M}}t_{j}}(\lambda_{1:M_{r}})}\
\right]\Big{|}\hat{\psi}_{j_{M}}(\lambda^{+}_{1:M_{r}})\Big{|}^{2}\
d\lambda_{1:M_{r}}.$

From the recursive formula (25) and (26) with the initial term (15), we know
that ${}^{M-1}\widetilde{G}^{(M_{r})}_{t}(\lambda_{1:M_{r}})$ satisfies

${}^{M-1}\widetilde{G}^{(M_{r})}_{t}(\lambda_{1:M_{r}})={}^{M-1}\widetilde{G}^{(M_{r})}_{0}(\lambda_{1:M_{r}})\exp\left(it\lambda^{+}_{1:M_{r}}\right).$

Hence,

$\displaystyle{}^{M}\\!F^{(M_{r})}_{2^{j_{M}}t_{i}}\otimes_{M_{r}}{}^{M}\\!F^{(M_{r})}_{2^{j_{M}}t_{j}}=\int_{\mathbb{R}^{M_{r}}}\exp\left(i2^{j_{M}}(t_{i}-t_{j})\lambda^{+}_{1:M_{r}}\right)\Big{|}{}^{M-1}\widetilde{G}^{(M_{r})}_{0}(\lambda_{1:M_{r}})\Big{|}^{2}\Big{|}\hat{\psi}_{j_{M}}(\lambda^{+}_{1:M_{r}})\Big{|}^{2}\
d\lambda_{1:M_{r}}.$

By changing of variables

$\displaystyle\left\\{\begin{array}[]{l}u_{k}=\lambda_{k}\ \textup{for}\
k=1,2,\ldots,M_{r}-1,\\\ z=2^{j_{M}}\lambda^{+}_{1:M_{r}},\end{array}\right.$

we get

$\displaystyle 2^{j_{M}}\
{}^{M}\\!F^{(M_{r})}_{2^{j_{M}}t_{i}}\otimes_{M_{r}}{}^{M}\\!F^{(M_{r})}_{2^{j_{M}}t_{j}}$
$\displaystyle=$
$\displaystyle\int_{\mathbb{R}^{M_{r}}}e^{i(t_{i}-t_{j})z}\Big{|}{}^{M-1}\widetilde{G}^{(M_{r})}_{0}(u_{1:M_{r}-1},2^{-j_{M}}z-u^{+}_{1:M_{r}-1})\Big{|}^{2}\Big{|}\hat{\psi}(z)\Big{|}^{2}\
du_{1:M_{r}-1}\ dz.$

Under the condition $2\alpha+\beta\geq 1$, we apply Lemma 4 and the Lebesgue
dominated convergence theorem to get

$\displaystyle\underset{j_{M}\rightarrow\infty}{\lim}2^{j_{M}}{}^{M}\\!F^{(M_{r})}_{2^{j_{M}}t_{i}}\otimes_{M_{r}}{}^{M}\\!F^{(M_{r})}_{2^{j_{M}}t_{j}}$
$\displaystyle=$
$\displaystyle\int_{\mathbb{R}}e^{i(t_{i}-t_{j})z}\left\\{\int_{\mathbb{R}^{M_{r}-1}}\Big{|}{}^{M-1}\widetilde{G}^{(M_{r})}_{0}(u_{1:M_{r}-1},-u^{+}_{1:M_{r}-1})\Big{|}^{2}\
du_{1:M_{r}-1}\right\\}\Big{|}\hat{\psi}(z)\Big{|}^{2}\ dz.$

Note that the integral inside the curly brackets above is just the constant
$c^{(M_{r})}$ defined in (39).

###  4.8 Proof of Theorem 3 (derivation of (49))

It suffices to derive the covariance function of the limiting process $G^{2}$.
From Proposition 3, the covariance function of the process $G$ has the
spectral representation

$\textup{Cov}\left(G(t_{1}),G(t_{2})\right)=\kappa\int_{\mathbb{R}}e^{i(t_{1}-t_{2})z}\Big{|}\hat{\psi}(z)\Big{|}^{2}\
dz,$

where $\kappa$ is a nonnegative constant defined as

$\kappa=\overset{2^{M-2}-1}{\underset{r=0}{\sum}}M_{r}!\ c^{(M_{r})}.$

By introducing a complex-valued Gaussian random measure $W^{{}^{\prime}}$ on
$\mathbb{R}$, which also satisfies (4), the Gaussian process $G$ can be
expressed as a Wiener integral

$G(t)=\sqrt{\kappa}\int_{\mathbb{R}}e^{itz}\hat{\psi}(z)\
W^{{}^{\prime}}(dz),\ t\in\mathbb{R}.$

By Lemma 1,

$\displaystyle G^{2}(t)=$
$\displaystyle\,\kappa\left(\int_{\mathbb{R}}e^{itz}\hat{\psi}(z)\
W^{{}^{\prime}}(dz)\right)^{2}$ $\displaystyle=$
$\displaystyle\,\kappa\int_{\mathbb{R}}|\hat{\psi}(z)|^{2}dz+\kappa\int^{{}^{\prime}}_{\mathbb{R}^{2}}e^{it(z_{1}+z_{2})}\hat{\psi}(z_{1})\hat{\psi}(z_{2})W^{{}^{\prime}}(dz_{1})W^{{}^{\prime}}(dz_{2}),$

that is,

$\displaystyle
G^{2}(t)-\mathbb{E}[G^{2}(t)]=\kappa\int^{{}^{\prime}}_{\mathbb{R}^{2}}e^{it(z_{1}+z_{2})}\hat{\psi}(z_{1})\hat{\psi}(z_{2})W^{{}^{\prime}}(dz_{1})W^{{}^{\prime}}(dz_{2}).$

By applying Lemma 1 to the product
$\left(G^{2}(t_{1})-\mathbb{E}[G^{2}(t_{1})]\right)\left(G^{2}(t_{2})-\mathbb{E}[G^{2}(t_{2})]\right)$
and noting that the expectation of Wiener integrals is equal to zero, we
obtain

$\displaystyle\textup{Cov}\left(G^{2}(t_{1}),G^{2}(t_{2})\right)=$
$\displaystyle\,2\kappa^{2}\int_{\mathbb{R}^{2}}e^{i(t_{1}-t_{2})(z_{1}+z_{2})}|\hat{\psi}(z_{1})|^{2}|\hat{\psi}(z_{2})|^{2}\
dz_{1}\ dz_{2}.$ $\displaystyle=$
$\displaystyle\,2\left[\kappa\int_{\mathbb{R}}e^{i(t_{1}-t_{2})z}|\hat{\psi}(z)|^{2}\
dz\right]^{2}$ $\displaystyle=$
$\displaystyle\,2\left[\textup{Cov}\left(G(t_{1}),G(t_{2})\right)\right]^{2}.$

## Reference

## References

  * [1] S. Mallat, Group invariant scattering, Communications on Pure and Applied Mathematics 65 (10) (2012) 1331–1398.
  * [2] V. Chudáček, J. Andén, S. Mallat, P. Abry, M. Doret, Scattering transform for intrapartum fetal heart rate variability fractal analysis: A case-control study, IEEE Transactions on Biomedical Engineering 61 (4) (2013) 1100–1108.
  * [3] G.-R. Liu, Y.-L. Lo, J. Malik, Y.-C. Sheu, H.-T. Wu, Diffuse to fuse EEG spectra–intrinsic geometry of sleep dynamics for classification, Biomedical Signal Processing and Control 55 (2020) 101576.
  * [4] G.-R. Liu, T.-Y. Lin, H.-T. Wu, Y.-C. Sheu, C.-L. Liu, W.-T. Liu, M.-C. Yang, Y.-L. Ni, K.-T. Chou, C.-H. Chen, et al., Large-scale assessment of consistency in sleep stage scoring rules among multiple sleep centers using an interpretable machine learning algorithm, Journal of Clinical Sleep Medicine 17 (2) (2021) 159–166.
  * [5] H.-T. Wu, R. Talmon, Y.-L. Lo, Assess sleep stage by modern signal processing techniques, IEEE Transactions on Biomedical Engineering 62 (4) (2014) 1159–1168.
  * [6] R. Balestriero, H. Glotin, Linear time complexity deep Fourier scattering network and extension to nonlinear invariants, arXiv preprint arXiv:1707.05841.
  * [7] J. Andén, S. Mallat, Multiscale scattering for audio classification., in: ISMIR, Miami, FL, 2011, pp. 657–662.
  * [8] J. Li, L. Ke, Q. Du, X. Ding, X. Chen, D. Wang, Heart sound signal classification algorithm: A combination of wavelet scattering transform and twin support vector machine, IEEE Access 7 (2019) 179339–179348.
  * [9] W. Czaja, W. Li, Analysis of time-frequency scattering transforms, Applied and Computational Harmonic Analysis 47 (1) (2019) 149–171.
  * [10] I. Kavalerov, W. Li, W. Czaja, R. Chellappa, 3-D Fourier scattering transform and classification of hyperspectral images, IEEE Transactions on Geoscience and Remote Sensing.
  * [11] T. Wiatowski, H. Bölcskei, Deep convolutional neural networks based on semi-discrete frames, in: 2015 IEEE International Symposium on Information Theory (ISIT), IEEE, 2015, pp. 1212–1216.
  * [12] X. Cheng, X. Chen, S. Mallat, Deep Haar scattering networks, Information and Inference: A Journal of the IMA 5 (2) (2016) 105–133.
  * [13] J. Andén, S. Mallat, Deep scattering spectrum, IEEE Transactions on Signal Processing 62 (16) (2014) 4114–4128.
  * [14] R. Balan, M. Singh, D. Zou, Lipschitz properties for deep convolutional networks, Contemporary Mathematics 706 (2018) 129–151.
  * [15] T. Wiatowski, H. Bölcskei, A mathematical theory of deep convolutional neural networks for feature extraction, IEEE Transactions on Information Theory 64 (3) (2017) 1845–1866.
  * [16] G.-R. Liu, Y.-C. Sheu, H.-T. Wu, Central and non-central limit theorems arising from the scattering transform and its neural activation generalization, arXiv preprint arXiv:2011.10801.
  * [17] V. Lostanlen, A. Cohen-Hadria, J. P. Bello, One or two frequencies? The scattering transform answers, in: 2020 28th European Signal Processing Conference (EUSIPCO), IEEE, 2021, pp. 2205–2209.
  * [18] J. Bruna, S. Mallat, E. Bacry, J. Muzy, Intermittent process analysis with scattering moments, The Annals of Statistics 43 (1) (2015) 323–351.
  * [19] I. Nourdin, G. Peccati, G. Reinert, Invariance principles for homogeneous sums: universality of Gaussian Wiener chaos, The Annals of Probability 38 (5) (2010) 1947–1985.
  * [20] P. Major, Muliple Wiener-It$\hat{\textup{o}}$ Integrals, Vol. 849, Springer, 2014\.
  * [21] I. Nourdin, G. Peccati, Stein’s method on Wiener chaos, Probability Theory and Related Fields 145 (1-2) (2009) 75–118.
  * [22] I. Nourdin, G. Peccati, Normal approximations with Malliavin calculus: from Stein’s method to universality, no. 192, Cambridge University Press, 2012.
  * [23] P. Doukhan, G. Oppenheim, M. Taqqu, Theory and applications of long-range dependence, Springer Science & Business Media, 2002.
  * [24] V. Pipiras, M. S. Taqqu, Long-range dependence and self-similarity, Vol. 45, Cambridge University press, 2017.
  * [25] V. Pipiras, M. S. Taqqu, Regularization and integral representations of Hermite processes, Statistics & Probability Letters 80 (23-24) (2010) 2014–2023.
  * [26] K. Itô, Stationary random distributions, Memoirs of the College of Science, University of Kyoto. Series A: Mathematics 28 (3) (1954) 209–223.
  * [27] A. M. Yaglom, Some classes of random fields in n-dimensional space, related to stationary random processes, Theory of Probability & Its Applications 2 (3) (1957) 273–320.
  * [28] D. Nualart, The Malliavin calculus and related topics, Vol. 1995, Springer, 2006\.

##  Appendix A A validation of the application of the stochastic Fubini
theorem to get (52)

First of all, we apply the mathematical induction method to prove that for
each fixed $t\in\mathbb{R}$,

$\displaystyle{}^{M}\widetilde{G}^{(2^{M}-2\ell)}_{t}\in
L^{2}(\mathbb{R}^{2^{M}-2\ell})\ \textup{for}\ \ell=0,1,...,2^{M-1}-1,$ (A.1)

(${}^{M}\widetilde{G}^{(0)}_{t}$ is a constant) and

${}^{M}F^{(2^{M-1}-2r)}_{t}\in L^{2}(\mathbb{R}^{2^{M-1}-2r})\ \textup{for}\
r=0,1,\ldots,2^{M-2}-1.$ (A.2)

$\bullet$ From the initial term (15), we know that for each fixed
$t\in\mathbb{R}$, ${}^{1}\widetilde{G}^{(2)}_{t}\in L^{2}(\mathbb{R}^{2})$
because the spectral density function $f_{X}$ is integrable and $\psi\in
L^{1}(\mathbb{R})$ implies that $\hat{\psi}$ is a bounded function. The
initial term (15) also shows that ${}^{1}\widetilde{G}^{(0)}_{t}$ is a
constant. Hence, (A.1) holds when $M=1$. On the other hand, when $M=2$, (26)
can be simplified as

${}^{2}F^{(2)}_{t}(\lambda_{1:2})={}^{1}\widetilde{G}^{(2)}_{t}(\lambda_{1:2})\hat{\psi}_{j_{M}}(\lambda^{+}_{1:2}).$

By the fact $\|\hat{\psi}\|_{\infty}<\infty$, we obtain ${}^{2}F^{(2)}_{t}\in
L^{2}(\mathbb{R}^{2})$. Hence, (A.2) holds when $M=2$. At the same time, (25)
can be simplified as

$\displaystyle{}^{2}\widetilde{G}^{(4-2\ell)}_{t}=\ell!\binom{2}{\ell}\binom{2}{\ell}\times{}^{2}F^{(2)}_{t}\widetilde{\otimes}_{\ell}{}^{2}F^{(2)}_{t}\
\textup{for}\ \ell=0,1,2.$

Clearly, for each fixed $t\in\mathbb{R}$, ${}^{2}\widetilde{G}^{(0)}_{t}$ is a
constant. For $\ell=0,1$, by the Minkowski inequality and the Cauchy-Schwarz
inequality,

$\displaystyle\|{}^{2}F^{(2)}_{t}\widetilde{\otimes}_{\ell}{}^{2}F^{(2)}_{t}\|_{2}\leq\|{}^{2}F^{(2)}_{t}\|_{2}^{2}.$

It implies that ${}^{2}\widetilde{G}^{(4-2\ell)}_{t}\in
L^{2}(\mathbb{R}^{4-2\ell})$ for $\ell=0,1$. In other words, the claim (A.1)
holds when $M=2$.

$\bullet$ Now, we suppose that (A.1) and (A.2) hold when $M=m$, where $m\geq
2.$

$\bullet$ When $M=m+1$, from (26), i.e.,

${}^{m+1}F^{(2^{m}-2r)}_{t}(\lambda_{1:2^{m}-2r})={}^{m}\widetilde{G}^{(2^{m}-2r)}_{t}(\lambda_{1:2^{m}-2r})\hat{\psi}_{j_{m+1}}(\lambda^{+}_{1:2^{m}-2r})\
\textup{for}\ r=0,1,\ldots,2^{m-1}-1,$

we have

$\|{}^{m+1}F^{(2^{m}-2r)}_{t}\|_{2}\leq\|{}^{m}\widetilde{G}^{(2^{m}-2r)}_{t}\|_{2}\|\hat{\psi}_{j_{m+1}}\|_{\infty}\
\textup{for}\ r=0,1,\ldots,2^{m-1}-1,$

By $\|\hat{\psi}\|_{\infty}<\infty$ and the induction hypothesis
${}^{m}\widetilde{G}^{(2^{m}-2r)}_{t}\in L^{2}(\mathbb{R}^{2^{m}-2r})$ for
$r=0,1,\ldots,2^{m-1}-1$, we know that

${}^{m+1}F^{(2^{m}-2r)}_{t}\in L^{2}(\mathbb{R}^{2^{m}-2r})\ \textup{for}\
r=0,1,\ldots,2^{m-1}-1.$ (A.3)

On the other hand, by (25),

$\displaystyle{}^{m+1}\widetilde{G}^{(2^{m+1}-2\ell)}_{t}=$
$\displaystyle\overset{2^{m-1}-1}{\underset{r,r^{{}^{\prime}}=0}{\sum}}(\ell-
r-r^{\prime})!\binom{2^{m}-2r}{\ell-
r-r^{\prime}}\binom{2^{m}-2r^{\prime}}{\ell-r-r^{\prime}}$
$\displaystyle\times{}^{m+1}F^{(2^{m}-2r)}_{t}\widetilde{\otimes}_{\ell-
r-r^{\prime}}{}^{m+1}F^{(2^{m}-2r^{\prime})}_{t}$

for $\ell=0,1,\ldots,2^{m}$. It shows that ${}^{m+1}\widetilde{G}^{(0)}_{t}$
is a constant. For $\ell=0,1,\ldots,2^{m}-1$, by the Minkowski inequality and
the Cauchy-Schwarz inequality,

$\displaystyle\|{}^{m+1}\widetilde{G}^{(2^{m+1}-2\ell)}_{t}\|_{2}\leq$
$\displaystyle\overset{2^{m-1}-1}{\underset{r,r^{{}^{\prime}}=0}{\sum}}(\ell-
r-r^{\prime})!\binom{2^{m}-2r}{\ell-
r-r^{\prime}}\binom{2^{m}-2r^{\prime}}{\ell-r-r^{\prime}}$
$\displaystyle\times\|{}^{m+1}F^{(2^{m}-2r)}_{t}\widetilde{\otimes}_{\ell-
r-r^{\prime}}{}^{m+1}F^{(2^{m}-2r^{\prime})}_{t}\|_{2}$ $\displaystyle\leq$
$\displaystyle\overset{2^{m-1}-1}{\underset{r,r^{{}^{\prime}}=0}{\sum}}(\ell-
r-r^{\prime})!\binom{2^{m}-2r}{\ell-
r-r^{\prime}}\binom{2^{m}-2r^{\prime}}{\ell-r-r^{\prime}}$
$\displaystyle\times\|{}^{m+1}F^{(2^{m}-2r)}_{t}\|_{2}\
\|{}^{m+1}F^{(2^{m}-2r^{\prime})}_{t}\|_{2}<\infty,$

where the last inequality follows from (A.3). Therefore, we finish the
inductive step and the proof of the claim (A.1) and (A.2).

For $M\geq 1$, from (2), (24) and the mean-zero property of the wavelet,

$\displaystyle
T[j_{1:M+1}]X(t)=\int_{\mathbb{R}}U[j_{1:M}]X(s)\psi_{j_{M+1}}(t-s)ds$
$\displaystyle=$
$\displaystyle\overset{2^{M-1}}{\underset{\ell=0}{\sum}}\int_{\mathbb{R}}I_{2^{M}-2\ell}\left({}^{M}\widetilde{G}^{(2^{M}-2\ell)}_{s}\right)\psi_{j_{M+1}}(t-s)ds$
$\displaystyle=$
$\displaystyle\overset{2^{M-1}-1}{\underset{\ell=0}{\sum}}\int_{\mathbb{R}}I_{2^{M}-2\ell}\left({}^{M}\widetilde{G}^{(2^{M}-2\ell)}_{s}\right)\psi^{+}_{j_{M+1}}(t-s)ds$
$\displaystyle-\overset{2^{M-1}-1}{\underset{\ell=0}{\sum}}\int_{\mathbb{R}}I_{2^{M}-2\ell}\left({}^{M}\widetilde{G}^{(2^{M}-2\ell)}_{s}\right)\psi^{-}_{j_{M+1}}(t-s)ds,$

where $\psi_{j_{M+1}}^{+}$ (resp. $\psi_{j_{M+1}}^{-}$) is the nonnegative
part (resp. the negative part) of $\psi_{j_{M+1}}$. Before applying the
stochastic Fubini theorem [25, Theorem 2.1] to change the order of the
deterministic integral $\int_{\mathbb{R}}\ \cdots\ \psi^{+}_{j_{M+1}}(t-s)ds$
and the Wiener integral $I_{2^{M}-2\ell}(\cdots)$, we need to verify the
condition

$\int_{\mathbb{R}}\|{}^{M}\widetilde{G}^{(2^{M}-2\ell)}_{s}\|_{2}\
\psi^{+}_{j_{M+1}}(t-s)ds<\infty$

This condition holds because (27) implies that
$\|{}^{M}\widetilde{G}^{(2^{M}-2\ell)}_{s}\|_{2}$ does not depend on $s$ and
$\psi^{+}_{j_{M+1}}\in L^{1}$. Similarly,
$\int_{\mathbb{R}}\|{}^{M}\widetilde{G}^{(2^{M}-2\ell)}_{s}\|_{2}\
\psi^{-}_{j_{M+1}}(t-s)ds<\infty.$ Therefore, the stochastic Fubini theorem
implies that

$\displaystyle T[j_{1:M+1}]X(t)=$
$\displaystyle\overset{2^{M-1}-1}{\underset{\ell=0}{\sum}}I_{2^{M}-2\ell}\left(\int_{\mathbb{R}}{}^{M}\widetilde{G}^{(2^{M}-2\ell)}_{s}\psi^{+}_{j_{M+1}}(t-s)ds-\int_{\mathbb{R}}{}^{M}\widetilde{G}^{(2^{M}-2\ell)}_{s}\psi^{-}_{j_{M+1}}(t-s)ds\right)$
$\displaystyle=$
$\displaystyle\overset{2^{M-1}-1}{\underset{\ell=0}{\sum}}I_{2^{M}-2\ell}\left({}^{M+1}F^{(2^{M}-2\ell)}_{t}\right),$

where the last equality follows from the definition (26).