# Random Multiple Operator Integrals

Shih Yu Chang Shih Yu Chang is with the Department of Applied Data Science,
San Jose State University, San Jose, CA, U. S. A. (e-mail:
shihyu.chang@sjsu.edu).

###### Abstract

The introduction of Schur multipliers into the context of Double Operator
Integrals (DOIs) was proposed by V. V. Peller in 1985. This work extends
theorem on Schur multipliers from measurable functions to their closure space
and generalizes the definition of DOIs to Multiple Operator Integrals (MOIs)
for integrand functions as Schur multipliersconstructible by taking the limit
of projective tensor product and by taking the limit of integral projective
tensor product. According to such closure space construction for integrand
functions, we demonstrate that any function defined on a compact set of a
Euclidean space can be expressed by taking the limit of the projective tensor
product of linear functions. We also generalize previous works about random
DOIs with respect to finite dimensional operators, tensors, to MOIs with
respect to random operators, which are defined from spectral decomposition
perspectives. Based on random MOIs definitions and their properties, we derive
several tail bounds for norms of higher random operator derivatives, higher
random operator difference and Taylor remainder of random operator-valued
functions.

Index terms— Schur multiplier, Double Operator Integrals (DOIs), Multiple
Operator Integrals (MOIs), random operators, tail bounds.

##  1 Introduction

Basic concepts about _Double Operator Integrals_(DOIs) were first mentioned in
[1] by considering integration and differentiation of functions of Hermitian
operators and their applications to the theory of perturbations. It was Birman
and Solomyak who established later the theory of double operator integrals [2,
3, 4]. Let $(\Lambda_{1},A_{1})$ and $(\Lambda_{2},A_{2})$ be spaces with
spectral measures $A_{1}$ and $A_{2}$ on a HIlbert space $\mathfrak{H}$. Given
a bounded measurable function $\psi$ and a operator $X$ on Hilber space
$\mathfrak{H}$, the DOI is defined as

$\displaystyle\int\limits_{\Lambda_{1}}\int\limits_{\Lambda_{2}}\psi(\lambda_{1},\lambda_{2})dA_{1}(\lambda_{1})XdA_{2}(\lambda_{2}),$
(1)

where $\psi$ is named as an _integrand function_. We use conventional notation
$\mathfrak{S}_{p}$ to represent the $p$-th Schatten ideal. For every
$X\in\mathfrak{S}_{1}$, we say that the integrand function $\psi$ is a _Schur
multiplier_ of $\mathfrak{S}_{1}$ associated with the spectral measures
$\Lambda_{1}$ and $\Lambda_{2}$ if we have

$\displaystyle\int\limits_{\Lambda_{1}}\int\limits_{\Lambda_{2}}\psi(\lambda_{1},\lambda_{2})dA_{1}(\lambda_{1})XdA_{2}(\lambda_{2})\in\mathfrak{S}_{1}.$
(2)

Note that the introduction of Schur multipliers into the context of double
operator integrals was proposed by [5]. This is an extension of the notion of
matrix Schur multipliers. The first contribution of this work is to extend the
Schur multipliers [6] from measurable functions to their closure space. We
also extend the definition of DOIs to _Multiple Operator Integrals_(MOIs) for
integrand as Schur multipliers constructible by the _limit of projective
tensor product, $\hat{\otimes}$_ and by the _limit of integral projective
tensor product, $\acute{\otimes}$_. Therefore, several Lemmas discussed in
this work about MOIs basic properties, e.g., continuity, perturbation, and
higher order derivative representations by MOIs, will be generalized to
functions constructible by taking the _limit of projective tensor product
$(\hat{\otimes})$_ of measurable functions, or by taking the _limit of
integral projective tensor product $(\acute{\otimes})$_ of measurable function
[7]. According to such closure space construction for integrand functions, we
will be able to represent any function
$f(\lambda_{1},\lambda_{2},\cdots,\lambda_{m})$ defined on a compact set of a
Euclidean space $\mathbb{R}^{m}$ by the _limit of projective tensor product_
of linear functions.

The linear operators $A_{1},A_{2}$ and $X$ adopted by Eq. (1) are assumed to
be deterministic. The consideration of linear operators under randomness
settings is discussed by Skorohod in [8]. Two senses of linear random
operators are defined there: strong random operators and weak random
operators. Hackenbroch generalizes the concept of a strong random operator by
replacing the index Hilbert space by a fixed dense subspace [9]. In this work,
he proves that the operators with a densely defined adjoint have a unique
closed extension with a measurable selection, particularly, including
symmetric operators. The classes of random operators considered are all
restricted by assuming that there is a dense nonrandom subspace, but the
results are extensions of the results on self-adjoint extensions obtained by
[8]. Thang and Quy present results on strongly random operators and bounded
strongly random operators between separable Banach spaces [10]. They show that
a bounded strongly random operator can be extended to a continuous linear
operator from the set of Banach valued random variables to the set of Banach
valued random operators equipped with the topology from convergence in
probability. Thang and Quy prove several versions of the spectral theorem,
including bounded self adjoint, and more generally bounded normal strongly
random operators. These results amplify the results obtained by [8]. Although
several existing works about random DOIs for finite dimensional operators,
e.g., tensors, have been studied [11, 12], we first attempt to consider random
operators with MOIs in this work. However, the randomness of linear operators
is equipped with a different randomness structure compared to aforementioned
random linear operators. All operators considered in this work are assumed to
have spectral decomposition characterized by eigenvalues and projector spaces
(unitary operators formed by eigenspaces), then the randomness of a given
linear operator is determined by the random variables of eigenvalues and
random unitary operators with Haar measure.

Besides completeness for theorem on Schur multipliers, the extension
definition of MOIs, integrand approximation by linear functions, our other
contributions include following: derivation properties of MOIs when the
integrand functions constructible by the _limit of projective tensor product,
$\hat{\otimes}$_ and by the _limit of integral projective tensor product,
$\acute{\otimes}$_; definition about random MOIs from spectral decomposition
perspectives; application MOIs to derive tail bounds for higher random
operator derivatives, higher random operator difference and Taylor remainder
of random operator-valued functions.

The remainder of this paper is organized as follows. In Section 2, we discuss
the completeness for Theorem on Schur multipliers and define MOIs with the
completeness of integrand functions. We represent any function
$f(\lambda_{1},\lambda_{2},\cdots,\lambda_{m})$ defined on a compact set of a
Euclidean space $\mathbb{R}^{m}$ by _limit of projective tensor product_ of
linear functions in Section 3. We will discuss MOI properties and random MOI
for the integrand function constructible by _limit of integral projective
tensor product_ in Section 4. Finally, we will apply MOIs to derive tail
bounds for higher random operator derivatives, higher random operator
difference and Taylor remainder of random operator-valued functions in Section
5.

##  2 Completenss for Theorem on Schur multipliers

The purpose of this section is to define MOI with the completeness of
integrand funtions.

###  2.1 Double Operator Integrals

We use $\Gamma(A_{1},A_{2})$ to represent the space of Schur multipliers of
$\mathfrak{S}_{1}$ associated with the spectral measures $\Lambda_{1}$ and
$\Lambda_{2}$. If the integrand function $\psi$ in Eq. (1) belongs to the
_projective tensor product_ as
$L^{\infty}(A_{1})\hat{\otimes}L^{\infty}(A_{2})$, i.e., the integrand
function $\psi$ can be expressed as

$\displaystyle\psi(\lambda_{1},\lambda_{2})=\sum\limits_{n\geq
0}f_{1,n}(\lambda_{1})f_{2,n}(\lambda_{2}),$ (3)

where $f_{1,n}\in L^{\infty}(A_{1})$, $f_{2,n}\in L^{\infty}(A_{2})$ and
$\sum\limits_{n\geq
0}\left\|f_{1,n}(\lambda_{1})\right\|_{L^{\infty}}\left\|f_{2,n}(\lambda_{2})\right\|_{L^{\infty}}<\infty$;
we have $\psi\in\Gamma(A_{1},A_{2})$. Then, for such integrand function
$\psi$, we have

$\displaystyle\int\limits_{\Lambda_{1}}\int\limits_{\Lambda_{2}}\psi(\lambda_{1},\lambda_{2})dA_{1}(\lambda_{1})dA_{2}(\lambda_{2})\in\mathfrak{S}_{1}=\sum\limits_{n\geq
0}\left(\int\limits_{\Lambda_{1}}f_{1,n}(\lambda_{1})dA_{1}(\lambda_{1})\right)X\left(\int\limits_{\Lambda_{2}}f_{2,n}(\lambda_{2})dA_{2}(\lambda_{2})\right).$
(4)

Instead the summation form provided by Eq. (3), the integrand function $\psi$
belongs to the _integral projective tensor product_ as
$L^{\infty}(A_{1})\acute{\otimes}L^{\infty}(A_{2})$, i.e., the integrand
function $\psi$ can be expressed as

$\displaystyle\psi(\lambda_{1},\lambda_{2})=\int\limits_{\Xi}f_{1}(\lambda_{1},x)f_{2}(\lambda_{2},x)d\mu(x),$
(5)

where $(\Xi,\mu)$ is a measure space, $f_{1}$ is a measurable function on
$\Lambda_{1}\times\Xi$, $f_{2}$ is a measurable function on
$\Lambda_{2}\times\Xi$, and

$\displaystyle\int\limits_{\Xi}\left\|f_{1}(\cdot,x))\right\|_{L^{\infty}(A_{1})}\left\|f_{2}(\cdot,x))\right\|_{L^{\infty}(A_{2})}d\mu(x)<\infty.$
(6)

Then, for such integrand function $\psi$, we have

$\displaystyle\int\limits_{\Lambda_{1}}\int\limits_{\Lambda_{2}}\psi(\lambda_{1},\lambda_{2})dA_{1}(\lambda_{1})XdA_{2}(\lambda_{2})\in\mathfrak{S}_{1}=\int\limits_{\Xi}\left(\int\limits_{\Lambda_{1}}f_{1}(\lambda_{1},x)dA_{1}(\lambda_{1})\right)X\left(\int\limits_{\Lambda_{2}}f_{2}(\lambda_{2},x)dA_{2}(\lambda_{2})\right)d\mu(x).$
(7)

Let $\Lambda_{1},\Lambda_{2}$ be compact sets, and, for any $x$, suppose we
have

$\displaystyle\overline{f}_{1}(\lambda_{1},x)$ $\displaystyle=$
$\displaystyle\lim\limits_{n\rightarrow\infty}f_{1,n}(\lambda_{1},x)$
$\displaystyle\overline{f}_{2}(\lambda_{2},x)$ $\displaystyle=$
$\displaystyle\lim\limits_{n\rightarrow\infty}f_{2,n}(\lambda_{2},x)$ (8)

where $f_{1,n}$ is a measurable function on $\Lambda_{1}\times\Xi$, and
$f_{2,n}$ is a measurable function on $\Lambda_{2}\times\Xi$. We also assume
that

$\displaystyle\int\limits_{\Xi}\left\|\overline{f}_{1}(\cdot,x))\right\|_{L^{\infty}(A_{1})}\left\|\overline{f}_{2}(\cdot,x))\right\|_{L^{\infty}(A_{2})}d\mu(x)<\infty.$
(9)

We define the function $\psi_{n}(\lambda_{1},\lambda_{2})$ as

$\displaystyle\psi_{n}(\lambda_{1},\lambda_{2})=\int\limits_{\Xi}f_{1,n}(\lambda_{1},x)f_{2,n}(\lambda_{2},x)d\mu(x).$
(10)

Then, the limiting function of $\psi_{n}(\lambda_{1},\lambda_{2})$, denoted by
$\overline{\psi}(\lambda_{1},\lambda_{2})$, can be expressed as

$\displaystyle\overline{\psi}(\lambda_{1},\lambda_{2})=\lim\limits_{n\rightarrow\infty}\psi_{n}(\lambda_{1},\lambda_{2}).$
(11)

If functions $f_{1,n},f_{2,n}$ are increasing with $n$, or they are dominated
by some integrable functions, we have

$\displaystyle\overline{\psi}(\lambda_{1},\lambda_{2})$ $\displaystyle=$
$\displaystyle\lim\limits_{n\rightarrow\infty}\psi_{n}(\lambda_{1},\lambda_{2})=\lim\limits_{n\rightarrow\infty}\int\limits_{\Xi}f_{1,n}(\lambda_{1},x)f_{2,n}(\lambda_{2},x)d\mu(x)$
(12) $\displaystyle=$
$\displaystyle\int\limits_{\Xi}\lim\limits_{n\rightarrow\infty}(f_{1,n}(\lambda_{1},x)f_{2,n}(\lambda_{2},x))d\mu(x)$
$\displaystyle=$
$\displaystyle\int\limits_{\Xi}\overline{f}_{1}(\lambda_{1},x)\overline{f}_{2}(\lambda_{2},x)d\mu(x).$

We can summarize the above completeness arguments with Theorem on Schur
multipliers given by [6] to have the following new theorem after taking the
limit of the integrand function.

###### Theorem 1

Let $\overline{\psi}$ be a measurable function of
$\Lambda_{1}\times\Lambda_{2}$, the following statements are equivalent:

  1. 1.

$\overline{\psi}\in\Gamma(A_{1},A_{2})$;

  2. 2.

$\overline{\psi}\in L^{\infty}(A_{1})\acute{\otimes}L^{\infty}(A_{2})$;

  3. 3.

there exist measurable functions
$\overline{f}_{1}=\lim\limits_{n\rightarrow\infty}f_{1,n}(\lambda_{1},x)$ is a
measurable function on $\Lambda_{1}\times\Xi$, and
$\overline{f}_{2}=\lim\limits_{n\rightarrow\infty}f_{2,n}(\lambda_{2},x)$ is a
measurable function on $\Lambda_{2}\times\Xi$ such that Eq. (12) holds and
$\int\limits_{\Xi}\left\|\overline{f}_{1}(\cdot,x))\right\|_{L^{\infty}(A_{1})}\left\|\overline{f}_{2}(\cdot,x))\right\|_{L^{\infty}(A_{2})}d\mu(x)<\infty$.

###### Remark 1

If we can find functions
$\overline{f}_{1,n}(\lambda_{1})=\lim\limits_{m\rightarrow\infty}f_{1,n,m}(\lambda_{1})$
and
$\overline{f}_{2,n}(\lambda_{2})=\lim\limits_{m\rightarrow\infty}f_{2,n,m}(\lambda_{2})$
such that functions $f_{1,n,m},f_{2,n,m}$ are increasing with $m$, or they are
dominated by some integrable functions, Theorem 1 will still be true by
replacing the statement 3 with

$\displaystyle\overline{\psi}(\lambda_{1},\lambda_{2})=\sum\limits_{n\geq
0}\overline{f}_{1,n}(\lambda_{1})\overline{f}_{2,n}(\lambda_{2}),$ (13)

and $\sum\limits_{n\geq
0}\left\|\overline{f}_{1,n}\right\|_{L^{\infty}(A_{1})}\left\|\overline{f}_{2,n}\right\|_{L^{\infty}(A_{1}2}<\infty$.

If a given bivariate function can be expressed by the right-hand side of Eq,
(13), this function is said constructible by the _limit of projective tensor
product, $\hat{\otimes}$_. On the other hand, if a given bivariate function
can be expressed by the right-hand side of Eq, (12), this function is said
constructible by the _limit of integral projective tensor product,
$\acute{\otimes}$_.

###  2.2 Multiple Operator Integrals

We can easily extend the definition of the limit of projective tensor product
and the limit of integral projective tensor product to three or more function
spaces. We have following two multiple operator integrals for the integrand
function constructible by _limit of projective tensor product,
$\hat{\otimes}$_ and by _limit of integral projective tensor product,
$\acute{\otimes}$_. We use $\underline{A}_{i}^{j}$ to represent the sequence
of operators $A_{i},A_{i+1},\cdots,A_{j}$, similarly, we apply
$\underline{X}_{i}^{j}$ to represent the sequence of operators
$X_{i},X_{i+1},\cdots,X_{j}$.

###### Definition 1

Let $(\Lambda_{1},A_{1}),(\Lambda_{2},A_{2}),\cdots,(\Lambda_{m},A_{m})$ be
spaces with spectral measures $\underline{A}_{1}^{m}$ on a HIlbert space
$\mathfrak{H}$. Given a bounded measurable function
$\overline{\psi}_{\hat{\otimes}}(\lambda_{1},\lambda_{2},\cdots,\lambda_{m})$
($m$-variables) constructible by the limit of projective tensor product, and a
set of $(m-1)$ operators $\underline{X}_{1}^{m-1}$ on Hilber space
$\mathfrak{H}$, the _Multiple Operator Integrals_(MOIs), denoted as
$T^{\underline{A}_{1}^{m}}_{\overline{\psi}_{\hat{\otimes}}}(\underline{X}_{1}^{m-1})$,
is defined as

$\displaystyle
T^{\underline{A}_{1}^{m}}_{\overline{\psi}_{\hat{\otimes}}}(\underline{X}_{1}^{m-1})\stackrel{{\scriptstyle\mbox{\tiny
def}}}{{=}}$ (14)
$\displaystyle\int\limits_{\Lambda_{1}}\int\limits_{\Lambda_{2}}\cdots\int\limits_{\Lambda_{m}}\overline{\psi}_{\hat{\otimes}}(\lambda_{1},\lambda_{2},\cdots,\lambda_{m})dA_{1}(\lambda_{1})X_{1}dA_{2}(\lambda_{2})X_{2}\cdots
X_{m-1}dA_{m}(\lambda_{m}),$

where
$\overline{\psi}_{\hat{\otimes}}(\lambda_{1},\lambda_{2},\cdots,\lambda_{m})$
can be expressed as

$\displaystyle\overline{\psi}_{\hat{\otimes}}(\lambda_{1},\lambda_{2},\cdots,\lambda_{m})$
$\displaystyle=$ $\displaystyle\sum\limits_{n\geq
0}\overline{f}_{1,n}(\lambda_{1})\overline{f}_{2,n}(\lambda_{2})\cdots\overline{f}_{m,n}(\lambda_{m}).$
(15)

###### Definition 2

Let $(\Lambda_{1},A_{1}),(\Lambda_{2},A_{2}),\cdots,(\Lambda_{m},A_{m})$ be
spaces with spectral measures $\underline{A}_{1}^{m}$ on a HIlbert space
$\mathfrak{H}$. Given a bounded measurable function
$\overline{\psi}_{\acute{\otimes}}(\lambda_{1},\lambda_{2},\cdots,\lambda_{m})$
($m$-variables) constructible by the limit of integral projective tensor
product, and a set of $(m-1)$ operators $\underline{X}_{1}^{m-1}$ on Hilber
space $\mathfrak{H}$, the _Multiple Operator Integrals_(MOIs), denoted as
$T^{\underline{A}_{1}^{m}}_{\overline{\psi}_{\acute{\otimes}}}(\underline{X}_{1}^{m-1})$,
is defined as

$\displaystyle
T^{\underline{A}_{1}^{m}}_{\overline{\psi}_{\acute{\otimes}}}(\underline{X}_{1}^{m-1})$
$\displaystyle\stackrel{{\scriptstyle\mbox{\tiny def}}}{{=}}$
$\displaystyle\int\limits_{\Lambda_{1}}\int\limits_{\Lambda_{2}}\cdots\int\limits_{\Lambda_{m}}\overline{\psi}_{\acute{\otimes}}(\lambda_{1},\lambda_{2},\cdots,\lambda_{m})dA_{1}(\lambda_{1})X_{1}dA_{2}(\lambda_{2})X_{2}\cdots
X_{m-1}dA_{m}(\lambda_{m}),$ (16)

where
$\overline{\psi}_{\acute{\otimes}}(\lambda_{1},\lambda_{2},\cdots,\lambda_{m})$
can be expressed as

$\displaystyle\overline{\psi}_{\acute{\otimes}}(\lambda_{1},\lambda_{2},\cdots,\lambda_{m})$
$\displaystyle=$
$\displaystyle\int\limits_{\Xi}\overline{f}_{1}(\lambda_{1},x)\overline{f}_{2}(\lambda_{2},x)\cdots\overline{f}_{m}(\lambda_{m},x)d\mu(x).$
(17)

Below, we will extend our previous work about DOI definition of Hermitian
tensors to MOI definition based on Definitions 1 or 2. We use the symbol
$\mathcal{A}\star_{k}\mathcal{B}$ to represent the multiplication operation
between two tensors with $k$ common indices [13]. From Theorem 3.2 in [13],
every Hermitian tensor $\mathcal{H}\in\mathbb{C}^{I_{1}\times\cdots\times
I_{N}\times I_{1}\times\cdots\times I_{N}}$ has the following decomposition

$\displaystyle\mathcal{H}$ $\displaystyle=$
$\displaystyle\sum\limits_{i=1}^{\mathbb{I}_{1}^{N}}\lambda_{i}\mathcal{U}_{i}\star_{1}\mathcal{U}^{H}_{i}\mbox{
\leavevmode\nobreak\ with\leavevmode\nobreak\ \leavevmode\nobreak\
$\langle\mathcal{U}_{i},\mathcal{U}_{i}\rangle=1$ and
$\langle\mathcal{U}_{i},\mathcal{U}_{j}\rangle=0$ for $i\neq j$,}$ (18)
$\displaystyle\stackrel{{\scriptstyle\mbox{\tiny def}}}{{=}}$
$\displaystyle\sum\limits_{i=1}^{\mathbb{I}_{1}^{N}}\lambda_{i}\mathcal{P}_{\mathcal{U}_{i}}$

where $\mathcal{U}_{i}\in\mathbb{C}^{I_{1}\times\cdots\times I_{N}\times 1}$,
and the tensor $\mathcal{P}_{\mathcal{U}_{i}}$ is defined as
$\mathcal{U}_{i}\star_{1}\mathcal{U}^{H}_{i}$. The values $\lambda_{i}$ are
named as _eigevalues_. A Hermitian tensor with the decomposition shown by Eq.
(18) is named as _eigen-decomposition_. A Hermitian tensor $\mathcal{H}$ is a
positive definite (or positive semi-definite) tensor if all its eigenvalues
are positive (or nonnegative).

Let $\mathcal{A}_{i}\in\mathbb{C}^{I_{1}\times\cdots\times I_{N}\times
I_{1}\times\cdots\times I_{N}}$ for $i=1,2,\cdots,m$ be $m$ Hermitian tensors
with the following eigen-decompositions:

$\displaystyle\mathcal{A}_{i}$ $\displaystyle=$
$\displaystyle\sum\limits_{j=1}^{\mathbb{I}_{1}^{N}}\lambda_{i,j}\mathcal{U}_{i,j}\star_{1}\mathcal{U}^{H}_{i,j}\stackrel{{\scriptstyle\mbox{\tiny
def}}}{{=}}\sum\limits_{j=1}^{\mathbb{I}_{1}^{N}}\lambda_{i,j}\mathcal{P}_{\mathcal{U}_{i,j}}.$
(19)

We define _multiple tensor integrals_ (MTIs) with respect to tensors
$\mathcal{A}_{i}$ and the function $\psi:\mathbb{R}^{m}\rightarrow\mathbb{R}$
which is either constructible by the _limit of projective tensor product_ or
by the _limit of integral projective tensor product_ , denoted as
$T^{\underline{\mathcal{A}}_{1}^{m}}_{\overline{\psi}}(\underline{\mathcal{X}}_{1}^{m-1})$,
which can be expressed as

$\displaystyle
T^{\underline{\mathcal{A}}_{1}^{m}}_{\overline{\psi}}(\underline{\mathcal{X}}_{1}^{m-1})=$
(20)
$\displaystyle\sum\limits_{i_{1}=1}^{\mathbb{I}_{1}^{N}}\sum\limits_{i_{2}=1}^{\mathbb{I}_{1}^{N}}\cdots\sum\limits_{i_{m}=1}^{\mathbb{I}_{1}^{N}}\overline{\psi}(\lambda_{1},\lambda_{2},\cdots,\lambda_{m})\mathcal{P}_{\mathcal{U}_{1,i_{1}}}\star_{N}\mathcal{X}_{1}\star_{N}\mathcal{P}_{\mathcal{U}_{2,i_{1}}}\star_{N}\mathcal{X}_{2}\star_{N}\cdots\star_{N}\mathcal{X}_{m-1}\star_{N}\mathcal{P}_{\mathcal{U}_{m,i_{m}}},$

where $\mathcal{X}_{i}\in\mathbb{C}^{I_{1}\times\cdots\times I_{N}\times
I_{1}\times\cdots\times I_{N}}$ for $i=1,2,\cdots,m-1$.

##  3 Integrand Approximation By Linear Functions

In this section, we will show that it is possible to represent any function
$f(\lambda_{1},\lambda_{2},\cdots,\lambda_{m})$ defined on a compact set of a
Euclidean space $\mathbb{R}^{m}$ by the _limit of projective tensor product_
of linear functions.

We begin with the following theorem from [14] which says that any multivariate
polynomial in $\mathbb{R}^{m}$ can be expressed by a polynomial of inner
product forms.

###### Theorem 2

Let $\mathfrak{P}_{k}^{m}$ denote the set of all polynomials in
$\mathbb{R}^{m}$ with polynomial degree at most $k$, and let
$\bm{x}^{\bm{j}_{i,d}}=x_{1}^{j_{i,d,1}}x_{2}^{j_{i,d,2}}\cdots
x_{m}^{j_{i,d,m}}$ such that $j_{i,d,1}+j_{i,d,2}+\cdots+j_{i,d,m}=i$ for any
$d$. For any polynomial in $\mathfrak{P}_{k}^{m}$, we can express it as

$\displaystyle
p(\bm{x})=\sum\limits_{i=0}^{k}\sum\limits_{d=1}^{n_{i}}a_{i,d}\bm{x}^{\bm{j}_{i,d}},$
(21)

where coefficients $a_{i,d}\in\mathbb{R}$ and $n_{i}={m+i-1\choose i}$. Note
that the number $n_{i}$ indicates the total number of $i$-th degree
polynomials in $m$ variables. Then, there exist a set of coefficients
$c_{i,d}\in\mathbb{R}$ and a set of real vectors
$\bm{v}_{i,d}\in\mathbb{R}^{m}$ such that Eq. (21) can be expressed as:

$\displaystyle
p(\bm{x})=\sum\limits_{i=0}^{k}\sum\limits_{d=1}^{n_{i}}c_{i,d}\langle\bm{x},\bm{v}_{i,d}\rangle^{i}.$
(22)

Following theorem will apply Theorem 2 to show that any multivariate function
$f(\lambda_{1},\lambda_{2},\cdots,\lambda_{m})$ defined on a compact set of a
Euclidean space $\mathbb{R}^{m}$ can be approximated by the _limit of
projective tensor product_ of linear functions.

###### Theorem 3

Given a continous function $f(\bm{x})$ with $\bm{x}\in\mathbb{R}^{m}$ defined
on a compact set $\mathscr{C}\in\mathbb{R}^{m}$ and any $\epsilon>0$, we can
find the following polynomial $p(\bm{x})$ for all $\bm{x}\in\mathscr{C}$
expressed as

$\displaystyle
p(\bm{x})=\sum\limits_{i=1}^{N}\prod_{j=1}^{i}\langle\check{\bm{x}},\bm{u}_{i,j}\rangle,$
(23)

where $\bm{u}_{i,j}\in\mathbb{R}^{m+1}$ and $\check{\bm{x}}=[\bm{x},1]$; such
that

$\displaystyle\left|f(\bm{x})-p(\bm{x})\right|<\epsilon.$ (24)

Note that $N=\sum\limits_{i=0}^{k}n_{i}$, where $n_{i}$ comes from the
expression for $p(\bm{x})$ provided by Eq. (21).

Proof:

Our first goal is to show that $p(\bm{x})\in\mathfrak{P}_{k}^{m}$ shown by Eq.
(21) can be expressed as

$\displaystyle
p(\bm{x})=\sum\limits_{i=1}^{N}\prod\limits_{j=1}^{i}\langle\check{\bm{x}},\bm{u}_{i,j}\rangle,$
(25)

where $N=\sum\limits_{i=0}^{k}n_{i}$ and $\bm{u}_{i,j}\in\mathbb{R}^{m+1}$.

From Theorem 2, we have

$\displaystyle p(\bm{x})$ $\displaystyle=$
$\displaystyle\sum\limits_{i=0}^{k}\sum\limits_{d=1}^{n_{i}}c_{i,d}\langle\bm{x},\bm{v}_{i,d}\rangle^{i}$
(26) $\displaystyle=$
$\displaystyle\sum\limits_{i=1}^{n_{0}}(\langle\bm{x},\bm{0}\rangle+c_{0,i})(\langle\bm{x},\bm{0}\rangle+1)^{i-1}$
$\displaystyle+\sum\limits_{i=n_{0}+1}^{n_{0}+n_{1}}(\langle\bm{x},c_{1,i-n_{0}}\bm{v}_{1,i-n_{0}}\rangle+0)(\langle\bm{x},\bm{0}\rangle+1)^{i-1}$
$\displaystyle+\cdots$
$\displaystyle+\sum\limits_{i=n_{0}+\cdots+n_{k-1}+1}^{n_{0}+\cdots+n_{k}}\Big{[}(\langle\bm{x},c_{k,i-(n_{0}+\cdots+n_{k-1})}\bm{v}_{k,i-(n_{0}+\cdots+n_{k-1})}\rangle+0)$
$\displaystyle\leavevmode\nobreak\ \leavevmode\nobreak\ \leavevmode\nobreak\
\cdot(\langle\bm{x},\bm{v}_{k,i-(n_{0}+\cdots+n_{k-1})}\rangle+0)^{k-1}(\langle\bm{x},\bm{0}\rangle+1)^{i-k}\Big{]}.$

On the other hand, from Eq. (25), we can express $p(\bm{x})$ as

$\displaystyle p(\bm{x})$ $\displaystyle=$
$\displaystyle\sum\limits_{i=1}^{N}\prod\limits_{j=1}^{i}\langle\check{\bm{x}},\bm{u}_{i,j}\rangle$
(27) $\displaystyle=$
$\displaystyle\sum\limits_{i=1}^{n_{0}}\prod\limits_{j=1}^{i}\langle\check{\bm{x}},\bm{u}_{i,j}\rangle$
$\displaystyle+\sum\limits_{i=n_{0}+1}^{n_{0}+n_{1}}\prod\limits_{j=1}^{i}\langle\check{\bm{x}},\bm{u}_{i,j}\rangle+\cdots$
$\displaystyle+\sum\limits_{i=n_{0}+\cdots+n_{k-1}+1}^{n_{0}+\cdots+n_{k}}\prod\limits_{j=1}^{i}\langle\check{\bm{x}},\bm{u}_{i,j}\rangle.$

Because the expression in Eq. (26) is a special case of Eq. (27), we have the
expression of polynomial $p(\bm{x})\in\mathfrak{P}_{k}^{m}$ indicated by Eq.
(22). Finally, this theorem is proved from generalized Weierstrass polynomial
approximation Theorem [15]. $\hfill\Box$

##  4 MOIs Properties and Random MOIs

In this section, we will discuss MOI properties and random MOI for the
integrand function constructible by the _limit of integral projective tensor
product, $\acute{\otimes}$_. Same arguments will also be applied to integrand
function constructible by the _limit of projective tensor product,
$\hat{\otimes}$_.

###  4.1 Properties of MOI

####  4.1.1 Algebraic Properties of MOI

###### Theorem 4

Let $(\Lambda_{1},A_{1}),(\Lambda_{2},A_{2}),\cdots,(\Lambda_{m},A_{m})$ be
spaces with spectral measures $\underline{A}_{1}^{m}$ on a HIlbert space
$\mathfrak{H}$.

  1. (a)

Let $\overline{\phi}_{\acute{\otimes}},\overline{\psi}_{\acute{\otimes}}\in
L^{\infty}(A_{1})\acute{\otimes}\cdots\acute{\otimes}L^{\infty}(A_{m})$ and
$\alpha,\beta\in\mathbb{C}$, then we have

$\displaystyle
T^{\underline{A}_{1}^{m}}_{\alpha\overline{\phi}_{\acute{\otimes}}+\beta\overline{\psi}_{\acute{\otimes}}}(\underline{X}_{1}^{m-1})=\alpha
T^{\underline{A}_{1}^{m}}_{\overline{\phi}_{\acute{\otimes}}}(\underline{X}_{1}^{m-1})+\beta
T^{\underline{A}_{1}^{m}}_{\overline{\psi}_{\acute{\otimes}}}(\underline{X}_{1}^{m-1}).$
(28)

  2. (b)

Given $\overline{\psi}_{\acute{\otimes},1},\in
L^{\infty}(A_{1})\acute{\otimes}\cdots\acute{\otimes}L^{\infty}(A_{k})$ and
$\overline{\psi}_{\acute{\otimes},2},\in
L^{\infty}(A_{k+1})\acute{\otimes}\cdots\acute{\otimes}L^{\infty}(A_{m})$, we
define
$\overline{\psi}_{\acute{\otimes},1}\oplus\overline{\psi}_{\acute{\otimes},2}$
as
$\overline{\psi}_{\acute{\otimes},1}\oplus\overline{\psi}_{\acute{\otimes},2}\stackrel{{\scriptstyle\mbox{\tiny
def}}}{{=}}\overline{\psi}_{\acute{\otimes},1}\overline{\psi}_{\acute{\otimes},2}$,
then we have

$\displaystyle\overline{\psi}_{\acute{\otimes},1}\oplus\overline{\psi}_{\acute{\otimes},2}$
$\displaystyle\in$ $\displaystyle
L^{\infty}(A_{1})\acute{\otimes}\cdots\acute{\otimes}L^{\infty}(A_{m})$ (29)

and

$\displaystyle
T^{(\underline{A}_{1}^{m})}_{\overline{\psi}_{\acute{\otimes},1}\oplus\overline{\psi}_{\acute{\otimes},2}}(\underline{X}_{1}^{m-1})=T^{(\underline{A}_{1}^{k})}_{\overline{\phi}_{\acute{\otimes},1}}(\underline{X}_{1}^{k-1})X_{k}T^{(\underline{A}_{k+1}^{m})}_{\overline{\psi}_{\acute{\otimes},2}}(\underline{X}_{k+1}^{m-1}).$
(30)

  3. (c)

Let $\overline{\phi}_{\acute{\otimes}}\in
L^{\infty}(A_{1})\acute{\otimes}\cdots\acute{\otimes}L^{\infty}(A_{m})$, and
$\overline{\psi}_{\acute{\otimes},i}\in
L^{\infty}(A_{i,1})\acute{\otimes}\cdots\acute{\otimes}L^{\infty}(A_{i,j_{i}})$
for $i=1,2,\cdots,\ell$, be integrand functions composed by the $i$-th
partition from $\underline{A}_{1}^{m}$ into $\ell$ segments such that each
segment will have at least one element. For example,
$(A_{1},A_{2},A_{3},A_{4},A_{5})$ is partitioned into $(A_{1},A_{2})$,
$(A_{3})$, $(A_{4},A_{5})$. Suppose we have the following relationship between
the function $\overline{\phi}_{\acute{\otimes}}$ and functions
$\overline{\psi}_{\acute{\otimes},i}$ for $i=1,2,\cdots,\ell$:

$\displaystyle\overline{\phi}_{\acute{\otimes}}(\lambda_{1},\cdots,\lambda_{m})=\prod\limits_{i=1}^{\ell}\overline{\psi}_{\acute{\otimes},i}(\underline{\lambda}_{i}),$
(31)

where $\underline{\lambda}_{i}$ are those $j_{i}$ eigenvalues corresponding to
spectral measures $A_{i,1},\cdots,A_{i,j_{i}}$. Then, we hve

$\displaystyle
T^{\underline{A}_{1}^{m}}_{\overline{\phi}_{\acute{\otimes}}}(\underline{X}_{1}^{m-1})=\left(\prod\limits_{i=1}^{\ell-1}T^{\underline{A}_{j_{i-1}+1}^{j_{i}}}_{\overline{\psi}_{\acute{\otimes},i}}(\underline{X}_{j_{i-1}+1}^{j_{i}-1})X_{j_{i}}\right)T^{\underline{A}_{j_{\ell-1}+1}^{j_{\ell}}}_{\overline{\psi}_{\acute{\otimes},\ell}}(\underline{X}_{j_{\ell-1}+1}^{j_{\ell}-1}),$
(32)

where $j_{0}=0$ and $j_{\ell}=m$.

Proof:

The proof for part (a) is trivial from linearity of integration.

For part (b), because we have

$\displaystyle\overline{\psi}_{\acute{\otimes},1}(\lambda_{1},\lambda_{2},\cdots,\lambda_{k})$
$\displaystyle=$
$\displaystyle\int\limits_{\Xi_{1}}\overline{f}_{1}(\lambda_{1},x)\overline{f}_{2}(\lambda_{2},x)\cdots\overline{f}_{k}(\lambda_{k},x)d\mu_{1}(x),$
(33)

and

$\displaystyle\overline{\psi}_{\acute{\otimes},2}(\lambda_{k+1},\lambda_{k+2},\cdots,\lambda_{m})$
$\displaystyle=$
$\displaystyle\int\limits_{\Xi_{2}}\overline{f}_{{k+1}}(\lambda_{k+1},x)\overline{f}_{k+2}(\lambda_{k+2},x)\cdots\overline{f}_{m}(\lambda_{m},x)d\mu_{2}(x),$
(34)

then,

$\displaystyle\overline{\psi}_{\acute{\otimes},1}\oplus\overline{\psi}_{\acute{\otimes},2}$
$\displaystyle=$
$\displaystyle\left(\int\limits_{\Xi_{1}}\overline{f}_{1}(\lambda_{1},x)\overline{f}_{2}(\lambda_{2},x)\cdots\overline{f}_{k}(\lambda_{k},x)d\mu_{1}(x)\right)$
(35)
$\displaystyle\cdot\left(\int\limits_{\Xi_{2}}\overline{f}_{{k+1}}(\lambda_{k+1},x)\overline{f}_{k+2}(\lambda_{k+2},x)\cdots\overline{f}_{m}(\lambda_{m},x)d\mu_{2}(x)\right)$
$\displaystyle=$
$\displaystyle\int\limits_{\Xi_{1}\times\Xi_{2}}\overline{f}_{1}(\lambda_{1},x)\cdots\overline{f}_{k}(\lambda_{k},x)\overline{f}_{{k+1}}(\lambda_{k+1},x)\cdots\overline{f}_{m}(\lambda_{m},x)d\mu_{1}(x)d\mu_{2}(x).$

Eq. (29) is true from the product measure space.

Since we have

$\displaystyle
T^{\underline{A}_{1}^{m}}_{\overline{\psi}_{\acute{\otimes},1}\oplus\overline{\psi}_{\acute{\otimes},2}}(\underline{X}_{1}^{m-1})$
$\displaystyle=$
$\displaystyle\int\limits_{\Lambda_{1}}\int\limits_{\Lambda_{2}}\cdots\int\limits_{\Lambda_{m}}\overline{\psi}_{\acute{\otimes},1}\overline{\psi}_{\acute{\otimes},2}dA_{1}(\lambda_{1})X_{1}dA_{2}(\lambda_{2})X_{2}\cdots
X_{m-1}dA_{m}(\lambda_{m})$ (36) $\displaystyle=$
$\displaystyle\left(\int\limits_{\Lambda_{1}}\cdots\int\limits_{\Lambda_{k}}\overline{\psi}_{\acute{\otimes},1}dA_{1}(\lambda_{1})X_{1}d\cdots
X_{k-1}dA_{k}(\lambda_{k})\right)X_{k}$
$\displaystyle\int\limits_{\Lambda_{k+1}}\cdots\int\limits_{\Lambda_{m}}\overline{\psi}_{\acute{\otimes},2}dA_{k+1}(\lambda_{k+1})X_{k+1}\cdots
X_{m-1}dA_{m}(\lambda_{m})$ $\displaystyle=$ $\displaystyle
T^{(\underline{A}_{1}^{k})}_{\overline{\phi}_{\acute{\otimes}},1}(\underline{X}_{1}^{k-1})X_{k}T^{(\underline{A}_{k+1}^{m})}_{\overline{\psi}_{\acute{\otimes}},2}(\underline{X}_{k+1}^{m-1}).$

Therefore, we have Eq. (30).

For part (c), we have

$\displaystyle
T^{\underline{A}_{1}^{m}}_{\overline{\phi}_{\acute{\otimes}}}(\underline{A}_{1}^{m-1})$
$\displaystyle=$
$\displaystyle\int\limits_{\Lambda_{1}}\int\limits_{\Lambda_{2}}\cdots\int\limits_{\Lambda_{m}}\overline{\psi}_{\acute{\otimes}}dA_{1}(\lambda_{1})X_{1}dA_{2}(\lambda_{2})X_{2}\cdots
X_{m-1}dA_{m}(\lambda_{m})$ (37) $\displaystyle=_{1}$
$\displaystyle\int\limits_{\Lambda_{1}}\int\limits_{\Lambda_{2}}\cdots\int\limits_{\Lambda_{m}}\left(\prod\limits_{i=1}^{\ell}\overline{\psi}_{\acute{\otimes},i}(\underline{\lambda}_{i})\right)dA_{1}(\lambda_{1})X_{1}dA_{2}(\lambda_{2})X_{2}\cdots
X_{m-1}dA_{m}(\lambda_{m})$ $\displaystyle=$
$\displaystyle\underbrace{\left(\int\limits_{\Lambda_{1}}\cdots\int\limits_{\Lambda_{j_{1}}}\overline{\psi}_{\acute{\otimes},1}(\underline{\lambda}_{1})dA_{1}(\lambda_{1})X_{1}\cdots
X_{j_{1}-1}dA_{j_{1}}(\lambda_{j_{1}})\right)}_{T^{\underline{A}_{1}^{j_{1}}}_{\overline{\psi}_{\acute{\otimes},1}}(\underline{X}_{1}^{j_{1}-1})}X_{j_{1}}$
$\displaystyle\cdot\underbrace{\left(\int\limits_{\Lambda_{j_{1}+1}}\cdots\int\limits_{\Lambda_{j_{2}}}\overline{\psi}_{\acute{\otimes},2}(\underline{\lambda}_{2})dA_{j_{1}+1}(\lambda_{j_{1}+1})X_{j_{1}+1}\cdots
X_{j_{2}-1}dA_{j_{2}}(\lambda_{j_{2}})\right)}_{T^{\underline{A}_{j_{1}+1}^{j_{2}}}_{\overline{\psi}_{\acute{\otimes},2}}(\underline{X}_{j_{1}+1}^{j_{2}-1})}X_{j_{2}}$
$\displaystyle\cdots\underbrace{\left(\int\limits_{\Lambda_{j_{\ell-1}+1}}\cdots\int\limits_{\Lambda_{j_{\ell}}}\overline{\psi}_{\acute{\otimes},\ell}(\underline{\lambda}_{\ell})dA_{j_{\ell-1}+1}(\lambda_{j_{\ell-1}+1})X_{j_{\ell-1}+1}\cdots
X_{j_{\ell}-1}dA_{j_{\ell}}(\lambda_{j_{\ell}})\right)}_{T^{\underline{A}_{j_{\ell-1}}^{j_{\ell}}}_{\overline{\psi}_{\acute{\otimes},\ell}}(\underline{X}_{j_{\ell-1}+1}^{j_{\ell}-1})}$
$\displaystyle=$
$\displaystyle\left(\prod\limits_{i=1}^{\ell-1}T^{\underline{A}_{j_{i-1}+1}^{j_{i}}}_{\overline{\psi}_{\acute{\otimes},i}}(\underline{X}_{j_{i-1}+1}^{j_{i}-1})X_{j_{i}}\right)T^{\underline{A}_{j_{\ell-1}+1}^{j_{\ell}}}_{\overline{\psi}_{\acute{\otimes},\ell}}(\underline{X}_{j_{\ell-1}+1}^{j_{\ell}-1}),$

where we apply Eq. (31) to $=_{1}$ and set $j_{0}=1$. Note that $j_{\ell}=m$.
$\hfill\Box$

####  4.1.2 Norm Estimations for MOIs

Following lemma will provide two norm estimations of MOIs.

###### Lemma 1

According to Definition 2, we have the following norm estimations for Eq.
(16):

  1. (a)

Given a bounded measurable function
$\overline{\psi}_{\acute{\otimes}}(\lambda_{1},\lambda_{2},\cdots,\lambda_{m})$
($m$-variables) constructible by limit of integral projective tensor product,
and a set of $(m-1)$ bounded linear operators $X_{1},X_{2},\cdots,X_{m-1}$ on
Hilber space $\mathfrak{H}$, we have

$\displaystyle\left\|\int\limits_{\Lambda_{1}}\int\limits_{\Lambda_{2}}\cdots\int\limits_{\Lambda_{m}}\overline{\psi}_{\acute{\otimes}}(\lambda_{1},\lambda_{2},\cdots,\lambda_{m})dA_{1}(\lambda_{1})X_{1}dA_{2}(\lambda_{2})X_{2}\cdots
X_{m-1}dA_{m}(\lambda_{m})\right\|$ (38) $\displaystyle\leq$
$\displaystyle\left\|\overline{\psi}_{\acute{\otimes}}(\lambda_{1},\lambda_{2},\cdots,\lambda_{m})\right\|\prod\limits_{i=1}^{m-1}\left\|X_{i}\right\|.\leavevmode\nobreak\
\leavevmode\nobreak\ \leavevmode\nobreak\ \leavevmode\nobreak\
\leavevmode\nobreak\ \leavevmode\nobreak\ \leavevmode\nobreak\
\leavevmode\nobreak\ \leavevmode\nobreak\ \leavevmode\nobreak\
\leavevmode\nobreak\ \leavevmode\nobreak\ \leavevmode\nobreak\
\leavevmode\nobreak\ \leavevmode\nobreak\ \leavevmode\nobreak\
\leavevmode\nobreak\ \leavevmode\nobreak\ \leavevmode\nobreak\
\leavevmode\nobreak\ \leavevmode\nobreak\ \leavevmode\nobreak\
\leavevmode\nobreak\ \leavevmode\nobreak\ \leavevmode\nobreak\
\leavevmode\nobreak\ \leavevmode\nobreak\ \leavevmode\nobreak\
\leavevmode\nobreak\ \leavevmode\nobreak\ \leavevmode\nobreak\
\leavevmode\nobreak\ \leavevmode\nobreak\ \leavevmode\nobreak\
\leavevmode\nobreak\ \leavevmode\nobreak\ \leavevmode\nobreak\
\leavevmode\nobreak\ \leavevmode\nobreak\ \leavevmode\nobreak\
\leavevmode\nobreak\ \leavevmode\nobreak\ \leavevmode\nobreak\
\leavevmode\nobreak\ \leavevmode\nobreak\ \leavevmode\nobreak\
\leavevmode\nobreak\ \leavevmode\nobreak\ \leavevmode\nobreak\
\leavevmode\nobreak\ \leavevmode\nobreak\ \leavevmode\nobreak\
\leavevmode\nobreak\ \leavevmode\nobreak\ \leavevmode\nobreak\
\leavevmode\nobreak\ \leavevmode\nobreak\ \leavevmode\nobreak\
\leavevmode\nobreak\ \leavevmode\nobreak\ \leavevmode\nobreak\
\leavevmode\nobreak\ \leavevmode\nobreak\ \leavevmode\nobreak\
\leavevmode\nobreak\ \leavevmode\nobreak\ \leavevmode\nobreak\
\leavevmode\nobreak\ \leavevmode\nobreak\ \leavevmode\nobreak\
\leavevmode\nobreak\ \leavevmode\nobreak\ $

  2. (b)

Given a bounded measurable function
$\overline{\psi}_{\acute{\otimes}}(\lambda_{1},\lambda_{2},\cdots,\lambda_{m})$
($m$-variables) constructible by limit of integral projective tensor product,
and a set of $(m-1)$ bounded linear operators $X_{1},X_{2},\cdots,X_{m-1}$ on
Hilber space $\mathfrak{H}$ such that $X_{i}\in\mathscr{S}_{p_{i}}$, where
$\mathscr{S}_{p_{i}}$ is a Schatten–von Neumann class with parameter $p_{i}$,
we also require that $\sum\limits_{i=1}^{m}\frac{1}{p_{i}}\leq 1$. Then, we
have

$\displaystyle\left\|\int\limits_{\Lambda_{1}}\int\limits_{\Lambda_{2}}\cdots\int\limits_{\Lambda_{m}}\overline{\psi}_{\acute{\otimes}}(\lambda_{1},\lambda_{2},\cdots,\lambda_{m})dA_{1}(\lambda_{1})X_{1}dA_{2}(\lambda_{2})X_{2}\cdots
X_{m-1}dA_{m}(\lambda_{m})\right\|_{\mathscr{S}_{q}}$ (39) $\displaystyle\leq$
$\displaystyle\left\|\overline{\psi}_{\acute{\otimes}}(\lambda_{1},\lambda_{2},\cdots,\lambda_{m})\right\|\prod\limits_{i=1}^{m-1}\left\|X_{i}\right\|_{\mathscr{S}_{p_{i}}},\leavevmode\nobreak\
\leavevmode\nobreak\ \leavevmode\nobreak\ \leavevmode\nobreak\
\leavevmode\nobreak\ \leavevmode\nobreak\ \leavevmode\nobreak\
\leavevmode\nobreak\ \leavevmode\nobreak\ \leavevmode\nobreak\
\leavevmode\nobreak\ \leavevmode\nobreak\ \leavevmode\nobreak\
\leavevmode\nobreak\ \leavevmode\nobreak\ \leavevmode\nobreak\
\leavevmode\nobreak\ \leavevmode\nobreak\ \leavevmode\nobreak\
\leavevmode\nobreak\ \leavevmode\nobreak\ \leavevmode\nobreak\
\leavevmode\nobreak\ \leavevmode\nobreak\ \leavevmode\nobreak\
\leavevmode\nobreak\ \leavevmode\nobreak\ \leavevmode\nobreak\
\leavevmode\nobreak\ \leavevmode\nobreak\ \leavevmode\nobreak\
\leavevmode\nobreak\ \leavevmode\nobreak\ \leavevmode\nobreak\
\leavevmode\nobreak\ \leavevmode\nobreak\ \leavevmode\nobreak\
\leavevmode\nobreak\ \leavevmode\nobreak\ \leavevmode\nobreak\
\leavevmode\nobreak\ \leavevmode\nobreak\ \leavevmode\nobreak\
\leavevmode\nobreak\ \leavevmode\nobreak\ \leavevmode\nobreak\
\leavevmode\nobreak\ \leavevmode\nobreak\ \leavevmode\nobreak\
\leavevmode\nobreak\ \leavevmode\nobreak\ \leavevmode\nobreak\
\leavevmode\nobreak\ \leavevmode\nobreak\ \leavevmode\nobreak\
\leavevmode\nobreak\ \leavevmode\nobreak\ \leavevmode\nobreak\
\leavevmode\nobreak\ \leavevmode\nobreak\ \leavevmode\nobreak\
\leavevmode\nobreak\ \leavevmode\nobreak\ \leavevmode\nobreak\
\leavevmode\nobreak\ \leavevmode\nobreak\ \leavevmode\nobreak\
\leavevmode\nobreak\ \leavevmode\nobreak\ $

where $\frac{1}{q}=1-\sum\limits_{i=1}^{m}\frac{1}{p_{i}}$.

Proof: By applying triangle inequality for integrals and Hölder’s inequality
to Eq. (16). $\hfill\Box$

####  4.1.3 Perturbation Formula for MOI

Before presenting our perturbation formula for MOI, we have to define divided
differences for function $\overline{\psi}_{\acute{\otimes}}(\lambda)$, which
is a function constructible by _limit of integral projective tensor product,
$\acute{\otimes}$_. We assume this function can have $n$ times continuously
differentiable on $\mathbb{R}$. The divided difference
$\overline{\psi}^{[n]}_{\acute{\otimes}}$ of order $n$ with respect to the
$j$-th variable is defined by

$\displaystyle\overline{\psi}^{[n+1]}_{\acute{\otimes}}(\lambda_{0},\cdots,\lambda_{n+1})\stackrel{{\scriptstyle\mbox{\tiny
def}}}{{=}}$ (40)
$\displaystyle\frac{\overline{\psi}^{[n]}_{\acute{\otimes}}(\lambda_{0},\cdots,\lambda_{j-1},\lambda_{j+1},\cdots,\lambda_{n+1})-\overline{\psi}^{[n]}_{\acute{\otimes}}(\lambda_{0},\cdots,\lambda_{j-2},\lambda_{j},\cdots,\lambda_{n+1})}{\lambda_{j}-\lambda_{j-1}}.$

We have the following perturbation formula for MOIs.

###### Lemma 2

Let $\overline{\psi}_{\acute{\otimes}}$ be $n$ times differentiable on
$\mathbb{R}$ and $C,D,A_{1},\cdots,A_{n-1}$ be operators. Then, we have

$\displaystyle
T^{\underline{A}_{1}^{j-1},C,\underline{A}_{j}^{m}}_{\overline{\psi}^{[m]}_{\acute{\otimes}}}(\underline{X}_{1}^{m})-T^{\underline{A}_{1}^{j-1},D,\underline{A}_{j}^{m}}_{\overline{\psi}^{[m]}_{\acute{\otimes}}}(\underline{X}_{1}^{m})=T^{\underline{A}_{1}^{j-1},C,D,\underline{A}_{j}^{m}}_{\overline{\psi}^{[m+1]}_{\acute{\otimes}}}(\underline{X}_{1}^{j-1},C-D,\underline{X}_{j}^{m}),$
(41)

where $m\leq n-1$.

Proof: For $1\leq j\leq m+1$, we define

$\displaystyle f_{j}(\lambda_{0},\cdots,\lambda_{m+1})$
$\displaystyle\stackrel{{\scriptstyle\mbox{\tiny def}}}{{=}}$
$\displaystyle\lambda_{j}\overline{\psi}^{[m+1]}_{\acute{\otimes}}(\lambda_{0},\cdots,\lambda_{m+1}),$
$\displaystyle g_{j}(\lambda_{0},\cdots,\lambda_{m+1})$
$\displaystyle\stackrel{{\scriptstyle\mbox{\tiny def}}}{{=}}$
$\displaystyle\overline{\psi}^{[m]}_{\acute{\otimes}}(\lambda_{0},\cdots,\lambda_{j-1},\lambda_{j+1},\cdots,\lambda_{m+1}).$
(42)

From Eq. (40), we have

$\displaystyle f_{j}-f_{j-1}=g_{j}-g_{j-1}.$ (43)

Then, we have

$\displaystyle
T^{\underline{A}_{1}^{j-1},C,D,\underline{A}_{j}^{m}}_{\overline{\psi}^{[m+1]}_{\acute{\otimes}}}(\underline{X}_{1}^{j-1},C-D,\underline{X}_{j}^{m})$
(44) $\displaystyle=$ $\displaystyle
T^{\underline{A}_{1}^{j-1},C,D,\underline{A}_{j}^{m}}_{\overline{\psi}^{[m+1]}_{\acute{\otimes}}}(\underline{X}_{1}^{j-1},C,\underline{X}_{j}^{m})-T^{\underline{A}_{1}^{j-1},C,D,\underline{A}_{j}^{m}}_{\overline{\psi}^{[m+1]}_{\acute{\otimes}}}(\underline{X}_{1}^{j-1},D,\underline{X}_{j}^{m})$
$\displaystyle=$ $\displaystyle
T^{\underline{A}_{1}^{j-1},C,D,\underline{A}_{j}^{m}}_{f_{j-1}}(\underline{X}_{1}^{j-1},Id,\underline{X}_{j}^{m})-T^{\underline{A}_{1}^{j-1},C,D,\underline{A}_{j}^{m}}_{f_{j}}(\underline{X}_{1}^{j-1},Id,\underline{X}_{j}^{m})$
$\displaystyle=$ $\displaystyle
T^{\underline{A}_{1}^{j-1},C,D,\underline{A}_{j}^{m}}_{f_{j-1}-f_{j}}(\underline{X}_{1}^{j-1},Id,\underline{X}_{j}^{m})$
$\displaystyle=_{1}$ $\displaystyle
T^{\underline{A}_{1}^{j-1},C,D,\underline{A}_{j}^{m}}_{g_{j-1}-g_{j}}(\underline{X}_{1}^{j-1},Id,\underline{X}_{j}^{m})$
$\displaystyle=$ $\displaystyle
T^{\underline{A}_{1}^{j-1},C,D,\underline{A}_{j}^{m}}_{g_{j-1}}(\underline{X}_{1}^{j-1},Id,\underline{X}_{j}^{m})-T^{\underline{A}_{1}^{j-1},C,D,\underline{A}_{j}^{m}}_{g_{j}}(\underline{X}_{1}^{j-1},Id,\underline{X}_{j}^{m})$
$\displaystyle=$ $\displaystyle
T^{\underline{A}_{1}^{j-1},C,\underline{A}_{j}^{m}}_{\overline{\psi}^{[m]}_{\acute{\otimes}}}(\underline{X}_{1}^{m})-T^{\underline{A}_{1}^{j-1},D,\underline{A}_{j}^{m}}_{\overline{\psi}^{[m]}_{\acute{\otimes}}}(\underline{X}_{1}^{m}),$

where $Id$ is the identity operator and $=_{1}$ comes from Eq. (43).
$\hfill\Box$

####  4.1.4 Continuity for MOIs

In the next lemma, we establish continuity of a MOI defined by Definition 2.

###### Lemma 3

Let $\overline{\psi}_{\acute{\otimes}}$ be $n$ times differentiable on
$\mathbb{R}$ and $A_{i}^{(m)}\rightarrow A_{i}$ as $m\rightarrow\infty$, i.e.,
$\left\|A_{i}^{(m)}-A_{i}\right\|\rightarrow 0$ as $m\rightarrow\infty$. We
use $\underline{A^{(m)}}_{i}^{j}$ to represent the sequence
$A^{(m)}_{i},A^{(m)}_{i+1},\cdots,A^{(m)}_{j}$. Then, we have

$\displaystyle
T^{\underline{A^{(m)}}_{1}^{n+1}}_{\overline{\psi}^{[n]}_{\acute{\otimes}}}(\underline{X}_{1}^{n})\rightarrow
T^{\underline{A}_{1}^{n+1}}_{\overline{\psi}^{[n]}_{\acute{\otimes}}}(\underline{X}_{1}^{n}),$
(45)

where $X_{1},X_{2},\cdots,X_{n}$ are bounded operators by assuming that the
norm for the function $\overline{\psi}^{[n+1]}_{\acute{\otimes}}$ and norms
for the oprators $X_{1},\cdots,X_{n}$ are bounded.

Proof: Because we have

$\displaystyle\left\|T^{\underline{A^{(m)}}_{1}^{n+1}}_{\overline{\psi}^{[n]}_{\acute{\otimes}}}(\underline{X}_{1}^{n})-T^{\underline{A}_{1}^{n+1}}_{\overline{\psi}^{[n]}_{\acute{\otimes}}}(\underline{X}_{1}^{n})\right\|$
$\displaystyle=_{1}$
$\displaystyle\left\|\sum\limits_{i=1}^{n+1}\left(T^{\underline{A}_{1}^{i-1},\underline{A^{(m)}}_{i}^{n+1}}_{\overline{\psi}^{[n]}_{\acute{\otimes}}}(\underline{X}_{1}^{n})-T^{\underline{A}_{1}^{i},\underline{A^{(m)}}_{i+1}^{n+1}}_{\overline{\psi}^{[n]}_{\acute{\otimes}}}(\underline{X}_{1}^{n})\right)\right\|$
(46) $\displaystyle=_{2}$
$\displaystyle\left\|\sum\limits_{i=1}^{n+1}T^{\underline{A}_{1}^{i-1},A^{(m)}_{i},A_{i},\underline{A^{(m)}}_{i+1}^{n+1}}_{\overline{\psi}^{[n+1]}_{\acute{\otimes}}}(\underline{X}_{1}^{i-1},A^{(m)}_{i}-A_{i},\underline{X}_{i}^{n})\right\|$

where $=_{1}$ comes from telescoping summation and $=_{2}$ comes from Lemma 2

Applying norm triangle inequality and Lemma 1, we can upper bound Eq. (46) as

$\displaystyle\left\|T^{\underline{A^{(m)}}_{1}^{n+1}}_{\overline{\psi}^{[n]}_{\acute{\otimes}}}(\underline{X}_{1}^{n})-T^{\underline{A}_{1}^{n+1}}_{\overline{\psi}^{[n]}_{\acute{\otimes}}}(\underline{X}_{1}^{n})\right\|$
$\displaystyle\leq$
$\displaystyle\left\|\overline{\psi}^{[n+1]}_{\acute{\otimes}}\right\|\sum\limits_{i=1}^{n+1}\left(\left\|A^{(m)}_{i}-A_{i}\right\|\prod\limits_{j=1}^{n}\left\|X_{j}\right\|\right).$
(47)

This theorem is provied when $m\rightarrow\infty$. $\hfill\Box$

###  4.2 Random MOIs

From the MOI definition provided by Definition 1 and Definition 2, the random
MOI considered in this work is to assume random operators $A_{i}$ having
spectrum decomposition as
$\int\limits_{\Lambda_{i}}\lambda_{i}dA_{i}(\lambda_{i})$, where
$i=1,2,\cdots,m$. The randomness of the operator $A_{i}$ is determined by
random variables $\lambda_{i}$ and random unitary operators
$dA_{i}(\lambda_{i})$ via Haar measure. The remaining parameters like the
integrand
function$\overline{\psi}_{\hat{\otimes}}(or\overline{\psi}_{\acute{\otimes}}$,
and operators $X_{1},\cdots,X_{m}-1$ are assumed to be deterministic. The
purpose of this section is to derive tail bounds for random MOI norms and
establish the continuity property of random MOI.

###### Theorem 5

We have the following tail bound for the norm estimations:

  1. (a)

Given a random MOI,
$T^{\underline{A}_{1}^{m+1}}_{\overline{\psi}^{[n]}_{\acute{\otimes}}}(\underline{X}_{1}^{m})$,
we have

$\displaystyle\mathrm{Pr}\left(\left\|T^{\underline{A}_{1}^{m+1}}_{\overline{\psi}^{[m]}_{\acute{\otimes}}}(\underline{X}_{1}^{m})\right\|\geq\theta\right)$
$\displaystyle\leq$
$\displaystyle\frac{\prod\limits_{i=1}^{m-1}\left\|X_{i}\right\|}{\theta}\mathbb{E}\left[\left\|\overline{\psi}_{\acute{\otimes}}(\lambda_{1},\lambda_{2},\cdots,\lambda_{m})\right\|\right],$
(48)

where $\theta>0$.

  2. (b)

Given a random MOI,
$T^{\underline{A}_{1}^{m+1}}_{\overline{\psi}^{[m]}_{\acute{\otimes}}}(\underline{X}_{1}^{m})$,
we have

$\displaystyle\mathrm{Pr}\left(\left\|T^{\underline{A}_{1}^{m+1}}_{\overline{\psi}^{[m]}_{\acute{\otimes}}}(\underline{X}_{1}^{m})\right\|_{\mathscr{S}_{q}}\geq\theta\right)$
$\displaystyle\leq$
$\displaystyle\frac{\prod\limits_{i=1}^{m-1}\left\|X_{i}\right\|_{\mathscr{S}_{p_{i}}}}{\theta}\mathbb{E}\left[\left\|\overline{\psi}_{\acute{\otimes}}(\lambda_{1},\lambda_{2},\cdots,\lambda_{m}),\right\|\right],$
(49)

where $\frac{1}{q}=1-\sum\limits_{i=1}^{m}\frac{1}{p_{i}}$ and $\theta>0$.

Proof:

For part (a), since we have

$\displaystyle\mathrm{Pr}\left(\left\|T^{\underline{A}_{1}^{m+1}}_{\overline{\psi}^{[m]}_{\acute{\otimes}}}(\underline{X}_{1}^{m})\right\|\geq\theta\right)$
$\displaystyle\leq_{1}$
$\displaystyle\mathrm{Pr}\left(\left\|\overline{\psi}_{\acute{\otimes}}(\lambda_{1},\lambda_{2},\cdots,\lambda_{m})\right\|\prod\limits_{i=1}^{m-1}\left\|X_{i}\right\|\geq\theta\right)$
(50) $\displaystyle=$
$\displaystyle\mathrm{Pr}\left(\left\|\overline{\psi}_{\acute{\otimes}}(\lambda_{1},\lambda_{2},\cdots,\lambda_{m})\right\|\geq\frac{\theta}{\prod\limits_{i=1}^{m-1}\left\|X_{i}\right\|}\right)$
$\displaystyle\leq_{2}$
$\displaystyle\frac{\prod\limits_{i=1}^{m-1}\left\|X_{i}\right\|}{\theta}\mathbb{E}\left[\left\|\overline{\psi}_{\acute{\otimes}}(\lambda_{1},\lambda_{2},\cdots,\lambda_{m})\right\|\right],$

where $\leq_{1}$ is due to part (a) of Lemma 1, and $\leq_{2}$ comes from
Markov inequality.

For part (b), since we have

$\displaystyle\mathrm{Pr}\left(\left\|T^{\underline{A}_{1}^{m+1}}_{\overline{\psi}^{[m]}_{\acute{\otimes}}}(\underline{X}_{1}^{m})\right\|_{\mathscr{S}_{q}}\geq\theta\right)$
$\displaystyle\leq_{1}$
$\displaystyle\mathrm{Pr}\left(\left\|\overline{\psi}_{\acute{\otimes}}(\lambda_{1},\lambda_{2},\cdots,\lambda_{m})\right\|\prod\limits_{i=1}^{m-1}\left\|X_{i}\right\|_{\mathscr{S}_{p_{i}}}\geq\theta\right)$
(51) $\displaystyle=$
$\displaystyle\mathrm{Pr}\left(\left\|\overline{\psi}_{\acute{\otimes}}(\lambda_{1},\lambda_{2},\cdots,\lambda_{m})\right\|\geq\frac{\theta}{\prod\limits_{i=1}^{m-1}\left\|X_{i}\right\|_{\mathscr{S}_{p_{i}}}}\right)$
$\displaystyle\leq_{2}$
$\displaystyle\frac{\prod\limits_{i=1}^{m-1}\left\|X_{i}\right\|_{\mathscr{S}_{p_{i}}}}{\theta}\mathbb{E}\left[\left\|\overline{\psi}_{\acute{\otimes}}(\lambda_{1},\lambda_{2},\cdots,\lambda_{m})\right\|\right],$

where $\leq_{1}$ is due to part (b) of Lemma 1, and $\leq_{2}$ comes from
Markov inequality. $\hfill\Box$

In the remainder of this section, we will establish continuity of random MOI.
We need the following definition about the convergence in mean for random
operators.

###### Definition 3

We say that a sequence of random operator $X^{(m)}$ converges in the $r$-th
mean towards the random operator $X$ with respect to the operator norm
$\left\|\cdot\right\|$, if we have

$\displaystyle\mathbb{E}\left(\left\|X^{(m)}\right\|^{r}\right)\leavevmode\nobreak\
\leavevmode\nobreak\ \leavevmode\nobreak\ \mbox{exists,}$ (52)

and

$\displaystyle\mathbb{E}\left(\left\|X\right\|^{r}\right)\leavevmode\nobreak\
\leavevmode\nobreak\ \leavevmode\nobreak\ \mbox{exists,}$ (53)

and

$\displaystyle\lim\limits_{m\rightarrow\infty}\mathbb{E}\left(\left\|X^{(m)}-X\right\|^{r}\right)=0.$
(54)

We adopt the notatation $X^{(m)}\xrightarrow[]{r}X$ to represent that random
operators $X^{(m)}$ converges in the $r$-th mean to the random operator $X$
with respect to the norm $\left\|\cdot\right\|$.

Following theorem is about the continuity in the $r$-th mean of random MOI .

###### Theorem 6

For $i=1,2,\cdots,n+1$, we have random self-adjoint operators
$A_{i}^{(m)},A_{i}$ such that

$\displaystyle A_{i}^{(m)}\xrightarrow[]{r}A_{i}\leavevmode\nobreak\
\leavevmode\nobreak\ \mbox{as $m\rightarrow\infty$},$ (55)

where $1\leq r<\infty$. Moreover, the norm for the real valued function
$\overline{\psi}^{[n+1]}_{\acute{\otimes}}$ and the norm for $X_{j}$ for
$j=1,2,\cdots,n$, are assumed bounded. Then, we have

$\displaystyle
T^{\underline{A^{(m)}}_{1}^{n+1}}_{\overline{\psi}^{[n]}_{\acute{\otimes}}}(\underline{X}_{1}^{n})\xrightarrow[]{r}T^{\underline{A}_{1}^{n+1}}_{\overline{\psi}^{[n]}_{\acute{\otimes}}}(\underline{X}_{1}^{n}).$
(56)

Proof:

From the proof in Lemma 3, we have

$\displaystyle\left\|T^{\underline{A^{(m)}}_{1}^{n+1}}_{\overline{\psi}^{[n]}_{\acute{\otimes}}}(\underline{X}_{1}^{n})-T^{\underline{A}_{1}^{n+1}}_{\overline{\psi}^{[n]}_{\acute{\otimes}}}(\underline{X}_{1}^{n})\right\|$
$\displaystyle\leq$
$\displaystyle\left\|\overline{\psi}^{[n+1]}_{\acute{\otimes}}\right\|\sum\limits_{i=1}^{n+1}\left(\left\|A^{(m)}_{i}-A_{i}\right\|\prod\limits_{j=1}^{n}\left\|X_{j}\right\|\right).$
(57)

By raising the power $r$ and taking the expectation at the both sides of the
inequality provided by Eq. (57), we have proved this theorem by conditions
given by Eq. (55) and the following inequality:

$\displaystyle\left(\sum\limits_{i=1}^{n+1}a_{i}\right)^{r}\leq(n+1)^{r-1}\left(\sum\limits_{i=1}^{n+1}a_{i}^{r}\right)\leavevmode\nobreak\
\leavevmode\nobreak\ \mbox{given $a_{i}\geq 0$}.$ (58)

$\hfill\Box$

##  5 Random MOIs Applications

In this section, we will apply random MOIs to obtain several tail bounds
related to several applications of MOIs. We will focus on functions
constructible by the _limit of integral projective tensor product,
$\acute{\otimes}$_. Same arguments will also be applied to functions
constructible by the _limit of projective tensor product, $\hat{\otimes}$_.

###  5.1 Tail Bound for Higher Random Operator Derivative

In this section, we will derive tail bounds for higher random operato
derivatives. We begin with a lemma to express the first derivative of an
operator-valued function by double operator integrals.

###### Lemma 4

Let $X(t)$ be self-adjoint operators indexed by $t$ with bounded norm, i.e.,
$\left\|X(t)\right\|<\infty$, and$\overline{\psi}_{\acute{\otimes}}$ be
differentiable on $\mathbb{R}$. Then, we have

$\displaystyle\frac{d\overline{\psi}_{\acute{\otimes}}(X(t))}{dt}$
$\displaystyle=$ $\displaystyle
T^{X(t),X(t)}_{\overline{\psi}_{\acute{\otimes}}^{[1]}}\left(\frac{dX(t)}{dt}\right).$
(59)

Proof: Suppose the function $\overline{\psi}_{\acute{\otimes}}(x)$ has the
polynomial form as $\overline{\psi}_{\acute{\otimes}}(x)=x^{m}$, by the chain
rule of dervative, we have

$\displaystyle\frac{d\overline{\psi}_{\acute{\otimes}}(X(t))}{dt}$
$\displaystyle=$
$\displaystyle\sum\limits_{i=0}^{m-1}X^{i}(t)\frac{dX(t)}{dt}X^{m-i-1}(t).$
(60)

Since the operator $X(t)$ can be represented by its spectrum as

$\displaystyle X(t)=\int\limits_{\Lambda_{t}}\lambda dA_{t}(\lambda),$ (61)

where $(\Lambda_{t},A_{t})$ is the spectral measure at the instant $t$;by the
spectral theorem, we can express Eq. (60) as

$\displaystyle\frac{d\overline{\psi}_{\acute{\otimes}}(X(t))}{dt}$
$\displaystyle=$
$\displaystyle\sum\limits_{i=0}^{m-1}\int\limits_{\Lambda_{t}}\lambda_{1}^{i}dA_{t}(\lambda_{1})\frac{dX(t)}{dt}\int\limits_{\Lambda_{t}}\lambda_{2}^{m-i-1}dA_{t}(\lambda_{2})$
(62) $\displaystyle=$
$\displaystyle\int\limits_{\Lambda_{t}}\int\limits_{\Lambda_{t}}\left(\sum\limits_{i=0}^{m-1}\lambda_{1}^{i}\lambda_{2}^{m-i-1}\right)dA_{t}(\lambda_{1})\frac{dX(t)}{dt}dA_{t}(\lambda_{2})$
$\displaystyle=$
$\displaystyle\int\limits_{\Lambda_{t}}\int\limits_{\Lambda_{t}}\underbracket{\frac{\lambda_{1}^{m}-\lambda_{2}^{m}}{\lambda_{1}-\lambda_{2}}}_{(x^{m})^{[1]}}dA_{t}(\lambda_{1})\frac{dX(t)}{dt}dA_{t}(\lambda_{2}).$

From MOIs defined by Eq. (2) as $m=2$ (DOIs), we have Eq. (59) for
$\overline{\psi}_{\acute{\otimes}}(x)=x^{m}$. By linearity, this lemma is also
true when $\overline{\psi}_{\acute{\otimes}}(x)$ is a polynomial function.

Suppose we can approximate $\overline{\psi}_{\acute{\otimes}}$ by a sequence
of polynomials $p_{n}(x)$, i.e.,
$\left\|\overline{\psi}_{\acute{\otimes}}-p_{n}\right\|\rightarrow 0$ as
$n\rightarrow\infty$. Then, we have

$\displaystyle\left\|T^{X(t),X(t)}_{\overline{\psi}_{\acute{\otimes}}^{[1]}}\left(\frac{dX(t)}{dt}\right)-T^{X(t),X(t)}_{p_{n}^{[1]}}\left(\frac{dX(t)}{dt}\right)\right\|\leq
Const\left\|\overline{\psi}_{\acute{\otimes}}-p_{n}\right\|\left\|X(t)\right\|.$
(63)

This lemma is proved due to we can approximate
$\overline{\psi}_{\acute{\otimes}}$ by polynomials. $\hfill\Box$

From Lemma 4, we are ready to have the following theorem about the tail bound
for $\left\|\frac{d\overline{\psi}_{\acute{\otimes}}(X(t))}{dt}\right\|$.

###### Theorem 7

Let $X(t)$ be self-adjoint operators indexed by $t$ with bounded norm, i.e.,
$\left\|X(t)\right\|<\infty$, and$\overline{\psi}_{\acute{\otimes}}$ be
differentiable on $\mathbb{R}$. We also assume that
$\left\|\frac{dX(t)}{dt}\right\|<\Upsilon_{X}$. Then,

$\displaystyle\mathrm{Pr}\left(\left\|\frac{d\overline{\psi}_{\acute{\otimes}}(X(t))}{dt}\right\|>\theta\right)$
$\displaystyle\leq$
$\displaystyle\frac{\Upsilon_{X}}{\theta}\mathbb{E}\left[\left\|\overline{\psi}^{[1]}_{\acute{\otimes}}\right\|\right].$
(64)

Proof: Because we have

$\displaystyle\mathrm{Pr}\left(\left\|\frac{d\overline{\psi}_{\acute{\otimes}}(X(t))}{dt}\right\|>\theta\right)$
$\displaystyle=$
$\displaystyle\mathrm{Pr}\left(\left\|T^{X(t),X(t)}_{\overline{\psi}_{\acute{\otimes}}^{[1]}}\left(\frac{dX(t)}{dt}\right)\right\|>\theta\right)$
(65) $\displaystyle\leq_{1}$
$\displaystyle\mathrm{Pr}\left(\left\|\overline{\psi}^{[1]}_{\acute{\otimes}}\right\|\left\|\frac{dX(t)}{dt}\right\|>\theta\right)$
$\displaystyle=$
$\displaystyle\mathrm{Pr}\left(\left\|\overline{\psi}^{[1]}_{\acute{\otimes}}\right\|>\frac{\theta}{\left\|\frac{dX(t)}{dt}\right\|}\right)$
$\displaystyle\leq$
$\displaystyle\mathrm{Pr}\left(\left\|\overline{\psi}^{[1]}_{\acute{\otimes}}\right\|>\frac{\theta}{\Upsilon_{X}}\right),$

where $\leq_{1}$ comes from Lemma 1. This lemma follows by Markov inequality.
$\hfill\Box$

Following lemma is about the expression of the $k$-th operator-valued function
by MOIs.

###### Lemma 5

Let $A,B$ be two operators, and $\overline{\psi}_{\acute{\otimes}}$ be
$k\in\mathbb{N}$ order differentiable function, then we have

$\displaystyle\frac{d^{k}\overline{\psi}_{\acute{\otimes}}(A+tB)}{dt^{k}}\Bigg{|}_{t=0}$
$\displaystyle=$ $\displaystyle
k!T^{\overbrace{A+tB,\cdots,A+tB}^{k+1}}_{\overline{\psi}^{[k]}_{\acute{\otimes}}}\left(\underbrace{B,\cdots,B}_{k}\right).$
(66)

Proof: We will prove this lemma by induction. The base case is proved by Lemma
4 for $k=1$. Suppose we have Eq. (66) for the $k-1$-th derivative, then

$\displaystyle\frac{d^{k}\overline{\psi}_{\acute{\otimes}}(A+tB)}{dt^{k}}\Bigg{|}_{t=0}$
$\displaystyle=$ $\displaystyle\lim\limits_{t\rightarrow
0}\frac{(k-1)!}{t}\left(T^{\overbrace{A+tB,\cdots,A+tB}^{k}}_{\overline{\psi}^{[k-1]}_{\acute{\otimes}}}\left(\underbrace{B,\cdots,B}_{k-1}\right)-\right.$
(67)
$\displaystyle\left.T^{\overbrace{A,\cdots,A}^{k}}_{\overline{\psi}^{[k-1]}_{\acute{\otimes}}}\left(\underbrace{B,\cdots,B}_{k-1}\right)\right)$
$\displaystyle=$ $\displaystyle\lim\limits_{t\rightarrow
0}\frac{(k-1)!}{t}\sum\limits_{i=0}^{k-1}\left(T^{\overbrace{A+tB,\cdots,A+tB}^{k-i},\overbrace{A,\cdots,A}^{i}}_{\overline{\psi}^{[k-1]}_{\acute{\otimes}}}\left(\underbrace{B,\cdots,B}_{k-1}\right)-\right.$
$\displaystyle\left.T^{\overbrace{A+tB,\cdots,A+tB}^{k-i-1},\overbrace{A,\cdots,A}^{i+1}}_{\overline{\psi}^{[k-1]}_{\acute{\otimes}}}\left(\underbrace{B,\cdots,B}_{k-1}\right)\right)$
$\displaystyle=_{1}$ $\displaystyle
k!T^{\overbrace{A+tB,\cdots,A+tB}^{k-i},\overbrace{A,\cdots,A}^{i+1}}_{\overline{\psi}^{[k]}_{\acute{\otimes}}}\left(\underbrace{B,\cdots,B}_{k}\right),$

where we apply perturbation formula, Lemma 2, in $=_{1}$.

From continuity property given by Lemma 3, we have

$\displaystyle\lim\limits_{t\rightarrow
0}T^{\overbrace{A+tB,\cdots,A+tB}^{k-i},\overbrace{A,\cdots,A}^{i+1}}_{\overline{\psi}^{[k]}_{\acute{\otimes}}}\left(\underbrace{B,\cdots,B}_{k}\right)=T^{\overbrace{A,\cdots,A}^{k+1}}_{\overline{\psi}^{[k]}_{\acute{\otimes}}}\left(\underbrace{B,\cdots,B}_{k}\right).$
(68)

This lemma is proved from Eq. (67) and Eq. (68). $\hfill\Box$

From Lemma 5, we are ready to have the following theorem about the tail bound
for
$\left\|\frac{d^{k}\overline{\psi}_{\acute{\otimes}}(A+tB)}{dt^{k}}\Bigg{|}_{t=0}\right\|$.

###### Theorem 8

Let $A$ be random operators, $B$ be a determinstic operator, and
$\overline{\psi}_{\acute{\otimes}}$ be $k\in\mathbb{N}$ order differentiable
function, then we have

$\displaystyle\mathrm{Pr}\left(\left\|\frac{d^{k}\overline{\psi}_{\acute{\otimes}}(A+tB)}{dt^{k}}\Bigg{|}_{t=0}\right\|>\theta\right)$
$\displaystyle\leq$
$\displaystyle\frac{k!\left\|B\right\|^{k}}{\theta}\mathbb{E}\left[\left\|\overline{\psi}^{[k]}_{\acute{\otimes}}\right\|\right].$
(69)

Proof: Because we have

$\displaystyle\mathrm{Pr}\left(\left\|\frac{d^{k}\overline{\psi}_{\acute{\otimes}}(A+tB)}{dt^{k}}\Bigg{|}_{t=0}\right\|>\theta\right)$
$\displaystyle=_{1}$
$\displaystyle\mathrm{Pr}\left(\left\|k!T^{\overbrace{A+tB,\cdots,A+tB}^{k+1}}_{\overline{\psi}^{[k]}_{\acute{\otimes}}}\left(\underbrace{B,\cdots,B}_{k}\right)\right\|>\theta\right)$
(70) $\displaystyle\leq_{2}$
$\displaystyle\mathrm{Pr}\left(k!\left\|\overline{\psi}^{[k]}_{\acute{\otimes}}\right\|\left\|B\right\|^{k}>\theta\right)$
$\displaystyle\leq_{3}$
$\displaystyle\frac{k!\left\|B\right\|^{k}}{\theta}\mathbb{E}\left[\left\|\overline{\psi}^{[k]}_{\acute{\otimes}}\right\|\right],$

where $=_{1}$ comes from Lemma 5, $\leq_{2}$ comes from Lemma 1, and
$\leq_{3}$ comes from Markov inequality again. $\hfill\Box$

###  5.2 Tail Bound for Higher Random Operator Difference

In this section, we will extend the operator first order difference, i.e.,
$\overline{\psi}_{\acute{\otimes}}(A)-\overline{\psi}_{\acute{\otimes}}(A+B)$,
to higher order difference. The higher opertor difference is defined as

$\displaystyle\Delta^{k}_{B}\overline{\psi}_{\acute{\otimes}}(A)$
$\displaystyle=$ $\displaystyle\sum\limits_{i=0}^{k}(-1)^{k-i}{k\choose
i}\overline{\psi}_{\acute{\otimes}}(A+iB),$ (71)

where $A,B$ are self-adjoint operators on Hilbert space. We will have the
following theorem about the tail bound for the higher opertor difference given
by Eq. (71).

###### Theorem 9

Let $A$ be a self-adjoint random operator, $B$ be a deterministic self-adjoint
operator and the eigenvalue for the operator $A+iB$ is represented by
$\lambda_{i+1}$ for $i=0,1,\cdots,k$. We assume that
$\max(\lambda_{j+1}-\lambda_{j})<\kappa$ for $j=1,2,\cdots,k$. Then, we have

$\displaystyle\mathrm{Pr}\left(\left\|\Delta^{k}_{B}\overline{\psi}_{\acute{\otimes}}(A)\right\|>\theta\right)$
$\displaystyle\leq$
$\displaystyle\frac{k\kappa\left\|B\right\|^{k}}{\theta}\mathbb{E}(\overline{\psi}_{\acute{\otimes}}).$
(72)

Proof: From the higher opertor difference provided by Eq. (71) and the divide
difference definition, we have

$\displaystyle\Delta^{k}_{B}\overline{\psi}_{\acute{\otimes}}(A)$
$\displaystyle=$
$\displaystyle(-1)^{k}\Delta^{k-1}_{B}\overline{\psi}_{\acute{\otimes}}(A)+(-1)^{k-1}\Delta^{k-1}_{B}\overline{\psi}_{\acute{\otimes}}(A+B)$
(73) $\displaystyle=$
$\displaystyle\sum\limits_{j=1}^{k}\left(\underbrace{\int\cdots\int}_{k+1}(\lambda_{j+1}-\lambda_{j})\overline{\psi}^{[k]}_{\acute{\otimes}}(\lambda_{1},\cdots,\lambda_{k+1})dP_{A}(\lambda_{1})BdP_{A+B}(\lambda_{2})B\right.$
$\displaystyle\left.\cdots BdP_{A+kB}(\lambda_{k+1})\right),$

where $P_{A}(\lambda_{1}),\cdots,dP_{A+kB}(\lambda_{k+1})$ are spectral
measures.

Then, we have

$\displaystyle\mathrm{Pr}\left(\left\|\Delta^{k}_{B}\overline{\psi}_{\acute{\otimes}}(A)\right\|>\theta\right)$
(74) $\displaystyle=_{1}$
$\displaystyle\mathrm{Pr}\left(\left\|\sum\limits_{j=1}^{k}\left(\underbrace{\int\cdots\int}_{k+1}(\lambda_{j+1}-\lambda_{j})\overline{\psi}^{[k]}_{\acute{\otimes}}(\lambda_{1},\cdots,\lambda_{k+1})dP_{A}(\lambda_{1})B\cdots
BdP_{A+kB}(\lambda_{k+1})\right)\right\|>\theta\right)$
$\displaystyle\leq_{2}$
$\displaystyle\mathrm{Pr}\left(\sum\limits_{j=1}^{k}\max(\lambda_{j+1}-\lambda_{j})\left\|\overline{\psi}^{[k]}_{\acute{\otimes}}\right\|\left\|B\right\|^{k}>\theta\right)$
$\displaystyle\leq_{3}$
$\displaystyle\mathrm{Pr}\left(k\kappa\left\|\overline{\psi}^{[k]}_{\acute{\otimes}}\right\|\left\|B\right\|^{k}>\theta\right),$

where $=_{1}$ comes from Eq. (73), $=_{2}$ comes from Lemma 1, and $\leq_{3}$
comes from assumptions about $\max(\lambda_{j+1}-\lambda_{j})$. The proof of
this theorem is complete from Markov inequality. $\hfill\Box$

###  5.3 Tail Bound for Taylor Remainder of Random Operator-valued Functions

In this section, we will derive tail bounds for Taylor remainder of random
operator-valued functions. We will consider two types of random operators:
self-adjoint and unitary. Let us begin with _multi-index_ notation
definitions.

Multi-indices are represented by vector symbols as $\bm{a}$ or $\bm{b}$:

$\displaystyle\bm{a}=(a_{1},a_{2},\cdots,a_{n}),\leavevmode\nobreak\
\bm{b}=(b_{1},b_{2},\cdots,b_{n}),$ (75)

where $a_{i},b_{i}\in\\{0,1,2,\cdots\\}$. If $\bm{a}$ is a multi-index, we
have the following notation defnitions:

$\displaystyle\left|\bm{a}\right|$ $\displaystyle=$ $\displaystyle
a_{1}+a_{2}+\cdots+a_{n},$ $\displaystyle\bm{a}!$ $\displaystyle=$
$\displaystyle a_{1}!a_{2}!\cdots a_{n}!,$ $\displaystyle\bm{X}^{\bm{a}}$
$\displaystyle=$ $\displaystyle X_{1}^{a_{1}}X_{2}^{a_{2}}\cdots
X_{n}^{a_{n}}\leavevmode\nobreak\ \mbox{where
$\bm{x}=(X_{1},X_{2},\cdots,X_{n})$}.$ (76)

####  5.3.1 Random Self-Adjoint Operators

All operators discussed in this section are assumed self-adjoint. Given a
multi-variate operator-valued function $f(X_{1},X_{2},\cdots,X_{n})$ and
perturbed operators $(H_{1},H_{2},\cdots,H_{n})$, we define the partial
derivative for $\partial^{\bm{a}}f(\bm{X}+\bm{t}\circ\bm{H})$ as

$\displaystyle\partial^{\bm{a}}f(\bm{X}+\bm{t}\circ\bm{H})=\frac{\partial^{\left|\bm{a}\right|}f(\bm{X}+\bm{t}\circ\bm{H})}{\partial
t_{1}^{a_{1}}\partial t_{2}^{a_{2}}\cdots\partial t_{n}^{a_{n}}}\,$ (77)

where $\bm{t}$ is the vector of real variables $(t_{1},t_{2},\cdots,t_{n})$
and $\circ$ is the Hadamard product. Then, we define the Taylor remainder for
$f(X_{1},X_{2},\cdots,X_{n})$ with perturbed operators
$(H_{1},H_{2},\cdots,H_{n})$ as

$\displaystyle R_{k,f,\bm{X}}(\bm{H})$ $\displaystyle=$ $\displaystyle
f(\bm{X}+\bm{H})-\sum\limits_{\left|\bm{a}\right|<k}\frac{\partial^{\bm{a}}f(\bm{X}+\bm{t}\circ\bm{H})}{\bm{a}!}\bigg{|}_{\bm{t}=\bm{0}},$
(78)

where $\bm{t}$ is the vector of real variables $(t_{1},t_{2},\cdots,t_{n})$
and $\circ$ is the Hadamard product.

If the function $f(X_{1},X_{2},\cdots,X_{n})$ can be decomposed as the
summation of function constructible by the limit of integral projective tensor
product $\acute{\otimes}$, i.e.,

$\displaystyle f(X_{1},X_{2},\cdots,X_{n})$ $\displaystyle=$
$\displaystyle\overline{\phi}_{\acute{\otimes},1}(X_{1})+\overline{\phi}_{\acute{\otimes},2}(X_{2})+\cdots+\overline{\phi}_{\acute{\otimes},n}(X_{n}),$
(79)

the expression given by Eq. (78) can be further expressed as

$\displaystyle R_{k,f,\bm{X}}(\bm{H})$ $\displaystyle=$
$\displaystyle\sum\limits_{j=1}^{n}\left(\overline{\phi}_{\acute{\otimes}}(X_{j}+H_{j})-\sum\limits_{\ell=0}^{k-1}\frac{1}{\ell!}\frac{d^{\ell}}{dt^{\ell}_{j}}\overline{\phi}_{\acute{\otimes},j}(X_{j}+t_{j}H_{j})\bigg{|}_{t_{j}=0}\right).$
(80)

Following lemma is given to express $R_{k,f,\bm{X}}(\bm{H})$ by MOIs.

###### Lemma 6

Let the function $f:\mathbb{R}^{n}\rightarrow\mathbb{R}$ can be decomposed as
$f(\bm{X})=\sum\limits_{j=1}^{n}\overline{\phi}_{\acute{\otimes},j}(X_{j})$,
we have

$\displaystyle R_{k,f,\bm{X}}(\bm{H})$ $\displaystyle=$
$\displaystyle\sum\limits_{j=1}^{n}T^{X_{j}+H_{j},\cdots,X_{j}}_{\overline{\phi}^{[k]}_{\acute{\otimes},j}}(\underbrace{H_{j},\cdots,H_{j}}_{k}).$
(81)

Proof: From Lemma 5, we have

$\displaystyle R_{k,\overline{\phi}_{\acute{\otimes},j},X_{j}}(H_{j})$
$\displaystyle=$ $\displaystyle
T^{X_{j}+H_{j},X_{j}}_{\overline{\phi}^{[1]}_{\acute{\otimes},j}}(H_{j})-\sum\limits_{\ell=1}^{k-1}T^{X_{j},\cdots,X_{j}}_{\overline{\phi}^{[k]}_{\acute{\otimes},j}}(\underbrace{H_{j},\cdots,H_{j}}_{k}).$
(82)

By applying Lemma 2 to
$T^{X_{j}+H_{j},X_{j}}_{\overline{\phi}^{[1]}_{\acute{\otimes},j}}(H_{j})$ at
the R.H.S. of Eq. (82), we have

$\displaystyle R_{k,\overline{\phi}_{\acute{\otimes},j},X_{j}}(H_{j})$
$\displaystyle=$ $\displaystyle
T^{X_{j}+H_{j},X_{j},X_{j}}_{\overline{\phi}^{[2]}_{\acute{\otimes},j}}(H_{j})-\sum\limits_{\ell=2}^{k-1}T^{X_{j},\cdots,X_{j}}_{\overline{\phi}^{[k]}_{\acute{\otimes},j}}(\underbrace{H_{j},\cdots,H_{j}}_{k}).$
(83)

This lemma is proved by repeating this procedure $k-1$ times and the
decomposition relation given by Eq. (80). $\hfill\Box$

Following theorem is about the tail bound for $R_{k,f,\bm{X}}(\bm{H})$.

###### Theorem 10

For $j=1,2,\cdots,n$, let $X_{j}$ be a self-adjoint random operator, $H_{j}$
be a deterministic self-adjoint operator. The function
$f:\mathbb{R}^{n}\rightarrow\mathbb{R}$ is assumed to be decomposed as
$f(\bm{X})=\sum\limits_{j=1}^{n}\overline{\phi}_{\acute{\otimes},j}(X_{j})$.
Then, we have

$\displaystyle\mathrm{Pr}\left(\left\|R_{k,f,\bm{X}}(\bm{H})\right\|>\theta\right)$
$\displaystyle\leq$
$\displaystyle\sum\limits_{j=1}^{n}\frac{n\left\|H_{j}\right\|^{k}}{\theta}\mathbb{E}(\overline{\psi}^{[k]}_{\acute{\otimes},j}).$
(84)

Proof: Because we have

$\displaystyle\mathrm{Pr}\left(\left\|R_{k,f,\bm{X}}(\bm{H})\right\|>\theta\right)$
$\displaystyle=_{1}$
$\displaystyle\mathrm{Pr}\left(\left\|\sum\limits_{j=1}^{n}T^{X_{j}+H_{j},\cdots,X_{j}}_{\overline{\phi}^{[k]}_{\acute{\otimes},j}}(\underbrace{H_{j},\cdots,H_{j}}_{k})\right\|>\theta\right)$
(85) $\displaystyle\leq_{2}$
$\displaystyle\mathrm{Pr}\left(\sum\limits_{j=1}^{n}\left\|T^{X_{j}+H_{j},\cdots,X_{j}}_{\overline{\phi}^{[k]}_{\acute{\otimes},j}}(\underbrace{H_{j},\cdots,H_{j}}_{k})\right\|>\theta\right)$
$\displaystyle\leq$
$\displaystyle\sum\limits_{j=1}^{n}\mathrm{Pr}\left(\left\|T^{X_{j}+H_{j},\cdots,X_{j}}_{\overline{\phi}^{[k]}_{\acute{\otimes},j}}(\underbrace{H_{j},\cdots,H_{j}}_{k})\right\|>\frac{\theta}{n}\right)$
$\displaystyle\leq_{3}$
$\displaystyle\sum\limits_{j=1}^{n}\frac{n\left\|H_{j}\right\|^{k}}{\theta}\mathbb{E}\left[\left\|\overline{\psi}^{[k]}_{\acute{\otimes},j}\right\|\right],$

where $=_{1}$ comes from Lemma 6, $\leq_{2}$ is due to norm triangle
inequality, and $\leq_{3}$ comes from Lemma 1 and Markov inequality again.
$\hfill\Box$

####  5.3.2 Random Unitary Operators

Given a multi-variate operator-valued function $f(X_{1},X_{2},\cdots,X_{n})$
and perturbed operators $(H_{1},H_{2},\cdots,H_{n})$, which are assumed to be
bounded self-adjoint operators, we use $e^{\iota\bm{t}\circ\bm{H}}$ to
represent the vector
$(e^{\iota t_{1}H_{1}},e^{\iota t_{2}H_{2}},\cdots,e^{\iota t_{n}H_{n}})$,
where $\iota\stackrel{{\scriptstyle\mbox{\tiny def}}}{{=}}\sqrt{-1}$. We
define the partial derivative for
$\partial^{\bm{a}}f(e^{\iota\bm{t}\circ\bm{H}}\circ\bm{X})$ as

$\displaystyle\partial^{\bm{a}}f(e^{\iota\bm{t}\circ\bm{H}}\circ\bm{X})=\frac{\partial^{\left|\bm{a}\right|}f(e^{\iota\bm{t}\circ\bm{H}}\circ\bm{X})}{\partial
t_{1}^{a_{1}}\partial t_{2}^{a_{2}}\cdots\partial t_{n}^{a_{n}}}.$ (86)

We first define the Taylor remainder for $f(X_{1},X_{2},\cdots,X_{n})$ with
perturbed operators $(H_{1},H_{2},\cdots,H_{n})$ as

$\displaystyle Q_{k,f,\bm{X}}(\bm{H})$ $\displaystyle=$ $\displaystyle
f(e^{\iota\bm{H}}\circ\bm{X})-\sum\limits_{\left|\bm{a}\right|<k}\frac{\partial^{\bm{a}}f(e^{\iota\bm{t}\circ\bm{H}}\circ\bm{X})}{\bm{a}!}\bigg{|}_{\bm{t}=\bm{0}}.$
(87)

If the function $f$ can also be decomposed as Eq. (79), we can rewrite Eq.
(87) as

$\displaystyle Q_{k,f,\bm{X}}(\bm{H})$ $\displaystyle=$
$\displaystyle\sum\limits_{j=1}^{n}\left(\overline{\phi}_{\acute{\otimes}}(e^{\iota
H_{j}}X_{j})-\sum\limits_{\ell=0}^{k-1}\frac{1}{\ell!}\frac{d^{\ell}}{dt^{\ell}_{j}}\overline{\phi}_{\acute{\otimes},j}(e^{\iota
t_{j}H_{j}}X_{j})\bigg{|}_{t_{j}=0}\right).$ (88)

Following lemma is given to express $R_{k,f,\bm{X}}(\bm{H})$ by MOIs.

###### Lemma 7

Let the function $f:\mathbb{R}^{n}\rightarrow\mathbb{R}$ can be decomposed as
$f(\bm{X})=\sum\limits_{j=1}^{n}\overline{\phi}_{\acute{\otimes},j}(X_{j})$,
we have

$\displaystyle Q_{k,f,\bm{X}}(\bm{H})=$ (89)
$\displaystyle\sum\limits_{j=1}^{n}\left(\sum\limits_{\ell=1}^{k}\sum\limits_{\shortstack{
$i_{1},\cdots,i_{\ell}\geq 1$ \\\ $\left|\bm{i}\right|=k$}}T^{e^{\iota
H_{j}}X_{j},X_{j},\cdots,X_{j}}_{\overline{\phi}^{[\ell]}_{\acute{\otimes},j}}\left(\underbrace{\sum\limits_{m=i_{1}}^{\infty}\frac{(\iota
H_{j})^{m}}{m!}X_{j},\frac{(\iota
H_{j})^{i_{2}}}{i_{2}!}X_{j},\cdots,\frac{(\iota
H_{j})^{i_{\ell}}}{i_{\ell}!}X_{j}}_{\ell}\right)\right).$

Proof: We will prove this lemma by induction. For basic case with $k=1$ and
any $j$ between $1$ and $n$, we have

$\displaystyle Q_{1,\overline{\phi}_{\acute{\otimes},j},X_{j}}(H_{j})$
$\displaystyle=$ $\displaystyle\overline{\phi}_{\acute{\otimes},j}(e^{\iota
H_{j}}X_{j})-\overline{\phi}_{\acute{\otimes},j}(X_{j})$ (90) $\displaystyle=$
$\displaystyle T^{e^{\iota
H_{j}}X_{j},X_{j}}_{\overline{\phi}^{[1]}_{\acute{\otimes},j}}(e^{\iota
H_{j}}X_{j}-X_{j})$ $\displaystyle=$ $\displaystyle T^{e^{\iota
H_{j}}X_{j},X_{j}}_{\overline{\phi}^{[1]}_{\acute{\otimes},j}}\left(\sum\limits_{m=1}^{\infty}\frac{(\iota
H_{j})^{m}}{m!}X_{j}\right).$

Therefore, by summing over $j$ with respect to Eq. (90), this lemma is valid
for the basic case.

Suppose that Eq. (89) is valid for $k=q$ and any $j$ between $1$ and $n$, we
have

$\displaystyle Q_{q+1,\overline{\phi}_{\acute{\otimes},j},X_{j}}(H_{j})$
$\displaystyle=$
$\displaystyle\sum\limits_{\ell=1}^{q}\sum\limits_{\shortstack{
$i_{1},\cdots,i_{\ell}\geq 1$ \\\ $\left|\bm{i}\right|=q$}}T^{e^{\iota
H_{j}}X_{j},X_{j},\cdots,X_{j}}_{\overline{\phi}^{[\ell]}_{\acute{\otimes},j}}\left(\underbrace{\sum\limits_{m=i_{1}}^{\infty}\frac{(\iota
H_{j})^{m}}{m!}X_{j},\frac{(\iota
H_{j})^{i_{2}}}{i_{2}!}X_{j},\cdots,\frac{(\iota
H_{j})^{i_{\ell}}}{i_{\ell}!}X_{j}}_{\ell}\right)$
$\displaystyle-\frac{1}{q!}\frac{d^{q}}{dt^{q}_{j}}\overline{\phi}_{\acute{\otimes},j}(e^{\iota
H_{j}}X_{j})\bigg{|}_{t_{j}=0}$ $\displaystyle=$
$\displaystyle\sum\limits_{\ell=1}^{q}\sum\limits_{\shortstack{
$i_{1},\cdots,i_{\ell}\geq 1$ \\\ $\left|\bm{i}\right|=q$}}T^{e^{\iota
H_{j}}X_{j},X_{j},\cdots,X_{j}}_{\overline{\phi}^{[\ell]}_{\acute{\otimes},j}}\left(\underbrace{\sum\limits_{m=i_{1}}^{\infty}\frac{(\iota
H_{j})^{m}}{m!}X_{j},\frac{(\iota
H_{j})^{i_{2}}}{i_{2}!}X_{j},\cdots,\frac{(\iota
H_{j})^{i_{\ell}}}{i_{\ell}!}X_{j}}_{\ell}\right)$
$\displaystyle-\sum\limits_{\ell=1}^{q}\sum\limits_{\shortstack{
$i_{1},\cdots,i_{\ell}\geq 1$ \\\
$\left|\bm{i}\right|=q$}}T^{X_{j},X_{j},\cdots,X_{j}}_{\overline{\phi}^{[\ell]}_{\acute{\otimes},j}}\left(\underbrace{\frac{(\iota
H_{j})^{i_{1}}}{i_{1}!}X_{j},\frac{(\iota
H_{j})^{i_{2}}}{i_{2}!}X_{j},\cdots,\frac{(\iota
H_{j})^{i_{\ell}}}{i_{\ell}!}X_{j}}_{\ell}\right)$ $\displaystyle=$
$\displaystyle\sum\limits_{\ell=1}^{q}\sum\limits_{\shortstack{
$i_{1},\cdots,i_{\ell}\geq 1$ \\\ $\left|\bm{i}\right|=q$}}\Bigg{[}T^{e^{\iota
H_{j}}X_{j},X_{j},\cdots,X_{j}}_{\overline{\phi}^{[\ell]}_{\acute{\otimes},j}}\left(\underbrace{\sum\limits_{m=i_{1}}^{\infty}\frac{(\iota
H_{j})^{m}}{m!}X_{j},\frac{(\iota
H_{j})^{i_{2}}}{i_{2}!}X_{j},\cdots,\frac{(\iota
H_{j})^{i_{\ell}}}{i_{\ell}!}X_{j}}_{\ell}\right)$ $\displaystyle-T^{e^{\iota
H_{j}}X_{j},X_{j},\cdots,X_{j}}_{\overline{\phi}^{[\ell]}_{\acute{\otimes},j}}\left(\underbrace{\frac{(\iota
H_{j})^{i_{1}}}{i_{1}!}X_{j},\frac{(\iota
H_{j})^{i_{2}}}{i_{2}!}X_{j},\cdots,\frac{(\iota
H_{j})^{i_{\ell}}}{i_{\ell}!}X_{j}}_{\ell}\right)$ $\displaystyle+T^{e^{\iota
H_{j}}X_{j},X_{j},\cdots,X_{j}}_{\overline{\phi}^{[\ell]}_{\acute{\otimes},j}}\left(\underbrace{\frac{(\iota
H_{j})^{i_{1}}}{i_{1}!}X_{j},\frac{(\iota
H_{j})^{i_{2}}}{i_{2}!}X_{j},\cdots,\frac{(\iota
H_{j})^{i_{\ell}}}{i_{\ell}!}X_{j}}_{\ell}\right)$
$\displaystyle-T^{X_{j},X_{j},\cdots,X_{j}}_{\overline{\phi}^{[\ell]}_{\acute{\otimes},j}}\left(\underbrace{\frac{(\iota
H_{j})^{i_{1}}}{i_{1}!}X_{j},\frac{(\iota
H_{j})^{i_{2}}}{i_{2}!}X_{j},\cdots,\frac{(\iota
H_{j})^{i_{\ell}}}{i_{\ell}!}X_{j}}_{\ell}\right)\Bigg{]}$
$\displaystyle=_{1}$
$\displaystyle\overbrace{\sum\limits_{\ell=1}^{q}\sum\limits_{\shortstack{
$i_{1},\cdots,i_{\ell}\geq 1$ \\\ $\left|\bm{i}\right|=q$}}T^{e^{\iota
H_{j}}X_{j},X_{j},\cdots,X_{j}}_{\overline{\phi}^{[\ell]}_{\acute{\otimes},j}}\left(\underbrace{\sum\limits_{m=i_{1}+1}^{\infty}\frac{(\iota
H_{j})^{m}}{m!}X_{j},\frac{(\iota
H_{j})^{i_{2}}}{i_{2}!}X_{j},\cdots,\frac{(\iota
H_{j})^{i_{\ell}}}{i_{\ell}!}X_{j}}_{\ell}\right)}^{\mbox{Sum}_{1}}$
$\displaystyle+\overbrace{\sum\limits_{\ell=1}^{q}\sum\limits_{\shortstack{
$i_{1},\cdots,i_{\ell}\geq 1$ \\\ $\left|\bm{i}\right|=q$}}T^{e^{\iota
H_{j}}X_{j},X_{j},\cdots,X_{j}}_{\overline{\phi}^{[\ell+1]}_{\acute{\otimes},j}}\left(\underbrace{\sum\limits_{m=1}^{\infty}\frac{(\iota
H_{j})^{m}}{m!}X_{j},\frac{(\iota H_{j})^{i_{1}}}{i_{1}!}X_{j},\frac{(\iota
H_{j})^{i_{2}}}{i_{2}!}X_{j},\cdots,\frac{(\iota
H_{j})^{i_{\ell}}}{i_{\ell}!}X_{j}}_{\ell+1}\right)}^{\mbox{Sum}_{2}}$

where $=_{1}$ comes from Lemma 2.

By changing the summation index range as $\alpha_{1}=i_{1}+1$ and
$\alpha_{p}=i_{p}$ for $2\leq p\leq\ell$, we can express the summation of
$\mbox{Sum}_{1}$ as

$\displaystyle\mbox{Sum}_{1}=$ (92)
$\displaystyle\sum\limits_{\ell=1}^{q}\sum\limits_{\shortstack{
$\alpha_{1}\geq 2,\alpha_{2},\cdots,\alpha_{\ell}\geq 1$ \\\
$\left|\bm{\alpha}\right|=q$}}T^{e^{\iota
H_{j}}X_{j},X_{j},\cdots,X_{j}}_{\overline{\phi}^{[\ell]}_{\acute{\otimes},j}}\left(\underbrace{\sum\limits_{m=\alpha_{1}}^{\infty}\frac{(\iota
H_{j})^{m}}{m!}X_{j},\frac{(\iota
H_{j})^{\alpha_{2}}}{\alpha_{2}!}X_{j},\cdots,\frac{(\iota
H_{j})^{\alpha_{\ell}}}{i_{\ell}!}X_{j}}_{\ell}\right).$

Similarly, by changing the summation index range from $\ell=1,2,\cdots,q$ to
$\ell=2,\cdots,q+1$, $\alpha_{1}=1$ and $\alpha_{p}=i_{p-1}$ for $2\leq
p\leq\ell$, we can express the summation of $\mbox{Sum}_{2}$ as

$\displaystyle\mbox{Sum}_{2}=$ (93)
$\displaystyle\sum\limits_{\ell=2}^{q+1}\sum\limits_{\shortstack{
$\alpha_{1}=1,\alpha_{2},\cdots,\alpha_{\ell}\geq 1$ \\\
$\left|\bm{\alpha}\right|=q+1$}}T^{e^{\iota
H_{j}}X_{j},X_{j},\cdots,X_{j}}_{\overline{\phi}^{[\ell]}_{\acute{\otimes},j}}\left(\underbrace{\sum\limits_{m=\alpha_{1}}^{\infty}\frac{(\iota
H_{j})^{m}}{m!}X_{j},\frac{(\iota
H_{j})^{\alpha_{2}}}{\alpha_{2}!}X_{j},\cdots,\frac{(\iota
H_{j})^{\alpha_{\ell}}}{\alpha_{\ell}!}X_{j}}_{\ell}\right).$

From Eqs. (5.3.2) (92) and (93), we have

$\displaystyle Q_{q+1,\overline{\phi}_{\acute{\otimes},j},X_{j}}(H_{j})$ (94)
$\displaystyle=$ $\displaystyle T^{e^{\iota
H_{j}}X_{j},X_{j}}_{\overline{\phi}^{[1]}_{\acute{\otimes},j}}\left(\sum\limits_{m=q+1}^{\infty}\frac{(\iota
H_{j})^{m}}{m!}X_{j}\right)$
$\displaystyle+\sum\limits_{\ell=2}^{q+1}\left(\sum\limits_{\shortstack{
$\alpha_{1}=1,\alpha_{2},\cdots,\alpha_{\ell}\geq 1$ \\\
$\left|\bm{\alpha}\right|=q+1$}}+\sum\limits_{\shortstack{ $\alpha_{1}\geq
2,\alpha_{2},\cdots,\alpha_{\ell}\geq 1$ \\\
$\left|\bm{\alpha}\right|=q+1$}}\right)$ $\displaystyle T^{e^{\iota
H_{j}}X_{j},X_{j},\cdots,X_{j}}_{\overline{\phi}^{[\ell]}_{\acute{\otimes},j}}\left(\underbrace{\sum\limits_{m=\alpha_{1}}^{\infty}\frac{(\iota
H_{j})^{m}}{m!}X_{j},\frac{(\iota
H_{j})^{\alpha_{2}}}{\alpha_{2}!}X_{j},\cdots,\frac{(\iota
H_{j})^{\alpha_{\ell}}}{\alpha_{\ell}!}X_{j}}_{\ell}\right)$
$\displaystyle+T^{e^{\iota
H_{j}}X_{j},X_{j},\cdots,X_{j}}_{\overline{\phi}^{[q+1]}_{\acute{\otimes},j}}\left(\underbrace{\sum\limits_{m=1}^{\infty}\frac{(\iota
H_{j})^{m}}{m!}X_{j},\iota H_{j}X_{j},\cdots,\iota H_{j}X_{j}}_{q+1}\right)$
$\displaystyle=$
$\displaystyle\sum\limits_{\ell=1}^{q+1}\sum\limits_{\shortstack{
$i_{1},\cdots,i_{\ell}\geq 1$ \\\ $\left|\bm{i}\right|=q+1$}}T^{e^{\iota
H_{j}}X_{j},X_{j},\cdots,X_{j}}_{\overline{\phi}^{[\ell]}_{\acute{\otimes},j}}\left(\underbrace{\sum\limits_{m=i_{1}}^{\infty}\frac{(\iota
H_{j})^{m}}{m!}X_{j},\frac{(\iota
H_{j})^{i_{2}}}{i_{2}!}X_{j},\cdots,\frac{(\iota
H_{j})^{i_{\ell}}}{i_{\ell}!}X_{j}}_{\ell}\right).$

Then, this lemma is proved by induction and summing Eq. (94) with respect to
the variable $j$. $\hfill\Box$

We require another lemma to bound MOIs associated to $Q_{k,f,\bm{X}}(\bm{H})$.

###### Lemma 8

Let $X_{j}$ be a random unitary operators, $H_{j}$ be an operator with bounded
norm and $\overline{\phi}_{\acute{\otimes},j}$ be a function constructible by
the _limit of integral projective tensor product_ , then we have

$\displaystyle\left\|T^{e^{\iota
H_{j}}X_{j},X_{j},\cdots,X_{j}}_{\overline{\phi}^{[\ell]}_{\acute{\otimes},j}}\left(\underbrace{\sum\limits_{m=i_{1}}^{\infty}\frac{(\iota
H_{j})^{m}}{m!}X_{j},\frac{(\iota
H_{j})^{i_{2}}}{i_{2}!}X_{j},\cdots,\frac{(\iota
H_{j})^{i_{\ell}}}{i_{\ell}!}X_{j}}_{\ell}\right)\right\|$ $\displaystyle\leq$
$\displaystyle\left\|\overline{\phi}^{[\ell]}_{\acute{\otimes},j}\right\|\rho(H_{j},\bm{i},\ell),$
(95)

where $\rho(H_{j},\bm{i},\ell)$ is defined as

$\displaystyle\rho(H_{j},\bm{i},\ell)$
$\displaystyle\stackrel{{\scriptstyle\mbox{\tiny def}}}{{=}}$
$\displaystyle\left(\sum\limits_{m=i_{1}}^{\infty}\frac{\left\|H_{j}\right\|^{m}}{m!}\right)\prod\limits_{p=2}^{\ell}\frac{\left\|H_{j}\right\|^{i_{p}}}{i_{p}!}.$
(96)

Proof: Since we have

$\displaystyle\left\|T^{e^{\iota
H_{j}}X_{j},X_{j},\cdots,X_{j}}_{\overline{\phi}^{[\ell]}_{\acute{\otimes},j}}\left(\underbrace{\sum\limits_{m=i_{1}}^{\infty}\frac{(\iota
H_{j})^{m}}{m!}X_{j},\frac{(\iota
H_{j})^{i_{2}}}{i_{2}!}X_{j},\cdots,\frac{(\iota
H_{j})^{i_{\ell}}}{i_{\ell}!}X_{j}}_{\ell}\right)\right\|$ (97)
$\displaystyle\leq_{1}$
$\displaystyle\left\|\overline{\phi}^{[\ell]}_{\acute{\otimes},j}\right\|\left\|\sum\limits_{m=i_{1}}^{\infty}\frac{(\iota
H_{j})^{m}}{m!}X_{j}\right\|\prod\limits_{p=2}^{\ell}\left\|\frac{(\iota
H_{j})^{i_{p}}}{i_{p}!}X_{j}\right\|$ $\displaystyle\leq_{2}$
$\displaystyle\left\|\overline{\phi}^{[\ell]}_{\acute{\otimes},j}\right\|\left(\sum\limits_{m=i_{1}}^{\infty}\frac{\left\|H_{j}\right\|^{m}}{m!}\right)\prod\limits_{p=2}^{\ell}\frac{\left\|H_{j}\right\|^{i_{p}}}{i_{p}!}$
$\displaystyle=$
$\displaystyle\left\|\overline{\phi}^{[\ell]}_{\acute{\otimes},j}\right\|\rho(H_{j},\bm{i},\ell),$

where $\leq_{1}$ comes from Lemma 1 and $\leq_{2}$ comes from that the
operator norm is submultiplicative and triangle inequality, this lemma is
proved by Eq. (96). $\hfill\Box$

We are ready to present the following theorem about the tail bound for
$Q_{k,f,\bm{X}}(\bm{H})$.

###### Theorem 11

For $j=1,2,\cdots,n$, let $X_{j}$ be a random unitary operator, $H_{j}$ be a
deterministic operator with bounded norm. The function
$f:\mathbb{R}^{n}\rightarrow\mathbb{R}$ is assumed to be decomposed as
$f(\bm{X})=\sum\limits_{j=1}^{n}\overline{\phi}_{\acute{\otimes},j}(X_{j})$.
Then, we have

$\displaystyle\mathrm{Pr}\left(\left\|Q_{k,f,\bm{X}}(\bm{H})\right\|>\theta\right)$
$\displaystyle\leq$
$\displaystyle\sum\limits_{j=1}^{n}\sum\limits_{\ell=1}^{k}\frac{kn\Theta(j,k,\ell)}{\theta}\mathbb{E}\left[\left\|\overline{\psi}^{[\ell]}_{\acute{\otimes},j}\right\|\right],$
(98)

where $\Theta(j,k,\ell)\stackrel{{\scriptstyle\mbox{\tiny
def}}}{{=}}\sum\limits_{\shortstack{ $i_{1},\cdots,i_{\ell}\geq 1$ \\\
$\left|\bm{i}\right|=k$}}\rho(H_{j},\bm{i},\ell)$. Note that
$\rho(H_{j},\bm{i},\ell)$ is defined by Eq. (96).

Proof: Because we have

$\displaystyle\mathrm{Pr}\left(\left\|Q_{k,f,\bm{X}}(\bm{H})\right\|>\theta\right)=_{1}$
(99)
$\displaystyle\mathrm{Pr}\left(\left\|\sum\limits_{j=1}^{n}\left[\sum\limits_{\ell=1}^{k}\sum\limits_{\shortstack{
$i_{1},\cdots,i_{\ell}\geq 1$ \\\
$\left|\bm{i}\right|=k$}}\right.\right.\right.$
$\displaystyle\left.\left.\left.T^{e^{\iota
H_{j}}X_{j},X_{j},\cdots,X_{j}}_{\overline{\phi}^{[\ell]}_{\acute{\otimes},j}}\left(\underbrace{\sum\limits_{m=i_{1}}^{\infty}\frac{(\iota
H_{j})^{m}}{m!}X_{j},\frac{(\iota
H_{j})^{i_{2}}}{i_{2}!}X_{j},\cdots,\frac{(\iota
H_{j})^{i_{\ell}}}{i_{\ell}!}X_{j}}_{\ell}\right)\right]\right\|>\theta\right)$
$\displaystyle\leq$
$\displaystyle\mathrm{Pr}\left(\sum\limits_{j=1}^{n}\sum\limits_{\ell=1}^{k}\sum\limits_{\shortstack{
$i_{1},\cdots,i_{\ell}\geq 1$ \\\ $\left|\bm{i}\right|=k$}}\right.$
$\displaystyle\left.\left\|T^{e^{\iota
H_{j}}X_{j},X_{j},\cdots,X_{j}}_{\overline{\phi}^{[\ell]}_{\acute{\otimes},j}}\left(\underbrace{\sum\limits_{m=i_{1}}^{\infty}\frac{(\iota
H_{j})^{m}}{m!}X_{j},\frac{(\iota
H_{j})^{i_{2}}}{i_{2}!}X_{j},\cdots,\frac{(\iota
H_{j})^{i_{\ell}}}{i_{\ell}!}X_{j}}_{\ell}\right)\right\|>\theta\right)$
$\displaystyle\leq_{2}$
$\displaystyle\mathrm{Pr}\left(\sum\limits_{j=1}^{n}\sum\limits_{\ell=1}^{k}\left\|\overline{\psi}^{[\ell]}_{\acute{\otimes},j}\right\|\underbrace{\sum\limits_{\shortstack{
$i_{1},\cdots,i_{\ell}\geq 1$ \\\
$\left|\bm{i}\right|=k$}}\rho(H_{j},\bm{i},\ell)}_{\stackrel{{\scriptstyle\mbox{\tiny
def}}}{{=}}\Theta(j,k,\ell)}>\theta\right)$ $\displaystyle\leq$
$\displaystyle\sum\limits_{j=1}^{n}\sum\limits_{\ell=1}^{k}\mathrm{Pr}\left(\left\|\overline{\psi}^{[\ell]}_{\acute{\otimes},j}\right\|>\frac{\theta}{kn\Theta(j,k,\ell)}\right)$
$\displaystyle\leq_{3}$
$\displaystyle\sum\limits_{j=1}^{n}\sum\limits_{\ell=1}^{k}\frac{kn\Theta(j,k,\ell)}{\theta}\mathbb{E}\left[\left\|\overline{\psi}^{[\ell]}_{\acute{\otimes},j}\right\|\right],$

where $=_{1}$ comes from Lemma 7, $\leq_{2}$ is due to Lemma 8, and $\leq_{3}$
Markov inequality again; this theorem is proved. $\hfill\Box$

## References

  * [1] J. L. Daletskii and S. G. Krein, “Integration and differentiation of functions of hermitian operators and applications to the theory of perturbations,” _AMS Translations (2)_ , vol. 47, no. 1-30, pp. 10–1090, 1965.
  * [2] M. S. Birman and M. Z. Solomyak, “Double stieltjes operator integrals and problems on multipliers,” in _Doklady Akademii Nauk_ , vol. 171, no. 6. Russian Academy of Sciences, 1966, pp. 1251–1254.
  * [3] M. S. Birman and M. Solomyak, “Double stieltjes operator integrals, problems math. phys. leningrad. univ. 1 (1966) 33–67,” _English transl.: Topics Math. Physics_ , vol. 1, pp. 25–54, 1967.
  * [4] ——, “Double operator integrals in a hilbert space,” _Integral equations and operator theory_ , vol. 47, no. 2, pp. 131–168, 2003.
  * [5] V. V. Peller, “Hankel operators in the perturbation theory of unitary and self-adjoint operators,” _Functional Analysis and Its Applications_ , vol. 19, no. 2, pp. 111–123, 1985.
  * [6] ——, “Multiple operator integrals and higher operator derivatives,” _Journal of Functional Analysis_ , vol. 233, no. 2, pp. 515–544, 2006.
  * [7] A. Skripka and A. Tomskova, _Multilinear Operator Integrals_. Springer, 2019.
  * [8] A. V. Skorohod, _Random linear operators_. Springer Science & Business Media, 2001, vol. 1.
  * [9] W. Hackenbroch, “Point localization and spectral theory for symmetric random operators,” _Archiv der Mathematik_ , vol. 92, no. 5, pp. 485–492, 2009\.
  * [10] D. H. Thang and T. X. Quy, “On the spectral theorem for random operators.” _Southeast Asian Bulletin of Mathematics_ , vol. 41, no. 2, 2017.
  * [11] S. Y. Chang, “Random double tensors integrals,” _arXiv preprint arXiv:2204.01927_ , 2022.
  * [12] ——, “Random parametrization double tensors integrals and their applications,” _arXiv preprint arXiv:2205.03523_ , 2022.
  * [13] M. Liang and B. Zheng, “Further results on Moore-Penrose inverses of tensors with application to tensor nearness problems,” _Computers and Mathematics with Applications_ , vol. 77, no. 5, pp. 1282–1293, March 2019.
  * [14] C. K. Chui and X. Li, “Realization of neural networks with one hidden layer,” in _Multivariate approximation: From CAGD to wavelets_. World Scientific, 1993, pp. 77–89.
  * [15] D. Pérez and Y. Quintana, “A survey on the weierstrass approximation theorem,” _arXiv preprint math/0611038_ , 2006.