2cm2cm2cm2cm

Function-measure kernels, self-integrability and uniquely-defined stochastic
integrals

Ricardo Carrizo Vergara
<EMAIL_ADDRESS>
March 2023

Abstract

In this work we study the self-integral of a function-measure kernel and its
importance on stochastic integration. A continuous-function measure kernel $K$
over $D\subset\mathbb{R}^{d}$ is a function of two variables which acts as a
continuous function in the first variable and as a real Radon measure in the
second. Some analytical properties of such kernels are studied, particularly
in the case of cross-positive-definite type kernels. The self-integral of $K$
over a bounded set $D$ is the “integral of $K$ with respect to itself”. It is
defined using Riemann sums and denoted $\int_{D}K(x,dx)$. Some examples where
such notion is well-defined are presented. This concept turns out to be
crucial for unique-definiteness of stochastic integrals, that is, when the
integral is independent of the way of approaching it. If $K$ is the cross-
covariance kernel between a mean-square continuous stochastic process $Z$ and
a random measure with measure covariance structure $M$, $\int_{D}K(x,dx)$ is
the expectation of the stochastic integral $\int_{D}ZdM$ when both are
uniquely-defined. It is also proven that when $Z$ and $M$ are jointly
Gaussian, self-integrability properties on $K$ are necessary and sufficient to
guarantee the unique-definiteness of $\int_{D}ZdM$. Results on integrations
over subsets, as well as potential $\sigma$-additive structures are obtained.
Three applications of these results are proposed, involving tensor products of
Gaussian random measures, the study of a uniquely-defined stochastic integral
with respect to fractional Brownian motion with Hurst index $H>\frac{1}{2}$,
and the non-uniquely-defined stochastic integrals with respect to orthogonal
random measures. The studied stochastic integrals are defined with great
generality on the integrand and the integrator and without use of martingale-
type conditions, providing a potential filtration-free approach to stochastic
calculus grounded on covariance structures.

Keywords Self-integral, kernel, random measure, stochastic integral, tensor
product.

##  1 Introduction

This work is focused on a concept which, to the best of our knowledge, has not
received much attention in the literature, neither as an interesting
mathematical object it its own right, nor as a potentially important tool in
applied mathematics. This concept is the self-integral of a function-measure
kernel, and its main application is in stochastic integration. A function-
measure kernel is a structure defined over the cross product of two measurable
spaces which acts as a function in one component and as a measure in the
other. The self-integral is a sort of integral of the kernel with respect to
itself, that is, of its first component with respect to its second component.
Such a notion may seem peculiar at a first sight but it is simple to conceive
examples where it is well-defined. The purpose of this work is triple: to
study some basic analytical properties of function-measure kernels, to define
and study the concept of self-integrability, and to show the importance of
such concept in stochastic integration, where it is found to be a key concept
regarding uniquely-defined stochastic integrals.

We consider a function $K:D\times\mathscr{E}\to\mathbb{R}$, with $D$ some
measurable space and $\mathscr{E}$ some $\sigma$-algebra (or $\delta$-ring),
for which $K(\cdot,A)$ is a function and $K(x,\cdot)$ is a measure. Such kind
of kernel is widely used in the literature, and maybe the most important
example is a transition or conditional probability kernel, in which case
$K(x,\cdot)$ is a probability measure and $K(\cdot,A)$ is a positive
measurable function. In this work though the kernel $K$ will be used for other
purposes. First of all, we consider different regularity properties:
$K(\cdot,A)$ will be taken to be a continuous function over $D$ and
$K(x,\cdot)$ will be a real Radon measure. We shall focus on the case
$D\subset\mathbb{R}^{d}$ and $\mathscr{E}=\mathcal{B}(\mathbb{R}^{p})$, though
many of our results can be generalized to the classical settings where Radon
measures are used (replace $D$ or $\mathbb{R}^{p}$ by any locally compact
Hausdorff topological space which can be written as a countable union of
compact sub-spaces). The main role of such kernel $K$ in a stochastic context
is to describe cross-covariance structures. This implies the use of kernels
$K$ which have an extra cross-positive-definiteness condition which allows
them to describe the covariance between a mean-square continuous stochastic
process and a second-order random measure.

The notion of self-integral can be conceived when
$\mathscr{E}=\mathcal{B}(D)$, hence the left and the right components of $K$
act over the same measurable space. The objective is to define the “integral
of $K$ with respect to itself”, which in this work is noted

$\int_{D}K(x,dx).$ (1.1)

The author is not aware of any literature which has tried to formalise such
notion. Nonetheless, some readers specialised in analysis may recognize that
the self-integral is a trace operator (Ryan, 2002, Section 1.3). Indeed,
members of the tensor product space $C(D)\otimes\mathscr{M}(D)$, where $C(D)$
is the space of continuous functions over $D$ and $\mathscr{M}(D)$ is the
space of real Radon measures over $D$, form a particular class of function-
measure kernels where the self-integral can be defined trivially (see Section
4). Since for $D$ compact $C^{\prime}(D)=\mathscr{M}(D)$ (Riesz
representation), the self-integral over $D$ can be defined as a trace operator
over $C(D)\otimes C^{\prime}(D)$, and it can be eventually extended to more
general cases using the completion of such tensor product with the projective
topology (Ryan, 2002, Chapter 2). This analytical approach will be studied
further in a future paper where we will try to define the self-integral
through series expansion methods. In this paper we focus on an apparently
simpler approach: using Riemann sums to define the self-integral. That is, we
do the same as with classical integrals: relying on Riemann partitions of the
integration domain, tag-points to evaluate the left component of $K$, and so
constructing a limit of Riemann sums. Such self-integral is not always
uniquely-defined: the final limit may depend upon the selected partition or
tag-points. It is hence necessary to study the cases where such limit is
uniquely-defined and so define the concept of self-integral for those.

The self-integral appeared naturally when trying to define uniquely the
stochastic integral between a general mean-square continuous stochastic
process and a second-order random measure (see Definition 2.3 for the exact
notion of random measure we use in this work). By uniquely, we mean that the
integral must not depend upon the way of approaching it. We use an approach
which in our opinion is simpler than most part of the approaches already
present in the literature: the definition using a limit of Riemann sums, and
the study of unique-integrability conditions (independence upon partitions and
tag-points) through the analysis of the cross-covariance structure between the
integrand and the integrator. Such cross-covariance structure can be described
through a function-measure kernel, the same mathematical object described
above. Here we show that the possibility of defining uniquely stochastic
integrals is intimately related to the possibility of defining the self-
integral of this cross-covariance kernel: it is indeed a necessary condition
for it, since the mean of the stochastic integral is the self-integral of the
cross-covariance kernel. In addition, when the integrator and the integrand
are jointly Gaussian, necessary and sufficient conditions for a uniquely-
defined stochastic integral rely entirely on self-integrability properties of
the cross-covariance kernel. Such conditions are exposed in Theorem 5.1, which
is the main result of this work. While, of course, the study of second-moment
structures is not enough to fully characterise the laws of the objects, and
hence it cannot cover the whole richness of the theory of stochastic
processes, we believe that these results are powerful and simple enough to
propose another way of tackling the subject, which may have potential both for
generalization to more complicated cases, as well as for applications. We also
remark that this approach can be easily applied for stochastic integrals over
$\mathbb{R}^{d}$, and it is done regardless (in principle) of any martingale-
type condition on the integrator or adaptability on the integrand.

###  1.1 Motivations and some bibliographical revision

Let us expose some thoughts about the current developments on stochastic
integrals. There is one simple but important question for which this work aims
to give some enlightening: when can we integrate a stochastic process with
respect to a random measure? One could try to mimic the definition of the
integral in the deterministic case, but it is widely known that in the
stochastic world things are not that simple. Classical stochastic integration
against (the derivative of) a Brownian motion uses intuitively constructed
stochastic integrals through Riemann sums and simple-functions approximations,
for which different ways of approaching the integral produce different results
(classical difference between the Itô and Stratonovich integrals, see
(Øksendal, 2003, Chapter 3)). When can we thus integrate the stochastic
process with respect to the random measure uniquely? Which properties must
satisfy both the process and the random measure in order to assure the limit
to be well-defined and independent of the way of approaching it?

It turns out that this question seems not to have been considered in the
literature. While the study of stochastic integrals has produced a succulent
quantity of work in many directions for over 100 years since the first works
of Wiener (Wiener, 1923), the particular question of conditions for uniquely-
defined stochastic integral seems not to have been taken into account. In
(Loève, 1978, Section 37.3), the author proposes a definition of a stochastic
integral in a quite general setting of a second-order process as integrand and
the increments of a second-order process whose covariance has bounded
variation as integrator111The notion of random measure that we use in this
work actually coincides with this one, although we explicitly work with the
covariance measure rather than with the increments of its primitive function..
In such definition it is required that the limit must be independent of the
selected tag-points in the Riemann-Stieltjes sum. However, the author does not
provide general criteria for such condition to hold, except for the particular
case when the integrator and the integrand are independent, case which we will
revisit in Section 5.1. It is also known that when the integrand or the
integrator is deterministic the stochastic integral is uniquely-defined, but
this is actually a particular case of independence (we shall revisit such
constructions in Section 2.2). Besides such cases, it is not easy to find
bibliographical sources with relatively simple approaches to stochastic
integration with huge generality and caring about a unique definition. For
instance, in (Huang Cambanis, 1978) a construction of a stochastic integral
with general Gaussian integrators and general (non-adapted) integrands is done
following an approach based on tensor products and extensions on Hilbert
spaces, but the approach has already been considered complicated (Borisov
Bystrov, 2006). Another attempt to properly define a particular form of
stochastic integral regardless on the way of approaching it is the one present
in (Rao, 2012, Chapter 5). Here the author looks for a definition of a
convolution between two second-order random measures, which implicitly uses a
stochastic integral. Roughly speaking, the author describes the convolution
between the covariance bi-measures of both random measures as a new covariance
bi-measure, and thanks to Kolmogorov Theorem (see Proposition 5.1.3 in (Rao,
2012)), they construct a new random measure following such covariance.
However, this is an abstract construction which does not link directly the
paths of the initial random measures with the paths of the final convolution.
For instance, a similar and maybe more exploitable result can be obtained if
we use the convolution of two independent second-order random measures, which
can be defined without any problem.

On typical approaches on stochastic integration there is another point worth
mentioning. Virtually every approach presents some special conditions on the
integrand and/or the integrator which may seem more restrictive than what we
could conceive at a first (maybe naive) sight. The integrator must satisfy a
martingale-type condition, rather than just belonging to a simple and general
enough class of random measures. A semi-martingale condition on the primitive
of the random measure seems to be the most general condition one may find
(Kunita Watanabe, 1967; Bichteler Jacod, 2006; Dellacherie, 2006). On the
other hand, the integrand must be adapted to the filtration induced by the
random measure, rather than being just an enough regular process (mean-square
continuous for instance). These conditions are usually asked for mathematical
convenience (Lebesgue Dominated Theorem holds, see (Bichteler Jacod, 2006;
Chong Klüppelberg, 2015)), hence mathematicians have built the whole
functional theory over them. It is also possible that these conditions are
preferred because they follow an intuition of “not-looking-into-the-future"
(Øksendal, 2003, Chapter 3). It seems that they are totally inspired from a
purely temporal point of view, i.e. for stochastic integrals over the real
line rather than the plane or other spaces. Nonetheless, stochastic integrals
over the plane have been developed by applying those same principles to the
spatial axes and thus doing iterated Itô integrals (Itô, 1951; Cairoli Walsh,
1975) . This procedure is of course mathematically possible and legitimate but
it seems “artificial” since, in principle, there is no reason or human
intuition for which a spatial process should follow an adaptability condition
along the spatial axes222When someone requires to a stochastic processes over
time to be adapted with respect to a filtration, there is an implicit idea of
causality or time arrow behind this mathematical intuition. One must not
forget that these notions are also conceptual abstractions which we can or not
make part of the mathematical models. We could in principle conceive
mathematical models which do not consider them.. We wonder hence if it is not
possible to tackle the general problem of stochastic integration without using
such concepts. The adaptability is a way of describing the dependence between
the process and the measure, but maybe other simpler methods could be more
exploitable in some circumstances. The already mentioned not-so-simple
approaches in (Rao, 2012; Huang Cambanis, 1978) are some examples which treat
in a relatively complete manner the question of doing stochastic integration
without adaptability conditions. Skorokhod integral (Skorokhod, 1976), which
has been developed using Malliavin calculus (Nualart, 2006), is also another
successful approach to stochastic integration which does not consider
adaptability conditions on the integrand, but it is usually restricted to the
White Noise random measure (derivative of Brownian motion) for its
construction. It can be extended to more general random measures (Mocioalca
Viens, 2005) but in general, it seems that it is necessary to fix the random
measure and then construct the integration theory with respect to it. Hence,
there is no a relatively free manipulation in both the integrand and the
integrator at the same time. There are also other more sophisticated
approaches to stochastic integration through the definition of stochastic
products between quite general stochastic structures. The stochastic integral
is a sort of multiplication between a random function and a random measure,
but theories which uses more general stochastic objects, such as integrators
with controlled $p$-variations and random distributions have been developed
during the last decades. The Wick product (Holden ., 2009), rough path theory
and para-controlled distributions (Gubinelli, 2004; Gubinelli ., 2015) and the
theory of regularity structures (Hairer, 2014) are examples of developments
which allow the definition of the product of abstract stochastic structures
under suitable contexts. However, these magnificent theories are of a
technicality and sophistication which could demotivate some readers from the
applied world.

One final motivation for this work comes from a topic in applied mathematics:
the SPDE approach in Geostatistics (Lindgren ., 2011; Carrizo Vergara, 2018).
This approach consists in the analysis of spatialised data-sets interpreted as
the realisation of the solution of a Stochastic Partial Differential Equation
(SPDE). We refer to the seminal paper (Lindgren ., 2011) as the first highly
popular paper which exploits this link between SPDEs and Spatial Statistics
techniques, and to (Lindgren ., 2022) as a recent review on how the topic has
evolved in the Spatial Statistics community. However, the SPDE approach is
until now applied in a purely linear setting: linear SPDEs and linear
statistical methods based on covariance structures. So, there is an open door
for an ambitious research program: the systematic study and exploitation of a
potential link between non-linear Geostatistical methods and non-linear SPDEs.
Unfortunately, the state-of-the-art of the theory of non-linear SPDEs seems to
be quite technical and sophisticated, and it is not clear how to proceed to
create systematically this link with enough generality and simplicity. In this
context, this work tries to propose a humble beginning into this aim:
constructing an stochastic calculus based on covariance and cross-covariance
structures, the basic Geostatistical tools, in a general enough and simple
manner. Note that the basis of any non-linear stochastic analysis is the
definition of products between stochastic objects (the stochastic integral
being a fundamental case). Those products are the ones involved in the
definition of a non-linear SPDE and on the expression of its potential
solutions. Hence, exploring the link between stochastic integrals and cross-
covariance structures may open the road for a link between Spatial Statistics
techniques and the analysis of non-linear SPDEs.

Because all these reasons, we consider that the approach of defining
stochastic integrals by simple Riemann sums, with the generality of mean-
square continuous stochastic processes as integrands and random measures as
integrators, and with their cross-covariance structure determining the
possibility of defining such integral, is worth of our work and time.

###  1.2 Structure of the paper

This paper is organized as follows. In Section 2 we introduce the basic
objects we use in this work which rely on measure theoretical results, as well
as the notion of random measure we use: a second-order random measure whose
covariance is identified to a measure over the double-dimension space. In
Section 3 we introduce continuous function-measure kernels $K$ and their
related total variation kernel $|K|$. We obtain some analytical results for
those concerning iterated integrals ans a Fubini Theorem in Section 3.1, and
in Section 3.2 we study the properties of cross-covariance kernels. Section 4
is devoted to the definition of the self-integral of a kernel $K$. It is here
proven that for some kernels such notion is not uniquely-defined. An auxiliary
concept of second-order kernel $K^{(2)}$ is introduced from a cross-tensor
product between $K$ and itself, and we define a weaker self-integral for it,
called the quasi-self-integral.Section 5 is the most important one. It is
devoted to the connection between self-integrability and uniquely-defined
stochastic integrals. In Section 5.1 we define a uniquely-defined stochastic
integral between a mean-square continuous process and a random measure over
bounded sets. We prove that the expectation of such integral is the self-
integral of the cross-covariance kernel between the integrand and the
integrator (the self-integrability being hence necessary for unique-
definiteness). Section 5.2 we present our main Theorem 5.1, which says that
when the integrand and the integrator are jointly Gaussian, the unique-
definiteness of the stochastic integral is equivalent to the self-
integrability of the cross-covariance kernel and the quasi-self-integrability
of its second order kernel. Section 5.3 focuses on stochastic and self-
integrability on subsets (which does not follow immediately from our
definitions). It turns out that the self-integral and the stochastic integral
are additive functions when uniquely-defined. Theorem 5.3 synthesize these
results in the jointly Gaussian case and presents formulae for the mean and
the covariance between stochastic integrals. In Section 5.4 the potential
measure structure of the self-integral and the stochastic integrals are
studied. We give sufficient conditions for those to determine measures over
the space. In Section 6 is devoted to some applications of these results. In
Section 6.1 we construct tensor products of jointly Gaussian random measures
and we give their mean and covariance structures. Section 6.2 focuses on the
case of fractional Brownian motion $B_{H}$ with Hurst index $H>\frac{1}{2}$,
for which its derivative is a random measure (Borisov Bystrov, 2006) enough
well-behaved so the stochastic integral of $B_{H}$ with respect to its
derivative is uniquely-defined. In Section 6.3 we study the case where the
derivative of the integrand is an orthogonal random measure without atoms. In
such case, the corresponding stochastic integral is never uniquely-defined.
This can be interpreted as follows: even if the tensor products random
measures can be uniquely-defined, the corresponding double integrals do not
have a canonical disintegration. We finalise this paper with the conclusive
Section 7. In Section 7.1 we propose some open questions inspired from the
developments here presented. In Section 7.2 we show that the results here
obtained can be used to study more multiplicative and tensor products between
more general stochastic structures in a manner analogous to the stochastic
integral.

##  2 Notation and Preliminaries

We denote $\mathbb{N}=\\{0,1,2,...\\}$. For a topological space $D$, $C(D)$
denotes the space of complex continuous functions over $D$, and $C_{c}(D)$ the
space of complex continuous functions with compact support. If
$D\subset\mathbb{R}^{d}$ is a Borel set, we denote $\mathcal{B}(D)$ the
$\sigma$-algebra of Borel subsets of $D$, and $\mathcal{B}_{B}(D)$ the sub-
collection of $\mathcal{B}(D)$ consistent of bounded sets (which is a
$\delta$-ring in general). We denote $\lambda$ the Lebesgue measure. The space
of complex (Borel) measurable functions over $D$ is denoted $\mathcal{M}(D)$.
$\mathcal{M}_{B}(D)$ and $\mathcal{M}_{B,c}(D)$ are the subspaces of
$\mathcal{M}(D)$ consisting of bounded functions and bounded compactly-
supported functions respectively. $\|\cdot\|_{\infty,A}$ denotes the supremum
norm over the set $A$ ($A$ is omitted if it is clear from context).
$\mathbbm{1}_{A}$ is the indicator function of the set $A$. LDCT stands for
Lebesgue Dominated Convergence Theorem.

Concerning probabilistic definitions, we work always with a fixed probability
space $(\Omega,\mathcal{A},\mathbb{P})$. A real stochastic process $Z$ is a
collection of real random variables indexed by a non-empty set $(Z(x))_{x\in
D}$ ($D$ is not necessarily a time interval). If $Z(x)\in
L^{2}(\Omega,\mathcal{A},\mathbb{P})$ for every $x\in D$, then $Z$ is said to
be a second-order process and the functions $m_{Z}(x)=\mathbb{E}(Z(x))$ and
$C_{Z}(x,y)=\mathbb{C}ov(Z(x),Z(y))$ are called the mean and the covariance
functions respectively. If $D\subset\mathbb{R}^{d}$, a stochastic process
$(Z(x))_{x\in D}$ is said to be mean-square continuous over $D$ if for every
$x_{0}\in D$, $Z(x)\xrightarrow[x\to x_{0}]{L^{2}(\Omega)}Z(x_{0})$. It is
known that $Z$ is mean-square continuous if and only if $m_{Z}\in C(D)$ and
$C_{Z}\in C(D\times D)$ (Sobczyk, 1991, Section 21).

###  2.1 Basic concepts in measure theory

We focus on the measurable space
$(\mathbb{R}^{d},\mathcal{B}(\mathbb{R}^{d}))$ and on subsets of it.

######  Definition 2.1.

A real measure over $D\in\mathcal{B}(\mathbb{R}^{d})$ is a function
$\mu:\mathcal{B}_{B}(D)\to\mathbb{R}$ which is $\sigma$-additive over
$\mathcal{B}_{B}(D)$.

We remark that $\mu$ may not be defined over the whole Borel $\sigma$-algebra
(some authors would use the word pre-measure for $\mu$ (Kupka, 1978)).
Definition 2.1 implies that $\mu$ is locally finite hence Radon since we are
working over the Euclidean space. The real vector space of real measures over
$D$ is denoted $\mathscr{M}(D)$. $\mu$ is said to be positive if it takes non-
negative values. The total-variation measure of $\mu$ is the function
$|\mu|:\mathcal{B}_{B}(D)\to\mathbb{R}^{+}$ defined by

$|\mu|(A):=\sup\left\\{\sum_{j\in J}|\mu(I_{j})|\ \big{|}\ (I_{j})_{j\in
J}\hbox{ Borel finite partition of }A\right\\}.$ (2.1)

$|\mu|$ is indeed a measure (Rudin, 1987, Chapter 6) and it is the smallest
positive measure such that $|\mu(A)|\leq|\mu|(A)$ for every
$A\in\mathcal{B}_{B}(D)$. If $D$ is compact the number $|\mu|(D)$ is called
the total-variation norm of $\mu$ over $D$, and it defines a norm over
$\mathscr{M}(D)$ which turns it into a Banach space.

Let $f\in\mathcal{M}(D)$ and $\mu\in\mathscr{M}(D)$. If $|f|$ is Lebesgue
integrable with respect to $|\mu|$ over $D$, then the Lebesgue integral
$\int_{D}fd\mu$ is defined, for instance, using the decomposition of $\mu$
into positive and negative parts (Rudin, 1987, Chapter 6). In such case, we
use the linear-functional notation for the integral

$\langle\mu,f\rangle:=\int_{D}fd\mu:=\int_{D}f(x)d\mu(x),$ (2.2)

if $D$ is clear from context. We shall use the bracket or the integral
notation depending on the ease of reading of the context. Of course we always
have $|\langle\mu,f\rangle|\leq|\mu|(D)\|f\|_{\infty,D}$.

We can also study measures through the Bourbakian approach (Bourbaki, 1965),
that is, exploiting the Riesz Representation Theorem interpreting measures as
members of dual spaces of continuous functions. We introduce such Theorem in
its topology free version over the Euclidean space.

######  Theorem 2.1 (Riesz Representation).

Let $T:C_{c}(\mathbb{R}^{d})\to\mathbb{C}$ be a real linear functional. Then,
the following are equivalent

  1. (i)

For every $K\subset\mathbb{R}^{d}$ compact there exists $C_{K}>0$ such that

$|\langle T,\varphi\rangle|\leq
C_{K}\|\varphi\|_{\infty,K},\quad\forall\varphi\in C_{c}(\mathbb{R}^{d})\hbox{
such that }\operatorname{supp}(\varphi)\subset K.$ (2.3)

  2. (ii)

There exists a unique $\mu\in\mathscr{M}(\mathbb{R}^{d})$ such that

$\langle T,\varphi\rangle=\int_{\mathbb{R}^{d}}\varphi
d\mu,\quad\forall\varphi\in C_{c}(\mathbb{R}^{d}).$ (2.4)

######  Remark 2.1.

An implication of Theorem 2.1 is that if $D\subset\mathbb{R}^{d}$ is compact
and $C(D)$ is endowed with the supremum norm (hence a Banach space), then
$C^{\prime}(D)=\mathscr{M}(D)$ in a sense analogue to (2.4). The total-
variation norm over $\mathscr{M}(D)$ coincides then with the operator norm
over $C^{\prime}(D)$.

See (Tao, 2022, Section 1.10) for a proof. In this work Theorem 2.1 is used
mainly to prove that some structures are effectively measures through their
identification with linear functionals satisfying (2.3). The following
auxiliary definition will be widely used.

######  Definition 2.2.

Let $D\in\mathcal{B}_{B}(\mathbb{R}^{d})$. For every $n\in\mathbb{N}$, we
consider a finite index set $J_{n}$ such that $|J_{n}|\nearrow\infty$333$|J|$
denotes the cardinality of the index set $J$., a (finite) Borel partition of
$D$, $(I_{j}^{n})_{j\in J_{n}}$ satisfying $\displaystyle\max_{j\in
J_{n}}\operatorname{diam}(I_{j}^{n})\xrightarrow[n\to\infty]{}0$, and
arbitrary tag-points $x_{j}^{n}\in I_{j}^{n}$. The so-constructed sequence
$\left((I_{j}^{n})_{j\in J_{n}},(x_{j}^{n})_{j\in
J_{n}}\right)_{n\in\mathbb{N}}$ is called a Riemann system of $D$.

If $D\in\mathcal{B}_{B}(\mathbb{R}^{d})$, $f\in C(\mathbb{R}^{d})$ and
$\mu\in\mathscr{M}(\mathbb{R}^{d})$, the Lebesgue integral $\int_{D}fd\mu$ can
be computed as the limit of Riemann sums

$\int_{D}fd\mu=\lim_{n\to\infty}\sum_{j\in J_{n}}f(x_{j}^{n})\mu(I_{j}^{n}),$
(2.5)

the limit being independent of the selected Riemann system of $D$ used. This
method of constructing integrals is the main focus of this work. Note that any
sub-Riemann system, that is, any sub-sequence of the sequence
$\left((I_{j}^{n})_{j\in J_{n}},(x_{j}^{n})_{j\in
J_{n}}\right)_{n\in\mathbb{N}}$ is also a Riemann system, and the associated
Riemann sums should converge to the same integral.

###  2.2 Random measures

In the literature it is possible to find many different notions of a random
measure. Maybe the most intuitive one consists in a random variable taking
values in a space of measures. A lot of work has been done around such concept
(Horowitz, 1986; Kallenberg, 2017; Baccelli ., 2020). A slightly weaker, but
also intuitive and probabilistic-friendly definition is to consider a random
variable which takes values in a space of measures almost-surely. This is the
case of Poisson and other point processes. However, some important examples of
other kinds of random measures do not satisfy these requirements. The most
important example is Gaussian White Noise over $\mathbb{R}^{d}$, whose
realisations have almost-surely infinite variation over bounded sets (Dalang
., 2009, Exercice 3.16).

Another notion of random measure which is often used (Morando, 1969; Rao,
2012) is the one of a vector measure taking values in a (topological) vector
space of random variables, such as $L^{p}(\Omega,\mathcal{A},\mathbb{P})$ for
$p\in[0,\infty]$. To be precise, if $(E,\mathscr{E})$ is a measurable space
($\mathscr{E}$ being a $\sigma$-algebra or a $\delta$-ring), a
$L^{p}(\Omega,\mathcal{A},\mathbb{P})$-random measure $M$ over $E$ is a
$\sigma$-additive function $M:\mathscr{E}\to
L^{p}(\Omega,\mathcal{A},\mathbb{P})$. The most popular case is $p=2$, where
$M$ is also called a second-order random measure (Thornett, 1979; Åberg
Podgórski, 2011). Gaussian White Noise over $\mathbb{R}^{d}$ enters in such
definition. For a second-order random measure, one can define mean and
covariance structures given by the functions
$A\in\mathscr{E}\mapsto\mathbb{E}(M(A))$ and
$(A,B)\in\mathscr{E}\times\mathscr{E}\mapsto\mathbb{C}ov(M(A),M(B))$
respectively. The first one defines a measure over $(E,\mathscr{E})$, noted
$m_{M}$ and called the mean measure. The second one defines a bi-measure over
$\mathscr{E}\times\mathscr{E}$, that is, a structure which defines a measure
over $A$ when $B$ is fixed and conversely. One important remark is that the
covariance structure does not necessarily define a measure over the cross-
space $(E\times E,\mathscr{E}\otimes\mathscr{E})$. That is, there are examples
of second-order random measures $M$ for which there exists no measure
$\Lambda$ over $(E\times E,\mathscr{E}\otimes\mathscr{E})$ such that
$\Lambda(A\times B)=\mathbb{C}ov(M(A),M(B))$ (Rao, 2012, Chapter 2, Example
2). Hence some extra precautions must be taken into account when using a
second-order random measure. To avoid those, we shall use the following
simplified definition.

######  Definition 2.3.

A second-order measure-covariance random measure (from now on, random measure)
over $D\in\mathcal{B}(\mathbb{R}^{d})$ is a second-order random measure
$M:\mathcal{B}_{B}(D)\to L^{2}(\Omega,\mathcal{A},\mathbb{P})$ such that there
exists a measure $C_{M}\in\mathscr{M}(D\times D)$ such that

$C_{M}(A\times B)=\mathbb{C}ov(M(A),M(B)),\quad\forall
A,B\in\mathcal{B}_{B}(D).$ (2.6)

Definition 2.3 is the only sense in which the words “random measure” will be
used in this work. Some authors have already remarked the convenience of this
measure-covariance supposition for different aims (Borisov Bystrov, 2006;
Carrizo Vergara, 2022). The measure $C_{M}$ is called the covariance measure
of $M$. The cautious reader will realize that, all along this work, the
supposition of $C_{M}$ being a measure will be of great help when proving most
part of the results.

If $\varphi\in\mathcal{M}(\mathbb{R}^{d})$ satisfies

$\langle|m_{M}|,|\varphi|\rangle<\infty,\quad\hbox{and}\quad\langle|C_{M}|,|\varphi|\otimes|\varphi|\rangle<\infty,$
(2.7)

then the stochastic integral

$\langle M,\varphi\rangle:=\int_{\mathbb{R}^{d}}\varphi dM$ (2.8)

can be defined. This is nothing but the Dunford-Schwartz integral (Dunford
Schwartz, 1958, Section IV.10) with respect to the
$L^{2}(\Omega,\mathcal{A},\mathbb{P})$-valued measure $M$. If $\varphi,\phi$
satisfy (2.7) then ((Borisov Bystrov, 2006) or (Carrizo Vergara, 2018,
Proposition 3.3.1))

$\mathbb{E}(\langle M,\varphi\rangle)=\langle
m_{M},\varphi\rangle\quad\hbox{and}\quad\mathbb{C}ov\left(\langle
M,\varphi\rangle\langle M,\phi\rangle\right)=\langle
C_{M},\varphi\otimes\overline{\phi}\rangle.$ (2.9)

Similarly as the case of deterministic measures, random measures can be
studied through the Bourbakian approach. The following Theorem is a stochastic
version of Riesz Representation Theorem 2.1 and it is a fundamental tool for
such developments. It states that a random measure can be defined when
studying only its action against continuous functions, and the integrals with
respect to more general measurable functions can be hence defined by extension
arguments. We will state in the setting of $C(D)$ with $D$ compact444It can be
extended to the case $C_{c}(D)$ following simple arguments which we omit..

######  Theorem 2.2.

Let $D\subset\mathbb{R}^{d}$ compact. Let $M:C(D)\to
L^{2}(\Omega,\mathcal{A},\mathbb{P})$ be such that there exist measures
$m_{M}\in\mathscr{M}(D)$ and $C_{M}\in\mathscr{M}(D\times D)$ such that (2.9)
hold for every $\varphi,\phi\in C(D)$. Then, there exists a unique random
measure $\tilde{M}:\mathcal{B}(D)\to L^{2}(\Omega,\mathcal{A},\mathbb{P})$
such that

$\langle M,\varphi\rangle=\int_{D}\varphi d\tilde{M},\quad\forall\varphi\in
C(D).$ (2.10)

Of course, the mean and covariance measures of $\tilde{M}$ in Theorem 2.2 are
$m_{M}$ and $C_{M}$, respectively. This Theorem also implies that the
definition of $M$ can be extended to $\mathcal{M}_{B}(D)$ linearly and
continuously (with the supremum norm). While this Theorem is powerful, it is
not the main focus of this work, and it will be used in an auxiliary manner.
Its proof is hence refereed to Appendix A.

We make explicit two examples of random measures in the sense of Definition
2.3.

######  Example 2.1.

Gaussian White Noise $W$ over $\mathbb{R}^{d}$ is a centred Gaussian
stochastic process $(W(A))_{A\in\mathcal{B}_{B}(\mathbb{R}^{d})}$ such that
$\mathbb{C}ov(W(A),W(B))=\lambda(A\cap B)$. Note that $C_{W}$ is the measure
related to the linear functional $\psi\in
C_{c}(\mathbb{R}^{d}\times\mathbb{R}^{d})\mapsto\langle
C_{W},\psi\rangle:=\int_{\mathbb{R}^{d}}\psi(x,x)dx$. The stochastic integral
$\langle W,\varphi\rangle$ can actually be extended to $\varphi\in
L^{2}(\mathbb{R}^{d})$ since in such case $\langle
C_{W},\varphi\otimes\overline{\varphi}\rangle=\int_{\mathbb{R}^{d}}|\varphi|^{2}dx<\infty$
and thus condition (2.9) hold. $\square$

######  Example 2.2.

Consider a mean-square continuous process $Z$ over $\mathbb{R}^{d}$, and let
$\mu\in\mathscr{M}(\mathbb{R}^{d})$. Then, one can define without ambiguity
through Riemann sums ((Soong TT, 1973, Theorem 4.5.2) or (Carrizo Vergara,
2018, 3.2.1)) the application
$A\in\mathcal{B}_{B}(\mathbb{R}^{d})\mapsto\int_{A}Z(x)d\mu(x)$. One has

$\mathbb{E}\left(\int_{A}Z(x)d\mu(x)\right)=\int_{A}m_{Z}d\mu\quad;\quad\mathbb{C}ov\left(\int_{A}Z(x)d\mu(x),\int_{B}Z(x)d\mu(x)\right)=\int_{A\times
B}C_{Z}d(\mu\otimes\mu).$ (2.11)

Thus, $A\mapsto\int_{A}Zd\mu$ has mean and covariance structures determined by
measures, hence it is a random measure over $\mathbb{R}^{d}$. $\square$

Other examples which will be treated later are fractional Brownian motion with
Hurst index $H>\frac{1}{2}$ (Section 6.2) and orthogonal random measures
(Section 6.3).

##  3 Function-measure kernels

In this Section we study basic analytical properties and operations on
function-measure kernels and we give the first insights on their importance on
stochastic integration. We begin with the main definitions.

######  Definition 3.1.

Let $K:\mathbb{R}^{d}\times\mathcal{B}_{B}(\mathbb{R}^{p})\to\mathbb{R}$ be a
mapping such that $K(x,\cdot)\in\mathscr{M}(\mathbb{R}^{p})$ for all
$x\in\mathbb{R}^{d}$, and $K(\cdot,A)\in C(\mathbb{R}^{d})$ for all
$A\in\mathcal{B}_{B}(\mathbb{R}^{p})$. Then, $K$ is called a continuous-
function$-$measure kernel (from now on, cf-m kernel) over
$\mathbb{R}^{d}\times\mathbb{R}^{p}$.

In the rest of this Section $K$ will always denote a cf-m kernel over
$\mathbb{R}^{d}\times\mathbb{R}^{p}$.

######  Definition 3.2.

The total-variation kernel of $K$ is the kernel
$|K|:\mathbb{R}^{d}\times\mathcal{B}_{B}(\mathbb{R}^{p})\to\mathbb{R}^{+}$
given by $|K|(x,A):=|K(x,\cdot)|(A)$.

$|K|$ can be computed using formula (2.1). Note that if $(I_{j})_{j}$ is a
finite partition of $A$, the sum $\sum_{j}|K(\cdot,I_{j})|$ defines a
continuous function. $|K|(\cdot,A)$ is hence the supremum of a set of
continuous functions, so it is lower semi-continuous and thus measurable.
However, it is not clear if it is locally bounded, which is a regularity
property we shall ask and study in the following Section.

###  3.1 Local boundedness and partial integrals

######  Definition 3.3.

$K$ is said to be locally bounded if

$\sup_{x\in A}|K|(x,B)<\infty,\quad\forall
A\in\mathcal{B}_{B}(\mathbb{R}^{d}),B\in\mathcal{B}_{B}(\mathbb{R}^{p}).$
(3.1)

In such case we define $\|K\|_{A,B,\varepsilon}:=\sup_{x\in A}|K|(x,B)$.

The notation $\|K\|_{A,B,\varepsilon}$ is inspired from the injective norm of
a tensor product of normed spaces, but we shall not give much attention to
this. The extra requirement of local boundedness allows to gain some
regularity when operating with $K$. Note that integrals of the form
$\int_{\mathbb{R}^{p}}\psi(x,y)dK(x,\cdot)(y)$ are well-defined for every
$\psi\in\mathcal{M}_{B,c}(\mathbb{R}^{d}\times\mathbb{R}^{p})$ and every $x$.

######  Proposition 3.1.

Let $\psi\in C_{c}(\mathbb{R}^{d}\times\mathbb{R}^{p})$. If $K$ is locally
bounded, then the function
$x\mapsto\int_{\mathbb{R}^{p}}\psi(x,y)dK(x,\cdot)(y)$ is continuous with
compact support.

Proof: Suppose $\operatorname{supp}(\psi)\subset D_{d}\times D_{p}$ with
$D_{d}\subset\mathbb{R}^{d}$ and $D_{p}\subset\mathbb{R}^{p}$ compacts. If
$\left((I_{j}^{n})_{j\in J_{n}},(y_{j}^{n})_{j\in
J_{n}}\right)_{n\in\mathbb{N}}$ is a Riemann system of $D_{p}$, then we have
the point-wise limit of continuous functions

$\int_{\mathbb{R}^{p}}\psi(x,y)dK(x,\cdot)(y)=\lim_{n\to\infty}\sum_{j\in
J_{n}}\psi(x,y_{j}^{n})K(x,I_{j}^{n}),\quad\forall x\in\mathbb{R}^{d}.$ (3.2)

(3.2) is null for $x\notin D_{d}$, hence the limit has compact support. Let us
prove the convergence is uniform. Let $\epsilon>0$. Since $\psi$ is uniformly
continuous over $D_{d}\times D_{p}$, there exists $\delta>0$ such that if
$|x-u|+|y-v|\leq\delta$,
$|\psi(x,y)-\psi(u,v)|\leq\epsilon\|K\|_{D_{d},D_{p},\varepsilon}^{-1}$. If
$n$ is big enough so $\max_{j\in
J_{n}}\operatorname{diam}(I_{j}^{n})\leq\delta$, then

$\displaystyle\sup_{x\in
D_{d}}\left|\int_{D_{p}}\psi(x,y)dK(x,\cdot)(y)-\sum_{j\in
J_{n}}\psi(x,y_{j}^{n})K(x,I_{j}^{n})\right|$ $\displaystyle=\sup_{x\in
D_{d}}\left|\sum_{j\in
J_{n}}\int_{I_{j}^{n}}\left(\psi(x,y)-\psi(x,y_{j}^{n})\right)dK(x,\cdot)(y)\right|$
(3.3) $\displaystyle\leq\sup_{x\in D_{d}}\sum_{j\in
J_{n}}\underbrace{\sup_{y\in
I_{j}^{n}}|\psi(x,y)-\psi(x,y_{j}^{n})|}_{\leq\epsilon\|K\|_{D_{d},D_{p},\varepsilon}^{-1}}|K|(x,I_{j}^{n})$
$\displaystyle\leq\sup_{x\in
D_{d}}\epsilon\|K\|_{D_{d},D_{p},\varepsilon}^{-1}\underbrace{\sum_{j\in
J_{n}}|K|(x,I_{j}^{n})}_{=|K|(x,D_{p})}=\epsilon$

This proves a uniform convergence hence the limit function (3.2) is
continuous. $\blacksquare$

A similar regularity result holds when $\psi$ is independent of $x$.

######  Proposition 3.2.

Let $\phi\in\mathcal{M}_{B,c}(\mathbb{R}^{p})$. If $K$ is locally bounded,
then the function $x\mapsto\int_{\mathbb{R}^{p}}\phi(y)dK(x,\cdot)(y)$ is
continuous.

Proof: Let $D_{p}\subset\mathbb{R}^{p}$ compact with
$\operatorname{supp}(\phi)\subset D_{p}$. Since $\phi$ is bounded it can be
expressed as a uniform-over-$D_{p}$ limit of simple functions
$(\phi_{n})_{n}$. For a simple function $\phi_{n}$ it is clear that
$x\mapsto\int_{\mathbb{R}^{p}}\phi_{n}(y)dK(x,\cdot)(y)$ is continuous. Now,
one has for every compact $D\subset\mathbb{R}^{d}$,

$\sup_{x\in
D}\left|\int_{D_{p}}\phi(y)-\phi_{n}(y)dK(x,\cdot)(y)\right|\leq\|K\|_{D,D_{p},\varepsilon}\|\phi-\phi_{n}\|_{\infty}\xrightarrow[n\to\infty]{}0.$
(3.4)

The function $x\mapsto\int_{\mathbb{R}^{p}}\phi(y)dK(x,\cdot)(y)$ is hence
continuous as a uniform-on-compacts limit of continuous functions.
$\blacksquare$

Consider now for any given $\mu\in\mathscr{M}(\mathbb{R}^{d})$ the iterated
integral

$\int_{\mathbb{R}^{d}}\int_{\mathbb{R}^{p}}\psi(x,y)dK(x,\cdot)(y)d\mu(x),$
(3.5)

which is well-defined for $\psi\in C_{c}(\mathbb{R}^{d}\times\mathbb{R}^{p})$.
This induces a real linear functional from
$C_{c}(\mathbb{R}^{d}\times\mathbb{R}^{p})$ to $\mathbb{C}$.

######  Proposition 3.3.

Let $\mu\in\mathscr{M}(\mathbb{R}^{d})$. If $K$ is locally bounded, there
exists a unique
$\tilde{K}_{\mu}\in\mathscr{M}(\mathbb{R}^{d}\times\mathbb{R}^{p})$ such that

$\int_{\mathbb{R}^{d}}\int_{\mathbb{R}^{p}}\psi(x,y)dK(x,\cdot)(y)d\mu(x)=\langle\tilde{K}_{\mu},\psi\rangle,\quad\forall\psi\in
C_{c}(\mathbb{R}^{d}\times\mathbb{R}^{p}).$ (3.6)

Proof: If $\operatorname{supp}(\psi)\subset D_{d}\times
D_{p}\subset\mathbb{R}^{d}\times\mathbb{R}^{p}$, $D_{d}$ and $D_{p}$ compact,
then

$\left|\int_{\mathbb{R}^{d}}\int_{\mathbb{R}^{p}}\psi(x,y)dK(x,\cdot)(y)d\mu(x)\right|\leq|\mu|(D_{d})\|K\|_{D_{d},D_{p},\varepsilon}\|\psi\|_{\infty}.$
(3.7)

From Riesz Representation Theorem 2.1, there exists a unique measure
$\tilde{K}_{\mu}$ identifying the linear functional
$\psi\mapsto\int_{\mathbb{R}^{d}}\int_{\mathbb{R}^{p}}\psi(x,y)dK(x,\cdot)(y)d\mu(x)$
. $\blacksquare$

It follows that the iterated integral (3.5) can be extended uniquely to any
function in $\mathcal{M}_{B,c}(\mathbb{R}^{d}\times\mathbb{R}^{p})$ through
the use of the measure $\tilde{K}_{\mu}$ having hence

$\int_{\mathbb{R}^{d}}\int_{\mathbb{R}^{p}}\psi(x,y)dK(x,\cdot)(y)d\mu(x)=\int_{\mathbb{R}^{d}\times\mathbb{R}^{p}}\psi(x,y)d\tilde{K}_{\mu}(x,y),\quad\forall\psi\in\mathcal{M}_{B,c}(\mathbb{R}^{d}\times\mathbb{R}^{p}).$
(3.8)

Note that for a fixed
$\psi\in\mathcal{M}_{B,c}(\mathbb{R}^{d}\times\mathbb{R}^{p})$, the function
$x\mapsto\int_{\mathbb{R}^{p}}\psi(x,y)dK(x,\cdot)(y)$, which is compactly
supported and bounded by $\|\psi\|_{\infty}\|K\|_{D_{d},D_{p},\varepsilon}$,
is nothing but the Radon-Nikodym derivative of the measure
$A\in\mathcal{B}_{B}(\mathbb{R}^{d})\mapsto\int_{A\times\mathbb{R}^{p}}\psi(x,y)d\tilde{K}_{\mu}(x,y)$
with respect to $\mu$. It is hence in $\mathcal{M}_{B,c}(\mathbb{R}^{d})$.

Let us now study the iterated integral “in the other sense”, first integrating
over $\mathbb{R}^{d}$ and then over $\mathbb{R}^{p}$. This would mean to give
sense to the integral

$``\int_{\mathbb{R}^{p}}d\left(\int_{\mathbb{R}^{d}}\psi(x,\cdot)K(x,\cdot)d\mu(x)\right)(y)".$
(3.9)

This notation makes no further sense until now for a general $\psi$. Note that
the second component of $\psi$ acts as part of the measure integrating over
$\mathbb{R}^{p}$. One option for giving a meaning to this is to use a
disintegration of the measure $\tilde{K}_{\mu}$ in the opposite order that in
(3.8). Rather than doing this, we will consider a direct construction of the
iterated integrals in the new order, starting from simple cases of $\psi$ and
then extending. Let $\varphi\in\mathcal{M}_{B,c}(\mathbb{R}^{d})$ and
$\mu\in\mathscr{M}(\mathbb{R}^{d})$. We consider the object

$``\int_{\mathbb{R}^{d}}\varphi(x)K(x,\cdot)d\mu(x)".$ (3.10)

Intuitively, this integral should define a measure over $\mathbb{R}^{p}$.
Indeed, the application
$A\in\mathcal{B}_{B}(\mathbb{R}^{p})\mapsto\int_{\mathbb{R}^{d}}\varphi(x)K(x,A)d\mu(x)$
is clearly well-defined and additive.

######  Proposition 3.4.

If $K$ is locally bounded,
$A\mapsto\int_{\mathbb{R}^{d}}\varphi(x)K(x,A)d\mu(x)$ defines a measure over
$\mathbb{R}^{p}$.

Proof: We need to prove the $\sigma$-additivity. Consider a sequence of
bounded Borel sets of $\mathbb{R}^{p}$, $(D_{n})_{n\in\mathbb{N}}$, with
$D_{n}\searrow\emptyset$. Set the functions $f_{n}:=K(\cdot,D_{n})$.
$(f_{n})_{n\in\mathbb{N}}$ converges point-wise to $0$ and dominated by
$x\mapsto|K|(x,D_{0})$, which is locally bounded since $K$ is locally bounded.
From LDCT it follows

$\lim_{n\to\infty}\int_{\mathbb{R}^{d}}K(x,D_{n})\varphi(x)d\mu(x)=\int_{\operatorname{supp}(\varphi)}\lim_{n\to\infty}f_{n}(x)\varphi(x)d\mu(x)=0.$
(3.11)

This proves the $\sigma-$additivity of
$A\mapsto\int_{\mathbb{R}^{d}}\varphi(x)K(x,A)d\mu(x)$. $\blacksquare$

Since (3.10) is a measure, the iterated integral

$\int_{\mathbb{R}^{p}}\phi(y)d\left(\int_{\mathbb{R}^{d}}K(x,\cdot)\varphi(x)d\mu(x)\right)(y),\quad\varphi\in\mathcal{M}_{B,c}(\mathbb{R}^{d}),\phi\in\mathcal{M}_{B,c}(\mathbb{R}^{p})$
(3.12)

makes sense. By linearity, we can give a meaning to (3.9) for every $\psi$ in
the tensor product space
$\mathcal{M}_{B,c}(\mathbb{R}^{d})\otimes\mathcal{M}_{B,c}(\mathbb{R}^{p})$.
Now, we ask ourselves if such iterated integral coincides with the iterated
integral (3.5). We begin with the continuous case.

######  Proposition 3.5.

Let $\mu\in\mathscr{M}(\mathbb{R}^{d})$, $\varphi\in C_{c}(\mathbb{R}^{d})$
and $\phi\in C_{c}(\mathbb{R}^{p})$. If $K$ is locally bounded, then

$\int_{\mathbb{R}^{d}}\varphi(x)\int_{\mathbb{R}^{p}}\phi(y)dK(x,\cdot)(y)d\mu(x)=\int_{\mathbb{R}^{p}}\phi(y)d\left(\int_{\mathbb{R}^{d}}K(x,\cdot)\varphi(x)d\mu(x)\right)(y).$
(3.13)

Proof: Let $D_{d},D_{p}$ compact with $\operatorname{supp}(\varphi)\subset
D_{d}$ and $\operatorname{supp}(\phi)\subset D_{p}$. Let
$\left((I_{j}^{n})_{j\in J_{n}},(y_{j}^{n})_{j\in
J_{n}}\right)_{n\in\mathbb{N}}$ be a Riemann system of $D_{p}$. Since the
convergence $\sum_{j\in
J_{n}}\phi(y_{j}^{n})K(x,I_{j}^{n})\to\int_{K_{p}}\phi(y)dK(x,\cdot)(y)$ is
uniform over $x\in D_{d}$ (proof of Proposition 3.1), we can interchange
integral and limit symbols:

$\displaystyle\int_{D_{p}}\phi(y)d\left(\int_{D_{d}}\varphi(x)K(x,\cdot)d\mu(x)\right)(y)$
$\displaystyle=\lim_{n\to\infty}\sum_{j\in
J_{n}}\phi(y_{j}^{n})\int_{D_{d}}\varphi(x)K(x,I_{j}^{n})d\mu(x)$ (3.14)
$\displaystyle=\int_{D_{d}}\varphi(x)\lim_{n\to\infty}\sum_{j\in
J_{n}}\phi(y_{j}^{n})K(x,I_{j}^{n})d\mu(x)$
$\displaystyle=\int_{D_{d}}\varphi(x)\int_{D_{p}}\phi(y)dK(x,\cdot)(y)d\mu(x).\quad\blacksquare$

By linearity, for all $\psi\in C_{c}(\mathbb{R}^{d})\otimes
C_{c}(\mathbb{R}^{p})$ we have

$\int_{\mathbb{R}^{p}}d\left(\int_{\mathbb{R}^{d}}\psi(x,\cdot)K(x,\cdot)d\mu(x)\right)(y)=\int_{\mathbb{R}^{d}}\int_{\mathbb{R}^{p}}\psi(x,y)dK(x,\cdot)(y)d\mu(x)=\langle\tilde{K}_{\mu},\psi\rangle.$
(3.15)

But since $\tilde{K}_{\mu}$ is a measure, its definition can be extended
uniquely to any $\psi\in C_{c}(\mathbb{R}^{d}\times\mathbb{R}^{p})$ using the
density of $C_{c}(\mathbb{R}^{d})\otimes C_{c}(\mathbb{R}^{p})$ in
$C_{c}(\mathbb{R}^{d}\times\mathbb{R}^{p})$ with the uniform-on-compacts
topology. The extension of the three integrals in (3.15) to any
$\psi\in\mathcal{M}_{B,c}(\mathbb{R}^{d}\times\mathbb{R}^{p})$ has thus a
unique meaning and they all must agree, giving a complete rigorous sense to
the integral (3.9). We conclude the following Theorem.

######  Theorem 3.1 (Fubini).

If $K$ is locally bounded, then for every
$\psi\in\mathcal{M}_{B,c}(\mathbb{R}^{d}\times\mathbb{R}^{p})$ the iterated
integral (3.9) is well-defined and the equalities (3.15) hold.

As we shall see later in Section 4, the integral we have defined in (3.9) is
the first non-trivial example of self-integral we have used in this work.

###  3.2 Cross-positive-definite kernels

The most important kind of cf-m kernels over
$\mathbb{R}^{d}\times\mathbb{R}^{p}$ we use in this work are those of the type

$K_{Z,M}(x,A):=\mathbb{C}ov(Z(x),M(A)),$ (3.16)

$Z$ being a mean-square continuous stochastic process over $\mathbb{R}^{d}$
and $M$ being random measure over $\mathbb{R}^{p}$. The fact that $K_{Z,M}$ is
indeed a cf-m kernel can be deduced from the continuity of $Z$ and the
$\sigma$-additivity of $M$. The kernel $K_{Z,M}$ is called the cross-
covariance kernel between $Z$ and $M$. We shall see that such kind of kernel
has particular extra regularity properties.

We recall that the covariance function $C_{Z}$ and the covariance measure
$C_{M}$ must have a positive-definite behaviour. To be precise, $C_{Z}$ is
such that if $(x_{1},...,x_{n})\in(\mathbb{R}^{d})^{n}$ and if
$(\alpha_{1},...,\alpha_{n})\in\mathbb{C}^{n}$, then
$\sum_{j,k=1}^{n}\alpha_{j}C_{Z}(x_{j},x_{k})\overline{\alpha_{k}}\geq 0.$
Similarly, $C_{M}$ is such that if
$(E_{1},...,E_{n})\in(\mathcal{B}_{B}(\mathbb{R}^{p}))^{n}$ and if
$(\alpha_{1},...,\alpha_{n})\in\mathbb{C}^{n}$, then
$\sum_{j,k=1}^{n}\alpha_{j}C_{M}(E_{j}\times E_{k})\overline{\alpha_{k}}\geq
0$. $C_{Z}$ and $C_{M}$ describe roughly the statistical properties of $Z$ and
$M$ separately, but $K_{Z,M}$ describes, also roughly in general, their
dependence interaction. Based on this, we have the following definition and
criterion of cross-positivity.

######  Definition 3.4.

We say that $K$ is a cross-positive-definite kernel (abbreviated cross-pos-def
kernel), or that it is a cross-covariance kernel, if there exist a covariance
function $C_{Z}\in C(\mathbb{R}^{d}\times\mathbb{R}^{d})$ and a covariance
measure $C_{M}\in\mathscr{M}(\mathbb{R}^{p}\times\mathbb{R}^{p})$ such that

$\left|\sum_{j=1}^{n}\sum_{k=1}^{m}\alpha_{j}\beta_{k}K(x_{j},E_{k})\right|\leq\frac{1}{2}\left[\sum_{i,j=1}^{n}\alpha_{j}C_{Z}(x_{i},x_{j})\overline{\alpha_{k}}+\sum_{k,l=1}^{m}\beta_{k}C_{M}(E_{k}\times
E_{l})\overline{\beta_{l}}\right].$ (3.17)

for any $n,m\geq 1$,
$(\alpha_{1},...,\alpha_{n},\beta_{1},...,\beta_{m})\in\mathbb{C}^{n+m}$,
$(x_{1},...,x_{n})\in(\mathbb{R}^{d})^{n}$ and
$(E_{1},...,E_{m})\in\mathcal{B}_{B}(\mathbb{R}^{p})^{m}$.

######  Proposition 3.6.

$K$ is cross-pos-def if and only if $K(x,A)=\mathbb{C}ov(Z(x),M(A))$ for a
mean-square continuous process $Z$ over $\mathbb{R}^{d}$ and a random measure
$M$ over $\mathbb{R}^{p}$.

Proof: For the sufficiency, condition (3.17) is actually equivalent to the
non-negative-definiteness of the covariance matrix of the random vector
$(Z(x_{1}),...,Z(x_{n}),M(E_{1}),...,M(E_{m}))$. For the necessity, one can
use the classical Kolmogorov existence Theorem to construct a probability
space where $Z$ and $M$ are well-defined jointly Gaussian processes, with
covariances $C_{Z}$ and $C_{M}$ respectively, and cross-covariance $K$.
Condition (3.17) guarantees the Kolmogorov compatibility conditions when
constructing the covariance matrices of finite-dimensional Gaussian random
vectors. Details are left to the reader. $\blacksquare$

The first self-evident (we omit the proof) consequence to is the following.

######  Proposition 3.7 (Cauchy-Schwarz inequality).

If $K$ is cross-pos-def, and $C_{Z},C_{M}$ are as in Definition 3.4, then

$\left|\sum_{j=1}^{n}\sum_{k=1}^{m}\alpha_{j}\beta_{k}K(x_{j},E_{k})\right|\leq\sqrt{\sum_{i,j=1}^{n}\alpha_{j}C_{Z}(x_{i},x_{j})\overline{\alpha_{k}}}\sqrt{\sum_{k,l=1}^{m}\beta_{k}C_{M}(E_{k}\times
E_{l})\overline{\beta_{l}}},$ (3.18)

for any $n,m\geq 1$,
$(\alpha_{1},...,\alpha_{n},\beta_{1},...,\beta_{m})\in\mathbb{C}^{n+m}$,
$(x_{1},...,x_{n})\in(\mathbb{R}^{d})^{n}$ and
$(E_{1},...,E_{m})\in\mathcal{B}_{B}(\mathbb{R}^{p})^{m}$.

The first really important result on cross-pos-def kernels is the following.

######  Theorem 3.2.

If $K$ is cross-pos-def, then it is locally bounded.

Proof:  Let $A\in\mathcal{B}_{B}(\mathbb{R}^{p})$ and
$D\in\mathcal{B}_{B}(\mathbb{R}^{d})$. Then, by definition

$\sup_{x\in D}|K|(x,A)=\sup_{x\in
D}\sup_{(E_{j})_{j}\in\mathscr{P}_{F}(A)}\sum_{j}|K(x,E_{j})|,$ (3.19)

where $\mathscr{P}_{F}(A)$ denotes the collection of all finite Borel
partitions of $A$. For any $x\in D$ and any
$(E_{j})_{j}\in\mathscr{P}_{F}(A)$, we set
$\alpha_{x,j}:=\operatorname{sgn}(K(x,E_{j}))$. These coefficients satisfy
hence $|\alpha_{x,j}|=1$ and $|K(x,E_{j})|=\alpha_{x,j}K(x,E_{j})$. From
Cauchy-Schwarz inequality (3.18), we obtain

$\displaystyle\sup_{x\in D}|K|(x,A)$ $\displaystyle=\sup_{x\in
D}\sup_{(E_{j})_{j}\in\mathscr{P}_{F}(A)}\left|\sum_{j}\alpha_{j,x}K(x,E_{j})\right|$
(3.20) $\displaystyle\leq\sup_{x\in
D}\sup_{(E_{j})_{j}\in\mathscr{P}_{F}(A)}\sqrt{C_{Z}(x,x)}\sqrt{\sum_{j,k}\alpha_{j,x}C_{M}(E_{j}\times
E_{k})\alpha_{k,x}}$ $\displaystyle\leq\sup_{x\in
D}\sqrt{C_{Z}(x,x)}\sup_{(E_{j})_{j}\in\mathscr{P}_{F}(A)}\sqrt{\sum_{j,k}|C_{M}|(E_{j}\times
E_{k})}$ $\displaystyle\leq\sup_{x\in D}\sqrt{C_{Z}(x,x)}\sqrt{|C_{M}|(A\times
A)}<\infty,$

where we have used that $C_{Z}$ is continuous and $C_{M}$ is a measure.
$\blacksquare$

Results obtained in Section 3.1 apply thus for cross-pos-def kernels. New
surprises are also present in this case. As said above, it is not clear in
general if the function $|K|(\cdot,A)$ is continuous. It turns out that for
cross-pos-def kernels an even stronger regularity property holds. To present
it, we will introduce the following auxiliary definition.

######  Definition 3.5.

We define the increment kernel of $K$ as the kernel $K_{I}$ over
$(\mathbb{R}^{d}\times\mathbb{R}^{d})\times\mathbb{R}^{p}$ given by
$K_{I}((x,y),A):=K(x,A)-K(y,A)$.

If $K$ is cross-pos-def, then $K_{I}$ is also cross-pos-def. This because if
$K$ is the cross-covariance kernel between $Z$ and $M$, then $K_{I}$ is the
cross-covariance kernel between the increment process

$I_{Z}(x,y):=Z(x)-Z(y)$ (3.21)

and $M$. We will use the notation $K_{I_{Z},M}$ for the increment kernel of
$K_{Z,M}$ in the cross-covariance case. Note that the covariance function of
$I_{Z}$ is given by

$C_{I_{Z}}((x,y),(u,v)):=C_{Z}(x,u)-C_{Z}(x,v)-C_{Z}(y,u)+C_{Z}(y,v),$ (3.22)

which is continuous over
$\mathbb{R}^{d}\times\mathbb{R}^{d}\times\mathbb{R}^{d}\times\mathbb{R}^{d}$,
hence $I_{Z}$ is mean-square continuous.

######  Theorem 3.3.

Let $D\subset\mathbb{R}^{d}$ compact and
$A\in\mathcal{B}_{B}(\mathbb{R}^{p})$. If $K$ is cross-pos-def, then for every
$\epsilon>0$ there exists $\delta>0$ such that

$\sup_{\stackrel{{\scriptstyle x,y\in
D}}{{|x-y|\leq\delta}}}\sup_{B\in\mathcal{B}(A)}|K(x,\cdot)-K(y,\cdot)|(B)\leq\epsilon.$
(3.23)

In particular, the family of functions $\\{|K|(\cdot,B)\ \big{|}\
B\in\mathcal{B}(A)\\}$ is uniformly equicontinuous over $D$.

Proof:  Let $\epsilon>0$. We note $K=K_{Z,M}$. Since the increment kernel
$K_{I_{Z},M}$ is cross-pos-def, we can use Cauchy-Schwarz inequality (3.18).
If $B\in\mathcal{B}(A)$, then

$\displaystyle\sup_{x,y\in D}|K_{I_{Z},M}((x,y),\cdot)|(B)$
$\displaystyle\leq\sup_{x,y\in
D}\sqrt{C_{I_{Z}}((x,y),(x,y))}\sqrt{|C_{M}|(B\times B)}$ (3.24)
$\displaystyle\leq\sup_{x,y\in
D}\sqrt{C_{I_{Z}}((x,y),(x,y))}\sqrt{|C_{M}|(A\times A)}.$

Since $C_{I_{Z}}$ is continuous and $D$ compact, we can use the uniform
continuity of $C_{I_{Z}}$ over $D\times D\times D\times D$ and take $\delta>0$
such that if $|x-y|\leq\delta$, then
$C_{I_{Z}}((x,y),(x,y))\leq\epsilon^{2}\frac{1}{|C_{M}|(A\times A)}$. The
inequality (3.23) comes then immediately from (3.24). The uniform
equicontinuity of the family $\\{|K|(\cdot,B)\ \big{|}\ B\in\mathcal{B}(A)\\}$
comes from
$||K_{Z,M}|(x,B)-|K_{Z,M}|(y,B)|\leq|K_{Z,M}(x,\cdot)-K_{Z,M}(y,\cdot)|(B)$,
which is just the triangular inequality applied to the total-variation norm
over $B$. $\blacksquare$

We finally consider the iterated integrals studied in Section 3.1 and their
relation to $Z$ and $M$ in the case of a cross-pos-def kernel.

######  Proposition 3.8.

Let $K_{Z,M}$ be the cross-covariance kernel between $Z$ and $M$. Then, for
every
$\mu\in\mathscr{M}(\mathbb{R}^{d}),\varphi\in\mathcal{M}_{B,c}(\mathbb{R}^{d})$,
and $\phi\in\mathcal{M}_{B,c}(\mathbb{R}^{p})$, one has

$\int_{\mathbb{R}^{d}}\int_{\mathbb{R}^{p}}\varphi(x)\overline{\phi}(y)dK_{Z,M}(x,\cdot)(y)d\mu(x)=\mathbb{C}ov\left(\int_{\mathbb{R}^{d}}Z(x)\varphi(x)d\mu(x),\int_{\mathbb{R}^{p}}\phi(y)dM(y)\right).$
(3.25)

Proof: Consider first the case $\phi=\mathbbm{1}_{A}$,
$A\in\mathcal{B}_{B}(\mathbb{R}^{p})$. Let $\left((I_{j}^{n})_{j\in
J_{n}},(x_{j}^{n})_{j\in J_{n}}\right)_{n\in\mathbb{N}}$ be a Riemann system
of $\operatorname{supp}(\varphi)$. Then,

$\begin{aligned}
\mathbb{C}ov\left(\int_{\mathbb{R}^{d}}\varphi(x)Z(x)d\mu(x),M(A)\right)&=\lim_{n\to\infty}\mathbb{C}ov\left(\sum_{j\in
J_{n}}Z(x_{j}^{n})\int_{I_{j}^{n}}\varphi d\mu\ ,\ M(A)\right)\\\
&=\lim_{n\to\infty}\sum_{j\in
J_{n}}K_{Z,M}(x_{j}^{n},A)\int_{I_{j}^{n}}\varphi d\mu\\\
&=\int_{\mathbb{R}^{d}}\varphi(x)K_{Z,M}(x,A)d\mu(x).\end{aligned}.$ (3.26)

For a general $\phi\in\mathcal{M}_{B,c}(\mathbb{R}^{p})$, we use a sequence
$(\phi_{n})_{n}\subset\mathcal{M}_{B,c}(\mathbb{R}^{p})$ of simple functions
converging uniformly to $\phi$, say
$\phi_{n}=\sum_{j\in\tilde{J}_{n}}a_{j}^{n}\mathbbm{1}_{A_{j}^{n}}$. Then,

$\displaystyle\mathbb{C}ov\left(\int_{\mathbb{R}^{d}}Z(x)\varphi(x)d\mu(x),\int_{\mathbb{R}^{p}}\phi(y)dM(y)\right)$
$\displaystyle=\lim_{n\to\infty}\mathbb{C}ov\left(\int_{\mathbb{R}^{d}}Z(x)\varphi(x)d\mu(x),\sum_{j\in\tilde{J}_{n}}a_{j}^{n}M(A_{j}^{n})\right)$
(3.27)
$\displaystyle=\lim_{n\to\infty}\sum_{j\in\tilde{J}_{n}}\overline{a_{j}^{n}}\int_{\mathbb{R}^{d}}K_{Z,M}(x,A_{j}^{n})\varphi(x)d\mu(x)$
$\displaystyle=\lim_{n\to\infty}\int_{\mathbb{R}^{p}}\overline{\phi_{n}}(y)d\left(\int_{\mathbb{R}^{d}}\varphi(x)K_{Z,M}(x,\cdot)d\mu(x)\right)(y)$
$\displaystyle=\int_{\mathbb{R}^{p}}\overline{\phi}(y)d\left(\int_{\mathbb{R}^{d}}\varphi(x)K_{Z,M}(x,\cdot)d\mu(x)\right)(y).$

The final result comes from Fubini Theorem (3.1). Note that we have used
measure (3.10). $\blacksquare$

##  4 The self-integral

This Section is mainly devoted to introduce new concepts. We consider a cf-m
kernel $K$ acting over $\mathbb{R}^{d}\times\mathbb{R}^{d}$ (the same space in
both components). In such case we shall simply say that $K$ is a cf-m kernel
over $\mathbb{R}^{d}$. Let $D\in\mathcal{B}_{B}(\mathbb{R}^{d})$. We would
like to integrate $K$ with respect to itself over $D$, integrating its first
component with respect to the second one. This number, if well-defined, will
be written as

$``\int_{D}K(x,dx)".$ (4.1)

A trivial example where this peculiar idea works is the following: consider
$K$ of the form $K(x,A)=f(x)\mu(A)$, with $f\in C(\mathbb{R}^{d})$ and
$\mu\in\mathscr{M}(\mathbb{R}^{d})$. In such case, we simply do

$\int_{D}K(x,dx):=\int_{D}fd\mu.$ (4.2)

The same idea can be applied if $K$ is a finite linear combination of kernels
of this form $K=\sum_{j}f_{j}\otimes\mu_{j}$, that is, members of
$C(D)\otimes\mathscr{M}(D)$, for which the self-integral can be defined
unambiguously as

$\int_{D}K(x,dx):=\sum_{j}\int_{D}f_{j}d\mu_{j}.$ (4.3)

An analyst may have the reflex extending the definition to a completion of the
space $C(D)\otimes\mathscr{M}(D)$ with a suitable topology. This approach will
be considered in a forthcoming paper. For now, we shall use Riemann sums to
try to define the self-integral. Let $\left((I_{j}^{n})_{j\in
J_{n}},(x_{j}^{n})_{j\in J_{n}}\right)_{n\in\mathbb{N}}$ be a Riemann system
of $D$. Grounded on the definition of Riemann-Stieltjes integrals, we ask
ourselves if the limit

$\int_{D}K(x,dx):=\lim_{n\to\infty}\sum_{j\in J_{n}}K(x_{j}^{n},I_{j}^{n}),$
(4.4)

exists and if it is independent of the Riemann system. One fact that many
people know, but which may have not been expressed in this way, is that such
limit is not always well-defined.

######  Proposition 4.1.

There are cross-pos-def kernels for which the self-integral (4.4) is not
uniquely-defined.

Proof: We give as counterexample the cross-covariance kernel between a White
Noise and its primitive when in $d=1$ (Brownian motion)555This is the
classical difference between Itô and Stratonovitch integrals. Consider
$D=[0,1)$ and the kernel $K(x,A)=\lambda(A\cap[0,x])$. Then, if we use
partitions of $[0,1)$ with $n$ same-length intervals
$I_{j}^{n}=[\frac{j-1}{n},\frac{j}{n})$, $j\in J_{n}=\\{1,...,n\\}$, and if we
use the left-points of the intervals as tag-points, we obtain

$\int_{[0,1)}K(x,dx)=\lim_{n\to\infty}\sum_{j\in
J_{n}}\lambda\left([\frac{j-1}{n},\frac{j}{n})\cap[0,\frac{j-1}{n}]\right)=0.$
(4.5)

But if we use the middle-points as tag-points, we obtain

$\int_{[0,1)}K(x,dx)=\lim_{n\to\infty}\sum_{j\in
J_{n}}\lambda\left([\frac{j-1}{n},\frac{j}{n})\cap[0,\frac{2j-1}{2n}]\right)=\lim_{n\to\infty}\sum_{j\in
J_{n}}\lambda\left([\frac{j-1}{n},\frac{2j-1}{2n}]\right)=\frac{1}{2}.$ (4.6)

Hence, the limit depends upon the Riemann system employed. $\blacksquare$

The previous result inspires the next definition.

######  Definition 4.1.

$K$ is said to be self-integrable over $D$ if there exists a unique
$\ell_{D}\in\mathbb{R}$ such that for every Riemann system of $D$, say
$\left((I_{j}^{n})_{j\in J_{n}},(x_{j}^{n})_{j\in
J_{n}}\right)_{n\in\mathbb{N}}$, it holds

$\ell_{D}=\lim_{n\to\infty}\sum_{j\in J_{n}}K(x_{j}^{n},I_{j}^{n}).$ (4.7)

In such case, $\ell_{D}$ is called the self-integral of $K$ over $D$, and it
is denoted

$\int_{D}K(x,dx):=\ell_{D}.$ (4.8)

Plenty of questions may be asked concerning this concept, but we shall only
mention some of them in the conclusive Section 7.1, since in this work we
shall focus on the application of this concept in stochastic integration. In
order to contrast with the not-self-integrable situation of Proposition 4.1,
we give the following non-trivial example of well-defined self-integral.

######  Example 4.1.

Consider the weirdly-noted integral (3.9) for which we gave a rigorous meaning
through the use of the measure $\tilde{K}_{\mu}$ obtained in Proposition 3.3.
We will show that it can be conceived as a self-integral. Consider the case
$\psi\in C_{c}(\mathbb{R}^{d}\times\mathbb{R}^{p})$ with
$\operatorname{supp}(\psi)\subset D_{d}\times D_{p}$. Then, we can consider
the kernel
$K_{\psi,\mu}:\mathbb{R}^{p}\times\mathcal{B}_{B}(\mathbb{R}^{p})\to\mathbb{R}$
given by

$K_{\psi,\mu}(y,A)=\int_{\mathbb{R}^{d}}\psi(x,y)K(x,A)d\mu(x),$ (4.9)

which is a cf-m kernel if $K$ is locally bounded. Note that $K_{\psi,\mu}=0$
for $x\notin D_{p}$. We claim that $K_{\psi,\mu}$ is self-integrable over
$D_{p}$, having

$\int_{D_{p}}K_{\psi,\mu}(y,dy)=\int_{\mathbb{R}^{p}}d\left(\int_{\mathbb{R}^{d}}\psi(x,\cdot)K(x,\cdot)d\mu(x)\right).$
(4.10)

Indeed, if $\left((I_{j}^{n})_{j\in J_{n}},(y_{j}^{n})_{j\in
J_{n}}\right)_{n\in\mathbb{N}}$ is any Riemann system of $D_{p}$, then

$\displaystyle\lim_{n\to\infty}\sum_{j\in
J_{n}}K_{\psi,\mu}(y_{j}^{n},I_{j}^{n})$
$\displaystyle=\lim_{n\to\infty}\sum_{j\in
J_{n}}\int_{\mathbb{R}^{d}}\psi(x,y_{j}^{n})K(x,I_{j}^{n})d\mu(x)$ (4.11)
$\displaystyle=\int_{\mathbb{R}^{d}}\lim_{n\to\infty}\sum_{j\in
J_{n}}\psi(x,y_{j}^{n})K(x,I_{j}^{n})d\mu(x)$
$\displaystyle=\int_{\mathbb{R}^{d}}\int_{\mathbb{R}^{p}}\psi(x,y)dK(x,\cdot)(y)d\mu(x)=\int_{\mathbb{R}^{p}}d\left(\int_{\mathbb{R}^{d}}\psi(x,\cdot)K(x,\cdot)d\mu(x)\right),$

where we have used LCDT and Fubini Theorem 3.1. The Riemann system of $D_{p}$
was arbitrary, thus the kernel $K_{\psi,\mu}$ is self-integrable over $D_{p}$
and the self-integral coincides with the iterated integral (3.9). $\square$

We shall now introduce an auxiliary concept which plays an important role in
future developments.

######  Definition 4.2.

Let $K$ be a cf-m kernel over $\mathbb{R}^{d}\times\mathbb{R}^{p}$. The
second-order kernel $K^{(2)}$ is defined as the kernel over
$(\mathbb{R}^{d}\times\mathbb{R}^{d})\times(\mathbb{R}^{p}\times\mathbb{R}^{p})$
given by $K^{(2)}((x,y),E):=(K(y,\cdot)\otimes K(x,\cdot))(E)$.

The second-order kernel is a kind of cross-tensor product measure (remark the
order $(y,x)$ and not $(x,y)$ in Definition 4.2). It is clearly well-defined
as a function-measure kernel, though it is not clear if it is in general
continuous in $(x,y)$. We shall see later that this is true for cross-pos-def
kernels, which is the case we shall focus on in the following sections. For
now, we shall introduce the next Definition.

######  Definition 4.3.

Suppose $K$ is such that $K^{(2)}$ is a cf-m kernel. Let
$A,B\in\mathcal{B}_{B}(\mathbb{R}^{d})$. We say that $K^{(2)}$ is quasi-self-
integrable over $A\times B$ if there exists $\ell_{A,B}\in\mathbb{R}$ such
that for every Riemann system of $A$, $\left((A_{j}^{n})_{j\in
J^{A}_{n}},(a_{j}^{n})_{j\in J^{A}_{n}}\right)_{n\in\mathbb{N}}$, and every
Riemann system of $B$, $\left((B_{j}^{n})_{j\in J^{B}_{n}},(b_{j}^{n})_{j\in
J^{B}_{n}}\right)_{n\in\mathbb{N}}$, the following double-limit holds

$\lim_{(n,m)\to\infty}\sum_{j\in J_{n}}\sum_{k\in
J^{B}_{m}}K^{(2)}((a_{j}^{n},b_{k}^{m}),A_{j}^{n}\times
B_{k}^{m})=\ell_{A,B}.$ (4.12)

In such case, we call $\ell_{A,B}$ the quasi-self-integral of $K^{(2)}$ over
$A\times B$, and we denote it

$\tilde{\int}_{A\times B}K^{(2)}(x,dx):=\ell_{A,B}.$ (4.13)

Definition 4.3 is weaker than the definition of a proper self-integral of
$K^{(2)}$ since it only considers a specific class of Riemann system:
partitions constructed with products $A_{j}^{n}\times B_{k}^{m}$. In addition,
Definition 4.3 only considers integration over sets of the form $A\times B$,
rather than general bounded Borel sets of
$\mathbb{R}^{d}\times\mathbb{R}^{d}$. It follows that if $K^{(2)}$ is self-
integrable over $E\in\mathcal{B}_{B}(\mathbb{R}^{d}\times\mathbb{R}^{p})$,
with $E=A\times B$, then it is quasi-self-integrable over $E$. In such case,
the quasi-self-integral and the self-integral coincide. It is not clear if
quasi-self-integrability implies self-integrability.

##  5 Self-integrability and stochastic integration

In this Section we show that the concept of self-integrability plays a central
role in the definition of stochastic integrals. All along this Section, $Z$ is
a centred mean-square continuous stochastic process over $\mathbb{R}^{d}$, $M$
is a centred random measure over $\mathbb{R}^{d}$, and
$D\in\mathcal{B}_{B}(\mathbb{R}^{d})$.

###  5.1 A first approach

The first simple question is whether we can define the stochastic integral

$``\int_{D}Z(x)dM(x)".$ (5.1)

Analogously to the deterministic case, we study the limit of the random
variables

$\lim_{n\to\infty}\sum_{j\in J_{n}}Z(x_{j}^{n})M(I_{j}^{n}),$ (5.2)

being $\left((I_{j}^{n})_{j\in J_{n}},(x_{j}^{n})_{j\in
J_{n}}\right)_{n\in\mathbb{N}}$ a Riemann system of $D$. We introduce hence
the following definition.

######  Definition 5.1.

We say that the stochastic integral of $Z$ with respect to $M$ over $D$ is
uniquely-defined if the limit (5.2) exists in the sense of
$L^{p}(\Omega,\mathcal{A},\mathbb{P})$ for a specified $p\in[1,\infty]$ and
that it does not depend upon the Riemann system of $D$ used. In such case we
denote

$\int_{D}ZdM:=\int_{D}Z(x)dM(x):=\lim_{n\to\infty}\sum_{j\in
J_{n}}Z(x_{j}^{n})M(I_{j}^{n}).$ (5.3)

In this work we will use $p=1$ or $p=2$. Note that the random variables in
(5.2) are always in $L^{1}(\Omega,\mathcal{A},\mathbb{P})$, hence it does make
sense to ask if they converge in the $L^{1}(\Omega,\mathcal{A},\mathbb{P})$
sense. However, in some cases the convergence in
$L^{2}(\Omega,\mathcal{A},\mathbb{P})$ is easier to analyse (such convergence
implies of course the $L^{1}(\Omega,\mathcal{A},\mathbb{P})$ convergence). We
shall always specify the sense of the convergence we are considering.

The first link between self-integrability and stochastic integrals is given by
the next Proposition.

######  Proposition 5.1.

If $\int_{D}ZdM$ is uniquely-defined in the
$L^{p}(\Omega,\mathcal{A},\mathbb{P})$ sense for any $p\in[1,\infty]$, then
the cross-covariance kernel $K_{Z,M}$ is self-integrable over $D$.

Proof: If the limit (5.2) converges to some random variable in
$L^{p}(\Omega,\mathcal{A},\mathbb{P})$, then the expectations also converge to
some real limit $\ell\in\mathbb{R}$. Hence

$\lim_{n\to\infty}\mathbb{E}\left(\sum_{j\in
J_{n}}Z(x_{j}^{n})M(I_{j}^{n})\right)=\lim_{n\to\infty}\sum_{j\in
J_{n}}K_{Z,M}(x_{j}^{n},I_{j}^{n})=\ell.$ (5.4)

Since this holds for every Riemann system of $D$, $K_{Z,M}$ must be self-
integrable over $D$. $\blacksquare$

The last Proposition does not only tell us that we need self-integrability of
$K_{Z,M}$ in order to define uniquely the stochastic integral, but it also
gives us the expectation formula

$\mathbb{E}\left(\int_{D}ZdM\right)=\int_{D}K_{Z,M}(x,dx).$ (5.5)

There are simple cases where the stochastic integral is uniquely-defined and
hence the associated cross-covariance kernel is self-integrable. We list three
important examples which will be used further.

######  Example 5.1.

Consider $Z$ and $M$ independent. Let us prove the Riemann sums (5.2) form a
Cauchy sequence in $L^{2}(\Omega,\mathcal{A},\mathbb{P})$. Using the measure
structure of $M$, we right the Cauchy gaps as

$\sum_{j\in J_{n}}Z(x_{j}^{n})M(I_{j}^{n})-\sum_{k\in
J_{m}}Z(x_{k}^{m})M(I_{k}^{m})=\sum_{j\in J_{n}}\sum_{k\in
J_{m}}(Z(x_{j}^{n})-Z(x_{k}^{m}))M(I_{j}^{n}\cap I_{k}^{m}).$ (5.6)

Taking $\mathbb{E}\left[(\cdot)^{2}\right]$ of this, one obtains the multi-sum

$\sum_{j,j^{\prime}\in J_{n}}\sum_{k,k^{\prime}\in
J_{m}}\mathbb{E}\left[(Z(x_{j}^{n})-Z(x_{k}^{m}))(Z(x_{j^{\prime}}^{n})-Z(x_{k^{\prime}}^{m}))M(I_{j}^{n}\cap
I_{k}^{m})M(I_{j^{\prime}}^{n}\cap I_{k^{\prime}}^{m})\right].$ (5.7)

From the independence condition and using the increment process $I_{Z}$ (See
(3.21) and (3.22)), we obtain

$\sum_{j,j^{\prime}\in J_{n}}\sum_{k,k^{\prime}\in
J_{m}}C_{I_{Z}}((x_{j}^{n},x_{k}^{m}),(x_{j^{\prime}}^{n},x_{k^{\prime}}^{m}))C_{M}((I_{j}^{n}\cap
I_{k}^{m})\times(I_{j^{\prime}}^{n}\cap I_{k^{\prime}}^{m})).$ (5.8)

Now we consider the uniform continuity of $C_{I_{Z}}$ over $\overline{D}^{4}$.
Note that $C_{I_{Z}}((x,x),(y,y))=0$. Hence, for every $\epsilon>0$ there
exists $\delta>0$ such that if $|x-u|+|y-v|<\delta$, then
$|C_{I_{Z}}((x,u),(y,v))|<\epsilon$. Consider hence $n_{0}$ such that if
$n,m\geq n_{0}$,
$\operatorname{diam}(I_{j}^{n})+\operatorname{diam}(I_{k}^{m})<\frac{\delta}{2}$
for every $(j,k)\in J_{n}\times J_{m}$. Thus, we claim that if $n,m\geq n_{0}$
then for every $j,j\in J_{n}$ and every $k,k^{\prime}\in J_{m}$ one has

$|C_{I_{Z}}((x_{j}^{n},x_{k}^{m}),(x_{j^{\prime}}^{n},x_{k^{\prime}}^{m}))C_{M}((I_{j}^{n}\cap
I_{k}^{m})\times(I_{j^{\prime}}^{n}\cap
I_{k^{\prime}}^{m}))|<\epsilon|C_{M}|((I_{j}^{n}\times
I_{k}^{m})\cap(I_{j}^{n}\times I_{k}^{m})).$ (5.9)

This holds since if $I_{j}^{n}\cap I_{k}^{m}=\emptyset$ then (5.9) equals $0$,
and if $I_{j}^{n}\cap I_{k}^{m}\neq\emptyset$, using
$\operatorname{diam}(I_{j}^{n})+\operatorname{diam}(I_{k}^{m})<\frac{\delta}{2}$,
we conclude $|x_{j}^{n}-x_{k}^{m}|<\frac{\delta}{2}$, and the same idea can be
used for $|x_{j^{\prime}}^{n}-x_{k^{\prime}}^{m}|$. Hence
$|C_{I_{Z}}((x_{j}^{n},x_{k}^{m}),(x_{j^{\prime}}^{n},x_{k^{\prime}}^{m}))|<\epsilon$.
It follows that (5.8) is bounded by

$\sum_{j,j^{\prime}\in J_{n}}\sum_{k,k^{\prime}\in
J_{m}}\epsilon|C_{M}|((I_{j}^{n}\cap I_{k}^{m})\times(I_{j^{\prime}}^{n}\cap
I_{k^{\prime}}^{m}))=\epsilon|C_{M}|(D\times D).$ (5.10)

The Riemann sums form thus a Cauchy sequence in
$L^{2}(\Omega,\mathcal{A},\mathbb{P})$, converging to a random variable with
finite variance. If
$\left((D_{j}^{n})_{j\in\tilde{J}_{n}},(y_{j}^{n})_{j\in\tilde{J}_{n}}\right)_{n\in\mathbb{N}}$
is another Riemann system of $D$, we can consider the gap

$\sum_{j\in
J_{n}}Z(x_{j}^{n})M(I_{j}^{n})-\sum_{k\in\tilde{J}_{n}}Z(y_{k}^{n})M(D_{k}^{n})=\sum_{j\in
J_{n}}\sum_{k\in\tilde{J}_{n}}(Z(x_{j}^{n})-Z(y_{k}^{n}))M(I_{j}^{n}\cap
D_{k}^{n}).$ (5.11)

An analogue argument can be used to prove that the
$L^{2}(\Omega,\mathcal{A},\mathbb{P})$-norm of (5.11) goes to $0$ as
$n\to\infty$. The limit is hence independent of the Riemann system chosen, so
the stochastic integral $\int_{D}ZdM$ is uniquely-defined in the sense of
$L^{2}(\Omega,\mathcal{A},\mathbb{P})$. Of course in this case $K_{Z,M}=0$,
which is trivially self-integrable. $\square$

######  Example 5.2.

Suppose $Z$ or $M$ has finite-dimensional image in
$L^{2}(\Omega,\mathcal{A},\mathbb{P})$. For instance, suppose $Z$ is of the
form

$Z=\sum_{\alpha\leq m}X_{\alpha}f_{\alpha},$ (5.12)

where $(X_{\alpha})_{\alpha\leq m}\subset
L^{2}(\Omega,\mathcal{A},\mathbb{P})$ and $(f_{\alpha})_{\alpha\leq m}\subset
C(\mathbb{R}^{d})$. Then, the stochastic integral $\int_{D}ZdM$ is uniquely-
defined since for every Riemann system $\left((I_{j}^{n})_{j\in
J_{n}},(x_{j}^{n})_{j\in J_{n}}\right)_{n\in\mathbb{N}}$ we have

$\int_{D}ZdM=\lim_{n\to\infty}\sum_{j\in
J_{n}}Z(x_{j}^{n})M(I_{j}^{n})=\sum_{\alpha\leq
m}X_{\alpha}\lim_{n\to\infty}\sum_{j\in
J_{n}}f_{\alpha}(x_{j}^{n})M(I_{j}^{n})=\sum_{\alpha\leq
m}X_{\alpha}\int_{D}f_{\alpha}(x)dM(x).$ (5.13)

In this case the cross-covariance kernel $K_{Z,M}$ has a simple form. Let
$\sigma_{X_{\alpha}}^{2}=\mathbb{V}ar(X_{\alpha})>0$, and let
$\nu_{\alpha}(A):=\sigma_{X_{\alpha}}^{-1}\mathbb{E}(X_{\alpha}M(A))$. Form
the $\sigma$-additivity of $M$, $\nu_{\alpha}$ defines a measure over
$\mathbb{R}^{d}$. Thus,

$K_{Z,M}(x,A)=\mathbb{E}(Z(x)M(A))=\sum_{\alpha\leq
m}\sigma_{X_{\alpha}}f_{\alpha}(x)\nu_{\alpha}(A).$ (5.14)

$K_{Z,M}$ is hence a linear combination of tensors. It is then self-integrable
over $D$. Analogously, this same situation is present when the random measure
$M$ is such that

$M=\sum_{\beta\leq m}Y_{\beta}\mu_{\beta},$ (5.15)

with $(Y_{\beta})_{\beta\leq m}\in L^{2}(\Omega,\mathcal{A},\mathbb{P})$, and
$(\mu_{\beta})_{\beta\leq m}\in\mathscr{M}(\mathbb{R}^{d})$. We shall explode
more in detail this kind of construction in a forthcoming paper. $\square$

######  Example 5.3.

Consider the case where $M$ is absolutely continuous with respect to a real
deterministic measure $\mu\in\mathscr{M}(\mathbb{R}^{d})$ in the sense that it
exists an enough-regular process ($\mu$-almost everywhere mean-square
continuous-locally-bounded, for instance) $U$ such that

$M(A)=\int_{A}U(y)d\mu(y),\quad\forall A\in\mathcal{B}_{B}(\mathbb{R}^{d}).$
(5.16)

The Riemann sums approaching the stochastic integrals are given by

$\sum_{j\in J_{n}}Z(x_{j}^{n})\int_{I_{j}^{n}}U(y)d\mu(y)$ (5.17)

When analysing the $L^{1}(\Omega,\mathcal{A},\mathbb{P})$-norm of the Cauchy
gap (5.6) we obtain

$\displaystyle\mathbb{E}\Big{(}\Big{|}\sum_{j\in J_{n}}\sum_{k\in
J_{m}}(Z(x_{j}^{n}$ $\displaystyle)-Z(x_{k}^{m}))\int_{I_{j}^{n}\cap
I_{k}^{m}}U(y)d\mu(y)\Big{|}\Big{)}\leq\sum_{j\in J_{n}}\sum_{k\in
J_{m}}\mathbb{E}\Big{(}|Z(x_{j}^{n})-Z(x_{k}^{m})|\Big{|}\int_{I_{j}^{n}\cap
I_{k}^{m}}U(y)d\mu(y)\Big{|}\Big{)}$ (5.18) $\displaystyle\leq\sum_{j\in
J_{n}}\sum_{k\in
J_{m}}\sqrt{\mathbb{E}(|Z(x_{j}^{n})-Z(x_{k}^{m})|^{2})}\sqrt{\mathbb{E}\Big{(}\Big{|}\int_{I_{j}^{n}\cap
I_{k}^{m}}U(y)d\mu(y)\Big{|}^{2}\Big{)}}$ $\displaystyle=\sum_{j\in
J_{n}}\sum_{k\in
J_{m}}\sqrt{C_{I_{Z}}((x_{j}^{n},x_{k}^{m}),(x_{j}^{n},x_{k}^{m}))}\sqrt{\int_{I_{j}^{n}\cap
I_{k}^{m}}\int_{I_{j}^{n}\cap I_{k}^{m}}C_{U}(y,z)d\mu(y)d\mu(z)},$

where $C_{U}$ is the covariance function of $U$. We can use the Cauchy-Schwarz
inequality for $C_{U}$ to obtain

$\displaystyle\int_{I_{j}^{n}\cap I_{k}^{m}}\int_{I_{j}^{n}\cap
I_{k}^{m}}C_{U}(y,z)d\mu(y)d\mu(z)$ $\displaystyle\leq\int_{I_{j}^{n}\cap
I_{k}^{m}}\int_{I_{j}^{n}\cap
I_{k}^{m}}\sqrt{C_{U}(y,y)}\sqrt{C_{U}(z,z)}d|\mu|(y)d|\mu|(z)$ (5.19)
$\displaystyle=\left(\int_{I_{j}^{n}\cap
I_{k}^{m}}\sqrt{C_{U}(y,y)}d|\mu|(y)\right)^{2}.$

With the same arguments as in (5.9), for any $\epsilon>0$ and for $n,m$ big
enough (5.18) is bounded by

$\sum_{j\in J_{n}}\sum_{k\in J_{m}}\epsilon\int_{I_{j}^{n}\cap
I_{k}^{m}}\sqrt{C_{U}(y,y)}d|\mu|(y)=\epsilon\int_{D}\sqrt{C_{U}(y,y)}d|\mu|(y).$
(5.20)

Thus the Riemann sums form a Cauchy sequence on
$L^{1}(\Omega,\mathcal{A},\mathbb{P})$ hence they converge. A similar
algebraic computation assures that the limit does not depend upon the Riemann
system chosen. The limit in the sense of
$L^{1}(\Omega,\mathcal{A},\mathbb{P})$ is hence $\int_{D}ZdM$. In this case
the cross-covariance kernel $K_{Z,M}$ has the form

$K_{Z,M}(x,A)=\int_{A}C_{Z,U}(x,y)d\mu(y),$ (5.21)

where $C_{Z,U}$ is the cross-covariance function between $Z$ and $U$. The
self-integral of $K_{Z,M}$ is given by

$\int_{D}K_{Z,M}(x,dx)=\int_{D}C_{Z,U}(x,x)d\mu(x).$ (5.22)

Note that with this analysis we can define
$\int_{D}Z(x)U(x)d\mu(x):=\int_{D}ZdM$, which should coincide with other
intuitive manners of constructing such integral.$\square$

###  5.2 Gaussian case: necessary and sufficient conditions

Consider the case where $Z$ and $M$ are jointly Gaussian. That is, for every
$(x_{1},...,x_{n})\in(\mathbb{R}^{d})^{n}$ and every
$(A_{1},...,A_{m})\in(\mathcal{B}_{B}(\mathbb{R}^{d}))^{m}$, the vector
$(Z(x_{1}),...,Z(x_{n}),M(A_{1}),...,M(A_{m}))$ is a Gaussian vector. In such
case, $C_{Z},C_{M}$ and $K_{Z,M}$ describe entirely the finite-dimensional
laws of the random vectors we can construct with $Z$ and $M$. Concerning the
unique definition of $\int_{D}ZdM$, note that in Examples 5.2 and 5.3 there
are extra regularity conditions on $Z$ or $M$ playing a role. In Example 5.1
there is no regularity requirement on $Z$ or $M$, but a strong independence
condition is used. It is then expected that conditions for the definition of
$\int_{D}ZdM$, without asking more regularity in $Z$ or $M$, will rely
entirely on the dependence between $Z$ and $M$, which is fully determined by
$K_{Z,M}$. The main result of this work is Theorem 5.1 which shows that this
intuition is correct. For introducing such Theorem we will need Definition 4.3
applied to the kernel $K_{Z,M}$. The next Proposition will helps us for that.

######  Proposition 5.2.

If $K$ is cross-pos-def, then $K^{(2)}$ is also cross-pos-def.

Proposition 5.2 and Theorem 3.3 imply that $K_{Z,M}^{(2)}$ is a cf-m kernel,
hence Definition 4.3 of the quasi-self-integral can be applied to it. In order
to prove Proposition 5.2 we shall use the following Lemma which introduces the
tensor product of independent random measures.

######  Lemma 5.1.

Let $M_{1}$ and $M_{2}$ be two independent random measures over
$\mathbb{R}^{d}$ and $\mathbb{R}^{p}$ respectively. Then, there exists a
unique random measure $M_{1}\otimes M_{2}$ over
$\mathbb{R}^{d}\times\mathbb{R}^{p}$ satisfying

$(M_{1}\otimes M_{2})(A\times B)=M_{1}(A)M_{2}(B),\quad\forall
A\in\mathcal{B}_{B}(\mathbb{R}^{d}),B\in\mathcal{B}_{B}(\mathbb{R}^{p}).$
(5.23)

Proof of Lemma 5.1: We define the random measure $M_{1}\otimes M_{2}$ acting
over functions in $C_{c}(\mathbb{R}^{d}\times\mathbb{R}^{p})$ through the
iterated stochastic integrals

$\langle M_{1}\otimes
M_{2},\psi\rangle:=\int_{\mathbb{R}^{d}}\int_{\mathbb{R}^{p}}\psi(x,y)dM_{2}(y)dM_{1}(x),\quad\forall\psi\in
C_{c}(\mathbb{R}^{d}\times\mathbb{R}^{p}).$ (5.24)

Note that the stochastic process
$x\mapsto\int_{\mathbb{R}^{p}}\psi(x,y)dM_{2}(y)$ is mean-square continuous
and independent of $M_{1}$, and hence we are in the case of Example 5.1 of a
uniquely-defined stochastic integral. The application $\psi\mapsto\langle
M_{1}\otimes M_{2},\psi\rangle$ has null expectation (Eq. (5.5)) and
covariance

$\mathbb{C}ov(\langle M_{1}\otimes M_{2},\psi_{1}\rangle,\langle M_{1}\otimes
M_{2},\psi_{2}\rangle)=\int_{\mathbb{R}^{d}\times\mathbb{R}^{d}}\int_{\mathbb{R}^{p}\times\mathbb{R}^{p}}\psi_{1}(x,u)\overline{\psi_{2}}(y,v)dC_{M_{2}}(u,v)dC_{M_{1}}(x,y),$
(5.25)

which defines a measure over
$(\mathbb{R}^{d}\times\mathbb{R}^{p})\times(\mathbb{R}^{d}\times\mathbb{R}^{p})$
since $C_{M_{1}}$ and $C_{M_{2}}$ are measures. From Theorem 2.2,
$M_{1}\otimes M_{2}$ defines a unique random measure over
$\mathbb{R}^{d}\times\mathbb{R}^{p}$ satisfying (5.23) (use
$\psi=\mathbbm{1}_{A}\otimes\mathbbm{1}_{B}$). $\blacksquare$

Proof of Proposition 5.2: Let $Z_{1},Z_{2}$ be centred mean-square continuous
processes over $\mathbb{R}^{d}$ and let $M_{1}$ and $M_{2}$ be centred random
measures over $\mathbb{R}^{p}$ such that $(Z_{1},M_{1})$ is independent of
$(Z_{2},M_{2})$ and such that $K$ is the cross-covariance kernel of both
pairs, $K=K_{Z_{1},M_{1}}=K_{Z_{2},M_{2}}$. Consider thus the centred and
mean-square continuous process $U(x,y):=Z_{2}(x)Z_{1}(y)$ and the random
measure $M_{1}\otimes M_{2}$. Then,

$\displaystyle\mathbb{E}\big{(}U(x,y)(M_{1}\otimes M_{2})(A\times B)\big{)}$
$\displaystyle=\mathbb{E}\big{(}Z_{1}(y)M_{1}(A)\big{)}\mathbb{E}\big{(}Z_{2}(x)M_{2}(B)\big{)}$
(5.26) $\displaystyle=K(y,A)K(x,B)$ $\displaystyle=K^{(2)}((x,y),A\times B),$

for every $A\in\mathcal{B}_{B}(\mathbb{R}^{d})$,
$B\in\mathcal{B}_{B}(\mathbb{R}^{p})$. Thus, $K^{(2)}$ is nothing but the
cross-covariance kernel between $U$ and $M_{1}\otimes M_{2}$, hence it is
cross-pos-def. $\blacksquare$

We state now Theorem 5.1.

######  Theorem 5.1.

If $Z$ and $M$ are jointly Gaussian, then $\int_{D}ZdM$ is uniquely-defined in
a $L^{2}(\Omega,\mathcal{A},\mathbb{P})$ sense if and only if $K_{Z,M}$ is
self-integrable over $D$ and $K^{(2)}_{Z,M}$ is quasi-self-integrable over
$D\times D$.

We shall use next two simple Lemmas, proper of basic Probability Theory.

######  Lemma 5.2.

If $(X_{1},X_{2},X_{3},X_{4})$ is a centred Gaussian vector, then

$\mathbb{C}ov(X_{1}X_{2},X_{3}X_{4})=\mathbb{E}(X_{1}X_{3})\mathbb{E}(X_{2}X_{4})+\mathbb{E}(X_{1}X_{4})\mathbb{E}(X_{2}X_{3}).$
(5.27)

Proof of Lemma 5.2: This is a particular case of Isserlis-Wick Theorem
(Isserlis, 1918; Wick, 1950). $\blacksquare$

######  Lemma 5.3.

Let $(X_{n})_{n},(Y_{n})_{n}$ be two sequences in
$L^{2}(\Omega,\mathcal{A},\mathbb{P})$. Then,

  1. (i)

$(X_{n})_{n}$ converges in $L^{2}(\Omega,\mathcal{A},\mathbb{P})$ if and only
if $\displaystyle\lim_{n\to\infty}\mathbb{E}(X_{n})$ and
$\displaystyle\lim_{(n,m)\to\infty}\mathbb{C}ov(X_{n},X_{m})$ exist.

  2. (ii)

If $X_{n}\stackrel{{\scriptstyle L^{2}(\Omega)}}{{\to}}X$, then
$Y_{n}\stackrel{{\scriptstyle L^{2}(\Omega)}}{{\to}}X$ if and only if
$\displaystyle\lim_{n\to\infty}\mathbb{E}(Y_{n})=\displaystyle\lim_{n\to\infty}\mathbb{E}(X_{n})$
and
$\displaystyle\lim_{(n,m)\to\infty}\mathbb{C}ov(Y_{n},Y_{m})=\displaystyle\lim_{(n,m)\to\infty}\mathbb{C}ov(X_{n},X_{m})=\displaystyle\lim_{(n,m)\to\infty}\mathbb{C}ov(Y_{n},X_{m})$.

Proof of Lemma 5.3: For i, the necessity is clear in the case of the mean, and
for the covariance it is enough to consider
$\mathbb{C}ov(X_{n},X_{m})=\frac{1}{2}\left(\mathbb{V}ar(X_{n})+\mathbb{V}ar(X_{m})-\mathbb{V}ar(X_{n}-X_{m})\right)$,
which converges to the variance of the limit as $(n,m)\to\infty$. For the
sufficiency, the convergence of the double-limit of the covariances implies in
particular (using $n=m$) the convergence of the variances
$\mathbb{V}ar(X_{n})$, whose limit coincide with
$\lim_{(n,m)\to\infty}\mathbb{C}ov(X_{n},X_{m})$. Using
$\mathbb{E}(|X_{n}-X_{m}|^{2})=\mathbb{V}ar(X_{n})+\mathbb{V}ar(X_{m})-2\mathbb{C}ov(X_{n},X_{m})+|\mathbb{E}(X_{n})-\mathbb{E}(X_{m})|^{2}$,
it follows that $(X_{n})_{n}$ is a Cauchy sequence hence convergent.

For ii the necessity in the case of the means is clear as well as the equality
$\displaystyle\lim_{(n,m)\to\infty}\mathbb{C}ov(Y_{n},Y_{m})=\displaystyle\lim_{(n,m)\to\infty}\mathbb{C}ov(X_{n},X_{m})$
(the limit is $\mathbb{V}ar(X)$), and the equality involving
$\mathbb{C}ov(Y_{n},X_{m})$ follows using
$\displaystyle\lim_{(n,m)\to\infty}\mathbb{E}(|X_{m}-Y_{n}|^{2})=0$ and
developing. For the sufficiency, from i $(Y_{n})_{n}$ is convergent, and
developing $\mathbb{E}(|Y_{n}-X_{n}|^{2})$ one proves that it goes to $0$ as
$n\to\infty$, hence $(Y_{n})_{n}$ and $(X_{n})_{n}$ have the same limit.
Details are left to the reader. $\blacksquare$

Proof of Theorem 5.1: Let $\left((I_{j}^{n})_{j\in J_{n}},(x_{j}^{n})_{j\in
J_{n}}\right)_{n\in\mathbb{N}}$,
$\left((E_{j}^{n})_{j\in\tilde{J}_{n}},(y_{j}^{n})_{j\in\tilde{J}_{n}}\right)_{n\in\mathbb{N}}$
be two Riemann systems of $D$. The covariance between the Riemann sums
approaching $\int_{D}ZdM$ is (Lemma 5.2)

$\displaystyle\mathbb{C}ov\Big{(}$ $\displaystyle\sum_{j\in
J_{n}}Z(x_{j}^{n})M(I_{j}^{n}),\sum_{k\in\tilde{J}_{m}}Z(y_{k}^{m})M(E_{k}^{m})\Big{)}$
(5.28) $\displaystyle=\sum_{j\in
J_{n}}\sum_{k\in\tilde{J}_{m}}\mathbb{E}(Z(x_{j}^{n})Z(y_{k}^{m}))\mathbb{E}(M(I_{j}^{n})M(E_{k}^{m}))+\mathbb{E}(Z(x_{j}^{n})M(E_{k}^{m}))\mathbb{E}(Z(y_{k}^{m})M(I_{j}^{n}))$
$\displaystyle=\sum_{j\in
J_{n}}\sum_{k\in\tilde{J}_{m}}C_{Z}(x_{j}^{n},y_{k}^{m})C_{M}(I_{j}^{n}\times
E_{k}^{m})+K_{Z,M}(x_{j}^{n},E_{k}^{m})K_{Z,M}(y_{k}^{m},I_{j}^{n})$
$\displaystyle=\sum_{j\in
J_{n}}\sum_{k\in\tilde{J}_{m}}C_{Z}(x_{j}^{n},y_{k}^{m})C_{M}(I_{j}^{n}\times
E_{k}^{m})+\sum_{j\in
J_{n}}\sum_{k\in\tilde{J}_{m}}K_{Z,M}^{(2)}((x_{j}^{n},y_{k}^{m}),I_{j}^{n}\times
E_{k}^{m}).$

Note now that

$\lim_{(n,m)\to\infty}\sum_{j\in
J_{n}}\sum_{k\in\tilde{J}_{m}}C_{Z}(x_{j}^{n},y_{k}^{m})C_{M}(I_{j}^{n}\times
E_{k}^{m})=\int_{D\times D}C_{Z}dC_{M},$ (5.29)

which holds since the concerned double-sum is nothing but a Riemann
approximation of the integral $\int_{D\times D}C_{Z}dC_{M}$.666If the reader
needs more details, one can use the uniform continuity of $C_{Z}$ over
$\overline{D}\times\overline{D}$ to prove that the function $\sum_{j\in
J_{n}}\sum_{k\in\tilde{J}_{m}}C_{Z}(x_{j}^{n},y_{k}^{m})\mathbbm{1}_{I_{j}^{n}\times
E_{k}^{m}}$ converges uniformly to $C_{Z}$ over $D\times D$ as
$(n,m)\to\infty$ and then conclude. It follows that

$\displaystyle\lim_{(n,m)\to\infty}\mathbb{C}ov\Big{(}\sum_{j\in
J_{n}}Z(x_{j}^{n})$ $\displaystyle M(I_{j}^{n})\
,\sum_{k\in\tilde{J}_{m}}Z(y_{k}^{m})M(E_{k}^{m})\Big{)}$ (5.30)
$\displaystyle=\int_{D\times D}C_{Z}dC_{M}+\lim_{(n,m)\to\infty}\sum_{j\in
J_{n}}\sum_{k\in\tilde{J}_{m}}K_{Z,M}^{(2)}((x_{j}^{n},y_{k}^{m}),I_{j}^{n}\times
E_{k}^{m}).$

We conclude hence the equivalence of the convergences of both double-limits in
(5.30) (if one converges then the other does and vice-versa). Concerning
expectations, using (5.4) it follows that the self-integrability of $K_{Z,M}$
is equivalent to the convergence of the expectations of the Riemann sums to a
limit independent of the chosen Riemann system.

It follows that if $\int_{D}ZdM$ is uniquely-defined, from Lemma 5.3 the left
side in (5.30) converges to $\mathbb{V}ar(\int_{D}ZdM)$, hence the right side
converges to the same value independently of both Riemann systems chosen, thus
$K_{Z,M}^{(2)}$ is quasi-self-integrable. Conversely, if $K_{Z,M}^{(2)}$ is
quasi-self-integrable then the double-limit at left side in (5.30) converges
to the same value independently of the Riemann systems. Applying then Lemma
5.3 point i, one proves using equal Riemann systems $\left((I_{j}^{n})_{j\in
J_{n}},(x_{j}^{n})_{j\in
J_{n}}\right)_{n\in\mathbb{N}}=\left((E_{j}^{n})_{j\in\tilde{J}_{n}},(y_{j}^{n})_{j\in\tilde{J}_{n}}\right)_{n\in\mathbb{N}}$
in equation (5.30) that the Riemann sums associated to such system converge in
$L^{2}(\Omega,\mathcal{A},\mathbb{P})$ to a random variable. The independence
of the Riemann system follows hence when using two different Riemann systems
in (5.30) and then applying Lemma 5.3 point ii, concluding thus that the
corresponding Riemann sums converge to the same random variable. It follows
hence that $\int_{D}ZdM$ is uniquely-defined as a limit in
$L^{2}(\Omega,\mathcal{A},\mathbb{P})$. $\blacksquare$

It is worth mention that Theorem 5.1 does not necessarily hold in a non-
Gaussian context. Consider the following counter-example: $D=[0,1)$,
$Z(t)=B^{2}(t)$, with $B$ a Gaussian Brownian motion and $M=B^{\prime}$ the
distributional derivative of $B$, which is a Gaussian White Noise. Then, we
can use the explicit link between the Itô Integral $\int_{[0,1)}B^{2}(t)dB(t)$
and the Stratonovich integral $\int_{[0,1)}B^{2}(t)\circ dB(t)$:

$\int_{[0,1)}B^{2}(t)\circ
dB(t)-\int_{[0,1)}B^{2}(t)dB(t)=\int_{[0,1)}B(t)dB(t),$ (5.31)

where we have used the traditional notation for Itô and Stratonovich integrals
(Kloeden Platen, 1995, Equation 5.5). The Itô integral $\int_{[0,1)}B(t)dB(t)$
is not null, hence there is a difference between both integrals, thus
$\int_{D}ZdM$ is not uniquely-defined. However, in this case
$K_{B^{2},B^{\prime}}=0$ (impair moments of a centred Gaussian vector are
involved, which are always null), which is trivially self-integral, as well as
$K_{B^{2},B^{\prime}}^{(2)}$.

###  5.3 Stochastic and self-integration over subsets

Both definitions of the self-integral and the uniquely-defined stochastic
integral we have used are focused on a fixed integration domain $D$. Form the
definitions, it is not clear this implies the possibility of integrating over
subsets of $D$. In this Section we tackle such problem.

######  Theorem 5.2.

Let $D\in\mathcal{B}_{B}(\mathbb{R}^{d})$. If the stochastic integral
$\int_{D}ZdM$ is uniquely-defined in the sense of
$L^{p}(\Omega,\mathcal{A},\mathbb{P})$ for some $p\in[1,\infty]$, then for
every $A\in\mathcal{B}(D)$ the stochastic integral $\int_{A}ZdM$ is uniquely-
defined in the sense of $L^{p}(\Omega,\mathcal{A},\mathbb{P})$ and the
additivity property holds:

$\int_{A}ZdM+\int_{D\setminus A}ZdM=\int_{D}ZdM.$ (5.32)

We shall use the following Lemma on sequences and sub-sequences on Banach
spaces.

######  Lemma 5.4.

Let $(E,\|\cdot\|_{E})$ be a Banach space. Let $(a_{n})_{n\in\mathbb{N}}$ and
$(b_{n})_{n\in\mathbb{N}}$ be two sequences in $E$ satisfying the following
property: there exists a unique $\ell\in E$ such that for every growing
function $\sigma:\mathbb{N}\to\mathbb{N}$,

$\lim_{n\to\infty}a_{n}+b_{\sigma(n)}=\ell.$ (5.33)

Then, both $(a_{n})_{n\in\mathbb{N}}$ and $(b_{n})_{n\in\mathbb{N}}$ converge.

Proof of Lemma 5.4: For any two growing functions
$\sigma,\sigma^{\prime}:\mathbb{N}\to\mathbb{N}$, one has from property (5.33)

$\lim_{n\to\infty}b_{\sigma(n)}-b_{\sigma^{\prime}(n)}=\lim_{n\to\infty}a_{n}+b_{\sigma(n)}-(a_{n}+b_{\sigma^{\prime}(n)})=\ell-\ell=0.$
(5.34)

Suppose now that $(b_{n})_{n\in\mathbb{N}}$ does not converge. Since $E$ is
Banach, $(b_{n})_{n}$ must not be a Cauchy sequence. There exists thus
$\epsilon>0$ such that for every $n\in\mathbb{N}$ one can find
$\tilde{n},\tilde{m}\geq n$ such that
$\|b_{\tilde{n}}-b_{\tilde{m}}\|_{E}\geq\epsilon$. We can hence construct two
growing functions $\sigma_{1},\sigma_{2}:\mathbb{N}\to\mathbb{N}$ as
following. For $n=0$, we take $\sigma_{1}(n),\sigma_{2}(n)\geq 1$ such that
$\|b_{\sigma_{1}(0)}-b_{\sigma_{2}(0)}\|_{E}\geq\epsilon$. Then, for every
$n\geq 1$ we take
$\sigma_{1}(n),\sigma_{2}(n)>\max\\{\sigma_{1}(n-1),\sigma_{2}(n-1)\\}$ such
that $\|b_{\sigma_{1}(n)}-b_{\sigma_{2}(n)}\|_{E}\geq\epsilon$. It follows
that both $\sigma_{1}$ and $\sigma_{2}$ are growing and that
$\|b_{\sigma_{1}(n)}-b_{\sigma_{2}(n)}\|_{E}\geq\epsilon$ for all
$n\in\mathbb{N}$. But this contradicts the result (5.34).
$(b_{n})_{n\in\mathbb{N}}$ must hence be Cauchy, and thus convergent since $E$
in Banach. From property (5.33) it follows immediately that
$(a_{n})_{n\in\mathbb{N}}$ must also converge. $\blacksquare$

Proof of Theorem 5.2: Let $A\in\mathcal{B}(D)$. Let $\left((A_{j}^{n})_{j\in
J^{A}_{n}},(a_{j}^{n})_{j\in J^{A}_{n}}\right)_{n\in\mathbb{N}}$ be any
Riemann system of $A$, and let $\left((\tilde{A}_{j}^{n})_{j\in
J^{\tilde{A}}_{n}},(\tilde{a}_{j}^{n})_{j\in
J^{\tilde{A}}_{n}}\right)_{n\in\mathbb{N}}$ be any Riemann system of
$D\setminus A$. Then, one can construct a new Riemann system of $D$, say
$\left((I_{j}^{n})_{j\in J_{n}},(x_{j}^{n})_{j\in
J_{n}}\right)_{n\in\mathbb{N}}$ through,

$J_{n}=J_{n}^{A}\cup J_{n}^{\tilde{A}};\quad
I_{j}^{n}=\begin{cases}A_{j}^{n}&\hbox{ if }j\in J_{n}^{A}\\\
\tilde{A}_{j}^{n}&\hbox{ if }j\in J_{n}^{\tilde{A}}\end{cases};\quad
x_{j}^{n}=\begin{cases}a_{j}^{n}&\hbox{ if }j\in J_{n}^{A}\\\
\tilde{a}_{j}^{n}&\hbox{ if }j\in J_{n}^{\tilde{A}}\end{cases}.$ (5.35)

The Riemann sum associated to such system is the sum of the Riemann sums over
$A$ and $D\setminus A$:

$\sum_{j\in J_{n}}Z(x_{j}^{n})M(I_{j}^{n})=\sum_{j\in
J_{n}^{A}}Z(a_{j}^{n})M(A_{j}^{n})+\sum_{j\in
J_{n}^{\tilde{A}}}Z(\tilde{a}_{j}^{n})M(\tilde{A}_{j}^{n}).$ (5.36)

If $\sigma:\mathbb{N}\to\mathbb{N}$ is a growing function, then
$\left((\tilde{A}_{j}^{\sigma(n)})_{j\in
J_{\sigma(n)}^{\tilde{A}}},(\tilde{a}_{j}^{\sigma(n)})_{j\in
J_{\sigma(n)}^{\tilde{A}}}\right)_{n\in\mathbb{N}}$ is a sub-Riemann system of
$D\setminus A$. Using it together with $\left((A_{j}^{n})_{j\in
J^{A}_{n}},(a_{j}^{n})_{j\in J^{A}_{n}}\right)_{n\in\mathbb{N}}$ for
constructing a new Riemann system of $D$ analogously as in (5.36), since the
stochastic integral $\int_{D}ZdM$ is uniquely-defined we must have

$\lim_{n\to\infty}\sum_{j\in J_{n}^{A}}Z(a_{j}^{n})M(A_{j}^{n})+\sum_{j\in
J_{\sigma(n)}^{\tilde{A}}}Z(\tilde{a}_{j}^{\sigma(n)})M(\tilde{A}_{j}^{\sigma(n)})=\int_{D}ZdM.$
(5.37)

From Lemma 5.4 using the Banach space $L^{p}(\Omega,\mathcal{A},\mathbb{P})$,
the Riemann sum $\sum_{j\in J_{n}^{A}}Z(a_{j}^{n})M(A_{j}^{n})$ must converge.
Set $\ell_{A}:=\lim_{n\to\infty}\sum_{j\in
J_{n}^{A}}Z(a_{j}^{n})M(A_{j}^{n})$. If $\left((\tilde{A}_{j}^{n})_{j\in
J^{\tilde{A}}_{n}},(\tilde{a}_{j}^{n})_{j\in
J^{\tilde{A}}_{n}}\right)_{n\in\mathbb{N}}$ denotes now any arbitrary Riemann
system of $D\setminus A$, then by constructing a Riemann system of $D$ as in
(5.35), it follows

$\lim_{n\to\infty}\sum_{j\in
J_{n}^{\tilde{A}}}Z(\tilde{a}_{j}^{n})M(\tilde{A}_{j}^{n})=\lim_{n\to\infty}\sum_{j\in
J_{n}}Z(x_{j}^{n})M(I_{j}^{n})-\sum_{j\in
J_{n}^{A}}Z(a_{j}^{n})M(A_{j}^{n})=\int_{D}Zd_{M}-\ell_{A}.$ (5.38)

Since the Riemann system of $D\setminus A$ was arbitrary, it follows that the
stochastic integral $\int_{D\setminus A}ZdM$ is uniquely-defined and equals
$\int_{D}Zd_{M}-\ell_{A}$. By taking arbitrary Riemann systems of $A$ and
$D\setminus A$ and constructing an associated Riemann system of $D$ through
(5.35), it follows that the stochastic integral $\int_{A}ZdM$ is also
uniquely-defined and equals $\ell_{A}$. The additivity property (5.32) follows
from (5.38). $\blacksquare$

The next Theorem is a résumé of the main results exposed until now and it
includes thus self-integrability over subsets for cross-covariance kernels. It
also makes precise the important relations of expectation and covariances
between stochastic integrals.

######  Theorem 5.3.

Suppose $Z$ and $M$ are jointly Gaussian and centred. Let
$D\in\mathcal{B}_{B}(\mathbb{R}^{d})$. Then, the following statements are
equivalent:

  1. (i)

$K_{Z,M}$ is self-integrable over $D$ and $K_{Z,M}^{(2)}$ is quasi-self-
integrable over $D\times D$.

  2. (ii)

$K_{Z,M}$ is self-integrable over every $A\in\mathcal{B}(D)$ and
$K_{Z,M}^{(2)}$ is quasi-self-integrable over every set of the form $A\times
B$ with $A,B\in\mathcal{B}(D)$.

  3. (iii)

The stochastic integral $\int_{D}ZdM$ is uniquely-defined as a limit in
$L^{2}(\Omega,\mathcal{A},\mathbb{P})$.

  4. (iv)

For every $A\in\mathcal{B}(D)$, the stochastic integral $\int_{A}ZdM$ is
uniquely-defined as a limit in $L^{2}(\Omega,\mathcal{A},\mathbb{P})$.

Moreover, if any of these statements holds, the expectation and covariance
formulae hold:

$\mathbb{E}\left(\int_{A}ZdM\right)=\int_{A}K(x,dx),\quad\forall
A\in\mathcal{B}(D),$ (5.39)
$\mathbb{C}ov\left(\int_{A}ZdM,\int_{B}ZdM\right)=\int_{A\times
B}C_{Z}dC_{M}+\tilde{\int}_{A\times B}K^{(2)}(x,dx),\quad\forall
A,B\in\mathcal{B}(D).$ (5.40)

Proof of Theorem 5.3: $(i)\Leftrightarrow(iii)$ is Theorem 5.1.
$(iii)\Leftrightarrow(iv)$ is Theorem 5.2. $(ii)\Rightarrow(i)$ is obvious. We
need only to prove $(iv)\Rightarrow(ii)$ and the formulae (5.39) and (5.40).
The self-integrability of $K_{Z,M}$ over $A$ and formula (5.39) come from
Proposition 5.1. For $K^{(2)}_{Z,M}$, let $\left((A_{j}^{n})_{j\in
J^{A}_{n}},(a_{j}^{n})_{j\in J^{A}_{n}}\right)_{n\in\mathbb{N}}$ and
$\left((B_{j}^{n})_{j\in J^{B}_{n}},(b_{j}^{n})_{j\in
J^{B}_{n}}\right)_{n\in\mathbb{N}}$ be two arbitrary Riemann systems of $A$
and $B$ respectively. Since the associated Riemann sums approaching the
stochastic integrals converge to $\int_{A}ZdM$ and $\int_{B}ZdM$ respectively,
we use the same arguments as in (5.28) and we conclude

$\mathbb{C}ov\Big{(}\int_{A}ZdM,\int_{B}ZdM\Big{)}=\int_{A\times
B}C_{Z}dC_{M}+\lim_{(n,m)\to\infty}\sum_{j\in J_{n}^{A}}\sum_{k\in
J_{m}^{B}}K_{Z,M}^{(2)}((a_{j}^{n},b_{k}^{m}),A_{j}^{n}\times B_{k}^{m}).$
(5.41)

The double-limit at the right side holds then for any arbitrary Riemann
systems of $A$ and $B$. It follows that $K_{Z,M}^{(2)}$ is quasi-self-
integrable over $A\times B$, and its quasi-self-integral satisfies equation
(5.40). $\blacksquare$

Let us now prove that a self-integrable kernel $K$ is also self-integrable on
subsets, without asking cross-pos-def properties or any other condition on
$K^{(2)}$.

######  Theorem 5.4.

Let $K$ be an arbitrary cf-m kernel over $\mathbb{R}^{d}$, and let
$D\in\mathcal{B}_{B}(\mathbb{R}^{d})$. If $K$ is self-integrable over $D$,
then it is self-integrable over every subset $A\in\mathcal{B}(D)$ and the
additivity property holds:

$\int_{A}K(x,dx)+\int_{D\setminus A}K(x,dx)=\int_{D}K(x,dx),\quad\forall
A\in\mathcal{B}(D).$ (5.42)

Proof of Theorem 5.4: The arguments are the same as in Theorem 5.2, so we give
the outline and leave details to the reader: use an arbitrary Riemann system
of $A$ and an arbitrary Riemann system of $D\setminus A$. Construct a Riemann
system of $D$ through (5.35). The sum of the respective Riemann sums converges
to $\int_{D}K(x,dx)$, as well as the sums when taking a sub-sequence of the
Riemann sum of $D\setminus A$. Apply Lemma 5.4 in the case $E=\mathbb{R}$ to
prove that the Riemann sums over $A$ and $D\setminus A$ converge. Argue then
by varying the Riemann system of $D\setminus A$, that $K$ is self-integrable
over $D\setminus A$, and conclude then that $K$ is self-integrable over $A$,
with the additivity property (5.42) holding. $\blacksquare$

For $K^{(2)}$, we only mention that, if $K$ is cross-pos-def, self-integrable
over $D$ and such that $K^{(2)}$ is quasi-self-integrable over $D\times D$,
then $K^{(2)}$ is quasi-self-integrable over every $A\times B$,
$A,B\in\mathcal{B}(D)$. This comes directly from Theorem 5.3. In this case we
have an additivity property for the quasi-self-integrals

$\tilde{\int}_{A\times A}K^{(2)}(x,dx)+\tilde{\int}_{A\times(D\setminus
A)}K^{(2)}(x,dx)+\tilde{\int}_{(D\setminus A)\times
A}K^{(2)}(x,dx)+\tilde{\int}_{(D\setminus A)\times(D\setminus
A)}K^{(2)}(x,dx)=\tilde{\int}_{D\times D}K^{(2)}(x,dx),$ (5.43)

which can be obtained through equation (5.40), using
$\int_{D}ZdM=\int_{A}ZdM+\int_{D\setminus A}ZdM$ and eliminating
correspondingly the integrals of the form $\int C_{Z}dC_{M}$ that there
appear.

###  5.4 Measure structure

Results in Section 5.3 have allowed us to define, under suitable self-
integrability conditions over $D$, the applications
$A\in\mathcal{B}(D)\mapsto\int_{A}K(x,dx)$, and
$(A,B)\in\mathcal{B}(D)\times\mathcal{B}(D)\mapsto\tilde{\int}_{A\times
B}K^{(2)}(x,dx)$. As we have seen, the first application is additive and it is
not difficult to prove that the second is additive in each component when the
other is fixed. However, it is not clear if $\sigma$-additivity properties can
be concluded for those applications, that is, if the first one defines a
measure over $D$ or if the second defines a measure over $D\times D$ (or a bi-
measure whatsoever). This question is not without any use, since in such case,
one can construct stochastic integrals which determine random measures. In
particular, the following Corollary follows immediately from equations (5.39)
and (5.40).

######  Corollary 5.1.

Suppose $Z$ and $M$ are jointly Gaussian. Then, the application
$A\mapsto\int_{A}ZdM$ determines a uniquely-defined random measure over $D$ if
and only if the self-integrals $A\mapsto\int_{A}K_{Z,M}(x,dx)$ and the quasi-
self-integrals $(A,B)\mapsto\tilde{\int}_{A\times B}K_{Z,M}^{(2)}(x,dx)$ are
well-defined and determine measures over $D$ and $D\times D$ respectively.

It is then important to determine cases where a measure structure is induced
in the self-integration process. Let us begin with the case of the self-
integral of $K$. Rather than proving a $\sigma$-additivity property, we will
construct directly the integrals of measurable functions with respect to the
potential measure. Suppose $D$ is compact and $K$ is self-integrable over $D$.
If $\varphi\in\mathcal{M}_{B}(D)$ is a simple function, say of the form
$\varphi=\sum_{i}a_{i}\mathbbm{1}_{A_{i}}$, then we can define using Theorem
5.4,

$\int_{D}\varphi(x)K(x,dx):=\sum_{i\in I}a_{i}\int_{A_{i}}K(x,dx).$ (5.44)

In order to extent this to a more general $\varphi$, we start, as in the case
of classical measures, with the case where $K$ is a positive kernel over $D$,
that is such that $K(x,A)\geq 0$, for every $(x,A)\in D\times\mathcal{B}(D)$.
in such case, one has for $\varphi$ simple

$\left|\int_{D}\varphi(x)K(x,dx)\right|\leq\sum_{i}|a_{i}|\int_{A_{i}}K(x,dx)\leq\|\varphi\|_{\infty}\int_{D}K(x,dx).$
(5.45)

For $\varphi\in C(D)$, we express it as a uniform limit of simple functions
$(\varphi_{n})_{n}$ and we define
$\int_{D}\varphi(x)K(x,dx)=\lim_{n\to\infty}\int_{D}\varphi_{n}(x)K(x,dx)$.
From (5.45) it follows that the limit is well-defined and it does not depend
upon the sequence approaching $\varphi$. Equation (5.45) also holds thus for
$\varphi\in C(D)$, and the application $\varphi\in
C(D)\mapsto\int_{D}\varphi(x)K(x,dx)$ defines a linear and continuous
application in the supremum norm. From Riesz Representation Theorem (Remark
2.1), there exists a unique measure representing such application, and hence
its definition can be extended to any $\varphi\in\mathcal{M}_{B}(D)$. We
conclude the next Proposition.

######  Proposition 5.3.

If $K\geq 0$ is self-integrable over $D$ compact, then the application
$A\mapsto\int_{A}K(x,dx)$ defines a (positive) measure.

For the general case when $K$ is not positive, we consider an intuitive
sufficient condition regarding the self-integrability of the total-variation
kernel $|K|$. We remark that Definition 4.1 of the self-integral only applies
for a cf-m kernel, hence we cannot in general apply it to $|K|$ since, as far
as we now, $|K|$ may not be continuous. We need hence to add this extra
requirement. Note that this anyway holds for $K$ cross-pos-def (Theorem 3.3).

######  Proposition 5.4.

Suppose $D$ is compact. If $K$ is a cf-m kernel such that $|K|$ is also a cf-m
kernel and if both $K$ and $|K|$ are self-integrable over $D$, then
$A\mapsto\int_{A}K(x,dx)$ defines a measure satisfying

$\left|\int_{D}\varphi(x)K(x,dx)\right|\leq\|\varphi\|_{\infty}\int_{D}|K|(x,dx),\quad\forall\varphi\in\mathcal{M}_{B}(D).$
(5.46)

Proof: If $\left((A_{j}^{n})_{j\in J^{A}_{n}},(a_{j}^{n})_{j\in
J^{A}_{n}}\right)_{n\in\mathbb{N}}$ is any Riemann system of $A$, then

$\left|\int_{A}K(x,dx)\right|=\lim_{n\to\infty}\left|\sum_{j\in
J_{n}^{A}}K(a_{j}^{n},A_{j}^{n})\right|\leq\lim_{n\to\infty}\sum_{j\in
J_{n}^{A}}|K|(a_{j}^{n},A_{j}^{n})=\int_{A}|K|(x,dx).$ (5.47)

It follows that for a simple function
$\varphi=\sum_{i}a_{i}\mathbbm{1}_{A_{i}}$, one has

$\left|\int_{D}\varphi(x)K(x,dx)\right|=\left|\sum_{i}a_{i}\int_{A_{i}}K(x,dx)\right|\leq\sum_{i}|a_{i}|\int_{A_{i}}|K|(x,dx)\leq\|\varphi\|_{\infty}\int_{D}|K|(x,dx).$
(5.48)

The linear application $\varphi\mapsto\int_{D}\varphi(x)K(x,dx)$ is hence
continuous in the sense of the supremum norm. It determines hence a measure
and its application can be extended to any $\varphi\in\mathcal{M}_{B}(D)$.
$\blacksquare$

The general question of whether the self-integrability of $K$ implies or is
implied by the self-integrability of $|K|$ remains open. Now, we can give
similar results concerning the quasi-self-integral of $K^{(2)}$.

######  Proposition 5.5.

Let $D\subset\mathbb{R}^{d}$ compact. If $K$ is positive, cross-pos-def and
self-integrable over $D$, and if such that $K^{(2)}$ is quasi-self-integrable
over $D\times D$, then $(A,B)\mapsto\int_{A\times B}K^{(2)}(x,dx)$ defines a
measure over $D\times D$.

Proof: We proceed similarly as in Proposition 5.3. Of course $K$ positive
implies $K^{(2)}$ positive. From Theorem 5.3, $K^{(2)}$ is quasi-self-
integrable over every subset $A\times B$, $A,B\in\mathcal{B}(D)$. Hence, we
can define a linear application over every simple function of the form
$\psi=\sum_{i}\sum_{j}a_{i,j}\mathbbm{1}_{A_{i}}\otimes\mathbbm{1}_{B_{k}}$ as

$\tilde{\int}_{D\times D}\psi(x)K^{(2)}(x,dx):=\sum_{i\in I}\sum_{j\in
J}a_{i,j}\tilde{\int}_{A_{i}\times B_{j}}K(x,dx).$ (5.49)

And one can easily prove also following the same idea as in (5.45) that for
such $\psi$

$\left|\tilde{\int}_{D\times
D}\psi(x)K^{(2)}(x,dx)\right|\leq\|\psi\|_{\infty}\tilde{\int}_{D\times
D}K^{(2)}(x,dx).$ (5.50)

Since every function in $C(D\times D)$ can be uniformly approached by
functions of the form of $\psi$, one can apply the same approximation
arguments and Riesz Representation Theorem to prove that the application
$\psi\mapsto\tilde{\int}_{D\times D}\psi(x)K^{(2)}(x,dx)$ defines a measure
and can be extended uniquely to any $\psi\in\mathcal{M}_{B}(D\times D)$.
$\blacksquare$

The next Proposition is concluded following similar arguments which we left to
the reader.

######  Proposition 5.6.

Suppose $D$ compact. Let $K$ be cross-pos-def, such that $K$ and $|K|$ are
self-integrable over $D$ and $K^{(2)}$ and $|K^{(2)}|$ are quasi-self-
integrable over $D\times D$. Then, the application
$(A,B)\mapsto\tilde{\int}_{A\times B}K^{(2)}(x,dx)$ defines a measure over
$D\times D$.

We finish this Section with the next Corollary which follows directly from
Propositions 5.4 and 5.6 and Theorem 5.3.

######  Corollary 5.2.

Suppose $Z$ and $M$ jointly Gaussian and $D$ compact. If $K_{Z,M}$ and
$|K_{Z,M}|$ are self-integrable over $D$, and if $K_{Z,M}^{(2)}$ and
$|K_{Z,M}^{(2)}|$ are quasi-self-integrable over $D\times D$, then
$A\mapsto\int_{A}ZdM$ defines a unique random measure over $D$ with mean and
covariance measures given by (5.39) and (5.40).

######  Remark 5.1.

When conditions in Corollaries 5.1 or 5.2 hold, since $\int_{A}ZdM$ defines a
random measure, the integrals of every $\varphi\in\mathcal{M}_{B}(D)$ with
respect to this measure are well-defined random variables, giving thus a
unique definition for the stochastic integral

$\int_{D}Z(x)\varphi(x)dM(x),$ (5.51)

and such random variable should coincide with other intuitive forms of
defining such integral. We remark that this allows, among many other things,
to define properly distributional derivatives of the product $``ZM"$ when
using $\varphi$ smooth with compact support.

##  6 Applications

In this Section we apply our results. We begin with two cases where conditions
in Theorem 5.1 hold with a surprising convenience, allowing the unique-
definition of some concepts involving stochastic integrals. We also present an
important examples without uniquely-defined stochastic integrals, showing some
limitations of our approach.

###  6.1 Tensor product of Gaussian random measures

In Lemma 5.1 we have defined the tensor product of two independent random
measures. Now, we will relax the hypothesis of independence but we will ask a
Gaussianity condition. Let $M_{1}$ and $M_{2}$ be two real centred jointly
Gaussian random measures over $\mathbb{R}^{d}$ and $\mathbb{R}^{p}$
respectively. Let us suppose, in addition, that the cross-covariance between
them is determined by a measure
$C_{M_{1},M_{2}}\in\mathscr{M}(\mathbb{R}^{d}\times\mathbb{R}^{p})$, that is
$\mathbb{E}(M_{1}(A)M_{2}(B))=C_{M_{1},M_{2}}(A\times B)$ for any
$A\in\mathcal{B}_{B}(\mathbb{R}^{d}),B\in\mathcal{B}_{B}(\mathbb{R}^{p})$.
$M_{1}\otimes M_{2}$ should then be a new random measure over
$\mathbb{R}^{d}\times\mathbb{R}^{p}$ satisfying $(M_{1}\otimes M_{2})(A\times
B)=M_{1}(A)M_{2}(B)$.

We follow the same approach as in the proof of Lemma 5.1. Let $\psi\in
C_{c}(\mathbb{R}^{d}\times\mathbb{R}^{p})$, with
$\operatorname{supp}(\psi)\subset D_{d}\times D_{p}$, $D_{d}$, and $D_{p}$
compacts. We propose to define the action of $M_{1}\otimes M_{2}$ over $\psi$
through an iterated stochastic integral as in (5.24). In order to verify that
such stochastic integral is well-defined, we consider that
$Z_{M_{2}}(x):=\int_{\mathbb{R}^{p}}\psi(x,y)dM_{2}(y)$ is a mean-square
continuous process over $\mathbb{R}^{d}$, null outside $D_{d}$. The cross-
covariance kernel between $Z_{M_{2}}$ and $M_{1}$ is

$K_{Z_{M_{2}},M_{1}}(x,A)=\int_{A\times\mathbb{R}^{p}}\psi(x,v)dC_{M_{1},M_{2}}(u,v)=\int_{D_{d}\times
D_{p}}\psi(x,v)\mathbbm{1}_{A}(u)dC_{M_{1},M_{2}}(u,v).$ (6.1)

If $\left((I_{j}^{n})_{j\in J_{n}},(x_{j}^{n})_{j\in
J_{n}}\right)_{n\in\mathbb{N}}$ is a Riemann system of $D_{d}$, the associated
Riemann sum approaching the potential self-integral of $K_{Z_{M_{2}},M_{1}}$
is

$\sum_{j\in J_{n}}\int_{D_{d}\times
D_{p}}\psi(x_{j}^{n},v)\mathbbm{1}_{I_{j}^{n}}(u)dC_{M_{1},M_{2}}(u,v)=\int_{D_{d}\times
D_{p}}\left(\sum_{j\in
J_{n}}\psi(x_{j}^{n},v)\mathbbm{1}_{I_{j}^{n}}(u)\right)dC_{M_{1},M_{2}}(u,v).$
(6.2)

From the uniform continuity of $\psi$ over $D_{d}\times D_{p}$, one has
$\sum_{j\in J_{n}}\psi(x_{j}^{n},v)\mathbbm{1}_{I_{j}^{n}}(u)\to\psi(u,v)$
uniformly over $D_{d}\times D_{p}$. Since $C_{M_{1},M_{2}}$ is a measure, this
implies the convergence of the integrals to a limit independent of the Riemann
system used. $K_{Z_{M_{2}},M_{1}}$ is hence self-integrable over $D_{d}$ with
self-integral

$\int_{D_{d}}K_{Z_{M_{2}},M_{1}}(x,dx)=\int_{\mathbb{R}^{d}\times\mathbb{R}^{p}}\psi(x,y)dC_{M_{1},M_{2}}(x,y).$
(6.3)

The second order kernel is given by (computations left to the reader)

$K_{Z_{M_{2}},M_{1}}^{(2)}((x,y),E)=\int_{D_{d}\times D_{p}}\int_{D_{d}\times
D_{p}}\psi(x,\tilde{v})\mathbbm{1}_{E}(u,\tilde{u})\psi(y,v)dC_{M_{1},M_{2}}(u,v)dC_{M_{1},M_{2}}(\tilde{u},\tilde{v}).$
(6.4)

$K_{Z_{M_{2}},M_{1}}^{(2)}$ is actually self-integrable over $D_{d}\times
D_{d}$ in the sense of Definition 4.1, which implies of course quasi-self-
integrability. To see this, we argue similarly as in the case of the self-
integrability of $K_{Z_{M_{2}},M_{1}}$ using the uniform continuity of $\psi$
which allows to conclude that for any Riemann system $\left((E_{j}^{n})_{j\in
J_{n}},(e_{j}^{n})_{j\in J_{n}}\right)_{n\in\mathbb{N}}$ of $D_{d}\times
D_{d}$, (say $e_{j}^{n}=(x_{j}^{n},y_{j}^{n})\in D_{d}\times D_{d}$) one has

$\sum_{j\in
J_{n}}\psi(x_{j}^{n},\tilde{v})\mathbbm{1}_{E_{j}^{n}}(u,\tilde{u})\psi(y_{j}^{n},v)\xrightarrow[n\to\infty]{}\psi(u,\tilde{v})\psi(\tilde{u},v)$
(6.5)

uniformly in $(u,v,\tilde{u},\tilde{v})\in D_{d}\times D_{p}\times D_{d}\times
D_{p}$. The self-integral of $K_{Z_{M_{2}},M_{1}}^{(2)}$ is hence given by

$\int_{D_{d}\times D_{d}}K_{Z_{M_{2}},M_{1}}^{(2)}(x,dx)=\int_{D_{d}\times
D_{p}\times D_{d}\times
D_{p}}\psi(u,\tilde{v})\psi(\tilde{u},v)d(C_{M_{1},M_{2}}\otimes
C_{M_{1},M_{2}})((u,v),(\tilde{u},\tilde{v})).$ (6.6)

Theorem 5.1 can be hence applied. The stochastic integrals of the form

$\langle M_{1}\otimes
M_{2},\psi\rangle:=\int_{\mathbb{R}^{d}}\int_{\mathbb{R}^{p}}\psi(x,y)dM_{2}(y)dM_{1}(x)$
(6.7)

for $\psi\in C_{c}(\mathbb{R}^{d}\times\mathbb{R}^{p})$ are uniquely-defined.
Let us describe the mean and covariance of such variables. For the mean, it
follows from (6.3) that

$\mathbb{E}\left(\langle M_{1}\otimes
M_{2},\psi\rangle\right)=\int_{\mathbb{R}^{d}\times\mathbb{R}^{p}}\psi(x,y)dC_{M_{1},M_{2}}(x,y).$
(6.8)

For the covariance consider for simplicity $\psi_{1}$ and $\psi_{2}$ real. We
then compute through Riemann sums and using Lemma 5.2

$\displaystyle\mathbb{C}ov$ $\displaystyle\left(\langle M_{1}\otimes
M_{2},\psi_{1}\rangle,\langle M_{1}\otimes M_{2},\psi_{2}\rangle\right)$ (6.9)
$\displaystyle=\lim_{(n,m)\to\infty}\sum_{j\in
J_{n}}\sum_{k\in\tilde{J}_{m}}\mathbb{C}ov\left(\int_{\mathbb{R}^{p}}\psi_{1}(x_{j}^{n},v)dM_{2}(v)M_{1}(I_{j}^{n}),\int_{\mathbb{R}^{p}}\psi_{2}(y_{k}^{m},\tilde{v})dM_{2}(\tilde{v})M_{1}(E_{k}^{m})\right)$
$\displaystyle=\lim_{(n,m)\to\infty}\sum_{j\in
J_{n}}\sum_{k\in\tilde{J}_{m}}\int_{\mathbb{R}^{p}\times\mathbb{R}^{p}}\psi_{1}(x_{j}^{n},v)\psi_{2}(y_{k}^{m},\tilde{v})dC_{M_{2}}(v,\tilde{v})C_{M_{1}}(I_{j}^{n}\times
E_{k}^{m})$ $\displaystyle\quad\quad+\sum_{j\in
J_{n}}\sum_{k\in\tilde{J}_{m}}\int_{\mathbb{R}^{d}\times\mathbb{R}^{p}}\psi_{1}(x_{j}^{n},v)\mathbbm{1}_{E_{k}^{m}}(u)dC_{M_{1},M_{2}}(u,v)\int_{\mathbb{R}^{d}\times\mathbb{R}^{p}}\psi_{2}(y_{k}^{m},v)\mathbbm{1}_{I_{j}^{n}}(\tilde{u})dC_{M_{1},M_{2}}(\tilde{u},\tilde{v})$
$\displaystyle=\int_{\mathbb{R}^{d}\times\mathbb{R}^{d}\times\mathbb{R}^{p}\times\mathbb{R}^{p}}\psi_{1}(\tilde{u},v)\psi_{2}(u,\tilde{v})d(C_{M_{1}}\otimes
C_{M_{2}})(u,\tilde{u},v,\tilde{v})$
$\displaystyle\quad\quad+\int_{\mathbb{R}^{d}\times\mathbb{R}^{p}\times\mathbb{R}^{d}\times\mathbb{R}^{p}}\psi_{1}(\tilde{u},v)\psi_{2}(u,\tilde{v})d(C_{M_{1},M_{2}}\otimes
C_{M_{1},M_{2}})(u,v,\tilde{u},\tilde{v}).$

The last double-limit is justified using similar uniform-approximation
arguments as in (6.5). Since $C_{M_{1}},C_{M_{2}}$ and $C_{M_{1},M_{2}}$ are
all measures, it follows that expression (6.9) defines a measure over
$\mathbb{R}^{d}\times\mathbb{R}^{p}\times\mathbb{R}^{d}\times\mathbb{R}^{p}$.
We can express these results in a more synthetic manner as

$m_{M_{1}\otimes M_{2}}:=C_{M_{1},M_{2}}\quad;\quad C_{M_{1}\otimes
M_{2}}=C_{M_{1}}``\otimes"C_{M_{2}}+C_{M_{1},M_{2}}``\otimes"C_{M_{1},M_{2}},$
(6.10)

where $C_{M_{1}}``\otimes"C_{M_{2}}$ and
$C_{M_{1},M_{2}}``\otimes"C_{M_{1},M_{2}}$ denote respectively the only
measures over
$\mathbb{R}^{d}\times\mathbb{R}^{p}\times\mathbb{R}^{d}\times\mathbb{R}^{p}$
such that for all
$A_{1},A_{2}\in\mathcal{B}_{B}(\mathbb{R}^{d}),B_{1},B_{2}\in\mathcal{B}_{B}(\mathbb{R}^{p})$
one has

$(C_{M_{1}}``\otimes"C_{M_{2}})(A_{1}\times B_{1}\times A_{2}\times
B_{2})=C_{M_{1}}(A_{1}\times A_{2})C_{M_{2}}(B_{1}\times B_{2});$ (6.11)
$(C_{M_{1},M_{2}}``\otimes"C_{M_{1},M_{2}})(A_{1}\times B_{1}\times
A_{2}\times B_{2})=C_{M_{1},M_{2}}(A_{1}\times
B_{2})C_{M_{1},M_{2}}(A_{2}\times B_{1}).$ (6.12)

Note that the measure $C_{M_{1}}``\otimes"C_{M_{2}}$ is not the measure
$C_{M_{1}}\otimes C_{M_{2}}$ since the latter is defined over
$\mathbb{R}^{d}\times\mathbb{R}^{d}\times\mathbb{R}^{p}\times\mathbb{R}^{p}$.
As well, the measure $C_{M_{1},M_{2}}``\otimes"C_{M_{1},M_{2}}$ is not
$C_{M_{1},M_{2}}\otimes C_{M_{1},M_{2}}$ since the former has the cross-
product property of connecting $A_{1}$ with $B_{2}$ rather than with $B_{1}$.
Using Theorem 2.2 and thus extending the definition of $M_{1}\otimes M_{2}$ to
functions in $\mathcal{M}_{B,c}(\mathbb{R}^{d}\times\mathbb{R}^{p})$, we
conclude the following Theorem.

######  Theorem 6.1.

Let $M_{1}$ and $M_{2}$ be jointly Gaussian random measures over
$\mathbb{R}^{d}$ and $\mathbb{R}^{p}$ respectively. Suppose that the cross-
covariance structure between $M_{1}$ and $M_{2}$ is determined by a measure
$C_{M_{1},M_{2}}\in\mathscr{M}(\mathbb{R}^{d}\times\mathbb{R}^{p})$. Then,
there exists a unique random measure over
$\mathbb{R}^{d}\times\mathbb{R}^{p}$, noted $M_{1}\otimes M_{2}$, which
satisfies

$(M_{1}\otimes M_{2})(A\times B)=M(A)M(B),\quad\forall
A\in\mathcal{B}_{B}(\mathbb{R}^{d}),B\in\mathcal{B}_{B}(\mathbb{R}^{p}).$
(6.13)

Consider the iterated integrals in (6.7) in the other sense
$\int_{\mathbb{R}^{p}}\int_{\mathbb{R}^{d}}\psi(x,y)dM_{1}(x)dM_{2}(y)$.
Following the same arguments as before, one can conclude that such iterated
integral is uniquely-defined. The mean and the covariance structures of the
application
$\psi\mapsto\int_{\mathbb{R}^{p}}\int_{\mathbb{R}^{d}}\psi(x,y)dM_{1}(x)dM_{2}(y)$
can be computed analogously as we did for (6.10), obtaining that both
structures are determined by measures. In addition, it is clear that
$\int_{\mathbb{R}^{p}}\int_{\mathbb{R}^{d}}\psi(x,y)dM_{1}(x)dM_{2}(y)$
coincide with
$\int_{\mathbb{R}^{d}}\int_{\mathbb{R}^{p}}\psi(x,y)dM_{2}(y)dM_{1}(x)$ for
every $\psi\in C_{c}(\mathbb{R}^{d})\otimes C_{c}(\mathbb{R}^{p})$. Since
$C_{c}(\mathbb{R}^{d})\otimes C_{c}(\mathbb{R}^{p})$ is dense in
$C_{c}(\mathbb{R}^{d}\times\mathbb{R}^{p})$ with the uniform-on-compacts
topology, and since both applications define continuous linear functions from
$C_{c}(\mathbb{R}^{d}\times\mathbb{R}^{p})$ to
$L^{2}(\Omega,\mathcal{A},\mathbb{P})$, both must coincide over the whole
space $C_{c}(\mathbb{R}^{d}\times\mathbb{R}^{p})$. We conclude the following
Fubini Theorem.

######  Theorem 6.2 (Fubini Theorem for random measures).

Let $M_{1}$ and $M_{2}$ be jointly Gaussian random measures over
$\mathbb{R}^{d}$ and $\mathbb{R}^{p}$ respectively such that the cross-
covariance structure between $M_{1}$ and $M_{2}$ is determined by a measure
$C_{M_{1},M_{2}}\in\mathscr{M}(\mathbb{R}^{d}\times\mathbb{R}^{p})$. Then, for
every $\psi\in C_{c}(\mathbb{R}^{d}\times\mathbb{R}^{p})$ the triple equality

$\int_{\mathbb{R}^{p}}\int_{\mathbb{R}^{d}}\psi(x,y)dM_{1}(x)dM_{2}(y)=\langle
M_{1}\otimes
M_{2},\psi\rangle=\int_{\mathbb{R}^{d}}\int_{\mathbb{R}^{p}}\psi(x,y)dM_{2}(y)dM_{1}(x)$
(6.14)

holds, the iterated integrals being uniquely-defined.

It is worth mentioning that Theorem 6.2 works only for $\psi$ continuous with
compact support. While it also works (trivially) for functions in
$\mathcal{M}_{B,c}(\mathbb{R}^{d})\otimes\mathcal{M}_{B,c}(\mathbb{R}^{p})$,
it does not work in general for any
$\psi\in\mathscr{M}_{B,c}(\mathbb{R}^{d}\times\mathbb{R}^{p})$. We show a
counter-example in Section 6.3. The main issue is that in such case the
iterated integrals may not be uniquely-defined, even if the broad tensor
product measure is. One way of expressing this fact is the following: tensor
products of Gaussian random measures do not have a canonical stochastic
disintegration.

We finish by mentioning that through the use of tensor products one can define
the convolution between two Gaussian random measures, at least in the case
where both $M_{1}$ and $M_{2}$ are finite over $\mathbb{R}^{d}$ ($C_{M_{1}}$
and $C_{M_{2}}$ have finite total mass over
$\mathbb{R}^{d}\times\mathbb{R}^{d}$) and a measure $C_{M_{1},M_{2}}$
describes their cross-covariance. In such case, the application $\varphi\in
C_{c}(\mathbb{R}^{d})\mapsto\langle M_{1}\otimes
M_{2},(x,y)\mapsto\varphi(x+y)\rangle$ should define a new random measure.
This is an advance in the research of a convolutive algebra between random
measures, at least in the Gaussian case, which may contribute to the questions
proposed in (Rao, 2012, Chapter 5) and (Park, 2016).

###  6.2 Fractional Brownian motion and its derivative

A fractional Brownian motion over $[0,1]$ with Hurst index $H\in(0,1)$ is a
centred Gaussian process $(B_{H}(t))_{t\in[0,1]}$ with covariance function

$C_{B_{H}}(t,s)=\frac{|t|^{2H}+|s|^{2H}-|t-s|^{2H}}{2}.$ (6.15)

Suppose $H>\frac{1}{2}$. If $B^{\prime}_{H}$ is the distributional derivative
of $B_{H}$, then its covariance is (Borisov Bystrov, 2006)

$C_{B_{H}^{\prime}}=\frac{\partial^{2}C_{B_{H}}}{\partial t\partial
s}=H(2H-1)|t-s|^{2H-2}.$ (6.16)

Since $\frac{1}{2}<H<1$, we have $-1<2H-2<0$. It follows that covariance
(6.16) is identified with a function over $[0,1]\times[0,1]$ which is
integrable but not continuous. $B_{H}^{\prime}$ is hence a random measure with
absolutely continuous covariance measure

$C_{B_{H}^{\prime}}(E)=H(2H-1)\int_{E}\frac{d(u,v)}{|u-v|^{2-2H}},\quad\forall
E\in\mathcal{B}([0,1]\times[0,1]).$ (6.17)

Some authors also call $B_{H}^{\prime}$ a long-range dependence process, see
(Gay Heyde, 1990; Anh ., 1999). Now, using $B_{H}(t)=B_{H}^{\prime}([0,t])$ we
can compute the cross-covariance kernel between $B_{H}$ and its derivative
$B_{H}^{\prime}$:

$\displaystyle
K_{B_{H},B_{H}^{\prime}}(t,A)=\mathbb{E}\left(B_{H}^{\prime}([0,t])B_{H}^{\prime}(A)\right)$
$\displaystyle=H(2H-1)\int_{A\times[0,t]}\frac{d(u,v)}{|u-v|^{2-2H}}$ (6.18)
$\displaystyle=H\int_{A}u^{2H-1}+|t-u|^{2H-1}\operatorname{sgn}(t-u)du.$

Let $\left((I_{j}^{n})_{j\in J_{n}},(t_{j}^{n})_{j\in
J_{n}}\right)_{n\in\mathbb{N}}$ be any Riemann system of $[0,1]$. Then,

$\sum_{j\in
J_{n}}K_{B_{H},B_{H}^{\prime}}(t_{j}^{n},I_{j}^{n})=H\int_{0}^{1}u^{2H-1}du+H\sum_{j\in
J_{n}}\int_{I_{j}^{n}}|t_{j}^{n}-u|^{2H-1}\operatorname{sgn}(t_{j}^{n}-u)du.$
(6.19)

For the second term at the right side of (6.19), we consider that $2H-1>0$ and
hence

$\displaystyle\left|\sum_{j\in
J_{n}}\int_{I_{j}^{n}}|t_{j}^{n}-u|^{2H-1}\operatorname{sgn}(t_{j}^{n}-u)du\right|$
$\displaystyle\leq\sum_{j\in
J_{n}}\operatorname{diam}(I_{j}^{n})^{2H-1}\lambda(I_{j}^{n})$ (6.20)
$\displaystyle\leq\max_{j\in
J_{n}}\operatorname{diam}(I_{j}^{n})^{2H-1}\xrightarrow[n\to\infty]{}0.$

It follows that $K_{B_{H},B_{H}^{\prime}}$ is self-integrable with self-
integral

$\int_{[0,1]\times[0,1]}K_{B_{H},B_{H}^{\prime}}(x,dx)=H\int_{0}^{1}u^{2H-1}du=\frac{1}{2}.$
(6.21)

On the other hand, the second order kernel $K_{B_{H},B_{H}^{\prime}}^{(2)}$ is
given by

$K_{B_{H},B_{H}^{\prime}}^{(2)}((t,s),A\times B)=H^{2}\int_{A\times
B}\left(u^{2H-1}+|s-u|^{2H-1}\operatorname{sgn}(s-u)\right)\left(v^{2H-1}+|t-v|^{2H-1}\operatorname{sgn}(t-v)\right)d(u,v).$
(6.22)

If
$\left((D_{j}^{n})_{j\in\tilde{J}_{n}},(s_{j}^{n})_{j\in\tilde{J}_{n}}\right)_{n\in\mathbb{N}}$
is another Riemann system of $[0,1]$, the double-Riemann sums approaching the
quasi-self-integral of $K_{B_{H},B_{H}^{\prime}}^{(2)}$ are given by the
integral over $[0,1]\times[0,1]$ of the double-sequence of functions

$(u,v)\mapsto H^{2}\sum_{j\in
J_{n}}\sum_{k\in\tilde{J}_{m}}\mathbbm{1}_{I_{j}^{n}}(u)\mathbbm{1}_{D_{k}^{m}}(v)\left(u^{2H-1}+|s_{k}^{m}-u|^{2H-1}\operatorname{sgn}(s_{k}^{m}-u)\right)\left(v^{2H-1}+|t_{j}^{n}-v|^{2H-1}\operatorname{sgn}(t_{j}^{n}-v)\right).$
(6.23)

Let us study the convergence of the functions (6.23) when $(n,m)\to\infty$.
Fix $u\in[0,1]$ and for each $n$ denote $j_{u}\in J_{n}$ the only element of
$J_{n}$ such that $u\in I_{j_{u}}^{n}$. It follows that $x_{j_{u}}^{n}\to u$
when $n\to\infty$. We do analogously with any $v\in[0,1]$ and
$k_{v}\in\tilde{J}_{m}$. Thus (6.23) can be re-written as

$(u,v)\mapsto
H^{2}\left(u^{2H-1}+|s_{k_{v}}^{m}-u|^{2H-1}\operatorname{sgn}(s_{k_{v}}^{m}-u)\right)\left(v^{2H-1}+|t_{j_{u}}^{n}-v|^{2H-1}\operatorname{sgn}(t_{j_{u}}^{n}-v)\right).$
(6.24)

If $u\neq v$, since the $\operatorname{sgn}$ function is continuous outside
$0$, expression (6.24) converges as $(n,m)\to\infty$ to

$H^{2}\left(u^{2H-1}+|v-u|^{2H-1}\operatorname{sgn}(v-u)\right)\left(v^{2H-1}+|u-v|^{2H-1}\operatorname{sgn}(u-v)\right).$
(6.25)

This last function is integrable over $[0,1]\times[0,1]$. In addition, the
functions (6.24) are all bounded by the constant
$H^{2}(1+\sqrt{2}^{2H-1})^{2}$, hence they converge almost-everywhere and
dominated to the function (6.25). It follows from LDCT Theorem for double-
sequences (see Lemma A.1), that the double-sequence of integrals of (6.23)
over $[0,1]\times[0,1]$ converges to

$H^{2}\int_{0}^{1}\int_{0}^{1}\left(u^{2H-1}+|v-u|^{2H-1}\operatorname{sgn}(v-u)\right)\left(v^{2H-1}+|u-v|^{2H-1}\operatorname{sgn}(u-v)\right)d(u,v).$
(6.26)

This integral, which is then the quasi-self-integral of
$K_{B_{H},B_{H}^{\prime}}^{(2)}$, can be computed using a long but elementary
calculus. The final result is

$\tilde{\int}_{[0,1]\times[0,1]}K_{B_{H},B_{H}^{\prime}}^{(2)}(x,dx)=\frac{1}{2}-H\mathbf{B}(2H,2H+1)-\frac{H}{2(4H-1)},$
(6.27)

where $\mathbf{B}$ is the beta function. From Theorems 5.1 and 5.3, the
stochastic integrals

$\int_{A}B_{H}(t)dB_{H}^{\prime}(t),\quad A\in\mathcal{B}([0,1])$ (6.28)

are uniquely-defined. In addition, $K_{B_{H},B_{H}^{\prime}}$ is positive, so
we are in the quite regular case of Corollary 5.2. The application
$A\mapsto\int_{A}B_{H}(t)dB_{H}^{\prime}(t)$ defines then a random measure
over $[0,1]$.

This property of fractional Brownian motion with $H>\frac{1}{2}$ makes a huge
contrast with the case of Browninan motion $(H=\frac{1}{2})$ briefly presented
in the proof of Proposition 4.1. At least in what concerns the definition of
simple stochastic integrals, the stochastic calculus with respect to $B_{H}$
is better behaved than with respect to $B$.

###  6.3 Processes whose derivatives are random measures

Let $Z$ and $M$ be such that $M$ is the (distributional) derivative of $Z$.
The example studied in Section 6.2 is a particular case, and other important
case is the one of Brownian motion and its derivative White Noise described
partially in the proof of Proposition 4.1. Let $D=[0,1]$. We can start from
$M$ and define

$Z(t):=M([0,t]),$ (6.29)

which is the only (distributional) primitive of $M$ such that
$Z(0)=M(\\{0\\})$. In general, such $Z$ is mean-square càdlàg (right-
continuous with left limits). Let us suppose, for simplicity, that $M$ is such
that $Z$ is mean-square continuous, hence $M$ must not have atoms
($M(\\{t\\})\stackrel{{\scriptstyle a.s.}}{{=}}0$ for every $t\in[0,1]$). The
cross-covariance kernel is

$K_{Z,M}(t,A)=C_{M}([0,t]\times A).$ (6.30)

In this case, Riemann sums approaching the self-integral are always bounded:

$\left|\sum_{j\in J_{n}}C_{M}((0,x_{j}^{n}],I_{j}^{n})\right|\leq\sum_{j\in
J_{n}}|C_{M}|(D\times I_{j}^{n})=|C_{M}|(D\times D).$ (6.31)

By taking a sub-sequence, one can always find a Riemann sum which converges,
but in general this does not imply self-integrability.

Let us consider an important particular case which is widely used in theory in
applications: $M$ is an orthogonal random measure. By this, we mean that $M$
has a covariance of the form

$C_{M}(A\times B)=\nu_{M}(A\cap B),\quad\forall
A,B\in\mathcal{B}_{B}(\mathbb{R}^{d}),$ (6.32)

being $\nu_{M}$ a positive measure over $\mathbb{R}^{d}$. Orthogonal random
measures take non-correlated values over disjoint subsets. Some authors
restrain the study of random measure to this class, sometimes with the
stronger requirement of independence in disjoint subsets (Kingman, 1967;
Passeggeri, 2020). Sadly, in this case the stochastic integral
$\int_{[0,1]}ZdM$ is not uniquely-defined. Indeed, in this case the cross-
covariance kernel (6.30) is given by $K_{Z,M}(t,A)=\nu_{M}([0,t]\cap A)$, and
the same problem as in Proposition 4.1 is present. Consider partitions of
$[0,1]$ consistent of left-closed intervals $(I_{j}^{n})_{j\in J_{n}}$, with
$a_{j}^{n}:=\inf I_{j}^{n}$ and $b_{j}^{n}:=\sup I_{j}^{n}$. Note that since
$M$ has no atoms, $\nu_{M}$ has no atoms as well. If we take as tag-points the
initial points $x_{j}^{n}=a_{j}^{n}$, one has

$\sum_{j\in J_{n}}K_{Z,M}(a_{j}^{n},I_{j}^{n})=\sum_{j\in
J_{n}}\nu_{M}([0,a_{j}^{n}]\cap I_{j}^{n})=\sum_{j\in
J_{n}}\nu_{M}(\\{a_{j}^{n}\\})=0.$ (6.33)

But using the uniform continuity of the function $t\mapsto\nu_{M}([0,t])$ over
$[0,1]$, one can always find, for every $n$, tag-points $(x_{j}^{n})_{j\in
J_{n}}$ such that $\sum_{j\in
J_{n}}\nu_{M}(I_{j}^{n}\cap(x_{j}^{n},b_{j}^{n}])\to 0$ as $n\to\infty$.
Hence,

$\sum_{j\in J_{n}}K_{Z,M}(x_{j}^{n},I_{j}^{n})=\sum_{j\in
J_{n}}\nu_{M}([a_{j}^{n},x_{j}^{n}])=\sum_{j\in
J_{n}}\nu_{M}(I_{j}^{n})-\nu_{M}(I_{j}^{n}\cap(x_{j}^{n},b_{j}^{n}])\xrightarrow[n\to\infty]{}\nu_{M}(D).$
(6.34)