# Canonical data-reconstructions via kernels, Hilbert space-valued Gaussian
processes, and quantum states

Palle E.T. Jorgensen  (Palle E.T. Jorgensen) Department of Mathematics, The
University of Iowa, Iowa City, IA 52242-1419, U.S.A.  palle-
<EMAIL_ADDRESS>and  James Tian  (James F. Tian) Mathematical Reviews, 416
4th Street Ann Arbor, MI 48103-4816, U.S.A<EMAIL_ADDRESS>
###### Abstract.

We offer new results and new directions in the study of operator-valued
kernels and their factorizations. Our approach provides both more explicit
realizations and new results, as well as new applications. These include: (i)
an explicit covariance analysis for Hilbert space-valued Gaussian processes,
(ii) optimization results for quantum gates (from quantum information), (iii)
new results for positive operator-valued measures (POVMs), and (iv) a new
approach/result in inverse problems for quantum measurements.

###### Key words and phrases:

Positive definite functions, kernels, Gaussian processes, covariance,
dilation, POVMs, PVMs, measurement, quantum states.

###### 2020 Mathematics Subject Classification:

46E22, 46N50, 47A20, 47B32, 47N50, 60G15, 81P15.

###### Contents

  1. 1 Introduction
  2. 2 Factorization of $\mathcal{L}\left(H\right)$-valued p.d. kernels
  3. 3 Relation to classical dilation theory
  4. 4 $H$-valued Gaussian processes
  5. 5 Completeness, POVMs, quantum states, and quantum information
    1. 5.1 An inverse problem

##  1\. Introduction

We present a new approach to inverse problems, combining classical and
quantum. Applications include new covariance analyses for Hilbert space-valued
Gaussian processes, and to a variety of dilation constructions. We further
present new results for optimization with quantum gates; a new approach/result
in inverse problems for quantum measurements, and to inference with positive
operator-valued measures (POVMs).

With our starting point being that of operator-valued positive definite
kernels, our present paper then proceeds to offer a new two-fold framework (i)
and (ii) for the following problems: one is that of Hilbert space and
representations, and the other, new classes of Gaussian processes. In more
detail: We present: (i) a new approach to the interplay between a variety of
systems which allow for realizations in specific and concrete Hilbert spaces
$H$, on the one hand, and which at the same time also admit extensions,
typically called dilations, in “larger” Hilbert spaces, say $L$. (By
“dilation,” we mean that the two systems are connected via an isometry $V$
from $H$ to $L$, which intertwines the respective systems/representations.)
And for (ii), we make use of these constructions in our analysis of Hilbert
space-valued Gaussian processes. The constructions we present in (i) are more
explicit than those in earlier related analyses, such as Gelfand Naimark Segal
(GNS), from states to representations; Stinespring, from completely positive
maps to representations; or Halmos, from powers contraction to unitary power
dilations; and finally from positive operator-valued measures (POVMs) to
projection valued measures. See, e.g., [DM20, PTD23, Ver22, BJ97, BJ99,
BJKW00, DJ12, DJ14, DHJ15, DJ15]. Our approach serves multiple purposes: one
is to unify theories, and the other is to make these constructions explicit.

Our new results include (i) an explicit covariance analysis for Hilbert space-
valued Gaussian processes (Section 4), (ii) optimization results for quantum
gates (Section 5), (iii) new results for positive operator-valued measures
(POVMs) (Sections 2–5), and (iv) a new approach/result in inverse problems for
quantum measurements (Section 5.1).

Notation. Throughout the paper, we use the physics convention that inner
products are linear in the second variable. Let
$\left|a\left\rangle\right\langle b\right|$ denote Dirac’s rank-1 operator,
$c\mapsto a\left\langle b,c\right\rangle$. $\mathcal{L}\left(H\right)$ denotes
the algebra of all bounded linear operators in a Hilbert space $H$.

For a positive definite (p.d.) kernel $K:S\times S\rightarrow\mathbb{C}$, let
$H_{K}$ be the corresponding reproducing kernel Hilbert space (RKHS). $H_{K}$
is the Hilbert completion of

$span_{\mathbb{C}}\left\\{K_{y}\left(\cdot\right):=K\left(\cdot,y\right)\mid
y\in S\right\\}$ (1.1)

with respect to the inner product

$\left\langle\sum_{i}c_{i}K\left(\cdot,x_{i}\right),\sum_{i}d_{j}K\left(\cdot,y_{j}\right)\right\rangle_{H_{K}}:=\sum_{i}\sum_{j}\overline{c}_{i}d_{j}K\left(x_{i},x_{j}\right).$
(1.2)

The following reproducing property holds:

$\varphi\left(x\right)=\left\langle
K\left(\cdot,x\right),\varphi\right\rangle_{H_{K}},\forall x\in
S,\>\forall\varphi\in H_{K}.$ (1.3)

Any scalar-valued kernel $K$ as above is associated with a zero-mean Gaussian
process, where $K$ is the covariance:

$K\left(s,t\right)=\mathbb{E}\left[\overline{W_{s}}W_{t}\right],$ (1.4)

and $W_{s}\sim N\left(0,K\left(s,s\right)\right)$.

An $\mathcal{L}\left(H\right)$-valued kernel $K:S\times
S\rightarrow\mathcal{L}\left(H\right)$ is p.d. if

$\sum_{i=1}^{n}\left\langle
a_{i},K\left(s_{i},s_{j}\right)a_{j}\right\rangle_{H}\geq 0$ (1.5)

for all $\left(a_{i}\right)_{1}^{n}$ in $H$, and all $n\in\mathbb{N}$.

For the theory of reproducing kernels, the literature is extensive, both pure
and recent applications, we refer to the following, including current, [AS56,
Aro48, AJ21, PR16, AFMP94, LYA23, CY17, Alp91, AD93, AL95, AB97, AD06, AJ15].

##  2\. Factorization of $\mathcal{L}\left(H\right)$-valued p.d. kernels

In this section, we present our universal construction in a general framework.
This will then, in adapted forms, go into a set of diverse applications. The
latter are addressed in subsequent sections.

In our analysis of kernels and factorizations we want to highlight the
following two distinctions: First is the distinction between the traditional
results which only yield existence of factorizations involving abstract and
unstructured completions, as opposed to arising as a universal and concrete
solution, realized directly from the given operator valued kernel as an
explicitly given space of functions. The second is the distinction between
universal, as opposed to case-by-case analyses. In both senses, our results
below fall in the category of universal factorizations.

######  Theorem 2.1 (Universal Factorization).

Let $K:S\times S\rightarrow\mathcal{L}\left(H\right)$ be a p.d. kernel. Then
there exists a RKHS $H_{\tilde{K}}$, and a family of operators
$V_{s}:H\rightarrow H_{\tilde{K}}$, $s\in S$, such that

$H_{\tilde{K}}=\overline{span}\left\\{V_{s}a:a\in H,s\in S\right\\}$ (2.1)

and

$K\left(s,t\right)=V_{s}^{*}V_{t}.$ (2.2)

Conversely, if there is a Hilbert space $L$ and operators $V_{s}:H\rightarrow
L$, $s\in S$, such that

$L=\overline{span}\left\\{V_{s}a:a\in H,s\in S\right\\}$ (2.3)

and (2.2) holds, then $L\simeq H_{\tilde{K}}$.

###### Proof.

First, assume that $K$ is of the form (2.2). For all
$\left(a_{i}\right)_{i=1}^{n}$ in $H$, and $\left(s_{i}\right)_{i=1}^{n}$ in
$S$,

$\displaystyle\sum_{i,j}\left\langle
a_{i},K\left(s_{i},s_{j}\right)a_{j}\right\rangle_{H}$
$\displaystyle=\sum_{i,j}\left\langle
a_{i},V_{s_{i}}^{*}V_{s_{j}}a_{j}\right\rangle_{H}$
$\displaystyle=\sum_{i,j}\left\langle
V_{s_{i}}a_{i},V_{s_{j}}a_{j}\right\rangle_{L}=\left\|\sum_{i}V_{s_{i}}a_{i}\right\|_{L}^{2}\geq
0.$

Thus, $K$ is p.d.

Now, given a p.d. kernel $K:S\times S\rightarrow\mathcal{L}\left(H\right)$,
let

$\tilde{K}:\left(S\times H\right)\times\left(S\times
H\right)\rightarrow\mathbb{C}$

be the scalar-valued kernel defined by

$\tilde{K}\left(\left(s,a\right),\left(t,b\right)\right):=\left\langle
a,K\left(s,t\right)b\right\rangle_{H}.$

Then $\tilde{K}$ is p.d. and let $H_{\tilde{K}}$ be the associated RKHS.

Set $V_{s}:H\rightarrow H_{\tilde{K}}$ by

$V_{s}a=\tilde{K}_{\left(s,a\right)}=\tilde{K}\left(\cdot,\left(s,a\right)\right):\left(S\times
H\right)\rightarrow\mathbb{C},\quad\forall a\in H.$

Since

$\left\langle
V_{s}a,\tilde{K}\left(\cdot,\left(t,b\right)\right)\right\rangle_{H_{\tilde{K}}}=\left\langle
a,K\left(s,t\right)b\right\rangle_{H},\quad\forall a,b\in H,\>\forall s,t\in
S,$

it follows that

$V_{s}^{*}\tilde{K}\left(\cdot,\left(t,b\right)\right)=K\left(s,t\right)b.$

One concludes that

$K\left(s,t\right)=V_{s}^{*}V_{t}.$

Finally, suppose there is a Hilbert space $L$ and family of operators
$V_{s}:H\rightarrow L$, $s\in S$, such that (2.2)–(2.3) hold. Then the mapping

$\tilde{K}\left(\cdot,\left(s,a\right)\right)\longmapsto V_{s}a$

extends (by linearity and density) to a unique isometric isomorphism from
$H_{\tilde{K}}$ onto $L$, and so any such space $L$ is isomorphic to the RKHS
$H_{\tilde{K}}$. ∎

######  Corollary 2.2.

Any function $F\in H_{\tilde{K}}$ satisfies that

$F\left(s,a\right)=\left\langle
a,V_{s}^{*}F\right\rangle_{H_{\tilde{K}}},\quad\left(s,a\right)\in S\times H.$
(2.4)

In particular, $F$ is conjugate liner in the second variable.

###### Proof.

This follows from the reproducing property (see (1.3)) of $H_{\tilde{K}}$:

$F\left(s,a\right)=\left\langle\tilde{K}\left(\cdot,\left(s,a\right)\right),F\right\rangle_{H_{\tilde{K}}}=\left\langle
V_{s}a,F\right\rangle_{H_{\tilde{K}}}=\left\langle
a,V_{s}^{*}F\right\rangle_{H}.$

∎

######  Corollary 2.3.

Let $\left(\varphi_{i}\right)$ be any orthonormal basis (ONB) in
$H_{\tilde{K}}$. The following operator identity holds:

$K\left(s,t\right)=\sum_{i}\left|V_{s}^{*}\varphi_{i}\left\rangle\right\langle
V_{t}^{*}\varphi_{i}\right|.$ (2.5)

###### Proof.

Using the identity in $H_{\tilde{K}}$,

$I_{H_{\tilde{K}}}=\sum_{i}\left|\varphi_{i}\left\rangle\right\langle\varphi_{i}\right|,$

we get that

$K\left(s,t\right)=V_{s}^{*}V_{t}=V_{s}^{*}I_{H_{\tilde{K}}}V_{t}=\sum_{i}\left|V_{s}^{*}\varphi_{i}\left\rangle\right\langle
V_{t}^{*}\varphi_{i}\right|.$

∎

##  3\. Relation to classical dilation theory

The notion of dilation theory dates back to the fundamental Hilbert space
axioms, including to the study of such basic notions in operator theory as
contractive operators in Hilbert space $H$, of states defined as positive
linear functionals (on algebras of operators), and of the notion and
properties of Stinespring-completely positive operator-valued maps. While the
early advances were motivated primarily by pure operator theory, and by
representation theory, recent studies have served to link much more directly
to modern quantum information, see also Section 5 below. The cast of
characters in the general context was (and is) perhaps focused on such areas
of dilation theory as: dilations for (i) subnormal operators, (ii) for
operator valued measures, (iii) for contraction operators, (iv) for operator
spaces, and their role as extensions in dilation theory, and (v) for commuting
sets of operators, and semigroups of completely positive maps. Early
applications include systems theory, prediction theory, and more recently
quantum information. The literature is vast, and here we call attention to
[Arv10], and the papers cited there. We now turn to the role our above results
(Section 2) play as tools in both classical and modern dilation theory; as
well as in sharpening the conclusions, and in presenting new directions.

Indeed, the setting in Theorem 2.1 is quite general. It includes and extends
well known constructions in classical dilation theory. Notably, passing from
$K$ to a scalar-valued p.d. kernel $\tilde{K}$ offers a function-based
approach to dilation theory, in contrast with traditional abstract spaces of
equivalence classes. It also allows for direct evaluation and manipulation of
functions, due to the RKHS structure of the dilation space, as well as the
associated $H$-valued Gaussian processes (Section 4), making it especially
relevant for practical applications.

For instance, consider a completely positive map $\varphi$ from a
$C^{*}$-algebra $\mathfrak{A}$ to the algebra of bounded operators on a
Hilbert space $H$. The Stinespring Dilation Theorem [Sti55] states that there
exists a Hilbert space $L$ (potentially larger than $H$), a representation
$\pi:\mathfrak{A}\rightarrow\mathcal{L}\left(L\right)$, and a bounded operator
$V:H\rightarrow L$, such that

$\varphi\left(A\right)=V^{*}\pi\left(A\right)V,\quad A\in\mathfrak{A}.$ (3.1)

Moreover, if $\varphi$ is unital, then $V$ is isometric and $H$ can be
identified as a subspace of $L$. The space $L$ is minimal if the span of
$\left\\{\pi\left(A\right)Va:a\in H,A\in\mathcal{L}\left(H\right)\right\\}$ is
dense in $L$, in which case $L$ is unique up to unitary equivalence.

A key step in the proof of Stinespring’s theorem is to introduce a
sesquilinear form on the algebraic tensor product $\mathfrak{A}\otimes H$,

$\left\langle\sum_{i}A_{i}\otimes a_{i},\sum_{j}B_{j}\otimes
b_{j}\right\rangle_{L}:=\sum_{i,j}\left\langle
a_{i},\varphi\left(A^{*}B\right)b_{j}\right\rangle_{H},$

and then setting $L$ to be the corresponding Hilbert completion, modulo
elements with zero norm, i.e., elements such that $\left\|\sum A_{i}\otimes
a_{i}\right\|_{L}=0$. The space $L$ is then an abstract Hilbert space of
equivalence classes.

To compare with Theorem 2.1, one may consider the
$\mathcal{L}\left(H\right)$-valued p.d. kernel

$K\left(A,B\right)=\varphi\left(A^{*}B\right)$

defined on $\mathfrak{A}\times\mathfrak{A}$, and define the scalar-valued p.d.
kernel $\tilde{K}:\left(\mathfrak{A}\times
H\right)\times\left(\mathfrak{A}\times H\right)\rightarrow\mathbb{C}$ by

$\tilde{K}\left(\left(A,a\right),\left(B,b\right)\right)=\left\langle
a,\varphi\left(A^{*}B\right)b\right\rangle_{H}.$

######  Corollary 3.1.

The space $L$ is then the RKHS $H_{\tilde{K}}$, consisting of functions $F$ on
$\mathfrak{A}\times H$, with the reproducing property:

$F\left(A,a\right)=\left\langle\tilde{K}\left(\cdot,\left(A,a\right)\right),F\right\rangle_{H_{\tilde{K}}}.$

Note that $H_{\tilde{K}}$ is a function space on $\mathfrak{A}\times H$, as
opposed to an abstract space of equivalence classes.

The operator $V$, and the representation $\pi$, are also explicit:

######  Lemma 3.2.

Set $V:H\rightarrow H_{\tilde{K}}$ by

$Va=\tilde{K}\left(\cdot,\left(I,a\right)\right).$

The adjoint $V^{*}:H_{\tilde{K}}\rightarrow H$ is given by

$V^{*}\tilde{K}\left(\cdot,\left(B,b\right)\right)=\varphi\left(B\right)b.$

Define $\pi:\mathfrak{A}\rightarrow\mathcal{L}\left(H_{\tilde{K}}\right)$ as

$\pi\left(A\right)\tilde{K}\left(\cdot,\left(B,b\right)\right)=\tilde{K}\left(\cdot,\left(AB,b\right)\right).$

Then (3.1) holds. Indeed, since, by definition,

$span\left\\{K\left(\cdot,\left(A,a\right)\right):A\in\mathfrak{A},a\in
H\right\\}$

is dense in $H_{\tilde{K}}$, $\left(\mathfrak{A},\pi,V,H_{\tilde{K}}\right)$
forms a minimal Stinespring dilation of $\varphi$.

###### Proof.

We verify the adjoint $V^{*}$ is as stated. Since the span of

$\left\\{\tilde{K}\left(\cdot,\left(A,a\right)\right):A\in\mathfrak{A},a\in
H\right\\}$

is dense in $H_{\tilde{K}}$, it is enough to check that

$\displaystyle\left\langle
Va,\tilde{K}\left(\cdot,\left(B,b\right)\right)\right\rangle_{H_{\tilde{K}}}$
$\displaystyle=\left\langle\tilde{K}\left(\cdot,\left(I,a\right)\right),\tilde{K}\left(\cdot,\left(B,b\right)\right)\right\rangle_{H_{\tilde{K}}}$
$\displaystyle=\left\langle a,\varphi\left(B\right)b\right\rangle_{H}$

which holds for all $a,b\in H$, and all $B\in\mathcal{L}\left(H\right)$. This
means
$V^{*}\tilde{K}\left(\cdot,\left(B,b\right)\right)=\varphi\left(B\right)b$,
for all $B\in\mathfrak{A}$, and all $b\in H$.

Then,

$\displaystyle V^{*}\pi\left(A\right)Va$
$\displaystyle=V^{*}\pi\left(A\right)\tilde{K}\left(\cdot,\left(I,a\right)\right)$
$\displaystyle=V^{*}\tilde{K}\left(\cdot,\left(A,a\right)\right)=\varphi\left(A\right)a$

which is (3.1).

Additionally, if $\varphi$ is unital, then

$\displaystyle\left\|Va\right\|_{H_{\tilde{K}}}^{2}$
$\displaystyle=\left\langle\tilde{K}\left(\cdot,\left(I,a\right)\right),\tilde{K}\left(\cdot,\left(I,a\right)\right)\right\rangle_{H_{\tilde{K}}}$
$\displaystyle=\left\langle a,K\left(I,I\right)a\right\rangle_{H}=\left\langle
a,\varphi\left(I\right)a\right\rangle_{H}=\left\langle
a,a\right\rangle_{H}=\left\|a\right\|_{H}^{2},$

so that $V$ is isometric. ∎

##  4\. $H$-valued Gaussian processes

Since the foundational work of Ito, the stochastic analysis of Gaussian
processes has relied heavily on Hilbert space and associated operators. For
reference, see works such as [Sch52, AJ15, AJ21] along with the early
literature cited therein. In this discussion, we highlight two distinct
scenarios: (i) scalar-valued Gaussian processes indexed by a Hilbert space,
often through an Ito-isometry; and (ii) Gaussian processes that take values in
a Hilbert space themselves, see e.g., [Lop23, Gue22, KM20, FS20, Kat19, JT23].
Our current focus is on the latter. This setting offers the advantage of
enabling more flexible representations of covariance structures, which are
increasingly relevant in the analysis of large data sets. We now proceed with
the construction details.

Let $S$ be a set, and $H$ a Hilbert space. We say
$\left\\{W_{s}\right\\}_{s\in S}$ is an $H$-valued Gaussian process if, for
all $a,b\in H$, and all $s,t\in S$,

$\mathbb{E}\left[\left\langle a,W_{s}\right\rangle_{H}\left\langle
W_{t},b\right\rangle_{H}\right]=C_{a,b}\left(s,t\right),$ (4.1)

where

$\left\langle W_{s},a\right\rangle_{H}\sim
N\left(0,C_{a,a}\left(s,s\right)\right),$ (4.2)

i.e., mean zero Gaussian, with variance $C_{a,a}\left(s,s\right)$.

######  Theorem 4.1.

Every operator-valued p.d. kernel $K:S\times
S\rightarrow\mathcal{L}\left(H\right)$ is associated with an $H$-valued
Gaussian process $\left\\{W_{s}\right\\}_{s\in S}$, such that

$\mathbb{E}\left[\left\langle a,W_{s}\right\rangle_{H}\left\langle
W_{t},b\right\rangle_{H}\right]=\left\langle
a,K\left(s,t\right)b\right\rangle_{H}.$

for all $s,t\in S$, and all $a,b\in H$.

Conversely, any $H$-valued Gaussian process $\left\\{W_{s}\right\\}_{s\in S}$
is obtained from such a p.d. kernel.

###### Proof.

Suppose $\left\\{W_{s}\right\\}_{s\in S}$ is an $H$-valued (mean zero)
Gaussian process realized in a probability space
$\left(\Omega,\mathbb{P}\right)$, with expectation
$\mathbb{E}\left(\cdot\right)=\int_{\Omega}\left(\cdot\right)d\mathbb{P}$.

Fix $s,t\in S$. For all $a,b\in H$,

$\left(a,b\right)\longmapsto\mathbb{E}\left[\left\langle
a,W_{s}\right\rangle_{H}\left\langle W_{t},b\right\rangle_{H}\right]$

is a bounded sesquilinear form on $H$, and so

$\mathbb{E}\left[\left\langle a,W_{s}\right\rangle_{H}\left\langle
W_{t},b\right\rangle_{H}\right]=\left\langle
a,K\left(s,t\right)b\right\rangle_{H}$

for some operator $K\left(s,t\right)\in\mathcal{L}\left(H\right)$. Also,

$K:S\times S\rightarrow\mathcal{L}\left(H\right)$

is positive definite, since

$\displaystyle\sum_{i,j=1}^{n}\left\langle
a_{i},K\left(s_{i},s_{j}\right)a_{j}\right\rangle_{H}$
$\displaystyle=\sum_{i,j=1}^{n}\mathbb{E}\left[\left\langle
a_{i},W_{s_{i}}\right\rangle_{H}\left\langle
W_{s_{j}},a_{j}\right\rangle_{H}\right]$
$\displaystyle=\mathbb{E}\left[\left|\sum_{i=1}^{n}\left\langle
W_{s_{i}},a_{i}\right\rangle_{H}\right|^{2}\right]\geq 0$

for all $a_{1},\cdots,a_{n}\in H$ and $s_{1},\cdots,s_{n}\in S$.

Conversely, given $K:S\times S\rightarrow\mathcal{L}\left(H\right)$ p.d.,
introduce the scalar-valued p.d. kernel $\tilde{K}:\left(S\times
H\right)\times\left(S\times H\right)\rightarrow\mathbb{C}$,

$\tilde{K}\left(\left(s,a\right),\left(t,b\right)\right)=\left\langle
a,K\left(s,t\right)b\right\rangle_{H}.$

Let $H_{\tilde{K}}$ be the RKHS of $\tilde{K}$, and let

$\left\\{W_{\left(s,a\right)}:\left(s,a\right)\in S\times H\right\\}$

be the associated Gaussian process (see (1.4), and also Theorem 4.3 for a
realization). That is,

$\tilde{K}\left(\left(s,a\right),\left(t,b\right)\right)=\left\langle
a,K\left(s,t\right)b\right\rangle_{H}=\mathbb{E}\left[\overline{W_{\left(s,a\right)}}W_{\left(t,b\right)}\right].$

Let $V_{s}:H\rightarrow H_{\tilde{K}}$ be given by
$V_{s}a=\tilde{K}\left(\cdot,\left(s,a\right)\right)$. Using the isometry

$H_{\tilde{K}}\ni
V_{s}a=\tilde{K}\left(\cdot,\left(s,a\right)\right)\longrightarrow
W_{\left(s,a\right)}\in L^{2}\left(\mathbb{P}\right)$

it follows that $H\ni a\rightarrow W_{\left(s,a\right)}$ is linear. The
assertion follows. ∎

######  Definition 4.2.

Consider the $H$-valued Gaussian process $W_{s}:\Omega\rightarrow H$,

$W_{t}=\sum_{i}\left(V_{t}^{*}\varphi_{i}\right)Z_{i},$ (4.3)

where $\left(\varphi_{i}\right)$ is an ONB in $H_{\tilde{K}}$. Following
standard conventions, here $\left\\{Z_{i}\right\\}$ refers to a choice of an
independent identically distributed (i.i.d.) system of standard scalar
Gaussians $N(0,1)$ random variables, and with an index matching the choice of
ONB.

######  Theorem 4.3.

We have

$\mathbb{E}\left[\left\langle a,W_{s}\right\rangle_{H}\left\langle
W_{t},b\right\rangle_{H}\right]=\left\langle
a,K\left(s,t\right)b\right\rangle_{H}.$ (4.4)

###### Proof.

Note that $\mathbb{E}\left[Z_{i}Z_{j}\right]=\delta_{i,j}$. From this, we get

$\displaystyle\text{LHS}_{\left(\ref{eq:d9}\right)}$
$\displaystyle=\mathbb{E}\left[\left\langle
a,W_{s}\right\rangle_{H}\left\langle W_{t},b\right\rangle_{H}\right]$
$\displaystyle=\sum_{i}\sum_{j}\left\langle
a,V_{s}^{*}\varphi_{i}\right\rangle\left\langle
V_{t}^{*}\varphi_{j},b\right\rangle\mathbb{E}\left[Z_{i}Z_{j}\right]$
$\displaystyle=\sum_{i}\left\langle
V_{s}a,\varphi_{i}\right\rangle_{H_{\tilde{K}}}\left\langle\varphi_{i},V_{t}b\right\rangle_{H_{\tilde{K}}}$
$\displaystyle=\left\langle
V_{s}a,V_{t}b\right\rangle_{H_{\tilde{K}}}=\left\langle
a,V_{s}^{*}V_{t}b\right\rangle_{H}=\left\langle
a,K\left(s,t\right)b\right\rangle_{H}.$

Also see (2.5). ∎

######  Remark 4.4.

The Gaussian process $\left\\{W_{t}\right\\}_{t\in S}$ in (4.3) is well
defined and possesses the stated properties. This is an application of the
central limit theorem to the choice $\left\\{Z_{i}\right\\}$ of i.i.d.
$N\left(0,1\right)$ Gaussians on the right-side of (4.3).

Varying the choices of ONBs $\left(\varphi_{i}\right)$ and i.i.d.
$N\left(0,1\right)$ Gaussian random variables $\left(Z_{i}\right)$ will result
in different Gaussian processes $\left\\{W_{t}\right\\}_{t\in S}$, but all
will adhere to the covariance condition specified in (4.4). It is also
important to note that if $\left(\varphi_{i}\right)$, $\left(Z_{i}\right)$ are
fixed, the resulting Gaussian process $\left\\{W_{t}\right\\}_{t\in S}$ is
uniquely determined by its first two moments.

##  5\. Completeness, POVMs, quantum states, and quantum information

In this section we apply our construction to the case of positive operator-
valued measures (POVMs). While POVMs have long played an important role in
operator theory, they have recently found applications in quantum information
[ANW17, EHD+13, CDP+13, KLSW12, KSW08, KW01, Wer01]. In more detail: For a
given Hilbert space $H$, and a measure space $\Omega$, a positive operator-
valued measure (POVM) on $\Omega$ is a sigma-additive measure whose values are
positive semi-definite operators on $H$. Thus the notion of POVM is a
generalization of projection-valued measures (PVMs). In quantum information,
the latter are called projection-measurements. (Here by a projection, we mean
selfadjoint projection $P$ in $H$.) Note that therefore quantum measurements
realized as POVMs are generalizations of quantum measurement described by
PVMs. By analogy, a POVM is to a PVM what a mixed state is to a pure state.
Recall that in quantum theory, mixed states specify those quantum states
occurring in subsystems of larger systems. Hence the study of purification of
quantum states. The POVMs describe the effect on a subsystem of a projection
measurement performed on a larger system. In other words, POVMs are the most
general kind of measurement in quantum mechanics, and they in the field of
quantum information. Here quantum information refers the information of states
of quantum systems, thus leading to a variety of quantum information
processing techniques. The literature is extensive. For current references we
here mention only a sample, [LK24, QY23, DKM24, HKLP24, WNGB24, FLZG24, Hol22,
RMC21, HCW21] and the papers cited there. Quantum information refers to both
the technical definition of von Neumann entropy as well as to the general
computational devices. Recently, quantum computing has become especially
active research area, serving as a key part of modern computation and
communication.

A main difference between classical probability, and its quantum counterpart
is played by the role of entangled quantum states, see e.g., [LKKK24, QLY+24,
FMBS24]. The latter in turn have their origin with the pioneers. But in the
past decade, they have come to play a crucial role in modern quantum
computing. (We make a note on their origin: Erwin Schrödinger, in studying the
Einstein-Podolsky-Rosen (EPR) state of two particles influencing each other
instantly at a distance, introduced the concept of entangled (quantum) states.
He used the German term “verschränkung”, which was subsequently translated
into English as entanglement.)

In quantum information theory, a measurement channel is a POVM $P$, defined on
a measurable space $\left(\Omega,\mathscr{B}_{\Omega}\right)$, acting in a
Hilbert space $H$. The set of quantum states (density operators)
$\mathcal{S}_{1}^{1}\left(H\right)$ consists of positive trace class operators
$\rho$ with $tr\left(\rho\right)=1$. A measurement channel $P$ is said to be
information-complete if

$\rho\mapsto tr\left(\rho P\left(\cdot\right)\right)$

from $\mathcal{S}_{1}^{1}\left(H\right)$ to the set of Borel probability
measures $\mathcal{M}_{1}\left(\Omega\right)$ on $\Omega$, is injective. In
other words, measurements uniquely determine states. In this context, an
$\mathcal{L}\left(H\right)$-valued p.d. kernel on $S\times S$ may be naturally
viewed as a generalized quantum measurement channel.

In what follows, let $\mathscr{K}\left(S,\mathcal{L}\left(H\right)\right)$ be
the set of all $\mathcal{L}\left(H\right)$-valued p.d. kernels on $S\times S$.
In particular, $\mathscr{K}\left(S,\mathbb{C}\right)$ is the set of all
scalar-valued p.d. kernels. $\mathcal{S}_{p}\left(H\right)$ denotes the $p$th
Schatten-class operators, with norm

$\left\|T\right\|_{p}=\left(\sum\nolimits_{n}s_{n}\left(T\right)^{p}\right)^{1/p}<\infty,$

where $s_{n}\left(T\right)$ are the singular numbers of $T$. Here,
$\mathcal{S}_{1}\left(H\right)$ is the trace class, and
$\mathcal{S}_{2}\left(H\right)$ the class of Hilbert-Schmidt operators.
$\mathcal{S}_{1}^{+}\left(H\right)$ is the cone of positive trace class
operators, and
$\mathcal{S}_{1}^{1}\left(H\right)=\left\\{T\in\mathcal{S}_{1}^{+}\left(H\right):tr\left(T\right)=1\right\\}$
consists of quantum states.

######  Lemma 5.1.

Fix $K\in\mathscr{K}\left(S,\mathcal{L}\left(H\right)\right)$. For all
$\rho\in\mathcal{S}_{1}^{+}\left(H\right)$, the kernel $c:S\times
S\rightarrow\mathbb{C}$,

$c\left(s,t\right):=tr\left(\rho K\left(s,t\right)\right)$

is positive definite, that is, $c\in\mathscr{K}\left(S,\mathbb{C}\right)$.

###### Proof.

Write

$\rho=\sum\lambda_{i}\left|\varphi_{i}\left\rangle\right\langle\varphi_{i}\right|,\quad\lambda_{i}>0,$

where $\left(\varphi_{i}\right)$ is an ONB in $H$. Then,

$\displaystyle\left(s,t\right)\longmapsto tr\left(\rho
K\left(s,t\right)\right)$
$\displaystyle=\sum\lambda_{i}\left\langle\varphi_{i},K\left(s,t\right)\varphi_{i}\right\rangle_{H}$

is a sum of scalar valued p.d. kernels, and so it is in
$\mathscr{K}\left(S,\mathbb{C}\right)$. ∎

######  Theorem 5.2.

Fix $K\in\mathscr{K}\left(S,\mathcal{L}\left(H\right)\right)$. Consider the
map
$\Psi:\mathcal{S}_{1}^{+}\left(H\right)\rightarrow\mathcal{K}\left(S,\mathbb{C}\right)$,
where

$\Psi\left(\rho\right)\left(s,t\right)=tr\left(\rho
K\left(s,t\right)\right),\quad\left(s,t\right)\in S\times S.$

Then $\Psi$ is 1-1 if and only if the double commutant
$\left\\{K\left(s,t\right)\right\\}^{\prime\prime}$ is weak* dense in
$\mathcal{L}\left(H\right)$.

###### Proof.

This follows from von Neumann’s double commutant theorem. Recall that
$\mathcal{S}_{1}^{+}\left(H\right)$ is the space of normal states, and it is
the predual of $\mathcal{L}\left(H\right)$. Therefore, the injectivity of
$\Psi$ in question is equivalent to the von Neumann algebra generated by
$\left\\{K\left(s,t\right):s,t\in S\right\\}$ being weak* dense in
$\mathcal{L}\left(H\right)$. ∎

######  Definition 5.3.

An $\mathcal{L}\left(H\right)$-valued p.d. kernel $K$ is said to be complete
if the map

$\mathcal{S}_{1}^{+}\left(H\right)\ni\rho\longmapsto tr\left(\rho
K\left(\cdot,\cdot\right)\right)\in\mathcal{K}\left(S,\mathbb{C}\right)$

is 1-1. That is, the scalar-valued kernel $\left(s,t\right)\mapsto
tr\left(\rho K\left(s,t\right)\right)$ on $S\times S$ uniquely determines
$K\in\mathscr{K}\left(S,\mathcal{L}\left(H\right)\right)$.

######  Corollary 5.4.

Fix $K\in\mathscr{K}\left(S,\mathcal{L}\left(H\right)\right)$. Let
$H_{\tilde{K}}$ be the RKHS of the corresponding scalar-valued kernel
$\tilde{K}$, defined on $\left(S\times H\right)\times\left(S\times H\right)$.
Let $\left\\{W_{t}\left(\cdot\right)\right\\}_{t\in S}$ be the associated
$H$-valued Gaussian process. Assume that $K$ is complete. Then the system of
vectors

$\left\\{V_{t}a=\tilde{K}\left(\cdot,\left(t,a\right)\right)\right\\}_{t\in
S}\subset H_{\tilde{K}}$

determines each $a\in H$ uniquely. Similarly, the scalar valued Gaussian
process

$\left\\{\left\langle W_{t},a\right\rangle_{H}\right\\}_{t\in S}\subset
L^{2}\left(\Omega,\mathbb{P}\right)$

determines each $a\in H$ uniquely.

###### Proof.

Let $a\in H$. Apply Theorem 5.2 to $\rho=\left|a\left\rangle\right\langle
a\right|\in\mathcal{S}_{1}^{+}\left(H\right)$, and use the fact that

$\left\langle
V_{s}a,V_{t}a\right\rangle_{H_{\tilde{K}}}=\mathbb{E}\left[\left\langle
a,W_{s}\right\rangle_{H}\left\langle
W_{t},a\right\rangle_{H}\right]=\left\langle
a,K\left(s,t\right)a\right\rangle_{H}=tr\left(\rho K\left(s,t\right)\right).$

∎

###  5.1. An inverse problem

In the theory of quantum measurement, a pertinent question is how to recover
the state of a system given measurement data. More precisely, for generalized
channels, one may consider the following:

######  Question 5.5.

Given a generalized channel
$K\in\mathscr{K}\left(S,\mathcal{L}\left(H\right)\right)$, and measurement
data $c\left(s,t\right)=tr\left(\rho K\left(s,t\right)\right)$, how to recover
the quantum state $\rho\in\mathcal{S}_{1}^{1}\left(H\right)$?

Alternatively, given $K$, $\left(s_{i}\right)_{i=1}^{N}\subset S$ and
$\left(c_{i,j}\right)_{i,j=1}^{N}\subset\mathbb{C}$, consider the following
optimization problem:

$\min\left\\{\sum_{i,j=1}^{N}\left|tr\left(\rho
K\left(s_{i},s_{j}\right)\right)-c_{i,j}\right|^{2}:\rho\in\mathcal{S}_{1}^{1}\left(H\right)\right\\}.$
(5.1)

Since every state $\rho\in\mathcal{S}_{1}^{+}\left(H\right)$ is a product of
Hilbert-Schmidt operators ($\rho=\left(\sqrt{\rho}\right)^{2}$), we will first
extend $\mathcal{L}\left(H\right)$-valued p.d. kernels to the case where
$H=\mathcal{S}_{2}$.

In fact, every $\mathcal{L}\left(H\right)$-valued p.d. kernel is also an
$\mathcal{L}\left(\mathcal{S}_{2}\right)$-valued kernel. Here,
$\mathcal{S}_{2}=\mathcal{S}_{2}\left(H\right)$ is the class of Hilbert-
Schmidt operators in $H$, which is a Hilbert space with inner product
$\left\langle A,B\right\rangle_{2}=tr\left(A^{*}B\right)$. More precisely, one
has:

######  Proposition 5.6.

$\mathscr{K}\left(S,\mathcal{L}\left(H\right)\right)\subset\mathscr{K}\left(S,\mathcal{L}\left(S_{2}\right)\right)$.

###### Proof.

Let $K\in\mathscr{K}\left(S,\mathcal{L}\left(H\right)\right)$. Note that
$K\in\mathscr{K}\left(S,\mathcal{L}\left(\mathcal{S}_{2}\right)\right)$ if and
only if

$\sum_{i,j}\left\langle
A_{i},K\left(s_{i},s_{j}\right)A_{j}\right\rangle_{2}=\sum_{i,j}tr\left(A_{i}^{*}K\left(s_{i},s_{j}\right)A_{j}\right)\geq
0,\quad\forall\left(A_{i}\right)_{i=1}^{N}\subset\mathcal{S}_{2}.$

It suffices to verify that

$\sum_{i,j}\left\langle\varphi,A_{i}^{*}K\left(s_{i},s_{j}\right)A_{j}\varphi\right\rangle_{H}\geq
0,\quad\forall\varphi\in H.$

This follows from the p.d. assumption of $K$, since

$\displaystyle\sum_{i,j}\left\langle\varphi,A_{i}^{*}K\left(s_{i},s_{j}\right)A_{j}\varphi\right\rangle_{H}$
$\displaystyle=\sum_{i,j}\left\langle
A_{i}\varphi,K\left(s_{i},s_{j}\right)A_{j}\varphi\right\rangle_{H}$
$\displaystyle=\sum_{i,j}\left\langle
h_{i},K\left(s_{i},s_{j}\right)h_{j}\right\rangle_{H}\geq 0,\quad
h_{i}:=A_{i}\varphi.$

∎

######  Remark.

Thus, there is an associated scalar-valued p.d. kernel
$\tilde{K}:\left(S\times\mathcal{S}_{2}\right)\times\left(S\times\mathcal{S}_{2}\right)\rightarrow\mathbb{C}$,
with

$\tilde{K}\left(\left(s,A\right),\left(t,B\right)\right)=\left\langle
A,K\left(s,t\right)B\right\rangle_{2}.$

And, an $\mathcal{S}_{2}$-valued Gaussian process $W$ satisfying that

$\mathbb{E}\left[\left\langle A,W_{s}\right\rangle_{2}\left\langle
W_{t},B\right\rangle_{2}\right]=\left\langle
A,K\left(s,t\right)B\right\rangle_{2}$

for all $s,t\in S$, and $A,B\in\mathcal{S}_{2}$.

Below, we analyze a simple case when $\dim H<\infty$, which reduces (5.1) to a
constraint kernel approximation (least-squares) problem.

Assume $H$ is finite-dimensional. Let

$K\left(s,t\right)=Q\left(s\cap t\right),\quad s,t\in\mathscr{B}_{\Omega},$
(5.2)

where
$Q:\left(\Omega,\mathscr{B}_{\Omega}\right)\rightarrow\mathcal{L}\left(H\right)$
is a POVM over a measurable space $\left(\Omega,\mathscr{B}_{\Omega}\right)$.
Recall that $Q\left(\Omega\right)=I_{H}$. Introduce $\tilde{K}$ and its RKHS
$H_{\tilde{K}}$ as follows:

$\displaystyle\tilde{K}\left(\left(s,A\right),\left(t,B\right)\right)$
$\displaystyle=\left\langle\tilde{K}_{\left(s,A\right)},\tilde{K}_{\left(t,B\right)}\right\rangle_{H_{\tilde{K}}}$
$\displaystyle=\left\langle A,Q\left(s\cap
t\right)B\right\rangle_{2}=tr\left(A^{*}Q\left(s\cap t\right)B\right),$

for all Borel sets $s,t\in\mathscr{B}_{\Omega}$, and all
$A,B\in\mathcal{S}_{2}\left(H\right)$ (i.e., all
$A,B\in\mathcal{L}\left(H\right)$, since $\dim H<\infty$).

Set $V:\mathcal{S}_{2}\left(H\right)\rightarrow H_{\tilde{K}}$ by

$VA=\tilde{K}\left(\cdot,\left(\Omega,A\right)\right).$

This is an isometric embedding of $\mathcal{S}_{2}\left(H\right)$ into
$H_{\tilde{K}}$, since

$\left\|VA\right\|_{H_{\tilde{K}}}^{2}=\left\langle\tilde{K}\left(\cdot,\left(\Omega,A\right)\right),\tilde{K}\left(\cdot,\left(\Omega,A\right)\right)\right\rangle_{H_{\tilde{K}}}=\left\langle
A,Q\left(\Omega\right)A\right\rangle_{2}=\left\|A\right\|_{2}^{2}.$

######  Lemma 5.7.

Given $\left(c_{i}\right)_{i=1}^{N}$ in $\mathbb{R}_{+}$, and
$\left(s_{i}\right)_{i=1}^{N}$ in $\mathscr{B}_{\Omega}$, we have

$\sum_{i}\left|tr\left(AQ\left(s_{i}\right)\right)-c_{i}\right|^{2}=\sum_{i}\left|\left\langle\tilde{K}_{\left(s_{i},I_{H}\right)},\tilde{K}_{\left(\Omega,A\right)}\right\rangle_{H_{\tilde{K}}}-c_{i}\right|^{2}.$

###### Proof.

Note that

$\displaystyle tr\left(AQ\left(s\right)\right)$
$\displaystyle=tr\left(I_{H}^{*}Q\left(s\right)A\right)$
$\displaystyle=\left\langle
I_{H},Q\left(s\right)A\right\rangle_{2}=\left\langle\tilde{K}_{\left(s,I_{H}\right)},\tilde{K}_{\left(\Omega,A\right)}\right\rangle_{H_{\tilde{K}}}.$

∎

######  Corollary 5.8.

In the current setting, (5.1) is converted to the following constraint kernel
approximation problem:

$\displaystyle\min\left\\{\sum_{i=1}^{N}\left|\left\langle\tilde{K}_{\left(s_{i},I_{H}\right)},\tilde{K}_{\left(\Omega,A\right)}\right\rangle_{H_{\tilde{K}}}-c_{i}\right|^{2}:A\in\mathcal{S}_{1}^{1}\left(H\right)\right\\}$
$\displaystyle\Updownarrow$
$\displaystyle\min\left\\{\sum_{i=1}^{N}\left|F\left(s_{i},I_{H}\right)-c_{i}\right|^{2}:F\in
V\left(\mathcal{S}_{1}^{1}\left(H\right)\right)\subsetneq
H_{\tilde{K}}\right\\}.$ (5.3)

###### Proof.

Recall that
$VA\left(\cdot\right)=\tilde{K}\left(\cdot,\left(\Omega,A\right)\right)$, so
that

$\left\langle\tilde{K}_{\left(s_{i},I_{H}\right)},\tilde{K}_{\left(\Omega,A\right)}\right\rangle_{H_{\tilde{K}}}=\left(VA\right)\left(s_{i},I_{H}\right)$

by the reproducing property of $H_{\tilde{K}}$. Then $F$ in (5.3) is of the
form $F=VA$, $A\in\mathcal{S}_{1}^{1}\left(H\right)$. ∎

######  Remark 5.9.

Kernel approximation methods are widely used in statistical learning for tasks
such as regression, classification, and density estimation. They are
particularly useful in settings where the underlying functional relationships
are complex and non-linear, and where a balance between fitting the training
data and maintaining generalizability to new data is crucial. The standard
formulation is as follows:

$\min\left\\{\sum_{i=1}^{N}\left|f\left(x_{i}\right)-y_{i}\right|^{2}+\lambda\left\|f\right\|_{H_{K}}^{2}:f\in
H_{K}\right\\}$ (5.4)

where $H_{K}$ is the RKHS of a scalar-valued kernel $K$ on some set $S\times
S$, $\left(y_{i}\right)\subset\mathbb{R}$, and $\lambda$ is a regularization
parameter.

A key difference between (5.3) and (5.4) is that, in (5.3), $F$ is restricted
to a convex subset, rather than being defined over the entire space
$H_{\tilde{K}}$. A standard method to solve (5.3) is to use projected gradient
descent.

######  Remark 5.10.

For a given POVM $Q$ as in (5.2), the construction of $H_{\tilde{K}}$ and $V$
in the above example can be used to show that $Q$ has a PVM dilation in
$H_{\tilde{K}}$. The verification is similar to that of the Stinespring
dilation theorem in Section 3 (also see [JT24, Section 2]).

######  Acknowledgement.

For numerous enlightening discussions, we are grateful to Prof Wayne Polyzou,
and to members of the Univ of Iowa weekly Math Physics seminar. We also
benefited from discussions with our other colleagues, including Profs Daniel
Alpay, Sergii Bezuglyi, Myung-Sin Song, Ilwoo Cho, and Shrey Sanadhya.

## References

  * [AB97] D. Alpay and V. Bolotnikov, _On tangential interpolation in reproducing kernel Hilbert modules and applications_ , Topics in interpolation theory (Leipzig, 1994), Oper. Theory Adv. Appl., vol. 95, Birkhäuser, Basel, 1997, pp. 37–68. MR 1473250
  * [AD93] Daniel Alpay and Harry Dym, _On a new class of structured reproducing kernel spaces_ , J. Funct. Anal. 111 (1993), no. 1, 1–28. MR 1200633
  * [AD06] D. Alpay and C. Dubi, _Some remarks on the smoothing problem in a reproducing kernel Hilbert space_ , J. Anal. Appl. 4 (2006), no. 2, 119–132. MR 2223568
  * [AFMP94] Gregory T. Adams, John Froelich, Paul J. McGuire, and Vern I. Paulsen, _Analytic reproducing kernels and factorization_ , Indiana Univ. Math. J. 43 (1994), no. 3, 839–856. MR 1305949
  * [AJ15] Daniel Alpay and Palle Jorgensen, _Spectral theory for Gaussian processes: reproducing kernels, boundaries, and $L^{2}$-wavelet generators with fractional scales_, Numer. Funct. Anal. Optim. 36 (2015), no. 10, 1239–1285. MR 3402823
  * [AJ21] Daniel Alpay and Palle E. T. Jorgensen, _New characterizations of reproducing kernel Hilbert spaces and applications to metric geometry_ , Opuscula Math. 41 (2021), no. 3, 283–300. MR 4302453
  * [AL95] Daniel Alpay and Juliette Leblond, _Traces of Hardy functions and reproducing kernel Hilbert spaces_ , Arch. Math. (Basel) 64 (1995), no. 6, 490–499. MR 1329822
  * [Alp91] Daniel Alpay, _Some reproducing kernel spaces of continuous functions_ , J. Math. Anal. Appl. 160 (1991), no. 2, 424–433. MR 1126127
  * [ANW17] Pablo Arrighi, Vincent Nesme, and Reinhard F. Werner, _Bounds on the speedup in quantum signaling_ , Phys. Rev. A 95 (2017), no. 1, 012331, 11. MR 3681182
  * [Aro48] N. Aronszajn, _Reproducing and pseudo-reproducing kernels and their application to the partial differential equations of physics_ , Harvard University, Graduate School of Engineering, Cambridge, MA, 1948, Studies in partial differential equations. Technical report 5, preliminary note. MR 31663
  * [Arv10] William Arveson, _Dilation theory yesterday and today_ , A glimpse at Hilbert space operators, Oper. Theory Adv. Appl., vol. 207, Birkhäuser Verlag, Basel, 2010, pp. 99–123. MR 2743416
  * [AS56] N. Aronszajn and K. T. Smith, _Functional spaces and functional completion_ , Ann. Inst. Fourier (Grenoble) 6 (1955/56), 125–185. MR 80878
  * [BJ97] Ola Bratteli and Palle E. T. Jorgensen, _Isometries, shifts, Cuntz algebras and multiresolution wavelet analysis of scale $N$_, Integral Equations Operator Theory 28 (1997), no. 4, 382–443. MR 1465320
  * [BJ99] by same author, _Iterated function systems and permutation representations of the Cuntz algebra_ , Mem. Amer. Math. Soc. 139 (1999), no. 663, x+89. MR 1469149
  * [BJKW00] O. Bratteli, P. E. T. Jorgensen, A. Kishimoto, and R. F. Werner, _Pure states on $\mathscr{O}_{d}$_, J. Operator Theory 43 (2000), no. 1, 97–143. MR 1740897
  * [CDP+13] Giulio Chiribella, Giacomo Mauro D’Ariano, Paolo Perinotti, Dirk Schlingemann, and Reinhard Werner, _A short impossibility proof of quantum bit commitment_ , Phys. Lett. A 377 (2013), no. 15, 1076–1087. MR 3038250
  * [CLPL19] Wen-Sheng Chen, Jingmin Liu, Binbin Pan, and Yugao Li, _Block kernel nonnegative matrix factorization for face recognition_ , Int. J. Wavelets Multiresolut. Inf. Process. 17 (2019), no. 1, 1850059, 28. MR 3911880
  * [CY17] Rupak Chatterjee and Ting Yu, _Generalized coherent states, reproducing kernels, and quantum support vector machines_ , Quantum Inf. Comput. 17 (2017), no. 15-16, 1292–1306. MR 3700848
  * [DHJ15] Dorin Ervin Dutkay, John Haussermann, and Palle E. T. Jorgensen, _Atomic representations of Cuntz algebras_ , J. Math. Anal. Appl. 421 (2015), no. 1, 215–243. MR 3250475
  * [DJ12] Dorin Ervin Dutkay and Palle E. T. Jorgensen, _Spectral measures and Cuntz algebras_ , Math. Comp. 81 (2012), no. 280, 2275–2301. MR 2945156
  * [DJ14] by same author, _Monic representations of the Cuntz algebra and Markov measures_ , J. Funct. Anal. 267 (2014), no. 4, 1011–1034. MR 3217056
  * [DJ15] by same author, _Representations of Cuntz algebras associated to quasi-stationary Markov measures_ , Ergodic Theory Dynam. Systems 35 (2015), no. 7, 2080–2093. MR 3394108
  * [DKM24] Ali Dadkhah, Mohsen Kian, and Mohammad Sal Moslehian, _Decomposition of tracial positive maps and applications in quantum information_ , Anal. Math. Phys. 14 (2024), no. 3, Paper No. 48. MR 4736299
  * [DM20] Nicolò Drago and Valter Moretti, _The notion of observable and the moment problem for $\ast$-algebras and their GNS representations_, Lett. Math. Phys. 110 (2020), no. 7, 1711–1758. MR 4114386
  * [EHD+13] Tobias Eberle, Vitus Händchen, Jörg Duhme, Torsten Franz, Fabian Furrer, Roman Schnabel, and Reinhard F. Werner, _Gaussian entanglement for quantum key distribution from a single-mode squeezing source_ , New J. Phys. 15 (2013), no. May, 053049, 14. MR 3081872
  * [FLZG24] Lingxuan Feng, Shunlong Luo, Yan Zhao, and Zhihua Guo, _Equioverlapping measurements as extensions of symmetric informationally complete positive operator valued measures_ , Phys. Rev. A 109 (2024), no. 1, Paper No. 012218, 10. MR 4705975
  * [FMBS24] Samuel Fedida, Anupam Mazumdar, Sougato Bose, and Alessio Serafini, _Entanglement entropy in scalar quantum electrodynamics_ , Phys. Rev. D 109 (2024), no. 6, Paper No. 065028. MR 4735667
  * [FS20] Martin Forde and Benjamin Smith, _The conditional law of the Bacry-Muzy and Riemann-Liouville log correlated Gaussian fields and their GMC, via Gaussian Hilbert and fractional Sobolev spaces_ , Statist. Probab. Lett. 161 (2020), 108732, 11. MR 4073554
  * [Gue22] J. C. Guella, _On Gaussian kernels on Hilbert spaces and kernels on hyperbolic spaces_ , J. Approx. Theory 279 (2022), Paper No. 105765, 36\. MR 4414825
  * [HCW21] Shan Huang, Zeng-Bing Chen, and Shengjun Wu, _Entropic uncertainty relations for general symmetric informationally complete positive operator-valued measures and mutually unbiased measurements_ , Phys. Rev. A 103 (2021), no. 4, Paper No. 042205, 9. MR 4256782
  * [HKLP24] Teiko Heinosaari, Oskari Kerppo, Leevi Leppäjärvi, and Martin Plávala, _Simple information-processing tasks with unbounded quantum advantage_ , Phys. Rev. A 109 (2024), no. 3, Paper No. 032627. MR 4732043
  * [Hol22] A. S. Holevo, _On optimization problem for positive operator-valued measures_ , Lobachevskii J. Math. 43 (2022), no. 7, 1646–1650. MR 4509531
  * [JT23] Palle E. T. Jorgensen and James Tian, _Operator-valued Gaussian processes and their covariance kernels_ , Infinite Dimensional Analysis, Quantum Probability and Related Topics 0 (2023), no. 0, 2350020.
  * [JT24] Palle E. T. Jorgensen and James Tian, _Hilbert space valued Gaussian processes, their kernels, factorizations, and covariance structure_ , arXiv: 2404.14685 (2024).
  * [JYZ21] Hao Jiang, Ming Yi, and Shihua Zhang, _A kernel non-negative matrix factorization framework for single cell clustering_ , Appl. Math. Model. 90 (2021), 875–888. MR 4160499
  * [Kat19] Victor Katsnelson, _On the completeness of Gaussians in a Hilbert functional space_ , Complex Anal. Oper. Theory 13 (2019), no. 3, 637–658. MR 3940383
  * [KLSW12] J. Kiukas, P. Lahti, J. Schultz, and R. F. Werner, _Characterization of informational completeness for covariant phase space observables_ , J. Math. Phys. 53 (2012), no. 10, 102103, 11. MR 3050559
  * [KM20] Vít Kubelka and Bohdan Maslowski, _Filtering of Gaussian processes in Hilbert spaces_ , Stoch. Dyn. 20 (2020), no. 3, 2050020, 22. MR 4101087
  * [KSW08] Dennis Kretschmann, Dirk Schlingemann, and Reinhard F. Werner, _A continuity theorem for Stinespring’s dilation_ , J. Funct. Anal. 255 (2008), no. 8, 1889–1904. MR 2462580
  * [KW01] M. Keyl and R. F. Werner, _Estimating the spectrum of a density operator_ , Phys. Rev. A (3) 64 (2001), no. 5, 052311, 5. MR 1878924
  * [LK24] A. Lebedev and A. Khrennikov, _Quantum-like modeling of the order effect in decision making: POVM viewpoint on the Wang-Busemeyer QQ-equality_ , Infinite dimensional analysis, quantum probability and related topics, QP–PQ: Quantum Probab. White Noise Anal., vol. 32, World Sci. Publ., Hackensack, NJ, [2024] ©2024, pp. 123–128. MR 4678079
  * [LKKK24] Su-Yong Lee, Jihwan Kim, Zaeill Kim, and Duk Y. Kim, _Asymmetric entanglement for quantum target sensing_ , Phys. Rev. A 109 (2024), no. 4, Paper No. 042429. MR 4740579
  * [Lop23] Oleh Lopushansky, _Bernstein-Jackson inequalities on Gaussian Hilbert spaces_ , J. Fourier Anal. Appl. 29 (2023), no. 5, Paper No. 58, 21. MR 4641110
  * [LYA23] Binglin Li, Changwon Yoon, and Jeongyoun Ahn, _Reproducing kernels and new approaches in compositional data analysis_ , J. Mach. Learn. Res. 24 (2023), Paper No. [327], 34. MR 4690276
  * [OCL24] Eoin O’Connor, Steve Campbell, and Gabriel T. Landi, _Fisher information rates in sequentially measured quantum systems_ , New J. Phys. 26 (2024), no. March, Paper No. 033048, 15. MR 4730881
  * [PPPR20] Satish K. Pandey, Vern I. Paulsen, Jitendra Prakash, and Mizanur Rahaman, _Entanglement breaking rank and the existence of SIC POVMs_ , J. Math. Phys. 61 (2020), no. 4, 042203, 14. MR 4085703
  * [PR16] Vern I. Paulsen and Mrinal Raghupathi, _An introduction to the theory of reproducing kernel Hilbert spaces_ , Cambridge Studies in Advanced Mathematics, vol. 152, Cambridge University Press, Cambridge, 2016. MR 3526117
  * [PTD23] J. E. Pascoe and Ryan Tully-Doyle, _Induced Stinespring factorization and the Wittstock support theorem_ , Results Math. 78 (2023), no. 4, Paper No. 135, 15. MR 4581177
  * [PTG15] Andrés Esteban Páez-Torres and Fabio A. González, _Online kernel matrix factorization_ , Progress in pattern recognition, image analysis, computer vision, and applications, Lecture Notes in Comput. Sci., vol. 9423, Springer, Cham, 2015, pp. 651–658. MR 3487535
  * [QL17] Likuan Qin and Vadim Linetsky, _Long-term factorization of affine pricing kernels_ , Math. Financ. Econ. 11 (2017), no. 4, 479–498. MR 3709384
  * [QLY+24] Ji Qi, Ke Li, Zhe Yang, Rui-Yang Yuan, and Bao-Cang Ren, _Error identification entanglement purification for stationary system using high-dimensional entanglement_ , Phys. Rev. A 109 (2024), no. 4, Paper No. 042423. MR 4740573
  * [QY23] Minglong Qin and Penghui Yao, _Decidability of fully quantum nonlocal games with noisy maximally entangled states_ , 50th International Colloquium on Automata, Languages, and Programming, LIPIcs. Leibniz Int. Proc. Inform., vol. 261, Schloss Dagstuhl. Leibniz-Zent. Inform., Wadern, 2023, pp. Art. No. 97, 20. MR 4612967
  * [RMC21] Benjamin Robinson, Bill Moran, and Doug Cochran, _Positive operator-valued measures and densely defined operator-valued frames_ , Rocky Mountain J. Math. 51 (2021), no. 1, 265–272. MR 4280112
  * [Sch52] Laurent Schwartz, _Théorie des noyaux_ , Proceedings of the International Congress of Mathematicians, Cambridge, Mass., 1950, vol. 1, Amer. Math. Soc., Providence, RI, 1952, pp. 220–230. MR 45307
  * [Sch64] by same author, _Sous-espaces hilbertiens d’espaces vectoriels topologiques et noyaux associés (noyaux reproduisants)_ , J. Analyse Math. 13 (1964), 115–256. MR 179587
  * [Shi05] Serguei Shimorin, _Commutant lifting and factorization of reproducing kernels_ , J. Funct. Anal. 224 (2005), no. 1, 134–159. MR 2139107
  * [Sti55] W. Forrest Stinespring, _Positive functions on $C^{*}$-algebras_, Proc. Amer. Math. Soc. 6 (1955), 211–216. MR 69403
  * [Ver22] Dominic Verdon, _A covariant Stinespring theorem_ , J. Math. Phys. 63 (2022), no. 9, Paper No. 091705, 49. MR 4482713
  * [Wer01] R. F. Werner, _All teleportation and dense coding schemes_ , vol. 34, 2001, Quantum information and computation, pp. 7081–7094. MR 1863141
  * [WNGB24] Elisabeth Wagner, Ramil Nigmatullin, Alexei Gilchrist, and Gavin K. Brennen, _Information flow in non-unitary quantum cellular automata_ , SciPost Phys. 16 (2024), no. 1, Paper No. 014, 41. MR 4710381


# Variational Auto-Encoder Based Deep Learning Technique For Filling Gaps in
Reacting PIV Data

Shashank Yellapantulaa,∗
aNational Renewable Energy Laboratory, Golden, CO, USA

###### Abstract

In this study, a deep learning based conditional density estimation technique
known as conditional variational auto-encoder (CVAE) is used to fill gaps
typically observed in particle image velocimetry (PIV) measurements in
combustion systems. The proposed CVAE technique is trained using time resolved
gappy PIV fields, typically observed in industrially relevant combustors.
Stereo-PIV (SPIV) data from a swirl combustor with very a high vector yield is
used to showcase the accuracy of the proposed CVAE technique. Various error
metrics evaluated on the reconstructed velocity field in the gaps are
presented from data sets corresponding to three sets of combustor operating
conditions. In addition to accurate data reproduction, the proposed CVAE
technique offers data compression by reducing the latent space dimension,
enabling the efficient processing of large-scale PIV data.

Keywords: VAE; PIV; Deep Learning; CNN; Combustor

Information for Colloquium Chairs and Cochairs, Editors, and Reviewers

1) Novelty and Significance Statement

The following are the novel contributions made by this study to advance
techniques for accurately filling large gaps in reacting PIV data:

  1. 1.

First of a kind application of a generative ML technique to fill gaps in
reacting PIV data. Technique solely relies on time varying vector field in the
valid regions.

  2. 2.

Usage of partial convolution operator to map spatially fine flow features,
solely in the valid regions, to network parameters.

  3. 3.

Novel use of conditioning variables to uniquely identify and reproduce
distinct flow features in each of the snapshots.

  4. 4.

Demonstrate the superior accuracy of CVAE in reproducing vector field in the
gaps even in extreme cases where 75% of the vectors were eliminated from the
training dataset.

This paper presents significant research that addresses a long-standing
challenge in processing gaps in large reacting PIV datasets commonly found in
industrially relevant combustion systems.

2) Author Contributions

The first and the only author designed and performed the research along with
writing and editing of the paper.

3) Authors’ Preference and Justification for Mode of Presentation at the
Symposium

The author prefers OPP presentation at the Symposium, for the following
reasons:

  * •

This study will benefit both the experimentalists and the modelers due to the
accurate reconstruction and data compression capability.

  * •

Presentation of a novel algorithm not applied previously in the experimental
diagnostics community.

  * •

This study presents a significant improvement over the most popular algorithms
for reacting PIV data processing.

##  1 Introduction

Particle image velocimetry (PIV) measurements in complex combustor flows often
suffer from low signal to noise ratio. This can be attributed to several
factors including flame luminosity, wide density variations and complex flow
structures leading to non-uniform particle distributions that lead to non-
uniform particle distributions. These factors can cause gaps in the velocity
vector field. This issue is further exacerbated at high pressures
representative of complex industrial combustion systems. Several techniques,
including gappy proper orthogonal decomposition (GPOD) [1] have been proposed
in literature to reconstruct the velocity field in these gaps. In this above
referenced study, authors proposed a median filter (MF) based GPOD technique
and compared the superior performance of MF GPOD method against some of the
existing techniques such as the method by Gunes [2], Raben [3] and the
original GPOD algorithm [4]. These techniques have been shown to perform very
well in certain conditions. However, they seem to struggle for cases where
large chunks of data were missing. Recently, studies focusing on improving the
GPOD techniques using spatial and temporal correlations have been proposed
[5]. Additionally, some machine learning based techniques have also been
proposed [6] where the authors used supervised machine learning techniques
trained on artificial particle images (APIs) and direct numerical simulation
(DNS) data to develop new data reconstruction method for PIV. Another very
recent study [7] proposed a technique that unifies deep neural networks and
GPOD for fluid and thermal reconstruction.

In the current study a deep learning-based generative technique called the
variational autoencoder (VAE) is proposed for PIV gap filling. This technique
leverages Bayesian inference to effectively handle large training data with
large gaps in vector field and achieve superior reconstruction accuracy. This
flavor of VAE with conditioning data, also known as Conditional Variational
Auto-Encoder (CVAE), is trained on the temporally evolving gappy vector field,
enabling it to infer the reconstructed field in the gaps. This approach
outperforms the above mentioned techniques by demonstrating higher
reconstruction accuracy even in extreme cases where 75% of vectors are
eliminated from the training snapshots.

##  2 Methodology

Gappy POD [1], as with all other forms of POD, is a dimension reduction
technique with both an encoding and a decoding component. In the POD encoder
stage, the high-dimensional data is projected onto the POD modes. This
projection results in coefficients that provide the contribution of each mode.
These coefficients form the latent space which is the low-dimensional
representation of the original high-dimensional data. During the decoding
phase, linear combination of these POD modes are employed to reconstruct the
original high-dimensional data. By varying the number of coefficients and
associated POD modes, the original data can be approximated with varying
levels of accuracy. Variational auto-encoder is a similar although a non-
linear approach to dimension reduction with some specific properties imposed
on the latent space.

###  2.1 Conditional Variational Auto-encoder (CVAE)

In the current study, a generative deep Learning Model known as Variational
auto-encoder (VAE) [8] is used to learn and predict the background
distribution of the velocity field and provide an approximation to the
velocity field in the gaps. A comprehensive and a detailed description of
underlying variational and the deep learning techniques are beyond the scope
of this article and the reader is referred to [8] and [9] for a detailed
discussion. In the following a brief description of how these methods are
implemented in the context of development of a technique to fill gaps in
reactive PIV data will be presented.

Traditionally, autoencoders (AE) have been used for dimension reduction and
therefore for data compression purposes. In this regard, AE is analogous to
POD. One critical difference between AE and POD is that AE is an inherently
non-linear technique with both encoder and decoder typically represented by
deep neural networks, fully connected layers and non-linear activation
functions. The non-linear capability within AE allows this technique to learn
representations of the input data in lower dimensions with minimal
reconstruction loss. However, the latent spaces with AE are low dimensional
vector spaces lacking any regularization, not necessarily continuous and
without any ability to interpolate within the latent space. Lack of
interpolation property within the latent space leads to AE reconstructing
incorrect data if the latent space regions away from regions corresponding to
input data are fed to the decoder network. For the current use case of
training datasets with gaps, AE latent space will likely suffer from
information loss regarding data in the gaps. This constraint with AE is
removed in VAEs where the high-dimensional input data is encoded to a
distribution over the latent space. Thereafter, the latent space distribution
is sampled and decoded to generate an approximation to the original high-
dimensional data. A regularization term, imposing constraints of completeness
and continuity over the latent space, is added to the mean square
reconstruction loss between the original and the decoded data. In practice,
this regularization is implemented by enforcing latent space distributions to
be close to a high-dimensional gaussian distribution with unit norm. In order
to compute the parameters of the encoder and the decoder network this loss
function is backpropagated using stochastic gradient descent. To allow for
gradient descent through a distribution with a random variable, the
reparameterization trick is applied [8]. The loss function can be summarized
as the following

$\mathcal{L}=-KL\left[\mathcal{N}(\mathbf{\mu}_{x},\mathbf{\sigma}_{x})||\mathcal{N}(0,I)\right]+MSE(\mathbf{x},\mathbf{x}_{d})$
(1)

where KL is the Kullback-Leibler divergence, MSE is mean square error,
$\mathbf{x}$ is the original data and $\mathbf{x}_{d}$ is the decoded data.

One novel step introduced in this study is to condition every snapshot in the
input dataset (temporally resolved vector field with gaps at every time
instant) with a few specific variables computed using the gappy field at that
time instant. Conditioning variables are generated to control the output data
at every instant and generate the most plausible vector field in the gaps
taking into account specific characteristics in the training snapshot at that
corresponding instant. This flavor of VAE is commonly referred to as
conditional variational auto-encoder (CVAE) [9]. During the training process
these conditioning variables are input to both the encoder and decoder. During
the prediction stage these conditioning variables along with a sample from the
latent space distribution are fed to the decoder to generate approximate
samples of full vector field at every instant.

The next section delves into implementation details of each of the component
of the CVAE employed in this study.

Fig. 1: Schematic of the CVAE network used in this study. Forward evaluation
loop is highlighted with a box with dashed edges.

###  2.2 Encoder and Decoder network

The encoder and the decoder networks in the CVAE are constructed using a
series of convolutional neural networks (CNN) and a couple of fully connected
layers. CNNs are well-suited for recognizing specific patterns in the input
data. In order to accommodate the irregularly shaped gaps in the PV data, a
technique known as partial convolutions [10] is used to modify the convolution
operators. This critical step ensures that only the valid data are used for
training the model. The gappy regions are masked off and are not utilized in
the training. Figure 1 shows the network used in this study. A 3$\times$3
kernel was used to detect spatially fine features. Due to the gaps in the
training dataset, partial convolution operators are used in the encoder
network. The decoder network contains the regular CNN operators. Batch
normalization [11] and rectified linear unit (ReLU) [12] activation function
along with fully connected layers are also shown in Fig. 1. For all the
results presented in this study, a 10 dimensional latent space was used. This
parameter and all the other hyperparameters related to training process such
as learning rates were optimized using SHERPA, a python based hyperparameter
optimization library [13].

####  2.2.1 Partial convolution

As mentioned in the previous section, only the data from valid regions are
used to train the variational auto-encoder (VAE). The spatial regions where
gaps were present are replaced with zeros in each of the snapshot of the PIV
dataset. Additionally a binary mask same size as the original data is used to
denote regions where vectors are eliminated. In order to not use the
eliminated vectors for training, partial convolutional filters [10], instead
of regular convolutional filters, are utilized in this study. With partial
convolutions, provided a binary mask, only the non-masked valid regions are
convolved at each layer of the network. Additionally, a mask update step is
performed at each layer providing regions of unmasked values for the next
layer. As described in [10], this partial convolution algorithm, relying only
on unmasked values, can be expressed as

$x^{\prime}=\begin{cases}\mathbf{W}^{T}\left(\mathbf{X}\odot
M\right)\frac{sum(l)}{sum(M)}+b,\\\
~{}~{}~{}~{}~{}~{}~{}~{}~{}if~{}~{}\mathbf{sum}(M)>0\\\
0,~{}~{}~{}otherwise\end{cases}$ (2)

where $\mathbf{W}$ are the convolution weights, $\mathbf{X}$ are the input
data, $M$ are the binary masks, $\odot$ implies element-wise multiplication.
The matrix $l$ has the same shape as $M$ but with all elements equal to 1 to
provide the right scaling.

###  2.3 Conditioning variables

From each sample in the temporally resolved PIV dataset, a set of 10
conditioning variables are generated. These conditioning variables only
utilize the valid vector field. These conditioning variables are designed to
uniquely tag each temporal snapshot of the dataset.

$\displaystyle\mathbf{C}^{i}$ $\displaystyle=\begin{bmatrix}\Delta t*i\\\
{}^{min}U_{x}^{i}\\\ {}^{max}U_{x}^{i}\\\ {}^{min}U_{y}^{i}\\\
{}^{max}U_{y}^{i}\\\ {}^{min}U_{z}^{i}\\\ {}^{max}U_{z}^{i}\\\ \sum
U_{x}^{i}\left(x,y\right)\\\ \sum U_{y}^{i}\left(x,y\right)\\\ \sum
U_{z}^{i}\left(x,y\right)\end{bmatrix}$ (3)

where $\Delta t$ is the temporal resolution of the dataset, $U_{1}^{i}$,
$U_{2}^{i}$ and $U_{3}^{i}$ are x, y and z components of $i^{th}$ snapshot,
and ${}^{min}U_{1}^{i}$ is the minimum of the x component of the $i^{th}$
sample gappy vector field.

###  2.4 Loss Function

Equation 1 describes the loss function used in this study for backpropagation
using stochastic gradient descent (SGD). The reconstruction loss,
$MSE(\mathbf{x},\mathbf{x}_{d})$, is only calculated over the valid vector
field which are tagged using the binary mask, $M$. The Adam optimizer [14], a
variant of SGD, is used in this study. A learning rate of $7.5e-4$ was used
network training.

###  2.5 Training and Forward evaluation

For each PIV dataset with gaps, a tailored CVAE network is trained. Once
trained, only a forward evaluation step is required. During training, all time
snapshots are randomly shuffled and grouped into batches of 32. The
corresponding conditioning variables are also computed and shuffled in the
same order. The encoder, utilizing multi-layered CNNs and incorporating
conditioning variables, aims to map the increasingly complex spatial patterns
in each snapshot, independent of the temporal order, to a unique latent space
region. In all the datasets presented in the paper a total of 1000 epochs were
used for training the respective models. The validation loss was found to
reduce by 2 orders of magnitude within 1000 epochs.

For a forward evaluation, the latent space (unit normal gaussian) distribution
is first sampled and is passed to the decoder network along with the
conditioning variables. For each snapshot, 32 independent evaluations are made
and a mean was computed over these 32 samples to generate a vector field at
that instant. The decoder network will generate a full vector field. However,
only the vector field in the gaps are retained. The remaining regions, away
from the gaps, are patched with velocity field from the training dataset.

###  2.6 Implementation Details

The python based software code for the algorithm described in this study is
implemented using the Pytorch library [15]. The code is designed to use GPUs,
whenever available, for efficient computation. The forward evaluation to
generate 5120 snapshots using the decoder network took 0.1 hours on a NVIDIA
Tesla V100.

##  3 Experimental Data

In order to evaluate the CVAE based algorithm to fill gaps, a PIV dataset with
high vector yield and low uncertainty is chosen. Similar to the procedure
described in [1], gaps with varying pixel widths are created in the data and
vectors in these gaps are completely eliminated. The performance of the CVAE
algorithm to fill these gaps is then investigated by comparing against
original data without any gaps.

###  3.1 Premixed Swirl Burner

In this study, Stereo-PIV data from the experimental study of the premixed
swirl burner [16, 17, 18] is considered. The experimental setup, shown in Fig.
2, was equipped with simultaneous 10 kHz OH planar laser induced fluorescence
(PLIF) and stereoscopic particle image velocimetry (S-PIV) diagnostic. The
experimental diagnostics and data processing techniques are discussed in
detailed in [16] and will not be discussed in this paper.

Fig. 2: Schematic of the experimental configuration [16]

Three different cases corresponding to three different operating conditions
were considered in this study. The following table lists out the details of
each of these cases. A total of 5120 snapshots for each of the test case was
used to train the CVAE network for each of the cases.

Table 1: Combustor operating conditions for all the cases considered in this
study

Case name | $T_{ad}$ | $T_{r}$ | $D_{r}$ | $\phi$ | $X_{air}$ | $X_{CH_{4}}$ | $X_{H_{2}}$
---|---|---|---|---|---|---|---
D2F3 | 1750 K | 400K | 4.4 | 0.6 | 0.93 | 0.055 | 0.014
D2F1 | 1750K | 400K | 4.4 | 0.6 | 0.94 | 0.060 | 0.0
D3F3 | 1700K | 500K | 3.4 | 0.53 | 0.94 | 0.049 | 0.012

The D in the case name refers to the reactant-to-product density ratio. This
parameter is one of the critical parameters impacting hydrodynamic
instabilities leading to distinct velocity fields. In this study, two cases
with different density ratios, D2 and D3, were considered. The letter F in the
case name is to denote the H2 addition in the fuel stream. The H2 fuel
addition was shown to play a critical role in increasing flame stability in
the experimental study using this rig [16]. For all the three cases considered
in this study, a complex flow field was observed, Fig. 3. The flow field is
characterized by the swirling reactant jet, inner and outer shear layers along
with a central recirculation zone (CRZ). For a detailed discussion regarding
the flow features and flame stabilization in this burner, the reader is
referred to [16]. The time resolved SPIV (3 velocity components on a 2D plane)
data in these experimental measurements had a very high vector yield with low
uncertainty. As is typically the case with S-PIV data the out of plane
velocity component, $U_{z}$, was found to have the highest uncertainty. Owing
to the high level of confidence in the vector field obtained from this
experimental study, this dataset was selected for studying the efficacy of
CVAE in generating velocity fields within gaps and comparing them to the
original data in those artificial gaps. The algorithm for creating these
artificial gaps will be discussed further.

Fig. 3: Time averaged velocity field from case D2F3.

###  3.2 Algorithm for vector elimination

Only 50% and 75% gappiness percentage (G), ratio of missing vectors to the
total number of possible vectors, are considered in this study. For the
G$\approx$50% case the total surface area of the gaps in the time-resolved
data are distributed across all snapshots following a Gaussian profile in
time, with the mean value of gappiness denoted by G. In each snapshot, these
artificial gaps are randomly distributed throughout the spatial domain. The
algorithm used in the gappy POD study [1] had a restriction of no single point
in the field was missing for all snapshots. This constraint has been removed
in the gap creation algorithm used in this study. Figure 4 shows the
distribution of the % of vector eliminated from each of the snapshot spread
across the entire 5120 snapshots in the dataset from each of the three cases.

Fig. 4: Probability density function (PDF) of % of vectors eliminated in each
of the 5120 images. Red line corresponds to the case D2F1, blue line is from
case D2F3 and black line corresponds to case D3F3.

##  4 Results and Discussion

To evaluate the effectiveness of the proposed CVAE algorithm, three distinct
networks were trained for each of the operating conditions, Tab. 1. For each
of the operating condition, a distinct CVAE network is trained using a dataset
with a mean G$\approx$50%, Fig. 4. The efficacy of CVAE will also be assessed
on a test case with G$\approx$75% and will be presented at the end of this
section. Such a significant data loss in a realistic PIV dataset will warrant
a significant change in the experimental methodology along with
instrumentation design and therefore will rarely be seen. However, this study
is focused on stretching the limits of the proposed CVAE technique and
providing a usable guide to the user to help them make an informed judgement
on CVAE for gap filling. For this reason a case with G$\approx$75% is also
being considered in this study. In terms of metrics to evaluate the
performance of the proposed technique, first a qualitative comparison between
the original 2D vector field and the reconstructed field will be shown.
Thereafter, reconstruction error (E), defined as below and also used in [1]
will be presented.

$E=\sqrt{\sum(U^{r}-U^{o})^{2}/\sum(U^{o})^{2}}$ (4)

Statistics of spatially local relative error,
$\mid\frac{U^{r}-U^{o}}{U^{o}}\mid$ where $U^{r}$ is the reconstructed field
and the original field is represented by $U^{o}$, will also be presented.

Figures 5, 6 and 7 shows the original clean vector field, vector field with
gaps and the reconstructed vector field using the CVAE technique for all the
three operating conditions, respectively. The snapshots shown in Fig. 5, 6 and
7 correspond to a time instant where 55%, 54% and 58%, respectively, of
vectors were fully eliminated.

Fig. 5: Instantaneous vector plot showing the original vector field, vector
field with 55% of vectors eliminated, CVAE reconstructed vector field from the
case D2F1.

Qualitatively, all the main flow features eliminated during the gap creation
was reconstructed accurately using the CVAE strategy. Despite the elimination
of large chunks of vector field along swirling arms of the reactant jet in
Fig. 6, CVAE is able to reproduce these flow features effectively. In Figs. 5
and 7 some of the low speed vortical structures, eliminated in the training
data, are successfully reconstructed by the CVAE.

Fig. 6: Instantaneous vector plot showing the original vector field, vector
field with 54% of vectors eliminated, CVAE reconstructed vector field from the
case D2F3. Fig. 7: Instantaneous vector plot showing the original vector
field, vector field with 58% of vectors eliminated, CVAE reconstructed vector
field from the case D3F3.

Figure 8 shows the distribution of reconstruction error, E, for all three
components of velocity over all the snapshots for each of the test case. The
reconstruction error, Eq. 4 without the term in the denominator, is used in
the loss function (Eq. 1) used to train the network. Figure 8 shows that
$U_{y}$ velocity component has the lowest error amongst all the components
followed by $U_{x}$ and $U_{z}$. This trend is similar to the trend presented
in [1]. The higher uncertainty and the noise in the out-plane velocity
components in SPIV [19] was attributed by the GPOD study [1] as main cause of
higher error in the $U_{z}$ component. By comparing the reconstruction errors
obtained by CVAE against the various GPOD variants presented in [1], it is
evident that CVAE achieves lower reconstruction errors. Moreover, this
superior performance of CVAE is attained using a training dataset with a
reduced number of valid vectors. To analyze the sources of these errors
distribution of spatially local relative error defined as,
$\mid\frac{U^{r}_{x}-U^{o}_{x}}{U^{o}_{x}}\mid$, is shown in Fig. 9. This
distribution of relative error, in the $U_{x}$ component, is computed over all
points and over all the snapshots. To highlight the regions where this error
is observed to be the highest, the relative error is conditioned on the local
$U_{x}$. Figure 9 shows that the regions with higher velocity have smaller
relative errors whereas the regions with negligible velocities dominate the
relative error metric. For the highest velocity zones CVAE predicted a
relative error $<0.2$ in the $U_{x}$ velocity component. The same trend in
relative error is observed for all the cases.

(a)  (b)

(c)

Fig. 8: PDF, computed over all the snapshots, of the reconstruction error, E
from the three test cases considered in this study.(a) Case D2F1, (b) Case
D2F3, (c) Case D3F3

(a)  (b)

(c)

Fig. 9: PDF, computed over an entire snapshot and over all the snapshots in
the dataset, of the relative error
$\mid\frac{U^{r}_{x}-U^{o}_{x}}{U^{o}_{x}}\mid$ of $U_{x}$ conditioned on a
range of $U_{x}$ values.(a) Case D2F1, (b) Case D2F3, (c) Case D3F3

To demonstrate the capability of CVAE in accurately reproducing velocity field
in gaps even in extreme cases, a test case is designed with some snapshots
with 75% of vectors eliminated. For this case the training dataset was
generated with a uniform distribution of % of vectors eliminated, ranging
between 35-75%, spread uniformly over all the 5120 snapshots, Fig. 10. The
dataset for the case D2F1 was used to generate the gappy training set. A
distinct CVAE network was trained with this dataset with gaps. Figure 11 shows
the instantaneous original vector field, field with artificial gaps and the
reconstructed field using CVAE for a snapshot where 75% of the vectors were
eliminated from the training set. Remarkably, the CVAE latent space along with
the decoder network were able to reproduce all the flow features seen in the
original vector field. Figure 12 shows the reconstruction error using the CVAE
trained with G$\approx$ 75% dataset and compares against the prediction made
by the network trained using the D2F1 dataset with G$\approx$ 50%. As expected
higher reconstruction errors are predicted with the CVAE network trained on
snapshots with gappy % shown in Fig. 10. However, even in this extreme case
reconstruction errors predicted by CVAE are similar to what was observed with
the MF GPOD method described in [1].

Fig. 10: Histogram showing number of snapshots with % of vectors eliminated
from the dataset corresponding to the case D2F1 Fig. 11: Instantaneous vector
plot showing the original vector field, vector field with 75% of vectors
eliminated, CVAE reconstructed vector field from the case D2F1. Fig. 12: PDF,
computed over all the snapshots, of the reconstruction error, E. Lines are
from the case with G 50% and lines with symbols are from the case with G 75%.

##  5 Conclusions

A deep learning-based generative technique called the Conditional Variational
Auto-Encoder (CVAE) is proposed for reconstructing velocity vector field in
gaps typically observed in reacting particle image velocimetry (PIV) data.
This non-linear dimensionality reduction technique, characterized by its
superior latent space properties, solely relies on the valid vector field and
is demonstrated to accurately reproduce vector field in the gaps. The
qualitative and quantitative performance of the CVAE technique is demonstrated
using stereo-PIV data from a premixed swirl combustor with artificial gaps
where up to 75% of the vector field was removed from some of the snapshots.
This highly versatile technique also offers excellent data compression
properties enabling the efficient processing of large-scale PIV data.

Declaration of competing interest

The author declares that he has no known competing financial interests or
personal relationships that could have appeared to influence the work reported
in this paper.

Acknowledgments

The author would like to gratefully acknowledge Adam Steinberg and Pankaj
Saini for sharing the SPIV dataset and source codes for generating gappy PIV
fields.

This research was supported by the Exascale Computing Project (ECP), Project
Number: 17-SC-20-SC, a collaborative effort of two DOE organizations - the
Office of Science and the National Nuclear Security Administration,
responsible for the planning and preparation of a capable exascale ecosystem,
including software, applications, hardware, advanced system engineering and
early testbed platforms, to support the nation’s exascale computing
imperative. The research was performed using computational resources sponsored
by the Department of Energy’s Office of Energy Efficiency and Renewable Energy
and located at the National Renewable Energy Laboratory.

## References

  * [1] P. Saini, C. M. Arndt, A. M. Steinberg, Development and evaluation of gappy-pod as a data reconstruction technique for noisy piv measurements in gas turbine combustors, Experiments in Fluids 57 (7) (2016) 122.
  * [2] H. Gunes, S. Sirisup, G. E. Karniadakis, Gappy data: To krig or not to krig?, Journal of Computational Physics 212 (1) (2006) 358–382.
  * [3] S. G. Raben, J. J. Charonko, P. P. Vlachos, Adaptive gappy proper orthogonal decomposition for particle image velocimetry data reconstruction, Measurement Science and Technology 23 (2) (2012) 025303.
  * [4] R. Everson, L. Sirovich, Karhunen–loeve procedure for gappy data, JOSA A 12 (8) (1995) 1657–1664.
  * [5] A. Nekkanti, O. T. Schmidt, Gappy spectral proper orthogonal decomposition, Journal of Computational Physics 478 (2023) 111950.
  * [6] M. Morimoto, K. Fukami, K. Fukagata, Experimental velocity data estimation for imperfect particle images using machine learning, Physics of Fluids 33 (8) (2021).
  * [7] X. Zhao, Z. Gong, X. Chen, W. Yao, Y. Zhang, A unified framework of deep neural networks and gappy proper orthogonal decomposition for global field reconstruction, in: 2023 International Joint Conference on Neural Networks (IJCNN), IEEE, 2023, pp. 1–8.
  * [8] D. P. Kingma, M. Welling, Auto-encoding variational bayes, arXiv preprint arXiv:1312.6114 (2013).
  * [9] K. Sohn, H. Lee, X. Yan, Learning structured output representation using deep conditional generative models, in: Advances in neural information processing systems, 2015, pp. 3483–3491.
  * [10] G. Liu, F. A. Reda, K. J. Shih, T.-C. Wang, A. Tao, B. Catanzaro, Image inpainting for irregular holes using partial convolutions, in: Proceedings of the European Conference on Computer Vision (ECCV), 2018, pp. 85–100.
  * [11] Batch normalization in pytorch, https://pytorch.org/docs/stable/ generated/torch.nn.BatchNorm2d.html, accessed: 2023-11-20.
  * [12] Rectified linear unit in pytorch, https://pytorch.org/docs/stable/ generated/torch.nn.ReLU.html, accessed: 2023-11-20.
  * [13] L. Hertel, J. Collado, P. Sadowski, J. Ott, P. Baldi, Sherpa: Robust hyperparameter optimization for machine learning, SoftwareXIn press. (2020).
  * [14] D. P. Kingma, J. Ba, Adam: A method for stochastic optimization, arXiv preprint arXiv:1412.6980 (2014).
  * [15] A. Paszke, S. Gross, S. Chintala, G. Chanan, E. Yang, Z. DeVito, Z. Lin, A. Desmaison, L. Antiga, A. Lerer, Automatic differentiation in pytorch, in: NIPS-W, 2017.
  * [16] Q. An, A. M. Steinberg, The role of strain rate, local extinction, and hydrodynamic instability on transition between attached and lifted swirl flames, Combustion and Flame 199 (2019) 267–278.
  * [17] W. Meier, P. Weigand, X. Duan, R. Giezendanner-Thoben, Detailed characterization of the dynamics of thermoacoustic pulsations in a lean premixed swirl flame, Combustion and Flame 150 (1-2) (2007) 2–26.
  * [18] V. Caux-Brisebois, A. M. Steinberg, C. M. Arndt, W. Meier, Thermo-acoustic velocity coupling in a swirl stabilized gas turbine model combustor, Combustion and Flame 161 (12) (2014) 3166–3180.
  * [19] A. K. Prasad, Stereoscopic particle image velocimetry, Experiments in fluids 29 (2) (2000) 103–116.