viewable as a $\mathcal{C}$-submodule of $\mathcal{Q}$. The inclusion map is
given by $\mathcal{P}\ni\psi\longmapsto 1\otimes\psi\in\mathcal{Q}$. So
$\mathcal{P}$ is cyclic for the $\mathcal{B}$-module $\mathcal{Q}$.

The left $\bar{\mathcal{C}}$-module structure on $\mathcal{P}$ is naturally
expressed via the projection map $\Pi\colon\mathcal{Q}\rightarrow\mathcal{Q}$
given by

(3.7) $\Pi(\bar{\alpha}\otimes\psi)=1\otimes(\bar{\alpha}\psi).$

Clearly, the image of the idempotent $\Pi$ is $\mathcal{P}$ and $\Pi$ is
$\bar{\mathcal{C}}$-linear.

######  Lemma 3.1.

The formula

(3.8)
$\raisebox{1.2pt}{$\scriptscriptstyle\blacktriangleleft$}\kern-2.8pt\langle\bar{\alpha}\otimes\varphi|\bar{\beta}\otimes\psi\rangle\kern-3.0pt\raisebox{1.2pt}{$\scriptscriptstyle\blacktriangleright$}=\sum_{k}\langle\beta_{k}\varphi,\alpha_{k}\psi\rangle$

where
$\sigma(\alpha\otimes\bar{\beta})=\sum_{k}\bar{\beta}_{k}\otimes\alpha_{k}$,
defines a sesqui-linear form on $\mathcal{Q}$ which restricted on
$\mathcal{P}$ reproduces the original scalar product. With respect of this
form, the left $\mathcal{B}$-module structure is symmetric.

###### Proof.

Remember that $\sigma(1\otimes 1)=1\otimes 1$ so
$\raisebox{1.2pt}{$\scriptscriptstyle\blacktriangleleft$}\kern-2.8pt\langle
1\otimes\varphi|1\otimes\psi\rangle\kern-3.0pt\raisebox{1.2pt}{$\scriptscriptstyle\blacktriangleright$}=\langle\varphi,\psi\rangle$.
Furthermore since
$\sigma(\beta\otimes\bar{\alpha})=\sum_{k}\bar{\alpha}_{k}\otimes\beta_{k}$ so

$\raisebox{1.2pt}{$\scriptscriptstyle\blacktriangleleft$}\kern-2.8pt\langle\bar{\beta}\otimes\psi|\bar{\alpha}\otimes\varphi\rangle\kern-3.0pt\raisebox{1.2pt}{$\scriptscriptstyle\blacktriangleright$}=\sum_{k}\langle\alpha_{k}\psi,\beta_{k}\varphi\rangle=\bigl{[}\sum_{k}\langle\beta_{k}\varphi,\alpha_{k}\psi\rangle\bigr{]}^{*}=\raisebox{1.2pt}{$\scriptscriptstyle\blacktriangleleft$}\kern-2.8pt\langle\bar{\alpha}\otimes\varphi|\bar{\beta}\otimes\psi\rangle\kern-3.0pt\raisebox{1.2pt}{$\scriptscriptstyle\blacktriangleright$}^{*}$

and hence the hermitian symmetry of
$\raisebox{1.2pt}{$\scriptscriptstyle\blacktriangleleft$}\kern-2.8pt\langle|\rangle\kern-3.0pt\raisebox{1.2pt}{$\scriptscriptstyle\blacktriangleright$}$.
In order to prove that the action of $\mathcal{B}$ is symmetric, it is
sufficient to check it for $\mathcal{C}$, or equivalently $\bar{\mathcal{C}}$.
If $c\in\mathcal{C}$ then

$\raisebox{1.2pt}{$\scriptscriptstyle\blacktriangleleft$}\kern-2.8pt\langle\bar{c}\bar{\alpha}\otimes\varphi|\bar{\beta}\otimes\psi\rangle\kern-3.0pt\raisebox{1.2pt}{$\scriptscriptstyle\blacktriangleright$}=\sum_{k}\langle
u_{k}\varphi,v_{k}\psi\rangle=\raisebox{1.2pt}{$\scriptscriptstyle\blacktriangleleft$}\kern-2.8pt\langle\bar{\alpha}\otimes\varphi|c(\bar{\beta}\otimes\psi)\rangle\kern-3.0pt\raisebox{1.2pt}{$\scriptscriptstyle\blacktriangleright$}\qquad\sum_{k}\bar{u}_{k}\otimes
v_{k}=\sigma(\alpha c\otimes\bar{\beta})$

and we have applied the pentagonal symmetry for $\sigma$ in twisting the
product $\alpha c$ with $\bar{\beta}$. ∎

######  Lemma 3.2.

The projection $\Pi$ is symmetric relative to
$\raisebox{1.2pt}{$\scriptscriptstyle\blacktriangleleft$}\kern-2.8pt\langle|\rangle\kern-3.0pt\raisebox{1.2pt}{$\scriptscriptstyle\blacktriangleright$}$.

###### Proof.

Remembering that $\sigma$ classically flips $1$ with everything, we obtain

$\displaystyle\raisebox{1.2pt}{$\scriptscriptstyle\blacktriangleleft$}\kern-2.8pt\langle\Pi(\bar{\alpha}\otimes\varphi)|\bar{\beta}\otimes\psi\rangle\kern-3.0pt\raisebox{1.2pt}{$\scriptscriptstyle\blacktriangleright$}$
$\displaystyle=\raisebox{1.2pt}{$\scriptscriptstyle\blacktriangleleft$}\kern-2.8pt\langle
1\otimes\bar{\alpha}\varphi|\bar{\beta}\otimes\psi\rangle\kern-3.0pt\raisebox{1.2pt}{$\scriptscriptstyle\blacktriangleright$}=\langle\beta\bar{\alpha}\varphi|\psi\rangle$
$\displaystyle=\langle\bar{\alpha}\varphi|\bar{\beta}\psi\rangle=\raisebox{1.2pt}{$\scriptscriptstyle\blacktriangleleft$}\kern-2.8pt\langle\bar{\alpha}\otimes\varphi|\Pi(\bar{\beta}\otimes\psi)\rangle\kern-3.0pt\raisebox{1.2pt}{$\scriptscriptstyle\blacktriangleright$}.$

So $\Pi$ is indeed symmetric. ∎

Let us now consider the null space of
$\raisebox{1.2pt}{$\scriptscriptstyle\blacktriangleleft$}\kern-2.8pt\langle|\rangle\kern-3.0pt\raisebox{1.2pt}{$\scriptscriptstyle\blacktriangleright$}$.
In general, it will be a non-trivial subspace of $\mathcal{Q}$. It is defined
by

$\mathcal{N}=\Bigl{\\{}\eta\in\mathcal{Q}\bigm{|}\raisebox{1.2pt}{$\scriptscriptstyle\blacktriangleleft$}\kern-2.8pt\langle\eta|\mathcal{Q}\rangle\kern-3.0pt\raisebox{1.2pt}{$\scriptscriptstyle\blacktriangleright$}=\\{0\\}\Bigr{\\}}.$

######  Lemma 3.3.

The space $\mathcal{N}$ is $\mathcal{B}$-invariant and moreover

$\Pi(\mathcal{N})=\\{0\\}=\mathcal{P}\cap\mathcal{N}.$

###### Proof.

The invariance property is a direct consequence of the symmetry of the action
of $\mathcal{B}$ with respect to
$\raisebox{1.2pt}{$\scriptscriptstyle\blacktriangleleft$}\kern-2.8pt\langle|\rangle\kern-3.0pt\raisebox{1.2pt}{$\scriptscriptstyle\blacktriangleright$}$.
Since $\Pi$ is symmetric, it also follows that $\mathcal{N}$ is
$\Pi$-invariant. On the other hand, $\Pi$ projects onto $\mathcal{P}$, and
there
$\raisebox{1.2pt}{$\scriptscriptstyle\blacktriangleleft$}\kern-2.8pt\langle|\rangle\kern-3.0pt\raisebox{1.2pt}{$\scriptscriptstyle\blacktriangleright$}$
reduces to the initial strictly positive scalar product. ∎

Consequently, the whole structure naturally projects on the factor module

(3.9) $\mathcal{R}=\mathcal{Q}/\mathcal{N}$

over the algebra $\mathcal{B}$. We shall use the same symbols
$\raisebox{1.2pt}{$\scriptscriptstyle\blacktriangleleft$}\kern-2.8pt\langle|\rangle\kern-3.0pt\raisebox{1.2pt}{$\scriptscriptstyle\blacktriangleright$}\colon\mathcal{R}\times\mathcal{R}\rightarrow\mathbb{C}$
and $\Pi\colon\mathcal{R}\rightarrow\mathcal{R}$ for the projected objects.
Clearly, the projected
$\raisebox{1.2pt}{$\scriptscriptstyle\blacktriangleleft$}\kern-2.8pt\langle|\rangle\kern-3.0pt\raisebox{1.2pt}{$\scriptscriptstyle\blacktriangleright$}$
is non-degenerate and $\mathcal{P}$ is a $\mathcal{C}$-submodule of
$\mathcal{R}$. Because of the symmetry of $\Pi$ we can write

(3.10) $\mathcal{R}=\mathcal{P}\oplus\mathcal{P}^{\bot},$

and $\mathcal{P}^{\bot}$ is also a $\mathcal{C}$-submodule of $\mathcal{R}$.
To every element of $\mathcal{B}$ we can associate a linear operator from
$\mathcal{A}$ via the symbol construction:

(3.11) $\mathcal{B}\ni b\longmapsto\Pi
b\Pi\colon\mathcal{P}\rightarrow\mathcal{P}.$

Explicitly, if $b=\sum_{k}\bar{\alpha}_{k}\otimes\beta_{k}$, then simply

(3.12) $\Pi b\Pi\leftrightarrow\sum_{k}\bar{\alpha}_{k}\beta_{k},$

and in particular this is an element of $\mathcal{A}$.

######  Remark 3.2.

These elements in (3.12) span the $\bar{\mathcal{C}}$–$\mathcal{C}$ bimodule
in $\mathcal{A}$ generated by $1\in\mathcal{A}$. In general, this will be
strictly smaller than $\mathcal{A}$.

The construction described here can be reversed, thereby giving its own
abstract characterization, which is very much in resonance with the
Stinespring construction (see [21]), which itself generalizes the GNS
construction to the level of appropriate completely positive maps. We can
start from the twisted product algebra
$\mathcal{B}\leftrightarrow\bar{\mathcal{C}}\otimes\mathcal{C}$ as above,
represented by formally adjointable operators in a linear space $\mathcal{R}$
equipped with a not necessarily positive regular scalar product
$\raisebox{1.2pt}{$\scriptscriptstyle\blacktriangleleft$}\kern-2.8pt\langle|\rangle\kern-3.0pt\raisebox{1.2pt}{$\scriptscriptstyle\blacktriangleright$}$.

Let $\mathcal{P}$ be a linear subspace of $\mathcal{R}$ satisfying the
following four structural properties:

  * •

It is positive, in the sense that
$\raisebox{1.2pt}{$\scriptscriptstyle\blacktriangleleft$}\kern-2.8pt\langle|\rangle\kern-3.0pt\raisebox{1.2pt}{$\scriptscriptstyle\blacktriangleright$}$
reduces to a strictly positive scalar product on $\mathcal{P}$;

  * •

It is orthocomplementable in $\mathcal{R}$ in the sense that

$\mathcal{R}=\mathcal{P}\oplus\mathcal{P}^{\bot}$

with respect to
$\raisebox{1.2pt}{$\scriptscriptstyle\blacktriangleleft$}\kern-2.8pt\langle|\rangle\kern-3.0pt\raisebox{1.2pt}{$\scriptscriptstyle\blacktriangleright$}$;

  * •

It is $\mathcal{C}$-invariant;

  * •

It is cyclic, in the sense that

$\bar{\mathcal{C}}\mathcal{P}=\Bigl{\\{}\sum\bar{c}\psi\,|\,c\in\mathcal{C},\psi\in\mathcal{P}\Bigr{\\}}=\mathcal{R}.$

######  Proposition 3.1.

Under the above assumptions, the entire system is naturally isomorphic to the
one constructed above. ∎

##  4\. Concluding Thoughts

Our Toeplitz quantization construction provides a kind of resonant bridge
going between two, in principle very different, algebras. On one side we have
a given polynomial quadratic *-algebra $\mathbb{C}[z,\bar{z},Q]$ generated by
abstract symbols $z$ and $\bar{z}$. On the other there is a concrete non-
commutative *-algebra of operators $\mathcal{A}$, acting within a Hilbert
space $\mathcal{H}$ of entire functions, possessing a dense common invariant
subspace $\mathcal{W}$, and also viewable as transformations of the whole
$\mathrm{H}(\mathbb{C})$, generated by the multiplication operator $z$ and its
companion  $z$ , interpretable as the adjoint of $z$ in terms of
$\mathcal{H}$.

There is an extended space $\mathcal{R}$, which is equipped with a non-
necessarily positive non-degenerate scalar product
$\raisebox{1.2pt}{$\scriptscriptstyle\blacktriangleleft$}\kern-2.8pt\langle|\rangle\kern-3.0pt\raisebox{1.2pt}{$\scriptscriptstyle\blacktriangleright$}$,
on which the *-algebra $\mathbb{C}[z,\bar{z},Q]$ symmetrically acts. The
common domain $\mathcal{W}$ for the operators of $\mathcal{A}$ is
isometrically realized as an orthocomplementable subspace of $\mathcal{R}$.
Then $\mathbb{C}[z,\bar{z},Q]$ is morphed into $\mathcal{A}$ with the help of
the orthogonal projection $\Pi\colon\mathcal{R}\rightarrow\mathcal{W}$. In
this sense $\mathcal{A}$ is interpreted as a Toeplitz quantization of
$\mathbb{C}[z,\bar{z},Q]$.

At a first sight, it might come as a surprise that we can perform the
construction on two a priori unrelated algebras such as
$\mathbb{C}[z,\bar{z},Q]$ and $\mathcal{A}$—coming from two apparently quite
different worlds.

On one hand, we believe that this reflects the flexibility and versatility of
the construction, which as we have seen admits an elegant purely algebraic
foundation. And after all, behind both algebras lies an intuitive idea of a
quantum object resembling the classical Euclidean plane. On the other hand, it
is natural to look for some additional geometrical context, in which the
relationship between $\mathbb{C}[z,\bar{z},Q]$ and $\mathcal{A}$ looks
especially harmonic.

One such a context is given by our last harmony condition, which requires that
$\raisebox{1.2pt}{$\scriptscriptstyle\blacktriangleleft$}\kern-2.8pt\langle|\rangle\kern-3.0pt\raisebox{1.2pt}{$\scriptscriptstyle\blacktriangleright$}$
be positive. As we have already mentioned, in the case of the standard
commutative product in $\mathbb{C}[z,\bar{z}]$ the positivity condition leads
to its moment problem of finding a finite measure on $\mathbb{C}$ reproducing
the scalar product. Such a measure can always be taken to be radial for
diagonal scalar products.

Our principal series of examples, as in [10], also comes from a diagonal
scalar product on $\mathbb{C}[z]$, defined by a positive sequence
$s_{n}=s_{nn}$ of squares of norms of mutually orthogonal $z^{n}$. There is a
natural generalization, which includes as a special case the parabolic
quadratic algebra considered in Section 2, and which is also invariant under
Euclidean transformations of $\mathbb{C}$. If we require that the polynomials
of the canonical orthonormal basis satisfy

$\phi_{n}(z)\mid\phi_{n+1}(z)$

for every $n\in\mathbb{N}$, that is to say, that zeroes of $\phi_{n}(z)$ are
included (multiplicities counted) in zeroes of $\phi_{n+1}(z)$, the whole
scalar product is encoded in two sequences. these are a sequence of positive
numbers $s_{n}$ and a sequence of complex numbers $\lambda_{n}$ so that

$\phi_{n}(z)=\frac{1}{\sqrt{\smash[b]{s_{n}}}}(z-\lambda_{1})\cdots(z-\lambda_{n})$

or equivalently

$z^{n}=\sum_{k=0}^{n}\sqrt{s_{k}}u_{n-k}(\lambda_{1},\cdots,\lambda_{k+1})\phi_{k}(z)$

where $u_{m}$ are symmetric functions computing the sum of all elementary
monomials of degree $m$ constructed from their arguments (for instance $u_{0}$
is always $1$ while $u_{1}$ sums its arguments and
$u_{7}(a,b)=(a+b)(a^{2}+b^{2})(a^{4}+b^{4})$).

The diagonal scenario is recovered as the special case $\lambda_{n}=0$ for all
$n$, and the parabolic quadratic algebra example corresponds to
$\lambda_{n}=n-1+h/2$. The algebraic part of Harmony Zero will be always
satisfied, as $Z^{*}\mathbb{C}_{n}[z]\subseteq\mathbb{C}_{n}[z]$ for every
$n\in\mathbb{N}$.

We have analyzed in this paper the ‘one-particle’ scenario, where there is
only one complex variable $z$ to quantize. In a very similar way, everything
is extendible to several complex variables. The methods should also be
extendible to classical subdomains in $\mathbb{C}^{n}$. Here we would expect a
natural emergence of non-Euclidean quantum geometries in resonance with the
results of [9], where quantum hyperbolic planes are studied, and the
interpretation of Poincaré models linked with the Hilbert spaces of analytic
functions on the open unitary disk $\mathbb{D}$.

Another very interesting and promising topic for future explorations is to
consider multi-dimensional quantum Euclidean spaces defined via the described
quantization, which preserve some important symmetries and develop
differential geometry on such spaces. It is tempting to procede in the spirit
of [11, 12], where it was revealed that classical structures with discrete
symmetries exhibit a ‘hidden’ quantum personality, which is describable in the
elegant geometrical language of quantum principal bundles.

##  Appendix A Positivity and Stieltjes Moment Condition

In this Appendix we shall discuss in more detail the positivity property for
the quadratic form
$\raisebox{1.2pt}{$\scriptscriptstyle\blacktriangleleft$}\kern-2.8pt\langle|\rangle\kern-3.0pt\raisebox{1.2pt}{$\scriptscriptstyle\blacktriangleright$}$,
induced on the classical polynomial *-algebra $\mathbb{C}[z,\bar{z}]$ by a
diagonal scalar product $\langle|\rangle$ on $\mathbb{C}[z]$ with the defining
sequence $s_{n}$ given in (2.12)

The geometry of $\mathbb{C}[z,\bar{z}]$ with such a
$\raisebox{1.2pt}{$\scriptscriptstyle\blacktriangleleft$}\kern-2.8pt\langle|\rangle\kern-3.0pt\raisebox{1.2pt}{$\scriptscriptstyle\blacktriangleright$}$
is reflected in the accompanying figure. We start with an integer lattice. The
white nodes correspond to the monomials $\bar{z}^{k}z^{l}$ with $k,l\geq 0$.
These monomials are a basis in the commutative *-algebra
$\mathbb{C}[z,\bar{z}]$. By allowing negative powers of $z$ and $\bar{z}$ such
that $k+l\geq 0$, this algebra is included in an extended commutative
*-algebra $\mathbb{E}[z,\bar{z}]$. Such monomials are represented by grey
nodes. To each node $a=(k,l)$ we define its value to be the number $|a|:=k+l$.
In terms of this picture the scalar product
$\raisebox{1.2pt}{$\scriptscriptstyle\blacktriangleleft$}\kern-2.8pt\langle|\rangle\kern-3.0pt\raisebox{1.2pt}{$\scriptscriptstyle\blacktriangleright$}$
is computed using (2.72) according to the following rule: if the nodes labeled
by $a$ and $b$ share one of the parallel diagonal lines, then the scalar
product is $s_{(|a|+|b|)/2}$. Otherwise the monomials are orthogonal. This
immediately reveals that
$\raisebox{1.2pt}{$\scriptscriptstyle\blacktriangleleft$}\kern-2.8pt\langle|\rangle\kern-3.0pt\raisebox{1.2pt}{$\scriptscriptstyle\blacktriangleright$}$
will be positive if and only if its restriction to all of the diagonal
subspaces is positive.

But the diagonal subspaces are of two canonical types: those consisting of
nodes with even value and those consisting of nodes with odd value. So the
positivity condition reduces to the positivity of the corresponding Gram
matrices,

$\begin{pmatrix}s_{0}&s_{1}&\cdots&s_{n}\\\ s_{1}&s_{2}&\cdots&s_{n+1}\\\
\vdots&\vdots&\ddots&\vdots\\\
s_{n}&s_{n+1}&\cdots&s_{2n}\end{pmatrix}\qquad\quad\begin{pmatrix}s_{1}&s_{2}&\cdots&s_{n}\\\
s_{2}&s_{3}&\cdots&s_{n+1}\\\ \vdots&\vdots&\ddots&\vdots\\\
s_{n}&s_{n+1}&\cdots&s_{2n-1}\end{pmatrix}$

of even and odd types respectively. But this is exactly the Stieltjes moment
condition for the existence of a measure $\mu$ on $[0,\infty)$ satisfying

$s_{n}=\int_{0}^{\infty}t^{n}\,d\mu(t)\qquad\quad\forall n\in\mathbb{N}.$

And this is equivalent to the existence of a rotationally invariant measure
$d\mu(z,\bar{z})$ on $\mathbb{C}$ reproducing the scalar product on
$\mathbb{C}[z,\bar{z}]$ in the standard way:

$\raisebox{1.2pt}{$\scriptscriptstyle\blacktriangleleft$}\kern-2.8pt\langle
f|g\rangle\kern-3.0pt\raisebox{1.2pt}{$\scriptscriptstyle\blacktriangleright$}=\int_{\mathbb{C}}\overline{f(z)}g(z)\,d\mu(z,\bar{z}).$

Interestingly, we can look at all of this another way around and use the
construction to prove the Stieltjes moment condition. Indeed, as is easily
seen from the figure, the positivity of
$\raisebox{1.2pt}{$\scriptscriptstyle\blacktriangleleft$}\kern-2.8pt\langle|\rangle\kern-3.0pt\raisebox{1.2pt}{$\scriptscriptstyle\blacktriangleright$}$
on $\mathbb{C}[z,\bar{z}]$ is equivalent to the positivity of
$\raisebox{1.2pt}{$\scriptscriptstyle\blacktriangleleft$}\kern-2.8pt\langle|\rangle\kern-3.0pt\raisebox{1.2pt}{$\scriptscriptstyle\blacktriangleright$}$
on $\mathbb{E}[z,\bar{z}]$. And such an extended positivity is equivalent,
according to the theorem of Stochel–Szafraniec [17], to the existence of an
underlying measure for the quadratic form. This measure, due to the rotational
symmetry of the system, can be always taken to be purely radial.

##  Appendix B On Hilbert Spaces of Holomorphic Functions

### Reproducing Kernels as Point Functionals

We shall here review some general properties of Hilbert spaces of holomorphic
functions, which are particularly relevant for our main considerations on
Toeplitz quantization.

We refer to [1] for a lovely self-contained exposition of the theory of
Hilbert function spaces, including those consisting of holomorphic functions.
In dealing with these structures, it is optimal to assume that everything
occurs in a mathematical universe in which linear functionals defined over
Hilbert spaces are always continuous. This leads to interesting foundational
issues in Functional Analysis, involving the axiom of choice, automatic
continuity and constructibility of objects. We refer to [16], especially
Chapter 6 of the book, for an in-depth discussion of these conceptual roots.

Let $\mathcal{H}$ be a Hilbert space consisting of holomorphic functions over
a domain $\Omega$. So $\mathcal{H}$ is a complex vector subspace of
$\mathrm{Hol}(\Omega)$. The structure of such a space is completely determined
by its reproducing kernel function
$K\colon\Omega\times\Omega\rightarrow\mathbb{C}$, which in the main text we
have met in the context $\Omega=\mathbb{C}$ of entire functions. The primary
properties we mentioned in that special context are valid for a general
domains. In particular, the kernel is holomorphic in its second argument and
anti-holomorphic in its first argument. And for every $w\in\Omega$ we can
define $[w]=w(z)=K(\bar{w},z)$ to be the corresponding ‘point wave function’.
These functions belong to $\mathcal{H}$ and reproduce the values of the
functions from $\mathcal{H}$ in the points $w$ via the scalar product $\langle
w(z),\psi(z)\rangle=\psi(w)$. So the scalar product between two point wave
functions $[v],[w]$ is the reproducing kernel of the points $v,w\in\Omega$
taken in the opposite order $K(\bar{w},v)=\langle
v(z)|w(z)\rangle=[w](v)=\overline{K(\bar{v},w)}$.

If $\Lambda$ is any subset of $\Omega$ possessing an accumulation point in
$\Omega$, then the point wave functions $\lambda(z)$ with $\lambda\in\Lambda$
generate the whole Hilbert space $\mathcal{H}$ or, in other words, the
orthocomplement to all of them is $\\{0\\}$.

And if $\Lambda$ is a finite non-empty subset of $\Omega$ we can construct a
quadratic matrix $K[\Lambda]$ whose entry $(v,w)$ is $K(\bar{w},v)$, where now
$w,v\in\Lambda$. Such a matrix is always positive. It is the Gram matrix for
the system of vectors $\lambda(z)$ with $\lambda\in\Lambda$. It turns out that
conversely, any bi-anti-holomorphic function $K$ on $\Omega\times\Omega$
satisfying the positivity condition for its all matrices $K[\Lambda]$
generates a Hilbert space of holomorphic functions.

We can naturally extend the notion of wave point function to include
derivative operators. For every $w\in\Omega$ and $n\in\mathbb{N}$ let us
define a function $[w{\wr}n]=[w{\wr}n](z)\in\mathcal{H}$ by

(B.1) $\langle[w{\wr}n]|\psi\rangle=\frac{\partial^{n}\psi}{\partial
z^{n}}(w)\qquad\forall\psi\in\mathcal{H}.$

This makes sense since the right side is a norm continuous functional of
$\psi\in\mathcal{H}$. So we have included the original point wave functions as
$[w{\wr}0]=[w]$. In terms of the reproducing kernel we have

(B.2) $[w{\wr}n](z)=\frac{\partial^{n}\\!K}{\partial\bar{w}^{n}}(\bar{w},z)$

and the classical Cauchy formula can be expressed as

(B.3) $[w{\wr}n]=\frac{n!}{2\pi i}\oint\frac{dc\,[c]}{(c-w)^{n+1}},$

where the integral is over a positively oriented simple curve in $\Omega$
around $w$. From the definition of these higher order point wave functions it
follows that

(B.4)
$\langle[c{\wr}m]|[w{\wr}n]\rangle=\frac{\partial^{n+m}}{\partial\bar{w}^{n}\partial
c^{m}}K(\bar{w},c).$

######  Lemma B.1.

For a given $w\in\Omega$ the functions
$\bigl{\\{}[w{\wr}n]:n\in\mathbb{N}\bigr{\\}}$ span a dense linear subspace in
$\mathcal{H}$.

###### Proof.

For a function from $\mathcal{H}$, being orthogonal to all $[w{\wr}n]$ with
fixed $w$ means vanishing at $w$, together with all its derivatives. Because
of the connectedness of $\Omega$, such a holomorphic function must be zero
identically. ∎

######  Lemma B.2.

For a given $w\in\Omega$ the functions
$\bigl{\\{}[w{\wr}n]:n\in\mathbb{N}\bigr{\\}}$ admit a biorthogonal system if
and only if $\mathbb{C}[z]\subset\mathcal{H}$. In this case the corresponding
biorthogonal system is given by
$\bigl{\\{}(z-w)^{n}/n!:n\in\mathbb{N}\bigr{\\}}$.

###### Proof.

If $b_{m}(z)$ are functions biorthogonal to $[w{\wr}n]$ then

$\langle[w{\wr}n]|b_{m}\rangle=\delta_{nm}=\frac{\partial^{n}}{\partial
z^{n}}b_{m}(z)\Bigm{|}_{z=w}$

and by expanding in Taylor series around $w$ we obtain
$b_{m}(z)=(z-w)^{m}/m!$. In particular $\mathbb{C}[z]\subset\mathcal{H}$.
Conversely, if polynomials are included in $\mathcal{H}$ then $b_{m}(z)$
defined by the same formula are clearly the biorthogonal system for
$[w{\wr}n](z)$. ∎

We see that the inclusion of the polynomials $\mathbb{C}[z]$ in $\mathcal{H}$
is a natural structural property of $\mathcal{H}$. However, the space
$\mathbb{C}[z]$ is not necessarily dense in $\mathcal{H}$. We can proceed in
the same spirit here, as in the context of entire functions, and consider the
array of harmony properties, related to the operator form of the complex
coordinate $z$, and its adjoint operator, introduce the common invariant
subspace $\mathcal{W}$ consisting of normal vectors, and obtain a non-
commutative *-algebra $\mathcal{A}$ which itself can be viewed as a form of
quantization of the domain $\Omega$, and also as the algebra of the Toeplitz
operators for the Toeplitz quantization of appropriate quadratic algebras.
Here it is natural to assume the maximality of $\Omega$ relative to
$\mathcal{H}$ in the sense that there is no common analytic extension beyond
$\Omega$, valid for all the elements of $\mathcal{H}$. Of course, when
$\Omega=\mathbb{C}$ this condition is trivial.

### Segal-Bargmann Space and $q$-Deformations

Let us first consider the Hilbert space $L^{2}(\mathbb{C},\rho)$ where
$\rho(z)=\exp(-|z|^{2})/\pi$. The Segal-Bargmann space then consists of all
entire functions belonging to the above Hilbert space. The polynomials in $z$
are everywhere dense, and the monomials in $z$ are mutually orthogonal. The
reproducing kernel is given by

(B.5) $K(\bar{w},z)=\exp(\bar{w}z).$

The defining sequence of weights (see (2.12)) is

(B.6) $s_{n}=s_{nn}=n!$

as is also immediately visible from the expansion of the exponent in the
reproducing kernel formula. Modulo normalizations this scalar product is the
unique scalar product on $\mathcal{P}=\mathbb{C}[z]$ with respect to which
$Z^{*}=\partial/\partial z$.

Let us now consider a $q$-deformation of the Segal-Bargmann space with the
defining sequence given by the $q$-factorials (2.78). As we have seen, the
reproducing kernel is given by the $q$-exponential function
$K(\bar{w},z)=E_{q}(\bar{w}z)$. There are infinitely many generating measures
on $\mathbb{C}$ for this reproducing kernel, which in particular means that
$\raisebox{1.2pt}{$\scriptscriptstyle\blacktriangleleft$}\kern-2.8pt\langle|\rangle\kern-3.0pt\raisebox{1.2pt}{$\scriptscriptstyle\blacktriangleright$}$
is strictly positive on $\mathbb{C}[z,\bar{z}]$.

Here are two distinguished measures, both rotationally symmetric. The first
one is discrete on the radial coordinate in $\mathbb{C}$ and based on the
discrete $q$-integral

(B.7) $\int_{0}^{\infty}f(t)\,d_{q}t=\sum_{n\in\mathbb{Z}}q^{n}f(q^{n}).$

A straightforward application of the $q$-integral formula

(B.8)
$\int_{0}^{\infty}\frac{t^{\alpha}\,d_{q}t}{E_{q}(t/q)}=\frac{(q|q)_{\alpha}}{(q-1|q)_{\alpha}(q/(q-1)|q)_{-\alpha}}=:!_{q}(\alpha)$

together with the observation that $!_{q}(\alpha)=s_{\alpha}$ for
$\alpha\in\mathbb{N}$, leads us to this radially discrete density function

(B.9)
$\rho(z,\bar{z})=\frac{1}{\pi}\frac{1}{E_{q}(|z|^{2}/q)}\sum_{n\in\mathbb{Z}}q^{n}\delta(|z|^{2}-q^{n}).$

We have used an extended definition for the $q$-symbols

(B.10) $(z|q)_{\alpha}=\frac{(z|q)_{\infty}}{(zq^{\alpha}|q)_{\infty}}$

valid for arbitrary complex numbers $\alpha\in\mathbb{C}$.

On the other hand, we can use the continuous variation

(B.11)
$\int_{0}^{\infty}\frac{t^{\alpha-1}\,dt}{(-t|q)_{\infty}}=(q^{1-\alpha}|q)_{\alpha}\frac{\pi}{\sin(\pi\alpha)}$

of the integral (B.8). This for $\alpha\in\mathbb{N}$ morphs into

(B.12)
$\int_{0}^{\infty}\frac{t^{\alpha}\,dt}{E_{q}(t/q)}=\frac{\log(1/q)}{1/q-1}s_{\alpha}$

and we naturally arrive to another density function, scaled by a positive
factor and free of discrete $\delta$-terms

(B.13) $\rho(z,\bar{z})=\frac{1/q-1}{\pi\log(1/q)}\frac{1}{E_{q}(|z|^{2}/q)}.$

In the limit $q\rightsquigarrow 1^{-}$ both measures become the standard
Segal-Bargmann measure.

It is interesting to observe that from the sequence of weights we can see
directly that the form
$\raisebox{1.2pt}{$\scriptscriptstyle\blacktriangleleft$}\kern-2.8pt\langle|\rangle\kern-3.0pt\raisebox{1.2pt}{$\scriptscriptstyle\blacktriangleright$}$
is strictly positive. Indeed, consider the following $n\times n$ matrix
determinant, with entries indexed by $i,j=0,\dots,n-1$ and composed of
$q$-factorials:

(B.14) $\left|\begin{matrix}&&\\\ &!_{q}(i+j+d)&\\\
&&\end{matrix}\right|=q^{-{\textstyle\binom{n}{2}}(d+{\textstyle\frac{2n-1}{3}})}\prod_{k=0}^{d-1}\frac{!_{q}(n+k)}{!_{q}(k)}\Bigl{\\{}\prod_{k=1}^{n-1}!_{q}(k)\Bigr{\\}}^{2}$

where $d\in\mathbb{N}$. For a positive $q$ these are all positive numbers (and
quickly growing very large if $q\leq 1$). Here is the sequence of positive
numbers whose partial products generate the above determinants

(B.15)
$\lambda_{n}=q^{-n^{2}}!_{q}(n)^{2}\qquad\lambda_{n}=q^{-n^{2}-nd}!_{q}(n+d)\,!_{q}(n)^{2}$

corresponding to $d=0$ and $d\geq 1$ respectively. From the analysis of the
previous Appendix it follows that the form
$\raisebox{1.2pt}{$\scriptscriptstyle\blacktriangleleft$}\kern-2.8pt\langle|\rangle\kern-3.0pt\raisebox{1.2pt}{$\scriptscriptstyle\blacktriangleright$}$
must be strictly positive on $\mathbb{C}[z,\bar{z}]$.

And here is a variation on the above sequences

(B.16)
$\lambda_{n}=q^{n^{2}}!_{q}(n)^{2}\qquad\qquad\>\lambda_{n}=q^{n^{2}}!_{q}(n+d)\,!_{q}(n)^{2}$

generating the determinants

(B.17) $\left|\begin{matrix}&&\\\ &q^{ij}!_{q}(i+j+d)&\\\
&&\end{matrix}\right|=q^{{\textstyle\binom{n}{2}}{\textstyle\frac{2n-1}{3}}}\prod_{k=0}^{d-1}\frac{!_{q}(n+k)}{!_{q}(k)}\Bigl{\\{}\prod_{k=1}^{n-1}!_{q}(k)\Bigr{\\}}^{2}$

via their partial products. The positivity of these numbers can be used to
derive the positivity of
$\raisebox{1.2pt}{$\scriptscriptstyle\blacktriangleleft$}\kern-2.8pt\langle|\rangle\kern-3.0pt\raisebox{1.2pt}{$\scriptscriptstyle\blacktriangleright$}$
on the space $\mathbb{C}[z,\bar{z},Q]$ for the Manin $q$-plane, so in this
scenario the property H4 holds, too. Note that the parameter $q$ is the same
for both algebras.

### Bergman Spaces

These spaces provide one of the earliest frameworks for quantizing Euclidean
domains, via complex functions theory. They were introduced by Stefan Bergman
[4] during the 20s of the 20th century.

Let us consider a bounded domain $\Omega$ equipped with its standard Euclidean
measure. Let $\mathcal{H}$ be the space of all square integrable holomorphic
functions in $\Omega$. It is easy to see that $\mathcal{H}$ is a closed
subspace of $L^{2}(\Omega)$ and thus, in the induced scalar product, it
becomes a Hilbert space of holomorphic functions.

The scalar product is given by

$\langle f,g\rangle=\int_{\Omega}\overline{f(w)}g(w).$

Because of the boundedness of $\Omega$, the multiplication operator by $z$ is
bounded and defined on all $\mathcal{H}$. Together with its adjoint, it
generates a non-commutative C*-algebra representing a quantized domain
$\Omega$.

We then have

$\int_{\Omega}|w(z)\rangle\langle
w(z)|=1\colon\mathcal{H}\rightarrow\mathcal{H}$

where the integral is in the weak operator topology. This is a direct
consequence of the definition of the space $\mathcal{H}$. It is worth noticing
that in the special case of $\Omega=\mathbb{D}$ the unit disk, the reproducing
kernel is given by

$K(\bar{w},z)=\frac{1}{\pi}\frac{1}{(1-\bar{w}z)^{2}}$

and the whole system is interpretable as a quantization of the Poincaré model
of the hyperbolic plane.

### The Toeplitz Extension

Another quantum model of the hyperbolic plane is obtained by considering the
square summable power series [7]. The unit disc $\mathbb{D}$ as the common
domain of holomorphicity, and the reproducing kernel is simply

$K(\bar{w},z)=\frac{1}{1-\bar{w}z}$

so that the monomials $z^{n}$ form an orthonormal basis in $\mathcal{H}$. The
coordinate $z$ acts as the unilateral shift operator in $\mathcal{H}$.

### Paley-Wiener $\&$ Euler Spaces

The Paley-Wiener space of index $a>0$ is generated by the reproducing kernel

$K(\bar{w},z)=\frac{\sin[a(\bar{w}-z)]}{\pi(\bar{w}-z)}.$

The elements of the space are the entire functions square-integrable over the
reals, and the scalar product is given by

$\langle f(z),g(z)\rangle=\int_{-\infty}^{\infty}\overline{f(t)}g(t)\,dt.$

The space can be viewed as the image of $L^{2}[-a,a]$ via the complexified
Fourier transform. More generally, if we consider an entire function $E(z)$
satisfying

$|E(x-iy)|<|E(x+iy)|$

where $x\in\mathbb{R}$ and $y>0$, then the Hilbert space $\mathcal{H}(E)$ is
given by the reproducing kernel

$K(\bar{w},z)=\frac{E^{*}(\bar{w})E(z)-E(\bar{w})E^{*}(z)}{2\pi
i(\bar{w}-z)}.$

The space $\mathcal{H}(E)$ consists of entire functions. They are all square
integrable over the reals with the weight function $1/|E(t)|^{2}$ so that the
scalar product is given by

$\langle
f(z),g(z)\rangle=\int_{-\infty}^{\infty}\frac{\overline{f(t)}g(t)}{|E(t)|^{2}}\,dt.$

The Paley-Wiener spaces are a special case when $E(z)=\exp(-iaz)$. We refer to
[6] for a detailed study of these ‘Euler spaces’. They are a rich collection
of examples of the Hilbert spaces of entire functions going beyond our harmony
properties. Nevertheless, they allow similar non-commutative constructions and
provide, with the geometrical interpretations appropriately refined, an
interesting complementary framework for quantization of the classical
Euclidean plane.

## References

  * [1] J. Agler, J. E. McCarthy: Pick Interpolation and Hilbert Function Spaces, American Mathematical Society, 2002.
  * [2] G. Andrews, R. Askey, R. Roy: Special Functions, Encyclopedia of Mathematics and its Applications 71, Cambridge University Press, 1999.
  * [3] V. Bargmann: On a Hilbert space of analytic functions and its associated integral transform. I, Commun. Pure Appl. Math. 14 (1961) 187–214.
  * [4] S. Bergman: The Kernel Function and Conformal Mapping, American Mathematical Society, 1950.
  * [5] M. S. Birman, M. Z. Solomjak: Spectral Theory of Self-Adjoint Operators in Hilbert Space, Mathematics and Its Applications 5, Springer, 1987.
  * [6] L. de Branges: Hilbert Spaces of Entire Functions, Prentice-Hall, 1968.
  * [7] L. de Branges, J. Rovnyak: Square Summable Power Series, Athena Series, Holt, Rinehart and Winston, 1966.
  * [8] A. Connes: Noncommutative Geometry, Academic Press, 1994.
  * [9] M. D urd evich, P. C. Lucio Peña, Geometric Structures on Quantum Hyperbolic Planes, Q-Preprint Q-2020.
  * [10] M. D urd evich, S. B. Sontz: Coherent States for the Manin Plane via Toeplitz Quantization, J. Math. Phys. 61, 023502 (2020).
  * [11] M. D urd evich, S. B. Sontz: Dunkl Operators as Covariant Derivatives in a Quantum Principal Bundle, SIGMA 9 040, 29 Pages (2013).
  * [12] M. D urd evich, S. B. Sontz: Dunkl Operators for Arbitrary Finite Groups, Adv. Oper. Theory 6, 37 (2021).
  * [13] G. Gasper, M. Rahman: Basic Hypergeometric Series, Encyclopedia of Mathematics and its Applications 96, Cambridge University Press, 2004.
  * [14] M. Reed, B. Simon, Methods of Modern Mathematical Physics, Vol. I. Functional Analysis, Academic Press, 1972.
  * [15] E. Prugovečki: Quantum Geometry–A Framework for Quantum General Relativity, Kluwer Academic Publishers, 1992.
  * [16] E. Schechter: Handbook of Analysis and its Foundations, Academic Press (1996).
  * [17] K. Schmüdgen: The Moment Problem, Springer, 2017.
  * [18] A. Shepler and S. Witherspoon, Poincaré-Birkhoff-Witt theorems, in: Commutative algebra and non-commutative algebraic geometry, Vol. I, 259–290, Math. Sci. Res. Inst. Publ. 67, Cambridge Univ. Press, 2015.
  * [19] S. B. Sontz: A Reproducing Kernel and Toeplitz Operators in the Quantum Plane, Commun. Math. 21 (2013) 137–160.
  * [20] S. B. Sontz: Toeplitz Quantization for Non-commuting Symbol Spaces such as $SU_{q}(2)$, Commun. Math. 24 (2016) 43–69.
  * [21] W. F. Stinespring: Positive Functions on C*-algebras, Proceedings of the American Mathematical Society, 6 (1955) 211–216.
  * [22] S. L. Woronowicz: Pseudospaces, Pseudogroups and Potryagin Duality, Proceedings of the International Conference of Mathematical Physics, Lausanne 1979, Lecture Notes in Physics 116, 407–412.


# A Multimodal Intermediate Fusion Network with Manifold Learning for Stress
Detection

1st Morteza Bodaghi  Dept. MCHE. Engineering
University of Louisiana
Lafayette, USA
<EMAIL_ADDRESS>2nd Majid Hosseini  Dept. C. Science
University of Louisiana
Lafayette, USA
<EMAIL_ADDRESS>3rd Raju Gottumukkala  Dept. MCHE. Engineering
University of Louisiana
Lafayette, USA
<EMAIL_ADDRESS>
###### Abstract

Multimodal deep learning methods capture synergistic features from multiple
modalities and have the potential to improve accuracy for stress detection
compared to unimodal methods. However, this accuracy gain typically comes from
high computational cost due to the high-dimensional feature spaces, especially
for intermediate fusion. Dimensionality reduction is one way to optimize
multimodal learning by simplifying data and making the features more amenable
to processing and analysis, thereby reducing computational complexity. This
paper introduces an intermediate multimodal fusion network with manifold
learning-based dimensionality reduction. The multimodal network generates
independent representations from biometric signals and facial landmarks
through 1D-CNN and 2D-CNN. Finally, these features are fused and fed to
another 1D-CNN layer, followed by a fully connected dense layer. We compared
various dimensionality reduction techniques for different variations of
unimodal and multimodal networks. We observe that the intermediate-level
fusion with the Multi-Dimensional Scaling (MDS) manifold method showed
promising results with an accuracy of 96.00% in a Leave-One-Subject-Out Cross-
Validation (LOSO-CV) paradigm over other dimensional reduction methods. MDS
had the highest computational cost among manifold learning methods. However,
while outperforming other networks, it managed to reduce the computational
cost of the proposed networks by 25% when compared to six well-known
conventional feature selection methods used in the preprocessing step.

###### Index Terms:

Psychological Stress Detection, Early Fusion, Intermediate Level Fusion,
Manifold Learning, Dimension Reduction, Multimodal Learning, Biometric
Signals, Facial Landmarks, video

##  I Introduction

Predicting stress in the real world is a very challenging problem because of
the high variability of how stress manifests across individuals and the
influence of environmental context. [1]. Multimodal learning integrates
signals from multiple modalities and captures complementary information across
sources and their cross-modal interaction. Prior work in multi-modal learning
from biometric signals, voice, and facial expression has been shown to improve
prediction accuracy [1]. However, as pointed out by prior research [2],
multimodal learning models come with high computational costs of data
processing, training, and validation. Training the model requires
synchronization across multiple modalities, and optimizing the model to
capture the correlation between different modalities gets quite expensive[3].
More research is needed to optimize the performance trade-offs [4, 5, 6].
Various strategies exist to optimize the performance trade-offs between
achieving good accuracy and computational needs, such as improving network
architecture [7], model compression [8], dimensionality reduction[9],
regularization[10] and hardware optimization [11]. In this paper, we study the
problem of dimensionality reduction using manifold learning.

Dimensional reduction benefits multimodal learning in various ways. First,
unnecessary features are eliminated, and a cleaner set of features is
retained. Second, the data size is reduced, reducing the storage and
processing requirements[12] that decreases memory and computational cost.
Finally, a reduced feature set minimizes the number of dimensions and
redundancy, thereby minimizing overfitting and improving model performance
[13]. Existing methods, such as PCA, offer good dimensionality reduction, but
these models work well for transformations where the combination of features
is predominantly linear. Emotion and stress-related features involve complex
geometric transformations that are inherently nonlinear [14, 15]. Manifold
learning assumes that the data lies on a two-dimensional manifold embedded in
a higher-dimensional space, and the structure can be preserved by geometric
space, so Manifold learning can potentially capture low-dimensional structures
within high-dimensional data where the structure is inherently geometric.
Manifold learning has been applied in image recognition [16], multiple sensor
translation [17], neuroscience and brain network analysis [18, 19]. In this
paper, we introduce a low-cost intermediate fusion model for stress detection
that implements manifold dimensional reduction and convolutional neural
network. We investigate different variations of multimodal learning techniques
with various manifold learning techniques and PCA to study the accuracy vs
computational cost trade-offs. The models are investigated using the LOSO-CV
method. We also compared the performancecost trade-offs of using manifold
learning in unimodal, early, and intermediate-level fusion neural networks.

We discuss closely related work in this area in Section 2; the details of the
proposed intermediate fusion network are discussed in Section 3; manifold
learning methods are explained in Section 4, and then we discuss results and
analysis in Section 5. Section 6 compares the computational cost of different
networks with and without using various dimensionality reduction techniques.
Finally, we discuss the limitations and conclusion in sections 7 and 8.

##  II Related Work

Multimodal stress detection is a valuable approach, as it leverages multiple
sensory data sources to uncover intriguing patterns and correlations across
different modalities simultaneously [20]. In this regard, researchers have
explored three distinct classes of multimodal learning models: early fusion,
late fusion, and intermediate-level fusion.

Early fusion has been extensively examined in existing literature. For
instance, Wu, Yujin, et al. [21] proposed an innovative geometric multimodal
LSTM-based stress detection framework, incorporating representations of
physiological and behavioral signals derived from covariance and cross-
covariance. Similarly, Seo, Wonju, et al. [22] introduced a deep learning
feature-level (early-fusion) fusion network that combined ECG, respiration,
and facial features, achieving a binary stress detection accuracy of 73.3%.
Additionally, [23] employed ECG signals, peripheral physiological signals, and
eye movement for emotion detection, reducing data dimensions through the ISO
algorithm at the feature level and subsequently feeding it into a
Convolutional Neural Network (CNN), achieving an accuracy of 90.05

Moving to intermediate-level fusion, [24] attained an accuracy of 75.5% using
a multimodal fusion model based on CNNs for subject-independent stress
detection. This model incorporated physiological signals, such as
Electrocardiogram (ECG) and Electrodermal Activity (EDA). Furthermore, [25]
successfully fused raw video and audio signals to create an emotion detection
system tailored for the elderly population, showing significant improvements
in detecting happiness and sadness.

Other approaches integrated various modalities for stress detection. For
instance, [26] integrated ECG, voice, and facial expression data, achieving an
accuracy of 85.1% in stress detection. Meanwhile, [27] proposed subject-
independent emotion recognition using EEG signals, employing a Semi-Supervised
Dimension Reduction algorithm (SSDR) to reduce deep feature dimensions and
improve classification accuracy. Notably, fusion after profound feature
reduction yielded the highest accuracy. Lastly, [28] combined EEG and speech
features for emotion recognition, employing a weighted fusion approach and
achieving an average accuracy of 86.59%.

These studies underscore the significance of employing multiple modalities for
stress detection, as they enable the integration of complementary information.
However, there remains a need for improved, more accurate, and faster stress
detection methods suitable for real-world scenarios, facilitating timely
stress monitoring and management.

This paper focuses on exploring the computational cost and performance trade-
offs of various Manifold-based dimensional reduction methods within an
intermediate-level fusion network. We compare these methods to unimodal and
early fusion networks for stress detection. Among these dimensionality
reduction techniques, our experiments reveal that Multidimensional Scaling
(MDS) achieves a remarkable accuracy of 96% when utilized in an intermediate-
level fusion network, particularly in a Leave-One-Subject-Out Cross-Validation
(LOSO-CV) framework.

##  III Multimodal Intermediate Level Fusion Network

Figure 1 illustrates our proposed intermediate fusion network. This model
employs manifold dimensional reduction before the fusion of the
representations of each modality separately.

Figure 1: Intermediate Level Fusion Network

In this network architecture, the process begins with the preprocessing of
biometric signals and facial landmarks. The biometric signals are input into a
$3\times 1$ CNN layer, while in parallel, a $3\times 3$ CNN layer is
responsible for generating representations of facial landmarks. Subsequently,
both sets of representations go through $(2\times 1)$ and $(2\times 2)$ max-
pooling layers, followed by flattening layers.

The representations derived from biometric signals and facial landmarks are
then combined through concatenation before being fed into a series of fully
connected Deep Neural Network (DNN) layers. The DNN layers consist of the
following components: A fully connected layer with 16 neurons employing a ReLU
activation function, followed by a 0.2 dropout, and a layer with eight neurons
utilizing ReLU activation, L2 regularization with a coefficient of 0.01,
followed by a 0.2 dropout. The final output layer comprises three neurons and
employs a softmax activation function for the purpose of classifying stress
into three distinct levels.

The source code and network architectures related to this work are made
publicly available through the CPHS lab at the University of Louisiana at
Lafayette [CPHSLAB].

##  IV Manifold learning based Dimensionality Reduction

Manifold learning is a technique that uncovers the underlying structure or
geometry within data and then transforms it into a space with fewer
dimensions. This approach allows us to gain insights into high-dimensional
data either by projecting it onto a lower-dimensional space or by learning the
mapping itself. By identifying a non-linear mapping that connects the high-
dimensional space to a lower-dimensional manifold, manifold learning preserves
the inherent relationships between the data points while simultaneously
simplifying computational complexity [29]. We employed six different dimension
reduction methods, including Locally Linear Embedding (LLE)[30], Spectral
Embedding (SE)[31], Multidimensional Scaling (MDS)[32], Isometric Mapping
(ISO)[33], t-Distributed Stochastic Neighbor Embedding (t-SNE)[34], and
Principal Component Analysis (PCA)[35]. The following sections explain various
manifold learning techniques.

###  IV-A Locally Linear Embedding (LLE)

The LLE method, introduced by [30], is an unsupervised dimensionality
reduction technique that attempts to retain the essential geometric
characteristics of the original nonlinear feature structure. The method aims
to find a lower-dimensional representation of the data while ensuring that the
distances between the k-nearest neighbors are maintained. It achieves this by
modeling the data as a linear combination of its neighboring points and then
determining the weights that minimize the dissimilarity between the original
data and its reconstructed version.

To illustrate the performance of LLE, we provide a 2D visualization of our
data in Figure 2, along with the time it took to reduce their dimension using
LLE. The plots clearly demonstrate that LLE encountered challenges in
effectively clustering the data while preserving the local structure. For this
visualization, we employed the following parameters: Number of neighbors = 10,
Number of Components = 2, method = ’standard’, and random state = 42.

Figure 2: Biometric and Landmarks data visualization using LLE

###  IV-B Spectral Embedding (SE)

SE is another manifold nonlinear dimension reduction technique introduced in
2002 by [31]. SE simplifies complex data relationships by analyzing the
connections between points using Laplacian Eigenmaps and creating a smaller
data version. This helps to understand complex relationships that do not
follow a straight line. To visualize our data in 2D using SE, we used the
following parameters: Number of neighbors = 5, Number of Components = 2, and
random state = 42. Figure 3 shows the 2D visualization of our data using SE.
As shown in the plots, SE exhibited nonsensical clustering in the data and
yielded the worst performance due to its limitation in capturing only the
global aspect of the data, making it unsuitable for our dataset.

Figure 3: Biometric and Landmarks data visualization using SE

###  IV-C Multi-Dimensional Scaling (MDS)

MDS measures distances or similarities between data points in high-dimensional
data and generates a new map that accurately retains these relationships. This
method captures a strong local structure of the data based on pairwise
distances between the data points [32]. The visualization of our data using 2D
MDS is illustrated in Figure 4. We used the following parameters for
visualization: Number of Components = 2 and random state = 42. Our study shows
that MDS plots effectively cluster the data, especially for biometric signals,
making it the best-performing technique in this study when using intermediate-
level fusion. However, the downside of MDS is its high computational cost,
which needs to be considered. Despite this, MDS is a powerful method with
notable strengths.

Figure 4: Biometric and Landmarks data visualization using MDS

###  IV-D Isometric Mapping (ISO)

ISO is an extension of MDS. It uses graph distance, which approximates the
geodesic distance between all pairs of points [33]; in Figure 5, you can see
the 2D visualizations of our data obtained using ISO. We used the following
parameters for visualization: Number of Neighbors = 10, Number of Components =
2, and random state = 42. The plots obtained through ISO exhibit effective
clustering, particularly for biometric signals data, while retaining the
global structure of the data. In this study, the accuracy yielded by ISO and
MDS was almost the same in an unimodal network for biometric signals.

Figure 5: Biometric and Landmarks data visualization using ISO

###  IV-E t-Distributed Stochastic Neighbor Embedding (t-SNE)

t-SNE models each high-dimensional object by a point so that nearby points
model similar objects, and dissimilar objects are modeled by distant points
with high probability, as described in [34]. We used t-SNE to visualize our
data in 2D, as shown in Figure 6. The visualization was performed using the
following parameters: perplexity = 30, Number of Components = 2, and random
state = 42. The t-SNE plots exhibit promising results in clustering the data,
primarily landmarks data, as they capture the local structure of the data.
t-SNE ranks as the second method in our study with a high computational cost,
and it also corresponds to the second-worst performer in terms of overall
performance in multimodal networks. This trade-off between clustering efficacy
and computational expense should be carefully considered when choosing t-SNE
for dimensionality reduction in similar studies.

Figure 6: Biometric and Landmarks data visualization using t-SNE

###  IV-F Principal Component Analysis (PCA)

PCA is a linear, well-known method for reducing the dimensions of high-
dimensional data. It works by finding the directions of maximum variance and
projecting them onto a new subspace with fewer dimensions [35]. The figures in
7 show how our data is visualized in 2D using PCA. We used the following
parameters to generate these visualizations: Number of Components = 2 and
random state = 42. PCA plots are effective in clustering, primarily for
biometric data and preserving the global structure of the data. The PCA
algorithm is more computationally efficient than the manifold methods used in
this study. PCA performs the best when using unimodal networks but yields
lower performance when it comes to multimodal learning.

Figure 7: Biometric and Landmarks data visualization using PCA

##  V Results and Analysis

We utilized the EmpathicSchool dataset, as introduced in [36]. This dataset
has biometric signals, including heart rate, electrodermal activity, skin
temperature, and accelerometer data, alongside video recordings, all of which
were employed for stress detection purposes. To efficiently process emotional
information without incurring excessive computational costs, we employed the
dlib library [37] to extract facial landmarks from video data.

For consistency with prior work [2], we set the signal frequency to 1 Hz. To
extract relevant features in both the time and frequency domains, we applied a
rolling window of 20 seconds with a 10-second overlap. Within each window, we
calculated the average stress level of the user, which served as the stress
label. To simplify the stress levels, originally ranging from 0 to 19, we
reclassified them into three categories: no-stress (0) for values less than or
equal to 6.5, medium-stress (1) for values between 6.5 and 13, and stressful
(2) for values exceeding 13.

We applied random down-sampling to address potential data imbalances and
minimize bias in our performance evaluation. Furthermore, we independently
normalized the minimum and maximum values of each signal for each user. This
normalization process helped maintain the original data distribution and
mitigated the impact of inter-subject variability.

The resulting dataset featured 1904 features derived from facial landmarks and
175 features from biometric signals. Subsequently, we reduced the data
dimension to 49 for facial landmarks and 20 for biometric signals.

We first compared the effect of six manifold learning methods on unimodal
networks, and then we applied the manifold learning to the multimodal models.

###  V-A Unimodal Networks

Figure 8 shows the unimodal network visualization. In these networks, after
preprocessing the signals, the data is fed into a $(3\times 1)$ CNN for the
biometric signals or a $(3\times 3)$ CNN for facial landmarks, followed by a
$(2\times 1)$ or a $(2\times 2)$ max-pooling, flattened layer, and DNN layers.
The DNN layers consist of a fully connected layer with 16 neurons and a ReLU
activation function, followed by a 0.2 dropout. The next layer consists of
eight neurons, ReLU activation, L2 regularization of (0.01), followed by 0.2
dropout. The final output layer includes three neurons and a softmax
activation function to be consistent with the proposed network. We compare
unimodal and multimodal networks to evaluate the effectiveness of multimodal
networks in capturing complementary information from different modalities. We
investigate the performance computational cost trade-offs of manifold learning
using unimodal models.

Figure 8: Separate unimodal networks for stress detection using biometric
signals and facial landmarks.

Table I and II show the performance of unimodal deep neural networks using
manifold learning. Using LOSO-CV, we trained and tested the deep neural
network models for biometric signals (Bio) and facial landmarks (Land)
separately. However, manifold learning could not outperform PCA in unimodal
networks. Moreover, the PCA has shown lower computational cost than the rest
of the dimensionality reduction methods.

TABLE I: Performance comparison of manifold learning methods on unimodal network using Biometric Signals method | LLE | SE | MDS | ISO | t-SNE | PCA
---|---|---|---|---|---|---
Acc(%) | 75.27 | 64.35 | 81.62 | 80.04 | 48.84 | 82.22
Pre(%) | 70.66 | 66.57 | 76.19 | 74.02 | 46.25 | 84.13
Rec(%) | 73.34 | 59.62 | 80.59 | 74.27 | 46.83 | 82.10
F1(%) | 69.77 | 54.40 | 76.52 | 72.85 | 42.66 | 80.11
TABLE II: Performance comparison of manifold learning methods on unimodal network using Facial Landmarks Method | LLE | SE | MDS | ISO | t-SNE | PCA
---|---|---|---|---|---|---
Acc(%) | 59.77 | 36.68 | 69.25 | 55.26 | 61.96 | 74.13
Pre(%) | 58.61 | 44.75 | 64.88 | 53.74 | 67.98 | 70.56
Rec(%) | 59.55 | 45.61 | 65.17 | 52.98 | 57.50 | 69.63
F1(%) | 54.46 | 31.64 | 62.40 | 46.68 | 54.53 | 67.52

###  V-B Multimodal Early Level Fusion Network

Figure 9 shows the multimodal early-fusion learning network. The dimension
reduction method was applied separately to each data type and then
concatenated and fed to a 1D-CNN layer followed by max-pooling, flattening,
and DNN hidden layers. The DNN layers were consistent throughout all models to
facilitate the comparisons. This allows the researchers to compare the effect
of adding the manifold learning models on the performance and computational
cost of different multimodal learning models.

Figure 9: Multimodal early fusion network for stress detection using biometric
signals and facial landmarks.

Table III summarizes the performance of the manifold dimension reduction
methods on the early fusion network. LLE showed promising results and
outperformed other methods with an accuracy of 90.99%, followed by ISO and
PCA. Manifold learning has enhanced feature integration, while multimodal
learning combines features from multiple modalities, leading to a richer data
representation [38]. LLE performed better than PCA in the early fusion
network, attributed to their ability to effectively capture and preserve the
intrinsic geometry [39] and structure of the combined biometric signals and
facial landmarks.

TABLE III: Performance comparison of manifold learning methods on Early Fusion network Method | LLE | SE | MDS | ISO | t-SNE | PCA
---|---|---|---|---|---|---
Acc(%) | 90.99 | 71.45 | 87.80 | 89.68 | 74.14 | 89.01
Pre(%) | 86.22 | 75.86 | 83.00 | 87.49 | 68.26 | 90.84
Rec(%) | 86.87 | 64.95 | 85.71 | 88.67 | 68.53 | 87.92
F1(%) | 86.13 | 61.62 | 82.84 | 87.29 | 66.30 | 86.76

###  V-C Multimodal Intermediate Level Fusion Network

Table IV shows the performance of the intermediate-level fusion network using
various manifold learning models discussed in Section 4. The results indicate
that utilizing the MDS method outperformed other techniques, achieving an
accuracy of 96%. PCA and ISO are followed closely, with accuracies of 93.14%
and 88.48%, respectively. The results show that the MDS accurately represents
the pairwise distance in our data and can preserve similarities between data
points. Biometric signals and facial landmarks are complex data types that
have non-linear relationships between their variables. MDS’s ability to
capture these relationships becomes crucial in this matter. Our study showed
that choosing a suitable dimension reduction method and the intermediate
fusion of convolution layers can result in accurate stress detection compared
to early multimodal fusion and unimodal networks. The strength of the 2D-CNN
model lies in its ability to extract spatial information from facial
landmarks, while the 1D-CNN model excels in capturing temporal dependencies
from biometric signals. This integration of spatiotemporal features is crucial
for stress detection, as stress manifests in both spatial and temporal
aspects. The findings underscore the significance of manifold learning methods
in enhancing stress detection performance and cost within the context of an
intermediate fusion network.

TABLE IV: Performance comparison of manifold learning methods on Intermediate Fusion network Method | LLE | SE | MDS | ISO | t-SNE | PCA
---|---|---|---|---|---|---
Acc(%) | 83.16 | 46.44 | 96.00 | 88.48 | 78.51 | 93.14
Pre(%) | 81.74 | 38.56 | 93.97 | 84.96 | 73.68 | 92.64
Rec(%) | 81.24 | 38.47 | 96.67 | 82.98 | 74.63 | 90.59
F1(%) | 79.04 | 31.57 | 94.99 | 82.96 | 72.20 | 89.63

##  VI Computational Cost

Table V shows the computational cost of each network in terms of the dimension
reduction (DR), training, and prediction times. We used a two-28-core (56
CPUs) node with an Intel(R) Xeon(R) CPU E5-2660 v4 @ 2.00GHz, with L1d, L1i,
L2, and L3 cache sizes of 32K, 32K, 256K, and 35840K, respectively. The node
was equipped with a P100 Tesla GPU for deep-learning computations.

Manifold dimension reduction methods had higher computational costs than PCA,
completing the process in 270 milliseconds for biometric signals (Bio) and
3.28 seconds for facial landmarks (Land). However, the MDS method took the
most dimension reduction time, requiring 368 and 483 seconds to reduce the Bio
and Land data dimensions, respectively (approximately 100 to 1000 times
slower).

[2] explored six different feature selection methods, determining that Lasso
regularization outperformed others and yielded the best performance for
multimodal deep learning models. The Lasso features selection method took 138
and 1001 seconds to select 20 features from biometric signals and 49 features
from facial landmarks, respectively. In contrast, the MDS approach achieved
the same dimensional reduction, resulting in higher overall performance in the
multimodal methods in a shorter time, showing a 25% improvement in
preprocessing time. This shows the effectiveness of manifold learning in
computational cost reduction.

TABLE V: Comparing Computational Cost of the Methods in Unimodal, Early, and
Intermediate Networks

Method |  | DR Time (s)
---
Number of Parameters | Train Time (s) | Test Time (s)
Bio | Land | Unimodal | Early | Intermediate | Unimodal | Early | Intermediate | Unimodal | Early | Intermediate
Bio | Land | Bio | Land | Bio | Land
LLE | 7.99 | 10.10 | 5,043 | 2,675 | 5,043 | 82,099 | 413 | 346 | 402 | 674 | 0.084 | 0.080 | 0.081 | 0.394
SE | 118 | 42.70 | 258 | 236 | 343 | 391 | 0.079 | 0.064 | 0.082 | 0.387
MDS | 368 | 483 | 350 | 254 | 413 | 466 | 0.077 | 0.108 | 0.097 | 0.403
ISO | 8.44 | 12.60 | 359 | 281 | 389 | 510 | 0.078 | 0.065 | 0.084 | 0.389
t-SNE | 62 | 414 | 298 | 282 | 435 | 592 | 0.078 | 0.065 | 0.080 | 0.375
PCA | 0.27 | 3.28 | 343 | 238 | 326 | 468 | 0.077 | 0.064 | 0.122 | 0.383

###  VI-A Effect of adding extra non-linearity using 1D-CNN

Adding an extra 1D-CNN layer, followed by a max-pooling, right after data
fusion yields performance improvement by adding non-linearity to the model,
enhancing the average performance of the models by an increase of 7.41% in
accuracy, 6.18% in precision, 9.23% in recall, and 9.42% in f1-score. This
final 1D-CNN layer plays a significant role in capturing higher-level patterns
by combining extracted features from both modalities [24].

##  VII Discussion and Future works

In this paper, we presented a streamlined intermediate-level fusion network
that leverages manifold learning techniques to enhance stress detection.
Within this model, manifold-based dimension reduction is separately applied to
each signal in the initial stages. This unique approach enables the extraction
of meaningful representations from biometric signals and facial landmarks,
both of which possess intricate and non-linear relationships, independently.
These representations are later combined to inform the stress detection
process. It’s worth noting that this approach can be customized to different
levels for various network configurations.

In this study, we demonstrate the efficacy of non-linear dimension reduction
methods within intermediate multimodal neural networks when compared to early
fusion and unimodal models. Notably, our model, incorporating Multidimensional
Scaling (MDS), outperforms the other models, while Locally Linear Embedding
(LLE) strikes a balance between performance and computational costs. Principal
Component Analysis (PCA) offers higher processing speed due to its efficiency.
However, the incorporation of manifold learning techniques results in a
substantial 41.69% reduction in errors (from 93.14% to 96%) compared to using
PCA.

We plan to investigate the effect of implementing manifold learning the
multimodal learning network at various levels of fusion: multimodal early,
late, and intermediate-level fusion models. Another research area is
investigating how manifold learning preserves complementary information to
enhance multimodal learning for stress detection.

##  VIII Summary and Conclusions

In summary, this study introduced a lightweight intermediate fusion network
that employs manifold learning for dimensionality reduction. We systematically
investigated the impact of incorporating manifold learning in the
preprocessing phase across unimodal, early, and intermediate-fusion networks.
Remarkably, manifold learning yielded a substantial 41.69% reduction in errors
when compared to Principal Component Analysis (PCA) within intermediate-level
fusion networks. Our analysis involved the examination of various manifold
dimension reduction methods, assessing their effects on stress detection
performance and computational costs. We evaluated six manifold learning
models, including LLE, SE, MDS, ISO, t-SNE, and PCA.

The results underscored the superiority of the intermediate-level fusion model
that integrated the MDS manifold learning dimensionality reduction method,
achieving an impressive 96% accuracy in the Leave-One-Subject-Out Cross-
Validation (LOSO-CV) paradigm. Furthermore, the implementation of data
balancing through down-sampling played a pivotal role in reducing dimension
reduction and train-test times, ensuring that performance assessments remained
unbiased in the context of stress detection.

Additionally, the inclusion of an extra 1D-CNN layer had a substantial
positive impact on the model’s performance, leading to a 7.41% increase in
accuracy, a 6.18% boost in precision, a 9.23% improvement in recall, and a
9.42% enhancement in the f1-score. These findings collectively highlight the
significance of manifold learning, data balancing, and architectural
enhancements in advancing the state-of-the-art in stress detection techniques.

## References

  * [1] Russell Li and Zhandong Liu  “Stress detection using deep neural networks”  In _BMC Medical Informatics and Decision Making_ 20 Springer, 2020, pp. 1–10
  * [2] Majid Hosseini et al.  “Multimodal Stress Detection Using Facial Landmarks and Biometric Signals”  In _arXiv preprint arXiv:2311.03606_ , 2023
  * [3] Seyedmajid Hosseini et al.  “A multimodal sensor dataset for continuous stress detection of nurses in a hospital”  In _Scientific Data_ 9.1 Nature Publishing Group UK London, 2022, pp. 255
  * [4] Ioannis Kapsouras, Spiros Nikolopoulos and Ioannis Kompatsiaris  “A Deep Learning Approach Towards Multimodal Stress Detection”  In _Proceedings of the 2nd International Workshop on Affective Computing_ , 2020, pp. 1–8
  * [5] Shengdong Zhao et al.  “Stress Detection via Multimodal Multitemporal-Scale Fusion: A Hybrid of Handcrafted and Deep Learning Features”  In _IEEE Transactions on Affective Computing_ IEEE, 2023
  * [6] Zhenyu Zhang et al.  “Multimodal Stress Detection from Multiple Heterogeneous Data Sources”  In _Proceedings of the 27th ACM International Conference on Multimedia_ , 2019, pp. 2410–2418
  * [7] Hector M Romero Ugalde et al.  “Computational cost improvement of neural network models in black box nonlinear system identification”  In _Neurocomputing_ 166 Elsevier, 2015, pp. 96–108
  * [8] Xunyu Zhu et al.  “A Survey on Model Compression for Large Language Models”  In _arXiv preprint arXiv:2308.07633_ , 2023
  * [9] Jing Wang, Haibo He and Danil V Prokhorov  “A folded neural network autoencoder for dimensionality reduction”  In _Procedia Computer Science_ 13 Elsevier, 2012, pp. 120–127
  * [10] Pushparaja Murugan and Shanmugasundaram Durairaj  “Regularization and optimization strategies in deep convolutional neural network”  In _arXiv preprint arXiv:1712.04711_ , 2017
  * [11] Abdullah Al-Dujaili and Saad Al-Saadi  “Review of deep learning: concepts, CNN architectures, challenges, applications, future directions”  In _Journal of Big Data_ 8.1 Springer, 2021, pp. 1–49
  * [12] Lam Ho Nguyen and Susan Holmes  “Ten quick tips for effective dimensionality reduction”  In _PLOS Computational Biology_ 15.6 Public Library of Science, 2019, pp. e1006907
  * [13] Saad Al-Saadi, Abdullah Al-Dujaili and Jingpeng Li  “Redundancy detection and removal in big data: A comprehensive review”  In _IEEE Access_ 7 IEEE, 2019, pp. 106328–106344
  * [14] Weilin Huang and Hujun Yin  “On nonlinear dimensionality reduction for face recognition”  In _Image and Vision Computing_ 30.4-5 Elsevier, 2012, pp. 355–366
  * [15] Moulay A Akhloufi, Abdelhakim Bendada and Jean-Christophe Batsale  “Multispectral face recognition using non linear dimensionality reduction”  In _Visual Information Processing XVIII_ 7341, 2009, pp. 152–161  SPIE
  * [16] Malte Hoffmann et al.  “Learning multi-modal image registration without real data”  In _arXiv preprint arXiv:2004.10282_ , 2020
  * [17] Aditya Dutt, Alina Zare and Paul Gader  “Shared Manifold Learning Using a Triplet Network for Multiple Sensor Translation and Fusion With Missing Data”  In _IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing_ 15 IEEE, 2022, pp. 9439–9456
  * [18] Qi Zhu et al.  “Multimodal brain network jointly construction and fusion for diagnosis of epilepsy”  In _Frontiers in Neuroscience_ 15 Frontiers, 2021, pp. 734711
  * [19] Chaofan Song et al.  “Multi-modal feature selection with self-expression topological manifold learning”  In _Mathematical Biosciences and Engineering_ 18.5 American Institute of Mathematical Sciences, 2021, pp. 6468–6482
  * [20] Rahee Walambe, Pranav Nayak, Ashmit Bhardwaj and Ketan Kotecha  “Employing Multimodal Machine Learning for Stress Detection”  In _arXiv preprint arXiv:2306.09385_ , 2023
  * [21] Yujin Wu et al.  “Fusion of Physiological and Behavioural Signals on SPD Manifolds with Application to Stress and Pain Detection”  In _2022 IEEE International Conference on Systems, Man, and Cybernetics (SMC)_ , 2022, pp. 2949–2955  IEEE
  * [22] Wonju Seo, Namho Kim, Cheolsoo Park and Sung-Min Park  “Deep learning approach for detecting work-related stress using multimodal signals”  In _IEEE Sensors Journal_ 22.12 IEEE, 2022, pp. 11892–11902
  * [23] Yong Zhang, Cheng Cheng and YiDie Zhang  “Multimodal emotion recognition based on manifold learning and convolution neural network”  In _Multimedia Tools and Applications_ 81.23 Springer, 2022, pp. 33253–33268
  * [24] K Radhika and V Ramana Murthy Oruganti  “Deep multimodal fusion for subject-independent stress detection”  In _2021 11th International Conference on Cloud Computing, Data Science & Engineering (Confluence)_, 2021, pp. 105–109  IEEE
  * [25] P Sreevidya, S Veni and OV Ramana Murthy  “Elder emotion classification through multimodal fusion of intermediate layers and cross-modal transfer learning”  In _Signal, image and video processing_ 16.5 Springer, 2022, pp. 1281–1288
  * [26] Jing Zhang et al.  “Real-time mental stress detection using multimodality expressions with a deep learning framework”  In _Frontiers in Neuroscience_ 16 Frontiers, 2022, pp. 947168
  * [27] Behrooz Zali-Vargahan, Asghar Charmin, Hashem Kalbkhani and Saeed Barghandan  “Deep time-frequency features and semi-supervised dimension reduction for subject-independent emotion recognition from multi-channel EEG signals”  In _Biomedical Signal Processing and Control_ 85 Elsevier, 2023, pp. 104806
  * [28] Qingshan She et al.  “Cross-subject EEG emotion recognition using multi-source domain manifold feature selection”  In _Computers in Biology and Medicine_ 159 Elsevier, 2023, pp. 106860
  * [29] Yunqian Ma and Yun Fu  “Manifold learning theory and applications”  CRC press, 2011
  * [30] Sam T Roweis and Lawrence K Saul  “Nonlinear dimensionality reduction by locally linear embedding”  In _science_ 290.5500 American Association for the Advancement of Science, 2000, pp. 2323–2326
  * [31] Mikhail Belkin and Partha Niyogi  “Laplacian eigenmaps for dimensionality reduction and data representation”  In _Neural computation_ 15.6 MIT Press, 2003, pp. 1373–1396
  * [32] Ingwer Borg and Patrick JF Groenen  “Modern multidimensional scaling: Theory and applications”  Springer Science & Business Media, 2005
  * [33] Joshua B Tenenbaum, Vin de Silva and John C Langford  “A global geometric framework for nonlinear dimensionality reduction”  In _science_ 290.5500 American Association for the Advancement of Science, 2000, pp. 2319–2323
  * [34] Laurens Van der Maaten and Geoffrey Hinton  “Visualizing data using t-SNE.”  In _Journal of machine learning research_ 9.11, 2008
  * [35] Ian T Jolliffe and Jorge Cadima  “Principal component analysis: a review and recent developments”  In _Philosophical transactions of the royal society A: Mathematical, Physical and Engineering Sciences_ 374.2065 The Royal Society Publishing, 2016, pp. 20150202
  * [36] Majid Hosseini et al.  “EmpathicSchool: A multimodal dataset for real-time facial expressions and physiological data analysis under different stress conditions”  In _arXiv preprint arXiv:2209.13542_ , 2022
  * [37] Davis E King  “Dlib-ml: A Machine Learning Toolkit”  In _Journal of Machine Learning Research_ 10, 2009, pp. 1755–1758
  * [38] Fadi Dornaika  “Multi-layer manifold learning with feature selection”  In _Applied Intelligence_ 50.6 Springer, 2020, pp. 1859–1871
  * [39] Bo Jiang, Chris Ding and Bin Luo  “Robust data representation using locally linear embedding guided PCA”  In _Neurocomputing_ 275 Elsevier, 2018, pp. 523–532

CPHSLAB