# Extended Gevrey regularity via weighted matrices

Nenad Teofanov  Department of Mathematics and Informatics, University of Novi
Sad, Novi Sad, Serbia<EMAIL_ADDRESS>and  Filip Tomić  Faculty
of Technical Sciences, University of Novi Sad, Novi Sad, Serbia
<EMAIL_ADDRESS>
###### Abstract.

The main aim of this paper is to compare two recent approaches for
investigating the interspace between the union of Gevrey spaces
$\mathcal{G}_{t}(U)$ and the space of smooth functions $C^{\infty}(U)$. The
first approach in the style of Komatsu is based on the properties of two
parameter sequences $M_{p}=p^{\tau p^{\sigma}}$, $\tau>0$, $\sigma>1$. The
other one uses weight matrices defined by certain weight functions. We prove
the equivalence of the corresponding spaces in the Beurling case by taking
projective limits with respect to matrix parameters, while in the Roumieu case
we need to consider a larger space then the one obtained as the inductive
limit of extended Gevrey classes.

###### Key words and phrases:

Ultradifferentiable functions; defining sequences; weight matrices; Gevrey
classes

###### 2000 Mathematics Subject Classification:

46F05, 46E10

##  1\. Introduction

Classes of ultradifferentable functions (also known as smooth functions of
ultra-rapid decay) are usually studied in the framework of one of the two most
widely used approaches. The first one is based on the properties of the
defining sequences $M_{p}$, $p\in\mathbf{N}$, which control the derivatives of
the functions, [8]. For the same purpose, the other approach uses weights with
the certain asymptotical properties, [2, 10]. The relation between these
weights and the so-called associated functions to $M_{p}$ sequences provides a
way to compare the theories of ultradifferentable functions and their dual
spaces of ultradistributions. In many situations these approaches are
equivalent. For example, it is proved in [10] that the corresponding classes
of functions are equal if the sequence $M_{p}$ satisfies Komatsu’s conditions
$(M.1)$, $(M.2)$ and $(M.3)$, see Section 2. These conditions are relaxed in
[1] where $(M.3)$ is replaced by

$(\exists Q\in\mathbf{N})\quad\liminf_{p\to\infty}\frac{m_{Qp}}{m_{p}}>1,$
(1.1)

with $\displaystyle m_{p}=M_{p}/M_{p-1}$.

In this paper we study the equivalence of the approaches by considering
specific sequences which do not satisfy $(M.2)$. To that end we exploit the
powerful technique based on weight matrices introduced in [14]. Broadly
speaking, weight matrices are families of sequences. For instance,
$\\{p!^{t}\\}_{t>0}$ is a weight matrix that consists of Gevrey sequences.
More generally, for a given weight function $\omega$ (see Subsection 1.1 for
the definition) one can observe matrices of the form
$\displaystyle{\mathcal{M}}=\\{M_{p}^{H}\\}_{H>0}$ where

$\displaystyle M_{p}^{H}=e^{\frac{1}{H}\varphi^{*}(Hp)},\quad p\in\mathbf{N},$
(1.2)

and $\varphi^{*}$ is the Young conjugate of
$\displaystyle\varphi(t)=\omega(e^{t})$, see (1.6). This approach allows to
prove that the corresponding classes of functions are equivalent in certain
situations even if $(M.2)$ is violated, see [14, 6].

We consider the two parameter defining sequences of the form
$M^{\tau,\sigma}_{p}=p^{\tau p^{\sigma}}$, $\tau>0$, $\sigma>1$, cf. [11].
Such sequences do not satisfy $(M.2)$ for any choice of parameters $\tau>0$
and $\sigma>1$, hence the corresponding classes of functions
$\mathcal{E}_{\tau,\sigma}(U)$ (extended Gevrey classes) are not
ultradifferentiable. However, related ultradifferentiable classes can be
obtained by taking their unions and intersections (inductive and projective
limits) with respect to the parameter $\tau$ (this follows from Proposition
2.1 iv)).

Extended Gevrey regularity turned out to be convenient when describing certain
aspects of some hyperbolic PDE’s. In particular, $\mathcal{E}_{1,2}(U)$
appears in the study of local solvability of strictly hyperbolic PDE’s, for
which the initial value problem is ill posed in the Gevrey settings (see [4]).
In addition, sequences $M^{\tau,\sigma}_{p}$ for $1<\sigma\leq 2$ are recently
used in [7] to study the surjectivity of Borel maps for ultraholomorphic
classes. For more details concerning $M^{\tau,\sigma}_{p}$ and
$\mathcal{E}_{\tau,\sigma}(U)$ we refer to [11, 12, 13].

In this paper we prove that the extended Gevrey classes are the special case
of classes investigated [6, 14] only when considering projective and inductive
limits with respect to the (matrix) parameter $\tau$. More precisely, in the
Beurling case we prove the equality of the corresponding spaces, while in the
Roumieu case the equivalence holds when the corresponding inductive limit is
replaced by a larger space of test functions (see (2.14) and (3.7)).

We start by proving that the function $T_{\tau,\sigma,h}(k)$ associated to the
sequence $M^{\tau,\sigma}_{p}=p^{\tau p^{\sigma}}$ is equivalent to a weight
function in the sense of [1] (see Theorem 3.1). For that purpose we need to
estimate $T_{\tau,\sigma,h}(k)$. This is done in [12, Theorem 2.1] by using
the properties of the Lambert $W$ function. In Proposition 3.1 (see also Lemma
2.2) we use another technique to obtain similar estimates. Consequently, we
conclude that $\displaystyle\\{M^{\tau,\sigma}_{p}\\}_{\tau>0}$ and
$\displaystyle\\{e^{\frac{1}{H}\varphi_{\sigma}^{*}(Hp)}\\}_{H>0}$ are
equivalent matrices for a suitable function $\varphi_{\sigma}$, which implies
that the corresponding classes of functions given by these matrices coincide.

Although Theorem 3.1, as the main result of the paper, connects the approach
from [11, 12, 13] with the one given in [14, 6], let us mention an important
difference between them. In contrast to the usual Carleman classes and
corresponding part in [14, 6], in the norm (2.8) we consider
$h^{|\alpha|^{\sigma}}$, $\sigma>1$, in denominator. Thus the parameter
$\sigma$ plays an important role in our construction which can not be revealed
by using the techniques from [14, 6]. For example, the spaces
$\mathcal{E}_{\tau,\sigma}(U)$ are closed under finite order differentiation
for any choice of parameters $\tau>0$ and $\sigma>1$. In addition, the
parameters $h$ and $\sigma$ provide a ”fine tuning” in the gap between the
union of Gevrey spaces and $C^{\infty}$ (see Proposition 2.1 i)).

We end this introductory section with a review of some basic notions.

###  1.1. Basic notions and notation

We will use the standat notation ${\bf N}$, $\mathbf{Z}_{+}$, ${\bf R}$, ${\bf
R}_{+}$, ${\bf C}$, for the sets of nonnegative integers, positive integers,
real numbers, positive real numbers and complex numbers, respectively. The
floor function of $x\in{\bf R}_{+}$ is denoted by $\lfloor
x\rfloor:=\max\\{m\in\mathbf{N}\,:\,m\leq x\\}$. For a multi-index
$\alpha=(\alpha_{1},\dots,\alpha_{d})\in{\bf N}^{d}$ we write
$\partial^{\alpha}=\partial^{\alpha_{1}}\dots\partial^{\alpha_{d}}$ and
$|\alpha|=|\alpha_{1}|+\dots|\alpha_{d}|$. By $\\#A$ we denote the number of
elements of the finite set $A$. We write $\ln_{+}x=\max\\{0,\ln x\\}$, $x>0$.

An essential role in our analysis is played by the _Lambert $W$ function_,
which is defined as the inverse of $ze^{z}$, $z\in{\bf C}$. By $W(x)$, $x\geq
0$, we denote the restriction of its principal branch, and we review some of
its basic properties as follows:

  * $(W1)\quad$

$W(0)=0$, $W(e)=1$, $W(x)$ is continuous, increasing and concave on
$[0,\infty)$,

  * $(W2)\quad$

$W(xe^{x})=x$ and $x=W(x)e^{W(x)}$, $x\geq 0$,

  * $(W3)\quad$

$\displaystyle\ln x-\ln(\ln x)\leq W(x)\leq\ln x-\frac{1}{2}\ln(\ln x)$,
$x\geq e$.

Note that $(W2)$ implies

$W(x\ln x)=\ln x,\quad x>1.$ (1.3)

By using $(W3)$ we obtain

$W(x)\sim\ln x,\quad x\to\infty,$ (1.4)

and therefore

$W(Cx)\sim W(x),\quad x\to\infty,$ (1.5)

for any $C>0$. We refer to [5] for more details concerning the Lambert
function.

Recall (see [1]), a non-negative, continuous, even and increasing function
$\omega$ defined on $\mathbf{R}$, $\omega(0)=0$, is called _weight function_
if it satisfies the following conditions:

  * ($\alpha$)

$\omega(2t)=O(\omega(t)),\quad t\to\infty,$

  * ($\beta$)

$\omega(t)=O(t),\quad t\to\infty$

  * ($\gamma$)

$o(\omega(t))=\log t,\quad t\to\infty,$

  * ($\delta$)

$\varphi(t)=w(e^{t}),\quad{\rm is\,convex}.$

Young’s conjugate of the function $\varphi$ (defined as above) is given by

$\displaystyle\varphi^{*}(k)=\sup_{t>0}(kt-\varphi(t)),\quad k\geq 0.$ (1.6)

Some classical examples of weight functions are

$\omega(t)=\ln^{s}_{+}|t|,\quad\quad\omega(t)=\frac{|t|}{\ln^{s-1}(e+|t|)},\quad
s>1,\,t\in\mathbf{R}.$ (1.7)

Moreover, $\omega(t)=|t|^{s}$ is a weight function if and only if $0<s\leq 1$.
Note that by (1.4) it follows that $\omega(t)=W(|t|)$ is not a weight
functions since the condition $(\gamma)$ is not satisfied.

Functions $f$ and $g$ are equivalent if $f=O(g)$ and $g=O(f)$, and we will
write $f\asymp g$. In particular, if $\omega$ is a weight function and
$\omega_{1}\asymp\omega$ then

$A\varphi^{*}(y/A)\leq\varphi_{1}^{*}(y)\leq B\varphi^{*}(y/B)\quad y>0,$
(1.8)

for some $A,B>0$, where $\varphi(t)=\omega(e^{t})$,
$\varphi_{1}(t)=\omega_{1}(e^{t})$ and $\varphi^{*}$, $\varphi_{1}^{*}$ are
their Young’s conjugates, respectively (see [10]).

Throughout the paper we assume that $\tau>0$ and $\sigma>1$, unless stated
otherwise.

##  2\. Preliminaries

In this section we recall the definitions of weight functions, weight
sequences and their associated functions, and classes of ultradifferentiable
functions related to the extended Gevrey regularity. We also list their main
properties that will be used in Section 3. We proceed with weight sequences
introduced in [11].

###  2.1. Weight sequences

In this subsection we consider sequences of the form
$M_{p}^{\tau,\sigma}=p^{\tau p^{\sigma}}$, $M^{\tau,\sigma}_{0}=1$, $\tau>0$,
$\sigma>1$. Since $\displaystyle(M^{\tau,\sigma}_{p})^{1/p}\to\infty$ when
$p\to\infty$ such sequences are examples of weight sequences considered in
[6].

Note that

$M^{\tau_{1},\sigma_{1}}_{p}\leq M^{\tau_{2},\sigma_{2}}_{p},\quad
0<\tau_{1}\leq\tau_{2},\quad 1<\sigma_{1}\leq\sigma_{2},\quad p\in\mathbf{N}.$
(2.1)

Moreover, $M_{p}^{\tau,\sigma}=p^{\tau p^{\sigma}}$, $\tau>0$, $\sigma>1$
($M^{\tau,\sigma}_{0}=1$), satisfies the following conditions (see [11] for
the proof):

$(M.1)$
$\displaystyle({M_{p}^{\tau,\sigma}})^{2}\leq{M_{p-1}^{\tau,\sigma}}{M_{p+1}^{\tau,\sigma}}$,
$p\in\mathbf{Z}_{+}$,

$\widetilde{(M.2)^{\prime}}$ $(\exists C>0)\quad M_{p+1}^{\tau,\sigma}\leq
C^{p^{\sigma}}M_{p}^{\tau,\sigma}$, $p\in\mathbf{N}$,

$\widetilde{(M.2)}$ $(\exists C>0)$ $M_{p+q}^{\tau,\sigma}\leq
C^{p^{\sigma}+q^{\sigma}}M_{p}^{2^{\sigma-1}\tau,\sigma}M_{q}^{2^{\sigma-1}\tau,\sigma},\quad
p,q\in\mathbf{N},$

$(M.3)^{\prime}$
$\displaystyle\sum\limits_{p=1}^{\infty}\frac{M_{p-1}^{\tau,\sigma}}{M_{p}^{\tau,\sigma}}<\infty,$

$\widetilde{(M.4)}$ $\displaystyle(\forall h>0)\,(\exists C>0)\quad
M^{\tau_{1},\sigma}_{p}\leq C\,h^{p^{\sigma}}M^{\tau_{2},\sigma}_{p},\quad
0<\tau_{1}<\tau_{2},\quad\sigma>1$,

$\widetilde{(M.5)}$ $\displaystyle(\forall h>0)\,(\exists C>0)\quad
M^{\tau_{1},\sigma_{1}}_{p}\leq
Ch^{p^{\sigma_{2}}}M^{\tau_{2},\sigma_{2}}_{p},\,\,\tau_{1},\tau_{2}>0,\,1<\sigma_{1}<\sigma_{2}.$

Note that $\widetilde{(M.4)}$ implies

$Ch^{p^{\sigma}}M^{\tau,\sigma}_{p}\geq M^{\tau/2,\sigma}_{p}\geq 1,\quad
C,h>0,\quad p\in\mathbf{N},$

and hence we obtain weaker condition

$\widetilde{(M.4)’}$ $(\forall h>0)\,(\exists
C>0)\quad{h^{p^{\sigma}}M_{p}^{\tau,\sigma}}\geq C,\quad p\in\mathbf{N}$.

######  Remark 2.1.

Let us briefly comment the case $\sigma=1$. Then the conditions
$\widetilde{(M.2)^{\prime}}$ and $\widetilde{(M.2)}$ are classical Komtasu’s
${(M.2)^{\prime}}$ and ${(M.2)}$ (respectively) for the Gevrey sequence
$M_{p}=p!^{\tau}$. Moreover, $\widetilde{(M.4)}$ also holds. The theory of
Gevrey functions is a classical one (see [9, 15] and references therein),
hence we are interested in the case $\sigma>1$.

Note that $\widetilde{(M.5)}$ is also true for the case
$\sigma_{2}>\sigma_{1}=1$ (see [11]).

Recall (see [6]), a family of weight sequences ${\mathcal{M}}$ is called
_weight matrix_ if

$(\forall M_{p},N_{p}\in{\mathcal{M}})\quad M_{p}\leq
N_{p}\,\,\vee\,\,N_{p}\leq M_{p},\quad p\in\mathbf{N}.$ (2.2)

######  Example 2.1.

For fixed $\sigma>1$ and $\sigma_{2}=\sigma_{1}=\sigma$, (2.1) implies that
${\mathcal{M}}_{\sigma}=\\{M^{\tau,\sigma}_{p}\\}_{\tau>0}$ is a weight
matrix. Similarly, ${\mathcal{M}}_{\tau}=\\{M^{\tau,\sigma}_{p}\\}_{\sigma>1}$
is a weight matrix for any given $\tau>0$. Nevertheless, if we observe
${\mathcal{M}}=\\{M^{\tau,\sigma}_{p}\\}_{\tau>0,\sigma>1}$ then for
$\tau_{1}>\tau_{2}$ and $\sigma_{1}<\sigma_{2}$ we can only prove that

$M^{\tau_{1},\sigma_{1}}_{p}\leq CM^{\tau_{2},\sigma_{2}}_{p},\quad
p\in\mathbf{N},$

for a large positive constant $C$ (see $\widetilde{(M.5)}$). Thus
${\mathcal{M}}=\\{M^{\tau,\sigma}_{p}\\}_{\tau>0,\sigma>1}$ does not satisfy
(2.2).

For two weight matrices ${\mathcal{M}}$ and ${\mathcal{N}}$ we write
${\mathcal{M}}\lesssim{\mathcal{N}}$ if

$(\forall M_{p}\in{\mathcal{M}})\,(\exists N_{p}\in{\mathcal{N}})\,(\exists
C>0)\quad M_{p}\leq C^{p}N_{p},\quad p\in\mathbf{N}.$

We say that ${\mathcal{M}}$ and ${\mathcal{N}}$ are _equivalent_ if
${\mathcal{M}}\lesssim{\mathcal{N}}$ and ${\mathcal{N}}\lesssim{\mathcal{M}}$
(see [6]).

######  Remark 2.2.

Let $\omega$ be a weight function and $\omega_{1}$ equivalent to $\omega$.
Notice that
$\displaystyle\\{M_{p}^{H}=e^{\frac{1}{H}\varphi^{*}(Hp)}\\}_{H>0}$ and
$\displaystyle\\{M_{p}^{H_{1}}=e^{\frac{1}{H_{1}}\varphi_{1}^{*}(H_{1}p)}\\}_{H_{1}>0}$
are equivalent matrices due to inequalities in (1.8). In particular, when
investigating matrices of the form (1.2) it is sufficient to consider
functions that are equivalent to weights.

Put

$\displaystyle
m^{\tau,\sigma}_{p}=\frac{M_{p}^{\tau,\sigma}}{M_{p-1}^{\tau,\sigma}},\qquad
p\in\mathbf{Z}_{+}.$ (2.3)

By $(M.1)$ it follows that $m^{\tau,\sigma}_{p}$ is an increasing sequence.
Moreover, the following Lemma holds.

######  Lemma 2.1.

Let $M_{p}^{\tau,\sigma}=p^{\tau p^{\sigma}}$, $M^{\tau,\sigma}_{0}=1$,
$\tau>0$, $\sigma>1$, and let $m^{\tau,\sigma}_{p}$ be given by (2.3). Then
the following inequalities hold

$\Big{(}\frac{e}{2^{\sigma}}\Big{)}^{\frac{\tau
p^{\sigma-1}}{2^{\sigma-1}}}p^{\frac{\tau\sigma
p^{\sigma-1}}{2^{\sigma-1}}}\leq m^{\tau,\sigma}_{p}\leq e^{\tau
p^{\sigma-1}}p^{\tau\sigma p^{\sigma-1}},\quad p\geq 2.$ (2.4)

###### Proof.

Set $\displaystyle f_{\tau,\sigma}(x)=\tau x^{\sigma}\ln x$, $x>0$. By the
mean value theorem, for every $p\in\mathbf{Z}_{+}$ there exists $\theta_{p}$
such that

$f_{\tau,\sigma}(p)-f_{\tau,\sigma}(p-1)=\tau\theta_{p}^{\sigma-1}\ln(e\theta_{p}^{\sigma}),\quad
p-1<\theta_{p}<p.$ (2.5)

For $p\geq 2\iff p/2\leq p-1$, we obtain

$\frac{\tau
p^{\sigma-1}}{2^{\sigma-1}}\ln\frac{ep^{\sigma}}{2^{\sigma}}\leq\tau(p-1)^{\sigma-1}\ln(e(p-1)^{\sigma})<\tau\theta_{p}^{\sigma-1}\ln(e\theta_{p}^{\sigma})<\tau
p^{\sigma-1}\ln(ep^{\sigma}),$

and by (2.5) we conclude

$\frac{\tau
p^{\sigma-1}}{2^{\sigma-1}}\ln\frac{ep^{\sigma}}{2^{\sigma}}\leq\tau
p^{\sigma}\ln p-\tau(p-1)^{\sigma}\ln(p-1)\leq\tau
p^{\sigma-1}\ln(ep^{\sigma}),\quad p\geq 2.$

Then (2.4) follows after taking exponentials. ∎

######  Remark 2.3.

Note that $\widetilde{(M.2)^{\prime}}$ follows from the right-hand side of
(2.4). In particular,

$M^{\tau,\sigma}_{p}\leq e^{\tau p^{\sigma-1}}p^{\tau\sigma
p^{\sigma-1}}M^{\tau,\sigma}_{p-1}\leq
C^{p^{\sigma}}M^{\tau,\sigma}_{p-1},\quad p\in\mathbf{Z}_{+},$

for suitable $C>0$.

###  2.2. Associated function

In this subsection we recall the definition and some elementary properties of
$T_{\tau,\sigma,h}(k)$, $h>0$, the associated function to the sequence
$M^{\tau,\sigma}_{p}=p^{\tau p^{\sigma}}$ given by

$\displaystyle
T_{\tau,\sigma,h}(k)=\sup_{p\in\mathbf{N}}\ln_{+}\frac{h^{p^{\sigma}}k^{p}}{M_{p}^{\tau,\sigma}},\;\;\;k>0.$
(2.6)

We refer to [12] for more details on $T_{\tau,\sigma,h}(k)$. One of the aims
of this paper is to prove that $\omega(k)=T_{\tau,\sigma,h}(|k|)$ is
equivalent to a weight function, see Theorem 3.1 i).

######  Remark 2.4.

Consider $1<\sigma\leq 2$. Then by $\widetilde{(M.4)^{\prime}}$ and Example 20
from [1] we obtain

$T_{\tau,\sigma,h}(k)\leq
A\sup_{p\in\mathbf{N}}\ln_{+}\frac{k^{p}}{e^{p^{\sigma}}}+B\leq
A_{1}\ln_{+}^{\frac{\sigma}{\sigma-1}}k+B_{1}\,\quad k>0,$

for suitable $A_{1}>0$ and $B_{1}\in\mathbf{R}$ (depending on
$\tau,\sigma,h$). Hence we conclude that $\displaystyle T_{\tau,\sigma,h}(k)$
is dominated by a weight function (see (1.7)). However this fact does not
imply that $T_{\tau,\sigma,h}(|k|)$ is equivalent to a weight function. We
will provide additional arguments in the proof of Theorem 3.1.

Sharp estimates for $T_{\tau,\sigma,h}(k)$ are given in [12], whenre it is
proved that for some $A_{1},A_{2}>0$ and $B_{1},B_{2}\in\mathbf{R}$ (depending
on $\tau,\sigma,h$) the following estimates hold:

$A_{1}{W^{-\frac{1}{\sigma-1}}({{\mathfrak{R}}(h,k)})}\,{\ln_{+}}^{\frac{\sigma}{\sigma-1}}k+B_{1}\leq
T_{\tau,\sigma,h}(k)\\\ \leq
A_{2}{W^{-\frac{1}{\sigma-1}}({{\mathfrak{R}}(h,k)})}\,{\ln_{+}}^{\frac{\sigma}{\sigma-1}}k+B_{2},$
(2.7)

where

$\displaystyle{\mathfrak{R}}(h,k):=h^{-\frac{\sigma-1}{\tau}}e^{\frac{\sigma-1}{\sigma}}\frac{\sigma-1}{\tau\sigma}\ln(e+k),\quad
h,k>0,$

and $W$ is the principal branch of the Lambert function. Note that these
estimates hold for any choice of parameters $h>0$, $\tau>0$ and $\sigma>1$.

We write $T_{\tau,\sigma}(k):=T_{\tau,\sigma,1}(k)$ for the classical
associated function associated to $M^{\tau,\sigma}_{p}$ (see [8]). We end this
subsection with a simple result which will be used in the sequel.

######  Lemma 2.2.

Let $T_{\tau,\sigma,h}(k)$ be given by (2.6), and let
$T_{\tau,\sigma}(k):=T_{\tau,\sigma,1}(k)$. Then for any given $h>0$ and
$\tau_{2}>\tau>\tau_{1}>0$ there exists $A,B\in{\mathbf{R}}$ such that

$T_{\tau_{2},\sigma}(k)+A\leq T_{\tau,\sigma,h}(k)\leq
T_{\tau_{1},\sigma}(k)+B,\quad k>0.$

###### Proof.

Note that $\widetilde{(M.4)}$ implies

$C_{2}\frac{k^{p}}{M^{\tau_{2},\sigma}_{p}}\leq\frac{\,h^{p^{\sigma}}k^{p}}{M^{\tau,\sigma}_{p}}\leq
C_{1}\frac{k^{p}}{M^{\tau_{1},\sigma}_{p}},\quad k>0,\quad p\in\mathbf{N},$

and the conclusion follows after taking logarithms and the supremum with
respect to $p\in\mathbf{N}$. ∎

###  2.3. Extended Gevrey classes

In this subsection we recall the definition of extended Gevrey classes and
some of their basic properties. We consider general sequences which satisfy
conditions $(M.1)-\widetilde{(M.5)}$.

Let $U$ be an open set in $\mathbf{R}^{d}$ and $K\subset\subset U$ be a
regular compact set. We denote by
$\displaystyle{\mathcal{E}}_{\tau,\sigma,h}(K)$ the Banach space of functions
$\phi\in C^{\infty}(K)$ such that

$\|\phi\|_{{\mathcal{E}}_{\tau,\sigma,h}(K)}=\sup_{\alpha\in\mathbf{N}^{d}}\sup_{x\in
K}\frac{|\partial^{\alpha}\phi(x)|}{h^{|\alpha|^{\sigma}}M_{|\alpha|}^{\tau,\sigma}}<\infty.\,$
(2.8)

Note that

$\displaystyle{\mathcal{E}}_{\tau_{1},\sigma_{1},h_{1}}(K)\hookrightarrow{\mathcal{E}}_{\tau_{2},\sigma_{2},h_{2}}(K),\;\;\;0<h_{1}<h_{2},\;0<\tau_{1}<\tau_{2},\;1<\sigma_{1}<\sigma_{2},$

where $\hookrightarrow$ denotes the strict and dense inclusion. We define
spaces of Roumieu and Beurling type by introducing the following inductive and
projective limit topologies (respectively)

${\mathcal{E}}_{\\{\tau,\sigma\\}}(U)=\varprojlim_{K\subset\subset
U}\varinjlim_{h\to\infty}{\mathcal{E}}_{\tau,\sigma,h}(K),$ (2.9)

${\mathcal{E}}_{(\tau,\sigma)}(U)=\varprojlim_{K\subset\subset
U}\varprojlim_{h\to 0}{\mathcal{E}}_{\tau,\sigma,h}(K).$ (2.10)

We omit the brackets if we consider either $\\{\tau,\sigma\\}$ or
$(\tau,\sigma)$.

######  Remark 2.5.

The condition $(M.3)^{\prime}$ implies that ${\mathcal{E}}_{\tau,\sigma}(U)$
contains compactly supported functions. The construction of smooth compactly
supported functions which are not in Gevrey classes but which belong to
${\mathcal{E}}_{\tau,\sigma}(U)$ can be found in [11].

Extended Gevrey classes given by (2.9) and (2.10) are studied in [11, 12, 13].
For the convenience of the reader, we collect some of their basic properties
in the following Proposition. Recall, the Gevrey class of index $t>1$ is given
by ${\mathcal{G}}_{t}(U)=\mathcal{E}_{\\{t,1\\}}(U)$, see (2.9).

######  Proposition 2.1.

Let $U$ be an open set in $\mathbf{R}^{d}$. Let
${\mathcal{E}}_{\\{\tau,\sigma\\}}(U)$ and ${\mathcal{E}}_{(\tau,\sigma)}(U)$
be given by (2.9) and (2.10) respectively, and let $\varinjlim$ and
$\varprojlim$ denote the corresponding inductive and projective limits
respectively. Then the following is true:

  * $i)$

For $\sigma_{2}>\sigma_{1}>1$ we have

$\varinjlim_{t\to\infty}{\mathcal{G}}_{t}(U)\hookrightarrow\varprojlim_{\tau\to
0}{\mathcal{E}}_{\\{\tau,\sigma_{1}\\}}(U)=\varprojlim_{\tau\to
0}{\mathcal{E}}_{(\tau,\sigma_{1})}(U)\hookrightarrow\varinjlim_{\tau\to\infty}{\mathcal{E}}_{(\tau,\sigma_{1})}(U)\\\
=\varinjlim_{\tau\to\infty}{\mathcal{E}}_{\\{\tau,\sigma_{1}\\}}(U)\hookrightarrow\varprojlim_{\tau\to
0}{\mathcal{E}}_{\\{\tau,\sigma_{2}\\}}(U)\hookrightarrow C^{\infty}(U).$
(2.11)

  * $ii)$

${\mathcal{E}}_{\tau,\sigma}(U)$ are closed under the pontwise multiplication.

  * $iii)$

${\mathcal{E}}_{\tau,\sigma}(U)$ are closed under finite order derivation.

  * $iv)$

For $a_{\alpha}\in{\mathcal{E}}_{(\tau,\sigma)}(U)$ (resp.
$a_{\alpha}\in{\mathcal{E}}_{\\{\tau,\sigma\\}}(U)$) define

$P(x,\partial)=\sum_{|\alpha|=0}^{\infty}a_{\alpha}(x)\partial^{\alpha},$

such that for every $K\subset\subset U$ there exists $L>0$ and for every $h>0$
there exists $A>0$ (resp. for every $K\subset\subset U$ there exists $h>0$ and
for every $L>0$ there exists $A>0$) so that

$\sup_{x\in K}|\partial^{\beta}a_{\alpha}(x)|\leq
Ah^{|\beta|^{\sigma}}M^{\tau,\sigma}_{|\beta|}\frac{L^{|\alpha|^{\sigma}}}{M^{2^{\sigma-1}\tau,\sigma}_{|\alpha|}}.$

Then $\displaystyle
P(x,\partial)\,:\,{\mathcal{E}}_{\tau,\sigma}(U)\to{\mathcal{E}}_{2^{\sigma-1}\tau,\sigma}(U)$
is a continuous and linear mapping.

Let $h=1$ in (2.8). We introduce the following spaces

$\mathcal{E}_{\\{\sigma\\}}(U)=\varinjlim_{\tau\to\infty}\mathcal{E}_{\tau,\sigma}(U),\quad\mathcal{E}_{(\sigma)}(U)=\varprojlim_{\tau\to
0}\mathcal{E}_{\tau,\sigma}(U),$ (2.12)

$\mathcal{E}_{\\{\sigma\\}}^{R}(U)=\varprojlim_{K\subset\subset
U}\varinjlim_{\tau\to\infty}{\mathcal{E}}_{\tau,\sigma,1}(K),\quad\mathcal{E}_{(\sigma)}^{B}(U)=\varprojlim_{K\subset\subset
U}\varprojlim_{\tau\to 0}{\mathcal{E}}_{\tau,\sigma,1}(K).$ (2.13)

Note that Proposition 2.1 i), and the order of quantifiers in the definition
of spaces (2.12) and (2.13) implies that

$\varinjlim_{t\to\infty}{\mathcal{G}}_{t}(U)\hookrightarrow\mathcal{E}_{(\sigma)}(U)=\mathcal{E}_{(\sigma)}^{B}(U)\hookrightarrow\mathcal{E}_{\\{\sigma\\}}(U)\hookrightarrow\mathcal{E}_{\\{\sigma\\}}^{R}(U)\hookrightarrow
C^{\infty}(U).$ (2.14)

Notice that, unlike $\mathcal{E}_{\tau,\sigma}(U)$,
$\mathcal{E}_{\\{\sigma\\}}(U)$, $\mathcal{E}_{(\sigma)}(U)$, are classes of
ultradifferentiable functions. This follows from Proposition 2.1 iv).
Moreover, ultradifferentiability of $\mathcal{E}_{\\{\sigma\\}}^{R}(U)$
follows from the arguments given in [14].

##  3\. Main result

In this section we first give an estimate for
$T_{\tau,\sigma}(k)=T_{\tau,\sigma,1}(k)$ which is introduced in Subsection
2.2. Note that $T_{\tau,\sigma}(k)$ satisfies estimates (2.7) when $h=1$.
Therefore the following Proposition follows directly from [12, Theorem 2.1].
However, here we give an independent proof.

######  Proposition 3.1.

Let $T_{\tau,\sigma}(k)=T_{\tau,\sigma,1}(k)$ be given by (2.6) with $h=1$,
and let $W(t)$, $t>0$, denote the restriction of the principal branch of the
Lambert $W$ function. If
$\displaystyle\varphi_{\sigma}(t)=\frac{t^{\frac{\sigma}{\sigma-1}}}{W^{\frac{1}{\sigma-1}}(t)}$,
$t>0$, and $\varphi_{\sigma}(0)=0$, then we have

$B_{\tau,\sigma}\varphi_{\sigma}(\ln_{+}k)+\widetilde{B}_{\tau,\sigma}\leq
T_{\tau,\sigma}(k)\leq
A_{\tau,\sigma}\varphi_{\sigma}(\ln_{+}k)+\widetilde{A}_{\tau,\sigma},\quad
k>0,$

for suitable constants $A_{\tau,\sigma},B_{\tau,\sigma}>0$ and
$\widetilde{A}_{\tau,\sigma},\widetilde{B}_{\tau,\sigma}\in\mathbf{R}$.

###### Proof.

For $\lambda>0$ we let

$\displaystyle
m_{\tau,\sigma}(\lambda)=\\#\\{p\in\mathbf{Z}_{+}\,|\,m^{\tau,\sigma}_{p}\leq\lambda\\},$

and note that $m_{\tau,\sigma}(\lambda)=0$ for all $0<\lambda<1$. This is due
to the fact that $m_{1}=1$ and $m^{\tau,\sigma}_{p}$ is increasing.

Since $M^{\tau,\sigma}_{p}$ satisfies $(M.1)$ we can write (see [8, 3])

$T_{\tau,\sigma}(k)=\int_{0}^{k}\frac{m_{\tau,\sigma}(\lambda)}{\lambda}d\lambda=\int_{1}^{k}\frac{m_{\tau,\sigma}(\lambda)}{\lambda}d\lambda.$

In the sequel we estimate $m_{\tau,\sigma}(\lambda)$ when $\lambda\geq 1$.

Put

$m^{C}_{\tau,\sigma}(\lambda)=\\#\\{p\in\mathbf{Z}_{+}\,|\,C^{p^{\sigma-1}}p^{\tau
p^{\sigma-1}}\leq\lambda\\},\quad C>0.$

Then (2.4) implies that

$m^{C_{1}}_{(\tau\sigma),\sigma}(\lambda)\leq m_{\tau,\sigma}(\lambda)\leq
m^{C_{2}}_{(\tau\sigma)/2^{\sigma-1},\sigma}(\lambda),\quad\lambda\geq 1,$

where $C_{1}=e^{\tau}$ and $C_{2}=(e/2^{\sigma})^{\tau/2^{\sigma-1}}$. In
particular,

$\int_{1}^{k}\frac{m^{C_{1}}_{(\tau\sigma),\sigma}(\lambda)}{\lambda}d\lambda\leq
T_{\tau,\sigma}(k)\leq\int_{1}^{k}\frac{m^{C_{2}}_{(\tau\sigma)/2^{\sigma-1},\sigma}(\lambda)}{\lambda}d\lambda,\quad
k>0.$ (3.1)

Next we note that

$C^{p^{\sigma-1}}p^{\tau p^{\sigma-1}}\leq\lambda\iff
C^{\frac{\sigma-1}{\tau}}p^{\sigma-1}\ln(C^{\frac{\sigma-1}{\tau}}p^{\sigma-1})\leq
C^{\frac{\sigma-1}{\tau}}\frac{\sigma-1}{\tau}\ln\lambda\\\
\iff\ln(C^{\frac{\sigma-1}{\tau}}p^{\sigma-1})\leq
W(C^{\frac{\sigma-1}{\tau}}\frac{\sigma-1}{\tau}\ln\lambda)\iff\\\ p\leq
C^{-\frac{1}{\tau}}{e}^{\frac{1}{\sigma-1}W(C^{\frac{\sigma-1}{\tau}}\frac{\sigma-1}{\tau}\ln\lambda)},\quad
C>0,\,\,\lambda\geq 1,$ (3.2)

where for the second equivalence we used property (1.3) of the Lambert
function.

This calculation shows that

$m^{C}_{\tau,\sigma}(\lambda)=\Big{\lfloor}C^{-\frac{1}{\tau}}{e}^{\frac{1}{\sigma-1}W(C^{\frac{\sigma-1}{\tau}}\frac{\sigma-1}{\tau}\ln\lambda)}\Big{\rfloor},\quad\lambda\geq
1.$

and therefore

$\int_{1}^{k}\frac{m^{C}_{\tau,\sigma}(\lambda)}{\lambda}d\lambda\sim
C^{-\frac{1}{\tau}}\int_{1}^{k}\frac{{e}^{\frac{1}{\sigma-1}W(C^{\frac{\sigma-1}{\tau}}\frac{\sigma-1}{\tau}\ln\lambda)}}{\lambda}d\lambda,\,\quad
k\to\infty.$

It remains to compute

$I^{C}_{\tau,\sigma}(k):=C^{-\frac{1}{\tau}}\int_{1}^{k}\frac{{e}^{\frac{1}{\sigma-1}W(C^{\frac{\sigma-1}{\tau}}\frac{\sigma-1}{\tau}\ln\lambda)}}{\lambda}d\lambda,\quad
C>0.$

Set $C_{\tau,\sigma}=C^{\frac{\sigma-1}{\tau}}\frac{\sigma-1}{\tau}$. Note
that after the substitution $t=C_{\tau,\sigma}\ln\lambda$ we obtain

$I^{C}_{\tau,\sigma}(k)=C^{-\frac{\sigma}{\tau}}\frac{\tau}{\sigma-1}\int_{0}^{C_{\tau,\sigma}\ln
k}{e}^{\frac{1}{\sigma-1}W(t)}dt.$ (3.3)

Another change of variables $W(t)=s$ $(t=se^{s},dt=(s+1)e^{s}ds)$, and
integration by parts yields

$\int{e}^{\frac{1}{\sigma-1}W(t)}dt=\int{e}^{\frac{\sigma
s}{\sigma-1}}(s+1)ds={e}^{\frac{\sigma
s}{\sigma-1}}\frac{\sigma-1}{\sigma}\Big{(}s+\frac{1}{\sigma}\Big{)},$ (3.4)

where we use indefinite integral just for the notational convenience.

Now using $(W2)$ property of the Lambert function and (1.5), by (3.3) and
(3.4) we have

$I^{C}_{\tau,\sigma}(k)=\frac{\tau}{\sigma}C^{-\frac{\sigma}{\tau}}{e}^{\frac{\sigma
s}{\sigma-1}}\Big{(}s+\frac{1}{\sigma}\Big{)}\Big{|}_{0}^{W(t)}=\\\
\frac{\tau}{\sigma}C^{-\frac{\sigma}{\tau}}\Big{(}\frac{t}{W(t)}\Big{)}^{\frac{\sigma}{\sigma-1}}(W(t)+\frac{1}{\sigma})\Big{|}_{0}^{C_{\tau,\sigma}\ln
k}\asymp\tau^{-\frac{1}{\sigma-1}}\varphi_{\sigma}(\ln_{+}k)+\widetilde{C}_{\tau,\sigma},\
k>0,$ (3.5)

for some $\widetilde{C}_{\tau,\sigma}\in\mathbf{R}$, where the hidden
constants are depending only on $\sigma$.

More precisely, using (3.1) and (3.5) we conclude

$B_{\sigma}\tau^{-\frac{1}{\sigma-1}}\varphi_{\sigma}(\ln_{+}k)+\widetilde{B}_{\tau,\sigma}\leq
T_{\tau,\sigma}(k)\leq
A_{\sigma}\Big{(}\frac{\tau}{2^{\sigma-1}}\Big{)}^{-\frac{1}{\sigma-1}}\varphi_{\sigma}(\ln_{+}k)+\widetilde{A}_{\tau,\sigma},\,\,k>0,$
(3.6)

for suitable $A_{\sigma},B_{\sigma}>0$ and
$\widetilde{A}_{\tau,\sigma},\widetilde{B}_{\tau,\sigma}\in\mathbf{R}$. This
completes the proof. ∎

Following [6, 14] we introduce the Banach space ${\mathcal{B}}_{H,\sigma}(K)$,
$K\subset\subset U$, with the norm

$\|\phi\|_{{\mathcal{B}}_{H,\sigma}(K)}=\sup_{\alpha\in\mathbf{N}^{d}}\sup_{x\in
K}{|\partial^{\alpha}\phi(x)|}e^{-\frac{1}{H}\varphi_{\sigma}^{*}(Hp)},\quad
H>0,$

where $\varphi^{*}_{\sigma}$ is Youngs conjugate of the function
$\displaystyle\varphi_{\sigma}$ introduced in Proposition 3.1.

We introduce the corresponding Roumieu and Beurling classes as follows:

${\mathcal{B}}_{\\{\sigma\\}}(U)=\varprojlim_{K\subset\subset
U}\varinjlim_{H\to\infty}{\mathcal{B}}_{H,\sigma}(U),\quad{\mathcal{B}}_{(\sigma)}(U)=\varprojlim_{K\subset\subset
U}\varprojlim_{H\to 0}{\mathcal{B}}_{H,\sigma}(U),$

respectively.

Now we can formulate the main result of the paper.

######  Theorem 3.1.

Fix $\sigma>1$ and let $\displaystyle\varphi_{\sigma}(t)$ be as in Proposition
3.1. Moreover, let $T_{\tau,\sigma,h}$ be given by (2.6). Then the following
is true.

  * i)

The function $\omega(k)=\varphi_{\sigma}(\ln_{+}|k|)$ is equivalent to a
weight function. Moreover, for every $h>0$ and $\tau>0$, the function
$\omega(k)=T_{\tau,\sigma,h}(|k|)$ is equivalent to a weight function.

  * ii)

The weight matrices ${\mathcal{M}}_{\sigma}=\\{p^{\tau
p^{\sigma}}\\}_{\tau>0}$ and ${\mathcal{N}}_{\sigma}=\\{\displaystyle
e^{\frac{1}{H}\varphi_{\sigma}^{*}(Hp)}\\}_{H>0}$ are equivalent. In
particular,

${\mathcal{B}}_{(\sigma)}(U)={\mathcal{E}}_{(\sigma)}^{B}(U),\quad{\mathcal{B}}_{\\{\sigma\\}}(U)={\mathcal{E}}_{\\{\sigma\\}}^{R}(U)$
(3.7)

where $\displaystyle{\mathcal{E}}_{(\sigma)}^{B}(U)$ and
${\mathcal{E}}_{\\{\sigma\\}}^{R}(U)$ are given in (2.13).

###### Proof.

i) By Proposition 3.1 it follows that functions
$T_{\tau,\sigma}(|k|)\asymp\varphi_{\sigma}(\ln_{+}|k|)$. Thus, it is
sufficient to show that $\displaystyle T_{\tau,\sigma}(|k|)$ is a weight
functions (see Remark 2.2).

Since $T_{\tau,\sigma}$ is the function associated to $M^{\tau,\sigma}_{p}$,
by [1, Lemma 12] it is sufficient to show that $m^{\tau,\sigma}_{p}$ given by
2.3 satisfies (1.1), i.e., that there exists $Q\in\mathbf{N}$ such that

$\liminf_{p\to\infty}\frac{m^{\tau,\sigma}_{Qp}}{m^{\tau,\sigma}_{p}}>1.$
(3.8)

Note that Lemma 2.1 implies

$\frac{m^{\tau,\sigma}_{3p}}{m^{\tau,\sigma}_{p}}\geq\frac{\Big{(}\frac{e}{2^{\sigma}}\Big{)}^{\frac{\tau(3p)^{\sigma-1}}{2^{\sigma-1}}}(3p)^{\frac{\tau\sigma(3p)^{\sigma-1}}{2^{\sigma-1}}}}{e^{\tau
p^{\sigma-1}}p^{\tau\sigma p^{\sigma-1}}}\\\
=\Big{(}\frac{3}{2}\Big{)}^{\tau\sigma(\frac{3p}{2})^{\sigma-1}}\exp\Big{\\{}\tau\Big{(}({3}/{2})^{\sigma-1}-1\Big{)}p^{\sigma-1}\Big{\\}}p^{\tau\sigma\Big{(}({3}/{2})^{\sigma-1}-1\Big{)}p^{\sigma-1}}\to\infty,\,\,p\to\infty,$
(3.9)

and (3.8) follows when $Q=3$. In addition, Lemma 2.2 together with Proposition
3.1 implies that $T_{\tau,\sigma,h}(|k|)\asymp\varphi_{\sigma}(\ln_{+}|k|)$.

ii) Let $A=A_{\sigma}>0$ and $B=B_{\sigma}>0$ be as in (3.6). For $\tau>0$ set

$H_{1}=B^{-1}\tau^{\frac{1}{\sigma-1}},\quad
H_{2}=A^{-1}(\tau/2^{\sigma-1})^{\frac{1}{\sigma-1}},$

and note that (3.6) implies

$C_{2}\exp\\{-\frac{1}{H_{2}}\varphi_{\sigma}(\ln_{+}k)\\}\leq\inf_{p\in\mathbf{N}}\frac{p^{\tau
p^{\sigma}}}{k^{p}}\leq
C_{1}\exp\\{-\frac{1}{H_{1}}\varphi_{\sigma}(\ln_{+}k)\\},\quad k>0.$

for suitable constants $C_{1},C_{2}>0$.

In particular, we have

$C_{2}\exp\\{\frac{1}{H_{2}}(H_{2}\,p\ln_{+}k-\varphi_{\sigma}(\ln_{+}k)\\}\leq
p^{\tau p^{\sigma}}\leq
C_{1}\exp\\{\frac{1}{H_{1}}(H_{1}\,p\ln_{+}k-\varphi_{\sigma}(\ln_{+}k)\\},$
(3.10)

for $p\in\mathbf{N}$ and $k>0$. Putting $t=\ln_{+}k$ and taking the supremum
with respect to $t\geq 0$, (3.10) implies

$C_{2}\exp\\{\frac{1}{H_{2}}\varphi_{\sigma}^{*}(H_{2}\,p)\\}\leq p^{\tau
p^{\sigma}}\leq
C_{1}\exp\\{\frac{1}{H_{1}}\varphi_{\sigma}^{*}(H_{1}\,p)\\},\quad
p\in\mathbf{N}.$ (3.11)

Therefore, matrices ${\mathcal{M}}_{\sigma}$ and ${\mathcal{N}}_{\sigma}$ are
equivalent.

It remains to prove (3.7). We give the proof for the Roumieu case
$\displaystyle{\mathcal{B}}_{\\{\sigma\\}}(U)={\mathcal{E}}_{\\{\sigma\\}}^{R}(U)$,
and omit the proof for the Beurling case, since it uses similar arguments.

Let $\phi\in{\mathcal{B}}_{{\sigma}}(U)$. Then for arbitrary $K\subset\subset
U$ there exists $H>0$ such that
$\|\phi\|_{{\mathcal{B}}_{H,\sigma}(K)}<\infty$. Putting
$\tau=(2AH)^{\sigma-1}$, (3.11) implies

$\|\phi\|_{{\mathcal{E}}_{\tau,\sigma}(K)}\leq
C^{\prime}\|\phi\|_{{\mathcal{B}}_{H,\sigma}(K)},$

for some $C^{\prime}>0$.

Conversely, if $\phi\in{\mathcal{E}}_{\\{\sigma\\}}^{R}(U)$ then for arbitrary
$K\subset\subset U$ there exists $\tau>0$ such that
$\displaystyle\|\phi\|_{{\mathcal{E}}_{\tau,\sigma}(K)}<\infty$. Choosing
$H=B^{-1}\tau^{\frac{1}{\sigma-1}}$, again by (3.11) we have

$\|\phi\|_{{\mathcal{B}}_{H,\sigma}(K)}\leq
C^{\prime\prime}\|\phi\|_{{\mathcal{E}}_{\tau,\sigma}(K)},$

for suitable $C^{\prime\prime}>0$. This completes the proof. ∎

######  Remark 3.1.

Note that $M^{\tau,\sigma}_{p}=p^{\tau p^{\sigma}}$ is not a weight sequence
in the sense of [1], since it does not satisfy $(M.2)^{\prime}$. Instead we
use $\widetilde{(M.2)^{\prime}}$ in a stronger form (see Remark 2.3).

Moreover, in the proof of Theorem 3.1 we use the part of [1, Lemma 12] for
which it is sufficient to assume $(M.1)$ and

$(M.0)\quad\quad(\exists C>0)\quad M^{\tau,\sigma}_{p}\geq Cp^{p},\quad
p\in\mathbf{N},$

which is obviously true (see $\widetilde{(M.4)^{\prime}}$).

We conclude the paper with the following Corollary which is an immediate
consequence of Theorem 3.1.

######  Corollary 3.1.

For $s>1$ function
$\displaystyle\omega(t)=\frac{\ln_{+}^{s}|t|}{\ln^{s-1}(\ln(e+|t|))}$,
$t\not=0$, $\omega(0)=0$, is equivalent to a weight function.

##  4\. Discussion

The equivalence of theories of ultradifferentiable functions given by
Komatsu’s or the Braun-Meise-Taylor approach are well established in the most
classical situations. Recent approach based on weighted matrices seems to
offer a very general construction, see [14, 6]. In parallel, it is
demonstrated in [11, 12, 13] that the two-parameter sequences of the form
$M_{p}=p^{\tau p^{\sigma}}$, $\tau>0$, $\sigma>1$, provide a useful extension
of the Gevrey type spaces.

In this paper we show that the projective limits of extended Gevrey classes
can be viewed as a part of the construction based on the weighted matrices.
The same conclusion holds when the inductive limits of extended Gevrey classes
are replaced by certain slightly larger spaces. At the same time, extended
Gevrey classes $\mathcal{E}_{\tau,\sigma}(U)$ for fixed $\tau>0$ and
$\sigma>1$, can not be characterized by weight matrices used in [14, 6] due to
the particular role played by the parameter $\sigma$.

While finishing the paper the authors learned about the work in progress ”A
comparison of two ways to generalize ultradifferentiable classes defined by
weight sequences” by J. Jiménez-Garrido, D. N. Nenning, and G. Schindl, which
is devoted to similar topic considered from a different point of view. We
thank the authors for their fruitful comments on the first version of this
paper.

## Acknowledgement

This research was funded by Ministry of Education, Science and Technological
Development, Republic of Serbia Projects no. 451-03-68/2022-14/200125 and
451-03-68/2022-14/200156.

## References

  * [1] Bonet, J., Meise, R., Melikhov. S., A comparison of two different ways to define classes of ultradifferentiable functions, Bull. Belg. Math. Soc. Simon Stevin 14 (3) 425 - 444, 2007.
  * [2] Braun, R.W., Meise, R., Taylor, B. A., Ultra-differentiable functions and Fourier analysis, Results Math. 17 (3-4), (1990), 206–237.
  * [3] Carmichael, R., Kaminski, A., Pilipović, S., Notes on Boundary Values in Ultradistribution Spaces, Lecture Notes Series of Seul University, 49, 1999.
  * [4] Cicognani M., Lorentz, D., Strictly hyperbolic equations with coefficients low-regular in time and smooth in space, J. Pseudo-Differ. Oper. Appl., 9, no. 3, 643–675, (2018).
  * [5] Corless, R.M. , Gonnet, G.H. , Hare, D.E.G. ,Jeffrey, D.J. ,Knuth, D.E. ,On the Lambert W function, Adv. Comput. Math. 5, (1996), 329–359.
  * [6] Fürdös, S., Nenning, D.N, Rainer, A, Schindl, Gerhard Almost analytic extensions of ultradifferentiable functions with applications to microlocal analysis. J. Math. Anal. Appl. 481 (2020), no. 1, 123451, 51 pp.
  * [7] Jiménez-Garrido, J., Lastra, A., Sanz, J.,  Extension operators for some ultraholomorphic classes defined by sequnces of rapid growth, https://doi.org/10.48550/arXiv.2204.01316 (2022).
  * [8] Komatsu, H., Ultradistributions, I: Structure theorems and a characterization. J. Fac. Sci. Univ. Tokyo, Sect. IA Math., 20 1 (1973), 25–105.
  * [9] Komatsu, H., An introduction to the theory of generalized functions, Lecture notes, Department of Mathematics Science University of Tokyo, 1999.
  * [10] Meise, R., Taylor, B.A. _Whitney’s extension theorem for ultradifferentiable functions of Beurling type_ , Ark. Mat. 26, no. 2, (1988), 265–287.
  * [11] Pilipović, S. ,Teofanov, N. , Tomić, F, On a class of ultradifferentiable functions, Novi Sad Journal of Mathematics, 45 (1), (2015), 125–142.
  * [12] Pilipović, S. ,Teofanov, N. , Tomić, F, A Paley–Wiener theorem in extended Gevrey regularity, J. Pseudo-Differ. Oper. Appl., 11 , no. 2, 593–612., 2020.
  * [13] Pilipović, S. ,Teofanov, N. , Tomić, F, Boundary values in ultradistribution spaces related to extended Gevrey regularity , Mathematics , 9(1), 7, 2021.
  * [14] Rainer, A., Schindl, G. Composition in ultradifferentiable classes, Studia Math., 224 2, 97 – 131 (2014)
  * [15] Rodino, L. , Linear Partial Differential Operators in Gevrey Spaces, World Scientific, 1993.


# Multimodal Emotion Recognition Model using Physiological Signals

Yuxuan Zhao1,2 , Xinyan Cao1, Jinlong Lin1, Dunshan Yu1, Xixin Cao1,∗
1School of Software and Microelectronics, Peking University, China
2Institute of Automation, Chinese Academy of Sciences, China
To whom correspondence should be addressed; E-mail<EMAIL_ADDRESS>
###### Abstract

As an important field of research in Human-Machine Interactions, emotion
recognition based on physiological signals has become research hotspots.
Motivated by the outstanding performance of deep learning approaches in
recognition tasks, we proposed a Multimodal Emotion Recognition Model that
consists of a 3D convolutional neural network model, a 1D convolutional neural
network model and a biologically inspired multimodal fusion model which
integrates multimodal information on the decision level for emotion
recognition. We use this model to classify four emotional regions from the
arousal valence plane, i.e., low arousal and low valence (LALV), high arousal
and low valence (HALV), low arousal and high valence (LAHV) and high arousal
and high valence (HAHV) in the DEAP and AMIGOS dataset. The 3D CNN model and
1D CNN model are used for emotion recognition based on electroencephalogram
(EEG) signals and peripheral physiological signals respectively, and get the
accuracy of 93.53% and 95.86% with the original EEG signals in these two
datasets. Compared with the single-modal recognition, the multimodal fusion
model improves the accuracy of emotion recognition by 5%~
25%, and the fusion result of EEG signals (decomposed into four frequency
bands) and peripheral physiological signals get the accuracy of 95.77%, 97.27%
and 91.07%, 99.74% in these two datasets respectively. Integrated EEG signals
and peripheral physiological signals, this model could reach the highest
accuracy about 99% in both datasets which shows that our proposed method
demonstrates certain advantages in solving the emotion recognition tasks.

##  1 Introduction

Emotion recognition plays a crucial role in human-machine interaction and
health care. The recognition method based on physiological signals has become
a research hotspot because the signals could represent the inner emotional
states and cannot be controlled subjectively compared with other signals as
facial expressions or speech.

The traditional machine learning approaches use well-designed classifiers with
hand-crafted features have been studied for many years. The most common
features[17, 39] contains Time Domain Features: Event Related Potentials
(ERP)[8], Statistics of Signal[28] (Power, Mean, Standard deviation, 1st
difference, 2nd difference et al.), Higher Order Crossings (HOC)[28] et al;
Frequency Domain Features: Power Spectra Density (PSD)[39], Higher Order
Spectra (HOS)[15] et al; Time Frequency Domain Features:
Hilbert-Huang Spectrum (HHS)[12], Magnitude Squared Coherence Estimate
(MSCE)[18] et al. A traditional approach which could achieve a high emotion
recognition accuracy mostly depended on the well-designed hand-crafted
features. So the new and effective feature extraction methods based on phase
space reconstruction [38, 37] and flexible analytic wavelet transform (FAWT)
[11] make a good performance in the emotion recognition task. Soroush [37]
made the EEG signals reconstructed in phase space and then in angle space,
then extracted features from angle variability and length variability, and
used Dempster-Shafer theory for emotion recognition, final used ten-fold cross
validation to evaluate their model. To our best knowledge, their method
achieved the best performance in traditional machine learning approaches -
classification accuracy was about 90% on average classified into four classes
in DEAP dataset.

The well-designed hand-crafted features based on comprehensive domain
knowledge which may be an obstacle for non-domain experts, on the other hand,
the domain knowledge may limit the performance of the model. Deep learning
approaches make a great success in pattern recognition domain, motivated by
the outstanding performance of deep learning many researchers used it in
emotion recognition tasks, such as the pre-trained deep learning model [23,
10], the 2D Convolutional Neural Network Model [13, 22], the 3D Convolutional
Neural Network Model [27, 34], the combined model of 2D CNN and LSTM [35] and
other models base on deep learning methods [2, 29, 22, 14]. Deep learning
approaches make a good performance in the emotion recognition task based on
physiological signals. A 3D CNN Model used to emotion recognition based on EEG
signals which we proposed in a previous work gets the accuracy of 93.53% in
four classification task of DEAP dataset.

Multimodal information be collected to using for emotion recognition task,
such as the electrocardiogram (ECG) signals, electromyogram (EMG) signals,
electrooculogram (EOG) signals, galvanic skin resistance (GSR) signals,
respiration amplitude signals, blood volume signals and skin temperature in
peripheral physiological signals and electroencephalogram (EEG) signals.
Integrate multimodal information to improve the model performance attracts
many researchers, most of the multimodal information fusion techniques
integrate different information on feature level [32, 22, 14] and intermediate
level [30, 36], less on decision level [3]. Compared with the feature level
fusion and intermediate level, the decision level fusion more easily because
the decisions resulting from multiple modalities usually have the same form of
data and every modality can utilize its best suitable classifier or model to
learn its features [25].

Here we proposed a 3D and 1D convolutional neural network model for single
modal emotion recognition based on EEG signals and various peripheral
physiological signals respectively and could achieve a high accuracy
especially used the EEG signals. Furthermore, we proposed a biologically
inspired multimodal fusion model that integrates multimodal information on the
decision level to improve the performance of the model.

The remainder of this paper is organized as follows: Section 2 provides a
review of research about the deep learning approaches and the multimodal
information fusion techniques used in emotion recognition tasks. Section 3
describes the method of data pre-processing, and the 3D and 1D convolutional
neural network model used for single-modal, and the biologically inspired
multimodal fusion model. Section 4 presents the description of the DEAP and
AMIGOS dataset which used in the experiment and the result to verify the
performance of the proposed model. In Section 5, we conclude our work.

##  2 Related work

Deep learning approaches make a great success in the emotion recognition task,
in particular, the deep learning approaches using EEG signals attracted a lot
of researchers. Lin [23] and Liu [10] converted the EEG signals into 2D image
format and used the pre-trained deep learning model AlexNet and ResNets to
extract depth level features respectively, then combine the hand-crafted
features which extract from peripheral physiological signals in Lin and EEG
signals in Liu for classification, and achieve the accuracy of 87.3%/85.5%
(two-category emotion based on the threshold of arousal and valence
respectively) in Lin, and 89.06%/90.39%( two-category emotion), 86.05% (four-
category emotion based on the threshold of arousal and valence) in Liu. Mei
[13] and Kwon [22] used 2D Convolutional Neural Network Model for feature
extraction and classification and achieve the accuracy of 73.1% and 73.43% in
four-category emotion recognition task respectively. These 2D conventional
methods ignore the spatial characteristics of EEG signals, so the spatial-
temporal features extraction methods had been proposed. Salama [27] proposed a
3D Convolutional Neural Network Model for spatial-temporal features extraction
and classification in EEG signals and achieve the accuracy of 88.49% for
arousal, 87.44% for valence in two-category emotion recognition task
respectively. Wang [34] converted the EEG channels into 2D electrode
topological plate which could include topological position information and
used the 3D CNN Model for spatial-temporal features extraction and
classification and found that compared with the 2D CNN Model used unconverted
data, the 3D CNN Model made a better performance with the accuracy of 73.3%
and 72.1% in two-category emotion recognition task. Yang [35] implemented the
2D CNN module and LSTM module extract spatial and temporal features
respectively and combined the features for classification, and achieved a high
accuracy rate of 91.03% and 90.8% in two-category emotion recognition task.
The research of emotion recognition based on peripheral physiological signals
is relatively few, especially using deep learning approaches. Machot [2]
proposed a 2D CNN architecture for emotion recognition in four-category
emotion based on Electrodermal Activity (EDA) signals, and achieve the
accuracy of 85% in DEAP dataset. Granados [29] proposed a 1D Convolutional
Neural Network Model model using electrocardiogram and galvanic skin response
signals in four-category emotion recognition tasks in AMIGOS dataset, and
achieve an accuracy of 65.25%.

Multimodal information fusion techniques used to improve the performance of
the system by integrating different information on feature level [32, 22, 14],
intermediate level [30, 36], and decision level [3]. The feature fusion is
most widely used compared with the others in emotion recognition tasks based
on physiological signals. Verma [32] proposed a multimodal fusion approach at
feature level, they extracted 25 features from EEG and peripheral signals by
Discrete Wavelet Transform, and they got the average accuracy of 81.45% by
Support Vector Machine (SVM) classifier for thirteen emotions classification
in DEAP dataset. Shu [30] proposed a fusion method to model the high-order
dependencies among multiple physiological signals by Restricted Boltzmann
Machine (RBM), the new feature representation generated from the features of
EEG signals and peripheral physiological signals by RBM, then they used
Support Vector Machine (SVM) classifier to emotion recognition and got the
accuracy of 64.6% and 60.7% in arousal and valence respectively in DEAP
dataset. Yin [36] proposed a multiple-fusion-layer based ensemble classifier
of stacked auto-encoder (MESAE) for emotion recognition, they extracted 425
salient physiological features from EEG signals and peripheral physiological
signals and the features split into non-overlapped physiological feature
subsets, then get the abstraction fusion features by the SAE, and used
Bayesian model to classification, the model got the accuracy of 84.18% and
83.04% in arousal and valence respectively in DEAP dataset. Kwon [22] proposed
a preprocessing method of EEG signals which using a wavelet transform while
considering time and frequency simultaneously, then extracted the EEG feature
convolution neural networks (CNN) model and combined them with the features
extracted from galvanic skin response (GSR) signals, and got the accuracy of
73.43% in four class classification task of DEAP Dataset. Bagherzadeh [3]
extracted spectral, time and non-linear features from peripheral and EEG
signals, then used multiple stacked autoencoders in a parallel form (PSAE) to
primary classification, the final decision about classification was performed
using the majority voting method, and they got the accuracy of 93.6% in four
class classification task of DEAP Dataset. Hassan [14] applied unsupervised
deep belief network (DBN) for depth level feature extraction from partial
peripheral signals and combined them in a feature fusion vector, then used the
Fine Gaussian Support Vector Machine (FGSVM) to classification, they got the
accuracy of 84.44% in five class classification task of DEAP Dataset.

##  3 Methods

Here we propose a Multimodal Emotion Recognition Model that consists of a 3D
convolutional neural network model, a 1D convolutional neural network model
and a biologically inspired multimodal fusion model on decision level for
multimodal emotion recognition.

###  3.1 Pre-processing

A pre-processing method with baseline signals - the signals be recorded when
the participant under no stimulus - which first elaborated by Yang [35] is an
effective way to improve recognition accuracy. They reported that the pre-
processing method can increase recognition accuracy by 32% approximately in
the emotion recognition task. The pre-processing method contains: extract the
baseline signals from all channels C and cut it in N segments with fixed
length L, get N segments C x L matrixes; calculate the mean value of the
baseline signals with segmented data, get the baseline signals mean value M, a
C x L matrixes; cut the EEG and Physiological signals which without baseline
signals with length L and minus the baseline signals mean value M, get the
preprocessed signals.

The raw EEG signals in the dataset lost the topological position information
of the electrodes. To solve this problem, the electrodes used in the dataset
are relocated to the 2D electrode topological structure based on the 10-20
system positioning. For each time sample point, the EEG signals are mapped
into a 9x9 matrixes. The unused electrodes are filled with zero. Z-score
normalization is used in each transformation.

###  3.2 3D Convolutional Neural Network Model

The 3D convolutional neural network model is used for emotion recognition
based on EEG signals. The architecture of the 3D convolutional neural network
model contains two convolution layers, each followed by a max-pooling layer,
and a fully-connected layer. A detailed illustration of the architecture is
shown in Fig.1. The input size is 9x9x128, the 9x9 is the 2D electrode
topological structure and the 128 is the number of the consecutive time sample
point processed at once. The kernel size of the convolution layer is 3x3x4,
which means the spatial-temporal features are generated based on a local
topology of 3x3 and a time period of 4-time sample points. To prevent missing
information of input data, the zero-padding be used in each convolutional
layers. The RELU activation function is used after the convolution operation.
The pooling size of a max-pooling layer is 1x1x2 which used to reduce the data
size in the temporal dimension and improve the robustness of extracted
features. The numbers of feature maps in the first and second convolutional
layers are 32 and 64 respectively. Before passing the 64 resulting feature
maps to the fully-connected layer, the output feature maps are reshaped in a
vector. The fully-connected layer maps the feature maps into a final feature
vector of 1024. And a dropout regularization after fully connected layers used
to avoid overfitting. The N in the output layer means the numbers of the label
in the task.

Figure 1: 3D Convolutional Neural Network Model

###  3.3 1D Convolutional Neural Network Model

The 1D convolutional neural network model is used for emotion recognition
based on physiological signals. The architecture of the 1D convolutional
neural network model contains two convolution layers, each followed by a max-
pooling layer, and a fully-connected layer. A detailed illustration of the
architecture is shown in Fig.2. The input size is 128x1 which represents a 128
consecutive time sample point of physiological signals in each channel. The
kernel size of the convolution layer is 3x1. To prevent missing information of
input data, the zero-padding be used in each convolutional layers. The RELU
activation function is used after the convolution operation. The pooling size
of a max-pooling layer is 2x1 which used to reduce the data size in the
temporal dimension and improve the robustness of extracted features. The
numbers of feature maps in the first and second convolutional layers are 16
and 32 respectively. Before passing the 32 resulting feature maps to the
fully-connected layer, the output feature maps are reshaped in a vector. The
fully-connected layer maps the feature maps into a final feature vector of
256. And a dropout regularization after fully connected layers used to avoid
overfitting. The N in the output layer means the numbers of the label in the
task.

Figure 2: 1D Convolutional Neural Network Model

###  3.4 Multimodal Fusion Model

The multisensory integration has been widely studied in neuroscience and
psychophysical [6]. A Bayes-optimal cue integration model been proposed and
has been proved its successful in visual and haptic information integration
[5], visual and proprioceptive information integration [31], visual and
vestibular information integration [7, 9], visual and auditory information
integration [19, 4], stereo and texture or texture and motion information
integration in vision research [20, 16]. This model estimates the result by
weighting the cues in proportion to their relative reliability which
proportional to the inverse variance. Take a spatial localization estimate by
visual and auditory information integration as an example as shown in Fig 3,
this model could describe as follows:

Figure 3: Bayes-optimal combination of multiple sensory cues.

The direction of an event is s, the estimate spatial location by visual
($x_{v}$) and auditory ($x_{a}$) will inconsistent with the true location
because of the noise in transmission and processing. According to Bayes’
theorem and the conditionally independent sources of visual and auditory, is
can be described as Equation 1.

$p(s|x_{v},x_{a})\propto p(x_{v}|s)*p(x_{a}|s)*p(s)$ (1)

If assumes that the p(s) is uniform prior distribution and the additional
simplifying assumption of Gaussian likelihood function,the average estimate
derived from an optimal Bayesian integrator is a weighted average of the
average estimates that would be derived from each cue alone, it could be
described as Equation 2.

$s^{\ast}=\omega_{v}\times x_{v}+\omega_{a}\times x_{a}$ (2)

where

$\omega_{v}=\frac{\displaystyle
1/\sigma_{v}^{2}}{1/\sigma_{v}^{2}+1/\sigma_{a}^{2}}\;and\;\omega_{a}=\frac{\displaystyle
1/\sigma_{a}^{2}}{1/\sigma_{v}^{2}+1/\sigma_{a}^{2}}$ (3)

It means that the greater a cue’s reliability, the more it contributes to the
final estimate.

In the research of affect, Russell’s valence-arousal scale [26] has been
widely used quantitatively describe emotions. After watching a video that
could induce the participant’s emotion, the participant rated a self-
assessment of arousal and valence on a scale from 1 to 9. In the emotion
recognition task, the researchers often divided the dataset into various
classes based on the threshold of arousal and valence. So the class labels
could mapping in the two-dimensional space by calculated the mean value of the
corresponding class labels on Arousal-Valence data. In this way, we can
calculate the Euclidean distance between various class labels, then using the
standard normal distribution to calculate the classification score reliability
of other labels in a certain label based on the Euclidean distance, as shown
in Equation 4.

$f(d_{ij})=\frac{1}{\sqrt{2\pi}}e^{-\frac{d_{ij}^{2}}{2}}$ (4)

The $f(d_{ij})$ is the classification score reliability between label $i$ and
label $j$. For example, the emotion dataset is divided into 4 classes based on
the threshold of arousal and valence: LALV (e.g. sad), HALV (e.g. stressed),
LAHV (e.g. relaxed), HAHV (e.g. happy). For a certain label HAHV, the distance
between HAHV and LALV is the farthest, so the classification score reliability
of LALV is lowest, similarly, the classification score reliability of HAHV is
lowest for a certain label LALV.

In each channel, we calculate the final classification score through
classification probability from CNN model and classification score reliability
as shown in Equation 5, the $NL$ is the number of labels, and the $PR_{c,i}$
represents the classification probability of label $i$ in channel $c$, the
$GauPR_{c,j}$ is the final classification score of label $j$ in channel $c$
with classification score reliability. Considering the difference between the
biological system and the machine systems, the classification probability from
the CNN model with Softmax Regression and the sum of all the classification
probability is 1, here we use the dispersion of final classification score to
represent the corresponding channel reliability, and the dispersion calculate
by the standard deviation as shown in Equation 6, the $S_{c}$ represents the
channel reliability of channel $c$. The more average the final classification
scores are, the smaller of the dispersion is, the lower the corresponding
channel reliability is. If one of the final classification scores in a channel
is high, the corresponding channel reliability is high because of the greater
dispersion.

$GauPR_{c,j}=\sum_{i=1}^{NL}PR_{c,i}\times f(d_{ij})$ (5)

$S_{c}=\sqrt{\frac{1}{NL-1}\sum_{j=1}^{NL}(GauPR_{c,j}-\overline{GauPR_{c}})^{2}}$
(6)

where

$\overline{GauPR_{c}}=\frac{1}{NL}\sum_{j=1}^{NL}GauPR_{c,j}$ (7)

Then we calculate the fusion result of label $j$ as shown in Equation 8, the
$NC$ represents the number of the channels. Then we use $argmax$ to select the
final result.

$F_{j}=\sum_{c=1}^{NC}GauPR_{c,j}\times S_{c}$ (8)

$R_{label}=argmax(F_{j})$ (9)

##  4 Result

We test this method in public database DEAP[21] and AMIGOS[24].

The proposed model is implemented by using the Tensorflow framework [1] and
deployed on NVIDIA Tesla K40c. The learning rate is set to 1E-3 with Adam
Optimizer, and the keep probability of dropout operation is 0.5. The batch
size for training and testing is set to 240. We use 10-fold cross-validation
to evaluate the performance of our model. The average accuracy of the 10-fold
validation processes is taken as the final result.

###  4.1 DEAP Dataset

The DEAP [21] is an open dataset for researchers to validate their model. This
dataset contains 32 channels EEG signals and 8 channels peripheral
physiological signals which be collected when 32 participants watched 40
videos each with one- minute duration. The EEG channels contain Fp1, AF3, F3,
F7, FC5, FC1, C3, T7, CP5, CP1, P3, P7, PO3, O1, Oz, Pz, Fp2, AF4, Fz, F4, F8,
FC6, FC2, Cz, C4, T8, CP6, CP2, P4, P8, PO4 and O2. The peripheral
physiological channels contain hEOG, vEOG, zEMG, tEMG, GSR, respiration
amplitude, blood volume, and skin temperature. Each trial contains 63s signals
and the first 3s are the baseline signals. The baseline signals are recorded
when the participant under no stimulus. After watching a minute video, the
participants rated a self-assessment of arousal, valence, liking, and
dominance on a scale from 1 to 9. A preprocessed version had been provided:
The data was down-sampled from 512Hz to 128Hz, and a bandpass frequency filter
from 4.0-45.0Hz was applied.

In the process of data pre-processing for one trial signals (40x8064), the
baseline signals (40x384) have been cut in 3 segments (3 40x128), and
calculate the mean value of the baseline signals (1 40x128). And the data
without baseline signals cut in 60 segments (60 40x128) then minus the
baseline signals mean value, get the preprocessed signals (40x7680). For each
time sample point, the 32 channels EEG signals are mapped into a 9x9 matrixes
(as shown in Fig.4), get the 2D electrode topological structure (7680 9x9)
with Z-score normalization. Final, the signals are cut into 60 segments with
1s length (60 9x9x128), and the 1s length was reported as the most suitable
time window length in [33]. The final data size of EEG signals after
processing is 76800 9x9x128. The 8 channels physiological signals cut into 60
segments with 1s length in each channel, and the final data size of
physiological signals in each channel after processing is 76800 128x1.

(a) 10-20 System

(b) 2D Electrode Topological Structure

Figure 4: (a) The international 10-20 system describes the location of scalp
electrodes, and the red nodes show the 32 electrodes used in DEAP dataset. (b)
The 32 channels EEG signals are mapped into a 9x9 matrixes.

The dataset could be segmented in four classes low arousal low valence (LALV),
high arousal low valence (HALV), low arousal high valence (LAHV), high arousal
high valence (HAHV) based on the arousal and valence value with the threshold
of 5 respectively, and the corresponding instance numbers are shown in Table
1.

Table 1: Corresponding instance numbers in DEAP Label | LALV | HALV | LAHV | HAHV
---|---|---|---|---
Arousal |  $\leq$5 | >5 |  $\leq$5 | >5
Valence |  $\leq$5 |  $\leq$5 | >5 | >5
Instances | 16440 | 17880 | 16140 | 26340
Total | 76800

Fig 5 shows the distribution of emotion classes in DEAP. The red points are
calculated by the mean value of the corresponding class labels on Arousal-
Valence data, and it could represent the labels in two-dimensional space such
as the LALV (2.95, 3.51), HALV (6.64, 3.07), LAHV (3.44, 6.42), HAHV (6.58,
7.11). Then we calculate the Euclidean distance between various class labels
and using the standard normal distribution to calculate the classification
score reliability. In each channel, we calculate the final classification
score through classification score from CNN model and classification score
reliability, and use the standard deviation of final classification score to
represent the corresponding channel reliability, then calculate the result of
multimodal information integration.

Figure 5: Distribution of emotion classes in DEAP, and the red points are the
mean value of the corresponding labels on Arousal-Valence data, such as LALV
(2.95, 3.51), HALV (6.64, 3.07), LAHV (3.44, 6.42), HAHV (6.58, 7.11).

The 3D CNN model and 1D CNN model are used for emotion recognition based on
EEG signals and peripheral physiological signals. To verify the multimodal
fusion framework of EEG signals, the signals decomposed into four parts
according to the four frequency bands of theta (4-7 Hz), alpha (8-13 Hz), beta
(14-30 Hz) and gamma (31-45 Hz). Table 2 shows the accuracy rate of each
single-modal and the increment in fusion result. It is easy to see, there is a
significant increase of accuracy rate in fusion result compared with the
single-modal results.

Table 2: Accuracy improvement in fusion result of EEG signals and peripheral physiological signals compared with the single-modal result (DEAP). The fusion result of four frequency bands EEG signals is 95.77% and the fusion result of 8 multimodal peripheral physiological signals is 97.27%. Modality | Single-modal Result |  | Improvement in
---
Fusion Result
EEG Signals
Alpha | 89.01% | 6.76%
Beta | 89.69% | 6.08%
Gamma | 73.09% | 22.68%
Theta | 81.97% | 13.80%
Peripheral Physiological Signals
hEOG | 72.57% | 24.70%
vEOG | 86.45% | 10.82%
zEMG | 73.66% | 23.61%
tEMG | 83.74% | 13.53%
GSR | 74.60% | 22.67%
Respiration belt | 66.33% | 30.94%
Plethysmograph | 72.94% | 24.33%
Temperature | 51.43% | 45.84%

Fig 6 shows the average accuracy of fusion results in the various numbers of
moda, and the average accuracy of the fusion result is proportional to the
numbers of the modal.

Figure 6: The average accuracy of fusion result in various number of modal.

The accuracy rate will rise further with the fusion between EEG signals and
peripheral physiological signals. Table 3 shows the fusion accuracy rate of
EEG signals and peripheral physiological signals respectively and the
increment in EEG signals + peripheral physiological signals fusion result.

Table 3: Accuracy improvement in fusion result of EEG signals + peripheral physiological signals compares with the fusion result of EEG signals and peripheral physiological signals respectively (DEAP). The fusion result of EEG signals + peripheral physiological signals achieves the accuracy rate of 99.17% with the decomposed EEG signals (EEG) and 98.52% with the original EEG signals (EEG*). Modality | Result |  | EEG
---
Peripheral
| EEG*
---
Peripheral
EEG* | 93.53% | 5.64% | 4.99%
Fusion EEG | 95.77% | 3.40% | 2.75%
Fusion Peripheral | 97.27% | 1.90% | 1.25%

Fig.7 shows the result of emotion recognition in each modal and the fusion
results.

Figure 7: The result in DEAP dataset. The red bar charts are the fusion
results.The EEG represents the signals decomposed into four frequency bands,
and the EEG* represents the original EEG signals.

###  4.2 AMIGOS Dataset

The AMIGOS [24] is a new open dataset. This dataset contains 14 channels EEG
signals and 3 channels peripheral physiological signals which be collected
when 40 participants watched 20 videos (16 short videos + 4 long videos). The
EEG channels contain AF3, F7, F3, FC5, T7, P7, O1, O2, P8, T8, FC6, F4, F8,
and AF4. The peripheral physiological channels contain ECG Right, ECG Left,
and GSR. Each trial contains 5s baseline signals in first and the signals
depend on the duration of the video. The baseline signals are recorded when
the participant under no stimulus. After watching the video, the participants
rated a self-assessment of arousal, valence, liking, and dominance on a scale
from 1 to 9. A preprocessed version had been provided: The data was down-
sampled to 128Hz, and a bandpass frequency filter from 4.0-45.0Hz was applied.

Here we use the signals which were recorded in short videos experiment. The
participant ID of 9, 12, 21, 22, 23, 24 and 33 has been removed because there
are some invalid data in the preprocessed version. The data pre-processing in
the AMIGOS dataset is the same as in the DEAP dataset, and the signals also be
segmented with 1s length. The 14 channels EEG signals are mapped into a 9x9
matrixes (as shown in Fig.8). The final data size of EEG signals after
processing is 45474 9x9x128. The 3 channels physiological signals cut into 60
segments with 1s length in each channel, and the final data size of
physiological signals in each channel after processing is 45474 128x1.

(a) 10-20 System

(b) 2D Electrode Topological Structure

Figure 8: (a) The international 10-20 system describes the location of scalp
electrodes, and the red nodes show the 14 electrodes used in AMIGOS dataset.
(b) The 14 channels EEG signals are mapped into a 9x9 matrixes.

The dataset could be segmented in four classes, i.e. low arousal low valence
(LALV), high arousal low valence (HALV), low arousal high valence (LAHV), high
arousal high valence (HAHV) based on the arousal and valence value with the
threshold of 5 respectively, and the corresponding instance numbers are shown
in Table 4.

Table 4: Corresponding instance numbers in AMIGOS Label | LALV | HALV | LAHV | HAHV
---|---|---|---|---
Arousal |  $\leq$5 | >5 |  $\leq$5 | >5
Valence |  $\leq$5 |  $\leq$5 | >5 | >5
Instances | 12295 | 12327 | 10606 | 10246
Total | 45474

Fig 9 shows the distribution of emotion classes in AMIGOS, and the labels
could be represented as LALV (2.95, 3.51), HALV (6.64, 3.07), LAHV (3.44,
6.42), HAHV (6.58, 7.11) in two-dimensional space. The process of calculating
the multimodal information integration result is the same as in DEAP.

Figure 9: Distribution of emotion classes in AMIGOS, and the red points are
the mean value of the corresponding labels on Arousal-Valence data, such as as
LALV (2.95, 3.51), HALV (6.64, 3.07), LAHV (3.44, 6.42), HAHV (6.58, 7.11).

As the process of emotion recognition in DEAP, the same model be used in
AMIGOS. Table 5 shows the accuracy rate of each single-modal and the increment
in fusion result. There is a significant increase of accuracy rate in fusion
result of four frequency bands EEG signals, but the increment is unobvious in
the fusion result of peripheral physiological signals compared with the ECG
signals.

Table 5: Accuracy improvement in fusion result of EEG signals and peripheral physiological signals compared with the single-modal result (AMIGOS). The fusion result of four frequency bands EEG signals is 91.07% and the fusion result of 3 multimodal peripheral physiological signals is 99.74% Modality |  | Single-modal
---
Result
| Improvement in
---
Fusion Result
EEG Signals
Alpha | 79.54% | 11.53%
Beta | 86.80% | 4.27%
Gamma | 85.39% | 5.68%
Theta | 75.88% | 15.19%
Peripheral Physiological Signals
ECG Right | 99.71% | 0.03%
ECG Left | 99.30% | 0.44%
GSR | 84.69% | 15.05%

Table 6 shows the fusion accuracy rate of EEG signals and peripheral
physiological signals respectively and the increment in EEG signals +
peripheral physiological signals fusion result. There is a significant
increase compared with the EEG modal, but make an unsatisfactory performance
compared with the fusion result of peripheral physiological modal.

Table 6: Accuracy improvement in fusion result of EEG signals + peripheral physiological signals compared with the fusion result of EEG signals and peripheral physiological signals respectively (AMIGOS). The fusion result of EEG signals + peripheral physiological signals achieves the accuracy rate of 99.04% with the decomposed EEG signals (EEG) and 99.89% with the original EEG signals (EEG*). Modality | Result |  | EEG
---
Peripheral
| EEG*
---
Peripheral
EEG* | 95.86% | 3.18% | 4.03%
Fusion EEG | 91.07% | 7.97% | 8.82%
Fusion Peripheral | 99.74% | -0.70% | 0.15%

Fig.10 shows the result of emotion recognition in each modal and the fusion
results.

Figure 10: The result in AMIGOS dataset. The red bar charts are the fusion
results. The EEG represents the signals decomposed into four frequency bands,
and the EEG* represents the original EEG signals.

##  5 Conclusions

In this paper, we have proposed a Multimodal Emotion Recognition Model that
consists of the 3D convolutional neural network model, the 1D convolutional
neural network model and the biologically inspired multimodal fusion model for
emotion recognition. In the single-modal emotion recognition, the model
achieves the highest accuracy rate of 89.69% and 86.45% based on the Beta band
of EEG signals and the vEOG of peripheral physiological signals in the DEAP
dataset respectively, and achieve the highest accuracy rate of 86.8% and
99.71% based on the Beta band of EEG signals and ECG Right of peripheral
physiological signals in AMIGOS dataset respectively. The accuracy rate
achieves 93.53% and 95.86% with original EEG signals in DEAP and AMIGOS
dataset respectively. The biologically inspired multimodal fusion model
improves the recognition accuracy of 5% ~25% based on the fusion of EEG
signals and peripheral physiological signals compared with single-modal, and
achieve the accuracy of 95.77%, 97.27% in the DEAP dataset and 91.07%, 99.74%
in AMIGOS dataset respectively. Compared with the result of original EEG
signals, the fusion result of various bands of EEG signals gets a higher
accuracy of 2.24% in the DEAP dataset and a lower accuracy of 4.79% in the
AMIGOS dataset. This result proves that the original EEG signals contain
enough information and could be used directly for emotion classification,
besides, the result of peripheral physiological signals shows that a robust
classification of human emotion is possible without EEG signals which signals
be collected a little difficult. The fusion result of EEG signals + peripheral
physiological signals gets the highest accuracy rate of 99.17% and 99.04% with
the EEG signals decomposed into four frequency bands, 98.52% and 99.89% with
the original EEG signals in DEAP dataset and AMIGOS dataset respectively. This
model makes a good performance in the DEAP dataset and AMIGOS dataset, in the
future, we will verify the universality of the model on varied dataset.
Furthermore, the biologically inspired multimodal fusion model uses the
correlation between various labels and integrates the information on the
decision level, the multimodal information fusion method could be used for
other multimodal pattern recognition tasks.

## References

  * [1] Martín Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving, and Michael Isard.  Tensorflow: A system for large-scale machine learning.  In 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI 16), pages 265–283.
  * [2] F. Al Machot, A. Elmachot, M. Ali, E. Al Machot, and K. Kyamakya.  A deep-learning model for subject-independent human emotion recognition using electrodermal activity sensors.  Sensors (Basel), 19(7), 2019.
  * [3] S. Bagherzadeh, K. Maghooli, J. Farhadi, and M. Zangeneh Soroush.  Emotion recognition from physiological signals using parallel stacked autoencoders.  Neurophysiology, 50(6):428–435, 2019.
  * [4] P. W. Battaglia, R. A. Jacobs, and R. N. Aslin.  Bayesian integration of visual and auditory signals for spatial localization.  Journal of the Optical Society of America A: Optics, Image Science, and Vision, 20(7):1391–1397, 2003.
  * [5] M. O. Ernst and M. S. Banks.  Humans integrate visual and haptic information in a statistically optimal fashion.  Nature, 415(6870):429–433, 2002.
  * [6] C. R. Fetsch, G. C. DeAngelis, and D. E. Angelaki.  Bridging the gap between theories of sensory cue integration and the physiology of multisensory neurons.  Nature Reviews: Neuroscience, 14(6):429–442, 2013.
  * [7] C. R. Fetsch, A. H. Turner, G. C. DeAngelis, and D. E. Angelaki.  Dynamic reweighting of visual and vestibular cues during self-motion perception.  Journal of Neuroscience, 29(49):15601–15612, 2009.
  * [8] Christos A Frantzidis, Bratsas Charalampos, Christos L Papadelis, Konstantinidis Evdokimos, Pappas Costas, and Panagiotis D Bamidis.  Toward emotion aware computing: an integrated approach using multichannel neurophysiological recordings and affective visual stimuli.  IEEE Transactions on Information Technology in Biomedicine, 14(3):589–597, 2010.
  * [9] Y. Gu, D. E. Angelaki, and G. C. Deangelis.  Neural correlates of multisensory cue integration in macaque mstd.  Nature Neuroscience, 11(10):1201–1210, 2008.
  * [10] Ningjie Liu; Yuchun Fang ; Ling Li ; Limin Hou ; Fenglei Yang ; Yike Guo.  Multiple feature fusion for automatic emotion recognition using eeg signals.  2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2018.
  * [11] Vipin Gupta, Mayur Dahyabhai Chopda, and Ram Bilas Pachori.  Cross-subject emotion recognition using flexible analytic wavelet transform from eeg signals.  IEEE Sensors Journal, 19(6):2266–2274, 2019.
  * [12] Stelios K. Hadjidimitriou and Leontios J. Hadjileontiadis.  Toward an eeg-based recognition of music liking using time-frequency analysis.  IEEE Transactions on Biomedical Engineering, 59(12):3498–3510, 2012\.
  * [13] Mei Han and Xu Xiangmin.  Eeg-based emotion classification using convolutional neural network.  In 2017 International Conference on Security, Pattern Analysis, and Cybernetics (SPAC), pages 130–135. IEEE, 2017.
  * [14] Mohammad Mehedi Hassan, Md Golam Rabiul Alam, Md Zia Uddin, Shamsul Huda, Ahmad Almogren, and Giancarlo Fortino.  Human emotion recognition using deep belief network architecture.  Information Fusion, 51:10–18, 2019.
  * [15] Seyyed Abed Hosseini, Mohammad Ali Khalilzadeh, Naghibi Sistani, Mohammad Bagher, and Vahid Niazmand.  Higher order spectra analysis of eeg signals in emotional stress states.
  * [16] Robert A. Jacobs.  Optimal integration of texture and motion cues to depth.  Vision Research, 39(21):3621–3629, 1999.
  * [17] Robert Jenke, Angelika Peer, and Martin Buss.  Feature extraction and selection for emotion recognition from eeg.  IEEE Transactions on Affective Computing, 5(3):327–339, 2014.
  * [18] Reza Khosrowabadi, Hiok Chai Quek, Abdul Wahab, and Keng Ang Kai.  Eeg-based emotion recognition using self-organizing map for boundary detection.  In 20th International Conference on Pattern Recognition.
  * [19] D. C. Knill and A. Pouget.  The bayesian brain: the role of uncertainty in neural coding and computation.  Trends in Neurosciences, 27(12):712–719, 2004.
  * [20] David C. Knill and Jeffrey A. Saunders.  Do humans optimally integrate stereo and texture information for judgments of surface slant?  Vision Research, 43(24):2539–2558, 2003.
  * [21] S. Koelstra, C. Muhl, M. Soleymani, Lee Jong-Seok, A. Yazdani, T. Ebrahimi, T. Pun, A. Nijholt, and I. Patras.  Deap: A database for emotion analysis using physiological signals.  IEEE Transactions on Affective Computing, 3(1):18–31, 2012.
  * [22] Y. H. Kwon, S. B. Shin, and S. D. Kim.  Electroencephalography based fusion two-dimensional (2d)-convolution neural networks (cnn) model for emotion recognition system.  Sensors (Basel), 18(5), 2018.
  * [23] Wenqian Lin, Chao Li, and Shouqian Sun.  Deep Convolutional Neural Network for Emotion Recognition Using EEG and Peripheral Physiological Signal, book section Chapter 33, pages 385–394.  Lecture Notes in Computer Science. 2017.
  * [24] Juan Abdon Miranda Correa, Mojtaba Khomami Abadi, Niculae Sebe, and Ioannis Patras.  Amigos: A dataset for affect, personality and mood research on individuals and groups.  IEEE Transactions on Affective Computing, pages 1–1, 2018.
  * [25] Soujanya Poria, Erik Cambria, Rajiv Bajpai, and Amir Hussain.  A review of affective computing: From unimodal analysis to multimodal fusion.  Information Fusion, 37:98–125, 2017.
  * [26] James A Russell.  A circumplex model of affect.  Journal of Personality and Social Psychology, 39(6):1161, 1980.
  * [27] Elham S. Salama, Reda A.El-Khoribi, Mahmoud E.Shoman, and Mohamed A.Wahby.  Eeg-based emotion recognition using 3d convolutional neural networks.  International Journal of Advanced Computer Science and Applications, 9(8), 2018.
  * [28] Anas Samara, Maria Luiza Recena Menezes, and Leo Galway.  Feature extraction for emotion recognition and modelling using neurophysiological data.  In International Conference on Ubiquitous Computing and Communications and International Symposium on Cyberspace and Security.
  * [29] Luz Santamaria-Granados, Mario Munoz-Organero, Gustavo Ramirez-Gonzalez, Enas Abdulhay, and N. Arunkumar.  Using deep convolutional neural network for emotion detection on a physiological signals dataset (amigos).  IEEE Access, 7:57–67, 2019.
  * [30] Yangyang Shu and Shangfei Wang.  Emotion recognition through integrating eeg and peripheral signals.  In IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 2871–2875.
  * [31] S. J. Sober and P. N. Sabes.  Flexible strategies for sensory integration during motor planning.  Nature Neuroscience, 8(4):490–497, 2005.
  * [32] G. K. Verma and U. S. Tiwary.  Multimodal fusion framework: a multiresolution approach for emotion classification and recognition from physiological signals.  NeuroImage, 102 Pt 1:162–172, 2014.
  * [33] Xiao-Wei Wang, Dan Nie, and Bao-Liang Lu.  Emotional state classification from eeg data using machine learning approach.  Neurocomputing, 129:94–106, 2014.
  * [34] Yi Wang and Zhiyi Huang.  Emotionet: A 3-d convolutional neural network for eeg-based emotion recognition.  In 2018 International Joint Conference on Neural Networks (IJCNN), pages 1–7. IEEE, 2018.
  * [35] Yilong Yang, Qingfeng Wu, Ming Qiu, Yingdong Wang, and Xiaowei Chen.  Emotion recognition from multi-channel eeg through parallel convolutional recurrent neural network.  In 2018 International Joint Conference on Neural Networks (IJCNN), pages 1–7. IEEE, 2018.
  * [36] Z. Yin, M. Zhao, Y. Wang, J. Yang, and J. Zhang.  Recognition of emotions using multimodal physiological signals and an ensemble deep learning model.  Computer Methods and Programs in Biomedicine, 140:93–110, 2017\.
  * [37] M. Zangeneh Soroush, K. Maghooli, S. K. Setarehdan, and A. M. Nasrabadi.  Emotion recognition through eeg phase space dynamics and dempster-shafer theory.  Medical Hypotheses, 127:34–45, 2019.
  * [38] Morteza Zangeneh Soroush, Keivan Maghooli, Seyed Kamaledin Setarehdan, and Ali Motie Nasrabadi.  A novel eeg-based approach to classify emotions through phase space dynamics.  Signal, Image and Video Processing, 13(6):1149–1156, 2019.
  * [39] Wei-Long Zheng, Jia-Yi Zhu, and Bao-Liang Lu.  Identifying stable patterns over time for emotion recognition from eeg.  IEEE Transactions on Affective Computing, pages 1–1, 2017.