# Multi-dimensional Kuramoto-Sivashinsky-Zakharov-Kuznetsov equation posed on
admisssible multi-dimensional domains

N. A. Larkin  Departamento de Matemática, Universidade Estadual de Maringá,
Av. Colombo 5790: Agência UEM, 87020-900, Maringá, PR, Brazil
nlarkine<EMAIL_ADDRESS>
###### Abstract.

An initial- boundary value problem for the n-dimensional ($n$ is a natural
number from the interval [2,7]) Kuramoto-Sivashinsky-Zakharov-Kuznetsov
equation posed on smooth bounded domains in ${\mathbb{R}}^{n}$ was considered.
The existence and uniqueness of global regular solutions as well as their
exponential decay have been established. A connection between an order of the
stationary part of the equation and admissible dimensions of a domain has been
revealed.

MSC 2010:35Q35; 35Q53.
Keywords: Kuramoto-Sivashinsky equation; Zakharov-Kuznetsov equation; Global
solutions; Decay in bounded Domains

##  1\. Introduction

In this work, we study the existence, uniqueness, regularity and exponential
decay of global solutions to an initial-boundary value problem for the
$n$-dimensional Kuramoto-Sivashinsky-Zakharov-Kuznetsov equation (KS-ZK)

$\displaystyle\phi_{t}+\Delta^{2}\phi+\Delta\phi+\Delta\phi_{x_{1}}+\frac{1}{2}|\nabla\phi|^{2}=0$
(1.1)

with the purpose to reveal a connection between an order of the stationary
part of the KS-ZK equation and admiissible dimensions of domains involved.
Here $n$ is a natural number from the interval [2,7], $\Delta$ and $\nabla$
are the Laplacian and the gradient in ${\mathbb{R}}^{n}.$ In [15], Kuramoto
studied the turbulent phase waves and Sivashinsky in [25] obtained an
asymptotic equation which modeled the evolution of a disturbed plane flame
front. See also [9]. Mathematical results on initial and initial-boundary
value problems for one-dimensional (1.1) are presented in [3, 4, 7, 8, 12, 16,
23, 26, 31], see references there for more information. Multi-dimensional
problems for equations of (1.1) type can be found in [2, 5, 7, 21, 22, 23, 24,
26], where some results on existence, regularity and nonlinear stability of
solutions have been presented. In one-dimensional case, Kuramoto-Sivashinsky
type equations which included the (KdV) term $\phi_{xxx}$ have been considered
[3, 4, 12, 16, 31]. The aim of these studies were intentions to understand
simultaneous influences of dispersion and dissipation on stability of systems
modeled by these equations. In [16], an initial-boundary value problem for the
KdV equation was regularized by initial-boundary value problems for the
Kuramoto-Sivashinsky equation with a consequent passage to the limit with
respect to regularization parameters.

In multi-dimensional cases, $\phi_{xxx}$ must be replaced by
$\Delta\phi_{x_{1}}$, Zakharov-Kuznetsov term, see [30], where the propagation
of nonlinear ionic-sonic waves in a plasma submitted to a magnetic field
directed along the $x_{1}$-axis was studied.This Kuramoto-Sivashinsky-
Zakharov-Kuznetsov (KS-ZK) equation may be useful in studies of nonlinear
stability of a three-dimensional viscous film flowing down an inclined
surface; see [29], where it has been observed that the three-dimensional
structure of waves can suppress an instability due to presence of the fourth-
order derivative term.

There is an extensive literature on existence, uniqueness and stability of
global solutions for both ZK equation, see [10, 11, 26] and for the Kuramoto-
Sivashinsky equation, see [7, 12, 18, 20, 22, 31]. Recently, appeared
published papers on solvability of initial-boundary value problems for two-
dimensional and three-dimensional Kuramoto-Sivashinsky-Zakharov-Kuznetsov
equation in bounded and unbounded domains, see [17, 19]. In [18], an initial-
boundary value problem for the n-dimensional Kuramoto-Sivashinsky equation
posed on smooth domains in ${\mathbb{R}}^{n},\;n\in[2,7]$ has been studied

This motivated us to consider n-dimensional Kuramoto-Sivashinsky-Zakharov-
Kuznetsov equation and to define natural $n$ which allow to prove the
existence, uniqueness and exponential decay of global regular solutions.
Consequently, we call (1.1) Kuramoto-Sivashinsky-Zakharov-Kuznetsov equation
(KS-ZK).
For $n$ dimensions, (1.1) can be rewritten in the form of the following
system:

$\displaystyle(u_{j})_{t}+\Delta^{2}u_{j}+\Delta u_{j}+(\Delta
u_{j})_{x_{1}}+\frac{1}{2}\sum_{i=1}^{n}(u_{i})^{2}_{x_{j}}=0,$ (1.2)
$\displaystyle(u_{i})_{x_{j}}=(u_{j})_{x_{i}},\;i\neq j,\;i,j=1,...,n,$ (1.3)

where $u_{j}=\phi_{x_{j}}.$ First essential problem that arises while one
studies stability either for (1.1) or for (1.2)-(1.3) is a destabilizing
effect of $\Delta u_{j}$ that may be damped by a dissipative term
$\Delta^{2}u_{j}$ provided a domain has some specific properties. Naturally,
so called ”thin domains” appear where some dimensions are small while the
others may be large, see [14, 24].

Our approach is based on the Faedo-Galerkin method and simple, transparent
conditions for the geometry of a domain which suppress instability. These
conditions were obtained using Steklov‘s inequalities.

Second essential problem is presence of semi-linear terms in (1.2) which are
interconnected. Differently from the one-dimensional case, this does not allow
to obtain the first estimate independent of time and solutions and leads to a
connection between geometric properties of a domain and initial data.
Our work has the following structure: Section I is Introduction. Section 2
contains notations and auxiliary facts. In Section 3, formulation of an
initial-boundary value problem for (1.2)-(1.3) posed on bounded smooth domains
in ${\mathbb{R}}^{n}$ with boundary conditions of a solution and the Laplace
operator of the solution on boundaries of domains are given. The existence and
uniqueness of a global regular solution, exponential decay of the $H^{2}$-norm
have been established. Moreover, a ”smoothing” effect has been observed here.
Section 4 contains conclusions.

##  2\. Notations and Auxiliary Facts

let $D_{n}$ be a suffuciently smooth domain in ${\mathbb{R}}^{n}$ satisfying
the Cone condition, [1], and $x=(x_{1},...,x_{n})\in D_{n},$ where $n$ is a
fixed natural number $n\in[2,7].$ We use the standard notations of Sobolev
spaces $W^{k,p}$, $L^{p}$ and $H^{k}$ for functions and the following
notations for the norms [1] for scalar functions $f(x,t):$

$\|f\|^{2}=\int_{D_{n}}|f|^{2}dx,\hskip
28.45274pt\|f\|_{L^{p}(D_{n})}^{p}=\int_{D_{n}}|f|^{p}\,dx,$
$\|f\|_{W^{k,p}(D_{n})}^{p}=\sum_{0\leq\alpha\leq
k}\|D^{\alpha}f\|_{L^{p}(D_{n})}^{p},\hskip
28.45274pt\|f\|_{H^{k}(D_{n})}=\|f\|_{W^{k,2}(D_{n})}.$

When $p=2$, $W^{k,p}(D_{n})=H^{k}(D_{n})$ is a Hilbert space with the scalar
product

$((u,v))_{H^{k}(D_{n})}=\sum_{|j|\leq
k}(D^{j}u,D^{j}v),\;\|u\|_{L^{\infty}(D_{n})}=ess\;sup_{D_{n}}|u(x)|.$

We use a notation $H_{0}^{k}(D_{n})$ to represent the closure of
$C_{0}^{\infty}(D_{n})$, the set of all $C^{\infty}$ functions with compact
support in $D_{n}$, with respect to the norm of $H^{k}(D_{n})$.

######  Lemma 2.1 (Steklov’s Inequality [27]).

Let $v\in H^{1}_{0}(0,L).$ Then

$\frac{\pi^{2}}{L^{2}}\|v\|^{2}\leq\|v_{x}\|^{2}.$ (2.1)

######  Lemma 2.2 (Differential form of the Gronwall Inequality).

Let $I=[t_{0},t_{1}]$. Suppose that functions $a,b:I\to{\mathbb{R}}$ are
integrable and a function $a(t)$ may be of any sign. Let $u:I\to{\mathbb{R}}$
be a positive differentiable function satisfying

$u_{t}(t)\leq a(t)u(t)+b(t),\text{ for }t\in I\text{ and }\,\,u(t_{0})=u_{0},$
(2.2)

then

$u(t)\leq
u_{0}e^{\int_{t_{0}}^{t}a(\tau)\,d\tau}+\int^{t}_{t_{0}}e^{\int_{t_{0}}^{s}a(r)\,dr}b(s)ds$

###### Proof.

Multiply (2.2) by the integrating factor $e^{\int_{t_{0}}^{t}a(\tau)d\tau}$
and integrate the result from $t_{0}$ to $t.$ ∎

The next Lemmas will be used in estimates:

######  Lemma 2.3 (See: [13], Theorem 9.1).

Let $n$ be a natural number from the interval $[2,7];\;D_{n}$ be a
sufficiently smooth bounded domain in ${\mathbb{R}}^{n}$ satisfying the Cone
condition and $v\in H^{4}(D_{n})\cap H^{1}_{0}(D_{n})$. then for all natural
$n\in[2,7]$

$\sup_{D_{n}}|v(x)|\leq C_{n}\|v\|_{H^{4}(D_{n})}.$ (2.3)

The constant $C_{n}$ depends on $n,D_{n}.$

######  Lemma 2.4.

Let $f(t)$ be a continuous positive function such that

$\displaystyle f^{\prime}(t)+(\alpha-kf^{n}(t))f(t)\leq
0,\;t>0,\;n\in{\mathbb{N}},$ (2.4) $\displaystyle\alpha-kf^{n}(0)>0,\;k>0.$
(2.5)

Then

$f(t)<f(0)$ (2.6)

for all $t>0$.

###### Proof.

Obviously, $f^{\prime}(0)+(\alpha-kf^{n}(0))f^{n}(0)\leq 0$. Since $f$ is
continuous, there exists $T>0$ such that $f(t)<f(0)$ for every $t\in[0,T)$.
Suppose that $f(0)=f(T)$. Integrating (2.4), we find

$f(T)+\int_{0}^{T}(\alpha-kf^{n}(t))f(t)\,dt\leq f(0).$

Since

$\int_{0}^{T}(\alpha-kf^{n}(t))f(t)\,dt>0,$

then $f(T)<f(0).$ This contradicts that $f(T)=f(0).$ Therefore, $f(t)<f(0)$
for all $t>0.$
The proof of Lemma 2.4 is complete. $\Box$

##  3\.  KS-ZK system posed on bounded smooth domains in ${\mathbb{R}}^{n}$.
Special basis

Let $\Omega_{n}$ be the minimal nD-parallelepiped containing a given bounded
smooth domain $\bar{D_{n}}\in{\mathbb{R}}^{n},\;n=2,...,7$:

$\Omega_{n}=\\{x\in{\mathbb{R}}^{n};x_{i}\in(0,L_{i})\\},\;\;i=1,...,n.$

Fixing any natural $n=2,...,7,$ consider in $Q_{n}=D_{n}\times(0,t)$ the
following initial-boundary value problem:

$\displaystyle(u_{j})_{t}+\Delta^{2}u_{j}+\Delta u_{j}+(\Delta
u_{j})_{x_{1}}+\frac{1}{2}\sum_{i=1}^{n}(u_{i})^{2}_{x_{j}}=0;$ (3.1)
$\displaystyle(u_{i})_{x_{j}}=(u_{j})_{x_{i}},\;i\neq j;$ (3.2) $\displaystyle
u_{j}|_{\partial D_{n}}=\Delta u_{j}|_{\partial D_{n}}=0,\;t>0;$ (3.3)
$\displaystyle u_{j}(x,0)=u_{j0}(x),\;j=1,...,n;\;x\in D_{n}.$ (3.4)

######  Lemma 3.1.

Let $f\in H^{4}(D_{n})\cap H^{1}_{0}(D_{n}),\;\Delta f|_{\partial D_{n}}=0$.
Then

$\displaystyle a\|f\|^{2}\leq\|\nabla f\|^{2},\;\;a^{2}\|f\|^{2}\leq\|\Delta
f\|^{2},\;\;a\|\nabla f\|^{2}\leq\|\Delta f\|^{2},$ (3.5) $\displaystyle
a^{2}\|\Delta f\|^{2}\leq\|\Delta^{2}f\|^{2},\;\;a\|\Delta
f_{x_{1}}\|^{2}\leq\|\Delta^{2}f\|^{2},\;\text{where}\;a=\sum_{i=1}^{n}\frac{\pi^{2}}{L_{i}^{2}}.$
(3.6)

###### Proof.

By definition,

$\|\nabla f\|^{2}=\sum_{i=1}^{n}\|f_{x_{i}}\|^{2}.$

Extending $f\in H^{1}_{0}(D_{n})$ by $0$ into the minimal parallelepiped
$\Omega_{n}$ as $\bar{f}$, making use of Steklov‘s inequalities in
$\Omega_{n}$ and taking into account that
$\|\nabla\bar{f}\|_{\Omega_{n}}=\|\nabla f\|_{D_{n}},$ we get

$\|\nabla
f\|^{2}\geq\sum_{i=1}^{n}\frac{\pi^{2}}{L_{i}^{2}}\|f\|^{2}=a\|f\|^{2}.$

On the other hand,

$a\|f\|^{2}\leq\|\nabla f\|^{2}=-\int_{D_{n}}f\Delta fdx\leq\|\Delta
f\|\|f\|.$

This implies

$a\|f\|\leq\|\Delta f\|\;\;\text{and}\;\;a^{2}\|f\|^{2}\leq\|\Delta f\|^{2}.$

Consequently, $a\|\nabla f\|^{2}\leq\|\Delta f\|^{2}.$ Similarly,

$\|\Delta
f\|^{2}=(\Delta^{2}f,f)\leq\|\Delta^{2}f\|\|f\|\leq\frac{1}{a}\|\Delta^{2}f\|\|\Delta
f\|,$ $\|\Delta f_{x_{1}}\|^{2}\leq\|\nabla\Delta f\|^{2}=-(\Delta^{2}f,\Delta
f)\leq\|\Delta^{2}f\|\|\Delta f\|\leq\frac{1}{a}\|\Delta^{2}f\|^{2}.$

Assertions of Lemma 3.1 follow from these inequalities. Proof of Lemma 3.1 is
complete. ∎

######  Remark 3.1.

Assertions of Lemma 3.1 are true if the function $f$ is replaced respectively
by $u_{j},\;j=1,...,n$.

######  Lemma 3.2.

Let $a>1$. In conditiions of Lemma 3.1,

$\displaystyle\|f\|^{2}(t)_{H^{2}(D_{n})}\leq 3\|\Delta f\|^{2}(t),$ (3.7)
$\displaystyle\|f\|^{2}(t)_{H^{4}(D_{n})}\leq 5\|\Delta^{2}f\|^{2}(t),$ (3.8)
$\displaystyle\sup_{D_{n}}|f(x)|\leq
C_{s}\|\Delta^{2}f\|,\;\text{where}\;C_{s}=5C_{n}.$ (3.9)

###### Proof.

To prove (3.8), making use of Lemma 3.1, we find

$\|f\|^{2}_{H^{4}(D_{n})}=\|f\|^{2}+\|\nabla f\|^{2}+\|\Delta
f\|^{2}+\|\nabla\Delta f\|^{2}+\|\Delta^{2}f\|^{2}$
$\leq\Big{(}\frac{1}{a^{4}}+\frac{1}{a^{3}}+\frac{1}{a^{2}}+\frac{1}{a}+1\Big{)}\|\Delta^{2}f\|^{2}.$

Since $a>1$, then (3.8) follows. Similarly, (3.7) can be proved. Moreover,
taking into account Lemma 2.3, we get (3.9). ∎

######  Theorem 3.1.

Let Let $n$ be a natural number from the closed interval [2,7];
$D_{n}\in{\mathbb{R}}^{n}$ be a bounded smooth domain satisfying the Cone
condition and $\Omega_{n}$ be the minimal $nD$-parallelepiped containing
$\bar{D_{n}}.$ Let

$\displaystyle
2a=2\sum_{i=1}^{n}\frac{\pi^{2}}{L_{i}^{2}}>3+5^{1/2},\;\;\theta=1-\frac{1}{a}-\frac{1}{a^{1/2}}>0.$
(3.10)

Given

$u_{j0}(D_{n})\in H^{2}(D_{n})\cap H^{1}_{0}(D_{n}),\;j=1,...,n$

such that

$\displaystyle\theta-\frac{2C_{s}^{2}7^{3}}{a\theta}\Big{(}\sum_{j=1}^{n}\|\Delta
u_{j}\|^{2}(0)\Big{)}>0,$ (3.11)

then there exists a unique global regular solution to (3.1)- (3.4):

$u_{j}\in L^{\infty}({\mathbb{R}}^{+};H^{2}(D_{n})\cap H^{1}_{0}(D_{n}))\cap
L^{2}({\mathbb{R}}^{+};H^{4}(D_{n})\cap H^{1}_{0}(D_{n}));$ $u_{jt}\in
L^{2}({\mathbb{R}}^{+};L^{2}(D_{n})),\;j=1,...,n.$

Moreover,

$\displaystyle\sum_{j=1}^{n}\|\Delta
u_{j}\|^{2}(t)\leq\Big{(}\sum_{j=1}^{n}\|\Delta
u_{j0}\|^{2}\Big{)}\exp\\{-a^{2}t\theta/2\\}$ (3.12)

and

$\displaystyle\sum_{i=1}^{n}\|\Delta
u_{i}\|^{2}(t)+\int_{0}^{t}\sum_{i=1}^{n}\|\Delta^{2}u_{i}\|^{2}(\tau)d\tau\leq
C\sum_{i=1}^{n}\|\Delta u_{i0}\|^{2},\;t>0.$

###### Proof.

Let $\\{w_{i}(x)$ be eigenfunctions of the following problem:

$\Delta^{2}w_{i}-\lambda_{i}w_{i}=0,\;x\in D_{n};\;w_{i}|_{\partial
D_{n}}=\Delta w_{i}|_{\partial D_{n}}=0.$

We construct approximate solutions to (3.1)-(3.4) in the form

$u^{N}_{j}(x,t)=\sum_{i=1}^{N}g_{i}^{j}(t)w_{i}(x);\;j=1,...,n.$

Unknown functions $g_{i}^{j}(t)$ satisfy the following initial problems:

$\displaystyle\frac{d}{dt}(u^{N}_{j},w_{j})(t)+(\Delta u^{N}_{j},\Delta
w_{j})(t)-(\nabla u^{N}_{j},\nabla w_{j})(t)$ $\displaystyle-((\nabla
u^{N}_{j})_{x_{1}},\nabla
w_{j})(t)+\frac{1}{2}\sum_{i=1}^{n}((u^{N}_{i})^{2}_{x_{j}},w_{j})(t)=0,$
(3.13) $\displaystyle g_{i}^{j}(0)=g_{i0}^{j},\;\j=1,...,n.$ (3.14)

By Caratheodory‘s existence theorem, [6], Theorems 1.2 of Chapter 1, there
exist solutions to (3.13)-(3.14) at least locally in $t$. All the estimates we
will prove will be done on smooth solutions to (3.1)-(3.4). Naturally, the
same estimates are true also for approximate solutions $u^{N}_{j}.$

Estimate I Multiply (3.1) by $2\Delta^{2}u_{j}$ to obtain

$\displaystyle\frac{d}{dt}\|\Delta
u_{j}\|^{2}(t)+2\|\Delta^{2}u_{j}\|^{2}(t)-2\|\Delta^{2}u_{j}\|(t)\|\Delta
u_{j}\|(t)$ $\displaystyle-2\|(\Delta
u_{j})_{x_{1}}\|(t)\|\Delta^{2}u_{j})\|(t)$
$\displaystyle+2\sum_{i=1}^{n}((u_{i})(u_{i})_{x_{j}},\Delta^{2}u_{j})(t)\leq
0,\;j=1,...,n.$

By Lemma 3.1, $a\|\Delta u_{j}\|\leq\|\Delta^{2}u_{j}\|,\;a^{1/2}\|(\Delta
u_{j})_{x_{1}}\|\leq\|\Delta^{2}u_{j}\|,$ hence the last inequality can be
rewritten as

$\displaystyle\frac{d}{dt}\|\Delta
u_{j}\|^{2}(t)+2\Big{(}1-\frac{1}{a}-\frac{1}{a^{1/2}}\Big{)}\|\Delta^{2}u_{j}\|^{2}(t)$
$\displaystyle+2\sum_{i=1}^{n}((u_{i})(u_{i})_{x_{j}},\Delta^{2}u_{j})(t)\leq
0,\;j=1,...,n.$

By definition, $\theta=1-\frac{1}{a}-\frac{1}{a^{1/2}}$ must be positive.
Solving the inequality

$1-\frac{1}{a}-\frac{1}{a^{1/2}}>0$

with respect to $a$, we find that $2a>3+5^{1/2}$, see (3.10). In [17, 19], it
was defined $a>3$. This condition is more restrictive, but more explicit.

Making use of Lemma 3.1 and definition of $\theta$, we get

$\displaystyle\frac{d}{dt}\|\Delta
u_{j}\|^{2}(t)+2\theta\|\Delta^{2}u_{j}\|^{2}(t)$ $\displaystyle\leq
2\sum_{i=1}^{n}(\sup_{D}|u_{i}|(t)\|\nabla
u_{i}\|(t))\|\Delta^{2}u_{j}\|(t),\;j=1,...,n.$ (3.15)

Taking into account Lemmas 2.5, 3.1, 3.2, we can rewrite (3.15) as

$\displaystyle\frac{d}{dt}\|\Delta
u_{j}\|^{2}(t)+2\theta\|\Delta^{2}u_{j}\|^{2}(t)$ $\displaystyle\leq
2\Big{[}\sum_{i=1}^{n}\sup_{D_{n}}|u_{i}(x,t)|\|\nabla
u_{i}\|(t)\Big{]}\|\Delta^{2}u_{j}\|(t)$ $\displaystyle\leq
2\Big{[}C_{s}\sum_{i=1}^{n}\|\Delta^{2}u_{i}\|(t)\|\nabla
u_{i}\|(t)\Big{]}\|\Delta^{2}u_{j}\|(t);j=1,...,n.$ (3.16)

Summing over $j=1,...,n$ and making use of Lemma 3.1, we rewrite (3.16) in the
form:

$\displaystyle\frac{d}{dt}\sum_{j=1}^{n}\|\Delta
u_{j}\|^{2}(t)+2\theta\sum_{j=1}^{n}\|\Delta^{2}u_{j}\|^{2}(t)$
$\displaystyle\leq 2C_{s}n\Big{(}\sum_{j=1}^{n}\|\nabla
u_{j}\|(t)\Big{)}\Big{[}\sum_{j=1}^{n}\|\Delta^{2}u_{j}\|^{2}(t)\Big{]}$
$\displaystyle\leq\Big{[}\frac{\theta}{2}+\frac{2C_{s}^{2}n^{2}}{\theta}\Big{(}\sum_{j=1}^{n}\|\nabla
u_{j}\|(t)\Big{)}^{2}\Big{]}\sum_{j=1}^{n}\|\Delta^{2}u_{j}\|^{2}(t)$
$\displaystyle\leq\Big{[}\frac{\theta}{2}+\frac{2C_{s}^{2}n^{3}}{\theta}\Big{(}\sum_{j=1}^{n}\|\nabla
u_{j}\|^{2}(t)\Big{)}\Big{]}\sum_{j=1}^{n}\|\Delta^{2}u_{j}\|^{2}(t)$
$\displaystyle\leq\Big{[}\frac{\theta}{2}+\frac{2C_{s}^{2}n^{3}}{a\theta}\Big{(}\sum_{j=1}^{n}\|\Delta
u_{j}\|^{2}(t)\Big{)}\Big{]}\sum_{j=1}^{n}\|\Delta^{2}u_{j}\|^{2}(t).$

Taking this into account, we transform (3.16) as follows:

$\displaystyle\frac{d}{dt}\sum_{j=1}^{n}\|\Delta
u_{j}\|^{2}(t)+\frac{\theta}{2}\sum_{j=1}^{n}\|\Delta^{2}u_{j}\|^{2}(t)$
$\displaystyle+\Big{[}\theta-\frac{2C_{s}^{2}n^{3}}{a\theta}\Big{(}\sum_{j=1}^{n}\|\Delta
u_{j}\|^{2}(t)\Big{)}\Big{]}\sum_{j=1}^{n}\|\Delta^{2}u_{j}\|^{2}(t)\leq 0.$
(3.17)

Condition (3.11) and Lemma 2.4 guarantee that

$\theta-\frac{2C_{s}^{2}n^{3}}{a\theta}\Big{(}\sum_{j=1}^{n}\|\Delta
u_{j}\|^{2}(t)\Big{)}>0,\;\;t>0.$

Hence, by Lemma 3.1, (3.17) can be rewritten as

$\displaystyle\frac{d}{dt}\sum_{j=1}^{n}\|\Delta
u_{j}\|^{2}(t)+\frac{a^{2}\theta}{2}\sum_{j=1}^{n}\|\Delta u_{j}\|^{2}(t)\leq
0.$ (3.18)

Integrating, we get

$\displaystyle\sum_{i=1}^{n}\|\Delta u_{j}\|^{2}(t)\leq\sum_{j=1}^{n}\|\Delta
u_{j0}\|^{2}\exp\\{-a^{2}\theta t/2\\}.$ (3.19)

Since

$\theta-\frac{2C_{s}^{2}n^{3}}{a\theta}\Big{(}\sum_{j=1}^{n}\|\Delta
u_{j}\|^{2}(t)\Big{)}>0,\;\;t>0,$

returning to (3.17), we find

$\displaystyle\sum_{i=1}^{n}\|\Delta
u_{i}\|^{2}(t)+\int_{0}^{t}\sum_{i=1}^{n}\|\Delta^{2}u_{i}\|^{2}(\tau)d\tau\leq
C\sum_{i=1}^{n}\|\Delta u_{i0}\|^{2}.$ (3.20)

By Lemma 3.1, $a^{1/2}\|\nabla\Delta u_{j}\|\leq\|\Delta^{2}u_{j}\|,$ then
directly from (3.1),

$(u_{j})_{t}\in L^{2}({\mathbb{R}}^{+};L^{2}(D_{n})),\;j=1,...,n.$

Obviously, these inclusions and estimates (3.19), (3.20) do not depend on $N$
that allow us to pass to the limit as $n\to\infty$ in (3.13), (3.14) and to
prove the existence part of Theorem 3.1.

######  Lemma 3.3.

A regular solution of Theorem 3.1 is uniquely defined.

###### Proof.

###### Proof.

Let $u_{j}$ and $v_{j}$, $j=1,...,n$ be two distinct solutions to (3.1)-(3.4).
Denoting $w_{j}=u_{j}-v_{j}$, we come to the following system:

$\displaystyle(w_{j})_{t}+\Delta^{2}w_{j}+\Delta
w_{j}+\Delta(u_{j})_{x_{1}}+\frac{1}{2}\sum_{i=1}^{n}\Big{(}u^{2}_{i}-v^{2}_{i}\Big{)}_{x_{j}}=0,$
(3.21) $\displaystyle(w_{j})_{x_{i}}=(w_{i})_{x_{j}},\;i\neq j,$ (3.22)
$\displaystyle w_{j}|_{\partial D_{n}}=\Delta w_{j}|_{\partial
D_{n}}=0,\;t>0;$ (3.23) $\displaystyle
w_{j}(x,0)=0\;\text{in}\;D_{n},\;\;j=1,...,n.$ (3.24)

Multiply (3.21) by $2w_{j}$ to obtain

$\displaystyle\frac{d}{dt}\|w_{j}\|^{2}(t)+2\|\Delta w_{j}\|^{2}(t)-2\|\nabla
w_{j}\|^{2}(t)$ $\displaystyle-2\|\nabla(w_{i})_{x_{1}}\|(t)\|\nabla
w_{j}\|(t)\leq\sum_{i=1}^{n}(|\\{u_{i}+v_{i}\\}w_{i},(w_{j})_{x_{j}}|)(t).$
(3.25)

Making use of Lemmas 2.3, 3.1, 3.2, we estimate

$I_{1}=2\|\nabla(w_{i})_{x_{1}}\|(t)\|\nabla w_{j}\|(t)\leq
2\frac{1}{a^{1/2}}\|\Delta w_{j}\|^{2}(t),$
$2\|w_{j}\|^{2}(t)\leq\frac{2}{a}\|\Delta w_{j}\|^{2}(t).$
$I_{2}=\sum_{i=1}^{n}(|\\{u_{i}+v_{i}\\}w_{i},(w_{j})_{x_{j}}|)\leq\sum_{i=1}^{n}(\sup_{D_{n}}|\\{u_{i}+v_{i}\\}|\|w_{i}\|\|(w_{j})_{x_{j}}\|)$
$\leq
C_{s}\sum_{i=1}^{n}\|w_{i}\|\Big{(}\|\Delta^{2}(u_{i}+v_{i})\|\Big{)}\|\nabla(w_{j})\|$
$\leq\frac{C_{s}}{a^{1/2}}\sum_{i=1}^{n}\|w_{i}\|\Big{(}\|\Delta^{2}(u_{i}+v_{i})\|\Big{)}\|\Delta
w_{j}\|$ $\leq\frac{\epsilon}{2}\|\Delta
w_{j}\|^{2}+\frac{nC_{s}^{2}}{2a\epsilon}\Big{[}\sum_{i=1}^{n}\\{\|\Delta^{2}u_{i}\|^{2}+\|\Delta^{2}v_{i}\|^{2}\\}\Big{]}\sum_{j=1}^{n}\|w_{j}\|^{2}.$

Here $\epsilon$ is an arbitrary positive number. Substituting $I_{1},I_{2}$
into (3.25), we get

$\displaystyle\frac{d}{dt}\|w_{j}\|^{2}(t)+(2\theta-\frac{\epsilon}{2})\|\Delta
w_{j}\|^{2}(t)$
$\displaystyle\leq\frac{nC_{s}^{2}}{2a\epsilon}\Big{[}\sum_{i=1}^{n}\\{\|\Delta^{2}u_{i}\|^{2}+\|\Delta^{2}v_{i}\|^{2}\\}\Big{]}\sum_{j=1}^{n}\|w_{j}\|^{2}.$
(3.26)

Taking $\epsilon=4\theta$ and summing up over $j=1,...,n,$ we transform (3.26)
as follows:

$\displaystyle\frac{d}{dt}\sum_{j=1}^{n}\|w_{j}\|^{2}(t)\leq
C\Big{[}\sum_{i=1}^{n}\\{\|\Delta^{2}u_{i}\|^{2}+\|\Delta^{2}v_{i}\|^{2}\\}\Big{]}\sum_{j=1}^{n}\|w_{j}\|^{2},\;i=1,...,n.$

Since by (3.20),

$\|\Delta^{2}u_{i}\|^{2}(t),\|\Delta^{2}v_{i}\|^{2}(t)\in
L^{1}({\mathbb{R}}^{+}),$

then by Lemma 2.2,

$\|w_{j}\|^{2}(t)\equiv 0\;\;j=1,...,n,\;\text{for all}\;\;t>0.$

Hence

$\displaystyle u_{j}(x,t)\equiv v_{j}(x,t);j=1,...,n.$

The proof of Lemma 3.3, and consequently, Theorem 3.1 are complete.

∎

∎

∎

##  4\.  Conclusions

In this work, we studied the initial-boundary value problem for the
n-dimensional Kuramoto-Sivashinsky-Zakharov-Kuznetsov system (1.2), (1.3)
posed on bounded smooth domains $D_{n}\in{\mathbb{R}}^{n}.$. The problem on a
bounded domains $D_{n}$ has homogeneous conditions for a solution and the
Laplace operator of the solution on the boundaries of the domains. Making use
of a special base, we established the existence, uniqueness and exponential
decay of the $H^{2}$-norm as well as ”smoothing effect”: for initial data from
$H^{2}(D_{n})\cap H^{1}_{0}(D_{n})$, we proved that for all natural
$n\in[2,7]$ solutions pertained to $L^{2}({\mathbb{R}}^{+};H^{4}(D_{n})).$ A
set of admissible domains has been defined which eliminate destabilizing
effects of terms $\Delta u_{j}$ by dissipation of $\Delta^{2}u_{j}.$
Since these problems do not admit the first estimate independent of $t$ and
solutions, in order to prove the existence of global solutions, we put
conditions connecting geometrical properties of domains with initial data, see
(3.11).

For $n=2,3,$ initial-boundary value problems have been studied in [17, 19],
nevertheless, we included these cases because they are covered by the same
scheme and are special cases of the general results.

## Conflict of Interests

The author declares that there is no conflict of interest regarding the
publication of this paper.

## References

  * [1] R.A. Adams and J.F. Fournier,  Sobolev Spaces.  Elsevier Science Ltd; Oxford OX5 IGB UK, @003o.
  * [2] S. Benachour, I. Kukavica, W. Rusin, M. Ziane,  Anisotropic estimates for the two-dimensional Kuramoto-Sivashinsky equation,  J. of Dynamics and Differential Equations, Springer Verlag, 2014, 26, pp. 461-476. 10.1007/s10884-014-9372-3. hal-00790207.
  * [3] D.J. Benney,  Long waves on liquid films,  J.M.P., 45 (1966),150-155.
  * [4] H.A. Biagioni, J.L. Bona, R.J. Iorio Jr., and M. Scialom,  On the Korteweg de Vries-Kuramoto-Sivashinsky Equation,  Adv. Diff. Eqs., 1(1) (1996),1-20.
  * [5] H.A. Biagioni, I. Gramchev,  Multi-dimensional Kuramoto-Sivashinsky type equations: Singular initial data and analytic regularity,  Matemática Contemporanea, 15 (1998), 21-42.
  * [6] E. Coddington, N. Levinson,  Theory of Ordinary Differential Equations.  New York: MacGraw-Hill; 1955.
  * [7] Guo Boling,  The existence and nonexistence of a global solution for the initial-value problem of generalized Kuramoto-Sivashinsky type equations,  J. of Mathematical Research and Exposition, vol. 11, N0 1,(1991), 57-69.
  * [8] A.T Cousin and N.A. Larkin,  Kuramoto-Sivashinsky equation in domains with moving boundaries,  Portugaliae Mathematica, vol. 59, fasc. 3 (2002), 335-349.
  * [9] M.C. Cross,  Pattern formation outside of equilibrium,  Review of Modern Physics. 65(3) (1993), 851-1086.
  * [10] G.G. Doronin, N.A. Larkin,  Stabilization of regular solutions for the Zakharov-Kuznetsov equation posed on bounded rectangles and on a strip,  Proceeding of the Edinburgh Mathematical Society, 2(58), N3, 661–682 (2015)
  * [11] A.V. Faminskii,  Initial-boundary value problems in a half-strip for two-dimensional Zakharov-Kuznetsov equation,  Ann. Inst. H. Poincaré (C) Analyse Non Linéaire 35, 1235-1265 (2018).
  * [12] Feng Bao-Feng, B.A. Malomed, T. Kawahara,  Stable Periodic Waves in Coupled Sivashinsky-Korteweg-de Vries Equations,  arXiv:nlin/0209003 v 1 [nlin. PS] 1Sep 2002.
  * [13] A. Friedman,  Partial differential equations,  Dover publication, Mineola, N.Y. 1997.
  * [14] D. Iftimie, G. Raugel,  Some Results on the Navier-Stokes Equations in Thin 3D Domains,  Journal of Differential Equations 169, (2001), 281-331.
  * [15] Y. Kuramoto and T. Tsuzuki,  On the formation of dissipative structures in reaction-diffusion systems,  Progr. Theor. Phys., 54 (1975), 687-699.
  * [16] N.A. Larkin,  Korteweg de Vries and Kuramoto-Sivashinsky equations in bounded domains.  J. Math. Anal. Appl., 297 (2004), 169-185.
  * [17] N.A. Larkin,  Existence and decay of global solutions for the three-dimensional Kuramoto-Sivashinsky-Zakharov-Kuznetsov equation ,  JMAA online, doi:org/101016/jmaa.2022.126046
  * [18] N.A. Larkin,  Global regular solutions for the multi-dimensional Kuramoto-Sivashinsky equation posed on smooth domains.  arXiv:2203.14739v1 [math.AP] 28 Mar 2022
  * [19] N.A. Larkin,  Existence and decay of global solutions for the Kuramoto-Sivashinsky-Zakharov-Kuznetsov equation posed on rectangles,  PDEA, doi: 10.1007/s42.985-022-00155-6
  * [20] N.A. Larkin,  Regularity and decay of solutions for the 3D Kuramoto-Sivashinsky equation posed on smooth domains and parallelepipeds,  Electron J. Math., 3 (2022), 1-15; doi:10.47443/ejm.2022.002
  * [21] D. Massatt,  On the well-posedness of the anisotropically-reduced two-dimensional Kuramoto-Sivashinsky equation.  DCDS, serie B, https://doi.org/10.3934/dcdsb.2021305
  * [22] L. Molinet,  Local dissipativity in $L^{2}$ for the Kuramoto-Sivashinsky equation in spatial dimension 2,  J. Dynam. Diff. Eq., 12 (2000), N0 3, 533-556.
  * [23] B. Nicolaenko, B. Scheurer and R. Temam,  Some global dynamical properties of the Kuramoto-Sivashinsky equations: nonlinear stability and attractors,  Phys. D 16 (1985), No 3, 155-183
  * [24] G.R. Sell and M. Taboada,  Local dissipativity and attractors for the Kuramoto-Sivashinsky equation in thin 2D domains,  Nonlin. Anal. 18 (1992), 671-687.
  * [25] G.I. Sivashinsky,  Nonlinear analysis of hydrodynamic instability in laminar flames-1. Derivation of basic equations,  Acta Astronautica, 4 (1977), 1177-1206
  * [26] J.-S. Saut, R. Temam, C. Wang,  An initial and boundary-value problem for the Zakharov-Kuznetsov equation in a bounded domain,  J. Math. Phys., 53, 115612 (2012) doi:10.1063/1.4752102
  * [27] A.V. Steklov,  The problem of cooling of an heterogeneous rigid rod,  Communs. Kharkov Math. Soc., Ser. 2, 5 (1896) 136-181 (Russian).
  * [28] R. Temam,  Infinite-Dimensional Dynamical Systems in Mechanics and Physics,  Springer, Berlin-Heidelberg, New York (1988).
  * [29] J. Topper and T. Kawahara,  Approximate Equations for Long Nonlinear Waves on a Viscous Fluid,  J.Phys. Soc. of Japan,44 No 2,(1978), 663-666.
  * [30] V. E. Zakharov and E. A. Kuznetsov,  On three-dimensional solitons,  Sov. Phys. JETP 39 (1974), 285–286.
  * [31] Jing Li, Bing-Yu Zhang and Zhixiong Zhang,  A non-homogeneous boundary value problem for the Kuramoto-Sivashinsky equation in a quarter plane,  arXiv:1607.00506v2 [math.AP] 14 Jul 2016.


figurec [firstpage,color=gray!90,angle=0,scale=0.28,
xpos=0in,ypos=-5in]Submitted for publication to the Journal of Audio
Engineering Society on October 20th, 2022.

# Enhanced Fuzzy Decomposition of Sound Into Sines, Transients and Noise

Leonardo Fierro and Vesa Välimäki
<EMAIL_ADDRESS>|<EMAIL_ADDRESS>Acoustics Lab, Department of Signal Processing and Acoustics, Aalto University, Espoo, Finland

###### Abstract

The decomposition of sounds into sines, transients, and noise is a
long–standing research problem in audio processing. The current solutions for
this three–way separation detect either horizontal and vertical structures or
anisotropy and orientations in the spectrogram to identify the properties of
each spectral bin and classify it as sinusoidal, transient, or noise. This
paper proposes an enhanced three–way decomposition method based on fuzzy
logic, enabling soft masking while preserving the perfect reconstruction
property. The proposed method allows each spectral bin to simultaneously
belong to two classes, sine and noise or transient and noise. Results of a
subjective listening test against three other techniques are reported, showing
that the proposed decomposition yields a better or comparable quality. The
main improvement appears in transient separation, which enjoys little or no
loss of energy or leakage from the other components and performs well for test
signals presenting strong transients. The audio quality of the separation is
shown to depend on the complexity of the input signal for all tested methods.
The proposed method helps improve the quality of various audio processing
applications. A successful implementation over a state-of-the-art time-scale
modification method is reported as an example.

##  1 INTRODUCTION

Decomposing an audio signal into its sinusoidal, transient, and noise (STN)
components has been drawing research interest for over two decades [1, 2, 3,
4]. It is a widely used tool in a variety of audio processing applications,
ranging from beat tracking [5] and tonality estimation [6] to reduction of
spectral complexity in cochlear implants [7] and to virtual bass enhancement
[8]. The STN separation is also helpful in time-scale modification [1, 9, 10],
where it has been combined with the notion of fuzzy logic in order to improve
[4, 11] or evaluate the audio quality [12]. In all these audio applications,
it is helpful to process sine, transient, and noise components independently
of each other. This paper proposes improvements to the fuzzy STN decomposition
of audio signals.

The STN separation relies on the assumption that any audio signal can be
described as a linear combination of three independent actors: tonal content
(sines), impulsive events (transients), and a residual part (noise) that does
not belong to either one of the other two classes and adds nuance to the
sound. Historically, additive synthesis modeled any sound as a sum of
sinusoidal components [13, 14, 15]. Serra and Smith expanded the additive
synthesis method by introducing the noise class, which was obtained as a
residual after a sinusoidal model was subtracted from the original signal
[16]. In the resulting method—called spectral modeling synthesis [16]—the
frequency, amplitude, and phase of the sinusoidal components were estimated
from the short-time Fourier transform (STFT) using a method similar to the
McAulay-Quatieri algorithm [17].

The three-way decomposition was first introduced by Verma et al. [1, 18, 19],
who showed that including a third component for transients was greatly
beneficial in the context of signal analysis and synthesis, as it avoided the
smearing of transients, which was a weakness in sines + noise models. Levine
and Smith also showed that the adaptiveness of the STN model made it suitable
for audio compression and for pitch– and time–scale modification [20].

Fitzgerald discovered that it was possible to decompose an audio signal into
its sinusoidal and transient components by using spectral masks extracted via
horizontal and vertical median filtering of the STFT [21]. Driedger et al. [2]
reintroduced the three-way separation by updating Fitzgerald’s method: the
noise component could be obtained by retrieving spurious information after
extracting the other two components with median filtering.

Füg et al. [3] proposed a follow-up method involving the use of structure
tensors (ST) to find predominant orientation angles and anisotropy in the
time-frequency signal representation, showing an improvement in the separation
quality for sounds with vibrato.

Other recent approaches for sines–transients separation include a kernel
additive matrix [22], non-negative matrix factorization [23], improved
sinusoidal modeling [24, 25], and neural networks [26]. However, these methods
do not involve a third class for the noise component, hence they are not
discussed further in this paper.

It should be noted that the STN decomposition does not directly relate to
traditional source separation, which usually aims at retrieving musical
instruments in a sound mixture [27] or speech from noisy background sources
[28]. According to the STN formulation, even strongly percussive sources, such
as drums, will have a sinusoidal and noise component—unless they are perfect,
synthetic pulses—and, similarly, strongly-harmonical sources, such as the
violin, will hold, in addition to a sinusoidal part, a transient component in
their attack and a noise component to describe the nuances, such as the bowing
noise.

While both Driedger et al. [2] and Füg et al. [3] applied hard binary masks to
define the sinusoidal, transient, and noise classes, Damskägg and Välimäki [4]
introduced the concept of fuzzy logic in the context of time-scale
modification. The fuzzy classification (FZ) allows spectral bins to
simultaneously contribute to the three classes, providing a more refined basis
for the three-way separation [4]. This decomposition method was then extended
to objective evaluation by Fierro and Välimäki [12] and improved by Moliner et
al. [8] to allow perfect reconstruction by ensuring that the three soft
spectral masks sum up to unity.

This work proposes a novel way to estimate fuzzy soft masks for STN
decomposition of audio signals. The proposed method allows for intermediate
classifications of the spectral bins between two components—sines v. noise and
transients v. noise. This two-stage decomposition is shown to improve the
overall sound quality of the separated components, in particular for
transients. The masks ensure perfect reconstruction and are optimized for each
class to have a large constant region followed by a fast but smooth transition
to the adjacent class. The transition slope is refined for both decomposition
stages to provide the best separation quality.

The rest of this paper is structured as follows. Sec. 2 discusses previous
three-way separation techniques. Sec. 3 introduces the new STN decomposition
method, which optimally extracts the sinusoidal and transient components. Sec.
4 evaluates the proposed method against three previous techniques. Sec. 5
applies the proposed method to time-scale modification, and Sec. 6 concludes.

##  2 RELATED WORK

This section summarizes three previous STN decomposition methods based on a
spectrogram representation of the input signal. A spectrogram ${X}$ is a
$K$-by-$M$ matrix representing the time-frequency behavior of audio signal
$x$. Each element $X(m,k)$ is computed using the STFT:

$X(m,k)=\sum_{n=0}^{L-1}x(n+mH)\ w(n)\ e^{-j\omega_{k}n},$ (1)

where $n$ is the sample index, $m=0,1,2,...\ M-1$ is the temporal frame index,
$k=0,1,2,...\ K-1$ is the spectral bin index, $w$ is the analysis window, $H$
is the hop size, $L$ is the window length in samples, which is assumed to be
even, $j$ is the imaginary unit, and $\omega_{k}$ is the normalized central
frequency of the $k^{\text{th}}$ spectral bin.

Figure 1: Spectrogram of a test signal consisting of the castanets and the
violin playing simultaneously.

###  2.1 Harmonic–Percussive–Residual Separation

The Harmonic–Percussive–Residual (HPR) separation [2] builds upon the
Harmonic–Percussive (HP) method for sines–transients decomposition [21].
Fitzgerald noted that since sinusoids form flat lines in time direction in the
spectrogram and, vice versa, impulsive events appear as flat lines in the
frequency direction, they can be detected (suppressed) using a median filter
[21]. Fig. 1 shows the spectrogram of a signal consisting of a mixture of
violin and castanets, whose time– and frequency–direction ridges are
noticeable.

Horizontal (time-oriented) and vertical (frequency-oriented) median filtering
can be applied to the spectrogram $X(m,k)$ to highlight the desired component
and suppress the other [21]:

$X_{\textrm{h}}(m,k)\\\
=\textrm{med}\Big{[}|X(m-\frac{L_{\textrm{h}}}{2}+1,k)|,...,|X(m+\frac{L_{\textrm{h}}}{2},k)|\Big{]}$
(2)

and

$X_{\textrm{v}}(m,k)\\\
=\textrm{med}\Big{[}|X(m,k-\frac{L_{\textrm{v}}}{2}+1)|,...,|X(m,k+\frac{L_{\textrm{v}}}{2})|\Big{]},$
(3)

where $\textrm{med}[\cdot]$ is the median function, and $X_{\textrm{h}}$ and
$X_{\textrm{v}}$ are the resulting horizontally- and vertically-enhanced
magnitude spectrograms, respectively. Parameters $L_{\textrm{h}}$ and
$L_{\textrm{v}}$ are the median filter lengths (in samples) in the time and
frequency directions, respectively.

Matrices $X_{\textrm{h}}$ and $X_{\textrm{v}}$ are then used to extract the
tonalness $R_{\textrm{s}}$ and transientness $R_{\textrm{t}}$ matrices with
the following elements [21]:

$R_{\textrm{s}}(m,k)=\frac{X_{\textrm{h}}(m,k)}{X_{\textrm{h}}(m,k)+X_{\textrm{v}}(m,k)}$
(4)

and

$R_{\textrm{t}}(m,k)=1-R_{\textrm{s}}(m,k)=\frac{X_{\textrm{v}}(m,k)}{X_{\textrm{h}}(m,k)+X_{\textrm{v}}(m,k)},$
(5)

respectively. Fitzgerald [21] used $R_{\textrm{s}}$ and $R_{\textrm{t}}$
directly as spectral masks, whereas Driedger et al. [2] later introduced a
controllable separation factor $\beta$ and a third class (noise) to describe
those parts of the sound that are neither sines nor transients.

From Eqs. (4) and (5), a set of hard spectral masks $S$ (sinusoidal), $T$
(transient), and $N$ (noise) can be derived as follows [2]:

$S(m,k)=\begin{cases}1,&\mbox{if
}R_{\textrm{s}}(m,k)/R_{\textrm{t}}(m,k)>\beta\\\
0,&\mbox{otherwise,}\end{cases}$ (6) $T(m,k)=\begin{cases}1,&\mbox{if
}R_{\textrm{t}}(m,k)/R_{\textrm{s}}(m,k)>\beta\\\
0,&\mbox{otherwise,}\end{cases}$ (7)

and

$N(m,k)=1-S(m,k)-T(m,k).$ (8)

Their relationship for a chosen $\beta$ is shown in Fig. 2. The spectral masks
are then imposed on $X(m,k)$ to retrieve the three desired spectral
components:

$X_{\textrm{s}}=S\odot X,\qquad X_{\textrm{t}}=T\odot X,\qquad
X_{\textrm{n}}=N\odot X,$ (9)

where $\odot$ represents the Hadamard product, or element-wise multiplication.

Figure 2: Hard masks for transients, noise, and sines, as used in the HPR
method [2], for separation factor $\beta=2.5$.

It has been observed that the quality of the HPR separation largely varies for
the sinusoidal and the transient components depending on the choice of the
analysis window length $L$ [29, 30, 2]. A large window length $L$ for the
STFT, ensuring sufficient frequency resolution but poor time resolution,
results in a faultless extraction of sines but a low-quality transient output;
conversely, a smaller value of $L$ leads to a better extraction of the
transient component but a worse description of sines.

To overcome the time-frequency limitation, Driedger et al. [2] divided the
decomposition process can be divided into two cascaded iterations [2]. In the
first stage, a longer analysis window is applied to extract the sinusoidal
component [2], while transients and noise remain mixed together:

$x_{\textrm{s}}=\text{ISTFT}\Big{[}S_{1}\odot X\Big{]},$ (10)
$x_{\textrm{res}}=\text{ISTFT}\Big{[}(T_{1}+N_{1})\odot X\Big{]},$ (11)

where ISTFT is the Inverse STFT. Subsequently, the residual from the first
stage is separated again with shorter windowing, leading to the final
decomposition [2]:

$x_{\textrm{t}}=\text{ISTFT}\Big{[}T_{2}\odot X_{\textrm{res}}\Big{]},$ (12)
$x_{\textrm{n}}=\text{ISTFT}\Big{[}(S_{2}+N_{2})\odot
X_{\textrm{res}}\Big{]}.$ (13)

The noise signal $x_{\textrm{n}}$ will also contain residuals of sine
components, unless they were perfectly separated on the first stage. Fig. 3
shows the separated STN components of the example audio signal used above
obtained with the HPR method.

(a) Sines

(b) Transients

(c) Noise

Figure 3:  STN separation performed over the mixture of castanets and violin
using HPR method.

###  2.2 STN Separation Based on Structure Tensor

Füg et al. [3] noted that sounds exhibiting vibrato, which carry tonal
information and are perceived as sines, do not present strictly horizontal
structures in the spectrogram. This results in a leakage of energy in-between
different spectral components. The strictness of the median filtering can be
overcome using a structure tensor, a widely used tool in image processing, to
obtain a measure of the frequency change rate and local anisotropies in the
spectrogram, which will then be used as features to define the spectral masks
[3].

The structure tensor matrix is obtained from the partial derivatives of the
spectrogram with respect to time and frequency, and the orientation angles
$\alpha$ and the anisotropy $C$ of the spectral bins are computed from the
eigenvalues and the eigenvectors of such a matrix, as described in [3]. The
instantaneous frequency change rate $R$ is computed for each bin from the
orientation angles:

$R(m,k)=\frac{{f_{\textrm{s}}}^{2}}{HM}\tan{[\alpha(m,k)]},$ (14)

where $f_{\textrm{s}}$ is the sample rate. The spectral masks are then
obtained as follows:

$S(m,k)=\begin{cases}1,&\mbox{if }|R(m,k)|\leq r_{\textrm{s}}\ \wedge\
C(m,k)>c\\\ 0,&\mbox{otherwise,}\end{cases}$ (15)
$T(m,k)=\begin{cases}1,&\mbox{if }|R(m,k)|\geq r_{\textrm{t}}\ \wedge\
C(m,k)>c\\\ 0,&\mbox{otherwise,}\end{cases}$ (16)

(a) Sines

(b) Transients

(c) Noise

Figure 4: STN decomposition obtained using ST method, cf. Fig. 3.

where $c$ is the anisotropy threshold and $r_{\textrm{s}}$ and
$r_{\textrm{t}}$ are the frequency rate thresholds for the sinusoidal and the
transient component, respectively. The noise mask is computed as described in
Eq. (8), and the spectral components are then derived as in Eq. (9).

Fig. 4 shows the separated STN components of the example audio signal using
the ST method. Some differences can be observed in comparison to the
separation results of the HPR method in Fig. 3, such as some holes in the
transient events at low and middle frequencies in Fig. 4(b).

###  2.3 Fuzzy Separation

Damskägg and Välimäki [4] introduced the concept of fuzzy classification of
the spectral bins, which corresponds to a non–binary classification using
continuous values between 0 and 1. This method was later extended by Moliner
et al. [8] to ensure perfect reconstruction, i.e. all masks summing up to
unity. In [8], a third membership function for noisiness $R_{\textrm{n}}$ is
derived from Eqs. (4) and (5):

$R_{\textrm{n}}(m,k)=1-\sqrt{|R_{\textrm{s}}(m,k)-R_{\textrm{t}}(m,k)|}.$ (17)

The soft spectral masks are computed as

$S(m,k)=R_{\textrm{s}}(m,k)-\frac{1}{2}\ R_{\textrm{n}}(m,k),$ (18)
$T(m,k)=R_{\textrm{t}}(m,k)-\frac{1}{2}\ R_{\textrm{n}}(m,k),\\\ $ (19)

and

$N(m,k)=1-S(m,k)-T(m,k)=R_{\textrm{n}}(m,k).$ (20)

Their relationship is shown in Fig. 5. The spectral masks are once again
imposed on $X(m,k)$ to obtain the spectral components using the Hadamard
product, as in Eq. (9).

Fig. 6 shows the separated STN components of the example signal using the
fuzzy masks. The results are again slightly different from those obtained with
the two previous techniques, presented in Figs. 3 and 4. One apparent feature
is the leakage of energy from the other components to the transient component
at frequencies below about 5 kHz, shown in Fig. 6(b).

Figure 5: Transient, noise, and sinusoidal masks, as used in the FZ method,
which ensures perfect reconstruction [8].

(a) Sines

(b) Transients

(c) Noise

Figure 6: STN decomposition obtained using the FZ method, cf. Figs. 3 and 4.

##  3 PROPOSED METHOD

Fuzzy logic has proved useful in time-scale modification, where the separation
of the STN components using a soft mask leads to the state-of-the-art
performance [4, 11]. However, the mask for each component must be designed
carefully to obtain the best performance. For example, the FZ separation
proposed in [8] is suboptimal for this task, due to the leakage caused by the
secondary lobes of the $S$ and $T$ masks and the peaky behavior of the $N$
mask, which can be seen in Fig. 5. Ideally, a good fuzzy masking approach
guarantees perfect reconstruction, smooth transitions between classes, and a
well defined dominant region per each class. In this section, a novel method
comprising an extension to the HPR concept of clustered STN regions with soft
masks resulting from fuzzy classification is proposed to fulfill this target.

###  3.1 Prototype Soft Masking

A prototype function meeting all the aforementioned requirements is the
raised–cosine function, also known as the Hann window:

$w(n)=\sin^{2}(\pi\ n/L),\qquad 0\leq n<L.$ (21)

It is possible to take advantage of the symmetry of the raised–cosine
function, using only its one wing (appropriately shifted) to describe the
different transitions. The spectral masks for sines and transients can then be
obtained as follows:

$\displaystyle S(m,k)$
$\displaystyle=\begin{cases}\sin^{2}{[\pi(R_{\textrm{s}}(m,k)+\frac{1}{2})]},&\mbox{if
}R_{\textrm{s}}(m,k)\geq\frac{1}{2}\\\ 0,&\mbox{otherwise,}\end{cases}$ (22)
$\displaystyle T(m,k)$
$\displaystyle=\begin{cases}\sin^{2}{[\pi(R_{\textrm{s}}(m,k)-\frac{1}{2})]},&\mbox{if
}R_{\textrm{s}}(m,k)\leq\frac{1}{2}\\\ 0,&\mbox{otherwise}\end{cases}$

with $N(m,k)$ being computed according to Eq. (8). Their relationship is shown
in Fig. 7. While the masks defined in Eq. (22) already provide an audible
improvement over the FZ masks, they are affected by a strong leakage of sines
and transients into the noise component, suggesting that transitions between
adjacent masks should be stricter.

Figure 7: Prototype soft masks for transients, noise, and sines, as computed
from Eq. (22).

###  3.2 Improving Noise Classification

Damskägg and Välimäki [4] suggested that the tonalness distribution of pure
noise, e.g. white or pink noise, can be used to verify the shape of the noise
mask $N(m,k)$. In the following, a description of the noise distribution over
the tonalness and, consequently, sines-to-noise and transients-to-noise
transitions is experimentally identified.

(a)

(b)

Figure 8: Normalized tonalness distribution for median-filtered white noise
with (a) long and (b) short analysis windows.

A set of 100 instances of random white noise were generated and their
tonalness was computed, independently, with a long window ($185$ ms, or
$L=8192$ samples at 44.1 kHz) and a short window ($11$ ms or $L=512$ samples).
Normalized histograms for tonalness values are shown, respectively, in Figs.
8(a) and 8(b). A visual inspection indicates that the noise component remains
relevant for a large range of tonalness values around 0.5 before quickly
decaying in both directions. This suggests that mask transitions should be
much steeper than in FZ (cf. Fig. 5). The success of hard masking of HPR also
indicates that a single-classification dominant region should be included in
each mask.

It is also noted in Figs. 8(a) and 8(b) that the shape of the tonalness
distribution of noise is asymmetric for different window lengths. The peak
value is not centered at $R_{\textrm{s}}=0.5$, but shifts towards one side or
another depending on $L$. The two wings of the distribution have different
degrees of steepness: a sharper “sinusoidal” (right) side and a more relaxed
“transient” (left) side can be identified for longer $L$ (Fig. 8(a)); the
opposite behavior is exhibited for shorter $L$ (Fig. 8(b)). As for HPR, this
could lead to a two–stage decomposition featuring masks with different
transition regions.

###  3.3 Enhanced Soft Masking with Fuzzy Logic

Following the considerations discussed above, a new set of masks can be
derived by altering Eq. (22) to include dominant and cutoff regions for each
mask while retaining the smoothness of the raised cosine function for the
transition region. Parameters $\beta_{\textrm{U}}$ and $\beta_{\textrm{L}}$
are introduced to control the limits of the transition region and the bounds
for, respectively, the dominant (upper) region and the cutoff (lower) region.
The enhanced fuzzy masks are obtained as follows:

$S(m,k)=f\left(R_{\textrm{s}}(m,k)\right),\quad
T(m,k)=f\left(R_{\textrm{t}}(m,k)\right)$ (23)

where

$\displaystyle f($ $\displaystyle a)=\begin{cases}1,&\mbox{if
}a\geq\beta_{\textrm{U}}\\\
\sin^{2}{\Big{(}\dfrac{\pi}{2}\dfrac{a-\beta_{\textrm{L}}}{\beta_{\textrm{U}}-\beta_{\textrm{L}}}}\Big{)},&\mbox{if
}\beta_{\textrm{L}}\leq a<\beta_{\textrm{U}}\\\
0,&\mbox{otherwise}\end{cases}$ (24)

and $N(m,k)$ is computed using Eq. (8). The relationship of the proposed soft
masks is shown in Fig. 9, when $\beta_{\textrm{U}}=0.8$ and
$\beta_{\textrm{L}}=0.7$.

Figure 9: Proposed enhanced transient, noise, and sines fuzzy masks for STN
decomposition, $\beta_{\textrm{U}}=0.8$, $\beta_{\textrm{L}}=0.7$.

It has been shown earlier that, with hard masks, the two–stage STN
decomposition yields better results than a single–stage separation [2, 30].
The same concept can be extended to fuzzy masks. Following Eqs. (10) and (13),
two sets are obtained:
$\\{S_{1},T_{1},N_{1},\beta_{\textrm{U,1}},\beta_{\textrm{L,1}}\\}$ for sines
extraction with a large $L$, and
$\\{S_{2},T_{2},N_{2},\beta_{\textrm{U,2}},\beta_{\textrm{L,2}}\\}$ for the
residual transient and noise separation using a small $L$. The analysis
process is summarized in Fig. 10.

Figure 10: Block diagram of the proposed two–stage STN decomposition method.

###  3.4 Choosing the Transition Area

In order to find suitable values for the $\beta_{\textrm{U}}$ and
$\beta_{\textrm{L}}$ parameters of the two decomposition stages, an
optimization algorithm was run over the STN decomposition of a mixture of
synthetic sounds, each belonging almost perfectly to a single class: a sum of
sinusoids (for $S$), a short Gaussian
monopulse111$x_{\text{GP}}(t)=\sqrt{e}2\pi f_{c}te^{-2(\pi
f_{\textrm{c}}t)^{2}}$, where $f_{\textrm{c}}$ is the center frequency. (for
$T$), and a white noise sequence (for $N$). As the original sources are known,
the decomposition error can be evaluated for each class. The goal is not to
find a single optimal pair of values for the interval, as it is known that the
quality of STN decomposition greatly varies with different audio inputs [2,
4]. Instead, a range of tonalness values for each bound yielding a
small–enough decomposition error can be identified; the following analysis was
conducted in order to find a pair of quasi–optimal values that suits multiple
audio inputs.

The genetic algorithm [31] was chosen for the optimization process, which is
divided in two stages. The first optimization run narrows down a set of paired
bounds $B_{1}=\\{(\beta_{\textrm{U,1}},\beta_{\textrm{L,1}})_{i}\\}$ over the
$S_{1}$ mask by minimizing the decomposition error over the sinusoidal part.
Following that, different optimizations can be run by fixing a single pair
$\\{\beta_{\textrm{U,1}},\beta_{\textrm{L,1}}\\}$ from $B_{1}$ and finding its
optimal pair $\\{\beta_{\textrm{U,2}},\beta_{\textrm{L,2}}\\}$ over the
$T_{2}$ mask. Finally, an audible comparison over multiple separations using
the obtained sets determines the final quasi–optimal set
$\bar{B}=\\{(\beta_{\textrm{U,1}},\beta_{\textrm{L,1}}),(\beta_{\textrm{U,2}},\beta_{\textrm{L,2}})\\}$,
which ensures that the decomposition quality remains similar for different
audio inputs. The results of this quasi–optimal choosing process are reported
in Table 1.

Table 1: Transition area bounds for each decomposition stage used in this study. Stage | Decomposition | $\beta_{\textrm{U}}$ | $\beta_{\textrm{L}}$
---|---|---|---
1 | Sines v Residual | 0.8 | 0.7
2 | Transients v Noise | 0.85 | 0.75

Fig. 11 shows the separated STN components of the example audio signal using
the proposed method with the chosen set $\bar{B}$. The separation results
differ somewhat from the previous separation examples. In particular, the
transients in Fig. 11(b) are unbroken and the pauses between them are
practically free from leakage from the other components, as desired.

(a) Sines

(b) Transients

(c) Noise

Figure 11: STN decomposition of the castanets and violin using the proposed
method, cf. Figs. 3, 4, and 6.

##  4 EVALUATION

The audio quality of STN decomposition is typically degraded by inter-
component leakage, loss of tonality, loss of presence, or other artifacts,
e.g. musical noise [32]. In previous works, the separation quality was
evaluated by means of audio blind source separation performance assessment
metrics, such as SDR (Signal-to-Distortion Ratio), SIR (Signal-to-Interference
Ratio), and SAR (Signal-to-Artifacts Ratio) [2, 3, 33]. However, those metrics
require a mixture of three independent sources, one per class, that is
subsequently decomposed again for the separation quality assessment. This
prevents non-synthetic audio inputs, e.g. music or speech, from being tested:
unless dealing with perfectly tonal, impulsive, or noisy sources, the input
sounds themselves are composed of a mixture of unknown sinusoidal, transient,
and noise parts.

Informal subjective listening tests on STN separation have previously been
conducted by asking the participants whether the sinusoidal and transient
components extracted by the method under test met the expectation of
representing the sines and transients of the audio reference [2]. The STN
decomposition algorithm proposed in this work is evaluated against other
techniques by extending the same idea to a formal blind listening test,
involving experienced listeners as participants and asking them to rate the
quality of the sines and transients extraction for different STN methods.

###  4.1 Listening Test Design

A formal blind listening test was conducted on a selection of 19 experienced
listeners, 17 of which reported previous experience in test design. No
participant reported any hearing impairments or relevant medical conditions.
The test software was run on a machine running MacOS 10.14.6, using a single
pair of Sennheiser HD 650 headphones, inside a sound–proof listening booth at
the Aalto Acoustics Lab, Espoo, Finland.

A set of nine audio samples of short duration (4 to 6 s) was selected,
consisting of two synthetic sounds and seven musical excerpts from various
genres, featuring different spectral contents. The test samples are listed in
Table 2.

Table 2: Audio samples used in the listening test.

Name | Description
---|---
Synth | Synthetic mix of tones, pulses and white noise
CastViol | Solo violin and castanets, from [34]
ICanSee | Excerpt from I Can See Clearly, by Holly Cole Trio
Eddie | Excerpt from Early in the Morning, by E. Rabbit
Jazz | Mix of trumpet, piano, bass, and drums, from [34]
Vocals | Excerpt from Tom’s Diner, by Suzanne Vega
Vibrato | Synthetic mix of vibrato, pulses, and pink noise
Billie | Intro of Billie Jean, by Michael Jackson
Drum | Solo performed on a drum set, from [34]

(a)

(b)

Figure 12: Mean opinion scores and confidence intervals for (a) sines and (b)
transients separation of nine audio samples, showing that the proposed method
is the winner or among the best methods in almost all cases.

In each trial of the test, subjects were presented with one of the audio
excerpts, referred to as reference, and were asked to blindly rate the quality
of the extraction of the sound component under test (sines or transients,
respectively) from such a reference, for four different STN decomposition
methods: HPR, ST, FZ and the proposed one (PROP). The original reference was
also included among the samples under test, to provide a lower bound. Subjects
were asked to rate each sample on a scale from 0 to 100, with the specific
request of assigning 0 to the sample identified as the reference. As there was
no ground truth separation for any of the synthetic sounds, it is anticipated
that none of the samples was perceived to be an ideal decomposition: hence,
there was no obligation to rate any of the samples as 100 (full scale).

The test was divided in two parts consisting of nine trials each, the first
focusing on sines extraction and the second on transients extraction, for a
total of 18 trials and 90 audio samples under investigation. Listeners were
allowed a short training session before starting the actual test to get
acquainted with the interface, the keyboard shortcuts, and the task itself.
The results of the training session were not included in the statistical
analysis. Prior to the test, familiarity of the subjects with the concepts of
sound separation and sines, transients, and noise was asserted. The experiment
was conducted over WebMushra [35], although the designed test did not follow
the MUSHRA recommendation. The processed audio excerpts and the test software
are available at the companion webpage [36].

###  4.2 Results

(a)

(b)

(c)

(d)

(e)

(f)

(g)

(h)

(i)

Figure 13: Thresholded $p$–values resulting from the Wilcoxon signed-rank test
over sinusoidal separation data. Statistical significance (p
$\leq\alpha,\alpha=0.05$) is highlighted by coloring the cell.

(a)

(b)

(c)

(d)

(e)

(f)

(g)

(h)

(i)

Figure 14: Thresholded $p$–values of transient separation data, cf. Fig. 13.
Statistical significance (p $\leq\alpha,\alpha=0.05$) is highlighted by
coloring the cell.

Mean Opinion Scores (MOS) were computed from the ratings given by the subjects
to estimate the quality of the decomposition methods under test. Boxplots
displaying the distribution of the data for the sines and transients
extraction are shown, respectively, in Figs. 12(a) and 12(b). Data
distribution can also be observed via histograms, available at the companion
webpage [36].

Overall, the proposed method consistently performed better than or as well as
HPR for every audio excerpt, for both sinusoidal and transient decompositions.
The proposed methods achieved a median MOS score between 3.0 and 5.0 (“good”
to “excellent”) in all but one test case, see Figs. 12(a) and 12(b). The HPR
method was below 3.0 in two cases. The ST method generally scored intermediate
values, with the exception of Vibrato, where ST outperformed the other
separation methods: this was expected, considering that ST was designed
purposely for sound mixtures presenting vibrato. FZ was the lowest-ranked
algorithm with a significant difference in the subjective ratings from the
other three methods, as appears from Figs. 12(a) and 12(b).

Considering the sines extraction only, the proposed method performed quite
similarly to HPR in Fig. 12(a) and significantly better only for the Vocals
and Drum excerpts, where the fuzzy soft masking helped in the preservation of
the tonality variations. However, the median value for the proposed method was
consistently higher than that of the HPR method.

A larger improvement was observed in the transient decomposition in Fig.
12(b). Considering the median of the distributions, the proposed method
surpassed HPR for all excerpts but Icansee and Jazz. ST proved to be a
competitive algorithm for Vocals. The large amount of variance in the data
came from the absence of a proper separation “reference”, i.e. an upper limit
for the subjective grading in each trial. The subjects had to apply their own
scale during the grading process, which consequently lead to data which are
distributed in a non–Gaussian fashion. This was also confirmed by an
inspection of data skewness and kurtosis, which is reported in the companion
website [36].

Results were validated by conducting further statistical analysis and
determining whether there was statistical significance in the data
distribution, i.e. the difference between the distribution of data for
different methods had statistical significance. In this case, the observation
of non–Gaussian distributions called for a non–parametric paired difference
test. The Wilcoxon signed–rank test was chosen for the task. For a 95%
confidence interval, statistical significance was achieved if the signed–rank
test returned a $p$-value below the threshold $\alpha=0.05$. Thresholded
resulting $p$–values for sinusoidal and transient separation data are shown,
respectively, in Figs. 13 and 14.

The results showed that statistical significance for the difference in data
distribution from the proposed and the competitive methods was achieved for at
least one of the two components (sinusoidal or transient separation) for seven
excerpts out of nine, with the transient separation being the discriminant
factor in six cases out of seven. Icansee and Jazz were the two most complex
mixtures among the collected audio samples: this hinted that the more complex
the sound mixture, the harder it became to discriminate the separation
performance. The traditional statistical analysis via ANOVA and paired
$t$–test, carrying similar results, is reported in the companion website [36].

(a)

(b)

(c)

Figure 15:  Comparison between (a) a section of CastViol and its modifications
via (b) fuzzy phase vocoder (FPV) [4] and (c) its STN enhancement (PROP), for
a TSM factor of 2. Note the different time scales in the subfigures.

##  5 APPLICATION TO TIME STRETCHING

The proposed method is adapted to audio time-scale modification (TSM), which
is a suitable application for the STN decompositions [4, 12, 34]. For this
purpose, the fuzzy phase vocoder developed by Damskägg and Välimäki [4], which
received the highest average score in a recent comparison of audio time-
stretching methods [37], is modified to include the proposed decomposition
method. The refined separation allows for the transients to be preserved and
repositioned onto the stretched time axis [38], while the sinusoidal and noise
components are processed via the phase vocoder with identity phase locking and
phase randomization, respectively.

The enhanced TSM processing for a section of the CastViol excerpt that has
been slowed down to half speed is shown in Fig. 15. The sound processed with
the original fuzzy phase vocoder visibly suffers from transients smearing, a
recurring phenomenon in phase-vocoder-based TSM [39], With the STN-enhanced
version, the transients appear much sharper and better resemble the ones
visible in the original signal.

(a)

(b)

Figure 16:  Subject preferences between original fuzzy phase vocoder (FPV) [4]
and the STN modification (PROP), for TSM factors (a) 1.5 and (b) 2.

###  5.1 Comparison

A preference test was conducted to compare the original fuzzy phase vocoder
(FPV) [4] with the proposed STN-modified one (PROP), in order to evaluate the
enhancement brought by STN decomposition to a suitable method. Eleven
experienced listeners participated in the test, which was realized on the same
hardware and with similar modalities as the one described in Sec. 4.1.
Subjects were asked to listen to a reference sound and to select their
preferred time-stretched version in terms of sound quality from two options,
processed respectively with FPV and PROP. Four audio excerpts (CastViol,
Billie, and Icansee from the previous test, plus a Guitar plucking sound) were
time-stretched with factors 1.5 and 2. Loudness normalization was applied to
compensate for the non-perfect reconstruction of FPV. Results are visualized
in Fig. 16, and all test sounds are available on the companion webpage [36].

Test subjects showed a strong preference for PROP over all samples for both
time-stretching factors. A minority expressed a preference for FPV for
Icansee, mentioning that while the transients’ “punch" was well retained by
the other method (PROP), that created a dissonance with the noisy part of the
transient, which goes through the phase vocoder and phase randomization and is
heavily smeared. This hints that a different processing for the noise
component, which can now be isolated through the STN decomposition, could
further improve the audio time-stretching performance.

##  6 CONCLUSION

In this paper, the three-way sound decomposition into sines, transients, and
noise using fuzzy logic was enhanced. A set of soft spectral masks was derived
to fulfill the task while preserving the perfect reconstruction property.
Using such soft masks, the novel two–stage STN decomposition method proposed
in this paper allows a single spectral bin to be simultaneously classified
either as sine and noise, or as transient and noise. Soft masking positively
affects the decomposition by attenuating or removing common artifacts, e.g.
musical noise or loss of transient presence.

The results of a subjective listening test against three other methods showed
that the proposed decomposition method typically improves the separation
quality in terms of transient extraction, with a comparable performance for
sines extraction with the previous best method. It was also shown how the
complexity of the audio signal affects the quality of the decomposition. For
instance, the proposed separation method struggles when the sinusoidal part
contains vibrato, as does the competing previous method.

The proposed method can help improve sound quality in many audio processing
tasks. A successful application to audio time stretching was shown to improve
the performance of the state-of-the-art algorithm.

##  7 ACKNOWLEDGMENTS

This work belongs to the activities of the “Nordic Sound and Music Computing
Network—NordicSMC”, NordForsk project number 86892. The work of Leonardo
Fierro was funded by the Aalto ELEC Doctoral School. The authors are grateful
to Dennis Bontempi for the helpful discussions, and to Alec Wright for
proofreading.

## References

  * Verma and Meng [2000] T. S. Verma and T. H. Y. Meng.  Extending spectral modeling synthesis with transient modeling synthesis.  _Computer Music J._ , 24(2):47–59, 2000.
  * Driedger et al. [2014] J. Driedger, M. Müller, and S. Disch.  Extending harmonic-percussive separation of audio signals.  In _Proceedings of the International Conference on Music Infirmation Retrieval (ISMIR)_ , pages 611–616, Taipei, Taiwan, October 2014.
  * Füg et al. [2016] R. Füg, A. Niedermeier, J. Driedger, S. Disch, and M. Müller.  Harmonic-percussive-residual sound separation using the structure tensor on spectrograms.  In _Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , pages 445–449, Shanghai, China, March 2016.
  * Damskägg and Välimäki [2017] E.-P. Damskägg and V. Välimäki.  Audio time stretching using fuzzy classification of spectral bins.  _Appl. Sci._ , 7(12):1293, December 2017.
  * Gkiokas et al. [2012] A. Gkiokas, V. Katsouros, G. Carayannis, and T. Stajylakis.  Music tempo estimation and beat tracking by applying source separation and metrical relations.  In _Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , pages 421–424, Kyoto, Japan, March 2012.
  * Faraldo Pérez [2018] Á. Faraldo Pérez.  _Tonality Estimation in Electronic Dance Music: A Computational and Musically Informed Examination_.  PhD thesis, Universitat Pompeu Fabra, Barcelona, Spain, March 2018.
  * Lentz et al. [2020] B. Lentz, A. Nagathil, J. Gauer, and R. Martin.  Harmonic/percussive sound separation and spectral complexity reduction of music signals for cochlear implant listeners.  In _Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , pages 8713–8717, Barcelona, Spain, May 2020.
  * Moliner et al. [2020] E. Moliner, J. Rämö, and V. Välimäki.  Virtual bass system with fuzzy separation of tones and transients.  In _Proceedings of the International Conference on Digital Audio Effects (DAFx)_ , pages 86–93, Vienna, Austria, September 2020.
  * Nsabimana and Zölzer [2008] F. X. Nsabimana and U Zölzer.  Audio signal decomposition for pitch and time scaling.  In _Proceedings of the International Symposium on Communications, Control and Signal Processing (ISCCSP)_ , pages 1285–1290, St. Julians, Malta, June 2008.
  * Driedger et al. [2013] J. Driedger, M. Müller, and S. Ewert.  Improving time-scale modification of music signals using harmonic-percussive separation.  _IEEE Signal Process. Letters_ , 21(1):105–109, January 2013.
  * Roberts and Paliwal [2019] T. Roberts and K. K. Paliwal.  Time-scale modification using fuzzy epoch-synchronous overlap-add (FESOLA).  In _Proceedings of the IEEE Workshop Appl. Signal Process. Audio Acoustics (WASPAA)_ , pages 31–34, New Paltz, NY, USA, October 2019.
  * Fierro and Välimäki [2020] L. Fierro and V. Välimäki.  Towards objective evaluation of audio time-scale modification methods.  In _Proceedings of the Sound and Music Computing Conference (SMC)_ , pages 457–462, Malaga, Spain, June 2020.
  * Beauchamp [1966] J W. Beauchamp.  Additive synthesis of harmonic musical tones.  _J. Audio Eng. Soc._ , 14(4):332–342, Oct. 1966\.
  * Risset and Mathews [1969] J.-C. Risset and M. V. Mathews.  Analysis of Musical-Instrument Tones.  _Phys._ , 2(22):23–30, Feb. 1969.
  * Moorer [1977] J. A. Moorer.  Signal processing aspects of computer music: A survey.  _Proc. IEEE_ , 65(8):1108–1137, August 1977.
  * Serra and Smith III [1990] X. Serra and J. Smith III.  Spectral modeling synthesis: A sound analysis/synthesis system based on a deterministic plus stochastic decomposition.  _Computer Music J._ , 14(4):12–24, 1990.
  * McAulay and Quatieri [1986] R. J. McAulay and T. F. Quatieri.  Speech analysis/synthesis based on a sinusoidal representation.  _IEEE Trans. Acoust. Speech Signal Process._ , 34(4):744–754, Aug. 1986.
  * Verma et al. [1997] T. S. Verma, S. N. Levine, and T. H. Y. Meng.  Transient modeling synthesis: A flexible analysis/synthesis tool for transient signals.  In _Proceedings of the International Computer Music Conference (ICMC)_ , pages 164–167, Thessaloniki, Greece, September 1997.
  * Verma and Meng [1998] T. S. Verma and T. H. Y. Meng.  An analysis/synthesis tool for transient signals that allows a flexible sines+transients+noise model for audio.  In _Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , volume 6, pages 3573–3576, Seattle, WA, USA, May 1998.
  * Levine and Smith III [1998] S. N. Levine and J. O. Smith III.  A sines+transients+noise audio representation for data compression and time/pitch scale modifications, September 1998.
  * Fitzgerald [2010] D. Fitzgerald.  Harmonic/percussive separation using median filtering.  In _Proceedings of the International Conference on Digital Audio Effects (DAFx)_ , Graz, Austria, September 2010.
  * Fitzgerald et al. [2014] D. Fitzgerald, A. Liukus, Z. Rafii, B. Pardo, and L. Daudet.  Harmonic/percussive separation using kernel additive modelling.  In _Proceedings of the IET Irish Signals and Systems Conference (ISSC)_ , Limerick, Ireland, June 2014.
  * Canadas-Quesada et al. [2017] F. J. Canadas-Quesada, D. Fitzgerald, P. Vera-Candeas, and N. Ruiz-Reyes.  Harmonic-percussive sound separation using rhythmic information from non-negative matrix factorization in single-channel music recordings.  In _Proceedings of the International Conference on Digital Audio Effects (DAFx)_ , pages 276–282, Edinburgh, UK, September 2017.
  * Neri and Depalle [2018] J. Neri and P. Depalle.  Fast partial tracking of audio with real-time capability through linear programming.  In _Proceedings of the International Conference on Digital Audio Effects (DAFx)_ , pages 326–333, Aveiro, Portugal, September 2018.
  * Masuyama et al. [2019] Y. Masuyama, K. Yatabe, and Y. Oikawa.  Phase-aware harmonic/percussive source separation via convex optimization.  In _Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , pages 985–989, Brighton, UK, May 2019.
  * Drossos et al. [2018] K. Drossos, P. Magron, Stylianos I. Mimilakis, and T. Virtanen.  Harmonic-percussive source separation with deep neural networks and phase recovery.  In _Proceedings of the International Workshop on Acoustic Signal Enhancement (IWAENC)_ , pages 421–425, Tokyo, Japan, September 2018.
  * Rafii et al. [2018] Z. Rafii, A. Liutkus, F.-R. Stöter, S. I. Mimilakis, D. Fitzgerald, and B. Pardo.  An overview of lead and accompaniment separation in music.  _IEEE Trans. Speech Audio Process._ , 26(8):1307–1335, Aug. 2018.
  * Makino et al. [2007] S. Makino, T.-W. Lee, and H. Sawada.  _Blind Speech Separation_ , volume 615.  Springer, 2007.
  * Bonada [2000] Jordi Bonada.  Automatic technique in frequency domain for near-lossless time-scale modification of audio.  In _Proceedings of the International Computer Music Conference (ICMC)_ , Berlin, Germany, August 2000.
  * Tachibana et al. [2013] H. Tachibana, N. Ono, and S. Sagayama.  Singing voice enhancement in monaural music signals based on two-stage harmonic/percussive sound separation on multiple resolution spectrograms.  _IEEE Trans Audio Speech Lang. Process._ , 22(1):228–237, October 2013.
  * Mirjalili [2019] S. Mirjalili.  _Evolutionary Algorithms and Neural Networks_.  Springer, 2019.
  * Fierro and Välimäki [2021] L. Fierro and V. Välimäki.  SiTraNo: A MATLAB app for sines-transient-noise decomposition of audio signals.  In _Proceedings of the International Conference on Digital Audio Effects (DAFx)_ , Vienna, Austria, September 2021.
  * Vincent et al. [2006] E. Vincent, R. Gribonval, and C. Févotte.  Performance measurement in blind audio source separation.  _IEEE Trans. Audio Speech Lang. Process._ , 14(4):1462–1469, July 2006.
  * Driedger and Müller [2014] J. Driedger and M. Müller.  TSM Toolbox: MATLAB implementations of time-scale modification algorithms.  In _Proceedings of the International Conference on Digital Audio Effects (DAFx)_ , pages 249–256, Erlangen, Germany, September 2014.
  * Schoeffler et al. [2018] M. Schoeffler, S. Bartoschek, F.-R. Stöter, M. Roess, S. Westphal, B. Edler, and J. Herre.  WebMUSHRA–—A comprehensive framework for web-based listening tests.  _J. Open Res. Softw._ , 6(1), February 2018.
  * [36] L. Fierro and V. Välimäki.  Enhanced fuzzy decomposition of sound into sines, transients and noise: Companion web page.  http://research.spa.aalto.fi/publications/papers/ieee-tals-stn/main.htm.  Accessed: Oct. 19, 2022.
  * Roberts et al. [2021] Timothy Roberts, Aaron Nicolson, and Kuldip K. Paliwal.  Deep learning-based single-ended quality prediction for time-scale modified audio.  _J. Audio Eng. Soc._ , 69(9):644–655, Sep. 2021\.  https://doi.org/10.17743/jaes.2021.0031.
  * Nagel and Walther [2009] Frederik Nagel and Andreas Walther.  A novel transient handling scheme for time stretching algorithms, October 2009.  paper 7926.
  * Röbel [2003] Axel Röbel.  A new approach to transient processing in the phase vocoder.  In _Proceedings of the International Conference on Digital Audio Effects (DAFx)_ , pages 344–349, London, United Kingdom, September 2003.