cannot compete with the two other algorithms. In comparison, SigGB performs
worst on the tri1 benchmark example where it is about four times slower than
BB vanilla and about ten times slower than BB optimized (62 sec vs. 16 sec vs.
6 sec). For other examples, such as lv2-100, the timings are closer together
but still in favor of the classical Buchberger algorithm (60 sec vs. 43 sec
vs. 46 sec).

This is mainly because of two reasons. First of all, when using the F5
criterion, the number of checks that have to be done for each S-polynomial
increases quadratically with the size of the set $G^{(\Sigma)}$, which becomes
computationally quite intense as $G^{(\Sigma)}$ grows. Additionally, the fact
that we are restricted to regular $\operatorname{\mathfrak{s}}$-reductions in
Algorithm 2 requires an additionally check before each
$\operatorname{\mathfrak{s}}$-reduction. This cost also adds up for longer
computations.

We will investigate whether it is possible to improve the performance of
Algorithm 2 to obtain a competitive algorithm in practice. One step towards
achieving this goal could be finding ways to also allow non-fair module
orderings such as a position-over-term ordering. Additionally, future research
will be focused on adapting the concepts developed in this paper to the
noncommutative F4 algorithm.

We also plan to leverage the algorithms developed here to find short
representations of ideal elements. This is particularly useful when proving
operator identities, where such short representations correspond to short
proofs of the statement about operators. In particular, the effective
description of the syzygy module provided by a signature Gröbner basis might
allow to compute the _shortest_ proof of certain operator identities.

## Acknowledgements

The first author was supported by the Austrian Science Fund (FWF) grant
P32301. The second author was supported by the Austrian Science Fund (FWF)
grants P31571-N32 and P34872. The authors thank the anonymous referees for
their helpful suggestions which improved this work a lot, as well as Clemens
G. Raab and Georg Regensburger for their careful reading and their valuable
comments.

## References

  * AL [94] William Adams and Philippe Loustaunau.  An Introduction to Gröbner Bases.  Graduate Studies in Mathematics, 1994.
  * AP [11] Alberto Arri and John Perry.  The F5 criterion revised.  Journal of Symbolic Computation, 46(9):1017–1029, 2011.
  * Ber [78] George M. Bergman.  The diamond lemma for ring theory.  Advances in Mathematics, 29:178–218, 1978.
  * BK [06] Holger Bluhm and Martin Kreuzer.  Gröbner Basis Techniques in the Computation of Two-Sided Syzygies.  In Combinatorial Group Theory, Discrete Groups, and Number Theory: A Conference in Honor of Gerhard Rosenberger, December 8-9, 2004, Fairfield University: AMS Special Session on Infinite Groups, October 8-9, 2005, Bard College, volume 10, page 45. American Mathematical Soc., 2006.
  * Bok [76] L. A. Bokut’.  Embeddings into simple associative algebras.  Algebra i Logika, 15(2):117–142, 1976.
  * Buc [65] Bruno Buchberger.  Ein Algorithmus zum Auffinden der Basiselemente des Restklassenringes nach einem nulldimensionalen Polynomideal.  PhD thesis, University of Innsbruck, Austria, 1965.
  * BW [93] Thomas Becker and Volker Weispfenning.  Gröbner bases, volume 141 of Graduate Texts in Mathematics.  Springer-Verlag, 1993.  A computational approach to commutative algebra, In cooperation with Heinz Kredel.
  * CHRR [20] Cyrille Chenavier, Clemens Hofstadler, Clemens G. Raab, and Georg Regensburger.  Compatible rewriting of noncommutative polynomials for proving operator identities.  In Proceedings of the 45th International Symposium on Symbolic and Algebraic Computation, pages 83–90, 2020.
  * [9] Cyrille Chenavier, Arthur Léonard, and Tristan Vaccon.  On the difficulty of computing non-commutative signature Gröbner bases.  In preparation.
  * EF [17] Christian Eder and Jean-Charles Faugère.  A survey on signature-based algorithms for computing Gröbner bases.  Journal of Symbolic Computation, 80:719–784, 2017.
  * EP [11] Christian Eder and John Edward Perry.  Signature-based algorithms to compute Gröbner bases.  In Proceedings of the 36th international symposium on Symbolic and algebraic computation, pages 99–106, 2011.
  * Fau [99] Jean-Charles Faugère.  A new efficient algorithm for computing Gröbner bases $(F_{4})$.  Journal of Pure and Applied Algebra, 139(1-3):61–88, 1999.
  * Fau [02] Jean-Charles Faugère.  A new efficient algorithm for computing Gröbner bases without reduction to zero $(F_{5})$.  In Proceedings of the 2002 International Symposium on Symbolic and Algebraic Computation, pages 75–83 (electronic). ACM, 2002\.
  * Gal [85] André Galligo.  Some algorithmic questions on ideals of differential operators.  Lecture Notes in Computer Science, page 413–421, 1985.
  * GGV [10] Shuhong Gao, Yinhua Guan, and Frank Volny IV.  A new incremental algorithm for computing Gröbner bases.  In Proceedings of the 2010 International Symposium on Symbolic and Algebraic Computation, pages 13–19, 2010.
  * GVW [15] Shuhong Gao, Frank Volny IV, and Mingsheng Wang.  A new framework for computing Gröbner bases.  Mathematics of Computation, 85(297):449–465, 2015.
  * Hof [20] Clemens Hofstadler.  Certifying operator identities and ideal membership of noncommutative polynomials.  Master’s thesis, Johannes Kepler University Linz, Austria, 2020.  Available at https://epub.jku.at/obvulihs/content/titleinfo/5013051.
  * HRR [19] Clemens Hofstadler, Clemens G. Raab, and Georg Regensburger.  Certifying operator identities via noncommutative Gröbner bases.  ACM Communications in Computer Algebra, 53(2):49–52, 2019.
  * HSW [98] J. William Helton, Mark Stankus, and John J. Wavrik.  Computer simplification of formulas in linear systems theory.  IEEE Transactions on Automatic Control, 43(3):302–314, 1998.
  * HW [94] J. William Helton and John J. Wavrik.  Rules for computer simplification of the formulas in operator model theory and linear systems.  In Nonselfadjoint operators and related topics, pages 325–354. Springer, 1994.
  * Kin [14] Simon A. King.  A non-commutative F5 algorithm with an application to the computation of Loewy layers.  Journal of Symbolic Computation, 65:111–129, 2014.
  * LSL [09] Roberto La Scala and Viktor Levandovskyy.  Letterplace ideals and non-commutative Gröbner bases.  Journal of Symbolic Computation, 44(10):1374–1393, 2009.
  * LSZ [20] Viktor Levandovskyy, Hans Schönemann, and Karim Abou Zeid.  Letterplace: a Subsystem of Singular for Computations with Free Algebras via Letterplace Embedding.  In Proceedings of the 45th International Symposium on Symbolic and Algebraic Computation, pages 305–311, 2020.
  * Mor [85] Ferdinando Mora.  Gröbner bases for non-commutative polynomial rings.  In International Conference on Applied Algebra, Algebraic Algorithms, and Error-Correcting Codes, pages 353–362. Springer, 1985.
  * Mor [94] Teo Mora.  An introduction to commutative and noncommutative Gröbner bases.  Theoretical Computer Science, 134(1):131–173, 1994.
  * Mor [16] Teo Mora.  Solving Polynomial Equation Systems IV: Volume 4, Buchberger Theory and Beyond, volume 158.  Cambridge University Press, 2016.
  * RRHP [21] Clemens G. Raab, Georg Regensburger, and Jamal Hossein Poor.  Formal proofs of operator identities by a single formal computation.  Journal of Pure and Applied Algebra, 225(5):106564, 2021.
  * RS [02] Gerhard Rosenberger and Martin Scheer.  Classification of the finite generalized tetrahedron groups.  Contemporary Mathematics, 296:207–230, 2002.
  * RS [12] Bjarke Hammersholt Roune and Michael Stillman.  Practical Gröbner basis computation.  In Proceedings of the 37th International Symposium on Symbolic and Algebraic Computation, pages 203–210, 2012.  Full version including the appendix available at arXiv:1206.6940.
  * Shi [62] A. I. Shirshov.  Some algorithmic problems for Lie algebras.  Sibirsk. Mat. Z., 3:292–296, 1962.  (in Russian); English translation in SIGSAM Bull. 33(2):3–6 (1999).
  * SL [20] Leonard Schmitz and Viktor Levandovskyy.  Formally verifying proofs for algebraic identities of matrices.  In International Conference on Intelligent Computer Mathematics, pages 222–236. Springer, 2020.
  * SW [11] Yao Sun and Dingkang Wang.  Solving detachability problem for the polynomial ring by signature-based Gröbner basis algorithms.  arXiv preprint arXiv:1108.1301, 2011.
  * SWMZ [12] Yao Sun, Dingkang Wang, Xiaodong Ma, and Yang Zhang.  A Signature-Based Algorithm for Computing Gröbner Bases in Solvable Polynomial Algebras.  Proceedings of the 37th International Symposium on Symbolic and Algebraic Computation - ISSAC ’12, 2012.
  * Xiu [12] Xingqiang Xiu.  Non-commutative Gröbner bases and applications.  PhD thesis, University of Passau, Germany, 2012.  Available at http://www.opus-bayern.de/uni-passau/volltexte/2012/2682/.

##  Appendix A Detailed example

######  Example 56 (continues=infinite-calculations).

Recall that in Example 31 we considered the ideal
$I=(f_{1},f_{2},f_{3})\subseteq K\langle X\rangle$ with

$f_{1}=xyx-xy,\qquad f_{2}=yxy,\qquad f_{3}=xyy-xxy,$

over a field $K$ in the variables $X=\\{x,y\\}$. Furthermore, we used
$\preceq_{\textup{deglex}}$ where we ordered the indeterminates as
$x\prec_{\textup{lex}}y$ and used $\preceq_{\textbf{top}}$ as a module
ordering. We claimed that the set

$G^{[\Sigma]}=\\{f_{1}^{[\varepsilon_{1}]},f_{2}^{[\varepsilon_{2}]},f_{3}^{[\varepsilon_{3}]},f_{4}^{[\alpha]}\\}\cup\\{g_{n}^{[\gamma_{n}]}\mid
n\geq 0\\}$

with $f_{4}=xxy$, $g_{n}=yx^{n+2}y$ and certain $\alpha,\gamma_{n}\in\Sigma$
such that $\operatorname{\mathfrak{s}}(\alpha)=\varepsilon_{1}y$ and
$\operatorname{\mathfrak{s}}(\gamma_{n})=y\varepsilon_{3}y^{n}$, is a minimal
labelled Gröbner basis of $I^{[\Sigma]}$ w.r.t. the family of generators
$f_{1},f_{2},f_{3}$. We postponed the verification that $G^{[\Sigma]}$ is
indeed a labelled Gröbner basis. We finish this proof here using Theorem 39.
To this end, we compute all regular ambiguities of $G^{[\Sigma]}$ and regular
$\operatorname{\mathfrak{s}}$-reduce the respective S-polynomials. We have the
following regular ambiguities $a_{ij}$ between $f_{i}$ and $f_{j}$:

$\displaystyle a_{11}$
$\displaystyle=(xyxyx,xy,yx,f_{1}^{[\varepsilon_{1}]},f_{1}^{[\varepsilon_{1}]}),$
$\displaystyle a_{12}$
$\displaystyle=(xyxy,x,y,f_{1}^{[\varepsilon_{1}]},f_{2}^{[\varepsilon_{2}]}),$
$\displaystyle a_{13}$
$\displaystyle=(xyxyy,xy,yy,f_{1}^{[\varepsilon_{1}]},f_{3}^{[\varepsilon_{3}]}),$
$\displaystyle a_{14}$
$\displaystyle=(xyxxy,xy,xy,f_{1}^{[\varepsilon_{1}]},f_{4}^{[\alpha]}),$
$\displaystyle a_{21}$
$\displaystyle=(yxyx,y,x,f_{2}^{[\varepsilon_{2}]},f_{1}^{[\varepsilon_{1}]}),$
$\displaystyle a_{22}$
$\displaystyle=(yxyxy,yx,xy,f_{2}^{[\varepsilon_{2}]},f_{2}^{[\varepsilon_{2}]}),$
$\displaystyle a_{23}$
$\displaystyle=(yxyy,y,y,f_{2}^{[\varepsilon_{2}]},f_{3}^{[\varepsilon_{3}]}),$
$\displaystyle a_{32}$
$\displaystyle=(xyyxy,xy,xy,f_{3}^{[\varepsilon_{3}]},f_{2}^{[\varepsilon_{2}]}),$
$\displaystyle a_{41}$
$\displaystyle=(xxyx,x,x,f_{4}^{[\alpha]},f_{1}^{[\varepsilon_{1}]}),$
$\displaystyle a_{42}$
$\displaystyle=(xxyxy,xx,xy,f_{4}^{[\alpha]},f_{2}^{[\varepsilon_{2}]}),$
$\displaystyle a_{43}$
$\displaystyle=(xxyy,x,y,f_{4}^{[\alpha]},f_{3}^{[\varepsilon_{3}]}).$

The corresponding S-polynomials are

$\displaystyle\operatorname{sp}(a_{11})$
$\displaystyle=(-xyyx+xyxy)^{[\varepsilon_{1}yx-xy\varepsilon_{1}]},$
$\displaystyle\operatorname{sp}(a_{12})$
$\displaystyle=-xyy^{[\varepsilon_{1}y-x\varepsilon_{2}]},$
$\displaystyle\operatorname{sp}(a_{13})$
$\displaystyle=(-xyyy+xyxxy)^{[\varepsilon_{1}yy-xy\varepsilon_{3}]},$
$\displaystyle\operatorname{sp}(a_{14})$
$\displaystyle=-xyxy^{[\varepsilon_{1}xy-xy\alpha]},$
$\displaystyle\operatorname{sp}(a_{21})$
$\displaystyle=yxy^{[\varepsilon_{2}x-y\varepsilon_{1}]},$
$\displaystyle\operatorname{sp}(a_{22})$ $\displaystyle=0^{[\varepsilon_{2}xy-
yx\varepsilon_{2}]},$ $\displaystyle\operatorname{sp}(a_{23})$
$\displaystyle=yxxy^{[\varepsilon_{2}y-y\varepsilon_{3}]},$
$\displaystyle\operatorname{sp}(a_{32})$
$\displaystyle=-xxyxy^{[\varepsilon_{3}xy-xy\varepsilon_{2}]},$
$\displaystyle\operatorname{sp}(a_{41})$ $\displaystyle=xxy^{[\alpha
x-x\varepsilon_{1}]},$ $\displaystyle\operatorname{sp}(a_{42})$
$\displaystyle=0^{[\alpha xy-xx\varepsilon_{2}]},$
$\displaystyle\operatorname{sp}(a_{43})$ $\displaystyle=xxxy^{[\alpha
y-x\varepsilon_{3}]}.$

Of these S-polynomials, all but two regular
$\operatorname{\mathfrak{s}}$-reduce to 0. The two exceptions are
$\operatorname{sp}(a_{12})$, which regular
$\operatorname{\mathfrak{s}}$-reduces to $-xxy^{[\alpha_{12}]}$ with
$\alpha_{12}=\varepsilon_{1}y-x\varepsilon_{2}+\varepsilon_{3}$, and
$\operatorname{sp}(a_{23})$, which is already regular
$\operatorname{\mathfrak{s}}$-reduced. Note that
$\operatorname{\mathfrak{s}}(\alpha_{12})=\varepsilon_{1}y=\operatorname{\mathfrak{s}}(\alpha)$.
Therefore, $-xxy^{[\alpha_{12}]}$ is singular top
$\operatorname{\mathfrak{s}}$-reducible by $f_{4}^{[\alpha]}$. Similarly,
$\operatorname{sp}(a_{23})$ is singular top
$\operatorname{\mathfrak{s}}$-reducible by $g_{0}^{[\gamma_{0}]}$ since
$\operatorname{\mathfrak{s}}(\varepsilon_{2}y-y\varepsilon_{3})=y\varepsilon_{3}=\operatorname{\mathfrak{s}}(\gamma_{0})$.

Additionally, for each $n\geq 0$, we have eight regular ambiguities between
the $f_{i}$ and $g_{n}$. The S-polynomials of these ambiguities all regular
$\operatorname{\mathfrak{s}}$-reduce to 0 except for
$\operatorname{sp}(\tilde{a}_{n3})=yx^{n+3}y^{[\gamma_{n}y-yx^{n+1}\varepsilon_{3}]}$,
which comes from the overlap ambiguity

$\tilde{a}_{n3}=(yx^{n+2}yy,yx^{n+1},y,g_{n}^{[\gamma_{n}]},f_{3}^{[\varepsilon_{3}]})$

between $f_{3}^{[\varepsilon_{3}]}$ and $g_{n}^{[\gamma_{n}]}$ and which is
already regular $\operatorname{\mathfrak{s}}$-reduced. Note that
$\operatorname{sp}(\tilde{a}_{n3})$ is singular top
$\operatorname{\mathfrak{s}}$-reducible by $g_{n+1}^{[\gamma_{n+1}]}$ since
$\operatorname{\mathfrak{s}}(\gamma_{n}y-yx^{n+1}\varepsilon_{3})=\operatorname{\mathfrak{s}}(\gamma_{n}y)=y\varepsilon_{3}y^{n+1}=\operatorname{\mathfrak{s}}(\gamma_{n+1})$.

Finally, we also have the following regular ambiguity between $g_{i}$ and
$g_{j}$ for all $i,j\geq 0$:

$a^{\prime}_{ij}=(yx^{i+2}yx^{j+2}y,yx^{i+2},x^{j+2}y,g_{i}^{[\gamma_{i}]},g_{j}^{[\gamma_{j}]}).$

The respective S-polynomial is
$\operatorname{sp}(a^{\prime}_{ij})=0^{[\gamma_{i}x^{j+2}y-yx^{i+2}\gamma_{j}]}$.

So, all regular S-polynomials of $G^{[\Sigma]}$ regular
$\operatorname{\mathfrak{s}}$-reduce to $0$ or to a singular top
$\operatorname{\mathfrak{s}}$-reducible element. Hence, Theorem 39 yields that
$G^{[\Sigma]}$ is a labelled Gröbner basis of $I^{[\Sigma]}$ w.r.t. the family
of generators $f_{1},f_{2},f_{3}$.

Furthermore, we also claimed that the set

$G^{[\Sigma^{\prime}]}=\\{f_{1}^{[\varepsilon_{1}]},f_{2}^{[\varepsilon_{2}]},f_{3}^{[\varepsilon_{3}]},f_{4}^{[\varepsilon_{4}]}\\}\cup\\{yxxy^{[\gamma_{0}]}\\}$

with $\operatorname{\mathfrak{s}}(\gamma_{0})=y\varepsilon_{3}$ is a minimal
labelled Gröbner basis of $I^{[\Sigma^{\prime}]}$ w.r.t. the family of
generators $f_{1},f_{2},f_{3},f_{4}$. To see why this is the case, we note
that with $f_{4}$ now being a basis element, it has the signature
$\varepsilon_{4}$ instead of $\varepsilon_{1}y$. Consequently,
$f_{4}^{[\varepsilon_{4}]}$ can now be used to regular
$\operatorname{\mathfrak{s}}$-reduce the S-polynomial
$\operatorname{sp}(\tilde{a}_{03})=yxxxy^{[y\varepsilon_{3}y-yx\varepsilon_{3}]}$
to zero. Therefore, $G^{[\Sigma^{\prime}]}$ is a labelled Gröbner basis of
$I^{[\Sigma^{\prime}]}$. The minimality of $G^{[\Sigma^{\prime}]}$ follows
from the fact that $\varepsilon_{3}\prec\varepsilon_{4}$ as therefore neither
$f_{3}^{[\varepsilon_{3}]}$ nor $yxxy^{[\gamma_{0}]}$ is
$\operatorname{\mathfrak{s}}$-reducible by $f_{4}^{[\varepsilon_{4}]}$.


# Towards the Development of a Real-Time Deepfake Audio Detection System in
Communication Platforms

###### Abstract

Deepfake audio poses a rising threat in communication platforms, necessitating
real-time detection for audio stream integrity. Unlike traditional non-real-
time approaches, this study assesses the viability of employing static
deepfake audio detection models in real-time communication platforms. An
executable software is developed for cross-platform compatibility, enabling
real-time execution. Two deepfake audio detection models based on Resnet and
LCNN architectures are implemented using the ASVspoof $2019$ dataset,
achieving benchmark performances compared to ASVspoof $2019$ challenge
baselines. The study proposes strategies and frameworks for enhancing these
models, paving the way for real-time deepfake audio detection in communication
platforms. This work contributes to the advancement of audio stream security,
ensuring robust detection capabilities in dynamic, real-time communication
scenarios.

Index Terms: Deepfake audio detection, Communication platforms, Deep learning.

##  1 Introduction

Deepfake audio detection constitutes a crucial task, wherein the objective is
to discern authentic (real/bonafide) audio samples from counterfeit
(fake/spoof) ones through machine learning methodologies [1, 2]. Given the
progress in text-to-speech (TTS) [3] and voice cloning (VC) [4] technologies,
particularly in the context of voice-centric applications, it becomes
essential to address the potential threats posed by audio deepfake attacks.
This necessitates the implementation of effective countermeasures to ensure
the security of services in such applications. The predominant focus of
studies in deepfake audio detection has revolved around the realm of automatic
speaker verification (ASV) systems [5]. Notably, the ASVspoof challenge [5, 6,
7, 8] has served as a catalyst for numerous advanced research endeavors in
this domain.

Despite the promising strides in recent developments pertaining to deepfake
audio detection and countermeasure strategies, it is noteworthy that a
significant portion of these solutions has been conceived and assessed within
the confines of static audio recordings. Here, the term _static audio
recordings_ refers to datasets comprising audio samples lasting between
$2$-$10$ seconds, characterized by limited variations in acoustic backgrounds,
the number of speakers, encountered artifacts and recording conditions [9,
10]. In this work, we denote _static deepfake audio models_ to represent
models trained using static audio recordings. It is important to acknowledge
that these static deepfake audio models may not consistently exhibit robust
performance when deployed in real-time scenarios, such as continuous audio
streams coming from applications on a source computer device or similar
scenarios in communication platforms. Consider, for instance, a Teams group
call within an organization, which represents the latter case. The observed
decline in the performance of static deepfake models during such events can
primarily be attributed to their lack of awareness regarding dynamic
variations inherent in real-time conversational speech data within
communication platforms.

In this paper, the viability of employing static deepfake audio detection
models in real-time and continuous conversational speech scenarios across a
communication platform is systematically assessed. Two benchmark models are
initially developed, leveraging Resnet and LCNN architectures as per [11] and
[12] respectively. Moreover, we developed an executable software application
compatible with diverse operating system environments, streamlining the
deployment of our deepfake audio detection systems. For real-time testing of
our models in a communication platform, a new dataset was curated using actual
Teams meeting sessions. To enhance the resilience of these models for real-
time audio streams within a communication platform, diverse strategies are
proposed as part of future works. The subsequent sections describe the
methodology (Section 2), dataset and experimental details (Section 3), the
deepfake audio detection models (Section 4), the software platform (Section
5), and the results, discussion, and future work strategies (Section 6).

##  2 Methodology

A block diagram illustrating the components of a static deepfake audio
detection system, a real-time deepfake audio detection system implemented on a
source device, and a real-time deepfake detection system within a
communication platform is depicted in Figure 1. This study focuses on
assessing and exploring the application of static deepfake audio detection
models within a real-time audio communication platform. The methodology
employed in this study is summarized as follows.

  * •

Two static deepfake detection models are initially implemented, utilizing
Resnet [11] and LCNN [12] architectures. These models are trained on the
ASVspoof $2019$ challenge dataset [7].

  * •

The benchmarking of the deepfake models is conducted by comparing their
performance against the baseline model scores from the ASVspoof $2019$
challenge [7].

  * •

An executable software platform is developed to facilitate the deployment of
the deepfake audio detection model. The software is designed to be compatible
with various operating systems.

  * •

Evaluation of the static deepfake models in real-time detection scenarios is
carried out during our daily group meeting sessions on Microsoft Teams. This
assessment involves the analysis of the model’s performance in differentiating
between real and fake voices in audio stream data over a communication
platform. A block diagram view of the entire system is depicted in Figure 2.

This study aims to analyze the challenges and shortcomings associated with
employing a static deepfake audio model for real-time deepfake audio detection
on audio stream data within a communication platform. Building upon the
insights gained from this analysis, we put forth several strategies to
implement an effective real-time deepfake audio detection model tailored for
deployment in a communication platform.

Figure 1: A block diagram of various deepfake audio detection systems. Figure
2: A block diagram showing the typical use-case of our deepfake audio
detection system.

##  3 Dataset and Experimental Details

The implementation of our deepfake audio detection models relies on the
ASVspoof $2019$ challenge dataset [7], which encompasses both logical access
(LA) and physical access (PA) challenges. In the context of voice-based LA
attacks, attackers seek to trick a system by imitating or altering an
individual’s voice, aiming to gain unauthorized digital access, such as
unlocking a smartphone or accessing secure online information. These attacks
are commonly executed through synthetic voice generation methods, including
text-to-speech (TTS), voice cloning, and voice conversion technologies. The LA
challenge is designed to distinguish between genuine voices and synthetic
voices. On the other hand, voice-based PA attacks involve attackers to
manipulate or deceive a voice-based biometric system by replaying a recorded
real voice of an individual. The objective of the PA challenge is to
distinguish between authentic voices and replayed voices.

The challenge comes with distinct LA and PA datasets, each comprising $8$ male
and $12$ female speakers in both training and development subsets. In the LA
challenge, the training subset encompasses $2580$ bonafide and $22800$ spoofed
utterances, while the PA challenge includes $5400$ bonafide and $48600$
spoofed utterances. Evaluation sets for LA and PA consist of approximately
$80000$ and $135000$ test utterances, respectively.

The initial phase involves the implementation of a Resnet model, adapted from
[11], to distinguish between real and synthetic voices. This model is
exclusively trained using the LA dataset. Similarly, an LCNN model, adapted
from [12], is implemented to distinguish between real and replayed voices,
utilizing only the PA dataset. Subsequently, the performance of these models
is assessed using the evaluation subset of the PA and LA datasets. The
evaluation metrics employed for benchmarking include the tandem detection cost
function (t-DCF) [13] and the equal error rate (EER) [14]. Lower scores in
these metrics indicate better model performance.

The evaluation of our models focuses primarily on the ASVspoof $2019$
evaluation data. For real-time testing in a communication platform, we use
Microsoft Teams group meeting sessions and create a new dataset specifically
for this purpose, known as the _Teams Meeting dataset_.

The dataset is generated through the following procedure: we develop an
executable software application capable of running our deepfake audio
detection models. During Teams meeting sessions, our application is run on a
participant’s device, with another participant speaking while keeping their
cameras turned off. This arrangement ensures the collection of authentic voice
samples for testing the model predictions. To introduce synthetic voice
events, we used voice cloning and voice conversion systems. A moderator is
assigned to instruct a speaker to conduct either a real or a fake trial,
allowing us to note the reference annotations associated with each trial. The
topics for the trials include reading exercises and conversational speech
between two speakers on selected topics. Each trial of a speaker lasts for
$10$ seconds, and a total of $10$ individuals participate in the experiment,
each contributing $5$ real and fake voice events from the reading exercises.
The number of instances created using the conversational speech setup for real
and fake scenarios is not equal.

To avoid scenarios where multiple speakers speak simultaneously (cocktail
party scenario), we organized individuals participation in the experiment
carefully with proper time-slot arrangement. Throughout the $10$-second voice
trail, model predictions are aggregated, and a majority voting scheme is
employed to classify the event as either real or fake. This approach enables
us to evaluate the individual accuracy of our static LA model (ResNet model)
and PA model (LCNN model) for predicting real-time audio stream data over a
communication platform.

##  4 Model pipeline and Training

The developmental and evaluation stages of our deepfake audio detection
systems follow a sequential pipeline, as depicted in Figure 3. The training
and development phase initiates with the construction of an acoustic model,
which learns a mapping function between audio samples and their reference
annotations. Raw audio samples undergo preprocessing in the feature extraction
unit, extracting compressed and information-rich audio feature
representations. Several audio feature representations, such as mel-
spectrogram cepstral coefficient (MFCC), power-spectrogram, and mel-
spectrogram features, are empirically experimented with for the development of
our deepfake audio model. Ultimately, mel-spectrogram features are selected
for the LA model, while power-spectrogram features are chosen for the PA
model. The label encoding unit transforms textual reference annotations
(real/fake) into a numeric format, utilizing binary encoding as the label
encoding function. In the detection phase, a threshold value, corresponding to
the minimum EER computed using the validation dataset, is applied to the
probability score from the model. This threshold aids in identifying whether a
given audio sample is classified as real or fake.

Our LA model is constructed using an adapted version of the Resnet model
architecture from [11], while our PA model employs an adapted version of the
LCNN model architecture as presented in [12]. Training our models involves
utilizing audio samples of a $3$-second duration. To standardize input
lengths, trimming and padding techniques are applied to long and short audio
samples in the LA and PA datasets, resulting in $3$-second audio samples. For
the LA model, a feature representation of $80$ log mel-bands spectrogram is
employed. This feature is extracted using a short-term Fourier transform
(STFT) with a fast Fourier transform (FFT) window of $512$, a hop length of
$160$, and a sample rate of $16$ kHz. On the other hand, the PA model uses a
power-spectrogram feature representation, extracted with the same feature
settings as above, but without the mel-transformation.

For both the PA and LA models, each convolutional layer activation undergoes
batch normalization and is subject to regularization with dropout (probability
= $0.3$). The weights of the convolutional layers are initialized using a
Glorot uniform distribution [15]. Training for each model spans $100$ epochs,
employing a binary cross-entropy loss function, as defined in equation (1),
and the Adam optimizer with a learning rate set at $0.001$. To mitigate
overfitting, early stopping is implemented, with termination set after $10$
epochs based on the validation loss score.

Figure 3: Pipeline of a deepfake audio detection system.

$\begin{split}\mathcal{L}&=-\sum_{n=1}^{N}{(y_{n}\log(p_{n})+(1-y_{n})\log(1-p_{n}))}\end{split}$
(1)

In Equation (1) $\mathcal{L}$ is the binary cross-entropy loss function, $N$
is the total number of audio samples in the training dataset and $y_{n}$ and
$p_{n}$ respectively denotes the true label and predicted probability for the
$n^{th}$ audio sample.

##  5 Software application development

We have developed a software application designed for use alongside popular
conferencing platforms such as Teams and Zoom, aimed at enhancing the
integrity of audio interactions during remote communications. Python serves as
the chosen programming language for this initiative, selected for its
versatility. The ecosystem of libraries and frameworks in Python provides an
ideal environment for implementing AI and advanced audio processing
techniques. Our program architecture adheres to the MoSCoW rules, ensuring a
selection of core functionalities. A flow diagram of the complete development
process in shown in Figure 4.

Figure 4: Flow diagram of the software application development process.

The graphical user interface (GUI) is designed, incorporating a secure login
mechanism and a multifunctional main screen. On the backend, the program
orchestrates the automated recording of audio from default speakers, real-time
analysis of audio clips, and the utilization of an AI model for classification
of audio authenticity. The key functionalities of our application are the
following.

  * •

GUI Interface: A user interface crafted for intuitive interaction,
encompassing a secure login screen and a main interface with essential
functionalities.

  * •

Automated Audio Recording: Designed to autonomously capture audio clips from
default speakers in real-time during a video conference session.

  * •

AI Enabled Deepfake Audio Detection Models: The pinnacle of our technological
innovation, an intricately crafted AI model leverages advanced techniques to
discern between authentic and fake audio streams.

  * •

Real-time Notifications: An Operation System level notification system to
promptly inform the user of the results.

  * •

Autonomous Operation: Equipped with automated processes, the program
consistently maintains its operations after initiation, ensuring continuous
functionality.

As illustrated in Figure 4, the codebase of our software program is
distinguished by structuring into modular components. This flow diagram shows
the architecture of the Python application that involves audio recording,
processing, and system notifications. It uses various libraries such as
PySimpleGUI, Soundcard, Librosa, Plyer, and TensorFlow to perform different
tasks within the application. The diagram illustrates the main components and
methods of the _MainApplication class_ , as well as the interactions and
dependencies among the libraries.

Each module serves a specific purpose, encapsulating functionalities for audio
recording, preprocessing, prediction, and GUI interactions. Threading
intricacies have been employed to enable parallel execution, aligning with
industry-best practices for code readability and maintainability. This
structure enhances the clarity of the codebase and also promotes scalability
and ease of future development. We test our system in stages through trial and
error, checking each part to confirm it works correctly, including testing on
different platforms.

##  6 Results and Discussion

In this section, experimental results are presented and analyzed using the
ASVspoof $2019$ evaluation data and our Teams Meeting dataset. It is crucial
to note that our models are trained solely using the ASVspoof $2019$
development dataset, and the Teams Meeting dataset is exclusively used for
real-time testing of our models.

###  6.1 ASVspoof 2019 Evaluation data

The deepfake audio detection performance of our LA and PA models on the
ASVspoof $2019$ evaluation data is summarized in Table 1. The baseline models,
denoted as Baseline 1 and Baseline 2 in Table 1, correspond to the baseline
models associated with the ASVspoof $2019$ challenge [7]. The results indicate
that our LA model achieves an EER of $7.39$, outperforming both baseline
models. Additionally, the LA model demonstrates a t-DCF score of $0.215$,
closely aligning with the score of the Baseline 1 model. Similarly, our PA
model exhibits an EER of $4.38$ and a t-DCF score of $0.121$, surpassing the
baseline model scores. These findings underscore the effectiveness of our
static PA and LA models in distinguishing between real and fake voices within
the context of the ASVspoof $2019$ dataset.

###  6.2 Teams Meeting dataset

Table 2 provides an overview of the performance metrics for the LA and PA
models when deployed in real-time audio stream data during Microsoft Teams
meetings. The metrics for evaluation include precision (P), recall (R), and
F-score (F), calculated based on the definitions provided in the equation (2),
where $TP$, $FP$, and $FN$ represent true positive, false positive, and false
negative rates, respectively. The precision values indicate the proportion of
correctly identified positive instances among all instances predicted as
positive. In this context, the LA model outperforms the PA model with a higher
precision of $0.48$ compared to $0.39$, suggesting a better ability to
correctly identify positive instances. Similarly looking at the recall scores,
it can be identified that the PA model is more prone to false negative
predictions.

$P=\frac{TP}{TP+FP},\quad R=\frac{TP}{TP+FN},\quad F=\frac{2PR}{P+R}\\\ $ (2)

The results reveal that the models exhibit poor performance in real-time
communication platform scenarios compared to their performance on the ASVspoof
$2019$ evaluation data. This underperformance can be attributed to the static
nature of the audio data used for training the models, which contrasts with
the dynamic nature of real-time audio stream data. The ASVspoof $2019$
dataset’s training and development subsets share similar spoofing algorithms
and conditions for both the LA and PA datasets. We assume that training on the
same types of spoofing attacks may lead to overfitting and limited
generalization on unseen attack conditions. Additionally, the dataset’s
limited variations in terms of the number of speakers, acoustic conditions,
and recording conditions contribute to poor generalization on real-time audio
stream data.

To address these issues and enhance the robustness of our deepfake audio
detection models for real-time communication platforms, we propose the
following strategies.

###  6.3 Recommended Strategies

####  6.3.1 Training Data Variability

To address real-time communication nuances, augment the ASVspoof $2019$
dataset with additional conversational, noisy, and fragmented speech data is
an immediate first step. Incorporating datasets like the _Spotify podcast_
collection [16], _FoR_ [17], _WaveFake_ [18], _LRPD_ [19], _In-the-wild_ [20],
and _VoxCeleb2_ [21] can introduce more diverse acoustic scenarios, enhancing
the model’s adaptability to real-world communication platform conditions. We
plan to investigate the effects of training with these datasets collectively
in our future work.

####  6.3.2 Data Augmentation

Develop novel data augmentation methods tailored to the specific variations
introduced by communication platforms. This may involve simulating platform-
specific artifacts, such as network-induced delays, packet loss, or
compression artifacts. These techniques aim to improve the model’s robustness
to the unique challenges posed by real-time audio streams in communication
environments. In our future work, we plan to explore a data augmentation
strategy that focuses on understanding the impact of data compression in the
communication platform.

####  6.3.3 Generative Model Approaches

Explore the integration of generative models to synthesize additional training
samples. By leveraging generative models like variational autoencoder (VAE)
[22] or similar approaches, synthetic data can be generated, providing the
model with a more extensive set of examples for distinguishing between genuine
and deepfake audio in real-time scenarios. In our future work, we plan to
design and implement a generative model-augmented classifier for real-time
deepfake detection, inspired by the approach proposed in [23].

Table 1: Evaluation performance of the LA and PA model using ASVspoof $2019$ evaluation dataset. Model | LA | PA
---|---|---
EER | t-DCF | EER | t-DCF
Baseline 1 | 8.09 | 0.211 | 13.54 | 0.301
Baseline 2 | 9.57 | 0.236 | 11.04 | 0.245
LA | 7.39 | 0.215 | - | -
PA | - | - | 4.38 | 0.121
Table 2: Evaluation performance of the LA and PA model on real-time audio stream data in Teams. Model | Precision | Recall |  $F$-score
---|---|---|---
PA | 0.39 | 0.41 | 0.40
LA | 0.48 | 0.42 | 0.45

##  7 Conclusion

This study aimed to develop a real-time deepfake audio detection system for
communication platforms. Two static deepfake audio detection models were
implemented using the ASVspoof $2019$ dataset. Subsequently, a software
application was created to assess the models performance in real-time
communication, specifically tested in Teams meeting sessions. The
investigation highlighted challenges associated with deploying static deepfake
detection models in real-time communication platforms. The study recommends
future work to focus on developing an efficient deepfake audio detection model
for real-time prediction in communication platforms.

## References

  * [1] Z. Almutairi and H. Elgibreen, “A review of modern audio deepfake detection methods: challenges and future directions,” _Algorithms_ , vol. 15, no. 5, p. 155, 2022.
  * [2] S. Lyu, “Deepfake detection: Current challenges and next steps,” in _2020 IEEE international conference on multimedia & expo workshops (ICMEW)_. IEEE, 2020, pp. 1–6.
  * [3] N. Kaur and P. Singh, “Conventional and contemporary approaches used in text to speech synthesis: A review,” _Artificial Intelligence Review_ , vol. 56, no. 7, pp. 5837–5880, 2023.
  * [4] S. Arik, J. Chen, K. Peng, W. Ping, and Y. Zhou, “Neural voice cloning with a few samples,” _Advances in neural information processing systems_ , vol. 31, 2018.
  * [5] Z. Wu, T. Kinnunen, N. Evans, J. Yamagishi, C. Hanilçi, M. Sahidullah, and A. Sizov, “Asvspoof 2015: the first automatic speaker verification spoofing and countermeasures challenge,” in _Sixteenth annual conference of the international speech communication association_ , 2015.
  * [6] T. Kinnunen, M. Sahidullah, H. Delgado, M. Todisco, N. Evans, J. Yamagishi, and K. A. Lee, “The asvspoof 2017 challenge: Assessing the limits of replay spoofing attack detection,” _Interspeech 2017_ , 2017.
  * [7] M. Todisco, X. Wang, V. Vestman, M. Sahidullah, H. Delgado, A. Nautsch, J. Yamagishi, N. Evans, T. H. Kinnunen, and K. A. Lee, “Asvspoof 2019: Future horizons in spoofed and fake audio detection,” _Interspeech 2019_ , 2019.
  * [8] J. Yamagishi, X. Wang, M. Todisco, M. Sahidullah, J. Patino, A. Nautsch, X. Liu, K. A. Lee, T. Kinnunen, N. Evans _et al._ , “Asvspoof 2021: accelerating progress in spoofed and deepfake speech detection,” in _ASVspoof 2021 Workshop-Automatic Speaker Verification and Spoofing Coutermeasures Challenge_ , 2021.
  * [9] J. Yamagishi, C. Veaux, and K. MacDonald, “CSTR VCTK Corpus: English multi-speaker corpus for CSTR voice cloning toolkit (version 0.92),” 2019.
  * [10] K. Ito and L. Johnson, “The LJ speech dataset,” https://keithito.com/LJ-Speech-Dataset/, 2017.
  * [11] M. Alzantot, Z. Wang, and M. B. Srivastava, “Deep residual neural networks for audio spoofing detection,” _arXiv preprint arXiv:1907.00501_ , 2019.
  * [12] A. Nautsch, X. Wang, N. Evans, T. H. Kinnunen, V. Vestman, M. Todisco, H. Delgado, M. Sahidullah, J. Yamagishi, and K. A. Lee, “Asvspoof 2019: spoofing countermeasures for the detection of synthesized, converted and replayed speech,” _IEEE Transactions on Biometrics, Behavior, and Identity Science_ , vol. 3, no. 2, pp. 252–265, 2021.
  * [13] T. Kinnunen, K. A. Lee, H. Delgado, N. Evans, M. Todisco, M. Sahidullah, J. Yamagishi, and D. A. Reynolds, “t-dcf: a detection cost function for the tandem assessment of spoofing countermeasures and automatic speaker verification,” _arXiv preprint arXiv:1804.09618_ , 2018.
  * [14] J.-M. Cheng and H.-C. Wang, “A method of estimating the equal error rate for automatic speaker verification,” in _2004 International Symposium on Chinese Spoken Language Processing_. IEEE, 2004, pp. 285–288.
  * [15] X. Glorot and Y. Bengio, “Understanding the difficulty of training deep feedforward neural networks,” in _Proceedings of the thirteenth international conference on artificial intelligence and statistics_. JMLR Workshop and Conference Proceedings, 2010, pp. 249–256.
  * [16] A. Clifton, S. Reddy, Y. Yu, A. Pappu, R. Rezapour, H. Bonab, M. Eskevich, G. Jones, J. Karlgren, B. Carterette _et al._ , “100,000 podcasts: A spoken english document corpus,” in _Proceedings of the 28th International Conference on Computational Linguistics_ , 2020, pp. 5903–5917.
  * [17] R. Reimao and V. Tzerpos, “For: A dataset for synthetic speech detection,” in _2019 International Conference on Speech Technology and Human-Computer Dialogue (SpeD)_. IEEE, 2019, pp. 1–10.
  * [18] J. Frank and L. Schönherr, “Wavefake: A data set to facilitate audio deepfake detection,” in _Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2)_ , 2021.
  * [19] I. Yakovlev, M. Melnikov, N. Bukhal, R. Makarov, A. Alenin, N. Torgashov, and A. Okhotnikov, “Lrpd: Large replay parallel dataset,” in _ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_. IEEE, 2022, pp. 6612–6616.
  * [20] B. Zi, M. Chang, J. Chen, X. Ma, and Y.-G. Jiang, “Wilddeepfake: A challenging real-world dataset for deepfake detection,” in _Proceedings of the 28th ACM international conference on multimedia_ , 2020, pp. 2382–2390.
  * [21] J. S. Chung, A. Nagrani, and A. Zisserman, “Voxceleb2: Deep speaker recognition,” _Interspeech 2018_ , 2018.
  * [22] D. P. Kingma, M. Welling _et al._ , “An introduction to variational autoencoders,” _Foundations and Trends® in Machine Learning_ , vol. 12, no. 4, pp. 307–392, 2019.
  * [23] B. Chettri, T. Kinnunen, and E. Benetos, “Deep generative variational autoencoding for replay spoof detection in automatic speaker verification,” _Computer Speech & Language_, vol. 63, p. 101092, 2020.