# High Chern numbers in a perovskite-derived dice lattice
(La$X$O3)3/(LaAlO3)3(111) with $X=$ Ti, Mn and Co

Okan Köksal  Department of Physics and Center for Nanointegration Duisburg-
Essen (CENIDE), University of Duisburg-Essen, Lotharstr. 1, 47057 Duisburg,
Germany  L. L. Li  Department of Physics and Center for Nanointegration
Duisburg-Essen (CENIDE), University of Duisburg-Essen, Lotharstr. 1, 47057
Duisburg, Germany  Rossitza Pentcheva  Department of Physics and Center for
Nanointegration Duisburg-Essen (CENIDE), University of Duisburg-Essen,
Lotharstr. 1, 47057 Duisburg, Germany

###### Abstract

The dice lattice, containing a stack of three triangular lattices, has been
proposed to exhibit nontrivial flat bands with nonzero Chern numbers, but
unlike the honeycomb lattice it is much less studied. By employing density-
functional theory (DFT) calculations with an on-site Coulomb repulsion term,
we explore systematically the electronic and topological properties of
(La$X$O3)3/(LaAlO3)3(111) superlattices with $X=$ Ti, Mn and Co, where a
LaAlO3 trilayer spacer confines the La$X$O3 (L$X$O) dice lattice. In the
absence of spin-orbit coupling (SOC) with symmetry constrained to P3, the
ferromagnetic (FM) phase of the L$X$O(111) trilayers exhibits a distinct spin-
polarized half-metallic state with multiple Dirac crossings and coupled
electron-hole pockets around the Fermi energy. Symmetry lowering induces a
significant rearrangement of bands and triggers a metal-to-insulator
transition. Inclusion of SOC leads to a substantial anomalous Hall
conductivity (AHC) around the Fermi energy reaching values up to
$\sim-3e^{2}/h$ for $X=$ Mn and Co in P3 symmetry and both in- and out-of-
plane magnetization directions in the first case and along [001] in the
latter. The dice lattice emerges as a promising playground to realise
nontrivial topological phases with high Chern numbers.

##  I Introduction

Transition metal oxides (TMO) comprise a class of materials where electronic
correlation and the interplay of charge, spin, orbital, and lattice degrees of
freedom can lead to fascinating properties ranging from magnetism to
superconductivity [1, 2]. Precise control of the layer thickness, growth
orientation, and epitaxial strain of TMO heterostructures provides essential
degrees of freedom to tune their functional properties [3]. Among the TMO
heterostructures, perovskite superlattices (SLs) have developed into an
excellent platform to explore interface- and confinement-induced phenomena
such as interfacial charge transfer, conductivity, magnetism, electronic
reconstruction and metal-to-insulator transition [4, 5, 6]. In particular,
(111)-oriented SLs have drawn attention due to the possibility to engineer
quantum Hall states [7], with view of applications in low-power electronics.
Along the [111]- direction LaO3 and $X$ layers alternate in La$X$O3, thereby,
two $X$ triangular lattices form a buckled honeycomb lattice, which is
topologically analogous to graphene. Model Hamiltonian studies in conjunction
with DFT calculations predicted a distinct set of four bands in (111) bilayers
of LaNiO3, two nearly flat interconnected by two dispersive ones with a
(Dirac) crossing at K and a quadratic band touching at $\Gamma$ [8, 9, 10,
11]. Systematic DFT+$U$ calculations have shown that this set of bands occurs
also for La$X$O3 ($X=$ Mn and Co [12]) and a Chern insulator phase driven by
spin-orbit coupling (SOC) can emerge in (111) bilayers of La$X$O3 ($X=$ Mn
[13, 12], Co [12], Ru, Os [14], Pd, and Pt [15, 16]). The (111)-oriented
LaMnO3 buckled honeycomb bilayer was predicted to have a nontrivial band gap
of $\sim$ 150 meV and a quantized Hall conductivity of $-e^{2}/h$ [12]. On the
other hand, the Chern insulating phases are often unstable with respect to
symmetry breaking that leads to trivial Mott insuling ground states, albeit
with properties distinct from the bulk compounds [12]. In the meantime, the
growth of (111)-oriented superlattices has been successfully demonstrated,
thus enabling the exploration of such exotic phases [17, 18, 19, 20, 21, 22].
Recent advances in the theoretical understanding, fabrication, and
characterization of correlated and topological phases in (111)-oriented
perovskite-derived heterostructures have been highlighted in Ref. [23].

In contrast to the honeycomb bilayers in (111)-oriented perovskites, less
attention has been paid to their trilayer counterparts. In the latter, a stack
of three triangular lattices forms a so-called dice lattice. Tight-binding
Hamiltonian studies have shown that the dice lattice and its one-dimensional
ribbon structure exhibit nontrivial electronic properties expressed in terms
of in-gap flat bands, nonzero Chern numbers, quantum anomalous Hall
conductance, and chiral edge states [24, 25]. In particular, a nearest-
neighbor tight-binding model on a bipartite lattice with distinct number of
neighbors for the outer vs. inner layer and Rashba-type SOC predicted nearly
flat bands with a Chern number $C\,=\,\pm 2$ [24]. Material-specific DFT
calculations for the dice lattice are rare: A half-metallic phase was
identified in the (111) trilayers of LaNiO3, with the interfacial Ni $e_{g}$
states contributing to a spin-polarized conduction [11]. To assess the
possibly nontrivial properties of the dice lattice, in this work we explore
(La$X$O3)3/(LaAlO3)3(111) SLs with $X=$ Ti, Mn and Co, where a LaAlO3 trilayer
spacer confines the La$X$O3 dice lattice, as displayed in Fig. 1. While the
model of Wang and Ran assumed an $s$-orbital [24], here we consider a $d^{1}$
configuration for $X=$ Ti ($t_{2g}$1) and an $e_{g}$1 for $X=$ Mn and Co. Such
an orbital configuration has been found essential in order to achieve
topologically nontrivial behavior for the honeycomb layers [12]. We study
systematically the electronic and topological properties of the considered SLs
by performing DFT calculations with an on-site Coulomb repulsion term. In
particular, we investigate how the interplay of ferromagnetism, SOC, and
lattice symmetry influence the band structure, Berry curvature, and anomalous
Hall conductivity of (La$X$O3)3/(LaAlO3)3(111) SLs. Parallels to the honeycomb
counterpart (La$X$O3)2/(LaAlO3)4(111) SLs [12] are also discussed.

Figure 1: (a) and (b) Side and top views of a (La$X$O3)3/(LaAlO3)3(111)
superlattice ($X=$ Ti, Mn and Co), where the La$X$O3 trilayer forms a dice
lattice that is confined to the LaAlO3 trilayer. In this dice lattice one of
the three $X$ sites is located in the central layer and the other two on the
interface layers. $\bm{a}_{1}$ and $\bm{a}_{2}$ are the lateral lattice
vectors of the dice lattice and $\bm{c}$ is the out-of-plane lattice vector.
(c) Two-dimensional Brillouin zone of the dice lattice with $\bm{b}_{1}$,
$\bm{b}_{2}$ are the reciprocal lattice vectors of the real-space counterparts
$\bm{a}_{1}$, $\bm{a}_{2}$ while $\Gamma$, K, $\textrm{K}^{\prime}$, M,
$\textrm{M}^{\prime}$ denote the high-symmetry k-points.

##  II Theoretical Approach

Systematic DFT calculations were performed for (La$X$O3)3/(LaAlO3)3(111) SLs
($X=$ Ti, Mn and Co) with 30 atoms in the primitive cell using the projector
augmented wave method, [26] as implemented in the VASP code [27]. The
generalized gradient approximation (GGA) was used for the exchange-correlation
functional, as parameterized by Perdew, Burke, and Enzerhof [28]. Static
correlation effects were included in the GGA$+U$ approach [29] by employing
$U=5$ eV for the $X$ $3d$ orbitals and $U=8$ eV for the La $4f$ orbitals, in
line with previous work [13, 12]. A detailed examination of the topological
properties for $X=$ Mn as a function of the $U$ parameter is provided in the
Supplemental Material, showing that the topological phases are robust for
variation of $U$ between 3.5 and 6.0 eV. A cutoff energy of 600 eV was used to
truncate the plane-wave expansion and a $\Gamma$-centered $k$-point mesh of
$8\times 8\times 2$ to sample the Brillouin zone (BZ). In order to model the
SLs grown on a LaAlO3(111) substrate, the lateral lattice constant was set to
$a_{\textrm{LAO}}\times\sqrt{2}$, ($a_{\textrm{LAO}}=3.79$ Å). The out-of-
plane lattice parameter and the internal coordinates were optimized until the
forces on atoms were less than 0.01 eV/Å and the change in total energy was
less than $10^{-6}$ eV. Octahedral rotations and distortions were fully taken
into account in the structural optimization. The Fermi surfaces were
calculated using wannier90 [30, 31] and plotted using Fermisurfer [32]. Spin-
orbit coupling (SOC) was included with magnetization direction parallel or
perpendicular to the [111]-direction. For the topological analysis, maximally
localized Wannier functions (MLWFs) [33] were constructed to compute the Berry
curvature and anomalous Hall conductivity of (La$X$O3)3/(LaAlO3)3(111) SLs on
a dense $k$-point mesh of $144\times 144\times 12$ using the wannier90 code
[30, 31].

##  III Results and Discussion

In the following, we consider the electronic and magnetic properties of
(La$X$O3)3/(LaAlO3)3(111) SLs ($X=$ Ti, Mn and Co) with ferromagnetic (FM)
order, which was found to be more stable than a layerwise antiferromagnetic
(AFM) arrangement (see Supplemental Material).

Figure 2: Spin-resolved/site-projected band structures, band-decomposed Fermi
surfaces, and top/side-view spin densities of (La$X$O3)3/(LaAlO3)3 (111) SLs:
(a-b) for $X=$ Ti, (c-d) for $X=$ Mn and (e-f) for $X=$ Co with the isosurface
values of 0.01 $e$/Å3 for $X=$ Ti, Mn and 0.05 $e$/Å3 for $X=$ Co. The band
structures and the spin densities are shown for both P3 (left) and P1 (right)
symmetries. In the band structures, color/black curves represent the
majority/minority bands. The Fermi level is set to zero and denoted by a
dashed line. Red, green, and blue colors represent the contributions from the
first interface (IF1), the central (C), and the second interface (IF2) $X$
layers, respectively. Side and top view of the Fermi surfaces are shown with
electron pockets in purple/blue and hole pockets in green. The spin densities
were integrated in the energy range between –8 eV to $E_{\rm F}$ except for
c-d) where the integration interval is from –1.3 eV to $E_{\rm F}$. We show
also the magnetic moments on the $X$ sites in units of $\mu_{B}$.

###  III.1 GGA+$U$ results

Table 1: Structural, magnetic, and electronic properties of ferromagnetic (La$X$O3)3/(LaAlO3)3(111) SLs ($X=$ Ti, Mn and Co) for P3 and P1 symmetries in the absence of SOC. $\Delta E$ is the energy difference of the system in P3 symmetry compared to P1, $c$ is the optimized out-of-plane lattice constant, $d_{X\textrm{-O}}$ (IF/C) the $X$-O bond lengths in Å in the interfacial/central $X$O6 octahedra, $E_{g}$ the band gap in eV, and $M_{S}$ the layer-resolved spin magnetic moments (in units of $\mu_{B}$) at the $X$ sites, IF1/IF2 denote the first/second interface layer and C the central one. $X$ | Symmetry |  $\Delta E$ (eV) |  $c$ (Å) |  $d_{X\textrm{-O}}$ (IF) |  $d_{X\textrm{-O}}$ (C) |  $E_{g}$ (eV) |  $M_{S}$ (IF1/C/IF2)
---|---|---|---|---|---|---|---
Ti | P3 | 2.57 | 14.23 | 2.06-2.07 | 1.96 | Metal | 1.32/0.22/1.32
P1 | 0 | 14.34 | 2.01-2.14 | 2.04-2.09 | 2.34 | 0.97/0.98/0.97
Mn | P3 | 0.64 | 13.78 | 1.96-2.00 | 1.96 | Metal | 4.04/3.99/4.04
P1 | 0 | 14.00 | 1.89-2.12 | 1.93-2.10 | 0.40 | 3.95/3.97/3.95
Co | P3 | 0.55 | 13.58 | 1.97-1.98 | 1.91 | Metal | 2.44/2.83/2.44
P1 | 0 | 13.77 | 1.87-2.08 | 1.92-2.08 | 0.44 | 3.14/2.32/3.14

To determine the most stable configuration we have considered both SLs
constrained to (P3) symmetry as well as released constraints, leading to (P1)
symmetry. Table 1 lists the structural, magnetic, and electronic properties of
(La$X$O3)3/(LaAlO3)3(111) SLs ($X=$ Ti, Mn and Co) for both P3 and P1
symmetries in the absence of SOC. For $X=$ Ti, Mn and Co, P1 symmetry is
energetically favored over P3 by $\sim 2.57\,(1.57)$, $0.64\,(0.64)$, and
$0.55\,(0.56)$ eV per 30-atom unit cell, respectively, where the values in the
brackets indicate the energy differences including spin-orbit coupling. The
reduction of symmetry is accompanied by an expansion of the out-of-plane
lattice constant $c$ from 14.23 Å (P3) to 14.34 Å (P1) for $X=$ Ti, from 13.78
Å (P3) to 14.00 Å (P1) for $X=$ Mn and from 13.58 Å (P3) to 13.77 Å (P1) for
$X=$ Co. Furthermore, the reduction of symmetry leads to a strong variation of
the $X$-O bond lengths. The shortest Ti-O bonds are obtained for the central
layer (1.96 Å) in P3 symmetry, the ones in the interface layer being
($\sim$2.07 Å). In contrast, a strong bond variation occurs for P1 symmetry in
the interface layers (2.01-2.14 Å) compared to the central layer (2.04-2.09
Å). For $X=$ Mn the bond lengths lie in a narrow range (1.96-2.00 Å) for P3
symmetry, but the disparity is enhanced to 1.89-2.12 Å in the interface layers
and to 1.93-2.10 Å in the central (C) layer for P1 symmetry, indicating a
strong Jahn-Teller (JT) effect. Similarly, the Co-O bonds change from
1.97-1.98 Å for P3 symmetry to 1.87-2.08 Å (IF1/IF2) and 1.92-2.08 Å (C) for
P1 symmetry.

The structural changes are closely related to the electronic properties. Fig.
2 displays the spin-resolved, site-projected band structures, the band-
decomposed Fermi surfaces, and the top/side-view of the spin densities of
(La$X$O3)3/(LaAlO3)3(111) SLs for both P3 and P1 symmetries. In all cases the
band structure close to the Fermi level is dominated by majority spin bands,
leading to halfmetallic behavior for P3 symmetry. We first discuss the high
symmetry cases (left panels). For $X$ = Ti the asymmetry of the interface vs.
central layer for the P3 case discussed above is reflected also in the band
structure (Fig. 2a): while the $3d$ bands of the central lie above 1 eV
(green), the four bands around the Fermi level have exclusively interface
character (purple) with two more dispersive just below $E_{\rm F}$ and two
relatively flat at the Fermi level, both sets touching at K and
$\textrm{K}^{\prime}$.

The band structures for $X=$ Mn and Co (Fig. 2c, e) with P3 symmetry show
similar features and are dominated by majority bands with multiple Dirac
crossings around the Fermi level as well as quadratic band touching at
$\Gamma$ slightly above the Fermi energy and at $\sim 1.5$ eV. These spin-
polarized bands are grouped into three distinct pairs: The lowest occupied and
the highest unoccupied pairs of bands are predominantly localized at the
central layer (green), whereas the middle pair of bands at the Fermi level is
of prevailing interface character (purple). Both the top and bottom pairs of
bands (for Co the latter overlaps with the O $2p$ bands) show two Dirac
crossings which reside at K and $\textrm{K}^{\prime}$. The middle pair
intersects the Fermi level leading to a half-metallic behavior and exhibits
three crossings, located along M-K, K-$\textrm{M}^{\prime}$ and
$\textrm{M}^{\prime}$-$\textrm{K}^{\prime}$.

As seen from the site-projected band structures (Fig. 2a, c, e), the Fermi
surface is dominated by bands of the interfacial layers (purple colors),
whereas the bands of the central layer (green color) are shifted from the
Fermi energy. This indicates that within P3 symmetry electron conductivity in
(La$X$O3)3/(LaAlO3)3(111) SLs is mainly contributed by the $3d$ bands of the
interface $X$-ions. Due to the halfmetallic nature and intertwined bands
around $E_{\rm F}$, the Fermi surface contains coupled electron-hole pockets,
which exhibit different features for $X=$ Ti, Mn and Co (Fig. 2): six electron
pockets (purple) around K and one hole pocket (green) around $\Gamma$ for $X=$
Mn and Co and v.v. for Ti.

Table 2: Electronic and magnetic properties of (La$X$O3)3/(LaAlO3)3(111) SLs ($X=$ Ti, Mn and Co) for P3 and P1 symmetries including SOC with magnetization direction along the [100] and [001] directions. $\Delta E=E_{[100]}-E_{[001]}$ is the magnetocrystalline anisotropy energy (MAE), $E_{g}$ the band gap, $M_{S}$ and $M_{L}$ are the spin and orbital moments (in units of $\mu_{B}$). $X$ | Symmetry | SOC |  $\Delta E$ (meV) |  $E_{g}$ (eV) |  $M_{S}$ (IF1/C/IF2) |  $M_{L}$ (IF1/C/IF2)
---|---|---|---|---|---|---
Ti | P3 | [100] | 0.0 | 0.83 | 1.03/0.92/1.02 | -0.04/-0.01/-0.04
[001] | 0.07 | 0.83 | 1.04/0.93/1.02 | -0.06/-0.03/-0.06
P1 | [100] | 0.18 | 2.33 | 0.97/0.98/0.99 | -0.01/-0.03/-0.01
[001] | 0.0 | 2.33 | 0.98/0.99/0.99 | -0.06/-0.07/-0.06
Mn | P3 | [100] | 0.0 | Metal | 3.99/3.99/3.99 | -0.02/-0.01/-0.01
[001] | 0.34 | Metal | 3.99/3.98/4.0 | -0.01/-0.01/-0.01
P1 | [100] | 0.02 | 0.39 | 3.91/4.0/3.91 | -0.01/-0.00/-0.01
[001] | 0.0 | 0.39 | 3.91/4.01/3.91 | -0.00/-0.02/-0.00
Co | P3 | [100] | 0.0 | Metal | 2.44/2.82/2.45 | 0.23/0.21/0.23
[001] | 4.79 | Metal | 2.43/2.83/2.43 | 0.16/0.07/0.16
P1 | [100] | 2.38 | 0.42 | 3.04/2.36/3.04 | 0.11/0.24/0.11
[001] | 0.0 | 0.42 | 3.35/1.96/3.35 | 0.21/0.16/0.21

The symmetry lowering and structural distortion from P3 to P1 triggers a
significant reconstruction of the band structures of $X=$ Ti, Mn and Co (Fig.
2b, d, f) leading in all cases to a metal-to-insulator transition. The
degeneracy of the M and $\textrm{M}^{\prime}$ points is lifted and the set of
spin-polarized bands around the Fermi energy splits into an occupied valence
and an empty conduction band separated by a substantial band gap of 2.34,
0.40, and 0.44 eV for $X=$ Ti, Mn and Co (cf. Table 1), respectively. For $X=$
Ti three narrow bands are occupied just below $E_{\rm F}$, the lowest one with
predominant central-layer contribution and the other two of prevailing
interface character. The highest valence band of $X=$ Mn has interface
character, whereas the conduction band is of mixed character with its bottom
at $\textrm{M}^{\prime}$ having a stronger contribution from the central
layer. On the other hand for $X=$ Co both the top valence and bottom
conduction band have a predominant contribution from the central layer and the
interface bands lie away from $E_{\rm F}$.

The spin densities and magnetic moments in the IF1/C/IF2 layers for P3 and P1
symmetries, shown in Fig. 2, the latter listed also in Table 1, provide
further insights into the electronic reconstruction. The pronounced asymmetry
between the central (C) and interface (IF1 and IF2) layers for $X=$ Ti in P3
symmetry is also reflected in the magnetic moments 1.32/0.22/1.32 $\mu_{\rm
B}$. This indicates a modulation of the Ti valence state: Ti3±δ in the
interface and Ti4+ in the central layer, which is consistent with the band
occupation pattern described above (Fig. 2a): a fully and a partially occupied
band for each interface layer and empty $d$ bands for the central layer. In
the (111)-oriented SLs the octahedral symmetry is reduced to trigonal which
splits the $t_{2g}$ triplet into an $a_{1g}$ singlet and an $e^{\prime}_{g}$
doublet. The spin density for P3 symmetry (Fig. 2) displays a preferential
occupation of the $e^{\prime}_{g}$ doublet in the interface layer. In
contrast, for P1 symmetry similar magnetic moments in all layers
0.97/0.98/0.97 $\mu_{\rm B}$ are obtained, consistent with a valence state of
Ti3+ in all layers. Interestingly, the spin density for P1 symmetry (Fig. 2)
reflects a staggered orbital polarization of $d_{xz}$ and $d_{yz}$ orbitals
instead of the expected $a_{1g}$ or $e^{\prime}_{g}$ orbitals. This is similar
to the behavior found in the honeycomb (LaTiO3)2/(LaAlO3)4(111) in P1 symmetry
[12].

For $X=$ Mn (Fig. 2c, d) the magnetic moments are almost unchanged between the
outer and inner layer and between P3 (4.04/3.99/4.04 $\mu_{B}$) and P1
symmetry (3.95/3.97/3.95 $\mu_{\rm B}$) and are consistent with a high spin
Mn3+ $d^{4}$ configuration. Moreover, the metal-to-insulator transition from
P3 to P1 symmetry can be understood as a result of a Jahn-Teller (JT)
distortion of the JT active Mn3+ ion, analogous to the behavior observed in
the honeycomb Mn-bilayer [12]. For $X=$ Co3+ in P3 symmetry (Fig. 2), the
magnetic moment at the Co sites in the interface layers is $2.44$ $\mu_{B}$
and the central layer acquires a magnetic moment of $2.83$ $\mu_{B}$,
indicating an intermediate-spin state ($t_{2g}^{5}$, $e_{1g}^{1}$) (Fig. 2).
In contrast, for P1 symmetry the sizes of magnetic moments are reversed with
enhanced magnetic moments in the interfacial layer (3.14 $\mu_{\rm B}$) rather
pointing to a high-spin state, while the magnetic moment in the central layer
is nearly 1 $\mu_{\rm B}$ smaller (2.32 $\mu_{\rm B}$). We note that different
spin states of Co were also reported for the honeycomb Co-bilayers [12] and
are related to the rich phase diagram of bulk LaCoO3 with respect to spin
degree of freedom, e.g., with transitions from a low-spin ($t_{2g}^{6}$)
ground state to a ferromagnetic intermediate- or high-spin state e.g. under
pressure or strain [34, 35].

###  III.2 Effect of SOC and Topological Analysis

In the following, we proceed with the effect of SOC and topological analysis
in (La$X$O3)3/(LaAlO3)3(111) SLs with $X=$ Ti, Mn and Co. For the analysis of
the topological properties, we performed a Wannier interpolation of the
relevant part of the DFT+$U$+SOC band structure around the Fermi energy with
prevailing $X$ $3d$ character and calculated both the Berry curvature and
anomalous Hall conductivity (AHC) for the low-energy $X$ $3d$ bands by
constructing the MLWFs [33] using the wannier90 code [30, 31]. The quality of
the Wannier fit is demonstrated in the Supplemental Material by superimposing
the Wannier interpolated bands on the original DFT bands for $X=$ Ti, Mn and
Co.

Figure 3: (a-b) Element-projected GGA+$U$+SOC band structures with the same
color coding as used in Fig. 2 for $X=$ Ti with P3 symmetry and magnetization
directions along [100] and [001]. Additionally, the electron density
distribution integrated in the energy range between –2.3 and –1.5 eV, as well
as –1.0 and –0.3 eV at the isosurface value of 0.01 $e$/Å3 is shown; (c-d)
Berry curvatures $\Omega_{xy}(\textbf{k})$ along the same k-path; (e-f) the
corresponding anomalous Hall conductivities $\sigma_{xy}^{\textrm{AHC}}$ vs.
the chemical potential in units of $e^{2}/h$.  Figure 4: (a-b) Element-
projected GGA+$U$+SOC band structures with the same color coding as used in
Fig. 2 for $X=$ Mn with P3 symmetry and magnetization directions along [100]
and [001]. Additionally, the electron density distribution of integrated in
the energy range between –1.4 and –0.7 eV, –0.3 and 0.22 eV and 0.23 and 1.55
eV at the isosurface value of 0.01 $e$/Å3 are displayed; (c-d) corresponding
Berry curvatures $\Omega_{xy}(\textbf{k})$ along the same k-path; (e-f)
anomalous Hall conductivities $\sigma_{xy}^{\textrm{AHC}}$ vs. the chemical
potential in units of $e^{2}/h$.

The Berry curvature is calculated using the Kubo formula [36, 37]

$\Omega_{xy}^{z}(\textbf{k})=-2\sum_{n\in occ}\sum_{m\neq
n}\textrm{Im}\frac{\left\langle\psi_{n\textbf{k}}\left|v_{x}\right|\psi_{m\textbf{k}}\right\rangle\left\langle\psi_{m\textbf{k}}\left|v_{y}\right|\psi_{n\textbf{k}}\right\rangle}{{(\epsilon_{m\textbf{k}}-{\epsilon}_{n\textbf{k}})}^{2}},$
(1)

where the sum over $n$ is restricted to the occupied bands,
$\psi_{n\textbf{k}}$ is the spinor wave function of the $n^{th}$ band,
$\epsilon_{n\textbf{k}}$ is the corresponding band energy, and $v_{x}$
($v_{y}$) is the velocity operator along the $x$ ($y$) direction. The
anomalous Hall conductivity is calculated by integrating the Berry curvature
over the Brillouin zone (BZ) which is transformed into a sum over k-points,

$\sigma_{xy}^{\textrm{AHC}}=-\frac{e^{2}}{\hbar}\frac{1}{N_{\textbf{k}}V_{c}}\sum_{\textbf{k}}\Omega_{xy}^{z}(\textbf{k}),$
(2)

where $V_{c}$ is the volume of the unit cell and $N_{\textbf{k}}$ the number
of k-points used for sampling the BZ.

Figure 5: (a-b) Element-projected GGA+$U$+SOC band structures with the same
color coding as used in Fig. 2 for $X=$ Co with P3 symmetry and magnetization
directions along [100] and [001]. Additionally the electron density
distribution integrated in the energy range between –0.5 and $E_{\rm F}$,
$E_{\rm F}$ and 0.74 eV and 0.8 and 1.0 eV at the isosurface value of 0.01
$e$/Å3 is displayed; (c-d) Berry curvatures $\Omega_{xy}(\textbf{k})$ along
the same k-path; (e-f) corresponding anomalous Hall conductivities
$\sigma_{xy}^{\textrm{AHC}}$ vs. the chemical potential in units of $e^{2}/h$.

Figures 3, 4 and 5 show the GGA+$U$+SOC band structures, the Berry curvatures
(BCs) and anomalous Hall conductivities of $X=$ Ti, Mn and Co for in- and out-
of-plane magnetization direction with P3 symmetry, for which we observe a
significant effect of SOC. The corresponding results for P1 symmetry are shown
in the Supplemental Material. The magnetocrystalline anisotropy and the spin
and orbital moments are listed in Table 2. It is noteworthy that in all cases
for the high symmetry the magnetic easy axis is found to be in-plane (see
Table 2), whereas for the systems with P1 symmetry the magnetic easy axis is
out-of-plane. For $X=$ Ti SOC has only a small effect for P1 symmetry (cf.
Fig. S1 in the Supplemental Material) leaving the band gap nearly unchanged
for both the in-plane and out-of-plane directions (see Table 2). In contrast,
for P3 symmetry SOC leads to a significant band reconstruction (Fig. 3a, b)
and a metal-to-insulator transition. The intertwined bands around the Fermi
level in the absence of SOC (Fig. 2a) are disentangled due to SOC and split
into occupied and unoccupied bands separated by a band gap of 0.83 eV for both
magnetization directions. While the single band, comprising the valence band
maximum (VBM) is mainly contributed by the central-layer, the lower two bands
in the energy range between –1.3 eV and –0.3 eV exhibit a predominant
interface character. Consistent with the band occupation, a Ti3+ valence state
in all layers is obtained with SOC, reflected in similar magnetic moments
1.03/0.92/1.02 $\mu_{\rm B}$. The electron density distribution in the two
energy intervals indicates a staggered $d_{xz}$ (IF), $d_{yz}$ (C) orbital
polarization, similar to results for the LaTiO3 bilayer [12]. Concerning the
topological properties of $X=$ Ti, the Berry curvature shows strong
oscillations between positive and negative values, leading to a vanishing AHC
for both magnetization directions (Fig. 3e, f).

For $X=$ Mn the band structures with P3 symmetry in the presence of SOC with
[100] (in-plane) and [001] (out-of-plane) magnetization directions (Fig. 4a,
b) exhibit no pronounced modification, compared to those in the absence of SOC
(Fig. 2c). Still, a closer inspection reveals that SOC lifts the quadratic
band touching at $\Gamma$ for both magnetization directions (Fig. 4a, b), the
in-plane magnetization being energetically favored by 0.34 meV (see Table 2).
In contrast to $X=$ Ti, only the middle pair of bands at the Fermi level shows
a distinct interface character, whereas the lowest occupied and the highest
unoccupied pairs of bands have contributions from the central with some
admixture of the interface layer. The electron density distribution in the
three selected energy ranges gives further insight into the contribution of
the different layers and suggest degenerate $e_{g}$ orbital occupation with
some contribution of neighboring O $2p$ states. The Berry curvature exhibits
significant negative contributions (cf. Fig. 4c, d) due to the avoided
crossings of bands along M-K and $\textrm{M}^{\prime}$-$\textrm{K}^{\prime}$.
This results in large negative spikes in the AHC close to $\sim-3e^{2}/h$ (cf.
Fig. 4e, f), indicating nontrivial pairs of bands with Chern numbers of $\pm
3$ just below/above $E_{\rm F}$. We have explored the effect of the $U$
parameter on the band structure and topological properties for $X=$ Mn and
find that the emergence of high Chern numbers is robust beyond $U\,=\,3.0$ eV
up to the studied maximum value of 6 eV. For more details, see Fig. S5 in the
Supplemental Material. Additionally, we investigated the relative stability
between the nontrivial P3 and trivial P1 phase under strain and find that the
P3 phase is stabilized under tensile strain for a lateral lattice constant
between $a=4.04$ Å and $a=4.14$ Å (see Fig. S6 in the Supplemental Material),
corresponding to the lattice parameters of, e.g., PrScO3 or LaScO3 [38].
Moreover, the topological properties of both P1 and P3 at $a=4.04$ Å (cf. Fig.
S7 in the Supplemental Material) were explored. In particular, the system with
P3 symmetry exhibits nontrivial topological bands accompanied by a
significant, nearly integer, AHC value of $\sim 0.94e^{2}/h$.

Figure 6: The calculated edge states for $X$ = Mn, Co superlattices shown in
(a-b) for (100) surfaces. Red-white range of colors represent higher local
DOS, the solid red lines correspond to the edge states connecting valence and
conduction bands. The blue regions denote the energy gap. The Fermi level is
set to zero.

Cobalt-containing compounds tend to have a large magneto-crystalline
anisotropy. This is also observed for the Co-based dice lattice which shows
the highest $\Delta E$ value among the systems in the current investigation:
4.79 meV and 2.38 meV (see Table 2) for the P3 and P1 symmetry, respectively.
Moreover, $X=$ Co acquires a substantial orbital moment of up to 0.23 and 0.24
$\mu_{B}$ for P3 and P1 symmetry with magnetization along [100] (cf. Table 2).
SOC also has a stronger effect on the band structure and splits the bands at K
and $\textrm{K}^{\prime}$ as well as the quadratic band touching point at
$\Gamma$, the effect being larger for the magnetization axis along [001]
compared to the [100] crystallographic direction (see Fig. 5a, b). The bands
around $E_{\rm F}$ show multiple crossings and a predominant interface
character in the occupied part, mixed contribution between $E_{\rm F}$ and 0.8
eV, whereas the extremely flat bands above 0.8 eV show prevailing central
layer contribution (see also integrated electron density distribution of the
three energy regions which also indicates substantial O $2p$ hybridization).
The larger effect of SOC for out-of-plane magnetization is also reflected in
larger negative contributions to the Berry curvature $\Omega(k)$ (cf. Fig. 5c,
d), in particular a peak arising due to the avoided crossing along $\Gamma$-M.
Analogous to $X=$ Mn (cf. Fig. 4e) the AHC for $X=$ Co (see Fig. 5f) reaches
values $\sim-3e^{2}/h$, however only for out-of-plane magnetization direction.
Interestingly, the sign of the Berry curvature and Hall conductivity bear
analogies to the ones identified for $X=$ Mn and Co in the (111)-oriented
bilayers of La$X$O3 [12]. However, due to the rather semimetallic character
with valence and conduction band touching close to the Fermi level the
formation of a quantized Hall plateau at $E_{\rm F}$ is hampered.
Nevertheless, the $\sim-3e^{2}/h$ peak at $E_{\rm F}$ indicates nontrivial
bands with $C=\pm 3$. Moreover, a quantized plateau of $\sim-e^{2}/h$ emerges
above $E_{\rm F}$ at 0.2 eV related to the avoided crossing at the K point. In
order to confirm the topological character for $X=$ Mn and Co, we carried out
edge state calculations by constructing the MLWFs. The edge Green’s function
and the local density of states (LDOS) can be simulated using an iterative
method [39, 40, 41]. As displayed in Fig. 6a, b the topologically protected
chiral edge states obtained for $X=$ Mn, Co show that valence and conduction
bands are connected but the edge states are obscured due to the crossing of
bands at $E_{\rm F}$ in both cases.

Previous model Hamiltonian studies indicate that the dice lattice can host
topological bands with nonzero AHC around the Fermi energy [24, 25]. Our
DFT+$U$+SOC calculations also predict nontrivial electronic bands with finite
AHC around the Fermi energy. While Wang and Ran [24] found bands with Chern
numbers $C=\pm 2$, our DFT+$U$ results indicate $C=\pm 3$ for $X=$ Mn and Co.
The difference can be attributed to the assumptions in the model study of an
$s$-orbital and a Rashba-type SOC, while in our study the SOC effect is not
related to a breaking of inversion symmetry. Moreover, the active orbital is a
$3d$ orbital ($e_{g}$). The DFT+$U$ results allow to gain insight into the
orbital character and address the role of lattice symmetry, atomic relaxation,
orbital hybridization and spin orientation. Disentangling these aspects is
pivotal to understand the electronic and topological properties of perovskite-
derived dice lattices.

##  IV Summary

We have performed a systematic DFT+$U$+SOC study of the electronic and
topological properties of (La$X$O3)3/(LaAlO3)3(111) SLs for $X=$ Ti, Mn and
Co, where the La$X$O3 (L$X$O) trilayer defines a dice lattice confined by the
band insulator LaAlO3. By considering lattice symmetry and SOC, we found that:
(i) In the absence of SOC, when the symmetry of the three $X$ sublattices is
constrained to P3, the FM phase of the L$X$O trilayer exhibits a set of spin-
polarized bands with predominand interface character around the Fermi energy,
leading to a distinct half-metallic state with multiple Dirac crossings and
coupled electron-hole pockets; (ii) By releasing the sublattice symmetry, the
FM phase of the L$X$O trilayer undergoes a significant electronic
reconstruction and a metal-to-insulator transition, due to a Jahn-Teller
effect ($X=$Mn) and accompanied by a modulation of magnetic moments for $X=$
Co; (iii) SOC for P3 symmetry leads to substantial band reconstruction for
$X=$ Ti resulting in a metal-to-insulator transition. While the effects of SOC
are more subtle for $X=$ Mn and Co, avoided crossings close to $E_{\rm F}$ and
lifting of the quadratic band touching at $\Gamma$ lead to anomalous Hall
conductivities reaching $\sim-3e^{2}/h$; (iv) The band structure, anomalous
Hall conductivity and Berry curvature depend strongly on the sublattice
symmetry and the magnetization direction, thereby providing several degrees of
freedom to tune the electronic and topological properties of perovskite-based
dice lattices. While in the studied Mn and Co-based dice lattices a quantized
AHC is hampered due to the (semi-) metallic character, robust Chern insulators
may be achieved as a function of strain or other choice of $X$. Furthermore,
electrostatic doping may be used to tune the Fermi level to the topologically
nontrivial bands which can be achieved, e.g., by using polar oxide surfaces
[42] or semiconductor interfaces [43, 44]. The presented results for
(La$X$O3)3/(LaAlO3)3(111) indicate that the dice lattice establishes a
promising and rich playground to achieve exotic electronic and topological
states beyond the honeycomb lattice.

## AUTHOR CONTRIBUTIONS

The project was conceived and supervised by R. Pentcheva. Initial DFT+$U$
calculations were carried out by L. L. Li and the project was completed by O.
Köksal. All authors contributed to the analysis and wrote the manuscript.

## DATA AVAILABILITY

The authors declare that the main data supporting the finding of this study
are available within the article and its Supplementary Information files.
Additional data can be provided upon request.

## ACKNOWLEDGMENTS

This work was supported by the German Science Foundation (DFG) within
SFB/TRR80 (project number 107745057) project G3 and computational time at the
Leibniz Supercomputer Center (project pr87ro).

## COMPETING INTERESTS

The authors declare no competing financial or non-financial interests.

## References

  * Tokura [2003] Y. Tokura, Phys. Today 56, 50 (2003).
  * Dagotto [2005] E. Dagotto, Science 309, 257 (2005).
  * Lorenz  _et al._ [2016] M. Lorenz, M. S. R. Rao, T. Venkatesan, E. Fortunato, P. Barquinha, R. Branquinho, D. Salgueiro, R. Martins, E. Carlos, A. Liu, F. K. Shan, M. Grundmann, H. Boschker, J. Mukherjee, M. Priyadarshini, N. DasGupta, D. J. Rogers, F. H. Teherani, E. V. Sandana, P. Bove, K. Rietwyk, A. Zaban, A. Veziridis, A. Weidenkaff, M. Muralidhar, M. Murakami, S. Abel, J. Fompeyrine, J. Zuniga-Perez, R. Ramesh, N. A. Spaldin, S. Ostanin, V. Borisov, I. Mertig, V. Lazenka, G. Srinivasan, W. Prellier, M. Uchida, M. Kawasaki, R. Pentcheva, P. Gegenwart, F. M. Granozio, J. Fontcuberta, and N. Pryds, J. Phys. D: Appl. Phys. 49, 433001 (2016).
  * Pentcheva and Pickett [2010] R. Pentcheva and W. E. Pickett, J. Phys.: Condens. Matter 22, 043001 (2010).
  * Hwang  _et al._ [2012] H. Y. Hwang, Y. Iwasa, M. Kawasaki, B. Keimer, N. Nagaosa, and Y. Tokura, Nature Mater 11, 103 (2012).
  * Chakhalian  _et al._ [2014] J. Chakhalian, J. W. Freeland, A. J. Millis, C. Panagopoulos, and J. M. Rondinelli, Rev. Mod. Phys. 86, 1189 (2014).
  * Xiao  _et al._ [2011] D. Xiao, W. Zhu, Y. Ran, N. Nagaosa, and S. Okamoto, Nat Commun 2, 1 (2011).
  * Yang  _et al._ [2011] K.-Y. Yang, W. Zhu, D. Xiao, S. Okamoto, Z. Wang, and Y. Ran, Phys. Rev. B 84, 201104 (2011).
  * Rüegg  _et al._ [2012] A. Rüegg, C. Mitra, A. A. Demkov, and G. A. Fiete, Phys. Rev. B 85, 245131 (2012).
  * Rüegg  _et al._ [2013] A. Rüegg, C. Mitra, A. A. Demkov, and G. A. Fiete, Phys. Rev. B 88, 115146 (2013).
  * Doennig  _et al._ [2014] D. Doennig, W. E. Pickett, and R. Pentcheva, Phys. Rev. B 89, 121110(R) (2014).
  * Doennig  _et al._ [2016] D. Doennig, S. Baidya, W. E. Pickett, and R. Pentcheva, Phys. Rev. B 93, 165145 (2016).
  * Weng  _et al._ [2015] Y. Weng, X. Huang, Y. Yao, and S. Dong, Phys. Rev. B 92, 195114 (2015).
  * Guo  _et al._ [2017] H. Guo, S. Gangopadhyay, O. Köksal, R. Pentcheva, and W. E. Pickett, Npj Quantum Mater. 2, 1 (2017).
  * Lu and Guo [2019] H.-S. Lu and G.-Y. Guo, Phys. Rev. B 99, 104405 (2019).
  * Köksal and Pentcheva [2019] O. Köksal and R. Pentcheva, Sci Rep 9, 1 (2019).
  * Herranz  _et al._ [2012] G. Herranz, F. Sánchez, N. Dix, M. Scigaj, and J. Fontcuberta, Sci. Rep. 2, 1 (2012).
  * Middey  _et al._ [2012] S. Middey, D. Meyers, M. Kareev, E. J. Moon, B. A. Gray, X. Liu, J. W. Freeland, and J. Chakhalian, Appl. Phys. Lett. 101, 261602 (2012).
  * Piamonteze  _et al._ [2015] C. Piamonteze, M. Gibert, J. Heidler, J. Dreiser, S. Rusponi, H. Brune, J.-M. Triscone, F. Nolting, and U. Staub, Phys. Rev. B 92, 014426 (2015).
  * Wei  _et al._ [2017] H. Wei, C. Yang, J. L. Barzola-Quiquia, M. Welke, R. Denecke, C. Patzig, T. Höche, P. Esquinazi, M. Grundmann, and M. Lorenz, J. Phys. D: Appl. Phys. 50, 43LT02 (2017).
  * Arab  _et al._ [2019] A. Arab, X. Liu, O. Köksal, W. Yang, R. U. Chandrasena, S. Middey, M. Kareev, S. Kumar, M.-A. Husanu, Z. Yang, L. Gu, V. N. Strocov, T.-L. Lee, J. Minár, R. Pentcheva, J. Chakhalian, and A. X. Gray, Nano Lett. 19, 8311 (2019).
  * Bisht  _et al._ [2022] R. S. Bisht, M. Mograbi, P. K. Rout, G. Tuvia, Y. Dagan, H. Yoon, A. G. Swartz, H. Y. Hwang, L. L. Li, and R. Pentcheva, arXiv:2102.07239 (2022).
  * Chakhalian  _et al._ [2020] J. Chakhalian, X. Liu, and G. A. Fiete, APL Mater. 8, 050904 (2020).
  * Wang and Ran [2011] F. Wang and Y. Ran, Phys. Rev. B 84, 241103 (2011).
  * Soni  _et al._ [2020] R. Soni, N. Kaushal, S. Okamoto, and E. Dagotto, Phys. Rev. B 102, 045105 (2020).
  * Kresse and Joubert [1999] G. Kresse and D. Joubert, Phys. Rev. B 59, 1758 (1999).
  * Kresse and Furthmüller [1996] G. Kresse and J. Furthmüller, Phys. Rev. B 54, 11169 (1996).
  * Perdew  _et al._ [1996] J. P. Perdew, K. Burke, and M. Ernzerhof, Phys. Rev. Lett. 77, 3865 (1996).
  * Dudarev  _et al._ [1998] S. L. Dudarev, G. A. Botton, S. Y. Savrasov, C. J. Humphreys, and A. P. Sutton, Phys. Rev. B 57, 1505 (1998).
  * Mostofi  _et al._ [2008] A. A. Mostofi, J. R. Yates, Y.-S. Lee, I. Souza, D. Vanderbilt, and N. Marzari, Comput. Phys. Commun. 178, 685 (2008).
  * Mostofi  _et al._ [2014] A. A. Mostofi, J. R. Yates, G. Pizzi, Y.-S. Lee, I. Souza, D. Vanderbilt, and N. Marzari, Comput. Phys. Commun. 185, 2309 (2014).
  * Kawamura [2019] M. Kawamura, Comput. Phys. Commun. 239, 197 (2019).
  * Marzari  _et al._ [2012] N. Marzari, A. A. Mostofi, J. R. Yates, I. Souza, and D. Vanderbilt, Rev. Mod. Phys. 84, 1419 (2012).
  * Hsu  _et al._ [2012] H. Hsu, P. Blaha, and R. M. Wentzcovitch, Phys. Rev. B 85, 140404 (2012).
  * Geisler and Pentcheva [2020] B. Geisler and R. Pentcheva, Phys. Rev. B 101, 165108 (2020).
  * Wang  _et al._ [2006] X. Wang, J. R. Yates, I. Souza, and D. Vanderbilt, Phys. Rev. B 74, 195118 (2006).
  * Yates  _et al._ [2007] J. R. Yates, X. Wang, D. Vanderbilt, and I. Souza, Phys. Rev. B 75, 195121 (2007).
  * Uecker  _et al._ [2017] R. Uecker, R. Bertram, M. Brützam, Z. Galazka, T. M. Gesing, C. Guguschev, D. Klimm, M. Klupsch, A. Kwasniewski, and D. G. Schlom, Journal of Crystal Growth 457, 137 (2017).
  * Wu  _et al._ [2018] Q. Wu, S. Zhang, H. Song, M. Troyer, and A. A. Soluyanov, Comput. Phys. Commun. 224, 405 (2018).
  * Sancho  _et al._ [1984] M. P. L. Sancho, J. M. L. Sancho, and J. Rubio, J. Phys. F 14, 1205 (1984).
  * Sancho  _et al._ [1985] M. P. L. Sancho, J. M. L. Sancho, and J. Rubio, J. Phys. F 15, 851 (1985).
  * Baidya  _et al._ [2016] S. Baidya, U. V. Waghmare, A. Paramekanti, and T. Saha-Dasgupta, Phys. Rev. B 94, 155405 (2016).
  * Miao  _et al._ [2012] M. S. Miao, Y. Q., C. G. Van de Walle, W. K. Lou, L. L. Li, and K. Chang, Phys. Rev. Lett. 109, 186803 (2012).
  * Zhang  _et al._ [2013] D. Zhang, W. Lou, M. Miao, S.-C. Zhang, and K. Chang, Phys. Rev. Lett. 111, 156402 (2013).


Christiaan Jacobs1, Nathanaël Carraz Rakotonirina2,5, Everlyn Asiko
Chimoto3,4,
Bruce A. Bassett3,4,5, Herman Kamper1

# Towards hate speech detection in low-resource languages:
Comparing ASR to acoustic word embeddings on Wolof and Swahili

###### Abstract

We consider hate speech detection through keyword spotting on radio
broadcasts. One approach is to build an automatic speech recognition (ASR)
system for the target low-resource language. We compare this to using acoustic
word embedding (AWE) models that map speech segments to a space where matching
words have similar vectors. We specifically use a multilingual AWE model
trained on labelled data from well-resourced languages to spot keywords in
data in the unseen target language. In contrast to ASR, the AWE approach only
requires a few keyword exemplars. In controlled experiments on Wolof and
Swahili where training and test data are from the same domain, an ASR model
trained on just five minutes of data outperforms the AWE approach. But in an
in-the-wild test on Swahili radio broadcasts with actual hate speech keywords,
the AWE model (using one minute of template data) is more robust, giving
similar performance to an ASR system trained on 30 hours of labelled data.

Index Terms: Keyword spotting, acoustic word embeddings, zero-resource speech
processing, hate speech detection.

##  1 Introduction

Hate speech is a pervasive problem in many parts of the world. In some
developing countries, radio broadcasts are the primary medium for
communicating news and information to the public [1]. They also serve as a
platform for public discourse, enabling freedom of expression [2]. Monitoring
hate speech on these radio broadcasts in low-resource languages is extremely
challenging due to factors such as background noise, speaker variability, and
the presence of accents or dialects in the audio.

Hate speech detection can be supported by using a keyword spotting (KWS)
system to search through an audio corpus for a predetermined set of keywords
indicative of hate speech. The conventional KWS approach involves transcribing
the audio corpus with an automatic speech recognition (ASR) system and then
searching for keywords in the output. However, for low-resource languages, the
absence of large quantities of transcribed speech can make it difficult to
train high-quality ASR models. Several studies have looked into this problem,
proposing methods to train models on limited amounts of labelled data [3, 4,
5, 6].

When we want to rapidly develop and deploy a KWS system for hate speech
detection in a new setting, it might be better to opt for an ASR-free
approach. The most popular methodology extends query-by-example (QbE), which
uses a spoken instance of a keyword to search through an audio corpus. To use
QbE for KWS, we need a small number of spoken templates to serve as queries
for the keywords of interest; this is the approach followed in [5, 7]. Dynamic
time warping is typically used to match the speech features of a spoken query
to a search utterance [8, 9, 10, 11].

Recent alternative approaches jointly map search segments and query segments
to a fixed vector space, allowing for faster comparisons. Various neural
networks have been considered to obtain these acoustic word embeddings (AWEs)
[12, 13, 14, 15, 16, 17]. One approach to quickly obtain robust embeddings for
a new language is to train a multilingual AWE model on labelled data from
multiple well-resourced languages, before applying it to the target low-
resource language [18, 19, 20, 21, 22]. Although these multilingual AWE models
have proven successful in controlled experiments, there has been limited work
investigating the effectiveness of these systems beyond the experimental
environment—and even less so for hate speech detection.

In this paper, we investigate hate speech detection in low-resource languages
through KWS. Our main goal is to compare ASR to a multilingual AWE-based
approach on real data. Specifically, we consider the performance of different
ASR and AWE-based systems on real Swahili radio broadcast data scraped from
stations in Kenya, a country in sub-Saharan Africa. For our ASR systems, we
build on recent advancements in multilingual speech recognition models,
enabling a pre-trained ASR model to be fine-tuned using much less training
data than that required for training an ASR model from scratch [23, 24, 25].
We investigate the trade-off between the amount of ASR training data and KWS
performance. We compare the fine-tuned ASR models to a KWS system using QbE
with multilingual AWEs [22, 26].

Building up to our final experiments on real radio broadcasts, we first
perform experiments in a controlled environment where training and test data
come from the same domain. Given that the pre-trained ASR model used in our
experiments includes Swahili as a pre-training language, we also consider
performance on another low-resource language, Wolof, in our controlled
experiments. Then, we put the ASR and AWE KWS systems to a real-life test by
evaluating performance on Swahili radio broadcasts. In the controlled
experiments we find that fine-tuning an ASR model using as little as five
minutes of labelled data outperforms the AWE-based KWS system which relies on
roughly one minute of template audio. However, in the out-of-domain experiment
on real scraped Swahili radio broadcasts, the AWE system proves to be more
robust by almost reaching the performance of an ASR model fine-tuned on 30
hours of data. Our findings suggest that KWS using multilingual AWEs is a
promising approach for quickly implementing hate speech detection in a new
unseen language if resources are severely limited.

##  2 Towards hate speech detection through low-resource keyword spotting

In this section, we briefly give the overall methodology that we follow
towards our end goal of hate speech detection on Swahili radio broadcasts.
Exact details are then given in what follows.

We start by developing Wolof and Swahili KWS systems in a controlled
environment where we use parallel audio-text data from experimental datasets
to train and test our models. These datasets [27, 28] contain high-quality,
noise-free read speech. We then test our models on data from these corpora. We
refer to these experiments as in-domain because the test data comes from the
same domain as the training data. In this controlled setup, we perform KWS
using a set of keywords drawn from development data. These are not necessarily
hate speech keywords. The benefit of this controlled setting is that we have
transcriptions for the test data, and can therefore quantify performance
exactly.

We then apply the models to the out-of-domain Swahili radio broadcasts. These
recordings were obtained from three radio stations in Kenya. They consist of a
mix of read and spontaneous speech, and also include non-speech audio like
music and advertisements. Additionally, there may be instances of different
accents and languages present within the recordings. This is the type of real-
world audio that we would like to monitor for hate speech using KWS. As our
keywords in this context, we utilise actual hate speech keywords. These were
labelled as inflammatory words by native Swahili experts familiar with the
media environment. We should emphasise that, when we detect occurrences of
these words, they might not necessarily respond to hate speech (e.g. the
English sentence ``do not kill your neighbour'' is not hate speech despite
containing the word ``kill''). In a deployed system the utterances flagged by
our KWS approach would be passed on to human moderators for further review and
hate speech assessment. In this work we also use a human moderator, but only
to mark whether a detected keyword actually occurred in an utterance. We do
this for our in-the-wild test since we do not have transcriptions for the
radio broadcasts.

##  3 Keyword spotting with ASR

ASR is one of the most straightforward approaches to hate speech detection via
KWS in low-resource languages. Given a set of keywords, the ASR model first
transcribes the audio into text, then the keywords are searched for in the
resulting text [29]. One property of this approach is that the downstream KWS
performance depends on the ASR model's performance.

We use an ASR model from the Wav2vec 2.0 XLS-R family [23, 30]. XLS-R is a
large-scale cross-lingual speech model pre-trained on half a million hours of
unlabelled speech data in 128 languages (including Swahili but not Wolof). The
speech representation learned during cross-lingual pre-training improves
performance for low-resource languages by leveraging data from the high-
resource languages on which it is pre-trained. Using the pre-trained model as
a starting point, the model is fine-tuned on labelled data from the language
of interest.

Previous work has shown that this fine-tuning approach leads to competitive
ASR performance even with limited amounts of labelled data [30]. In our
experiments (Table 3 specifically) we show that the sample efficiency of the
XLS-R model also extends to KWS when training on just five minutes of labelled
data.

##  4 Keyword spotting with multilingual acoustic word embeddings

For a severely under-resourced language, there might not even be limited
amounts of labelled data available to fine-tune a pre-trained ASR model. In
this scenario, collecting only a few spoken instances of a keyword of interest
would allow for KWS through query-by-example (QbE) search [5, 7]. QbE is the
task of retrieving utterances from a search corpus using a spoken instance of
a keyword, instead of a written keyword.

Figure 1: Search utterances ($u_{1},u_{2},\ldots,u_{N}$) are segmented and
mapped to a fixed-dimensional space, along with a query segment $q$, to
produce embeddings $\mathbf{z}_{n}^{i}$ and $\mathbf{z}_{q}$, respectively.
Subsequently, search utterances are ranked based on the shortest distance
between a search segment $\mathbf{z}_{n}^{i}$ and the query vector
$\mathbf{z}_{q}$.

The workings of an AWE-based QbE system are displayed in Figure 1. First, the
search corpus is segmented into word-like segments. Ideally, we would like to
use true word boundaries to segment each utterance in the search collection.
However, in an under-resourced setting, word boundaries are not available.
Therefore, we follow an exhaustive segmentation technique where utterances are
split into overlapping segments of some minimum to some maximum duration [31].
Concretely, utterances from the search corpus ($u_{1},u_{2},\ldots,u_{N}$) are
segmented to produce a set of variable-length speech segments
$S=\\{s_{n}^{i}\\}$, where $i$ is the $i^{\text{th}}$ search segment extracted
from utterance $n$.

A multilingual AWE model is then applied to the segmented search corpus and
query segment, mapping them to the same fixed-dimensional vector space. We
specifically use the CAE-RNN architecture [20, 21] to train a multilingual AWE
model. This model is trained using an encoder-decoder recurrent neural network
(RNN) structure, where the encoder takes a variable-length word segment as
input to produce a fixed-dimensional output. The output vector of the encoder
is then fed as input to a decoder RNN, which reconstructs a target that is not
identical to the input but rather an instance of the same word type. This
model is trained on word pairs using labelled data from multiple languages
(but not the target language). At inference time, we take the projection of
the final encoder RNN hidden state as the AWE of an input segment.

Applying the AWE model to search segments $S$ and query segment $q$ produces
search embeddings $\mathbf{z}_{n}^{i}$ and query embedding $\mathbf{z}_{q}$,
respectively. Utterances in the search corpus can then be ranked based on the
segment-query distances $d(\mathbf{z}_{n}^{i},\mathbf{z}_{q})$. We can then
predict that all query-search pairs with a distance below some threshold are
positive matches (this requires a threshold to be tuned on development data).
With these predictions we can calculate common KWS metrics and compare them to
those achieved by the ASR KWS systems.

##  5 Experimental setup

Data. We train and evaluate our models on Swahili and Wolof data, collected
from the Common Voice [28] and ALFFA [27] datasets, respectively. For both
datasets, we use the default train, development, and test splits. In our
controlled experiments, we evaluate in-domain KWS performance on the Swahili
and Wolof test splits. The test split of both languages serves as the search
collection and is not used during model development. The Swahili search
collection contains 8 941 utterances and has a duration of approximately 14
hours, while the Wolof set contains 2 000 utterances and has a duration of
approximately 2.5 hours.

We perform our in-the-wild test on recorded radio broadcasts from three
different radio stations in Kenya. We do not have any transcriptions, speaker
information or content information on these audio clips. We segment this data
using a diarisation system that also attempts to remove music. Apart from this
preprocessing step, we simply apply our systems to this out-of-domain data
without further changes or calibration. After this preprocessing step, 19 716
utterances are obtained, which is our out-of-domain search collection. The
duration of all utterances is between three and 30 seconds. As explained
below, a human expert is used to evaluate the system's performance on this
set.

ASR model. For the ASR KWS systems (Section 3), we control the amount of
training data and train three ASR models for Wolof and Swahili. Specifically,
we fine-tune the XLS-R model [30] for each language on 30 hours, one hour, and
five minutes of labelled training data.

Table 1: Swahili hate speech keywords used for in-the-wild KWS on Swahili
radio broadcasts, with their English translations.

Swahili |  English |  Swahili |  English
---|---|---|---
vita |  war |  wezi |  thieves
damu |  blood |  majimbo |  states
hama |  move |  wakora |  conmen
kabila |  tribe |  panga |  machete
utapeli |  fraud |  takataka |  garbage
mende |  cockroaches |  mjinga |  stupid
kitendawili |  riddle |  fala |  stupid

AWE model. For the AWE-based KWS system (Section 4), we train a multilingual
AWE model on five different Common Voice languages: Abkhazian (ab), Czech
(cs), Basque (eu), Swedish (sv), Tamil (ta). Word boundaries are obtained
using the Montreal forced-aligner [32]. We pool the data from all languages
and extract 300k positive word pairs which we use to train a multilingual CAE-
RNN AWE model [20, 21]. The encoder and decoder each consist of three
unidirectional RNNs with 400-dimensional hidden vectors and an embedding size
of 100 dimensions. As inputs to the encoder, we use XLS-R to extract speech
features; we use the output of the $12^{\text{th}}$ transformer layer,
producing input features with $1024$ dimensions. Preliminary experiments
showed using these features outperforms mel-frequency cepstral coefficients in
AWE modelling. We employ Adam optimisation [33] with a learning rate of
$0.001$.

Controlled evaluation. To measure in-domain KWS performance, we apply the ASR
and AWE-based KWS systems to the same search corpora. Each utterance is
labelled as either 1 (if the keyword is present) or 0 (if the keyword is not
present). We report precision, recall and $F_{1}$-score. For the ASR KWS
systems, we simply check whether a keyword occurs in the predicted transcript.
For the AWE-based KWS, sub-segments for utterances in the search collection
are obtained by extracting windows ranging from 20 to 35 frames with a 5-frame
overlap and then applying an AWE model to each segment. We use a set of ten
templates of 36 unique keywords with a minimum character length of five for
Swahili, and 15 unique keywords with a minimum character length of four, for
Wolof. These keywords were randomly sampled from a larger set of words that
appear at least ten times in both the development and test sets. Query
templates are drawn from development data, embedded, and averaged to obtain a
single AWE embedding representing a keyword. The similarity between a query
and search segment is calculated using cosine distance. The threshold value in
the AWE-based KWS is tuned for the highest $F_{1}$-score across all keywords
on the controlled test data.

In-the-wild evaluation. For the in-the-wild KWS, we apply the systems directly
to the out-of-domain Swahili radio broadcasts. We use a set of keywords
labelled as inflammatory by expert analysts familiar with the media
environment for the purpose of hate speech detection. These keywords with
their English translations are given in Table 1. For the AWE-based KWS system,
ten query templates per keyword are extracted from the in-domain Swahili data.
We evaluate KWS performance by asking each approach to give 100 utterances out
of the search collection that are most likely to contain any of the hate
speech keywords. In the absence of per-word confidence scores for the ASR
models, here we simply take 100 random utterances that were predicted to
contain a keyword (for most models, the total number of utterances was around
150).

For the AWE-based KWS, we use the 100 highest-ranked utterances. Because we do
not have transcriptions for this data, we provide a native Swahili speaker
with untranscribed recordings of the utterances that were predicted to contain
a keyword. They then mark whether the keyword was indeed present.

##  6 Results

Table 2:  ASR and AWE KWS results (%) on in-domain test data. For the ASR
systems, the XLS-R model is fine-tuned on each target language, controlling
the amount of training data. For the AWE KWS system, a supervised multilingual
AWE is trained on multiple well-resourced languages and applied to the two
low-resource target languages.

Model | Swahili | Wolof
---|---|---
|  Prec. |  Rec. |  $F_{1}$ |  Prec. |  Rec. |  $F_{1}$
ASR: |  |  |  |  |  |
XLS-R (30-h) |  97.6 |  98.6 |  98.0 |  93.1 |  87.1 |  90.0
XLS-R (1-h) |  96.4 |  76.1 |  85.1 |  93.1 |  74.3 |  82.7
XLS-R (5-min) |  95.4 |  42.6 |  58.9 |  88.0 |  45.6 |  60.0
Multilingual AWE: |  |  |  |  |  |
CAE-RNN (ab+cs+eu+sv+ta) |  57.1 |  56.1 |  56.6 |  44.0 |  58.4 |  50.2

###  6.1 KWS results in a controlled environment

We first look at results in a controlled test on in-domain Swahili and Wolof
data. The first three lines in Table 2 report the ASR KWS results. The results
show that fine-tuning XLS-R on only five minutes of labelled data achieves
decent KWS performance, with $F_{1}$-scores of 59% and 60% for Swahili and
Wolof, respectively. The multilingual AWE system using roughly one minute of
template data, performs worse on most metrics. The only metric on which the
AWE system is better is recall, where it achieves higher scores than the
5-minute ASR system on both languages. In practice, the AWE system could
therefore be a better option if recall is important.

We briefly consider how ASR performance is affected by the amount of training
data. For reference, the word error rates (WERs) for the 30-hour, 1-hour, and
5-minute models are respectively 9%, 36%, and 62% on Swahili. For Wolof, the
WERs are 27%, 44%, and 68%. As expected, more training data gives better ASR
and KWS scores. But it is noteworthy that with just five minutes of training
data, we can already spot keywords with high precision (95% and 88% for
Swahili and Wolof, respectively). This is especially useful for our use case,
where we want to rapidly develop KWS applications in severely low-resourced
settings. The other noteworthy finding from the ASR results is that scores for
Swahili are not notability higher than Wolof, although the former is one of
XLS-R's pretraining languages.

Table 3:  Supervised AWE KWS results (%) on in-domain Swahili test data. The
supervised monolingual CAE-RNN AWE model is trained using labelled Swahili
data. It is applied with two segmentation configurations: using ground-truth
word boundaries (true segm.), and using a variable-length window which is
swept across the search collection (random segm.). Here we also report the
standard QbE metrics P@10 and P@N.

Model |  Prec. |  Rec. |  $F_{1}$ |  P@10 |  P@N
---|---|---|---|---|---
Supervised monolingual: |  |  |  |  |
CAE-RNN (true segm.) |  95.5 |  90.8 |  93.1 |  98.6 |  94.3
CAE-RNN (random segm.) |  79.2 |  76.1 |  77.6 |  92.2 |  90.9
Supervised multilingual: |  |  |  |  |
CAE-RNN ab+cs+eu+sv+ta (random segm.) |  57.1 |  56.1 |  56.6 |  87.8 |  64.7

We now turn to the AWE-based approach; specifically, we ask what the upper
bound on performance would be if we had more training data or a more idealised
search setting. Therefore, for a moment, we assume we have labelled data
available and perform top-line experiments on the in-domain Swahili data,
shown in Table 3. A supervised Swahili AWE model is trained on 30 hours of
labelled data and applied to the search collection that is segmented using
true word boundaries. Results are shown in the first row of Table 3, serving
as the top-line performance for AWE KWS, with a high $F_{1}$-score of 93%.
Compared to the 30-hour ASR system (Table 2), the idealised AWE system comes
closer to the $F_{1}$ of 98%. The second row in Table 3 shows KWS performance
using a supervised Swahili AWE model but applied to search segments extracted
by sliding a variable-length window across the search collection—the way we
apply the AWE approach in practice and in Table 2. Segmentation without true
word boundaries incurs a significant penalty, resulting in an $F_{1}$-score
drop to 77%. It is clear that the sliding window approach has a large effect
on downstream performance, so future work should consider more sophisticated
unsupervised word segmentation techniques.

###  6.2 KWS results in the wild

We now turn to our main research question: comparing ASR to multilingual AWE-
based KWS on real-life, out-of-domain audio in a low-resource setting. As
mentioned in Section 5, systems are applied to out-of-domain Swahili radio
broadcasts, after which, for each system, the top 100 utterances predicted to
contain a hate speech keyword (Table 1) are manually reviewed. The results are
given in Table 4.

The table reports precision: the proportion of retrieved top-100 utterances
that correctly contain a hate speech keyword. Surprisingly, we see that the
multilingual AWE KWS system achieves a precision of 45%—better than the
5-minute and even the 1-hour ASR system in this in-the-wild test. This is in
contrast to the in-domain KWS results, where the 1-hour ASR model outperformed
the AWE KWS system (Table 2).

Further investigation is required to understand exactly why the relative
performance of the ASR and AWE systems are affected differently when applied
to out-of-domain data. However, it is worth noting that several studies have
shown that ASR system performance can drop dramatically when it is applied to
data outside of its training domain [34, 35, 36]. One example where this can
be seen in this case is how many of the ASR retrievals contain music (which we
asked the human annotator to mark).

The in-the-wild search collection has been diarised automatically, which
included a step to remove music segments. But this preprocessing step is not
perfect: the search collection still ends up with some music (which neither
the ASR nor AWE systems have seen in training). Table 4 shows that, out of the
top-100 utterances for the 30-hour ASR system, 30% contained music. This
decreases for the 1-hour (29%) and 5-minute (17%) ASR systems. But the AWE KWS
system only retrieves two utterances containing music. And of these two, one
utterance actually did contain a hate speech keyword in the music lyrics
(there are also examples of such correct matches in the ASR music retrievals).
Nevertheless, this shows that the AWE approach seem to be more robust to
domain mismatch compared to training an ASR system on one hour or five minutes
of labelled data.

Table 4:  In-the-wild KWS results (%) on Swahili radio broadcasts. For each
system, we report the percentage of utterances retrieved containing a keyword
(precision) and the percentage of utterances retrieved containing music.

Model |  Precision |  Music
---|---|---
ASR: |  |
XLS-R (30-h) |  52 |  30
XLS-R (1-h) |  42 |  29
XLS-R (5-min) |  36 |  17
Multilingual AWE: |  |
CAE-RNN ab+cs+eu+sv+ta |  45 |  2

##  7 Conclusion

This paper considered the problem of keyword spotting (KWS) for the purpose of
hate speech detection in low-resource languages. We compare two KWS systems
for data from Swahili and Wolof: a fine-tuned automatic speech recognition
(ASR) model using different amounts of training data, and an ASR-free KWS
system that utilises multilingual acoustic word embeddings (AWEs). The results
show that fine-tuning a pre-trained multilingual ASR model using even a small
amount of labelled data can outperform an AWE-based KWS system in controlled
environments where training and test data come from the same domain. However,
the AWE-based KWS system is more robust on out-of-domain radio broadcast data
and achieves comparable results to an ASR model fine-tuned on 30 hours of
labelled data. In the end, the merits of ASR vs AWE KWS will come down to the
practical setting: it will depend on whether labelled training data can be
collected from the target domain or not, and whether precision or recall is
more important.

##  8 Acknowledgements

UPF has received funding from the European Research Council (ERC) under the
European Union's Horizon 2020 research and innovation programme (grant no.
101019291). We thank the VoxCroft Analytics machine translation and labelling
team for providing the data used for this work.

## References

  * [1] F. Chalk, ``Hate Radio in Rwanda,'' in _The Path of a Genocide: The Rwanda Crisis from Uganda to Zaire_ , 1999.
  * [2] K. Somerville, ``Violence, hate speech and inflammatory broadcasting in Kenya: The problems of definition and identification,'' _Ecquid Novi: African Journalism Studies_ , vol. 32, no. 1, pp. 82–101, 2011.
  * [3] G. Chen, C. Parada, and G. Heigold, ``Small-footprint keyword spotting using deep neural networks,'' in _Proc. ICASSP_ , 2014.
  * [4] K. Audhkhasi, A. Rosenberg, A. Sethy, B. Ramabhadran, and B. Kingsbury, ``End-to-end ASR-free keyword search from speech,'' in _Proc. ICASSP_ , 2017.
  * [5] R. Menon, H. Kamper, J. Quinn, and T. Niesler, ``Fast ASR-free and almost zero-resource keyword spotting using DTW and CNNs for humanitarian monitoring,'' in _Proc. Interspeech_ , 2018.
  * [6] E. T. Mekonnen, A. Brutti, and D. Falavigna, ``End-to-end low resource keyword spotting through character recognition and beam-search re-scoring,'' in _Proc. ICASSP_ , 2022.
  * [7] E. van der Westhuizen, H. Kamper, R. Menon, J. Quinn, and T. Niesler, ``Feature learning for efficient ASR-free keyword spotting in low-resource languages,'' _Comput. Speech Lang._ , vol. 71, p. 101275, 2022.
  * [8] A. S. Park and J. R. Glass, ``Unsupervised pattern discovery in speech,'' _IEEE Trans. Audio, Speech, Lang. Process._ , vol. 16, no. 1, pp. 186–197, 2008.
  * [9] T. J. Hazen, W. Shen, and C. White, ``Query-by-example spoken term detection using phonetic posteriorgram templates,'' in _Proc. ASRU_ , 2009.
  * [10] Y. Zhang and J. R. Glass, ``Unsupervised spoken keyword spotting via segmental DTW on Gaussian posteriorgrams,'' in _Proc. ASRU_ , 2009.
  * [11] A. Jansen and B. V. Durme, ``Indexing raw acoustic features for scalable zero resource search,'' in _Proc. Interspeech_ , 2012.
  * [12] Y.-A. Chung, C.-C. Wu, C.-H. Shen, H.-Y. Lee, and L.-S. Lee, ``Unsupervised learning of audio segment representations using sequence-to-sequence autoencoder,'' in _Proc. Interspeech_ , 2016.
  * [13] H. Kamper, W. Wang, and K. Livescu, ``Deep convolutional acoustic word embeddings using word-pair side information,'' in _Proc. ICASSP_ , 2016\.
  * [14] S. Settle and K. Livescu, ``Discriminative acoustic word embeddings: Recurrent neural network-based approaches,'' in _Proc. SLT_ , 2016.
  * [15] S. Settle, K. Levin, H. Kamper, and K. Livescu, ``Query-by-example search with discriminative neural acoustic word embeddings,'' in _Proc. Interspeech_ , 2017.
  * [16] H. Kamper, ``Truly unsupervised acoustic word embeddings using weak top-down constraints in encoder-decoder models,'' in _Proc. ICASSP_ , 2019.
  * [17] B. M. Abdullah, M. Mosbach, I. Zaitova, B. Mobius, and D. Klakow, ``Do acoustic word embeddings capture phonological similarity? An empirical study,'' in _Proc. Interspeech_ , 2021.
  * [18] D. Ram, L. Miculicich, and H. Bourlard, ``Neural network based end-to-end query by example spoken term detection,'' _IEEE Trans. Audio, Speech, Lang. Process._ , vol. 28, no. 1, pp. 1416–1427, 2019.
  * [19] Y. Hu, S. Settle, and K. Livescu, ``Multilingual jointly trained acoustic and written word embeddings,'' in _Proc. Interspeech_ , 2020.
  * [20] H. Kamper, Y. Matusevych, and S. Goldwater, ``Improved acoustic word embeddings for zero-resource languages using multilingual transfer,'' _IEEE Trans. Audio, Speech, Lang. Process._ , vol. 29, pp. 1107–1118, 2021.
  * [21] C. Jacobs, Y. Matusevych, and H. Kamper, ``Acoustic word embeddings for zero-resource languages using self-supervised contrastive learning and multilingual adaptation,'' in _Proc. SLT_ , 2021.
  * [22] C. Jacobs and H. Kamper, ``Multilingual transfer of acoustic word embeddings improves when training on languages related to the target zero-resource language,'' in _Proc. Interspeech_ , 2021.
  * [23] A. Baevski, Y. Zhou, A. Mohamed, and M. Auli, ``wav2vec 2.0: A framework for self-supervised learning of speech representations,'' in _Proc. NeurIPS_ , 2020.
  * [24] J. Zhao and W.-Q. Zhang, ``Improving automatic speech recognition performance for low-resource languages with self-supervised models,'' _IEEE J. Sel. Top. Signal Process._ , vol. 16, no. 6, pp. 1227–1241, 2022.
  * [25] L. M. Nguyen, S. Nayak, and M. Coler, ``Improving Luxembourgish speech recognition with cross-lingual speech representations,'' in _Proc. SLT_ , 2023.
  * [26] Y. Hu, S. Settle, and K. Livescu, ``Acoustic span embeddings for multilingual query-by-example search,'' in _Proc. SLT_ , 2021.
  * [27] E. Gauthier, L. Besacier, S. Voisin, M. Melese, and U. P. Elingui, ``Collecting resources in sub-Saharan African languages for automatic speech recognition: a case study of Wolof,'' in _Proc. LREC_ , 2016.
  * [28] R. Ardila, M. Branson, K. Davis, M. Henretty, M. Kohler, J. Meyer, R. Morais, L. Saunders, F. M. Tyers, and G. Weber, ``Common Voice: A massively-multilingual speech corpus,'' in _Proc. LREC_ , 2020.
  * [29] A. Saeb, R. Menon, H. Cameron, W. Kibira, J. Quinn, and T. R. Niesler, ``Very low resource radio browsing for agile developmental and humanitarian monitoring,'' in _Proc. Interspeech_ , 2017.
  * [30] A. Babu, C. Wang, A. Tjandra, K. Lakhotia, Q. Xu, N. Goyal, K. Singh, P. von Platen, Y. Saraf, J. Pino, A. Baevski, A. Conneau, and M. Auli, ``XLS-R: Self-supervised cross-lingual speech representation learning at scale,'' in _Proc. Interspeech_ , 2022.
  * [31] H. Kamper, A. Anastassiou, and K. Livescu, ``Semantic query-by-example speech search using visual grounding,'' in _Proc. ICASSP_ , 2019.
  * [32] M. McAuliffe, M. Socolof, S. Mihuc, M. Wagner, and M. Sonderegger, ``Montreal forced aligner: Trainable text-speech alignment using Kaldi,'' in _Proc. Interspeech_ , 2017.
  * [33] D. P. Kingma and J. Ba, ``Adam: A method for stochastic optimization,'' in _Proc. ICLR_ , 2017.
  * [34] M. L. Seltzer, D. Yu, and Y. Wang, ``An investigation of deep neural networks for noise robust speech recognition,'' in _Proc. ICASSP_ , 2013.
  * [35] T. Likhomanenko, Q. Xu, V. Pratap, P. Tomasello, J. Kahn, G. Avidov, R. Collobert, and G. Synnaeve, ``Rethinking evaluation in ASR: Are our models robust enough?'' _arXiv preprint arXiv:2010.11745_ , 2021.
  * [36] W.-N. Hsu, A. Sriram, A. Baevski, T. Likhomanenko, Q. Xu, V. Pratap, J. Kahn, A. Lee, R. Collobert, G. Synnaeve, and M. Auli, ``Robust wav2vec 2.0: Analyzing domain shift in self-supervised pre-training,'' _arXiv preprint arXiv:2104.01027_ , 2021.