# Heavy flavor as a probe of hot QCD matter produced in proton-proton
collisions

Jiaxing Zhao, Joerg Aichelin, Pol Bernard Gossiaux, and Klaus Werner
SUBATECH, Nantes University, IMT Atlantique, IN2P3/CNRS, 4 rue Alfred Kastler,
44307 Nantes cedex 3, France

###### Abstract

We investigate yields, elliptic flow $v_{2}$, baryon-to-meson ratios, and
correlations of various charmed hadrons in high energy proton-proton ($pp$)
collisions, employing the newly developed EPOS4HQ approach, which reproduces
basic experimental data concerning light hadrons. All the above-mentioned
heavy flavor observables can be well explained, essentially due to the
presence of deconfined matter, which in particular affects $v_{2}$
dramatically, although the energy loss is small.

_Introduction.–_ There exists ample evidence that a new deconfined phase of
matter, predicted by Lattice QCD calculations [1, 2] and called quark-gluon
plasma (QGP), is created in relativistic heavy-ion collisions. The plasma
expands rapidly leading to a continuous decrease in temperature and density.
Finally, the energy density gets below a critical value and the QGP hadronizes
into hadrons, which are finally observed. Heavy mesons, which contain either a
heavy quark $Q$ or a heavy antiquark $\bar{Q}$, have turned out to be an ideal
probe to study the QGP due to two reasons: a) The heavy quark mass is much
larger than the QCD cutoff, $m_{Q}\gg\Lambda_{QCD}$. Therefore their
production can be well described by the perturbative QCD (pQCD) and hence
their initial momentum distribution is known; b) The heavy quark mass is much
larger than the typical temperature of the hot medium, $m_{Q}\gg T_{QGP}$.
Their masses are unchanged in the hot medium and their number is conserved
during the evolution. The transverse momentum spectrum of heavy mesons (as
compared to that in proton-proton collisions) can only be understood if a QGP
is created in these collisions. This is confirmed by another key observable,
the elliptic flow, $v_{2}$, which heavy quarks can only acquire through
interactions with the QGP because initially they are formed in hard processes,
which are azimuthally isotropic.

It came as a surprise that in (high multiplicity) $pp$ collisions observations
have been made, which are considered in heavy-ion collisions as a proof of the
existence of a QGP like the observations of long-range correlations (also
called near-side “ridge”) [3, 4, 5, 6], strangeness enhancement [7], and
baryon-to-meson ratio [8], even a finite elliptic flow of $D^{0}$ mesons [9].
It was even more astonishing that the multiplicity of charmed baryons at mid-
rapidity in these collisions is considerably higher than expected from the
analysis of $e^{+}e^{-}$ collisions, questioning the process independence of
fragmentation functions [10, 11, 12, 13]. Several works have been advanced in
recent years [14, 15, 16, 17] and show the hadronization mechanism of heavy
quark in high energy $pp$ collisions is quite different from the $e^{+}e^{-}$.
The purpose of this letter is to show that the recently advanced EPOS4(HQ)
approach, which assumes the creation of a QGP if the energy density exceeds a
critical energy density, describes $pp$ collisions as well and can reproduce
quantitatively the experimental observations. Therefore $pp$ collisions are
just the low system size limit of $AA$ collisions.

_EPOS4 primary interactions.–_ In the EPOS4 approach, we distinguish “primary
interactions” and “secondary interactions”. The former refer to parallel
partonic scatterings, which happen at very high energies instantaneously, at
$t=0$ and not in any sequential ordering. The theoretical tool is a particular
form of the proton-proton scattering S-matrix. In EPOS4 [18, 19, 20] we can
accommodate simultaneously: (1) rigorous parallel scattering, (2) energy-
momentum sharing, and (3) validity of the AGK theorem [21] and factorization
[22] for hard processes, by introducing saturation, compatible with recent
“low-x-physics” considerations [23, 24, 25, 26, 27, 28, 29]. Although energy-
momentum sharing makes things complicated, it is not only mandatory for a
consistent picture, it also allows one to understand the connection between
factorization and saturation.

The validity of AGK means that we can do the same as models based on
factorization to study very hard processes, but this represents only a very
small fraction of all possible applications, and there are very interesting
cases outside the applicability of that approach. A prominent example concerns
collective phenomena in small systems, one of the most challenging theoretical
questions in our domain since it has been shown that high-multiplicity $pp$
events show very similar collective features as earlier observed in heavy ion
collisions [3]. High multiplicity means automatically “multiple parton
scattering”. Consequently, we cannot employ the usual parton distribution
functions (representing the partonic structure of a fast nucleon), but have to
treat the different scatterings (happening in parallel) individually. For each
one, we have a parton evolution according to some evolution function $E$
(representing the space-like (SL) cascade), as sketched in Fig. 1.

Figure 1: Rigorous parallel scattering scenario, here for three parallel
scatterings. The red symbols should remind us that the parts of the diagram
representing nonlinear effects (like gluon fusion) are replaced by simply
using saturation scales.

We still have DGLAP evolution [30, 31, 32], for each of the scatterings, but
we introduce saturation scales. Most importantly, these scales are not
constants, they depend on the number of scatterings as well as on $x^{+}$ and
$x^{-}$, the momentum fraction of the partons of in the two protons. [18]. In
Fig. 1, we show for simplicity only emitted gluons and not their further
development.

Concerning heavy flavor, we use the general notation of $Q$ for quarks and
$\bar{Q}$ for antiquarks. A heavy flavor may be produced in different ways, as
shown in Fig. 2.

(a) (b) (c)

Figure 2: Different possibilities to create heavy flavor, (a) in the space-
like cascade (SLC), (b) in the Born process, (c) in the time-like cascade
(TLC).

Starting from a gluon, a $Q\bar{Q}$ pair may be produced in the space-like
cascade, as shown in Fig. 2(a), provided the virtuality is large enough. The
number of allowed flavors is considered to be depending on the virtuality
(variable flavor number scheme). It is also possible to create a $Q\bar{Q}$
pair in the Born process, via $g+g\to Q+\bar{Q}$ or $q+\bar{q}\to Q+\bar{Q}$
(for light flavor quarks $q$), as shown in Fig. 2(b), and finally a $Q\bar{Q}$
pair may be produced in the time-like cascade, via $g\to Q+\bar{Q}$, as shown
in Fig. 2(c) after the hard process.

Once the parton emissions are determined, one considers the corresponding
Feynman diagram, and constructs a color flow picture, which defines chains of
partons by following the color flow, as shown in Fig. 3. These chains of
partons are then mapped (in a unique fashion) to kinky strings, where each
part corresponds to a kink.

Figure 3: The chains 1-2-3-4-5, 6-7-8, and 9-10-11 are mapped to kinky strings
(red lines). The black points indicate the kinks, which carry the parton
momenta.

For more details see [19]. The general mapping procedure (chains of partons to
kinky strings), as well as the string decay procedures into “string segments”
(which finally correspond to hadrons), are described in detail in [33]. Now we
use the term “prehadrons” for the string segments, not knowing yet if they
eventually become hadrons.

_EPOS4 core-corona method and fluid evolution.–_ From the above-mentioned
primary interactions, we obtain a number of prehadrons. We employ a core-
corona procedure [34, 35, 36], where the prehadrons, considered at a given
proper time $\tau_{0}$, are separated into “core” and “corona” prehadrons,
depending on the energy loss of each prehadron when traversing the “matter”
composed of all the others. Corona prehadrons are those that escape, whereas
core prehadrons constitute what we call “core”, which acts as an initial
condition for a hydrodynamic evolution [36, 37].

Figure 4: Energy density in the transverse plane ($x,y)$ for proton-proton
scattering involving 6 Pomerons. The left plot represents the start time
$\tau_{0}$ (of the hydro evolution), and the right plot a later time
$\tau_{1}$, close to final freeze-out.

The evolution of the core ends whenever the energy density falls below a
critical value $\epsilon_{\mathrm{FO}}=0.57\rm GeV/fm^{3}$ (the corresponding
critical temperature is $T_{FO}\approx 167\rm MeV$), which marks the point
where the fluid “decays” into hadrons. It is not a switch from fluid to
particles, it is a sudden decay, called “hadronization”. The details of the
energy-momentum flow and the energy and momentum conserving the creation of
hadrons through the “freeze-out (FO) hypersurface” at
$\epsilon_{\mathrm{FO}}$, can be found in [20]. The heavy quark hadronization
will be described later.

Let us consider a (randomly chosen, but typical) 7 TeV proton-proton
scattering event with 6 Pomerons (representing roughly three times the
average). In Fig. 4, we plot the energy density in the transverse plane
($x,y)$. We consider two snapshots, namely at the start time of the hydro
evolution $\tau_{0}=0.40\,\mathrm{fm/c}$ (left plot) and a later time
$\tau_{1}$ close to final freeze-out (right plot). The initial distribution
has an elongated shape (just by accident, due to the random positions of
interacting partons). One can clearly see that the final distribution is as
well elongated, but perpendicular to the initial ones, as expected in a
hydrodynamical expansion.

_Heavy quark energy loss in EPOS4HQ.–_ In contradistinction to EPOS4, in
EPOS4HQ heavy quarks interact with the partons of the equilibrated fluid,
formed by the core partons. Heavy quarks will lose or gain energy by
interacting with the thermal core partons. Both, collisional [38] and
radiative energy loss [39], are included in this study. The scattering cross
sections with gluons and quarks are calculated by pQCD with a running coupling
constant. The pQCD cross sections diverge for small momentum transfers in the
$t$ and $u$ channels. These infrared divergences are healed by the Debye
screening mass $m_{D}(T)$ of gluons in the hot medium, which is given by the
hard thermal loop (HTL) approach. Scattering at high momentum transfer can be
described by free propagators whereas at a low momentum transfer HTL
propagators are required. A smooth transition between both regimes can be
assured by an effective Debye mass $m_{\rm eff}=\kappa m_{D}(T)$ in the gluon
propagator [38] with $\kappa=0.2$. This approach has successfully described
heavy meson data in heavy-ion collisions at LHC energies with EPOS3 [40, 41]
and therefore we employ it also in this new EPOS4 version.

_Heavy flavor hadronization in EPOS4HQ.–_ When the heavy quark crosses the
freeze-out hypersurface, it will convert into colorless heavy flavor hadrons.
This process is indeed a non-perturbative process and is usually related to
soft gluon radiation. Two processes contribute in EPOS4HQ to hadronization:
(i) fragmentation which is also employed in $e^{+}e^{-}$ collisions and in
standard EPOS4 and (ii) coalescence, when the Q combines at the moment of
hadronization with one/two nearby core partons to form final-state mesons or
baryons.

In the coalescence model, the heavy quarks coalesce with light quarks in the
same cell to a hadron m when the cell passes the critical temperature. The
light (anti) quarks are assumed to be thermalized in the cell rest system. The
probability that a heavy hadron is formed is given by :

$\displaystyle{dN\over d^{3}{\bf P}}$ $\displaystyle=$ $\displaystyle
g_{H}\int\prod_{i=1}^{k}{d^{3}p_{i}\over(2\pi)^{3}E_{i}}p_{i}\cdot
d\Sigma_{i}F({\bf p}_{1},..,{\bf p}_{k})$ (1) $\displaystyle\times$
$\displaystyle W_{m}({\bf p}_{1},..,{\bf p}_{k})\,\delta^{(3)}\left({\bf
P}-\sum_{i=1}^{N}{\bf p}_{i}\right),$

where $g_{H}$ is the degeneracy factor of color and spin. ${\bf P}$ and ${\bf
p}_{i}$ are the momenta of heavy flavor hadrons and constituent quarks,
respectively. The delta function conserves the momentum. $d\Sigma_{i}$ is the
hypersurface element. $F({\bf p}_{1},..,{\bf p}_{k})$ with $k=2(3)$ for mesons
(baryons) is the momentum space distribution of the constituents of the heavy
hadron. $W_{m}({\bf p}_{1},..,{\bf p}_{k})$ is the Wigner density of a given
heavy hadron $m$, which can be constructed from the hadron wave function and
which is approximated here by three-dimensional harmonic oscillator states
with the same root mean square radius. For the ground state, we find

$\displaystyle W(p_{r})$ $\displaystyle=$ $\displaystyle
g_{h}{(2\sqrt{\pi}\sigma)^{3}\over V}e^{-\sigma^{2}{p_{r}}^{2}}.$ (2)

$g_{h}$ is the statistical factor, $p_{r}$ is the relative momentum between
the two constituent quarks in the center-of-mass (CM) frame,
$p_{r}={|E_{2}{\bf p}_{1}-E_{1}{\bf p}_{2}|\over E_{1}+E_{2}}$, $E_{1}(p_{1})$
and $E_{2}(p2)$ are the energies and momenta of the quark or antiquark in the
CM frame, respectively. Baryons are treated as two two-body systems (baryons
are produced by recombining two particles first and then using their center of
mass to recombine with the third one.).

Figure 5: Momentum-dependent coalescence probability of charm quark in a hot
and static medium with temperature $T=165\rm MeV$. Here $D$ is the summation
of $D^{0}$ and $D^{+}$.

Instead of calculating the Wigner density of all excited states explicitly,
their contribution is included with the help of the statistic model in which
the density of hadrons $i$ at the temperature $T_{c}$ is given by [42],
$n_{i}={g_{i}\over 2\pi^{2}}T_{c}m_{i}^{2}K_{2}\left(m_{i}\over T_{c}\right)$.
$g_{i}$ is the degeneracy, which considers both spin and isospin. $m_{i}$ is
the mass of the hadron. $K_{2}$ is the second-order Bessel function and
$T_{c}$ is the temperature corresponding to the critical energy density. The
contribution of the excited states is then given by $R=n_{\rm excited}/n_{\rm
ground}$. In our study, we consider almost all possible excited states, also
the missing baryons, which are predicted by the quark model [43] and lattice
QCD [44, 45].

The light quarks are assumed to be thermalized. The quark masses are
$m_{u/d}=0.1\rm GeV$, $m_{s}=0.3\rm GeV$, $m_{c}=1.5\rm GeV$, and
$m_{b}=4.5\rm GeV$. The heavy quark coalescence probability $P_{\rm coal}$ in
a static hot medium with a temperature $T=165\rm MeV$ is obtained by
integrating Eq. 1 with a delta distribution $\delta^{3}({\bf p}_{i}-{\bf
p}_{c})$ for heavy quark and a thermal distribution for light quark. Here, we
take a constant root-mean-square radius $\sqrt{\langle r^{2}\rangle}=0.94\rm
fm$, which is actually the $D^{0}$ radius at 165MeV calculated by two-body
Dirac equation [46], for all charmed mesons and the two-quark system in
baryons. Heavy quarks, which do not hadronize via the coalescence, will
fragment into a heavy-flavor hadron. The fragmentation probability is
therefore $1-P_{\rm coal}$. The fragmentation we used comes from the heavy
quark effective theory [47, 48] and the fragmentation ratios to various
charmed hadrons are taken as the $e^{+}e^{-}$ collisions [49].

_Results and analysis.–_ When in $pp$ collisions a QGP is formed, because
locally the energy density exceeds the critical energy density, there are two
ways in which heavy hadrons are formed: by fragmentation and by coalescence,
as described in the last section. It is the coalescence contribution that
changes the observables as compared to low multiplicity $pp$ events. We focus
in the following on four observables, namely the transverse momentum spectra,
yield ratios, elliptic flow, and correlations.

Figure 6: $p_{T}$ spectrum of $D^{0}$, $D^{+}$, $D_{s}$, $\Lambda_{c}$,
$\Xi_{c}^{0}$, and $\Omega_{c}$ in 5.02TeV $pp$ collisions. The experiment
data are from the ALICE [10, 11] and CMS [12].

Transverse momentum spectrum: Our results can well reproduce the experimental
measured transverse momentum spectra of $D^{0}$, $D^{+}$, $D_{s}$,
$\Lambda_{c}$, $\Xi_{c}^{0}$, and $\Omega_{c}$ up to 20 GeV, as shown in Fig.
6.

Figure 7: Yield ratio of $\Lambda_{c}/D^{0}$ and $\Xi_{c}^{0}/D^{0}$ as a
function of transvers momentum $p_{T}$ in 5.02TeV $pp$ collisions. The
experiment data are from the ALICE [10, 11] and CMS [12].

Yield ratio: The measured yield ratio of charmed baryons and mesons,
$\Lambda_{c}/D^{0}$ and $\Xi_{c}^{0}/D^{0}$, at low $p_{T}$ in $pp$ collisions
[3], show a strong enhancement compared to the expectations based on the
fragmentation function extracted from $e^{+}e^{-}$ collisions. Our EPOS4HQ
results can well reproduce this enhancement, as shown in Fig. 7, whereas
EPOS4, where hadrons are produced exclusively via fragmentation, does not show
such an enhancement. This indicates that the origin of the enhancement is the
rescattering of heavy quarks with core particles and the subsequent
hadronization via coalescence.

Figure 8: Elliptic flow of charm quarks at creation, after energy loss, after
hadronization, and light hadron $K$. The experimental data ($v_{2}\\{2\\}$) of
kaons is from the CMS [6]. Figure 9: $D^{0}$ meson $v_{2}$ as a function of
transvers momentum $p_{T}$ in 13TeV $pp$ collisions. The experiment data are
from the ATLAS [50] and CMS [9].

Elliptic flow $v_{2}$ of $D^{0}$ meson: Initially heavy quarks show an
isotropic distribution in the azimuthal direction, see black dotted line in
Fig. 8. They gain $v_{2}$ by interactions with the core partons before
hadronization (magenta-dashed line). Final hadronic interactions modify the
distribution only a little (red line). To be sure that this is not an artifact
due to a too high $v_{2}$ value of the core partons, we compare in Fig. 8 as
well $v_{2}(p_{T})$ of kaons (orange line) with experiment and find that our
calculation agrees with the experimental finding. Hence the $v_{2}$ of the
core partons is correctly reproduced. We note that $v_{2}(p_{T})$ of
$D$-mesons is well below the $v_{2}(p_{T})$ of the kaons. If we trigger the
experimental multiplicity bin, we can compare our calculations with the
experimental data of the ATLAS and CMS collaboration. This is shown in Fig. 9.
The calculated (as compared to the kaons) reduced $v_{2}$ values agree
quantitatively with the experimental data for different multiplicities as
measured by two different collaborations. In standard EPOS4 calculations,
where heavy hadrons are produced by fragmentation, $v_{2}$ of heavy hadrons is
zero. Due to the dominance of the t-channel elastic scattering, we can obtain
a finite $v_{2}$, even if the energy loss of the heavy quarks is small, as
shown in Fig 6 where the black line is charm quark distribution at creation
while blue dashed line is that before the hadronization.

Figure 10:  $c\bar{c}$ and $D\bar{D}$ correlations in 5.02TeV $pp$ collisions.
A selection of $p_{T}>3$ GeV/c for both trigger and associated particles is
used in all cases.

Correlations: Pure EPOS4 reproduces well the correlations both $DD$ and
$D\bar{D}$ [19]. As can be seen from Fig. 10 neither the interaction of heavy
quarks with core particles (blue-dashed line) nor the subsequent hadronic
rescattering (red line) have an influence on these correlations and therefore
EPOS4 and the EPOS4HQ give the same result.

_Conclusion.–_ We have investigated the yield of heavy baryons, the elliptic
flow $v_{2}$, and the correlation of heavy flavor hadrons in $pp$ collisions,
employing the new EPOS4HQ approach, which reproduces the light hadron
observables in $pp$ as well as in heavy-ion collisions. We find that also in
$pp$ collisions an energy density can be reached at which a QGP is formed. The
creation of a QGP and its subsequent hadronization modifies the $v_{2}$ of
$D$-mesons as well as the heavy baryon/meson ratio, bringing both in agreement
with the experimental findings. We can therefore conclude that, independent of
the system size, the observables can be well understood assuming that there is
system independent critical energy density for the creation of a QGP. If this
happens in $pp$ collisions hadron production by standard fragmentation is
complemented by coalescence. Both together can well describe the experimental
data.

Acknowledgement: This work is funded by the European Union’s Horizon 2020
research and innovation program under grant agreement No. 824093
(STRONG-2020).

## References

  * Bernard  _et al._ [2005] C. Bernard, T. Burch, E. B. Gregory, D. Toussaint, C. E. DeTar, J. Osborn, S. Gottlieb, U. M. Heller, and R. Sugar (MILC), Phys. Rev. D 71, 034504 (2005), arXiv:hep-lat/0405029 .
  * Bazavov  _et al._ [2012] A. Bazavov _et al._ , Phys. Rev. D 85, 054503 (2012), arXiv:1111.1710 [hep-lat] .
  * Khachatryan  _et al._ [2010] V. Khachatryan _et al._ (CMS), JHEP 09, 091 (2010), arXiv:1009.4122 [hep-ex] .
  * Aad  _et al._ [2016] G. Aad _et al._ (ATLAS), Phys. Rev. Lett. 116, 172301 (2016), arXiv:1509.04776 [hep-ex] .
  * Khachatryan  _et al._ [2016] V. Khachatryan _et al._ (CMS), Phys. Rev. Lett. 116, 172302 (2016), arXiv:1510.03068 [nucl-ex] .
  * Khachatryan  _et al._ [2017] V. Khachatryan _et al._ (CMS), Phys. Lett. B 765, 193 (2017), arXiv:1606.06198 [nucl-ex] .
  * Adam  _et al._ [2017] J. Adam _et al._ (ALICE), Nature Phys. 13, 535 (2017), arXiv:1606.07424 [nucl-ex] .
  * Acharya  _et al._ [2019] S. Acharya _et al._ (ALICE), Phys. Rev. C 99, 024906 (2019), arXiv:1807.11321 [nucl-ex] .
  * Sirunyan  _et al._ [2021] A. M. Sirunyan _et al._ (CMS), Phys. Lett. B 813, 136036 (2021), arXiv:2009.07065 [hep-ex] .
  * Acharya  _et al._ [2021a] S. Acharya _et al._ (ALICE), Phys. Rev. Lett. 127, 202301 (2021a), arXiv:2011.06078 [nucl-ex] .
  * Acharya  _et al._ [2021b] S. Acharya _et al._ (ALICE), JHEP 10, 159 (2021b), arXiv:2105.05616 [nucl-ex] .
  * Tumasyan  _et al._ [2023] A. Tumasyan _et al._ (CMS), (2023), arXiv:2307.11186 [nucl-ex] .
  * Acharya  _et al._ [2023] S. Acharya _et al._ (ALICE), (2023), arXiv:2308.04873 [hep-ex] .
  * He and Rapp [2019] M. He and R. Rapp, Phys. Lett. B 795, 117 (2019), arXiv:1902.08889 [nucl-th] .
  * Minissale  _et al._ [2021] V. Minissale, S. Plumari, and V. Greco, Phys. Lett. B 821, 136622 (2021), arXiv:2012.12001 [hep-ph] .
  * Li  _et al._ [2021] H.-h. Li, F.-l. Shao, and J. Song, Chin. Phys. C 45, 113105 (2021), arXiv:2103.14900 [hep-ph] .
  * Beraudo  _et al._ [2023] A. Beraudo, A. De Pace, D. Pablos, F. Prino, M. Monteno, and M. Nardi, (2023), arXiv:2306.02152 [hep-ph] .
  * Werner [2023a] K. Werner, (2023a), arXiv:2301.12517 [hep-ph] .
  * Werner and Guiot [2023] K. Werner and B. Guiot, Phys. Rev. C 108, 034904 (2023), arXiv:2306.02396 [hep-ph] .
  * Werner [2023b] K. Werner, (2023b), arXiv:2306.10277 [hep-ph] .
  * Abramovsky  _et al._ [1973] V. A. Abramovsky, V. N. Gribov, and O. V. Kancheli, Yad. Fiz. 18, 595 (1973).
  * Collins  _et al._ [1989] J. Collins, D. Soper, and G. Sterman, in Perturbative Quantum Chromodynamics, edited by A.H. Mueller, World Scientific, Singapore (1989).
  * Gribov  _et al._ [1983] L. V. Gribov, E. M. Levin, and M. G. Ryskin, Phys. Rept. 100, 1 (1983).
  * McLerran and Venugopalan [1994] L. D. McLerran and R. Venugopalan, Phys. Rev. D 49, 3352 (1994), arXiv:hep-ph/9311205 .
  * Kovner  _et al._ [1995] A. Kovner, L. McLerran, and H. Weigert, Phys. Rev. D52, 3809 (1995), hep-ph/9505320 .
  * Kovchegov [1996] Y. V. Kovchegov, Phys. Rev. D54, 5463 (1996), hep-ph/9605446 .
  * Jalilian-Marian  _et al._ [1997a] J. Jalilian-Marian, A. Kovner, L. McLerran, and H. Weigert, Phys. Rev. D55, 5414 (1997a), hep-ph/9606337 .
  * Jalilian-Marian  _et al._ [1997b] J. Jalilian-Marian, A. Kovner, A. Leonidov, and H. Weigert, Nucl. Phys. B504, 415 (1997b), hep-ph/9701284 .
  * Jalilian-Marian  _et al._ [1999] J. Jalilian-Marian, A. Kovner, A. Leonidov, and H. Weigert, Phys. Rev. D59, 034007 (1999), hep-ph/9807462 .
  * Gribov and Lipatov [1972] V. N. Gribov and L. N. Lipatov, Sov. J. Nucl. Phys. 15, 438 (1972).
  * Altarelli and Parisi [1977] G. Altarelli and G. Parisi, Nuclear Physics B. 126, 298 (1977).
  * Dokshitzer [1977] Y. L. Dokshitzer, Sov. Phys. JETP 46, 641 (1977).
  * Drescher  _et al._ [2001] H. J. Drescher, M. Hladik, S. Ostapchenko, T. Pierog, and K. Werner, Phys. Rept. 350, 93 (2001), arXiv:hep-ph/0007198 .
  * Werner [2007] K. Werner, Phys. Rev. Lett. 98, 152301 (2007), arXiv:0704.1270 [nucl-th] .
  * Werner  _et al._ [2010] K. Werner, I. Karpenko, T. Pierog, M. Bleicher, and K. Mikhailov, Phys. Rev. C 82, 044904 (2010), arXiv:1004.0805 [nucl-th] .
  * Werner  _et al._ [2014] K. Werner, B. Guiot, I. Karpenko, and T. Pierog, Phys. Rev. C 89, 064903 (2014), arXiv:1312.1233 [nucl-th] .
  * Karpenko  _et al._ [2014] I. Karpenko, P. Huovinen, and M. Bleicher, Computer Physics Communications 185, 3016 (2014), arXiv:1312.4160 .
  * Gossiaux and Aichelin [2008] P. B. Gossiaux and J. Aichelin, Phys. Rev. C 78, 014904 (2008), arXiv:0802.2525 [hep-ph] .
  * Aichelin  _et al._ [2014] J. Aichelin, P. B. Gossiaux, and T. Gousset, Phys. Rev. D 89, 074018 (2014), arXiv:1307.5270 [hep-ph] .
  * Gossiaux  _et al._ [2009] P. B. Gossiaux, R. Bierkandt, and J. Aichelin, Phys. Rev. C 79, 044906 (2009), arXiv:0901.0946 [hep-ph] .
  * Gossiaux  _et al._ [2010] P. B. Gossiaux, J. Aichelin, T. Gousset, and V. Guiho, J. Phys. G 37, 094019 (2010), arXiv:1001.4166 [hep-ph] .
  * Andronic  _et al._ [2008] A. Andronic, P. Braun-Munzinger, K. Redlich, and J. Stachel, Phys. Lett. B 659, 149 (2008), arXiv:0708.1488 [nucl-th] .
  * Ebert  _et al._ [2011] D. Ebert, R. N. Faustov, and V. O. Galkin, Phys. Rev. D 84, 014025 (2011), arXiv:1105.0583 [hep-ph] .
  * Bazavov  _et al._ [2014] A. Bazavov _et al._ , Phys. Lett. B 737, 210 (2014), arXiv:1404.4043 [hep-lat] .
  * Padmanath  _et al._ [2015] M. Padmanath, R. G. Edwards, N. Mathur, and M. J. Peardon, PoS LATTICE2014, 084 (2015), arXiv:1410.8791 [hep-lat] .
  * Zhao  _et al._ [2018] J. Zhao, S. Shi, N. Xu, and P. Zhuang, (2018), arXiv:1805.10858 [hep-ph] .
  * Braaten  _et al._ [1995] E. Braaten, K.-m. Cheung, S. Fleming, and T. C. Yuan, Phys. Rev. D 51, 4819 (1995), arXiv:hep-ph/9409316 .
  * Cacciari  _et al._ [2005] M. Cacciari, P. Nason, and R. Vogt, Phys. Rev. Lett. 95, 122001 (2005), arXiv:hep-ph/0502203 .
  * Lisovyi  _et al._ [2016] M. Lisovyi, A. Verbytskyi, and O. Zenaiev, Eur. Phys. J. C 76, 397 (2016), arXiv:1509.01061 [hep-ex] .
  * Aad  _et al._ [2020] G. Aad _et al._ (ATLAS), Phys. Rev. Lett. 124, 082301 (2020), arXiv:1909.01650 [nucl-ex] .


∎

e1Corresponding author<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>
11institutetext: Department of Physics, Azarbaijan Shahid Madani University,
53714-161, Tabriz, Iran

# Solutions of generalized supergravity equations with the BTZ black hole
metric

Ali Eghbalie1  Simin Ghasemi-Sorkhabie2  Adel Rezaei-Aghdame3

(Received: date / Accepted: date)

††journal: TTMP

## Abstract

We proceed to investigate the solutions of generalized supergravity equations
(GSE) in three dimensions. Our candidate is the metric of BTZ black hole. It
is shown that only the cases with $J=M=0$ and $J=0,~{}M\neq 0$ of the BTZ
metric satisfy the GSE. In the former, we find a family of solutions including
the field strength $H_{{}_{r\varphi t}}=2r/l$, the cosmological constant
$\Lambda=-1/l^{2}$, one-form $Z_{\mu}$ and a vector field which is obtained to
be a linear combination of the directions of the time translation and
rotational symmetries. In the latter, the solutions possess the same field
strength as before, while the cosmological constant $\Lambda$, one-form
$Z_{\mu}$ and vector field $I$ will be different from the previous case.
Finally, we show that the charged black string solution found by Horne and
Horowitz, which is Abelian T-dual to the the BTZ black hole solution, can be
considered as a solution for the GSE.

Keywords: Generalized supergravity equations; Standard supergravity; BTZ
metric

##  1  Introduction

Supergravity is a modern field theory combining the principles of
supersymmetry and general relativity. The 10- dimensional supergravity theory
describes the dynamics of massless string excitations and arises in string
theory as a low-energy effective theory. Recently, new string backgrounds have
been discovered that satisfy a more general set of motion equations than
ordinary supergravity. These equations, which are a generalization of the
standard supergravity equations, are called the GSE. It’s worth noting that
the primary difference between standard supergravity and GSE is the absence of
a scalar dilaton. Arutyunov, et al. r1  introduced the GSE in order to
investigate integrable deformations of the $AdS_{5}\times S^{5}$ type II
superstring sigma model r2 ; r3 111In order to construct the Yang-Baxter
deformations of $AdS_{5}\times S^{5}$, one must employ the classical r-matrix
as a solution of the homogeneous classical Yang-Baxter equation r4 . It has
been shown that the Yang-Baxter deformed background of $AdS_{5}\times S^{5}$
satisfies the equations of motion of type IIB supergravity if the classical
r-matrix satisfies the unimodularity condition r5  (see, also, Sheikh-Jabbari1
; Sheikh-Jabbari2 ; Sheikh-Jabbari3 ; Sheikh-Jabbari4 ). Otherwise, the
background is a solution of the GSE., which is closely related to non-Abelian
T-duality transformations r6 ; r7 ; r8 ; r9 . This generalized system in
string theory includes additional vector fields $I$ in addition to the
standard component fields of type IIB supergravity. Up to now, the
corresponding classical action has not been discovered, and only the equations
of motion are presented. Additionally, Ref. r11  presents a solution of
standard supergravity with a linear dilaton that has been mapped to a GSE
solution through a formal T-duality transformation along a specific direction.
These findings emphasize the importance of treating solutions of standard
supergravity and GSE equally in the context of string theory, as T-duality is
a fundamental symmetry in string theory. As a great progress in the recent
study of string theory, Tseytlin and Wulff r10  demonstrated that the GSE can
be reproduced by solving the $\kappa$-symmetry constraints. In fact, their
results show that the $\kappa$-symmetry of the Green-Schwarz action requires
the background supergravity fields to satisfy the GSE.

Later, the GSE attracted the attention of many researchers, in such a way that
it was shown that yuho1  the whole bosonic part of the type II GSE can be
reproduced from the T-duality covariant equations of motion of the double
field theory when one chooses a non-standard solution of the strong
constraint. Additionally, in Ref. yuho1 , the Weyl invariance of the bosonic
sigma model on a generalized gravity background has been shown by using the
doubled formalism. We think that the results of work of Ref. yuho1  have
provided positive evidence that superstring theories defined on solutions of
the GSE are Weyl invariant (see, also, yuho1.1 ). In Ref. yuho2 , without
introducing a T-duality manifest formulation of string theory, it has been
constructed a local counterterm that cancels out the Weyl anomaly of bosonic
string theory defined in generalized supergravity backgrounds; the results of
their work support the Weyl invariance of string theory in generalized
supergravity backgrounds. Also, it has been shown that the equations of motion
of generalized supergravity can be followed from the sigma model once the
Killing vector $I$ is identified with the trace of the structure constants r8
. In this regard, for the Bianchi spacetimes, it was shown that r8  the non-
Abelian T-duals with respect to non-semisimple groups are solutions to
generalized supergravity.

As a spin off from this progress, it would be interesting to present new
solutions for the GSE in three dimensions. In this work, we obtain some new
solutions for the GSE, including special cases of the BTZ metric ($J=M=0$ and
$J=0,~{}M\neq 0$). Notably, when we select the vector field $I$ based on our
perspective, it becomes evident that the cases with $J\neq 0$, $M=0$ and
$J\neq 0,~{}M\neq 0$ do not satisfy the GSE. As an interesting result, we show
that the charged black string solution found by Horne and Horowitz Horowitz1 ,
which is Abelian T-dual to the the BTZ black hole solution, can be considered
as a solution for the GSE.

The structure of the paper is as follows. We start with a short overview of
the GSE in section 2, and introduce our notation. Next, in section 3, after
the introduction of the BTZ black hole metric, we consider the BTZ black hole
solutions in the context of the low energy approximation. Section 4 contains
the original results of the work: we look into the BTZ metrics in the context
of the GSE and show that only the cases $J=M=0$ and $J=0,~{}M\neq 0$ of the
BTZ metric can satisfy the GSE. In section 5, the charged black string
solution is considered as a solution for the GSE. Finally, conclusion is
reported in section 6.

##  2  A brief review of the GSE

The subject of this section is a brief overview of the GSE. In the absence of
Ramond-Ramond fields, these equations in D dimensions take the following form
r1

$\displaystyle
R_{\mu\nu}-\frac{1}{4}~{}H_{\mu\rho\sigma}~{}H^{\rho\sigma}_{\nu}+(\nabla_{\mu}~{}X_{\nu}+\nabla_{\nu}~{}X_{\mu})=0,$
(1)
$\displaystyle\frac{1}{2}~{}\nabla^{\lambda}~{}H_{\lambda\mu\nu}-X^{\lambda}~{}H_{\lambda\mu\nu}-\nabla_{\mu}~{}X_{\nu}+\nabla_{\nu}~{}X_{\mu}=0,$
(2) $\displaystyle
R-\frac{1}{12}H^{2}+4\nabla_{\mu}~{}X^{\mu}-4X_{\mu}~{}X^{\mu}-4~{}\Lambda=0,$
(3)

where $R_{\mu\nu}$ and $R$ are the respective Ricci tensor and Gauss curvature
that are calculated from the metric $G_{\mu\nu}$, and $\Lambda$ is the
cosmological constant. Here, the $D$-dimensional indices $\mu,~{}\nu,...$ of
coordinates $x^{\mu}$ are raised or lowered with the metric $G_{\mu\nu}$. The
covariant derivative $\nabla_{\mu}$ is the conventional Levi-Civita connection
associated with $G_{\mu\nu}$. The field strength $H_{\mu\nu\rho}$
corresponding to anti-symmetry tensor field B is defined as

$H_{\mu\nu\rho}=\partial_{\mu}~{}B_{\nu\rho}+\partial_{\nu}~{}B_{\rho\mu}+\partial_{\rho}~{}B_{\mu\nu}.$
(4)

In addition, $X_{\mu}$ is defined to be $X_{\mu}=I_{\mu}+Z_{\mu}$ in which
$I=I^{\mu}~{}\partial_{\mu}$ is a vector field, while $Z=Z_{\mu}~{}dx^{\mu}$
is a one-form. They satisfy r1

$\displaystyle\mathcal{L}_{{}_{I}}~{}G_{\mu\nu}=0,$ (5)
$\displaystyle\mathcal{L}_{{}_{I}}~{}B_{\mu\nu}=0,$ (6)
$\displaystyle\nabla_{\mu}~{}Z_{\nu}-\nabla_{\nu}~{}Z_{\mu}+I^{\lambda}~{}H_{\lambda\mu\nu}=0,$
(7) $\displaystyle I^{\lambda}~{}Z_{\lambda}=0,$ (8)

where $\mathcal{L}$ stands for the Lie derivative. The conventional dilaton is
included in $Z_{\mu}$ as follows:

$Z_{\mu}=\partial_{\mu}~{}\Phi+B_{\nu\mu}~{}I^{\nu}.$ (9)

Here, $\Phi$ is a scalar dilaton field hidden within $Z_{\mu}$. Note that if
we set $I^{\mu}=0$, then one gets that $X_{\mu}=\partial_{\mu}~{}\Phi$, and
thus, the GSE reduce to the standard supergravity equations.

In this work, we present new solutions for the GSE (1)-(3) together with
(5)-(8), including special cases of the BTZ metric, field strength $H$, one-
form $Z_{\mu}$, and an appropriate vector field $I$. In this manner, we will
derive a family of solutions for the cases $J=0,~{}M=0$ and $J=0,~{}M\neq 0$
within the BTZ metric. As mentioned earlier, the case $I^{\mu}=0$ of the GSE
corresponds to the standard supergravity equations. It has already been shown
that Horowitz  a slight modification of the BTZ black hole solution yields an
exact solution to the standard supergravity equations. In the next section,
after a brief overview of the BTZ black hole, we consider the BTZ metric in
the context of the low energy approximation.

##  3  The BTZ black hole metric as a solution of the standard supergravity
equations

First of all, let us introduce the metric of BTZ black hole. The BTZ black
hole, discovered by Banados, Teitelboim, and Zanelli r12 , is a
$2+1$-dimensional solution of Einstein’s equations with negative cosmological
constant, mass, angular momentum, and charge. Unlike its higher-dimensional
counterparts, the BTZ black hole is asymptotically anti-de Sitter and lacks a
curvature singularity at the origin. The line element for the black hole
solutions is as follows r12 :

$\displaystyle ds^{2}$ $\displaystyle=$
$\displaystyle(M-\frac{r^{2}}{l^{2}})~{}dt^{2}-J~{}dt~{}d\varphi+r^{2}~{}d\varphi^{2}$
(10) $\displaystyle+$
$\displaystyle(\frac{r^{2}}{l^{2}}-M+\frac{J^{2}}{4r^{2}})^{-1}~{}dr^{2},~{}~{}~{}~{}~{}~{}0\leq\varphi\leq
2\pi.$

where the radius $l$ is related to the cosmological constant by
$l=(-\Lambda)^{-1/2}$. The constants of motion, denoted by $M$ and $J$,
represent the mass and angular momentum of the BTZ black hole, respectively.
The line element (10) describes a black hole solution with outer and inner
horizons at $r=r_{+}$ and $r=r_{-}$ , respectively,

$r_{\pm}=l\bigl{(}{{M}\over
2}\bigr{)}^{1/2}\biggl{\\{}1\pm\biggl{(}1-{{J^{2}}\over{{M^{2}}l^{2}}}\biggr{)}^{1/2}\biggr{\\}}^{1/2}.$
(11)

The mass $M$ and angular momentum $J$ are related to $r=r_{\pm}$ by
$M=(r_{+}^{2}+r_{-}^{2})/l^{2}$ and $J=2~{}r_{+}~{}r_{-}/l^{2}$. Solutions
with $-1<M<0$ and $J=0$ describe point particle sources with naked conical
singularities at $r=0$. The metric with $J=0$ and $M=-1$ can be recognized as
ordinary anti-de Sitter space, and it is separated by a mass gap from the case
where $J=0$ and $M=0$. The vacuum state, representing empty space, is obtained
by letting the horizon size go to zero. This corresponds to $M\rightarrow 0$,
which requires $J\rightarrow 0$. It’s worth noting that the metric for the
$J=0$ and $M=0$ black hole is not the same as the $AdS_{3}$ metric, which has
negative mass, $M=-1$.

As mentioned in the previous section, a modified BTZ black hole solution was
obtained to a $2+1$-dimensional string theory with a matter source given by
anti-symmetric $B$-field with the contribution of the cosmological constant
$\Lambda$ Horowitz . There, Horowitz and Welch showed that very solution to
3-dimensional general relativity with $\Lambda<0$ can be considered as a
solution to the standard supergravity equations with a constant dilaton field,
$\Lambda=-1/l^{2}$ and field strength
$H_{{}_{\mu\nu\rho}}=2\epsilon_{{}_{\mu\nu\rho}}/l$, where
$\epsilon_{{}_{\mu\nu\rho}}$ stands for the volume form in three
dimensions222Note that a special property of three dimensions is that the
field strength $H_{{}_{\mu\nu\rho}}$ must be proportional to the volume form
$\epsilon_{{}_{\mu\nu\rho}}$.. In addition, it was shown that Horowitz  the
solution of BTZ black hole (10) along with a constant dilaton field,
$\Lambda=-1/l^{2}$ and field strength $H_{{}_{r\varphi t}}=2r/l$ satisfy the
equations of motion of the standard supergravity. It is worth mentioning that
3-dimensional black hole solutions to (10) do not exist assumed that
$H_{{}_{\mu\nu\rho}}=0$.

##  4  Solutions of the GSE with the BTZ metric

In what follows we shall look into the BTZ solutions in the context of the
GSE. We show that the cases $J=M=0$ and $J=0,~{}M\neq 0$ of the BTZ metric can
be considered as solutions of the GSE. As mentioned in section 3, the ${M}$
and $J$ are the mass and angular momentum of the BTZ black hole, respectively.
They are appeared due to the time translation symmetry and rotational symmetry
of the metric, corresponding to the Killing vectors ${\partial/\partial t}$
and ${\partial/\partial\varphi}$, respectively. On the other hand, relation
(5) is called Killing equation, which in terms of a Killing vector field
$K_{a}=K_{a}^{~{}\mu}\partial_{\mu}$ it can be written as

$\displaystyle\partial_{\mu}K_{a}^{~{}\lambda}~{}G_{{}_{\lambda\nu}}+K_{a}^{~{}\lambda}\partial_{\lambda}G_{{}_{\mu\nu}}+\partial_{\nu}K_{a}^{~{}\lambda}~{}G_{{}_{\mu\lambda}}=0.$
(12)

Accordingly, the vector field $I$ can be a Killing vector or a linear
combination of the Killing vectors corresponding to metric (10). The Killing
vectors $K_{a}$ corresponding to the BTZ metric can be derived by solving
Killing equation (12), giving us six linearly independent vectors. Here, to
construct an appropriate vector field $I$ we use the Killing vectors of the
BTZ metric. We assume that the choice of

$\displaystyle I=\alpha_{1}K_{1}+\cdots+\alpha_{6}K_{6},$ (13)

for some constants $\alpha_{i}$, can be a suitable candidate for solving the
GSE (1)-(3) together with (5)-(8). It should be noted the fact that the BTZ
solutions must be single-valued in the angular direction. Therefore, one
should be careful in choosing the vector field $I$. As the first example, we
look into the case $J=0,M=-\lambda^{2}<0$ of (10) in full detail.

###  4.1 Solutions with J=0, M=-$\lambda^{2}<0$

In this subsection we shall investigate the solutions of the GSE for the case
$J=0,M=-\lambda^{2}<0$ of the BTZ metric. Before proceeding to do this, let us
first write down the BTZ metric for the case $J=0,M=-\lambda^{2}<0$. Using
relation (10), it is given by

$ds^{2}=-(\lambda^{2}+\frac{r^{2}}{l^{2}})~{}dt^{2}+(\lambda^{2}+\frac{r^{2}}{l^{2}})^{-1}~{}dr^{2}+r^{2}~{}d\varphi^{2}.$
(14)

The Killing vectors of the metric (14) can be derived by solving Killing
equations. They are then read off

$\displaystyle K_{1}$ $\displaystyle=$
$\displaystyle\frac{\partial}{\partial{\varphi}},$ $\displaystyle K_{2}$
$\displaystyle=$
$\displaystyle\sqrt{\lambda^{2}{l^{2}}+{r^{2}}}~{}~{}\Big{[}\frac{rl}{\lambda^{2}{l^{2}}+r^{2}}~{}\cos(\frac{\lambda
t}{l})\sin(\lambda\varphi)~{}\frac{\partial}{\partial{t}}$
$\displaystyle+\lambda\sin(\frac{\lambda
t}{l})\sin(\lambda\varphi)\frac{\partial}{\partial{r}}+\frac{1}{r}\sin(\frac{\lambda
t}{l})\cos(\lambda\varphi)\frac{\partial}{\partial{\varphi}}\Big{]},$
$\displaystyle K_{3}$ $\displaystyle=$
$\displaystyle\sqrt{\lambda^{2}{l^{2}}+{r^{2}}}~{}~{}\Big{[}-\frac{rl}{\lambda^{2}{l^{2}}+r^{2}}~{}\sin(\frac{\lambda
t}{l})\sin(\lambda\varphi)~{}\frac{\partial}{\partial{t}}$
$\displaystyle+\lambda\cos(\frac{\lambda
t}{l})\sin(\lambda\varphi)\frac{\partial}{\partial{r}}+\frac{1}{r}\cos(\frac{\lambda
t}{l})\cos(\lambda\varphi)\frac{\partial}{\partial{\varphi}}\Big{]},$
$\displaystyle K_{4}$ $\displaystyle=$
$\displaystyle\sqrt{\lambda^{2}{l^{2}}+{r^{2}}}~{}~{}\Big{[}-\frac{rl}{\lambda^{2}{l^{2}}+r^{2}}~{}\cos(\frac{\lambda
t}{l})\cos(\lambda\varphi)~{}\frac{\partial}{\partial{t}}$
$\displaystyle-\lambda\sin(\frac{\lambda
t}{l})\cos(\lambda\varphi)\frac{\partial}{\partial{r}}+\frac{1}{r}\sin(\frac{\lambda
t}{l})\sin(\lambda\varphi)\frac{\partial}{\partial{\varphi}}\Big{]},$
$\displaystyle K_{5}$ $\displaystyle=$
$\displaystyle\sqrt{\lambda^{2}{l^{2}}+{r^{2}}}~{}~{}\Big{[}\frac{rl}{\lambda^{2}{l^{2}}+r^{2}}~{}\sin(\frac{\lambda
t}{l})\cos(\lambda\varphi)~{}\frac{\partial}{\partial{t}}$
$\displaystyle-\lambda\cos(\frac{\lambda
t}{l})\cos(\lambda\varphi)\frac{\partial}{\partial{r}}+\frac{1}{r}\cos(\frac{\lambda
t}{l})\sin(\lambda\varphi)\frac{\partial}{\partial{\varphi}}\Big{]},$
$\displaystyle K_{6}$ $\displaystyle=$
$\displaystyle-l^{2}\frac{\partial}{\partial{t}}.$ (15)

Fortunately, for integer values of $\lambda$, the resulting Killing vectors
are single-valued in the angular direction. Now, we apply the Killing vectors
(15) to construct an appropriate vector field by using formula (13). As we
will see, our solutions are classified into two special cases. In both cases
of solutions, the anti-symmetry tensor field $B$ is considered to be
$B=-{r^{2}}/{l}~{}dt\wedge d\varphi$. Then, one can employ formula (5) to
calculate the field strength corresponding to this $B$-field, giving us

$H=-\frac{2r}{l}~{}dr\wedge dt\wedge d\varphi.$ (16)

Here, the forms of our solutions including the metric (14) and field strength
(16) are given by the following two cases I and II:
$\bullet$ Case I: In this case, the GSE (1)-(3) together with (5)-(8) are
fulfilled with the metric (14) and field strength (16) if the vector field
$I$, one-form $Z$ and cosmological constant $\Lambda$ can now be expressed in
the following forms

$I=-\alpha_{6}l^{2}~{}\frac{\partial}{\partial{t}},~{}~{}~{}~{}~{}Z=\alpha_{6}lr^{2}~{}d\varphi,~{}~{}~{}~{}\Lambda=-\frac{1}{l^{2}}+\lambda^{2}{\alpha_{6}}^{2}l^{4}.$
(17)

Using these, the dilaton field is constant, and the components of $X_{\mu}$
read off

$X_{t}=\alpha_{6}l^{2}(\lambda^{2}+\frac{r^{2}}{l^{2}}),~{}~{}~{}~{}~{}X_{\varphi}=\alpha_{6}lr^{2},~{}~{}~{}~{}~{}~{}X_{r}=0.$
(18)

$\bullet\bullet$ Case II: In this case of solutions, the vector field $I$ is a
rotational symmetry, which together with one-form $Z$ and $\Lambda$ are given
as follows:

$I=\alpha_{1}~{}\frac{\partial}{\partial{\varphi}},~{}~{}~{}~{}Z=\alpha_{1}l(\lambda^{2}+\frac{r^{2}}{l^{2}})dt,~{}~{}~{}\Lambda=-\frac{1}{l^{2}}+\lambda^{2}{\alpha_{1}}^{2}l^{2}.~{}$
(19)

Then, one finds that

$X_{t}=\alpha_{1}l(\lambda^{2}+\frac{r^{2}}{l^{2}}),~{}~{}~{}~{}~{}X_{\varphi}=\alpha_{1}r^{2},~{}~{}~{}~{}~{}~{}X_{r}=0.$
(20)

The corresponding dilaton field to this case of the solutions is time-
dependent, $\Phi=c_{{}_{0}}+\lambda^{2}{\alpha_{1}}lt$.

Let us now discuss the solutions for all positive and negative values of $M$,
when $J$ is zero. The BTZ metric with $J=0,M\neq 0$ is given by

$ds^{2}=(M-\frac{r^{2}}{l^{2}})~{}dt^{2}+(\frac{r^{2}}{l^{2}}-M)^{-1}~{}dr^{2}+r^{2}~{}d\varphi^{2}.$
(21)

In this case, the solutions are similar to those of the case
$J=0,M=-\lambda^{2}$, so that one should use $M$ instead of $-\lambda^{2}$ in
the solutions (17) and (19). Then, the solutions take the forms

$I=-\alpha_{6}l^{2}~{}\frac{\partial}{\partial{t}},~{}~{}~{}~{}~{}Z=\alpha_{6}lr^{2}~{}d\varphi,~{}~{}~{}~{}\Lambda=-\frac{1}{l^{2}}-M{\alpha_{6}}^{2}l^{4},$
(22)

and

$I=\alpha_{1}~{}\frac{\partial}{\partial{\varphi}},~{}~{}~{}~{}Z=\alpha_{1}l(\frac{r^{2}}{l^{2}}-M)dt,~{}~{}~{}\Lambda=-\frac{1}{l^{2}}-M{\alpha_{1}}^{2}l^{2},~{}$
(23)

and in the same way for the components of $X_{\mu}$.

###  4.2 Solution with J=0, M=0

The BTZ metric with $J=0,~{}M=0$ is simply found by using the formula (10),
giving us

$ds^{2}=-\frac{r^{2}}{l^{2}}~{}dt^{2}+\frac{l^{2}}{r^{2}}~{}dr^{2}+r^{2}~{}d\phi^{2}.$
(24)

In order to construct the vector field $I$, one may determine the Killing
vectors corresponding to the metric (24) similar to what was done in (15).
Analogously, only non-zero component of the tensor field $B$ is considered to
be $B_{\varphi t}={r^{2}}/{l}$ for which the field strength is calculated to
be as in (16). Now, we solve the GSE with the metric (24), field strength (16)
and a constant dilaton field. The equations are then satisfied if the vector
field $I$, one-form $Z$ and $\Lambda$ have the following forms333Note that our
solution including the metric (24), $B_{\varphi t}={r^{2}}/{l}$ and relations
given by equation (25) together with a constant dilaton field may be related
to a GSE solution found in equation (4.12) of Ref. Sakatani . There, it has
been obtained a 10-dimensional solution for the GSE (with a non-zero Killing
vector $I$) whose metric is locally $AdS_{3}\times S^{3}\times T^{4}$. If we
look at the $AdS_{3}$ part of solution (4.12) of Sakatani , it may be locally
the same as our BTZ solution.

$\displaystyle I$ $\displaystyle=$
$\displaystyle-\alpha_{6}l^{2}~{}\frac{\partial}{\partial{t}}+\alpha_{3}~{}\frac{\partial}{\partial{\varphi}},$
$\displaystyle Z$ $\displaystyle=$
$\displaystyle\frac{\alpha_{3}r^{2}}{l}dt+\alpha_{6}lr^{2}d\varphi,$
$\displaystyle\Lambda$ $\displaystyle=$ $\displaystyle-\frac{1}{l^{2}}.$ (25)

Using the first two relations and also the $B$-field $B_{\varphi
t}={r^{2}}/{l}$, one then finds that the components of $X_{\mu}$ are

$\displaystyle X_{t}$ $\displaystyle=$
$\displaystyle(\alpha_{3}+\alpha_{6}l)~{}\frac{r^{2}}{l},$ $\displaystyle
X_{\varphi}$ $\displaystyle=$ $\displaystyle(\alpha_{3}+\alpha_{6}l)~{}r^{2},$
$\displaystyle X_{r}$ $\displaystyle=$ $\displaystyle 0.$ (26)

At the end of this section let us compare the solutions with J=0, M=0 and J=0,
M$\neq$0\. As showed in the above, for the case of J=0, M=0 we obtained the
vector $I$ as a linear combination of the directions of the time translation
and rotational symmetries, while this is not true for the case of J=0,
M$\neq$0\. In the case M$\neq$0 of the solutions, if one uses a linear
combination of (22) and (23) for the vector $I$ and one-form $Z$, then,
equation (8) becomes $\alpha_{1}\alpha_{6}l^{3}M=0$. This means that one of
the constants $\alpha_{1}$ or $\alpha_{6}$ should be zero. Therefore, unlike
the case of J=0, M=0, a linear combination of (22) and (23) cannot be a
solution for the GSE.

##  5  Abelian target space dual of the BTZ as a solution of the GSE

One of the interesting issues in the context of GSE is that to investigate the
solutions of the GSE under the T-duality. The T-duality symmetry is one of the
most interesting properties of string theory connecting seemingly different
backgrounds in which the strings propagate. We say that the duality is Abelian
if it is constructed on an Abelian isometry group Busher . By making use of
the Buscher’s duality transformations Busher , it was shown that the BTZ black
hole solution discussed at the end of section 3 is, under the Abelian
T-duality, equivalent to the charged black string solution discussed in Ref.
Horowitz1 . There, Horne and Horowitz found a family of solutions to low
energy string theory describing charged black strings in three dimensions.
This family of solutions is given by

$\displaystyle{d{\tilde{s}}}^{2}$ $\displaystyle=$
$\displaystyle-\big{(}1-\frac{{\bf
M}}{\hat{r}}\big{)}d\hat{t}^{2}+\Big{(}1-\frac{{{\bf
Q}}^{2}}{{{M}}\hat{r}}\Big{)}d\hat{x}^{2}$ (27)
$\displaystyle~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}+\big{(}1-\frac{{\bf
M}}{\hat{r}}\big{)}^{{}^{-1}}\Big{(}1-\frac{{{\bf Q}}^{2}}{{{\bf
M}}\hat{r}}\Big{)}^{{}^{-1}}~{}\frac{l^{2}d\hat{r}^{2}}{4\hat{r}^{2}},$
$\displaystyle{\tilde{H}}$ $\displaystyle=$ $\displaystyle\frac{{\bf
Q}}{\hat{r^{2}}}~{}d\hat{r}\wedge d{\hat{t}}\wedge d\hat{x},$ (28)
$\displaystyle{\tilde{\Phi}}$ $\displaystyle=$
$\displaystyle-\frac{1}{2}\ln(l\hat{r}),$ (29)

where ${\bf M}={r_{+}^{2}}/l$ and ${\bf Q}={J}/2$ are the respective the mass
and charge of the black string. It can be easily shown that the above metric
admits two Killing vectors $\partial/\partial\hat{t}$ and
$\partial/\partial\hat{x}$. Here, we shall show that the metric (27) and field
strength (28) can satisfy the GSE, if the vector field I, one-form Z, dilaton
field and cosmological constant express as follows:

$\displaystyle I$ $\displaystyle=$
$\displaystyle\alpha_{1}~{}\frac{\partial}{\partial\hat{t}},$ $\displaystyle
Z$ $\displaystyle=$
$\displaystyle-\frac{1}{2\hat{r}}d\hat{r}+\alpha_{1}(\frac{{\bf M}}{{\bf
Q}}-\frac{{\bf Q}}{\hat{r}})d\hat{x},$ $\displaystyle{\Phi}$ $\displaystyle=$
$\displaystyle c_{{}_{0}}-\frac{1}{2}\ln\hat{r}+\alpha_{1}\frac{{\bf M}}{{\bf
Q}}\hat{x},$ $\displaystyle\Lambda$ $\displaystyle=$
$\displaystyle\frac{2}{l^{2}}+\frac{2{\alpha_{1}}^{2}({\bf M}^{2}-{\bf
Q}^{2})}{{\bf Q}^{2}}.$ (30)

In addition to this solution, one can see that the GSE (1)-(3) together with
(5)-(8) are fulfilled with the metric (27) and field strength (28) if the
vector field $I$, one-form $Z$, dilaton field and cosmological constant can
now be expressed in the following forms

$\displaystyle I$ $\displaystyle=$
$\displaystyle\alpha_{2}~{}\frac{\partial}{\partial\hat{x}},$ $\displaystyle
Z$ $\displaystyle=$ $\displaystyle-\frac{1}{2\hat{r}}d\hat{r}+\alpha_{2}{\bf
Q}(\frac{1}{\hat{r}}-\frac{1}{{\bf M}})d\hat{t},$ $\displaystyle{\Phi}$
$\displaystyle=$ $\displaystyle
c_{{}_{0}}-\frac{1}{2}\ln\hat{r}-\alpha_{2}\frac{{\bf Q}}{{\bf M}}\hat{t},$
$\displaystyle\Lambda$ $\displaystyle=$
$\displaystyle\frac{2}{l^{2}}+\frac{2{\alpha_{2}}^{2}({\bf M}^{2}-{\bf
Q}^{2})}{{\bf M}^{2}}.$ (31)

##  6  Conclusion

In this study, we have obtained some new solutions for the GSE in three
dimensions. Our solutions include the special cases $J=0,M=0$ and $J=0,M\neq
0$ of the BTZ metric together with the field strength $H=-({2r}/{l})dr\wedge
dt\wedge d\varphi$. In each case we have calculated the the vector field $I$,
one-form $Z$ and the cosmological constant $\Lambda$. To determine the vector
field $I$ we have used the Killing vectors corresponding to the BTZ metrics.
By comparing the solutions with $J=0,M=0$ and $J=0,M\neq 0$, we concluded that
unlike the case of $J=0,M=0$, a linear combination of (22) and (23) cannot be
a solution for the GSE. However, we have derived a family of solutions as
presented in relations (22), (23) and (25). Notably, our results confirm that
these solutions are single-valued in the angular direction. On the other hand,
we have checked that the cases $J\neq 0,M=0$ and $J\neq 0,M\neq 0$ for the BTZ
metric do not satisfy the GSE.

While we considered the BTZ metrics, one can easily repeat for other geometric
metrics such as Thurston geometries. As mentioned in section 3, the BTZ metric
with $J=0$ and $M=-1$ represents ordinary anti-de Sitter space ($AdS_{3}$),
which is nothing but one of Lorentzian Thurston geometries. Therefore, we have
investigated that the $AdS_{3}$ space can be considered as a solution for the
GSE. Finally, as an interesting result, we have shown that the charged black
string solution (27), which is Abelian T-dual to the the BTZ black hole
solution, can be also a solution for the GSE. In fact, this result helps to
answer the question whether the solutions of the GSE are, under the Abelian
T-duality, preserved. It will be an interesting future direction to be
addressed.

###### Acknowledgements.

This work has been supported by the research vice chanceller of Azarbaijan
Shahid Madani University under research fund No. $1402/231$.

## References

  * (1) G. Arutyunov, S. Frolov, B. Hoare, R. Roiban, A. A. Tseytlin, Nucl. Phys. B 903, 262 (2016)
  * (2) F. Delduc, M. Magro, B. Vicedo, Phys. Rev. Lett. 112, 051601 (2014)
  * (3) F. Delduc, M. Magro, B. Vicedo, J. High Energy Phys. 10, 132 (2014)
  * (4) I. Kawaguchi, T. Matsumoto, K. Yoshida, J. High Energy Phys. 04, 153 (2014)
  * (5) R. Borsato, L. Wulff, J. High Energy Phys. 10, 045 (2016)
  * (6) T. Araujo, I. Bakhmatov, E. O Colgain, J. Sakamoto, M. M. Sheikh-Jabbari, K. Yoshida, Phys. Rev. D 95, 105006 (2017)
  * (7) T. Araujo, I. Bakhmatov, E. O Colgain, J. Sakamoto, M. M. Sheikh-Jabbari, K. Yoshida, J. Phys. A 51, 235401 (2018)
  * (8) T. Araujo, E. O Colgain, J. Sakamoto, M. M. Sheikh-Jabbari, K. Yoshida, Eur. Phys. J. C 77, 739 (2017)
  * (9) I. Bakhmatov, O. Kelekci, E. O Colgain, M. M. Sheikh-Jabbari, Phys. Rev. D 98, 021901 (2018)
  * (10) D. Orlando, S. Reffert, J. I. Sakamoto, K. Yoshida, J. Phys. A 49, 445403 (2016)
  * (11) B. Hoare, A. A. Tseytlin, J. Phys. A 49, 494001 (2016)
  * (12) M. Hong, Y. Kim, E. O. Colgain, Eur. Phys. J. C 78, 1025 (2018)
  * (13) R. Borsato, L. Wulff, J. High Energy Phys. 08, 027 (2018)
  * (14) B. Hoare, A. A. Tseytlin, J. High Energy Phys. 10, 060 (2015)
  * (15) L. Wulff, A. A. Tseytlin, J. High Energy Phys. 06, 174 (2016)
  * (16) J. Sakamoto, Y. Sakatani, K. Yoshida, PTEP 053B07 (2017)
  * (17) Y. Sakatani, S. Uehara, K. Yoshida, J. High Energy Phys. 04, 123 (2017)
  * (18) J. J. Fernandez-Melgarejo, J. Sakamoto, Y. Sakatani, K. Yoshida, Phys. Rev. Lett. 122, 111602 (2019)
  * (19) J. Horne and G. Horowitz, Nucl. Phys. B 368, 444 (1992)
  * (20) G. Horowitz, D. Welch, Phys. Rev. Lett. 71, 328 (1993)
  * (21) M. Banados, C. Teitelboim, J. Zanelli, Phys. Rev. Lett. 69, 1849 (1992)
  * (22) Y. Sakatani, Prog. Theor. Exp. Phys. 073B04 (2019)
  * (23) T. Buscher, Phys. Lett. B 194, 59 (1987); Phys. Lett. B 201, 466 (1988).


# Homophone Reveals the Truth: A Reality Check for Speech2Vec

Guangyu Chen
Renmin University of China
<EMAIL_ADDRESS>
###### Abstract

Generating spoken word embeddings that possess semantic information is a
fascinating topic. Compared with text-based embeddings, they cover both
phonetic and semantic characteristics, which can provide richer information
and are potentially helpful for improving ASR and speech translation systems.
In this paper, we review and examine the authenticity of a seminal work in
this field: Speech2Vec. First, a homophone-based inspection method is proposed
to check the speech embeddings released by the author of Speech2Vec. There is
no indication that these embeddings are generated by the Speech2Vec model.
Moreover, through further analysis of the vocabulary composition, we suspect
that a text-based model fabricates these embeddings. Finally, we reproduce the
Speech2Vec model, referring to the official code and optimal settings in the
original paper. Experiments showed that this model failed to learn effective
semantic embeddings. In word similarity benchmarks, it gets a correlation
score of 0.08 in MEN and 0.15 in WS-353-SIM tests, which is over 0.5 lower
than those described in the original paper. Our data and code are
available111https://github.com/my-yy/s2v_rc.

##  1 Introduction

Word embeddings are fixed-length representations of words, which carry
syntactic and semantic information and become the building block in natural
language processing tasks, such as named entity recognition [24] and question
answering [23]. Speech, as another carrier of information, also reflects
semantic meaning, which implies the possibility of directly using it to learn
semantic word representations. To this end, some attempts have been made to
generate semantic speech embeddings [7, 18, 5, 6]. Compared with learning
text-based embeddings, this is a much more difficult task. Because there are
intrinsic interferences between phonetics and semantics [5]. For example, the
words ‘ate’ and ‘eight’ have almost the same pronunciation but are
semantically different. If the model can not distinguish these sound-alike
inputs, it is impossible to obtain effective semantic representations.

Figure 1:  The Speech2Vec model (skip-gram style).

In this paper, we focus on Speech2Vec [6, 7], one of the field’s earliest and
most influential studies. It can be viewed as a speech version of Word2Vec
[17], which leverages the skip-gram and CBOW strategy to learn semantic speech
embeddings. Despite being an imitator of Word2Vec, it seems to have magically
resolved the mentioned phonetic-semantic interference and has been reported to
exceed the Word2Vec on 13 word similarity benchmarks [10]. This phenomenon
aroused our interest in exploring the validity of this study. Specifically, we
analyze the authenticity of Speech2Vec from two perspectives of _the official-
released speech embeddings_ and _our reproduction results_. The conclusions
can be summarized as follows:

1) The official speech embeddings fail to reflect speech characteristics,
implying that they are not generated by the Speech2Vec model.

2) Analysis of the vocabulary composition reveals that these released
embeddings may come from a text-based model.

3) Our reproduction results show that the Speech2Vec model failed to generate
valid semantic speech embeddings. Even after 500 epochs of training, its
performance is still negligible in most benchmark metrics.

##  2 Recap Speech2Vec

Speech2Vec can be regarded as a speech version of Word2Vec, which relies on
the skip-gram or CBOW strategy to learn semantic speech embeddings. Herein, we
focused on the skip-gram style of Speech2Vec, which is reported to have better
performance. Fig. 1 shows its structure.

When training, the input speech sentence is segmented into spoken word
sequences in advance. Then the encoder network converts the current spoken
word into an embedding vector. And this vector is used for decoding context
words. In the corpus, the audio segments corresponding to the same text word
appear multiple times (with different speakers, environment, emotions, or
talking speed). To obtain a deterministic word representation, the average of
all embeddings corresponding to the same text word is taken as the final
embedding.

The official code of Speech2Vec is not publicly available, but the pre-trained
speech embeddings are released on GitHub
222https://github.com/iamyuanchung/speech2vec-pretrained-vectors. These
embeddings are described from a skip-gram style Speech2Vec model trained on
500 hours of LibriSpeech [19] dataset:

> … In this repository, we release the speech embeddings of different
> dimensionalities learned by Speech2Vec using skip-grams as the training
> methodology. The model is trained on a corpus consisting of about 500 hours
> of speech from LibriSpeech (the clean-360 + clean-100 subsets) …

We further provide a fork to this repository 333https://github.com/my-
yy/speech2vec-pretrained-vectors.

##  3 Reality Check

This section aims to check the authenticity of the released speech embeddings.
We analyze them from two perspectives: phonetic characteristics and vocabulary
composition.

###  3.1 Homophone Inspection

Homophones refer to words that have the same pronunciation but reflect
different meanings (e.g., ‘ate, eight’,‘ dear, deer’, and ‘whine, wine’).
Since these spoken words are consistent in pronunciation, in a speech
recognition system, the speech context and language model [3] are usually
required to distinguish them. For the skip-gram style Speech2Vec, its input is
a single spoken word that does not contain context information. Therefore,
this model should not be able to distinguish these homophones effectively. And
the embedding similarity of the homophone pair should have a higher value. We
used 307 homophone pairs444 Sourced from:
http://www.singularis.ltd.uk/bifroest/misc/homophones-list.html and calculated
their similarities. The results are shown in Tab. 1.

Embedding | Dim | Pair Similarity
---|---|---
Homophone | Random
Official | 50 | 0.33±0.15 | 0.39±0.17
100 | 0.28±0.14 | 0.35±0.15
200 | 0.23±0.12 | 0.31±0.14
300 | 0.21±0.11 | 0.29±0.14
Ours | 50 | 0.95±0.05 | 0.77±0.10
Table 1:  Cosine similarities of word pairs (mean ± std). Random: Similarities
between 307 randomly selected pairs. Official: Embeddings released by the
author of Speech2Vec. Ours: Embeddings generated by our reproduced model
(detailly described in Sec. 4. )

As we can see, for the official embeddings, their homophone similarities are
counterintuitively lower than the random results. It suggested that the
official model may not be affected by phonetic similarity and can successfully
distinguish these highly similar inputs. However, our reproduced model did not
‘show the magic’. Although it had a 0.77 similarity for random pairs (which
implies the embedding space is ill-distributed), it still obtained a higher
similarity of 0.95 for the homophone pairs.

Homophone Pair | Similarity
---|---
Raw | Official | Ours
ate | eight | 1.00 | 0.22 | 1.00
hail | hale | 0.99 | 0.34 | 0.97
know | no | 1.00 | 0.63 | 0.97
made | maid | 1.00 | 0.25 | 0.98
meat | meet | 1.00 | 0.13 | 0.99
peace | piece | 1.00 | 0.15 | 0.99
sea | see | 1.00 | 0.31 | 0.98
steal | steel | 1.00 | 0.19 | 0.99
tale | tail | 1.00 | 0.23 | 1.00
whine | wine | 1.00 | 0.13 | 0.97
MEAN | 0.99 | 0.32 | 0.95
Table 2:  Examples of homophone similarities. The Official stands for the official 50-dimensional embeddings. The appendix gives the full 307 results.  Word | Official | Ours
---|---|---
K Nearest Neighbor | HR | K Nearest Neighbor | HR
ate | eating | drank | eat | 25,099 | eight | agent | aid | 1
hail | thunder | blast | thunders | 15,253 | hell | handle | held | 9
know | tell | think | why | 2,305 | narrow | natural | no | 3
made | making | gave | make | 35,582 | me | mid | necessity | 15
meat | roasting | roasted | venison | 33,108 | meet | mate | mute | 1
peace | tranquility | liberty | deliverer | 33,609 | plates | place | patience | 7
sea | ocean | shore | waters | 18,610 | city | safety | succeed | 100
steal | flay | stealing | kill | 29,506 | still | steel | special | 2
tale | story | adventure | tales | 28,617 | tail | title | tackle | 1
whine | snarling | growls | growl | 34,575 | watering | washing | wide | 4
MEAN |  |  |  | 25,626 |  |  |  | 14
Table 3: The k nearest neighbor results (k=3). HR: Homophone ranking.

Tab. 2 further gives some pair examples. In this table, we introduce a ‘Raw’
metric to measure the similarity of audio input. This metric is the cosine
similarity of extracted features from a pre-trained wav2vec [21] model.

It is clear to see that most homophone pairs have a Raw score close to 1.0,
which means there is a high degree of similarity between these homophone
words. And the wav2vec model failed to distinguish them. In addition, our
reproduced model has similar behavior, which gets an average similarity of
0.95. However, the official embeddings get relatively lower similarities.
Taking the ‘ate, eight’ pair as an example, the similarities of Raw and ours
are both 1.00, while the official gives a 0.22 similarity score.

The k-nearest neighbor results (Tab. 3) further reveals the anomaly of the
official embedding. Although the official’s nearest neighbors reflect similar
semantics, there are considerable differences in pronunciation. For example,
the nearest neighbors of ‘hail’ are ‘thunder, blast, thunders’, while its
homophone counterpart (‘hale’) is ranked as high as 15,253. In contrast, the
homophone rank in our reproduction is 9.

The above anomaly behavior of the official embeddings can be explained by a
bold hypothesis: the official embeddings come from a text-based model. As in a
text-based model (like Word2Vec), words of different spellings are assigned
with different coding values. In the learning process, the model does not rely
on pronunciation to identify words; therefore, it will not be disturbed by
them. Moreover, because homophone pairs have different meanings, their
similarities may be smaller than the random pairs. This explains why the
official results have smaller similarities than the random in Tab. 1.

###  3.2 Vocabulary Analysis

####  3.2.1 Checking the Released Vocabulary

Corpus Division | Hours | Speakers
---|---|---
Clean | train-clean-100 | 100.6 | 475 | 251 | 1,252
train-clean-360 | 363.6 | 921
dev-clean | 5.4 | 40
test-clean | 5.4 | 40
Other | train-other-500 | 496.7 | 507 | 1,166 | 1,232
dev-other | 5.3 | 33
test-other | 5.1 | 33
Table 4: The statistics of the LibrSpeech corpus.

The Seepch2Vec paper claimed that the training is based on 500 hours of
LibriSpeech [19], which contains 1,252 speakers. We refer to this dataset as
Libri-Clean. Its statistics are shown in Tab. 4. We counted distinct words in
Libri-Clean’s transcripts and compared them with the official. The results are
shown in Tab. 5.

| Official | Libri-Clean | Libri-All
---|---|---|---
| - | count$\geq$5 | count$\geq$5
Vocab | 37,622 | 66,721 | 27,454 | 37,622
Missing | - | 1,685 | 10,168 | 0
Table 5:  The vocabulary composition.

We found that 1.6k official words are not in the Libri-Clean. If we only
consider frequent words of occurrence count $\geq$ 5, this number rises to
10k. However, if we use the entire LibriSpeech corpus and filter out
infrequent words, the vocabulary composition is exactly the same as the
official. This proves that the released embeddings are derived from all
LibriSpeech corpus, not the claimed ‘500 hours’ data.

In addition, we believe that these embeddings are generated by a text-based
model, which directly uses text transcripts for training. Because the
LibriSpeech dataset needs to be pre-processed for training, each speech
sentence should be aligned with the transcript to locate word boundaries.
However, this alignment process is not ideal, which may cause data loss.

In our analysis, we used the alignment results based on the widely used
Montreal Forced Aligner [16] 555https://github.com/CorentinJ/librispeech-
alignments. It aligned the entire corpus of 292,367 speech sentences and
encountered 127 failures (0.04% fail rate). After removing these unaligned
speeches, the corpus changes, which causes additional 17 missing words in the
vocabulary. Therefore, if the authors of Speech2Vec do not realize an ideal
alignment, their vocab size will not equal 37,622 exactly. Otherwise, their
embeddings come from a text-based model.

Benchmark | Not Found Pairs
---|---
Paper Data | Libri-Clean
MEN [4] | 122 | 231
MTurk-287 [20] | 13 | 29
MTurk-771 [12] | 22 | 43
Rare-Word [15] | 783 | 1027
SimLex-999 [13] | 0 | 11
SimVerb-3500 [11] | 126 | 118
Verb-143 [2] | 0 | 5
WS-353 [26] | 21 | 34
WS-353-REL [1] | 12 | 24
WS-353-SIM [1] | 7 | 16
YP-130 [26] | 0 | 9
Table 6:  Not found pairs in word similarity benchmarks.

####  3.2.2 Checking The Paper Data

Although the vocabulary size is not mentioned in the paper of Speech2Vec, we
can still find some clues from the word similarity benchmark results. In the
original paper, these benchmarks are used to evaluate the semantic information
carried by embeddings. Each benchmark contains a series of predefined word
pairs with the corresponding oracle correlation. In an early version of
Speech2vec [6], the author reported the ‘not found’ pairs in these benchmarks
(Tab. 6).

If we use all the 66,721 words in Libri-Clean to construct the vocabulary,
there are still more not found pairs in these tests (except for SimVerb-3500).
For example, the original paper reported 122 not found pairs in the MEN test,
but we found 231. This illustrates that the authors of Speech2Vec used a
larger corpus other than the claimed 500h Libri-Clean.

##  4 Our Reproduction

###  4.1 Reproduction Details

####  4.1.1 Code Implementation

Our reproduction is based on the incomplete code provided by the author of
Speech2Vec. This code covers the model implementation and training logic while
lacking the data pre-processing. We complement the missing parts and train the
Speech2Vec, referring to the optimal setting in the original paper.
Specifically, the embedding dimension is set as 50. The window size is taken
as 3. The encoder is a single-layer bidirectional LSTM. And the decoder is a
single-layer unidirectional LSTM. During the decoding process, the attention
mechanism [14] is used to reference the encoder’s outputs (Fig. 5 in the
appendix gives a detailed overview of its structure).

####  4.1.2 Training Details

In accordance with Speech2Vec’s paper, the Libri-Clean dataset is used for
training. Each speech sentence is segmented into spoken words in advance. Then
we filter out the low-frequency words that appear less than four times and
finally get a vocabulary of size 30,788. For each spoken word, we convert it
into 13-dim MFCC frames.

During the mini-batch construction, we set batch size as 4096 and trained the
model for 500 epochs with SGD optimizers of 0.001 learning rate. This training
process took 8.4 days on an AMD 3900XT + RTX3090 device. The output models of
every epoch are saved, and the 500th epoch’s model is used for evaluation.

Figure 2:  Comparisons on word similarity benchmarks. Paper Data: The results
claimed in the paper of Speech2Vec [7]. Rand Init: The randomly initialized
model without training. 500-Epoch: Train the model for 500 epochs.  Figure 3:
The learning curve. X-axis: training epochs. Y-axis: Spearman coefficient
(except for Loss).

###  4.2 Results & Analysis

Same with the official, we tested our reproduced model on 13 similarity
benchmarks [10]. The evaluation score is Spearman’s rank correlation
coefficient $\rho$. It has a range of [-1,1], and a larger value represents a
better result. The results are shown in Fig. 2.

Obviously, the reproduced model gives trivial results on these benchmarks.
Compared with the random initialization, the 500-epoch model only has minor
improvements or even decreases in indicators of SimLex-999 and SimVerb-3500.
Moreover, compared with the claimed paper data, there is a substantial
performance drop, even over 0.5 in some metrics (MC-30, MEN, RG-65,
WS-353-SIM).

We further depicted the training process in Fig. 3. With the loss decreasing,
most indicators showed a rising trend. But the increasing speed is relatively
slow and is finally limited to a small range (e.g., 0$\sim$0.022 for
MTurk-287). More training epochs may provide better results on these metrics,
but it should be much larger than the claimed ‘500 epochs’. Moreover, two
indicators seem to have passed their peak period, with performance turning
from rising to falling (WS-353-REL and MEN). However, their peak performance
is still relatively lower than the original paper. There are two indicators
whose performance is consistently declining, which means that the required
semantics have not been learned successfully (SimLex-999 and SimVerb-3500).

As mentioned in Sec. 2, each text word corresponds to multiple speech
segments. In Fig. 4, we visualize the embedding distribution of specific
words. Specifically, the multi-dimensional scaling (MDS) [25] is used for
visualization. Compared with the popular T-SNE, the MDS preserves distance and
global structure.

As we can see, the homophone words are mixed, while words with different
pronunciations tend to be separated. This result confirms that Speech2Vec’s
encoder can not successfully distinguish homophones. Moreover, we noticed some
mixed points for the ‘need-thus’ pair. This shows that the model did not have
a good discernibility even for not sound-alike pairs, which may be caused by
the interference between phonetics and semantics.

Figure 4: MDS visualization of speech word embeddings.

##  5 Related Works

###  5.1 Works on Learning Semantic Speech Embeddings

Five years since the Speech2Vec was proposed, there is rarely work in the
field of learning semantic speech embeddings. The Speech2Vec has received 158
citations so far, but no article has directly compared with it, and no
subsequent work has proved its validity.

####  5.1.1 Chen et al.’s Work

Chen et al. [5] pointed out that when learning semantic speech embeddings, the
phonetics and semantics inevitably disturb each other. Therefore, they
designed a two-stage learning scheme to reduce the learning difficulty. In the
first stage, an encoder-decoder structure generates phonetic embeddings in
which the speaker characteristics are disentangled. In the second stage, these
embeddings possess semantic meaning through a context prediction task.
Experiments have verified their embeddings’ phonetic and semantic
characteristics. However, this work is not compared with Speech2Vec on the
similarity benchmarks.

####  5.1.2 CAWE

CAWE [18] is an encoder-decoder-based speech recognition model, which relies
on an attention mechanism to automatically segment speech sentences into
spoken words and generate speech embeddings. This model is tested on 16
sentence evaluation tasks and shows competitive results to Word2Vec. Compared
with skip-gram style Speech2Vec, the CAWE can leverage the context
information. Thus their effectiveness can not laterally prove Speech2Vec’s
validity. Moreover, we have not found any successful third-party replications
of CAWE.

####  5.1.3 Audio2Vec

Audio2Vec [22] is a CNN-based model which uses skip-gram, CBOW, and temporal-
gap strategies to generate fixed-length audio representations. Although it has
a similar training style to Speech2Vec, the goal of Audio2Vec is to produce
general-purpose audio representations. In the training process, the speech
sentence is segmented into fixed-length slices without reference to word
boundaries. Thus this model was not tested on the word similarity benchmarks
and not compared with Speech2Vec.

###  5.2 Works Directly Based on Speech2Vec Embedding

Ideally, if the experimental data is questionable, the conclusion’s validity
cannot be guaranteed. We note that there are some works [8, 9, 27] directly
based on the embeddings of Speech2Vec. Among them, two works come from the
authors of Speech2vec [8, 9], which align the speech space and text space to
realize cross-language speech-to-text translation.

Yi et al. [27] reported that they successfully improved punctuation prediction
performance by involving the released Speech2Vec embeddings in their input.
But they did not explore how these speech embeddings work. We think their
results are authentic but unintentionally provide a wrong interpretation. As
we have analyzed, these Speech2Vec embeddings cannot reflect phonetic
information. Thus their improvements come from involving more input features.
And it has nothing to do with whether these features come from a text-based or
speech-based model. Therefore, we believe that the above works cannot provide
evidence for the authenticity of Speech2Vec.

###  5.3 Replication Attempts by Other Researchers

We investigated the replication attempts by other researchers, including
Jang’s666https://github.com/yjang43/Speech2Vec and
Wang’s777https://github.com/ZhanpengWang96/pytorch-speech2vec. Like us, they
fail to achieve the results claimed in Speech2Vec’s paper. Jang reported that
the model gets trivial results. As for Wang’s, the reproduced model is also
vulnerable, which only gets $\rho=0.08$ in WS-353-REL and $\rho=-0.15$ in
YP-130.

##  6 Disscussion

“If I have seen further, it is by standing on the shoulders of Giants. ——Isaac
Newton,1675.”

Scientific progress builds on the innovations of predecessors. If an
influential innovation is falsified, it is not ‘the shoulder of a giant’.
Instead, it becomes a _stumbling block_ and brings huge obstacles to
developing the field. We think science is not a kind of religion, especially
when it comes to computer science. The authenticity of innovation should
depend on reproducibility rather than the reputation of the author,
institution, or paper grade.

In 2018, a master’s student was assigned to begin his academic research in an
emerging field. Unluckily, he took faith in a bogus study and get off the
wrong road. He was passionate about it, striving for years to achieve an
impossible goal, and ended up with nothing. Four years after setting out, he
reflected on the failures of his life and wrote this article. To prevent
others from repeating his past, this is the value of this study.

## References

  * [1] Eneko Agirre, Enrique Alfonseca, Keith Hall, Jana Kravalova, Marius Pasca, and Aitor Soroa.  A study on similarity and relatedness using distributional and wordnet-based approaches.  2009\.
  * [2] Simon Baker, Roi Reichart, and Anna Korhonen.  An unsupervised model for instance level subcategorization acquisition.  In EMNLP, pages 278–289, 2014.
  * [3] Yoshua Bengio, Réjean Ducharme, and Pascal Vincent.  A neural probabilistic language model.  Advances in neural information processing systems, 13, 2000.
  * [4] Elia Bruni, Gemma Boleda, Marco Baroni, and Nam-Khanh Tran.  Distributional semantics in technicolor.  In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 136–145, 2012.
  * [5] Yi-Chen Chen, Sung-Feng Huang, Chia-Hao Shen, Hung-Yi Lee, and Lin-Shan Lee.  Phonetic-and-semantic embedding of spoken words with applications in spoken content retrieval.  In 2018 IEEE Spoken Language Technology Workshop (SLT), pages 941–948. IEEE, 2018.
  * [6] Yu-An Chung and James Glass.  Learning word embeddings from speech.  arXiv preprint arXiv:1711.01515, 2017.
  * [7] Yu-An Chung and James Glass.  Speech2vec: A sequence-to-sequence framework for learning word embeddings from speech.  In INTERSPEECH, 2018.
  * [8] Yu-An Chung, Wei-Hung Weng, Schrasing Tong, and James Glass.  Unsupervised cross-modal alignment of speech and text embedding spaces.  Advances in neural information processing systems, 31, 2018.
  * [9] Yu-An Chung, Wei-Hung Weng, Schrasing Tong, and James Glass.  Towards unsupervised speech-to-text translation.  In ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 7170–7174. IEEE, 2019.
  * [10] Manaal Faruqui and Chris Dyer.  Community evaluation and exchange of word vectors at wordvectors. org.  In Proceedings of 52nd Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pages 19–24, 2014.
  * [11] Daniela Gerz, Ivan Vulić, Felix Hill, Roi Reichart, and Anna Korhonen.  Simverb-3500: A large-scale evaluation set of verb similarity.  arXiv preprint arXiv:1608.00869, 2016.
  * [12] Guy Halawi, Gideon Dror, Evgeniy Gabrilovich, and Yehuda Koren.  Large-scale learning of word relatedness with constraints.  In Proceedings of the 18th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 1406–1414, 2012.
  * [13] Felix Hill, Roi Reichart, and Anna Korhonen.  Simlex-999: Evaluating semantic models with (genuine) similarity estimation.  Computational Linguistics, 41(4):665–695, 2015.
  * [14] Minh-Thang Luong, Hieu Pham, and Christopher D Manning.  Effective approaches to attention-based neural machine translation.  arXiv preprint arXiv:1508.04025, 2015.
  * [15] Minh-Thang Luong, Richard Socher, and Christopher D Manning.  Better word representations with recursive neural networks for morphology.  In Proceedings of the seventeenth conference on computational natural language learning, pages 104–113, 2013.
  * [16] Michael McAuliffe, Michaela Socolof, Sarah Mihuc, Michael Wagner, and Morgan Sonderegger.  Montreal forced aligner: Trainable text-speech alignment using kaldi.  In Interspeech, volume 2017, pages 498–502, 2017.
  * [17] Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean.  Distributed representations of words and phrases and their compositionality.  Advances in neural information processing systems, 26, 2013.
  * [18] Shruti Palaskar, Vikas Raunak, and Florian Metze.  Learned in speech recognition: Contextual acoustic word embeddings.  In ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 6530–6534. IEEE, 2019.
  * [19] Vassil Panayotov, Guoguo Chen, Daniel Povey, and Sanjeev Khudanpur.  Librispeech: an asr corpus based on public domain audio books.  In 2015 IEEE international conference on acoustics, speech and signal processing (ICASSP), pages 5206–5210. IEEE, 2015.
  * [20] Kira Radinsky, Eugene Agichtein, Evgeniy Gabrilovich, and Shaul Markovitch.  A word at a time: computing word relatedness using temporal semantic analysis.  In Proceedings of the 20th international conference on World wide web, pages 337–346, 2011.
  * [21] Steffen Schneider, Alexei Baevski, Ronan Collobert, and Michael Auli.  wav2vec: Unsupervised pre-training for speech recognition.  arXiv preprint arXiv:1904.05862, 2019.
  * [22] Marco Tagliasacchi, Beat Gfeller, Félix de Chaumont Quitry, and Dominik Roblek.  Self-supervised audio representation learning for mobile devices.  arXiv preprint arXiv:1905.11796, 2019.
  * [23] Stefanie Tellex, Boris Katz, Jimmy Lin, Aaron Fernandes, and Gregory Marton.  Quantitative evaluation of passage retrieval algorithms for question answering.  In Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval, pages 41–47, 2003.
  * [24] Joseph Turian, Lev Ratinov, and Yoshua Bengio.  Word representations: a simple and general method for semi-supervised learning.  In Proceedings of the 48th annual meeting of the association for computational linguistics, pages 384–394, 2010.
  * [25] Florian Wickelmaier.  An introduction to mds.  Sound Quality Research Unit, Aalborg University, Denmark, 46(5):1–26, 2003.
  * [26] Dongqiang Yang and David MW Powers.  Verb similarity on the taxonomy of WordNet.  Masaryk University, 2006.
  * [27] Jiangyan Yi and Jianhua Tao.  Self-attention based model for punctuation prediction using word and speech embeddings.  In ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 7270–7274. IEEE, 2019.

##  Appendix A Encoding-Decoding Details

Fig. 5 depicts the encoding-decoding process. For clarity, this illustration
takes a single spoken word as input, while the model is trained in batch mode.
Since the input words have various time lengths, the length uniformity is
implemented in the official code. The official code lacks the data processing
part, and we don’t know the exact time length. According to statistics, our
dataset has an average MFCC frame of 10, and there are 94.6% of words less
than 21 frames. Thus we padded (or truncated) the input word to fixed 20 MFCC
frames.

In the encoding stage, the official code used the `pack_padded_sequence` of
PyTorch to eliminate the padding effects. However, in the decoding stage, the
reconstruction loss corresponding to the padded frames is still considered. We
fixed this issue in our reproduction. More details can be found in our
released code.

(a) Encoding process

(b) One decoding step

Figure 5:  Illustration of the encoding-decoding steps. Best view in color.

##  Appendix B Comparisons on Similarity Benchmarks

We note that the optimizer in the official code is the Adam optimizer (not the
described SGD in the official paper). To exclude the optimizer’s impact, we
tried the Adam setting. However, we did not observe essential differences in
the results, as shown in Tab. 7.

Benchmark | Paper Data | SGD | Adam
---|---|---|---
Rand Init | 500-Epoch | Rand Init | 500-Epoch
MC-30 | 0.85 | 0.18 | 0.24↓0.60 | 0.37 | -0.28↓1.13
MEN | 0.62 | 0.08 | 0.08↓0.54 | 0.06 | 0.04↓0.58
MTurk-287 | 0.47 | 0.00 | 0.02↓0.45 | 0.04 | 0.08↓0.39
MTurk-771 | 0.52 | -0.00 | 0.03↓0.49 | -0.00 | 0.04↓0.48
RG-65 | 0.79 | 0.11 | 0.18↓0.61 | 0.25 | 0.05↓0.74
Rare-Word | 0.32 | 0.10 | 0.12↓0.21 | 0.10 | 0.04↓0.28
SimLex-999 | 0.29 | 0.02 | -0.05↓0.34 | -0.06 | -0.04↓0.33
SimVerb-3500 | 0.16 | 0.00 | -0.02↓0.17 | -0.03 | -0.03↓0.18
Verb-143 | 0.32 | 0.24 | 0.29↓0.02 | 0.34 | 0.24↓0.07
WS-353 | 0.51 | 0.11 | 0.14↓0.36 | 0.18 | 0.12↓0.39
WS-353-REL | 0.35 | 0.10 | 0.11↓0.24 | 0.12 | 0.11↓0.24
WS-353-SIM | 0.66 | 0.10 | 0.15↓0.51 | 0.22 | 0.16↓0.51
YP-130 | 0.32 | 0.01 | 0.07↓0.25 | 0.11 | -0.01↓0.33
Table 7:  Results on semantic benchmarks. The ↓ represents the gap with the
Paper Data. Note that the optimizer type does not affect the ‘Rand Init’
results. We used different seeds for these two runs; that’s why their Rand
Init results are different.

##  Appendix C The Homophone List

The full 307 homophone pairs are listed in LABEL:tab:homo_full_list. Since our
reproduction is based on Libri-Clean, which has a smaller vocab than the
released embedding. Thus there are 33 missing pairs in our reproduction
(marked by ‘-’).

Table 8: The homophone list.  Homophone Pairs | Raw | Official | Ours | Homophone Pairs | Raw | Official | Ours
---|---|---|---|---|---|---|---
ad | add | 0.99 | 0.09 | 0.92 | lieu | loo | 0.98 | 0.43 | 0.91
ail | ale | 0.98 | 0.49 | 0.75 | links | lynx | 1.00 | 0.24 | 0.96
air | heir | 1.00 | 0.07 | 0.97 | lo | low | 0.99 | 0.22 | 0.98
all | awl | - | 0.40 | - | loan | lone | 1.00 | 0.09 | 0.94
allowed | aloud | 1.00 | 0.21 | 0.96 | loot | lute | 0.99 | 0.17 | 0.92
alms | arms | 0.95 | 0.26 | 0.95 | made | maid | 1.00 | 0.25 | 0.98
altar | alter | 0.99 | 0.13 | 0.95 | mail | male | 1.00 | 0.23 | 0.98
arc | ark | 0.98 | 0.19 | 0.91 | main | mane | 0.99 | 0.17 | 0.97
aren’t | aunt | 0.96 | 0.44 | 0.95 | maize | maze | 0.99 | 0.25 | 0.93
ate | eight | 1.00 | 0.22 | 1.00 | manna | manner | 0.97 | 0.33 | 0.89
auger | augur | - | 0.51 | - | mantel | mantle | 1.00 | 0.43 | 0.95
awe | oar | 0.92 | 0.26 | 0.78 | mare | mayor | 0.99 | 0.38 | 0.96
axel | axle | 0.99 | 0.41 | 0.86 | marshal | martial | 1.00 | 0.60 | 0.98
aye | eye | 0.99 | 0.29 | 0.89 | marten | martin | - | 0.36 | -
bail | bale | 0.99 | 0.36 | 0.98 | mask | masque | - | 0.51 | -
bait | bate | 0.98 | 0.41 | 0.95 | mean | mien | 0.98 | 0.24 | 0.90
baize | bays | 0.99 | 0.50 | 0.91 | meat | meet | 1.00 | 0.13 | 0.99
bald | bawled | 0.99 | 0.21 | 0.96 | medal | meddle | 0.99 | 0.11 | 0.90
bard | barred | 0.98 | 0.01 | 0.96 | metal | mettle | 1.00 | 0.31 | 0.92
bare | bear | 1.00 | 0.51 | 1.00 | meter | metre | 0.99 | 0.59 | 0.95
bark | barque | 0.96 | 0.33 | 0.86 | might | mite | 0.99 | 0.38 | 0.96
baron | barren | 1.00 | 0.07 | 0.97 | mind | mined | - | 0.21 | -
base | bass | 0.99 | 0.28 | 0.98 | miner | minor | 1.00 | 0.17 | 0.95
bazaar | bizarre | 0.98 | 0.12 | 0.92 | missed | mist | 0.99 | 0.20 | 0.98
be | bee | 0.99 | 0.35 | 0.98 | moat | mote | 0.99 | 0.46 | 0.93
beach | beech | 1.00 | 0.47 | 0.97 | mode | mowed | 0.99 | 0.21 | 0.94
bean | been | 0.92 | 0.29 | 0.96 | moor | more | 0.99 | 0.41 | 0.94
beat | beet | - | 0.41 | - | morning | mourning | 1.00 | 0.46 | 0.99
beer | bier | 0.99 | 0.24 | 0.96 | muscle | mussel | 0.99 | 0.48 | 0.95
bel | bell | 0.99 | 0.29 | 0.98 | naval | navel | - | 0.34 | -
berry | bury | 0.99 | 0.16 | 0.97 | nay | neigh | - | 0.34 | -
berth | birth | 1.00 | 0.08 | 1.00 | none | nun | 0.99 | 0.35 | 0.84
bitten | bittern | - | 0.63 | - | ode | owed | 0.99 | 0.30 | 0.95
blew | blue | 1.00 | 0.48 | 1.00 | oh | owe | 0.97 | 0.55 | 0.82
boar | bore | 0.99 | 0.39 | 0.96 | one | won | 1.00 | 0.39 | 1.00
board | bored | 1.00 | 0.13 | 0.99 | pail | pale | 1.00 | 0.27 | 0.99
boarder | border | 0.99 | 0.34 | 0.92 | pain | pane | 1.00 | 0.38 | 0.98
bold | bowled | 0.99 | 0.50 | 0.92 | pair | pare | 0.99 | 0.35 | 0.95
born | borne | 1.00 | 0.29 | 0.99 | palate | palette | 0.98 | 0.50 | 0.88
bough | bow | 0.99 | 0.48 | 0.97 | pause | paws | 1.00 | 0.34 | 0.99
boy | buoy | 0.96 | 0.35 | 0.88 | pea | pee | 0.93 | 0.54 | 0.87
braid | brayed | - | 0.60 | - | peace | piece | 1.00 | 0.15 | 0.99
brake | break | 1.00 | 0.38 | 0.99 | peak | peek | - | 0.30 | -
bread | bred | 1.00 | 0.24 | 0.99 | peal | peel | 0.99 | 0.21 | 0.96
bridal | bridle | 1.00 | 0.36 | 0.97 | peer | pier | 0.99 | 0.33 | 0.96
broach | brooch | 0.98 | 0.52 | 0.89 | plain | plane | 1.00 | 0.40 | 0.99
but | butt | 0.97 | 0.37 | 0.92 | pleas | please | - | 0.42 | -
buy | by | 0.99 | 0.25 | 0.98 | plum | plumb | 0.99 | 0.57 | 0.96
calendar | calender | 0.98 | 0.39 | 0.95 | pole | poll | 0.98 | 0.19 | 0.96
canvas | canvass | 0.99 | 0.41 | 0.98 | practice | practise | 1.00 | 0.74 | 0.99
cast | caste | 0.99 | 0.33 | 0.98 | praise | prays | 0.99 | 0.51 | 0.99
caught | court | 0.95 | 0.11 | 0.91 | principal | principle | 1.00 | 0.48 | 1.00
ceiling | sealing | 1.00 | 0.63 | 0.95 | profit | prophet | 1.00 | 0.30 | 0.99
cell | sell | 0.99 | 0.06 | 0.95 | quarts | quartz | 0.99 | 0.49 | 0.95
censer | censor | 0.98 | 0.35 | 0.78 | rain | reign | 1.00 | 0.18 | 0.99
cent | scent | 0.99 | 0.00 | 0.98 | raise | rays | 1.00 | 0.10 | 0.99
check | cheque | 0.99 | 0.61 | 0.98 | rap | wrap | 0.99 | 0.40 | 0.96
chord | cord | 1.00 | 0.42 | 0.98 | raw | roar | 0.96 | 0.18 | 0.88
cite | sight | 0.99 | 0.24 | 0.97 | read | reed | 0.97 | 0.28 | 0.97
clew | clue | 0.99 | 0.83 | 0.96 | real | reel | 1.00 | 0.27 | 0.97
climb | clime | 0.99 | 0.29 | 0.93 | reek | wreak | 0.98 | 0.33 | 0.92
coarse | course | 1.00 | 0.20 | 0.99 | rest | wrest | 0.99 | 0.54 | 0.95
coign | coin | - | 0.37 | - | right | rite | 1.00 | 0.19 | 0.96
colonel | kernel | 0.98 | 0.02 | 0.94 | ring | wring | 0.99 | 0.48 | 0.97
complacent | complaisant | - | 0.54 | - | road | rode | 1.00 | 0.66 | 0.96
complement | compliment | 1.00 | 0.27 | 0.96 | roe | row | 0.95 | 0.23 | 0.80
coo | coup | 0.98 | 0.35 | 0.87 | role | roll | 1.00 | 0.18 | 0.99
cops | copse | 0.98 | 0.39 | 0.96 | rood | rude | - | 0.32 | -
council | counsel | 1.00 | 0.52 | 0.99 | root | route | 0.98 | 0.06 | 0.98
creak | creek | 0.99 | 0.37 | 0.98 | rose | rows | 1.00 | 0.37 | 0.99
crews | cruise | 1.00 | 0.67 | 0.95 | rote | wrote | 0.98 | 0.46 | 0.91
currant | current | 0.99 | 0.11 | 0.97 | rough | ruff | 0.99 | 0.51 | 0.94
dam | damn | 0.99 | 0.21 | 0.95 | rung | wrung | 1.00 | 0.33 | 0.97
days | daze | 0.99 | 0.47 | 0.94 | rye | wry | 0.99 | 0.35 | 0.92
dear | deer | 0.99 | 0.08 | 0.98 | sale | sail | 1.00 | 0.32 | 0.99
descent | dissent | 1.00 | 0.42 | 0.98 | sane | seine | - | 0.09 | -
desert | dessert | 0.99 | 0.14 | 0.97 | sauce | source | 0.97 | 0.22 | 0.97
dew | due | 1.00 | 0.25 | 0.98 | saw | soar | 0.96 | 0.33 | 0.92
die | dye | 0.99 | 0.26 | 0.89 | scene | seen | 1.00 | 0.41 | 1.00
done | dun | 0.99 | 0.21 | 0.97 | sea | see | 1.00 | 0.31 | 0.98
draft | draught | 0.99 | 0.30 | 0.96 | seam | seem | 0.98 | 0.30 | 0.93
dual | duel | 0.98 | 0.48 | 0.88 | sear | seer | 0.98 | 0.57 | 0.86
earn | urn | 0.99 | 0.04 | 0.89 | seas | sees | 0.99 | 0.18 | 0.96
ewe | yew | 0.99 | 0.45 | 0.90 | sew | so | 0.99 | 0.55 | 0.98
faint | feint | 0.99 | 0.55 | 0.95 | shear | sheer | 0.99 | 0.25 | 0.94
fair | fare | 1.00 | 0.42 | 0.99 | side | sighed | 1.00 | 0.31 | 0.99
farther | father | 0.98 | 0.29 | 0.99 | sign | sine | - | 0.39 | -
feat | feet | 1.00 | 0.37 | 0.99 | slay | sleigh | 0.99 | 0.30 | 0.97
few | phew | - | 0.38 | - | sole | soul | 1.00 | 0.43 | 0.98
find | fined | 0.99 | 0.24 | 0.97 | some | sum | 0.99 | 0.34 | 0.99
fir | fur | 1.00 | 0.32 | 0.99 | son | sun | 1.00 | 0.26 | 1.00
flaw | floor | 0.97 | 0.30 | 0.87 | sort | sought | 0.96 | 0.21 | 0.96
flea | flee | 0.99 | 0.36 | 0.98 | staid | stayed | 1.00 | 0.55 | 0.99
floe | flow | 0.99 | 0.42 | 0.89 | stair | stare | 1.00 | 0.27 | 0.99
flour | flower | 1.00 | 0.30 | 0.99 | stake | steak | 1.00 | 0.19 | 0.98
for | fore | 0.95 | 0.26 | 0.99 | stalk | stork | 0.96 | 0.55 | 0.92
fort | fought | 0.95 | 0.42 | 0.93 | stationary | stationery | 0.99 | 0.27 | 0.85
forth | fourth | 1.00 | 0.25 | 1.00 | steal | steel | 1.00 | 0.19 | 0.99
foul | fowl | 1.00 | 0.36 | 0.97 | stile | style | 0.99 | 0.21 | 0.89
franc | frank | 0.98 | 0.31 | 0.95 | storey | story | 0.99 | 0.37 | 0.95
freeze | frieze | - | 0.38 | - | straight | strait | 1.00 | 0.39 | 0.97
furs | furze | - | 0.56 | - | sweet | suite | 1.00 | 0.03 | 0.99
gait | gate | 1.00 | 0.31 | 0.99 | tacks | tax | 0.98 | 0.25 | 0.96
gamble | gambol | - | 0.41 | - | tale | tail | 1.00 | 0.23 | 1.00
gild | guild | - | 0.41 | - | taught | taut | 0.99 | 0.12 | 0.93
gilt | guilt | 1.00 | 0.21 | 0.99 | team | teem | - | 0.28 | -
gnaw | nor | 0.95 | 0.38 | 0.84 | tear | tier | 0.98 | 0.24 | 0.92
grate | great | 1.00 | 0.15 | 0.97 | teas | tease | 0.99 | 0.35 | 0.97
greys | graze | 0.98 | 0.65 | 0.90 | tern | turn | - | 0.32 | -
grisly | grizzly | 0.99 | 0.64 | 0.88 | there | their | 0.99 | 0.34 | 0.98
groan | grown | 0.99 | 0.25 | 0.95 | threw | through | 0.99 | 0.39 | 0.99
guessed | guest | 1.00 | 0.33 | 0.99 | throes | throws | 0.99 | 0.31 | 0.95
hail | hale | 0.99 | 0.34 | 0.97 | throne | thrown | 1.00 | 0.30 | 0.96
hair | hare | 1.00 | 0.32 | 0.98 | thyme | time | 0.99 | 0.25 | 0.97
hall | haul | 1.00 | 0.13 | 0.93 | tide | tied | 1.00 | 0.10 | 0.99
hart | heart | 0.99 | 0.23 | 0.94 | tire | tyre | 0.99 | 0.21 | 0.90
haw | hoar | 0.91 | 0.12 | 0.71 | to | too | 0.97 | 0.63 | 0.97
hay | hey | 0.98 | 0.37 | 0.92 | toad | toed | 0.98 | 0.55 | 0.96
he’d | heed | 0.99 | 0.39 | 0.98 | told | tolled | 0.98 | 0.33 | 0.96
heal | heel | 1.00 | 0.37 | 0.95 | ton | tun | - | 0.55 | -
hear | here | 1.00 | 0.58 | 0.98 | vain | vane | 0.99 | 0.19 | 0.96
heard | herd | 0.99 | 0.32 | 0.97 | vale | veil | 1.00 | 0.65 | 0.98
hew | hue | 0.99 | 0.34 | 0.94 | vial | vile | 0.99 | 0.33 | 0.94
hi | high | 0.98 | 0.18 | 0.98 | wain | wane | 0.98 | 0.73 | 0.86
higher | hire | 1.00 | 0.27 | 0.98 | waist | waste | 1.00 | 0.13 | 0.99
him | hymn | 0.99 | 0.28 | 0.98 | wait | weight | 1.00 | 0.21 | 0.99
ho | hoe | 0.98 | 0.40 | 0.96 | waive | wave | 0.99 | 0.22 | 0.95
hoard | horde | 0.99 | 0.47 | 0.89 | war | wore | 0.99 | 0.23 | 0.96
hoarse | horse | 1.00 | 0.21 | 0.99 | ware | wear | 0.99 | 0.32 | 0.97
hour | our | 0.98 | 0.28 | 0.94 | warn | worn | 1.00 | 0.02 | 0.98
idle | idol | 1.00 | 0.46 | 0.97 | watt | what | - | 0.52 | -
in | inn | 0.98 | 0.38 | 0.95 | wax | whacks | - | 0.48 | -
it’s | its | 0.99 | 0.20 | 0.95 | way | weigh | 0.99 | 0.36 | 0.96
key | quay | 0.97 | 0.33 | 0.95 | we | wee | 0.97 | 0.21 | 0.96
knave | nave | 0.99 | 0.06 | 0.90 | we’d | weed | 0.96 | 0.26 | 0.94
knead | need | 0.99 | 0.51 | 0.98 | weak | week | 1.00 | 0.10 | 1.00
knew | new | 1.00 | 0.31 | 0.99 | weal | we’ll | 0.94 | 0.15 | 0.90
knight | night | 1.00 | 0.26 | 0.99 | wean | ween | - | 0.57 | -
knob | nob | - | 0.59 | - | weather | whether | 0.99 | 0.21 | 0.95
knot | not | 1.00 | 0.26 | 0.98 | weir | we’re | - | 0.08 | -
know | no | 1.00 | 0.63 | 0.97 | were | whirr | 0.96 | 0.27 | 0.83
knows | nose | 1.00 | 0.18 | 0.99 | which | witch | 0.99 | 0.28 | 0.94
lac | lack | - | 0.10 | - | whig | wig | 0.99 | 0.16 | 0.96
lain | lane | 0.99 | 0.30 | 0.96 | whine | wine | 1.00 | 0.13 | 0.97
laps | lapse | - | 0.32 | - | whirl | whorl | 0.97 | 0.37 | 0.78
larva | lava | - | 0.45 | - | whirled | world | 1.00 | 0.26 | 0.96
law | lore | 0.96 | 0.33 | 0.89 | whit | wit | 0.99 | 0.50 | 0.96
lea | lee | 0.98 | 0.53 | 0.89 | white | wight | 0.99 | 0.32 | 0.95
lead | led | 0.94 | 0.59 | 0.97 | who’s | whose | 0.99 | 0.26 | 0.99
lessen | lesson | 1.00 | 0.26 | 0.92 | woe | whoa | 0.97 | 0.23 | 0.90
levee | levy | 0.98 | 0.45 | 0.82 | wood | would | 0.98 | 0.26 | 0.96
liar | lyre | 0.99 | 0.31 | 0.93 | yoke | yolk | 0.99 | 0.32 | 0.88
licence | license | 0.99 | 0.72 | 0.94 | you’ll | yule | 0.95 | 0.48 | 0.91
lie | lye | 0.97 | 0.47 | 0.71 |  |  |  |  |