The short life of a drop

Guillermo Hernández-Cruz1, Minerva Vargas2, Heberto Pérez1

J. Arturo Pimentel3, Gabriel Corkidi3 and Eduardo Ramos1

1Center for Energy Research, Universidad Nacional Autónoma de México

2 Instituto Tecnológico de Zacatepec

3 Biotechnology Institute, Universidad Nacional Autónoma de México

###### Abstract

This is the companion paper of the Fluid Dynamics Video ”The short life of a
drop” where it is argued that the geometry of the sediment of a drop of water
with particles in suspension can be correlated with the dynamics of the fluid
inside the drop during the drying process.

We study the dynamics of the flow inside a sessile drop of water with a
suspension of polystyrene spheres of 1$\mu$m in diameter. According to our
observations it is possible to correlate the motion of the fluid inside the
drop with the pattern of sediments. The initial volume of the drops is 1$\mu$l
and the motion is recorded with a micro PIV equipment that comprises an
Inverted Optical Microscope (Olympus IX71) and a video camera (Optronis
CR5000x2) that captures 512 $\times$ 512 images at a rate of 60 fps. The
microscope has an amplification of 4X and the objective has a nominal depth of
field of 40 $\mu$m. The drop sits on a soda lime glass substrate that is kept
at 20 ∘C. The initial footprint of the drop is 2 mm in diameter and the
suspension has 5.67 $\times 10^{6}$ spheres per $\mu$l of suspension. The
evaporation process lasts typically several minutes with the exact duration
depending on the relative humidity in the room. The evaporation process can be
divided in four parts. In the first, the drop takes the shape of a spherical
cap and the velocity of the fluid inside the drop is very slow and difficult
to resolve but particles emigrate steadily towards the contact line and build
up to start forming the coffee ring. In the second part, the drop is almost
flat and the velocity field inside the drop is mostly radial to compensate for
the preferential evaporation near the contact line. In this part, the radial
velocity near the edge diverges as the thickness of the liquid layer is
reduced and the rapid accumulation of the particles near the contact line
result in a quick thickening of the coffee ring. In these two first stages,
the edge of the drop remains pinned and can be well approximated by a circle.
The third part of the process starts when the liquid film is sufficiently thin
and rips off from the coffee ring and a complicated interaction, of surface
tension (which pulls the liquid to reduce the area) and evaporation, determine
the internal flow. In the initial stages of this part a transient formation of
a secondary coffee ring slows down the motion of the retracting contact line
forming relatively thick sediment segments with the shape of short circular
arcs. Subsequently, the geometry of the outer boundary of the drop becomes
unstable and small perturbations modify its local curvature to generate a time
dependent non-circular edge. During the fourth part of the process, local
irregular features of the drop edge are smoothed out by surface tension. This
effect also generates fast inward radial flows which sweep particles forming
thin radial segments of sediments. This situation prevails until the liquid
totally evaporates.


††thanks: This project was supported by Brookhaven Science Associates, LLC
under Contract No. DESC0012704 with the U.S. Department of Energy.

Also at ]Brookhaven National Laboratory

# BAGELS: A General Method for Minimizing the Rate of
Radiative Depolarization in Electron Storage Rings

Matthew G. Signorelli<EMAIL_ADDRESS>Georg H. Hoffstaetter de Torquat  [
Department of Physics, Cornell University, Ithaca, NY 14853, USA  Yunhai Cai
SLAC National Accelerator Laboratory, 2575 Sand Hill Road, Menlo Park,
California 94205, USA

###### Abstract

We present a novel method for minimizing the effects of radiative
depolarization in electron storage rings by use of vertical orbit bumps in the
arcs. Electron polarization is directly characterized by the RMS of the so-
called spin orbit coupling function in the bends. In the Electron Storage Ring
(ESR) of the Electron-Ion Collider (EIC), as was the case in HERA, this
function is excited by the spin rotators. Individual vertical corrector coils
in the arcs can have varying impacts on this function globally. In this
method, we use a singular value decomposition of the response matrix of the
spin-orbit coupling function with each coil to define a minimal number of most
effective groups of coils, motivating the name “Best Adjustment Groups for
ELectron Spin” (BAGELS) method. The BAGELS method can be used to minimize the
depolarizing effects in an ideal lattice, and to obtain fine-tuning knobs to
restore the minimization in rings with realistic closed orbit distortions.
Furthermore, the least effective groups can instead be chosen for other
applications where no impact on polarization is desirable, e.g. global
coupling compensation or vertical emittance creation. Application of the
BAGELS method has significantly increased the polarization in simulations of
the 18 GeV ESR, beyond achievable with conventional methods.

††preprint: APS/123-QED

##  I Introduction

Letting $s$ specify the azimuthal position along the closed orbit and
$\langle...\rangle_{s}$ denote an average over all phase space coordinates
$\vec{r}=(x,p_{x},y,p_{y},z,\delta)$ at the azimuth $s$, the rate of radiative
depolarization in an electron ring can be calculated as

$\tau_{dep}^{-1}=\frac{5\sqrt{3}}{8}\frac{r_{e}\gamma^{5}\hbar}{m}\frac{1}{C}\oint
ds\left\langle
g^{3}\frac{11}{18}\left\lvert\frac{\partial\hat{n}}{\partial\delta}\right\rvert^{2}\right\rangle_{s}\
,$ (1)

where $r_{e}$ is the classical electron radius, $\gamma$ is the Lorentz
factor, $C$ is the circumference of the ring, $g=g(s)$ is the local bending
strength ($g=1/\rho$ where $\rho$ is the bending radius), $\delta=\frac{\Delta
p}{p_{0}}$ is the relative momentum deviation, and
$\hat{n}=\hat{n}(\vec{r},s)$ is the invariant spin field (ISF) [1]. The ISF is
a special $2\pi$-periodic spin field that solves the Thomas-BMT equation along
orbital trajectories,

$\displaystyle\hat{n}(\vec{r},s)$
$\displaystyle=\mathbf{R}(\vec{r}_{0},s_{0};s)\hat{n}(\vec{r}_{0},s_{0})\ ,$
(2a) $\displaystyle\hat{n}(\vec{r},s_{0}+C)$
$\displaystyle=\hat{n}(\vec{r},s_{0})\ .$ (2b)

On the closed orbit, the ISF reduces to $\hat{n}_{0}$, which is the 1-turn
periodic spin solution on the closed orbit. The projection of a particle’s
spin along the ISF, or the spin action $J_{s}=\hat{n}\cdot\vec{S}$, is an
adiabatic invariant [2]. However, in the presence of stochastic radiation, a
particle will experience an instantaneous change in its phase space
coordinates (kick to $\delta$) when emitting a photon in the bends. The
invariant spin direction at its new phase space coordinate will in general be
different from the invariant spin direction at its initial phase space
coordinate, thus changing the spin action. This effect leads to a rapid
decoherence of the particles’ spins in an ensemble if left uncorrected [3]. In
a perfectly flat ring with only horizontal dipoles, regular quadrupoles, and
regular sextupoles, there are no radiative depolarization effects; the beam
will lie almost entirely in the midplane, having an approximately zero
vertical beam size due to radiation damping. All magnetic fields in the
midplane are vertical in such a ring, and so not only is $\hat{n}_{0}$
vertical, but so is the ISF over the entire phase space occupied by the beam.
Therefore, $\partial\hat{n}/\partial\delta\approx 0$. However, once spin
rotators are introduced, the regions where $\hat{n}_{0}$ is not vertical will
excite radiative depolarization, because particles’ spins will precess
different amounts than $\hat{n}_{0}$ does depending on the phase space
coordinates. This causes $\partial\hat{n}/\partial\delta\neq 0$.

Traditionally to minimize the effects of radiative polarization, the following
procedure is executed [4]: first, the magnet strengths and spin rotator
configuration are set so that, at as many azimuthal positions as possible, the
spin dependence after 1-turn on the horizontal, vertical, and longitudinal
positions/momenta are removed; this is setting the $\mathbf{G}$-matrices in
the SLIM formalism equal to zero [5], or equivalently the spin response
functions in the spin response formalism equal to zero [6], and is called
strong synchro-beta spin matching. Then in a realistic ring, the random closed
orbit distortions will cause a tilt of $\hat{n}_{0}$ in the arcs outside of
the spin rotator. This tilt, which will precess with frequency $2\pi
a\gamma_{0}$, will excite $\partial\hat{n}/\partial\delta$. Harmonic closed
orbit spin matching is performed to correct the tilt by using vertical orbit
bumps to cancel out integer spin harmonics nearest to $a\gamma_{0}$ [7]. By
cancelling out these harmonics, the tilt is reduced. This was used in HERA
with success [8]. In summary, the traditional procedure focuses on reducing
the spin-orbit coupling and the integer spin harmonics, which we then expect
to also reduce the RMS $\partial\hat{n}/\partial\delta$ globally.

Here we present a new method, which we call the “Best Adjustment Groups for
ELectron Spin” (BAGELS) method, that instead focuses directly on the RMS
$\partial\hat{n}/\partial\delta$ itself in the bends. The BAGELS method is
executed by first calculating the response of $\partial\hat{n}/\partial\delta$
at each bend in the ring for each possible “unit vertical closed orbit bump”.
Then, a singular value decomposition of the response matrix is performed to
obtain a minimal number of best “groups”, or knobs, of corrector coils for
electron spin. The BAGELS method can be used for both spin matching the ideal
lattice (traditionally performed with strong synchro-beta spin matching) using
only vertical corrector coils, and obtaining fine-tuning knobs to correct the
spin match with random closed orbit distortions (traditionally performed with
harmonic closed orbit spin matching). Furthermore, BAGELS can be used to
alternatively obtain knobs that have no impact on polarization, for purposes
such as delocalized vertical dispersion and/or coupling creation for vertical
emittance creation. In this manuscript, we describe how to execute the BAGELS
method in detail, and showcase four results of using BAGELS in simulations of
the planned 18 GeV Electron Storage Ring (ESR) of the Electron-Ion Collider
(EIC): (1) significantly improving polarization in the ideal 1-colliding
interaction region (IR) lattice, and more than doubling the polarization in
the 2-colliding IR lattice, only using four knobs and a maximum orbit
excursion of $<$1.3 mm, (2) restoring the spin match when including random
magnet errors (translational/rotational misalignments, field errors) using
only four knobs for 10 error seeds, (3) creating a “polarization-transparent”
delocalized vertical dispersion-creating knob for vertical emittance creation,
and (4) creating a “polarization-transparent” delocalized coupling-creating
knob for vertical emittance creation.

##  II The BAGELS Method

Because $\hat{n}_{0}$ is vertical in the arcs, horizontal orbit bumps in the
arcs (with vertical magnetic fields) will have essentially zero impact on the
radiative depolarization. Therefore, vertical orbit bumps are to be used. Let
$\theta_{i}$ represent the strength of some arbitrary, single vertical orbit
bump. For example, it could be the strength of both coils in a standard
$\pi$-bump or a $3\pi$-bump, etc. We will refer to this bump as a unit bump,
for reasons that will become apparent shortly. The values of
$|\partial\hat{n}/\partial\delta|$ at all $m$ bends in the ring can be written
as a function of all $n$ unit bumps in the ring, to first-order, as

$\displaystyle\begin{pmatrix}|\partial\hat{n}/\partial\delta|_{1}\\\ \vdots\\\
|\partial\hat{n}/\partial\delta|_{m}\end{pmatrix}=\mathbf{M}_{\textrm{m}\times\textrm{n}}\begin{pmatrix}\theta_{1}\\\
\vdots\\\
\theta_{n}\end{pmatrix}+\begin{pmatrix}|\partial\hat{n}/\partial\delta|_{1}\\\
\vdots\\\ |\partial\hat{n}/\partial\delta|_{m}\end{pmatrix}_{0}\ ,$
$\displaystyle\mathbf{M}_{\textrm{m}\times\textrm{n}}=\begin{pmatrix}\frac{\partial(|\partial\hat{n}/\partial\delta|_{1})}{\partial\theta_{1}}&\cdots&\frac{\partial(|\partial\hat{n}/\partial\delta|_{1})}{\partial\theta_{n}}\\\
\vdots&\ddots&\vdots\\\
\frac{\partial(|\partial\hat{n}/\partial\delta|_{m})}{\partial\theta_{1}}&\cdots&\frac{\partial(|\partial\hat{n}/\partial\delta|_{m})}{\partial\theta_{n}}\end{pmatrix}$
(3)

$\mathbf{M}_{\textrm{m}\times\textrm{n}}$ is the response matrix of
$|\partial\hat{n}/\partial\delta|$ at the ends of each bend, for each unit
bump. The spin-orbit coupling function can be evaluated using any method
(SLIM, spin response, SITF, etc). The derivatives w.r.t each unit bump can be
calculated using automatic differentiation if supported by the software, or
finite differencing with an appropriately chosen step size (i.e. individually
turning on and off every unit bump and calculating the response). We note that
one could alternatively use $1/L\int ds\
g^{3}\left\lvert\partial\hat{n}/\partial\delta\right\rvert^{2}$ evaluated at
each bend (with the integral perhaps approximated using trapezoidal rule), to
directly observe the response of the contribution to $\tau_{dep}^{-1}$.
However, we did not find this to be necessary, and in fact all of the results
shown in this manuscript only use the response of the $x$-component of the
spin-orbit coupling function at each bend.

For spin matching, we now must determine a minimal number of vectors, or
“groups”, or “knobs”, that scale the matrix most significantly, so we can get
the most “bang-for-our-buck” using minimal coil strengths and orbit
excursions. If $\mathbf{M}_{\textrm{m}\times\textrm{n}}$ were a square matrix,
we could just take its eigenvectors with the largest eigenvalues. However, for
non-square matrices, we must use singular value decomposition (SVD). The
compact SVD factors a non-square real matrix into the form

$\mathbf{M}_{\textrm{m}\times\textrm{n}}=\mathbf{U}_{\textrm{m}\times\textrm{r}}\mathbf{\Sigma}_{\textrm{r}\times\textrm{r}}(\mathbf{V}_{\textrm{n}\times\textrm{r}})^{\textrm{T}}\
,\ r=\min{(n,m)}$ (4)

where $\mathbf{U}^{\textrm{T}}\mathbf{U}=\mathbf{V}^{\textrm{T}}\mathbf{V}=1$
and $\mathbf{\Sigma}$ is a diagonal matrix with the real non-negative singular
values $\sigma_{i}$ in descending order along the diagonal. The columns of
$\mathbf{V}$ and $\mathbf{U}$ contain the right- and left-singular vectors
respectively, such that

$\mathbf{M}\mathbf{v}_{i}=\sigma_{i}\mathbf{u}_{i}\ ,\
\mathbf{M}^{T}\mathbf{u}_{i}=\sigma_{i}\mathbf{v}_{i}\ .$ (5)

For spin matching, we therefore choose a minimal number of right-singular
vectors with the largest corresponding singular values as our groups, for a
maximal impact on the polarization. For other purposes, we generally choose
the right-singular vectors with the smallest singular values, for a minimal
impact on polarization. These are our “knobs”, that describe how to scale each
corrector coil accordingly for a unit change to the knob to optimally achieve
the goal.

We now explicitly lay out the BAGELS method, step-by-step:

  1. 1.

Choose a unit bump. The choice of unit bump should be guided by the goal: e.g.
for spin matching, a bump that automatically cancels its own generated
vertical dispersion and coupling (if there are sextupoles inbetween the coils)
is ideal. For global coupling compensation using vertical orbits in
sextupoles, a unit bump that intentionally creates delocalized coupling should
be used instead.

  2. 2.

Calculate the response matrix. The change in $\partial\hat{n}/\partial\delta$
at each bend for each individual unit bump must be obtained to construct the
response matrix in Eq. (3). This can be calculated with automatic
differentiation or finite differencing with an appropriate step size.

  3. 3.

Perform an SVD of the response matrix to get the Best Adjustment Groups for
ELectron Spin. The “best” groups will be those vectors with either the largest
singular values for maximal impact on polarization, or the smallest singular
values for minimal impact on polarization, depending on the goal.

##  III Results

###  III.1 Spin Matching the ESR of the EIC

The 18 GeV ESR of the EIC notably lacks a longitudinal spin match using
traditional strong synchro-beta spin matching, due to unfeasibly large
solenoid strengths required to achieve such [9]. The lack of a longitudinal
spin match reduces the asymptotic polarization $P_{dk}$ in the 1-colliding IR
(1-IP, with only one spin rotator) lattice from 61% to 35%, and in the
2-colliding IR (2-IP, with two spin rotators) lattice from 47% to 17%. Without
any additional correction, the polarization in the ideal 1-IP lattice
marginally satisfies requirements, and the polarization in the ideal 2-IP
lattice falls short of the requirements. After application of the BAGELS
method, we bring the asymptotic polarization to 66% in the 1-IP lattice and
58% in the 2-IP lattice, well-exceeding the polarization requirements.

To apply the BAGELS method, we first choose a unit bump. For spin matching, a
unit bump that keeps the generated vertical dispersion and coupling (created
by a vertical orbit in sextupoles) localized within the bump is desirable, for
minimal impact on the optics. The 18 GeV ESR has a 90∘ phase advance in the
arc cells, with two sextupole families per plane. To localize the coupling in
this lattice, we can use a $2\pi n$-bump, as shown in Fig. 1. With such a
bump, all skew-quadrupole terms that the closed orbit sees are cancelled out
by an equal but opposite sign skew-quadrupole term in the second half of each
oscillation.

Figure 1: Schematic of a $2\pi n$-bump that creates localized coupling but
delocalized vertical dispersion in arc cells with a 90∘ degree phase advance
and four total sextupole families (two per plane). The blue rectangles
represent sextupoles, and the red vertical corrector coils.

To cancel the generated vertical dispersion wave by this bump, we can place a
second equal (opposite) $2\pi n$-bump that is out of phase (in phase) with the
first. A diagram showing this type of unit bump is shown in the appendix in
Fig. 12. We therefore choose our unit bump to be all possible $2\pi n$-bumps
with equal (opposite) $2\pi n$-bump out of phase (in phase) in the ring per
Fig. 12. Note that we can overlap these bumps with the same dispersion and
coupling cancellation, to first-order.

After calculating the response of $\partial\hat{n}/\partial\delta$ at each
bend for each unit bump, we calculate the SVD of the response matrix and
obtain the best groups. Using only the four best knobs, we achieved a
significant improvement in the polarization. Figure 2 shows the difference in
the asymptotic polarization $P_{\infty}$ before and after turning the four
knobs, obtained with the BAGELS method, in the 1-IP lattice for maximum
polarization, and Fig. 3 shows the difference in
$\partial\hat{n}/\partial\delta$, dispersion, coupling, and orbit before and
after turning the knobs.

Figure 2: Asymptotic polarization before (orange) and after (blue) applying
the BAGELS method for spin matching. 3rd order map tracking refers to
calculating $P_{\infty}$ using $\tau_{dep}$ obtained from 3rd order Monte
Carlo tracking with radiation. Figure 3: The 18 GeV 1-IP ESR lattice before
(left) and after (right) varying four knobs calculated with the BAGELS method.
Figure 4: The 18 GeV 2-IP ESR lattice before (left) and after (right) varying
four knobs calculated with the BAGELS method.

The polarization requirements are well-exceeded in the 1-IP lattice after
application of the BAGELS method, and only four knobs with a maximal 1.3 mm
orbit was necessary. Repeating the BAGELS method on the 2-IP ESR, similarly
only the best four knobs were needed. Figure 5 shows the difference in the
polarization before and after turning the knobs for maximum polarization for
the 2-IP lattice and Fig. 4 shows the difference in
$\partial\hat{n}/\partial\delta$, dispersion, coupling, and orbit before and
after turning the knobs. The polarization requirements are now exceeded after
application of the BAGELS method.

Figure 5: Asymptotic polarization before (orange) and after (blue) applying
the BAGELS method for spin matching. 3rd order map tracking refers to
calculating $P_{\infty}$ using $\tau_{dep}$ obtained from 3rd order Monte
Carlo tracking with radiation.

###  III.2 Correcting the Spin Match with Random Closed Orbit Distortions

Figure 6: RMS errors used in sensitivity analysis of the 1-IP 18 GeV ESR.

In actual operation, there will inevitably be random closed orbit distortions
caused by magnet misalignments and field errors. Without any correction, this
will degrade the spin match. While traditionally harmonic closed orbit spin
matching was used to obtain knobs to restore the spin match, instead the
BAGELS method could be used to obtain the best knobs.

The RMS errors in Fig. 6 were used to generate 10 different error seeds of the
1-IP 18 GeV ESR. After correction of the orbit and coupling, five knobs
calculated using BAGELS were varied to correct the spin match. Figure 7 shows
the analytical $P_{\infty}$ obtained after correction of the spin match using
the BAGELS knobs for all 10 error seeds.

Figure 7: Asymptotic polarization for 10 different error seeds in the 1-IP ESR
after correction of the spin match using five knobs calculated with the BAGELS
method.

###  III.3 $\epsilon_{y}$-Creation with Delocalized Vertical Dispersion

Electron beams in flat rings will have vertical beam sizes very close to zero,
due to radiation damping. However, in order to not hurt the proton/light-ion
beam lifetime, the electron beam size must be matched to a certain degree with
the ion beam at the interaction point(s). One way of achieving beam size
matching is to increase the vertical emittance $\epsilon_{y}$ of the electron
beam by generating a delocalized vertical dispersion wave around the ring.
However, in general the creation of such a wave will have negative effects on
polarization unless done carefully. The BAGELS method can also be applied to
create vertical closed orbit bumps that have no impact on polarization, by
instead choosing the groups to be those with the smallest singular values for
minimal impact on polarization.

Figure 8: Asymptotic polarization in the 18 GeV 1-IP ESR after using
“polarization-transparent” vertical dispersion-creating knobs calculated with
the BAGELS method to achieve the desired $\epsilon_{y}$ for beam size
matching. 3rd order map tracking refers to calculating $P_{\infty}$ using
$\tau_{dep}$ obtained from 3rd order Monte Carlo tracking with radiation. The
nonlinear tracking gave an equilibrium $\epsilon_{y}=$ 3.0 nm vs. the
analytical calculation 2.1 nm.

Following the procedure, we first choose a unit bump that is best-suited to
achieve our goal. In this case, we need to create only delocalized vertical
dispersion, without delocalized coupling. Lone $2\pi$-bumps, shown in Fig. 13
give exactly that. Using our BAGELS-spin-matched ESR 1-IP lattice, we reapply
the BAGELS method instead using lone $2\pi$-bumps, and now choose our best
groups as those with the smallest singular values. These knobs can be turned
with essentially zero impact on the polarization, until the desired vertical
emittance is obtained. Figure 9 shows the $\partial\hat{n}/\partial\delta$,
dispersion, coupling, and orbit after setting the vertical emittance to 2.1
nm. Comparison of the analytical calculation of
$\partial\hat{n}/\partial\delta$ with that in Fig. 3 shows basically no
difference, however now the emittance requirements for beam-size matching are
satisfied using delocalized vertical dispersion. Figure 8 shows a scan of the
asymptotic polarization with the desired emittance, with nonlinear tracking.
We note that the nonlinearities do cause a greater disagreement in both
polarization and the equilibrium vertical emittance (calculated in tracking to
be 3.0 nm), however the result is still a significant improvement over the
baseline without any vertical emittance creation.

Figure 9: The 18 GeV 1-IP ESR lattice after using “polarization-transparent”
vertical dispersion-creating knobs calculated with the BAGELS method to
achieve the desired $\epsilon_{y}$ for beam size matching.

###  III.4 $\epsilon_{y}$-Creation with Delocalized Coupling

Figure 10: Asymptotic polarization in the 18 GeV 1-IP ESR after using
“polarization-transparent” coupling-creating knobs calculated with the BAGELS
method to achieve the desired $\epsilon_{y}$ for beam size matching. 3rd order
map tracking refers to calculating $P_{\infty}$ using $\tau_{dep}$ obtained
from 3rd order Monte Carlo tracking with radiation. The nonlinear tracking
gave an equilibrium $\epsilon_{y}=$ 4.6 nm vs. the analytical calculation 2.0
nm due to a higher-order synchro-beta resonance.

Another way to create vertical emittance is by decoupling some of the
horizontal emittance into the vertical. To create only delocalized coupling
without any delocalized vertical dispersion in this particular lattice with
90∘ phase advance and two sextupole families per plane, two equal $\pi$-bumps
separated by $2\pi n$ in betatron phase advance can be used, as shown in Fig.
14. Applying the BAGELS method using this type of bump as our unit bump, the
knobs with the smallest singular values can be varied with minimal impact on
the polarization. Figure 11 shows the $\partial\hat{n}/\partial\delta$,
dispersion, coupling, and orbit after setting the vertical emittance to 2.1
nm. Comparison of the analytical calculation of
$\partial\hat{n}/\partial\delta$ with that in Fig. 3 shows very little
difference, however now the emittance requirements for beam-size matching are
satisfied using delocalized coupling. Figure 10 shows a scan of the asymptotic
polarization with the desired emittance, with nonlinear tracking. We note that
a significant higher-order synchro-beta resonance was observed in nonlinear
tracking, suspected to be $Q_{y}-Q_{x}-Q_{s}$, causing the vertical emittance
to grow to 4.6 nm. While this is likely the cause for the greater disagreement
of the polarization calculated with nonlinear depolarizing effects vs. the
linear calculation, the result is still a significant improvement over the
baseline even with the very large vertical emittance.

Figure 11: The 18 GeV 1-IP ESR lattice after using “polarization-transparent”
coupling-creating knobs calculated with the BAGELS method to achieve the
desired $\epsilon_{y}$ for beam size matching.

##  IV Conclusions

The Best Adjustment Groups for ELectron Spin (BAGELS) method provides a
general approach for obtaining the best knobs of vertical corrector coils for
electron spin for a wide range of scenarios. The BAGELS method was used to
significantly improve the polarization in both the 1- and 2-colliding IR 18
GeV ESR lattices, in the latter case more than doubling the asymptotic
polarizations, with maximum orbit excursions of ¡ 1.3 mm and only four knobs
necessary. The BAGELS method was also used to obtain the best knobs to restore
the spin match when including random closed orbit distortions with 10 error
seeds. Finally, the BAGELS method was used to obtained “polarization-
transparent” knobs that create one of either delocalized vertical dispersion,
or delocalized coupling, for vertical emittance creation to achieve beam size
matching without any impact on polarization.

###### Acknowledgements.

We thank Yuri Nosochkov for the suggestion of using a second $2\pi n$-bump to
cancel out the dispersion wave created by the first, and Eliana Gianfelice-
Wendt for previous work using harmonic bumps to increase polarization in the
ESR.

##  Appendix A Unit Bumps

Figure 12:  Selected unit bump for spin matching using the BAGELS method, with
localized vertical dispersion and localized coupling: (Left) Two equal $2\pi
n$-bumps out of betatron phase with each other to cancel the vertical
dispersion wave generated by the first. (Right) Two opposite $2\pi n$-bumps in
betatron phase with each other to cancel the vertical dispersion wave
generated by the first.  Figure 13:  Selected unit bump ($2\pi$-bump) for
creating only delocalized vertical dispersion, with coupling locally
cancelled. Figure 14:  Selected unit bump (equal $\pi$-bumps in-phase with
each other) for creating only delocalized coupling, with vertical dispersion
locally cancelled.

## References

  * Derbenev and Kondratenko [1973] Y. S. Derbenev and A. M. Kondratenko, Polarization kinematics of particles in storage rings, Sov. Phys. JETP 37, 968 (1973).
  * Hoffstaetter [2006] G. H. Hoffstaetter,  _High-Energy Polarized Proton Beams: A Modern View_, Vol. 218 (2006).
  * Montague [1984] B. W. Montague, Polarized beams in high energy storage rings, Physics Reports 113, 1 (1984).
  * Barber and Ripken [2023] D. P. Barber and G. Ripken, Handbook of Accelerator Physics and Engineering (3rd Edition) (World Scientific Publishing Company, 2023) Chap. 2.6.6 - 2.6.8.
  * Chao [1981] A. W. Chao, Evaluation of radiative spin polarization in an electron storage ring, Nucl. Inst. and Meth. 180, 29 (1981).
  * Ptitsyn  _et al._ [2009] V. I. Ptitsyn, Y. M. Shatunov, and S. R. Mane, Spin response formalism in circular accelerators, Nucl. Inst. and Meth. 608, 225 (2009).
  * Barber  _et al._ [1985] D. P. Barber, H. Mais, G. Ripken, and R. Rossmanith, A General Harmonic Spin Matching Formalism for the Suppression of Depolarization Caused by Closed Orbit Distortions in Electron Storage Rings, DESY-85-044 (1985).
  * Barber  _et al._ [1994] D. P. Barber _et al._ , High spin polarization at the HERA Electron Storage Ring, Nucl. Instrum. Meth. A 338, 166 (1994).
  * Signorelli  _et al._ [2023] M. G. Signorelli, G. H. Hoffstaetter, and V. Ptitsyn, Electron polarization preservation in the EIC, JACoW IPAC2023, MOPA052 (2023).


# Developing a Multi-variate Prediction Model For The Detection of COVID-19
From Crowd-sourced Respiratory Voice Data

Wafaa Aljbawi, Sami O. Simmons, and Visara Urovi

###### Abstract

COVID-19 has affected more than 223 countries worldwide. There is a pressing
need for non-invasive, low-costs and highly scalable solutions to detect
COVID-19, especially in low-resource countries where PCR testing is not
ubiquitously available. Our aim is to develop a deep learning model
identifying COVID-19 using voice recordings data spontaneously provided by the
general population (voice recordings and a short questionnaire) via their
personal devices. The novelty of this work is in the development of a deep
learning model for the identification of COVID-19 patients from voice
recordings.

Methods: We used the Cambridge University dataset consisting of 893 audio
samples, crowd-sourced from 4352 participants that used a COVID-19 Sounds app.
Voice features were extracted using a Mel-spectrogram analysis. Based on the
voice data, we developed deep learning classification models to detect
positive COVID-19 cases. These models included Long-Short Term Memory (LSTM)
and Convolutional Neural Network (CNN). We compared their predictive power to
baseline classification models, namely Logistic Regression and Support Vector
Machine.

Results: LSTM based on a Mel-frequency cepstral coefficients (MFCC) features
achieved the highest accuracy (89%,) with a sensitivity and specificity of
respectively 89% and 89%, The results achieved with the proposed model suggest
a significant improvement in the prediction accuracy of COVID-19 diagnosis
compared to the results obtained in the state of the art.

Conclusion: Deep learning can detect subtle changes in the voice of COVID-19
patients with promising results. As an addition to the current testing
techniques this model may aid health professionals in fast diagnosis and
tracing of COVID-19 cases using simple voice analysis.

###### Index Terms:

COVID-19, Voice Analysis, Deep Learning, Mel-Spectrogram, Machine Learning.

##  1 Introduction

The coronavirus or severe acute respiratory syndrome coronavirus 2 (SARS-
CoV-2) can cause severe respiratory disease in humans and has become a
potential threat to human health and the global economy. On February 1, 2022,
more than 5.5 million deaths from coronavirus disease 2019 (COVID-19) were
confirmed [1]. Since the outbreak in late December 2019, it has spread
worldwide [2]. One of the most concerning aspects of SARS-CoV-2 is its rapid
spread; the virus easily spreads through surfaces [3], air, breathing, talking
or physical contact [4], and thus it is possible to affect large populations
in a very short time. As a result, it is important to quickly identify who is
infected. Several research efforts have been carried out in order to avoid the
rapid spread of the epidemic and to effectively control the number of infected
people [5],[6], and[7]. In order to facilitate the detection of cases, several
researchers have been exploring the possibility of utilizing auditory data
produced by the human body (such as breathing [8], heart rate [9], and
vibration sounds [10]) to diagnose and track disease progression [11].

The most common symptoms of the virus include fever, dry cough, loss of smell
and taste, headache, muscle aches, diarrhea, conjunctivitis, and in more
severe cases, shortness of breath, chest pain, and loss of speech or movement
[12]. The respiratory track is impacted which can implicitly impact the voice
of the affected patients by resulting in a lack of speech energy and a loss of
voice due to shortness of breath and upper airway congestion [13]. Recurrent
dry coughs can also cause alterations in the vocal cords, reducing the quality
of one’s voice. A recent study found that individuals with COVID-19 had
changes in their voice’s acoustic characteristics due to inadequate airflow
through the vocal tract as a result of pulmonary and laryngological
dysfunction [13]. As a consequence, COVID-19 as a respiratory condition may
cause patients’ voices to become distinctive, leading to distinguishable voice
signatures.

With the emergence of COVID-19, several attempts were made to complement
standard testing procedures with effective automated diagnosis solutions. The
World Health Organization (WHO) considers nucleic acid-based real time reverse
transcription polymerase chain reaction (RT-PCR) to be the standard technique
for diagnosing COVID-19 [12]. Due to the high cost and restricted access to
the test, as well as the risk of exposing healthcare professionals and medical
staffs to the virus, it is also infeasible to go to medical centers and
undergo RT-PCR after every cough or uncomfortable feeling [12]. The Rapid
Antigen Test (RAT) is an alternative that does not require laboratory
processing and eliminates the time constraint of RT-PCR; however, its
sensitivity declines with lower viral loads, resulting in false negative
results in patients with lower levels of the SARS-CoV-2 virus [13]. As a
result, finding novel accessible and non-invasive techniques for diagnosing
this disease with greater precision is critical, as it can alleviate
unnecessary concerns and motivates patients to take necessary measures.

In this research, we developed an AI-based algorithm that identifies patients
who report being COVID-19 positive based on the results of their voice
analysis. Several other works have been proposed to tackle this problem such
as [5],[6],[7],[14],[15], and [16]. The novelty of our work is that we trained
our models on a large multi-modal dataset of COVID-19 respiratory sounds.
Besides, we developed both deep learning classification models and basic
machine learning models to determine which model is optimal for diagnosing
COVID-19.

The reminder of the paper is organized as follows. Section 2 provides a
literature review on voice analysis for COVID-19 diagnosis. The architecture
of the proposed models and the used dataset in this research are explained in
detail in section 3. Section 4 describes the features used to train our models
and other preprocessing procedures. Section 5 explains the experiments that
are conducted to evaluate the performance of the used models. This is followed
by a description of the results and their interpretation in section 6.
Finally, section 7 summarises the work and identifies future directions.

##  2 Background and Related Work

Voice samples provide a plethora of health-related information [17]. As a
result, scientists believe that minor voice signals might reveal underlying
medical issues or disease risk [17]. Voice analysis technologies have the
potential to be a reliable, efficient, affordable, convenient, and simple-to-
use technique for predicting, diagnosing, and treating health problems.
Various approaches are used to extract certain voice and acoustic features
(referred to as vocal biomarkers) from audio voice samples. These features are
then analyzed for important patterns and cues to give insights about an
individual’s health.

Mild or severe changes in one’s voice can be a sign of a variety of diseases,
making vocal biomarkers a noninvasive tool to monitor patients, grade the
severity, the stages of a disease or for drug development [17]. Patients
suffering from Parkinson’s disease, for example, have a decrease of voice
volume and start talking more quickly [18]. Patients with Alzheimer’s disease
experience longer pauses between words, have difficulty finding their words,
and frequently substitute nouns with pronouns like ”that” and ”it” [19]. As a
result, modifications in voice, which are normally undetectable by human ears,
are now being investigated in order to identify early indicators of health
problems. For instance, in 2019, researchers successfully implemented a model
for diagnosing Parkinson’s disease [20]. Their model was able to distinguish
between Parkinson’s disease patients and those in the control group 85% of the
time [20]. Interestingly, the proposed model outperformed the clinical
diagnosis set by nonspecialist doctors (with accuracy of 74% at the time) and
diagnosis set by disorder experts (with accuracy on average 80%) [20].

Similarly, several other studies have proposed automatic COVID-19 screening by
analyzing coughs or respiratory signals, to detect the presence of alterations
due to the COVID-19 infection. A few other studies, instead, have indicated
solutions for the detection of COVID-19 disorders based on an analysis of
voice samples. To illustrate, in [21], the authors propose a technique for
identifying this illness based on analyzing a specific sentence (”I believe my
data may assist to manage the virus pandemic.”). This sentence was repeated
1-3 times and was reported along with the COVID-19 test results. Cambridge
University shares this dataset, which is crowdsourced from more than 10
different countries including samples from 366 subjects with 308 COVID-19
positive patients, and 893 recordings. The authors achieved an AUC of 72.1%
for distinguishing COVID-19 subjects from non-COVID-19 subjects using Support
Vector Machine (SVM) algorithm on COMPARE Acoustic Feature Set that contains
6373 static features resulting from the computation of functionals
(statistics) over low-level descriptor (LLD) contours.

Besides, researcers have investigated several methods to extract sound
features. Mel Frequency Cepstrum Coefficient (MFCC) is a technique for
extracting audio features that is extensively utilized in different audio
recognition applications such as speech emotion identification and abnormal
voice recognition. In fact the most important point about speech is that the
sounds produced by a human are filtered by the shape of the vocal tract, which
includes the tongue, teeth, and so on. This shape controls how the sound is
produced. The shape of the vocal tract reveals itself in the envelope of the
short time power spectrum, and the goal of MFCCs is to appropriately capture
this envelope. Moreover, a recent study showed the remarkable proficiency of
MFCCs in detecting COVID-19 [14]. The researchers have extracted MFCC features
from coughing, breathing, and speaking sounds using a dataset of 1159 sound
samples obtained from 592 participants with 213 COVID-19 infected patients
[14]. These were then processed using the Long Short-Term Memory (LSTM)
architecture, which achieved 98.9% accuracy, precision, F1-score, and recall
[14].

With the development of deep learning and machine learning, neural networks
have played an important role in audio recognition. For instance, LSTM [22],
SVM [23], CNN [24], ANN [25], KNN [26], and other methods are widely used for
speech recognition. Compared with traditional methods, deep and machine
learning models can extract and learn more complex and robust features and
make intelligent decisions on its own.

In conclusion, there are relatively few studies in the literature related to
the identification of COVID-19 using an analysis of speech sounds, most likely
due to the virus’s recent spread and the pandemic’s ongoing development. The
majority of these efforts have been conducted on relatively small and, in many
cases, inaccessible datasets, limiting their accessibility to the larger
research community and limiting the future development of classification
algorithms on standardized datasets.

##  3 Methods

In this section we will describe the data used for this analysis, the steps
followed to clean and the pre-processing performed prior to the analysis.
Figure 1 illustrates the model development steps into a pipeline.

Figure 1: The used pipeline for both basic Machine learning classifiers and
Deep Learning classifiers for COVID-19 binary classification (e.g., COVID-19
vs. non-COVID-19).

###  3.1 Dataset Description

In this work we used the University of Cambridge dataset [11] which was crowd-
sourced and collected from a web-based platform, an android application, and
an iOS application. As reported by [11], participants are asked to report
their demographics, medical history, and smoking status. In addition, they are
required to report their COVID-19 test results, hospitalisation status, and
symptoms (if any). To record respiratory sounds, participants are prompted to
i) cough three times; ii) breathe deeply through their mouth three to five
times; iii) read a short sentence on-screen and repeat it three times. After a
year of data collection, 4352 users participants worldwide have contributed
their data, and 893 audio samples (308 COVID-19 positive samples) are
released. In Figure 2, user characteristics are shown. The average age of our
participants is 46 where the max and min age are 90 and 9.5 years old,
respectively. The dominant gender is male with 503 males 385 females
participants. Also, most of the participants (593 out of 893) are non-smokers.
However, the majority of the participants don’t have a medical condition but
high blood pressure is so common among other participants. Finally the
symptoms they suffered from include dry cough, smell and taste loss, sore
throat, wet cough, short breath, muscle ache, and headache.

The training data set was divided into training and testing sets. 70% of cases
were selected randomly and were considered as the training set, and the rest
were designated as the testing set. Thus, the training set size was 319, and
the testing set size was 138.

Our goal is to determine whether a patient has COVID-19 or not based on the
voice data. Therefore, we developed several classification models to detect
positive COVID-19 cases. These models included Long-Short Term Memory (LSTM)
and Convolutional Neural Network (CNN). We compared their predictive power to
baseline classification models, namely Logistic Regression and Support Vector
Machine. The following section explains the architecture of the suggested
models.

(a) Age distribution of the participants

(b) Gender distribution of the participants

(c) Smoking status of the participants

(d) The number of participants who admitted to the hospital

(e) A COVID-19 test result

Figure 2: Users characteristics (a) age, (b) gender, (c) smoking status, (d) hospitalized or not (e) their COVID-19 test result  Attribute | Variable | Value | Count | Percentage
---|---|---|---|---
Demographics | Gender | Female | 385 | 43%
Male | 503 | 56%
| Other | 5 | 0.6%
| Age | Max age | 90 |
| Min age | 9.5 |
| Avg age | 46 |
Symptoms |  | None | 375 | 42%
| Dry cough | 118 | 13%
| Smell and taste loss, Sore throat | 47 | 5.3%
| Wet cough | 37 | 4.1%
| Short breath | 25 | 2.8%
| Sore throat | 22 | 2.5%
| Muscleache | 15 | 1.7%
| Headache | 14 | 1.6%
| Dry cough, Tightness | 12 | 1.3%
Prior Medical History |  | None | 630 | 71%
| High blood pressure | 112 | 13%
| Pulmonary | 50 | 6%
| Other | 35 | 4%
| Diabetes | 23 | 2.6%
| Cardiovascular | 4 | 0.4%
Smoking Status |  | Non-smoker | 586 | 71%
| Ex-smoker | 191 | 13%
| 1-10 cigarettes per day | 63 | 6%
| 11-20 cigarettes per day | 28 | 4%
| Prefer not to say | 13 | 2.6%
| Smoked Once | 7 | 0.4%
| 21+ cigarettes per day | 5 | 0.4%
Hospitalized |  | Yes | 880 | 98.5%
| No | 12 | 1.5%
COVID-19 test result |  | negative | 585 | 65.5%
| positive | 308 | 34.5%
TABLE I: Patient characteristics in the dataset

###  3.2 Basic Machine Learning Classifiers

We started with simple models as a baseline and progressively dived into more
complex ones. This allows us to solve problems efficiently, by reasoning at
the simplest useful level. In addition, this enables us to explore in which
direction we should refine the more complex model to address the hard parts
better. We have evaluated Logistic Regression (LR) and Support Vector Machine
(SVM) models. These models were trained on the patient’s metadata, medical
history, smoking status, hospitalisation status, and symptoms.

###  3.3 Convolutional Neural Network (CNN)

With the development of deep learning, more and more deep learning methods are
applied to various scenarios, such as image recognition, image classification,
speech recognition, machine translation, etc. CNNs contain three main layers:
convolutional layer, pooling layer, and fully connected layer. Due to the
outstanding performance of neural networks, CNN has solved several complex
challenges of computer vision. Therefore, a pre-trained ResNet50 convolutional
neural network was used to process Mel-spectrogram images that were extracted
from the audio recordings.

The Mel-spectrogram is an effective tool to extract hidden features from audio
and visualize them as an image. A CNN model can effectively extract features
from images, and then complete tasks such as classification and recognition.
Therefore, we use the CNN model to effectively classify the audio and to
realize the accurate recognition and detection of voice.

The CNN was trained by an Adam optimizer, whose learning rate is 0.0001. The
max epoch and batch size were 50 and 32, respectively.

###  3.4 Long-Short Term Memory (LSTM)

Long Short-Term Memory (LSTM) [27] is an advanced variant of recurrent neural
network(RNN) that is suited to handle sequential data and describe temporal
dependencies in the data. [28] have shown that LSTM based RNN architectures
makes more effective use of model parameters to train acoustic models for
large vocabulary speech recognition. Recent works have successfully employed
LSTM models for disease detection (i.e. [29, 30] The intuition behind this
choice is that LSTM allows the neural network to retain (and gradually forget)
information about previous time instants taking advantage of the strong
temporal dependency that exists between consecutive frames in the speech
signals. Our LSTM model was implemented in Python using the Keras library.

We extracted Mel-frequency cepstral coefficients (MFCC) from the recorded
audio signals. These features were used to train the LSTM model. MFCC is a
fundamental feature that is widely utilized in many domains (i.e. speech
recognition or emotion analysis) Its success stands in the ability to capture
in a compact form and on capturing in the model the ways human hearing
perceive sound [31] (i.e. humans cannot perceive frequencies over 1000 Hz).
Specifically, MFCC is based on the known variations of the human ear’s
critical bandwidth frequency.

Consequently, the input consists of the patient’s voice. Then, the MFCC
features are extracted, and the output is passed through the LSTM model for
classification.

##  4 Experimental Setting

###  4.1 Metadata Preprocessing

Preprocessing is performed to account for the potential missing, incomplete or
noisy data in the dataset. After data investigation, we did not have any
missing values in our dataset but some columns have ”none” values or ”prefer
not to say” entries. We started by categorizing the medical history of the
patients in a more general categories as shown in Table II. After that, we
used one hot encoding to indicate whether a patient has a certain medical
condition by using 1 if the patient has a certain medical condition and 0
otherwise. For example: a patient with high blood pressure, diabetes and
cardiovascular disease. Therefore we label 1 the corresponding columns and 0
otherwise. The same technique was used for the patient’s gender. We created 3
columns for females, males, and unprovided gender. The smoking status includes
non-smoker, prefer not to say, ex-smoker, smoked once, 1-10 cigarettes per
day, 11-20 cigarettes per day, and 21+ cigarettes per day. Some smoking
categories could be removed without loss of information, such as ”ItOnce” can
be included into ”non-smoker” (as it means that the patient has tried smoking
but it is not a smoker). Also, patients who smoke ”21+ cigarettes per day” can
be merged with the ”11-20 cigarettes per day”, resulting in a group of
patients who smoke ”11 or more cigarettes per day”. One-hot encoding was also
applied to the smoking status. As for patient’s age, patients who preferred to
not say their age and tested negative were removed from the dataset. Then we
created age ranges instead of an exact age of a patient. An average of the age
range was used, for instance, if the patient is between 40-49 years old, its
age will be the average of 40-49 which is 44.5 years old. The age values were
then normalized to have similar values as the values in other columns.

The label ”COVID-19 test result”, encodes the patients who tested tested by 1
and patients who tested negative by 0. Similarly, for ”Hospitalized” which is
an indicator whether the patient is in hospital, we encode ”yes” as 1 and ”no”
as 0.

Patient’s symptoms, were also processed similarly to patient’s medical
history. Since most patients have no symptoms but some of them have multiple
symptoms, we created a column for each symptom. If a patient has multiple
symptoms, it will have the value 1 for the mentioned symptom. For example, if
a patient suffers from dry cough and short breath, it will have 1 in the dry
cough and short breath columns and 0 in the remaining columns of symptoms

Disease |  Category
---|---
Hpb |  high blood pressure
Asthma |  pulmonary
Pnts |  other
Longterm |  other
Lung |  pulmonary
Heart |  cardiovascular
Valvular |  cardiovascular
Cancer |  Cancer
Diabetes |  Diabetes
Copd |  pulmonary
Hiv |  other
otherHeart |  cardiovascular
Cystic |  pulmonary
Angina |  other
TABLE II: Medical History Categorization

###  4.2 Feature extraction

We extracted vocal features from the audio files that were then used into our
models. In the proposed work, the extracted features for the speech signal
optimum representation are the Mel-frequency cepstral coefficients (MFCC).
MFCC is a fundamental feature that is utilized in speaker and emotion
recognition by virtue of the advanced representation of human auditory
perception it provides.

MEL spectrum images were also extracted. The Mel spectrum contains a short-
time Fourier transform (STFT) for each frame of the spectrum (energy/amplitude
spectrum), from the linear frequency scale to the logarithmic Mel-scale, the
filter bank will then determine the eigenvector. Eigenvalues can be roughly
expressed as the distribution of signal energy on the Mel-scale frequency. We
transformed the audio recordings into Mel-spectrograms so that we can train
the convolutional neural networks to distinguish between positive and negative
patients.

Finally, other additional feature were extracted such as the mean, standard
deviation, maximum value, minimum value, median, kurtosis, 1st quartile, 3rd
quartile, interquartile, skewness and mode of the audio signals.

###  4.3 Imbalanced dataset

Our dataset consists of 893 voice recordings with 308 positively tested
patients. First, we split our dataset into training, testing, and validation
sets before balancing the data. That way, we ensure that the test dataset is
as unbiased as it can be and reflects a true evaluation for our model. The
training set has 243 negative cases and 72 positive cases. We can see that the
training set is highly imbalanced, so we need to take that into consideration.
Also, the testing set has 189 negative cases and 94 positive cases. Finally,
the validation set has 152 negative cases and 142 positive cases. Given that
the validation set big, we re-assigned some of positively tested patients from
the validation set to the training set to increase the size of the positively
test patients in training data set and thus solve the imbalance dataset
problem. Consequently, the size of the new training set is with 214 positive
and 243 negative cases, respectively. This new training set will be used as an
input for all the models that are described in this paper.

##  5 Experiments

###  5.1 Experiments with neural network models

Neural networks have been frequently utilized to analyse data for the
diagnosis of various disease. Some of the most famous classifiers in pattern
recognition and classification tasks include CNN, LSTM and Generative
adversarial network (GAN). To investigate the effectiveness of such networks
in detecting COVID-19, we employed CNN and LSTM to identify the best model for
COVID-19 classification. These models have been trained on different data
inputs. The CNN was trained on MEL-spectogram images extracted from the voice
recordings. Whereas, the LSTM model was fed MFCC features.

###  5.2 Experiments with basic Machine Learning models

To test whether basic Machine Learning models improve COVID-19 prediction
accuracy, two models were developed. In this paper, we examine the performance
of Logistic Regression and Support Vector Machine. Thus, the proposed models
were trained on the pre-processed patient’s data.

##  6 Results

For each experiment, several performance measures obtained during model
evaluation are reported for every model. An overview of the performances and
parameters of the used models are summarized in Table III

Model |  Parameters |  Accuracy |  Sensitivity |  Specificity
---|---|---|---|---
LR |  Input = patient’s medical history, gender, smoking status, age, symptoms, hospitalized |  75% |  70% |  80%
SVM |  Input = patient’s medical history, gender, smoking status, age, symptoms, hospitalized, kernel= rbf, C=1,gamma= auto |  75% |  71% |  79%
CNN |  Type = ResNet50, input = Mel-spectrogram images ,input shape=(150, 150,3), Trainable parameters $\approx$ 23M, non-trainable parameters $\approx$ 53K , loss= binary crossentropy, optimizer= adam, activation = softmax |  80% |  79% |  80%,
LSTM |  Input= MFCC features, total parameters $\approx$ 849K ,loss= binary crossentropy, optimizer= adam, activation = softmax |  89% |  89% |  83%
TABLE III:  A summary of the performances and parameters of the used models

####  6.0.1 Performance Measurements

In order to better evaluate the performance of the model, we list several
indicators used to evaluate the model.

Accuracy: Measures the ratio between the number of correctly predicted data
samples out of all the data samples. Sensitivity: Sensitivity measures the
ratio of correctly predicted positive observations to the all observations in
the positive class. The question it answers is: Of all the patients that truly
were positive, how many did the model label as positive?

Specificity: is the model ability to correctly identify non-infected
participants. The question answered by this metric is: Out of all the people
that do not have the disease, how many received negative results?

Confusion matrix: Defines a matrix summarizing the classification outcome in
relation to the ground truth. Thus, the rows of each of the confusion matrices
correspond to the ground-truth labels, and the columns illustrate the
predicted labels.

ROC (receiver operating characteristic) curve: is a graph showing the
performance of a classification model at all classification thresholds. The
plot shows the diagnostic ability of the binary classification models as a
trade-off between sensitivity and specificity. Models with ROC curves closer
to the top-left corner indicate a better performance.

###  6.1 Experimental Results of the deep learning models

This section highlights the importance of deep learning models in the
implementation of sophisticated diagnostic systems. Two proposed models were
assessed using two types of neural networks: CNN, and LSTM. The performance of
each model is illustrated in the next sections.

####  6.1.1 CNN model

Experiments were carried out to investigate the performance of a CNN model on
Mel-spectogram images. As demonstrated in Table IV, this model performs well
since 50 of the 63 positively tested patients (about 79 percent) are
identified accurately as positive. Similarly, 60 of 75 patients (80% of the
patients) who tested negative are expected to be COVID-19 free.

|  Negative |  Positive
---|---|---
Negative |  60 |  15
Positive |  13 |  50
TABLE IV: Confusion Matrix for CNN Model

Figure 3 shows the ROC curve of the CNN classifier. The curve shows good
performance suggesting that the CNN model is quite accurate in detecting
features from MEL-spectorgram images, that assisted in the diagnosis of
patient voice recordings.

Figure 3: ROC curve for CNN Model

Finally, the sensitivity and specificity of our model was 79% and 80%,
respectively. Sensitivity of 79% is a measure of how well the model could
correctly identify patients with COVID-19 disease. Also, the high specificity
indicates how well the model can correctly identify people without the
disease.

####  6.1.2 LSTM model

To test the performance of the LSTM model, we ran an experiment with the MFCC
features extracted from the audio recordings. According to Table V, it is
apparent that our model correctly diagnosed about 88% percent of our positive
cases. Similarly, around 89% of non-infected individuals were predicted to be
negative.

|  Negative |  Positive
---|---|---
Negative |  67 |  8
Positive |  7 |  56
TABLE V: Confusion Matrix for LSTM Model

Figure 4 shows the ROC curve for the LSTM classifier. The curve shows good
performance. The ROC curve also reveals whether or not our model can
discriminate between classes. Consequently, the high AUC we obtained suggests
that LSTM is very good at predicting zero classes as zero and one classes as
one. In other words, the higher the AUC, the better the model distinguishes
between patients with and without the illness.

Figure 4: ROC curve for LSTM Model

Lastly, we obtained with the LSTM, sensitivity and specificity of 89%. Such
high sensitivity means that there are few false negative results, and thus
fewer cases of disease are missed. In addition, the high specificity of the
LSTM means that the model is able to accurately designate an individual who
does not have a disease as negative.

###  6.2 Experimental Results of basic Machine Learning models

We trained two basic classifiers to predict COVID-19 test result based on
demographics, smoking status, symptoms, and some clinical information.

####  6.2.1 Logistic regression

According to Table VI, the LR model correctly recognised 79% of the positive
cases. In addition, the model accurately classified 70% of our negative
samples.

|  Negative |  Positive
---|---|---
Negative |  60 |  15
Positive |  19 |  44
TABLE VI: Confusion Matrix for Logistic Regression Model

As depicted by Figure 5, the LR model produced a curve that is relatively
close to the top-left corner, indicating an acceptable performance.

Figure 5: ROC curve for Logistic Regression Model

Finally, the sensitivity and specificity of our model was 70% and 80%,
respectively.

####  6.2.2 SVM model

Table VII shows 45 of 63 individuals who were classified as having a positive
test result, they actually have the disease. Furthermore, 59 of the 75 healthy
individuals were correctly diagnosed as being COVID-19-free.

|  Negative |  Positive
---|---|---
Negative |  59 |  16
Positive |  18 |  45
TABLE VII: Confusion Matrix for SVM Model

The ROC curve obtained with SVM is similar to the LR model’s curve. The
resulting curve, as illustrated in Figure 6, demonstrates satisfactory
performance.

Figure 6: ROC curve for SVM Model

SVM obtained a sensitivity of 70% and a specificity of 80%.

We generated moderately to significantly accurate predictive models for
COVID-19 detection based on voice features, demographics, symptoms, medical
history, and other variables. The LSTM classifier is the algorithm with the
highest performance. CNN turned out to be the second best. Given the accuracy
of CNN, the model appears to distinguish voice features from the images of the
MEL-spectogram images provided in input. LR and SVM classifiers also have a
high accuracy. As demonstrated in Table III, the detection sensitivity of the
LSTM is higher than the other models, achieving 89%. As shown in Table III,
detection specificity rose with LSTM, reaching 89%.

##  7 Conclusion

Non-intrusive and easy to collect voice parameters may provide valuable
information COVID-19 diagnosis. Using voice parameters extracted from audio
recordings we defined a predictive model that can classify COVID-19 patients
as positive and negative. This exploratory work suggest hat the best
performance is obtained with an LSTM model trained on voice features extracted
as mel-frequency-cepstral-coefficients (MFCCs). In future works we plan to
further explore which features extracted from the voice signals are the most
useful to distinguish COVID-19 patients. Furthermore we plan to analyse the
performance of the models with a larger dataset. We will seek to externally
validate the models with other voice COVID-19 datasets collected in other
circumstances. More research needed to evaluate if the created models are able
to distinguish between COVID-19 and another respiratory conditions.

## References

  * [1] Worldometer, “Covid-19 coronavirus outbreak,” 2022.
  * [2] C.-C. Lai, T.-P. Shih, W.-C. Ko, H.-J. Tang, and P.-R. Hsueh, “Severe acute respiratory syndrome coronavirus 2 (sars-cov-2) and coronavirus disease-2019 (covid-19): The epidemic and the challenges,” _International journal of antimicrobial agents_ , vol. 55, no. 3, p. 105924, 2020.
  * [3] N. C. for Immunization _et al._ , “Science brief: Sars-cov-2 and surface (fomite) transmission for indoor community environments,” in _CDC COVID-19 Science Briefs [Internet]_. Centers for Disease Control and Prevention (US), 2021.
  * [4] R. Ningthoujam, “Covid 19 can spread through breathing, talking, study estimates,” _Current medicine research and practice_ , vol. 10, no. 3, p. 132, 2020.
  * [5] J. Han, T. Xia, D. Spathis, E. Bondareva, C. Brown, J. Chauhan, T. Dang, A. Grammenos, A. Hasthanasombat, A. Floto _et al._ , “Sounds of covid-19: exploring realistic performance of audio-based digital testing,” _NPJ digital medicine_ , vol. 5, no. 1, pp. 1–9, 2022.
  * [6] B. Stasak, Z. Huang, S. Razavi, D. Joachim, and J. Epps, “Automatic detection of covid-19 based on short-duration acoustic smartphone speech analysis,” _Journal of Healthcare Informatics Research_ , vol. 5, no. 2, pp. 201–217, 2021.
  * [7] S. de la Fuente Garcia, F. Haider, and S. Luz, “Covid-19: Affect recognition through voice analysis during the winter lockdown in scotland,” in _2021 43rd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC)_. IEEE, 2021, pp. 2326–2329.
  * [8] A. Hassan, I. Shahin, and M. B. Alsabek, “Covid-19 detection system using recurrent neural networks,” in _2020 International conference on communications, computing, cybersecurity, and informatics (CCCI)_. IEEE, 2020, pp. 1–5.
  * [9] M. A. Mehrabadi, S. A. H. Aqajari, I. Azimi, C. A. Downs, N. Dutt, and A. M. Rahmani, “Detection of covid-19 using heart rate and blood pressure: Lessons learned from patients with ards,” in _2021 43rd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC)_. IEEE, 2021, pp. 2140–2143.
  * [10] J.-S. Liang and K. Wang, “Vibration feature extraction using audio spectrum analyzer based machine learning,” in _2017 International conference on information, Communication and Engineering (ICICE)_. IEEE, 2017, pp. 381–384.
  * [11] C. Brown, J. Chauhan, A. Grammenos, J. Han, A. Hasthanasombat, D. Spathis, T. Xia, P. Cicuta, and C. Mascolo, “Exploring automatic diagnosis of covid-19 from crowdsourced respiratory sound data,” _arXiv preprint arXiv:2006.05919_ , 2020.
  * [12] A. Vahedian-Azimi, A. Keramatfar, M. Asiaee, S. S. Atashi, and M. Nourbakhsh, “Do you have covid-19? an artificial intelligence-based screening tool for covid-19 using acoustic parameters,” _The Journal of the Acoustical Society of America_ , vol. 150, no. 3, pp. 1945–1953, 2021.
  * [13] V. Despotovic, M. Ismael, M. Cornil, R. Mc Call, and G. Fagherazzi, “Detection of covid-19 from voice, cough and breathing patterns: Dataset and preliminary results,” _Computers in Biology and Medicine_ , vol. 138, p. 104944, 2021\.
  * [14] A. B. Nassif, I. Shahin, M. Bader, A. Hassan, and N. Werghi, “Covid-19 detection systems using deep-learning algorithms based on speech and image data,” _Mathematics_ , vol. 10, no. 4, p. 564, 2022.
  * [15] Y. Chang, X. Jing, Z. Ren, and B. W. Schuller, “Covnet: A transfer learning framework for automatic covid-19 detection from crowd-sourced cough sounds,” _Frontiers in Digital Health_ , vol. 3, 2021.
  * [16] M. Aly, K. H. Rahouma, and S. M. Ramzy, “Pay attention to the speech: Covid-19 diagnosis using machine learning and crowdsourced respiratory and speech recordings,” _Alexandria Engineering Journal_ , vol. 61, no. 5, pp. 3487–3500, 2022.
  * [17] G. Fagherazzi, A. Fischer, M. Ismael, and V. Despotovic, “Voice for health: The use of vocal biomarkers from research to clinical practice,” _Digital biomarkers_ , vol. 5, no. 1, pp. 78–88, 2021.
  * [18] G. Solana-Lavalle and R. Rosas-Romero, “Analysis of voice as an assisting tool for detection of parkinson’s disease and its subsequent clinical interpretation,” _Biomedical Signal Processing and Control_ , vol. 66, p. 102415, 2021.
  * [19] A. Almor, D. Kempler, M. C. MacDonald, E. S. Andersen, and L. K. Tyler, “Why do alzheimer patients have difficulty with pronouns? working memory, semantics, and reference in comprehension and production in alzheimer’s disease,” _Brain and language_ , vol. 67, no. 3, pp. 202–227, 1999.
  * [20] T. J. Wroge, Y. Özkanca, C. Demiroglu, D. Si, D. C. Atkins, and R. H. Ghomi, “Parkinson’s disease diagnosis using machine learning and voice,” in _2018 IEEE Signal Processing in Medicine and Biology Symposium (SPMB)_. IEEE, 2018, pp. 1–7.
  * [21] B. W. Schuller, A. Batliner, C. Bergler, C. Mascolo, J. Han, I. Lefter, H. Kaya, S. Amiriparian, A. Baird, L. Stappen _et al._ , “The interspeech 2021 computational paralinguistics challenge: Covid-19 cough, covid-19 speech, escalation & primates,” _arXiv preprint arXiv:2102.13468_ , 2021.
  * [22] S. Hochreiter and J. Schmidhuber, “Long short-term memory,” _Neural computation_ , vol. 9, no. 8, pp. 1735–1780, 1997.
  * [23] C. Cortes and V. Vapnik, “Support-vector networks,” _Machine learning_ , vol. 20, no. 3, pp. 273–297, 1995.
  * [24] K. O’Shea and R. Nash, “An introduction to convolutional neural networks,” _arXiv preprint arXiv:1511.08458_ , 2015.
  * [25] W. S. McCulloch and W. Pitts, “A logical calculus of the ideas immanent in nervous activity,” _The bulletin of mathematical biophysics_ , vol. 5, no. 4, pp. 115–133, 1943.
  * [26] N. S. Altman, “An introduction to kernel and nearest-neighbor nonparametric regression,” _The American Statistician_ , vol. 46, no. 3, pp. 175–185, 1992\.
  * [27] F. A. Gers, J. Schmidhuber, and F. Cummins, “Learning to forget: Continual prediction with lstm,” _Neural computation_ , vol. 12, no. 10, pp. 2451–2471, 2000.
  * [28] H. Sak, A. Senior, and F. Beaufays, “Long short-term memory based recurrent neural network architectures for large vocabulary speech recognition,” _arXiv preprint arXiv:1402.1128_ , 2014.
  * [29] D. R. Rizvi, I. Nissar, S. Masood, M. Ahmed, and F. Ahmad, “An lstm based deep learning model for voice-based detection of parkinson’s disease,” _Int. J. Adv. Sci. Technol_ , vol. 29, no. 8, 2020.
  * [30] C. Xue, C. Karjadi, I. C. Paschalidis, R. Au, and V. B. Kolachalama, “Detection of dementia on voice recordings using deep learning: a framingham heart study,” _Alzheimer’s research & therapy_, vol. 13, no. 1, pp. 1–15, 2021.
  * [31] B. Logan, “Mel frequency cepstral coefficients for music modeling,” in _In International Symposium on Music Information Retrieval_. Citeseer, 2000.