# HAZMAT. VIII. A Spectroscopic Analysis of the Ultraviolet Evolution of K
Stars: Additional Evidence for K Dwarf Rotational Stalling in the First
Gigayear

Tyler Richey-Yowell School of Earth and Space Exploration, Arizona State
University, Tempe, AZ 85287, USA<EMAIL_ADDRESS>Evgenya L. Shkolnik School of
Earth and Space Exploration, Arizona State University, Tempe, AZ 85287, USA
R. O. Parke Loyd School of Earth and Space Exploration, Arizona State
University, Tempe, AZ 85287, USA  Eureka Scientific, 2452 Delmer Street Suite
100, Oakland, CA, 94602-3017, USA  James A. G. Jackman School of Earth and
Space Exploration, Arizona State University, Tempe, AZ 85287, USA  Adam C.
Schneider U.S. Naval Observatory, 10391 W Naval Observatory Rd, Flagstaff, AZ
86001, USA  Department of Physics and Astronomy, George Mason University,
MS3F3, 4400 University Drive, Fairfax, VA 22030, USA  Marcel A. Agüeros
Department of Astronomy, Columbia University, 550 West 120th Street, New York,
NY 10027, USA  Travis Barman Lunar and Planetary Laboratory, University of
Arizona, Tucson, AZ 85721, USA  Victoria S. Meadows NASA Astrobiology
Institute, Virtual Planetary Laboratory, University of Washington, Seattle, WA
85215, USA  Rose Gibson  Department of Astronomy, Columbia University, 550
West 120th Street, New York, NY 10027, USA  Stephanie T. Douglas Department of
Physics, Lafayette College, 730 High St, Easton, PA 18042, USA

(Accepted March 11, 2022)

###### Abstract

Efforts to discover and characterize habitable zone planets have primarily
focused on Sun-like stars and M dwarfs. K stars, however, provide an appealing
compromise between these two alternatives that has been relatively unexplored.
Understanding the ultraviolet (UV) environment around such stars is critical
to our understanding of their planets, as the UV can drastically alter the
photochemistry of a planet’s atmosphere. Here we present near-UV and far-UV
Hubble Space Telescope’s Cosmic Origins Spectrograph observations of 39 K
stars at three distinct ages: 40 Myr, 650 Myr, and $\approx$5 Gyr. We find
that the K star (0.6 – 0.8 M⊙) UV flux remains constant beyond 650 Myr before
falling off by an order of magnitude by field age. This is distinct from early
M stars (0.3 – 0.6 M⊙), which begin to decline after only a few hundred Myr.
However, the rotation-UV activity relation for K stars is nearly identical to
that of early M stars. These results may be a consequence of the spin-down
stalling effect recently reported for K dwarfs, in which the spin-down of K
stars halts for over a Gyr when their rotation periods reach $\approx$10 d,
rather than the continuous spin down that G stars experience. These results
imply that exoplanets orbiting K dwarfs may experience a stronger UV
environment than thought, weakening the case for K stars as hosts of potential
“super-habitable” planets.

stars: evolution, stars: low-mass

††journal: ApJ††software: Astropy (Price-Whelan et al., 2018), Matplotlib
(Hunter, 2007), Numpy (van der Walt et al., 2011), Scipy (Jones et al., 2001),
emcee (Foreman-Mackey et al., 2013), lightkurve (Lightkurve Collaboration et
al., 2018), stardate (Angus et al., 2019)

##  1 Introduction

While often overlooked in favor of M and G stars, K stars have been gaining
traction as potential “Goldilocks” hosts for habitable planets. Many factors
go into determining the potential habitability of a planet from the stellar
perspective, e.g., frequency and lifetime of the star, contraction time onto
the main sequence, the size and longevity of the habitable zone (HZ),
ultraviolet (UV) and X-ray emission, stellar variability, and the potential
for tidal locking (e.g., Meadows & Barnes, 2018). Compared to solar-type
stars, K stars are more abundant, maintain longer main-sequence lifetimes, and
their planets are more suitable to observations based on the mass and radius
ratios of the planet to star. While M stars also excel in these regards, they
have other potentially life-destroying disadvantages compared to K dwarfs. For
example, superflares (flares more energetic than the largest recorded solar
events, i.e., Carrington 1859; Tsurutani et al. 2003) appear to occur daily on
young M stars (Loyd et al., 2018). Yet the probability of finding a habitable
planet around solar-type stars decreases significantly due to the stars’
shorter lifetimes, the lower frequency of the stars, and more importantly, our
difficulty in detecting and characterizing their planets. HZ planets around
old solar-type stars are too faint to detect directly, have smaller RV
amplitudes, and transit only once per year.

The UV radiation incident on a planet (both through quiescent emission and
flaring) is a key factor in assessing the habitability the planet, as the UV
has the potential to alter the chemistry of planetary atmospheres by ionizing
and photo-dissociating both the most important molecules for the development
of life and signatures that would be indicative of life, with the potential
for complete erosion of the atmosphere (Airapetian et al., 2017). Therefore,
in order to gain an understanding of the atmospheric environments of planets
around any type of star today, it is essential to characterize the
distribution of UV properties and how these properties evolve with age.

###  1.1 The Effect of UV Radiation on Planetary Atmospheres

The near-UV (NUV, 1700-3200 Å), far-UV (FUV, 912-1700 Å), and extreme-UV (EUV,
100-912 Å) each affect planetary atmospheres differently. The NUV and FUV
radiation will photo-dissociate the molecules in the atmosphere, including
those most important for the development and detection of life (e.g. H2O, O2,
OCS, and NH3) (e.g. Lichtenegger et al., 2010; Segura et al., 2010; Hu et al.,
2012). The FUV to NUV flux ratio determines the lifetimes of biogenic species
such as CH4, N2O, and CH3Cl, allowing for a greater probability of detection
with a greater FUV to NUV ratio (Segura et al., 2005), but will also increase
abiotic oxygen and ozone, thus producing higher chances of false-positive
biosignatures (e.g. Domagal-Goldman et al., 2014; Tian et al., 2014; Harman et
al., 2015). Richey-Yowell et al. (2019) measured this ratio to be up to 10
times smaller for K stars than for M stars, which have been measured
photometrically to be between $\approx 0.1-0.3$ (Schneider & Shkolnik, 2018)
and spectroscopically to be between $\approx 0.5-3$ (France et al., 2013),
giving K stars an added advantage for the detection of real biosignatures. The
EUV values are crucial since the EUV will heat and ionize the atmosphere,
powering thermal atmospheric escape (Lammer et al., 2003, 2007; Koskinen et
al., 2010). However, the EUV flux from stars other than the Sun is
predominantly absorbed by the interstellar medium. Limited data exist in the
EUV and must therefore be estimated from empirical relationships, differential
emission measure models, or stellar atmospheric models constrained by data at
FUV, NUV, or X-ray wavelengths (e.g., Pagano, 2009; Linsky & Güdel, 2015;
Fontenla et al., 2016; Peacock et al., 2019, 2020; Tilipman et al., 2021).

The photo-dissociation of the molecules in planetary atmospheres can also lead
to increased production of hazes in reducing atmospheres of sub-Neptunes and
Archean Earth (Earth 4.0 – 2.5 Myr ago) analogs, thus drastically altering the
spectrum and the detectability of biosignatures (Zerkle et al., 2012; Arney et
al., 2017). For future observations such as with the James Webb Space
Telescope  (JWST) that will focus on planetary spectra, hazes could limit the
detectability of spectral features in the atmosphere (e.g., Fauchez et al.,
2019). While no HZ planets around K dwarfs are expected to be seen by TESS due
to orbital period constraints (Barclay et al., 2018), $\sim$30 HZ K dwarf
terrestrial planets are expected to be discovered in the next decade with
ESA’s PLATO mission (PLATO Definition Study Report, 2017).

###  1.2 The K Dwarf Advantage

Several studies have recently focused on K stars as optimal hosts for
habitable planets. Both Cuntz & Guinan (2016) and Heller & Armstrong (2014)
found that early-type K stars may offer the most suitable conditions for life
and may even be “super-habitable,” i.e., even more habitable than an Earth-
sized planet around a G2V star. From a photochemistry perspective, studies
have demonstrated that K6V stars ($\approx$0.7 M⊙) may be the most suitable
laboratories for producing detectable CH4 and O2 (Arney et al., 2017; Arney,
2019), while the increased O2 of planets around these stars would permit
complex life to develop much faster (Lingam & Loeb, 2017).

###  1.3 K Star Activity Evolution

Using the Galaxy Evolution Explorer (GALEX) broadband UV photometry of K stars
ranging in age from 10 Myr to $\approx$5 Gyr, Richey-Yowell et al. (2019)
compared the photometric UV evolution of K stars to that of early- and late-
type M stars (Shkolnik & Barman, 2014; Schneider & Shkolnik, 2018). They
reported that the intrinsic NUV radiation of K stars only decreased slightly
(half an order of magnitude) between 10 Myr – $\approx$5 Gyr (the average age
for the field stars), and the FUV radiation decreased similarly from 10 – 650
Myr before falling off more steeply (an order of magnitude) by 5 Gyr. This
trend is different for late M stars, which are active longer because of their
slower spin-down rate, and early M stars, which remain constant through 150
Myr before declining sharply (1.5 orders of magnitude) in both the NUV and
FUV. However, GALEX observations only give us a narrow photometric window into
the overall evolution of K star UV flux, as some of the strongest and thus
most important emission features of these stars are outside of the GALEX
bandpasses (e.g. C III, Si III, N V, C II, and Mg II).

Table 1: Target information broken down by age.

Name | 2MASS Name | RA | Dec. | SpT | Mass | DistanceaaDistance measurements from Gaia EDR3 (Brown et al., 2021). | Derived AgebbAges either from the literature for moving group members (§2.1) or derived from the stardate program by Angus et al. (2019) for field stars. See §3.3 for discussion.  | Rotation PeriodccErrors reported where specified by the original authors, otherwise assumed to be 10% of the rotation period. | Ref. | Rossby
---|---|---|---|---|---|---|---|---|---|---
|  |  |  |  | [M⊙] | [pc] | [Myr] | [days] |  | Number
Tucana-Horologium Members (40 Myr)
HIP 1910 | J00240899-6211042 | 00 24 08.98 | -62 11 04.29 | K7.2 | 0.69 | 44.23 $\pm$ 1.07 | 40 $\pm$ 5 | 1.751 $\pm$ 0.003 | 1 | 0.044
CT Tuc | J00251465-6130483 | 00 25 14.66 | -61 30 48.25 | K5.7 | 0.72 | 44.16 $\pm$ 0.06 | 40 $\pm$ 5 | 4.190 $\pm$ 0.419 | 1 | 0.121
CD-46 644 | J02105538-4603588 | 02 10 55.39 | -46 03 58.65 | K4.2 | 0.80 | 81.78 $\pm$ 0.61 | 40 $\pm$ 5 | 1.116 $\pm$ 0.001 | 1 | 0.060
CD-35 1167 | J03190864-3507002 | 03 19 08.66 | -35 07 00.29 | K5.7 | 0.77 | 45.56 $\pm$ 0.07 | 40 $\pm$ 5 | 8.5 $\pm$ 0.1 | 1 | 0.275
CD-44 1173 | J03315564-4359135 | 03 31 55.64 | -43 59 13.54 | M0 | 0.60 | 45.28 $\pm$ 0.07 | 40 $\pm$ 5 | 2.94 $\pm$ 0.01 | 1 | 0.092
TYC 8083-45-5 | J04480066-5041255 | 04 48 00.66 | -50 41 25.66 | K5.7 | 0.72 | 59.60 $\pm$ 0.27 | 40 $\pm$ 5 | 8.44 $\pm$ 0.05 | 1 | 0.224
TYC 8098-41 | J05341467-5145545 | 05 33 25.58 | -51 17 13.10 | K4.9 | 0.77 | 53.95 $\pm$ 2.67 | 40 $\pm$ 5 | 5.22 $\pm$ 0.52 | 2 | 0.154
HIP 107345 | J21443012-6058389 | 21 44 30.12 | -60 58 38.87 | K7.2 | 0.69 | 46.36 $\pm$ 0.05 | 40 $\pm$ 5 | 4.54 $\pm$ 0.02 | 1 | 0.120
2MASS J23261069-7323498 | J23261069-7323498 | 23 26 10.70 | -73 23 49.89 | K7.7 | 0.66 | 46.29 $\pm$ 0.06 | 40 $\pm$ 5 | 0.57 $\pm$ 0.02 | 2 | 0.014
UCAC3 13-31217 | J23585674-8339423 | 23 58 56.74 | -83 39 42.3 | K5.8 | 0.72 | 55.63 $\pm$ 0.06 | 40 $\pm$ 5 | … | … | …
Hyades Members (650 Myr)
2MASS J03510309+2354134 | J03510309+2354134 | 03 51 03.11 | +23 54 13.14 | K6 | 0.79 | 40.69 $\pm$ 0.08 | 650 $\pm$ 70 | 12.57 $\pm$ 1.26 | 3 | 0.603
2MASS J03524101+2548159 | J03524101+2548159 | 03 52 41.00 | +25 48 15.97 | K7 | 0.62 | 45.28 $\pm$ 0.12 | 650 $\pm$ 70 | 14.66 $\pm$ 1.47 | 3 | 0.498
2MASS J03550142+1229081 | J03550142+1229081 | 03 55 01.44 | +12 29 08.11 | K0 | 0.63 | 45.92 $\pm$ 0.09 | 650 $\pm$ 70 | 11.66 $\pm$ 1.17 | 3 | 0.628
2MASS J04070122+1520062 | J04070122+1520062 | 04 07 01.22 | +15 20 06.10 | K4 | 0.75 | 45.09 $\pm$ 0.24 | 650 $\pm$ 70 | 14.03 $\pm$ 1.40 | 3 | 0.639
2MASS J04081110+1652229 | J04081110+1652229 | 04 08 11.09 | +16 52 23.11 | K7 | 0.62 | 40.10 $\pm$ 0.07 | 650 $\pm$ 70 | 13.63 $\pm$ 1.36 | 3 | 0.398
2MASS J04082667+1211304 | J04082667+1211304 | 04 08 26.66 | +12 11 30.64 | K5 | 0.60 | 46.33 $\pm$ 0.15 | 650 $\pm$ 70 | 12.96 $\pm$ 1.30 | 3 | 0.467
2MASS J04172512+1901478 | J04172512+1901478 | 04 17 25.15 | +19 01 47.67 | K5 | 0.71 | 47.86 $\pm$ 0.18 | 650 $\pm$ 70 | 12.84 $\pm$ 1.28 | 3 | 0.557
2MASS J04240740+2207079 | J04240740+2207079 | 04 24 07.42 | +22 07 07.92 | K5 | 0.71 | 45.78 $\pm$ 0.12 | 650 $\pm$ 70 | 13.01 $\pm$ 1.30 | 3 | 0.477
2MASS J04293897+2252579 | J04293897+2252579 | 04 29 38.99 | +22 52 57.79 | K5.5 | 0.68 | 59.35 $\pm$ 0.19 | 650 $\pm$ 70 | 14.95 $\pm$ 1.50 | 3 | 0.671
V925 Tau | J04303819+2254289 | 04 30 38.19 | +22 54 28.83 | M3 | 0.66 | 50.93 $\pm$ 0.26 | 650 $\pm$ 70 | 2.50 $\pm$ 0.25 | 3 | 0.040
2MASSJ04333716+2109030 | J04333716+2109030 | 04 33 37.18 | +21 09 03.06 | K7 | 0.62 | 43.80 $\pm$ 0.09 | 650 $\pm$ 70 | 13.59 $\pm$ 1.36 | 3 | 0.543
2MASS J04334192+1900504 | J04334192+1900504 | 04 33 41.92 | +19 00 50.52 | K7 | 0.62 | 47.80 $\pm$ 0.16 | 650 $\pm$ 70 | 12.64 $\pm$ 1.26 | 3 | 0.542
STKM 1-495ddIdentified in this work as a spectroscopic binary and was therefore not included in the analysis. See §3.1 for discussion. | J04350255+0839304 | 04 35 02.55 | +08 39 30.55 | M1 | 0.79 | 59.34 $\pm$ 2.64 | 650 $\pm$ 70 | 11.76 $\pm$ 1.18 | 4 | 0.284
UCAC4 518-008871ddIdentified in this work as a spectroscopic binary and was therefore not included in the analysis. See §3.1 for discussion. | J04451961+1334274 | 04 45 19.60 | +13 34 27.34 | K6 | 0.66 | 51.71 $\pm$ 0.71 | 650 $\pm$ 70 | 2.22 $\pm$ 0.2203 | 3 | 0.035
2MASS J04470892+2052564 | J04470892+2052564 | 04 47 08.93 | +20 52 56.32 | K4 | 0.75 | 42.30 $\pm$ 0.09 | 650 $\pm$ 70 | 10.54 $\pm$ 1.05 | 3 | 0.595
2MASS J04472618+2303032 | J04472618+2303032 | 04 47 26.18 | +23 03 03.32 | K4 | 0.75 | 55.09 $\pm$ 0.17 | 650 $\pm$ 70 | 11.79 $\pm$ 1.18 | 3 | 0.579
2MASS J04514917+1716255 | J04514917+1716255 | 04 51 49.18 | +17 16 25.43 | K5 | 0.71 | 54.47 $\pm$ 0.15 | 650 $\pm$ 70 | 13.01 $\pm$ 1.30 | 3 | 0.515
Field Stars ($\approx$5 Gyr)
CD-23 1056 | J02464286-2305119 | 02 46 42.88 | -23 05 11.80 | K5 | 0.70 | 23.39 $\pm$ 0.02 | 1436 ${}^{+5981}_{-379}$ | 19 $\pm$ 2 | 5 | 0.582
BD+16 502 | J03435253+1640198 | 03 43 52.56 | +16 40 19.30 | K6 | 0.65 | 17.21 $\pm$ 0.01 | 1428 ${}^{+1033}_{-402}$ | 21.81 $\pm$ 2.18 | 6 | 0.610
HD 266611 | J06570468+3045235 | 06 57 04.68 | +30 45 23.37 | K5 | 0.70 | 18.89 $\pm$ 0.04 | 5889 ${}^{+4947}_{-4247}$ | … | … | …
HD 85512 | J09510700-4330097 | 09 51 07.05 | -43 30 10.02 | K6 | 0.65 | 11.28 $\pm$ 0.01 | 4766 ${}^{+5130}_{-3520}$ | … | … | …
BD+57 1274 | J10314321+5706571 | 10 31 43.22 | +57 06 57.10 | K5 | 0.70 | 17.56 $\pm$ 0.01 | 6219 ${}^{+4884}_{-4311}$ | … | … | …
HD 99492 | J11264627+0300229 | 11 26 46.27 | +03 00 22.75 | K3 | 0.79 | 18.21 $\pm$ 0.06 | 8774 ${}^{+3069}_{-3766}$ | … | … | …
BD+49 2126 | J12150885+4843574 | 12 15 08.84 | +48 43 57.25 | K6 | 0.65 | 22.46 $\pm$ 0.04 | 6469 ${}^{+4817}_{-4366}$ | … | … | …
BD+05 2767 | J13342150+0440026 | 13 34 21.50 | +04 40 02.63 | K5 | 0.70 | 20.44 $\pm$ 0.03 | 6123 ${}^{+4735}_{-4189}$ | … | … | …
HD 128311 | J14360055+0944474 | 14 36 00.56 | +09 44 47.45 | K3 | 0.80 | 16.34 $\pm$ 0.02 | 2672 ${}^{+3962}_{-1929}$ | 11.54 $\pm$ 1.15 | 7 | 0.702
BD+47 2936 | J19505021+4804508 | 19 50 50.24 | +48 04 51.09 | K4 | 0.74 | 37.81 $\pm$ 0.03 | 1639 ${}^{+3891}_{-76}$ | 19.08 $\pm$ 0.06 | 8 | 0.778
WASP-80 | J20124017-0208391 | 20 12 40.16 | -02 08 39.19 | K7 | 0.61 | 49.86 $\pm$ 0.12 | 6067 ${}^{+4328}_{-3238}$ | … | … | …
WASP-69 | J21000618-0505398 | 21 00 06.19 | -05 05 40.03 | K5 | 0.70 | 50.03 $\pm$ 0.13 | 5442 ${}^{+4919}_{-3679}$ | … | … | …
HD 201091 | J21065341+3844529 | 21 06 53.93 | +38 44 57.89 | K5 | 0.70 | 3.50 $\pm$ 0.00 | 4803 ${}^{+4534}_{-1238}$ | 35.37 $\pm$ 3.54 | 9 | 1.889
WASP-29 | J23513108-3954241 | 23 51 31.08 | -39 54 24.25 | K4 | 0.74 | 87.82 $\pm$ 0.31 | 4003 ${}^{+5016}_{-2876}$ | … | … | …

References. — (1) Messina et al. (2010); (2) Jayasinghe et al. (2019); (3)
Douglas et al. (2019); (4) Kiraga & Stępień (2013); (5) Astudillo-Defru et al.
(2017); (6) Lu et al. (2019); (7) Strassmeier et al. (2000); (8) McQuillan et
al. (2014); (9) Mamajek & Hillenbrand (2008).

Table 2: Summary of observations.  Star | Grating | $T_{exp}$ | First | Last | Program
---|---|---|---|---|---
|  |  | Observation | Observation | HST #
|  | [s] | [MM/DD/YY] | [MM/DD/YY] |
Tucana-Horologium Members (40 Myr)
HIP 1910 | G130M | 1636 | 08/30/17 | … | 14784
| G160M | 1502 | 08/30/17 | … | 14784
| G230L | 284 | 08/30/17 | … | 14784
CT Tuc | G130M | 5369 | 03/06/20 | … | 15955
| G160M | 1786 | 03/06/20 | … | 15955
| G230L | 562 | 03/06/20 | … | 15955
CD-46 | G130M | 5173 | 04/27/20 | … | 15955
| G160M | 1837 | 04/27/20 | … | 15955
| G230L | 364 | 04/27/20 | … | 15955
CD-35 | G130M | 4483 | 02/27/20 | … | 15955
| G160M | 3592 | 02/27/20 | … | 15955
| G230L | 578 | 02/27/20 | … | 15955
CD-44 1173 | G130M | 10117 | 07/20/17 | … | 14784
| G160M | 10810 | 07/20/17 | … | 14784
| G230L | 4561 | 07/19/17 | … | 14784
TYC 8083 | G130M | 4280 | 04/18/20 | … | 15955
| G160M | 2068 | 04/18/20 | … | 15955
| G230L | 644 | 04/18/20 | … | 15955
TYC 8098 | G130M | 5932 | 06/18/20 | … | 15955
| G160M | 3512 | 06/18/20 | … | 15955
| G230L | 662 | 06/18/20 | … | 15955
HIP 107345 | G130M | 5201 | 03/13/20 | … | 15955
| G160M | 1845 | 03/13/20 | … | 15955
| G230L | 662 | 03/13/20 | … | 15955
J23261 | G130M | 1352 | 08/18/17 | … | 14784
| G160M | 1402 | 08/18/17 | … | 14784
| G230L | 606 | 08/18/17 | … | 14784
UCAC3 13 | G130M | 6334 | 02/16/21 | … | 15955
| G160M | 1531 | 02/16/21 | … | 15955
| G230L | 1506 | 02/16/21 | … | 15955
Hyades Members (650 Myr)
J03510 | G230L | 88 | 09/29/19 | … | 15091
J03524 | G230L | 868 | 08/07/19 | … | 15091
J03550 | G230L | 208 | 09/18/19 | … | 15091
J04070 | G230L | 395 | 07/29/19 | … | 15091
J04081 | G230L | 777 | 09/22/19 | … | 15091
J04082 | G230L | 271 | 08/24/19 | … | 15091
J04172 | G230L | 678 | 09/06/19 | … | 15091
J04240 | G230L | 934 | 11/01/19 | … | 15091
J04293 | G230L | 1167 | 09/13/19 | … | 15091
V925 Tau | G130M | 5580 | 02/26/20 | … | 15955
| G160M | 2472 | 02/26/20 | … | 15955
| G230L | 1160 | 02/26/20 | … | 15955
J04333 | G230L | 632 | 09/24/19 | … | 15091
J04334 | G230L | 501 | 09/13/19 | … | 15091
STKM 1-495 | G130M | 5556 | 02/23/20 | … | 15955
| G160M | 2806 | 02/23/20 | … | 15955
| G230L | 440 | 02/23/20 | … | 15955
UCAC4 518 | G130M | 5564 | 09/07/20 | … | 15955
| G160M | 2979 | 09/07/20 | … | 15955
| G230L | 832 | 09/07/20 | … | 15955
J04470 | G230L | 356 | 12/02/19 | … | 15091
J04472 | G230L | 571 | 09/29/19 | … | 15091
J04514 | G230L | 1179 | 09/15/19 | … | 15091
Field Stars ($\approx$5 Gyr)
CD-23 | G130M | 5580 | 12/26/20 | … | 15955
| G160M | 5370 | 12/26/20 | … | 15955
| G230L | 1522 | 12/26/20 | … | 15955
BD+16 | G130M | 5578 | 03/03/20 | … | 15955
| G160M | 3428 | 03/03/20 | … | 15955
| G230L | 654 | 03/03/20 | … | 15955
HD 266611 | G130M | 5626 | 02/25/20 | … | 15955
| G160M | 3617 | 02/25/20 | … | 15955
| G230L | 674 | 02/25/20 | … | 15955
HD 85512 | G130M | 4631 | 03/14/15 | … | 13650
| G160M | 400 | 03/14/15 | … | 13650
| G230L | 458 | 03/14/15 | … | 13650
BD+57 | G130M | 6044 | 05/02/20 | … | 15955
| G160M | 3910 | 05/02/20 | … | 15955
| G230L | 846 | 05/02/20 | … | 15955
HD 99492 | G130M | 1920 | 03/31/17 | … | 14633
BD+49 | G130M | 5836 | 06/10/20 | … | 15955
| G160M | 5626 | 06/10/20 | … | 15955
| G230L | 1614 | 06/10/20 | … | 15955
BD+05 | G130M | 11103 | 06/12/20 | … | 15955
| G160M | 2668 | 06/12/20 | … | 15955
| G230L | 1738 | 06/12/20 | … | 15955
HD 128311 | G130M | 1920 | 06/10/17 | … | 14633
BD+47 | G130M | 38375 | 12/16/16 | 05/21/17 | 14767
WASP-80 | G130M | 43273 | 07/22/18 | 06/28/19 | 14767
WASP-69 | G130M | 38226 | 10/12/17 | 10/31/17 | 14767
HD 201091 | G130M | 1300 | 03/28/10 | … | 11687
WASP-29 | G130M | 39020 | 05/19/18 | 11/04/18 | 14767

In this paper, we present an examination of the evolution of upper atmosphere
UV emission lines of 39 K stars, building upon the photometric K star
evolution work of Richey-Yowell et al. (2019), as well as expanding the M star
UV spectral analysis of Loyd et al. (2021) into the K star regime. Using the
Cosmic Origins Spectrograph (COS) on the Hubble Space Telescope (HST), we
obtained spectra of K stars at ages 40 Myr (Kraus et al., 2014), 650 Myr
(Martín et al., 2018), and $\approx$5 Gyr (the average age of the field stars)
in both the NUV (2000-3100 Å) and FUV (1100-1800 Å) in order to measure the UV
temporal and rotation evolution of K stars. In §2, we discuss the target
selection strategy followed by a description of the methods for analysis in
§3. Finally, we present the results in §4 and discussion the implications for
these conclusions in §5.

##  2 Observations

###  2.1 Target Selection

To explore the UV evolution of K stars, we selected stars at three distinct
ages: young, intermediate, and old. The young star sample of 10 K stars was
taken from the Tucana-Horologium young moving group (40 $\pm$ 5 Myr, Kraus et
al. 2014) and confirmed to be bona-fide members through their 3D space
velocity motions and lithium depletion by Kraus et al. (2014). Similarly, our
intermediate stellar sample of 15 K stars was selected from members of the
Hyades moving group (650 $\pm$ 70 Myr, Martín et al. 2018), confirmed through
3D space velocity kinematics by Goldman et al. (2013). Finally, we identified
14 field stars within 30 pc using the SIMBAD database and assumed an age of 5
Gyr for our field age sample, as this is the average age of the stellar field.

K stars with the same spectral type will have different masses at different
ages. Therefore, we must consider the evolution of stars of similar mass in
time. We estimated the masses of the stars using Table 1 of Richey-Yowell et
al. (2019), which were calculated from the spectral type to effective
temperature transformations of Pecaut & Mamajek (2013) and the model
isochrones of Baraffe et al. (2015). We select stars only from 0.6 – 0.8 M⊙ to
be included in our samples. A summary of the targets is presented in Table
1.3.

###  2.2 Hubble Space Telescope Observations

The 39 targets were observed in the NUV and/or FUV with the Hubble Space
Telescope (HST) Cosmic Origins Spectrograph (COS) instrument. Of these, 16 are
new observations from GO-15955 (PI: Richey-Yowell) and 14 are new observations
from SNAP-15091 (PI: Agüeros); the remaining are publicly archived
observations from GO-14767 (PI: Sing), GO-14784 (PI: Shkolnik), GO-13650 (PI:
France), SNAP-14633 (PI: France), and SNAP-11687 (PI: Ayres). The HST program
number is identified with each target in Table 2.

For the new targets from GO-15955 presented in this paper, each was visited
once by HST and was exposed in the COS G230L (NUV), G160M (FUV), and G130M
(FUV) gratings, with central wavelengths 2950 Å, 1577 Å, and 1291 Å
respectively. Total exposure times ranged from $\approx$250 s to
$\approx$12,000 s depending on the brightness of the object. Time between
exposures were typically $\approx$100 s or $\approx$45 minutes if the Earth
was occulting the target during HST’s orbit. The individual exposures were
dithered on the detector to reduce noise that may be introduced into the
spectra from observations at a single location. For the new targets from
SNAP-15091, each was visited once by HST and exposed in the COS G230L grating
with central wavelength 2950 Å. Exposure times ranged from $\approx$90 s to
$\approx$1,200 s. The additional archived targets were observed in at least
one of the COS G230L, G160M, or G130M gratings, although the central
wavelengths varied from program to program. A summary of the observations can
be found in Table 2.

##  3 Analysis

Table 3: Properties of the UV emission lines studied in this work.

Ion | Integration Region [Å] |  $\lambda_{rest}$ [Å] | log${}_{10}(T_{peak}$/[K])
---|---|---|---
C III$\ast$$\ast$footnotemark:  | 1174.5 – 1177.0 | 1174.93 | 4.7
|  | 1175.26 | 4.7
|  | 1175.59 | 4.7
|  | 1175.71 | 4.7
|  | 1175.99 | 4.7
|  | 1176.37 | 4.7
Si III$\ast$$\ast$footnotemark:  | 1205.8 – 1207.2 | 1206.51 | 4.7
N V$\ast$$\ast$footnotemark:  | 1238.0 – 1244.1 | 1238.82 | 5.2
|  | 1242.80 | 5.2
C II$\ast$$\ast$footnotemark:  | 1333.3 – 1336.8 | 1334.52 | 4.5
|  | 1335.71 | 4.5
Si IV | 1393.0 – 1395.0 | 1393.76 | 4.9
| 1401.9 – 1403.5 | 1402.77 | 4.9
C IV | 1547.0 – 1552.0 | 1548.20 | 4.8
|  | 1550.77 | 4.8
He II | 1639.0 – 1642.0 | 1640.4 | 4.9
Mg II$\ast$$\ast$footnotemark:  | 2792.0 – 2807.0 | 2796.35 | 3.6 – 4.0
|  | 2808.53 | 3.6 – 4.0
| Integration Regions [Å]
FUV Pseudo-Continuum | 1160-1172, 1250-1290, 1320-1330,
| 1340-1390, 1406-1540, 1590-1635,
| 1680-1740
NUV Pseudo-Continuum | 2810-3200

∗∗footnotetext: Outside of the GALEX NUV or FUV bandpasses.

Figure 1: HST/COS spectra of HIP 107345 (40 Myr), V925 Tau (650 Myr), and
BD+49 2126 ($\approx$5 Gyr), as representative spectra of the entire sample.
The upper panel shows the luminosity per unit wavelength of the spectra, while
the lower panel shows the surface flux per unit wavelength. The data have been
binned into 1-Å intervals, except for the dotted portion of the spectra that
represent 2$\sigma$ limits of low S/N data (S/N < 5 when binned to 10-Å
intervals).

The 40 Myr and 650 Myr spectra appear quite similar to each other, while the
field star spectrum is almost an order of magnitude lower in the FUV. Note
that the Ly$\alpha$ (1216 Å) and O I (1302 Å) lines between the spectra are
not comparable due to different levels of geocoronal contamination and ISM
absorption.

Figure 2: All HST spectra, separated by age. Flares have been removed for the
direct comparison of quiescent emission and the data have been binned into 1-Å
intervals, except for the dotted portion of the spectra that represent
2$\sigma$ limits of low S/N data (S/N < 5 when binned to 10-Å intervals). Only
one Hyades (650 Myr) member has data in the FUV.

There is clear residual variability between the spectra, particularly at the
younger ages, partly due to a larger spread in spectral mass. Note that
Ly$\alpha$ (1216 Å) and O I (1302 Å) lines between the spectra are not
comparable due to different levels of geocoronal contamination and ISM
absorption.

Figure 3: Color-magnitude diagram of Gaia objects taken from the stardate
package by Angus et al. (2019). The field stars in this study are overlaid as
black points. These stars were aged using stardate with the G, BP, and RP
magnitudes and parallaxes from Gaia EDR3; the J, H, and K magnitudes from
2MASS; and the rotation period where available.

###  3.1 Data Reduction

Initial data reduction was carried out through the CALCOS pipeline version
3.3.10. Since we are interested in the evolution of the quiescent UV flux of
these stars, we removed any flares in the data. If a flare was found, we
removed the flare by discarding the time region where the flare occurred using
the python program costools. The remaining data were then re-run through the
CALCOS pipeline to produce flux-calibrated 1D spectra. While these spectra
ideally represents a quiescent stellar state, unresolved flares may still be
present.

To identify flares, we created light curves by extracting the spectrum (in
terms of [counts s-1 Å-1]) from the time-tagged (corrtag) files and
subtracting the background region (see Figure 3 of Loyd & France 2014). After
removing regions contaminated by Earth’s geocoronal emission (such as
Ly$\alpha$ and O I), we then applied the flux calibration from the original
x1d spectrum to the time-tagged spectrum, to create a conversion from [counts
s-1 Å-1] to [erg s-1 cm-2 Å-1]. After this, we integrated over the bandpass to
create a flux-calibrated light curve for each target binned in 5 second
intervals. A dozen flares above 3$\sigma$ were identified and will be analyzed
separately in a future paper. No additional flare candidates were identified
by eye.

For spectra that were observed during multiple visits, the data were coadded.
We used the coaddx1d111https://github.com/cosmonaut/python-coaddx1d (Danforth
et al., 2010, 2016) code originally in IDL and translated into python by
Nicholas Nell to coadd the data, weighting by error.

The final spectra were shifted to the rest frame of the star and reviewed
individually. Examples representative of the spectra at each age are shown in
Figure 1.

Two of the observed targets from GO-15955, STKM 1-495 and UCAC4 518-008871
showed clear binarity in our spectroscopic observations. Additionally,
archival Keck NIRC2 images (Proposal I.D. #H246N2, PI: Gaidos) confirm that
STKM 1-495 is in fact a binary system. These two targets were thus not
considered in our study due to both blending and potential increased activity
due to tidal and/or magnetic interactions between their companions.

###  3.2 Measuring Line and Continua Fluxes

We analyze the strongest chromospheric and transition region emission features
in the COS bandpasses: C III, Si III, N V, C II, Si IV, C IV, He II, and Mg
II. These lines were also analyzed in Loyd et al. (2021) and thus we can make
direct comparisons with the UV evolution of early M stars. We list the rest
wavelength and formation temperature in Table 3, as well as the wavelength
range over which we integrate to calculate the surface fluxes.

For more accurate line fluxes, we used the astropy module specutils to fit the
continuum immediately outside the spectral regions identified in Table 3 with
a cubic Chebyshev polynomial and subtract the fit. The accuracy of these fits
were confirmed by eye for each line of each target.

We additionally analyze the NUV continuum as well as the FUV continuum of each
spectrum. The continua in these regions are dominated by multiple weak
absorption and emission lines and therefore are more accurately considered to
be "psuedo-continua". To calculate the pseudo-continua, we consider the same
regions as Loyd et al. (2021), which cumulatively cover 251 Å. The wavelengths
of the regions over which we integrated are additionally found in Table 3.
These regions exclude areas of strong emission lines not considered in this
study and non-stellar background contamination such as geo-coronal emission
features. For the archived field-age targets which only had data in one of the
two FUV gratings, we take the ratio of the target pseudo-continuum available
to the same region in HD 266611 (the target with the FUV pseudo-continuum flux
closest to the average of the FUV pseudo-continuum flux for the entire
sample), and multiply the FUV pseudo-continuum flux of HD 266611 by that
ratio. This assumes that the shape of the continuum does not vary
significantly between targets, as is the case in Figure 2.

In order to calculate the surface flux from the observed flux, we estimated
the radius of the star first using the spectral type to effective temperature
relationship in Pecaut & Mamajek (2013) and then using the effective
temperature to mass and radius relationships in Baraffe et al. (2015).

Figure 4: Surface flux as a function of stellar age for the 39 K stars in our
sample. 40 Myr stars are in blue, 650 Myr in orange, and field stars in green.
Only one Hyades (650 Myr) member has data in the FUV. The ages of the field
stars were determined using stardate (Angus et al., 2019). Model fits to
Equation 1 are shown in gray. Single gray lines well away from the average are
all outliers compared to the total data set. The averages of the fits are
shown in black and the resulting equations for the power law decline are
shown, presented in full in Table 4. The red lines are the model fits for
early M stars from Loyd et al. (2021). Figure 5: Same as Figure 4, but for the
evolution of the surface flux with rotation period. Figure 6: Same as Figure
4, but for the evolution of the surface flux with Rossby number.

###  3.3 Determining Ages, Rotation Periods, and Rossby Numbers

While the ages of the moving group members are well constrained, the ages of
field stars are typically estimated from isochrones and gyrochrones. We
utilized the stardate package by Angus et al. (2019) to determine these ages,
providing the $G,BP,$ and $RP$ magnitudes and parallaxes from $Gaia$ EDR3
(Brown et al., 2021), the $J,H,$ and $K$ magnitudes from 2MASS (Skrutskie et
al., 2006), and the rotation period if available. A summary of these data are
seen in Figure 3.

The rotation periods were gathered from the literature where available. For
stars with no reported error in their rotation period, we assume 10%. For
those without measured rotation periods, we carried out a Lomb-Scargle (Lomb,
1976; Scargle, 1982) search of $TESS$ lightcurves using the program lightkurve
(Lightkurve Collaboration et al., 2018). The lightkurve periodograms test
frequencies up to the Nyquist frequency, using grid steps of
$1/\textrm{observation time}$. UCAC3 13-31217, HD 266611, HD 85512, BD+57
1274, BD+49 2126, and BD+05 2767 had TESS observations; however, no clear
variability was identified, which is not uncommon for older stars.

The Rossby number is defined as the ratio of a star’s rotation period to its
convective turnover time (i.e. the time it takes for circulation within a
convective cell): $R_{o}=P_{rot}/\tau$. This number was first shown by Noyes
et al. (1984) to be correlated to the magnetic activity of stars, and
therefore should relate directly to the chromospheric, transition region, and
coronal surface fluxes of stars which are driven by this magnetic activity. To
calculate the convective turnover time, we use the empirical relationship
between $V-K_{s}$ color and the turnover timescale of convection by Wright et
al. (2018). For two stars in our data without $V-K_{s}$ colors, we instead use
the mass – convective turnover time relationship, again established by Wright
et al. (2018). These were found to produce consistent Rossby numbers with the
rest of the data set.

The complete list of estimated stellar ages, measured rotation periods, and
derived Rossby numbers can be found in Table 1.3.

###  3.4 Model Fitting

To fit the surface flux, $F$, as a function of age, rotation period, or Rossby
number, we adopt a piecewise model that includes a period of saturated
(constant) activity, followed by a power law decline, as utilized in other
chromospheric studies (e.g. Wright et al., 2011; Jackson et al., 2012; Loyd et
al., 2021; Pineda et al., 2021):

$F=\begin{cases}F_{sat},&x\leq x_{sat}\\\
F_{sat}(x/x_{sat})^{-\alpha},&x>x_{sat}\\\ \end{cases}$ (1)

where the data $x$ can be age, rotation period, or Rossby number; and the free
parameters are the surface flux at saturation, $F_{sat}$; end of saturation,
$x_{sat}$; and the power-law index, $\alpha$. We additionally include a
hyperparameter, $f$, representing a fractional amount by which the error is
underestimated due to intrinsic variation among stars. We fit this model using
the Markov Chain Monte Carlo (MCMC) package emcee (Foreman-Mackey et al.,
2013). We use uniform priors on the the power-law index, restricting it to
$0\leq\alpha\leq 3$, as values between this range are what is known for M
stars (Loyd et al., 2021) and G stars (Güdel, 2007, and references therein).
Placing a prior on the power-law index additionally helps constrain the age at
which saturation would occur. We ran 10,000 steps, taking the result to be the
50th percentile value, and the errors as the 16th and 84th percentile values.
These results can be found in Table 4.

##  4 Results

Figure 1 shows examples of K star spectra representative of each sample at the
three different ages used in this study: 40 Myr, 650 Myr, and $\approx$5 Gyr.
The 40 Myr and 650 Myr spectra appear to be quite similar in both luminosity
and surface flux per unit wavelength, while the 5 Gyr spectrum is an order of
magnitude lower in the FUV, and half an order of magnitude lower in the NUV.

These results are notably different than those of the early M stars presented
in Loyd et al. (2021). In their work, the authors show that the early M dwarf
FUV spectra at each age are an order of magnitude separate from each other,
with the exception of the rotation-period-normalized spectra, which lie on top
of each other. In the NUV, the luminosity and surface flux per unit wavelength
of the Hyades and field-age spectra are similar, while the young star spectrum
is an order of magnitude larger. However, the FUV spectra of both the 40 Myr
and 650 Myr K stars have similar luminosity densities and surface flux
densities to the young M stars.

As well as comparing the broadband characteristics of the spectra, we also
analyze specific spectral lines that probe the chromosphere and transition
region of the stars and see how those evolve in time. Figure 4 shows the
evolution of the surface flux of the strong UV emission lines C III, Si III, N
V, C II, Si IV, C IV, He II, Mg II, and the FUV and NUV pseudo-continua as a
function of stellar age. While the ages are well constrained for the moving
group samples, large errorbars persist for the ages of the field age stars,
even for those with measured rotation periods to aid in gyrochronological and
isocohronal dating, as discussed in Section 3. We again see in each emission
line region that the K dwarfs appear to have a prolonged saturation period as
compared to M stars, which fall off by only a couple hundred million years
old. While the evolution for M stars is now quite well known, the age at which
K stars actually do begin to decline in UV activity remains poorly
constrained.

Most (32/39) of our targets have measured rotation periods. The rotation
periods of the 40 Myr and 650 Myr samples overlap slightly, while the field-
age stars are distinct in their longer rotation. Figure 5 shows the surface
flux evolution of the emission lines and pseudo-continuum as a function of
rotation period for those with known rotation periods. The surface flux
remains roughly constant from $P_{rot}\approx 0.5-10$ days for all emission
lines and the pseudo-continuum, at which point the flux decays.

Since our stars are of similar masses, the evolution of the surface flux of
the emission lines with Rossby number is similar to that of the rotation
period evolution, as seen in Figure 6. The Rossby number at which the flux
starts to decline is $Ro\sim 0.2$. Since $Ro$ is the rotation period
normalized by the convective turnover time, it is more appropriate to compare
the K and M stars in terms of $Ro$ instead of rotation. Interestingly, we see
that the emission line evolution fits between the partially convective K stars
and early M stars are nearly identical to each other. This implies that the
evolution of K dwarf rotation period with age is distinct from that of early M
stars and is the primary driver of the observed differences in activity
evolution with age.

In each of these cases, the slope of the Mg II surface flux evolution is more
shallow than for the other emission lines. Mg II is the only chromospheric
line analyzed here, while the others map to the transition region of the star.
The difference in the slopes may represent a difference in heating mechanism
between the chromosphere and the transition region. However, ISM absorption
may also play a role, since the Tuc-Hor and Hyades moving groups lie at a
distance of 40 – 60 pc and thus may have a greater Mg column density than for
the field stars, which range from 3 – 50 pc. A greater Mg absorption in the
younger stars may artificially create a descreased slope between the young and
field-aged stars. For direct comparison among all of the lines, the model fits
for the UV surface flux evolution with age, rotation period, and Rossby number
are presented in Table 4.

Table 4: Results of the MCMC fits to the data seen in Figures 4, 5, and 6.  Line | log${}_{10}(x_{sat}$) | log${}_{10}(F_{sat}$) | $\alpha$ | log($f$)
---|---|---|---|---
| [year] | [erg s-1 cm-2] |  |
Age |  |  |  |
Mg II | $8.920_{-0.168}^{+0.275}$ | $6.016_{-0.041}^{+0.041}$ | $0.613_{-0.150}^{+0.277}$ | $-0.859_{-0.153}^{+0.174}$
C II | $8.779_{-0.130}^{+0.126}$ | $5.012_{-0.070}^{+0.064}$ | $1.090_{-0.161}^{+0.195}$ | $-0.649_{-0.222}^{+0.253}$
Si III | $8.760_{-0.153}^{+0.148}$ | $4.637_{-0.077}^{+0.071}$ | $1.142_{-0.185}^{+0.230}$ | $-0.534_{-0.238}^{+0.269}$
C III | $8.708_{-0.129}^{+0.136}$ | $4.818_{-0.075}^{+0.066}$ | $1.298_{-0.182}^{+0.223}$ | $-0.592_{-0.237}^{+0.266}$
Si IV | $8.797_{-0.149}^{+0.148}$ | $4.626_{-0.082}^{+0.072}$ | $1.250_{-0.210}^{+0.277}$ | $-0.496_{-0.238}^{+0.275}$
He II | $8.753_{-0.179}^{+0.194}$ | $5.131_{-0.064}^{+0.054}$ | $1.331_{-0.261}^{+0.356}$ | $-0.885_{-0.297}^{+0.327}$
C IV | $8.847_{-0.111}^{+0.153}$ | $5.299_{-0.079}^{+0.066}$ | $1.533_{-0.244}^{+0.352}$ | $-0.635_{-0.266}^{+0.316}$
N V | $8.778_{-0.124}^{+0.123}$ | $4.580_{-0.076}^{+0.069}$ | $1.239_{-0.182}^{+0.220}$ | $-0.580_{-0.235}^{+0.274}$
FUV Cont. | $8.534_{-0.258}^{+0.215}$ | $6.246_{-0.087}^{+0.082}$ | $0.736_{-0.138}^{+0.153}$ | $-0.354_{-0.226}^{+0.265}$
NUV Cont. | $9.368_{-0.309}^{+0.112}$ | $7.341_{-0.089}^{+0.080}$ | $1.688_{-0.901}^{+0.875}$ | $-0.010_{-0.206}^{+0.247}$
Rotation Period |  |  |  |
Mg II | $1.073_{-0.034}^{+0.029}$ | $6.070_{-0.045}^{+0.046}$ | $2.217_{-0.683}^{+0.506}$ | $-0.945_{-0.170}^{+0.188}$
C II | $0.694_{-0.158}^{+0.115}$ | $5.117_{-0.069}^{+0.077}$ | $1.577_{-0.314}^{+0.325}$ | $-0.938_{-0.224}^{+0.259}$
Si III | $0.842_{-0.061}^{+0.049}$ | $4.717_{-0.067}^{+0.064}$ | $2.526_{-0.403}^{+0.302}$ | $-0.848_{-0.229}^{+0.281}$
C III | $0.823_{-0.039}^{+0.034}$ | $4.901_{-0.060}^{+0.055}$ | $2.773_{-0.276}^{+0.161}$ | $-0.979_{-0.238}^{+0.278}$
Si IV | $0.786_{-0.094}^{+0.068}$ | $4.714_{-0.072}^{+0.073}$ | $2.088_{-0.317}^{+0.265}$ | $-0.767_{-0.239}^{+0.277}$
He II | $0.814_{-0.048}^{+0.039}$ | $5.200_{-0.039}^{+0.038}$ | $2.642_{-0.351}^{+0.245}$ | $-1.524_{-0.311}^{+0.343}$
C IV | $0.844_{-0.047}^{+0.046}$ | $5.369_{-0.069}^{+0.062}$ | $2.730_{-0.407}^{+0.197}$ | $-0.875_{-0.262}^{+0.310}$
N V | $0.807_{-0.069}^{+0.049}$ | $4.682_{-0.060}^{+0.058}$ | $2.466_{-0.395}^{+0.307}$ | $-0.995_{-0.243}^{+0.277}$
FUV Cont. | $0.841_{-0.073}^{+0.060}$ | $6.322_{-0.084}^{+0.083}$ | $2.428_{-0.348}^{+0.301}$ | $-0.598_{-0.232}^{+0.283}$
NUV Cont. | $1.046_{-0.034}^{+0.030}$ | $7.459_{-0.097}^{+0.092}$ | $2.596_{-0.620}^{+0.297}$ | $-0.160_{-0.211}^{+0.266}$
Rossby Number |  |  |  |
Mg II | $-0.506_{-0.502}^{+0.164}$ | $6.040_{-0.048}^{+0.054}$ | $0.152_{-0.105}^{+0.252}$ | $-0.763_{-0.159}^{+0.187}$
C II | $-1.025_{-0.192}^{+0.262}$ | $5.123_{-0.090}^{+0.103}$ | $0.976_{-0.184}^{+0.288}$ | $-0.782_{-0.233}^{+0.273}$
Si III | $-0.833_{-0.344}^{+0.327}$ | $4.705_{-0.093}^{+0.105}$ | $1.087_{-0.378}^{+0.843}$ | $-0.618_{-0.244}^{+0.306}$
C III | $-0.775_{-0.235}^{+0.169}$ | $4.888_{-0.079}^{+0.088}$ | $1.552_{-0.476}^{+0.594}$ | $-0.727_{-0.252}^{+0.294}$
Si IV | $-0.847_{-0.250}^{+0.216}$ | $4.710_{-0.092}^{+0.108}$ | $1.267_{-0.302}^{+0.384}$ | $-0.620_{-0.239}^{+0.291}$
He II | $-0.700_{-0.041}^{+0.032}$ | $5.199_{-0.037}^{+0.036}$ | $2.687_{-0.330}^{+0.216}$ | $-1.563_{-0.306}^{+0.344}$
C IV | $-0.710_{-0.040}^{+0.038}$ | $5.381_{-0.057}^{+0.055}$ | $2.746_{-0.336}^{+0.182}$ | $-1.030_{-0.257}^{+0.302}$
N V | $-0.860_{-0.198}^{+0.167}$ | $4.689_{-0.077}^{+0.088}$ | $1.449_{-0.358}^{+0.498}$ | $-0.867_{-0.254}^{+0.296}$
FUV Cont. | $-0.720_{-0.160}^{+0.121}$ | $6.336_{-0.083}^{+0.088}$ | $1.692_{-0.346}^{+0.324}$ | $-0.663_{-0.248}^{+0.289}$
NUV Cont. | $-0.474_{-0.382}^{+0.136}$ | $7.385_{-0.103}^{+0.091}$ | $0.138_{-0.103}^{+0.247}$ | $0.015_{-0.221}^{+0.280}$

###  4.1 Comparison with GALEX photometry

Using GALEX broadband UV photometry of 400 K stars ranging in age from 10 Myr
to $\approx$5 Gyr, Richey-Yowell et al. (2019) saw a decrease (half an order
of magnitude) in median flux by 650 Myr; however, the interquartiles (the
middle 50% of the population) continued to overlap with the younger stars.
Figure 7 shows the relative GALEX UV surface fluxes of the stars in this work
compared to their complete sample. The stars in this work are typical compared
to other K stars and fall within the interquartiles of the GALEX sample.

It should also be noted that the X-ray evolution of K stars (e.g. Richey-
Yowell et al., 2019; Jackson et al., 2012) starts to decline after 10 Myr,
much earlier than seen in the UV, unlike for M stars where both the UV and
X-ray decline at similar ages (Shkolnik & Barman, 2014). In comparing the
fractional X-ray (coronal) and Ly$\alpha$ (chromospheric) flux to bolometric
luminosity, Linsky et al. (2020) observed that the relative emission from the
chromosphere and corona show different distributions between M and K stars,
potentially representing different heating mechanisms between the
chromospheres and coronae of these stars. The corona can also heat the
transition region through thermal conduction in a process known as back-
heating (e.g., Kopp & Kuperus, 1968; Athay, 1990). Therefore, the FUV lines in
this study that map the transition region may also experience a different
evolution than the X-rays or other UV chromospheric lines.

Figure 7: Relative GALEX photometric UV fluxes of the stars in our sample
compared to the entire sample studied in Richey-Yowell et al. (2019).
Artificial age scatter has been added to both samples for clarity. The colored
stars represent targets in our sample, whereas gray points are the entire
photometric sample. White diamonds represent the median flux of that age bin,
with the interquartiles (middle 50%) presented as the error bars. Our targets
typically lie within the interquartiles and do not appear to be unique to the
total sample.

##  5 Discussion: What is causing the prolonged saturation period in K stars?

###  5.1 A Result of Different Initial Rotation Periods?

Several studies (e.g. Tu et al., 2015; Gondoin, 2018; Magaudda et al., 2020)
have shown that the initial rotation period of low-mass stars affects the
evolution of the X-ray flux. Stars with faster initial rotation periods are
saturated for longer and decline at a faster rate; conversely, stars with
slower initial rotation periods decline in flux at younger ages, but decline
at a lower rate. This trend most likely continues into the UV and accounts
partially for the large scatter seen in Richey-Yowell et al. (2019) among K
dwarfs. However, the rotation rates of our sample span a range of periods, yet
we see constant flux for those at shorter or longer periods within the same
age group. Therefore, it is unlikely that the prolonged period of saturated
activity observed is due to a selection of only stars with faster original
rotation periods.

###  5.2 A Result of a Stalling in K Dwarf Rotational Spin-Down?

Using K2 data from open clusters, Curtis et al. (2019, 2020) show that current
gyrochronological models fail to predict the rotation period of K stars beyond
650 Myr. Instead, the K stars appear to stall their spin-down, potentially
maintaining their rotation period for over 1.3 Gyr before resuming to spin
down. This effect is not seen in the same rotation data from Curtis et al.
(2019) for G stars and it remains unknown if this occurs in M stars.

Spada & Lanzafame (2020) were able to closely recreate these observations
using a two-zone model for the star, where differential rotation between the
convective envelope and the radiative core develops as magnetic wind braking
reaches a maximum. The stalling may occur due to a balance between the spin-
down torque from the magnetic wind braking and a spin-up torque from the
transfer of angular momentum from the more quickly rotating radiative core.
The redistribution of angular momentum is highly mass dependent, therefore the
lower mass stars (i.e., K stars) are stalled for longer periods. Curtis et al.
(2020), however, do not rule out other theories such as reduced magnetic
braking. If the core-envelope recoupling is the underlying method for the
stalling, then this would imply that M dwarfs would not experience this
effect, since stars become more fully convective with lower mass, and that
there is perhaps a dominant relative size of the convection zone at which
stalling is more pronounced.

This phenomenon could explain the prolonged saturation period seen in the
results presented above. If the K stars remain at a faster rotation period
through a couple of Gyrs, than we would expect the evolution with age to
follow a similar trend to that of the evolution of rotation period with age,
maintaining a faster rotation period and thus a higher flux for longer periods
of time. Similarly, flux as a function of rotation period would not be
dissimilar from a case in which the spin-down of the star did not stall, which
indeed is what we see when comparing the K stars with the M stars.

##  6 Summary

In this paper, we present HST/COS UV spectra of 39 K stars at ages 40 Myr, 650
Myr, and $\approx$5 Gyr. Combining new observations of 28 K stars with 11
archived field-aged K dwarfs, we compared the evolutionary trends of K stars
as a function of age, rotation, and Rossby number.

Specifically, we analyze the evolution of the surface flux of the strong UV
emission lines, i.e. C III, Si III, N V, C II, Si IV, C IV, He II, Mg II, as
well as the FUV and NUV continua. We find that for each emission line and the
pseudo-continua, the flux remains saturated between the 40 Myr and 650 Myr
stars, before dropping off by up to two orders of magnitude. The UV evolution
of K stars with age is thus distinct from that of early M stars, which begin
to decline by a couple hundred Myr and show a clear decrease already by 650
Myr. However, when comparing the evolution of K and M star UV activity with
rotation or Rossby number, this distinction goes away and the fits appear
quite similar.

This may imply that the rotational evolution of K star with age is dissimilar
than that of M stars. This supports recent works by Curtis et al. (2019,
2020), which show that K dwarfs experience a period of time where their spin-
down stalls, and the stars remain at a constant rotation period. This
phenomenon is primarily thought to be caused by a core-envelope recoupling
event and would weaken toward later type stars as the relative size of the
convection zone grows, thus leading the early M dwarfs to decline in rotation
rate and UV activity sooner than for K stars.

K dwarfs have recently garnered attention for providing such a hospitable
environment as to host potentially “superhabitable” planets (e.g. Cuntz &
Guinan, 2016; Heller & Armstrong, 2014; Arney, 2019); yet, a prolonged period
of saturated flux weakens this case. Further observations of open cluster
members between 650 Myr – 3 Gyr will be required to resolve at what age K
dwarf rotational spin-down resumes. The delayed time evolution of K stars
would increase the accumulated UV flux incident on their planets if the
saturated period lasts for multiple Gyr. Additionally, a comparison of the
flare evolution between these two types of stars will contribute to
understanding the atmospheres of planets around these types of stars.

The authors would like to thank Dr. Jeff Linsky for a timely and insightful
referee report. Support for this work was provided by NASA through grants
numbered HST-GO-15091.001-A and HST-GO-15955.01 from the Space Telescope
Science Institute, which is operated by AURA, Inc., under NASA contract NAS
5-26555. T.R.Y. would like to acknowledge additional support from the Future
Investigators in NASA Earth and Space Exploration (FINESST) award
19-ASTRO20-0081 and thanks Aishwarya Iyer for useful MCMC discussions. This
research has made use of the SIMBAD database, operated at CDS, Strasbourg,
France. This work has made use of data from the European Space Agency (ESA)
mission Gaia (https://www.cosmos.esa.int/gaia), processed by the Gaia Data
Processing and Analysis Consortium (DPAC,
https://www.cosmos.esa.int/web/gaia/dpac/consortium)

## References

  * Airapetian et al. (2017) Airapetian, V. S., Glocer, A., Khazanov, G. V., et al. 2017, ApJ, 836, L3, doi: 10.3847/2041-8213/836/1/L3
  * Angus et al. (2019) Angus, R., Morton, T. D., Foreman-Mackey, D., et al. 2019, AJ, 158, 173, doi: 10.3847/1538-3881/ab3c53
  * Arney (2019) Arney, G. N. 2019, ApJ, 873, L7, doi: 10.3847/2041-8213/ab0651
  * Arney et al. (2017) Arney, G. N., Meadows, V. S., Domagal-Goldman, S. D., et al. 2017, ApJ, 836, 1, 49, 19 pp. (2017)., 836, doi: 10.3847/1538-4357/836/1/49
  * Astudillo-Defru et al. (2017) Astudillo-Defru, N., Delfosse, X., Bonfils, X., et al. 2017, A&A, 600, A13, doi: 10.1051/0004-6361/201527078
  * Athay (1990) Athay, R. G. 1990, ApJ, 362, 364, doi: 10.1086/169272
  * Baraffe et al. (2015) Baraffe, I., Homeier, D., Allard, F., & Chabrier, G. 2015, A&A, 577, A42, doi: 10.1051/0004-6361/201425481
  * Barclay et al. (2018) Barclay, T., Pepper, J., & Quintana, E. V. 2018, ApJS, 239, 2, doi: 10.3847/1538-4365/aae3e9
  * Brown et al. (2021) Brown, A. G. A., Vallenari, A., Prusti, T., et al. 2021, Astronomy & Astrophysics, 649, A1, doi: 10.1051/0004-6361/202039657
  * Carrington (1859) Carrington, R. C. 1859, MNRAS, 20, 13, doi: 10.1093/mnras/20.1.13
  * Cuntz & Guinan (2016) Cuntz, M., & Guinan, E. F. 2016, ApJ, 827, 1, 79, 9 pp. (2016)., 827, doi: 10.3847/0004-637X/827/1/79
  * Curtis et al. (2019) Curtis, J. L., Agüeros, M. A., Douglas, S. T., & Meibom, S. 2019, ApJ, 879, 49, doi: 10.3847/1538-4357/ab2393
  * Curtis et al. (2020) Curtis, J. L., Agüeros, M. A., Matt, S. P., et al. 2020, ApJ, 904, 140, doi: 10.3847/1538-4357/abbf58
  * Danforth et al. (2010) Danforth, C. W., Keeney, B. A., Stocke, J. T., Shull, J. M., & Yao, Y. 2010, ApJ, 720, 976, doi: 10.1088/0004-637X/720/1/976
  * Danforth et al. (2016) Danforth, C. W., Keeney, B. A., Tilton, E. M., et al. 2016, ApJ, 817, 111, doi: 10.3847/0004-637X/817/2/111
  * Domagal-Goldman et al. (2014) Domagal-Goldman, S. D., Segura, A., Claire, M. W., Robinson, T. D., & Meadows, V. S. 2014, ApJ, 792, 90, doi: 10.1088/0004-637X/792/2/90
  * Douglas et al. (2019) Douglas, S. T., Curtis, J. L., Agüeros, M. A., et al. 2019, ApJ, 879, 100, doi: 10.3847/1538-4357/ab2468
  * Fauchez et al. (2019) Fauchez, T. J., Turbet, M., Villanueva, G. L., et al. 2019, ApJ, 887, 194, doi: 10.3847/1538-4357/ab5862
  * Fontenla et al. (2016) Fontenla, J. M., Linsky, J. L., Witbrod, J., et al. 2016, ApJ, 830, 154, doi: 10.3847/0004-637X/830/2/154
  * Foreman-Mackey et al. (2013) Foreman-Mackey, D., Hogg, D. W., Lang, D., & Goodman, J. 2013, PASP, 125, 306, doi: 10.1086/670067
  * France et al. (2013) France, K., Froning, C. S., Linsky, J. L., et al. 2013, ApJ, 763, 149, doi: 10.1088/0004-637X/763/2/149
  * Goldman et al. (2013) Goldman, B., Röser, S., Schilbach, E., et al. 2013, A&A, 559, A43, doi: 10.1051/0004-6361/201321727
  * Gondoin (2018) Gondoin, P. 2018, ArXiv e-prints.  https://arxiv.org/abs/1808.01824
  * Güdel (2007) Güdel, M. 2007, Living Reviews in Solar Physics, 4, 3, doi: 10.12942/lrsp-2007-3
  * Harman et al. (2015) Harman, C. E., Schwieterman, E. W., Schottelkotte, J. C., & Kasting, J. F. 2015, ApJ, 812, 137, doi: 10.1088/0004-637X/812/2/137
  * Heller & Armstrong (2014) Heller, R., & Armstrong, J. 2014, Astrobiology, Vol. 14, No. 1, p. 50-66., 14, 50, doi: 10.1089/ast.2013.1088
  * Hu et al. (2012) Hu, R., Seager, S., & Bains, W. 2012, ApJ, 761, 2, 166, 29 pp. (2012)., 761, doi: 10.1088/0004-637X/761/2/166
  * Hunter (2007) Hunter, J. D. 2007, Computing in Science and Engineering, 9, 90, doi: 10.1109/MCSE.2007.55
  * Jackson et al. (2012) Jackson, A. P., Davis, T. A., & Wheatley, P. J. 2012, MNRAS, 422, 2024, doi: 10.1111/j.1365-2966.2012.20657.x
  * Jayasinghe et al. (2019) Jayasinghe, T., Stanek, K. Z., Kochanek, C. S., et al. 2019, MNRAS, 485, 961, doi: 10.1093/mnras/stz444
  * Jones et al. (2001) Jones, E., Oliphant, T., Peterson, P., et al. 2001, SciPy: Open source scientific tools for Python.  http://www.scipy.org/
  * Kiraga & Stępień (2013) Kiraga, M., & Stępień, K. 2013, Acta Astron., 63, 53.  https://arxiv.org/abs/1304.3236
  * Kopp & Kuperus (1968) Kopp, R. A., & Kuperus, M. 1968, Sol. Phys., 4, 212, doi: 10.1007/BF00148082
  * Koskinen et al. (2010) Koskinen, T. T., Yelle, R. V., Lavvas, P., & Lewis, N. K. 2010, ApJ, 723, 1, pp. 116-128 (2010)., 723, 116, doi: 10.1088/0004-637X/723/1/116
  * Kraus et al. (2014) Kraus, A. L., Shkolnik, E. L., Allers, K. N., & Liu, M. C. 2014, AJ, 147, 146, doi: 10.1088/0004-6256/147/6/146
  * Lammer et al. (2003) Lammer, H., Selsis, F., Ribas, I., et al. 2003, ApJ, 598, L121, doi: 10.1086/380815
  * Lammer et al. (2007) Lammer, H., Lichtenegger, H. I., Kulikov, Y. N., et al. 2007, Astrobiology, 7, 185, doi: 10.1089/ast.2006.0128
  * Lichtenegger et al. (2010) Lichtenegger, H., Lammer, H., Grießmeier, J.-M., et al. 2010, Icarus, 210, 1, doi: 10.1016/j.icarus.2010.06.042
  * Lightkurve Collaboration et al. (2018) Lightkurve Collaboration, Cardoso, J. V. d. M., Hedges, C., et al. 2018, Lightkurve: Kepler and TESS time series analysis in Python, Astrophysics Source Code Library.  http://ascl.net/1812.013
  * Lingam & Loeb (2017) Lingam, M., & Loeb, A. 2017, eprint arXiv:1710.11134.  https://arxiv.org/abs/1710.11134
  * Linsky & Güdel (2015) Linsky, J. L., & Güdel, M. 2015, in Astrophysics and Space Science Library, Vol. 411, Characterizing Stellar and Exoplanetary Environments, ed. H. Lammer & M. Khodachenko, 3, doi: 10.1007/978-3-319-09749-7_1
  * Linsky et al. (2020) Linsky, J. L., Wood, B. E., Youngblood, A., et al. 2020, ApJ, 902, 3, doi: 10.3847/1538-4357/abb36f
  * Lomb (1976) Lomb, N. R. 1976, Ap&SS, 39, 447, doi: 10.1007/BF00648343
  * Loyd & France (2014) Loyd, R. O. P., & France, K. 2014, ApJS, 211, 9, doi: 10.1088/0067-0049/211/1/9
  * Loyd et al. (2018) Loyd, R. O. P., Shkolnik, E. L., Schneider, A. C., et al. 2018, ApJ, 867, 70, doi: 10.3847/1538-4357/aae2ae
  * Loyd et al. (2021) —. 2021, ApJ, 907, 91, doi: 10.3847/1538-4357/abd0f0
  * Lu et al. (2019) Lu, H.-p., Zhang, L.-y., Shi, J., et al. 2019, ApJS, 243, 28, doi: 10.3847/1538-4365/ab2f8f
  * Magaudda et al. (2020) Magaudda, E., Stelzer, B., Covey, K. R., et al. 2020, A&A, 638, A20, doi: 10.1051/0004-6361/201937408
  * Mamajek & Hillenbrand (2008) Mamajek, E. E., & Hillenbrand, L. A. 2008, ApJ, 687, 1264, doi: 10.1086/591785
  * Martín et al. (2018) Martín, E. L., Lodieu, N., Pavlenko, Y., & Béjar, V. J. S. 2018, ApJ, 856, 40, doi: 10.3847/1538-4357/aaaeb8
  * McQuillan et al. (2014) McQuillan, A., Mazeh, T., & Aigrain, S. 2014, ApJS, 211, 24, doi: 10.1088/0067-0049/211/2/24
  * Meadows & Barnes (2018) Meadows, V. S., & Barnes, R. K. 2018, Factors Affecting Exoplanet Habitability, ed. H. J. Deeg & J. A. Belmonte, 57, doi: 10.1007/978-3-319-55333-7_57
  * Messina et al. (2010) Messina, S., Desidera, S., Turatto, M., Lanzafame, A. C., & Guinan, E. F. 2010, A&A, 520, A15, doi: 10.1051/0004-6361/200913644
  * Noyes et al. (1984) Noyes, R. W., Weiss, N. O., & Vaughan, A. H. 1984, ApJ, 287, 769, doi: 10.1086/162735
  * Pagano (2009) Pagano, I. 2009, Ap&SS, 320, 115, doi: 10.1007/s10509-008-9821-8
  * Peacock et al. (2019) Peacock, S., Barman, T., Shkolnik, E. L., et al. 2019, ApJ, 886, 77, doi: 10.3847/1538-4357/ab4f6f
  * Peacock et al. (2020) —. 2020, ApJ, 895, 5, doi: 10.3847/1538-4357/ab893a
  * Pecaut & Mamajek (2013) Pecaut, M. J., & Mamajek, E. E. 2013, ApJS, 208, 9, doi: 10.1088/0067-0049/208/1/9
  * Pineda et al. (2021) Pineda, J. S., Youngblood, A., & France, K. 2021, ApJ, 911, 111, doi: 10.3847/1538-4357/abe8d7
  * Price-Whelan et al. (2018) Price-Whelan, A. M., Sipócz, B. M., Günther, H. M., et al. 2018, AJ, 156, 123, doi: 10.3847/1538-3881/aabc4f
  * Richey-Yowell et al. (2019) Richey-Yowell, T., Shkolnik, E. L., Schneider, A. C., et al. 2019, ApJ, 872, 17, doi: 10.3847/1538-4357/aafa74
  * Scargle (1982) Scargle, J. D. 1982, ApJ, 263, 835, doi: 10.1086/160554
  * Schneider & Shkolnik (2018) Schneider, A. C., & Shkolnik, E. L. 2018, AJ, 155, 3, 122, 16 pp. (2018)., 155, doi: 10.3847/1538-3881/aaaa24
  * Segura et al. (2005) Segura, A., Kasting, J. F., Meadows, V., et al. 2005, Astrobiology, 5, 6, pp. 706-725., 5, 706, doi: 10.1089/ast.2005.5.706
  * Segura et al. (2010) Segura, A., Walkowicz, L., Meadows, V., Kasting, J., & Hawley, S. 2010, Astrobiology, 10, 7, pp. 751-771., 10, 751, doi: 10.1089/ast.2009.0376
  * Shkolnik & Barman (2014) Shkolnik, E. L., & Barman, T. S. 2014, AJ, 148, 4, 64, 14 pp. (2014)., 148, doi: 10.1088/0004-6256/148/4/64
  * Skrutskie et al. (2006) Skrutskie, M. F., Cutri, R. M., Stiening, R., et al. 2006, AJ, 131, 1163, doi: 10.1086/498708
  * Spada & Lanzafame (2020) Spada, F., & Lanzafame, A. C. 2020, A&A, 636, A76, doi: 10.1051/0004-6361/201936384
  * Strassmeier et al. (2000) Strassmeier, K., Washuettl, A., Granzer, T., Scheck, M., & Weber, M. 2000, A&AS, 142, 275, doi: 10.1051/aas:2000328
  * Tian et al. (2014) Tian, F., France, K., Linsky, J. L., Mauas, P. J. D., & Vieytes, M. C. 2014, Earth and Planetary Science Letters, 385, 22, doi: 10.1016/j.epsl.2013.10.024
  * Tilipman et al. (2021) Tilipman, D., Vieytes, M., Linsky, J. L., Buccino, A. P., & France, K. 2021, ApJ, 909, 61, doi: 10.3847/1538-4357/abd62f
  * Tsurutani et al. (2003) Tsurutani, B. T., Gonzalez, W. D., Lakhina, G. S., & Alex, S. 2003, Journal of Geophysical Research (Space Physics), 108, 1268, doi: 10.1029/2002JA009504
  * Tu et al. (2015) Tu, L., Johnstone, C. P., Güdel, M., & Lammer, H. 2015, A&A, 577, L3, doi: 10.1051/0004-6361/201526146
  * van der Walt et al. (2011) van der Walt, S., Colbert, S. C., & Varoquaux, G. 2011, Computing in Science and Engineering, 13, 22, doi: 10.1109/MCSE.2011.37
  * Wright et al. (2011) Wright, N. J., Drake, J. J., Mamajek, E. E., & Henry, G. W. 2011, ApJ, 743, 48, doi: 10.1088/0004-637X/743/1/48
  * Wright et al. (2018) Wright, N. J., Newton, E. R., Williams, P. K. G., Drake, J. J., & Yadav, R. K. 2018, MNRAS, 479, 2351, doi: 10.1093/mnras/sty1670
  * Zerkle et al. (2012) Zerkle, A. L., Claire, M. W., Domagal-Goldman, S. D., Farquhar, J., & Poulton, S. W. 2012, Nature Geoscience, 5, 359, doi: 10.1038/ngeo1425


Tsu-Yuan Hsu1∗, Chen-An Li2∗, Tung-Yu Wu3, Hung-yi Lee4

# Model Extraction Attack against Self-supervised Speech Models

###### Abstract

Self-supervised learning (SSL) speech models generate meaningful
representations of given clips and achieve incredible performance across
various downstream tasks. Companies can provide services of these models by
building APIs. However, each of these APIs may suffer a model extraction
attack (MEA), which refers to an adversary stealing the model functionality
with limited query access. In this work, we propose an MEA framework against
the SSL speech models. Our MEA framework learns multiple output
representations of given clips to extract the target SSL speech models. We
demonstrate various selection methods on speech corpus to construct limited
query access. We also study the MEA on different speech corpus. We evaluate
the effectiveness of our MEA framework on four diverse downstream tasks. To
our knowledge, this is the first attempt to steal a large-scale speech model.

Index Terms: Self-supervised learning, speech representation learning, model
extraction attack ††*equal contributions

##  1 Introduction

Recent advances in self-supervised learning (SSL) speech models [1, 2, 3]
build meaningful representations of speech and achieve incredible performance
in many tasks [4]. Regarding the current SSL-based natural language processing
APIs, such as official GPT-3 [5] APIs, which provide services for generating
data embeddings, it can be expected that SSL speech-processing APIs would also
come into sight in the future. This kind of APIs take text or speech provided
by users as input and generate corresponding feature representations for the
training of downstream models.

However, each of these APIs may suffer a model extraction attack (MEA), which
refers to an adversary stealing the model functionality by limited query
access. MEA has posed a non-negligible threat to online deep learning
applications. Previous work [6] has shown that the adversary may extract
models used in remote APIs simply by querying them. Since the training of
models and the collection of datasets may have cost a tremendous amount of
time and money, this kind of attack causes sizeable financial losses to
victimized companies. Hence, it is an urgent task for researchers to study how
the adversary may perform the attack.

MEAs against different classes of machine learning and deep learning models
are broadly studied. Through some direct queries to the remote API, simple
regression models and multilayer perceptrons (MLPs) can be easily stolen [6].
Convolutional neural networks (CNNs) are also vulnerable to the attack [7, 8].
[7] randomly selects images on hand, queries the API with selected images to
fetch fake labels, and utilizes the image-label pairs to train the local
surrogate model. Knockoff Nets [8] adopts reinforcement learning to actively
sample images, which surpasses the random-selection approach. Recurrent Neural
Networks (RNNs) have also been studied to be attacked. [9] studies on attacks
based on features of RNNs and LSTMs for classification and regression tasks.
Advanced models such as graph neural networks (GNNs) have also been examined
[10, 11]. For instance, [10] demonstrates that, after collecting query graphs,
a surrogate model can be effectively learned by minimizing RMSE loss between
the remote GNN-based API's and its graph responses. Besides model-type-
specific extraction attacks, there are also some works [12, 13] committed to
stealing certain information of remote APIs. Model hyperparameters have been
pointed out as a potential target, and a framework is proposed to verify the
feasibility of hyperparameter extraction [12]. Metamodel methods [13] learn a
classifier to predict model attributes, such as model architectures, adopted
optimizers, and types of training datasets.

Large-scale SSL models [1, 2, 3, 14, 15, 16] are more critical potential
targets of MEA since the SSL-model-based APIs can serve as a powerful feature
extractor to generate representations of input data that help users implement
various applications such as text question answering (QA) with BERT [14] and
automatic speech recognition (ASR) with wav2vec 2.0 [1]. The training and
fine-tuning of an SSL model require much time and effort. However, the MEAs
against SSL models are still underexplored. Though there have been some works
[17, 18, 19, 20] investigating MEA against text models, to our knowledge,
approaches for speech models have not been discussed. Moreover, current text-
model-based methods still exist some restrictions. [17, 18, 19] apply random
sampling strategy to select query data, while [20] presents a hybrid strategy
that integrates algebraic-based and learning-based methods. Nonetheless, they
all assume attackers have access to output logits of downstream tasks and that
each time the attack merely focuses on stealing one task. ††Note that this
work aims to capture the research community’s attention to potential issues in
terms of speech-based SSL APIs, rather than simply an attempt to conduct MEA.
With this work, we anticipate more discussions in this field to help build a
more robust and comprehensive ecosystem of speech-based Machine Learning as a
Service (MLaaS).

In this work, we propose and implement the MEA against SSL speech models. In
particular, we design several clip-selection methods that identify informative
data. The selected clips are further used as queries to the victim model to
get corresponding representations for the local surrogate model's supervised
training. This enables our surrogate model to approximate the victim model's
performances with only a small number of clips, i.e., queries. We demonstrate
MEA on various selection methods and different speech corpus. Specifically,
four diverse downstream tasks are conducted in Section 3 to evaluate the
effectiveness of our proposed clip selection methods and the whole extraction
pipeline. To our knowledge, this is the first attempt to steal a large-scale
speech model. Furthermore, since our framework's presented active-sampling
methods only need access to data embeddings instead of logits, we can extract
the remote SSL speech model directly rather than just a downstream task.

Figure 1: Illustration of our model extraction attack framework.

##  2 Methods

In this paper, the victim model refers to the model being queried, such as the
APIs mentioned in Section 1. The pre-trained surrogate model refers to the
model pre-trained on unlabeled corpus and the extracted surrogate model refers
to the model after performing model extraction on the victim model. We assume
the requests to the victim model are limited, and the knowledge of the victim
model's architecture and its training corpus is lacking. The query limitation
is defined as the total length of waveforms. The information returned by the
victim model includes representations of input data.

As shown in Figure 1, our model extraction attack process includes several
steps:

  1. (1)

Pre-train a model on unlabeled corpus $\mathcal{X}$ to get the Pre-trained
Surrogate.

  2. (2)

An active selection method is applied to sample a small portion of clips from
dataset $\mathcal{X}$ to form a subset $\mathcal{X_{S}}$. The sampling process
is done until the total length of the waveforms in $\mathcal{X_{S}}$ is no
less than the preset length limitation $H$.

  3. (3)

The Pre-trained Surrogate is trained with $\mathcal{X_{S}}$ and the obtained
Victim's representations to perform the model extraction, i.e., steal the
functionality of the Victim to get the Extracted Surrogate.

###  2.1 Selection Methods

In this section, we elaborate on several proposed clip selection methods used
to construct $\mathcal{X_{S}}$ for model extraction.

####  2.1.1 SSL Pre-training Loss Selection

We state that SSL pre-training loss can serve as a good metric to sample
waveforms. Clips with high pre-training loss are regarded as hard samples
which baffle the current surrogate model and are thus worth engaging in the
afterward teacher-student supervised training. As a result, we evaluate the
dataset $\mathcal{X}$ on the pre-trained surrogate model and calculate each
waveform's pre-training loss. The dataset $\mathcal{X_{S}}$ iteratively
samples waveform with the highest loss in $\mathcal{X}$ until its total clip
length reaches $H$.

####  2.1.2 Content-based Selection

A selection approach based on the acoustic content of speech clips is also
proposed. We argue that, with the knowledge of each recording's acoustic
information, it is possible to sample a small portion of clips that represents
the overall distribution of the whole corpus. To achieve this, we first
leverage the pre-trained surrogate model to generate all the waveform
representations. Secondly, each waveform timestamp's token is determined by
feeding its representation into a clustering model fit on 10% of the
representations, with consecutive same class removed. We then take these
tokens as the content of a speech clip. Finally, we iteratively sample the
corpus with the farthest point sampling (FPS) [21]. To be specific, after
randomly picking the first clip, we calculate its token-based trigram Jaccard
distance to all other clips:

$\displaystyle D_{x,y}=1-J(X,Y)=1-\frac{|X\cap Y|}{|X\cup Y|},$ (1)

where $x$ and $y$ are two distinct clips (a sampled clip and an unsampled one
in our case), while $X$ and $Y$ are their token trigram sets. Specifically,
each element in the set is a set containing three consecutive tokens generated
by the clustering model. $J(X,Y)$ is the Jaccard similarity. Among all
unselected clips, we choose the one with the farthest distance (lowest
similarity) to the previously-selected clip as the second sampled clip. If
there have been multiple clips already sampled, an unsampled clip's distance
to them is defined as the Jaccard distance to its nearest sampled clip. The
FPS process is repeated until the total length of sampled clips is no less
than $H$.

####  2.1.3 Transcription-based Selection

In this method, we select clips with different content. We assume that every
speech in the dataset has its corresponding transcription. We utilize a pre-
trained language model to generate [CLS] token embeddings for speech
transcriptions. Next, a clustering model is fit on [CLS] token embeddings to
get clustering labels. Finally, we evenly select the corresponding clips in
$\mathcal{X}$ from each cluster to increase the data diversity into
$\mathcal{X}_{s}$.

Table 1: Evaluation results with querying dataset LibriSpeech 960-hour for KS,
IC, and ER in accuracy and SD in diarization error rate. The first three rows
are w/o any query. BaselineR refers to performing model extraction on Random
Surrogate w/ random selection. BaselineP, Transcription, SSL, and Content
refer to the Extracted Surrogate w/ random selection and the selection methods
elaborated in Section 2. ToplineH and ToplineW refer to the Extracted
Surrogate w/ unlimited queries to the HuBERT Base and WavLM Base+,
respectively. Best result among BaselineR, BaselineP, Transcription (if any),
SSL, and Content of each experiment is marked in bold.

gray!20 |  | KS (Acc $\uparrow$) | IC (Acc $\uparrow$) | ER (Acc $\uparrow$) | SD (DER $\downarrow$)
---|---|---|---|---|---
Pre-trained Surrogate | 92.41 | 77.91 | 57.7 | 7.86
HuBERT Base | 96.30 | 98.34 | 64.92 | 5.88
WavLM Base+ | 97.37 | 99.00 | 68.65 | 3.50
ToplineH | 95.85 | 94.83 | 62.74 | 6.76
ToplineW | 96.33 | 95.68 | 64.02 | 6.72
gray!20Method | Victim Model | 0.1h | 1h | 10h | 0.1h | 1h | 10h | 0.1h | 1h | 10h | 0.1h | 1h | 10h
BaselineR | HuBERT Base | 83.47 | 89.65 | 94.26 | 63.67 | 79.73 | 87.29 | 56.41 | 57.72 | 60.39 | 11.02 | 10.05 | 7.56
BaselineP | HuBERT Base | 92.05 | 94.16 | 94.35 | 75.43 | 87.43 | 88.98 | 56.91 | 60.28 | 60.06 | 9.15 | 7.64 | 7.18
Transcription | HuBERT Base | 92.11 | 93.57 | 94.22 | 76.19 | 84.29 | 90.17 | 58.68 | 59.78 | 61.48 | 9.61 | 7.63 | 7.61
SSL | HuBERT Base | 93.12 | 95.30 | 95.07 | 79.09 | 88.74 | 90.43 | 59.34 | 60.03 | 62.11 | 9.14 | 8.32 | 7.16
Content | HuBERT Base | 92.08 | 94.45 | 95.29 | 77.54 | 88.72 | 91.27 | 57.99 | 60.36 | 61.49 | 9.28 | 7.80 | 6.93
BaselineR | WavLM Base+ | 81.50 | 88.93 | 93.02 | 60.77 | 78.01 | 86.45 | 55.18 | 58.00 | 61.55 | 12.46 | 10.48 | 7.56
BaselineP | WavLM Base+ | 93.22 | 93.51 | 94.35 | 78.30 | 87.77 | 91.48 | 59.41 | 61.56 | 61.95 | 9.62 | 7.31 | 7.52
SSL | WavLM Base+ | 93.67 | 94.68 | 95.46 | 82.92 | 90.11 | 92.33 | 59.87 | 61.78 | 63.71 | 8.96 | 7.74 | 7.23
Content | WavLM Base+ | 93.18 | 94.55 | 95.13 | 79.41 | 90.4 | 92.17 | 58.84 | 62.29 | 62.70 | 8.74 | 7.77 | 6.84

###  2.2 Model Extraction

In this paper, we adopt the objective function of DistilHuBERT [22] to perform
model extraction. We assume that the victim model returns multiple
representations per query because weighted-sum multiple hidden states of the
pre-train model could significantly improve the downstream performance [4].
The surrogate model is followed by multiple separate prediction heads which
learn the victim model's representations from different layers.

Given victim model's $n$-th output representation $h^{(n)}$ and the prediction
head vector learn from victim model’s $n$-th output $\hat{h}^{(n)}$, The
objection function $\mathcal{L}_{\text{extract}}$ can be shown as follows:

$\displaystyle\mathcal{L}_{\text{extract}}^{(n)}$
$\displaystyle=\mathcal{L}_{cos}^{(n)}+\mathcal{L}_{l1}^{(n)}$
$\displaystyle=\sum_{t=1}^{T}\left[-\log\sigma\left(\cos\left(h_{t}^{(n)},\hat{h}_{t}^{(n)}\right)\right)+\frac{1}{D}\lVert
h_{t}^{(n)}-\hat{h}_{t}^{(n)}\rVert_{1}\right]$ (2)
$\displaystyle\mathcal{L}_{\text{extract}}$ $\displaystyle=\sum_{n\in
N}\mathcal{L}_{\text{extract}}^{(n)}=\sum_{n\in
N}\left[\mathcal{L}_{cos}^{(n)}+\mathcal{L}_{l1}^{(n)}\right]$ (3)

where $t\in[T]$, $T$ is the number of timestamps, $N$ is the set of the
layers' indices, $\sigma(\cdot)$ denotes sigmoid function, $cos(\cdot,\cdot)$
denotes cosine similarity function, $D$ is the feature dimension of the
representation.

##  3 Experiments

###  3.1 Experimental Setup

Experiments are implemented with s3prl v0.3.4 and fairseq [23] 0.12.2. We use
fairseq for pre-training our surrogate model. For s3prl, we use it for
performing model extraction and evaluation.

####  3.1.1 Data and Preprocessing

We use LibriSpeech 960-hour [24] to pre-train the surrogate model and as the
querying dataset to the victim model in our main results. Extensive
experiments are conducted using Wall Street Journal [25] and Aishell-1 [26] as
querying datasets. Considering the victim model should have an instance-wise
querying-length constraint, the maximum input sequence length is set to 15.6
seconds, and each clip longer than 15.6s is split. Also, we set the minimum
waveform length to 2s, which means the clip less than 2 seconds will be
dropped.

####  3.1.2 Victim Model

We consider HuBERT Base and WavLM Base+ as the victim models. Each of them has
a 12-layer transformer encoder. Features from different layers contain various
information [3, 22], such as speaker-related, semantic, and content
information. Therefore, we assume the victim model returns the representations
of each transformer layer, and the 4${}^{\text{th}}$, 8${}^{\text{th}}$, and
12${}^{\text{th}}$-layer representations are used to perform the model
extraction.

The pre-training dataset of HuBERT Base is exactly the same as our querying
dataset, i.e. LibriSpeech 960-hour while that of WavLM Base+ is a 94k-hour
dataset, implying the overlap with our querying dataset is less than 1.1%. We
assume the latter one to be more realistic. That is, the API (victim model) is
pre-trained on a large-scale dataset and should cope with speech clips from
various domains well. However, we only have a small piece of the dataset, and
we intend to query the victim model as efficiently as possible.

####  3.1.3 Surrogate Model

Our surrogate model is initialized as a 7-layer CNN extractor following a
2-layer transformer encoder, called Random Surrogate. HuBERT pre-training is
applied on Random Surrogate with LibriSpeech 960-hour to obtain Pre-trained
Surrogate. The first iteration is trained for 250k steps by the MFCC clustered
labels. The second iteration is trained for 400k steps by the clustered labels
generated from the $1^{\text{st}}$ transformer encoder layer features of
first-iteration Pre-trained Surrogate.

In the selection stage, the corresponding features of sampled clips are
retrieved by querying the victim model, where the clips are obtained by either
random, pre-training loss, content-based, or transcription-based selection. We
adopt the k-means model [27] as the clustering model used for content- and
transcription-based selection where the number of clusters is 250. For SSL
pre-training loss selection, we use the self-supervised cluster-prediction
loss of HuBERT. For transcription-based selection, we use RoBERTa [16] as our
language model.

In the model extraction stage, our surrogate model (Random Surrogate or Pre-
trained Surrogate) is trained on the clip-feature pairs for $10k\times H$
steps with batch size 24, where $H$ is the hour of sampled clips and
$H=0.1,1,10$ are considered in this paper. The learning rate linearly
increases to 0.0002 in the first 7% of the total steps and then linearly
decreases to zero in the left steps.

###  3.2 Evaluation

We use SUPERB [4] benchmark to evaluate our surrogate model. In SUPERB, the
surrogate model is frozen and a trainable linear layer is adopted to weigh
each layer's output features of our surrogate model, including one CNN
extractor's, two transformer encoder layers', and three prediction heads'
outputs. The downstream models then use the weighted feature as the input. In
this paper, four downstream tasks, including keyword spotting (KS) [28],
intent classification (IC) [29], emotion recognition (ER) [30], and speaker
diarization (SD) [31], are selected to evaluate the surrogate model.

Table 2: Performances of SID and SD with victim model WavLM Base+. Results
better than Pre-trained Surrogate (65.80 for SID and 7.86 for SD) are marked
in bold.

| SID | SD
---|---|---
Method | 0.1h | 1h | 10h | 0.1h | 1h | 10h
BaselineP | 55.17 | 65.90 | 71.42 | 9.62 | 7.31 | 7.52
Most Speakers | 56.55 | 64.76 | 71.55 | 9.83 | 8.35 | 7.74

(a) Accuracy

(b) Agreement

Figure 2: Performances of BaselineP, Content, SSL, and Topline on KS with
victim model WavLM Base+. Agreement refers to the prediction consistency
between the victim model and the surrogate model.

###  3.3 Main Results

In Table 1, BaselineP shows an overall-better performance compared to
BaselineR, especially in the highly low-resource setting, which indicates the
effectiveness of pre-training the surrogate model. Note that ``low-resource''
in this paper refers to the limitation on the number of queries.

For the proposed selection methods discussed in Section 2.1, Transcription has
a similar performance compared to BaselineP. This indicates that selecting the
queries based on speech transcriptions is not effective. From Table 1 and
Figure 2, though there’s still room for improvements, our methods have
generally outperformed the random selection baseline on KS, IC, and ER, making
a step forward in efficient data selection and MEA. Note that similar
agreement results as Figure 2(b) can be obtained on IC and
ER.${}^{\text{\ref{note2}}}$

From Table 1, we observed that all methods and baselines suffer performance
degradation in low-resource situations on SD. Therefore, we conducted
extensive experiments on another speaker task: speaker identification (SID)
[32], and provide Most Speakers, referring to the most-speaker selection,
which samples as many speakers as possible, to tackle speaker tasks. The
results are shown in Table 2111Only part of the results are shown due to page
limitation.. However, the performance still does not improve under the low-
resource setting.

###  3.4 Mismatched Querying Datasets

Table 3: Performances of SD with the victim model HuBERT Base. The querying
datasets are denoted in parentheses.

|  SD (Aishell-1) |  SD (WSJ)
---|---|---
Method | 0.1h | 1h | 10h | 0.1h | 1h | 10h
BaselineP | 9.09 | 8.25 | 7.21 | 8.55 | 8.44 | 7.80
SSL | 8.31 | 7.46 | 6.90 | 9.45 | 7.73 | 7.88
Content | 9.21 | 8.20 | 6.88 | 9.50 | 7.82 | 7.93

(a) Aishell-1

(b) WSJ

Figure 3: IC performances of Content, SSL, and BaselineP with querying dataset
Aishell-1 and WSJ.

The checkpoints of the pre-trained models are often released while the pre-
training datasets are not publicly available due to commercial use, privacy
issue, etc. Therefore, we examine WSJ and AISHELL-1 as querying datasets to
simulate the situation that the pre-training dataset of Pre-trained Surrogate
is unknown. It is worth mentioning that AISHELL-1 is a Chinese corpus, which
means the surrogate model's pre-training dataset and our querying dataset are
in different languages.

As shown in Figure 3, there is no obvious performance difference between our
proposed methods and BaselineP (the same conclusion can be drawn on KS, ER,
and SD). From Table 1 and Table 3${}^{\text{\ref{note2}}}$, we observe that
extracting the models with the AISHELL-1 corpus achieves a slightly better
performance on SD than with the WSJ and the LibriSpeech corpus. On the other
hand, extracting the victim model with the LibirSpeech corpus usually
outperforms the other two corpora in other tasks, which means that performing
model extraction with the different datasets pre-trained on the surrogate
model may significantly affect the performance.

##  4 Conclusion and Future Works

This work makes the first attempt to conduct the model extraction attack
against SSL speech models. Experimental results on four diverse tasks in
SUPERB show that our proposed selection methods outperform the naive random
data selection. In the future, we expect to explore a more effective data
selection method and find a way to avoid ineffective data selection resulting
from the mismatch between pre-training and querying dataset as mentioned in
Section 3.4.

The authors would like to thank ISCA and the organising committees of past
INTERSPEECH conferences for their help and for kindly providing the previous
version of this template.

As a final reminder, the 5th page is reserved exclusively for references. No
other content must appear on the 5th page. Appendices, if any, must be within
the first 4 pages. The references may start on an earlier page, if there is
space.

## References

  * [1] A. Baevski, Y. Zhou, A. Mohamed, and M. Auli, ``wav2vec 2.0: A framework for self-supervised learning of speech representations,'' _Advances in neural information processing systems_ , vol. 33, pp. 12 449–12 460, 2020.
  * [2] W.-N. Hsu, B. Bolte, Y.-H. H. Tsai, K. Lakhotia, R. Salakhutdinov, and A. Mohamed, ``Hubert: Self-supervised speech representation learning by masked prediction of hidden units,'' _IEEE/ACM Transactions on Audio, Speech, and Language Processing_ , vol. 29, pp. 3451–3460, 2021.
  * [3] S. Chen, C. Wang, Z. Chen, Y. Wu, S. Liu, Z. Chen, J. Li, N. Kanda, T. Yoshioka, X. Xiao _et al._ , ``Wavlm: Large-scale self-supervised pre-training for full stack speech processing,'' _IEEE Journal of Selected Topics in Signal Processing_ , 2022.
  * [4] S. wen Yang, P.-H. Chi, Y.-S. Chuang, C.-I. J. Lai, K. Lakhotia, Y. Y. Lin, A. T. Liu, J. Shi, X. Chang, G.-T. Lin, T.-H. Huang, W.-C. Tseng, K. tik Lee, D.-R. Liu, Z. Huang, S. Dong, S.-W. Li, S. Watanabe, A. Mohamed, and H. yi Lee, ``SUPERB: Speech Processing Universal PERformance Benchmark,'' in _Proc. Interspeech 2021_ , 2021, pp. 1194–1198.
  * [5] T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell _et al._ , ``Language models are few-shot learners,'' _Advances in neural information processing systems_ , vol. 33, pp. 1877–1901, 2020.
  * [6] F. Tramèr, F. Zhang, A. Juels, M. K. Reiter, and T. Ristenpart, ``Stealing machine learning models via prediction $\\{$APIs$\\}$,'' in _25th USENIX security symposium (USENIX Security 16)_ , 2016, pp. 601–618.
  * [7] J. R. Correia-Silva, R. F. Berriel, C. Badue, A. F. de Souza, and T. Oliveira-Santos, ``Copycat cnn: Stealing knowledge by persuading confession with random non-labeled data,'' in _2018 International Joint Conference on Neural Networks (IJCNN)_. IEEE, 2018, pp. 1–8.
  * [8] T. Orekondy, B. Schiele, and M. Fritz, ``Knockoff nets: Stealing functionality of black-box models,'' in _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , 2019, pp. 4954–4963.
  * [9] T. Takemura, N. Yanai, and T. Fujiwara, ``Model extraction attacks on recurrent neural networks,'' _Journal of Information Processing_ , vol. 28, pp. 1010–1024, 2020.
  * [10] Y. Shen, X. He, Y. Han, and Y. Zhang, ``Model stealing attacks against inductive graph neural networks,'' in _2022 IEEE Symposium on Security and Privacy (SP)_. IEEE, 2022, pp. 1175–1192.
  * [11] B. Wu, X. Yang, S. Pan, and X. Yuan, ``Model extraction attacks on graph neural networks: Taxonomy and realisation,'' in _Proceedings of the 2022 ACM on Asia Conference on Computer and Communications Security_ , 2022, pp. 337–350.
  * [12] B. Wang and N. Z. Gong, ``Stealing hyperparameters in machine learning,'' in _2018 IEEE symposium on security and privacy (SP)_. IEEE, 2018, pp. 36–52.
  * [13] S. J. Oh, B. Schiele, and M. Fritz, ``Towards reverse-engineering black-box neural networks,'' in _Explainable AI: Interpreting, Explaining and Visualizing Deep Learning_. Springer, 2019, pp. 121–144.
  * [14] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, ``BERT: Pre-training of deep bidirectional transformers for language understanding,'' in _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_. Minneapolis, Minnesota: Association for Computational Linguistics, Jun. 2019, pp. 4171–4186. [Online]. Available: https://aclanthology.org/N19-1423
  * [15] Z. Yang, Z. Dai, Y. Yang, J. Carbonell, R. R. Salakhutdinov, and Q. V. Le, ``Xlnet: Generalized autoregressive pretraining for language understanding,'' _Advances in neural information processing systems_ , vol. 32, 2019.
  * [16] Y. Liu, M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, and V. Stoyanov, ``Roberta: A robustly optimized bert pretraining approach,'' _arXiv preprint arXiv:1907.11692_ , 2019.
  * [17] K. Krishna, G. S. Tomar, A. P. Parikh, N. Papernot, and M. Iyyer, ``Thieves on sesame street! model extraction of bert-based apis,'' in _International Conference on Learning Representations_ , 2020. [Online]. Available: https://openreview.net/forum?id=Byl5NREFDr
  * [18] C. Chen, X. He, L. Lyu, and F. Wu, ``Killing one bird with two stones: Model extraction and attribute inference attacks against bert-based apis,'' _arXiv e-prints_ , pp. arXiv–2105, 2021.
  * [19] X. He, L. Lyu, L. Sun, and Q. Xu, ``Model extraction and adversarial transferability, your BERT is vulnerable!'' in _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_. Online: Association for Computational Linguistics, Jun. 2021, pp. 2006–2012. [Online]. Available: https://aclanthology.org/2021.naacl-main.161
  * [20] S. Zanella-Beguelin, S. Tople, A. Paverd, and B. Köpf, ``Grey-box extraction of natural language models,'' in _International Conference on Machine Learning_. PMLR, 2021, pp. 12 278–12 286.
  * [21] Y. Eldar, M. Lindenbaum, M. Porat, and Y. Y. Zeevi, ``The farthest point strategy for progressive image sampling,'' _IEEE Transactions on Image Processing_ , vol. 6, no. 9, pp. 1305–1315, 1997.
  * [22] H.-J. Chang, S.-w. Yang, and H.-y. Lee, ``Distilhubert: Speech representation learning by layer-wise distillation of hidden-unit bert,'' in _ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_. IEEE, 2022, pp. 7087–7091.
  * [23] M. Ott, S. Edunov, A. Baevski, A. Fan, S. Gross, N. Ng, D. Grangier, and M. Auli, ``fairseq: A fast, extensible toolkit for sequence modeling,'' in _Proceedings of NAACL-HLT 2019: Demonstrations_ , 2019.
  * [24] V. Panayotov, G. Chen, D. Povey, and S. Khudanpur, ``Librispeech: an asr corpus based on public domain audio books,'' in _2015 IEEE international conference on acoustics, speech and signal processing (ICASSP)_. IEEE, 2015, pp. 5206–5210.
  * [25] D. B. Paul and J. Baker, ``The design for the wall street journal-based csr corpus,'' in _Speech and Natural Language: Proceedings of a Workshop Held at Harriman, New York, February 23-26, 1992_ , 1992.
  * [26] H. Bu, J. Du, X. Na, B. Wu, and H. Zheng, ``Aishell-1: An open-source mandarin speech corpus and a speech recognition baseline,'' in _2017 20th conference of the oriental chapter of the international coordinating committee on speech databases and speech I/O systems and assessment (O-COCOSDA)_. IEEE, 2017, pp. 1–5.
  * [27] S. Lloyd, ``Least squares quantization in pcm,'' _IEEE transactions on information theory_ , vol. 28, no. 2, pp. 129–137, 1982.
  * [28] P. Warden, ``Speech commands: A public dataset for single-word speech recognition.'' _Dataset available from http://download.tensorflow.org/data/speech_commands_v0.01.tar.gz_ , 2017.
  * [29] L. Lugosch, M. Ravanelli, P. Ignoto, V. S. Tomar, and Y. Bengio, ``Speech Model Pre-Training for End-to-End Spoken Language Understanding,'' in _Proc. Interspeech 2019_ , 2019, pp. 814–818. [Online]. Available: http://dx.doi.org/10.21437/Interspeech.2019-2396
  * [30] C. Busso, M. Bulut, C.-C. Lee, A. Kazemzadeh, E. Mower, S. Kim, J. N. Chang, S. Lee, and S. S. Narayanan, ``Iemocap: Interactive emotional dyadic motion capture database,'' _Language resources and evaluation_ , vol. 42, pp. 335–359, 2008.
  * [31] J. Cosentino, M. Pariente, S. Cornell, A. Deleforge, and E. Vincent, ``Librimix: An open-source dataset for generalizable speech separation,'' _arXiv preprint arXiv:2005.11262_ , 2020.
  * [32] A. Nagrani, J. S. Chung, W. Xie, and A. Zisserman, ``Voxceleb: Large-scale speaker verification in the wild,'' _Computer Speech & Language_, vol. 60, p. 101027, 2020.