difference image, blinking between the 1995 and 2011 epochs, is available in
the online Journal. The animation is 12 seconds in duration with a cadence of
1 epoch/second; it is not annotated. The FITS files used to construct this
difference image are available as the data behind the figure.  Figure 19: A
difference image showing proper motions and changes in the north 2 (21) field
between the 2011 and 1995 epoch HST images in the portion of the northern lobe
of S106. The interval between images is 15.1 years. An animated version of
this difference image, blinking between the 1995 and 2011 epochs, is available
in the online Journal. The animation is 12 seconds in duration with a cadence
of 1 epoch/second; it is not annotated. The FITS files used to construct this
difference image are available as the data behind the figure.

##  Appendix C Spitzer and Herschel mid-IR Images

Figure 20 shows that the Hii region sits inside a roughly cylindrical,
$\sim$6′ (1.9 pc) long cavity bounded by warm dust and PAH emission at 3.6, to
70.0 $\mu$m. This cavity is more than a factor of two longer and wider than
the Hii region at visual, near-IR, and radio wavelengths. S106 IR is displaced
from the center of this cavity and located near its eastern edge.

Figure 20 shows a color composite image showing 8, 70, and 160 $\mu$m
emission. The cavity walls are clearly seen in the shorter wavelengths (blue
in Figure 20). Warm dust in the surrounding molecular cloud is traced by the
70 $\mu$m emission. Note the V-shaped filaments fanning away from the center
of the Hii region towards the north an northeast. Also note the bright bar
located about 1.3′ south of S106 IR. This feature is also seen at radio
frequencies and marks the southern end of the Hii region. The dark feature
located 30″ to 60″ west of S106 IR remains dark at all wavelengths to 500
$\mu$m. Thus, this feature may be a cavity in the cloud surrounding S106 that
is shielded from illumination by S106 IR by the dense molecular clump
containing the Class 0 source, S106 FIR (Furuya et al., 1999, 2000).

Figure 20: Left: A wide-field view showing the cavities surrounding the S106
Hii region in the Spitzer/IRAC 3.6 $\mu$m (blue), 4.6 $\mu$m (green), and 8.0
$\mu$m (red) images. The location of S106 IR is indicated by the red dot in
the brightest part of the image. Right: A wide-field view showing the cavities
surrounding the S106 Hii region in the Spitzer/IRAC 8.0 $\mu$m (blue),
Herschel/PACS 70 $\mu$m (green) and 160.0 $\mu$m (red) images.

##  Appendix D near-IR narrow-band images

Figure 21: Near-infrared image showing S106 in a narrow-band filter centered
on the $\lambda$=2.12 $\mu$m H2 emission line. Figure 22: Near-infrared image
showing S106 in a narrow-band filter centered at $\lambda$=2.13 $\mu$m off the
H2 emission line.

Figure 21 shows the narrow-band H2 image taken with the APO 3.5 meter
telescope at a wavelength of 2.12 $\mu$m with a filter pass-band of 0.5%.
Figure 22 shows the narrow-band H2\- off-line image taken with the APO 3.5
meter telescope at a wavelength of 2.13 $\mu$m with a filter pass-band of
0.5%. These images show some of the members of the S106 cluster.

Figure 23 shows a deep cut of Figure 7 that emphasizes the faint feature of H2
emission. Figure 24 shows a deep cut of Figure 6 that emphasizes the faint
feature of H2 emission.

Figure 23: Continuum-subtracted near-infrared image showing S106 in the
$\lambda$=2.12 $\mu$m H2 emission line. The intensity scale on this log
display is set to emphasize faint features. Figure 24: Continuum-subtracted
near-infrared image showing S106 in the $\lambda$=2.16 $\mu$m Br-$\gamma$
emission line. The intensity scale on this log display is set to emphasize
faint features.


# X-ray observation of the Roche-lobe filling white dwarf plus hot subdwarf
system ZTF J213056.71+442046.5

S. Mereghetti INAF – Istituto di Astrofisica Spaziale e Fisica Cosmica, Via A.
Corti 12, I-20133 Milano, Italy  N. La Palombara INAF – Istituto di
Astrofisica Spaziale e Fisica Cosmica, Via A. Corti 12, I-20133 Milano, Italy
T. Kupfer Department of Physics and Astronomy, Texas Tech University, PO Box
41051, Lubbock, TX 79409, USA  T.R. Marsh Department of Physics, University of
Warwick, Gibbet Hill Road, Coventry CV4 7AL, UK  C.M. Copperwheat
Astrophysics Research Institute, Liverpool John Moores University, IC2,
Liverpool Science Park, L3 5RF, UK  K. Deshmukh Department of Physics and
Astronomy, Texas Tech University, PO Box 41051, Lubbock, TX 79409, USA  P.
Esposito Scuola Universitaria Superiore IUSS Pavia, Palazzo del Broletto,
piazza della Vittoria 15, 27100 Pavia, Italy  INAF – Istituto di Astrofisica
Spaziale e Fisica Cosmica, Via A. Corti 12, I-20133 Milano, Italy  T.
Maccarone Department of Physics and Astronomy, Texas Tech University, PO Box
41051, Lubbock, TX 79409, USA  F. Pintore INAF – IASF Palermo, via U. La Malfa
153, 90146 Palermo, Italy  M. Rigoselli INAF – Istituto di Astrofisica
Spaziale e Fisica Cosmica, Via A. Corti 12, I-20133 Milano, Italy  L. Rivera
Sandoval Department of Physics and Astronomy, University of Texas Rio Grande
Valley, Brownsville, TX 78520, USA  A. Tiengo Scuola Universitaria Superiore
IUSS Pavia, Palazzo del Broletto, piazza della Vittoria 15, 27100 Pavia, Italy
INAF – Istituto di Astrofisica Spaziale e Fisica Cosmica, Via A. Corti 12,
I-20133 Milano, Italy  Istituto Nazionale di Fisica Nucleare, Sezione di
Pavia, via A. Bassi 6, I-27100 Pavia, Italy

(Received March 28, 2022 ; Revised April 18, 2022; Accepted April 19, 2022)

###### Abstract

ZTF J213056.71+442046.5 is the prototype of a small class of recently
discovered compact binaries composed of a white dwarf and a hot subdwarf that
fills its Roche-lobe. Its orbital period of only 39 min is the shortest known
for the objects in this class. Evidence for a high orbital inclination
($i$=86∘) and for the presence of an accretion disk has been inferred from a
detailed modeling of its optical photometric and spectroscopic data. We report
the results of an _XMM-Newton_ observation carried out on 2021 January 7. ZTF
J213056.71+442046.5 was clearly detected by the Optical Monitor, which showed
a periodic variability in the UV band (200-400 nm), with a light curve similar
to that seen at longer wavelengths. Despite accretion on the white dwarf at an
estimated rate of the order of 10-9 ${\rm M}_{\odot}$ yr-1, no X-rays were
detected with the EPIC instrument, with a limit of $\sim$1030 erg s-1 on the
0.2-12 keV luminosity. We discuss possible explanations for the lack of a
strong X-ray emission from this system.

White dwarf stars, B subdwarf stars, Compact binary stars, Stellar accretion.

††journal: ApJ

##  1 Introduction

A class of close binary systems consisting of Roche-lobe filling hot subdwarfs
with white dwarf (WD) companions has been recently identified thanks to
optical surveys devoted to the discovery of transient and variable sources
(Kupfer et al., 2020a). Hot subdwarfs (see Heber 2016, for a review) are low
mass stars ($\sim$0.3–0.8 ${\rm M}_{\odot}$), spectrally classified as sdO or
sdB based on their effective temperature above or below 38 kK, respectively.
They are believed to originate from the evolution of stars that have lost most
of their envelopes during the red-giant phase. One of the possible mechanisms
responsible for the loss of the massive hydrogen envelopes necessary to form
hot subdwarfs, is non-conservative mass transfer in a binary. This is
observationally supported by the large number of hot subdwarfs, particularly
of sdB type, found in close binary systems (Morales-Rueda et al., 2003;
Napiwotzki et al., 2004). Evolutionary models predict that most of the
companions of sdB stars in systems with orbital periods shorter than $\sim$10
days should be either late type main sequence stars or WDs (Han et al., 2002,
2003).

Hot subdwarfs with WD companions are particularly interesting because they
might be among the progenitors of type Ia supernovae (Iben & Tutukov, 1994)
and some of the nearby systems with ultrashort orbital period are expected to
be sources of gravitational waves detectable by LISA (Kupfer et al., 2018).
Furthermore, the determination of the mass and evolutionary stage of the two
components, as done, e.g., for KPD 0422+5421 (Orosz & Wade, 1999), KPD
1930+2752 (Geier et al., 2007), and HD 49798 (Mereghetti et al., 2009), can
shed light on the poorly known processes that take place during the common-
envelope evolutionary phase.

The hot subdwarfs with WD companions that exit the common-envelope phase at
orbital periods $P_{\rm orb}\lower
2.15277pt\hbox{$\;\buildrel<\over{\sim}\;$}2$ hr will reach contact while the
sdB is still burning He (Bauer & Kupfer, 2021). Due to the emission of
gravitational waves, the orbit will shrink until the sdB star fills its Roche-
lobe at an orbital period dependent on the evolutionary stage of the subdwarf
(Iben & Tutukov, 1991; Yungelson, 2008). The known population of sdB binaries
consists mostly of systems with orbital periods too large to start accretion
before the sdB turns into a WD (Kupfer et al., 2015). Until recently, only
five binaries of this kind with $P_{\rm orb}$$\;\buildrel<\over{\sim}\;$ 2 hr
were known (Vennes et al., 2012; Kupfer et al., 2017a, b; Pelisoli et al.,
2021; Kupfer et al., 2022). The sdBs in these systems do not fill their Roche-
lobe. X-ray observations of one of them (CD –30∘ 11223, $P_{\rm orb}$=70 min)
provided an upper limit on the X-ray luminosity indicating a wind mass-loss
rate from the sdB star lower than $\sim 3\times 10^{-13}$ ${\rm M}_{\odot}$
yr-1 (Mereghetti et al., 2014).

The first Roche-lobe filling subdwarf with a WD companion, ZTF
J213056.71+442046.5 (hereafter ZTF J2130, Kupfer et al. 2020b) was identified
in a search for periodic objects in a catalog of $\sim$ 40,000 hot subdwarf
candidates (Geier et al., 2019) and first discovered in a dedicated high-
cadence survey with the Zwicky Transient Facility (Bellm et al., 2019). ZTF
J2130 consists of a sdB star with mass $\sim 0.35{\rm M}_{\odot}$ and radius
0.13$\pm$0.1 $R_{\odot}$ and a WD of $\sim 0.56{\rm M}_{\odot}$. The orbital
period of this system, $P_{\rm orb}$ = 39 min (also independently discovered
by G.
Murawski111https://www.aavso.org/vsx/index.php?view=detail.top&oid=689728), is
the shortest of any known hot subdwarf binary. The radial velocity curve of
ZTF J2130 indicates a circular orbit with a velocity semi-amplitude $K$ = 429
$\pm$ 28 km s-1. The periodic variability is primarily caused by the
ellipsoidal modulation of the subdwarf, due to its tidal deformation under the
gravitational influence of the companion. However, the optical light-curve
cannot be described with a simple model composed of two detached stars.

Kupfer et al. (2020b) showed that the subdwarf in this system fills its Roche-
lobe and the WD is surrounded by an optically thick accretion disk of size
$\sim 0.14R_{\odot}$. The formation of a disk is not surprising in a Roche-
lobe filling system, and its presence suggests the possible emission of
accretion-powered X-rays. A short pointing (1 ks) carried out with the X-ray
Telescope on the _Neil Gehrels Swift Observatory_ in 2019 did not detect
X-rays (Rivera Sandoval et al., 2019), giving a flux upper limit of the order
of a few 10-13 erg cm-2 s-1. Here we report on a much deeper observation of
ZTF J2130 obtained with the _XMM-Newton_ satellite. In the following we use
the source distance of 1.29${}^{+0.06}_{-0.04}$ kpc, derived from the Gaia
EDR3 parallax (Bailer-Jones et al., 2021).

##  2 Data analysis and results

The field of ZTF J2130 was observed with the _XMM-Newton_ satellite for about
65 ks starting at 23:47 UT of 2021 January 6. We used the X-ray data obtained
with the EPIC instrument, comprising one pn (Strüder et al., 2001) and two MOS
cameras (Turner et al., 2001) operating in the 0.2–12 keV energy range, and
with the Optical Monitor (OM, Mason et al. 2001). The three EPIC cameras were
used in full frame mode with the thin optical filter. The OM provided data in
the UVW1 band ($\lambda_{eff}$=291 nm, $\Delta\lambda$= 83 nm) for the first
26.4 ks and in the UVW2 band ($\lambda_{eff}$=212 nm, $\Delta\lambda$=50 nm )
for the remaining part of the observation. All the data were processed with
Version 19.0.0 of the Science Analysis Software (sas)222https://xmm-
tools.cosmos.esa.int/external/xmm_user_support/ documentation/sas_usg/USG/.

###  2.1  EPIC X-ray data

The EPIC data were affected by periods of high background caused by soft
proton flares. Removing the corresponding time intervals resulted in net
exposure times of 31 ks for the pn and 37 ks for the MOS. We extracted images
from the three cameras in five different energy ranges (0.2-0.5, 0.5-1, 1-2,
2-4.5, and 4.5-12 keV), both using the cleaned data and the whole observation.
Many sources were clearly visible in these images, but none at the position of
ZTF J2130.

We performed a more detailed analysis by means of a source detection procedure
over the whole field of view using the data of the three cameras
simultaneously. This was done in the five energy bands given above using the
SAS task edetect_chain333https://xmm-
tools.cosmos.esa.int/external/sas/current/ doc/edetect_chain/index.html, as
described in the Appendix, and resulted in the detection of 51 sources (see
Table 2) above a threshold likelihood $L$ = 10, corresponding to a confidence
level of 99.9955 %. ZTF J2130 was not detected and we derived an upper limit
of 1.68 counts ks-1 on its count rate in the 0.2-12 keV energy range (with a
threshold likelihood $L$ = 8, i.e. 99.97 % confidence level). The limits in
the 0.2-0.5 and 0.5-4.5 keV ranges were 0.69 and 1.34 counts ks-1,
respectively.

###  2.2  OM ultraviolet data

ZTF J2130 was clearly detected by the OM, with average count rates of 22
counts s-1 in the UVW1 filter (291 nm) and 2.5 counts s-1 in the UVW2 filter
(212 nm) (Table 1). Figure 1 shows the OM light curves in the two filters
folded at the orbital period according to the ephemeris of Kupfer et al.
(2020b). In both filters, the orbital modulation is consistent with that seen
in the optical band, as it is shown by the superimposed solid lines
representing the profiles obtained in the g-band with HiPERCAM (Kupfer et al.,
2020b). Some small residuals remain. These are most likely to be indicating
factor of $\sim 2$ variability in the accretion disc flux.

A search of periodicities was performed using the OM-UVW1 data, but the
periodogram did not reveal any additional periodicity beyond the known orbital
period of 39 min.

Table 1: Ultraviolet and X-ray measurements of ZTF J2130 obtained with _XMM-Newton_. Errors in luminosity include the error in distance and are given at $1\sigma$. Magnitudes are not dereddened. The upper limit in the X-ray luminosity is valid for the blackbody and thermal bremsstrahlung models.  Instrument | Band | $\lambda_{eff}$ | Exposure | Mean Magnitude | Flux | Luminosity
---|---|---|---|---|---|---
|  | (nm) | (ks) | (Vega) |  $\times 10^{-11}$ erg cm-2 s-1 |  $\times 10^{33}$ erg s-1
OM | $UVW1$ | 291 | 26.4 | $13.849\pm 0.289$ | $3.16\pm 0.84$ | $6.3\pm 1.4$
OM | $UVW2$ | 212 | 37.32 | $13.809\pm 0.003$ | $3.27\pm 0.01$ | $6.51\pm 0.02$
EPIC | 0.2-12 keV | – | 31 (pn), 37 (MOS) | – | $<0.0012$ | $<0.0025$
Figure 1: The black points show the Optical Monitor light curves of ZTF J2130
in two UV bands folded at the orbital period. The red lines show for
comparison the g-band light curves. The second and fourth panels from top show
the ratios between the UV and g-band light curves. The WD is between the
observer and the sdB star at phase 0.5. Figure 2: Upper limits on the
luminosity of ZTF J2130 as a function of temperature, for an assumed blackbody
(red) or thermal bremsstrahlung (blue) spectrum. The solid and dashed lines
give the unabsorbed luminosity in the 0.2-12 keV range for $N_{\rm H}$=
1.7$\times 10^{21}$ cm-2 and 2.4$\times 10^{21}$ cm-2, respectively. The
dotted lines give the bolometric (0.001-100 keV) unabsorbed luminosity for
$N_{\rm H}$= 1.7$\times 10^{21}$ cm-2.

##  3 Discussion

Given the well known distance of ZTF J2130, the most important factors
affecting the conversion of the count rate upper limits derived in Section 2.1
to an upper limit on the luminosity are the spectral shape assumed for the
X-ray emission and the amount of interstellar absorption. Accreting WDs have
complex X-ray spectra, with different components depending on many factors,
including, e.g., the WD mass and magnetic field, the geometry of the accretion
flow, the binary orientation wrt the line of sight (see, e.g., Mukai 2017, for
a review). Optically thin thermal emission originates from the shocked plasma,
which in magnetic WDs is usually channeled in an accretion column. Depending
on the plasma temperature, significant line emission can be present, in
addition to the continuum bremsstrahlung flux. Part of the emission from the
accretion column can heat the star surface and will then be re-radiated as an
optically thick soft thermal component. Other spectral components can arise
from Compton reflection of the hardest X-rays, from the inner part of an
accretion disk, from the boundary layer, and from a hot corona above the disk.
Detailed models for many of these emission components have been developed, but
for our purpose it is sufficient to adopt two simple models, blackbody and
thermal bremsstrahlung, which provide an adequate phenomenological description
of the hard and soft components observed in accreting WDs.

The 3-D extinction maps of Green et al. (2019) give a reddening of
$E(g-r)$=0.18 at the distance of ZTF J2130. This corresponds to $A_{V}\sim$0.6
mag, which, using the relation between $A_{V}$ and interstellar absorption
derived by Bahramian et al. (2015), gives $N_{\rm H}$=1.7$\times 10^{21}$
cm-2. This value is consistent with the upper limit provided by the total
column density in this direction, $N_{\rm H}$=2.4$\times$1021 cm-2 (HI4PI
Collaboration et al., 2016).

Therefore, we computed the count rate to flux conversion factors for blackbody
and thermal bremsstrahlung models and two representative values of absorption
(the nominal and the maximum values, 1.7 and 2.4 $\times 10^{21}$ cm-2,
respectively). This was done using the response matrix for the EPIC instrument
(i.e. the sum of the pn and two MOS) computed for the position of ZTF J2130 in
the instrument field of view. The resulting upper limits on the luminosity are
plotted as a function of the spectral temperature in Fig. 2. Except for the
case of very soft spectral shapes, for which most of the emission falls
outside the energy range sampled by the EPIC instrument, the limits are quite
constraining and indicate a luminosity below $\sim 10^{30}$ erg s-1 for a wide
range of assumed temperature values.

By modeling the evolutionary history of ZTF J2130, Kupfer et al. (2020b)
estimated that the WD in this system is accreting at a rate of $\sim 10^{-9}$
${\rm M}_{\odot}$ yr-1. Taken at face value, this would imply an accretion-
powered luminosity of

$L_{acc}=G\frac{M}{R}\dot{M}=5\times 10^{33}\dot{M}_{\rm-9}~{}~{}{\rm
erg~{}s^{-1}},$ (1)

where G is the gravitational constant, $\dot{M}_{\rm-9}$ is the mass accretion
rate in units of 10-9 ${\rm M}_{\odot}$ yr-1, and we used the values of
$M$=0.545 ${\rm M}_{\odot}$ and $R$=9400 km appropriate for this WD. This
luminosity is orders of magnitude larger than the upper limit derived with
_XMM-Newton_ , but there are possible explanations for the lack of detectable
X-ray emission in ZTF J2130.

First, it must be considered that about half of the accretion power is
dissipated in the accretion disk, which is too cold to emit in the X-ray band.
Most of the disk luminosity occurs at IR/optical/UV wavelengths and is
outshined by the much brighter emission from the hot subdwarf present in this
system. The latter has an effective temperature of 42,400 K and a bolometric
luminosity of 41$\pm$9 $L_{\odot}$. From the absence of optical emission
lines, Kupfer et al. (2020b) concluded that the accretion disk contributes at
most 3% of the overall luminosity, which is consistent with the above estimate
of Lacc.

X-rays from accreting WDs can be emitted from the boundary layer between the
disk and the star surface or, in the case of intermediate polars, where the
magnetic field is sufficiently strong to channel the accretion flow, from the
shocked plasma in the accretion column. In this case, the maximum plasma
temperature $T$ in the post shock region is an increasing function of the WD
mass. A simple estimate (e.g. Frank et al. 2002) is

$T=\frac{3}{8}\frac{GM\mu m_{p}}{kR},$ (2)

which gives kT = 19 keV for ZTF J2130 (mp is the proton mass and we used a
mean molecular weight $\mu$ =0.615). Our luminosity upper limit for a thermal
bremsstrahlung spectrum of this temperature is compared in Fig. 3 with the
luminosities of a sample of intermediate polars observed in the hard X-ray
range (Suleimanov et al., 2019). It is clear that, even if the mass accretion
rate derived by Kupfer et al. (2020b) were overestimated by up to a factor
$\sim$100, the _XMM-Newton_ limit implies a luminosity much lower than that of
intermediate polars, that disfavors the presence of a magnetic WD in this
system.

Figure 3: Luminosity versus mass accretion rate in a sample of intermediate
polars studied by Suleimanov et al. (2019). The values predicted by eq. (1)
are shown by the blue and green lines for MWD=1 and 0.5 ${\rm M}_{\odot}$,
respectively. The red line shows the upper limit we derived on the X-ray
luminosity of ZTF J2130.

If the WD in ZTF J2130 has a low magnetic field, the available accretion power
is mainly released in the boundary layer between the disk and the star, which
might be optically thick, and from the heated WD surface. The resulting
spectrum would peak at soft X-ray/EUV energies and thus escape detection in
the energy range covered by the EPIC instrument. Considering the high
inclination of this system, it is also possible that the emitting region is
occulted by the accretion disk. In this respect, it is interesting to note the
anticorrelation between soft X-ray luminosity and inclination found in non-
magnetic cataclysmic variables by van Teeseling et al. (1996).

Finally, we note the possibility that the accretion rate derived from the
evolutionary model used by Kupfer et al. (2020b) has been overestimated. An
evolutionary phase with accretion rate at $\sim 10^{-9}$ ${\rm M}_{\odot}$
yr-1 is required to explain the current properties of this system, but it is
possible that this phase is now ending and the sdOB has already started to
shrink and underfill the Roche lobe.

##  4 Conclusions

ZTF J2130 is the WD with a hot subdwarf binary companion with the shortest
known orbital period. The evidence, derived from optical observations, for an
accretion disk around the WD, caused by Roche-lobe overflow of its sdB
companion, motivated a search for X-ray emission.

Our _XMM-Newton_ observation, despite reaching a flux limit more than two
orders of magnitude below the previously available one, could not reveal X-ray
emission. The luminosity upper limit is in the range $\sim$(0.5–2.5)$\times
10^{30}$ erg s-1, depending on the assumed spectral temperature. For the hard
Bremsstrahlung spectra typical of accreting magnetic WDs, this limit is
significantly below the luminosity expected from the mass accretion rate of
$\sim 10^{-9}$ ${\rm M}_{\odot}$ yr-1 deduced from the evolution model of this
binary. This suggests that ZTF J2130 contains a non-magnetic WD, that can have
a luminosity up to a few $10^{33}$ erg s-1, consistent with the theoretical
accretion rate, but characterized by a soft spectrum peaking in the EUV range
and thus undetectable by X-ray observatories. Alternative explanations for the
lack of observable X-rays are either intrinsic absorption in the system, due
to its high inclination ($i$=86∘), or an overestimate of the current accretion
rate implying that the Roche-lobe filling phase is nearly ending.

We acknowledge support via ASI/INAF Agreement n. 2019-35-HH and PRIN-MIUR 2017
UnIAM (Unifying Isolated and Accreting Magnetars, PI S. Mereghetti). TK
acknowledges support from the National Science Foundation through grant AST
#2107982, from NASA through grant 80NSSC22K0338 and from STScI through grant
HST-GO-16659.002-A. TRM acknowledges support from the UK’s Science and
Technology Facilities Council (STFC), grant number ST/T000406/1. This work is
based on data obtained with XMM-Newton, an ESA science mission with
instruments and contributions directly funded by ESA Member States and the USA
(NASA). This research has made use of the SIMBAD database, operated at CDS,
Strasbourg, France.

## References

  * Bahramian et al. (2015) Bahramian, A., Heinke, C. O., Degenaar, N., et al. 2015, MNRAS, 452, 3475, doi: 10.1093/mnras/stv1585
  * Bailer-Jones et al. (2021) Bailer-Jones, C. A. L., Rybizki, J., Fouesneau, M., Demleitner, M., & Andrae, R. 2021, AJ, 161, 147, doi: 10.3847/1538-3881/abd806
  * Bauer & Kupfer (2021) Bauer, E. B., & Kupfer, T. 2021, ApJ, 922, 245, doi: 10.3847/1538-4357/ac25f0
  * Bellm et al. (2019) Bellm, E. C., Kulkarni, S. R., Graham, M. J., et al. 2019, PASP, 131, 018002, doi: 10.1088/1538-3873/aaecbe
  * Frank et al. (2002) Frank, J., King, A., & Raine, D. J. 2002, Accretion Power in Astrophysics: Third Edition
  * Gaia Collaboration et al. (2016) Gaia Collaboration, Prusti, T., de Bruijne, J. H. J., et al. 2016, A&A, 595, A1, doi: 10.1051/0004-6361/201629272
  * Gaia Collaboration et al. (2021) Gaia Collaboration, Brown, A. G. A., Vallenari, A., et al. 2021, A&A, 649, A1, doi: 10.1051/0004-6361/202039657
  * Geier et al. (2007) Geier, S., Nesslinger, S., Heber, U., et al. 2007, A&A, 464, 299, doi: 10.1051/0004-6361:20066098
  * Geier et al. (2019) Geier, S., Raddi, R., Gentile Fusillo, N. P., & Marsh, T. R. 2019, A&A, 621, A38, doi: 10.1051/0004-6361/201834236
  * Green et al. (2019) Green, G. M., Schlafly, E., Zucker, C., Speagle, J. S., & Finkbeiner, D. 2019, ApJ, 887, 93, doi: 10.3847/1538-4357/ab5362
  * Han et al. (2003) Han, Z., Podsiadlowski, P., Maxted, P. F. L., & Marsh, T. R. 2003, MNRAS, 341, 669, doi: 10.1046/j.1365-8711.2003.06451.x
  * Han et al. (2002) Han, Z., Podsiadlowski, P., Maxted, P. F. L., Marsh, T. R., & Ivanova, N. 2002, MNRAS, 336, 449, doi: 10.1046/j.1365-8711.2002.05752.x
  * Heber (2016) Heber, U. 2016, PASP, 128, 082001, doi: 10.1088/1538-3873/128/966/082001
  * HI4PI Collaboration et al. (2016) HI4PI Collaboration, Ben Bekhti, N., Flöer, L., et al. 2016, A&A, 594, A116, doi: 10.1051/0004-6361/201629178
  * Høg et al. (2000) Høg, E., Fabricius, C., Makarov, V. V., et al. 2000, A&A, 355, L27
  * Iben & Tutukov (1994) Iben, I. J., & Tutukov, A. V. 1994, ApJ, 431, 264, doi: 10.1086/174484
  * Iben & Tutukov (1991) Iben, Icko, J., & Tutukov, A. V. 1991, ApJ, 370, 615, doi: 10.1086/169848
  * Kupfer et al. (2015) Kupfer, T., Geier, S., Heber, U., et al. 2015, A&A, 576, A44, doi: 10.1051/0004-6361/201425213
  * Kupfer et al. (2017a) Kupfer, T., van Roestel, J., Brooks, J., et al. 2017a, ApJ, 835, 131, doi: 10.3847/1538-4357/835/2/131
  * Kupfer et al. (2017b) Kupfer, T., Ramsay, G., van Roestel, J., et al. 2017b, ApJ, 851, 28, doi: 10.3847/1538-4357/aa9522
  * Kupfer et al. (2018) Kupfer, T., Korol, V., Shah, S., et al. 2018, MNRAS, 480, 302, doi: 10.1093/mnras/sty1545
  * Kupfer et al. (2020a) Kupfer, T., Bauer, E. B., Burdge, K. B., et al. 2020a, ApJ, 898, L25, doi: 10.3847/2041-8213/aba3c2
  * Kupfer et al. (2020b) Kupfer, T., Bauer, E. B., Marsh, T. R., et al. 2020b, ApJ, 891, 45, doi: 10.3847/1538-4357/ab72ff
  * Kupfer et al. (2022) Kupfer, T., Bauer, E. B., van Roestel, J., et al. 2022, ApJ, 925, L12, doi: 10.3847/2041-8213/ac48f1
  * Lasker et al. (2008) Lasker, B. M., Lattanzi, M. G., McLean, B. J., et al. 2008, AJ, 136, 735, doi: 10.1088/0004-6256/136/2/735
  * Mason et al. (2001) Mason, K. O., Breeveld, A., Much, R., et al. 2001, A&A, 365, L36, doi: 10.1051/0004-6361:20000044
  * Mereghetti et al. (2014) Mereghetti, S., La Palombara, N., Esposito, P., et al. 2014, MNRAS, 441, 2684, doi: 10.1093/mnras/stu773
  * Mereghetti et al. (2009) Mereghetti, S., Tiengo, A., Esposito, P., et al. 2009, Science, 325, 1222, doi: 10.1126/science.1176252
  * Morales-Rueda et al. (2003) Morales-Rueda, L., Maxted, P. F. L., Marsh, T. R., North, R. C., & Heber, U. 2003, MNRAS, 338, 752, doi: 10.1046/j.1365-8711.2003.06088.x
  * Mukai (2017) Mukai, K. 2017, PASP, 129, 062001, doi: 10.1088/1538-3873/aa6736
  * Napiwotzki et al. (2004) Napiwotzki, R., Karl, C. A., Lisker, T., et al. 2004, Ap&SS, 291, 321, doi: 10.1023/B:ASTR.0000044362.07416.6c
  * Orosz & Wade (1999) Orosz, J. A., & Wade, R. A. 1999, MNRAS, 310, 773, doi: 10.1046/j.1365-8711.1999.02989.x
  * Pelisoli et al. (2021) Pelisoli, I., Neunteufel, P., Geier, S., et al. 2021, Nature Astronomy, 5, 1052, doi: 10.1038/s41550-021-01413-0
  * Rivera Sandoval et al. (2019) Rivera Sandoval, L. E., Maccarone, T., & Murawski, G. 2019, The Astronomer’s Telegram, 12847, 1
  * Rosen et al. (2016) Rosen, S. R., Webb, N. A., Watson, M. G., et al. 2016, A&A, 590, A1, doi: 10.1051/0004-6361/201526416
  * Strüder et al. (2001) Strüder, L., Briel, U., Dennerl, K., & et al. 2001, A&A, 365, L18, doi: 10.1051/0004-6361:20000066
  * Suleimanov et al. (2019) Suleimanov, V. F., Doroshenko, V., & Werner, K. 2019, MNRAS, 482, 3622, doi: 10.1093/mnras/sty2952
  * Turner et al. (2001) Turner, M. J. L., Abbey, A., Arnaud, M., & et al. 2001, A&A, 365, L27, doi: 10.1051/0004-6361:20000087
  * van Teeseling et al. (1996) van Teeseling, A., Beuermann, K., & Verbunt, F. 1996, A&A, 315, 467
  * Vennes et al. (2012) Vennes, S., Kawka, A., O’Toole, S. J., Németh, P., & Burton, D. 2012, ApJ, 759, L25, doi: 10.1088/2041-8205/759/1/L25
  * Yungelson (2008) Yungelson, L. R. 2008, Astronomy Letters, 34, 620, doi: 10.1134/S1063773708090053

We performed the detection, characterization, and identification of the
several X-ray sources present in the EPIC images of the sky region around ZTF.
To this aim, we applied the SAS task edetect_chain, with the default
configuration and input parameters, using the images in five energy ranges
(0.2-0.5, 0.5-1, 1-2, 2-4.5, and 4.5-12 keV) of the three EPIC cameras
simultaneously. This resulted in the detection of 51 distinct sources (Fig.
4). For each of them we obtained the X-ray position and its uncertainty, the
detection likelihood (defined as L = -lnP, where P is the probability of a
spurious detection due to a Poissonian random fluctuation of the background)
and the determination of its point or extended nature. We derived the source
fluxes from the measured count rates assuming the same absorbed power-law
model (with photon index $\Gamma$ = 1.7 and hydrogen column density $N_{\rm
H}$= 3$\times 10^{20}$ cm-2) used in the 3XMM source catalogue (Rosen et al.,
2016).

Figure 4: EPIC image in the 0.2-12 keV energy range of the sky region around
ZTF J213056.71+442046.5 with the position of the 51 detected X-ray sources.
North is to the top, east to the left. The white circle indicates the position
of ZTF J2130.

We used the task eposcorr to correct the position of the detected sources,
through their cross-correlation with the Gaia EDR3 source catalogue (Gaia
Collaboration et al., 2016, 2021). The average RA and DEC correction offset
applied to the X-ray positions was 2.26 and 1.60 arcsec, respectively.

The last step of our analysis was the identification of each detected source
with a known optical counterpart, or at least the assessment of its stellar or
non-stellar nature. We performed this task with the cross-correlation with two
different optical catalogues. The first choice was again the Gaia EDR3
catalogue, which to date is the most complete and accurate optical catalogue.
For each X-ray source, we looked for all Gaia sources closer than three times
the uncertainty of the X-ray position: due to the high accuracy of the Gaia
positions, this approach gives us high confidence that no potential Gaia
counterpart is missed. In this way we found at least one candidate Gaia
counterpart for only 29 of the sources. Therefore, we integrated our analysis
by cross-correlating the X-ray sources (with the same criteria) also with the
Guide Star Catalogue (GSC, Lasker et al.
(2008))444http://gsss.stsci.edu/webservices/GSC2/WebForm.aspx, which also
provides a tentative classification of the source
nature555http://gsss.stsci.edu/Catalogs/GSC/GSC2/gsc23/gsc23_release_notes.htm#ClassificationCodes.
We found a candidate GSC counterpart for 11 of the X-ray sources without Gaia
counterparts. For 12 X-ray sources we found two or more candidate
counterparts, while 11 X-ray sources remained uncorrelated to any optical
counterpart.

In Table 2 we report the main parameters of the detected X-ray sources and of
their candidate counterparts, together with the proposed identification. For
each possible counterpart, we report the distance from the X-ray source, the
source class given in the GSC catalogue (Star, Non-Star, or Unclassified), the
magnitude type and value, and the estimated value and error (in logarithm) of
the X-ray-to-optical flux ratio. Three of the brightest sources (#10, #13, and
#17) can unambiguously be identified with a known bright optical source of the
Tycho catalogue (Høg et al., 2000).

Two sources (#8 and #14) are clearly extended. The analysis of the EPIC images
shows that they are clearly visible between 0.5 and $\simeq$ 5 keV, while
outside this energy range they are confused with the background signal. Since
for both objects there is no clear identification, we investigated their
spectrum in order to obtain additional information on their properties. In
both cases the count rate is very low (2.7$\times 10^{-2}$ and 7.8$\times
10^{-3}$ cts s-1, respectively), so the count statistics is limited. The
spectrum of source #8 is rather hard and extends up to 10 keV. It can be
equally well described with a power-law model with a photon index
$\Gamma\simeq$ 1.6, or with a collisionally-heated plasma emission model with
a high temperature (kT $\sim$ 10 keV). With both models the estimated flux (in
the energy range 0.2-10 keV) is $\simeq 10^{-13}$ erg cm-2 s-1. The high
temperature of the thermal model suggests that the identification with a
cluster of galaxies is unlikely. The hard power-law spectrum suggests a pulsar
wind nebula nature, but no pulsars at this positions are known. The spectrum
of source #14 is significantly softer: it can be described with either a
power-law model with $\Gamma\simeq$ 2 or with a plasma model with kT $\sim$ 5
keV. Therefore, in this case the estimated plasma temperature is consistent
with a cluster of galaxies or with a supernova remnant. However, the limited
count statistics prevents to clearly discriminate between a thermal or non-
thermal emission model. The estimated flux of this source is $\simeq 6\times
10^{-14}$ erg cm-2 s-1. In summary, based on the EPIC data, we cannot suggest
any identification of these extended sources with a known class of X-ray
sources.

Table 2: Main parameters of the 51 detected X-ray sources. The sources are
sorted by decreasing detection likelihood.

(1) | (2) | (3) | (4) | (5) | (6) | (7) | (8) | (9) | (10) | (11) | (12) | (13) | (14)
---|---|---|---|---|---|---|---|---|---|---|---|---|---
N | RAX | DECX | ERR | CR | Flux ($\times 10^{-14}$ | Detection | Extended | $d$ | Class | Magnitude | X/O ratio | Counterpart
n/a | ∘ | ∘ | ” | cts s-1 | erg cm-2 s-1) | likelihood | source | ” | n/a | Type | Value | Log10 |
1 | 322.9037 | 44.3588 | 0.2 | 0.119 $\pm$ 0.004 | 12.50 $\pm$ 0.38 | 3343.42 | NO | 0.5 | Star | GaiaG | 11.2 | -3.62 $\pm$ 0.05 | Gaia EDR3 1970952112669846528
|  |  |  |  |  |  |  |  |  |  |  |  | 1RXS J213137.1+442125
2 | 322.7730 | 44.4847 | 0.2 | 0.141 $\pm$ 0.004 | 22.50 $\pm$ 0.58 | 3254.26 | NO | 0.4 | Star | GaiaG | 14.4 | -2.07 $\pm$ 0.04 | Gaia EDR3 1970957197911088256
3 | 322.7798 | 44.3510 | 0.2 | 0.061 $\pm$ 0.002 | 6.25 $\pm$ 0.18 | 2269.45 | NO | 0.2 | - | GaiaG | 14.3 | -2.68 $\pm$ 0.01 | Gaia EDR3 1970951219316446848
4 | 322.7973 | 44.3185 | 0.4 | 0.024 $\pm$ 0.001 | 7.11 $\pm$ 0.37 | 537.44 | NO | 0.6 | Non-Star | SDSS_r | 22.4 | 1.38 $\pm$ 0.14 | GSC2 N0331212284370
5 | 322.6702 | 44.2730 | 0.4 | 0.022 $\pm$ 0.001 | 7.70 $\pm$ 0.46 | 400.99 | NO | - | - | - | - | - | ?
6 | 322.7772 | 44.2866 | 0.4 | 0.019 $\pm$ 0.001 | 4.79 $\pm$ 0.30 | 301.96 | NO | 0.5 | Non-Star | SDSS_g | 21.7 | 0.70 $\pm$ 0.10 | GSC2 N0331212271069
7 | 322.5740 | 44.1716 | 0.6 | 0.016 $\pm$ 0.002 | 9.42 $\pm$ 0.90 | 199.27 | NO | - | - | - | - | - | ?
8 | 322.8676 | 44.3626 | 1.0 | 0.073 $\pm$ 0.005 | 14.20 $\pm$ 1.05 | 194.75 | YES | 2.9 | Non-Star | SDSS_r | 21.4 | 1.30 $\pm$ 0.15 | GSC2 N0331212285756
9 | 322.5840 | 44.3718 | 0.5 | 0.023 $\pm$ 0.002 | 4.96 $\pm$ 0.49 | 183.53 | NO | - | - | - | - | - | ?
10 | 322.7726 | 44.2233 | 0.6 | 0.014 $\pm$ 0.001 | 1.33 $\pm$ 0.11 | 171.66 | NO | 1.2 | Star | GaiaG | 10.2 | -4.98 $\pm$ 0.11 | TYC 3195-1291-1
|  |  |  |  |  |  |  |  |  |  |  |  | (Gaia EDR3 1970937647223334784)
11 | 322.8890 | 44.5620 | 0.8 | 0.042 $\pm$ 0.005 | 23.70 $\pm$ 2.90 | 154.79 | NO | 1.1 | Star | GaiaG | 14.9 | -1.85 $\pm$ 0.14 | Gaia EDR3 1971050931274339456
12 | 322.8995 | 44.5525 | 0.9 | 0.021 $\pm$ 0.005 | 7.74 $\pm$ 1.92 | 112.35 | NO | 2.7 | Star | GaiaG | 18.6 | -0.87 $\pm$ 0.44 | Gaia EDR3 1971050720812613632
13 | 322.9422 | 44.2552 | 0.9 | 0.011 $\pm$ 0.001 | 0.98 $\pm$ 0.12 | 94.56 | NO | 0.9 | Non-Star | SDSS_i | 14.7 | -2.44 $\pm$ 0.29 | GSC2 N0331212310632
|  |  |  |  |  |  |  | 1.4 | Star | GaiaG | 11.4 | -4.62 $\pm$ 0.29 | TYC 3195-1009-1
|  |  |  |  |  |  |  |  |  |  |  |  | (Gaia EDR3 1970938781091225472)
|  |  |  |  |  |  |  | 2.0 | Non-Star | SDSS_g | 13.2 | -3.39 $\pm$ 0.29 | GSC2 N0331212271457
14 | 322.5631 | 44.4418 | 0.7 | 0.045 $\pm$ 0.005 | 9.83 $\pm$ 1.03 | 91.29 | YES | - | - | - | - | - | ?
15 | 322.7897 | 44.1830 | 0.8 | 0.013 $\pm$ 0.002 | 3.93 $\pm$ 0.51 | 75.79 | NO | 0.7 | Star | GaiaG | 19.8 | -0.68 $\pm$ 0.20 | Gaia EDR3 1970937372330018944
|  |  |  |  |  |  |  | 1.1 | - | GaiaG | 17.1 | -1.76 $\pm$ 0.06 | Gaia EDR3 1970937372341684864
16 | 322.6268 | 44.2773 | 0.7 | 0.010 $\pm$ 0.001 | 5.43 $\pm$ 0.61 | 70.91 | NO | 1.4 | Non-Star | SDSS_i | 21.8 | 1.15 $\pm$ 0.10 | GSC2 N0331212276990
|  |  |  |  |  |  |  | 2.0 | Unclassified | WISE_w1 | 15.7 | - | GSC2 N0331212380540
17 | 322.8324 | 44.1495 | 0.8 | 0.017 $\pm$ 0.002 | 6.91 $\pm$ 0.90 | 70.69 | NO | 0.8 | Non-Star | SDSS_r | 11.4 | -3.03 $\pm$ 0.16 | GSC2 N0331212325476
|  |  |  |  |  |  |  | 1.3 | Star | GaiaG | 11.2 | -3.89 $\pm$ 0.15 | TYC 3195-1810-1
|  |  |  |  |  |  |  |  |  |  |  |  | (Gaia EDR3 1970936375909299712)
|  |  |  |  |  |  |  | 2.2 | Star | GaiaG | 19.5 | -0.56 $\pm$ 0.15 | Gaia EDR3 1970936375899213440
18 | 322.6491 | 44.4884 | 1.1 | 0.014 $\pm$ 0.002 | 4.83 $\pm$ 0.65 | 66.51 | NO | 1.3 | Non-Star | SDSS_g | 21.9 | 0.78 $\pm$ 0.20 | GSC2 N0331212254829
19 | 322.8823 | 44.2828 | 0.9 | 0.009 $\pm$ 0.001 | 2.60 $\pm$ 0.35 | 62.60 | NO | 1.1 | Star | GaiaG | 16.8 | -2.07 $\pm$ 0.20 | Gaia EDR3 1970939537005448064
20 | 322.6726 | 44.2914 | 0.9 | 0.008 $\pm$ 0.001 | 0.76 $\pm$ 0.09 | 54.98 | NO | 2.1 | Star | GaiaG | 16.3 | -2.80 $\pm$ 0.27 | Gaia EDR3 1970944931484039552
21 | 322.7768 | 44.4750 | 0.9 | 0.010 $\pm$ 0.001 | 1.44 $\pm$ 0.21 | 47.15 | NO | 0.5 | Star | GaiaG | 18.8 | -1.52 $\pm$ 0.36 | Gaia EDR3 1970956437691046144
22 | 322.7357 | 44.1784 | 0.9 | 0.014 $\pm$ 0.002 | 2.25 $\pm$ 0.36 | 44.78 | NO | - | - | - | - | - | ?
23 | 322.8111 | 44.5572 | 1.3 | 0.011 $\pm$ 0.002 | 0.59 $\pm$ 0.13 | 37.34 | NO | 2.7 | Non-Star | SDSS_r | 21.7 | 0.01 $\pm$ 0.96 | GSC2 N0331212239468
|  |  |  |  |  |  |  | 3.3 | Non-Star | SDSS_r | 21.6 | -0.03 $\pm$ 0.96 | GSC2 N0331212254752
|  |  |  |  |  |  |  | 3.3 | Non-Star | SDSS_i | 21.2 | -0.05 $\pm$ 0.96 | GSC2 N0331212265386
24 | 322.7257 | 44.3285 | 1.0 | 0.006 $\pm$ 0.001 | 2.10 $\pm$ 0.28 | 37.10 | NO | - | - | - | - | - | ?
25 | 322.5279 | 44.3729 | 1.1 | 0.008 $\pm$ 0.001 | 1.98 $\pm$ 0.32 | 34.78 | NO | 2.0 | Star | GaiaG | 17.9 | -1.72 $\pm$ 0.26 | Gaia EDR3 1970996157547913856
26 | 322.6523 | 44.4078 | 0.8 | 0.005 $\pm$ 0.001 | 0.75 $\pm$ 0.12 | 34.74 | NO | - | - | - | - | - | ?
27 | 322.6700 | 44.4790 | 1.1 | 0.008 $\pm$ 0.001 | 1.13 $\pm$ 0.19 | 34.26 | NO | 0.7 | Star | GaiaG | 14.8 | -3.22 $\pm$ 0.36 | Gaia EDR3 1971002827645828352
28 | 322.6544 | 44.4462 | 1.0 | 0.007 $\pm$ 0.001 | 1.12 $\pm$ 0.22 | 30.88 | NO | 2.2 | Non-Star | SDSS_g | 21.9 | 0.14 $\pm$ 0.61 | GSC2 N0331212233476
29 | 322.7616 | 44.3734 | 0.9 | 0.007 $\pm$ 0.003 | 2.82 $\pm$ 0.98 | 29.88 | NO | 0.9 | Non-Star | SDSS_i | 22.0 | 0.94 $\pm$ 0.39 | GSC2 N0331212287517
30 | 322.9213 | 44.4183 | 1.0 | 0.009 $\pm$ 0.002 | 2.52 $\pm$ 0.45 | 29.64 | NO | 2.7 | Star | GaiaG | 18.2 | -1.53 $\pm$ 0.31 | Gaia EDR3 1970952524975861504
|  |  |  |  |  |  |  | 3.0 | Star | GaiaG | 13.3 | -3.48 $\pm$ 0.31 | Gaia EDR3 1970952524986698624
31 | 323.0323 | 44.4024 | 1.3 | 0.013 $\pm$ 0.002 | 1.29 $\pm$ 0.20 | 29.19 | NO | 2.5 | Star | GaiaG | 16.2 | -2.62 $\pm$ 0.38 | Gaia EDR3 1971034266801276288
32 | 322.5873 | 44.3178 | 1.0 | 0.006 $\pm$ 0.001 | 2.34 $\pm$ 0.38 | 28.27 | NO | 1.6 | Non-Star | SDSS_i | 21.1 | 0.49 $\pm$ 0.20 | GSC2 N0331212281960
33 | 322.6738 | 44.2218 | 1.0 | 0.007 $\pm$ 0.001 | 1.10 $\pm$ 0.17 | 27.42 | NO | 2.3 | Star | GaiaG | 20.1 | -1.13 $\pm$ 0.29 | Gaia EDR3 1970944446140502272
34 | 322.7184 | 44.3332 | 0.9 | 0.004 $\pm$ 0.001 | 0.60 $\pm$ 0.09 | 25.85 | NO | 1.4 | Star | GaiaG | 17.3 | -2.48 $\pm$ 0.24 | Gaia EDR3 1970950978798273920
35 | 322.6870 | 44.4670 | 1.1 | 0.008 $\pm$ 0.001 | 4.28 $\pm$ 0.78 | 23.26 | NO | 2.9 | Non-Star | SDSS_r | 21.6 | 0.85 $\pm$ 0.17 | GSC2 N0331212262403
36 | 322.6012 | 44.3770 | 1.2 | 0.006 $\pm$ 0.001 | 3.43 $\pm$ 0.61 | 22.24 | NO | 1.6 | Star | GaiaG | 20.9 | -0.29 $\pm$ 0.16 | Gaia EDR3 1970949260801367296
37 | 322.7944 | 44.4414 | 1.1 | 0.006 $\pm$ 0.001 | 1.95 $\pm$ 0.37 | 22.00 | NO | 1.7 | Star | GaiaG | 15.0 | -2.89 $\pm$ 0.30 | Gaia EDR3 1970956167118744320
38 | 323.0129 | 44.4497 | 1.0 | 0.015 $\pm$ 0.003 | 11.40 $\pm$ 2.40 | 21.17 | NO | 2.9 | Star | GaiaG | 20.3 | -0.03 $\pm$ 0.09 | Gaia EDR3 1971046292698993152
|  |  |  |  |  |  |  | 3.1 | Star | GaiaG | 17.9 | -1.00 $\pm$ 0.17 | Gaia EDR3 1971046288408505344
39 | 322.8485 | 44.3143 | 1.0 | 0.006 $\pm$ 0.001 | 1.93 $\pm$ 0.31 | 20.26 | NO | 1.8 | Non-Star | SDSS_g | 22.2 | 0.48 $\pm$ 0.20 | GSC2 N0331212276215
40 | 322.7466 | 44.5755 | 2.3 | 0.011 $\pm$ 0.004 | 7.22 $\pm$ 2.36 | 17.86 | NO | 2.2 | Star | GaiaG | 15.3 | -2.24 $\pm$ 0.35 | Gaia EDR3 1971004957949785728
|  |  |  |  |  |  |  | 2.6 | Non-Star | SDSS_g | 20.7 | 0.48 $\pm$ 0.35 | GSC2 N0331212232458
41 | 322.8268 | 44.1844 | 1.4 | 0.003 $\pm$ 0.001 | 1.51 $\pm$ 0.72 | 16.89 | NO | 2.2 | Star | GaiaG | 20.1 | -0.97 $\pm$ 0.50 | Gaia EDR3 1970936650776371968
|  |  |  |  |  |  |  | 4.2 | Star | GaiaG | 20.0 | -1.04 $\pm$ 0.50 | Gaia EDR3 1970936650787455744
42 | 322.6930 | 44.4533 | 1.1 | 0.005 $\pm$ 0.001 | 1.70 $\pm$ 0.37 | 16.48 | NO | - | - | - | - | - | ?
43 | 322.5266 | 44.5201 | 1.0 | 0.011 $\pm$ 0.006 | 8.61 $\pm$ 4.49 | 16.15 | NO | 2.1 | Non-Star | GaiaG | 21.1 | 0.16 $\pm$ 0.23 | Gaia EDR3 1971006572852951296
44 | 322.6004 | 44.1886 | 1.3 | 0.004 $\pm$ 0.001 | 0.40 $\pm$ 0.14 | 15.54 | NO | - | - | - | - | - | ?
45 | 322.4930 | 44.2919 | 1.1 | 0.007 $\pm$ 0.001 | 1.70 $\pm$ 0.35 | 15.06 | NO | - | - | - | - | - | ?
46 | 322.7744 | 44.3828 | 1.0 | 0.004 $\pm$ 0.001 | 1.34 $\pm$ 0.24 | 15.00 | NO | 0.6 | - | GaiaG | 17.7 | -1.99 $\pm$ 0.08 | Gaia EDR3 1970951528542823296
|  |  |  |  |  |  |  | 1.1 | Star | GaiaG | 20.1 | -1.04 $\pm$ 0.21 | Gaia EDR3 1970951528554080768
|  |  |  |  |  |  |  | 2.5 | Star | GaiaG | 20.9 | -0.73 $\pm$ 0.21 | Gaia EDR3 1970951528543900544
|  |  |  |  |  |  |  | 2.5 | Non-Star | SDSS_z | 19.8 | -0.18 $\pm$ 0.21 | GSC2 N0331212259744
47 | 322.8249 | 44.4733 | 1.3 | 0.006 $\pm$ 0.002 | 4.47 $\pm$ 1.05 | 13.65 | NO | - | - | - | - | - | ?
48 | 322.7561 | 44.4806 | 1.2 | 0.005 $\pm$ 0.001 | 0.35 $\pm$ 0.08 | 11.87 | NO | 2.3 | Star | GaiaG | 17.0 | -2.87 $\pm$ 0.92 | Gaia EDR3 1970957094831654784
|  |  |  |  |  |  |  | 3.5 | Star | GaiaG | 19.4 | -1.89 $\pm$ 0.92 | Gaia EDR3 1970957094820313856
49 | 322.4694 | 44.3742 | 2.0 | 0.006 $\pm$ 0.001 | 0.87 $\pm$ 0.20 | 10.76 | NO | 1.6 | Star | GaiaG | 18.9 | -1.68 $\pm$ 0.38 | Gaia EDR3 1970996672943997568
50 | 322.9151 | 44.4709 | 2.4 | 0.008 $\pm$ 0.002 | 2.14 $\pm$ 0.50 | 10.68 | NO | 3.8 | Non-Star | SDSS_z | 20.5 | 0.30 $\pm$ 0.37 | GSC2 N0331212267844
|  |  |  |  |  |  |  | 6.2 | Star | GaiaG | 18.3 | -1.55 $\pm$ 0.37 | Gaia EDR3 1971047284838488704
|  |  |  |  |  |  |  | 6.9 | Non-Star | SDSS_i | 20.6 | 0.26 $\pm$ 0.37 | GSC2 N0331212251938
51 | 322.5518 | 44.3149 | 1.5 | 0.006 $\pm$ 0.002 | 3.45 $\pm$ 0.93 | 10.03 | NO | 3.0 | Star | GaiaG | 16.3 | -2.13 $\pm$ 0.18 | Gaia EDR3 1970948504896789632

Note. — Column description: (1) X-ray source ID number; (2), (3) X-ray
position; (4) X-ray position uncertainty; (5) count rate in the range 0.2-12
keV; (6) flux in the range 0.2-12 keV, assuming an absorbed power-law spectrum
with photon index $\Gamma$ = 1.7 and hydrogen column density $N_{\rm H}$=
3$\times 10^{20}$ cm-2; (7) detection likelihood; (8) source morphology; (9)
angular distance of the candidate optical counterpart from the X-ray position;
(10) classification of the candidate counterpart; (11) magnitude type; (12)
magnitude value; (13) logarithmic value of the X-ray-to-optical flux ratio;
(14) proposed source identification.


# Schwarzian derivative in higher-order Riccati equations

Benoy Talukdar<EMAIL_ADDRESS>Department of Physics, Visva-Bharati
University, Santiniketan 731235, India  Supriya Chatterjee  Department of
Physics, Bidhannagar College, EB-2, Sector-1, Salt Lake, Kolkata 700064, India
Golam Ali Sekh  Department of Physics, Kazi Nazrul University, Asansol 713303,
India

###### Abstract

The Sturm-Liouville equation represents the linearized form of the first-order
Riccati equation. This provides an evidence for the connection between
Schwarzian derivative and this first-order nonlinear differential equation.
Similar connection is not obvious for higher-order equations in the Riccati
chain because the corresponding linear equations are of order greater than
two. With special attention to the second- and third-order Riccati equations
we demonstrate that Schwarzian derivative has a natural space in higher
Riccati equations.

Higher-order Riccati equations; Linearized form; Schwarzian derivative

###### pacs:

02.03.Hq; 02.40 Dr; 02.70.-c

## 1\. Introduction

Let $f(x)$ represent a well behaved function of a single variable $x$ (real or
complex) and primes over it denote appropriate derivatives, then $S(f(x))$
written as

$S(f(x))=\frac{f^{\prime\prime\prime}(x)}{f^{\prime}(x)}-\frac{3}{2}(\frac{f^{\prime\prime}(x)}{f^{\prime}(x)})^{2}$
(1)

is called the Schwarzian derivative of $f(x)$ or Schwarzian in short 1 ; 2 .
The expression in (1) appears in a wide variety of mathematical and physical
context ranging from classical complex analysis to conformal field theory 3 .

In the theory of complex variables the linear fractional or Möbius
transformation given by

$f(x)=\frac{ax+b}{cx+d}$ (2)

posses a number of remarkable properties. In particular, the transformation
(2) with real coefficients forms a group of symmetries in real projective
space 4 ; 5 .The Schwarzian derivative of $f(x)$ in (2) is zero. Thus for any
arbitrary function, $S(f(x))$ measures how much $f$ differs from being a
Möbius transformation. It is important to note that the Schwarzian is not a
function but represents the difference between a function and a more
complicated tensor field 6 .

An important property of the Swarzian is that $S(f)=S(g)$ if and only if
$g(x)=(af(x)+b)/(cf(x)+d)$ with $ad-bc\neq 0$. It is interesting to note that
if $\psi_{1}(x)$ and $\psi_{2}(x)$ represent two linearly independent
solutions of the general Sturm-Liouville equation

$\psi^{\prime\prime}(x)+p(x)\psi^{\prime}(x)+q(x)\psi(x)=0$ (3a) and
$f(x)=\frac{\psi_{1}(x)}{\psi_{2}(x)},$ (3b)

then

$S(f(x))=-\frac{1}{2}p(x)^{2}+2q(x)-p^{\prime}(x).$ (4)

For $p(x)=0$, (3a) corresponds to the time-independent Schrödinger equation
with $q(x)$ as the potential function. Using $p(x)=p^{\prime}(x)=0$ in (4) we
see that the Schwarzian reconstructs the potential.

Assuming

$\psi(x)=u(x)v(x)$ (5)

as a complete solution of (3) it is straightforward to write the equation in
the normal form

$v^{\prime\prime}(x)+\lambda(x)v(x)=0$ (6)

with $\lambda(x)=S(f(x))/2$. This shows that when the first derivative is
removed from the Sturm-Liouville equation the coefficient of the dependent
variable is equal to half the Schwarzian derivative. Apparently, this simple
fact has remained un-noticed in the literature. The object of the present
paper is to exploit this fact to demonstrate the appearance of Schwarzian
derivative in higher-order equations in the Riccati chain 7  rather than
following the method applicable for the first-order equation. In this context
we shall see that linearization of the higher-order Riccati equation leads to
differential equations of order greater than 2. This poses an awkward
analytical constraint to look for the appearance of $S(f(x))$ in these
equations.

## 2\. Riccati equations and Schwarzian

The Riccati equation represented by 8

$\frac{d\omega(x)}{dx}+\alpha(x)+\beta(x)\omega(x)+\omega(x)^{2}=0$ (7)

is the only nonlinear differential equation that is of Painleve type 9 . It
appears in many different fields of physics and mathematics 10 ; 11 ; 12  and
can be linearized using the Cole-Hopf transformation 13 ; 14

$\omega(x)=\frac{\phi^{\prime}}{\phi}$ (8)

to write

$\phi^{\prime\prime}(x)+\beta(x)\phi^{\prime}(x)+\alpha(x)\phi(x)=0.$ (9)

Thus the Schwarzian derivative of (9) will be of the same form as that in (4)
with $b(x)=\beta(x)$ and $c(x)=\alpha(x)$. As expected if (9) is expressed in
the normal form removing the first derivative by the use of $\phi(x)=u(x)v(x)$
then the coefficient of $v(x)$ will be half of the Schwarzian derivative of
(9).

Following Grundland and Levi 7  we write the $n$-th order equation in the
Riccati chain as

$L^{n}\omega(x)+\sum_{i=1}^{n}\alpha_{i}(x)(L^{i-1}\omega(x))+\alpha_{0}(x)=0.$
(10)

Here $n$ denotes the order of the Riccati equation and $L$ stands for a
differential operator represented by

$L=\frac{d}{dx}+c\omega(x).$ (11)

The transformation in (8) re-written as $\omega(x)=\phi^{\prime}(x)/c\phi(x)$
converts the equation in (10) into a linear ordinary differential given by

$\sum_{i=0}^{n}\alpha_{i}(x)\frac{d^{i}\phi(x)}{dx^{i}}+\frac{d^{n+1}\phi}{dx^{n+1}}=0.$
(12)

The second-order Riccati equation and corresponding linear equation obtained
by using $n=2$ in (10) and (12) are

$\begin{split}\omega^{\prime\prime}(x)+(3cf(x)+\alpha_{2}(x))\omega^{\prime}(x)+c\alpha_{2}(x)\omega(x)^{2}\\\
+c^{2}\omega(x)^{3}+\alpha(x)\omega(x)+\alpha_{0}(x)=0\end{split}$ (13)

and

$\phi^{(3)}(x)+\alpha_{2}(x)\phi^{\prime\prime}(x)+\alpha_{1}(x)\phi^{\prime}(x)+c\alpha_{0}(x)\phi(x)=0.$
(14)

The space of solutions of (14) is three dimensional spanned by three linearly
independent eigen functions, say, $\phi_{1}(x)$, $\phi_{2}(x)$ and
$\phi_{3}(x)$. This tends to pose some problem to construct the Schwarzian
corresponding to (14). Clearly, the same is true for other equations in the
hierarchy. However, the appearance of the Schwarzian derivative in the second-
order Riccati equation can be seen as follows.

Making use of $\phi(x)=u(x)v(x)$ in (14) we get

$v^{(3)}(x)+a_{2}^{2}(x)v^{\prime\prime}(x)+a_{3}^{2}(x)v^{\prime}(x)+a_{4}^{2}(x)v(x)=0,$
(15)

where

$a_{2}^{2}(x)=\alpha_{2}(x)+\frac{3u^{\prime}(x)}{u(x)},$ (16a) linearized
forms of the second- and third-order Riccati equations when the second highest
derivatives of the linear e
$a_{3}^{2}(x)=\alpha_{1}(x)+\frac{2\alpha_{2}(x)u^{\prime}(x)}{u(x)}+\frac{3u^{\prime\prime}(x)}{u(x)}$
(16b) and
$a_{4}^{2}(x)=c\alpha_{0}(x)+\frac{\alpha_{1}(x)u^{\prime}(x)}{u(x)}+\frac{\alpha_{2}(x)u^{\prime\prime}(x)}{u(x)}+\frac{u^{(3)}(x)}{u(x)}$
(16c)

We now choose $u(x)$ such that $a_{2}^{2}(x)=0$. This gives

$u(x)=e^{-\frac{1}{3}\int\alpha_{2}(x)dx}.$ (17)

From (15) and (17) we find

$v^{(3)}(x)+b_{3}^{2}(x)v^{\prime}(x)+b_{4}^{2}(x)v(x)=0$ (18)

with

$b_{3}^{2}=\alpha_{1}(x)-\frac{\alpha_{2}(x)^{2}}{3}-\alpha_{2}^{\prime}(x)$
(19a) and
$b_{4}^{2}=c\alpha_{0}(x)-\frac{\alpha_{1}(x)\alpha_{2}(x)^{2}}{3}-\frac{2\alpha_{2}(x)^{2}}{27}-\frac{\alpha_{2}^{\prime\prime}}{3}.$
(19b)

Equation (18) represents a third-order linear differential equation with the
second derivative removed and this result has been found from the second-order
Riccati equation. It is easy to see that the coefficient $b_{3}^{2}$ of
$v^{\prime}(x)$ in (18) stands for the Schwarzian derivative of the Sturm-
Liouvlle equation (3) with $p(x)=2\alpha_{2}(x)/3$ and $q(x)=\alpha_{1}(x)/3$.
This is how the Schwarzian derivative appears in the second-order Riccati
equation.

The expression for $b_{3}^{2}$, the coefficient of $v(x)$ in (18) involves the
second-order derivative of $\alpha_{2}(x)$ and thus it appears that
$b_{4}^{2}$ is unlikely have any connection with Schwarzian. We, however found
that the combination $(b_{4}^{2}(x)-b_{3}^{\prime 2}(x)/3)/\alpha_{2}(x)$
under the constraint $\alpha_{1}^{\prime}(x)=3c\alpha_{0}(x)$ represents a
Schwarzian of (3) with $p(x)=2\alpha_{2}(x)/3$ and $q(x)=\alpha_{2}(x)/2$.

The third-order Riccati equation is given by

$\begin{split}\omega^{(3)}(x)+(4c\omega(x)+\alpha_{3}(x))\omega^{\prime\prime}(x)+(6c^{2}\omega(x)^{2}\\\
+3c\alpha_{3}(x)\omega(x)+\alpha_{2}(x))\omega^{\prime}(x)+3c\omega^{\prime}(x)^{2}\\\
+c^{3}\omega(x)^{4}+c^{2}\alpha_{3}(x)\omega(x)^{3}\\\
+c\alpha_{2}(x)\omega(x)^{2}+\alpha_{1}(x)\omega(x)+\alpha_{0}(x)=0.\end{split}$
(20)

The corresponding linear equation reads

$\begin{split}\phi^{(4)}(x)+\alpha_{3}(x)\phi^{(3)}(x)+\alpha_{2}(x)\phi^{\prime\prime}(x)\\\
+\alpha_{1}(x)\phi^{\prime}(x)+c\alpha_{0}(x)=0.\end{split}$ (21)

When the third derivative term in (21) is removed we get

$v^{(4)}(x)+b_{3}^{3}(x)v^{\prime\prime}(x)+b_{4}^{3}(x)v^{\prime}(x)+b_{5}^{3}(x)v(x)=0$
(22)

with

$b_{3}^{3}(x)=\alpha_{2}(x)-\frac{3\alpha_{3}(x)^{2}}{8}-\frac{3\alpha_{3}^{\prime}(x)}{2},$
(23a)
$b_{4}^{3}(x)=\alpha_{1}(x)-\frac{\alpha_{2}(x)\alpha_{3}(x)}{2}+\frac{\alpha_{3}(x)^{3}}{8}-\alpha_{3}^{\prime\prime}(x)$
(23b) and
$\begin{split}b_{5}^{3}(x)=c\alpha_{0}(x)-\frac{\alpha_{1}(x)\alpha_{3}(x)}{4}+\frac{\alpha_{2}(x)\alpha_{3}(x)^{2}}{16}-\frac{3\alpha_{3}(x)^{4}}{256}\\\
-\frac{\alpha_{2}(x)\alpha_{3}^{\prime}(x)}{4}+\frac{3\alpha_{3}(x)^{2}\alpha_{3}^{\prime}(x)}{32}+\frac{3\alpha_{3}^{\prime}(x)^{2}}{16}-\frac{\alpha_{3}(x)^{(3)}}{4}.\end{split}$
(23c)

The expression for the coefficient $b_{3}^{3}(x)$ of $v^{\prime\prime}(x)$ in
(22) involves only the first derivative of $\alpha_{3}(x)$. Thus , as in the
case of second Riccati equation, the result for $b_{3}^{3}(x)$ represents a
Schwarzian derivative corresponding to (3) with $p(x)=\alpha_{3}(x)/2$ and
$q(x)=\alpha_{2}(x)/6$. From $b_{3}^{3}(x)$ and $b_{4}^{3}(x)$ we have
verified that under the constraint $\alpha_{2}^{\prime}(x)=3\alpha_{1}(x)/2$
the quantity $(b_{4}^{3}(x)-2b_{3}^{\prime 2}(x))/\alpha_{3}(x)$ is a
Schwarzian $-\alpha_{2}(x)+\alpha_{3}(x)^{2}/4+\alpha_{3}^{\prime}(x)$.

In the above we have seen that Schwarzian derivative appear in the linearized
forms of the second- and third-order Riccati equations when the second highest
derivatives of the linear eqations are removed. It is straightforward to
verify that the method suggested works for verifying the presence of
Schwarzian in any higher-order equation in the Riccati chain.

## 3\. Conclusion

In the recent past Carillo et 15  used the properties of Schwarzian derivative
to obtain solutions of the second-order Riccati equation. In the present work
we examine the connection between the Schwarzian derivative and higher-order
equations in the Riccati chain. Broadly speaking, we achieve this by removing
the $(n-1)$-th order derivative term from the $n$-th order linear differential
equation associated with $n$-th order Riccati equation. About fifty years ago
such a reduced equation was considered by Kim 16  in order to look for some
generalization of the Schwarzian derivative. We note that the generalized
derivatives so obtained lead to highly complicated expressions for the
coefficients of first- and zeroth-order derivative terms of a third-order
equation. As opposed to this we derived a transparent method to demonstrate
how the so-called traditional Schwarzian derivative enters into the higher-
order equations of the Riccati Chain.

## References

  * (1) Nehari Z 1949 The Schwarzian derivative and schlicht functions Bull. Amer. Math. Soc.55, 545
  * (2) de Melo W and van Strien S 1988 One-dimensional dynamics: The Schwarzian derivative and beyond Bull. Amer. Math. Soc. (N.S.) 18, 159
  * (3) Carne K 1990 The Schwarzian derivative for conformal maps J. Reine Angew. Math. 408, 10
  * (4) Ablowitz M J and Fokas A S 1997 Complex Variables: Introduction and Applications Cambridge University Press (UK) pp366
  * (5) Olver P J 1993 Application of Lie groups to differential equations, Springer-Verlag, New York pp146
  * (6) Son D N 2018 The Schwarzian derivative and Möbius equation on strictly pseudo-convex CR manifold Comm. Anal. and Geom. 26, 237
  * (7) Grundland A M and Levi D 1999 On higher-order Riccati equations as Bäcklund transformations J. Phys. A: Math. Gen. 32, 3931
  * (8) Reid W T 1972 Riccati Differential Equations – Mathematics in Science an Engineering Vol. 86 Academic Press (NY)
  * (9) Ince E L 1976 Ordinary Differential Equations Dover Publications (NY)
  * (10) Calogero F 1967 Variable Phase Approach to Potential Scattering Academic Press (NY)
  * (11) Bera P K , Nandi T K and Talukdar B 1993 Combined variable-phase supersymmetric quantum mechanics J. Phys. A: Math. Gen. 26L 1073
  * (12) Cariñena J F and Lucas J D 2011 Integrability of Lie systems through Riccati equations J. nonlinear Math. Phys. 18, 29
  * (13) Hole J D 1951 On a quasilinear parabolic equation occurring in aerodynamics Quart. Appl. Math. 9, 225
  * (14) Hopf E 1950 The partial differential equation Comm. Pure Appl. Math. 3, 201
  * (15) Carillo S, Chinchurin A, Filipuk G and Zullo F 2022 Schwarzian derivative, Painleve XXV-Ermakov equation and Backlund transformation arXiv:2201.02267v1 [nonlin SI] 6 Jan 2022
  * (16) Kim W J 1969 The Schwarzian derivative and multivalance Pacific J. Math. 31, 717


# TCM-GPT: Efficient Pre-training of Large Language Models for Domain
Adaptation in Traditional Chinese Medicine

Guoxing Yang$1$, Jianyu Shi$1$, Zan Wang$1$, Xiaohong Liu$2$,∗, Guangyu
Wang$1$,∗ $1$State Key Laboratory of Networking and Switching Technology,
Beijing University of Posts and Telecommunications, Beijing 100876, China
$2$UCL Cancer Institute, University College London, London, United Kingdom
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>*Corresponding Author: Xiaohong
Liu, Guangyu Wang Email<EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

Pre-training and fine-tuning have emerged as a promising paradigm across
various natural language processing (NLP) tasks. The effectiveness of
pretrained large language models (LLM) has witnessed further enhancement,
holding potential for applications in the field of medicine, particularly in
the context of Traditional Chinese Medicine (TCM). However, the application of
these general models to specific domains often yields suboptimal results,
primarily due to challenges like lack of domain knowledge, unique objectives,
and computational efficiency. Furthermore, their effectiveness in specialized
domains, such as Traditional Chinese Medicine, requires comprehensive
evaluation.

To address the above issues, we propose a novel domain specific TCMDA (TCM
Domain Adaptation) approach, efficient pre-training with domain-specific
corpus. Specifically, we first construct a large TCM-specific corpus, TCM-
Corpus-1B, by identifying domain keywords and retreving from general corpus.
Then, our TCMDA leverages the LoRA which freezes the pretrained model’s
weights and uses rank decomposition matrices to efficiently train specific
dense layers for pre-training and fine-tuning, efficiently aligning the model
with TCM-related tasks, namely TCM-GPT-7B. We further conducted extensive
experiments on two TCM tasks, including TCM examination and TCM diagnosis.
TCM-GPT-7B archived the best performance across both datasets, outperforming
other models by relative increments of 17% and 12% in accuracy, respectively.
To the best of our knowledge, our study represents the pioneering validation
of domain adaptation of a large language model with 7 billion parameters in
TCM domain. We will release both TCM-Corpus-1B and TCM-GPT-7B model once
accepted to facilitate interdisciplinary development in TCM and NLP, serving
as the foundation for further study.

###### Index Terms:

Deep Learning, Traditional Chinese Medicine, Large Language Model,
Pretraining, Domain adaptation

##  I Introduction

In recent years, the paradigm of pre-training language models followed by
fine-tuning has gained significant attention in the field of Natural Language
Processing (NLP). Based on a wide variety of textual content, pretrained
language models (PLMs) have shown their capacit in capturing deep semantic
insights, significantly enhancing performance across numerous NLP tasks [1].
For instance, BERT [2] employs stacked Transformer encoders for comprehensive
corpus pre-training, resulting in marked performance gains in downstream
tasks. Concurrently, GPT [3] adopts a Transformer-based decoder architecture,
illustrating notable generative capabilities. Recent studies indicate that by
scaling up both the model size and training data, large language models (LLMs)
can achieve impressive results [4]. For example, models such as GPT-3 [5]
amassing 175 billion parameters, and PaLM [6] with its remarkable 540 billion
parameters, have been trained on datasets with over a trillion tokens. These
datasets include a diverse range of sources, from programming code snippets to
news articles, forum threads, and books. Through comprehensive training, these
models demonstrate emergent behaviors, including few-shot learning and zero-
shot learning, enabling them to excel in tasks such as writing, programming,
and reasoning.

Despite the impressive performance of large language models, their direct
application to specific domains encounters challenges, including complex
domain knowledge, unique objectives, and computational efficiency. A primary
hurdle is the challenge faced by a general LLM in incorporating the
specialized domain-specific knowledge without pretraining, which can cause
performance degradation when adapting models to diverse contexts. The lack of
domain knowledge can even lead to the generation of hallucination or false
information. Gururangan et al. [7] demonstrated that domain-adaptive
pretrained models consistently outperformered that without pretraining,
underscoring the indispensability of pretraining, when evaluated on four
domains, including biomedical, computer science, news, and reviews. Moreover,
Arumae et al. [8] and Jin et al. [9] found that parameter regularization
techniques for pretrained models can yield superior results in domain
adaptation. Shah, R. S., et al.[10] presents FLANG, a domain-specific language
model tailored for financial contexts. By utilizing financial keywords and
phrases, it optimizes its training process and offers benchmarks for five
unique NLP tasks within the financial domain. Another challenge is the
computational cost of large-scale pre-training, which is often prohibitively
high due to large-scale parameter size or stacked network layers. To address
this, ALBERT [11] employs cross-layer parameter sharing in the Transformer
architecture, which substantially reduces the model size. However, this leads
to an obvious performance degradation. DistilBERT [12] serves as a distilled
version of large pretrained models, reducing parameters with maintaining
commendable performance. Edward et al. [13] introduced the Low-Rank Adaptive
(LoRA) approach, wherein they freeze the pretrained model’s weights and use
rank decomposition matrices to efficiently train specific dense layers,
resulting in significantly reducing the number of trainable parameters without
compromising performance on subsequent tasks.

In the field of biomedicine, there exist research that leverages pre-
trainedlanguage models in specialized domains, demonstrating promising
results. For instance, BioBERT [14], a domain-specific BERT-based model pre-
trained on biomedical text, has demonstrated applicability across downstream
tasks including biomedical named entity recognition and relation extraction.
Venigalla et al. [15] introduced BioMedLM, a GPT-based mode trained on the
PubMed abstracts and full text documents, achieving state-of-the art results
on medical question and answer tasks. Despite the existing research of pre-
trained language models, few studies have explored scaling them within the
clinical domain, due to the sensitive nature of clinical narratives and the
substantial computing power required. Furthermore, their effectiveness in
specialized domains, such as Traditional Chinese Medicine (TCM), remains to be
validated.

In this study, we aim to investigate the adaptation of large language models
to TCM domain and subsequently conduct performance evaluations. This effort
carries considerable significance, as Traditional Chinese Medicine represents
a cherished heritage that has been passed down through generations in China,
presenting a noteworthy complement to Western medicine [16]. There have been a
few studies focused on NLP tasks in TCM. For example, Zhang et al. [17]
proposed the Traditional Chinese Medicine Language System (TCMLS), aiming to
standardize TCM terminology to enhance information extraction. Nevertheless,
constructing such a system incurred high data acquisition costs. However, the
integration of large language models into the TCM is currently not well
developed compared to Western medicine. This discrepancy can be attributed to
a range of challenges. Firstly, in contrast to other medical domains,
Traditional Chinese Medicine faces a scarcity of publicly available resources.
Many pre-trained models are predominantly fine-tuned on generalized corpora,
potentially leading to suboptimal performance when confronted with the nuanced
language of TCM [18, 19]. Moreover, the computational resources available for
training domain-specific models are generally scarcer compared to the
abundance available for generic models.

To address these challenges, we explored the process of tailoring general-
purpose LLMs to the specific TCM domain through the integration of contextual
data and domain-specific knowledge. We introduced a novel approach, TCMDA,
aimed at efficiently customizing pre-training models for the TCM field. Our
contributions are summarized as follows: Firstly, to overcome the limited
availability of pre-training data in TCM, we adopted a strategic approach by
extracting domain text from general corpus. By incorporating domain
terminologies and extracting task-relevant keywords, we constructed a set of
keywords specific to TCM. This set was then utilized to retrieve TCM-relevant
text, resulting in the TCM-Corpus-1B, a substantial collection of TCM-related
textual data. Secondly, to ensure the alignment of large language models to
the TCM domain, we introduced a novel model named TCM-GPT-7B. This model
adopts the LoRA technique for domain adaptation during both pre-training and
fine-tuning phases. Furthermore, we performed a comprehensive evaluation of
TCM-GPT-7B across two distinct TCM tasks: TCM examinations and TCM diagnosis.
The experimental results demonstrate our models archive state-of-the-art
performance after pre-training using the TCM-Corpus-1B, revealing the
necessity and effectiveness of domain-specific pre-training. To our knowledge,
TCM-GPT-7B represents a pioneer as the first 7B-scale large language model
pre-trained exclusively on TCM data. Finally, we are committed to publicly
releasing both the curated the TCM-Corpus-1B and the TCM-GPT-7B model, to
advance the cross-disciplinary research in both TCM and NLP.

Figure 1: The overview of TCMDA

##  II Methods

In this section, we describe our proposed TCMDA. Our method can be divided
into three steps: domain-related keyword extraction, associated corpus
retrieval, and LLM pretraining and fine-tuning, as shown in Figure 1.

###  II-A Domain-related Keyword Extraction

We utilized the TextRank algorithm [20] to extract a set of keywords related
to task, denoted as $K^{T}=\\{k_{1},\cdots,k_{n}\\}$, where $k_{i}\in K^{T}$
is a keyword, $n$ represents the number of keywords. TextRank is a graph-based
ranking algorithm, used for automates keyword extraction in text. It builds a
graph by measuring word similarity, employing iteration to assess word
importance and derive keyword rankings. By applying TextRank to the text
extracted from all samples, we acquired the top five keywords for each
individual sample. To augment keyword diversity and enhance the domain
coverage, we incorporated a publicly accessible TCM
lexicon111https://pinyin.sogou.com/dict/cate/index/135 into the keyword
collection, denoted as $K^{L}$. Consequently, this fusion of both $K^{T}$ and
$K^{L}$ led to the formation of the comprehensive keyword set, $K^{D}$, which
served as the foundation for querying the corpus.

Furthermore, during the keyword extraction process for each sample, we
incorporated quantity information to enhance the effectiveness of retrieving
task-relevant corpus. Specifically, we employed the count of keyword
occurrences as weights when retrieval. The weight was computed as follows:

$\displaystyle w_{i}={1+{\log(n_{i})}},$ (1)

where $n_{i}$ represents the count of keyword $k_{i}$.

###  II-B Associated Corpus Retrieval

We collected an extensive language corpus from both Baidu
Baike222https://baike.baidu.com/ and Wikipedia
sources333https://zh.wikipedia.org/. Before extracting retrieval from the
corpus, a thorough data cleaning process was performed. We have removed the
special tags and characters, and standardized the format for retrieved
contents. To effectively retrieve domain-specific content from the collected
corpus, we employed the BM25 algorithm [21], as it’s simple, efficient, and
widely used in information retrieval tasks[22]. BM25 is based on the TF-IDF
(Term Frequency-Inverse Document Frequency) model. It treats text as a
collection of terms and calculates a score $score(D,Q)$ between a given
document $D$ from database and a query $Q$. To enhance the efficacy of BM25,
we adjusted a query keyword’s weight by a previous obtained weight $w_{i}$.
Specifically, we repeated a keyword $k_{i}$ with $\min(\lfloor
w_{i}\rfloor,3)$ times. The resultant modified query $Q^{\prime}$ is then used
to score the documents, ultimately yielding the final relevance score
$score(D,Q^{\prime})$.

###  II-C LLM Pretraining and Fine-tuning with LoRA

To address the challenges associated with excessive parameter size and the
risk of overfitting, we adopted the LoRA approach, we employed the LoRA [13]
to for both pre-training and fine-tuning, as an alternative to conventional
full-parameter fine-tuning strategies. The LoRA methodology involves the
freezing of pretrained model matrix weights $W$, coupled with the introduction
of trainable rank decomposition matrices $B$ and $A$ into each layer of the
Transformer architecture, greatly reducing the number of trainable parameters
for downstream tasks. Specifically, given a linear layer $h=Wx$, the LoRA
transforms this layer into $h=(W+W^{\prime})x$, where $W$ represents frozen
parameters, $W^{\prime}=BA$ represents trainable parameters,
$B\in\mathbb{R}^{d_{1}\times r}$, $A\in\mathbb{R}^{r\times d_{2}}$, and the
rank $r\ll\min(d_{1},d_{2})$. During pre-training process, we optimize a
causal language model (CLM) loss function designed to predict the subsequent
token:

$\displaystyle L_{CLM}=-\frac{1}{n}\sum_{i=1}^{n}\log
P_{{\theta}^{\prime}}(w_{i}|w_{1},...,w_{i-1})$ (2)

where $w_{1},\cdots,w_{i-1}$ denotes the preceding $i-1$ tokens in a sentence,
$w_{i}$ denotes the next token. During instruction supervised fine-tuning
(SFT) process, for a given input prompt $I=w_{1:m}$ and response
$R=w_{m+1:m+n}$, the loss function is:

$\displaystyle L_{SFT}$ $\displaystyle=-\log P_{\theta^{\prime\prime}}(R|I)$
(3) $\displaystyle=-\frac{1}{n}\sum_{i=m+1}^{m+n}{\log
P_{\theta^{\prime\prime}}(w_{i}|w_{1},...,w_{m-1},w_{m},..,w_{i-1})}$ (4)

where $n$ and $m$ represent the lengths of the response and input prompt,
respectively.

##  III Experiments

###  III-A Datasets

The data utilized in this study primarily consists of three components: a
general corpus, a TCM examination dataset (denoted as TCM-EXAM), and a TCM
Electronic Health Record dataset (denoted as TCM-EHR). The general corpus was
collected from Baidu Baike and Wikipedia sources. After undergoing a data
cleaning process, this corpus encompasses a substantial volume of information,
totally 20GB in size. The TCM-EXAM dataset comprises TCM-specific examination
questions from diverse sources. It encompasses a total of 6,325 multiple-
choice questions. Within this dataset, we constructed a test subset that
concentrates on questions related to Diagnostics (denoted as DT) and Formula
Science (denoted as FS) in TCM, containing 214 and 207 questions,
respectively. The remaining 5,904 questions are used as the train set. The
TCM-EHR dataset is collected from a traditional Chinese medicine hospital and
encompasses 7,783 electronic health records. Each individual record
encapsulates information including chief complaints, history of present
illness, physical examinations, and diagnoses, among others. For this dataset,
a targeted test subset was generated, including specific disease groups,
namely ”Etiology and syndrome” (denoted as ES) and ”Skin and mucosal diseases”
(denoted as SMD) within TCM. This test subset encompasses 150 and 150 records,
while the remaining 7,483 records are used as the training set.

###  III-B Evaluation Metric

We conducted performance evaluations for our model across two distinct tasks:
the TCM examination task using the TCM-EXAM dataset, and the TCM diagnosis
task using the TCM-EHR dataset. For the TCM examination task, we employed the
examination questions along with their corresponding answer choices as input.
We then extracted the predicted option from the model-generated response using
regular expressions. Regarding the TCM diagnosis task, in order to facilitate
quantitative evaluation, we transformed the diagnostic inquiries into five-
choice multiple-choice questions. Specifically, we augmented the questions
with additional disease options from the same disease group. Subsequently,
patient information and disease options were fed as inputs, and the predicted
option was extracted from the model-generated response using regular
expressions. For both of these tasks, we employed accuracy as the evaluation
metric. This involved computing accuracy by comparing the extracted options
with the accurate answers.

###  III-C Training Details

We conducted a series of comprehensive experiments, following [23]. We opted
to utilize the BLOOM-7B model [24] as our foundational architecture, given its
multilingual capabilities and public accessibility. During the pretraining
process, we configured the learning rate to 1e-4, employed a batch size of
128, a maximum sequence length of 1024 tokens, spanned across 2 epochs. During
the supervised fine-tuning process, we established the learning rate to 5e-5,
with a batch size of 128, a maximum sequence length of 1024, training across 3
epochs. We adopted LoRA with the rank $r$ to 8, the scaling factor to 32, and
a dropout rate of 0.1.

##  IV Results

###  IV-A Evaluation on Two TCM Tasks

TABLE I: Evaluation Results of TCM Examination Task. | TCD | FS | Average
---|---|---|---
Baseline | 0.252 | 0.242 | 0.247
Random | 0.278 (+10.3%) | 0.268 (+10.7%) | 0.273 (+10.5%)
TCMDA | 0.315 (+25.0%) | 0.265 (+9.5%) | 0.290 (+17.4%)
TABLE II: Evaluation Results of TCM Diagnosis Task. | ES | SMD | Average
---|---|---|---
Baseline | 0.235 | 0.234 | 0.235
Random | 0.269 (+14.4%) | 0.213 (-8.9%) | 0.241 (+2.5%)
TCMDA | 0.252 (+7.2%) | 0.277 (+18.3%) | 0.264 (+12.3%)

To evaluate the impact of various pre-training strategies on the performance
of models in TCM tasks, we conducted a comprehensive evaluation using TCM-EXAM
and TCM-EHR datasets. The models encompassed three pre-training strategies,
followed by fine-tuning: Baseline, models without any additional pre-training.
Random, models pre-trained on a random corpus of size equivalent to TCM-
Corpus-1B. TCMDA (ours), models pre-trained on TCM-specific corpus, namely
TCM-Corpus-1B. Tables I and II present the evaluation results for the TCM
examination and diagnosis tasks, respectively. The percentages within
parentheses indicate the relative performance compared to the baseline model
(Baseline).

As shown in Tables I and II, we found that models with further pretraining
consistently outperform those without additional pretraining. Specifically,
the Random model exhibited performance improvements of $10.5\%$ and $2.5\%$ on
the TCM-EXAM and TCM-EHR tasks, respectively. Our domain-specific TCMDA model
showcased more enhancements with increases of $17.4\%$ for TCM-EXAM and
$12.3\%$ for TCM-EHR, achieving accuracies of $0.290$ and $0.264$,
respectively. Notably, Random exhibited a performance boost with pretraining
on a random corpus. This enhancement can possibly be attributed to the fact
that BLOOM, our pre-training framework, lacks exposure to Chinese language
data, thus providing certain benefits in a different context. However, the
most remarkable advancements were observed with the TCMDA model, indicating
the effectiveness of domain-specific pre-training. It’s worth noting that the
Random model exhibited an accuracy decline of $8.9\%$ in the SMD task of TCM-
EXAM. Conversely, the TCMDA model consistently outperformed across all tasks,
demonstrating that domain-specific pre-training not only enhances performance
but also improves model robustness across various scenarios. The greater
enhancement in TCM-EXAM ($17.4\%$) compared to TCM-EHR ($12.3\%$) could be
attributed to the broader scope of TCM-EXAM, which benefits more from domain
pre-training. While TCM-EHR primarily deals with clinical records. Even though
extracting specialized content can be challenging, our domain-specific pre-
training significantly improved the model’s comprehension of these records.

###  IV-B Effect of Corpus Size

Figure 2: The Impact of Corpus Size on Performance

To study the effect of corpus size on model performance, we conducted
experiments with varying amounts of corpus for pretraining, as shown in Figure
2. Our TCMDA model, pretrained using TCM-related corpus, consistently
demonstrated superior performance compared to Random. As demonstrated in I and
II, the Random exhibited improved Chinese language capabilities over Baseline,
as also evidenced in Figure 2. However, with the expansion of the corpus size,
Random did not display a trend of increasing in performance. In contrast, the
TCMDA model, pretrained using TCM-related corpus, exhibited a upward trend in
performance as the size of the corpus grew. This observation also suggests the
rationale for further experimental validation across a broader spectrum of
corpora in the future study.

###  IV-C Case Study

TABLE III: A sample question extracted from TCM-EXAM test set. Question
---
脉在筋骨，乍疏乍密，散乱无序者，称为
回答选项：A. 鱼翔脉; B. 虾游脉; C. 雀啄脉; D. 解索脉
请分析并给出正确选项。
Translation:
Pulse in the tendons and bones, sometimes sparse and sometimes dense,
scattered and disordered, is called
Answer options: A. Waving fish pulse; B. Darting shrimp pulse; C. Pecking
sparrow pulse; D. Untwining rope pulse)
Please analyze and provide the correct option.
Response 1 (TCMDA)
脉在筋骨，乍疏乍密，散乱无序者，称为解索脉。解索脉是指脉搏在筋骨之间出现，乍疏乍密，散乱无序的状态。脉搏在筋骨之间出现，意味着脉搏在筋骨之间的血管中流动。乍疏乍密，散乱无序的状态，意味着脉搏在筋骨之间出现时，脉搏的强弱、节律和节奏都不稳定。因此，解索脉是脉搏在筋骨之间出现的一种状态。因此，正确选项是D.
解索脉。
Translation:
The pulse in the muscles and bones, sparse and dense at first, scattered and
disorderly, is called untwining rope pulse. Untwining rope pulse refers to the
state in which the pulse appears between muscles and bones, initially sparse
and dense, scattered and disorderly. The pulse appears between the muscles and
bones, indicating that it flows through the blood vessels between the muscles
and bones. The state of being scattered and disorderly means that the
strength, rhythm, and rhythm of the pulse are unstable when it appears between
muscles and bones. Therefore, untwining rope pulse is a state in which the
pulse appears between muscles and bones. Therefore, the correct option is D
Untwining rope pulse.
Response 2 (Random)
脉是人体经络系统的一部分，在筋骨中运行。脉的形态和结构非常复杂，乍疏乍密，散乱无序。这种形态和结构的特点与鱼翔脉的形态和结构相似。因此，脉在筋骨，乍疏乍密，散乱无序者，称为鱼翔脉。因此，正确选项是A.
鱼翔脉。
Translation:
Pulse is a part of the human meridian system that operates in the muscles and
bones. The morphology and structure of veins are very complex, with sparse and
dense patterns, scattered and disorderly. The characteristics of this form and
structure are similar to those of fish-swimming pulse. Therefore, the pulse in
the muscles and bones, which are sparse and dense at first, scattered and
disorderly, are called the waving fish pulse. Therefore, the correct option is
A Waving fish pulse.

Table III illustrates a task sample extracted from the TCM-EXAM test dataset.
Response 1 is generated by our TCMDA model, while Response 2 is the output of
the Random model. Notably, Response 1 correctly addresses the query, whereas
Response 2 falls short in providing the accurate answer. Despite both
responses exhibit linguistic fluency, Response 2 lacks meaningful and
informative content.

For example, within Response 1, the sentence ”The pulse in the muscles and
bones, sparse and dense at first, scattered and disorderly, is called
untwining rope pulse.” effectively describes the definition of untwining rope
pulse, with subsequent analysis stemming from this definition leading to the
accurate answer. This demonstrates how information gained from the TCM corpus
enriches the quality of model-generated responses, thereby augmenting the
overall performance of the model. In contrast, Response 2’s sentence ”Pulse is
a part of the human meridian system that operates in the muscles and bones.
The morphology and structure of veins are very complex, with sparse and dense
patterns, scattered and disorderly.” merely echoes the original question and
subsequently arrives at an incorrect answer. This result shows that without
domain-specific knowledge learned through pretraining, models struggle to
generate useful insights and provide accurate responses.

##  V Conclusion

In this paper, we have proposed a strategic domain adaptation method for TCM,
TCMDA, to overcome the limited availability of pre-training data in TCM. To
the best of our knowledge, TCMDA represents the first attempt of domain
adaptation of a large language model with 7 billion parameters in TCM domain.
The core idea is to construct a TCM-specic corpus, TCM-Corpus-1B, by
retrieving from general corpus and adapt our models on it. To align our models
to the TCM domain, we use the LoRA technique for domain adaptation during pre-
training and fine-tuning. Furthermore, we conduct a multi-level evaluation for
our approach, including simple TCM examination task and complex real-world TCM
diagnosis task. Our experiments show that TCMDA helps to enhance the model
performance on various TCM downstream tasks, revealing the necessity and
effectiveness of domain-specific pre-training. We will release both TCM-
Corpus-1B and TCM-GPT-7B model to facilitate interdisciplinary development in
TCM and NLP.

## Acknowledgment

This study was funded by the National Natural Science Foundation of China
(grant 62272055), New Cornerstone Science Foundation through the XPLORER
PRIZE, and Young Elite Scientists Sponsorship Program by CAST (2021QNRC001).
The authors appreciate Nanjing Institute of InforSuperBahn for providing the
traning and evaluation platform.

## References

  * [1] X. Han, Z. Zhang, N. Ding, Y. Gu, X. Liu, Y. Huo, J. Qiu, Y. Yao, A. Zhang, L. Zhang _et al._ , “Pre-trained models: Past, present and future,” _AI Open_ , vol. 2, pp. 225–250, 2021.
  * [2] S. Alaparthi and M. Mishra, “Bidirectional encoder representations from transformers (bert): A sentiment analysis odyssey,” _arXiv preprint arXiv:2007.01127_ , 2020.
  * [3] A. Radford, K. Narasimhan, T. Salimans, I. Sutskever _et al._ , “Improving language understanding by generative pre-training,” 2018.
  * [4] W. X. Zhao, K. Zhou, J. Li, T. Tang, X. Wang, Y. Hou, Y. Min, B. Zhang, J. Zhang, Z. Dong _et al._ , “A survey of large language models,” _arXiv preprint arXiv:2303.18223_ , 2023.
  * [5] T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell _et al._ , “Language models are few-shot learners,” _Advances in neural information processing systems_ , vol. 33, pp. 1877–1901, 2020.
  * [6] A. Chowdhery, S. Narang, J. Devlin, M. Bosma, G. Mishra, A. Roberts, P. Barham, H. W. Chung, C. Sutton, S. Gehrmann _et al._ , “Palm: Scaling language modeling with pathways,” _arXiv preprint arXiv:2204.02311_ , 2022.
  * [7] S. Gururangan, A. Marasović, S. Swayamdipta, K. Lo, I. Beltagy, D. Downey, and N. A. Smith, “Don’t stop pretraining: Adapt language models to domains and tasks,” _arXiv preprint arXiv:2004.10964_ , 2020.
  * [8] K. Arumae, Q. Sun, and P. Bhatia, “An empirical investigation towards efficient multi-domain language model pre-training,” _arXiv preprint arXiv:2010.00784_ , 2020.
  * [9] X. Jin, D. Zhang, H. Zhu, W. Xiao, S.-W. Li, X. Wei, A. Arnold, and X. Ren, “Lifelong pretraining: Continually adapting language models to emerging corpora,” _arXiv preprint arXiv:2110.08534_ , 2021.
  * [10] R. S. Shah, K. Chawla, D. Eidnani, A. Shah, W. Du, S. Chava, N. Raman, C. Smiley, J. Chen, and D. Yang, “When flue meets flang: Benchmarks and large pre-trained language model for financial domain,” _arXiv preprint arXiv:2211.00083_ , 2022.
  * [11] Z. Lan, M. Chen, S. Goodman, K. Gimpel, P. Sharma, and R. Soricut, “Albert: A lite bert for self-supervised learning of language representations,” _arXiv preprint arXiv:1909.11942_ , 2019.
  * [12] V. Sanh, L. Debut, J. Chaumond, and T. Wolf, “Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter,” _arXiv preprint arXiv:1910.01108_ , 2019.
  * [13] E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, and W. Chen, “Lora: Low-rank adaptation of large language models,” _arXiv preprint arXiv:2106.09685_ , 2021.
  * [14] J. Lee, W. Yoon, S. Kim, D. Kim, S. Kim, C. H. So, and J. Kang, “Biobert: a pre-trained biomedical language representation model for biomedical text mining,” _Bioinformatics_ , vol. 36, no. 4, pp. 1234–1240, 2020.
  * [15] M. C. Abhinav Venigalla, Jonathan Frankle. (2023) Biomedlm: a domain-specific large language model for biomedical text. [Online]. Available: https://www.mosaicml.com/blog/introducing-pubmed-gpt
  * [16] J.-L. Tang, B.-Y. Liu, and K.-W. Ma, “Traditional chinese medicine,” _The Lancet_ , vol. 372, no. 9654, pp. 1938–1940, 2008.
  * [17] T. Zhang, Y. Wang, X. Wang, Y. Yang, and Y. Ye, “Constructing fine-grained entity recognition corpora based on clinical records of traditional chinese medicine,” _BMC medical informatics and decision making_ , vol. 20, no. 1, pp. 1–17, 2020.
  * [18] Z. Yin, Q. Sun, Q. Guo, J. Wu, X. Qiu, and X. Huang, “Do large language models know what they don’t know?” _arXiv preprint arXiv:2305.18153_ , 2023.
  * [19] H. He, H. Zhang, and D. Roth, “Rethinking with retrieval: Faithful large language model inference,” _arXiv preprint arXiv:2301.00303_ , 2022.
  * [20] R. Mihalcea and P. Tarau, “Textrank: Bringing order into text,” in _Proceedings of the 2004 conference on empirical methods in natural language processing_ , 2004, pp. 404–411.
  * [21] S. Robertson, H. Zaragoza _et al._ , “The probabilistic relevance framework: Bm25 and beyond,” _Foundations and Trends® in Information Retrieval_ , vol. 3, no. 4, pp. 333–389, 2009.
  * [22] X. Yao, Y. Zheng, X. Yang, and Z. Yang, “Nlp from scratch without large-scale pretraining: A simple and efficient framework,” in _International Conference on Machine Learning_. PMLR, 2022, pp. 25 438–25 451.
  * [23] G. Wang, G. Yang, Z. Du, L. Fan, and X. Li, “Clinicalgpt: Large language models finetuned with diverse medical data and comprehensive evaluation,” _arXiv preprint arXiv:2306.09968_ , 2023.
  * [24] T. L. Scao, A. Fan, C. Akiki, E. Pavlick, S. Ilić, D. Hesslow, R. Castagné, A. S. Luccioni, F. Yvon, M. Gallé _et al._ , “Bloom: A 176b-parameter open-access multilingual language model,” _arXiv preprint arXiv:2211.05100_ , 2022.