2021/12/17 2022/05/30

††affiliationtext: MAXI team, RIKEN, 2-1 Hirosawa, Wako, Saitama 351-0198,
Japan

accretion, accretion disks — stars:neutron— X-rays:binaries

# Decades’ long-term variations in NS-LMXBs observed with MAXI/GSC, RXTE/ASM
and Ginga/ASM

Kazumi Asai Tatehiro Mihara and Masaru Matsuoka<EMAIL_ADDRESS>
###### Abstract

We investigated the decades’ long-term X-ray variations in bright low-mass
X-ray binaries containing a neutron star (NS-LMXB). The light curves of
MAXI/GSC and RXTE/ASM covers $\sim$ 26 yr, and high-quality X-ray light curves
are obtained from 33 NS-LMXBs. Among them, together with Ginga/ASM, two
sources (GX 3$+$1 and GX 9$+$1) showed an apparent sinusoidal variation with
the period of $\sim 5$ yr and $\sim 10$ yr in the 34 yr light curve. Their
X-ray luminosities were $(1-4)\times 10^{37}$ erg s-1 in the middle of the
luminosity distribution of the NS-LMXB. Other seven sources (Ser X-1, 4U
1735–444, GX 9$+$9, 4U 1746$-$37, 4U 1708$-$40, 4U 1822$-$000, and 1A
1246$-$588) have also similar sinusoidal variation, although the profiles
(amplitude, period, and phase) are variable. Compering the 21 sources with
known orbital periods, a possible cause of the long-term sinusoidal variation
might be the mass transfer cycles induced by the irradiation to the donor
star.

##  1 Introduction

Many of the bright X-ray sources are low mass X-ray binaries with a weakly
magnetized neutron star (NS-LMXB: see [Barret (2001)] for a review). Based on
the temporal activity, NS-LMXBs are divided into two types: persistent type
and transient type. Furthermore, many persistent NS-LMXBs and bright phases of
transient NS-LMXBs are divided into two groups: Z sources and Atoll sources,
based on their behavior on the color–color diagram and the hardness–intensity
diagram (Hasinger & van der Klis, 1989). Z sources are very bright, and the
luminosities sometimes become close to the Eddington luminosity ($L_{\rm E}$).
On the other hand, Atoll sources are generally less bright ( $\ltsim$ 0.5
$L_{\rm E}$).

The orbital periods of NS-LMXBs range from minutes to $\sim 20$ d (Liu et al.,
2007). Moreover, long-term variations on the time scale beyond the orbital
period are known. These super-orbital periods range in tens to hundreds of
days, which are thought to be related to the properties of the accretion disk,
such as radiation-induced warping and precession (see Charles et al. (2008)
for a review). On the other hand, several NS-LMXBs display very long-term
quasi-periodic modulations (approximately several to tens of years). The
variations are thought to have a different origin. Kotze and Charles (2010)
(hereafter KC10) suggest that the long-term variations are due to the
variation of the mass-transfer rate from the donor, which is a consequence of
solar-like magnetic cycles (Applegate & Patterson (1987), Warner (1988)).
Solar-like cycles of $\sim$ 10 yr are observed from many late type stars
(Baliunas et al., 1995).

It is pointed out the importance of irradiation in NS-LMXBs for the outburst
properties and their long-term evolution (Ritter, 2008). The former is
relevant to irradiating the accretion disk. Irradiation during an outburst
leads to drastic changes in the outburst properties because the irradiation
changes the conditions for the occurrence of disk instabilities. The latter is
relevant to the irradiating donor star. The irradiation of the donor star can
destabilize mass transfer and lead to irradiation-driven mass transfer cycles,
i.e., to a secular evolution. However, it is not clear to estimate the effect
of irradiation for the secular evolution because of several unclear factors.

GX 3$+$1 shows the long-term variation on the time scale of years superimposed
with the short-term variations on the time scale of hours (Seifina &
Titarchuk, 2012). The short-term variations are due to the transitions between
branches in terms of its color–color diagram, which are independent of the
long-term variation. The spectral index is constant during the long-term
variations.

In this paper, we pay attention to the sinusoidal variation and report the
analysis of the long-term variations during 1996–2021 for 41 NS-LMXBs. In
section 2, we present the details of X-ray light curve analysis. We show the
results in section 3. We discuss the cause of the sinusoidal variation in
section 4.

##  2 Observations and data analysis

List of NS-LMXBs observed with MAXI/GSC and RXTE/ASM.  Name
Type**footnotemark: $*$ Lave \dagger\daggerfootnotemark: $\dagger$ Distance
Porb {\ddagger}{\ddagger}footnotemark: ${\ddagger}$ ASM/GSC\S\Sfootnotemark:
$\S$ Comment \|\|footnotemark: $\|$ Reference\sharp\sharpfootnotemark:
$\sharp$ (1036 erg s-1) (kpc) (hr) Sco X-1 Z 222 2.8 18.90 0.95 NP (1) GX
17$+$2 Z 209 12.6 – 0.97 NP (2) GX 5$-$1 Z 173 9 – 0.94 NP (3) Cyg X-2 Z 129
11 235.2 0.97 NP (4) GX 349$+$2 Z 126 5 22.5 0.88 NP (3) LMC X-2 Z 99 50 8.16
1.02 NP (5) GX 340$+$0 Z 96 11 – 0.98 NP (3) GX 13$+$1 Z, A 35 7 577.6 0.92 NP
(1) 4U 1820$-$303 A, UCXB 34 7.6 0.19 1.02 FV (1) Ser X-1 A 33 8.4 2 0.96 MP
(1) GX 9$+$1 A 29 5.0 – 0.96 CP (1) 4U 1705$-$440 A 21 7.4 – 1.00 FV (1) 4U
1735$-$444 A 22 8.5 4.65 1.00 MP (4) 4U 1624$-$490 ADC 21 15 20.89 1.06 NP (7)
SAX J1747.0$-$2853 T 20 9 – – – (8) GX 9$+$9 A 12 5.0 4.20 0.98 NP (3) 4U
1254$-$690 A 11 13 3.93 0.99 NP (10) Cir X-1 Z, A 12 7.8 398.4 1.04 LV (6) GX
3$+$1 A 10 4.5 – 1.14 CP (9) GS 1826$-$238 T 8.4 7.0 2.088 1.41 LV (11) 4U
1746$-$37 A 6.8 11.0 5.16 0.76 MP (1) 4U 1708$-$40 - 5.3 8 – 0.83 MP (12) 4U
1724$-$307 - 4.1 7.4 – – – (1) 4U 1543$-$624 UCXB 3.9 9.2 0.303 1.02 NP (1) 4U
1636$-$536 A 3.8 6.0 3.80 0.99 LV (1) Aql X-1 T 2.7 5.0 18.95 – – (1) 4U
2127$+$119 (M15 X-2) UCXB 2.4 10.3 0.376 0.90 NP (13) 4U 0513$-$40 UCXB 2.3 12
– 1.03 NP (14) EXO 1745$-$248 T 2.0 5.9 – – – (15) 4U 1608$-$522 T 1.9 4.1
12.89 – – (1) 4U 1822$-$000 - 1.8 6.3 3.2 0.92 MP (16) XTE J1709$-$267 T 1.4
8.5 – – – (17) 4U 1916$-$053 UCXB 1.2 8.9 0.83 1.40 NP (4) 4U 1745$-$203 T 1.1
8.5 – – – (15) 4U 0614$+$091 UCXB 1.1 3.2 – 0.94 NP (18) SLX 1735$-$269 - 1.0
7.3 – 2.64 NP (4) HETE J1900.1$-$2455 T 0.9 5 1.39 – – (4) 1H 0918$-$548 UCXB
0.5 4.8 – 0.94 NP (19) 1A 1246$-$588 UCXB 0.4 5 – 0.83 MP (20) 4U 1323$-$619 -
0.3 4.2 2.93 0.64 NP (21) 1H 1556$-$605 - 0.3 4 9.1 1.58 NP (3)
**footnotemark: $*$ The source types are indicated by ’Z’-Z source, ’A’-Atoll
source, ’ADC’-ADC source, ’UCXB’-Ultra Compact X-ray Binary, ’T’-transient.
{\dagger}{\dagger}footnotemark: ${\dagger}$ Luminosity in the 2–10 keV band in
the unit of $10^{36}$ erg s-1 from MJD=55100 to  MJD=59662.
{\ddagger}{\ddagger}footnotemark: ${\ddagger}$ Orbital periods of systems are
adopted from Liu et al. (2007) except fot Ser X-1 (Cornelisse et al., 2013).
\S\Sfootnotemark: $\S$ ASM/GSC for MJD=55100– 55460.
\|\|footnotemark: $\|$ Type of variation: ’CP’ show a clear periodic
variation, ’MP’ show a modified periodic variation, ’NP’ show no periodic
variation, ’FV’ show fast variability, ’LV’ show large variability, ’–’ are
Transient and Contamination sources.
\sharp\sharpfootnotemark: $\sharp$ Reference of distance. (1) Liu et al.
(2007), (2) Lin et al. (2012), (3) Christian and Swank (1997), (4) Galloway et
al. (2008), (5) Freedman et al. (2001) (6) D’Ai et al. (2012), (7) Xiang et
al. (2007), (8) Natalucci et al. (2000) (9) Kuulkers and van der Klis (2000),
(10) in’t Zand et al. (2003), (11) Barret et al. (2000), (12) Revnivtsev et
al. (2011), (13) White and Angelini (2001), (14) Harris (1996), (15) Valenti
et al. (2007), (16) Shahbaz et al. (2007), (17) Ludlam et al. (2017), (18)
Kuulkers et al. (2010), (19) Jonker and Nelemans (2004), (20) Jonker et al.
(2007), (21) Gambino et al. (2016).

The long-term X-ray activity is continuously monitored with MAXI (Monitor of
All-sky X-ray Image: Matsuoka et al. (2009)) since 2009 August. We obtained
long-term one-day bin light curves of MAXI/GSC (Gas Slit Camera: Mihara et al.
(2011); Sugizaki et al. (2011)) 111$<$http://maxi.riken.jp/pubdata/v7l/$>$.
for 41 NS-LMXBs from 2009 August to 2021 December. We also analyzed the data
of the same sources observed with the ASM (All Sky Monitor: Levine et al.
(1996)) onboard RXTE (Rossi X-ray Timing Explorer: Bradt et al. (1993)) in
2–10 keV from 1996 February to 2011 December. The ASM data are obtained from
the archived results provided by the RXTE-ASM teams at MIT and NASA/GSFC.
222$<$http://xte.mit.edu/$>$.

The obtained count rates of GSC and ASM were converted to luminosities by
assuming a Crab-like spectrum (Kirsch et al., 2005) and the distance listed in
table 2. ASM data are converted to a flux in Crab unit using the nominal
relation of 1 Crab = 75 counts s-1 for ASM.
333$<$http://xte.mit.edu/XTE/ASM_lc.html$>$. GSC data are converted to a flux
in Crab unit using the nominal relation of 1 Crab = 3.45 photons s-1 cm-2 in
2–10 keV for GSC. 444$<$http://maxi.riken.jp/top/readme.html$>$. The
assumption of a Crab-like spectrum is acceptable in the hard state (low
luminosity $\leq 5\times 10^{36}$ erg s-1) because the energy spectrum is
approximated by a power law with a photon index of 1–2. On the other hand, in
the soft state (high luminosity $\geq 10^{37}$ erg s-1), the energy spectrum
is dominated by the thermal emission, and the luminosity obtained by assuming
Crab-like spectrum is underestimated in the 2–10 keV band (Asai et al., 2015).
In this paper, we do not care about it because we handle only the relative
difference.

We excluded following data in each source.

GX 3$+$1:

Data of MJD = 56187–56287 and 56652–56997 for contamination by Swift
J174510.8$-$262411\. Data of MJD = 55916–55922 for another south-west source.

SLX 1735$-$269:

Data of MJD = 56187–56287 and 56652–56997 for contamination by Swift
J174510.8$-$262411.

4U 1624$-$490:

Data of MJD = 58460–58580 for contamination by MAXI J1631$-$479.

4U 1708$-$40:

Data of the count rate above 1 photons cm-2 s-1 by solar X-ray leakage.

4U 1916$-$053:

Data of MJD = 55945–55951 and MJD = 56311–56315 by solar X-ray leakage.

4U 1323$-$619:

Data of MJD = 58509–58610 for contamination by MAXI J1348$-$630.

Ser X-1:

Data with the errors above 0.012 Crab in 2–10 keV.

We analyzed ASM and GSC data of the 41 NS-LMXBs and investigated the long-term
variability during $\sim 26$ yr (see table 1). The X-ray light curves are
shown in the Appendix. Here we used GSC data for the overlapped period during
MJD=55100–55800. Table 1 shows the flux ratio of ASM and GSC during the
overlapped period.

We estimated the luminosity in the 2–10 keV energy band from MJD=55100 to
MJD=59662 for the 41 NS-LMXBs using GSC data and listed in $L_{\rm ave}$.
High-quality persistent light curves are obtained from 33 sources. The
remaining eight sources are excluded from the following analyses. Six of
excluded eight sources are transients sources. The active period of HETE
J1900.1$-$2455 is too short ($\leq$10 yr) to investigate the long-term
variability. Other sources (EXO 1745$-$248, Aql X-1, XTE J1709-267, 4U
1608$-$522, and 4U 1745$-$203) are below the GSC detection limit during
quiescence. The other excluded two sources, SAX J1747.0$-$2853 and 4U
1724$-$307, have contamination from nearby sources.

KC10 indicated that all the 20 sources in their paper were considered to be
better fitted with a single sine wave than with a constant value. We also
focus on the properties of the sinusoidal variation. Here, we tried to
classify observed variations into five types as follows. The types are
described in the comment column of table 1.

CP (clear periodic variation):

Two bright Atoll sources (GX 3$+$1 and GX 9$+$1) show clear sinusoidal
variations. We define CP sources as can be fitted with a single sinusoidal
curve and a tilted line. We focus on these sources in the next session.

MP (modified periodic variation):

Seven sources (Ser X-1, 4U 1735$-$444, GX 9$+$9, 4U 1746$-$37, 4U 1708$-$40,
4U 1822$-$000, and 1A 1246$-$588) show the modified periodic variation. It is
difficult to fit their light curves with the periodic model functions. We also
focus on these sources in the next session.

NP (no periodic variation):

Nineteen sources show no periodic variation. Eight sources (Z sources and GX
13$+$1) of them show almost constant baseline although there are small
variations around baseline. Other eight sources (X-ray luminosities $<$ $\sim
3\times 10^{37}$ erg s-1) of them show almost constant. The remaining three
sources (4U 1543$-$624, 4U1916$-$053, and 1H 1556$-$605) show a decreasing
trend in luminosity.

FV (fast variation):

Two sources (4U1820$-$303 and 4U1705$-$440) have the luminosity change with a
shorter variability than 1 yr. The variation seems to have a different origin
from that of the sinusoidal variation investigated in this paper.

LV (large variation):

Three sources (Cir X-1, 4U 1636$-$536, and GS 1826$-$238) show the large
luminosity change of 1–2 orders of magnitude. Again, the large luminosity
change seems to have a different origin from that of the sinusoidal variation
investigated in this paper.

Figure 1a shows the average luminosity against the binary separation for 21
sources. The orbital periods of 21 sources are known among 33 sources. The
binary separation was estimated by Kepler’s third law assuming a neutron star
mass of 1.4 $\Mo$ and the mass of the donor star of 0.5 $\Mo$ although their
actual masses are uncertain. We also display the types in the figure.
Especially, filled marks indicate periodic variation, which are relevant to
the long-term variation investigated in this paper.

Figure 1b shows the irradiating flux on the donor star against the binary
separation. Here we treated the donor star as a point source and calculated
simply irradiation flux $F=L/4\pi d^{2}$, where $L$ and $d$ denote the average
luminosity and the binary separation, respectively.

Figure 1: (a) Average luminosity in the 2–10 keV against binary separation.
(b) Average irradiating flux on the donor star. We assumed the donor star mass
is 0.5 $M_{\odot}$. Filled squares MP (five sources), open circles NP, open
triangles FV, and open squares LV. Two CP sources (GX 3$+$1 and GX 9$+$1) are
not plotted since the orbital period is not known.

Next, we pick up CP and MP sources with periodic and modified periodic
variation. To confirm the property of variation, we also analyzed the archive
data of the X-ray All Sky Monitor (ASM: Tsunemi et al. (1989)) on board the
Ginga satellite (Makino & ASTRO-C Team, 1987). The Ginga/ASM data from 1987 to
1991 are obtained from the archived site of DARTS. 555
$<$https://data.darts.isas.jaxa.jp/pub/ginga/GINGA-ASM-1.2/$>$.  The 1–6 keV
counts s-1 are converted to the 2–10 keV flux assuming the spectrum of Crab
nebula. For 1A1246$-$588 (MP), there were no data of Ginga/ASM.

##  3 Results

Figure 2 shows the light curves of two CP sources (GX 3$+$1 and GX 9$+$1) with
the apparent sinusoidal variation from 1987 to 2021. First, we fitted the
light curves of GX 3$+$1 and GX 9$+$1 with a sinusoidal and linear function
model. The fitting parameters are shown in table 2.

Figure 2:  Light curves observed by Ginga/ASM, RXTE/ASM, and MAXI/GSC. For GX
3$+$1 and GX 9$+$1, we show the fitted sinusoidal curves in red (Color
online). The model function and parameters are shown in table 2. The data of
GX 9$+$1 has a discrepancy between ASM and GSC fluxes. MAXI data are processed
in a regular way as for other sources. There is no contamination source nor
background uncertainty by the ridge emission. So we plotted the data as they
are.

Fitting results of a sinusoidal and linear function model. Plots in figure 2.
Name Model**footnotemark: $*$ Average luminosity Sin Slope\S\Sfootnotemark:
$\S$ ($10^{36}$ erg s-1) $A${\dagger}{\dagger}footnotemark: ${\dagger}$
$P$\ddagger\ddaggerfootnotemark: $\ddagger$ (d) ($10^{36}$ erg s-1 d-1) GX
3$+$1 CONS$+$LINR$+$SIN 9.7 0.28 2046 $-4.81\times 10^{-4}$ GX 9$+$1
CONS$+$LINR$+$SIN 29.5 0.09 3542 $+1.90\times 10^{-5}$ **footnotemark: $*$
Model components. CONS: constant component, LINR: linear component, SIN:
sinusoidal component.
{\dagger}{\dagger}footnotemark: ${\dagger}$ Amplitude of sinusoidal component
against average luminosity.
{\ddagger}{\ddagger}footnotemark: ${\ddagger}$ Period of sinusoidal component.
\S\Sfootnotemark: $\S$ Slope of baseline.

Next, in order to investigate the variability of the sinusoidal profile of two
CP soures and seven MP sources, we fit each peak with a Gaussian profile
(figure 3 and 4). The parameters are the width (sigma) of Gaussian profile
(GW) and the peak luminosity (GN). Those correspond to the period and
amplitude of the sinusoidal variation, respectively. In order to see
variability of the peak luminosity and width, we plotted the GN against the GW
in figure 5. The amplitude and periods of two CP sources (GX 3$+$1 and GX
9$+$9) are more stable than those of MP sources. The average GWs for each
source are $\sim$ 300–2300 d, which correspond approximately to a half
sinusoidal period of GX 3$+$1 ($\sim$ 2.5 yr) and a period of GX 9$+$1 ($\sim$
10 yr), respectively.

Figure 3: Light curves of two CP sources fitted the gaussian peaks. Parameters
are shown in table 3. The fitting results were shown in red color curve.
(Color online)

Figure 4: Light curves of seven MP sources fitted the gaussian peaks.
Parameters are shown in table 3. The fitting results were shown in red color
curve. For 1A1246$-$588, there were no data of Ginga/ASM. (Color online)

Fitting results of gaussian components. Plots in figure 3 and 4.  Name Type
MJD GC**footnotemark: $*$ GW**footnotemark: $*$ GN**footnotemark: $*$ GX 3$+$1
CP 47000–48600 $47580\pm 28$ $985^{+70}_{-58}$ $15.4\pm 0.3$ 50000–51000
$50438\pm 4$ $546\pm 7$ $14.59\pm 0.05$ 51500–53300 $52282\pm 3$ $864\pm 5$
$16.44\pm 0.03$ 53700–55550 $54524\pm 2$ $678\pm 2$ $16.33\pm 0.04$
55550–57500 $56347\pm 3$ $784\pm 3$ $15.21\pm 0.04$ 57400–59400 $58410\pm 3$
$907\pm 4$ $9.98\pm 0.02$ GX 9$+$1 CP 50000–53600 $51595\pm 5$ $2718\pm 11$
$30.25\pm 0.03$ 53600–57000 $55490\pm 4$ $2141\pm 8$ $30.94\pm 0.03$
57000–59660 $58418\pm 5$ $2058\pm 13$ $31.20\pm 0.04$ Ser X-1 MP 46870–47700
$47220\pm 14$ $544\pm 34$ $36.1\pm 0.5$ 51000–52300 $51484\pm 6$ $1172\pm 17$
$34.73\pm 0.06$ 52600–55000 $53611\pm 5$ $1486\pm 9$ $37.42\pm 0.05$
55000–59650 $58778\pm 37$ $4132\pm 55$ $36.92\pm 0.06$ 4U 1735$-$444 MP
51000–55000 $52884\pm 3$ $1545\pm 5$ $36.46\pm 0.06$ 55000–56800 $55847\pm 3$
$708\pm 4$ $27.7\pm 0.1$ 56900–58300 $57575\pm 3$ $618\pm 5$ $24.4\pm 0.1$ GX
9$+$9 MP 51100–51900 $51383\pm 13$ $825\pm 36$ $13.91\pm 0.05$ 52400–53900
$53260\pm 8$ $1304\pm 21$ $15.77\pm 0.04$ 54100–55500 $54374\pm 12$ $1103\pm
14$ $17.70\pm 0.04$ 55500–56600 $56181\pm 6$ $846\pm 14$ $11.79\pm 0.03$
56600–5960 $57589\pm 39$ $4212\pm 140$ $12.02\pm 0.02$ 4U 1746$-$37 MP
50400–51080 $50824\pm 10$ $322\pm 18$ $9.1\pm 0.2$ 51080–51250 $51157\pm 5$
$92\pm 13$ $15\pm 1$ 51330–52050 $51597\pm 8$ $338\pm 14$ $13.0\pm 0.2$
52050–52440 $52248\pm 6$ $180\pm 13$ $8.8\pm 0.3$ 52470–53500 $52876\pm 6$
$375\pm 9$ $14.4\pm 0.3$ 53500–54800 $54164\pm 13$ $557\pm 22$ $8.4\pm 0.2$
55250–57400 $56216\pm 6$ $634\pm 8$ $11.5\pm 0.1$ 57500–58000 $57680\pm 5$
$183\pm 7$ $10.7\pm 0.2$ 4U 1708$-$40 MP 46850–48400 $47295^{+67}_{-101}$
$721^{+156}_{-98}$ $5.6\pm 0.4$ 50100–51050 $50343\pm 24$ $363\pm 21$ $4.1\pm
0.1$ 51300–52500 $52127\pm 13$ $478\pm 16$ $5.2\pm 0.1$ 52500–54150 $53380\pm
22$ $929^{+49}_{-43}$ $5.0\pm 0.1$ 54500–55800 $55418\pm 13$ $652\pm 19$
$5.87\pm 0.05$ 55830–56800 $56243\pm 8$ $582\pm 18$ $7.5\pm 0.1$ 56800–58950
$57379\pm 17$ $1057\pm 17$ $6.67\pm 0.04$ 4U 1822$-$000 MP 50700–54500
$52177^{+59}_{-67}$ $3322^{+191}_{-164}$ $2.71\pm 0.02$ 54800–55800 $55340\pm
13$ $665^{+35}_{-30}$ $2.78\pm 0.03$ 55800–57000 $55920^{+41}_{-50}$
$748^{+42}_{-36}$ $2.33\pm 0.04$ 57000–59000 $58103\pm 11$ $841\pm 18$
$1.90\pm 0.03$ 1A1246$-$588 MP 50600–52600 $53737^{+42}_{-46}$
$657^{+62}_{-52}$ $0.50\pm 0.03$ 56000–56700 $56315\pm 9$ $209\pm 11$ $0.58\pm
0.02$ 56700–57900 $57218\pm 17$ $483\pm 26$ $0.59\pm 0.01$ 57900–59500
$58128^{+74}_{-98}$ $807^{+59}_{-63}$ $0.47\pm 0.02$ **footnotemark: $*$ GC is
the center MJD of gaussian component, GW is the width (sigma) of gaussian
component, and GN is the center luminosity of gaussian component.

Figure 5:  The center luminosity of Gaussian model (GN) as a function of the
width of Gaussian model (GW). Filled circles represent data of CP sources.
Filled triangles and open squares represent data of MP sources. To avoid the
confusion of data points, we use two kind of marks.

The individual properties in the nine sources are as follows:

GX 3$+$1 (figure 2a and 3a):

The sinusoidal variation is clear, although the shapes of sinusoidal profiles
are complex with some peaks. The baseline decreases. The long-term period is
$\sim 6$ yr, which is similar to the results of KC10. The long-term periodic
variation turned to be stable over the twice time-length in KC10.

GX 9$+$1 (figure 2b and 3b):

The sinusoidal variation is also clear but with a little longer period ($\sim
11$ yr) than that of GX 3$+$1\. The baseline has gradually increased in $\sim
25$ yr, which is against the KC10 report that the baseline was constant for
$\sim 13$ yr observed by RXTE/ASM. The period of long-term is similar to the
results ($\sim 12$ yr) of KC10.

Ser X-1 (figure 4a):

Quasi periodic modulation is clear. The period is not constant, but seems to
become longer. KC10 reported the period of $\sim 7.3$ yr observed by RXTE/ASM.
In the MAXI/GSC era, the period is longer and amplitude may be larger.

4U 1735$-$444 (figure 4b):

The sinusoidal variation is clear. However, the period is not constant, but
seems to become shorter. KC10 reported the period of $\sim 10$ yr from the
large hump around MJD=53000. In the MAXI/GSC era, the period is shorter, about
a half, and amplitude may be smaller.

GX 9$+$9 (figure 4c):

Although KC10 reported that the baseline was increasing, it cannot be
extrapolated in the MAXI/GSC era. The light curve dropped after RXTE/ASM.
Since then the baseline has stayed almost constant. However, the long-term
period is similar to the results ($\sim 4$ yr) of KC10 throughout by RXTE/ASM
and MAXI/GSC. The amplitude of the sinusoidal variation seems to be smaller in
the MAXI/GSC era.

4U 1746$-$37 (figure 4d):

Quasi periodic modulation is clear. However, the period and the amplitude is
not constant. KC10 reported the period of $\sim 4.36$ yr observed by RXTE/ASM.

4U 1708$-$40 (figure 4e):

Quasi periodic modulation is clear. However, the period and the amplitude is
not constant. Although we divided the data into small peak with short period,
the data of MAXI/GSC era can also be fitted with one gaussian profile for
large one peak. KC10 did not report this source, because they focused on the Z
sources and Atoll sources.

4U 1822$-$000 (figure 4f):

Quasi periodic modulation is clear. However, the period and the amplitude is
not constant. Although we divided the data between MJD=54800 and 57000 into
two small peak with short period, the data can also be fitted with one
gaussian profile for large one peak. KC10 did not report this source.

1A 1246$-$588 (figure 4g):

Quasi periodic modulation is seen. However, the period and the amplitude is
not constant. KC10 did not report this source.

##  4 Discussion

The long-term sinusoidal variations were presented for the two CP sources (GX
3$+$1 and GX 9$+$1). The long-term periods range in $\sim 5$ yr and $\sim 10$
yr as shown in table 2–3 and figure 2–3. We also investigated the seven MP
sources with modified periodic variation, and estimated periods of quasi-
periodic modulation (table 3 and figure 4). The rage of each average period
was approximately from $\sim$ 2.5 yr to $\sim$ 10 yr. These time scales are
much longer than the orbital period ($\sim$ 2–5 hr), although, strictly
speaking, those of the four sources (CP: GX 3$+$1, GX 9$+$1 and MP:4U
1708$-$40, 1A 1246$-$588) are unknown.

We discuss mechanisms of long-term variation. First, we focus on the physical
motion of the accretion disk. Precession of accretion disk may occur due to
excitation of resonances in case of mass ratio of $q=M_{\rm c}/M_{\rm NS}\sim$
0.25–0.33 (Whitehurst & King, 1991). It is likely to occur in the case of
LMXBs with donor mass of 0.35$\Mo$–0.46$\Mo$. Although the donor mass of our
CP and MP sources are not known, precession of accretion disk could be
possible. In general, according to Inoue (2012), precession occurs when the
ratio of the precession period $P_{\rm p}$ to the binary orbital period
$P_{\rm B}$ is $P_{\rm p}/P_{\rm B}$ = 10–100. The ratios of our results are
$P_{\rm p}/P_{\rm B}\geq 10^{4}$. There is no such case in figure 2 of Inoue
(2012). However, if we extrapolate one line for $P_{\rm p}/P_{\rm B}=10^{4}$,
the range of donor mass for 0.14$\Mo$–0.42$\Mo$ ($q\sim$0.1–0.3) would be
possible. Since donor mass is not known, the disk precession scenario could be
possible.

On the other hand, another possibility of radiation-induced warping is
excluded. Kotze and Charles (2012) explicitly discuss the stability of
radiation-induced warping. In the region below the bottom solid line in their
figure 1, a radiation-induced warping of the disk is unlikely, and there is in
fact no super-orbital cycle observed in this region. Our four sources lie
below the bottom solid line for the typical values of mass ratio: $q=M_{\rm
c}/M_{\rm NS}\sim 0.3$ and orbital period $\sim 2-5$ hr. Thus the radiation-
induced warping of the accretion disk is unlikely.

Next, we consider the possibility of the variation of mass transfer rate from
the donor. KC10 suggested that they are consequence of the solar-like magnetic
cycles seen in the late-type star (Applegate & Patterson (1987), Warner
(1988)). They pointed out that the flux modulation of a sin wave of $\leq 30$
percent is plausible by the magnetic cycles. In our result, the flux
modulation (see the amplitude of table 2) is $\leq 30$ percent. Thus the solar
cycle-like variations can be responsible for its long-term variations.

Here, we discuss another possibility of the variation of mass transfer rate
from the donor. It is the irradiation by the central X-ray source to the donor
star. Ritter (2008) showed that irradiation-driven mass transfer cycles could
only occur when the irradiation is sustained for a sufficiently long time.
Büning and Ritter (2004) shows the numerical results in terms of irradiation-
driven mass transfer cycles and then indicates the possibility that NS-LMXB
undergoes those cycles. Their numerical results show that the stability of the
mass transfer rate from the donor star is a function of the orbital period.

In figure 1, filled marks represent quasi periodic modulation (MP: Ser X-1, 4U
1735$-$444, GX 9$+$9, 4U 1746$-$37, 4U 1822$-$000) in our analysis. In figure
1b, the filled marked five MP sources are located in the region which is
middle in the binary separation and high irradiation average flux. The region
may tend to show a periodic modulation.

However, in the region, there are also three sources (GS 1826$-$238, 4U
1636$-$536, and 4U 1254$-$690) of non-periodic modulation. The 2 LV sources
(GS 1826$-$238 and 4U 1636$-$536) show the large luminosity change of 1–2
orders of magnitude, and it is difficult to see the flux modulation of the
amplitude ( $\leq 30$ percent ) which a change of the mass transfer rate
induces. The one NP source (4U 1254$-$690) has similar binary properties to
that of GX 9$+$9 (CP). The cause of no clear periodic variation is unclear.

We discuss the features of the region in which a long-term variation trend to
occur. One feature is high irradiation average flux ($\geq 1\times 10^{13}$
erg s-1 cm-2). Even in the high irradiation-flux, there are four NP sources
(Sco X-1, GX 349$+$2, LMC X-2 and 4U1624$-$490) without a long-term variation
in the larger binary-separation region. Here, the three sources (Sco X-1, GX
349$+$2, and LMC X-2) are Z sources. KC10 reported that the amplitudes of
long-term variation of the Z sources are small and noted that it is because
their luminosity is close to the Eddington. Although the luminosity of 4U
1624$-$490 is not close to Eddington luminosity, the intrinsic luminosity is
uncertain because it is an ADC source. Here, we focus on the donor star. The
three sources (Sco X-1, GX 349$+$2 and LMC X-2) are Z sources, and the donor
stars are suggested to be evolved stars (Hasinger & van der Klis (1989) for Z
sources, Cherepashchuk et al. (2021) for Sco X-1). 4U 1642$-$490 did not
belong to Z sources. However, Jones and Watson (1989) reported that the
flaring behavior of the source shows similarity to the flaring blackbody
component (in temperature and radius) of Sco X-1 and other Z sources. Although
the donor star of 4U 1624$-$490 is not identified, it is possible to be an
evolved star, similar to the Z sources. In this case, the mass transfer rate
on to the neutron star may be above the Eddington luminosity.

On the other hand, in the high irradiation-flux, but in the smaller binary-
separation region, there are four sources (FV: 4U 1820$-$303 and three NPs: 4U
1543$-$624, 4U 2127$+$119, and 4U 1916$-$053). The sources are Ultra Compact
X-ray Binaries (UCXBs). The donors in UCXBs may be white dwarfs (WDs) or He
stars (4U 1820$-$303: Rappaport et al. (1987), 4U 1543$-$624: Nelemans et al.
(2004), 4U 2127$+$119: Dieball et al. (2005), and 4U 1916$-$053: Joss (1978),
Nelemans et al. (2006)). Lü et al. (2017) suggests that, if the donor star is
a WD, the irradiation flux can only penetrate into a very thin layer of the WD
surface, and the irradiation hardly affects the evolution of the persistent
UCXB. The irradiation flux would not affect the mass transfer rate from the
donor star.

In summary, the intense irradiation may induce variation of the mass transfer
rate, and a sinusoidal periodic variation may appear. Our results also seem to
show a dependence on the donor. A WD donor would not be affected by the
irradiation. The periodicity might be related to the mass transfer cycles
caused by the irradiation pointed out by Ritter (2008).

## Light curves of 41 NS-LMXB observed with MAXI/GSC and RXTE/ASM

The light curve used to analyze the long-term variation are presented in this
Appendix (figure 6–14). The energy band is 2–10 keV band, and th e period of
data is from 1996 February to 2021 December. The left side of the vertical
dash line is the data of RXTE/ASM, and the right is MAXI/GSC.

Figure 6: Light curve in 2–10 keV of sources with Z sources and GX 13$+$1 (NP:
no periodic variation). The left side of the vertical dash line was the data
of RXTE/ASM, and the right was MAXI/GSC. The MAXI/ASM flux ratio is not
adjusted.

Figure 7: As figure 6, but for CP (clear periodic variation) sources.

Figure 8: As figure 6, but for MP (modified periodic variation) sources.

Figure 9: As figure 6, but for NP (no periodic variation) and almost constant
sources.

Figure 10: As figure 6, but for NP (no periodic variation) and decrease
sources.

Figure 11: As figure 6, but for FV (fast variability) sources.

Figure 12: As figure 6, but for LV (large variability) sources.

Figure 13: As figure 6, but for Transient sources.

Figure 14: As figure 6, but for the source with contamination.

## References

  * Applegate & Patterson (1987) Applegate, J. H. & Patterson, J. 1987, ApJ, 322, L99
  * Asai et al. (2015) Asai, K., Mihara, T., Matsuoka, M., & Sugizaki, M. 2015, PASJ, 67, 92
  * Baliunas et al. (1995) Baliunas, S. L., Donahue, R. A., Soon, W. H., et al. 1995, ApJ, 438, 269
  * Barret et al. (2000) Barret, D., Olive, J. F., Boirin, L., Done, C., Skinner, G. K., & Grindlay, J. E. 2000, ApJ, 533, 329
  * Barret (2001) Barret, D. 2001, Advances in Space Research, 28, 307
  * Büning & Ritter (2004) Büning, A. & Ritter, H. 2004, A&A, 423, 281
  * Bradt et al. (1993) Bradt, H. V., Rothschild, R. E., & Swank, J. H. 1993, A&AS, 7, 355
  * Christian and Swank (1997) Christian, D. J., & Swank, J. H. 1997, ApJS, 109, 177
  * Charles et al. (2008) Charles, P.A.,, Clarkson, W., Cornelisse, R. & Shih, C. 2008, New Astron. Rev., 51, 768
  * Cornelisse et al. (2013) Cornelisse, R., Casares, J., Charles, P. A., et al. 2013, MNRAS, 432, 1361
  * Cherepashchuk et al. (2021) Cherepashchuk, A. M., Khruzina, T. S., & Bogomazov, A. I. 2021, MNRAS, 508, 1389
  * Cornelisse et al. (2007) Cornelisse, R., Steeghs, D., Casares, J., Charles, P. A., Shih, I. C., Hynes, R. I. & O’Brien, K. 2007, MNRAS, 381, 194
  * D’Ai et al. (2012) D’Ai et al. 2012, A&A, 543, 20
  * Dieball et al. (2005) Dieball, A., Knigge, C., Zurek, D. R., et al. 2005, ApJ, 634, L105
  * Freedman et al. (2001) Freedman, W. L. et al. 2001, ApJ, 553, 47
  * Galloway et al. (2008) Galloway, D. K., Muno, M. P., Hartman, J. M., Psaltis, D., & Chakrabarty, D. 2008, ApJS, 179, 360
  * Gambino et al. (2016) Gambino et al. 2016, A&A, 58, 34
  * Hasinger & van der Klis (1989) Hasinger, G. & van der Klis, M. 1989, A&A, 225, 79
  * Harris (1996) Harris, W. E. 1996, AJ, 112,1487
  * Inoue (2012) Inoue, H. 2012, PASJ, 64, 40
  * in’t Zand et al. (2003) in’t Zand, J. J. M., Kuulkers, E., Verbunt, F., Heise, J. & Cornelisse, R. 2003, A&A, 411, L487
  * Jones and Watson (1989) Jones, M. H. & Watson, M. G. 1989, Two Topics in X-Ray Astronomy, Volume 1: X Ray Binaries. Volume 2: AGN and the X Ray Background, 1, 439
  * Jonker and Nelemans (2004) Jonker, P. G., & Nelemans, G. 2004, MNRAS, 354, 355
  * Jonker et al. (2007) Jonker, P. G., in’t Zand, J. J. M., Méndez, M., & van der Klis, M., 2007, MNRAS, 378, 1187
  * Joss (1978) Joss, P. C. 1978, ApJ, 225, L123
  * Kirsch et al. (2005) Kirsch, M. G.,et al. 2005, Proc. SPIE, 5898, 22
  * Kotze & Charles (2010) Kotze, M.M., & Charles, P.A. 2010, MNRAS, 402, L16 (KC10)
  * Kotze & Charles (2012) Kotze, M.M., & Charles, P.A. 2012, MNRAS, 420, 1575
  * Kuulkers and van der Klis (2000) Kuulkers, E., & van der Klis, M. 2000, A&A, 356, L45
  * Kuulkers et al. (2010) Kuulkers et al. 2010, A&A, 514, 65
  * Levine et al. (1996) Levine, A. M., Bradt, H., Cui, W., Jernigan, J. G., Morgan, E. H., Remillard, R., Shirey, R. E., & Smith, D. A. 1996, ApJ, 469, L33
  * Lin et al. (2012) Lin, D., Remillard, R. A., Homan, J., & Barret, D. 2012, ApJ, 756, 34
  * Liu et al. (2007) Liu, Q. Z., van Paradijs, J., & van den Heuvel, E. P. J. 2007, A&A, 469, 807
  * Lü et al. (2017) Lü, G., Zhu, C., Wang, Z., et al. 2017, ApJ, 847, 62
  * Ludlam et al. (2017) Ludlam, R. M., Miller, J. M., Cackett, E. M., Degenaar, N., & Bostrom, A. C. 2017, ApJ. 838, 79
  * Makino & ASTRO-C Team (1987) Makino, F. & ASTRO-C Team 1987, Astrophys. Lett., 25, 223
  * Matsuoka et al. (2009) Matsuoka, M., et al.2009, PASJ, 61, 999
  * Mihara et al. (2011) Mihara, T., et al. 2011, PASJ, 63, S623
  * Natalucci et al. (2000) Natalucci, L., Bazzano, A., Cocchi, M., Ubertini, P., Heise, J., Kuulkers, E., & in’t Zand, J. J. M. 2000, ApJ, 543, L73
  * Nelemans et al. (2004) Nelemans, G., Jonker, P. G., Marsh, T. R., et al. 2004, MNRAS, 348, L7
  * Nelemans et al. (2006) Nelemans, G., Jonker, P. G., & Steeghs, D. 2006, MNRAS, 370, 255
  * Rappaport et al. (1987) Rappaport, S., Nelson, L. A., Ma, C. P., et al. 1987, ApJ, 322, 842
  * Revnivtsev et al. (2011) Revnivtsev, M., Postnov, K., Kuranov, A., & Ritter, H. 2011, A&A, 526, 994
  * Ritter (2008) Ritter, H. 2008, New Astronomy Reviews, 51, 869
  * Shahbaz et al. (2007) Shahbaz, T., Watson, C. A., & Hernandez-Peralta, H. 2007, MNRAS, 376, 1886
  * Seifina & Titarchuk (2012) Seifina, E., & Titarchuk, L. 2012, ApJ, 747, 99
  * Sugizaki et al. (2011) Sugizaki, M., et al. 2011, PASJ, 63, S635
  * Tsunemi et al. (1989) Tsunemi, H., Kitamoto, S., Manabe, M., et al. 1989, PASJ, 41, 391
  * Valenti et al. (2007) Valenti, E., Ferraro, F. R., & Origlia, L. 2007, AJ, 133, 1287
  * Warner (1988) Warner, B. 1988, Nature, 336, 129
  * White and Angelini (2001) White, N. E., & Angelini, L. 2001, ApJ, 561, L101
  * Whitehurst & King (1991) Whitehurst, R. & King, A. 1991, MNRAS, 249, 25
  * Xiang et al. (2007) Xiang, J., Lee, J. C., & Nowak, M. A. 2007, ApJ, 660, 1309


# Talking with Machines: A Comprehensive Survey of Emergent Dialogue Systems

William Tholke
University of California, Berkeley
<EMAIL_ADDRESS>
###### Abstract

From the earliest experiments in the 20th century to the utilization of large
language models and transformers, dialogue systems research has continued to
evolve, playing crucial roles in numerous fields. This paper offers a
comprehensive review of these systems, tracing their historical development
and examining their fundamental operations. We analyze popular and emerging
datasets for training and survey key contributions in dialogue systems
research, including traditional systems and advanced machine learning methods.
Finally, we consider conventional and transformer-based evaluation metrics,
followed by a short discussion of prevailing challenges and future prospects
in the field.

##  1 Introduction

The early 1960s saw the start of research into dialogue systems, advanced
programs designed to emulate human-like interactions in two-way conversation
with users111Word count: 2190.. These early dialogue systems laid the
foundation for monumental advancements in the field of natural language
processing (NLP), giving rise to models such as ELIZA, PARRY, and GPT, the
highly sophisticated large language model developed by OpenAI.

Modern dialogue systems have proven their utility in a wide array of fields,
including but not limited to academia, customer service, healthcare, and
entertainment. Nevertheless, despite their ubiquity, navigating the
complexities of these systems and their underlying models can be a daunting
task.

This paper aims to demystify dialogue systems, starting with a brief history
of their development, followed by a discussion of their underlying processes.
We describe popular and emerging corpora used for training and survey
significant contributions in the realm of dialogue systems research, including
both traditional and cutting-edge machine learning systems and techniques. We
further explore the evaluation metrics employed to assess system performance,
and finish with a consideration of the challenges and future prospects in the
field of dialog systems research.

##  2 Historical Development

In his landmark 1966 paper, Joseph Weizenbaum of the Massachusetts Institute
of Technology (MIT) AI Laboratory introduced ELIZA, the first rule-based
dialogue system for emulating human conversation. Weizenbaum’s ELIZA
implementd rule-based "scripting," identifying the most significant keyword in
the input sequence, finding minimal context surrounding the keyword, and
applying its associated rule to generate a response222For instance, when given
the sentence ”I am very unhappy these days,” ELIZA may detect the keywords ”I
am” to be of the structure ”I am [predicate]” and then transform the input
text to the output ”How long have you been [predicate]?” (Weizenbaum, 1966).
for the user (Weizenbaum, 1966). Then came PARRY, the dialogue system
developed by Kenneth Mark Colby along with graduate students from Stanford
University and the University of California, Los Angeles (UCLA). PARRY was a
rule-based system that was designed to model the thinking and behavior
patterns of paranoid psychiatric patients, but unlike ELIZA, it contained
advanced parsing and interpretation-action modules, allowing the system to
make inferences about the beliefs and intentions of the user as well as
maintain an internal state representation333See 3.2: Dialogue State Tracking &
Management. (Colby, 1981).

As soon as 1990, researchers began to consider the use of statistical methods,
already proven useful in automatic speech recognition and lexicography, for
NLP tasks. IBM researcher Peter Brown implemented a machine translation model
that assigned probabilities to sentence pairs, allowing for the use of Bayes’
theorem to compute translation probabilities. However, roughly $89$ percent of
the available multilingual text data was insufficient for training the model’s
parameters, contributing to the system’s limited success rate of $48$ percent
(Brown et al., 1990).

As the 20th century progressed, the availability of computing power and
diverse text data grew substantially, leading to the development of more
advanced corpus-based and data-driven dialogue systems Serban et al. (2017).
These systems, which leverage incredibly large corpora derived from real-world
data,444See 4: Dialogue System Datasets. remain the state-of-the-art in
dialogue systems research.

##  3 Dialogue System Tasks

Before discussing the application of large corpora in dialogue systems, it is
essential to first examine the tasks that most state-of-the-art systems are
designed to perform. In doing so, we provide important context for a better
understanding of the varying architectures that are used to implement them.

###  3.1 Natural Language Understanding (NLU)

Natural language understanding (NLU) refers to the set of tasks that involve
the processing and interpretation of natural language text input. This
includes tokenization, part-of-speech tagging, dependency parsing, and named
entity recognition, among others.

Tokenization is a fundamental task that splits the input text into
constitutent tokens, such as words or numbers, and removes meaningless units
of text like punctuation and non-textual characters. These tokens may
represent unique objects or concepts, such as people, pronouns, events, dates,
places, and so on, called named entities.

Part-of-speech (POS) tagging is essential for assigning a grammatical POS
tag–NN for nouns, VB for verbs, JJ for adjectives, etc.–to each of these
tokens. There is also the task of identifying the syntactic relationship(s)
between tokens, known as dependency parsing, where the system predicts the
token that governs the grammatical structure of a sentence (Yu et al., 2020).
This is particularly useful for named entity recognition, the process of
identifying and classifying named entities, which has been greatly improved
with the advent of Bidirectional Encoder Representations from Transformers
(BERT) Yu et al. (2020).

###  3.2 Dialogue State Tracking & Management

Dialogue state tracking and management is essential for every dialogue system,
as it involves keeping track of the user’s goals in dialogues. This task has
typically been restricted to unimodal input, where specific slots for
placeholders of information, called slot-value pairs, are defined by specific
database schema and limited to specific knowledge domains. However, recent
advances in multimodal state tracking, which utilizes multiple
modalities555The term ”modalities” refers to types or channels of input such
as text, speech, video, and so on. of input, have demonstrated higher F1
metrics and overall performance gains for each individual modality Le et al.
(2022).

###  3.3 Natural Language Generation (NLG)

Natural language generation (NLG) can be described as the critical processes
related to converting a dialog system’s internal representation of data into
natural language text output. One of these processes is content determination,
the identification of appropriate domain or subject matter needs for the
generation of output text. After content determination, the system is then
able to perform lexicalization, the selection of suitable words to express the
contents of the message, and document structuring, to create a word-ordering
for the output text.

In addition to these tasks, it is crucial to distinguished named entities from
one another. This is accomplished by referring expression generation (REG),
which is often coupled with sentence aggregation, the process of constructing
a clear and readable text output through the removal of redundant information
Santhanam and Shaikh (2019).

##  4 Dialogue System Datasets

In our analysis of text-based datasets for training dialog systems, we examine
both popular and emerging public datasets.

###  4.1 Schema Guided Dialogue (SGD)

The Schema-Guided Dialogue (SGD) dataset, released by Google Research in 2020,
offers a challenging testbed for dialogue systems with more than 16,000 multi-
domain conversations from 26 services and APIs across 16 domains. As one of
the largest public task-oriented dialogue corpora, it includes evaluation sets
that contain services not present in the training set, providing a valuable
opportunity to assess model performance on previously unseen services. In
total, the SGD dataset is comprised of 16,142 dialogues, 329,964 turns, and
30,352 unique tokens, along with 214 and 14,139 slots and slot-values,
respectively (Rastogi et al., 2020).

###  4.2 MultiWoZ & GlobalWoZ

The Multi-Domain Wizard-of-Oz (MultiWoZ) dataset is a large collection of
human-to-human conversations that captures natural conversations between
tourists and information center clerks in touristic cities. With 10,438
dialogues, 115,424 turns, and a total of 1,520,970 tokens, alongside 25 slots
and 4,510 slot-values, the MultiWoZ dataset is slightly smaller in magnitude
than the SGD dataset (Budzianowski et al., 2018; Rastogi et al., 2020).

GlobalWoz, which is based on MultiWoZ, is a multilingual task-oriented
dialogue (ToD) dataset that is characterized by its ability to accommodate
foreign speakers using ToD in foreign-language and English-speaking countries.
This dataset expanded the potential applications of the existing multi-domain
dataset beyond the standard application of an English speaker using ToD in an
English-speaking country (Ding et al., 2022).

###  4.3 SciNLI & SciBERT

The newly developed SciNLI corpus, designed for natural language inference
(NLI), is unique in its ability to capture formality in scientific writing.
Comprised of 107,412 sentence pairs extracted from academic papers on NLP and
computational linguistics, SciNLI is notably smaller in size than the SNLI and
MNLI datasets, which consist of 570,152 and 432,702 sentence pairs Bowman et
al. (2015); Williams et al. (2018).

The corpus is unique in that it provides a comprehensive exploration of the
various types of inferences found in scientific writing. As noted by Sadat and
Caragea (2022), SciNLI still has lots of room for improvement, having achieved
a macro-averaged F1 score666The macro-averaged F1 score is computed by taking
the average of the F1 scores across all classes in a multi-class
classification problem. See 6: Evaluation Metrics for other notable metrics.
of only 78.18 percent.

Another noteworthy mention is SciBERT, a BERT-based pre-trained language model
that addresses the shortage of large-scale, high-quality labeled scientific
data Beltagy et al. (2019).

###  4.4 The Pile

The Pile is a massive 825-gigabyte English text corpus that was built to
facilitate the training of large-scale language models. It comprises 22
diverse and high-quality datasets, including those that are popular, such as
Project Gutenberg (PG-19) (Rae et al., 2019) and Open-Subtitles (Tiedemann,
2016), and those that are new, such as the 56.21 and 95.126 gigabytes of raw
data collected from GitHub and ArXiv, respectively.

##  5 Approaches to Dialogue Systems

We present an overview of various approaches to developing dialog systems,
including both traditional and deep learning methods.

###  5.1 Traditional Systems

####  5.1.1 Rule-based

Rule-based dialogue systems are characterized by their utilization of
predefined scripts or templates and can be either script-based, such as ELIZA,
or production-based, encoding rules as "if-then" statements. At a fundamental
level, these systems operate by matching a token in the input text to a
corresponding rule in order to generate a response.

While dialogue flows in these systems are pre-determined by hard-coded rules,
as explained by (Ni et al., 2023), these rules consistently yield high-
quality, controlled responses (Liu and Mei, 2020). Unfortunately, rule-based
systems are outperformed by statistical and machine learning methods, which
can generalize better to unseen states (Ni et al., 2023; Lemon and Pietquin,
2007).

####  5.1.2 Retrieval-based

Retrieval-based dialogue systems search through a database of dialogues,
selecting responses that align most closely with the given context. Due to
their small set of hand-tuned parameters, these systems are capable of
generating sensible responses to queries without the need for human
annotation. However, just as with rule-based systems, they perform poorly in
generalizing to unseen states (Serban et al., 2017). Implementing pre-trained
language models like BERT can help improve this poor performance (Han et al.,
2021).

###  5.2 Machine Learning Methods

####  5.2.1 Convolutional Neural Networks (CNNs)

A subset of Deep Neural Networks (DNNs), Convolutional Neural Networks (CNNs)
are powerful multi-layered models that are adept at transforming multimodal
input into output classifications.777The reader may consult (Zeiler and
Fergus, 2013) for detailed visualizations and constructions of CNNs. In the
context of dialogue systems, CNNs are typically made up of a number of layers,
which we describe simply.

The primary purpose of the first layer is to accept and transmute textual
input into numerical data, passing it to the convolutional layer, where
filters are applied to form feature maps. Following this, non-linearity is
introduced by the ReLU activitation in the Rectified Linear Unit (ReLU). The
pooling layer then downsamples the feature maps to reduce computational
complexity, which are then flattened into a one-dimensional vector in the
fully connected layer. The output layer takes these processed features and
computes the final outputs Ni et al. (2023).

####  5.2.2 Recurrent Neural Networks (RNNs)

Recurrent Neural Networks (RNNs) differ from CNNs in that they operate
sequentially rather than in parallel. With a hidden layer that sustains a form
of memory across time steps, RNNs are highly effective for modeling temporal
dependencies in dialogue and preserving context throughout conversations (Ni
et al., 2023). However, RNNs may suffer from the problem of vanishing or
exploding gradients, which happens when errors are backpropogated until they
evolve exponentially. Moreover, RNNs tend to struggle with modeling long-term
dependencies (Hochreiter et al., 2001).

To address these issues, Long Short-Term Memory (LSTM) models were introduced
Hochreiter and Schmidhuber (1997), which leverage gating mechanisms to
overcome problems with gradients. LSTMs inspired the development of the Gated
Recurrent Unit (GRU) model888See also (Sutskever et al., 2014). (Cho et al.,
2014). The reader may also be interested in sequence-to-sequence learning with
DNNs, as described in (Sutskever et al., 2014).

####  5.2.3 Generative Pre-trained Transformers (GPT) for Dialogue Systems

The third iteration in OpenAI’s Generative Pre-trained Transformer (GPT)
series, known as GPT-3, is a massive autoregressive language model that
demonstrates exceptional performance in dialogue-related tasks. At its release
on June of 2020, GPT-3 had 175 billion parameters and was one of the largest
language models available. (Brown et al., 2020).

OpenAI’s latest model, GPT-4, surpassed the last model and as such is even
better at performing dialogue-related tasks, as is shown by its novel
implementation in ChatGPT (OpenAI, 2023).

##  6 Evaluation Metrics

Conventional automatic language evaluation metrics such as BLEU Papineni et
al. (2002) and METEOR Banerjee and Lavie (2005) are commonly used to assess
the performance of dialog systems. While BLEU is a rule-based metric, it is
limited in its ability to reflect grammatical and semantic nuances while
preserving sentence meaning. Similarly, METEOR, while consisting of the
additional features of synonymy and stemming, is also limited in its ability
to evaluate dialogue system output (Liu et al., 2016).

Recent research has introduced dialog-specific evaluation metrics that exhibit
stronger correlations with human judgments than existing metrics. One such
metric is FrugalScore, developed by OpenAI, which learns a low-cost version of
any expensive NLG evaluation metric. FrugalScore maintains 96.8 percent of the
original metric’s performance, has 25 less parameters, and runs 24 times
faster (Kamal Eddine et al., 2022). In addition, large pre-trained language
models, such as RoBERTa (Liu et al., 2019), the variant of BERT that was
trained on ten times more data,999RoBERTa has shown better performance in
dialog-specific metrics Liu et al. (2019). are commonly employed in these
evaluation metrics.

The majority of these metrics101010(Yeh et al., 2021) gives an overview of
roughly two dozen dialog-specific metrics. rely on human evaluation, which can
be expensive, time-consuming, and prone to subjective inconsistencies Smith et
al. (2022). Thus, as proposed by (Reddy, 2022), it is imperative that
alternative metrics be developed to reduce reliance on human evaluation,
although it is still useful for assessing performance (Ghandeharioun et al.,
2019).

##  7 Concluding Remarks

Dialogue systems research has come a long way since the first rule-based
system was built in 1966, trending away from high-maintenance rule sets in
smaller models towards self-sufficient data-driven models. As the field leans
in this direction, there is a rising need for more robust evaluation metrics
and methods for mitigating bias in model responses.

Moreover, as we get closer to developing true Artificial General Intelligence
(AGI), it is important that state-of-the-art models limit the risks of model
hallucinations, disinformation, cybersecurity threats, and overreliance OpenAI
(2023). Although our best models struggle with factual accuracy, self-
contradiction, and maintaining character identity, as examined in great detail
by Shuster et al. (2022), there will come a day when a language model that
underlies one of these dialogue systems will give sparks of AGI. Thus, as more
powerful dialogue systems are developed, it is crucial that researchers keep
them aligned with our ethical and moral values.

## References

  * Banerjee and Lavie (2005) Satanjeev Banerjee and Alon Lavie. 2005.  METEOR: An automatic metric for MT evaluation with improved correlation with human judgments.  In _Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization_ , pages 65–72, Ann Arbor, Michigan. Association for Computational Linguistics.
  * Beltagy et al. (2019) Iz Beltagy, Kyle Lo, and Arman Cohan. 2019.  SciBERT: A pretrained language model for scientific text.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pages 3615–3620, Hong Kong, China. Association for Computational Linguistics.
  * Bowman et al. (2015) Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Manning. 2015\.  A large annotated corpus for learning natural language inference.  In _Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing_ , pages 632–642, Lisbon, Portugal. Association for Computational Linguistics.
  * Brown et al. (1990) Peter F. Brown, John Cocke, Stephen A. Della Pietra, Vincent J. Della Pietra, Fredrick Jelinek, John D. Lafferty, Robert L. Mercer, and Paul S. Roossin. 1990\.  A statistical approach to machine translation.  _Computational Linguistics_ , 16(2):79–85.
  * Brown et al. (2020) Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020.  Language models are few-shot learners.
  * Budzianowski et al. (2018) Paweł Budzianowski, Tsung-Hsien Wen, Bo-Hsiang Tseng, Iñigo Casanueva, Stefan Ultes, Osman Ramadan, and Milica Gašić. 2018.  MultiWOZ - a large-scale multi-domain Wizard-of-Oz dataset for task-oriented dialogue modelling.  In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_ , pages 5016–5026, Brussels, Belgium. Association for Computational Linguistics.
  * Cho et al. (2014) Kyunghyun Cho, Bart van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014.  Learning phrase representations using RNN encoder–decoder for statistical machine translation.  In _Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 1724–1734, Doha, Qatar. Association for Computational Linguistics.
  * Colby (1981) Kenneth Mark Colby. 1981.  Modeling a paranoid mind.  _Behavioral and Brain Sciences_ , 4(4):515–560.
  * Ding et al. (2022) Bosheng Ding, Junjie Hu, Lidong Bing, Mahani Aljunied, Shafiq Joty, Luo Si, and Chunyan Miao. 2022.  GlobalWoZ: Globalizing MultiWoZ to develop multilingual task-oriented dialogue systems.  In _Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 1639–1657, Dublin, Ireland. Association for Computational Linguistics.
  * Ghandeharioun et al. (2019) Asma Ghandeharioun, Judy Hanwen Shen, Natasha Jaques, Craig Ferguson, Noah Jones, Agata Lapedriza, and Rosalind Picard. 2019.  Approximating interactive human evaluation with self-play for open-domain dialog systems.
  * Han et al. (2021) Janghoon Han, Taesuk Hong, Byoungjae Kim, Youngjoong Ko, and Jungyun Seo. 2021.  Fine-grained post-training for improving retrieval-based dialogue systems.  In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 1549–1558, Online. Association for Computational Linguistics.
  * Hochreiter et al. (2001) Sepp Hochreiter, Yoshua Bengio, Paolo Frasconi, and Jürgen Schmidhuber. 2001.  Gradient flow in recurrent nets: The difficulty of learning long-term dependencies.
  * Hochreiter and Schmidhuber (1997) Sepp Hochreiter and Jürgen Schmidhuber. 1997.  Long short-term memory.  _Neural computation_ , 9:1735–80.
  * Kamal Eddine et al. (2022) Moussa Kamal Eddine, Guokan Shang, Antoine Tixier, and Michalis Vazirgiannis. 2022\.  FrugalScore: Learning cheaper, lighter and faster evaluation metrics for automatic text generation.  In _Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 1305–1318, Dublin, Ireland. Association for Computational Linguistics.
  * Le et al. (2022) Hung Le, Nancy Chen, and Steven Hoi. 2022.  Multimodal dialogue state tracking.  In _Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 3394–3415, Seattle, United States. Association for Computational Linguistics.
  * Lemon and Pietquin (2007) Oliver Lemon and Olivier Pietquin. 2007.  Machine learning for spoken dialogue systems.
  * Liu and Mei (2020) Bing Liu and Chuhe Mei. 2020.  Lifelong knowledge learning in rule-based dialogue systems.
  * Liu et al. (2016) Chia-Wei Liu, Ryan Lowe, Iulian Serban, Mike Noseworthy, Laurent Charlin, and Joelle Pineau. 2016.  How NOT to evaluate your dialogue system: An empirical study of unsupervised evaluation metrics for dialogue response generation.  In _Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing_ , pages 2122–2132, Austin, Texas. Association for Computational Linguistics.
  * Liu et al. (2019) Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019.  Roberta: A robustly optimized bert pretraining approach.
  * Ni et al. (2023) Jinjie Ni, Tom Young, Vlad Pandelea, Fuzhao Xue, and Erik Cambria. 2023.  Recent advances in deep learning based dialogue systems: a systematic survey.  _Artificial Intelligence Review_ , 56:3055–3155.
  * OpenAI (2023) OpenAI. 2023.  GPT-4 technical report.
  * Papineni et al. (2002) Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002.  BLEU: A method for automatic evaluation of machine translation.  In _Proceedings of the 40th Annual Meeting on Association for Computational Linguistics_ , ACL ’02, page 311–318, USA. Association for Computational Linguistics.
  * Rae et al. (2019) Jack W. Rae, Anna Potapenko, Siddhant M. Jayakumar, and Timothy P. Lillicrap. 2019\.  Compressive transformers for long-range sequence modelling.
  * Rastogi et al. (2020) Abhinav Rastogi, Xiaoxue Zang, Srinivas Sunkara, Raghav Gupta, and Pranav Khaitan. 2020.  Towards scalable multi-domain conversational agents: The schema-guided dialogue dataset.
  * Reddy (2022) Sujan Reddy. 2022.  Automating human evaluation of dialogue systems.  In _Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Student Research Workshop_ , pages 229–234, Hybrid: Seattle, Washington + Online. Association for Computational Linguistics.
  * Sadat and Caragea (2022) Mobashir Sadat and Cornelia Caragea. 2022.  SciNLI: A corpus for natural language inference on scientific text.  In _Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 7399–7409, Dublin, Ireland. Association for Computational Linguistics.
  * Santhanam and Shaikh (2019) Sashank Santhanam and Samira Shaikh. 2019.  A survey of natural language generation techniques with a focus on dialogue systems - past, present and future directions.
  * Serban et al. (2017) Iulian Vlad Serban, Ryan Lowe, Peter Henderson, Laurent Charlin, and Joelle Pineau. 2017.  A survey of available corpora for building data-driven dialogue systems.
  * Shuster et al. (2022) Kurt Shuster, Jack Urbanek, Arthur Szlam, and Jason Weston. 2022.  Am I me or you? state-of-the-art dialogue models cannot maintain an identity.  In _Findings of the Association for Computational Linguistics: NAACL 2022_ , pages 2367–2387, Seattle, United States. Association for Computational Linguistics.
  * Smith et al. (2022) Eric Smith, Orion Hsu, Rebecca Qian, Stephen Roller, Y-Lan Boureau, and Jason Weston. 2022.  Human evaluation of conversations is an open problem: comparing the sensitivity of various methods for evaluating dialogue agents.  In _Proceedings of the 4th Workshop on NLP for Conversational AI_ , pages 77–97, Dublin, Ireland. Association for Computational Linguistics.
  * Sutskever et al. (2014) Ilya Sutskever, Oriol Vinyals, and Quoc V. Le. 2014.  Sequence to sequence learning with neural networks.
  * Tiedemann (2016) Jörg Tiedemann. 2016.  Finding alternative translations in a large corpus of movie subtitle.  In _Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC’16)_ , pages 3518–3522, Portorož, Slovenia. European Language Resources Association (ELRA).
  * Weizenbaum (1966) Joseph Weizenbaum. 1966.  ELIZA—a computer program for the study of natural language communication between man and machine.  _Communications of the ACM_ , 9(1):36–45.
  * Williams et al. (2018) Adina Williams, Nikita Nangia, and Samuel Bowman. 2018.  A broad-coverage challenge corpus for sentence understanding through inference.  In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)_ , pages 1112–1122, New Orleans, Louisiana. Association for Computational Linguistics.
  * Yeh et al. (2021) Yi-Ting Yeh, Maxine Eskenazi, and Shikib Mehri. 2021.  A comprehensive assessment of dialog evaluation metrics.  In _The First Workshop on Evaluations and Assessments of Neural Conversation Systems_ , pages 15–33, Online. Association for Computational Linguistics.
  * Yu et al. (2020) Juntao Yu, Bernd Bohnet, and Massimo Poesio. 2020.  Named entity recognition as dependency parsing.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 6470–6476, Online. Association for Computational Linguistics.
  * Zeiler and Fergus (2013) Matthew D Zeiler and Rob Fergus. 2013.  Visualizing and understanding convolutional networks.