# Three Ultraluminous X-ray Sources Hosted by Globular Clusters in NGC 1316

Kristen C. Dage,1,2 Arunav Kundu,3 Erica Thygesen,4 Arash Bahramian,5 Daryl
Haggard, 1,2 Jimmy A. Irwin,6 Thomas J. Maccarone,7 Sneha Nair,1 Mark B.
Peacock,3 Jay Strader,4 Stephen E. Zepf 4
1 Department of Physics, McGill University, 3600 University Street, Montréal,
QC H3A 2T8, Canada
2 McGill Space Institute, McGill University, 3550 University Street, Montréal,
QC H3A 2A7, Canada
3Eureka Scientific, Inc., 2452 Delmer Street, Suite 100 Oakland, CA 94602, USA
4Department of Physics and Astronomy, Michigan State University, East Lansing,
MI 48824
5 International Centre for Radio Astronomy Research $--$ Curtin University,
GPO Box U1987, Perth, WA 6845, Australia
6Department of Physics and Astronomy, University of Alabama, Box 870324,
Tuscaloosa, Alabama, 35487, USA
7Department of Physics, Box 41051, Science Building, Texas Tech University,
Lubbock, TX 79409-1051, USA
E-mail<EMAIL_ADDRESS>
(Accepted XXX. Received YYY; in original form ZZZ)

###### Abstract

We have identified three ultraluminous X-ray sources (ULXs) hosted by globular
clusters (GCs) within NGC 1316’s stellar system. These discoveries bring the
total number of known ULXs in GCs up to 20. We find that the X-ray spectra of
the three new sources do not deviate from the established pattern of spectral
behaviour of the other known GC ULXs. The consistency of the X-ray spectral
behaviour for these sources points to multiple paths of formation and
evolution mechanisms for these rare and unique sources. Using the now larger
sample of GC ULXs, we compare the optical properties of the entire known
population of GC ULXs to other GCs across five galaxies and find that the
properties of clusters that host ULXs are quite different from the typical
clusters. Lastly, any trend of GC ULXs being preferentially hosted by metal-
rich clusters is not strongly significant in this sample.

###### keywords:

NGC 1316: globular clusters: general – stars: black holes – X-rays: binaries

††pubyear: 2021††pagerange: Three Ultraluminous X-ray Sources Hosted by
Globular Clusters in NGC 1316–Three Ultraluminous X-ray Sources Hosted by
Globular Clusters in NGC 1316

##  1 Introduction

Ultraluminous X-ray sources (ULXs) are off-nuclear X-ray point sources with
luminosities exceeding the Eddington limit for a 10 solar mass black hole
(BH), typically $\gtrsim$1039 erg s-1. Most ULXs tend to occur in young, star-
forming regions of spiral galaxies, and a handful have identified neutron star
accretors, identified by pulsations, possibly with high magnetic fields
(Bachetti et al., 2014; Brightman et al., 2018; Hu et al., 2021, and
references therein). Statistical evidence for such sources in elliptical
galaxies is poor (Irwin et al., 2004), but starting with the source XMMU
J122939.+075333 in the globular cluster RZ 2109 in the galaxy NGC 4472
(Maccarone et al., 2007), a growing number of GC ULXs in elliptical galaxies
has been discovered. With the globular cluster association, the problem of
distinguishing between background AGN projected against a galaxy and bona fide
members of the galaxies is ameliorated, and secure identifications of ULXs in
these older populations can be made.

For these GC ULXs, BH accretors offer the most likely explanation. Since these
ULXs exist in crowded, older stellar populations, the only remaining donor
stars are low mass and hydrogen deficient. In these dense stellar
environments, binaries are formed dynamically. Given the number of differences
between the old, crowded GC environment, and the young regions where NS ULXs
have been identified, this implies the accretion phenomena and compact objects
of GC ULXs may be fundamentally different than the physics behind the younger
NS ULXs (see also theoretical work such as Wiktorowicz et al. (2021) which
suggests that the physics powering ultraluminous X-ray sources in young
($<$100 Myr) environments is vastly different than the physics in environments
$>$ 2Gyr).

Identifying these BH candidates in extragalactic globular clusters is exciting
because, while many studies have identified black holes and black hole
candidates in Galactic globular clusters (Strader et al., 2012a; Chomiuk et
al., 2013; Miller-Jones et al., 2015; Giesers et al., 2018; Shishkovsky et
al., 2018; Giesers et al., 2019), the Milky Way is home to fewer than 300
globular clusters. By contrast, the total number of clusters in even a single
elliptical galaxy can be larger than $\sim$100 the number in the Milky Way,
offering tens of thousands of clusters for study. This allows for broader
statistical studies of the nature of BH-hosting clusters than can be
accomplished in the Milky Way.

The nature and number of these sources have important implications for the
progenitors of the black hole-black hole binary mergers detected by LIGO, as
globular clusters are a likely birthplace of these sources (Abbott et al.,
2016; Rodriguez et al., 2019). Black holes are also important for
understanding cluster evolution (Giersz et al., 2019; Kremer et al., 2019), as
black holes may play an important role in the cluster dynamics (Ye et al.,
2020). This marks a dramatic shift from early theoretical work, which
predicted that while black holes certainly formed in globular clusters, they
would be thrown out of the cluster either due to natal kicks or gravitational
interactions (e.g. Spitzer 1969).

The first ULX associated with a globular cluster was identified by Maccarone
et al. (2007). The source displayed unusual short-term variability on the
scale of hours, as well as luminous and broad [OIII] emission lines, visible
above the host cluster continuum (Zepf et al., 2008; Steele et al., 2011).
Analysis by Peacock et al. (2012) measured a sizeable oxygen nebula in the
system. Steele et al. (2014) placed limits on any excess hydrogen emission
([OIII]/H$\beta$ =106) and identified a carbon-oxygen white dwarf as the very
likely donor star of the system. The long-term X-ray and optical properties of
the source were studied by Dage et al. (2018); Dage et al. (2019b), which
showed that the X-ray luminosity varied by several orders of magnitude over 16
years, and the optical luminosity of the line declined over a 7 year period.
Subsequent observations indicate that the optical luminosity has been rising
for the last two years, placing constraints on the size of the oxygen nebula
(between $10^{-3}$ pc at minimum, but possibly up to 2pc).

In 2010, two GC ULXs were identified in NGC 1399, the central galaxy in the
Fornax Cluster. One of these showed short variability on the timescales of
hours, and was bright in X-rays until 2003, but has not been detected since
(Shih et al., 2010). The other GC ULX did not show variability within an
observation, but showed narrow [OIII] and [NII] emission lines above the host
cluster continuum (Irwin et al., 2010).

Maccarone et al. (2011) discovered a second GC ULX hosted by one of NGC 4472’s
clusters, and Roberts et al. (2012) identified a GC ULX in an NGC 4649
cluster. Optical spectroscopy of the NGC 4649 cluster revealed no emission
lines beyond the host cluster continuum, implying that not every GC ULX is
host to optical emission. Irwin et al. (2016) discovered X-ray sources hosted
by two globular clusters that flared above the Eddington limit on the
timescale of just minutes. Similar sources were also identified by Sivakoff et
al. (2005).

Dage et al. (2019a) conducted the first ever large-scale, long-term X-ray
analysis of GC ULXs in NGC 4472, NGC 4649 and NGC 1399, comparing the X-ray
spectral properties of these and other sources. One of the major results of
this work was that the two clusters that were known to exhibit optical
emission lines showed markedly different X-ray behaviour than the rest of the
sample, with lower best-fit inner disk temperatures and spectral shapes
uncorrelated with major changes in X-ray luminosity.

Seven more GC ULXs were identified by Dage et al. (2020), associated with the
large GC population of M87. The X-ray spectra of these sources behaved like
the larger GC ULX sample (and unlike the optical emission line GC ULXs). A
third source with significant short-term variability was identified. Lastly,
it was shown that the ULXs were preferentially hosted by brighter (and hence,
presumably, more massive) clusters, but did not show evidence for a
correlation between the cluster colour and presence of a ULX.

In this work, we look to NGC 1316 to identify new GC ULX sources. NGC 1316 is
a giant early-type galaxy that underwent a merger as recently as 1–3Gyr ago
(Goudfrooij et al. 2001a, see also work by Kleiner et al. (2021) for a
discussion of the galaxy’s history). Studies of the galaxy’s globular cluster
system show that it is made up of two distinct populations of younger and
older clusters (Goudfrooij et al., 2001b). Studies of the HI content of the
galaxy give further agreement that NGC 1316 is the product of a merger of a
lenticular and spiral galaxy (Kleiner et al., 2021) .

We use archival Chandra and HST data to identify three new GC ULXs in NGC
1316, and compare the X-ray spectral properties of the sources to the
previously-identified GC ULXs, as well as the optical properties of the host
clusters. Section 2 describes the data and analysis techniques used for both
optical and X-ray data reduction. Section 3 discusses the results of this work
and compares the X-ray and optical properties of the 20 known GC ULXs.
Finally, the implications of this analysis are discussed in Section 4.

##  2 Observations and Analysis

Figure 1: Left: X-ray image of NGC 1316 (ObsID 20341) with positions of the
three GC ULXs overlaid. Right: HST F475W (Sloan $g$ band) image of NGC 1316
with the GC ULXs overlaid.

We analysed archival X-ray and optical data of NGC 1316 to identify three ULXs
hosted by globular clusters in this galaxy (see Figure 1). The data are
displayed in Table 1, and include six Chandra X-ray observations, with one
taken in 2001 and five spread across a single month in 2019. We use HST
observations in the F475W and F850LP filters (Sloan $g$ and $z$ bands in the
AB magnitude system) to identify potential globular cluster counterparts, as
described in Section 2.2.

###  2.1 Identification of ULX sources

To identify the X-ray point source populations, we use the function wavdetect
from ciao on each X-ray image, which were cleaned and filtered of background
flares, with the exposure map centered at 2.3 keV, and wavelet scales of 1.0,
2.0, 4.0, 8.0 and 16.0 pixels. We use an enclosed count fraction (e.c.f.) of
0.3, and significance threshold of $10^{-6}$. This corresponds to about one
false detection per chip, although the false detection will not have a flux
corresponding to a ULX. We then estimate the fluxes of the identified sources
by using the function srcflux, assuming a fixed power-law of $\Gamma$=1.7, the
typical power-law index for X-ray binaries, and fixed the hydrogen column
density, $N_{H}$, to the line of sight density, 2.13 $\times 10^{20}$ cm-2
(Dickey & Lockman, 1990). We use the distance of 20.0 Mpc 111The distance to
the Fornax cluster used in Dage et al. (2019a). to estimate the X-ray
luminosity for all of the sources. This yields three unique X-ray point
sources with luminosities near or exceeding the Eddington limit. While this
analysis is only concerned with sources that have X-ray luminosities exceeding
the Eddington limit for a 10 solar mass BH, it is worth noting that there are
a number of less bright X-ray sources associated with globular clusters in the
sample from Jordán et al. (2009).

Table 1: Left column: Chandra observations of NGC 1316, all observations are ACIS-S. Right column: HST observations for $g$ and $z$ (AB mag) observations. ObsID (Chandra) | Date | Exposure (ks) | ObsID (HST) | Date | Filter | Exposure (s)
---|---|---|---|---|---|---
2022 | 2001-04-17 | 30 | j90x01020 | 2005-02-16 | F475W ($g$) | 760
20340 | 2019-04-10 | 45 | j90x01010 | 2005-02-16 | F850LP ($z$) | 1130
20341 | 2019-04-22 | 52 |  |  |  |
22179 | 2019-04-17 | 39 |  |  |  |
22180 | 2019-04-20 | 14 |  |  |  |
22187 | 2019-04-25 | 53 |  |  |  |

###  2.2 HST analysis and globular cluster counterpart identification

We use catalogs of the globular cluster population of NGC 1316 from Goudfrooij
et al. (2001b) and Jordán et al. (2016) to identify three new ultraluminous
X-ray sources hosted by globular clusters. Two ULX sources, GCULX1 and GCULX2,
match to globular cluster candidates in the Jordán et al. (2016) survey.
GCULX3’s optical counterpart, identified in Allak et al. (2020), also had
characteristics typical of extragalactic globular clusters. We use DAOPHOT
(Stetson, 1987) to perform photometry on all three sources, as well as other
known GCs in the field, using images that corresponds to the $g$, $z$ bands,
and found that GCULX3’s colour and magnitude are typical for a globular
cluster. The ACS and WFC3 zero-points adopted from the HST ACS and WFC3 data
handbook 222https://www.stsci.edu/hst/documentation.

We measure the half-light radius with BAOlab (Larsen, 1999) using the KING30
model convolved with a PSF made in Tiny Tim (Krist et al., 2011), and found
the size of GCULX3 was on par with the globular clusters identified as having
a 100% probability of being a globular cluster in Jordán et al. (2016).

The coordinates of the three GC ULX sources are listed in Table 2, along with
their measured colour and magnitude in the AB system. The optical properties
of the host clusters compared to the rest of the cluster population in the NGC
1316 are displayed in Figure 2.

Table 2: X-ray coordinates of the three ULXs and lower X-ray luminosity
sources coincident with candidate globular clusters. These sources have been
identified in Kim & Fabbiano (2003), Swartz et al. (2004), and Allak et al.
(2020). Sources marked with o were identified by Goudfrooij et al. (2001b),
sources with $\dagger$ were identified by Jordán et al. (2016). X-ray
luminosity estimates are based off of srcflux calculations from ObsID 22187.
Source counts and errors are based off of wavdetect measurements of ObsID
22187.

CXOU J032238-371228 was only measured in the Vega magnitude system by
Goudfrooij et al. (2001b), and hence there are no AB magnitudes to report.
Name RA Dec $L_{X}$ Estimate Counts z-band $g-z$ $\times 10^{39}$erg s-1
0.5-7.0 keV (AB mag) CXOU J032242.5-371222 (GCULX1) $\dagger$ 03:22:42.48
-37:12:21.15 1.39 112 $\pm$ 11 20.88 1.67 CXOU J032241.07-371235.3 (GCULX2)
$\dagger$ 03:22:41.11 -37:12:34.73 1.05 61 $\pm$ 9 21.98 1.49 CXOU
J032240.8-371224 (GCULX3) 03:22:40.82 -37:12:23.28 2.15 159 $\pm$ 16 23.10
0.94 CXOU J032244-371310 o 03:22:44.71 -37:13:10.07 0.15 9 $\pm$3 20.81 1.11
CXOU J032241-371304 o $\dagger$ 03:22:41.89 -37:13:04.49 0.33 17 $\pm$4 22.64
1.36 CXOU J032242-371258 o 03:22:42.43 -37:12:58.83 0.85 61 $\pm$8 22.37 1.72
CXOU J032240-371244 o 03:22:40.42 -37:12:44.51 0.13 16 $\pm$5 22.02 1.40 CXOU
J032237-371251 o 03:22:37.69 -37:12:51.11 0.21 16 $\pm$4 18.19 1.27 CXOU
J032238-371211 o 03:22:38.19 -37:12:11.58 0.12 9 $\pm$3 22.06 1.02 CXOU
J032239-371148 o $\dagger$ 03:22:39.12 -37:11:48.00 0.70 64 $\pm$8 21.55 1.47
CXOU J032242-371218 o 03:22:42.06 -37:12:18.25 0.41 35 $\pm$7 18.55 1.40 CXOU
J032242-371216 o $\dagger$ 03:22:42.73 -37:12:16.62 0.41 36 $\pm$6 18.16 1.24
CXOU J032242-371206 o $\dagger$ 03:22:42.60 -37:12:06.25 0.20 10 $\pm$3 22.40
0.99 CXOU J032242-371123 o 03:22:42.10 -37:11:23.92 0.42 47 $\pm$7 23.91 1.23
CXOU J032241-371117 o $\dagger$ 03:22:41.29 -37:11:17.13 0.30 19 $\pm$4 21.65
1.08 CXOU J032238-3712 $\dagger$ 03:22:38.78 -37:12:28.66 0.13 17 $\pm$4 N/A
N/A

Figure 2: Colour and magnitude of the three GC ULXs in NGC 1316 compared to
the overall population of globular clusters.

###  2.3 X-ray Analysis

Table 3: Total background subtracted counts in the 0.5-8.0keV band and best fit spectral parameters for combined spectra of the three GC-ULXs.  |  | power-law |  | diskbb |
---|---|---|---|---|---
Source | Total Counts (0.5-8.0 keV) | $\Gamma_{\text{PL}}$ |  $\chi_{\nu}^{2}$/d.o.f. | F-Test prob. (%) |  | kT${}_{\text{in}}$ (keV) |  $\chi_{\nu}^{2}$/d.o.f. | F-Test prob. (%) |
GCULX1 | 272 | 1.4 ($\pm 0.3$) | 0.42/11 | – |  | 1.5 (${}^{+0.7}_{-0.4}$) | 0.62/11 | – |
GCULX2 | 613 | 1.8 ($\pm$0.1) | 1.29/26 | 5 |  | 1.2 ($\pm$0.2) | 1.08/26 | 53 |
GCULX3 | 484 | 1.6($\pm 0.1$) | 1.01/21 | — |  | 1.3 ($\pm$0.2) | 0.87/21 | – |
Table 4: Chandra Fit Parameters and Fluxes (0.5-8 keV) for spectral best fit
single-component models, tbabs*pegpwrlw andtbabs*diskbb for NGC 1316-GCULX1.
Hydrogen column density ($N_{H}$) frozen to 2.13 $\times 10^{20}$cm-2. All
fluxes shown are unabsorbed in the 0.5-8 keV band. In ObsID 22180, GCULX1 has
a background subtracted count rate of 8.84 $\times 10^{-4}$ counts/sec in the
0.5-8 keV range. The upper limit was calculated using a fixed power-law index
of $\Gamma$ = 1.4.  NGC 1316 GCULX1
---
| tbabs*pegpwrlw |  | tbabs*diskbb
ObsID (Date) | $\Gamma$ |  $\chi^{2}_{\nu}$/d.o.f. | PL Flux |  | $T_{in}$ | Disk Norm |  $\chi^{2}_{\nu}$/d.o.f. | Disk Flux
|  |  | ($10^{-14}$ erg s-1 cm-2) |  | (keV) | ($10^{-4}$) |  | ($10^{-14}$ erg s-1 cm-2)
2022 (2001-04-17) | 1.3 ($\pm$0.4) | (44.36/51) | 2.9 (${}^{+1.2}_{-0.9}$) |  | 1.7 $(^{+2.2}_{-0.6})$ |  $\leq$1.7 | (44.44/51) | 2.4 (${}^{+1.3}_{-0.7}$)
20340 (2019-04-10) | 1.2 ($\pm$ 0.5) | (70.61/63) | 1.7 ($\pm$0.6) |  | 2.0 (${}^{+4.5}_{-0.8}$) |  $\leq$0.5 | (69.02/63) | 1.5 ($\pm$ 0.6)
20341 (2019-04-22) | 1.6 ($\pm$0.6) | (30.78/34) | 1.3 ($\pm$0.5) |  | 1.5 (${}^{+1.8}_{-0.6}$) |  $\leq$ 1.1 | (31.62/34) | 1.1 ($\pm$ 0.5)
22179 (2019-04-17) | 1.9 ($\pm$0.7) | (32.05/30) | 1.2($\pm$0.5) |  | 1.4(${}^{+2.3}_{-0.6}$) |  $\leq$ 1.3 | (35.09/30) | 1.0 ($\pm$ 0.5)
22180 (2019-04-20) | $1.4$ | - |  $\leq$ 1.3 |  | - | - | - | -
22187 (2019-04-25) | 1.4 $(\pm 0.4)$ | (47.22/60) | 1.9 ($\pm$ 0.5) |  | 1.6 (${}^{+1.3}_{-0.5}$ |  $\leq$ 1.2 | (47.74/60) | 1.7 ($\pm$ 0.6)
Table 5: Chandra Fit Parameters and Fluxes (0.5-8 keV) for spectral best fit
single-component models, tbabs*pegpwrlw andtbabs*diskbb for NGC 1316-GCULX2.
Hydrogen column density ($N_{H}$) frozen to 2.13 $\times 10^{20}$cm-2. All
fluxes shown are unabsorbed in the 0.5-8 keV band.  NGC 1316 GCULX2
---
| tbabs*pegpwrlw |  | tbabs*diskbb
ObsID (Date) | $\Gamma$ |  $\chi^{2}_{\nu}$/d.o.f. | PL Flux |  | $T_{in}$ | Disk Norm |  $\chi^{2}_{\nu}$/d.o.f. | Disk Flux
|  |  | ($10^{-14}$ erg s-1 cm-2) |  | (keV) | ($10^{-3}$) |  | ($10^{-14}$ erg s-1 cm-2)
2022 (2001-04-17) | 1.6 ($\pm$0.7) | 0.362/3 | 3.3 (${}^{+3.0}_{-1.3}$) |  | 0.7 $(^{+1.6}_{-0.25})$ |  $\leq$3.7 | 0.23/3 | 1.7 (${}^{+2.4}_{-0.5}$)
20340 (2019-04-10) | 1.5 ($\pm$ 0.5) | 1.139/61 | 2.2 ($\pm$0.6) |  | 1.4 (${}^{+0.9}_{-0.4}$) |  $\leq$0.2 | 0.9385/61 | 1.8 ($\pm$ 0.5)
20341 (2019-04-22) | 1.4 ($\pm$0.6) | 0.848/3 | 3.1 (${}^{+1.2}_{-0.9}$) |  | 1.2 (${}^{+1.8}_{-0.4}$) |  $\leq$ 0.5 | 0.561/3 | 2.2(${}^{+1.3}_{-0.6}$)
22179 (2019-04-17) | 2.2($\pm$0.6) | (65.04/52) | 2.0($\pm$0.5) |  | 0.9(${}^{+0.5}_{-0.3}$) |  $\leq$ 1.6 | ( 69.92 /52) | 1.5 ($\pm$ 0.5)
22180 (2019-04-20) | 1.9 ($\pm$ 0.8) | (38.26/29) | 2.7 (${}^{+1.2}_{-0.9}$) |  | 1.1(${}^{+1.6}_{-0.4}$) |  $\leq$0.7 | (38.15/29) | 2.1 (${}^{+1.2}_{-0.8}$)
22187 (2019-04-25) | 1.7 $(\pm 0.4)$ | 0.93/5 | 3.7($\pm$ 0.6) |  | 1.2 (${}^{+0.6}_{-0.3}$ |  $\leq$ 0.7 | 1.29/5 | 2.9 ($\pm$ 0.6)
Table 6: Chandra Fit Parameters and Fluxes (0.5-8 keV) for spectral best fit
single-component models, tbabs*pegpwrlw andtbabs*diskbb for NGC 1316-GCULX3.
Hydrogen column density ($N_{H}$) frozen to 2.13 $\times 10^{20}$cm-2. All
fluxes shown are unabsorbed in the 0.5-8 keV band. In ObsID 22180, the upper
limit for GCULX3 was estimated using a background subtracted count rate of
9.58 $\times 10^{-4}$ counts/sec in the 0.5-8 keV range. NGC 1316 GCULX3
---
| tbabs*pegpwrlw |  | tbabs*diskbb
ObsID (Date) | $\Gamma$ |  $\chi^{2}_{\nu}$/d.o.f. | PL Flux |  | $T_{in}$ | Disk Norm |  $\chi^{2}_{\nu}$/d.o.f. | Disk Flux
|  |  | ($10^{-14}$ erg s-1 cm-2) |  | (keV) | ($10^{-3}$) |  | ($10^{-14}$ erg s-1 cm-2)
2022 (2001-04-17) | 1.2 ($\pm$0.5) | 0.08/2 | 1.7 ($\pm$ 0.6) |  | 0.8 $(^{+0.5}_{-0.3})$ |  $\leq$ 1.6 | 0.26/2 | 1.2($\pm$ 0.5)
20340 (2019-04-10) | 1.7 ($\pm$ 0.5) | ( 46.78/48) | 1.7 ($\pm$0.7) |  | 1.2 (${}^{+0.9}_{-0.4}$) |  $\leq$0.3 | (47.78/48) | 1.4 ($\pm$ 0.5)
20341 (2019-04-22) | 1.6 ($\pm$0.5) | 1.16/3 | 3.4 (${}^{+1.2}_{-0.5}$) |  | 1.1 (${}^{+1.0}_{-0.3}$) |  $\leq$0.9 | 0.98/3 | 2.3 (${}^{+1.1}_{-0.6}$)
22179 (2019-04-17) | 1.6 ($\pm$0.6) | (34.44/51) | 2.3($\pm$0.7) |  | 1.4(${}^{+1.9}_{-0.3}$) |  $\leq$0.3 | (34.48/51) | 1.9 (${}^{+0.8}_{-0.5}$)
22180 (2019-04-20) | $1.6$ | – |  $\leq$ 1.7 |  | - | - | - | -
22187 (2019-04-25) | 1.2 ($\pm 0.5$)) | 0.34/3 | 4.5(${}^{+1.4}_{-1.1}$) |  | 2.1 (${}^{+2.1}_{-0.9}$ |  $\leq$0.1 | 0.422/3 | 3.7(${}^{+2.1}_{-1.1}$)

These sources were analysed in the same manner as Dage et al. (2019a, 2020),
in order to compare the X-ray properties of all of the 20 identified GC ULXs
to each other. The sources were extracted in each observation using ciao, and
the background regions were created by placing regions of a similar size to
the source region near the source, but avoiding any nearby point sources. We
use xspec (Arnaud, 1996) to fit the spectra in the 0.5-8.0 keV band with
single component models: tbabs*diskbb and tbabs*pegpwrlw. Any observation with
greater than 100 counts is binned by 20, and any observation with fewer than
100 counts is binned by 1 and fit using the Cash statistic (Cash, 1979). In
ObsID 22180, both GCULX2 and GCULX3 have too few counts to extract a spectrum,
and we estimate an upper limit using pimms. The X-ray fits for each source can
be found in Tables 4-6. Finally, we use combine_spectra to combine all of the
extracted spectra for each source (see Table 3 for the best-fit values). The
best fit power-law index of the combined spectra was used to estimate the
upper limits for GCULX2 and GCULX3 in ObsID 22180. We also fit the two
component model tbabs*(diskbb+pegpwrlw) to the deep spectra, however, only
GCULX2 could be fit by the two component model. We use F-test to determine
whether the two component model was statistically a better fit than either
single component model, but given that the F-test null hypothesis probability
was high for either case, this implies that the complexity of the two-
component model is not justified. Since this is likely due to the source
having many more counts than the other two sources, we do not report the
values.

Lastly, we extract background subtracted light-curves using dmextract and find
no evidence of strong variability (see also analysis by Allak et al. (2020)
for lack of variability in GCULX3). This is typical for most GCULXs, currently
only three of them show strong variability (see Maccarone et al. (2007); Shih
et al. (2010); Dage et al. (2020) for more details).

##  3 Results

We identified three new GC ULX candidates in NGC 1316, using archival Chandra
and HST observations. The HST observations revealed three globular cluster
candidates with $z$ and $g-z$, which all had similar sizes. We extracted and
fit the X-ray spectra of each source across the six Chandra observations. We
fit two different single component models to the individual spectra, and also
fit models to the combined spectra. Of these, only GCULX2 was able to be fit
by a two component model over a single component model333and it is worth
noting that GCULX2 has significantly more counts than either of the other two
sources, but in either case the F-test probability was high, meaning that it
is not statistically reasonable to add the extra component. Very few GCULXs
can be well-fit by a two-component model, only two known sources (RZ2109 (Dage
et al., 2018), and M87-GCULX1 (Dage et al., 2020) have any evidence for the
extra component statistically being a better fit.

Both on the one month and the 18 year time baselines probed, there is no
evidence for variations for more than a factor of $\sim$ 2, which is
consistent with most of the known GC ULXs (Dage et al., 2019a). Only the
source in RZ 2109 in NGC 4472 (Maccarone et al., 2007), the faded GC ULX
identified by Shih et al. (2010) in NGC 1399, and the source SC302 in M87
(Dage et al., 2020) show strong variability on the timescale of hours. Given
that most of the observations are clustered within a single month in 2019,
with just one observation occurring in 2001 (see Figure 3), it is difficult to
characterise any long-term variability of these sources.

We also identified thirteen X-ray sources coincident with GCs (Table 2), with
estimated X-ray luminosities on the order of $10^{38}$ erg/s. Given the
paucity of the data, we do not perform spectral fits, however, a comprehensive
study of similar sources has previously been studied in Irwin et al. (2003).

Figure 3: Long-term X-ray luminosity for the three GC ULX sources. Left panel
shows the observation from April, 2001 (ObsID 2022), and the right panel shows
the five observations taken in April, 2019, including upper limits from ObsID
22180 (triangles).

###  3.1 Host Cluster Colour and Magnitude

In both Dage et al. (2019a) and Dage et al. (2020), we noted that while the
ULXs are indeed preferentially hosted by brighter clusters, there is not yet
evidence that ULXs hold an affinity for metal rich clusters (unlike the rest
of the less-luminous X-ray binary population which shows red clusters
outnumbering bluer clusters by three to one as LMXB hosts Kundu et al. 2002;
Sarazin et al. 2003). Initial studies on a low population of ULX hosting
clusters by Maccarone et al. (2011) suggested that it was likely that ULXs are
also preferentially hosted by metal rich clusters, however the sample size in
this initial study was very small. With the new, larger sample of ULXs hosted
by globular clusters, this question can now be revisited.Nineteen of the GC
ULXs have colour and magnitudes measured in the AB magnitude system (see Dage
et al. (2019a, 2020) for more). As seen in Figure 4, the majority of ULXs are
not hosted by typical globular clusters for these galaxies, which certainly
highlights the uniqueness of these already-rare sources. We used the Anderson
Darling test (Anderson & Darling, 1952, 1954) from
kSamples444https://cran.r-project.org/web/packages/kSamples/index.html to
independently compare the optical colour and magnitude distributions of the
ULX hosts and other globular clusters. For the magnitude, the statistic is
22.1 and the significance level is $5.9\times 10^{-10}$. For colour, the
statistic value is 3.22 and the significance level is 0.016. The low
significance levels imply that the magnitude of the ULX hosts come from very
different distributions than the rest of the clusters in these galaxies.
However, the tests cannot yet reject the null hypothesis concerning the colour
distribution.

NGC 1316 is known to host an intermediate age population, and therefore the
optical colours might not be linearly correlated metallicity for some of NGC
1316’s GCs. Infrared colours are known to be more sensitive to metallicity
(Kundu & Zepf, 2007). To check the metallicities of the NGC 1316 ULX hosting
cluster, we also performed photometry in the H-band (F160W, HST observation
ib3n03050). GCULX1 was $H$=20.0$\pm$0.2, GCULX2 was $H$= 20.1$\pm$0.4 and
GCULX3 was $H$= 21.3$\pm$0.4. The very red infrared colours (z-H) imply that
all three ULX hosting clusters of NGC 1316 may be very metal rich. The
apparent bluer optical colour of GCULX3 might be because these GCs may be from
an intermediate age population (Goudfrooij et al., 2001b).

Interestingly, while the NGC 1316 clusters that hosted lower luminosity X-ray
sources showed a similar preference to be hosted by brighter clusters (see
Figure 5). The statistic value comparing the magnitude of the lower luminosity
X-ray cluster hosts to the overall NGC 1316 cluster population is 12.6 and the
significance level is $4.3\times 10^{-7}$. However, the statistic value for
optical colour is 1.52, with a significance level of 0.17, although this is
could be due to the smaller sample size.

Figure 4: Colour versus Magnitude (AB, dereddened) for the globular cluster ULXs, superimposed on the properties of non-ULX hosting clusters in the five galaxies (NGC 1316, NGC 1399, NGC 4472, NGC 4649 and M87, using data from Jordán et al. (2009); Strader et al. (2012b); Peacock et al. (2014); Jordán et al. (2016)).  |
---|---
Figure 5: Left: Cumulative distribution functions for $z$ and $g-z$ for the
ULX hosting clusters and non-ULX hosting clusters in the five galaxies (NGC
1316, NGC 1399, NGC 4472, NGC 4649 and M87 (Jordán et al., 2009; Strader et
al., 2012b; Peacock et al., 2014; Jordán et al., 2016). Right: Cumulative
distribution functions for $z$ and $g-z$ for the NGC 1316 clusters and low
X-ray luminosity hosting NGC 1316 clusters.

###  3.2 Spatial Distribution of ULX Hosting Clusters

As shown in Figures 6-7, NGC 1316’s globular cluster ULXs are all located near
the galaxy centre, with the lower luminosity X-ray hosting clusters more
spread out in the system.

Figure 6: Spatial distribution of globular clusters in NGC 1316, using
globular clusters identified by Jordán et al. (2016). We caution that the
asymmetrical distribution of globular clusters may be an observational bias
caused by obscuration from the dust lanes of the galaxy, although
observational evidence for anisotropic globular cluster distribution has been
observed in other galaxies such as NGC 4261 (Giordano et al., 2005).  Figure
7: Distance of NGC 1316’s X-ray hosting clusters (Table 2) from NGC 1316’s
galaxy centre as a function of effective radius (taken from the 2MASS catalog
(Skrutskie et al., 2006)).

The full data-set of the now 20 discovered GC ULXs can help address the
question of the spatial distribution of the ULX hosting clusters, and whether
they are preferentially located near the galaxy centres or outskirts (see
Figure 8). The three farthest ULXs were hosted by clusters over two effective
radii from their galaxy centres. Little is known about the nature of the
distant NGC 1399 source, as it has not been bright in X-rays since 2003 (Shih
et al., 2010), but the NGC 4472 source, RZ2109, is one of the most well-
studied GC ULXs, and is thought to be a stellar mass black hole accreting from
a white dwarf. The host of SC302, the farthest GC ULX identified in M87, by
contrast, has optical properties that are ambiguous as to whether the ULX is
hosted by a globular cluster or perhaps a stripped nucleus (Dage et al.,
2020). In the latter case, this may imply that the accretor of SC302 may be an
intermediate mass black hole.

Figure 8: Distance of ULX hosting clusters from their respective galaxy
centres as a function of effective radius (taken from the 2MASS catalog
Skrutskie et al. 2006). The most distant ULX hosts are the NGC 1399 GC ULX
observed by Shih et al. (2010), the well-studied and highly variable source
RZ2109 in NGC 4472 (Maccarone et al., 2007), and SC302 in M87 (Dage et al.,
2020).

Given the current data, ULXs seem to be over-represented in the inner regions
of the galaxy centre, although we caution that for some galaxies, there may
not be complete lists of the globular clusters, or sufficient X-ray
observations to cover the outer regions of all of these galaxies.

###  3.3  Power-Law Model Fits

Using data from (Dage et al., 2018, 2019a, 2020), we can compare the power-law
fits of the three NGC 1316 GC ULXs to the previously studied sample. Figure 9
displays the best fit power-law index ($\Gamma$) versus the X-ray luminosity
(calculated assuming D=20.0 Mpc). The three NGC 1316 GC ULXs seem to follow
the same trends as the previously studied sources. Interestingly, GCULX8 in
NGC 1399 (where the X-ray luminosity was calculated with D=20.0 Mpc) is still
an outlier compared to the rest of the sources, and is intrinsically much
brighter, even compared to sources at a comparable distance.

Figure 9: $L_{X}$ versus $\Gamma$ for NGC 1316’s GC ULX population compared to
fit parameters for other known GC ULXs (Dage et al., 2019a, 2020).

###  3.4  Black-body Disk Model Fits

Figure 10 compares the changes in the measured inner disk temperature
($T_{in}$) to the X-ray luminosity. These sources also appear to follow the
same trends as many of the other GC ULXs, where $L_{X}$ and kT have been
established to be correlated in some cases (Dage et al., 2019a).Both RZ2109 in
NGC 4472 and GCU7 in NGC 1399 behave differently than the rest of the GC ULXs,
with both having low temperatures (kT $<$ 0.5 keV) which appear to be
independent of the X-ray luminosity. These sources again highlight that the
accretion astrophysics is not the same for all of the GC ULXs.

Figure 10: $L_{X}$ versus $T_{in}$ for NGC 1316’s GC ULX population compared
to fit parameters for other known GC ULXs (Dage et al., 2018, 2019a, 2020).

##  4 Conclusions

The three new GC ULXs identified in NGC 1316 bring the total known population
of globular cluster ULXs to 20. Their behaviour in X-ray is neither as
luminous as GCU8 in NGC 1399, nor as strange as RZ2109 in NGC 4472 or GCU7 in
NGC 1399. Of the sources studied, the GC ULXs appear to still fall into the
same three categories of spectral behaviour, (1.) the low kT sources like
RZ2109 and GCU7, (2.) the “intermediate" sources with high kT where both kT
and $L_{X}$ vary, and (3.) the sources best fit by a power-law spectrum.

The intermediate sources in the second group, which have a wide range of best-
fit spectral parameters, may be more comparable to well-studied bright
Galactic X-ray binaries (Tetarenko et al., 2016). Many of these sources behave
like Galactic X-ray binaries, only with consistently brighter luminosities, or
show behaviour similar to sources like GRS 1915 (Tetarenko et al., 2016)
(although formed in an older population) which are bright, and only pass the
Eddington limit for a 10 solar mass BH for a small fraction of the total
observations.

The separate X-ray spectral behaviours of GC ULXs implies that the sources
have different combinations of accretor masses/donor stars, and therefore also
multiple paths to evolution and formation channels. Given that the X-ray
binary formation process in a globular cluster is highly dynamic, and involves
multiple interactions with other bodies in the cluster, as well as pair
exchanges, it is no surprise that the binary makeup and accretion physics of
these systems are also diverse.

Studies of the optical properties of the host clusters show that while the
ULXs tend to be more often hosted by luminous clusters, evidence is finally
emerging that the host clusters of ULXs are also preferentially redder.

The nature of these sources prompt a number of open questions, both in terms
of the high energy astrophysical phenomena that drive them, as well as the
dynamical evolution and formation of these sources. However, given that they
are potential tracers of BHs in GCs, they have important implications for the
possible origin of merging black holes observed by LIGO, for which black holes
in GCs are one of the leading theoretical possibilities (Chatterjee et al.,
2017; Rodriguez et al., 2021).

It is currently unclear what properties of the early-type galaxies or their
cluster systems will predict whether or not they host ULXs, aside from
brighter clusters preferentially hosting ULXs. For instance, Brassington et
al. (2010) surveyed bright X-ray sources in NGC 3379, and while three X-ray
sources that were found to coincide with globular clusters were luminous, they
were still below the Eddington limit.

NGC 1316 also is home to luminous, but not super-Eddington, X-ray point
sources aligned with its globular clusters. Given the recent merger of the
galaxy, and the two different ages of cluster populations, studies of the
X-ray luminosity function of the low mass X-ray binary population are
interesting possibilities for a future study.

## Data Availability Statement

The X-ray data in this article is publicly available through the Chandra
archive555https://cda.harvard.edu/chaser/ The optical data is available
through MAST 666https://archive.stsci.edu/.

## Acknowledgements

This research has made use of data obtained from the Chandra Data Archive and
the Chandra Source Catalog. We also acknowledge use of NASA’s Astrophysics
Data System and arXiv. KCD and DH acknowledge funding from the Natural
Sciences and Engineering Research Council of Canada (NSERC), the Canada
Research Chairs (CRC) program, and the McGill Bob Wares Science Innovation
Prospectors Fund. KCD acknowledges fellowship funding from the McGill Space
Institute. AK and MBP acknowledge support from NASA through grant number
GO-14738 from StSci. SEZ acknowledges support from Chandra grants GO9-20080X
and AR0-21008B. KCD thanks Carol-Rose Little, Will Clarkson, Eric Koch, Dane
Kleiner and Paolo Serra for their helpful discussion. The following software
and packages were used for analysis: ciao, software provided by the Chandra
X-ray Center (CXC), heasoft obtained from the High Energy Astrophysics Science
Archive Research Center (HEASARC), a service of the Astrophysics Science
Division at NASA/GSFC and of the Smithsonian Astrophysical Observatory’s High
Energy Astrophysics Division, SAOImage DS9, developed by Smithsonian
Astrophysical Observatory, numpy (van der Walt et al., 2011), and matplotlib
(Hunter, 2007). This publication makes use of data products from the Two
Micron All Sky Survey, which is a joint project of the University of
Massachusetts and the Infrared Processing and Analysis Center/California
Institute of Technology, funded by the National Aeronautics and Space
Administration and the National Science Foundation.

## References

  * Abbott et al. (2016) Abbott B. P., et al., 2016, ApJ, 833, L1
  * Allak et al. (2020) Allak S., Akyuz A., Aksaker N., Ozdogan Ela M., Avdan S., Soydugan F., 2020, MNRAS, 499, 5682
  * Anderson & Darling (1952) Anderson T. W., Darling D. A., 1952, The Annals of Mathematical Statistics, 23, 193
  * Anderson & Darling (1954) Anderson T. W., Darling D. A., 1954, Journal of the American Statistical Association, 49, 765
  * Arnaud (1996) Arnaud K. A., 1996, in ASP Conf. Ser. 101: Astronomical Data Analysis Software and Systems V. p. 17
  * Bachetti et al. (2014) Bachetti M., et al., 2014, Nature, 514, 202
  * Brassington et al. (2010) Brassington N. J., et al., 2010, ApJ, 725, 1805
  * Brightman et al. (2018) Brightman M., et al., 2018, Nature Astronomy, 2, 312
  * Cash (1979) Cash W., 1979, ApJ, 228, 939
  * Chatterjee et al. (2017) Chatterjee S., Rodriguez C. L., Kalogera V., Rasio F. A., 2017, ApJ, 836, L26
  * Chomiuk et al. (2013) Chomiuk L., Strader J., Maccarone T. J., Miller-Jones J. C. A., Heinke C., Noyola E., Seth A. C., Ransom S., 2013, ApJ, 777, 69
  * Dage et al. (2018) Dage K. C., Zepf S. E., Bahramian A., Kundu A., Maccarone T. J., Peacock M. B., 2018, ApJ, 862, 108
  * Dage et al. (2019a) Dage K. C., Zepf S. E., Peacock M. B., Bahramian A., Noroozi O., Kundu A., Maccarone T. J., 2019a, MNRAS, 485, 1694
  * Dage et al. (2019b) Dage K. C., et al., 2019b, MNRAS, 489, 4783
  * Dage et al. (2020) Dage K. C., Zepf S. E., Thygesen E., Bahramian A., Kundu A., Maccarone T. J., Peacock M. B., Strader J., 2020, MNRAS, 497, 596
  * Dickey & Lockman (1990) Dickey J. M., Lockman F. J., 1990, ARA&A, 28, 215
  * Giersz et al. (2019) Giersz M., Askar A., Wang L., Hypki A., Leveque A., Spurzem R., 2019, MNRAS, 487, 2412
  * Giesers et al. (2018) Giesers B., et al., 2018, MNRAS, 475, L15
  * Giesers et al. (2019) Giesers B., et al., 2019, A&A, 632, A3
  * Giordano et al. (2005) Giordano L., Cortese L., Trinchieri G., Wolter A., Colpi M., Gavazzi G., Mayer L., 2005, ApJ, 634, 272
  * Goudfrooij et al. (2001a) Goudfrooij P., Mack J., Kissler-Patig M., Meylan G., Minniti D., 2001a, MNRAS, 322, 643
  * Goudfrooij et al. (2001b) Goudfrooij P., Alonso M. V., Maraston C., Minniti D., 2001b, MNRAS, 328, 237
  * Hu et al. (2021) Hu C.-P., Ueda Y., Enoto T., 2021, ApJ, 909, 5
  * Hunter (2007) Hunter J. D., 2007, Computing in Science and Engineering, 9, 90
  * Irwin et al. (2003) Irwin J. A., Athey A. E., Bregman J. N., 2003, ApJ, 587, 356
  * Irwin et al. (2004) Irwin J. A., Bregman J. N., Athey A. E., 2004, ApJ, 601, L143
  * Irwin et al. (2010) Irwin J. A., Brink T. G., Bregman J. N., Roberts T. P., 2010, ApJ, 712, L1
  * Irwin et al. (2016) Irwin J. A., et al., 2016, Nature, 538, 356
  * Jordán et al. (2009) Jordán A., et al., 2009, ApJS, 180, 54
  * Jordán et al. (2016) Jordán A., Peng E. W., Blakeslee J. P., Cote P., Eyheramendy S., Ferrarese L., 2016, VizieR Online Data Catalog, p. J/ApJS/221/13
  * Kim & Fabbiano (2003) Kim D.-W., Fabbiano G., 2003, ApJ, 586, 826
  * Kleiner et al. (2021) Kleiner D., et al., 2021, arXiv e-prints, p. arXiv:2101.10347
  * Kremer et al. (2019) Kremer K., Chatterjee S., Ye C. S., Rodriguez C. L., Rasio F. A., 2019, ApJ, 871, 38
  * Krist et al. (2011) Krist J. E., Hook R. N., Stoehr F., 2011, in Kahan M. A., ed., Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series Vol. 8127, Optical Modeling and Performance Predictions V. p. 81270J, doi:10.1117/12.892762
  * Kundu & Zepf (2007) Kundu A., Zepf S. E., 2007, ApJ, 660, L109
  * Kundu et al. (2002) Kundu A., Maccarone T. J., Zepf S. E., 2002, ApJ, 574, L5
  * Larsen (1999) Larsen S. S., 1999, A&AS, 139, 393
  * Maccarone et al. (2007) Maccarone T. J., Kundu A., Zepf S. E., Rhode K. L., 2007, Nature, 445, 183
  * Maccarone et al. (2011) Maccarone T. J., Kundu A., Zepf S. E., Rhode K. L., 2011, MNRAS, 410, 1655
  * Miller-Jones et al. (2015) Miller-Jones J. C. A., et al., 2015, MNRAS, 453, 3918
  * Peacock et al. (2012) Peacock M. B., et al., 2012, ApJ, 759, 126
  * Peacock et al. (2014) Peacock M. B., Zepf S. E., Maccarone T. J., Kundu A., Gonzalez A. H., Lehmer B. D., Maraston C., 2014, ApJ, 784, 162
  * Roberts et al. (2012) Roberts T. P., et al., 2012, ApJ, 760, 135
  * Rodriguez et al. (2019) Rodriguez C. L., Zevin M., Amaro-Seoane P., Chatterjee S., Kremer K., Rasio F. A., Ye C. S., 2019, Phys. Rev. D, 100, 043027
  * Rodriguez et al. (2021) Rodriguez C. L., Kremer K., Chatterjee S., Fragione G., Loeb A., Rasio F. A., Weatherford N. C., Ye C. S., 2021, arXiv e-prints, p. arXiv:2101.07793
  * Sarazin et al. (2003) Sarazin C. L., Kundu A., Irwin J. A., Sivakoff G. R., Blanton E. L., Rand all S. W., 2003, ApJ, 595, 743
  * Shih et al. (2010) Shih I. C., Kundu A., Maccarone T. J., Zepf S. E., Joseph T. D., 2010, ApJ, 721, 323
  * Shishkovsky et al. (2018) Shishkovsky L., et al., 2018, ApJ, 855, 55
  * Sivakoff et al. (2005) Sivakoff G. R., Sarazin C. L., Jordán A., 2005, ApJ, 624, L17
  * Skrutskie et al. (2006) Skrutskie M. F., et al., 2006, AJ, 131, 1163
  * Steele et al. (2011) Steele M. M., Zepf S. E., Kundu A., Maccarone T. J., Rhode K. L., Salzer J. J., 2011, ApJ, 739, 95
  * Steele et al. (2014) Steele M. M., Zepf S. E., Maccarone T. J., Kundu A., Rhode K. L., Salzer J. J., 2014, ApJ, 785, 147
  * Stetson (1987) Stetson P. B., 1987, PASP, 99, 191
  * Strader et al. (2012a) Strader J., Chomiuk L., Maccarone T. J., Miller-Jones J. C. A., Seth A. C., 2012a, Nature, 490, 71
  * Strader et al. (2012b) Strader J., et al., 2012b, ApJ, 760, 87
  * Swartz et al. (2004) Swartz D. A., Ghosh K. K., Tennant A. F., Wu K., 2004, ApJS, 154, 519
  * Tetarenko et al. (2016) Tetarenko B. E., Sivakoff G. R., Heinke C. O., Gladstone J. C., 2016, ApJS, 222, 15
  * Wiktorowicz et al. (2021) Wiktorowicz G., Lasota J.-P., Belczynski K., Lu Y., Liu J., Iłkiewicz K., 2021, arXiv e-prints, p. arXiv:2103.02026
  * Ye et al. (2020) Ye C. S., Fong W.-f., Kremer K., Rodriguez C. L., Chatterjee S., Fragione G., Rasio F. A., 2020, ApJ, 888, L10
  * Zepf et al. (2008) Zepf S. E., et al., 2008, ApJ, 683, L139
  * van der Walt et al. (2011) van der Walt S., Colbert S. C., Varoquaux G., 2011, Computing in Science and Engineering, 13, 22


# Primacy Effect of ChatGPT

Yiwei Wang†11footnotemark: 1 Yujun Cai‡ Muhao Chen§ Yuxuan Liang¶ Bryan Hooi∥
† University of California, Los Angeles ‡ Meta
§ University of California, Davis ∥ National University of Singapore
¶ Hong Kong University of Science and Technology (Guangzhou)
<EMAIL_ADDRESS>Equal contribution.

###### Abstract

Instruction-tuned large language models (LLMs), such as ChatGPT, have led to
promising zero-shot performance in discriminative natural language
understanding (NLU) tasks. This involves querying the LLM using a prompt
containing the question, and the candidate labels to choose from. The
question-answering capabilities of ChatGPT arise from its pre-training on
large amounts of human-written text, as well as its subsequent fine-tuning on
human preferences, which motivates us to ask: Does ChatGPT also inherits
humans’ cognitive biases? In this paper, we study the primacy effect of
ChatGPT: the tendency of selecting the labels at earlier positions as the
answer. We have two main findings: i) ChatGPT’s decision is sensitive to the
order of labels in the prompt; ii) ChatGPT has a clearly higher chance to
select the labels at earlier positions as the answer. We hope that our
experiments and analyses provide additional insights into building more
reliable ChatGPT-based solutions. We release the source code at
https://github.com/wangywUST/PrimacyEffectGPT.

##  1 Introduction

Humans tend to recall information presented at the start of a list better than
information at the middle or end. This phenomenon is known as the primacy
effect Asch (1946), which is a cognitive bias that relates to humans’
attention spans Crano (1977), rehearsal Tan and Ward (2000), and memory
systems Li (2010). Similarly, in advertisement systems and search engines,
humans tend to interact with items in higher positions regardless of the
items’ actual relevance Chen et al. (2023). Primacy effect influences humans’
behaviors to make unfair decisions. Similarly, if it exists in machine
learning models, it may lead to worse performance.

Recently, instruction-tuned large language models (LLMs), represented by
ChatGPT OpenAI (2022), have received wide attention on their capabilities of
imitating humans in question-answering and problem-solving. However, this
underlying behavioral similarity between ChatGPT and humans naturally leads to
an intriguing question: Is ChatGPT also affected by the primacy effect?

Figure 1:  Primacy Effect of ChatGPT: ChatGPT tends to return labels in
earlier positions as the answer. This plot shows the distribution of ChatGPT’s
predicted label indices in TACRED (42 classes), where we randomly shuffle
labels before every prediction (see Sec. 2.2).

ChatGPT provides a convenient way to achieve the discriminative natural
language understanding (NLU) Li et al. (2023); Wei et al. (2023); Yuan et al.
(2023). People only need to list the labels in the prompt and asking ChatGPT
to select the label(s) that match the input text. In this work, to analyze the
primacy effect of ChatGPT, we start by testing with random label shuffling,
i.e., shuffling labels listed in the prompt before every prediction. We
compare the predictions on the same instance with two different label orders.
Then, we count the predicted label indices on many instances with label
shuffling. The motivation is that: a fair NLU model should give the same
prediction on an input instance regardless of how the labels are ordered;
consequently, it should produce uniformly distributed label indices under
label shuffling for any instance.

Through extensive experiments with a series of NLU datasets, we find that

  * •

ChatGPT’s prediction is sensitive to the order of labels in the prompt.
Specifically, ChatGPT’s prediction changes after a label shuffling on 87.9% of
the instances in TACRED.

  * •

ChatGPT is affected by the primacy effect: ChatGPT tends to select labels in
earlier positions in the prompt (see Fig. 1), which present clear bias with
respect to the label order.

On the whole, our work contributes to a better understanding of ChatGPT’s
behaviors and building more faithful ChatGPT-based NLU solutions.

##  2 Primacy Effect of ChatGPT

In this section, we first introduce the general prompt design of ChatGPT in
discriminative natural language understanding (NLU). Then, we analyze the
primacy effect of ChatGPT using label shuffling in prompts.

###  2.1 Prompts for ChatGPT

Prompts are a key component to the effective use of ChatGPT on discriminative
NLU tasks Wei et al. (2023); Yuan et al. (2023). Generally, prompts for such
tasks involves two key components: (i) label definitions, and (ii) a task
description and input text (see an example in Fig. 2).

ChatGPT’s capability of understanding instructions in the prompt benefits from
its training with human feedback OpenAI (2022), but this also creates the
risks of inheriting humans’ cognitive biases. In this paper, we discuss a
cognitive bias in ChatGPT: the primacy effect, which indicates the tendency of
selecting labels in earlier positions in the prompt.

###  2.2 Analysis with Label Shuffling

Analyzing the primacy effect requires us to distill the effects of label
orders in the prompts. However, this is non-trivial because there are many
factors influencing ChatGPT’s decisions, such as the input text and label
definitions. In our work, to distinguish the primacy effect of ChatGPT from
other factors, we conduct random shuffling for labels listed in the prompts.
Specifically, before every prediction, we shuffle the labels as visualized in
Fig. 3. Label shuffling erases the discriminative semantics of the specific
label orders in the prompts. Ideally, a fair model should return the same
prediction when labels are shuffled, and consequently, the predicted label
index should follow a uniform distribution under random shuffling.

Next, we introduce our two ways of using random label shuffling to analyze
ChatGPT.

Prompts for Zero-shot Intent Detection Label Definitions
Label 1: change pin Label 2: card arrival Label 3: activate my card … Target
Text: I need a new PIN. Which Label matches the intent expressed in the Target
Text? Figure 2: A prompt example for ChatGPT. Figure 3:  We analyze the
primacy effects of ChatGPT by randomly shuffling the labels in the prompts.

#### Prediction Comparison on an Instance

A reliable and consistent classifier is expected to consistently choose the
same label for the same instance irrespective of the label order. To evaluate
such consistency of ChatGPT, we perform the random shuffling for the same
instance twice to produce two prompts. We feed these two prompts to ChatGPT
and compare the corresponding two predictions with each other. We apply the
above process to all the test instances and compute the fraction of the
instances where the prediction changed after label shuffling. The higher the
fraction is, the more sensitive ChatGPT is to the label order.

#### Statistics of Predicted Indices

Taking a further step, we perform statistical analysis on the predicted
indices for instances where the prediction changed after label shuffling. If
ChatGPT does not have any preference on the label orders, its predicted label
indices should be uniformly distributed. By comparing the predicted label
index distribution of ChatGPT to the uniform distribution, we can assess its
fairness and preferences regarding label orders.

##  3 Experiments

We analyze the primacy effect based on the aforementioned strategies using
three relation extraction datasets and an intent detection dataset.

###  3.1 Experiment Setup

We mainly chose relation extraction and intent detection tasks in our
experiments since these tasks naturally come with adequately sized decision
spaces to illustrate the underlying primacy effect of labels. For relation
extraction, we experiment on three benchmark datasets including TACRED Zhang
et al. (2017), TACREV Alt et al. (2020), and Re-TACRED Stoica et al. (2021).
For intent detection, we conducted experiments on Banking77 Casanueva et al.
(2020a) and MASSIVE FitzGerald et al. (2022). MASSIVE FitzGerald et al. (2022)
is a parallel dataset of massive utterances with annotations for the Natural
Language Understanding tasks of intent prediction. Utterances span 60 intents.

We additionally conducted experiments on the NLP datasets: GoEmotions Demszky
et al. (2020) and 20 Newsgroups Albishre et al. (2015) for a more
comprehensive evaluation. GoEmotions Demszky et al. (2020) is a dataset for
fine-grained emotion classification. It is a corpus of 58k carefully curated
comments extracted from Reddit, with human annotations for 27 emotion
categories and a neutral one. The 20 Newsgroups Albishre et al. (2015) dataset
is a collection of approximately 20,000 newsgroup documents, partitioned
across 20 different newsgroups.

We follow the existing work Wei et al. (2023); Li et al. (2023) to apply
ChatGPT to these tasks via the OpenAI API gpt-3.5-turbo. Specifically, we set
the temperature as 0.0 to minimize the randomness of ChatGPT’s outputs. For
comparison, we adopt the existing work Casanueva et al. (2020b); Zhou and Chen
(2021) to fine-tune the BERT model with an MLP classification head.

###  3.2 Consistency under Label Shuffling

First, we observe the low consistency of ChatGPT confronted under label
shuffling. As shown in Table 1, ChatGPT changes its label prediction after
label shuffling in over 85% of the test instances on the TACRED, TACREV, and
Re-TACRED datasets, and in 35.7% of instances on Banking77. Also, ChatGPT
changes its label prediction after label shuffling in over 69% of the test
instances on the datasets of GoEmotions and in more than 30% of instances on
MASSIVE and 20 Newsgroups. In contrast, the fine-tuned BERT classifier
maintains consistent predictions after label shuffling. This discrepancy
challenges the widely-held belief that ChatGPT can comprehend human
instructions and provide consistent responses. One possible explanation is
that ChatGPT’s understanding of the prompt is obtained by training on human-
labeled data, which inherits humans’ cognitive bias of treating labels at
different positions unfairly.

It is worth noting that the ratio of instances with changed predictions is
consistently high across the relation extraction datasets but lower on intent
detection. This discrepancy can be attributed to the fact that information
extraction tasks are shown to be challenging for ChatGPT and other LLMs Wang
et al. (2023); Li et al. (2023). In more difficult tasks, ChatGPT lacks
sufficient discriminative semantic understanding from the input text and may
be more affected by the label order.

Method | TACRED | TACREV | Re-TACRED | Banking77 | GoEmotions | MASSIVE | 20 Newsgroups
---|---|---|---|---|---|---|---
ChatGPT w/ Prompt | 87.9 | 85.9 | 88.6 | 35.7 | 69.3 | 32.8 | 34.1
BERT w/ MLP | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0
Table 1: Fraction of the instances that have their predicted label changed
after a label shuffling.

(a) TACRED

(b) TACREV

(c) Re-TACRED

(d) Banking77

Figure 4: The distribution of predicted indices of the test instances with
label shuffling before every prediction.

###  3.3 Primacy Effect of ChatGPT

The empirical results in Section 3.2 indicate that ChatGPT’s predictions are
affected by label order. To deeper delve into the effects of label orders on
ChatGPT, we analyze the distribution of predicted label indices (e.g., if the
prediction is the first label, the label index is $1$), as introduced in
Section 2.2. We visualize the distributions in Fig. 4. Notably, the
distribution of ChatGPT’s predictions consistently deviates from the uniform
distribution, displaying a consistent bias towards smaller indices across
different datasets. In contrast, BERT exhibits no preference for label orders
and consistently demonstrates a uniform distribution in its predicted label
indices.

We term this tendency of ChatGPT as the primacy effect, where the model tends
to favor the labels presented earlier in the prompt. The magnitude of these
primacy effects varies across tasks, as illustrated in Fig. 4. Notably, the
influence of primacy effects is higher in more challenging tasks. This
observation aligns with the results discussed in Sec. 3.3, wherein the impact
of primacy effects is greater when ChatGPT tackles more difficult tasks. In
the next section, we will quantitatively analyze the primacy effects of
ChatGPT.

Method | TACRED | TACREV | Re-TACRED | Banking77
---|---|---|---|---
ChatGPT w/ Prompts | 57.9 | 57.8 | 58.1 | 18.8
ChatGPT w/ CoT | 57.6 | 57.9 | 58.3 | 18.6
BERT w/ MLP | 1.8 | 1.9 | 2.3 | 2.1
Table 2: Experimental results (unfairness; %) on the test sets of TACRED,
TACRED-Revisit, Re-TACRED, and Banking77 (lower is better). The best results
in each column are highlighted in bold font.

###  3.4 Evaluation on Fairness

The fairness of a trained model can be assessed by examining the imbalance or
skewness in its predictions Sweeney and Najafian (2019). Following prior
studies Xiang et al. (2020); Sweeney and Najafian (2019); Qian et al. (2021);
Wang et al. (2022), we employ the JS divergence Fuglede and Topsoe (2004) as
the metric to evaluate how imbalanced/skewed/unfair a prediction $P$ is. The
measurement is symmetric (i.e., $\mathrm{JS}(P\|U)=\mathrm{JS}(U\|P)$) and
strictly scoped.

To evaluate the label order bias of ChatGPT, we compute the average relative
label order imbalance ($\mathrm{LOI}$): $\mathrm{LOI}$ is defined as the JS
divergence between the predicted label index distribution $P$ and the uniform
distribution $U$:

$\mathrm{LOI}=\mathrm{JS}(P({x|x\in\mathcal{D}}),U),$ (1)

where $x$ represents an input instance, $\mathcal{D}$ is the test set, $P(x)$
is the predicted label index, and $U$ is the uniform distribution.
$\mathrm{LOI}$ captures the disparity between the predicted indices and a
uniform distribution.

We conduct the fairness evaluation following the experimental settings
described in Section 3.3, and the results are presented in Table 2. The
findings demonstrate that ChatGPT exhibits unfair treatment of label indices
when making relation label predictions for input texts. Furthermore, the
degree of unfairness increases with the task’s difficulty, which aligns with
the empirical results discussed in Sections 3.2 and 3.3. In contrast, BERT
demonstrates significantly better fairness, as its predictions are not
influenced by label orders.

We additionally test the performance of ChatGPT with CoT (Chain-of-thoughts)
Wei et al. (2022). With CoT, ChatGPT still exhibits the primacy effect. The
above results show that with or without CoT, ChatGPT consistently exhibits the
primacy effect. A reason for this phenomenon could be that the CoT encourages
the LLMs for “slow thinking” about the question but does not necessarily
mitigate the cognitive bias in the reasoning steps of CoT.

##  4 Related Work

Large Language Models (LLMs) Brown et al. (2020); Rae et al. (2021); Thoppilan
et al. (2022); Smith et al. (2022), such as GPT-3 Brown et al. (2020), LaMDA
Thoppilan et al. (2022) and PaLM Chowdhery et al. (2022), refer to large scale
pretrained model that contain more than a hundred billion parameters. Based on
the the highly parallelizable Transformer architecture Vaswani et al. (2017),
these Large Language models have shown powerful capability to produce
reasonable results with very few samples or task descriptions as input.

A key milestone in the development process of LLMs is ChatGPT, which is
developed by OpenAI based on InstructGPTOuyang et al. (2022). ChatGPT is able
to interact with humans through multiple turns of dialogue, understand user
intent, accomplish instructions and return human-like responses. This attracts
huge attention from research field, motivating numerous recent work Zhang et
al. (2022); Ma et al. (2023); Wan et al. (2023); Zhong et al. (2023); Susnjak
(2023) to utilize ChatGPT to different tasks.

As ChatGPT is a proprietary model, and OpenAI does not disclose its training
specifics, researchers are actively investigating its associated implications
and capabilities. There have been some work analyzing the performance,
robustness, faithfulness, and explain-ability of ChatGPT Gao et al. (2023);
Han et al. (2023); Li et al. (2023). For example, Malinka et al. (2023)
investigates the educational integrity of ChatGPT and evaluates the ChatGPT’s
abilities to solve assignments of various levels in computer security
specialization. Haque et al. (2022) and Krügel et al. (2023) investigate the
ethical risks of ChatGPT.

Before ChatGPT, LLMs’ inference has been accompanied by in-context learning
(ICL) which adds a few demonstrations in the prompt Dong et al. (2022); Fei et
al. (2023). Accordingly, some work investigates the effects of demonstration
orders for the LLMs before ChatGPT Lu et al. (2021). Zhao et al. (2021) finds
the majority label, recency, and common token biases of LLMs’ ICL.

Different from the above work, we focus on a new phenomenon of ChatGPT: the
primacy effect, which is the tendency of selecting the first labels as the
answer. The Primacy effect seriously influence ChatGPT’s fairness.
Collectively, our findings provide a new understanding of how ChatGPT works
given the instructional prompts.

##  5 Conclusion

While previous work often takes ChatGPT as a universal method applicable to
all text-related tasks, we argue that its flexibility comes with the risk of
inheriting human’s cognitive biases. These biases lead to unfair judgments
which can affect the performance of the machine learning model. This work
studies a cognitive bias of ChatGPT: primacy effects. We propose a simple yet
effective label shuffling method to analyze the influence of label orders on
ChatGPT. We discover the primacy effects of ChatGPT and finds that it highly
influence the fairness of ChatGPT in NLU. Our work contributes to better
understanding the behaviors of ChatGPT and building more faithful solutions
with ChatGPT in NLU applications.

## Limitation

Our work has a few potential limitations. Firstly, we primarily evaluate the
primacy effect of ChatGPT, which is one of the most widely-used instruction-
legacy models for each task. It would be beneficial to assess this effect on
other LLMs models (such as Google Bard, vicuna Chiang et al. (2023)) and
explore additional tasks to examine this primacy effect. Secondly, this work
focused on analyzing the primacy effect of ChatGPT through experiments. We
encourage further studies to propose effective solutions that can mitigate the
negative impacts associated with the primacy effect.

## Acknowledgement

The authors would like to thank the anonymous reviewers for their discussion
and feedback. Yiwei Wang and Bryan Hooi are supported by NUS ODPRT Grant
A-0008067-00-00, NUS ODPRT Grant R252-000-A81-133, and Singapore Ministry of
Education Academic Research Fund Tier 3 under MOEs official grant number
MOE2017-T3-1-007. Muhao Chen is supported by the NSF Grant IIS 2105329, the
NSF Grant ITE 2333736, the DARPA MCS program under Contract No. N660011924033
with the United States Office Of Naval Research, a Cisco Research Award, two
Amazon Research Awards, and a Keston Research Award.

## References

  * Albishre et al. (2015) Khaled Albishre, Mubarak Albathan, and Yuefeng Li. 2015.  Effective 20 newsgroups dataset cleaning.  In _2015 IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology (WI-IAT)_ , volume 3, pages 98–101. IEEE.
  * Alt et al. (2020) Christoph Alt, Aleksandra Gabryszak, and Leonhard Hennig. 2020.  TACRED revisited: A thorough evaluation of the TACRED relation extraction task.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 1558–1569, Online. Association for Computational Linguistics.
  * Asch (1946) Solomon E Asch. 1946.  Forming impressions of personality.  _The Journal of Abnormal and Social Psychology_ , 41(3):258.
  * Brown et al. (2020) Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020.  Language models are few-shot learners.  _Advances in neural information processing systems_ , 33:1877–1901.
  * Casanueva et al. (2020a) Iñigo Casanueva, Tadas Temcinas, Daniela Gerz, Matthew Henderson, and Ivan Vulic. 2020a.  Efficient intent detection with dual sentence encoders.  In _Proceedings of the 2nd Workshop on NLP for ConvAI - ACL 2020_.  Data available at https://github.com/PolyAI-LDN/task-specific-datasets.
  * Casanueva et al. (2020b) Iñigo Casanueva, Tadas Temčinas, Daniela Gerz, Matthew Henderson, and Ivan Vulić. 2020b.  Efficient intent detection with dual sentence encoders.  _arXiv preprint arXiv:2003.04807_.
  * Chen et al. (2023) Jiawei Chen, Hande Dong, Xiang Wang, Fuli Feng, Meng Wang, and Xiangnan He. 2023\.  Bias and debias in recommender system: A survey and future directions.  _ACM Transactions on Information Systems_ , 41(3):1–39.
  * Chiang et al. (2023) Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez, Ion Stoica, and Eric P. Xing. 2023.  Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality.
  * Chowdhery et al. (2022) Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. 2022.  Palm: Scaling language modeling with pathways.  _arXiv preprint arXiv:2204.02311_.
  * Crano (1977) William D Crano. 1977.  Primacy versus recency in retention of information and opinion change.  _The Journal of Social Psychology_ , 101(1):87–96.
  * Demszky et al. (2020) Dorottya Demszky, Dana Movshovitz-Attias, Jeongwoo Ko, Alan Cowen, Gaurav Nemade, and Sujith Ravi. 2020.  Goemotions: A dataset of fine-grained emotions.  _arXiv preprint arXiv:2005.00547_.
  * Dong et al. (2022) Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Zhiyong Wu, Baobao Chang, Xu Sun, Jingjing Xu, and Zhifang Sui. 2022.  A survey for in-context learning.  _arXiv preprint arXiv:2301.00234_.
  * Fei et al. (2023) Yu Fei, Yifan Hou, Zeming Chen, and Antoine Bosselut. 2023.  Mitigating label biases for in-context learning.  _arXiv preprint arXiv:2305.19148_.
  * FitzGerald et al. (2022) Jack FitzGerald, Christopher Hench, Charith Peris, Scott Mackie, Kay Rottmann, Ana Sanchez, Aaron Nash, Liam Urbach, Vishesh Kakarala, Richa Singh, et al. 2022\.  Massive: A 1m-example multilingual natural language understanding dataset with 51 typologically-diverse languages.  _arXiv preprint arXiv:2204.08582_.
  * Fuglede and Topsoe (2004) Bent Fuglede and Flemming Topsoe. 2004.  Jensen-shannon divergence and hilbert space embedding.  In _International Symposium onInformation Theory, 2004. ISIT 2004\. Proceedings._ , page 31. IEEE.
  * Gao et al. (2023) Jinglong Gao, Xiao Ding, Bing Qin, and Ting Liu. 2023.  Is chatgpt a good causal reasoner? a comprehensive evaluation.  _arXiv preprint arXiv:2305.07375_.
  * Han et al. (2023) Ridong Han, Tao Peng, Chaohao Yang, Benyou Wang, Lu Liu, and Xiang Wan. 2023.  Is information extraction solved by chatgpt? an analysis of performance, evaluation criteria, robustness and errors.  _arXiv preprint arXiv:2305.14450_.
  * Haque et al. (2022) Mubin Ul Haque, Isuru Dharmadasa, Zarrin Tasnim Sworna, Roshan Namal Rajapakse, and Hussain Ahmad. 2022.  " i think this is the most disruptive technology": Exploring sentiments of chatgpt early adopters using twitter data.  _arXiv preprint arXiv:2212.05856_.
  * Krügel et al. (2023) Sebastian Krügel, Andreas Ostermaier, and Matthias Uhl. 2023.  The moral authority of chatgpt.  _arXiv preprint arXiv:2301.07098_.
  * Li et al. (2023) Bo Li, Gexiang Fang, Yang Yang, Quansen Wang, Wei Ye, Wen Zhao, and Shikun Zhang. 2023.  Evaluating chatgpt’s information extraction capabilities: An assessment of performance, explainability, calibration, and faithfulness.  _arXiv preprint arXiv:2304.11633_.
  * Li (2010) Cong Li. 2010.  Primacy effect or recency effect? a long-term memory test of super bowl commercials.  _Journal of Consumer Behaviour: An International Research Review_ , 9(1):32–44.
  * Lu et al. (2021) Yao Lu, Max Bartolo, Alastair Moore, Sebastian Riedel, and Pontus Stenetorp. 2021\.  Fantastically ordered prompts and where to find them: Overcoming few-shot prompt order sensitivity.  _arXiv preprint arXiv:2104.08786_.
  * Ma et al. (2023) Yubo Ma, Yixin Cao, YongChing Hong, and Aixin Sun. 2023.  Large language model is not a good few-shot information extractor, but a good reranker for hard samples!  _arXiv preprint arXiv:2303.08559_.
  * Malinka et al. (2023) Kamil Malinka, Martin Peresíni, Anton Firc, Ondrej Hujnak, and Filip Janus. 2023.  On the educational impact of chatgpt: Is artificial intelligence ready to obtain a university degree?  In _Proceedings of the 2023 Conference on Innovation and Technology in Computer Science Education V. 1_ , pages 47–53.
  * OpenAI (2022) OpenAI. 2022.  Introducing chatgpt https://openai.com/blog/chatgpt).
  * Ouyang et al. (2022) Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022\.  Training language models to follow instructions with human feedback.  _Advances in Neural Information Processing Systems_ , 35:27730–27744.
  * Qian et al. (2021) Chen Qian, Fuli Feng, Lijie Wen, Chunping Ma, and Pengjun Xie. 2021.  Counterfactual inference for text classification debiasing.  In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_ , pages 5434–5445.
  * Rae et al. (2021) Jack W Rae, Sebastian Borgeaud, Trevor Cai, Katie Millican, Jordan Hoffmann, Francis Song, John Aslanides, Sarah Henderson, Roman Ring, Susannah Young, et al. 2021.  Scaling language models: Methods, analysis & insights from training gopher.  _arXiv preprint arXiv:2112.11446_.
  * Smith et al. (2022) Shaden Smith, Mostofa Patwary, Brandon Norick, Patrick LeGresley, Samyam Rajbhandari, Jared Casper, Zhun Liu, Shrimai Prabhumoye, George Zerveas, Vijay Korthikanti, et al. 2022.  Using deepspeed and megatron to train megatron-turing nlg 530b, a large-scale generative language model.  _arXiv preprint arXiv:2201.11990_.
  * Stoica et al. (2021) George Stoica, Emmanouil Antonios Platanios, and Barnabás Póczos. 2021.  Re-tacred: Addressing shortcomings of the tacred dataset.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , volume 35, pages 13843–13850.
  * Susnjak (2023) Teo Susnjak. 2023.  Applying bert and chatgpt for sentiment analysis of lyme disease in scientific literature.  _arXiv preprint arXiv:2302.06474_.
  * Sweeney and Najafian (2019) Chris Sweeney and Maryam Najafian. 2019.  A transparent framework for evaluating unintended demographic bias in word embeddings.  In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_ , pages 1662–1667.
  * Tan and Ward (2000) Lydia Tan and Geoff Ward. 2000.  A recency-based account of the primacy effect in free recall.  _Journal of Experimental Psychology: Learning, Memory, and Cognition_ , 26(6):1589.
  * Thoppilan et al. (2022) Romal Thoppilan, Daniel De Freitas, Jamie Hall, Noam Shazeer, Apoorv Kulshreshtha, Heng-Tze Cheng, Alicia Jin, Taylor Bos, Leslie Baker, Yu Du, et al. 2022.  Lamda: Language models for dialog applications.  _arXiv preprint arXiv:2201.08239_.
  * Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017.  Attention is all you need.  In _Advances in neural information processing systems_ , pages 5998–6008.
  * Wan et al. (2023) Zhen Wan, Fei Cheng, Zhuoyuan Mao, Qianying Liu, Haiyue Song, Jiwei Li, and Sadao Kurohashi. 2023.  Gpt-re: In-context learning for relation extraction using large language models.  _arXiv preprint arXiv:2305.02105_.
  * Wang et al. (2023) Fei Wang, Wenjie Mo, Yiwei Wang, Wenxuan Zhou, and Muhao Chen. 2023.  A causal view of entity bias in (large) language models.  _arXiv preprint arXiv:2305.14695_.
  * Wang et al. (2022) Yiwei Wang, Muhao Chen, Wenxuan Zhou, Yujun Cai, Yuxuan Liang, Dayiheng Liu, Baosong Yang, Juncheng Liu, and Bryan Hooi. 2022.  Should we rely on entity mentions for relation extraction? debiasing relation extraction with counterfactual analysis.  _arXiv preprint arXiv:2205.03784_.
  * Wei et al. (2022) Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022.  Chain-of-thought prompting elicits reasoning in large language models.  _Advances in Neural Information Processing Systems_ , 35:24824–24837.
  * Wei et al. (2023) Xiang Wei, Xingyu Cui, Ning Cheng, Xiaobin Wang, Xin Zhang, Shen Huang, Pengjun Xie, Jinan Xu, Yufeng Chen, Meishan Zhang, et al. 2023.  Zero-shot information extraction via chatting with chatgpt.  _arXiv preprint arXiv:2302.10205_.
  * Xiang et al. (2020) Liuyu Xiang, Guiguang Ding, and Jungong Han. 2020.  Learning from multiple experts: Self-paced knowledge distillation for long-tailed classification.  In _European Conference on Computer Vision_ , pages 247–263. Springer.
  * Yuan et al. (2023) Chenhan Yuan, Qianqian Xie, and Sophia Ananiadou. 2023.  Zero-shot temporal relation extraction with chatgpt.  _arXiv preprint arXiv:2304.05454_.
  * Zhang et al. (2022) Bowen Zhang, Daijun Ding, and Liwen Jing. 2022.  How would stance detection techniques evolve after the launch of chatgpt?  _arXiv preprint arXiv:2212.14548_.
  * Zhang et al. (2017) Yuhao Zhang, Victor Zhong, Danqi Chen, Gabor Angeli, and Christopher D Manning. 2017\.  Position-aware attention and supervised data improve slot filling.  In _Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing_ , pages 35–45.
  * Zhao et al. (2021) Zihao Zhao, Eric Wallace, Shi Feng, Dan Klein, and Sameer Singh. 2021.  Calibrate before use: Improving few-shot performance of language models.  In _International Conference on Machine Learning_ , pages 12697–12706. PMLR.
  * Zhong et al. (2023) Qihuang Zhong, Liang Ding, Juhua Liu, Bo Du, and Dacheng Tao. 2023.  Can chatgpt understand too? a comparative study on chatgpt and fine-tuned bert.  _arXiv preprint arXiv:2302.10198_.
  * Zhou and Chen (2021) Wenxuan Zhou and Muhao Chen. 2021.  An improved baseline for sentence-level relation extraction.  _arXiv preprint arXiv:2102.01373_.