[a,b]Michael Pitt

# Constraining models of hadronic showers using proton-Oxygen collisions at
the LHC involving proton/neutron tagging

###### Abstract

The study of hadronic showers, which are produced by cosmic rays penetrating
Earth’s atmosphere, is essential for shedding light on the origins and
characteristics of high-energy particles originating from space and reaching
our planet. At the Large Hadron Collider at CERN, there are plans to conduct a
short run of proton–oxygen collisions toward the final years of LHC Run 3,
anticipated in 2024, in order to refine the modeling of hadronic showers. This
work explores the potential impact on constraining models of hadronic showers
by measuring interactions facilitated by color-neutral objects such as
photons, pomerons, and pions. These interactions are often characterized by
high-energy protons or neutrons produced at forward rapidities, and detecting
and analyzing these events is possible through the use of dedicated detectors
for forward neutrons and protons.

##  1 Introduction

Cosmic rays (CRs) are high-energy particles originating from outer space, with
some possessing energies up to 1021 eV. Unraveling the nature of ultra-high-
energy CRs is at the forefront of astrophysical research. A vital component in
demystifying the properties of CRs is determination of the mass and energy
spectra of the extended air showers that they produce. Such assessments depend
on precise measurements and simulations of the air shower profiles, modeled
through hadronic Monte Carlo (MC) simulations that are often tuned using data
from the Large Hadron Collider (LHC) [1, 2, 3, 4]. Nonetheless, discrepancies
between different model predictions persist, even at LHC energies, leading to
substantial systematic uncertainties in our understanding of the composition
of CRs [5]. To improve the modeling of hadronic interactions, plans are
underway to conduct a short run of proton–oxygen ($pO$) collisions during LHC
Run 3 [6].

For the upcoming proton–oxygen collision runs, the primary focus of the
standard research program will predominantly be on non-diffractive
interactions [7]; however, interactions involving color-neutral objects can
also be explored by tagging forward protons in $pO\to pX$ interactions or
forward neutrons in $pO\to nY$ interactions, providing a unique opportunity to
study those components with better precision. Schematic diagrams of processes
of interest are illustrated in Figure 1.

Figure 1: Schematic diagrams of $pO$ collisions with an intact proton (left)
or a neutron (right) produced at very forward rapidities.

The color-neutral interactions are weakly constrained at the LHC, resulting in
substantial discrepancies between the experimental data and the predictions of
MC simulations (e.g., in proton–lead collisions [8]), suggesting there are
missing interactions not included in these event generators, which can be
further probed using forward neutron and proton detectors.

##  2 The forward neutron and proton detectors at the LHC

The inclusion of forward neutron and proton detectors at the LHC has
significantly broadened the research capabilities of the ATLAS and CMS
experiments within the heavy ion and proton–proton physics programs. These
detectors can assist in studying the properties of color-neutral interactions
during forthcoming proton–oxygen collisions at the LHC. The arrangement of the
detector devices along the LHC beamline, on both sides of the interaction
point (IP), is schematically illustrated in Figure 2.

Figure 2: Schematic layout of the insertion region between the interaction
point and quadrupole magnet Q6. Figure taken from Ref. [13].

###  2.1 The Forward Proton Detectors (FPDs)

During LHC Run 2, forward proton detectors (FPDs) were successfully deployed
as part of the ATLAS and CMS collaborations. FPDs are specialized near-beam
detectors located approximately 200 meters from the IP. These detectors were
operational throughout the standard high-luminosity runs at the LHC, where
they were primarily dedicated to studying the central exclusive production
processes in proton–proton collisions. The ATLAS forward proton (AFP) detector
[14] and the CMS-TOTEM Precision Proton Spectrometer (CT-PPS) [15] have been
seamlessly integrated during standard LHC runs and delivered a broad range of
physics results [9, 10].

The kinematic acceptance for forward protons is contingent upon the
arrangement of the LHC magnetic field. Protons in diffractive interactions
lose a fraction of their momentum (denoted by $\xi=\Delta p/p$) and are
consequently deflected from their initial beam trajectory. During standard LHC
runs, $\xi$ typically ranges from 1.5% to 15%. This acceptance range is also
expected to apply to the upcoming proton–oxygen run, providing the capability
to measure the diffractive component of the total $pO$ cross-section by
tagging the intact protons.

###  2.2 Zero Degree Calorimeter (ZDC)

The zero degree calorimeter (ZDC) is a specialized detector positioned at a
zero-degree angle relative to the beamline, contributing substantially to the
heavy ion physics program at the LHC [11, 12]. Its primary function is to
detect forward neutral particles produced in $AA$ and $pA$ collisions at the
LHC, primarily spectators from ion disintegration. The ZDC detectors for both
ATLAS and CMS experiments are hosted in a dedicated slot inside the neutral
beam absorbers (TAN) at a distance of 140 meters from the IP, which shields
the LHC machine components against neutral particles emerging from the IP.

The ZDC plays a crucial role in detecting forward neutrons and photons with
pseudorapidities greater than 8.5 in $pp$, $pA$, and $AA$ collisions. It
consists of an electromagnetic section, approximately 30 radiation lengths
long, and three hadronic modules, each about 1.15 interaction lengths long
[11].

##  3 New constrains on MC hadronic models

In the forthcoming proton–oxygen run, colorless interactions (including
elastic, diffractive, and pion exchange processes), which account for
approximately 20% of the total cross-section, can be tagged by operating the
ZDC and FPDs downstream of the proton beam. The typical spectra of forward
protons and neutrons originating from color-neutral interactions in
proton–oxygen collisions at $\sqrt{S_{\text{NN}}}=9.9$ TeV using different
Monte Carlo event generators are depicted in Figure 3 and Figure 4,
respectively.

Figure 3: Differential cross-section as a function of proton energy (left) and
transverse momentum (right) for $pO$ collisions assuming integrated luminosity
of $L_{\text{int}}=1nb^{-1}$ at center-of-mass energy per nucleon pair at
$\sqrt{S_{\text{NN}}}=9.9$ TeV, obtained using the EPOS-LHC (magenta), Pythia8
Angantyr (yellow), Sibyll 2.3d (green), and QGSJETII-04 (blue) MC event
generators.

Figure 4: Differential cross-section as a function of neutron energy (left)
and transverse momentum (right) for $pO$ collisions assuming integrated
luminosity of $L_{\text{int}}=1nb^{-1}$ at center-of-mass energy per nucleon
pair at $\sqrt{S_{\text{NN}}}=9.9$ TeV, obtained using the EPOS-LHC (magenta),
Pythia8 Angantyr (yellow), Sibyll 2.3d (green), and QGSJETII-04 (blue) MC
event generators.

Elastic and diffractive interactions within proton–oxygen collisions are
characterized by large gaps in the rapidity distribution of the final-state
particles, defined as $\Delta\eta_{F}$. In contrast to non-diffractive
inelastic events, where the probability of finding a continuous rapidity
region $\Delta\eta_{F}$ free of particles is exponentially suppressed, color-
neutral interactions such as those involving pomeron or pion exchange are
distinctive for their clear rapidity gaps, and discriminating between these
topologies is only achievable through proton and neutron tagging. Figure 5
illustrates the contribution from diffractive processes, where a proton is
measured by an FPD, or a neutron is measured by the ZDC, as a function of the
$\Delta\eta_{\text{F}}$ spectra.

Figure 5: Contribution from $pO\to pX$ and $pO\to nX$ interactions with a
proton within the FPD acceptance or a neutron within the ZDC acceptance. The
peak in the $\Delta\eta_{\text{F}}$ distribution corresponds to zero-bias
events (i.e., no particles with energy above 1 GeV are detected within the
pseudorapidity range of $|\Delta\eta|<4.5$). Figure taken from Ref. [13]

##  4 Conclusions

As preparations for proton–oxygen collision runs at the LHC continue, forward
neutron and proton detectors are expected to become increasingly vital,
allowing the scope of the existing physics research program to be augmented
with oxygen beams. These detectors will broaden the range of the observable
phase space, which is pivotal in refining our understanding of color-neutral
interactions. They are poised to provide precise constraints on diffractive
and elastic interactions in proton–ion collisions, including the first
measurement of the elastic component of proton–oxygen interactions. If
successfully executed, this research could serve as a stepping stone for
future measurements in heavy ion runs at the LHC, leveraging both the forward
proton detector (FPD) and the zero degree calorimeter (ZDC) to explore new
frontiers in particle physics.

## References

  * [1] T. Pierog, I. Karpenko, J. M. Katzy, E. Yatsenko and K. Werner, _EPOS LHC: Test of collective hadronization with data measured at the CERN Large Hadron Collider_. _Phys. Rev. C_ 92 (2015) 3, 034906.
  * [2] F. Riehn, R. Engel, A. Fedynitch, T. K. Gaisser and T. Stanev, _Hadronic interaction model Sibyll 2.3d and extensive air showers_. _Phys. Rev. D_ 102 (2020) 6, 063002.
  * [3] S. Ostapchenko, _Monte Carlo treatment of hadronic interactions in enhanced Pomeron scheme: I QGSJET-II model_. _Phys. Rev. D_ 83 (2011), 014018.
  * [4] C. Bierlich, G. Gustafson, L. Lönnblad and H. Shah, _The Angantyr model for Heavy-Ion Collisions in PYTHIA8_. _JHEP_ 10 ((2018), 134.
  * [5] A. D. Supanitsky, _Determination of the Cosmic-Ray Chemical Composition: Open Issues and Prospects_. _Galaxies_ 10 (2022) 3, 75.
  * [6] R. Bruce, R. Alemany-Fernández, H. Bartosik, M. Jebramcik, J. Jowett and M. Schaumann, _Studies for an LHC Pilot Run with Oxygen Beams_ _Proc. IPAC’21_ , Campinas, SP, Brazil, May 2021, pp. 53–56.
  * [7] Brewer, Jasmine and Mazeliauskas, Aleksas and van der Schee, Wilke, _Opportunities of OO and $p$O collisions at the LHC_. CERN-TH-2021-028
  * [8] CMS Collaboration. First measurement of the forward rapidity gap distribution in pPb collisions at $\sqrt{s_{\mathrm{NN}}}$ = 8.16 TeV. _Phys. Rev. D_ 108, (2023) 9, 092004.
  * [9] A. Sopczak, Overview of ATLAS Forward Proton (AFP) detectors in Run-2 and outlook for Run-3 analyses. arXiv:2307.09780 [hep-ex].
  * [10] C. Royon, Recent results from the CMS Proton Precision Spectrometer. arXiv:2307.05321 [hep-ph].
  * [11] O. Surányi, A. Al-Bataineh, J. Bowen, S. Cooper, M. Csanád, V. Hagopian, D. Ingram, C. Ferraioli, T. Grassi and R. Kellogg, et al. Performance of the CMS Zero Degree Calorimeters in pPb collisions at the LHC. _JINST_ 16, (2021) 5, P05008.
  * [12] ATLAS Collaboration. Evidence for light-by-light scattering in heavy-ion collisions with the ATLAS detector at the LHC. _Nature Phys._ 13, (2017) 9, 852-858.
  * [13] M. Pitt, _Reducing model uncertainties using proton-oxygen collisions with proton/neutron tagging at the LHC_ , _PoS_ ICRC2023, 426 (2023).
  * [14] L. Adamczyk et al., _Technical Design Report for the ATLAS Forward Proton Detector_. CERN-LHCC-2015-009, ATLAS-TDR-024. https://cds.cern.ch/record/2017378
  * [15] CMS and TOTEM Collaborations, _CMS-TOTEM Precision Proton Spectrometer_. CERN-LHCC-2014-021 ; TOTEM-TDR-003 ; CMS-TDR-013 https://cds.cern.ch/record/1753795


XX XX x May Ethical ChatGPT: Concerns, Challenges, and Commandments

# Ethical ChatGPT: Concerns, Challenges, and Commandments

Jianlong Zhou  Data Science Institute, University of Technology Sydney,
Sydney, NSW 2007, Australia  Heimo Müller  Human-Centered AI Lab, Medical
University Graz, Austria  Andreas Holzinger  Human-Centered AI Lab, University
of Natural Resources and Life Sciences Vienna and Medical University Graz,
Austria  Fang Chen  Data Science Institute, University of Technology Sydney,
Sydney, NSW 2007, Australia

(2023)

###### Abstract

Large language models, e.g. ChatGPT are currently contributing enormously to
make artificial intelligence even more popular, especially among the general
population. However, such chatbot models were developed as tools to support
natural language communication between humans. Problematically, it is very
much a “statistical correlation machine" (correlation instead of causality)
and there are indeed ethical concerns associated with the use of AI language
models such as ChatGPT, such as Bias, Privacy, and Abuse. This paper
highlights specific ethical concerns on ChatGPT and articulates key challenges
when ChatGPT is used in various applications. Practical commandments for
different stakeholders of ChatGPT are also proposed that can serve as
checklist guidelines for those applying ChatGPT in their applications. These
commandment examples are expected to motivate the ethical use of ChatGPT.

Chat Generative Pre-Trained Transformer (also known as ChatGPT), can fluently
answer questions from users and has the ability to generate human-like text
with a seemingly logical connection between different sections. Individuals
have reportedly used ChatGPT to formulate university essays, scholarly
articles with references [1], debug computer program code, compose music,
write poetry, give restaurant reviews, generate advertising copy and solve
exams [2], co-author journal articles [3] and many others.

ChatGPT models are basically massive neural networks with billions of
parameters, which resulted in gains in quality, accuracy, and breadth of
generated content. Their behaviors are learned from a large amount of text
data of Internet resources such as web pages, books, research articles and
social chatter, not programmed explicitly. They are trained with two phases:
1) the initial “pre-training” phase learns to predict the next word in a
sentence with a large amount of Internet text from a vast array of
perspectives; and 2) the second phase “fine-tunes” models with the use of
datasets that human reviewers crafted to narrow down system behavior [4]. Such
combination of unsupervised pre-training and supervised fine-tuning helps to
generate human-like responses to queries and in particular provide responses
to queried topics that resemble that of a human expert [5].

The rapid widespread adoption of ChatGPT after release has demonstrated its
tremendous powerfulness of potential uses in different areas ranging from
technical assistance such as coding, essay writing, business letters, to
customer engagement as well as many others [6]. Despite the powerful capacity
of ChatGPT to help people with various writing tasks and experiments
engendering both positive and adverse impacts, the society has critical
concerns on allowing users to cheat and plagiarize especially in academy and
education communities, potentially spreading misinformation, and enabling
unethical business practices as well as other ethical issues [7].

Weidinger et al. [8] summarises the ethical risk landscape with Large Language
Models (LLM), identifying six ethical concerns: 1) Discrimination, Exclusion,
and Toxicity, 2) Information Hazards, 3) Misinformation Harms, 4) Malicious
Uses, 5) Human-Computer Interaction Harms, and 6) Automation, Access, and
Environmental Harms. ChatGPT shares not only the similar ethical issues with
other AI solutions including fairness, privacy and security, transparency,
accountability, etc. [9, 10], it may also introduce additional ethical
concerns because of its specific characteristics. For example, people have
difficulty to distinguish facts and fake with the ChatGPT’s human-like
conversations; in education, teachers may have difficulty to differentiate the
authorship between human and AI in home work; in the creative areas such as
designing and creative writing, ChatGPT may introduce changes to not only the
authorship, but also the creativity of human in the long time.

This paper first highlights specific ethical concerns on ChatGPT and
articulates key challenges when ChatGPT is used in various applications. We
then propose practical commandments for different stakeholders of ChatGPT that
can serve as checklist guidelines for those applying ChatGPT in their
applications.

## ETHICAL CONCERNS

This section demonstrates typical ethical concerns on ChatGPT as shown in
Figure 1.

Figure 1: Examples of ethical concerns on ChatGPT

### Bias

Similar to many other AI solutions, ChatGPT could also demonstrate bias in its
answers. These biases have arisen because of different reasons such as the
machine learning algorithms used for modelling and the data used for training
and fine-tuning. Despite the use of human data labellers following
instructions by ChatGPT for training and fine-tuning datasets, it must be
recognised that the data labellers are not representative for diverse
viewpoints and perspectives which introduces biases to data. Furthermore,
training data is primarily from massive Internet resources which not only have
limited diversity but also may have biases within itself. For example, [11]
showed that such large datasets significantly over-represent younger users,
especially people from developed countries and English speakers. Any biases
present in these data will be reflected in the output of the model. Such bias
is hard to overcome [12]. OpenAI lists this issue in its announcement blog
post saying that ChatGPT is “often excessively verbose and overuses certain
phrases, such as restating that it’s a language model trained by OpenAI. These
issues arise from biases in the training data (trainers prefer longer answers
that look more comprehensive) and well-known over-optimization issues”.

These biases can result in unavoidable unfair results of ChatGPT answers,
particularly for vulnerable groups.

### Privacy and Security

ChatGPT generates answers based on the input it receives. Such input-output
pairs may also be used to fine-tune ChatGPT. These may inadvertently reveal
sensitive information of users. Individuals’ interaction histories with
ChatGPT may also be used to track and profile individuals. In addition, many
of the databases that ChatGPT can use come from the Internet even social
platforms such as Twitter, which means that ChatGPT may learn content that may
leaks privacy of individuals and lacks fact-checking, and further not only
generate incorrect or wrong information, but also cause privacy issues.

### Transparency

Figure 2 shows two main steps in building ChatGPT. However, OpenAI did not
release much information about ChatGPT. For example, what training data is
used, what are the training and testing data and their sizes, what model is
used, what are the review instructions, and who are reviewers are not
transparent. But OpenAI heavily emphasized its performance on question
answering. Therefore, ChatGPT’s inner workings are opaque to users, which can
make it difficult to understand how it arrives at its responses. The lack of
transparency affects user trust in ChatGPT the ability of users to make
informed decisions about how to use the model.

Figure 2: Two main steps in building ChatGPT (adapted from [4])

Furthermore, ChatGPT’s answer is “random” based on statistical models. The
same question may give slightly different answers in differet queries with
ChatGPT. The lack of awareness of randomness makes ChatGPT less trustworthy
[13].

### Abuse

The primary goal of ChatGPT is to generate seemly reasonable human-like text
responses to inputs using natural language. However, trained with
reinforcement learning, it currently does not have source of truth and does
not include the accuracy. The ability to generate human-like text could result
in the misue and abuse of the technology such as spreading misinformation or
impersonating individuals.

For example, programmers acted firstly and are more likely to use such
generative AI tools. StackOverflow, a popularr question-and-answer site for
programmers, had already moved to ban the submission of ChatGPT-generated
answers as the site explained, “Overall, because the average rate of getting
correct answers from ChatGPT is too low, the posting of answers created by
ChatGPT is substantially harmful to the site and to users who are asking and
looking for correct answers” [14]. Some big financial companies also banned
the use of ChatGPT in their work mainly concerning its accountability.

Furthermore, phishing email scams, online job hunting and dating scams, and
even political propaganda may benefit from human-like text from ChatGPT. Just
imagine, in the past, cross-border fraudulent emails were often exposed due to
insufficient language translation, but with AI capable of translation and text
generation, it may be even more difficult to detect.

### Authorship

Since ChatGPT can generate human-like writings using natural language, it may
be used in different situations that need text writings. For example, students
may use ChatGPT in their home work or report writing. More and more academics
from universities pointed out that they received text reports generated by
students using ChatGPT. They have difficulty to differentiate the authorship
for the plagiarism concerns between students and AI so that teachers can
evaluate students’ performance objectively. Furthermore, ChatGPT has been used
in creative work such as creative writing and music composing, which
introduces not only the authorship concerns but also the human’s creativity
concerns.

## CHALLENGES

Despite the potential of AI and ChatGPT to greatly enhance many areas of life,
including communication and problem-solving in various domains, even in
medicine [15], as with any new technology, it is important to be aware of
potential challenges that may arise. The use of ChatGPT in various
applications has prompted the identification of the following challenges:

  * Blind trust — Over-reliance on AI systems without proper validation or verification can lead to incorrect or inappropriate decision making, potentially resulting in harm to users or other negative consequences. ChatGPT lacks the source of truth of responses, the over-reliance on ChatGPT in decision making may result in unexpected harm to users. While the check of the truth of ChatGPT responses is a challenge.

  * Over-regulation (no guts, no glory) — Excessive regulation could prevent innovation and progress, as overly strict regulations could limit the ability of private and commercial users to experiment and take well-known risks with new AI technologies. ChatGPT has been demonstrating its strong capabilities since its first release. However, some counties and organisations have banned its uses in their organisations because of various adverse concerns. It is a challenge for stakeholders in the regulation of its uses. Although the regulation of the uses of ChatGPT is highly important, the over-regulation may affect the innovation progress of new technologies.

  * Dehumanization — AI systems that replace human interaction and compassion in human-to-human relationships can lead to a loss of empathy and decreased satisfaction in society. The human-like responses from ChatGPT may result in the difficulty in differentiating machines and humans and thus affects human-to-human relationships. It is currently still a challenge to differentiate responses from ChatGPT and humans.

  * Wrong targets in optimization — AI systems that prioritize metrics that do not align with social norms can result in social dislocations. Such norms are unwritten rules or expectations that guide behavior and interactions within a community. Social norms can be formal or informal, and they can vary based on cultural, social, and historical contexts. ChatGPT is also without exception in lacking the alignment with social norms fully to prioritize its performance metrics. It is a challenge to consider such social norms in ChatGPT.

  * Over-informing and false forecasting — AI systems that generate too much information or provide false predictions can lead to confusion and decreased trust in the technology. Users can get any number of responses for one query from ChatGPT and there is no any accuracy information on those responses. Therefore, it is a challenge for ChatGPT to foster trust under such conditions. AI systems that rely solely on statistical models without considering individual user circumstances can lead to incorrect or inappropriate actions. Responses from ChatGPT are randomly statistically generated. Different responses may be generated for a same query. ChatGPT may generate incorrect responses because of its statistical characteristics, and it is a challenge to generalise its responses.

  * Self-reference (AI-based) monitoring — AI systems that rely solely on themselves for evaluation, without independent oversight, can lead to a lack of accountability and decreased transparency in decision making. As shown in Figure 2, ChatGPT uses both supervised and unsupervised learning to train the model. OpenAI did not open much information on how ChatGPT is evaluated and monitored despite the use of self-reference approach commonly used in the community.

## RECOMMENDATIONS

Considering significant challenges of ChatGPT as discussed above, this section
provides recommendations to different stakeholders in ChatGPT for its
responsible uses. We first identify various stakeholders involved in ChatGPT.
Recommendations are then suggested to different stakeholders.

### Stakeholders in ChatGPT

There are various stakeholders involved in ChatGPT. Here are some examples:

  * Researchers and developers — These stakeholders are involved in developing and improving ChatGPT technologies. They may work for academic institutions, research organizations, or private companies.

  * Users and consumers — These stakeholders are the end-users of ChatGPT technologies. They may use ChatGPT for various purposes, such as information retrieval, language translation, and creative writing.

  * Regulators and policymakers — They are responsible for establishing legal and ethical guidelines for the development and use of ChatGPT technologies. They may work for government agencies, international organizations, or industry associations. These stakeholders collaborate closely with advocacy groups and civil society organizations, which represent the interests of various groups affected by ChatGPT technologies, such as privacy advocates, human rights groups, and marginalized communities. Advocacy groups may lobby for policy changes or raise public awareness about the risks and benefits of these technologies.

  * Ethicists and social scientists are stakeholders who focus on the ethical and social implications of ChatGPT technologies. They may study the impact of these technologies on society, culture, and human behavior. Their role is to reflect on the developments and provide guidance on how to address any ethical or social issues that arise. While they may not directly influence the development of these technologies, their work helps ensure that ChatGPT technologies are developed and used responsibly.

### Recommendations for researchers and developers

Considering challenges ChatGPT faces and characteristics of researchers and
developers, the recommendations for researchers and developers of ChatGPT are:

  * Do not be a algorithmic pied piper and seduce and deliberately mislead your users. Take responsibility for providing background information about bias, privacy in an active way. If possible, offer a feature to explain why a particular statement was made in ChatGPT.

  * Protect the vulnerable — It is important to protect vulnerable individuals who may not fully understand the disclaimer in ChatGPT. This includes children, young people, and individuals with cognitive disabilities or lower cognitive function, who may require additional protection.

  * Give reasons for answers, avoid made-up sentences unless they are explicitly requested — This command is important for ChatGPT because it emphasises the importance of providing clear and well-justified responses to users. When the ChatGPT generates an outcome or response, it should be able to explain the reasoning behind it, rather than simply providing a result without any explanation.

Providing justification for a response can help build trust and credibility
with the user, and can also help the user better understand the bot’s thought
process and decision-making. Additionally, this command emphasises that the
bot should only produce outcomes when they are deliberately requested by the
user, rather than providing unsolicited responses.

  * Connect ChatGPT to domain knowledge — Connecting ChatGPT models to domain-specific knowledge representations curated by a community and/or experts can greatly enhance the accuracy and relevance of the responses provided by the model. These knowledge representations can take different forms, such as taxonomies, ontologies, or knowledge graphs, and can be both human-readable and machine-readable. By including domain-specific knowledge in the training data, the model can learn to incorporate this information into its responses. This approach may require significant domain expertise to curate and annotate the training data.

### Recommendations for users

The recommendations for users of ChatGPT include:

  * Check, re-check, double-check., if users intend to use the result of a ChatGPT conversation as fact. This is a fundamental principle of reliable science as well as trustworthy journalism that emphasises the importance of verifying information before publishing it. Before sharing any information on ChatGPT, make sure to check the source and ensure that it is credible and reliable. Avoid sharing information from unknown or unverified sources. Also be aware of own biases and those of others in the chat. Double-check any information that seems too good to be true or aligns too closely with your own beliefs. When using ChatGPT, it is important to critically evaluate the information presented, distinguishing between fact and fiction, and considering how the responses were generated.

  * Don’t mix facts and fiction — To use ChatGPT responsibly, it is important to distinguish between reality and fiction and to contextualize the information obtained from the platform. While it is not necessary to rely solely on factual information, it is crucial to differentiate between statements that are part of a fictional story and those that are intended to be true. For instance, scientific statements can be presumed to be accurate, whereas statements in a work of fiction may not be. Therefore, when using ChatGPT, it is essential to put the used text into the right context.

  * Don’t use a result of ChatGPT that you don’t understand — This rule emphasises the importance of understanding the meaning and implications of a statement before using it. This rule can also be applied to users of ChatGPT to help ensure that messages being sent are clear and accurately reflect the intended meaning. If you come across a technical term or jargon in a message that you are not familiar with, avoid using it in your own message. Instead, take the time to research the meaning and ensure that you understand it fully before using it.

  * Don’t get into ’waffling’ and try to convince by the sheer amount of text generated by a machine — emphasises the importance of being clear and concise in communication, and avoiding the use of overly complex language or convoluted sentence structures. And also do not to blind with superficiality which can be easy generated by ChatGPT.

  * Do not assign ChatGPT any responsibility to you who has not explicitly accepted it in its terms and conditions — This rule emphasises the importance of understanding the terms and conditions of using ChatGPT. While this may be legally necessary, it also emphasises the importance of reading and understanding the terms and conditions of any platform or service before using it.

  * Ignore emotional language — Despite the human-like qualities, ChatGPT does not have emotions and feelings, but it can sometimes fake such emotions. Emotional language from ChatGPT could be ignored.

### Recommendations for regulators and policymakers

The recommendations for regulators and policymakers are:

  * Don’t over regulate — Finding the right balance between regulation and free use can be a challenging task for regulators. On the one hand, regulation can be necessary to protect individuals and ensure fair competition. On the other hand, excessive regulation can stifle innovation and limit the benefits of new technologies or services. To strike the right balance, regulators should consider a variety of factors, including the potential risks and benefits of the technology or service, the impact of regulation on users and businesses, and the potential for self-regulation or market-based solutions. In addition, regulators should engage with stakeholders, including users, businesses, and experts, to ensure that their approach is informed by a range of perspectives. Ultimately, the goal should be to create a regulatory environment that promotes innovation and growth while protecting the public interest.

  * Thou shalt not concentrate information and communication in one place — Concentrating information and communication in one place can create imbalances of power and increase the potential for abuse. When one entity has exclusive control over information and communication channels, they can use that power to manipulate or exploit others. This can occur in a variety of contexts, including social media platforms, news media organizations, and government agencies but also with ChatGPT in the future. To prevent these imbalances of power, it is important to distribute information and communication across multiple platforms and systems, promoting competition and diversity. This can help to ensure that no single entity has too much control or influence over the flow of information and communication, and that individuals and groups have the freedom to express themselves and access the information they need.

### Recommendations for Ethicists

The recommendations for ethicists are:

  * Understand ethics roles fully in innovative technologies — AI ethics concerns with the human moral behaviour as they design, construct, use and treat artificially intelligent beings, as well as concerns with the moral behaviour of AI agents [16, 17]. Ethicists need to fully understand the roles of ethics in ChatGPT in order to not only guide the ethical development of ChatGPT, but also provide guidance on the ethical use of ChatGPT more effectively.

  * Collaborate with experts closely from multiple disciplinaries — The conversation about ethics of ChatGPT is a philosophical discussion and needs to be elevated to a sufficiently high level from different fields. For example, legal or social experts are usually good at ethical issues related to data governance, but they may not have deep knowledge on how an AI model such as LLMs is built with a large number of parameters as AI experts be. Therefore, ethicists need to collaborate with experts that span the fields of AI, engineering, law, science, economics, ethics, philosophy, politics, and health as well as other.

## DISCUSSION

LLMs such as ChatGPT can be used to fulfil typical language tasks such as
summarising text paragraphs and writing news articles, answering difficult
questions, generating ideas, programming computer code, writing interesting
novels, as well as others [5]. However, it lacks a firm moral stance, and it
is suggested that users do not carelessly follow ChatGPT’s advice [18]. We
need to ensure individuals and organisations use the ChatGPT ethically,
legally, and responsibly.

There are no widely accepted guidelines and standards for the use of ChatGPT
yet. Chan [12] argued to first focus on the regulation of professional AI
developers and users by government regulatory agencies, and high-quality
curated datasets for less harmful language model outputs. There is also a need
to fill the gap between abstract ethical principles and practical applications
[17]. Furthermore, public education and digital literacy are important
measures to address potential intentional misuse for manipulation and
unintentional harm caused by bias from language models.

This paper proposed commandments for different stakeholders for the
responsible use of ChatGPT. For example, a known limitation of ChatGPT is that
it may provide answers to questions that are simply wrong. The fact-check of
answers is highly important for its responsible uses. Furthermore, a feature
to explain why a particular statement was made in ChatGPT will foster user
trust in responses of ChatGPT. Therefore, the development of fact-check and
justification of responses of ChatGPT is highly suggested for the responsible
use of ChatGPT. Developers must also develop systems to detect and mitigate
bias, ensure user privacy and security, differentiate responses from ChatGPT
and humans, and prevent misuse of the technology.

In summary, ChatGPT has been demonstrating extensive applications with human-
like writings. However, users have shown various ethical concerns such as
intentional misuse for manipulation and unintentional harm caused by bias,
because of the data it uses and opaque models as well as human-like responses.
There are still various challenges to address these ethical concerns. The
commandments presented in this paper will motivate stakeholders for the
ethical use of ChatGPT.

## CONCLUSION

ChatGPT has been becoming a widely discussed topic because of its powerfulness
in generating human-like text for various purposes since it was released in
November 2022. However, there are various ethical concerns associated with the
use of ChatGPT. This paper highlighted typical ethical concerns on ChatGPT
such as Bias, Privacy, and Abuse, and articulates key challenges when ChatGPT
is used in various applications. The proposed practical commandments for
different stakeholders of ChatGPT can serve as guidelines for those applying
ChatGPT in their applications. The future of this work will focus on the
development of guidelines and tools for fact-check and justification of
responses for the responsible use of ChatGPT.

## ACKNOWLEDGMENTS

The Acknowledgments is always plural even if there is a single acknowledgment.
The author(s) would like to thank A, B, and C. This work was supported by XYZ
under Grant ###.

## REFERENCES

  * [1] M. Liebrenz, R. Schleifer, A. Buadze, D. Bhugra, and A. Smith, “Generating scholarly content with chatgpt: ethical challenges for medical publishing,” _The Lancet Digital Health_ , 2023.
  * [2] Elon University News Bureau, “How ChatGPT is changing the way we use artificial intelligence,” https://www.elon.edu/u/news/2023/02/13/how-chatgpt-is-changing-the-way-we-use-artificial-intelligence/, February 2023, accessed: 2023-03-29.
  * [3] J. V. Pavlik, “Collaborating with chatgpt: Considering the implications of generative artificial intelligence for journalism and media education,” _Journalism & Mass Communication Educator_, p. 10776958221149577, 2023.
  * [4] OpenAI, “How should AI systems behave, and who should decide?” https://openai.com/blog/how-should-ai-systems-behave, February 2023, accessed: 2023-03-29.
  * [5] L. Floridi and M. Chiriatti, “Gpt-3: Its nature, scope, limits, and consequences,” _Minds and Machines_ , vol. 30, pp. 681–694, 2020.
  * [6] L. Tung, “ChatGPT can write code. now researchers say it’s good at fixing bugs, too,” https://www.zdnet.com/article/chatgpt-can-write-code-now-researchers-say-its-good-at-fixing-bugs-too/, March 2023, ZDNET,Accessed: 2023-03-29.
  * [7] C. Stokel-Walker, “Chatgpt listed as author on research papers: many scientists disapprove,” _Nature_ , January 2023.
  * [8] L. Weidinger, J. Mellor, M. Rauh, C. Griffin, J. Uesato, P.-S. Huang, M. Cheng, M. Glaese, B. Balle, A. Kasirzadeh _et al._ , “Ethical and social risks of harm from language models,” _arXiv preprint arXiv:2112.04359_ , 2021.
  * [9] J. Zhou, F. Chen, A. Berry, M. Reed, S. Zhang, and S. Savage, “A survey on ethical principles of ai and implementations,” in _2020 IEEE Symposium Series on Computational Intelligence (SSCI)_. IEEE, 2020, pp. 3010–3017.
  * [10] C. Zhang, C. Zhang, C. Li, Y. Qiao, S. Zheng, S. K. Dam, M. Zhang, J. U. Kim, S. T. Kim, J. Choi, G.-M. Park, S.-H. Bae, L.-H. Lee, P. Hui, I. S. Kweon, and C. S. Hong, “One Small Step for Generative AI, One Giant Leap for AGI: A Complete Survey on ChatGPT in AIGC Era,” Apr. 2023, arXiv:2304.06488 [cs]. [Online]. Available: http://arxiv.org/abs/2304.06488
  * [11] E. M. Bender, T. Gebru, A. McMillan-Major, and S. Shmitchell, “On the dangers of stochastic parrots: Can language models be too big?” in _Proceedings of the 2021 ACM conference on fairness, accountability, and transparency_ , 2021, pp. 610–623.
  * [12] A. Chan, “Gpt-3 and instructgpt: technological dystopianism, utopianism, and “contextual” perspectives in ai ethics and industry,” _AI and Ethics_ , vol. 3, pp. 53–64, 2023.
  * [13] S. Krügel, A. Ostermaier, and M. Uhl, “The moral authority of ChatGPT,” _arxiv: 2301.07098 [cs]_ , January 2023, arXiv:2301.07098. [Online]. Available: http://arxiv.org/abs/2301.07098
  * [14] StackOverflow, “Temporary policy: Chatgpt is banned,” https://meta.stackoverflow.com/questions/421831/temporary-policy-chatgpt-is-banned/, December 2022, accessed: 2023-4-4.
  * [15] H. Mueller, M. T. Mayrhofer, E.-B. V. Veen, and A. Holzinger, “The ten commandments of ethical medical ai,” _IEEE COMPUTER_ , vol. 54, no. 7, pp. 119–123, 2021.
  * [16] A. Jobin, M. Ienca, and E. Vayena, “The global landscape of ai ethics guidelines,” _Nature Machine Intelligence_ , vol. 1, no. 9, pp. 389–399, 2019.
  * [17] J. Zhou and F. Chen, “Ai ethics: from principles to practice,” _AI & SOCIETY_, pp. 1–11, 2022.
  * [18] S. Krügel, A. Ostermaier, and M. Uhl, “The moral authority of chatgpt,” _arXiv preprint arXiv:2301.07098_ , 2023.


# In-context Learning as Maintaining Coherency: A Study of On-the-fly Machine
Translation Using Large Language Models

Suzanna Sia
Johns Hopkins University
<EMAIL_ADDRESS>
&Kevin Duh
Johns Hopkins University
<EMAIL_ADDRESS>

###### Abstract

The phenomena of in-context learning has typically been thought of as
"learning from examples". In this work which focuses on Machine Translation,
we present a perspective of in-context learning as the desired generation task
maintaining coherency with its context, i.e., the prompt examples. We first
investigate randomly sampled prompts across 4 domains, and find that
translation performance improves when shown in-domain prompts. Next, we
investigate coherency for the in-domain setting, which uses prompt examples
from a moving window. We study this with respect to other factors that have
previously been identified in the literature such as length, surface
similarity and sentence embedding similarity. Our results across 3 models
(GPTNeo2.7B, Bloom3B, XGLM2.9B), and three translation directions
(en$\rightarrow${pt, de, fr}) suggest that the long-term coherency of the
prompts and the test sentence is a good indicator of downstream translation
performance. In doing so, we demonstrate the efficacy of In-context Machine
Translation for on-the-fly adaptation.

##  1 Introduction

In-context Machine Translation is a relatively new paradigm that uses large
autoregressive Language Models to carry out the task of Machine Translation
(MT) by being shown translation pairs in the prefix. From a practitioner’s
viewpoint, In-context learning presents itself as an attractive approach for
rapidly adapting a Translation model on-the-fly. Previous strategies for
adapting a pre-trained MT model still require additional engineering or
training of the model, e.g fine-tuning with in-domain data using adaptor
layers philip-etal-2020-monolingual. Instead, simply changing the inputs to
the model might be an effective way to adapt on-the-fly without any model
modification.

The in-context learning paradigm describes a phenomena where large
autoregressive language models perform a task when shown examples (known as
prompts) in the prefix brown2020language; bommasani2021opportunities. Previous
work approaches the role of the prompt context as allowing the model to "learn
by examples". This intuitive approach to formulating the task of prompt
selection has led to the suggestion of selecting examples that are similar to
the source sentence being translated. Semantic similarity based on sentence
embeddings liu-etal-2022-makes and BM25 have been proposed to select examples
to present as “demonstrations" rubin2021learning. This approach was further
expanded by agrawal2022context who show that BM25 and a heuristic version
optimizing for word coverage, is effective for selecting examples.

We focus on Machine Translation as a complex conditional generation task and
offer an alternate perspective: the in-context paradigm depends on maintaining
coherency. Coherence is an aspect of natural language that reflects the
overall semantic and syntatic consistency in a body of text
flowerdew2009lexical. We investigate this by first exploring the model’s
behavior when showing matching and mismatching domains in the context and the
test sentence. Next we consider a stricter notion of coherency using a moving
window of previous gold translations directly preceding the test source
sentence to be next translated. Our experiments compare the coherence factor
with similarity based factors for prompt selection, additionally controlling
for length xie2021explanation which is typically overlooked but is important
to consider for performance and available labeling (translation) budget. The
contributions of this work are

Translate English to French. |  |
---|---|---
English: A discomfort which lasts .. | French: | Un malaise qui dure
English: HTML is a language for formatting | French: | HTML est un langage de formatage
… |  | …
English: After you become comfortable with formatting .. | French: |
Table 1: A single continuous input sequence presented to the model for
decoding a single test source sentence “After you become comfortable with
formatting..”. Given the entire sequence as input, the model proceeds to
generate the target sequence.

  * •

We identify coherency of prompt examples with respect to test sentence as a
critical factor for translation performance. Experiments across 3 models
(GPTNeo2.7B, Bloom3B, XGLM2.9B) and 4 domains (Medical, Social Media,
Wikipedia, and TED Talks) suggest that models perform better when prompts are
randomly drawn from the same domain.

  * •

Within the TED talks domain, we investigate local coherence using document-
level translation experiments, by adopting a moving window directly preceding
the test source sentence to be translated. Overall, our results across the 3
models and three translation directions (en$\rightarrow${pt, de, fr}) suggest
that the coherency of the prompts with regard to the test sentence is a good
indicator of translation performance.

##  2 Preliminaries

###  2.1 In-context Machine Translation

In an in-context learning setup, several formatting decisions need to be made
on how to present the prompt examples to the model. We adopt the following
commonly used prompt format where the instructions are straightforwardly
provided as in the following (Table 1).111We also experiment with a different
separator ”=” used in lin2021few (instead of “English” and “French”), but find
that this does not perform significantly better. In this work, we consider
both sentence level translation (subsection 5.1) and an on-the-fly document-
level setting (subsection 5.3).

###  2.2 Coherence in Natural Language Text

The computational linguistics literature holds many competing definitions of
coherence in text wang2014short. We consider two aspects of coherence, first
from a more global level where we investigate domain effects, and also from a
local sentence level, where we consider a coherent context as a moving window
of previous (gold) translations which directly precede a test sentence. A
similar working definition of coherence has been used in discrimination tasks
that require a model to identifying the right order of (shuffled) sentences
elsner2007unified; barzilay2008modeling; laban2021can.

##  3 Factors which affect In-context MT

We outline several factors studied in this paper related to example selection
for In-context MT. While we emphasise the notion of Coherence (subsection
2.2), by studying the domain factor (subsection 3.4) and local coherence
(subsection 3.5), our experiments seek to compare this against other factors
that have been highlighted in previous literature. Namely, length (subsection
3.1), surface similarity (subsection 3.2) and semantic similarity (subsection
3.3). To demonstrate, in Table 1, the first sentence is semantically similar
and the second sentence has surface similarity with the test sentence.

Figure 1: Factors identified and studied in this paper. Selecting from
matching Domain increases coherence (Appendix C) and each domain has different
length distributions (subsection 5.2). Surface similarity and embedding
similarity are associated (Table 4). Surface similarity selection also results
in longer sentences (subsection 5.4) Rectangle boxes next to the node are
measures of these factors. We describe and quantify positive and negative
interference of the model for translation performance in subsection 6.3.

###  3.1 Length (Translation Budget)

One previously overlooked factor for in-context MT is the length (number of
words) in the prompt examples. The perspective of In-context Learning as
implicit Bayesian Inference argues that longer examples provide more evidence
to the model on the desired task pattern xie2021explanation. Longer examples
are also more likely to contain non-trivial translation exemplars, although it
is not clear whether this affects downstream performance. We find example
length to be correlated with the domain (Figure 2), and it may thus be a
confounding factor for in-context MT.

##### Controlling for Length

We adopt the notion of a “Translation Budget" which is the total word count of
all the prompt examples provided (excluding the test sentence). Examples can
be selected as long as they satisfy the budget constraint. A generalized
algorithm is provided in subsection 4.3. From a resource perspective, this
reflects the work of the human annotator in providing example translations.

###  3.2 Surface Similarity

####  3.2.1 BM25

BM25 robertson2009probabilistic is a bag-of-words unsupervised retrieval
function that ranks a set of documents based on the query terms appearing in
the documents. agrawal2022context report that using BM25 to retrieve similar
prompt examples outperforms random selection. They also advocate for a variant
of BM25 with increased coverage of test sentence source words although with
marginal gains (<1 BLEU point) increase. Following agrawal2022context, we
order the examples according to their similarity to the source, with the most
similar examples on the left in all our experiments.

####  3.2.2 Maximising Surface Similarity Coverage

To maximise word overlap across all prompts and the source sentence, we adopt
Submodular optimisation by Maximal Marginal Relevance carbonell1998use;
lin2010multi. Formally we are given a finite size set of objects $U$ (the size
of the prompt bank). A valuation function $f:2^{U}\rightarrow\mathcal{R}_{+}$
returns a non-negative real value for any subset $X\subset U$. The function
$f$ is said to be submodular if it satisfies the property of “diminishing
returns", namely, for all $X\subset Z$ and $Z\notin U$, we have $f(X\cup
u)-f(X)\geq f(Z\cup u)-f(Z)$. The algorithm optimises for sentences with
maximal word overlap weighted by the BM25 score.

###  3.3 Semantic Similarity (Nearest Neighbors)

The semantic similarity of prompts based on their sentence embeddings has also
been advocated for selecting good in-context examples. liu2021makes apply a
pre-trained Roberta-large sentence encoder to the test sentence, and query for
its nearest neighbors to use as in-context demonstrations. In our experiments
we apply a similar strategy using MPNet base song2020mpnet which achieved
highest scores on HuggingFace sentence embedding and semantic search
benchmarks.222https://www.sbert.net/docs/pretrained_models.html We do not
consider training a prompt retriever rubin2021learning or fine-tuning the
sentence encoder liu2021makes in this study, as these are no longer "light-
weight" retrieval methods that are comparable with the other unsupervised
strategies.

###  3.4 Domain Coherence

GPT is able to do style transfer just from instructions or from being shown
surface prompt examples reif-etal-2022-recipe. Simply providing demonstrations
from the same domain may induce the large language model (LLM) to generate a
similar style which is coherent with the target text. Another possibility is
that particular lexical translation exemplars which match the source sentence
may be present. However, due to the very high dimensionality of the raw
vocabulary, this is less likely if translation examples are randomly sampled.

Domain may also present spurious correlations which are confounded by the
training data of LLMs. For instance, there may be certain domains which are
better at eliciting Translation behavior from the model, regardless of what
the test domain is.

###  3.5 Local Coherence (Moving Window)

We hypothesise that the local coherence (subsection 2.2) of the context to the
test sentence to be translated may be an important factor for performance. To
test this, we adopt a moving context window of the previously translated gold
sentence pairs as the prompt examples. To our knowledge, subsection 3.4 and
subsection 3.5 are previously unexplored for In-context Machine Translation.

##  4 Experiments

###  4.1 Data

##### Domain Coherence

We organise our experiments investigating four en$\rightarrow$fr domains,
WMT19 Biomedical (MED) (bawden-etal-2019-findings), a social media dataset,
MTNT (michel-neubig-2018-mtnt), multilingual TED Talks, and Wikipedia-based
FLORES flores. Except for MED, all other datasets have a wide range of topics
in the train (prompt bank) and test set which are shuffled in random sampling,
and thus the domain experiments are more focused on the writing style of the
text. We use standard train-test splits, with the trainset being used as the
prompt bank. Scores are reported using SacreBLEU (post-2018-call).333nrefs:1
$\mid$ case:lower $\mid$ eff:no $\mid$ tok:13a $\mid$ smooth:exp $\mid$
version:2.0.0

##### Local Coherence (document level)

We use the Multitarget TED Talks dataset from duh18multitarget. The original
dataset has 30 documents in the test set, where each document corresponds to a
10-20 minute TED talk. To increase the size of the test set, we partition the
"original" trainset into a train (prompt bank) and test split, where talks
with a minimum of 100 lines were used as the test and talks with less than 100
lines were used as the "out-of-document" prompt bank. We used 120 test
documents that had a minimum of 100 lines, and we evaluated each up to 120
lines, where each TED talk is a document. The document level BLEU scores are
reported for three language directions en$\rightarrow${fr, pt, de}. We do not
use a dev set as there is no training or any tuning of any hyperparameters.
Since this is a non-standardised data split, we provide the numbers in the
following table.

|  Talks (Docs) |  Lines per doc |  Total Lines
---|---|---|---
"Outside-doc" Prompt Bank |  450 |  <100 |  26000+
"Within-doc" Prompt Bank |  1 |  100-120 |  120
Test |  120 |  100-120 |  12000+

###  4.2 Models

We use three models, GPTNeo2.7B (gpt-neo), XGLM2.9B lin2021few, and Bloom3B
scao2022bloom which are open access LLMs available on HuggingFace wolf-
etal-2020-transformers. The later two have been advertised as "Multilingual
Language Models". GPTNeo2.7B is a GPT3 replicate pretrained on The Pile
gao2020pile, while XGLM adopts a similar architecture trained on a
multilingual corpus (CC100-XL). Bloom3B has been trained on the ROOTS Corpus
laurenccon2022bigscience, a collection of huggingface datasets of 1.6 TB of
text. To our knowledge, there has not been any reports of sentence level
parallel corpora in the training datasets of these models.

| GPTNeo2.7B | Bloom3B | XGLM2.9B
---|---|---|---
Prompt / Test | FLORES | MED | MTNT | TED | FLORES | MED | MTNT | TED | FLORES | MED | MTNT | TED
FLORES | 24.6 | 19.7 | 23.1 | 24.6 | 36.7 | 28.5 | 28.5 | 31.1 | 29.3 | 20.9 | 24.7 | 25.7
MED | 23.0 | 19.2 | 21.1 | 23.2 | 34.5 | 28.7 | 26.2 | 29.5 | 27.5 | 21.4 | 22.9 | 24.4
MTNT | 23.7 | 18.6 | 22.4 | 23.7 | 35.5 | 27.7 | 29.1 | 30.6 | 27.9 | 21.2 | 25.0 | 25.4
TED | 23.2 | 18.6 | 22.1 | 23.6 | 36.1 | 27.9 | 29.1 | 31.2 | 27.8 | 21.1 | 24.2 | 24.8

Table 2: Crosstable of BLEU scores from sampling and testing in different
domains. We present the average BLEU scores across 5 randomly sampled prompt
sets. The size of the prompt sets (number of translation pair examples) is 5.
We bold the largest value column-wise.

###  4.3 Algorithm for Greedy selection with Length Constraint

In our experiments, we investigate BM25 (subsubsection 3.2.1), BM25 with
submodular optimisation (BM25-s; subsubsection 3.2.2), and semantic similarity
(nn; subsection 3.3). To control for length effects, we employ an algorithm
for selection with length constraints (algorithm 1) which closely follows
greedy submodular algorithms krause2008beyond. Retrieval methods adopts a
utility function: $f$, which is used to retrieve highest scoring sentences.
For BM25 and BM25-s, $f$is BM25, while $u_{i}$ is selected by $f(\\{u\\})$,
and $f(\\{u\\}|X_{i})$ respectively. While for nn, $f$ is the L2 embedding
similarity between prompt sentence and test query.

1 Input: (Submodular) function $f:2^{U}\rightarrow\mathbb{R}_{+}$, cost
function $m$, budget $b$, finite prompt bank $U$

2 Output: $X_{k}$ where $k$ is the number of iterations/prompts.

3 Set $X_{0}\leftarrow$; $i\leftarrow 0$;

4 while  _$m(X_{i}) <b$_ do

5 $u_{i}=\textrm{argmax}_{u\in U\backslash X_{i}}f(\\{u\\}\mid X_{i})$

6 $X_{i+1}\leftarrow X_{i}\cup{u_{i}}$;

7 $i\leftarrow i+1$

Algorithm 1 Generalised greedy (submodular) algorithm with length budget

##  5 Analysis of Factors

###  5.1 Domain Coherence [Table 2]

Does coherence of domain allow models to adapt on the fly? If models are
adapting to the domain shown in the context, sampling and testing within the
same domain should result in the highest translation performance, as compared
to being shown examples out of domain. For example, if we are testing on the
TED domain, is it important that the prompt be also drawn from TED or is it
sufficient to have sentence pairs from any domain illustrating the translation
task? To account for prompt selection and ordering effects, all inference runs
were repeated with 5 randomly sampled prompt sets from the training data. We
focus on en $\rightarrow$ fr which is common across datasets.

##### Results and Discussion

  * •

Models are able to perform some form of domain adaptation on-the-fly. There
appears to be evidence of domain adaptation in Bloom3B and XGLM, as sampling
and testing within the same domain (e.g., sample from MED test with MED)
mostly results in the highest performance column-wise. We also observe that
matching domains result in lower conditional source sentence perplexity
(Appendix C).

  * •

For GPTNeo, sampling from FLORES results in the best translation performance
across all test sentences even with domain mismatch. This suggests that
translation performance in GPTNeo is best induced using FLORES and is less
adaptive to the domain. Note that the second best column wise result for
GPTNeo tends to occur when there is matching prompt and test domain.

###  5.2 Domain controlling for Length

How does length of prompts affect translation across different domains? In
Figure 2, we randomly sample 1000 sentences from each domain’s training set.
Randomly sampled sentences from different domains show distinct length
effects. We study the impact of these length effects by selecting either a
5-10 word or 15-20 word long sentences for translation examples, and compare
the differences in scores for the non-filtered scenario (Table 3).

Figure 2: Histograms of sentence lengths (word counts) randomly sampled from
different domains, which has implications for the total prompt length when
sampling from these domains. FLORES sentences tend to be nearly twice as long
as MTNT and TED sentences.

Prompt / Test | FLORES | MED | MTNT | TED
---|---|---|---|---
FLORES | - | - | - | -
MED |  $\Downarrow$22.4 (0.3) |  $\Downarrow$18.5 (0.3) |  $\downarrow$20.8 (0.8) |  $\Downarrow$22.5 (0.8)
MTNT |  $\Downarrow$23.2 (0.4) |  $\downarrow$18.3 (0.5) |  $\Downarrow$21.9 (1.2) |  $\downarrow$23.5 (0.5)
TED |  $\Downarrow$21.7 (1.4) |  $\Downarrow$17.6 (0.6) |  $\Downarrow$20.1 (1.8) |  $\Downarrow$22.3 (1.5)
5-10 words long sentences; GPTNeo 2.7B

Prompt / Test | FLORES | MED | MTNT | TED
---|---|---|---|---
FLORES | 24.2 (0.2)$\downarrow$ | 19.6 (0.3) | 22.7 (0.8)$\downarrow$ | 24.3 (0.5)$\downarrow$
MED | 22.9 (0.6) | 19.3 (0.1) | 21.1 (0.9) | 22.8 (0.7) $\downarrow$
MTNT | 24.0 (0.4) $\uparrow$ | 18.9 (0.6)$\uparrow$ | 22.5 (0.0) | 24.3 (0.3)$\Uparrow$
TED | 23.8 (0.4)$\Uparrow$ | 19.0 (0.4)$\uparrow$ | 22.9 (0.2)$\Uparrow$ | 23.8 (0.4)
15-20 words long sentences; GPTNeo 2.7B

Table 3: Selecting for short source sentences (5-10 words) vs longer source
sentences (15-20 words) as translation examples. $\downarrow$and
$\uparrow$refers to differences $>0.3$, and $\Downarrow$and $\Uparrow$refers
to differences $>0.5$ when compared to the no-length filter scenario in Table
2.

##### Results and Discussion

  * •

When source prompt sentences are 5-10 words, all BLEU scores decrease. For
15-20 words sentences which is "long" for MTNT and TED, but "short" for
FLORES, the BLEU score of the former increases while the latter decreases.
BLEU scores are similar for MED as 15-20 words is close to the mean of MED
length distribution.

  * •

We inspect the length of generation under different prompt lengths, and find
that average differences in generation length are marginal (only 1-2 words
difference) indicating that poorer performance is not simply due to a
difference in generation lengths.

|  | GPTNeo2.7B(BLEU) | Bloom3B(BLEU) | XGLM2.9B(BLEU) | L2 | coverage
---|---|---|---|---|---|---
| In/outdoc | en$\rightarrow$fr | en-pt | en-de | en-fr | en-pt | en-de | en-fr | en-pt | en-de | - | -
random | out | 26.3 | 27.1 | 16.6 | 35.2 | 35.5 | 7.9 | 27.1 | 26.7 | 18.9 | 1.35 | 0.31
nn | out | 26.8 | 26.9 | 16.9 | 35.1 | 35.1 | 8.2 | 25.6 | 26.6 | 18.3 | 0.98 | 0.49
bm25 | out | 27.1 | 27.4 | 17.3 | 35.1 | 35.3 | 9.4 | 25.3 | 27.0 | 18.4 | 1.21 | 0.75
bm25-s | out | 27.2 | 27.5 | 17.4 | 34.8 | 34.9 | 9.1 | 25.6 | 27.4 | 18.7 | 1.25 | 0.80
random | within | 27.4 | 27.3 | 17.3 | 35.9 | 35.8 | 7.8 | 26.6 | 28.8 | 19.6 | 1.28 | 0.34
window | within | 28.1 | 28.3 | 17.9 | 36.9 | 37.0 | 8.8 | 28.6 | 31.6 | 21.2 | 1.22 | 0.40

Table 4: BLEU score comparison of similarity-based retrieval methods from out
of document, and moving window (window) from within the document. Coverage
(Rouge1-precision) refers to the word overlap between prompt source sentences
and test source sentence. L2 refers to the average L2 Euclidean distance
between source prompt sentence embeddings and the test sentence embedding.

###  5.3 Local Coherence [Table 4]

How important is a coherent context (as compared to other prompt selection
methods?) subsection 5.1 showed that models are able to adapt when shown
prompts from a matching domain. We hypothesise that coherence of the prompts
with respect to the test source sentence (subsection 2.2) is an important
factor for performance.

We use the TED talks dataset (data preparation described in subsection 4.1),
and consider a moving window of previous gold translations (window) as a
coherent context for the model.444Preliminary experiments using model
generated instead of gold translations performed worse than random. We compare
this against the baselines of (BM25; subsubsection 3.2.1), (BM25-s;
subsubsection 3.2.2), and Nearest Neighbor retrieval of sentence embeddings
(nn; subsection 3.3) from a large prompt bank outside the document. We use a
prompt set of 5 examples for all experiments, and randomly sample from outside
of the document if the available window is smaller than 5. Document level BLEU
scores are averaged across 120 documents and reported in Table 4.

##### Quantifying Similarity

We report the ROUGE1-precision (coverage; lin2004rouge) and the L2 Euclidean
distance (L2) of the source sentences in the prompt set, with the test source
sentence to be translated. If translation performance is due to word overlap
or embedding similarity, then we expect that having a higher coverage or lower
L2 would have better performance than window. Note that all similarity based
retrieval methods depend only on the source sentences, and is model and target
language independent. i.e., the single coverage and L2 value applies for all
results columns in Table 4.

##### Results and Discussion

  * •

The moving window (window) outperforms all other baselines across the 3 models
and 3 language directions, with the exception of Bloom3B on en$\rightarrow$de
direction. The gains are from 0.5 to 2.6 BLEU points from the next best
performing retrieval method. Importantly, coverage and L2 shows that the
performance is not due to similarity or word overlap.

  * •

Interestingly, randomly sampling sentences from within the document (talk)
performs well compared to other similarity based retrieval methods from
outside of the document. This further highlights that coherence is a critical
factor for In-context Machine Translation.

  * •

Similarity based retrieval mostly does better than randomly sampled prompt
sets, which is consistent with existing literature which did not consider the
factor of coherence. A notable exception is XGLM en$\rightarrow$fr results,
where similarity based methods are doing poorly compared to that reported by
agrawal2022context. We find that the similarity based retrieval methods does
better for XGLM when the number of prompts is increased from 5 to 15. The same
trend is observed at 15 prompts, window continues to outperform the other
methods (results in Appendix B).

Crucially, this set of experiments show that similarity based methods are not
as critical for translation as compared to coherency, a new factor that we
identify in this work.

###  5.4 Similarity based Retrieval within the Document

How well do similarity based retrieval methods perform for previous on-the-fly
translations? In subsection 5.3, we established that using a moving window
(local coherence) outperforms retrieval from outside the document with
similarity-based retrieval methods. Here we apply bm25, bm25-s, nn for
retrieval within the document. We consider the more realistic "on-the-fly" or
computer-aided translation scenario, where the human translator works with MT
systems, and translation examples in the document can only be selected prior
to the test sentence alabau2014casmacat.

##### Controlling for Length

When doing retrieval based methods within the document for an "on-the-fly"
setting, length factors in and longer sentences are retrieved on average. We
thus investigate budgeting for the length constraint to be same as the moving
window (window). For every test sentence, we compute the budget used by it’s
own moving window, and apply it as a length constraint to for the other
retrieval based methods as described in subsection 4.3. Results are presented
in Figure 3.

Figure 3: Comparison of Retrieval methods controling for budget: No budget or
same budget as moving window. Model is GPTNeo2.7B on en->fr. random is sampled
within the document.

##### Results and Discussion

  * •

We observe similar performance for all retrieval methods, with bm25-s doing
slightly better than bm25 and nearest neighbors (nn).

  * •

Without any budget restriction, performance of retrieval methods outperforms
window. However when restricted to the same budget as window, we find that the
performance is within 0.1-0.5 BLEU score difference. Furthermore, the coverage
is only 0.01-0.03 less if not using similarity based retrieval, indicating
that most of the differences in contributions could be coming from the length
effect and not because of similarity.

##  6 Further Analysis and Discussion

In this section, we focus on GPTNeo2.7B and in the en$\rightarrow$fr
direction.

###  6.1 Perplexity and Coverage

One natural question that arises is the relationship between Coverage,
Coherence, and translation performance. Although there is no widely accepted
measure of general coherence, we can formulate this with respect to the
particular model being studied. We consider the model’s conditional perplexity
of the test sentence given the context. Perplexity is a widely used measure of
suprisal in text and has also been used as a measure in topic coherence
newman2010automatic. Concurrent work by gonen2022demystifying argue that total
perplexity of the input sequence is related to In-context performance.

In Figure 4, we produce scatterplots of Sentence BLEU scores, source
perplexity and Coverage (word overlap). We observe that there is a negative
relationship between source perplexity and Sentence BLEU (-0.22 Pearson’s r),
but very noisy relationship between Sentence BLEU and word overlap, and word
overlap and source perplexity.

Figure 4: Scatterplots of Sentence BLEU Scores, with Source Perplexity and
Word Overlap

###  6.2 Studying Local Coherence [Table 5]

We compare the window with other baselines which may give some indication of
what is important in the document in terms of local coherence.

  * •

Shuffle simulates whether the model is affected by the the local coherence by
shuffling sentences within window.

  * •

Static refers to the first $k$ (window size) translation sentences of the
document which is then held fix throughout when translating the rest of the
document.

retrieval | bleu | L2 | Coverage | ppl_s
---|---|---|---|---
static | 26.6 | 1.22 | 0.41 | 16.8
random | 27.4 | 1.28 | 0.31 | 14.9
window | 28.1 | 1.22 | 0.40 | 11.1
shuffle | 28.3 | 1.22 | 0.40 | 12.0

Table 5: Ordering effects within document. All retrieval methods are within
documnent.

Interestingly, shuffling the set of prompts within the moving window which
breaks the natural ordering of the document "coherence" does not deteriorate
in-context translation performance. This finding is consistent across several
models and languages Appendix D. The ordering of the document does affect
source perplexity, with perplexity increasing from 11.1 $\rightarrow$ to 12.0,
however this does not negatively affect translation performance. This suggests
that the relationship between coherence and translation is indirect or non-
linear, and the way models use context might be counter-intuitive; a view
increasingly advocated by recent research webson2021prompt; min2022rethinking.
Overall this suggests we may benefit from methods which perform selection from
within the document which we leave to future work.

###  6.3 Do we need Translation examples at all?

Given the rise of instruction-following GPT ouyang2022training a reasonable
question is whether prompt example selection will still be relevant in future
models. For a large language model, merely providing the instruction
"Translate English to French" without any prompt examples (zero-shot) can
still elicit a translation. In spite of zero-shot success, a common finding
(for MT as well as other NLP tasks) is that providing more prompt examples
typically results in better performance albeit with diminishing effect. Since
examples are not strictly necessary for translation but can enhance the
model’s downstream translation ability, what is the role of prompt examples?

| random | bm25 | bm25-s | nn | window
---|---|---|---|---|---
Positive | 0.56 | 0.62 | 0.61 | 0.6 | 0.62
Negative | 0.32 | 0.31 | 0.31 | 0.32 | 0.29
No Change | 0.12 | 0.07 | 0.08 | 0.08 | 0.09

Table 6: Positive, Negative and No change (proportions) in BLEU scores across
different prompt selection methods. For positive row, higher is better. For
negative row, lower is better.

##### Positive vs Negative Task Interference

One curiosity that we observe across all of our experiments, is that prompt
sets do better on-average rather than across all examples, relative to the
Zero-shot, instructions only setting. This suggests a notion of interference;
examples may guide generation towards a poorer translation (negative
interference) or better translation (positive interference). A closely related
concept is task location reynolds2021prompt.

Table 6 quantifies this across different methods corresponding to the results
in Table 4 for GPTNeo2.7B en$\rightarrow$ fr direction. window has both the
highest positive interference and lowest negative interference. From Figure 5,
the major role of prompting methods compared to the zero-shot scenario is to
have greater positive interference chiefly over sentence BLEU of 20-60 and for
some extreme cases of 100 BLEU, although a large proportion of sentences still
lie in the low-scoring region.

Figure 5: Histograms of sentence BLEU scores for zeroshot (no prompts),
random, and window.

##  7 Conclusion

In-context Learning has typically been thought of as learning from examples.
In this work, we introduce a different perspective of coherency of the context
with the test sentence. We first showed that models are mostly able to adapt
to different writing styles when the prompt bank and test set are
matching/consistent in domain. Experiments across 3 models and 3 languages
show that a moving window is up to 2.6 BLEU points better than previously
reported similarity based retrieval methods from outside the document. From
this perspective, the problem of prompt selection for in-context MT is one of
maintaining a coherency for text generation. Preliminary analysis on local
coherence effects, and the presence of negative interference compared to the
zero-shot setting, suggests avenues for future work on investigating more
careful mechanisms for controlling in-context Machine Translation.

##  8 Limitations

This section details several limitations and ethical concerns associated with
this work.

  * •

While we have identified coherency of domain and document as a factor for in-
context MT, we expect there should be other factors that could be more
predictive of downstream performance, such as activation of attention patterns
from source to target sentence during generation.

  * •

We studied GPTNeo, Bloom and XGLM which have different training data but
similar sizes. Due to GPU memory limitations we did not study larger models
and it is not clear whether findings generalise to even larger models.

  * •

Although the TED talks dataset is a good overall testbed because it covers
many topics and combines formal language and informal text, we did not
quantify whether coherence is more likely to affect formal or informal
language and might be studied with other datasets.

  * •

This paper focuses heavily on MT as a complex generative task to study
coherence of context, and it is not immediately clear whether the findings
would also generalise to other longer-context generation tasks such as
document summarization or how this would affect simple classification.

##  9 Ethical Concerns

Large LMs are known to hallucinate text content, potentially produce toxic
speech or misinformation. While we did not observe this frequently in our
experiments, we did not quantify the extent of this across various methods.

##  Appendix A Resources

##### Software

  * •

Implementation of Nearest Neighbor Retrieval with FAISS library
johnson2019billion

  * •

Huggingface library was used for LLMs, model weights and calculation of
perplexity.

##### Hardware

  * •

All experiments can be run with a single NVIDIA-TITAN RTX GPU (24GB).

##### Datasets

All datasets for the experiments are open-source.

##  Appendix B Local Coherence (nprompts=15)

Ablation experiments for subsection 5.3 prompt set size of 15 shown in Table
7. The same trend is observed for 5 and 15 prompts.

|  | GPTNeo2.7B(BLEU) | Bloom3B(BLEU) | XGLM2.9B(BLEU)
---|---|---|---|---
| In/outdoc | en-fr | en-pt | en-de | en-fr | en-pt | en-de | en-fr | en-pt | en-de
random | out | 27.2 | 27.3 | 16.9 | 35.3 | 35.4 | 8.0 | 29.2 | 31.2 | 20.7
nn | out | 27.3 | 28.2 | 17.1 | 35.6 | 35.9 | 9.1 | 30.0 | 32.0 | 21.6
bm25 | out | 27.9 | 29.0 | 17.4 | 36.1 | 36.4 | 10.8 | 31.2 | 33.0 | 22.2
bm25-s | out | 27.7 | 29.1 | 17.3 | 35.2 | 36.0 | 9.1 | 29.8 | 32.0 | 21.6
random | within | 28.1 | 29.2 | 17.6 | 36.8 | 37.3 | 8.9 | 30.9 | 33.3 | 22.3
window | within | 28.9 | 29.8 | 18.2 | 37.8 | 38.1 | 9.6 | 31.7 | 34.4 | 23.0

Table 7: BLEU score comparison of similarity-based retrieval methods from out
of document, and moving window (window) from within the document. 15 prompt
examples used.

##  Appendix C Domain vs Perplexity

We report the perplexity of the source sentences when randomly sampling and
testing from different domains. Although there is no widely accepted measure
of general coherence, we can formulate this with respect to the particular
model being studied. We consider the model’s conditional perplexity of the
test sentence given the context. Perplexity is a widely used measure of
suprisal in text and has also been used as a measure in topic coherence
(reference). Concurrent work by gonen2022demystifying argue that total
perplexity of the input sequence is related to In-context performance. We
report the conditional perplexity from sampling and testing in different
domains for GPTNeo2.7B in Table 8 and Bloom3B in Table 9. We did not report
XGLM2.9B because the model log likelihood is very poorly calibrated.

Prompt / Test | FLORES | MED | MTNT | TED
---|---|---|---|---
FLORES | 21.3 | 24.5 | 54.9 | 25.5
MED | 24.1 | 16.2 | 62.4 | 27.0
MTNT | 25.4 | 26.9 | 40.8 | 23.3
TED | 24.0 | 24.9 | 52.2 | 19.6
Table 8: Source sentence perplexity conditioned on prompts randomly sampled from the domain computed with GPTNeo2.7B. Lower perplexity indicates greater coherence. Prompt / Test | FLORES | MED | MTNT | TED
---|---|---|---|---
FLORES | 21.5 | 23.4 | 61.8 | 28.5
MED | 25.1 | 16.3 | 74.8 | 33.4
MTNT | 25.2 | 25.7 | 47.2 | 26.1
TED | 24.1 | 23.5 | 60.1 | 22.1
Table 9: Source sentence perplexity conditioned on prompts randomly sampled
from the domain computed with Bloom3B. Lower perplexity indicates greater
coherence.

##  Appendix D Local Coherence Shuffle Effects

BLEU scores for comparing window with other baselines, accompanying appendix
section to subsection 6.2 which reports BLEU scores for GPTNeo2.7B
en$\rightarrow$fr. we find that results generalise across several models and
languages that we further investigated.

|  GPTNeo (en-pt) |  GPTNeo (en-de) |  XGLM (en-fr) |  Bloom (en-fr)
---|---|---|---|---
static |  27.1 |  16.8 |  27.7 |  34.9
random |  27.3 |  17.3 |  26.6 |  35.9
window |  28.3 |  17.9 |  28.5 |  36.9
shuffle |  28.5 |  17.9 |  28.7 |  36.9

Table 10: BLEU scores for different ordering effects within document. All
retrieval methods are within document.