# MBTI Personality Prediction for Fictional Characters Using Movie Scripts

Yisi Sang1 Xiangyang Mou2∗ Mo Yu3∗ Dakuo Wang4 Jing Li5 Jeffrey Stanton1
1Syracuse University 2Rensselaer Polytechnic Institute 3Pattern Recognition
Center, WeChat AI
4IBM Research, Northeastern University 5New Jersey Institute of Technology
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>Authors contributed equally
to this paper. Mo Yu is the corresponding author.

###### Abstract

An NLP model that understands stories should be able to understand the
characters in them. To support the development of neural models for this
purpose, we construct a benchmark, Story2Personality. The task is to predict a
movie character’s MBTI or Big 5 personality types based on the narratives of
the character. Experiments show that our task is challenging for the existing
text classification models, as none is able to largely outperform random
guesses. We further proposed a multi-view model for personality prediction
using both verbal and non-verbal descriptions, which gives improvement
compared to using only verbal descriptions. The uniqueness and challenges in
our dataset call for the development of narrative comprehension techniques
from the perspective of understanding characters.111Our code and data are
released at https://github.com/YisiSang/Story2Personality

##  1 Introduction

Character comprehension is commonly regarded as the cornerstone to
comprehending stories in psychology and education Bower and Morrow (1990);
Paris and Paris (2003); Zhao et al. (2022). The NLP community has done some
work on character comprehension in reading comprehension tasks, but most of
the existing studies focus on short or expository texts (e.g., story
summaries) Urbanek et al. (2019); Brahman et al. (2021). Moreover, most of
them are limited in factoid understanding of characters, such as coreference
resolution Chen and Choi (2016) and character relationships Iyyer et al.
(2016), and few studies have explored deeper comprehension of characters’
persona Flekova and Gurevych (2015); Sang et al. (2022a), on which humans can
generally do well.

Figure 1: An example excerpt from the movie script of “The Matrix”. Blue
utterances are the character Morpheus’s scene descriptions, red are his
dialogues. Morpheus’s MBTI personality was rated as ENFJ by 300 user votes.

We propose Story2Personality, a new narrative understanding benchmark to
encourage the study of character understanding. The goal of Story2Personality
is to predict personality according to the character’s narrative texts in the
script.

Personality prediction from narratives has many challenges. First, stories
often use a variety of narrative clues (e.g., scenery changes), sequence
(e.g., flashback) and rhetorical techniques (e.g., metaphor) Xu et al.
(2022b). Second, the inputs of the task are long ($>$10K words on average),
challenging the applications of Transformer-based models Vaswani et al.
(2017). Third, both the scene descriptions and dialogues are informative for
the prediction, requiring models to jointly consider multiple views of inputs.

This study makes the following contributions:

  * •

We establish a large-scale dataset for personality prediction of narrative
characters that can support the development of neural models. Our dataset
consists of 3,543 characters from 507 movies with MBTI labels of four
dimensions. In comparison, the only existing related dataset Flekova and
Gurevych (2015) contains only 298 characters and focuses on a single
dimension. Our dataset is proved challenging — on this binary classification
task, none of the baselines achieve higher than 60% macro-F1.

  * •

We develop a movie script parser to automatically process a script to a
structured form with the verbal character dialogues and the non-verbal scene
descriptions illustrating backgrounds. Human study shows that our parser is
more accurate compared to previous rule-based tools.

  * •

We propose an extension to BERT classifier Devlin et al. (2018) to handle the
long and multi-view (verbal and non-verbal) inputs. Our model improves 2-3%
over the baselines. This shows the potential of exploiting both verbal and
non-verbal narratives of characters, which is consistent with psychological
theory McCroskey and Richmond (1996); Richmond et al. (2008); and suggests
directions of future model design.

##  2 Related Work

#### Character-Centric Narrative Understanding

There have been existing studies on character-centric narrative understanding.
While many of them Massey et al. (2015); Srivastava et al. (2016); Brahman et
al. (2021) work on summaries of stories or summaries of characters. Their
scopes thus have a different assessment purpose from ours, and have the
challenge on understanding long narrative inputs greatly reduced.

For works that use long narratives, most of them study the inter-character
relationship Elson et al. (2010); Elsner (2012); Elangovan and Eisenstein
(2015); Iyyer et al. (2016); Chaturvedi et al. (2016, 2017); Kim and Klinger
(2019). Inter-character relationship is also related to social network
theories. Various of relationships have been considered in these studies,
while most of them rely on unsupervised learning and do not provide labeled
data for a direct automatic evaluation. TVSHOWGUESS explored multiple
perspectives of persona using long narratives but the task format is different
from us Sang et al. (2022b).

Finally, there is work on fundamental NLP annotating techniques over books and
screenplays, such as named entity recognition Bamman et al. (2019),
coreference resolution Chen and Choi (2016), event-centric extraction Xu et
al. (2022a), and entity-centric natural language modeling Clark et al. (2018)
which is different from narrative understanding. Their techniques can be
helpful to our task but the scope of their research is different from
character-centric comprehension.

#### Latent Persona Induction

Besides Flekova and Gurevych (2015) that is similar to our work in terms of
the focus on personality classification, there is another line of related work
on latent persona induction Bamman et al. (2013). The work learns a topic
model over character behaviors from books, and each latent topic corresponds
to an induced persona. The induced persona vectors can be then applied to
potential applications as a type of character representation.

From the perspective of practicality, our work and Bamman et al. (2013) have
their own strengths. From our motivation of story comprehension assessment,
the difference is whether we provide a direct evaluation of the character
understanding or evaluate it in down-streaming tasks – similar to the
aforementioned relationship detection work, it is also difficult to provide an
automatic and objective evaluation for the task of Bamman et al. (2013). The
advantage of our task is that it supports direct automatic evaluation by
itself, without the need for further downstream tasks; and it can be also used
to evaluate the methods for the task of Bamman et al. (2013). Moreover,
compared to a direct evaluation, the performance on a down-streaming task can
be affected by other factors other than persona so a good performance on
downstream tasks may not come directly caused by a good persona
representation. The cons of our task is that it is limited to the personality
types that have human annotations.

##  3 Background of MBTI

Personality is a “stable and measurable” individual characteristic Vinciarelli
and Mohammadi (2014) which can “distinguish internal properties of the person
from overt behaviors” Matthews et al. (2003). Understanding the personalities
of the characters is essential for grasping the story’s greater message. The
Myers–Briggs Type Indicator (MBTI) Myers (1962) and the Big-5 Personality are
two of the most popular personality scales. We used MBTI as the annotation
criteria since despite some validity controversy in self-report measurement,
research shows that a person’s friend can accurately judge his/her MBTI
personality Cohen et al. (1981). In our narrative comprehension scenario, a
fictional character’s MBTI personality is judged by other human raters in an
online community, which is quite similar to the third-person evaluation
scenario, and should yield a reasonable validity. We also conducted our study
on Big-5 and reported the results in Appendix 6.

MBTI assess the psychological preferences in how people perceive the world and
make decisions in four dimensions: E/I: extravert (E) is seen as being
generally active and objective while the intravert (I) is seen as generally
passive and subjective Sipps and Alexander (1987). S/N: sensing (S) is seen as
attending to sensory stimuli; intuition (N) describes a more detached,
insightful analysis of events and stimuli Boyle (1995). T/F: thinking (T)
involves logical reasoning and decision making; feeling (F) involves a more
subjective and interpersonal approach Thomas (1983). J/P: judging (J) attitude
is associated with prompt decision making; perception (P) involves greater
patience and waiting for more information before making a decision. An
individual’s MBTI type has a label based on her dominant preference for each
dimension. In Figure 1, Morpheus is an extraversion person, understanding the
world with intuition, dealing with things with feeling, and organize the world
around him by judging. Together gives an ENFJ type.

##  4 Story2Personality Dataset

We constructed our dataset in three stages: extracting movie scripts from the
Internet Movie Script Database (IMSDB 222https://imsdb.com/), parsing the
collected movie scripts into dialogue and scene sections, matching characters’
personality types from The Personality Database(PDB333https://www.personality-
database.com/) with their dialogues and scenes.

###  4.1 Movie Scripts Collection

We collected HTML files from IMSDB combined with movie scripts in NarrativeQA
Kočiskỳ et al. (2018). After removing corrupted or empty files, we got 1,464
usable scripts.

###  4.2 Our Statistical Movie Script Parser

As shown in Figure 1, a movie script usually has four basic format elements
Riley (2009): Scene Headings, one line description of each scene’s type,
location, and time (i.e., INT. ROOM 1313); Scene description, the description
of the actions of the characters (i.e., text in blue); Dialogues, names of
characters and actual words they speak (i.e., text in red); Transitions,
instructions for linking scenes together (i.e., FADE IN ON).

In order to extract dialogues and scene descriptions in a structured form, we
first split the scripts to sections, i.e., text chunks between two adjacent
bolded chunks which are scene headings or character names and stored the
bolded texts as section titles. Then we designed a statistical method to
classify the section types:

Rule-Based Pre-Processing  We start with a rule to classify the sections into
dialogues and scenes. As Figure 1 shows, a common format of movie scripts is
to align the shot headings, transitions and scene descriptions vertically, and
uses a larger indentat for dialogues. So, the indent size can be used to
identify dialogues. Since the indentat size may vary across different scripts.
Our rule assumes the sections as dialogues if they have larger indent compared
to FADE IN in the same script and the others as scenes.

Silver Parses Construction  The rule-based pre-processing introduces many
noises. We then designed a statistical method to automatically determine the
threshold indent of dialogues. First, we compute the averaged ratio $\mu$ of
dialogues in a script and its standard variation $\sigma$. Second, we keep
adding sections with the largest indent sizes to the set of dialogues, until
the ratio of added sections becomes larger than $\mu$$+$$\sigma$. Finally, we
keep the left sections as scenes. If none of the indentation size can reach
the ratio of dialogues in the range of $\mu$$\pm$$\sigma$, the movie script
was seen as a failure case. We designated the successfully processed scripts
with the dialogues/scene labels as the “silver” set which consists of 29% of
the scripts.

Section Classifier  For the failure scripts from the previous step and the
scripts without FADE IN markers, we trained a BERT-based section classifier
using 137,042 labeled sections from the silver set to label them. The
classifier achieved 99.31% accuracy on a held out validation set. The outputs
are our final parses.

###  4.3 Personality Collection and Mapping

We collect human rated MBTI types from _PDB_. Movie scripts are the blueprint
for the actor’s performance. An actor’s body language, dialogue, and contexts
are all described in the scripts Jhala (2008). Human rater’s perception of a
character’s personality from the movies would be consistent with the script’s
description. In total, we collected MBTI types of 28,653 characters. Each
character has an id, name, vote count, and voters’ agreement on each MBTI
dimension. For example, the MBTI profile in Figure 1 has 300 voters, with
different agreement rate along each dimension. To ensure the quality of
personality voting, we removed character profiles with $<$3 voters and $<$60%
agreement rate so some characters do not have all the 4 dimensions. We include
more details in Table 5 in the appendix. PDB’s When the user starts rating,
the rating interface hides the previous rater’s choices. Thus, the rater would
not have prior bias. We then matched the characters’ personality profiles to
the scripts, if the name can be softly matched to the dialogue title or the
recognized named entities in the scenes (details and example of the final
processed data in Appendix A). Table 1 shows the core statistics of our
dataset. The numbers of data points can also be found in Table 6.

|  Dimension | Train(%) | Dev(%) | Test(%)
---|---|---|---|---
(a) |  E/I | 45.9/51.8 | 49.6/49.0 | 52.6/44.2
N/S | 36.6/60.4 | 41.8/54.0 | 41.4/55.0
T/F | 54.7/43.2 | 45.8/50.8 | 46.0/52.8
J/P | 46.4/51.3 | 47.2/51.2 | 45.6/53.0

|  | Mean | Min | Max
---|---|---|---|---
(b) | # dialogues/character | 76.90 | 0 | 776
# words/dialogue | 917.74 | 1 | 12, 536
# scenes/character | 41.08 | 0 | 495
# words/scene | 1,381.47 | 1 | 25,457

Table 1: Distribution of two personality types per dimension (a) and core
statistics (b) in Story2Personality.

##  5 Dataset Analysis

We conduct human study to verify the advantage of our script parser; then
provide the human performance on our dataset.

Script Parsing results  We compared our parsing results with the results of
the state-of-the-art open-sourced script parser Ramakrishna et al. (2017),
which employs many human written rules, with a human study. We randomly
selected five scene descriptions and five dialogue sections in 10 common
movies, giving 100 snippets for evaluation (40 from the silver set). Then we
manually compared the parsing results with the original movie scripts. Table 2
shows the results. Our parser outperforms Ramakrishna et al. (2017) with a
large margin. Most mistakes of Ramakrishna et al. (2017) is to recognize
scenes as dialogues. There are other parsers but did not publish the code or
data, so we cannot conduct human study for comparison. A state-of-the-art
learning model Agarwal et al. (2014) reports 91% accuracy on line-level
classification. In a preliminary study, we achieve 99% on this task, but
finally choose do conduct more accurate section-level classification as in
Section 4.2.

| Correct scene | Correct dialogue
---|---|---
Ramakrishna et al. (2017) | 85% | 93%
Our parser | 97% | 100%

Table 2: Comparison of correct parsing results.

Human performance  We take the majority vote of each character’s MBTI types as
the groundtruth. This gives an averaged 93.54% human accuracy across the four
personality dimensions on our test data. Computing humans’ macro-F1 score
lacks an analytical form from the agreement scores. Therefore we make an
approximation by sampling three voters (the minimum number of voters in our
dataset) for each character and treating them like the predictions of three
different models. This gives overall $>$95% scores which is much higher than
model performance (in Table 3). The statistics of human agreement on MBTI
dimensions is shown in Table 6 in the appendix.

Figure 2: MR-MV BERT model architecture.

##  6 Experiments

#### Baselines

We build two baseline models. $\bullet$SVM, the LinearSVC from sklearn.svm. We
extracted top 20K word unigram, bigram, and trigram features according to term
frequency after removing stop words. We set $C$$=$0.1. $\bullet$ BERT, fine-
tuning the out-of-box BERT, with a linear head on the ‘[CLS]’ token’s final
layer embedding for classification.

#### Our Method

We propose the multi-view multi-row BERT (MV-MR BERT) classifier (Fig. 2)
which is an extension of BERT to deal with the long inputs and handle the
verbal and non-verbal information differently. First, to handle the long input
per character, we borrow the idea from fusion-in-decoder Izacard and Grave
(2020). Since the complexity of Transformers is $O(RL^{2})$ (with $R$ the
number of rows and $L$ the length per row), when $L$ is very large, we can
split it into multiple segments to reduce the quadratic term. Next, we rely on
the attention over all the segments to fuse the information. Specifically, we
split the input content $\mathcal{D}$ of a character into multiple segments
$\mathcal{D}=\\{\mathcal{S}_{i}\\}^{R}_{i=1}$, and encode all the segments in
a minibatch as $\mathbf{H}=\text{BERT}(\mathcal{S}_{i})\in\mathbb{R}^{R\times
L\times d}$, where $d$ is the hidden state size. Then a linear head is applied
to get the attention score across tokens in all the rows as
$\mathbf{\alpha}=\text{softmax}(\mathbf{H}W+b)\in[0,1]^{R\times L}$. The final
summarized representation of the input $\mathcal{D}$ is thus the weighted
summation
$\mathbf{h}_{\mathcal{D}}=\sum_{i=1}^{R}\sum_{j=1}^{L}\alpha_{ij}\mathbf{H}_{ij}$.
Second, to handle both the dialogue and behavioral description a character,
our multi-view model receives an input pair
$(\mathcal{D}^{\text{dial}},\mathcal{D}^{\text{scene}})$, then uses a shared
BERT and separated linear heads to compute the summarized states
$\mathbf{h}_{\mathcal{D}}^{\text{dial}}$ and
$\mathbf{h}_{\mathcal{D}}^{\text{scene}}$. The two vectors are fed into a
fully-connected layer for prediction. For the scene descriptions, we prepend a
special token “[ent]” to the target character’s name to denote its position.
The attention $\alpha^{\text{scene}}$ is only computed on these special
tokens.

#### Results and Analysis

Following Flekova and Gurevych (2015), we use macro-averaged F1 as evaluation
metric. Table 3 shows the main results on the four MBTI dimensions. Peak
performance was achieved by our MV-MR BERT. The result suggests using both
dialog and action scene descriptions consistently improved model performance.

The results are generally low compared to human performance, showing the task
is challenging to existing models. We analyzed the learning curve of BERT
model by adding the training data from 1K to 2.5K characters (Table 7 in
Appendix D). The model performance did not change a lot in the development
dataset. Figure 3 in Appendix D gives further evidence for the challenge of
our task, which shows the dev and test results are not highly-correlated,
meaning that by achieving near perfect accuracy on the training data, the
models largely overfit the noises instead of capturing real clues.

Model | E/I | N/S | T/F | J/P
---|---|---|---|---
SVM | 54.65 | 55.41 | 52.83 | 56.18
BERT | 56.06${}_{\pm\textrm{0.73}}$ | 55.59${}_{\pm\textrm{3.36}}$ | 57.13${}_{\pm\textrm{0.97}}$ | 57.59${}_{\pm\textrm{1.40}}$
MV-MR BERT | 57.50${}_{\pm\textrm{2.04}}$ | 57.42${}_{\pm\textrm{4.27}}$ | 60.33${}_{\pm\textrm{0.93}}$ | 59.83${}_{\pm\textrm{1.42}}$
\- multiview | 57.30${}_{\pm\textrm{1.91}}$ | 57.05${}_{\pm\textrm{1.80}}$ | 57.04${}_{\pm\textrm{2.05}}$ | 57.39${}_{\pm\textrm{2.21}}$
Human Perf. | 98.19${}_{\pm\textrm{0.60}}$ | 97.82${}_{\pm\textrm{0.10}}$ | 98.51${}_{\pm\textrm{0.67}}$ | 98.03${}_{\pm\textrm{0.19}}$

Table 3: Macro F1 scores on the four dimensions

#### Model Performance on the Big Five Personality Test

We collected a variation of Big 5, the Global 5, from the PDB. The Global 5
adaptation of the Big Five Digman (1997) consists of Extroversion, Emotional
Stability, Orderliness, Accommodation, and Intellect. The SLOAN nation system
is the scoring format for Global 5 test. SLOAN nation keys are: Social/
Reserved, Limbic/ Calm, Organized/ Unstructured, Accommodating/ Egocentric,
Non-curious/ Inquisitive. The number of people who evaluated the movie
characters with big5 was very small, and after eliminating the characters
without dialogues, we only got 1,346 characters’ data. Such a small amount of
data deeply affects the training of neural-based model. Table 4 shows the
model performance.

Model | S/R | L/C | O/U | A/E | N/I
---|---|---|---|---|---
SVM | 57.31 | 55.93 | 59.40 | 53.51 | 55.89
MV-MR BERT | 58.98 | 54.20 | 62.26 | 54.00 | 59.04
Table 4: Macro F1 scores on the Big 5 dimensions

##  7 Limitation

Movie scripts are the blueprint for the actor’s performance. An actor’s
dialogue, body language, and the contexts are well described in the scripts.
There is sufficient information in the scripts for readers to understand the
characters. However, the actors’ portrayal have the potential to add
additional cues to influence the audience’s perception of the fictional
characters’ personalities. In future work we will try to use multi-modality
data as input.

##  8 Conclusion

We develop a movie script parser and proposed a new narrative understanding
benchmark, Story2Personality, which enables neural model training for
understanding characters. We evaluate several classifiers on our task – while
our multi-view multi-view BERT model achieves a substantial improvement over
the SVM and BERT baselines, there is a huge gap compared to human performance.
This indicates our dataset a valuable and challenging task for future
research. In the future we will expand our dataset.

## References

  * Agarwal et al. (2014) Apoorv Agarwal, Sriramkumar Balasubramanian, Jiehan Zheng, and Sarthak Dash. 2014\.  Parsing screenplays for extracting social networks from movies.  In _Proceedings of the 3rd Workshop on Computational Linguistics for Literature (CLFL)_ , pages 50–58.
  * Bamman et al. (2013) David Bamman, Brendan O’Connor, and Noah A Smith. 2013.  Learning latent personas of film characters.  In _Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 352–361.
  * Bamman et al. (2019) David Bamman, Sejal Popat, and Sheng Shen. 2019.  An annotated dataset of literary entities.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , pages 2138–2144.
  * Bower and Morrow (1990) Gordon H Bower and Daniel G Morrow. 1990.  Mental models in narrative comprehension.  _Science_ , 247(4938):44–48.
  * Boyle (1995) Gregory J Boyle. 1995.  Myers-briggs type indicator (mbti): some psychometric limitations.  _Australian Psychologist_ , 30(1):71–74.
  * Brahman et al. (2021) Faeze Brahman, Meng Huang, Oyvind Tafjord, Chao Zhao, Mrinmaya Sachan, and Snigdha Chaturvedi. 2021.  " let your characters tell their story": A dataset for character-centric narrative understanding.  _arXiv preprint arXiv:2109.05438_.
  * Chaturvedi et al. (2017) Snigdha Chaturvedi, Mohit Iyyer, and Hal Daume III. 2017.  Unsupervised learning of evolving relationships between literary characters.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , volume 31.
  * Chaturvedi et al. (2016) Snigdha Chaturvedi, Shashank Srivastava, Hal Daume III, and Chris Dyer. 2016.  Modeling evolving relationships between characters in literary novels.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , volume 30.
  * Chen and Choi (2016) Yu-Hsin Chen and Jinho D Choi. 2016.  Character identification on multiparty conversation: Identifying mentions of characters in tv shows.  In _Proceedings of the 17th Annual Meeting of the Special Interest Group on Discourse and Dialogue_ , pages 90–100.
  * Clark et al. (2018) Elizabeth Clark, Yangfeng Ji, and Noah A Smith. 2018.  Neural text generation in stories using entity representations as context.  In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)_ , pages 2250–2260.
  * Cohen et al. (1981) David Cohen, Marilye Cohen, and Herbert Cross. 1981.  A construct validity study of the myers-briggs type indicator.  _Educational and Psychological Measurement_ , 41(3):883–891.
  * Devlin et al. (2018) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018.  Bert: Pre-training of deep bidirectional transformers for language understanding.  _arXiv preprint arXiv:1810.04805_.
  * Digman (1997) John M Digman. 1997.  Higher-order factors of the big five.  _Journal of personality and social psychology_ , 73(6):1246.
  * Elangovan and Eisenstein (2015) Vinodh Krishnan Elangovan and Jacob Eisenstein. 2015.  " you’re mr. lebowski, i’m the dude": Inducing address term formality in signed social networks.  In _HLT-NAACL_.
  * Elsner (2012) Micha Elsner. 2012.  Character-based kernels for novelistic plot structure.  In _Proceedings of the 13th Conference of the European Chapter of the Association for Computational Linguistics_ , pages 634–644.
  * Elson et al. (2010) David K Elson, Kathleen McKeown, and Nicholas J Dames. 2010.  Extracting social networks from literary fiction.
  * Flekova and Gurevych (2015) Lucie Flekova and Iryna Gurevych. 2015.  Personality profiling of fictional characters using sense-level links between lexical resources.  In _Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing_ , pages 1805–1816.
  * Iyyer et al. (2016) Mohit Iyyer, Anupam Guha, Snigdha Chaturvedi, Jordan Boyd-Graber, and Hal Daumé III. 2016.  Feuding families and former friends: Unsupervised learning for dynamic fictional relationships.  In _Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 1534–1544.
  * Izacard and Grave (2020) Gautier Izacard and Edouard Grave. 2020.  Leveraging passage retrieval with generative models for open domain question answering.  _arXiv preprint arXiv:2007.01282_.
  * Jhala (2008) Arnav Jhala. 2008.  Exploiting structure and conventions of movie scripts for information retrieval and text mining.  In _Joint International Conference on Interactive Digital Storytelling_ , pages 210–213. Springer.
  * Jung (2016) Carl Jung. 2016.  _Psychological types_.  Routledge.
  * Kim and Klinger (2019) Evgeny Kim and Roman Klinger. 2019.  Frowning frodo, wincing leia, and a seriously great friendship: Learning to classify emotional relationships of fictional characters.  In _Proceedings of NAACL-HLT_ , pages 647–653.
  * Kočiskỳ et al. (2018) Tomáš Kočiskỳ, Jonathan Schwarz, Phil Blunsom, Chris Dyer, Karl Moritz Hermann, Gábor Melis, and Edward Grefenstette. 2018.  The narrativeqa reading comprehension challenge.  _Transactions of the Association for Computational Linguistics_ , 6:317–328.
  * Massey et al. (2015) Philip Massey, Patrick Xia, David Bamman, and Noah A Smith. 2015.  Annotating character relationships in literary texts.  _arXiv preprint arXiv:1512.00728_.
  * Matthews et al. (2003) Gerald Matthews, Ian J Deary, and Martha C Whiteman. 2003.  _Personality traits_.  Cambridge University Press.
  * McCroskey and Richmond (1996) James C McCroskey and Virginia P Richmond. 1996.  _Fundamentals of human communication: An interpersonal perspective_.  Waveland PressInc.
  * Myers (1962) Isabel Briggs Myers. 1962.  The myers-briggs type indicator: Manual (1962).
  * Paris and Paris (2003) Alison H Paris and Scott G Paris. 2003.  Assessing narrative comprehension in young children.  _Reading Research Quarterly_ , 38(1):36–76.
  * Ramakrishna et al. (2017) Anil Ramakrishna, Victor R Martínez, Nikolaos Malandrakis, Karan Singla, and Shrikanth Narayanan. 2017.  Linguistic analysis of differences in portrayal of movie characters.  In _Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 1669–1678.
  * Richmond et al. (2008) Virginia P Richmond, James C McCroskey, and Mark Hickson. 2008.  _Nonverbal behavior in interpersonal relations_.  Allyn & Bacon.
  * Riley (2009) Christopher Riley. 2009.  _The Hollywood standard: the complete and authoritative guide to script format and style_.  Michael Wiese Productions.
  * Sang et al. (2022a) Yisi Sang, Xiangyang Mou, Jing Li, Jeffrey Stanton, and Mo Yu. 2022a.  A survey of machine narrative reading comprehension assessments.  _arXiv preprint arXiv:2205.00299_.
  * Sang et al. (2022b) Yisi Sang, Xiangyang Mou, Mo Yu, Shunyu Yao, Jing Li, and Jeffrey Stanton. 2022b.  Tvshowguess: Character comprehension in stories as speaker guessing.  _arXiv preprint arXiv:2204.07721_.
  * Sipps and Alexander (1987) Gary J Sipps and Ralph A Alexander. 1987.  The multifactorial nature of extraversion-introversion in the myers-briggs type indicator and eysenck personality inventory.  _Educational and Psychological Measurement_ , 47(3):543–552.
  * Srivastava et al. (2016) Shashank Srivastava, Snigdha Chaturvedi, and Tom Mitchell. 2016.  Inferring interpersonal relations in narrative summaries.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , volume 30.
  * Thomas (1983) Charles R Thomas. 1983.  Field independence and myers-briggs thinking individuals.  _Perceptual and motor skills_.
  * Urbanek et al. (2019) Jack Urbanek, Angela Fan, Siddharth Karamcheti, Saachi Jain, Samuel Humeau, Emily Dinan, Tim Rocktäschel, Douwe Kiela, Arthur Szlam, and Jason Weston. 2019.  Learning to speak and act in a fantasy text adventure game.  _arXiv preprint arXiv:1903.03094_.
  * Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017.  Attention is all you need.  In _Advances in neural information processing systems_ , pages 5998–6008.
  * Vinciarelli and Mohammadi (2014) Alessandro Vinciarelli and Gelareh Mohammadi. 2014.  A survey of personality computing.  _IEEE Transactions on Affective Computing_ , 5(3):273–291.
  * Xu et al. (2022a) Guangxuan Xu, Paulina Toro Isaza, Moshi Li, Akintoye Oloko, Bingsheng Yao, Aminat Adebeyi, Yufang Hou, Nanyun Peng, and Dakuo Wang. 2022a.  Nece: Narrative event chain extraction toolkit.  _arXiv preprint arXiv:2208.08063_.
  * Xu et al. (2022b) Ying Xu, Dakuo Wang, Mo Yu, Daniel Ritchie, Bingsheng Yao, Tongshuang Wu, Zheng Zhang, Toby Li, Nora Bradford, Branda Sun, Tran Hoang, Yisi Sang, Yufang Hou, Xiaojuan Ma, Diyi Yang, Nanyun Peng, Zhou Yu, and Mark Warschauer. 2022b.  Fantastic questions and where to find them: FairytaleQA – an authentic dataset for narrative comprehension.  In _Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 447–460, Dublin, Ireland. Association for Computational Linguistics.
  * Zhao et al. (2022) Zhenjie Zhao, Yufang Hou, Dakuo Wang, Mo Yu, Chengzhong Liu, and Xiaojuan Ma. 2022\.  Educational question generation of children storybooks via question type distribution learning and event-centric summarization.  _arXiv preprint arXiv:2203.14187_.

##  Appendix A Details of Dataset Construction

#### Soft Name Matching Algorithm

We created two movie-character dictionaries to associate the characters with
the movies using the characters’ full names and their subcategories (i.e.,
movie names) in personality profile data, as well as section titles (i.e.,
character names or scene headings) and movie names in the movie script data.
Then, we tokenized and lowercased the character names. We matched both the
exact same full names and the intersections of tokens such as the first or
last name of the full name when the movie names are matched. To identify a
character’s scene descriptions, we extracted named entities from scene
descriptions and then matched the characters and scenes based on their names
using the same method. After matching the character name with the movie name,
we store the MBTI personality, vote count, dialogue and scene descriptions
into a dictionary for each character.

#### Example Data Item for One Character

Figure 4 shows the example of information for one character Gary from the
movie “Joker” in our Story2Personality, stored in json format. The data item
contains the ID (‘id’), character name (‘mbti_profile’) and movie name
(‘subcategory’) in the PDB website; together with the human voted MBTI types
and the number of votes. Finally we save the dialogues of the character and
the scenes he appears in two separated entries. For scenes, we save both the
scene texts and the soft matched name mentions in the texts for the target
character. The name mention is used to prepend the special tokens in our MV-MR
BERT model.

##  Appendix B Additional Information of the Dataset

Table 5 lists the distribution of all the 16 MBTI types in our dataset,
together with a representative movie character for each type.

Personality | % | Example
---|---|---
ISTJ | 8.41% |  Darth Vader (“Star Wars”)
ISTP | 8.07% |  Shrek (“Shrek”)
ESTP | 8.21% |  Han Solo (“Star Wars”)
ESTJ | 6.52% |  Boromir (“The Lord of the Rings”)
ISFJ | 6.41% |  Forrest Gump (“Forrest Gump”)
ISFP | 6.49% |  Harry Potter (“Harry Potter”)
ESFJ | 4.88% |  Cher Horowitz (“Clueless”)
ESFP | 7.06% |  Jack Dawson (“Titanic”)
INFJ | 4.80% |  Edward Cullen (“Twilight”)
INFP | 5.42% |  Amélie Poulain (“Amélie”)
ENFP | 3.90% |  Anna (“Frozen”)
ENFJ | 3.75% |  Judy Hopps (“Zootopia”)
INTJ | 4.26% |  Michael Corleone (“The God Father”)
INTP | 3.75% |  Neo (“The Matrix”)
ENTP | 4.94% |  Tyler Durden (“Fight Club”)
ENTJ | 4.88% |  Patrick Bateman (“American Psycho”)

Table 5: Distribution of the 16 MBTI personality types in Story2Personality

##  Appendix C Statistics of Human Agreement

Table 6 lists the human agreement score on each MBTI dimension, on which we
compute the human accuracy and approximate the human macro-F1 scores.

The raters are most divided in annotation of N, with an average agreement is
91.06% and the standard deviation 0.11. One reason is that the perceptual
style dimension N/S measures how the individual obtain information. Comparing
with dimensions related to attitudes (E/I) or decision making (T/F, J/P) Jung
(2016) perceptual style is more implicit. Specifically, S is seen as attending
to sensory stimuli, while N describes a more detached, insightful analysis of
events and stimuli Boyle (1995). They are more difficult to determine from the
explicit story narratives.

| Mean | Min | Max | STD | #Character
---|---|---|---|---|---
I | 94.43% | 60% | 100% | 0.10 | 1,783
E | 94.22% | 60% | 100% | 0.10 | 1,679
N | 91.06% | 60% | 100% | 0.11 | 1,347
S | 93.32% | 60% | 100% | 0.11 | 2,082
T | 94.22% | 60% | 100% | 0.10 | 1,851
F | 93.68% | 60% | 100% | 0.10 | 1,617
P | 93.68% | 60% | 100% | 0.10 | 1,825
J | 93.72% | 60% | 100% | 0.10 | 1,644
Table 6: Descriptive statistics of voters’ agreement Figure 3: Dev vs. test F1
scores of BERT-based models.

##  Appendix D Additional Model Performance

Figure 3 gives further evidence for the challenge of our task, which plots the
dev versus test scores during our model selection. It shows the dev and test
results are not highly-correlated, meaning that by achieving near perfect
accuracy on the training data, the models largely overfit the noises instead
of capturing real clues. Both length and multiview have an improvement on
model performance, but length has a slightly smaller impact, as shown in Table
8, when increasing the number of input tokens, the performance is not greatly
affected.

| 1K | 1.5K | 2K |  $\sim$2.5K
---|---|---|---|---
SVM | 50.33 | 52.19 | 54.56 | 55.41
BERT | 54.32 | 55.42 | 55.58 | 55.59
Table 7: Learning curve with varying amount of training data (on N/S). | 0.5K | 1K | 2K | 4K
---|---|---|---|---
J/P | 59.79 | 58.18 | 60.35 | 58.77
T/F | 59.91 | 64.31 | 63.32 | 65.42
N/S | 56.15 | 53.86 | 55.65 | 57.18
I/E | 61.64 | 61.05 | 62.87 | 62.69
Table 8: Ablation experiment on input length. Figure 4: An example except from
Story2Personality. Each character has ID (’id’), character name
(’mbti_profile’), movie name (’subcategory’), dialogue, and scene
descriptions.

##  Appendix E Model Checklist

We implement our baselines based on HuggingFace
Transformers.444https://github.com/huggingface/transformers We use the pre-
trained bert-base-uncased models. We train all the models with the Adam
optimizer.

We train our model on a single V100 GPU. It takes around 2 hour and 10 minutes
to train a multi-row BERT model. For all the models, we train in total 20
epochs.

#### Hyperparameters

We set the number of rows in MV-MR BERT to 20, to maximize the usage of GPU
memory. We set the learning rate to 2e-5. We report the average performance of
five runs.


# Few-Shot and Zero-Shot Learning for Historical Text Normalization

Marcel Bollmann♣ Natalia Korchagina♢ Anders Søgaard♣
♣Department of Computer Science, University of Copenhagen
♢Institute of Computational Linguistics, University of Zurich
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

Historical text normalization often relies on small training datasets. Recent
work has shown that multi-task learning can lead to significant improvements
by exploiting synergies with related datasets, but there has been no
systematic study of different multi-task learning architectures. This paper
evaluates 63 multi-task learning configurations for sequence-to-sequence-based
historical text normalization across ten datasets from eight languages, using
autoencoding, grapheme-to-phoneme mapping, and lemmatization as auxiliary
tasks. We observe consistent, significant improvements across languages when
training data for the target task is limited, but minimal or no improvements
when training data is abundant. We also show that zero-shot learning
outperforms the simple, but relatively strong, identity baseline.

##  1 Introduction

Historical text normalization is the task of mapping variant spellings in
historical documents—e.g., digitized medieval manuscripts—to a common form,
typically their modern equivalent. The aim is to make these documents amenable
to search by today’s scholars, processable by NLP tools, and accessible to lay
people. Many historical documents were written in the absence of standard
spelling conventions, and annotated datasets are rare and small, making
automatic normalization a challenging task (cf. Piotrowski, 2012; Bollmann,
2018).

In this paper, we experiment with datasets in eight different languages:
English, German, Hungarian, Icelandic, Portuguese, Slovene, Spanish, and
Swedish. We use a standard neural sequence-to-sequence model, which has been
shown to be competitive for this task (e.g., Korchagina, 2017; Bollmann, 2018;
Tang et al., 2018). Our main focus is on analyzing the usefulness of multi-
task learning strategies (a) to leverage whatever supervision is available for
the language in question (few-shot learning), or (b) to do away with the need
for supervision in the target language altogether (zero-shot learning).

Bollmann et al. (2017) previously showed that multi-task learning with
grapheme-to-phoneme conversion as an auxiliary task improves a sequence-to-
sequence model for historical text normalization of German texts; Bollmann et
al. (2018) showed that multi-task learning is particularly helpful in low-
resource scenarios. We consider three auxiliary tasks in our
experiments—grapheme-to-phoneme mapping, autoencoding, and lemmatization—and
focus on extremely low-resource settings.

Our paper makes several contributions:

  1. (a)

We evaluate 63 multi-task learning configurations across ten datasets in eight
languages, and with three different auxiliary tasks.

  2. (b)

We show that in few-shot learning scenarios (ca. 1,000 tokens), multi-task
learning leads to robust, significant gains over a state-of-the-art, single-
task baseline.111We note that 1,000 tokens is more instances than is typically
considered in few-shot learning; e.g., Kimura et al. (2018) use up to 200
instances. We argue that for structured prediction it is reasonable to assume
more data, yet we also consider scenarios down to as little as 100 instances.

  3. (c)

We are, to the best of our knowledge, the first to consider zero-shot
historical text normalization, and we show significant improvements over the
simple, but relatively strong, identity baseline.

While our focus is on the specific task of historical text normalization, we
believe that our results can be of interest to anyone looking to apply multi-
task learning in low-resource scenarios.

Dataset/Language | Tokens (Dev)
---|---
DEA | German (Anselm) | 45,996
DER | German (RIDGES) | 9,712
EN | English | 16,334
ES | Spanish | 11,650
HU | Hungarian | 16,707
IS | Icelandic | 6,109
PT | Portuguese | 26,749
SLB | Slovene (Bohorič) | 5,841
SLG | Slovene (Gaj) | 20,878
SV | Swedish | 2,245
Table 1: Historical datasets used in our experiments and the size of their
development sets. (Size of the training sets is fixed in all our experiments.)

#### Datasets

We consider ten datasets spanning eight languages, taken from Bollmann
(2019).222The datasets are available from:
https://github.com/coastalcph/histnorm Table 1 gives an overview of the
languages and the size of the development set, which we use for evaluation.

##  2 Model architecture

We use a standard attentional encoder–decoder architecture (Bahdanau et al.,
2014) with words as input sequences and characters as input symbols. 333Our
implementation uses the XNMT toolkit (Neubig et al., 2018,
https://github.com/neulab/xnmt). Following the majority of previous work on
this topic (cf. Sec. 5), we limit ourselves to word-by-word normalization,
ignoring problems of contextual ambiguity. Our model consists of the following
parts (which we will also refer to using the bolded letters):

  * •

Source embedding layer: transforms input characters into dense vectors.

  * •

Encoder: a single bidirectional LSTM that encodes the embedded input sequence.

  * •

Attention layer: calculates attention from the encoded inputs and the current
decoder state using a multi-layer perceptron (as in Bahdanau et al., 2014).

  * •

Target embedding layer: transforms output characters into dense vectors.

  * •

Decoder: a single LSTM that decodes the encoded sequence one character at a
time, using the attention vector and the embedded previous output characters
as input.

  * •

Prediction layer: a final feed-forward layer that linearly transforms the
decoder output and performs a softmax to predict a distribution over all
possible output characters.

#### Hyperparameters

We tuned our hyperparameters on the English development section. We use
randomly initialized embeddings of dimensionality 60, hidden layers of
dimensionality 300, a dropout of 0.2 and a batch size of 30. We train the
model for an unspecified number of epochs, instead relying on early stopping
on a held-out validation set. Since we experiment with varying amounts of
training data, we choose to derive this held-out data from the given training
set, using only 90% of the tokens as actual training data and the remaining
10% to determine early stopping.

##  3 Multi-task learning

Multi-task learning (MTL) is a technique to improve generalization by training
a model jointly on a set of related tasks. We follow the common approach of
hard parameter sharing suggested by Caruana (1993), in which certain parts of
a model architecture are shared across all tasks, while others are kept
distinct for each one. Such approaches have been applied successfully to a
variety of problems, e.g., machine translation Dong et al. (2015), sequence
labelling Yang et al. (2016); Peng and Dredze (2017), or discourse parsing
Braud et al. (2016).

#### Auxiliary tasks

We experiment with the following auxiliary tasks:

  * •

Autoencoding. We use data extracted from Wikipedia444Whenever possible, we
used the dumps provided by the Polyglot project:
https://sites.google.com/site/rmyeid/projects/polyglot
Since an Icelandic text dump was not available from Polyglot, we generated one
ourselves using the Cirrus Extractor: https://github.com/attardi/wikiextractor
All dumps were cleaned from punctuation marks. and train our model to recreate
the input words. In the normalization task, large parts of the input words
often stay the same, so autoencoding might help to reinforce this behavior in
the model.

  * •

Grapheme-to-phoneme mapping (g2p). This task uses the data by Deri and Knight
(2016) to map words (i.e., sequences of graphemes) to sequences of phonemes.
Bollmann et al. (2017) previously showed that this task can improve historical
normalization, possibly because changes in spelling are often motivated by
phonological processes, an assumption also made by other normalization systems
(Porta et al., 2013; Etxeberria et al., 2016).

  * •

Lemmatization. We use the UniMorph dataset (Kirov et al.,
2018)555https://unimorph.github.io/ to learn mappings from inflected word
forms to their lemmas. This task is similar to normalization in that it maps a
set of different word forms to a single target form, which typically bears a
high resemblance to the input words.

Since we train separate models for each historical dataset, we always use
auxiliary data from the same language as the dataset.

#### Training details

When training an MTL model, we make sure that each training update is based on
a balanced combination of main and auxiliary task inputs; i.e., for each batch
of 30 tokens of the historical normalization task, the model will see 10
tokens from each auxiliary task. Epochs are still counted based on the
normalization task only. This way, we try to make up for the imbalanced
quantity of different auxiliary datasets.

0%20%40%60%80%SEADPSEATDSADPSEATPSEDETDPSETPSETD$\displaystyle\cdots$ No
MTL$\displaystyle\cdots$ ATSTEPATD Figure 1: Normalization accuracy on the
English-1k dataset, trained jointly with all three auxiliary tasks; letters
indicate which model components (cf. Sec. 2) are shared between tasks.
1234560%65%70%75% Figure 2: Quartiles of the normalization accuracies (on
English-1k) by the number of shared components in the MTL model; bottom dashed
line indicates no shared components (= single-task), top dashed line indicates
all (= 6) shared components.

###  3.1 Experiment 1: What to share?

In previous work on multi-task learning, there is no clear consensus on which
parts of a model to share and which to keep separate. Bollmann et al. (2017)
share all parts of the model except for the final prediction layer, while
other multi-task sequence-to-sequence models keep task-specific encoders and
decoders (cf. also Sec. 5). In principle, though, the decision to share
parameters between tasks can be made for each of the encoder–decoder
components individually, allowing for many more possible MTL configurations.

#### Setup

We explore the effect of different sharing configurations. The architecture
described in Sec. 2 leaves us with $2^{6}=64$ possible model configurations.
When all parameters are shared, this is identical to training a single model
to perform all tasks at once; when none are shared, this is identical to a
single-task model trained on historical normalization only. We identify an MTL
configuration using letters (cf. the bold letters from Sec. 2) to indicate
which parts of the model are shared; e.g., an “SE” model would share the
source embeddings and the encoder, an “SEATD” model would share everything
except the final prediction layer, and so on.

In Experiment 1, we only use the first 1,000 tokens of the English historical
dataset for training. We combine this with all three auxiliary tasks (using
their full datasets) and train one MTL model for each of the 64 different
sharing configurations.

#### Results

Figure 1 shows an excerpt of the results, evaluated on the dev set of the
English dataset. The best MTL model achieves a normalization accuracy of
75.9%, while the worst model gets 58.6%. In total, 49 configurations
outperform the single-task model, showing the general effectiveness of the MTL
approach. Sharing more is generally better; nine out of the top ten
configurations share at least four components. Figure 2 visualizes the
accuracy distribution by the number of shared components in the MTL model,
supporting this conclusion.

DEADERENESHUISPTSLBSLGSV-10%0%10%20%30%Auxiliary taskAutoencodingGrapheme-to-
phonemeLemmatization Figure 3: Error reduction for the SEADP configuration by
auxiliary task, using 1,000 tokens from the historical datasets for training.

###  3.2 Experiment 2: Which auxiliary tasks?

In the previous experiment, we trained the models using all three auxiliary
tasks at the same time. However, not all of these tasks might be equally
helpful for learning the normalization task. While Bollmann et al. (2017) show
the effectiveness of the grapheme-to-phoneme task, they only evaluate on
German, and autoencoding and lemmatization have so far not been evaluated at
all for improving historical text normalization.

#### Setup

We want to investigate the improvements from each auxiliary task in isolation
compared to (a) the single-task baseline and (b) the previous approach of
training with all three auxiliary tasks simultaneously. For this, we select
the best MTL configuration from Sec. 3.1, which is to share everything except
the target embeddings (“SEADP”), and train one single-task model and four MTL
models per dataset: one for each of the three auxiliary tasks, and one that
uses all three tasks at the same time.

As before, we only use the first 1,000 tokens of each historical dataset. This
also makes the results more comparable across datasets, as the size of the
training set for the main task can affect the usefulness of multi-task
learning.666The same is true, of course, for the size of the auxiliary
datasets. We try to balance out this factor by balancing the training updates
as described in Sec. 3 “Training details”, but we also note that we do not
observe a correlation between auxiliary dataset size and its effectiveness for
MTL in Fig. 3.

Dataset | Single | Multi-task
---|---|---
|  | Autoenc | Lemma | g2p | All 3
DEA | 54.84 | 56.41 | 56.55 | 55.99 | 56.52
DER | 56.72 | 65.05 | 63.79 | 60.25 | 64.49
EN | 66.95 | 76.94 | 73.84 | 68.72 | 72.01
ES | 74.68 | 77.87 | 76.97 | 78.45 | 79.09
HU | 42.44 | 40.39 | 40.49 | 40.07 | 38.64
IS | 63.40 | 67.31 | 67.02 | 66.31 | 68.51
PT | 72.23 | 76.28 | 73.89 | 74.27 | 75.55
SLB | 74.06 | 74.44 | 74.54 | 75.59 | 74.39
SLG | 86.34 | 87.86 | 86.15 | 87.40 | 89.45
SV | 69.98 | 70.29 | 72.34 | 65.97 | 73.05
Micro-Avg | 64.46 | 67.46 | 66.47 | 65.79 | 67.04
(a) Single-task vs. multi-task models

Dataset | Best in (a) | from Bollmann (2019)
---|---|---
|  | Norma | SMT | NMT
DEA | 56.55 | 61.27 | 58.60 | 52.74
DER | 65.05 | 73.62 | 75.04 | 60.61
EN | 76.94 | 84.53 | 83.81 | 66.93
ES | 79.09 | 86.21 | 85.89 | 76.32
HU | 42.44 | 55.75 | 53.00 | 40.52
IS | 68.51 | 70.86 | 72.30 | 62.80
PT | 76.28 | 82.94 | 82.00 | 71.43
SLB | 75.59 | 78.97 | 82.90 | 73.83
SLG | 89.45 | 84.36 | 90.00 | 86.31
SV | 73.05 | 74.54 | 78.51 | 66.43
Micro-Avg | 68.13 | 73.30 | 73.07 | 63.80
(b) Comparison to previous work

Table 2: Normalization accuracy on dev sets after training on 1,000 tokens.
Best results highlighted in bold.

#### Results

Figure 3 shows the error reduction of the MTL models compared to the single-
task setup. For most datasets, MTL improves the results; the main exception is
Hungarian, where all three auxiliary tasks lead to a loss in accuracy. The
results show that not all auxiliary tasks are equally beneficial. Autoencoding
provides the largest error reduction in most cases, while lemmatization is
often slightly worse, but provides the best result for German (Anselm) and
Swedish. The grapheme-to-phoneme task, on the other hand, performs worst on
average, yielding much less benefits on German (Ridges) and English, and even
increases the error on Swedish.

Table 2(a) shows the accuracy scores for all datasets and models. The full MTL
model—training jointly on all tasks—only achieves the best performance on four
of the datasets. Since the dev sets used for this evaluation vary strongly in
size, we also calculate the micro-average of the accuracy scores, i.e., the
accuracy obtained over the concatenation of all datasets. Here, we can see
that using only autoencoding as an auxiliary task actually produces the
highest average accuracy.

###  3.3 Experiment 3: How much training data?

All previous experiments have used 1,000 tokens from each historical dataset
for training. Bollmann et al. (2018) show that the benefits of multi-task
learning depend on training data quantity, so it is unclear whether the
findings generalize to smaller or larger datasets.

#### Setup

We analyze the benefit of MTL depending on the amount of training data that is
used for the main task. We do this by training MTL models (using all three
auxiliary tasks, as in Sec. 3.1) with varying amounts of historical training
data, ranging from 100 tokens to 50,000 tokens. Different sharing
configurations might conceivably give different benefits based on the training
set size. We therefore evaluate each of the top three MTL configurations from
Sec. 3.1, as well as the single-task model, across different data sizes.

#### Results

Figure 4 shows learning curves for all of our historical datasets. The
quantity of improvements from MTL differs between datasets, but there is a
clear tendency for MTL to become less beneficial as the size of the
normalization training set increases. In some cases, using MTL with larger
training set sizes even results in _lower_ accuracy compared to training a
single-task model to do normalization only. This suggests that multi-task
learning—at least with the auxiliary tasks we have chosen here—is mostly
useful when the training data for the main task is sparse.

Since the accuracy scores of the different models are often within close range
of each other, Figure 5 visualizes the three MTL configurations in terms of
error reduction compared to the single-task model, averaged over all ten
datasets. This again highlights the decreasing gains from MTL with increasing
amounts of training data.

30%50%70%90%EnglishGerman (Anselm)German
(RIDGES)30%50%70%90%HungarianIcelandicPortuguese30%50%70%90%Slovene
(Bohorič)1002505001k2.5k5k10k25k50kSlovene
(Gaj)1002505001k2.5k5k10k25k50kSpanish1002505001k2.5k5k10k25k50k30%50%70%90%SwedishNo
MTLSEATDSEADPSADP Figure 4: Learning curves for all datasets, showing the
normalization accuracy of a single-task and three multi-task learning models
in relation to the training set size; note that the $x$-axis is log-scaled.

###  3.4 Comparison to previous work

Bollmann (2019) compares normalization models when trained with different
amounts of data, including a setting with 1,000 tokens for training, allowing
us to directly compare our results with those reported there.777Bollmann
(2019) only shows graphical plots for these results, but the exact figures
were released at:
https://github.com/coastalcph/histnorm/blob/master/appendix_tab6.pdf These
results are shown in Table 2(b). Comparing our single-task system with their
NMT model (which is very similar to ours), we see that the scores are overall
comparable, suggesting that our implementation is sound. At the same time, our
best scores with MTL are still far below those produced by SMT or the rule-
based “Norma” tool. This, unfortunately, is a negative result for the neural
approach in this low-resource scenario, and the diminishing gains from MTL
that were shown in Sec. 3.3 suggest that our presented approach will not be
sufficient for elevating the neural model above its non-neural alternatives
for this particular task.

###  3.5 Experiment 4: Zero-shot learning

Most previous work on historical text normalization has focused on a
supervised scenario where some labeled data is available for the target
domain, i.e., the particular historical language you are interested in. Since
spelling variation is highly idiosyncratic in the absence of normative
spelling guidelines, models are not expected to generalize beyond specific
language stages, or sometimes even manuscript collections. This means that
many historical text normalization projects require resources to annotate new
data. This paper is the first to experiment with a zero-shot learning scenario
that leverages existing data from other languages, but assumes no labeled data
for the target language.

#### Setup

For the zero-shot experiments, we use the same model as for the single-task
baseline; in other words, all layers are shared between all tasks and
languages. Instead, to allow the model to discern between languages and tasks,
we prepend two extra symbols to all model inputs: a _language identifier_ and
a _task identifier._ For each language, we then train a single model on all
tasks—normalization, lemmatization, autoencoding, and grapheme-to-phoneme
transduction—and all languages, _except_ for the normalization task of the
target language. This way, the model can observe data from the normalization
task (albeit in other languages) and from the target language (albeit from
auxiliary tasks only), but does not see any normalization data from the target
language. In those cases where there are two datasets from the same language,
we leave out both of them from the training step. The model is similar to
previous work on zero-shot neural machine translation Johnson et al. (2016).

As before, we include only 1,000 tokens from each historical dataset for
training. In each training update, we use an equal number of samples from each
dataset/task combination, and define an epoch to consist of 1,000 samples from
each of these combinations. Since we do not want to feed the model any
normalization data from the target language during training, we cannot use
early stopping, but instead train for a fixed number of 10 epochs.

1002505001k2.5k5k10k25k50k0%10%20%30%SEATDSEADPSADP Figure 5: Error reduction for three MTL configurations by training set size, (micro-)averaged over all datasets. Dataset | Identity | Zero-shot
---|---|---
DEA | 30.16 | 40.94
DER | 43.57 | 55.92
EN | 75.47 | 56.31
ES | 72.29 | 64.39
HU | 17.81 | 20.58
IS | 47.77 | 42.95
PT | 65.18 | 67.64
SLB | 39.84 | 50.21
SLG | 85.58 | 84.99
SV | 59.24 | 50.65
Micro-Avg | 50.17 | 52.96
Table 3: Normalization accuracy on dev sets for zero-shot experiments. Best
results highlighted in bold.

30%40%50%60%70%80%90%Accuracy of auxiliary task model–10%0%10%20%30%Error
reduction of MTL modelENDEADERHUISESPTSLGSLBSVr = 0.63415959 (a) Autoencoding

10%20%30%40%50%60%70%Accuracy of auxiliary task
model–10%0%10%20%ENDEADERHUISESPTSLGSLBSVr = 0.61624035 (b) Lemmatization

Figure 6: Correlations (with 95% confidence intervals) between the performance
of an auxiliary task model applied to normalization data and the error
reduction when using this task in a multi-task learning setup.
20%30%40%50%60%70%80%Accuracy of identity baseline0%10%20%30%Error reduction
of full MTL modelENDEADERHUISESPTSLGSLBSVr = 0.81910840 Figure 7: Correlation
(with 95% confidence interval) between the identity baseline and the error
reduction of the full MTL model with all three auxiliary tasks.

#### Results

Table 3 shows the accuracy of zero-shot normalization compared to the naive
identity baseline, i.e., the accuracy obtained by simply leaving the input
word forms unchanged. The zero-shot approach improves over this baseline for
half of the datasets, sometimes by up to 12 percentage points (DER). Micro-
averaging the results shows an overall advantage for zero-shot learning.

##  4 Analysis

The experiment in Sec. 3.2 has shown that not all auxiliary tasks are equally
useful; furthermore, autoencoding is, on average, the most useful auxiliary
task of the three, closely followed by lemmatization. This gives rise to the
hypothesis that MTL mostly helps the model learn the identity mappings between
characters.

To analyze this, we feed the historical data into the auxiliary models; i.e.,
we treat them as if they were a historical text normalization model. We then
correlate their normalization accuracy with the error reduction over the
baseline of the MTL model using this auxiliary task. Figure 6(a) shows a
strong correlation for the autoencoding task, suggesting that the synergy
between autoencoding and historical text normalization is higher when the two
tasks are very related. Figure 6(b) shows the same correlation for
lemmatization.

We can also compare the error reduction from MTL to the identity baseline (cf.
Tab. 3). Figure 7 shows the correlation of these scores for the full MTL model
trained with all three auxiliary tasks.888The correlation is similar when
using longest common subsequence or Levenshtein distance instead of accuracy.
The strong correlation suggests that the regularization effect introduced by
MTL is particularly helpful with tasks where there is a strong similarity
between input and output; or, in other words, that multi-task learning
prevents the model from over-generalizing based on the training data.

1002505001k2.5k5k10k25k50k10%30%50%70%90%KnownsNo
MTLSEATDSEADPSADP1002505001k2.5k5k10k25k50kUnknowns

1002505001k2.5k5k10k25k50k10%30%50%70%90%IdentityNo
MTLSEATDSEADPSADP1002505001k2.5k5k10k25k50kNon-identity

Figure 8: Learning curves, micro-averaged over all datasets, for different
subsets of the data.

The previous correlation scores only consider the performance of models
trained on 1,000 tokens of historical data. Sec. 3.3 showed that the benefit
of MTL diminishes when the size of the historical training sets gets larger.
Figure 8 presents learning curves that have been micro-averaged over all ten
datasets, but evaluated on different subsets of the data: (a) tokens that have
been seen during training (“knowns”) or not (“unknowns”); and (b) tokens that
stay identical in the reference normalization or not. On average, the
performance of the MTL models is comparable to that of the single-task model
for known tokens and non-identity normalizations. In other words, most of the
gain from MTL comes from helping the model learn the identity mappings, which
becomes less relevant the more historical training data is available.

##  5 Related work

On previous approaches to historical text normalization, Bollmann (2019, Sec.
2) gives an extensive overview. Common approaches include rule-based
algorithms—with either manually crafted or automatically learned rules—or
distance metrics to compare historical spellings to modern lexicon forms Baron
and Rayson (2008); Bollmann (2012); Pettersson et al. (2013a). Finite-state
transducers are sometimes used to model this, but also to explicitly encode
phonological transformations which often underlie the spelling variation Porta
et al. (2013); Etxeberria et al. (2016).

Character-based statistical machine translation (CSMT) has been successfully
applied to normalization on many languages Pettersson et al. (2013b); Scherrer
and Erjavec (2016); Domingo and Casacuberta (2018); neural encoder–decoder
models with character-level input can be seen as the neural equivalent to the
statistical MT approach (Bollmann et al., 2017; Tang et al., 2018) and have
been shown to be competitive with it (Robertson and Goldwater, 2018;
Hämäläinen et al., 2018), although Bollmann (2019) suggests that they are
still inferior to CSMT in low-resource scenarios.

All these methods rely on individual word forms as their input; there is
almost no work on incorporating sentence-level context for this task (but cf.
Jurish, 2010).

#### MTL architectures

In Sec. 3.1, we explored _what to share_ between tasks in our multi-task
architecture. A common approach is to share only the first layers (e.g., Yang
et al., 2016; Peng and Dredze, 2017). Multi-task encoder–decoder models will
often keep the whole encoder and decoder task- or language-specific Dong et
al. (2015); Luong et al. (2015). Firat et al. (2016) explore the effect of
sharing the attentional component across all languages, while Anastasopoulos
and Chiang (2018) compare both parallel and cascading model configurations.

A different MTL approach is to share _all_ parts of a model, but prepend a
task-specific symbol to the input string to enable it to learn task-specific
features (cf. Sec. 3.5). Milde et al. (2017) use this approach for grapheme-
to-phoneme conversion; Kann et al. (2017) apply it to morphological paradigm
completion.

#### Auxiliary tasks for MTL

For _which auxiliary task(s) to use_ (Sec 3.2), few systematic studies exist.
Most approaches use tasks that are deemed to be related to the main task—e.g.,
combining machine translation with syntactic parsing (Kiperwasser and
Ballesteros, 2018)—and justify their choice by the effectiveness of the
resulting model. Bingel and Søgaard (2017) analyze beneficial task relations
for MTL in more detail, but only consider sequence labelling tasks. For zero-
shot learning (Sec. 3.5), we use an architecture very similar to Johnson et
al. (2016), also used for grapheme-to-phoneme mapping in Peters et al. (2017).

##  6 Conclusion

We performed an extensive evaluation of a neural encoder–decoder model on
historical text normalization, using little or even no training data for the
target language, and using multi-task learning (MTL) strategies to improve
accuracy. We found that sharing more components between main and auxiliary
tasks is usually better, and autoencoding generally provides the most benefit
for our task. Analysis showed that this is mainly because MTL helps the model
learn that most characters should stay the same, and that its beneficial
effect vanishes as the size of the training set increases. While our models
did not beat the non-neural models of Bollmann (2019), we believe our work
still provides interesting insights into the impact of MTL for low-resource
scenarios.

## Acknowledgments

We would like to thank the anonymous reviewers of this as well as previous
iterations of this paper for several helpful comments.

This research has received funding from the European Research Council under
the ERC Starting Grant LOWLANDS No. 313695. Marcel Bollmann was partly funded
from the European Union’s Horizon 2020 research and innovation programme under
the Marie Skłodowska-Curie grant agreement No. 845995.

## References

  * Anastasopoulos and Chiang (2018) Antonios Anastasopoulos and David Chiang. 2018.  Tied multitask learning for neural speech translation.  In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)_ , pages 82–91, New Orleans, Louisiana. Association for Computational Linguistics.
  * Bahdanau et al. (2014) Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014.  Neural machine translation by jointly learning to align and translate.  _CoRR_ , abs/1409.0473.
  * Baron and Rayson (2008) Alistair Baron and Paul Rayson. 2008.  VARD 2: A tool for dealing with spelling variation in historical corpora.  In _Proceedings of the Postgraduate Conference in Corpus Linguistics_.
  * Bingel and Søgaard (2017) Joachim Bingel and Anders Søgaard. 2017.  Identifying beneficial task relations for multi-task learning in deep neural networks.  In _Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers_ , pages 164–169, Valencia, Spain. Association for Computational Linguistics.
  * Bollmann (2012) Marcel Bollmann. 2012.  (Semi-)automatic normalization of historical texts using distance measures and the Norma tool.  In _Proceedings of the Second Workshop on Annotation of Corpora for Research in the Humanities (ACRH-2)_ , Lisbon, Portugal.
  * Bollmann (2018) Marcel Bollmann. 2018.  Normalization of historical texts with neural network models.  _Bochumer Linguistische Arbeitsberichte_ , 22.
  * Bollmann (2019) Marcel Bollmann. 2019.  A large-scale comparison of historical text normalization systems.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , pages 3885–3898. Association for Computational Linguistics.
  * Bollmann et al. (2017) Marcel Bollmann, Joachim Bingel, and Anders Søgaard. 2017.  Learning attention for historical text normalization by learning to pronounce.  In _Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 332–344, Vancouver, Canada. Association for Computational Linguistics.
  * Bollmann et al. (2018) Marcel Bollmann, Anders Søgaard, and Joachim Bingel. 2018.  Multi-task learning for historical text normalization: Size matters.  In _Proceedings of the Workshop on Deep Learning Approaches for Low-Resource NLP_ , pages 19–24. Association for Computational Linguistics.
  * Braud et al. (2016) Chloé Braud, Barbara Plank, and Anders Søgaard. 2016.  Multi-view and multi-task training of RST discourse parsers.  In _Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers_ , pages 1903–1913, Osaka, Japan. The COLING 2016 Organizing Committee.
  * Caruana (1993) Rich Caruana. 1993.  Multitask learning: A knowledge-based source of inductive bias.  In _Proceedings of the 10th International Conference on Machine Learning (ICML)_ , pages 41–48.
  * Deri and Knight (2016) Aliya Deri and Kevin Knight. 2016.  Grapheme-to-phoneme models for (almost) any language.  In _Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 399–408, Berlin, Germany. Association for Computational Linguistics.
  * Domingo and Casacuberta (2018) Miguel Domingo and Francisco Casacuberta. 2018.  Spelling normalization of historical documents by using a machine translation approach.  In _Proceedings of the 21st Annual Conference of the European Association for Machine Translation_ , pages 129–137.
  * Dong et al. (2015) Daxiang Dong, Hua Wu, Wei He, Dianhai Yu, and Haifeng Wang. 2015.  Multi-task learning for multiple language translation.  In _Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_ , pages 1723–1732, Beijing, China. Association for Computational Linguistics.
  * Etxeberria et al. (2016) Izaskun Etxeberria, Iñaki Alegria, Larraitz Uria, and Mans Hulden. 2016.  Evaluating the noisy channel model for the normalization of historical texts: Basque, Spanish and Slovene.  In _Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC 2016)_ , pages 1064–1069, Portorož, Slovenia. European Language Resources Association (ELRA).
  * Firat et al. (2016) Orhan Firat, Kyunghyun Cho, and Yoshua Bengio. 2016.  Multi-way, multilingual neural machine translation with a shared attention mechanism.  _CoRR_ , abs/1601.01073.
  * Hämäläinen et al. (2018) Mika Hämäläinen, Tanja Säily, Jack Rueter, Jörg Tiedemann, and Eetu Mäkelä. 2018.  Normalizing early english letters to present-day english spelling.  In _Proceedings of the Second Joint SIGHUM Workshop on Computational Linguistics for Cultural Heritage, Social Sciences, Humanities and Literature_ , pages 87–96. Association for Computational Linguistics.
  * Johnson et al. (2016) Melvin Johnson, Mike Schuster, Quoc V. Le, Maxim Krikun, Yonghui Wu, Zhifeng Chen, Nikhil Thorat, Fernanda B. Viégas, Martin Wattenberg, Greg Corrado, Macduff Hughes, and Jeffrey Dean. 2016.  Google’s multilingual neural machine translation system: Enabling zero-shot translation.  _CoRR_ , abs/1611.04558.
  * Jurish (2010) Bryan Jurish. 2010.  More than words: using token context to improve canonicalization of historical German.  _Journal for Language Technology and Computational Linguistics_ , 25(1):23–39.
  * Kann et al. (2017) Katharina Kann, Ryan Cotterell, and Hinrich Schütze. 2017.  One-shot neural cross-lingual transfer for paradigm completion.  In _Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 1993–2003, Vancouver, Canada. Association for Computational Linguistics.
  * Kimura et al. (2018) Akisato Kimura, Zoubin Ghahramani, Koh Takeuchi, Tomoharu Iwata, and Naonori Ueda. 2018.  Few-shot learning of neural networks from scratch by pseudo example optimization.  _arXiv e-prints_ , abs/1802.03039.
  * Kiperwasser and Ballesteros (2018) Eliyahu Kiperwasser and Miguel Ballesteros. 2018.  Scheduled multi-task learning: From syntax to translation.  _Transactions of the Association for Computational Linguistics_ , 6:225–240.
  * Kirov et al. (2018) Christo Kirov, Ryan Cotterell, John Sylak-Glassman, Géraldine Walther, Ekaterina Vylomova, Patrick Xia, Manaal Faruqui, Arya D. McCarthy, Sandra Kübler, David Yarowsky, Jason Eisner, and Mans Hulden. 2018.  UniMorph 2.0: Universal morphology.  In _Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018)_ , pages 1868–1873. European Language Resources Association (ELRA).
  * Korchagina (2017) Natalia Korchagina. 2017.  Normalizing medieval German texts: from rules to deep learning.  In _Proceedings of the NoDaLiDa 2017 Workshop on Processing Historical Language_ , pages 12–17, Gothenburg. Linköping University Electronic Press.
  * Luong et al. (2015) Minh-Thang Luong, Quoc V. Le, Ilya Sutskever, Oriol Vinyals, and Lukasz Kaiser. 2015\.  Multi-task sequence to sequence learning.  _CoRR_ , abs/1511.06114.
  * Milde et al. (2017) Benjamin Milde, Christoph Schmidt, and Joachim Köhler. 2017.  Multitask sequence-to-sequence models for grapheme-to-phoneme conversion.  In _Proceedings of Interspeech 2017_ , pages 2536–2540.
  * Neubig et al. (2018) Graham Neubig, Matthias Sperber, Xinyi Wang, Matthieu Felix, Austin Matthews, Sarguna Padmanabhan, Ye Qi, Devendra Singh Sachan, Philip Arthur, Pierre Godard, John Hewitt, Rachid Riad, and Liming Wang. 2018.  XNMT: The extensible neural machine translation toolkit.  In _Conference of the Association for Machine Translation in the Americas (AMTA) Open Source Software Showcase_ , Boston.
  * Peng and Dredze (2017) Nanyun Peng and Mark Dredze. 2017.  Multi-task domain adaptation for sequence tagging.  In _Proceedings of the 2nd Workshop on Representation Learning for NLP_ , pages 91–100, Vancouver, Canada. Association for Computational Linguistics.
  * Peters et al. (2017) Ben Peters, Jon Dehdari, and Josef van Genabith. 2017.  Massively multilingual neural grapheme-to-phoneme conversion.  In _Proceedings of the First Workshop on Building Linguistically Generalizable NLP Systems_ , pages 19–26, Copenhagen, Denmark. Association for Computational Linguistics.
  * Pettersson et al. (2013a) Eva Pettersson, Beáta Megyesi, and Joakim Nivre. 2013a.  Normalisation of historical text using context-sensitive weighted Levenshtein distance and compound splitting.  In _Proceedings of the 19th Nordic Conference of Computational Linguistics (NODALIDA 2013)_ , pages 163–179, Oslo, Norway. Linköping University Electronic Press.
  * Pettersson et al. (2013b) Eva Pettersson, Beáta Megyesi, and Jörg Tiedemann. 2013b.  An SMT approach to automatic annotation of historical text.  In _Proceedings of the Workshop on Computational Historical Linguistics at NODALIDA 2013_ , NEALT Proceedings Series 18, pages 54–69. Linköping University Electronic Press.
  * Piotrowski (2012) Michael Piotrowski. 2012.  _Natural Language Processing for Historical Texts_.  Number 17 in Synthesis Lectures on Human Language Technologies. Morgan & Claypool.
  * Porta et al. (2013) Jordi Porta, José-Luis Sancho, and Javier Gómez. 2013.  Edit transducers for spelling variation in Old Spanish.  In _Proceedings of the Workshop on Computational Historical Linguistics at NODALIDA 2013_ , NEALT Proceedings Series 18, pages 70–79. Linköping University Electronic Press.
  * Robertson and Goldwater (2018) Alexander Robertson and Sharon Goldwater. 2018.  Evaluating historical text normalization systems: How well do they generalize? In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers)_ , pages 720–725. Association for Computational Linguistics.
  * Scherrer and Erjavec (2016) Yves Scherrer and Tomaž Erjavec. 2016.  Modernising historical Slovene words.  _Natural Language Engineering_ , 22(6):881–905.
  * Tang et al. (2018) Gongbo Tang, Fabienne Cap, Eva Pettersson, and Joakim Nivre. 2018.  An evaluation of neural machine translation models on historical spelling normalization.  In _Proceedings of the 27th International Conference on Computational Linguistics_ , pages 1320–1331, Santa Fe, New Mexico, USA. Association for Computational Linguistics.
  * Yang et al. (2016) Zhilin Yang, Ruslan Salakhutdinov, and William W. Cohen. 2016.  Multi-task cross-lingual sequence tagging from scratch.  _CoRR_ , abs/1603.06270.


# A Multiwavelength study of the ongoing Changing-look AGN AT2021fxu

Yukta Ajay  Indian Institute of Science Education and Research (IISER),
Tirupati
Tirupati 517 507, Andhra Pradesh, India
Muryel Guolo  Department of Physics and Astronomy, Johns Hopkins University
3400 N. Charles St., Baltimore MD 21218, USA  Dheeraj Pasham  MIT Kavli
Institute for Astrophysics and Space Research
Cambridge, MA 02139, USA

###### Abstract

We present multiwavelength analyses of an active optical transient AT2021fxu
which shows the appearance of previously absent broad emission lines in a
recent optical spectrum, suggesting a Changing-look (CL) Active Galactic
Nucleus (AGN). During the spectral transition, the brightness in the
individual photometric bands increased up to $\approx$0.6 in the optical bands
and up to $\approx$1.1 magnitudes in the UV bands. The brightening was
accompanied by a blueward shift of the optical spectrum. AT2021fxu shows high
X-ray (0.3-10 keV) flux variability before and after the outburst, with the
average X-ray flux increasing by a factor of $\approx$2 post-outburst.
However, the X-ray spectral shape remains roughly the same, with no
significant change in the line-of-sight column density. AT2021fxu’s overall
properties are consistent with an accretion-rate-driven transition from a
Type-II to a Type-1 AGN.

##  1 Introduction

Accreting active galactic nuclei (AGN) can be identified by their optical
emission line features such as broad and/or narrow high ionization lines.
Doppler broadened Broad Emission Lines (BELs) originate from the
gravitationally-bound broad-line region (BLR). In contrast, the narrow-line
regions (NLRs) can extend beyond a few hundred parsecs, although the ionizing
AGN continuum considerably influences the NLR. AGN that show both the broad-
and the narrow-lines are classified as Type-I AGN, while those that do not
show BELs are called as Type-II AGN. The AGN unification model (Antonucci,
1993; Urry & Padovani, 1995) attributes differences in type to the viewing
angle towards an axisymmetric, parsec scale toroidal structure (the so-called
“torus”). This structure can obstruct our direct view of the AGN central
engine along our line-of sight, and block our view of the BLR in Type-II AGN.

A small subset of AGN, referred to as changing-look (CL) AGN have been
reported in recent years (e.g., LaMassa et al., 2015; Guolo et al., 2021;
Zeltyn et al., 2022). These show strong spectral variability in their BELs and
continuum, and can therefore appear to transition between the AGN classes over
months to several years timescale. CL-AGN transitions have been explained
mainly by variable obscuration or a strong variation in the accretion rate
(see Ricci & Trakhtenbrot, 2022, for a recent review).

AT2021fxu/ZTF21aalxxzn/Gaia22dgm is an optical transient at a redshift of
0.1097 identified by the Zwicky Transient Facility (ZTF) on 13 February 2021
(De, 2021). Prior to the optical outburst, it was cataloged as an AGN (2MASX
J13471874+0210579/SWIFT J1347.7+0212) by the BAT AGN Spectroscopic Survey
(BASS, Koss et al., 2017). Here we present analysis of AT2021fxu’s
multiwavelength data taken over a temporal baseline of over two decades.

##  2 Data and Analysis

###  2.1 Properties of optical spectra

In Fig.1a, we show AT2021fxu’s optical spectra before and after the optical
transition. The archival optical spectrum from Sloan Digital Sky Survey (SDSS)
in 2001 shows a continuum dominated by stellar population with no clear BELs.
Although a specific AGN class is hard to pin down due to contamination of sky
lines in the H$\alpha$ region, the general features are consistent with either
a low-ionization nuclear emission-line region (LINER, Heckman, 1980) or a
Type-II AGN. In the optical spectrum captured by the Faulkes Telescope North
in 2021 (Fig.1c), a bright blue continuum and BELs typical for a Type-I AGN
are evident. This indicates a change in the spectral class of the AGN from
Type-II to Type-I.

###  2.2 Optical and UV photometry

Swift’s Ultra-Violet/Optical Telescope (UVOT) performed observations of the
source before and after the optical outburst. The data were processed with the
uvotsource command within the HEASoft v6.29c package. We also performed point
spread function (PSF) photometry on all publicly available ZTF data at the
location of AT2021fxu using the ZTF forced-photometry service (Masci et al.,
2019) in the g- and r-bands. The measured magnitudes have been corrected for
Galactic extinction using E(B  $-$ V) = 0.021 from Schlafly & Finkbeiner
(2011).

Fig.1c shows the optical lightcurves since Decmeber 2020 ($\sim$MJD 59200),
which is nearly flat until July 2021 ($\sim$MJD 59420). Additionally, the
optical magnitudes recorded by SDSS in May 2000 (18.24$\pm$0.01 in the
optical-g and 17.33$\pm$0.01 in the optical-r, not shown here) are consistent
with the average optical magnitudes between March 2018 to July 2021
(18.22$\pm$0.08 (g-band) and 17.41$\pm$0.05 (r-band)). This suggests that
AT2021fxu’s optical spectrum did not change between 2000 and 2021. Further,
the optical brightness of the source increased by $\approx$0.6 and
$\approx$0.34 magnitudes in the g- and r-bands, respectively, during the
optical outburst between 1 April 2022 and 27 May 2022. This was followed by an
optical decline of $\approx$0.34 magnitudes in the g-band and $\approx$0.2
magnitudes in the r-band over the next two months, until 19 July 2022.

Comparing the average UV magnitudes from about a decade prior to the outburst
with the values recorded during the optical decline, the magnitudes have
changed from $\approx$18.3 to $\approx$17.7 in the U-band, from $\approx$19.3
to $\approx$18.2 in the UVW1 band, and from $\approx$19.6 to $\approx$18.4 in
the UVW2 band, indicating that the brightness in all the UV bands increased
post-outburst.

There is a shift to a bluer spectral energy distribution (SED), with the
optical (g  $-$ r) color decreasing from 0.80$\pm$0.06 to 0.46$\pm$0.04,
before and after the optical outburst, respectively. At the optical peak, the
(g-r) difference is minimum, and is $\approx$0.38. Similarly, the average UV
(W1  $-$ U) magnitude difference drops from 1.02$\pm$0.07 to 0.5$\pm$0.01, and
the average UV (W2  $-$ U) magnitude decreases from 1.28$\pm$0.05 to
0.74$\pm$0.02, again indicative of a bluer SED.

###  2.3 X-ray spectral analysis

The archival Swift X-Ray Telescope (XRT) observations between 4 December 2011
to 13 August 2012 suggest that the source was highly variable in X-rays, with
the 0.3-10 keV count rate increasing by a factor of 10 in less than 2 months.
We fit the combined 0.3-10 keV XRT spectrum using the
TBabs*zTBabs*zashift(powerlaw) model in XSPEC (Arnaud, 1996) with the galactic
column density ($N_{H}$) fixed at 1.5$\times$1020 cm-2 (HI4PI Collaboration et
al., 2016), and the host galaxy $N_{H}$ left as a free parameter. The best-fit
powerlaw index and the neutral column density were $1.71_{-0.19}^{+0.42}$ and
$<$$8\times 10^{20}$ cm-2, respectively. The observed luminosity and flux at
these times were $2.72_{-0.64}^{+0.48}\times 10^{43}$ ergs s-1 and $0.91\pm
0.20\times 10^{-12}$ ergs cm-2 s-1, respectively. All the errorbars reported
in this work represent 90% confidence values.

Swift/XRT observations between August 2021 and August 2022 coincided with the
recent optical outburst. We fit the combined XRT spectrum during the optical
decline (mean countrate of $0.038\pm 0.002$ cps) with the same powerlaw model
as before and obtained a best-fit photon index and column denstiy values of
$1.83_{-0.16}^{+0.32}$, and $<7.5\times 10^{20}$ cm-2, respectively. The
observed 0.3-10 keV luminosity and flux of the post-outburst XRT spectrum were
found to be $6.54_{-1.37}^{+0.76}\times 10^{43}$ ergs s-1 and
$2.17_{-0.35}^{+0.31}\times 10^{-12}$ ergs cm-2 s-1, respectively.

Following XRT’s detection, the Neutron Star Interior Composition
Explorer(NICER) performed multiple exposures between 28 June 2022 and 4 July
2022 during the optical decline. The source was variable during this time
(Fig.1d) with a mean background-subtracted count rate of $1.33\pm 0.02$ cps in
the 0.3-2 keV band. We fit a combined NICER 0.3-2.0 keV spectrum using the
powerlaw model described above. The best-fit power law index and the host
$N_{H}$ column upper limits are $2.01^{+0.13}_{-0.06}$ and $1.5\times 10^{20}$
cm-2, respectively, consistent with the post-outburst values from Swift/XRT.
The observed average luminosity and flux in the 0.3-10 keV energies from NICER
are $7.98_{-0.82}^{+0.73}\times 10^{43}$ ergs s-1 and
$2.60_{-0.30}^{+0.25}\times 10^{-12}$ ergs cm-2 s-1, respectively.

The X-ray lightcurves from Swift/XRT and NICER (Fig.1d) show a doubling in the
X-ray flux post-outburst and high flux variability with almost no change in
the X-ray spectral shape over time. Furthermore, the powerlaw index and the
host column density values do not show considerable variation between 2011-12
and 2021-22.

Figure 1: (a) Optical spectrum of the host galaxy in 2001 (black), and the
optical spectrum post-outburst in May 2022 (blue) showing the appearance of
BELs and a strong blue continuum; (b) Unfolded spectra and models for
Swift/XRT data between 2011-2012 (before the optical outburst), and Swift/XRT
and NICER between June-August 2022 (post-outburst); (c) Optical and UV light
curves from Swift/UVOT; (d) Swift/XRT and NICER X-ray light curves, zoomed in
plot shows short-term X-ray variability.

##  3 Discussion and Conclusion

We have analyzed AT2021fxu’s optical, UV and X-ray data over approximately two
decades. The recent optical spectrum clearly indicates the appearance of BELs,
suggesting that AT2021fxu underwent a spectral transition from a Type-II to a
Type-I AGN. Since the optical g- and r- magnitudes have remained constant
between May 2000 to July 2021, it is clear that optical spectrum remained the
same during these years, and the recent optical outburst in April-May 2022
marks the onset of the CL transition in this AGN.

X-ray spectral analysis shows that there is minimal variation in the column
density over the observed time. This eliminates the possibility of variable
obscuration driving the spectral transition, implying instead that the CL
transition is due to a variable accretion rate. This mechanism of transition
due to increased accretion is further confirmed by an increase in the flux in
all energy bands and a shift of the SED toward bluer wavelengths.

In summary, we show that AT2021fxu is an accretion-drive CL transition in the
AGN 2MASX J13471874+021057/SWIFT J1347.7+0212, where the increase in the
ionization flux is able to produce strong broad Balmer lines that drives the
change in spectral classification.

Interestingly, during the writing of this note, the source showed further
optical brightening by $\approx$0.12 magnitudes. We have initiated follow-up
observations on Swift and NICER to further analyze this new outburst, and we
encourage observations in other wavelengths to study the ongoing CL
transition.

## References

  * Ricci & Trakhtenbrot (2022) Ricci, C. & Trakhtenbrot, B. 2022, arXiv:2211.05132
  * Antonucci (1993) Antonucci, R. 1993, ARA&A, 31, 473. doi:10.1146/annurev.aa.31.090193.002353
  * Urry & Padovani (1995) Urry, C. M. & Padovani, P. 1995, PASP, 107, 803. doi:10.1086/133630
  * Heckman (1980) Heckman, T. M. 1980, A&A, 87, 152
  * Koss et al. (2017) Koss, M., Trakhtenbrot, B., Ricci, C., et al. 2017, ApJ, 850, 74. doi:10.3847/1538-4357/aa8ec9
  * De (2021) De, K. 2021, Transient Name Server Discovery Report, 2021-792
  * Arnaud (1996) Arnaud, K. A. 1996, Astronomical Data Analysis Software and Systems V, 101, 17
  * HI4PI Collaboration et al. (2016) HI4PI Collaboration, Ben Bekhti, N., Flöer, L., et al. 2016, A&A, 594, A116. doi:10.1051/0004-6361/201629178
  * Guolo et al. (2021) Guolo, M., Ruschel-Dutra, D., Grupe, D., et al. 2021, MNRAS, 508, 144. doi:10.1093/mnras/stab2550
  * LaMassa et al. (2015) LaMassa, S. M., Cales, S., Moran, E. C., et al. 2015, ApJ, 800, 144. doi:10.1088/0004-637X/800/2/144
  * Zeltyn et al. (2022) Zeltyn, G., Trakhtenbrot, B., Eracleous, M., et al. 2022, ApJ, 939, L16. doi:10.3847/2041-8213/ac9a47
  * Schlafly & Finkbeiner (2011) Schlafly, E. F. & Finkbeiner, D. P. 2011, ApJ, 737, 103. doi:10.1088/0004-637X/737/2/103
  * Masci et al. (2019) Masci, F. J., Laher, R. R., Rusholme, B., et al. 2019, PASP, 131, 018003. doi:10.1088/1538-3873/aae8ac