# Improving Span Representation for Domain-adapted Coreference Resolution

Nupoor Gandhi†, Anjalie Field†, Yulia Tsvetkov §
Carnegie Mellon University †, University of Washington §
{nmgandhi,anjalief}<EMAIL_ADDRESS>
###### Abstract

Recent work has shown fine-tuning neural coreference models can produce strong
performance when adapting to different domains. However, at the same time,
this can require a large amount of annotated target examples. In this work, we
focus on supervised domain adaptation for clinical notes, proposing the use of
concept knowledge to more efficiently adapt coreference models to a new
domain. We develop methods to improve the span representations via (1) a
retrofitting loss to incentivize span representations to satisfy a knowledge-
based distance function and (2) a scaffolding loss to guide the recovery of
knowledge from the span representation. By integrating these losses, our model
is able to improve our baseline precision and F-1 score. In particular, we
show that incorporating knowledge with end-to-end coreference models results
in better performance on the most challenging, domain-specific spans111Code
publicly available at https://github.com/nupoorgandhi/i2b2-coref-public.

##  1 Introduction

Recent work has achieved high performance on coreference resolution in
standard benchmark datasets like OntoNotes Kirstain et al. (2021); Joshi et
al. (2020); Weischedel et al. (2013). However, in many real world settings
where coreference resolution would be valuable, text differs greatly from
these standard datasets. For example, coreference resolution over clinical
notes can enable tracking a patient’s progress and treatment history. However,
clinical notes contain acronyms and medical terminology. Annotating new
training data for every domain of interest is expensive and time-consuming,
and coreference models trained on existing benchmark datasets perform worse on
other domains Srivastava et al. (2020); Xu and Choi (2020); Joshi et al.
(2020). In this work, we develop a domain-adaptation model for coreference
resolution that requires only a small number of target training examples and
target domain knowledge.

Our primary approach involves incorporating domain-knowledge into the span
representations learned by an end-to-end neural system Lee et al. (2017). A
span representation is a vector representation of a contiguous set of tokens.
When determining if a given mention refers to an antecedent, span
representations are used by the model to (1.) select a set of candidate
mentions and (2.) select an antecedent from the candidates for the given
mention. Thus, a high-quality span representation encodes the semantic meaning
of the span tokens and their local context. Joshi et al. (2020) introduced
SpanBERT, a pre-training method extending BERT, designed to improve
performance on span-selection tasks that involves masking contiguous spans
rather than tokens. Span representations are derived by concatenating the pre-
trained transformer outputs at the boundary tokens with an attention-weighted
vector over the span tokens. These representations are fed into a coreference
resolution model, thus integrating SpanBERT into an end-to-end coreference
resolution system.

SpanBERT is able to capture coreference structure implicitly in rich span
representations. The expressiveness of the SpanBERT representation is apparent
from extrinsic coreference performance, but also through probing tasks that
have shown that span representations can capture headedness, coreference arcs,
and other linguistic features of coreference Kahardipraja et al. (2020). The
best coreference performance and span representations are obtained by training
the end-to-end model with SpanBERT using labeled coreference data.

When adapting a coreference model to a new domain, fine-tuning or continued
training can greatly improve performance, but this approach can be
computationally expensive and requires a large amount of labelled documents
from the target domain Gururangan et al. (2020); Xia and Van Durme (2021).
Neural models have also been criticized for largely relying on shallow
heuristics in the text, suggesting this data-driven learning method requires
many target examples to learn a new target distribution Lu and Ng (2020);
Rosenman et al. (2020).

The presence of out-of-vocabulary words in a new domain can create additional
challenges. SpanBERT uses wordpiece tokenization, which can lead to misleading
meaning representation for spans when a single wordpiece belongs to spans with
different meanings Joshi et al. (2020); Poerner et al. (2020b). Consider for
example, the spans euthmyia and dementia, both of which are common medical
terminology but out-of-vocabulary words for SpanBERT, which tokenizes them to
contain a common wordpiece: “##ia”. As described by Poerner et al. (2020b),
this can lead to a coreference model incorrectly predicting the spans
coreferent, since the suffix “##ia” is commonly associated with diseases. A
coreference model could correct this by learning a more meaningful
representation for the prefix tokens and downweighting the suffix “##ia”
token, but this would take many target domain training examples.

Instead, we propose a more efficient method for integrating domain-specific
knowledge into SpanBERT-based span representations, which requires only a
small number of target training samples and leverages domain-specific concept
knowledge.

We take a set of spans with some similarity in meaning to be a concept, and we
use concepts of varying granularity (e.g., problem, headache).

First, we introduce a retrofitting loss (§2.4) which guides the representation
learning of span pairs to satisfy a knowledge-based distance function. This
distance function reflects pairwise span relationships from our concept
knowledge. As a result, we are able to align the target domain coreference
structure encoded in the span representation with the global meanings of the
spans. This allows the end-to-end model to more efficiently build more
meaningful span representations for the target domain.

We also introduce an auxiliary scaffolding loss (§2.5) for a concept
prediction task in order to ensure that knowledge relevant to the coreference
task can be recovered from the span representation. This usage of an auxiliary
task to produce a useful inductive bias was introduced in Swayamdipta et al.
(2018) to add a syntactic labeling loss for coreference resolution since
syntactic constituents are often coreferent. Spans belonging to the same
concept within our concept knowledge usually corefer, so we generalize this
technique to a broad, knowledge-based lexicon in our domain adaptation
setting. While our retrofitting loss integrates relative knowledge into the
span representation, we are able to supplement the span representation with
global meaning using the scaffolding loss.

To evaluate our models, we take OntoNotes as our source domain and the i2b2/VA
corpus of clinical notes as our target domain. We train our model on the
source domain and 200 examples from the target domain, we evaluate model
performance on the target domain. Our knowledge concepts are from the i2b2/VA
dataset and the UMLS Metathesaurus.

First, we describe our methodology and introduce our retrofitting and
scaffolding loss functions in §2. Then, we describe our experiments to
quantify model performance on our target domain in §3, and finally we
demonstrate the performance improvement over our baseline and on rare/OOV
spans §4.

The main contribution of this work is in developing methods to integrate
concept knowledge into coreference resolution systems to improve domain
adaptation. We outperform our baseline primarily by improving precision, and
in doing so, we demonstrate the utility of a set of knowledge concepts for
adapting span representations to a new domain. We show that our model’s
performance does not deteriorate on highly domain-specific spans containing
OOD frequently subdivided vocabulary.

##  2 Model

Figure 1: Span representations produced by SpanBERT for mentions “euthymia”
and “dementia” are fed into a co-reference resolution model. The baseline
Joshi et al. (2020), which uses a single coreference loss, CL, produces
similar span representations for “euthymia” and “dementia” (top). When we
incorporate knowledge concepts, $C_{1}^{k}=\\{\text{euthymia,
pyrexia},\ldots\\},C^{k}_{2}=\\{\text{dementia, polymyalgia}\\},\ldots$, into
the span representation using our proposed losses SL and RL, the span
representations for “euthmyia” and “dementia” are further apart (bottom),
accurately reflecting that “euthmyia” is a symptom, while “dementia”is a
disease, even though they share a wordpiece “ia”.

The objective of coreference resolution is to identify a set of coreference
clusters from a document containing entities, where each coreference cluster
contains mentions referencing a single entity. The set of mention candidates
are referred to as spans, where a span is any contiguous set of tokens in the
document. We can use the coreference clusters $\\{C_{1},C_{2},\ldots,C_{m}\\}$
to define an unconnected graph where the set of spans are the vertices
$V=\\{s_{1},s_{2},\ldots,s_{n}\\}$, and there are edges only between spans
that belong to the same cluster. First, we describe the basic model setup for
a neural coreference resolution system, and we then describe our proposed
approach, that involves two new losses.

###  2.1 End-to-end Coreference Model

For the set of possible spans, we first produce span representations. The span
representation is learned as a part of the neural end-to-end framework
introduced in Lee et al. (2017). Given span representations, each span
representation $\bm{h}_{i}$ is assigned a unary mention score. The mention
score reflects the likelihood that a given span is in fact a mention. This
score is used to obtain a set of candidate mentions. Each span pair
$\bm{h}_{i},\bm{h}_{j}$ is assigned a pairwise antecedent score reflecting the
likelihood that $\bm{h}_{i}$ is the antecedent of $\bm{h}_{j}$. For an
arbitrary span pair, the overall score is composed of the antecedent score and
the mention scores for each span. The scoring functions are learned using
standard feed-forward neural networks, allowing us to derive a distribution
over all possible antecedents for a given span $x$.

$\displaystyle\textit{P}(y)=\frac{e^{\textit{s}(x,y)}}{\sum_{y^{\prime}\in\textit{Y}}e^{\textit{s}(x,y^{\prime})}}$

where s is the scoring function as defined in Joshi et al. (2020). We maximize
the likelihood of the correct antecedents from the set of gold mentions,
giving us a coreference loss $(\textbf{CL})$:

$\displaystyle\textbf{CL}=\log\prod_{i=1}^{\textit{N}}\sum_{\hat{y}\in\mathcal{Y}(i)\cap\text{GOLD}(i)}\textit{P}(\hat{y})$

$\text{GOLD}(i)$ denotes the set of spans in the gold cluster containing span
$i$. Our baseline span representation is produced using SpanBERT with the
single objective of minimizing the coreference loss (CL).

###  2.2 SpanBERT representation

SpanBERT is a pre-training method extending BERT that masks contiguous spans
and also trains the span boundary representations to predict the masked span.
The span representation $\bm{h}_{i}$ is the concatenation of the two SpanBERT
transformer states of the span endpoints (first and last word pieces)
$\bf{x}_{\text{START}(i)},\bf{x}_{\text{END}(i)}$ and an attention vector
$\bf{\hat{x}_{i}}$ computed over all the word pieces in the span Joshi et al.
(2019, 2020).

$\displaystyle\bm{h}_{i}=\left[\bf{x}_{\text{START}(i)},\bf{x}_{\text{END}(i)},\bf{\hat{x}_{i}},\phi(i)\right]$

The attention vector $\bf{\hat{x}_{i}}$ is intended to best represent the
internal span itself (e.g. head word), whereas the endpoints better represent
the context Lee et al. (2017). This suggests that the $\bf{\hat{x}_{i}}$
component of the overall span representation is the most natural part of the
span to align with global, non-contextual knowledge.

###  2.3 Integrating Knowledge into Span Representation

We aim to create a span representation such that knowledge can be easily
aligned with the coreference structure. Then, we can learn a span
representation geometry such that connected spans are close and disconnected
spans are far. In constructing such a vector space, we gain some flexibility
to integrate any type of knowledge that shares the same structure as the
coreference clusters. We can represent knowledge sources with concept clusters
$\\{C_{1}^{k},C_{2}^{k},\ldots C_{m}^{k}\\}$ to resemble coreference clusters,
so that we can impose that the coreference cluster graph is consistent with
the concept cluster graph via two additional losses.

We propose two complementary approaches to integrating knowledge. A pairwise
retrofitting loss is intended to encode relative knowledge and a unary
“scaffold” loss is intended to encode global knowledge into the span
representation.

###  2.4 Retrofitting with Concept Knowledge

We want the span representation $\bf{\hat{x}_{i}}$ to be close to coreferring
spans $\bf{\hat{x}_{j}}$. Similarly span representations belonging to the same
cluster should be close. Similar to Faruqui et al. (2015), we enforce this
objective by defining a custom distance function. But unlike Faruqui et al.
(2015), our custom distance function is task specific and instead of using a
lexicon, we are using broad concepts.

Concept Knowledge Distance metric We define our distance function to be
composed of two elements: coreference information and concept knowledge.

$\displaystyle\textit{d}_{\textit{T}}(s_{i},s_{j})$
$\displaystyle=\alpha_{c}d_{c}(s_{i},s_{j})+\alpha_{k}d_{k}(s_{i},s_{j})$

Variables $\alpha_{c},\alpha_{k}$ each denote weights that we tune, and $T$
references the document from which valid span pairs are passed into the
function.

Coreference information To capture the coreference graph, we recreate the
distance between span pairs that do not corefer $(d_{c})$. Note that this
distance does not discriminate between span pairs that belong to separate
coreference clusters and span pairs where one span does not belong to any
coreference cluster at all. We define $d_{c}(s_{i},s_{j})=1$ if the spans do
not corefer, otherwise $0$.

Knowledge From our knowledge, we can obtain concepts, or sets of spans with
some level of similarity of non-contextual meaning:
$\\{C_{1}^{k},C_{2}^{k},\ldots,C_{m}^{k}\\}$. Coreferent spans refer to the
same entity, and as a result, the concept type (e.g. person) must be
consistent for any pair of coreferent spans. Thus, given a set of concepts, we
want spans belonging to the same concept to have similar representations.
Accordingly, we define $d_{k}(s_{i},s_{j})=0$ if both spans belong to the same
concept type, otherwise $1$.

Retrofitting Loss (RL) We want to create a span representation with a geometry
defined by our custom distance function. We can optimize the end-to-end model
to satisfy a loss applied to the $\bf{\hat{x}_{i}}$ component of the span
representation:

$\displaystyle\textbf{RL}=\sum_{\ell}\frac{1}{|r^{\ell}|}\sum_{i,j}|d_{T^{\ell}}(s_{i}^{\ell},s_{j}^{\ell})-d(\hat{\bm{x}}_{i}^{\ell},\hat{\bm{x}}_{j}^{\ell})|$

Here $|r^{\ell}|$ denotes the number of span pairs internal to one document,
which we use to normalize, $\ell$ identifies the document that span
$s_{i},s_{j}$ belongs to, and the function $d$ is cosine distance.

###  2.5 Concept Identification as an Auxiliary Task

We introduce a concept identification auxiliary task to guide the model to
construct a span representation from which the concept can be recovered.
Swayamdipta et al. (2018) introduces the notion of a “scaffold” or auxiliary
supervised loss function that is related to the primary task. Since
coreferring spans nearly always belong to the same concept in our concept
knowledge, concepts are a good choice for a scaffold. By sharing SpanBERT
parameters optimizing for the scaffold loss and the overall coreference loss,
we are able to encode the concept type in the span representation.

Auxiliary Scaffolding Loss (SL) Following from Swayamdipta et al. (2018), we
assign a distribution over the set of concepts

$p(s_{i}\in
C\mid\bf{\hat{x}_{i}})=\text{softmax}_{c}\bm{w}_{c}\cdot\bf{\hat{x}_{i}}$

where $\bm{w}_{c}$ is a parameter vector associated with a concept $C$. This
gives us the loss:

$\textbf{SL}=\sum_{\ell}\frac{1}{|r^{\ell}|}\sum_{i}\log
p\left(c_{i}^{k}\mid\bf{\hat{x}_{i}}\right)$

where $c_{i}^{k}$ is the concept associated with span $s_{i}$.

Finally, we optimize a summation of these losses, weighting each loss with a
hyperparameter that we tune.

$\displaystyle\mathcal{L}=\beta_{1}\textbf{CL}+\beta_{2}\textbf{RL}+\beta_{3}\textbf{SL}$

##  3 Experiments

###  3.1 Datasets

Our target corpus is a medical notes dataset, released as a part of the
i2b2/VA Shared-Task and Workshop in 2011 Uzuner et al. (2011). The dataset
contains 251 train documents, 51 of which we have randomly selected for
development and 173 test documents. Our dev set is used to select some model
parameters (e.g., loss function weights $\beta_{1},\beta_{2},\beta_{3}$). The
average length of these documents is 962.62 tokens with average coreference
chain containing 4.48 spans. For our source domain corpus, we use OntoNotes,
with documents on average half as long as the clinical notes and similar
average chain length 4.21.

####  3.1.1 Knowledge Lexicons

i2b2 Concepts (i2b2): In addition to coreference chains, the i2b2/VA dataset
includes broad concept labels for spans. We focus on four concepts: person
(e.g. the patient, Dr. X), treatment (e.g. abdominal hysterectomy, the
procedure), problem (e.g. coronary artery disease, slurred speech), test (e.g.
MRI, echocardiogram). The i2b2 dataset annotates coreference chains s.t.
corefering spans must belong to the same concept. In Table 1, we report how
these i2b2 concepts are distributed among the coreference chains.

i2b2 Concept | Avg. Chain length | # of Chains
---|---|---
Train | Test | Train | Test
Problem | 2.96 | 2.9 | 1704 | 1186
Test | 2.31 | 2.51 | 568 | 360
Person | 14.16 | 12.54 | 754 | 571
Treatment | 2.66 | 2.63 | 1262 | 1063

Table 1: Breakdown of i2b2 concepts in coreference chains

Unified Medical Language System Concepts (UMLS): The Unified Medical Language
System (UMLS) defines concept unique identifier (CUI) codes in the UMLS
Metathesuarus tool Bodenreider (2004). Each UMLS concept links synonymous
spans, so the UMLS concepts are much more fine-grained than those defined in
the i2b2 dataset. For example, a CUI for headache would map to {headaches,
cranial pain, head pain cephalgia}. We used string match to assign a UMLS
concept to spans in the training set. This resulted in the identification of
3,500 unique CUI’s for the spans. We also experimented with using a partial
string match to assign UMLS concepts to spans which we refer to as “UMLS
overlap”.

###  3.2 Baseline

For our supervised domain adaptation approach, we use a familiar approach of
training a model on a source domain and tuning this model on a target domain.
We take the current state of the art end-to-end coreference model from Joshi
et al. (2020) for our baseline.

First, we train this SpanBERT-based end-to-end model on OntoNotes using the
hyperparameters from Joshi et al. (2020). Then, we continue training this
model using the target i2b2 training dataset. Continued training has been
shown to be effective for coreference resolution in out-of-domain settings Xia
and Van Durme (2021).

###  3.3 Model Setup

In order to improve the SpanBERT-based span representation, we introduce the
i2b2 and UMLS concept knowledge in two ways: we retrofit the span
representation to the concept knowledge (§2.4) and introduce an auxiliary task
of concept identification as a scaffold (§2.5).

In our implementation, we add the corresponding retrofitting loss (RL) and
scaffolding loss (SL) from these two objectives to the coreference loss (CL)
to produce an overall loss which we optimize for. Aside from the difference in
the loss function, the training process for our model resembles that of our
baseline. We train our model first with the source domain OntoNotes and then
continue training on the i2b2 dataset.

For the retrofitting loss, we experiment with the knowledge lexicons i2b2 and
UMLS individually and together. Recall that the knowledge lexicon distance
metric relies on two main components: coreference clusters
$C_{1},C_{2},\ldots$ and knowledge clusters $C_{1}^{k},C_{2}^{k},\ldots$. When
using i2b2 or UMLS knowledge concepts individually, we experiment with
$\alpha_{k},\alpha_{c}$ values between $(0,1]$ at intervals of $.1$, and we
found that $\alpha_{k}=.2$ and $\alpha_{c}=1.0$ performs best over the dev
set. When using i2b2 and UMLS concepts together, we found that assigning more
weight ($\alpha_{k}=.5$) to the broader i2b2 concepts than the UMLS concepts
($\alpha_{k}=.2$) performs best over the dev set. When training our model on
OntoNotes, we do not have the same knowledge lexicon available, so
effectively, we have $\alpha_{k}=0$ until we begin training on the i2b2 data.

For the concept identification auxiliary task, we use only i2b2 concepts for
our knowledge lexicon, since using the fine-grained UMLS concepts would induce
3,500 class labels. Additionally, since the i2b2 knowledge lexicon is not
available for OntoNotes, we ignore SL ($\beta_{3}=0$).

For our model, we choose max_seq_length of 512, BERT learning rate of $2e-5$,
and task specific learning rates of $1e-4$. Similar to Joshi et al. (2020), we
fine-tune 20 epochs for OntoNotes and the i2b2 training examples.

##  4 Results

For successful domain adaptation, our model aims to demonstrate the value of
incorporating concept knowledge. We evaluate overall coreference performance
improvement as a result of the span retrofitting and auxiliary concept
identification task in §4.1. Then, we inspect whether representations for
highly domain-specific spans are better for the coreference resolution task in
our model than in the baseline (§4.2).

Model Losses | Knowledge | Model Performance
---|---|---
MUC | B-cubed | CEAFE | averages
|  | R | P | F-1 | R | P | F-1 | R | P | F-1 | R | P | F-1
Baseline (CL) | NA | 70.93 | 72.51 | 71.71 | 64.91 | 66.48 | 65.69 | 54.57 | 58.44 | 56.44 | 63.47 | 65.81 | 64.61
CL \+ RL \+ SL | i2b2, UMLS | 71.15 | 73.64 | 72.37 | 65.03 | 67.59 | 66.28 | 54.77 | 60.64 | 57.56 | 63.65 | 68.04 | 65.41
CL \+ RL | i2b2, UMLS | 70.66 | 73.88 | 72.23 | 64.36 | 67.89 | 66.06 | 54.39 | 60.38 | 57.22 | 63.14 | 67.28 | 65.17
CL \+ SL | i2b2 | 70.28 | 74.14 | 72.16 | 64.22 | 68.27 | 66.18 | 54.69 | 60.43 | 57.41 | 63.43 | 67.17 | 65.24

Table 2: Overall coreference performance for various combinations of loss
functions and knowledge resources. Our model surpasses our Baseline (CL)
largely as a result of an improvement in precision (scores averaged over 6
random seed initializations).

###  4.1 Coreference Performance

In Table 2, we see that combining both of the losses we introduce in this work
(CL + RL + SL) improves the model precision by 2.23% resulting in a .8%
improvement in the F-1 score.

Combining the retrofitting loss and the auxiliary scaffolding loss performs
better than using each individually. It is possible that adding the
scaffolding loss alone is not as helpful because it does not contain the UMLS
knowledge.

While our model does improve precision and the overall F-1 score, recall
largely remains constant. The retrofitting loss pushes unrelated spans
belonging to different concepts further apart, and consequently we penalize
the detection of any valid mention that does not appear in our concept
knowledge. Incompleteness of our concept knowledge may be a contributing
factor to the lack of recall improvement.

It is expected that precision should be affected most by the additional losses
SL and RL, since both are designed to integrate knowledge in the pairwise
relationships between spans.

The scaffolding loss guides the model to be able to distinguish spans
belonging to different concepts from the span representation, and the
retrofitting loss enforces a knowledge-based distance function between spans.
Model recall is partly determined by a unary mention score used to identify
candidate mentions from spans in our model. This mention score is impacted by
our losses, since span representation is taken as input to the scoring
function. However, the pairwise knowledge integrated into the span
representation is much more useful for selecting an antecedent among a set of
candidates for a given span. Consequently, since our loss functions have a
bigger impact on the antecedent scoring function, then they will also have a
bigger impact on model precision.

Model Losses | Metric | i2b2 Concepts
---|---|---
Person | Problem | Treatment | Test
Baseline (CL) | Avg. R | 63.47 | 50.92 | 54.85 | 48.08
Avg. P | 83.76 | 79.98 | 82.98 | 84.43
Avg. F-1 | 69.92 | 58.0 | 62.24 | 54.74
CL \+ RL \+ SL | Avg. R | 62.66 | 51.98 | 53.17 | 48.67
Avg. P | 86.23 | 83.20 | 86.22 | 86.67
Avg. F-1 | 69.98 | 59.56 | 61.84 | 55.62

Table 3: Performance on coreference chains belonging to a specific concept.
Our model outperforms the baseline on more domain-specific spans indicating
our model improves domain adaptation problem and test coreference clusters

Model Losses | Metric | Avg. # of wordpieces/span range
---|---|---
[0.0,1.7) | [1.7,3.4) | [3.4,5.1) | [5.1,6.8) | [6.8,8.5)
Baseline (CL) | Avg. R | 47.19 | 31.46 | 35.33 | 27.77 | 26.49
Avg. P | 79.94 | 64.55 | 60.63 | 54.32 | 49.28
Avg. F-1 | 54.64 | 38.40 | 41.16 | 35.10 | 29.81
CL \+ RL \+ SL | Avg. R | 47.18 | 29.41 | 36.88 | 30.76 | 27.93
Avg. P | 83.04 | 67.42 | 65.31 | 54.74 | 60.17
Avg. F-1 | 55.19 | 37.39 | 43.57 | 37.28 | 33.78
CL \+ RL \+ SL (Overlap UMLS) | Avg. R | 48.68 | 31.73 | 38.03 | 31.73 | 29.59
Avg. P | 79.22 | 63.06 | 57.89 | 52.26 | 51.65
Avg. F-1 | 55.50 | 38.25 | 41.86 | 37.22 | 33.103
CL \+ RL (Overlap UMLS) | Avg. R | 47.32 | 29.71 | 34.67 | 29.74 | 36.82
Avg. P | 80.63 | 64.58 | 65.58 | 55.67 | 56.89
Avg. F-1 | 54.92 | 37.03 | 42.03 | 37.07 | 38.11

Table 4: We take the set of coreference chains s.t. the average number of
wordpieces per span falls withing the range, and evaluate model of the subset.
We observe that our model outperforms the baseline coreference chains with
spans that are subdivided more frequently  TP Coreferent Span Pair Examples
---
Hereafter, wife noted development of left sided weakness, facial droop,
slurring of speech …with past medical history of atrial fibrillation on
coumadin, coronary artery disease, hyperlipidemia, dementia with sudden onset
left sided weakness, dysarthria
evaluation and treatment of adenocarcinoma involving the transverse colon and
gallbladder …DISCHARGE DIAGNOSIS: 1. Metastatic gallbladder cancer
He is status post a hemiarthroplasty on 10/17/97 …decreased hematocrit prior
to his humeral fixation surgery
An angiogram was done which disclosed possible subsegmental pulmonary emboli
of the upper lobes as well …patient was bolused with intravenous heparin due
to concern for pulmonary embolism
Table 5: Examples of coreferent span pairs missed by Baseline (CL), identified
by our model (CL \+ RL \+ SL). In these cases, we can see that wordpiece
tokenization is likely misleading the baseline model, since the spans in each
pair have few wordpieces in common. FP Non-coreferent Span Pair Examples
---
She underwent an open laparoscopy …The patient is now admitted for exploratory
laparotomy
Right heart catheterization and coronary angiography on October 15 …urgently
transferred by Dr. Lenni Factor for possible angioplasty
78-yo male with atrial fibrillation…Mechanical mitral valve: Anticoagulation
was reversed
He had a cardiac catheterization performed which revealed …management after
this hospitalization and has done very well
Table 6: Examples of non-coreferent span pairs correctly missed our model (CL
\+ RL \+ SL), but identified by Baseline (CL). In these cases, we can see that
wordpiece tokenization is likely misleading the baseline model, since the
spans in each pair have wordpieces in common.

###  4.2 Performance on Domain-specific Spans

In addition to overall performance, we are interested in comparing performance
on rare spans that do not occur or occur infrequently in the source domain.
Specifically, we are interested in addressing performance degradation that can
occur as a result of wordpiece tokenization.

For example, consider a coreference cluster containing spans that are on
average subdivided many times. This is an indication that the vocabulary of
spans in the cluster is out of domain. The attention weighted vector over the
tokens in the spans may be a misleading representation since the token
embeddings correspond to shorter, less meaningful subwords. We report in Table
6 some examples of span pairs that the baseline model incorrectly predicts as
coreferent.

Consider the spans “open laparoscopy” and “exploratory laporotomy”. Their
tokenizations would include the subwords ["lap", "##aro", "##tom", "##y"] and
["lap", "##aro", "##sco", "##py"]. This overlap of the first few subwords
might lead the baseline to conclude that the spans are similar in non-
contextual meaning and consequently coreferent. From our fine-grained UMLS
knowledge, we know that laparoscopy and laparotomy belong to distinct
concepts. Our loss functions $\textbf{RL},\textbf{SL}$ guide the model to
produce disparate span representations since the spans map to distinct
concepts. Our model uses knowledge to learn to upweight the wordpieces that
are meaningful in context of the target domain (e.g., the suffix tokens in
this example). We report examples of challenging span pairs that our model
identifies and the baseline fails to identify in Table 5.

To collect quantitative evidence for our model’s performance on the most
challenging spans, we evaluate how model performance changes as the average
number of wordpieces per span increases in Table 4. We can observe that as the
average number of wordpieces per span increases beyond $3.4$, we start to see
an increasing F-1 performance over the baseline. Similar to the overall
results for the entire test set, we see that performance improvements in
precision are largely responsible.

Beyond average number of wordpieces per span, we can also use the concept
labels annotated in the i2b2 dataset to verify the performance of our model on
domain-specific spans. We verified that concepts person and treatment are
respectively dominated by spans like “doctor”, “patient” “surgery”, and
“procedure”, which are more likely to appear in the source domain than spans
like “afebrile”, “basal cell carcinoma”, “asculation” from the problem and
test concepts. Therefore, the performance improvement in Table 3 for concepts
problem and test suggests that our model can outperform the baseline on
domain-specific spans.

###  4.3 Visualization

We want to visualize how the span representation captures the mention-
antecedent relationship for different types of concepts – specifically how
consistently information is extracted from the mention span representation to
arrive at a predicted antecedent. We randomly select $200$ span pairs with a
mention-antecedent relationship. For each span pair, we extract the attention
weighted vector over the span tokens, which is the same piece of the span used
to compute SL, RL. For these 768-dimensional vectors, we take the projection
vectors from the mention to the antecedent vector. Then, in Figure 2 we
transform these projections into a 2-dimensional space using PCA and plot them
in $\mathbb{R}^{2}$ similar to Faruqui et al. (2015).

For the baseline model (top), most of the mention-antecedent vectors share a
similar direction regardless of concept type. However, for the CF \+ RL \+ SL
model (bottom), there is a clear distinction in mention-antecedent vectors for
each concept. This suggests that our model is able to construct a span
representation that can capture the mention-antecedent relationship in a way
that is (1) specific to the concept (2) consistent across all mention-
antecedent pairs belonging to the concept.

Figure 2: Two-dimensional PCA projections of attention-weighted span tokens
for vector pairs holding the mention-antecedent relation for the baseline
(top) and our model CL + SL + RL (bottom). Our model uses concept knowledge to
construct a span representation that more consistently captures the mention-
antecedent relationship specific to each concept.

##  5 Related Work

Prior to the introduction of end-to-end neural models in Lee et al. (2017),
coreference resolution for the clinical domain used pipelined approaches,
allowing for the propagation of errors from other core NLU tasks and relying
on hand-crafted rules for resolving each type of entity in the domain Jindal
et al. (2014). Feature-based methods like Jindal et al. (2014) using knowledge
in the coreference model rely on the availability of knowledge at test time.
We are focused instead on the case where there may be no concept knowledge
available when our model is deployed.

There has been some limited work in domain adaptation for coreference
resolution. Yang et al. (2012) adapts a model trained on the MUC-6 and ACE
2005 datasets to the biomedical domain using an active learning approach,
applying data augmentation and pruning techniques. Zhao and Ng (2014) propose
a feature-based active learning method to learn cross-domain knowledge. Unlike
these works, we take advantage of the modern expressive power of the SpanBERT
representation. With the introduction of SpanBERT, there was a marked
performance improvement for several NLU tasks including coreference
resolution. Joshi et al. (2020) showed that SpanBERT could be fine-tuned to
perform well on several datasets, e.g., GLU and ACE Wang et al. (2018);
Doddington et al. (2004).
However, Lu and Ng (2020) found that SpanBERT coreference resolvers generally
rely more on mentions than context, so they are susceptible to small
perturbations (e.g., changing all the names/nominal mentions in the test set).
More generally, for NLU tasks several studies have found that neural models
rely heavily on shallow heuristics in the text rather than learning the
underlying structure of the linguistic phenomenon Peng et al. (2020); Rosenman
et al. (2020), leading to the misinterpretation of context Alt et al. (2020).

This poses a challenge for adapting a coreference model to noisy domains like
clinical text and mostly OOD spans. With the recent success of fine-tuning for
domain adaptation, a natural approach would be to fine-tune the SpanBERT
representation, the coreference model, or both Gururangan et al. (2020). Fine-
tuning the pretrained SpanBERT method alone can be expensive – Gururangan et
al. (2020) showed that the best performing scheme requires 180K documents. In
settings where there are fewer documents available in the target domain, it is
still possible to fine-tune a coreference model with SpanBERT Joshi et al.
(2020).

However, a persisting challenge with adapting the span representation is
associated with the wordpiece tokenization employed by SpanBERT. For highly
technical domain-specific language, it is natural that there is a higher
average number of subwords per span, since it is unlikely that many spans or
spans split only once belong to the limited 300,000 word SpanBERT vocabulary.
Poerner et al. (2020a) show that wordpiece tokenization in the biomedical
domain can lead to misleading span representations. Purely fine-tuning
approaches fail to address this issue, since the SpanBERT vocabulary is
constant.

At the same time, SpanBERT implicitly learns a geometry that encodes rich
information related to the coreference task. Hewitt and Manning (2019) show
that it is possible to learn a linear projection space from BERT embeddings to
capture linguistic phenomena like syntactic dependencies. In Kahardipraja et
al. (2020), the authors use a Feedforward Neural Network (FFNN) to probe for
properties of coreference structure, finding evidence that the SpanBERT
representation can be used to predict coreference arcs and the head word of
the span with >90% F1. Therefore, it is likely that overall performance
greatly depends on the quality of span representation.

In our work, we develop multiple techniques to adapt the span representation
to a new domain with concept knowledge, allowing the model to be fine-tuned
with fewer target domain examples and perform better on highly domain-specific
entities. In particular, by incorporating knowledge into the span
representation, we are able to restore a global, non-contextual meaning to
excessively subtokenized spans.

##  6 Conclusion

We present methods to efficiently adapt coreference resolution models to a new
domain using domain-specific concept knowledge. We demonstrate that we can
integrate knowledge into the span representation using two losses to (1)
retrofit the span representation to the concept knowledge and (2) ensure that
knowledge can be recovered from the span representation using an auxiliary
concept identification task. Using these methods, we are able to improve the
performance of our baseline, especially for highly domain-specific spans.

## Acknowledgements

Thanks to Alex Chouldechova, Amanda Coston, David Steier, and the Allegheny
County Department of Human Services for valuable feedback on this work. This
work is supported by the Block Center for Technology and Innovation, and A.F.
is supported by a Google PhD Fellowship.

## References

  * Alt et al. (2020) Christoph Alt, Aleksandra Gabryszak, and Leonhard Hennig. 2020.  Probing linguistic features of sentence-level representations in neural relation extraction.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 1534–1545, Online. Association for Computational Linguistics.
  * Bodenreider (2004) Olivier Bodenreider. 2004.  The unified medical language system (umls): integrating biomedical terminology.  _Nucleic acids research_ , 32(suppl_1):D267–D270.
  * Doddington et al. (2004) George R Doddington, Alexis Mitchell, Mark A Przybocki, Lance A Ramshaw, Stephanie M Strassel, and Ralph M Weischedel. 2004.  The automatic content extraction (ace) program-tasks, data, and evaluation.  In _Lrec_ , volume 2, pages 837–840. Lisbon.
  * Faruqui et al. (2015) Manaal Faruqui, Jesse Dodge, Sujay Kumar Jauhar, Chris Dyer, Eduard Hovy, and Noah A. Smith. 2015.  Retrofitting word vectors to semantic lexicons.  In _Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 1606–1615, Denver, Colorado. Association for Computational Linguistics.
  * Gururangan et al. (2020) Suchin Gururangan, Ana Marasović, Swabha Swayamdipta, Kyle Lo, Iz Beltagy, Doug Downey, and Noah A. Smith. 2020.  Don’t stop pretraining: Adapt language models to domains and tasks.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 8342–8360, Online. Association for Computational Linguistics.
  * Hewitt and Manning (2019) John Hewitt and Christopher D Manning. 2019.  A structural probe for finding syntax in word representations.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , pages 4129–4138.
  * Jindal et al. (2014) Prateek Jindal, Dan Roth, and Carl A Gunter. 2014.  Joint inference for end-to-end coreference resolution for clinical notes.  In _Proceedings of the 5th ACM Conference on Bioinformatics, Computational Biology, and Health Informatics_ , pages 192–201.
  * Joshi et al. (2020) Mandar Joshi, Danqi Chen, Yinhan Liu, Daniel S Weld, Luke Zettlemoyer, and Omer Levy. 2020.  Spanbert: Improving pre-training by representing and predicting spans.  _Transactions of the Association for Computational Linguistics_ , 8:64–77.
  * Joshi et al. (2019) Mandar Joshi, Omer Levy, Luke Zettlemoyer, and Daniel Weld. 2019.  BERT for coreference resolution: Baselines and analysis.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pages 5803–5808, Hong Kong, China. Association for Computational Linguistics.
  * Kahardipraja et al. (2020) Patrick Kahardipraja, Olena Vyshnevska, and Sharid Loáiciga. 2020.  Exploring span representations in neural coreference resolution.  In _Proceedings of the First Workshop on Computational Approaches to Discourse_ , pages 32–41.
  * Kirstain et al. (2021) Yuval Kirstain, Ori Ram, and Omer Levy. 2021.  Coreference resolution without span representations.  In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 2: Short Papers)_ , pages 14–19, Online. Association for Computational Linguistics.
  * Lee et al. (2017) Kenton Lee, Luheng He, Mike Lewis, and Luke Zettlemoyer. 2017.  End-to-end neural coreference resolution.  In _Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing_ , pages 188–197, Copenhagen, Denmark. Association for Computational Linguistics.
  * Lu and Ng (2020) Jing Lu and Vincent Ng. 2020.  Conundrums in entity reference resolution.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 6620–6631.
  * Peng et al. (2020) Hao Peng, Tianyu Gao, Xu Han, Yankai Lin, Peng Li, Zhiyuan Liu, Maosong Sun, and Jie Zhou. 2020.  Learning from Context or Names? An Empirical Study on Neural Relation Extraction.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 3661–3672, Online. Association for Computational Linguistics.
  * Poerner et al. (2020a) Nina Poerner, Ulli Waltinger, and Hinrich Schütze. 2020a.  E-BERT: Efficient-yet-effective entity embeddings for BERT.  In _Findings of the Association for Computational Linguistics: EMNLP 2020_ , pages 803–818, Online. Association for Computational Linguistics.
  * Poerner et al. (2020b) Nina Poerner, Ulli Waltinger, and Hinrich Schütze. 2020b.  Inexpensive domain adaptation of pretrained language models: Case studies on biomedical NER and covid-19 QA.  In _Findings of the Association for Computational Linguistics: EMNLP 2020_ , pages 1482–1490, Online. Association for Computational Linguistics.
  * Rosenman et al. (2020) Shachar Rosenman, Alon Jacovi, and Yoav Goldberg. 2020.  Exposing Shallow Heuristics of Relation Extraction Models with Challenge Data.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 3702–3710, Online. Association for Computational Linguistics.
  * Srivastava et al. (2020) Ankit Srivastava, Piyush Makhija, and Anuj Gupta. 2020.  Noisy text data: Achilles’ heel of bert.  In _Proceedings of the Sixth Workshop on Noisy User-generated Text (W-NUT 2020)_ , pages 16–21.
  * Swayamdipta et al. (2018) Swabha Swayamdipta, Sam Thomson, Kenton Lee, Luke Zettlemoyer, Chris Dyer, and Noah A. Smith. 2018.  Syntactic scaffolds for semantic structures.  In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_ , pages 3772–3782, Brussels, Belgium. Association for Computational Linguistics.
  * Uzuner et al. (2011) Özlem Uzuner, Brett R South, Shuying Shen, and Scott L DuVall. 2011.  2010 i2b2/va challenge on concepts, assertions, and relations in clinical text.  _Journal of the American Medical Informatics Association_ , 18(5):552–556.
  * Wang et al. (2018) Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman. 2018.  GLUE: A multi-task benchmark and analysis platform for natural language understanding.  In _Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP_ , pages 353–355, Brussels, Belgium. Association for Computational Linguistics.
  * Weischedel et al. (2013) Ralph Weischedel, Martha Palmer, Mitchell Marcus, Eduard Hovy, Sameer Pradhan, Lance Ramshaw, Nianwen Xue, Ann Taylor, Jeff Kaufman, Michelle Franchini, Mohammed El-Bachouti, Robert Belvin, and Ann Houston. 2013.  OntoNotes Release 5.0.
  * Xia and Van Durme (2021) Patrick Xia and Benjamin Van Durme. 2021.  Moving on from ontonotes: Coreference resolution model transfer.  _arXiv preprint arXiv:2104.08457_.
  * Xu and Choi (2020) Liyan Xu and Jinho D. Choi. 2020.  Revealing the myth of higher-order inference in coreference resolution.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 8527–8533, Online. Association for Computational Linguistics.
  * Yang et al. (2012) Jian Bo Yang, Qi Mao, Qiao Liang Xiang, Ivor Wai-Hung Tsang, Kian Ming A Chai, and Hai Leong Chieu. 2012.  Domain adaptation for coreference resolution: An adaptive ensemble approach.  In _Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning_ , pages 744–753.
  * Zhao and Ng (2014) Shanheng Zhao and Hwee Tou Ng. 2014.  Domain adaptation with active learning for coreference resolution.  In _Proceedings of the 5th International Workshop on Health Text Mining and Information Analysis (Louhi)_ , pages 21–29.


that saturate the inequalities (2.9), because it allows to evolve arbitrarily
strong shockwaves. Moreover, this choice also guarantees that the field
redefinition terms are as small as possible.

In the hydrodynamic studies of the QGP, the initial data for the stress tensor
is obtained by some prehydrodynamic procedure. If we want to implement the
relativistic Navier-Stokes equations in this context, one needs to address the
question of constructing initial data in a certain causal frame. One way to
proceed is to diagonalize a given stress tensor and find the data for the
evolution variables in the Landau frame; then, use the prescription of
effective field theory (2.7) to find the corresponding quantities in the
causal frame. Therefore, one needs to address the effects of changing frames
in the initial data.

To study the effects of changing frames in the initial data, in this paper we
have used solutions obtained in a certain frame for a time interval to
construct initial data at some $t>0$ in a new frame using (2.7), see Figs. 5
and 7. The lesson that we have learned from this exercise is that the change
of frame in the initial data has a small effect in the subsequent evolution.
That is, the difference in the two evolutions in different frames due to the
change of frame in the initial data is comparable to the difference introduced
upon time evolution in different frames when using similar initial data that
satisfies the ideal equations. In these constructions, the equations of ideal
hydrodynamics are nearly satisfied, which implies that the effects of changing
frames in the initial data are necessarily small.

We also studied a situation in which the the ideal hydrodynamics equations are
far from being satisfied initially, see Fig. 6; this situation arises, for
example, when the time derivatives in the initial data are set to zero. We
found that this introduces large differences in the initial stress tensor
under frame changes and this difference is sustained upon time evolution. To
be precise, this difference is large compared to the difference generated
solely upon time evolution with initial data that satisfies the ideal
equations. This can be simply understood because having large error terms in
the ideal equations implies that initial data in different frames is
significantly different; then Cauchy stability implies that the subsequent
evolutions in different frames will also differ significantly. In practical
applications, this situation should not arise since one wants to initialize a
hydrodynamic code to describe a system that is in the regime of hydrodynamics,
and hence the ideal equations should nearly satisfied.

We now comment on future directions. We have limited our studies to conformal
fluids as a natural starting point to perform a detailed analysis of the
consequences of using different causal frames. However, physical systems of
interest like neutron star mergers or the QGP created in heavy-ion collisions
are not conformal, and it would be interesting to extend our analysis to the
non-conformal case. Dynamical evolutions of the relativistic first-order
viscous hydrodynamics equations in the non-conformal case have been studied in
[34].

An interesting direction for studies of relativistic hydrodynamics is to
perform comparisons with microscopic real-time evolutions in quantum field
theories. Such comparisons allow to explore quantitatively under which
circumstances the effective field theory provides a good description of the
microscopic theory. In particular, one would be able to estimate the
acceptable threshold values for the different criteria that we have proposed.
However, obtaining real time evolutions in interacting quantum field theories
is in general a very difficult task. In this context, holography turns out to
be a very useful tool. We can think of a specific holographic theory as a toy
model, specific to certain quantum field field theories that are very
different from QCD in many aspects, but useful nevertheless to obtain
information about the real time dynamics in a controlled way; such a
computation is not feasible in QCD. The qualitative conclusions obtained from
this analysis, even if obtained with holography, might apply more generally,
and in particular to finite temperature QCD. We initiated this line of
research in [32] and we will extend these studies in the future. This paper is
part of a long term program that aims to contribute to the implementation of
the relativistic Navier-Stokes equations in studies of physical systems of
interest such as the QGP created in heavy-ion collisions or astrophysical
systems.

## Acknowledgments

We thank Jorge Casalderrey-Solana, David Mateos and Alexandre Serantes for
very useful discussions. Y.B. is supported by a Maria Zambrano postdoctoral
fellowship from the University of Barcelona, by the “Unit of Excellence MdM
2020-2023” award to the Institute of Cosmos Sciences (CEX2019-000918-M), and
by grants PID2019-105614GB-C22, 2021-SGR-872 and PID2022-136224NB-C22, funded
by MCIN/AEI/ 10.13039/501100011033/FEDER, UE. P.F. was supported by a Royal
Society University Research Fellowship (Grant No. UF140319 and URF\R\201026)
and is currently supported by the STFC Consolidated Grant ST/X000931/1.

##  Appendix A Convergence tests

Our code uses finite difference stencils of order 6 for the spatial
derivatives and a fifth order explicit Runge-Kutta-Nyström Generalized (RKNG)
time integration method for the equations written in second order (both in
time and space) form. See [32] for more details. In [32] we presented a
convergence test showing a convergence factor of 4 or even higher. In this
paper we use a 1+1 version of that code, and we present convergence tests for
some representative runs of this paper. For the runs presented in this paper
we typically use $N_{x}=2048$ points along the spatial dimension and Courant
factor 0.1. For a set of representative runs, we perform a convergence test
for which we use three different resolutions $N_{x}/4$,$N_{x}/2$ and $N_{x}$.
In Fig. 13 we show convergence tests for the runs presented in Figs. 3 and 6,
and in Fig. 14 convergence tests for the runs presented in Figs. 7 and 10. In
some cases there is an initial transient, but after it in all cases the
convergence factor is between 4 and 5, consistent with our method.

Figure 13: Convergence test corresponding to the small amplitude sinusoidal
perturbation runs presented in Figs. 3 and 6, with initial data (3.3) and
(3.9) respectively. (Left) Runs with $k/\overline{T}\simeq 0.184$, (right)
runs with $k/\overline{T}\simeq 1.84$. The convergence rate approaches 4, or
even goes above, as one would expect given our differencing and time
integration schemes.

Figure 14: (Left) Convergence test corresponding to the gaussian perturbation
runs presented in Fig. 7 with initial data (3.10). The convergence rate for
all the runs is close to 5. (Right) Convergence test for the Riemann problem
presented in Fig. 10 with initial data (3.11).

## References

  * [1] P. Romatschke and U. Romatschke, _Relativistic Fluid Dynamics In and Out of Equilibrium_ , Cambridge Monographs on Mathematical Physics, Cambridge University Press (5, 2019), 10.1017/9781108651998, [1712.05815].
  * [2] W. Busza, K. Rajagopal and W. van der Schee, _Heavy Ion Collisions: The Big Picture, and the Big Questions_ , _Ann. Rev. Nucl. Part. Sci._ 68 (2018) 339 [1802.04801].
  * [3] STAR Collaboration collaboration, _STAR Experiment Results From Beam Energy Scan Program_ , _Phys. Atom. Nucl._ 86 (2023) 758.
  * [4] M.G. Alford, L. Bovard, M. Hanauske, L. Rezzolla and K. Schwenzer, _Viscous Dissipation and Heat Conduction in Binary Neutron-Star Mergers_ , _Phys. Rev. Lett._ 120 (2018) 041101 [1707.09475].
  * [5] L.D. Landau and E.M. Lifshitz, _Fluid Mechanics_ , _Pergamon_ (1987) .
  * [6] M. Chabanov, L. Rezzolla and D.H. Rischke, _General-relativistic hydrodynamics of non-perfect fluids: 3+1 conservative formulation and application to viscous black-hole accretion_ , 2102.10419.
  * [7] M. Shibata, K. Kiuchi and Y.-i. Sekiguchi, _General relativistic viscous hydrodynamics of differentially rotating neutron stars_ , _Phys. Rev. D_ 95 (2017) 083005 [1703.10303].
  * [8] S. Fujibayashi, K. Kiuchi, N. Nishimura, Y. Sekiguchi and M. Shibata, _Mass Ejection from the Remnant of a Binary Neutron Star Merger: Viscous-Radiation Hydrodynamics Study_ , _Astrophys. J._ 860 (2018) 64 [1711.02093].
  * [9] S. Fujibayashi, M. Shibata, S. Wanajo, K. Kiuchi, K. Kyutoku and Y. Sekiguchi, _Viscous evolution of a massive disk surrounding stellar-mass black holes in full general relativity_ , _Phys. Rev. D_ 102 (2020) 123014 [2009.03895].
  * [10] M. Chabanov and L. Rezzolla, _Impact of bulk viscosity on the post-merger gravitational-wave signal from merging neutron stars_ , 2307.10464.
  * [11] M. Chabanov and L. Rezzolla, _Numerical modelling of bulk viscosity in neutron stars_ , 2311.13027.
  * [12] J.L. Ripley, A. Hegade K. R., R.S. Chandramouli and N. Yunes, _First constraint on the dissipative tidal deformability of neutron stars_ , 2312.11659.
  * [13] C. Eckart, _The Thermodynamics of irreversible processes. 3.. Relativistic theory of the simple fluid_ , _Phys. Rev._ 58 (1940) 919.
  * [14] W.A. Hiscock and L. Lindblom, _Generic instabilities in first-order dissipative relativistic fluid theories_ , _Phys. Rev. D_ 31 (1985) 725.
  * [15] I. Muller, _Zum Paradoxon der Warmeleitungstheorie_ , _Z. Phys._ 198 (1967) 329.
  * [16] W. Israel, _Nonstationary irreversible thermodynamics: A Causal relativistic theory_ , _Annals Phys._ 100 (1976) 310.
  * [17] W. Israel and J.M. Stewart, _Transient relativistic thermodynamics and kinetic theory_ , _Annals Phys._ 118 (1979) 341.
  * [18] R. Baier, P. Romatschke, D.T. Son, A.O. Starinets and M.A. Stephanov, _Relativistic viscous hydrodynamics, conformal invariance, and holography_ , _JHEP_ 04 (2008) 100 [0712.2451].
  * [19] G.S. Denicol, H. Niemi, E. Molnar and D.H. Rischke, _Derivation of transient relativistic fluid dynamics from the Boltzmann equation_ , _Phys. Rev. D_ 85 (2012) 114047 [1202.4551].
  * [20] R.P. Geroch and L. Lindblom, _Dissipative relativistic fluid theories of divergence type_ , _Phys. Rev. D_ 41 (1990) 1855.
  * [21] L. Lehner, O.A. Reula and M.E. Rubio, _Hyperbolic theory of relativistic conformal dissipative fluids_ , _Phys. Rev. D_ 97 (2018) 024013 [1710.08033].
  * [22] P. Montes, M.E. Rubio and O.A. Reula, _Numerical simulations of divergence-type theories for conformal dissipative fluids_ , 2304.08584.
  * [23] R.P. Geroch, _Relativistic theories of dissipative fluids_ , _J. Math. Phys._ 36 (1995) 4226.
  * [24] F.S. Bemfica, M.M. Disconzi and J. Noronha, _Causality and existence of solutions of relativistic viscous fluid dynamics with gravity_ , _Phys. Rev. D_ 98 (2018) 104064 [1708.06255].
  * [25] F.S. Bemfica, M.M. Disconzi and J. Noronha, _Nonlinear Causality of General First-Order Relativistic Viscous Hydrodynamics_ , _Phys. Rev. D_ 100 (2019) 104020 [1907.12695].
  * [26] F.S. Bemfica, M.M. Disconzi and J. Noronha, _First-Order General-Relativistic Viscous Fluid Dynamics_ , _Phys. Rev. X_ 12 (2022) 021044 [2009.11388].
  * [27] P. Kovtun, _First-order relativistic hydrodynamics is stable_ , _JHEP_ 10 (2019) 034 [1907.08191].
  * [28] F.S. Bemfica, M.M. Disconzi, V. Hoang, J. Noronha and M. Radosz, _Nonlinear Constraints on Relativistic Fluids Far from Equilibrium_ , _Phys. Rev. Lett._ 126 (2021) 222301 [2005.11632].
  * [29] C. Plumberg, D. Almaalol, T. Dore, J. Noronha and J. Noronha-Hostler, _Causality violations in realistic simulations of heavy-ion collisions_ , _Phys. Rev. C_ 105 (2022) L061901 [2103.15889].
  * [30] H. Freistuhler, _Nonexistence and existence of shock profiles in the Bemfica-Disconzi-Noronha model_ , _Phys. Rev. D_ 103 (2021) 124045 [2103.16661].
  * [31] A. Pandya and F. Pretorius, _Numerical exploration of first-order relativistic hydrodynamics_ , _Phys. Rev. D_ 104 (2021) 023015 [2104.00804].
  * [32] H. Bantilan, Y. Bea and P. Figueras, _Evolutions in first-order viscous hydrodynamics_ , _JHEP_ 08 (2022) 298 [2201.13359].
  * [33] A. Pandya, E.R. Most and F. Pretorius, _Conservative finite volume scheme for first-order viscous relativistic hydrodynamics_ , 2201.12317.
  * [34] A. Pandya, E.R. Most and F. Pretorius, _Causal, stable first-order viscous relativistic hydrodynamics with ideal gas microphysics_ , _Phys. Rev. D_ 106 (2022) 123036 [2209.09265].
  * [35] R.E. Hoult and P. Kovtun, _Stable and causal relativistic Navier-Stokes equations_ , _JHEP_ 06 (2020) 067 [2004.04102].
  * [36] H.S. Reall and C.M. Warnick, _Effective field theory and classical equations of motion_ , _J. Math. Phys._ 63 (2022) 042901 [2105.12028].
  * [37] P. Van and T.S. Biro, _First order and stable relativistic dissipative hydrodynamics_ , _Phys. Lett. B_ 709 (2012) 106 [1109.0985].
  * [38] H. Freistuhler and B. Temple, _Causal dissipation and shock profiles in the relativistic fluid dynamics of pure radiation_ , _Proceedings of the Royal Society of London Series A 470_ 20140055 (2014) .
  * [39] H. Freistuhler and B. Temple, _Causal dissipation for the relativistic dynamics of ideal gases_ , _Proceedings of the Royal Society of London Series A 473_ 20160729 (2017) .
  * [40] H. Freistuhler and B. Temple, _Causal dissipation in the relativistic dynamics of barotropic fluids_ , _Journal of Mathematical Physics 59_ 063101 (2018) .
  * [41] M.M. Disconzi, _On the existence of solutions and causality for relativistic viscous conformal fluids_ , _Analysis_ 18 (2019) 1567 [1708.06572].
  * [42] F.S. Bemfica, M.M. Disconzi, C. Rodriguez and Y. Shao, _Local well-posedness in Sobolev spaces for first-order conformal causal relativistic viscous hydrodynamics_ , 1911.02504.
  * [43] G. Policastro, D.T. Son and A.O. Starinets, _The Shear viscosity of strongly coupled N=4 supersymmetric Yang-Mills plasma_ , _Phys. Rev. Lett._ 87 (2001) 081601 [hep-th/0104066].
  * [44] P. Kovtun, D.T. Son and A.O. Starinets, _Holography and hydrodynamics: Diffusion on stretched horizons_ , _JHEP_ 10 (2003) 064 [hep-th/0309213].
  * [45] T. Schäfer and D. Teaney, _Nearly Perfect Fluidity: From Cold Atomic Gases to Hot Quark Gluon Plasmas_ , _Rept. Prog. Phys._ 72 (2009) 126001 [0904.3107].
  * [46] S. Bhattacharyya, V.E. Hubeny, S. Minwalla and M. Rangamani, _Nonlinear Fluid Dynamics from Gravity_ , _JHEP_ 02 (2008) 045 [0712.2456].
  * [47] L. Rezzolla and O. Zanotti, _Relativistic Hydrodynamics_ , .