# On the Impact of Temporal Representations on Metaphor Detection

Giorgio Ottolina
Systems and Communication
University of Milano-Bicocca
Milan, Italy
<EMAIL_ADDRESS>
Matteo Palmonari
Systems and Communication
University of Milano-Bicocca
Milan, Italy
<EMAIL_ADDRESS>
Mehwish Alam
FIZ Karlsruhe - Leibniz Institute for Information Infrastructure
Karlsruhe Institute of Technology
Karlsruhe, Germany
<EMAIL_ADDRESS>
Manuel Vimercati
Systems and Communication
University of Milano-Bicocca
Milan, Italy
<EMAIL_ADDRESS>

###### Abstract

State-of-the-art approaches for metaphor detection compare their literal - or
core - meaning and their contextual meaning using sequential metaphor
classifiers based on neural networks. The signal that represents the literal
meaning is often represented by (non-contextual) word embeddings. However,
metaphorical expressions evolve over time due to various reasons, such as
cultural and societal impact. Metaphorical expressions are known to co-evolve
with language and literal word meanings, and even drive, to some extent, this
evolution. This rises the question whether different, possibly time-specific,
representations of literal meanings may impact on the metaphor detection task.
To the best of our knowledge, this is the first study which examines the
metaphor detection task with a detailed explorative analysis where different
temporal and static word embeddings are used to account for different
representations of literal meanings. Our experimental analysis is based on
three popular benchmarks used for metaphor detection and word embeddings
extracted from different corpora and temporally aligned to different state-of-
the-art approaches. The results suggest that different word embeddings do
impact on the metaphor detection task and some temporal word embeddings
slightly outperform static methods on some performance measures. However,
results also suggest that temporal word embeddings may provide representations
of words’ core meaning even too close to their metaphorical meaning, thus
confusing the classifier. Overall, the interaction between temporal language
evolution and metaphor detection appears tiny in the benchmark datasets used
in our experiments. This suggests that future work for the computational
analysis of this important linguistic phenomenon should first start by
creating a new dataset where this interaction is better represented.

_Keywords_ Metaphor Detection $\cdot$ Temporal Word Embeddings $\cdot$ Word
Embeddings

##  1 Introduction

Accounting for figurative language is one of the key challenges in Natural
Language Processing (NLP). Many recent methods have proposed solutions to this
problem based on machine learning methods (Recupero et al., 2019; Shutova,
2015; Leong et al., 2020). Figurative language often contains metaphorical
expressions which map one concept from a source domain to another concept in a
target domain. For instance, in the sentence “The wheels of Stalin’s regime
were well-oiled and already turning", a political system (target concept) is
viewed in terms of a mechanism (source concept) which can function, break,
have wheels, etc. This association allows us to transfer knowledge from the
domain of mechanical engineering to that of politics. Therefore, political
systems are thought about in terms of mechanisms, leading to multiple
metaphorical expressions. The phenomenon of source-target domain mapping was
first introduced by George Lakoff known as Conceptual Metaphor Theory (Lakoff
and Johnson, 1980). Due to previously defined characteristics, the presence of
metaphorical expression in text causes misinterpretation in the algorithms
such as machine translation or sentiment analysis (Saif Mohammad, 2016). For
example, in machine translation, the text may be translated literally choosing
inappropriate words, instead of those ones that capture the metaphorical
meaning. Similarly, in the case of sentiment analysis, the polarity of the
sentences can be misinterpreted and even inverted due to the presence of
metaphors. Therefore, several studies have addressed the problem of detecting
metaphors in text.

The studies addressing the metaphor detection problem in the last decade
(Shutova, 2015) usually exploit word embeddings to encode word meaning. Word
embeddings are distributed representations, i.e., vectors, derived from the
usage of words in large text corpora: words occurring in similar contexts have
similar meanings and are eventually close to each other in a vector space
(Almeida and Xexéo, 2019). In metaphor detection, these vector-based
representations can be used as signals to classify sentences or individual
words as being used with a metaphorical meaning or not. The key intuition is
to recognize that words are used in a context that is different from their
usual context, i.e., that their usage in a specific context significantly
differs from their usage in their most frequent contexts, which qualify their
literal meanings. In the previous example, “wheels" is collocated close to
“Stalin" and “regime", which define a context different from the contexts in
which it usually appears, i.e., in the domain of mechanical engineering. In
other words, the classifier, usually implemented as a neural network, is
expected to compare literal word meanings with word meanings in specific
contexts to detect if in that context a metaphor is used. Most of the recent
approaches have therefore combined non-contextual and contextual word
embeddings to provide signals for this comparison (e.g., exploiting the
concatenation of non-contextualized and contextualized word vectors) (Mao et
al., 2019; Swarnkar and Singh, 2018). For example, Mao et al. (2019);
Gulordava and Baroni (2011); Tomas Mikolov and Dean (2013a) combine non-
contextual GloVe embeddings (Jeffrey Pennington, 2014) with contextual ELMo
embeddings (Peters M, 2018) within a BiLSTM neural network for sequence
labelling. Glove embeddings account for literal word meanings, while ELMo
embeddings account for contextual word meanings. The network tries to learn
when the comparison between the two vectors indicates a metaphorical word
usage.

Word embedding techniques such as GloVe or Word2Vec (Tomas Mikolov and Dean,
2013a) are static embeddings that associate each word with one, context
independent, representation. Contextual word embeddings, such as ELMo and BERT
(Devlin et al., 2018), associate each word with one representation per
sentence. For example, in the following two sentences: “Apple sells phones”
and “I eat an apple”, contextual embeddings will represent “apple” differently
in each sentence, while static embedding can not distinguish the semantic
difference between the two references of “apple”. Usually, the latter
represents a word with its core meaning, i.e., with the meaning in those
contexts that are more frequent in the corpus used to train the embeddings
(when more meanings compete as core meanings, words may be associated with
vectors that mediate across contexts). An important linguistic phenomenon that
is not accounted in static and contextual word embeddings is language
evolution. While contextual word embeddings are expected to be sentence-
specific and thus de-contextualized with respect to time, static word
embeddings are expected to account for core meanings - and literal meanings in
metaphor detection. Core meanings of course change over time and several
approaches have been proposed to capture this changes (William L. Hamilton,
2018). The trait of evolution of the meaning over time is also shared by
metaphorical expressions which can be due to various reasons such as cultural
and societal impact. Metaphorical expressions are known to co-evolve with
language and literal word meanings derive this evolution to some extent (Smith
and Höfler, 2015a; Aitchison, 2003). This leads to the question whether
different, possibly time-specific, representations of literal meanings impact
the task of metaphor detection. In conclusion, if metaphor detection
approaches tend to compare a sentence-specific and a literal meaning, we must
be aware that literal meaning as accounted in static word embeddings 1)
depends on the corpus and method used to train the embeddings and 2) evolve
over time.

To this end, the current empirical study focuses on analyzing the impact of
different embeddings accounting for literal word meaning on the task of
metaphor detection. In particular, we want to study the interactions between
metaphor detection approaches and different word embeddings used to account
for the literal meaning of words in these approaches. We want to dedicate a
special attention to possible interactions between metaphor detection and
time-specific (non–contextual) word representations used to account for
literal meanings at different times. To the best of our knowledge, this is the
first study seeking for interactions between time-dependent word
representations and metaphor detection approaches. The empirical study
discussed in this paper aims to make a first step into addressing the co-
evolution of metaphors and language evolution that is known to be an important
factor for language evolution itself (Smith and Höfler, 2015a; Aitchison,
2003).

The methodology adopted in our study consists in following protocol. First, we
select a state-of-the-art Recurrent Neural Networks (RNN)-based model (Ge Gao
and Zettlemoyer, 2018) for metaphor detection which uses static word
embeddings to account for literal word meaning; the model performs metaphor
detection as a sequence classification task where each word occurrence is
labeled as either a metaphor usage or a literal usage. Second, we select three
benchmark data sets widely used to evaluate the performance of metaphor
detection approaches. Third, we feed the RNN-based model with literal meaning
vectors obtained from different (non contextual) word embeddings spaces; these
spaces differs for the corpora used to train them and, especially include
temporal word embeddings computed for different decades and aligned with
state-of-the-art alignment methods, such as Procrustes (Edouard Grave, 2018)
and the Compass method (first referred to as Temporal Word Embeddings with a
Compass (Di Carlo et al., 2019) \- TWEC and later as Compass-aligned
Distributional Embeddings (Bianchi et al., 2020). The experimental results
indicate that different word embeddings impact the metaphor detection task and
some temporal word embeddings slightly outperform static methods on some
performance measures. These quantitative results are then explained with the
help of a qualitative analysis of the predictions made by the models.

During this analysis, some interesting recurring patterns were observed. Some
patterns concern the interactions between literal meanings and domains of
discourse. For example, in sentences containing correctly identified
metaphors, the topics related to economics, politics, and emotions are the
most recurring ones. Besides, verbs having a literal meaning characterized by
physical connotations, often assume figurative meanings when used in the
sentences related to the previously listed contexts.

Some patterns concern indeed the interaction between time and language.
Studying the predictions obtained with the embeddings of one specific time
period, we noticed that none of the sentences belonging to the “news" domain
of the VUA dataset were correctly predicted. This could indicate that for that
specific time period (1990 decade), words’ representations of that domain are
biased towards their metaphorical meaning, and this would prevent the neural
networks from correctly identifying the metaphors. Furthermore, if temporal
word embeddings provided words’ representations that are more inclined towards
their literal core meaning (and not the metaphorical one), models exploiting
them would correctly identify metaphors more easily. One way to investigate
these hypotheses further is to explore the nearest neighbors of a word in the
word embeddings used in a figurative way inside a sentence, both in a static
(atemporal) word embedding space, e.g., obtained with GloVe (Jeffrey
Pennington, 2014) and in a decade-specific temporal space, e.g., obtained from
the CoHa111https://www.english-corpora.org/coha/ corpus with Procrustes
(Edouard Grave, 2018).This pattern is highlighted by the following sentence
example: “The virus attacked Argonne National Laboratory outside Chicago
starting at 11.54 pm EST Wednesday and throughout the night". If we
investigate the ten nearest neighbors of “virus", in the temporal embedding we
find words such as “infection, respiratory and organism", while in the
atemporal one there are for example “malware and spyware", that diverge from
the core literal meaning and are related to a modern connotation of the word.
When exploiting the temporal word embedding, the model is able to understand
that the “virus" in this sentence is a computer one, and therefore that it is
used in a metaphorical way along with the verb “attacked".

The paper is organized as follows: Section 2 discusses the related work about
metaphor detection as well as temporal word embeddings. Section 3 discusses
the methodology followed while Section 4 shows the experimental results of the
paper. Finally, Section 5 concludes the paper.

##  2 State of the Art

Metaphors arise due to systematic association between distinct and seemingly
unrelated concepts. The existence of this association allows us to transfer
knowledge and inferences from the domain of mechanisms and discuss them using
the mechanism terminology. The semantic distance between two concepts is,
according to Lakoff and Johnson (1980), what makes a metaphor “more
metaphorical" than another one. Wilks, in his work “Making Preferences more
active" (Wilks, 2015), which is considered the most influential one regarding
metaphor recognition for automatic metaphor recognition in text, affirms that
metaphors are “a violation of semantic constraints put by verbs onto their
arguments". Fass (1997)’ approach “Processing Metonymy and Metaphor" was one
of the first real approaches to metaphor detection, and it consisted in the
distinction between literalness, metonymy, metaphor, and anomaly: phrases
could be tested for being a metonymic relation using hand-coded patterns
(e.g.: container-for-content). One of the biggest problems with this approach
was interpretation being always context-dependent. A phrase could either be
metaphorical or literal, for example: “All men are animals". Wim and Ivonne
Peters (1998) made use of “WordNet" (a large lexical database of English
nouns, verbs, adjectives and adverbs grouped into sets of cognitive synonyms
called synsets, each expressing a distinct concept) hierarchy to group close
senses and to find hyponymy relations. The semantic field of a hyponym is
included within that of its hyperonym, for example: “spoon" is a hyponym of
“cutlery". If two words are not included in WordNet’s hyponymy/hypernymy
hierarchy, then they are most likely part of a metaphorical phrase. CorMet was
the first system ever to automatically discover source-target domain mappings.
Following Fass (1997)’ work, state of the art approaches to metaphor detection
have begun spreading. Shutova (2015)’s survey, “Design and Evaluation of
Metaphor Processing Systems", provides a detailed look of the various methods.

Numerous BiLSTM systems take advantage of both contextualised and pre-trained
embeddings in the classification layer (Mao et al., 2019; Swarnkar and Singh,
2018). In particular, the Di-LSTM Contrast system (Swarnkar and Singh, 2018)
encodes the left and right side context of a target word through forward and
backward LSTMs. The classification is based on a concatenation of the target
word representation and its difference with the encoded context (Xiaoyu Tong,
2021). Mao et al. (2019) combined GloVe (Jeffrey Pennington, 2014) and BiLSTM
hidden states for sequence labelling. Some of the most recently available
systems adopt a fine-tuning approach, employing pre-trained contextual
language models such as Bidirectional Encoder Representations from
Transformers (Devlin J, 2018) (BERT (Devlin et al., 2018) and RoBERTa (Yinhan
Liu and Stoyanov, 2019)), and taking advantage of contextual embeddings (Wang,
2020). For example, Dankers et al. (2020) fine-tuned a BERT (Devlin et al.,
2018) model, which gets a discourse fragment as input. Hierarchical attention
computes both token and sentence level attention (Goro Kobayashi, 2020) after
the encoded layers, leading to better results compared to those obtained by
applying general attention to all tokens.

Temporal language evolution and semantic change computation have long been of
interest to both academic circle and the general public. According to
Tomasello (2003), “[l]anguage is a complex outcome of human cognitive and
social processes taking place in evolutionary, historical and ontogenetic
time. And different aspects of language –for example, symbols and grammar –
may have involved different processes and different evolutionary times.". In
Smith and Höfler (2015b), it is shown that the relevant cognitive competences
are general-purpose mechanisms which are crucially not specific to language;
in fact, they also foster non-linguistic communication, where the same
processes lead to the emergence of apparently arbitrary symbols. Previously,
instead, evolutionary linguists generally agreed that the emergence of human
language, as opposed to other primate communication systems, was characterised
by two key and distinct phenomena, namely the use of symbols, and the use of
grammatical structure (Tomasello, 2003). In the “Historical Linguistics and
Language Change" survey, Lass (1997) explored metaphors as part of his
discussion of standards that should be followed in historiographical
activities, such as reconstructing and explaining. Lass noted that “metaphors
are often thought of as vague sloppy things, as compared with good theories"
(Aitchison, 2003); “But one could argue on the contrary that theory often
(maybe normally) is the formalization of metaphor" (Lass, 1997). By studying
the English Great Vowel Shift, he proves “how metaphors can populate history
with new objects and kinds, and provide both access to interesting new worlds,
and great field-internal success" (Lass, 1997).

Based on the observations that language is always changing and a dynamic flux
(Group’ et al., ; Francesco La Mantia, 2017; Massip-Bonet and Bastardas Boada,
2013), linguists have formulated different theories and models searching for
rules and regularities in semantic change, such as the “Diachronic Prototype
Semantics" (Geeraerts, 1997; Geeraerts et al., 1999), the Invited Inference
Theory of Semantic Change (Traugott and Dasher, 2001), and semantic change
based on metaphor and metonymy (Heine et al., 1991). Semantic change
computation deals with the dynamic change of semantics, including the
phenomenon of polysemy, a perennial challenge for NLP. A better understanding
of polysemy should help improve any NLP task that involves semantic
processing. For instance, applications for semantic search can increase the
relevance of the query result by taking into account the new senses of words
(Mitra et al., 2015; Yao et al., 2018).

Historically, much of the theoretical work on semantic shifts has been devoted
to documenting and categorizing various types of semantic shifts (Breal, 1897;
Stern, 1975). Linguistic processes that cause semantic shifts generally
involve the interaction between words of the vocabulary and their meanings.
Semantic shifts are naturally separated into two important classes:
“linguistic drifts" (slow and steady changes in core meaning of words) and
“cultural shifts" (changes in associations of a given word determined by
cultural influences). Gulordava and Baroni (2011), for instance, showed that
distributional models capture cultural shifts, like the word “sleep" acquiring
more negative connotations related to sleep disorders domain, when comparing
its 1960s contexts to its 1990s contexts. Researchers studying semantic shifts
from a computational point of view have empirically shown the existence of
this division (William L. Hamilton, 2016).

Diachronic corpora provide empirical resources for semantic change
computation. The construction of diachronic corpora is based on different
factors such as size, balance, and representativeness of the corpora in
question. The most frequently used corpus is the Google Books Ngram Corpus
(Lin et al., 2012), due to its size, temporal range, types of language
included in the corpus and public availability. The availability of large
corpora have enabled the development of new methodologies for the study of
lexical semantic shifts within general linguistics (Traugott and Dasher,
2001). A key assumption is that changes in a word’s collocational patterns
reflect changes in word meaning (Hilpert, 2013), thus providing a usage-based
account of semantics (Gries, 1999). The usage-based view of lexical semantics
aligns well with the assumptions underlying the distributional semantic
approach (Firth, 1957) often employed in NLP. To summarize, semantic shifts
are often reflected in large corpora through change in the context of the word
which is affected by the shift, as measured by co-occurring words.

The two main metaphor detection approaches used in literature are:

  1. 1.

Metaphor Detection in Single Sentence: local contextual clues within a short
text are used in order to detect metaphors;

  2. 2.

Metaphor Detection in Discourse: the global context of discourse is used in
order to detect metaphors, thanks to a combination of global and local
features.

Some models interpret the identified metaphors by paraphrasing them into their
literal counterparts so that they can be better translated by machines. This
allows metaphor identification and interpretation in whole sentences (Rui Mao,
2018).

  1. 1.

Metaphor Identification \- this approach is often based on: violation of
selectional preferences (Fass, 1997); clustering (Birke and Sarkar, 2006) and
Ekaterina Shutova (2010); lexical relations in WordNet (Saisuresh
Krishnakumaran, ); contrast between literal and non-literal use of a target
expression in text and source-target domain mappings (Mason, 2004). Recent
advances have been made by proposing neural network base methods (cf. Section
3.2).

  2. 2.

Metaphor Interpretation \- Ekaterina Shutova, in 2013 (Bollegala and Shutova,
2013), performed Metaphor Identification and Interpretation simultaneously. At
first, much of the computational work had focused on detecting and uncovering
the intended meaning behind metaphors.

In MIP (Metaphor Identification Procedure, (Group, 1997)) and SPV (Selectional
Preferences Violation, (Wilks, 2015)) procedures, the modeling of the contrast
between literal and contextual representations of metaphors is theoretically
similar: humans are usually able to infer the contextual meanings of a word
conditioned on its context. While MIP is based on the contrast between a
word’s contextual and literal meanings, SPV is based on the contrast between a
target word and its context.

##  3 Methodology

###  3.1 Intuition

In MIP (Group, 1997) and SPV (Wilks, 2015), the modeling of the contrast
between literal and contextual representations of metaphors is theoretically
similar. In Neural Metaphor Detection in Context, Ge Gao et al. (Ge Gao and
Zettlemoyer, 2018) presented end-to-end neural models for detecting
metaphorical words used in a context. They showed that BiLSTM (Graves and
Schmidhuber, 2005) models which operate on complete sentences work well for
the task of metaphor detection. These models establish a new state-of-the-art
on existing verb metaphor detection benchmarks and show strong performance on
simultaneously predicting the metaphoricity of all words in a running text.
Two common task formulations are investigated:

  * •

Given a target verb in a sentence, classifying whether it is metaphorical or
not;

  * •

Given a sentence, detecting all of the metaphorical words (independent of
their POS tags).

In BiLSTM, the hidden states, used as contextual meaning representations, are
extracted through the forward and the backward contexts and itself: Graves and
Schmidhuber’s work (Graves and Schmidhuber, 2005) was the first one ever with
bidirectional training being applied on a LSTM network. The input is presented
forwards and backward to two separate recurrent networks, both of which are
connected to the same output layer. Pretrained GloVe (Jeffrey Pennington,
2014) representations of each dataset are used as the input, i.e., literal
meaning representation, as words have been embedded with their most common
senses (trained on Web crawled data). The main idea of our work is to replace
the traditional GloVe embeddings used in the two state-of-the-art models
discussed in the previous section (Ge Gao and Zettlemoyer, 2018) with the SGNS
Historical Word2Vec ones, and see whether temporal semantic information can
enhance metaphor detection. Each temporal embeddings’ slice spans a decade.
The state-of-the-art models must be modified in order to be able to replace
the GloVe embedding (the literal representation) with the new embeddings.

###  3.2 Selected Metaphor Detection Approaches

The state-of-the-art models that have been modified to conduct the experiments
with temporal embeddings are the ones used for the End-to-End Sequential
metaphor Identification task in Ge Gao and Zettlemoyer (2018):

  1. 1.

Recurrent Neural Network Hidden GloVe (RNN HG): Based on the MIP (the metaphor
is classified by the contrast between a word’s contextual and literal
meanings) metaphor identification procedure, here the GloVe embeddings serve
as the literal representation and joins the hidden state from the BiLSTM
(contextual representation).

Figure 1: RNN_HG model architecture based on MIP procedure, with tensors
concatenated along the last dimension.

In the RNN HG model, The GloVe and ELMO (applied in the same way as in Ge Gao
and Zettlemoyer (2018) and concatenated with GloVe (Peters M, 2018) inputs
feed the BiLSTM network. Then literal and contextual representations get
compared in the so-called comparison stage. The BiLSTM has a hidden state
which is the contextual representation of the sentences: this hidden state
${h_{t}}$ joins the input literal representations and is:

$\left.h_{t}=f_{BiLSTM}(\left[g_{t};e_{t}\right]\vec{h}_{t-1},\vec{h}_{t+1}\right)$
(1)

Two vectors that are two different representations of the same word (literal
and contextual) and that are located in two different encoding spaces are
concatenated. The last step consists of the softmax function $\sigma$, which
calculates the probability of a label prediction $\hat{y}$ for a target word
at position t, conditioned on both its contextual and literal meaning
representations. This is expressed by the equation:

$p\left(\hat{y}_{t}\mid
h_{t},g_{t}\right)=\sigma\left(w^{\top}\left(h_{t};g_{t}\right)+b\right)$ (2)

  2. 2.

Recurrent Neural Network Multi-Head Context Attention (RNN MHCA): based on the
SPV procedure, here a label prediction is conditioned on a hidden state of a
target word and its attentive context representation. The target word
representation is compared with its context by concatenating them, just like
GloVe and ELMO (Peters M, 2018) were in the Hidden Glove model.

The RNN HG classifier, after training, understands whether these two
aforementioned vectors portray similar or different meanings: in the first
scenario, the sense will be literal, while in the latter a metaphor will be
detected. In the Recurrent Neural Network MHCA model’s architecture instead,
the contextual embedding is used as the target word representation and gets
concatenated with the word’s attentive context representation. These two
representations can be located in the same encoding space (leading to better
results) or in two different ones, as it happens for the RNN HG model. After a
few initial experiments, we noticed that the RNN HG model provides better
results with temporal embeddings, and therefore we stopped using the RNN MHCA
model for the remaining tests.

###  3.3 Temporal Embeddings and Embedding Training Corpora

The Corpus of Historical American English (CoHa) is the largest structured
corpus of historical English, and it is related to many other corpora of
English. The CoHa corpus contains more than 400 million words of text from the
1820s-2000s (which makes it 50-100 times as large as other comparable
historical corpora of English). We bought the full CoHa corpus and we used its
linear text format data: this format gives a textID for each text, and then
the entire text on the same line. Here words are not annotated for part of
speech or lemma.

HistWords is a collection of tools and datasets for analyzing language change
using word embeddings (William L. Hamilton, 2018). The historical word vectors
in HistWords were used to study the semantic evolution of more than 30,000
words across four languages. This lead to the formulation of the law of
conformity (words that are used with more frequency change less and are
characterized by more stable meanings over time) and the law of innovation
(polysemous words, that have more than one meaning and change at faster
rates), (William L. Hamilton, 2016). HistWords embeddings were originally
aligned using Procrustes alignment (Edouard Grave, 2018), one of the oldest
alignment methods in the literature, which takes on the task of aligning two
sets of points in high dimension (which has many applications in natural
language processing), through the joint estimation of an orthogonal matrix and
a permutation matrix. A stochastic algorithm is proposed to minimize the cost
function on large-scale problems, and the method is finally evaluated on the
problem of unsupervised word translation. This is done by aligning word
embeddings trained on monolingual data.

We also used a newer alignment method, CADE - Compass Alignment (Di Carlo et
al., 2019; Bianchi et al., 2020), an extension of one of the two original
Word2vec models (Tomas Mikolov and Dean, 2013a, b), that can be applied to
sliced corpora in order to create a set of aligned word embeddings. In order
to conduct the experiments shown in this paper, pretrained historical word
embeddings for the English language spanning all decades from 1800 to 2000
have been used. The four main SGNS HistWords embeddings are the following:

  1. 1.

All English (1800s-1990s) (from Google N-Grams eng-all): These datasets were
created in July 2012 (Version 2) and July 2009 (Version 1) - Each of the files
constituting the corpora is compressed tab-separated data. In Version 2 each
line has the following format - ngram, year, match_count, volume_count

  2. 2.

English Fiction (1800s-1990s) (from Google N-Grams eng-fiction-all): see
description above

  3. 3.

Genre-Balanced American English (1830s-2000s) (from Corpus of Historical
American English)

  4. 4.

Genre-Balanced American English, word lemmas (1830s-2000s) (from Corpus of
Historical American English)

###  3.4 Datasets

Table 1: Dataset Statistics Dataset | N. of sentences | Train/Test/Split | Temporal Annotation? |  Creation Detail
---|---|---|---|---
MOH-X | 646 | No | No |  Derived from MOH dataset. The verbs are used as metaphors.
VUAsequence | 5323 | Yes | Yes (1985-1994) |  117 fragments sampled across 4 genres from British National Corpus (academic, news, conversation, and fiction)
TroFi | 3737 | No | Yes (1987-1989) |  The sentences (each one with a single annotated target verb) are taken from ‘87-‘89 Wall Street Journal Corpus.

For showing the feasibility of the proposed claims the experiments have been
performed on three datasets. Table 1 provides a glimpse of their main
characteristics.

MOH-X dataset (Saif Mohammad, 2016) is derived from the subset of MOH dataset
used by Ekaterina et. al (Ekaterina Shutova, 2016). Mohammad et al. annotated
different senses of WordNet (Rui Mao, 2018) verbs for metaphoricity. They
extracted verbs that had between three and ten senses in WordNet along with
their glosses. The verbs were annotated for metaphoricity with the help of
crowd-sourcing. Ten annotators were recruited for each sentence and only those
verbs were selected which were annotated positive for metaphoricity by at
least 70% of the annotators. The final dataset consisted of 647 verb-noun
pairs, 316 metaphorical, and 331 literal.

VUA dataset consists of 117 fragments sampled across four genres from the
British National Corpus, i.e., Academic, News, Conversation, and Fiction (Chee
Wee, Ben). The data was annotated using the MIP-VU procedure (Gerard Steen,
2010) which is based on the MIP procedure (Group, 1997). The tagset defined by
MIP procedure is rich and organized hierarchically, detecting various types of
metaphors, words that flag the pre-sense of metaphors, etc. The majority of
sentences in this dataset have the timestamp for the decade 1985-1994.

The TroFi dataset contains feature lists consisting of the stemmed nouns and
verbs in a sentence, with target or seed words. The frequent words consist of
332 most frequent words in the British National Corpus along with the
contractions, single letters, and numbers ranging from 0 to 10. These frequent
words have been removed from the TroFi dataset. The target set is built using
the ‘88-‘89 Wall Street Journal Corpus (WSJ) tagged using the Ratnaparkhi
(1996) tagger and the Joshi (1999) SuperTagger.

##  4 Experiments

###  4.1 Experimental Design

For the quantitative experiments, we used the previously introduced RNN-HG and
RNN-MHCA models discussed in Section 3 with the datasets discussed in Section
3.4. We use word embeddings (GloVe), contextual embeddings, and especially
temporal word embeddings such as Procrustes and CADE aligned models. We
implemented the aforementioned representations inside the state-of-the-art
Recurrent Neural Network Hidden GloVe and Multi-Head Context Attention models,
in conjunction with the ELMO vectors and replacing the standard GloVe
embeddings. Pre-trained ELMO representations for each of the three datasets
were used. RNN-HG and RNN-MHCA models are used to perform metaphor detection
as a sequence classification task, in order to detect single metaphorical
words (nouns, verbs, adjectives, and so on). The following research questions
were targeted:

  1. 1.

Can the results of the state-of-the-art metaphor detection algorithms be
improved by using different word embeddings, especially temporal embeddings
such as HistWords - SGNS?

  2. 2.

Are there any observable patterns that can lead to the assumption that
metaphor detection tasks performed with temporal embeddings and
representations impact datasets with known temporal connotations more than
others?

  3. 3.

Are representations obtained through CADE and Compass alignment more effective
for metaphor detection than embeddings aligned with traditional methods (e.g.:
Procrustes)?

  4. 4.

Are specific word embeddings’ architectures more effective for metaphor
detection tasks than others?

The first experiments have been performed using the HistWords - SGNS temporal
embeddings (English All, English-Fiction, CoHa Word, CoHa Lemma). The datasets
discussed in Section 3.4 were used for evaluation. At this point, in order to
see whether it was indeed the temporal connotation of the SGNS embeddings that
improved metaphor detection task performances, or just their Word2Vec
architecture as opposed to one of GloVe vectors, previous results have been
compared to those obtained using a-temporal Word2Vec embeddings trained on the
Wikipedia corpus. Finally, experiments have been performed using word
embeddings obtained by aligning all different decades slices of the CoHa
corpus (ranging from 1820 to 2000) with CADE - Compass alignment method.
Slices of the CoHa corpus for each decade needed to be aligned with CADE -
Compass in order to perform equivalent experiments to the ones with other
embeddings. The CoHa corpus in text format is composed of each decade ranging
from 1820 to 2000. The first step consisted in concatenating all text files,
obtaining a final corpus for all the decades. The pre-processing procedures
applied to clean all the resulting corpora were: stripping HTML tags; removing
text between square brackets; replacing all contractions; removing stopwords;
cleaning the resulting text through regex expressions. At this point, the
alignment using CADE and Compass could be performed through the following
steps:

  1. 1.

Creating the main compass file by concatenating all the processed CoHa decade
slices;

  2. 2.

Training the obtained compass-aligned embeddings;

  3. 3.

Training all the different slices from the compass obtaining their respective
models;

  4. 4.

Converting the CoHA compass models in Word2Vec format, so that they could have
the same architecture of the HistWords - SGNS and Wikipedia embeddings, and be
exploited inside the modified Recurrent Neural Network model.

Only the aligned models of the decades slices ranging from 1900 to 2000 were
kept so that the results could be comparable to the previous ones. A Full CoHa
Word CADE model was also obtained by training the compass on all the
aforementioned decade slices: this will also be used for qualitative analyses.

#### Qualitative Analysis.

For our qualitative analyses, we took a closer look at the actual predictions
made by the RNN HG model with the different types of embeddings used so far,
analyzing the characteristics of the words that were correctly or mistakenly
identified as metaphors. Since the huge range of our previous experiments, it
was not feasible to inspect all the results. We finally decided to focus on
MOH-X, VUA, and TroFi datasets’ predictions obtained with 4 specific
embeddings:

  1. 1.

Full CoHa Word CADE;

  2. 2.

GloVe (state of the art representation);

  3. 3.

CoHa Word CADE 1990 Decade Slice;

  4. 4.

CoHa Word HistWords SGNS 1990 Decade Slice.

The first two representations are trained on full corpora (in particular the
Full CoHa CADE one spans the 19th and the 20th centuries). The last two
representations, instead, were obtained from a specific slice. We chose the
1990 decade slice because of VUA and TroFi datasets’ sentences temporal
connotations, because this slice generally provided good quantitative
performances, and to make the qualitative analyses of these two embeddings
comparable. Therefore, the usage of these 4 corpora allowed us to look at
predictions made by the RNN HG model with both temporal and a-temporal
embeddings, and with both full and singular corpora slices. In order to check
all the predictions made by our model for MOH-X and TroFi, we combined each
one of their 10-folds intermediate results, since these two datasets are not
split into train, validation, and test sets like VUA. For each one of the
three state of the art datasets, we checked:

  1. 1.

Correctly identified metaphors;

  2. 2.

Mistakenly identified metaphors.

The different focuses of the qualitative analysis for the three datasets were
the following:

  1. 1.

MOH-X: verbs (a column for nouns is also given in the dataset, but they are
not included in the evaluation labels);

  2. 2.

VUA: most frequent patterns of parts-of-speech’ positions in the sentences and
genres of the sentences coming from the British National Corpus;

  3. 3.

TroFi: verbs.

These analyses were mostly performed to check MOH-X and TroFi predictions
since they provide precise information about metaphorical verbs and nouns.

###  4.2 Quantitative Results

Tables 2, 3, and 4 provide the overall quantitative performances and scores
for each dataset. Combining the observations gathered from all the performed
experiments, following conclusions can be drawn:

  1. 1.

Word2vec architecture (HistWords - SGNS, CoHa and Wikipedia embeddings) works
better than GloVe architecture for metaphor detection;

  2. 2.

HistWords - SGNS embeddings achieve better results than GloVe;

  3. 3.

HistWords - SGNS temporal embeddings perform better on the datasets with known
temporal connotations (TroFi and VUA) compared to the a-temporal Wikipedia
embeddings;

  4. 4.

MOH-X dataset is generally the one that achieves better results with the
different types of word representations;

  5. 5.

Procrustes alignment (HistWords - SGNS) and CADE - Compass alignment methods
(CoHa corpus) lead to similar performances and results. Although, while the
latter performs better on the TroFi dataset (data extracted from Wall Street
Journal corpus), the first one impacts slightly more the VUA dataset (data
extracted from the British National Corpus).

Figure 2: Results related to MOH-X dataset, with every single embedding.
Figure 3: Results related to VUA dataset, with every single embedding.  Figure
4: Results related to TroFi dataset, with every single embedding.

###  4.3 Qualitative Results

The various qualitative analyses that we simultaneously performed both on the
predictions of all three state-of-the-art metaphor datasets and on the
embeddings themselves, revealed and confirmed several patterns. The most
important ones are the following: Topics related to economics, politics, and
emotions are the most recurring ones in sentences containing correctly
identified metaphors. We could also notice this in the fine-tuning experiments
with contextual embeddings, even though they were performed using a different
test set, so this is a very important confirmation. Verbs having a literal
meaning characterized by physical connotations, often assume
metaphorical/figurative meanings when used in sentences related to the
contexts listed before. This pattern was at first observed especially in TroFi
predictions, but thanks to the nearest neighbors analyses we were able to
detect it even in the other datasets. In several cases, words that are similar
to verbs or nouns related to one of the aforementioned contexts, are found in
sentences alongside correctly classified metaphorical words that are connected
to another one of those most recurring topics. Different results related to
the domains of the sentences have been observed in the VUA dataset’s
predictions. With Full CoHa CADE embedding, only one sentence belonging to the
academic genre was correctly classified, whereas as far as CoHa SGNS 1990
slice is concerned, no sentences belonging to the news genre were correctly
predicted. The latter result could indicate that for that specific time
period, SGNS words’ representations of the news genre are biased towards their
metaphorical meaning (words are used in metaphorical contexts way more than in
literal ones). This would prevent the RNN-based models from correctly
identifying the words as metaphors. In fact, the models generally recognize
metaphors because of the mismatch between the words’ signals and those of the
contexts of the sentences in which they are located.

##  5 Discussion and Conclusion

In the previous sections, we analyzed the results of our work. As an
additional step to evaluate the impact of temporal embeddings on the evolution
of language and the meaning of the words, we also performed our experiments
using the words with the biggest semantic shift across time. The words’ list
was retrieved from the SemEval 2020 Task 1: Unsupervised Lexical Semantic
Change Detection
Competition222https://competitions.codalab.org/competitions/20948#participate-
get_data.We tried building a custom dataset by searching for metaphorical and
literal sentences that would contain all these words so that it would be
possible to perform metaphor detection tasks exploiting different temporal
embeddings. Unfortunately, this final experiment was not feasible, due to the
fact that there were not enough metaphorical statements in the competition
dataset or in the state-of-the-art ones with the aforementioned words being
used in a proper way. We performed metaphor detection as a sequence
classification task in order to identify words with figurative meanings inside
sentences. This approach allowed us to take advantage of different types of
word representations, especially temporal ones, and evaluate their impact on
metaphor detection.

Looking at the numerous and diversified results, we can affirm that temporal
word embeddings do generally improve the performance of the task of metaphor
detection, however, their overall impact is rather limited. Besides,
independently from the absolute performance, the interaction effect between
the specificity of the embeddings (especially their temporal specificity) and
metaphor detection is found in the experiments conducted in this study. In
fact, these experiments verify that if the core meaning of the words of
interest in a sentence is too similar to their figurative one in the word
embedding, a metaphorical sentence could get misclassified as literal.
Moreover, when temporal word embeddings provide words’ representations that
are more inclined towards their literal core meaning (and not the metaphorical
one), models exploiting end up correctly identifying metaphors more easily.
Furthermore, embeddings of words belonging to some language domains in
specific time periods can be biased towards their metaphorical meaning,
leading to words being used in metaphorical contexts way more than in literal
ones. This would prevent neural models from correctly identifying the words as
metaphors. We could verify these hypotheses by examining the examples analyzed
in section 1. We explored the nearest neighbors of the target metaphorical
words of sentences extracted from state of the art datasets, both in a static
and atemporal word embedding such as GloVe (Jeffrey Pennington, 2014) and CoHa
SGNS respectively. When exploiting the temporal word embedding, the model
could correctly understand that the words (in our examples: “apple, virus,
attack, hearts, and glow") were used in a figurative way, thus correctly
classifying them as metaphors. Furthermore, the entire sentences were
correctly classified as metaphorical too, since the words’ core meanings were
closer to their literal core ones. This also explains the fluctuations across
slices, because corpora are never fully representative, and some contexts may
be represented more than others in one specific decade.

Finally, we performed qualitative analyses on the predictions made by sequence
classification neural networks-based approaches and fine-tuned models for
metaphor detection. The results suggest that topics related to economics,
politics, and emotions are the most recurring ones in sentences containing
correctly identified metaphors and that verbs having a literal meaning
characterized by physical connotations, often assume a metaphorical meaning
when used in sentences related to the aforementioned contexts.

## References

  * Recupero et al. [2019] Diego Reforgiato Recupero, Mehwish Alam, Davide Buscaldi, Aude Grezka, and Farideh Tavazoee.  Frame-based detection of figurative language in tweets [application notes].  _IEEE Comput. Intell. Mag._ , 14(4):77–88, 2019\.  doi:10.1109/MCI.2019.2937614.  URL https://doi.org/10.1109/MCI.2019.2937614.
  * Shutova [2015] Ekaterina Shutova.  Design and evaluation of metaphor processing systems.  _Computational Linguistics_ , 41, December 2015.
  * Leong et al. [2020] Chee Wee (Ben) Leong, Beata Beigman Klebanov, Chris Hamill, Egon Stemle, Rutuja Ubale, and Xianyang Chen.  A report on the 2020 VUA and TOEFL metaphor detection shared task.  In _Proceedings of the Second Workshop on Figurative Language Processing_ , pages 18–29, Online, July 2020. Association for Computational Linguistics.  doi:10.18653/v1/2020.figlang-1.3.  URL https://aclanthology.org/2020.figlang-1.3.
  * Lakoff and Johnson [1980] George Lakoff and Mark Johnson.  _L aTeX: Metaphors we live by._ Online, 1980.
  * Saif Mohammad [2016] Peter Turney Saif Mohammad, Ekaterina Shutova.  Metaphor as a medium for emotion: An empirical study.  _Proceedings of the Fifth Joint Conference on Lexical and Computational Semantics_ , page 23–33, August 2016.
  * Almeida and Xexéo [2019] Felipe Almeida and Geraldo Xexéo.  Word embeddings: A survey, 01 2019.
  * Mao et al. [2019] Rui Mao, Chenghua Lin, and Frank Guerin.  End-to-end sequential metaphor identification inspired by linguistic theories.  In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_ , pages 3888–3898, Florence, Italy, July 2019. Association for Computational Linguistics.  doi:10.18653/v1/P19-1378.  URL https://aclanthology.org/P19-1378.
  * Swarnkar and Singh [2018] Krishnkant Swarnkar and Anil Kumar Singh.  Di-LSTM contrast : A deep neural network for metaphor detection.  In _Proceedings of the Workshop on Figurative Language Processing_ , pages 115–120, New Orleans, Louisiana, June 2018. Association for Computational Linguistics.  doi:10.18653/v1/W18-0914.  URL https://aclanthology.org/W18-0914.
  * Gulordava and Baroni [2011] Kristina Gulordava and Marco Baroni.  A distributional similarity approach to the detection of semantic change in the google books ngram corpus.  pages 67–71, 07 2011.
  * Tomas Mikolov and Dean [2013a] Greg Corrado Tomas Mikolov, Kai Chen and Jeffrey Dean.  Efficient estimation of word representations in vector space.  _In 1st International Conference on Learning Representations, ICLR 2013, Scottsdale, Arizona, USA, Workshop Track Proceedings._ , 2013a.
  * Jeffrey Pennington [2014] Christopher D. Manning Jeffrey Pennington, Richard Socher.  Glove: Global vectors for word representation.  _Empirical Methods in Natural Language Processing (EMNLP)_ , pages 1532––1543, 2014.
  * Peters M [2018] Iyyer M Gardner M Clark C Lee K Zettlemoyer L. Peters M, Neumann M.  Deep contextualized word representations.  _Proceedings of the 2018 conference of the north American chapter of the association for computational linguistics: human language technologies, Association for Computational Linguistics, New Orleans, Louisiana_ , 1 (Long Papers):2227––2237, 2018.
  * Devlin et al. [2018] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova.  Bert: Pre-training of deep bidirectional transformers for language understanding.  10 2018.
  * William L. Hamilton [2018] Dan Jurafsky William L. Hamilton, Jure Leskovec.  Diachronic word embeddings reveal statistical laws of semantic change.  2018\.
  * Smith and Höfler [2015a] Andrew Smith and Stefan Höfler.  _The pivotal role of metaphor in the evolution of human language_ , pages 123–.  January 2015a.  ISBN 978-3-11-033543-9.  doi:10.1515/9783110335453.123.
  * Aitchison [2003] Jean Aitchison.  _Motives for Language Change_.  Cambridge University Press, 2003.  doi:10.1017/CBO9780511486937.
  * Ge Gao and Zettlemoyer [2018] Yejin Choi Ge Gao, Eunsol Choi and Luke Zettlemoyer.  Neural metaphor detection in context.  _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_ , pages 607–613, 2018.
  * Edouard Grave [2018] Quentin Berthet Edouard Grave, Armand Joulin.  Unsupervised alignment of embeddings with wasserstein procrustes.  May 2018.
  * Di Carlo et al. [2019] Valerio Di Carlo, Federico Bianchi, and Matteo Palmonari.  Training temporal word embeddings with a compass.  _Proceedings of the AAAI Conference on Artificial Intelligence_ , 33(01):6326–6334, Jul. 2019.  doi:10.1609/aaai.v33i01.33016326.  URL https://ojs.aaai.org/index.php/AAAI/article/view/4594.
  * Bianchi et al. [2020] Federico Bianchi, Valerio Di Carlo, Paolo Nicoli, and Matteo Palmonari.  Compass-aligned distributional embeddings for studying semantic differences across corpora, 2020.
  * Wilks [2015] Yorick Wilks.  Making preferences more active.  _Artificial Intelligence_ , 11(3):197–223, December 2015.
  * Fass [1997] Dan Fass.  Processing metonymy and metaphor.  _Contemporary Studies in Cognitive Science & Technology_, 1997.
  * Wim and Ivonne Peters [1998] Wim and Piek Vossen Ivonne Peters.  Automatic sense clustering in eurowordnet.  _Artificial Intelligence_ , 1998.
  * Xiaoyu Tong [2021] Martha Lewis Xiaoyu Tong, Ekaterina Shutova.  Recent advances in neural metaphor processing: A linguistic, cognitive and social perspective.  _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , page 4673–4686, June 6–11 2021.
  * Devlin J [2018] Lee K Toutanova K. Devlin J, Chang MW.  Bert: Pre-training of deep bidirectional transformers for language understanding.  _arXiv preprint arXiv:1810.04805_ , 2018.
  * Yinhan Liu and Stoyanov [2019] Naman Goyal Jingfei Du Mandar Joshi Danqi Chen Omer Levy Mike Lewis Luke Zettlemoyer Yinhan Liu, Myle Ott and Veselin Stoyanov.  Roberta: A robustly optimized bert pretraining approach.  2019\.
  * Wang [2020] Hou Y. Che W. et al. Wang, Y.  From static to dynamic word representations: a survey.  _Int. J. Mach. Learn. & Cyber_, 11:1611––1630, 2020\.
  * Dankers et al. [2020] Verna Dankers, Karan Malhotra, Gaurav Kudva, Volodymyr Medentsiy, and Ekaterina Shutova.  Being neighbourly: Neural metaphor identification in discourse.  In _Proceedings of the Second Workshop on Figurative Language Processing_ , pages 227–234, Online, July 2020. Association for Computational Linguistics.  doi:10.18653/v1/2020.figlang-1.31.  URL https://aclanthology.org/2020.figlang-1.31.
  * Goro Kobayashi [2020] Sho Yokoi Kentaro Inui Goro Kobayashi, Tatsuki Kuribayashi1.  Attention is not only a weight: Analyzing transformers with vector norms.  pages 7057–7075, 2020.
  * Tomasello [2003] Michael Tomasello.  Constructing a language: A usage based theory of language acquisition.  8, 01 2003.
  * Smith and Höfler [2015b] Andrew Smith and Stefan Höfler.  _The pivotal role of metaphor in the evolution of human language_ , pages 123–.  01 2015b.  ISBN 978-3-11-033543-9.  doi:10.1515/9783110335453.123.
  * Lass [1997] Roger Lass.  _Historical Linguistics and Language Change_.  Cambridge Studies in Linguistics. Cambridge University Press, 1997.  doi:10.1017/CBO9780511620928.
  * [33] The ’Five Graces Group’, Clay Beckner, Richard Blythe, Joan Bybee, Morten H. Christiansen, William Croft, Nick C. Ellis, John Holland, Jinyun Ke, Diane Larsen-Freeman, and Tom Schoenemann.  Language is a complex adaptive system: Position paper.  _Language Learning_ , 59(s1):1–26.  doi:https://doi.org/10.1111/j.1467-9922.2009.00533.x.  URL https://onlinelibrary.wiley.com/doi/abs/10.1111/j.1467-9922.2009.00533.x.
  * Francesco La Mantia [2017] Pietro Perconti Francesco La Mantia, Ignazio Licata.  Language in complexity.  _Lecture Notes in Morphogenesis_ , 2017.
  * Massip-Bonet and Bastardas Boada [2013] Àngels Massip-Bonet and Albert Bastardas Boada.  _Complexity perspectives on language, communication and society_.  01 2013.  ISBN 978-3-642-32816-9.
  * Geeraerts [1997] Dirk Geeraerts.  _Diachronic Prototype Semantics: A Contribution to Historical Lexicology_.  Oxford University Press, 1997.
  * Geeraerts et al. [1999] Dirk Geeraerts, Stefan Grondelaers, and Dirk Speelman.  _Convergentie en divergentie in de Nederlandse woordenschat. Een onderzoek naar kleding- en voetbaltermen._ 01 1999.  ISBN ISBN 90 70389 60 6.
  * Traugott and Dasher [2001] Elizabeth Closs Traugott and Richard B. Dasher.  _Regularity in Semantic Change_.  Cambridge Studies in Linguistics. Cambridge University Press, 2001.  doi:10.1017/CBO9780511486500.
  * Heine et al. [1991] Bernd Heine, Ulrike Claudi, and Friederike Hunnemeyer.  Grammaticalization: A conceptual framework.  _Bibliovault OAI Repository, the University of Chicago Press_ , 01 1991.
  * Mitra et al. [2015] Sunny Mitra, Ritwik Mitra, Suman Maity, Martin Riedl, Chris Biemann, Pawan Goyal, and Animesh Mukherjee.  An automatic approach to identify word sense changes in text media across timescales.  _Natural Language Engineering_ , -1:1–26, 04 2015.  doi:10.1017/S135132491500011X.
  * Yao et al. [2018] Zijun Yao, Yifan Sun, Weicong Ding, Nikhil Rao, and Hui Xiong.  Dynamic word embeddings for evolving semantic discovery.  _Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining_ , Feb 2018.  doi:10.1145/3159652.3159703.  URL http://dx.doi.org/10.1145/3159652.3159703.
  * Breal [1897] Michel Breal.  _Essai de semantique (Science des significations)_.  Hachette Paris, 1897.
  * Stern [1975] Gustaf Stern.  _Meaning and Change of Meaning: With Special Reference to the English Language_.  Greenwood Press, 1975.
  * William L. Hamilton [2016] Dan Jurafsky William L. Hamilton, Jure Leskovec.  Cultural shift or linguistic drift? comparing two computational measures of semantic change.  _In Proceedings of the Conference on Empirical Methods in Natural Language Processing_ , page 2116–2121, 2016.
  * Lin et al. [2012] Yuri Lin, Jean-Baptiste Michel, Erez Aiden, Jon Orwant, Will Brockman, and Slav Petrov.  Syntactic annotations for the google books ngram corpus.  volume July, pages 169–174, 07 2012.
  * Hilpert [2013] Martin Hilpert.  _Germanic Future Constructions: A usage-based approach to language change_.  01 2013.  doi:10.1075/cal.7.
  * Gries [1999] Stefan Gries.  Particle movement: A cognitive and functional approach.  _Cognitive Linguistics_ , 10, 01 1999.  doi:10.1515/cogl.1999.005.
  * Firth [1957] J. R. Firth.  A synopsis of linguistic theory 1930-55.  1952-59:1–32, 1957.
  * Rui Mao [2018] Frank Guerin Rui Mao, Chenghua Lin.  Word embedding and wordnet based metaphor identification and interpretation.  _In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics_ , Volume 1: Long Papers:1222–1231, 2018\.
  * Birke and Sarkar [2006] Julia Birke and Anoop Sarkar.  A clustering approach for the nearly unsupervised recognition of nonliteral language.  _Conference: EACL 2006, 11st Conference of the European Chapter of the Association for Computational Linguistics, Proceedings of the Conference, April 3-7, 2006, Trento, Italy_ , January 2006.
  * Ekaterina Shutova [2010] Lin Sun Ekaterina Shutova.  Metaphor identification using verb and noun clustering.  _In COLING 2010, 23rd International Conference on Computational Linguistics, Proceedings of the Conference_ , pages 23–27, August 2010.
  * [52] Xiaojin Zhu Saisuresh Krishnakumaran.  Hunting elusive metaphors using lexical resources.  _In Proceedings of the Workshop on Computational Approaches to Figurative Language_ , pages 13–20.
  * Mason [2004] Zachary J. Mason.  Cormet: A computational, corpus-based conventional metaphor extraction system.  March 2004.
  * Bollegala and Shutova [2013] Danushka Bollegala and Ekaterina Shutova.  Metaphor interpretation using paraphrases extracted from the web.  _PLOS ONE_ , 8(9):1–10, 09 2013.  doi:10.1371/journal.pone.0074304.  URL https://doi.org/10.1371/journal.pone.0074304.
  * Group [1997] Pragglejaz Group.  Mip: A method for identifying metaphorically used words in discourse.  1997\.
  * Graves and Schmidhuber [2005] Alex Graves and Jürgen Schmidhuber.  Framewise phoneme classification with bidirectional lstm and other neural network architectures.  _Neural Networks_ , 18(5)(ISSN 0893-6080):602–610, 2005.
  * Tomas Mikolov and Dean [2013b] Kai Chen Greg S Corrado Tomas Mikolov, Ilya Sutskever and Jeff Dean.  Distributed representations of words and phrases and their compositionality.  _In C. J. C. Burges, L. Bottou, M. Welling, Z. Ghahramani, and K. Q. Weinberger, editors, Advances in Neural Information Processing Systems_ , 26:3111–3119, 2013b.
  * Ekaterina Shutova [2016] Jean Maillard Ekaterina Shutova, Douwe Kiela.  Black holes and white rabbits: Metaphor identification with visual features.  _Proceedings of NAACL-HLT 2016, pages 160–170, San Diego, California, June 12-17, 2016. c 2016 Association for Computational Linguistics_ , pages 160–170, June 2016.
  * Chee Wee [Ben] Ekaterina Shutova Chee Wee (Ben) Leong, Beata Beigman Klebanov.  A report on the 2018 vua metaphor detection shared task.  In _Proceedings of the Workshop on Figurative Language Processing_ , pages 56–66, Online, 2018.
  * Gerard Steen [2010] Anna Kaal J. Berenike Herrmann Gerard Steen, Lettie Dorst.  A method for linguistic metaphor identification: From mip to mipvu.  2010\.
  * Ratnaparkhi [1996] Adwait Ratnaparkhi.  A maximum entropy model for part-of-speech tagging.  _Conference on Empirical Methods in Natural Language Processing_ , 1996.
  * Joshi [1999] Aravind K. Joshi.  Supertagging: An approach to almost parsing.  1999\.