# Making “fetch” happen: The influence of social and linguistic context on
nonstandard word growth and decline

Ian Stewart and Jacob Eisenstein
School of Interactive Computing
Georgia Institute of Technology
Atlanta, GA 30318
<EMAIL_ADDRESS>
###### Abstract

In an online community, new words come and go: today’s haha may be replaced by
tomorrow’s lol. Changes in online writing are usually studied as a social
process, with innovations diffusing through a network of individuals in a
speech community. But unlike other types of innovation, language change is
shaped and constrained by the grammatical system in which it takes part. To
investigate the role of social and structural factors in language change, we
undertake a large-scale analysis of the frequencies of nonstandard words in
Reddit. Dissemination across many linguistic contexts is a predictor of
success: words that appear in more linguistic contexts grow faster and survive
longer. Furthermore, social dissemination plays a less important role in
explaining word growth and decline than previously hypothesized.

##  1 Introduction

Stop trying to make “fetch” happen! It’s not going to happen! – Regina George
(_Mean Girls_ , 2005)

With the fast-paced and ephemeral nature of online discourse, language change
in online writing is both prevalent Androutsopoulos (2011) and noticeable
Squires (2010). In social media, new words emerge constantly to replace even
basic expressions such as laughter: today’s haha is tomorrow’s lol Tagliamonte
and Denis (2008). Why do some nonstandard words, like lol, succeed and spread
to new contexts, while others, like fetch, fail to catch on? Can a word’s
growth be predicted from patterns of usage during its early days?

Language change can be treated like other social innovations, such as the
spread of hyperlinks Bakshy et al. (2011) or hashtags Romero et al. (2011);
Tsur and Rappoport (2015). A key aspect of the adoption of a new practice is
its _dissemination_ : is it used by many people, and in many social contexts?
High dissemination enables words to achieve greater exposure among social
groups Altmann et al. (2011), and may signal that the innovation is positively
evaluated.

In addition to social constraints, language change is also shaped by
grammatical constraints D’Arcy and Tagliamonte (2015). New words and phrases
rarely change the rules of the game but must instead find their place in a
competitive ecosystem with finely-differentiated linguistic roles, or “niches”
MacWhinney (1989). Some words become valid in a broad range of linguistic
contexts, while others remain bound to a small number of fixed expressions. We
therefore posit a structural analogue to social dissemination, which we call
_linguistic dissemination_.

We compare the fates of such words to determine how linguistic and social
dissemination each relate to word growth, focusing on the adoption of
nonstandard words in the popular online community Reddit. The following
hypotheses are evaluated:

  * •

H1: Nonstandard words with higher initial social dissemination are more likely
to grow. Following the intuition that words require a large social base to
succeed, we hypothesize a positive correlation between social dissemination
and word growth.

  * •

H2-weak: Nonstandard words with higher linguistic dissemination in the early
phase of their history are more likely to grow. This follows from work in
corpus linguistics showing that words and grammatical patterns with a higher
diversity of collocations are more likely to be adopted Ito and Tagliamonte
(2003); Partington (1993).

  * •

H2-strong: Nonstandard words with higher linguistic dissemination are more
likely to grow, even after controlling for social dissemination. This follows
from the intuition that linguistic context and social context contribute
differently to word growth.

To address H2, we develop a novel metric for characterizing linguistic
dissemination, by comparing the observed number of $n$-gram contexts to the
number of contexts that would be predicted based on frequency alone. Our
analysis of word growth and decline includes: (1) prediction of frequency
change in growth words (as in prior work); (2) causal inference of the
influence of dissemination on probability of word growth; (3) binary
prediction of future growth versus decline; and (4) survival analysis, to
determine the factors that predict when a word’s popularity begins to decline.
All tests indicate that linguistic dissemination plays an important role in
explaining the growth and decline of nonstandard words.

##  2 Related Work

#### Lexical change online

Language changes constantly, and one of the most notable forms of change is
the adoption of new words Metcalf (2004), sometimes referred to as lexical
entrenchment Chesley and Baayen (2010). New nonstandard words may arise
through the mutation of existing forms by processes such as truncation (e.g,
favorite to fave; Grieve et al., 2016) and blending (e.g., web+log to weblog
to blog; Cook and Stevenson, 2010). The fast pace and interconnected nature of
online communication is particularly conducive to innovation, and social media
provides a “birds-eye view” on the process of change Danescu-Niculescu-Mizil
et al. (2013); Kershaw et al. (2016); Tsur and Rappoport (2015).

The most closely related work is a contemporaneous study that explored the
role of weak social ties in the dissemination of linguistic innovations on
Reddit, which also proposed the task of quantitatively predicting the success
or failure of lexical innovations Tredici and Fernández (2018). One
distinguishing feature of our work is the emphasis on _linguistic_ (rather
than social) context in explaining these successes and failures. In addition
to predicting the binary distinction between success and failure, we also take
on the more fine-grained task of predicting the length of time that each
nonstandard word will survive.

#### Social dissemination

Language changes as a result of transmission across generations Labov (2007)
as well as diffusion across individuals and social groups Bucholtz (1999).
Such diffusion can be quantified with _social dissemination_ , which Altmann
et al. (2011) define as the count of social units (e.g., users) who have
adopted a word, normalized by the expected count under a null model in which
the word is used with equal frequency across the entire population. Altmann et
al. (2011) use dissemination of words across forum users and threads to
predict the words’ change in frequency in Usenet, finding a positive
correlation between frequency change and both kinds of social dissemination.
In contrast, Garley and Hockenmaier (2012) use the same metric to predict the
growth of English loanwords on German hip-hop forums, and find that social
dissemination has less predictive power than expected. We seek to replicate
these prior findings, and to extend them to the broader context of Reddit.

#### Linguistic dissemination

In historical linguistics, the distribution of a new word or construction
across lexical contexts can signal future growth Partington (1993).
Furthermore, grammatical and lexical factors can explain a speaker’s choice of
linguistic variant Ito and Tagliamonte (2003); Cacoullos and Walker (2009) and
can provide more insight than social factors alone. Our study proposes a
generalizable method of measuring the dissemination of a word across lexical
contexts with _linguistic_ dissemination and compares social and linguistic
dissemination as predictors of language change.

##  3 Data

Our study examines the adoption of words on social media, and we focus on
Reddit as a source of language change. Reddit is a social content sharing site
separated into distinct sub-communities or “subreddits” that center around
particular topics Gilbert (2013). Reddit is a socially diverse and dynamic
online platform, making it an ideal environment for research on language
change Kershaw et al. (2016). Furthermore, because Reddit data is publicly
available we expect that this study can be more readily replicated than a
similar study on other platforms such as Facebook or Twitter, whose data is
less easily obtained.

We analyze a set of public monthly Reddit comments111From
http://files.pushshift.io/reddit/comments/ (Accessed 1 October 2016). posted
between 1 June 2013 and 31 May 2016, totalling $T=36$ months of data. This
dataset has been analyzed in prior work Hessel et al. (2016); Tan and Lee
(2015) and has been noted to have some missing data Gaffney and Matias (2018),
although this issue should not affect our analysis. To reduce noise in the
data, we filter all comments generated by known bots and spam users222The same
list used in Tan and Lee (2015): https://chenhaot.com/data/multi-
community/README.txt (Accessed 1 October 2016). and filter all comments
created in well-known non-English subreddits.333We randomly sampled 100 posts
from the top 500 subreddits and labelled a subreddit as non-English if fewer
than 90% of its posts were identified by langid.py Lui and Baldwin (2012) as
English. The final data collected is summarized in Table 1.

| Total | Monthly mean
---|---|---
Comments | 1,625,271,269 | 45,146,424
Tokens | 56,674,728,199 | 1,574,298,006
Subreddits | 333,874 | 48,786
Users | 14,556,010 | 2,302,812
Threads | 102,908,726 | 3,079,780
Table 1: Data summary statistics.

We replace all references to subreddits and users (marked by the convention
r/subreddit and u/user) with r/SUB and u/USER tokens, and all hyperlinks with
a URL token. We also reduce all repeated character sequences to a maximum
length of three (e.g., loooool to loool). The final vocabulary includes the
top 100,000 words by frequency.444We restricted the vocabulary because of the
qualitative analysis required to identify nonstandard words. We replace all
OOV words with UNK tokens, which comprise 3.95% of the total tokens.

###  3.1 Finding growth words

Our work seeks to study the growth of nonstandard words, which we identify
manually instead of relying on pre-determined lists Tredici and Fernández
(2018).To detect such words, we first compute the Spearman correlation
coefficient between the time steps $\\{1...T\\}$ and each word $w$’s frequency
time series $f^{(w)}_{(1:T)}$ (frequency normalized and log-transformed). The
Spearman correlation coefficient captures monotonic, gradual growth that
characterizes the adoption of nonstandard words Grieve et al. (2016); Kershaw
et al. (2016).

The first set of words is filtered by a Spearman correlation coefficient above
the $85^{\text{th}}$ percentile ($N=15,017$). From this set of words, one of
the authors manually identified 1,120 words in set $\mathcal{G}$ (“growth”)
that are neither proper nouns (berniebot, killary, drumpf) nor standard words
(election, voting).555Code and word lists available at:
https://github.com/ianbstewart/nonstandard_word_dissemination. These words
were removed because their growth may be due to exogenous influence. A
“standard” word is one that can plausibly be found in a newspaper article,
which follows from the common understanding of newspaper text as a more formal
and standard register. Therefore, a “nonstandard” word is one that cannot
plausibly be found in a newspaper article, a judgment often used by linguists
to determine what counts as slang Dumas and Lighter (1978). In ambiguous
cases, one of the authors inspected a sample of comments that included the
word. We validate this process by having both authors annotate the top 200
growth candidates for standard/proper versus nonstandard (binary), obtaining
inter-annotator agreement of $\kappa$=0.79.

###  3.2 Finding decline words

To determine what makes the growth words successful, we need a control group
of “decline” words, which are briefly adopted and later abandoned. Although
these words may have been successful before the time period investigated,
their decline phase makes them a useful comparison for the growth words. We
find such words by fitting two parametric models to the frequency series.

#### Piecewise linear fit

We fit a two-phase piecewise linear regression on each word’s frequency time
series $f_{(1:T)}$, which splits the time series into $f_{(1:\hat{t})}$ and
$f_{(\hat{t}+1:T)}$. The goal is to select a split point $\hat{t}$ to minimize
the sum of the squared error between observed frequency $f$ and predicted
frequency $\hat{f}$:

$\hat{f}(m_{1},m_{2},b,t)=\begin{cases}b+m_{1}t&t\leq\hat{t}\\\
b+m_{1}\hat{t}+m_{2}(t-\hat{t})&t>\hat{t},\end{cases}$ (1)

where $b$ is the intercept, $m_{1}$ is the slope of the first phase, and
$m_{2}$ is the slope of the second phase. Decline words $\mathcal{D}_{p}$
(“piecewise decline”) display growth in the first phase ($m_{1}>0$), decline
in the second phase ($m_{2}<0$), and a strong fit between observed and
predicted data, indicated by $R^{2}(f,\hat{f})$ above the $85^{th}$ percentile
(36.1%); this filtering yields 14,995 candidates.

#### Logistic fit

To account for smoother growth-decline trajectories, we also fit the growth
curve to a logistic distribution, which is a continuous unimodal distribution
with support over the non-negative reals. We identify the set of candidates
$\mathcal{D}_{l}$ (“logistic decline”) as words with a strong fit to this
distribution, as indicated by $R^{2}$ above the $99^{th}$ percentile (82.4%),
yielding 998 candidates. The logistic word set partially overlaps with the
piecewise set, because some words’ frequency time series show a strong fit to
both the piecewise function and the logistic distribution.

#### Combined set

We combine the sets $\mathcal{D}_{p}$ and $\mathcal{D}_{l}$ to produce a set
of decline word candidates ($N=15,665$). Next, we filter this combined set to
exclude standard words and proper nouns, yielding a total of 530 decline words
in set $\mathcal{D}$. Each word is assigned a split point $\hat{t}$ based on
the estimated time of switch between the growth phase and the decline phase,
which is the split point $\hat{t}$ for piecewise decline words and the center
of the logistic distribution $\hat{\mu}$ for the logistic decline words.

Word set |  Examples
---|---
$\mathcal{G}$ |  idk, lmao, shitpost, tbh, tho
$\mathcal{D}_{l}$ |  atty, eyebleach, iifym, obeasts, trashy
$\mathcal{D}_{p}$ |  brojob, nparent, rekd, terpers, wot
Table 2: Examples of nonstandard words in all word sets: growth
($\mathcal{G}$), logistic decline ($\mathcal{D}_{l}$) and piecewise decline
($\mathcal{D}_{p}$).

Examples of both growth and decline words are shown in Table 2. The growth
words include several acronyms (tbh, “to be honest”; lmao, “laughing my ass
off”), while the decline words include clippings (atty, “atomizer”),
respellings (rekd, “wrecked”; wot, “what”) and compounds (nparent,
“narcissistic parent”).

|  Clipping |  Compound |  Respelling |  Other | Total
---|---|---|---|---|---
$\mathcal{G}$ |  198 (17.7%) |  334 (29.8%) |  83 (7.4%) |  505 (45.1%) | 1,120
$\mathcal{D}$ |  53 (10.0%) |  100 (18.9%) |  108 (20.4%) |  269 (50.8%) | 530
Table 3: Word formation category counts in growth ($\mathcal{G}$) and decline
($\mathcal{D}$) word sets.

We also provide a distribution of the words across word generation categories
in Table 3, including compounds and clippings in similar proportions to prior
work Kulkarni and Wang (2018). Because the growth and decline words exhibit
similar proportions of category counts, we do not expect that this will be a
significant confound in differentiating growth from decline.

##  4 Predictors

We now outline the predictors used to measure the degree of social and
linguistic dissemination in the growth and decline words.

###  4.1 Social dissemination

We rely on the dissemination metric proposed by Altmann et al. (2011) to
measure the degree to which a word occupies a specific social niche (e.g., low
dissemination implies limited niche). To compute user dissemination $D^{U}$
for word $w$ at time $t$, we first compute the number of individual users who
used word $w$ at time $t$, written $U^{(w)}_{t}$. We then compare this with
the expectation $\tilde{U}^{(w)}_{t}$ under a model in which word frequency is
identical across all users. The user dissemination is the log ratio,

$\log\frac{U_{t}^{(w)}}{\tilde{U}_{t}^{(w)}}=\log
U^{(w)}_{t}-\log\tilde{U}^{(w)}_{t}.$ (2)

Following Altmann et al. (2011), the expected count $\tilde{U}_{t}^{(w)}$ is
computed as,

$\tilde{U}_{t}^{(w)}=\sum_{u\in\mathcal{U}_{t}}(1-e^{-f_{t}^{(w)}m_{t}^{(u)}}),$
(3)

where $m_{t}^{(u)}$ equals the total number of words contributed by user $u$
in month $t$, and $\mathcal{U}_{t}$ is the set of all users active in month
$t$. This corresponds to a model in which each token from a user has identical
likelihood $f_{t}^{(w)}$ of being word $w$. In this way, we compute
dissemination for all users ($D^{U}$), subreddits ($D^{S}$) and threads
($D^{T}$) for each month $t\in\\{1...T\\}$.

###  4.2 Linguistic dissemination

Linguistic dissemination captures the diversity of linguistic contexts in
which a word appears, as measured by unique $n$-gram counts. We compute the
log count of unique trigram666Pilot analysis with bigram contexts gave similar
results. contexts for all words ($C^{3}$) using all possible trigram
positions: in the sentence “that’s cool af haha”, the term af appears in three
unique trigrams, that’s cool af, cool af haha, af haha <END>.

The unique log number of trigram contexts is strongly correlated with log word
frequency ($\rho(C^{3},f)=0.904$), as implied by Heaps’ law Egghe (2007). We
therefore adjust this statistic by comparing with its expected value
$\tilde{C}^{3}$. At each timestep $t$, we fit a linear regression between log-
frequency and log-unique $n$-gram counts, and then compute the residual
between the observed log count of unique trigrams and its expectation,
$D^{L}=C^{3}_{t}-\tilde{C}^{3}_{t}$. The residual $D^{L}$, or _linguistic
dissemination_ , identifies words with a higher or lower number of lexical
contexts than expected.

Figure 1: Distribution of mean linguistic dissemination ($D^{L}$) across part
of speech groups.

Linguistic dissemination can separate words by grammatical category, as shown
in Figure 1 where the mean $D^{L}$ values are computed for words across common
part-of-speech categories. Part-of-speech tags were computed over the entire
corpus using a Twitter-based tagger Gimpel et al. (2011), and each word type
was assigned the most likely POS tag to provide an approximate distribution of
tags over the vocabulary. Interjections have a lower median $D^{L}$ than other
word categories due to the tendency of interjections to occur in limited
lexical contexts. Conversely, verbs have a higher median $D^{L}$ due to the
flexibility of verbs’ arguments (e.g., subject and object may both be open-
class nouns).

##  5 Results

The hypotheses about social and linguistic dissemination are tested under four
analyses: correlation against frequency change in growth words; causal
inference on probability of word growth; binary prediction of word growth; and
survival analysis of decline words.

###  5.1 Correlational analysis

To test the relative importance of the linguistic and social context on word
growth, we correlate these metrics with frequency change
($\Delta_{f_{t}}=f_{t}-f_{t-k}$) across all growth words. This replicates the
methodology in prior work by Altmann et al. (2011) and Garley and Hockenmaier
(2012), who analyzed different internet forums. Focusing on long-term change
with $k=12$ (one year) and $k=24$ (two years), we compute the proportion of
variance in frequency change explained by the covariates using a relative
importance regression Kruskal (1987).777Relative importance regression
implemented in the _relaimpo_ package in R:
https://cran.r-project.org/package=relaimpo

|  | Variance
---
explained
Lower, upper 95%
$f_{t-12}$ | 10.8% | [10.2%, 11.5%]
$D^{L}_{t-12}$ | 0.584% | [0.461%, 0.777%]
$D^{U}_{t-12}$ | 0.307% | [0.251%, 0.398%]
$D^{S}_{t-12}$ | 0.120% | [0.0852%, 0.191%]
$D^{T}_{t-12}$ | 0.246% | [0.171%, 0.379%]
$f_{t-24}$ | 21.4% | [20.4%, 22.4%]
$D^{L}_{t-24}$ | 1.29% | [1.05%, 1.64%]
$D^{U}_{t-24}$ | 0.400% | [0.346%, 0.493%]
$D^{S}_{t-24}$ | 0.287% | [0.201%, 0.392%]
$D^{T}_{t-24}$ | 0.272% | [0.226%, 0.380%]
Table 4: Percent of variance explained in frequency change, computed over all
growth words $\mathcal{G}$. $N=26,880$ for $k=12$, $N=13,440$ for $k=24$.

The results of the regression are shown in Table 4. All predictors have
relative importance greater than zero, according to a bootstrap method to
produce confidence intervals Tonidandel et al. (2009). Frequency is the
strongest predictor ($f_{t-12},f_{t-24}$), because words with low initial
frequency often show the most frequency change. In both short- and long-term
prediction, linguistic dissemination ($D^{L}_{t-12},D^{L}_{t-24}$) has a
higher relative importance than each of the social dissemination metrics. The
social dissemination metrics have less explanatory power, in comparison with
the other predictors and in comparison to the prior results of Garley and
Hockenmaier (2012), who found $1.5\%$ of variance explained by $D^{U}$ and
$1.9\%$ for $D^{T}$ at $k=24$. Our results were robust to the exclusion of the
predictor $D^{L}$, meaning that a model with only the social dissemination
metrics as predictors resulted in a similar proportion of variance explained.
The weakness of social dissemination could be due to the fragmented nature of
Reddit, compared to more intra-connected forums. Since users and threads are
spread across many different subreddits, and users may not visit multiple
subreddits, a higher social dissemination for a particular word may not lead
to immediate growth.

###  5.2 Causal analysis

While correlation can help explain the relationship between dissemination and
frequency change, it only addresses the weak version of H2: it does not
distinguish the causal impact of linguistic and social dissemination. To test
the strong version of H2, we turn to a causal analysis, in which the _outcome_
is whether a nonstandard word grows or declines, the _treatment_ is a single
dissemination metric such as linguistic dissemination, and the _covariates_
are the remaining dissemination metrics. The goal of this analysis is to test
the impact of each dissemination metric, while holding the others constant.

Figure 2: Average dose response function for all treatment variables, where
outcome is probability of word growth. 95% confidence intervals plotted in
red, chance rate of 50% marked with dotted black line.

Causal inference typically uses a binary treatment/control distinction Angrist
et al. (1996), but in this case the treatment is continuous. We therefore turn
to an adapted model known as the _average dose response function_ to measure
the causal impact of dissemination Imbens (2000). To explain the procedure for
estimating the average dose response, we adopt the following terminology: $Z$
for treatment variable, $X$ for covariates, $Y$ for outcome.888Average dose
response function implemented in the _causaldrf_ package in R:
https://cran.r-project.org/package=causaldrf

  1. 1.

A linear model is fit to estimate the treatment from the covariates,

$Z_{i}\mid X_{i}\sim\mathcal{N}(\beta^{\top}X_{i},\sigma^{2}).$ (4)

The output of this estimation procedure is a vector of weights $\hat{\beta}$
and a variance $\hat{\sigma}^{2}$.

  2. 2.

The generalized propensity score (GPS) $R$ is the likelihood of observing the
treatment given the covariates, $P(Z_{i}\mid X_{i})$. It is computed from the
parameters estimated in the previous step:

$\hat{R}_{i}=\frac{1}{\sqrt{2\pi\hat{\sigma}^{2}}}\exp\left(-\frac{(Z_{i}-\hat{\beta}^{\top}X_{i})^{2}}{2\hat{\sigma}^{2}}\right).$
(5)

  3. 3.

A logistic model is fit to predict the outcome $Y_{i}$ using the treatment
$Z_{i}$ and the GPS $\hat{R}_{i}$:

$\hat{Y}_{i}=\text{Logistic}(\hat{\alpha}_{0}+\hat{\alpha}_{1}Z_{i}+\hat{\alpha}_{2}\hat{R}_{i}).$
(6)

This involves estimating the parameters
$\\{\hat{\alpha}_{0},\hat{\alpha}_{1},\hat{\alpha}_{2}.\\}$ By incorporating
the generalized propensity score $\hat{R}_{i}$ into this predictive model over
the outcome, it is possible to isolate the causal effect of the treatment from
the other covariates (Hirano and Imbens, 2004).

  4. 4.

The range of treatments is divided into levels (quantiles). The _average dose
response_ for a given treatment level $s_{z}$ is the mean estimated outcome
for all instances at that treatment level,

$\hat{\mu}(s_{z})=\frac{1}{|s_{z}|}\sum_{z_{i}\in s_{z}}\hat{Y}_{i}.$ (7)

The average dose response function is then plotted for all treatment levels.

Each dissemination metric is considered separately as a treatment. We consider
all other dissemination metrics and frequency as covariates: e.g., for
treatment variable $D^{L}$, the covariates are set to $[f,D^{U},D^{S},D^{T}]$.
We bootstrap the above process 100 times with different samples to produce
confidence intervals. To balance the outcome classes, we sample an equal
number of growth and decline words for each bootstrap iteration.

The average dose response function curves in Figure 2 show that linguistic
dissemination ($D^{L}$) produces the most dramatic increase in word growth
probability. For linguistic dissemination, the lowest treatment quantile
(0%-10%) yields a growth probability below 40% (significantly less than
chance), as compared to the highest treatment quantile (90-100%), which yields
a growth probability nearly at 70% (significantly greater than chance). This
supports the strong form of H2, which states that linguistic dissemination is
predictive of growth, even after controlling for the frequency and the other
dissemination metrics. Subreddit dissemination also shows a mild causal effect
on word growth, up to 60% in the highest treatment quantile. The other social
dissemination metrics prove to have less effect on word growth.

###  5.3 Predictive analysis

We now turn to prediction to determine the utility of linguistic and social
dissemination: using the first $k$ months of data, can we predict whether a
word will grow or decline in popularity? This is similar to previous work in
predicting the success of lexical innovations Kooti et al. (2012), but our
goal is to compare the relative predictive power of various dissemination
metrics, rather than to maximize accuracy.

Figure 3: Prediction accuracy for different feature sets using $k={1...12}$
months of training data. $f$ indicates frequency-only, $f+L$ frequency plus
linguistic dissemination, $f+S$ frequency plus social dissemination, $f+L+S$
all features.

We use logistic regression with 10-fold cross-validation over four different
feature sets: frequency-only (f), frequency plus linguistic dissemination
(f+L), frequency plus social dissemination (f+S) and all features (f+L+S).
Each fold is balanced for classes so that the baseline accuracy is 50%. Figure
3 shows that linguistic dissemination provides more predictive power than
social dissemination: the accuracy is consistently higher for the models with
linguistic dissemination than for the frequency-only and social dissemination
models. The accuracies converge as the training data size increases, which
suggests that frequency is a useful predictor if provided sufficient
historical trajectory.

Figure 4: Distribution of $D^{L}$ values across growth and decline words,
grouped by part of speech tag. * indicates $p<0.05$ in one-tailed t-test
between growth and decline $D^{L}$ values.

#### Part-of-speech robustness check

Considering the uneven distribution of linguistic dissemination across part-
of-speech groups (Figure 1), the prediction results may be explained by an
imbalance of word categories between the growth and decline words. This issue
is addressed through two robustness checks: within-group comparison and
prediction.

First, we compare the distribution of linguistic dissemination values between
growth and decline words, grouped by the most common POS tags (computed in §
4.2). Each decline word is matched with a growth word based on similar mean
frequency in the first $k=12$ months, and their mean linguistic dissemination
values during that time period are compared, grouped within POS tag groups.
The differences in Figure 4 show that across all POS tags, the growth words
show a tendency toward higher linguistic dissemination with significant
($p<0.05$) differences in the interjections, adjectives and verbs.

Next, we add POS tags as additional features to the frequency-only model in
the binary prediction task. The accuracy of a predictive model with access to
frequency and POS features at ${k=1}$ is 54.8%, which is substantially lower
than the accuracy of the model with frequency and linguistic dissemination
(cf. Figure 3).999Higher $k$ values yield similar results. Thus, linguistic
dissemination thus contributes predictive power beyond what is contributed by
part-of-speech alone.

###  5.4 Survival analysis

Having investigated what separates growth from decline, we now focus on the
factors that precede a decline word’s “death” phase Drouin and Dury (2009).

Predicting the time until a word’s decline can be framed as survival analysis
Klein and Moeschberger (2005), in which a word is said to “survive” until the
beginning of its decline phase at split point $\hat{t}$. In the Cox
proportional hazards model Cox (1972), the hazard of death $\lambda$ at each
time $t$ is modeled as a linear function of a vector of predictors,

$\lambda_{i}(t)=\lambda_{0}(t)\exp(\mathbf{\beta}\cdot\mathbf{x}_{i}),$ (8)

where $\mathbf{x}_{i}$ is the vector of predictors for word $i$, and
$\mathbf{\beta}$ is the vector of coefficients. Each cell $x_{i,j}$ is set to
the mean value of predictor $j$ for word $i$ over the training period
$t=\\{1...k\\}$ where $k=3$.

For words which begin to decline in popularity in our dataset, we treat the
point of decline as the “death” date. The remaining words are viewed as
_censored_ instances: they may begin to decline in popularity at some point in
the future, but this time is outside our frame of observation. We use
frequency, social dissemination and linguistic dissemination as predictors in
a Cox regression model.101010Cox regression implemented in the _lifelines_
package in Python: https://lifelines.readthedocs.io/en/latest/.

Predictor | $\beta$ | std. error | $Z$ | $p$
---|---|---|---|---
$\mathrm{f}$ | -0.207 | 0.0492 | -4.21 | ***
$\mathrm{D^{L}}$ | -0.330 | 0.0385 | -8.56 | ***
$\mathrm{D^{U}}$ | 0.0053 | 0.0518 | 0.102 |
$\mathrm{D^{S}}$ | -0.156 | 0.0807 | -1.928 |
$\mathrm{D^{T}}$ | 0.0825 | 0.0662 | 1.25 |
Table 5: Cox regression results for predicting word death with all predictors
(f+L+S) averaged over first $k=3$ months. *** indicates $p<0.001$, otherwise
$p>0.05$.

The estimated coefficients from the regression are shown in Table 5. We find a
negative coefficient for linguistic dissemination ($\beta=-0.330,p<0.001$),
which mirrors the results from § 5.2: higher $D^{L}$ indicates a lower hazard
of word death, and therefore a higher likelihood of survival. We also find
that higher subreddit dissemination has a weak but insignificant correlation
with a lower likelihood of word death ($\beta=-0.156,p>0.05$). Both of these
results lend additional support to the strong form of the hypothesis H2.

Figure 5: Distribution of concordance scores (10-fold cross-validation) of the
Cox regression models across feature sets.

The predictive accuracy of survival analysis can be quantified by a
_concordance score_. A score of 1.0 on heldout data indicates that the model
perfectly predicts the order of death times; a score of 0.5 indicates that the
predictions are no better than a chance ordering. We perform 10-fold cross-
validation of the survival analysis model, and plot the results in Figure 5.
The model with access to linguistic dissemination (f+L) consistently achieves
higher concordance than the baseline frequency-only model (f),
($t=4.29,p<0.001$), and the model with all predictors f+L+S significantly
outperforms the model with access only to frequency and social dissemination
f+S ($t=4.64,p<0.001$). The result is reinforced by testing the goodness-of-
fit for each model with model _deviance_ , or difference from the null model.
The f+L model has lower deviance, i.e. better fit, than the null model
($\chi^{2}=93.3,p<0.01$), and the f+L+S does not have a significantly lower
deviance than the f+L model ($\chi^{2}=4.6,p=0.80$), suggesting that adding
social dissemination does not significantly improve model fit.

##  6 Discussion

All four analyses support H2: linguistic dissemination was the strongest
predictor of monthly frequency changes in growth words, the best
differentiator of growth and decline words in causal and predictive tasks, and
the most effective warning sign that a word is about to decline. Linguistic
dissemination can be related to theories such as the FUDGE factors Chesley and
Baayen (2010); Cook (2010); Metcalf (2004), in which a word’s growth depends
on frequency (F), unobtrusiveness (U), diversity of users and situations (D),
generation of other forms and meanings (G), and endurance (E). Linguistic
dissemination provides an example of “diversity of situation.”

The effectiveness of linguistic dissemination is exemplified in pairs of
semantically similar growth and decline words. In the first $k=3$ months of
growth, the growth word kinda has a relatively high ratio of linguistic to
frequency ($\frac{D^{L}}{f}=0.270$) as compared with the semantically similar
decline word sorta (0.055). This pattern holds for other pairs of semantically
similar growth/decline words: fuckwit and fuckboy; lolno and lmao; yup and
yas. While not exhaustive, such a trend suggests that the growth words were
able to reach a wider range of lexical contexts and therefore succeed where
the decline words failed.

Regarding H1, we generally found a positive role for social dissemination as
well, although these results were not consistent across all metrics and tests,
particularly in the survival analysis. This matches the conclusion from Garley
and Hockenmaier (2012), who argued that social dissemination is less
predictive of word adoption than Altmann et al. (2011) originally suggested.
One possible explanation is the inclusion of word categories such as proper
nouns in the analysis of Altmann et al. (2011); the adoption of such terms may
rely on social dynamics more than the adoption of nonstandard terms. The lower
predictive power of thread and user dissemination is also interesting and
suggests that subreddits are more socially salient in terms of exposing
nonstandard words to potential adopters.

#### Limitations

One limitation in the study was the exclusion of orthographic and
morphological features such as affixation, which has been noted as a predictor
of word growth Kershaw et al. (2016). Future work should incorporate these
features as additional predictors. Our study also omitted borrowings, unlike
prior work in word adoption that has focused on borrowings Chesley and Baayen
(2010); Garley and Hockenmaier (2012). Our early language-filtering steps
eliminated most non-English words from the vocabulary, although it would have
been interesting to examine loanword use in English-language posts. Finally,
our study was limited by the focus on nonstandard words rather than memetic
phrases (e.g., like a boss) which may show a similar correlation between
dissemination, growth and decline Bybee (2006).

#### Future work

We approximate linguistic dissemination using trigram counts, because they are
easy to compute and they generalize across word categories. In future work, a
more sophisticated approach might estimate linguistic dissemination with
syntactic features such as appearance across different phrase heads Kroch
(1989); Ito and Tagliamonte (2003) or across nouns of different semantic
classes D’Arcy and Tagliamonte (2015). Future work should also investigate
more semantically-aware definitions of linguistic dissemination. The existence
of semantic “neighbors” occurring in similar contexts (e.g., the influence of
standard intensifier very on nonstandard intensifier af) may prevent a new
word from reaching widespread popularity Grieve (2018).

## Acknowledgments

The authors thank the anonymous reviewers, the audience at NWAV 46 for
discussion of an early presentation of this study, and the members of Georgia
Tech’s Computational Linguistics Lab for their help throughout the project.
This research was supported by NSF award IIS-1452443, AFOSR award
FA9550-14-1-0379, and NIH award R01-GM112697-03.

## References

  * Altmann et al. (2011) Eduardo Altmann, Janet Pierrehumbert, and Adilson Motter. 2011.  Niche as a determinant of word fate in online groups.  _PLoS ONE_ , 6(5):1–12.
  * Androutsopoulos (2011) Jannis Androutsopoulos. 2011.  Language change and digital media: a review of conceptions and evidence.  In Kristiansen Tore and Nikolas Coupland, editors, _Standard Languages and Language Standards in a Changing Europe_ , pages 145–159. Novus Press, Oslo.
  * Angrist et al. (1996) Joshua D Angrist, Guido W Imbens, and Donald B Rubin. 1996.  Identification of Causal Effects Using Instrumental Variables.  _Source Journal of the American Statistical Association_ , 91(434):444–455.
  * Bakshy et al. (2011) Eytan Bakshy, Jake Hofman, Winter Mason, and Duncan Watts. 2011.  Everyone’s an influencer: quantifying influence on Twitter.  In _Proceedings of the International Conference on Web Search and Data Mining_ , pages 65–74.
  * Bucholtz (1999) Mary Bucholtz. 1999.  "Why be normal?": Language and identity practices in a community of nerd girls.  _Language in Society_ , 28:203–223.
  * Bybee (2006) Joan L. Bybee. 2006.  From Usage to Grammar: The Mind’s Response to Repetition.  _Language_ , 82(4):711–733.
  * Cacoullos and Walker (2009) Rena Torres Cacoullos and James Walker. 2009.  The Present of the English Future: Grammatical Variation and Collocations in Discourse.  _Language_ , 85(2):321–354.
  * Chesley and Baayen (2010) Paula Chesley and Harald Baayen. 2010.  Predicting new words from newer words: Lexical borrowings in French.  _Linguistics_ , 48(6):1343–1374.
  * Cook (2010) Paul Cook. 2010.  _Exploiting linguistic knowledge to infer properties of neologisms_.  Ph.D. thesis, University of Toronto.
  * Cook and Stevenson (2010) Paul Cook and Suzanne Stevenson. 2010.  Automatically identifying the source words of lexical blends in English.  _Computational Linguistics_ , 36(1):129–149.
  * Cox (1972) David Cox. 1972.  Regression models and life tables.  _Journal of the Royal Statistical Society_ , 34:187–220.
  * Danescu-Niculescu-Mizil et al. (2013) Cristian Danescu-Niculescu-Mizil, Robert West, Dan Jurafsky, and Christopher Potts. 2013.  No Country for Old Members: User Lifecycle and Linguistic Change in Online Communities.  _Proceedings of the International Conference on World Wide Web_ , pages 307–317.
  * D’Arcy and Tagliamonte (2015) Alexandra D’Arcy and Sali Tagliamonte. 2015.  Not always variable: probing the vernacular grammar.  _Language Variation and Change_ , 27(3):255–285.
  * Drouin and Dury (2009) Patrick Drouin and Pascaline Dury. 2009.  When terms disappear from a specialized lexicon: A semi-automatic investigation into necrology.  In _Actes de la conférence internationale “Language for Special Purposes”_.
  * Dumas and Lighter (1978) BK Dumas and Jonathan Lighter. 1978.  Is slang a word for linguists?  _American Speech_ , 53(1):5–17.
  * Egghe (2007) Leo Egghe. 2007.  Untangling Herdan’s law and Heaps’ Law: Mathematical and informetric arguments.  _Journal of the American Society for Information Science and Technology_ , 58(5):702–709.
  * Gaffney and Matias (2018) Devin Gaffney and J. Nathan Matias. 2018.  Caveat emptor, computational social science: Large-scale missing data in a widely-published reddit corpus.  _PLoS ONE_ , 13(7).
  * Garley and Hockenmaier (2012) Matt Garley and Julia Hockenmaier. 2012.  Beefmoves: dissemination, diversity, and dynamics of English borrowings in a German hip hop forum.  In _Proceedings of the Association of Computational Linguistics_ , pages 135–139.
  * Gilbert (2013) Eric Gilbert. 2013.  Widespread Underprovision on Reddit.  In _Proceedings of the Conference on Computer-Supported Cooperative Work_ , pages 803–808.
  * Gimpel et al. (2011) Kevin Gimpel, Nathan Schneider, Brendan O’Connor, Dipanjan Das, Daniel Mills, Jacob Eisenstein, Michael Heilman, Dani Yogatama, Jeffrey Flanigan, and Noah Smith. 2011.  Part-of-speech tagging for Twitter: Annotation, features, and experiments.  In _Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics_ , pages 42–47.
  * Grieve (2018) Jack Grieve. 2018.  Natural selection in the modern English Lexicon.  In _International Conference on Language Evolution_ , pages 153–157.
  * Grieve et al. (2016) Jack Grieve, Andrea Nini, and Diansheng Guo. 2016.  Analyzing lexical emergence in Modern American English online.  _English Language and Linguistics_ , 20(2):1–29.
  * Hessel et al. (2016) Jack Hessel, Chenhao Tan, and Lillian Lee. 2016.  Science, AskScience, and BadScience: On the coexistence of highly related communities.  In _Proceedings of the International Conference on Social and Web Media_ , pages 171–180.
  * Hirano and Imbens (2004) Keisuke Hirano and Guido W Imbens. 2004.  The propensity score with continuous treatments.  In Andrew Gelman and Xiao-Li Meng, editors, _Applied Bayesian modeling and causal inference from incomplete-data perspectives_ , pages 73–84. Wiley, Chichester.
  * Imbens (2000) Guido W Imbens. 2000.  The role of the propensity score in estimating dose-response functions.  _Biometrika_ , 87(3):706–710.
  * Ito and Tagliamonte (2003) Rika Ito and Sali Tagliamonte. 2003.  Well weird, right dodgy, very strange, really cool: Layering and recycling in English intensifiers.  _Language in Society_ , 32:257–279.
  * Kershaw et al. (2016) Daniel Kershaw, Matthew Rowe, and Patrick Stacey. 2016.  Towards Modelling Language Innovation Acceptance in Online Social Networks.  _Proceedings of the ACM International Conference on Web Search and Data Mining_ , pages 553–562.
  * Klein and Moeschberger (2005) John Klein and Melvin Moeschberger. 2005.  _Survival analysis: techniques for censored and truncated data_.  Springer Science & Business Media, New York.
  * Kooti et al. (2012) Farshad Kooti, Winter A Mason, Krishna P Gummadi, and Meeyoung Cha. 2012.  Predicting emerging social conventions in online social networks.  In _Proceedings of the International Conference on Information and Knowledge Management_ , pages 445–454.
  * Kroch (1989) Anthony S Kroch. 1989.  Reflexes of grammar in patterns of language change.  _Language Variation and Change_ , 1(3):199–244.
  * Kruskal (1987) William Kruskal. 1987.  Relative importance by averaging over orderings.  _The American Statistician_ , 41(1):6–10.
  * Kulkarni and Wang (2018) Vivek Kulkarni and William Yang Wang. 2018.  Simple Models for Word Formation in English Slang.  In _In Proceedings of the North American Association of Computational Linguistics_ , pages 1424–1434.
  * Labov (2007) William Labov. 2007.  Transmission and Diffusion.  _Language_ , 83(2):344–387.
  * Lui and Baldwin (2012) Marco Lui and Timothy Baldwin. 2012.  langid. py: An off-the-shelf language identification tool.  In _Proceedings of the Association of Computational Linguistics_ , pages 25–30.
  * MacWhinney (1989) Brian MacWhinney. 1989.  Competition and Lexical Categorization.  _Linguistic Categorization_ , pages 195–242.
  * Metcalf (2004) Allan Metcalf. 2004.  _Predicting new words: The secrets of their success_.  Houghton Mifflin, New York.
  * Partington (1993) Alan Partington. 1993.  Corpus evidence of language change: The case of the intensifier.  In Mona Baker, Gill Francis, and Elena Tognini-Bonelli, editors, _Text and Technology: In Honour of John Sinclair_ , pages 177–192. John Benjamins Publishing, Philadelphia.
  * Romero et al. (2011) Daniel Romero, Brendan Meeder, and Jon Kleinberg. 2011.  Differences in the mechanics of information diffusion across topics: idioms, political hashtags, and complex contagion on Twitter.  In _Proceedings of the International Conference on World Wide Web_ , pages 695–704. ACM.
  * Squires (2010) Lauren Squires. 2010.  Enregistering internet language.  _Language in Society_ , 39:457–492.
  * Tagliamonte and Denis (2008) Sali Tagliamonte and Derek Denis. 2008.  Linguistic ruin? LOL! Instant messaging and teen language.  _American Speech_ , 83(1):3–34.
  * Tan and Lee (2015) Chenhao Tan and Lillian Lee. 2015.  All who wander: On the prevalence and characteristics of multi-community engagement.  In _Proceedings of the International Conference on World Wide Web_ , pages 1056–1066.
  * Tonidandel et al. (2009) Scott Tonidandel, James LeBreton, and Jeff Johnson. 2009.  Determining the statistical significance of relative weights.  _Psychological methods_ , 14(4):387.
  * Tredici and Fernández (2018) Marco Del Tredici and Raquel Fernández. 2018.  The Road to Success: Assessing the Fate of Linguistic Innovations in Online Communities.  In _Proceedings of the International Conference on Computational Linguistics_ , pages 1591–1603.
  * Tsur and Rappoport (2015) Oren Tsur and Ari Rappoport. 2015.  Don’t Let Me Be #Misunderstood: Linguistically Motivated Algorithm for Predicting the Popularity of Textual Memes.  In _Proceedings of the International Conference on Social and Web Media_ , pages 426–435.


# Uncertainty Surrogates for Deep Learning

Radhakrishna Achanta and Natasa Tagasovska
Swiss Data Science Center
EPFL and ETH Zurich
Switzerland
radhakrishna.achanta<EMAIL_ADDRESS>
###### Abstract

In this paper we introduce a novel way of estimating prediction uncertainty in
deep networks through the use of uncertainty surrogates. These surrogates are
features of the penultimate layer of a deep network that are forced to match
predefined patterns. The patterns themselves can be, among other
possibilities, a known visual symbol. We show how our approach can be used for
estimating uncertainty in prediction and out-of-distribution detection.
Additionally, the surrogates allow for interpretability of the ability of the
deep network to learn and at the same time lend robustness against adversarial
attacks. Despite its simplicity, our approach is superior to the state-of-the-
art approaches on standard metrics as well as computational efficiency and
ease of implementation. A wide range of experiments are performed on standard
datasets to prove the efficacy of our approach.

##  1 Introduction

Deep learning has made large strides in recent times, providing breakthroughs
in several computer vision and machine learning problems. However, deep
learning models remain unable to assess their confidence when performing under
unforeseen situations. As argued by Begoli _et al_. [3], uncertainty
quantification is a problem of paramount importance when deploying machine
learning models in sensitive contexts such as information security [31],
engineering [36], transportation [38], or medicine [3].

Figure 1: For three different corruptions of the original image, namely noise
addition, random cropping, and rotation, compared to a competing technique
[35], our approach shows the uncertainty in prediction that better reflects
the qualitative deterioration of the original image.

Uncertainty estimation can be used to assess risk or decide when to abstain
from using the prediction. Such abstention is a reasonable strategy to deal
with anomalies [6], outliers [19], out-of-distribution examples [30], to
detect and defend against adversaries [32], or to delegate high-risk
predictions to humans [8, 12, 7]. Knowing uncertainty is the backbone of
active learning [29], when it comes to optimally choosing examples a human
should annotate. Importantly, uncertainty quantification takes us closer to
model interpretability [1].

In this paper, we address this outstanding problem, namely, quantification of
the degree of uncertainty of the inference done by a deep network, with a
simple yet powerful method. Our approach consists of assigning, to each class,
a fixed binary pattern, often visually meaningful (see Figure 2), and forcing
the activations of the penultimate layer to resemble it using an additional
reconstruction loss.

Training with this additional loss turns the penultimate layer activations
into proxies for estimating the uncertainty of classification results. Our
method is hence termed _Classification Uncertainty from Surrogate Prediction_
or CUSP, in short. Additionally, in the rest of the paper we refer to the
predefined patterns used for training uncertainty surrogates as _surrogate
patterns_.

|  |  |  |  |  |  |  |  |  |  |  |  |  |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---
0.094 | 0.097 | 0.117 | 0.119 | 0.121 | 0.145 | 0.154 | 0.166 | 0.198 | 0.204 | 0.213 | 0.254 | 0.368 | 0.612 | 0.64
|  |  |  |  |  |  |  |  |  |  |  |  |  |
0.108 | 0.127 | 0.128 | 0.146 | 0.158 | 0.337 | 0.344 | 0.347 | 0.354 | 0.393 | 0.328 | 0.337 | 0.341 | 0.348 | 0.362
Figure 2: Visualizations using our approach correspondng to FashionMNIST and
CIFAR-10 datasets. The FashionMNIST images are trained with ten $16\times 16$
digit images, while the CIFAR-10 images are trained with ten $16\times 32$
text images containing the corresponding class name. In each of the two sets,
the rows from top contain: the original images from the CIFAR-10 dataset, our
uncertainty surrogates, the pattern corresponding to the predicted label, the
pattern corresponding to the true label, the uncertainty score as a colored
bar (green for correctly predicted and red for incorrectly predicted
examples), and the numerical value of the uncertainty score. In the columns
from left to right: first five images show low uncertainty and are correctly
classified, the second five images show high uncertainty but are still
correctly classified, and last five images show high uncertainty and are
wrongly classified. As can be seen the uncertainty computed through our
surrogates is a good indicator of the quality of the decision.

Despite its apparent simplicity, in one fell swoop CUSP addresses several
outstanding issues related to prediction uncertainty, which are otherwise
separately addressed by existing works. Our contributions are as follows:

  * •

Uncertainty surrogates help quantify prediction uncertainty through reliable,
yet computationally efficient measures.

  * •

We show through experiments how anomalies, outliers, and corrupted examples,
i.e out-of-distribution examples, can be detected with CUSP.

  * •

The regularization introduced by CUSP during training improves robustness
against adversarial attacks.

  * •

Due to the visual nature of the surrogates, we show that a secondary detector
can be trained to quantify the uncertitude of the classifier.

  * •

CUSP allows for visual inspection of the training progress as well the test
results, which along with quantified uncertainty values provides
interpretability.

We run several experiments over standard datasets to show the efficacy of our
approach and its superiority as compared to the state-of-the-art methods that
usually are able to address only a subset of the issues we tackle.

The rest of the paper is organized as follows: we present how uncertainty is
defined and categorized in literature along with the work in section 2. This
positions our method CUSP, which we introduce in detail in section 3. In
section 4 we provide extensive empirical results for the applicability and
trustworthiness of CUSP. Finally, we discuss and conclude in section 5.

##  2 Background

The lack of confidence estimates being an essential drawback for the
advancements of deep learning at production level, has inspired lot of recent
works addressing this challenge [18, 11, 21, 20, 33].

Research in the domain of uncertainty estimation could be categorized in
multiple ways: based on _implementation technique_ : augmenting the network
architecture, the loss function, or the data itself; based on the _machine
learning task_ : classification, regression, reinforcement learning etc; or
based on the _source of uncertainty_ : aleatoric or epistemic.

Figure 3: A deep network $\mathcal{F}$ generates a surrogate feature layer
$\mathbf{s}$ from an input image $\mathbf{x}$, which is then used for
predicting $\mathbf{y}$ as well as for assessing the uncertainty of the
prediction $\mathbf{y}$ by comparing the pre-determined pattern that
$\mathbf{s}$ is expected to resemble.

For the sake of computational efficiency, one of the most popular approaches
is using Monte-Carlo dropout [11], that provides an estimate for the
predictive uncertainty by aggregating results from multiple dropout masks.
Another method favoured for its intuitive approach is ensembling, wherein
multiple models are trained and the inherent variance between the prediction
of the models is used as a proxy for uncertainty [21]. Both of these methods
require custom loss functions and/or modifications to the architecture.

Several variants of these initial models were introduced as an attempt to
either consider the two sources of uncertainty [20] for MC dropout, or to
reduce the computational overhead [14, 34, 37], or to combine dropout with
ensembles [9].

Since it is quite challenging to develop methods targeting a specific ML
application or architecture, several existing approaches focus on the task at
hand. Some focus on classification [28, 15, 17, 10, 25], while others address
regression tasks [26, 4, 27] with the goal of obtaining high quality
prediction intervals, say, by allowing for non-Gaussian, non i.i.d noise in
the data.

A convenient choice for estimating uncertainty is to leverage the feature
representations at the final layer to distinguish between in-distribution and
out-of-distribution samples [33, 5, 22]. Tagasovska _et al_. [33] show that
while there are novel methods being introduced, baseline methods such as the
entropy of the softmax layer and its variants already provide hard-to-beat
uncertainty estimates for out-of-distribution detection. Our work, CUSP relies
on the pen-utlimate layer of networks, which can be related in spirit to these
simple yet seasoned methods.

Formally, uncertainty in machine learning is defined with regards to its
source. The term _aleatoric_ (from the latin word “alea” meaning randomness)
uncertainty relates to the inherent stohasticity in the data, i.e. noise,
which is irreducible, but, should be accounted for in a model. The term
_epistemic_ (from the Greek word “episteme” i.e. knowledge) uncertainty
relates to incomplete knowledge, whether that emanates from the lack of
observations in certain input regions, the presence of unmeasured variables,
or the propagation of errors from preceding operations for computing the
target values.

In Table 1, we distinguish as well as summarize the cases one should consider
when deciding on which type of uncertainty is most relevant for their
application. For example, guarding against adversarial attacks is related to
data noise in the input or target variables, whereas anomaly detection or
distribution shift is related to the epistemic uncertainty of the model.

| Aleatoric | Epistemic
---|---|---
Inputs $X$ |  | noisy measurements;
---
faulty equipment
| unobs. variables
---
unobs. input regions
Targets $Y$ |  |  $y$ derived from noisy $x$
---
disagreed class labels
| wrong numerical
---
/labeling model

Table 1: Overview of the categories of data-related uncertainty machine
learning.

Owing to methodological limitations or developing approaches with a certain
application in mind, most of the existing methods determining uncertainty
address only one or two of the cells in Table 1. In this work we are
interested computing an uncertainty estimate for deep learning that is
simultaneously sensitive to both aleatoric and epistemic uncertainty for the
task of classification.

In Figure 1 we show how uncertainties obtained from our approach CUSP relate
to three different situations. In the first row, we notice how the uncertainty
score increases with an additive random noise. In the second row we erase
parts of the image at random and again we notice the surrogates are very
sensitive to the missing patches. Both of these corruptions are related to the
aleatoric source of noise.

If, on the other hand, we rotate a given image from 0 to 180 degrees,
essentially creating out-of-distribution samples, we notice that the our
uncertainty estimation clearly reflects this. This is a representation of
unseen data, i.e. epistemic source of uncertainty in the third row in Figure
1.

Our choice of experiments is guided by Table 1: we will evaluate our method
CUSP w.r.t. input noise-specific experiments by way of adversarial attacks, to
target noise using label flips, and to out-of-distribution detection (both in-
domain and out-of-domain).

##  3 Uncertainty Surrogates

The crux of our work is in forcing the pen-utlimate layer of a deep network to
resemble a known pattern. This layer then serves as a proxy for a measure of
prediction uncertainty, which we term _uncertainty surrogate_. An Uncertainty
surrogate can be trained using a visual pattern, in which case it can
additionally provide qualitative feedback.

###  3.1 Problem setup

We are considering the case of a deep classifier $\mathcal{F}$ as composed of
a feature creating hidden part $h$ that computes feature vector
$\mathbf{s}\in\mathbb{R}^{m}$, which is then used by the usual linear
classifier $f$ to compute the prediction vector $\mathbf{y}$. That is,
$\mathbf{s}=f(\mathbf{x})$ and $\mathbf{y}=f(\mathbf{s})$. The Figure 3
explains this schematically.

The network $\mathcal{F}$ is trained on observations from a dataset
$\mathcal{D}\sim(X,T)$ where the inputs are random variables
$X\in\mathbb{R}^{d}$ with $d>m$ commonly, and $T\in\mathcal{K}\sim\\{1,2\dots
k\\}$ a categorical variable representing at most $k$ classes. We train our
model end-to-end on pairs of observations
$\\{\mathbf{x_{i}},\mathbf{t}_{i}\\}_{i=1}^{N}$ and simultaneously obtain
features $\mathbf{s}$ and prediction $\mathbf{y}$.

It is commonly understood that the pen-ultimate layer features $\mathbf{s}$,
give a high level representation of the data that is most informative for
downstream tasks. In our work, we aim to take advantage of this property of
$\mathbf{s}$ by additionally enforcing it to resemble a class-dependent visual
pattern. To this end, we introduce class-specific patterns,
$\\{\mathbf{p}_{k}|k,\forall k\in\mathcal{K}\\}$, where
$\mathbf{p_{k}}\in\mathbb{R}^{m}$ same as $\mathbf{s}$.

While training, we optimize over a sum of two losses, one for predicting the
correct class label (classification) and the other, which forces the surrogate
features to resemble the visual pattern (reconstruction). The loss function
therefore takes the form:

$\displaystyle\mathcal{L}\>\>$
$\displaystyle=\mathcal{L}_{1}+\alpha*\mathcal{L}_{2},\text{where}$ (1)
$\displaystyle\mathcal{L}_{1}$
$\displaystyle=-\sum_{i}^{\mathcal{K}}t_{i}\log(y_{i}),$ (2)
$\displaystyle\mathcal{L}_{2}$
$\displaystyle=-\sum_{i}^{m}p_{t_{i}}\log({s_{i}})+(1-p_{t_{i}})\log(1-s_{i}),$
(3)

with $\mathcal{L}_{1}$ being the categorical cross entropy (CCE) loss for
classification into $C$ classes w.r.t. ground truth $\mathbf{t}$, and
$\mathcal{L}_{2}$ being the logits-based binary cross entropy (BCE) loss for
reconstruction of the $m$ pixels of the surrogate pattern $\mathbf{p}$. We
choose $\alpha=0.5$ for all our experiments.

We use the binary cross entropy loss for training the network because for our
binary patterns it provided better visual quality as compared to mean square
error (MSE). Please note that the patterns need not be limited to a single
layer binary or gray-scale ones, they can be in color, or be composed of
multiple binary/gray-scale patterns, as long as the shape of the surrogate
layer $\mathbf{s}$ can be adapted to match that of the surrogate pattern
$\mathbf{p}$.

Apart from using an additional loss function, the only other modification done
to the regular deep learning process to make the size of the pen-ultimate
layer equal to the size of the surrogate pattern of choice. That is, if we use
$16\times 16$ patterns, we set the size of the penultimate layer to $256$.

|  |  |  |  |  |  |  |  |
---|---|---|---|---|---|---|---|---|---
|  |  |  |  |  |  |  |  |
|  |  |  |  |  |  |  |  |
|  |  |  |  |  |  |  |  |
|  |  |  |  |  |  |  |  |
Figure 4: Examples of surrogate patterns, _i.e_. fixed patterns, each
associated with a distinct class. The bright and dark pixels represent ones
and zeros, respectively. The top-row patterns are orthogonal. All others are
non-orthogonal patterns. The patterns can even be error-correcting codes like
QR-codes. Please note that such patterns can also be in color although we
stick to binary ones in this paper.

###  3.2 Choice of surrogate patterns

To train the uncertainty surrogates, the choice of the type for the surrogate
patterns can provide different properties. In Figure 4, we introduce an
initial proposal of visual cues that we use as patterns. If the patterns are
binary (or gray-scale) in pixel values, as is the case our work, the surrogate
patterns for each class can be orthogonal to each other or non-orthogonal.

If orthogonal patterns are chosen, the network is encouraged to introduce
orthogonality in the decision making earlier than at the output. Intuitively,
and according to literature, orthogonality of activations improves the
accuracy of deep classifiers [2]. However, the orthogonal patterns may not
carry any explicit visual meaning (see first row of Figure 4). More
importantly, the number of bright to dark pixels can drop significantly if the
number of classes is large. This may introduce far too much sparsity (because
the dark pixels in the patterns represent zeros) than may be acceptable for a
reasonable performance.

Non-orthogonal patterns can take any visual symbol and can have any desired
degree of sparsity. They can even be error correcting codes like QR codes,
which rely on Reed-Solomon error correction and do not necessarily have a
visual meaning. It is also possible to express domain knowledge in the choice
of surrogate patterns. For example, if two of the several classes in a
classification scenario are too similar and hence difficult to separate, the
surrogate patterns may be designed or chosen to ensure they are as dissimilar
as possible according to some metric like euclidean distance. This will force
the activations to be dissimilar making the classification task more
discriminative for this.

###  3.3 Obtaining prediction uncertainty

Having trained a network with the surrogate patterns, at inference time, for a
new sample $\mathbf{x}^{*}$, the deep model outputs
$\mathbf{y}^{*},\mathbf{s}^{*}=\mathcal{F}(\mathbf{x^{*}})$. We compute a
scalar uncertainty score

$u=\delta(\mathbf{p_{y*}},\mathbf{s^{*}}),$ (4)

which is the distance between $\mathbf{x}^{*}$ and the surrogate pattern
$\mathbf{p_{y*}}$ corresponding to the predicted label. For $\delta$ in
Equation 4, one could choose either BCE, which was used during training or MSE
to measure the distance, i.e the uncertainty between the surrogate pattern and
the reconstruction. We find MSE to be as effective or better than BCE as shown
in the experiments section.

The function $\delta$ need not be limited to MSE or BCE. Interestingly, since
the reconstructed pattern is of a visual nature, we can use a light CNN to
replace the $\delta$ function. This approach of using a secondary network for
uncertainty prediction is superior to using the BCE or MSE reconstruction
error as shown in Figure 5 albeit at a slightly higher computational cost.

As for the complexity of CUSP, since at test time we obtain the surrogates at
the same time as the label predictions, our method requires only computing
element wise distance in the dimension of the feature layer, i.e. pattern
shape $m$. Both the distance functions are quite efficient computationally,
linear in the number of pixels of the surrogate pattern, with MSE having a
slight edge over BCE.

##  4 Experiments

|  |
---|---|---
|  |
Figure 5: Plots of precision versus recall (top row) and ROC (bottom row) to
compare the uncertainty estimate provided by BCE, MSE, and the secondary CNN
trained on the recreated pattern. The three plots from left to right are for
the datasets of FashionMNIST, CIFAR-10, CIFAR-100, respectively. The plots
show that BCE, MSE, and our secondary CNN peform better than competing
methods.

In what follows we introduce an extensive empirical evaluation for the
benefits and various use cases of the surrogates uncertainties. In the coming
sub-sections we organize our results that address the following questions in
order:

  1. 1.

Is CUSP applicable to cases where the source of uncertainty is epistemic?

  2. 2.

Is CUSP applicable to cases where the source of uncertainty is aleatoric?

  3. 3.

How reliable is the CUSP score?

  4. 4.

Does CUSP increase the robustness of a deep model?

  5. 5.

Does CUSP improve the interpretability of the model?

#### Experimental setup

In all experiments we use the implementation of PreActResNet18 [24] with
modification only to the last layer to adjust for the dimensions of the
surrogate patterns. We use the same hyperparameters and architecture for CUSP
and all baselines with the only difference being the added reconstruction loss
term to the CUSP case.

For CUSP we have three parameters the user can set: the distance measure for
$\delta$ in Equation 4, the surrogate pattern and the size of the pattern and
consequently the feature layer.

We pick the datasets of MNIST and fashionMNIST for grayscale images, and those
of CIFAR10, SVHN and CIFAR100 for color images. All experiments were done on
Tesla GPUs P100-PCIE.

#### Baselines

We compare against multiple state-of-the-art uncertainty estimates that rely
on the last or penultimate layer of a classifier. We follow a similar setup as
the one of Tagasovska _et al_. [33]. We include as baselines, Bayesian linear
regression (covariance) baseline, distance to nearest training points
(distance), largest softmax score [16, largest], absolute difference between
the two largest softmax scores (functional), softmax entropy (entropy),
geometrical margin [35, geometrical], ODIN [22], random network distillation
[5, distillation], principal component analysis (PCA), a random baseline
(random), and an oracle trained to separate the in- and out-of- domain
examples. As a sanity check we also add a random score and an “oracle”, which
means we compute the score knowing the ground truth labels.

#### Metrics

As commonly done in such setups we use the _classification accuracy_ of a
network as well as the ROC _area under the curve_ (AUC). Essentially, ROC is a
probability curve and the area under it represents a measure of class
separability. The higher the AUC, the better the model is at predicting 0s as
0s and 1s as 1s.

###  4.1 Epistemic uncertainty

To confirm the applicability of CUSP under an epistemic sources of uncertainty
we propose out-of-distribution experiments. We address the two cases of 1)
same domain out-of-distribution detection and 2) different domain out-of-
distribution detection. In both cases we are interested in the AUC score,
which should give us an estimate on how different are the scores for samples
seen during training as opposed to novel ones.

Different domain out-of-distribution detection We consider one full dataset as
in-distribution and another one as out-of-distribution. In particular we
consider the following cases: training on CIFAR10, while testing on SVHN, and
vice versa; and training on MNIST, while testing on fashionMNIST, and vice-
versa. From Table 2 we notice that CUSP has the best results on MNIST and
SVHN. Moreover, we also note that it has consistently high scores across all
datasets, which is not the case with the rest of the baselines.

#### Same domain out-of-distribution detection

Here we split each of the four datasets at random into five “in-domain”
classes and five “out-of-domain” classes. We note that this experimental setup
is much more challenging than the previous one since in this case the in and
out of distribution examples are more similar than if taken from completely
different datasets.

From Table 3 we notice that CUSP has the highest AUC scores across all
datasets in this more challenging setup and with a very small variance across
different configurations and datasets.

| CIFAR10 | fashion | MNIST | SVHN | Average
---|---|---|---|---|---
certificates | $0.92\pm 0.01$ | $0.90\pm 0.05$ | $0.68\pm 0.10$ | $0.81\pm 0.00$ | 0.83 $\pm$ 0.09
covariance | $0.63\pm 0.00$ | $0.81\pm 0.00$ | $0.99\pm 0.00$ | $0.58\pm 0.00$ | 0.75 $\pm$ 0.16
distillation | $0.74\pm 0.10$ | $0.81\pm 0.02$ | $0.96\pm 0.02$ | $0.55\pm 0.01$ | 0.76 $\pm$ 0.15
entropy | 0.96 $\pm$ 0.02 | 0.95 $\pm$ 0.01 | $0.77\pm 0.03$ | 0.95 $\pm$ 0.00 | $0.91\pm 0.08$
functional | $0.94\pm 0.01$ | $0.93\pm 0.01$ | $0.79\pm 0.01$ | $0.94\pm 0.00$ | $0.90\pm 0.06$
geometrical | $0.87\pm 0.10$ | $0.72\pm 0.07$ | 0.98 $\pm$ 0.01 | $0.79\pm 0.15$ | $0.84\pm 0.09$
largest | $0.96\pm 0.02$ | $0.92\pm 0.02$ | $0.75\pm 0.03$ | $0.95\pm 0.00$ | $0.89\pm 0.08$
odin | $0.93\pm 0.07$ | $0.95\pm 0.01$ | $0.77\pm 0.04$ | $0.94\pm 0.01$ | $0.90\pm 0.07$
pca | $0.63\pm 0.16$ | $0.61\pm 0.06$ | $0.66\pm 0.10$ | $0.52\pm 0.01$ | $0.60\pm 0.05$
random | $0.51\pm 0.00$ | $0.51\pm 0.01$ | $0.50\pm 0.0$ | $0.51\pm 0.01$ | $0.50\pm 0.00$
oracle | $0.94\pm 0.00$ | $0.99\pm 0.01$ | $0.99\pm 0.00$ | $0.99\pm 0.01$ | $0.99\pm 0.00$
CUSP digits | 0.80443 | 0.91854 | 0.94160 | 0.92626 | $0.90\pm 0.05$
CUSP non ortho | 0.89647 | 0.80717 | 0.95507 | 0.95507 | $0.90\pm 0.06$
CUSP ortho | 0.85852 | 0.90069 | 0.98671 | 0.96140 | $0.93\pm 0.05$
CUSP BCE | $0.85\pm 0.04$ | $0.87\pm 0.04$ | $0.96\pm 0.01$ | $0.93\pm 0.05$ | $0.90\pm 0.04$
CUSP digits | 0.85528 | 0.80395 | 0.97382 | 0.95556 | $0.90\pm 0.07$
CUSP non ortho | 0.92223 | 0.93617 | 0.96616 | 0.95641 | $0.94\pm 0.01$
CUSP ortho | 0.90031 | 0.94077 | 0.93507 | 0.96002 | $0.93\pm 0.02$
CUSP MSE | $0.89\pm 0.03$ | $0.89\pm 0.03$ | $0.96\pm 0.01$ | 0.96 $\pm$ 0.01 | 0.92 $\pm$ 0.03

Table 2: AUC scores for out-of-distribution uncertainty. By column: average
and standard deviation across different hyper-parameters. By row: average and
standard deviation per method across datasets.

| CIFAR10 | fashion | MNIST | SVHN | Average
---|---|---|---|---|---
certificates | $0.75\pm 0.01$ | $0.74\pm 0.00$ | $0.87\pm 0.01$ | $0.87\pm 0.03$ | $0.80\pm 0.06$
covariance | $0.54\pm 0.00$ | $0.70\pm 0.08$ | $0.88\pm 0.09$ | $0.62\pm 0.00$ | $0.69\pm 0.12$
distance | $0.62\pm 0.03$ | $0.79\pm 0.00$ | $0.87\pm 0.09$ | $0.67\pm 0.02$ | $0.74\pm 0.10$
distillation | $0.56\pm 0.03$ | $0.68\pm 0.04$ | $0.72\pm 0.13$ | $0.79\pm 0.03$ | $0.69\pm 0.08$
entropy | $0.73\pm 0.00$ | $0.81\pm 0.00$ | $0.93\pm 0.02$ | $0.92\pm 0.01$ | $0.85\pm 0.08$
functional | $0.72\pm 0.00$ | $0.83\pm 0.01$ | $0.94\pm 0.01$ | $0.92\pm 0.01$ | $0.85\pm 0.08$
geometrical | $0.70\pm 0.08$ | $0.67\pm 0.10$ | $0.80\pm 0.11$ | $0.80\pm 0.09$ | $0.74\pm 0.05$
largest | $0.69\pm 0.03$ | $0.82\pm 0.00$ | $0.89\pm 0.04$ | $0.92\pm 0.01$ | $0.83\pm 0.08$
odin | $0.71\pm 0.03$ | $0.77\pm 0.05$ | $0.89\pm 0.01$ | $0.92\pm 0.02$ | $0.82\pm 0.08$
pca | $0.57\pm 0.06$ | $0.56\pm 0.05$ | $0.54\pm 0.05$ | $0.58\pm 0.05$ | $0.56\pm 0.14$
random | $0.50\pm 0.00$ | $0.51\pm 0.00$ | $0.50\pm 0.0$ | $0.52\pm 0.01$ | $0.50\pm 0.00$
oracle | $0.94\pm 0.00$ | $1.00\pm 0.01$ | $0.99\pm 0.00$ | $0.99\pm 0.01$ | $0.99\pm 0.00$
CUSP ortho | 0.78775 | 0.88854 | 0.96838 | 0.93540 | $0.90\pm 0.06$
CUSP non ortho | 0.73533 | 0.86589 | 0.96487 | 0.92602 | $0.87\pm 0.09$
CUSP digits | 0.79648 | 0.84193 | 0.94251 | 0.94348 | $0.88\pm 0.06$
CUSP BCE | $0.77\pm 0.02$ | 0.87 $\pm$ 0.02 | 0.96 $\pm$ 0.01 | 0.93 $\pm$ 0.01 | 0.88 $\pm$ 0.02
CUSP digits | 0.75920 | 0.74232 | 0.90327 | 0.87009 | $0.82\pm 0.07$
CUSP non ortho | 0.79259 | 0.78787 | 0.95465 | 0.92378 | $0.86\pm 0.07$
CUSP ortho | 0.79780 | 0.89320 | 0.95070 | 0.89052 | $0.88\pm 0.05$
CUSP MSE | 0.78 $\pm$ 0.02 | $0.81\pm 0.06$ | $0.94\pm 0.02$ | $0.90\pm 0.02$ | $0.86\pm 0.06$

Table 3: AUC scores for same-domain uncertainty. By column: average and
standard deviation across different hyper-parameters. By row: average and
standard deviation per method across datasets.

###  4.2 Aleatoric uncertainty

To investigate the reliability of CUSP under noise uncertainty in the targets,
we propose the following “flipped class” experiment. We chose pairs of
visually similar classes, and for 30% of the training samples we flip the
target labels. More specifically, some 1s get labeled as 7, some 4s as 9 and
some 3s as 8. In Figure 6 we observe that CUSP is indeed sensitive to
heteroscedastic noise, _i.e_., as desired, in the case of flipped labels the
uncertainty scores are consistently higher.

Figure 6: Sensitivity of CUSP scores to heteroskedastic noise. Uncertainty is
higher when trained on flipped cases. Experiment done with orthogonal patterns
and MSE loss.

###  4.3 Reliability of CUSP

In this section we evaluate the sensitivity of the surrogates with regards to
different choice of parameters.

Figure 7: Accuracy of classifiers under adversarial attacks. Error bars come
from different $\epsilon$ values for the FGM. It can be noticed that at any
level of adversarial attacks, the network trained with CUSP remains robust.

The setup is as follows: we split each of the datasets into three parts, in
the ratio 10:1:1. The first part is used to train a PreActResNet18 model to
predict the given patterns and the class labels. The second part is used to
predict the patterns, classes, and the MSE based uncertainties using the
trained PreActResNet18. The predicted patterns are now assigned binary labels
of _true_ or _false_ depending on whether the PreActResNet18 predicts the
ground truth labels correctly or not. Given these binary labels as targets,
the predicted patterns corresponding original patterns, and the predicted MSE-
based uncertainty scores are used to train a light 3-layer CNN to predict the
binary label with a single sigmoid output and BCE loss. The complement of this
scalar output is then treated as the uncertainty prediction. The third part of
the dataset is used to test the secondary CNN. As a note, since the number of
correct predictions largely exceeds incorrect ones, we use focal loss [23] for
training with $\gamma=2$.

Figure 5 shows the comparison of BCE, MSE, and the secondary CNN using an ROC
curve. To plot the curve, we threshold each of the three uncertainties at a
100 thresholds ranging from 0 to 1. Each threshold splits the given patterns
into _true_ and _false_ predictions, which are then compared with the binary
ground truth labels we collected on the second part of the dataset. This
permits us to compute counts of true positives, false positives, true
negatives, and false negatives at each threshold value of uncertainty. The
curves show that in all three datasets the CUSP scores whether computed with
BCE, MSE, or the second classifer (CNN proba) consistently outperforms the
strongest baselines.

###  4.4 CUSP robustness check

Adversarial attacks could be considered as a form of an aleatoric noise in the
inputs, but if the produced example is actually outside of the training
manifold, this becomes an epistemic source of uncertainty as well. In this
experiment, we are interested in actually, how sensitive will a CUSP network
be if exposed to a white-box attack. To this end we evaluate the accuracy of
the classifier when a network has been trained with the surrogate loss as
opposed to standard training. We introduce different levels of additive
adversarial noise following the Fast Gradient Method, (FGM) [13], and at each
level we measure the impact on the classifier accuracy.

The gradient sign method uses the gradient of the underlying model to produce
adversarial examples. An image $x$ is manipulated by adding or subtracting a
small error $\epsilon$ to each pixel. Adding errors in the direction of the
gradient means that the image is intentionally altered so that the model
classification fails. The following formula describes the fast gradient sign
method:

$x^{\prime}=x+\epsilon\text{sign}(\nabla_{x}J(\theta,x,y))$

$\nabla_{x}J(x,y)$ is the gradient of the models’ loss function with respect
to the original input pixel vector $x$, $y$ is the true label vector for $x$
and $\theta$ is the model parameter vector.

From Figure 7, we notice that that when confronted with adversarial attacks,
the networks with surrogate regularization manage to consistently keep higher
accuracy as compared to their baseline counterparts. This confirms the
increased robustness due to the CUSP training.

| entropy | largest | functional | BCE | MSE | CNN
---|---|---|---|---|---|---
Fashion | 0.8100 | 0.8216 | 0.8210 | 0.9693 | 0.9922 | 0.9927
CIFAR-10 | 0.7017 | 0.7613 | 0.7585 | 0.8841 | 0.9239 | 0.9325
CIFAR100 | 0.8023 | 0.8304 | 0.8220 | 0.9986 | 0.9985 | 0.9813

Table 4: AUC for the RoC curves of Figure 5. Compared to the state-of-the-art,
the uncertainty predicted by our approach is consistently better w.r.t. AUC
scores.

##  5 Conclusion

Providing the uncertainty of prediction for deep learning, which is mandatory
for any critical application of machine learning, was difficult or reserved to
computationally expensive methods. In this paper, we introduce a simple method
to make a regular deep network provide the prediction and the confidence of
its prediction, better than existing methods as shown qualitatively and
quantitatively. To achieve this, we place a simple constraint on the readily
available features of the penultimate layer to resemble a pre-determined
visual pattern. Not only does this provide better uncertainty estimates in a
computationally efficient manner, but it also allows us to detect out-of-
distribution examples and to resist adversarial attacks at no extra cost.
Additionally, the reconstructions of the predetermined patterns provide visual
feedback during training as well as testing, which, along with the uncertainty
estimates, renders the training process and the predictions of the network
more interpretable. Finally, we also hope that the visual interpretability of
our uncertainty score will open new directions for communicating the behavior
and results of deep models to the end users.

## References

  * [1] David Alvarez-Melis and Tommi Jaakkola.  A causal framework for explaining the predictions of black-box sequence-to-sequence models.  In EMNLP, 2017.
  * [2] Nitin Bansal, Xiaohan Chen, and Zhangyang Wang.  Can we gain more from orthogonality regularizations in training deep cnns?  arXiv preprint arXiv:1810.09102, 2018.
  * [3] Edmon Begoli, Tanmoy Bhattacharya, and Dimitri Kusnezov.  The need for uncertainty quantification in machine-assisted medical decision making.  Nature Machine Intelligence, 2019.
  * [4] Axel Brando, Jose A Rodríguez-Serrano, Jordi Vitria, and Alberto Rubio.  Modelling heterogeneous distributions with an uncountable mixture of asymmetric laplacians.  NeurIPS, 2019.
  * [5] Yuri Burda, Harrison Edwards, Amos Storkey, and Oleg Klimov.  Exploration by random network distillation.  arXiv, 2018.
  * [6] Varun Chandola, Arindam Banerjee, and Vipin Kumar.  Anomaly detection: A survey.  ACM computing surveys (CSUR), 2009.
  * [7] Tongfei Chen, Jiří Navrátil, Vijay Iyengar, and Karthikeyan Shanmugam.  Confidence scoring using whitebox meta-models with linear classifier probes.  arXiv, 2018.
  * [8] Corinna Cortes, Giulia DeSalvo, and Mehryar Mohri.  Learning with rejection.  In ALT, 2016.
  * [9] Nikita Durasov, Timur Bagautdinov, Pierre Baque, and Pascal Fua.  Masksembles for uncertainty estimation.  to appear CVPR, 2021.
  * [10] Michael Dusenberry, Ghassen Jerfel, Yeming Wen, Yian Ma, Jasper Snoek, Katherine Heller, Balaji Lakshminarayanan, and Dustin Tran.  Efficient and scalable bayesian neural nets with rank-1 factors.  In International conference on machine learning, pages 2782–2792. PMLR, 2020.
  * [11] Yarin Gal.  Uncertainty in Deep Learning.  PhD thesis, University of Cambridge, 2016.
  * [12] Yonatan Geifman and Ran El-Yaniv.  Selective classification for deep neural networks.  In NIPS, 2017.
  * [13] Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy.  Explaining and harnessing adversarial examples.  arXiv preprint arXiv:1412.6572, 2014.
  * [14] Marton Havasi, Rodolphe Jenatton, Stanislav Fort, Jeremiah Zhe Liu, Jasper Snoek, Balaji Lakshminarayanan, Andrew M Dai, and Dustin Tran.  Training independent subnetworks for robust prediction.  arXiv preprint arXiv:2010.06610, 2020.
  * [15] Matthias Hein, Maksym Andriushchenko, and Julian Bitterwolf.  Why relu networks yield high-confidence predictions far away from the training data and how to mitigate the problem.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 41–50, 2019.
  * [16] Dan Hendrycks and Kevin Gimpel.  A baseline for detecting misclassified and out-of-distribution examples in neural networks.  ICLR, 2017.
  * [17] Dan Hendrycks, Norman Mu, Ekin D Cubuk, Barret Zoph, Justin Gilmer, and Balaji Lakshminarayanan.  Augmix: A simple data processing method to improve robustness and uncertainty.  ICLR, 2020.
  * [18] José Miguel Hernández-Lobato and Ryan Adams.  Probabilistic backpropagation for scalable learning of bayesian neural networks.  In ICML, 2015.
  * [19] Victoria Hodge and Jim Austin.  A survey of outlier detection methodologies.  Artificial intelligence review, 2004.
  * [20] Alex Kendall and Yarin Gal.  What uncertainties do we need in bayesian deep learning for computer vision?  In NeurIPS, 2017.
  * [21] Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell.  Simple and scalable predictive uncertainty estimation using deep ensembles.  In NeurIPS, 2017.
  * [22] S. Liang, Y. Li, and R. Srikant.  Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks.  ICLR, 2018.
  * [23] Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár.  Focal loss for dense object detection.  In Proceedings of the IEEE international conference on computer vision, pages 2980–2988, 2017.
  * [24] Kuang Liu, 2018.
  * [25] Andrey Malinin and Mark Gales.  Prior networks for detection of adversarial attacks.  arXiv preprint arXiv:1812.02575, 2018.
  * [26] Tim Pearce, Alexandra Brintrup, Mohamed Zaki, and Andy Neely.  High-quality prediction intervals for deep learning: A distribution-free, ensembled approach.  In International Conference on Machine Learning, pages 4075–4084. PMLR, 2018.
  * [27] Xin Qiu, Elliot Meyerson, and Risto Miikkulainen.  Quantifying point-prediction uncertainty in neural networks via residual estimation with an i/o kernel.  ICLR, 2020.
  * [28] Murat Sensoy, Lance Kaplan, and Melih Kandemir.  Evidential deep learning to quantify classification uncertainty.  arXiv preprint arXiv:1806.01768, 2018.
  * [29] Burr Settles.  Active learning literature survey. 2010.  Computer Sciences Technical Report, 2014.
  * [30] Alireza Shafaei, Mark Schmidt, and James J Little.  Does your model know the digit 6 is not a cat? a less biased evaluation of” outlier” detectors.  arXiv, 2018.
  * [31] H Jeff Smith, Tamara Dinev, and Heng Xu.  Information privacy research: an interdisciplinary review.  MIS quarterly, 2011.
  * [32] Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus.  Intriguing properties of neural networks, 2014.
  * [33] Natasa Tagasovska and David Lopez-Paz.  Single-model uncertainties for deep learning.  NeurIPS, 2019.
  * [34] Linh Tran, Bastiaan S Veeling, Kevin Roth, Jakub Swiatkowski, Joshua V Dillon, Jasper Snoek, Stephan Mandt, Tim Salimans, Sebastian Nowozin, and Rodolphe Jenatton.  Hydra: Preserving ensemble diversity for model distillation.  arXiv preprint arXiv:2001.04694, 2020.
  * [35] Dan Wang and Yi Shang.  A new active labeling method for deep learning.  In IJCNN, 2014.
  * [36] YK Wen, BR Ellingwood, D Veneziano, and J Bracci.  Uncertainty modeling in earthquake engineering.  MAE center project FD-2 report, 2003.
  * [37] Yeming Wen, Dustin Tran, and Jimmy Ba.  Batchensemble: an alternative approach to efficient ensemble and lifelong learning.  ICLR, 2020.
  * [38] Lingxue Zhu and Nikolay Laptev.  Deep and confident prediction for time series at uber.  In ICDMW, 2017.