# “More than words”: Linking Music Preferences and Moral Values through Lyrics

###### Abstract

This study explores the association between music preferences and moral values
by applying text analysis techniques to lyrics. Harvesting data from a
Facebook-hosted application, we align psychometric scores of 1,386 users to
lyrics from the top 5 songs of their preferred music artists as emerged from
Facebook Page Likes. We extract a set of lyrical features related to each
song’s overarching narrative, moral valence, sentiment, and emotion. A machine
learning framework was designed to exploit regression approaches and evaluate
the predictive power of lyrical features for inferring moral values. Results
suggest that lyrics from top songs of artists people like inform their
morality. Virtues of hierarchy and tradition achieve higher prediction scores
($.20\leq r\leq.30$) than values of empathy and equality ($.08\leq r\leq.11$),
while basic demographic variables only account for a small part in the models’
explainability. This shows the importance of music listening behaviours, as
assessed via lyrical preferences, alone in capturing moral values. We discuss
the technological and musicological implications and possible future
improvements.

##  1 Introduction

The field of music recommender systems has a lot to gain from the fields of
music psychology and sociology [1, 2], where researchers have found converging
evidence that people listen to music that reflects their personality needs [3,
4, 5, 6, 7] and helps express their values [8, 9, 10]. For example,
extroverted people tend to choose more energetic and rhythmic tunes, while
listeners holding values of understanding and tolerance prefer more
sophisticated and complex music. Indeed, operationalising knowledge of how
personality traits relate to listener taste and preferences has already been
shown to improve music recommendations [11, 12] and to make them more diverse
[13]. Yet personality dispositions alone may not suffice to explain, and thus
model, our music listening behaviours.

Aiming to advance an integrative view of the music listener, which may benefit
music recommender system scenarios, we set to explore the less attended
relation between moral values and music preferences. If personal values are
conceived as intrinsic motivational goals, moral values reflect traits learned
under the influence of society, culture, and religion, amongst others, which
bond people together into groups. Considering music as an evolved tool of
social affiliation and bonding [14, 15], it is reasonable to speculate that
people may like certain music styles and genres because they provide stimuli
that match their morality-related needs.

We further hypothesise that moral values are expressed more clearly in a
verbal rather than non-verbal manner and examine their influence on musical
taste through lyrics. When people listen to sung music, their preferences are
driven by not only the audio content but also the content of lyrics [16].
Lyrics convey rich, multifaceted messages about societal issues such as love,
life and death, but also political or religious concepts, often independently
from melodic and other audio information [17]. Lyrical messages can support
listeners’ mental health [18]. Nonetheless, little is known about whether
lyrical information manifests links between psychological traits and music
preferences [19]. To what extent are moral values reflected in the lyrics of
one’s favourite songs? Do lyrics predict the moral traits of listeners?

To tackle these questions, we used data from the LikeYouth.org project, a
Facebook-hosted application developed specifically for research purposes as a
surveying tool and was mainly deployed in Italy. Upon providing their informed
consent, participants completed validated psychometric questionnaires for
personality, moral traits and basic human values, basic demographic
information such as age and gender, while agreed to share their Page Likes
(see [20] for a detailed description of the complete dataset). For the purpose
of this study we only analysed moral values scores and Likes on music artist
Pages. Combining these with information from the genius.com music database, we
obtained the lyrics from the five most popular songs per artist. We performed
both sentiment [21] and emotion [22] analysis on the obtained lyrics, assessed
their moral narratives employing the MoralStrength lexicon [23], and examined
themes and overarching narratives through topic modelling [24].

We built a series of regression models that infer moral traits from lyrical
content, demographics, and Likes-based features (e.g., artist popularity). Our
findings show that peoples’ worldviews and moral values are indeed reflected
in their music preferences as modelled through lyrics, in line with recent
literature [25, 26]. Extracting topic and moral features from lyrics
specifically increased model performance over sentiment, emotion, and
demographic features.

We contribute to the growing literature studying the interplay between music
and psychology, with findings that clearly link the preferences of people to
artists and songs that are in line with their moral values. Personalised
recommendations for streaming on-demand music can be greatly enriched by
including notions of moral worldviews about their listeners instead of only
shallow psychological attributes [5, 1, 10]. Such knowledge can be directly
implemented in psychologically aware music recommender systems, improving
music streaming services and contributing to listener wellbeing [27]. On a
different key, the relationship between moral worldviews and music preference
is crucial to inform communication experts about their choice of the most
appropriate music piece to accompany a social campaign.

##  2 Background

We operationalise morality via the Moral Foundations Theory (MFT) [28], which
expresses the psychological basis of moral reasoning in terms of five innate
foundations, namely Care/Harm, Fairness/Cheating, Loyalty/Betrayal,
Authority/Subversion, and Purity/Degradation. These can further collapse into
two superior foundations: Individualising (Care and Fairness), indicative of a
more liberal perspective, and Binding (Purity, Authority and Loyalty),
indicative of a more conservative outlook.

Moral foundations are considered to be higher psychological constructs than
the more commonly investigated personality traits [29]. They have been
associated with attitudes towards complex situations such as politics [30,
31], climate change [32], and vaccination [33, 34].

However, moral values have attracted less attention from music scientists.
Using data from an ad-hoc online survey comprising, among other items, MFQ
scores and preferences ratings on 13 music genres, Preniqi et al. [25] found
that people with higher levels of Binding foundations (e.g., more
authoritarian individuals) tend to listen to country and Christian music, the
lyrics of which often foster notions of tradition [3]. Those with lower levels
of Binding traits tend to prefer music genres such as punk and hip-hop, where
lyrics are known to challenge traditional values, and the status quo [8].
Individualising foundations were overall harder to predict (cf. [35]).
Furthermore, including demographic information (e.g., age, gender, political
views, education) improved MFT predictions marginally, indicating the ability
of music preferences alone to explain one’s moral values.

In the computational social science field, recent work has demonstrated the
predictability of MFT traits from a variety of digital data, including
gameplay [36], smartphone usage and web browsing [35]. Moral values can also
be explained by verbal data, as they can be more clearly communicated through
thoughts and opinions [34, 37, 23]. Several dictionary-based approaches for
predicting moral values expressed in texts such as tweets and other social
media posts have been proposed, including the Moral Foundations Dictionary
[38, 37] and the MoralStrength lexicon [23]. Here we employ the latter to
uncover moral narratives in song lyrics, which we then use to predict the
moral traits of listeners.

The relation between lyrics and music preferences has only recently started to
receive attention across music and social psychology disciplines. Some studies
have suggested associations between the personality or mental health of
songwriters and their lyrics [39, 40]. On the listener side, neurotic
individuals tend to listen to songs with more complex and less repetitive
lyrics that express negative emotions [41, 19]. More conscientious individuals
tend to prefer lyrics talking about achievements [19] but also about love
[42]. Importantly, preferences for lyrics are found to be predictive of
personality traits distinctly from audio or melodic preferences [19, 42].

Concerning moral values, in recent work, they have been found to explain a
unique and significant portion of the variance in the lyrical preferences of
different metal music sub-genre fans that was not already accounted for by
personality traits [26]. For example, preferring lyrics about celebrating
metal culture and unity was related to higher levels of the Loyalty foundation
and higher levels of extroversion. In U.S. popular music, an increase in
lyrics related to self-focus and -promotion since the 1980s has been shown to
manifest the increasing individualism of American society [43, 44].

|  | Census |  | MFT
---
All data
$n=3,920$
| MFT & $\geq$10
---
Page Likes
$n=1,386$
Gender | M | 48% | 54% | 53%
F | 52% | 46% | 47%
Age |  $<$25 | 23% | 21% | 29%
$\geq$25 | 77% | 79% | 71%
Table 1: Demographic breakdown of our data according to gender and age. The
“Census” column reports the national distribution per attribute according to
the statistics provided by the official census bureau [45].

##  3 Data Collection

The LikeYouth Facebook-hosted application was initially launched in March
2016, while the data used here were downloaded in September 2019. It was
deployed mainly in Italy, where approximately 64,000 people entered the
platform, from whom 3,920 users (90% geolocated in Italian territory) filled
out the MFT questionnaire correctly.

Of those, 47% did not provide their age due to the facultative nature of
LikeYouth. Because we wished to include age as a demographic predictor
variable, we inferred the missing values from all (e.g., not just music artist
related) Page Likes of the 3,920 users. Similar to [46] we created a sparse
matrix representation of Page Likes per user and applied sparse singular value
decomposition to reduce dimensionality, while binning the age attribute
(median $=25$) as “younger” ($<25$) and “older” ($\geq 25$) allowed to
approximate the official census distribution [45]. We then employed an XGBoost
classifier, to predict missing age values [35], with an estimated $AUROC=0.79$
and standard deviation $=0.018$. Acknowledging that age inference might add
bias to our models, we only use age as a predictor in isolated experiments
(see Table 4). We also run the same experiments keeping only users who
provided their age. Predictions were similar for Binding and slightly lower
for Individualising.

To ensure the stability of our regression models, we applied a simple activity
threshold. After extensive experimentation we chose to drop users with less
than 10 Facebook Page Likes related to music artists (Page category
selection), resulting in a reduced final dataset of 1,386 users. Table 1
reports the demographic breakdown of our data sample in terms of gender and
age, which follows closely the population distribution of the official Italian
census [45].

For the final 1,386 users, we retrieved song lyrics corresponding to their
music artist Page Likes using genius.com. Querying the Genius API, we
initially obtained the 10 most popular songs per artist alongside the
respective lyrics. We assume that if a user liked the Page of a specific
artist, then that artist’s most famous songs (as per Genius) reflect the music
preferences of the user. We carried out predictive tasks using the $n=$ 10, 5,
or 3 most popular songs from an artist and found that $n=$ 5 gave the best
compromise in terms of predictions, computational resources, and within-
musician variability in lyrical and audio content (see future work discussion)
while maintaining an optimal number of musicians and songs for our lyrics
data. Finally, we used the spaCy library [47, 48] to identify songs with
English lyrics only, resulting in 3,179 artists and 15,895 songs.

We also considered two additional, more shallow digital trace features that
can potentially convey information about user’s music habits, namely the
number of Page Likes per user (mean $=35.11$, standard deviation $=33.95$) and
a built-in feature of artist popularity from LikeYouth, based on the number of
Page followers.

We use LikeYouth because, to our best knowledge, it is the only dataset
providing MFT scores of individuals alongside a potential proxy of their music
preferences (e.g., artist Page Likes). A limitation of this approach is that
the data provided by LikeYouth are static and may thus refer to a snapshot of
music interests in time. Streaming platforms could offer richer information
about habitual music listening [7, 42]. Nonetheless, there is substantial
evidence that Facebook Page Likes can capture personality needs and personal
values [5, 20, 46]. Another limitation is that LikeYouth user MFT scores and
thus our predictive models cannot be made publicly available due to privacy
implications [34]. Instead, we have shared the lyrics data and related source
code for lyrical feature modeling in a GitHub
repository.111https://github.com/vjosapreniqi/lyrics-content-features

Type | Method | Features
---|---|---
Topics | LDA |  | Death/Fear/Violence,
---
Obscene, Romantic,
World/Time/Life
Morals | MoralStrength |  | Care, Fairness, Loyalty,
---
Authority, Purity
Sentiment | VADER |  | Negative, Positive,
---
Neutral, Compound
Emotions | NRC |  | Anger, Disgust, Fear,
---
Sadness, Anticipation,
Surprise, Joy, Trust
Table 2: Summary of lyrical features used in this study. Topic | Artist | Song Title | %
---|---|---|---
Romantic (0.39) | Mike Williams | Give it up | 99
Marc Anthony | I need to know | 99
NSYNC | I want you back | 99
Willie Nelson | Always on my mind | 98
Alexia | Because I miss you | 97
Obscene (0.24) | Tyga | Rack city | 98
Fat Joe | Yellow tape | 96
Cardi B | Bartier cardi | 95
Chamillionaire | Ridin’ | 95
21 Savage | Bank account | 91
World/Time/ Life (0.22) | Holly Herndon | Morning sun | 99
Noisecontrollers | The day | 97
Nathan East | Finally home | 96
Dave Gahan | Tomorrow | 94
Gabrielle Aplin | Start of time | 90
Death/Fear/ Violence (0.15) | Hatebreed | Destroy everything | 99
Fear Factory | Edgecrusher | 97
Eomac | Mandate for murder | 95
Destruction | Thrash till death | 92
Sabaton | Attack of dead men | 91
Table 3: LDA topic modelling: overall topic prevalence (in brackets below
topic descriptions) and 5 manually selected songs per topic as ranked by
descending topic proportion.

##  4 Lyrics Content Analysis

We extracted a set of textual features related to each song lyrics’
overarching narrative (topic modelling), moral valence, sentiment, and
emotion. Based on the corresponding feature modeling method, we applied
different levels of text preprocessing. Sentiment detection required only a
general cleanup while keeping punctuation and capitalization within the text.
For the other methods, we extracted Part Of Speech (POS) lemmas using the
spaCy lemmatizer [47]. On average, each lyrics contained 273 words and 108
lemmas.

###  4.1 Topic Modelling

Initially, we aimed to uncover common patterns in the lyrics narratives by
applying a topic modelling approach based on Latent Dirichlet Allocation (LDA)
[24]. We used LDA due to its simplicity, high accuracy in topic modelling, and
good computational efficiency [49]. The input of the LDA model is a term
frequency matrix of the corpus created by the song lyrics. To eliminate very
common terms that can lead to irrelevant topics, we ignored words with
frequency higher than 90%.

To derive the optimum number of topics $k$, we optimized the topic coherency
($C_{v}$ metric [50]) for models with $k\in[2,16]$ using a step size of 2. The
number of topics for which coherency was maximised was $k=4$. For $k>4$, we
obtained topics that were either generic or hard to characterise due to the
mixture of different words belonging to multiple topics. While for $k=4$, the
topics obtained were in line with previous literature [51, 52]. Table 3
depicts examples of manually selected songs of 5 artists for each topic,
ranked by descending weight in the specific topic.

###  4.2 Moral Valence

We assess the moral narratives by employing the MoralStrength lexicon [23],
which holds the state-of-the-art performance in moral text prediction. This
expands the Moral Foundation Dictionary by offering three times more moral-
annotated lemmas. The lexicon provides, along with each lemma, the moral
valence score, a numeric assessment that indicates both the polarity and the
intensity of the lemma in each of the five moral foundations (MFT traits).
Moral valence is expressed on a Likert scale from one to nine, with five
considered neutral. When lower than 5, scores reflect notions closer to Harm,
Cheating, Betrayal, Subversion, and Degradation, while values higher than 5
indicate Care, Fairness, Loyalty, Authority, and Purity, respectively.

We obtained a moral valence score for each lemma in a song’s lyrics and each
MFT trait, which is then averaged across lemmas for each song. Negation
correction was not applied, as moral foundation polarities do not directly
translate as opposites (e.g., “not care” is not the same as “harm”). The
MoralStrength lexicon has a limited linguistic coverage; as a result, we could
not predict moral valence for 16% of the collected lyrics. Instead, we
assigned them the value 5, the neutral point of the moral valence Likert
scale. This approach pushes the observed mean towards the center of the scale,
but captures the variability of the moral values across all the lyrical data.

###  4.3 Sentiment and Emotion Analysis

In textual data, emotions, as brief and preconscious phenomena, can be defined
via descriptions of appraisal, physiological reaction, expressive display,
feeling, or action tendency, while sentiments, as lasting and conscious
emotional dispositions, tend to be modelled in terms of text polarity
(positive, negative, neutral) [53].

We applied the commonly used VADER (Valence Aware Dictionary and sEntiment
Reasoner) model [21] on the lyrical text to obtain information about the
sentiment of each song. The VADER model is shown to perform well both with
long and short text, providing for each song a score for positive, neutral,
negative, and compound sentiment (see Table 2). We also estimated the eight
basic emotions defined in the Plutchik wheel of emotions [54] employing the
NRC Word–Emotion Association Lexicon [22]. This lexicon was shown to be
efficient with unlabeled data [55]. Each song lyrics was annotated with the
eight emotions (see Table 2) by averaging its word emotion association scores.

##  5 Experiments and Results

Figure 1: Spearman’s Rank correlations of MFT super foundations with
demographics, artist likes and lyrics features. We report only ones that were
significant at $p\leq.01$. “Artist liked no” refers to the number of artist
Likes per user.

Initially, we explored the relationship between users’ moral values as emerged
from the self-reported questionnaires, basic demographic attributes, and their
respective music preferences, as expressed in the linguistic components of the
lyrics. Figure 1 depicts the statistically significant correlations
($p\leq.01$) obtained for the two superior foundations, namely Individualising
and Binding. We observed that people who value more Individualising
foundations prefer artists whose songs prevalently talk about anticipation and
trust. On the other side, those concerned more about social order and Binding
foundations tend to prefer artists who deal with more romantic topics in their
songs instead of existential and social issues. Overall, participants with
strong Binding foundations display a tendency to dislike songs with negative
valence and emotions such as sadness, fear, or disgust. Yet both the
Individualising and Binding groups resonate with positive and joyful songs,
showing that despite often profound differences in sociopolitical stances,
music is a shelter to everyone.

ID | Features
---|---
EX1 | Sentiment (VADER)
EX2 | Emotions (NRC)
EX3 | Sentiment + Emotions
EX4 | Best of {EX1, EX2, EX3} + Morals
EX5 | Best of {EX1, EX2, EX3} + Topics
EX6 | Best of {EX1, EX2, EX3} + Morals + Topics
EX7 | EX6 + Age + Gender
EX8 | EX7 + Artist Likes + Artist Popularity
Table 4: Summary of performed experiments with corresponding features used as
predictors. Moral Foundations - Regression Models
---
| EX1 | EX2 | EX3 | EX4 | EX5 | EX6 | EX7 | EX8
C | .08 [.08, .09] | .10 [.10, .11] | .10 [.10, .11] | .11 [.11, .12] | .10 [.10, .11] | .12 [.11, .12] | .12 [.12, .13] | .11 [.11, .12]
F | .04 [.04, .05] | .06 [.05, .06] | .05 [.04, .05] | .06 [.05, .06] | .06 [.05, .06] | .05 [.05, .05] | .08 [.07, .08] | .05 [.05, .05]
L | .12 [.12, .13] | .16 [.16, .17] | .18 [.17, .18] | .20 [.20, .21] | .19 [.18, .19] | .19 [.19, .20] | .20 [.20, .21] | .20 [.20, .21]
A | .19 [.19, .19] | .21 [.21, .22] | .23 [.23, .24] | .26 [.26, .26] | .24 [.24, .24] | .25 [.25, .26] | .26 [.26, .26] | .27 [.26, .27]
P | .19 [.18, .19] | .20 [.20, .21] | .24 [.23, .24] | .25 [.25, .26] | .23 [.22, .23] | .25 [.24, .25] | .24 [.24, .25] | .26 [.26, .26]
I | .08 [.07, .08] | .10 [.10, .11] | .10 [.10, .11] | .10 [.09, .10] | .09 [.09, .10] | .10 [.10, .11] | .11 [.10, .11] | .10 [.10, .11]
B | .20 [.19, .20] | .24 [.23, .24] | .26 [.26, .27] | .28 [.28, .29] | .26 [.26, .27] | .28 [.27, .28] | .27 [.27, .28] | .30 [.30, .31]
Table 5: Moral foundations regression with Random Forest using different
feature combinations (see Table 4): Pearson’s correlation [95% confidence
intervals] between predicted and the actual values averaged across 5-fold
cross-validation. C: Care; F: Fairness; L: Loyalty; A: Authority; P: Purity;
I: Individualising; B: Binding. Figure 2: Top 4 individual feature
contributions (via SHAP values) for the five basic moral foundations from
experiment EX8 (see Table 4). The higher the SHAP value, the more the feature
contributes to the prediction model.

Next, we proposed a series of experimental designs to infer moral values of
the participants from their music preferences and the respective linguistic
content. Table 4 summarises the performed experiments. We employed four
algorithms, namely Support Vector Regressor, Random Forest, XGBoost, and
ElasticNet, to predict moral values using a multivariate regression approach
over a 5-fold cross-validation setting. For each participant, the features
were aggregated and normalised. Here, we report only the results from the
Random Forest since it slightly outperformed the rest. We used the Pearson’s
correlation coefficients between the predicted and actual moral values scores
to measure the model’s goodness fit. This metric was commonly used in papers
that predicted personality based on users’ music preferences and listening
behaviours [5, 56].

To comprehend the general behaviour of our models and evaluate the importance
of each feature, we estimated the SHAP values. SHAP (SHapley Additive
exPlanations) is a game theory approach designed to illustrate the features’
contribution to the final output of any machine learning model [57].

Following the incremental experimental design reported in Table 4, we trained
one model per each moral foundation and presented the best results obtained by
each feature in Table 5. In line with recent literature [25] that shows higher
prediction accuracy for Binding rather than Individualising foundations, we
noticed a similar behaviour also when inferring from linguistic features of
song lyrics.

When adding demographics and artist Facebook information (EX8), the results
slightly improved for both super foundations, implying that the more
information we have about users’ demographics and music preferences, the more
precise our models become. Despite that, the model trained on just emotions,
sentiment and moral information (EX4) achieved almost as good results as those
who are aware of the demographics and the general artist information (EX7 and
EX8). This highlights the importance of music preferences in portraying our
goals and decisions whose motivations go far beyond basic demographic
knowledge.

Figure 2 depicts the most important individual (only top 4 due to page
restrictions) features for predicting each of the five moral foundations.
While Figure 3 illustrates the impact individual (top 8) and grouped features
in inferring the two superior foundations when considering all predictor
variables (EX8). In line with observed correlations, feature importance
representations for regression models show that lyrics linked to objective and
subtle emotions (e.g., joy, trust, and anticipation) effectively predict Care
and Fairness. Whereas more intense and opposite polarities of sentiment and
emotions (e.g., fear, sadness, lyrics positive and negative valence) account
for better predictions of Loyalty, Authority and Purity. We noticed that those
who value more the Binding foundations appear to be sensitive to the
popularity of the song, which reflects their worldview of prioritising group-
focus over self-focus.

Figure 3: Individual (top 8) and grouped feature contributions (via SHAP
values) for the two superior moral foundations from experiment EX8 (see Table
4). The higher the SHAP value, the more the feature contributes to the
prediction model.

##  6 Conclusion

This paper discussed the link between lyrical information and moral values. We
presented a wide range of lyrics processing techniques and features for
measuring the power of linguistic aspects in predicting complex psychological
traits such as moral values. Besides, we explored and compared the impact of
user demographics and shallow digital traces in inferring moral foundations
against the song lyrics components.

We noticed that Binders express their views throughout their music preference
and lyrical styles. In contrast, Individualising views are more complex to be
captured solely by people’s music lyrics preferences. Thus, using the proposed
framework, it was easier to infer moral values of Binding ($.20\leq r\leq.30$)
between predicted and target values than Individualising foundations ($.08\leq
r\leq.11$).

We demonstrated that lyrics features extracted from the naturally emerging
music preferences in social media, to some extent, allow for constructing
reliable inferences of moral values. Considering the expanded presence of
online music streaming services our findings may have direct implications for
music recommendation and personalisation algorithms [5, 58, 1]. Since moral
values are a key element of the decision making process in several societal
issues [59, 35] and highly linked to political leanings [60], our research
implications can help future studies to tackle aspects of why and how music is
or can be used for mass stimulation and persuasion in social and political
campaigns, raising awareness on what our digital music behaviours can reveal.

In future work, we intend to combine audio and lyrical content analysis
together in a multimodal framework to further expand our understanding of
music and moral affiliations, especially for Individualising foundations that
remain hard to predict. Recent work highlights that preferences for both
lyrics and audio features are important in predicting, often distinctly,
personality traits [42]. We will also use additional data from LikeYouth to
investigate if moral foundations can explain variance in music preferences
that cannot be accounted for by personality traits and personal values (cf.
[26]). We ultimately aim to integrate our findings into novel psychologically
aware music recommender systems, but also beyond the music domain to other
media.

##  7 Acknowledgements

This work was supported by the QMUL Centre for Doctoral Training in Data-
informed Audience-centric Media Engineering (2021–2025) as part of a PhD
studentship awarded to VP. KK acknowledges support from the Lagrange Project
of the Institute for Scientific Interchange Foundation (ISI Foundation) funded
by Fondazione Cassa di Risparmio di Torino (Fondazione CRT). We would like to
thank the three anonymous reviewers and the meta-reviewer for their thoughtful
comments.

## References

  * [1] L. Porcaro, C. Castillo, and E. Gómez Gutiérrez, “Diversity by design in music recommender systems,” _Transactions of the International Society for Music Information Retrieval. 2021; 4 (1)._ , 2021.
  * [2] A. Laplante, “Improving music recommender systems: What can we learn from research on music tastes?” in _ISMIR_ , 2014, pp. 451–456.
  * [3] P. J. Rentfrow and S. D. Gosling, “The do re mi’s of everyday life: the structure and personality correlates of music preferences.” _Journal of personality and social psychology_ , vol. 84, no. 6, p. 1236, 2003.
  * [4] D. M. Greenberg, M. Kosinski, D. J. Stillwell, B. L. Monteiro, D. J. Levitin, and P. J. Rentfrow, “The song is you: Preferences for musical attribute dimensions reflect personality,” _Social Psychological and Personality Science_ , vol. 7, no. 6, pp. 597–605, 2016.
  * [5] G. Nave, J. Minxha, D. M. Greenberg, M. Kosinski, D. Stillwell, and J. Rentfrow, “Musical preferences predict personality: evidence from active listening and facebook likes,” _Psychological science_ , vol. 29, no. 7, pp. 1145–1158, 2018.
  * [6] S. P. Devenport and A. C. North, “Predicting musical taste: Relationships with personality aspects and political orientation,” _Psychology of Music_ , vol. 47, no. 6, pp. 834–847, 2019.
  * [7] I. Anderson, S. Gil, C. Gibson, S. Wolf, W. Shapiro, O. Semerci, and D. M. Greenberg, ““just the way you are”: Linking music listening on spotify and personality,” _Social Psychological and Personality Science_ , vol. 12, no. 4, pp. 561–572, 2021.
  * [8] A. Gardikiotis and A. Baltzis, “‘rock music for myself and justice to the world!’: Musical identity, values, and music preferences,” _Psychology of Music_ , vol. 40, no. 2, pp. 143–163, 2012.
  * [9] V. Swami, F. Malpass, D. Havard, K. Benford, A. Costescu, A. Sofitiki, and D. Taylor, “Metalheads: The influence of personality and individual differences on preference for heavy metal.” _Psychology of Aesthetics, Creativity, and the Arts_ , vol. 7, no. 4, p. 377, 2013.
  * [10] S. Manolios, A. Hanjalic, and C. C. Liem, “The influence of personal values on music taste: Towards value-based music recommendations,” in _Proceedings of the 13th ACM Conference on Recommender Systems_ , 2019, pp. 501–505.
  * [11] R. Hu and P. Pu, “Enhancing collaborative filtering systems with personality information,” in _Proceedings of the fifth ACM conference on Recommender systems_ , 2011, pp. 197–204.
  * [12] Y. Jin, N. Tintarev, N. N. Htun, and K. Verbert, “Effects of personal characteristics in control-oriented user interfaces for music recommender systems,” _User Modeling and User-Adapted Interaction_ , vol. 30, no. 2, pp. 199–249, 2020.
  * [13] F. Lu and N. Tintarev, “A diversity adjusting strategy with personality for music recommendation.” in _IntRS@ RecSys_ , 2018, pp. 7–14.
  * [14] C. Loersch and N. L. Arbuckle, “Unraveling the mystery of music: Music as an evolved group process.” _Journal of Personality and Social Psychology_ , vol. 105, no. 5, p. 777, 2013.
  * [15] P. E. Savage, P. Loui, B. Tarr, A. Schachner, L. Glowacki, S. Mithen, and W. T. Fitch, “Music as a coevolved system for social bonding,” _Behavioral and Brain Sciences_ , vol. 44, 2021.
  * [16] A. M. Demetriou, A. Jansson, A. Kumar, and R. M. Bittner, “Vocals in music matter: the relevance of vocals in the minds of listeners.” in _ISMIR_ , 2018, pp. 514–520.
  * [17] S. O. Ali and Z. F. Peynircioğlu, “Songs and emotions: are lyrics and melodies equal partners?” _Psychology of music_ , vol. 34, no. 4, pp. 511–534, 2006.
  * [18] J. H. Lee, A. Bhattacharya, R. Antony, N. K. Santero, and A. Le, “Finding home: Understanding how music supports listener mental health through a case study of bts,” in _ISMIR_ , 2021, pp. 358–365.
  * [19] L. Qiu, J. Chen, J. Ramsay, and J. Lu, “Personality predicts words in favorite songs,” _Journal of Research in Personality_ , vol. 78, pp. 25–35, 2019\.
  * [20] A. Urbinati, K. Kalimeri, A. Bonanomi, A. Rosina, C. Cattuto, and D. Paolotti, “Young adult unemployment through the lens of social media: Italy as a case study,” in _International Conference on Social Informatics_. Springer, 2020, pp. 380–396.
  * [21] C. Hutto and E. Gilbert, “Vader: A parsimonious rule-based model for sentiment analysis of social media text,” in _Proceedings of the international AAAI conference on web and social media_ , vol. 8, no. 1, 2014, pp. 216–225.
  * [22] S. M. Mohammad and P. D. Turney, “Crowdsourcing a word–emotion association lexicon,” _Computational intelligence_ , vol. 29, no. 3, pp. 436–465, 2013\.
  * [23] O. Araque, L. Gatti, and K. Kalimeri, “Moralstrength: Exploiting a moral lexicon and embedding similarity for moral foundations prediction,” _Knowledge-based systems_ , vol. 191, p. 105184, 2020.
  * [24] D. M. Blei, A. Y. Ng, and M. I. Jordan, “Latent dirichlet allocation,” _Journal of machine Learning research_ , vol. 3, no. Jan, pp. 993–1022, 2003\.
  * [25] V. Preniqi, K. Kalimeri, and C. Saitis, “Modelling moral traits with music listening preferences and demographics,” _arXiv preprint arXiv:2107.00349_ , 2021.
  * [26] K. J. Messick and B. E. Aranda, “The role of moral reasoning & personality in explaining lyrical preferences,” _PLoS one_ , vol. 15, no. 1, p. e0228057, 2020.
  * [27] Y. Mejova and K. Kalimeri, “Effect of values and technology use on exercise: implications for personalized behavior change interventions,” in _Proceedings of the 27th ACM Conference on User Modeling, Adaptation and Personalization_ , 2019, pp. 36–45.
  * [28] J. Graham, J. Haidt, S. Koleva, M. Motyl, R. Iyer, S. P. Wojcik, and P. H. Ditto, “Moral foundations theory: The pragmatic validity of moral pluralism,” in _Advances in experimental social psychology_. Elsevier, 2013, vol. 47, pp. 55–130.
  * [29] D. P. McAdams and J. L. Pals, “A new big five: fundamental principles for an integrative science of personality.” _American psychologist_ , vol. 61, no. 3, p. 204, 2006.
  * [30] A. Miles and S. Vaisey, “Morality and politics: Comparing alternate theories,” _Social Science Research_ , vol. 53, pp. 252–269, 2015.
  * [31] J. Haidt and J. Graham, “When morality opposes justice: Conservatives have moral intuitions that liberals may not recognize,” _Social Justice Research_ , vol. 20, no. 1, pp. 98–116, 2007.
  * [32] C. Wolsko, H. Ariceaga, and J. Seiden, “Red, white, and blue enough to be green: Effects of moral framing on climate change attitudes and conservation behaviors,” _Journal of Experimental Social Psychology_ , vol. 65, pp. 7–19, 2016.
  * [33] A. B. Amin, R. A. Bednarczyk, C. E. Ray, K. J. Melchiori, J. Graham, J. R. Huntsinger, and S. B. Omer, “Association of moral values with vaccine hesitancy,” _Nature Human Behaviour_ , vol. 1, no. 12, pp. 873–880, 2017\.
  * [34] K. Kalimeri, M. G. Beiró, A. Urbinati, A. Bonanomi, A. Rosina, and C. Cattuto, “Human values and attitudes towards vaccination in social media,” in _Companion Proceedings of The 2019 World Wide Web Conference_ , 2019, pp. 248–254.
  * [35] K. Kalimeri, M. G. Beiró, M. Delfino, R. Raleigh, and C. Cattuto, “Predicting demographics, moral foundations, and human values from digital behaviours,” _Comput. Hum. Behav._ , vol. 92, pp. 428–445, 2019.
  * [36] E. Kim, R. Iyer, J. Graham, Y.-H. Chang, and R. Maheswaran, “Moral values from simple game play,” in _International Conference on Social Computing, Behavioral-Cultural Modeling, and Prediction_. Springer, 2013, pp. 56–64.
  * [37] Y. Lin, J. Hoover, G. Portillo-Wightman, C. Park, M. Dehghani, and H. Ji, “Acquiring background knowledge to improve moral value prediction,” in _2018 ieee/acm international conference on advances in social networks analysis and mining (asonam)_. IEEE, 2018, pp. 552–559.
  * [38] J. Graham, J. Haidt, and B. A. Nosek, “Liberals and conservatives rely on different sets of moral foundations.” _Journal of personality and social psychology_ , vol. 96, no. 5, p. 1029, 2009.
  * [39] D. M. Greenberg, S. C. Matz, H. A. Schwartz, and K. R. Fricke, “The self-congruity effect of music.” _Journal of Personality and Social Psychology_ , 2020.
  * [40] E. J. Lightman, P. M. McCarthy, D. F. Dufty, and D. S. McNamara, “Using computational text analysis tools to compare the lyrics of suicidal and non-suicidal songwriters,” in _Proceedings of the Annual Meeting of the Cognitive Science Society_ , vol. 29, no. 29, 2007.
  * [41] J. Shriram, S. Paruchuri, and V. Alluri, “How much do lyrics matter? analysing lyrical simplicity preferences for individuals at risk of depression,” _arXiv preprint arXiv:2109.07227_ , 2021.
  * [42] L. Sust, G. Kudchadker, R. Schoedel, T. Schuwerk, M. Bühner, and C. Stachl, “Personality computing with naturalistic music listening data,” _PsyArXiv Preprints_ , 2022.
  * [43] C. N. DeWall, R. S. Pond Jr, W. K. Campbell, and J. M. Twenge, “Tuning in to psychological change: Linguistic markers of psychological traits and emotions over time in popular us song lyrics.” _Psychology of Aesthetics, Creativity, and the Arts_ , vol. 5, no. 3, p. 200, 2011.
  * [44] P. McAuslan and M. Waung, “Billboard hot 100 songs: Self-promoting over the past 20 years.” _Psychology of Popular Media Culture_ , vol. 7, no. 2, p. 171, 2018.
  * [45] I.Stat, “Italian statistics,” http://dati.istat.it/, accessed: 2019-09.
  * [46] M. Kosinski, Y. Bachrach, P. Kohli, D. Stillwell, and T. Graepel, “Manifestations of user personality in website choice and behaviour on online social networks,” _Machine learning_ , vol. 95, no. 3, pp. 357–380, 2014.
  * [47] M. Honnibal and I. Montani, “spaCy 2: Natural language understanding with Bloom embeddings, convolutional neural networks and incremental parsing,” 2017, to appear.
  * [48] M. Neumann, D. King, I. Beltagy, and W. Ammar, “ScispaCy: Fast and robust models for biomedical natural language processing,” in _Proceedings of the 18th BioNLP Workshop and Shared Task_. Florence, Italy: Association for Computational Linguistics, Aug. 2019, pp. 319–327. [Online]. Available: https://aclanthology.org/W19-5034
  * [49] A. Lancichinetti, M. I. Sirer, J. X. Wang, D. Acuna, K. Körding, and L. A. N. Amaral, “High-reproducibility and high-accuracy method for automated topic classification,” _Physical Review X_ , vol. 5, no. 1, p. 011007, 2015.
  * [50] M. Röder, A. Both, and A. Hinneburg, “Exploring the space of topic coherence measures,” in _WSDM_ , 2015, pp. 399–408.
  * [51] S. Sasaki, K. Yoshii, T. Nakano, M. Goto, and S. Morishima, “Lyricsradar: A lyrics retrieval system based on latent topics of lyrics.” in _Ismir_ , 2014, pp. 585–590.
  * [52] L. Misael, C. Forster, E. Fontelles, V. Sampaio, and M. França, “Temporal analysis and visualisation of music,” in _Anais do XVII Encontro Nacional de Inteligência Artificial e Computacional_. SBC, 2020, pp. 507–518.
  * [53] M. Munezero, C. S. Montero, E. Sutinen, and J. Pajunen, “Are they different? affect, feeling, emotion, sentiment, and opinion detection in text,” _IEEE transactions on affective computing_ , vol. 5, no. 2, pp. 101–111, 2014\.
  * [54] R. Plutchik, “A psychoevolutionary theory of emotions,” 1982.
  * [55] E. Çano and M. Morisio, “Moodylyrics: A sentiment annotated lyrics dataset,” in _Proceedings of the 2017 International Conference on Intelligent Systems, Metaheuristics & Swarm Intelligence_, 2017, pp. 118–124.
  * [56] A. Anderson, L. Maystre, I. Anderson, R. Mehrotra, and M. Lalmas, “Algorithmic effects on the diversity of consumption on spotify,” in _Proceedings of The Web Conference 2020_ , 2020, pp. 2155–2165.
  * [57] S. M. Lundberg and S.-I. Lee, “A unified approach to interpreting model predictions,” _Advances in neural information processing systems_ , vol. 30, 2017.
  * [58] J. R. Ogden, D. T. Ogden, and K. Long, “Music marketing: A history and landscape,” _Journal of Retailing and Consumer Services_ , vol. 18, no. 2, pp. 120–125, 2011.
  * [59] J. Haidt and C. Joseph, “Intuitive ethics: How innately prepared intuitions generate culturally variable virtues,” _Daedalus_ , vol. 133, no. 4, pp. 55–66, 2004.
  * [60] G. Lakoff, _Moral politics: How liberals and conservatives think_. University of Chicago Press, 2010.


Lemma 1.8 (ii)). By compactness of $K$, we may assume that $K^{\prime}$ has
finitely many connected components. Replacing $K$ with a $G$-orbit of
connected components of $K^{\prime}$, we may assume that $K/G$ is connected.

We may suppose that $f\neq 0$. After shrinking $S$, we may assume that $S/G$
is connected, that $f\in\mathcal{O}(S)^{G}$, and that $f$ is nonnegative on
$S^{G}$. Let $p:T\to S$ be the finite surjective $G$-equivariant holomorphic
map of normal $G$-equivariant Stein spaces associated with the finite étale
$\mathcal{M}(S)^{G}$-algebra ${F:=\mathcal{M}(S)^{G}[x]/\langle
x^{2}+f\rangle}$ by Proposition 5.4. The nonnegativity hypothesis on $f$ and
the fact that $-f$ is a square in $\mathcal{M}(T)^{G}$, hence in
$\mathcal{O}(T)^{G}$ by normality of $T$, imply that $T^{G}$ is contained in
the nowhere dense closed analytic subset $\\{f=0\\}$ of $T$. Let $L\subset T$
be a $G$-orbit of connected components of $p^{-1}(K)$. It is a $G$-invariant
Stein compact subset of $T$ by [GR79, V, §1.1, Theorem 1 d)]. By Corollary
5.8, the field $\mathcal{M}(L)^{G}$ has finite cohomological dimension, and
hence cannot be ordered.

As
$\mathcal{M}(K)^{G}\subset\mathcal{M}(L)^{G}\subset\mathcal{M}(K)^{G}[x]/\langle
x^{2}+f\rangle$, one has $\mathcal{M}(L)^{G}=\mathcal{M}(K)^{G}$ or
$\mathcal{M}(L)^{G}=\mathcal{M}(K)^{G}[x]/\langle x^{2}+f\rangle$. In the
first case, $f$ is a sum of squares in $\mathcal{M}(K)^{G}$ by [Lam05, VIII,
Theorem 1.10 and Proposition 1.1 (2)]. In the second case, $f$ is a sum of
squares in $\mathcal{M}(K)^{G}$ by [Lam05, VIII, Theorem 1.10 and Basic Lemma
1.4]. ∎

######  Remark 6.2.

Theorem 6.1 can be proven by the more elementary methods of [Jaw86], but one
could not recover in this way the quantitative results of Theorem 6.4.

###  6.2. Sums of few squares

If $A$ is a ring, we let $\mathrm{cd}_{2}(A)$ denote the cohomological
$2$-dimension of the étale site of $\mathrm{Spec}(A)$ (see [Sch94, Definition
7.1]).

######  Proposition 6.3.

Let $F$ be a field such that $\mathrm{cd}_{2}(F[x]/\langle x^{2}+1\rangle)\leq
n$. If $f\in F$ is a sum of squares in $F$, then it is a sum of $2^{n}$
squares in $F$.

###### Proof.

If $\mathrm{char}(F)=2$, then sums of squares in $F$ are squares in $F$, so we
may assume that $\mathrm{char}(F)\neq 2$. We may also assume that $f\neq 0$.
Denote by $\\{a\\}\in H^{1}(F,\mathbb{Z}/2)$ the class induced by $a\in F^{*}$
via the Kummer isomorphism $F^{*}/(F^{*})^{2}\ext@arrow
0359\arrowfill@\relbar\relbar{\hbox{$\rightarrow$}}{}{\,\sim\,}H^{1}(F,\mathbb{Z}/2)$.

As $f$ is a sum of squares, it is positive with respect to all the field
orderings of $F$. By a theorem of Arason [Ara75, Satz 3], the class
$\\{f\\}\cdot\\{-1\\}^{N}\in H^{N+1}(F,\mathbb{Z}/2)$ vanishes for $N\gg 0$.
Consider the étale $F$-algebra $F^{\prime}:=F[x]/\langle x^{2}+1\rangle$ and
set $\Gamma:=\mathrm{Gal}(F^{\prime}/F)$. For all $N\geq 0$, the short exact
sequence

$0\to\mathbb{Z}/2\to\mathbb{Z}/2[\Gamma]\to\mathbb{Z}/2\to 0$

of $\Gamma$-modules induces an exact sequence

$H^{N}(F^{\prime},\mathbb{Z}/2)\to
H^{N}(F,\mathbb{Z}/2)\xrightarrow{\cdot\\{-1\\}}H^{N+1}(F,\mathbb{Z}/2).$

A decreasing induction on the degree shows that $\\{f\\}\cdot\\{-1\\}^{N}=0$
for all $N\geq n$. The Milnor conjectures proven by Voevodsky [Voe03] now
imply that $f$ is a sum of $2^{n}$ squares in $F$ (see [Ben20, Proposition
2.1]). ∎

The next theorem is a quantitative improvement of Theorem 6.1 in the spirit of
Pfister’s theorem [Pfi67, Theorem 1].

######  Theorem 6.4.

Let $K$ be a $G$-invariant Stein compact subset of a reduced $G$-equivariant
Stein space $S$ of dimension $n$. Let $f\in\mathcal{O}(K)^{G}$ be nonnegative
on a neighborhood of $K^{G}$ in $S^{G}$. Then $f$ is a sum of $2^{n}$ squares
in $\mathcal{M}(K)^{G}$.

###### Proof.

Arguing as in the proof of Theorem 6.1, we may assume that $S$ is normal and
that $K/G$ is connected. Then $F:=\mathcal{M}(K)^{G}$ is a field and
${F[x]/\langle x^{2}+1\rangle\simeq\mathcal{M}(K)}$ has cohomological
dimension $\leq n$ by Corollary 5.11 applied to the connected components of
$K$. Apply Theorem 6.1 and Proposition 6.3 to conclude. ∎

###  6.3. Real-analytic geometry

Theorem 6.4 has applications to real-analytic variants of Hilbert’s $17$th
problem. We follow the conventions of [GMT86] and refer to [GMT86, II,
Definition 1.4] for the definitions of real-analytic spaces and real-analytic
varieties. If $K$ is a closed subset of a real-analytic space $M$, we denote
by $\mathcal{O}(K)$ (resp.$\mathcal{M}(K)$) the ring of germs of real-analytic
functions (resp.of real-analytic meromorphic functions) in a neighborhood of
$K$.

######  Theorem 6.5.

Let $M$ be a real-analytic space of dimension $n$ with reduced local rings.
Let $K\subset M$ be a compact subset. Let $f\in\mathcal{O}(K)$ be nonnegative
on a neighborhood of $K$ in $M$. Then $f$ is a sum of $2^{n}$ squares in
$\mathcal{M}(K)$.

###### Proof.

By [GMT86, III, Theorems 3.6 and 3.10], there exist a reduced $G$-equivariant
Stein space $S$ of dimension $n$, and an isomorphism $M\ext@arrow
0359\arrowfill@\relbar\relbar{\hbox{$\rightarrow$}}{}{\,\sim\,}S^{G}$ of real-
analytic spaces. In addition, there exists a proper injective $G$-equivariant
holomorphic map $i:S\to\mathbb{C}^{N}$ for $N\gg 0$ (see [GMT86, V, Theorem
3.7]).

The compact subset $i(K)\subset\mathbb{R}^{N}$ admits a basis of Stein
neighborhoods in $\mathbb{C}^{N}$ (see [Siu69, Lemma 5]) which we may choose
to be $G$-invariant as the intersection of two Stein open subsets is Stein
(see [GR79, p. 127]). We deduce that $K$ admits a basis of $G$-invariant Stein
neighborhoods in $S$ (use [GR79, V, §1.1, Theorem 1 d)]). Use [GMT86, III,
Proposition 1.8] to choose one such neighborhood $S$ with the property that
$f$ extends to a holomorphic map $g:S\to\mathbb{C}$. After replacing $g$ with
$(g+\overline{g\circ\sigma})/2$, we may assume that it is $G$-equivariant. One
may now apply Theorem 6.4 to the function $g$ to conclude. ∎

######  Remark 6.6.

Theorem 6.5 applies to normal real-analytic varieties of pure dimension by
[GMT86, IV, Proposition 3.8], hence to real-analytic manifolds.

## References

  * [ABF14] F. Acquistapace, F. Broglia and J. F. Fernando, _On Hilbert’s 17th problem and Pfister’s multiplicative formulae for the ring of real analytic functions_ , Ann. Sc. Norm. Super. Pisa Cl. Sci. (5) 13 (2014), no. 2, 333–369.
  * [ABF19] by same author, _Normalization of complex analytic spaces from a global viewpoint_ , J. Geom. Anal. 29 (2019), no. 3, 2888–2930.
  * [ABF22] by same author, _Topics in global real analytic geometry_ , Springer Monographs in Math., Springer, Cham, 2022.
  * [ABFR05] F. Acquistapace, F. Broglia, J. F. Fernando and J. M. Ruiz, _On the Pythagoras numbers of real analytic surfaces_ , Ann. Sci. ENS 38 (2005), no. 5, 751–772.
  * [ABFR10] by same author, _On the finiteness of Pythagoras numbers of real meromorphic functions_ , Bull. SMF 138 (2010), no. 2, 231–247.
  * [ADCR03] C. Andradas, A. Díaz-Cano and J. M. Ruiz, _The Artin–Lang property for normal real analytic surfaces_ , J. Reine Angew. Math. 556 (2003), 99–111.
  * [Ara75] J. K. Arason, _Primideale im graduierten Wittring und im ${\rm mod}$ $2$ Cohomologiering_, Math. Z. 145 (1975), no. 2, 139–143.
  * [Art27] E. Artin, _Über die Zerlegung definiter Funktionen in Quadrate_ , Abh. Math. Sem. Univ. Hamburg 5 (1927), no. 1, 100–115.
  * [AT19] D. Abramovich and M. Temkin, _Functorial factorization of birational maps for qe schemes in characteristic 0_ , Algebra Number Theory 13 (2019), no. 2, 379–424.
  * [Ben20] O. Benoist, _Sums of squares in function fields over Henselian local fields_ , Math. Ann. 376 (2020), no. 1-2, 683–692.
  * [Bha12] B. Bhatt, _Annihilating the cohomology of group schemes_ , Algebra Number Theory 6 (2012), no. 7, 1561–1577.
  * [Bin76a] J. Bingener, _Holomorph-prävollständige Resträume zu analytischen Mengen in Steinschen Räumen_ , J. Reine Angew. Math. 285 (1976), 149–171.
  * [Bin76b] by same author, _Schemata über Steinschen Algebren_ , Schr. Math. Inst. Univ. Münster (2), vol. 10, 1976.
  * [BKS81] J. Bochnak, W. Kucharz and M. Shiota, _On equivalence of ideals of real global analytic functions and the 17th Hilbert problem_ , Invent. Math. 63 (1981), no. 3, 403–421.
  * [BM21] B. Bhatt and A. Mathew, _The arc-topology_ , Duke Math. J. 170 (2021), no. 9, 1899–1988.
  * [Bou06] N. Bourbaki, _Éléments de mathématique. Algèbre commutative. Chapitres 1 à 4_ , reprint of the 1985 original ed., Springer, Berlin, 2006.
  * [BR75] J. Bochnak and J.-J. Risler, _Le théorème des zéros pour les variétés analytiques réelles de dimension $2$_, Ann. Sci. ENS 8 (1975), no. 3, 353–363.
  * [BW21] O. Benoist and O. Wittenberg, _The tight approximation property_ , J. Reine Angew. Math. 776 (2021), 151–200.
  * [Car57a] H. Cartan, _Quotient d’un espace analytique par un groupe d’automorphismes_ , Algebraic geometry and topology. A symposium in honor of S. Lefschetz, Princeton University Press, Princeton, N.J., 1957, pp. 90–102.
  * [Car57b] by same author, _Variétés analytiques réelles et variétés analytiques complexes_ , Bull. Soc. Math. France 85 (1957), 77–99.
  * [Cho22] C.-Y. Chough, _Proper base change for étale sheaves of spaces_ , JEMS 24 (2022), no. 11, 3679–3702.
  * [CM86] M. Colţoiu and N. Mihalache, _On the homology groups of Stein spaces and Runge pairs_ , J. Reine Angew. Math. 371 (1986), 216–220.
  * [Cox79] D. A. Cox, _The étale homotopy type of varieties over ${\bf R}$_, Proc. AMS 76 (1979), no. 1, 17–22.
  * [CTP90] J.-L. Colliot-Thélène and R. Parimala, _Real components of algebraic varieties and étale cohomology_ , Invent. Math. 101 (1990), no. 1, 81–99.
  * [EGA42] A. Grothendieck, _Éléments de géométrie algébrique: IV. Étude locale des schémas et des morphismes de schémas, II_ , Publ. Math. IHES 24 (1965).
  * [EGA43] by same author, _Éléments de géométrie algébrique: IV. Étude locale des schémas et des morphismes de schémas, III_ , Publ. Math. IHES 28 (1966).
  * [Fer21] J. F. Fernando, _Positive semidefinite analytic functions on real analytic surfaces_ , J. Geom. Anal. 31 (2021), no. 12, 12375–12410.
  * [For17] F. Forstnerič, _Stein manifolds and holomorphic mappings_ , second ed., Ergeb. Math. Grenzgeb. (3), vol. 56, Springer, Cham, 2017.
  * [Fri67] J. Frisch, _Points de platitude d’un morphisme d’espaces analytiques complexes_ , Invent. Math. 4 (1967), 118–138.
  * [GG73] M. Golubitsky and V. Guillemin, _Stable mappings and their singularities_ , Graduate Texts in Math., vol. 14, Springer-Verlag, New York-Heidelberg, 1973.
  * [GMT86] F. Guaraldo, P. Macrì and A. Tancredi, _Topics on real analytic spaces_ , Advanced Lectures in Math., Friedr. Vieweg & Sohn, Braunschweig, 1986.
  * [God58] R. Godement, _Topologie algébrique et théorie des faisceaux_ , Actualités Scientifiques et Industrielles, vol. 1252, Hermann, Paris, 1958, Publ. Math. Univ. Strasbourg. No. 13.
  * [GR65] R. C. Gunning and H. Rossi, _Analytic functions of several complex variables_ , Prentice-Hall, Inc., Englewood Cliffs, N.J., 1965.
  * [GR79] H. Grauert and R. Remmert, _Theory of Stein spaces_ , Grundlehren der Mathematischen Wissenschaften, vol. 236, Springer-Verlag, Berlin-New York, 1979\.
  * [GR84] by same author, _Coherent analytic sheaves_ , Grundlehren der Mathematischen Wissenschaften, vol. 265, Springer-Verlag, Berlin, 1984.
  * [Gra58] H. Grauert, _On Levi’s problem and the imbedding of real-analytic manifolds_ , Ann. of Math. (2) 68 (1958), 460–472.
  * [Gur88] R. M. Guralnick, _Matrices and representations over rings of analytic functions and other one-dimensional rings_ , Visiting scholars’ lectures—1987 (Lubbock, TX), Texas Tech Univ. Math. Ser., vol. 15, Texas Tech Univ., Lubbock, TX, 1988, pp. 15–35.
  * [Hak72] M. Hakim, _Topos annelés et schémas relatifs_ , Ergeb. Math. Grenzgeb., vol. 64, Springer-Verlag, Berlin-New York, 1972.
  * [Ham83] H. A. Hamm, _Zur Homotopietyp Steinscher Räume_ , J. Reine Angew. Math. 338 (1983), 121–135.
  * [Har77] R. Hartshorne, _Algebraic geometry_ , Graduate Texts in Math., vol. 52, Springer-Verlag, New York-Heidelberg, 1977.
  * [Hil88] D. Hilbert, _Ueber die Darstellung definiter Formen als Summe von Formenquadraten_ , Math. Ann. 32 (1888), no. 3, 342–350.
  * [HL98] G. Henkin and J. Leiterer, _The Oka–Grauert principle without induction over the base dimension_ , Math. Ann. 311 (1998), no. 1, 71–93.
  * [Hou61] C. Houzel, _Géométrie analytique locale III_ , Familles d’espaces complexes et fondements de la géométrie analytique, exposé 20, Séminaire H. Cartan E.N.S., vol. 13, 1960/61.
  * [Iss66] Hej Iss’sa, _On the meromorphic function field of a Stein variety_ , Ann. of Math. (2) 83 (1966), 34–46.
  * [Jaw82] P. Jaworski, _Positive definite analytic functions and vector bundles_ , Bull. Acad. Polon. Sci. Sér. Sci. Math. 30 (1982), 501–506.
  * [Jaw86] by same author, _Extensions of orderings on fields of quotients of rings of real analytic functions_ , Math. Nachr. 125 (1986), 329–339.
  * [Jaw92] by same author, _The 17th Hilbert problem for noncompact real analytic manifolds_ , Real algebraic geometry (Rennes, 1991), Lecture Notes in Math., vol. 1524, Springer, Berlin, 1992, pp. 289–295.
  * [Kie67] R. Kiehl, _Note zu der Arbeit von J. Frisch: “Points de platitude d’un morphisme d’espaces analytiques complexes”_ , Invent. Math. 4 (1967), 139–141.
  * [KS21] K. Kurano and K. Shimomoto, _Ideal-adic completion of quasi-excellent rings (after Gabber)_ , Kyoto J. Math. 61 (2021), no. 3, 707–722.
  * [Kuc05] W. Kucharz, _Meromorphic functions and factoriality_ , Proc. AMS 133 (2005), no. 7, 2013–2021.
  * [Lam05] T. Y. Lam, _Introduction to quadratic forms over fields_ , Graduate Studies in Math., vol. 67, Amer. Math. Soc., Providence, RI, 2005.
  * [Loj64] S. Lojasiewicz, _Triangulation of semi-analytic sets_ , Ann. Scuola Norm. Sup. Pisa Cl. Sci. (3) 18 (1964), 449–474.
  * [Mar75] A. Markoe, _Invariance of holomorphic convexity under proper mappings_ , Math. Ann. 217 (1975), no. 3, 267–270.
  * [Mat80] H. Matsumura, _Commutative algebra_ , second ed., Math. Lecture Note Series, vol. 56, Benjamin/Cummings Publishing Co., Inc., Reading, Mass., 1980\.
  * [Mih96] N. Mihalache, _Special neighbourhoods of subsets in complex spaces_ , Math. Z. 221 (1996), no. 1, 49–58.
  * [Nar60] R. Narasimhan, _Imbedding of holomorphically complete complex spaces_ , Amer. J. Math. 82 (1960), 917–934.
  * [Nar61] by same author, _The Levi problem for complex spaces_ , Math. Ann. 142 (1961), 355–365.
  * [Nar62] by same author, _The Levi problem for complex spaces. II_ , Math. Ann. 146 (1962), 195–216.
  * [NS77] F. Norguet and Y.-T. Siu, _Holomorphic convexity of spaces of analytic cycles_ , Bull. Soc. Math. Fr. 105 (1977), no. 2, 191–223.
  * [Pfi67] A. Pfister, _Zur Darstellung definiter Funktionen als Summe von Quadraten_ , Invent. Math. 4 (1967), 229–237.
  * [Rui85] J. M. Ruiz, _On Hilbert’s 17th problem and real Nullstellensatz for global analytic functions_ , Math. Z. 190 (1985), no. 3, 447–454.
  * [Sch70] H. W. Schuster, _Infinitesimale Erweiterungen komplexer Räume_ , Comment. Math. Helv. 45 (1970), 265–286.
  * [Sch94] C. Scheiderer, _Real and étale cohomology_ , Lecture Notes in Math., vol. 1588, Springer-Verlag, Berlin, 1994.
  * [Ser94] J.-P. Serre, _Cohomologie galoisienne_ , fifth ed., Lecture Notes in Math., vol. 5, Springer-Verlag, Berlin, 1994.
  * [SGA41] A. Grothendieck, _Théorie des topos et cohomologie étale des schémas (SGA 4 I)_ , Lecture Notes in Math., vol. 269, Springer-Verlag, Berlin-New York, 1972.
  * [SGA42] by same author, _Théorie des topos et cohomologie étale des schémas (SGA 4 II)_ , Lecture Notes in Math., vol. 270, Springer-Verlag, Berlin-New York, 1972.
  * [SGA43] by same author, _Théorie des topos et cohomologie étale des schémas (SGA 4 III)_ , Lecture Notes in Math., vol. 305, Springer-Verlag, Berlin-New York, 1973.
  * [Sho94] V. V. Shokurov, _Riemann surfaces and algebraic curves_ , Algebraic geometry, I, Encyclopaedia Math. Sci., vol. 23, Springer, Berlin, 1994, pp. 1–166.
  * [Siu69] Y.-T. Siu, _Noetherianness of rings of holomorphic functions on Stein compact subsets_ , Proc. AMS 21 (1969), 483–489.
  * [SP] A. J. de Jong et al., _The Stacks Project_ , https://stacks.math.columbia.edu.
  * [Swa98] R. G. Swan, _Néron–Popescu desingularization_ , Algebra and geometry (Taipei, 1995), Lect. Algebra Geom., vol. 2, Int. Press, Cambridge, MA, 1998, pp. 135–192.
  * [Tem08] M. Temkin, _Desingularization of quasi-excellent schemes in characteristic zero_ , Adv. Math. 219 (2008), no. 2, 488–522.
  * [Tog67] A. Tognoli, _Proprietà globali degli spazi analitici reali_ , Ann. Mat. Pura Appl. (4) 75 (1967), 143–218.
  * [Voe96] V. Voevodsky, _Homology of schemes_ , Selecta Math. (N.S.) 2 (1996), no. 1, 111–153.
  * [Voe03] by same author, _Motivic cohomology with ${\bf Z}/2$-coefficients_, Publ. Math. IHES (2003), no. 98, 59–104.
  * [Zam76] W. R. Zame, _Holomorphic convexity of compact sets in analytic spaces and the structure of algebras of holomorphic germs_ , Trans. AMS 222 (1976), 107–127.


# Detrive: Imitation Learning with Transformer Detection for End-to-End
Autonomous Driving

1st Daoming Chen
3rd Feng Chen  Bristol Robotics Laboratory
University of Bristol
Bristol, UK
<EMAIL_ADDRESS>Zhejiang VIE Science & Technology Co. Ltd.
Zhejiang, China
<EMAIL_ADDRESS>2nd Ning Wang
4th Tony Pipe  Bristol Robotics Laboratory
University of the West of England
Bristol, UK
<EMAIL_ADDRESS>Bristol Robotics Laboratory
University of the West of England
Bristol, UK
<EMAIL_ADDRESS>
###### Abstract

This Paper proposes a novel Transformer-based end- to-end autonomous driving
model named Detrive. This model solves the problem that the past end-to-end
models cannot detect the position and size of traffic participants. Detrive
uses an end-to-end transformer based detection model as its perception module;
a multi-layer perceptron as its feature fusion network; a recurrent neural
network with gate recurrent unit for path planning; and two controllers for
the vehicle’s forward speed and turning angle. The model is trained with an
on-line imitation learning method. In order to obtain a better training set, a
rein- forcement learning agent that can directly obtain a ground truth
bird’s-eye view map from the Carla simulator as a perceptual output, is used
as teacher for the imitation learning. The trained model is tested on the
Carla’s autonomous driving benchmark. The results show that the Transformer
detector based end-to- end model has obvious advantages in dynamic obstacle
avoidance compared with the traditional classifier based end-to-end model.

###### Index Terms:

autonomous driving, imitation learning, trans- former

##  I Introduction

Since 1988, the first autonomous vehicle, ALVINN, has been recognised as a
successful example of the use of neural networks for autonomous driving [1].
Modern autonomous driving technology was started since 2004, as DARPA [2]
hosted its first grand challenge in self-driving, and attracted the brightest
minds for studying this topic. After 2007, with the appearance of Lidar, high-
resolution sensors, and the success of the DARPA Urban Challenge, the
foundation of autonomous driving technology was built. Companies like Google
(Waymo) started their research in autonomous vehicles (AV), and the so-called
AV 1.0 was formed.

One of the most important technology revolutions for au- tonomous driving is
the rise of Deep Learning in 2012. Since then, the AV commercial companies
have flourished around the world. In the early stages, Artificial Intelligence
(AI) was applied only to computer vision in the AV industry. In 2021, the
concept of AV 2.0 was stated by a company called Wayve [20]. AV 2.0 means an
end-to-end autonomous driving approach that uses only one large neural network
for all driving challenges. It replaces the old control-based decision making
module with a neural network. This is the key to solving the scale-up problem
that AVs face now.

One of the biggest breakthroughs in AI in the last five years is the module
called transformer [10]. Since its emergence, almost all models in the Natural
Language Process (NLP) field have been inseparable from the transformer
module. In recent years, researchers start to explore the possibility of
applying transformer in the computer vision field [17] [22] [15] [27], and
those transformer based models are breaking the limitation of traditional CNN
based models. However, there are not enough studies of the application of
transformers in autonomous driving.

Most previous end-to-end autonomous driving methods only use a classification
model as their perception module [11] [4] [12] [16] [29] [26]. One of the
reasons why detection models like Faster-RCNN [5] or YOLO [7] are not used in
end- to-end AD models is that neither of those models are end-to- end.
Therefore, to solve this problem, the end-to-end detection model, DETR [15],
is used in this paper’s proposed model, which has not been studied before.

Motivated by the problem mentioned above, this paper aims to explore the
application of using transformer models as the perception module for an end-
to-end autonomous driving. The model is developed based on the open source
CARLA simu- lator [8], and the model is evaluated on the CARLA leader- board
[28] to fairly compare with other people’s methods. The source code of the
model used on CARLA leaderboard is released on https://github.com/Alexbeast-
CN/Detrive

##  II RELATED WORK

Since the 1980s, more and more researchers have been working on autonomous
vehicles using the end-to-end paradigm [1]. This long-history paradigm now has
two dif- ferent branches. One branch is based on deep reinforcement learning
and another branch is based on imitation learning. The deep reinforcement
learning method is to allow the robot coverage to an optimal policy [13]. The
imitation learning method uses an expert to make labels for the deep neural
network to learn from [3].

The advantage of the end-to-end paradigm is that using neural networks is
easier to scale up comparing to those hand- crafted, rule-based conventional
methods. Autonomous driving is a typical problem affected by the long tail
effect. There are countless corner cases in everyday driving scenarios which
are not covered by the training dataset. The modular pipeline models have to
firstly define those cases and then hand code the solution. On the other hand,
the AI-based end-to-end model can solve these problems by collecting data
which is much easier [20].

However, the deep learning methods are still a black box to human. When an
end-to-end AV made a wrong decision and caused an accident, it’s difficult to
explain how the decision was made and how to prevent such problems in the
future [18].

###  II-A Intermediate representation

The intermediate representation paradigm was first proposed in 2015 [4]. It is
a hybrid version of the end-to-end paradigm and the pipeline paradigm. It
emphasises using state-of-the- art deep learning for the perception module to
obtain more comprehensive and accurate state observations. These state
observations are also known as affordances. As defined in [12], the
affordances are objects or rules that temporally or spatially limited the
movement of the ego vehicle. The items of affordances are listed as follows:

  * •

The distance from the ego vehicle’s centre point to the centre line of the
lane $d$.

  * •

The relative angle between the ego vehicle’s head direction and the centre
line of the lane $\psi$.

  * •

The distance from the ego vehicle to the front vehicle $l_{1}$.

  * •

The distance from the ego vehicle to a pedestrian $l_{2}$.

  * •

The traffic lights.

  * •

The speed sign.

In recent years, this paradigm has been widely accepted by industry and
academia, but in a bird’s-eye-view (BEV) form instead [23] [21] [25] [30]. The
BEV intermediate representation is to map the sensor input into a top down
view. The first step is to use a neural network to detect the major traffic
participants and estimate their relative position to the ego vehicle. Then, a
sensor fusion algorithm will transform the features into a BEV map.

###  II-B Conditional Imitation Learning

One of the breakthroughs for autonomous driving via imita- tion learning is
proposed by Codevilla et al [11] [14]. Compared to previous methods that only
use images as input, Conditional Imitation Learning (CIL) embeds other sensor
input such as speed and the navigational command. The embedding method uses a
multilayer perceptron neural network (MLP) to transform the raw data into the
same dimension as the output image features. Those image features together
with other feature is then feed into another MLP to generate actions. The
architecture of the model is shown in Figure 1. With these auxiliary inputs,
the CIL model is able to drive the vehicle in many complex scenarios.
Therefore, it is one of the strongest baselines for end-to-end autonomous
driving. Until today, the architecture of conditional imitation learning can
still be found in the state of the art models [26].

Figure 1: The architecture of conditional imitation learning[11]

##  III METHODOLOGY

###  III-A System Overview

Previous end-to-end imitation learning approach, such as CILRS [11] is able to
drive in urban streets, but it is very likely to collide with other traffic
participants in dense traffic sce- narios. One of the possible reasons for
this defect comes from the perception module. The perception module in CILRS
is an ImageNet pretrained ResNet [6], which can only classify images into 1000
object labels. However, this doesn’t make sense for autonomous driving, since
the location and shape of each object can’t be detected by ResNet. Therefore,
to improve driving ability, the classification perception module needs to be
replaced by a detection model. The detailed illustration of this model is in
the Section III-B.

Except for the architecture of the imitation learning network, the dataset for
training the model is another key component for good results. Previous methods
normally use a human as an expert to generate datasets. However, to update a
neural network with over a million parameters will take the expert thousands
of hours of demonstration. It is not only time- consuming, but also a waste of
financial investment and human effort. Another way to collect an expert
dataset is to hand code an autopilot based on the traffic rules. Those
autopilots can access the ground truth information from the simulator. In most
cases, the autopilot can drive the vehicle safely. However, it is very
difficult to hand code an rule-based autopilot that can perfectly handle all
driving scenarios. To tackle this problem, this paper uses a reinforcement
learning agent which is trained based on the ground truth Bird’s-Eye-View
(BEV) map. The RL expert is much more robust than the rule-based autopilot
when evaluated on the NoCrash-dense benchmark [24].

Figure 2: The overall architecture of the Detrive model

The model proposed in this paper is called Detrive. It is an autonomous
driving model that uses DETR as its perception module. The overall
architecture of the Detrive model is shown in Figure 2. There are two models,
the RL model uses no sensor but a ground truth BEV map for generating actions
that form the training dataset. The imitation learning model is an agent that
can be deployed on an ego vehicle. There are three main modules in the
imitation learning model, the perception module can detect both the label and
bounding box of an object; the planning module takes the detection result as
input and generates future waypoints; the control module use those waypoints
to generate driving actions. The inputs of the ego vehicle are a front view
monocular camera, speedometer, and a Global Navigation Satellite System (GNSS)
that can give a global navigation from point A to point B.

The imitation learning model is another key component of this paper. The
overview of the model is shown in Figure 3. As mentioned above, the model can
be divided into three main modules. The perception module is modified from the
DETR model. It uses a backbone to extract features from the input images. The
extracted features will then be positional encoded and pass through a
transformer module. The transformer module will transform those features into
a vector of object classification and bounding boxes. The vector then will be
concatenated with tensors from other sensors. The concate- nated vector is the
input of the trajectory planning network. The planning module first uses a
multilayer perceptron (MLP) to perform information fusion on the input vector.
Then this vector is passed together with the goal point tensor, into a gated
recurrent unit (GRU) network for predicting future way points. Lastly, the
control module uses a PID controller to generate steering angle, throttle, and
brake action to fit the predicted trajectory.

###  III-B Perception Module

The perception module uses a DETR-like network architec- ture that is divided
into four parts, a backbone for extracted features from image input; a
positional encoding to encode feature maps with positional information;
transformer encoder and decoder for object detection.

####  III-B1 Backbone

DETR has no special requirements for backbone, any neural network that can do
image classification can be used as its backbone. However, the efficiency of
the network and the prediction accuracy needs to be considered before
deploying DETR to a task. Due to the real-time requirements of the automatic
driving system and the limitation of available computing power, the backbone
can only choose a medium-sized classification network. After some testing,
ResNet50 [6] was finally selected as the backbone of this model. The backbone
takes a list of images $\\{x_{1}$, $x_{2}$,…, $x_{n}\\}$ as its input where
$x_{i}\in\mathrm{R}^{3\times H_{0}\times W_{0}}$. The output of the backbone
is a list of feature maps $\\{f_{1}$, $f_{2}$,…, $f_{n}\\}$, where
$f_{i}\in\mathrm{R}^{c\times H\times W}$, $c=2048$, $H=H_{0}/32$,
$W=W_{0}/32$. In this work, the size of the input image is $256\times 256$

####  III-B2 Positional encoding

In order to understand the context of a full picture, the Transformer’s
attention mechanism shuffles the ordering of the input signal. Therefore the
feature maps passed into the Transformer must be positional encoded to output
meaningful results.

####  III-B3 Transformer Encoder

After the backbone, the dimension of the feature map needs to be reduced by a
$1\times 1$ convolutional layer to form a new feature map
$z\in\mathrm{R}^{d\times H\times W}$. In this model $d=256$. Because the
transformer encoder only takes two dimensions, the third-dimension tensor $z$
needs to be flattened to generate the input vector $z\in\mathrm{R}^{d\times
HW}$. The last process for the transformer encoder is to positional encode the
vector $z$.

####  III-B4 Transformer Decoder

The decoder also has the same structure as the original transformer. It uses
the output from encoder as the input keys and values of the multi-head self-
attention layer. However, the difference is that it uses **N** objects as the
queries of the multi-head self-attention layer. Then, the decoder will
transform the object queries into a latent layer that contains the fused
information of that object. The latent layer can be further decoded into
confidence of that object’s label and a bounding box of the object’s position
by a Feed Forward Network (FFN). The structure of the FFN is very simple. It
is a network with three fully connected layers with Rectified Linear Unit
(ReLU) as the activation function between layers. There are two different FFNs
in DETR, one is used to predict the object labels, another one is for
predicting the bounding box.

###  III-C Planning Module

Figure 3:  The architecture of the proposed Detrive model

The planning module consists of a feature fusion network and a Recurrent
Neural Network (RNN) with Gated Recurrent Units (GRU) Cells. The planning
module is also known as the decision module. It takes the output of the
perception network as input, and generates dynamic trajectory waypoints. This
predicted trajectory is updated at a high frequency to allow the vehicle to
cope with dynamic obstacles and traffic signals.

####  III-C1 Feature Fusion

The feature fusion network is used to fuse the perception outputs with other
sensor data. It provides a scene understanding ability for the agent by
combining perception modules with other sensor information. The structure of
the fusion network is shown in Figure 4. The outputs of the perception module
are object labels and object bounding box. As shown in the figure, the shape
of the outputs are respectively $(b,\,N,\,1)$ and $(b,\,N,\,4)$, where $b$ is
the batch size, $N$ is the number of objects. In order to reduce the feature
loss caused by the deepening of the network, a residual structure is used. It
will pass the feature map from the backbone to a MLP then concatenate with
object label and bounding box tensors. The last two dimensions of the
concatenation tensor will be flattened into one dimension. The the shape of
the new tensor is $(b,\,(5\times N))$. Afterwards, the new tensor will be fed
into a single-layer MLP that can fuse the data and transform the shape into
$(b,d)$.

Figure 4: The architecture of the feature fusion network

Navigational command and speed are the other two inputs of the model in
addition to the RGB camera. There are six kinds of navigation commands in
GNSS, they are to move along the current lane, change to the left lane, change
to the right lane, turn left at the intersection, turn right at the
intersection, and go straight at the intersection. These six commands will be
one-hot encoded into numbers from 0 to 5. Then, the navigational command and
speed will be transformed into tensors and fed into a similar MLP to generate
vector with the shape of $(b,d)$. After that, the two tensors will be added
element-wise to form a new feature tensor. Finally, another three-layer MLP
will further fuse the information and reduce tensor dimension to $(b,c)$.

####  III-C2 Recurrent Neural Network

The recurrent neural network (RNN) is the most important part of the planning
module. Compared with CNN, RNN is better at dealing with sequence problems.
The trajectory prediction problem is similar to the one-to-many or many-to-
many model in the sequence problem. In RNN, $F$ is the parametric network, $h$
and $x$ are the inputs of the network. These three elements form the basic
cell of an RNN. It can be formalized as Equation 1, where $W$ is the learnable
parameters matrix, $h$ is the hidden output, and $x$ is the input. More
specifically, $h_{0}$ is the current position of the ego vehicle, $x$ is the
tensor connected by the output of the feature fusion network and the target
point of the ego vehicle.

$h_{t}=tanh(W_{hh}h_{t-1}+W_{hx}x_{h}+B_{h})$ (1)

The output of the RNN is a sequence of Way points, denoted as $Wp_{t}$. They
are calculated using Equation 2.

$Wp_{t}=W_{hy}h_{t}+B_{y}$ (2)

The RNN cell used in the model is the Gated Recurrent Units (GRU). By
repeatedly stacking these GRU cells, an RNN network can be obtained. In this
work, the output $h_{t}$ of each cell is dynamic. The order of the generated
way points is from the target point to the current position of the ego
vehicle, as shown in Figure 5.

###  III-D Control Module

The aim of the control module is to generate actions from the trajectory. The
control module will output the steering wheel angle, as well as the use of the
throttle. The process is formalized in Equation 3 to Equation 5.

$P=\frac{1}{K}\sum W_{k}$ (3)

$v=\frac{1}{K}\sum\frac{L_{2}(w_{k}-w_{k-1})}{\Delta t}$ (4)

$\delta=tan^{-1}(\frac{p_{y}}{p_{x}})$ (5)

$P$ is the midpoint of those path points; $v$ is the desired speed; $\Delta t$
is the Time interval ; $\delta$ is the angle between the head of the vehicle
ego and the point $P$. Then, $v$ will be mapped to the throttle, and the
$\delta$ will be mapped to the steer. The steering wheel is in [-1, 1], where
-1 represents the maximum angle of left turn and 1 represents the maximum
angle of right turn. The throttle is also in [-1, 1], where -1 is the maximum
value

Figure 5: The GRU cells network for planner module

for braking and 1 is the maximum value for acceleration. The process is
formalized in Equation 6 and Equation 7:

$\mathrm{steer}=\mathrm{Lateral\\_PID}(\delta)$ (6)

$\mathrm{throttle}=\mathrm{Longitudinal\\_PID}(v)$ (7)

###  III-E Reinforcement Learning Expert

The training data is collected by an informed reinforcement leaning expert
called Roach [24]. This paper uses the pretrained Roach agent as the
demonstrator for the imitation learning agent. The expert is informed because
takes the real-time ground truth BEV map from the CARLA simulator as its
input, and generates the control actions through a Proximal Policy
Optimization (PPO) [9] neural network.

The dataset is generated on all routes and scenarios provided by the CARLA
leaderboard training suite. The training kit has five maps named, Town01,
Town03, Town04, Town06, Town08 and a hundred routes in total within those
maps. The data collection process is to let experts go through all routes and
scenarios while sampling key information. The frequency of data sampling is 2
frame per second. Considering the generality of the dataset, more information
is collected than is required for this model.

##  IV Experiments

###  IV-A Implementation

The model is trained on a GeForce RTX 3090 ti GPU with 24 GB of video memory.
The simulator is the released version of CARLA 0.9.10.1. The hyperparameters
used in the final model are:

  * •

The number of objects to detect in an image is 100.

  * •

The number of GRU cells are 4.

  * •

The input image resolution is 256 by 256.

  * •

The learning rate for training the network is $1e^{-4}$.

  * •

The number of epochs for training the network is 501.

  * •

The batch size for traning the network is 128.

In addition to the network itself needs to be tuned, the controller parameters
of the vehicle are also critical to the final performance of the model. The
parameters of the vehicle controller include the following:

  * •

The lateral PID controller: $K_{P}=1.25$, $K_{I}=0.75$, $K_{D}=0.3$, $n=30$

  * •

The longitudinal pid controller: $K_{P}=5.0$, $K_{I}=0.5$, $K_{D}=1.0$,
$n=40$.

  * •

The max throttle is 0.75.

  * •

The maximum value of acceleration is 0.2 $m/s^{2}$.

###  IV-B Training Methods

The data used in the training process of this work are the speed of the
vehicle, the image of the front camera, goal point, and way points. The
training program uses stochastic gradient descent as the training method and
Adam as its optimizer. The loss function is
$L_{1}=\sum_{k}(|\hat{W}_{px}-W_{Px}|+|\hat{W}_{py}-W_{Py}|)$.

During the training process of imitation learning, the imitation learning
agent will first use pre-collected data for offline learning. Offline learning
does not require the agent to interact with the environment in the simulator.
It updates the model parameters through labels of dataset. After 500 episodes
of offline training, the model will fit a policy similar to the expert.
However, the generality of this policy is not good enough. The ego vehicle
will perform badly, if it encounters scenes it has not seen in the training
set. Therefore, the data aggregation method is applied to solve this problem.
After the offline training of N sets is completed, the expert roach will
intervene in the training to provide the imitation learning agent with on
policy annotations. During this process, the imitation learning agent will run
its policy in the Carla simulator, and at the same time, Roach will get the
ground truth information of the current state from the simulator, and make
decisions based on it. The information produced by the imitation learning
agent and the Roach expert in this process will be recorded as a new dataset.
Afterwards, the data of these two datasets will be mixed half and half to form
a new dataset. A new round of offline learning will be based on this new
dataset to generate a new driving policy. After repeating the above steps 12
times, a robust driving policy is created.

###  IV-C Evaluation

The Detrive model is evaluated on the Carla leaderboard’s NoCarsh-dense
benchmark. Carla Leaderboard is the most commonly used autonomous driving
evaluation system in academia. The competition has been running since 2019 and
each year collects entries in cooperation with some international academic
conferences. During these years, the benchmark of the competition was updated
once in 2020. The benchmark currently used for the competition is the Carla
Grand Challenge 2020. The aim of the challenge is to design an autonomous
agent to drive the vehicle through a sequence of predefined routes, where each
route consists of only a few key waypoints. The agent is required to follow
the traffic rules and drive through each key point without collision. The
routes in the challenge include cities, villages, and highways in both the
United States style and European style. In addition, the challenge also has 14
different weather simulations, such as daylight, sunset, rain, fog, and night.
There are 26 routes, and 3 maps in total. The evaluation maps are Town02,
Town04, Town06.

The Challenge has designed many common traffic scenarios in its tracks, such
as: keep the current lane on congested roads; lane merging on the highway and
urban streets; change to the turn lane ahead of the intersection; negotiations
at traffic intersections and roundabouts, handling traffic lights and traffic
signs, avoid collision with pedestrians and cyclists who suddenly cross the
road. It subtracts points for the ego vehicle based on how and how many times
it violates traffic rules.

##  V Results

When evaluated by the CARLA leaderboard, the Detrive model shows higher total
score than other imitation learning based models. As shown in Table I, the
Detrive model effectively reduced the collisions. However, due to the lack of
labeled traffic light, and stop sign data, the model finds it hard to tell if
the traffic light is red or green, and if the traffic sign is stop or speed
limitation. Therefore, the scores on those metrics are relatively low.

TABLE I: CARLA LEADERBOARD COMPARISON RESULTS Model | OURS | WOR [19] | LBC [16] | CILRS [14]
---|---|---|---|---
Driving score | 34.49 | 31.37 | 8.94 | 5.37
Route completion | 67.37 | 57.65 | 17.54 | 14.4
Infraction penalty | 0.59 | 0.56 | 0.73 | 0.55
Collisions pedestrians | 0 | 0.61 | 0 | 2.69
Collisions vehicles | 0.32 | 1.35 | 0.4 | 1.48
Collisions layout | 0.49 | 1.02 | 1.16 | 2.35
Red light infractions | 0.78 | 0.79 | 0.71 | 1.62
Stop sign infractions | 0.15 | 0 | 0 | 0
Off-road infractions | 0.45 | 0.96 | 1.52 | 4.55
Route deviations | 0 | 1.69 | 0.03 | 4.14
Route timeouts | 0.03 | 0 | 0 | 0
Agent blocked | 1.32 | 0.47 | 4.69 | 4.28

##  VI Conclusion

This paper has proposed a novel end-to-end autonomous driving model that uses
a DETR-like structure for perceiving the surrounding driving environment. The
overall training method of this work is imitation learning. It contains a
informed reinforcement learning expert that can read ground-truth information
directly from the simulator, and a student network that can mimic the action
of the expert. The focus of this work is to improve the perception module of
the imitation learning agent. Instead of the commonly used CNN classification
network, a DETR-like detection model is used for perception. Another
innovation of this paper is the structure of the feature fusion network. The
fusion network, composed of the multi-layer MLP structure and the vector
concatenation network can effectively fuse the data of different modalities
and reduce the dimensionality. The last part of the Detrive network is a GRU-
based RNN, which is used to generate waypoints. Then those waypoints can be
mapped into actions by the vehicle control algorithm.

This paper verifies that the use of a transformer-based object detection model
end-to-end autonomous driving model can significantly improve the driving
performance compared with those classification-based perception models.
However, the perception model is only trained with a universal vision dataset,
rather than with a dedicated driving scene dataset. This leads to the fact
that some traffic objects are not labeled in the dataset, and are therefore
hard to be detected by the perception network.

As a future direction to extend our work and to thoroughly evaluate the
efficacy of our proposed end-to-end autonomous driving model, we plan to train
the proposed model with a driving scene dataset. In addition, we will consider
the use of BEV view to help let the vehicle understand the spatial
relationship, and therefore resulting in better path planning.

## References

  * [1] Dean A Pomerleau  “Alvinn: An autonomous land vehicle in a neural network”  In _Advances in neural information processing systems_ 1, 1988
  * [2] darpa  “darpa grand challenge”, 2007  URL: http://archive.darpa.mil/grandchallenge/
  * [3] Brenna D Argall, Sonia Chernova, Manuela Veloso and Brett Browning  “A survey of robot learning from demonstration”  In _Robotics and autonomous systems_ 57.5 Elsevier, 2009, pp. 469–483
  * [4] Chenyi Chen, Ari Seff, Alain Kornhauser and Jianxiong Xiao  “Deepdriving: Learning affordance for direct perception in autonomous driving”  In _Proceedings of the IEEE international conference on computer vision_ , 2015, pp. 2722–2730
  * [5] Shaoqing Ren, Kaiming He, Ross Girshick and Jian Sun  “Faster r-cnn: Towards real-time object detection with region proposal networks”  In _Advances in neural information processing systems_ 28, 2015
  * [6] Kaiming He, Xiangyu Zhang, Shaoqing Ren and Jian Sun  “Deep residual learning for image recognition”  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2016, pp. 770–778
  * [7] Joseph Redmon, Santosh Divvala, Ross Girshick and Ali Farhadi  “You only look once: Unified, real-time object detection”  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2016, pp. 779–788
  * [8] Alexey Dosovitskiy et al.  “CARLA: An open urban driving simulator”  In _Conference on robot learning_ , 2017, pp. 1–16  PMLR
  * [9] John Schulman et al.  “Proximal policy optimization algorithms”  In _arXiv preprint arXiv:1707.06347_ , 2017
  * [10] Ashish Vaswani et al.  “Attention is all you need”  In _Advances in neural information processing systems_ 30, 2017
  * [11] Felipe Codevilla et al.  “End-to-end driving via conditional imitation learning”  In _2018 IEEE international conference on robotics and automation (ICRA)_ , 2018, pp. 4693–4700  IEEE
  * [12] Axel Sauer, Nikolay Savinov and Andreas Geiger  “Conditional affordance learning for driving in urban environments”  In _Conference on Robot Learning_ , 2018, pp. 237–252  PMLR
  * [13] Richard S Sutton and Andrew G Barto  “Reinforcement learning: An introduction”  MIT press, 2018
  * [14] Felipe Codevilla, Eder Santana, Antonio M López and Adrien Gaidon  “Exploring the limitations of behavior cloning for autonomous driving”  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , 2019, pp. 9329–9338
  * [15] Nicolas Carion et al.  “End-to-end object detection with transformers”  In _European conference on computer vision_ , 2020, pp. 213–229  Springer
  * [16] Dian Chen, Brady Zhou, Vladlen Koltun and Philipp Krähenbühl  “Learning by cheating”  In _Conference on Robot Learning_ , 2020, pp. 66–75  PMLR
  * [17] Alexey Dosovitskiy et al.  “An image is worth 16x16 words: Transformers for image recognition at scale”  In _arXiv preprint arXiv:2010.11929_ , 2020
  * [18] Ardi Tampuu et al.  “A survey of end-to-end driving: Architectures and training methods”  In _IEEE Transactions on Neural Networks and Learning Systems_ 33.4 IEEE, 2020, pp. 1364–1384
  * [19] Dian Chen, Vladlen Koltun and Philipp Krähenbühl  “Learning to drive from a world on rails”  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , 2021, pp. 15590–15599
  * [20] Jeffrey Hawke, Vijay Badrinarayanan and Alex Kendall  “Reimagining an autonomous vehicle”  In _arXiv preprint arXiv:2108.05805_ , 2021
  * [21] Anthony Hu et al.  “Fiery: Future instance prediction in bird’s-eye view from surround monocular cameras”  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , 2021, pp. 15273–15282
  * [22] Ze Liu et al.  “Swin transformer: Hierarchical vision transformer using shifted windows”  In _Proceedings of the IEEE/CVF international conference on computer vision_ , 2021, pp. 10012–10022
  * [23] tesla  “Tesla ai day”, 2021  URL: https://www.tesla.com/AI
  * [24] Zhejun Zhang et al.  “End-to-end urban driving by imitating a reinforcement learning coach”  In _Proceedings of the IEEE/CVF international conference on computer vision_ , 2021, pp. 15222–15232
  * [25] Zhiqi Li et al.  “Bevformer: Learning bird’s-eye-view representation from multi-camera images via spatiotemporal transformers”  In _European conference on computer vision_ , 2022, pp. 1–18  Springer
  * [26] Penghao Wu et al.  “Trajectory-guided control prediction for end-to-end autonomous driving: A simple yet strong baseline”  In _Advances in Neural Information Processing Systems_ 35, 2022, pp. 6119–6132
  * [27] Hao Zhang et al.  “Dino: Detr with improved denoising anchor boxes for end-to-end object detection”  In _arXiv preprint arXiv:2203.03605_ , 2022
  * [28] carla  “CARLA leaderboard”, 2023  URL: https://leaderboard.carla.org/
  * [29] Kashyap Chitta et al.  “Transfuser: Imitation with transformer-based sensor fusion for autonomous driving”  In _IEEE Transactions on Pattern Analysis & Machine Intelligence_ 45.11 IEEE Computer Society, 2023, pp. 12878–12895
  * [30] Zhijian Liu et al.  “Bevfusion: Multi-task multi-sensor fusion with unified bird’s-eye view representation”  In _2023 IEEE International Conference on Robotics and Automation (ICRA)_ , 2023, pp. 2774–2781  IEEE