# Enhancement of Pitch Controllability
using Timbre-Preserving Pitch Augmentation in FastPitch

###### Abstract

The recently developed pitch-controllable text-to-speech (TTS) model, i.e.
FastPitch, was conditioned for the pitch contours. However, the quality of the
synthesized speech degraded considerably for pitch values that deviated
significantly from the average pitch; i.e. the ability to control pitch was
limited. To address this issue, we propose two algorithms to improve the
robustness of FastPitch. First, we propose a novel timbre-preserving pitch-
shifting algorithm for natural pitch augmentation. Pitch-shifted speech
samples sound more natural when using the proposed algorithm because the
speaker’s vocal timbre is maintained. Moreover, we propose a training
algorithm that defines FastPitch using pitch-augmented speech datasets with
different pitch ranges for the same sentence. The experimental results
demonstrate that the proposed algorithms improve the pitch controllability of
FastPitch.

Index Terms: timbre-preserving pitch-shifting algorithm, pitch augmentation,
text-to-speech, FastPitch, VocGAN

##  1 Introduction

Dynamic pitch contour is a crucial factor in synthesizing natural speech.
Therefore, recently, studies have been actively conducted on pitch-
controllable text-to-speech (TTS) models have been actively researched [1, 2,
3]. Among them, FastPitch [2], that controls the phoneme-level pitch and
duration of synthesized speech by conditioning the pitch and duration values,
has recently become popular.

However, in our pilot experiments, we observed that larger pitch adjustment
values in FastPitch significantly degraded the quality of the synthesized
speech. For example, the pronunciation clarity and intelligibility of the
synthesized speech were reduced when the input pitch values deviated
considerably from the average pitch range; further details on these results
are discussed in Section 3. This problem is primarily caused by the unbalanced
ratio of the speech database. Generally, speech samples in a database are
mainly distributed around an average pitch value, and the number of speech
samples with pitch ranges that are significantly different from the average
value are relatively small. This distribution prevents FastPitch from learning
sufficiently over a wide pitch range; the model’s ability to control pitch is
thus limited.

A simple approach to address this issue is to record a sufficient number of
speech samples for various pitch ranges. However, building such a database is
not favorable under realistic conditions, owing to the cost and time required.
Data augmentation can be an alternative approach. In [4, 5, 6, 7], the speech
data obtained from external sources such as audiobooks and crowd-sourced data
was utilized to train TTS models. These studies primarily focused on
increasing the quantity of speech database but enhancing the pitch
controllability of FastPitch is necessary to augment speech data over a wide
range of pitches. The pitch augmentation can be achieved by shifting the pitch
of speech data. The key point here is that the pitch-shifted speech should be
perceptually similar to the speaker’s vocal timbre, which is known as vocal
color. In this paper, these pitch-shifting (PS) algorithm is called a timbre-
preserving PS algorithm.

Timbre-preserving PS algorithms are classified into two main types: parametric
and non-parametric methods. First, parametric methods are based on parametric
vocoders, such as WORLD [8] vocoder. These method extract vocoder parameters,
also called acoustic features, from the speech signal, and then, re-synthesize
the pitch-shifted speech with parameters including modified pitch values.
Second, non-parametric methods are performed directly on time-domain signals.
The time-domain pitch-synchronous overlap and add (TD-PSOLA) technique [9] is
utilized. The TD-PSOLA technique divides pitch-synchronous windowed speech
segments at pitch marks, which are time stamps placed at the peaks of the
periodic signals, and then, combines the speech segments at the pitch marks
adjusted to the desired pitch interval.

These methods successfully control the pitch of the speech signals, while
preserving the vocal timbre. Despite this successful pitch-control ability,
the quality of the resultant pitch-shifted speech generated by these methods
depends on the vocoder models or on the degree of pitch adjustment. In the
parametric method, regardless of the PS, a well-designed vocoder model is
required to obtain synthesized speech with high audible quality. In addition,
even with a well-designed vocoder model, the quality of generated speech is
affected by factors including inaccurate vocoded parameter values estimated
using pitch or spectral envelope estimation algorithms. In the TD-PSOLA
method, a slight pitch adjustment maintains the speaker’s timbre; however, an
excessive pitch adjustment affects the timbre of the pitch-shifted speech.

To address these issues, we propose (1) a novel timbre-preserving PS algorithm
that does not require extra algorithms, such as pitch tracking or spectral
envelope estimation, and (2) a FastPitch training algorithm that uses the
pitch-augmented speech datasets. Based on our proposed methods, FastPitch can
generate a high-quality synthesized speech even for larger pitch adjustment
values. In other words, it enhances the pitch controllability of FastPitch.
More detail processes are as follows. First, the pitch-augmented speech
datasets are obtained using the proposed PS algorithm, which utilizes the
characteristics of the VocGAN [10], a neural vocoder. This algorithm raises or
lowers the pitch of the speech samples, while maintaining the speaker’s vocal
timbre. Since the proposed algorithm is not based on a parametric vocoder, it
has the advantage of being less affected by inaccurate by inaccurately
extracted vocoder features. Also, the pitch adjustment range is relatively
less restricted compared to the TD-PSOA method. After augmenting the speech
data for a wide range of pitches, FastPitch is trained using a training
algorithm that was proposed specifically for pitch-augmented datasets.
FastPitch is updated using the original and pitch-augmented datasets
alternately for every epoch. Because the pitch-augmented datasets comprise
speech samples with different pitches for the same sentence, the pitch and
duration predictors are fixed while updating the FastPitch with them. Please
note that the duration and pitch predictors cannot predict different values
for the same text.

##  2 Proposed Method

###  2.1 VocGAN-based PS algorithm

Figure 1: Mel-spectrograms, where (a) represents recorded speech, (b) and (c)
represent the output waveforms from the source and filter gates, respectively,
and (d) represents their power spectra.

VocGAN [10] is a neural vocoder comprising a multi-scale waveform generator
and discriminator. In the model training step, five waveforms that are
generated with low to high resolutions are used for training. In the inference
step, only the waveform with the highest resolution is used. In our
preliminary experiments, we referred the two low-resolution gates as the
source gate; and the three high-resolution gates as the (vocal tract) filter
gate. We applied the same mel-spectrogram to the source and filter gates of
the pre-trained VocGAN model and perceptually observed that each waveform had
different characteristics. The output signal from the source gate had a pitch
that was perceptually similar to the input speech; however, it contained less
linguistic information. In contrast, the output signal from the filter gate
contained more linguistic information from the input speech without a pitch-
related information. Figure 1 depicts the mel-spectrograms of the input speech
and the output signals of the source and filter gates.

Based on these characteristics, we propose the VocGAN-based PS algorithm
(VocGAN-PS), which shifts the pitch of the input speech according to the pitch
adjustment value $\alpha$, while maintaining the speaker’s timbre by
preserving the spectral envelopes; the $\alpha$ is represented in semitone
(ST) units. Figure 2 depicts the schematic of the VocGAN-PS. For the filter
gate, the original mel-spectrogram is applied to preserve the spectral
envelopes, whereas for the source gate, the pitch-shifted mel-spectrogram is
applied to obtain the target pitch information. The pitch-shifted mel-
spectrogram is obtained by the sampling rate conversion [11] in the frequency-
domain. Finally, the output speech of VocGAN is obtained with the correct
timbre of the speaker.

Figure 2: Schematic of the VocGAN-PS and Spectral envelopes of the three types
of speech waveforms.

###  2.2 Availability of pitch-augmented data for training

Figure 3: Schematic diagram of proposed method.

Although the VocGAN-PS preserves the vocal timbre of the input speech well,
the training data of the TTS model should have high pronunciation accuracy and
should be perceptually similar to the target speaker’s speech. Therefore, we
evaluate the speech quality and speaker similarity for the pitch-shifted
speech samples and use only those samples that meet the constraints as the
augmented dataset, as shown in Figure 3a.

###  2.3 Data preparation

Figure 3b depicts the process of the data preparation. First, we extracted the
character sequence $\mathbf{x}$ and mel-spectrogram $\mathbf{y}$. The mel-
spectrogram using an 80-bins mel filter bank, which was computed by the FFT
with size of $1024$, hop size of $256$, and window size of $1024$ in the
frequency range $0-8000$ Hz. We then extracted the character-level durations
$\mathbf{d}$ and pitches $\mathbf{p}$ using the attention matrix of the pre-
trained deep-convolutional TTS model [12], which can easily train an attention
matrix by guided-attention loss.

###  2.4 Training FastPitch using pitch-augmented dataset

Figure 3c depicts the proposed training algorithm. The FastPitch
$\mathcal{M}_{\mathrm{FastPitch}}$ is updated using the original and augmented
datasets, namely $D_{\textrm{orig}}$ and $D_{\textrm{aug}}$, alternately for 1
epoch each.

For $D_{\textrm{orig}}$, the text encoder first outputs a hidden
representation $\mathbf{h}$ corresponding to the character sequence
$\mathbf{x}$ of the input text. The pitch predictor and duration predictor
then predict the duration sequence $\hat{\mathbf{d}}$ and pitch sequence
$\hat{\mathbf{p}}$, respectively, corresponding to the input $\mathbf{h}$. For
training, the teacher forcing technique [13] was used; this means that the
ground-truths $\mathbf{d}$ and $\mathbf{p}$ were used for upsampling and as
the inputs to the mel decoder. Then, $\mathbf{h}_{\mathrm{up}}$ and
$\mathbf{p}_{\mathrm{up}}$, which are upsampled using a length regulator, pass
through the mel decoder that predicts the output mel-spectrogram
$\hat{\mathbf{y}}$.

For $D_{\textrm{aug}}$, $\mathcal{M}_{\mathrm{FastPitch}}$ generates
$\mathbf{\hat{y}}$ by applying the pitch-shifted pitch $\mathbf{p}+\alpha$. In
this step, the losses are calculated for the pitch-shifted mel-spectrograms
only. Note that the duration and pitch predictors are not used in this step
because it is impossible for the pitch predictor to predict
$\mathbf{p}+\alpha$ instead of $\mathbf{p}$ for the same $\mathbf{h}$.

##  3 Experiments

Figure 4: Objective evaluation results for the PS algorithms, where (a), (b),
(c) and (d) represent the SoX-, TD-PSOLA-, WORLD- and VocGAN-PSs,
respectively.

In this study, we used $14000$ speech samples of a single Korean female
speaker (target speaker), which were sampled at $22.05$ kHz, amounting to
approximately $20$ h. The dataset was divided into the training, validation,
and test sets with $13400$, $100$, and $500$ samples, respectively. The
distribution of mean-referred pitch values of target speaker is
$\mathbf{p}\sim\mathcal{N}(0,4.11^{2})$, which is represented in ST units by
the equation
$\mathbf{p}=12\cdot\mathrm{log}_{2}(\mathbf{f}_{0}/\bar{\mathbf{f}}_{0})$,
with the average fundamental frequency $\bar{\mathbf{f}}_{0}$ = $248$ Hz.

###  3.1 Pitch augmentation using the VocGAN-PS

Before generating the augmented dataset for the training set, we generated the
pitch-shifted speech samples using $500$ samples of the test set for
$|\alpha|\leq 4$ (less than $1$ standard deviation). We then conducted the
objective and subjective tests to evaluate the speech quality and speaker
similarity of the pitch-shifted speech samples as mentioned in Section 2.2.

The SoX- (vocal timbre is not preserved), TD-PSOLA- and WORLD-PS algorithms
were also evaluated to compare with the VocGAN-PS. We used the open sources
[14, 15, 16] and its default settings except for the VocGAN-PS. In case of the
VocGAN, we implemented it and trained using the same training set.

Table 1: MOS ($\uparrow$) results for speech quality of the three timbre-preserving PS algorithms with 95% confidence intervals (CIs). | $\mathbf{-4}$ | $\mathbf{-3}$ | $\mathbf{-2}$ | $\mathbf{+2}$ | $\mathbf{+3}$ | $\mathbf{+4}$
---|---|---|---|---|---|---
TD-PSOLA $\pm$ CI | $1.49$ | $1.69$ | $1.93$ | $1.99$ | $1.67$ | $1.73$
$0.13$ | $0.12$ | $0.14$ | $0.16$ | $0.13$ | $0.14$
WORLD $\pm$ CI | $2.64$ | $3.16$ | $2.95$ | $3.05$ | $3.11$ | $2.81$
$0.17$ | $0.18$ | $0.16$ | $0.17$ | $0.18$ | $0.17$
VocGAN $\pm$ CI | $\mathbf{3.61}$ | $\mathbf{4.03}$ | $\mathbf{4.35}$ | $\mathbf{4.35}$ | $\mathbf{4.11}$ | $\mathbf{3.85}$
$0.15$ | $0.13$ | $0.14$ | $0.13$ | $0.15$ | $0.15$

Figure 5: Spectrogram of the input speech and the pitch-shifted samples of
each algorithm, where $\alpha=3$.

Figure 6: Similarity results of the VocGAN-PS. Red dots represent their mean
value.

Figure 7: SER results for pronunciation accuracy.

To evaluate the speech quality, we first conducted the objective test
calculating the mel-cepstral distortion (MCD) [17], change in the average
pitch ($\Delta\bar{\mathbf{p}}$), and syllable error rate (SER) using the
Kaldi speech recognition system [18]. Second, we conducted the mean opinion
score (MOS) test on the speech quality by randomly selecting five samples from
the test set for the three timbre-preserving PS algorithms of $\alpha\in\\{\pm
2,\pm 3,\pm 4\\}$ for a total of $30$ samples for each algorithm. A total of
15 native Koreans participated in this effort and scored the samples on a
scale of $1$ (Bad) to $5$ (Excellent) referred by [19].

These results are summarized in Figure 4 and Table 1. Comparing the timbre-
preserving algorithms for all $\alpha$, the speech quality of the VocGAN-PS
outperform the other PS algorithms overall. Figure 5 depicts the spectrogram
of the input speech and pitch-shifted samples of each algorithm. Comparing the
areas in the red box, where the pitch values were lower than about $200$ Hz,
the TD-PSOLA-PS generated the low speech quality samples caused by the PSOLA
technique. Comparing the areas in the blue box, where is the unvoiced/voiced
transition, the WORLD-PS occurred the distortion caused by the inaccurately
estimated pitch values.

To evaluate the speaker similarity, we conducted the listening test to
evaluate the speaker similarity between the input recorded speech and pitch-
shifted speech. We selected five samples for every value of $\alpha\in\\{\pm
2,\pm 3,\pm 4\\}$ randomly for a total of $30$ samples. A total of 15 native
Koreans participated in this effort and scored the samples on a four-point
scale: 1 (Different, absolutely sure), 2 (Different, not sure), 3 (Same, not
sure) and 4 (Same, absolutely sure) referred by [20]; these results are
summarized in Figure 6. For $|\alpha|\leq 3$, about $70\%$ or more of the
pitch-shifted samples were judged by participants to be the same as the target
speaker.

Hence, in subsequent experiments, we used the pitch-shifted speech samples for
$|\alpha|\leq 3$ as the augmented dataset.

###  3.2 Performance evaluation of the FastPitch

Table 2: MOS results for speech quality of the two FastPitch models with 95% CIs. $\alpha$ | $\mathbf{-4}$ | $\mathbf{-2}$ | $\mathbf{0}$ | $\mathbf{+2}$ | $\mathbf{+4}$
---|---|---|---|---|---
$\mathcal{M}_{\mathrm{Base}}$ $\pm$ CI | $2.88$ | $3.97$ | $4.46$ | $4.05$ | $3.18$
$0.18$ | $0.15$ | $0.16$ | $0.16$ | $0.18$
$\mathcal{M}_{\mathrm{Aug}}$ $\pm$ CI | $\mathbf{3.63}$ | $\mathbf{4.27}$ | $\mathbf{4.58}$ | $\mathbf{4.15}$ | $\mathbf{3.55}$
$0.16$ | $0.15$ | $0.14$ | $0.14$ | $0.17$

We compared $\mathcal{M}_{\mathrm{Base}}$ and $\mathcal{M}_{\mathrm{Aug}}$
trained by the basic FastPitch algorithm and proposed training algorithm,
respectively. Both models were trained up to $300$ epochs each with a mini-
batch size of $16$ and the Adam optimizer [21] with initial learning rate of
$0.0002$. The hyper-parameters of Adam optimizer were
$({\beta}_{1},{\beta}_{2})=(0.5,0.9)$, and ${\epsilon}=10^{-6}$.

To evaluate the pitch controllability of FastPitch, we explored the maximum
pitch adjustment value $\alpha^{*}$ for each model. This means that the model
generates speech for $\hat{\mathbf{p}}\pm\alpha^{*}$ without performance
deterioration of pronunciation clarity and speech quality. We set a range in
which (1) the SER is lower than $7\%$ and (2) the speech quality score is
higher than $3$ (Fair) as $\alpha^{*}$. We observed that the pitch-shifted
speech generated by the SoX-PS algorithm had SERs greather than $7$% without
the deterioration of pronunciation clarity; however, its timbre sounded
perceptually inhuman or unnatural similar to the effect of helium on a human
voice. Therefore, we set an SER of $7\%$ as the constraint for evaluating the
pronunciation clarity of the FastPitch models.

To explore $\alpha^{*}$, we generated speech samples for $500$ texts of the
test set by adding $\alpha$ to $\mathbf{\hat{p}}$, as shown in Figure 3d;
$\alpha$ was set from $-5$ to $+5$. Figure 7 depicts the SERs of both models.
For $\mathcal{M}_{\mathrm{Aug}}$, the SER is maintained lower than $7\%$ for
all $\alpha$, whereas for $\mathcal{M}_{\mathrm{Base}}$, the SER increases as
$\alpha$ increases.

We conducted the MOS test on speech quality by randomly selecting eight
samples for each case of $\alpha\in\\{0,\pm 2,\pm 4\\}$ for a total of $40$
samples for each model; this test is the same participant and scale in Section
3.1. Table 2 shows that the results of the MOS test.
$\mathcal{M}_{\mathrm{Aug}}$ model outperforms the
$\mathcal{M}_{\mathrm{Base}}$ model overall; in addition,
$\mathcal{M}_{\mathrm{Aug}}$ received a score of $3.63$ even when $\alpha$ was
equal to $-4$, whereas the quality of $\mathcal{M}_{\mathrm{Base}}$ was
significantly degraded.

In summary, the value of $\alpha^{*}$ for $\mathcal{M}_{\mathrm{Aug}}$ was
determined as $4$ and that for $\mathcal{M}_{\mathrm{Base}}$ was $2$; this
proved that the proposed training algorithms improved the pitch
controllability of FastPitch.

Audio samples can be found online111https://nc-
ai.github.io/speech/publications/vocgan-ps-fastpitch.

##  4 Conclusion

This paper proposed two algorithms to enhance the robustness of pitch
controllability of FastPitch. The proposed VocGAN-PS first augmented the
speech data successfully such that it contained a wide range of pitches while
preserving vocal timbre. Then, a training algorithm was proposed for FastPitch
such that the pitch-augmented data could be used. Thus, FastPitch generated
speech with stable quality even at higher and lower pitch ranges. We verified
the effectiveness of the proposed methods through various quantitative and
qualitative evaluations.

## References

  * [1] Y. Ren _et al._ , “Fastspeech 2: Fast and high-quality end-to-end text-to-speech,” _International Conference on Learning Representations (ICLR)_ , 2021.
  * [2] A. Łańcucki, “FastPitch: Parallel Text-to-Speech with Pitch Prediction,” in _IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , 2021.
  * [3] T. Park, J. S. Bae, H. Bae, Y. I. Kim, and H. Y. Cho, “Fastpitchformant: Source-filter based decomposed modeling for speech synthesis,” in _Annual Conference of the International Speech Communication Association (Interspeech)_ , 2021.
  * [4] W. Hsu _et al._ , “Disentangling correlated speaker and noise for speech synthesis via data augmentation and adversarial factorization,” in _IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , 2019, pp. 5901–5905.
  * [5] C. V. Botinhao, X. Wang, S. Takaki, and J. Yamagishi, “Speech enhancement for a noise-robust text-to-speech synthesis system using deep recurrent neural networks,” in _Annual Conference of the International Speech Communication Association (Interspeech)_ , 2016, pp. 342–356.
  * [6] N. Adiga, Y. P. V. Tsiaras, and Y. Stylianou, “Speech Enhancement for Noise-Robust Speech Synthesis Using Wasserstein GAN,” in _Annual Conference of the International Speech Communication Association (Interspeech)_ , 2019, pp. 1821–1825.
  * [7] H. Bae, J. S. Bae, Y. S. Joo, Y. I. Kim, and H. Y. Cho, “A neural text-to-speech model utilizing broadcast data mixed with background music,” in _IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , 2021.
  * [8] M. Morise, F. Yokomori, and K. Ozawa, “World: a vocoder-based high-quality speech synthesis system for real-time applications,” in _IEICE TRANSACTIONS on Information and Systems_ , vol. 99.7, 2016, pp. 1877–1884.
  * [9] C. Hamon, E. Mouline, and F. Charpentier, “A diphone synthesis system based on time-domain prosodic modifications of speech,” in _IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , vol. 1, 1989, pp. 238–241.
  * [10] J. Yang, J. Lee, Y. I. Kim, H. Y. Cho, and I. Kim, “VocGAN: A High-Fidelity Real-time Vocoder with a Hierarchically-nested Adversarial Network,” in _Annual Conference of the International Speech Communication Association (Interspeech)_ , 2020.
  * [11] J. Laroche, “Time and pitch scale modification of audio signals,” in _Applications of digital signal processing to audio and acoustics_. Springer, 2002.
  * [12] H. Tachibana, K. Uenoyama, and S. Aihara, “Efficiently Trainable Text-to-Speech System Based on Deep Convolutional Networks with Guided Attention,” in _IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , 2018, pp. 4784–4788.
  * [13] R. J. Williams and D. Zipser, “A learning algorithm for continually running fully recurrent neural networks,” _Neural Computation_ , vol. 1, no. 2, pp. 270–280, 1989.
  * [14] “SoX Sound eXchange,” https://sox.sourceforge.net, (Accessed Mar. 1, 2022).
  * [15] S. Wager, “open source of TD-PSOLA,” https://github.com/sannawag/TD-PSOLA, (Accessed Mar. 1, 2022).
  * [16] M. Morise, “open source of WORLD,” https://github.com/mmorise/World, (Accessed Mar. 1, 2022).
  * [17] R. Kubichek, “Mel-cepstral distance measure for objective speech quality assessment,” in _Proceedings of IEEE Pacific Rim Conference on Communications Computers and Signal Processing_ , vol. 1. IEEE, 1993, pp. 125–128.
  * [18] D. Povey _et al._ , “The kaldi speech recognition toolkit,” in _IEEE 2011 workshop on automatic speech recognition and understanding_. IEEE Signal Processing Society, 2011.
  * [19] F. Ribeiro, D. Florêncio, C. Zhang, and M. Seltzer, “Crowdmos: An approach for crowdsourcing mean opinion score studies,” in _IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , 2011, pp. 2416–2419.
  * [20] Y. Zhao, W. Huang, X. Tian, J. Yamagishi, R. K. Das, T. Toda, T. Kinnunen, and Z. Ling, “Voice conversion challenge 2020 — intra-lingual semiparallel and cross-lingual voice conversion,” in _ISCA Joint Workshop for the Blizzard Challenge and Voice Conversion Challenge 2020_ , 2020.
  * [21] D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” in _International Conference on Learning Representations (ICLR)_ , 2015.


# Detecting Offensive Language on Social Networks: An End-to-end Detection
Method based on Graph Attention Networks

Zhenxiong Miao  Xingshu Chen  Haizhou Wang<EMAIL_ADDRESS>Rui Tang  Zhou
Yang  Wenyi Tang  School of Cyber Science and Engineering, Sichuan University,
Chengdu, 610207, China  Cyber Science Research Institute, Sichuan University,
Chengdu, 610207, China

###### Abstract

The pervasiveness of offensive language on the social network has caused
adverse effects on society, such as abusive behavior online. It is urgent to
detect offensive language and curb its spread. Existing research shows that
methods with community structure features effectively improve the performance
of offensive language detection. However, the existing models deal with
community structure independently, which seriously affects the effectiveness
of detection models. In this paper, we propose an end-to-end method based on
community structure and text features for offensive language detection (CT-
OLD). Specifically, the community structure features are directly captured by
the graph attention network layer, and the text embeddings are taken from the
last hidden layer of BERT. Attention mechanisms and position encoding are used
to fuse these features. Meanwhile, we add user opinion to the community
structure for representing user features. The user opinion is represented by
user historical behavior information, which outperforms that represented by
text information. Besides the above point, the distribution of users and
tweets is unbalanced in the popular datasets, which limits the generalization
ability of the model. To address this issue, we construct and release a
dataset with reasonable user distribution. Our method outperforms baselines
with the F1 score of 89.94%. The results show that the end-to-end model
effectively learns the potential information of community structure and text,
and user historical behavior information is more suitable for user opinion
representation.

###### keywords:

Social networks, Deep learning, Offensive language detection, Graph attention
networks

###### MSC:

[2010] 00-01, 99-00

††journal: Journal of LaTeX Templates

##  1 Introduction

In recent years, more and more offensive language appears in social networks,
which leads to cyber violence and abusive behavior. The offensive language has
become a serious concern for governments and social media platforms [1]. There
are several reasons why offensive language is proliferating in social
networks. On one hand, users hide their identities in social networks easily.
The anonymity of social networks increases the chances that users adopt
aggressive behavior. On the other hand, social networks provide an environment
where offensive language spread quickly [2].

Usually, social platforms will adopt detection algorithms to find offensive
language automatically and eliminate them. However, it is not easy to
automatically identify these contents, especially the offensiveness is
contained in the hidden rhetoric and irony [3]. Offensive language can be
divided into explicit offensive language and implicit offensive language [4,
5]. Explicit offensive language refers to language that is explicitly abusive
in its meaning, such as racial discrimination. Implicit offensive language
refers to language that does not directly imply abuse, and it is difficult to
judge whether it is offensive without understanding the context [4, 6].

For offensive language detection, existing research [7, 8, 9, 10] have done
some works based on text, confirming that n-gram features have good
performance. Meanwhile, some researchers [11, 12, 13] have combined user
profile features such as age and gender with text features to improve the
performance of detection. Recently, more and more methods based on deep
learning were applied [14, 15, 16, 17]. The BERT-based methods started to
appear in offensive language-related tasks of International Workshop on
Semantic Evaluation (SemEval) [18, 19, 20]. In addition, some works [21, 22]
have further studied the representation method of user features. They
introduced community structure into offensive language detection, which
significantly improved the detection performance.

However, there are limitations in existing methods using community structures.
These methods [21, 22] deal with community structure independently, which
seriously affects the effectiveness of user representations. This limits the
performance of the offensive language detection model. On the other hand, the
existing datasets are mainly collected by searching with the keyword in
tweets, only considering the tweets containing explicit offensive words, such
as [12], [10], [1]. Some abusive tweets that do not contain predefined
keywords cannot be collected. This causes the existing models unable to learn
relevant feature information and reduces the detection performance and
generalization ability of the model in detecting implicit offensive language
[23]. Furthermore, some popular datasets may be affected by “author bias” [24,
25]. This means that the classifier can be affected by the unbalanced
distribution of users, which limits the generalization ability of the model.
For the dataset of Waseem and Hovy [12], more than 70% of the sexist tweets
originate from two users, and 99% of the racist tweets originate from one
user. Existing research shows that detection models based on such datasets
could produce severe user overfitting [25].

To solve these problems, we conduct the work in this paper with the following
contributions.

  * 1.

An end-to-end method based on community structure and text features are
proposed for offensive language detection (CT-OLD). It uses Graph Attention
Networks (GAT) [26] to learn user representations from community structure and
fine-tuned BERT [27] to learn the text representations. The model learns these
features at the same time. In addition, attention mechanisms and position
coding are used for feature fusion. The CT-OLD achieves an F1 score of 89.94%
with our dataset. Moreover, our method has better robustness in different
ratios of training and test set. The experimental results show that the end-
to-end model significantly improves the current detection performance.

  * 2.

A novel method is proposed to represent users with user historical behavior
information. The user historical behavior information indicates the opinion
tendency of users to post offensive language. This method reduces the number
of parameters in the graph neural network and improves the performance of the
model. For users whose historical behavior is unknown, four feature
initialization methods are compared. Our method improves the F1 score by 2.57%
over the user representation method in [22].

  * 3.

A new dataset with community structure and reasonable user distribution is
constructed 111The dataset will be released on https://github.com/mzx4936/CT-
OLD-Dataset. A method that depend on the lexicon indirectly is used to
construct the dataset. Specifically, we first collect the users and
relationships rather than tweets. Then the tweets of the users are collected.
We collect over 400,000 tweets and over 500,000 users. Then we annotate 12,780
tweets posted by 1260 users. The offensive tweets are posted by 220 users,
accounting for 17.5% of the users.

##  2 Related Work

Recently, research based on social networks has been increasing [28, 29, 30,
31]. In this section, we summarize the research on offensive language
detection in social networks in recent years. The methods in offensive
language detection research are divided into two main categories: classical
machine learning-based methods [7, 9, 11, 12, 13, 32, 33, 21, 34, 35], and
deep learning-based methods [14, 18, 19, 20, 22, 23, 27, 36, 37, 38, 39, 40].

###  2.1 Classical machine learning-based methods

Classical machine learning methods have been widely used in offensive language
detection task. It is divided into two categories: methods based on text
features [7, 9, 12, 32, 33, 34], and methods combining text and user features
[11, 13, 21, 35].

####  2.1.1 Text-based feature methods

Most of the text feature-based offensive language detection methods involve
supervised text classification tasks, which generally requires feature
engineering, such as text features, sentiment features, linguistic features.
Text features are calculated by bag-of-words models or word embeddings.
Sentiment features of the sentence are obtained by predefined word sentiments.
Linguistic features refer to the statistical results of uppercase letters,
lowercase letters, and punctuation marks. Using feature engineering methods,
various classifiers can be supervised training.

Razavi et al. [32] constructed a lexicon containing insulting and abusive
language. Assigning a weight to each word in the lexicon to indicate its
impact, which effectively improved the accuracy of detection. Sood et al. [33]
pointed out that sometimes some words appearing in the blacklist would not
cause offensiveness. At the same time, they realized that offensive language
may hide offensive words through accidental or deliberate spelling errors, so
edit distance was used to find potentially offensive words. Finally, a
combination of Support Vector Machine (SVM), lexicon, and edit distance is
used for classification. The integrated method achieved the highest F1 score
compared to the single method. Moreover, since pronouns can be used to avoid
detecting offensive terms, Magu et al. [7] added known offensive word
counterparts to their lexicon.

In addition, some studies used n-gram features for offensive language
detection. Waseem and Hovy [12] compared word n-grams and character n-grams
with Logistic Regression (LR) for classification. The results showed that
character n-grams performed best. Nobata et al. [9] combined multiple features
to detect offensive language. They combine features such as n-gram features
obtained by text, linguistic features obtained by statistics, and distributed
semantic features obtained by word2vec to detect offensive language. Through
experiments on multiple datasets, they found that fusing multiple features
greatly improves detection performance compared to single feature. Salminen et
al. [34] proposed an improved n-gram method. To improve the effectiveness of
n-grams, they tried to weight n-grams using word frequency and TF-IDF. The
classification was performed using LR, Decision Tree, Random Forest (RF),
Adaboost, and SVM. Finally, the n-grams weighted using TF-IDF with SVM
achieved 96% of F1 score in binary classification.

####  2.1.2 Methods of combining text features with user features

Several researchers have combined personal characteristics such as age,
gender, and activity history of the users in detecting offensiveness. Dadvar
et al. [11] combined textual content and user-based features. They considered
the history of user activity, the number of offensive appearances in
previously posted content, and the age of the user. The addition of user
features improved the F1 score by 4%, with the highest contribution from the
number of users’ historical offensive language. Unsvåg and Gambäck [13] used
user profile information (gender, geographic location, etc.), social network-
based user information (number of followers and followees), and activity-based
user information (number of favorites, etc.) as user features in combination
with n-grams of text for classification. Their work showed that user features
could effectively improve the performance of offensive language detection.

Offensive language often comes from users who share common stereotypes [21],
while existing offensive language detection methods completely ignore the
structure of user communities. Mishra et al. [21] proposed a new method of
using community information. They collected user and social relationship
information on the basis of the Waseem and Hovy dataset [12]. The users were
used as nodes and follower-followee relationships were used as edges to form
an undirected social graph. Then the user node embeddings were obtained by
node2vec [35] and text features were obtained by n-grams. Finally, the LR
classifier was trained for classification, which greatly improved the
performance of detection and obtained state of the art at that time. Their
work illustrated the importance of using community-based information for
offensive language detection.

Classical machine learning methods need to analyze the data and complex
features manually, which is probably losing potential features. Therefore,
more and more researchers have started to apply deep learning methods to
offensive language detection.

###  2.2 Deep learning-based methods

Deep learning-based methods automatically represent text or users through deep
learning models, greatly reducing the reliance on feature engineering. It is
divided into the following two types of methods: the method of using text [14,
18, 19, 20, 27, 36, 37, 38], and the method of combining text information with
user information [22, 23, 39, 40].

####  2.2.1 Text-based embedding methods

Recently, deep learning techniques have been applied to offensive language
detection task. Such methods usually rely on word embeddings to obtain text
representation and then train them by sequential or convolutional models.
Badjatiya et al. [14] proposed the methods based on deep learning models and
compared them with machine learning methods. The authors compared two word
vector initialization methods, random embedding and Glove embedding. After
that, the classifiers of CNN, LSTM, and FastText were used to classification.
The highest F1 score of deep learning models reached 83.9%, while the highest
F1 score of the baseline method using machine learning was 81.6%. In addition,
the method combining LSTM and GBDT achieves the highest F1 score of 93.0%. In
this method, the word vectors were trained by a deep neural network after
random initialization of the word vectors, and then input into GBDT as
features for classification. Chen et al. [36] considered that for short texts
in social networks, it was difficult to fully learn the information of each
word with a highly context-dependent method such as word embeddings.
Therefore, they proposed a method based on TF-IDF and CNN, which transformed
the 1-dimensional vector obtained by TF-IDF into 2-dimensional matrix,
allowing CNN to convolve the word features in a larger field. Compared with
randomly initialized word embeddings, the accuracy rate was increased by 9%
and the AUC value was increased by 0.12. Rajamanickam et al. [37] proposed a
multi-task learning detection model for emotion and offensive language. They
used offensive language detection as the primary task and emotion detection as
the auxiliary task. The output of the auxiliary task was added to the primary
task by encoding the text with BiLSTM. And the information flow was controlled
by learnable parameters. When compared on two popular datasets, the method
using multi-task learning improved F1 by 1% over the single-task method.

With the rise of large-scale pre-trained language models, BERT [27] has
occupied an important position in natural language processing. In the
SemEval-2019 Task 6 task, Liu et al. [18] obtained first place in detecting
whether a text is offensive (subtask A) by fine-tuning the BERT model for
transfer learning only. Dai et al. [19] achieved better results by fine-tuning
the BERT model for multi-task learning based on Liu et al. and experimenting
on the same dataset.

Although the BERT model has achieved good performance on numerous NLP tasks,
when it is applied to non-standard language types such as social network
posts, the detection performance may fluctuate greatly due to its chaotic
nature. Thus Caselli et al.[38] proposed the HateBERT model, a retraining BERT
model for detecting offensive language in English. The model was trained on a
large-scale Reddit English comment dataset that included posts from
communities banned for being offensive, abusive, and promoting hate speech,
containing a total of 1,492,740 messages. The authors compared with the
generic BERT model on three English general-purpose datasets, and the F1 score
of HateBERT were all higher than those of generic BERT. Safaya et al. [20]
combined BERT and CNN to obtain better detection results than BERT. The
authors split the tweets obtained from Twitter, transforming Hashtags into raw
text, e.g., _“#SomeHashtagText”_ into _“Some Hashtag Text”_. Subsequently, the
text was split by Wordpiece, a BERT splitter, and the text was fed into a
twelve-layer BERT model. The output context embeddings of the last four hidden
layers of BERT were provided to the CNN to maximize the semantic knowledge in
BERT. Finally, the output of the CNN was used for classification. Experiments
were performed on three language datasets (Arabic, Greek, Turkish) and the F1
score was improved by 1% over BERT alone.

####  2.2.2 Methods of combining text information with user information

The user information provides information that is difficult to obtain at the
textual granularity. Several researchers have attempted to combine user
information with textual information.

Qian et al. [39] proposed a method for offensive language detection using
intra-user and inter-user representations. They collected and analyzed users’
historical tweets to model intra-user tweets. For inter-user tweets, they
could be collected from the dataset by locally sensitive hashing (LSH). Adding
the inter-user representation and intra-user representation improved the F1
score by 7.1% over the baseline using BiLSTM. Mishra et al. [22] used GCN [40]
to learn user embeddings and combined with the bag-of-words representation of
the text for offensive language detection. They constructed a heterogeneous
graph of users and tweets. The user representation was learned through two
layers of GCNs, with the labels of the tweet nodes as the targets of GCN
learning. The output of the first layer of GCNs was used as the user
embedding. After that, LR was applied to detection with the user embeddings
and the bag-of-words representation of the text. This heterogeneous graph
representation was able to model the community structure and the linguistic
behavior of authors in the community. Therefore, their method achieved an F1
score of 85.4%, refreshing the state of the art of offensive language
detection.

Due to the noise of social network texts and the subjectivity of offensive
language, the dataset faced shortcomings in construction. There might be an
oversimplification in offensive language detection. Therefore, Ribeiro et al.
[23] turned to identifying whether users were offensive or not. They collected
over 100,000 users and labeled 4,972 of them. For each user, the authors used
the number of followers and followees, the number of favorites, centrality
measurements (including betweenness, eigenvector centrality, and the in/out
degree of each node), and the tweet features obtained by GloVe. The results of
the three methods of GradBoost, AdaBoost, and GraphSage for classification
were compared. The semi-supervised node embedding algorithm GraphSage was
found to outperform the other methods, with a 15% improvement in F1 in the
task of detecting offensive users.

In existing studies, n-grams are effective for offensive language detection
task. Therefore, classical machine learning methods mostly combine n-grams
with other features. With the development of deep learning, especially the
emergence of BERT, researchers apply word embeddings to text representations
rather than complex feature engineering. During the research on offensive
language detection task, the question of how to use user features has been
explored. Some researchers applied personal information such as gender and age
to improve detection performance. Furthermore, the relationships between users
have been taken into consideration. The structure of communities was composed
into graphs for graph embedding learning. In the past, user features were
obtained after the user model was trained. Then user features are combined
with text features. However, the user features obtained by this method may not
work well for the offensive language detection task. Meanwhile, the “author
bias” in the existing datasets also limits the generalization ability of the
model. In our work, we construct the dataset with a method that relies on
lexicon indirectly. The user historical behavior feature [11] is used to
represent the opinion of users. Finally, the end-to-end model is used to learn
user features and text features for classification.

##  3 Methodology

In this section, we will introduce the data collection and annotation module,
text pre-processing module, social graph construction module, and detection
model module. The framework of this paper is shown in Figure 1.

Figure 1: The architecture of the end-to-end offensive language detection
methods

  1. 1.

Data collection and annotation module: This module is responsible for
collecting users, relationships, and tweets from Twitter to provide data
support for other modules. In this module, a web crawler is developed to
collect data from Twitter. Then the data is annotated according to the
proposed seven annotation criteria.

  2. 2.

Social graph construction module: This module processes the collected user and
social relationship into a social graph. It takes users as nodes, social
relationships as edges, and historical behavioral information as node features
to construct a directed isomorphic social graph.

  3. 3.

Text pre-processing module: This module is used to pre-process the collected
tweets. In this module, the emojis in tweets are replaced by words with the
same meaning. The hashtag is reduced to words or phrases. In addition, URLs,
usernames, etc. are identified uniformly.

  4. 4.

Detection model: We detect offensive language by an end-to-end model that
combines the text representation with community structure. The model is based
on the GAT, the BERT, and the attention mechanism.

###  3.1 Dataset

####  3.1.1 Collection and annotation

##### Data discuss

The widely used datasets, including Waseem and Hovy [12], Davidson et al.
[10], and Zampieri et al. [1], start collecting through the tweets containing
discrimination-related words such as religion, gender, and minority. Such a
way of collecting the data would make the dataset have a much higher
percentage of offensive language. However, it is difficult to collect tweets
with more subtle offensive features. In addition, the datasets such as Waseem
and Hovy [12] have severe “author bias”. More than 70% of the sexist tweets
originate from two users, and 99% of the racist tweets originate from one user
[24]. This bias affects the generalization ability of the model. In this case,
Ribeiro et al. [23] suggest collecting users firstly, and then collecting
their tweets. Table 1 shows the percentage of offensive language in English
language datasets.

Table 1: The statistics of current english datasets Dataset |  | The number
---
of offensive
tweets
| The number
---
of tweets
| The ratio
---
of offensive
tweets
Data type
Waseem and Hovy[12] | 5,355 | 16,914 | 31.66% | tweets
Davidson et al.[10] | - | 24,802 | 76.00% | tweets
Zampieri et al.[1] | 4,640 | 14,100 | 32.91% | tweets
Mandl et al.[41] | 4,456 | 7,005 | 63.61% | tweets
Founta et al.[42] | - | 80,000 | 17.50% | tweets
Mishra et al.[21] | 5,355 | 16,914 | 31.66% |  | tweets, users,
---
relationships
Ours | 1,009 | 12,780 | 7.90% |  | tweets, users,
---
relationships

##### Data collection

Based on the above discussions, we want to build a dataset that the
distribution of users and tweets is closer to the real-world environment. In
most existing datasets, tweets are collected by keyword [23]. Specifically,
they first collect tweets using keywords from the lexicon. Then they collect
the authors of the tweets and the relationships of the authors. However, they
can’t collect the tweets without keywords from the lexicon. In our method, we
first collect the users based on the lexicon. Then we collect their
relationships and tweets. This method will not lose any tweets.

Table 2: Raw data and labeled data Data Type | Raw Data | Labeled Data
---|---|---
Tweets | 443,400 | 12,780
Users | 387,680 | 1,260
Relationship | 559,735 | 8,877

Finally, our program collected 443,400 tweets, 387,680 users, and 559,735
relationships from Twitter. These tweets were posted from January 2018 to
March 2021. Details of data collection are as follows.

  1. 1.

Instead of using offensive words, we construct lexicon by topic-related words.
Then the users and the social relationships are collected with this lexicon.
Specifically, the first-degree and second-degree friends and tweets are
collected. Due to a large number of second-degree friends, we only collect
tweets from first-degree friends. Finally, raw data includes 443,400 tweets,
387,680 users, and 559,735 relationships.

  2. 2.

In the raw data, we randomly select users and expand them to second-degree
nodes. Then the users without tweets and social relationships are removed.
Meanwhile, we focus on English offensive language detection and filter the
others. After the tweets are annotated, only the users with annotated tweets
and their relationships are retained. Finally, 1,260 users, 8,877
relationships, and 12,780 tweets are obtained. Among them, the number of
offensive tweets is 1,009, accounting for 7.90%.

##### Annotation

The offensiveness of tweets includes racism, sexism, geography, etc. [2]. The
offensiveness of many tweets is not obvious, so the determination of offensive
tweets is a difficult task. Therefore, we need a clear discriminant to guide
us in labeling tweets. Our dataset is annotated using the following criteria.
The criteria refer to the annotation recommendations of Waseem et al.[12] and
Zampieri et al.[1]. Tweets that meet one of the following are labeled as
offensive.

  1. 1.

Criticizing or satirizing the target without well-founded arguments [12].

  2. 2.

Abusing directed at individuals and groups [1].

  3. 3.

Blatant misrepresentation of facts or attempting to misrepresent the view of
the subject with unfounded claims [12].

  4. 4.

Negative stereotyping of an object.

  5. 5.

Discrimination against an aspect of the subject.

  6. 6.

Criticism of negative facts with a strong subjective negative bias.

  7. 7.

Causing discomfort.

We label the tweets offensively and non-offensively, and this labeling is done
by three graduate students. Two graduate students label the same tweet, and if
the results are different, the third graduate student will make a judgment.

####  3.1.2 Dataset analysis

To analyze the user distribution and community distribution of the dataset, we
perform the following statistics.

##### User analysis

Among the 1,260 users, the median number of labeled tweets for each user is
11. There are 1,009 offensive tweets in the dataset, and these tweets are
posted by 220 users. The distribution is shown in Figure 2.

Figure 2: Distribution of users of offensive language

The horizontal axis of the figure above shows the number of offensive tweets
posted by each user. The vertical axis shows the number of users who posted
offensive tweets. As we can see, the majority of users posted between 1 and 11
tweets. The largest number of offensive tweets posted by one user is 54,
accounting for 5.40% of the total. The data distribution is more reasonable
than the existing datasets [21, 24]. The offensive tweets are posted by a
small subset of users on social networks.

##### Community analysis

All users are divided into 22 communities by the algorithms of Blondel et al.
[43] and Lambiotte et al. [44]. We show the user distribution of communities
with the number of nodes greater than one. The communities are sorted by the
proportion of offensive nodes, as shown in Table 3.

Table 3: Statistics on community distribution of offensive users Community id |  The number of nodes in community |  The number of offensive nodes in community |  The percentage of offensive nodes in community
---|---|---|---
1 | 250 | 70 | 27.97%
2 | 241 | 15 | 6.22%
3 | 219 | 55 | 25.10%
4 | 201 | 34 | 16.90%
5 | 130 | 15 | 11.52%
6 | 88 | 6 | 6.81%
7 | 75 | 20 | 26.63%
8 | 23 | 1 | 4.34%
9 | 21 | 1 | 4.75%

The Percentage of offensive nodes shows the ratio of offensive nodes in the
total nodes of each community. More than 20% of users in communities 1, 3, and
7 have posted offensive language, and they are shown in Figure 3(a). Figure
3(b) shows the users of other communities. The blue nodes are users who have
not posted offensive language. And the red nodes are users who have posted
offensive language.

(a) The users in communities 1, 3, 7

(b) The users in other communities

(c) All users

Figure 3: The distribution of users in the communities

Combined with the discussions in dataset analysis, we draw the following
conclusions.

  1. 1.

Users who have posted offensive language are likely to continue to post
offensive language.

  2. 2.

Users who post offensive language congregate in some communities. These
communities contain the majority of users who post offensive language.

###  3.2 Social graph building

The online social network is characterized by a complex network in which users
are represented as nodes and the follower-followee information as edges [45,
46, 47]. We construct a directed social graph. The direction of the edge is
from the follower to the followee. If user $x$ follows user $y$ then an edge
will be generated that points from $x$ to $y$. The attributes of the graph
nodes only include information about the user historical behavior.

The offensiveness of users is associated with user historical behavior
information rather than community structure [22]. Therefore, we use user
historical behavior information to represent the offensiveness of users. The
ratio of non-offensive language and offensive language in the training data is
counted as user historical behavior information. For example, the user posted
three offensive tweets and five non-offensive tweets, and the user historical
behavior information is $(3,5)$. It is used to indicate the opinion of this
user to post offensive language. This feature is a node attribute on the
social graph. If a user has no tweets, the default value $(1,1e-6)$ is used.
Training model with this social graph allows the user features to represent
the offensiveness of that user.

In this paper, the built graph consists of 1,260 nodes with 8,878 edges. We
add self-loops to each node. The nodes have an average degree of 6.66. During
the model prediction, the labels of tweets and user historical behavior
information of the users are unknown. Therefore, the relevant information of
the test set is removed from the social graph. The social graph we used for
training and testing is shown in Figure 4. The red node indicates that the
user’s tweets are in the test set. The green node indicates that the user’s
tweets are in the training set. The purple node indicates that both the
training set and the test set contain its tweets.

Figure 4: Training and test graph

###  3.3 Text pre-processing

The processing of this part is directly related to the effectiveness of the
offensive language detection task. We process the text from the following
aspects.

  1. 1.

Emojis. Tweets contain a large number of emojis. Emojis play an important role
in the sentiment tendency of tweets. If the emojis are removed directly, this
part of sentiment information will be lost. We use Python’s third-party
libraries emoji222https://github.com/carpedm20/emoji and
ekphrasis333https://github.com/cbaziotis/ekphrasis [48] to convert emojis into
words with similar meanings.

  2. 2.

URLs. To avoid the impact of URLs on semantics of tweet embeddings, we replace
all URLs with _“http”_ [19]. For example, _“Refinancing during Covid-19.
https://t.co/EzjVxwoLq7”_ into _“Refinancing during Covid-19. http”_

  3. 3.

Hashtag segmentation. To utilize the textual information in hashtags, they are
converted into phrases. This is done using the Python third-party library
ekphrasis, e.g. _“#SomeHashtagText”_ into _“Some Hashtag Text”_.

  4. 4.

Other standardization. Tweets containing user references, email addresses,
time, currency, date, etc. may have an impact on the semantics of tweet
embeddings. Thus, we convert these uniformly to _“ $<$category$>$”_, e.g.
_“@Kevin”_ to _“ $<$user$>$”_, which uses ekphrasis.

###  3.4 Then end-to-end model

We design the end-to-end model for offensive language detection. The model
consists of graph attention network layer, BERT, attention layer, and feed-
forward layer. The learning rate of the GAT layer is set to 1e-2 and the rest
are set to 5e-5. The parameters are initialized by the Xavier normal
distribution [49]. In addition, the Adam optimizer [50] is used as
optimization. The structure of this model is shown as Figure 5.

Figure 5: Structure of the end-to-end offensive language detection model

####  3.4.1 GAT layer

Graph Attention Networks were proposed by Veličković et al. [26] in 2018. GAT
uses the attention mechanism to assign different weights to different nodes,
which is suitable for graph representation on social networks. The social
graph is represented as $G=\left(V,E\right)$, $V$ is the set of user nodes and
$E$ is the set of edges. The node features are represented as
$h=\\{\vec{h}_{1},\vec{h}_{2},...,\vec{h}_{n}\\},\vec{h}_{i}\in R^{F}$, where
$n$ is the number of nodes and $F$ is the dimensionality of the node features.
We can obtain a high-level representation of the feature $h$ by the following
linear transformation.

$z_{i}^{(l)}=W^{(l)}h_{i}^{(l)}$ (1)

Where $W$ is the weight matrix, $h_{i}$ is the feature of each node, and
$z_{i}$ is the transformed feature expression. The $(l)$ represents that this
formula is the matrix calculation of layer $l$.

For node $i$, the correlation coefficient is calculated by node $i$ and its
one-degree neighbor $j$. The nonlinear function LeakyReLU is used as the
activation function to obtain the attention scores. After that, the attention
scores are normalized by softmax.

$e_{ij}^{(l)}=LeakyReLU(\vec{a}^{(l)^{T}}(z_{i}^{(l)}|z_{j}^{(l)}))$ (2)

$a_{ij}^{(l)}=\frac{\exp(e_{ij}^{(l)})}{\sum_{k\in
N_{(i)}}\exp(e_{ik}^{(l)})}$ (3)

Where $a$ is a single-layer feedforward neural network and $e_{ij}$ is the
attention scores. The $a_{ij}$ is the normalized attention coefficient.

After that, the attention coefficient is used to perform linear transformation
with the corresponding node features. Finally, it is passed through the
activation function as the output.

$h_{i}^{(l+1)}=\sigma(\sum_{j\in N_{(i)}}\alpha_{ij}^{(l)}z_{j}^{(l)})$ (4)

User representations are obtained by the multi-head attention mechanism, which
stacks the attention of each head. In addition, to reduce the impact of the
network on the original features, we perform a linear transformation on the
original features and then stack them with the features calculated by
attention. The formula is as follows.

$h_{i}^{(l+1)}=(||_{k=1}^{K}\sigma(\sum_{j\in
N_{(i)}}\alpha_{ij}^{k}W^{k}h_{j}^{(l)}))||(Wh_{j}^{(l)})$ (5)

This part uses a single-layer graph attention network with eight attention
heads and the hidden layer size is set to 768. The output of this layer is
represented as $H=\\{H_{1},H_{2},\ldots,H_{n}\\}$, where $n$ is the number of
users. For the user $i$,
$H_{i}=\left\\{\vec{h}_{i}^{\prime(1)},\vec{h}_{i}^{\prime(2)},\ldots,\vec{h}_{i}^{\prime(N)},\vec{r}_{i}\right\\}$,
and $N$ is the number of attention heads. The $\vec{r}_{i}$ is the residual,
$\vec{r}_{i}=\vec{h}_{i}$ .

####  3.4.2 BERT

Google released Bidirectional Encoder Representation from Transformer (BERT)
[27] based on the Encoder of Transformer [51] in 2018 with improved
performance for several NLP tasks. The BERT uses a large-scale corpus for pre-
training. The model can be fine-tuned for downstream tasks when performing
specific tasks.

This part mainly consists of the BERT Base model. The tweets are fed into the
pre-trained BERT model. Then they are passed through 12 layers of self-
attention to obtain the embeddings of the context. The attention dropout is
set to 0.5 and the hidden dropout is set to 0.1. Meanwhile, texts longer than
512 are truncated. The input texts are denoted as
$T=\left\\{t_{1},t_{2},\ldots,t_{s}\right\\}$, and $s$ is the number of texts
in the current batch. For a text, it is expressed as
$t_{i}=\left\\{t_{i}^{(1)},t_{i}^{(2)},\ldots,t_{i}^{(M)}\right\\}$, and $M$
is the length of the text. The output vectors are denoted as
$F=\left\\{F_{1},F_{2},\ldots,F_{n}\right\\}$, where
$F_{i}=\left\\{\vec{f}_{i}^{(1)},\vec{f}_{i}^{(2)},\ldots,\vec{f}_{i}^{(M)}\right\\}$.

####  3.4.3 Attention layer

The attention mechanism focuses the model’s attention on key information [52].
We use the attention mechanism to fuse the user features with the tweet
features. The inputs of this layer are the user features $H$ from the GAT
layer and the text features $F$ from the BERT. The users and the texts are not
in one-to-one correspondence, but one user corresponds to multiple texts.
Since the final task is at the text level, only the user features
corresponding to the text are retained in the current batch. Then the user
features are appended onto the text features, denoted as
$X_{i}=\left\\{\vec{f}_{i}^{(1)},\ldots,\vec{f}_{i}^{(M)},\vec{h}_{i}^{\prime(1)},\ldots,\vec{h}_{i}^{\prime(N)},\vec{r}_{i}\right\\}$.
In addition, to reinforce the location information of the sequence, we inject
the position encoding according to the method in Transformer [51]. Note that
the user features from different attention heads of GCNs use the same position
encoding. The resulting features are multiplied by the three weight matrices
$W$ to obtain the matrices $Q$, $K$, $V$ after performing layer normalization
[53], and the attention is computed as follows.

$Attention(Q,K,V)=softmax\left(\frac{QK^{T}}{\sqrt{d_{k}}}\right)V$ (6)

$head_{i}=Attention\left(QW_{i}^{Q},KW_{i}^{K},VW_{i}^{V}\right)$ (7)

$MultiHead(Q,K,V)=Concat\left(head_{1},\ldots,head_{h}\right)W^{O}$ (8)

Where $d_{k}$ is the dimensionality of matrix $K$ and $W_{i}^{Q}$,
$W_{i}^{K}$, $W_{i}^{V}$, $W_{i}^{O}$ is the parameter matrix. The output of
multi-headed attention is denoted as
$X_{i}^{\prime}=\left\\{\vec{x}_{1},\ldots,\vec{x}_{M},\vec{x}_{M+1},\ldots\vec{x}_{M+N+1}\right\\}$.
After this, we add residual linking and layer normalization.

####  3.4.4 Feed-Forward layer

Following the attention layer is a feedforward network layer, which contains a
ReLU activation function and a linear transformation.

$FFN(x)=\max(0,x)W+b$ (9)

Where $W$ is the weight of the fully connected layer and $b$ is the bias.

####  3.4.5 Loss function

For the data imbalance problem, we use Focal Loss [54] to mitigate the
classification difficulty of imbalanced samples.

$L_{fl}=\left\\{\begin{array}[]{cc}-\alpha\left(1-y^{\prime}\right)^{\gamma}\log
y^{\prime},\quad y=1\\\
-(1-\alpha)y^{\prime\gamma}\log\left(1-y^{\prime}\right),y=0\end{array}\right.$
(10)

Where $\alpha$ and $\gamma$ are both adjustable hyperparameters, we set
$\alpha=0.25$ and $\gamma=2$. The $y^{{}^{\prime}}$ is the model prediction
value, which is between $(0-1)$. When $y=1$, $y^{{}^{\prime}}$ tends to $1$,
indicating easy positive samples, and its contribution to the weights tends to
$0$. When $y=0$, $y^{{}^{\prime}}$ tends to $0$, indicating easy negative
samples, and its contribution to the weights tends to $0$.

##  4 Experiments and Results

In this section, we evaluate the performance of the proposed method. First, we
describe the experimental setup. Second, our method is compared with the
current state-of-the-art methods. Third, we evaluate the effectiveness of the
modules in CT-OLD. Then we compare the performance of feature initialization
methods. Finally, we explore the differences introduced by different social
graph modeling methods.

###  4.1 Experimental setup

To evaluate the effectiveness of the end-to-end detection method proposed in
this paper for offensive language detection, we conducted a series of
experiments. Our method uses a batch size of 64, and the rest of the methods
are executed according to the original papers. All experiments are conducted
on 12,780 samples, 1,260 users, and 8,877 relations. The dataset is divided
7:3 by tweets and the average results are calculated by 10 independent
replications. All experiments are executed on a server with 128G RAM, Intel
Xeon Gold 6130 2.10GHz, and NVIDIA Tesla V100. In experiments, all models are
trained with maximum 20 epochs. Besides, the early stop strategy is used if
the F1 score doesn’t increase in five continuous epochs.

####  4.1.1 Baseline analysis

To perform performance evaluation, we conducted a series of baseline studies
of these state-of-the-art methods for offensive language detection as follows.

BERT. Liu et al. [18] won the first place in subtask A of OffensEval-2019 with
the fine-tuned BERT model. The text embeddings are obtained by the BERT model
and then classified by linear layer.

BERT+CNN. Safaya et al. [20] combined BERT with CNN for offensive language
detection. Their method ranked in the top four on all three languages in
subtask A of OffensEval-2020. BERT+CNN learns text representations through the
BERT model. Then the output of the last four layers of BERT is fed into CNN
networks with different size filters.

LR+AUTH. 444The code of this baseline is taken from
https://github.com/pushkarmishra/AuthorProfilingAbuseDetection. LR+AUTH [21]
is the first work to combine community structure and texts for offensive
speech detection. First, social relationships are constructed as social
graphs. Then user representations are learned by node2vec. For each tweet, the
user features are appended to the n-gram features of the tweet, and the LR
classifier is trained.

LR+GCN. LR+GCN [22] models users and tweets to construct a heterogeneous
graph. It uses bags of words as the feature of users. This method uses two
layers of GCNs to train by minimizing the cross-entropy of the labeled nodes
in the graph. The output of the first hidden layer of the trained GCNs is used
as the user embeddings. For each tweet, user features are appended onto the
character n-gram features of the tweet for training the LR classifier. State-
of-the-art results were obtained at that time.

The LR+AUTH code is publicly available and directly used in our experiments.
The rest of the baseline is our implementation based on the original paper.

####  4.1.2 Performance metrics

To quantify model effects, we show AUC, Accuracy, Macro Recall, Macro
Precision, and Macro F1 score.

$AUC=\frac{\sum_{i\in positiveclass}rank_{i}-\frac{M\cdot(M+1)}{2}}{M\cdot N}$
(11)

Where $\sum_{i\in positiveclass}rank_{i}$ is the sum of the positive samples
ranked by probability score from smallest to largest, $M$ is the number of
positive samples, and $N$ is the number of negative samples. When the
probability scores are the same, the $rank$ value is the average of the $rank$
of the same probability scores.

For the other four metrics, the confusion matrix is calculated first, and for
each classification, the true positive (TP), the false negative (FN), the
false positive (FP), and the true negative (TN) are calculated as follows:

$Accuracy=\frac{TP+TN}{TP+TN+FP+FN}$ (12)

$Recall_{i}=\frac{TP_{i}}{TP_{i}+FN_{i}}$ (13)

$Precision_{i}=\frac{TP_{i}}{TP_{i}+FP_{i}}$ (14)

$F1_{i}=\frac{2\cdot Precision_{i}\cdot Recall_{i}}{Precision_{i}+Recall_{i}}$
(15)

The subscript $i$ of each metric represents the $i$-th category, and the
number of categories is represented by $L$. After calculating the Recall,
Precision, and F1 for each category, the average value is used to obtain the
macro value. The formula for calculating Macro-F1 is given below, and the rest
of the metrics are the same.

$Macro-F1=\frac{1}{L}\sum_{i=1}^{L}F1_{i}$ (16)

###  4.2 Evaluation of the proposed detection approach

####  4.2.1 Comparison with baseline method

Table 4: The baselines vs. CT-OLD Model | Accuracy | Precision | Recall | F1
---|---|---|---|---
LR+AUTH[21] | 93.87 | 82.97 | 71.57 | 75.80
LR+GCN[22] | 95.62 | 89.70 | 79.23 | 83.49
BERT[18] | 95.41 | 83.70 | 88.80 | 86.02
BERT+CNN[20] | 95.64 | 84.65 | 88.65 | 86.51
CT-OLD(Ours) | 96.92 | 89.83 | 90.06 | 89.94
Figure 6: ROC-AUC of baselines and CT-OLD

Table 4 shows the AUC, Accuracy, Precision, Recall, and F1 score of the
baseline methods and the CT-OLD on our dataset. In LR+AUTH and LR+GCN, text
features are extracted by n-grams, resulting in relatively poor performance.
In addition, they are classified after calculating user embeddings and text
embeddings separately. This divides the training process into two parts.
LR+AUTH learns user embeddings only through community structure and has the
worst performance. LR+GCN considers user features represented by text, which
greatly improves the performance of detection. The BERT-based method obtains
competitive scores by considering only text. And BERT-CNN has further
improvement, but it is not significant.

CT-OLD obtains the first place in all evaluation metrics, and the F1 score was
3.43% higher than the second. For the ROC curve, we find that CT-OLD’s curve
covers all other methods. This proves that our method has a great advantage in
the offensive language detection task.

####  4.2.2 Effect of different training ratios

This experiment shows the performance of the baseline method with CT-OLD at
different training ratios from 0.1 to 0.9.

Figure 7: Accuracy, Precision, Recall, and F1 of baselines and CT-OLD for
training sets with percentages between 0.1 and 0.9

It can be seen from Figure 7 that different training ratios have an impact on
all methods. Meanwhile, each performance metric gradually increases with the
increase of the training ratio. The difference between BERT and BERT-CNN in
different training ratios is small. The F1 score of LR+AUTH fluctuates the
most. And it is lower than the other methods at all ratios.

The F1 score of CT-OLD increases more obviously when the training ratio is
from 0.1 to 0.4. In addition, the difference between the F1 scores of CT-OLD
and BERT is also increasing. This is because when the training set is small,
the node features are relatively sparse on the social graph. With increasing
the training data, the opinion of users to post offensive language is learned
obviously. As a whole, CT-OLD achieves a significant advantage in all metrics.

####  4.2.3 Detection model ablation tests

To explore the influence of the graph attention network layer, the BERT, the
residuals of the GAT layer, the multi-headed attention of the GAT layer, and
the attention layer used in the CT-OLD, we perform ablation tests.
Specifically, we remove only one of the above modules from the CT-OLD and show
the performance of the model. The results are shown in Table 5

Table 5: The impact of modules on detection performance Model | AUC | Accuracy | Precision | Recall | F1
---|---|---|---|---|---
CT-OLD\GAT | 0.9775 | 95.83 | 87.02 | 84.90 | 85.92
CT-OLD\BERT | 0.9225 | 92.44 | 75.48 | 85.33 | 79.31
CT-OLD\Residual of GAT | 0.9802 | 96.35 | 88.18 | 87.75 | 87.96
| CT-OLD\Multi-head
---
Attention of GAT
0.9824 | 96.50 | 87.63 | 90.40 | 88.95
CT-OLD\Attention Layer | 0.9852 | 96.56 | 88.13 | 89.86 | 88.97
CT-OLD | 0.9853 | 96.92 | 89.83 | 90.06 | 89.94

We find that the removal of any module will reduce the model performance. All
modules contribute to the effectiveness of the CT-OLD. The CT-
OLD$\backslash$GAT removes the graph attention network layer, with similar
results to the two BERT-based methods used as baselines. The CT-
OLD$\backslash$BERT removes the text embeddings obtained by BERT. The
connection between the GAT and our detection target is only the user
historical behavior information. Meanwhile, it only detects the granularity of
the author. In such a coarse-grained detection, the F1 score has reached
79.31%, exceeding the LR+AUTH. The GAT uses community structure with user
behavior information rather than textual content. So, the information it
learns is different from the text-based method. In addition, the text-based
BERT captures semantic information well. Therefore, the end-to-end detection
method combining the two features significantly improves the detection
performance.

In the remaining four experiments, the CT-OLD$\backslash$Residual of GAT
removes the residual links from the graph attention network and has the most
degraded performance. This is because the residual connection enhances the
influence of the original features.

####  4.2.4 Comparison of initialization of user features

The node features are an important factor for graph representation learning.
However, for users whose tweets exclude in the training set, their initial
features are not available during the training. This requires feature
initialization for this subset of users. Therefore, we consider four
initialization methods.

  1. 1.

All-0 initialization. The number of tweets for both categories is unknown. It
is set to $[0,0]$ by default.

  2. 2.

All-1 initialization. The probability of a user posting tweets in both
categories is the same. Set to $[1,1]$.

  3. 3.

Average initialization. For each category of tweets, the average number of
user posts is used as the initial value.

  4. 4.

Non-offensive initialization. There are a few users who post offensive
language in social networks. And it is assumed that users only post non-
offensive tweets. It is set to $[1,1e-6]$.

In addition to the default case of 0.7 training ratio, we also show the
detection performance in the sparse data case of 0.1 training ratio. Table 6
shows the performance of different feature initialization methods.

Table 6: Performance of different feature initialization methods training ratio | Model | Accuracy | Precision | Recall | F1
---|---|---|---|---|---
Sparse data | All-0 | 94.17 | 80.51 | 77.85 | 79.11
All-1 | 94.60 | 81.76 | 80.44 | 81.08
Avg | 94.70 | 82.79 | 78.79 | 80.63
Non-off | 95.16 | 84.68 | 80.19 | 82.24
Default | All-0 | 96.45 | 87.05 | 91.22 | 89.00
All-1 | 96.40 | 86.79 | 91.34 | 88.90
Avg | 96.61 | 87.70 | 91.31 | 89.40
Non-off | 96.92 | 89.83 | 90.06 | 89.94

The effectiveness of user historical behavior information decreases in the
case of sparse data. The detection performance is more dependent on the user
feature initialization. The experiments show that non-offensive initialization
performs the best. By default, only a small part of users rely on feature
initialization. The performance of the four methods is similar. The difference
between the highest and lowest F1 score is only less than 1%.

####  4.2.5 Comparison of social graph modeling

In the past, modeling of community information could be done by computing node
embeddings through the network structure of the communities [21]. This method
is able to capture the structure of communities but ignores the association
with offensive language detection.

Mishra et al. [22] built user features through the n-gram features of tweets,
which effectively combined community structure and text information. In this
method, community structure and linguistic features are captured together. The
user features obtained by this method contain all the information of the
tweets posted by the user. However, the user representations of this method
depend on the quality of the tweet representations deeply.

We compare three social graph modeling methods for users.

  1. 1.

Using the modeling method of Mishra et al. [22], but we only model the users.
The user features are constructed by binary bag-of-words representations of
the user’s posted tweets. This method is called the bow feature.

  2. 2.

We tag users based on whether they post offensive language. If the user has
posted offensive language, the tag is 1, otherwise, it is 0. This method we
call the hard feature.

  3. 3.

In our proposed method, we use the user historical behavior information as
user features. This method is called the soft feature.

Table 7: The performance of different social graph modeling methods Model | Accuracy | Precision | Recall | F1
---|---|---|---|---
GAT-bow | 92.33 | 75.20 | 84.56 | 78.88
GAT-hard | 85.08 | 66.75 | 87.02 | 70.57
GAT-soft | 92.44 | 75.48 | 85.33 | 79.31
CT-OLD-bow | 96.01 | 86.15 | 88.70 | 87.37
CT-OLD-hard | 96.35 | 88.42 | 87.32 | 87.86
CT-OLD-soft | 96.92 | 89.83 | 90.06 | 89.94

Table 7 shows the comparison in different methods of social graph modeling.
Among the three methods of GAT, the F1 score using the soft feature is the
highest, the bow feature is the second-highest, and the hard feature is the
lowest. This may be because the hard feature contains too little information.
As long as users have posted offensive language, their tweets are judged to be
offensive. The bow feature contains complete information, but large-scale
features also have more uncertainty during training. The soft feature has more
constraints than the bow feature and more complete information than the hard
feature.

The soft feature still gets the highest F1 score among the three CT-OLD
methods, while the difference between the bow feature and hard feature is not
obvious. The reason for this variation is that the graph learning module using
the hard feature and soft feature has a clear purpose to learn the
representation of users to post offensive language. The bow feature, on the
other hand, uses raw features of text embeddings with complex features,
resulting in difficulties in learning effective information. Therefore, the
hard feature has stronger performance than the bow feature. However, the hard
feature will be mislabeled if the training set excludes the offensive tweets
of a user and the test set does, affecting the detection performance. The soft
feature based on user historical behavior information has the advantages of
others and achieves better performance.

##  5 Conclusion

In this paper, we proposed an end-to-end method for detecting offensive
language with graph attention networks and BERT. The model captures community
structure and text information at the same time and fuses them by attention
mechanisms. Specifically, we used user historical behavior information to
represent user opinion. This information is a node attribute on the social
graph that associates the social graph with the offensiveness of the user. We
evaluated the effectiveness of our method on the dataset we constructed. The
experimental results showed that the proposed method is effective for
detecting offensive language, and it has better performance than other
methods.

In online social networks, the generalization ability of the offensive
language detection model that learns community structure and text at the same
time will exceed that of the text-based model. But it is difficult to obtain
more datasets with different user distribution and community structure. In the
future, we hope to further investigate the impact of social graphs on
generalization ability.

## References

  * [1] M. Zampieri, S. Malmasi, P. Nakov, S. Rosenthal, N. Farra, R. Kumar, Predicting the type and target of offensive posts in social media, in: Proceedings of the 17th Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Minneapolis, USA, 2019, pp. 1415–1420.
  * [2] P. Fortuna, S. Nunes, A survey on automatic detection of hate speech in text, ACM Computing Surveys 51 (4) (2018) 1–30.
  * [3] E. Mosca, M. Wich, G. Groh, Understanding and interpreting the impact of user context in hate speech detection, in: Proceedings of the 9th International Workshop on Natural Language Processing for Social Media, Online, 2021, pp. 91–102.
  * [4] Z. Waseem, T. Davidson, D. Warmsley, I. Weber, Understanding abuse: A typology of abusive language detection subtasks, in: Proceedings of the 1st Workshop on Abusive Language Online, Vancouver, Canada, 2017, pp. 78–84.
  * [5] P. Kapil, A. Ekbal, A deep neural network based multi-task learning approach to hate speech detection, Knowledge-Based Systems 210 (2020) 106458.
  * [6] A. Sheth, V. L. Shalin, U. Kursuncu, Defining and detecting toxicity on social media: context and knowledge are key, Neurocomputing (2021) Online.
  * [7] R. Magu, K. Joshi, J. Luo, Detecting the hate code on social media, in: Proceedings of the 11th International Conference on Web and Social Media, Montréal, Canada, 2017, pp. 608–611.
  * [8] N. Djuric, J. Zhou, R. Morris, M. Grbovic, V. Radosavljevic, N. Bhamidipati, Hate speech detection with comment embeddings, in: Proceedings of the 24th International Conference on World Wide Web, Florence, Italy, 2015, pp. 29–30.
  * [9] C. Nobata, J. Tetreault, A. Thomas, Y. Mehdad, Y. Chang, Abusive language detection in online user content, in: Proceedings of the 25th International Conference on World Wide Web, Montréal, Canada, 2016, pp. 145–153.
  * [10] T. Davidson, D. Warmsley, M. Macy, I. Weber, Automated hate speech detection and the problem of offensive language, in: Proceedings of the 11th International Conference on Web and Social Media, Montréal, Canada, 2017, pp. 512–515.
  * [11] M. Dadvar, D. Trieschnigg, R. Ordelman, F. de Jong, Improving cyberbullying detection with user context, in: Proceedings of the 35th European Conference on Information Retrieval, Moscow, Russia, 2013, pp. 693–696.
  * [12] Z. Waseem, D. Hovy, Hateful symbols or hateful people? predictive features for hate speech detection on twitter, in: Proceedings of the 14th Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, San Diego, USA, 2016, pp. 88–93.
  * [13] E. F. Unsvåg, B. Gambäck, The effects of user features on twitter hate speech detection, in: Proceedings of the 2nd Workshop on Abusive Language Online, Brussels, Belgium, 2018, pp. 75–85.
  * [14] P. Badjatiya, S. Gupta, M. Gupta, V. Varma, Deep learning for hate speech detection in tweets, in: Proceedings of the 26th International Conference on World Wide Web Companion, Perth, Australia, 2017, pp. 759–760.
  * [15] J. H. Park, P. Fung, One-step and two-step classification for abusive language detection on twitter, in: Proceedings of the 1st Workshop on Abusive Language Online, Vancouver, Canada, 2017, pp. 41–45.
  * [16] S. Yoon, J. Shin, K. Jung, Learning to rank question-answer pairs using hierarchical recurrent encoder with latent topic clustering, in: Proceedings of the 16th Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, New Orleans, Louisiana, 2018, pp. 1575–1584.
  * [17] P. Le-Hong, Diacritics generation and application in hate speech detection on vietnamese social networks, Knowledge-Based Systems 233 (2021) 107504.
  * [18] P. Liu, W. Li, L. Zou, Nuli at semeval-2019 task 6: Transfer learning for offensive language detection using bidirectional transformers, in: Proceedings of the 13th International Workshop on Semantic Evaluation, Minneapolis, USA, 2019, pp. 87–91.
  * [19] W. Dai, T. Yu, Z. Liu, P. Fung, Kungfupanda at semeval-2020 task 12: Bert-based multi-tasklearning for offensive language detection, in: Proceedings of the 14th Workshop on Semantic Evaluation, Online, 2020, pp. 2060–2066.
  * [20] A. Safaya, M. Abdullatif, D. Yuret, Kuisail at semeval-2020 task 12: Bert-cnn for offensive speech identification in social media, in: Proceedings of the 14th Workshop on Semantic Evaluation, Online, 2020, pp. 2054–2059.
  * [21] P. Mishra, M. Del Tredici, H. Yannakoudakis, E. Shutova, Author profiling for abuse detection, in: Proceedings of the 27th International Conference on Computational Linguistics, Santa Fe, USA, 2018, pp. 1088–1098.
  * [22] P. Mishra, M. Del Tredici, H. Yannakoudakis, E. Shutova, Abusive language detection with graph convolutional networks, in: Proceedings of the 17th Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Minneapolis, USA, 2019, pp. 2145–2150.
  * [23] M. H. Ribeiro, P. H. Calais, Y. A. Santos, V. A. Almeida, W. Meira Jr, Characterizing and detecting hateful users on twitter, in: Proceedings of the 12th International Conference on Web and Social, Stanford, USA, 2018, pp. 676–679.
  * [24] M. Wiegand, J. Ruppenhofer, T. Kleinbauer, Detection of abusive language: the problem of biased datasets, in: Proceedings of the 17th conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Minneapolis, USA, 2019, pp. 602–608.
  * [25] A. Arango, J. Pérez, B. Poblete, Hate speech detection is not as easy as you may think: A closer look at model validation, in: Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, Paris, France, 2019, pp. 45–54.
  * [26] P. Veličković, G. Cucurull, A. Casanova, A. Romero, P. Lio, Y. Bengio, Graph attention networks, in: Proceedings of the 6th International Conference on Learning Representations, Vancouver, Canada, 2018, pp. 1–12.
  * [27] J. Devlin, M.-W. Chang, K. Lee, K. Toutanova, Bert: Pre-training of deep bidirectional transformers for language understanding, in: Proceedings of the 17th Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Minneapolis, USA, 2019, pp. 4171–4186.
  * [28] W. Tang, G. Luo, Y. Wu, L. Tian, X. Zheng, Z. Cai, A second-order diffusion model for influence maximization in social networks, IEEE Transactions on Computational Social Systems 6 (4) (2019) 702–714.
  * [29] W. Tang, L. Tian, X. Zheng, G. Luo, Z. He, Susceptible user search for defending opinion manipulation, Future Generation Computer Systems 115 (2021) 531–541.
  * [30] W. Tang, B. Hui, L. Tian, G. Luo, Z. He, Z. Cai, Learning disentangled user representation with multi-view information fusion on social networks, Information Fusion 74 (2021) 77–86.
  * [31] W. Tang, X. Xu, G. Luo, Z. He, K. Zhan, Budgeted persuasion on user opinions via varying susceptibility, in: Proceedings of the 39th IEEE International Performance Computing and Communications Conference, Austin, USA, 2020, pp. 1–8.
  * [32] A. H. Razavi, D. Inkpen, S. Uritsky, S. Matwin, Offensive language detection using multi-level classification, in: Proceedings of the 23rd Canadian Conference on Artificial Intelligence, Ottawa, Canada, 2010, pp. 16–27.
  * [33] S. O. Sood, J. Antin, E. Churchill, Using crowdsourcing to improve profanity detection, in: Proceedings of the 21st AAAI Spring Symposium Series: Wisdom of the Crowd, Stanford, USA, 2012, pp. 69–74.
  * [34] J. Salminen, H. Almerekhi, M. Milenković, S.-g. Jung, J. An, H. Kwak, B. J. Jansen, Anatomy of online hate: developing a taxonomy and machine learning models for identifying and classifying hate in online news media, in: Proceedings of the 12th International Conference on Web and Social Media, Stanford, USA, 2018, pp. 330–339.
  * [35] A. Grover, J. Leskovec, Node2vec: Scalable feature learning for networks, in: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, USA, 2016, pp. 855–864.
  * [36] J. Chen, S. Yan, K.-C. Wong, Verbal aggression detection on twitter comments: Convolutional neural network for short-text sentiment analysis, Neural Computing and Applications 32 (15) (2020) 10809–10818.
  * [37] S. Rajamanickam, P. Mishra, H. Yannakoudakis, E. Shutova, Joint modelling of emotion and abusive language detection, in: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Online, 2020, pp. 4270–4279.
  * [38] T. Caselli, V. Basile, J. Mitrović, M. Granitzer, Hatebert: Retraining bert for abusive language detection in english, arXiv preprint arXiv:2010.12472.
  * [39] J. Qian, M. ElSherief, E. Belding, W. Y. Wang, Leveraging intra-user and inter-user representation learning for automated hate speech detection, in: Proceedings of the 16th Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, New Orleans, Louisiana, 2018, pp. 118–123.
  * [40] T. N. Kipf, M. Welling, Semi-supervised classification with graph convolutional networks, in: Proceedings of the 5th International Conference on Learning Representations, Toulon, France, 2017, pp. 1–14.
  * [41] T. Mandl, S. Modha, P. Majumder, D. Patel, M. Dave, C. Mandlia, A. Patel, Overview of the hasoc track at fire 2019: Hate speech and offensive content identification in indo-european languages, in: Proceedings of the 11th Forum for Information Retrieval Evaluation, Kolkata, India, 2019, pp. 14–17.
  * [42] A. M. Founta, C. Djouvas, D. Chatzakou, I. Leontiadis, J. Blackburn, G. Stringhini, A. Vakali, M. Sirivianos, N. Kourtellis, Large scale crowdsourcing and characterization of twitter abusive behavior, in: Proceedings of the 12th International Conference on Web and Social Media, Stanford, USA, 2018, pp. 491–500.
  * [43] V. D. Blondel, J.-L. Guillaume, R. Lambiotte, E. Lefebvre, Fast unfolding of communities in large networks, Journal of Statistical Mechanics: Theory and Experiment 2008 (10) (2008) 10008.
  * [44] R. Lambiotte, J.-C. Delvenne, M. Barahona, Laplacian dynamics and multiscale modular structure in networks, arXiv preprint arXiv:0812.1770.
  * [45] R. Tang, S. Jiang, X. Chen, H. Wang, W. Wang, W. Wang, Interlayer link prediction in multiplex social networks: An iterative degree penalty algorithm, Knowledge-Based Systems 194 (2020) 105598.
  * [46] R. Tang, X. Chen, C. Wei, Q. Li, W. Wang, H. Wang, W. Wang, Interlayer link prediction based on multiple network structural attributes, Computer Networks 203 (2022) 108651.
  * [47] R. Tang, Z. Miao, S. Jiang, X. Chen, H. Wang, W. Wang, Interlayer link prediction in multiplex social networks based on multiple types of consistency between embedding vectors, IEEE Transactions on Cybernetics (2021) 1–14.
  * [48] C. Baziotis, N. Pelekis, C. Doulkeridis, Datastories at semeval-2017 task 4: Deep lstm with attention for message-level and topic-based sentiment analysis, in: Proceedings of the 11th International Workshop on Semantic Evaluation, Vancouver, Canada, 2017, pp. 747–754.
  * [49] X. Glorot, Y. Bengio, Understanding the difficulty of training deep feedforward neural networks, in: Proceedings of the 13th International Conference on Artificial Intelligence and Statistics, Chia Laguna Resort, Italy, 2010, pp. 249–256.
  * [50] D. P. Kingma, J. Ba, Adam: A method for stochastic optimization, in: Proceedings of the 3rd International Conference on Learning Representations, San Diego, USA, 2015, pp. 1–15.
  * [51] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, I. Polosukhin, Attention is all you need, in: Proceedings of the 31st Advances in Neural Information Processing Systems, Long Beach, USA, 2017, pp. 6000–6010.
  * [52] M.-T. Luong, H. Pham, C. D. Manning, Effective approaches to attention-based neural machine translation, arXiv preprint arXiv:1508.04025.
  * [53] J. L. Ba, J. R. Kiros, G. E. Hinton, Layer normalization, arXiv preprint arXiv:1607.06450.
  * [54] T.-Y. Lin, P. Goyal, R. Girshick, K. He, P. Dollár, Focal loss for dense object detection, IEEE Transactions on Pattern Analysis and Machine Intelligence 42 (2) (2020) 318–327.