# Overview of Bachelors Theses 2021

Vitaly Aksenov, ITMO University
<EMAIL_ADDRESS>
(July 2021)

##  1 Development of a Streaming Algorithm for the Decomposition of Graph
Metrics to Tree Metrics

Student:  Fafurin Oleg, ITMO University

External Supervisor: Michael Kapralov, EPFL

The embedding problem. We are given a graph $G$. We want to embed this graph
onto some tree $T$, so that the shortest distance $d_{G}(u,v)$ between any
pair of vertices $u$ and $v$ does not change much. In other words, we want to
minimize $\max\limits_{(u,v)}\frac{d_{T}(u,v)}{d_{G}(u,v)}$. This value is
named the _distortion_. Obviously, the distortion is upper bounded by the
maximal distortion of edges.

There exists an algorithm that embeds any graph on a tree with distortion
$O(\log^{2}n)$ in the streaming model, i.e., it can use only
$O(n\cdot\mathrm{polylog}\,n)$ memory. It consists of two parts.

In the first part, we insert edges one by one and if for a given edge $(u,v)$
the current distance is less than $t$ then we do not insert it. This
algorithm, obviously, provides a distortion $O(t)$ for each edge and it can be
proven that the total number of edges will not exceed $O(n^{1+\frac{1}{t}})$
[4]. Taking $t=\log n$, we get $O(\log n)$ distortion and
$O(n\cdot\mathrm{polylog}\,n)$.

In the second part, we use a streaming algorithm named FRT [5], it takes a
graph with $O(n\cdot\mathrm{polylog}\,n)$ edges and gets a tree with
distortion $O(\log^{2}n)$.

As the first result, we improved the distortion of this algorithm by taking
$t$ to be $O(\frac{\log n}{\log\log n})$ in the first part and, thus, giving
$O(\frac{\log n}{\log\log n})$ distortion with $O(n\cdot\mathrm{polylog}n)$
edges in the graph. So, in total, the algorithm gives
$O(\frac{\log^{2}n}{\log\log n})$ distortion.

The resulting distortion is the upper bound. We decided to find graphs for
which the distortion matches that upper bound. The following two graphs
satisfy.

Regular graph. We build a regular graph with degree $O(\frac{2\log n}{\log\log
n})$: at first, put all $n$ vertices on a cycle, and then connect each vertex
with $O(\frac{\log n}{\log\log n})$ neighbours in both sides.

Star. Consider $0<\alpha<1$. One of the vertices is a center, from which there
are $n^{\alpha}$ chains with length $n^{1-\alpha}$. Then, we take all the
vertices on the distance at most $O(\frac{\log n}{\log\log n})$ from the
center and add all the edges between them.

Then, we implement the algorithm. The complexity of the first part appeared to
be $O(mn\log n)$ where $m$ is the number of edges and $n$ is the number of
vertices. The complexity of the second part is $O(n^{2}\log n)$.

We run the resulting algorithm on several different open-source network
graphs.

The following plot shows the distortion of paths after the first part of the
algorithm on different graphs such as Facebook [7] and scale-free graphs [6].

The following plot shows the distortion of edges after the second part of the
algorithm (FRT) on different scale-free graphs with different base.

##  2 Development of Memory-friendly Concurrent Data Structures

Student:  Roman Smirnov, ITMO University

External Supervisor: Petr Kuznetsov, Telecom Paris

The main idea of this work is to implement the skip-list so that each node can
store up to $k$ elements instead of one. We designed and implemented the
algorithm using locks. This thesis is mostly technical and the main results
are the experiments.

At first, we chose the best $k$—it appeared to be $32$. Then we compared our
approach with two well-known concurrent data structures based on the skip-
list: ConcurrentSkipListSet [1] from Java standard library and
NonBlockingFriendlySkipListSet [11]. Please, note, that we compared sets and
not maps. It can be seen as that our approach does not lose the performance
much.

Then, we decided to replace Objects in the previous implementation by
integers. For that, we rewrote our algorithm and ConcurrentSkipListSet. This
improved the performance of our data structure almost $2$ times since now $k$
elements reside on the same cache line, while the results of
ConcurrentSkipListSet barely changed.

As the result, we can say that the idea of batching the elements from
different nodes into one seems to be a reasonable approach.

##  3 Theoretical Analysis of the Performance of Concurrent Data Structures

Student:  Daniil Bolotov, ITMO University

External Supervisor: Petr Kuznetsov, Telecom Paris

In this work we tried to predict the performance of MCS lock [9] and Treiber
stack [10]. The prediction is done in the similar manner as in [3].

For MCS lock, we consider a data structure that takes MCS lock, perform the
critical section of size $C$, releases the lock, and then perform the parallel
section of size $P$. Thus, we can get the following code that emulates such
data structure.

⬇

1class Node:

2 bool locked // shared, atomic

3 Node next = null

4

5 tail = null // shared, global

6 threadlocal myNode = null // per process

7operation():

8 myNode = Node()

9 myNode.locked = true

10 pred = tail.getAndSet(myNode) // $W$ or $X$

11 if pred != null:

12 pred.next = myNode

13 while myNode.locked: // pass // $R_{I}$

14 // CS started

15 for i in 1..C: // $C$

16 //nop

17 // CS finished

18 if myNode.next == null: // $R_{I}$

19 if tail.CAS(myNode,null): // $W$ or $X$

20 return

21 else:

22 while myNode.next == null: // $R_{I}$

23 //pass

24 myNode.next.locked = false // $W$

25 //Parallel section

26 for i in 1..P: // $P$

27 //nop

By considering different schedules we can prove that the throughput is equal
to:

$\begin{cases}\frac{\alpha}{2R_{I}+C+2W}&\text{, if
}P+W\leq(N-1)\cdot(2W+C+R_{I})\\\ \frac{\alpha\cdot
N}{(2W+C+R_{I})+(P+W)}&\text{, else}\end{cases},$

where $C$ is the size of the critical section, $P$ is the size of the parallel
section, $W$ is the cost of a write, $R_{I}$ is the cost of a read, and $N$ is
the number of processes.

On Intel Xeon and $15$ processes we get the following throughput, where red is
the prediction and blue is the real execution:

On AMD Opteron and $15$ processes we get the following throughput:

Now, we consider Treiber stack. The pseudocode is the following:

⬇

1class Node:

2 T data;

3 Node next

4

5head = null //shared, atomic

6

7push(data):

8 newHead = Node(data)

9 while !success:

10 oldHead = atomic_read(head) // $M$ or $X$

11 newHead.next = oldHead

12 success = head.compareAndSet(oldHead, newHead) // $W$

13

14pop():

15 Node oldHead

16 while !success:

17 oldHead = atomic_read(head) // $M$ or $X$

18 if (oldHead == null) {

19 return DEFAULT_VALUE // corner case

20 }

21 newHead = oldHead.next

22 success = head.compareAndSet(oldHead, newHead) // $W$

23

24 return oldHead.data

One can see that push and pop operations are similar and we can write them as
one generic function as follows:

⬇

1pop_or_push_operation():

2 while !success do

3 current = atomic_read(head)

4 new = critical_work(current)

5 success = head.compareAndSet(current, new)

Then, we simulate the application of the Treiber stack: we take an element
from the stack and then we perform an execution of size $P$.

⬇

1class Node:

2 T data;

3 Node next

4

5head = null //shared, atomic

6

7operation():

8 newHead = Node(data)

9 while !success:

10 oldHead = atomic_read(head) // $M$ or $X$

11 newHead.next = oldHead

12 success = head.compareAndSet(oldHead, newHead); // $W$

13

14 for i in 1..P: / / $P$

15 //nop

By considering different schedules we can prove that the throughput is equal
to:

$\begin{cases}\frac{\alpha}{M+W}&\text{, if }P\leq(N-1)\cdot(M+W)\\\
\frac{\alpha\cdot N}{(P+M+W)}&\text{, else}\end{cases}$

On Intel Xeon and $15$ processes we get the following results:

On AMD Opteron and $15$ processes we get the following results:

As a result, we get pretty good theoretical approximation of the throughput.

##  4 Parallel Batched Interpolation Search Tree

Student: Alena Martsenyuk, MIPT

In this thesis, we show how to design _parallel batched_ implementation of
Interpolation Search Tree [8]. “Parallel batched” means that we ask the data
structure to apply multiple operations together in parallel.

We developed the data structure that applies a batch of $m$ operations in
$O(m\log\log n)$ work and $O(\log m\log\log n)$ span, where $n$ is the current
size of the tree.

For experiments, we used an Intel Xeon machine with $16$ threads. On this
plot, you can see how much time (OY-axis) it takes to apply $m$ (OX-axis)
operations using different number of processes into a tree of size $2.5\cdot
10^{7}$.

On this plot, you can see how much time (OY-axis) it takes to apply $10^{6}$
operations using different number of processes into a tree of size $n$ (OX-
axis).

Finally, we insert $10^{6}$ elements into the tree of size $5\cdot 10^{7}$ and
check the speedup. The speedup is approximately $11$ on $16$ processes.

##  5 Parallel Batched Self-adjusting Data Structures

Student: Vitalii Krasnov, MIPT

In this thesis, we show how to design parallel batched self-adjusting binary
search tree. We based our data structure on CBTree data structure [2].

We proved that the resulting data structure is static-optimal, i.e., the total
work is equal to $O(\sum\limits_{x}c_{x}\cdot\frac{m}{c_{x}})$ where $m$ is
the total number of operations from the start of the existence of the data
structure and $c_{x}$ is the number of times $x$ is requested. The span of the
algorithm is $\frac{m}{C}$ where $C$ is $\min\limits_{x}c_{x}$.

For experiments, we used an Intel Xeon machine with $16$ threads. All our
experiments has the following construction: we continuously add $10^{3}$
elements to the same tree until it becomes very large—so, the tree is always
the same but growing. On the first plot, one can see how much time (OY-axis)
it takes to apply batches of size $10^{3}$ into a growing tree (OX-axis). The
speedup is approximately $9$ on $12$ processes.

On the second plot, one can see how much time (OY-axis) it takes to apply
batches of size $10^{3}$ taken from a normal distribution into a growing tree
(OX-axis).

Also, our data structure outperforms the set data structure from the standard
C++ library in the sequential setting.

##  6 Parallel Batched Persistent Binary Search Trees

Student: Ildar Zinatulin, MIPT

In this thesis, we show how to design a persistent parallel batched binary
search tree. We consider persistence in the sense of versions. Suppose we are
asked to apply operations $op_{1},op_{2},\ldots,op_{m}$. A result of any
operation is the new version of the tree, and operations should be applied in
some “sequential” order $op_{\pi(1)},\ldots,op_{\pi(m)}$, i.e., a version of
the tree after operation $op_{\pi(j)}$ should be the initial tree after an
application of all first $j$ operations $op_{\pi(1)},\ldots,op_{\pi(j)}$.

We designed a persistent binary search tree that applies the operations in the
order of their arguments. The idea is a little bit complicated and is similar
to the scan function — we make two traversals from top to bottom. The work of
the resulting algorithm is $O(m\log n)$ and the span is $O(\log n\log m)$.

For experiments, we used an Intel Xeon machine with $16$ threads. We performed
only one experiment — the speedup of an application of a batch with size
$10^{5}$ to a tree with size $10^{6}$. As for the binary search tree we used
Treap. The blue dot on the plot is the sequential algorithm for the persistent
Treap.

## References

  * [1] Java concurrentskiplistset, 2021.
  * [2] Y. Afek, H. Kaplan, B. Korenfeld, A. Morrison, and R. E. Tarjan.  CBTree: A practical concurrent self-adjusting search tree.  In Lecture Notes in Computer Science, pages 1–15. Springer Berlin Heidelberg, 2012.
  * [3] V. Aksenov, D. Alistarh, and P. Kuznetsov.  Brief-announcement: Performance prediction for coarse-grained locking.  Proceedings of the thirty seventh annual ACM Symposium on Principles of distributed computing (PODC), pages 411–413, 2018.
  * [4] I. Althöfer, G. Das, D. P. Dobkin, D. Joseph, and J. Soares.  On sparse spanners of weighted graphs.  Discrete and Computational Geometry, (9):81–100, 1993.
  * [5] J. Fakcharoenphol, S. Rao, and K. Talwar.  A tight bound on approximating arbitrary metrics by tree metrics.  Journal of Computer and System Sciences, (69):485–497, 2004.
  * [6] D. Fasino, A. Tonetto, and F. Tudisco.  Generating large scale-free networks with the chung–lu random graph model.  2019\.
  * [7] J. McAuley and J. Leskovec.  Learning to discover social circles in ego networks, 2012.
  * [8] K. Mehlhorn and A. Tsakalidis.  Dynamic interpolation search.  In Automata, Languages and Programming, pages 424–434. Springer-Verlag, 1985.
  * [9] J. M. Mellor-Crummey and M. L. Scott.  Algorithms for scalable synchronization on shared-memory multiprocessors.  ACM Transactions on Computer Systems (TOCS), 9(1):21–65, 1991.
  * [10] R. K. Treiber.  Systems programming: Coping with parallelism.  International Business Machines Incorporated, Thomas J. Watson Research …, 1986.
  * [11] M. R. Tyler Crain, Vincent Gramoli.  A contention-friendly, non-blocking skip list.  2012\.


[1]Asma<EMAIL_ADDRESS>
# Suicidal Ideation Detection on Social Media: A Review of Machine Learning
Methods

Asma Abdulsalam  Department of Computer Science,Faculty of Computing and
Information Technology,King AbdulAziz University,Jeddah,Saudi Arabia  Areej
Alhothali  Department of Computer Science,Faculty of Computing and Information
Technology,King AbdulAziz University,Jeddah,Saudi Arabia

###### Abstract

Social media platforms have transformed traditional communication methods by
allowing users worldwide to communicate instantly, openly, and frequently.
People use social media to express their opinion and share their personal
stories and struggles. Negative feelings that express hardship, thoughts of
death, and self-harm are widespread in social media, especially among young
generations. Therefore, using social media to detect and identify suicidal
ideation will help provide proper intervention that will eventually dissuade
others from self-harming and committing suicide and prevent the spread of
suicidal ideations on social media. Many studies have been carried out to
identify suicidal ideation and behaviors in social media. This paper presents
a comprehensive summary of current research efforts to detect suicidal
ideation using machine learning algorithms on social media. This review 24
studies investigating the feasibility of social media usage for suicidal
ideation detection is intended to facilitate further research in the field and
will be a beneficial resource for researchers engaged in suicidal text
classification.

## Introduction

Millions of individuals regularly use social media such as chat rooms,
blogging websites, and social networking platforms, with $3.96$ billion people
actively utilizing the internet [1]. Facebook, Twitter, Snapchat, and other
social media networking sites allow users to share material and interact with
others. Many users prefer to utilize social media networks to share their
thoughts and emotions, and their daily experiences, problems, and issues.
Suicidal ideation, death, and self-harming thoughts are among the most widely
discussed themes on social media.

Suicide is described as a person’s deliberate attempt to take their own life
[2]. Suicide is a multifaceted occurrence that results from a complex
interaction of biological, psychological, social, cultural, and spiritual
variables [3]. Suicide is a manifestation of underlying suffering caused by a
mix of events, including underlying mental diseases that generate
psychological pain [4]. Suicide ideation, suicide planning, and suicide
attempts are three types of suicidal behavior [2, 3, 4]. Suicide ideation
refers to a person’s ideas or intentions to end their life without actually
trying to do so. In contrast, a suicide plan is a specific technique a person
can use to end their life, and a suicide attempt is an act of self-harm that
results in death with the intended purpose being to die [2, 3, 4].

Suicide has ramifications for people, families, communities, and even
countries [4]. Suicide is the second largest cause of mortality among young
people, killing more people than diabetes, liver disease, stroke, or infection
[5]. More than 40% of individuals who seek primary care are reluctant to
address their depressive symptoms because of the stigma associated with mental
disorders. Suicidal thoughts and acts necessitate quick intervention, and
there is no reliable approach for managing, assessing, or preventing suicide
[5]. Traditional suicide ideation detection approaches rely on the knowledge
of psychologists and self-reported questionnaires [4]. Patient Health
Questionaire-9 (PHQ-9) and Columbia Suicide Severity Rating Scale (C-SSRS) are
two examples of public forum questionnaires that can screen for suicide and
identify depressive symptoms [5]. These approaches are effective and quick,
but they are subject to false negatives due to participant concealment. They
are also difficult to carry out over a lengthy period or on a very large scale
[5].

The task of identifying suicidality has attracted researchers in different
fields to investigate linguistic and psychological signs and other factors
that aid in diagnosing and identifying individuals with suicidal thoughts [4].
Social media posts provide a valuable source of information about individuals’
lives and their emotional and psychological states. For various reasons, many
individuals are unable to share their personal stories and express their
emotions in real life and instead choose to write blogs about their feelings
or suicide plans. Unfortunately, these suicide posts are often either
overlooked or ignored. This information can help to perform screening of
suicidality on a wide scale.

To detect suicidal individuals or who may have suicidal thoughts from their
tweets or blogs is very important, because early detection of suicidal people
could save many lives even though people who know that they are suffering from
suicidal thoughts may not get the appropriate treatment for many reasons.
Therefore, using a suicidal detection system could help many people and can
have a significant impact on their treatments.

The studies reviewed in this paper have examined social media content to
detect automatically suicidal ideation and behaviors. This article presents a
detailed overview of current research efforts in social media platforms that
use machine learning techniques to detect and identify suicidal ideation.
Several specific tasks and datasets are introduced and summarized according to
their practice. This article is intended for researchers who are interested in
developing applications that leverage text classification methods or suicidal
text classification. Also, to aid future study in the field and investigate
the feasibility of using social media to detect suicidal ideation. In this
research, the terms suicidal ideation, suicidal thoughts, and suicidality will
be used interchangeably. The contributions of our survey are summarized as
follows.

  * •

To the best of our knowledge, this is the first comprehensive review of
research into suicidal ideation detection using social media, including the
datasets that have been constructed and the methods employed from a machine
learning perspective.

  * •

We introduce and discuss classical and modern machine learning techniques on
different social media platforms and identify the best performing algorithm in
the context of the platform used in the study and how the dataset was
collected and annotated.

The literature search was performed through two databases for retrieving
scientific works: Scopus and Google Scholar. These databases include most of
the important papers in the area. The inclusion and exclusion criteria is
shown in fig 1 and can be summarized as follow. First, we included all papers
from 2014 to 2020 that contain the following keywords in its title: (suicide
OR suicidal OR suicidality OR suicide-related OR behavior OR ideation OR
intent OR risk OR psychiatric stressors OR expressions OR detection OR
detecting OR prediction) AND (deep OR machine OR learning OR algorithms OR
classification OR feature selection OR social media OR Twitter OR Facebook OR
Reddit OR Microblogs OR online communities). We then excluded out of scope
studies, thesis, secondary studies (e.g., surveys, systematic literature
reviews), and papers that had been written in a language other than English.

Figure 1: Flow diagram for a systematic reviews which included searches of
databases

The remainder of this article is organized as follows. Sections 1 and 1.1
detail the dataset collection procedures followed in the current research
studies and annotation techniques. Section 2 covers details of feature
extraction and algorithms used in the classification process. Section 3
provides a summary and discussion of the current research in the field.
Section 4 gives a conclusion of the survey paper.

##  1 Datasets

Users’ posts and interactions on social media platforms provide a wealth of
information for many researchers. Several sets of information, social media
platforms, and data sources were investigated to identify suicide-related
posts. This section gives an overview of current practice in the detection of
suicidal thoughts. In particular, an overview of types of data (i.e.,
linguistic/semantic, psycholinguistic, metadata or interaction data), the
language of the content (i.e., English, Chinese, and others), social media
platforms (i.e., Twitter, Reddit) data collection procedure (including search
keywords) and annotation scheme (i.e., number of classes) are given.

###  1.1 Type of Data

The studies surveyed in this paper examined several types of data categorized
into linguistic data, psycholinguistic data, metadata, and interaction data
[6]. Linguistic data was central to a series of NLP applications and
includes,for example, authorship attribution and forensic linguistics, gender
detection, and personality type detection [7]. Many studies show that the
linguistic and semantic features of social media users’ posts could help
indicate and clarify the mental state of the poster [8]. Mapping words often
obtain psycholinguistic features words into pre-defined psychological and
affective categories. The Linguistic Inquiry Word Count (LIWC) is one of the
most widely used psycholinguistic dictionaries in related NLP tasks[6]. The
LIWC consists of a large number of words along with different categories
started by two effective classes (positive, negative emotion) and more than
$80$ categories (e.g., anxiety, anger, sadness) [7, 6]. The LIWC has been used
in different domains such as social relations and mental health [7, 9].
Metadata features are pieces of information that describe digital data, which
can be account metadata or post/message metadata. Account metadata are the
data that describe the account, such as the owner’s name, profile information,
biography, and location. Post or message metadata are the data that describe
posts, such as the author, location, likes, number of shares, date/time,
links, and hashtags. Interaction data are associated with what users produce
in their daily interactions and communication in the digital world [10].
Several interactive features were examined, which include user temporal
posting patterns.

###  1.2 Languages of Textual Data

Authors have examined the mental state of social media users in many
languages.The majority of papers in the field are written in English [11, 8,
12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23]. The Chinese language was the
second most used language in the published studies [24, 25, 26, 27]. Further
studies were completed in Spanish [28, 29], and Russian [30], and even
Japanese [21] and Filipino or Taglish [1].The distribution of articles over
the platforms can be observed in Figure 2. As Figure 2 shows, English-language
articles predominate; of 24 articles, only seven used other languages.

Figure 2: Distribution of Articles over Languages.

###  1.3 Platforms

A wide range of social media platforms has been used for creating suicidality
detection datasets, with most of the studies using Twitter [31, 8, 12].
Twitter is a free social media broadcast site, and any registered user can
communicate with other users using 140 characters each time they post. Other
social media platforms have been the subject of similar research, including
Reddit [9, 25]. Reddit is a community-driven platform for commenting,
submitting, and rating links and text posts [32]. The Chinese microblog Weibo
has been studied [23, 17, 19, 18], Weibo also has a limit of 140 characters in
a post and has witnessed exponential growth, particularly in China [33]. In
Russia, a popular platform is Vkontakte in which users can create groups and
invite users to join them, discuss different topics, and meet other users [24,
34]. Figure 3 shows the distribution of articles over these platforms and
shows that, Twitter is the most used platform in studying suicidal posts.

Figure 3: Distribution of Articles over Platforms

###  1.4 Data Collection Procedure and Annotation schema

Several datasets were developed for suicidality detection that vary in size,
target (individual tweets or user histories), and data collection procedures.
Sawhney et al. used the Twitter timeline dataset of user data [31] to filter
$32,558$ user profiles with a mean number of tweets history of $748$ tweets. A
lexicon of $143$ suicidal phrases was used and then annotated by two clinical
psychology students as Suicidal Intent (SI) Present or Suicidal Intent (SI)
Absent [8]. O’dea et al. gathered $1,820$ tweets to study suicide-related
posts using English words or phrases consistent with the vernacular of
suicidal ideation. Each tweet was then classified by three mental health
researchers and two computer scientists [12]. Valeriano et al. collected
$2,068$ Spanish tweets by translating a list of English keywords used to
express a suicidal tendency to Spanish and then annotated the tweets with the
help of bilingual assistants [13]. Burnap et al. collected four million tweets
using suicidal keywords extracted from four well-known websites dedicated to
suicide prevention and support. The dataset was then annotated using a crowd-
sourcing online service and randomly sampled with $800$ suicidal tweets and
$200$ undirect suicidal ideation tweets [14]. Vioules et al. proposed an
approach to detecting suicidal thoughts to identify sudden changes in users’
online behavior by analyzing users’ behavioral and textual features. They
collected $5,446$ tweets using special key phrases obtained from a generated
list of suicide risk factors and warning signs. Eight researchers and a mental
health professional then manually annotated tweets [15].

Moulahi et al. exploited a list of key phrases generated from the American
Psychological Association (APA) list of suicide risk factors and keywords from
the American Association of Suicidology (AAS) list of warning signs. They only
considered users’ accounts that show in their online behavior serious suicide
symptoms, collect $29,887$ tweets from $60$ users. To avoid over-fitting, they
included $60$ normal accounts that used the same keywords [16]. Sawhney et al.
extracted $4,314$ posts from four well-known Suicide web forums to create a
suicidal language. Also, user posts with ’suicide’ tag from social media sites
such as Tumblr and Reddit were included to the collection. As a result, 300
posts were chosen from each suicide forum, and 2000 posts were chosen at
random from Tumblr and Reddit. After manually annotating these posts and
utilizing Term Frequency/Inverse Document Frequency (TF-IDF) to determine the
most often occurring terms, a list of 108 words/phrases associated with
Suicidal Intent was created. To validate the model’s performance in terms of
various elements, three datasets were constructed using different strategies:
($2726$ suicidal, $9160$ non-suicidal) using words/phrases. The second dataset
followed the same method and users whose tweets or posts were classified as
suicidal but didn’t include any hashtags associated with suicidal ideation,
and the last dataset used both datasets with no overlap. To assess the
effectiveness of the proposed methodology, three clinical psychology students
annotated each of the three datasets, which included suicidal and nonsuicidal
tweets. [35]. Astoveza et al. gathered a dataset using keywords of potential
warning signs and hints from psychological associations and online
organizations and keywords used in similar studies. The chosen keywords were
translated to the Filipino language to gather $3,055$ English and $2,119$
Filipino or Taglish tweets and annotated by trained psychologists and a
resident guidance counselor [1].

Shah et al.used Reddit to gather $7,098$ English posts. The dataset consisted
of $3,549$ user-posts containing suicidal ideation taken from a sub-Reddit
called SuicideWatch and labeled ”1”. A further $3,549$ pieces of data of
different popular Reddit posts that do not contain suicidal ideation and
labeled ”0”are also included [9]. The dataset consists of $594$ suicidal
ideation tweets out of $10,288$ tweets using a keyword filtering technique
including suicidal words and phrases such as, e.g., suicide, die, and end my
life. The text is then manually annotated to Suicide, Nonsuicide text [28].
Questionnaires are also considered textual data sources. Jain et al. used two
datasets, one from questionnaires and the second is from Reddit and Twitter
and a labeled dataset from Kaggle [11].

In2012,a Chinese college student, Zoufan, hung herself after leaving a suicide
note on Weibo, the largest open social media platform in China. People still
paid attention and left messages below her last blog, with some of the
messages reveal suicidal thoughts.Y. Huang et al. created a dataset by sorting
through $65,352$ messages below Zoufan’s last blog entry. Three experts who
specializing in psychology and suicidal behavior labeled $8,548$ blogs as
suicide and $10,000$ as non-suicide blogs [19]. A further, another dataset
used Zoufan’s blog and crawled $5,000$ Chinese posts from the Weibo website to
be used in Dual attention mechanism (DAM) to improve the performance of social
media based suicide risk detection [17]. Huang et al. identified $53$ users
who posted suicidal content on Weibo before their deaths and collected more
than $30,000$ posts, in addition to another, they also collected $600,000$
posts collected from $1,000$ thousand random non-suicidal users. The
researchers curated all suicidal users’ posts and obtained $614$ suicidal
posts. They then randomly sampled $6,140$ posts from the set of non-suicidal
users for a total of $6,754$ posts. After filtering some blank posts, they
obtained $6,704$ posts [18].

Researchers in another study acquired messages from VKontakte, Europe’s
second-largest social network after Facebook. They gathered $35,000$ Russian
messages from individuals diagnosed with depression (i.e., chronic, severe,
and persistent), and $50,000$ postings with unfavorable sentiments on other
topics were obtained to generate a balanced dataset [24].

Figure 4 shows the frequency usage of each annotation type and shows that the
data is annotated manually in most studies. They could be students of clinical
psychology or mental health researchers. Two studies used a website like
Kaggle to annotate their data. In one study, the data were annotated based on
the source [9], further explained in the next section.

Figure 4: Annotation Scheme used by Articles

###  1.5 Annotation Classes

Data annotation is an important aspect of data preparation because supervised
machine learning models spot patterns in annotated data. Each example in the
dataset should be labeled with one of the pre-defined categories to train
machine learning models to distinguish between potential suicide ideation and
nonsuicide ideation posts.

The problem of suicidal ideation detection is often formulated in binary and
multiclass classification. Binary suicidal classification determines whether a
given post contains suicidal thoughts or a user is at risk of suicide. Figure
5 shows that in most studies data is labeled as a binary classification of 0
(non-suicidal), or 1(suicidal) [9, 8, 19, 25, 26, 30, 35, 1, 13]. Shah et al.
labeled data according to the source assigning the label ”1” to the data that
are from Suicide-Watch and ”0” to the data from other sub-Reddit forums [9].
Jain et al. labeled the second dataset (Reddit and Twitter) using two classes,
risky and non-risky and annotated the first dataset (questionnaire) using five
levels of depression severity [11]. Multiclass classification tasks are
formulated based on the assumption that each sample is assigned to several
pre-defined classes. For example, O’Dea et al. classified posts into three
levels: ”Strongly concerning,” ”Possibly concerning,” and ”Safe to ignore.”
Annotators were instructed to choose only one of the three levels and, if in
doubt, to choose ”Safe to ignore” [12]. Four levels of classification were
used by Vioules et al. ranging from normal to suicidal [15]. Burnap et al.
classified Twitter text into seven classes, including suicidal intent, or
other suicide-related topics such as suicide campaigning, support or
prevention of suicidality, reporting of suicide, flippant reference to
suicide, or none of the above [14, 22, 29].

Figure 5: Number of class used in Classification in each article

##  2 Methodology

The classification of suicidal-related posts or blogs aims to determine
whether the user has a suicidal tendency or not. Machine learning methods and
other techniques have also been applied to solve this problem. The
classification method often requires employing feature extraction/ text
representation technique before employing machine and deep learning models.
Figure 6 shows a general procedure used by most studies discussed in this
article. Step one was data collection and involved constructing a dataset
using one or more social media platforms. The second step, annotation,
involved labeling datasets using different techniques, as discussed in section
1.5. The third step, is feature extraction, is applied before employing
machine and deep learning models.

Figure 6: Architecture of Suicide Detection Methodology

###  2.1 Feature extraction

Many techniques have been used to extract features from social media posts to
identify whether they reflect suicidal thoughts or not.

TF-IDF matrixes were used for textual features to reflect the importance of
words to distinguish between suicidal and non-suicidal posts [11, 12, 24, 13,
19, 20, 35]. N-gram features were also utilized to find the probability of $n$
words in a given document:in this case utilized to process blog content and
identify terms in the blog corpus. [19, 35, 1]. N-grams are known as a base
feature for sentiment analysis of tweets. Due to the character limitation in
Twitter posts, it leads to choosing short N-grams [15].

Some studies use textual features in addition to psycholinguistic features
obtained from (LIWC)[6]. LIWC is also used to count the frequency of a
specific word, and LIWC has categories to identify syntactical elements
(e.g.,nouns, pronouns, verbs, and adverbs) [21, 19, 35].

Computational features and linguistic features (TF-IDF, N-gram and 30 best
features for LIWC) were used by Shah et al. to propose a hybrid method [9].
Several features were extracted, including statistical, linguistic, syntactic,
topic features, and word embedding. These features were used to detect online
suicidal users through their online content [28], including the language of
the tweet and the emotional historic spectrum feature in a time-aware manner
[8]. Important data that can give information about users on social networks
include users’ behavior (daily activities, social network size, etc.) [15].
Profile, text content such as messages, publications, and comments are used to
extract features of a general-purpose classification [24]. A combination of
textual features such as BoW or N-grams and word embedding with social network
and psychological features include lexical, behavioral, and sentiment
analysis. These and other features can be mapped to social media context using
certain signs, symptoms, and image-based features [21].

###  2.2 Classification Methods

Many studies have utilized machine classification techniques to study and
analyze the content users generate on social media. First, researchers have
focused on three strategies to tackle the problem of suicide detection.
Researchers formulated the problem as a time-series problem to detect changes
in users’ behavior. Second, the task is see as a text classification
(supervised) problem to identify linguistic connotations associated with
suicide.Third, the problem is approached as unsupervised (clustering) to group
examples of user posts into different groups based on their features.

Several supervised algorithms were examined in the literature, including
Support Vector Mchine (SVM) [9, 11, 21], NB [9, 29, 20], K-nearest neighbor
algorithm (KNN) [9, 15], Logistic Regression (LR) [11, 12, 21], Decision Tree
(DT) classifier [11, 23], and Extreme Gradient Boost (XGBoost) algorithm [11,
30, 28]. Time-aware Long Short-Term MemoryLSTM (T-LSTM) was used to propose
Suicidality Assessment Time-Aware Temporal Network (STATENet)[8].
Convolutional Neural networks (CNN) and Recurrent Neural Networks (RNN) were
also used to classify suicidal posts.

####  2.2.1 Temporal Behavior Problem

The Multi-Layer Perceptron classifier was also used with $1,500$ best features
out of $5,000$ features. The classifier was able to classify $90.2\%$ of the
non-risky tweets and only misclassified $9.0\%$. However, only $65.1\%$ of the
risky tweets were classified correctly [1]. A study in the Japanese language
used ordinary least squares (OLS) regression model to study the relationship
between suicide cases and the suicidal keyword “kietai” (“I want to
disappear”). The researchers also studied the linguistic context changes at
different hours of the day for the suicidal keyword. They found a clear
pattern with the use of suicidal keywords peaking from 1 am to 5 am. This
trend showed a positive correlation among suicide deaths for people aged 15 to
44 years but negative among adults over 45 years old. Nighttime tweets showed
a significant relationship between self-disgust words and words that indicate
direct suicidal ideation [27].

A probabilistic framework based on Conditional Random Fields (CRF) was used by
Moulahi et al. to track suicidal ideation. They studied mental states as a
sequence of events, considering the context and users’ online activities that
may lead to suicide. They evaluated their approach by comparing it with other
machine learning methods: SVM, NB, J48, RF, and multilayer perceptron.
Different CRF configurations were run, and no sequences of observations were
considered to compare their approach. The researchers noted that both CRF
configurations outperform in terms of all the testing criteria average
Precision, recall, and F1-score measures. Their approach had the best
performance Precision of $81.6\%$, recall $75.2\%$, and F1-score $71.1$ [16].
The Firefly algorithm is a metaheuristic-based approach that seeks to increase
classifier accuracy while attempting to reduce the amount of features in order
to reduce computational cost, complexity, and redundancy. Sawhney et al. used
the Binary Firefly Algorithm (BFA) which is a discrete-space modification of
the firefly algorithm used for feature selection. They used firefly algorithm
as a wrapper over the four classifiers (Random Forest (RF), SVM, LR, and
XGBoost). RF and BFA combined gave the highest performance with $89.2\%$
Precision, $87.4\%$ recall, and $88.3\%$ F1-score[35]

Vioules et al. detect the change in the data streams by passing textual and
behavior features to a martingale framework. They needed two datasets
sufficiently large annotated set and another smaller set of selected Twitter
users to study their history. They found that the two-step classification
performed well in the test set. They reached $82.9\%$ precision, $81.7\%$ for
recall, and F1-score[15].

A DAM finds the correlation between text and image from the same post and
better detects the user’s implicit suicide risk. They have then compared their
model with other five models: NB, SVM with TF-IDF features, Long Short-Term
Memory (LSTM), CNN, and Species Distribution Models (SDM) deep learning model
based on layered attention and suicide-oriented word embeddings. Experiments
showed that the DAM performed better than most suicide risk detection models
and obtained competitive results on the proposed dataset. The model performed
better when people’s posts contained images [17].

####  2.2.2 Text Classification Problems

Classification algorithms such as SVM and LR were examined to identify a tweet
with a tendency to suicide from a non-suicidal tweet [13]. Narynov et al. used
supervised (Gradient Boosting, RF) and unsupervised algorithms (K-means) and
tested them with TF-IDF and Word2Vec. They found that RF with TF-IDF had the
best performance with $96\%$ accuracy [24].

Six supervised learning classifiers were used: SVM, RF, gradient boost
decision tree (GBDT) for classification, XGBoost, and feed-forward neural
network with several sets of features (statistics, POS counts, LIWC features,
TF-IDF vectors, and topic probability features) and found that combining more
features increases the performance of all methods. RF gained better
performance than most models except for the metric of Precision, in which the
neural network model achieves slightly better results [28]. Different
classifiers were also examined by X. Huang et al., including SVM, NB, LR, J48
classifier, RF, and Sequential minimal optimization (SMO) with three N-gram
features. SVM classifier achieves the best performance in comparison with
other classifiers, with an F1-score of $68.3\%$, a Precision of $78.9\%$, a
recall of $60.3\%$, and accuracy over $94.0\%$ [18].

Different machine learning algorithms and ensemble approaches have been used,
such as NB, decision trees, multinomial NB, LR, RF, resulting in $98.5\%$
accuracy, $98.7\%$ Precision, and $98.2\%$ recall yielded using RF that gave
the best performance [20].

Two machine learning algorithms were used (SVM, DT) by Y. Huang et al. to
build a classification model with three features sets (automated machine
learning dictionary, Chinese suicide dictionary, and Simplified Chinese Micro-
Blog Word Count (SCMBWC)). Each feature set was used with two machine learning
algorithms separately to generate six detection results. Those were input to
an LR. It has been found that SVM with feature set extracted using automated
machine learning dictionary from real blog data-driven by N-gram had the best
performance [19].

Tadesse et al. combined two models, LSTM and CNN, to explore the potential of
each algorithm separated and their combined model applied in classifying the
sentences with suicidal and non-suicidal content. The proposed algorithm was
compared with CNN, LSTM separated and other machine learning classifiers such
as SVM, NB, RF, and XGBoost. They found the proposed model improved the
accuracy with $93.8\%$, F1-score $93.4\%$, recall $94.1\%$ and Precision
$93.2\%$ [26].

SVM and NB were incorporated as an ensemble approach known as Rotation Forest
(RF). They tested the RF approach with three classifiers: DT, SVM, and NB.
They reached $69.0\%$ for F1-score, the Precision performance of $64.4\%$, and
recall of $74.4\%$[14]. Interestingly, another study [22] used four machine
classifiers (DT, NB, RF, and SVM) on the same dataset [14] and DT had the best
performance with an F1-score of $87.9\%$ and $79.0\%$ accuracy for a
multiclass dataset. A third study completed by Chiroma et al. made the same
dataset [14] with the same pre-processing technique. The prism algorithm was
first introduced in 1987 by Cendrowska [36]. It can select attributes based on
their importance to a specific class [29]. They compared the performance of
the Prism algorithm against the popular machine learning algorithms (SVM, DT,
NB, and RF). They found that the Prism algorithm had $84\%$ Precision, recall,
and F1-score, which is the best performance compared to all other classifiers
in all measures[29].

Sentiment dictionaries were adopted into Latent Dirichlet Allocation (LDA) by
X. Huang et al. and evaluated against traditional LDA on a different number of
topics (100- 1000). Also, they trained and tested different classifiers SVM,
J48 classifier, LR, random tree, RF, and decision table. They found the best
performing algorithm was the J48 classifier with an accuracy of 94.3,
Precision $80.2\%$, recall $48.3\%$, and F1-score $60.3\%$ [23].

Vader sentiment analysis was used by Rajesh Kumar et al. to give a score for
each word with different classifiers such as NB, RF, XGBoost, and logistic
regression. Vader sentiment analysis helped separate the sentence to
distinguish the sentences into positive, or neutral. They achieved $99.6\%$
accuracy using the RF method [30]. CNN to select suicide-related tweets and
RNN to extract stressors were used by Du et al. to build an automatic
psychiatric stressors binary classifier. They compare their proposed model
with other machine/deep learning approaches SVM, ET, RF, LR, Bi-LSTM. CNN had
the highest recall: $90\%$ and F1-score:$83\%$ [25].The studies are summarized
in Table 1.

Table 1: Results of Each Study Included in This Review. * indicates best performing algorithm. Acc,P,R, and F1 are abbreviation for Accuracy, Precision, Recall, and F1-score, respectively.SNPSY: social networks and psychological features. Ref/Year. | Source | language | N(Posts) | Features | Algorithms | Performance
---|---|---|---|---|---|---
[9]/2020 | Reddit | English |  7098 post |  Unigram, Bigram, Trigram, TF-IDF, LIWC |  NB*, SVM, KNN, RF |
Acc:73.6%
P:70.5%
R:89.7%
F1:76.7%
[11]/2019 |  Twitter Reddit qustinayr | English | - | TF-IDF |  LR*, DT, XGBoost, SVM | 86.5
[8]/2020 | Twitter | English |  34,306 tweets |  LIWC, N-grams , POS |  RF, LSTM, SDM, CNN, STATENet* |
Acc:85.1%
R:81.0%
F1:79.9%
[24]/2019 | VKontakte | Russian |  85,000 posts |  TF-IDF, Word2Vec |  GB*, RF, k-means |
P:96.0%
R:95.0%
F1:95.0%
[21]/2020 | Twitter | Spanish |  1200 users |  LIWC, BoW, N-gram, SNPSY, images |  RF, LR, MLP, SVM* , CNN |
Acc:86.0%
P:91.0%
R:81.0%
F1:86.0%
[20]/2020 | Twitter | English |  4266 tweets |  TF-IDF, BoW |  NB, DT, SVM, RF*, LR, and others |  Acc:98.5% P:98.7% R:98.2%
[13]/2020 | Twitter | Spanish |  2068 tweets |  TF-IDF, Word2Vec. |  SVM, LR* |  Acc:79.0% P:79.0% R:79.0% F1:79.0%
[23]/2015 | Weibo | Chinese |  7978 blogs |  Word2Vec, POS, LDA, meta features, N-gram |  SVM, J48*, LR, RT, RF, DT |  Acc:94.3% P:80.2% R:48.3% F1:60.3%
[22]/2018 | Twitter | English |  1000 tweets |  POS, BOW, TF-IDF |  DT, NB, RF, SVM |  P:86.4% R:89% F1:87.9%
[29]/2018 | Twitter | English |  1000 tweets |  TF-IDF, N-gram, POS, BOW, LIWC |  Prism algorithm*, DT, NB, RF, SVM |  P:84.0% R:84.0% F1:84.0%
[30]/2020 | Twitter | English |  54720 tweets |  statistical, BOW, Word frequency |  NB, RF*, LR XGBoost |  P:99.6% R:99.1% F1:99.8%
[17]/2020 | Weibo | Chinese |  5,000 users | TF-IDF |  SDM, CNN, LSTM, NB, SVM, DAM* |
Acc:91.8%
F1:91.5%
[12]/2015 | Twitter | English |  1820 tweets |  freq, TF-IDF, filter | SVM* ,LR |  Acc:76.0% P:80.0% R:53.0% F1:64.0%
[19]/2019 | Weibo | Chinese |  18548 blogs |  N-gram, TF-IDF, LIWC | SVM*, DT |  P:89.0% R:88.0% F1:88.0%
[28]/2018 |  Reddit Twitter | English |  10882 tweets |  Statistical, POS, LIWC, Word2Vec, LDA |  SVM, RF*, GBDT, LSTM and others |  Acc: 96.4% P: 96.4% R: 99.2% F1: 96.5%
[26]/2019 | Reddit | English |  7201 posts |  TF-IDF, BoW, Statistical, Word2Vec |  SVM, NB, RF, XGBoost, LSTM, CNN, LSTM-CNN* |  Acc:93.8% P:93.2% R:94.1% F1:93.4%
[15]/ 2018 | Twitter | English |  5,446 tweets |  N-grams, symptoms, pronouns, swear |  NB, SMO , J48, LR, RF, and others |  P:83.0% R:82.0% F1:82.0%
[25]/2017 | Twitter | English |  6,263 tweets |  GloVe Twitter embedding |  CNN*, SVM, ET, RF,LR, Bi-LSTM |  P:78.0% R:88.0% F1:83.0%
[18]/2014 | Weibo | Chinese |  614 posts |  Unigram, Bigram, Trigram |  SVM*, NB, LR, J48, RF, SMO |  Acc:94.0% P:78.9% R:60.3% F1:68.3%
[16]/2017 | Twitter | English |  29887 tweets |  POS , sentiment (Psychological and emotional lexicon) , contextual |  SVM, NB, J48, RF, DARE* and others. |  P:81.6% R:75.2% F1:71.1%
[14]/2015 | Twitter | English |  1000 tweets |  TF-IDF, N-gram, POS, LIWC |  NB, SVM, J48, RF, NB+SVM* |  P:64.4% R:74.4% F1:69%
[27]/2020 | Twitter | Japanese |  2,889,190 tweets | - |  OLS regression |
[35]/2019 | Twitter | English |  36548 tweets |  Unigrams, Bigrams, LIWC, TF-IDF, POS, LDA unigrams, Sentiment Ratio, Emoji Sentiment |  RF, SVM, LR, RNN, LSTM, RF + BFA* and others |  P:89.2% R:87.4% F1:88.3%
[1]/2018 | Twitter |  English Filipino |  5,174 tweets |  Unigrams, Bigrams | MLP | Acc:89.2%

##  3 Discussion

Detecting suicidal people using new technologies is an important and very
active research area. Many studies have been developed to detect suicidal
ideation using different machine learning techniques automatically. Users’
posts and their interaction on different social media platforms is a novel
area of inquiry. This review paper discusses different studies that use
machine learning techniques on social media platforms to detect and identify
suicidal ideation. Both supervised and unsupervised machine learning
algorithms were used on different social media platforms such as Twitter,
Reddit, and other microblogs, adopting different languages such as English,
Chinese, Spanish, Japanese, and Russian, as shown in Table 1. Several datasets
have been developed using different procedures for suicidal ideation detection
purposes. The most commonly used procedures are keyword and suicidal phrases
extracted from suicide dictionaries or translated from other languages,
obtained from websites or lists of suicidal supports. A subset of studies
investigated metadata or interaction data, but most studies used linguistic
data. Metadata can show how and when a person is active, indicating a person’s
psychological state. Linguistic and sentiment analysis of users’ posts also
showed a good understanding of users’ emotional and mental health.

Most studies used and compared their work using popular machine learning
algorithms such as LR, DT, SVM, RF, and NB. In other studies, deep learning
algorithms like CNN and LSTMs were used. Figure 7 shows the frequency of
module usage. SVM and RF are the most used models, and followed by LR and NB.
The classification was most commonly observed in this review, with a small
number of studies using time-frame, and other studies using both. Most studies
classified posts, although some classified users. There are varying numbers of
classes or labels for both classification types to determine the level of
concern. Most studies used only two classes (suicidal and non-suicidal),
although some used additional classes for uncertainty, and other studies used
three to five levels. Different sets of features were used, including
statistical, syntactic, linguistic, and topic features. Most researchers use
different textual features such as TF-IDF, N-gram, and LIWC. Meta features
were also used, like posting time and social relationships. Methods with
automatic feature learning increased the performance of suicidal ideation
detection. Table 1 provides an overview of all studies mentioned in this
article.

Figure 7: Distribution of Most used Module

##  4 Conclusion

Using social media platforms to express experiences and feelings has created
new opportunities to analyze and detect suicidal ideation and other mental
disorders. The early detection of suicidal ideation on social media networks
will reduce suicide, provide an automatic and wide-ranging screening for
suicidal tendencies and prevent the spread of suicidal content in social
media. This survey investigates existing methods that use social media to
detect suicidal ideation using machine learning methods. A significant amount
of research has confirmed the effectiveness and feasibility of using social
media such as Twitter, Reddit, and Weibo for suicidal ideation detection. Most
studies have focused on suicidal ideation detection techniques for widely used
languages such as English, but less attention has been paid to Arabic. Thus,
with the growing number of social media users in the Arab region, research is
needed for Arabic suicidal ideation detection.

## References

  * [1] G. Astoveza, R. J. P. Obias, R. J. L. Palcon, R. L. Rodriguez, B. S. Fabito, and M. V. Octaviano, “Suicidal behavior detection on twitter using neural network,” in _TENCON 2018 - 2018 IEEE Region 10 Conference_ , 2018, pp. 0657–0662.
  * [2] M. K. Nock, G. Borges, E. J. Bromet, C. B. Cha, R. C. Kessler, and S. Lee, “Suicide and suicidal behavior,” _Epidemiologic reviews_ , vol. 30, no. 1, pp. 133–154, 2008.
  * [3] A. T. Beck, M. Kovacs, and A. Weissman, “Assessment of suicidal intention: the scale for suicide ideation.” _Journal of consulting and clinical psychology_ , vol. 47, no. 2, p. 343, 1979.
  * [4] D. Liu, Q. Fu, C. Wan, X. Liu, T. Jiang, G. Liao, X. Qiu, and R. Liu, “Suicidal ideation cause extraction from social texts,” _IEEE Access_ , vol. 8, pp. 169 333–169 351, 2020.
  * [5] A. N. Weber, M. Michail, A. Thompson, and J. G. Fiedorowicz, “Psychiatric emergencies: assessing and managing suicidal ideation,” _Medical Clinics_ , vol. 101, no. 3, pp. 553–571, 2017.
  * [6] Y. R. Tausczik and J. W. Pennebaker, “The psychological meaning of words: Liwc and computerized text analysis methods,” _Journal of Language and Social Psychology_ , vol. 29, no. 1, pp. 24–54, 2010. [Online]. Available: https://doi.org/10.1177/0261927X09351676
  * [7] C. Danescu-Niculescu-Mizil, M. Gamon, and S. Dumais, “Mark my words! linguistic style accommodation in social media,” in _Proceedings of the 20th international conference on World wide web_ , 2011, pp. 745–754.
  * [8] R. Sawhney, H. Joshi, S. Gandhi, and R. Shah, “A time-aware transformer based model for suicide ideation detection on social media,” in _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , 2020, pp. 7685–7697.
  * [9] F. M. Shah, F. Haque, R. U. Nur, S. Al Jahan, and Z. Mamud, “A hybridized feature extraction approach to suicidal ideation detection from social media post,” in _2020 IEEE Region 10 Symposium (TENSYMP)_. IEEE, 2020, pp. 985–988.
  * [10] B. Perez, M. Musolesi, and G. Stringhini, “You are your metadata: Identification and obfuscation of social media users using metadata information,” in _Twelfth International AAAI Conference on Web and Social Media_ , 2018.
  * [11] S. Jain, S. P. Narayan, R. K. Dewang, U. Bhartiya, N. Meena, and V. Kumar, “A machine learning based depression analysis and suicidal ideation detection system using questionnaires and twitter,” in _2019 IEEE Students Conference on Engineering and Systems (SCES)_. IEEE, 2019, pp. 1–6.
  * [12] B. O’dea, S. Wan, P. J. Batterham, A. L. Calear, C. Paris, and H. Christensen, “Detecting suicidality on twitter,” _Internet Interventions_ , vol. 2, no. 2, pp. 183–188, 2015.
  * [13] K. Valeriano, A. Condori-Larico, and J. Sulla-Torres, “Detection of suicidal intent in spanish language social networks using machine learning,” _International Journal of Advanced Computer Science and Applications_ , 2020\.
  * [14] P. Burnap, W. Colombo, and J. Scourfield, “Machine classification and analysis of suicide-related communication on twitter,” in _Proceedings of the 26th ACM conference on hypertext & social media_, 2015, pp. 75–84.
  * [15] M. J. Vioules, B. Moulahi, J. Azé, and S. Bringay, “Detection of suicide-related posts in twitter data streams,” _IBM Journal of Research and Development_ , vol. 62, no. 1, pp. 7–1, 2018.
  * [16] B. Moulahi, J. Azé, and S. Bringay, “Dare to care: a context-aware framework to track suicidal ideation on social media,” in _International Conference on Web Information Systems Engineering_. Springer, 2017, pp. 346–353.
  * [17] Y. Ma and Y. Cao, “Dual attention based suicide risk detection on social media,” in _2020 IEEE International Conference on Artificial Intelligence and Computer Applications (ICAICA)_. IEEE, 2020, pp. 637–640.
  * [18] X. Huang, L. Zhang, D. Chiu, T. Liu, X. Li, and T. Zhu, “Detecting suicidal ideation in chinese microblogs with psychological lexicons,” in _2014 IEEE 11th Intl Conf on Ubiquitous Intelligence and Computing and 2014 IEEE 11th Intl Conf on Autonomic and Trusted Computing and 2014 IEEE 14th Intl Conf on Scalable Computing and Communications and Its Associated Workshops_. IEEE, 2014, pp. 844–849.
  * [19] Y. Huang, X. Liu, and T. Zhu, “Suicidal ideation detection via social media analytics,” in _International Conference on Human Centered Computing_. Springer, 2019, pp. 166–174.
  * [20] S. T. Rabani, Q. R. Khan, and A. M. U. D. Khanday, “Detection of suicidal ideation on twitter using machine learning & ensemble approaches,” _Baghdad Science Journal_ , vol. 17, no. 4, pp. 1328–1328, 2020.
  * [21] D. Ramírez-Cifuentes, A. Freire, R. Baeza-Yates, J. Puntí, P. Medina-Bravo, D. A. Velazquez, J. M. Gonfaus, and J. Gonzàlez, “Detection of suicidal ideation on social media: multimodal, relational, and behavioral analysis,” _Journal of medical internet research_ , vol. 22, no. 7, p. e17758, 2020.
  * [22] F. Chiroma, H. Liu, and M. Cocea, “Text classification for suicide related tweets,” in _2018 International Conference on Machine Learning and Cybernetics (ICMLC)_ , vol. 2. IEEE, 2018, pp. 587–592.
  * [23] X. Huang, X. Li, T. Liu, D. Chiu, T. Zhu, and L. Zhang, “Topic model for identifying suicidal ideation in chinese microblog,” in _Proceedings of the 29th Pacific Asia Conference on Language, Information and Computation_ , 2015, pp. 553–562.
  * [24] S. NARYNOV, D. MUKHTARKHANULY, I. KERIMOV, and B. OMAROV, “Comparative analysis of supervised and unsupervised learning algorithms for online user content suicidal ideation detection,” _Journal of Theoretical and Applied Information Technology_ , vol. 97, no. 22, pp. 3304–3317, 2019.
  * [25] J. Du, Y. Zhang, J. Luo, Y. Jia, Q. Wei, C. Tao, and H. Xu, “Extracting psychiatric stressors for suicide from social media using deep learning,” _BMC medical informatics and decision making_ , vol. 18, no. 2, pp. 77–87, 2018.
  * [26] M. M. Tadesse, H. Lin, B. Xu, and L. Yang, “Detection of suicide ideation in social media forums using deep learning,” _Algorithms_ , vol. 13, no. 1, p. 7, 2020.
  * [27] R. A. Fahey, J. Boo, and M. Ueda, “Covariance in diurnal patterns of suicide-related expressions on twitter and recorded suicide deaths,” _Social Science & Medicine_, vol. 253, p. 112960, 2020.
  * [28] S. Ji, C. P. Yu, S.-f. Fung, S. Pan, and G. Long, “Supervised learning for suicidal ideation detection in online user content,” _Complexity_ , vol. 2018, 2018.
  * [29] F. Chiroma, H. Liu, and M. Cocea, “Suiciderelated text classification with prism algorithm,” in _2018 International Conference on Machine Learning and Cybernetics (ICMLC)_ , vol. 2. IEEE, 2018, pp. 575–580.
  * [30] E. Rajesh Kumar, K. Rama Rao, S. R. Nayak, and R. Chandra, “Suicidal ideation prediction in twitter data using machine learning techniques,” _Journal of Interdisciplinary Mathematics_ , vol. 23, no. 1, pp. 117–125, 2020.
  * [31] R. Mishra, P. Prakhar Sinha, R. Sawhney, D. Mahata, P. Mathur, and R. Ratn Shah, “SNAP-BATNET: Cascading author profiling and social network graphs for suicide ideation detection on social media,” in _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Student Research Workshop_. Minneapolis, Minnesota: Association for Computational Linguistics, Jun. 2019, pp. 147–156. [Online]. Available: https://www.aclweb.org/anthology/N19-3019
  * [32] P. Singer, F. Flöck, C. Meinhart, E. Zeitfogel, and M. Strohmaier, “Evolution of reddit: from the front page of the internet to a self-referential community?” in _Proceedings of the 23rd international conference on world wide web_ , 2014, pp. 517–522.
  * [33] L. Zhang and I. Pentina, “Motivations and usage patterns of weibo,” _Cyberpsychology, Behavior, and Social Networking_ , vol. 15, no. 6, pp. 312–317, 2012.
  * [34] D. Suleymanova, “Tatar groups in vkontakte: The interplay between ethnic and virtual identities on social networking sites,” _Digital Icons: Studies in Russian, Eurasian and Central European New Media_ , vol. 1, no. 2, pp. 37–55, 2009.
  * [35] R. Sawhney, R. R. Shah, V. Bhatia, C.-T. Lin, S. Aggarwal, and M. Prasad, “Exploring the impact of evolutionary computing based feature selection in suicidal ideation detection,” in _2019 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE)_ , 2019, pp. 1–6.
  * [36] J. Cendrowska, “Prism: An algorithm for inducing modular rules,” _International Journal of Man-Machine Studies_ , vol. 27, no. 4, pp. 349–370, 1987. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0020737387800032