# Classification of multivariate weakly-labelled time-series with attention

Surayez Rahman

Chang Wei Tan

Faculty of Information Technology  Monash University, Clayton Campus
Melbourne, Australia<EMAIL_ADDRESS>Faculty of Information
Technology  Monash University, Clayton Campus  Melbourne, Australia
<EMAIL_ADDRESS>
(06 November 2020)

###### Abstract

This research identifies a gap in weakly-labelled multivariate time-series
classification (TSC), where state-of-the-art TSC models do not perform well.
Weakly labelled time-series are time-series containing noise and significant
redundancies. In response to this gap, this paper proposes an approach of
exploiting context relevance of subsequences from previous subsequences to
improve classification accuracy. To achieve this, state-of-the-art Attention
algorithms are experimented in combination with the top CNN models for TSC
(FCN and ResNet), in an CNN-LSTM architecture. Attention is a popular strategy
for context extraction with exceptional performance in modern sequence-to-
sequence tasks. This paper shows how attention algorithms can be used for
improved weakly labelled TSC by evaluating models on a multivariate EEG time-
series dataset obtained using a commercial Emotiv headsets from participants
performing various activities while driving. These time-series are segmented
into subsequences and labelled to allow supervised TSC.

###### keywords:

Weakly-labelled, time-series, EEG, classification, CNN-LSTM, attention

##  1 Introduction

Time-series classification (TSC) is one of the most well-researched areas of
modern computer science [1, 5, 13, 18, 6, 8]. With the increase in time-series
data, accurate real-time TSC has become more important than ever. Despite the
huge progress in TSC over the last decade [1, 6, 8, 18, 5], most of these
works make assumptions that are unrealistic in the real-world setting. This
includes the assumption that the patterns (subsequences) that exist in the
time-series are of fixed equal lengths and are always correctly labelled [11].
However, in the real-world settings, these patterns can be of unequal length,
the class label of the pattern may contain errors and may only correspond to a
general section of the data [20]. For example, detecting distracted driving
using Electroencephalography (EEG) signal is a challenging task that violates
the unrealistic assumptions required by most TSC algorithms. Figure 1 shows an
EEG signal obtained from a participant during driving using the Emotiv Epoc+
EEG headset111https://www.emotiv.com/.

The 14-channel EEG signal can be interpreted as a time-series with 14
dimensions. A time-series analysis algorithm can be applied to analyse these
brain waves of the driver and detect the state of the driver to be either
focused or distracted. The different colors, as indicated in the legend,
represent the different activities performed by the driver during driving.
These activities are weakly labelled because it is challenging to correctly
label the driver’s transition from driving to using an iPad and vice versa.
Besides, these subsequences may also contain noise or redundant patterns that
are not related to the activity that the driver is performing at that point in
time. In fact, these redundant patterns may occur in different activity
patterns, which makes it challenging to differentiate the different
activities. Moreover, most real-world time-series dataset are highly
imbalanced where they contain more of the “normal” label. In this example,
there are more driving labels than distracted labels, thus making the task of
predicting a distracted activity extremely challenging.

Figure 1: Example of a weakly labelled EEG data

Quite simply, current state-of-the-art TSC algorithms are not capable of
handling weakly labelled time-series data. It makes sense because these models
were mainly designed for univariate TSC [1, 5]. While this strategy works well
for noise-free time-series with proper labelling, it fails to extract
discriminative features from weakly-labelled time-series as the signal-to-
noise ratio is low. Hence, most TSC algorithms [6, 13] cannot distinguish
between noise and useful data, especially when the class labels are highly
imbalanced.

This research aims to develop models that can accurately classify multivariate
weakly-labelled time-series. The approach explores the use of “attention”
mechanisms [2] to extract the contextual relevance of subsequences while
minimising the effect of noise and redundancies. The intuition behind this is
the chronological order of time-series, such that under realistic conditions,
they follow certain chronological trends. For example, for the task of
predicting distracted driving, the EEG data will typically show a driver who
is distracted at one moment has a higher probability of being distracted in
the next moment or show gradual changes, rather than a sudden spike in
concentration. Therefore, this research studies a set of deep learning models
that can classify each subsequence of the time-series and understand sequence
correlations using attention.

This paper is organised as follows. Section 2 discusses some background
information and related work in state-of-the-art TSC algorithms and introduces
existing attention strategies. Section 3 highlights the improvements made
using a CNN-LSTM architecture to implement the idea of contextual relevance.
It also describes the use of LSTM and attention mechanism. Finally, sections 4
describe the experimental setup with the weakly labelled time-series evaluated
in this research, followed by the discussion of improved results discussion
obtained as a result of this study.

##  2 Background and Related Work

The field of TSC has advanced significantly in the last decade, with each
algorithm significantly more accurate than its predecessors [1, 6, 5, 13, 8].
This section outlines the strength and limitations of some of the existing
state-of-the-art TSC algorithms used in this research and some of the popular
attention mechanisms [26, 2, 19] available till date.

###  2.1 State-of-the-art TSC Algorithms

Recently, Dempster et. al. [6] proposed ROCKET, a TSC classifier that achieves
state-of-the-art performances with a fraction of time of existing state-of-
the-art methods. ROCKET extract features from a time-series using a large
number of random convolutional kernels. These kernels have random length,
weights, bias, dilation, and padding, and when applied to a time-series
produce a feature map. Then the maximum value and the proportion of positive
values are computed from each feature map, producing two real-valued numbers
as features per kernel. With the default 10,000 kernels, ROCKET produces
20,000 features. These features are then used to train a ridge regression
classifier. ROCKET was found to be the most accurate TSC classifier compared
with other state-of-the-art algorithms such as HIVE-COTE [18], InceptionTime
[8] and ResNet [13] when bench-marked on the 85 TSC dataset [5]. Although
designed for univariate time-series, it can easily be adapted for multivariate
time-series with similar performance. This research applies ROCKET on weakly-
labelled multivariate time-series, however as shown later, ROCKET does not
perform well on weakly labelled time-series because it was designed using
clean, well-labelled time-series dataset [5]. In fact, none of the state-of-
the-art TSC algorithms are designed to classify weakly labelled time-series.

Ismail et. al. [13] reviewed a few deep-learning models for TSC. They found
two models that achieve state-of-the-art performances, Fully Convolutional
Network (FCN) and Residual Network (ResNet). ResNet was ranked as the most
accurate univariate TSC deep learning model bench-marked on 85 univariate TSC
dataset [5]. It is a convolutional neural network (CNN) that consists of three
residual blocks with three convolutional layers in each block, followed by a
global average pooling (GAP) layer and a softmax layer for classification.
With the help of residual connections, ResNet eliminates vanishing gradient
effect[10] in time-series where changes in data are very small (small learning
gradient).

On the other hand, when bench-marked on 12 multivariate TSC dataset, FCN is
the most accurate multivariate TSC deep learning model although it is not
significantly different from ResNet [13].  FCN consists of three 1D
convolutional blocks with batch normalisation and ReLU activation function to
learn features from the time series. Then, a GAP layer, applied to the last
convolutional block summarises the features which are then passed to a softmax
classifier for classification.

Perslev et. al. [22] proposed U-Time, a segmentation algorithm which uses a
feed-forward network (FFN) with convolutional encoders and decoders to segment
subsequences of long time-series. U-Time was designed for segmentation and
demonstrated high segmentation accuracy on a range of multivariate weakly
labelled time-series such as real-life sleep-staging EEG time-series. This
research is interested in the classification of weakly labelled data. Hence
U-Time is not being evaluated in this work and will be considered as future
work.

All of these state-of-the-art algorithms have shown superior performance on a
set of benchmark dataset [5]. Most of these datasets are not realistic, i.e.
they are properly labelled, equal in length and do not suffer from class
imbalanced. Hence these state-of-the-art algorithms are not capable of
handling weakly labelled dataset with unequal length and highly imbalanced,
such as the task of predicting distracted driving using EEG signal, as shown
in our experiments in Section 4. Therefore, this research identifies the lack
of suitable algorithms to classify multivariate weakly-labelled time-series
with high accuracy.

Among these existing state-of-the-art models, this research focuses on
improving FCN and ResNet because their neural network architecture allows them
to be upgraded. More specifically, this paper explores various attention
mechanisms to handle weakly labelled data.

###  2.2 Attention

The attention concept was initially proposed in 2015 by Bahdanau et. al. [2]
to utilise the contextual information in sequence-to-sequence (Seq-2-Seq)
problems such as natural language processing (NLP) [2]. Seq-2-Seq models are
neural networks that take in a sequence of data as input and output another
sequence, such as language translation. These models typically consist of an
encoder and a decoder [3]. The encoder tries to summarise the input sequence
into a concise feature vector while the decoder maps the feature vector to an
output sequence. Using the example of language translation, the input can be
an English sentence and the output can be a French sentence. When input
sequences are long, generating the contextual information becomes a
bottleneck. To prevent this problem, attention strategy was introduced to
decouple the encoders and decoders, and allow long sentences to be translated
accurately using alignment functions.

####  2.2.1 Attention by Luong and Bahdanau et. al.

Since the introduction of attention, several other designs were introduced by
a number of subsequent papers, mostly focused on language translation. In
2015, Luong et. al [19] proposed an attention model in a Long Short-Term
Memory (LSTM) encoder-decoder architecture. Their model was based on the
intuition to keep track of all the words in a sentence so that the relevance
of each word to the current word can be used to make better translation
decisions. Luongs’ attention was an improvement to the previous strategy by
Bahdanau et. al. [2], because it uses multiple calculations to derive the
context. However, the benefit of the model by Bahdanau et. al. [2] was that it
extracted a bi-directional sequence context.

####  2.2.2 Self-Attention and Multi-Head Attention

Later in 2017, a state-of-the-art attention model, known as Multi-Head
Attention, was proposed by Vaswani et. al. [26] To learn the context in
sequences, this model builds correlation functions using a neural network
model known as Self-Attention and use a complex multi-layered encoder-decoder
architecture, known as the Transformer [26]. This model can focus on multiple
sections of sequences in parallel to extract context.

Since the introduction of these attention models, they have been adopted in
various research across different problem domains [27]. In this research, some
of the above-mentioned attention strategies are explored in combination with
the existing CNN TSC algorithms to demonstrate significant improvement in TSC.

##  3 Methodology

This section outlines the approach taken in this study to improve existing TSC
models on weakly labelled data. The first sub-section discusses the intuition
behind the use of context-relevance to improve weakly labelled TSC. Then the
following sub-sections introduce the CNN-LSTM architecture implemented in this
research and explain the implementation of existing attention strategies for
complex context extraction in TSC. The final sub-section lists all the neural
network models created in this study.

###  3.1 Context relevance of subsequences

Relevance information in time-series can be described as the additional
context between subsequences. For example, if a subsequence currently being
classified has the same class as the previous two subsequences, they are
likely to have similar patterns that can help to improve TSC. When the models
incorrectly learn the features from subsequences due to noise and erroneous
data, this additional context can help the models identify similar sections of
a subsequence with previous subsequences to correctly identify its category.
This means that even if the current subsequence is much longer or shorter in
length, finding relevant sections from previously known subsequences can help
the model to understand the correct category of the subsequence. Therefore,
this idea eliminates the effect caused by subsequence length variations in
TSC.

Similarly, if the data is imbalanced and the model does not have enough data
to learn the features of the minority category, the contextual information can
tip the classification decision for a subsequence towards the correct category
as soon as it detects relevant sections with previous subsequences which also
belong to that minority category.

Since time-series data are ordered in time, the level of noise or redundancy
is expected to show continuous changes. If a model can constantly keep track
of the noise and redundancies by comparing the relevance of sections from
previous subsequences, it can continuously learn and cancel out the effect of
noise or redundancies. With this intuition, two aspects of a desired model can
be defined; it must be supervised to firstly allow the model to learn correct
features corresponding to categories and secondly, it must be able to keep
track of the contextual information for every subsequence with its nearest
previous ones.

Following the idea of context relevance, this study initially employs a CNN-
LSTM structure to learn the relationship between subsequences.

###  3.2 CNN-LSTM Architecture

CNN-LSTMs have shown great success for similar tasks such as activity
recognition, image description and video description [7]. The authors [7]
investigated different combinations of CNNs and LSTMs to create models that
are suitable for each of the tasks.

Typically, in a CNN-LSTM model, the CNNs are used to extract distinctive
features from subsequences while LSTMs are used to learn the relationship
between subsequences. LSTM is a popular Recurrent Neural Network (RNN) and the
benefit of using an RNN layer is its ability to “remember from the past” [9].
In particular, LSTM can be used to store context because it is influenced by
both its given input weights and hidden-state vectors that represent context
from previous inputs.

In this study, state-of-the-art CNN TSC models are combined with two LSTM
layers to form a CNN-LSTM architecture, illustrated in Figure 2. Note that the
CNN refers to either FCN or ResNet for TSC [13] while LSTM or Attention (will
be described in the next section) layers can be placed in between CNN and
final LSTM for subsequence context extraction. This design is applicable to
both univariate and multivariate time-series.

Figure 2: A general CNN-LSTM architecture, where $N$ is the number of
subsequences.

As illustrated in Figure 2, $N$ number of subsequences can be provided to this
CNN-LSTM. The first $N-1$ subsequences are used to extract context and when
combined with the feature-maps produced from the CNN, it is expected that the
final subsequence in the batch can be classified more accurately. Based on
this architecture, this study creates two models: FCN_LSTM and ResNet_LSTM.

The intuition behind these models is that the state-of-the-art CNNs are able
to extract discriminative features from subsequences which are then passed to
the LSTMs. For example, when the CNN in Figure 2 is FCN [13], it contains 3
convolutional layers with 128, 256 and 128 filters respectively. The LSTMs
then extract context from the feature maps generated by the CNNs. The feature
maps are then passed to two LSTM layers with 64 hidden units to extract
context from the feature maps. The output of the final LSTM layer is passed to
a softmax layer for classification.

Table 1 shows the improvements made using CNN-LSTM. While this approach
improves the classification performance when compared to existing CNNs, the
performance can be further improved using more complex context relevance
analysis. To explore this, attention mechanisms are added using an encoder-
decoder architecture for context extraction.

###  3.3 Attention for context extraction

Generally, seq-2-seq models with attention use LSTM encoders and LSTM decoders
[17]. However, as shown in Figure 2, this study implements Attention in a CNN-
LSTM architecture between the CNN encoders and LSTM decoders, because the CNNs
have shown to achieve state of the art performances on various TSC tasks [13].

While LSTM in CNN-LSTM models can decode the contextual information from CNN
feature-maps, attention mechanisms can be used to pay more “attention” to
certain subsequences in the whole time series. Attention mechanisms use
various techniques to align previous subsequences with the current subsequence
to identify strong pattern matches.

This research studies some state-of-the-art attention mechanisms.
Specifically, the following attention mechanisms are used to improve on the
CNN-LSTM architecture mentioned in the previous section.

  1. 1.

Alignment Attention, introduced by Luong et. al. [19]

  2. 2.

Bi-directional LSTM with Attention, by Bahdanau et. al. [2]

  3. 3.

Locally and Globally focused Self-Attention Model, designed by Vaswani et. al.
[26]

  4. 4.

Multi-Head Attention Model, also by Vaswani et. al. [26]

The method of how each attention mechanism extracts context relevance for TSC
is explained across the next few subsections. It is important to note, while
almost all of the attention algorithms discussed in this study are integrated
with FCN and ResNet, some of them are implemented independently to evaluate
their standalone performance.

####  3.3.1 Luongs’ Attention

Since the original Luong et. al. [19] attention was designed for seq-2-seq
models, this study implements the same architecture for time-series.

After receiving $N$ sets of feature-maps from the CNN encoders, Luongs’
attention creates an alignment-score vector. These alignment scores represent
how well each of the subsequences matches with the features extracted from the
other subsequences.

The alignment score vector is multiplied with the encoder output for the last
subsequence which is a method of identifying how much of the first $N-1$
subsequences “align” with the last subsequence. In other terms, this dot
product generates a context vector, that is the weighted sum of the feature-
maps of the last subsequence which is relevant to the previous subsequences.

Theoretically, the LSTM decoder can use these weights to understands how well
the last sequence is relevant to the previous sequences. This is done by
concatenating this context vector with the CNN feature-maps to update the
contextual information to the feature-maps.

####  3.3.2 Bi-directional Alignment Attention

Similar to Luongs’ strategy, the attention concept proposed by Bahdanau et.
al. [2] is implemented to extract the context of subsequences in forward and
reverse direction. The intuition behind their algorithm is to understand the
relevance of previous subsequences affecting the final one, and how the class
of the previous subsequences could be affected based on the classification
made on the final subsequence.

Figure 3: Bi-directional LSTM with attention

The model begins by receiving a batch of $N$ subsequence feature-maps. To
allow analysis in both directions, this algorithm uses a bi-directional LSTM
to learn the representation of the subsequences in order and backwards.

Figure 3 illustrates how each direction of the bi-directional LSTM create a
set of hidden states, labelled $[E1,E2,E3,E4]$ for forward and $[B1,B2,B3,B4]$
for backwards when considering $N=4$ subsequences. Similar to Luongs’ model
[19], these hidden states represent the alignment scores of subsequences
individually for each direction and hence, provides much more context for TSC.

The algorithm then concatenates these hidden-states together, to allow each of
them to collectively determine the overall alignment score of subsequences in
both forward and backward directions. The weights of the vector after
concatenation represent the context vector that is returned to the decoders.

####  3.3.3 Self-Attention

The Self-Attention model is made of complex “Transformer” encoders-decoders,
which are series of Self-Attention layers and Feed-Forward Neural Networks
[26]. The core Self-Attention (SelfA) library implemented in this study was
adapted from a Keras implementation by CyberZHG222https://github.com/CyberZHG.

The transformer generates and maintains Query(Q), Key(K) and Value(V) vectors
[26], which are responsible to keep track of the context of subsequences.
After receiving feature maps from the CNN encoders, SelfA passes these through
the Transformer encoders to calculate the alignment scores of subsequences.
Then these alignment scores are multiplied with its initially created value(V)
vector with certain calculations to flush out irrelevant values from the
context vector [26]. In the case of weakly labelled time-series, this is
expected to clean irrelevant context from the first three subsequences
efficiently, to allow stronger matches to be found with the last subsequence.

The SelfA model also introduces the concept of “soft” vs. “hard” attention
[26]. In terms of TSC, soft attention creates a context vector of a window
centred around the position of the current subsequence for maximum focus, and
it stretches along the length of previous subsequences so the model gets an
overall perspective of the context. On the contrary, hard attention
aggressively focuses on specific areas of input subsequences and therefore,
identifies the relevance of definite sections of the previous subsequences to
the current.

This study implements both global and local SelfA algorithms, where global
SelfA uses soft attention to learn overall patterns across the previous
subsequences, and local SelfA uses a balance of soft and hard attention to
extract alignment score for subsequences.

This study also implements the two different alignment score functions
proposed by the SelfA algorithm [26]. The additive function calculates the
scores by passing the context vectors to a Feed-Forward Network (FFN), whereas
the multiplicative method passes the vectors into a softmax layer where the
scores are calculated as the following equation.

$Attention(Q,K,V)=softmax(\frac{QK^{T}}{\sqrt{d_{k}}})V$ (1)

For context extraction of subsequences, the additive function is expected to
outperform multiplicative function, especially when the encoder key dimension,
dk, is large. This dk value depends on the complexity of time-series
subsequences, that is, the higher the number of features of the subsequence,
the higher the dimensions and vice versa. In the case of multiplicative
approach, large dk means large magnitudes of alignment scores are produced and
therefore, extreme gradient values are generated which would negatively impact
context extraction. For weakly labelled time-series, the number of features
extracted from subsequences is expected to be high particularly due to
additional noise and redundancies. However, assuming that the complexity is
not definitive, this research evaluates SelfA strategy with both
multiplicative and additive functions.

The performance comparisons between global vs. local SelfA, and multiplicative
vs. additive SelfA, are discussed later in Section 4.

####  3.3.4 Multi-Head Attention

Finally, one of the most advanced attention mechanisms, Multi-Head Attention
(MHA), proposed by [26], is a strategy which adopts the idea of the previously
discussed SelfA, but with more focused attention to various sections of
previous subsequences in parallel.

This algorithm runs several self-attention models together to perform context
extraction, which makes it computationally heavy. The MHA implemented in this
study is primarily based on a Keras implementation by CyberZHG 2, with
improvements made from the original implementation, which is available as a
Tensorflow library 333https://www.tensorflow.org/tutorials/text/transformer.

Figure 4: An example of MHA focusing on different time-series subsequence
positions

From the CNN encoders, MHA receives a set of $N$ feature-maps per batch, again
with the aim to classify the last one. This algorithm creates its initial
vectors similar to SelfA but this is replicated “h” times, where h is the
number of heads selected as a hyper-parameter.

With regards to TSC, MHA runs the parallel SelfA encoders to understand the
relevance of multiple areas of input subsequence at the same time. Figure 4
illustrates how MHA determines that the last subsequence is relevant to the
two immediate-previous subsequences, but not to the first one. The multiple
heads in MHA allow the model to focus on individual positions within these two
previous subsequences to compare the various sections with the subsequence,
and at the same time flushing out the irrelevant sections using self-attention
as discussed before.

While the multiple heads allow a parallel focus on previous subsequences, it
is important that the model considers the order. For TSC, it is only useful to
keep track of neighbouring subsequences if they are part of a running trend.
Nevertheless, MHA takes care of the order using an additional vector known as
the positional encoding. Theoretically, this keeps track of the distance
between each extracted context locations, giving the model a sense of
subsequence order.

MHA decoders are masked which means they will only consider previous
subsequences. This is important in this study because it enables the model to
suit real-time classifications where future subsequences are not known. While
the previously discussed attention algorithms follow this restriction by
default, MHA requires this additional masking due to multiple Self-Attention
running in parallel and because MHA cannot centrally control the individual
attention encoder/decoder positions.

###  3.4 Multi-Step CNN-ATTENTION

Since the CNN-LSTM encoder-decoder architecture is designed to receive $N$
subsequences per batch, this study labels these models as Multi-Step CNN-
ATTENTION models, where $N>1$. The list of these models implemented and
studied in this research are as follows.

  * •

FCN_LATTN: Luong’s Attention with FCN

  * •

ResNet_LATTN: Luong’s Attention with ResNet

  * •

FCN_BATTN: Bahdanau’s Attention with FCN

  * •

ResNet_BATTN: Bahdanau’s Attention with ResNet

  * •

SelfA: Local Additive Self-Attention

  * •

FCN_SelfA: Local Additive Self-Attention with FCN

  * •

FCN_SelfA_Global: Global Self-Attention with FCN

  * •

FCN_SelfA_Multiplicative: Local Multiplicative Self-Attention with FCN

  * •

ResNet_SelfA: Local Self-Attention with ResNet

  * •

MHA: Multi-Head Attention

  * •

FCN_MHA: Multi-Head Attention with FCN

  * •

ResNet_MHA: Multi-Head Attention with ResNet

###  3.5 Single-Step CNN-ATTENTION

The multi-step models extract the features and context from the past $N-1$
subsequences to predict the final one. However, to create a baseline of how
the models perform when only one subsequence ($N=1$) is passed, a set of
Single-Step CNN-ATTENTION models are created.

The single-step models do not consider the relationship between subsequences,
and will only consider the relevance of sections within a single sequence. For
example, in the case of the EEG time-series, past three subsequences may show
that the driver is continuously in a state of distraction and in the last
subsequence, the first half show distraction trends, but the second half may
start revealing increasing driving concentration. The single-step models are
designed to disregard the relevance of the previous subsequence and evaluate
the performance by only detecting context within single subsequences.

The single-step models comprise of FCN and ResNet combined with MHA. Only the
MHA algorithm is selected because of its ability to focus on various positions
of a subsequence using parallel attention “heads”. Each of these heads are
expected to apply the self-attention strategy to different positions within
the single subsequence to extract the context. Similar to the multi-step
models, the CNN layers learn the features of the subsequence while the MHA
layer simply adds the context vector.

The single-step models implemented in this study are as follows.

  * •

LSA: Luong’s Single-step Attention

  * •

MHSA: Multi-Head Single-step Attention

  * •

FCN_MHSA: Multi-Head Single-step Attention with FCN

  * •

ResNet_MHSA: Multi-Head Single-step Attention with ResNet

##  4 Experimentation

This section discusses the weakly labelled dataset used in this study. It also
discusses the various performance metrics improved with the use of Attention
for TSC.

###  4.1 Experimental Setup

All the models in this study are evaluated on two weakly labelled time-series
datasets Emotiv266 and EmotivRaw, supplied by the bioinformatics company
Emotiv 1. The Emotiv datasets are collections of EEG brain signals data,
captured using a light-weight and wireless Emotiv EPOC+ headset1, from a
40-minute driving simulation by 18 participants performing 16 different tasks
(defined in Figure 1 legend) while driving.

####  4.1.1 Details of the datasets by Emotiv

The EmotivRaw dataset captures 14 individual EEG channels in parallel, at a
frequency of 128Hz (illustrated in Figure 1). As the participants perform
various activities during driving, their brain activity alters and these are
recorded in the EEG time-series. To remove ocular artifacts, a 4-40Hz band-
pass filter and Fast Fourier Transform (FFT) [21] was applied to every 2
seconds window with 0.25-second intervals. Each of the channels was further
split into several frequency bands to extract features including average
power, peak power and peak frequency from each of the frequency bands.
Additional features were also created based on the aggregated data from
various brain regions and occipital power within each specified band, and by
accumulating EEP power. In total, 266 variables (channels) per EEG signal are
created and combined to form Emotiv266.

####  4.1.2 Data Preparation

Among the 18 participants, 12 participants are randomly selected as the
‘training’ set, another 2 as ‘validation’ set and the remaining are used as
‘test’ set to evaluate the models.

EmotivRaw and Emotiv266 are long multivariate time-series. To allow these
time-series to be accepted by the deep-learning models, the long time-series
is segmented into equal-length “subsequences”, as shown in Figure 5.

Figure 5: Time-series segmentation of Emotiv266

The subsequences are extracted with a sliding window with 50% overlap. For
Emotiv266, these segmented subsequences are of dimensions $266\times 40$,
which refers to 266 parallel time-series channels of 40 time-steps in length,
corresponding to a 4.25 seconds window. Similarly for EmotivRaw with 14
channels, sampled at 128Hz, the time series are segmented into subsequences of
$14\times 256$, representing 2 seconds window. To create a suitable simulation
of a real-time stream, all the multi-step models are evaluated with 4
subsequences per batch, that is $N=4$. The four subsequences together
represent a window of 11.75s (160 time-steps samples) for Emotiv266 and 3.5s
(1024 time-steps samples) for EmotivRaw. The number of subsequences in
training, validation and test set of Emotiv266 are 7656, 995 and 2010
respectively, and in EmotivRaw are 18040, 2513 and 3947.

####  4.1.3 Label Enhancements

In this study, only considering binary classification are considered, where a
class 0 represents distracted and class 1 represents focused driving
respectively. Multi-class classification will be considered as future work
where a more complex model is required. Typically each of these subsequences
are labelled with the majority label. This may introduce some unwanted signals
in the subsequence which may lead to poor performance. Hence, a slightly
different approach is considered in this study. If the whole subsequence does
not contain multiple groups of labels, then the same label is assigned to the
subsequence. However, if there are two different groups of labels across the
subsequence, the subsequence is further split into two and each subsequence is
filled with insignificant placeholder values to fill them to window sizes of
40. This ensure that each subsequence extracted has consistent labels. If the
window has over two label variations (which is unlikely in Emotiv), the
windows are discarded. This reduces any negative impact on TSC experimentation
that can be caused by data preparation.

####  4.1.4 Up-sampling

Similar to most real-world datasets, the Emotiv datasets are also highly
imbalanced. The majority label is “focused” driving, which is almost twice the
number of “distracted” driving (a ratio of 3828 to 2032 samples). Although the
attention models are expected to mitigate the effect of imbalanced data, this
study explores a simple method to balance out the classes in the data by
random up-sampling the minority class in the training data before training.
Note that other approaches such as SMOTE [4] are not considered in this study
as random up-sampling has found to work well in practice without the
complexity of generating synthetic time series data. The effect of up-sampling
data on the performance of the models of this study is analysed in detail
later in Section 4.

The python library “imbalanced-learn” 444https://imbalanced-
learn.readthedocs.io is used to repeatedly duplicate random records from the
minority group (distracted category). This balances the number of records for
each label and provides a fair learning opportunity for the models.

###  4.2 Evaluating Performance

All the models implemented in this study are trained with the categorical
cross-entropy loss for 100 epochs and optimised with ADAM [15]. The learning
rate is adjusted based on the loss gradient. This ensures the model can reach
a minimum plateau in validation loss, and hence perform TSC with minimum loss.
Moreover, the experiments are designed to minimise empirical risk [25] by
selecting the best models with minimum loss among the epochs. The code for
this study has been made publicly available at
https://github.com/Surayez/EmotivDriverDistraction.

Table 1 outlines the average test scores of each model on Emotiv266 and
EmotivRaw after five runs.

| Emotiv266 | EmotivRaw
---|---|---
| Accuracy | AUC | F1 Distracted | Accuracy | AUC | F1 Distracted
Models | ($\mathrm{\char 37\relax}$) | ($\mathrm{\char 37\relax}$) | ($\mathrm{\char 37\relax}$) | ($\mathrm{\char 37\relax}$) | ($\mathrm{\char 37\relax}$) | ($\mathrm{\char 37\relax}$)
Existing Models |  |  |  |  |  |
ROCKET | 59.49 | 41.09 | 54.1 | 59.21 | 41.38 | 54.73
FCN | 55.06 | 58.64 | 50.96 | 57.3 | 50.55 | 50.6
ResNet | 53.24 | 54.59 | 45.75 | 56.99 | 40.02 | 53.32
white LSTM Models |  |  |  |  |  |
FCN_LSTM | 60.23 | 60.31 | 48.57 | 60.94 | 49.81 | 53.2
ResNet_LSTM | 61.51 | 53.43 | 48.72 | 60.69 | 53.12 | 46.12
Single-Step CNN-ATTENTION Models |  |  |  |  |  |
LSA | 55.34 | 40.21 | 48.67 | 56.82 | 41.1 | 47.39
MHSA | 61.88 | 54.56 | 59.62 | 60.47 | 43.03 | 46.97
FCN_MHSA | 63.91 | 58.51 | 46.96 | 62.63 | 44.78 | 47.55
ResNet_MHSA | 62.83 | 53.91 | 40.01 | 60.11 | 40.12 | 44.29
white Multi-Step CNN-ATTENTION Models |  |  |  |  |  |
SelfA | 60.95 | 58.2 | 42.1 | 61.65 | 42.09 | 48.57
MHA | 64.32 | 60.95 | 50.01 | 62.19 | 48.51 | 39.47
FCN_SelfA | 68.32 | 61.94 | 54.71 | 64.12 | 52.36 | 54.55
FCN_SelfA_Global | 65.81 | 60.43 | 52.29 | 62.07 | 50.88 | 52.71
FCN_MHA | 66.16 | 63.51 | 54.21 | 64.25 | 47.11 | 52.39
FCN_LATTN | 60.4 | 56.56 | 48.66 | 59.29 | 40.44 | 49.23
FCN_BATTN | 60.76 | 59.01 | 50.63 | 59.92 | 44.37 | 46.91
ResNet_MHA | 61.11 | 54.84 | 41.39 | 60.83 | 52.09 | 48.36
ResNet_SelfA | 62.21 | 60.4 | 54.4 | 59.87 | 42.85 | 54.1
ResNet_LATTN | 54.99 | 48.7 | 50.85 | 58.91 | 39.01 | 52.06
ResNet_BATTN | 63.19 | 50.4 | 52.83 | 60.56 | 42.09 | 53.8
Table 1: Performance of each model on both Emotiv dataset, averaged over 5
runs. The ones in bold indicate the best performing models.

####  4.2.1 Accuracy

Immediately noticeable from Table 1, most of the attention implementations
perform with higher accuracies than the existing deep learning models. The
addition of the LSTM layer, for basic context relevance, improved the accuracy
scores of FCN and ResNet by almost 5% on both datasets, which is slightly
higher than the accuracy by achieved by ROCKET.

However, the most notable models are FCN_SelfA, MHA and FCN_MHA, which show
significant improvements in classification accuracy. On average, FCN_SelfA,
classifies approximately 13% more accurately compared to the existing FCN
model and over 7% than FCN_LSTM on Emotiv266. Again on EmotivRaw FCN_SelfA
shows 7% and 4% enhancement over FCN and FCN_LSTM. Following this, FCN_MHA and
MHA perform with over 10% higher accuracies than FCN, and 4% higher accuracies
when compared to FCN_LSTM on Emotiv266, and almost 7% and 3% on EmotivRaw.
These show dramatic improvements in weakly labelled TSC, in comparison to the
existing models.

In fact, the accuracy scores of SelfA and MHA by themselves are surprisingly
high. It indicates that the position based context extraction of SelfA and MHA
can solely perform TSC more accurately on weakly labelled data than the
existing models. However, the high loss metrics of these two models, discussed
later, indicate that this is not always the case.

The simpler LATTN and BATTN models also demonstrate improved TSC accuracy. In
comparison to existing models, FCN_LATTN and FCN_BATTN are almost 5% more
accurate on both dataset, but when compared with FCN_LSTM the performance is
almost identical. This indicates that the additional attention in these models
are not improving the performance significantly. This is largely due to
underfitting, as discussed in more detail in Section 4.2.8.

####  4.2.2 AUC

To further establish the improvement using attention algorithms, the AUC
scores are evaluated as part of this analysis. AUC scores are important for
this binary classification evaluation as it eliminates the effect of bias in
the target categories when comparing the average performance of classification
algorithms [12].

The CNN-ATTENTION models clearly demonstrate significant improvements in AUC
metrics. FCN_MHA performs with an almost 5% higher AUC score compared to the
FCN model, which has the highest AUC score among the existing TSC models.
Similarly, FCN_SelfA shows approximately 3% improvement compared to FCN on
both datasets. Even when LSTM extracts the basic context, it improves the AUC
by almost 2% on Emotiv266. This demonstrates that the higher accuracy scores
of FCN are, in fact, improving the overall performance of TSC.

On the other hand, the ResNet models show much lower AUC scores in comparison
to FCN models. This indicates that the accuracy improvements when using
attention with ResNet may not always be improving the overall performance of
the models. The primary reason for this is that ResNet is overfitting the
training data, and this issue is discussed in detail later in section 4.2.7.

####  4.2.3 F1 Score

The F1 scores of each model are important in this study because the initial
Emotiv data is highly imbalanced [24]. Table 1 shows the maximum average F1
score among the existing models is at 54% achieved by ROCKET. Although
FCN_SelfA and FCN_MHA improved the F1 score by almost 4% when compared to FCN
on Emotiv266 and 2% on EmotivRaw, this is not a drastic improvement
considering the F1 score by ROCKET is around the same.

However, the performance of CNN-ATTENTION in terms of F1 score is reflected
when the models are evaluated with imbalanced data. Table 2 compares the F1
and accuracy scores by the existing FCN and ResNet, and the CNN-ATTENTION
model with the highest accuracy from Table 1, which is FCN_SelfA, before and
after the training set of Emotiv266 is up-sampled.

| Up- | F1 Distr. | F1 Driving | Accuracy
---|---|---|---|---
Models | sampled | ($\mathrm{\char 37\relax}$) | ($\mathrm{\char 37\relax}$) | ($\mathrm{\char 37\relax}$)
FCN | No | 22.47 | 79.88 | 66.51
| Yes | 50.96 | 60.25 | 55.06
ResNet | No | 20.58 | 80.62 | 65.27
| Yes | 45.49 | 62.73 | 53.24
FCN_SelfA | No | 35.21 | 77.21 | 65.55
| Yes | 54.71 | 74.89 | 68.32
Table 2: Performance variations based on up-sampling

Table 2 shows that the accuracy and F1 driving scores of FCN and ResNet are
quite high when the data is imbalanced. However, the F1 distracted score is
significantly low which means that the models can accurately classify most of
the driving (dominant) subsequences, but only a few of the distracted (minor)
category. The overall accuracy is high due to the higher true positives where
in reality, most of the distracted category classifications are incorrect.

When the data is up-sampled (balanced), the number of accurate distracted
classifications improve, hence the distracted F1 scores rise sharply. Now that
FCN and ResNet are equally false-classifying for both categories, the overall
accuracy falls. Similar behaviour is observed on EmotivRaw.

Nevertheless, the F1 scores of FCN_SelfA show that the effect of up-sampling
has a minor effect on both the accuracy and F1 scores for driving. Moreover,
the difference between the distracted F1 scores before and after up-sampling
is much smaller than in the case of FCN and ResNet. This proves that the
attention strategy, as expected, is assisting the FCN model to make better
decisions of the minority category when data is imbalanced. Using the context
information from previous subsequences, it is able to eliminate the effect of
imbalanced data. Therefore, it can be concluded that the improved accuracy of
FCN_SelfA would be preserved even when the input time-series is imbalanced, as
in a real-world setting.

####  4.2.4 FCN and ResNet

The performance evaluations of this study indicate that ResNet has
inconsistent improvements when combined with LSTM or attention algorithms.
Table 1 shows that ResNet performs well when combined with SelfA and BATTN,
but show lower improvements with MHA and LATTN. The overall accuracy metrics
suggest that FCN is performing significantly better with additional context
information than ResNet.

However, the reason for this behaviour can be pointed to the more complex
architecture of ResNet. The feature-maps returned from 3 convolution blocks of
FCN seems to be better interpreted by the attention, than with that from 9
convolution blocks of ResNet. Because of this, ResNet overfits the training
set slightly more than FCN and hence, lower accuracies are reflected in Table
1.

####  4.2.5 Single-Step Baseline Models

When the single-step models are evaluated on both the Emotiv dataset, they did
not demonstrate exceptional performance metrics. The results from Table 1
indicate that the single-step models slightly improve TSC when compared to FCN
and ResNet, but not as much as the multi-step models.

This confirms the intuition of this research that the multi-step models are
indeed using subsequence relevance to increase accuracy. It also suggests that
understanding the context of various positions within the single subsequence
is not enough to minimise the effect of weakly labelled data in Emotiv time-
series. The attention models examining a larger window for context (multi-step
models) are performing much better.

However, these performance metrics do not disqualify the single-step models
from weakly labelled TSC. Intuitively, the models are expected to prove very
useful when subsequences of time-series are of higher lengths and show a more
repetitive pattern. For the case of Emotiv266 or EmotivRaw, each subsequence
block is not exceptionally long to take advantage of such context analysis and
therefore, the single-step models show insignificant improvements compared to
the existing models.

####  4.2.6 Validation Loss

Although there is an increase in overall accuracy, the loss graphs generated
by the FCN_SelfA (shown in Figure 6) and FCN_MHA models indicate overfitting
of the training set. This is identified by the increasing and fluctuating
validation loss while the training loss decrease exponentially.

Due to overfitting, when the models are given an unseen data (validation set),
the false predictions are heavily penalised by significantly increasing loss
and correct predictions have a minor effect on decreasing the loss [23]. This
explains the behaviour of Self-Attention and MHA classifying with high
accuracy but also with higher loss.

On the contrary, LATTN and BATTN (shown in Figure 7) models have a better loss
pattern despite their minor improvements in accuracy. Each of its validation
loss shows an exponential decrease, similar to its training loss. Once again,
this is a typical indication that the models are underfitting the training
data and is discussed in section 4.2.8.

Figure 6: Training and validation loss of FCN_SelfA Figure 7: Training and
validation loss of FCN_BATTN

####  4.2.7 Effect of Overfitting

Figure 8 shows the training, validation and test accuracies in a sample
evaluation of the models on the Emotiv266 time-series. It is clear that almost
all the models have a much higher training accuracy than test accuracy, which
identifies that the models are largely overfitting the training set [14].

Figure 8: Accuracy of TSC models with 100 epochs

The reason for this behaviour is due to the complex architectures of FCN and
ResNet. When these models are trained on a weakly labelled dataset before
attention is applied, they learn the noise and redundancies of the training
data too accurately. This clouds their judgement of the noise as distinctive
features of subsequences. As the models are then evaluated on test data, they
fail to identify the new patterns of noise and redundancies in test set,
leading to low test set accuracies.

As expected, the attention layers significantly improve the accuracy of the
models by cutting down this effect. The improved results indicate that
attention can successfully support TSC by extracting context of subsequences.
However, this effect is not completely eliminated by attention.

####  4.2.8 Effect of Underfitting

Attention algorithms by Luong et. al. [19] and Bahdanau et. al. [2] are
considering subsequence relevance information because it shows minor
improvements, but the simplicity of the alignment calculation algorithm fails
to retrieve enough context in comparison to SelfA or MHA. The more complex
internal encoders and decoders of SelfA and MHA provide them with powerful
computational support to ’attend’ more sections within the previous
subsequences, and therefore better alignments to be found. This explains the
underfitting behaviour of LATTN and BATTN which have low accuracies despite
having a low validation loss metric.

Nevertheless, LATTN and BATTN are re-evaluated in this study on longer epochs
(300 epochs) with the expectation that further training may eliminate the
effect of underfitting. Unfortunately, the performance did not improve even
with the longer training period.

####  4.2.9 Global vs. Local SelfA

SelfA is one of the best performing models of this study and further analysis
are conducted to explore various aspects of SelfA for TSC. Firstly, this
research evaluates the performance of global and localised SelfA algorithms on
Emotiv266. As discussed earlier, local SelfA is expected to be more accurate
for Emotiv266 because it considers various positions within previous
subsequences, rather than a global “soft” focus. The results are consistent
with this expectation because on average, local SelfA performed 4% more
accurately than global SelfA. Similar performance is observed on EmotivRaw.

####  4.2.10 Additive vs. Multiplicative SelfA

Furthermore, the performance of additive and multiplicative SelfA are
evaluated on Emotiv266. When local FCN_SelfA is evaluated with the additive
alignment function, it performed 5% more accurately than when evaluated with
multiplicative alignment function.

The reason for this performance difference is the previously discussed FFN
layer, which is present in the additive algorithm but is replaced with a
softmax calculation in Equation (1) in the multiplicative algorithm. Due to
the noise and data redundancies, the subsequences in Emotiv are of high
complexities. The extra FFN layer helps the additive model interpret the
encoder keys with larger dimensions of the Emotiv subsequences, and causes the
multiplicative calculation to produce extreme gradient values which negatively
impact context extraction.

####  4.2.11 Effect of normalisation

| Accuracy with normalisation:
---|---
| None | Subsequence | Series | Both
Models | ($\%$) | ($\%$) | ($\%$) | ($\%$)
FCN | 69.29 | 63.12 | 54.32 | 55.06
ResNet | 65.54 | 64.42 | 52.16 | 53.24
FCN_LSTM | 70.12 | 67.78 | 70.64 | 60.23
ResNet_LSTM | 67.36 | 66.41 | 64.51 | 61.51
FCN_SelfA | 71.96 | 68.09 | 71.29 | 68.32
FCN_MHA | 71.13 | 67.26 | 70.35 | 66.16
Table 3: Normalisation Analysis

Table 3 shows the average outcomes when the models are evaluated on time-
series with:

  * •

No normalisation

  * •

Only normalising the subsequences

  * •

Only normalising the time series

  * •

Both normalisation as above

While it is a standard practice to normalise time-series for TSC using deep-
learning [16], the performance of the models without the two types of
normalisation applied in this study is evaluated to further reinforce the TSC
improvements using attention. The time-series are normalised when loading the
dataset and subsequences are normalised during segmentation. The outcomes in
Table 3 highlights the improved performance using attention by both Self-
Attention and Multi-Head Attention models on Emotiv266. Exact patterns are
observed for EmotivRaw

##  5 Conclusion

From the experiments on the real-world Emotiv dataset, it can be concluded
that the attention mechanism improves the performance of existing state-of-
the-art TSC algorithms on weakly-labelled time-series. To eliminate the effect
of noise, redundancies and data imbalances, as in a real-world setting, the
extraction and interpretation of subsequence context information help
supervised CNNs to improve their performance in TSC. In particular, the
performance of Self-Attention with FCN (with additive function in localised
context) and Multi-Head Attention with FCN displays the most dramatic
improvements, and can therefore be concluded as the top models of this study.

The set of models developed as part of this research can complement any
further studies for real-time TSC. The CNN-LSTM architecture can be used to
further analyse the performance of these models on datasets from different
domains.

## References

  * [1] A. Bagnall, J. Lines, A. Bostrom, J. Large, and E. Keogh.  The great time series classification bake off: a review and experimental evaluation of recent algorithmic advances.  Data Min. Knowl. Discov., 31(3):606–660, 2017.
  * [2] D. Bahdanau, K. Cho, and Y. Bengio.  Neural machine translation by jointly learning to align and translate.  In 3rd International Conference on Learning Representations, ICLR 2015, 2015.
  * [3] S. Chaudhari, G. Polatkan, R. Ramanath, and V. Mithal.  An attentive survey of attention models.  arXiv preprint arXiv:1904.02874, 2019.
  * [4] N. V. Chawla, K. W. Bowyer, L. O. Hall, and W. P. Kegelmeyer.  Smote: synthetic minority over-sampling technique.  Journal of artificial intelligence research, 16:321–357, 2002.
  * [5] H. A. Dau, A. Bagnall, K. Kamgar, C.-C. M. Yeh, Y. Zhu, S. Gharghabi, C. A. Ratanamahatana, and E. Keogh.  The ucr time series archive.  IEEE/CAA Journal of Automatica Sinica, 6(6):1293–1305, 2019.
  * [6] A. Dempster, F. Petitjean, and G. I. Webb.  ROCKET: exceptionally fast and accurate time series classification using random convolutional kernels.  Data Min. Knowl. Discov., 34(5):1454–1495, Sept. 2020.
  * [7] J. Donahue, L. Anne Hendricks, S. Guadarrama, M. Rohrbach, S. Venugopalan, K. Saenko, and T. Darrell.  Long-term recurrent convolutional networks for visual recognition and description.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2625–2634, 2015.
  * [8] H. I. Fawaz, B. Lucas, G. Forestier, C. Pelletier, D. F. Schmidt, J. Weber, G. I. Webb, L. Idoumghar, P.-A. Muller, and F. Petitjean.  Inceptiontime: Finding alexnet for time series classification.  Data Mining and Knowledge Discovery, 34(6):1936–1962, 2020.
  * [9] F. A. Gers, J. Schmidhuber, and F. Cummins.  Learning to forget: continual prediction with lstm.  In 1999 Ninth International Conference on Artificial Neural Networks ICANN 99. (Conf. Publ. No. 470), volume 2, pages 850–855 vol.2, 1999\.
  * [10] S. Hochreiter.  The vanishing gradient problem during learning recurrent neural nets and problem solutions.  Internat. J. Uncertain. Fuzziness Knowledge-Based Systems, 6(2):107–116, Apr. 1998.
  * [11] B. Hu, Y. Chen, and E. Keogh.  Classification of streaming time series under more realistic assumptions.  Data Min. Knowl. Discov., 30(2):403–437, Mar. 2016.
  * [12] J. Huang and C. X. Ling.  Using AUC and accuracy in evaluating learning algorithms.  IEEE Trans. Knowl. Data Eng., 17(3), Mar. 2005.
  * [13] H. Ismail Fawaz, G. Forestier, J. Weber, L. Idoumghar, and P.-A. Muller.  Deep learning for time series classification: a review.  Data Min. Knowl. Discov., 33(4):917–963, July 2019.
  * [14] H. Jabbar and R. Z. Khan.  Methods to avoid over-fitting and under-fitting in supervised machine learning (comparative study).  Computer Science, Communication and Instrumentation Devices, pages 163–172, 2015.
  * [15] D. P. Kingma and J. Ba.  Adam: A method for stochastic optimization.  In Y. Bengio and Y. LeCun, editors, 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, 2015.
  * [16] Y. Lei and Z. Wu.  Time series classification based on statistical features.  EURASIP Journal on Wireless Communications and Networking, 2020(1):1–13, 2020.
  * [17] Y. Li, Z. Zhu, D. Kong, H. Han, and Y. Zhao.  Ea-lstm: Evolutionary attention-based lstm for time series prediction.  Knowledge-Based Systems, 181:104785, 2019.
  * [18] J. Lines, S. Taylor, and A. Bagnall.  HIVE-COTE: The hierarchical vote collective of transformation-based ensembles for time series classification.  In Proceedings of the 16th IEEE international conference on data mining (ICDM), pages 1041–1046, 2016.
  * [19] M.-T. Luong, H. Pham, and C. D. Manning.  Effective approaches to attention-based neural machine translation.  In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 1412–1421, 2015.
  * [20] P. Misra and S. Chaurasia.  Machine learning and time series: Real world applications, 05 2017.
  * [21] U. Oberst.  The fast fourier transform.  SIAM J. Control Optim., 46(2):496–540, Jan. 2007.
  * [22] M. Perslev, M. H. Jensen, S. Darkner, P. J. Jennum, and C. Igel.  U-Time: A fully convolutional network for time series segmentation applied to sleep staging.  pages 4415–4426, Oct. 2019.
  * [23] S. Salman and X. Liu.  Overfitting mechanism and avoidance in deep neural networks.  Jan. 2019.
  * [24] M. Sokolova, N. Japkowicz, and S. Szpakowicz.  Beyond accuracy, f-score and roc: A family of discriminant measures for performance evaluation.  volume Vol. 4304, pages 1015–1021, 01 2006.
  * [25] O. Taiwo.  A logical approach for empirical risk minimization in machine learning for data stratification.
  * [26] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin.  Attention is all you need.  pages 5998–6008, June 2017.
  * [27] P. Veličković, G. Cucurull, A. Casanova, A. Romero, P. Lio, and Y. Bengio.  Graph attention networks.  arXiv preprint arXiv:1710.10903, 2017.