# DeepRA: Predicting Joint Damage from Radiographs Using CNN with Attention

Neelambuj Chaturvedi
Advanced Data Science Group, ZS Associates
Bengaluru, India
<EMAIL_ADDRESS>
###### Abstract

Joint damage in Rheumatoid Arthritis (RA) is assessed by manually inspecting
and grading radiographs of hands and feet. This is a tedious task which
requires trained experts whose subjective assessment leads to low inter-rater
agreement. An algorithm which can automatically predict the joint level damage
in hands and feet can help optimize this process, which will eventually aid
the doctors in better patient care and research. In this paper, we propose a
two-staged approach which amalgamates object detection and convolution neural
networks with attention which can efficiently and accurately predict the
overall and joint level narrowing and erosion from patients radiographs. This
approach has been evaluated on hands and feet radiographs of patients
suffering from RA and has achieved a weighted root mean squared error (RMSE)
of 1.358 and 1.404 in predicting joint level narrowing and erosion Sharp/van
der Heijde (SvH) scores which is 31% and 19% improvement with respect to the
baseline SvH scores, respectively. The proposed approach achieved a weighted
absolute error of 1.456 in predicting the overall damage in hands and feet
radiographs for the patients which is a 79% improvement as compared to the
baseline. Our method also provides an inherent capability to provide
explanations for model predictions using attention weights, which is essential
given the black box nature of deep learning models. The proposed approach was
developed during the RA2 Dream Challenge hosted by Dream Challenges
111http://dreamchallenges.org/challenges/ and secured 4th and 8th position in
predicting overall and joint level narrowing and erosion SvH scores from
radiographs.

_Keywords_ Attention $\cdot$ CNNs $\cdot$ Rheumatoid Artheritis $\cdot$ Object
Detection

##  1 Introduction

Rheumatoid arthritis (RA) is an inflammatory rheumatic disease with
progressive course affecting articular and extra-articular structures
resulting in pain, disability, and mortality Birch and Bhattacharya (2010).
Persistent inflammation leads to erosive joint damage and functional
impairment in the vast majority of patients Combe (2009). The extent of joint
damage can be assessed from radiographs using the established and validated
Sharp/van der Heijde (SvH) method Van der Heijde et al. (1995). However, this
requires experienced radiologists to manually inspect and grade the images.
This is not only very time consuming, but also highly subjective, ultimately
leading to low inter-rater reliability, where even trained experts often
disagree on the final score Renshaw and Gould (2005). An automated scoring
algorithm which can efficiently and accurately predict the narrowing and
erosion SvH scores is an important and unmet need for better patient care for
a number of reasons. First, RA-associated joint damage is not quantitative in
clinical practice, as radiology reports typically state mild, moderate, or
severe damage, which is highly subjective. Second, existing radiographs are a
very objective measure of patient outcomes, but are not optimally used in
patient care because of the lack of a method to quickly quantify damage.
Third, there is often a disagreement between patient symptoms and the degree
of damage in radiographs RA2 (2019 (accessed January, 2021). This discordance
makes early identification and treatment of damage critical, as many biologics
have clinical indications to prevent structural radiographic damage.
Convolution Neural Networks (CNNs) have delivered outstanding performance in
solving problems related to the medical imaging domain in the past, such as
Classification of skin cancer Fujisawa et al. (2019), prediction of lung
cancer mutations Coudray et al. (2018), prediction of covid 19 from chest
X-Rays Boudrioua (2020) where the authors leverage deep neural networks to
extract the hidden representation from the images and accurately predict the
outcome and have achieved outstanding results. With recent developments in
computer vision multitude of works have incorporated attention mechanism
within CNNs which helps in boosting their representation learning capacity,
such as CBAM, Dynamic convolution, Attention augmented CNNs Woo et al. (2018);
Bello et al. (2019); Chen et al. (2020). Attention mechanisms have also been
implemented in the domain of medical imaging such as CABNet He et al. (2020)
where the authors proposed an attention module which can be leveraged within
CNNs for imbalanced diabetic retinopathy grading. Integrating attention blocks
enhances the CNNs to focus on salient regions in the image but also acts as an
inherent ability to explain the predictions made by the models.

In this paper, we propose a two staged approach for predicting joint narrowing
and erosion SvH scores from hands and feet radiographs for patients using
object detection and CNN with an attention mechanism. First stage being object
detection, RetinaNet object detection models are trained on hands and feet
radiographs to detect fingers/wrist joints from the radiographs. The second
stage deals with developing joint level models which comprise of CNNs with
attention to predict the joint-wise narrowing and erosion SvH scores on top of
joints extracted from the first stage. Incorporating an attention mechanism
assists the joint level models to focus on the salient regions in the joint
radiographs to make damage predictions, additionally visualizing these
attention maps helps in explaining the model predictions.

To summarize the contribution of this work

  * $\bullet$

We deal with the problem of non-availability of annotations for detecting
fingers and wrist in hands and feet radiographs by training object detection
models which can accurately detect these joints for further downstream tasks

  * $\bullet$

We tackle the time consuming nature of grading radiographs by training joint
level attention based CNN models which provide accurate predictions for
narrowing and erosion SvH scores which also provides the overall RA damage for
the patients

  * $\bullet$

Given the black-box nature of deep learning models, our method provides
inherent explanability using attention maps, which establishes confidence in
the model predictions

Rest of the paper is organized as follows. In Section 2, we briefly describe
the related works in this domain. We describe the overall methodology in
detail i.e., object detection, CNN with attention in Section 3. In Section 4,
we discuss the experiments and results and present the attention maps
generated from the joint level models, we finally conclude the paper in
Section 5.

##  2 Related Work

Earlier works on the radiographic scoring of patients with rheumatoid
arthritis (RA) such as Snekhalatha and Anburajan (2017); Langs et al. (2008)
focuses on solving the problem related to joint space narrowing, these methods
lack extensive validation using the recently modified Sharp/van der Heijde
(SvH) scores and these approaches also involve some degree of manual effort in
generating hand crafted features. Other traditional methods to score
radiographs are Hoving et al. (2004); Døhn et al. (2008) which rely on
magnetic resonance imaging and sonography for damage predictions in patients
with Rheumatoid arthritis. Some recent work which deals with scoring
radiographs also exists for the degenerative, non-inflammatory osteoarthritis
such as in Tiulpin et al. (2018) the authors proposed a deep siamese CNN for
scoring the severity of osteoarthritis in knee radiographs and in Xue et al.
(2017) authors propose a method to detect osteoarthritis in hip radiographs
using pretrained VGG Simonyan and Zisserman (2014) models. More recently, the
representation learning capabilities of CNNs have also been leveraged to
categorize the level of joint damage from radiographs in limited works such as
Rohrbach et al. (2019) where the author developed a VGG Simonyan and Zisserman
(2014) inspired CNN model on hand radiographs at joint level to categorize the
joint damage, another work which classifies the degree of joint damage from
hand radiographs has been proposed in Hirano et al. (2019) where the authors
use haar cascade features for joint detection and further leverage a CNN model
to categorize the joint level damage, classifying the degree of joint damage
can be higly subjective, additionally these approaches lack the inherent
explainability of the model predictions. Other approaches developed during the
challenge extracted joint patches from hands and feet radiographs using object
detection and employed CNN models to predict the overall and joint level
damage of the patients, although the performance of these approaches is
promising however these approaches lacked inherent explanability for the model
predictions. In this work, we propose a novel solution where we detect finger
rather than finger joints from hands and feet radiographs and develop a
attention-based CNN architecture which provides accurate predictions at par
with the joint level approaches developed during the challenge and in prior
works, additionally our approach has an inherent explainability for the model
predictions.

##  3 Methodology

In this section, we discuss object detection using RetinaNet and Convolution
Neural Networks with attention, which are leveraged in the proposed approach.

###  3.1 Object Localization

We address the challenge of localizing joints from radiographs using object
detection, which exploits RetinaNet Lin et al. (2017a) a one-stage detector
model which is able to accurately detect fingers/wrists from the radiographs.
Object detectors are generally classified into two major categories i.e., two-
stage detectors and one-stage detectors, where the former performs better in
terms of accuracy in identifying the objects whereas the latter outperforms in
terms of speed and simplicity. RetinaNet being a one-stage detector
outperforms two-stage detectors in terms of accuracy, speed, and simplicity by
utilizing a modified loss function known as focal loss Lin et al. (2017a)
which tends to give higher weightage to hard examples and give a lower weight
to easily classified examples, focal loss also reduces the contribution from
easy examples and increases the importance of correcting misclassified
examples. The architecture of RetinaNet constitutes of three major components,
i.e., a backbone network for deep feature extraction, Feature Pyramid Network
(FPN) to extract multi-scale features, and two subnetworks, i.e., one which
predicts the class of the object and the second which predicts the coordinates
of the bounding boxes in the input images. As shown In Fig. 1. RetinaNet
employs backbone networks, such as residual networks (ResNet) He et al.
(2016), convolutional neural networks named by Visual Geometry Group (VGG)
Simonyan and Zisserman (2014), or densely connected convolutional networks
(DenseNet) Huang et al. (2017) to extract higher semantic feature maps and
then applies FPNs Lin et al. (2017b) to extract the multi-scale features from
the backbone network, it adopts a pyramidal top-down architecture with lateral
connections to capture the high-level feature maps at all scales and helps the
model to detect objects of smaller size Lin et al. (2017b). FPN extracts the
features from the images at different scales and merges them with the features
of consecutive layers of the backbone network, which effectively propagate
these at different levels and scales to the succeeding layer Li and Ren
(2019). The pyramidal features are then sent to the classification and
regression subnets to classify and locate the desired objects.

Figure 1: RetinaNet architecture consists of 3 major components a) Backbone
Networks, b) Feature Pyramid Networks (FPN) and the c) Sub Networks. Backbone
extracts the deep features from the image, FPN extracts the multi-scale
features and merge these using pyramidal network structure. Finally, Sub
Networks classifies and detects the objects in the image Lin et al. (2014);
Wang et al. (2019) Figure 2: The Model architecture leverages EfficientNet
B3-B5 Tan and Le (2019) backbones which are then passed on through the batch-
normalization layer and further sub divided into two sub networks where one
network convolves the feature maps and generate the attention weights which
are then multiplied with second sub network element-wise and finally generate
the narrowing/erosion scores after passing through the dense layers

###  3.2 Convolution with Attention

Convolutional Neural Network (CNN) has been recently employed to solve
problems from both the computer vision and medical image analysis domains.
Various research groups have developed innovative CNN architectures such as
DenseNets Huang et al. (2017), ResNets He et al. (2016), VGG Simonyan and
Zisserman (2014) which tend to be computationally expensive, Inception network
utilizes a unique architecture which lowers the computational cost compared to
the previous CNN architectures and achieves state-of-the-art performance in
the ImageNet Large-Scale Visual Recognition Challenge 2014 (ILSVRC14) Szegedy
et al. (2015). In recent years, many attempts has been made to improve CNN
performance by expanding them in different dimensions i.e width (W), depth (D)
and resolution (R) with limited computational budget, although it is possible
to scale two or three dimensions arbitrarily, this involves tedious manual
tuning and generally yields substandard accuracy and efficiency. Increasing
the width of the network helps the model to learn more complex features which
causes a problem of vanishing gradients which has been mitigated with the use
of skip connections as mentioned in He et al. (2016). Wider network
corresponds to the use of a higher number of channels which aim to capture
more fine-grained patterns in the images. Using images with higher
resolutions, Huang et al. (2019) have also achieved state-of-the-art ImageNet
accuracy. However, as we increase the W, D and R of the network, the accuracy
quickly saturates after a certain point Tan and Le (2019). Inspired by these
observations more recently, a family of models named EfficientNets Tan and Le
(2019) has been proposed, which tend to improve the performance of CNNs by
scaling them in three dimensions, i.e., W, D, and R using a set of fixed
scaling coefficients that meet a specific computational constraint. The family
of EfficientNet models ranges from B0-B7 where B0 is the baseline network
which was developed by neural architecture search (NAS) which tends to arrive
at the optimal values for W, D, and R for a CNN given a computational
constraint. Rescaling the architecture of the B0 model along three dimensions,
i.e., W, D, and R constitute of 7 other models denoted by B1-B7. The
EfficientNet-B1 is 7.6x smaller and 5.7x faster than ResNet-152 and
EfficientNet-B7 achieves new state-of-the-art 84.3% top-1 accuracy on ImageNet
task but is 8.4x smaller and 6.1x faster than GPipe Huang et al. (2019) as
mentioned in Tan and Le (2019). The proposed approach leveraged the
EfficientNet B3-B5 models as backbones in the joint level models to grade
radiographs.

####  3.2.1 Attention

Attention mechanism has revolutionized the Natural Language Processing (NLP)
domain. More recently numerous efforts have been made to incorporate attention
mechanisms in CNNs such as Xu et al. (2015) where authors introduce the
concept of soft and hard attention in images to generate image captions. We
incorporate an attention mechanism in the EfficientNet architecture which
enhances the representation learning capacity of the EfficientNet
architecture. This proposed approach is inspired by Schlemper et al. (2019) in
which the authors apply attention gated CNN network for object localization in
a 3D CT abdominal dataset. Attention module not only helps the model to
eventually learn the salient regions in the radiographs while making
predictions but also adds a layer of explainability by extraction of attention
weights. These weights are obtained by passing the input feature maps from the
EfficientNet backbone through various convolution layers and finally utilizing
a locally connected 2D layer with sigmoid activation which computes the
weights at a pixel level ranging from 0-1. A element-wise multiplication of
these attention weights with the feature maps helps the model to turn off the
pixel values where the weights are zero. Fig. 2. depicts the final model
architecture which is trained on the extracted joints from the first stage,
i.e., object detection. We employ task-specific EfficientNet backbones and
train an end-to-end network which predicts the joint damage from the
radiographs. Code for this approach is publicly available at
https://github.com/NC717/DeepRA.

##  4 Experimentation and Results

###  4.1 Dataset used

The Dataset used, for the analyses in this paper was contributed by University
of Alabama at Birmingham. This dataset was obtained as part of the RA2-DREAM
Challenge: Automated Scoring of Radiographic Damage in Rheumatoid Arthritis
through Synapse ID [syn20545111] Bridges Jr et al. (2010); Ormseth et al.
(2016). The training dataset consisted of 1,468 radiographs corresponding to
367 patients with manually annotated narrowing and erosion SvH scores at a
patient level. Leaderboard performance was evaluated on hands and feet
radiographs for 300 patients which along-with the final evaluation test set
was not available to the participants during the course of the RA2 Dream
Challenge. SvH scoring system considers joints in the hands/wrist and feet
that are typical targets of inflammation in rheumatoid Arthritis (RA), which
includes joints in the wrists, metacarpal phalangeal (MCP) and proximal
interphalangeal (PIP) joints of the hands, and the metatarsal phalangeal (MTP)
and the PIP joints in feet. The challenge was to predict narrowing and erosion
scores corresponding to these joints at a patient level, which can further be
used to get the overall damage in the radiographs Sun et al. (2021).

###  4.2 Joint localization

First stage of the proposed approach, i.e., object detection, was performed to
extract the fingers and wrist from the radiographs for both hands and feet.

####  4.2.1 Implementation Details

We trained RetinaNet model with ResNet50 He et al. (2016) as the backbone
network, which was pretrained on COCO image dataset Lin et al. (2014). All
weights of the backbone network were trainable owing to the inherent
differences in the data distributions in COCO dataset and the hands and feet
radiographs. Fingers in hands and feet and wrist in hand were manually
annotated for 914 radiographs using an open source tool
labellmg222https://github.com/tzutalin/labelImg, which were finally used to
train and validate two separate detector models to detect fingers/wrist from
the radiographs. First model was developed to detect fingers and wrist in hand
and the second model detected fingers in feet. We had 341 and 57 radiographs
in the training and validation sets, respectively, to develop the hand
finger/wrist detection model, whereas the feet finger detection model
consisted of 355 and 61 radiographs in the training and validation sets,
respectively. During training, we limit the dimensions of the input
radiographs with image Min/Max dimension bounded at 1,000 and 1,400
respectively. We employ Adam Kingma and Ba (2014) optimizer with a learning
rate of 0.001 which is reduced during training by a factor of 0.1 if the mean
Average Precision (mAP) has not improved after 4 consecutive epochs. Batch
size of 8 is used during training and to reduce the overfitting, early
stopping is used if the mAP has not improved for 5 consecutive epochs on the
validation set. We use Tesla K80 GPU on Google Colab to train and test the
models.

####  4.2.2 Results

We evaluate the detector models on the test dataset which comprised of 50
Hands/Feet radiographs respectively, these radiographs were not used during
the training and validation. The detector model, for the feet was able to
achieve a mAP of 0.965, whereas the hand model achieved a mAP of 0.981 at an
intersection over union threshold (IOU) of 0.6. In Table 1, we report the
joint wise average precision (AP), we observe that the wrist joint being the
easiest to detect, achieves the highest AP, whereas all other joints have a
relatively similar AP in the case of both detector models. Fig. 4 represents
the joints detected for a patient in the test set.

(a)  Hand Fingers/Wrist detected by the model

(b)  Feet Fingers/Toe detected by the model

Figure 3: Fingers, wrists and thumb/toe identified by the detection models for a patient Table 1: Overall and joint wise Average Precision (AP) at IOU threshold of 0.6 | Joint Type
---
| Hand Model
---
Joint Wise AP
| Feet Model
---
Joint Wise AP
Thumb/Toe | 0.986 | 0.983
Finger 1 | 0.979 | 0.973
Finger 2 | 0.983 | 0.955
Finger 3 | 0.971 | 0.947
Finger 4 | 0.977 | 0.965
Wrist | 0.993 | -
Overall mAP | 0.981 | 0.965

###  4.3 Joint Level Models

Second stage of the proposed approach consists of training joint level models,
to predict the joint narrowing and erosion SvH scores from the radiographs. We
describe the implementation details and the results in the following sections.

####  4.3.1 Implementation Details

We train 6 joint level models which are based on the EfficientNet [B3-B4-B5]
backbones pretrained on ImageNet dataset along-with the attention mechanism.
Finger joints extracted from object detection stage were resized to $256\times
256$ whereas wrist joints were resized to $300\times 300$ while training. We
use a 80:20 train test split to train all joint level models. During training,
we use Adam optimizer Kingma and Ba (2014) and mean squared error (MSE) as the
loss function with an initial learning rate of 0.001 which is decayed by a
factor of 0.2 if the validation RMSE does not improve within 5 epochs. We
train the models for 200 epochs and finally save the model which has the least
RMSE on the validation set. To make the model robust, we employ horizontal
flipping of the joint images in every joint model. During training 70% weights
of the EfficientNet backbones were unfreezed since the distribution of the
medical radiographs is entirely different from the ImageNet dataset which was
originally used to train these models. Approximately 70% of the joints had a
Narrowing/Erosion SvH score of zero in the training data which caused the
issue of imbalance while training. We tackled this by training the model by
randomly sampling the equivalent number of joints with non-zero
Narrowing/Erosion scores from zero damage joint radiographs, so that there is
no class imbalance while training the model. We use a batch-size of 32 and all
the models are implemented with Keras using Tensorflow backend. All the
experiments are performed on Tesla K80 GPUs on Google Colab. In Table 2, we
describe the training specifications for the joint level models which describe
the EfficientNet backbones utilized, the number of joint images extracted from
the radiographs using the detector models, and the number of predictions to be
made in every model.

$MSE=(\frac{1}{n})\sum_{i=1}^{n}(y_{i}-x_{i})^{2}$

Table 2: Training specifications for Joint level models: This table represents the EfficientNet backbones used, the number of joints extracted from object detection stage and number of predictions to be made for every joint level model Model type |  | Finger Joints
---
| EfficientNet
---
Backbone
| Total Joint
---
Images
| Number of
---
Predictions
Erosion | Hand Fingers & Thumb/Toe | B3 | 4,404 | 2
Narrowing | Hand Fingers & Toe | B3 | 3,670 | 2
Narrowing | Feet Fingers & Thumb | B4 | 3,670 | 1
Erosion | Feet Fingers | B4 | 2,936 | 1
Erosion & Narrowing | Wrist | B5 | 734 | 6

####  4.3.2 Evaluation Metrics

The RA2 Dream challenge utilized the weighted (Wt.) root mean squared error
(RMSE) to evaluate the joint level narrowing and erosion SvH scores, whereas
weighted (Wt.) absolute error was used to evaluate the overall RA damage at a
patient level. The weights used for calculating the joint wise RMSE were not
released during the RA2 Dream Challenge.

$Weighted\hskip
4.2679ptRMSE=\sqrt{(\frac{1}{n})\sum_{i=1}^{n}(w_{i})(y_{i}-x_{i})^{2}}$
$Weighted\hskip
4.2679ptMAE=(\frac{1}{n})\sum_{i=1}^{n}(w_{i})\left|y_{i}-x_{i}\right|$

####  4.3.3 Results

The proposed approach was evaluated on leaderboard and the final test dataset
which comprised on 1200 radiographs corresponding to 300 patients each, these
datasets were not available to the participants during the challenge. The RA2
Dream baseline uses the distribution of scores in the training data to create
a random prediction. In Table 3, we describe multiple experiments and the
associated Weighted AE for overall RA damage prediction and Weighted RMSE for
joint narrowing and erosion SvH score prediction on the Leaderboard/Final test
datasets. Initial experiments were performed with a DenseNet-169 model
pretrained on ImageNet dataset which is fine tuned further on hands and feet
radiographs without joint detection and attention to predict the joint wise
and overall SvH scores, which showed a minor improvement w.r.t baseline score,
whereas when we include joint detection and attention into the same model
architecture we were able to improve the model performance significantly. We
further went on and incorporated various pretrained EfficientNet backbones
into the joint level model architecture and observed that the EfficientNet
models with joint detection and attention outperformed all the other
experiments which were executed during the challenge. Our EfficientNet based
joint models with Joint detection and attention achieved 4th place in
predicting the overall damage and 8th place in predicting the joint Narrowing
and Erosion SvH scores on the final test dataset.

Table 3: Model performance on the public/private leaderboard:  EfficientNets with attention outperforms other models in joint-wise narrowing prediction and overall damage prediction from the radiographs Experiments |  | Weighted Abs. Error
---
(Overall)
| Weighted RMSE
---
(Narrowing)
| Weighted RMSE
---
(Erosion)
Baseline | 7.023 | 1.96 | 1.728
| DenseNet-169 w/o Joint detection & Attention
---
2.979 | 1.659 | 1.487
| DenseNet-169 w. Joint Detection & Attention
---
1.465 | 1.449 | 1.394
| EfficientNets w. Joint Detection & Attention
---
1.371 | 1.433 | 1.397
| Final Performance (EfficientNet & Attention)
---
1.456 | 1.358 | 1.404

###  4.4 Attention Map Analysis

While dealing with healthcare related problems we need to be sure of what we
are predicting for a patient. In this case since we are predicting joint level
narrowing and erosion SvH scores we need to have some evidence in our model
predictions, otherwise it is tough to acknowledge that the predictions are
accurate. Here we demonstrate that the attention mechanism we introduced
within the model architecture is not only capable of boosting the model
performance but also provides a layer of explanability for the model
predictions. We extracted the attention weights and overlayed them onto the
joint images. In Fig. 5, we demonstrate the attention maps for narrowing SvH
score prediction problem for hand/feet fingers, and wrist.

(a) Feet finger

(b) Hand finger

(c) Wrist

Figure 4: Attention weights corresponding to various joints while predicting
narrowing scores from extracted fingers and wrists from the radiographs,
yellow regions demonstrate the region where the model assigns higher weights
in the input image

In Fig. 5(a), and 5(b), the joint region in the feet and hand fingers is
activated to provide the final predictions, similarly, we see that in Fig.
5(c), all wrist joints are given a higher weightage by the model while making
the predictions. Our attention map provides an inbuilt explanability component
which can aid the radiologist to believe in the model predictions.

##  5 Conclusion

In this paper, we propose an automated method for detecting joints from
radiographs and predicting the associated damage in the joints by leveraging
attention based CNN framework. Due to the lack of availability of annotations
for the the joints we propose an object detection model which is capable of
correctly identifying and localizing the joints in the radiographs which can
be leveraged for downstream tasks. Since manual grading of radiographs
requires experienced radiologists, we tackle this with an attention-based CNN
method which can accurately grade the radiographs and aid radiologists in
better patient care and research. Not only the model is able to give better
predictions but it also provides a sense of the salient regions in the joint
radiographs where the model latches on to make those predictions. While the
achieved performance is encouraging, further evaluation of this approach on
broader dataset is required to have a more reliable estimation of the model
predictions. In the future, we aim to develop joint level detector models
instead of a finger level models and employ the similar model architecture
which could provide a great deal of understanding while predicting the scores
and assist the radiologists in making their decisions, also directly comparing
the explainability of inherent attention maps and other offline explainability
methods may instill confidence in the predictions.

## References

  * Birch and Bhattacharya [2010] James T Birch and Shelley Bhattacharya.  Emerging trends in diagnosis and treatment of rheumatoid arthritis.  _Primary Care: Clinics in Office Practice_ , 37(4):779–792, 2010.
  * Combe [2009] Bernard Combe.  Progression in early rheumatoid arthritis.  _Best Practice & Research Clinical Rheumatology_, 23(1):59–69, 2009.
  * Van der Heijde et al. [1995] DMFM Van der Heijde, MA Van Leeuwen, PLCM Van Riel, and LBA Van de Putte.  Radiographic progression on radiographs of hands and feet during the first 3 years of rheumatoid arthritis measured according to sharp’s method (van der heijde modification).  1995\.
  * Renshaw and Gould [2005] Andrew A Renshaw and Edwin W Gould.  Comparison of disagreement and error rates for three types of interdepartmental consultations.  _American journal of clinical pathology_ , 124(6):878–882, 2005.
  * RA2 [2019 (accessed January, 2021] Ra2 dream challenge.  https://www.synapse.org/#!Synapse:syn20545111/wiki/597242, 2019 (accessed January, 2021).
  * Fujisawa et al. [2019] Y Fujisawa, Y Otomo, Y Ogata, Y Nakamura, R Fujita, Y Ishitsuka, R Watanabe, N Okiyama, K Ohara, and M Fujimoto.  Deep-learning-based, computer-aided classifier developed with a small dataset of clinical images surpasses board-certified dermatologists in skin tumour diagnosis.  _British Journal of Dermatology_ , 180(2):373–381, 2019.
  * Coudray et al. [2018] Nicolas Coudray, Paolo Santiago Ocampo, Theodore Sakellaropoulos, Navneet Narula, Matija Snuderl, David Fenyö, Andre L Moreira, Narges Razavian, and Aristotelis Tsirigos.  Classification and mutation prediction from non–small cell lung cancer histopathology images using deep learning.  _Nature medicine_ , 24(10):1559–1567, 2018.
  * Boudrioua [2020] Mohamed Samir Boudrioua.  Covid-19 detection from chest x-ray images using cnns models: Further evidence from deep transfer learning.  _Available at SSRN 3630150_ , 2020.
  * Woo et al. [2018] Sanghyun Woo, Jongchan Park, Joon-Young Lee, and In So Kweon.  Cbam: Convolutional block attention module.  In _Proceedings of the European conference on computer vision (ECCV)_ , pages 3–19, 2018.
  * Bello et al. [2019] Irwan Bello, Barret Zoph, Ashish Vaswani, Jonathon Shlens, and Quoc V Le.  Attention augmented convolutional networks.  In _Proceedings of the IEEE International Conference on Computer Vision_ , pages 3286–3295, 2019.
  * Chen et al. [2020] Yinpeng Chen, Xiyang Dai, Mengchen Liu, Dongdong Chen, Lu Yuan, and Zicheng Liu.  Dynamic convolution: Attention over convolution kernels.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 11030–11039, 2020.
  * He et al. [2020] Along He, Tao Li, Ning Li, Kai Wang, and Huazhu Fu.  Cabnet: Category attention block for imbalanced diabetic retinopathy grading.  _IEEE Transactions on Medical Imaging_ , 2020.
  * Snekhalatha and Anburajan [2017] U Snekhalatha and M Anburajan.  Computer-based measurements of joint space analysis using metacarpal morphometry in hand radiograph for evaluation of rheumatoid arthritis.  _International journal of rheumatic diseases_ , 20(9):1120–1131, 2017.
  * Langs et al. [2008] Georg Langs, Philipp Peloschek, Horst Bischof, and Franz Kainberger.  Automatic quantification of joint space narrowing and erosions in rheumatoid arthritis.  _IEEE transactions on medical imaging_ , 28(1):151–164, 2008.
  * Hoving et al. [2004] Jan Lucas Hoving, Rachelle Buchbinder, Stephen Hall, Gary Lawler, Peter Coombs, Stephen McNealy, Paul Bird, and David Connell.  A comparison of magnetic resonance imaging, sonography, and radiography of the hand in patients with early rheumatoid arthritis.  _The Journal of Rheumatology_ , 31(4):663–675, 2004.
  * Døhn et al. [2008] Uffe Møller Døhn, Bo J Ejbjerg, Maria Hasselquist, Eva Narvestad, Jakob Møller, Henrik S Thomsen, and Mikkel Østergaard.  Detection of bone erosions in rheumatoid arthritis wrist joints with magnetic resonance imaging, computed tomography and radiography.  _Arthritis research & therapy_, 10(1):R25, 2008\.
  * Tiulpin et al. [2018] Aleksei Tiulpin, Jérôme Thevenot, Esa Rahtu, Petri Lehenkari, and Simo Saarakkala.  Automatic knee osteoarthritis diagnosis from plain radiographs: a deep learning-based approach.  _Scientific reports_ , 8(1):1–10, 2018.
  * Xue et al. [2017] Yanping Xue, Rongguo Zhang, Yufeng Deng, Kuan Chen, and Tao Jiang.  A preliminary examination of the diagnostic value of deep learning in hip osteoarthritis.  _PLoS One_ , 12(6):e0178992, 2017.
  * Simonyan and Zisserman [2014] Karen Simonyan and Andrew Zisserman.  Very deep convolutional networks for large-scale image recognition.  _arXiv preprint arXiv:1409.1556_ , 2014.
  * Rohrbach et al. [2019] Janick Rohrbach, Tobias Reinhard, Beate Sick, and Oliver Dürr.  Bone erosion scoring for rheumatoid arthritis with deep convolutional neural networks.  _Computers & Electrical Engineering_, 78:472–481, 2019\.
  * Hirano et al. [2019] Toru Hirano, Masayuki Nishide, Naoki Nonaka, Jun Seita, Kosuke Ebina, Kazuhiro Sakurada, and Atsushi Kumanogoh.  Development and validation of a deep-learning model for scoring of radiographic finger joint destruction in rheumatoid arthritis.  _Rheumatology advances in practice_ , 3(2):rkz047, 2019.
  * Lin et al. [2017a] Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár.  Focal loss for dense object detection.  In _Proceedings of the IEEE international conference on computer vision_ , pages 2980–2988, 2017a.
  * He et al. [2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Deep residual learning for image recognition.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pages 770–778, 2016.
  * Huang et al. [2017] Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger.  Densely connected convolutional networks.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pages 4700–4708, 2017.
  * Lin et al. [2017b] Tsung-Yi Lin, Piotr Dollár, Ross Girshick, Kaiming He, Bharath Hariharan, and Serge Belongie.  Feature pyramid networks for object detection.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pages 2117–2125, 2017b.
  * Li and Ren [2019] Yixing Li and Fengbo Ren.  Light-weight retinanet for object detection.  _arXiv preprint arXiv:1905.10011_ , 2019.
  * Lin et al. [2014] Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick.  Microsoft coco: Common objects in context.  In _European conference on computer vision_ , pages 740–755. Springer, 2014.
  * Wang et al. [2019] Yuanyuan Wang, Chao Wang, Hong Zhang, Yingbo Dong, and Sisi Wei.  Automatic ship detection based on retinanet using multi-resolution gaofen-3 imagery.  _Remote Sensing_ , 11(5):531, 2019.
  * Tan and Le [2019] Mingxing Tan and Quoc V Le.  Efficientnet: Rethinking model scaling for convolutional neural networks.  _arXiv preprint arXiv:1905.11946_ , 2019.
  * Szegedy et al. [2015] Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich.  Going deeper with convolutions.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pages 1–9, 2015.
  * Huang et al. [2019] Yanping Huang, Youlong Cheng, Ankur Bapna, Orhan Firat, Dehao Chen, Mia Chen, HyoukJoong Lee, Jiquan Ngiam, Quoc V Le, Yonghui Wu, et al.  Gpipe: Efficient training of giant neural networks using pipeline parallelism.  In _Advances in neural information processing systems_ , pages 103–112, 2019.
  * Xu et al. [2015] Kelvin Xu, Jimmy Ba, Ryan Kiros, Kyunghyun Cho, Aaron Courville, Ruslan Salakhudinov, Rich Zemel, and Yoshua Bengio.  Show, attend and tell: Neural image caption generation with visual attention.  In _International conference on machine learning_ , pages 2048–2057, 2015.
  * Schlemper et al. [2019] Jo Schlemper, Ozan Oktay, Michiel Schaap, Mattias Heinrich, Bernhard Kainz, Ben Glocker, and Daniel Rueckert.  Attention gated networks: Learning to leverage salient regions in medical images.  _Medical image analysis_ , 53:197–207, 2019.
  * Bridges Jr et al. [2010] S Louis Bridges Jr, Zenoria L Causey, Paula I Burgos, B Quynh N Huynh, Laura B Hughes, Maria I Danila, Amalia Van Everdingen, Stephanie Ledbetter, Doyt L Conn, Ashutosh Tamhane, et al.  Radiographic severity of rheumatoid arthritis in african americans: results from a multicenter observational study.  _Arthritis care & research_, 62(5):624–631, 2010.
  * Ormseth et al. [2016] Michelle J Ormseth, Patricia G Yancey, Joseph F Solus, S Louis Bridges Jr, Jeffrey R Curtis, MacRae F Linton, Sergio Fazio, Sean S Davies, L Jackson Roberts, Kasey C Vickers, et al.  Effect of drug therapy on net cholesterol efflux capacity of high-density lipoprotein–enriched serum in rheumatoid arthritis.  _Arthritis & rheumatology_, 68(9):2099–2105, 2016.
  * Sun et al. [2021] Dongmei Sun, Thanh M. Nguyen, Robert J. Allaway, Jelai Wang, Verena Chung, Thomas V Yu, Michael Mason, Isaac Dimitrovsky, Lars Ericson, Hongyang Li, Yuanfang Guan, Ariel Israel, Alex Olar, Balint Armin Pataki, RA2 DREAM Challenge Community, Gustavo Stolovitzky, Justin Guinney, Percio S. Gulko, Mason B. Frazier, James C. Costello, Jake Y. Chen, and S. Louis Bridges.  A crowdsourcing approach to develop machine learning models to quantify radiographic joint damage in rheumatoid arthritis.  _medRxiv_ , 2021.  doi: 10.1101/2021.10.25.21265495.  URL https://www.medrxiv.org/content/early/2021/10/26/2021.10.25.21265495.
  * Kingma and Ba [2014] Diederik P Kingma and Jimmy Ba.  Adam: A method for stochastic optimization.  _arXiv preprint arXiv:1412.6980_ , 2014.


# Analyzing Machine Learning Approaches for Online Malware Detection in Cloud

Jeffrey C Kimmell1, Mahmoud Abdelsalam2, and Maanak Gupta3
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>13Dept. of Computer Science, Tennessee Technological University, Cookeville,
Tennessee 38505, USA
2Dept. of Computer Science, Manhattan College, Riverdale, NY, USA

###### Abstract

The variety of services and functionality offered by various cloud service
providers (CSP) have exploded lately. Utilizing such services has created
numerous opportunities for enterprises infrastructure to become cloud-based
and, in turn, assisted the enterprises to easily and flexibly offer services
to their customers. The practice of renting out access to servers to clients
for computing and storage purposes is known as Infrastructure as a Service
(IaaS). The popularity of IaaS has led to serious and critical concerns with
respect to the cyber security and privacy. In particular, malware is often
leveraged by malicious entities against cloud services to compromise sensitive
data or to obstruct their functionality. In response to this growing menace,
malware detection for cloud environments has become a widely researched topic
with numerous methods being proposed and deployed. In this paper, we present
online malware detection based on process level performance metrics, and
analyze the effectiveness of different baseline machine learning models
including, Support Vector Classifier (SVC), Random Forest Classifier (RFC),
K-Nearest Neighbor (KNN), Gradient Boosted Classifier (GBC), Gaussian Naive
Bayes (GNB) and Convolutional Neural Networks (CNN). Our analysis conclude
that neural network models can most accurately detect the impact malware have
on the process level features of virtual machines in the cloud, and therefore
are best suited to detect them. Our models were trained, validated, and tested
by using a dataset of 40,680 malicious and benign samples. The dataset was
complied by running different families of malware (collected from VirusTotal)
in a live cloud environment and collecting the process level features.

##  I Introduction and Motivation

Cloud computing’s convenience and scalability have made it the go to resource
for many entities in both private and public sectors. One of the major cloud
characteristics is offering resources on-demand, if and when needed using the
pay-as you go model. In general, IT specialists at enterprises have to control
and manage these resources which hinders the advantages offered by the cloud.
Cloud automation tools have become the norm where IT personnel are able to
automatically provision resources in the cloud. Such automation is achieved
through tools (e.g., Puppet111Puppet. https://puppet.com/ and Chef222Chef.
https://www.chef.io/) by writing configuration scripts that are able to
create, modify, and delete resources in the cloud. Just as such orchestration
tools offer huge benefits to DevOps teams, they widen the security attack
surface. In particular, VMs are often spawned using automatic configuration
tools which means that a large group VM are similarly configured, if not exact
copies. The inherent redundancies in these VMs could allow for malware to
easily propagate across VMs, especially if there are flaws in these
configuration scripts. The repercussions of compromising a group of VMs far
outweighs those of a single compromised VM. Cloud infrastructure requires
considerably major security implementations due to its inherent complexity and
dynamic environment where threats are always changing and evolving. For the
same reason, developing malware detection methods that are both accurate and
fast is imperative [1].

Malware is a major threat to cloud infrastructures. Multiple malware detection
methods have been proposed with pros and cons. Static malware detection [2, 3,
3, 4] is a popular method where the signature of an executable is analyzed and
compared to a database of known malware signatures. Attackers have tried to
limit the effectiveness of static analysis by implementing techniques such as
obfuscation and packing. In addition, static malware analysis is limited to
known malware executables and is unable to detect the ever-evolving zero-day
malware. These two major limitations have led to extensive research on
behavioral malware detection methods. Dynamic and online malware detection are
two behavioral based methods. Dynamic malware detection methods work by
running the malware executables in a secure environment, such as a sandbox and
analyzing their behavior. By doing this, the detection system is able to
analyze novel zero-day malware since it is not relying on previously known
signatures but the actual behavior of the executable. However, attackers have
been able to implement malware that can detect the use of tools such as a
sandbox, and cease behaving maliciously in order to avoid its detection.
Dynamic and static methods also share the same limitation where the detection
system focus on identifying malware in the given executables before they are
run on actual systems. However, it is common for malware to get into a system
through vulnerabilities, hence bypassing these primitive detection approaches.
Online malware detection [5, 6, 7, 8] focuses on the behavior of a machine
that it is trying to protect from malware. Rather than analyzing executables
and their behavior, online methods monitor the performance of the entire
virtual machine, and raises an alert if traces of malicious behavior is found
at any time. As such, online malware detection methods are considered
continuous real-time detection system, and overcomes the limitations of static
and dynamic malware detection approaches.

Machine Learning (ML) and neural networks techniques are widely utilized in
order to capture the behavior of malware in an accurate and efficient way [9].
This is due to the models’ abilities to quickly process a significant amount
of data generated by a VM to classify executables as malicious or benign.
Online malware detection techniques are significantly impacted by the features
chosen to capture the behavior of the malware present in a particular machine.
For example, several works [2, 10, 11] are using system calls (the most widely
used); however, it is very resource consuming and data can only be fetched
through on-host collecting agent. Only few works [5, 6, 7, 1, 8, 12, 13] use
resource utilization metrics (also known as performance metrics) due to the
fact that they are less expressive than system calls in terms of capturing the
low-activity malicious behavior. However, performance metrics are more
suitable to cloud environments since they are cheaper and can easily be
fetched from the hypervisor (e.g., VMs introspection [14]).

In this paper, we analyze and compare the effectiveness of different online
malware detection approaches that utilize process-level performance metrics.
We provide an in depth analysis of various machine learning models which will
work as a baseline for other works which focus primarily on one machine
learning model. This is critical to motivate the use of expensive deep
learning models which require huge amounts of training data and to prove their
efficacy with respect to more fundamental machine learning models. To our
understanding and literature review, this is the first work focusing on
analyzing the efficiency of baseline machine learning models, which is
important to justify the use of expensive deep learning techniques.

The main contributions of this paper are as follows:

  * •

We analyze the effectiveness of different machine learning models for online
malware detection.

  * •

We demonstrate how the set of processes running in a VM can be represented as
a sequence of system features.

  * •

We conjecture that a Convolutional Neural Network (CNN) model is better suited
for malware detection compared to traditional ML methods.

The remainder of the paper is structured as follows. Section II discusses the
related works regarding cloud malware detection and use of various machine
learning models. We also elaborate different ML models used in our work.
Section III discusses the experimental cloud set up and methodology. The
results generated by each of the models are discussed in Section IV. Section V
offers comparison and analysis of different ML approaches used followed by
limitations in Section VI. Section VII summarizes our work.

##  II Background

We will outline related work in malware detection and summarize different ML
models used in our work.

###  II-A Related Works

TABLE I: This table shows the differences between our work and related works. The sections in red indicate a difference in features used, environment that the solution was tested in, or focus of the paper. A ✓ indicates that a particular paper possesses this attribute or model, whereas a blank cell indicates an absence of this attribute or model. | Features | Domain | Focus | Models Used
---|---|---|---|---
Paper |  API Calls |  Performance Metrics |  System Calls |  Performance Counters |  Memory Features |  Cloud Environment |  Traditional Host-Based Environment |  Dynamic Malware Detection |  Online Malware Detection |  Anomaly Detection |  KNN |  Naive Bayes |  Neural Network |  Random Forest |  Boosted Trees |  SVC |  Clustering |  Decision Trees |  No Machine Learning
Firdausi et al. 2010 [15] |  |  | ✓ |  |  |  | ✓ | ✓ |  |  | ✓ | ✓ |  |  |  | ✓ |  | ✓ |
Azmandian et al. 2011 [16] |  |  | ✓ |  |  | ✓ |  |  | ✓ | ✓ | ✓ |  |  |  |  |  | ✓ |  |
Guan et al. 2012 [17] |  | ✓ |  |  |  | ✓ |  |  | ✓ | ✓ |  | ✓ |  |  |  |  |  | ✓ |
Pannu et al. 2012 [18] |  | ✓ |  |  |  | ✓ |  |  |  | ✓ |  |  |  |  |  | ✓ |  |  |
Demme et al. 2013 [19] |  |  |  | ✓ |  |  | ✓ |  | ✓ |  | ✓ |  |  | ✓ |  |  |  | ✓ |
Pirscoveanu et al. 2015 [20] |  | ✓ |  |  |  |  | ✓ | ✓ |  |  |  |  |  | ✓ |  |  |  |  |
Watson et al. 2015 [1] |  | ✓ |  |  |  | ✓ |  |  | ✓ | ✓ |  |  |  |  |  | ✓ |  |  |
Luckett et al. 2016 [11] |  |  | ✓ |  |  |  | ✓ | ✓ |  |  |  |  | ✓ |  |  |  |  |  |
Fan et al. 2016 [21] | ✓ |  |  |  |  |  | ✓ | ✓ |  |  | ✓ |  |  |  |  |  |  |  |
Tobiyama et al. 2016 [9] | ✓ |  |  |  |  |  | ✓ | ✓ |  |  |  |  | ✓ |  |  |  |  |  |
Abdelsalam et al. 2017[6] |  | ✓ |  |  |  | ✓ |  |  | ✓ | ✓ |  |  |  |  |  |  | ✓ |  |
Xu et al. 2017 [22] |  |  |  |  | ✓ |  | ✓ |  | ✓ |  |  |  |  | ✓ |  |  |  |  |
Abdelsalam et al. 2018 [7] |  | ✓ |  |  |  | ✓ |  |  | ✓ |  |  |  | ✓ |  |  |  |  |  |
Dawson et al. 2018 [10] |  |  | ✓ |  |  | ✓ |  |  | ✓ | ✓ |  |  | ✓ |  |  |  |  |  | ✓
Joshi et al. 2018 [23] |  | ✓ |  |  |  |  | ✓ | ✓ |  |  |  |  |  | ✓ |  |  |  |  |
Our Approach |  | ✓ |  |  |  | ✓ |  |  | ✓ |  | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |  |  |

There has been substantial work in the field of malware detection. Most
recently, approaches that rely on machine learning techniques have gained
traction. The high increase in cloud activity has also called for more
attention towards methods that are specific to the cloud environment [7, 6, 5,
1, 10, 20, 21]. Table I shows some of the closely related work. We categorize
the work with respect to the focus of the paper, the targeted environment, and
the features used for detection as well as the ML algorithms used.

Dynamic Malware Detection. Dynamic malware detection approaches focus on
running malware executables in a sandbox and closely monitor its behavior or
system wide behavior. Most works target traditional host-based systems.
Research in [15, 11] utilize system calls as features to train classical
machine learning models (i.e. KNN, NB, SVC and DT) and neural networks,
respectively. Other works [9, 21, 20] analyze the effectiveness of using CNN,
RF and KNN models for malware detection and rely on features extracted from
API calls. In addition, Joshi et al. [23] use the random forest classifier and
monitor a VM’s process behavior. However, their analysis of these approaches
is not analyzed on a cloud environment. The main limitation of such approaches
is the fact that they are performed in an isolated environment neglecting the
unique cloud topology, including the infrastructure and its network
communication channels. Even though dynamic analysis approaches can be adopted
and used in online settings, collecting real-time metrics generated from the
cloud environment is essential for cloud malware detection.

Online Malware Detection. Unlike static and dynamic analysis approaches where
an executable is analyzed or monitored before it runs on a system, online
malware detection approaches focus on continuously monitoring the entire
systems under the assumption that a malware will eventually make its way into
the system. The authors in [19] introduced a malware detection method that
utilizes performance counters and [22] proposed the use of memory features;
however, both of these works are targeting traditional host-based environment.
Other works specifically target the cloud. Abdelsalam et al. [7] presented a
CNN solution that focused on process level performance metrics with a
relatively successful accuracy score of 90%. However, this work only examines
CNN and does not provide a baseline of comparison with respect to traditional
machine learning algorithm, which we aim to accomplish in this paper. In
addition, we also categorize anomaly detection based approaches as online
techniques, since they naturally focus on continuous monitoring of their
target systems. Pannu et al. [18] use cloud performance metrics features and
analyzed the effectiveness SVM and Gaussian based approaches. Even though
their work focused on general anomaly detection within the cloud, it can be
easily adopted and tailored to detect malware specifically. Similarly, Guan et
al. [17] considered anomaly detection within a cloud environment where they
analyzed system calls based on an ensemble of Bayesian predictors and decision
trees. Their work also focused on cloud computing systems failure, not
specifically malware. Other anomaly detection based approaches are focused on
malware. Azmandian et al. [16] presented an intrusion detection system using
system calls as features. Abdelsalam et al. [6] proposed a novel k-means
clustering algorithm for detection purposes. This approach succeeded in
detecting highly active malware, but was not successful in detecting low
activity malware. Dawson et al. [10] fetch API calls through hypervisor to be
used as features and use a non linear phase-space algorithm to detect
anomalous behavior. Watson et al. [1] use performance metrics to build a one
class SVC; however, the authors experimented on highly active malware which is
easy to detect.

In this paper, we aim to address the following limitations:

  1. 1.

Unlike traditional host-based approaches in [15, 19, 20, 11, 21, 9, 22, 23],
we aim to focus on developing a cloud-specific approach. Our experiment
deployment, which consists of a commonly used three-tier web architecture,
gives our collected data the added benefit of being generated in an extremely
realistic cloud environment. The different layers of this architecture allows
the utilization of the cloud topology and provides an in-depth view at how a
real system could be affected by malware.

  2. 2.

Unlike dynamic analysis approaches in [15, 20, 11, 21, 9], we aim to focus on
developing an online malware detection approach that is well suited for cloud
environments.

  3. 3.

Unlike the work in [15, 16, 19, 11, 21, 9, 22, 10], we focus on performance-
level metrics which is more practical for cloud than costly features like
system or API calls.

  4. 4.

Unlike the work in [1], we aim to focus on a broad range of malware by using
low-active malware from seven different categories.

  5. 5.

Unlike the work in [7], we aim to provide a baseline comparison by employing
various traditional machine learning algorithms to convey the importance of
using deep learning algorithms for online malware detection in cloud.

###  II-B Baseline Machine Learning Models

Here we outline different ML models used in our analysis.

####  II-B1 Support Vector Classifier (SVC)

SVC are supervised learning models that are used for classification. SVC’s
ability to use a non-linear kernel gives this method the ability to
efficiently perform non-linear classifications. This also reduces the
computational power required to calculate relationships in infinite
dimensions. There is not always a possible linear classification discernible
between features, so finding higher dimensional relationships between the
supplied features allows SVC to make classifications that other methods, such
as logistic regression, would not be able to make.

####  II-B2 Random Forest Classifier (RFC)

RFC are supervised learning models used for classification. An RFC works by
fitting a collection of decision trees[24], and is therefore considered an
ensemble learning method since it uses a collection of classifiers [25]. RFCs
choose the best parameter at each node at random as opposed to decision trees
where the best parameter is selected based on all of the features [23]. This
gives RFCs better scalability as well as reducing the risk of overfitting.

####  II-B3 Nearest Neighbor

Also known as k-Nearest Neighbor (KNN), it is a supervised learning method of
classification that relies on measuring the distances of samples that have
close proximity. KNN assumes that samples with the same classification will be
closer in distance, and uses this assumption to classify a new sample based on
the k closest neighbors.

####  II-B4 Gradient Boosted Trees

Gradient Boosted Trees, or Gradient Boosted Classifiers (GBC), like RFC, is an
ensemble supervised learning method. GBCs work by creating many decision trees
that handle specific decisions. A weak learner is considered a model that is
only slightly better than guessing. However, weak learners are often able to
make correct decisions on a very specific portion of the sample. The idea is
to create an ensemble of enough weak learners so that the model as a whole can
uses the decisions from the various learners to generate an overall accurate
classification.

####  II-B5 Naive Bayes

A Naive Bayes classifier generates classifications using the Bayes Theorem.
Bayes Theorem is a method used to calculate conditional probability based off
a set of features, however it requires a large pool of computational
resources. Bayes Theorem assumes that all features are dependent on one
another, this is what leads to the theorem becoming so computationally
demanding. To remedy this, a simplified or Naive method was created by
assuming that each of the features are independent; this assumption allows the
theorem to be simplified which therefore reduces the required computational
resources.

####  II-B6 CNN

Convolutional Neural Networks are most commonly used for image recognition or
use cases that involve visual data. We can imitate this with our data by
shaping it so that it is represented in a two dimensional array, similar to an
image. The CNN model that we employ in our paper is DenseNet-121 model, which
is one of the state-of-the-art CNN models.

DenseNets [26] attempt to solve the issue of the ”vanishing gradient.” This
problem arises when the neural network becomes so deep that the standard back-
propagation fails to update the neurons at the early layers of the network
from the changes made at the output. Because of this issue, neural networks
were limited in deepness and complexity. DenseNets work around this issue by
creating more channels that connect the hidden layers. In DenseNets, the
outputs of every layer are passed to subsequent layers. Because of this,
DenseNets do not need as many feature maps at every hidden layer since these
feature maps are being used by every subsequent layer and not just the
successor layer. In addition, DenseNets borrow the identity mapping feature
from residual networks. This feature allows the gradient to flow through the
model easily via the use of skip-connections. DenseNets are comprised of dense
blocks and connected with transition layers that are made of a convolution and
pooling layer. For simplicity, we will refer to it as CNN for the remainder of
the paper.

##  III Methodology and Experimental Setup

###  III-A Experimental Setup

Figure 1: Experimental Cloud Testbed Setup

####  III-A1 Testbed

Simulating real world data is an imperative aspect to creating reliable
malware prediction models. In order to achieve this, we set up a cloud
environment with traffic to emulate real world cloud behavior. OpenStack, a
popular cloud platform, was installed on the testbed and consisted of a single
control node and four compute nodes as shown in Figure 1. The control node is
responsible for tasks such as the dashboard, storage, network, identity, and
computing. The compute nodes only handle computing services, and each compute
node is also supplied with agents for networking, polling, and collecting.
Allowing malware to behave naturally is also key in the data collection
process. To ensure the malware was able to behave as intended, all of the
experiments to collect system features were conducted on machines that were
connected to the Internet. This was done since some malware has the capability
to detect a closed environment such as a sandbox, a limitation of dynamic
malware analysis approaches. If a closed environment is detected, the malware
may try to act as a benign process or may cease its behaviour as to not be
detected. The Internet connection ensures that malware is able to communicate
with its necessary command and control servers needed for malware to act in
some cases. In addition, all firewalls and antivirus were disabled.

Figure 2: Experiment Deployment

####  III-A2 Malware Samples

The malware that was injected during our experiments were obtained from
VirusTotal333https://www.virustotal.com/gui/. 113 samples were gathered in
total and were chosen at random consisting of diverse malware families such as
DoS, Backdoor, Trojan, Virus, among others.

####  III-A3 Experiment Deployment

In order to simulate a real world scenario, a three-tiered web architecture
was used as shown in Figure 2. This architecture consisted of web-servers,
application servers, and a database server. A front load balancer is tasked
with handling and distributing clients requests to the appropriate web
servers. An internal load balancer is used to connect web servers to the
application servers and to distribute requests among the application servers,
and the application servers are all connected to a single database server. An
auto scaling policy is also utilized which is based on the average CPU usage
and is applied independently to both the web and application servers. The
policy states that if the average CPU utilization of all VMs belonging to the
web or application layer exceeds 70%, new VMs are created and attached to the
corresponding load balancer. Inversely, if the average CPU utilization falls
below 40%, VMs are deleted to reduce resource usage. During our experiments,
anywhere between 2 to 10 servers were spawned at each layer, depending on the
overall traffic load. In order to uphold integrity, the traffic/requests were
generated based on an ON/OFF Pareto distribution, this is done to mimic the
realistic dynamic behavior of cloud infrastructures. A main control VM is used
for keeping the malware executables in the database, injecting a single
malware sample into an application server at a specific time, and
deploying/destroying the experiment stack. OpenStack Heat orchestration
service is used to deploy/destroy experiment stacks using yaml scripts.

Figure 3: Experiment Timeline

####  III-A4 Unique Processes

We collect system features (e.g. memory, cpu, input/output etc.) from all
process that are running on the VM at certain times. Many of these processes
are short lived, and also have their process IDs reassigned by the operating
system. Due to these characteristics, it is often difficult to analyze the
behavior of these processes. To remedy this, we utilize a method known as
unique processes inspired by [7] which reduces dynamism. Traditional operating
systems identify processes with a pid, whereas unique processes consider the
actual behavior of a process and is identified using a tuple of two elements,
process name and the command used to run the process. Processes that share
common values in both fields within the tuple are clustered by taking the
average of their measurements. By using this approach, we are also able to
reduce the total number of processes within a given sample.

####  III-A5 Data Collection

A total of 113 experiments were conducted each using a different malware
executable for a total of 40,680 samples. Each of the experiments lasted for 1
hour and is split up into benign and infected phases as shown in Figure 3. The
first 30 minutes is the benign phase, during this time there is no malware
injected into the machine. Between minute 30 and 40, a single malware is
injected into one of the application servers. The malware injection and
execution times varies which adds a more dynamic nature to the experiments and
ensures that a rigid injection and execution would not skew the results.
Minute 40 is referred to as the malicious phase. During this time, malware is
openly running on the machine. Data samples are collected every 10 seconds,
this results in a total of 360 samples in total for each experiment which are
stored in the database. After the experiment is completed, the main control VM
destroys the entire experiment stack in order to prevent contamination between
experiments.

####  III-A6 Model Training

We used the scikit-learn444scikit-learn. https://scikit-learn.org/stable/
library for implementing our classical machine learning models, and
Keras555Keras. https://keras.io/, a high-level API that runs on top of
TensorFlow666Tensorflow. https://www.tensorflow.org/, for implementing our CNN
model. All the ML models were trained on a Windows machine equipped with an
AMD Ryzen 5 2600 processor and 16 GB of RAM.

###  III-B Evaluation

We use five standard metrics to measure the performance of different models,
accuracy, precision, recall, and F1 as defined below where TP (True Positive),
TN (True Negative), FP (False Positive), and FN (False Negative).

$\displaystyle Accuracy$ $\displaystyle=\frac{TP+TN}{TP+TN+FP+FN}$
$\displaystyle Precision$ $\displaystyle=\frac{TP}{TP+FP}$ $\displaystyle
Recall$ $\displaystyle=\frac{TP}{TP+FN}$ $\displaystyle F1~{}Score$
$\displaystyle=2\times\frac{Precision\times Recall}{Precision+Recall}$

##  IV Results

TABLE II: Detailed Performance Results for the Different ML Models Model | Accuracy | Precision | Recall | F1
---|---|---|---|---
CNN | 92.9% | 100% | 84.6% | 91.5%
SVC | 87.56% | 86.2% | 80.91% | 83.47%
RFC | 89.36% | 99.71% | 72.80% | 84.15%
KNN | 72.34% | 66.6% | 57.67% | 61.81%
GBC | 81.47% | 75.22% | 77.87% | 76.57%
GNB | 58.09% | 48.06% | 98.57% | 64.61%

Table II shows the performance metric scores from each of our models. In
regards to overall performance, the CNN model outperformed every other model
with an F1 score of 91.5%. We use the F1 score as a basis for overall
performance since it takes the precision and recall metrics into consideration
and is therefore better at describing a model’s overall performance than the
accuracy metric. The RFC and SVC models had the next highest F1 scores with
84.15% and 83.47% respectively. The GBC, GNB, and KNN models did not perform
as well with F1 scores of 76.57%, 64.61%, and 61.81%.

The recall metric quantifies the percentage of the infected samples that were
detected, and the model that scored the highest in this metric was the GNB
model with a score of 98.57%. However there is a caveat to this, its precision
score, which measures how many of the samples that were labeled as infected
were actually infected, was very low with a score of 48.06%. The CNN model
achieved a recall score of 84.6% and a precision score of 100%. The next best
model in regards to recall was the SVC model with a score of 80.91% and a
precision score of 86.2%. The RFC model achieved a recall score of 72.80% but
also had a very high precision score of 99.71%. The KNN model performed the
worst, which was indicated earlier by the low F1 score, with a recall score of
57.67% and a precision score of 66.6%. The findings when analyzing the
accuracy scores are consistent with our other findings, the CNN model
outperforms the other models with the RFC and SVC models not too far behind.

Figure 4 shows the ROC curves and AUC scores for each of our models. Once
again, the CNN model has the best AUC score of more than 99%. The RFC and SVC
models follow closely scoring a 94% and a 93% respectively. The GBC model was
able to score and AUC of 90% whereas the KNN model scored a 77% and the GNB
model performed the worst with a score of 65%. These results are consistent
with the results depicted in Table II.

Another important metric to consider when comparing the performance of various
models is the training time of the model, as shown in Table III. Neural
networks require models that use this methodology be trained for a certain
number of epochs. If the model is trained for too many epochs overfitting
could occur, but if the model is not trained for enough epochs, underfitting
will occur. As such, we stop training if the validation accuracy is not
increasing for specific number of epochs and we choose the model with the
highest validation accuracy achieved during these epochs. The CNN model took
1683 seconds to train and required a total of 32 epochs. Other models do not
train based on a certain number of epochs, therefore, the same technique
doesn’t apply when training the remainder of the models. In such case, the SVC
model took the longest to train by far at 989 seconds. The GBC model took 167
seconds, the KNN model took 28 seconds, the RFC model took 20 seconds, and the
GNB model only took 2 second to train. Note that the reported times solely
include training time, excluding the time it takes to read and load the data
samples.

Figure 4: Receiver Operating Characteristic (ROC) Curves

##  V Comparison and Analysis

TABLE III: Time Cost for the Models Model | Time to Train (s) | Detection Time (ms)
---|---|---
CNN | 1683 | 164
SVC | 989 | 11
RFC | 20 | 3900
KNN | 28 | 118
GBC | 167 | 40
GNB | 2 | .9
Figure 5: Performance Metrics Comparison for Different Machine Learning Models

The superior metrics generated by the CNN model clearly indicate that this
model is the best suited for our use case. The comparison of all the models
can be found in figure 5. The CNN model was able to detect 84.6% of all of the
infected samples while also not falsely labeling benign samples as infected.
In malware detection, while detecting every instance of malware is ideal, a
high number of false positives can be just as much of a disruption as malware.
That is why the high recall rate achieved by the GNB model is not as promising
as it may seem. Its extremely low precision score of 48.06% indicates that
this model generated a high number of false positives. In fact, 52% of the
samples labeled as infected were actually benign. In a business case, having a
model that generated these many false positives could hinder day to day
activities by labeling essential, benign processes as malware. The SVC and RFC
models produced similar accuracy and precision scores but differed in the
recall and precision metrics. The SVC model was able to detect 80% of the
total infected samples, but also has a lower precision score of 86%. The RFC
model had a lower recall score of 72% but generated nearly no false positives
with a precision score of 99%. These models will need security professionals
to make a decision when implementing a malware detection system. There could
be a situation where a company is willing to have a lower recall rate in
exchange for nearly no false positives to ensure that the malware detection
implementation does not severely hinder necessary activities. On the other
hand, there may be a security critical use case where a high number of false
negatives is very detrimental. In that case, it may be beneficial with a
system which is able to detect nearly all malwares even if a large portion of
those classifications are false positives.

###  V-A Cost Analysis

The time it takes to train these models, as shown in Table III, can also have
an affect on which use case each model will be suitable for. There is a clear
pattern of the more successful models requiring more time to train. The CNN
model took the longest to train by far but also outperformed every other model
by a convincing margin. The RFC model gave promising results with an F1 score
of 84% and also only required 20 seconds to train. The RFC model reigns
supreme of the SVC model in regards to time since the SVC model required over
900 seconds to train while still yielding similar results to the RFC model. In
a general sense, it is usually worth sacrificing some time in order to train
models that are as accurate as possible. That is why deploying a deep learning
method such as the CNN model is preferred to the faster trained models such as
SVC and RFC. Once these models are trained and deployed for online detection,
an important aspect is how long will the models take to decide whether a given
sample is benign or malicious. Detection time, also found in Table III, shows
how long it took each model to classify a single sample. This metric indicates
how fast each model will be able to process the input data and correctly
classify an infected sample as such. The GNB model has the fastest detection
time, however this is due to this model having a high false positive rate
indicated by its low precision score. The RFC is the slowest model by far,
this could be due to the input data having to parse through the various trees
that make up the RFC which would cause this model to take longer to generate a
prediction. A faster detection time is of course preferred, however if a model
generates fast predictions that are incorrect, the benefit of a faster model
is overshadowed by its inability to produce accurate results.

###  V-B Overall Analysis

Despite its longer training time, the CNN model has proven to be the optimal
model in our use case. The CNN model was able to achieve the highest metric
scores. The CNN model achieved high, balanced values between all of the
metrics indicating that this model is able to correctly classify our samples
as benign or infected, more so than the other models. The CNN model’s longer
train time can be attributed to its deep architecture. This model utilized the
state of the art DenseNet-121 model, indicating that it contains 121 different
layers within the network.

##  VI Limitations

One limitation of our work is due to the relatively small number of malware
samples used. We conducted 113 different experiments each with a different
type of malware, but conducting more experiments with a wider range of malware
could give us a better look into how malware affects the behavior of VMs in a
cloud environment. Another limitation lies in the assumption that each VM can
only be infected by a single malware, which helps in simplifying our analysis.
In practice, a machine can be infected with multiple malware at the same time.
That being said, our work aims to provide a fundamental step towards a more
rigorous and complex analysis for multiple malware infection. Further work is
needed in order to determine if our approach would be feasible given a
situation where a VM is infected by multiple malwares.

In addition, the use of the unique processes approach could allow malware
behavior to go unnoticed. Since a unique process sample is the average of a
collection of processes sharing the same name and command line, a malware that
mimics these same attributes will be counted within the average of this
sample. This is a common drawback to methodologies that utilize meta-stats
(e.g. average, standard deviation, etc.) However, the drawback of using meta-
stats is confined to each unique process independently. This makes our
approach partially immune to the meta-stats limitation as opposed to other
approaches that use meta-stats of the entire system.

##  VII Conclusion

In this paper we analyzed a variety of machine learning methods in order to
determine which method is best for online malware detection in cloud. We find
that, although it takes the longest to train, the DenseNet-121 (CNN) model has
the best overall performance. The SVC and RFC models produced promising
results that are not too far behind those produced by the CNN model, as well
as being much quicker to train. However, when it comes to malware detection,
taking the time to train a more accurate model is mostly preferred. The
remaining models, KNN, GBC, and GNB, simply could not compete with the others.
Due to the CNN model’s success, it can be concluded that deep learning models
are more adept at detecting malware within our dataset in cloud IaaS.

## Acknowledgement

This research is partially supported by NSF Grants 2025682 at Tennessee
Technological University and 2025686 at Manhattan College.

## References

  * [1] M. R. Watson, A. K. Marnerides _et al._ , “Malware detection in cloud computing infrastructures,” _IEEE Transactions on Dependable and Secure Computing_ , vol. 13, no. 2, pp. 192–205, 2015.
  * [2] M. Alazab, S. Venkatraman _et al._ , “Zero-day malware detection based on supervised learning algorithms of api call signatures,” 2010.
  * [3] A. Shalaginov, S. Banin, A. Dehghantanha, and K. Franke, “Machine learning aided static malware analysis: A survey and tutorial,” in _Cyber Threat Intelligence_. Springer, 2018, pp. 7–45.
  * [4] S. A. Roseline, S. Geetha, S. Kadry, and Y. Nam, “Intelligent vision-based malware detection and classification using deep random forest paradigm,” _IEEE Access_ , vol. 8, pp. 206 303–206 324, 2020.
  * [5] M. Abdelsalam, R. Krishnan, and R. Sandhu, “Online malware detection in cloud auto-scaling systems using shallow convolutional neural networks,” in _IFIP Annual Conference on Data and Applications Security and Privacy_. Springer, 2019, pp. 381–397.
  * [6] ——, “Clustering-based IaaS cloud monitoring,” in _IEEE International Conference on Cloud Computing (CLOUD)_ , 2017, pp. 672–679.
  * [7] M. Abdelsalam, R. Krishnan _et al._ , “Malware detection in cloud infrastructures using convolutional neural networks,” in _IEEE International Conference on Cloud Computing (CLOUD)_ , 2018, pp. 162–169.
  * [8] A. McDole _et al._ , “Analyzing CNN based behavioural malware detection techniques on cloud IaaS,” in _International Conference on Cloud Computing (CLOUD)_. Springer, 2020, pp. 64–79.
  * [9] S. Tobiyama _et al._ , “Malware detection with deep neural network using process behavior,” in _IEEE Annual Computer Software and Applications Conference (COMPSAC)_ , vol. 2, 2016, pp. 577–582.
  * [10] J. A. Dawson _et al._ , “Phase space detection of virtual machine cyber events through hypervisor-level system call analysis,” in _IEEE International Conference on Data Intelligence and Security_ , 2018, pp. 159–167.
  * [11] P. Luckett _et al._ , “Neural network analysis of system call timing for rootkit detection,” in _IEEE Cybersecurity Symposium_ , 2016.
  * [12] A. McDole _et al._ , “Deep Learning Techniques for Behavioral Malware Analysis in Cloud IaaS,” in _Malware Analysis using Artificial Intelligence and Deep Learning_. Springer, pp. 269–285.
  * [13] J. C. Kimmel _et al._ , “Recurrent neural networks based online behavioural malware detection techniques for cloud infrastructure,” _IEEE Access_ , vol. 9, pp. 68 066–68 080, 2021.
  * [14] T. Garfinkel _et al._ , “A virtual machine introspection based architecture for intrusion detection.” in _NDSS_ , vol. 3, no. 2003, pp. 191–206.
  * [15] I. Firdausi _et al._ , “Analysis of machine learning techniques used in behavior-based malware detection,” in _IEEE Int. conference on advances in computing, control, and telecommunication technologies_ , 2010.
  * [16] F. Azmandian _et al._ , “Virtual machine monitor-based lightweight intrusion detection,” _ACM SIGOPS Operating Systems Review_ , vol. 45, 2011\.
  * [17] Q. Guan, Z. Zhang, and S. Fu, “Ensemble of bayesian predictors and decision trees for proactive failure management in cloud computing systems,” _Journal of Communications_ , vol. 7, no. 1, pp. 52–61, 2012.
  * [18] H. S. Pannu, J. Liu, and S. Fu, “Aad: Adaptive anomaly detection system for cloud computing infrastructures,” in _IEEE Symposium on Reliable Distributed Systems_ , 2012, pp. 396–397.
  * [19] J. Demme _et al._ , “On the feasibility of online malware detection with performance counters,” _ACM SIGARCH Computer Architecture News_ , vol. 41, no. 3, pp. 559–570, 2013.
  * [20] R. S. Pirscoveanu _et al._ , “Analysis of malware behavior: Type classification using machine learning,” in _IEEE Int. conference on cyber situational awareness, data analytics and assessment_ , 2015, pp. 1–7.
  * [21] Y. Fan, Y. Ye, and L. Chen, “Malicious sequential pattern mining for automatic malware detection,” _Expert Systems with Applications_ , vol. 52, pp. 16–25, 2016.
  * [22] Z. Xu _et al._ , “Malware detection using machine learning based analysis of virtual memory access patterns,” in _Design, Automation & Test in Europe Conference & Exhibition (DATE)_. IEEE, 2017, pp. 169–174.
  * [23] S. Joshi _et al._ , “Machine learning approach for malware detection using random forest classifier on process list data structure,” in _Proc. of the Int. Conference on Information System and Data Mining_ , 2018, pp. 98–102.
  * [24] L. Breiman, J. Friedman, C. J. Stone, and R. A. Olshen, _Classification and regression trees_. CRC press, 1984\.
  * [25] L. Breiman, “Random forests,” _Machine learning_ , vol. 45, no. 1, pp. 5–32, 2001.
  * [26] G. Huang, Z. Liu, L. Van Der Maaten, and K. Q. Weinberger, “Densely connected convolutional networks,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2017, pp. 4700–4708.