Deep Neural Networks based Meta-Learning for Network Intrusion Detection
The digitization of different components of industry and inter-connectivity
among indigenous networks have increased the risk of network attacks. Designing
an intrusion detection system to ensure security of the industrial ecosystem is
difficult as network traffic encompasses various attack types, including new
and evolving ones with minor changes. The data used to construct a predictive
model for computer networks has a skewed class distribution and limited
representation of attack types, which differ from real network traffic. These
limitations result in dataset shift, negatively impacting the machine learning
models' predictive abilities and reducing the detection rate against novel
attacks. To address the challenges, we propose a novel deep neural network
based Meta-Learning framework; INformation FUsion and Stacking Ensemble
(INFUSE) for network intrusion detection. First, a hybrid feature space is
created by integrating decision and feature spaces. Five different classifiers
are utilized to generate a pool of decision spaces. The feature space is then
enriched through a deep sparse autoencoder that learns the semantic
relationships between attacks. Finally, the deep Meta-Learner acts as an
ensemble combiner to analyze the hybrid feature space and make a final
decision. Our evaluation on stringent benchmark datasets and comparison to
existing techniques showed the effectiveness of INFUSE with an F-Score of 0.91,
Accuracy of 91.6%, and Recall of 0.94 on the Test+ dataset, and an F-Score of
0.91, Accuracy of 85.6%, and Recall of 0.87 on the stringent Test-21 dataset.
These promising results indicate the strong generalization capability and the
potential to detect network attacks.
Anabia Sohail, Bibi Ayisha, Irfan Hameed, Muhammad Mohsin Zafar, Hani Alquhayz, Asifullah Khan
# Deep Neural Networks based Meta-Learning for Network Intrusion Detection

###### Abstract

Designing an intrusion detection system is difficult as network traffic encompasses various attack types, including new and evolving ones with minor changes. The data used to construct a predictive model has a skewed class distribution and limited representation of attack types, which differ from real network traffic. These limitations result in dataset shift, negatively impacting the machine learning models' predictive abilities and reducing the detection rate against novel attacks. To address the challenge of dataset shift, we introduce the INformation FUsion and Stacking Ensemble (INFUSE) for network intrusion detection. This approach further improves its predictive power by employing a deep neural network-based Meta-Learner on top of INFUSE. First, a hybrid feature space is created by integrating decision and feature spaces. Five different classifiers are utilized to generate a pool of decision spaces. The feature space is then enriched through a deep sparse autoencoder that learns the semantic relationships between attacks. Finally, the deep Meta-Learner acts as an ensemble combiner to analyze the hybrid feature space and make a final decision. Our evaluation on stringent benchmark datasets and comparison to existing techniques showed the effectiveness of INFUSE with an F-Score of 0.91, Accuracy of 91.6%, and Recall of 0.94 on the Test\({}^{+}\) dataset, and an F-Score of 0.91, Accuracy of 85.6%, and Recall of 0.87 on the stringent Test-21 dataset. These promising results indicate the proposed technique has strong generalization capability and the potential to detect network attacks.

 Intrusion Detection, Deep Neural Networks, Autoencoder, Deep Stacking Ensemble, Information Fusion, Deep Meta-Learner.
Introduction

The widespread use of various technologies such as 4G/5G networks, the Internet of Things, smart devices, and cloud-based services has significantly increased the size of global networks [1]. This expansion has made network attacks more sophisticated and increased the risk of network-based attacks due to prevailing vulnerabilities in the network [2, 3, 4]. The emergence of COVID-19 in late 2019 completely transformed the perception of internet-based activities among the general population, leading to a shift in trade and education to online mode [5, 6]. This transition has greatly expanded the number of network users, adding pace to the already-growing global network, with a projected growth of 62.5% of the world population in 2022, up from 56.7% [7, 8].

The large population relying on the global network makes network security a primary area of focus, as a considerable amount of privacy-sensitive data is being generated and distributed across multiple network nodes. Network traffic is targeted by a diverse range of attacks, including Probing, Denial of Service, User to Root, SQL injections, Cross-site Scripting, Web attacks, and many others [9]. In November 2021, a DDoS attack three times larger than previous records was initiated by approximately 15,000 bots, with a peak throughput of 3.45 Tbps. Recently, during the Russia-Ukraine war, the digital infrastructure of Ukraine [10] was crippled by cyber attacks [11].

The growing need for network security and the continuous evolution of intrusion methods demands an active research in the development of intrusion detection systems (IDS). Machine learning (ML) techniques are seen as a valuable tool for developing anomaly-based IDS because of their ability to model anomalous behaviour [12, 13]. However, the learning paradigm of ML techniques assumes that the probability distribution of the test set is the same as the training set [14]. In real-world applications, especially in network traffic, the data distribution often shifts between training and testing data. The continuous emergence of new and variant attack types poses a challenge of dataset shift in intrusion detection systems [15]. This raises concerns about poor performance of ML models on the test set due to underlying inductive bias of the model.

Dataset shift has received relatively little attention in the context of IDS, despite the fact that new attacks are emerging every day. Therefore, there is a pressing need to develop a technique that can effectively address the non-stationary nature of attacks. It is challenging to choose a model with the right bias and optimal hypothesis space that can accurately generalize the deviation in
attack distribution. Thus, it is necessary to develop a technique that can model the deviation pattern of attacks, rather than focusing solely on specific attack types.

This work proposes a novel Information Fusion and Stacking Heterogeneous Ensemble-based Model for Network Intrusion Detection (INFUSE) by performing both decision- and feature-level information fusion to address the dataset shift problem. The performance of the proposed INFUSE is assessed on the benchmarked network traffic dataset. The distinct contributions of this work are:

* Diverse feature spaces were explored to model the data distribution, which can address unseen emerging attack variations. The strong representational learning ability of weight regularized deep sparse autoencoders is utilized to encode the semantic relevance of normal and abrupt traffic.
* A strong decision space was produced by utilizing multiple base-classifiers with varying hypothesis spaces to reduce the inductive bias of each hypothesis space.
* A stacking-based deep heterogeneous ensemble was developed that uses a deep neural network as a meta-learner to systematically evaluate a hybrid space of different decision spaces along with multiple feature spaces.
* The proposed INFUSE has been analysed by evaluating several existing techniques and different ensembles, and the performance comparison shows a significant increase in detection rate and accuracy.

The paper is structured as follows: Section 2 analyses the relevant literature on intrusion detection. The dataset is described in Section 3, and methodology of the proposed ensemble is presented in Section 4. Section 5 discusses the results in detail and finally, concludes the study in Section 6.

## 2 Background Literature

In the past, several techniques have been developed for the analysis of network traffic, which involve either binary classification of intrusions in network traffic or multiclass segregation of attacks into specific categories. To achieve this, different approaches have been explored, including classical ML, deep learning, and ensemble learning techniques, utilizing publicly available datasets [16][17, 18, 19]. A brief overview of the existing techniques is provided below.
Initially, classical ML techniques were used to detect attacked samples in network traffic by focusing on the extraction or selection of informative representations from the data. Aslahi-Shahri et al. [20] developed a hybrid model that used genetic algorithms to select an optimal feature subset from the NSL-KDD dataset, followed by SVM classification. The technique achieved a 0.97 F-score by selecting 10 features out of 45. In another study, Liu et al. [21] used both unsupervised and supervised learning to address network intrusion. They first identified similar groups within the dataset using k-means clustering, and then used a Random Forest (RF) algorithm to classify these clusters as normal or attacked traffic. They followed this up with a multi-class classification of attacked samples using deep CNN and LSTM. This approach achieved accuracies of 85% and 99% for NSL-KDD and CIS-IDS2017 datasets, respectively. Another study [22] used the Naive Bayes algorithm to extract new feature embeddings from the NSL-KDD dataset and assigned the transformed feature space to SVM for maximum separability. Finally, a study by [23] proposed an evolutionary neural network for intrusion detection, using a multiverse optimizer to select optimal parameters during training. One major limitation of classical ML techniques is their reliance on different feature extraction techniques to deal with dataset variance effectively. A highly efficient intrusion detection system requires immediate detection of any attack.

Many researchers have utilized deep neural networks for network traffic analysis due to their strong representation mapping capacity. Neural network-based intrusion detection systems typically consist of two stages: in the first stage, deep neural networks are used for feature space encoding, and in the second stage, classification of attacked samples is performed [24]. Qureshi et al. [25] utilized the idea of self-taught learning and autoencoders to classify records as normal or malicious. In this study, a deep sparse autoencoder was used for feature extraction and classification phases. Firstly, a pre-trained autoencoder for the regression problem was used to derive a new feature set which was then concatenated with the original feature set. After this, an autoencoder without a decoder was trained end-to-end on the augmented feature space for classification. Al-Qataf et al. [26] proposed a hybrid learning scheme that utilized the idea of self-taught learning for pretraining of an autoencoder. They extracted a new low-dimensional feature space from the NSL-KDD dataset and assigned it to an SVM for classification.

In another study, the performance was improved by combining four different CNN architectures [27]. The feature space of the NSL-KDD dataset was transformed into an image to take
advantage of the learning capacity of CNN. Four different CNN architectures were merged before the fully connected layer and trained with a single loss function. The final decision was based on a 256-dimensional feature space, but the use of multiple CNNs increased the complexity. Naseer et al. [28] investigated various techniques, including deep CNN, LSTM, and AE, to differentiate between attacked samples and normal data flow. However, due to the significant difference between the attack samples in the training and testing sets, these techniques exhibit a low detection rate for attacked samples.

Ensemble learning has shown promise in improving individual classifiers for handling noise sensitivity, domain shift, scalability, and inability to detect diverse attacks. Gao et al. developed an adaptive learning-based ensemble to overcome the complexity of intrusion datasets [29]. Five different classifiers, including decision tree, RF, kNN, and DNN, were used as base learners in the ensemble. Majority voting was used to make a decision by assigning a weightage to each classifier's decision. This ensemble technique was used for intrusion detection in NSL-KDD Test+. Similarly, another study suggested an ensemble of \(n\) modified Adaboost algorithms. Cost-sensitive base-classifiers were developed by optimizing the Adaboost algorithm using the area under the curve to address the class imbalance problem. Salo et al. selected significant features by applying PCA and Information Gain [30]. They enhanced the learning capacity by developing an ensemble of SVM, instance-based learning algorithms, and multilayer perceptron.

Zhang et al. [31] proposed using multiple feature fusion and homogenous stacking ensemble mechanisms to detect irregularities in network traffic. A diverse set of features was generated to train \(n\) number of homogenous base classifiers. The predictions of the base classifiers were combined using RF as a meta-classifier to draw a final decision. The idea of heterogeneous ensembles has been proposed to enhance the learning capacity by addressing the shortcomings of homogenous ensembles. Zhou et al. developed a voting-based heterogeneous ensemble. Initially, they used a hierarchical feature extraction algorithm CFS-BA to boost the feature representation at the preprocessing step. The proposed approach exploited shallow algorithms such as Forest Penalizing Attributes, C4.5, and RF on the extracted representation, and an average voting strategy was used to combine the base classifiers' probabilities [32].

The main issue with the existing approach is that it ignores the non-stationary nature of attacks and uses accuracy as the performance metric, which can underestimate the detection rate of the
minority class. In real network traffic, a wide range of attacks exist, and the proportion of attacks is often imbalanced with respect to normal traffic. Therefore, it is essential to take into account the issue of dataset shift and prioritize the detection ability of the ML model for highly imbalanced malicious attack data.

## 3 Dataset Details

NSL-KDD dataset was utilized in this study for the analysis of network traffic intrusion. It is considered a benchmark dataset for the development of Network Intrusion Detection Systems [33]. The dataset's high level of difficulty, class imbalance, and presence of unique attacks in the test set make it suitable for analyzing the effectiveness of intrusion detection systems against a dataset shift problem. The presence of unique attacks in the test set motivates the development of a model that can handle emerging attacks and the challenge of distribution shift. The dataset's records are classified as normal or attacked, with the attacked samples falling into five main categories commonly found in network traffic. Table 1 provides a description of the dataset's characteristics, while Table 2 lists the various types of attacks reported in the dataset. The training set includes normal data and 22 attack types, while the test dataset contains seven new attack types in addition to those present in the training data.

The distinct features of NSL-KDD dataset are as follows:

* There is no repetition of records in the train and test set contrary to other datasets. Sample recurrence in the test set makes the performance evaluation criterion unrealistic. NSL-KDD dataset makes an equal contribution to each sample to remove bias.
* The difficulty level was assigned to both the train and test set. Test set was made stringent by adding samples from 17 different unique attacks in the Test\({}^{+}\) and Test-21 other than attacks reported in Train\({}^{+}\). The additional stringent dataset Test\({}^{-21}\) was generated by the test examples that were misclassified by the 21 classifiers.
* The train and test sets are highly imbalanced that increases the difficulty level. The frequency of normal and attacked traffic in the Train, Test\({}^{+}\) and Test-21 are represented in Figure 1.
### Statistical Evaluation of Dataset Shift

In this work, the Kolmogorov-Smirnov test is applied to analyse a dataset shift between train and test set. It is a non-parametric test and compares how likely that two datasets are drawn with the same probability distribution [34]. Hypothesis test for comparison of test distribution with train is formulated as:

\begin{table}
\begin{tabular}{|c|c|c|} \hline
**Category** & \multicolumn{2}{|c|}{**Description**} & **Data type** \\ \hline Basic Features & Features representing TCP/ICP connection & Symbolic \& \\  & without considering the payload & Continuous \\ \hline Content Features & Features required to accessed the payload & \multirow{2}{*}{\begin{tabular}{} \end{tabular} } \\ Content Features & of TCP packet and suspicious behaviour & \\  & within payload & \\ \hline Time based Traffic & Network features were analysed with 2s & \multirow{2}{*}{\begin{tabular}{} \end{tabular} } \\ Features & temporal window and provide a statistical & \\  & information & \\ \hline Host based Traffic & Feature used to analyse the attack within & \multirow{2}{*}{
\begin{tabular}{} \end{tabular} } \\ features & interval longer than 2s & \\ \hline \end{tabular}
\end{table}
Table 1: Feature representation of NSL KDD dataset.

\begin{table}
\begin{tabular}{|c|c|} \hline
**Category** & \multicolumn{2}{|c|}{**Attacks**} \\ \hline
**Probing** & \multicolumn{2}{|c|}{IP-sweep, Port-sweep, NMAP, Satan} \\ \hline
**Root to Local** & FTP-Write, Guess-password, IMAP, Multi-HOP, Phf, SPY, Warezclient, Warezmaster \\ \hline
**User to Root** & \multicolumn{2}{|c|}{Buffer Overflow, Load-module, Perl, Root-kit} \\ \hline
**Denial of Service** & Back, Land, Neptune, Ping-of-Death, Smurf, Teardrop \\ \hline
**Unique attacks in Test+** & \multicolumn{2}{|c|}{\begin{tabular}{} \end{tabular} } \\ and Test-21 & \multicolumn{2}{|c|}{
\begin{tabular}{} \end{tabular} } \\ \hline \end{tabular}
\end{table}
Table 2: Types of attacks in the Train, Test\(+\) and Test-21.

Figure 1: The proportion of attacked samples in the train and test sets (Test\(+\) and Test-21).
H\({}_{0}\): train and test sets are sampled from same data distributions

H\({}_{\Lambda}\): train and test sets have different data distributions

The Kolmogorov-Smirnov test computes the test statistics D by regressing the empirical distribution function of the test set on cumulative distribution function of reference dataset (train data) using the following formula:

\[D_{s,m}=\max\mid F(x)-E(x)\mid \tag{1}\]

In Eq. (1), \(n\) is the sample size of cumulative distribution function ( \(F(x)\)) of the train set, whereas \(m\) is the sample size of empirical distribution function ( \(E(x)\)) of the test set.

The test statistics D = 0.0128 show that there is a significant difference between two distributions with significant p-value (p-value= 6.82e-280 \(<\) 0.05).

### Feature space based Evaluation of Dataset Shift

Feature space of the train \(p_{train}(X_{tr})\) and test set \(p_{test}(X_{tr})\)is visualized using t-SNE (Figure 2) to qualitatively analyse the shift in distribution. Figure 2 shows that the distribution of the test set is not only different from the train set \(p_{train}(x_{tr}\mid y_{tr})\neq p_{test}(x_{tr}\mid y_{tr})\) but also that the test set has some unique attacks \(p_{train}(y_{tr}\mid x_{tr})\neq p_{test}(y_{tr}\mid x_{tr})\) that are not part of the training set.

### Dataset Representation and Preprocessing

NSL-KDD dataset comprises 41 network features that are measured on continuous and ordinal scales. The features are listed in Table 2, and symbolic features are represented by indicator variables in this study, which increases the feature dimension from 41 to 121.

The NSL-KDD dataset has a highly skewed feature distribution, leading to biased variable contributions and reduced classifier performance. To address this issue, we applied data

Figure 2: Visualization of the first two components of the feature space projected via t-SNE.
normalization to scale the Train, Test\({}^{+}\), and Test-21 sets while preserving their original distributions.

### Dataset Division

In this study, we used a stratified hold-out strategy to randomly split the training dataset into a 60:40% for training of base-classifiers and the Meta-Learner, respectively. We select an optimal set of hyperparameters for base-classifiers by performing 5-fold cross-validation on the 60% reserved training data. The Meta-Learner is trained on 80% of the 40% split, while 20% is utilized for validation and hyperparameter selection. Finally, we assess the generalization performance of the proposed ensemble on separately provided test sets (test\({}^{+}\) and test-21) by NSL KDD.

The proposed INformation Fusion and Stacking Heterogeneous Ensemble for Network Intrusion Detection (INFUSE)

The network data is heterogeneous in nature and vulnerable to various types of attacks, posing a challenge to developing a robust NIDS that can handle dataset shift problems. To address this challenge, we propose an INformation FUSion and Stacking Ensemble called "INFUSE". The proposed ensemble works sequentially in two phases: first, developing an information-rich

Figure 3: A detailed layout of the proposed INFUSE.
feature space, and second, developing a meta-learner to draw a final decision from the hybrid feature space. Figure 3 provides a detailed workflow of the proposed INFUSE ensemble.

### Motivation

The concept of information fusion is utilized at both the feature and decision level to tackle the dataset shift problem. Additionally, an idea of stacking ensemble is used to enhance the generalization of the NIDS. A machine learning model has an inherent bias towards a particular data distribution, which can affect its performance when dealing with dataset shift. Ensemble-based classifiers address this limitation by introducing diversity in sample space, feature space, or decision space, which can overcome the bias and variance associated with individual learners [35]. Therefore, the use of ensembles in this study is expected to improve accuracy and robustness against distribution shift in the test set compared to a single learner.

The learning potential of an ensemble is primarily defined by diversity and the combination rule [36]. In this study, the proposed ensemble "INFUSE" induces diversity by combining the predictive power of multiple classifiers and incorporating diversity in feature representation to handle diverse attacks. Meanwhile, an effective combination rule is generated via a deep neural network. The proposed technique works based on the following ideas:

* The idea of information fusion is exploited to improve the diversity and develop an information-rich feature space that can address the challenge of distribution shift.
* A set of supervised base-classifiers varying in their hypothesis spaces are used. Thus, the instances that cannot be tackled via single learner can be corrected by other classifiers.
* The representational learning capacity of artificial NN is exploited using unsupervised weight regularized deep sparse autoencoders to generate effective feature representations. In this regard, a new attack instance that lies outside the distribution can be modeled based on semantic relevance.
* A stacking based heterogeneous ensemble is developed to integrate multiple information spaces. A deep artificial NN based meta-learner is used that makes a decision by analysing the information-rich feature-space collectively in an intelligent way.
### Diversity Improvement using Decision Spaces

In this work, we have used five heterogeneous base classifiers \(H=\{H_{1},H_{2},H_{3},H_{4},H_{5}\}\) and combined their decision Score \(Z=\{Z_{1},Z_{2},Z_{3},Z_{4},Z_{5}\}\) to generate a pool of diverse hypothesis varying in their learning biases. Diversity in the hypothesis space of base classifiers is discussed below. The parameters of the base classifiers are mentioned in Table 3.

#### 4.2.1 Base Classifier 1

The first base classifier was selected based on the property of structural risk minimization. In this regard, SVM is used to define an optimal decision boundary [37] by incorporating a geometric interpretation of the hyper-plane during training. The mathematical representation of SVM is shown in Eqs. (2 & 3).

\[\mathbf{w}^{T}x+b=0 \tag{2}\]

\[\min_{w,\zeta_{i}}C\sum_{i}^{N}\zeta_{i}+\frac{1}{2}\left\|\,\mathbf{w}\, \right\|^{2} \tag{3}\]

In Eq. (2), an input instance is represented by \(x\), whereas \(\,\mathbf{w}^{T}\,\) is a weight vector and \(b\) is a bias. Misclassifications made by a learner are represented by \(\,\zeta_{i}\,\), and \(C\) is a hyper-parameter that establishes tradeoff between generalization and empirical error on a training set.

#### 4.2.2 Base Classifier 2

The second base classifier kNN was nonparametric in nature and makes an instance-wise classification. It locally approximates the target function by simply storing the training instances instead of learning the explicit definition of the target function. This gives the advantage for classifying new attacks as it assigns the class to new instance based on smallest distance with its \(k\) nearest neighbours as shown in Eq. (4) [38]. Eq. (4) shows that Euclidian distance is computed between train instance \(X\,\) and test instance \(j\,\).

\[D(X,j)=(\sum_{i=1}^{n}\left|x_{i}-j\right|^{p})^{1/p} \tag{4}\]

#### 4.2.3 Base Classifier 3

Rule-based approach is used in the third base classifier [29] to appropriately deal with categorical and nominal datasets. In intrusion detection, most features are binary in nature and can be well classified using a rule-based approach. For this, decision tree is implemented using entropy and it is mathematically represented in Eq. (5).
\[H(L)=-\sum_{i=1}^{k}p(L=l_{i})^{p}\log_{2}p(L=l_{i}) \tag{5}\]

#### 4.2.4 Base Classifier 4

Furthermore, idea of ensemble learning is employed in base classifier-4 using Random Forest (RF). RF improves robustness towards attacks by manipulating both data distribution and features during training. RF draws the final decision by taking an average of all the predictions made by the forest of trees. It is mathematically expressed in Eq. (6). RF reduces the variance and sensitivity associated with individual trees [39].

\[\hat{R}=\frac{1}{B}\sum_{b=1}^{B}\hat{f_{b}}(x^{\cdot}) \tag{6}\]

In Eq. (6), \(B\) is the number of decision trees that are considered during prediction, and \(b\) is the current decision tree. Data sub-set is defined by \(x^{\cdot}\), \(\hat{f_{b}}(.)\) is the function that fits the decision tree on feature subset, and final output is denoted by \(\hat{R}\).

#### 4.2.5 Base Classifier 5

AdaBoost reduces the bias associated with each classifier using boosting strategy [40]. It performs the serial training of a set of weak base-learners in \(n\) iterations, and weighted sampling is performed during training. Thus, each classifier focuses on correctly classifying the samples misclassified in the previous iteration. The final decision is drawn by taking a decision from all the learners and combining them through a majority vote rule.

### Diversity Improvement using Feature Space

In this work, we improved the feature space by training two AEs in an unsupervised manner to learn semantics of the data and generated a new representation reflecting the underlying distribution of network traffic. The newly generated laten representation \(L=\{L_{1},L_{2}\}\) is combined with the original feature space \(X\) to provide more discriminative features \(F=f_{c}(L_{1}\|\,L_{2}\,\|\,X)\).

#### 4.3.1 Weight Regularized Deep Sparse AE

We exploited two weight regularized deep sparse AEs with depth of 8- and 10- layers to capture the latent representation of attacks. The description of the proposed AEs is mentioned in Table 4
and Figure 4. AE is an effective representational learning algorithm based on unsupervised NNs. The learning principle of AE is based on identity function and follows the encoder-decoder paradigm [41]. The mathematical representation of AE is shown in Eqs. 7 & 8.

\[\overset{\wedge}{L_{1}}=f_{enc}(X)=\partial(\textbf{w}_{n}(\textbf{ w}_{n-1}..(\textbf{w}_{1}X+\textbf{b}_{1})+\textbf{b}_{n-1})+\textbf{b}_{n}) \tag{7}\] \[X^{{}^{\prime}}=f_{dec}(\overset{\wedge}{L})=\partial(\textbf{ w}_{n}^{{}^{\prime}}(\textbf{w}_{n-1}..(\textbf{w}_{1}^{{}^{\prime}} \overset{\wedge}{L+\textbf{b}_{1}^{{}^{\prime}}})+\textbf{b}_{n-1}^{{}^{\prime} })+\textbf{b}_{n}^{{}^{\prime}}) \tag{8}\]

In Eq. (7) \(X\) is the original input, **W**\(=\{\)**w\({}_{1}\),...,**w\({}_{n}\}\)** represents weights, **B**\(=\{\)**b\({}_{1}\),..., **b\({}_{n}\}\)** is the added biases. AE maps \(X\) to a latent representation \(\overset{\wedge}{L}\) using an encoding function \(f_{enc}(.)\). In Eq. (8), the reconstructed input is represented by \(X^{{}^{\prime}}\) that is generated from the latent representation ( \(L\) ) using the decoding function \(f_{dec}(.)\).

In this work, we enhance the representation learning ability of the AE by making it deep and adding the sparsity penalty. Deep AE maps the original input to useful representation by hierarchically learning multi-level feature representations while each hierarchy level corresponds to a different level of abstraction. The L2 weight regularization was applied as a sparsity penalty in the mean square error loss function that enforces the AE to learn the effective low-dimensional feature representation from the NSL-KDD dataset.

\[MSE=f_{error}(\frac{1}{n}\sum_{i=1}^{n}\left\|X_{i}-X^{{}^{\prime}}_{i}\right\| ^{2}+\lambda\left\|\textbf{w}\right\|^{2}) \tag{9}\]

The sparsity penalty for AE is shown in Eq. (9). L2 weight regularization penalizes the model in proportion to the magnitude of the weights. Thus, it enforces weights near zero and makes AE to learn a reduced feature space by discovering the structure hidden in the high dimensional feature space. A latent representation is generated by learning a semantic relevance between features corresponding to the same group.
### The proposed Meta-Learner for Information Fusion Analysis

A hybrid feature space is developed by fusing multiple feature representations with decision spaces to generate a strong representative context. A 6-layers deep, fully connected NN is used as a Meta-Learner (Figure 5) to approximate the target function from a hybrid information-rich feature-space intelligently. Multiple levels of nonlinearities make deep NNs efficient in learning a complex non-linear representation.

Figure 4: Architectural overview of the proposed weight regularized AE.
### Parameter Setting

The parameter setting of the Base and Meta-Learner is performed on the validation dataset that is separated from the training set. Tables- 3 and -4 show the parameter on which proposed models give the optimal performance.

\begin{table}
\begin{tabular}{|l|c|c|c|c|c|} \hline
**Model** & **Layers** & **Learning rate** & **Optimizer** & **Loss** & **Weight decay** \\ \hline INFUSE & 6 Layers & 0.00008 & SGD & BCELoss & - \\ \hline Autoencoder 1 & 10 Layers & 0.00008 & Adam & MSELoss & 1e-5 \\ \hline Autoencoder 2 & 8 Layers & 0.0001 & Adam & MSELoss & 1e-5 \\ \hline \end{tabular}
\end{table}
Table 4: Parameter setting of the proposed weight regularized autoencoder and Meta-Leraner.

Figure 5: Architectural overview of the proposed deep NN based Meta-Learner.
Results

The significance of the proposed ensemble INFUSE against dataset shift has been analyzed on the Test\({}^{+}\) and Test-21 of the NSL-KDD dataset. The performance has been evaluated using several performance measures and compared with other techniques.

### Evaluation Measures

Multiple evaluation metrics, including accuracy (Acc.), F-score, detection rate (Recall), false negative rate (FNR), ROC, and PR curves, are considered to estimate the performance of the proposed ensemble INFUSE against dataset shift. Detection rate is used to evaluate the model's capacity to recognize attacks, and Accuracy and F-score are also reported. The statistical significance of the proposed ensemble is assessed via the McNemar test, and the standard error is computed at a 95% confidence interval using z-statistics. True Positives (TPs) are the samples correctly recognized as attacks, and True Negatives (TNs) are the number of normal instances correctly classified. Accuracy defines the percentage of correct predictions regardless of the normal and attacked class. The performance measures are defined as follows.

\[\textit{Detection Rate}\left(\text{Re}\,\textit{call}\right)=\textit{TP} \textit{/}\textit{TP}+\textit{FN} \tag{10}\]

\[\textit{Specificity}=\textit{TN}\textit{/}\textit{TN}+\textit{FP} \tag{11}\]

\[\textit{Accuracy}=\textit{TP}+\textit{TN}\textit{/}\textit{TP}+\textit{TN}+ \textit{FP}+\textit{FN} \tag{12}\]

\[\textit{F-Score}=2.\begin{array}{c}\text{Pr}\,\textit{ecision}\times\, \text{Re}\,\textit{call}\end{array} \tag{13}\]

\[\textit{FNR}=\textit{FN}\textit{/}\textit{/}\textit{TP}+\textit{FN} \tag{14}\]

### Performance Estimation of the proposed Ensemble (INFUSE)

Initially, the performance of the proposed INFUSE was analyzed by comparing it to standard baseline models, namely MLP (2-layers deep) and a deep fully connected neural network (6-layers deep). Table 5 shows the F-score, accuracy, and recall, indicating that the proposed ensemble INFUSE outperforms the baseline models on both datasets. The remarkable performance of INFUSE on the Test\({}^{21}\) dataset, which has a high distribution shift, suggests that the proposed technique is effective in addressing the dataset shift problem.
### Performance comparison of INFUSE with Base-Classifier

The performance of the base-classifiers is presented in Table 6, where SVM has the highest accuracy of 82.80% and 67% on Test\({}^{+}\) and Test-21, respectively. The F-scores of base-classifiers range from 0.69 to 0.83 on Test\({}^{+}\) and 0.55 to 0.76 on Test-21. While the base-classifiers show reasonable performance on KDD Test\({}^{+}\), their performance deteriorates on KDD Test-21, a stringent dataset with a high class imbalance of unseen attacks that are difficult to classify. The empirical analysis in Table 6 suggests that the proposed ensemble INFUSE outperforms the base-classifiers and significantly addresses the problem of dataset shift.

### Statistical Analysis

McNemar's test is applied to statistically evaluate the significance of the proposed ensemble INFUSE [42]. The hypothesis test is formulated as:

H\({}_{0}\): the proposed ensemble is equivalent to base-classifier in performance

H\({}_{1}\): the proposed ensemble performs better than base-classifier

The performance is accessed by comparing performance with best performing base-classifier using Eq. (15).

\begin{table}
\begin{tabular}{|c|c|c|c|c|c|c|} \hline  & \multicolumn{3}{c|}{**NSL-KDD Test+**} & \multicolumn{3}{c|}{**NSL-KDD Test-21**} \\ \cline{2-7}  & **F-Score\(\pm\)S.E** & **Acc. (\%)** & **Recall** & **F-Score\(\pm\)S.E** & **Acc. (\%)** & **Recall** \\ \hline
**Proposed INFUSE** & **0.91\(\pm\)0.003** & **91.64** & **0.94** & **0.91\(\pm\)0.005** & **85.6** & **0.87** \\ \hline Base classifier-1 (SVM) & 0.83\(\pm\)0.005 & 82.80 & 0.73 & 0.76\(\pm\)0.0076 & 67.41 & 0.64 \\ \hline Base classifier-2 (KNN) & 0.76\(\pm\)0.0055 & 76.76 & 0.64 & 0.66\(\pm\)0.0085 & 55.83 & 0.53 \\ \hline Base classifier-3 (Decision Tree) & 0.77\(\pm\)0.0055 & 78.48 & 0.64 & 0.68\(\pm\)0.0083 & 59.16 & 0.53 \\ \hline Base classifier-4 (Random Forest) & 0.69\(\pm\)0.006 & 72.65 & 0.53 & 0.55\(\pm\)0.009 & 48.30 & 0.39 \\ \hline Base classifier-5 (AdaBoost) & 0.76\(\pm\)0.0055 & 76.56 & 0.64 & 0.66\(\pm\)0.0085 & 55.49 & 0.53 \\ \hline \end{tabular}
\end{table}
Table 6: Performance comparison of the proposed INFUSE with base classifiers.

\begin{table}
\begin{tabular}{|c|c|c|c|c|c|c|} \hline \multirow{2}{*}{**Technique**} & \multicolumn{3}{c|}{**NSL-KDD Test+**} & \multicolumn{3}{c|}{**NSL-KDD Test-21**} \\ \cline{2-7}  & **F-Score\(\pm\)S.E** & **Acc. (\%)** & **Recall** & **F-Score\(\pm\)S.E** & **Acc. (\%)** & **Recall** \\ \hline
**Proposed INFUSE** & **0.91\(\pm\)0.003** & **91.64** & **0.94** & **0.91\(\pm\)0.005** & **85.6** & **0.87** \\ \hline MLP & 0.84\(\pm\)0.005 & 83.05 & 0.77 & 0.79\(\pm\)0.007 & 69.18 & 0.69 \\ \hline Deep NN & 0.87\(\pm\)0.004 & 84.70 & 0.89 & 0.86\(\pm\)0.006 & 76.90 & 0.84 \\ \hline \end{tabular}
\end{table}
Table 5: Performance evaluation of the proposed ensemble INFUSE and baseline classifiers.
\[\chi^{2}=\frac{\left(b-c\right)^{2}}{b+c} \tag{15}\]

In the above equation, \(b\) is the number of misclassifications made by the base-classifier whereas \(c\) is the total misclassification by ensemble model.

The test statistics suggest a statistically significant difference in performance of the proposed ensemble compared to best performing base-classifer with statistics \(=566.7\), p-value \(=2.88\)e-125 \(<0.005\) for test\(+\) and statistics \(=753.5\), p-value \(=6.803\)e-166 \(<0.005\) for test-21.

### ROC and PR Curve based Analysis

The ROC curve graphically illustrates the ratio of the true positive rate to the false-positive rate at multiple confidence intervals, while the PR curve is for the imbalanced dataset that shows the

Figure 6: PR and ROC curves based analysis of the proposed INFUSE and base classifiers.
ratio of correctly classified examples among the positively predicted samples at several confidence intervals. As shown in Figure 6, the performance of the proposed ensemble improves on unseen data. Figure 6 also provides a graphical illustration of AUC-ROC and AUC-PR on Test\({}^{+}\) and Test-21.

### Ablation Study

We also conducted a thorough analysis to assess the importance of information fusion in the proposed ensemble by excluding each of the feature spaces one by one. The results are presented in Tables 7 and 8, which indicate that the proposed INFUSE shows better performance on both the datasets than that of the individual feature spaces. The results show that the proposed technique considerably gains an increase in F-score and accuracy on Test-21 and achieves a good trade-off between recall and specificity.
### 5.7 Significance Analysis of the Proposed Ensemble INFUSE

The significance of the proposed ensemble against dataset shift was evaluated by assigning the information-fused hybrid feature space to SVM and XGBoost as Meta-Learners. Additionally, a performance comparison was made with different classical ensemble techniques, and the results are shown in Table 9. The empirical evaluation demonstrated that the proposed deep learning-based Meta-Learner significantly outperformed other Meta-Learners as well as max-weighted, average, and majority voting-based ensemble techniques.

### Detection Rate Analysis and Feature space Visualization

In IDS, detecting each type of attack is crucial for system security. Figure 7 shows the detection rate and FNR for the proposed ensemble and other techniques. As the type and nature of attacks change every day, IDS should be robust against new attack types. Therefore, the detection rate of the proposed ensemble INFUSE is analyzed for all the unique attack profiles included in the test sets. In the test set, there are 7 attack profiles - mscan, processtable, snmpguess, sainl, apache, httptunnel, and mailbomb - that are unique attack variants and were not seen by the classifier during training. The results, shown in Figure 8, propose that the proposed ensemble INFUSE achieve a considerable detection rate for new attacks that were not seen by the classifier during training.

\begin{table}
\begin{tabular}{|c|c|c|c|c|c|} \hline \multirow{2}{*}{**Technique**} & \multirow{2}{*}{**Type**} & \multicolumn{2}{c|}{**NSL-KDD Test+**} & \multicolumn{2}{c|}{**NSL-KDD Test-21**} \\ \cline{3-6}  & & **F-Score+S.E** & **Acc. (\%)** & **F-Score+S.E** & **Acc. (\%)** \\ \hline
**Proposed INFUSE** & & **0.91\(\pm\)0.003** & **91.64** & **0.91\(\pm\)0.005** & **85.6** \\ \hline Proposed Hybrid space with SVM Meta-Learner & Stacking Ensemble & 0.78\(\pm\)0.0055 & 78.80 & 0.69\(\pm\)0.0083 & 59.76 \\ \hline Proposed Hybrid space with XGBoost Meta-Learner & & 0.76\(\pm\)0.0055 & 77.69 & 0.66\(\pm\)0.0085 & 57.59 \\ \hline Max-weighted voting & & 0.79\(\pm\)0.0053 & 80.00 & 0.72\(\pm\)0.0080 & 62.05 \\ \hline Average voting & Classical Ensemble & 0.768\(\pm\)0.0055 & 78.00 & 0.67\(\pm\)0.0084 & 58.40 \\ \hline Majority voting & & 0.77\(\pm\)0.0055 & 78.38 & 0.67\(\pm\)0.0084 & 58.89 \\ \hline \end{tabular}
\end{table}
Table 9: Performance comparison of the proposed ensemble with different ensemble learning strategies on Test\({}^{+}\).
Feature space visualization using t-SNE shows that the proposed ensemble INFUSE significantly separates the attacked samples from normal traffic samples (Figure 9).
### Performance Comparison with Existing Techniques

The proposed ensemble INFUSE has been evaluated by making performance comparison with state-of-the-art studies that employed ML, deep learning, and ensemble learning techniques. The results, shown in Table 10, indicate that the proposed ensemble INFUSE significantly outperforms other classifiers both on Test\({}^{+}\) and Test-21.

Figure 9: Feature space is visualized via t-SNE. Panels (a) and (b) show original and feature space of the last layer of the proposed INFUSE, respectively.
## 6 Conclusion

Efficient detection against various network traffic attacks is crucial for ensuring smooth flow of network traffic without interruption. In this study, we developed a robust stacking heterogeneous ensemble using the idea of information fusion to address the problem of dataset shift. Our proposed approach, INFUSE, improves the detection rate by utilizing weight regularized deep sparse autoencoders, while achieving specificity towards attacks through exploitation of multiple decision spaces. In the decision stage, a deep neural network-based Meta-Learner intelligently draws a final decision from the hybrid feature space, establishing a good trade-off between specificity and recall. Our performance analysis of the proposed technique on the NSL-KDD dataset in terms of accuracy (Test\(+\): 91.6%, Test-21: 85.6%) and detection rate (Test\(+\): 0.94, Test-21: 0.87) demonstrates its effectiveness. Comparison with other ensemble learning techniques and existing techniques highlights the strong detection ability of INFUSE towards

\begin{table}
\begin{tabular}{|c|c|c|c|} \hline \multirow{2}{*}{**Reference**} & \multirow{2}{*}{**Technique**} & **NSL-KDD Test\(+\)** & **NSL-KDD Test-21** \\ \cline{3-4}  & & **Acc. (\%)** & **Acc. (\%)** \\ \hline
**Proposed INFUSE** & **Information fusion and deep Meta learner** & **91.6** & **85.6\%** \\ \hline Mushtaq et al. [43] & LSTM\(+\)AE & 89.84 & - \\ \hline Vinet et al. [44] & Clustering \(+\) SVM & 91.3 & - \\ \hline Zhang et al. [31] & Stacking Ensemble & 84 & - \\ \hline Sham et al. [45] & Deep CNN & 81.00 & - \\ \hline Zhou et al. [32] & Voting based Ensemble & 87.37 & 73.57 \\ \hline Tama et al. [46] & Two-Stage Ensemble & 85.8 & 72.52 \\ \hline Qureshi et al. [25] & Autoencoder & 84.60 & 79.90 \\ \hline Chohan et al. [47] & Deep CNN & 89.41 & 80.36 \\ \hline Singh et al. [48] & RNN & 84.03 & 69.75 \\ \hline Li et al. [27] & Multi-CNN fusion & 86.95 & 76.67 \\ \hline Gao et al. [49] & Ensemble & 84.54 & 71.29 \\ \hline Naseer et al. [28] & Deep CNN & 85.00 & 70.00 \\ \hline Qatf et al. [26] & Sparse Autoencoder \& SVM & 84.96 & 79.42 \\ \hline Ashfaq et al. [50] & Fuzzy based semi supervised Learning & 84.12 & 68.82 \\ \hline Yin et al. [51] & RNN & 81.29 & 64.67 \\ \hline \end{tabular}
\end{table}
Table 10: Performance comparison with existing state-of-the-art techniques.
unseen attacks. In the future, this approach may prove useful for Zero-day attack detection and other malware analysis problems, enabling improved generalization.

**CRediT authorship contribution statement**

**Anabia Sohail:** Methodology, Software, Supervision, Writing - original draft, Writing - review & editing, **Ayisha Abdullah:** Methodology, Software, Visualization, **Irfan Hameed:** Validation, **Muhammad Mohsin Zafar**: Writing - original draft & review, **Asifullah Khan:** Supervision, Methodology, Writing - review & editing, Project administration, Resources.

**Declaration of Competing Interest**

The authors declare no competing interest.

**Acknowledgments**

The authors thank Pattern Recognition lab at DCIS, PIEAS, for providing computational facilities.
