Graph Neural Networks based Log Anomaly Detection and Explanation
Event logs are widely used to record the status of high-tech systems, making
log anomaly detection important for monitoring those systems. Most existing log
anomaly detection methods take a log event count matrix or log event sequences
as input, exploiting quantitative and/or sequential relationships between log
events to detect anomalies. Unfortunately, only considering quantitative or
sequential relationships may result in low detection accuracy. To alleviate
this problem, we propose a graph-based method for unsupervised log anomaly
detection, dubbed Logs2Graphs, which first converts event logs into attributed,
directed, and weighted graphs, and then leverages graph neural networks to
perform graph-level anomaly detection. Specifically, we introduce One-Class
Digraph Inception Convolutional Networks, abbreviated as OCDiGCN, a novel graph
neural network model for detecting graph-level anomalies in a collection of
attributed, directed, and weighted graphs. By coupling the graph representation
and anomaly detection steps, OCDiGCN can learn a representation that is
especially suited for anomaly detection, resulting in a high detection
accuracy. Importantly, for each identified anomaly, we additionally provide a
small subset of nodes that play a crucial role in OCDiGCN's prediction as
explanations, which can offer valuable cues for subsequent root cause
diagnosis. Experiments on five benchmark datasets show that Logs2Graphs
performs at least on par with state-of-the-art log anomaly detection methods on
simple datasets while largely outperforming state-of-the-art log anomaly
detection methods on complicated datasets.
Zhong Li, Jiayang Shi, Matthijs van Leeuwen
# Graph Neural Network based Log Anomaly Detection and Explanation

###### Abstract.

Event logs are widely used to record the status of high-tech systems, making log anomaly detection important for monitoring those systems. Most existing log anomaly detection methods take a log event count matrix or log event sequences as input, exploiting quantitative and/or sequential relationships between log events to detect anomalies. Unfortunately, only considering quantitative or sequential relationships may result in many false positives and/or false negatives. To alleviate this problem, we propose a graph-based method for unsupervised log anomaly detection, dubbed _Log3Graphs_, which first converts event logs into attributed, directed, and weighted graphs, and then leverages graph neural networks to perform graph-level anomaly detection. Specifically, we introduce One-Class Digraph Inception Convolutional Networks, abbreviated as OCDiGCN, a novel graph neural network model for detecting graph-level anomalies in a collection of attributed, directed, and weighted graphs. By coupling the graph representation and anomaly detection steps, OCDiGCN can learn a representation that is especially suited for anomaly detection, resulting in a high detection accuracy. Importantly, for each identified anomaly, we additionally provide a small subset of nodes that play a crucial role in OCDiGCN's prediction as explanations, which can offer valuable cues for subsequent root cause diagnosis. Experiments on five benchmark datasets show that _Log3Graphs_ performs at least on par state-of-the-art log anomaly detection methods on simple datasets while largely outperforming state-of-the-art log anomaly detection methods on complicated datasets.

Log Analysis, Log Anomaly Detection, Graph Neural Networks +
Footnote †: ccs: 2024
most existing log anomaly detection methods focus exclusively on detection performance without giving any explanations.

To overcome these limitations, we propose _Logs2Graphs_, a graph-based unsupervised log anomaly detection approach by designing a novel one-class graph neural network. Specifically, _Logs2Graphs_ first utilises off-the-shelf methods to learn a semantic embedding for each log event, and then assigns log messages to different groups. Second, _Logs2Graphs_ converts each group of log messages into an attributed, directed, and weighted graph, with each node representing a log event, the node attributes containing its semantic embedding, a directed edge representing how an event is followed by another event, and the corresponding edge weight indicating the number of times the events follow each other. Third, by coupling the graph representation learning and anomaly detection objectives, we introduce One-Class Digraph Inception Convolutional Networks (OCDiGCN) as a novel method to detect anomalous graphs from a set of graphs. As a result, _Logs2Graphs_ leverages the rich and expressive power of attributed, directed and edge-weighted graphs to represent logs, followed by using graph neural networks to effectively detect graph-level anomalies, taking into account both semantic information of log events and structure information (including sequential information as a special case) among log events. Importantly, by decomposing the anomaly score of a graph into individual nodes and visualizing these nodes based on their contributions, we provide straightforward and understandable explanations for identified anomalies.

Overall, our contributions can be summarised as follows: (1) We introduce _Logs2Graphs_, which formalises log anomaly detection as a graph-level anomaly detection problem and represents log sequences as directed graphs to capture more structure information than previous approaches; (2) We introduce OCDiGCN, the first end-to-end unsupervised graph-level anomaly detection method for attributed, directed and edge-weighted graphs. By coupling the graph representation and anomaly detection objectives, we improve the potential for accurate anomaly detection over existing approaches; (3) For each detected anomaly, we identify important nodes as explanations, offering valuable cues for subsequent root cause diagnosis; (4) We empirically compare our approach to eight state-of-the-art log anomaly detection methods on five benchmark datasets, showing that _Logs2Graphs_ performs at least on par and often better than its competitors.

The reminder of this paper is organised as follows. Section 2 revisits related work, after which Section 3 formalises the problem. Section 4 describes Digraph Inception Convolutional Networks (Dosovitskiy et al., 2016), which are used for _Logs2Graphs_ in Section 5. We then evaluate _Logs2Graphs_ in Section 6 and conclude in Section 7.

## 2. Related Work

Graph-based log anomaly detection methods usually comprise five steps: log parsing, log grouping, graph construction, graph representation learning, and anomaly detection. In this paper we focus on graph representation learning, log anomaly detection and explanation, thus only revisiting related work in these fields.

### Graph Representation Learning

Graph-level representation learning methods, such as GIN (Shi et al., 2017) and Graph2Vec (Kipf and Welling, 2017), are able to learn a mapping from graphs to vectors. Further, graph kernel methods, including Weisfeiler-Lehman (WL) (Welker and Hinton, 2010) and Propagation Kernels (PK) (Kipf and Welling, 2017), can directly provide pairwise distances between graphs. Both types of methods can be combined with off-the-shelf anomaly detectors, such as OCSVM (Kipf and Welling, 2017) and iForest (Kipf and Welling, 2017), to perform graph-level anomaly detection.

To improve on these naive approaches, efforts have been made to develop graph representation learning methods especially for anomaly detection. For instance, OCGIN (Wang et al., 2017) and GLAM (Wang et al., 2017) combine the GIN (Shi et al., 2017) representation learning objective with the SVDD objective (Dosovitskiy et al., 2016) to perform graph-level representation learning and anomaly detection in an end-to-end manner. GLocalKD (Kipf and Welling, 2017) performs random distillation of graph and node representations to learn 'normal' graph patterns. Further, OCGTL (Shi et al., 2017) combines neural transformation learning and one-class classification to learn graph representations for anomaly detection. Although these methods are unsupervised or semi-supervised, they can only deal with attributed, undirected, and unweighted graphs.

iGAD (Wang et al., 2017) considers graph-level anomaly detection as a graph classification problem and combines attribute-aware graph convolution and substructure-aware deep random walks to learn graph representations. However, iGAD is a supervised method, and can only handle attributed, undirected, and unweighted graphs. CODEtect (Kipf and Welling, 2017) takes a pattern-based modelling approach using the minimum description length (MDL) principle and identifies anomalous graphs based on _motifs_. CODEtect can (only) deal with labelled, directed, and edge-weighted graphs, but is computationally very expensive. To our knowledge, we introduce the first unsupervised method for graph-level anomaly detection that can handle attributed, directed and edge-weighted graphs.

### Log Anomaly Detection and Explanation

Log anomaly detection methods can be roughly subdivided into three categories: 1) traditional,'shallow' methods, such as principal component analysis (PCA) (Shen et al., 2016), one-class SVM (OCSVM) (Kipf and Welling, 2017), isolation forest (iForest) (Kipf and Welling, 2017) and histogram-based outlier score (HOS) (Kipf and Welling, 2017), which take a log event count matrix as input and analyse quantitative relationships; 2) deep learning based methods, such as DeepLog (Chen et al., 2017), LogAnomaly (Kipf and Welling, 2017), and AutoEncoder (Chen et al., 2017), which employ sequences of log events (and sometimes their semantic embeddings) as input, analysing sequential information and possibly semantic information of log events to identify anomalies; and 3) graph-based methods, such as TCFG (Kipf and Welling, 2017) and GLAD-PAW (Wang et al., 2017), which first convert logs into graphs and then perform graph-level anomaly detection.

To our knowledge, only a few works (Wang et al., 2017; Wang et al., 2017; Wang et al., 2017) have capitalised on the powerful learning capabilities of graph neural networks for log anomaly detection. GLAD-PAW (Wang et al., 2017) first transforms logs into attributed and undirected graphs and then uses a Position Aware Weighted Graph Attention Network to identify anomalies. However, converting logs into undirected graphs may result in loss of important sequential information. Further, DeepTraLog (Wang et al., 2017) combines traces and logs to generate a so-called Trace Event Graph, which is an attributed and directed graph. On this basis, they train
a Gated Graph Neural Networks based Deep Support Vector Data Description model to identify anomalies. However, their approach requires the availability of both traces and logs, and is unable to handle edge weights. In contrast, like LogGD (Wang et al., 2017), our proposed _Logs2Graphs_ approach is applicable to generic logs by converting logs into attributed, directed, and edge-weighted graphs. However, LogGD is a supervised method that requires fully labelled training data, which is usually impractical and even impossible. In contrast, our proposed algorithm OCDiGCN is the first _unsupervised_ graph-level anomaly detection method for attributed, directed, and edge-weighted graphs.

Although anomaly explanation has received much attention in traditional anomaly detection (Kumar et al., 2017), only a few studies (Wang et al., 2017) considered log anomaly explanation. Specifically, PLELog (Wang et al., 2017) offers explanations by quantifying the significance of individual log events within an anomalous log sequence, thereby facilitating improved identification of relevant log events by operators. Similarly, our method provides straightforward explanations for anomalous log groups by identifying and visualising a small subset of important nodes.

## 3. Problem Statement

Before we state the log anomaly detection problem, we first introduce the necessary notations and definitions regarding event logs and graphs.

**Event logs**. _Logs_ are used to record system status and important events, and are usually collected and stored centrally as log files. A _log file_ typically consists of many _log messages_. Each _log message_ is composed of three components: a timestamp, an event type (_log event_ or _log template_), and additional information (_log parameter_). _Log parsers_ are used to extract log events from log messages.

Further, log messages can be grouped into _log groups_ (a.k.a. _log sequences_) using certain criteria. Specifically, if a _log identifier_ is available for each log message, one can group log messages based on such identifiers. Otherwise, one can use a _fixed_ or _sliding window_ to group log messages. The _window size_ can be determined according to timestamp or the number of observations. Besides, counting the occurrences of each log event within a log group results in a _event count vector_. Consequently, for a log file consisting of many log groups, one can obtain a _event count matrix_. The process of generating an _event count matrix_ (or other feature matrix) is known as _feature extraction_. Extracted features are often used as input to an anomaly detection algorithm to identify _log anomalies_, i.e., log messages or log groups that deviate from what is considered 'normal'.

**Graphs**. We consider an attributed, directed, and edge-weighted graph \(\mathcal{G}=(\mathcal{V},\mathcal{E},\mathbf{X},\mathbf{Y})\), where \(\mathcal{V}=\{v_{1},...,v_{|\mathcal{V}|}\}\) denotes the set of _nodes_ and \(\mathcal{E}=\{e_{1},...,e_{|\mathcal{E}|}\}\subseteq\mathcal{V}\times\mathcal{ V}\) represents the set of edges between nodes. If \((v_{i},v_{j})\in\mathcal{E}\), then there is an edge from node \(v_{i}\) to node \(v_{j}\). Moreover, \(\mathbf{X}\in\mathbb{R}^{|\mathcal{V}|\times d}\) is the node attribute matrix, with the \(i\)-th row representing the attributes of node \(v_{i}\), and \(d\) is the number of attributes. Besides, \(\mathbf{Y}\in\mathbb{N}^{|\mathcal{E}|\times|\mathcal{E}|}\) is the edge-weight matrix, where \(\mathbf{Y}_{ij}\) represents the weight of the edge from node \(v_{i}\) to \(v_{j}\).

Equivalently, \(\mathcal{G}\) can be described as \((\mathbf{A},\mathbf{X},\mathbf{Y})\), with adjacency matrix \(\mathbf{A}\in\mathbb{R}^{|\mathcal{V}|\times|\mathcal{V}|}\), where \(\mathbf{A}_{ij}=\mathbb{I}[(v_{i},v_{j})\in\mathcal{E}]\) indicates whether there is an edge from node \(v_{i}\) to node \(v_{j}\), for \(i,j\in\{1,...,|\mathcal{V}|\}\).

### Graph-based Log Anomaly Detection

Given a set of log files, we let \(\mathcal{L}=\{L_{1},...,L_{|\mathcal{L}|}\}\) denote the set of unique log events. We divide the log messages into \(M\) log groups \(\mathbf{Q}=\{\mathbf{q}_{1},...,\mathbf{q}_{m},...,\mathbf{q}_{M}\}\), where \(\mathbf{q}_{m}=\{\mathbf{q}_{m1},...,\mathbf{q}_{mmv},...,\mathbf{q}_{mN}\}\) is a log group and \(\mathbf{q}_{mn}\) a log message.

For each log group \(\mathbf{q}_{m}\), we construct an attributed, directed, and edge-weighted graph \(\mathcal{G}_{m}=(\mathcal{V}_{m},\mathcal{E}_{m},\mathbf{X}_{m},\mathbf{Y}_{m})\) to represent the log messages and their relationships. Specifically, each node \(v_{i}\in\mathcal{V}_{m}\) corresponds to exactly one log event \(L\in\mathcal{L}\) (and vice versa). Further, an edge \(e_{ij}\in\mathcal{E}_{m}\) indicates that log event \(i\) is at least once immediately followed by log event \(j\) in \(\mathbf{q}_{m}\). Attributes \(\mathbf{x}_{i}\in\mathbf{X}_{m}\) represent the semantic embedding of log event \(i\), and \(y_{ij}\in\mathbf{Y}_{m}\) is the weight of edge \(e_{ij}\), representing the number of times event \(i\) was immediately followed by event \(j\). In this manner, we construct a set of log graphs \(\{\mathcal{G}_{1},...,\mathcal{G}_{m},...,\mathcal{G}_{M}\}\).

We can use these definitions to define graph-level anomaly detection:

**Problem 1 (Graph-based Log Anomaly Detection)**. _Given a set of attributed, directed, and weighted graphs that represent logs, find those graphs that are notably different from the majority of graphs._

What we mean by 'notably different' will have to be made more specific when we define our method, but we can already discuss what types of anomalies can potentially be detected. Most methods aim to detect two types of anomalies:

* A log group (namely a graph) is considered a _quantitative anomaly_ if the occurrence frequencies of some events in the group are higher or lower than expected from what is commonly observed. For example, if a file is opened (event \(A\)) twice, it should normally also be closed (event \(B\)) twice. In other words, the number of event occurrences \(\#A=\#B\) in a normal pattern and an anomaly is detected if \(\#A\neq\#B\).
* A log group (namely a graph) is considered to contain _sequential anomalies_ if the order of certain events violates the normal order pattern. For instance, a file can be closed only after it has been opened in a normal workflow. In other words, the order of event occurrences \(A\to B\) is considered normal while \(B\to A\) is considered anomalous.

An advantage of graph-based anomaly detection is that it detect these two types of anomalies, but also anomalies reflected in the structure of the graphs. Moreover, no _unsupervised_ log anomaly detection approaches represent event logs as attributed, directed, weighted graphs, which allow for even higher expressiveness than undirected graphs (and thus limiting the information loss resulting from the representation of the log files as graphs).

## 4. Preliminaries: Digraph Inception Convolutional Nets

To learn node representations for attributed, directed, and edge-weighted graphs, (Wang et al., 2017) proposed Digraph Inception Convolutional Networks (DiGCN).
Specifically, given a graph \(\mathcal{G}\) described by an adjacency matrix \(\mathbf{A}\in\mathbb{R}^{|\mathcal{V}|\times|\mathcal{V}|}\), a node attribute matrix \(\mathbf{X}\in\mathbb{R}^{|\mathcal{V}|\times d}\), and an edge-weight matrix \(\mathbf{Y}\in\mathbb{R}^{|\mathcal{V}|\times|\mathcal{V}|}\), DiGCN defines the \(k\)-th order digraph convolution as

\[\mathbf{Z}^{(k)}=\begin{cases}\mathbf{X}\Theta^{(0)}&k=0\\ \Psi\mathbf{X}\mathbf{Q}^{(1)}&k=1\\ \Phi\mathbf{X}\mathbf{Q}^{(k)}&k\geq 2,\end{cases} \tag{1}\]

where \(\Psi=\frac{1}{2}\left(\Pi^{(1)}\frac{1}{2}\mathbf{P}^{(1)}\Pi^{(1)}\Pi^{(1)} \frac{-1}{2}+\Pi^{(1)}\frac{-1}{2}\mathbf{P}^{(1)T}\Pi^{(1)}\frac{1}{2}\right)\) and \(\Phi=\mathbf{W}^{(k)}\frac{-1}{2}\mathbf{P}^{(k)}\mathbf{W}^{(k)}\frac{-1}{2}\). Particularly, \(\mathbf{Z}^{(k)}\in\mathcal{R}^{|\mathcal{V}|\times f}\) denote the convolved output with \(f\) output dimension, and \(\Theta^{(0)},\Theta^{(1)},\Theta^{(k)}\) represent the trainable parameter matrices.

Moreover, \(\mathbf{P}^{(k)}\) is the \(k\)-th order proximity matrix defined as

\[\mathbf{P}^{(k)}=\begin{cases}\mathbf{I}&k=0\\ \tilde{\mathbf{D}}^{-1}\tilde{\mathbf{A}}&k=1\\ Ins\left((\mathbf{P}^{(1)})^{(k-1)}(\mathbf{P}^{(1)T})^{(k-1)}\right)&k\geq 2,\end{cases} \tag{2}\]

where \(\mathbf{I}\in\mathcal{R}^{|\mathcal{V}|\times|\mathcal{V}|}\) is an identity matrix, \(\tilde{\mathbf{A}}=\mathbf{A}+\mathbf{I}\), and \(\tilde{\mathbf{D}}\) denotes the diagonal degree matrix with \(\tilde{\mathbf{D}}_{ii}=\sum_{j}\tilde{\mathbf{A}}_{ij}\). Besides, \(Ins\left((\mathbf{P}^{(1)})^{(k-1)}(\mathbf{P}^{(1)T})^{(k-1)}\right)\) is defined as

\[\frac{1}{2}Intersect\left((\mathbf{P}^{(1)})^{(k-1)}(\mathbf{P}^{(1)T})^{(k-1) },(\mathbf{P}^{(1)T})^{(k-1)}(\mathbf{P}^{(1)})^{(k-1)}\right) \tag{3}\]

, with \(Intersect(\cdot)\) denoting the element-wise intersection of two matrices (see [33] for computation details). In addition, \(\mathbf{W}^{(k)}\) is the diagonalized weight matrix of \(\mathbf{P}^{(k)}\), and \(\Pi^{(1)}\) is the approximate diagonalized eigenvector of \(\mathbf{P}^{(1)}\). Particularly, the approximate diagonalized eigenvector is calculated based on personalised PageRank [2], with a parameter \(\alpha\) to control the degree of conversion from a digraph to an undirected graph. We omit the details to conserve space, and refer to [33] for more details.

After obtaining the multi-scale features \(\{\mathbf{Z}^{(0)},\mathbf{Z}^{(1)},...,\mathbf{Z}^{(k)}\}\), DiGCN defines an Inception block as

\[\mathbf{Z}=\sigma\left(\Gamma\left(\mathbf{Z}^{(0)},\mathbf{Z}^{(1)},..., \mathbf{Z}^{(k)}\right)\right), \tag{4}\]

where \(\sigma\) represents an activation function, and \(\Gamma(\cdot)\) denotes a fusion operation, which can be summation, normalisation, and concatenation. In practice, we often adapt a fusion operation that keeps the output dimension unchanged, namely \(\mathbf{Z}\in\mathcal{R}^{|\mathcal{V}|\times f}\). As a result, the \(i\)-th row of \(\mathbf{Z}\) (namely \(\mathbf{Z}_{i}\)) denotes the learned vector representation for node \(v_{i}\) in a certain layer.

## 5. Graph-Based Anomaly Detection for Event Logs

We propose _Logs2Graphs_, a graph-based log anomaly detection method tailored to event logs. The overall pipeline consists of the usual main steps, i.e., log parsing, log grouping, graph construction, graph representation learning, and anomaly detection, and is illustrated in Figure 1. Note that we couple the graph representation learning and anomaly detection steps to accomplish end-to-end learning once the graphs have constructed.

First, after collecting logs from a system, the _log parsing_ step extracts log events and log parameters from raw log messages. Since log parsing is not the primary focus of this article, we use Drain [11] for this task. Drain is a log parsing technique with fixed depth tree, and has been shown to generally outperform its competitors [47]. We make the following assumptions on the log files:

* Logs files are written in English;
* Each log message contains at least the following information: date, time, operation detail, and log identifier;
* The logs contain enough events to make the mined relationships (quantitative, sequential, structural) statistically meaningful, i.e., it must be possible to learn from the logs what the 'normal' behaviour of the system is.

Second, the _log grouping_ step uses the log identifiers to divide the parsed log messages into log groups. Third, for each resulting group of log messages, the _graph construction_ steps builds an attributed, directed, and edge-weighted graph, as described in more detail in Subsection 5.1. Fourth and last, in an integrated step for _graph representation learning and anomaly detection_, we learn a One-Class Digraph Inception Convolutional Network (OCDiGCN) based on the obtained set of log graphs. The resulting model can be used for graph-level anomaly detection. This model couples the graph representation learning objective and anomaly detection objective, and is thus trained in an end-to-end manner. The model, its training, and its use for graph-level anomaly detection are explained in detail in Subsection 5.2.

### Graph Construction

We next explain how to construct an attributed, directed, and edge-weighted graph given a group of parsed log messages, and illustrate this in Figure 2. Particularly, the motivation of graph construction is to keep everything relevant in log data.

First, we construct nodes to represent the different log events. That is, the number of nodes depends on the number of unique log events that occur in the log group. Second, starting from the first line of log messages in chronological order, we add an directed edge from log event \(L_{i}\) to \(L_{j}\) and set its edge-weight to \(1\) if the next event after \(L_{i}\) is \(L_{j}\). If the corresponding edge already exists, we increase its edge-weight by \(1\). In this manner, we obtain a labelled, directed, and edge-weighted graph.

However, using only the labels (e.g., _open_ or _write_) of log events for graph construction may lead to missing important information. That is, we can improve on this by explicitly taking the semantic information of log events into account, by which we mean that we should look at the text of the log event in entirety. Specifically, we generate a vector representation for each log event as follows:

1. _Preprocessing:_ for each log event, we first remove non-character words and stop words, and split compound words into separate words;
2. _Word embedding:_ we use Glove [25], a pre-trained word embedding model with \(200\) embedding dimensions to generate a vector representation for each English word in a log event;
3. _Sentence embedding:_ we generate a vector representation for each log event. Considering that the words in a sentence are usually not of equal importance, we use Term Frequency-Inverse Document frequency (TF-IDF) [27] to measure the importance of words. As a result, the weighted sum of word embedding vectors composes the vector representation of a log event.
By augmenting the nodes with the vector representations of the log events as attributes, we obtain an attributed, directed, and edge-weighted graph.

### OCDiGCN: One-Class Digraph Inception Convolutional Nets

We next describe One-Class Digraph Inception Convolutional Networks, abbreviated as OCDiGCN, a novel method for end-to-end graph-level anomaly detection. We chose to build on Digraph Inception Convolutional Networks (DiGCN) (Shen et al., 2017) for their capability to handle directed graphs, which we argued previously is an advantage in graph-based log anomaly detection.

Considering that DiGCN was designed for node representation learning, we repurpose it for graph representation learning as follows:

\[\mathbf{z}=\text{Readout}(\mathbf{Z}_{i}\mid i\in\{1,2,...,|\mathcal{V}|\}). \tag{4}\]

That is, at the final iteration layer, we utilise a so-called Readout(\(\cdot\)) function to aggregate node vector representations to obtain a graph vector representation. Importantly, Readout(\(\cdot\)) can be a simple permutation-invariant function such as maximum, sum or mean, or a more advanced graph-level pooling function (Zhu et al., 2017).

Next, note that DiGCN work did not explicitly enable learning edge features (i.e., \(\mathbf{Y}\)). However, as DiGCN follows the Message Passing Neural Network (MPNN) framework (Dong et al., 2017), incorporating \(\mathbf{Y}\) into Equation (1) and conducting computations in Equations (2-4) analogously enables learning edge features.

Now, given a set of graphs \(\{\mathcal{G}_{1},...,\mathcal{G}_{m},...,\mathcal{G}_{M}\}\), we can use Equation (4) to obtain an explicit vector representation for each graph, respectively. We denote the vector presentation of \(\mathcal{G}_{m}\) learned by the DiGCN model as DiGCN(\(\mathcal{G}_{m}\);\(\mathcal{H}\)).

In graph anomaly detection, anomalies are typically identified based on a reconstruction or distance loss (Krizhevsky et al., 2014). In particular, the One-Class Deep SVDD objective (Wang et al., 2017) is commonly used for two reasons: it can be easily combined with other neural networks, and more importantly, it generally achieves a state-of-the-art performance (Wang et al., 2017). Therefore, to detect anomalies, we train a one-class classifier by optimising the following One-Class Deep SVDD objective:

\[\min_{\mathcal{H}}\frac{1}{M}\sum_{m=1}^{M}\lVert\text{DiGCN}(\mathcal{G}_{m} ;\mathcal{H})-\mathbf{o}\rVert_{2}^{2}+\frac{\lambda}{2}\sum_{l=1}^{L}\lVert \mathbf{H}^{(l)}\rVert_{\text{F}}^{2}, \tag{5}\]

where \(\mathbf{H}^{(l)}\) represents the trainable parameters of DiGCN at the \(l\)-th layer, namely \((\Theta^{(0)(l)},\Theta^{(1)(l)},...,\Theta^{(k)(l)})^{T},\mathcal{H}\) denotes \(\{\mathbf{H}^{(1)},...,\mathbf{H}^{(L)}\}\), \(\lambda>0\) represents the weight-decay hyperparameter, \(\lVert\cdot\rVert_{2}\) is the Euclidean norm, and \(\lVert\cdot\rVert_{F}\) denotes the Frobenius norm. Moreover, \(\mathbf{o}\) is the center of the hypersphere in the learned representation space. Ruff et al. (Ruff et al., 2017) found empirically that setting \(\mathbf{o}\) to the average of the

Figure 1. The Logs2Graphs pipeline. We use attributed, directed, and weighted graphs for representing the log files with high expressiveness, and integrate representation learning and anomaly detection for accurate anomaly detection. We use off-the-shelf methods for log parsing, log grouping, and graph construction.

Figure 2. The construction of an attributed, directed, and edge-weighted graph from a group of log messages.
network representations (i.e., graph representations in our case) obtained by performing an initial forward pass is a good strategy.

Ruff et al. (Ruff et al., 2017) also pointed out, however, that One-Class Deep SVDD classification may suffer from a hypersphere collapse, which will yield trivial solutions, namely mapping all graphs to a fixed center in the representation space. To avoid a hypersphere collapse, the hypersphere center \(\mathbf{o}\) is set to the average of the network representations, the bias terms in the neural networks are removed, and unbounded activation functions such as ReLU are preferred.

After training the model on a set of non-anomalous graphs (or with a very low proportion of anomalies), given a test graph \(\mathcal{G}_{m}\), we define its distance to the center in the representation space as its anomaly score, namely

\[score(\mathcal{G}_{m})=\|\mathrm{DiGCN}(\mathcal{G}_{m};\mathcal{H})-\mathbf{o }\|_{2}. \tag{6}\]

**Training and hyperparameters:** In summary, OCDiGCN is composed of an \(L\)-layer DiGCN architecture to learn node representations, plus a Readout(\(\cdot\)) function to obtain the graph representation. It is trained in an end-to-end manner via optimising the SVDD objective, which can be optimised using stochastic optimisation techniques such as Adam (Kingmaa and Ba, 2014). Overall, OCDiGCN takes a collection of non-anomalous graphs and a set of hyperparameters, which are outlined in Table 2, as inputs. Importantly, the pseudo-code for Logs2Graphs is given in Algorithm 1.

```
0: Training dataset \(D_{tr}\), testing dataset \(D_{ts}\), model \(\theta\)
0: Predicted labels and explanations for \(D_{ts}\)
1: Parse \(D_{tr}\) and \(D_{ts}\) using Drain (Kingmaa and Ba, 2014)\(\rightarrow\) Obtain parsed datasets \(\hat{D}_{tr}\) and \(\hat{D}_{ts}\)
2: Group \(\hat{D}_{tr}\) and \(\hat{D}_{ts}\) based on log identifier \(\rightarrow\) Obtain grouped dataset \(\hat{D}_{tr}\) and \(\hat{D}_{ts}\)
3: Construct graphs using \(\hat{D}_{tr}\) and \(\hat{D}_{ts}\)\(\rightarrow\) Obtain graph sets \(\mathbf{Q}_{tr}\) and \(\mathbf{Q}_{ts}\)
4: Train the OCDiGCN model using Equation (5) with \(\mathbf{Q}_{tr}\)\(\rightarrow\) Obtain trained model \(\hat{\theta}\)
5: Use \(\hat{\theta}\) to predict anomalies in \(\mathbf{Q}_{ts}\)\(\rightarrow\) Obtain a set of anomalies \(\{Q_{1},...,Q_{n}\}\)
6: Generate explanations for \(Q_{i}\in\{Q_{1},...,Q_{n}\}\)
```

**Algorithm 1** Pseudo-code of Logs2Graphs

### Anomaly Explanation

Our anomaly explanation method can be regarded as a decomposition method (Kingmaa and Ba, 2014), that is, we build a score decomposition rule to distribute the prediction anomaly score to the input space. Concretely, a graph \(\mathcal{G}_{m}\) is identified as anomalous if and only if its graph-level representation has a large distance to the hyper-sphere center (Equation 6). Further, the graph-level representation is obtained via a Readout(\(\cdot\)) function applied on the node-level representations (Equation 4). Therefore, if the Readout(\(\cdot\)) function is attributable (such as sum or mean), we can easily obtain the a small subset of important nodes (in the penultimate layer) whose node embeddings contribute the most to the distance. Specifically, the importance score of node \(v_{j}\) (in the penultimate layer) in a graph \(\mathcal{G}_{m}\) is defined as:

\[\frac{|score(\mathcal{G}_{m})-score(\mathcal{G}_{m}/\{\mathbf{Z}_{j}\})|}{score (\mathcal{G}_{m})} \tag{7}\]

where \(score(\mathcal{G}_{m})\) is defined in Equation 6 and \(score(\mathcal{G}_{m}/\{\mathbf{Z}_{j}\})\) is the anomaly score by removing the embedding vector of \(v_{j}\) (namely \(\mathbf{Z}_{j}\)) when applying Readout function to obtain the graph-level representation.

Next, for each important node in the penultimate layer, we extend the LRP (Layerwise Relevance Propagation) algorithm (Bogorst and Welling, 2014) to obtain a minor set of important nodes in the input layer (this is not the contribution of our paper and we simply follow the practice in (Bogorst and Welling, 2014; Lees and Vanhoucke, 2015)). If certain of these nodes are connected by edges, the resulting subgraphs can provide more meaningful explanations. As the LRP method generates explanations utilizing the hidden features and model weights directly, its explanation outcomes are deemed reliable and trustworthy (Kingmaa and Ba, 2014).

\begin{table}
\begin{tabular}{l l l l l l} \hline \hline Name & \#Events & \#Graphs & \#Anomalies & \#Nodes & \#Edges \\ \hline HDFS & 48 & 575,061 & 16,838 & 7 & 20 \\ Hadoop & 683 & 978 & 811 & 34 & 120 \\ BGL & 1848 & 69,251 & 31,374 & 10 & 30 \\ Spirit & 834 & 10,155 & 4,432 & 6 & 24 \\ Thunderbird & 1013 & 52,160 & 6,814 & 16 & 52 \\ \hline \hline \end{tabular}
\end{table}
Table 1. Summary of datasets. #Events refers to the number of log event templates obtained using log parser Drain (Kingmaa and Ba, 2014). #Groups means the number of generated graphs. #Anomalies represents the number of anomalous graphs. #Nodes denotes the average number of nodes in generated graphs. #Edges indicates the average number of edges in the generated graphs.

\begin{table}
\begin{tabular}{l l l} \hline \hline
**Symbol** & **Meaning** & **Range** \\ \hline _Bs_ & Batch size & \{16, 32, 64, **128**, 256, 512, 1024, 1536, 2048, 2560\} \\ \hline _Op_ & optimisation method & Adam, **SGD** \\ \hline \(L\) & number of layers & \{\(1,2,3,4,5\)\} \\ \hline \(\lambda\) & weight decay parameter & \{**0.0001**, 0.001, 0.01, 0.1\} \\ \hline \(\eta\) & learning rate & \{0.0001, 0.001, **0.01\} \\ \hline \(k\) & proximity parameter & \{\(1,2\)\} \\ \hline \(\alpha\) & teleport probability & \{0.05, **0.1**, 0.2\} \\ \hline \(\Gamma\) & fusion operation if \(k\geq 2\) & sum, concatenation \\ \hline _Re_ & readout function & **mean**, sum, max \\ \hline \(d\) & embedding dimension & \{32, 64, **128**, 256, 300\} \\ \hline _Ep_ & Epochs for training & range(100,1000,50) \\ \hline \hline \end{tabular}
\end{table}
Table 2. Description of hyperparameters involved in OCDiGCN. Range indicates the values that we have tested, and boldfaced values represent the values suggested to use in experiments. Particularly, for the embedding dimensions: 300 is suggested for BGL and 128 for others. For the batch sizes: 32 is suggested for HDFS and 128 for others. For the training epochs: 100 for BGL and Thunderbird, 200 for HDFS, 300 for Hadoop and 500 for Spirit are suggested.
## 6. Experiments

We perform extensive experiments to answer the following questions:

1. **Detection accuracy:** How effective is _Logs2Graphs_ at identifying log anomalies when compared to state-of-the-art methods?
2. **Directed vs. undirected graphs:** Is the directed log graph representation better than the undirected version for detecting log anomalies?
3. **Node Labels vs. Node Attributes**: How important is it to use semantic embedding of log event template as node attributes?
4. **Robustness analysis:** To what extent is Logs2Graphs robust to contamination in training data?
5. **Ability to detect structural anomalies:** Can Logs2Graphs better capture structural anomalies and identify structurally equivalent normal instances than other contenders?
6. **Explainability Analysis:** How understandable are the anomaly detection results delivered by Logs2Graphs?
7. **Sensitivity analysis:** How do the values of the hyperparameters influence the detection accuracy?
8. **Runtime analysis:** What are the runtime for different methods?

### Experiment Setup

#### 6.1.1. Datasets

The five datasets that we use, summarised in Table 1, were chosen for three reasons: 1) they are commonly used for the evaluation of log anomaly detection methods; 2) they contain ground truth labels that can be used to calculate evaluation metrics; and 3) they include log identifiers that can be used for partitioning log messages into groups. For each group of log messages in a dataset, we label the group as anomalous if it contains at least one anomaly. More details are given as follows:

* HDFS (Han et al., 2017) consists of Hadoop Distributed File System logs obtained by running 200 Amazon EC2 nodes. These logs contain _block_id_, which can be used to group log events into different groups. Moreover, these logs are manually labeled by Hadoop experts.
* Hadoop (Han et al., 2017) was collected from a Hadoop cluster consisting of 46 cores over 5 machines. The _ContainerID_ variable is used to divide log messages into different groups.
* BGL, Spirit and Thunderbird contain system logs collected from the BlueGene/L (BGL) supercomputing system, Spirit supercomputing system, and Thunderbird supercomputing system located at Sandia National Labs, respectively. For those datasets, each log message was manually inspected by engineers and labelled as normal or anomalous. For BGL, we use all log messages, and group log messages based on the _Node_ variable. For Spirit and Thunderbird, we only use the first 1 million and first 5 million log messages for evaluation, respectively. Furthermore, for these two datasets, the _User_ is used as log identifier to group log messages. However, considering that an ordinary user may generate hundreds of thousands of logs, we regard every 100 consecutive logs of each user as a group. If the number of logs is less than 100, we also consider it as a group.

#### 6.1.2. Baselines

To investigate the performance of _Logs2Graphs_, we compare it with the following seven state-of-the-art log anomaly detection methods: Principal Component Analysis (PCA) (Han et al., 2017), OneClass SVM (OCSVM) (Krishnan et al., 2016), Isolation Forest (iForest) (Han et al., 2017), HBOS (Han et al., 2017), DeepLog (Chen et al., 2017), LogAnomaly (Han et al., 2017) and AutoEncoder (Chen et al., 2017), and one state-of-the-art graph level anomaly detection method: GLAM (Zhou et al., 2017).

We choose these methods as baselines because they are often regarded to be representatives of traditional machine learning-based (PCA, OCSCM, IForest, HBOS) and deep learning-based approaches (DeepLog, LogAnomaly and AutoEncoder), respectively. All methods are unsupervised or semi-supervised methods that do not require labeled anomalous samples for training the models.

#### 6.1.3. Evaluation Metrics

The Area Under Receiver Operating Characteristics Curve (AUC ROC) and the Area Under the Precision-Recall Curve (AUC PRC) are widely used to quantify the detection accuracy of anomaly detection. Therefore, we employ both to evaluate and compare the different log anomaly detection methods. AUC PRC is also known as Average Precision (AP). For both AUC ROC and AUC PRC, values closer to 1 indicate better performance.

### Model Implementation and Configuration

Traditional machine learning based approaches--such as PCA, OCSVM, iForest, and HBOS--usually first transform logs into log event count vectors, and then apply traditional anomaly detection techniques to identify anomalies. For these methods, we utilise their resource implementations provided in PyOD (Zhou et al., 2017). Meanwhile, for deep learning methods DeepLog, LogAnomaly, and AutoEncoder, we use their open-source implementations in Deep-Loglizer (Chen et al., 2017). For these competing methods, we use their default hyperparameter values.

For all deep learning based methods, the experimental design adopted in this study follows a train/validation/test strategy with a distribution of 70% : 5% : 25% for normal instances. Specifically, the model was trained using 70% of normal instances, while 5% of normal instances and an equal number of abnormal instances were employed for validation (i.e., hyperparameter tuning). The remaining 25% of normal instances and the remaining abnormal instances were used for testing. Specifically, Table 2 summarises the hyperparameters involved in OCDiGCN as well as their recommended values.

We implemented and ran all algorithms in Python 3.8 (using PyTorch (Paszasz et al., 2017) and PyTorch Geometric (Chen et al., 2017) libraries when applicable), on a computer with Apple M1 chip 8-core CPU and 16GB unified memory. For reproducibility, all code and datasets will be released on GitHub.

### Comparison to the state of the art

We first compare _Logs2Graphs_ to the state of the art. The results are shown in Table 3, based on which we make the following main observations:

* In terms of AUC ROC, _Logs2Graphs_ achieves the best performance against its competitors on four out of five datasets. Particularly, _Logs2Graphs_ outperforms the closet competitor on BGL with 9.6% and delivers remarkable results (i.e., an AUC ROC larger than 0.99) on Spirit and Thunderbird. Similar observations can be made for Average Precision.
* Deep learning based methods generally outperform the traditional machine learning based methods. One possible reason is that traditional machine learning based methods only leverage log event count vectors as input, which makes them unable to capture and exploit sequential relationships between log events and the semantics of the log templates.
* The performance of (not-graph-based) deep learning methods is often inferior to that of _Log2Graphs_ on the more complex datasets, i.e., Hadoop, BGL, Spirit, and Thunderbird, which all contain hundreds or even thousands of log templates. This suggests that LSTM-based models may not be well suited for logs with a large number of log templates. One possible reason is that the test dataset contains many unprecedented log templates, namely log templates that are not present in the training dataset.
* In terms of ROC AUC score, all methods except for OCSVM and AutoEncoder achieve impressive results (with \(RC>0.91\)) on HDFS. One possible reason is that HDFS is a relatively simple log dataset that contains only 48 log templates. Concerning AP, PCA and LSTM-based DeepLog achieve impressive results (with \(AP>0.89\)) on HDFS. Meanwhile, _Logs2Graphs_ obtains a competitive performance (with \(AP=0.87\)) on HDFS.

### Directed vs. undirected graphs

To investigate the practical added value of using _directed_ log graphs as opposed to _undirected_ log graphs, we convert the logs to attributed, undirected, and edge-weighted graphs, and apply GLAM (Wang et al., 2019), a graph-level anomaly detection method for undirected graphs. We use the same graph construction method as for _Logs2Graphs_, except that we use undirected edges. Similar to our method, GLAM also couples the graph representation learning and anomaly detection objectives by optimising a single SVDD objective. The key difference with OCDiGCN is that GLAM leverages GIN (Wang et al., 2019), which can only tackle undirected graphs, while OCDiGCN utilises DiGCN (Wang et al., 2019) that is especially designed for directed graphs.

The results in Table 3 indicate that GLAM's detection performance is comparable to that of most competitors. However, it consistently underperforms on all datasets, except for Hadoop, when compared to _Logs2Graphs_. Given that the directed vs undirected representation of the log graphs is the key difference between the methods, a plausible explanation is that directed graphs have the capability to retain the temporal sequencing of log events, whereas undirected graphs lack this ability. Consequently, GLAM may encounter difficulties in detecting sequential anomalies and is outperformed by _Logs2Graphs_.

### Node Labels vs. Node Attributes

To investigate the importance of using semantic embedding of log event template as node attributes, we replace the node semantic attributes with one-hot-encoding of node labels (i.e., using an integer to represent a log event template). The performance comparisons in terms of ROC AUC for Logs2Graphs are depicted in Figure 3, which shows that using semantic embedding is always superior to using node labels. Particularly, it can lead to a substantial performance improvement on Hadoop, Spirit and HDFS datasets. The PR AUC results show a similar behaviour and thus are omitted.

### Robustness to Contamination

To investigate the robustness of Logs2Graphs when the training dataset is contaminated, we report its performance in terms of ROC AUC under a wide range of contamination levels. Figure 4 shows that the performance of Logs2Graphs decreases with an increase of contamination in the training data. The PR AUC results show a similar behaviour and thus are omitted. Hence, it is important to ensure that the training data contains only normal graphs (or with a very low proportion of anomalies).

### Ability to Detect Structural Anomalies and Recognise Unseen Normal Instances

To showcase the effectiveness of different neural networks in detecting structural anomalies, we synthetically generate normal and anomalous directed graphs as shown in Figure 5. As Deeplog, LogAnomaly and AutoEncoder require log sequences as inputs, we convert directed graphs into sequences by sequentially presenting the endpoints pair of each edge. Moreover, for GLAM we convert directed graphs into undirected graphs by turning each directed edge into undirected edge.

Figure 4. ROC results of Logs2Graphs w.r.t. a wide range of contamination levels. Results are averaged over 10 runs. Particularly, HDFS contains only 3% anomalies and thus results at 5% and 10% are not available.

Figure 3. The comparative performance analysis of Logs2Graphs, measured by ROC AUC, demonstrating the distinction between utilizing node semantic attributes and node labels.
Moreover, to investigate their capability of recognising unseen but structurally equivalent normal instances, we generate the following normal log sequences based on the synthetic normal graph as training data: \(A\to B\to C\to D\to A\) (1000), \(B\to C\to D\to A\to B\) (1000) and \(C\to D\to A\to B\) (1000), and the following as test dataset: \(D\to A\to B\to C\to D\) (1000).

Specifically, the results in Table 4 indicate that Logs2Graphs, Deelog and LogAnomaly can effectively detect structural anomalies while AutoEncoder and GLAM fail in some cases. However, log sequences based methods, namely Deelog, LogAnomaly and AutoEncoder, can lead to high positive rates due to their inability of recognising unseen but structurally equivalent normal instances.

### Anomaly Explanation

Particularly, Figure 6 provides an example of log anomaly explanation with the HDFS dataset. For each detected anomalous log graph (namely a group of logs), we first quantify the importance of nodes according to the description in Section 5.3. Next, we visualise the anomalous graph by assigning darker shade of red to more important nodes. In this example, the node "WriteBlock(WithException)" contributes the most to the anomaly score of an anomalous log group and thus is highlighted in red.

### Sensitivity Analysis

We examine the effects of three hyperparameters in OCDiGCN on the detection performance.

**The Number of Convolutional Layers:**\(L\) is a potentially important parameter as it determines how many convolutional layers to use in OCDiGCN. Figure 7 (top row) depicts PR AUC and ROC AUC for the five benchmark datasets when \(L\) is varied from 1 to 5. We found that \(L=1\) yields consistently good performance. As the value of \(L\) is increased, there is only a slight enhancement in the resulting performance or even degradation, while the associated

\begin{table}
\begin{tabular}{l c c c c c} \hline \hline
**Case** & \multicolumn{2}{c}{Deelog LogAnomaly AutoEncoder GLAM Ours} \\ \hline S1 (RUC) & 1.0 & 1.0 & 0.0 & 0.0 & 1.0 \\ S2 (RUC) & 1.0 & 1.0 & 0.50 & 1.0 & 1.0 \\ S3 (RUC) & 1.0 & 1.0 & 1.0 & 1.0 & 1.0 \\ S4 (RUC) & 1.0 & 1.0 & 1.0 & 1.0 & 1.0 \\ \hline N1 (FPR) & 100\% & 100\% & 100\% & 0\% & 0\% \\ \hline \hline \end{tabular}
\end{table}
Table 4. ROC AUC Results (higher is better) of detecting structural anomalies and False Positive Rate (lower is better) of recognising unseen normal instances. S1: Reverse Edge Direction; S2: Change Edge Endpoint; S3: Delete Edge; S4: Add Edge; N1: Unseen normal instances.

Figure 5. Synthetic generation of normal (10000) and structurally anomalous (200 each) graphs.

\begin{table}
\begin{tabular}{l c c c c c c c c c} \hline \hline  & \multicolumn{2}{c}{**HDFS**} & \multicolumn{2}{c}{**Hadoop**} & \multicolumn{2}{c}{**BGL**} & \multicolumn{2}{c}{**Spirit**} & \multicolumn{2}{c}{**Thunderbird**} \\ \cline{2-11} Method & AP & RC & AP & RC & AP & RC & AP & RC & AP & RC \\ \hline PCA & 0.91\(\pm\)0.03 & **1.0**\(\pm\)0.00 & 0.84\(\pm\)0.00 & 0.52\(\pm\)0.00 & 0.73\(\pm\)0.01 & 0.82\(\pm\)0.00 & 0.31\(\pm\)0.00 & 0.19\(\pm\)0.00 & 0.11\(\pm\)0.00 & 0.34\(\pm\)0.01 \\ OCSVM & 0.18\(\pm\)0.01 & 0.88\(\pm\)0.01 & 0.83\(\pm\)0.00 & 0.45\(\pm\)0.00 & 0.47\(\pm\)0.00 & 0.47\(\pm\)0.01 & 0.34\(\pm\)0.00 & 0.29\(\pm\)0.00 & 0.12\(\pm\)0.00 & 0.45\(\pm\)0.01 \\ IForest & 0.73\(\pm\)0.04 & 0.97\(\pm\)0.01 & 0.85\(\pm\)0.01 & 0.55\(\pm\)0.01 & 0.79\(\pm\)0.01 & 0.83\(\pm\)0.01 & 0.32\(\pm\)0.03 & 0.23\(\pm\)0.02 & 0.11\(\pm\)0.01 & 0.24\(\pm\)0.10 \\ HBOS & 0.74\(\pm\)0.04 & 0.99\(\pm\)0.00 & 0.84\(\pm\)0.00 & 0.50\(\pm\)0.00 & 0.84\(\pm\)0.02 & 0.87\(\pm\)0.03 & 0.35\(\pm\)0.00 & 0.22\(\pm\)0.00 & 0.15\(\pm\)0.01 & 0.29\(\pm\)0.05 \\ \hline DeepLog & **0.92\(\pm\)**0.07 & 0.97\(\pm\)0.04 & **0.96\(\pm\)**0.00 & 0.47\(\pm\)0.00 & 0.89\(\pm\)0.00 & 0.72\(\pm\)0.00 & 0.99\(\pm\)0.00 & 0.97\(\pm\)0.00 & 0.91\(\pm\)0.01 & 0.96\(\pm\)0.00 \\ LogAnomaly & 0.89\(\pm\)0.09 & 0.95\(\pm\)0.05 & **0.96\(\pm\)**0.00 & 0.47\(\pm\)0.00 & 0.89\(\pm\)0.00 & 0.72\(\pm\)0.00 & 0.99\(\pm\)0.00 & 0.97\(\pm\)0.00 & 0.90\(\pm\)0.01 & 0.96\(\pm\)0.00 \\ AutoEncoder & 0.71\(\pm\)0.03 & 0.84\(\pm\)0.01 & **0.96\(\pm\)**0.00 & 0.52\(\pm\)0.00 & 0.91\(\pm\)0.01 & 0.79\(\pm\)0.02 & 0.96\(\pm\)0.00 & 0.92\(\pm\)0.01 & 0.44\(\pm\)0.02 & 0.46\(\pm\)0.05 \\ \hline GLAM & 0.78\(\pm\)0.08 & 0.89\(\pm\)0.04 & 0.95\(\pm\)0.00 & **0.61\(\pm\)**0.00 & 0.94\(\pm\)0.02 & 0.90\(\pm\)0.03 & 0.93\(\pm\)0.00 & 0.91\(\pm\)0.00 & 0.75\(\pm\)0.02 & 0.85\(\pm\)0.01 \\ Logs2Graphs & 0.87\(\pm\)0.04 & 0.91\(\pm\)0.02 & 0.95\(\pm\)0.00 & 0.59\(\pm\)0.00 & **0.96\(\pm\)**0.01 & **0.93\(\pm\)**0.01 & **1.0\(\pm\)**0.00 & **1.0\(\pm\)**0.00 & **0.99\(\pm\)**0.00 & **1.0\(\pm\)**0.00 \\ \hline \hline \end{tabular}
\end{table}
Table 3. Anomaly detection accuracy on five benchmark datasets for _Logs2Graphs_ and its eight competitors. AP and RC denote Average Precision and AUC ROC, respectively. HDFS, BGL, and Thunderbird have been downsampled to 10,000 graphs each while maintaining the original anomaly rates. For each method on each dataset, to mitigate potential biases arising from randomness, we conducted ten experimental runs with varying random seeds and report the average values along with standard deviations of AP and RC. Moreover, we highlight the best results with bold and the runner-up with underline.
computational burden increases substantially. We thus recommend and use \(L=1\).

**The Embedding Dimension \(d\):** From Table 7 (middle row), one can see that \(d=128\) yields good performance on Spirit, Hadoop, HDFS and Thunderbird, while further increasing \(d\) obtains negligible performance improvement or even degradation. However, an increase of \(d\) on BGL leads to significantly better performance. One possible reason is that BGL is a complex dataset wherein anomalies and normal instances are not easily separable on lower dimensions.

**The Proximity Parameter \(k\):** As this parameter increases, a one can gain more information from its further neighbours. Figure 7 (bottom row) contrasts the detection performance when \(k\) is set to \(1\) and \(2\), respectively. Particularly, we construct one Inception Block when \(k=2\), using concatenation to fuse the results.

We observe that there is no significant improvement in performance when using a value of \(k=2\) in comparison to \(k=1\). It is important to recognize that a node exhibits 0th-order proximity with itself and 1st-order proximity with its immediately connected neighbors. If \(k=2\), a node can directly aggregate information from its \(2\)-order neighbours. As described in Table 1, graphs generated from logs usually contain a limited number of nodes, varying to \(6\) to \(34\). Therefore, this is no need to utilise the Inception Block, which was originally designed to handle large graphs in (Zhu et al., 2018).

### Runtime Analysis

Note that traditional machine learning methods, including PCA, OCSVM, IForest and HBOS, usually perform log anomaly detection in a transductive way. In other words, they require the complete dataset beforehand and do not follow a train-and-test strategy. In contrast, neural network based methods, such as DeepLog, LogAnomaly, AutoEncoder, and Logs2Graphs, perform log anomaly detection in an inductive manner, namely following a train-and-test strategy.

Figure 8 shows that most computational time demanded by _Logs2Graphs_ is allocated towards the graph generation phase. In contrast, the training and testing phases require a minimal time budget. The graph generation phase can be amenable to parallelisation though, thereby potentially reducing the overall processing time. As a result, _Logs2Graphs_ shows great promise in performing online log anomaly detection. Meanwhile, other neural networks based models--such as DeepLog, LogAnomaly, and AutoEncoder--demand considerably more time for the training and testing phases.

## 7. Threats to Validity

We have discerned several factors that may pose a threat to the validity of our findings.

**Limited Datasets.** Our experimental protocol entails utilizing five publicly available log datasets, which have been commonly employed in prior research on log-based anomaly detection. However, it is important to acknowledge that these datasets may not fully encapsulate the entirety of log data characteristics. To address this limitation, our future work will involve conducting experiments on additional datasets, particularly those derived from industrial settings, in order to encompass a broader range of real-world scenarios.

**Limited Competitors.** This study focuses solely on the experimental evaluation of eight competing models, which are considered representative and possess publicly accessible source code. However, it is worth noting that certain models such as GLAD-PAW did not disclose their source code and it requires non-trivial efforts

Figure 8. Runtime for all eight methods on all datasets, wherein HDFS, BGL, and Thunderbird have been downsampled to 10,000 graphs. Runetimes are averaged over 10 repetitions. We report the training time per epoch for neural network based methods.

Figure 7. The effects of the number of layers (top row), the embedding dimensions (middle row) and the proximity parameter (bottom row) on AP (left column) and AUC ROC (right column).
to re-implement these models. Moreover, certain models such as CODEtect require several months to conduct the experiments on our limited computing resources. For these reasons, we exclude them from our present evaluation. In subsequent endeavors, we intend to re-implement certain models and attain more computing resources to test more models.

**Purity of Training Data.** The purity of training data is usually hard to guarantee in practical scenarios. Although Logs2Graphs is shown to be robust to very small contamination in the training data, it is critical to improve the model robustness by using techniques such as adversarial training (Bahmani et al., 2010) in the future.

**Graph Construction.** The graph construction process, especially regarding the establishment of edges and assigning edge weights, adheres to a rule based on connecting consecutive log events. However, this rule may be considered overly simplistic in certain scenarios. Therefore, more advanced techniques will be explored to construct graphs in the future.

## 8. Conclusions

We introduced _Logs2Graphs_, a new approach for unsupervised log anomaly detection. It first converts log files to attributed, directed, and edge-weighted graphs, translating the problem to an instance of graph-level anomaly detection. Next, this problem is solved by OCDDiGCN, a novel method based on graph neural networks that performs graph representation learning and graph-level anomaly detection in an end-to-end manner. Important properties of OCDDiGCN include that it can deal with directed graphs and do unsupervised learning.

Extensive results on five benchmark datasets reveal that _Logs2Graphs_ is at least comparable to and often outperforms state-of-the-art log anomaly detection methods such as DeepLog and LogAnomaly. Furthermore, a comparison to a similar method for graph-level anomaly detection on _undirected_ graphs demonstrates that directed log graphs lead to better detection accuracy in practice.

## Acknowledgments

**Zhong Li and Matthijs van Leeuwen:** this publication is part of the project Digital Twin with project number P18-03 of the research programme TTV Perspective, which is (partly) financed by the Dutch Research Council (NWO). **Jiayang Shi:** This research is co-financed by the European Union H2020-MSCA-ITN-2020 under grant agreement no. 956172 (xCTing).
