# FeDiSa: A Semi-asynchronous Federated Learning Framework for Power System
Fault and Cyberattack Discrimination
††thanks: This work is the result of the research project funded by the
Faculty of Science, Engineering and Built Environment (SEBE), Deakin
University under the scheme ’Mini ARC Analogue Program (MAAP)’.

M. A. Husnoo†, A. Anwar†, H. T. Reda†, N. Hosseinzadeh‡, S. N. Islam‡, A. N.
Mahmood∗and R. Doss†

†Centre for Cyber Security Research and Innovation, School of IT, Deakin
University, Geelong, Australia ‡Centre for Smart Power and Energy Research,
School of Engineering, Deakin University, Geelong, Australia ∗Department of
Computer Science & IT, Latrobe University, Melbourne, Australia Email:
{mahusnoo, adnan.anwar, haftu.reda, nasser.hosseinzadeh, shama.i,
<EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

With growing security and privacy concerns in the Smart Grid domain, intrusion
detection on critical energy infrastructure has become a high priority in
recent years. To remedy the challenges of privacy preservation and
decentralized power zones with strategic data owners, Federated Learning (FL)
has contemporarily surfaced as a viable privacy-preserving alternative which
enables collaborative training of attack detection models without requiring
the sharing of raw data. To address some of the technical challenges
associated with conventional synchronous FL, this paper proposes FeDiSa, a
novel Semi-asynchronous Federated learning framework for power system faults
and cyberattack Discrimination which takes into account communication latency
and stragglers. Specifically, we propose a collaborative training of deep
auto-encoder by Supervisory Control and Data Acquisition sub-systems which
upload their local model updates to a control centre, which then perform a
semi-asynchronous model aggregation for a new global model parameters based on
a buffer system and a preset cut-off time. Experiments on the proposed
framework using publicly available industrial control systems datasets reveal
superior attack detection accuracy whilst preserving data confidentiality and
minimizing the adverse effects of communication latency and stragglers.
Furthermore, we see a 35% improvement in training time, thus validating the
robustness of our proposed method.

###### Index Terms:

Cyberattack, Asynchronous Federated Learning, Anomaly Detection, Internet of
Things (IoT), Smart Grid

##  I Introduction

Figure 1: An overview of the FL-based intrusion detection system model. Each
edge-based IED in a respective power grid zone uploads its data securely to a
SCADA sub-system. Once the FL training is initiated, the control centre
initializes a global model and broadcasts it to all SCADA sub-systems. Using
their local datasets, the SCADA sub-systems will update the received models
and send it back to the control centre. Once a preset cut-off time is reached,
the control centre will start the model parameters aggregation. Any model
updates received after the cut-off time will be updated into a buffer to be
used for the next communication round.

The widespread proliferation of Internet of Things (IoT) devices and
groundbreaking advances in communication technology within the Smart Grid (SG)
paradigm have fuelled an unprecedented increase in data generation by edge
devices; however, the evolution of cyberattack crafting methods, coupled with
a lack of proper attack detection strategies, has rendered the SG ecosystem an
easy prey for cyberattacks launched by malicious adversaries seeking illicit
financial and/or political gains [1]. Therefore, it is peremptory to develop
effective countermeasures to protect critical assets of SGs from cyberthreats.

###  I-A Motivation

In recent years, numerous literature [2, 3, 4] have proposed novel defence
strategies against cyberattacks in SGs by leveraging state-of-the-art machine
learning and deep learning solutions. However, such centralized cyberattack
defence solutions However, such centralized cyberattack defence solutions come
with limited storage capabilities, communication bottlenecks, and, most
notably, privacy issues. Previous studies inherently assumed attack-free
transmission of sensitive power related information which subsequently fail to
provide confidentiality guarantees in real-world applications. Consequently,
to address the aforementioned challenges, Federated Learning (FL)-based
cyberattack countermeasures [5, 6, 7, 8, 9, 10] emerged as viable privacy-
preserving promising solutions which exploit the concept of distributed
learning by restricting data sharing and enabling collaborative on-device
training of models at the edge. For instance, the authors in [10] put forward
a transformer-based federated false data injection attack detection mechanism
and claimed superior and effective attack mitigation.

Hitherto, prior FL-based cyberattack countermeasures make use of classic
synchronous aggregation protocols (including FedAvg [11] and its extensions)
whereby at each communication round, the central orchestrator broadcasts the
model to the clients, waits for updates from all clients participating in the
training and, eventually aggregates the local updates for the subsequent round
until convergence. However, due to resource constraints of several Intelligent
Electronic Devices (IEDs) within the real-world scenario, studies [12]
revealed few short-comings of existing FL-based cyberattack detection
methodologies: 1) Stragglers: Current state-of-the-art assume delay-free
communication between the client nodes and the central orchestrator. In
reality, wireless Supervisory Control and Data Acquisition (SCADA)
communication transmission protocols (such as the IEC 61850) may experience
delays due to unforeseen circumstances and/or unexpected dropout of
participating client nodes. 2) Communication inefficiency: Due to stragglers,
the central orchestrator is required to wait for local model updates from all
participating clients nodes prior to aggregation. Consequently, faster nodes
are penalized due to global learning suspensions and timeouts. 3) Resource
wastage: As a result of node selection within large set-ups, multiple
competent client nodes are more likely to remain idle and are unable to
participate in the training process. To overcome these mentioned issues,
asynchronous FL [13, 14, 15] was proposed as an alternative to the classical
synchronous FL method whereby the central orchestrator initiates the
aggregation procedure without waiting for straggler updates. Nonetheless,
asynchronous FL methodologies presume physical homogeneity of data on client
nodes which is impractical within the SG paradigm due to perpetual data
sensing by IEDs.

###  I-B Contribution and Paper Outline

In this paper, we put forward the suitability of a semi-asynchronous federated
cyberattack detection framework to discriminate between adversarial
cyberattacks and natural power system disturbances within distributed power
grid zone settings while considering straggler nodes. Specifically, the major
contributions of this paper are summarized as follows: 1) To relax physical
homogeneity and resource assumptions, we propose a light-weight semi-
asynchronous privacy-preserving on-device collaborative cyberattack detection
framework for power control systems, termed as FeDiSa. 2) We leverage the use
of a representation learning-based Deep Auto-Encoder model to improve the
accuracy of anomaly detection in power control systems. 3) Lastly, we
extensively evaluate our proposed framework on the Mississippi State
University and Oak Ridge National Laboratory Power System Attack (MSU-ORNL
PSA) Dataset [16] to validate that our proposed FeDiSaframework still achieves
high detection performance under physical heterogeneity and resource
constraints. We compare our proposed framework against other state-of-the-art
models and observe that our proposed solution achieves a superior detection
rate (accuracy of 92.4%) with a 35% decrease in training time required in the
presence of stragglers, which concludes the effectiveness and efficiency of
our FeDiSa framework.

The remainder of this paper is structured as follows: Section II briefly
discusses the system model and the main problem faced by synchronous FL-based
cyberattack detection methods. The proposed FeDiSa framework is detailed in
Section III. Section IV presents simulation scenarios for validating the
efficacy of our proposed approach on publicly available datasets. Finally,
Section V concludes the paper.

##  II System Model and Problem Formulation

Figure 2: A schematic illustration of our attack detection module architecture
with an example of a data point passing through the anomaly detection process.
The detection module consists of three components: 1) Encoder: The encoder
compresses the input into a lower dimensional latent space representation. 2)
Latent Distribution: By imposing a bottleneck in the network, we enforce a
compressed knowledge representation of the original input such that sample
latent vectors can be fed to the decoder. 3) Decoder: The decoder outputs a
reconstruction of the original input which is then compared against the
original input.

Throughout this paper, we consider a distributed power grid system which is
partitioned into $N$ number of zones where $N\in\mathbb{R}^{+}$. As shown in
Figure 1, our proposed system model consists of three major actors: 1) Control
Centre: The control centre typically coordinates and monitors grid operations.
We assume that the control centre has sufficient computational resources and
acts as the central orchestrator of our federated intrusion detection
framework. 2) SCADA Sub-systems: Equipped with advanced data acquisition
capabilities, SCADA sub-systems are responsible for the collection and
monitoring of power data sensed by IEDs. The SCADA sub-systems will
collaboratively train the federated models with their local data and update
their local model parameters to the control centre. 3) Power Grid Zones: Power
grid zones consists with several IEDs and sensor networks which continually
sense power-related data such as voltage phase angle, current, etc. We assume
that IEDs are connected to the SCADA sub-systems via high speed communication
networks for continuous transmission of sensed data.

The canonical goal of our proposed FeDiSa framework is to solve the objective
function $f_{k}$ for a shared global model parameter $Z\in\mathbb{R}^{M}$ such
that
$f_{k}(Z)=\dfrac{1}{N_{k}}\displaystyle\sum_{i=1}^{N_{k}}\ell\left(\\{a_{k}^{i},b_{k}^{i}\\};Z\right)$,
where $\ell(.):\mathbb{R}^{M}\times\mathbb{R}\rightarrow\mathbb{R}$ is the
loss to be optimized for the data point $\left(a_{k}^{i},b_{k}^{i}\right)$.
Furthermore, the global objective function $F$ over all the distributed $K$
datasets is defined as
$F(Z)=\displaystyle\sum_{k=1}^{K}\dfrac{N_{k}}{N}f_{k}(Z)$ which is the sum of
the local loss functions for each $k$, where $N$ is the total data points for
training. FedAvg and its improvements [17] are the most common FL optimization
algorithms currently in use for aggregation of local weights from all
federated client nodes through multiple iterations in view of achieving model
convergence.

However, as opposed to previous works on FL-based cyberattack detection [6, 5,
10] which assume delay-free and error-free communication between the central
FL orchestrator and the client nodes, within a real-world scenario, wireless
communication protocols used for data transmission are subject to high latency
which may be due to unanticipated occurrences such as weather conditions,
geographical locations, high magnetic fields, etc. Indeed, for cyberattack
detection within distributed grid systems, communication latency is a pressing
issue that has been understudied in the context of federated learning.
Additionally, active client node dropout due to power or connectivity
constraints is not uncommon. The limited availability of studies that take
into consideration the effect of stragglers motivates the pressing need to
develop a robust and fault-tolerant FL-based framework for power system
disturbances and cyberattack discrimination which is addressed by our
manuscript.

##  III Proposed Method

In this section, we first present the attack detection module considered for
power system disturbances and cyberattack discrimination followed by the
proposed robust asynchronous FL-based intrusion detection framework.

###  III-A Cyberattack Detection Module

We propose to address the challenge of cyberattack detection in power network
systems by leveraging the use of a Deep Auto-encoder (DAE) neural network, as
schematically illustrated in Figure 2. DAEs are unsupervised representation
learning-based models with the objective of achieving identity mapping between
inputs and outputs [18]. Auto-encoders learn important features by parsing the
input vector, $x_{i}$, to an encoder, $f_{\alpha}$, which compresses $x_{j}$
to a latent space representation, $y^{(j)}$, such that
$y^{(j)}=f_{\alpha}(x_{j})=a(Zx_{j}+c)$, where $a$ is an element-wise
activation function, $Z$ is the weight matrix and $c$ is the bias vector of
the encoder. The sample latent vector, $y_{j}$ is then parsed to the decoder,
$g_{\alpha^{\prime}}$, which performs a reconstruction of the original input,
$x_{j}$, through
$x_{j}=g_{\alpha^{\prime}}(y_{i})=a(Z^{\prime}y_{j}+c^{\prime})$ while
curtailing the reconstruction error. The parameters of the model $Z$,
$Z^{\prime}$, $c$ and $c^{\prime}$ are iteratively updated during the
federated training process such that $\alpha$,
$\alpha^{\prime}=arg\displaystyle{\min_{\alpha,\alpha^{\prime}}}\ell(x_{j},y_{j})$
where $\ell(x_{j},y_{j})$ is the reconstruction error function to be
optimized.

Our DAE network architecture comprises of two symmetrical deep belief networks
with five shallow layers to represent both the model’s encoder and decoder.
Restricted Boltzmann Machines (RBMs) are used as the building blocks of the
DAE layers and are trained using Contrastive Divergence Algorithm. Rectified
Linear Unit (ReLU) is used as the activation function while Mean Squared Error
(MSE) is chosen as the metric for measurement of reconstruction error/anomaly
degree. The anomaly threshold $\tau$ is determined by sorting the
reconstruction errors for the training set in ascending order and choosing the
optimal value $\tau$ at the inflection point of the error distribution.
Therefore, if the reconstruction error for a certain data point exceeds
$\tau$, existence of an attack event is validated. Additionally, we add soft-
max layers after the RBM stack for classification. The batch size and learning
rate for training are configured to 100 and 10-3 respectively. The model
serves as the basis for cyberattack detection in the our proposed semi-
asynchronous FL framework.

###  III-B Semi-asynchronous FL Framework

As depicted in Figure 1, throughout this manuscript, we consider a federated
power system set-up whereby $K$ distributed SCADA sub-systems are linked to
the control centre via communication protocols (e.g. IEC 61850, etc.). In our
proposed framework, we take into consideration that the transmission protocol
between the SCADA sub-systems and the control centre id not delay-free and
error-free. Given the heterogeneous natural circumstances of the edge-based
SCADA sub-systems, it is impossible to ensure stable communication between the
control centre and the SCADA sub-systems. In such context, asynchronous FL is
a viable alternative to overcome stragglers and the training time consumed by
synchronous FL during the wait for all local model updates. However, such
approach exacerbates the staleness of the local updates. Therefore, to remedy
this problem, we propose a semi-asynchronous framework that relies upon a
preset cut-off time for model updates aggregation.

Input: learning rate $\eta$, one-bit quantizer $sign(.)$, $K$ SCADA sub-
systems, Local training set $D_{k}$ for $k\in[1,K]$ Initialize common
unanimous model parameter $Z_{0}$, model parameter buffer $B$, aggregation
time $T_{a}$, SCADA sub-systems time cost [$s_{0},s_{1},...,s_{k}$].

for  _each communication round $t$ in $T_{cl}\in(1,n)$_ do

Control Centre:

Obtain gradient ($g^{t}_{k}$, $t_{k}$) from each SCADA sub- system $k$.
$s_{k}\leftarrow$ update $t_{k}$. $B\xleftarrow{+}$ ($g^{t}_{k}$, $t_{k}$) if
_Aggregation time $T_{a}$ is reached_ then

$B^{\prime}\leftarrow$ cluster by model version $t$ and compute aggregation.
for  _each ( $g^{t}_{k}$, $t_{k}$) in $B^{\prime}$_ do

Update new global $g$ and $t$. end

Update newly aggregated global parameters ($g$, $t$) to source SCADA sub-
systems $k$. Clear $B$. end

SCADA Sub-System:

for  _each $k\in[K]$_ do

Compute local gradient $g^{t}_{k}=\nabla f_{k}(Z^{t})$ using $D_{k}$. Compute
time cost of each client $t_{k}$ Update ($g^{t}_{k}$, $t_{k}$) to control
centre. end

end Output: Global Model

Algorithm 1 Proposed FeDiSa Framework

During a particular $t$th federated training round, our proposed FeDiSa
framework, as presented in Algorithm 1 consists of the following
implementation phases: 1) The control centre distributes the global DAE model
parameters $Z^{t}$ to all $K$ SCADA-subsystems. 2) Each SCADA sub-system $k$
for $k\in$ [$k$] then computes the local model gradient $g^{t}_{k}=\nabla
f_{k}(Z^{t})$ utilizing its local training set. 3) Each $k$ SCADA sub-system
updates its locally computed gradient $g^{t}_{k}$ to the control centre. 4)
The control centre updates the model parameter buffer and the time cost for
each $k$ SCADA sub-system. 5) Once the preset cut-off time $T_{a}$ is reached,
the control centre clusters the model updates by the time version and
aggregates the updates of each group using the method proposed in [19]. 6)
Lastly, the control centre then updates the new model parameters to each
SCADA-subsystem $k\in[K]$.

##  IV Experimental Results and Discussions

In this section, we first discuss the dataset used for the experimental set-up
followed by the achieved results that validate the attack detection
performance and the robustness of our proposed FeDiSa framework detailed in
Section III.

###  IV-A Dataset Description and Preparation

We use the publicly available industrial control system MSU-ORNL PSA dataset
[16] to experimentally evaluate the effectiveness of our proposed FeDiSa
framework. The datasets are a binary class comprised of 15 distinct data
files, each with a different power system event scenario. The event scenarios
include natural events (power system disturbances) and attack events
(cyberattacks). The natural events include faults of varying level across the
distributed grids, namely 10-19%, 20-79%, and 80-90%. Similarly, the attack
events are as follows: 1) A remote tripping command injection (i.e., an attack
vector that involves sending a command to an IED which renders a circuit
breaker to open/close). 2) A change in IED settings (i.e., an attack vector
targeting the IEDs to change their setting to disable their normal operation,
for example, to falsely trip a relay). 3) Then, a false data injection attack
(i.e., an attack vector that intelligently emulates a valid physical fault by
modifying power system values across voltage, current, and other physical
parameters to blind system operators and render consequential impacts). The
datasets have been simulated and recorded based on 128 features (such as
frequency, current phase angle, current phase magnitude, etc.).

A basis of 100 features have been chosen for training our DAE-based attack
detection model after applying Principal Component Analysis (PCA) to cull out
the unnecessary features and reduce training complexity. To deal with missing
values within the dataset, we use K-Nearest Neighbour (KNN) imputation. The
input features are normalized in the range of (0,1). Lastly, we split the
dataset into a train-test split ratio of 70% and 30% respectively.

TABLE I: Comparison of the average attack detection performance of our proposed approach against state-of-the-art models.  | Federated Models
---|---
Metric (%) | Prop. Approach | CNN | LSTM | RBM | RNN
Accuracy | 92.4 | 89.6 | 78.5 | 73.3 | 69.3
Precision | 90.5 | 87.7 | 79.1 | 72.5 | 68.8
Recall | 87.9 | 85.9 | 78.3 | 71.7 | 69.5
F1-Measure | 88.4 | 87.6 | 79.2 | 71.3 | 69.7

###  IV-B Attack Detection Performance

Figure 3: Comparison of the average attack detection performance of our
proposed framework against four competing deep learning models based on
percentage accuracy, precision, recall and F-Score metrics.

We validated the attack detection performance of our proposed semi-
asynchronous FL-based cyberattack detection framework against four other
state-of-the-art models used for intrusion detection in SGs namely
Convolutional Neural Network (CNN), Long Short-Term Memory (LSTM), Recurrent
Neural Network (RNN) and Restricted Boltzmann Machine (RBM). The rivalling
models are all trained in a similar semi-asynchronous FL-based configuration.
After training, the global models are evaluated on the test set. From Figure 3
and Table I, we evaluate the average attack detection performance of our
proposed approach using four different classification metrics namely accuracy,
precision, recall and F1-Measure. The experimental validations reveals that
our proposed FeDiSa framework achieves superior average cyberattack detection
accuracy over other state-of-the-art deep learning models. For instance, our
proposed approach achieved an accuracy rate of 92.4% in comparison with CNN
(89.6%), LSTM (78.5%), RBM (73.3%) and RNN (69.3%). Similarly, in terms of
precision rate, our proposed approach achieved 90.5% as opposed to CNN
(87.7%), LSTM (79.1%), RBM (72.5%) and RNN (68.8%). In addition, we observe
that RNN produces the lowest detection performance overall. Next, we evaluate
the cyberattack detection rate of our proposed framework using the fifteen
different data files within the MSU-ORNL PSA dataset. From the experimental
results in Figure 4, we observe that our proposed framework outperforms other
neural networks over all 15 data files. For example, the accuracy of FeDiSa on
data1 is 92.6% as compared to CNN (88.9%) or LSTM (78.4%). Similarly, for
data15, we achieve improved accuracy detection rate of 94.9% as opposed to RNN
(68.9%) and RBM (73.9%). Therefore, we conclude that our proposed semi-
asynchronous FL-based power system disturbance and cyberattack discrimination
framework achieves sufficient attack detection accuracy as opposed to other
deep learning models whilst guaranteeing data confidentiality and minimizing
the effect of communication delays and stragglers.

Figure 4: Comparison of the average attack detection accuracy of our proposed
approach against four other neural networks (trained in similar fashion) over
different datasets. Figure 5: Comparison of the robustness of our proposed
semi-asynchronous approach against two classical synchronous FL aggregation
methods based on accuracy with increasing affected nodes.

###  IV-C Robustness against Stragglers

Taking into account the effect of communication latency and straggler client
nodes in a real-world Smart Grid scenario, we propose a semi-asynchronous FL-
based approach to train a global cyberattack detection model. We set the cut-
off aggregation time to twice the average waiting time of nodes in a federated
set-up. To simulate the effect of stragglers and communication latency, we
initially set $K$ to 10 and set a pause mechanism for a set portion of SCADA
sub-systems such that they will be non-responsive for that period of time. We
compare the performance of our proposed approach against that of two classical
synchronous FL aggregation solutions, FedAvg and Federated Stochastic Gradient
Descent (FedSGD). From Figure 5, we observe that the accuracy rate of
cyberattack detection of our proposed approach stays relatively consistant
with increasing number of affected nodes. On the other hand, the synchronous
FL aggregation protocols oversee a significant decline in their performance
with increasing number of impacted SCADA sub-systems. This signifies that our
semi-asynchronous attack detection solution is robust to communication lags
and straggler SCADA sub-systems.

Figure 6: Comparison of training time of our proposed approach against FedAvg
and FedSGD on varying number of impacted SCADA sub-systems.

Furthermore, we evaluate the training time taken to achieve model convergence
within a classical Federated Learning environment using FedAvg and FedSGD in
different scenarios of affected nodes. As shown in Figure 6, we note that the
training time of our proposed method in the presence of stragglers and
communication delays is reduced as compared to the synchronous FedAvg and
FedSGD aggregation algorithms. Specifically, it is highlighted that the
training time for our proposed FeDiSa solution is approximately 35% less that
the rivalling aggregation methods. However, we also notice that there is an
increase in the training time of our proposed solution with increasing number
of impacted SCADA sub-systems. This is be due to the increasing number of
training rounds required to achieve model convergence with an increase in
affected SCADA sub-systems nodes.

##  V Conclusion

This paper proposes a novel semi-asynchronous FL-based power system detection
and cyberattack detection strategy to mitigate the effects of communication
unreliability and stragglers in real-world SG applications on decentralized
power grid systems. Furthermore, a representation learning-based DAE model is
leveraged for timely and accurate detection of cyberthreats. Experimental
validations performed on a publicly available dataset (the MSU-ORNL PSA
dataset) revealed that the proposed solution outperforms other deep learning
models in terms of detection accuracy. while being computationally efficient
and achieving a 35% reduction in training time compared to state-of-the-art
synchronous solutions, thus proving the effectiveness and efficiency of the
proposed FeDiSa. For future works, we will explore security aspects of FL-
based detection frameworks, which are highly vulnerable to byzantine threats
as per our previous work [20].

## References

  * [1] M. A. Husnoo, A. Anwar, N. Hosseinzadeh, S. N. Islam, A. N. Mahmood, and R. Doss, “False data injection threats in active distribution systems: A comprehensive survey,” _Future Generation Computer Systems_ , vol. 140, p. 344–364, Mar 2023.
  * [2] A. Anwar, A. Mahmood, B. Ray, M. A. Mahmud, and Z. Tari, “Machine learning to ensure data integrity in power system topological network database,” _Electronics_ , vol. 9, no. 4, p. 693, Apr 2020.
  * [3] B. M. R. Amin, A. Anwar, and M. J. Hossain, “Distinguishing between cyber injection and faults using machine learning algorithms,” in _2018 IEEE Region Ten Symposium (Tensymp)_ , 2018, pp. 19–24.
  * [4] A. Anwar, A. N. Mahmood, and Z. Shah, “A data-driven approach to distinguish cyber-attacks from physical faults in a smart grid,” in _Proceedings of the 24th ACM International on Conference on Information and Knowledge Management_ , ser. CIKM ’15. New York, NY, USA: Association for Computing Machinery, 2015, p. 1811–1814.
  * [5] W.-T. Lin, G. Chen, and Y. Huang, “Incentive edge-based federated learning for false data injection attack detection on power grid state estimation: A novel mechanism design approach,” _Applied Energy_ , vol. 314, p. 118828, May 2022\.
  * [6] M. Wen, R. Xie, K. Lu, L. Wang, and K. Zhang, “Feddetect: A novel privacy-preserving federated learning framework for energy theft detection in smart grid,” _IEEE Internet of Things Journal_ , vol. 9, no. 8, pp. 6069–6080, 2022.
  * [7] Y. Xu, M. Z. A. Bhuiyan, T. Wang, X. Zhou, and A. K. Singh, “C-fdrl: Context-aware privacy-preserving offloading through federated deep reinforcement learning in cloud-enabled iot,” _IEEE Transactions on Industrial Informatics_ , vol. 19, no. 2, pp. 1155–1164, 2023.
  * [8] X. Xu, H. Peng, M. Z. A. Bhuiyan, Z. Hao, L. Liu, L. Sun, and L. He, “Privacy-preserving federated depression detection from multisource mobile health data,” _IEEE Transactions on Industrial Informatics_ , vol. 18, no. 7, pp. 4788–4797, 2022.
  * [9] X. Xu, W. Liu, Y. Zhang, X. Zhang, W. Dou, L. Qi, and M. Z. A. Bhuiyan, “Psdf: Privacy-aware iov service deployment with federated learning in cloud-edge computing,” _ACM Trans. Intell. Syst. Technol._ , vol. 13, no. 5, oct 2022\. [Online]. Available: https://doi.org/10.1145/3501810
  * [10] Y. Li, X. Wei, Y. Li, Z. Dong, and M. Shahidehpour, “Detection of false data injection attacks in smart grid: A secure federated deep learning approach,” _IEEE Transactions on Smart Grid_ , vol. 13, no. 6, pp. 4862–4872, 2022.
  * [11] M. A. Husnoo, A. Anwar, N. Hosseinzadeh, S. N. Islam, A. N. Mahmood, and R. Doss, “Fedrep: Towards horizontal federated load forecasting for retail energy providers,” 2022. [Online]. Available: https://arxiv.org/abs/2203.00219
  * [12] W. Wu, L. He, W. Lin, R. Mao, C. Maple, and S. Jarvis, “Safa: A semi-asynchronous protocol for fast federated learning with low overhead,” _IEEE Transactions on Computers_ , vol. 70, no. 5, pp. 655–668, 2021.
  * [13] L. Cui, Y. Qu, G. Xie, D. Zeng, R. Li, S. Shen, and S. Yu, “Security and privacy-enhanced federated learning for anomaly detection in iot infrastructures,” _IEEE Transactions on Industrial Informatics_ , vol. 18, no. 5, pp. 3492–3500, 2022.
  * [14] Y. Lu, X. Huang, Y. Dai, S. Maharjan, and Y. Zhang, “Differentially private asynchronous federated learning for mobile edge computing in urban informatics,” _IEEE Transactions on Industrial Informatics_ , vol. 16, no. 3, pp. 2134–2143, 2020.
  * [15] J. Lin, J. Ma, and J. Zhu, “Privacy-preserving household characteristic identification with federated learning method,” _IEEE Transactions on Smart Grid_ , vol. 13, no. 2, pp. 1088–1099, 2022.
  * [16] U. Adhikari, S. Pan, T. Morris, R. Borges, and J. Beaver. [Online]. Available: https://sites.google.com/a/uah.edu/tommy-morris-uah/ics-data-sets
  * [17] A. Nilsson, S. Smith, G. Ulm, E. Gustavsson, and M. Jirstrand, “A performance evaluation of federated learning algorithms,” in _Proceedings of the Second Workshop on Distributed Infrastructures for Deep Learning_ , ser. DIDL ’18. New York, NY, USA: Association for Computing Machinery, 2018, p. 1–8. [Online]. Available: https://doi.org/10.1145/3286490.3286559
  * [18] C. Zhou and R. C. Paffenroth, “Anomaly detection with robust deep autoencoders,” ser. KDD ’17. New York, NY, USA: Association for Computing Machinery, 2017, p. 665–674.
  * [19] C. Xie, S. Koyejo, and I. Gupta, “Asynchronous federated optimization,” 2019. [Online]. Available: https://arxiv.org/abs/1903.03934
  * [20] M. A. Husnoo, A. Anwar, N. Hosseinzadeh, S. N. Islam, A. N. Mahmood, and R. Doss, “A secure federated learning framework for residential short term load forecasting,” 2022. [Online]. Available: https://arxiv.org/abs/2209.14547


# An Extensive Study of User Identification via Eye Movements across Multiple
Datasets

Sahar Mahdie Klim Al Zaidawi<EMAIL_ADDRESS>Database Lab,
University of Bremen , Germany ,  Martin H.U. Prinzler
<EMAIL_ADDRESS>Database Lab,
University of Bremen , Germany ,  Jonas Lührs<EMAIL_ADDRESS>Database
Lab,
University of Bremen , Germany and  Sebastian Maneth<EMAIL_ADDRESS>Database Lab,
University of Bremen , Germany

###### Abstract.

Several studies have reported that biometric identification based on eye
movement characteristics can be used for authentication. This paper provides
an extensive study of user identification via eye movements across multiple
datasets based on an improved version of method originally proposed by George
and Routray. We analyzed our method with respect to several factors that
affect the identification accuracy, such as the type of stimulus, the IVT
parameters (used for segmenting the trajectories into fixation and saccades),
adding new features such as higher-order derivatives of eye movements, the
inclusion of blink information, template aging, age and gender. We find that
three methods namely selecting optimal IVT parameters, adding higher-order
derivatives features and including an additional blink classifier have a
positive impact on the identification accuracy. The improvements range from a
few percentage points, up to an impressive 9 % increase on one of the
datasets.

Eye Tracking; Eye Movement Biometrics; User Identification; Machine Learning;
IVT algorithm

##  1\. Introduction

Eye movements can comprise rich and sensitive information about an individual,
including biometric identity, gender, age, ethnicity, personality traits, drug
consumption habits, moods and emotions, skills, preferences, cognitive
processes, and physical and mental health conditions (Kröger et al., 2019).
Eye movements are an excellent predictor of human desires and focus. They are
inextricably linked with human cognitive and perceptual processes.

Eye tracking is a method to record a persons’s eye movements. Today, a vast
number of different eye tracking devices are available and have been used by
researchers to produce many high-quality datasets. These datasets are analyzed
in different contexts; we only mention a few examples: disorder detection
(Armstrong and Olatunji, 2012; Benfatto et al., 2016; Billeci et al., 2017),
gaming (Lin et al., 2004; Alkan and Cagiltay, 2007; Lankes and Stoeckl, 2020),
gender prediction (Sargezeh et al., 2019; Moss et al., 2012; Zaidawi et al.,
2020), and user identification (Kasprowski and Ober, 2004; George and Routray,
2016; Rigas and Komogortsev, 2017; Schröder et al., 2020)

Let us consider the state-of-the-art approaches to user identification via eye
movement data. In the 2015 BioEye competition (Rigas and Komogortsev, 2017)
different systems for user identification were competing. The winning system
is based on first segmenting the eye tracking trajectory into fixations and
saccades using the “Velocity-Threshold Identification” (IVT) method (see e.g.
(Salvucci and Goldberg, 2000), where this methods is analyzed and some prior
references are given). Their version of IVT uses two parameters which are set
to certain default values. It was not investigated, whether other settings of
these parameters would give higher identification accuracies.

Most of the previous studies of eye movement biometrics have tested their
approach using a few number of participants, utilizing either one or two
datasets e.g. (George and Routray, 2016; Jäger et al., 2019; Krishna et al.,
2019). Furthermore, since most prior studies train and assess their models on
datasets compiled over a short time span, the permanence of eye movements
remains unexamined. Our study fills these gaps by assessing these methods with
datasets containing a larger number of participants and some that were not
utilized for eye movement biometrics.

We use two datasets from the 2015 Bioeye competition (Rigas and Komogortsev,
2017). They consists of 153 participants which looked at two different
stimuli: random moving dots (RAN) and a poem (TEX). The third biometrics
dataset is about a visual searching task (VST) (Li et al., 2018) and comprises
58 participants. The final dataset is known as the gaze on faces (GOF) and
contains 378 participants. The GOF dataset was used to identify “scanning
strategies” that are different for men than for women (Coutrot et al., 2016).
The entire data utilized in our study provides a high number of participants
and a broad range of age groups and stimuli.

The main contribution of this paper is an extensive study of user
identification via eye movements across four different datasets. We
investigate the effect of stimuli, gender, and age on user identification.
Additionally, we propose the following improvements:

  * •

We optimize the IVT parameters which can increase the accuracy by 3 % for RAN,
by 2 % for TEX, by 5 % for GOF, and by 9 % for VST datasets.

  * •

Adding higher derivative features can increase the accuracy by 2 % for RAN, by
1 % for TEX and VST, and by 3 % for GOF datasets.

  * •

Adding blinking features can increase the accuracy by 1 % for RAN and VST, by
0.5 % for TEX datasets.

  * •

Combining the above three methods of improvement (IVT parameters, higher order
derivative features, blink classifier) can increase the accuracy by 4 % for
RAN, by 3 % for TEX, and by 9 % for VST datasets.

Related Work. There is vast and fast growing literature on eye tracking
biometrics. Some surveys that provide a good overview are (Esfahani, 2016;
Kröger et al., 2019; Rigas and Komogortsev, 2017; Galdi et al., 2016). When
comparing results, it is important to keep in mind that the fewer participants
there are, the higher the prediction accuracies will be; thus, it is hard to
compare results that have different numbers of participants. The winner of the
2015 BioEye competition (George and Routray, 2016) achieves an accuracy of 89
% (over a single run) over 153 participants by computing fixation and saccade
based features and using them with an RBFN based classifier; this was improved
in (Schröder et al., 2020) to an accuracy of 94.1 % from one run (92.6 % over
50 runs as shown in the current paper), basically by adding more features. A
different biometrics approach has been implemented in (Li et al., 2018) for
the VST dataset over 58 participants. They split their stimuli (images) into
different grids and consider the frequencies of gaze points per grid component
(using the Gabor wavelet transform). The highest accuracy in this work is
97.35 % (termed as rank-1 identification rate) using 10 runs (data was split
into 70 % train and 30 % test) with their Fixation Density Map (FDM) method.

##  2\. Proposed System

This section gives an overview of our user identification pipeline including
the used datasets, data pre-processing and segmentation methods, feature
extraction, Machine Learning (ML) classifier and accuracy metric used in this
study.

###  2.1. Datasets

Four datasets with different stimuli are used: Bioeye TEX, Bioeye RAN, Visual
Searching Task, and Gaze on Faces. See Table 1 for an overview.

Table 1. Overview of Datasets. Dataset | Participants | Age | Trajectory | Blink
---|---|---|---|---
M | F | T | Range | Length [s] | Info.
TEX | N.A. | N.A. | 153 | 18–46 | 60 | Y
RAN | N.A. | N.A. | 153 | 18–46 | 100 | Y
VST | 24 | 34 | 58 | 21–33 | 180 | Y
GOF | 193 | 185 | 378 | 20–72 | 60 | N

####  2.1.1. Bioeye (TEX/RAN)

Two datasets with different stimuli were used in the _BioEye 2015 competition_
(Rigas and Komogortsev, 2017) 111The data was provided by Oleg Komogortsev..
Each has two recordings per participant, which were separated by a pause of 30
minutes in between.

  * TEX

60 second recordings of reading a poem. 1(b) shows the gaze trajectory of a
sample participant from this dataset along with the reading stimulus.

  * RAN

100 second recordings of observing a randomly moving dot. 1(a) visualizes the
gaze trajectory of a sample participant.

Both datasets consists of eye movement data from a total of 153 participants
including males and females from the age group 18 to 46.

The participants were seated from the monitor screen at a distance of 550 mm.
The dimensions and resolution of the monitor were $474\times 297$ mm and
$1680\times 1050$ pixels respectively. The participant’s heads were supported
with a chin rest to ensure stability during the sessions to avoid potential
eye-tracking artifacts that stem from notable head movements. The device used
for recording was an EyeLink-1000 eye-tracker (1000 Hz) but the data was down
sampled to 250 Hz using an anti-aliasing FIR filter which interpolated between
the invalid gaze points (Rigas and Komogortsev, 2017). Despite this
interpolation, the dataset still provides the explicit information about the
validity of the samples in the recording which can be attributed to device
specific faults or user specific reasons (e.g. blinking, loss of attention
etc.).

####  2.1.2. Visual Searching Task (VST)

This dataset (Li et al., 2018) includes the recording of gaze trajectories of
a total of 58 participants (24 males, 34 females) aged 21–33. The participants
performed a visual search task which is a series of number search questions
carried out with pictures. The participants were asked to compare the target
number with the comparison numbers in a form to find the longest matched
number. 1(c) shows an example of this stimulus and the gaze trajectory of one
of the participants. The recording duration is at least 4 minutes for each
participant in each session. The data collection experiment was divided into
two trials with at least two weeks between. In each trial there were 160
questions divided into four sessions separated by two minutes, each consisting
of 40 questions. Participants took a two minutes rest between these tests.
Overall, $160\times 2=320$ gaze trajectories for each participant were
collected.

The participants were seated from the monitor screen at a distance of 600 mm.
The dimensions and resolution of the monitor were $474\times 297$ mm and
$1920\times 1080$ pixels respectively. The device used for recording was an
Tobii TX300 eye tracking system running at 300 Hz. In contrast to the Bioeye
dataset, no explicit information about the validity of the gaze trajectory was
provided. Nevertheless, the gaze trajectory consisted of NaNs which we
consider as invalid and hence a source for blink information. We interpolated
across the invalid segments in order to have a connected gaze trajectory.

(a) RAN

(b) TEX

(c) VST

(d) GOF

Figure 1. Various stimuli and their gaze trajectories. (a) A participant
looking at random dots from the RAN dataset (b) A participant reading a text
from the TEX dataset, (c) A participant searching for number, (d) A
participant gazing on face image.

####  2.1.3. Gaze on Faces (GOF)

This dataset is provided in the study of (Coutrot et al., 2016) and it
involves the recording gaze trajectories of participants while they observe
faces. The participants looked at multiple images of a single actor gazing
towards them for varying durations ($0.1\,s$ to $10.3\,s$) in 32 different
trials. Overall, 8 actors were cast for the videos comprising four females and
four males. Originally the dataset consisted of a total number of 405
candidates from 58 countries, varying in age from 18 to 72. However,
participants that were found with erratic, absent, or damaged data were
eliminated to avoid inaccuracies from the study. Hence, the research in this
paper was carried out using the remaining 378 participants of which 193 were
males and 185 were females, ranging from ages 20 to 72.

The participants were seated at a distance of 570 mm from a monitor
($1280\times 1024$ pixels). The width and height of the stimuli were
$429\times 720$ pixels. Eye-tracking data was collected using the EyeLink 1000
kit eye-tracker at 250 Hz. 1(d) shows the gaze trajectory of a participant
while looking at the face of an actress. The selected dataset is utilized as
it offers an opportunity to study a set of participants that contains a large
number of male and female participants with a broader age range. In this
dataset, no invalid parts in the gaze trajectories were found and hence no
blink information could be deduced.

###  2.2. Preprocessing and Segmentation

While VST and GOF datasets provide the recorded pixel coordinates directly,
RAN and TEX datasets provide viewing angles w.r.t. the x and y axes. We
convert them to screen pixel coordinates $(x_{screen},y_{screen})$ as follows:

$\\{x|y\\}_{screen}=\left(\frac{d\times\\{w|h\\}_{pix}}{\\{w|h\\}}\right)tan(\theta_{\\{x|y\\}})+\frac{\\{w|h\\}_{pix}}{2}.$

The distance from the screen and viewing angles in the $x$ and $y$ directions
are denoted by $d$, $\theta_{x}$, and $\theta_{y}$ respectively. $w_{pix}$ and
$h_{pix}$ indicate the screen’s resolution and $w$ and $h$ the physical size
(width and height). The inverse of the above equation is used to compute the
viewing angle coordinates from the pixel coordinates for VST and GOF datasets.

Noise may be present in raw gaze trajectory data which amplifies further in
the calculation of velocity, acceleration and other higher order time
derivatives of the gaze trajectory. We implemented a Savitzky-Golay (Savitzky
and Golay, 1964) filter to reduce the noise (see also (Schafer, 2011)). This
filter applies a symmetric polynomial over several points. In our work we used
the polynomial order of 6 and a frame size of 15 as used in the works of
(George and Routray, 2016; Schröder et al., 2020).

The Velocity Threshold (IVT) algorithm, utilized in numerous publications, see
e.g. (Holland and Komogortsev, 2011, 2012; Olsen and Matos, 2012), is used to
segment the filtered gaze trajectories into a sequence of fixations and
saccades. The IVT algorithm has been described in different ways in the
literature. In (Sen and Megaw, 1984; Salvucci and Goldberg, 2000; Andersson et
al., 2017), the IVT algorithm is implemented with only one parameter i.e.
velocity threshold (VT). This segmentation might produce very short fixations.
These very short fixations are often not meaningful because the brain requires
some time to register the visual input (Olsen, 2012). Therefore, many
researchers remove the short fixations by using a second parameter named
Minimum fixation duration (MFD). Hence, in various studies (Holland and
Komogortsev, 2012; Rakoczi et al., 2013; Kasneci et al., 2021), this algorithm
uses both the VT and MFD. We use the IVT algorithm as described in (George and
Routray, 2016). The algorithm defines as fixation all consecutive gaze points
resulting in eye rotation velocities below the VT, unless the fixation would
be shorter than the MFD. All other segments are identified as saccades.
Commonly used parameters in IVT algorithm include VT = 50 °/s and MFD = 100 ms
(George and Routray, 2016; Schröder et al., 2020).

The eye tracking data can have invalid data (NaNs) or outliers (e.g invalid
gaze point). This can be due to user-specific reasons such as blinking, loss
of attention (micro-sleeping) or eye tracker faults (e.g. solo missed gaze
points) (Rigas and Komogortsev, 2017). However, the majority of the outliers
are reasoned by blinks. Physiologically, the blinking behavior can encode some
information about the participants (Kröger et al., 2019). Actual blinking
rates vary by individual averaging around 10 blinks per minute and the
duration of a blink is on average between 100–400 ms according to the Harvard
Database of Useful Biological Numbers222https://bionumbers.hms.harvard.edu/
[accessed 11-August-2021]. Blinking behavior is arguably different in men and
women (Doughty, 2002) and it has been found that adults blink more often than
infants (Juan, 2006). These studies motivate us to extract the blink
information of the participants in addition to our fixations and saccades when
available (which is the case of RAN, TEX and VST datasets). We extracted the
blink segments from the explicitly labeled invalid data (in case of RAN & TEX)
and NaN segments (in case of VST) using a duration threshold between 80–500
ms, since duration of more than 500 ms are considered as micro-sleeping
(Schleicher et al., 2008; Wang et al., 2011) and less than 80 ms can be device
faults or other unknown reasons.

###  2.3. Feature Extraction

Feature extraction is a basic way to reduce the dimension of high-dimensional
data. For each fixation, saccade and blink (when available), various features
are extracted separately.

Let $X$ and $Y$ denote the sequences of gaze coordinates in each
fixation/saccade where $X={x_{1},x_{2},...,x_{N}}$ and
$Y={y_{1},y_{2},...,y_{N}}$, and $N$ is the number of points.

We compute a number of basic features, such as duration, path length, fix/sac
ratio (ratio of maximum fix/sac angular velocity to fix/sac duration), fix/sac
angle, amplitude, dispersion, distance with the centroid of previous fix/sac,
angle with the centroid of previous fix/sac and average velocity are computed.
All these features are used in (George and Routray, 2016) and the
corresponding formulae are given in that paper.

All the derivatives, such as velocity, acceleration, jerk, etc. are computed
using the “forward difference method” as detailed in Figure 2.

Since for $k\geq 1$, the $k$-th order derivative can only be computed for
segments (fixations or saccades) of at least $(k+1)$ points. We exclude,
whenever we use such higher-order derivatves, segments that are shorter than
$(k+1)$ points (by appropriately merging the neighboring segments, in
accordance to our IVT algorithm). We always remove saccades consisting of only
one or two points. As shown in Table 2, various statistical features namely
mean, median, max, standard deviation, skewness and kurtosis are computed for
different types of velocities and derivatives (in the following refereed as
M3S2K features).

To the best of our knowledge, previous approaches to eye movement biometrics
only include derivaties until order two, i.e., they never use derivatives
beyond that of acceleration. In this paper, we demonstrate that including
higher order derivatives beyond acceleration can indeed be beneficial for eye
movement biometrics. There are works within eye tracking reserach that include
up to fourth order derivatives (to predict saccade movements, see (Wang et
al., 2017)). In general, it is well known that humans’ predictive capabilities
in their perception-action loop can be capture by higher order derivatives of
the perception or action trajectories (Sargolzaei et al., 2016).

Lastly, seven features namely number of blinks, duration of each blink, total
of duration, mean of duration, minimum of duration, maximum of duration, and
variance of duration are computed from the blink segments as shown in Table 3.
We tried on blink duration with different sets of the M3S2K statistical
features, and these seven features gave the best results.

In all cases, the features are normalized using the Z-score
standardization/normalization method implemented by (Pedregosa et al., 2011)
(sklearn.preprocessing.StandardScaler). The method of calculation here is to
determine the mean and standard deviation for each feature. Next we subtract
the mean from each feature and divide the obtained value by its standard
deviation. This ensures that each feature’s values are in the similar range
(all features are centered around 0 and have variance in the same order) and
therefore contribute equally to the classification.

Table 2. User identification features.

|  | Fix./Sac.
---
Features
|  | Fix./Sac.
---
Features
1 | Duration | 16–21 | Angular velocity*
2 | Path length | 22–27 | Velocity X*
3 | Skew X | 28–33 | Velocity Y*
4 | Skew Y | 34–39 | Angular acceleration*
5 | Kurt X | 40–45 | Acceleration X*
6 | Kurt Y | 46–51 | Acceleration Y*
7 | STD of X | 52–57 | Angular jerk*
8 | STD of Y | 58–63 | Jerk X*
9 | Fix/Sac ratio | 64–69 | Jerk Y*
10 | Fix/Sac angle | 70–75 | Angular jounce*
11 | Amplitude | 76–81 | Jounce X*
12 | Dispersion | 82–87 | Jounce Y*
13 | Dist. with previous Fix/Sac | 88–93 | Angular crackle*
14 | Angle with previous Fix/Sac | 94–99 | Crackle X*
15 | Average velocity | 100–105 | Crackle Y*

*M3S2K-Statistical features:
*Mean, Median, Max, Std, Skewness, Kurtosis

Table 3. Blinks features. | Blinks Features |  | Blinks Features
---|---|---|---
1 | Duration | 5 | Minimum of the duration
2 | Number of blinks | 6 | Maximum of the duration
3 | Mean of the duration | 7 | Variance of the duration
4 | Total of the duration |  |

$\overbrace{\left(x_{1},y_{1}\right)}^{P_{1}}$, |  $\overbrace{\left(x_{2},y_{2}\right)}^{P_{2}}$, |  |  |  |  |  | … |  |  |  |  |  | ,$\overbrace{\left(x_{n},y_{n}\right)}^{P_{n}}$ |  | Time per sample: | $t$ | $=$ | $\frac{1}{\textrm{sample rate}}$ |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---
|  | $\underbrace{\hskip 34.1433pt}$ | $\underbrace{\hskip 34.1433pt}$ |  |  |  | . |  |  |  |  | $\underbrace{\hskip 34.1433pt}$ |  |  |  |  |  |
|  |  | ${d_{1}}$ |  |  | ${d_{2}}$ |  |  |  |  | … |  |  |  |  |  |  $d_{n-1}$ |  | $\leftarrow$ | Euclidian Distance: | $d_{i}$ | $=$ | $\left\|P_{i+1}-P_{i}\right\|_{2}$ | $1\leq i\leq n-1$
|  |  | $\downarrow$ |  |  | $\downarrow$ |  |  |  |  |  |  |  |  |  |  |  $\downarrow$ |  |  |  |  |  |  |
|  |  | ${v_{1}}$ |  |  | ${v_{2}}$ |  |  |  |  | … |  |  |  |  |  |  $v_{n-1}$ |  | $\leftarrow$ | Velocity: | $v_{i}$ | $=$ | $\frac{d_{i}}{t}$ | $1\leq i\leq n-1$
|  |  |  | $\underbrace{\hskip 34.1433pt}$ |  |  |  |  |  |  |  |  |  | $\underbrace{\hskip 34.1433pt}$ |  |  |  |  |  |  |
|  |  |  |  | ${a_{1}}$ |  |  | ${a_{2}}$ |  |  | … |  |  |  |  |  $a_{n-2}$ |  |  | $\leftarrow$ | Acceleration: | $a_{i}$ | $=$ | $\frac{v_{i+1}-v_{i}}{t}$ | $1\leq i\leq n-2$
|  |  |  |  |  | $\underbrace{\hskip 34.1433pt}$ |  |  |  |  |  |  | $\underbrace{\hskip 34.1433pt}$ |  |  |  |  |  |  |  |
|  |  |  |  |  |  | $J_{1}$ |  | $J_{2}$ |  | … |  |  |  |  $J_{n-3}$ |  |  |  | $\leftarrow$ | Jerk: | $J_{i}$ | $=$ | $\frac{a_{i+1}-a_{i}}{t}$ | $1\leq i\leq n-3$
|  |  |  |  |  |  |  | $\underbrace{\hskip 34.1433pt}$ |  |  |  | $\underbrace{\hskip 34.1433pt}$ |  |  |  |  |  |  |  |  |
|  |  |  |  |  |  |  |  | $j_{1}$ |  |  $j_{2}$ | … |  |  |  $j_{n-4}$ |  |  |  |  | $\leftarrow$ | Jounce: | $j_{i}$ | $=$ | $\frac{J_{i+1}-J_{i}}{t}$ | $1\leq i\leq n-4$
|  |  |  |  |  |  |  |  |  | $\underbrace{\hskip 34.1433pt}$ |  | $\underbrace{\hskip 34.1433pt}$ |  |  |  |  |  |  |  |  |  |
|  |  |  |  |  |  |  |  |  |  |  $C_{1}$ |  | … |  |  $C_{n-5}$ |  |  |  |  |  | $\leftarrow$ | Crackle: | $C_{i}$ | $=$ | $\frac{j_{i+1}-j_{i}}{t}$ | $1\leq i\leq n-5$

with
$\left\|P_{i+1}-P_{i}\right\|_{2}=\sqrt{\left(x_{i+1}-x_{i}\right)^{2}+\left(y_{i+1}-y_{i}\right)^{2}}$

Figure 2. Computation of higher order derivative features.

###  2.4. Machine Learning Classifier

For given trajectories of eye movements of a number of participants, we seek
an algorithm that, given an unseen trajectory, is able to detect which
participant has generated it. This type of problem is known as a
_classification problem_ , the algorithm that carries out the classification
is known as _classifier_.

The machine learning classifier Radial Basis Function Networks (RBFN)
(Broomhead and Lowe, 1988) has been used in (George and Routray, 2016) and
also the work in (Schröder et al., 2020). We compared RBFN with Random Forest
(RF) (Breiman, 2001) and we found that RBFN gave better accuracies. Therefore,
we use RBFN in this paper. A maximum of three instances of the RBFN
classifiers are trained: one to predict the user from fixation segments, the
second to predict the user from saccade segments and third to predict user
from blink segments (whenever available). The final prediction probability
$p^{i}_{\text{final}}$ is the weighted average of the probabilities of
fixation ($p^{i}_{\text{fix}}$), saccade ($p^{i}_{\text{sac}}$) and blink
($p^{i}_{\text{blink}}$) classifiers for each class $i$ (user ID):

$p^{i}_{\text{final}}=p^{i}_{\text{fix}}w_{\text{fix}}+p^{i}_{\text{sac}}w_{\text{sac}}+p^{i}_{\text{blink}}w_{\text{blink}},$

where $w_{\text{fix}}$, $w_{\text{sac}}$, and $w_{\text{blink}}$ are the
weights for the fixation, saccade and blink classifiers respectively. In case
the blink classifier is absent, $w_{\text{fix}}$, $w_{\text{sac}}$ are
typically selected as 0.5 each (Schröder et al., 2020; George and Routray,
2016). For $m$ classes, the class having the maximum probability
$p_{\text{max}}=\max\\{p^{i}_{\text{final}}\mid i\in(1,\ldots,m)\\}$ is final
outcome of the ML classifier.

###  2.5. Performance Metrics

In our case a class is a user of an eye tracking device in one of the
experiments. For a single experiment, we always calculate predictions for all
available users. As accuracy of user identification we divide the number of
correct predictions by the total number of predictions (equal to the number of
users). Since the result of the RBFN algorithm depends randomly on the
initialization of its internal state, we perform a cross-validation with 50
different states (seeds) and take the average accuracy as the final accuracy
of user identification for each experiment in this work. Together with the
final accuracy, we also report the _standard error of the mean_ :
$\sigma_{\mu}=\frac{\sigma}{\sqrt{k}}$, where $\sigma$ is the standard
deviation of the results and $k$ the number of runs.

##  3\. User Identification Experiments

This section introduces different experiments that are employed to investigate
the individual effects towards prediction accuracy of stimuli, IVT parameters,
including higher order speed derivatives, including blink features, and
including gender and age across the different datasets.

###  3.1. Effect of Stimuli

We study the accuracy of user identification in the four different datasets
with different stimuli. We use the default IVT parameters of VT = 50 °/s, MFD
= 100 ms for the RAN, TEX, and VST datasets. For GOF VT = 15 °/s is applied,
since the default parameters lead to zero fixations for some participants in
this dataset. We use the first 51 features shown in Table 2 and an equal
weighting of the saccade and fixation classifiers to compute the user
prediction probabilities. In the following, the user identification
experiments are explained and. The results are summarized in Table 4.

Table 4. Performance metrics with different datasets using 51 features over 50 runs. | Data
---
set
| Identification
---
Accuracy
| Number of
---
participants
| Trajectory
---
Length [s]
RAN | 92.62 $\pm$ 0.13 % | 153 | 100
TEX | 90.90 $\pm$ 0.10 % | 153 | 60
VST | 85.69 $\pm$ 0.16 % | 58 | 180
GOF | 77.91 $\pm$ 0.51 % | 153 | 60

##### Bioeye TEX/RAN

For both datasets, the ML classifier is trained with all the 153 participants
of the second session and tested with the first session. In our previous work
we achieved an identification accuracy of 94.10 % using RAN data and 90.80 %
using TEX data with one run (Schröder et al., 2020). In this work, the average
accuracy achieved over 50 runs with the RAN dataset is 92.62 $\pm$ 0.13 %
(maximum accuracy = 94.77 %) and with TEX dataset is 90.90 $\pm$ 0.10 %
(maximum accuracy = 92.28 %) which is a more stable prediction accuracy of our
classifier compared to our previously reported results in (Schröder et al.,
2020).

##### VST

The ML classifier is trained with all the 58 participants and use training and
testing sessions that are recorded in the same day. The accuracy achieved with
this data set is 85.69 $\pm$ 0.16 %.

##### GOF

This dataset has a large number of participants (193 males and 185 females in
different age groups). We used the first 16 trials for training and the
remaining 16 trials for testing. For better comparison with BioEye data, 153
participants (77 males and 76 females) are chosen randomly over 50 runs and
the average accuracy is 77.91 $\pm$ 0.51 %.

###  3.2. Effect of IVT Parameters

As mentioned previously, the IVT algorithm has two parameters namely velocity
threshold (VT) and minimum fixation duration (MFD). In order to study the
effect of changing the IVT parameters, a systematic parameter variation is
conducted to determine which IVT parameter leads to the highest accuracy of
our ML classifier.

An initial set of experiment is carried out with the RAN and TEX datasets. In
the first stage, we vary the VT with a fixed MFD of 100 ms and in the second
stage we fix the VT (at the value with the highest accuracy from the previous
stage) and vary the MFD. The parameter range under consideration was 10–100
°/s for VT and 50–150 ms for MFD. In each stage, first a broad variation is
done in steps of 10, then a fine variation is executed in steps of 1. For each
setting, we perform a cross validation with random 80 % subsets of the users
(i.e. 122 participants) for a total of 50 runs. 3(a) and 3(b) show the
accuracy of user identification along with the number of fixations as the
velocity threshold is varied. It can be noted that the highest accuracy is
always obtained around the highest number of fixations. The best accuracy is
achieved with a VT and MFD respectively of 26 °/s and 98 ms for TEX, 27 °/s
and 96 ms for RAN. With MFD = 100 ms the obtained accuracies are almost
identical (the difference is in the order of 0.05 %).

(a) RAN

(b) TEX

Figure 3. Variation of Velocity thresholds of BioEye Data with fixing MFD at
100 ms.

The above approach requires a cross validation in the initial stages to
compute the best parameters which is a time consuming process. Therefore, in
an alternative approach we first tune the VT parameter to achieve the highest
number of fixation. This is done as follows. We first prepare a plot such as
Figure 4 (which is for the RAN dataset). The $x$-coordinate represents the
number of fixations over all the participants in the data set and the
$y$-coordinate shows the velocity thresholds. The colored stripes are a
representation of the number of fixations of all the participants in the data
set for each velocity threshold. The beginning of the line indicates the
minimum number of fixations, while the end of the line indicates the maximum
number of fixations on each specific velocity threshold. The blue dots on the
colored stripes mark the average fixation number. From the plot we can
determine a peak in the number of fixations at a VT value of approximately
$24$. We then determine the precise accuracies for this VT value, plus a few
neighboring VT values and choose the VT value that gives the highest accuracy
within this range.

We performed the above described procedure and obtained the following results
for the four data sets using 51 features in fixation and saccade classifiers:

  * RAN

The highest number of fixations in RAN data set occurred with a velocity
threshold of 24 °/s with accuracy of 94.89 %, while the best accuracy of 95.96
% is achieved with a VT of 27 °/s and MFD of 100 ms over 50 runs.

  * TEX

The best accuracy of 93.23 % is achieved with a VT of 26 °/s and MFD of 100
ms, while the highest number of fixations occurred with a VT of 30 °/s with
accuracy of 92.24 % using TEX data set.

  * VST

The best accuracy of 94.82 % is attained with a VT of 100 °/s, MFD of 100 ms
and the highest number of fixation occurred at a VT of 120 °/s with accuracy
of 94.31 %.

  * GOF

In this data set, The VT of 21 °/s produced the highest number of fixations
with accuracy of 82.35 % and the best accuracy of 83.45 % is achieved with the
VT of 22 °/s and MFD of 100 ms over 152 participants (76 males and 76 females)
which are selected randomly over 50 runs from all the data users.

In all the above cases, we were able to significantly increase the accuracies
of user identification (by 3.34 % for RAN, 2.33 % for TEX, 9.03 % for VST and
4.76 % for GOF) by selecting the optimal IVT parameters (compare with Table
4).

Figure 4. Velocity threshold of IVT against number of fixation for training
session (RAN data set / 153 participants).

###  3.3. Effect of Higher-Order Derivatives

In order to study the effect of higher order derivative features on the
accuracy of user identification, we start with a minimal set of first 13
position based features and duration as a general feature. See the first 14
features in Table 2. Next we add 19 velocity based features and than, step by
step, we include 18 statistical features based on each of the other higher
order derivatives (acceleration, jerk, jounce, crackle). For RAN, TEX and VST,
the IVT VT parameter is chosen as 50 °/s and for GOF dataset, it is 15 °/s
since the default parameters lead to zero fixations for some participants in
this dataset.

Table 5 shows the accuracy with an increasing number of higher order
derivative features for the four different datasets. The accuracy of user
identification increases until the inclusion of jounce features for the RAN,
TEX and GOF datasets. For the crackle based features, the accuracy decreases
again. For the VST dataset, the accuracy rises only until the jerk level
features and decreases already for jounce.

The shown observations attest our hypothesis that including higher order
derivatives (up to a certain level) is indeed a useful way to capture
meaningful information that contribute to an increase in the accuracy of user
identification.

Table 5. Performance metrics over 50 runs with varying number of features of higher order derivatives of the gaze trajectory of all the datasets. | Derivative |  | Number
---
of
Features
| RAN (153 participants)
---
Acc. VT=50 °/s,
MFD=100 ms
| TEX (153 participants)
---
Acc. VT=50 °/s,
MFD=100 ms
| VST (58 participants)
---
Acc. VT=50 °/s,
MFD=100 ms
| GOF (153 participants)
---
Acc. VT=15 °/s,
MFD=100 ms
0. | Position | 14 | 83.02 $\pm$ 0.22 % | 84.30 $\pm$ 0.17 % | 84.79 $\pm$ 0.22 % | 46.82 $\pm$ 0.44 %
1. | Velocity | 33 | 89.67 $\pm$ 0.15 % | 89.81 $\pm$ 0.11 % | 85.03 $\pm$ 0.15 % | 69.46 $\pm$ 0.54 %
2. | Acceleration | 51 | 92.62 $\pm$ 0.13 % | 90.90 $\pm$ 0.10 % | 85.69$\pm$ 0.16 % | 77.91 $\pm$ 0.51 %
3. | Jerk | 69 | 93.54 $\pm$ 0.10 % | 91.73 $\pm$ 0.12 % | 86.52 $\pm$ 0.23 % | 80.08 $\pm$ 0.50 %
4. | Jounce | 87 | 94.58 $\pm$ 0.10 % | 91.96 $\pm$ 0.08 % | 85.13 $\pm$ 0.17 % | 81.02 $\pm$ 0.53 %
5. | Crackle | 105 | 94.15 $\pm$ 0.09 % | 90.86 $\pm$ 0.14 % | 83.82 $\pm$ 0.14 % | 80.54 $\pm$ 0.54 %

###  3.4. Effect of Blinking Features

We study the effect of including the blink classifier as introduced in Section
2.4 in addition to fixation and saccade classifiers. Since, the identification
accuracy of the blink classifier is lower than the fixation and saccade
classifiers, we can not weigh all of them equally to compute the final
accuracy like done before. In order to find the optimal weights for our three
classifiers, we use the Nelder-Mead Method (Nelder and Mead, 1965) which is a
popular direct search method (based on function comparison (Gao and Han,
2012)) and is suited for optimization problems for which derivatives may not
be known. As shown in Table 6, the accuracy increases by 1.35 %, 1.25 %, and
0.5 % with use of the blink classifier in VST, RAN, and TEX dataset
respectively.

Table 6. Performance metrics over 50 runs using blink classifier in RAN, TEX, and VST datasets (VT = 50 °/s and MFD = 100 ms). Dataset |  | Sac/Fix/blink
---
features
| CLF weights
---
Sac/Fix/Blink
| Identification
---
Accuracy
RAN | 51/51/0 | 0.5/0.5/0.0 | 92.62 $\pm$ 0.13 %
| 51/51/7 | 0.408/0.578/0.015 | 93.87 $\pm$ 0.12 %
TEX | 51/51/0 | 0.5/0.5/0.0 | 90.90 $\pm$ 0.10 %
| 51/51/7 | 0.4453/0.5453/0.0094 | 91.40 $\pm$ 0.10 %
VST | 51/51/0 | 0.5/0.5/0.0 | 85.69 $\pm$ 0.16 %
| 51/51/7 | 0.568/0.3938/0.0381 | 87.04 $\pm$ 0.24 %

###  3.5. Effect of Gender and Age Groups on User Identification

We study the effect of gender and age on the accuracy of user identification
using the GOF dataset. It has large number of participants with gender and age
information of the participants (Table 7).

Table 7. Demographics of Males and Females in GOF dataset. Age Group | Males | Females | Total
---|---|---|---
20 – 40 | 151 | 157 | 308
41 – 72 | 42 | 28 | 70

####  3.5.1. Gender

To study the effect of gender, user identification experiments are performed
in three different groups with each 150 participants: a balanced group with 75
Males and 75 Females, a group exclusively with male participants and a group
exclusively with female participants. The participants of the groups were
randomly chosen from the complete dataset 50 times each. The average of the
user identification accuracies are reported in Table 8. For the mixed group,
the accuracy achieved is 83.25 $\pm$ 0.48 % . In isolated groups of males and
females, the accuracy is found to be higher in the female group (88.85 $\pm$
0.32 %) when compared to the male one (77.37 $\pm$ 0.50 %). This shows that
user identification is biased by gender and is found to be more accurate in
female than male group of participants.

####  3.5.2. Age

To study the effect of age, the participants were split into two age groups
20–40 years and 41–72 years. Both age groups contain 56 participants with an
equal number of males and females. The user identification can be performed in
the older age group with a five percentage points higher accuracy in
comparison to the younger age group (Table 8).

Table 8. Effect of gender and age groups on average user identification accuracy over 50 runs with VT = 22 using 51 features. Age group |  | Number
---
of participants
M | F |  | Identification
---
Accuracy
20 – 72 | 150 | 75 | 75 | 83.25 $\pm$ 0.48 %
20 – 72 | 150 | 150 | 0 | 77.37 $\pm$ 0.50 %
20 – 72 | 150 | 0 | 150 | 88.85 $\pm$ 0.32 %
20 – 40 | 56 | 28 | 28 | 85.96 $\pm$ 0.79 %
41 – 72 | 56 | 28 | 28 | 91.43 $\pm$ 0.47 %

###  3.6. Effect of the Time Gap Between Train and Test Data

Finally, we study the effect of the time gap between train and test data using
the RAN, TEX and VST data sets. The effect is also referred to as template
aging (George and Routray, 2016).

####  3.6.1. BioEye datasets

Out of 153 participants, this dataset has 37 participants for which recordings
are also available after one year. The first experiment in this dataset is
conducted with these participants. The IVT parameters used in these
experiments are VT = 50 °/s and MFD = 100 ms. For the RAN dataset, the average
accuracy over 50 runs is 83.51 $\pm$ 0.18 % and with TEX the accuracy is 75.24
$\pm$ 0.32 %. In comparison with the work of George and Routray (George and
Routray, 2016) running the same experiment yields 81.08 % and 78.38 % with one
run for RAN and TEX respectively.

####  3.6.2. VST dataset

This dataset has a time gap of more than two weeks between the two trials and
two hours between four sessions in each trial. The accuracies of user
identification were noted as $(94.6\pm 0.1)\,\%$ (2 hour time gap i.e. train
with session 1 and test with session 4 both in trial 1) and $(76.0\pm
0.2)\,\%$ (2 week time gap i.e. train with session 1, trial 1 and test with
session 1, trial 2).

As expected, the accuracy of user identification decreases for both datasets
when there is a significant time gap between train and test sessions. This may
be attributed to changing physiological parameters of the participants, device
characteristics, and some other inexplicable effects.

##  4\. Study of Combined Factors

In this section, we combine the effects of above factors which leads to
further improvement in user identification accuracy.

Table 9. Performance metrics over 50 runs with varying number of features of higher order derivatives of the gaze trajectory of all the datasets with their best Velocity Threshold (VT). | Derivatives
---
order
| Features
---
Number
| RAN (153 participants)
---
Acc. VT=27 °/s,
MFD= 96 ms
| TEX (153 participants)
---
Acc. VT=26 °/s,
MFD= 98 ms
| VST (58 participants)
---
Acc. VT=100 °/s,
MFD= 100 ms
| GOF (153 participants)
---
Acc. VT=22 °/s,
MFD= 100 ms
0 | 14 | 88.08 $\pm$ 0.19 % | 85.77 $\pm$ 0.13 % | 91.86 $\pm$ 0.22 % | 57.53 $\pm$ 0.53 %
1 | 33 | 93.71 $\pm$ 0.15 % | 91.09 $\pm$ 0.16 % | 94.41 $\pm$ 0.13 % | 76.55 $\pm$ 0.55 %
2 | 51 | 95.96$\pm$ 0.09 % | 93.23 $\pm$ 0.13 % | 94.72 $\pm$ 0.08 % | 82.67 $\pm$ 0.52 %
3 | 69 | 95.16$\pm$ 0.12 % | 93.12 $\pm$ 0.12 % | 93.59 $\pm$ 0.12 % | 84.48 $\pm$ 0.44 %
4 | 87 | 95.37$\pm$ 0.08 % | 93.39 $\pm$ 0.09 % | 91.59 $\pm$ 0.20 % | 84.08 $\pm$ 0.46 %
5 | 105 | 94.89$\pm$ 0.10 % | 93.04 $\pm$ 0.11 % | 90.34 $\pm$ 0.19 % | 83.20 $\pm$ 0.48 %

(a) RAN

(b) TEX

(c) VST

(d) GOF

Figure 5. Performance metrics over 50 runs with different number features of
higher order derivatives of the gaze trajectory for different datasets.

###  4.1. Combination of IVT Parameter Tuning and Higher-Order Derivatives
Features

Here, we take the best IVT parameters with the approach described in Section
3.2 for the four datasets and repeat the user identification experiments with
increasing number of higher order derivative features. Table 9 shows the
average of accuracies over 50 runs of increasing number of higher derivative
features of fixation and saccade using the best IVT parameters for the four
different datasets. Similarly, 5(a), 5(b), 5(c), and 5(d) compare the
accuracies between the default and optimal IVT parameters for RAN, TEX, VST
and GOF datasets respectively for including an increasing number of higher
order derivative features. It can be noticed that for the RAN and VST datasets
while using the optimal IVT parameters, the best accuracy were observed with
fewer features (until acceleration level) in comparison to the case when the
default IVT parameters were used (see Table 5). For TEX and GOF datasets, the
best accuracy were achieved with including jounce (93.39 $\pm$ 0.09 % for Tex)
and jerk (84.48 $\pm$ 0.44 % for GOF) level features.

Overall, one can conclude that finding the best IVT parameters and including
higher order derivative features are somewhat complementary: When the default
threshold is used, including higher order derivatives always increases the
best accuracy for all the four datasets. However, finding the best IVT
threshold may yield similar best case accuracies with lower derivative
features as this is the case for RAN and VST datasets. Nevertheless, for TEX
and GOF datasets, the accuracy still increases when jounce and jerk level
features in TEX and GOF datasets respectively are included even after using
the optimal IVT parameters.

###  4.2. Combining IVT Tuning and Higher-Order Derivatives with Blink
Classifier

Here, we study the case of including the blink classifier with the optimal IVT
threshold and set of higher order derivatives features. As mentioned in
Section 3.4, we employ the Nelder-Mead optimization method to find the optimal
weights for combining the fixation, saccade and blink classifiers. The
achieved accuracy for the three datasets are reported in Table 10. The
accuracy improves by 0.70 % for the RAN dataset yielding the final accuracy of
96.64 $\pm$ 0.07 %, by 0.15 % for the TEX dataset leading to 93.53 $\pm$ 0.11
%, and by 0.07 % for the VST dataset leading to 94.79 $\pm$ 0.03 %.

Table 10. Performance metrics over 50 runs using blink classifier in RAN data (VT = 27 °/s and MFD = 96 ms), TEX data (VT = 26 °/s and MFD = 98 ms), and VST data (VT = 100 °/s and MFD = 100 ms). Dataset |  | Fix/Sac/blink
---
features
| CLF weights
---
Sac/Fix/Blink
| Identification
---
Accuracy
RAN | 51/51/7 | 0.5/0.5/0.0 | 95.96 $\pm$ 0.09 %
| 51/51/7 | 0.543/0.447/0.010 | 96.64 $\pm$ 0.07 %
TEX | 87/87/0 | 0.5/0.5/0.0 | 93.39 $\pm$ 0.09 %
| 87/87/7 | 0.529/0.466/0.005 | 93.53 $\pm$ 0.11 %
VST | 51/51/0 | 0.5/0.5/0.0 | 94.72 $\pm$ 0.08 %
| 51/51/7 | 0.513/0.477/0.010 | 94.79 $\pm$ 0.03 %
Table 11. A comparison of performance metrics over 50 runs using 51 features with 58 participants of all the datasets. | Datasets
---
| VT, (MFD = 100 ms)
---
| Identification
---
Accuracy
RAN | $50\,^{\circ}\\!/s$ | 90.37 $\pm$ 0.48 %
TEX | $50\,^{\circ}\\!/s$ | 90.90 $\pm$ 0.10 %
VST | $50\,^{\circ}\\!/s$ | 73.59 $\pm$ 0.39 %
GOF | $15\,^{\circ}\\!/s$ | 84.48 $\pm$ 0.79 %
RAN | $27\,^{\circ}\\!/s$ | 92.82 $\pm$ 0.38 %
TEX | $26\,^{\circ}\\!/s$ | 93.23 $\pm$ 0.13 %
VST | $100\,^{\circ}\\!/s$ | 71.62 $\pm$ 0.35 %
GOF | $22\,^{\circ}\\!/s$ | 85.21 $\pm$ 0.64 %
Table 12. Identification accuracy, number of fixations in BioEye data sets with two different VT and different trajectory length of RAN data. | RAN | (MFD = 100 ms) |
---|---|---|---
Vel. | Fix. No. |  | Identification
---
Accuracy
Trajectory
threshold |  |  | length
50 | 16891 | 84.43 $\pm$ 0.23 % | 60 sec (start)
27 | 18831 | 89.75 $\pm$ 0.16 % | 60 sec (start)
50 | 17451 | 82.37 $\pm$ 0.20 % | 60 sec (end)
| TEX | (MFD = 100 ms) |
50 | 32944 | 90.90 $\pm$ 0.10 % | 60 sec
26 | 33584 | 93.23 $\pm$ 0.13 % | 60 sec

###  4.3. Effect of Stimuli after Homogenizing the Different Datasets

Here we revisit the question which stimuli is the best for user prediction.
This was previously studied in Section 3.1. To get a fair comparison we
homogenize the different aspects, i.e., number of participants, trajectory
length, and age group of these datasets. As mentioned previously, each dataset
has a different number of participants (RAN: 153, TEX: 153, VST: 58 and GOF:
378). For a fair comparison, the minimum number of participants among all the
datasets has been selected (i.e. 58 participants). The participants are drawn
at random when the datasets have a higher number of available participants. A
fixed trajectory length (first minute) is selected to test the effect of the
stimuli on the user identification accuracy. Since, RAN, TEX and VST datasets
have participants only in the age group 20–40, we selected only participants
of the corresponding age group 1 of the GOF dataset.

The results of these experiments are reported in Table 11. One can notice that
the TEX data accuracy is the highest: 93.23 % with the optimal IVT parameter
VT = 26 °/s. This is according to the state of art e.g. (Friedman et al.,
2017; Lohr et al., 2020) since the eye movements during reading have been used
to achieve some of the best biometric performances. However, it is important
to consider the number of fixations (if recorded with the same eye tracker)
before concluding, which stimuli is the best, as the number of fixations plays
a vital role in attaining a higher accuracy. The higher the number of
fixations, the better the accuracy achieved. Table 12 notes the number of
fixations for the RAN datasets for a trajectory length of 1 minute. It is much
lower than in the TEX dataset, which might influence that the accuracy for TEX
is higher than that of RAN.

##  5\. Conclusion and future work

This paper presents an approach for user identification which works
consistently and robustly well with four different datasets of varying
stimuli. The achieved identification accuracies are 96.64 % in RAN, 93.53 % in
TEX, 94.72 % in VST, and 84.48 % in GOF data. Additionally, an extensive study
has been done to investigate different factors (for example, IVT parameters,
higher order derivative features, effect of gender and age, template aging
effect, blinks etc.) that can affect user identification performance. Our
results suggest that selecting the best VT and higher-order derivative
features have the greatest impact on accuracy. We also found that user
identification works better in the solo female group than in solo male groups
and hence is biased towards gender. Similarly, it works better in the older
age group than in a younger age group of participants. We believe more work is
needed on improving the accuracy of user identification when there is a
significant time gap between train and test sessions.

## Acknowledgment

The first author would like to acknowledge the support of DAAD PhD scholarship
(award number 91645228) for funding this research.

## References

  * (1)
  * Alkan and Cagiltay (2007) Serkan Alkan and Kursat Cagiltay. 2007.  Studying computer game learning experience through eye tracking.  _British Journal of Educational Technology_ 38, 3 (2007), 538–542.
  * Andersson et al. (2017) Richard Andersson, Linnea Larsson, Kenneth Holmqvist, Martin Stridh, and Marcus Nyström. 2017\.  One algorithm to rule them all? An evaluation and discussion of ten eye movement event-detection algorithms.  _Behavior Research Methods_ 49, 2 (2017), 616–637.  https://doi.org/10.3758/s13428-016-0738-9
  * Armstrong and Olatunji (2012) Thomas Armstrong and Bunmi O. Olatunji. 2012.  Eye tracking of attention in the affective disorders: A meta-analytic review and synthesis.  _Clinical psychology review_ 32, 8 (2012), 704–723.
  * Benfatto et al. (2016) Mattias Nilsson Benfatto, Gustaf Öqvist Seimyr, Jan Ygge, Tony Pansell, Agneta Rydberg, and Christer Jacobson. 2016.  Screening for dyslexia using eye tracking during reading.  _PloS one_ 11, 12 (2016), e0165508.
  * Billeci et al. (2017) Lucia Billeci, Antonio Narzisi, Alessandro Tonacci, Beatrice Sbriscia-Fioretti, Luca Serasini, Francesca Fulceri, Fabio Apicella, Federico Sicca, Sara Calderoni, and Filippo Muratori. 2017.  An integrated EEG and eye-tracking approach for the study of responding and initiating joint attention in Autism Spectrum Disorders.  _Scientific Reports_ 7, 1 (2017), 1–13.
  * Breiman (2001) Leo Breiman. 2001\.  Random Forests.  _Machine Learning_ 45, 1 (2001), 5–32.  https://doi.org/10.1023/A:1010933404324
  * Broomhead and Lowe (1988) David S. Broomhead and David Lowe. 1988.  _Radial basis functions, multi-variable functional interpolation and adaptive networks_.  Technical Report. Royal Signals and Radar Establishment Malvern (United Kingdom).
  * Coutrot et al. (2016) Antoine Coutrot, Nicola Binetti, Charlotte Harrison, Isabelle Mareschal, and Alan Johnston. 2016\.  Face exploration dynamics differentiate men and women.  _Journal of vision_ 16, 14 (2016), 16–16.
  * Doughty (2002) Michael J. Doughty. 2002\.  Further assessment of gender-and blink pattern-related differences in the spontaneous eyeblink activity in primary gaze in young adult humans.  _Optometry and Vision Science_ 79, 7 (2002), 439–447.
  * Esfahani (2016) Nastaran Maus Esfahani. 2016\.  A Brief Review of Human Identification Using Eye Movement.  _Journal of Pattern Recognition Research_ 11, 1 (2016), 15–24.  https://doi.org/10.13140/RG.2.1.3466.3924
  * Friedman et al. (2017) Lee Friedman, Mark S. Nixon, and Oleg V. Komogortsev. 2017\.  Method to assess the temporal persistence of potential biometric features: Application to oculomotor, gait, face and brain structure databases.  _PloS one_ 12, 6 (2017), e0178501.
  * Galdi et al. (2016) Chiara Galdi, Michele Nappi, Daniel Riccio, and Harry Wechsler. 2016.  Eye movement analysis for human authentication: a critical survey.  _Pattern Recognition Letters_ 84 (2016), 272–283.  https://doi.org/10.1016/j.patrec.2016.11.002
  * Gao and Han (2012) Fuchang Gao and Lixing Han. 2012.  Implementing the Nelder-Mead simplex algorithm with adaptive parameters.  _Computational Optimization and Applications_ 51, 1 (2012), 259–277.
  * George and Routray (2016) Anjith George and Aurobinda Routray. 2016.  A score level fusion method for eye movement biometrics.  _Pattern Recognition Letters_ 82 (2016), 207–215.  https://doi.org/10.1016/j.patrec.2015.11.020
  * Holland and Komogortsev (2011) Corey Holland and Oleg V. Komogortsev. 2011.  Biometric identification via eye movement scanpaths in reading. In _2011 IEEE International Joint Conference on Biometrics, IJCB 2011, Washington, DC, USA, October 11-13, 2011_. 1–8.  https://doi.org/10.1109/IJCB.2011.6117536
  * Holland and Komogortsev (2012) Corey D. Holland and Oleg V. Komogortsev. 2012.  Biometric verification via complex eye movements: The effects of environment and stimulus. In _IEEE Fifth International Conference on Biometrics: Theory, Applications and Systems, BTAS 2012, Arlington, VA, USA, September 23-27, 2012_. 39–46.  https://doi.org/10.1109/BTAS.2012.6374556
  * Jäger et al. (2019) Lena A. Jäger, Silvia Makowski, Paul Prasse, Sascha Liehr, Maximilian Seidler, and Tobias Scheffer. 2019.  Deep Eyedentification: Biometric identification using micro-movements of the eye. In _Joint European Conference on Machine Learning and Knowledge Discovery in Databases_. Springer, 299–314.
  * Juan (2006) Stephen Juan. 30 Jun 2006\.  Why do babies blink less often than adults?  https://www.theregister.com/2006/06/30/the_odd_body_blinking/.  [Online; accessed 11-August-2021].
  * Kasneci et al. (2021) Enkelejda Kasneci, Gjergji Kasneci, Ulrich Trautwein, Tobias Appel, Maike Tibus, Susanne M. Jaeggi, and Peter Gerjets. 2021.  Do your eye movements reveal your performance on an IQ test? A study linking eye movements and socio-demographic information to fluid intelligence.  _PsyArXiv_ (Mar 2021).  https://doi.org/10.31234/osf.io/dru93
  * Kasprowski and Ober (2004) Pawel Kasprowski and Józef Ober. 2004.  Eye Movements in Biometrics. In _Biometric Authentication, ECCV 2004 International Workshop, BioAW 2004, Prague, Czech Republic, May 15, 2004, Proceedings_. 248–258.  https://doi.org/10.1007/978-3-540-25976-3_23
  * Krishna et al. (2019) Vrishab Krishna, Yi Ding, Aiwen Xu, and Tobias Höllerer. 2019\.  Multimodal biometric authentication for VR/AR using EEG and eye tracking. In _Adjunct of the 2019 International Conference on Multimodal Interaction_. 1–5.
  * Kröger et al. (2019) Jacob Leon Kröger, Otto Hans-Martin Lutz, and Florian Müller. 2019.  What does your gaze reveal about you? On the privacy implications of eye tracking. In _IFIP International Summer School on Privacy and Identity Management_. Springer, 226–241.
  * Lankes and Stoeckl (2020) Michael Lankes and Andreas Stoeckl. 2020.  Gazing at Pac-Man: Lessons Learned from a Eye-Tracking Study Focusing on Game Difficulty. In _ACM Symposium on Eye Tracking Research and Applications_ (Stuttgart, Germany) _(ETRA ’20 Short Papers)_. Association for Computing Machinery, New York, NY, USA, Article 62, 5 pages.  https://doi.org/10.1145/3379156.3391840
  * Li et al. (2018) Chunyong Li, Jiguo Xue, Cheng Quan, Jingwei Yue, and Chenggang Zhang. 2018.  Biometric recognition via texture features of eye movement trajectories in a visual searching task.  _PloS one_ 13, 4 (2018), e0194475.
  * Lin et al. (2004) Chern-Sheng Lin, Chia-Chin Huan, Chao-Ning Chan, Mau-Shiun Yeh, and Chuang-Chien Chiu. 2004.  Design of a computer game using an eye-tracking device for eye’s activity rehabilitation.  _Optics and lasers in engineering_ 42, 1 (2004), 91–108.
  * Lohr et al. (2020) Dillon J. Lohr, Samantha Aziz, and Oleg Komogortsev. 2020\.  Eye Movement Biometrics Using a New Dataset Collected in Virtual Reality. In _ACM Symposium on Eye Tracking Research and Applications_ (Stuttgart, Germany) _(ETRA ’20 Adjunct)_. Association for Computing Machinery, New York, NY, USA, Article 40, 3 pages.  https://doi.org/10.1145/3379157.3391420
  * Moss et al. (2012) Felix Joseph Mercer Moss, Roland Baddeley, and Nishan Canagarajah. 2012.  Eye movements to natural images as a function of sex and personality.  _PLoS One_ 7, 11 (2012), e47870.
  * Nelder and Mead (1965) J. A. Nelder and R. Mead. 1965.  A Simplex Method for Function Minimization.  _Comput. J._ 7, 4 (01 1965), 308–313.  https://doi.org/10.1093/comjnl/7.4.308 arXiv:https://academic.oup.com/comjnl/article-pdf/7/4/308/1013182/7-4-308.pdf
  * Olsen (2012) Anneli Olsen. 2012\.  The Tobii I-VT fixation filter.  _Tobii Technology_ 21 (2012).
  * Olsen and Matos (2012) Anneli Olsen and Ricardo Matos. 2012.  Identifying parameter values for an I-VT fixation filter suitable for handling data sampled with various sampling frequencies. In _Proceedings of the 2012 Symposium on Eye-Tracking Research and Applications, ETRA 2012, Santa Barbara, CA, USA, March 28-30, 2012_. 317–320.  https://doi.org/10.1145/2168556.2168625
  * Pedregosa et al. (2011) F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. 2011.  Scikit-learn: Machine Learning in Python.  _Journal of Machine Learning Research_ 12 (2011), 2825–2830.
  * Rakoczi et al. (2013) Gergely Rakoczi, Andrew Duchowski, Helena Casas-Tost, and Margit Pohl. 2013.  Visual perception of international traffic signs: influence of e-learning and culture on eye movements. In _Proceedings of the 2013 Conference on Eye Tracking South Africa_. 8–16.
  * Rigas and Komogortsev (2017) Ioannis Rigas and Oleg V. Komogortsev. 2017.  Current research in eye movement biometrics: An analysis based on BioEye 2015 competition.  _Image Vision Computing_ 58 (2017), 129–141.  https://doi.org/10.1016/j.imavis.2016.03.014
  * Salvucci and Goldberg (2000) Dario D. Salvucci and Joseph H. Goldberg. 2000.  Identifying fixations and saccades in eye-tracking protocols. In _Proceedings of the Eye Tracking Research & Application Symposium, ETRA 2000, Palm Beach Gardens, Florida, USA, November 6-8, 2000_. 71–78.  https://doi.org/10.1145/355017.355028
  * Sargezeh et al. (2019) Bahman Abdi Sargezeh, Niloofar Tavakoli, and Mohammad Reza Daliri. 2019.  Gender-based eye movement differences in passive indoor picture viewing: An eye-tracking study.  _Physiology & behavior_ 206 (2019), 43–50.
  * Sargolzaei et al. (2016) Arman Sargolzaei, Mohamed Abdelghani, Kang K. Yen, and Saman Sargolzaei. 2016.  Sensorimotor control: computing the immediate future from the delayed present.  _BMC bioinformatics_ 17, 7 (2016), 501–509.
  * Savitzky and Golay (1964) Abraham Savitzky and Marcel J. E. Golay. 1964.  Smoothing and Differentiation of Data by Simplified Least Squares Procedures.  _Analytical Chemistry_ 36, 8 (1964), 1627–1639.  https://doi.org/10.1021/ac60214a047 arXiv:https://doi.org/10.1021/ac60214a047
  * Schafer (2011) Ronald W. Schafer. 2011\.  What Is a Savitzky-Golay Filter? [Lecture Notes].  _IEEE Signal Process. Mag._ 28, 4 (2011), 111–117.  https://doi.org/10.1109/MSP.2011.941097
  * Schleicher et al. (2008) Robert Schleicher, Niels Galley, Susanne Briest, and Lars Galley. 2008.  Blinks and saccades as indicators of fatigue in sleepiness warnings: looking tired?  _Ergonomics_ 51, 7 (2008), 982–1010.
  * Schröder et al. (2020) Christoph Schröder, Sahar Mahdie Klim Al Zaidawi, Martin H. U. Prinzler, Sebastian Maneth, and Gabriel Zachmann. 2020.  Robustness of Eye Movement Biometrics Against Varying Stimuli and Varying Trajectory Length. In _CHI ’20: CHI Conference on Human Factors in Computing Systems, Honolulu, HI, USA, April 25-30, 2020_. ACM, 1–7.  https://doi.org/10.1145/3313831.3376534
  * Sen and Megaw (1984) Tayyar Sen and Ted Megaw. 1984.  The effects of task variables and prolonged performance on saccadic eye movement parameters.  In _Advances in Psychology_. Vol. 22. Elsevier, 103–111.
  * Wang et al. (2017) Shuhang Wang, Russell L. Woods, Francisco M. Costela, and Gang Luo. 2017.  Dynamic gaze-position prediction of saccadic eye movements using a Taylor series.  _Journal of vision_ 17, 14 (2017), 1–11.  Issue 3.
  * Wang et al. (2011) Yanfang Wang, Sonia S. Toor, Ramesh Gautam, and David B. Henson. 2011.  Blink frequency and duration during perimetry and their relationship to test–retest threshold variability.  _Investigative ophthalmology & visual science_ 52, 7 (2011), 4546–4550.
  * Zaidawi et al. (2020) Sahar Mahdie Klim Al Zaidawi, Martin H. U. Prinzler, Christoph Schröder, Gabriel Zachmann, and Sebastian Maneth. 2020.  Gender Classification of Prepubescent Children via Eye Movements with Reading Stimuli. In _Companion Publication of the 2020 International Conference on Multimodal Interaction, ICMI Companion 2020, Virtual Event, The Netherlands, October, 2020_. ACM, 1–6.  https://doi.org/10.1145/3395035.3425261