# SuperGF: Unifying Local and Global Features for Visual Localization

Wenzheng Song1,2 Ran Yan1 Boshu Lei1,4 Takayuki Okatani2,3
1Megvii 2GSIS, Tohoku University 3RIKEN Center for AIP 4Xi’an Jiaotong
University
{song<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

Advanced visual localization techniques encompass image retrieval challenges
and 6 Degree-of-Freedom (DoF) camera pose estimation, such as hierarchical
localization. Thus, they must extract global and local features from input
images. Previous methods have achieved this through resource-intensive or
accuracy-reducing means, such as combinatorial pipelines or multi-task
distillation. In this study, we present a novel method called SuperGF, which
effectively unifies local and global features for visual localization, leading
to a higher trade-off between localization accuracy and computational
efficiency. Specifically, SuperGF is a transformer-based aggregation model
that operates directly on image-matching-specific local features and generates
global features for retrieval. We conduct experimental evaluations of our
method in terms of both accuracy and efficiency, demonstrating its advantages
over other methods. We also provide implementations of SuperGF using various
types of local features, including dense and sparse learning-based or hand-
crafted descriptors.

##  1 Introduction

Visual localization is a key component in computer vision tasks such as
Structure-from-Motion (SfM) or SLAM, which is a fundamental problem in
numerous applications, such as autonomous driving, mobile robotics, or
augmented reality. This growing range of applications of visual localization
calls for reliable operation in both large-scale changing indoor and outdoor
environments, irrespective of the weather, illumination, or seasonal changes.
Visual localization is the problem of estimating the 6 Degree-of-Freedom (DoF)
camera pose from which a given image was taken relative to a reference scene
representation.

Advanced visual localization approaches are hierarchical, encapsulating image
retrieval problems and 6-DoF camera pose estimation [35, 50, 42]. i.e., given
a query image of the current view, related candidates are determined by
performing image retrieval in a database; then, perform local feature matching
between the query image and related candidates to establish point
correspondences between 2D to 3D, solving a PnP and estimate 6 DoF camera pose
of the query image.

Figure 1: Illustration of the feature-level gap between the two tasks. i.e.,
(a) image retrieval and (b) image matching. Where (a) and (b) are activation
maps of local features generated by NetVLAD [1] and SuperPoint [10],
respectively.

Naturally, two types of image features are needed to perform such a
hierarchical visual localization approach, i.e., global image features for
retrieval and local image features for image-matching. However, existing
studies struggle to unify these two types of image features. For instance, the
image retrieval task also involves these two types of features for the two-
stage retrieval strategy [5], i.e., global features are used for large-scale
coarse retrieval to get prior candidates, then re-rank it based on the number
of inliers obtained from local feature matching with geometry verification.
However, such local features tend to deviate from needs from 6-DoF pose
estimation. Specifically, local features yielded by a retrieval model always
tend to be semantically rich and lose spatial invariance [26, 14]. Moreover,
retrieval models don’t provide key-point detection, so these local features
lack corresponding exact pixel coordinate information.

Most studies employ two models for the hierarchical visual localization.,
i.e., one for image retrieval, such as NetVLAD [1], and another for image
matching, such as SIFT [24] or SuperPoint [10]. However, such a method is not
competitive as far as efficiency is concerned due to the absence of shared
computing. In addition, it is difficult to trade off efficiency and accuracy
in model selection. Specifically, accurate image retrieval can improve the
accuracy and efficiency of localization within a certain range. However,
advanced image retrieval models tend to be heavy to pursue higher accuracy,
which causes an increase in computational cost. Still, these improvements do
not result in corresponding gains for localization.

A recent study unifies global and local features for visual localization using
multitask distillation, i.e., HF-Net [35]. It uses a CNN that jointly
estimates local and global features with a shared encoder. It improves
computational efficiency by using a compression model, i.e., MobileNetVLAD
(MNV). However, the distillation inevitably harms inference accuracy,
especially in challenging cases, e.g., night scenes. One possible reason is
that they do not consider the conflict between the two tasks, i.e., image
retrieval and image-matching, at the feature level; see Figure 1.
Specifically, learning global features for image retrieval tends to induce
features in the backbone to be more semantic and less localizable, eventually
making it more sparse. Conversely, learning local features for image-matching
emphasizes feature invariance, which leads the features to focus more on local
information and lacks global semantics. This issue has been demonstrated
earlier [5] but needs to be handled more appropriately.

Most recently, transformers have been used for feature extraction in computer
vision tasks and have led to state-of-the-art results [26, 47]. It benefits
from the desirable property of the self-attention mechanism, which can break
through spatial constraints to establish global correlations, and aggregate
task-relevant features naturally.

In this paper, we propose a novel holistic model, SuperGF, which unifies
global and local features for visual localization. It works directly on the
local features generated by the image-matching model and aggregates to a
global image feature, similar to BoW [29, 30] or Fisher Vector [33]. A
transformer is adopted to perform feature aggregation, which is more accurate
and resource-friendly. It integrates global contextual information and
establishes global correlations between feature tokens by the self-attention
mechanism; thus, semantic cues can be learned automatically from local
features of the image-matching model. As a result, the transformer module can
bridge the feature-level gap between the two tasks and yield robust global
features for image retrieval in an efficient manner. We experimentally
evaluate global features yield by SuperGF on several benchmarks. Then we also
assess the performance of visual localization using the holistic model
combining different kinds of local features. The results show the advantage of
our model compared to existing methods.

##  2 Related Work

###  2.1 Approaches for Visual Localization

The approaches of previous works on visual localization can be summarized as
follow:

Structure-based Localization. Previous visual localization approaches mainly
rely on estimating correspondences between 2D keypoints in the query and 3D
points in a sparse model using local descriptors. The map is usually composed
of a 3D point cloud constructed via Structure-from-Motion (SfM), where each 3D
point is associated with one or more local feature descriptors. The query pose
is obtained by feature matching and solving a Perspective-n-Point problem
(PnP) [22]. However, direct matching methods tend to be resource-intensive or
fragile and challenging to apply in large-scale localization.

Image-based Localization. Visual localization in large-scale urban
environments is often approached as an image retrieval problem. Specifically,
the location of a given query image is predicted by transferring the geotag of
the most similar image retrieved from a geotagged database [1, 3, 6, 21, 37,
43, 44]. This approach scales to entire cities thanks to compact image
descriptors and efficient indexing techniques [2, 34, 13, 19] and can be
further improved by spatial re-ranking [5], informative feature selection [9,
8] or feature weighting [15, 18, 37, 44]. Image-based localization approaches
have recently shown promising results in robustness and efficiency but are not
competitive in terms of accuracy [1, 23], which output only an approximate
location of the query, not an exact 6-DoF pose.

Hierarchical Localization. Hierarchical localization takes an approach,
dividing the problem into a global, coarse search followed by a fine pose
estimation. It shows advantages in terms of efficiency and accuracy compared
to the above two approaches, which can be applied to large-scale. The
intermediate retrieval step of hierarchical localization limits the downstream
feature matching to a reasonable range, which reduces the computational cost
significantly while improving the localization performance by reducing the
influence of feature repetition. [36] proposed to search at the map level
using image retrieval and localize by matching hand-crafted local features
against retrieved 3D points. However, its robustness and efficiency are
limited by the underlying local descriptors and heterogeneous structure. Taira
_et al_. applied learning-based features to camera pose estimation but in a
dense, expensive manner [42]. Recently, HF-Net [35] integrated learning-based
models of image retrieval and image-matching, by model distillation that
simultaneously predicts keypoints as well as global and local descriptors for
accurate 6-DoF localization.

###  2.2 Global and Local Image Features

Before the emergence of deep learning, hand-crafted local features, such as
SIFT [24], ORB [32], and SURF [4], are widely applied in computer vision
fields such as image matching. Moreover, traditional aggregation methods [30,
33] are developed for generating global image features for image retrieval
using these hand-crafted local features. However, hand-crafted local features
are limited in invariance due to only involving low-level information.

Recent features emerging from convolutional neural networks (CNN) exhibit
unrivaled robustness at a low computing cost. However, it tends to be task-
specific. Specifically, task-specific local or global image features are
generated using different models in an end-to-end manner. Even though they
achieve superior performances in their respective domain, such as image
retrieval [26, 47, 16, 5] or image matching [10, 34, 13, 19, 28], there are
still problems in unifying for multi-task.

More recently, transformers have been adopted for feature extraction in
computer vision fields and achieved state-of-art performances [26, 47]. It
benefits from the desirable property of the self-attention mechanism, which
can naturally aggregate task-relevant features. Recent studies have applied
transformers to each component method for visual localization, i.e., image
retrieval [26, 47, 49] and image matching [34, 40].

##  3 Unify Local and Global features for Visual Localization.

###  3.1 Overview

SuperGF essentially plays the role of feature aggregation, i.e., aggregate
local image features used for image-matching, which are more localizable but
lack semantics, into glocal image features with rich semantic information used
for image retrieval. Figure. 2 illustrates the framework of SuperGF.

Given an input image, local features are extracted by a detector and
descriptor at first. In practice, SuperGF is designed for both a hand-crafted
descriptor and a learning-based descriptor, i.e., SIFT [24] and SuperPoint
[10], respectively. Moreover, we consider two different forms of the
descriptors separately, i.e., dense or sparse. Specifically, in the case of
the sparse version, only local descriptors where keypoints are taken into
count. In the case of the dense version, we adopt dense descriptors, i.e.,
dense SIFT or feature map output by the encoder of SuperPoint.

Then, we process the input of local features into tokens. In the case of the
dense version, we perform the local descriptors by a tokenizer consisting of a
one-layer CNN with downsampling operations. Then, we perform position
embedding for these tokens. In the case of the sparse version, the input of
local features is a variable-length sequence. To save computational costs, we
downsample them by performing clustering. Before that, we integrate three
features of descriptors, keypoints, and confidence scores by performing
positional encoding.

Finally, these tokens generated by the previous step are input into the three
layers of the vision transformer encoder. And a global image feature is
aggregated from the transformer’s output using a GeM pooling layer.

Figure 2: The framework of SuperGF. We provide two implementations of SuperGF,
i.e., the dense version and the sparse version. The former works on dense
local descriptors, such as dense SIFT or feature maps output by the SuperPoint
encoder. The latter works on sparse local features used for image matching,
i.e., keypoints ($p_{i}$), descriptors ($d_{i}$), and confidence scores
($r_{i}$). The modules indicated by green baskets contain learnable
parameters.

###  3.2 Local Feature Processing

Dense version. Given an input image $I\in\mathbb{R}^{H\times{W}}$, the feature
map of local descriptors of $I$ can be denoted as
$\mathbf{F}\in\mathbb{R}^{{128}\times\frac{H}{8}\times\frac{W}{8}}$. Let’s
define
$\mathcal{F}:\xi\rightarrow\mathbb{R}^{D\times\frac{H}{16}\times\frac{W}{16}}$
to denote a CNN layer encodes feature map $\mathbf{F}\in\xi$ into a feature
with the shape of $(D,\frac{H}{16},\frac{W}{16})$, then, the output of the
tokenizer can be denoted as:

$\bm{T}_{raw}=ReLU(\mathcal{F}(\mathbf{F}))+\bm{\mathcal{P}}$ (1)

where
$\bm{T}_{raw},\bm{\mathcal{P}}\in\mathbb{R}^{D\times\frac{H}{16}\times\frac{W}{16}}$,
$\bm{\mathcal{P}}$ is the tokens of Position Embedding, $D$ means the number
of channels of raw patch tokens, specifically, $D=512$. Finally, the raw patch
tokens $\bm{T}_{raw}$ is reshaped into a sequence of flattened 2D pattern
$(\frac{H}{16}\times\frac{W}{16},D)$ as input tokens for transformer encoder
layers. In practice, $7\times 7$ convolutional kernel is used, and the stride
is $2$.

Sparse version. Given an input image $I\in\mathbb{R}^{H\times{W}}$, the sparse
local features of $I$ can be denoted as $d_{i}\in\mathbb{R}^{i\times d}$:
local descriptors, $p_{i}\in\mathbb{R}^{i\times 2}$: keypoints position, and
$r_{i}\in\mathbb{R}^{i\times 1}$: confidence scores. $d=128$ when using SIFT,
and $d=256$ when using SuperPoint.

The initial representation $t_{i}\in\mathbb{R}^{i\times d}$ for each keypoint
$i$ combines its visual appearance and location. We embed the keypoint
position into a high-dimensional vector with a Multilayer Perceptron (MLP) as:

$t_{i}=d_{i}+MLP_{enc}(p_{i},r_{i})$ (2)

This encoder enables the graph network to later reason about both appearance
and position jointly, especially when combined with attention, called
Positional Encoding.

Inspired by recent studies , we propose a clustering module base on slot
attention, which takes an input of $t_{i}$ and outputs an ordered set of
features of clusters, denoted by $\bm{\zeta}$. Let the clustering module be
represented by function $\Phi(\bullet):\mathbb{R}^{i\times
d}\to\mathbb{R}^{N\times D}$, which can be defined as an iterative module:

$\Phi(\bullet)=\bm{\zeta}^{L},\quad\bm{\zeta}^{l}=\phi(\bullet;\bm{\zeta}^{l-1})$
(3)

where $\phi$ denotes the core function of the module applied $L$ times, and
$\bm{\zeta}^{0}\in\mathbb{R}^{N\times D}$ denotes a set of learnable templates
of clusters, which are initialized randomly. In practice, we set $N=512$ and
$L=6$. The final output is progressively formed by iterative refinement of the
templates and each token of $\bm{\zeta}$ is a function of all input of
$t_{i}$.

The architecture of the core function $\phi$ is inspired by [45, 12] and is
composed of a dot-product attention function $\psi$, followed by an MLP. The
function $\psi$ receives three inputs, i.e., key, value, and query,
represented by $K$, $Q$, and $V$, respectively. The corresponding input
generates the $K$, $Q$, and $V$, passing through layer normalization and fed
to three linear projection functions that project them to dimensions $d_{k}$,
$d_{q}$, and $d_{v}$, respectively. In practice, we set
$d_{k}=d_{q}=d_{v}=D=512$, $K$ and $V$ are generated by $t_{i}$, and Q is
generated by $\bm{\zeta}$. The functions $\phi$ and $\psi$ is given by:

$\phi(\bullet)=\psi(\bullet)+MLP(\psi(\bullet)),\quad\psi(\bullet)=\bm{\alpha}\cdot
V+\bm{\zeta}^{l}$ (4)

where $\bm{\alpha}$ denotes the attention maps over $t_{i}$. Thus, we have $N$
attention maps in total. i.e., $\bm{\alpha}\in\mathbb{R}^{i\times N}$
$\bm{\alpha}$ can be calculated as follow:

$\bm{\alpha}=\mathcal{N}_{L_{1}}(\mathrm{softmax}(\frac{QK^{\top}}{\sqrt{d_{k}}}))$
(5)

where $\mathcal{N}_{L_{1}}$ denotes $L_{1}$ normalization.

###  3.3 Aggregate to Global Image Features

SuperGF employs vision transformer encoders to achieve feature aggregation,
integrating global contextual information and establishing global correlations
between feature tokens by self-attention mechanism, and semantic cues can be
learned automatically from the input of local features. In practice, we follow
the implementation of the vision transformer encoder, which consists of a
stack of Multi-headed Self-Attention (MSA) and Multi-Layer Perceptron (MLP)
modules [45, 12], and add skip-connection between two transformer encoders.
Moreover, we discarded the operation of patch embedding and instead it by the
operation of the previous step; see Sec. 3.2. Let’s define the input as
$\mathbf{T}_{0}$, where $\mathbf{T}_{0}$ denotes the output of the previous
step, i.e., $\mathbf{T}_{raw}$ or $\bm{\zeta}^{L}$. Then define function
$\bm{\Theta}_{n}(\bullet):n\in[1,4]$ to denote the $n$-layer of vision
transformer encoders. The output tokens $\mathbf{T}_{out}$ is given by:

$\begin{split}&\bm{T}_{n}=\sum_{n=1}^{3}\left[\bm{T}_{n-1}+\bm{\Theta}_{n}(\bm{T}_{n-1})\right],\\\
&\mathbf{T}_{out}=MLP(\mathcal{L}(\bm{T}_{3}))\end{split}$ (6)

where $\mathcal{L}$ denotes the layer normalization, In practice, we double
the dimension of the input in the MLP. Thus,
$\mathbf{T}_{out}\in\mathbb{R}^{(\frac{H}{16}\times\frac{W}{16})\times 1024}$
in the case of the dense version or $\mathbf{T}_{out}\in\mathbb{R}^{N\times
1024}$ in the case of the sparse version. As a result, $\mathbf{T}_{out}$
tends to focus on task-relevant local features.

Finally, $\mathbf{T}_{out}$ is aggregated to global image features by
performing Generalized Mean (GeM) pooling , which is a flexible way to
aggregate local features with a learnable parameter $p$. The aggregated global
image feature $\bm{\mathfrak{D}}$ is given by:

$\begin{split}&\bm{\mathfrak{D}}=\mathcal{N}_{L_{2}}(GeM(\mathbf{T}_{out},p)),\\\
&GeM(\mathbf{T}_{out},p)=\left[\frac{1}{\lambda}\sum_{i=0}^{\lambda}{(T_{out}^{i})^{p}}\right]^{\frac{1}{p}}\end{split}$
(7)

where $\bm{\mathfrak{D}}\in\mathbb{R}^{1\times 1024}$, $\mathcal{N}_{L_{2}}$
denotes $L_{2}$ normalization, and $\lambda$ denotes the special size of
$\mathbf{T}_{out}$, i.e., $\lambda=\frac{H}{16}\times\frac{W}{16}$ in the case
of the dense version and $\lambda=N$ in the case of the sparse version. In
practice, we set $p=3$ in parameter initialization.

###  3.4 Training Strategy

####  3.4.1 Loss Function

In most studies, image retrieval or Visual Place Recognition (VPR) is treated
as a binary problem. Models are trained using pairs or triplets of image
samples labeled as either positive or negative with a metric learning loss,
such as the triplet loss [17, 11] or a contrastive loss [46]. However as
mentioned in existing studies [23, 31], since the distribution of scenes in
the real world is continuous, in the VPR task, the similarity between images
in VPR cannot be strictly defined in a binary fashion. Thus as we observed, in
most VPR datasets, there is a large ambiguity between the positive and
negative samples of the training data.

Moreover, VPR is essentially a problem of ranking, not classification. For
training a VPR model, metric learning losses distinguish between positive and
negative samples by defining a margin, which easily leads to a distance
ambiguity. This results in the performance being very sensitive to the loss
settings so it is often difficult to achieve optimal performance by optimizing
a metric learning loss directly.

Considering the above, we adopt a listwise loss. i.e., Average Precision (AP)
loss [7, 31, 19], to train our model. The AP loss train models by optimizing
ranking results directly. We follow the implementation of AP loss as Chen  _et
al_. [31]. As illustrated in Fig. 3, after inference an image batch of
$[I_{Q},I_{P},\bm{I}_{N}]$, we obtain global image descriptors
$[\bm{\mathfrak{D}}_{Q},\bm{\mathfrak{D}}_{P},\bm{\mathfrak{D}}_{N}]$. Then we
calculate cosine similarities
$\bm{\mathcal{S}}=\mathcal{S}_{i}:{i}\in[1,\alpha+\beta]$ between
$\bm{\mathfrak{D}}_{Q}$ and
$\bm{\mathfrak{D}}_{P}\cup{\bm{\mathfrak{D}}_{N}}$:

$\mathcal{S}_{i}=sim(\bm{\mathfrak{D}}_{Q},\bm{\mathfrak{D}}^{i})=\bm{\mathfrak{D}}_{Q}^{\top}\bm{\mathfrak{D}}^{i}$
(8)

where
$\bm{\mathfrak{D}}^{i}\in\bm{\mathfrak{D}}_{P}\cup{\bm{\mathfrak{D}}_{N}}$,
$\mathcal{S}_{i}\in[-1,1]$. Let’s define the function $AP(\cdot)$ denotes the
ranking metric AP for $\bm{\mathfrak{D}}_{Q}$, The final loss function can be
defined as:

$L_{AP}=1-AP(\bm{\mathcal{S}},\bm{S})$ (9)

where $\bm{S}\in[0,1]$ denotes the label of image similarity scores between
$I_{Q}$ and $I_{P}\cup{\bm{I}_{N}}$.

Figure 3:  Illustration of the training strategy. Each sample of training data
includes a batch composed of one query image $I_{Q}$, one positive sample
$I_{P}$, and $\alpha+\beta$ negatives; of these, $\alpha$ soft negative
samples and $\beta$ negative samples. i.e.,
$I_{N}^{i}\in\bm{I}_{N}:{i}=[1,...,\alpha+\beta]$. In practice, $\alpha=2$ and
$\beta=100$.

In the case of the sparse version, we further adopt an attention decorrelation
loss which aims at reducing the spatial correlation between attention maps. It
makes the output of the slot attention clustering module, i.e., $\bm{\zeta}$,
as complementary as possible. Specifically, we encourage them to attend to
different local features, i.e., different locations of the image. Let’s define
$\bm{\alpha}=[\alpha_{1},...,\alpha_{N}]$, The attention decorrelation loss is
given by:

$L_{attn}=\frac{1}{N(N-1)}\sum_{i\neq
j}\frac{\alpha_{i}\cdot\alpha_{j}}{\left\|\alpha_{i}\right\|_{2}\left\|\alpha_{j}\right\|_{2}}$
(10)

where $i,j\in\left\\{1,...,N\right\\}$. In practice, We train our sparse
version model with the above two losses jointly.

####  3.4.2 Training Setting

First, we aim to design robust global image descriptors that can be used for
place recognition for SuperPiont dependencies. So we fixed the SuperPoint
encoder during the training process. We perform image-level supervision using
the AP loss. In practice, we adopt the data annotation provided by Vallina _et
al_. [23] as the soft label, i.e, $\bm{S}$ for our model training, which
represents Field-of-View (FoV) overlap between query and database images.

Specifically, for each query image, we split the database images into three
kinds based on FoV overlap, i.e., positive samples where $S\in[0.5,1]$, soft
negative samples where $S\in(0,0.5)$, and negative samples where $S=0$. For
each query image $I_{Q}$, we compute the AP loss between it with 105 images in
the database, which contains 1 positive sample, 2 soft negative samples and
100 negative samples. For each iteration of the training process, we load 4
batches to compute loss and optimize the parameters of our model. We observe
that choosing more negative samples for each query image tends to result in
better performance. However, selecting too many negative samples takes up more
computational resources and provides limited gain to the model.

##  4 Experiments

We experimentally evaluate the proposed SuperGF on several benchmark datasets
compared with some state-of-the-art methods. We evaluate our method on two
tasks, i.e., VPR and visual localization. The former aims mainly to examine
the performance of global image features generated by SuperGF. We give the
details of experimental settings, datasets, evaluation metrics, and compared
methods in the following.

###  4.1 Implementation details

Model Settings. We set the latent embedding dimension of the transformer
encoders as 512, and the hidden dimension of MLP blocks as 1024. We trained
our model on the MSLS training set with settings reported in Sec. 3.4.2. We
selected 10,000 query samples randomly from all sub-cities for one epoch and
trained our model on 350 epochs in total. We adopted the AdamW optimizer and
set the parameter of weight decay $=10^{-4}$ in training. We set an initial
learning rate to $10^{-4}$ and it will finally decline to $10^{-6}$. We set
the input image size to $640\times 480$.

Experiment Settings for VPR. We perform the retrieval based on the $L_{2}$
distance between global image features to obtain the prior candidates. We
further perform geometry verification by matching sparse descriptors provided
by the front-end model, i.e., SIFT or SuperPoint. In practice, given an image
pair of the query and database, we obtain keypoints and sparse local
descriptors for both. Then, we perform feature matching to obtain initial
point correspondences, using Nearest Neighbor searching (NN) or SuperGlue
[34]. Finally, we estimate the homography with RANSAC as a robust estimator
and re-rank the prior candidates based on the number of inliers. We keep the
top 100 candidates retrieved by global image features and perform the geometry
verification.

Experiment Settings for Localization. Firstly, a sparse reference model is
built with SfM using database images. Specifically, we adopt prebuilt
reference models with COLMAP using RootSIFT or SuperPoint, provided by [25]
and [35]. They are each used in the case of the corresponding descriptor
employed in SuperGF to avoid the influence of using different detectors on
feature matching. For localization, given a query image, we retrieve the
top-$n$ candidates from the database using global image features generated by
SuperGF and match them one by one. In practice, we adopt NN and also SuperGlue
when SuperPoint is selected. Then, we keep all matches where a corresponding
3D point is included in the reference SfM model. Finally, we can lift them to
2D-3D matches, and the 6-DoF camera pose of the query image is obtained by
solving PnP with RANSAC.

###  4.2 Evaluation Datasets

We evaluated our method on several public benchmark datasets: MSLS[48],
Pitts30k [44], Nordland [27] and Tokyo247 [43] for VPR; The RobotCar Seasons
dataset [25] for localization. All of these datasets contain different
challenging appearance variations, such as changes in day-night, weather, and
season. More details of dataset usage are given in Supplementary Material. All
images are resized to $640\times 480$ while evaluation.

###  4.3 Metrics

As illustrated in Sec. 4.2, for VPR datasets, we use Recall@$N$ metric, which
computes the percentage of query images that are correctly localized. A query
image is considered retrieved successfully if at least one of the top $N$
ranked reference images is within a threshold distance from the ground truth
location of the query image. Default threshold definitions are used for all
datasets [16, 47, 1].

For the localization dataset, i.e., the RobotCar Seasons dataset, we report
the pose recall at position and orientation thresholds different for each
sequence, as defined by the benchmark [25].

###  4.4 Compared Methods

We experimentally compared SuperGF against several models on both VPR and
visual localization tasks with settings introduced in Sec. 4.1. VPR methods.
We compare methods in the VPR task considering both two approaches, i.e.,
single-pass and two-stage retrieval. The former only uses global image
features, and the latter performs re-ranking with geometry verification
additionally. For the experiment of single-pass retrieval, we compare with a
widely used baseline, i.e., NetVLAD [1], several state-of-the-art CNN-based
methods, including SFRS [14], ResNet50-GeM-GCL [23], GeM-AP [31], and also two
state-of-the-art transformer-based methods, i.e., SOLAR [26] and TransVPR
[47]. For the experiment of two-stage retrieval, we compare with DELG [5], and
two state-of-the-art methods, i.e., Patch-NetVLAD [16] and TransVPR. As
mentioned earlier in the paper, local features generated by all the above
methods don’t support 6-DoF camera pose estimation. In addition, we further
compare with a strong hybrid baseline, NV-SP-SG, which re-ranks NetVLAD
retrieved candidates by using SuperGlue matcher to match SuperPoint local
features.

Localization methods. Since image-based localization approaches lack accuracy,
we only consider localization approaches that can obtain 6-DoF camera poses;
see Sec. 2.1. In practice, we compare ours with two methods based on
structure-based localization approaches, i.e., Active Search (AS) [38] and
City Scale Localization (CSL) [41], hierarchical localization approach based
on hybrid pipelines including NV+SIFT, NV+SP, and also HF-Net [35]. NV+SIFT
and NV+SP mean using global features extracted by NetVLAD and local features
extracted by SIFT or SuperPoint.

##  5 Results and Discussion

###  5.1 Single-pass Retrieval

Table 1 shows the experimental results for single-pass retrieval. We can make
the following observations.

Table 1: Single-pass retrieval results with three metrics of recall, where
’R-’ means ’Recall’.

| MSLS val | MSLS challenge | Pitts30k test | Nordland test | Tokyo247 test
---|---|---|---|---|---
Method | R@1 | R@5 | R@10 | R@1 | R@5 | R@10 | R@1 | R@5 | R@10 | R@1 | R@5 | R@10 | R@1 | R@5 | R@10
NetVLAD | 47.7 | 62.8 | 70.9 | 30.7 | 41.9 | 46.4 | 82.1 | 91.4 | 93.8 | 12.5 | 21.4 | 26.1 | 57.1 | 72.1 | 77.1
SFRS | 69.2 | 80.3 | 83.1 | 41.5 | 52.0 | 56.3 | 89.4 | 94.7 | 95.9 | 18.8 | 32.8 | 39.8 | 85.4 | 91.1 | 93.3
ResNet50-GeM-GCL | 66.2 | 78.9 | 81.9 | 43.3 | 59.1 | 65.0 | 72.3 | 87.2 | 91.3 | 27.2 | 41.1 | 49.2 | 44.1 | 61.0 | 66.7
GeM-AP | 64.1 | 75.0 | 78.2 | 33.7 | 44.5 | 49.4 | 80.7 | 91.4 | 94.0 | 11.8 | 18.4 | 22.7 | 11.4 | 22.9 | 30.5
SOLAR | 78.3 | 87.2 | 89.6 | 45.2 | 60.5 | 69.3 | 85.4 | 92.6 | 94.8 | 36.6 | 51.3 | 58.8 | 76.2 | 84.4 | 88.3
TranVPR | 70.8 | 85.1 | 89.6 | 48.0 | 67.1 | 73.6 | 73.8 | 88.1 | 91.9 | 15.9 | 38.6 | 49.4 | – | – | –
SuperGF-DenseSP | 72.8 | 81.5 | 85.2 | 50.8 | 66.0 | 71.6 | 68.4 | 82.5 | 87.3 | 58.4 | 78.6 | 85.5 | 23.2 | 37.8 | 46.0
SuperGF-SparseSP | 71.3 | 81.3 | 85.0 | 48.2 | 65.3 | 70.4 | 60.2 | 77.6 | 83.1 | 57.1 | 75.4 | 82.0 | 25.7 | 39.4 | 44.4
SuperGF-DenseSIFT | 60.2 | 72.8 | 75.3 | 39.7 | 56.2 | 63.9 | 53.2 | 71.6 | 78.1 | 35.6 | 50.7 | 56.7 | 14.7 | 26.8 | 36.3
SuperGF-SparseSIFT | 49.7 | 67.5 | 74.4 | 11.2 | 22.7 | 37.5 | 37.7 | 59.1 | 69.2 | 15.5 | 30.5 | 38.4 | 8.6 | 15.9 | 21.0

Table 2: Two-stage retrieval results with three metrics of recall, where ’R-’
means ’Recall’.

| MSLS val | MSLS challenge | Pitts30k test | Nordland test | Tokyo247 test
---|---|---|---|---|---
Method | R@1 | R@5 | R@10 | R@1 | R@5 | R@10 | R@1 | R@5 | R@10 | R@1 | R@5 | R@10 | R@1 | R@5 | R@10
DELG | 83.2 | 90.0 | 91.1 | 52.2 | 61.9 | 65.4 | 89.9 | 95.4 | 96.7 | 51.3 | 66.8 | 69.8 | 78.8 | 86.7 | 90.0
NV-SP-SG | 78.1 | 81.9 | 84.3 | 50.6 | 56.9 | 58.3 | 87.2 | 94.8 | 96.4 | 29.1 | 33.5 | 34.3 | 71.2 | 82.0 | 78.8
Patch-NetVLAD | 79.5 | 86.2 | 87.7 | 48.1 | 57.6 | 60.5 | 88.7 | 94.5 | 95.9 | 46.4 | 58.0 | 60.4 | 95.9 | 96.8 | 97.1
TransVPR | 86.8 | 91.2 | 92.4 | 63.9 | 74.0 | 77.5 | 89.0 | 94.9 | 96.2 | 58.8 | 75.0 | 78.7 | – | – | –
SuperGF-DenseSP | 87.7 | 92.0 | 93.2 | 65.2 | 76.5 | 80.1 | 84.6 | 92.1 | 94.0 | 87.4 | 95.3 | 97.2 | 68.9 | 72.4 | 73.7
SuperGF-SparseSP | 86.4 | 91.5 | 92.9 | 64.4 | 74.3 | 79.1 | 83.3 | 90.8 | 93.8 | 85.3 | 94.7 | 97.1 | 63.2 | 70.6 | 72.8
SuperGF-DenseSIFT | 71.3 | 81.3 | 85.0 | 50.2 | 57.1 | 59.3 | 60.2 | 77.6 | 83.1 | 57.1 | 75.4 | 82.0 | 25.7 | 39.4 | 44.4
SuperGF-SparseSIFT | 49.7 | 67.5 | 74.4 | 45.5 | 50.2 | 59.8 | 55.2 | 67.1 | 78.9 | 46.8 | 54.1 | 60.2 | 25.2 | 37.9 | 43.1

First, the performance of single-pass retrieval only based on global image
features is highly relevant to the dataset, which denotes global features as a
compact representation of the entire image, which exhibits undesirable
generalizability, especially for high-precision retrievals, such as R@$1$ and
R@$5$.

Next, in general, even if we do not use separate generated task-specific local
features as other methods do, i.e., we adopt local features designed for
image-matching. Our method based on learned local features, such as SuperGF-
denseSP or SuperGF-sparseSP, can still generate global image features on par
with state-of-the-art retrieval models. It demonstrates the advantages of the
transformer in feature aggregation.

Finally, from the comparison between different implementations of our method,
we can observe that using learned local features for the aggregation, i.e.,
SuperGF-denseSP and SuperGF-sparseSP, show significantly better results than
using hand-craft local features, i.e., SuperGF-denseSIFT and SuperGF-
sparseSIFT. In addition, using dense local features show better results than
using sparse local features, especially for hand-craft local features.

###  5.2 Two-stage Retrieval

Table 2 shows the experimental results for two-stage retrieval. We can make
the following observations.

First, compared to single-pass retrieval, the robustness is significantly
improved by performing re-ranking based on geometry verification, which
illustrates the importance of spatial information for image retrieval, which
is discarded by global image features.

Next, using sparse local features designed for image-matching for the re-
ranking show on par even better performance than using those task-specific
local features. It is noteworthy that the former performs matching sparse
local features, which is more competitive in terms of efficiency than the
latter, which performs matching local features densely.

Finally, our methods with the SuperGlue matcher, i.e., SuperGF-denseSP-SG and
SuperGF-sparseSP-SG, show state-of-the-art level performance compared to
others.

Table 3: Localization results on the RobotCar Seasons dataset. We report the
recall [$\%$] at different distance and orientation thresholds, i.e., $\Delta
t$ and $\Delta r$, on different conditions. i.e., dusk, sun, night, and night-
rain.

| dusk | sun | night | night-rain
---|---|---|---|---
$\Delta t$@ | .25 / .50 / 5.0 | .25 / .50 / 5.0 | .25 / .50 / 5.0 | .25 / .50 / 5.0
Method $\Delta r$@ | 2 / 5 / 10 | 2 / 5 / 10 | 2 / 5 / 10 | 2 / 5 / 10
AS | 44.7 / 74.6 / 95.9 | 25.0 / 46.5 / 69.1 | 0.5 / 1.1 / 3.4 | 1.4 / 3.0 / 5.2
CSL | 56.6 / 82.7 / 95.9 | 28.0 / 47.0 / 70.4 | 0.2 / 0.9 / 5.3 | 0.9 / 4.3 / 9.1
NV+SIFT | 55.6 / 83.5 / 95.3 | 46.3 / 67.4 / 90.9 | 4.1 / 9.1 / 24.4 | 2.3 / 10.2 / 20.5
NV+SP | 54.8 / 83.0 / 96.2 | 51.7 / 73.9 / 92.4 | 6.6 / 17.1 / 32.2 | 5.2 / 17.0 / 26.6
NV+SP+SG | 62.4 / 83.5 / 97.2 | 52.8 / 80.2 / 96.1 | 29.0 / 66.9 / 90.2 | 46.4 / 76.1 / 92.0
HF-Net | 53.9 / 81.5 / 94.2 | 48.5 / 69.1 / 85.7 | 2.7 / 6.6 / 15.8 | 4.7 / 16.8 / 21.8
SuperGF-SparseSIFT | 55.8 / 83.6 / 95.0 | 45.8 / 67.2 / 91.1 | 3.6 / 8.8 / 20.8 | 2.6 / 11.5 / 19.8
SuperGF-DenseSIFT | 56.2 / 84.2 / 97.2 | 48.4 / 68.9 / 91.2 | 6.9 / 17.9 / 33.1 | 6.2 / 17.3 / 32.6
SuperGF-SparseSP | 55.1 / 83.0 / 96.7 | 52.2 / 75.7 / 96.1 | 8.0 / 19.9 / 36.8 | 8.2 / 22.0 / 33.9
SuperGF-DenseSP | 56.6 / 83.5 / 97.0 | 53.1 / 76.4 / 95.8 | 12.4 / 26.3 / 45.3 | 13.0 / 25.2 / 35.8
SuperGF-SparseSP-SG | 64.2 / 84.3 / 98.5 | 54.2 / 81.5 / 97.1 | 33.6 / 69.9 / 92.6 | 49.2 / 78.1 / 92.4
SuperGF-DenseSP-SG |  65.1 / 84.5 / 98.5 |  55.8 / 82.5 / 97.5 |  34.2 / 70.3 / 93.0 |  49.9 / 78.2 / 92.8

###  5.3 Visual Localization

We report the pose recall at position and orientation thresholds different for
each sequence, as defined by the benchmark [25]. Table 3 shows the
localization results for the different methods. We can make the following
observations.

First, compared to methods that intermediate a retrieval step, structure-based
methods, i.e., AS and CSL, show competitive results on the dusk sequence,
where the accuracy tends to saturate. In the more challenging sequences,
methods based on a hierarchical approach tend to work significantly better
than structure-based methods, which suffer from the increased ambiguity of the
matches.

Next, in contrasting the two pipelines of NV+SIFT and NV+SP, we can observe
that NV+SIFT shows slightly better results than NV+SP on easy cases, such as
dusk. In contrast, the NV+SP shows significantly better results than NV+SIFT
in those challenging cases, especially at night. It indicates that the SIFT
descriptors perform better than SuperPoint on camera pose estimation in normal
cases, but the conclusion is the opposite in the situation of challenging
cases. It is consistent with the findings of previous studies [39, 20].

Finally, by comparing our method with baselines, we can observe that although
the two-stage localization fuses image retrieval and camera pose estimation,
it is clear that the latter plays a dominant role. Especially for simple
cases, the use of SuperGF, which can be regarded as a robust global feature,
for retrieval has limited improvement for hierarchical localization. But for
challenging cases, we can observe the localization improvement using SuperGF.
Overall, by implanting SuperGF, our pipelines achieve state-of-art performance
on 6-DoF localization compared with others.

Table 4: Comparisons of model size and latency. Latency is measured on an
NVIDIA TITAN RTX GPU.

Method | Memory (MB) | Extraction latency (ms)
---|---|---
NetVLAD | 14.8 | 15.5
SFRS | 149.0 | 16.4
ResNet50-GeM | 23.5 | 11.1
SOLAR | 56.2 | 19.5
SuperGF-dense | 6.1 | 4.4
SuperGF-sparse | 2.2 | 2.3

###  5.4 Latency and Memory

In real-world applications, latency and scalability are important factors that
need to be considered for real-time visual localization. We report the
computational time and memory requirements for compared methods of extracting
glocal image features to process a single query image; see Table 4. In
practice, we unify the input image size as $480\times 640$ for all methods. In
the case of SuperGF-sparse, we randomly selected 100 images to generate sparse
local features for the test. SuperGF shows clear advantages in both terms than
others.

Combining the results of previous sections, we can conclude that SuerGF uses
minimal resources but generates SOTA-level global image features for image
retrieve and lead to better performance of 6-DoF localization. In particular,
the sparse version of SuperGF causes a loss of retrieval accuracy to some
extent, but it has obvious advantages in terms of feature extraction
efficiency. It is worth noting that SuperGF is affected by the adopted local
descriptor significantly. We prefer to use learning-based local features, such
as SuperPoint, joined with SuperGF. We tried to use SuperGF to generate global
features for hand-craft local features, i.e., SIFT, but the results were
unsatisfactory. One possible reason is the limited representative ability of
the hand-craft local features, which only jointly low-level information of
images.

##  6 Summary and Conclusion

This paper has presented a novel method for global feature extraction, i.e.,
SuperGF. It is transformer-based and designed for 6-DoF localization, which
acts directly on local features provided by descriptors of image-matching. The
results show our method’s advantages in terms of accuracy and efficiency.

We provide different implementations of SuperGF. i.e., different types of
local features with both learning-based and hand-craft descriptors. Users can
choose different versions according to their needs. Moreover, we encourage
using the learning-based descriptor, i.e., SuperPoint, joint with SuperGF. In
addition, if you pursue higher retrieval accuracy while localization, e.g.,
using for loop closure, then the dense version is more suitable. For 6-DoF
localization alone, the sparse version is better in efficiency.

## References

  * [1] Relja Arandjelovic, Petr Gronat, Akihiko Torii, Tomas Pajdla, and Josef Sivic.  Netvlad: Cnn architecture for weakly supervised place recognition.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 5297–5307, 2016.
  * [2] R. Arandjelović and A. Zisserman.  Three things everyone should know to improve object retrieval.  In Proc. CVPR, 2012.
  * [3] Relja Arandjelović and Andrew Zisserman.  Dislocation: Scalable descriptor distinctiveness for location recognition.  In Asian conference on computer vision, pages 188–204. Springer, 2014.
  * [4] H. Bay, T. Tuytelaars, and L. Van Gool.  Surf: Speeded up robust features.  In Proc. ECCV, 2006.
  * [5] Bingyi Cao, Andre Araujo, and Jack Sim.  Unifying deep local and global features for image search.  In European Conference on Computer Vision, pages 726–743. Springer, 2020.
  * [6] David Chen, Sam Tsai, Vijay Chandrasekhar, Gabriel Takacs, Huizhong Chen, Ramakrishna Vedantham, Radek Grzeszczuk, and Bernd Girod.  Residual enhanced visual vectors for on-device image matching.  In 2011 Conference Record of the Forty Fifth Asilomar Conference on Signals, Systems and Computers (ASILOMAR), pages 850–854. IEEE, 2011.
  * [7] Kean Chen, Weiyao Lin, Jianguo Li, John See, Ji Wang, and Junni Zou.  Ap-loss for accurate one-stage object detection.  IEEE Transactions on Pattern Analysis and Machine Intelligence, 43(11):3782–3798, 2020.
  * [8] Ondřej Chum, Andrej Mikulik, Michal Perdoch, and Jiří Matas.  Total recall ii: Query expansion revisited.  In CVPR 2011, pages 889–896. IEEE, 2011.
  * [9] Ondrej Chum, James Philbin, Josef Sivic, Michael Isard, and Andrew Zisserman.  Total recall: Automatic query expansion with a generative feature model for object retrieval.  In 2007 IEEE 11th International Conference on Computer Vision, pages 1–8. IEEE, 2007.
  * [10] D. DeTone, T. Malisiewicz, and A. Rabinovich.  Superpoint: Self-supervised interest point detection and description.  In Proc. CVPRW, 2018.  https://github.com/magicleap/SuperPointPretrainedNetwork.
  * [11] Xingping Dong and Jianbing Shen.  Triplet loss in siamese network for object tracking.  In Proceedings of the European conference on computer vision (ECCV), pages 459–474, 2018.
  * [12] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al.  An image is worth 16x16 words: Transformers for image recognition at scale.  arXiv preprint arXiv:2010.11929, 2020.
  * [13] M. Dusmanu, I. Rocco, T. Pajdla, M. Pollefeys, J. Sivic, A. Torii, and T. Sattler.  D2-net: A trainable cnn for joint detection and description of local features.  In Proc. CVPR, 2019.
  * [14] Yixiao Ge, Haibo Wang, Feng Zhu, Rui Zhao, and Hongsheng Li.  Self-supervising fine-grained region similarities for large-scale image localization.  In European conference on computer vision, pages 369–386. Springer, 2020.
  * [15] Petr Gronat, Guillaume Obozinski, Josef Sivic, and Tomas Pajdla.  Learning and calibrating per-location classifiers for visual place recognition.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 907–914, 2013.
  * [16] Stephen Hausler, Sourav Garg, Ming Xu, Michael Milford, and Tobias Fischer.  Patch-netvlad: Multi-scale fusion of locally-global descriptors for place recognition.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14141–14152, 2021.
  * [17] Alexander Hermans, Lucas Beyer, and Bastian Leibe.  In defense of the triplet loss for person re-identification.  arXiv preprint arXiv:1703.07737, 2017.
  * [18] Hervé Jégou, Matthijs Douze, and Cordelia Schmid.  On the burstiness of visual elements.  In 2009 IEEE conference on computer vision and pattern recognition, pages 1169–1176. IEEE, 2009.
  * [19] R. Jerome, W. Philippe, R. S. César, and H. Martin.  R2D2: repeatable and reliable detector and descriptor.  In Proc. NeurIPS, 2019.  https://github.com/naver/r2d2.
  * [20] Y. Jin, D. Mishkin, A. Mishchuk, J. Matas, P. Fua, K. M. Yi, and E. Trulls.  Image matching across wide baselines: From paper to practice.  IJCV, 2020.
  * [21] Hyo Jin Kim, Enrique Dunn, and Jan-Michael Frahm.  Learned contextual feature reweighting for image geo-localization.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2136–2145, 2017.
  * [22] Vincent Lepetit, Francesc Moreno-Noguer, and Pascal Fua.  Epnp: An accurate o (n) solution to the pnp problem.  International journal of computer vision, 81(2):155–166, 2009.
  * [23] María Leyva-Vallina, Nicola Strisciuglio, and Nicolai Petkov.  Generalized contrastive optimization of siamese networks for place recognition.  arXiv preprint arXiv:2103.06638, 2021.
  * [24] D. G. Lowe.  Distinctive image features from scale-invariant keypoints.  IJCV, 60(2):91–110, 2004.
  * [25] W. Maddern, G. Pascoe, C. Linegar, and P. Newman.  1 year, 1000 km: The oxford robotcar dataset.  International Journal of Robotics Research, 36(1):3–15, 2017.
  * [26] Tony Ng, Vassileios Balntas, Yurun Tian, and Krystian Mikolajczyk.  Solar: second-order loss and attention for image retrieval.  In European conference on computer vision, pages 253–270. Springer, 2020.
  * [27] Daniel Olid, José M Fácil, and Javier Civera.  Single-view place recognition under seasonal changes.  arXiv preprint arXiv:1808.06516, 2018.
  * [28] Y. Ono, E. Trulls, P. Fua, and K. M. Yi.  Lf-net: learning local features from images.  In Proc. NeurIPS, 2018.
  * [29] Filip Radenović, Giorgos Tolias, and Ondřej Chum.  Cnn image retrieval learns from bow: Unsupervised fine-tuning with hard examples.  In European conference on computer vision, pages 3–20. Springer, 2016.
  * [30] F. Radenović, G. Tolias, and O. Chum.  Cnn image retrieval learns from bow: Unsupervised fine-tuning with hard examples.  In Proc. ECCV, 2016.
  * [31] Jerome Revaud, Jon Almazán, Rafael S Rezende, and Cesar Roberto de Souza.  Learning with average precision: Training image retrieval with a listwise loss.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 5107–5116, 2019.
  * [32] E. Rublee, V. Rabaud, K. Konolige, and G. Bradski.  Orb: An efficient alternative to sift or surf.  In Proc. ICCV, 2011.
  * [33] Jorge Sánchez, Florent Perronnin, Thomas Mensink, and Jakob Verbeek.  Image classification with the fisher vector: Theory and practice.  International journal of computer vision, 105(3):222–245, 2013\.
  * [34] P. Sarlin, D. DeTone, T. Malisiewicz, and A. Rabinovich.  Superglue: Learning feature matching with graph neural networks.  In Proc. CVPR, 2020.  https://github.com/magicleap/SuperGluePretrainedNetwork.
  * [35] Paul-Edouard Sarlin, Cesar Cadena, Roland Siegwart, and Marcin Dymczyk.  From coarse to fine: Robust hierarchical localization at large scale.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12716–12725, 2019.
  * [36] Paul-Edouard Sarlin, Frédéric Debraine, Marcin Dymczyk, Roland Siegwart, and Cesar Cadena.  Leveraging deep visual descriptors for hierarchical efficient localization.  In Conference on Robot Learning, pages 456–465. PMLR, 2018.
  * [37] Torsten Sattler, Michal Havlena, Konrad Schindler, and Marc Pollefeys.  Large-scale location recognition and the geometric burstiness problem.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1582–1590, 2016.
  * [38] Torsten Sattler, Bastian Leibe, and Leif Kobbelt.  Efficient & effective prioritized matching for large-scale image-based localization.  IEEE transactions on pattern analysis and machine intelligence, 39(9):1744–1756, 2016.
  * [39] Wenzheng Song, Masanori Suganuma, Xing Liu, Noriyuki Shimobayashi, Daisuke Maruta, and Takayuki Okatani.  Matching in the dark: A dataset for matching image pairs of low-light scenes.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 6029–6038, 2021.
  * [40] Jiaming Sun, Zehong Shen, Yuang Wang, Hujun Bao, and Xiaowei Zhou.  Loftr: Detector-free local feature matching with transformers.  In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8922–8931, 2021.
  * [41] Linus Svärm, Olof Enqvist, Fredrik Kahl, and Magnus Oskarsson.  City-scale localization for cameras with known vertical direction.  IEEE transactions on pattern analysis and machine intelligence, 39(7):1455–1461, 2016.
  * [42] Hajime Taira, Masatoshi Okutomi, Torsten Sattler, Mircea Cimpoi, Marc Pollefeys, Josef Sivic, Tomas Pajdla, and Akihiko Torii.  Inloc: Indoor visual localization with dense matching and view synthesis.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 7199–7209, 2018.
  * [43] Akihiko Torii, Relja Arandjelovic, Josef Sivic, Masatoshi Okutomi, and Tomas Pajdla.  24/7 place recognition by view synthesis.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1808–1817, 2015.
  * [44] Akihiko Torii, Josef Sivic, Tomas Pajdla, and Masatoshi Okutomi.  Visual place recognition with repetitive structures.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 883–890, 2013.
  * [45] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin.  Attention is all you need.  Advances in neural information processing systems, 30, 2017.
  * [46] Feng Wang and Huaping Liu.  Understanding the behaviour of contrastive loss.  In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 2495–2504, 2021.
  * [47] Ruotong Wang, Yanqing Shen, Weiliang Zuo, Sanping Zhou, and Nanning Zheng.  Transvpr: Transformer-based place recognition with multi-level attention aggregation.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13648–13657, 2022.
  * [48] Frederik Warburg, Soren Hauberg, Manuel Lopez-Antequera, Pau Gargallo, Yubin Kuang, and Javier Civera.  Mapillary street-level sequences: A dataset for lifelong place recognition.  In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 2626–2635, 2020.
  * [49] Philippe Weinzaepfel, Thomas Lucas, Diane Larlus, and Yannis Kalantidis.  Learning super-features for image retrieval.  arXiv preprint arXiv:2201.13182, 2022.
  * [50] Nan Yang, Rui Wang, Jorg Stuckler, and Daniel Cremers.  Deep virtual stereo odometry: Leveraging deep depth prediction for monocular direct sparse odometry.  In Proceedings of the European Conference on Computer Vision (ECCV), pages 817–833, 2018.


# Theoretical study of topological properties of ferromagnetic pyrite CoS2

I. Robredo1,2, N. B. M. Schröter3,4, A. Reyes-Serrato1,5, A. Bergara1,2,6, F.
de Juan1,7, L. M. Schoop8, M.G. Vergniory1,9 1 Donostia International Physics
Center, 20018 Donostia-San Sebastian, Spain.  2 Physics Department, University
of the Basque Country UPV/EHU, 48080 Bilbao, Spain.  3 Swiss Light Source,
Paul Scherrer Institute, CH-5232 Villigen PSI, Switzerland.  4Max Planck
Institute of Microstructure Physics, 06120 Halle, Germany  5 Centro de
Nanociencias y Nanotecnología, Universidad Nacional Autónoma de México,
Apartado Postal 14, Ensenada Baja California, 22800, México.  6 Centro de
Física de Materiales, Centro Mixto CSIC -UPV/EHU, 20018 Donostia, Spain.  7
IKERBASQUE, Basque Foundation for Science, Maria Diaz de Haro 3, 48013 Bilbao,
Spain  8 Department of Chemistry, Princeton University, Princeton, NJ 08540,
USA.  9 Max Planck Institute for Chemical Physics of Solids , 01187 Dresden,
Germany.
inigo.robredo@dipc.org,maiagvergniory<EMAIL_ADDRESS>
(today)

###### Abstract

Since the discovery of the first topological material 15 years ago, the search
for material realizations of novel topological phases has become the driving
force of the field. While oftentimes we search for new materials, we forget
that well established materials can also display very interesting topological
properties. In this work, we revisit CoS2, a metallic ferromagnetic pyrite
that has been extensively studied in the literature due to its magnetic
properties. We study the topological features of its electronic band structure
and identify Weyl nodes and Nodal lines, as well as a symmetry-protected
4-fold fermion close to the Fermi level. Looking at different surface cleavage
planes, we observe both spin polarized Fermi arcs in the majority channel and
drumhead states. These findings suggest that CoS2 is a promising platform to
study topological phenomena, as well as a good candidate for spintronic
applications.

Keywords: Weyl semimetal, Fermi arcs, nodal lines, spintronics

##  1 Introduction

Topological semimetals are a class of materials that display protected band
crossings close to the Fermi level with nonzero Berry phase [1, 2, 3, 4, 5,
6]. In 3 dimensions (3D) one can find three types of topological band
crossings, namely, nodes, lines and planes. When nodes have a linear
dispersion in all directions and they carry a topological charge, they are
named Weyl nodes. Weyl nodes can only exist in systems with broken inversion
or time-reversal symmetry (TRS). They are stable even in the absence of
symmetries and have very interesting surface states called Fermi arcs, that
connect the surface projection of the nodes. From each Weyl node with
topological charge 1, there will be a Fermi arc stemming from its projection
in the surface. Important examples of inversion symmetry breaking Weyl
semimetals are transition metal monophosphides [7], MoTe2 [8] and TaAs [9, 10,
11]. Contrary to the prolific discovery of inversion symmetry-broken Weyl
semimetals, time reversal symmetry-broken (magnetic) ones are still scarce.
They were first predicted in pyrochlore iridates [12], HgCr2Se4 [13] and the
Heusler family of XCo2Z (X=IVB or VB; Z=IVA or IIIA) [14], and recently
measured experimentally in EuCd2As2 [15], Co3Sn2S2 [16, 17] and Co2MnGa[18].
These materials usually display a large anomalous Hall effect [16], which is
important for both electronic and spintronic devices [19], and they played a
fundamental role in the first experimental realization of the chiral anomaly
[20]. Higher order generalizations of Weyl nodes have been proposed [21] and
confirmed experimentally [22, 23, 24, 25, 26]. These so called multifold
fermions have higher Chern numbers, which produce more Fermi arcs in the
surface. It has also been shown that space group chirality (no inversion or
mirror symmetries) has an important effect on the connectivity of such Fermi
arcs [22, 27]. Even if the search for multifold fermions has been focused on
non-magnetic materials, there has been an increased shift of attention to
magnetic systems. There are, however, few material predictions that can host
magnetic multifold fermions [28, 29], thus, making it a priority to find new
material realizations of such multifold fermions.

Another notable topological feature are Nodal lines. These crossings are
protected by inversion-TRS, spin rotation or reflection symmetries [30, 31,
32, 33, 34, 35, 36, 37]. Compared to Weyl semimetals, Nodal line semimetals
are more difficult to diagnose, since the topological invariants [38] and,
thus, the surface spectrum, are protected by crystalline symmetries, so
surfaces not preserving them will not display topologically protected surface
states [39, 40, 41, 42]. In the facets where symmetries are preserved,
however, there will appear what are known as drumhead states, a set of surface
states covering the projection of the nodal line in the surface. In the cases
in which spin-orbit coupling (SOC) can be neglected, Nodal lines protected by
inversion-TRS can occur at any point in the Brillouin zone (BZ), as well as in
high symmetry planes. When including SOC, though, inversion-TRS can no longer
protect the crossings, and these will generally be gapped. This is the case
for spin rotation symmetry protected Nodal lines too, because SOC mixes spin
components and gaps the crossings. Then, in the cases in which SOC cannot be
neglected, only mirror symmetries can protect Nodal lines. This is why there
are few examples of real materials that host Nodal lines when SOC is taken
into account [43, 39], and why finding new ones is exciting both theoretically
and experimentally. Like in Weyl semimetals, Nodal line semimetals have
recently been found to have large Spin Hall current [44], expanding the family
of topological materials promising for topological electronic applications.
Generalizing Nodal lines, non-symmorphic symmetries can force crossings on
entire high symmetry planes in the boundary of the BZ. These are named nodal
planes, which give rise to Topological Protectorates; regions of the Fermi
surface that intersect the Nodal planes. They have been predicted to be large
sources of Berry curvature [45, 46].

Pyrites of the form XS2 (with X being Fe, Ni and Co) are a notable family of
TRS-breaking materials that have been studied for decades due to the large
size of pure crystals that can be found in nature or easily grown
experimentally [47, 48]. Among this family, CoS2 has been experimentally
confirmed to be ferromagnetic and extensively studied for its spin
polarization and magnetic properties. Unlike many magnetic materials, CoS2 is
an itinerant ferromagnet [49, 50], that is, the ferromagnetism does not stem
from highly localized electrons. Thus, electron-electron interactions can be
neglected when studying the electronic properties of the system. This implies
that standard mean field approximations such as density functional theory
(DFT) within the local spin density approximation (LSDA) are an excellent way
to obtain reliable and accurate electronic properties [51].

Topological condensed matter physics is a relatively new field of physics.
Thus, it is possible that ‘old’ materials could display topological properties
that have been missed. This is the case on point; it has been recently
reported experimentally that CoS2 is a Weyl semimetal, with spin-polarized
single and double Fermi arcs stemming from the projections of the Weyl nodes
in the surface [52]. In this work, we perform an extensive study of the
topological properties as a function of its crystal symmetries and also report
several Nodal line structures near the Fermi level that survive the addition
of SOC interaction. We also find novel drumhead surface states emerging from
the Nodal lines.

The presence of both Nodal lines and Weyl nodes near the Fermi level, along
with the big size of pure crystals and ferromagnetic nature of the pyrite,
make CoS2 a promising platform to probe topological phenomena in an accessible
material that can be used in all sorts of electronic applications, from new
magnetic memories to spin injector junctions, in which the spin polarized
Fermi arcs will contribute to reduce the weight of the minority spin pocket
[53].

This work is structured as follows: First we study the symmetry of the crystal
and electronic structures. Next, we analyze the symmetry enforced 4-fold
degeneracy arising at the $M=(\frac{1}{2},\frac{1}{2},0)$ point and the effect
of spin-orbit coupling on it. Then, we study the Nodal lines and Weyl nodes in
CoS2. Finally, we summarize the results of this work.

##  2 Results and discussion

CoS2 crystallizes in SG Pa$\bar{3}$ (205). This is a non-symmorphic cubic
Space Group, generated by {I$|$0}, {C${}_{2z}|\frac{1}{2}0\frac{1}{2}$} and
{C${}_{31}|$0}, whose combination produces glide symmetry planes,
$\\{m_{x}|1/2,1/2,0\\}$, $\\{m_{y}|0,1/2,1/2\\}$ and $\\{m_{z}|1/2,0,1/2\\}$.
Cobalt atoms sit in the 4a Wyckoff position, while sulfur atoms sit in the 8c
Wyckoff position. The structure is shown in Fig. 1(a). Experimental
measurements [54, 55] reveal that the system has ferromagnetic ordering, which
mainly stems from the cobalt d orbitals. This is represented by red arrows on
cobalt atoms in Fig. 1(a). Experimental measurements determine that the
magnetization is on the (100) direction family. We choose the direction of the
magnetization in the z direction, without loss of generality.

When taking into account magnetization, the symmetry of the system is lowered.
The 3-fold axes are broken, as well as TRS. The only surviving unitary
symmetries are the 2-fold screw {C${}_{2z}|\frac{1}{2}0\frac{1}{2}$}, the
glide $\\{m_{z}|1/2,0,1/2\\}$ and inversion symmetry. The screws in the x and
y directions, as well as the glides orthogonal to $x$ and $y$ axis are
preserved in combination with TRS. All these properties are summarized in its
magnetic space group, which is Pb’c’a (No. 61.436) [56, 57, 58].

To study the electronic structure, we performed density functional theory
(DFT) calculations as implemented in the Vienna ab initio simulation package
(VASP) [59, 60, 61, 62]. As it was shown in previous studies [52, 63], the
appropriate method to compute the electronic structure is the local spin
density approximation (LSDA) with no Hubbard U correction [51], with the
Dudarev simplified exchange correlation term, together with PAW
pseudopotentials. We used a grid of 7x7x7 k-points for the self-consistent
calculation.

The resulting band structures, with and without the effect of spin-orbit
coupling are shown in Fig. 1(b) and Fig. 1(c). Notice that the bands below the
Fermi level are polarized mainly in the direction of the majority spin (up,
following the magnetization), with a small minority pocket at the $R$ point,
as experimentally demonstrated in Ref. [52].

Without SOC in a ferromagnet, the Schrödinger equation can be decoupled into
spin up and spin down sectors where spin is labeled in the quantization axis
of the ferromagnet. In one of the spin sectors, the symmetry thus appears to
be the non-magnetic one Pa$\bar{3}$ (205), regardless of the orientation of
the magnetization. This artifice breaks down when SOC is included. Since SOC
interaction has small effect on the band structure (see Fig. 1(b) and Fig.
1(c)), we expect the symmetry breaking effects to be small. In the band
structure plots shown in Fig. 1(b) and Fig. 1(c), SOC couples both spins and
opens gaps when spin-up and spin-down bands meet (see light green circles). In
addition, the 3-fold degeneracy at the $\Gamma$ point in the SOC-free plot is
lifted when including SOC (see dark green circles). As expected, the gaps are
small.

Figure 1: (a) Crystal and magnetic structure of CoS2. Electronic band
structures near the Fermi level without (b) and with (c) SOC interaction
included. Light green, dark green and blue circles enclosed gapless/gapped
band crossing owing to SOC interaction mixing spins, 3-fold/2-fold degeneracy
w/o and with SOC interaction included and the symmetry enforced 4-fold
degeneracy, respectively.

In Fig. 1(b) we observe a 4-fold degeneracy near the Fermi level in k-point
$M=(\frac{1}{2},\frac{1}{2},0)$ (see blue circles), which survives the
addition of SOC as shown in Fig. 1(c). Since we are interested in band
crossings close to the Fermi level, we analyse this degeneracy. It is a
symmetry-protected degeneracy, following the 4-dimensional irreducible co-
representation at the $M$ point. Following Group Theory tools, we can
construct the most general, symmetry allowed, low-energy Hamiltonian that
describes the 4-fold degeneracy in the vicinity of the $M$ point. The
resulting $\bf k\cdot p$ Hamiltonian at first order in momentum and
magnetization111We include the magnetization as a finite parameter in the
expansion, because we will be interested in computing the model on the other
$M$ points later on. Then, we include a general magnetization
$(B_{x},B_{y},B_{z})$, even though only $B_{z}$ is non-zero in our case. reads

$H(\bm{k})=B_{x}\bm{f\cdot\sigma}\otimes\tau_{3}+k_{x}v_{x}B_{z}\sigma_{0}\otimes\tau_{1}+k_{y}\bm{v_{y}\cdot\sigma}\otimes\tau_{1}+k_{z}v_{z}\sigma_{0}\otimes\tau_{3},$
(1)

where momentum $\bm{k}=(k_{x},k_{y},k_{z})$ is measured from $M$,
$\sigma,\,\tau$ are Pauli matrices, $B_{x}$ is the magnetic field in the x
direction and $\bm{f},v_{x},\bm{v_{y}},v_{z}$ are undetermined constants. This
resembles the Hamiltonian of a Dirac fermion, but there are two main
differences: first, the momentum-independent
$B_{x}\bm{f\cdot\sigma}\otimes\tau_{3}$ term, which gaps the 4-fold when
$B_{x}\neq 0$ and second that both $k_{x}$ and $k_{y}$ go with $\tau_{1}$ and
there is no $\tau_{2}$, so the resulting energy dispersion does not result in
a 3D cone:

$E(\bm{k})=\pm\sqrt{(B_{x}f)^{2}+(v_{x}B_{z}k_{x}+v_{y}k_{y})^{2}+(v_{z}k_{z})^{2}}$
(2)

where $f=\sqrt{\bm{f}^{2}}$ and $v_{y}=\sqrt{\bm{v}_{y}^{2}}$. To see that
this is not a cone, we can do a rotation of coordinates such that
$k_{1}=v_{x}B_{z}k_{x}+v_{y}k_{y}$ and $k_{2}$ is orthogonal to $k_{1}$ and
lies in the $k_{z}=0$ plane. In that case, the resulting energy dispersion
would be cone-like in $k_{1},\,k_{z}$ but completely flat in $k_{2}$. Thus, we
cannot compute topological quantities such as the Chern number of half-filled
bands, because they are completely degenerate in the $k_{2}$ direction.

Furthermore, we can rotate the model to find the corresponding $\bf k\cdot p$
model in the other M points, $M_{2}=(\frac{1}{2},0,\frac{1}{2})$ and
$M_{3}=(0,\frac{1}{2},\frac{1}{2})$. $\\{C_{31}|0\\}$ symmetry transforms
momentum and magnetization in the same way,
$C_{31}(k_{x},k_{y},k_{z})=(k_{z},k_{x},k_{y})$ and
$C_{31}(B_{x},B_{y},B_{z})=(B_{z},B_{x},B_{y})$. Then,

$\displaystyle\hfill
C_{31}M_{1}=C_{31}(\frac{1}{2},\frac{1}{2},0)=(0,\frac{1}{2},\frac{1}{2})=M_{3}$
$\displaystyle\hfill C_{31}B_{x}=C_{31}(B,0,0)=(0,B,0)=B_{y}$
$\displaystyle\hfill
C_{31}M_{3}=C_{31}(0,\frac{1}{2},\frac{1}{2})=(\frac{1}{2},0,\frac{1}{2})=M_{2}$
$\displaystyle\hfill C_{31}B_{y}=C_{31}(0,B,0)=(0,0,B)=B_{z}.$ (3)

Notice that the momentum independent term that goes with $B_{x}$ in $M_{1}$
will go with $B_{z}$ in $M_{2}$. Since we have chosen the direction of the
magnetization in the $z$ axis, we predict that the 4-fold will gap in the
$M_{2}$ point. We confirmed this by computing the energy bands in the 3
$M_{i}$ points (see Fig. 2(a)) and we found two of them keep the band
crossings ($M_{1},M_{3})$ and the other one has a gap of 5.5 meV, as shown in
Fig. 2(b). The size of the gap is directly related to the energy scale of the
SOC interaction. Since SOC effect is small, the size of the gap must be small
too.

Figure 2: (a) Brillouin zone of the MSG Pb’c’a (No. 61.436) depicting the
position of nonequivalent $M_{i}$ points (extracted from BCS [56, 57, 58]).
(b) Dispersion from $\Gamma$ to the different $M_{i}$ points. The red arrow
indicates the gap opening in $M_{2}$, as predicted by the $\bf k\cdot p$
model.

Apart from 4-fold degeneracy at the $M_{1}$ and $M_{3}$ points, we also found
that the planes $k_{x}=\pm 0.5$ and $k_{y}=\pm 0.5$ only have 2-dimensional
magnetic irreducible co-representations. This implies that the bands will
always be 2-fold degenerate at those planes, thus, they are Nodal planes.
However, due to the presence of inversion symmetry, they will not have
topological charge [45].

Figure 3: (a) Position of the Nodal lines in the BZ. Each red rectangle
represents an integrated Berry phase of $\pi$. (b) Projection of the Nodal
lines in the (001) direction. (c) and (d) Berry phase calculations on the
paths depicted in (b). (e) Surface calculation in the (001) cleavage plane,
computed along the red path shown in (b). Notice that the drumhead surface
states gap in regions where the Berry phase is equal to $0$. Figure 4: (a)
Position of the Weyl nodes in the BZ. (b) Surface calculation in the (100)
cleavage plane in the path depicted in (d). (c) Zoomed-in schematic depiction
of the surface states in the (100) cleavage plane near the
$\bar{M}_{2}$-$\bar{R}$ line. The location of the projection of the Weyls into
the surface is shown by red and blue dots. Dots with a surrounding circle
represent the projection of two Weyl nodes with equal chirality. (d) Spin
polarization of Fermi arcs in the surface calculation. They are polarized in
the direction of the majority spin, just like the bulk bands.

We now study the topological properties of CoS2 focusing on both Weyl nodes
and Nodal lines. The calculations in what follows are based on the
interpolated Tight Binding model in the Wannier basis [64] constructed from
VASP ab-initio calculation [59, 60, 61, 62]. We performed Berry phase
calculations, as well as surface calculations as implemented in WannierTools
[65].

We focus on the crossings between the last two valence bands near the Fermi
level (the two top bands polarized in the majority spin in Fig. 1(c)). We
found two nodal lines, one in the plane $k_{z}=0$ and another one in
$k_{z}=\pi$. Both planes are left invariant by $\\{m_{z}|1/2,0,1/2\\}$ (see
Fig. 3(a)), which is the symmetry that protects both Nodal lines. In order to
characterize the topological protection of the Nodal line, we computed the
Berry phase around a loop enclosing it. The location of the Berry phase
integration paths (red rectangles) is shown Fig. 3(a). Each red rectangle
represents an integrated Berry phase of $\pi$, thus, non-trivial $Z_{2}$
index. We conclude that the Nodal lines in both planes are topologically
protected.

In Fig. 3(b) we show the projection of both Nodal lines in the (001)
direction. We showed that the Nodal lines are topologically protected,
however, which region of the $k_{z}=0$ plane will host surface states depends
on the choice of surface termination [66]. We fix our unit cell as shown in
Fig. 1(a) and compute the Berry phases (integrating on $k_{z}$) along the
momentum paths depicted in Fig. 3(b). The results of the calculation are shown
in Fig. 3(c) and Fig. 3(d). The calculations show that the Berry phase inside
the blue Nodal line is equal to $\pi$, while the Berry phase outside, or
inside the projection of both Nodal lines is equal to 0. Thus, we will only
find drumhead states in the region where nodal lines do not overlap and the
Berry phase is equal to $\pi$. Berry phase and surface spectrum calculations
shown in Fig 3 (d) and Fig. 3(e) respectively, have been performed in the same
momentum path, as depicted in Fig. 3(b). We choose the surface termination to
coincide with the periodic repetition of the unit cell. We observe that
drumhead surface states survive only in regions where the Berry phase of Fig.
3(d) is equal to $\pi$, while they gap in regions where it is equal to 0.

By carefully examining the region close to the Fermi level we found a total of
8 Weyl points close to the Fermi level, whose location in momentum space is
displayed in Fig. 4(a). In Fig. 4(b), we show (100) surface spectrum for
$k_{y}=0$, along the momentum path depicted in Fig. 4(d) in green. In this
path, we cross the projection of 4 Weyl nodes, projected pairwise with the
same chirality. We can see two bright surface states that connect the
projection of the Weyl nodes, one within the BZ and the other going through
the boundary. Notice that Fermi arcs connect Weyl nodes across the boundary of
the BZ but they avoid the Nodal planes, which are located at $k_{y}=\pm\pi$.
In the (100) cleavage plane, we can observe 4 different Fermi arcs as depicted
in Fig. 4(c). The Fermi arc close to the $\bar{R}$ point connects the
projection of two opposite chirality Weyl nodes. Near the $\bar{M}$ points,
though, we can see two Fermi arcs connecting the projections, instead of one.
This is due to pairs of Weyl nodes (with the same chirality) being projected
in the same point. In Fig. 4(d), we see the Fermi surface calculation on the
(100) surface, schematized in Fig. 4(c). The connection of the Fermi arcs is
the same one as in the scheme of Fig. 4(c), with single Weyl nodes connected
by a single Fermi arc and double Weyl nodes connected by two Fermi arcs. From
the ab initio calculation we know that the bands in the relevant energy window
are predominantly polarized in the direction of the majority spin. We computed
the spin polarization of the Fermi arcs to see if they inherit the
polarization from the bulk. We show in Fig 4 (d) that the Fermi arcs are
completely polarized in the direction of the majority spin, as is the case in
the bulk.

Our results have implications in a wide range of fields. In Quantum Optics, it
has been shown that Weyl semimetals could be used to realize a Veselago lens
for electrons [67] (negative refractive index). It has also been confirmed
experimentally that Weyl semimetals have a quantized circular photogalvanic
effect [68, 69] due to their topological charge. In the field of electronics,
Weyl semimetals show signatures of the chiral anomaly [70]. There are
materials that show a large magnetorresistance, such as the family of TaAs
[9], which can be harnessed for the next generation of memory devices. The
spin polarization of the Fermi arcs can also influence the performance of a
Weyl semimetal as a spin injector, improving the expected bulk results [71].

##  3 Conclusions

In this work we revisited the recently discovered Weyl semimetal CoS2
ferromagnetic pyrite. We analyzed both Nodal lines and Weyl nodes found on the
ferromagnetic system. We found two nodal lines close to the Fermi level, with
surface drumhead states in the (001) surface. We also found a family of Weyl
nodes close to the Fermi level, with their distinctive Fermi arcs on the (100)
surface. These surface states increase the number of available electrons in
the surface, thus improving its metallic properties for junctions. Following
the spin polarization in the direction of the majority in the bulk, we checked
if the Fermi arcs are polarized in the same direction too. We answer this
question in the affirmative, adding this material to the family of spin
polarized Fermi arcs in real materials [72]. The Fermi arcs would contribute
to the spin majority polarization, making it an interesting material for
spintronics, like a possible spin injector [73].

##  4 Acknowledgements

M.G.V. and I.R. acknowledge the Spanish Ministerio de Ciencia e Innovacion
(grant number PID2019109905GB-C21) and Programa Red Guipuzcoana de Ciencia,
Tecnología e Innovación 2021 No. 2021-CIEN-000070-01 Gipuzkoa Next. A.R.S. is
grateful to PASPA-DGAPA-UNAM for the sabbatical scholarship and to the DIPC
for their support during the research stay. A.B. acknowledges financial
support from the Spanish Ministry of Science and Innovation
(PID2019-105488GBI00). N.B.M.S. was supported by Microsoft. This work was
supported by the Gordon and Betty Moore Foundation through Grant GBMF9064 to
L.M.S.

## References

## References

  * [1] Chiu C K, Teo J C Y, Schnyder A P and Ryu S 2016 Rev. Mod. Phys. 88(3) 035005 URL https://link.aps.org/doi/10.1103/RevModPhys.88.035005
  * [2] Armitage N P, Mele E J and Vishwanath A 2018 Rev. Mod. Phys. 90(1) 015001 URL https://link.aps.org/doi/10.1103/RevModPhys.90.015001
  * [3] Burkov A 2018 Annual Review of Condensed Matter Physics 9 359–378 (Preprint https://doi.org/10.1146/annurev-conmatphys-033117-054129) URL https://doi.org/10.1146/annurev-conmatphys-033117-054129
  * [4] Yan B and Felser C 2017 Annual Review of Condensed Matter Physics 8 337–354 (Preprint https://doi.org/10.1146/annurev-conmatphys-031016-025458) URL https://doi.org/10.1146/annurev-conmatphys-031016-025458
  * [5] Vergniory M G, Elcoro L, Felser C, Regnault N, Bernevig B A and Wang Z 2019 Nature 566 480–485
  * [6] Xu Y, Elcoro L, Song Z D, Wieder B J, Vergniory M, Regnault N, Chen Y, Felser C and Bernevig B A 2020 Nature 586 702–707
  * [7] Weng H, Fang C, Fang Z, Bernevig B A and Dai X 2015 Phys. Rev. X 5(1) 011029 URL https://link.aps.org/doi/10.1103/PhysRevX.5.011029
  * [8] Sun Y, Wu S C, Ali M N, Felser C and Yan B 2015 Phys. Rev. B 92(16) 161107 URL https://link.aps.org/doi/10.1103/PhysRevB.92.161107
  * [9] Lv B Q, Weng H M, Fu B B, Wang X P, Miao H, Ma J, Richard P, Huang X C, Zhao L X, Chen G F, Fang Z, Dai X, Qian T and Ding H 2015 Phys. Rev. X 5(3) 031013 URL https://link.aps.org/doi/10.1103/PhysRevX.5.031013
  * [10] Yang L X, Liu Z K, Sun Y, Peng H, Yang H F, Zhang T, Zhou B, Zhang Y, Guo Y F, Rahn M, Prabhakaran D, Hussain Z, Mo S K, Felser C, Yan B and Chen Y L 2015 Nature Physics 11 728–732 ISSN 1745-2481 URL https://doi.org/10.1038/nphys3425
  * [11] Xu S Y, Belopolski I, Alidoust N, Neupane M, Bian G, Zhang C, Sankar R, Chang G, Yuan Z, Lee C C, Huang S M, Zheng H, Ma J, Sanchez D S, Wang B, Bansil A, Chou F, Shibayev P P, Lin H, Jia S and Hasan M Z 2015 Science 349 613–617 ISSN 0036-8075 (Preprint https://science.sciencemag.org/content/349/6248/613.full.pdf) URL https://science.sciencemag.org/content/349/6248/613
  * [12] Wan X, Turner A M, Vishwanath A and Savrasov S Y 2011 Phys. Rev. B 83(20) 205101 URL https://link.aps.org/doi/10.1103/PhysRevB.83.205101
  * [13] Xu G, Weng H, Wang Z, Dai X and Fang Z 2011 Phys. Rev. Lett. 107(18) 186806 URL https://link.aps.org/doi/10.1103/PhysRevLett.107.186806
  * [14] Wang Z, Vergniory M G, Kushwaha S, Hirschberger M, Chulkov E V, Ernst A, Ong N P, Cava R J and Bernevig B A 2016 Phys. Rev. Lett. 117(23) 236401 URL https://link.aps.org/doi/10.1103/PhysRevLett.117.236401
  * [15] Soh J R, de Juan F, Vergniory M G, Schröter N B M, Rahn M C, Yan D Y, Jiang J, Bristow M, Reiss P, Blandy J N, Guo Y F, Shi Y G, Kim T K, McCollam A, Simon S H, Chen Y, Coldea A I and Boothroyd A T 2019 Phys. Rev. B 100(20) 201102 URL https://link.aps.org/doi/10.1103/PhysRevB.100.201102
  * [16] Morali N, Batabyal R, Nag P K, Liu E, Xu Q, Sun Y, Yan B, Felser C, Avraham N and Beidenkopf H 2019 Science 365 1286–1291 ISSN 0036-8075 (Preprint https://science.sciencemag.org/content/365/6459/1286.full.pdf) URL https://science.sciencemag.org/content/365/6459/1286
  * [17] Liu D F, Liang A J, Liu E K, Xu Q N, Li Y W, Chen C, Pei D, Shi W J, Mo S K, Dudin P, Kim T, Cacho C, Li G, Sun Y, Yang L X, Liu Z K, Parkin S S P, Felser C and Chen Y L 2019 Science 365 1282–1285 ISSN 0036-8075 (Preprint https://science.sciencemag.org/content/365/6459/1282.full.pdf) URL https://science.sciencemag.org/content/365/6459/1282
  * [18] Belopolski I, Manna K, Sanchez D S, Chang G, Ernst B, Yin J, Zhang S S, Cochran T, Shumiya N, Zheng H, Singh B, Bian G, Multer D, Litskevich M, Zhou X, Huang S M, Wang B, Chang T R, Xu S Y, Bansil A, Felser C, Lin H and Hasan M Z 2019 Science 365 1278–1281 ISSN 0036-8075 (Preprint https://science.sciencemag.org/content/365/6459/1278.full.pdf) URL https://science.sciencemag.org/content/365/6459/1278
  * [19] Liu C X, Zhang S C and Qi X L 2016 Annual Review of Condensed Matter Physics 7 301–321 (Preprint https://doi.org/10.1146/annurev-conmatphys-031115-011417) URL https://doi.org/10.1146/annurev-conmatphys-031115-011417
  * [20] Adler S L 1969 Phys. Rev. 177(5) 2426–2438 URL https://link.aps.org/doi/10.1103/PhysRev.177.2426
  * [21] Bradlyn B, Cano J, Wang Z, Vergniory M G, Felser C, Cava R J and Bernevig B A 2016 Science 353 ISSN 0036-8075 (Preprint http://science.sciencemag.org/content/353/6299/aaf5037.full.pdf) URL http://science.sciencemag.org/content/353/6299/aaf5037
  * [22] Schröter N B M, Pei D, Vergniory M G, Sun Y, Manna K, de Juan F, Krieger J A, Süss V, Schmidt M, Dudin P, Bradlyn B, Kim T K, Schmitt T, Cacho C, Felser C, Strocov V N and Chen Y 2019 Nature Physics 15 759–765 ISSN 1745-2481 URL https://doi.org/10.1038/s41567-019-0511-y
  * [23] Xu X, Wang X, Cochran T A, Sanchez D S, Chang G, Belopolski I, Wang G, Liu Y, Tien H J, Gui X, Xie W, Hasan M Z, Chang T R and Jia S 2019 Phys. Rev. B 100(4) 045104 URL https://link.aps.org/doi/10.1103/PhysRevB.100.045104
  * [24] Sanchez D S, Belopolski I, Cochran T A, Xu X, Yin J X, Chang G, Xie W, Manna K, Süß V, Huang C Y, Alidoust N, Multer D, Zhang S S, Shumiya N, Wang X, Wang G Q, Chang T R, Felser C, Xu S Y, Jia S, Lin H and Hasan M Z 2019 Nature 567 500–505 ISSN 1476-4687 URL https://doi.org/10.1038/s41586-019-1037-2
  * [25] Takane D, Wang Z, Souma S, Nakayama K, Nakamura T, Oinuma H, Nakata Y, Iwasawa H, Cacho C, Kim T, Horiba K, Kumigashira H, Takahashi T, Ando Y and Sato T 2019 Phys. Rev. Lett. 122(7) 076402 URL https://link.aps.org/doi/10.1103/PhysRevLett.122.076402
  * [26] Kumar N, Yao M, Nayak J, Vergniory M G, Bannies J, Wang Z, Schröter N B M, Strocov V N, Müchler L, Shi W, Rienks E D L, Mañes J L, Shekhar C, Parkin S S P, Fink J, Fecher G H, Sun Y, Bernevig B A and Felser C 2020 Advanced Materials 32 1906046 (Preprint https://onlinelibrary.wiley.com/doi/pdf/10.1002/adma.201906046) URL https://onlinelibrary.wiley.com/doi/abs/10.1002/adma.201906046
  * [27] Schröter N B M, Stolz S, Manna K, de Juan F, Vergniory M G, Krieger J A, Pei D, Schmitt T, Dudin P, Kim T K and et al 2020 Science 369 179–183 ISSN 1095-9203 URL http://dx.doi.org/10.1126/science.aaz3480
  * [28] Vergniory M G, Elcoro L, Orlandi F, Balke B, Chan Y H, Nuss J, Schnyder A P and Schoop L M 2018 The European Physical Journal B 91 213 ISSN 1434-6036 URL https://doi.org/10.1140/epjb/e2018-90302-7
  * [29] Schoop L M, Topp A, Lippmann J, Orlandi F, Müchler L, Vergniory M G, Sun Y, Rost A W, Duppel V, Krivenkov M, Sheoran S, Manuel P, Varykhalov A, Yan B, Kremer R K, Ast C R and Lotsch B V 2018 Science Advances 4 eaar2317
  * [30] Chiu C K and Schnyder A P 2014 Phys. Rev. B 90(20) 205136 URL https://link.aps.org/doi/10.1103/PhysRevB.90.205136
  * [31] Yang B J, Bojesen T A, Morimoto T and Furusaki A 2017 Phys. Rev. B 95(7) 075135 URL https://link.aps.org/doi/10.1103/PhysRevB.95.075135
  * [32] Bian G, Chang T R, Sankar R, Xu S Y, Zheng H, Neupert T, Chiu C K, Huang S M, Chang G, Belopolski I, Sanchez D S, Neupane M, Alidoust N, Liu C, Wang B, Lee C C, Jeng H T, Bansil A, Chou F, Lin H and Zahid Hasan M 2015 arXiv e-prints arXiv:1505.03069 (Preprint 1505.03069)
  * [33] Yamakage A, Yamakawa Y, Tanaka Y and Okamoto Y 2016 Journal of the Physical Society of Japan 85 013708 (Preprint https://doi.org/10.7566/JPSJ.85.013708) URL https://doi.org/10.7566/JPSJ.85.013708
  * [34] Xie L S, Schoop L M, Seibel E M, Gibson Q D, Xie W and Cava R J 2015 APL Materials 3 083602 (Preprint https://doi.org/10.1063/1.4926545) URL https://doi.org/10.1063/1.4926545
  * [35] Kim Y, Wieder B J, Kane C L and Rappe A M 2015 Phys. Rev. Lett. 115(3) 036806 URL https://link.aps.org/doi/10.1103/PhysRevLett.115.036806
  * [36] Zeng M, Fang C, Chang G, Chen Y A, Hsieh T, Bansil A, Lin H and Fu L 2015 arXiv e-prints arXiv:1504.03492 (Preprint 1504.03492)
  * [37] Huang H, Liu J, Vanderbilt D and Duan W 2016 Phys. Rev. B 93(20) 201114 URL https://link.aps.org/doi/10.1103/PhysRevB.93.201114
  * [38] Fang C, Chen Y, Kee H Y and Fu L 2015 Phys. Rev. B 92(8) 081201 URL https://link.aps.org/doi/10.1103/PhysRevB.92.081201
  * [39] Bian G, Chang T R, Zheng H, Velury S, Xu S Y, Neupert T, Chiu C K, Huang S M, Sanchez D S, Belopolski I, Alidoust N, Chen P J, Chang G, Bansil A, Jeng H T, Lin H and Hasan M Z 2016 Phys Rev B 93 121113 (Preprint 1508.07521)
  * [40] Weng H, Liang Y, Xu Q, Yu R, Fang Z, Dai X and Kawazoe Y 2015 Phys. Rev. B 92(4) 045108 URL https://link.aps.org/doi/10.1103/PhysRevB.92.045108
  * [41] Yu R, Weng H, Fang Z, Dai X and Hu X 2015 Phys. Rev. Lett. 115(3) 036807 URL https://link.aps.org/doi/10.1103/PhysRevLett.115.036807
  * [42] Muechler L, Topp A, Queiroz R, Krivenkov M, Varykhalov A, Cano J, Ast C R and Schoop L M 2020 Phys. Rev. X 10(1) 011026 URL https://link.aps.org/doi/10.1103/PhysRevX.10.011026
  * [43] Bian G, Chang T R, Sankar R, Xu S Y, Zheng H, Neupert T, Chiu C K, Huang S M, Chang G, Belopolski I, Sanchez D S, Neupane M, Alidoust N, Liu C, Wang B, Lee C C, Jeng H T, Zhang C, Yuan Z, Jia S, Bansil A, Chou F, Lin H and Hasan M Z 2016 Nature Communications 7 10556 ISSN 2041-1723 URL https://doi.org/10.1038/ncomms10556
  * [44] Hou W, Liu J, Zuo X, Xu J, Zhang X, Liu D, Zhao M, Zhu Z G, Luo H G and Zhao W 2021 npj Computational Materials 7 37 ISSN 2057-3960 URL https://doi.org/10.1038/s41524-021-00504-w
  * [45] Wilde M A, Dodenhöft M, Niedermayr A, Bauer A, Hirschmann M M, Alpin K, Schnyder A P and Pfleiderer C 2021 Nature 594 374–379 ISSN 1476-4687 URL https://doi.org/10.1038/s41586-021-03543-x
  * [46] Xiao M, Ye L, Qiu C, He H, Liu Z and Fan S 2020 Science Advances 6 (Preprint https://advances.sciencemag.org/content/6/8/eaav2360.full.pdf) URL https://advances.sciencemag.org/content/6/8/eaav2360
  * [47] Wu P F, Shi j B, Cheng B, Lin H and Lee H 2019 Materials Science Forum 947 91–95
  * [48] Paul F 1945 American Mineralogist 30 483–497
  * [49] Matthias B, Clogston A, Williams H, Corenzwit E and Sherwood R 1961 Physical Review Letters 7 7
  * [50] Matthias B and Bozorth R 1958 Physical Review 109 604
  * [51] Kwon S K, Youn S J and Min B I 2000 Phys. Rev. B 62(1) 357–360 URL https://link.aps.org/doi/10.1103/PhysRevB.62.357
  * [52] Schröter N B M, Robredo I, Klemenz S, Kirby R J, Krieger J A, Pei D, Yu T, Stolz S, Schmitt T, Dudin P, Kim T K, Cacho C, Schnyder A, Bergara A, Strocov V N, de Juan F, Vergniory M G and Schoop L M 2020 Science Advances 6 (Preprint https://advances.sciencemag.org/content/6/51/eabd5000.full.pdf) URL https://advances.sciencemag.org/content/6/51/eabd5000
  * [53] Li P, Wu W, Wen Y, Zhang C, Zhang J, Zhang S, Yu Z, Yang S A, Manchon A and Zhang X x 2018 Nature Communications 9 3990 ISSN 2041-1723 URL https://doi.org/10.1038/s41467-018-06518-1
  * [54] Miyahara S and Teranishi T 1968 Journal of Applied Physics 39 896–897 (Preprint https://doi.org/10.1063/1.1656326) URL https://doi.org/10.1063/1.1656326
  * [55] Teruya A, Suzuki F, Aoki D, Honda F, Nakamura A, Nakashima M, Amako Y, Harima H, Uchima K, Hedo M, Nakama T and Yonuki 2017 Journal of Physics: Conference Series 807 012001 URL https://doi.org/10.1088/1742-6596/807/1/012001
  * [56] Perez-Mato J, Orobengoa D, Tasci E, De la Flor Martin G and Kirov A 2011 Bulgarian Chemical Communications 43 183–197
  * [57] Aroyo M I, Perez-Mato J M, Capillas C, Kroumova E, Ivantchev S, Madariaga G, Kirov A and Wondratschek H 01 Jan 2006 Zeitschrift für Kristallographie \- Crystalline Materials 221 15 – 27 URL https://www.degruyter.com/view/journals/zkri/221/1/article-p15.xml
  * [58] Aroyo M I, Kirov A, Capillas C, Perez-Mato J M and Wondratschek H 2006 Acta Crystallographica Section A 62 115–128 URL https://doi.org/10.1107/S0108767305040286
  * [59] Kresse G and Hafner J 1993 Phys. Rev. B 47(1) 558–561 URL https://link.aps.org/doi/10.1103/PhysRevB.47.558
  * [60] Kresse G and Hafner J 1994 Phys. Rev. B 49(20) 14251–14269 URL https://link.aps.org/doi/10.1103/PhysRevB.49.14251
  * [61] Kresse G and Furthmüller J 1996 Computational Materials Science 6 15–50 ISSN 0927-0256 URL https://www.sciencedirect.com/science/article/pii/0927025696000080
  * [62] Kresse G and Furthmüller J 1996 Phys. Rev. B 54(16) 11169–11186 URL https://link.aps.org/doi/10.1103/PhysRevB.54.11169
  * [63] Brown P J, Neumann K U, Simon A, Ueno F and Ziebeck K R A 2005 Journal of Physics: Condensed Matter 17 1583–1592 URL https://doi.org/10.1088/0953-8984/17/10/013
  * [64] Pizzi G, Vitale V, Arita R, Blügel S, Freimuth F, Géranton G, Gibertini M, Gresch D, Johnson C, Koretsune T, Ibañez-Azpiroz J, Lee H, Lihm J M, Marchand D, Marrazzo A, Mokrousov Y, Mustafa J I, Nohara Y, Nomura Y, Paulatto L, Poncé S, Ponweiser T, Qiao J, Thöle F, Tsirkin S S, Wierzbowska M, Marzari N, Vanderbilt D, Souza I, Mostofi A A and Yates J R 2020 Journal of Physics: Condensed Matter 32 165902 URL https://doi.org/10.1088%2F1361-648x%2Fab51ff
  * [65] Wu Q, Zhang S, Song H F, Troyer M and Soluyanov A A 2018 Computer Physics Communications 224 405 – 416 ISSN 0010-4655 URL http://www.sciencedirect.com/science/article/pii/S0010465517303442
  * [66] Naumov I I and Hemley R J 2016 Phys. Rev. Lett. 117(20) 206403 URL https://link.aps.org/doi/10.1103/PhysRevLett.117.206403
  * [67] Hills R D Y, Kusmartseva A and Kusmartsev F V 2017 Phys. Rev. B 95(21) 214103 URL https://link.aps.org/doi/10.1103/PhysRevB.95.214103
  * [68] de Juan F, Grushin A G, Morimoto T and Moore J E 2017 Nature Communications 8 15995 ISSN 2041-1723 URL https://doi.org/10.1038/ncomms15995
  * [69] Rees D, Manna K, Lu B, Morimoto T, Borrmann H, Felser C, Moore J E, Torchinsky D H and Orenstein J 2020 Science Advances 6 eaba0509
  * [70] Hirschberger M, Kushwaha S, Wang Z, Gibson Q, Liang S, Belvin C, Bernevig B A, Cava R J and Ong N P 2016 Nature Materials 15 1161–1165 ISSN 1476-4660 URL https://doi.org/10.1038/nmat4684
  * [71] Zhao B, Karpiak B, Khokhriakov D, Johansson A, Hoque A M, Xu X, Jiang Y, Mertig I and Dash S P 2020 Advanced Materials 32 2000818 (Preprint https://onlinelibrary.wiley.com/doi/pdf/10.1002/adma.202000818) URL https://onlinelibrary.wiley.com/doi/abs/10.1002/adma.202000818
  * [72] Xu S Y, Belopolski I, Sanchez D S, Neupane M, Chang G, Yaji K, Yuan Z, Zhang C, Kuroda K, Bian G, Guo C, Lu H, Chang T R, Alidoust N, Zheng H, Lee C C, Huang S M, Hsu C H, Jeng H T, Bansil A, Neupert T, Komori F, Kondo T, Shin S, Lin H, Jia S and Hasan M Z 2016 Phys. Rev. Lett. 116(9) 096801 URL https://link.aps.org/doi/10.1103/PhysRevLett.116.096801
  * [73] Huang Z, Lane C, Cao C, Zhi G X, Liu Y, Matt C E, Kuthanazhi B, Canfield P C, Yarotski D, Taylor A J and Zhu J X 2020 Phys. Rev. B 102(23) 235167 URL https://link.aps.org/doi/10.1103/PhysRevB.102.235167


# Just Another Method to Compute MTTF from Continuous Time Markov Chain Models

Eduardo M. Vasconcelos
Academic Department of Superior Courses
Federal Institute of Pernambuco
Recife, PE 50740-540
<EMAIL_ADDRESS>

###### Abstract

The Meantime to Failure is a statistic used to determine how much time a
system spends to enter one of its absorption states. This statistic can be
used in most areas of knowledge. In engineering, for example, can be used as a
measure of equipment reliability, and in business, as a measure of processes
performance. This work presents a method to obtain the Meantime to Failure
from a Continuous Time Markov Chain models. The method is intuitive and is
simpler to be implemented, since, it consists of solving a system of linear
equations.

_Keywords_ Meantime to Failure, Continuous Time Markov Chain, Stochastic
Process, Business Processes

##  1 Introduction

The Meantime To Failure (MTTF) is a statistic used for system analysis in
several knowledge areas. This value represents the average time to the system
enters into one of the possible states of fault, without considering system
repairs. Although MTTF be considered to analyze systems with fault states, it
also can be used to perform analysis on processes, since it can be used to
represent the meantime to one process finishes, given that, processes can be
represented by state machine models.

This work presents a method to compute MTTF from Continuous Time Markov Chain
(CTMC) models. There are no arguments that demonstrate that this method
performs better than other methods, but this method has a simpler
implementation and is intuitive. This method also allows computing the
absorption probabilities and the average holding time of each state without
additional steps.

##  2 Calculating the Meantime to Failure

This section presents a method to compute the MTTF from a CTMC model with
absorption states. This method consists in generating a secondary model from
the original, adding on it unitary repairs.

To extract the MTTF from a CTMC model, it´s necessary that this model
accomplishes two conditions: (i) the model need to have a initial state
$s_{\iota}$; (ii) the model need to have at last one absorption state. Let´s
consider a CTMC $Z=\\{S,T\\}$, with $Z$ containing a set of states
$S=\\{s_{1},s_{2},...,s_{n}\\}$ and a set of transitions $T=\\{t_{i,j}\\}$
with $t_{i,j}=1$ if there is a transition between states $s_{i}$ and $s_{j}$,
and $t_{i,j}=0$, otherwise. A state is considered an absorption state $s_{f}$
if $t_{f,k}=0,\forall k$. The occurrence rate of a transition is $q_{i,j}>0$
if $t_{i,j}=1$ and $q_{i,j}=0$ if $t_{i,j}=0$, with
$q_{i,i}=-\sum{q_{i,\beta}}$ and $\beta\neq i$.

To compute the MTTF we need to create a secondary model
$Z^{\prime}=\\{[S_{A},S_{F}],[T,T^{\prime}]\\}$, with $S_{A}$ the set of
reachable states and the initial state and $S_{F}$ the set of fault states.
After creating $Z^{\prime}$ we have to add repair transitions $T^{\prime}$
between states $S_{F}$ and $s_{\iota}$ with $q_{f,\iota}=\mu$ as showed on
Figure 1. The figure presents $Z$ with the addiction of repair transitions
$T^{\prime}$ with rate $\mu$, which are represented by dashed arrows.

Figure 1: A General View of $Z^{\prime}$

From $Z^{\prime}$, which corresponds to $Z$ with addiction of $T^{\prime}$,
being $Z^{\prime}$ a closed chain, it´s possible to compute the steady-state
availability $A^{\prime}$. The process of computing the availability can be
seen in (Bolch et al., 1998). To compute the MTTF from $Z^{\prime}$ and
$A^{\prime}$, we can use the expression 1.

$A^{\prime}=\frac{MTTF}{MTTF+MTTR}$ (1)

Where MTTR corresponds to the Meantime to repair the system. As transitions
$T^{\prime}$ were artificially added on $Z^{\prime}$, the MTTR can be computed
according to equation 2.

$MTTR=\sum{P[sf_{u}]\mu^{-1}=\mu^{-1}}\sum{P[sf_{u}]}$ (2)

Where $P[.]$ is the probability of a fault state be reached at the end of the
process and $sf_{u}\in S^{\prime}_{F}$. As $\sum{P[sf_{u}]}=1$, given that
every fault states can be reached, $MTTR=\mu^{-1}$. So, we have:

$MTTF=-\frac{A^{\prime}\mu^{-1}}{A^{\prime}-1}=\frac{A^{\prime}\mu^{-1}}{U^{\prime}}$
(3)

Where $U^{\prime}$ is the unavailability of the system.

Although the MTTF of equation 3 be computed from the availability of
$Z^{\prime}$, this value corresponds to the MTTF of $Z$, since the set of
states and transitions of $Z^{\prime}$ are the same as the original. The
method to calculate the MTTF is easier to be implemented than the methods
presented in (Trivedi and Bobbio, 2017) since it´s just necessary to solve a
system of linear equations presented on equation 4 (Vasconcelos, 2017).

$\pi^{\prime}_{i}=\frac{\sum{\pi^{\prime}_{j}q_{j,i}t_{j,i}}}{\sum{q_{i,k}t_{i,k}}}$
(4)

From the MTTF and $A^{\prime}$ it is possible to compute other values of
interest to the analysis of $Z$. As $A^{\prime}=\sum{\pi^{\prime}_{a}}$ with
$\pi^{\prime}_{a}$ representing the steady probability of state $s_{a}\in
S_{A}$, and $U^{\prime}=1-A^{\prime}=\sum{\pi^{\prime}_{f}}$ with
$\pi^{\prime}_{f}$ as the steady probability of $s_{f}\in S_{F}$, we have the
following values: (i) $\tau_{a}=MTTF.(\pi^{\prime}_{a}A^{\prime-1})$, (ii)
$\rho_{f}=\pi^{\prime}_{f}U^{\prime-1}$. Where $\tau_{a}$ corresponding to the
average holding time of state $s_{a}$, $\rho_{f}$ corresponding to the
probability of $s_{f}$ be reached.

The value $\tau_{a}$ is obtained from a proportion of MTTF since the MTTF is
the sum of the average holding time of states $S_{A}$ as showed in (Trivedi
and Bobbio, 2017).

##  3 Conclusions

This work presented a method to compute the Meantime to Failure (MTTF) from
Continuous Time Markov Chain Models with absorption states. The method
consists of creating a second model and adding repair transitions to it. From
the secondary model, it is possible to calculate the steady-state availability
and, from this value, calculate the MTTF. Beyond the MTTF, it´s possible to
compute probabilities of the process reaching the fault states, as well as the
states holding time.

## References

  * Bolch et al. [1998] Gunter Bolch, Stefan Greiner, Hermann de Meer, and Kishor S. Trivedi.  _Queueing Networks and Markov Chains: Modeling and Performance Evaluation with Computer Science Applications_.  Wiley-Interscience, USA, 1998.  ISBN 0471193666.
  * Trivedi and Bobbio [2017] Kishor S. Trivedi and Andrea Bobbio.  _Continuous-Time Markov Chain: Reliability Models_ , page 357–422.  Cambridge University Press, 2017.  doi:10.1017/9781316163047.014.
  * Vasconcelos [2017] Eduardo M. Vasconcelos.  Steady state availability general equations of decision and sequential processes in continuous time markov chain models.  _CoRR_ , abs/1701.06415, 2017.  URL http://arxiv.org/abs/1701.06415.