2021 Full Paper – MIDL 2021 Steffen Czolbe<EMAIL_ADDRESS>
Oswin Krause<EMAIL_ADDRESS>
Department of Computer Science, University of Copenhagen, Denmark and Aasa
Feragen<EMAIL_ADDRESS>
DTU Compute, Technical University of Denmark, Denmark

# Semantic similarity metrics for learned image registration

###### Abstract

We propose a semantic similarity metric for image registration. Existing
metrics like Euclidean Distance or Normalized Cross-Correlation focus on
aligning intensity values, giving difficulties with low intensity contrast or
noise. Our approach learns dataset-specific features that drive the
optimization of a learning-based registration model. We train both an
unsupervised approach using an auto-encoder, and a semi-supervised approach
using supplemental segmentation data to extract semantic features for image
registration. Comparing to existing methods across multiple image modalities
and applications, we achieve consistently high registration accuracy. A
learned invariance to noise gives smoother transformations on low-quality
images. Code and experiments are available at
github.com/SteffenCzolbe/DeepSimRegistration.

###### keywords:

Image Registration, Deep Learning, Representation Learning

##  1 Introduction

Deformable registration, or nonlinear image alignment, is a fundamental tool
in medical imaging. Registration models find correspondences between a set of
images and derive a geometric transformation to align them. Most algorithmic
and deep learning-based methods solve the registration problem by the
minimization of a loss function, consisting of a similarity metric and a
regularization term ensuring smoothness of the transformation. The similarity
metric is essential to the optimization; it judges the quality of the match
between registered images and has a strong influence on the result.

Pixel-based similarity metrics like euclidean distance and patch-wise cross-
correlation are well explored within algorithmic [Avants et al. (2011), Avants
et al. (2008), Faisal Beg et al. (2005), Rueckert et al. (1999), Thirion
(1998), Vercauteren et al. (2007)] and deep learning based [Alvén et al.
(2019), Balakrishnan et al. (2019), Dalca et al. (2018), Dalca et al. (2019),
de Vos et al. (2019), Liu et al. (2019), Lee et al. (2019), Hu et al. (2019a),
Hu et al. (2019b), Yang et al. (2017), Xu and Niethammer (2019)] image
registration. These metrics assume that if the image intensities are aligned,
or strongly correlated, the images are well aligned. Each choice of metric
adds additional assumptions on the characteristics of the specific dataset.
Thus, a common methodological approach is to trial registration models with
multiple different pixel-based metrics, and choose the metric performing best
on the dataset [Balakrishnan et al. (2019), Hu et al. (2019a)].

The shortcomings of pixel-based similarity metrics, such as blur in the
generated images, have been studied substantially in the image generation
community [Hou et al. (2017), Zhang et al. (2018)] and have been superseded by
deep similarity metrics approximating human visual perception. Here, features
are extracted from neural networks pre-trained on image-classification tasks
[Deng et al. (2009)]. Performance can further be improved by fine-tuning the
features to human perception [Czolbe et al. (2020), Zhang et al. (2018)],
leading to generative models that produce photo-realistic images. We propose
to apply deep similarity metrics within image registration to achieve a
similar increase of performance for registration models.

### Contribution

We propose a data-driven similarity metric for image registration based on the
alignment of semantic features. We explore both unsupervised (using auto-
encoders) and semi-supervised methods (using a segmentation model) to learn
filters of semantic importance to the dataset. We use the learned features to
construct a similarity metric used for training a registration model, and
validate our approach on three biomedical datasets of different image
modalities and applications. Across all datasets, our method achieves
consistently high registration accuracy, outperforming even metrics utilizing
supervised information on two out of three datasets, while yielding an
inconclusive result on the third one. Our models learn to ignore noisy image
patches, leading to smoother transformations on low-quality data.

##  2 Background & related work

### Image registration

Most image registration frameworks model the problem as finding a
transformation $\Phi\colon\Omega\to\Omega$ that aligns a moving image
$\mathbf{I}\colon\Omega\to\mathbb{R}$ to a fixed image
$\mathbf{J}\colon\Omega\to\mathbb{R}$. The morphed source image, obtained by
applying the transformation, is expressed by function composition as
$\mathbf{I}\circ\Phi$. The domain $\Omega$ denotes the set of all coordinates
$\mathbf{x}\in\mathbb{R}^{d}$ within the image111While the domain $\Omega$ is
continuous in $\mathbb{R}^{d}$, recorded images and computations thereon are
discrete. For simplicity of notation, we denote both the continuous and
discrete domain as $\Omega$. We implement $\sum_{\mathbf{p}\in\Omega}$ as a
vectorized operation over the discrete pixel/voxel-coordinates and calculate
$|\Omega|$ as the total count of discrete pixels/voxels of the image. The
transformation $\Phi$ is implemented as a map from a discrete domain to a
continuous one, and the sampling of continuous points from a discrete image is
implemented via bi-/tri-linear interpolation.. Images record intensity at
discrete pixel-coordinates $\mathbf{p}$ but can be viewed as a continuous
function by interpolation.

The transformation is found through iterative algorithms [Avants et al.
(2008), Faisal Beg et al. (2005), Rueckert et al. (1999), Thirion (1998),
Vercauteren et al. (2007)], or predicted with learning based techniques
[Balakrishnan et al. (2019), Dalca et al. (2018), de Vos et al. (2019), Liu et
al. (2019), Lee et al. (2019), Hu et al. (2019b), Yang et al. (2017), Xu and
Niethammer (2019)]. In both cases, the optimal transformation is found by
minimization of a similarity measure $D$ and a $\lambda$-weighted regularizer
$R$, expressed via the loss function

$L(\mathbf{I},\mathbf{J},\Phi)=D(\mathbf{I}\circ\Phi,\mathbf{J})+\lambda
R(\Phi)\enspace.$ (1)

As many non-linear transformation models are over-parametrized and have
multiple minima, regularization is necessary. Smooth transformation fields,
that avoid folds or gaps, are assumed to be physically plausible and
encouraged by the regularizer. We use the diffusion regularizer throughout
this paper, which is defined as

$\Phi(\mathbf{p})=\mathbf{p}+\mathbf{u}(\mathbf{p}),\enspace
R(\Phi)=\sum_{\mathbf{p}\in\Omega}\|\nabla\mathbf{u}(\mathbf{p})\|^{2}\enspace,$
(2)

with the gradient of the displacement field $\nabla\mathbf{u}(\mathbf{p})$
approximated via finite differences.

### Similarity metrics for image registration

Denote by $D$ the (dis-)similarity between the morphed moving image
$\mathbf{I}\circ\Phi$ and the fixed image $\mathbf{J}$. Pixel-based metrics
are well explored within algorithmic image registration, a comparative
evaluation is given by Avants et al. (2011). We briefly recall the two most
popular choices, mean squared error ($\operatorname{MSE}$) and normalized
cross correlation ($\operatorname{NCC}$). The pixel-wise MSE is intuitive and
computationally efficient. It is derived from maximizing the negative log-
likelihood of a Gaussian normal distribution, making it an appropriate choice
under the assumption of Gaussian noise. On a grid of discrete points
$\mathbf{p}$ from domain $\Omega$, the $\operatorname{MSE}$ is defined as
$\operatorname{MSE}(\mathbf{I}\circ\Phi,\mathbf{J})=\frac{1}{|\Omega|}\sum_{\mathbf{p}\in\Omega}\|\mathbf{I}\circ\Phi(\mathbf{p})-\mathbf{J}(\mathbf{p})\|^{2}$.

Patch-wise normalized cross correlation is robust to variations in brightness
and contrast, making it a popular choice for images recorded with different
acquisition tools and protocols, or even across image modalities. For two
image patches $\mathbf{A},\mathbf{B}$, represented as column-vectors of length
$N$ with patch-wise means $\bar{\mathbf{A}},\bar{\mathbf{B}}$ and variance
$\sigma^{2}_{\mathbf{A}},\sigma^{2}_{\mathbf{B}}$, it is defined as

$\operatorname{NCC}_{\text{patch}}(\mathbf{A},\mathbf{B})=\sum_{n=1}^{N}\frac{(\mathbf{A}_{n}-\bar{\mathbf{A}})(\mathbf{B}_{n}-\bar{\mathbf{B}})}{\sigma_{\mathbf{A}}\sigma_{\mathbf{B}}}\enspace.$
(3)

The Patch-wise similarities are then averaged over the image Gee et al.
(1993); Avants et al. (2008). Note that an alternative, computationally
efficient variant of $\operatorname{NCC}$ is sometimes used in image
registration Avants et al. (2011). If annotations are available, these
unsupervised similarity measures can be extended by a supervised component to
measure both intensity differences and the alignment of annotated label maps
Balakrishnan et al. (2019).

### Learned similarity metrics for image registration

While deep-learning-based image registration has received much interest
recently, similarity metrics utilizing the compositional and data-driven
advantages of neural networks remain under-explored. Some current works
explore how to incorporate scale-space into learned registration models, but
similarity metrics remain pixel-based Hu et al. (2019a); Li and Fan (2018).
Learned similarity metrics are proposed by Haskins et al. (2019) and Krebs et
al. (2017), but both approaches require ground truth registration maps, which
are either synthetically generated or manually created by a medical expert.
Lee et al. (2019) propose to learn annotated structures of interest as part of
the registration model to aid alignment, but the method discards sub-regional
and non-annotated structures.

Learned common data representations and similarity metrics are frequently used
in multi-modal image registration Heinrich et al. (2012); Chen et al. (2016);
Simonovsky et al. (2016); Pielawski et al. (2020). While these approaches
learn common representations from well-aligned images of multiple modalities,
we aim to find a semantically augmented representation of images of a single
modality.

##  3 Method

### A discussion of NCC

Our design of a semantic similarity metric starts by examining the popular
$\operatorname{NCC}$ metric. We see that $\operatorname{NCC}$ between image
patches $\mathbf{A}$ and $\mathbf{B}$ is equivalent to the cosine-similarity
between the corresponding mean-centered vectors
$f(\mathbf{A})=\mathbf{A}-\bar{\mathbf{A}}$ and
$f(\mathbf{B})=\mathbf{B}-\bar{\mathbf{B}}$:

$\operatorname{NCC}_{\text{patch}}(\mathbf{A},\mathbf{B})=\frac{\big{<}f(\mathbf{A}),f(\mathbf{B})\big{>}}{\|f(\mathbf{A})\|\|f(\mathbf{B})\|}\enspace,$
(4)

with scalar product $\langle\cdot,\cdot\rangle$ and euclidean norm $\|\cdot\|$
. Thus, an alternative interpretation of the $\operatorname{NCC}$ similarity
measure is the cosine-similarity between two feature descriptors in a high-
dimensional space. The descriptor is given by the intensity values of a
centered image patch centered at a pixel $\mathbf{p}$. We will construct a
similar metric, using semantic feature descriptors instead.

### A semantic similarity metric for image registration

To align areas of similar semantic value, we propose a similarity metric based
on the agreement of semantic feature representations of two images. Semantic
feature maps are obtained by a feature extractor, which is pre-trained on a
surrogate task. To capture alignment of both localized, concrete features, and
global, abstract ones, we calculate the similarity at multiple layers of
abstraction. Given a set of feature-extracting functions
$F^{l}\colon\mathbb{R}^{\Omega\times C}\to\mathbb{R}^{\Omega_{l}\times C_{l}}$
for $L$ layers, we define

$\displaystyle\operatorname{DeepSim}(\mathbf{I}\circ\Phi,\mathbf{J})=\frac{1}{L}\sum_{l=1}^{L}\frac{1}{|\Omega_{l}|}\sum_{{\mathbf{p}}\in\Omega_{l}}\frac{\big{<}F^{l}_{\mathbf{p}}(\mathbf{I}\circ\Phi),F^{l}_{\mathbf{p}}(\mathbf{J})\big{>}}{\|F^{l}_{\mathbf{p}}(\mathbf{I}\circ\Phi)\|\|F^{l}_{\mathbf{p}}(\mathbf{J})\|}\enspace,$
(5)

where $F^{l}_{\mathbf{p}}(\mathbf{J})$ denotes the $l^{th}$ layer feature
extractor applied to image $\mathbf{J}$, at spatial coordinate $\mathbf{p}$.
It is a vector of $C_{l}$ output channels, and the spatial size of the
$l^{th}$ feature map is denoted by $|\Omega_{l}|$.

Similarly to $\operatorname{NCC}$, the neighborhood of the pixel is considered
in the metric, as we compose $F^{l}$ of convolutional filters with increasing
receptive area of the composition. In contrast, it is not necessary to zero-
mean the feature descriptors, as the semantic feature representations are
trained to be robust to variances in image brightness present in the training
data.

DeepSim
---
Warp
---
Feature Extractor
---
Registration Network
---
$\mathbf{I}$
---
$\hat{\mathbf{S}}$
---
$\mathbf{I}$
---
$\mathbf{J}$
---
$\mathbf{J}$
---
$\mathbf{I}\circ\Phi$
---
$F^{l}(\mathbf{J})$
---
$F^{l}(\mathbf{I}\circ\Phi)$
---
$\Phi$
---
$\mathbf{I}$
---
$\hat{\mathbf{I}}$
---
Segmentation
---
Autoencoder
---
Figure 1: Two-step training: First, the feature extractor (yellow) is trained.
We test both a segmentation model, and an auto-encoder. Next, the feature
extractor weights are frozen and used in the loss computation of the
registration network (blue).

### Feature extraction

To aid registration, the functions $F^{l}(\cdot)$ should extract features of
semantic relevance for the registration task, while ignoring noise and
artifacts. We extract features from the encoding branch of networks trained on
two surrogate tasks.

First, if segmentation masks are available, we can learn features on a
supplementary segmentation task. Segmentation models excel at learning
relevant kernels for the data while attaining invariance towards non-
predictive features like noise, but require an annotated dataset for training.
We denote the proposed similarity metric with feature extractors conditioned
on this task as $\operatorname{DeepSim_{seg}}$.

Second, we can learn an abstract feature representation of the dataset in an
unsupervised setting with auto-encoders. Auto-encoders learn efficient data
encoding by training the network to ignore signal noise. A benefit of this
approach is that no additional annotations are required. While variational
methods for encoding tasks have several advantages, we choose a deterministic
auto-encoder for its simplicity and lack of hyperparameters. We denote the
similarity metric with feature extractors conditioned on this task as
$\operatorname{DeepSim_{ae}}$.

[Qualitative comparison]  N/A for 3D Data [Detail view]

Figure 2: Left: Qualitative comparison of registration models. Rows: Datasets
Brain-MRI, Platelet-EM, PhC-U373. Columns 3-8: Registration models trained
with various similarity metrics. Right: Detail view of highlighted noisy
background areas on the Platelet-EM dataset. Select segmentation classes
annotated in color. The transformation is visualized by morphed grid-lines.

##  4 Experiments

Figure 3: Hyperparameter tuning. For each model, we select the regularizer
weighting factor $\lambda$ with the highest validation mean dice overlap for
further evaluation. Color scheme follows Figure 4.

We empirically compare registration models trained with the unsupervised
$\operatorname{DeepSim_{ae}}$ and semi-supervised
$\operatorname{DeepSim_{seg}}$ to the baselines $\operatorname{MSE}$,
$\operatorname{NCC}$, $\operatorname{NCC_{sup}}$ ($\operatorname{NCC}$ with
supervised information), and $\operatorname{VGG}$ (a VGG-net based deep
similarity metric from image generation). Our implementation of the baseline
methods follows Avants et al. (2011), Balakrishnan et al. (2019), and Hou et
al. (2017).

As our goal is not to advance the state of the art for any particular
registration task, but instead to explore the value of our loss-function in a
generic setting, we use well-established 2D and 3D U-Net Ronneberger et al.
(2015) architectures for both registration and segmentation models, sketched
in Figure 1. For the auto-encoder task, we use the same architecture, but
without the shortcut connections. Each network consists of three encoder and
decoder stages. Each stage consists of one batch normalization Ioffe and
Szegedy (2015), two convolutional, and one dropout layer Gal, Yarin and
Ghahramani (2016). After the final decoder step, we smooth the model output
with three more convolutional layers. We experimented with deeper
architectures but found they do not increase performance. The activation
function is LeakyReLu throughout the network, Softmax for the final layer of
the segmentation network, Sigmoid for the final layer of the auto-encoder, and
linear for the final layer of the registration network. The stages have
$64,128,256$ channels for 2d datasets, and $32,64,128$ channels for 3d.

The segmentation model is trained with a cross-entropy loss function, the
auto-encoder with the mean squared error, and the registration network with
the loss given by Eq. 1. The optimization algorithm for all models is ADAM
Kingma and Ba (2015), the initial learning rate is $10^{-4}$, decreasing by a
factor of $10$ each time the validation loss plateaus. All models are trained
until convergence. Due to the large 3D volumes involved, the choice of batch-
size is often limited by available memory. We sum gradients over multiple
passes to arrive at effective batch-sizes of 3-5 samples. The hyperparameter
$\lambda$ is tuned for each model independently on the validation set, we plot
the validation score of tested values in Figure 3.

To show that our approach applies to a variety of registration tasks, we
validate it on three 2D and 3D datasets of different modalities: 4000
T1-weighted Brain-MRI scans Di Martino et al. (2014); LaMontagne et al.
(2019), 74 slices of human blood cells of the Platelet-EM dataset Quay et al.
(2018), and 230 time-steps of the cell-tracking video PhC-U373 Maška et al.
(2014); Ulman et al. (2017). All datasets are pre-aligned through affine
transformations and split into train, validation, and test sections. We
augment each pair of images with random affine transformations during
training.

Figure 4: Mean dice overlap of registration models on the test set. Baselines in shades of blue, ours in red. Label of the best metric in bold, 2nd best black, others in grey. Boxplot with median, quartiles, deciles and outliers. Dataset | Method | Baseline
---|---|---
|  | $\operatorname{MSE}$ | $\operatorname{NCC}$ | $\operatorname{NCC}_{\text{sup}}$ | $\operatorname{VGG}$
|  | $p$ | $d$ | $p$ | $d$ | $p$ | $d$ | $p$ | $d$
Brain-MRI | $\operatorname{DeepSim_{ae}}$ | $<\\!0.001$ | $0.14$ | $<\\!0.001$ | $0.43$ | $<\\!0.001$ | $0.10$ | – | –
$\operatorname{DeepSim_{seg}}$ | $<\\!0.001$ | $0.30$ | $<\\!0.001$ | $0.60$ | $<\\!0.001$ | $0.25$ | – | –
Platelet-EM | $\operatorname{DeepSim_{ae}}$ | $\color[rgb]{0.55,0.55,0.55}\definecolor[named]{pgfstrokecolor}{rgb}{0.55,0.55,0.55}\pgfsys@color@gray@stroke{0.55}\pgfsys@color<EMAIL_ADDRESS>| $0.12$ | $<\\!0.001$ | $1.14$ | $<\\!0.001$ | $0.51$ | $<\\!0.001$ | $1.06$
$\operatorname{DeepSim_{seg}}$ | $\color[rgb]{0.55,0.55,0.55}\definecolor[named]{pgfstrokecolor}{rgb}{0.55,0.55,0.55}\pgfsys@color@gray@stroke{0.55}\pgfsys@color<EMAIL_ADDRESS>| $0.10$ | $<\\!0.001$ | $1.12$ | $<\\!0.001$ | $0.49$ | $<\\!0.001$ | $1.04$
PhC-U373 | $\operatorname{DeepSim_{ae}}$ | $<\\!0.001$ | $0.10$ | $<\\!0.001$ | $0.11$ | $<\\!0.001$ | $\color[rgb]{0.55,0.55,0.55}\definecolor[named]{pgfstrokecolor}{rgb}{0.55,0.55,0.55}\pgfsys@color@gray@stroke{0.55}\pgfsys@color<EMAIL_ADDRESS>| $<\\!0.001$ | $\color[rgb]{0.55,0.55,0.55}\definecolor[named]{pgfstrokecolor}{rgb}{0.55,0.55,0.55}\pgfsys@color@gray@stroke{0.55}\pgfsys@color<EMAIL_ADDRESS>
$\operatorname{DeepSim_{seg}}$ | $<\\!0.001$ | $0.12$ | $<\\!0.001$ | $0.13$ | $0.002$ | $\color[rgb]{0.55,0.55,0.55}\definecolor[named]{pgfstrokecolor}{rgb}{0.55,0.55,0.55}\pgfsys@color@gray@stroke{0.55}\pgfsys@color<EMAIL_ADDRESS>| $\color[rgb]{0.55,0.55,0.55}\definecolor[named]{pgfstrokecolor}{rgb}{0.55,0.55,0.55}\pgfsys@color@gray@stroke{0.55}\pgfsys@color<EMAIL_ADDRESS>| $\color[rgb]{0.55,0.55,0.55}\definecolor[named]{pgfstrokecolor}{rgb}{0.55,0.55,0.55}\pgfsys@color@gray@stroke{0.55}\pgfsys@color<EMAIL_ADDRESS>
Table 1: Results of the statistical significance test, performed with the
Wilcoxon signed rank test for paired samples. Effect size measured with
Cohen’s d. Statistically insignificant results (significance level 0.05,
Bonferroni-corrected to $p>0.002$) and very small effect sizes ($|d|<0.1$) in
grey.

##  5 Results

### Registration accuracy

We measure the mean Sørensen Dice coefficient on the unseen test-set in Figure
4. Statistical significance testing of the results is performed with the
Wilcoxon signed rank test for paired samples. A significance level of $5\%$
gives Bonferroni-adjusted significance threshold $p=0.002$. We further measure
the effect size with Cohen’s d and show the results in Table 1. Models trained
with our proposed $\operatorname{DeepSim_{ae}}$ and
$\operatorname{DeepSim_{seg}}$ outperform all baselines on the Brain-MRI and
Platelet-EM datasets, with strong statistical significance and effect size. On
the PhC-U373 dataset, all models achieve high dice-overlaps of $>0.97$.

We monitor the mean dice overlap during training. The training accuracy is,
with few exceptions, similar to the test accuracy, indicating that results
generalize well. The empirical convergence speeds of the tested metrics
differ. We observe that the $\operatorname{DeepSim_{seg}}$ converges faster
than the baselines, especially in the first few epochs of training. See
appendix A for a convergence plot.

### Qualitative examples & transformation grids

We plot the fixed and moving images $\mathbf{I},\mathbf{J}$ and the morphed
image $\mathbf{I}\circ\Phi$ for each similarity metric model along with a more
detailed view of a noisy patch of the Platelet-EM dataset in Figure 2, and
perform a quantitative analysis of the transformation in Appendix B. On models
trained with the baselines, we find strongly distorted transformation fields
in noisy areas of the images. In particular, models trained with
$\operatorname{NCC}$ and $\operatorname{NCC}_{\text{sup}}$ produce very
irregular transformations, despite careful tuning of the regularization-hyper-
parameter. The model trained by $\operatorname{DeepSim_{seg}}$ is more
invariant towards noise.

### Anatomical regions

The Brain-MRI dataset is annotated with the anatomical regions of the brain.
We plot the dice overlap per region in a boxplot in Figure 5, and highlight
regions where both of our metrics perform better than all baselines bold.
Baseline methods (blue) perform very similar, despite
$\operatorname{NCC}_{\text{sup}}$ as a supervised metric requiring more
information over the unsupervised $\operatorname{MSE}$ and
$\operatorname{NCC}$.

Figure 5: Dice overlaps of the anatomical regions of the Brain-MRI dataset.
Baselines in shades of blue, our methods in red. Bold labels for regions where
both of our methods score higher than all baselines. We combined labels of the
left and right brain hemispheres into a single class. The boxplot shows
median, quartiles, deciles and outliers.

##  6 Discussion & Conclusion

Registration models trained with $\operatorname{DeepSim}$ achieve high
registration accuracy across datasets, allowing improved downstream analysis
and diagnosis. Its consistency makes testing multiple traditional metrics
unnecessary; instead of empirically determining whether $\operatorname{MSE}$
or $\operatorname{NCC}$ captures the characteristics of a data-set best, we
can use $\operatorname{DeepSim}$ to learn the relevant features from the data.

Our experiments show that the availability of annotated anatomical regions can
help in learning semantic features, but is not a necessity. Gains in
performance in deep learning often require large, annotated datasets, which
are expensive and time-consuming to obtain in biomedical settings. The feature
extractor of $\operatorname{DeepSim_{ae}}$ was trained on the unsupervised
autoencoding task, requiring no additional data aside from the intensity
images to be registered. It performed similarly to the semi-supervised
$\operatorname{DeepSim_{seg}}$ on two of the three datasets and outperformed
the baselines.

The analysis of noisy patches in Figure 2 and Appendix B highlights a learned
invariance to noise. The pixel-based similarity metrics are distracted by
artifacts, leading to overly-detailed transformation fields. Models trained
with $\operatorname{DeepSim}$ do not show this problem. While smoother
transformation fields can be obtained for all metrics by strengthening the
regularizer, this would negatively impact the registration accuracy of
anatomically significant regions. Accurate registration of noisy, low-quality
images allows for shorter acquisition time and reduced radiation dose in
medical applications.

A weakness of $\operatorname{DeepSim}$ is the need to train a separate model
for feature extraction. The design, training, and testing of a second model
takes additional resources, and the presented approach necessitates a dataset
to train the feature extractor with. In the context of deep learning-based
registration, $\operatorname{DeepSim_{ae}}$ requires no additional data to
what is required to train the registration model, while
$\operatorname{DeepSim_{seg}}$ requires additional label maps. Compared to
algorithmic registration methods, which optimize the registration map for each
pair of input images separately, any deep-learning approach requires
additional data.

$\operatorname{DeepSim}$ is a general metric, applicable to image registration
tasks of all modalities and anatomies. Beyond the presented datasets, our good
results in the presence of noise let us hope that $\operatorname{DeepSim}$
will improve registration accuracy in domains such as low dose CT, ultrasound,
or microscopy, where details are often hard to identify, and image quality is
poor. We further emphasize that the application of $\operatorname{DeepSim}$ is
limited neither to deep learning nor to image registration. In Algorithmic
image registration, a similarity-based loss is minimized via gradient descent-
based methods. $\operatorname{DeepSim}$ can be applied to drive algorithmic
methods, improving their performance by aligning deep, semantic feature
embeddings. Similarly, $\operatorname{DeepSim}$ could be used for other image
regression tasks, such as image synthesis, -translation, or -reconstruction.

This work was funded in part by the Novo Nordisk Foundation through the Center
for Basic Machine Learning Research in Life Science (grant no. 0062606), and
in part through the Lundbeck Foundation (grant no. R218-2016-883). We further
thank Matthew Quay, the Cell Tracking Challenge, and the Cancer Imaging
Archive for the provision of the datasets.

## References

  * Alvén et al. (2019) Jennifer Alvén, Kerstin Heurling, Ruben Smith, Olof Strandberg, Michael Schöll, Oskar Hansson, and Fredrik Kahl.  A Deep Learning Approach to MR-less Spatial Normalization for Tau PET Images.  In _International Conference on Medical Image Computing and Computer-Assisted Intervention_ , pages 355–363. 2019.
  * Avants et al. (2008) B. B. Avants, C. L. Epstein, M. Grossman, and J. C. Gee.  Symmetric diffeomorphic image registration with cross-correlation: Evaluating automated labeling of elderly and neurodegenerative brain.  _Medical Image Analysis_ , 12(1):26–41, 2008\.
  * Avants et al. (2011) Brian B. Avants, Nicholas J. Tustison, Gang Song, Philip A. Cook, Arno Klein, and James C. Gee.  A reproducible evaluation of ANTs similarity metric performance in brain image registration.  _NeuroImage_ , 54(3):2033–2044, 2011.
  * Balakrishnan et al. (2019) Guha Balakrishnan, Amy Zhao, Mert R. Sabuncu, John Guttag, and Adrian V. Dalca.  VoxelMorph: A Learning Framework for Deformable Medical Image Registration.  _IEEE Transactions on Medical Imaging_ , 38(8):1788–1800, 2019.
  * Chen et al. (2016) Xi Chen, Yan Duan, Rein Houthooft, John Schulman, Ilya Sutskever, and Pieter Abbeel.  InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets.  In _Advances in Neural Information Processing Systems_ , 2016.
  * Czolbe et al. (2020) Steffen Czolbe, Oswin Krause, Ingemar Cox, and Christian Igel.  A Loss Function for Generative Neural Networks Based on Watson’s Perceptual Model.  _Advances in Neural Information Processing Systems_ , 2020.
  * Dalca et al. (2018) Adrian V. Dalca, Guha Balakrishnan, John Guttag, and Mert R. Sabuncu.  Unsupervised Learning for Fast Probabilistic Diffeomorphic Registration.  _Medical Image Computing and Computer Assisted Intervention_ , pages 729–738, 2018.
  * Dalca et al. (2019) Adrian V. Dalca, Marianne Rakic, John Guttag, and Mert R. Sabuncu.  Learning Conditional Deformable Templates with Convolutional Networks.  _Advances in Neural Information Processing Systems_ , pages 806—-818, 2019.
  * de Vos et al. (2019) Bob D. de Vos, Floris F. Berendsen, Max A. Viergever, Hessam Sokooti, Marius Staring, and Ivana Išgum.  A deep learning framework for unsupervised affine and deformable image registration.  _Medical Image Analysis_ , 52:128–143, 2019.
  * Deng et al. (2009) Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei.  Imagenet: A large-scale hierarchical image database.  In _Conference on Computer Vision and Pattern Recognition_ , pages 248–255, 2009.
  * Di Martino et al. (2014) Adriana Di Martino, Chao-Gan Yan, Qingyang Li, and Others.  The autism brain imaging data exchange: towards a large-scale evaluation of the intrinsic brain architecture in autism.  _Molecular psychiatry_ , 19(6):659–667, 2014\.
  * Faisal Beg et al. (2005) Mirza Faisal Beg, Michael I Miller, Alain Trouvétrouv, and Laurent Younes.  Computing Large Deformation Metric Mappings via Geodesic Flows of Diffeomorphisms.  _International Journal of Computer Vision_ , 61(2):139–157, 2005.
  * Gal, Yarin and Ghahramani (2016) Zoubin Gal, Yarin and Ghahramani.  Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning.  In _International Conference on Machine Learning_ , pages 1050—-1059, 2016.
  * Gee et al. (1993) James C Gee, Martin Reivich, and Ruzena Bajcsy.  Elastically deforming a three-dimensional atlas to match anatomical brain images.  _IRCS Technical Reports Series_ , (192), 1993.
  * Haskins et al. (2019) Grant Haskins, Jochen Kruecker, Uwe Kruger, Sheng Xu, Peter A. Pinto, Brad J. Wood, and Pingkun Yan.  Learning deep similarity metric for 3D MR–TRUS image registration.  _International Journal of Computer Assisted Radiology and Surgery_ , 14(3):417–425, 2019.
  * Heinrich et al. (2012) Mattias P. Heinrich, Mark Jenkinson, Manav Bhushan, Tahreema Matin, Fergus V. Gleeson, Sir Michael Brady, and Julia A. Schnabel.  MIND: Modality independent neighbourhood descriptor for multi-modal deformable registration.  _Medical Image Analysis_ , 16(7):1423–1435, 2012\.
  * Hou et al. (2017) Xianxu Hou, Linlin Shen, Ke Sun, and Guoping Qiu.  Deep feature consistent variational autoencoder.  In _Winter Conference on Applications of Computer Vision_ , pages 1133–1141. IEEE, 2017.
  * Hu et al. (2019a) Xiaojun Hu, Miao Kang, Weilin Huang, Matthew R. Scott, Roland Wiest, and Mauricio Reyes.  Dual-Stream Pyramid Registration Network.  In _International Conference on Medical Image Computing and Computer-Assisted Intervention_ , pages 382–390. 2019a.
  * Hu et al. (2019b) Yipeng Hu, Eli Gibson, Dean C. Barratt, Mark Emberton, J. Alison Noble, and Tom Vercauteren.  Conditional Segmentation in Lieu of Image Registration.  In _International Conference on Medical Image Computing and Computer-Assisted Intervention_ , pages 401–409. 2019b.
  * Ioffe and Szegedy (2015) Sergey Ioffe and Christian Szegedy.  Batch normalization: Accelerating deep network training by reducing internal covariate shift.  In _International Conference on Machine Learning_ , pages 448–456. International Machine Learning Society (IMLS), 2015.
  * Kingma and Ba (2015) Diederik P. Kingma and Jimmy Lei Ba.  Adam: A method for stochastic optimization.  In _International Conference on Learning Representations_ , 2015.
  * Krebs et al. (2017) Julian Krebs, Tommaso Mansi, Hervé Delingette, Li Zhang, Florin C. Ghesu, Shun Miao, Andreas K. Maier, Nicholas Ayache, Rui Liao, and Ali Kamen.  Robust non-rigid registration through agent-based action learning.  In _Lecture Notes in Computer Science_ , volume 10433, pages 344–352. Springer Verlag, 2017.
  * LaMontagne et al. (2019) Pamela J LaMontagne, Tammie L S Benzinger, John C Morris, and Others.  OASIS-3: Longitudinal Neuroimaging, Clinical, and Cognitive Dataset for Normal Aging and Alzheimer Disease.  _medRxiv_ , 2019.
  * Lee et al. (2019) Matthew C. H. Lee, Ozan Oktay, Andreas Schuh, Michiel Schaap, and Ben Glocker.  Image-and-Spatial Transformer Networks for Structure-Guided Image Registration.  In _International Conference on Medical Image Computing and Computer-Assisted Intervention_ , pages 337–345. 2019.
  * Li and Fan (2018) Hongming Li and Yong Fan.  Non-rigid image registration using self-supervised fully convolutional networks without training data.  In _Proceedings - International Symposium on Biomedical Imaging_ , volume 2018-April, pages 1075–1078. IEEE Computer Society, 2018.
  * Liu et al. (2019) Lihao Liu, Xiaowei Hu, Lei Zhu, and Pheng-Ann Heng.  Probabilistic Multilayer Regularization Network for Unsupervised 3D Brain Image Registration.  In _International Conference on Medical Image Computing and Computer-Assisted Intervention_ , pages 346–354. 2019.
  * Maška et al. (2014) Martin Maška, Vladimír Ulman, David Svoboda, Pavel Matula, Petr Matula, Cristina Ederra, Ainhoa Urbiola, Tomás España, Subramanian Venkatesan, Deepak M W Balak, Pavel Karas, Tereza Bolcková, Markéta Štreitová, Craig Carthel, Stefano Coraluppi, Nathalie Harder, Karl Rohr, Klas E G Magnusson, Joakim Jaldén, Helen M Blau, Oleh Dzyubachyk, Pavel Krížek, Guy M Hagen, David Pastor-Escuredo, Daniel Jimenez-Carretero, Maria J Ledesma-Carbayo, Arrate Muñoz-Barrutia, Erik Meijering, Michal Kozubek, and Carlos Ortiz-de Solorzano.  A benchmark for comparison of cell tracking algorithms.  _Bioinformatics_ , 30(11):1609–1617, 2014.
  * Pielawski et al. (2020) Nicolas Pielawski, Elisabeth Wetzer, Johan Öfverstedt, Jiahao Lu, Carolina Wählby, Joakim Lindblad, and Nataša Sladoje.  CoMIR: Contrastive Multimodal Image Representation for Registration.  _Advances in neural information processing systems_ , 2020.
  * Quay et al. (2018) Matthew Quay, Zeyad Emam, Adam Anderson, and Richard Leapman.  Designing deep neural networks to automate segmentation for serial block-face electron microscopy.  In _International Symposium on Biomedical Imaging_ , volume 2018-April, pages 405–408. IEEE Computer Society, 2018.
  * Ronneberger et al. (2015) Olaf Ronneberger, Philipp Fischer, and Thomas Brox.  U-net: Convolutional networks for biomedical image segmentation.  In _International Conference on Medical Image Computing and Computer-Assisted Intervention_ , volume 9351, pages 234–241. Springer Verlag, 2015.
  * Rueckert et al. (1999) Daniel Rueckert, Luke I Sonoda, Carmel Hayes, Derek LG Hill, Martin O Leach, and David J Hawkes.  Nonrigid registration using free-form deformations: Application to breast mr images.  _IEEE Transactions on Medical Imaging_ , 18(8):712–721, 1999.
  * Simonovsky et al. (2016) Martin Simonovsky, Benjamín Gutiérrez-Becker, Diana Mateus, Nassir Navab, and Nikos Komodakis.  A deep metric for multimodal registration.  In _Lecture Notes in Computer Science_ , volume 9902, pages 10–18. Springer Verlag, 2016.
  * Thirion (1998) Jean-Philippe Thirion.  Image matching as a diffusion process: an analogy with Maxwell’s demons.  Technical Report 3, 1998.
  * Ulman et al. (2017) Vladimír Ulman, Martin Maška, Klas E.G. Magnusson, Olaf Ronneberger, Carsten Haubold, Nathalie Harder, Pavel Matula, Petr Matula, David Svoboda, Miroslav Radojevic, Ihor Smal, Karl Rohr, Joakim Jaldén, Helen M. Blau, Oleh Dzyubachyk, Boudewijn Lelieveldt, Pengdong Xiao, Yuexiang Li, Siu Yeung Cho, Alexandre C. Dufour, Jean Christophe Olivo-Marin, Constantino C. Reyes-Aldasoro, Jose A. Solis-Lemus, Robert Bensch, Thomas Brox, Johannes Stegmaier, Ralf Mikut, Steffen Wolf, Fred A. Hamprecht, Tiago Esteves, Pedro Quelhas, Ömer Demirel, Lars Malmström, Florian Jug, Pavel Tomancak, Erik Meijering, Arrate Muñoz-Barrutia, Michal Kozubek, and Carlos Ortiz-De-Solorzano.  An objective comparison of cell-tracking algorithms.  _Nature Methods_ , 14(12):1141–1152, 2017.
  * Vercauteren et al. (2007) Tom Vercauteren, Xavier Pennec, Aymeric Perchant, and Nicholas Ayache.  Non-parametric diffeomorphic image registration with the demons algorithm.  In _Lecture Notes in Computer Science_ , volume 4792, pages 319–326, 2007.
  * Xu and Niethammer (2019) Zhenlin Xu and Marc Niethammer.  DeepAtlas: Joint Semi-supervised Learning of Image Registration and Segmentation.  In _International Conference on Medical Image Computing and Computer-Assisted Intervention_ , pages 420–429. 2019.
  * Yang et al. (2017) Xiao Yang, Roland Kwitt, Martin Styner, and Marc Niethammer.  Quicksilver: Fast predictive image registration - A deep learning approach.  _NeuroImage_ , 158:378–396, 2017.
  * Zhang et al. (2018) Richard Zhang, Phillip Isola, Alexei A. Efros, Eli Shechtman, and Oliver Wang.  The Unreasonable Effectiveness of Deep Features as a Perceptual Metric.  _Conference on Computer Vision and Pattern Recognition_ , pages 586–595, 2018.

##  Appendix A Optimization convergence

Figure 6: Mean dice overlap of the registration models during training. The
training duration per model on a single RTX 2080 GPU is approximately seven
days for the Brain-MRI dataset and one day for the Platelet, PhC-U373
datasets.

##  Appendix B Regularity of the transformation

Method | Dataset
---|---
| Brain-MRI | Platelet-EM | PhC-U373
| $\sigma^{2}(J_{\Phi})$ | $|J_{\Phi}|<0\,(\%)$ | $\sigma^{2}(J_{\Phi})$ | $|J_{\Phi}|<0\,(\%)$ | $\sigma^{2}(J_{\Phi})$ | $|J_{\Phi}|<0\,(\%)$
$\operatorname{MSE}$ | $0.19$ | $0.42$ | $0.23$ | $0.40$ | $0.03$ | $0.02$
$\operatorname{NCC}$ | $0.23$ | $0.93$ | $0.54$ | $4.15$ | $0.29$ | $0.71$
$\operatorname{NCC}_{\text{sup}}$ | $0.11$ | $0.28$ | $0.54$ | $4.03$ | $0.28$ | $0.57$
$\operatorname{VGG}$ | – | – | $0.09$ | $0.00$ | $0.16$ | $0.13$
$\operatorname{DeepSim_{ae}}$ | $0.13$ | $0.20$ | $0.10$ | $0.04$ | $0.20$ | $0.35$
$\operatorname{DeepSim_{seg}}$ | $0.09$ | $0.12$ | $0.13$ | $0.14$ | $0.18$ | $0.32$
Table 2: Regularity of the transformation. The determinant of the Jacobian of
the transformation $|J_{\Phi}|$ is a measure of how the image volume is
compressed or stretched by the transformation. We assess transformation
smoothness by the variance of the voxel-wise determinant
$\sigma^{2}(J_{\Phi})$, a lower variance indicates a more volume-preserving
transformation. We assess the regularity of the transformation by measuring
the percentage of voxels for which the determinant is $<0$, which indicates
domain folding. Figure 7: Addendum to Figure 2. Detail view of noisy
background areas on the Platelet-EM dataset. The cell-boundary is annotated in
blue. The transformation is visualized by morphed grid-lines.


# Local mechanism of valence bond formation in IrTe2

T. Ritschel  Institute of Solid State and Materials Physics, TU Dresden, 01069
Dresden, Germany  Q. Stahl  Institute of Solid State and Materials Physics, TU
Dresden, 01069 Dresden, Germany  M. Kusch  Institute of Solid State and
Materials Physics, TU Dresden, 01069 Dresden, Germany  J. Trinckauf  Leibniz
Institute for Solid State and Materials Research IFW Dresden, Helmholtzstr.
20, D-01069 Dresden, Germany  G. Garbarino  European Synchrotron Radiation
Facility, BP 220, F-38043 Grenoble Cedex, France  V. Svitlyk  European
Synchrotron Radiation Facility, BP 220, F-38043 Grenoble Cedex, France  M.
Mezouar  European Synchrotron Radiation Facility, BP 220, F-38043 Grenoble
Cedex, France  J. Yang  Department of Physics, New Jersey Institute of
Technology, Newark, NJ 01702, USA  S.W. Cheong  Rutgers Center for Emergent
Materials and Department of Physics and Astronomy, Rutgers, The State
University of New Jersey, 136 Frelinghuysen Road, Piscataway, New Jersey
08854-8019 USA  J. Geck<EMAIL_ADDRESS>Institute of Solid State
and Materials Physics, TU Dresden, 01069 Dresden, Germany  Würzburg-Dresden
Cluster of Excellence ct.qmat, Technische Universität Dresden, 01062 Dresden,
Germany

###### Abstract

Doped IrTe2 is considered a platform for topological superconductivity and
therefore receives currently a lot of interest. In addition, the
superconductivity in these materials exists in close vicinity of electronic
valence bond crystals, which we explore here by means of high-pressure single
crystal x-ray diffraction in combination with density functional theory. Our
crystallographic refinements provide unprecedented information about the
structural evolution as a function of applied pressure up to 42 GPa. Using
this structural information for density functional theory calculations, we
show that the valence bond formation in IrTe2 is driven by changes in the Ir-
Te-Ir bond angle. When a valence bond is formed, this bond angle decreases
drastically, leading to a stabilization of local valence bonds large enough to
push them out of a broad band continuum. This unusual local mechanism of
valence bond formation in an itinerant material provides a natural explanation
for the different valence bond orders in IrTe2, implies a very strong
electron-phonon coupling and is most likely relevant for the superconductivity
as well.

##  I Introduction

Novel quantum states and their mutual interactions are a major topic of
contemporary condensed matter science. Famous and intensively studied examples
are unconventional superconductivityStewart (2017), charge density wavesArpaia
_et al._ (2019), quantum spin liquidsSavary and Balents (2017), topological
states of matterYan and Felser (2017) or Dirac materialsWehling  _et al._
(2014). Here two branches of research are attracting particularly much
attention: The first one is the role of fluctuating electronic order and
quantum magnetism for superconductivity. The second one, is the combination of
topological electrons and superconductivity in the very sough-after
topological superconductors.

In this regard, transition metal dichalcogenides (TMDs) of the type $TX_{2}$
($T$: transition metal, $X$: chalcogenide) are extremely interesting, as they
are in fact notorious for harbouring a variety of intriguing quantum
phenomena. To name just a few cases in point, both superconductiviy and charge
density waves occur in NbSe2, TaSe2 and TaS2, while WTe2 presently attracts a
lot of interest in the context of topological Weyl-physicsRossnagel (2011);
Yan and Felser (2017); Yang  _et al._ (2017); Gye  _et al._ (2019). Typical
TMDs can be regarded as stacks of two-dimensional planes with relatively weak
interactions between the planes. As a result, the electronic structure of TMDs
is usually very anisotropic and possess a pronounced two-dimensional
character, which, in turn, promotes electronic instabilities and
unconventional electronic ground states.

The TMD material IrTe2 is no different in this respect and, indeed, turns out
to be an extremely interesting case: In its trigonal $1T$-structure (cf. Fig.
1 (a,b)) IrTe2 exhibits bulk Dirac points, spin-orbit driven gap inversions
and the corresponding topological surface states, which all emerge from the Te
$5p$ derived bandsBahramy  _et al._ (2018); Nicholson  _et al._ (2021); Fei
_et al._ (2018); Jiang  _et al._ (2020). It is particularly exciting that this
system not only hosts a topological electronic band structure but also
superconductivity below 3 K, raising great hopes for intrinsic topological
superconductivity Kiswandhi  _et al._ (2013); Kudo  _et al._ (2013); Kamitani
_et al._ (2013); Oh  _et al._ (2013).

Figure 1: Trigonal 1T (spacegroup: 164) host crystal structure of IrTe2 (a,b)
and schematic of the experimental high-pressure XRD setup at the beamline ID27
of the ESRF (c).

But apart from this, there is also electronic order. In fact, various ordered
states have been reported, which, referring to the trigonal space group
$P\overline{3}m1$ (#164, CdI2-structure), can be characterized by wave vectors
$\vec{q}_{n}=(1/(3n+2),0,1/(3n+2))$ with $n=1,2,\dots,\infty$ where
$\vec{q}_{\infty}=(1/6,0,1/6)$. The orderings can be constructed from simple
basic building blocksPascut  _et al._ (2014a) containing subunits with very
short Ir-Ir distances. This Ir-pair is part of a larger object, which is
commonly referred to as dimer in the literature and which involves both Te-
and Ir-statesSaleh and Artyukhin (2020). Bulk IrTe2 at ambient pressure
exhibits an ordered lattice of these dimers with $\vec{q}_{n=1}=(1/5,0,1/5)$
below 280 K which transforms into a lattice with $\vec{q}_{n=2}=(1/8,0,1/8)$
at 180 K upon further coolingPascut  _et al._ (2014a). Different ordered
states can also coexist in different regions of the samplePascut  _et al._
(2014a); Hsu  _et al._ (2013); Chen  _et al._ (2017), although it appears that
with decreasing temperature T larger $n$ become stable.

In bulk crystals, the ordering of dimers competes with superconductivity. As
soon as this preemptive static order is suppressed, superconductivity can
emerge in bulk crystals at low temperatures. In case of IrTe2-nanoflakes that
are only a few atomic layers thick, however, the situation has been found to
be different. As reported in a very recent study, the superlattice of dimers
in these nanoflakes does actually support two-dimensional superconductivity
Park  _et al._ (2021). Superconductivity and ordered dimers are therefore not
just competing but interacting in a more complex manner. One corner stone for
a better understanding of the interplay between dimers, topological electrons
and superconductivity in IrTe2 is therefore an improved understanding of the
dimerization process in IrTe2. Especially since this will expose key
interactions at play in this material.

So far, various mechanisms have been introduced to explain the formation of
dimers in IrTe2, including partial Fermi-surface nestingYang  _et al._ (2012),
an orbitally induced Peierls effectOotsuki  _et al._ (2012), multicenter bonds
of Ir and TeSaleh and Artyukhin (2020), the polymerization-depolymerization of
Te-bondsOh  _et al._ (2013) as well as electronic instabilities caused by a
van Hove singularity at the Fermi-levelQian  _et al._ (2014). The multitude of
scenarios quoted above demonstrates that, despite extensive research efforts,
important aspects of the dimerization in IrTe2 remain controversial. To
address this issue and to shed light onto the physics at work, we performed
high-pressure x-ray diffraction studies of pristine IrTe2 single crystals. In
this way, we obtain unprecedented structural information of great detail,
which we then use for a thorough analysis within density functional theory.
The key result of this combined study is a dramatic change in the Ir-Te-Ir
bond angle within the dimers, which is instrumental for their stabilization by
the formation of localized valence bonds.

##  II Results

### High-pressure x-ray diffraction

Especially, when dealing with competing phases, as is the case here, external
pressure is a particularly valuable tool, because it is a clean control
parameter that avoids unwanted side effects, which may be caused by local
changes induced by chemical substitution or doping. We therefore performed
high-pressure XRD experiments at pressures $P$ up to 42 GPa and temperatures
between room temperature and 420 K (see methods section). The experimental
setup is sketched in Fig. 1 (c).

Figure 2: Reconstructed reciprocal space maps for the trigonal host phase
(a,b,i) at 300 K, the triclinic 1/5-phase (c,d,j) at 300 K and the monoclinic
1/6-phase (g-h,k,l) at 350 K. The pressure at which the data has been taken is
indicated on top of the panels. Reciprocal lattice vectors and Miller indices
$h,k$ and $l$ refer to the hexagonal cell and an approximately hexagonal cell
for the 1/5- and 1/6-phase. $hk0$-planes are shown in (a,c,e,f) and
$h\bar{1}l$-planes in (b,d,f,h). (i-l) show a magnification of the region
indicated by the red rectangle in (b). Green arrows in (j-l) depict the
$q$-vector.

For each pressure point, one image has been collected by continuously
recording the diffracted intensity on the detector during a large
$\omega$-rotation over $\pm$ 31∘. These measurements provide a qualitative
overview allowing quick detection of structural changes. Hence we will refer
to them as overview scans in the following. At selected pressure values,
identified using these overview scans, single crystal XRD data sets have been
collected. These data sets consist of images integrating over $\Delta\omega$ =
0.5∘. The obtained single crystal data has then been used for the
crystallographic structure refinements (see methods section). Detailed results
of the refinements are given in the supplementary information.

In Fig. 2 we present representative reciprocal space maps of the data at
different pressure-temperature points. At small pressures, here $P=0.4$ GPa in
panels a),b) and i), we observe the trigonal phase ($P\overline{3}m1$) as
expected. Fig. 2 a) show a $(h,k,0)$-plane of the reciprocal lattice, while a
$(h,-1,l)$-plane is presented in Fig. 2 b,i). The vertices of the dashed lines
in the panels indicate the positions of the reciprocal lattice of the trigonal
phase, which coincide perfectly with all reflections originating from the
sample. The additional spots outside the trigonal grid are all due to the
diamond anvils of the DAC.

Figure 3: Pressure-temperature phase diagram of IrTe2 as deduced from the
present XRD-data. Starting from the trigonal phase without dimers, a
metastability region is entered upon increasing pressure. In this region
different ordered phases appear and sometimes coexist. With further increasing
pressure, the long ranged ordered 1/6-phase is reached, which remains stable
up to 42 GPa. Small (black) crosses indicate points where an overview scan has
been taken. Large (red) crosses indicate points where single crystal
diffraction data sets have been recorded (further explanations in the text).

Upon increasing the pressure to $P=1.4$ GPa, the diffraction pattern changes
drastically, as can be observed in panels c),d) and j) of Fig. 2. Most
prominently, additional reflections appear in the $(h,-1,l)$-plane (cf. Fig. 2
d,j)). The width and intensity of these additional reflections is comparable
to that of the Bragg-reflections in the trigonal phase, i.e. we are not
dealing with a small lattice modulation as, for example, in a typical weak
coupling charge density wave phase. As can nicely be observed in Fig. 2 c) by
direct inspection, the Bragg-spots at $P=1.4$ GPa deviate slightly from the
trigonal reciprocal lattice (dashed lines). In fact, we find a triclinic
lattice symmetry under the present conditions ($P\overline{1}$). Nonetheless,
still referring to the trigonal low-pressure phase, this new structure can be
characterized by the wave vector $\vec{q}_{1}=(1/5,0,1/5)$, as it is commonly
done in the literature. We will refer to this phase as the 1/5-phase in the
following. Note that the peak splitting –particularly apparent in Fig. 2 i)–
is not caused by a phase coexistence, but is due to the twinning of the sample
under the present conditions. The sample therefore shows a pure 1/5-phase,
which enabled us to fully refine its structure based on the present data set
(cf. supplementary information).

Upon increasing $P$ further, the 1/6-phase characterized by
$\vec{q}_{\infty}=(1/6,0,1/6)$ is entered. This is shown by the data sets for
$P=6.5$ GPa and 42 GPa presented in Fig. 2. This phase is found to be
monoclinic ($C2/c$). The 1/6-phase has been found to remain stable up to the
highest pressures. No significant change in position or width of the detected
peaks, other than due to the compression of the lattice, has been observed up
to 42 GPa. Interestingly, the probed sample volume in the 1/6-phase was found
not to be twinned, as can be observed in Fig. 2. A possible reason for this
detwinning is a small pressure gradient within the DAC. Also the intensities
of the 1/6-phase could be refined successfully yielding a full determination
of this phase (cf. supplementary information).

The measurements of our HP-XRD experiments are summarized in the $pT$ phase
diagram displayed in Fig. 3. In the region between the trigonal and the
1/6-phase, we observed the 1/5-, 1/8- and the 1/11-phase ($n=1$, 2 and 3) as
well as a coexistence of some of these. For this reason it is denoted as
”metastability region”.

Our refined 1/5- and 1/6-structures at $p<6.5$ GPa resemble very closely the
corresponding structures found in Se-substituted IrTe2 at ambient
pressurePascut  _et al._ (2014a, b), where the concentration of dimers
increases with $n$: for the 1/5-phase ($n=1$) the fraction of dimerized Ir is
2/5, for the 1/8-phase ($n=2$) it is already 1/2 and for the 1/6 ($n=\infty$)
it becomes 2/3.

The evolution of the lattice structure with increasing $P$ up to 42 GPa is
illustrated in Fig. 4. As described above, the dimerized phases are no longer
trigonal but mono- or triclinic. However, in order to facilitate the
comparison among the different phases, these lower symmetry structures have
been mapped onto an approximate trigonal cell. Selected structural parameters
obtained by this mapping are shown in Fig. 4. While in the truly trigonal
phase $a=b$, for the structures of the dimerized phase only $a\simeq b$ is
fulfilled. This can be observed in Fig. 4 b), where $a$ and $b$ split up upon
entering the dimerized phases with increasing $P$. Panel b) and the
$c/a$-ratio in panel d) show that, as expected, the $c$-axis is much more
compressible than the $a$\- and $b$-axis. More specifically, we obtain
$4.7\times 10^{-3}\mathrm{GPa}^{-1}\simeq\kappa_{c}>\kappa_{a,b}\simeq
1.3\times 10^{-3}\,\mathrm{GPa}^{-1}$ for the axial compressibilities. At 42
GPa the $c$-axis and the unit cell volume are reduced by about 12% and 25%,
respectively.

Figure 4: Structural parameters as determined from the single crystal data as
a function of pressure. (a): Sketch of one Te-Ir-Te layer in the $q=1/6$
structure. The short Ir-Ir distances inside the dimers are highlighted in red.
All parameters in (b-e) are derived from a transformation of the supercell
back to an approximately trigonal cell. The corresponding (pseudo)-trigonal
in-plane lattice vectors $\mathbf{a}$ and $\mathbf{b}$ are indicated as black
arrows. The lattice vectors corresponding to the monoclinic $q=1/6$ structure
($\mathbf{a_{s},b_{s}}$ and $\mathbf{c_{s}}$) are shown in gray. Note that the
in-plane directions in terms of the monoclinic cell are given by
$\mathbf{a_{s}}+\mathbf{c_{s}}$ and $\mathbf{b_{s}}$. (b,c): The
(pseudo)-trigonal lattice parameters and cell volume as a function of
pressure. (d): $c/a$-ratio (green) and ratio between long and short Ir-Ir
bonds (blue). (e): Ir-Te-Ir angle $\alpha$ as indicated in (a). Square,
triangular and circular symbols correspond to data from the trigonal,
triclinic $q=1/5$\- and monoclinic $q=1/6$-phase. Solid lines are guides to
the eye.

In the trigonal phase, each Ir-site has 6 nearest neighbor Ir-sites at the
same distance $d\simeq 3.92$ Åwithin the IrTe2-layer. In the dimerized phases
this changes and every Ir in a dimer has 1 short intra-dimer distance $d_{s}$
and 5 longer nearest neighbor distances of average length $d_{l}$. The
formation of the dimers can therefore be tracked very nicely by the ratio
$\gamma=d_{l}/d_{s}$, which is presented in Fig. 4 c). With increasing $P$,
this ratio jumps from $\gamma=1$ in the trigonal phase (no dimers,
$d=d_{s}=d_{l}$) to $\gamma\simeq 1.27$ ($d_{s}\simeq 3.11$ Å, $d_{l}\simeq
3.96$ Å) in the 1/5-phase. Since $\gamma$ can be interpreted as an order
parameter of the local dimerization, its large value implies a very strong
dimerization in IrTe2. Furthermore, the monotonous increase of $\gamma$ with
$P$ shows that the dimers are more and more stabilized upon pressurization.

A key result of the present refinements concerns the Ir-Te-Ir bond angle
$\alpha$ (cf. Fig. 4 a)): Across the $P$-driven transition into the 1/5-phase,
the angle $\alpha$ inside the dimers collapses from 95.8 ∘ to 73.0 ∘, as shown
in Fig. 4 e). Note, the evolution of $\alpha$ with further increasing $P$ is
smooth and monotonous also across the transition into the 1/6-phase. The
reduced value of $\alpha$ is not significantly influenced by the specific long
range order, but rather appears to be a characteristic property of the local
dimers.

Compared to $\alpha$, the nearest neighbor Ir-Te distances change much more
moderately: The single Ir-Te distance of $2.645$ Å in the trigonal phase
splits up upon entering the dimerized phases (see supplementary). This
splitting is of the order of $\pm 0.06$ Å, which amounts to about only 2% of
$2.645$ Å. Also note that the Ir-Te distances of dimerized and non-dimerized
Ir-sites exhibit a very similar rate of compression upon increasing $P$.

The above analysis identifies a key structural parameter for the dimerization,
which escaped attention so far: the Ir-Te-Ir bond angle $\alpha$. The question
now obviously is, which role $\alpha$ plays for the stabilization of dimers in
IrTe2.

Figure 5: Electronic structure as a function of hydrostatic pressure. (a-d):
Fat bands representing the Te-5p character for the trigonal structure at
ambient conditions (a), the $q=1/5$ structure (b) and the $q=1/6$ structure
(c,d). Band weights of the $q=1/5$ and $q=1/6$ phase were unfolded into the
Brillouin zone of the trigonal structure. A magnification of the regions
around the BDP is shown in the top right corner for each panel. The high
symmetry points refer to the trigonal Brillouin zone (BZ) shown in green in
(e). The BZ of the monoclinic 1/6-phase is illustrated in gray in (e). (f):
Total density of states (DOS) at ambient conditions (normal cell) and for the
1/6-phase at 6.5 GPa. The green arrow indicates the shift of the dimer bonding
states to lower energies with increasing pressure. (g): Comparison of the band
structure along the $q$-direction of the 1/6-phase with the normal cell band
structure.

### Density functional theory

To address precisely this question and to elucidate the relation between the
formation of dimers and the electronic structure, we performed density
functional theory (DFT) calculations based on the experimentally determined
lattice structures. Calculations for 4 different structures were performed,
namely (i) the trigonal structure, (ii) the pure 1/5-phase, (iii) the
1/6-phase at 6.5 GPa and (iv) the 1/6-phase at 42 GPa (cf. structural
parameters in the supplementary information). The electronic bands obtained
for the 1/5- and 1/6-phase have been unfolded onto the original Brillouin zone
of the trigonal structure, using the approach described in Ref. Ku  _et al._ ,
2010. In this way the electronic bands for the three different structures can
be compared directly.

We first discuss the evolution of the bulk DFT band structure with increasing
hydrostatic pressure. As described very nicely in Ref. Bahramy  _et al._ ,
2018, the chalcogenide p-orbital manifold is the most important one for the
formation of Dirac-cones, topological surface states and topological surface
resonances. In Fig. 5 a)–d) we therefore show so-called fat bands in which the
Te $5p_{x,y,z}$-character is represented by the line thickness and the colour.

The results for the trigonal phase in Fig. 5 a) agree very well with
previously published DFT- and ARPES-resultsQian  _et al._ (2014), including
the van-Hove singularity at the Fermi-level along the $AL$-direction. Our DFT
band structure also exhibits precisely the features that have been discussed
in Ref. Bahramy  _et al._ , 2018: Firstly, there is a type II bulk Dirac point
(BDP) along the $\Gamma A$-direction, which is protected by the
$C_{3v}$-symmetry of the lattice. More to the point, the $C_{3v}$-symmetry
forbids hybridization of Te $5p_{x,y}$ and $5p_{z}$ bands along the $\Gamma
A$-direction so that there is no hybridization gap. The two bands therefore
cross at the BDP, forming states that correspond to massless Dirac-fermions.
In the present case, however, the BDP is above the Fermi-level. Secondly,
inverted band gaps of the bulk electronic structure are obtained as well.
These inverted band gaps are expected to result in topological surface states
and resonances Bahramy  _et al._ (2018), which indeed appears to be consistent
with a very recent ARPES-study Nicholson  _et al._ (2021).

Figure 6: Charge density distribution of the bonding dimer states for the
1/6-phase at 6.5 GPa (further explanations in the text). Left: Cuts through
the charge density parallel to two different planes placed into the lattice
structure. The short Ir-Ir distances within a dimer are indicated by red
lines. Gray shaded ring-like structures indicate the charge distribution of
the bonding states of the dimers. Right: Contour plots of the charge densities
of these covalent bonds for plane i and plane ii. Position of the planes is
indicated in the left panel.

Using the structural refinements described above as input for DFT, we can now
address the evolution of the electronic structure with increasing hydrostatic
pressure. As is obvious from Figs. 5 b)–d), the pressure-induced appearance of
dimers causes drastic changes in the electronic band structure: Starting from
the triclinic 1/5-phase, a strong splitting of the bands is apparent with
states entering the energy regions of the inverted band gaps. The band
crossing of the BDP is lost and a gap opens, which corresponds to the
formation of massive Dirac states, due to the broken $C_{3v}$-symmetry. Upon
increasing the applied pressure further, the monoclinic 1/6-phase occurs. In
this phase the band gap at the former BDP opens up further.

By comparing the energies of the $5p_{z}$-states at $\Gamma$ and $A$, one can
see that the energy spread of these states increases with increasing pressure
as expected from the reduced $c/a$-ratio. But also the $5p_{xy}$-type bands
change strongly. In particular we also find that the van-Hove singularity
along the $AL$-direction is pushed further below the Fermi-level in agreement
with in Ref. Qian  _et al._ , 2014.

Along the $\Gamma L$-direction, which is parallel to the the wave vectors of
the ordered dimers, a gap opens between the occupied and the unoccupied states
(Fig. 5 g)) This agrees perfectly with the earlier conclusion that –in direct
space– the electrons become confined to the two-dimensional layers between
neighbouring dimer-walls Pascut  _et al._ (2014a); Park  _et al._ (2021).

Fig. 5 f) shows the evolution of the density of states (DOS) with pressure.
Also here the drastic changes in the electronic structure can clearly be
discerned. Indeed, the changes caused by applying $P$ correspond closely to
what has been found for the 1/5-phase at ambient pressure but lower
temperatureSaleh and Artyukhin (2020). Most importantly for the following
discussion are the bonding states formed by Te $5p$ and Ir $5d$ that are
pushed out of the broad band continuum between 0 eV and -6 eV, once the dimers
form. Using the results from our XRD-measurements, we can now follow the
evolution of these bonding dimer-states with increasing $P$. As can be
observed in Fig. 5 f), these bonding dimer-states shift very strongly to lower
and lower energies with $P$ growing from 1.4 GPa to 41 GPa. Indeed there is a
huge shift of about 2 eV, which amounts to a relative change of approximately
30 %.

The energy resolved charge distribution of these bonding dimer-states is
illustrated in Fig. 6. As before, the short Ir-Ir distances within the dimers
in panel a) are indicated by red lines. Inspection of Fig. 6 a) reveals a
ring-shaped charge-density distribution going around every short Ir-Ir link
(gray colored charge clouds which correspond to an isolevel of $6\times
10^{-4}$ e/Å-3). This can be also seen in the cuts through the charge density
along the planes i) and ii) in Fig. 6 a). The same contour plots are also
shown in more detail in Figs. 6 b) and c). Note the strong distortion of the
angle $\alpha$ of the Ir-Te-Ir links that is clearly apparent in Figs. 6 a,b).

The most bonding states of the dimer are therefore not due to direct Ir-Ir
interactions. A detailed analysis of the chemical bonds in IrTe2 indeed showed
that direct Ir-Ir bonding is not relevant Saleh and Artyukhin (2020). Instead
we find that the ring-shaped covalent bond, which is formed by Ir $5d$ and Te
$5p$ states, plays a key role for the stabilization of the dimers. This is
verified by the strong $P$-driven stabilization of this bond, which goes hand
in hand with the experimentally observed change in $\alpha$.

##  III Discussion

The important role of the ring-shaped covalent bonds for the dimers has also
been found in Ref. Saleh and Artyukhin, 2020. The key new result of the
present study concerns the stabilization of these bonds: Our XRD data reveals
that the bond angle $\alpha$ changes drastically with pressure, while the Ir-
Te distances change only slightly and continuously across the phase
transitions. At the same time, our DFT results show that with increasing $P$
the localized covalent bonds are pushed out of the broad band continuum formed
by itinerant Ir $5d$ and Te $5p$ states. The very large energy shifts of up to
2 eV cannot be explained by the small changes of the Ir-Te bond lengths,
which, furthermore, change by essentially the same amount for Ir-sites within
and outside a dimer. We therefore conclude that $\alpha$ is the essential
structural degree of freedom for the stabilization of the ring-shapes covalent
bond, which correspond to a valence bond on two edge-sharing IrTe6-octahedra
formed by Te $5p$\- and Ir $5d$-states.

One way to rationalize this, is to consider two Wannier-orbitals of
$e_{g}$-symmetry on each of the two edge-sharing IrTe6-octahedra of a dimer.
In the edge-sharing geometry, the overlap of these two orbitals changes very
strongly with $\alpha$. More specifically, the overlap increases rapidly as
$\alpha$ deviates from $90^{\circ}$. As a result, the observed reduction of
$\alpha$ towards $70^{\circ}$ with increasing $P$ yields a strong
stabilization of the valence bond between the two IrTe6-octahedra.

There are two key features of this mechanism: First, it is essentially local,
since it corresponds to the formation of a valence bond centered on two edge-
sharing IrTe6-octahedra. Importantly, this valence bond is not stabilized by
direct Ir-Ir interactions, which is very surprising considering strong
shortening of the Ir-Ir distance within a dimer. Second, the strong change of
the electronic binding energy with $\alpha$ implies a very strong coupling of
electrons and phonons. The large structural distortions observed
experimentally directly reflect this strong electron-phonon coupling.

The mechanism identified in this study is different from other mechanisms
discussed previously, especially those referring to weak coupling scenarios
based on the electronic band structure. Such scenarios are not easily
reconciled with the various $\vec{q}_{n}$ observed experimentally, because
they involve specific wave vectors defined by the band structure of the
trigonal phase. The local $\alpha$-dependent hybridization, however, is not
tied to details of the band structure, i.e. it can easily accommodate the
different and often coexisting $\vec{q}_{n}$ found in experiment. We emphasize
that the specific type of long-range order under certain conditions, i.e. the
specific $\vec{q}_{n}$, may very well be determined by the underlying
electronic band structure. According to our analysis, however, the primary
stabilization mechanism of the dimers is the local $\alpha$-dependent
hybridization between neighboring IrTe6-octahedra.

The formation of localized valence bonds will compete against the
delocalization of charges in band-like states. In other words, the local
stabilization mechanism must be strong enough to pull the bonding states out
of the broad band continuum. According to DFT this is indeed the case here.
Nonetheless, this competition may explain why so far no phase, in which all
Ir-sites are part of a valence bond –or a dimer for that matter– have been
observed and all valence bond crystals still feature mobile charge carriers.

We also note that the valence bond formation described here does not involve
charge order in the sense of Ir-sites with significantly different valences.
In the present multi-site situation with extended Ir- and Te-states it is
hardly possible to assign a certain charge to a specific lattice site. In
addition we are dealing with very similar electron affinities of Ir and Te. A
charge ordering scenario therefore appears to be inadequate. Notwithstanding,
the valence bond crystals of course posses inequivalent Ir-sites which, for
instance, explain the peak splittings observed in x-ray photo emission
spectroscopyNicholson  _et al._ (2021) – even without different valences of
Ir.

The formation of the valence bonds involves changes in the occupied Te- and
Ir-states, which is at least qualitatively consistent with the
depolymerization scenarios discussed earlierOh  _et al._ (2013). Referring to
this scenario, our analysis provides a mechanism underlying the
depolymerization: The formation of strong intra-layer valence bonds on
neighboring edge-sharing IrTe6-octahedra results in a weakening of the inter-
layer Te-Te bonds.

Since the stabilizing mechanism of the valence bonds is essentially local, it
does not depend on details of the band structure and should be weakly
dependent on doping. This mechanism can therefore be expected to be active in
doped IrTe2 as well. It is hence very interesting to ask whether local and
possibly dynamic valence bonds also exist in the trigonal phases and whether
they could play a role for the superconductivity in doped IrTe2. Going beyond
the specific case of IrTe2, it will now also be important to investigate the
relevance of this mechanism in other materials featuring extended $p$ and $d$
valence states, especially with regard to magnetic instabilities, topology and
superconducting states. These questions certainly deserve to by scrutinized in
future studies.

##  IV Methods

High-pressure X-ray diffraction: The high-pressure XRD measurements have been
performed at the beamline ID27 of the European Synchrotron Radiation Facility
(ESRF) in Grenoble, using a monochromatic beam with a photon energy of 33 keV
($\lambda$=0.3738 Å) and a spot size of 3x3 $\mu$m. High-quality single
crystals have been loaded into a membrane-driven diamond anvil cell (DAC) with
helium as pressure transmitting medium. The pressure inside the DAC has been
monitored in-situ using the R1,2 fluorescence of Cr-centers in a ruby sphere
close to the sample. For the present temperature dependent measurements
between room temperature and 420 K, the DAC has been installed in an oven,
which was itself mounted on a single axis ($\omega$) goniometer that has been
equipped with a MARCCD area detector for efficient detection of the diffracted
intensities.

X-ray data analysis and structure refinement: All data processing was
performed in the CrysAlisPro software suite (version 171.39.46) Rigaku Oxford
Diffraction (2018). Empirical absorption correction was applied using
spherical harmonics, implemented in SCALE3 ABSPACK scaling algorithm. The
subsequent structure solution and weighted fullmatrix least-squares refinement
on F2 were done with SHELXT-2014/5 (Ref. Sheldrick, 2015a) and SHELXL-2018/3
(Ref. Sheldrick, 2015b) as implemented in the WinGx 2018.3 program suite (Ref.
Farrugia, 2012). As reciprocal space coverage is significantly reduced by the
DAC, the crystal structures were exclusively refined with isotropic
displacement parameters. The ratio of independent reflections to refined
parameters exceeds a value of ten for all data sets, with the exception of the
data set for the trigonal phase measured at $P=0.4$ GPa. The parameters of the
data collection and the results of the structural refinement are summarized in
Supplementary Table. The atomic positions and isotropic displacement
parameters are listed in Supplementary Table.

Density functional Theory: The first-principle calculations have been done
using the FPLO package (version 18) Koepernik and Eschrig (1999). We used the
local density approximation (LDA) and the generalized gradient approximation
(GGA) of the exchange-correlation potential as parameterized by Perdew, Wang
(Ref. Perdew and Wang, 1992) and Perdew, Burke and Ernzerhof (Ref. Perdew  _et
al._ , 1996), respectively. We found very little differences in the resulting
band structure for the two approximations. Therefore we present the results
from the calculations performed within LDA. In order to account for spin-orbit
coupling the calculations were performed within the full 4-component Dirac-
Kohn-Sham theory as implemented in FPLO Eschrig  _et al._ (2004). The total
density was converged on a grid of $24\times 24\times 24$ and $12\times
12\times 12$ irreducible k-points for the normal cell and the super cell
calculations, respectively. Brillouin zone integration was done using the
tetrahedron method.

## Acknowledgements

This research has been supported by the Deutsche Forschungsgemeinschaft
through the the projects C06 and C09 of the SFB 1143 (project-id 247310070)
and the Würzburg-Dresden Cluster of Excellence on Complexity and Topology in
Quantum Matter–ct.qmat (EXC 2147, project-id 390858490). The work at Rutgers
University was supported by the DOE under Grant No. DOE: DE-FG02-07ER46382.

## Author contributions

S.-W.C. and J.G conceived the high-pressure diffraction experiment. S.-W.C.
and J.Y provided IrTe2 single crystals. T.R, J.T., M.K., G.G., V.S. and JG
conducted the diffraction experiment at the ESRF. T.R., Q.S., M.K. and J.G.
analysed the diffraction data, T.R. performed the DFT-study. The manuscript
has been written with contributions from all authors.

## References

  * Stewart (2017) G. R. Stewart, Advances in Physics 66, 75 (2017).
  * Arpaia  _et al._ (2019) R. Arpaia, S. Caprara, R. Fumagalli, G. De Vecchi, Y. Y. Peng, E. Andersson, D. Betto, G. M. De Luca, N. B. Brookes, F. Lombardi, M. Salluzzo, L. Braicovich, C. Di Castro, M. Grilli, and G. Ghiringhelli, Science 365, 906 (2019).
  * Savary and Balents (2017) L. Savary and L. Balents, Reports on Progress in Physics 80, 016502 (2017).
  * Yan and Felser (2017) B. Yan and C. Felser, Annual Review of Condensed Matter Physics 8, 337 (2017).
  * Wehling  _et al._ (2014) T. O. Wehling, A. M. Black-Schaffer, and A. V. Balatsky, Advances in Physics 63, 1 (2014).
  * Rossnagel (2011) K. Rossnagel, Journal of Physics: Condensed Matter 23, 213001 (2011).
  * Yang  _et al._ (2017) H. Yang, S. W. Kim, M. Chhowalla, and Y. H. Lee, Nature Physics 13, 931 (2017).
  * Gye  _et al._ (2019) G. Gye, E. Oh, and H. W. Yeom, Phys. Rev. Lett. 122, 016403 (2019).
  * Bahramy  _et al._ (2018) M. S. Bahramy, O. J. Clark, B. J. Yang, J. Feng, L. Bawden, J. M. Riley, I. Marković, F. Mazzola, V. Sunko, D. Biswas, S. P. Cooil, M. Jorge, J. W. Wells, M. Leandersson, T. Balasubramanian, J. Fujii, I. Vobornik, J. E. Rault, T. K. Kim, M. Hoesch, K. Okawa, M. Asakawa, T. Sasagawa, T. Eknapakul, W. Meevasana, and P. D. C. King, Nature Materials 17, 21 (2018).
  * Nicholson  _et al._ (2021) C. W. Nicholson, M. Rumo, A. Pulkkinen, G. Kremer, B. Salzmann, M.-L. Mottas, B. Hildebrand, T. Jaouen, T. K. Kim, S. Mukherjee, K. Ma, M. Muntwiler, F. O. von Rohr, C. Cacho, and C. Monney, Communications Materials 2, 25 (2021).
  * Fei  _et al._ (2018) F. Fei, X. Bo, P. Wang, J. Ying, J. Li, K. Chen, Q. Dai, B. Chen, Z. Sun, M. Zhang, F. Qu, Y. Zhang, Q. Wang, X. Wang, L. Cao, H. Bu, F. Song, X. Wan, and B. Wang, Advanced Materials 30, 1801556 (2018).
  * Jiang  _et al._ (2020) J. Jiang, S. Lee, F. Fei, F. Song, E. Vescovo, K. Kaznatcheev, F. J. Walker, and C. H. Ahn, APL Materials 8, 061106 (2020), https://doi.org/10.1063/5.0011549 .
  * Kiswandhi  _et al._ (2013) A. Kiswandhi, J. S. Brooks, H. B. Cao, J. Q. Yan, D. Mandrus, Z. Jiang, and H. D. Zhou, Phys. Rev. B 87, 121107 (2013).
  * Kudo  _et al._ (2013) K. Kudo, M. Kobayashi, S. Pyon, and M. Nohara, Journal of the Physical Society of Japan 82, 085001 (2013), https://doi.org/10.7566/JPSJ.82.085001 .
  * Kamitani  _et al._ (2013) M. Kamitani, M. S. Bahramy, R. Arita, S. Seki, T. Arima, Y. Tokura, and S. Ishiwata, Phys. Rev. B 87, 180501 (2013).
  * Oh  _et al._ (2013) Y. S. Oh, J. J. Yang, Y. Horibe, and S.-W. Cheong, Phys. Rev. Lett. 110, 127209 (2013).
  * Pascut  _et al._ (2014a) G. L. Pascut, T. Birol, M. J. Gutmann, J. J. Yang, S.-W. Cheong, K. Haule, and V. Kiryukhin, Phys. Rev. B 90, 195122 (2014a).
  * Saleh and Artyukhin (2020) G. Saleh and S. Artyukhin, The Journal of Physical Chemistry Letters 11, 2127 (2020), pMID: 32079398, https://doi.org/10.1021/acs.jpclett.0c00012 .
  * Hsu  _et al._ (2013) P.-J. Hsu, T. Mauerer, M. Vogt, J. J. Yang, Y. S. Oh, S.-W. Cheong, M. Bode, and W. Wu, Phys. Rev. Lett. 111, 266401 (2013).
  * Chen  _et al._ (2017) C. Chen, J. Kim, Y. Yang, G. Cao, R. Jin, and E. W. Plummer, Phys. Rev. B 95, 094118 (2017).
  * Park  _et al._ (2021) S. Park, S. Y. Kim, H. K. Kim, M. J. Kim, T. Kim, H. Kim, G. S. Choi, C. J. Won, S. Kim, K. Kim, E. F. Talantsev, K. Watanabe, T. Taniguchi, S.-W. Cheong, B. J. Kim, H. W. Yeom, J. Kim, T.-H. Kim, and J. S. Kim, Nature Communications 12, 3157 (2021).
  * Yang  _et al._ (2012) J. Yang, Y. Choi, Y. Oh, A. Hogan, Y. Horibe, K. Kim, B. Min, and S. Cheong, Physical Review Letters 108, 116402 (2012).
  * Ootsuki  _et al._ (2012) D. Ootsuki, Y. Wakisaka, S. Pyon, K. Kudo, M. Nohara, M. Arita, H. Anzai, H. Namatame, M. Taniguchi, N. L. Saini, and T. Mizokawa, Phys. Rev. B 86, 014519 (2012).
  * Qian  _et al._ (2014) T. Qian, H. Miao, Z. J. Wang, X. Shi, Y. B. Huang, P. Zhang, N. Xu, L. K. Zeng, J. Z. Ma, P. Richard, M. Shi, G. Xu, X. Dai, Z. Fang, A. F. Fang, N. L. Wang, and H. Ding, New Journal of Physics 16, 123038 (2014).
  * Pascut  _et al._ (2014b) G. L. Pascut, K. Haule, M. J. Gutmann, S. A. Barnett, A. Bombardi, S. Artyukhin, T. Birol, D. Vanderbilt, J. J. Yang, S.-W. Cheong, and V. Kiryukhin, Phys. Rev. Lett. 112, 086402 (2014b).
  * Ku  _et al._ (2010) W. Ku, T. Berlijn, and C.-C. Lee, Phys. Rev. Lett. 104, 216401 (2010).
  * Rigaku Oxford Diffraction (2018) Rigaku Oxford Diffraction, “CrysAlisPro Software system version 1.171.39.46, Rigaku Corporation, Oxford,” (2018).
  * Sheldrick (2015a) G. M. Sheldrick, Acta Crystallographica Section A Foundations and Advances 71, 3 (2015a).
  * Sheldrick (2015b) G. M. Sheldrick, Acta Crystallographica Section C Structural Chemistry 71, 3 (2015b).
  * Farrugia (2012) L. J. Farrugia, Journal of Applied Crystallography 45, 849 (2012).
  * Koepernik and Eschrig (1999) K. Koepernik and H. Eschrig, Phys. Rev. B 59, 1743 (1999).
  * Perdew and Wang (1992) J. P. Perdew and Y. Wang, Phys. Rev. B 45, 13244 (1992).
  * Perdew  _et al._ (1996) J. P. Perdew, K. Burke, and M. Ernzerhof, Phys. Rev. Lett. 77, 3865 (1996).
  * Eschrig  _et al._ (2004) H. Eschrig, M. Richter, and I. Opahle,  _Relativistic Solid State Calculations, in: Relativistic Electronic Structure Theory, Part 2. Applications_ , edited by P. Schwerdtfeger, Vol. 13 (Elsevier, 2004) pp. 723–776.