# StyleDyRF: Zero-shot 4D Style Transfer for Dynamic Neural Radiance Fields

Hongbin Xu1, Weitao Chen2, Feng Xiao1, Baigui Sun2, Wenxiong Kang1
1South China University of Technology
2Alibaba Group
<EMAIL_ADDRESS>
###### Abstract

4D style transfer aims at transferring arbitrary visual style to the
synthesized novel views of a dynamic 4D scene with varying viewpoints and
times. Existing efforts on 3D style transfer can effectively combine the
visual features of style images and neural radiance fields (NeRF) but fail to
handle the 4D dynamic scenes limited by the static scene assumption.
Consequently, we aim to handle the novel challenging problem of 4D style
transfer for the first time, which further requires the consistency of
stylized results on dynamic objects. In this paper, we introduce
StyleDyRF111Our code and model weights are released here., a method that
represents the 4D feature space by deforming a canonical feature volume and
learns a linear style transformation matrix on the feature volume in a data-
driven fashion. To obtain the canonical feature volume, the rays at each time
step are deformed with the geometric prior of a pre-trained dynamic NeRF to
render the feature map under the supervision of pre-trained visual encoders.
With the content and style cues in the canonical feature volume and the style
image, we can learn the style transformation matrix from their covariance
matrices with lightweight neural networks. The learned style transformation
matrix can reflect a direct matching of feature covariance from the content
volume to the given style pattern, in analogy with the optimization of the
Gram matrix in traditional 2D neural style transfer. The experimental results
show that our method not only renders 4D photorealistic style transfer results
in a zero-shot manner but also outperforms existing methods in terms of visual
quality and consistency.

##  1 Introduction

Given a reference style image, neural style transfer on 3D scene aims to
render novel views which not only contain the target style but also hold
consistency across different viewpoints in 3D scene. The substantial
prerequisite of 3D style transfer is the advances in 2D image-based neural
style transfer [1, 2, 3, 4] that allows transferring arbitrary styles in a
zero-shot manner. With the flourishing of 3D implicit representation of neural
radiance fields (NeRF), many efforts have been conducted to combine the NeRF
representation with neural style transfer techniques. These prior studies [5,
6, 7, 8] have proved that 3D style transfer should optimize novel view
synthesis and style transfer jointly instead of naively stacking these
components together to avoid the problem of multi-view inconsistency or poor
stylization quality.

Nevertheless, all these approaches assume a static 3D scene without moving
objects. Despite their resounding success on static scenes, existing
techniques on 3D style transfer are still deficient in handling dynamic
scenes. Consequently, in this paper, we first consider the novel challenge of
zero-shot 4D style transfer (Fig. LABEL:fig:motivation) by relaxing the static
assumption in 3D style transfer to 4D dynamic conditions, made of both static
and moving/deforming objects. Specifically, the imperative requirements of
this task can be summarized as follows: 1) Zero-shot generalization towards
arbitrary styles; 2) Multi-view consistency of rendered stylized novel views;
3) Cross-time consistency of rendered stylized novel views;

Before elaborating on the detailed solution of this work, we would like to
first revisit the typical pipeline of existing 3D zero-shot style transfer
methods [9, 8] to explore the bottleneck. The typical pipeline contains 2 key
ingredients: 1) _Explicit feature volume_ : As a radiance field can
effectively capture the geometry and color information, the extension to
feature space can endow expressive visual feature representation with 3D prior
for great generalization performance. 2) _Style transformation on the feature
volume_ : The style transformation on the 3D-aware feature space can
effectively maintain the multi-view consistency of the predicted stylized
results.

In dynamic scenes, the explicit 3D feature volume suffers from the ambiguity
of correspondences on moving objects, leading to an inaccurate 3D feature
space. Furthermore, the existing 3D style transformations vary with the
changing 3D scenes over time, ignoring the global coherence across the time
sequence. In consequence, the core problem lies in: _How to endow these 2 key
components with cross-time consistency without losing their capability of
generalization and multi-view consistency?_

In this paper, we propose StyleDyRF to resolve the challenging zero-shot 4D
style transfer problem. According to the aforementioned discussion, our
StyleDyRF can be decomposed into 2 innovative designs: 1) Canonical Feature
Volume (CFV): To model the 4D temporal feature volume in the dynamic scene,
our core idea is to decompose the feature radiance field into a canonical 3D
feature volume and a deformation network based on the canonical space
assumption [10]. The deformation network learns to map the 4D point
$(x,y,z,t)$ at each time to the unified 3D canonical space, and sample feature
by interpolating the canonical feature volume to render the 2D feature map at
time $t$. 2) Canonical Style Transformation (CST): The Whitening&Coloring
Transformation (WCT) [3] in 2D style transfer reflects a direct matching of
feature covariance from content to style. To ensure the temporal consistency
of style transformation in 4D space, we attempt to learn the transformation
matrix in a data-driven manner [4]. Specifically, we estimate the whitening
and coloring transformation matrix with a light-weight 3D CNN on the global
canonical feature volume and a 2D CNN on style images respectively.
Furthermore, for photorealistic 4D style transfer, we also present a post-
processing module that can filter the distortions on the stylized novel views
with the guidance of the volume-rendered images from the backbone NeRF.

The contribution of this work is summarized as follows: 1) We first
investigate the novel challenge of zero-shot 4D style transfer, which has the
following requirements: zero-shot generalization to arbitrary styles, multi-
view consistency, and cross-time consistency. 2) We introduce StyleDyRF, a
novel zero-shot 4D style transfer framework that can effectively synthesize
stylized novel views within the canonical feature space of a radiance field.
3) Extensive experiments show that our StyleDyRF can achieve superior
performance in zero-shot 4D style transfer.

##  2 Related Work

2D Style Transfer. Neural style transfer aims to synthesize a new image that
can preserve the content structure of one image and meantime contain the style
pattern of another. The profound work [1] treats the style transfer as an
iterative optimization task to match the correlations (Gram matrix) on the
intermediate feature maps of pre-trained CNNs between images. To speed up
rendering, many followers [11, 2, 12, 4, 3, 13, 14, 15, 16] apply feature
transformations on the feature map extracted by pre-trained CNNs to
approximate the time-consuming optimization process. The feature
transformations can be achieved by matching the mean and variance of features
[2], matrix operation of whitening/coloring transformation [3, 4], self-
attention transformation [11, 13, 14], etc. Video style transfer techniques
extend previous progress in style transfer to videos by applying temporal
constraints [17, 18, 19, 20] across adjacent video frames. Though they can
generate smooth videos, the missing awareness of underlying 3D geometry limits
their applications in rendering consistent frames in arbitrary novel views.

3D Style Transfer. 3D style transfer is a burgeoning field aiming to generate
stylized content with multi-view consistency. A simple solution is to combine
the existing 2D style transfer techniques with neural radiance fields (NeRFs)
[21, 22, 23, 24, 10, 25, 26, 27] by first rendering novel images and then
applying style transformation to the images. Recent studies [5, 9, 6, 7, 28,
29, 30, 8, 31, 32] have shown that the naive combination of NeRF and 2D style
transfer may result in unexpected multi-view inconsistency and poor
stylization quality. Huang et al [5] first handle the 3D style transfer by
back-projecting image features to construct a point cloud and then perform
style transformations on the point cloud. StylizedNeRF [6] proposes a novel
mutual learning framework with a 2D image stylization network and NeRF for
style optimization. To embed the generalization ability into the 3D style
transfer model, [9] utilize a hypernetwork comprised of MLPs to transfer the
style information into the scene representation. StyleRF [8] achieves state-
of-the-art performance in 3D style transfer by transforming the content and
style feature in the 3D space. However, all these prior works concentrate on
tackling the 3D style transfer problem, neglecting the 4D scenes with dynamic
objects. The dynamic motion in 4D scenes may corrupt the static assumption in
all these prior works and thus deteriorate the rendering performance. In this
paper, we aim to handle the 4D style transfer in a zero-shot manner, and our
StyleDyRF can effectively handle the dynamic scenes and provide high-quality
stylization on novel view synthesis.

##  3 Method

Figure 1: The overview framework of StyleDyRF.

The overview framework of our proposed StyleDyRF is provided in Fig. 1, which
contains multiple stages for training. In the NeRF pre-training stage (Sec.
3.1), a dynamic NeRF is trained via volume rendering the images. In the
Canonical Feature Distillation stage (Sec. 3.2), the geometry branch of the
pre-trained dynamic NeRF is used to guide the rendering from the canonical
feature volume. In the Canonical Style Transformation stage (Sec. 3.3), we
learn a linear style transformation matrix with lightweight networks from the
canonical feature volume and the style image.

###  3.1 Preliminary of Dynamic NeRF Pretraining

We extend the basic NeRF [10] to a dynamic one by attaching a deformation
network comprised of MLPs parameterized by $\theta_{d}$. The deformation
network can estimate the deformation on the sampled point $x\in\mathbb{R}^{3}$
at time $t\in\mathbb{R}$:

${\Delta x}_{t}=\text{MLP}_{\theta_{d}}(x,t)$ (1)

Then we can project point $x$ back to the canonical space and sample the
corresponding feature from the canonical appearance volume
$V_{a}\in\mathbb{R}^{N_{x}\times N_{y}\times N_{z}\times C_{a}}$ and canonical
geometry volume $V_{g}\in\mathbb{R}^{N_{x}\times N_{y}\times N_{z}\times
C_{g}}$:

$(v_{a},v_{g})=(V_{a}[x+{\Delta x}_{t}],V_{g}[x+{\Delta x}_{t}])$ (2)

where $[x+{\Delta x}_{t}]$ means the trilinear interpolation on the volume
space based on the position of $x+{\Delta x}_{t}$.
$v_{a}\in\mathbb{R}^{C_{a}}$ and $v_{g}\in\mathbb{R}^{C_{g}}$ are the sampled
features. After that, the density $\sigma_{t}\in\mathbb{R}$ and color
$c_{t}\in\mathbb{R}^{3}$ can be computed via:

$(c_{t},\sigma_{t})=(\text{MLP}_{\theta_{c}}(v_{a},d,t),\text{MLP}_{\theta_{g}}(v_{g},d,t))$
(3)

where $d\in\mathbb{R}^{3}$ is the view direction. The color $c_{t}$ and
density $\sigma_{t}$ are respectively predicted with MLPs paramterized by
$\theta_{c}$ and $\theta_{\sigma}$.

Then, the approximated volume rendering along the ray $r$ is used to calculate
the pixel color:

$\displaystyle\hat{I}_{t}(r)=\sum_{i=1}^{N_{s}}\omega_{t,i}c_{t,i}$ (4)
$\displaystyle\text{where}\;\;\omega_{t,i}=\text{exp}\left(-\sum_{j=1}^{i-1}\sigma_{t,j}\delta_{j}\right)\left(1-\text{exp}(-\sigma_{t,i}\delta_{i})\right)$
(5)

where $N_{s}$ points are sampled along the ray $r$. $\sigma_{t,i}$ and
$c_{t,i}$ are respectively the density and color predicted by Eq. 3.
$\delta_{i}$ is the distance between two adjacent points sampled on the ray.
Then we can optimize the radiance fields by minimizing the following
photometric loss:

$L_{\text{pho}}=\|\hat{I}_{t}(r)-I_{t}(r)\|_{2}^{2}$ (6)

where $I_{t}$ is the ground truth image at time $t$.

###  3.2 Canonical Feature Distillation

To model the entire scene with deep features, we aim to find a volumetric
representation to capture the representative features distilled from pre-
trained CNNs like VGG [33]. In previous works, StyleRF [8] builds a feature
volume in 3D space and optimizes it by rendering the 2D feature maps to
approximate the ones extracted from given views. However, such a 3D feature
volume is insufficient to meet the need for 4D style transfer in dynamic
scenes, because of the motion ambiguity caused by static assumption. A naive
solution is to directly extend the 3D feature volume to a 4D one, it is too
memory-intensive to be embedded into the training framework. Furthermore, the
temporal consistency among different times should also be considered because
the moving points should also be transformed consistently. Consequently, we
propose Canonical Feature Volume (CFV) to extend the canonical space
assumption to the 3D feature space. It can provide a compact 4D representation
in the canonical 3D space and preserve the equivalent transformation across
time simultaneously.

With the deformation network pre-trained in Sec. 3.1, we can sample the
feature from the canonical feature volume $V_{f}\in\mathbb{R}^{N_{x}\times
N_{y}\times N_{z}\times C_{f}}$ to construct the rendered feature map:

$\hat{F}_{t}(r)=\sum_{i=1}^{N_{s}}w_{t,i}V_{f}[x+\text{MLP}_{\theta_{d}}(x,t)]$
(7)

where $w_{t,i}$ is calculated from Eq. 5.
$\hat{F}_{t}\in\mathbb{R}^{N_{h}\times N_{w}\times C_{f}}$ is the rendered
feature map and $C_{f}$ is the feature dimension. Note that the 2D feature map
is directly rendered without using any other MLP modules to aggregate the
multi-view consistency in the canonical 3D space.

Aiming to align the latent feature space of our canonical feature volume and a
pre-trained autoencoder $\phi$ [33, 4] pre-trained on large-scale 2D image
datasets, we further utilize the following loss functions for training:

$L_{\text{fea}}=\|\hat{F_{t}}-\phi_{\text{enc}}(I_{t})\|_{2}^{2}+\|\phi_{\text{dec}}(\hat{F}_{t})-I_{t}\|_{2}^{2}$
(8)

where $\hat{F}_{t}$ is the rendered 2D feature map of all pixels/rays.
$\phi_{\text{enc}}$ and $\phi_{\text{dec}}$ are respectively the encoder and
decoder networks of the pre-trained autoencoder [33].

To reduce the huge computation and memory cost of volume-based rendering, we
further apply the vector-matrix tensor decomposition [24] to factorize the
tensors into compact vectors and matrices.

###  3.3 Canonical Style Transformation

The combination of the canonical feature volume in Sec. 3.2 and the
deformation network in Sec. 3.1 can provide a compact representation of the 4D
feature space. In this section, our goal is to find a linear style
transformation matrix that is consistent on the 4D feature space. A naive
solution is to apply the adaptive style normalization technique proposed by
StyleRF [8] to the 3D feature volume at each frame. However, the sampled
points from the canonical space are exactly different at different times in a
4D scene, leading to a changing per-frame 3D feature volume. Since the
deferred style transformation in StyleRF depends on the mean and variance
information of the changing per-frame volume, it inevitably results in an
unexpected variation in the style transformation toward the whole scene thus
deteriorating the temporal consistency. Consequently, we propose canonical
style transformation to learn a universal whitening/coloring transformation
matrix from the 4D content scene and the 2D style image.

####  3.3.1 Universal Transformation on Canonical Space

Denote that $F_{s}\in\mathbb{R}^{C_{f}\times N_{h}N_{w}}$ is the flattened
feature map extracted by the pre-trained encoder $\phi_{\text{enc}}$ from
style image $I_{s}$. The canonical feature volume $V_{f}$ is reshaped to a
matrix form $F_{c}=\pi(V_{f})\in\mathbb{R}^{C_{f}\times N_{x}N_{y}N_{z}}$,
where $\pi(\cdot)$ is the reshaping function.

We aim to find the optimal linear transformation matrix
$T\in\mathbb{R}^{C_{f}\times C_{f}}$ that can transform the latent space of
the canonical feature volume $V_{f}$ to the feature space of $F_{s}$. The
style transfer problem [1] can be converted to an optimization problem to
minimize the Gram matrix between the content and style:

$T^{*}=\arg\min_{T}\frac{1}{C_{f}}\|\overline{F}_{cs}\overline{F}_{cs}^{T}-\overline{F}_{s}\overline{F}_{s}^{T}\|_{F}^{2},s.t.\>\>\overline{F}_{cs}=T\overline{F}_{c}$
(9)

where $\overline{F}_{c}=F_{c}-\mu(F_{c})$ and
$\overline{F}_{s}=F_{s}-\mu(F_{s})$. $\mu(\cdot)$ calculates the mean value of
the input and $T^{*}\in\mathbb{R}^{C_{f}\times C_{f}}$.

The optimal transformation matrix $T$ can be obtained when:

$T^{*}\overline{F}_{c}{\overline{F}_{c}}^{T}{T^{*}}^{T}=\overline{F}_{s}{\overline{F}_{s}}^{T}$
(10)

By applying singular value decomposition (SVD) to the covariance matrix, we
can get: $\overline{F}_{c}{\overline{F}_{c}}^{T}=W_{c}\Sigma_{c}{W_{c}}^{T}$,
$\overline{F}_{s}{\overline{F}_{s}}^{T}=W_{s}\Sigma_{s}{W_{s}}^{T}$.
Substitute them in Eq. 10:

$T^{*}W_{c}\Sigma_{c}{W_{c}}^{T}{T^{*}}^{T}=W_{s}\Sigma_{s}{W_{s}}^{T}$ (11)

Then it is easy to get the closed-form solution of $T^{*}$:

$T^{*}=T_{s}T_{c}=\left(W_{s}\Sigma_{s}^{\frac{1}{2}}{W_{s}}^{T}\right)\left(W_{c}\Sigma_{c}^{-\frac{1}{2}}{W_{c}}^{T}\right)$
(12)

3D Whitening Transform. Based on Eq. 12, we can obtain the whitened 3D feature
volume $\hat{F}_{c}\in\mathbb{R}^{C_{f}\times N_{x}N_{y}N_{z}}$:

$\tilde{F}_{c}=T_{c}\overline{F}_{c}=W_{c}\Sigma_{c}^{-\frac{1}{2}}{W_{c}}^{T}\overline{F}_{c}$
(13)

where $\tilde{F}_{c}$ satisfies the constraints of
$\tilde{F}_{c}{\tilde{F}_{c}}^{T}=I$, which means it is an uncorrelated
feature volume without any style.

3D Coloring Transform. Based on Eq. 12, we can obtain
$\overline{F}_{cs}\in\mathbb{R}^{C_{f}\times N_{x}N_{y}N_{z}}$ which contains
the target style satisfying the constraints:
$\overline{F}_{cs}{\overline{F}_{cs}}^{T}=\overline{F}_{s}{\overline{F}_{s}}^{T}$.

$\overline{F}_{cs}=T_{s}\tilde{F}_{c}=W_{s}\Sigma_{s}^{\frac{1}{2}}{W_{s}}^{T}\tilde{F}_{c}$
(14)

Then we re-center the feature volume with the mean value of the style:
$F_{cs}=\overline{F}_{cs}+\mu(F_{s})$.

Feature Rendering. Finally, we can render the transformed feature volume to 2D
feature map via volume rendering in Eq. 5.

$\hat{F}_{t,cs}(r)=\sum_{i=1}^{N_{s}}w_{t,i}\pi^{-1}(F_{cs})[\tilde{x}_{t}]$
(15)

where $\pi^{-1}$ is the inverse reshaping function that converts matrix
$\hat{F}_{cs}$ to a feature volume. Note that we only need to compute the
universal transformation for one time and we can render the novel views at
different time $t$ with the weight $w_{t,i}$ calculated by Eq. 5. The rendered
feature map is further fed to the pre-trained decoder $\phi_{\text{dec}}$ to
reconstruct the stylized RGB image:

$I_{t,cs}=\phi_{\text{dec}}(\hat{F}_{t,cs})$ (16)

####  3.3.2 Learning Transformation from Data

Solving the closed-form solution from Eq. 9 to 12 creates a huge burden in
memory and computation, especially the SVD operations on the whole 3D feature
volume. Hence, we aim to design an alternative way to estimate these linear
transformation matrices with feed-forward networks in a data-driven manner
instead of calculating the closed-form solution.

The 3D style transformations can be rewritten in one equation:

$\hat{F}_{t,cs}(r)=\sum_{i=1}^{N_{s}}w_{t,i}\left(\pi^{-1}\left(T_{s}T_{c}(\pi(V_{f})-\mu_{f})+\mu_{s}\right)[\tilde{x}_{t}]\right)$
(17)

where $T_{s}\in\mathbb{R}^{C_{f}\times C_{f}}$ is the coloring matrix and
$T_{s}\in\mathbb{R}^{C_{f}\times C_{f}}$ is the whitening matrix.
$\mu_{f}=\mu(\pi(V_{f}))$ is the mean value of the canonical feature volume
$V_{f}$. $\mu_{s}=\mu(F_{s})$ is the mean value of the style feature. We can
further organize the formula as follows:

$\hat{F}_{t,cs}(r)=\pi^{-1}\left(T_{s}T_{c}\left(\pi\left(\sum_{i=1}^{N_{s}}w_{t,i}V_{f}[\tilde{x}_{t}]\right)-\mu_{f}\right)+\mu_{s}\right)$
(18)

Substituting the corresponding terms of Eq. 18 with Eq. 8:

$\hat{F}_{t,cs}(r)=\pi^{-1}\left(T_{s}T_{c}\left(\pi\left(\hat{F}_{t}(r)\right)-\mu_{f}\right)+\mu_{s}\right)$
(19)

From the above equation, we can introduce the key design of the proposed
canonical style transformation according to the following findings:

1) $\mu_{f}$ is fixed value because the canonical feature volume $V_{f}$ is
fixed. Hence we can calculate $\mu_{f}$ in advance to retrench the
computation.

2) $T_{c}$ is only conditioned by the content feature volume $V_{f}$. We adopt
a lightweight 3D CNN $\psi_{\text{3D}}$ to process the content feature volume:

$V_{z}=\psi_{\text{3D}}(V_{f}-\mu_{f})$ (20)

where $V_{z}\in\mathbb{R}^{N_{x}\times N_{y}\times N_{z}\times C_{z}}$ is the
output volume. $C_{z}$ is the output channel dimension which is smaller than
the original feature dimension $C_{f}$. Then, we can compute the covariance
matrix of $V_{z}$ and adopt a MLP parameterized by $\theta_{cz}$ to predict
the whitening matrix:

$T_{c}=\text{MLP}_{\theta_{cz}}(\text{cov}(\pi(V_{z})))$ (21)

where $\text{cov}(\pi(V_{z}))\in\mathbb{R}^{C_{z}\times C_{z}}$ is the
covariance matrix of $\pi(V_{z})$. $\pi$ is the reshaping function to flatten
a volume to a matrix. The MLP maps the squeezed feature dimension from $C_{z}$
to the original $C_{f}$ in $\hat{T}_{c}\in\mathbb{R}^{C_{f}\times C_{f}}$.

3) $T_{s}$ is only conditioned by the style feature $F_{s}$. We can use a
lightweight 2D CNN $\psi_{\text{2D}}$ to compress the dimension of feature map
$F_{s}$:

$F_{z}=\psi_{\text{2D}}(F_{s}-\mu_{s})$ (22)

where $F_{z}$ has a channel dimension of $C_{z}$. The coloring matrix
$\hat{T}_{s}\in\mathbb{R}^{C_{f}\times C_{f}}$ can be estimated with another
MLP paramterized by $\theta_{sz}$ mapping the covariance matrix of $F_{z}$
from $C_{z}$ to the original dimension of $C_{f}$:

$T_{s}=\text{MLP}_{\theta_{sz}}(\text{cov}(\pi(F_{z})))$ (23)

4) Eq. 19 is an equivalent form of Eq. 17. The matrix transformations in Eq.
17 are applied on the whole 3D feature volume which is pretty computation-
intensive. Whereas, the rendered feature map $\hat{F}_{t}$ in Eq. 19 can be
computed in advance, and the computation can be significantly reduced because
only 2D feature map is required.

In summary, we replace the original closed-form matrix transformations in Eq.
19 with the predictions of feed-forward networks. The equivalent form in Eq.
19 can lower the computation complexity of the volume-based operations from
$\mathcal{O}(n^{3})$ in Eq. 17 to $\mathcal{O}(n^{2})$. It allows us to
conduct a universal style transformation towards the canonical volume and
consequently preserve the cross-time consistency. For training, the predicted
feature map $\hat{F}_{t,cs}$ can be supervised by traditional style loss [1]:

$\displaystyle
L_{\text{sty}}=\lambda_{\text{con}}*\|\phi_{\text{enc}}(\phi_{\text{dec}}(\hat{F}_{t,cs}))-\phi_{\text{enc}}(I_{t})\|_{2}^{2}+$
(24)
$\displaystyle\lambda_{\text{sty}}*\|\text{Gram}(\hat{F}_{t,cs})-\text{Gram}(\phi_{\text{enc}}(I_{s}))\|_{2}^{2}$

where the former term is the content constraint and the latter one is the
style constraint. $\text{Gram}(\cdot)$ calculates the Gram matrix [1] of the
input. In default, $\lambda_{\text{con}}=1$ and $\lambda_{\text{sty}}=10$.

Figure 2: Comparison of our StyleDyRF with other methods on Nvidia Dataset.
Our StyleDyRF produces better stylized novel views with temporal and multi-
view consistency in the provided samples of dynamic scenes. Figure 3:
Qualitative results of our StyleDyRF on DAVIS dataset. Our StyleDyRF can
render stylized novel views with realistic quality and 4D consistency at
different times.

####  3.3.3 Learning Photo-realistic Style Transfer

Photo-realistic 4D style transfer is another important application that aims
to render novel views without structural distortions. It is proved by prior
works [4] that the stylized results may contain apparent artifacts because of
the distortion caused by the pre-trained deep auto-encoder [33]. Our StyleDyRF
can be simply extended to a photo-realistic style transfer framework by
applying an additional post-processing network. Inspired by [4], we can apply
a convolution spatial propagation network (CSPN) [34] as a filter network for
distortions. The network takes the rendered novel views $\hat{C}_{t}$ by Eq. 4
and 5 as the conditional input and filters the stylized output
$\hat{F}_{t,cs}$ of Eq. 19.

$I_{t,p}=\text{CSPN}(\phi_{\text{dec}}(\hat{F}_{t,cs}),\text{Whiten}(\hat{C}_{t}))$
(25)

where the rendered RGB image $\hat{C}_{t}$ at novels view is firstly whitened
to obtain an uncorrelated map in the RGB space via $\text{Whiten}(\cdot)$
function which is a degraded 2D version of Eq. 13. To train the CSPN network,
we apply the following loss for training:

$L_{fil}=\|\phi_{\text{enc}}(I_{t,p})-\phi_{\text{enc}}(I_{t})\|_{2}^{2}+\|\nabla(I_{t,p})-\nabla(I_{t})\|_{2}^{2}$
(26)

where the former term is the content constraints and the latter one is the
gradient regularization loss on the image.

##  4 Experiments

We conduct qualitative experiments (Sec. 4.1) and quantitative experiments
(Sec. 4.2) to evaluate our method. Furthermore, we evaluate the proposed
method with ablation studies (Sec. 4.3) on the impact of the ingredients in
StyleDyRF and a new application of multi-style interpolation (Sec. 4.4). We
evaluate our method on 2 public datasets: Nvidia dataset [35] and DAVIS
dataset [36]. Following StyleRF [8], we select WikiArt [37] as our style
dataset. The Dynamic NeRF-pretraining stage requires 10,000 iterations, while
each of the remaining 2 training stages requires 2,500 iterations. The whole
training process takes about 1 day on a single NVIDIA V100. Further
implementation details are provided in the supplementary materials.

###  4.1 Qualitative Results

Fig. 2 shows the qualitative comparisons on the Nvidia dataset between our
StyleDyRF and other methods including: the state-of-the-art NeRF-based style
transfer method StyleRF [8], and a video-based style transfer method ReReVST
[20]. In the figure, StyleDyRF-A is the artistic style transfer model of our
proposed method, and StyleDyRF-P has an additional post-processing propagation
module introduced in Sec. 3.3.3. From the rendered novel views in the figure,
we can find that our StyleDyRF can generate novel views with better stylized
novel views compared with other methods. StyleRF fails to handle the dynamic
objects, resulting in vague artifacts in dynamic regions. Many detailed
texture information is dropped because of the excessive regularization in
ReReVST. We also provide the qualitative results on the DAVIS dataset in Fig.
2. It shows that our method can robustly generalize to new style patterns in a
zero-shot manner meantime preserving multi-view consistency and cross-time
consistency during stylization.

###  4.2 Quantitative Results

Method | _Balloon1_ | _Balloon2_ | _Jumping_ | _Skating_ | _Playground_ | _Truck_ | _Umbrella_ | _Mean_
---|---|---|---|---|---|---|---|---
LPIPS | RMSE | LPIPS | RMSE | LPIPS | RMSE | LPIPS | RMSE | LPIPS | RMSE | LPIPS | RMSE | LPIPS | RMSE | LPIPS | RMSE
WCT [3] | 0.347 | 0.201 | 0.351 | 0.186 | 0.341 | 0.204 | 0.375 | 0.193 | 0.303 | 0.191 | 0.336 | 0.163 | 0.360 | 0.204 | 0.345 | 0.192
LinearWCT [4] | 0.219 | 0.101 | 0.148 | 0.075 | 0.195 | 0.095 | 0.163 | 0.086 | 0.201 | 0.092 | 0.141 | 0.084 | 0.164 | 0.091 | 0.176 | 0.089
AdaIN [2] | 0.253 | 0.073 | 0.170 | 0.057 | 0.208 | 0.072 | 0.198 | 0.072 | 0.220 | 0.070 | 0.183 | 0.068 | 0.207 | 0.072 | 0.206 | 0.069
MCCNet [38] | 0.229 | 0.088 | 0.136 | 0.059 | 0.189 | 0.080 | 0.162 | 0.072 | 0.199 | 0.075 | 0.137 | 0.071 | 0.160 | 0.076 | 0.173 | 0.074
ReReVST [20] | 0.179 | 0.082 | 0.102 | 0.055 | 0.149 | 0.068 | 0.125 | 0.067 | 0.156 | 0.065 | 0.103 | 0.061 | 0.125 | 0.071 | 0.134 | 0.067
StyleRF [8] | 0.159 | 0.105 | 0.114 | 0.085 | 0.192 | 0.106 | 0.130 | 0.081 | 0.130 | 0.079 | 0.137 | 0.085 | 0.120 | 0.083 | 0.140 | 0.089
StyleDyRF-A | 0.094 | 0.060 | 0.095 | 0.047 | 0.086 | 0.055 | 0.092 | 0.047 | 0.096 | 0.060 | 0.103 | 0.053 | 0.100 | 0.051 | 0.095 | 0.053
StyleDyRF-P | 0.086 | 0.054 | 0.089 | 0.045 | 0.073 | 0.050 | 0.068 | 0.044 | 0.097 | 0.060 | 0.070 | 0.040 | 0.079 | 0.040 | 0.080 | 0.048

Table 1: Quantitative comparisons on short-range consistency. We compare the
consistency scores of LPIPS ($\downarrow$) and RMSE ($\downarrow$) between
stylized images at 2 adjacent novel views to evaluate the short-range
consistency.

Method | _Balloon1_ | _Balloon2_ | _Jumping_ | _Skating_ | _Playground_ | _Truck_ | _Umbrella_ | _Mean_
---|---|---|---|---|---|---|---|---
LPIPS | RMSE | LPIPS | RMSE | LPIPS | RMSE | LPIPS | RMSE | LPIPS | RMSE | LPIPS | RMSE | LPIPS | RMSE | LPIPS | RMSE
WCT [3] | 0.349 | 0.200 | 0.361 | 0.200 | 0.339 | 0.212 | 0.375 | 0.204 | 0.289 | 0.202 | 0.337 | 0.174 | 0.373 | 0.221 | 0.346 | 0.202
LinearWCT [4] | 0.223 | 0.094 | 0.183 | 0.090 | 0.260 | 0.120 | 0.206 | 0.097 | 0.248 | 0.106 | 0.170 | 0.085 | 0.229 | 0.103 | 0.217 | 0.099
AdaIN [2] | 0.236 | 0.067 | 0.195 | 0.065 | 0.250 | 0.078 | 0.226 | 0.073 | 0.238 | 0.074 | 0.199 | 0.061 | 0.253 | 0.080 | 0.228 | 0.071
MCCNet [38] | 0.222 | 0.082 | 0.171 | 0.074 | 0.250 | 0.098 | 0.209 | 0.081 | 0.248 | 0.089 | 0.162 | 0.068 | 0.220 | 0.095 | 0.212 | 0.084
ReReVST [20] | 0.181 | 0.078 | 0.133 | 0.061 | 0.218 | 0.096 | 0.181 | 0.088 | 0.207 | 0.082 | 0.124 | 0.053 | 0.185 | 0.084 | 0.176 | 0.077
StyleRF [8] | 0.198 | 0.119 | 0.133 | 0.068 | 0.279 | 0.141 | 0.199 | 0.109 | 0.193 | 0.098 | 0.194 | 0.096 | 0.172 | 0.089 | 0.195 | 0.103
StyleDyRF-A | 0.118 | 0.069 | 0.113 | 0.058 | 0.107 | 0.070 | 0.103 | 0.061 | 0.114 | 0.074 | 0.117 | 0.058 | 0.114 | 0.061 | 0.112 | 0.065
StyleDyRF-P | 0.112 | 0.063 | 0.111 | 0.058 | 0.101 | 0.068 | 0.086 | 0.060 | 0.113 | 0.073 | 0.084 | 0.046 | 0.097 | 0.050 | 0.101 | 0.060

Table 2: Quantitative comparisons on long-range consistency. We compare the
consistency scores of LPIPS ($\downarrow$) and RMSE ($\downarrow$) between
stylized images at 2 novel views which are far away in time sequence to
evaluate the short-range consistency.

Similar to the 3D style transfer task, there are few metrics for quantitative
evaluations of stylization quality in our 4D style transfer task. Previous
works [9, 8] attempt to evaluate the multi-view consistency of the generated
novel views of different methods. In our experiments, the evaluation of cross-
time consistency is also required to be evaluated in our benchmark besides the
multi-view consistency. Consequently, the evaluation also includes the
variation along the time sequence. To compute the quantitative metrics, we can
warp one view to another view according to the optical flow [39] using softmax
splitting [40] and evaluate the LPIPS score [41] and RMSE score to measure the
consistency after stylization. Following prior works [8, 9], we evaluate the
consistency in two different ways: 1) short-range consistency between adjacent
views, i.e. $t$ and $t+1$ frames in the testing video; 2) long-range
consistency between far-way novel views, i.e. $t$ and $t+T/3$ frames in the
test video with $T$ frames in total. The quantitative comparison of short-
range consistency and long-range consistency on the Nvidia dataset are
respectively provided in Tab. 1 and 2. The comparison includes: 1) Image-based
style transfer techniques: WCT [3], LinearWCT [4], AdaIN [2]; 2) Video-based
style transfer methods: MCCNet [38], ReReVST [20]; 3) NeRF-based style
transfer methods: StyleRF [8]. From the tables, it can be seen that our
StyleDyRF significantly outperforms other approaches in both short-range and
long-range consistency. It demonstrates that our StyleDyRF can effectively
stylize the novel’s views with consistency in dynamic scenes.

###  4.3 Ablation Studies

Figure 4: Ablation results of our proposed canonical feature volume. CFV can
effectively model dynamic objects in 4D scenes, providing coherent stylization
with the whole scene. Figure 5: Ablation results of our proposed canonical
style transformation. CST can better preserve the global consistency of the 4D
scene in the stylized outputs.

Ablation Study on Canonical Feature Volume. We first evaluate the design of
Canonical Feature Volume (CFV) in Fig. 4. In the figure, the baseline ”w/o
CFV” removes the CFV in our StyleDyRF, and it only utilizes a static 3D
feature volume during training. We can find that the dynamic objects are
invisible in the stylized results of ”w/o CFV”. The involvement of CFV can
effectively model the dynamic objects in 4D scenes meantime conducting
coherent style transformation with the whole scene.

Ablation Study on Canonical Style Transformation. We also evaluate the design
of Canonical Style Transformation (CST) discussed in Sec. 3.3 in Fig. 5. The
1st and 3rd row shows two different novel views, and the 2nd row shows the
zoomed region. For comparison, the baseline ”w/o CST” first renders the RGB
images and feeds them to the traditional style transfer method [4]. From the
comparison, our CST (3-rd column) produces much less distortions than the
baseline. It demonstrates the superiority of the proposed CST towards cross-
time consistency and multi-view consistency.

Ablation Study on Propagation Module. We further evaluate the effect of the
post-processing CSPN module discussed in Sec. 3.3.3 in Fig. 6. As the figure
shows, the structure distortions caused by style transformation in artistic
style transfer can be effectively filtered with the well-trained CSPN module.
It can provide high-quality stylization results in the 4D photo-realistic
style transfer task.

Figure 6: Ablation results of the spatical propagation module. The propagation
module in StyleDyRF can effectively filter the structure distortions [4]
caused by the pre-trained autoencoder.

###  4.4 Multi-style Interpolation

Figure 7: Multiple-style interpolation. Our StyleDyRF can smoothly interpolate
between different styles via interpolating the linear transformation matrix in
CST.

Since the designed CST is a linear transformation on the latent feature space,
our StyleDyRF can smoothly merge different style patterns by linearly
interpolating the coloring matrix $F_{s}$ and the mean feature $\mu_{s}$ in
Eq. 20. In Fig. 7, we provide the interpolation results of 4 different style
patterns in the same dynamic scene. It demonstrates the expressive
generalization ability of CST in the latent space and further shows the
potential of generalization to unseen style patterns via multi-style
interpolation.

##  5 Conclusion

In this paper, we first consider the problem of 4D style transfer that aims to
transfer arbitrary style patterns to the synthesized novel views in a dynamic
4D scene. To handle this issue, we propose StyleDyRF, a novel method that
models the 4D feature space with a canonical feature volume and learns the
style transformation matrix on the feature volume in a learning-based fashion.
We utilize a compact representation of canonical feature volume to model the
4D scene by deforming it with a pre-trained deformation network in dynamic
NeRF. To ensure temporal consistency, we propose a learnable fashion to
approximate the style transformation matrix based on the 3D canonical feature
volume and the style images. The experimental results demonstrate that
StyleDyRF can effectively render high-quality stylized novel views
consistently, and also be extended to potential applications for artistic 4D
scenes.

## References

  * [1] Leon A Gatys, Alexander S Ecker, and Matthias Bethge.  Image style transfer using convolutional neural networks.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2414–2423, 2016.
  * [2] Xun Huang and Serge Belongie.  Arbitrary style transfer in real-time with adaptive instance normalization.  In Proceedings of the IEEE international conference on computer vision, pages 1501–1510, 2017.
  * [3] Yijun Li, Chen Fang, Jimei Yang, Zhaowen Wang, Xin Lu, and Ming-Hsuan Yang.  Universal style transfer via feature transforms.  Advances in neural information processing systems, 30, 2017.
  * [4] Xueting Li, Sifei Liu, Jan Kautz, and Ming-Hsuan Yang.  Learning linear transformations for fast image and video style transfer.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3809–3817, 2019.
  * [5] Hsin-Ping Huang, Hung-Yu Tseng, Saurabh Saini, Maneesh Singh, and Ming-Hsuan Yang.  Learning to stylize novel views.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 13869–13878, 2021.
  * [6] Yi-Hua Huang, Yue He, Yu-Jie Yuan, Yu-Kun Lai, and Lin Gao.  Stylizednerf: consistent 3d scene stylization as stylized nerf via 2d-3d mutual learning.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 18342–18352, 2022.
  * [7] Fangzhou Mu, Jian Wang, Yicheng Wu, and Yin Li.  3d photo stylization: Learning to generate stylized novel views from a single image.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16273–16282, 2022.
  * [8] Kunhao Liu, Fangneng Zhan, Yiwen Chen, Jiahui Zhang, Yingchen Yu, Abdulmotaleb El Saddik, Shijian Lu, and Eric P Xing.  Stylerf: Zero-shot 3d style transfer of neural radiance fields.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8338–8348, 2023.
  * [9] Pei-Ze Chiang, Meng-Shiun Tsai, Hung-Yu Tseng, Wei-Sheng Lai, and Wei-Chen Chiu.  Stylizing 3d scene via implicit representation and hypernetwork.  In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 1475–1484, 2022.
  * [10] Albert Pumarola, Enric Corona, Gerard Pons-Moll, and Francesc Moreno-Noguer.  D-nerf: Neural radiance fields for dynamic scenes.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10318–10327, 2021.
  * [11] Yingying Deng, Fan Tang, Weiming Dong, Wen Sun, Feiyue Huang, and Changsheng Xu.  Arbitrary style transfer via multi-adaptation network.  In Proceedings of the 28th ACM international conference on multimedia, pages 2719–2727, 2020.
  * [12] Justin Johnson, Alexandre Alahi, and Li Fei-Fei.  Perceptual losses for real-time style transfer and super-resolution.  In Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part II 14, pages 694–711. Springer, 2016.
  * [13] Songhua Liu, Tianwei Lin, Dongliang He, Fu Li, Meiling Wang, Xin Li, Zhengxing Sun, Qian Li, and Errui Ding.  Adaattn: Revisit attention mechanism in arbitrary neural style transfer.  In Proceedings of the IEEE/CVF international conference on computer vision, pages 6649–6658, 2021.
  * [14] Dae Young Park and Kwang Hee Lee.  Arbitrary style transfer with style-attentional networks.  In proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 5880–5888, 2019.
  * [15] Lu Sheng, Ziyi Lin, Jing Shao, and Xiaogang Wang.  Avatar-net: Multi-scale zero-shot style transfer by feature decoration.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 8242–8250, 2018.
  * [16] Xiaolei Wu, Zhihao Hu, Lu Sheng, and Dong Xu.  Styleformer: Real-time arbitrary style transfer via parametric style composition.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 14618–14627, 2021.
  * [17] Dongdong Chen, Jing Liao, Lu Yuan, Nenghai Yu, and Gang Hua.  Coherent online video style transfer.  In Proceedings of the IEEE International Conference on Computer Vision, pages 1105–1114, 2017.
  * [18] Haozhi Huang, Hao Wang, Wenhan Luo, Lin Ma, Wenhao Jiang, Xiaolong Zhu, Zhifeng Li, and Wei Liu.  Real-time neural style transfer for videos.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 783–791, 2017.
  * [19] Manuel Ruder, Alexey Dosovitskiy, and Thomas Brox.  Artistic style transfer for videos and spherical images.  International Journal of Computer Vision, 126(11):1199–1219, 2018\.
  * [20] Wenjing Wang, Jizheng Xu, Li Zhang, Yue Wang, and Jiaying Liu.  Consistent video style transfer via compound regularization.  In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 12233–12240, 2020.
  * [21] Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng.  Nerf: Representing scenes as neural radiance fields for view synthesis.  Communications of the ACM, 65(1):99–106, 2021.
  * [22] Sara Fridovich-Keil, Alex Yu, Matthew Tancik, Qinhong Chen, Benjamin Recht, and Angjoo Kanazawa.  Plenoxels: Radiance fields without neural networks.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5501–5510, 2022.
  * [23] Cheng Sun, Min Sun, and Hwann-Tzong Chen.  Direct voxel grid optimization: Super-fast convergence for radiance fields reconstruction.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5459–5469, 2022.
  * [24] Anpei Chen, Zexiang Xu, Andreas Geiger, Jingyi Yu, and Hao Su.  Tensorf: Tensorial radiance fields.  In European Conference on Computer Vision, pages 333–350. Springer, 2022.
  * [25] Keunhong Park, Utkarsh Sinha, Jonathan T Barron, Sofien Bouaziz, Dan B Goldman, Steven M Seitz, and Ricardo Martin-Brualla.  Nerfies: Deformable neural radiance fields.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 5865–5874, 2021.
  * [26] Jiemin Fang, Taoran Yi, Xinggang Wang, Lingxi Xie, Xiaopeng Zhang, Wenyu Liu, Matthias Nießner, and Qi Tian.  Fast dynamic radiance fields with time-aware neural voxels.  In SIGGRAPH Asia 2022 Conference Papers, pages 1–9, 2022.
  * [27] Yu-Lun Liu, Chen Gao, Andreas Meuleman, Hung-Yu Tseng, Ayush Saraf, Changil Kim, Yung-Yu Chuang, Johannes Kopf, and Jia-Bin Huang.  Robust dynamic radiance fields.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13–23, 2023.
  * [28] Thu Nguyen-Phuoc, Feng Liu, and Lei Xiao.  Snerf: stylized neural implicit representations for 3d scenes.  arXiv preprint arXiv:2207.02363, 2022.
  * [29] Kai Zhang, Nick Kolkin, Sai Bi, Fujun Luan, Zexiang Xu, Eli Shechtman, and Noah Snavely.  Arf: Artistic radiance fields.  In European Conference on Computer Vision, pages 717–733. Springer, 2022.
  * [30] Lukas Höllein, Justin Johnson, and Matthias Nießner.  Stylemesh: Style transfer for indoor 3d scene reconstructions.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6198–6208, 2022.
  * [31] Yaosen Chen, Qi Yuan, Zhiqiang Li, Yuegen Liu, Wei Wang, Chaoping Xie, Xuming Wen, and Qien Yu.  Upst-nerf: Universal photorealistic style transfer of neural radiance fields for 3d scene.  arXiv preprint arXiv:2208.07059, 2022.
  * [32] Zhiwen Fan, Yifan Jiang, Peihao Wang, Xinyu Gong, Dejia Xu, and Zhangyang Wang.  Unified implicit neural stylization.  In European Conference on Computer Vision, pages 636–654. Springer, 2022.
  * [33] Karen Simonyan and Andrew Zisserman.  Very deep convolutional networks for large-scale image recognition.  arXiv preprint arXiv:1409.1556, 2014.
  * [34] Xinjing Cheng, Peng Wang, and Ruigang Yang.  Learning depth with convolutional spatial propagation network.  IEEE transactions on pattern analysis and machine intelligence, 42(10):2361–2379, 2019.
  * [35] Zhengqi Li, Simon Niklaus, Noah Snavely, and Oliver Wang.  Neural scene flow fields for space-time view synthesis of dynamic scenes.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6498–6508, 2021.
  * [36] Federico Perazzi, Jordi Pont-Tuset, Brian McWilliams, Luc Van Gool, Markus Gross, and Alexander Sorkine-Hornung.  A benchmark dataset and evaluation methodology for video object segmentation.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 724–732, 2016.
  * [37] Wendy Kan small yellow duck.  Painter by numbers, 2016.
  * [38] Yingying Deng, Fan Tang, Weiming Dong, Haibin Huang, Chongyang Ma, and Changsheng Xu.  Arbitrary video style transfer via multi-channel correlation.  In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 1210–1217, 2021.
  * [39] Zachary Teed and Jia Deng.  Raft: Recurrent all-pairs field transforms for optical flow.  In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part II 16, pages 402–419. Springer, 2020.
  * [40] Simon Niklaus and Feng Liu.  Softmax splatting for video frame interpolation.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5437–5446, 2020.
  * [41] Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang.  The unreasonable effectiveness of deep features as a perceptual metric.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 586–595, 2018.


# Pressure-induced magnetic and topological transitions in non-centrosymmetric
MnIn2Te4

Anumita Bose<EMAIL_ADDRESS>Solid State and Structural Chemistry
Unit, Indian Institute of Science, Bangalore 560012, India  Rajdeep Banerjee
<EMAIL_ADDRESS>Solid State and Structural Chemistry Unit, Indian
Institute of Science, Bangalore 560012, India  Awadhesh Narayan
<EMAIL_ADDRESS>Solid State and Structural Chemistry Unit, Indian
Institute of Science, Bangalore 560012, India

###### Abstract

The discovery of time-reversal-invariant topological states has drawn great
attention in recent decades. However, despite the potential of displaying a
variety of exotic physics, the study of magnetic topological phases lags
behind due to underlying added complexity of magnetism. In this work, we
predict the interplay of magnetism and topology in the non-centrosymmetric
ternary manganese compound MnIn2Te4, using first-principles calculations. At
ambient pressure, the ground state of the system is an antiferromagnetic
insulator. With the application of small hydrostatic pressure ($\sim$0.50
GPa), it undergoes a magnetic transition and the ferromagnetic state becomes
energetically favourable. At $\sim$2.92 GPa, the system undergoes a transition
into a Weyl semimetallic phase, which hosts multiple Weyl points in the bulk
and is associated with non-trivial surface Fermi arcs. Remarkably, we discover
that the number of Weyl points in this system can be controlled by pressure
and that these manifest in an anomalous Hall conductivity (AHC). In addition
to proposing a new candidate magnetic topological material, our work
demonstrates that pressure can be an effective way to induce and control
topological phases, as well as AHC, in magnetic materials. These properties
may allow our proposed material to be used as a novel pressure-controlled Hall
switch.

Introduction: In recent times, topological phases of matter have attracted
considerable attention in condensed matter physics and materials science. The
study of time-reversal-invariant topological states in quantum materials has
made tremendous progress and created fruitful outcomes in both the theoretical
and experimental fields of research Hasan and Kane (2010); Hasan and Moore
(2011); Qi and Zhang (2011); Ando (2013); Armitage  _et al._ (2018); Burkov
(2016); Gao  _et al._ (2019); Weng  _et al._ (2016); Kou  _et al._ (2017).

The interplay between magnetism and topology further leads to exotic phases,
such as quantum anomalous Hall effect (QAHE) He  _et al._ (2014); Liu  _et
al._ (2016); Nomura and Nagaosa (2011); Chang  _et al._ (2013), topological
magneto-electric effect Nomura and Nagaosa (2011), topological axion
insulators Li  _et al._ (2010), and topological superconductors Qi  _et al._
(2010). However, due to the added complexity of magnetism, the study of
magnetic topological phases has not progressed as rapidly. Traditionally, two
methods have been used to unite non-trivial topology and magnetism: (i) doping
magnetic elements into systems hosting non trivial topological phases Chang
_et al._ (2013); Liu  _et al._ (2017); Xu  _et al._ (2020), and (ii)
constructing hetero-structures out of the two Lang  _et al._ (2014); Mogi  _et
al._ (2017); Yasuda  _et al._ (2016). However, in both cases, systems are
usually far from satisfactory due to inhomogeneity induced in the doped
systems and inhomogeneous band bending near the interface in hetero-
structures. Also, since the coupling between topological and magnetic states
is usually weak and sensitive to the interface properties, experimental
realization of robust systems is quite challenging. Thus, it has become
crucial to find magnetic topological phases which are either intrinsic or can
be obtained via small perturbations.

In recent years, the study of MnBi2Te4 family of materials have provided a
promising platform to explore the combined effect of magnetism and topology
intrinsically Otrokov  _et al._ (2019a); Li  _et al._ (2019). These van der
Waals materials show a wide variety of topological states depending on the
number of layers Zhang  _et al._ (2019); Otrokov  _et al._ (2019b). Several
other intrinsic magnetic topological phases have been theoretically predicted
in the last decade Xu  _et al._ (2011); Wan  _et al._ (2011); Tang  _et al._
(2016); Hua  _et al._ (2018). Time reversal symmetry broken Weyl points have
been recently studied both theoretically and experimentally in centrosymmetric
crystals, such as Fe3GeTe2 Kim  _et al._ (2018), Co3Sn2S2 Wang  _et al._
(2018); Liu  _et al._ (2018) and Co2MnGa Belopolski  _et al._ (2019).
Importantly, these materials exhibit a giant anomalous Hall effect (AHE) as an
outcome of the Berry curvature around the Weyl nodes.

Figure 1: Structure and phase diagram of MnIn2Te4. (a) Tetragonal unit cell of
MnIn2Te4 with space group $I\bar{4}2m$ (number 121), comprising of two Mn,
four In and eight Te atoms which are denoted by pink, blue and yellow spheres,
respectively. The stacking of atomic layers is along the $c$ direction. FM
arrangement of Mn spins is depicted as red arrows. (b) Brillouin zone with the
high symmetry points and paths marked as blue dots and lines respectively.
Rectangular surface with blue dash-dotted line denotes the (010) surface with
projection of bulk high symmetry points marked as red dots. (c) The phase
transitions with pressure in MnIn2Te4 between magnetic insulator (shaded with
light red) and magnetic Weyl semimetal (shaded with light blue). Red line
shows the variation of gap between the top-most valence band and bottom-most
conduction band, as a function of hydrostatic pressure, which gradually
decreases with increasing pressure. Blue line indicates the number of Weyl
pairs formed between those two bands in the magnetic Weyl semimetallic phase.
Figure 2: Bulk electronic structure of ferromagnetic MnIn2Te4. (a) The zero
pressure band structure of ferromagnetic MnIn2Te4, which is a trivial
insulator with a direct band gap of nearly 250 meV at $\Gamma$. With
increasing hydrostatic pressure, the gap decreases. At 2.92 GPa, VBM and CBM
touch each other and the gap becomes zero, which is shown in (b). On
increasing the hydrostatic pressure further, overlap between the bands at the
Fermi energy increases, which leads to the FM Weyl semimetallic phase. (c) The
FM Weyl semimetallic phase at 6.76 GPa, where nodes appear near the $\Gamma$
point close to the Fermi energy.

Furthermore, external perturbations such as temperature Monserrat and
Vanderbilt (2016); Antonius and Louie (2016); Monserrat and Narayan (2019),
hydrostatic pressure Kirshenbaum  _et al._ (2013); He  _et al._ (2016); Zhang
_et al._ (2017); Sadhukhan  _et al._ (2022); Juneja  _et al._ (2018), strain
Liu  _et al._ (2014); Zeljkovic  _et al._ (2015); Narayan (2015); Bose and
Narayan (2021) and light Rodriguez-Vega  _et al._ (2021) allow manipulating
topological ordering and its underlying mechanisms. Among these stimuli, the
application of hydrostatic pressure is particularly of great utility, as it
allows studying the interplay of charge, spin, orbital and lattice degrees of
freedom under controlled conditions Mao  _et al._ (2018).

In this work, we report the first-principles prediction of topological
properties of non-centrosymmetric magnetic ternary compound MnIn2Te4. We show
that the ground state of MnIn2Te4 is an antiferromagetic (AFM) insulator and
the material undergoes a magnetic phase transition to a ferromagnetic (FM)
insulating phase at $\sim$0.5 GPa. Further increase in hydrostatic pressure
results in reduction of the band gap and the emergence of a semimetallic phase
with multiple pairs of Weyl points in the system. We confirm the presence of
Weyl points by calculating the resulting non-trivial Fermi arcs. Our results
also reveal a tunable material system where the number of Weyl points can be
controlled by hydrostatic pressure – these manifest in a non-zero anomalous
Hall conductivity (AHC) around the Fermi level. In a nutshell, we propose a
new member in the catalogue of magnetic topological materials where both the
magnetic ordering (AFM versus FM) and the topology can be controlled by
externally applied hydrostatic pressure. The system shows a pressure dependent
AHC, which makes the system a potential candidate for designing a pressure-
controlled Hall switch.

Methods:  Our first-principles calculations were performed within the density
functional theory framework as implemented in the Quantum Espresso package
Giannozzi  _et al._ (2009, 2017), using the Perdew-Burke-Ernzerhof exchange-
correlation functional Perdew  _et al._ (1996) and fully relativistic ultra-
soft pseudopotentials Vanderbilt (1990). Semi-empirical Grimme’s DFT-D2
correction Grimme (2006) was also included to take into account the van der
Waals forces. Self-consistent calculations for the bulk were performed using a
plane wave cutoff of $50$ Ry and $8\times 8\times 4$ $\Gamma$-centered
Monkhorst-Pack $k$-point grid Monkhorst and Pack (1976). We used an onsite
Hubbard $U$ correction for Mn $d$ orbitals, ($U=5.34$ eV) Liechtenstein  _et
al._ (1995). During structural optimization the system was relaxed until
Hellmann-Feynmann forces on the atoms became less than $10^{-3}$ Ry/bohr. In
order to create hydrostatic pressure on the system in our simulations, we
applied a chosen strain along the $a$, $b$ and $c$ directions, allowing the
atoms to fully relax. The structures for which the diagonal elements of the
stress tensor become equal were chosen for the particular hydrostatic pressure
value. We further constructed tight-binding models obtained from maximally
localized Wannier functions using the wannier90 code Mostofi  _et al._ (2008),
with Mn 3$d$, In 5$p$ and Te 5$p$ orbitals as the basis. In order to calculate
the position of the bulk Weyl nodes, we used the Nelder and Mead’s Downhill
Simplex Method Nelder (1965), as implemented in WannierTools Wu  _et al._
(2018), which takes a uniform $k$-mesh in the three-dimensional Brillouin zone
as a set of starting points for the calculation. After obtaining the positions
of gapless points, a small sphere is constructed around each and the enclosed
Weyl point’s chirality is determined by tracking the evolution of the sum of
hybrid Wannier charge centers as a function of the polar angle. Surface
Green’s function method for semi-infinite systems Guinea  _et al._ (1983);
Sancho  _et al._ (1984, 1985), implemented in WannierTools package was
employed to calculate the surface electronic spectra.

Structural properties: MnIn2Te4 is an existing compound which was first
synthesized in 1974 Range and Hübner (1975). This ternary manganese compound
crystallizes in a non-centrosymmetric tetragonal space group I$\bar{4}2m$
(number 121). The unit cell has two Mn atoms, four In atoms and eight Te
atoms, which are represented as pink, blue and yellow spheres, respectively,
in Fig. 1 (a). Due to the presence of Mn spins, the system shows a magnetic
nature with broken time reversal symmetry (TRS). Our calculated average
magnetic moment per Mn atom is very close to 5 ${\mu}_{B}$ [denoted as red
arrows in Fig. 1 (a)], suggesting a half-filed $d$-orbital manifold. At
ambient conditions (zero applied pressure), the ground state of the system is
found to be an AFM. Although application of hydrostatic pressure makes the FM
state energetically more stable compared to the AFM one, as we shall see. The
cell parameters for both the FM and AFM systems at ambient pressure obtained
from our ab-initio calculations are listed in TABLE 1 along with the
experimentally measured values Range and Hübner (1975); Döll  _et al._ (1991).
We find that the optimized lattice parameters match well with the experimental
values. At ambient pressure conditions, AFM phase is energetically more stable
(EAFM-EFM = 7.16 meV). However, at small hydrostatic pressure values of
$\sim$0.5 GPa, the FM state becomes energetically more favourable. Next, we
will look at the pressure variation of the electronic structure.

Table 1: Comparison of cell parameters for different magnetic orders. The calculated cell parameters obtained from first principles calculations for FM and AFM magnetic orders. Experimental lattice parameters are from Refs. Range and Hübner (1975); Döll  _et al._ (1991). Magnetic order |  $a=b$ (Å) |  $c$ (Å)
---|---|---
AFM |  6.138 |  12.229
FM |  6.172 |  12.324
Experimental |  6.191 |  12.382

Bulk electronic structure:  In Fig. 2, we examine the bulk electronic band
structures of the FM MnIn2Te4 system, at different values of hydrostatic
pressure, including spin orbit coupling (SOC), along the high symmetry
directions. At zero pressure, the system is a trivial magnetic insulator
[shown in Fig. 2 (a)] with a direct band gap of nearly 250 meV at the $\Gamma$
point. The top most valence band has mainly Te $p$ orbital character, while Te
$p$ and In $s$ orbitals contribute most to the lowest conduction band. For a
comparison of the band structure of the AFM phase, see supplemental material
abc . With increasing hydrostatic pressure, the gap between valence band
maximum (VBM) and conduction band minimum (CBM) shrinks continuously.
Variation of the direct band gap as a function of hydrostatic pressure can be
seen in Fig. 1 (c). The band gap decreases almost linearly with increasing
hydrostatic pressure. Here, the region shaded with light red shows the gapped
phase. As the hydrostatic pressure is further increased, the valence and
conduction band edges meet at the Fermi energy at the $\Gamma$ point, where
the gap vanishes and we observe a phase transition to a semimetallic phase at
$\sim$2.92 GPa, which can be seen from Fig. 2 (b). Fig. 2 (c) shows the bulk
bandstructure of the semimetallic phase at a representative pressure value of
6.76 GPa. The bands around $\Gamma$ near the Fermi energy get inverted,
creating gapless points around the center of the BZ. Although, we note that,
the nodes appear away from the high symmetric directions. For a list of the
position of Weyl nodes in the three-dimensional momentum space along with
their respective chirality values, see supplemental material abc . The blue
dots in Fig. 1 (c) indicate the number of Weyl point pairs present in the
semi-metallic phase near the Fermi energy. Remarkably, we find that the number
of Weyl points monotonically increases as one varies the hydrostatic pressure
in the system from 2.92 GPa to 8.77 GPa. Thus, hydrostatic pressure not only
creates Weyl points in our system, but also allows control over the number of
Weyl points. This intriguing feature may be directly probed in near-future
experiments. We further note that being non-centrosymmetric, MnIn2Te4 lacks
inversion symmetry. On the other hand, TRS is also broken in the system due to
its magnetic nature. This means that the Weyl points in this system occur in
absence of both the inversion and time reversal symmetry, indicating the
robustness of the these points to structural distortions.

Figure 3: Surface electronic spectra. (a) The spectrum on the (010) surface
for a hydrostatic pressure value 6.76 GPa. Existence of surface states is
observed along $\bar{M}-\bar{\Gamma}-\bar{R}$ direction. (b) The Fermi arc as
a function of the in-plane momenta $k_{x}$ and $k_{z}$ at the Fermi level
[denoted with the black dashed line in (a)].

Surface properties: The existence of non-trivial states on the surface is one
of the most significant consequences of the occurrence of bulk Weyl points in
a three-dimensional system. In order to further demonstrate non-trivial bulk
topology of MnIn2Te4, we explore the surface electronic spectra of the system
on the (010) surface. On this surface, bulk high symmetry points M, $\Gamma$
and Z project on $\bar{M}$, $\bar{\Gamma}$ and $\bar{Z}$, respectively, as
shown in Fig. 1 (b). Fig. 3 (a) shows the (010) surface state along
$\bar{M}-\bar{\Gamma}-\bar{R}$ direction connecting the bulk Weyl cones
extended in the energy window of -0.015 eV to 0.08 eV, for the representative
pressure value of 6.76 GPa. With the Weyl points with opposite chirality being
energetically separated, we expect to observe Fermi arcs for a range of energy
values. Surface states as a function of in-plane momenta $k_{x}$ and $k_{z}$,
taken at constant energy equal to the Fermi energy $E=E_{F}$ [shown by the
dashed line in Fig. 3 (a)], is shown in Fig. 3 (b). Here the arcs, connecting
the Weyl points with opposite chirality, take the shape of an ellipse. As one
goes towards higher energies, the size of the ellipse increases, which can
also be observed from Fig. 3 (a). We have also studied the surface spectra for
other surfaces, for e.g. (100) and (110), and obtained non-trivial surface
states.

Figure 4: Pressure tuned anomalous Hall conductivity. Violet, green and red
curves denote the AHC ($\sigma_{xy}$) for hydrostatic pressure values 3.50
GPa, 6.76 GPa and 8.77 GPa, respectively. For 3.50 GPa, $\sigma_{xy}$ at the
Fermi level is small but non-zero. At 6.76 GPa, it experiences a sudden jump,
which is saturated for higher pressure values. With increasing pressure, the
number of Weyl points increases, leading to an increase in $\sigma_{xy}$ at
Fermi level with pressure.

Anomalous Hall effect:  Next, we explore another intriguing aspect of our
system, namely the anomalous Hall effect. Recent studies have established that
intrinsic AHC, $\sigma_{xy}$, which originates from a nonzero Berry curvature,
can be induced in systems with Weyl points Fang  _et al._ (2003); Burkov
(2014); Nayak  _et al._ (2016); Manna  _et al._ (2018). The AHC is defined as
the integral of the Berry curvature of Bloch wave functions over the entire
BZ. For the $n$-th band with $z$ component of the Berry curvature
$\Omega_{n}^{z}(\textbf{k})$, it reads

$\sigma_{xy}=\frac{e^{2}}{\hbar}\sum_{n}\int_{BZ}f(\epsilon_{n}(\textbf{k}))\Omega_{n}^{z}(\textbf{k})\frac{d\textbf{k}}{(2\pi)^{3}},$
(1)

where $f(\epsilon_{n}(\textbf{k}))$ denotes the Fermi-Dirac distribution. We
next study the AHC for the Weyl phase of our predicted material.

Fig. 4 shows the variation of AHC component $\sigma_{xy}$ with energy for
different values of the applied hydrostatic pressure. We can observe, for 3.50
GPa, $\sigma_{xy}$ is quite small near the Fermi energy, although it increases
for 6.76 GPa and gets saturated for higher pressure values. From Fig. 1 (c) we
can see that the number of Weyl points increases almost linearly with the
increase in hydrostatic pressure. This observation may encourage us to expect
a linear increase in the AHC, as opposed to the observed saturation. To
explain this phenomenon, instead of looking at the total number of Weyl
points, we focus only on the Weyl points very close to the Fermi energy. Here,
to include the nearest Weyl point energy at 3.50 GPa, we take the window of
$|E-E_{F}|\leq$ 0.015 eV, so that at 3.50 GPa we have one Weyl point ‘close’
to the Fermi energy. If this Weyl point is responsible for the small AHC at
3.50 GPa, then we expect the number of Weyl points close to the Fermi energy
to increase when the pressure is increased from 3.50 GPa to 6.76 GPa. We see
that indeed the number of Weyl points close to Fermi energy, jumps from 1 to
5. A further increase in the pressure from 6.76 GPa to 8.77 GPa does not have
much effect on the AHC, and the number of Weyl points close to the Fermi
energy also remain fixed at 5. We note that the system becomes metallic at
10.43 GPa, and there are 2 Weyl points close to the Fermi energy at that
pressure. The above observations indicate that the number Weyl points very
close to the Fermi energy are the ones that control the behaviour of AHC in
the vicinity of the Fermi level.

This property enables us to propose this system as a pressure-controlled Hall
switch. In an electrical switch a voltage controls the flow of current through
a circuit, and beyond a certain voltage, one obtains a current flow. In an
analogous manner, here, the pressure controls the AHC at the Fermi energy. Up
to 3.50 GPa pressure, the AHC is close to zero. Beyond a certain pressure
(6.76 GPa), it becomes significant ($\sim$51 $\ohm^{-1}cm^{-1}$) and maintains
the AHC value up to 8.77 GPa. The breakdown pressure of the switch is 10.43
GPa, where the system becomes metallic.

Summary and conclusions: In summary, we have introduced a non-centrosymmetric
magnetic material MnIn2Te4 as a platform to explore the interplay of magnetism
and topology. The system is an AFM at ambient pressure conditions and
undergoes a magnetic transition to the FM phase at a pressure value $\sim$0.50
GPa. Increase in hydrostatic pressure in FM MnIn2Te4 further induces a
transition from a trivial magnetic insulator to a magnetic Weyl semimetal at
$\sim$2.92 GPa, which is experimentally easily achievable. Remarkably, we have
shown that the number of Weyl pairs can be tuned by the applied hydrostatic
pressure. The existence of these robust Weyl points is confirmed by the
presence of non-trivial surface state near the Fermi level and Fermi arcs. As
a result of the occurrence of Weyl points, the non-trivial magnetic semi-
metallic phase gives rise to a pressure dependent non zero AHC at the Fermi
level in the system. This property paves way for its possible application as a
pressure-controlled Hall switch. Our findings demonstrate that pressure is a
useful tool to unite magnetism with topology and hence produce non-trivial
magnetic topological phases.

Acknowledgements: AB acknowledges Prime Minister’s Research Fellowship for
financial support. RB and AN acknowledge support from the start-up grant
(SG/MHRD-19-0001) of the Indian Institute of Science. AN also acknowledges
support from DST-SERB (project number SRG/2020/000153).

##  I Supplementary Material

In this supplementary material, we present the following: (I) bandstructure of
the antiferromagnetic (AFM) MnIn2Te4, (II) density of states of ferromagnetic
MnIn2Te4, (III) position of Weyl points in the three-dimensional momentum
space, and (IV) anomalous Hall conductivity (AHC).

##  II Bandstructure of antiferromagnetic MnIn2Te4

Antiferromagnetic MnIn2Te4 is a trivial magnetic insulator with a direct band
gap of $\sim$0.46 eV at $\Gamma$ (see Fig. S1, which is almost double that of
the ferromagnetic one, discussed in main text). To explore the Weyl physics in
antiferromagnetic system, we expect the application of higher pressure. As we
noted in the main manuscript, the ferromagnetic structure becomes
energetically favorable upon application of small pressure (see main
manuscript).

Figure S1: The band structure of antiferromagnetic MnIn2Te4 at ambient
pressure.

##  III Density of States

Fig. S2 shows the partial density of states of ferromagnetic MnIn2Te4. At non-
zero pressures, contribution from the Mn $d$ orbitals increases near the Fermi
energy.

Figure S2: Orbital contribution to the density of states of the ferromagnetic
MnIn2Te4 at ambient (left) and 6.76 GPa (right) pressure.

##  IV Position of Weyl nodes

Table 2 lists the location of Weyl points in the momentum space, along with
their chirality values for the ferromagnetic MnIn2Te4, at 6.76 GPa. The system
contains six Weyl points near the Fermi level – three points with chirality +1
and the other three with chirality -1. Distribution of these Weyl points in
the three-dimensional momentum space is shown in Fig. S3.

Table 2: Position of bulk nodes at 6.76 GPa. $k_{x}$ (1/Å) |  $k_{y}$ (1/Å) |  $k_{z}$ (1/Å) |  Chirality
---|---|---|---
0.09629025 |  0.01028182 |  0.00658574 |  $+$1
0.05202581 |  $-$0.09052557 |  $-$0.00492802 |  $+$1
$-$0.04290352 |  $-$0.07375086 |  0.05560153 |  $+$1
$-$0.09621852 |  $-$0.00930774 |  0.00493105 |  $-$1
$-$0.02149350 |  $-$0.09390641 |  0.00941639 |  $-$1
0.08997877 |  $-$0.03492382 |  $-$0.02169327 |  $-$1
Figure S3: Distribution of Weyl points in the three dimensional momentum
space. Red and green dots represent the Weyl nodes with topological charge +1
and -1, respectively, for the ferromagnetic MnIn2Te4 at 6.76 GPa.

##  V Anomalous Hall Conductivity

In the main text we analyzed the pressure dependence of the AHC signal. Fig.
S4 shows the components of AHC as a function of the Fermi energy. We find that
the major contribution to the AHC comes from $\sigma_{xy}$, whereas
$\sigma_{yz}$ and $\sigma_{zx}$ contribute negligibly.

Figure S4: Blue, maroon and light green colours denote the three components of
AHC, $\sigma_{xy}$, $\sigma_{yz}$ and $\sigma_{zx}$, respectively. The
contributions to the conductivity coming from $\sigma_{yz}$ and $\sigma_{zx}$
are negligible compared to that from $\sigma_{xy}$.  Figure S5: (a) The
anomalous Hall conductivity of MnIn2Te4 at representative 6.76 GPa pressure.
The systems shows non-zero ($\sim$50 $\Omega^{-1}$ cm-1) AHC at Fermi energy
and maintains highest value of $\sim$800 $\Omega^{-1}$ cm-1 around 0.8 eV
below the Fermi energy. (b) The difference in the density of spin up
($\rho_{up}$) and spin down ($\rho_{down}$) states.

Fig. S5(a) shows the AHC of the semimetallic phase of FM MnIn2Te4 at a
representative hydrostatic pressure value of 6.76 GPa, calculated as a
function of Fermi energy. In the energy range of 0.2 to 0.95 eV, $\sigma_{xy}$
is almost zero. When $E-E_{F}$ is tuned to zero, i.e., we are at the Fermi
energy, we obtain a non-vanishing value of AHC ($\sim$ 50 $\Omega^{-1}$ cm-1).
It shows a maximum of nearly 800 $\Omega^{-1}$ cm-1 around 0.8 eV below the
Fermi level. The behaviour of $\sigma_{xy}$, although not directly
proportional to the density of available states, matches well with the nature
of difference in density of spin up ($\rho_{up}$) and down ($\rho_{down}$)
states. Fig. S5(b) shows the density difference of up and down spins,
$\rho_{up}-\rho_{down}$. We find that the nature of $\sigma_{xy}$
qualitatively follows the density of available states of majority carriers (up
or down spin). Close to the Fermi energy, electronic transport is mainly
controlled by the Berry curvature around the Weyl nodes. This explains a non-
zero $\sigma_{xy}$ at this value in spite of having negligible
$\rho_{up}-\rho_{down}$. We note that the AHC can be directly controlled by
pressure, since the number of Weyl points is pressure tunable, as we discussed
in the main manuscript.

## References

  * Hasan and Kane (2010) M. Z. Hasan and C. L. Kane, Reviews of modern physics 82, 3045 (2010).
  * Hasan and Moore (2011) M. Z. Hasan and J. E. Moore, Annu. Rev. Condens. Matter Phys. 2, 55 (2011).
  * Qi and Zhang (2011) X.-L. Qi and S.-C. Zhang, Reviews of Modern Physics 83, 1057 (2011).
  * Ando (2013) Y. Ando, Journal of the Physical Society of Japan 82, 102001 (2013).
  * Armitage  _et al._ (2018) N. Armitage, E. Mele, and A. Vishwanath, Reviews of Modern Physics 90, 015001 (2018).
  * Burkov (2016) A. Burkov, Nature materials 15, 1145 (2016).
  * Gao  _et al._ (2019) H. Gao, J. W. Venderbos, Y. Kim, and A. M. Rappe, Annual Review of Materials Research 49, 153 (2019).
  * Weng  _et al._ (2016) H. Weng, X. Dai, and Z. Fang, Journal of Physics: Condensed Matter 28, 303001 (2016).
  * Kou  _et al._ (2017) L. Kou, Y. Ma, Z. Sun, T. Heine, and C. Chen, The journal of physical chemistry letters 8, 1905 (2017).
  * He  _et al._ (2014) K. He, Y. Wang, and Q.-K. Xue, National Science Review 1, 38 (2014).
  * Liu  _et al._ (2016) C.-X. Liu, S.-C. Zhang, and X.-L. Qi, Annual Review of Condensed Matter Physics 7, 301 (2016).
  * Nomura and Nagaosa (2011) K. Nomura and N. Nagaosa, Physical review letters 106, 166802 (2011).
  * Chang  _et al._ (2013) C.-Z. Chang, J. Zhang, X. Feng, J. Shen, Z. Zhang, M. Guo, K. Li, Y. Ou, P. Wei, L.-L. Wang, _et al._ , Science 340, 167 (2013).
  * Li  _et al._ (2010) R. Li, J. Wang, X.-L. Qi, and S.-C. Zhang, Nature Physics 6, 284 (2010).
  * Qi  _et al._ (2010) X.-L. Qi, T. L. Hughes, and S.-C. Zhang, Physical Review B 82, 184516 (2010).
  * Liu  _et al._ (2017) C. Liu, Y. Zang, W. Ruan, Y. Gong, K. He, X. Ma, Q.-K. Xue, and Y. Wang, Physical review letters 119, 176809 (2017).
  * Xu  _et al._ (2020) Y. Xu, L. Elcoro, Z.-D. Song, B. J. Wieder, M. Vergniory, N. Regnault, Y. Chen, C. Felser, and B. A. Bernevig, Nature 586, 702 (2020).
  * Lang  _et al._ (2014) M. Lang, M. Montazeri, M. C. Onbasli, X. Kou, Y. Fan, P. Upadhyaya, K. Yao, F. Liu, Y. Jiang, W. Jiang, _et al._ , Nano letters 14, 3459 (2014).
  * Mogi  _et al._ (2017) M. Mogi, M. Kawamura, A. Tsukazaki, R. Yoshimi, K. S. Takahashi, M. Kawasaki, and Y. Tokura, Science advances 3, eaao1669 (2017).
  * Yasuda  _et al._ (2016) K. Yasuda, R. Wakatsuki, T. Morimoto, R. Yoshimi, A. Tsukazaki, K. Takahashi, M. Ezawa, M. Kawasaki, N. Nagaosa, and Y. Tokura, Nature Physics 12, 555 (2016).
  * Otrokov  _et al._ (2019a) M. M. Otrokov, I. I. Klimovskikh, H. Bentmann, D. Estyunin, A. Zeugner, Z. S. Aliev, S. Gaß, A. Wolter, A. Koroleva, A. M. Shikin, _et al._ , Nature 576, 416 (2019a).
  * Li  _et al._ (2019) J. Li, Y. Li, S. Du, Z. Wang, B.-L. Gu, S.-C. Zhang, K. He, W. Duan, and Y. Xu, Science Advances 5, eaaw5685 (2019).
  * Zhang  _et al._ (2019) D. Zhang, M. Shi, T. Zhu, D. Xing, H. Zhang, and J. Wang, Physical review letters 122, 206401 (2019).
  * Otrokov  _et al._ (2019b) M. Otrokov, I. P. Rusinov, M. Blanco-Rey, M. Hoffmann, A. Y. Vyazovskaya, S. Eremeev, A. Ernst, P. M. Echenique, A. Arnau, and E. V. Chulkov, Physical review letters 122, 107202 (2019b).
  * Xu  _et al._ (2011) G. Xu, H. Weng, Z. Wang, X. Dai, and Z. Fang, Physical review letters 107, 186806 (2011).
  * Wan  _et al._ (2011) X. Wan, A. M. Turner, A. Vishwanath, and S. Y. Savrasov, Physical Review B 83, 205101 (2011).
  * Tang  _et al._ (2016) P. Tang, Q. Zhou, G. Xu, and S.-C. Zhang, Nature Physics 12, 1100 (2016).
  * Hua  _et al._ (2018) G. Hua, S. Nie, Z. Song, R. Yu, G. Xu, and K. Yao, Physical Review B 98, 201116 (2018).
  * Kim  _et al._ (2018) K. Kim, J. Seo, E. Lee, K.-T. Ko, B. Kim, B. G. Jang, J. M. Ok, J. Lee, Y. J. Jo, W. Kang, _et al._ , Nature materials 17, 794 (2018).
  * Wang  _et al._ (2018) Q. Wang, Y. Xu, R. Lou, Z. Liu, M. Li, Y. Huang, D. Shen, H. Weng, S. Wang, and H. Lei, Nature communications 9, 1 (2018).
  * Liu  _et al._ (2018) E. Liu, Y. Sun, N. Kumar, L. Muechler, A. Sun, L. Jiao, S.-Y. Yang, D. Liu, A. Liang, Q. Xu, _et al._ , Nature physics 14, 1125 (2018).
  * Belopolski  _et al._ (2019) I. Belopolski, K. Manna, D. S. Sanchez, G. Chang, B. Ernst, J. Yin, S. S. Zhang, T. Cochran, N. Shumiya, H. Zheng, _et al._ , Science 365, 1278 (2019).
  * Monserrat and Vanderbilt (2016) B. Monserrat and D. Vanderbilt, Physical review letters 117, 226801 (2016).
  * Antonius and Louie (2016) G. Antonius and S. G. Louie, Physical review letters 117, 246401 (2016).
  * Monserrat and Narayan (2019) B. Monserrat and A. Narayan, Physical Review Research 1, 033181 (2019).
  * Kirshenbaum  _et al._ (2013) K. Kirshenbaum, P. Syers, A. Hope, N. Butch, J. Jeffries, S. Weir, J. Hamlin, M. Maple, Y. Vohra, and J. Paglione, Physical review letters 111, 087001 (2013).
  * He  _et al._ (2016) L. He, Y. Jia, S. Zhang, X. Hong, C. Jin, and S. Li, npj Quantum Materials 1, 1 (2016).
  * Zhang  _et al._ (2017) C. Zhang, J. Sun, F. Liu, A. Narayan, N. Li, X. Yuan, Y. Liu, J. Dai, Y. Long, Y. Uwatoko, _et al._ , Physical Review B 96, 155205 (2017).
  * Sadhukhan  _et al._ (2022) S. Sadhukhan, B. Sadhukhan, and S. Kanungo, arXiv preprint arXiv:2201.02347 (2022).
  * Juneja  _et al._ (2018) R. Juneja, R. Shinde, and A. K. Singh, The Journal of Physical Chemistry Letters 9, 2202 (2018).
  * Liu  _et al._ (2014) Y. Liu, Y. Li, S. Rajput, D. Gilks, L. Lari, P. Galindo, M. Weinert, V. Lazarov, and L. Li, Nature Physics 10, 294 (2014).
  * Zeljkovic  _et al._ (2015) I. Zeljkovic, D. Walkup, B. A. Assaf, K. L. Scipioni, R. Sankar, F. Chou, and V. Madhavan, Nature nanotechnology 10, 849 (2015).
  * Narayan (2015) A. Narayan, Physical Review B 92, 220101 (2015).
  * Bose and Narayan (2021) A. Bose and A. Narayan, Journal of Physics: Condensed Matter 33, 375002 (2021).
  * Rodriguez-Vega  _et al._ (2021) M. Rodriguez-Vega, Z.-X. Lin, A. Leonardo, A. Ernst, M. Vergniory, and G. A. Fiete, arXiv preprint arXiv:2107.14361 (2021).
  * Mao  _et al._ (2018) H.-K. Mao, X.-J. Chen, Y. Ding, B. Li, and L. Wang, Reviews of Modern Physics 90, 015007 (2018).
  * Giannozzi  _et al._ (2009) P. Giannozzi, S. Baroni, N. Bonini, M. Calandra, R. Car, C. Cavazzoni, D. Ceresoli, G. L. Chiarotti, M. Cococcioni, I. Dabo, _et al._ , Journal of physics: Condensed matter 21, 395502 (2009).
  * Giannozzi  _et al._ (2017) P. Giannozzi, O. Andreussi, T. Brumme, O. Bunau, M. B. Nardelli, M. Calandra, R. Car, C. Cavazzoni, D. Ceresoli, M. Cococcioni, _et al._ , Journal of Physics: Condensed Matter 29, 465901 (2017).
  * Perdew  _et al._ (1996) J. P. Perdew, K. Burke, and M. Ernzerhof, Physical review letters 77, 3865 (1996).
  * Vanderbilt (1990) D. Vanderbilt, Physical review B 41, 7892 (1990).
  * Grimme (2006) S. Grimme, Journal of computational chemistry 27, 1787 (2006).
  * Monkhorst and Pack (1976) H. J. Monkhorst and J. D. Pack, Physical review B 13, 5188 (1976).
  * Liechtenstein  _et al._ (1995) A. Liechtenstein, V. I. Anisimov, and J. Zaanen, Physical Review B 52, R5467 (1995).
  * Mostofi  _et al._ (2008) A. A. Mostofi, J. R. Yates, Y.-S. Lee, I. Souza, D. Vanderbilt, and N. Marzari, Computer physics communications 178, 685 (2008).
  * Nelder (1965) J. Nelder, The Computer Journal 7, 13 (1965).
  * Wu  _et al._ (2018) Q. Wu, S. Zhang, H.-F. Song, M. Troyer, and A. A. Soluyanov, Computer Physics Communications 224, 405 (2018).
  * Guinea  _et al._ (1983) F. Guinea, C. Tejedor, F. Flores, and E. Louis, Physical Review B 28, 4397 (1983).
  * Sancho  _et al._ (1984) M. L. Sancho, J. L. Sancho, and J. Rubio, Journal of Physics F: Metal Physics 14, 1205 (1984).
  * Sancho  _et al._ (1985) M. L. Sancho, J. L. Sancho, J. L. Sancho, and J. Rubio, Journal of Physics F: Metal Physics 15, 851 (1985).
  * Range and Hübner (1975) K.-J. Range and H.-J. Hübner, Zeitschrift für Naturforschung B 30, 145 (1975).
  * Döll  _et al._ (1991) G. Döll, A. Anghel, J. Baumann, E. Bucher, A. Ramirez, and K.-J. Range, physica status solidi (a) 126, 237 (1991).
  * (62) “See supplemental material for additional discussions of band structures, densities of states and analysis of the Weyl points.” .
  * Fang  _et al._ (2003) Z. Fang, N. Nagaosa, K. S. Takahashi, A. Asamitsu, R. Mathieu, T. Ogasawara, H. Yamada, M. Kawasaki, Y. Tokura, and K. Terakura, Science 302, 92 (2003).
  * Burkov (2014) A. Burkov, Physical review letters 113, 187202 (2014).
  * Nayak  _et al._ (2016) A. K. Nayak, J. E. Fischer, Y. Sun, B. Yan, J. Karel, A. C. Komarek, C. Shekhar, N. Kumar, W. Schnelle, J. Kübler, _et al._ , Science advances 2, e1501870 (2016).
  * Manna  _et al._ (2018) K. Manna, L. Muechler, T.-H. Kao, R. Stinshoff, Y. Zhang, J. Gooth, N. Kumar, G. Kreiner, K. Koepernik, R. Car, _et al._ , Physical Review X 8, 041045 (2018).