# UMFA: A photorealistic style transfer method based on U-Net and multi-layer
feature aggregation

Dongyu Rao  Jiangnan University, Jiangsu Provincial Engineerinig Laboratory of
Pattern Recognition and Computational Intelligence, School of Artificial
Intelligence and Computer Science, Lihu Avenue, Wuxi, China, 214122  Xiao-Jun
Wu  Jiangnan University, Jiangsu Provincial Engineerinig Laboratory of Pattern
Recognition and Computational Intelligence, School of Artificial Intelligence
and Computer Science, Lihu Avenue, Wuxi, China, 214122  Hui Li  Jiangnan
University, Jiangsu Provincial Engineerinig Laboratory of Pattern Recognition
and Computational Intelligence, School of Artificial Intelligence and Computer
Science, Lihu Avenue, Wuxi, China, 214122  Josef Kittler  University of
Surrey, Centre for Vision, Speech and Signal Processing, Guildford, UK., GU2
7XH  Tianyang Xu  University of Surrey, Centre for Vision, Speech and Signal
Processing, Guildford, UK., GU2 7XH

###### Abstract

In this paper, we propose a photorealistic style transfer network to emphasize
the natural effect of photorealistic image stylization. In general, distortion
of the image content and lacking of details are two typical issues in the
style transfer field. To this end, we design a novel framework employing the
U-Net structure to maintain the rich spatial clues, with a multi-layer feature
aggregation (MFA) method to simultaneously provide the details obtained by the
shallow layers in the stylization processing. In particular, an encoder based
on the dense block and a decoder form a symmetrical structure of U-Net are
jointly staked to realize an effective feature extraction and image
reconstruction. Besides, a transfer module based on MFA and ”adaptive instance
normalization” (AdaIN) is inserted in the skip connection positions to achieve
the stylization. Accordingly, the stylized image possesses the texture of a
real photo and preserves rich content details without introducing any mask or
post-processing steps. The experimental results on public datasets demonstrate
that our method achieves a more faithful structural similarity with a lower
style loss, reflecting the effectiveness and merit of our approach.

###### keywords:

photorealistic style transfer, U-Net, deep learning, AdaIN, Dense block

*Xiao-Jun Wu<EMAIL_ADDRESS>
##  1 Introduction

Photorealistic image stylization aims at realizing a style transfer between
the two real photos. The typical styles to be transferred include season [1],
weather and illumination etc. [2]. It can be regarded as a sub-problem of the
wider class of style transfers. It differs from the general style transfer
task which employs realistic and artistic images to obtain artistic content
images. The task of photorealistic style transfer is to achieve a realistic
photo texture with natural stylized output. Therefore, the photorealistic
image stylization has a wide range of applications and excellent development
prospects in photo generation, picture processing, etc..

In the field of computer vision, style transfer is usually studied as a
generalization problem of texture synthesis [3], i.e., extracting the texture
from the source and transmitting it to the target [4]. Traditional methods
mainly focus on model building and texture synthesis [5]. However, traditional
methods can only extract shallow features of the image without involving deep
features, resulting in the poor quality of the final appearance of the style
transfer for complex images. Besides, the constructed model is effective for a
predefined specific style only.

With the emergence of deep learning, the use of deep features extracted by
neural networks for transforming a style has been widely developed [6, 7, 8].
Gatys et al. [6] proposed one of the first style transfer methods based on
deep convolutional neural networks [9]. This method uses a convolutional
neural network to extract deep features and exploits the Gram matrix to
characterize the style in the image. By iteratively updating the reconstructed
image, it realizes the stylization of the input content image. This method can
achieve a transfer of style effects even between complex images. In addition,
it solves the problem of applying a single model to different styles. This
seminal work has opened up a new field of research named Neural Style Transfer
(NST) [10]. However, given a new image, each process of the stylization model
has to be retrained, increasing the computation. Moreover, the tendency to
distort the content image is notable. Subsequently, methods to improve the
quality of stylization and to accelerate the speed have been proposed [11, 12,
13, 14]. Although these methods achieve impressive artistic style transfer
effects, they cannot suppress the distortion and deformation of the content.

With the development of the neural style transfer field, in addition to the
study of artistic style transfer, other aspects of style transfer techniques
are also of interest, such as style transfer of high-resolution images, which
is used to process HD images [15, 16], text style transfer that changes text
attributes [17, 18] and photorealistic style transfer between real images [19,
20].

Traditional methods of photorealistic style transfer are mainly based on color
and tone matching [21, 22, 23]. With the development of the neural style
transfer, researchers have found that neural style transfer could be used to
realize the task of photorealistic image conversion. Based on the algorithm
proposed by Gatys, Fujun Luan et al. [2] were the first to tackle the
photorealistic style transfer task. Using the local affine transformation,
inspired by the Matting Laplacian [24] in the color space (RGB), the
distortion of the content in the generated image is significantly reduced.
Subsequently, a series of algorithms, such as PhotoWCT [25], WCT2 [20], and
other methods [26, 19] were proposed with promising performance. They improved
the quality of the generated images by changing the structure of the existing
methods and introducing post-processing steps. However, the appearance of
stylized images produced by these methods is not natural. In addition, these
methods introduce more operating steps, increasing the processing time.

The key issue regarding the photorealistic image style transfer is how to keep
the structure and details of the content image while introducing the target
style to achieve a natural stylized effect. Meanwhile, there is one problem to
be solved, i.e., realizing the photorealistic style transfer in real-time for
the needs of real-world applications. Current methods cannot solve this
problem very well.

The specific objective of this study is to remove the distortion of the
content in the stylized image and to accelerate the natural stylization
processing. In [25, 20, 27], the VGG network is used to extract deep features
to achieve image stylization. But these methods do not make full use of the
contextual information. In contrast, we propose to use dense block-based U-Net
for feature extraction and image reconstruction, retaining more image detail.
Besides, feature aggregation is an effective feature enhancement method in
computer vision tasks. In order to tightly connect the features, a multi-layer
feature aggregation (MFA) method is proposed and used to enhance the
representation ability of the extracted features. In summary, we propose a new
photorealistic style transfer method named UMFA based on U-Net [28] and multi-
layer feature aggregation [29]. The main contributions of our method UMFA are
as follows:

(1) A dense neural network architecture unit is used in a U-Net based style
transfer framework. The dense block enhances the quality of feature
transferring. The benefit of the combination of dense block and U-Net is the
retention of spatial detail and the ability to express image content.

(2) A multi-layer feature aggregation method is used to fuse multi-scale
features. The MFA module strengthens the ability to express features,
improving the stylization effect by introducing aggregate features in each
layer.

(3) The stylization process is carried out in the skip connection. Through a
cascade processing method, the directly stylized features are inserted into
the image reconstruction process to achieve a natural stylization effect.

The rest of our paper is structured as follows. In Section 2, we briefly
review the photorealistic style transfer method based on deep learning and the
network structure used in this paper. In Section 3, we present the proposed
UMFA method in detail. In Section 4 we report, analyze, and compare the
experimental results. Finally, we summarize the main achievements and draw the
paper to the conclusion in Section 5.

##  2 RELATED WORKS

Recently, many researchers have adopted deep learning-based methods to realize
photorealistic style transfer between images and achieved very promising
results[2, 25, 20]. In this section, first, we review several photorealistic
image stylization methods based on convolutional neural networks. Then we
introduce the computational process known as the U-Net[28] structure. Finally,
the problem of feature aggregation is highlighted in the context of automatic
style transfer and the main feature aggregation mechanisms are discussed [26].

###  2.1 Deep Learning-based Photorealistic Style Transfer Methods

In 2016, Gatys et al. proposed the use of a VGG network[30] based method to
extract features and to perform art style image transfer. The authors deployed
a pre-trained VGG model to obtain deep features of the input image and a style
image, and showed that a stylized image with both the target content and the
required style can be generated by optimizing a random noise image
iteratively. At the same time, their method of expressing style information
with Gram matrix has also been widely used. The Gram matrix is composed of the
inner product of the two vectors, so the Gram matrix can reflect the internal
relationship between the vectors in the group of vectors. The deep feature of
an image can represent the structure, texture and other information of the
image, so when the Gram of the deep features of two images are similar, they
have similar internal connections, that is, they have similar styles.

Subsequently, Johnson et al.[31] proposed an image style transfer method that
uses a perceptual loss to iteratively generate an optimized model named rapid
style transfer. It greatly improves the speed of the algorithm, but the model
can work only for one style.

In 2017, Huang et al.[27] proposed an adaptive instance normalization (AdaIN)
method inspired by the notion of instance normalization (IN)[32, 33] for
transferring arbitrary image styles. From a statistical point of view, the
style information of an image is also a kind of statistical information. When
two images have the same distribution, they also have similar styles. The
”adaptive instance normalization” is a method which changes the mean value and
variance of the content image feature and keeps it consistent with the mean
value and variance of the style image to realize style transfer.

Li et al.[34] developed a whitening and color transform (WCT) method for style
transfer based on a feature transform. These methods can achieve a real-time,
arbitrary style transfer.

In 2018, Li et al.[25] proposed a technique called PhotoWCT, which further
improves the baseline established in [34]. This method changes the upsampling
step to the unpooling operation in the decoder, to smooth the final result.
This method maintains the spatial consistency of the content image and the
stylized image. In 2019, Yoo et al.[20] proposed WCT2 method inspired by WCT.
This method uses wavelet pooling instead of maximum pooling in [25]. Wavelet
pooling retains the image details, while the model introduces richer style
elements through multi-level stylization operations.

The wide application of deep learning makes the design of the neural network
structure an important factor affecting the model performance. Neural
architecture search (NAS)[35] is a technology for automatically designing
efficient neural network structures. In 2020, An et at.[26] designed a model
for ultrafast photorealistic style transfer through neural architecture
search. The basic structure of this method is similar to WCT2[20]. They
searched for a neural architecture performing a multi-layer stylization to
optimise speed and convergence.

In addition to stylization methods based on the encoder and decoder networks,
generative adversarial networks (GAN) are also a hot research topic. Owing to
their characteristics, the images generated by a GAN tend to have realistic
photo textures and natural appearance. Some methods , e.g. Pix2Pix[36],
AutoGAN[37] and others[38, 39] have reported remarkably good style transfer
results.

###  2.2 The U-Net Architecture

In 2015, Ronneberger et al.[28] proposed a network structure for medical image
segmentation named U-Net. The structure consists of two main parts: encoder
and decoder. The encoder includes convolutional layers and a pooling layer,
which is constituted by a downsampling module. The decoder is a corresponding
upsampling module. A skip connection between the corresponding layers is added
to retain rich detailed image information. In addition to medical image
segmentation, U-Net has achieved good results in semantic segmentation, image
restoration, and enhancement[40]. Thanks to the U-Net structure, the image
context is preserved and effectively used to reconstruct the input image.

###  2.3 The Feature Aggregation

With the emergence of deep learning, deep features play an important role in
computer vision tasks. In order to improve the performance, the architecture
of deep neural networks is increasingly deeper and wider [41, 42] to enhance
the modeling capacity and flexibility. Besides, tightening connections between
layers is also an important factor affecting the performance of deep neural
networks, as proved by the idea of skip connection[43]. Feature aggregation
can closely connect features of different scales and levels to achieve a
better quality of image reconstruction, as demonstrated in [29].

---
Figure 1:  The framework of our method.

Inspired by these works, we adopt the U-Net structure for the task of style
transfer and enhance its modeling capability by endowing it with a multi-layer
feature aggregation mechanism. Finally, the merits of the proposed UMFA method
based on U-Net and multi-layer feature aggregation are demonstrated by
extensive experiments.

##  3 PROPOSED PHOTOREALISTIC STYLE TRANSFER METHOD

In this section, we introduce the proposed UMFA method. First, we present the
UMFA framework in Section 3.1. The details of its training are presented in
Section 3.2.

###  3.1 UMFA: Photorealistic Style Transfer Network

As shown in Fig. 1, UMFA consists of three main parts: encoder, decoder, and a
transfer module. The encoder and decoder constitute the basic symmetric
structure of U-Net. The encoder includes DDB blocks and a Conv Layer block.
The decoder includes UCB blocks and a Conv Layer block. The remaining part of
the architecture is the ”transfer module” which includes MFA blocks and AdaIN
blocks. In the encoder, the red and blue solid lines represent the features of
the style image and content image, respectively.

In the ”transfer module”, blue solid line represents the characteristics of
the content image. To make it easier to distinguish in the figure, the red
dashed line is used to represent the characteristics of the style image. The
black lines indicate stylized features. The features of the style and content
images are extracted by the encoder as a prerequisite to a style transfer
through the ”transfer module”. The decoder reconstructs the output image.

In Fig. 1, ”Conv Layer” represents convolutional layer. ”DDB” means
downsampling dense block[44] which contains a max-pooling layer, a dense
block, and two convolutional layers. The framework of dense block is shown in
Fig. 2 (b). Three convolutional layers are densely connected to form a dense
block. ”UCB” denotes upsampling convolutional block which has an upsampling
layer, three convolutional layers and a concatenation module that connects
information coming from the ”transfer module”. ”MFA” represents multi-layer
feature aggregation at skip connection. In Fig.2 (a), the structure of ”MFA”
is displayed, where ”Concat Layer” represents the cascade of the features
passed from the previous with the features of layer. Then the AdaIN block uses
”adaptive instance normalization” to transfer the style. The input images pass
through a convolutional layer containing 3$\times$3 filters. The number of
channels is increased to 32 without changing the feature size, which
facilitates subsequent convolution and pooling operations. Each of the four
downsampling blocks consists of a max-pooling layer, a dense block and two
convolutional layers. Among these, two convolutional layers with 1$\times$1
filters are used to change the feature dimension. The max-pooling layer
implements downsampling operations to halve the feature size. Thus, with each
pass through a downsampling block, the width and height of the feature map
becomes half of the original size and the number of channels is doubled. In
this way, we can obtain multi-scale image information and enhance the model’s
ability to process high-resolution images.

---
Figure 2:  (a)The framework of multi-layer feature aggregation (MFA);(b)The
framework of Dense Block.

The decoder mirrors the encoder structure, which is composed of 4 upsampling
blocks to form a U-Net structure. The upsampling block consists of an
upsampling layer, three convolutional layers and a concatenation module that
receives the information coming from the ”transfer module”. The concatenation
module combines the upsampled feature and the corresponding encoder feature
received from the ”transfer module”. Using the convolutional layer to restore
the required number of channels, we enhance the multiplexing of features and
recover more details of the input image.

After extracting multi-scale features, the ”transfer module” is used to
process the features of the content and style images with the aim of style
transfer. We introduce a feature aggregation mechanism[26] in the ”transfer
module” to connect the multi-scale features of each layer. Feature aggregation
enables the network to integrate spatial information from different scales and
retain image detail. In order to keep more detailed information in each layer,
we propose a multi-feature aggregation (MFA). As shown in Fig. 2, we insert
the features of the previous to the next layer, and repeat the same feature
aggregation process at the subsequent layers. Then the AdaIN method is used to
stylize these aggregated features in each layer.

---
Figure 3:  The framework of the training process of UMFA.

###  3.2 Trainig Phase

In the training phase, we train the network end-to-end to learn a
photorealistic style transfer model. The network framework is shown in Fig. 3.
In order to better extract and use image features, we use a pre-trained VGG-16
network. The trapezoids of different colors in the loss network represent
different blocks. Here we use the features extracted from the first four
blocks. The $relui_{-}j$ indicates the output in loss network, that $i$
indicates which block it is in VGG and $j$ indicates which layer is in the
block.

In the training phase, the loss function $L_{total}$ given in Eq.1 is defined
as

$L_{total}=\alpha L_{style}+\beta L_{content}+\gamma L_{SSIM}$ (1)

where $L_{content}$ and $L_{SSIM}$ respectively are content loss and structure
similarity(SSIM) loss between the content image and output image. $L_{style}$
is style loss between the style image and the output image. $L_{content}$ and
$L_{style}$ relate to the perceptual loss[31]. $\alpha$, $\beta$ and $\gamma$
are the weights of the above three losses. $O$, $C$ and $S$ represent the
output image, content image and style image, respectively.

$l^{\varphi,j}_{content}$ is defined as the L2 norm squared of the feature
which is extracted by ReLU layer in the loss network $\varphi$:

$l^{\varphi,j}_{content}(O,C)=\frac{1}{C_{j}H_{j}W_{j}}||\varphi_{j}(O)-\varphi_{j}(C)||^{2}_{2}$
(2) $L_{content}(O,C)=l^{\varphi,relu2_{-}2}_{content}(O,C)$ (3)

Different from the perceptual loss, we choose the $relu2_{-}2$ as the feature
layer for the content loss.

On the basis of the former, Gram matrix of the feature is used to represent
the style feature. $l^{\varphi,j}_{style}$ is calculated by Eq. 4

$l^{\varphi,j}_{style}(O,S)=\frac{1}{C_{j}H_{j}W_{j}}||G(\varphi_{j}(O))-G(\varphi_{j}(S))||^{2}_{2}$
(4)

$L_{style}(O,S)=\sum_{j}^{relu1_{-}2\cdots
relu4_{-}3}l^{\varphi,j}_{style}(O,S)$ (5)

where $G(\cdot)$ denotes the calculation of Gram matrix.$C,H$ and $W$ are the
number of channels, height and width of the j-th feature map(64*256*256,
128*128*128, 256*64*64, 512*32*32). Finally, the style loss $L_{style}$ is the
sum of all outputs from the four ReLU layers.

The SSIM loss $L_{style}$ is calculated by Eq. 6

$L_{SSIM}=1-SSIM(O,C)$ (6)

where SSIM($\cdot$) represents a structure similarity measure[45]. Small
values of $L_{SSIM}$ and $L_{content}$ means the output image $O$ and the
content image $C$ are similar. A small value of $L_{style}$ signifies that the
output image and the style image are similar.

##  4 EXPERIMENTAL RESULTS AND ANALYSIS

In this section, we introduce the experimental settings used during the
testing phase. Then we report the results of an ablation study. Finally, we
present a subjective evaluation and some objective evaluation indicators to
compare the proposed method with the state of the art.

###  4.1 Experimental Setting

For photorealistic style transfer, any two photoes can be used as style and
content images. In the training, we randomly divide the 80,000 pictures from
MS-COCO[46] into two equal parts as the data set, one part is the style image,
the other part is the content image. The training process includes two epochs.
The training image size is set to 256$\times$256 pixels. The Adam optimizer is
used for training. The learning rate is 0.0001 and unchanged. Our model is
implemented with NVIDIA TITAN Xp and Pytorch.

In the test, we use the image pairs provided in [2] as data set. And we
verified it on other more test sets[20]. Each set includes content, style. No
masking or post-processing is used in the evaluation in order to verify the
effectiveness of the methods.

We choose three typical methods for comparison, including: ”adaptive instance
normalization” (AdaIN)[27], ”photorealistic image stylization based on the
whitening and coloring transform” (PhotoWCT)[25] and ”photorealistic style
transfer via wavelet transforms” (WCT2)[20]. The results are obtained using
publicly available codes for these methods and compared with UMFA on the same
benchmark.

Two evaluation criteria are used to quantitatively compare UMFA with other
methods: structural similarity (SSIM) and Gram loss. Structural similarity
(SSIM) measures the similarity of the stylized image with the content image.
The Gram loss is the mean square error between the stylized image Gram matrix
and the style image Gram matrix that represents the similarity of the style.
Larger SSIM and smaller Gram loss mean better performance.

###  4.2 Ablation Study

1) Feature aggregation strategy: Feature aggregation can enrich the details of
stylized images. Inspired by the bottom feature aggregation method (BFA)[26],
we propose a multi-layer feature aggregation (MFA). In order to verify the
effectiveness of the multi-layer feature aggregation structure, we experiment
with three models: no aggregation, BFA and MFA. At this time, other parameters
are set as follows: $\alpha$=0.8, $\beta$:$\gamma$ is 1:1 ($\beta$=1,
$\gamma$=1). In Fig. 4, we show a set of images to compare the results
obtained with different aggregaation strategy. The plots of the total loss and
style loss are shown in the Fig. 8. The results of the three aggregaation
strategies are summarized in Table 1.

---
Figure 4:  The photorealistic stylization empowered by feature aggregation.
(input pair (top: content, bottom: style))
---
Figure 5:  A plot of the evolution of the total loss(a) and style loss(b)
during training (feature aggregation strategy). Each scale on the abscissa
represents 100 iterations.

As shown in Fig. 4. BFA and MFA method have similar performance, and their
effect is better than inapplicable feature aggregation. The color of the sky
in the red frame is closer to the style image. In Fig. 5, the convergence
curves of the three methods are very close, so we operate on the values (the
reciprocal of the negative logarithm). We can find that the blue line(MFA)
fluctuates less during convergence which means that its convergence process is
more stable. In Table 1, the results of the three methods have similar SSIM,
but the result of the MFA method has a smaller Gram loss which means the
results of MFA method are closer to the style of the targets. Therefore, the
MFA method is used as a feature aggregation strategy.

Table 1: The quantitative evaluation results achieved by different feature aggregation strategies |  w/o BFA |  BFA |  MFA
---|---|---|---
Gram loss |  9.476 |  9.486 |  8.996
SSIM |  0.620 |  0.625 |  0.612

2) Parameter($\alpha$) in Loss Function: The parameter $\alpha$ controls the
weight of the style loss. It is set to 0.2, 0.5, 0.8. Other parameters are set
as follows: $\beta$:$\gamma$ is 1:1 ($\beta$=1, $\gamma$=1), MFA is feature
aggregation strategy. In Fig. 6, we show a set of images to compare the
results obtained with different values of parameter $\alpha$. The plots of the
total loss and style loss are shown in the Fig. 7. The results of the two
quantitative evaluation criteria are summarized in Table 2.

Fig. 6 shows the results obtained with different values of parameter $\alpha$.
As $\alpha$ increases, the degree of stylization increases visibly. The
details in the red box are completely preserved, and its style is changed. In
Fig. 7, the network quickly converges no matter what value the parameter
$\alpha$ takes at 200 iterations. As the parameters $\alpha$ increases, the
speed of the network convergence also increases. The final total loss
converges to a close level. As for the style loss, the initial value is quite
different for different values of parameter $\alpha$. But with fast
convergence, the style loss values of becoming similar, regardless of the
value of $\alpha$. This shows that no matter what the value of $\alpha$ is,
our model is stable and convergent. In the test, the largest initial value
shows the smallest loss of style. This shows that we can control the
stylization of the generated image by adjusting the parameters $\alpha$. Since
the content and style information are antagonistic, content information will
be affected more when the weight of style loss is greater. As shown in Table
2, when $\alpha$ is large, the style evaluation is excellent but the content
index is poor. Based on these observations, the value of $\alpha$ is set to
0.8.

Figure 6:  The photorealistic stylization results with different parameter $\alpha$. (input pair (top: content, bottom: style))  Figure 7:  The evolution of the total loss(a) and style loss(b) during training (paremeter $\alpha$).  Table 2: Quantitative evaluation of the impact of different parameter $\alpha$ |  0.2 |  0.5 |  0.8
---|---|---|---
Gram loss |  14.308 |  11.938 |  8.996
SSIM |  0.8056 |  0.663 |  0.612

3) Parameters ($\beta$:$\gamma$) of the Loss Function: In the [44], Li et al.
control the performance of image fusion by the ratio of SSIM loss and content
loss. We adopt the same strategy in our work. Accordingly, this parameter is
set to 1:1, 1:10 and 1:100 ($\beta$=1, $\gamma$=1, 10, 100). Other parameters
are set as follows: $\alpha$=0.8, MFA is used for feature aggregation. Since
the change of this parameter will affect the value of the total loss, we only
observe the change of style loss. Fig. 8 shows the results obtained for
different values of parameter $\beta$:$\gamma$. The plot of style loss is
shown in the Fig. 9. Table 3 summarises the results measured using the two
quantitative evaluation criteria.

---
Figure 8:  The photorealistic stylization results for different parameter
values $\beta$:$\gamma$. (input pair (top: content, bottom: style))
---
Figure 9:  The evolution of the style loss during training (parameter $\beta$:$\gamma$). Table 3: The impact of meta parameter $\beta$:$\gamma$ |  1:1 |  1:10 |  1:100
---|---|---|---
Gram loss |  8.996 |  12.726 |  13.922
SSIM |  0.612 |  0.705 |  0.748

As shown in Fig. 8, the clouds in the red frame gradually become clear because
of the increasing $\gamma$. In addition, the color distribution of the
stylized image is closer to the content image, but it still retains some style
features, such as red mountains. This proves that the change of parameter
$\beta$:$\gamma$ can enhance the details of the content while retaining the
style elements. In Fig. 9, the style loss drops rapidly by the first 200
iterations. When $\beta$:$\gamma$ is set to 1:1, the convergence of the style
loss is the fastest. In addition, the blue line ($\beta$:$\gamma$=1:100)
always exhibits large fluctuations, which are difficult to stabilize. The
other two lines basically converge to the same level when the parameter
$\beta$:$\gamma$ is set as 1:1 and 1:10. As shown in Table 3, we can refine
the image content by changing $\beta$:$\gamma$, but it will lead to a loss of
style. Based on these results, we choose $\beta$:$\gamma$=1:1 ($\beta$=1,
$\gamma$=1).

---
Figure 10:  Photorealistic stylization results: (a) input pair (top: content,
bottom: style); (b) AdaIN; (c) PhotoWCT (without mask); (d) WCT2 (without
mask); (e) Our method (UMFA).

###  4.3 Result Analysis

####  4.3.1 Subjective evaluation

In Fig. 10, we show the results obtained by existing methods and UMFA. The
results suggest that the AdaIN[27] method renders the highest degree of
abstraction. PhotoWCT[25] and WCT2[20], improved by whitening and coloring
transforms (WCT), retain the structural information of the image better.
However, the stylization effect of these two methods is unnatural. Besides,
there are too many style elements that affect the realism of the photo. UMFA
produces a natural stylization effect, closest to a real photo with minimum
artifacts.

In order to better verify the effect of our method on the subjective
experience of people, we also set up a comparison of stylized results produced
by different methods. We scramble the pictures generated by different methods
in Fig. 10, and ask 100 testers to choose the pictures that they think have
the best photorealistic style transfer effect. As shown in Table 4, the
percentage indicates the proportion of the tester choosing the result of this
method as the ”Best stylization”. Since they can only choose one method, the
sum of the overall proportions is one. Among them, the proportion of choosing
our method(UMFA) is the highest, that the result of UMFA is the most popular.

---
Figure 11:  The result of using different structured content and style images. Table 4: Subjective evaluation results of different methods | AdaIN | PhotoWCT | WCT2) | Ours(UMFA)
---|---|---|---|---
|  | (w/o Mask) | (w/o Mask) |
Best stylization | 13.2% | 25.1% | 24.8% | 36.9%

The main purpose of photorealistic style transfer is to change the style
elements of the image without changing the main body of the image content.
Therefore, in order to achieve a better stylization effect, users usually use
pictures with similar structures when using algorithms. In addition to the
similar image structure, we also performed photorealistic style transfer tests
on other images to prove the effectiveness of our method. In Fig. 11, we show
four sets of photorealistic stylized images, where the upper left corner of
each part is the content image, the lower left corner is the style image, and
the right side is the stylized image.

####  4.3.2 Objective evaluation

Table 5 presents the average values of the two objective criteria used for the
evaluation of stylized images (Gram loss and SSIM). The best values are
indicated in bold. As shown in Table 5, UMFA has the smallest Gram loss and
the largest SSIM which means that our photorealistic stylization result has a
similar style to the style image and retains the structural information of the
content image.

Table 5: Quantitative evaluation results of different methods  | AdaIN | PhotoWCT | WCT2) | Ours(UMFA)
---|---|---|---|---
|  | (w/o Mask) | (w/o Mask) |
Gram loss | 11.356 | 11.367 | 9.323 | 8.996
SSIM | 0.347 | 0.423 | 0.516 | 0.612

###  4.4 Runtime

Table 6 shows the runtime of AdaIN[27], PhotoWCT[25], WCT2[20] and our
method(UMFA). The numbers in the table represent the time required for various
methods to generate a stylized picture at different image sizes. The unit of
the number is seconds. The smallest values are indicated in bold which means
fastest running speed. The results of our method are denoted in red. For
PhotoWCT and WCT2, we do not use mask to compare various methods at the same
level. The post-processing time of the PhotoWCT method is too long. We use the
time of stylization process as the comparison objective. AdaIn is the fastest
method because it is stylized by matching the mean and variance requiring the
small amount of calculation. In addition, it only uses the deepest features
for style transfer, reducing the amount of calculation. But for the same
reason, the stylization effect of the AdaIN method is the worst. PhotoWCT and
WCT2 take a similar amount of time because they are based on the same method,
but PhotoWCT needs more processing time to achieve the effect closed to WCT2.
Compared with the Adain, the method based on whitening and coloring transform
requires a larger amount of calculation. Other operations introduced in the
encoder and decoder process further increase the runtime. Among the four
methods, UMFA does not take the least time. However, it can achieve real-time
photorealistic style transfer and is an order of magnitude faster than the
other two popular methods. Inspired by AdaIN, we changed the network
structure, added MFA blocks and perform multi-layer stylization operations. As
a result, we have achieved a balance between photorealistic stylization and
time consumption.

Table 6: A comparison of runtime for different methods(Time/sec) Iamge Size | AdaIN | PhotoWCT | WCT2 | Ours(UMFA)
---|---|---|---|---
256 $\times$ 256 | 0.046 | 3.44 | 5.04 | 0.14
512 $\times$ 512 | 0.12 | 3.48 | 5.64 | 0.24
1024 $\times$ 1024 | 0.35 | 3.66 | 6.06 | 0.64

##  5 CONCLUSION

In this paper, we propose a photorealistic style transfer model named UMFA
based on U-Net and multi-layer feature aggregation. First, the encoder, which
uses the dense block in the downsampling module, can make full use of the
image information to capture details. The ”transfer module” is designed to
enrich multi-scale features and enhance the expressiveness of stylization
results by AdaIN and MFA blocks. The upsampling and skip connection of the
U-Net structure are designed for the decoder to reconstruct and output the
stylized image. The experimental results show that the UMFA method enhances
the content detail in the stylized image and retains the texture of the photo,
achieving style transfer with a natural photorealistic effect.

##  6 Acknowledgments

This work was supported by the National Key Research and Development Program
of China (Grant No. 2017YFC1601800), the National Natural Science Foundation
of China (61672265, U1836218), the 111 Project of Ministry of Education of
China (B12018), and the UK EPSRC (EP/N007743/1, MURI/EPSRC/DSTL,
EP/R018456/1).

## References

  * [1] P.-Y. Laffont, Z. Ren, X. Tao, et al., “Transient attributes for high-level understanding and editing of outdoor scenes,” ACM Transactions on graphics (TOG) 33(4), 1–11 (2014).
  * [2] F. Luan, S. Paris, E. Shechtman, et al., “Deep photo style transfer,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 4990–4998 (2017).
  * [3] A. Akl, C. Yaacoub, M. Donias, et al., “A survey of exemplar-based texture synthesis methods,” Computer Vision and Image Understanding 172, 12–24 (2018).
  * [4] A. A. Efros and W. T. Freeman, “Image quilting for texture synthesis and transfer,” in Proceedings of the 28th annual conference on Computer graphics and interactive techniques, 341–346 (2001).
  * [5] Z.-C. Song and S.-G. Liu, “Sufficient image appearance transfer combining color and texture,” IEEE Transactions on Multimedia 19(4), 702–711 (2016).
  * [6] L. A. Gatys, A. S. Ecker, and M. Bethge, “Image style transfer using convolutional neural networks,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2414–2423 (2016).
  * [7] X. Zhang, Y. Luan, and X. Li, “Real-time image style transformation based on deep learning,” Journal of Electronic Imaging 27(4), 043045 (2018).
  * [8] H. Zhang and K. Dana, “Multi-style generative network for real-time transfer,” in Proceedings of the European Conference on Computer Vision (ECCV), 0–0 (2018).
  * [9] A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,” in Advances in neural information processing systems, 1097–1105 (2012).
  * [10] Y. Jing, Y. Yang, Z. Feng, et al., “Neural style transfer: A review,” IEEE transactions on visualization and computer graphics (2019).
  * [11] L. A. Gatys, A. S. Ecker, M. Bethge, et al., “Controlling perceptual factors in neural style transfer,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 3985–3993 (2017).
  * [12] S. Gu, C. Chen, J. Liao, et al., “Arbitrary style transfer with deep feature reshuffle,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 8222–8231 (2018).
  * [13] M.-M. Cheng, X.-C. Liu, J. Wang, et al., “Structure-preserving neural style transfer,” IEEE Transactions on Image Processing 29, 909–920 (2019).
  * [14] Z. Hu, J. Jia, B. Liu, et al., “Aesthetic-aware image style transfer,” in Proceedings of the 28th ACM International Conference on Multimedia, 3320–3329 (2020).
  * [15] O. Texler, D. Futschik, J. Fišer, et al., “Arbitrary style transfer using neurally-guided patch-based synthesis,” Computers & Graphics 87, 62–71 (2020).
  * [16] M. Li, C. Ye, and W. Li, “High-resolution network for photorealistic style transfer,” arXiv preprint arXiv:1904.11617 (2019).
  * [17] S. Yang, W. Wang, and J. Liu, “Te141k: Artistic text benchmark for text effect transfer,” IEEE Transactions on Pattern Analysis and Machine Intelligence (2020).
  * [18] G. Li, J. Zhang, D. Chen, et al., “Chinese flower-bird character generation based on pencil drawings or brush drawings,” Journal of Electronic Imaging 28(3), 033029 (2019).
  * [19] L. Wang, Z. Wang, X. Yang, et al., “Photographic style transfer,” The Visual Computer 36(2), 317–331 (2020).
  * [20] J. Yoo, Y. Uh, S. Chun, et al., “Photorealistic style transfer via wavelet transforms,” in Proceedings of the IEEE International Conference on Computer Vision, 9036–9045 (2019).
  * [21] K. Sunkavalli, M. K. Johnson, W. Matusik, et al., “Multi-scale image harmonization,” ACM Transactions on Graphics (TOG) 29(4), 1–10 (2010).
  * [22] F. Pitie, A. C. Kokaram, and R. Dahyot, “N-dimensional probability density function transfer and its application to color transfer,” in Tenth IEEE International Conference on Computer Vision (ICCV’05) Volume 1, 2, 1434–1439, IEEE (2005).
  * [23] J.-D. Yoo, M.-K. Park, J.-H. Cho, et al., “Local color transfer between images using dominant colors,” Journal of Electronic Imaging 22(3), 033003 (2013).
  * [24] A. Levin, D. Lischinski, and Y. Weiss, “A closed-form solution to natural image matting,” IEEE transactions on pattern analysis and machine intelligence 30(2), 228–242 (2007).
  * [25] Y. Li, M.-Y. Liu, X. Li, et al., “A closed-form solution to photorealistic image stylization,” in Proceedings of the European Conference on Computer Vision (ECCV), 453–468 (2018).
  * [26] J. An, H. Xiong, J. Huan, et al., “Ultrafast photorealistic style transfer via neural architecture search.,” in AAAI, 10443–10450 (2020).
  * [27] X. Huang and S. Belongie, “Arbitrary style transfer in real-time with adaptive instance normalization,” in Proceedings of the IEEE International Conference on Computer Vision, 1501–1510 (2017).
  * [28] O. Ronneberger, P. Fischer, and T. Brox, “U-net: Convolutional networks for biomedical image segmentation,” in International Conference on Medical image computing and computer-assisted intervention, 234–241, Springer (2015).
  * [29] F. Yu, D. Wang, E. Shelhamer, et al., “Deep layer aggregation,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2403–2412 (2018).
  * [30] K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” in International Conference on Learning Representations, (2015).
  * [31] J. Johnson, A. Alahi, and L. Fei-Fei, “Perceptual losses for real-time style transfer and super-resolution,” in European conference on computer vision, 694–711, Springer (2016).
  * [32] V. Dumoulin, J. Shlens, and M. Kudlur, “A learned representation for artistic style,” arXiv preprint arXiv:1610.07629 (2016).
  * [33] D. Ulyanov, A. Vedaldi, and V. Lempitsky, “Improved texture networks: Maximizing quality and diversity in feed-forward stylization and texture synthesis,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 6924–6932 (2017).
  * [34] Y. Li, C. Fang, J. Yang, et al., “Universal style transfer via feature transforms,” in Advances in neural information processing systems, 386–396 (2017).
  * [35] B. Zoph and Q. Le, “Neural architecture search with reinforcement learning,” in International Conference on Learning Representations, (2017).
  * [36] P. Isola, J.-Y. Zhu, T. Zhou, et al., “Image-to-image translation with conditional adversarial networks,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 1125–1134 (2017).
  * [37] B. Cao, H. Zhang, N. Wang, et al., “Auto-gan: self-supervised collaborative learning for medical image synthesis,” in Proceedings of the AAAI Conference on Artificial Intelligence, 34(07), 10486–10493 (2020).
  * [38] Y.-F. Zhou, R.-H. Jiang, X. Wu, et al., “Branchgan: Unsupervised mutual image-to-image transfer with a single encoder and dual decoders,” IEEE Transactions on Multimedia 21(12), 3136–3149 (2019).
  * [39] J.-Y. Zhu, T. Park, P. Isola, et al., “Unpaired image-to-image translation using cycle-consistent adversarial networks,” in Proceedings of the IEEE international conference on computer vision, 2223–2232 (2017).
  * [40] D. Liu, B. Wen, X. Liu, et al., “When image denoising meets high-level vision tasks: A deep learning approach,” arXiv preprint arXiv:1706.04284 (2017).
  * [41] K. He, X. Zhang, S. Ren, et al., “Identity mappings in deep residual networks,” in European conference on computer vision, 630–645, Springer (2016).
  * [42] K. He, X. Zhang, S. Ren, et al., “Deep residual learning for image recognition,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 770–778 (2016).
  * [43] G. Huang, Z. Liu, L. Van Der Maaten, et al., “Densely connected convolutional networks,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 4700–4708 (2017).
  * [44] H. Li and X.-J. Wu, “Densefuse: A fusion approach to infrared and visible images,” IEEE Transactions on Image Processing 28(5), 2614–2623 (2018).
  * [45] Z. Wang, A. C. Bovik, H. R. Sheikh, et al., “Image quality assessment: from error visibility to structural similarity,” IEEE transactions on image processing 13(4), 600–612 (2004).
  * [46] T.-Y. Lin, M. Maire, S. Belongie, et al., “Microsoft coco: Common objects in context,” in European conference on computer vision, 740–755, Springer (2014).

Dongyu Rao received the B.E. degree in School of Metallurgy and Environment
from Central South University, China, in 2019. He is a master student at the
Jiangsu Provincial Engineering Laboratory of Pattern Recognition and
Computational Intelligence, School of Artificial Intelligence and Computer
Science, Jiangnan University. His research interests include style transfer,
machine learning and deep learning.

Xiao-Jun Wu received the Ph.D. degree in pattern recognition and intelligent
system from the Nanjing University of Science and Technology, Nanjing, in
2002. He has been with the School of Information Engineering, Jiangnan
University since 2006, where he is a Professor of pattern recognition and
computational intelligence. He has published over 300 papers in his fields of
research. His current research interests include pattern recognition, computer
vision, and computational intelligence.

Hui Li received the B.Sc. degree in School of Internet of Things Engineering
from Jiangnan University, China, in 2015. He is currently a PhD student at the
Jiangsu Provincial Engineerinig Laboratory of Pattern Recognition and
Computational Intelligence, School of Artificial Intelligence and Computer
Science, Jiangnan University. His research interests include image fusion,
machine learning and deep learning.

Josef Kittler received the B.A., Ph.D., and D.Sc. degrees from the University
of Cambridge, in 1971, 1974, and 1991, respectively. He is a distinguished
Professor of Machine Intelligence at the Centre for Vision, Speech and Signal
Processing, University of Surrey, U.K. He conducts research in biometrics,
video and image database retrieval, medical image analysis, and cognitive
vision. He published the textbook Pattern Recognition: A Statistical Approach
and over 700 scientific papers.

Tianyang Xu received the B.Sc. degree in electronic science and engineering
from Nanjing University, Nanjing, China, in 2011. He received the PhD degree
at the School of Internet of Things Engineering, Jiangnan University, Wuxi,
China, in 2019. He is currently a research fellow at the Centre for Vision,
Speech and Signal Processing (CVSSP), University of Surrey, Guildford, United
Kingdom. His research interests include visual tracking and deep learning.

###### List of Figures

  1. 1  The framework of our method.
  2. 2  (a)The framework of multi-layer feature aggregation (MFA);(b)The framework of Dense Block.
  3. 3  The framework of the training process of UMFA.
  4. 4  The photorealistic stylization empowered by feature aggregation. (input pair (top: content, bottom: style))
  5. 5  A plot of the evolution of the total loss(a) and style loss(b) during training (feature aggregation strategy). Each scale on the abscissa represents 100 iterations.
  6. 6  The photorealistic stylization results with different parameter $\alpha$. (input pair (top: content, bottom: style))
  7. 7  The evolution of the total loss(a) and style loss(b) during training (paremeter $\alpha$).
  8. 8  The photorealistic stylization results for different parameter values $\beta$:$\gamma$. (input pair (top: content, bottom: style))
  9. 9  The evolution of the style loss during training (parameter $\beta$:$\gamma$).
  10. 10  Photorealistic stylization results: (a) input pair (top: content, bottom: style); (b) AdaIN; (c) PhotoWCT (without mask); (d) WCT2 (without mask); (e) Our method (UMFA).
  11. 11  The result of using different structured content and style images.

###### List of Tables

  1. 1 The quantitative evaluation results achieved by different feature aggregation strategies
  2. 2 Quantitative evaluation of the impact of different parameter $\alpha$
  3. 3 The impact of meta parameter $\beta$:$\gamma$
  4. 4 Subjective evaluation results of different methods
  5. 5 Quantitative evaluation results of different methods
  6. 6 A comparison of runtime for different methods(Time/sec)


# Quantum complexity and topological phases of matter

Pawel Caputa and Sinong Liu  Faculty of Physics, University of Warsaw, ul.
Pasteura 5, 02-093 Warsaw, Poland.

###### Abstract

We find that the complexity of quantum many-body states, defined as a spread
in the Krylov basis, may serve as a probe that distinguishes topological
phases of matter. We illustrate this analytically in one of the representative
examples, the Su-Schrieffer-Heeger model. Moreover, in the same setup, we
analyze exactly solvable quench protocols where the evolution of the spread
complexity shows distinct features depending on the topological vs non-
topological phase of the initial state as well as the quench Hamiltonian.

## Introduction

Recently, the problem of quantifying complexity in quantum systems has been at
the center of attention in different branches of physics. To some extent, this
interest in complexity was stimulated by the holographic correspondence Ma
and puzzles related to the nature of black holes in quantum gravity. Among
others, complexity played essential roles in discussions on the firewall
Harlow:2013tf , fast scrambling near black hole horizons Susskind ; Aaronson ;
Stanford:2014jda ; Brown:2015bva , quantum chaos Maldacena:2015waa , and
finally the nature of holographic dictionary itself Bouland:2019pvu .

What makes holography powerful, is the dual description of quantum gravity in
terms of a strongly interacting quantum system. This way, the “complexity” of
black hole processes can, in principle, be quantified in the dual model and
indeed, many recent works advocated for the most useful probe of complexity in
(holographic) quantum field theories (QFTs) (see reviews Chapman:2021jbh ;
Chen:2021lnq ). Even though these explorations are still in a relative early
stage, as the dust is starting to settle, we are provided with new,
potentially powerful, tools for exploring the complexity frontier of field
theories.

One of the interesting directions where complexity measures can be tested and
may provide a new looking glass, is the physics of topological phases of
matter Hasan:2010xy ; Qi:2010qag ; Schnyderetal . Indeed, the importance of
complexity in topological order was realised early on (see e.g. DoritA ;
MilMiy ). On the other hand, applying geometric ideas from holography to
topological phases proved fruitful in Wen:2016bud ; Aguado:2007oza ;
Matsuura:2010vi  and recently, frameworks for classification of topological
phases were suggested as guiding principle for complexity in holography
Czech:2022wtt . Clearly, these two subjects may have a lot to learn from each
other. Nevertheless, actual studies of state complexity measures in models
with topological phases are in their infancy.

So far, only the Nielsen-type approach to complexity Nielsen1 ;
Jefferson:2017sdb ; Chapman:2017rqy , was tested in a couple of examples
Ali:2018aon ; Liu:2019aji ; Xiong:2019xoh . Since this framework, tailor-made
for computer science, gives rise to numerous ambiguities in many-body physics,
such as the choice of gates with their penalties, cost functions etc., it is
not surprising that conclusions on whether complexity is sensitive enough
differed between Ali:2018aon  and Liu:2019aji ; Xiong:2019xoh . Still
Liu:2019aji ; Xiong:2019xoh  argued that, with an appropriate choice for the
ambiguities, one may detect different phases by discontinuities in Nielsen’s
complexity. Based on the above, whether or not complexity is a universal probe
correlated with topological phases remains an open question.

In this work we begin filling this gap by analyzing a new measure of state
complexity proposed in Balasubramanian:2022tpr  called spread complexity
(defined in the next section) in one of the simplest models with topological
phases, the Su-Schrieffer-Heeger (SSH) model Su:1979ua . This measure is much
less ambiguous, relatively straightforward to compute and already proved handy
in diagnosing quantum chaos Balasubramanian:2022tpr . We first analytically
compute the spread complexity of formation of the ground state and find that
it is sensitive to the two phases of the model. Next, we test its evolution
following quantum quench protocols and find that it also distinguishes the
phases even in this non-equilibrium scenario. Our analytical results not only
reinforce the spread complexity as a powerful tool but also give a hope for
universality in studying topological phases from the perspective of
complexity.

This letter is organised as follows. We first briefly review the spread
complexity based on the Krylov basis Balasubramanian:2022tpr . Next, apply it
in the SSH model and evaluate complexity of formation for the ground state.
Then, we consider the time evolution of complexity following quantum quench
with different SSH Hamiltonians. Finally, we conclude and discuss future
directions. Technical details are included in three appendices.

##  I Spread Complexity of Quantum States

We begin with a brief review of the state complexity measure introduced in
Balasubramanian:2022tpr  and the Krylov basis that plays the main role in the
computations. Further details and more pedagogical review can be found in
Balasubramanian:2022tpr ; Caputa:2021sib . The starting point of our
discussion is a general quantum state $\left|\Psi(s)\right>$ related to some
initial $\left|\Psi_{0}\right>$ by a unitary transformation

$\left|\Psi(s)\right>=e^{-iHs}\left|\Psi_{0}\right>.$ (1)

In the quantum computation parlance, $\left|\Psi(s)\right>$ can be referred to
as a “target state” related to a “reference state” $\left|\Psi_{0}\right>$ by
a “unitary circuit” with a “circuit Hamiltonian” $H$. Parameter $s$, usually
$s\in[0,1]$, denotes a “circuit time”, can be also taken arbitrary and
regarded as the physical time $t$ (see below).
A useful measure of quantum state complexity can be defined as a spread of
$\left|\Psi(s)\right>$ in the Hilbert space (relative to
$\left|\Psi_{0}\right>$) Balasubramanian:2022tpr . More precisely, the spread
complexity of $\left|\Psi(s)\right>$ is estimated by the minimum over all
choices of basis
$\mathcal{B}=\left\\{\left|B_{n}\right>,n=0,1,2,...|\left|B_{0}\right>=\left|\Psi_{0}\right>\right\\}$
of the following cost function

$\mathcal{C}(s)=\min_{\mathcal{B}}\left(\sum_{n}n|\langle\Psi(s)|B_{n}\rangle|^{2}\right).$
(2)

The fact that makes this definition powerful and computable is that the
minimum is attained (see Balasubramanian:2022tpr  for proofs) when basis
$\mathcal{B}$ is the so-called Krylov basis that we now review.

The idea behind the Krylov basis for state (1) is to consider states with all
the different powers of the circuit Hamiltonian acting on the initial state
$\left|\Psi_{0}\right>$, i.e.,
$\\{\left|\Psi_{0}\right>,H\left|\Psi_{0}\right>,H^{2}\left|\Psi_{0}\right>,...\\}$,
and apply the Gram–Schmidt orthogonalization procedure, known as the Lanczos
algorithm LanczosVish , to this set. In this new basis $\left|K_{n}\right>$,
the circuit Hamiltonian $H$ is generally tri-diagonal and acts as

$H\left|K_{n}\right>=a_{n}\left|K_{n}\right>+b_{n}\left|K_{n-1}\right>+b_{n+1}\left|K_{n+1}\right>,$
(3)

where coefficients $a_{n}$ and $b_{n}$, the so-called Lanczos coefficients,
can be recursively constructed using the Lanczos algorithm. The information
about them is also contained in the moments of the return-amplitude (auto-
correlator) $S(s)\equiv\langle\Psi(s)|\Psi(0)\rangle$.

Having constructed the basis that minimizes (2), we expand our state as

$\left|\Psi(s)\right>=\sum_{n}\psi_{n}(s)\left|K_{n}\right>,$ (4)

where, by construction, the complex coefficients in this expression satisfy a
discrete Schrodinger equation

$i\partial_{s}\psi_{n}(s)=a_{n}\psi_{n}(s)+b_{n}\psi_{n-1}(s)+b_{n+1}\psi_{n+1}(s).$
(5)

With the knowledge of the Lanczos coefficients, we can solve it with initial
condition $\psi_{n}(0)=\delta_{n,0}$ (so that we start from
$\left|K_{0}\right>=\left|\Psi_{0}\right>$), and determine (4). Note that
unitarity implies that $\sum_{n}(p_{n}(s)\equiv|\psi_{n}(s)|^{2})=1$.
Moreover, the return amplitude is related to the first coefficient by
$S(s)=\psi^{*}_{0}(s)$. Last but not the least, the number of the independent
Krylov vectors depends on the Hamiltonian as well as the initial state
$\left|\Psi_{0}\right>$.
Most importantly, in the Krylov basis, the complexity (2) becomes

$\mathcal{C}(s)=\sum_{n}n|\psi_{n}(s)|^{2},$ (6)

and for all practical purposes, this will be our working definition of the
spread complexity in the remaining part of this paper. This measure naturally
generalizes the Krylov complexity of operators Parker:2018yvk  to quantum
states (see e.g. Yates:2021lrt  for applications in the SSH model). Among
other interesting properties, under time evolution in random matrix theory,
spread complexity exhibits universal features analogous to the spectral form
factor Cotler:2016fpe  that make it a promising candidate for a probe of
quantum chaos. In the following, we will employ (6) and test wether it does
equally well in integrable systems and in particular, whether it can detect
topological phases.

##  II Spread complexity in the SSH model

Our basic example will be the SSH model of polyacetylene Su:1979ua  (see e.g.
TopInS  for a pedagogical introduction, here we closely follow the conventions
of RH ; Miyaji:2014mca ) given by the Hamiltonian

$\displaystyle H$ $\displaystyle=$ $\displaystyle
t_{1}\sum_{i}\left(c_{Ai}^{\dagger}c_{Bi}+\text{h.c.}\right)-t_{2}\sum_{i}\left(c^{\dagger}_{Bi}c_{A,i+1}+\text{h.c.}\right)$
(7) $\displaystyle+$
$\displaystyle\mu_{s}\sum_{i}\left(c^{\dagger}_{Ai}c_{Ai}-c^{\dagger}_{Bi}c_{Bi}\right),$

where $(c_{Ai},c_{Bi})$ represent two-flavours of fermion annihilation
operators defined at site $i$ on a 1d lattice, and $t_{1},t_{2},\mu_{s}$ are
real parameters. We will assume $t_{1}\geq 0$ and $t_{2}\geq 0$ and
$\mu_{s}=0$. Depending on these couplings, the model is in one of the two
phases: a non-topological phase for $t_{1}>t_{2}$ or a topological phase
(topological insulator) for $t_{2}>t_{1}$, separated by a critical point at
$t_{1}=t_{2}$.

We will first compute the complexity of the ground state that, depending on
parameters $t_{1}$ and $t_{2}$, belongs to one of the above-mentioned phases
of the model. Strictly speaking, we should call it the spread complexity of
formation of the ground-sate and it is a slight generalisation of the
definition for real time $t$ in Balasubramanian:2022tpr . Nevertheless, since
the main step in computing the spread complexity is to rewrite the ground-sate
as a quantum circuit (1), the framework works equally well for the circuit
time $s$.

For that, as well as for later purposes, it will be convenient to re-write the
Hamiltonian in momentum space as (see Miyaji:2014mca , and appendix A)

$H=\sum_{k}\left[2R_{3}J^{(k)}_{0}+iR_{1}\left(J^{(k)}_{+}-J^{(k)}_{-}\right)\right],$
(8)

where the coefficients are $R_{1}=t_{1}-t_{2}\cos(k)$, $R_{3}=t_{2}\sin(k)$
and, for each momentum mode, we denoted the $SU(2)$ algebra generators

$[J^{(k)}_{0},J^{(k)}_{\pm}]=\pm
J^{(k)}_{\pm},\qquad[J^{(k)}_{+},J^{(k)}_{-}]=2J^{(k)}_{0}.$ (9)

Figure 1: Behaviour of the spread complexity of formation
$\mathcal{C}(t_{1},t_{2})$ (equation (16)) for the ground state of the SSH
model $|\Omega\rangle$.

Then, the ground state can be written in terms of the $SU(2)$ coherent states
as

$\left|\Omega\right>=\prod_{k>0}\mathcal{N}_{k}e^{-i\tan\left({\phi_{k}\over
2}\right)(J^{(k)}_{+}+J^{(-k)}_{+})}\left|{1\over 2},-{1\over 2}\right>_{k},$
(10)

where $\mathcal{N}_{k}$ stands for the normalization, $J^{(\pm k)}$ correspond
to two decoupled $SU(2)$ algebras for positive and negative momenta and
$\left|1/2,-1/2\right>$ denotes a tensor product over the lowest-weight states
of the $j=1/2$ representation ($J^{(\pm k)}_{-}\left|1/2,-1/2\right>_{\pm
k}=0$). Moreover, the relation between $\phi_{k}$ and the physical parameters
is given by

$\sin\phi_{k}={|R_{1}|\over R},\qquad\cos\phi_{k}={R_{3}\over R},$ (11)

where we also denoted

$R=\sqrt{t_{1}^{2}+t_{2}^{2}-2t_{1}t_{2}\cos(k)}.$ (12)

Without loss of generality, we can just compute the spread complexity for
positive momenta and the full result will have an additional factor of 2 from
the $-k$ sector.

First, for a single momentum $k>0$, using the $SU(2)$ Baker–Campbell–Hausdorff
formula and with a slight abuse of notation, we write the relevant part of the
state in a circuit form (1)

$\left|\Omega_{k}(s)\right>=e^{-i{s\phi_{k}\over
2}\left(J^{(k)}_{+}+J^{(k)}_{-}\right)}\left|{1\over 2},-{1\over
2}\right>_{k},$ (13)

where $s\in[0,1]$ and our ground state is the “target state” at $s=1$. Note
that in these circuits we took a natural reference state as the ground state
of the left and right Hamiltonians (see appendix A). This way of writing makes
transparent the connection with coherent states and we can directly apply the
tools from Caputa:2021sib  (see appendix B) to expand our state in the Krylov
basis as (4). Because $j=1/2$, we will only have two basis vectors and two
amplitudes

$\psi_{0}(s)=\cos\left({s\phi_{k}\over
2}\right),\qquad\psi_{1}(s)=-i\sin\left({s\phi_{k}\over 2}\right),$ (14)

that satisfy (5) with appropriate Lanczos coefficients. As a result, we get
the contribution to our complexity from a single momentum mode

$\mathcal{C}_{k}(s=1)=\sin^{2}{\phi_{k}\over 2}={1\over 2}-{t_{2}\sin(k)\over
2\sqrt{t_{1}^{2}+t_{2}^{2}-2t_{1}t_{2}\cos(k)}}.\ $ (15)

The complexity of the ground state is obtained by integrating over all the
momenta and multiplying by 2 from $k<0$. This yields

$\displaystyle\mathcal{C}(t_{1},t_{2})=2\int_{0}^{\pi}{dk\over
2\pi}\mathcal{C}_{k}={1\over 2}-{t_{1}+t_{2}-|t_{1}-t_{2}|\over 2\pi t_{1}}.$
(16)

Observe that we took the continuum limit so this result is proportional to the
volume $L$ but, to keep our equations compact, we rescaled this factor (our
$\mathcal{C}$ are complexity “densities”).

This surprisingly simple formula, shown on Fig. 1, indeed shows two very
different behaviours of the spread complexity for the two distinct phases of
the model. Namely, for the non-topological phase with $t_{1}>t_{2}$,
complexity linearly depends on the ratio $t_{2}/t_{1}$ but in the topological
phase, with $t_{2}>t_{1}$, it is constant. This is one of our main results. We
also performed analogous computation for the 1d Kitaev-chain Kitaev:2000nmw ,
and found that complexity becomes constant when crossing from a non-
topological to a topological phase in specific cases PCSL . Note that unlike
entanglement entropy or Nielsen-type complexities that require numerics, (16)
is fully analytical.

##  III Complexity during quantum quench

Another framework where we can probe the spread complexity is given by the so-
called quantum quenches. A typical quench protocol considers a unitary time
evolution of an initial state $\left|\Omega_{i}\right>$ of some initial
Hamiltonian $H_{i}$, performed with a different Hamiltonian $H_{f}$ for which
$\left|\Omega_{i}\right>$ is an excited state. Universal features of the
evolution of entanglement and complexity have been extensively studied in the
literature (see e.g. Mitra ; Calabrese:2016xau ; Camargo:2018eof ; Liu:2019qyx
and review of closely related dynamical quantum phase transitions Heyl:2017blm
).

Here, we focus on the so-called instantaneous quench in the SSH model and
consider the state

$|\Psi(t)\rangle=e^{-iH_{f}t}|\Omega_{i}\rangle,$ (17)

where $|\Omega_{i}\rangle$ is taken as the ground state of the initial
Hamiltonian $H_{i}$ with parameters $(t^{i}_{1},t^{i}_{2})$, and the evolution
is performed with the SSH Hamiltonian $H_{f}$ with different parameters
$(t^{f}_{1},t^{f}_{2})$. For this state, we will be interested in the Krylov
basis and universal features of the real-time evolution of the spread
complexity defined above.

Curiously, the Nielsen-type geometric complexity measures for free fields
Jefferson:2017sdb  as well as complexity based on the Fubini-Study distance
Chapman:2017rqy , were analyzed during such quenches in the SSH model before
Ali:2018aon . In this work, authors found that above-mentioned complexities
were neither sensitive enough to distinguish topological phases nor whether
the evolution is driven by $H_{f}$ or $H_{i}$. This gives us another strong
motivation to test the spread complexity and its sensitivity out of
equilibrium.

Similarly to the ground state, since the SSH Hamiltonian consists of the
$SU(2)$ generators (for each momentum mode), we can employ coherent state
techniques to expand the quench state (17) in the Krylov basis and find
contribution to spread complexity from each mode. This analysis is based on
the return amplitude (also known as Loschmidt amplitude) that for each mode
$k>0$ reads

$S_{k}(t)=\left<\Psi(t)\right|\Psi(0)\rangle=\cos(R_{f}t)-i\cos(\phi_{f}-\phi_{i})\sin(R_{f}t).$
(18)

In this formula we used the notation $R_{i/f}$ that simply stands for (12)
with parameters $(t^{i/f}_{1},t^{i/f}_{2})$ and similarly $\phi_{i/f}$ are
related to the momentum and parameters as in (11) with
$(t^{i/f}_{1},t^{i/f}_{2})$. From (18) we can repeat the whole procedure to
extract Lanczos coefficients, derive $\psi_{n}(t)$ and compute spread
complexity as before. The contribution from a single momentum mode becomes

$\mathcal{C}_{k}(t)=\sin^{2}(\phi_{f}-\phi_{i})\sin^{2}(R_{f}t).$ (19)

Clearly, for each mode, complexity is periodic in time with periods governed
by the parameters of the evolving Hamiltonian $H_{f}$. The spread complexity
is then not symmetric under the change of parameters of initial and final
Hamiltonians (as in Ali:2018aon ). The amplitude depends on both, initial and
final parameters.

Figure 2: Evolution of spread complexity starting from $H_{i}$ with
$(t^{i}_{1}=1,t^{i}_{2}=0.2)$ and $(t^{f}_{1}=0.7,t^{f}_{2}=1.5)$ (in black)
and the same parameters in $H_{i}$ and $H_{f}$ switched (in red). At late
times complexity approaches constant (23) (in blue).

The total complexity is obtained by integrating over the momenta and
multiplying by the factor of 2 from negative $k$. In the continuum, this can
be written in a compact form as an integral (again a density)

$\mathcal{C}(t)=2\int_{0}^{\pi}{dk\over
2\pi}{\left(t^{f}_{1}t^{i}_{2}-t^{i}_{1}t^{f}_{2}\right)^{2}\sin^{2}(k)\sin^{2}\left(R_{f}t\right)\over
R^{2}_{f}R^{2}_{i}},$ (20)

and for general parameters we need to integrate it numerically. Let us first
study some of the analytical examples. Clearly, the spread complexity vanishes
if $t^{i}_{1}=t^{f}_{1}=0$ or $t^{i}_{2}=t^{f}_{2}=0$. Hence, this measure
assigns zero costs to evolution of states with Hamiltonians on the axes of the
$t_{1}-t_{2}$ plane, as long as they are evolved with Hamiltonian on the same
axis (that may have a different parameter $t^{i/f}_{2}$ in the first or
$t^{i/f}_{1}$ in the second family).
A more interesting analytical example is derived if one of the parameters of
$H_{f}$ vanishes i.e., $t^{f}_{1(2)}=0$. The total spread complexity is then
oscillating with frequency $t^{f}_{2(1)}$

$\mathcal{C}(t)={\left(t^{i}_{1}+t^{i}_{2}-|t^{i}_{1}-t^{i}_{2}|\right)^{2}\over
8(t^{i}_{2(1)})^{2}}\sin^{2}\left(t\,t^{f}_{2(1)}\right),$ (21)

and the amplitude of these oscillations distinguishes the two phases of the
initial quench state $\left|\Omega_{i}\right>$.

Generally, we can just pick two Hamiltonians, each with a pair of generic
parameters in each of the two phases, and compare the evolution of the ground
state $\left|\Omega_{i}\right>$ of $H_{i}$ in the topological phase with
$H_{f}$ in the non-topological phase, and other way around (by just switching
parameters $t^{i}_{a}$ with $t^{f}_{a}$ in (20)). This is plotted in Fig. 2
and, unlike Nielsen complexities studied in Ali:2018aon , we can verify that
our complexity measure can distinguish the two different quench protocols
CommentSH . This is our second key finding.

In addition, we can derive universal results analytically for the early as
well as late time evolution of the spread complexity. For that, it is useful
to parametrize $t^{i}_{2}\equiv\alpha\,t^{i}_{1}$ and
$t^{f}_{2}\equiv\beta\,t^{f}_{1}$, such that the topological and non-
topological phases are separated by $\alpha,\beta=1$. Then, in the early times
we have a universal quadratic growth

$\mathcal{C}(t)\sim(\alpha-\beta)^{2}{(1+\alpha-|1-\alpha|)^{2}\over
8\alpha^{2}}(t^{f}_{1})^{2}t^{2},$ (22)

so this early time regime distinguishes whether $\left|\Omega_{i}\right>$ is
in the topological or non-topological phase.
Also for late times, as it can be observed from Fig. 2, complexity approaches
a constant that is given by

$\displaystyle\mathcal{C}(t\to\infty)={(\alpha-\beta)\over
8\alpha\beta}\times$
$\displaystyle\left(\beta-\alpha+{\alpha(\beta+1)|\beta-1|-\beta(\alpha+1)|\alpha-1|\over
1-\alpha\beta}\right).$ (23)

Clearly, this constant only depends on the two combinations, $\alpha$ and
$\beta$, of the four parameters of the model and is symmetric under their
exchange. Therefore, it is the same for both protocols (see Fig. 2) discussed
above but it depends on the phases of the model. Note that this constant is
the effect of summing of all the, otherwise periodic, contributions from
different momenta. We have analyzed it in more detail in appendix C and it is
clear that this spread complexity constant can be also used as a probe of the
late-time physics of the quench state. More detailed studies of this constant
in different models with topological phases will be presented in PCSL .

##  IV Conclusions and discussion

In this letter, we argued that the spread complexity based on the Krylov basis
Balasubramanian:2022tpr  is a sensitive probe of topological phases. We found
that, for the ground state of the SSH model, the complexity depends on the
ratio of parameters $t_{1}/t_{2}$ in the trivial phase, whereas it becomes
constant in the topological insulator regime. Based on other preliminary
studies PCSL , we expect that this behaviour is universal and further tests
(including numerics) in more general models will be very important to verify
this observation.

For the quantum quench, our main goal was to demonstrate the advantage of the
spread complexity over more geometric measures studied in Ali:2018aon . Not
only was this clearly shown but we also found various universal features of
the evolution, e.g. at early and late times, that are sensitive to the
topological phases. Still, more analysis of the dynamics of spread complexity
(e.g. during slow or fast-type quenches, with time-dependent Hamiltonians or
Floquet driving) is needed and we leave as a promising future direction. Along
the same lines, it will be very interesting to study the spread complexity
(and generally Krylov complexity tools) in the context of dynamical quantum
phase transitions Heyl:2017blm .

Last but not the least, it will be very important to understand the relation
between the spread complexity and other topological order parameters already
studied in the literature, such as Berry phases or Chern number, that can even
be extracted experimentally using certain quench protocols Tarnowski . In
fact, there is a striking similarity in the computation of the Krylov or the
spread complexity and Berry phases. For example, in Caputa:2021sib , it was
noted that Krylov complexity is proportional to the volume in the Hilbert
space measured using the Fubini-Study metric or, equivalently, to an integral
of a certain Berry connection. This link may play an important role in
sharpening the spread complexity as a tool in condensed matter physics as well
as in the holographic explorations on the complexity frontier of black holes.

Acknowledgements We are grateful to Jan Boruch, Diptarka Das, Sumit Das,
Dongsheng Ge, Shajid Haque, Javier Magan, Pratik Nandy, Tokiro Numasawa,
Dimitrios Patramanis, Shinsei Ryu and Christof Weitenberg for useful
discussions and comments. This work is supported by NAWA “Polish Returns 2019”
PPN/PPO/2019/1/00010/U/0001 and NCN Sonata Bis 9 2019/34/E/ST2/00123 grants.

## References

  * (1) J. M. Maldacena, “The Large N limit of superconformal field theories and supergravity” Adv. Theor. Math. Phys. 2 (1998) 231 [Int. J. Theor. Phys. 38 (1999) 1113] [arXiv:hep-th/9711200];
  * (2) D. Harlow and P. Hayden, “Quantum Computation vs. Firewalls,” JHEP 06 (2013), 085 [arXiv:1301.4504 [hep-th]].
  * (3) S. Aaronson, “The Complexity of Quantum States and Transformations: From Quantum Money to Black Holes”, [arXiv:1607.05256 [hep-th]].
  * (4) L. Susskind, “Computational Complexity and Black Hole Horizons,” Fortsch. Phys. 64 (2016) 24 [arXiv:1403.5695 [hep-th], arXiv:1402.5674 [hep-th]];
  * (5) D. Stanford and L. Susskind, “Complexity and Shock Wave Geometries,” Phys. Rev. D 90 (2014) no.12, 126007 [arXiv:1406.2678 [hep-th]].
  * (6) A. R. Brown, D. A. Roberts, L. Susskind, B. Swingle and Y. Zhao, “Holographic Complexity Equals Bulk Action?,” Phys. Rev. Lett. 116 (2016) no.19, 191301 [arXiv:1509.07876 [hep-th]].
  * (7) J. Maldacena, S. H. Shenker and D. Stanford, “A bound on chaos,” JHEP 08 (2016), 106 [arXiv:1503.01409 [hep-th]].
  * (8) A. Bouland, B. Fefferman and U. Vazirani, “Computational pseudorandomness, the wormhole growth paradox, and constraints on the AdS/CFT duality,” [arXiv:1910.14646 [quant-ph]].
  * (9) S. Chapman and G. Policastro, “Quantum computational complexity from quantum information to black holes and back,” Eur. Phys. J. C 82 (2022) no.2, 128 [arXiv:2110.14672 [hep-th]].
  * (10) B. Chen, B. Czech and Z. z. Wang, “Quantum information in holographic duality,” Rept. Prog. Phys. 85 (2022) no.4, 046001 [arXiv:2108.09188 [hep-th]].
  * (11) M. Z. Hasan and C. L. Kane, “Topological Insulators,” Rev. Mod. Phys. 82 (2010), 3045 [arXiv:1002.3895 [cond-mat.mes-hall]].
  * (12) X. L. Qi and S. C. Zhang, “Topological insulators and superconductors,” Rev. Mod. Phys. 83 (2011) no.4, 1057-1110 [arXiv:1008.2026 [cond-mat.mes-hall]].
  * (13) A. P. Schnyder, S. Ryu, A. Furusaki, A.W. W. Ludwig “Classification of topological insulators and superconductors in three spatial dimensions” Phys. Rev. B 78, 195125 (2008)
  * (14) D. Aharonov, L. Eldar, “On the complexity of Commuting Local Hamiltonians, and tight conditions for Topological Order in such systems” [arXiv:1102.0770 [quant-ph]]
  * (15) J. Miller, A. Miyake, “Latent Computational Complexity of Symmetry-Protected Topological Order with Fractional Symmetry” Phys. Rev. Lett. 120, 170503 (2018) [arXiv:1612.08135 [quant-ph]]
  * (16) X. Wen, G. Y. Cho, P. L. S. Lopes, Y. Gu, X. L. Qi and S. Ryu, “Holographic Entanglement Renormalization of Topological Insulators,” Phys. Rev. B 94 (2016) no.7, 075124 [arXiv:1605.07199 [cond-mat.mes-hall]].
  * (17) M. Aguado and G. Vidal, “Entanglement renormalization and topological order,” Phys. Rev. Lett. 100 (2008), 070404 [arXiv:0712.0348 [cond-mat.str-el]].
  * (18) S. Matsuura and S. Ryu, “Momentum space metric, non-local operator, and topological insulators,” Phys. Rev. B 82 (2010), 245113 [arXiv:1007.2200 [cond-mat.mes-hall]].
  * (19) B. Czech, “Holographic State Complexity from Group Cohomology,” [arXiv:2201.01303 [hep-th]].
  * (20) M. A. Nielsen, “A geometric approach to quantum lower bounds”, [arXiv:0502070 [quant-ph]].
M. A. Nielsen, M. R. Dowling, M. Gu and A. C. Doherty “Quantum computation as
geometry”, Science 311, 1133 (2006), [arXiv:0603161 [quant-ph]].

  * (21) R. Jefferson and R. C. Myers, “Circuit complexity in quantum field theory,” JHEP 10 (2017), 107 [arXiv:1707.08570 [hep-th]].
  * (22) S. Chapman, M. P. Heller, H. Marrochio and F. Pastawski, “Toward a Definition of Complexity for Quantum Field Theory States,” Phys. Rev. Lett. 120 (2018) no.12, 121602 [arXiv:1707.08582 [hep-th]].
  * (23) T. Ali, A. Bhattacharyya, S. Shajidul Haque, E. H. Kim and N. Moynihan, “Post-Quench Evolution of Complexity and Entanglement in a Topological System,” Phys. Lett. B 811 (2020), 135919 [arXiv:1811.05985 [hep-th]].
  * (24) F. Liu, S. Whitsitt, J. B. Curtis, R. Lundgren, P. Titum, Z. C. Yang, J. R. Garrison and A. V. Gorshkov, “Circuit complexity across a topological phase transition,” Phys. Rev. Res. 2 (2020) no.1, 013323 [arXiv:1902.10720 [quant-ph]].
  * (25) Z. Xiong, D. X. Yao and Z. Yan, “Nonanalyticity of circuit complexity across topological phase transitions,” Phys. Rev. B 101 (2020) no.17, 174305 [arXiv:1906.11279 [cond-mat.str-el]].
  * (26) V. Balasubramanian, P. Caputa, J. Magan and Q. Wu, “Quantum chaos and the complexity of spread of states,” [arXiv:2202.06957 [hep-th]].
  * (27) W. P. Su, J. R. Schrieffer and A. J. Heeger, “Solitons in polyacetylene,” Phys. Rev. Lett. 42 (1979), 1698-1701
  * (28) P. Caputa, J. M. Magan and D. Patramanis, “Geometry of Krylov complexity,” Phys. Rev. Res. 4 (2022) no.1, 013041 [arXiv:2109.03824 [hep-th]].
  * (29) C. Lanczos, “An iteration method for the solution of the eigenvalue problem of linear differential and integral operators,” J. Res. Natl. Bur. Stand. B 45 (1950), 255-282
V. Viswanath, G. Müller, “The Recursion Method: Application to Many-Body
Dynamics”, vol. 23 Science Business Media

  * (30) D. E. Parker, X. Cao, A. Avdoshkin, T. Scaffidi and E. Altman, “A Universal Operator Growth Hypothesis,” Phys. Rev. X 9 (2019) no.4, 041017 [arXiv:1812.08657 [cond-mat.stat-mech]].
  * (31) D. J. Yates, A. G. Abanov and A. Mitra, “Long-lived period-doubled edge modes of interacting and disorder-free Floquet spin chains,” Commun. Phys. 5 (2022), 43 [arXiv:2105.13766 [cond-mat.str-el]].
D. J. Yates and A. Mitra, “Strong and almost strong modes of Floquet spin
chains in Krylov subspaces,” Phys. Rev. B 104 (2021) no.19, 195121
[arXiv:2105.13246 [cond-mat.str-el]].

  * (32) J. S. Cotler, G. Gur-Ari, M. Hanada, J. Polchinski, P. Saad, S. H. Shenker, D. Stanford, A. Streicher and M. Tezuka, “Black Holes and Random Matrices,” JHEP 05 (2017), 118 [erratum: JHEP 09 (2018), 002] [arXiv:1611.04650 [hep-th]].
  * (33) J.K. Asbóth, L. Oroszlány, A. Pályi, “A short course on topological insulators,” Lecture notes in physics, 2016 - Springer
  * (34) S. Ryu and Y. Hatsugai, “Entanglement entropy and the Berry phase in the solid state,”, Phys. Rev. B 73, 245115 (2006) [arXiv:0601237 [cond-mat.mes-hall]]
  * (35) M. Miyaji, S. Ryu, T. Takayanagi and X. Wen, “Boundary States as Holographic Duals of Trivial Spacetimes,” JHEP 05 (2015), 152 [arXiv:1412.6226 [hep-th]].
  * (36) A. Y. Kitaev, “Unpaired Majorana fermions in quantum wires,” Phys. Usp. 44 (2001) no.10S, 131-136 [arXiv:cond-mat/0010440 [cond-mat.mes-hall]].
  * (37) P. Caputa, S. Liu, work in progress
  * (38) A. Mitra, “Quantum quench dynamics”, Annual Review of Condensed Matter Physics, Vol. 9:245-259 (2018) [ arXiv:1703.09740 [cond-mat.quant-gas]]
  * (39) P. Calabrese and J. Cardy, “Quantum quenches in 1 + 1 dimensional conformal field theories,” J. Stat. Mech. 1606 (2016) no.6, 064003 [arXiv:1603.02889 [cond-mat.stat-mech]].
  * (40) H. A. Camargo, P. Caputa, D. Das, M. P. Heller and R. Jefferson, “Complexity as a novel probe of quantum quenches: universal scalings and purifications,” Phys. Rev. Lett. 122 (2019) no.8, 081601 [arXiv:1807.07075 [hep-th]].
  * (41) S. Liu, “Complexity and scaling in quantum quench in $1+1$ dimensional fermionic field theories,” JHEP 07 (2019), 104 [arXiv:1902.02945 [hep-th]].
  * (42) M. Heyl, “Dynamical quantum phase transitions: a review,” Rept. Prog. Phys. 81 (2018) no.5, 054001 [arXiv:1709.07461 [cond-mat.stat-mech]].
  * (43) Since the analysis in Ali:2018aon  was numerical and mostly focused on late times, it may still be possible that a more careful study of this quenches (along the lines of Ali:2018fcz ) at early times will allow to distinguish the two setups. We thank Shajid Haque for his feedback.
  * (44) T. Ali, A. Bhattacharyya, S. Shajidul Haque, E. H. Kim and N. Moynihan, “Time Evolution of Complexity: A Critique of Three Methods,” JHEP 04 (2019), 087 [arXiv:1810.02734 [hep-th]].
  * (45) Tarnowski, M., Ünal, F.N., Fläschner, N. et al. ,“Measuring topology from dynamics by obtaining the Chern number from a linking number.” Nat Commun 10, 1728 (2019).
  * (46) A. M. Perelomov, “Coherent states for arbitrary Lie group”, Commun. Math. Phys. 26 (1972) 222.
  * (47) J.P. Gazeau, “Coherent States in Quantum Physics”, 2009, Wiley

##  Appendix A Appendix A: Details on SSH

In this appendix we explain some of the details of the SSH model and few steps
to express it in the language of $SU(2)$ coherent states.
First, via a Fourier transformation

$\begin{pmatrix}c_{Al}\\\ c_{Bl}\\\
\end{pmatrix}=N^{-1/2}\sum_{k\in\text{BZ}}e^{ikl}\begin{pmatrix}\tilde{c}_{k}\\\
\tilde{d}_{k}\\\ \end{pmatrix},$ (24)

and a rotation of Pauli matrices $(\sigma_{x},\sigma_{y},\sigma_{z})$ $\to$
$(\sigma_{x},\sigma_{z},-\sigma_{y})$, the Hamiltonian in position space (7)
turns into a free fermion one in momentum space

$H=\sum_{k\in\text{BZ}}\Psi^{\dagger}_{k}\vec{R}(k)\cdot\vec{\sigma}\Psi_{k},\quad\Psi_{k}=\begin{pmatrix}c_{k}\\\
d_{k}\\\ \end{pmatrix}=e^{-{i\pi\over
4}\sigma_{x}}\begin{pmatrix}\tilde{c}_{k}\\\ \tilde{d}_{k}\\\ \end{pmatrix},$
(25)

where the vector of parameters reads

$\vec{R}(k)=\begin{pmatrix}R_{1}\\\ R_{2}\\\ R_{3}\\\
\end{pmatrix}=\begin{pmatrix}t_{1}-t_{2}\cos k\\\ -\mu_{s}\\\ t_{2}\sin k\\\
\end{pmatrix}.$

The Hamiltonian (25) can be further diagonalised into

$H=\sum_{k\in\text{BZ}}\chi_{k}^{\dagger}\operatorname{diag}(R,-R)(k)\chi_{k},$
(26)

where we write the norm $R(k)=\left|\vec{R}(k)\right|$, and

$\chi_{k}=\begin{pmatrix}\chi_{+,k}\\\ \chi_{-.k}\\\
\end{pmatrix}=\begin{pmatrix}\vec{v}_{+}&\vec{v}_{-}\\\
\end{pmatrix}^{\dagger}(k)\Psi_{k}$ (27)

where we introduced orthonomal eigenvectors $\\{\vec{v}_{\pm}\\}$

$\vec{v}_{\pm}(k)={1\over\sqrt{2R(R\mp R_{3})}}\begin{pmatrix}R_{1}-iR_{2}\\\
\pm R-R_{3}\\\ \end{pmatrix}.$ (28)

Thus, the ground state of SSH Hamiltonian is

$|\Omega\rangle=\prod_{k}\chi_{-,k}^{\dagger}|0\rangle$ (29)

where $|0\rangle$ is the Fock vacuum of (25).

On the other hand, (25) consists of three parts – analogues of left- and
right-moving continuum Dirac Hamiltonian,

$H_{L}=\sum_{k}R_{3}c^{\dagger}_{k}c_{k},\qquad
H_{R}=-\sum_{k}R_{3}d^{\dagger}_{k}d_{k},$ (30)

and the mass part

$H_{LR}=\sum_{k}\Psi^{\dagger}_{k}\left(R_{1}\sigma_{1}+R_{2}\sigma_{2}\right)\Psi_{k}.$
(31)

The ground states of $H_{L}$ and $H_{R}$, denoted as $|G_{L}\rangle$ and
$|G_{R}\rangle$, respectively, are

$|G_{L}\rangle=\prod_{k<0}c^{\dagger}_{k}|0\rangle_{L},\qquad|G_{R}\rangle=\prod_{k>0}d^{\dagger}_{k}|0\rangle_{R}.$
(32)

where $|0\rangle_{L,R}$ are the Fock vacuum of $H_{L,R}$, respectively. It is
not hard to see (27) is a Bogoliubov transformation from the creation and
annihliation operators of $H_{L}+H_{R}$ in (30) and $H$ in (25). Therefore the
(unnormalized) ground state $|\Omega\rangle$ defined in (29) can be expressed
in terms of $|G_{L}\rangle$ and $|G_{R}\rangle$ as

$\left|\Omega\right>=\exp\left[-\sum_{k>0}\left({v_{k}\over
u_{k}}c^{\dagger}_{k}d_{k}+{u_{-k}\over
v_{-k}}d^{\dagger}_{-k}c_{-k}\right)\right]\left|G_{L}\right>\otimes\left|G_{R}\right>,$
(33)

where

$\begin{pmatrix}u_{k}\\\ v_{k}\\\
\end{pmatrix}={1\over\sqrt{2R(R+R_{3})}}\begin{pmatrix}R+R_{3}\\\
R_{1}-iR_{2}\\\ \end{pmatrix}.$ (34)

We then introduce the $\operatorname{SU}(2)$ algebra generators $J^{(k)}_{0}$,
$J^{(k)}_{\pm}$ (and similarly $J^{(-k)}_{0}$, $J^{(-k)}_{\pm}$) in terms of
the fermion creation and anihilation operators

$\displaystyle J^{(k)}_{+}=-ic^{\dagger}_{k}d_{k},$ $\displaystyle
J^{(k)}_{-}=id^{\dagger}_{k}c_{k},$ $\displaystyle J^{(k)}_{0}$
$\displaystyle=$ $\displaystyle{1\over
2}(c^{\dagger}_{k}c_{k}-d^{\dagger}_{k}d_{k}),$ (35) $\displaystyle
J^{(-k)}_{+}=ic^{\dagger}_{-k}d_{-k},$ $\displaystyle
J^{(-k)}_{-}=-id^{\dagger}_{-k}c_{-k},$ $\displaystyle J^{(-k)}_{0}$
$\displaystyle=$ $\displaystyle{1\over
2}(c^{\dagger}_{-k}c_{-k}-d^{\dagger}_{-k}d_{-k}),$ (36)

where $k>0$. Thus, the ground state (29) can be rewritten in terms of
$\operatorname{SU}(2)$ coherent states

$\displaystyle|\Omega\rangle$ $\displaystyle=$
$\displaystyle\prod_{k>0}\cos^{2}{\phi_{k}\over 2}e^{-i\tan{\phi_{k}\over
2}\left(e^{-i\psi_{k}}J_{+}^{(k)}+e^{i\psi_{k}}J_{+}^{(-k)}\right)}$ (37)
$\displaystyle\qquad\qquad\bigotimes_{k\in\text{BZ}}\left|1/2;-1/2\right\rangle_{k}.$

Finally, the Hamiltonian of SSH model can be rewritten in terms of
$\operatorname{SU}(2)$ algebra generators as

$\displaystyle H=\sum_{k>0}R$
$\displaystyle\left[2\cos\phi_{k}\left(J_{0}^{(k)}+J_{0}^{(-k)}\right)\right.$
(38)
$\displaystyle\quad+i\sin\phi_{k}\left(e^{-i\psi_{k}}J_{+}^{(k)}+e^{i\psi_{k}}J_{+}^{(-k)}\right)$
$\displaystyle\quad\left.-i\sin\phi_{k}\left(e^{i\psi_{k}}J_{-}^{(k)}+e^{i\psi_{k}}J_{-}^{(-k)}\right)\right].$

Above we have used spherical coordinates to rewrite $\vec{R}(k)$, in
particular,

$\displaystyle\sin\phi_{k}\cos\psi_{k}$ $\displaystyle=$
$\displaystyle{R_{1}\over R},$ $\displaystyle\sin\phi_{k}\sin\psi_{k}$
$\displaystyle=$ $\displaystyle{R_{2}\over R},$ $\displaystyle\cos\phi_{k}$
$\displaystyle=$ $\displaystyle{R_{3}\over R}.$ (39)

In the main text we have worked with $\mu_{s}=0$ that corresponds to
$R_{2}=\psi_{k}=0$ in these expressions.

##  Appendix B Appendix B: SU(2) coherent states

To make this article self-contained, we collect some of the tools from $SU(2)$
coherent states that were used in the main text (see more in e.g. coherent2 ;
gazeau2009 ). We start with $SU(2)$ algebra
$[J_{i},J_{j}]=i\epsilon_{ijk}J_{k}$, expressed in terms of the ladder
operators $J_{\pm}=J_{1}\pm iJ_{2}$ and $J_{0}=J_{3}$ as

$[J_{+},J_{-}]=2J_{0},\qquad[J_{0},J_{\pm}]=\pm J_{\pm}.$ (40)

The relevant coherent states are defined by action of a displacement operator
on the lowest-weight state $\left|j,-j\right>$ (i.e.,
$J_{-}\left|j,-j\right>=0$) labeled by (half or) integer $j$

$\left|z,j\right>=e^{\xi
J_{+}-\bar{\xi}J_{-}}\left|j,-j\right>=e^{zJ_{+}}e^{\ln(1+z\bar{z})J_{0}}e^{-\bar{z}J_{-}}\left|j,-j\right>.$
(41)

In the above we defined complex parameter

$\xi={\theta\over 2}e^{-i\varphi},\qquad 0\leq\theta<\pi,\quad
0\leq\varphi\leq 2\pi,$ (42)

with its complex conjugate $\bar{\xi}$ as well as

$z=\tan\left({\theta\over 2}\right)e^{-i\varphi},$ (43)

with complex conjugate $\bar{z}$.
These coherent states are employed in the Krylov/spread complexity context
when we e.g. chose $\xi=-is{\phi\over 2}$, so $\theta=s\phi$ and
$\varphi=\pi/2$, such that the state becomes

$\displaystyle\left|z,j\right>=e^{-is{\phi\over
2}(J_{+}+J_{-})}\left|j,-j\right>=\sum^{2j}_{n=0}\psi_{n}(s)\left|K_{n}\right>,$
(44)

where

$\psi_{n}(s)={(-i)^{n}\tan^{n}\left({s\phi\over
2}\right)\over\left(\cos{s\phi\over 2}\right)^{-2j}}\sqrt{{\Gamma(2j+1)\over
n!\Gamma(2j-n+1)}},$ (45)

and Krylov basis vectors are

$\left|K_{n}\right>=\left|j,-j+n\right>=\sqrt{{\Gamma(2j-n+1)\over
n!\Gamma(2j+1)}}J^{n}_{+}\left|j,-j\right>.$ (46)

The amplitude (45) satisfies a discrete Schrodinger equation (5) with
$a_{n}=0$ and $2j+1$ coefficients

$b_{n}={\phi\over 2}\sqrt{n(2j-n+1)},\qquad n=0,...,2j.$ (47)

The Krylov or spread complexity in this case becomes

$\mathcal{C}(s)=\sum_{n}n|\psi_{n}(s)|^{2}=2j\sin^{2}\left({s\phi\over
2}\right).$ (48)

In the main text, we only needed these results for the $j=1/2$ representation
where

$\mathcal{C}(s)=|\Psi_{1}(s)|^{2}=\sin^{2}\left({s\phi\over 2}\right).$ (49)

More generally, we may consider time evolution

$\left|\Psi(t)\right>=e^{-iHt}\left|j,-j\right>,$ (50)

with coherence preserving Hamiltonian

$H=\gamma J_{0}+\alpha(J_{+}+J_{-}).$ (51)

Then the state can be written as

$\left|\Psi(t)\right>=\sum^{2j}_{n=0}\Psi_{n}(t)\left|K_{n}\right>,$ (52)

with Krylov basis vectors (46) and coefficients

$\Psi_{n}(t)=e^{-jB}A^{n}\sqrt{{\Gamma(2j+1)\over n!\Gamma(2j-n+1)}},$ (53)

where

$\displaystyle A$ $\displaystyle=$ $\displaystyle{1\over
i\sqrt{1+{\gamma^{2}\over 4\alpha^{2}}}\cot\left(\alpha
t\sqrt{1+{\gamma^{2}\over 4\alpha^{2}}}\right)-{\gamma\over 2\alpha}},$ (54)
$\displaystyle e^{-jB}$ $\displaystyle=$ $\displaystyle\left(\cos\left(\alpha
t\sqrt{1+{\gamma^{2}\over 4\alpha^{2}}}\right)+\right.$ (55)
$\displaystyle\left.{i\gamma\over 2\alpha\sqrt{1+{\gamma^{2}\over
4\alpha^{2}}}}\sin\left(\alpha t\sqrt{1+{\gamma^{2}\over
4\alpha^{2}}}\right)\right)^{2j}.$

Coefficients $\Psi_{n}(t)$ in (53) are solutions of the Schrodinger equation
(5) with Lanczos coefficients

$a_{n}=\gamma(-j+n),\qquad b_{n}=\alpha\sqrt{n(2j-n+1)}.$ (56)

Finally, the complexity becomes

$\mathcal{C}(t)=\sum_{n}n|\Psi_{n}(t)|^{2}={2j\over 1+{\gamma^{2}\over
4\alpha^{2}}}\sin^{2}\left(\alpha t\sqrt{1+{\gamma^{2}\over
4\alpha^{2}}}\right).$ (57)

For the quench, we will also needed the following return amplitude

$S(t)=\left<j,-j\right|D^{\dagger}(z_{i})e^{iH_{f}t}D(z_{i})\left|j,-j\right>$
(58)

where the $SU(2)$ displacement operator is

$D(z_{i})=e^{-i{\phi_{i}\over 2}(J_{+}+J_{-})},$ (59)

and a Hamiltonian $H_{f}$ has a form (51) with $\gamma=2R_{3}$ and
$\alpha=iR_{1}$.
Generally, using the BCH relation we derive

$S(t)=\left(\cos(R_{f}t)-i\cos(\phi_{f}-\phi_{i})\sin(R_{f}t)\right)^{2j}.$
(60)

For $j=1/2$, the moments of this return amplitude are

$\mu_{n}=R^{n}_{f}\left(\cos{\pi n\over 2}-i\cos(\phi_{f}-\phi_{i})\sin{\pi
n\over 2}\right),$ (61)

and they correspond to the two Lanczos coefficients

$\displaystyle a_{0}$ $\displaystyle=$ $\displaystyle-
R_{f}\cos(\phi_{f}-\phi_{i}),\qquad a_{1}=R_{f}\cos(\phi_{f}-\phi_{i}),$
$\displaystyle b_{1}$ $\displaystyle=$ $\displaystyle
R_{f}|\sin(\phi_{f}-\phi_{i})|,$ (62)

while all the higher ones vanishing. This again allowed us to map our state
expanded in the Krylov basis to $SU(2)$ coherent state

$\displaystyle\left|\Psi(t)\right>=e^{-iH_{t}t}\left|\Omega_{i}\right>=\sum^{1}_{n=0}\psi_{n}(t)\left|K_{n}\right>,$
(63)

where

$\psi_{0}(t)=S(t)^{*},\quad\psi_{1}(t)=-i|\sin(\phi_{f}-\phi_{i})|\sin(R_{f}t),$
(64)

and the two basis vectors $\left|K_{0}\right>=\left|1/2,-1/2\right>$,
$\left|K_{1}\right>=\left|1/2,1/2\right>$. This leads to the spread complexity
per mode (19).
Finally, observe that for $j=1/2$ representations of $SU(2)$, since we only
have two complex amplitudes $\Psi_{0}(t)$ and $\Psi_{1}(t)$, we can relate the
spread complexity $\mathcal{C}(t)=|\psi_{1}(t)|^{2}$ to the return amplitude
$S(t)=\bar{\psi}_{0}(t)$. Indeed, by unitarity, probabilities sum up to
identity and we have

$|\Psi_{0}|^{2}+|\Psi_{1}|^{2}=1\leftrightarrow\mathcal{C}(t)=1-|S(t)|^{2}.$
(65)

This simple relation will not hold for a more general time evolution that
requires higher number of Krylov basis vectors. Nevertheless, understanding
the relation between spread complexity and Loschmidt echo is an interesting
future problem. We thank Diptarka Das for pointing this to us.

##  Appendix C Appendix C: Early and late times

This appendix contains more details on universal behaviours of the evolution
of complexity in the early and late times.

Figure 3: Spread complexity constant at late times
$\mathcal{C}(t\to\infty;\alpha,\beta)$, shown in (23).

Firstly, at the early times, after of the quench, i.e., $t\sim 0^{+}$, the
time evolution of complexity (20) becomes

$\mathcal{C}(t\sim 0)\simeq
t^{2}\times\int_{0}^{\pi}{dk\over\pi}{\left(t^{f}_{1}t^{i}_{2}-t^{i}_{1}t^{f}_{2}\right)^{2}\sin^{2}(k)\over
R^{2}_{i}}.$ (66)

In terms of the ratios of $t_{2}^{i,f}/t_{1}^{i,f}$, denoted as $\alpha$ and
$\beta$ for initial and final parameters, respectively, the spread complexity
is

$\displaystyle\mathcal{C}(t\sim 0)$ $\displaystyle=$
$\displaystyle(t\,t^{f}_{1})^{2}\left(\alpha-\beta\right)^{2}{\left(|1+\alpha|-|1-\alpha|\right)^{2}\over
8\alpha^{2}}$ $\displaystyle=$ $\displaystyle
t^{2}\left(t^{f}_{1}t^{i}_{2}-t^{i}_{1}t^{f}_{2}\right)^{2}{\left(t_{1}^{i}+t_{2}^{i}-\left|t_{1}^{i}-t_{2}^{i}\right|\right)^{2}\over
8\left(t_{1}^{i}t_{2}^{i}\right)^{2}}.$

On the other hand, for late times, i.e., $t\to\infty$, we notice that the time
evolution of (20) can be estimated as follows. First, (20) is equivalent to

$\mathcal{C}(t)=2\int_{0}^{\pi}{dk\over
2\pi}{\left(t^{f}_{1}t^{i}_{2}-t^{i}_{1}t^{f}_{2}\right)^{2}\sin^{2}(k)\over
R^{2}_{f}R^{2}_{i}}\times{1-\cos 2R_{f}t\over 2}.$ (68)

We can then change the integration variable in the second term such that

$\begin{split}&\int_{0}^{\pi}{dk\over
2\pi}{\left(t^{f}_{1}t^{i}_{2}-t^{i}_{1}t^{f}_{2}\right)^{2}\sin^{2}(k)\over
R^{2}_{f}R^{2}_{i}}\cos 2R_{f}t\\\
&={\left(t^{f}_{1}t^{i}_{2}-t^{i}_{1}t^{f}_{2}\right)^{2}\over
t_{1}^{f}t_{2}^{f}}\int_{k(R_{f})\in(0,\pi)}{dR_{f}\over 2\pi}{\sin
k(R_{f})\over R_{f}\cdot R^{2}_{i}(R_{f})}\cos 2R_{f}t.\end{split}$ (69)

Now, because for $t\to\infty$ the cosine function is highly oscillating in the
interval $(0,|t_{1,f}\pm t_{2,f}|]$, where the integral range belongs to, this
integral is always negligible, no matter what the concrete form of ${\sin
k(R_{f})\over R_{f}\cdot R^{2}_{i}(R_{f})}$ in terms of $R_{f}$ is. This way
we find the late time constant as

$\mathcal{C}(t\to\infty)\simeq\int_{0}^{\pi}{dk\over
2\pi}{\left(t^{f}_{1}t^{i}_{2}-t^{i}_{1}t^{f}_{2}\right)^{2}\sin^{2}(k)\over
R^{2}_{f}R^{2}_{i}}.$ (70)

This is also equivalent to replacing the time dependent factor
$\sin^{2}(R_{f}t)$ by $1/2$. This expression integrates to (23) and is plotted
in Fig. 3. Note that naively, (23) is singular for $\alpha\beta=1$ (the white
diagonal on Fig. 3), $\alpha=0$ and $\beta=0$. These cases should be analyzed
more carefully starting from (20) with these values of $\alpha$ and $\beta$
and only then preforming the integral over momenta.