# Pinning-depinning transitions in two classes of discrete elastic-string
models in (2+1)-dimensions

Yongxin Wu  School of Materials Science and Physics, China University of
Mining and Technology, Xuzhou 221116, China  Hui Xia  E-mail<EMAIL_ADDRESS>School of Materials Science and Physics, China University of Mining and
Technology, Xuzhou 221116, China

###### Abstract

The pinning-depinning phase transitions of interfaces for two classes of
discrete elastic-string models are investigated numerically. In the
(1+1)-dimensions, we revisit these two elastic-string models with slight
modification on growth rule, and compare with the previous numerical results.
In the (2+1)-dimensional case, we perform extensive simulations on pinning-
depinning transitions in these quenched models. For full comparisons in the
physically relevant spatial dimensions, we also perform numerically two
distinct quenched universality classes, including the quenched Edwards-
Wilkinson (QEW) and the quenched Kardar-Parisi-Zhang (QKPZ) equations with and
without external driving forces. The critical exponents of these quenched
systems are numerically estimated. Our results show that the critical
exponents satisfy scaling relations well, and these two discrete elastic-
string models do not fall into the existing universality classes. In order to
visually comparisons of these quenched systems in (2+1)-dimensional cases, we
exhibit surface morphologies with various external driving forces during the
saturated time regimes.

††preprint: APS/123-QED

##  I Introduction

In recent decades, much attention has been attached to the pinning-depinning
transition of a driven interface in quenched random media [1, 2, 3, 4, 5, 6,
7, 8, 9], which has served for interpretation of many other physical phenomena
such as immiscible displacements of fluids in porous media [10], field-driven
motion of domain walls in magnetic systems [11], vortices and flux lines in
superconductors [12, 3, 4, 13, 14, 15, 16, 17].

As a typical paradigm, the Kardar-Parisi-Zhang (KPZ) equation is widely
utilized in describing the evolution of surface and interface [18, 19, 20].
Driven by thermal noise $\eta(x,t)$ [13], the KPZ equation can be written as
[2]

$\frac{\partial h(\vec{r},t)}{\partial
t}=\nu\nabla^{2}h+\frac{\lambda}{2}(\nabla h)^{2}+\eta(\vec{r},t),$ (1)

where $\nu$ represents the surface tension smoothing the interface, and the
nonlinear term with $\lambda$ reflects lateral growth.

However, the quenched noise $\eta(\vec{r},h)$ usually wins the competition
with thermal noise in disordered medium. The former plays a more important
role in the pinning-depinning transition, which is generated by the quenched
disorder [12, 13, 21, 22, 23, 24]. Thus, Eq. (1) becomes quenched KPZ (QKPZ),

$\frac{\partial h(\vec{r},t)}{\partial
t}=F+\nu\nabla^{2}h+\frac{\lambda}{2}(\nabla h)^{2}+\eta(\vec{r},h).$ (2)

Here $F$ is the external driving force, and the quenched noise
$\eta(\vec{r},h)$ is regarded as Gaussian noise with zero mean, and satisfies
[2]

$\left\langle\eta(\vec{r},h)\eta(\vec{r^{\prime}},h^{\prime})\right\rangle=\delta^{d}(\vec{r}-\vec{r^{\prime}})\Delta(h-h^{\prime}),$
(3)

where the brackets $\langle\dots\rangle$ is the sample average. Many studies
have been devoted to the scaling behavior of the QKPZ model in the
(1+1)-dimensional case [25, 26, 27, 28]. However, (2+1)-dimensional case is
the most physically relevant, for example, after a cubic sponge contacts water
surface, the phenomenon that the fluid driven by the capillary force
propagates in the sponge could be described by (2+1)-dimensional quenched
models [2]. Therefore, we mainly focus on the pinning-depinning transition of
(2+1)-dimensional quenched models in the current work.

Firstly, we introduce briefly these elastic strings models [29, 30, 31]. In
these quenched models, the external driving force $f$ is applied to an elastic
string evolving in a medium with quenched disorder. While the interface is
driven by external force, it is also hindered by the pinning force $f_{p}$
generated by quenched disorder. In addition, there is another force called
elastic force $f_{el}$, which is originated by the surrounding sites owing to
elastic stretching of the string. When a site falls behind the surrounding
ones, the elastic force acts as a driving force. Oppositely, when the site
exceeds the surrounding ones, the elastic force acts as a resistance. In other
words, the elastic force tends to smooth the interface, whose function is
similar to that of the diffusion term $\nu\nabla^{2}h$ in the QKPZ model. If
the driving force $f$ is too small, the interface may be pinned by the
quenched disorder, i.e., the long-term velocity of interface is zero.
Inversely, when the driving force increases until it exceeds a threshold
$f_{c}$, the interface will be driven through the medium with a non-zero
velocity. Near $f_{c}$ , many interesting properties deserve to be studied,
and various scaling exponents characterize the pinning-depinning transitions.
Considering previous work focused mainly on the (1+1)-dimensional stochastic
systems. Thus, we will be committed to the generalizations and improvements on
these quenched models with elastic strings. For full comparisons in the
physically relevant case of (2+1)-dimensions, the QKPZ equation is also chosen
to investigate numerically as the well-known universality class. Actually,
when $\lambda\to 0$, QKPZ reduces to the quenched Edwards-Wilkinson (QEW) [32,
2]. And then, the comparison with one existing discrete model belonging to QEW
class [33] will also be given.

The paper is organised as follows: Firstly, we introduce some basic concepts
and numerical methods to describe quantitatively these quenched systems
mentioned above. Then, we present our numerical results from two classes of
discrete elastic-string models in the (1+1)- and (2+1)-dimensions,
respectively, and compare with those from QEW and QKPZ for the
(2+1)-dimensional cases. Finally, discussions and conclusions are provided.

Figure 1: Log-log plots of average velocity $v$ as a function of time $t$ in
Model I. (a) The driving force $f=0.680,0.676,\dots,0.640$ from the top to the
bottom with the system size $L=1024$ in the (1+1)-dimensions. (b)
$f=0.633,0.631,\dots,0.621,0.617,\dots,0.605$ from the top to the bottom with
system size $L\times L=110\times 110$ in the (2+1)-dimensions. Insets show $v$
versus $t$ near the critical thresholds: (a) $f_{c}\approx 0.660$ and (b)
$f_{c}\approx 0.621$.

Figure 2: Log-log plots of stationary velocity $v$ as a function of $f-f_{c}$
near the critical thresholds in Model I: (a) (1+1)-dimensional, and (b)
(2+1)-dimensional cases. Insets show log-log plots of $W(L,t)$ as a function
of $t$ at $f_{c}$, and the growth exponents are estimated correspondingly.

Figure 3: Near the critical thresholds, log-log plots of $vt^{\delta}$ as a
function of $t|f-f_{c}|^{\nu}$ in Model I. The critical exponents are chosen
in order to well fit data collapse: (a) $\delta=0.730$ and $\nu=1.50$ in the
(1+1)-dimensions, (b) $\delta=0.900$ and $\nu=1.48$ in the (2+1)-dimensions.
These lines for $f>f_{c}$ curve upward, and curve downward for $f<f_{c}$.

Figure 4: Log-log plots of $v$ as a function of time $t$ in Model II. The
elastic force is fixed $F=0.6$, and the driving force
$f=2.9995,2.9997,\dots,3.0009$ are chosen from bottom to top. (a) shows the
(1+1)-dimensional results, and (b) exhibits the (2+1)-dimensional results. As
is indicated in the insets, near $f_{c}$, the estimated values of critical
exponent $\delta$ are obtained correspondingly. Figure 5: Log-log plot of $v$
as a function of time $t$ in the (2+1)-dimensional Model II with the external
driving force $f=3.0000$, and the elastic force $F=0.5996,0.5997,\dots,0.6006$
from bottom to top. Inset shows $v$ versus $t$ near the critical threshold
$F_{c}\approx 0.6000$ and the estimated value of critical exponent
$\delta^{\prime}$.

Figure 6: Log-log plots of stationary velocity $v$ as a function of $f-f_{c}$
in Model II. (a) $f_{c}\approx 3.0000,\theta=0.32\pm 0.01$ in the
(1+1)-dimensions. (b) $f_{c}\approx 3.0000,\theta=0.41\pm 0.02$ in the
(2+1)-dimensions. Insets show log-log plots of $W$ as a function of $t$ near
$f_{c}$, and the values of growth exponent $\beta$ are estimated
correspondingly.

Figure 7: Log-log plots of $vt^{\delta}$ as a function of $t|f-f_{c}|^{\nu}$
in Model II. The critical exponents are chosen in order to well fit data
collapse: (a) $\delta=0.330$ and $\nu=0.970$ in the (1+1)-dimensions, (b)
$\delta=0.460$ and $\nu=0.90$ in the (2+1)-dimensions. Figure 8: Log-log plot
of $v$ as a function of time $t$ of (2+1)-dimensional QKPZ with the nonlinear
coefficient $\lambda=5$ and the external force
$F=-1.40,-1.30,\dots,-0.60,-0.55$. Inset shows $v$ versus $t$ near critical
threshold $F_{c}\approx-1.10$, and the critical exponent $\delta$ is estimated
correspondingly. Figure 9: Log-log plot of stationary velocity $v$ as a
function of $F-F_{c}$ of the (2+1)-dimensional QKPZ with $F_{c}\approx-1.10$.
Inset shows log-log plot of $W$ as a function of $t$ near $F_{c}$ and the
estimated value of $\beta$.  Figure 10: Log-log plot of $vt^{\delta}$ as a
function of $t|F-F_{c}|^{\nu}$ of the (2+1)-dimensional QKPZ system. The
critical exponents $\delta=0.775$ and $\nu=0.95$ are chosen in order to well
fit data collapse.

Figure 11: Surface morphologies in the (2+1)-dimensional quenched systems:
(a)-(c) in Model I, (d)-(f) in Model II, (g)-(i) in QEW, and (j)-(l) in QKPZ
during the saturated time regimes with three different external driving forces
including low $F$($f$), $F_{c}$($f_{c}$), and high $F$ ($f$).

##  II methods and basic concepts

Here we briefly introduce two typical elastic-string models proposed
originally in [30], slightly modify in the (1+1)-dimensions, and then
generalize to the (2+1)-dimensional case. As a comparison, the discretized
scheme of Eq. (2) in (2+1)-dimensions is also introduced. Then we present some
basic concepts used in the following numerical simulations.

###  II.1 Model I

Model I follows the following evolving equation,

$h\left(\vec{r},t+1\right)=h\left(\vec{r},t\right)+\left|f_{el}\right|\hat{S}(\vec{r},t)+\eta\left(\vec{r},h\left(\vec{r},t\right)\right)+f,$
(4)

with $h(\vec{r},t)$ is the height of element with coordinate $\vec{r}$ at time
$t$. $f$ is external driving force. The expression of $\hat{S}(\vec{r},t)$ can
be interpreted as the description on elastic force in SEC. I. More
specifically, $\hat{S}(\vec{r},t)=sgn(h(x+1,t)+h(x-1,t)-2h(x,t))$ for
(1+1)-dimensions, and $\hat{S}(\vec{r},t)=sgn(h(x,y-1,t)+h(x,y+1,t)$
$+h(x-1,y,t)+h(x+1,y,t)-4h(x,y,t))$ for (2+1)-dimensional case.
$\eta(\vec{r},h(\vec{r},t))$ is the quenched disorder in medium, which is
similar to the impurities distributed in paper towels in the (1+1)-dimensions
or porous media in sponge block in the (2+1)-dimensions. The magnitude of the
elastic force in the (1+1)-dimensional case is determined by following formula
[30],

$|f_{el}|=\frac{1}{L}\sum_{x}\left\\{\sqrt{\left[h(x,t)-h(x+1,t)\right]^{2}+1}-1\right\\},$
(5)

and the corresponding (2+1)-dimensional form reads,

$|f_{el}|=\frac{1}{L^{2}}\sum_{x,y}\biggl{\\{}\sqrt{\left[h(x,y,t)-h(x+1,y,t)\right]^{2}+1}\\\
+\sqrt{\left[h(x,y,t)-h(x,y+1,t)\right]^{2}+1}-2\biggr{\\}}.$ (6)

In order to investigate the pinning-depinning transition of Model I, we need
to discretize Eq. (4),

$h(\vec{r},t+\Delta t)=\begin{cases}h(\vec{r},t)+g_{1}(\vec{r},t)\Delta
t&\text{if}\quad g_{1}(\vec{r},t)>0,\\\ h(\vec{r},t)&\text{if}\quad
g_{1}(\vec{r},t)\leq 0,\end{cases}$ (7)

where
$g_{1}(\vec{r},t)=|f_{el}|\hat{S}(\vec{r},t)+\eta\left(\vec{r},\widetilde{h}(\vec{r},t)\right)+f$,
$\widetilde{h}$ is the integer part of $h(\vec{r},t)$, and $\Delta t$ is time
step.

###  II.2 Model II

For Model II, we set the magnitude of the elastic force $F$ to be constant.
Firstly, we revisit this quenched model, and well reproduced numerical results
in Ref. [30], and on this basis, we propose a new noise form
$\eta\left(\vec{r},h(\vec{r},t)\right)$, i.e., the noise is not only a
function of $\vec{r}$, but also a function of $h(\vec{r},t)$ for position
$\vec{r}$ at time $t$, which acts as a quenched disorder generated by impurity
in heterogeneous material, where the phenomenon of crack front propagation
[34, 35] occurs. This kind of the evolving quenched equation for height can be
rewritten as:

$h(\vec{r},t+1)=h(\vec{r},t)+F\hat{S}(\vec{r},t)+\eta\left(\vec{r},h(\vec{r},t)\right)+f,$
(8)

with constant $F$.

Similar to Model I, we need to discretize Eq. (8) in order to simulate
effectively, and the discretized version reads,

$h(\vec{r},t+\Delta t)=\begin{cases}h(\vec{r},t)+g_{2}(\vec{r},t)\Delta
t,&\text{if}\quad g_{2}(\vec{r},t)>0,\\\ h(\vec{r},t),&\text{if}\quad
g_{2}(\vec{r},t)\leq 0,\end{cases}$ (9)

where
$g_{2}(\vec{r},t)=F\hat{S}(\vec{r},t)+\eta\left(\vec{r},h(\vec{r},t)\right)+f$.
Here the meanings of $\hat{S}$ and $f$ are the same with the corresponding
variables in Model I.

###  II.3 (2+1)-dimensional QKPZ

We integrate numerically Eq. (2) in (2+1)-dimensions using the finite-
difference (FD) method [36, 37, 38, 39]. The height $h(x,y,t)$ is updated by
the evolving discretized scheme,

$h(x,y,t+\Delta t)=h(x,y,t)+\Delta t\biggl{\\{}\nu\bigl{[}h(x+1,y,t)\\\
+h(x-1,y,t)+h(x,y+1,t)+h(x,y-1,t)-4h(x,y,t)\bigr{]}\\\
+\frac{\lambda}{8}\bigl{[}h(x+1,y,t)+h(x-1,y,t)\bigr{]}^{2}\\\
+\frac{\lambda}{8}\bigl{[}h(x,y+1,t)+h(x,y-1,t)\bigr{]}^{2}\\\
+\eta(x,y,\tilde{h})+F\biggr{\\}}.$ (10)

In the following simulations, we set $\nu=1$, $\Delta t=0.01$. $\tilde{h}$ is
the integer part of height, and $\eta(x,y,\tilde{h})$ is uniformly distributed
in [-6 , 6]. The simulating processes based on time evolution of Eq. (10)
begin from $h(x,y,0)=0$ with periodic boundary condition.

###  II.4 Basic concepts

In order to depict the pinning-depinning transition quantitatively, some
characteristic physical quantities need to be introduced. In the process of
time evolution, the average velocity of moving interface can be defined as

$v(t)=\frac{1}{L}\sum_{i=1}^{L}{\left(h(i,t)-h(i,t-1\right))}.$ (11)

At the critical point, the average velocity of interface follows,

$v(t)\sim t^{-\delta},$ (12)

where $\delta$ is the critical exponent. Once identifying successfully the
critical point, one could calculate some of the scaling exponents related to
the pinning-depinning transition.

The interface width, namely, the mean square fluctuation can help us
characterize the roughness properties of interface,

$W(L,t)=\frac{1}{\sqrt{L}}\langle\sum_{i=1}^{L}\left[h(i,t)-\langle
h(t)\rangle\right]^{2}\rangle^{\frac{1}{2}},$ (13)

which follows the scaling ansatz [2],

$W(L,t)\sim\begin{cases}t^{\beta}\quad&\text{if}\quad t\ll t_{\times}\\\
L^{\alpha}\quad&\text{if}\quad t\gg t_{\times},\end{cases}$ (14)

where $t_{\times}\sim L^{z}$, and $\beta$ is the growth exponent. One could
expect $v\left(t\right)\sim W\left(L,t\right)/t$, implying $\beta+\delta=1$
[25, 40, 41].

With the time evolution, the stationary velocity above the critical point
follows,

$v\left(t\rightarrow\infty,f\right)\sim\left(f-f_{c}\right)^{\theta},$ (15)

where $\theta$ is the velocity exponent. Interestingly, when $F=0$, one could
find the critical point by adjusting the amplitude of nonlinear coefficient.
And there also exists a scaling ansatz above the critical $\lambda_{c}$,

$v\left(t\rightarrow\infty,\lambda\right)\sim\left(\lambda-\lambda_{c}\right)^{\theta^{\prime}}.$
(16)

In order to further obtain the critical exponent near the critical point, we
consider finite-size scaling ansatz as follows [42, 30],

$v\left(f,t\right)\sim t^{-\delta}F\left(t\left|f-f_{c}\right|^{\nu}\right),$
(17)

where $\nu$ is the exponent related to the diverging correlation time. One
could expect that the scaling relation $\nu=\theta/\delta$ always satisfies.
Correspondingly, when $F=0$, the scaling ansatz is replaced by

$v\left(\lambda,t\right)\sim
t^{-\delta^{\prime}}\Lambda\left(t\left|\lambda-\lambda_{c}\right|^{\nu^{\prime}}\right).$
(18)

##  III Numerical simulations and results

###  III.1 Model I

In this subsection, we focus on simulations and comparisons of the critical
exponents of pinning-depinning transition in Model I. For this purpose, our
priority is to look for the critical threshold. Considering that the magnitude
of the elastic force is variable in this model, and $|f_{el}|$ depends on Eq.
(5) or Eq. (6), we need to tune the magnitude of driving force $f$ to induce
the critical state. In our numerical simulations, different system sizes are
used: $L=1024$ in the (1+1)-dimensions, and $L\times L=110\times 110$ in the
(2+1)-dimensional case. The power-law scaling of the average velocity $v$
versus time $t$ in both cases is shown in Fig. 1. For the pinned region
($f<f_{c}$), as the absolute value of curve slope increases, $v$ decays faster
until $v=0$. On the contrary, in the depinned region ($f>f_{c}$), $v$ decays
following a power-law at early growth time, and then the curves bend upward.
Only near $f_{c}$, $v$ shows a power-law scaling as $v\sim t^{-\delta}$. We
obtain the critical exponents $\delta=0.730\pm 0.005$ with $f_{c}\approx
0.660$ in the (1+1)-dimensions, and $\delta=0.900\pm 0.005$ with $f_{c}\approx
0.621$ in the (2+1)-dimensions, as shown in the insets of Fig. 1.

When the critical threshold is determined, the scaling properties near the
critical point are investigated. As is mentioned in SEC. II, we exhibit the
power-law scaling of surface roughness $W(L,t)$ as a function of time $t$.
Typical curves of $W(L,t)$ at $f_{c}$ are shown in the insets of Fig. 2. The
growth exponents $\beta=0.28\pm 0.01$ and $\beta=0.120\pm 0.005$ are obtained
from the (1+1)- and (2+1)-dimensional cases, respectively. Our results show
that $\delta+\beta\approx 1.01$ in the (1+1)-dimensions, and
$\delta+\beta\approx 1.02$ in the (2+1)-dimensions, which implies that these
critical exponents satisfy the scaling relation $\beta+\delta=1$ very well in
these two dimensions. In the depinned region, we plot the saturated velocity
$v$ as a function of $f-f_{c}$, as shown in Fig. 2. Thus, we obtain that the
critical exponents $\theta=1.27\pm 0.03$ for the (1+1)- and $\theta=1.34\pm
0.05$ for the (2+1)-dimensional cases, respectively. Furthermore, based on
scaling ansatz (17), we perform data collapses to determine the critical
exponent $\nu$. As shown in Fig. 3, these data collapse into two lines nicely
with $\delta=0.730$, $\nu=1.50\pm 0.05$ in the (1+1)-dimensions, and
$\delta=0.900$, $\nu=1.48\pm 0.03$ in the (2+1)-dimensions. Thus, we verify
that another scaling relation $\nu=\theta/\delta$ could satisfy well in Model
I.

###  III.2 Model II

Following the similar research steps as Model I, we first determine the
criticality of Model II. Interestingly, the critical threshold here is not
entirely dependent on the external driving force $f$, but also depends on the
magnitude of the elastic force $F$ between the elastic strings, so how to deal
with the interplay between them becomes very important.

Actually, using extensive numerical simulations, we find that $f$ and $F$ all
play a central role in promoting the whole evolving process, while quenched
noise takes vital effect in resistance (pinning force $f_{p}$). Since the
quenched noise is generated in a set of non-positive random numbers, the
random number with the largest absolute value is the maximum resistance
$f_{pmax}$. Therefore, in Model II, we can summarize the formula to determine
the critical threshold:

$f_{pmax}=-Min=f+F,$ (19)

where $Min$ is the left end of the non-positive random number interval when
the quenched noise is generated. We set $Min=-3.6$ and $F=0.6$. Obviously,
according to Eq. (19), the critical threshold of the process could be reached
when $f=3.0000$, which will be verified numerically below.

As shown in Fig. 4, the scaling law of velocity versus time is similar to
Model I. Near $f_{c}\approx 3.0000$, $v$ always decays in power-law $v\sim
t^{-\delta}$, and one could obtain a nice straight line in the insets of Fig.
4 with $\delta=0.330\pm 0.005$ for the (1+1)-dimensions and $\delta=0.460\pm
0.005$ for the (2+1)-dimensions. It is easy to find that numerical results
agrees well with Eq. (19). Even if the driving force deviates from the
critical force to a small degree, the curve changes dramatically.

Furthermore, we investigate the sensitivity of growth interface to the changes
in the elastic force $F$ for (2+1)-dimensional Model II. We fix the driving
force $f=3.0000$ and adjust the values of $F$ to observe the pinning-depinning
transition, and the critical threshold $F_{c}\approx 0.6000$ is obtained,
which further confirms Eq. (19). Figure 5 shows the changes in trends of
different lines by adjusting $F$, which is similar to those by adjusting $f$
in Fig. 4. By comparing the two figures, it is easy to find that the pinning-
depinning transition is not only sensitive to changes in the driving force,
but also to changes in the elastic force. And the estimated value
$\delta^{\prime}=0.460\pm 0.005$ is the same as that of $\delta$ within the
error range.

Near $f_{c}$, the interface width follows power-law as Eq. (14) with
$\beta=0.670\pm 0.005$ in the (1+1)-dimensions, and $\beta=0.527\pm 0.003$ in
the (2+1)-dimensions, as shown in the insets of Fig. 6. So our numerical
results satisfy the relation $\beta+\delta=1$ very well. It has been noted
that the evolving process is sensitive to adjusting the magnitude of the
driving force , which helps us to calculate the velocity exponent more
accurately. In the depinned region, we plot the saturated velocity $v$ as a
function of $f-f_{c}$, and obtain the critical exponents $\theta=0.32\pm 0.01$
for the (1+1)-dimensions and $\theta=0.41\pm 0.02$ for the (2+1)-dimensions,
as shown in Fig. 6. Furthermore, based on power-law scaling (17), we obtain
another scaling exponents $\nu=0.970\pm 0.005$ for the (1+1)-dimensions and
$\nu=0.90\pm 0.01$ for the (2+1)-dimensions, as shown in Fig. 7. The scaling
relation $\nu=\theta/\delta$ is also verified numerically.

Table 1: Summary of scaling exponents at the critical pinning-depinning transitions in the (1+1)- and (2+1)-dimensions. Models | Dimensions | $\beta$ | $\delta$ | $\theta$ | $\nu$
---|---|---|---|---|---
Model I | 1+1 [30] | $0.35\pm 0.05$ | $0.60\pm 0.01$ | $0.83\pm 0.01$ | $1.35\pm 0.05$
1+1 | $0.28\pm 0.01$ | $0.730\pm 0.005$ | $1.27\pm 0.03$ | $1.50\pm 0.05$
2+1 | $0.120\pm 0.003$ | $0.900\pm 0.005$ | $1.34\pm 0.05$ | $1.48\pm 0.03$
Model II | 1+1 [30] | $0.65\pm 0.05$ | $0.34\pm 0.01$ | $1.00\pm 0.01$ | $2.95\pm 0.05$
1+1 | $0.670\pm 0.005$ | $0.330\pm 0.005$ | $0.32\pm 0.01$ | $0.970\pm 0.005$
2+1 | $0.527\pm 0.003$ | $0.460\pm 0.005$ | $0.41\pm 0.02$ | $0.90\pm 0.01$
QEW | 1+1 [43, 42] | $0.85\pm 0.03$ | $0.128$ | $0.24\pm 0.03$ | $1.96(6)$
2+1 [33] | $0.493(3)$ | $0.504(2)$ | $0.631(2)$ | $1.252(12)$
2+1 | $0.412\pm 0.003$ | $0.490\pm 0.002$ | $0.610\pm 0.003$ | $1.35\pm 0.05$
QKPZ | 1+1 [43, 25] | $0.67\pm 0.05$ | $0.360(1)$ | $0.64\pm 0.12$ | $1.711$
2+1 ($F=0$) | $0.425\pm 0.005$ | $0.520\pm 0.005$ | $0.84\pm 0.03$ | $1.40\pm 0.05$
2+1 ($F\neq 0$) | $0.320\pm 0.005$ | $0.775\pm 0.005$ | $0.90\pm 0.01$ | $0.95\pm 0.05$

###  III.3 (2+1)-dimensional QEW and QKPZ equations

In this subsection, we investigate numerically QEW and QKPZ equations as two
well-known universality classes in the physically relevant case of
(2+1)-dimensions. The time evolution of interface is updated following Eq.
(10). We find that the nonlinear parameter $\lambda$ is negatively correlated
with the external parameter $F$. Actually, for any chosen $F$, the
corresponding critical nonlinear parameter $\lambda_{c}$ could always be found
to reach pinning-depinning transition. Firstly, we set $\lambda=5$ ensuring
this system far away from the EW universality class. Figure 8 shows that the
power-law scaling of average velocity $v$ versus time $t$ is similar to that
of the quenched models mentioned above. In this case, we obtain that the
critical force $F_{c}\approx-1.10$. Near $F_{c}$, $v$ always decays in power-
law $v\sim t^{-\delta}$ with $\delta=0.775\pm 0.005$, as shown in the inset of
Fig. 8. Meanwhile, we measured $W(L,t)$ as a function of $t$ and obtain
$\beta=0.320\pm 0.005$. Then, other scaling exponents near the critical point
mentioned above are also calculated, as shown in Figs. 9-10. The scaling
relations $\beta+\delta=1$ and $\nu=\theta/\delta$ are verified numerically.

Furthermore, we demonstrate explicitly the equivalence of the fixed nonlinear
parameter with a small value and the fixed driving force simulations on the
(2+1)-dimensional QKPZ model. Two special cases are chosen: (i) QKPZ with a
smaller $\lambda=2$ and $F_{c}\approx 0.6$. (ii) QKPZ with $F=0$ and
$\lambda_{c}\approx 3.0$. Interestingly, we obtain the same scaling exponents
within a certain error range from these two cases:
$\delta=\delta^{\prime}=0.520\pm 0.005,\beta=0.425\pm
0.005,\theta=\theta^{\prime}=0.84\pm 0.03$ and $\nu=\nu^{\prime}=1.4\pm 0.05$,
implying that adjusting $F$ and $\lambda$ has equivalent effect on pinning-
depinning transition, which has also been mentioned in [39] for
(1+1)-dimensional cases. Notably, it is obvious that our results in these two
special cases are deviated from those of QKPZ universality class in the
(2+1)-dimensions. As a matter of fact, QKPZ in the absence of an external
driving force or with small nonlinear coefficient, all could reduce gradually
to EW universality class.

When the nonlinear coefficient $\lambda$ decreases to $0$, the QKPZ equation
recover gradually to QEW [32, 2]. With system size $L\times L=256\times 256$,
we obtain that $F_{c}\approx 2.2,\delta=0.490\pm 0.002,\beta=0.412\pm
0.003,\theta=0.610\pm 0.003$ and $\nu=1.35\pm 0.05$. It is clear that these
exponents are similar to those of two special case of QKPZ (i) and (ii)
mentioned above. Furthermore, comparing our results to one of the discrete
versions of QEW model [33], we find that these results are consistent with
each other.

In order to visually give comparisons of these (2+1)-dimensional quenched
systems, we exhibit surface morphologies with various external driving forces
during the saturated time regimes, as shown in Figs. 11. When $F<F_{c}$, we
show typical snapshots of the saturated surfaces at pinning in the left column
of Figs. 11, including four subgraphs: (a), (d), (g), and (j). In these cases,
the velocities are almost zero since the surfaces are pinned, and we confirm
these features by measuring average height field at different time instants.
Typical snapshot configurations are obtained when $F=F_{c}$, as shown in the
middle column of Figs. 11, including subgraphs: (b), (e), (h), and (k). It is
easy to observe that these surfaces happen from pinning to depinning phases
near the critical thresholds for different quenched models. And surface
morphologies change evidently with time evolution in comparison with those at
pinning phases. And when $F>F_{c}$, the evolving surfaces move forward very
obviously, and the average velocities remain almost unchanged for a certain
quenched model. However, the velocity magnitudes are significant differences
among these quenched models, which depend mainly on magnitudes of various
parameters, external forces, and quenched noises.

##  IV discussions and conclusions

In this paper, we revisit two discrete elastic-string models, and then
generalize to (2+1)-dimensional cases to investigate the pinning-depinning
transition. As the existing universality classes, the (2+1)-dimensional QKPZ
and QEW equations are also studied numerically. Two discrete quenched models
show the transition very clearly, especially, Model II is very sensitive to
external parameters even in smaller system size. It should be emphasized that
the results of Model I in the (1+1)-dimensional case are a little different
with those of [30]. As mentioned before, we introduce term
$\eta\left(\vec{r},h(\vec{r},t)\right)$ to Model II to replace term
$\eta(\vec{r})$ in [30]. We consider that Model II driven by
$\eta\left(h(\vec{r},t)\right)$ is closer to the models describing crack front
propagation [44, 45]. The comparisons among our estimated values and previous
results are shown in Table 1.

We perform similar research steps in these quenched systems. Firstly, by a
power-law decay of the interface velocity, we could find the critical
threshold. And then the scaling exponents $\delta,\beta,\theta$, and $\nu$
near the critical point are estimated based on various power-law scaling
ansatzs. Our results show that the estimated values of these exponents satisfy
$\beta+\delta=1$ and $\nu=\theta/\delta$ very well. All exponents calculated
are listed in Table 1 and the previous results are also compared. It should be
noted that these discrete quenched models do not fall into the existing
universality classes.

For the QKPZ system, we generalize (1+1)-dimensional case [39] to
(2+1)-dimensions.Our simulations show that one could enter the pinning-
depinning transition of the true KPZ region by adjusting $F$ and $\lambda$,
and then obtain the scaling exponents. Furthermore, we also focus on two
special cases including small $\lambda$ region with $F_{c}$ and $\lambda_{c}$
region with $F=0$. Notably, these two special cases belong to QEW universality
class instead of QKPZ. Through numerical simulations, we obtain similar
scaling exponents by adjusting $F$ and $\lambda$ near their respective
critical point based on Eqs. (15-18). It seems to indicate that the $\lambda$
terms in Eq. (10) is equivalent to a constant that acts as an external driving
force $F$ to some extent. Furthermore, we also calculate the critical
exponents in the (2+1)-dimensional QEW, and the critical values obtained here
are in line with those of the discrete model belonging to QEW universality
class in [33], the detailed comparisons are shown in Table 1.

The potential applications of these quenched models studied in current work
deserve further investigations. The relationship between two elastic string
models and fracture front propagation [44, 46, 47, 48, 49, 31, 34, 35] is a
meaningful issue. Elastic description is also related to self-organized
critical phase transition in some special cases [50]. And elastic energy [51]
in elastic-string model for fracture is worth studying. The different
properties of elastic force of two models may correspond to different
materials such as ductile and brittle materials [44, 46, 34, 35]. Besides, as
mentioned in SEC. I, the wetting fronts [52, 2, 53] in disordered materials
seem to be relevant for (2+1)-dimensional QKPZ with $F=0$, such as wetting
cubic sponge without external pressure, the interface could propagate in
quenched random media driven by local capillary forces, which implies that
pinning-depinning transition occurs even in the absence of external driving
force.

## References

  * Tang and Leschhorn [1992] L.-H. Tang and H. Leschhorn, Phys. Rev. A 45, R8309 (1992).
  * Barabási  _et al._ [1995] A.-L. Barabási, H. E. Stanley, _et al._ ,  _Fractal Concepts in Surface Growth_ (Cambridge university press, 1995).
  * Lee  _et al._ [2000] J. H. Lee, S. K. Kim, and J. M. Kim, Phys. Rev. E 62, 3299 (2000).
  * Reichhardt  _et al._ [2022] C. Reichhardt, A. del Campo, and C. Reichhardt, arXiv preprint arXiv:2202.12388 (2022).
  * Zapperi  _et al._ [1998] S. Zapperi, P. Cizeau, G. Durin, and H. E. Stanley, Phys. Rev. B 58, 6353 (1998).
  * Derrida and Retaux [2014] B. Derrida and M. Retaux, J. Stat. Phys. 156, 268 (2014).
  * Şenbil  _et al._ [2019] N. Şenbil, M. Gruber, C. Zhang, M. Fuchs, and F. Scheffold, Phys. Rev. Lett. 122, 108002 (2019).
  * Park and Pleimling [2012] H. Park and M. Pleimling, Eur. Phys. J. B 85, 1 (2012).
  * Brown  _et al._ [2019] B. L. Brown, U. C. Täuber, and M. Pleimling, Phys. Rev. B 100, 024410 (2019).
  * Rubio  _et al._ [1989] M. A. Rubio, C. A. Edwards, A. Dougherty, and J. P. Gollub, Phys. Rev. Lett. 63, 1685 (1989).
  * Huse and Henley [1985] D. A. Huse and C. L. Henley, Phys. Rev. Lett. 54, 2708 (1985).
  * Campi  _et al._ [2015] G. Campi, A. Bianconi, N. Poccia, G. Bianconi, L. Barba, G. Arrighetti, D. Innocenti, J. Karpinski, N. D. Zhigadlo, S. M. Kazakov, _et al._ , Nature 525, 359 (2015).
  * Fisher  _et al._ [1991] D. S. Fisher, M. P. A. Fisher, and D. A. Huse, Phys. Rev. B 43, 130 (1991).
  * Chaturvedi  _et al._ [2020] H. Chaturvedi, U. Dobramysl, M. Pleimling, and U. C. Täuber, Phys. Rev. B 101, 024515 (2020).
  * Täuber  _et al._ [1995] U. C. Täuber, H. Dai, D. R. Nelson, and C. M. Lieber, Phys. Rev. Lett. 74, 5132 (1995).
  * Täuber and Nelson [1995] U. C. Täuber and D. R. Nelson, Phys. Rev. B 52, 16106 (1995).
  * Dobramysl  _et al._ [2014] U. Dobramysl, M. Pleimling, and U. C. Täuber, Phys. Rev. E 90, 062108 (2014).
  * Corwin [2012] I. Corwin, Random Matrices-Theo. 01, 1130001 (2012).
  * Sasamoto and Spohn [2010] T. Sasamoto and H. Spohn, Phys. Rev. Lett. 104, 230602 (2010).
  * Priyanka  _et al._ [2021] Priyanka, U. C. Täuber, and M. Pleimling, J. Phys. A: Math. Theor. 54, 154002 (2021).
  * Dotsenko [1995] V. S. Dotsenko, Phys.-Usp. 38, 457 (1995).
  * Mardanya  _et al._ [2018] S. Mardanya, U. Bhattacharya, A. Agarwal, and A. Dutta, Phys. Rev. B 97, 115443 (2018).
  * Pan and Zhou [2020] J. Pan and L. Zhou, Phys. Rev. B 102, 094305 (2020).
  * Deutschländer  _et al._ [2013] S. Deutschländer, T. Horn, H. Löwen, G. Maret, and P. Keim, Phys. Rev. Lett. 111, 098301 (2013).
  * Lee and Kim [2005] C. Lee and J. M. Kim, J. Korean Phys. Soc. 47, 13 (2005).
  * Song and Kim [2007] H. S. Song and J. M. Kim, J. Korean Phys. Soc. 51, 1630 (2007).
  * Jeong  _et al._ [1999] H. Jeong, B. Kahng, and D. Kim, Phys. Rev. E 59, 1570 (1999).
  * Huergo  _et al._ [2014] M. A. C. Huergo, N. E. Muzzio, M. A. Pasquale, P. H. P. González, A. E. Bolzán, and A. J. Arvia, Phys. Rev. E 90, 022706 (2014).
  * Duemmer and Krauth [2007] O. Duemmer and W. Krauth, J. Stat. Mech.: Theory Exp. 2007 (01), P01019.
  * Biswas and Chakrabarti [2011] S. Biswas and B. K. Chakrabarti, arXiv preprint arXiv:1108.1707 (2011).
  * Tanguy  _et al._ [1998] A. Tanguy, M. Gounelle, and S. Roux, Phys. Rev. E 58, 1577 (1998).
  * Edwards and Wilkinson [1982] S. F. Edwards and D. R. Wilkinson, Proc. R. Soc. London, Ser. A 381, 17 (1982).
  * Song and Kim [2008] H. S. Song and J. M. Kim, J. Korean Phys. Soc. 53, 1802 (2008).
  * Ponson and Bonamy [2010] L. Ponson and D. Bonamy, Int. J. Fract. 162, 21 (2010).
  * Alava  _et al._ [2006] M. J. Alava, P. K. Nukala, and S. Zapperi, Adv. Phys. 55, 349 (2006).
  * Amar and Family [1990] J. G. Amar and F. Family, Phys. Rev. A 41, 3399 (1990).
  * Csahók  _et al._ [1993] Z. Csahók, K. Honda, E. Somfai, M. Vicsek, and T. Vicsek, Phys. A 200, 136 (1993).
  * Lee and Kim [2006] C. Lee and J. M. Kim, Phys. Rev. E 73, 016140 (2006).
  * Ramasco  _et al._ [2001] J. J. Ramasco, J. M. López, and M. A. Rodríguez, Phys. Rev. E 64, 066109 (2001).
  * Kolton  _et al._ [2006] A. B. Kolton, A. Rosso, E. V. Albano, and T. Giamarchi, Phys. Rev. B 74, 140201 (2006).
  * Dickman and Muñoz [2000] R. Dickman and M. A. Muñoz, Phys. Rev. E 62, 7632 (2000).
  * Kim and Choi [2006] J. M. Kim and H. Choi, J. Korean Phys. Soc. 48, 241 (2006).
  * Amaral  _et al._ [1995] L. A. N. Amaral, A.-L. Barabási, H. A. Makse, and H. E. Stanley, Phys. Rev. E 52, 4087 (1995).
  * Démery  _et al._ [2014] V. Démery, A. Rosso, and L. Ponson, Europhys. Lett. 105, 34003 (2014).
  * Basu and Chakrabarti [2019] A. Basu and B. K. Chakrabarti, Philos. Trans. R. Soc., A 377, 20170387 (2019).
  * Bonamy and Bouchaud [2011] D. Bonamy and E. Bouchaud, Phys. Rep. 498, 1 (2011).
  * Ponson [2009] L. Ponson, Phys. Rev. Lett. 103, 055501 (2009).
  * Le Priol  _et al._ [2020] C. Le Priol, J. Chopin, P. Le Doussal, L. Ponson, and A. Rosso, Phys. Rev. Lett. 124, 065501 (2020).
  * Måløy  _et al._ [2006] K. J. Måløy, S. Santucci, J. Schmittbuhl, and R. Toussaint, Phys. Rev. Lett. 96, 045501 (2006).
  * Bonamy  _et al._ [2008] D. Bonamy, S. Santucci, and L. Ponson, Phys. Rev. Lett. 101, 045501 (2008).
  * Ponson [2016] L. Ponson, Int. J. Fract. 201, 11 (2016).
  * Le Doussal  _et al._ [2009] P. Le Doussal, K. J. Wiese, S. Moulinet, and E. Rolley, Europhys. Lett. 87, 56001 (2009).
  * Moulinet  _et al._ [2004] S. Moulinet, A. Rosso, W. Krauth, and E. Rolley, Phys. Rev. E 69, 035103 (2004).


0.1pt 10

# StarGAN v2: Diverse Image Synthesis for Multiple Domains

Yunjey Choi111footnotemark: 1 Youngjung Uh111footnotemark: 1 Jaejun
Yoo211footnotemark: 1 Jung-Woo Ha1

1Clova AI Research, NAVER Corp. 2EPFL

###### Abstract

A good image-to-image translation model should learn a mapping between
different visual domains while satisfying the following properties: 1)
diversity of generated images and 2) scalability over multiple domains.
Existing methods address either of the issues, having limited diversity or
multiple models for all domains. We propose StarGAN v2, a single framework
that tackles both and shows significantly improved results over the baselines.
Experiments on CelebA-HQ and a new animal faces dataset (AFHQ) validate our
superiority in terms of visual quality, diversity, and scalability. To better
assess image-to-image translation models, we release AFHQ, high-quality animal
faces with large inter- and intra-domain differences. The code, pretrained
models, and dataset are available at clovaai/stargan-v2.

Figure 1: Diverse image synthesis results on the CelebA-HQ dataset and the
newly collected animal faces (AFHQ) dataset. The first column shows input
images while the remaining columns are images synthesized by StarGAN v2.

##  1 Introduction

††* indicates equal contribution

Image-to-image translation aims to learn a mapping between different visual
domains [20]. Here, domain implies a set of images that can be grouped as a
visually distinctive category, and each image has a unique appearance, which
we call style. For example, we can set image domains based on the gender of a
person, in which case the style includes makeup, beard, and hairstyle (top
half of Figure StarGAN v2: Diverse Image Synthesis for Multiple Domains). An
ideal image-to-image translation method should be able to synthesize images
considering the diverse styles in each domain. However, designing and learning
such models become complicated as there can be arbitrarily large number of
styles and domains in the dataset.

To address the style diversity, much work on image-to-image translation has
been developed [1, 16, 34, 28, 38, 54]. These methods inject a low-dimensional
latent code to the generator, which can be randomly sampled from the standard
Gaussian distribution. Their domain-specific decoders interpret the latent
codes as recipes for various styles when generating images. However, because
these methods have only considered a mapping between two domains, they are not
scalable to the increasing number of domains. For example, having K domains,
these methods require to train K(K-1) generators to handle translations
between each and every domain, limiting their practical usage.

To address the scalability, several studies have proposed a unified framework
[2, 7, 17, 30]. StarGAN [7] is one of the earliest models, which learns the
mappings between all available domains using a single generator. The generator
takes a domain label as an additional input, and learns to transform an image
into the corresponding domain. However, StarGAN still learns a deterministic
mapping per each domain, which does not capture the multi-modal nature of the
data distribution. This limitation comes from the fact that each domain is
indicated by a predetermined label. Note that the generator receives a fixed
label (_e.g_. one-hot vector) as input, and thus it inevitably produces the
same output per each domain, given a source image.

To get the best of both worlds, we propose StarGAN v2, a scalable approach
that can generate diverse images across multiple domains. In particular, we
start from StarGAN and replace its domain label with our proposed domain-
specific style code that can represent diverse styles of a specific domain. To
this end, we introduce two modules, a mapping network and a style encoder. The
mapping network learns to transform random Gaussian noise into a style code,
while the encoder learns to extract the style code from a given reference
image. Considering multiple domains, both modules have multiple output
branches, each of which provides style codes for a specific domain. Finally,
utilizing these style codes, our generator learns to successfully synthesize
diverse images over multiple domains (Figure StarGAN v2: Diverse Image
Synthesis for Multiple Domains).

We first investigate the effect of individual components of StarGAN v2 and
show that our model indeed benefits from using the style code (Section 3.1).
We empirically demonstrate that our proposed method is scalable to multiple
domains and gives significantly better results in terms of visual quality and
diversity compared to the leading methods (Section 3.2). Last but not least,
we present a new dataset of animal faces (AFHQ) with high quality and wide
variations (Appendix A) to better evaluate the performance of image-to-image
translation models on large inter- and intra-domain differences. We release
this dataset publicly available for research community.

(a) Generator (b) Mapping network (c) Style encoder (d) Discriminator

Figure 2: Overview of StarGAN v2, consisting of four modules. (a) The
generator translates an input image into an output image reflecting the
domain-specific style code. (b) The mapping network transforms a latent code
into style codes for multiple domains, one of which is randomly selected
during training. (c) The style encoder extracts the style code of an image,
allowing the generator to perform reference-guided image synthesis. (d) The
discriminator distinguishes between real and fake images from multiple
domains. Note that all modules except the generator contain multiple output
branches, one of which is selected when training the corresponding domain.

##  2 StarGAN v2

In this section, we describe our proposed framework and its training objective
functions.

###  2.1 Proposed framework

Let ${\cal X}$ and ${\cal Y}$ be the sets of images and possible domains,
respectively. Given an image $\mathbf{x}\in{\cal X}$ and an arbitrary domain
$y\in{\cal Y}$, our goal is to train a single generator $G$ that can generate
diverse images of each domain $y$ that corresponds to the image $\mathbf{x}$.
We generate domain-specific style vectors in the learned style space of each
domain and train $G$ to reflect the style vectors. Figure 2 illustrates an
overview of our framework, which consists of four modules described below.

Generator (Figure 2a). Our generator $G$ translates an input image
$\mathbf{x}$ into an output image $G(\mathbf{x},\mathbf{s})$ reflecting a
domain-specific style code $\mathbf{s}$, which is provided either by the
mapping network $F$ or by the style encoder $E$. We use adaptive instance
normalization (AdaIN) [15, 22] to inject $\mathbf{s}$ into $G$. We observe
that $\mathbf{s}$ is designed to represent a style of a specific domain $y$,
which removes the necessity of providing $y$ to $G$ and allows $G$ to
synthesize images of all domains.

Mapping network (Figure 2b). Given a latent code $\mathbf{z}$ and a domain
$y$, our mapping network $F$ generates a style code
$\mathbf{s}={F}_{y}(\mathbf{z})$, where ${F}_{y}(\cdot)$ denotes an output of
$F$ corresponding to the domain $y$. $F$ consists of an MLP with multiple
output branches to provide style codes for all available domains. $F$ can
produce diverse style codes by sampling the latent vector
$\mathbf{z}\in\cal{Z}$ and the domain $y\in\cal{Y}$ randomly. Our multi-task
architecture allows $F$ to efficiently and effectively learn style
representations of all domains.

Style encoder (Figure 2c). Given an image $\mathbf{x}$ and its corresponding
domain $y$, our encoder $E$ extracts the style code
$\mathbf{s}={E}_{y}(\mathbf{x})$ of $\mathbf{x}$. Here, ${E}_{y}(\cdot)$
denotes the output of $E$ corresponding to the domain $y$. Similar to $F$, our
style encoder $E$ benefits from the multi-task learning setup. $E$ can produce
diverse style codes using different reference images. This allows $G$ to
synthesize an output image reflecting the style $\mathbf{s}$ of a reference
image $\mathbf{x}$.

Discriminator (Figure 2d). Our discriminator $D$ is a multi-task discriminator
[30, 35], which consists of multiple output branches. Each branch ${D}_{y}$
learns a binary classification determining whether an image $\mathbf{x}$ is a
real image of its domain $y$ or a fake image $G(\mathbf{x},\mathbf{s})$
produced by $G$.

###  2.2 Training objectives

Given an image $\mathbf{x}\in\cal{X}$ and its original domain $y\in\cal{Y}$,
we train our framework using the following objectives.

Adversarial objective. During training, we sample a latent code
$\mathbf{z}\in\cal{Z}$ and a target domain $\widetilde{y}\in\cal{Y}$ randomly,
and generate a target style code
$\widetilde{\mathbf{s}}={F}_{\widetilde{y}}(\mathbf{z})$. The generator $G$
takes an image $\mathbf{x}$ and $\widetilde{\mathbf{s}}$ as inputs and learns
to generate an output image $G(\mathbf{x},\widetilde{\mathbf{s}})$ via an
adversarial loss

$\begin{split}\mathcal{L}_{adv}=&\thinspace\mathbb{E}_{\mathbf{x},y}\left[\log{{D}_{y}(\mathbf{x})}\right]\>\>+\\\
&\thinspace\mathbb{E}_{\mathbf{x},\widetilde{y},\mathbf{z}}[\log{(1-{D}_{\widetilde{y}}(G(\mathbf{x},\widetilde{\mathbf{s}})))}],\end{split}$
(1)

where ${D}_{y}(\cdot)$ denotes the output of $D$ corresponding to the domain
$y$. The mapping network $F$ learns to provide the style code
$\widetilde{\mathbf{s}}$ that is likely in the target domain $\widetilde{y}$,
and $G$ learns to utilize $\widetilde{\mathbf{s}}$ and generate an image
$G(\mathbf{x},\widetilde{\mathbf{s}})$ that is indistinguishable from real
images of the domain $\widetilde{y}$.

Style reconstruction. In order to enforce the generator $G$ to utilize the
style code $\widetilde{\mathbf{s}}$ when generating the image
$G(\mathbf{x},\widetilde{\mathbf{s}})$, we employ a style reconstruction loss

$\mathcal{L}_{sty}=\mathbb{E}_{\mathbf{x},\widetilde{y},\mathbf{z}}\left[{||\widetilde{\mathbf{s}}-{E}_{\widetilde{y}}(G(\mathbf{x},\widetilde{\mathbf{s}}))||}_{1}\right].$
(2)

This objective is similar to the previous approaches [16, 54], which employ
multiple encoders to learn a mapping from an image to its latent code. The
notable difference is that we train a single encoder $E$ to encourage diverse
outputs for multiple domains. At test time, our learned encoder $E$ allows $G$
to transform an input image, reflecting the style of a reference image.

Style diversification. To further enable the generator $G$ to produce diverse
images, we explicitly regularize $G$ with the diversity sensitive loss [34,
48]

$\mathcal{L}_{ds}=\mathbb{E}_{\mathbf{x},\widetilde{y},\mathbf{z}_{1},\mathbf{z}_{2}}\left[{\lVert
G(\mathbf{x},\widetilde{\mathbf{s}}_{1})-G(\mathbf{x},\widetilde{\mathbf{s}}_{2})\lVert}_{1}\right],$
(3)

where the target style codes $\widetilde{\mathbf{s}}_{1}$ and
$\widetilde{\mathbf{s}}_{2}$ are produced by $F$ conditioned on two random
latent codes $\mathbf{z}_{1}$ and $\mathbf{z}_{2}$ (_i.e_.
$\widetilde{\mathbf{s}}_{i}={F}_{\widetilde{y}}(\mathbf{z}_{i})$ for
$i\in\\{1,2\\}$). Maximizing the regularization term forces $G$ to explore the
image space and discover meaningful style features to generate diverse images.
Note that in the original form, the small difference of
${\lVert\mathbf{z}_{1}-\mathbf{z}_{2}\lVert}_{1}$ in the denominator increases
the loss significantly, which makes the training unstable due to large
gradients. Thus, we remove the denominator part and devise a new equation for
stable training but with the same intuition.

Preserving source characteristics. To guarantee that the generated image
$G(\mathbf{x},\widetilde{\mathbf{s}})$ properly preserves the domain-invariant
characteristics (_e.g_. pose) of its input image $\mathbf{x}$, we employ the
cycle consistency loss [7, 24, 53]

$\mathcal{L}_{cyc}=\mathbb{E}_{\mathbf{x},y,\widetilde{y},\mathbf{z}}\left[{||\mathbf{x}-G(G(\mathbf{x},\widetilde{\mathbf{s}}),\hat{\mathbf{s}})||}_{1}\right],$
(4)

where $\hat{\mathbf{s}}={E}_{y}(\mathbf{x})$ is the estimated style code of
the input image $\mathbf{x}$, and $y$ is the original domain of $\mathbf{x}$.
By encouraging the generator $G$ to reconstruct the input image $\mathbf{x}$
with the estimated style code $\hat{\mathbf{s}}$, $G$ learns to preserve the
original characteristics of $\mathbf{x}$ while changing its style faithfully.

Full objective. Our full objective functions can be summarized as follows:

$\begin{split}\min_{G,F,E}\max_{D}\quad&\mathcal{L}_{adv}+{\lambda}_{sty}\thinspace\mathcal{L}_{sty}\\\
&-{\lambda}_{ds}\thinspace\mathcal{L}_{ds}+{\lambda}_{cyc}\thinspace\mathcal{L}_{cyc},\end{split}$
(5)

where ${\lambda}_{sty}$, ${\lambda}_{ds}$, and ${\lambda}_{cyc}$ are
hyperparameters for each term. We also further train our model in the same
manner as the above objective, using reference images instead of latent
vectors when generating style codes. We provide the training details in
Appendix B.

##  3 Experiments

In this section, we describe evaluation setups and conduct a set of
experiments. We analyze the individual components of StarGAN v2 (Section 3.1)
and compare our model with three leading baselines on diverse image synthesis
(Section 3.2). All experiments are conducted using unseen images during the
training phase.

Baselines. We use MUNIT [16], DRIT [28], and MSGAN [34] as our baselines, all
of which learn multi-modal mappings between two domains. For multi-domain
comparisons, we train these models multiple times for every pair of image
domains. We also compare our method with StarGAN [7], which learns mappings
among multiple domains using a single generator. All the baselines are trained
using the implementations provided by the authors.

Datasets. We evaluate StarGAN v2 on CelebA-HQ [21] and our new AFHQ dataset
(Appendix A). We separate CelebA-HQ into two domains of male and female, and
AFHQ into three domains of cat, dog, and wildlife. Other than the domain
labels, we do not use any additional information (_e.g_. facial attributes of
CelebA-HQ or breeds of AFHQ) and let the models learn such information as
styles without supervision. For a fair comparison, all images are resized to
$256\times 256$ resolution for training, which is the highest resolution used
in the baselines.

Evaluation metrics. We evaluate both the visual quality and the diversity of
generated images using Frechét inception distance (FID) [14] and learned
perceptual image patch similarity (LPIPS) [52]. We compute FID and LPIPS for
every pair of image domains within a dataset and report their average values.
The details on evaluation metrics and protocols are further described in
Appendix C.

| Method |  FID |  LPIPS
---|---|---|---
a |  Baseline StarGAN [7] |  98.4 |  -
b | \+ Multi-task discriminator |  91.4 |  -
c |  \+ Tuning (_e.g_. ${R}_{1}$ regularization) |  80.5 |  -
d | \+ Latent code injection |  32.3 |  0.312
e |  \+ Replace (d) with style code |  17.1 |  0.405
f | \+ Diversity regularization |  13.7 |  0.452
Table 1: Performance of various configurations on CelebA-HQ. Frechet inception
distance (FID) indicates the distance between two distributions of real and
generated images (lower is better), while learned perceptual image patch
similarity (LPIPS) measures the diversity of generated images (higher is
better).

Figure 3: Visual comparison of generated images using each configuration in
Table 1. Note that given a source image, the configurations (a) - (c) provide
a single output, while (d) - (f) generate multiple output images.

Figure 4: Reference-guided image synthesis results on CelebA-HQ. The source
and reference images in the first row and the first column are real images,
while the rest are images generated by our proposed model, StarGAN v2. Our
model learns to transform a source image reflecting the style of a given
reference image. High-level semantics such as hairstyle, makeup, beard and age
are followed from the reference images, while the pose and identity of the
source images are preserved. Note that the images in each column share a
single identity with different styles, and those in each row share a style
with different identities.

Source MUNIT [16] DRIT [28] MSGAN [34] Ours Source MUNIT [16] DRIT [28] MSGAN
[34] Ours
(a) Latent-guided synthesis on CelebA-HQ (b) Latent-guided synthesis on AFHQ

Figure 5: Qualitative comparison of latent-guided image synthesis results on
the CelebA-HQ and AFHQ datasets. Each method translates the source images
(left-most column) to target domains using randomly sampled latent codes. (a)
The top three rows correspond to the results of converting male to female and
vice versa in the bottom three rows. (b) Every two rows from the top show the
synthesized images in the following order: cat-to-dog, dog-to-wildlife, and
wildlife-to-cat.

###  3.1 Analysis of individual components

We evaluate individual components that are added to our baseline StarGAN using
CelebA-HQ. Table 1 gives FID and LPIPS for several configurations, where each
component is cumulatively added on top of StarGAN. An input image and the
corresponding generated images of each configuration are shown in Figure 3.
The baseline configuration (a) corresponds to the basic setup of StarGAN,
which employs WGAN-GP [11], ACGAN discriminator [39], and depth-wise
concatenation [36] for providing the target domain information to the
generator. As shown in Figure 3a, the original StarGAN produces only a local
change by applying makeup on the input image.

We first improve our baseline by replacing the ACGAN discriminator with a
multi-task discriminator [35, 30], allowing the generator to transform the
global structure of an input image as shown in configuration (b). Exploiting
the recent advances in GANs, we further enhance the training stability and
construct a new baseline (c) by applying $R_{1}$ regularization [35] and
switching the depth-wise concatenation to adaptive instance normalization
(AdaIN) [9, 15]. Note that we do not report LPIPS of these variations in Table
1, since they are yet to be designed to produce multiple outputs for a given
input image and a target domain.

To induce diversity, one can think of directly giving a latent code
$\mathbf{z}$ into the generator $G$ and impose the latent reconstruction loss
${||\mathbf{z}-{E}(G(\mathbf{x},\mathbf{z},y))||}_{1}$ [16, 54]. However, in a
multi-domain scenario, we observe that this baseline (d) does not encourage
the network to learn meaningful styles and fails to provide as much diversity
as we expect. We conjecture that this is because latent codes have no
capability in separating domains, and thus the latent reconstruction loss
models domain-shared styles (_e.g_. color) rather than domain-specific ones
(_e.g_. hairstyle). Note that the FID gap between baseline (c) and (d) is
simply due to the difference in the number of output samples.

Instead of giving a latent code into $G$ directly, to learn meaningful styles,
we transform a latent code $\mathbf{z}$ into a domain-specific style code
$\mathbf{s}$ through our proposed mapping network (Figure 2b) and inject the
style code into the generator (e). Here, we also introduce the style
reconstruction loss (Eq. (2)). Note that each output branch of our mapping
network is responsible to a particular domain, thus style codes have no
ambiguity in separating domains. Unlike the latent reconstruction loss, the
style reconstruction loss allows the generator to produce diverse images
reflecting domain-specific styles. Finally, we further improve the network to
produce diverse outputs by adopting the diversity regularization (Eq. (3)),
and this configuration (f) corresponds to our proposed method, StarGAN v2.
Figure 4 shows that StarGAN v2 can synthesize images that reflect diverse
styles of references including hairstyle, makeup, and beard, without hurting
the source characteristics.

Source Reference DRIT [28] MSGAN [34] Ours Source Reference DRIT [28] MSGAN
[34] Ours
(a) Reference-guided synthesis on CelebA-HQ (b) Reference-guided synthesis on
AFHQ

Figure 6: Qualitative comparison of reference-guided image synthesis results
on the CelebA-HQ and AFHQ datasets. Each method translates the source images
into target domains, reflecting the styles of the reference images.

###  3.2 Comparison on diverse image synthesis

In this section, we evaluate StarGAN v2 on diverse image synthesis from two
perspectives: latent-guided synthesis and reference-guided synthesis.

Latent-guided synthesis. Figure 5 provides a qualitative comparison of the
competing methods. Each method produces multiple outputs using random noise.
For CelebA-HQ, we observe that our method synthesizes images with a higher
visual quality compared to the baseline models. In addition, our method is the
only model that can successfully change the entire hair styles of the source
images, which requires non-trivial effort (_e.g_. generating ears). For AFHQ,
which has relatively large variations, the performance of the baselines is
considerably degraded, while our method still produces images with high
quality and diverse styles.

|  | CelebA-HQ | AFHQ
---|---|---|---
Method | FID | LPIPS | FID | LPIPS
|  MUNIT [16] |  31.4 |  0.363 |  41.5 |  0.511
|  DRIT [28] |  52.1 |  0.178 |  95.6 |  0.326
|  MSGAN [34] |  33.1 |  0.389 |  61.4 |  0.517
| StarGAN v2 |  13.7 |  0.452 |  16.2 |  0.450
| Real images |  14.8 |  - |  12.9 |  -
Table 2: Quantitative comparison on latent-guided synthesis. The FIDs of real
images are computed between the training and test sets. Note that they may not
be optimal values since the number of test images is insufficient, but we
report them for reference.

As shown in Table 2, our method outperforms all the baselines by a large
margin in terms of visual quality. For both CelebA-HQ and AFHQ, our method
achieves FIDs of 13.7 and 16.2, respectively, which are more than two times
improvement over the previous leading method. Our LPIPS is also the highest in
CelebA-HQ, which implies our model produces the most diverse results given a
single input. We conjecture that the high LPIPS values of the baseline models
in AFHQ are due to their spurious artifacts.

Reference-guided synthesis. To obtain the style code from a reference image,
we sample test images from a target domain and feed them to the encoder
network of each method. For CelebA-HQ (Figure 6a), our method successfully
renders distinctive styles (_e.g_. bangs, beard, makeup, and hair-style),
while the others mostly match the color distribution of reference images. For
the more challenging AFHQ (Figure 6b), the baseline models suffer from a large
domain shift. They hardly reflect the style of each reference image and only
match the domain. In contrast, our model renders distinctive styles (_e.g_.
breeds) of each reference image as well as its fur pattern and eye color. Note
that StarGAN v2 produces high quality images across all domains and these
results are from a single generator. Since the other baselines are trained
individually for each pair of domains, the output quality fluctuates across
domains. For example, in AFHQ (Figure 6b), the baseline models work reasonably
well in dog-to-wildlife (2nd row) while they fail in cat-to-dog (1st row).

|  | CelebA-HQ | AFHQ
---|---|---|---
Method | FID | LPIPS | FID | LPIPS
|  MUNIT [16] |  107.1 |  0.176 |  223.9 |  0.199
|  DRIT [28] |  53.3 |  0.311 |  114.8 |  0.156
|  MSGAN [34] |  39.6 |  0.312 |  69.8 |  0.375
| StarGAN v2 |  23.8 |  0.388 |  19.8 |  0.432
| Real images |  14.8 |  - |  12.9 |  -
Table 3: Quantitative comparison on reference-guided synthesis. We sample ten
reference images to synthesize diverse images.

Table 3 shows FID and LPIPS of each method for reference guided synthesis. For
both datasets, our method achieves FID of 23.8, and 19.8, which are about
1.5$\times$ and 3.5$\times$ better than the previous leading method,
respectively. The LPIPS of StarGAN v2 is also the highest among the
competitors, which implies that our model produces the most diverse results
considering the styles of reference images. Here, MUNIT and DRIT suffer from
mode-collapse in AFHQ, which results in lower LPIPS and higher FID than other
methods.

Human evaluation. We use the Amazon Mechanical Turk (AMT) to compare the user
preferences of our method with baseline approaches. Given a pair of source and
reference images, the AMT workers are instructed to select one among four
image candidates from the methods, whose order is randomly shuffled. We ask
separately which model offers the best image quality and which model best
stylizes the input image considering the reference image. For each comparison,
we randomly generate 100 questions, and each question is answered by 10
workers. We also ask each worker a few simple questions to detect unworthy
workers. The number of total valid workers is 76. As shown in Table 4, our
method obtains the majority of votes in all instances, especially in the
challenging AFHQ dataset and the question about style reflection. These
results show that StarGAN v2 better extracts and renders the styles onto the
input image than the other baselines.

|  | CelebA-HQ | AFHQ
---|---|---|---
Method | Quality | Style | Quality | Style
|  MUNIT [16] |  6.2 |  7.4 |  1.6 |  0.2
|  DRIT [28] |  11.4 |  7.6 |  4.1 |  2.8
|  MSGAN [34] |  13.5 |  10.1 |  6.2 |  4.9
| StarGAN v2 |  68.9 |  74.8 |  88.1 |  92.1
Table 4: Votes from AMT workers for the most preferred method regarding visual
quality and style reflection (%). StarGAN v2 outperforms the baselines with
remarkable margins in all aspects.

##  4 Discussion

We discuss several reasons why StarGAN v2 can successfully synthesize images
of diverse styles over multiple domains. First, our style code is separately
generated per domain by the multi-head mapping network and style encoder. By
doing so, our generator can only focus on using the style code, whose domain-
specific information is already taken care of by the mapping network (Section
3.1). Second, following the insight of StyleGAN [22], our style space is
produced by learned transformations. This provides more flexibility to our
model than the baselines [16, 28, 34], which assume that the style space is a
fixed Gaussian distribution (Section 3.2). Last but not least, our modules
benefit from fully exploiting training data from multiple domains. By design,
the shared part of each module should learn domain-invariant features which
induces the regularization effect, encouraging better generalization to unseen
samples. To show that our model generalizes over the unseen images, we test a
few samples from FFHQ [22] with our model trained on CelebA-HQ (Figure 7).
Here, StarGAN v2 successfully captures styles of references and renders these
styles correctly to the source images.

Source Reference Output

Figure 7: Reference-guided synthesis results on FFHQ with the model trained on
CelebA-HQ. Despite the distribution gap between the two datasets, StarGAN v2
successfully extracts the style codes of the references and synthesizes
faithful images.

##  5 Related work

Generative adversarial networks (GANs) [10] have shown impressive results in
many computer vision tasks such as image synthesis [4, 31, 8], colorization
[18, 50] and super-resolution [27, 47]. Along with improving the visual
quality of generated images, their diversity also has been considered as an
important objective which has been tackled by either devoted loss functions
[34, 35] or architectural design [4, 22]. StyleGAN [22] introduces a non-
linear mapping function that embeds an input latent code into an intermediate
style space to better represent the factors of variation. However, this method
requires non-trivial effort when transforming a real image, since its
generator is not designed to take an image as input.

Early image-to-image translation methods [20, 53, 29] are well known to learn
a deterministic mapping even with stochastic noise inputs. Several methods
reinforce the connection between stochastic noise and the generated image for
diversity, by marginal matching [1], latent regression [54, 16], and diversity
regularization [48, 34]. Other approaches produce various outputs with the
guidance of reference images [5, 6, 32, 40]. However, all theses methods
consider only two domains, and their extension to multiple domains is non-
trivial. Recently, FUNIT [30] tackles multi-domain image translation using a
few reference images from a target domain, but it requires fine-grained class
labels and can not generate images with random noise. Our method provides both
latent-guided and reference-guided synthesis and can be trained with coarsely
labeled dataset. In parallel work, Yu et al. [51] tackle the same issue but
they define the style as domain-shared characteristics rather than domain-
specific ones, which limits the output diversity.

##  6 Conclusion

We proposed StarGAN v2, which addresses two major challenges in image-to-image
translation; translating an image of one domain to diverse images of a target
domain, and supporting multiple target domains. The experimental results
showed that our model can generate images with rich styles across multiple
domains, remarkably outperforming the previous leading methods [16, 28, 34].
We also released a new dataset of animal faces (AFHQ) for evaluating methods
in a large inter- and intra domain variation setting.

Acknowledgements. We thank the full-time and visiting Clova AI members for an
early review: Seongjoon Oh, Junsuk Choe, Muhammad Ferjad Naeem, and Kyungjune
Baek. All experiments were conducted based on NAVER Smart Machine Learning
(NSML) [23, 43].

Figure 8: Examples from our newly collected AFHQ dataset.

##  Appendix A The AFHQ dataset

We release a new dataset of animal faces, Animal Faces-HQ (AFHQ), consisting
of 15,000 high-quality images at $512\times 512$ resolution. Figure 8 shows
example images of the AFHQ dataset. The dataset includes three domains of cat,
dog, and wildlife, each providing 5000 images. By having multiple (three)
domains and diverse images of various breeds ($\geq$ eight) per each domain,
AFHQ sets a more challenging image-to-image translation problem. For each
domain, we select 500 images as a test set and provide all remaining images as
a training set. We collected images with permissive licenses from the
Flickr111https://www.flickr.com and Pixabay222https://www.pixabay.com
websites. All images are vertically and horizontally aligned to have the eyes
at the center. The low-quality images were discarded by human effort. We have
made dataset available at https://github.com/clovaai/stargan-v2.

##  Appendix B Training details

For fast training, the batch size is set to eight and the model is trained for
100K iterations. The training time is about three days on a single Tesla V100
GPU with our implementation in PyTorch [41]. We set ${\lambda}_{sty}=1$,
${\lambda}_{ds}=1$, and ${\lambda}_{cyc}=1$ for CelebA-HQ and
${\lambda}_{sty}=1$, ${\lambda}_{ds}=2$, and ${\lambda}_{cyc}=1$ for AFHQ. To
stabilize the training, the weight ${\lambda}_{ds}$ is linearly decayed to
zero over the 100K iterations. We adopt the non-saturating adversarial loss
[10] with ${R}_{1}$ regularization [35] using $\gamma=1$. We use the Adam [25]
optimizer with ${\beta}_{1}=0$ and ${\beta}_{2}=0.99$. The learning rates for
$G,D,$ and $E$ are set to ${10}^{-4}$, while that of $F$ is set to
${10}^{-6}$. For evaluation, we employ exponential moving averages over
parameters [21, 49] of all modules except $D$. We initialize the weights of
all modules using He initialization [12] and set all biases to zero, except
for the biases associated with the scaling vectors of AdaIN that are set to
one.

##  Appendix C Evaluation protocol

This section provides details for the evaluation metrics and evaluation
protocols used in all experiments.

Frechét inception distance (FID) [14] measures the discrepancy between two
sets of images. We use the feature vectors from the last average pooling layer
of the ImageNet-pretrained Inception-V3 [44]. For each test image from a
source domain, we translate it into a target domain using 10 latent vectors,
which are randomly sampled from the standard Gaussian distribution. We then
calculate FID between the translated images and training images in the target
domain. We calculate the FID values for every pair of image domains (_e.g_.
female $\rightleftarrows$ male for CelebA-HQ) and report the average value.
Note that, for reference-guided synthesis, each source image is transformed
using 10 reference images randomly sampled from the test set of a target
domain.

Learned perceptual image patch similarity (LPIPS) [52] measures the diversity
of generated images using the ${L}_{1}$ distance between features extracted
from the ImageNet-pretrained AlexNet [26]. For each test image from a source
domain, we generate 10 outputs of a target domain using 10 randomly sampled
latent vectors. Then, we compute the average of the pairwise distances among
all outputs generated from the same input (_i.e_. 45 pairs). Finally, we
report the average of the LPIPS values over all test images. For reference-
guided synthesis, each source image is transformed using 10 reference images
to produce 10 outputs.

##  Appendix D Additional results

We provide additional reference-guided image synthesis results on both CelebA-
HQ and AFHQ (Figure 9 and 10). In CelebA-HQ, StarGAN v2 synthesizes the source
identity in diverse appearances reflecting the reference styles such as
hairstyle, and makeup. In AFHQ, the results follow the breed and hair of the
reference images preserving the pose of the source images. Interpolation
results between styles can be found at https://youtu.be/0EVh5Ki4dIY.

Figure 9: Reference-guided image synthesis results on CelebA-HQ. The source
and reference images in the first row and the first column are real images,
while the rest are images generated by our proposed model, StarGAN v2. Our
model learns to transform a source image reflecting the style of a given
reference image. High-level semantics such as hairstyle, makeup, beard and age
are followed from the reference images, while the pose and identity of the
source images are preserved. Note that the images in each column share a
single identity with different styles, and those in each row share a style
with different identities.

Figure 10: Reference-guided image synthesis results on AFHQ. All images except
the sources and references are generated by our proposed model, StarGAN v2.
High-level semantics such as hair are followed from the references, while the
pose of the sources are preserved.

##  Appendix E Network architecture

In this section, we provide architectural details of StarGAN v2, which
consists of four modules described below.

Generator (Table 5). For AFHQ, our generator consists of four downsampling
blocks, four intermediate blocks, and four upsampling blocks, all of which
inherit pre-activation residual units [13]. We use the instance normalization
(IN) [45] and the adaptive instance normalization (AdaIN) [15, 22] for down-
sampling and up-sampling blocks, respectively. A style code is injected into
all AdaIN layers, providing scaling and shifting vectors through learned
affine transformations. For CelebA-HQ, we increase the number of downsampling
and upsampling layers by one. We also remove all shortcuts in the upsampling
residual blocks and add skip connections with the adaptive wing based heatmap
[46].

Mapping network (Table 6). Our mapping network consists of an MLP with K
output branches, where K indicates the number of domains. Four fully connected
layers are shared among all domains, followed by four specific fully connected
layers for each domain. We set the dimensions of the latent code, the hidden
layer, and the style code to 16, 512, and 64, respectively. We sample the
latent code from the standard Gaussian distribution. We do not apply the pixel
normalization [22] to the latent code, which has been observed not to increase
model performance in our tasks. We also tried feature normalizations [3, 19],
but this degraded performance.

Style encoder (Table 7). Our style encoder consists of a CNN with K output
branches, where K is the number of domains. Six pre-activation residual blocks
are shared among all domains, followed by one specific fully connected layer
for each domain. We do not use the global average pooling [16] to extract fine
style features of a given reference image. The output dimension “D” in Table 7
is set to 64, which indicates the dimension of the style code.

Discriminator (Table 7). Our discriminator is a multi-task discriminator [35],
which contains multiple linear output branches 333The original implementation
of the multi-task discriminator can be found at
https://github.com/LMescheder/GAN_stability.. The discriminator contains six
pre-activation residual blocks with leaky ReLU [33]. We use K fully-connected
layers for real/fake classification of each domain, where K indicates the
number of domains. The output dimension “D” is set to 1 for real/fake
classification. We do not use any feature normalization techniques [19, 45]
nor PatchGAN [20] as they have been observed not to improve output quality. We
have observed that in our settings, the multi-task discriminator provides
better results than other types of conditional discriminators [36, 37, 39,
42].

Layer | Resample | Norm | Output Shape
---|---|---|---
Image $\mathbf{x}$ | - | - | $\makebox[0.0pt][c]{256}\times\makebox[0.0pt][c]{256}\times\makebox[0.0pt][c]{3}$
Conv1$\times$1 | - | - | $\makebox[0.0pt][c]{256}\times\makebox[0.0pt][c]{256}\times\makebox[0.0pt][c]{64}$
ResBlk | AvgPool | IN | $\makebox[0.0pt][c]{128}\times\makebox[0.0pt][c]{128}\times\makebox[0.0pt][c]{128}$
ResBlk | AvgPool | IN | $\makebox[0.0pt][c]{64}\times\makebox[0.0pt][c]{64}\times\makebox[0.0pt][c]{256}$
ResBlk | AvgPool | IN | $\makebox[0.0pt][c]{32}\times\makebox[0.0pt][c]{32}\times\makebox[0.0pt][c]{512}$
ResBlk | AvgPool | IN | $\makebox[0.0pt][c]{16}\times\makebox[0.0pt][c]{16}\times\makebox[0.0pt][c]{512}$
ResBlk | - | IN | $\makebox[0.0pt][c]{16}\times\makebox[0.0pt][c]{16}\times\makebox[0.0pt][c]{512}$
ResBlk | - | IN | $\makebox[0.0pt][c]{16}\times\makebox[0.0pt][c]{16}\times\makebox[0.0pt][c]{512}$
ResBlk | - | AdaIN | $\makebox[0.0pt][c]{16}\times\makebox[0.0pt][c]{16}\times\makebox[0.0pt][c]{512}$
ResBlk | - | AdaIN | $\makebox[0.0pt][c]{16}\times\makebox[0.0pt][c]{16}\times\makebox[0.0pt][c]{512}$
ResBlk | Upsample | AdaIN | $\makebox[0.0pt][c]{32}\times\makebox[0.0pt][c]{32}\times\makebox[0.0pt][c]{512}$
ResBlk | Upsample | AdaIN | $\makebox[0.0pt][c]{64}\times\makebox[0.0pt][c]{64}\times\makebox[0.0pt][c]{256}$
ResBlk | Upsample | AdaIN | $\makebox[0.0pt][c]{128}\times\makebox[0.0pt][c]{128}\times\makebox[0.0pt][c]{128}$
ResBlk | Upsample | AdaIN | $\makebox[0.0pt][c]{256}\times\makebox[0.0pt][c]{256}\times\makebox[0.0pt][c]{64}$
Conv1$\times$1 | - | - | $\makebox[0.0pt][c]{256}\times\makebox[0.0pt][c]{256}\times\makebox[0.0pt][c]{3}$
Table 5: Generator architecture. Type | Layer | Actvation | Output shape
---|---|---|---
Shared | Latent $\mathbf{z}$ | - | 16
Shared | Linear | ReLU | 512
Shared | Linear | ReLU | 512
Shared | Linear | ReLU | 512
Shared | Linear | ReLU | 512
Unshared | Linear | ReLU | 512
Unshared | Linear | ReLU | 512
Unshared | Linear | ReLU | 512
Unshared | Linear | - | 64
Table 6: Mapping network architecture. Layer | Resample | Norm | Output shape
---|---|---|---
Image $\mathbf{x}$ | - | - | $\makebox[0.0pt][c]{256}\times\makebox[0.0pt][c]{256}\times\makebox[0.0pt][c]{3}$
Conv1$\times$1 | - | - | $\makebox[0.0pt][c]{256}\times\makebox[0.0pt][c]{256}\times\makebox[0.0pt][c]{64}$
ResBlk | AvgPool | - | $\makebox[0.0pt][c]{128}\times\makebox[0.0pt][c]{128}\times\makebox[0.0pt][c]{128}$
ResBlk | AvgPool | - | $\makebox[0.0pt][c]{64}\times\makebox[0.0pt][c]{64}\times\makebox[0.0pt][c]{256}$
ResBlk | AvgPool | - | $\makebox[0.0pt][c]{32}\times\makebox[0.0pt][c]{32}\times\makebox[0.0pt][c]{512}$
ResBlk | AvgPool | - | $\makebox[0.0pt][c]{16}\times\makebox[0.0pt][c]{16}\times\makebox[0.0pt][c]{512}$
ResBlk | AvgPool | - | $\makebox[0.0pt][c]{8}\times\makebox[0.0pt][c]{8}\times\makebox[0.0pt][c]{512}$
ResBlk | AvgPool | - | $\makebox[0.0pt][c]{4}\times\makebox[0.0pt][c]{4}\times\makebox[0.0pt][c]{512}$
LReLU | - | - | $\makebox[0.0pt][c]{4}\times\makebox[0.0pt][c]{4}\times\makebox[0.0pt][c]{512}$
Conv4$\times$4 | - | - | $\makebox[0.0pt][c]{1}\times\makebox[0.0pt][c]{1}\times\makebox[0.0pt][c]{512}$
LReLU | - | - | $\makebox[0.0pt][c]{1}\times\makebox[0.0pt][c]{1}\times\makebox[0.0pt][c]{512}$
Reshape | - | - | 512
Linear * K | - | - |  D * K
Table 7: Style encoder and discriminator architectures. D and K represent the
output dimension and number of domains, respectively.

## References

  * [1] A. Almahairi, S. Rajeshwar, A. Sordoni, P. Bachman, and A. Courville.  Augmented cyclegan: Learning many-to-many mappings from unpaired data.  In ICML, 2018.
  * [2] A. Anoosheh, E. Agustsson, R. Timofte, and L. Van Gool.  Combogan: Unrestrained scalability for image domain translation.  In CVPRW, 2018.
  * [3] J. L. Ba, J. R. Kiros, and G. E. Hinton.  Layer normalization.  In arXiv preprint, 2016.
  * [4] A. Brock, J. Donahue, and K. Simonyan.  Large scale gan training for high fidelity natural image synthesis.  In ICLR, 2019.
  * [5] H. Chang, J. Lu, F. Yu, and A. Finkelstein.  Pairedcyclegan: Asymmetric style transfer for applying and removing makeup.  In CVPR, 2018.
  * [6] W. Cho, S. Choi, D. K. Park, I. Shin, and J. Choo.  Image-to-image translation via group-wise deep whitening-and-coloring transformation.  In CVPR, 2019.
  * [7] Y. Choi, M. Choi, M. Kim, J.-W. Ha, S. Kim, and J. Choo.  Stargan: Unified generative adversarial networks for multi-domain image-to-image translation.  In CVPR, 2018.
  * [8] J. Donahue and K. Simonyan.  Large scale adversarial representation learning.  In NeurIPS, 2019.
  * [9] V. Dumoulin, E. Perez, N. Schucher, F. Strub, H. d. Vries, A. Courville, and Y. Bengio.  Feature-wise transformations.  In Distill, 2018.
  * [10] I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio.  Generative adversarial networks.  In NeurIPS, 2014.
  * [11] I. Gulrajani, F. Ahmed, M. Arjovsky, V. Dumoulin, and A. C. Courville.  Improved training of wasserstein gans.  In NeurIPS, 2017.
  * [12] K. He, X. Zhang, S. Ren, and J. Sun.  Delving deep into rectifiers: Surpassing human-level performance on imagenet classification.  In ICCV, 2015.
  * [13] K. He, X. Zhang, S. Ren, and J. Sun.  Identity mappings in deep residual networks.  In ECCV, 2016.
  * [14] M. Heusel, H. Ramsauer, T. Unterthiner, B. Nessler, and S. Hochreiter.  Gans trained by a two time-scale update rule converge to a local nash equilibrium.  In NeurIPS, 2017.
  * [15] X. Huang and S. Belongie.  Arbitrary style transfer in real-time with adaptive instance normalization.  In ICCV, 2017.
  * [16] X. Huang, M.-Y. Liu, S. Belongie, and J. Kautz.  Multimodal unsupervised image-to-image translation.  In ECCV, 2018.
  * [17] L. Hui, X. Li, J. Chen, H. He, and J. Yang.  Unsupervised multi-domain image translation with domain-specific encoders/decoders.  In ICPR, 2018.
  * [18] K. Hyunsu, J. Ho Young, P. Eunhyeok, and Y. Sungjoo.  Tag2pix: Line art colorization using text tag with secat and changing loss.  In ICCV, 2019.
  * [19] S. Ioffe and C. Szegedy.  Batch normalization: Accelerating deep network training by reducing internal covariate shift.  In ICML, 2015.
  * [20] P. Isola, J.-Y. Zhu, T. Zhou, and A. A. Efros.  Image-to-image translation with conditional adversarial nets.  In CVPR, 2017.
  * [21] T. Karras, T. Aila, S. Laine, and J. Lehtinen.  Progressive growing of GANs for improved quality, stability, and variation.  In ICLR, 2018.
  * [22] T. Karras, S. Laine, and T. Aila.  A style-based generator architecture for generative adversarial networks.  In CVPR, 2019.
  * [23] H. Kim, M. Kim, D. Seo, J. Kim, H. Park, S. Park, H. Jo, K. Kim, Y. Yang, Y. Kim, et al.  Nsml: Meet the mlaas platform with a real-world case study.  arXiv preprint arXiv:1810.09957, 2018.
  * [24] T. Kim, M. Cha, H. Kim, J. K. Lee, and J. Kim.  Learning to discover cross-domain relations with generative adversarial networks.  In ICML, 2017.
  * [25] D. P. Kingma and J. Ba.  Adam: A method for stochastic optimization.  In ICLR, 2015.
  * [26] A. Krizhevsky, I. Sutskever, and G. E. Hinton.  Imagenet classification with deep convolutional neural networks.  In NeurIPS, 2012.
  * [27] C. Ledig, L. Theis, F. Huszár, J. Caballero, A. Cunningham, A. Acosta, A. Aitken, A. Tejani, J. Totz, Z. Wang, et al.  Photo-realistic single image super-resolution using a generative adversarial network.  In CVPR, 2017.
  * [28] H.-Y. Lee, H.-Y. Tseng, J.-B. Huang, M. K. Singh, and M.-H. Yang.  Diverse image-to-image translation via disentangled representations.  In ECCV, 2018.
  * [29] M.-Y. Liu, T. Breuel, and J. Kautz.  Unsupervised image-to-image translation networks.  In NeurIPS, 2017.
  * [30] M.-Y. Liu, X. Huang, A. Mallya, T. Karras, T. Aila, J. Lehtinen, and J. Kautz.  Few-shot unsupervised image-to-image translation.  In ICCV, 2019.
  * [31] M. Lucic, M. Tschannen, M. Ritter, X. Zhai, O. Bachem, and S. Gelly.  High-fidelity image generation with fewer labels.  In ICML, 2019.
  * [32] L. Ma, X. Jia, S. Georgoulis, T. Tuytelaars, and L. Van Gool.  Exemplar guided unsupervised image-to-image translation with semantic consistency.  In ICLR, 2019.
  * [33] A. L. Maas, A. Y. Hannun, and A. Y. Ng.  Rectifier nonlinearities improve neural network acoustic models.  In ICML, 2013.
  * [34] Q. Mao, H.-Y. Lee, H.-Y. Tseng, S. Ma, and M.-H. Yang.  Mode seeking generative adversarial networks for diverse image synthesis.  In CVPR, 2019.
  * [35] L. Mescheder, S. Nowozin, and A. Geiger.  Which training methods for gans do actually converge?  In ICML, 2018.
  * [36] M. Mirza and S. Osindero.  Conditional generative adversarial nets.  In arXiv preprint, 2014.
  * [37] T. Miyato and M. Koyama.  cGANs with projection discriminator.  In ICLR, 2018.
  * [38] S. Na, S. Yoo, and J. Choo.  Miso: Mutual information loss with stochastic style representations for multimodal image-to-image translation.  In arXiv preprint, 2019.
  * [39] A. Odena, C. Olah, and J. Shlens.  Conditional image synthesis with auxiliary classifier gans.  In ICML, 2017.
  * [40] T. Park, M.-Y. Liu, T.-C. Wang, and J.-Y. Zhu.  Semantic image synthesis with spatially-adaptive normalization.  In CVPR, 2019.
  * [41] A. Paszke, S. Gross, S. Chintala, G. Chanan, E. Yang, Z. DeVito, Z. Lin, A. Desmaison, L. Antiga, and A. Lerer.  Automatic differentiation in pytorch.  In NeurIPSW, 2017.
  * [42] S. Reed, Z. Akata, X. Yan, L. Logeswaran, B. Schiele, and H. Lee.  Generative adversarial text to image synthesis.  In ICML, 2016.
  * [43] N. Sung, M. Kim, H. Jo, Y. Yang, J. Kim, L. Lausen, Y. Kim, G. Lee, D. Kwak, J.-W. Ha, et al.  Nsml: A machine learning platform that enables you to focus on your models.  arXiv preprint arXiv:1712.05902, 2017.
  * [44] C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna.  Rethinking the inception architecture for computer vision.  In CVPR, 2016.
  * [45] D. Ulyanov, A. Vedaldi, and V. Lempitsky.  Instance normalization: The missing ingredient for fast stylization.  In arXiv preprint, 2016.
  * [46] X. Wang, L. Bo, and L. Fuxin.  Adaptive wing loss for robust face alignment via heatmap regression.  In ICCV, 2019.
  * [47] X. Wang, K. Yu, S. Wu, J. Gu, Y. Liu, C. Dong, Y. Qiao, and C. Change Loy.  Esrgan: Enhanced super-resolution generative adversarial networks.  In ECCV, 2018.
  * [48] D. Yang, S. Hong, Y. Jang, T. Zhao, and H. Lee.  Diversity-sensitive conditional generative adversarial networks.  In ICLR, 2019.
  * [49] Y. Yazıcı, C.-S. Foo, S. Winkler, K.-H. Yap, G. Piliouras, and V. Chandrasekhar.  The unusual effectiveness of averaging in gan training.  In ICLR, 2019.
  * [50] S. Yoo, H. Bahng, S. Chung, J. Lee, J. Chang, and J. Choo.  Coloring with limited data: Few-shot colorization via memory augmented networks.  In CVPR, 2019.
  * [51] X. Yu, Y. Chen, T. Li, S. Liu, and G. Li.  Multi-mapping image-to-image translation via learning disentanglement.  In NeurIPS, 2019.
  * [52] R. Zhang, P. Isola, A. A. Efros, E. Shechtman, and O. Wang.  The unreasonable effectiveness of deep features as a perceptual metric.  In CVPR, 2018.
  * [53] J.-Y. Zhu, T. Park, P. Isola, and A. A. Efros.  Unpaired image-to-image translation using cycle-consistent adversarial networkss.  In ICCV, 2017.
  * [54] J.-Y. Zhu, R. Zhang, D. Pathak, T. Darrell, A. A. Efros, O. Wang, and E. Shechtman.  Toward multimodal image-to-image translation.  In NeurIPS, 2017.