aainstitutetext: Department of Physics and State Key Laboratory of Nuclear
Physics and Technology, Peking University, Beijing 100871,
Chinabbinstitutetext: CAS Key Laboratory of Theoretical Physics, Institute of
Theoretical Physics, Chinese Academy of Sciences, Beijing 100190, P. R.
Chinaccinstitutetext: School of Physical Sciences, University of Chinese
Academy of Sciences, Beijing 100049, P.R. Chinaddinstitutetext: School of
Fundamental Physics and Mathematical Sciences, Hangzhou Institute for Advanced
Study, University of Chinese Academy of Sciences, Hangzhou 310024,
Chinaeeinstitutetext: International Center for Theoretical Physics Asia-
Pacific, Beijing/Hanzhou, China

# A Tale of Two-U(1) Axion Models

Dong Hu a Hao-Ran Jiang b Hao-Lin Li b Ming-Lei Xiao b, c, d, e Jiang-Hao Yu
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

The $U(1)$ global symmetry to solve the strong $CP$ problem could be the
remnant of multi-$U(1)$ symmetries from QCD and hidden strong dynamics. Both
Peccei-Quinn $U(1)$ and dynamical $U(1)$ are described uniformly, based on
which we classify various mixed two-$U(1)$ models to solve both strong $CP$
and quality problems. We propose a moose diagram method with different fermion
assignments to directly read relations between $CP$ phases, which illustrate
how strong $CP$ problem is solved in terms of cancellation between $CP$
phases. In two axion models, we find the lightest axion is still same as QCD
axion at infrared region, while one axion model with $Z_{2}$ symmetry enhances
the axion mass spectrum. Our discussions can be extended to multi-axion cases.

##  1 Introduction

In the 1970s, ’t Hooft proposed that QCD has a nontrivial vacuum structure and
solved the $U(1)_{A}$ problem Hooft1976 ; Hooft1976a . The nontrivial vacuum
structure suggests that there is an additional topological term which violates
$CP$ symmetry in the QCD Lagrangian. The $CP$ violating term brings a free
parameter denoted as $\theta$. Due to the axial anomaly, a chiral rotation of
quarks changes $\theta$ and the argument of Yukawa couplings of quarks, while
the sum of them is invariant. Thus, the observable strong $CP$ phase in the
Standard Model (SM) is

$\overline{\theta}=\theta+\arg\det(Y_{u}Y_{d}),$ (1)

where $u,d$ represent up and down type quarks respectively. The measurement of
neutron electric dipole moment (EDM) suggests that $|\bar{\theta}|<10^{-10}$
Baker2006 . How to understand the extreme smallness of $\overline{\theta}$ is
the well-known strong $CP$ problem.

There are several ways to solve the strong $CP$ problem. One solution is the
Nelson-Barr mechanism Nelson1984NWCV ; Barr1984StSwtPQS , which assumes $CP$
to be conserved at high energy scale, i.e. the $CP$-violating phases of the
CKM matrix and $\overline{\theta}$ are both zero at the high energy scale.
While at the low energy scale the $CP$-violating phase $\delta$ of the CKM
matrix is reproduced and $\overline{\theta}$ is still fixed to be zero. An
alternative way is utilizing the chiral rotation of fermions to absorb
$\overline{\theta}$, when a massless quark or an additional global chiral
$U(1)$ symmetry exists. However, the solution with massless quark has been
disfavored by the results of Lattice QCD with $m_{u}=2.2^{+0.5}_{-0.4}~{}MeV$
Gasser1982 . The additional global chiral $U(1)$ solution was first proposed
by Peccei and Quinn Peccei1977 ; Peccei1977a . Later, Weinberg
Weinberg1978ANLB  and Wilczek Wilczek1978PoSitPoI  predicted a pseudoscalar as
the Goldstone boson called axion from the spontaneous breaking of this
additional $U(1)_{PQ}$ symmetry. Although the original Weinberg-Wilczek axion
model has been ruled out by experiments Bardeen1987 , derived models still
survive. The KSVZ Kim1979a ; Shifman1980a  and the DFSZ Dine1981a ;
zhitnitskii1980possible  are the most typical models among them.

Although the KSVZ and the DFSZ survive from experimental constrains, there
exists an additional theoretical issue, the quality problem. Constraints from
Astrophysics give the lower bound of the decay constant of axion to be
$f_{a}\gtrsim 10^{8}\mathrm{GeV}$ RaffeltAAB . There is a general consensus
that gravitational effects generate operators suppressed by the Planck scale
$M_{\mathrm{Pl}}$, which explicitly break global symmetries. As for the axion
model, the explicit breaking effect is estimated by the operators Barr1992 ;
Kamionkowski1992

$V(\phi)=g\frac{|\phi|^{2m}\phi^{n}}{M_{\mathrm{Pl}}^{2m+n-4}}+h.c..$ (2)

Although these operators are suppressed by the Planck scale, the smallness of
strong $CP$ phase and high scale of $\langle\phi\rangle\sim f_{a}$ cause non-
negligible effects from the lower dimension operators. Consequentially, the
minimum of the scalar potential is shifted and the $CP$ phase
$\overline{\theta}$ reappears. In order to solve this problem, one can impose
a discrete symmetry $Z_{N}$ with large $N$ on $\phi$ so that operators with
dimension less than $N$ are forbidden. For $f_{a}=10^{12}$ $\mathrm{GeV}$, a
scale that the axion can serve as dark matter, operators with dimension
$2m+n<14$ are forbidden Hook2018TLotSCPaA . Another idea is to reduce the VEV
$\langle\phi\rangle$ while preserving a large $f_{a}$. The implementation is
accomplished in the multiple axion models using the alignment mechanism
Kim2005 ; Choi2014a ; Higaki2016  or the Clockwork mechanism Bonnefoy2018 . On
the other hand, very heavy aixon with $m_{a}\gtrsim O(100)\mathrm{MeV}$ evades
the astrophysical constraints Fukuda2015AMoVQA , thus a small $f_{a}$ is
allowed to relax the quality problem. Furthermore, in some composite axion
models, gauge invariance forbids operators of high dimension by arranging
fermions suitably Randall1992CAMaPSP .

Additional non-Abelian gauge group is often introduced in various new physics,
such as Hidden Valley Strassler2006EoaHVaHC , Vectorlike Confinement
Kilic2009VCatL  and Twin Higgs Chacko2006NEBfaMS . These extensions of the SM
are usually motivated by the Hierarchy problem or dark matter. Furthermore,
the $U(1)$ global symmetry to solve the strong $CP$ problem could be the
remnant of multi-$U(1)$ symmetry from the hidden strong dynamics. We call this
kind of non-Abelian gauge groups as “hidden QCD" in this paper.

Similar to the SM QCD, the hidden QCD may also contain the $CP$ violation
source from the $\theta^{\prime}G^{\prime}\tilde{G^{\prime}}$ term and the
relevant fermion Yukawa couplings, though it may or may not influcence the
observed neutron EDM. Besides the QCD theta term, there are three types of
effective operators directly contributing to the neutron EDM, including the
quark EDM, the quark chromo EDM and the three-gluon Weinberg operator
Weinberg1989LHBETitNEDM ; Pospelov2005EDMAPoNP :

$\displaystyle O_{d}$
$\displaystyle=-\frac{i}{2}d\bar{q}\sigma^{\mu\nu}\gamma_{5}qF_{\mu\nu},$ (3)
$\displaystyle O_{\tilde{d}}$
$\displaystyle=-\frac{i}{2}\tilde{d}\bar{q}\sigma^{\mu\nu}t^{a}\gamma_{5}qG_{\mu\nu}^{a},$
$\displaystyle O_{w}$
$\displaystyle=\frac{1}{3}wf^{abc}G_{\mu\nu}^{a}\widetilde{G}_{\nu\beta}^{b}G_{\beta\mu}^{c},$

where $d$ denotes EDM, $\tilde{d}$ denotes chromo EDM, $t^{a}$ is the
generator of the QCD group and $f^{abc}$ denotes the QCD structure constant.
The $\theta^{\prime}G^{\prime}\tilde{G^{\prime}}$ term in hidden QCD is not
directly related to these operators. However, two typical scenarios will
result in the sensitivity of the neutorn EDM to the CP violation in the hidden
sector:

  * •

Two strong $CP$ angels can be associated with each other by introducing a
pseudo-scalar. The interaction between the pseudo-scalar and gauged fermions
and the chiral rotation of the pseudo-scalar are

$\displaystyle\mathcal{L}$ $\displaystyle\sim
y_{q}e^{ina/f_{a}}q\bar{q}+y_{Q}e^{ima/f_{a}}Q\bar{Q}+h.c.,$ (4)
$\displaystyle a/f_{a}$ $\displaystyle\rightarrow
a/f_{a}+\alpha,\qquad\theta\rightarrow\theta-n\alpha,\qquad\theta^{\prime}\rightarrow\theta^{\prime}-m\alpha,$

where $q$ and $Q$ notate fermions charged under QCD and hidden QCD
respectively. $a$ is the pseudo-scalar. $n$ and $m$ are constants.

  * •

Fermions charged under both QCD and hidden QCD can link these two $CP$ phases.
The chiral rotation of these fermions can change both two phases as

$\psi_{L}(N_{c},N_{h})\rightarrow\psi_{L}(N_{c},N_{h})e^{i\beta},\qquad\theta\rightarrow\theta-
N_{h}\beta,\qquad\theta^{\prime}\rightarrow\theta^{\prime}-N_{c}\beta,$ (5)

where $N_{c}$ and $N_{h}$ are representation of fermions in QCD and hidden QCD
respectively.

Under such circumstance, these two angles are needed to be small
simultaneously to explain the strong $CP$ problem. An additional global chiral
$U(1)$ symmetries is not enough. Therefore, two global chiral $U(1)$
symmetries are needed, or there is one chiral $U(1)$ symmetry with a $Z_{2}$
among the two sectors. In this work, we will focus on the first scenario,
while the later one is discussed in Ref. Rubakov1997 ; Berezhiani2001a ;
Albaid2015SCaS ; Fukuda2015AMoVQA ; Hook2014ASttSCP ; Chiang2016 .

Chiral $U(1)$ symmetries are classified into two kinds, according to whether
the corresponding axion is elementary or composite. We refer to the one
generating the elementary axion as the $U(1)_{PQ}$ arinsing from the phase of
a complex scalar and the one generating composite axion as (also called
dynamical axion Choi1985DA ) the $U(1)_{A}$ arising when massless fermions of
the hidden QCD condensate at the high energy scale. In both scenarios, axions
are the Goldstones of these chiral $U(1)$ symmetries. Instanton effects
explicitly break these symmetries and determine the properties of axion such
as mass and axion-photon coupling. Therefore, new instanton effects of the
hidden QCD could enlarge axion mass in some models with $Z_{2}$ symmetry
Rubakov1997 ; Fukuda2015AMoVQA ; Gherghetta2016a  and it could also enhance
axion-photon coupling in Agrawal2017ETfPCotQA . Moreover, an additional strong
dynamics also provides solutions to quality problem and domain wall problem
Sikivie1982ADWatEU .

With hidden QCD introduced, we find new mixed $U(1)$ solutions which contain
two sorts of chiral $U(1)$ symmetries. In these cases, spontaneous symmetry
breaking of $U(1)_{PQ}$ could be triggered by the dynamical symmetry breaking
in the hidden sector, besides the conventional Ginzburg-Landau potential
method. The cancellation of $CP$ phases is always viable, except alignment
situation. The cancellation of $CP$ phases in the QCD and the hidden QCD
sector is shown in section 3. The lightest axion in these models are similar
to the QCD axion. Some phenomena of these models are discussed in section 4.

In addition to the new mixed $U(1)$ solution mentinoed above, we also propose
a “moose-like” diagram method to visualize the cumbersome relations among
gauge groups, new fields and CP phases in multi-axion models, from which the
$U(1)$ charges for the fields and the potential of the relevant axions and CP
phases can be easily read. Moreover, the diagram method helps to construct
models containing more $U(1)$ and gauge groups.

This paper is organized as follows. In section 2, we discuss two typical
patterns to realize Chiral $U(1)$. Next, in section 3, we give solutions of
the strong $CP$ problem, and propose a diagram method to present these
solutions. In section 4, we study the axion mass, axion-photon coupling and
axion deacy constant of one-axion solutions and two typical mixed two-axion
solutions. Finally, we summarize the result in section 5.

##  2 Pecci-Quinn or Dynamical Solution

A large class of models solve the strong $CP$ problem by adding additional
global $U(1)$ symmetries. There are mainly two kinds of $U(1)$s: the one in
the Pecci-Quinn mechanism Peccei1977 ; Peccei1977a , denoted as $U(1)_{PQ}$,
is associated with a elementary scalar $\phi$; the other is the axial
$U(1)_{A}$ of some massless fermions charged under gauge groups, which induces
the dynamical solution Choi1985DA .

Although $U(1)_{PQ}$ and $U(1)_{A}$ symmetries are different, they solve the
strong $CP$ problem with the same philosophy: to make the $CP$ phase
dynamically cancelled by introducing a $U(1)$ pseudo-Goldstone boson (PGB)
with an anomaly-induced potential. For both scenarios, there are fermions
axially charged under the global $U(1)$,

$f_{L}\rightarrow e^{i\alpha}f_{L},\qquad f_{R}\rightarrow e^{-i\alpha}f_{R}.$
(6)

However, the $U(1)$ is broken in different ways: for Pecci-Quinn mechanism, it
is broken by the scalar potential $V(\phi)$, and the PGB turns out to be
$a\sim\arg\phi$, the axion; in the dynamical solution, it is broken by
condensation of the fermions, and the PGB is a composite of the fermion, also
known as the dynamical axion. The common infrared behavior is that both PGB
correspond to some axial currents, whose conservation is broken only by
anomaly. It inspires a general description of these models, as presented in
the following.

In this section, we propose a “moose-like” diagram method to uniformly
illustrate the structures of the models solving the strong $CP$ problem. The
notation of all possible additional fermions introduced are listed in TABLE 1.
As an example, we show the structure of the aligned axion model(left) Kim2005
and the dynamical axion model(right) Choi1985DA  in Fig. 1. At the left end,
we list all the axions, each representing a global $U(1)$, normalized as their
proper contributions to the CP phase. Dashed lines link the axions to the
fermions, and the numbers on them indicate the corresponding $U(1)$ charges of
the fermions. The solid lines with tags show the non-trivial representations
of the fermions under the linked gauge groups at the right end, like the color
$SU(3)_{c}$ and the hidden $SU(3)_{h}$. Circles ($\circ$) on the vertices
represent massive fermions; Crosses ($\times$) on the vertexes represent
massless fermions.

Table 1: Fermion charge and representation | $SU(3)_{c}$ | $SU(3)_{h}$ | $U(1)_{PQ}$
---|---|---|---
$\psi$ | 3 | 3 | 0
$\chi$ | 1 | 3 | 0
$Q$ | 3 | 1 | m
$Q^{\prime}$ | 1 | 3 | n
Figure 1:  In diagram (a), the solid line between Q and $SU(3)_{c}$ shows that
Q is under representation 3 of QCD and the solid line between $Q^{\prime}$ and
$SU(3)_{h}$ shows that Q is under representation 3 of hidden QCD. The red
dashed line between $a_{1}$ and Q($Q^{\prime}$) shows that the charge of
Q($Q^{\prime}$) under $U(1)_{PQ}$ is $m_{1}$($n_{1}$). Similarly, blue dashed
line shows that the charge of Q($Q^{\prime}$) under $U(1)_{PQ^{\prime}}$ is
$m_{2}$($n_{2}$). $1/f_{a_{1}}$ and $1/f_{a_{2}}$ are scale factors for two
$U(1)$s. In diagram (b), solid lines show that $\psi$ is under representation
3 for both QCD and hidden QCD, and $\chi$ is a singlet for QCD but under
representation 3 for hidden QCD. Red dashed lines show that the ratio of
$\psi$ and $\chi$ in $\eta_{h}^{\prime}$ current is 1:1, which is similar to
$U(1)$ charge in the left one at infrared region. The only difference is one
more normalization factor $\sqrt{2}/2$ in front of the scale factor
$1/f_{\pi_{h}}$. Similarly, blue lines show that the ratio of $\psi$ and
$\chi$ in $\pi_{h}$ current is 1:-3. The normalization factor is $\sqrt{6}/6$
and the scale factor is $1/f_{\pi_{h}}$.

The alignment axion model has been widely discussed in the literature Kim2005
; Choi2014a ; Kappl2014a . $SU(3)_{h}$ is introduced with a new free parameter
of $CP$-violation $\theta^{\prime}$. As shown in Fig. 1(a), massive fermions
$Q(3,1)$ and $Q^{\prime}(1,3)$ are charged under both $U(1)_{PQ}$ symmetry and
$U(1)_{PQ^{\prime}}$ symmetry. When $U(1)_{PQ}$ is broken, axion is left as a
pseudo-Goldstone boson. The corresponding current is

$\displaystyle J^{\mu}_{PQ}$
$\displaystyle=f_{a_{1}}\partial_{\mu}a_{1}+m_{1}\overline{Q}\gamma^{\mu}\gamma_{5}Q+n_{1}\overline{Q^{\prime}}\gamma^{\mu}\gamma_{5}Q^{\prime},$
(7) $\displaystyle J^{\mu}_{PQ^{\prime}}$
$\displaystyle=f_{a_{2}}\partial_{\mu}a_{2}+m_{2}\overline{Q}\gamma^{\mu}\gamma_{5}Q+n_{2}\overline{Q^{\prime}}\gamma^{\mu}\gamma_{5}Q^{\prime},$
$\displaystyle\partial_{\mu}J^{\mu}_{PQ}$
$\displaystyle=\frac{m_{1}g^{2}}{16\pi^{2}}G_{a}^{\mu\nu}\tilde{G}_{a\mu\nu}+\frac{n_{1}{g^{\prime}}^{2}}{16\pi^{2}}{G^{\prime}}_{A}^{\mu\nu}\tilde{G^{\prime}}_{A\mu\nu},$
$\displaystyle\partial_{\mu}J^{\mu}_{PQ^{\prime}}$
$\displaystyle=\frac{m_{2}g^{2}}{16\pi^{2}}G_{a}^{\mu\nu}\tilde{G}_{a\mu\nu}+\frac{n_{2}{g^{\prime}}^{2}}{16\pi^{2}}{G^{\prime}}_{A}^{\mu\nu}\tilde{G^{\prime}}_{A\mu\nu}.$

where $G_{a}^{\mu\nu}$ and ${G^{\prime}}_{A}^{\mu\nu}$ are strength tensors of
gauge fields for QCD and hidden QCD respectively. $g$ and $g^{\prime}$ are
couplings of gauge interactions. In some papers, the hidden sector is extended
to a “mirrored SM” with massive fermions $u^{\prime}$ and $d^{\prime}$ that do
not carry $U(1)_{PQ}$ charge like u and d in the SM Gherghetta2016a ;
Hook2014ASttSCP .

The main structure of the dynamical solution Choi1985DA  is shown in Fig 1(b).
Similar to the aligned axion model, $SU(3)_{h}$ is introduced with a new $CP$
parameter $\theta^{\prime}$. Two massless fermions $\psi$ and $\chi$ are
introduced to absorb $CP$-violating phases $\theta$ and $\theta^{\prime}$
through a $U(1)_{A}$ transformation. It’s common to assume that $SU(3)_{h}$
will confine just like QCD at a scale $f_{\pi_{h}}$ which is much higher than
QCD confining scale $f_{\pi}$. The related currents are

$\displaystyle J^{\mu}_{A}(\psi)$
$\displaystyle=\overline{\psi}\gamma^{\mu}\gamma_{5}\psi,\quad
J^{\mu}_{A}(\chi)=\overline{\chi}\gamma^{\mu}\gamma_{5}\chi$ (8)
$\displaystyle\partial_{\mu}J^{\mu}_{A}(\psi)$
$\displaystyle=\frac{3g^{2}}{16\pi^{2}}F_{a}^{\mu\nu}\tilde{F}_{a\mu\nu}+\frac{3g^{\prime
2}}{16\pi^{2}}G_{A}^{\mu\nu}\tilde{G}_{A\mu\nu},$
$\displaystyle\partial_{\mu}J^{\mu}_{A}(\chi)$ $\displaystyle=\frac{g^{\prime
2}}{16\pi^{2}}G_{A}^{\mu\nu}\tilde{G}_{A\mu\nu},$

Below the scale $f_{\pi_{h}}$, we assume the hidden sector has a dynamical
chiral symmetry breaking caused by the fermion condensate with

$\langle\bar{\psi}\psi\rangle\approx-
c_{\psi}f_{\pi_{h}}^{3},\qquad\langle\bar{\chi}\chi\rangle\approx-
c_{\chi}f_{\pi_{h}}^{3},$ (9)

where $c_{\psi}$,$c_{\chi}$ are constants and $f_{\pi_{h}}\gg f_{\pi}$.
Considering QCD as an additional “flavor symmetry”, there is a
$SU(4)_{L}\times SU(4)_{R}$ symmetry between $\psi$ and $\chi$. After the
condensate of hidden QCD, $\pi_{h}^{a}$ are Goldstones corresponding to coset
generators of $SU(4)_{L}\times SU(4)_{R}/SU(4)_{V}$. The decomposition of
$\pi_{h}^{a}$ into $SU(3)_{c}$ is

$15=8+3+\overline{3}+1.$ (10)

The QCD color-singlet scalar is denoted as $\pi_{h}\equiv\pi_{h}^{15}$ for
short. And the rest 14 colored scalars are supposed to be heavy because of QCD
condensate. However, there is one more color-singlet scalar
$\eta_{h}^{\prime}$ related to the $U(1)_{A}$ symmetry. The corresponding
currents related to these two fields are

$\displaystyle J_{\mu}(\pi_{h})$
$\displaystyle=\dfrac{1}{\sqrt{6}}(\overline{\psi}^{c}\gamma_{\mu}\gamma_{5}\psi_{c}-3\overline{\chi}\gamma_{\mu}\gamma_{5}\chi),$
(11) $\displaystyle J_{\mu}(\eta_{h}^{\prime})$
$\displaystyle=\dfrac{1}{\sqrt{2}}(\overline{\psi}^{c}\gamma_{\mu}\gamma_{5}\psi_{c}+\overline{\chi}\gamma_{\mu}\gamma_{5}\chi),$

where ‘c’ is the color index for $\psi$. These currents have been normalized
and we have chosen $tr(T^{a}T^{b})=2\delta^{ab}$ as normalization of $SU(4)$
generators. The specific matrices of $SU(4)$ generators and details of
derivation are shown in Appendix A. As a result of condensate, we can use
$\pi_{h}$ and $\eta_{h}^{\prime}$ standing for the $U(1)_{A}$ symmetry at low
energy scale. $\theta$ and $\theta^{\prime}$ can be offseted when $\pi_{h}$
and $\eta_{h}^{\prime}$ take VEVs.

##  3 Cancellation of the Strong $CP$ Phase

When considering the extension of the SM, new contributions to the strong $CP$
violation are introduced. Therefore, the strong $CP$ phase $\overline{\theta}$
needs to be modified. For distinction, the modified strong $CP$ phase is
notated as $\theta_{phy}$. In this section, we discuss these new contributions
and methods to solve the strong $CP$ problem. The difficulty is that
$\overline{\theta}$ is at order $1$ but $\theta_{phy}$ is at order $10^{-10}$.
Axion models and dynamic solutions both solve this difficulty by canceling
$\theta_{phy}$ at the minimal point of goldstone potential, which links
$\theta_{phy}$ with $\overline{\theta}$. To calculate $\theta_{phy}$ which
describes $CP$ violation effects at the scale lower than $\Lambda_{QCD}$,
Chiral Perturbation Theory (ChPT) is used to match fields of quarks with
fields of hadrons. Indeed, it is sufficient to consider ChPT with two light
quarks $u$ and $d$. The Lagrangian for QCD with an axion at hadron level is
Kuster2008A

$\mathcal{L}_{2}=\dfrac{f_{\pi}^{2}}{4}\mathrm{Tr}\left(\partial_{\mu}U\partial^{\mu}U^{\dagger}\right)+Af_{\pi}^{3}\mathrm{Tr}\left(MU^{\dagger}+UM^{\dagger}\right)+Bf_{\pi}^{4}\left(\theta+\frac{a}{f_{a}}+\frac{i}{2}\mathrm{Tr}\left(\log
U-\log U^{\dagger}\right)\right)^{2},$ (12)

with

$U=\exp\left[\dfrac{i\left(\pi^{a}\tau^{a}+I\eta\right)}{f_{\pi}}\right],\qquad
M=\begin{pmatrix}m_{u}&0\\\ 0&m_{d}\end{pmatrix},$ (13)

where $\pi^{a}$, $\eta$ are mesons and $a$ is the aixon. $A$ and $B$ are
dimensionless parameters matched by meson masses. Here $\tau^{a}$ are Pauli
matrices and $I$ is the identity matrix. For convenience, we set the matrix of
quark mass to be real in the rest of this paper, leading to
$\theta=\overline{\theta}$. The potential part is shown as

$V=-2Af_{\pi}^{3}\left[m_{u}\cos(\frac{\pi^{0}+\eta}{f_{\pi}})+m_{d}\cos(\frac{\pi^{0}-\eta}{f_{\pi}})\right]+Bf_{\pi}^{4}\left(\frac{2\eta}{f_{\pi}}+\theta+\frac{a}{f_{a}}\right)^{2}.$
(14)

For convenience, $\left\langle\pi^{0}\right\rangle/f_{\pi}$,
$\left\langle\eta\right\rangle/f_{\pi}$ and $\left\langle
a\right\rangle/f_{a}$ are defined as phases $\phi_{1}$, $\phi_{2}$ and
$\phi_{a}$ respectively. Considering $CP$ transformation, $\pi^{0}$, $\eta$
and $a$ are changed into $-\pi^{0}$, $-\eta$ and $-a$. Once $CP$ is conserved
in eq. (14), these phases need to meet

$\phi_{1}+\phi_{2}=0,\qquad\phi_{1}-\phi_{2}=0,\qquad
2\phi_{2}+\theta+\phi_{a}=0,$ (15)

which is equivalent to

$\phi_{1}=\phi_{2}=0,\qquad\theta+\phi_{a}=0.$ (16)

Considering the derivative of the effective potential in eq. (14) with respect
to $\pi^{0}$, $\eta$ and $a$, eq. (16) is exactly the solution at the minimal
point of the potential. New contributions from $U(1)$ symmetries are described
by phase $\phi_{a}$. Finally, the $CP$-violating observable $\theta_{phy}$ is
gotten as

$\theta_{phy}=\theta+\phi_{a}=\theta+\frac{\left\langle
a\right\rangle}{f_{a}}.$ (17)

The strong $CP$ problem is solved by the offset of axion VEV and the phase
$\theta$ to ensure $\theta_{phy}=0$. Axion mass can also be derived from eq.
(14) as

$m_{a}^{2}=m_{\pi}^{2}\frac{f_{\pi}^{2}}{f_{a}^{2}}\frac{m_{u}m_{d}}{(m_{u}+m_{d})^{2}}.$
(18)

From this QCD axion model, it can be found that $\theta_{phy}$ needs to be
modified due to the new contribution from axion. Similar absorption happens in
other axion models and dynamical solutions.

For a general hidden QCD model with new fermions added, two UV free
parameters, $\theta$ and $\theta^{\prime}$, have to be introduced. In general
cases, the potential term induced by the instanton effect is

$V_{\theta}=B_{1}f_{\pi}^{4}\left(\sum_{i=1}^{2}m_{i}\frac{a^{i}}{f_{a_{i}}}+\theta\right)^{2}+B_{2}f_{\pi_{h}}^{4}\left(\sum_{i=1}^{2}n_{i}\frac{a^{i}}{f_{a_{i}}}+\theta^{\prime}\right)^{2},$
(19)

where $m$ and $n$ stand for different anomalous charges. 111It’s more complex
for dynamical solutions, because normalization factor also has to be included
in this $m$ and $n$.

Strong $CP$ problems for both QCD and hidden QCD can be solved only satisfying

$\displaystyle\theta_{phy}$
$\displaystyle=\theta+\frac{\left\langle\eta^{\prime}\right\rangle}{f_{\pi}}+\theta_{U(1)}=0,$
(20) $\displaystyle\theta^{\prime}_{phy}$
$\displaystyle=\theta^{\prime}+\theta^{\prime}_{U(1)}=0.$

In the preceding equation, $\left\langle\eta^{\prime}\right\rangle$ means the
contribution from VEV of $\eta^{\prime}$, which can be absorbed by
redefinition of $\theta$. We are not interested in it, so we will absorb it
with $\theta$ in the following discussion. $\theta_{U(1)}$ and
$\theta^{\prime}_{U(1)}$ come from axion VEVs. In following subsections, we
discuss the $\theta$ cancellation of eq. (20) in some specific models with
different $U(1)$ symmetries.

###  3.1 One U(1)

As shown in eq. (20), there are two degrees of freedom needed to absorb both
$\theta$ and $\theta^{\prime}$. When only one $U(1)$ is introduced, there is
only one new degree of freedom. That means $\theta$ and $\theta^{\prime}$
cannot be absorbed simultaneously and the strong $CP$ problem cannot be
solved. To solve this problem, a mirrored $Z_{2}$ symmetry has to be
introduced between $SU(3)_{c}$ in the SM and $SU(3)_{h}$ in the hidden sector.
In UV theories, some models can achieve this by embedding these two $SU(3)$
into a larger gauge symmetry group $SU(6)$ Gherghetta2016a . Consequently, two
$CP$ phases are forced to be identical, which means

$\theta=\theta^{\prime}.$ (21)

As shown in Fig. 2, the additional $U(1)$ could be achieved by two methods.
One method introduces an axion with $U(1)_{PQ}$(left) and the other uses
massless $\psi(3,3)$ with $U(1)_{A}$(right). There are detailed discussions
about both two methods in literature Rubakov1997 ; Berezhiani2001a ;
Hook2014ASttSCP ; Fukuda2015AMoVQA ; Albaid2015SCaS ; Chiang2016 . For the
left one, cancellation eq. (20) becomes

$\theta_{phy}^{\prime}=\theta_{phy}=\theta+\frac{\left\langle
a\right\rangle}{f_{a}}=0.$ (22)

For the right one, as shown in Appendix A, the cancellation eq. (20) is
deduced to

$\theta_{phy}^{\prime}=\theta_{phy}=\theta+\sqrt{6}\dfrac{\left\langle\eta^{\prime}_{h}\right\rangle}{f_{\pi_{h}}}=0.$
(23)

In both cases, the strong $CP$ problem will be solved by only one $U(1)$
symmetry. And the $CP$-violating effects from $\theta$ and $\theta^{\prime}$
will be counteracted by $\left\langle a\right\rangle$ or
$\left\langle\eta^{\prime}_{h}\right\rangle$.

Figure 2: (a): Structure of $U(1)_{PQ}$ models with mirror symmetry. (b):
Structure of $U(1)_{A}$ dynamic solutions with mirror symmetry.

###  3.2 Two Similar U(1)s

As in the previous subsection, models with two $U(1)$ freedoms can also be
sorted by the different mechanism of $U(1)$ symmetries. As shown in Fig. 1,
models could contain two $U(1)_{PQ}$ or two $U(1)_{A}$ symmetries. For these
kinds of models, axial currents are determined by representations of fermions
in $SU(3)_{c}$ and $SU(3)_{h}$. The cancellation equation of $\theta$ and
$\theta^{\prime}$ will be determined by these currents and different $U(1)$
charge for fermions.

Here shows another advantage of our diagram method that the cancellation
equation can be read easily. Fig. 1(a) shows models with two $U(1)_{PQ}$
symmetries. Choosing $SU(3)_{c}$ as the starting point responsible for
$\theta$, we can find all possible links end at left side. Notice these links
can only go from right to left. The cancelling phase of each link is the
product of the charge number on the dashed line and the scale factor at the
end point on the left. Finally we can sum all offset terms from different
links together. For $\theta^{\prime}$, the starting point will be changed into
$SU(3)_{h}$, then repeat these steps. The cancellation equations from our
diagram read

$\displaystyle\theta_{phy}$ $\displaystyle=\theta+m_{1}\dfrac{\left\langle
a_{1}\right\rangle}{f_{a_{1}}}+m_{2}\dfrac{\left\langle
a_{2}\right\rangle}{f_{a_{2}}}=0,$ (24) $\displaystyle\theta_{phy}^{\prime}$
$\displaystyle=\theta^{\prime}+n_{1}\dfrac{\left\langle
a_{1}\right\rangle}{f_{a_{1}}}+n_{2}\dfrac{\left\langle
a_{2}\right\rangle}{f_{a_{2}}}=0.$

This two-axion model is widely discussed. When the angle between vectors
$(m_{1},m_{2})$ and $(n_{1},n_{2})$ is near zero (but not zero) on the
contrast to the large m and n, it is exactly the ‘alignment axion’ model
Kim2005 ; Choi2014a .

Models with two $U(1)_{A}$ symmetries provided by massless fermions have
similar expressions for $CP$-violating angles. In Fig. 1(b), $M_{\psi}=0$ and
$M\chi=0$ are supposed. The corresponding currents has been given in eq. (11).
It’s a little more complex to read cancellation equations for this model from
the figure. There will be two more things to notice. First, the normalization
factor has been already written together with scale factor. Second, the
degeneracy should be multiplied for each link, which comes from the color
index of the other $SU(3)$ outside the link. The degeneracy can be read from
the number for representation on other solid lines attached to the fermion
field in the link. For example, the degeneracy of the link,
“$SU(3)_{c}\rightarrow\psi\rightarrow\pi_{h}$", will be the number ‘3’ on the
solid line attached $\psi$ and $SU(3)_{h}$. When sum all links together, this
link should be multiplied by 3. The final cancellation equations are

$\displaystyle\theta_{phy}$
$\displaystyle=\theta+\frac{\sqrt{6}}{2}\dfrac{\left\langle\pi_{h}\right\rangle}{f_{h}}+\frac{3\sqrt{2}}{2}\dfrac{\left\langle\eta^{\prime}_{h}\right\rangle}{f_{h}}=0,$
(25) $\displaystyle\theta_{phy}^{\prime}$
$\displaystyle=\theta^{\prime}+2\sqrt{2}\dfrac{\left\langle\eta^{\prime}_{h}\right\rangle}{f_{h}}=0.$

The cancellation equations of other figures can also be read in this way.

###  3.3 Two Different U(1)s

In these two-$U(1)$ solutions, $U(1)$s need not to be the same, which means
models of one $U(1)_{PQ}$ and one $U(1)_{A}$ is possible. Here we show two
simple examples of that new kind of models in Fig. 3.

Figure 3: Structure of models with mixed $U(1)$s. (a) Model A: the spontaneous
breaking of $U(1)_{PQ}$ is independent with hidden QCD. (b) Model B: the
dynamical symmetry breaking of hidden QCD induces $U(1)_{PQ}$ breaking.

The mixed solution of $U(1)_{PQ}$ and $U(1)_{A}$ would be more complex, for
the reason that two independent scales, $f_{h}$ and $f_{a}$, are
simultaneously involved. $f_{h}$ is the condensate scale of hidden QCD
fermions, while $f_{a}$ is the spontaneously breaking scale of the Peccei-
Quinn symmetry. Fig. 3(a) shows model A in which $U(1)_{PQ}$ is spontaneous
broken independently with the hidden QCD. The result of cancellation equations
in this model is

$\displaystyle\theta_{phy}$
$\displaystyle=\theta+\sqrt{6}\dfrac{\left\langle\eta^{\prime}_{h}\right\rangle}{f_{h}}+m\dfrac{\left\langle
a\right\rangle}{f_{a}}=0,$ (26) $\displaystyle\theta_{phy}^{\prime}$
$\displaystyle=\theta^{\prime}+\sqrt{6}\dfrac{\left\langle\eta^{\prime}_{h}\right\rangle}{f_{h}}+n\dfrac{\left\langle
a\right\rangle}{f_{a}}=0.$

Fig. 3(b) shows model B in which the spontaneous breaking of $U(1)_{PQ}$ is
induced by the hidden QCD . The cancellation equations are

$\displaystyle\theta_{phy}$
$\displaystyle=\theta+\dfrac{3}{\sqrt{2}}\dfrac{\left\langle\eta^{\prime}_{h}\right\rangle}{f_{h}}+\dfrac{3}{\sqrt{6}}\dfrac{\left\langle\pi_{h}\right\rangle}{f_{h}}=0,$
(27) $\displaystyle\theta_{phy}^{\prime}$
$\displaystyle=\theta^{\prime}+2\sqrt{2}\dfrac{\left\langle\eta^{\prime}_{h}\right\rangle}{f_{h}}+n\dfrac{\left\langle
a\right\rangle}{f_{a}}=0.$

It seems that redundancy exists in these equations, as there are three VEVs to
absorb two $\theta$ parameters. However, there is an additional equation
constraining these three VEVs from the Yukawa term of $Q^{\prime}$, as shown
in (46) in the next section.

##  4 Phenomenology

In this section, we study phenomena in the one-axion solution and the mixed
two-axion solution, which can not directly be acquired from diagrams in the
last section. We compare the axion masses in these two kinds of solutions. We
also discuss the relation between the physical axion decay constant $F_{a}$
and the axion-photon coupling $g_{a\gamma\gamma}$ in the mixed two-axion
solution.

###  4.1 One-axion Solution

The one axion solution is exactly the implementation of the visible axion
model in which axion mass is enhanced by the condensate scale of the hidden
QCD. We illustrate this with a specific model shown in Fig. 2(a). In this
model, there is only one $U(1)_{PQ}$ chiral symmetry. The corresponding
current and its divergence are

$J^{\mu}_{PQ}=f_{a}\partial_{\mu}a+m\overline{Q}\gamma^{\mu}\gamma_{5}Q+m\overline{Q^{\prime}}\gamma^{\mu}\gamma_{5}Q^{\prime},\qquad\partial_{\mu}J^{\mu}_{PQ}=\frac{mg^{2}}{16\pi^{2}}F_{a}^{\mu\nu}\tilde{F}_{a\mu\nu}+\frac{mg^{\prime
2}}{16\pi^{2}}G_{A}^{\mu\nu}\tilde{G}_{A\mu\nu}.$ (28)

At the low energy scale, the axion mass term stems from the instanton effects
of QCD and the hidden QCD. Therefore, the mass terms are

$-\mathcal{L}_{\text{mass}}=\cdots+\Lambda_{c}^{4}\left(\frac{ma}{f_{a}}\right)^{2}+\Lambda_{h}^{4}\left(\frac{ma}{f_{a}}\right)^{2},$
(29)

where $\Lambda_{c}$ and $\Lambda_{h}$ denote the confinement scale of QCD and
the hidden QCD respectively. Due to the contribution from the high energy
scale $\Lambda_{h}$, the axion mass could be much heavier than KSVZ or DFSZ
axion. Different phenomena of the heavy axion may appear in collider
experiments or rare decay of mesons Gherghetta2016a .

###  4.2 Two-axion Solution

In two-axion solutions with extra hidden QCD, the light mass eigenstate always
exists and can be treated as an invisible axion, even if $\Lambda_{h}$ is much
higher than QCD scale $\Lambda_{c}$. We illustrate this with the mass matrix
of axions by a perturbative method. The Lagrangian of the mass part can be
expressed as

$-\mathcal{L}_{\text{mass}}=\cdots+\Lambda_{c}^{4}\left(\frac{m_{1}a_{1}}{f_{a_{1}}}+\frac{m_{2}a_{2}}{f_{a_{2}}}\right)^{2}+\Lambda_{h}^{4}\left(\frac{n_{1}a_{1}}{f_{a_{1}}}+\frac{n_{2}a_{2}}{f_{a_{2}}}\right)^{2},$
(30)

where $f_{a_{1}}$ and $f_{a_{2}}$ are scale factors. Anomaly coefficients of
axions are denoted as $m_{1},\ m_{2},\ n_{1}$ and $n_{2}$. The corresponding
mass matrix can be expressed as

$\Lambda_{c}^{4}\begin{pmatrix}m_{1}^{2}\frac{1}{f_{a_{1}}^{2}}&m_{1}m_{2}\frac{1}{f_{a_{1}}f_{a_{2}}}\\\
m_{1}m_{2}\frac{1}{f_{a_{1}}f_{a_{2}}}&m_{1}^{2}\frac{1}{f_{a_{2}}^{2}}\end{pmatrix}+\Lambda_{h}^{4}\begin{pmatrix}n_{1}^{2}\frac{1}{f_{a_{1}}^{2}}&n_{1}n_{2}\frac{1}{f_{a_{1}}f_{a_{2}}}\\\
n_{1}n_{2}\frac{1}{f_{a_{1}}f_{a_{2}}}&n_{1}^{2}\frac{1}{f_{a_{2}}^{2}}\end{pmatrix}.$
(31)

Since $\Lambda_{h}\gg\Lambda_{c}$, the first matrix with respect to
$\Lambda_{c}$ can be treated as perturbation. Considering zero order
approximation, the heavy degree of freedom is denoted as

$A\propto\frac{n_{1}a_{1}}{f_{a_{1}}}+\frac{n_{2}a_{2}}{f_{a_{2}}},$ (32)

while the massless one is in the orthogonal direction with $A$ in field space,
which is

$a\propto\frac{n_{2}a_{1}}{f_{a_{2}}}-\frac{n_{1}a_{2}}{f_{a_{1}}}.$ (33)

Replacing $(a_{1},\ a_{2})$ with $(A,\ a)$, eq. (30) becomes

$-\mathcal{L}_{\text{mass}}=\cdots+\Lambda_{c}^{4}\left(\cos\alpha\frac{a}{F_{a}}+\sin\alpha\frac{A}{F_{A}}\right)^{2}+\Lambda_{h}^{4}\left(\frac{A}{F_{A}}\right)^{2},$
(34)

where $F_{A}$ ($F_{a}$) is the decay constant of heavy axion ( light axion)
after diagonalization and normalization. $\alpha$ is the mixing angle. In
general, $F_{a}$ and $F_{A}$ are comparable. Therefore, the heavier one has a
mass of $m_{A}^{2}\sim\Lambda_{h}^{4}/F_{A}^{2}$ and the lighter one has a
mass of $m_{a}\sim\Lambda_{c}^{4}/F_{a}^{2}$. The lighter axion is exactly the
invisible QCD axion.

In the following part, we discuss two specific models of different breaking
mechanisms in mixed two-axion solutions. The model with two independently
breaking $U(1)$ symmetries (model A) is shown in Fig. 3(a) and the model with
one breaking $U(1)$ symmetry induced by the other (model B) is shown in Fig.
3(b). The main difference between these two models is whether the symmetry
breaking of $U(1)_{PQ}$ is induced by the breaking of $U(1)_{A}$. Chiral
symmetries are same in both cases. The corresponding currents and its
divergence of these models are

$\displaystyle J^{\mu}_{A}$
$\displaystyle=\dfrac{\sqrt{6}}{3}\overline{\psi}\gamma^{\mu}\gamma_{5}\psi,\qquad
J^{\mu}_{PQ}=f_{a}\partial_{\mu}a+(m\overline{Q}\gamma^{\mu}\gamma_{5}Q)+n\overline{Q^{\prime}}\gamma^{\mu}\gamma_{5}Q^{\prime},$
(35) $\displaystyle\partial_{\mu}J^{\mu}_{A}$
$\displaystyle=\frac{\sqrt{6}g^{2}}{16\pi^{2}}G_{a}^{\mu\nu}\tilde{G}_{a\mu\nu}+\frac{\sqrt{6}g^{\prime
2}}{16\pi^{2}}{G^{\prime}}_{A}^{\mu\nu}\tilde{G^{\prime}}_{A\mu\nu},$
$\displaystyle\partial_{\mu}J^{\mu}_{PQ}$
$\displaystyle=\frac{mg^{2}}{16\pi^{2}}G_{a}^{\mu\nu}\tilde{G}_{a\mu\nu}+\frac{ng^{\prime
2}}{16\pi^{2}}{G^{\prime}}_{A}^{\mu\nu}\tilde{G^{\prime}}_{A\mu\nu},$

where we set the anomaly coefficient $m=0$ in the second model for simplicity.

####  4.2.1 Model A

In model A, the spontaneous breaking of two chiral $U(1)$ symmetries comes
from different mechanisms. The $U(1)_{PQ}$ symmetry is broken by the effective
potential of the complex scalar $\phi$, while $U(1)_{A}$ is broken by the
condensate of massless fermion $\psi$. Therefore, these two scales are
independent. The only source of explicit breaking comes from the instanton
effects contributing to masses of both axions. The mass-relevant Lagrangian is

$-\mathcal{L}_{\text{mass}}=\cdots+bf_{\pi}^{4}\left(\frac{\sqrt{6}\eta^{\prime}_{h}}{f_{\pi_{h}}}+\frac{ma}{f_{a}}\right)^{2}+b^{\prime}f_{\pi_{h}}^{4}\left(\frac{\sqrt{6}\eta^{\prime}_{h}}{f_{\pi_{h}}}+\frac{na}{f_{a}}\right)^{2},$
(36)

where the first(second) term stems from QCD(hidden QCD) instanton effects with
$f_{\pi_{h}}\gg f_{\pi}$. After diagonalizing the mass matrix, we find the
lighter mass eigenstate $a_{phy}$ to be

$a_{\text{phy}}=\dfrac{nf_{\pi_{h}}\eta^{\prime}_{h}-\sqrt{6}f_{a}a}{\sqrt{{6f^{2}_{a}+n^{2}}f_{\pi_{h}}^{2}}},$
(37)

with its mass and decay constant are

$m_{a_{\text{phy}}}^{2}F_{a_{\text{phy}}}^{2}=m_{\pi}^{2}f_{\pi}^{2}\frac{m_{u}m_{d}}{(m_{u}+m_{d})^{2}},\qquad
F_{a_{\text{phy}}}=\dfrac{\sqrt{{6f^{2}_{a}+n^{2}}f_{\pi_{h}}^{2}}}{\sqrt{6}|m-n|}.$
(38)

Furthermore, the axion-photon coupling $g_{{a_{phy}}\gamma\gamma}$ can be
defined as

$\displaystyle\mathcal{L}_{{a_{phy}}\gamma\gamma}=\frac{1}{4}g_{{a_{\text{phy}}}\gamma\gamma}aF_{\mu\nu}\tilde{F}^{\mu\nu}=\frac{1}{4}(g_{{a_{\text{phy}}}\gamma\gamma}^{\text{IR}}+g_{{a_{\text{phy}}}\gamma\gamma}^{\text{UV}})aF_{\mu\nu}\tilde{F}^{\mu\nu},$
(39)

where $g_{{a_{\text{phy}}}\gamma\gamma}^{\text{IR}}$ comes from the mixing
between axion and $\pi^{0}$, The latest result is accurate to NLO Cortona2015
, which is

$g_{{a_{\text{phy}}}\gamma\gamma}^{\text{IR}}=-1.92(4)\frac{\alpha_{\mathrm{em}}}{2\pi
F_{a_{\text{phy}}}}.$ (40)

The UV part is model-dependent. In this model, $g_{a\gamma\gamma}^{\text{UV}}$
has contribution from the mixing of $\eta^{\prime}_{h}$ and $a$. the
interactions by anomaly effect between $\eta^{\prime}_{h}$, $a$ and photon can
be written as

$\mathcal{L}_{a\gamma\gamma}=\left(\sqrt{6}q^{2}_{\psi}\frac{\eta^{\prime}_{h}}{f_{\pi_{h}}}+6(q_{Q}^{2}+q_{Q^{\prime}}^{2})\frac{a}{f_{a}}\right)\frac{\alpha_{\mathrm{em}}}{4\pi}F_{\mu\nu}\tilde{F}^{\mu\nu},$
(41)

where $q_{\psi}$, $q_{Q}$ and $q_{Q^{\prime}}$ denote the electric charge of
each particles. Therefore,

$g_{{a_{\text{phy}}}\gamma\gamma}^{\text{UV}}=\dfrac{nq^{2}_{\psi}-6(q_{Q}^{2}+q_{Q^{\prime}}^{2})}{2|m-n|}\frac{\alpha_{\mathrm{em}}}{2\pi
F_{a_{\text{phy}}}}.$ (42)

When setting $n\gg|m-n|$, the UV contribution of the axion-photon coupling is
enlarged. Meanwhile the physical axion decay constant meets the condition
$F_{a_{\text{phy}}}\gg f_{a}$ as shown in eq. (38). This result is actually a
special case of ‘alignment axion’ models Kim2005 ; Choi2014a .

####  4.2.2 Model B

In model B, the spontaneous breaking of $U(1)_{PQ}$ is triggered by the
condensate of the light fermion $Q^{\prime}$ in the hidden sector. Thus two
breaking scales are related. We assume the potential of the complex scalar
$\phi$ containing only a quadratic term with $\mu^{2}>0$. The Yukawa term of
$Q^{\prime}$ will induce a tadpole term of $\phi$ after the chiral symmetry
breaking in the hidden sector. The Lagrangian after the chiral symmery
breaking is

$\displaystyle\mathcal{L}_{\text{CSB}}$
$\displaystyle=\partial_{\mu}\phi\partial^{\mu}\phi^{*}-\mu^{2}\phi^{2}+\dfrac{f_{\pi_{h}}^{2}}{4}\text{Tr}\partial_{\mu}\Sigma\partial^{\mu}\Sigma^{\dagger}+a^{\prime}f_{\pi_{h}}^{3}\text{Tr}\left(H\Sigma^{\dagger}+H^{\dagger}\Sigma\right)+\cdots,$
(43)

where

$\Sigma=\exp\left[\dfrac{i\left(\pi_{h}^{a}T^{a}+\frac{1}{\sqrt{2}}I_{4\times
4}\eta^{\prime}_{h}\right)}{f_{\pi}}\right],\qquad H=\begin{pmatrix}0&&&\\\
&0&&\\\ &&0&\\\ &&&y\phi\end{pmatrix}.$ (44)

Here $T^{a}$ are generators of the $SU(4)$ group meeting the trace condition
$\text{Tr}(T^{a}T^{b})=2\delta^{ab}$. The anomalous term is omitted in eq.
(43). Matrix $H$, the Yukawa terms of $\psi$ and $Q^{\prime}$, is treated as a
spurion field transforming as an adjoint representation under the $SU(4)_{V}$.
The third term in eq. (43) is the linear term of $\phi$ which triggers
$U(1)_{PQ}$ breaking. Specifically, the complex scalar $\phi$ is parameterized
as $\phi=\rho e^{i\left(na/{f_{a}}\right)}$. The effective potential of $\rho$
and $a$ is

$V(\rho,a)=\mu^{2}\rho^{2}-2a^{\prime}y\rho
f_{\pi_{h}}^{3}\cos\left(n\frac{a}{f_{a}}+\cdots\right),$ (45)

where $``\cdots"$ omits contributions from other composite scalars. Then we
can estimate the VEV of $\rho$ to be
$\left\langle\rho\right\rangle=f_{a}/\sqrt{2}\sim
2y\dfrac{a^{\prime}f_{\pi_{h}}^{3}}{\mu^{2}}$. Utilizing the anomalous
currents in eq. (35) and currents of $\pi_{h}$ and $\eta^{\prime}_{h}$ in eq.
(11), the mass terms of scalars in Lagrangian eq. (43) becomes

$-\mathcal{L}_{\text{mass}}=bf_{\pi}^{4}\left[\sqrt{6}\left(\dfrac{1}{2}\dfrac{\pi_{h}}{f_{\pi_{h}}}+\dfrac{\sqrt{3}}{2}\dfrac{\eta^{\prime}_{h}}{f_{\pi_{h}}}\right)\right]^{2}+a^{\prime}y\dfrac{f_{a}f_{\pi_{h}}^{3}}{\sqrt{2}}\left(\dfrac{na}{f_{a}}-\dfrac{3}{\sqrt{6}}\frac{\pi_{h}}{f_{\pi_{h}}}+\dfrac{1}{\sqrt{2}}\frac{\eta_{h}^{\prime}}{f_{\pi_{h}}}\right)^{2}+b^{\prime}f_{\pi_{h}}^{4}\left(\frac{2\sqrt{2}\eta^{\prime}_{h}}{f_{\pi_{h}}}\right)^{2},$
(46)

where the second term stems from the Yukawa term of $Q^{\prime}$. The
$na/f_{a}$ in second term could move to the third term by a chiral rotation
$\theta^{\prime}\rightarrow\theta^{\prime}+na/f_{a}$ and generates additional
differential interaction of $a$. By means of the same steps in the former
subsection, the lightest mass eigenstate is

$a_{phy}=\dfrac{nf_{\pi_{h}}\pi_{h}+3/\sqrt{6}f_{a}a}{\sqrt{{3/2f^{2}_{a}+n^{2}}f_{\pi_{h}}^{2}}},\qquad
F_{a_{\text{phy}}}=\dfrac{2\sqrt{3/2f^{2}_{a}+n^{2}f_{\pi_{h}}^{2}}}{\sqrt{6}|n|}.$
(47)

The $g_{a\gamma\gamma}^{\text{UV}}$ coming from $\pi_{h}$ to $\gamma\gamma$ is

$\mathcal{L}_{\pi_{h}\gamma\gamma}=\dfrac{\sqrt{6}}{2}(q^{2}_{\psi}-q^{2}_{Q^{\prime}})\frac{\pi_{h}}{f_{\pi_{h}}}\frac{\alpha_{\mathrm{em}}}{4\pi}F^{\mu\nu}\tilde{F}_{\mu\nu}.$
(48)

Therefore, the UV part contribution of light axion is

$g_{{a_{\text{phy}}}\gamma\gamma}^{\text{UV}}=2(q^{2}_{\psi}-q^{2}_{Q^{\prime}})\frac{\alpha_{\mathrm{em}}}{2\pi
F_{a_{\text{phy}}}}.$ (49)

In this model, the axion-photon coupling $g_{{a_{\text{phy}}}\gamma\gamma}$ is
independent of the anomaly coefficient $n$, while $F_{a_{\text{phy}}}$ is
inversely proportional to $|n|$. Furthermore, there is another colorless
scalar $\pi_{h}$ with $m^{2}_{\pi_{h}}\sim f_{a}f_{\pi_{h}}$.

##  5 Conclusion and Discussion

We investigate all possible solutions to the strong $CP$ problem with
anomalous $U(1)$ global symmetries when the hidden QCD is introduced, which
include both the Peccei-Quinn mechanism and dynamical solutions. We find these
two mechanisms can exist together, and a new class of solutions for the strong
CP problem are proposed. Thus we classify two $U(1)$ models in an unified way,
and discuss both one-$U(1)$ and two-$U(1)$ solutions in this work.

A “moose-like” diagram method is used to illustrate how the strong CP problem
is solved via cancellation between CP phases, which could be read directly.
When the gauge interaction and new particles are given, a diagram can be
determined with a breaking mode of $U(1)$ symmetries. From these diagrams,
cancellation between $CP$ phases can be easily read without detailed analysis
of the model. The information is important for solving the strong $CP$
problem. This method could be used to find new types of solutions and easily
extended to cases of multi-$U(1)$ and multi-gauge interaction.

In one-axion solutions, additional $Z_{2}$ symmetry keeps
$\theta=\theta^{\prime}$ to solve the strong $CP$ problem. In two-axion
solutions, two chiral $U(1)$s (either $U(1)_{PQ}$ or $U(1)_{A}$) have to be
introduced. As a consequence, there always exists an invisible light axion and
a heavy axion.

Using the diagram method, we find two new mixed solutions. In Model A, the
spontaneous breaking of $U(1)_{PQ}$ symmetry is triggered by the scalar
potential, independent of the confinement of the hidden QCD, and the axion-
photon couplings $g_{{a_{\text{phy}}}\gamma\gamma}^{UV}$ and the light axion
decay constant $F_{a_{\text{phy}}}$ are both inversely proportional to the
difference of anomalous charges $|m-n|$. In Model B, the $U(1)_{PQ}$ is
induced by the confinement of the hidden QCD, and
$g_{{a_{\text{phy}}}\gamma\gamma}^{UV}$ is independent of the anomaly
coefficient $n$ of $Q^{\prime}$ while $F_{a_{\text{phy}}}$ is inversely
proportional to $|n|$. At low energy scales, it is hard to distinguish this
light axion from the QCD axion, even extending to multi-$U(1)$ symmetries and
multi-gauge interactions. However, this kind of solutions could be probed by
detecting particles at the hidden QCD scale. Those heavy particles may leave
some hints in cosmology and at the colliders.

###### Acknowledgements.

We would like to thank Qing-Hong Cao and Shou-hua Zhu for valuable discussions
and their support. D.H. is supported in part by the National Science
Foundation of China under Grants No. 11635001, 11875072. H.R.J is supported in
part by the National Science Foundation of China under Grant Nos. 11725520,
11675002, 11635001. H.L.L and J.H.Y. are supported by the National Science
Foundation of China (NSFC) under Grants No. 11875003. M.L.X. is supported by
the National Natural Science Foundation of China (NSFC) under grant
No.2019M650856 and the 2019 International Postdoctoral Exchange Fellowship
Program. J.H.Y. is also supported by the National Science Foundation of China
(NSFC) under Grants No. 11947302.

##  Appendix A Dynamical Symmetry Breaking in Hidden Sector

This appendix shows the details of dynamical symmetry breaking in hidden
sector. Two specific examples in Fig. 1(b) and Fig. 2(b) are discussed. The
$su(N)$-algebra is also shown in this appendix.

In dynamical solutions, the global symmetries in the hidden sector are broken
as

$SU(N)_{L}\times SU(N)_{R}\times U(1)_{A}\times U(1)_{V}\rightarrow
SU(N)_{V}\times U(1)_{V},$ (50)

where $U(1)_{A}$ is explicitly broken by instanton effects. The effective
theory at low energy is assumed to be

$\mathcal{L}=\dfrac{f_{\pi}^{2}}{4}\mathrm{Tr}\left(\partial_{\mu}U\partial^{\mu}U^{\dagger}\right)+B^{\prime}f_{\pi}^{4}\left(\theta^{\prime}+\frac{i}{2}\mathrm{Tr}\left(\log
U-\log U^{\dagger}\right)\right)^{2},$ (51)

with

$U=\exp\left[\dfrac{i\left(\pi_{h}^{a}T^{a}+I\eta^{\prime}_{h}\right)}{f_{\pi_{h}}}\right],$
(52)

where $\pi_{h}^{a}$ are Goldstones corresponding to the $SU(N)_{A}$ symmetry
and $\eta^{\prime}_{h}$ is a pseudo Goldstone corresponding to the $U(1)_{A}$
symmetry. $T^{a}$ are basis elements of the $su(N)$-algebra and $I$ is the
$N$-dimension identity matrix. We choose $tr(T^{a}T^{b})=2\delta^{ab}$ as the
normalization condition. Therefore $T^{a}$ are shown as

$\displaystyle T_{1}$ $\displaystyle=\left(\begin{array}[]{ccccc}0&1&0&&0\\\
1&0&0&\vdots&0\\\ 0&0&0&&0\\\ &\cdots&&\ddots&\vdots\\\
0&0&0&\cdots&0\end{array}\right),\quad
T_{2}=\left(\begin{array}[]{ccccc}0&-i&0&&0\\\ i&0&0&\vdots&0\\\ 0&0&0&&0\\\
&\cdots&&\ddots&\vdots\\\ 0&0&0&\cdots&0\end{array}\right),\quad
T_{3}=\left(\begin{array}[]{ccccc}1&0&0&&0\\\ 0&-1&0&\vdots&0\\\ 0&0&0&&0\\\
&\cdots&&\ddots&\vdots\\\ 0&0&0&\cdots&0\end{array}\right),$ (53)
$\displaystyle T_{4}$ $\displaystyle=\left(\begin{array}[]{ccccc}0&0&1&&0\\\
0&0&0&\vdots&0\\\ 1&0&0&&0\\\ &\cdots&&\ddots&\vdots\\\
0&0&0&\cdots&0\end{array}\right),\quad\cdots,\quad\\!T_{N^{2}-1}=\sqrt{\dfrac{2}{N(N-1)}}\left(\begin{array}[]{ccccc}1&0&0&&0\\\
0&1&0&\vdots&0\\\ 0&0&1&&0\\\ &\cdots&&\ddots&\vdots\\\
0&0&0&\cdots&1-N\end{array}\right).$

In general, scalars corresponding to diagonal generators and identity matrix
ought to be mixed. $\eta^{\prime}_{h}$ could be treated as axion when
instanton effects are the only source that explicitly breaks $U(1)_{A}$.
Therefore, scalars corresponding to diagonal generators and identity matrix
are important to solve the strong CP problem. Specific forms of diagonal
generators and identity matrix are

$\displaystyle T_{3}$ $\displaystyle=\mathop{\mathrm{diag}}(1,-1,0,0,\cdots)$
(54) $\displaystyle T_{8}$
$\displaystyle=\dfrac{1}{\sqrt{3}}\mathop{\mathrm{diag}}(1,1,-2,0,\cdots)$
$\displaystyle T_{15}$
$\displaystyle=\dfrac{1}{\sqrt{6}}\mathop{\mathrm{diag}}(1,1,1,-3,\cdots)$
$\displaystyle T_{N^{2}-1}$
$\displaystyle=\sqrt{\dfrac{2}{N(N-1)}}\mathop{\mathrm{diag}}(1,1,1,\cdots,1-N)$
$\displaystyle I$
$\displaystyle=\dfrac{1}{\sqrt{N}}\mathop{\mathrm{diag}}(1,1,1,\cdots,1)$

In the model shown in Fig. 1(b), vectorlike fermions $\psi(3,3)$ and
$\chi(1,3)$ are introduced. In this case,
$\Psi=(\psi_{1}^{C},\psi_{2}^{C},\psi_{3}^{C},\chi^{C})$ has a
$SU(4)_{L}\times SU(4)_{R}$ flavor symmetry in the hidden sector, under which
$\Psi$ transforms as:

$\displaystyle\Psi\to
e^{i(\theta^{L}_{a}T^{a}P_{L}+\theta^{R}_{a}T^{a}P_{R})}\Psi=e^{i(\theta^{V}_{a}T^{a}+\theta^{A}_{a}T^{a}\gamma^{5})}\Psi.$
(55)

For $SU(4)$, diagonal generators are $T_{3},\ T_{8}$ and $T_{15}$. After the
condensate of hidden QCD, $\pi_{h}^{a}$ are Goldstones corresponding to coset
generators of $SU(4)_{L}\times SU(4)_{R}/SU(4)_{V}$. The decomposition of
$\pi_{h}^{a}$ into $SU(3)_{c}$ is

$15=8+3+\bar{3}+1.$ (56)

Colorless scalars are $\pi^{15}_{h}$ and $\eta^{\prime}_{h}$ corresponding to
$\gamma^{5}\otimes T_{15}$ and $\gamma^{5}\otimes I$ respectively. The rest 14
colored scalars are supposed to be heavy because of QCD condensate. Currents
corresponding to these generators and their divergences are

$\displaystyle J_{\mu}(\pi^{15}_{h})$
$\displaystyle=\overline{\Psi}\gamma_{\mu}\gamma_{5}T^{15}\Psi,$ (57)
$\displaystyle J_{\mu}(\eta^{\prime}_{h})$
$\displaystyle=\overline{\Psi}\gamma_{\mu}\gamma_{5}I_{4\times 4}\Psi,$
$\displaystyle\partial_{\mu}J^{\mu}(\pi^{15}_{h})$
$\displaystyle=\dfrac{3}{\sqrt{6}}\frac{g^{\prime
2}}{16\pi^{2}}G_{a}^{\mu\nu}\tilde{G}_{a\mu\nu},$
$\displaystyle\partial_{\mu}J^{\mu}(\eta^{\prime}_{h})$
$\displaystyle=2\sqrt{2}\frac{g^{\prime
2}}{16\pi^{2}}G_{a}^{\mu\nu}\tilde{G}_{a\mu\nu}+\dfrac{3}{\sqrt{2}}\frac{g^{2}}{16\pi^{2}}{G^{\prime}}_{A}^{\mu\nu}\tilde{G^{\prime}}_{A\mu\nu}.$

Matching with these currents, we can get the potential of these composite
axions at low energy as

$V=B_{1}f_{\pi}^{4}\left(\dfrac{3}{\sqrt{6}}\dfrac{\pi_{h}}{f_{\pi_{h}}}+2\sqrt{2}\dfrac{\eta^{\prime}_{h}}{f_{\pi_{h}}}+\theta^{\prime}\right)^{2}+B_{2}f_{\pi_{h}}^{4}\left(\dfrac{3}{\sqrt{2}}\dfrac{\eta^{\prime}_{h}}{f_{\pi_{h}}}+\theta\right)^{2}.$
(58)

At the minimal point of the potential, cancellation equations of $CP$ phases
are

$\begin{array}[]{l}\theta_{phy}=\theta+\dfrac{\sqrt{6}}{2}\dfrac{\left\langle\pi_{h}\right\rangle}{f_{h}}+\dfrac{3\sqrt{2}}{2}\dfrac{\left\langle\eta_{h}^{\prime}\right\rangle}{f_{h}}=0\\\
\theta_{phy}^{\prime}=\theta^{\prime}+2\sqrt{2}\dfrac{\left\langle\eta_{h}^{\prime}\right\rangle}{f_{h}}=0\end{array}$
(59)

In the model shown in Fig. 2(b),The fermion $\psi(3,3)$ is introduced. In this
case, $N=3$, $\Psi=(\psi_{1}^{C},\psi_{2}^{C},\psi_{3}^{C})$. The only
colorless scalar is $\eta^{\prime}_{h}$ corresponding to $\gamma^{5}\otimes
I$. The current and its divergence are

$\displaystyle J_{\mu}(\eta^{\prime}_{h})$
$\displaystyle=\overline{\Psi}\gamma_{\mu}\gamma_{5}I_{3\times 3}\Psi,$ (60)
$\displaystyle\partial_{\mu}J^{\mu}(\eta^{\prime}_{h})$
$\displaystyle=\frac{\sqrt{6}g^{2}}{16\pi^{2}}G_{a}^{\mu\nu}\tilde{G}_{a\mu\nu}+\frac{\sqrt{6}g^{\prime
2}}{16\pi^{2}}{G^{\prime}}_{A}^{\mu\nu}\tilde{G^{\prime}}_{A\mu\nu},$

The corresponding potential of the composite axion at low energy is

$V=B_{1}f_{\pi}^{4}\left(\dfrac{\sqrt{6}\eta^{\prime}_{h}}{f_{\pi_{h}}}+\theta\right)^{2}+B_{2}f_{\pi_{h}}^{4}\left(\dfrac{\sqrt{6}\eta^{\prime}_{h}}{f_{\pi_{h}}}+\theta^{\prime}\right)^{2},$
(61)

At the minimal point of the potential, the cancellation equation of $CP$
phases is

$\theta_{phy}^{\prime}=\theta_{phy}=\theta+\sqrt{6}\dfrac{\left\langle\eta_{h}^{\prime}\right\rangle}{f_{\pi_{h}}}=0.$
(62)

## References

  * (1) G. ‘t Hooft, _Symmetry breaking through bell-jackiw anomalies_ , _Physical Review Letters_ 37 (1976) 8.
  * (2) G. ‘t Hooft, _Computation of the quantum effects due to a four-dimensional pseudoparticle_ , _Physical Review D_ 14 (1976) 3432.
  * (3) C. A. Baker, D. D. Doyle, P. Geltenbort, K. Green, M. G. D. van der Grinten, P. G. Harris et al., _Improved experimental limit on the electric dipole moment of the neutron_ , _Physical Review Letters_ 97 (2006) .
  * (4) A. Nelson, _Naturally weak CP violation_ , _Physics Letters B_ 136 (1984) 387.
  * (5) S. M. Barr, _Solving the strong cp problem without the peccei-quinn symmetry_ , _Physical Review Letters_ 53 (1984) 329.
  * (6) J. Gasser and H. Leutwyler, _Quark masses_ , _Physics Reports_ 87 (1982) 77.
  * (7) R. D. Peccei and H. R. Quinn, _CPConservation in the presence of pseudoparticles_ , _Physical Review Letters_ 38 (1977) 1440.
  * (8) R. D. Peccei and H. R. Quinn, _Constraints imposed by CP conservation in the presence of pseudoparticles_ , _Physical Review D_ 16 (1977) 1791.
  * (9) S. Weinberg, _A new light boson?_ , _Physical Review Letters_ 40 (1978) 223.
  * (10) F. Wilczek, _Problem of StrongPandTInvariance in the presence of instantons_ , _Physical Review Letters_ 40 (1978) 279.
  * (11) W. A. Bardeen, R. Peccei and T. Yanagida, _Constraints on variant axion models_ , _Nuclear Physics B_ 279 (1987) 401.
  * (12) J. E. Kim, _Weak-interaction singlet and StrongCPInvariance_ , _Physical Review Letters_ 43 (1979) 103.
  * (13) M. Shifman, A. Vainshtein and V. Zakharov, _Can confinement ensure natural CP invariance of strong interactions?_ , _Nuclear Physics B_ 166 (1980) 493.
  * (14) M. Dine, W. Fischler and M. Srednicki, _A simple solution to the strong CP problem with a harmless axion_ , _Physics Letters B_ 104 (1981) 199.
  * (15) A. Zhitnitskii, _Possible suppression of axion-hadron interactions_ , _Sov. J. Nucl. Phys.(Engl. Transl.);(United States)_ 31 (1980) .
  * (16) G. G. Raffelt, _Astrophysical axion bounds_ , in _Lecture Notes in Physics_ , pp. 51–71, Springer Berlin Heidelberg, DOI.
  * (17) S. M. Barr and D. Seckel, _Planck-scale corrections to axion models_ , _Physical Review D_ 46 (1992) 539.
  * (18) M. Kamionkowski and J. March-Russell, _Planck-scale physics and the peccei-quinn mechanism_ , _Physics Letters B_ 282 (1992) 137.
  * (19) A. Hook, _Tasi lectures on the strong cp problem and axions_ , http://arxiv.org/abs/1812.02669v1.
  * (20) J. E. Kim, H. P. Nilles and M. Peloso, _Completing natural inflation_ , _Journal of Cosmology and Astroparticle Physics_ 2005 (2005) 005.
  * (21) K. Choi, H. Kim and S. Yun, _Natural inflation with multiple sub-planckian axions_ , 1404.6209v4.
  * (22) T. Higaki, K. S. Jeong, N. Kitajima and F. Takahashi, _Quality of the peccei-quinn symmetry in the aligned qcd axion and cosmological implications_ , _Journal of High Energy Physics_ 2016 (2016)  [http://arxiv.org/abs/1603.02090v2].
  * (23) Q. Bonnefoy, E. Dudas and S. Pokorski, _Axions in a highly protected gauge symmetry model_ , _The European Physical Journal C_ 79 (2019)  [http://arxiv.org/abs/1804.01112v2].
  * (24) H. Fukuda, K. Harigaya, M. Ibe and T. T. Yanagida, _A model of visible qcd axion_ , 1504.06084v2.
  * (25) L. Randall, _Composite axion models and planck scale physics_ , _Physics Letters B_ 284 (1992) 77.
  * (26) M. J. Strassler and K. M. Zurek, _Echoes of a hidden valley at hadron colliders_ , http://arxiv.org/abs/hep-ph/0604261v2.
  * (27) C. Kilic, T. Okui and R. Sundrum, _Vectorlike confinement at the lhc_ , http://arxiv.org/abs/0906.0577v4.
  * (28) Z. Chacko, H.-S. Goh and R. Harnik, _Natural electroweak breaking from a mirror symmetry_ , _Physical Review Letters_ 96 (2006) .
  * (29) S. Weinberg, _Larger higgs-boson-exchange terms in the neutron electric dipole moment_ , _Physical Review Letters_ 63 (1989) 2333.
  * (30) M. Pospelov and A. Ritz, _Electric dipole moments as probes of new physics_ , http://arxiv.org/abs/hep-ph/0504231v2.
  * (31) V. A. Rubakov, _Grand unification and heavy axion_ , _Journal of Experimental and Theoretical Physics Letters_ 65 (1997) 621 [hep-ph/9703409v2].
  * (32) Z. Berezhiani, L. Gianfagna and M. Giannotti, _Strong CP problem and mirror world: the weinberg–wilczek axion revisited_ , _Physics Letters B_ 500 (2001) 286.
  * (33) A. Albaid, M. Dine and P. Draper, _Strong cp and suz 2_, _Journal of High Energy Physics_ (2015) 1 [1510.03392v2].
  * (34) A. Hook, _Anomalous solutions to the strong CP problem_ , _Physical Review Letters_ 114 (2015)  [1411.3325v2].
  * (35) C.-W. Chiang, H. Fukuda, M. Ibe and T. T. Yanagida, _750 gev diphoton resonance in a visible heavy qcd axion model_ , 1602.07909v3.
  * (36) K. Choi and J. E. Kim, _Dynamical axion_ , _Physical Review D_ 32 (1985) 1828.
  * (37) T. Gherghetta, N. Nagata and M. Shifman, _A visible qcd axion from an enlarged color group_ , 1604.01127v3.
  * (38) P. Agrawal, J. Fan, M. Reece and L.-T. Wang, _Experimental targets for photon couplings of the qcd axion_ , 1709.06085v2.
  * (39) P. Sikivie, _Axions, domain walls, and the early universe_ , _Physical Review Letters_ 48 (1982) 1156.
  * (40) R. Kappl, S. Krippendorf and H. P. Nilles, _Aligned natural inflation: Monodromies of two axions_ , _Physics Letters B_ 737 (2014) 124 [1404.7127v3].
  * (41) M. Kuster, G. Raffelt and B. Beltrán, eds., _Axions_. Springer Berlin Heidelberg, 2008, 10.1007/978-3-540-73518-2.
  * (42) G. G. di Cortona, E. Hardy, J. P. Vega and G. Villadoro, _The QCD axion, precisely_ , _Journal of High Energy Physics_ 2016 (2016)  [1511.02867v2].


# Deblurring Processor for Motion-Blurred Faces Based on Generative
Adversarial Networks

Shiqing Fan
School of Informatics
Xiamen University
Xiamen, Fujian 361005, China
<EMAIL_ADDRESS>
Ye Luo
School of Informatics
Xiamen University
Xiamen, Fujian 361005, China
<EMAIL_ADDRESS><EMAIL_ADDRESS>

###### Abstract

Low-quality face image restoration is a popular research direction in today’s
computer vision field. It can be used as a pre-work for tasks such as face
detection and face recognition. At present, there is a lot of work to solve
the problem of low-quality faces under various environmental conditions. This
paper mainly focuses on the restoration of motion-blurred faces. In
increasingly abundant mobile scenes, the fast recovery of motion-blurred faces
can bring highly effective speed improvements in tasks such as face matching.
In order to achieve this goal, a deblurring method for motion-blurred facial
image signals based on generative adversarial networks(GANs) is proposed. It
uses an end-to-end method to train a sharp image generator, i.e., a processor
for motion-blurred facial images. This paper introduce the processing progress
of motion-blurred images, the development and changes of GANs and some basic
concepts. After that, it give the details of network structure and training
optimization design of the image processor. Then we conducted a motion blur
image generation experiment on some general facial data set, and used the
pairs of blurred and sharp face image data to perform the training and testing
experiments of the processor GAN, and gave some visual displays. Finally,
MTCNN is used to detect the faces of the image generated by the deblurring
processor, and compare it with the result of the blurred image. From the
results, the processing effect of the deblurring processor on the motion-
blurred picture has a significant improvement both in terms of intuition and
evaluation indicators of face detection.

CCS CONCEPTS · Computing methodologies · Artificial intelligence · Computer
vision
Keywords— Generative adversarial networks, Image processing, Face detection

##  1 Introduction

###  1.1 Motivation

Face detection[1] and face recognition [2] are among the most popular and
widely used tasks in the field of computer vision. In practical applications,
data sets are often affected by changes in the surrounding environment and
scenes, and have different characteristics. Generally speaking, it frequently
happens that the raw face images are not ideal for detection, such as images
with some missing parts, of low resolution, or being fogged or blurred. There
have already been a lot of researchers proposing much cutting-edge work to
help solve problems related to the detection and recognition of low-quality
face data. For instance, the work of Yeh _et al._ focused on repairing
defective images [3], and Chen _et al._ presented an approach about face image
super-resolution [4]. There is also some research work about low-resolution
face recognition [5], single image dehazing [6] etc. Under the current
increasing demand for mobile face recognition, the detection and recognition
of blurred facial images, especially of motion-blurred facial images, have
become highly significant. The purpose of this paper is to present a picture-
processing algorithm applied to motion-blurred face pictures, providing an
enhanced detectability for face detection.

After Generative Adversarial Nets (GANs) was proposed by Goodfellow _et al._ ,
it has become an exceedingly popular model in deep learning, especially in the
field of image generation. These image-generating models based on GAN can
afford very natural output, such as face generation [7], face aging [8], image
style transfer [9], and face beautification [10]. It can be said that GAN
generative models have reached the level of being fake but not seen through.

Applying the GAN model to image processing is also an interesting idea.
Traditional generative models get different outputs from noise, while GAN in
image processing uses defective pictures in the real world as input, and
builds a generator, though which excellent pictures can be obtained. This end-
to-end model is very intuitive, a generator that can transform defective
pictures to excellent pictures is trained to be an image processor for some
pictures that are difficult to be processed by traditional algorithms to.
Therefore, the goal of this article is to apply the GAN structure to the
motion-blurred face image processing to form a GAN-based motion-blurred face
images processor, and conduct a follow-up face detection experimental
comparison test on the effectiveness of the processor.

###  1.2 Contribution

In this paper, some changes are made on the basis of the traditional GAN
structure and applied to the deblurring processing of the motion-blurred
images of human faces. First, we perform motion blur simulation processing on
some public data set of face pictures to obtain pairs of motion-blurred and
sharp face images, and then we design the generator in the GAN structure to
use motion-blurred faces as input and sharp faces as output. The discriminator
in the network judges the generated images and the sharp images separately.
The generated image and the sharp images have pseudo image tags and real image
tags respectively. We used several different GAN loss function designs for
specific training. After training, the generator part in our network is the
face deblur processor we ultimately need. Finally, we use this processor to
test the image processing of face images, and conduct comparative experimental
statistics for face detection to test the capabilities of the processor. We
found that the face images through the processor has a significant improvement
in the success rate of face detection and the positioning accuracy of the
position of faces and also the facial features.

We introduced the related work of generative adversarial networks and
deblurring technology for image motion blur in Section 2, and introduced some
formulaic basic knowledge, including GAN, motion blur and face detection, in
Section 3. In Section 4, we give an intuitive explanation and detailed
introduction of the network used in this article. Section 5 is about the whole
experimental process and effect demonstration. Finally, we give a summary of
the paper.

##  2 Related Work

###  2.1 Image Deblurring

The previous early work on the processing of blurred images was mainly non-
blind, meaning that the cause of the blur and the specific effect and degree
of the blur are known and formulaic expressions can be studied, that is, the
blur kernel k in the following Equation 2.1 is known.

######  Definition 2.1.

The common blur model can be formulated as

$I_{B}=k(M)*I_{S}+N$

where $I_{B}$ is a blurred image, $I_{S}$ is a sharp image, $k(M)$ are unknown
blur kernels determined by motion field $M$. $N$ is other noise.

The formula indicates that the blurred image is formed by the convolution of
the blur kernel and the clear image with some noise added. There has been a
lot of research work on fuzzy image processing with prior knowledge, that is,
the knowledge about the blur kernels. For example, some work relying on the
classic Lucy-Richardson algorithm, obtaining $I_{S}$ estimates by performing
deconvolution operations with Wiener or Tikhonov filters. Starting from the
success of Fergus _et al._ [11], many methods have been extended in recent
years [12] [13][14][15]. These methods assume that the blur is uniform
throughout the image and try to compensate for the blur caused by camera
shake. They first estimate the motion of the camera based on the blur kernel
caused by the scene, and then reverse the effect by performing a deconvolution
operation. With the success of deep learning, in the past few years, some
methods based on Convolutional Neural Networks (CNNs) have emerged [16][17].
All these methods use CNNs to estimate the unknown ambiguity function. Later,
some kernelless end-to-end methods appeared. Noorozi _et al._ [18] and Nah _et
al._ [19] used multi-scale CNNs to directly blur the images. Ramakrishnan _et
al._ [20] uses a combination of the pix2pix framework [9] and a densely
connected convolutional network [21] to perform kernel-free blind image
deblurring. The advantage of these new methods is that they can handle
different sources of blur. And Kupyn _et al._ [22] even pioneered the use of
Generative Adversarial Networks(GAN) structure for image deblurring to help
object detection.

###  2.2 Generative Adversarial Networks

GAN was proposed by Goodfellow _et al._[7]. It uses the game learning of
generative network and adversarial network to finally produce excellent
generative output, which has great influence in unsupervised learning of
complex distribution. Two models, generator and discriminator, are used in the
GAN framework to train at the same time, similar to the relationship between a
spear and a shield. During the training process, the discriminator’s
discriminatory ability is getting stronger and stronger, and at the same time,
the generator’s forgery ability also getting stronger in order to fool the
discriminator. In an ideal state, the final result of the game is when the
discriminator’s judgment result of the generator’s forged output with a
probability of 0.5 is true, which indicates that the generator is sufficient
to produce a image that cannot be distinguished from the real image. Then the
generative model can be used to other downstream tasks. After the basic GAN
model was proposed, many improvements to the details of GAN were also
proposed. Mao _et al._ [23] proposed a least squares generative adversarial
networks(LSGANs), using the least squares loss function in the discriminator,
the purpose is to solve the problem of gradient disappearance that may exist
in traditional GANs to generate more stable and high-quality images. The
Wasserstein GAN(WGAN) proposed by Arjovsky _et al._ [24] focuses on the
balance between generator and discriminator. For example, once discriminator
is trained too well, generator cannot learn effective gradients, and WGAN
truncates the part of discriminator when updating parameters, that is,
”gradient clipping”. What’s more, their later work [25] improved the model to
use gradient penalty.

##  3 BASIC CONCEPTS

In this section, we introduce several GAN network ideas in formulaic form, and
give the specific form of motion blur discussed in this paper, and finally
give some basic concepts of face detection, which provides the basic knowledge
beforehand for our GAN-based motion blur facial image processor.

###  3.1 GAN Loss

In the general GAN structure, we train two models at the same time, one is the
generator $G$ and the other is the discriminator $D$. In general tasks, we
will have some real data $\boldsymbol{x}$, and some network inputs, such as
random noise, set as $\boldsymbol{z}$. The output of $\boldsymbol{z}$ through
G in the network is $G(\boldsymbol{z})$. The main task of G is to learn the
distribution $p_{\mathrm{data\ }}(\boldsymbol{x})$ of data $\boldsymbol{x}$,
its output $G(\boldsymbol{z})$ should be as close to $\boldsymbol{x}$ as
possible. On the other hand, the main task of the D is to distinguish between
$\boldsymbol{x}$ and $\boldsymbol{z}$, that is, to accurately determine that
$\boldsymbol{x}$ is real data and $\boldsymbol{z}$ is forged data. From the
above ideas, we can see that the goal of traditional GAN can be formulated as
Definition 3.1 . Definition 3.2 shows the form of the LSGANs, which is similar
to the traditional one, but with the cross entropy in the traditional GAN
changed to the least square loss. As for the WGANs, the Earth-Mover (also
called Wasserstein-1) distance $W(q,p)$ is used instead, in order to transform
the distribution $q$ into the distribution $p$. With reference to Kantorovich-
Rubinstein duality [26], we get the WGAN value function of Definition 3.3(i),
and then the improved version WGAN-GP taking into account gradient penalty is
shown in Definition 3.3(ii).

######  Definition 3.1.

The original GANs

$\min_{G}\max_{D}V(D,G)=\mathbb{E}_{x\sim p_{\text{data }}(x)}[\log
D(x)]+\mathbb{E}_{z\sim p_{z}(z)}[\log(1-D(G(z)))]$

######  Definition 3.2.

LSGANs [23]

$\begin{array}[]{l}\min_{D}V_{LSGAN}(D)=\frac{1}{2}\mathbb{E}_{\boldsymbol{x}\sim
p_{\text{data
}}(\boldsymbol{x})}\left[(D(\boldsymbol{x})-b)^{2}\right]+\frac{1}{2}\mathbb{E}_{\boldsymbol{Z}\sim
p_{\boldsymbol{z}}(\boldsymbol{z})}\left[(D(G(\boldsymbol{z}))-a)^{2}\right]\\\
\min_{G}V_{LSGAN}(G)=\frac{1}{2}\mathbb{E}_{\boldsymbol{Z}\sim
p_{\boldsymbol{z}}(\boldsymbol{z})}\left[(D(G(\boldsymbol{z}))-c)^{2}\right]\end{array}$

where a and b are the labels for fake data and real data, respectively. c
denotes the value that G wants D to believe for fake data.

######  Definition 3.3.

Wasserstein GANs

  1. (i)

WGAN [24]

$\operatorname{minmax}_{G}\underset{D\in\mathcal{D}}{\mathbb{E}}_{\boldsymbol{x}\sim\mathbb{P}_{r}}[D(\boldsymbol{x})]-\underset{\widetilde{\boldsymbol{x}}\sim\mathbb{P}_{g}}{\mathbb{E}}[D(\widetilde{\boldsymbol{x}})]$

where $\widetilde{\boldsymbol{x}}=G(z)$, $\mathbb{P}_{r}$ and $\mathbb{P}_{g}$
respectively represent the distribution of $\boldsymbol{x}$ and
$\widetilde{\boldsymbol{x}}$, and $\mathcal{D}$ is the set of 1-Lipschitz
functions. The purpose of the equation is to minimize
$W(\mathbb{P}_{r},\mathbb{P}_{g})$.

  2. (ii)

WGAN-GP [25]

$L=\underbrace{\underset{\tilde{\boldsymbol{x}}\sim\mathbb{P}_{g}}{\mathbb{E}}[D(\widetilde{\boldsymbol{x}})]-\underset{\boldsymbol{x}\sim\mathbb{P}_{r}}{\mathbb{E}}[D(\boldsymbol{x})]}_{\text{Original
loss
}}+\underbrace{\lambda_{\widehat{\boldsymbol{x}}\sim\mathbb{P}_{\hat{\boldsymbol{x}}}}\mathbb{E}\left[\left(\left\|\nabla_{\widehat{\boldsymbol{x}}}D(\widehat{\boldsymbol{x}})\right\|_{2}-1\right)^{2}\right]}_{\text{Gradient
penalty }}$

where $\hat{\boldsymbol{x}}\sim\mathbb{P}_{\hat{\boldsymbol{x}}}$ is random
samples whose gradient will be punished.

###  3.2 Motion-Blurred Image Generation

We use the blur kernels generation algorithm proposed by [22] to get the
blurred face image data. This method refers to the random trajectory
generation idea of Boracchi _et al._ [27]. After generating the trajectory,
sub-pixel interpolation is applied to the trajectory vector to generate a blur
kernel. This method can simulate a more realistic and complex blur kernel.
This is summarized in the Algorithm 1. In order to test the effect of our
deblurring processor, we set a relatively high level of motion blur.

Algorithm 1 Motion Blur Image Generation

Initialize the velocity vector of the trajectory points(1);

Initialize the position of trajectory points to zeros;

For the number of iterations, do

Generate the velocity of the next moment(2);

Get the position of the next point according to the current speed;

End

Get the trajectory vector x;

Generate blur kernel by sub pixel interpolation(x);

Get Blur image by convolution operation of blur kernels and sharp images.

_(1) According to the previously set maximum movement length and number of
iterations._

_(2)Based on the previous point velocity and position, gaussian perturbation,
impulse perturbation and_

_deterministic inertial component._

###  3.3 Face Detection

Face Detection is to find out the position of all faces in an image. It is
usually framed by a rectangular frame. The input is an image, and the output
is a number of rectangular frame positions like $(x,y,w,h)$ containing faces.

The evaluation indicators for face detection include the following:

_Recall Rate_ : Since the number of faces contained in each image is
uncertain, it is measured by the ratio of detection. The closer the
rectangular frame detected by the detector is to the manually labeled
rectangular frame, the better the detection result. Usually, if the
intersection ratio is greater than 0.5, it is considered to be detected. So
recall = (the number of detected faces)/(the total number of faces in the
image).

_False Positives_ : We use the absolute number of detection errors to express.
In contrast to recall, if the IoU(Intersection over Union) of the rectangular
box detected by the detector and any manually labeled box is less than 0.5,
the detection result is considered to be a false detection.

_Detection Speed_ : Usually expressed by frame-per-second (FPS).

##  4 PROCESSOR FOR BLURRED IMAGES OF HUMAN FACES

###  4.1 Processor Network Structure

Our motion face deblurring processor model is shown in Figure 1. It should be
noted that this structure is our final processor model, which is part of the
generator in the GAN structure. The generator is composed of an up-sampling
part including three convolution blocks, a ResNet [28] part including 6 ResNet
blocks, and a down-sampling part including 2 transposed convolution blocks and
the final output convolution block.

Figure 1: Deblurring Processor for Motion-Blurred Faces(Generator of GAN)

###  4.2 Model Training

As we mentioned above, our GAN structure includes a generator and a
discriminator, and they need to be trained at the same time. When training,
the input is a pair of blurred and sharp images, model gets the restored
images from generator, and both the restored images and the sharp images are
input to the discriminator. For the generator, the restored picture is marked
as the real picture for the loss calculation. Meanwhile the training of the
discriminator uses the calculated value obtained by marking the restored
images as fake and the sharp images as real as the loss, so as to achieve the
game effect of G and D. We use three GAN types at the same time, that is,
three training loss calculation methods mentioned in Subsection 3.1 for
experiments.

Specifically, as shown in Figure 2, the training loss function of the
generator G consists of two parts, one part is the result of the restored
images G(z) generated by the generator after passing the discriminator, and
the loss calculated by the label REAL, which is used to improve the forgery
ability of the generator. The other part is the content loss between restored
images $\boldsymbol{z}$ and sharp images $\boldsymbol{x}$, they first pass
through a content function(similar to VGG network[29]) and then calculate the
difference, which is also to improve the generator’s ability to generate
images that are more similar to sharp images. While the loss function of the
discriminator is calculated from the discriminant results of $\boldsymbol{x}$
and $G(\boldsymbol{z})$, where the discriminant label of the picture is real.

Figure 2: Training a GAN for motion blur face image processing.

##  5 EXPERIMENT DISPLAY

###  5.1 Motion-Blurred Face Data Set

We used the LFW (Labeled Faces in the Wild Home) [30] dataset for experiments.
Figure 3 shows some examples of motion-blurred face images obtained by
Algorithm 1.

Figure 3: Display of some sharp images and motion-blurred images of human
faces.

###  5.2 Image Processing and Face Detection

Figure 4: Deblurring processor under different GAN types help improve face
detection results.

We implemented three deblurring models under different types of GANs on
PyTorch. The training used 1000 image pairs generated by the motion blur
generation algorithm from the face images in LFW dataset, and we uniformly
crop and resize the images to 256x256 during training. After the training is
completed, we use other blurred face images that are also generated for the
motion blur algorithm to perform deblurring tests, and put all sharp images,
motion-blurred images, restored images into MTCNN [31] for face detection
comparison. From some of the results in Figure 4, we can see that the
deblurring image processor not only restores the blur caused by motion
visually, but also effectively helps MTCNN detect faces and the location of
key-points.

As for the detailed face detection results, we conducted experiments on
another 1,000 motion-blurred face image test sets under MTCNN, and compared
the processed images with the unprocessed images as shown in the Table 1. The
detection failure rate of pictures after the deblurring processor has dropped
a lot under all the 3 types, which means that the recall rate has increased
significantly. What’s more, the detection confidence of the faces is also
increased.

Table 1: Motion-blurred faces detection results

Processor GAN Type Detection Failure Rate Detection Confidence Mean GAN 7.36%
0.9918 LSGAN 6.61% 0.9916 WGAN-GP 6.23% 0.9908 Untreated blurred images 21.9%
0.9729 Sharp images 3.23% 0.9945

##  6 Conclusion

This paper introduced an image processor for motion-blurred faces. It uses GAN
structure, which can adapt well to blurred images without prior knowledge.
From the face detection results of the processed face images, this kind of
processor can also help face images perform more downstream tasks. In future
work, we hope to improve the network structure and perform more targeted
processing on face images. For example, when optimizing the network generator,
we will consider adding the feature extraction and alignment of key points of
the faces to the optimization goal to further improve the naturalness and
restoration of the images generated by the GAN.

## References

  * [1] Henry A Rowley, Shumeet Baluja, and Takeo Kanade.  Neural network-based face detection.  IEEE Transactions on pattern analysis and machine intelligence, 20(1):23–38, 1998.
  * [2] Wenyi Zhao, Rama Chellappa, P Jonathon Phillips, and Azriel Rosenfeld.  Face recognition: A literature survey.  ACM computing surveys (CSUR), 35(4):399–458, 2003.
  * [3] Raymond A Yeh, Chen Chen, Teck Yian Lim, Alexander G Schwing, Mark Hasegawa-Johnson, and Minh N Do.  Semantic image inpainting with deep generative models.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 5485–5493, 2017.
  * [4] Yu Chen, Ying Tai, Xiaoming Liu, Chunhua Shen, and Jian Yang.  Fsrnet: End-to-end learning face super-resolution with facial priors.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2492–2501, 2018.
  * [5] Wilman WW Zou and Pong C Yuen.  Very low resolution face recognition problem.  IEEE Transactions on image processing, 21(1):327–340, 2011.
  * [6] Dong Yang and Jian Sun.  Proximal dehaze-net: A prior learning-based deep network for single image dehazing.  In Proceedings of the European Conference on Computer Vision (ECCV), pages 702–717, 2018.
  * [7] I Goodfellow, J Pouget-Abadie, M Mirza, B Xu, D Warde-Farley, S Ozair, A Courville, and Y Bengio.  Generative adversarial nets [c]//advances in neural information processing systems.  New York: ACM, 26722680, 2014.
  * [8] Grigory Antipov, Moez Baccouche, and Jean-Luc Dugelay.  Face aging with conditional generative adversarial networks.  In 2017 IEEE international conference on image processing (ICIP), pages 2089–2093. IEEE, 2017.
  * [9] Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A Efros.  Image-to-image translation with conditional adversarial networks.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1125–1134, 2017.
  * [10] Tingting Li, Ruihe Qian, Chao Dong, Si Liu, Qiong Yan, Wenwu Zhu, and Liang Lin.  Beautygan: Instance-level facial makeup transfer with deep generative adversarial network.  In Proceedings of the 26th ACM international conference on Multimedia, pages 645–653, 2018.
  * [11] Rob Fergus, Barun Singh, Aaron Hertzmann, Sam T Roweis, and William T Freeman.  Removing camera shake from a single photograph.  In ACM SIGGRAPH 2006 Papers, pages 787–794. 2006.
  * [12] Li Xu, Shicheng Zheng, and Jiaya Jia.  Unnatural l0 sparse representation for natural image deblurring.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1107–1114, 2013.
  * [13] Li Xu and Jiaya Jia.  Two-phase kernel estimation for robust motion deblurring.  In European conference on computer vision, pages 157–170. Springer, 2010.
  * [14] Daniele Perrone and Paolo Favaro.  Total variation blind deconvolution: The devil is in the details.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2909–2916, 2014.
  * [15] S Derin Babacan, Rafael Molina, Minh N Do, and Aggelos K Katsaggelos.  Bayesian blind deconvolution with general sparse image priors.  In European conference on computer vision, pages 341–355. Springer, 2012.
  * [16] Dong Gong, Jie Yang, Lingqiao Liu, Yanning Zhang, Ian Reid, Chunhua Shen, Anton Van Den Hengel, and Qinfeng Shi.  From motion blur to motion flow: A deep learning solution for removing heterogeneous motion blur.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2319–2328, 2017.
  * [17] Jian Sun, Wenfei Cao, Zongben Xu, and Jean Ponce.  Learning a convolutional neural network for non-uniform motion blur removal.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 769–777, 2015.
  * [18] Mehdi Noroozi, Paramanand Chandramouli, and Paolo Favaro.  Motion deblurring in the wild.  In German conference on pattern recognition, pages 65–77. Springer, 2017.
  * [19] Seungjun Nah, Tae Hyun Kim, and Kyoung Mu Lee.  Deep multi-scale convolutional neural network for dynamic scene deblurring.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3883–3891, 2017.
  * [20] Sainandan Ramakrishnan, Shubham Pachori, Aalok Gangopadhyay, and Shanmuganathan Raman.  Deep generative filter for motion deblurring.  In Proceedings of the IEEE International Conference on Computer Vision Workshops, pages 2993–3000, 2017.
  * [21] Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger.  Densely connected convolutional networks.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4700–4708, 2017.
  * [22] Orest Kupyn, Volodymyr Budzan, Mykola Mykhailych, Dmytro Mishkin, and Jiří Matas.  Deblurgan: Blind motion deblurring using conditional adversarial networks.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 8183–8192, 2018.
  * [23] Xudong Mao, Qing Li, Haoran Xie, Raymond YK Lau, Zhen Wang, and Stephen Paul Smolley.  Least squares generative adversarial networks.  In Proceedings of the IEEE international conference on computer vision, pages 2794–2802, 2017.
  * [24] Martin Arjovsky, Soumith Chintala, and Léon Bottou.  Wasserstein generative adversarial networks.  In International conference on machine learning, pages 214–223. PMLR, 2017.
  * [25] Ishaan Gulrajani, Faruk Ahmed, Martin Arjovsky, Vincent Dumoulin, and Aaron Courville.  Improved training of wasserstein gans.  arXiv preprint arXiv:1704.00028, 2017.
  * [26] Cédric Villani.  Optimal transport: old and new, volume 338.  Springer Science & Business Media, 2008.
  * [27] Giacomo Boracchi and Alessandro Foi.  Modeling the performance of image restoration from motion blur.  IEEE Transactions on Image Processing, 21(8):3502–3517, 2012.
  * [28] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Deep residual learning for image recognition.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016.
  * [29] Karen Simonyan and Andrew Zisserman.  Very deep convolutional networks for large-scale image recognition.  arXiv preprint arXiv:1409.1556, 2014.
  * [30] Gary B Huang, Marwan Mattar, Tamara Berg, and Eric Learned-Miller.  Labeled faces in the wild: A database forstudying face recognition in unconstrained environments.  In Workshop on faces in’Real-Life’Images: detection, alignment, and recognition, 2008.
  * [31] Kaipeng Zhang, Zhanpeng Zhang, Zhifeng Li, and Yu Qiao.  Joint face detection and alignment using multitask cascaded convolutional networks.  IEEE Signal Processing Letters, 23(10):1499–1503, 2016.