#  EPHOU-21-010 KEK-TH-2333 Symplectic modular symmetry in heterotic string
vacua:
flavor, CP, and $R$-symmetries

Keiya Ishiguroa*** E-mail address<EMAIL_ADDRESS>, Tatsuo Kobayashib†††
E-mail address<EMAIL_ADDRESS>and Hajime Otsukac‡‡‡
E-mail address<EMAIL_ADDRESS>
aGraduate University for Advanced Studies (Sokendai),
1-1 Oho, Tsukuba, Ibaraki 305-0801, Japan.
bDepartment of Physics, Hokkaido University, Sapporo 060-0810, Japan
cKEK Theory Center, Institute of Particle and Nuclear Studies, KEK,
1-1 Oho, Tsukuba, Ibaraki 305-0801, Japan

Abstract

We examine a common origin of four-dimensional flavor, CP, and $U(1)_{R}$
symmetries in the context of heterotic string theory with standard embedding.
We find that flavor and $U(1)_{R}$ symmetries are unified into the
$Sp(2h+2,\mathbb{C})$ modular symmetries of Calabi-Yau threefolds with $h$
being the number of moduli fields. Together with the $\mathbb{Z}_{2}^{\rm CP}$
CP symmetry, they are enhanced to $GSp(2h+2,\mathbb{C})\simeq
Sp(2h+2,\mathbb{C})\rtimes\mathbb{Z}_{2}^{\rm CP}$ generalized symplectic
modular symmetry. We exemplify the $S_{3},S_{4},T^{\prime},S_{9}$ non-Abelian
flavor symmetries on explicit toroidal orbifolds with and without resolutions
and $\mathbb{Z}_{2},S_{4}$ flavor symmetries on three-parameter examples of
Calabi-Yau threefolds. Thus, non-trivial flavor symmetries appear in not only
the exact orbifold limit but also a certain class of Calabi-Yau threefolds.
These flavor symmetries are further enlarged to non-Abelian discrete groups by
the CP symmetry.

###### Contents

  1. 1 Introduction
  2. 2 Generalized symplectic modular symmetry
    1. 2.1 Symplectic structure of moduli fields in Calabi-Yau compactifications
    2. 2.2 Symplectic modular flavor symmetry of matter fields
    3. 2.3 $U(1)_{R}$ symmetry
    4. 2.4 Unification of symplectic modular flavor, CP, and $R$-symmetries
  3. 3 Examples
    1. 3.1 6D toroidal orbifolds
    2. 3.2 Small resolutions of $T^{6}/\mathbb{Z}_{3}$ orbifold
    3. 3.3 Small resolutions of $T^{6}/(\mathbb{Z}_{3}\times\mathbb{Z}_{3}^{\prime})$ orbifold
    4. 3.4 Geometrical regime of blown-up $T^{6}/(\mathbb{Z}_{3}\times\mathbb{Z}_{3}^{\prime})$ orbifold
    5. 3.5 Three-parameter examples of Calabi-Yau threefolds
  4. 4 Conclusions and discussions
  5. A Modular form of weight 1 for $T^{\prime}$

##  1 Introduction

The origin of flavor and CP symmetries is one of the unsolved and fundamental
issues in the Standard Model. Understanding the underlying structure would
give us a clue to find out the ultra-violet completion of the Standard Model.
It was proposed in the pioneering works [1] that these symmetries are closely
connected with the geometrical symmetry of extra-dimensional spaces. In
particular, $SL(2,\mathbb{Z})$ modular symmetry is of particular interest in
the bottom-up approach because the non-Abelian discrete flavor symmetries,
which are used to explain flavor structure of quarks and leptons such as
$A_{4}$, $S_{4}$, $A_{5}$ in the past decades, are naturally embedded into the
$SL(2,\mathbb{Z})$, called the modular flavor symmetry [2]. The inclusion of
the CP symmetry enlarges $SL(2,\mathbb{Z})$ to the generalized modular group
$GL(2,\mathbb{Z})$ [3, 4].

From the viewpoint of top-down approach such as the string theory, the
$SL(2,\mathbb{Z})$ symmetry appears in the low-energy effective action from
the higher-dimensional theories on toroidal backgrounds [5, 6] 111 Also, the
modular symmetry, in particular, its anomaly was studied in Refs. [7, 8, 9]..
Furthermore, the flavor symmetry can be a quotient of $SL(2,\mathbb{Z})$, and
CP symmetry can be regarded as an outer automorphism of $SL(2,\mathbb{Z})$.
They can be treated in a unified manner [10]. The stabilization mechanism of
moduli values providing observed masses and mixing angles of quarks/leptons is
proposed in the context of string theory [11, 12, 13]. Hence, both flavor and
CP symmetries have a common origin in the geometrical symmetries of toroidal
backgrounds, and this approach would be useful to reveal the nature of extra-
dimensional spaces relevant to particle physics.

In this paper, we examine the existence of modular flavor symmetry on Calabi-
Yau (CY) threefolds which are promising backgrounds of the string theory. The
complex structure and Kähler moduli spaces of CY threefolds are described by
the symplectic structure with $Sp(2h+2,\mathbb{Z})$ symplectic groups.222We
refer the details about the symplectic modular symmetry to Refs. [14, 15].
Here, $h$ is determined by the number of the complex structure moduli
$h=h^{2,1}$ or the Kähler moduli $h=h^{1,1}$. Such a symplectic extension was
recently discussed in Refs. [16, 13, 17, 18] in both bottom-up and top-down
approaches, and these flavor symmetries are called the symplectic modular
flavor symmetries. Since moduli fields and matter fields are in one-to-one
correspondence with each other in the context of heterotic string theory with
standard embedding, the flavor symmetries of matter fields naturally lie in
symplectic modular flavor symmetries. In particular, it turned out that four-
dimensional (4D) CP symmetry was regarded as the outer automorphism of
$Sp(2h+2,\mathbb{Z})$ on CY backgrounds [13]. These observations motivate us
to examine the symplectic modular symmetry involving the flavor and CP
symmetries in the context of CY symplectic moduli spaces.

The purpose of this paper is to formulate the symplectic modular flavor
symmetries together with CP symmetry in the context of heterotic string theory
with standard embedding and exemplify these symmetries on several toroidal and
CY backgrounds. As discussed in detail in Sec. 2, the flavor symmetries of
fundamental and anti-fundamental representations of $E_{6}$ gauge group are
governed by the symplectic modular symmetries in a general class of CY
threefolds with large volume/complex structure, although the flavor structure
depends on the intersection number of CY threefolds. Together with the results
of Ref. [13], we find that symplectic modular flavor symmetries are further
enlarged to $GSp(2h+2,\mathbb{Z})\simeq
Sp(2h+2,\mathbb{Z})\rtimes\mathbb{Z}_{2}^{\rm CP}$, called “generalized
symplectic modular symmetry”. Furthermore, the rotation of holomorphic three-
form of CY threefolds induces that of Killing spinors of 4D supersymmetry,
namely the $U(1)_{R}$ symmetry [19]. As a result, the flavor and the
$U(1)_{R}$ symmetries are described by $Sp(2h+2,\mathbb{C})$ symplectic
modular symmetry, which results in the unification of all flavor, CP, and
$U(1)_{R}$ symmetries in the context of $GSp(2h+2,\mathbb{C})$ symmetry. We
explicitly demonstrate generalized symplectic modular symmetries on toroidal
orbifolds with/without resolutions and three-parameter examples of CY
threefolds. It turns out that there exist $G_{\rm
flavor}=\mathbb{Z}_{2},S_{3},S_{4},T^{\prime},S_{9}$ non-Abelian flavor
symmetries, and they are further enhanced into $G_{\rm
flavor}\rtimes\mathbb{Z}_{2}^{\rm CP}$ together with the CP symmetry. So far,
several techniques have been developed to understand $G_{\rm flavor}$ on
toroidal orbifolds, but our approach is useful to find out the flavor
structure of smooth CY threefolds, including the known toroidal orbifold
regime. Indeed, our results indicate that a certain class of CY threefolds has
the $S_{4}$ flavor symmetry on matter fields associated with the moduli
fields.

This paper is organized as follows. In Sec. 2, we briefly review the
symplectic structure of moduli fields in CY compactifications and propose the
generalized symplectic modular symmetries of matter fields in the context of
heterotic string theory with standard embedding, where fundamental and anti-
fundamental representations of $E_{6}$ group are directly related to Kähler
moduli and complex structure moduli, respectively. The symplectic modular
flavor symmetries and their enlarged symmetries by the CP transformation are
explicitly demonstrated in Sec. 3. Sec. 4 is devoted to conclusions and
discussions. In Appendix A, we give the modular form of weight 1 for
$T^{\prime}$.

##  2 Generalized symplectic modular symmetry

In Sec. 2.1, we review the symplectic structure of moduli fields in CY
compactifications with an emphasis on the large volume/complex structure
regime. In Sec. 2.2, we give a prescription of the symplectic modular flavor
symmetry of matter fields associated with the complex structure moduli and
Kähler moduli in the context of heterotic string theory with standard
embedding. Furthermore, the symplectic transformations of matter fields
naturally incorporate the existence of $U(1)_{R}$ symmetry appearing in the 4D
effective action as shown in Sec. 2.3. The CP transformations exchanging the
fundamental and anti-fundamental representations of $E_{6}$ are regarded as
the anti-holomorphic transformations of moduli fields. In Sec. 2.4, we find
that these anti-holomorphic transformations enlarge the symplectic modular
symmetry into the generalized symplectic modular symmetry. As a result, the
symplectic modular, flavor, CP, and $U(1)_{R}$ symmetries are treated in a
unified manner.

###  2.1 Symplectic structure of moduli fields in Calabi-Yau compactifications

It was well known that the CY compactifications lead to the symplectic
structure in the 4D effective action. (For more details about the symplectic
structure of CY moduli spaces, see, e.g., Refs. [14, 15].) The moduli fields
associated with the CY metric are categorized into two types of moduli fields,
i.e., the complex structure moduli and the Kähler moduli. Their Kähler
potentials in the units of the reduced Planck mass are given by

$\displaystyle K_{\rm cs}$ $\displaystyle=-\ln[-i\int_{{\cal
M}}\Omega\wedge\macc@depth\char
1\relax\frozen@everymath{\macc@group}\macc@set@skewchar\macc@nested@a
111{\Omega}\biggl{]}$ $\displaystyle K_{\rm ks}$
$\displaystyle=-\ln[\int_{\cal M}J_{c}\wedge J_{c}\wedge J_{c}\biggl{]},$
(2.1)

where $\Omega$ and $J_{c}$ denote the holomorphic three-form and
(complexified) Kähler form of CY threefolds ${\cal M}$, respectively. Here,
$J_{c}=B+iJ$ consists of real Kalb-Ramond two-form $B$ and Kähler form $J$.
Their moduli spaces possess the symplectic structure. For concreteness, we
will focus on the complex structure moduli with large complex structure,
paying attention to the symplectic transformations of the moduli fields and
the prepotential in subsequent discussions. The purpose of reviewing the
symplectic structure is to understand the flavor structure of matter fields in
the context of heterotic string theory with standard embedding, as will be
discussed in the next section.

The holomorphic three-form $\Omega$ is expanded on the symplectic basis
$(\alpha_{I},\beta^{I})$ of $H^{3}({\cal M},\mathbb{Z})$,333Note that the
$Sp(2h^{2,1}+2,\mathbb{C})$ appears in the classical moduli space, but loop
and quantum corrections break it to $Sp(2h^{2,1}+2,\mathbb{Z})$, namely
duality transformations of the string theory.

$\displaystyle\Omega=\sum_{I=0}^{h^{2,1}}\left(X^{I}\alpha_{I}-{\cal
F}_{I}\beta^{I}\right),$ (2.2)

where $X^{I}$ are projective coordinates and the structure of ${\cal
F}_{I}=\partial_{I}{\cal F}$ is determined by the prepotential ${\cal F}$. The
orthogonal relation of the symplectic basis $(\alpha_{I},\beta^{I})$

$\displaystyle\int_{\cal
M}\alpha_{I}\wedge\beta^{J}=\delta^{J}_{\,\,\,\,I},\quad\int_{\cal
M}\beta^{J}\wedge\alpha_{I}=-\delta^{J}_{\,\,\,\,I},$ (2.3)

is invariant under the symplectic transformation:

$\displaystyle\begin{pmatrix}\alpha_{I}\\\ \beta^{I}\\\
\end{pmatrix}\rightarrow\begin{pmatrix}a&b\\\ c&d\\\
\end{pmatrix}\begin{pmatrix}\alpha_{I}\\\ \beta^{I}\\\ \end{pmatrix},$ (2.4)

with

$\displaystyle\begin{pmatrix}a&b\\\ c&d\\\ \end{pmatrix}\in
Sp(2h^{2,1}+2,\mathbb{Z}).$ (2.5)

Correspondingly, the vector $\\{X^{I},{\cal F}_{I}\\}^{T}$ (the so-called
period integrals of holomorphic three-form)

$\displaystyle\Pi=\begin{pmatrix}\int_{A^{I}}\Omega\\\ \int_{B_{I}}\Omega\\\
\end{pmatrix}=\begin{pmatrix}X^{I}\\\ {\cal F}_{I}\\\ \end{pmatrix}$ (2.6)

transforms as

$\displaystyle\begin{pmatrix}X^{I}\\\ {\cal F}_{I}\\\
\end{pmatrix}\rightarrow\begin{pmatrix}\widetilde{X}^{I}\\\ \widetilde{{\cal
F}}_{I}\\\ \end{pmatrix}=\begin{pmatrix}d&c\\\ b&a\\\
\end{pmatrix}\begin{pmatrix}X^{I}\\\ {\cal F}_{I}\\\ \end{pmatrix},$ (2.7)

due to the fact that the holomorphic three-form is invariant under the
symplectic transformation. Here, $\widetilde{{\cal F}}_{I}$ is required to be
a derivative of new function $\widetilde{{\cal F}}(\widetilde{X})$, namely
$\widetilde{{\cal F}}_{I}=\partial\widetilde{{\cal
F}}(\widetilde{X})/\partial\widetilde{X}^{I}$, to remain within the same class
of 4D effective action. Note that the 4D ${\cal N}=2$ supersymmetric action is
invariant under the symplectic transformation of the prepotential,
$\widetilde{{\cal F}}(\widetilde{X})={\cal F}(X)$. Furthermore, when

$\displaystyle\widetilde{{\cal F}}(\widetilde{X})={\cal F}(\widetilde{X}),$
(2.8)

the Lagrangian of ${\cal N}=2$ supergravity with $n$ Abelian vector multiplets
is invariant under the symplectic transformations [20], although the
prepotential itself is not invariant under them [21]. This structure holds for
the system of our interest, i.e., string compactifications on $(2,2)$
superconformal field theories [22, 23, 24, 25].

From the symplectic transformation (2.7), it is found that $\\{X^{I}\\}$ takes
a vector-valued modular form of $Sp(2h^{2,1}+2,\mathbb{Z})$, namely

$\displaystyle X^{I}$ $\displaystyle\rightarrow\widetilde{X}^{I}=(c{\cal
F}+d)^{I}_{J}X^{J}=\frac{\partial\widetilde{X}^{I}}{\partial X^{J}}X^{J},$
(2.9)

and the symmetric tensor ${\cal
F}_{IJK}=\partial_{I}\partial_{J}\partial_{K}{\cal F}$ obeys

$\displaystyle{\cal F}_{IJK}$ $\displaystyle\rightarrow\widetilde{{\cal
F}}_{IJK}=\frac{\partial X^{L}}{\partial\widetilde{X}^{I}}\frac{\partial
X^{M}}{\partial\widetilde{X}^{J}}\frac{\partial
X^{N}}{\partial\widetilde{X}^{K}}{\cal F}_{LMN},$ (2.10)

where it is required that $\partial\widetilde{X}^{I}/\partial X^{J}$ is a non-
singular matrix. For instance, the invertible relation between
$\widetilde{X}^{I}$ and $X^{I}$ is lost when the prepotential is a linear
function of $X^{I}$ as pointed out in Ref. [21]. This formula is useful to
understand the symplectic transformation of Yukawa couplings among matter
fields as analyzed in detail below.

In the large complex structure regime444Throughout this paper, we analyze the
classical CY moduli space., the prepotential is of the explicit form

$\displaystyle{\cal
F}(X)=\frac{1}{3!}\kappa_{ijk}\frac{X^{i}X^{j}X^{k}}{X^{0}}=(X^{0})^{2}F(u),$
(2.11)

where $\kappa_{ijk}$ denote the triple intersection numbers and the complex
structure moduli $u^{i}$ are set by

$\displaystyle u^{i}\equiv\frac{X^{i}}{X^{0}}.$ (2.12)

Here and in what follows, we still call
$F(u)=\frac{1}{3!}\kappa_{ijk}u^{i}u^{j}u^{k}$ the prepotential and take the
flat coordinates

$\displaystyle X^{0}=1,\quad X^{i}=u^{i}.$ (2.13)

Then, the symplectic transformation of the complex structure moduli $u^{i}$ is
given by

$\displaystyle u^{i}\equiv X^{i}$
$\displaystyle\rightarrow\widetilde{u}^{i}=\frac{\widetilde{X}^{i}}{\widetilde{X}^{0}}=\frac{\partial\widetilde{X}^{i}}{\partial
X^{J}}\frac{X^{J}}{\widetilde{X}^{0}}=\frac{\partial\widetilde{X}^{i}}{\partial
X^{j}}\frac{X^{j}}{\widetilde{X}^{0}}=\frac{\partial\widetilde{X}^{i}}{\partial
X^{j}}\frac{u^{j}}{\widetilde{X}^{0}}=(c{\cal
F}+d)^{i}_{j}\frac{u^{j}}{\widetilde{X}^{0}}.$ (2.14)

Let us discuss the symplectic transformation of Yukawa couplings ($F_{ijk}$)
among complex structure moduli. They are determined by the triple intersection
numbers $\kappa_{ijk}$,

$\displaystyle
F_{ijk}\equiv\partialderivative{u^{i}}\partialderivative{u^{j}}\partialderivative{u^{k}}F(u)=X^{0}{\cal
F}_{ijk}(X)=\kappa_{ijk},$ (2.15)

which correspond to the holomorphic Yukawa couplings of matter fields in the
context of heterotic string theory with standard embedding. The symplectic
transformation of Yukawa couplings $F_{ijk}$ can be read off from Eq. (2.10),
i.e.,

$\displaystyle\widetilde{F}_{ijk}$
$\displaystyle=\partialderivative{\widetilde{u}^{i}}\partialderivative{\widetilde{u}^{j}}\partialderivative{\widetilde{u}^{k}}\widetilde{F}(\widetilde{u})=\partialderivative{u^{l}}{\widetilde{u}^{i}}\partialderivative{u^{m}}{\widetilde{u}^{j}}\partialderivative{u^{n}}{\widetilde{u}^{k}}F_{lmn},$
(2.16)

or

$\displaystyle\widetilde{F}_{ijk}$
$\displaystyle=\widetilde{X}^{0}\widetilde{{\cal
F}}_{ijk}=\widetilde{X}^{0}\partialderivative{X^{L}}{\widetilde{X}^{i}}\partialderivative{X^{M}}{\widetilde{X}^{j}}\partialderivative{X^{N}}{\widetilde{X}^{k}}\,{\cal
F}_{LMN}=\widetilde{X}^{0}\partialderivative{X^{l}}{\widetilde{X}^{i}}\partialderivative{X^{m}}{\widetilde{X}^{j}}\partialderivative{X^{n}}{\widetilde{X}^{k}}\,F_{lmn}.$
(2.17)

Hence, these two expressions lead to

$\displaystyle\partialderivative{u^{l}}{\widetilde{u}^{i}}=(\widetilde{X}^{0})^{1/3}\partialderivative{X^{l}}{\widetilde{X}^{i}}.\quad$
(2.18)

So far, we have focused on the symplectic structure of the complex structure
moduli space, but the same structure appears in the Kähler moduli space as
discussed in detail in Ref. [15]. When we introduce a basis of $H^{2}({\cal
M},\mathbb{Z})$ as $e_{a}$, the Kähler moduli $t^{a}$ are defined by

$\displaystyle t^{a}e_{a}=B+iJ=J_{c},$ (2.19)

whose prepotential in the large volume regime is given by

$\displaystyle{\cal
G}(Y)=\frac{1}{3!}\kappa_{abc}\frac{Y^{a}Y^{b}Y^{c}}{Y^{0}}=(Y^{0})^{2}G(t),$
(2.20)

with

$\displaystyle t^{a}=\frac{Y^{a}}{Y^{0}}.$ (2.21)

Here, $\kappa_{abc}$ denote the triple intersection numbers among two cycles
of CY threefolds, and $\\{Y^{A}\\}$ represent the projective coordinates on
the Kähler moduli space. In particular, $Y^{0}$ is identified with the inverse
of the axio-dilaton. Analogous to the complex structure moduli, the Kähler
potential (2.1) is written by

$\displaystyle K_{\rm cs}$ $\displaystyle=-\ln[-i\int_{{\cal
M}}\Omega\wedge\macc@depth\char
1\relax\frozen@everymath{\macc@group}\macc@set@skewchar\macc@nested@a
111{\Omega}\biggl{]}=-\ln[-i(\macc@depth\char
1\relax\frozen@everymath{\macc@group}\macc@set@skewchar\macc@nested@a
111{X}^{I}{\cal F}_{I}-X^{I}\macc@depth\char
1\relax\frozen@everymath{\macc@group}\macc@set@skewchar\macc@nested@a
111{{\cal F}}_{I})\biggl{]},$ $\displaystyle K_{\rm ks}$
$\displaystyle=-\ln[\int_{\cal M}J_{c}\wedge J_{c}\wedge
J_{c}\biggl{]}=-\ln[-i(\macc@depth\char
1\relax\frozen@everymath{\macc@group}\macc@set@skewchar\macc@nested@a
111{Y}^{A}{\cal G}_{A}-Y^{A}\macc@depth\char
1\relax\frozen@everymath{\macc@group}\macc@set@skewchar\macc@nested@a
111{{\cal G}}_{A})\biggl{]},$ (2.22)

with $I=0,1,\cdots,h^{2,1}$, $A=0,1,\cdots,h^{1,1}$, and one can deal with the
symplectic structure as summarized in Table 1.

| Complex structure | Kähler structure
---|---|---
Projective coordinates | $X^{I}$ | $Y^{A}$
| $(I=1,2,\cdots,h^{2,1}+1)$ | $(A=1,2,\cdots,h^{1,1}+1)$
Moduli fields | $u^{i}$ | $t^{a}$
| $(i=1,2,\cdots,h^{2,1})$ | $(a=1,2,\cdots,h^{1,1})$
Prepotential | ${\cal F}(X)=(X^{0})^{2}F(u)$ | ${\cal G}(Y)=(Y^{0})^{2}G(t)$
Kähler potential | $-\ln[-i(\macc@depth\char 1\relax\frozen@everymath{\macc@group}\macc@set@skewchar\macc@nested@a 111{X}^{I}{\cal F}_{I}-X^{I}\macc@depth\char 1\relax\frozen@everymath{\macc@group}\macc@set@skewchar\macc@nested@a 111{{\cal F}}_{I})\biggl{]}$ | $-\ln[-i(\macc@depth\char 1\relax\frozen@everymath{\macc@group}\macc@set@skewchar\macc@nested@a 111{Y}^{A}{\cal G}_{A}-Y^{A}\macc@depth\char 1\relax\frozen@everymath{\macc@group}\macc@set@skewchar\macc@nested@a 111{{\cal G}}_{A})\biggl{]}$
Yukawa couplings | $\partial_{i}\partial_{j}\partial_{k}F=\kappa_{ijk}$ | $\partial_{a}\partial_{b}\partial_{c}G=\kappa_{abc}$
Symplectic modular group | $Sp(2h^{2,1}+2,\mathbb{Z})$ | $Sp(2h^{1,1}+2,\mathbb{Z})$
Symplectic transformations | $u^{i}\rightarrow(\widetilde{X}^{0})^{-1}(c{\cal F}+d)^{i}_{j}u^{j}$ | $t^{a}\rightarrow(\widetilde{Y}^{0})^{-1}(g{\cal G}+h)^{a}_{b}t^{b}$
| $\begin{pmatrix}a&b\\\ c&d\\\ \end{pmatrix}\in Sp(2h^{2,1}+2,\mathbb{Z})$ | $\begin{pmatrix}e&f\\\ g&h\\\ \end{pmatrix}\in Sp(2h^{1,1}+2,\mathbb{Z})$

Table 1: Moduli spaces of the complex structure and the Kähler structure.

In the subsequent sections, we discuss the symplectic modular transformations
of matter fields associated with the complex structure moduli and Kähler
moduli. The flavor structure of matter fields can be read off from the
symplectic structure of moduli Yukawa couplings.

###  2.2 Symplectic modular flavor symmetry of matter fields

In general, it is difficult to derive the flavor structure of matter fields in
smooth CY compactifications due to the lack of analytical expression of the CY
metric. Hence, we concentrate on the heterotic string theory with standard
embedding to overcome this problem. The matter fields $27_{a}$ and
$\macc@depth\char
1\relax\frozen@everymath{\macc@group}\macc@set@skewchar\macc@nested@a
111{27}_{i}$ under the $E_{6}$ gauge group are in one-to-one correspondence
with the Kähler moduli and the complex structure moduli, respectively. It
indicates that the flavor structure of matter fields is directly related to
the symplectic structure in the moduli spaces. In the following discussions,
we assume that the matter zero-modes appear from either the complex structure
moduli sector or the Kähler moduli sector since the net number of zero-modes
is determined by

$\displaystyle\chi=\frac{1}{2}|h^{2,1}-h^{1,1}|,$ (2.34)

through the Atiyah-Singer index theorem. We will focus on the matter fields
associated with either the complex structure moduli or the Kähler moduli in
order to reveal the relation between the flavor symmetry of matter fields and
symplectic modular symmetry appearing in the moduli effective action.

It was known that the matter Kähler metric is of the form555The overall factor
$e^{\pm\frac{1}{3}(K_{\rm cs}-K_{\rm ks})}$ takes the different form on
toroidal orbifolds in the example of section 3 due to the enlarged
symmetry[26].

$\displaystyle K^{(27)}_{a\macc@depth\char
1\relax\frozen@everymath{\macc@group}\macc@set@skewchar\macc@nested@a 111{b}}$
$\displaystyle=e^{\frac{1}{3}(K_{\rm cs}-K_{\rm ks})}(K_{\rm
ks})_{a\macc@depth\char
1\relax\frozen@everymath{\macc@group}\macc@set@skewchar\macc@nested@a
111{b}},$ $\displaystyle K^{(\macc@depth\char
1\relax\frozen@everymath{\macc@group}\macc@set@skewchar\macc@nested@a
111{27})}_{i\macc@depth\char
1\relax\frozen@everymath{\macc@group}\macc@set@skewchar\macc@nested@a 111{j}}$
$\displaystyle=e^{-\frac{1}{3}(K_{\rm cs}-K_{\rm ks})}(K_{\rm
cs})_{i\macc@depth\char
1\relax\frozen@everymath{\macc@group}\macc@set@skewchar\macc@nested@a
111{j}},$ (2.35)

where $(K_{\rm cs})_{i\macc@depth\char
1\relax\frozen@everymath{\macc@group}\macc@set@skewchar\macc@nested@a
111{j}}=\partial_{i}\partial_{\bar{j}}K_{\rm cs}$, $(K_{\rm
ks})_{a\macc@depth\char
1\relax\frozen@everymath{\macc@group}\macc@set@skewchar\macc@nested@a
111{b}}=\partial_{a}\partial_{\bar{b}}K_{\rm ks}$, $A^{i}$ and $A^{a}$ denote
the matter chiral superfields, respectively.666In the following, we call
matter chiral superfields as matters for simplicity. At the moment, we study
the relation between $\macc@depth\char
1\relax\frozen@everymath{\macc@group}\macc@set@skewchar\macc@nested@a
111{27}_{i}$ matter fields and the complex structure moduli $u^{i}$ for
concreteness. The Kähler potential of the complex structure moduli in the
large complex structure regime

$\displaystyle K_{\rm cs}$ $\displaystyle=-\ln[-i\int_{\cal
M}\Omega\wedge\macc@depth\char
1\relax\frozen@everymath{\macc@group}\macc@set@skewchar\macc@nested@a
111{\Omega}\biggl{]}=-\ln[-i|X^{0}|^{2}\frac{\kappa_{ijk}}{6}(u^{i}-\macc@depth\char
1\relax\frozen@everymath{\macc@group}\macc@set@skewchar\macc@nested@a
111{u}^{i})(u^{j}-\macc@depth\char
1\relax\frozen@everymath{\macc@group}\macc@set@skewchar\macc@nested@a
111{u}^{j})(u^{k}-\macc@depth\char
1\relax\frozen@everymath{\macc@group}\macc@set@skewchar\macc@nested@a
111{u}^{k})\biggl{]},$ (2.36)

indicates that the matter Kähler metric obeys

$\displaystyle K^{(\macc@depth\char
1\relax\frozen@everymath{\macc@group}\macc@set@skewchar\macc@nested@a
111{27})}_{i\macc@depth\char
1\relax\frozen@everymath{\macc@group}\macc@set@skewchar\macc@nested@a
111{j}}\propto|X^{0}|^{2/3}(K_{\rm cs})_{i\macc@depth\char
1\relax\frozen@everymath{\macc@group}\macc@set@skewchar\macc@nested@a
111{j}},$ (2.37)

up to the factor relevant to the Kähler moduli. Hence, symplectic
transformations of $\macc@depth\char
1\relax\frozen@everymath{\macc@group}\macc@set@skewchar\macc@nested@a
111{27}_{i}$ matter fields are almost the same with those of the moduli fields
$u^{i}$, up to the $X^{0}$ factor. Indeed, the matter Kähler metric transforms
under the symplectic modular transformations as

$\displaystyle K^{(\macc@depth\char
1\relax\frozen@everymath{\macc@group}\macc@set@skewchar\macc@nested@a
111{27})}_{i\macc@depth\char
1\relax\frozen@everymath{\macc@group}\macc@set@skewchar\macc@nested@a
111{j}}\rightarrow|\widetilde{X}^{0}|^{2/3}(K_{\rm
cs})_{\widetilde{l}\widetilde{\macc@depth\char
1\relax\frozen@everymath{\macc@group}\macc@set@skewchar\macc@nested@a
111{m}}}\partialderivative{\widetilde{u}^{i}}{u^{l}}\partialderivative{\widetilde{\macc@depth\char
1\relax\frozen@everymath{\macc@group}\macc@set@skewchar\macc@nested@a
111{u}}^{j}}{\macc@depth\char
1\relax\frozen@everymath{\macc@group}\macc@set@skewchar\macc@nested@a
111{u}^{m}}$ (2.38)

up to the factor $e^{K_{\rm ks}/3}$. It turns out that the Kähler potential of
$\macc@depth\char
1\relax\frozen@everymath{\macc@group}\macc@set@skewchar\macc@nested@a
111{27}_{i}$ matter fields has a modular invariance under the following
symplectic transformation of matter fields:

$\displaystyle
A^{i}\rightarrow\widetilde{A}^{i}=(\widetilde{X}^{0})^{1/3}\partialderivative{\widetilde{u}^{i}}{u^{l}}A^{l}=\frac{\partial\widetilde{X}^{i}}{\partial
X^{j}}A^{j},$ (2.39)

where we use Eq. (2.18). Since the matrix

$\displaystyle\frac{\partial\widetilde{X}^{i}}{\partial X^{j}}=(c{\cal
F}+d)^{i}_{j}$ (2.40)

is an element of the subgroup of $Sp(2h^{2,1}+2,\mathbb{Z})$, the flavor
structure of matter fields is governed by the symplectic structure. In what
follows, we adopt the gauge $X^{0}=1$ unless we specify it. As will be
discussed later, the above modular transformation is indeed consistent with
the Kähler invariance of the action by checking the Kähler invariant quantity.

On the other hand, the matter superpotential takes the form

$\displaystyle W$ $\displaystyle=F_{ijk}A^{i}A^{j}A^{k},$ (2.41)

where the matter Yukawa couplings are the same as the Yukawa couplings of the
moduli fields. As we derived before, the $Sp(2h^{2,1}+2,\mathbb{Z})$ modular
transformations of their Yukawa couplings are provided by

$\displaystyle\widetilde{F}_{ijk}$
$\displaystyle=\widetilde{X}^{0}\partialderivative{X^{l}}{\widetilde{X}^{i}}\partialderivative{X^{m}}{\widetilde{X}^{j}}\partialderivative{X^{n}}{\widetilde{X}^{k}}\,F_{lmn}.$
(2.42)

Hence, we arrive at the symplectic modular transformation of the Kähler
potential and the superpotential:

$\displaystyle K_{\rm cs}$ $\displaystyle\rightarrow K_{\rm
cs}-\ln|\widetilde{X}^{0}|^{2},\quad W\rightarrow\widetilde{X}^{0}W,$ (2.43)

from which the Kähler invariant quantity

$\displaystyle e^{K}|W|^{2}$ (2.44)

is invariant under the symplectic modular transformation.

We conclude that the flavor symmetry of matter fields $\macc@depth\char
1\relax\frozen@everymath{\macc@group}\macc@set@skewchar\macc@nested@a
111{27}_{i}$, $G_{\rm flavor}$, belongs to the $Sp(2h^{2,1}+2,\mathbb{Z})$
modular symmetry,

$\displaystyle G_{\rm flavor}\subset Sp(2h^{2,1}+2,\mathbb{Z}),$ (2.45)

and the flavor structure is governed by the symplectic structure of the
complex structure moduli. The same statements also hold for the $27_{a}$
matters associated with the Kähler moduli, whose prepotential is given in Eq.
(2.20) in the large volume regime. The flavor structure of $27_{a}$ matters is
determined by the holomorphic Yukawa coupling $\kappa_{abc}$. For that reason,
we call the flavor symmetry of matter fields the symplectic modular flavor
symmetry. The symplectic transformations of matter zero-modes, moduli, and
holomorphic Yukawa couplings are summarized in Table 2. It is a natural
generalization of $Sp(2,\mathbb{Z})\simeq SL(2,\mathbb{Z})$ discussed in the
toroidal background [1]. Since it is difficult to derive the flavor symmetries
in a background-independent way, we explicitly demonstrate the existence of
symplectic modular flavor symmetries on toroidal orbifolds with and without
resolutions and some classes of CY threefolds in Sec. 3.

|  $\macc@depth\char 1\relax\frozen@everymath{\macc@group}\macc@set@skewchar\macc@nested@a 111{27}$ matter fields |  $27$ matter fields
---|---|---
| (Complex structure) | (Kähler structure)
Matter fields | $A^{i}$ | $A^{a}$
Moduli fields | $u^{i}$ | $t^{a}$
Superpotential | $F_{ijk}A^{i}A^{j}A^{k}$ | $G_{abc}A^{a}A^{b}A^{c}$
Symplectic transformations | $A^{i}\rightarrow\widetilde{A}^{i}=\frac{\partial\widetilde{X}^{i}}{\partial X^{j}}A^{j}$ | $A^{a}\rightarrow\widetilde{A}^{a}=\frac{\partial\widetilde{Y}^{a}}{\partial Y^{b}}A^{b}$
| $u^{i}\rightarrow\widetilde{u}^{i}=(\widetilde{X}^{0})^{-1}\frac{\partial\widetilde{X}^{i}}{\partial X^{j}}u^{j}$ | $t^{a}\rightarrow\widetilde{t}^{a}=(\widetilde{Y}^{0})^{-1}\frac{\partial\widetilde{Y}^{a}}{\partial Y^{b}}t^{b}$
| $F_{ijk}\rightarrow\widetilde{F}_{ijk}=\widetilde{X}^{0}\partialderivative{X^{l}}{\widetilde{X}^{i}}\partialderivative{X^{m}}{\widetilde{X}^{j}}\partialderivative{X^{n}}{\widetilde{X}^{k}}F_{lmn}$ | $G_{abc}\rightarrow\widetilde{G}_{abc}=\widetilde{Y}^{0}\partialderivative{Y^{d}}{\widetilde{Y}^{a}}\partialderivative{Y^{e}}{\widetilde{Y}^{b}}\partialderivative{Y^{f}}{\widetilde{Y}^{c}}G_{def}$
| $\frac{\partial\widetilde{X}^{i}}{\partial X^{j}}=(c{\cal F}+d)^{i}_{j}$ | $\frac{\partial\widetilde{Y}^{a}}{\partial Y^{b}}=(g{\cal G}+h)^{a}_{b}$
Flavor symmetry | $G_{\rm flavor}\subset Sp(2h^{2,1}+2,\mathbb{Z})$ | $G_{\rm flavor}\subset Sp(2h^{1,1}+2,\mathbb{Z})$

Table 2: Symplectic modular flavor symmetries.

###  2.3 $U(1)_{R}$ symmetry

In addition to the symplectic modular flavor symmetry, there exists a
$U(1)_{R}$ symmetry in the 4D ${\cal N}=1$ supersymmetric effective action.
From the cubic matter superpotential (2.41), one can assign the $R$-charge 2/3
for the matter fields in a conventional way due to the fact that the
superpotential has $R$-charge 2. Here, we examine the $\macc@depth\char
1\relax\frozen@everymath{\macc@group}\macc@set@skewchar\macc@nested@a
111{27}_{i}$ matter fields, but $27_{a}$ matter fields also have $R$-charge
2/3 due to the same cubic superpotential. In this section, we discuss the
relation between the symplectic modular symmetry and $U(1)_{R}$ symmetry.

Let us consider the symplectic transformations in the basis where the Yukawa
couplings do not have the factor $\tilde{X}^{0}$,

$\displaystyle A^{i}$
$\displaystyle\rightarrow\widetilde{A}^{i}=(\widetilde{X}^{0})^{1/3}\partialderivative{\widetilde{u}^{i}}{u^{l}}A^{l},$
$\displaystyle F_{ijk}$
$\displaystyle\rightarrow\widetilde{F}_{ijk}=\partialderivative{u^{l}}{\widetilde{u}^{i}}\partialderivative{u^{m}}{\widetilde{u}^{j}}\partialderivative{u^{n}}{\widetilde{u}^{k}}F_{lmn}.$
(2.56)

Since the $U(1)_{R}$ symmetry is not the flavor symmetry, the
$(\widetilde{X}^{0})^{1/3}$ factor appearing in the symplectic transformation
of matter fields would play a role of $R$-symmetric transformation. In this
respect, let us set

$\displaystyle\widetilde{X}^{0}=e^{2i\alpha},$ (2.57)

with $\alpha$ being a constant relevant to the $R$-symmetric
transformation[27]. This assignment is consistent with the symplectic modular
symmetry and it gives rise to the $R$-charge 2 for the moduli fields. Indeed,
the $R$-symmetric transformations for the period integrals in the large
complex structure regime correspond to

$\displaystyle\begin{pmatrix}X^{I}\\\ {\cal F}_{I}\\\
\end{pmatrix}\rightarrow\begin{pmatrix}\widetilde{X}^{I}\\\ \widetilde{{\cal
F}}_{I}\\\ \end{pmatrix}=\widetilde{X}^{0}\begin{pmatrix}X^{I}\\\ {\cal
F}_{I}\\\ \end{pmatrix},$ (2.58)

with

$\displaystyle\\{X^{0},X^{i},{\cal F}_{0},{\cal F}_{i}\\}$
$\displaystyle=\left\\{1,u^{i},-\frac{1}{3!}\kappa_{ijk}u^{i}u^{j}u^{k},\frac{1}{2}\kappa_{ijk}u^{j}u^{k}\right\\},$
$\displaystyle\\{\widetilde{X}^{0},\widetilde{X}^{i},\widetilde{{\cal
F}}_{0},\widetilde{{\cal F}}_{i}\\}$
$\displaystyle=\widetilde{X}^{0}\left\\{1,\widetilde{u}^{i},-\frac{1}{3!}\kappa_{ijk}\widetilde{u}^{i}\widetilde{u}^{j}\widetilde{u}^{k},\frac{1}{2}\kappa_{ijk}\widetilde{u}^{j}\widetilde{u}^{k}\right\\}.$
(2.59)

As stated in Ref. [19], $R$-symmetric transformation corresponds to the
holomorphic transformation $f$ mapping the holomorphic three-form of CY
threefolds $\Omega$ into a multiple of itself,

$\displaystyle f\cdot\Omega=e^{i\gamma}\Omega,$ (2.60)

with $\gamma$ being the constant. Recalling that the $\Omega$ is constructed
by a covariantly constant positive-chirality spinor $\eta$, namely
$\Omega_{ijk}=\eta^{T}\Gamma_{ijk}\eta$ with three products of Gamma matrix
$\Gamma_{ijk}$, the holomorphic mapping $f$ rotates the $\eta$,

$\displaystyle f\cdot\eta=e^{i\gamma/2}\eta,$ (2.61)

namely the 4D ${\cal N}=1$ supersymmetry by a phase of $e^{i\gamma/2}$ for
positive chirality spinor. The negative chirality spinor is rotated by an
amount of $e^{-i\gamma/2}$. In this way, the phase shift of $\Omega$ gives
rise to the $R$-symmetry in 4D ${\cal N}=1$ supersymmetric theory. In our
setup, our finding $R$-symmetric transformation
$\widetilde{X}^{0}=e^{2i\alpha}$ corresponds to $\gamma=2\alpha$. Note that
the existence of $U(1)_{R}$ symmetry is related to the discrete
$\mathbb{Z}_{3}^{R}$ $R$-symmetry appearing in the matter superpotential,

$\displaystyle W=F_{ijk}A^{i}A^{j}A^{k},$ (2.62)

by assigning the charge 1 for all the matter fields. When we focus on the
overall complex structure modulus $u^{1}=u^{2}=\cdots=u^{h^{2,1}}$, the action
is invariant under the overall $SL(2,\mathbb{Z})_{\rm overall}$ symmetry. From
the fact that the superpotential has modular weight -3 under
$SL(2,\mathbb{Z})_{\rm overall}$, the matter fields have charge -1. That would
be related to the origin of charge 1 under the discrete $\mathbb{Z}_{3}^{R}$
$R$-symmetry 777The relations between $R$-symmetry and the modular symmetry,
in particular, their anomalies were studied in Refs. [28, 29].. Similar
statements also hold for the Kähler moduli space, by identifying $Y^{0}$ with
the $R$-symmetric transformations of $27_{a}$ matter fields. Hence,
$R$-symmetric transformations would be related to the phase rotation of the
axio-dilaton. It is notable that the Kähler form consists of Killing spinors,
$J_{m\bar{n}}=i\eta^{\dagger}\Gamma_{m\bar{n}}\eta$ with two products of Gamma
matrix $\Gamma_{m\bar{n}}$ in the complex basis $z^{m}$ with $m=1,2,3$, but it
does not change under the $R$-transformation.

As a result, $U(1)_{R}$ symmetry is also unified in the context of symplectic
modular symmetry, i.e.,

$\displaystyle Sp(2h+2,\mathbb{C})\supset Sp(2h+2,\mathbb{Z})\times U(1)_{R},$
(2.63)

with $h=h^{2,1}$ or $h^{1,1}$.

###  2.4 Unification of symplectic modular flavor, CP, and $R$-symmetries

Following Refs. [30, 31, 32], one can identify the 4D CP symmetry with the
simultaneous transformation of 4D parity and the 6D orientation reversing.
They belong to the 10D proper Lorentz transformation. We recall that the 10D
Majorana-Weyl spinor $16$ decomposes under the 10D Lorentz symmetry
$SO(1,9)=SO(1,3)\times SO(6)$,

$\displaystyle 16=(2_{L},4_{+})\oplus(2_{R},\macc@depth\char
1\relax\frozen@everymath{\macc@group}\macc@set@skewchar\macc@nested@a
111{4}_{-})$ (2.64)

where $2_{L}$ and $2_{R}$ denote the left- and right-handed spinors of
$SL(2,\mathbb{C})$, and $4_{+},\macc@depth\char
1\relax\frozen@everymath{\macc@group}\macc@set@skewchar\macc@nested@a
111{4}_{-}$ represent the positive and negative chirality spinors of
$SU(4)\simeq SO(6)$, respectively. Hence, the 4D parity and 6D orientation
reversing exchanges $(2_{L},4_{+})$ into $(2_{R},\macc@depth\char
1\relax\frozen@everymath{\macc@group}\macc@set@skewchar\macc@nested@a
111{4}_{-})$ which is in one-to-one correspondence with the 4D CP
transformation of $E_{6}$ matter fields: $27\rightarrow\macc@depth\char
1\relax\frozen@everymath{\macc@group}\macc@set@skewchar\macc@nested@a
111{27}$. It suggests that the 4D CP transformation is identified with the
geometrical symmetry of the internal manifold.

The 6D orientation reversing changes the sign of the volume form of CY
threefolds. In the local coordinate of CY threefolds $z^{m}$ with $m=1,2,3$,
the following transformation

$\displaystyle z^{m}\rightarrow-\macc@depth\char
1\relax\frozen@everymath{\macc@group}\macc@set@skewchar\macc@nested@a
111{z}^{m}$ (2.65)

changes the sign of the volume form.888The minus sign is a matter of
convention. To see the orientation reversing of the volume form, we examine
the transformation of the holomorphic three-form $\Omega$ and Kähler form $J$.
Eq. (2.65) gives rise to the $\mathbb{Z}_{2}^{\rm CP}$ transformation:

$\displaystyle\begin{split}\Omega\rightarrow-\macc@depth\char
1\relax\frozen@everymath{\macc@group}\macc@set@skewchar\macc@nested@a
111{\Omega},\\\ J_{c}\rightarrow\macc@depth\char
1\relax\frozen@everymath{\macc@group}\macc@set@skewchar\macc@nested@a
111{J_{c}},\end{split}$ (2.66)

where the anti-holomorphic transformation of $\Omega$ is consistent with its
local expansion $\Omega=dz^{1}\wedge dz^{2}\wedge dz^{3}$. On CY threefolds,
the 6D volume form $dV$ is expressed by the $\Omega$,

$\displaystyle i\Omega\wedge\macc@depth\char
1\relax\frozen@everymath{\macc@group}\macc@set@skewchar\macc@nested@a
111{\Omega}=|\Omega|^{2}dV,$ (2.67)

thereby the transformation (2.66) changes the sign of the volume form, namely
the 6D orientation reversing.

We comment on the orientation reversing of the Kähler form $J_{c}$ which is
locally given by $J_{c}=B+iJ=(b_{m\bar{n}}+ig_{m\bar{n}})dz^{m}\wedge
d\macc@depth\char
1\relax\frozen@everymath{\macc@group}\macc@set@skewchar\macc@nested@a
111{z}^{\bar{n}}$, respectively. Recall that the real part of complexified
Kähler form ($b_{m\bar{n}}$) originates from the Kalb-Ramond field, leading to
the pseudo scalars (axions) in the 4D effective action through the hodge
duality. Since the axions are 4D CP-odd fields through the 4D Chern-Simons
couplings, $b_{m\bar{n}}$ is required to be transformed as
$b_{m\bar{n}}\rightarrow-b_{m\bar{n}}$. By contrast, the imaginary part of
complexified Kähler form ($g_{m\bar{n}}$) is a CP-even field. Hence, we arrive
at the CP transformation of the complexified Kähler form as in Eq. (2.66).

As discussed in Ref. [33], the anti-holomorphic transformation of the
holomorphic three-form induces the anti-holomorphic transformation of the
complex structure moduli, and the CP-transformation of the complexified Kähler
form also induces the anti-holomorphic transformation of the Kähler moduli due
to the fact that $b_{m\bar{n}}$ and $g_{m\bar{n}}$ are CP-odd and -even
fields, respectively. Then, we arrive at the CP transformation of the moduli
fields:

$\displaystyle u^{i}\rightarrow-\macc@depth\char
1\relax\frozen@everymath{\macc@group}\macc@set@skewchar\macc@nested@a
111{u}^{i},\quad t^{a}\rightarrow-\macc@depth\char
1\relax\frozen@everymath{\macc@group}\macc@set@skewchar\macc@nested@a
111{t}^{a},$ (2.68)

where we take the convention with ${\rm Im}(u^{i})>0$ and ${\rm Im}(t^{a})>0$.
Furthermore, since the real part of the axio-dilaton $Y^{0}$ originating from
the Kalb-Ramond six-form is the universal axion, the CP transformation gives
rise to

$\displaystyle Y^{0}\rightarrow-\macc@depth\char
1\relax\frozen@everymath{\macc@group}\macc@set@skewchar\macc@nested@a
111{Y}^{0}.$ (2.69)

It is trivial to check that these anti-holomorphic transformations are not the
element of symplectic modular groups, but they belong to the generalized
symplectic modular group $GSp(2g+2,\mathbb{Z})\simeq
Sp(2g+2,\mathbb{Z})\rtimes\mathbb{Z}_{2}^{\rm CP}$ with $g=h^{2,1},h^{1,1}$.
Indeed, the $\mathbb{Z}_{2}^{\rm CP}$ transformation acts on the projective
coordinates in the complex structure and the Kähler structure moduli
spaces999Here, we adopt the minus sign for the CP transformation in the
complex structure moduli space, although the positive sign is allowed.

$\displaystyle\begin{pmatrix}X^{0}\\\ X^{i}\\\ {\cal F}_{0}\\\ {\cal F}_{i}\\\
\end{pmatrix}\rightarrow-\begin{pmatrix}\macc@depth\char
1\relax\frozen@everymath{\macc@group}\macc@set@skewchar\macc@nested@a
111{X}^{0}\\\ -\macc@depth\char
1\relax\frozen@everymath{\macc@group}\macc@set@skewchar\macc@nested@a
111{X}^{i}\\\ -\macc@depth\char
1\relax\frozen@everymath{\macc@group}\macc@set@skewchar\macc@nested@a
111{{\cal F}}_{0}\\\ \macc@depth\char
1\relax\frozen@everymath{\macc@group}\macc@set@skewchar\macc@nested@a
111{{\cal F}}_{i}\\\ \end{pmatrix}={\cal CP}\begin{pmatrix}X^{0}\\\ X^{i}\\\
{\cal F}_{0}\\\ {\cal F}_{i}\\\ \end{pmatrix},\quad\begin{pmatrix}Y^{0}\\\
Y^{a}\\\ {\cal G}_{0}\\\ {\cal G}_{a}\\\
\end{pmatrix}\rightarrow-\begin{pmatrix}\macc@depth\char
1\relax\frozen@everymath{\macc@group}\macc@set@skewchar\macc@nested@a
111{Y}^{0}\\\ -\macc@depth\char
1\relax\frozen@everymath{\macc@group}\macc@set@skewchar\macc@nested@a
111{Y}^{a}\\\ -\macc@depth\char
1\relax\frozen@everymath{\macc@group}\macc@set@skewchar\macc@nested@a
111{{\cal G}}_{0}\\\ \macc@depth\char
1\relax\frozen@everymath{\macc@group}\macc@set@skewchar\macc@nested@a
111{{\cal G}}_{a}\\\ \end{pmatrix}={\cal CP}\begin{pmatrix}Y^{0}\\\ Y^{a}\\\
{\cal G}_{0}\\\ {\cal G}_{a}\\\ \end{pmatrix},$ (2.70)

with

$\displaystyle{\cal CP}=\begin{pmatrix}-1&0&0&0\\\ 0&\bf{1}&0&0\\\ 0&0&1&0\\\
0&0&0&-\bf{1}\\\ \end{pmatrix}\notin
Sp(2h^{2,1}+2,\mathbb{Z}),Sp(2h^{1,1}+2,\mathbb{Z}).$ (2.71)

Here, we use the shorthand notation for the identity matrix $\bf{1}$, which is
rank $h^{2,1}$ for the complex structure moduli and $h^{1,1}$ for the Kähler
moduli, respectively. Hence, they do not belong to the element of
$Sp(2h^{2,1}+2,\mathbb{Z})$ and $Sp(2h^{1,1}+2,\mathbb{Z})$, due to the fact
that the CP-transformation does not have the symplectic structure ${\cal
CP}^{T}\cdot\Sigma\cdot{\cal CP}=\Sigma^{T}\neq\Sigma$. Rather, the CP
transformation is regarded as an outer automorphism of the symplectic modular
group, i.e., ${\cal CP}\cdot\gamma\cdot{\cal CP}^{-1}$ with $\gamma$ being the
transformation of the $Sp(2h^{2,1}+2,\mathbb{Z})$ or
$Sp(2h^{1,1}+2,\mathbb{Z})$ modular group[13]101010Outer automorphisms of
flavor symmetries were studied as generalized CP symmetries [34, 35, 36]..

As a result, the symplectic modular group together with the CP transformation
is enlarged to the generalized symplectic modular group. In particular, the
flavor symmetry belonging to the symplectic modular symmetry is also enhanced
to the non-Abelian symmetry,

$\displaystyle GSp(2h^{2,1}+2,\mathbb{Z})$ $\displaystyle\supset G_{\rm
flavor}\rtimes\mathbb{Z}_{2}^{\rm CP},$ $\displaystyle
GSp(2h^{1,1}+2,\mathbb{Z})$ $\displaystyle\supset G_{\rm
flavor}\rtimes\mathbb{Z}_{2}^{\rm CP},$ (2.72)

as exemplified in the next section. The $U(1)_{R}$ symmetry further enhances
them into

$\displaystyle GSp(2h^{2,1}+2,\mathbb{C})$ $\displaystyle\supset G_{\rm
flavor}\rtimes\mathbb{Z}_{2}^{\rm CP}\times U(1)_{R},$ $\displaystyle
GSp(2h^{1,1}+2,\mathbb{C})$ $\displaystyle\supset G_{\rm
flavor}\rtimes\mathbb{Z}_{2}^{\rm CP}\times U(1)_{R}.$ (2.73)

In the following analysis, we explicitly demonstrate the symplectic modular
flavor symmetry on several backgrounds. This approach would be deserved to be
the ultra-violet completion of the phenomenological model buildings in the
bottom-up approach, where the symplectic modular flavor symmetry is used to
explain the flavor structure of quarks and leptons.

##  3 Examples

In this section, we show the existence of generalized modular symmetry
involving the flavor and CP symmetries on several backgrounds. Starting from
the simplest 6D toroidal orbifolds in Sec. 3.1, we deal with several blown-up
toroidal orbifolds, where the size of blow-up radii is considered small enough
in Secs. 3.2 and 3.3 and large enough in Sec. 3.4. The three-parameter
examples of CY threefolds are analyzed in Sec. 3.5.

###  3.1 6D toroidal orbifolds

We begin with toroidal orbifolds with an emphasis on the flavor symmetry of
untwisted modes. The twisted (blow-up) modes are analyzed in detail in the
next sections. The simplest but non-trivial examples realizing the flavor
symmetries of untwisted modes are based on $T^{6}/\mathbb{Z}_{N}$ and
$T^{6}/(\mathbb{Z}_{N}\times\mathbb{Z}_{M})$ with $h^{1,1}_{\rm untw}=3$,
whose prepotential is given by111111The following discussions are applicable
to the sector of $h^{2,1}=3$ complex structure moduli sector as well.

$\displaystyle G=t^{1}t^{2}t^{3},$ (3.1)

leading to the Kähler potential[26]

$\displaystyle K_{\rm ks}=-\ln[i(t^{1}-\macc@depth\char
1\relax\frozen@everymath{\macc@group}\macc@set@skewchar\macc@nested@a
111{t}^{1})(t^{2}-\macc@depth\char
1\relax\frozen@everymath{\macc@group}\macc@set@skewchar\macc@nested@a
111{t}^{2})(t^{3}-\macc@depth\char
1\relax\frozen@everymath{\macc@group}\macc@set@skewchar\macc@nested@a
111{t}^{3})-\sum_{a}|A^{a}|^{2}\biggl{]}-\ln|Y^{0}|^{2}.$ (3.2)

Here, we include $Y^{0}$ for the later purpose, and the matter fields are
assumed to be expanded around $|A^{a}|\ll 1$. Correspondingly, the matter
superpotential is of the form

$\displaystyle W=Y^{0}A^{1}A^{2}A^{3}.$ (3.3)

Obviously, there appear the $\Pi_{a=1}^{3}SL(2,\mathbb{Z})_{a}$ symmetries
generated by

$\displaystyle S^{(a)}:t^{a}\rightarrow-1/t^{a},\quad T^{(a)}:t^{a}\rightarrow
t^{a}+1,$ (3.4)

for $\Pi_{a=1}^{3}SL(2,\mathbb{Z})_{a}$. They originate from factorizable
three tori: $T^{6}=(T^{2})^{3}$. In addition, we find that there exists a
$S_{4}$ symmetry by assigning $\\{t^{1},t^{2},t^{3}\\}$ for $S_{4}$ triplet
and $Y^{0}$ for $S_{4}$ singlet. Although the $S_{4}$ group has two triplets
$\\{\mathbf{3},\mathbf{3}^{\prime}\\}$ and two singlets
$\\{\mathbf{1},\mathbf{1}^{\prime}\\}$, the superpotential is invariant under
two cases: (i) $\\{t^{1},t^{2},t^{3}\\}$ for $\mathbf{3}$ and $Y^{0}$ for
$\mathbf{1}$, (ii) $\\{t^{1},t^{2},t^{3}\\}$ for $\mathbf{3}^{\prime}$ and
$Y^{0}$ for $\mathbf{1}^{\prime}$. (See for non-Abelian discrete symmetries
and notation of representation, Refs. [37, 38]. ) When we take the gauge
$Y^{0}=1$, namely $S_{4}$ trivial singlet $\mathbf{1}$, the transformation of
the $S_{4}$ triplet $\\{t^{1},t^{2},t^{3}\\}$ is given by

$\displaystyle P$ $\displaystyle:t^{1}\rightarrow t^{3},\quad t^{2}\rightarrow
t^{1},\quad t^{3}\rightarrow t^{2},$ $\displaystyle Q$
$\displaystyle:t^{1}\rightarrow-t^{1},\quad t^{2}\rightarrow-t^{3},\quad
t^{3}\rightarrow t^{2},$ (3.5)

where $P$ corresponds to $S_{3}$ permutation group of three
$SL(2,\mathbb{Z})_{a}$ [39]. In the context of heterotic string theory with
standard embedding, these symmetries are identified with the flavor symmetries
of matter fields. Indeed, the matter fields $\\{A^{1},A^{2},A^{3}\\}$ are also
identified with the $S_{4}$ triplet representation under which the effective
action of matter fields is invariant.

Our purpose in this section is to check whether these symmetries together with
the CP transformation can be embedded into the $GSp(8,\mathbb{Z})$ generalized
symplectic modular symmetry. The symplectic structure is understood from the
symplectic transformation of the period vector

$\displaystyle\begin{pmatrix}Y^{A}\\\ {\cal G}_{A}\\\
\end{pmatrix}=\begin{pmatrix}Y^{0}\\\ Y^{1}\\\ Y^{2}\\\ Y^{3}\\\ {\cal
G}_{0}\\\ {\cal G}_{1}\\\ {\cal G}_{2}\\\ {\cal G}_{3}\\\
\end{pmatrix}=\begin{pmatrix}Y^{0}\\\ Y^{1}\\\ Y^{2}\\\ Y^{3}\\\
-Y^{1}Y^{2}Y^{3}/(Y^{0})^{2}\\\ Y^{2}Y^{3}/Y^{0}\\\ Y^{1}Y^{3}/Y^{0}\\\
Y^{1}Y^{2}/Y^{0}\\\ \end{pmatrix}=Y^{0}\begin{pmatrix}1\\\ t^{1}\\\ t^{2}\\\
t^{3}\\\ -t^{1}t^{2}t^{3}\\\ t^{2}t^{3}\\\ t^{1}t^{3}\\\ t^{1}t^{2}\\\
\end{pmatrix},$ (3.6)

leading to the Kähler potential (3.2). Since the period vector is a
fundamental representation of $Sp(8,\mathbb{Z})$, the symplectic
transformation is represented by

$\displaystyle\begin{pmatrix}Y^{A}\\\ {\cal G}_{A}\\\
\end{pmatrix}=Y^{0}\begin{pmatrix}1\\\ t^{1}\\\ t^{2}\\\ t^{3}\\\
-t^{1}t^{2}t^{3}\\\ t^{2}t^{3}\\\ t^{1}t^{3}\\\ t^{1}t^{2}\\\
\end{pmatrix}\rightarrow\widetilde{Y}^{0}\begin{pmatrix}1\\\
\widetilde{t}^{1}\\\ \widetilde{t}^{2}\\\ \widetilde{t}^{3}\\\
-\widetilde{t}^{1}\widetilde{t}^{2}\widetilde{t}^{3}\\\
\widetilde{t}^{2}\widetilde{t}^{3}\\\ \widetilde{t}^{1}\widetilde{t}^{3}\\\
\widetilde{t}^{1}\widetilde{t}^{2}\\\ \end{pmatrix}=Y^{0}\times{\cal
R}\begin{pmatrix}1\\\ t^{1}\\\ t^{2}\\\ t^{3}\\\ -t^{1}t^{2}t^{3}\\\
t^{2}t^{3}\\\ t^{1}t^{3}\\\ t^{1}t^{2}\\\ \end{pmatrix},$ (3.7)

with ${\cal R}\subset Sp(8,\mathbb{Z})$.

It turns out that $\Pi_{a=1}^{3}SL(2,\mathbb{Z})_{a}$ and $S_{4}$ symmetries
are indeed subgroups of $Sp(8,\mathbb{Z})$. For instance, the generators of
$SL(2,\mathbb{Z})_{1}$ and $S_{4}$ are chosen as121212It is straightforward to
obtain the generators of other $SL(2,\mathbb{Z})$ by flipping the
corresponding moduli fields.

$\displaystyle\begin{split}S_{1}&=\begin{pmatrix}0&-1&0&0&0&0&0&0\\\
1&0&0&0&0&0&0&0\\\ 0&0&0&0&0&0&0&-1\\\ 0&0&0&0&0&0&-1&0\\\ 0&0&0&0&0&-1&0&0\\\
0&0&0&0&1&0&0&0\\\ 0&0&0&1&0&0&0&0\\\ 0&0&1&0&0&0&0&0\end{pmatrix},\quad
T_{1}=\begin{pmatrix}1&0&0&0&0&0&0&0\\\ 1&1&0&0&0&0&0&0\\\ 0&0&1&0&0&0&0&0\\\
0&0&0&1&0&0&0&0\\\ 0&0&0&0&1&-1&0&0\\\ 0&0&0&0&0&1&0&0\\\ 0&0&0&1&0&0&1&0\\\
0&0&1&0&0&0&0&1\end{pmatrix},\\\ P&=\begin{pmatrix}1&0&0&0&0&0&0&0\\\
0&0&0&1&0&0&0&0\\\ 0&1&0&0&0&0&0&0\\\ 0&0&1&0&0&0&0&0\\\ 0&0&0&0&1&0&0&0\\\
0&0&0&0&0&0&0&1\\\ 0&0&0&0&0&1&0&0\\\ 0&0&0&0&0&0&1&0\end{pmatrix},\quad
Q=\begin{pmatrix}1&0&0&0&0&0&0&0\\\ 0&-1&0&0&0&0&0&0\\\ 0&0&0&-1&0&0&0&0\\\
0&0&1&0&0&0&0&0\\\ 0&0&0&0&1&0&0&0\\\ 0&0&0&0&0&-1&0&0\\\ 0&0&0&0&0&0&0&-1\\\
0&0&0&0&0&0&1&0\end{pmatrix},\end{split}$ (3.8)

where the fundamental representation of $Sp(8,\mathbb{Z})$ is split into

$\displaystyle\mathbf{8}=\mathbf{1}+\mathbf{3}+\mathbf{1}+\mathbf{3},$ (3.9)

under $S_{4}$ symmetry. We have focused on the transformation of moduli
fields, but the $S_{4}$ flavor symmetry of matter fields is also realized by
$P$ and $Q$. Indeed, the submatrices of $P$ and $Q$ acting on
$(t^{1},t^{2},t^{3})^{T}$

$\displaystyle\begin{pmatrix}0&0&1\\\ 1&0&0\\\ 0&1&0\\\
\end{pmatrix},\quad\begin{pmatrix}-1&0&0\\\ 0&0&-1\\\ 0&1&0\\\ \end{pmatrix},$
(3.10)

correspond to the transformations for untwisted matters
$\\{A^{1},A^{2},A^{3}\\}$ with the $S_{4}$ triplet. Note that the holomorphic
Yukawa coupling is invariant under this $S_{4}$ symmetry.

Let us discuss the origin of the $S_{4}$ symmetry in the effective action. We
recall that imaginary parts of moduli ${\rm Im}(t^{a})$ behave as a triplet
representation under $SO(3)$ group due to the fact that the volume form of
factorizable toroidal background ${\cal V}$ is given by ${\cal
V}\simeq\otimes_{a=1}^{3}{\rm Im}(t^{a})dx^{a}\wedge dy^{a}$. Since the
$SO(3)$ triplet representation corresponds to the $S_{4}$ triplet
representation131313See, e.g., Ref. [40] and references therein. , $S_{4}$
symmetry exists in the moduli sector $\\{{\rm Im}(t^{1}),{\rm Im}(t^{2}),{\rm
Im}(t^{3})\\}$. On the complexified Kähler form, the Kähler moduli consist of
volume moduli ${\rm Im}(t^{a})$ and the Kähler axions ${\rm Re}(t^{a})$. This
fact enhances $SO(3)$ into $SU(3)$ symmetry. On the other hand, matter fields
are originated from 10D gauge bosons and gauginos. Since 10D ${\cal N}=1$
supersymmetry gives rise to 4D ${\cal N}=4$ supersymmetry equipping
$SU(4)_{R}$ symmetry on $T^{6}$ background without orbifolding, the
fundamental representation of $SU(4)_{R}$ symmetry can be split into

$\displaystyle\mathbf{4}=\mathbf{1}+\mathbf{3},$ (3.11)

under the $SU(3)_{R}$ symmetry. The matter fields $\\{A^{a}\\}$ of our
interest have triplet representation of $SU(3)_{R}$ symmetry, corresponding to
the $S_{4}$ triplet representation. Indeed, the $SU(3)_{R}$ symmetry is
embedded into $Sp(8,\mathbb{C})$ as follows:

$\displaystyle\begin{split}R&=\begin{pmatrix}1&0&0&0&0&0&0&0\\\
0&a_{1}&a_{2}&a_{3}&0&0&0&0\\\ 0&a_{4}&a_{5}&a_{6}&0&0&0&0\\\
0&a_{7}&a_{8}&a_{9}&0&0&0&0\\\ 0&0&0&0&1&0&0&0\\\
0&0&0&0&0&a_{1}&a_{2}&a_{3}\\\ 0&0&0&0&0&a_{4}&a_{5}&a_{6}\\\
0&0&0&0&0&a_{7}&a_{8}&a_{9}\end{pmatrix},\quad{\rm
with}\begin{pmatrix}a_{1}&a_{2}&a_{3}\\\ a_{4}&a_{5}&a_{6}\\\
a_{7}&a_{8}&a_{9}\\\ \end{pmatrix}\in SU(3)_{R},\end{split}$ (3.12)

where we used that $t^{1}t^{2}t^{3}$ ($A^{1}A^{2}A^{3}$) is a $SU(3)_{R}$
singlet.

In this way, $S_{4}$ flavor symmetry naturally lies in the symplectic modular
flavor symmetry. Together with the CP transformation as discussed in Sec. 2.4,
the symplectic modular symmetry is generalized to the generalized symplectic
modular symmetry. It turns out that the $S_{4}$ flavor symmetry is enhanced to

$\displaystyle S_{4}\rtimes\mathbb{Z}_{2}^{\rm CP},$ (3.13)

in a generic moduli space of $\\{t^{a}\\}$. Here, $\mathbb{Z}_{2}^{\rm CP}$
transformation is given by Eqs. (2.70) and (2.71).

###  3.2 Small resolutions of $T^{6}/\mathbb{Z}_{3}$ orbifold

In this section, we incorporate the twisted (blow-up) modes in the effective
action on the 6D toroidal orbifolds. We mainly deal with the blown-up
$T^{6}/\mathbb{Z}_{3}$ orbifold, which is directly connected with the
effective action of untwisted modes in the previous section in the blow-down
limit, i.e., the vanishing blow-up radii. On the blown-up
$T^{6}/\mathbb{Z}_{3}$ geometry, only the Kähler structure deformations remain
on the blown-up $T^{6}/\mathbb{Z}_{3}$ background. The totally $h^{1,1}=36$
Kähler moduli include $h^{1,1}_{\rm untw}=9$ untwisted Kähler moduli and
$h^{1,1}_{\rm tw}=27$ twisted Kähler moduli $c^{d}$ ($d=1,2,\cdots,27$) from
the viewpoint of toroidal orbifold.141414The following discussion is
applicable to the T-dual of $T^{6}/\mathbb{Z}_{3}$ orbifold, where the
massless deformations correspond to the complex structure moduli accompanying
the $\macc@depth\char
1\relax\frozen@everymath{\macc@group}\macc@set@skewchar\macc@nested@a
111{27}_{i}$ matter fields. For more details, see, Ref. [41] and references
therein. For simplicity, let us focus on the three diagonal untwisted Kähler
moduli $t^{a}$ ($a=1,2,3$) whose moduli space is given by
$\Pi_{a}SL(2,\mathbb{Z})_{a}$. The prepotential receives the corrections from
the blow-up modes [26]

$\displaystyle G=t^{1}t^{2}t^{3}-\frac{i}{4}\sum_{d=1}^{27}(c^{d})^{2},$
(3.14)

and the following vector

$\displaystyle\begin{pmatrix}Y^{A}\\\ {\cal G}_{A}\\\
\end{pmatrix}=\begin{pmatrix}1\\\ t^{1}\\\ t^{2}\\\ t^{3}\\\ c^{d}\\\
2G-t^{A}\partial_{A}G\\\ t^{2}t^{3}\\\ t^{1}t^{3}\\\ t^{1}t^{2}\\\
-\frac{i}{2}c^{d}\end{pmatrix}=\begin{pmatrix}1\\\ t^{1}\\\ t^{2}\\\ t^{3}\\\
c^{d}\\\ -t^{1}t^{2}t^{3}\\\ t^{2}t^{3}\\\ t^{1}t^{3}\\\ t^{1}t^{2}\\\
-\frac{i}{2}c^{d}\end{pmatrix},$ (3.15)

provides the Kähler potential

$\displaystyle K_{\rm ks}=-\ln[-i(\macc@depth\char
1\relax\frozen@everymath{\macc@group}\macc@set@skewchar\macc@nested@a
111{Y}^{A}{\cal G}_{A}-Y^{A}\macc@depth\char
1\relax\frozen@everymath{\macc@group}\macc@set@skewchar\macc@nested@a
111{{\cal G}}_{A})\biggl{]}=-\ln[i(t^{1}-\macc@depth\char
1\relax\frozen@everymath{\macc@group}\macc@set@skewchar\macc@nested@a
111{t}^{1})(t^{2}-\macc@depth\char
1\relax\frozen@everymath{\macc@group}\macc@set@skewchar\macc@nested@a
111{t}^{2})(t^{3}-\macc@depth\char
1\relax\frozen@everymath{\macc@group}\macc@set@skewchar\macc@nested@a
111{t}^{3})-\sum_{d}|c^{d}|^{2}\biggl{]},$ (3.16)

which is consistent with Ref. [6]. Correspondingly, the matter Kähler metric
is calculated through Eq. (2.35).

The allowed Yukawa couplings of matter zero-modes are restricted by
$\mathbb{Z}_{3}$ symmetry. The matter superpotential takes the form

$\displaystyle
W=A^{1}A^{2}A^{3}-\sum_{d,e,f=1}^{27}f(t^{1},t^{2},t^{3})_{def}C^{d}C^{e}C^{f},$
(3.17)

where $C^{d}$ denotes the twisted matters associated twisted Kähler moduli,
and their Yukawa couplings $f(t^{1},t^{2},t^{3})_{def}$ are moduli-dependent
in general[42, 43, 44]. The holomorphic Yukawa couplings of untwisted modes
enjoy the $S_{4}$ flavor symmetry as previously discussed in the orbifold
regime, but the flavor symmetry of the twisted matter fields is complicated.
Before going into the detail of the Yukawa couplings among twisted matters, we
comment on the $S_{4}$ symmetry of twisted matter fields. Let us denote them
by fixed point labels, $C_{n_{1},n_{2},n_{3}}$, where $n_{i}=0,1,2$ represents
the fixed point label on the $i$-th $T^{2}/\mathbb{Z}_{3}$. The matter fields
$C_{n,n,n}$ are obviously $S_{4}$ singlets. The other $C_{n_{1},n_{2},n_{3}}$
are triplets because they transform each other under $S_{4}$. Thus, $27$
twisted matter fields decompose into three $S_{4}$ singlets and eight $S_{4}$
triplets.

In the blow-down limit treated in this section, the relevant modular symmetry
in the untwisted Kähler moduli space is
$\Pi_{a=1}^{3}SL(2,\mathbb{Z})_{a}\subset Sp(8,\mathbb{Z})$. It was known that
the Kähler metrics of untwisted and twisted matters take the form [26, 8]

$\displaystyle K_{a\bar{b}}^{(27)}\bigl{|}_{\rm untw}$ $\displaystyle=(K_{\rm
ks})_{a\bar{b}}e^{-K_{a}}=\frac{1}{i(t^{a}-\bar{t}^{a})}\delta_{a\bar{b}},$
$\displaystyle K_{d\bar{e}}^{(27)}\bigl{|}_{\rm tw}$ $\displaystyle=(K_{\rm
ks})_{d\bar{e}}e^{-K_{\rm
ks}/3}=\frac{1}{\left\\{i(t^{1}-\bar{t}^{1})(t^{2}-\bar{t}^{2})(t^{3}-\bar{t}^{3})\right\\}^{2/3}}\delta_{d\bar{e}},$
(3.18)

with $K_{a}=-\ln(-i(t^{a}-\bar{t}^{a}))$, from which we deduce the modular
weight of both matter fields:

$\displaystyle A^{1}\,$ $\displaystyle:\,\left\\{-1,0,0\right\\},\quad
A^{2}\,:\,\left\\{0,-1,0\right\\},\quad
A^{3}\,:\,\left\\{0,0,-1\right\\},\quad
C^{d}\,:\,\left\\{-\frac{2}{3},-\frac{2}{3},-\frac{2}{3}\right\\},$ (3.19)

under $\\{SL(2,\mathbb{Z})_{1},SL(2,\mathbb{Z})_{2},SL(2,\mathbb{Z})_{3}\\}$.
These modular weights play an important role in checking the modular
invariance of the action. Indeed, the matter superpotential of untwisted modes
$A^{1}A^{2}A^{3}$ is invariant under the $\Pi_{a}SL(2,\mathbb{Z})_{a}$ modular
symmetry due to its modular weight -1 under all $SL(2,\mathbb{Z})_{a}$.
However, 3-point couplings of twisted modes $C^{d}C^{e}C^{f}$ have modular
weight $-2$ under all $SL(2,\mathbb{Z})_{a}$. Hence, the modular invariance
requires the moduli-dependent couplings $f(t^{1},t^{2},t^{3})_{def}$ with
modular weight $1$ for each $SL(2,\mathbb{Z})_{a}$. It was known that the
twisted modes belong to the triplet representation of double cover of
$(A_{4})_{a}$, namely $(T^{\prime})_{a}\subset SL(2,\mathbb{Z})_{a}$ in the
blow-down limit[45, 6], although this $T^{\prime}$ triplet is a reducible
representation as seen below. Their holomorphic Yukawa couplings are expected
to be a $\Gamma(3)$ modular form of non-Abelian discrete group $A_{4}$, which
bring into the $(T^{\prime})_{a}$-invariant superpotential. We will come back
to this point later.

To see the $T^{\prime}$ structure in more detail, let us discuss $T^{\prime}$
transformations in the symplectic basis. Note that on this blown-up geometry,
$S_{4}$ symmetry among untwisted modes lies in $Sp(8,\mathbb{Z})\subset
Sp(62,\mathbb{Z})$ in a way similar to the previous analysis. For
concreteness, we focus on a subsector of twisted modes, i.e., three twisted
fields $\\{C^{1},C^{2},C^{3}\\}$ which transform under the
$SL(2,\mathbb{Z})_{1}$ symmetry. Then, the representations of these twisted
matters under the $S_{1}$ and $T_{1}$ transformations of
$SL(2,\mathbb{Z})_{1}$ are given by [6, 45]

$\displaystyle{\cal S}_{\rm tw}=-\frac{i}{\sqrt{3}}\begin{pmatrix}1&1&1\\\
1&w&w^{2}\\\ 1&w^{2}&w\end{pmatrix},\quad{\cal T}_{\rm
tw}=\begin{pmatrix}w&0&0\\\ 0&1&0\\\ 0&0&1\end{pmatrix},$ (3.20)

with $w=e^{2\pi i/3}$. They satisfy the following algebraic relations:

$\displaystyle{\cal S}_{\rm tw}^{2}=-\mathbb{I},\qquad({\cal S}_{\rm tw}{\cal
T}_{\rm tw})^{3}={\cal T}_{\rm tw}^{3}=\mathbb{I},$ (3.21)

which is the $T^{\prime}$ symmetry [37, 38]. Note that these matrices are
rotated into

$\displaystyle{\cal U}_{\rm tw}^{-1}{\cal S}_{\rm tw}{\cal U}_{\rm
tw}=-\frac{i}{\sqrt{3}}\begin{pmatrix}1&\sqrt{2}&0\\\ \sqrt{2}&-1&0\\\
0&0&i\end{pmatrix},\quad{\cal U}_{\rm tw}^{-1}{\cal T}_{\rm tw}{\cal U}_{\rm
tw}={\cal T}_{\rm tw},$ (3.22)

by the unitary transformation:

$\displaystyle{\cal U}_{\rm tw}=\begin{pmatrix}1&0&0\\\
0&\frac{1}{\sqrt{2}}&-\frac{1}{\sqrt{2}}\\\
0&\frac{1}{\sqrt{2}}&\frac{1}{\sqrt{2}}\end{pmatrix}.$ (3.23)

It indicates that $T^{\prime}$ triplet of our interest is a reducible
representation decomposed into the singlet and the doublet representations.

We find that these transformations lie in the symplectic modular group as
follows:

$\displaystyle\begin{split}S_{\rm
tw}&=\left(\begin{array}[]{cccccccccccc}0&-1&0&0&&0&0&0&0&\\\
1&0&0&0&&0&0&0&0&\\\ 0&0&0&0&&0&0&0&-1&\\\ 0&0&0&0&&0&0&-1&0&\\\ &&&&{\cal
S}_{\rm tw}&&&&&\\\ 0&0&0&0&&0&-1&0&0&\\\ 0&0&0&0&&1&0&0&0&\\\
0&0&0&1&&0&0&0&0&\\\ 0&0&1&0&&0&0&0&0&\\\ &&&&&&&&&{\cal S}_{\rm tw}\\\
\end{array}\right),\\\ T_{\rm
tw}&=\left(\begin{array}[]{cccccccccccc}1&0&0&0&&0&0&0&0&\\\
1&1&0&0&&0&0&0&0&\\\ 0&0&1&0&&0&0&0&0&\\\ 0&0&0&1&&0&0&0&0&\\\ &&&&{\cal
T}_{\rm tw}&&&&&\\\ 0&0&0&0&&1&-1&0&0&\\\ 0&0&0&0&&0&1&0&0&\\\
0&0&0&1&&0&0&1&0&\\\ 0&0&1&0&&0&0&0&1&\\\ &&&&&&&&&{\cal T}_{\rm tw}\\\
\end{array}\right),\end{split}$ (3.24)

where we omit 0 in the blank space for simplicity. Here, we focused on three
twisted modes, but it is straightforward to extend this analysis on fully
resolved toroidal orbifolds with small resolutions. In that case, the
effective action enjoys the $(T^{\prime})^{3}$ flavor symmetry belonging to
$Sp(62,\mathbb{Z})$ modular symmetry.

Similar to the untwisted modes, $T^{\prime}$ flavor symmetry naturally lies in
the symplectic modular flavor symmetry because it is a quotient of
$SL(2,\mathbb{Z})$. The twisted modes have non-trivial representations under

$\displaystyle(T^{\prime})^{3}\times S_{4}.$ (3.25)

Thus, their vacuum expectation values may lead non-trivial breaking of
$(T^{\prime})^{3}\times S_{4}$. Together with the CP transformation as
discussed in Sec. 2.4, symplectic modular symmetry is enlarged to the
generalized symplectic modular symmetry. In the current setup, each
$T^{\prime}$ flavor symmetry is enhanced to

$\displaystyle T^{\prime}\rtimes\mathbb{Z}_{2}^{\rm CP}\simeq S_{4},$ (3.26)

as an element of $GSp(62,\mathbb{Z})$. Note that $\mathbb{Z}_{2}^{\rm CP}$
transformation exchanges between the irreducible representations of
$T^{\prime}$, i.e., the outer automorphism of $T^{\prime}$.

To understand the nature of Yukawa couplings among twisted matters, we remind
the couplings of twisted matters on $T^{2}/\mathbb{Z}_{3}$ geometry whose
fixed points are labeled by $n=0,1,2$. The total 27 fixed points on
$T^{6}/\mathbb{Z}_{3}$ orbifold is reproduced by the product of three fixed
points on $T^{2}/\mathbb{Z}_{3}$. As a consequence of $\mathbb{Z}_{3}$
symmetry, Yukawa couplings of twisted modes localized on

$\displaystyle(0,0,0),\quad(1,1,1),\quad(2,2,2),\quad(0,1,2)$ (3.27)

are only allowed on $T^{2}/\mathbb{Z}_{3}$. In particular, their Yukawa
couplings are of the form[42, 43, 44]

$\displaystyle f(t^{1},t^{2},t^{3})_{def}\propto e^{-{\cal A}},$ (3.28)

where ${\cal A}$ denotes the area satisfying boundary conditions relevant to
twisted strings. Furthermore, these twisted strings wrap the cycle of toroidal
orbifolds many times. Summing up these effects, the Yukawa couplings

$\displaystyle f(t^{1},t^{2},t^{3})_{def}\propto\sum_{m}e^{-{\cal A}_{m}}$
(3.29)

will be elliptic functions such as theta functions as explicitly demonstrated
on orbifolds. Note that the leading terms of holomorphic Yukawa couplings for
$(n,n,n)$ with $n=0,1,2$ are constant, but the other one $(0,1,2)$ is
exponentially suppressed with respect to the torus radii due to the fact that
twisted strings propagate the fixed points with another. That is world-sheet
instanton effects. The couplings among the fixed points $(0,1,2)$ would vanish
if we neglect such world-sheet instanton effects.

Let us revisit these results in the context of symplectic modular symmetry,
focusing on the modular transformation of Yukawa couplings on
$T^{2}/\mathbb{Z}_{3}$. We denote the Yukawa coupling among three twisted
modes on the same fixed point by $Y_{0}$, while we denote the Yukawa couplings
for three different fixed points (0,1,2) and (2,1,0) by $Y_{1}$ and $Y_{2}$,
following Ref. [46]. The triplet $(Y_{0},Y_{1},Y_{2})$ transforms in the same
way as the three twisted modes, i.e., Eq. (3.20), up to an overall phase.
Thus, they decompose the singlet and doublet under $T^{\prime}$, but the
singlet corresponds to $(Y_{1}-Y_{2})$ and it vanishes automatically. The
doublet corresponds to the modular forms of weight 1 for $T^{\prime}$. Indeed,
the modular forms of weight 1 for $T^{\prime}$ have two degrees of freedom,
corresponding to the $T^{\prime}$ doublet [47]. Appendix A shows the modular
form of weight 1 for $T^{\prime}$ in a representation basis 151515 Similar to
twisted modes and their Yukawa couplings, zero-modes and their Yukawa
couplings in magnetized D-brane models have non-trivial behavior under the
modular symmetry [48, 49, 50, 51, 52, 53, 54, 55].. Indeed, this fact is well
in accord with our observations. The constant Yukawa couplings localized at
the fixed points $(n,n,n)$ with $n=0,1,2$ are the remnant of such modular
functions in the large volume regime, whereas other one at $(0,1,2)$ are
exponentially suppressed due to the structure of elliptic functions. The large
volume expansion is nothing but the $q=e^{2\pi it}$ expansion in the modular
form. So far, we have focused on $T^{2}/\mathbb{Z}_{3}$ orbifold, but the
structure discussed above holds for the Yukawa couplings of twisted modes
associated with 27 fixed points on $T^{6}/\mathbb{Z}_{3}$ orbifold.

In the limit that the values of blow-up moduli are sufficiently large, $27$
twisted modes may have the flavor symmetry $S_{27}$ in $Sp(54,\mathbb{Z})$
modular symmetry as the model in Sec. 3.4. However, in the blow-down orbifold
limit, the symmetry $S_{27}$ as well as $Sp(54,\mathbb{Z})$ loses the
geometrical meaning. However, twisted modes transform non-trivially under the
$SL(2,\mathbb{Z})$ modular group of untwisted modes as the $T^{\prime}$
reducible triplet because of the structure of prepotential (3.14) and Kähler
potential (3.16). That is a quite trivial transition of modular symmetric
behavior of the twisted modes. However, we have understood just two regimes,
i.e., sufficiently large blow-up moduli regime and the blow-down orbifold
limit. The middle regime is unclear in our analysis. We hope to report on the
whole picture of modular symmetric behavior in the future.

Finally, we comment on the other untwisted moduli whose moduli space is given
by $\frac{SU(3,3)}{SU(3)\times SU(3)\times U(1)}$. From the prepotential of
the untwisted modes

$\displaystyle G=\sum_{a,b,c=1}^{9}\kappa_{abc}t^{a}t^{b}t^{c}$ (3.30)

with

$\displaystyle\kappa_{123}=\kappa_{468}=\kappa_{579}=1,\quad\kappa_{169}=\kappa_{258}=\kappa_{347}=-1,$
(3.31)

and 0 otherwise, the Kähler potential of untwisted modes is provided by

$\displaystyle K_{\rm ks}=-\ln[-i\frac{\kappa_{abc}}{6}(t^{a}-\macc@depth\char
1\relax\frozen@everymath{\macc@group}\macc@set@skewchar\macc@nested@a
111{t}^{a})(t^{b}-\macc@depth\char
1\relax\frozen@everymath{\macc@group}\macc@set@skewchar\macc@nested@a
111{t}^{b})(t^{c}-\macc@depth\char
1\relax\frozen@everymath{\macc@group}\macc@set@skewchar\macc@nested@a
111{t}^{c})\biggl{]}.$ (3.32)

Here, we take the $Y^{0}=1$ gauge. It is also known that the matter Käher
potential and corresponding metric are given by[56]

$\displaystyle K$ $\displaystyle=-\ln\det(-i({\cal T}-\macc@depth\char
1\relax\frozen@everymath{\macc@group}\macc@set@skewchar\macc@nested@a
111{{\cal T}})-A_{\alpha}A_{\bar{\alpha}})^{\hat{a}\bar{\hat{b}}}=K_{\rm
ks}+(-i({\cal T}-\macc@depth\char
1\relax\frozen@everymath{\macc@group}\macc@set@skewchar\macc@nested@a
111{{\cal
T}}))^{-1}_{\hat{a}\bar{\hat{b}}}A^{\hat{a}}_{\alpha}A^{\bar{\hat{b}}}_{\bar{\alpha}}+{\cal
O}(|A|^{4}),$ $\displaystyle K_{\hat{a}\bar{\hat{b}}}^{(27)}$
$\displaystyle=(-i({\cal T}-\macc@depth\char
1\relax\frozen@everymath{\macc@group}\macc@set@skewchar\macc@nested@a
111{{\cal T}}))^{-1}_{\hat{a}\bar{\hat{b}}},$ (3.33)

where

$\displaystyle{\cal T}=\begin{pmatrix}t^{1}&t^{4}&t^{5}\\\
t^{7}&t^{2}&t^{6}\\\ t^{8}&t^{9}&t^{3}\\\ \end{pmatrix},$ (3.34)

with $\hat{a},\hat{b}=1,2,3$. Note that $\alpha$ stands for the indices of
$SU(3)\subset E_{8}$ gauge group which are different from the indices of
$SU(3)$ isometry $\hat{a}$. The superpotential of untwisted modes is written
by161616The kinetic mixing of matter fields will play an important role of
realizing the hierarchical structure of matter fields [57].

$\displaystyle
W=\sum_{\hat{a},\hat{b},\hat{c},\alpha,\beta,\gamma}\kappa_{\hat{a}\hat{b}\hat{c}}\epsilon^{\alpha\beta\gamma}A^{\hat{a}}_{\alpha}A^{\hat{b}}_{\beta}A^{\hat{c}}_{\gamma},$
(3.35)

with $\kappa_{\hat{1}\hat{2}\hat{3}}=1$ and 0 otherwise, up to the overall
factor, where $\epsilon^{\alpha\beta\gamma}$ are the anti-symmetric tensors of
$SU(3)\subset E_{8}$ gauge group, and the upper and lower indices correspond
to the $SU(3)$ isometry and $SU(3)\subset E_{8}$ gauge group, respectively.
The $S_{4}$ flavor symmetry appears in three untwisted matters, e.g.,
$\\{A_{1}^{\hat{1}},A_{1}^{\hat{2}},A_{1}^{\hat{3}}\\}$ which can be embedded
into the symplectic modular group following the previous procedures. In this
way, the 9 untwisted modes transform as 3 triplet representations under the
$S_{4}$ symmetry, that is,
$\\{A_{\alpha}^{\hat{a}},A_{\alpha}^{\hat{b}},A_{\alpha}^{\hat{c}}\\}$ for
arbitrary $\alpha$.

###  3.3 Small resolutions of
$T^{6}/(\mathbb{Z}_{3}\times\mathbb{Z}_{3}^{\prime})$ orbifold

We deal with another example to exemplify the existence of generalized
symplectic modular symmetry. We focus on the small resolutions of
$T^{6}/(\mathbb{Z}_{3}\times\mathbb{Z}_{3}^{\prime})$ orbifold by further
imposing $\mathbb{Z}_{3}^{\prime}$ projection on $T^{6}/\mathbb{Z}_{3}$
orbifold. By introducing the complex coordinates of $\mathbb{C}^{3}$ as
$z^{m}$, $m=1,2,3$, we identify them with

$\displaystyle z^{m}\simeq z^{m}+1,\quad z^{m}\simeq z^{m}+\beta,$ (3.36)

with $\beta=e^{2\pi i/6}$ to construct the three torus. Then, we specify the
$\mathbb{Z}_{3}\times\mathbb{Z}_{3}^{\prime}$ action following Ref. [33]:

$\displaystyle\mathbb{Z}_{3}\,$
$\displaystyle:\,Q(z^{m})\simeq\beta^{2}z^{m},$
$\displaystyle\mathbb{Z}_{3}^{\prime}\,$
$\displaystyle:\,Q^{\prime}(z^{m})\simeq\beta^{2m}z^{m}+\frac{1+\beta}{3},$
(3.37)

where $\mathbb{Z}_{3}^{\prime}$ is a freely acting symmetry. From 27 fixed
points under $\mathbb{Z}_{3}$ symmetry

$\displaystyle z_{\rm fix}=\frac{1+\beta}{3}(n_{1},n_{2},n_{3}),$ (3.38)

with $n_{m}=0,1,2$, we find 9 orbifold-invariant combinations:

$\displaystyle
f_{1}:(0,0,0)+(1,1,1)+(2,2,2),\,\,f_{2}:(0,1,2)+(1,2,0)+(2,0,1),\,\,f_{3}:(0,2,1)+(1,0,2)+(2,1,0),$
(3.39) $\displaystyle
f_{4}:(1,0,0)+(2,1,1)+(0,2,2),\,\,f_{5}:(0,1,0)+(1,2,1)+(2,0,2),\,\,f_{6}:(0,0,1)+(1,1,2)+(2,2,0),$
(3.40) $\displaystyle
f_{7}:(2,0,0)+(0,1,1)+(1,2,2),\,\,f_{8}:(0,2,0)+(1,0,1)+(2,1,2),\,\,f_{9}:(0,0,2)+(1,1,0)+(2,2,1).$
(3.41)

Hence, there exist $h^{1,1}_{\rm tw}=9$ twisted Kähler moduli in addition to
$h^{1,1}_{\rm untw}=3$ untwisted Kähler moduli.

The orbifold projection (3.37) breaks $(T^{\prime})^{3}$ symmetry in Sec. 3.2
to $S_{3}$. For instance, Eq. (3.39) represents the $S_{3}$ singlet, and Eqs.
(3.40) and (3.41) become $S_{3}$ triplets, although the triplet representation
is the reducible representation decomposed into the single and doublet. In
addition, $27$ twisted modes on $T^{6}/\mathbb{Z}_{3}$ correspond to three
singlets and eight triplets under the $S_{4}$ flavor symmetry of untwisted
modes, where the matter fields $C_{n,n,n}$ correspond to the $S_{4}$ singlets,
while the other $C_{n_{1},n_{2},n_{3}}$ are $S_{4}$ triplets. All of the
matter fields $C_{n,n,n}$ correspond to three fixed points in the linear
combination $f_{1}$. Here, the matter fields corresponding to both $f_{2}$ and
$f_{3}$ become $S_{4}$ singlets because of the orbifold projection (3.37) . On
the other hand, the matter fields corresponding to $(f_{4},f_{5},f_{6})$ and
$(f_{7},f_{8},f_{9})$ still represent the $S_{4}$ triplets. Thus, the orbifold
projection (3.37) reduces three $S_{4}$ singlets and eight $S_{4}$ triplets to
three $S_{4}$ singlets and two $S_{4}$ triplets. From the total $Sp(2\times
12+2,\mathbb{Z})$ symplectic group, we can derive the $S_{3}$ flavor group,

$\displaystyle Sp(26,\mathbb{Z})\supset G_{\rm flavor}=S_{3},$ (3.42)

where the embedding $S_{3}\subset Sp(8,\mathbb{Z})\subset Sp(26,\mathbb{Z})$
is realized by following the same step in the previous section. The $S_{3}$
generator corresponds to $P$ in Eq. (3.5). Then, the $S_{3}$ flavor symmetry
is enlarged to $D_{6}\simeq S_{3}\rtimes\mathbb{Z}_{2}^{\rm CP}\subset
GSp(26,\mathbb{Z})$, taking into account the CP symmetry. Here,
$\mathbb{Z}_{2}^{\rm CP}$ transformation is given by Eqs. (2.70) and (2.71).

In the following, we move to the geometrical regime where the size of blow-up
radii is above the string length and check whether there exist such discrete
modular flavor symmetries.

###  3.4 Geometrical regime of blown-up
$T^{6}/(\mathbb{Z}_{3}\times\mathbb{Z}_{3}^{\prime})$ orbifold

In the previous sections, we examined the blown-up toroidal orbifolds with
small resolutions, where the $S_{3}$, $S_{4}$, and $T^{\prime}$ flavor
symmetries among untwisted and twisted matters are independent. Together with
the CP symmetry, all of them belong to the generalized symplectic modular
symmetry and are enlarged to $D_{6}$, $S_{4}\rtimes\mathbb{Z}_{2}^{\rm CP}$
and $S_{4}$, respectively.

In contrast to the previous analysis assuming the small vacuum expectation
values of blow-up modes, we discuss more non-trivial examples on smooth CY
threefolds. In this section, we focus on
$T^{6}/(\mathbb{Z}_{3}\times\mathbb{Z}_{3}^{\prime})$ background, where the
blown-up cycles are larger than 1 in string units, in comparison with the
analysis of Sec. 3.3. Among $h^{1,1}=12$ Kähler moduli, there exist
$h^{1,1}_{\rm untw}=3$ untwisted Kähler moduli spanned by $e_{a}$
corresponding to $idz^{a}\wedge d\macc@depth\char
1\relax\frozen@everymath{\macc@group}\macc@set@skewchar\macc@nested@a
111{z}^{\macc@depth\char
1\relax\frozen@everymath{\macc@group}\macc@set@skewchar\macc@nested@a 111{b}}$
from the viewpoint of toroidal orbifold, where $z^{a}$ ($a=1,2,3$) denote the
coordinates of 3-torus. In addition, we have $h^{1,1}_{\rm tw}=9$ twisted
Kähler moduli spanned by $e_{r}$ $(r=1,2,...,9)$, corresponding to complex
projective spaces $\mathbb{P}^{2}$’s, where fixed points are now resolved as
$\mathbb{P}^{2}$ of a $\mathbb{C}^{3}/\mathbb{Z}_{3}$ singularity. On this
geometry, there is no complex structure deformation.

When we expand the Kähler form

$\displaystyle J=\sum_{a=1}^{3}t^{a}e_{a}+\sum_{r=1}^{9}v^{r}e_{r},$ (3.43)

the Kähler potential is evaluated as [58]

$\displaystyle\begin{split}K_{\rm ks}&=-\ln\int J_{c}\wedge J_{c}\wedge
J_{c}=-\ln\frac{1}{8}\biggl{[}81(t^{1}-\macc@depth\char
1\relax\frozen@everymath{\macc@group}\macc@set@skewchar\macc@nested@a
111{t}^{1})(t^{2}-\macc@depth\char
1\relax\frozen@everymath{\macc@group}\macc@set@skewchar\macc@nested@a
111{t}^{2})(t^{3}-\macc@depth\char
1\relax\frozen@everymath{\macc@group}\macc@set@skewchar\macc@nested@a
111{t}^{3})-\frac{3}{2}\sum_{r}(v^{r}-\macc@depth\char
1\relax\frozen@everymath{\macc@group}\macc@set@skewchar\macc@nested@a
111{v}^{r})^{3}\biggl{]}.\end{split}$ (3.44)

Here, the prepotential is given by

$\displaystyle
G=\frac{\kappa_{ABC}}{6}t^{A}t^{B}t^{C}=81t^{1}t^{2}t^{3}-\frac{3}{2}\sum_{r}(v^{r})^{3},$
(3.45)

with the nonvanishing triple intersection numbers,

$\displaystyle\kappa_{123}=81,\quad\kappa_{rrr}=-9.$ (3.46)

Hence, the corresponding matter superpotential takes the form

$\displaystyle W=81A^{1}A^{2}A^{3}-9\sum_{r=1}^{9}(V^{r})^{3},$ (3.47)

where $A^{1,2,3}$ and $V^{r}$ denote the matter fields associated with the
untwisted and twisted moduli fields from the viewpoint of toroidal orbifolds,
respectively. The matter Kähler metric is calculated through Eq. (2.35). We
find that the matter superpotential is invariant under two flavor symmetries:
$S_{4}$ symmetry for the untwisted matters $\\{A^{1},A^{2},A^{3}\\}$ and
$S_{9}$ symmetry for the twisted matters $\\{V^{1},V^{2},...,V^{9}\\}$ in
addition to the overall modular symmetry $SL(2,\mathbb{Z})_{\rm overall}$.

To see the relation between flavor symmetries of matter fields and symplectic
modular symmetry, let us consider the period vector providing the Kähler
potential (3.44). Since the blown-up
$T^{6}/(\mathbb{Z}_{3}\times\mathbb{Z}_{3}^{\prime})$ manifold have totally 12
Kähler moduli fields, the moduli space is described by $Sp(26,\mathbb{Z})$
symplectic modular symmetry. Correspondingly, the period vector is given by

$\displaystyle\begin{pmatrix}Y^{A}\\\ {\cal G}_{A}\\\
\end{pmatrix}=\begin{pmatrix}1\\\ t^{1}\\\ t^{2}\\\ t^{3}\\\ v^{r}\\\
2G-t^{A}\partial_{A}G\\\ 81t^{2}t^{3}\\\ 81t^{1}t^{3}\\\ 81t^{1}t^{2}\\\
-\frac{9}{2}(v^{r})^{2}\end{pmatrix}=\begin{pmatrix}1\\\ t^{1}\\\ t^{2}\\\
t^{3}\\\ v^{a}\\\ -81t^{1}t^{2}t^{3}+\frac{3}{2}\sum_{r}(v^{r})^{3}\\\
81t^{2}t^{3}\\\ 81t^{1}t^{3}\\\ 81t^{1}t^{2}\\\
-\frac{9}{2}(v^{r})^{2}\end{pmatrix}.$ (3.48)

Similar to the previous example, the $S_{4}$ flavor symmetry of the untwisted
moduli $\\{t^{1},t^{2},t^{3}\\}$ can be embedded into $Sp(8,\mathbb{Z})\subset
Sp(26,\mathbb{Z})$. Accordingly, the matter fields $A^{1,2,3}$ transform as in
Eq. (3.10). In addition, we find that the generators of $S_{9}$

$\displaystyle{\cal P}_{rs}:v^{r}\leftrightarrow v^{s}$ (3.49)

is the element of $Sp(26,\mathbb{Z})$ as follows. For an illustrative purpose,
let us focus on ${\cal P}_{12}$ exchanging $v^{1}$ and $v^{2}$. To shorten the
expression of the generator ${\cal P}_{12}$, we focus on the relevant period
vector transformed under ${\cal P}_{12}$,

$\displaystyle\begin{pmatrix}v^{1}\\\ v^{2}\\\ -\frac{9}{2}(v^{1})^{2}\\\
-\frac{9}{2}(v^{2})^{2}\end{pmatrix},$ (3.50)

on which ${\cal P}_{12}$ acts on

$\displaystyle\begin{split}\begin{pmatrix}0&1&0&0\\\ 1&0&0&0\\\ 0&0&0&1\\\
0&0&1&0\\\ \end{pmatrix}\begin{pmatrix}v^{1}\\\ v^{2}\\\
-\frac{9}{2}(v^{1})^{2}\\\
-\frac{9}{2}(v^{2})^{2}\end{pmatrix}=\begin{pmatrix}v^{2}\\\ v^{1}\\\
-\frac{9}{2}(v^{2})^{2}\\\ -\frac{9}{2}(v^{1})^{2}\end{pmatrix}.\end{split}$
(3.51)

${\cal P}_{12}$ acts the identity operators on the other elements of period
vector. In this way, we show that the generator ${\cal P}_{12}$ is the element
of $Sp(26,\mathbb{Z})$, and it is possible to prove the other ${\cal P}_{rs}$
as the element of $Sp(26,\mathbb{Z})$ by changing the indices of $v^{r,s}$.

These transformations are directly related to the $S_{9}$ flavor symmetry of
matter fields from ${\cal P}_{rs}$. Indeed, the submatrix of ${\cal P}_{rs}$
acting on $(v^{1},v^{2})^{T}$

$\displaystyle\begin{pmatrix}0&1\\\ 1&0\\\ \end{pmatrix}$ (3.52)

corresponds to the transformation between $V^{1}$ and $V^{2}$. Combining all
${\cal P}_{ab}$ acting on

$\displaystyle(v^{1},v^{2},v^{3},v^{4},v^{5},v^{6},v^{7},v^{8},v^{9})^{T},$
(3.53)

one can realize the transformations of matter fields under $S_{9}$ modular
flavor symmetry.

We conclude that there exist $G_{\rm flavor}=S_{4}\times S_{9}$ flavor
symmetries in the effective action of blown-up
$T^{6}/(\mathbb{Z}_{3}\times\mathbb{Z}_{3}^{\prime})$ orbifold. They originate
from the $Sp(26,\mathbb{Z})$ symplectic group,

$\displaystyle Sp(26,\mathbb{Z})\supset G_{\rm flavor}=S_{4}\times S_{9},$
(3.54)

where $S_{9}$ symmetry is broken down to $S_{3}$ in the regime with small
resolutions as in Sec. 3.3. That would be related to the spontaneous breaking
of flavor symmetry triggered by the blow-up modes. We will leave the more
detailed discussion of this symmetry breaking involving the stabilization of
blow-up modes to future work. When we take into account the CP transformation,
two flavor symmetries are enhanced to

$\displaystyle S_{4}\rtimes\mathbb{Z}_{2}^{\rm CP},\quad
S_{9}\rtimes\mathbb{Z}_{2}^{\rm CP}.$ (3.55)

In the next section, we show the existence of non-Abelian discrete flavor
symmetries on other classes of CY threefolds.

This example suggests that some class of models with $h$ blow-up modes may
lead to the superpotential

$\displaystyle W=\sum_{i=1}^{h}(A^{i})^{3},$ (3.56)

up to an overall factor. This superpotential seems to have the $S_{h}$ flavor
symmetry, which is the permutation among $A^{i}$. This type of superpotential
has a specific property for $h=3$. We start with the following superpotential:

$\displaystyle W=\tilde{A}^{1}\tilde{A}^{2}\tilde{A}^{3}.$ (3.57)

Here, we redefine the matter fields,

$\displaystyle\tilde{A}^{1}$ $\displaystyle=A^{1}+A^{2}+A^{3},$
$\displaystyle\tilde{A}^{1}$ $\displaystyle=A^{1}+\omega
A^{2}+\omega^{2}A^{3},$ (3.58) $\displaystyle\tilde{A}^{1}$
$\displaystyle=A^{1}+\omega^{2}A^{2}+\omega A^{3}.$

Then, we can write

$\displaystyle
W=\tilde{A}^{1}\tilde{A}^{2}\tilde{A}^{3}=(A^{1})^{3}+(A^{2})^{2}+(A^{3})^{2}.$
(3.59)

The $S_{3}$ symmetry is enlarged to $S_{4}$. Hence, this phenomenon can be
observed only for $h=3$.

###  3.5 Three-parameter examples of Calabi-Yau threefolds

So far, we have focused on the toroidal orbifolds with and without
resolutions. In this section, we analyze the smooth CY threefolds, emphasizing
the flavor symmetry of matter fields. In particular, we deal with the Kreuzer-
Skarke data set of CY threefolds, given by total 473,800,776 ambient toric
fourfolds [59]. The ambient toric varieties are described by the corresponding
(reflexive) polytope, which might involve singularities in general. The smooth
CY hypersurfaces are obtained through “triangulation” of the polytope where
the singularities of the ambient space are resolved. For the favorable CY
hypersurface $X$171717The favorable CY means that the hodge number $h^{1,1}$
of $X$ is the same with that of the ambient space., the harmonic (1,1)-forms
for $H^{2}(X,\mathbb{Z})$, $J_{r}$ with $r=1,2,\cdots,h^{1,1}$, can be
obtained by the pull-backs of (1,1)-forms on the ambient space. Then, the
intersection numbers $d_{rst}$ of $X$ can be calculated by

$\displaystyle d_{rst}=\int_{X}J_{r}\wedge J_{s}\wedge J_{t}.$ (3.60)

Here, the $J_{r}$ stand for the basis of the Picard group. Finally, we
describe the Kähler cone of $X$, determining the allowed Kähler moduli
$t^{r}$. For the favorable CY hypersurface, the Kähler cone of $X$ is also
derived from that of the ambient space whose Kähler cone is calculated by the
data of polytope. When we expand the (1,1)-form $J$ as $J=t^{r}J_{r}$, the
Kähler moduli $t^{r}$ satisfy

$\displaystyle\sum_{r}K^{\hat{r}}_{r}t^{r}\geq 0,$ (3.61)

where we introduce the Kähler cone matrix $K=[K^{\hat{r}}_{r}]$ and $\hat{r}$
runs over the facets of the Kähler cone. Note that the two examples we analyze
later have a few different triangulation corresponding to a partial resolution
for the ambient toric varieties. In that case, the Kähler cone is given by the
union of several sub-cones:

$\displaystyle K=\bigcup_{j}K_{j}.$ (3.62)

Among them, only 16 manifolds have a non-trivial fundamental group which is an
important ingredient in introducing Wilson-lines on quotient CY threefolds
[60]. We focus on these particular class of CY threefolds in the context of
heterotic string theory with standard embedding.181818The heterotic string
models in the context of non-standard embedding are discussed on such a
particular class of CY threefolds [61]. For illustrative purposes, we deal
with five quotient CY threefolds. (For more details about the toric data as
well as topological data of CY threefolds, we refer to Ref. [61].) In
particular, we focus on the Kähler moduli sector by assuming that the complex
structure moduli sector is stabilized by some mechanism at a high scale.

  1. 1.

The first example of CY threefold with non-trivial fundamental group has
$h^{1,1}=3$ Kähler moduli, $h^{2,1}=59$ complex structure moduli, i.e., Euler
number $\chi=-112$ (corresponding to the “downstairs” CY threefold $X_{14}$ in
Ref. [61]). We will focus on the Kähler moduli sector since the complex
structure moduli sector is complicated to analyze. The prepotential in the
Kähler moduli sector is given by

$\displaystyle G=t^{1}t^{2}t^{3},$ (3.63)

with non-vanishing triple intersection number $\kappa_{123}=1$. The Kähler
cone is given by $K={\rm diag}(1,1,1)$. Hence, the superpotential of matter
fields associated with the Kähler moduli has the same functional form as one
in the example of Sec. 3.2,

$\displaystyle W=A^{1}A^{2}A^{3}.$ (3.64)

Hence, in addition to $\Pi_{a=1}^{3}SL(2,\mathbb{Z})_{a}$, there exists the
$S_{4}$ flavor symmetry, which can be enhanced to
$S_{4}\rtimes\mathbb{Z}_{2}^{CP}$ symmetry, taking into account the CP
transformation. The symmetry is then embedded into the generalized symplectic
$Sp(8,\mathbb{Z})$ flavor symmetry in the Kähler moduli space.

  2. 2.

The second example of three Kähler parameters of CY has $h^{2,1}=59$ complex
structure moduli, i.e., Euler number $\chi=-112$ (corresponding to the
downstairs CY threefold $X_{6}$ in Ref. [61]). When we focus on the Kähler
moduli sector, the prepotential is given by

$\displaystyle G=t^{1}t^{2}t^{3}-2t^{2}(t^{3})^{2},$ (3.65)

with non-vanishing triple intersection numbers $\kappa_{123}=1$ and
$\kappa_{233}=-4$. Here, the intersection number is negative, but it is
possible to redefine the modulus field as $\widetilde{t}^{1}\equiv
t^{1}-2t^{3}$, corresponding to the Kähler cone $K={\rm diag}(1,1,1)$. It
provides the superpotential of matter fields associated with the Kähler
moduli,

$\displaystyle W=A^{1}A^{2}A^{3}-2A^{2}(A^{3})^{2},$ (3.66)

which has a flavor structure similar to the first example of CY. Indeed, when
we redefine the matter fields as $\widetilde{A}^{1}\equiv A^{1}-2A^{3}$, the
matter superpotential reduces to

$\displaystyle W=\widetilde{A}^{1}A^{2}A^{3}.$ (3.67)

Hence, we find that $S_{4}$ flavor symmetry is realized on this class of CY
threefold and its enhanced $S_{4}\rtimes\mathbb{Z}_{2}^{CP}$ symmetry together
with the CP transformation can be embedded into the generalized symplectic
$Sp(8,\mathbb{Z})$ flavor symmetry.

  3. 3.

The third example of three Kähler parameters of CY has $h^{2,1}=43$ complex
structure moduli, i.e., Euler number $\chi=-80$ (corresponding to the
downstairs CY threefold $X_{5}$ in Ref. [61]). The prepotential in the Kähler
moduli sector is given by

$\displaystyle
G=t^{1}t^{2}t^{3}-2t^{1}(t^{3})^{2}+2t^{2}(t^{3})^{2}-8(t^{3})^{3},$ (3.68)

with non-vanishing triple intersection numbers $\kappa_{123}=1$,
$\kappa_{133}=-4$, $\kappa_{233}=4$ and $\kappa_{333}=-48$. Here, the Kähler
cone is given by

$\displaystyle K=\begin{pmatrix}0&1&-2\\\ 1&0&0\\\ 0&0&1\\\ \end{pmatrix},$
(3.69)

constraining the Kähler moduli to satisfy Eq. (3.61). It provides the
superpotential of matter fields associated with the Kähler moduli,

$\displaystyle
W=A^{1}A^{2}A^{3}-2A^{1}(A^{3})^{2}+2A^{2}(A^{3})^{2}-8(A^{3})^{3}.$ (3.70)

There is the overall modular symmetry $SL(2,\mathbb{Z})_{\rm overall}$. It
seems that other non-trivial flavor symmetries are absent in this system.
There is $\mathbb{Z}_{2}$ symmetry such as

$\displaystyle A^{1}\to A^{2},\qquad A^{2}\to A^{1},\qquad A^{3}\to-A^{3},$
(3.71)

up to $U(1)_{R}$ symmetry $W\to-W$.

  4. 4.

The fourth example of three Kähler parameters of CY has $h^{2,1}=75$ complex
structure moduli, i.e., Euler number $\chi=-144$ (corresponding to the
downstairs CY threefold $X_{7}$ in Ref. [61]). The prepotential in the Kähler
moduli sector is given by

$\displaystyle
G=t^{1}t^{2}t^{3}-2(t^{2})^{2}t^{3}-2t^{1}(t^{3})^{2}+8(t^{3})^{2},$ (3.72)

with non-vanishing triple intersection numbers $\kappa_{123}=1$,
$\kappa_{223}=\kappa_{133}=-4$ and $\kappa_{333}=48$. Here, the Kähler cone is
given by

$\displaystyle K=\begin{pmatrix}1&-2&0\\\ 0&0&1\\\ 0&1&-2\\\ \end{pmatrix},$
(3.73)

constraining the Kähler moduli to satisfy Eq. (3.61). It provides the
superpotential of matter fields associated with the Kähler moduli,

$\displaystyle
W=A^{1}A^{2}A^{3}-2(A^{2})^{2}A^{3}-2A^{1}(A^{3})^{2}+8(A^{3})^{2}.$ (3.74)

There is the overall modular symmetry $SL(2,\mathbb{Z})_{\rm overall}$, but
other non-trivial flavor symmetries are absent in this system.

  5. 5.

The last example of three Kähler parameters of CY has $h^{2,1}=75$ complex
structure moduli, i.e., Euler number $\chi=-144$ (corresponding to the
downstairs CY threefold $X_{8}$ in Ref. [61]). The prepotential in the Kähler
moduli sector is given by

$\displaystyle
G=t^{1}t^{2}t^{3}-2t^{1}(t^{3})^{2}-2t^{2}(t^{3})^{2}+8(t^{3})^{2},$ (3.75)

with non-vanishing triple intersection numbers $\kappa_{123}=1$,
$\kappa_{133}=\kappa_{233}=-4$ and $\kappa_{333}=48$. Here, the Kähler cone is
given by

$\displaystyle K=\begin{pmatrix}0&0&1\\\ 1&0&-2\\\ 0&1&-2\\\ \end{pmatrix},$
(3.76)

constraining the Kähler moduli to satisfy Eq. (3.61). It provides the
superpotential of matter fields associated with the Kähler moduli,

$\displaystyle
W=A^{1}A^{2}A^{3}-2A^{1}(A^{3})^{2}-2A^{2}(A^{3})^{2}+8(A^{3})^{2}.$ (3.77)

There is the overall modular symmetry $SL(2,\mathbb{Z})_{\rm overall}$. In
addition, there exists a $\mathbb{Z}_{2}\simeq S_{2}$ flavor symmetry between
$A^{1}$ and $A^{2}$ and it is further enhanced to
$\mathbb{Z}_{2}\rtimes\mathbb{Z}_{2}^{CP}$ symmetry, taking into account the
CP transformation. Since the $\mathbb{Z}_{2}$ transformation is the same with
${\cal P}_{12}$ in Eq. (3.51), the symmetry is then embedded into the
$Sp(8,\mathbb{Z})$ generalized symplectic flavor symmetry.

We find that non-trivial flavor symmetries on three-parameter examples of CY
threefolds are given by $S_{4}$ and $\mathbb{Z}_{2}$ symmetries, enlarging to
$S_{4}\rtimes\mathbb{Z}_{2}^{\rm CP}$ and
$\mathbb{Z}_{2}\rtimes\mathbb{Z}_{2}^{\rm CP}$ taking into account the
$\mathbb{Z}_{2}^{\rm CP}$ symmetry. Hence, the $S_{4}$ flavor symmetry appears
in not only the exact orbifold limit but also a certain class of CY threefolds
such as the first and second examples. In addition, all examples enjoy the
overall modular symmetry $SL(2,\mathbb{Z})_{\rm overall}$, which looks
$\mathbb{Z}_{3}$ symmetry, $A^{i}\to e^{2\pi i/3}A^{i}$ in the field-
theoretical viewpoint. The above $S_{4}$ and $\mathbb{Z}_{2}$ symmetries are
commutable with this $\mathbb{Z}_{3}$ symmetry.

Our analysis is limited to specific CY threefolds, but it is interesting to
examine the flavor symmetries on a more broad class of CY threefolds. We leave
it for future work.

##  4 Conclusions and discussions

In this paper, we examined the origin of flavor symmetry in the context of
heterotic string theory on CY threefolds, including toroidal orbifolds. The
flavor structure of matter zero-modes in the context of heterotic string
theory with standard embedding is governed by the geometrical structure of CY
threefolds because the matter and moduli fields are in one-to-one
correspondence with each other. It indicates that the flavor structure is
closely related to the symplectic structure of CY moduli spaces. Furthermore,
the 4D CP and the $U(1)_{R}$ symmetries were regarded as the outer
automorphism of the symplectic modular group [13] and the rotation of
holomorphic three-form of CY threefolds [19], respectively. These observations
motivate us to thoroughly investigate the relation among flavor, CP, and
$U(1)_{R}$ symmetries on the basis of 4D effective action. This approach would
yield the ultra-violet origin of models with modular flavor symmetry in the
bottom-up approach.

In Sec. 2, we gave a unified picture of these flavor, CP, and $U(1)_{R}$
symmetries from the viewpoint of generalized symplectic symmetry
$GSp(2h+2,\mathbb{C})\simeq Sp(2h+2,\mathbb{C})\rtimes\mathbb{Z}_{2}^{\rm
CP}\times U(1)_{R}$, with $h$ being the number of either complex structure
moduli or Kähler moduli. It was known that the traditional flavor symmetry was
reflected by the flavor structure of Yukawa couplings, but our results suggest
that the flavor symmetry $G_{\rm flavor}$ is enlarged to some non-Abelian
groups $G_{\rm flavor}\rtimes\mathbb{Z}_{2}^{\rm CP}$ by the CP symmetry on a
general class of CY threefolds with standard embedding. To exemplify these
results, we deal with toroidal orbifolds with and without resolutions, and
three-parameter examples of CY threefolds. It turned out that three untwisted
modes associated with three Kähler moduli enjoy $S_{4}$ flavor symmetry on
toroidal orbifolds. On the $T^{6}/\mathbb{Z}_{3}$ and
$T^{6}/(\mathbb{Z}_{3}\times\mathbb{Z}_{3}^{\prime})$ orbifolds with small
resolutions, there exist $(T^{\prime})^{3}$ and $S_{3}$ symmetries among
twisted modes, respectively. On the geometrical regime of
$T^{6}/(\mathbb{Z}_{3}\times\mathbb{Z}_{3}^{\prime})$ orbifold where the size
of blow-up modes is larger than the string length, we find the $S_{9}$ flavor
symmetry among blow-up models (corresponding to twisted modes in the blow-down
limit). The three-parameter examples of CY threefolds also enjoy
$\mathbb{Z}_{2}$ and $S_{4}$ flavor symmetries as discussed in Sec. 3.5.
Hence, non-trivial flavor symmetries such as $S_{4}$ appear in not only the
exact orbifold limit but also a certain class of CY threefolds. All the flavor
symmetries we analyzed have an origin in the symplectic modular symmetry and
are enlarged to non-Abelian groups by the CP symmetry.

In this paper, we have proposed the unification of flavor, CP, and $U(1)_{R}$
symmetries originated from the symplectic modular symmetry, and exemplify the
flavor symmetries on CY threefolds, including toroidal results. However, there
exist several future directions to pursue.

  * •

Yukawa couplings on CY threefolds

On the toroidal backgrounds, moduli-dependent Yukawa couplings are estimated
by the modular weight of matter fields, and their functional form is described
by a certain modular form of $SL(2,\mathbb{Z})$ symmetry. On the other hand,
the results of Sec. 3.5 exhibit the existence of $S_{4}$ flavor symmetry of
matter fields on some CY threefolds equipping the effective $SL(2,\mathbb{Z})$
modular symmetry. Hence, the Yukawa couplings including quantum corrections
would be described by the modular form of $S_{4}$ flavor symmetry, controlled
by the underlying symplectic modular symmetry. In this paper, we have focused
on the CY moduli spaces at the classical level, but loop and instanton
corrections calculated in the topological string theory appear in the
prepotential as well as Yukawa couplings [62]. Also, massive modes generate
the higher-order couplings, which are described in an appropriate derivative
of the prepotential [63]. It would be fascinating to figure out the flavor
structure of matter fields, including these corrections. We hope to report on
this interesting direction in the future.

  * •

Spontaneous breaking of flavor symmetry

From the results in Secs. 3.3 and 3.4, there the $S_{3}$ and $S_{9}$ flavor
symmetries of twisted modes on
$T^{6}/(\mathbb{Z}_{3}\times\mathbb{Z}_{3}^{\prime})$ orbifold with small
resolutions and its geometrical regime, respectively. It indicates the
spontaneous breaking of flavor symmetry would be induced by the blow-up modes.
To understand the underlying structure, it is required to interpolate both the
orbifold and geometrical regime. Including the stabilization of blow-up modes,
we leave this topic for future research.

  * •

Modular symmetric behavior of the twisted modes

In the limit that the size of blow-up moduli is sufficiently large, 9 twisted
modes in the example of Sec. 3.4 have $S_{9}$ flavor symmetry in
$Sp(18,\mathbb{Z})$ modular symmetry. Similarly, 27 twisted modes in the
example of Sec. 3.2 may also have the $S_{27}$ flavor symmetry in
$Sp(54,\mathbb{Z})$ modular symmetry. On the other hand, the symmetry $S_{27}$
as well as $Sp(54,\mathbb{Z})$ loses the geometrically meaning in the blow-
down orbifold limit. However, twisted modes transform non-trivially under the
$SL(2,\mathbb{Z})$ modular symmetry of untwisted modes because of the
structure of prepotential and Kähler potential. That is a quite trivial
transition of modular symmetric behavior of the twisted modes. In this paper,
we have understood just two regimes, i.e., sufficiently large blow-up moduli
regime $({\rm Im}(v^{r})\gg 1)$ and the blow-down orbifold limit $(|c^{r}|\ll
1)$ in the context of symplectic modular symmetry. The middle regime is
unclear in our analysis. It was conjectured by Ref. [64] that the blow-up
moduli in both regimes would be related by $c^{r}\simeq\exp(2\pi iv^{r})$. It
is interesting to find out a similar relation of twisted matters, which brings
us to understand the whole picture of modular symmetric behavior.

### Acknowledgements

The authors would like to thank Hiroshi Ohki for useful discussions, and
especially to the anonymous referee for a careful reading of the manuscript
and constructive suggestions. H. O. was supported in part by JSPS KAKENHI
Grant Numbers JP19J00664 and JP20K14477.

##  Appendix A Modular form of weight 1 for $T^{\prime}$

Following Ref. [47], we define the following functions,

$\displaystyle\hat{e}_{1}(\tau)=\frac{\eta^{3}(3\tau)}{\eta(\tau)},\qquad\hat{e}_{2}(\tau)=\frac{\eta^{3}(\tau/3)}{\eta(\tau)},$
(A.1)

by using the Dedekind eta function $\eta(\tau)$,

$\displaystyle\eta(\tau)=q^{1/24}\prod^{\infty}_{n=1}(1-q^{n}),$ (A.2)

with $q=e^{2\pi i\tau}$. Under the $T$-transformation, they transform

$\displaystyle\hat{e}_{1}\rightarrow\omega\hat{e}_{1},\qquad\hat{e}_{2}\rightarrow
3(1-\omega)\hat{e}_{1}+\hat{e}_{2},$ (A.3)

with $w=e^{2\pi i/3}$. Under the $S$-transformation, they transform

$\displaystyle\hat{e}_{1}\rightarrow
3^{-3/2}i(-\tau)\hat{e}_{2},\qquad\hat{e}_{2}\rightarrow
3^{3/2}i(-\tau)\hat{e}_{1}.$ (A.4)

By using these functions, we can construct the doublet of $T^{\prime}$, which
transforms under the $T$-transformation,

$\displaystyle\left(\begin{array}[]{c}x_{1}\\\
y_{1}\end{array}\right)\rightarrow\rho(T)\left(\begin{array}[]{c}x_{1}\\\
y_{1}\end{array}\right),$ (A.9)

and under the $S$-transformation

$\displaystyle\left(\begin{array}[]{c}x_{1}\\\
y_{1}\end{array}\right)\rightarrow(-\tau)\rho(S)\left(\begin{array}[]{c}x_{1}\\\
y_{1}\end{array}\right),$ (A.14)

where

$\displaystyle\rho(T)=\left(\begin{array}[]{cc}\omega&0\\\
0&1\end{array}\right),\qquad\rho(S)=-\frac{i}{\sqrt{3}}\left(\begin{array}[]{cc}1&\sqrt{2}\\\
\sqrt{2}&-1\end{array}\right).$ (A.19)

We follow the representation basis in Refs. [37, 38], which is different from
one in Ref. [47].

Explicitly, the modular form of weight 1 for the $T^{\prime}$ doublet can be
written by

$\displaystyle\left(\begin{array}[]{c}x_{1}\\\
y_{1}\end{array}\right)=\left(\begin{array}[]{c}-\sqrt{2}\hat{e}_{1}\\\
\hat{e}_{1}+\frac{1}{3}\hat{e}_{2}\end{array}\right).$ (A.24)

Their $q$-expansions can be written by

$\displaystyle x_{1}=-\sqrt{2}q^{1/3}+\cdots,$ $\displaystyle
y_{1}=1/3+\cdots.$ (A.25)

## References

  * [1] F. Feruglio, arXiv:1706.08749 [hep-ph]; T. Kobayashi, K. Tanaka and T. H. Tatsuishi, Phys. Rev. D 98, no. 1, 016004 (2018) [arXiv:1803.10391 [hep-ph]]; J. T. Penedo and S. T. Petcov, Nucl. Phys. B 939, 292 (2019) [arXiv:1806.11040 [hep-ph]]; J. C. Criado and F. Feruglio, SciPost Phys. 5, no. 5, 042 (2018) [arXiv:1807.01125 [hep-ph]]; T. Kobayashi, N. Omoto, Y. Shimizu, K. Takagi, M. Tanimoto and T. H. Tatsuishi, JHEP 1811, 196 (2018) [arXiv:1808.03012 [hep-ph]]; P. P. Novichkov, J. T. Penedo, S. T. Petcov and A. V. Titov, JHEP 1904, 005 (2019) [arXiv:1811.04933 [hep-ph]]; JHEP 1904, 174 (2019) [arXiv:1812.02158 [hep-ph]]; F. J. de Anda, S. F. King and E. Perdomo, arXiv:1812.05620 [hep-ph]; H. Okada and M. Tanimoto, Phys. Lett. B 791, 54 (2019) [arXiv:1812.09677 [hep-ph]]; T. Kobayashi, Y. Shimizu, K. Takagi, M. Tanimoto, T. H. Tatsuishi and H. Uchida, Phys. Lett. B 794, 114 (2019) [arXiv:1812.11072 [hep-ph]]; P. P. Novichkov, S. T. Petcov and M. Tanimoto, Phys. Lett. B 793, 247 (2019) [arXiv:1812.11289 [hep-ph]].
  * [2] R. de Adelhart Toorop, F. Feruglio and C. Hagedorn, Nucl. Phys. B 858, 437-467 (2012) [arXiv:1112.1340 [hep-ph]].
  * [3] H. P. Nilles, M. Ratz, A. Trautner and P. K. S. Vaudrevange, Phys. Lett. B 786 (2018) 283 [arXiv:1808.07060 [hep-th]].
  * [4] P. P. Novichkov, J. T. Penedo, S. T. Petcov and A. V. Titov, JHEP 07 (2019), 165 [arXiv:1905.11970 [hep-ph]].
  * [5] S. Ferrara, D. Lust, A. D. Shapere and S. Theisen, Phys. Lett. B 225, 363 (1989)
  * [6] S. Ferrara, D. Lust and S. Theisen, Phys. Lett. B 233 (1989), 147-152.
  * [7] J. P. Derendinger, S. Ferrara, C. Kounnas and F. Zwirner, Nucl. Phys. B 372, 145 (1992).
  * [8] L. E. Ibanez and D. Lust, Nucl. Phys. B 382, 305 (1992) [hep-th/9202046].
  * [9] T. Kobayashi, S. Nagamoto and S. Uemura, PTEP 2017, no.2, 023B02 (2017) [arXiv:1608.06129 [hep-th]].
  * [10] A. Baur, H. P. Nilles, A. Trautner and P. K. S. Vaudrevange, Phys. Lett. B 795 (2019) 7 [arXiv:1901.03251 [hep-th]].
  * [11] T. Kobayashi and H. Otsuka, Phys. Rev. D 101 (2020) no.10, 106017 [arXiv:2001.07972 [hep-th]].
  * [12] T. Kobayashi and H. Otsuka, Phys. Rev. D 102 (2020) no.2, 026004 [arXiv:2004.04518 [hep-th]].
  * [13] K. Ishiguro, T. Kobayashi and H. Otsuka, Nucl. Phys. B 973 (2021), 115598 [arXiv:2010.10782 [hep-th]].
  * [14] A. Strominger, Commun. Math. Phys. 133 (1990), 163-180.
  * [15] P. Candelas and X. de la Ossa, Nucl. Phys. B 355 (1991), 455-481.
  * [16] G. J. Ding, F. Feruglio and X. G. Liu, JHEP 01 (2021), 037 [arXiv:2010.07952 [hep-th]].
  * [17] A. Baur, M. Kade, H. P. Nilles, S. Ramos-Sanchez and P. K. S. Vaudrevange, Phys. Lett. B 816 (2021), 136176 [arXiv:2012.09586 [hep-th]].
  * [18] H. P. Nilles, S. Ramos-Sanchez, A. Trautner and P. K. S. Vaudrevange, [arXiv:2105.08078 [hep-th]].
  * [19] E. Witten, Nucl. Phys. B 258 (1985), 75.
  * [20] B. de Wit, F. Vanderseypen and A. Van Proeyen, Nucl. Phys. B 400 (1993), 463-524 [arXiv:hep-th/9210068 [hep-th]].
  * [21] B. de Wit and A. Van Proeyen, Nucl. Phys. B 245 (1984), 89-117.
  * [22] D. J. Gross, J. A. Harvey, E. J. Martinec and R. Rohm, Nucl. Phys. B 256 (1985) 253.
  * [23] D. J. Gross, J. A. Harvey, E. J. Martinec and R. Rohm, Nucl. Phys. B 267 (1986) 75.
  * [24] N. Seiberg, Nucl. Phys. B 303 (1988), 286-304.
  * [25] S. Cecotti, S. Ferrara and L. Girardello, Int. J. Mod. Phys. A 4 (1989), 2475.
  * [26] L. J. Dixon, V. Kaplunovsky and J. Louis, Nucl. Phys. B 329 (1990), 27-82.
  * [27] M. Billo, R. D’Auria, S. Ferrara, P. Fre, P. Soriani and A. Van Proeyen, Int. J. Mod. Phys. A 12 (1997), 379-418 [arXiv:hep-th/9505123 [hep-th]].
  * [28] T. Araki, K. S. Choi, T. Kobayashi, J. Kubo and H. Ohki, Phys. Rev. D 76, 066006 (2007) [arXiv:0705.3075 [hep-ph]].
  * [29] T. Araki, T. Kobayashi, J. Kubo, S. Ramos-Sanchez, M. Ratz and P. K. S. Vaudrevange, Nucl. Phys. B 805, 124-147 (2008) [arXiv:0805.0207 [hep-th]].
  * [30] A. Strominger and E. Witten, Commun. Math. Phys. 101 (1985) 341.
  * [31] M. Dine, R. G. Leigh and D. A. MacIntire, Phys. Rev. Lett. 69 (1992) 2030 [hep-th/9205011].
  * [32] K. w. Choi, D. B. Kaplan and A. E. Nelson, Nucl. Phys. B 391 (1993) 515 [hep-ph/9205202].
  * [33] A. Strominger, NSF-ITP-85-109.
  * [34] F. Feruglio, C. Hagedorn and R. Ziegler, JHEP 1307, 027 (2013) [arXiv:1211.5560 [hep-ph]].
  * [35] M. Holthausen, M. Lindner and M. A. Schmidt, JHEP 1304, 122 (2013) [arXiv:1211.6953 [hep-ph]].
  * [36] M. C. Chen, M. Fallbacher, K. T. Mahanthappa, M. Ratz and A. Trautner, Nucl. Phys. B 883, 267 (2014) [arXiv:1402.0507 [hep-ph]].
  * [37] H. Ishimori, T. Kobayashi, H. Ohki, Y. Shimizu, H. Okada and M. Tanimoto, Prog. Theor. Phys. Suppl. 183 (2010), 1-163 [arXiv:1003.3552 [hep-th]].
  * [38] H. Ishimori, T. Kobayashi, H. Ohki, H. Okada, Y. Shimizu and M. Tanimoto, Lect. Notes Phys. 858, 1-227 (2012)
  * [39] D. Shevitz, Nucl. Phys. B 338 (1990), 283-293.
  * [40] B. L. Rachlin and T. W. Kephart, JHEP 08, 110 (2017) [arXiv:1702.08073 [hep-ph]].
  * [41] K. Ishiguro and H. Otsuka, JHEP 12 (2021), 017 [arXiv:2104.15030 [hep-th]].
  * [42] S. Hamidi and C. Vafa, Nucl. Phys. B 279 (1987), 465-513
  * [43] L. J. Dixon, D. Friedan, E. J. Martinec and S. H. Shenker, Nucl. Phys. B 282 (1987), 13-73.
  * [44] T. T. Burwick, R. K. Kaiser and H. F. Muller, Nucl. Phys. B 355 (1991), 689-711.
  * [45] W. Lerche, D. Lust and N. P. Warner, Phys. Lett. B 231 (1989), 417-424.
  * [46] J. Lauer, J. Mas and H. P. Nilles, Phys. Lett. B 226, 251 (1989); Nucl. Phys. B 351, 353 (1991).
  * [47] X. G. Liu and G. J. Ding, JHEP 08, 134 (2019) [arXiv:1907.01488 [hep-ph]].
  * [48] T. Kobayashi, S. Nagamoto, S. Takada, S. Tamba and T. H. Tatsuishi, Phys. Rev. D 97, no. 11, 116002 (2018) [arXiv:1804.06644 [hep-th]].
  * [49] T. Kobayashi and S. Tamba, Phys. Rev. D 99 (2019) no.4, 046001 [arXiv:1811.11384 [hep-th]].
  * [50] H. Ohki, S. Uemura and R. Watanabe, Phys. Rev. D 102, no.8, 085008 (2020) [arXiv:2003.04174 [hep-th]].
  * [51] S. Kikuchi, T. Kobayashi, S. Takada, T. H. Tatsuishi and H. Uchida, Phys. Rev. D 102, no.10, 105010 (2020) [arXiv:2005.12642 [hep-th]].
  * [52] S. Kikuchi, T. Kobayashi, H. Otsuka, S. Takada and H. Uchida, JHEP 11, 101 (2020) [arXiv:2007.06188 [hep-th]].
  * [53] S. Kikuchi, T. Kobayashi and H. Uchida, [arXiv:2101.00826 [hep-th]].
  * [54] Y. Almumin, M. C. Chen, V. Knapp-Perez, S. Ramos-Sanchez, M. Ratz and S. Shukla, [arXiv:2102.11286 [hep-th]].
  * [55] Y. Tatsuta, JHEP 10 (2021), 054 [arXiv:2104.03855 [hep-th]].
  * [56] M. Cvetic, B. A. Ovrut and J. Louis, Phys. Rev. D 40 (1989), 684.
  * [57] K. Ishiguro, T. Kobayashi and H. Otsuka, JHEP 07 (2021), 064 [arXiv:2103.10240 [hep-th]].
  * [58] O. DeWolfe, A. Giryavets, S. Kachru and W. Taylor, JHEP 07 (2005), 066 [arXiv:hep-th/0505160 [hep-th]].
  * [59] M. Kreuzer and H. Skarke, Adv. Theor. Math. Phys. 4 (2002), 1209-1230 [arXiv:hep-th/0002240 [hep-th]].
  * [60] V. Batyrev and M. Kreuzer, [arXiv:math/0505432 [math.AG]].
  * [61] Y. H. He, S. J. Lee, A. Lukas and C. Sun, JHEP 06 (2014), 077 [arXiv:1309.0223 [hep-th]].
  * [62] R. Gopakumar and C. Vafa, [arXiv:hep-th/9809187 [hep-th]]; R. Gopakumar and C. Vafa, [arXiv:hep-th/9812127 [hep-th]].
  * [63] M. Bershadsky, S. Cecotti, H. Ooguri and C. Vafa, Commun. Math. Phys. 165 (1994), 311-428 [arXiv:hep-th/9309140 [hep-th]].
  * [64] S. Groot Nibbelink, J. Held, F. Ruehle, M. Trapletti and P. K. S. Vaudrevange, JHEP 03 (2009), 005 [arXiv:0901.3059 [hep-th]].