# Lepton masses and mixing in a scotogenic model

E. C. F. S. Fortes<EMAIL_ADDRESS>Universidade Federal do Pampa
Rua Luiz Joaquim de Sá Brito, s/n, Promorar,
Itaqui - RS, 97650-000, Brazil  A. C. B. Machado<EMAIL_ADDRESS>Laboratorio
de Física Teórica e Computacional
Universidade Cruzeiro do Sul – Rua Galvão Bueno 868
São Paulo, SP, Brazil, 01506-000  J. Montaño<EMAIL_ADDRESS>Instituto
de Física Teórica–Universidade Estadual Paulista
R. Dr. Bento Teobaldo Ferraz 271, Barra Funda
São Paulo - SP, 01140-070, Brazil  V. Pleitez<EMAIL_ADDRESS>Instituto
de Física Teórica–Universidade Estadual Paulista
R. Dr. Bento Teobaldo Ferraz 271, Barra Funda
São Paulo - SP, 01140-070, Brazil

(05/26/17)

###### Abstract

We consider an extension of the standard model with three Higgs doublet model
and $S_{3}\times\mathbb{Z}_{2}$ discrete symmetries. Two of the scalar
doublets are inert due to the $\mathbb{Z}_{2}$ symmetry. We have calculated
all the mass spectra in the scalar and lepton sectors and accommodated the
leptonic mixing matrix as well. We also show that the model has scalar and
pseudoscalar candidates to dark matter. Constraints on the parameters of the
model coming from the decay $\mu\to e\gamma$ were considered and we found
signals between the current and the upcoming experimental limits, and from
that decay we can predict the one-loop $\mu\to ee\bar{e}$ channel.

###### pacs:

14.60.Pq, 14.60.St, 13.35.Bv

##  I Introduction

Although since 2012 we know that there exist a neutral spin-0 resonance with
properties (mass and couplings) that are compatible, within the experimental
error, with the Higgs boson of the standard model (SM) Aad:2012tfa ;
Chatrchyan:2012xdj , the data do not exclude the existence of more scalar
fields and almost all extensions of the SM include extra Higgs doublets. This
is the reason for considering multi-Higgs models. Moreover, although many
scalar doublets may exist in nature, it is possible that only one of them is
the responsible for the electroweak spontaneous symmetry breaking and the
generation of the charged fermion masses. In this case, the other scalar
multiplets may be inert ones: they do not couple to fermions, do not
contribute to the vector bosons masses, and interact only with vector bosons
and with other scalars. This possibility was put forward many years ago in
Ref. Deshpande:1977rw  in which a $\mathbb{Z}_{2}$ symmetry was imposed to
keep inert one of the doublets.

On the other hand, an indication that there must be physics beyond the SM is
the origin of the neutrino masses. In fact, the generation of masses smaller
than 0.1 eV demand the introduction of new degrees of freedom even in the
context of the gauge symmetries of the SM. An interesting possibility is that
the extra scalar fields that may exist as an extension of the SM also induce
the appropriate neutrino mass. In particular, neutrino mass generation in a
model with two doublets, being one of them inert, was considered by Ma
Ma:2006km . This is the so called scotogenic mechanism for generating neutrino
masses through one-loop corrections involving the inert neutral components.

One interesting feature of the mechanism is that it includes by construction
one or more dark matter (DM) candidates, or the implementation of the baryon
asymmetry in the Universe, relating in this way three of the more important
problems in elementary particle physics: the generation of the neutrino
masses, the nature of the DM, and the observed asymmetry between matter and
anti-matter, see Ref. Aoki:2014lha  and references therein. Moreover, the
existence of many components of DM is interesting by their own. In this case
DM may decay from heavier to lighter components, and also co-annihilate in two
dark particles Dienes:2014via . In fact, this might be the only possibility to
accommodate several astrophysical observations. For instance, the
electron/positron excesses observed by many experiments and recently confirmed
by AMS-02 Accardo:2014lma  and the excess of gamma rays peaking at energies of
several GeV from the region surrounding the Galactic Center Goodenough:2009gk
; Daylan:2014rsa  must need at least two component DM. Moreover, the latter
case can avoid some constraints on the one component DM from the AMS-02 data
Geng:2014dea . Notwithstanding there are alternative interpretations for these
gamma rays excess, see Abazajian:2015raa ; Fermi-LAT:2016uux . It is
interesting that the one-doublet inert model has at least two DM components.
However see Liu:2017drf .

Here we will work out a similar mechanism but in the context of the model with
two inert scalar doublets proposed in Ref. Machado:2012ed . Moreover, the
inert character is due to the $\mathbb{Z}_{2}$ symmetry, and the $S_{3}$
symmetry makes the scalar potential more predictive and easier to be analyzed.
Although three right-handed neutrinos are introduced, the active neutrino
masses do not arise through the type-I seesaw mechanism but via the scotogenic
mechanism, at the 1-loop level. We need to add two real singlet scalar fields
in order to accommodate the charged lepton masses and the Pontecorvo-Maki-
Nakagawa-Sakata (PMNS) mixing matrix. In the context of inert doublet models,
the latter issue is to the best of our knowledge done for the first time.

The outline of this paper is as follows. In the next section we discuss the
model while in Sec. III analyse the scalar sector. Lepton mass matrices and
the leptonic mixing matrix is shown in Sec. IV. In Sec. V we show that the
model provides a multi-component DM spectra, although we do not consider the
most general case. In Sec. VI we consider the decays $\mu\to e\gamma$, Subsec.
VI.1, and $\mu\to e\bar{e}e$ in Subsec. VI.2. Our conclusions appear in Sec.
VII.

##  II The Model

In Ref. Machado:2012ed  it was proposed an extension of the electroweak SM
with three Higgs scalars $H_{1,2,3}$ transforming as doublets under $SU(2)$
and having $Y=+1$. One $H_{1}\equiv S$ transforms as singlet of $S_{3}$, and
the others as doublets, $D=(H_{2},H_{3})~{}\equiv~{}(D_{1},D_{2})$. Here we
will extend the model of Ref. Machado:2012ed  by adding three right-handed
sterile neutrinos, $N_{1R}$ transforming as singlet, and
$N_{d}=(N_{2R},N_{3R})$ transforming as doublets of $S_{3}$, and two real
scalar singlets of $SU(2)$ ($Y=0$) but doublets of $S_{3}$,
$\zeta_{d}=(\zeta_{1},\zeta_{2})$. See the other quantum numbers in Table 1.
The vacuum alignment is given by $\langle S\rangle=v_{SM}/\sqrt{2}$, and
$\langle D_{1},D_{2}\rangle=0$, $\langle\zeta_{1,2}\rangle=v_{\zeta}$. In the
charged lepton and quark sectors, all usual fields of SM transform as singlet
under $S_{3}$.

With these fields the Yukawa interactions in the lepton sector, invariant
under the gauge, $S_{3}$ and $\mathbb{Z}_{2}$ symmetries (see Table 1) are
given by

$\displaystyle-\mathcal{L}^{leptons}_{Yukawa}$ $\displaystyle=$ $\displaystyle
G^{l}_{ij}\bar{L}_{i}l_{jR}S+G^{\nu}_{id}\bar{L}_{i}[N_{d}\,\tilde{D}]_{1}+\frac{1}{\Lambda}G^{\nu}_{is}\bar{L}_{i}[N_{s}[\tilde{D}\zeta_{d}]_{1^{\prime}}]_{1}$
$\displaystyle+$
$\displaystyle\frac{1}{2}M_{s}\overline{N^{c}_{s}}N_{s}+\frac{1}{2}M_{d}[\overline{N^{c}_{d}}N_{d}]_{1}+H.c.,$

where $i,j=e,\mu,\tau$ (we omit summation symbols), $L_{i}(l_{iR})$ and
$N_{s,d}$ denote the usual left-handed lepton doublets (right-handed singlets)
and the right-handed neutrinos, respectively;
$[\tilde{D}\,\zeta_{d}]_{1^{\prime}}=\tilde{D}_{1}\zeta_{2}-\tilde{D}_{2}\zeta_{1}$.
$[N_{d}\,D]_{1}=N_{2R}D_{1}+N_{3R}D_{2}$, according to the $S_{3}$
multiplication rules, and $\tilde{D}_{1,2}=i\tau^{2}D^{*}_{1,2}$. Notice that
the doublets $D$ and $\zeta_{d}$ couple only with neutrinos. We assume that
$\langle\zeta_{1,2}\rangle\lesssim\Lambda$, where $\Lambda$ is an energy scale
much larger than the electroweak one. It is also interesting to note that the
right-handed neutrinos in the $S_{3}$ doublet are mass degenerated, with mass
$M_{d}$, which is different from the mass of the right-handed neutrino in
singlet of $S_{3}$, which has a mass denoted by $M_{s}$. Notice that at three
level active neutrinos are still massless.

Symmetry | $L_{i}$ | $l_{jR}$ | $N_{s}$ | $N_{d}$ | $S$ | $D$ | $\zeta_{d}$
---|---|---|---|---|---|---|---
$S_{3}$ | 1 | 1 | $1^{\prime}$ | 2 | 1 | 2 | 2
$\mathbb{Z}_{2}$ | 1 | 1 | -1 | 1 | 1 | 1 | -1
Table 1: Transformation properties of the fermion and scalar fields under
$S_{3}$ and $\mathbb{Z}_{2}$ symmetries. Quarks and charged leptons are
singlets of $S_{3}$ and even under $\mathbb{Z}_{2}$.

##  III The scalar sector

The scalar sector of the model is presented as follows:

$S=\left(\begin{array}[]{c}S^{+}\\\
\frac{1}{\sqrt{2}}(v_{SM}+\textrm{Re}S^{0}+i\textrm{Im}S^{0})\end{array}\right),\quad
D_{1,2}=\left(\begin{array}[]{c}D^{+}_{1,2}\\\
\frac{1}{\sqrt{2}}(\textrm{Re}D^{0}_{1,2}+i\textrm{Im}D^{0}_{1,2}).\end{array}\right),$
(2)

plus the singlets
$\zeta_{i}=v_{i}+\textrm{Re}\zeta_{i}+i\textrm{Im}\zeta_{i}$, $i=1,2$.

The scalar potential invariant under the gauge and
$S_{3}\otimes\mathbb{Z}_{2}$ symmetries is

$\displaystyle V_{S_{3}}$ $\displaystyle=$
$\displaystyle\mu^{2}_{s}S^{\dagger}S+\mu^{2}_{d}[D^{\dagger}\otimes
D]_{1}+\mu^{2}_{\zeta_{D}}[\zeta_{D}\otimes\zeta_{D}]_{1}+\lambda_{1}([D^{\dagger}\otimes
D]_{1})^{2}+\mu^{2}_{12}\zeta_{1}\zeta_{2}+a_{2}[[D^{\dagger}\otimes
D]_{1^{\prime}}[D^{\dagger}\otimes D]_{1^{\prime}}]$ (3)
$\displaystyle+a_{3}[(D^{\dagger}\otimes D)_{2^{\prime}}(D^{\dagger}\otimes
D)_{2^{\prime}}]_{1}+a_{4}(S^{\dagger}S)^{2}+a_{5}[D^{\dagger}\otimes
D]_{1}S^{\dagger}S+a_{6}[[S^{\dagger}D]_{2^{\prime}}[S^{\dagger}D]_{2^{\prime}}]_{1}$
$\displaystyle+H.c.]+a_{7}S^{\dagger}[D\otimes
D^{\dagger}]_{1}S+b_{1}S^{\dagger}S[\zeta_{D}\otimes\zeta_{D}]_{1}+b_{2}[D^{\dagger}\otimes
D]_{1}[\zeta_{D}\otimes\zeta_{D}]_{1}$
$\displaystyle+b_{3}[[D^{\dagger}\otimes
D]_{2^{\prime}}[\zeta_{D}\otimes\zeta_{D}]_{2^{\prime}}]_{1}+c_{1}([\zeta_{D}\otimes\zeta_{D}]_{1})^{2}+c_{2}[[\zeta_{D}\otimes\zeta_{D}]_{2^{\prime}}[\zeta_{D}\otimes\zeta_{D}]_{2^{\prime}}]_{1},$

with $\mu^{2}_{d}>0$ that is guaranteed by the $\mathbb{Z}_{2}$ symmetry.

We can write Eq. (3) explicitly as

$V(S,D,\zeta_{d})=V^{(2)}+V^{(4a)}+V^{(4b)}+V^{(4c)},$ (4)

where

$\displaystyle V^{(2)}$ $\displaystyle=$
$\displaystyle\mu^{2}_{SM}S^{\dagger}S+\mu^{2}_{d}(D^{\dagger}_{1}D_{1}+D^{\dagger}_{2}D_{2})+\mu^{2}_{\zeta}(\zeta^{2}_{1}+\zeta^{2}_{2})+\mu^{2}_{12}\zeta_{1}\zeta_{2},$
$\displaystyle V^{(4a)}$ $\displaystyle=$ $\displaystyle
a_{1}(D^{\dagger}_{1}D_{1}+D^{\dagger}_{2}D_{2})^{2}+a_{2}(D^{\dagger}_{1}D_{2}-D^{\dagger}_{2}D_{1})^{2}$
$\displaystyle+a_{3}[(D^{\dagger}_{1}D_{2}+D^{\dagger}_{2}D_{1})^{2}+(D^{\dagger}_{1}D_{1}-D^{\dagger}_{2}D_{2})^{2}]+a_{4}(S^{\dagger}S)^{2}+a_{5}(D^{\dagger}_{1}D_{1}+D^{\dagger}_{2}D_{2})S^{\dagger}S$
$\displaystyle+a_{6}[(S^{\dagger}D_{1}S^{\dagger}D_{1}+S^{\dagger}D_{2}S^{\dagger}D_{2})+H.c.]+a_{7}S^{\dagger}(D_{1}D^{\dagger}_{1}+D_{2}D^{\dagger}_{2})S,$
$\displaystyle V^{(4b)}$ $\displaystyle=$ $\displaystyle
b_{1}S^{\dagger}S(\zeta^{2}_{1}+\zeta^{2}_{2})+b_{2}(D^{\dagger}_{1}D_{1}+D^{\dagger}_{2}D_{2})(\zeta^{2}_{1}+\zeta^{2}_{2})+b_{3}[(D^{\dagger}_{1}D_{2}+D^{\dagger}_{2}D_{1})(\zeta_{1}\zeta_{2}+\zeta_{1}\zeta_{2})$
$\displaystyle\times(D^{\dagger}_{1}D_{1}-D^{\dagger}_{2}D_{2})(\zeta^{2}_{1}-\zeta^{2}_{2})+H.c.],$
$\displaystyle V^{(4c)}$ $\displaystyle=$ $\displaystyle
c_{1}(\zeta^{2}_{1}+\zeta^{2}_{2})^{2}+c_{2}[(\zeta_{1}\zeta_{2}+\zeta_{2}\zeta_{1})^{2}+(\zeta^{2}_{1}-\zeta^{2}_{2})^{2}],$
(5)

where we have used
$[\zeta_{d}\,\zeta_{d}]_{2^{\prime}}=(\zeta_{1}\zeta_{2}+\zeta_{2}\zeta_{1},\zeta_{1}\zeta_{1}-\zeta_{2}\zeta_{2})$.
Notice that the term $\mu^{2}_{12}$ breaks softly the $S_{3}$ symmetry but not
the $\mathbb{Z}_{2}$. Notice also that the $\mathbb{Z}_{2}$ symmetry forbids
trilinear terms in the scalar potential $[D^{\dagger}\otimes D]_{1}\zeta_{i}$
and $\zeta^{3}_{i}$.

From derivation of Eq. (5), we obtain the constraint equations:

$\displaystyle
v_{SM}[2\mu^{2}_{SM}+2a_{4}v^{2}_{SM}+b_{1}(v^{2}_{1}+v^{2}_{2})]$
$\displaystyle=$ $\displaystyle 0,$ $\displaystyle
v_{1}[4\mu^{2}_{\zeta}+2b_{1}v^{2}_{SM}+4c(v^{2}_{1}+v^{2}_{2})+4\mu^{2}_{12}\frac{v_{2}}{v_{1}}]$
$\displaystyle=$ $\displaystyle 0,$ $\displaystyle
v_{2}[4\mu^{2}_{\zeta}+2b_{1}v^{2}_{SM}+4c(v^{2}_{1}+v^{2}_{2})+4\mu^{2}_{12}\frac{v_{1}}{v_{2}}]$
$\displaystyle=$ $\displaystyle 0,$ (6)

where we have defined $c=c_{1}+c_{2}$. Notice from (6) that neither $v_{1}=0$
nor $v_{2}=0$ are allowed if $\mu^{2}_{12}\not=0$, hence we have that
$v_{1}\not=0,v_{2}\not=0$, or $v_{1}=v_{2}\equiv v_{\zeta}$. We have chosen
the latter case, so the constraint equations become

$v_{SM}[\mu^{2}_{SM}+a_{4}v^{2}_{SM}+b_{1}v^{2}_{\zeta}]=0\ ,\
v_{\zeta}[2\mu^{2}_{\zeta}+b_{1}v^{2}_{SM}+4cv_{\zeta}+2\mu^{2}_{12}]=0.$ (7)

The scalar potential has to be bounded from below to ensure its stability. In
the SM it is easy to ensure the stability of this potential, we just have to
ensure that $\lambda>0$. In theories in which the number of scalars is
increased, it is more difficult to ensure that the potential is bounded from
below, in all directions. A scalar potential has a quadratic form in the
quadratic couplings, i.e. $A_{ab}\phi^{2}_{a}\phi^{2}_{b}$, where
$\phi^{2}_{a}$ and $\phi^{2}_{b}$ represents the scalar fields, $S$, $D$ and
$\zeta_{d}$. If the matrix $A_{ab}$ is copositive it is possible to ensure
that the potential has a global minimum. Assuming a quadratic form (e.g.
considering only the quartic terms of the potential) is valid, even if there
exist trilinear terms, because in the case where the fields assume large
values, the terms of order 2 and 3 are negligible compared to the terms of
order 4. For more detail see Refs. ping ; Kannike:2012pe . We consider all
quartic couplings positive, i.e. all $A_{ij}$ are positive. Below we will
denote $B=a_{1}-2a_{2}-a_{3}$ and $C=a_{5}-2a_{6}+a_{7}$. And finally the
following limits guarantee that the scalar potential is bounded the from
below:

$\displaystyle a_{4}\geq 0,\;\frac{a_{5}}{2}\geq 0,\;C\geq 0,\;b_{1}\geq
0,\;a_{1}+a_{3}\geq 0,$ $\displaystyle b_{1}+b_{2}\geq 0,\;B\geq
0,\;c_{1}+c_{2}\geq 0.$ (8)

Next, we consider the scalar mass spectra. In the $C\\!P$-even sector the mass
matrix becomes in block diagonal form with one $3\times 3$, $\mathcal{M}_{1R}$
sub-matrix and one $2\times 2$ matrix, $\mathcal{M}_{2R}$. The first one in
the basis $(\textrm{Re}S^{0},\zeta_{1},\zeta_{2})$ is given by

$\mathcal{M}^{2}_{1R}=\left(\begin{array}[]{ccc}2a_{4}v^{2}_{s}&b_{1}v_{SM}v_{\zeta}&b_{1}v_{SM}v_{\zeta}\\\
&-\mu^{2}_{12}+2cv^{2}_{\zeta}&\mu^{2}_{12}+2cv^{2}_{\zeta}\\\
&&-\mu^{2}_{12}+2cv^{2}_{\zeta}\end{array}\right).$ (9)

The respective eigenvalues are

$\displaystyle m^{2}_{1}=-2\mu^{2}_{12},$ $\displaystyle
m^{2}_{2}=\frac{1}{2}\left[a_{4}v_{SM}^{2}+2cv^{2}_{\zeta}-\sqrt{a_{4}^{2}v^{4}_{SM}+2(b_{1}^{2}-2a_{4}c)v^{2}_{\zeta}v^{2}_{SM}+4c^{2}v_{\zeta}^{2}}\,\right],$
$\displaystyle
m^{2}_{3}=\frac{1}{2}\left[[4a_{4}v_{SM}^{2}+8cv^{2}_{\zeta}]^{2}+\sqrt{a_{4}^{2}v^{4}_{SM}+2(b_{1}^{2}-2a_{4}c)v^{2}_{\zeta}v^{2}_{SM}+4c^{2}v_{\zeta}^{2}}\,\right].$
(10)

From $m^{2}_{1}$ we see that $\mu^{2}_{12}<0$, hence $m^{2}_{1}$ may have
large mass. The SM-like Higgs boson may be identified with the scalar with
mass $m_{3}$ in Eq. (10). In order to see this just make $v^{2}_{\zeta}=0$.

The second mass matrix $M_{2R}$ in the basis
$(\textrm{Re}D^{0}_{1},\textrm{Re}D^{0}_{2})$ reads

$\mathcal{M}^{2}_{2R}=\left(\begin{array}[]{cc}\mu^{2}_{d}+\frac{a^{\prime}}{2}v_{SM}^{2}+b_{2}v^{2}_{\zeta}&b_{3}v^{2}_{\zeta}\\\
&\mu^{2}_{d}+\frac{a^{\prime}}{2}v_{SM}^{2}+b_{2}v^{2}_{\zeta}\\\
\end{array}\right),$ (11)

with the eigenvalues

$\mathrm{m}^{2}_{R1}=2\mu^{2}_{d}+a^{\prime}v_{SM}^{2}+2(b_{2}-b_{3})v^{2}_{\zeta},\quad\mathrm{m}^{2}_{R2}=2\mu^{2}_{d}+a^{\prime}v_{SM}^{2}+2(b_{2}+b_{3})v^{2}_{\zeta}.$
(12)

The mass matrix in the $C\\!P$ odd sector has the form in the basis
$(\textrm{Im}D^{0}_{1},\textrm{Im}D^{0}_{2})$ (the would-be Goldstone bosons
has been already decoupled)

$M_{I}=\left(\begin{array}[]{cc}\mu^{2}_{d}+\frac{a^{\prime\prime}}{2}v_{SM}^{2}+b_{2}v^{2}_{\zeta}&b_{3}v^{2}_{\zeta}\\\
&\mu^{2}_{d}+\frac{a^{\prime\prime}}{2}v_{SM}^{2}+b_{2}v^{2}_{\zeta}\end{array}\right),$
(13)

with eigenvalues

$m^{2}_{I1}=2\mu^{2}_{d}+a^{\prime\prime}v_{SM}^{2}+2(b_{2}-b_{3})v^{2}_{\zeta},\quad
m^{2}_{I2}=2\mu^{2}_{d}+a^{\prime\prime}v_{SM}^{2}+2(b_{2}+b_{3})v^{2}_{\zeta}.$
(14)

Above we have defined $a^{\prime}=a_{5}+a_{7}+2a_{6}$ and
$a^{\prime\prime}=a_{5}+a_{7}-2a_{6}$.

The model allows four neutral scalars with different masses that could
contribute to the DM relic density in different proportion: two $C\\!P$ even
and two $C\\!P$ odd. Notice also that $a_{6}$ is the term in the scalar
potential that transfer the $L$ violation to the active neutrino sector.

In the charged scalars sector, besides the charged would-be Goldstone boson,
we have two charged scalar fields (we have already omitted the charged would-
be Goldstone boson)

$M^{2}_{C}=\left(\begin{array}[]{cc}\mu^{2}_{d}+\frac{a_{5}}{2}v_{SM}^{2}+b_{2}v^{2}_{\zeta}&b_{3}v^{2}_{\zeta}\\\
&\mu^{2}_{d}+\frac{a_{5}}{2}v_{SM}^{2}+b_{2}v^{2}_{\zeta}\end{array}\right),$
(15)

with the non-zero eigenvalues given by

$\displaystyle
m^{2}_{+1}=\mu^{2}_{d}+\frac{a_{5}}{2}v_{SM}^{2}+(b_{2}-b_{3})v^{2}_{\zeta}\
,\
m^{2}_{+2}=\mu^{2}_{d}+\frac{a_{5}}{2}v_{SM}^{2}+(b_{2}+b_{3})v^{2}_{\zeta},$
(16)

Notice that

$\displaystyle m^{2}_{R1}-m^{2}_{+1}$ $\displaystyle=$
$\displaystyle\mu^{2}_{d}+\left(\frac{a_{5}}{2}+a_{7}+2a_{6}\right)v^{2}_{SM}+(b_{2}-b_{3})v^{2}_{\zeta},$
$\displaystyle m^{2}_{R2}-m^{2}_{+2}$ $\displaystyle=$
$\displaystyle\mu^{2}_{d}+\left(\frac{a_{5}}{2}+a_{7}+2a_{6}\right)v^{2}_{SM}+(b_{2}+b_{3})v^{2}_{\zeta}.$
(17)

Notice that $\mu^{2}_{d}>0$ does not disappear in the mass difference above
because, in oder to reproduce the Klein-Gordon equation for each component, a
real scalar has a $1/2$ factor in the mass term related to the mass of a
complex scalar.

##  IV Lepton masses and the PMNS matrix

In Sec. II we have seen that at tree level the neutrinos are massless, the
$a_{6}$ term in Eq. (5) induce diagrams like those in Fig. 1 and it is
possible to implement the mechanism of Ref. Ma:2006km  for radiative
generation of neutrinos mass. In fact, the diagram in Fig. 1 are exactly
calculable from the exchange of Re$D^{0}_{1,2}$ and Im$D^{0}_{1,2}$

$(M_{\nu})_{ij}=\sum_{a,k}\frac{Y_{ik}Y_{jk}M_{k}}{32\pi^{2}}\left[\frac{m^{2}_{Ra}}{m^{2}_{Ra}-M^{2}_{k}}\ln\frac{m^{2}_{Ra}}{M_{k}^{2}}-\frac{\mathrm{m}^{2}_{Ia}}{m^{2}_{Ia}-M^{2}_{k}}\ln\frac{m_{Ia}^{2}}{M_{k}^{2}}\right],$
(18)

where $a=1,2;k=s,d$; $m_{Ra}$ and $m_{Ia}$ are the masses of
$\textrm{Re}D^{0}_{1,2}$ and $\textrm{Im}D^{0}_{1,2}$, respectively. In Eq.
(18) $Y_{ik}Y_{jk}$ corresponds to $G^{\nu}_{is}G^{\nu}_{js}$ when the
coupling is with the $N_{s}$; and $Y_{ik}Y_{jk}$ corresponds to
$G^{\nu}_{id}G^{\nu}_{jd}$ when the coupling is with the $N_{d}$, and finally
$M_{s}$ is the mass of the right-handed neutrino $N_{s}$, and $M_{d}$ is the
common mass of the right-handed neutrino in the doublet of $S_{3}$, $N_{d}$
i.e, $M_{2}=M_{3}\equiv M_{d}$. We can define
$\Delta^{2}_{a}=m^{2}_{Ra}-~{}m^{2}_{Ia}=~{}4a_{6}v^{2}_{SM}$, and
$m^{2}_{0a}=(m^{2}_{Ra}+m^{2}_{aI})/2$, $a=1,2$.

If $\Delta^{2}\ll m^{2}_{a0}$ we obtain

$\displaystyle(M_{\nu})_{ij}$ $\displaystyle=$
$\displaystyle\frac{a_{6}v^{2}_{SM}}{16\pi^{2}}\left[\frac{G^{\nu}_{id}G^{\nu}_{jd}M_{d}}{m^{2}_{01}-M_{d}^{2}}\left(1-\frac{M_{d}^{2}}{m^{2}_{01}-M_{d}^{2}}\ln\frac{m^{2}_{01}}{M_{d}^{2}}\right)+\frac{G^{\nu}_{is}G^{\nu}_{js}M_{s}}{m^{2}_{01}-M^{2}_{s}}\left(1-\frac{M^{2}_{s}}{m^{2}_{01}-M^{2}_{s}}\ln\frac{m^{2}_{01}}{M^{2}_{s}}\right)\right.$
(19)
$\displaystyle\left.+\frac{G^{\nu}_{id}G^{\nu}_{jd}M_{d}}{m^{2}_{02}-M_{d}^{2}}\left(1-\frac{M_{d}^{2}}{m^{2}_{02}-M_{d}^{2}}\ln\frac{m^{2}_{02}}{M_{d}^{2}}\right)+\frac{G^{\nu}_{is}G^{\nu}_{js}M_{s}}{m^{2}_{02}-M^{2}_{s}}\left(1-\frac{M^{2}_{s}}{m^{2}_{02}-M^{2}_{s}}\ln\frac{m^{2}_{02}}{M^{2}_{s}}\right)\right],$

where $M_{s}$ is the mass of the right-handed neutrino $N_{s}$ and $M_{d}$ is
the common mass of the neutrinos $N_{d}$. Under the condition in which the
scalars are mass degenerated i.e., $b_{3}=0$ in (17) we obtain just a factor 2
in Eq.(19). Below, for simplicity, we will consider the case $b_{3}=0$.

In order to obtain the active neutrinos masses we assume a normal hierarchy
and, without loss of generality, that $M_{s}\sim M_{d}$ and will be
represented from now on by $M_{R}$. $M^{\nu}$ is diagonalized with a unitary
matrix $V_{L}^{\nu}$ i.e., $\hat{M}^{\nu}~{}=~{}V^{\nu
T}_{L}M^{\nu}V^{\nu}_{L}$, where
$\hat{M}^{\nu}=diag(m_{1},m_{2},m_{3})\approx(0,\sqrt{\delta
m^{2}_{12}},\sqrt{\delta m^{2}_{23}})$. Taken the central values in PDG we
have $\hat{M}^{\nu}\approx(0,x,y)$.

In the charged lepton sector we assume their masses at the central values in
PDG $\hat{M^{l}}=(0.510,105.658,1776.86)$ GeV. It is important to note from
these considerations, that there exist a multitude of other possibilities
which satisfy also the masses squared differences and the astrophysical limits
in the active neutrino sector. Each one corresponds to different
parameterization of the unitary matrices $V^{l}_{L,R},V^{\nu}_{L}$.

We will obtain the neutrinos masses from Eq. (19). We have as free parameters
$a_{6}$, $M_{R}$ and the Yukawas [$m_{+1}=m_{+2}\equiv m_{+}$ are also still
free but they will enter only in the leptonic decays considered in Sec. VI].
In the Fig. 2 we show the dependence of $a_{6}$ with respect to the main
Yukawas $G_{\tau d}^{\nu}$ in (a) and $G_{es}^{\nu}$ in (b) for fixed $M_{R}$
values. Notice that $G_{\tau d}^{\nu}$ are essentially of the same order of
magnitude, while the rest $G_{ed,\mu d,\mu s,\tau s}^{\nu}$ are suppressed by
four orders of magnitude when comparing with any specific value of those
$G_{\tau d,es}^{\nu}$.

$V_{L}^{l}$ parameterization | Masses in TeV | $G^{l}_{11}$ | $G^{l}_{12}$ | $G^{l}_{13}$ | $G^{l}_{22}$ | $G^{l}_{23}$ | $G^{l}_{33}$
---|---|---|---|---|---|---|---
P1 |  $M_{R}=2.8$, $M_{0}=2.2$ | 0.000421836 | 0.000514741 | -0.000800772 | 0.00374767 | -0.00356758 | 0.00367645
P2 |  $M_{R}=2\ \ $, $M_{0}=2.2$ | 0.000421864 | 0.000514825 | -0.000800852 | 0.00374768 | -0.00356756 | 0.00367641
P3 |  $M_{R}=1.5$, $M_{0}=2.2$ | 0.000421896 | 0.000514922 | 0.000800945 | 0.0037477 | -0.00356754 | 0.00367636
P4 |  $M_{R}=1\ \ $, $M_{0}=2.2$ | 0.00042195 | 0.000515086 | -0.000801103 | 0.00374772 | -0.00356751 | 0.00367628
P5 |  $M_{R}=0.5$, $M_{0}=2.2$ | 0.00042205 | 0.000515393 | -0.000801398 | 0.00374778 | -0.00356744 | 0.00367612
Table 2: Masses of the scalars in Scotogenic model (in GeV).

The mass matrices in the charged lepton sector $M^{l}$ are diagonalized by a
bi-unitary transformation $\hat{M}^{l}=V^{l\dagger}_{L}M^{l}V^{l}_{R}$ and
$\hat{M}^{l}=diag(m_{e},m_{\mu},m_{\tau})$. The relation between symmetry
eigenstates (primed) and mass (unprimed) fields are
$l^{\prime}_{L,R}=V^{l}_{L,R}l_{L,R}$ and
$\nu^{\prime}_{L}=V^{\nu}_{L}\nu_{L}$, where
$l^{\prime}_{L,R}=(e^{\prime},\mu^{\prime},\tau^{\prime})^{T}_{L,R}$,
$l_{L,R}=(e,\mu,\tau)^{T}_{L,R}$,
$\nu^{\prime}_{L}=(\nu_{e},\nu_{\mu},\nu_{\tau})^{T}_{L}$ and
$\nu_{L}=(\nu_{1},\nu_{2},\nu_{3})_{L}$. Defining the lepton mixing matrix as
$V_{PMNS}=V^{l\dagger}_{L}V^{\nu}_{L}$, it means that this matrix appears in
the charged currents coupled to $W_{\mu}^{+}$. We have tested the robustness
of our fitting of the lepton masses and the leptonic mixing matrix by using
several parametrization corresponding to the values of the Yukawa couplings
given in Table 2. We omit the respective matrices $V^{l}_{L}$ and
$V^{\nu}_{L}$ but in all cases we have obtained:

$|V_{PMNS}|\approx\left(\begin{array}[]{ccc}0.815&0.565&0.132\\\
0.479&0.527&0.702\\\ 0.327&0.635&0.700\\\ \end{array}\right),$ (20)

which is in agreement within the experimental error data at 3$\sigma$ given by
GonzalezGarcia:2012sz

$|V_{PMNS}|\approx\left(\begin{array}[]{ccc}0.795-0.846&0.513-0.585&0.126-0.178\\\
0.4205-0.543&0.416-0.730&0.579-0.808\\\ 0.215-0.548&0.409-0.725&0.567-0.800\\\
\end{array}\right),$ (21)

and we see that it is possible to accommodate all lepton masses and the PMNS
matrix. Here we do not consider $CP$ violation.

##  V Dark Matter

As we said before, the present model may have a multi-component DM spectrum,
which means that many particles may contribute to the relic density of DM, but
we will consider the simplest example where one of the $C\\!P$ even scalar,
say $R_{1}$, and one of the $C\\!P$ odd scalar, say $I_{1}$, as the dark
matter candidates, each case is considered separately for simplicity. A two
inert doublet model without right-handed neutrinos and scalar singlet was
considered in Ref. Fortes:2014dca .

As usual, in order to determine the relic density, we solve the Boltzmann
equation. Firstly, considering $R$ as the candidate, we have

$\frac{dn_{R}}{dt}+3Hn_{R}=-\langle\sigma|v|\rangle[(n_{R})^{2}-(n_{R}^{eq})^{2}],$
(22)

where $\langle\sigma|v|\rangle$ is the annihilation cross section already
thermally averaged and $H$ is the Hubble constant. In the thermal equilibrium,
the number density of DM Beltran:2008xg  is

$n^{eq}_{R}=g\left(\frac{m_{R}\,T}{2\pi}\right)^{3/2}\exp\left(-\frac{m_{R}}{T}\right),$
(23)

where $g=1$ for a scalar DM. When solving the Boltzmann equation we obtain the
equation for the relic density:

$\Omega_{R}h^{2}\approx\frac{1.04\times
10^{9}x_{F}}{M_{Pl}\sqrt{g_{*}}(a+3b/x_{F})},$ (24)

where $M_{Pl}=1.22\times 10^{19}$ GeV is the Planck mass, $x_{F}=m_{R}/T_{F}$,
where $m_{R}$ is the mass of the neutral scalar and $T_{F}$ is the temperature
at freeze-out, the terms $a$ and $b$ result from the partial wave expansion of
$\sigma|v|=a+bv^{2}$. The number of relativistic degrees of freedom
$g_{*}=118.375$ is a result of the SM particles plus three right-handed
neutrinos, five neutral scalars, two pseudo-scalars and two charged scalars.
The evaluation of $x_{F}$ leads to

$x_{F}=\ln\left[\widetilde{c}(\widetilde{c}+2)\sqrt{\frac{45}{8}}\frac{gm_{R}M_{Pl}(a+6b/x_{F})}{2\pi^{3}\sqrt{g_{*}(x_{F})}}\right],$
(25)

where the unitary parameter $\widetilde{c}\approx 5/4$.

Here we will consider the solution for the relic density which, at the same
time, solves the charged lepton masses and neutrino masses given in Eq. (18)
for the sets of parameters showed in Tables 3 and 4, in order to obtain the
PMNS matrix. We call them scenario 1 and 2, when $R_{1}$ and $I_{1}$ is the DM
candidate, respectively. In both scenarios the Yukawas values adjust the
squared masses differences for the neutrinos and the PMNS.

Scenario | $G^{\nu}_{es}$ | $G^{\nu}_{\mu s}$ | $G^{\nu}_{\tau s}$ | $G^{\nu}_{ed}$ | $G^{\nu}_{\mu d}$ | $G^{\nu}_{\tau d}$
---|---|---|---|---|---|---
1 | 1.17$\times 10^{-7}$ | $10^{-11}$ | $10^{-11}$ | $10^{-11}$ | $10^{-11}$ | 2.81$\times 10^{-7}$
2 | 1.56$\times 10^{-7}$ | $10^{-11}$ | $10^{-11}$ | $10^{-11}$ | $10^{-11}$ | 3.75$\times 10^{-7}$
Table 3: The Yukawas values for Eq. (19) that solve DM. Scenario | $M_{+1}$ | $M_{+2}$ | $m_{\zeta 1}$ | $m_{\zeta 2}$ | $M_{R1}$ | $M_{R2}$ | $M_{I1}$ | $M_{I2}$ | $M_{d}$ | $M_{s}$ | $a_{6}$
---|---|---|---|---|---|---|---|---|---|---|---
1 | 109.02 | 1477.64 | 749.81 | 3747.26 | 85.20 | 2085.74 | 161.84 | 2090.27 | 240 | 240 | -1.16$\times 10^{-1}$
2 | 109.12 | 1477.65 | 749.81 | 3747.26 | 257.25 | 2099.82 | 113.70 | 2087.10 | 240 | 240 | 2.6$\times 10^{-1}$
Table 4: Masses (in GeV) of the scalars in this model. $a_{6}$ is a
dimensionless coupling in the non-Hermitian quartic scalar interaction which
transfer the $L$ violation to the active neutrino sector.

With the numbers in Tables 3 and 4 we obtain the mixing matrix, that is:

$V^{\nu}_{L}\approx\left(\begin{array}[]{ccc}0.00010637&-1&0.0000444397\\\
-1&-0.000106374857&0.00007634160462\\\
0.000076336876851&0.000044447836932&1\end{array}\right),$ (26)

and the charged leptons has the following Yukawas: $G^{l}_{11}=0.0004219$,
$G^{l}_{12}=0.000515$, $G^{l}_{22}=0.00374772$, $G^{l}_{13}=-0.000801115$,
$G^{l}_{23}=-0.0035675$, $G^{l}_{33}=0.00367627$, and we obtain $m_{e}=0.510$
MeV, $m_{\mu}=105.658$ MeV and $m_{\tau}=1776.86$ MeV, and the mixing matrix
is

$V^{l}_{L}=\left(\begin{array}[]{ccc}0.564902&0.52704&0.634914\\\
0.814515&-0.479341&-0.326799\\\
-0.132104&-0.701756&0.700062\end{array}\right).$ (27)

As previously defined the mixing matrix for the leptonic sector
$V_{PMNS}=V^{l\dagger}_{L}V^{\nu}_{L}$, From Eqs. (26) and (27) we obtain
again Eq.(20).

To perform DM calculation we have used MicrOmegas package micromegas . For
instance, let us consider scenario 1, where $R_{1}$ is the DM candidate. In
the range of parameters used by us, DM annihilates mainly in $W^{+}W^{-}$.
Once again we emphasize that other solutions in other annihilation channels do
exist. We have chosen the following parameters for the couplings, and vacuum
expected value: $G_{\tau d}=2.81\times 10^{-7}$, $G_{\tau s}=1\times
10^{-11}$, $v_{\zeta}\lesssim\Lambda=1000$ GeV, $\lambda^{\prime}=2.7\times
10^{-2}$, $\lambda^{\prime\prime}=0.34$,
$\mu^{2}_{d}=2.809\,(\textrm{TeV})^{2}$, for values of other parameters see
Table 3. With this parameters choice $m_{R1}=85.15$ GeV. The parameter
dependence of $m_{R1}$ is presented in the last sections. So, the dominant
contributions for $\Omega$ are 99% in $R_{1}R_{1}\rightarrow W^{+}W^{-}$. In
this case, $x_{F}\sim 23.8$. The annihilation cross section is $\langle\sigma
v\rangle=1.0\times 10^{-26}$ cm${}^{3}/$s and the DM-nucleus cross section for
spin-independent elastic scattering is numerically given by
$\sigma_{SI}^{p}=5.84\times 10^{-46}$ cm2 and $\sigma_{SI}^{n}=6.70\times
10^{-46}$ cm2.

In scenario 2 we consider $I_{1}$ as the DM candidate. In this case, as a
result of the parameter choice ($G_{\tau d}=3.75\times 10^{-7}$, $G_{\tau
s}=1\times 10^{-11}$, $v_{\zeta}\lesssim\Lambda=1000$ GeV,
$\lambda^{\prime}=1$, $\lambda^{\prime\prime}=0.12$, $\mu^{2}_{d}=2830.24$
TeV2), $I_{1}$ annihilates 97% in $I_{1}I_{1}\rightarrow hh$ and 2% in
$I_{1}I_{1}\rightarrow b\overline{b}$. The value of other parameters can be
seen in Table 3. The annihilation cross section is $\langle\sigma
v\rangle=4.55\times 10^{-28}$ cm${}^{3}/$s and the DM-nucleus cross section
for spin-independent elastic scattering is numerically given by
$\sigma_{SI}^{p}=6.40\times 10^{-45}$ cm2 and $\sigma_{SI}^{n}=7.35\times
10^{-45}$ cm2.

In these two scenarios, we had set $R_{1}$ and $I_{1}$ as DM candidates,
making them lighter than the others possible neutral scalars. We emphasize
that other choices for DM are possible so that other annihilation channels may
also give interesting signatures. There is also the possibility that two,
three or even four of the neutral scalars contribute partially to the DM
density, but this case is beyond the scope of this paper.

In Fig. 3 we present the fluxes of photons, positrons and antiprotons in the
scenario 1 with $m_{DM}=85.15$ GeV and the scenario 2 with $m_{DM}=113.70$
GeV, where the upper limits of the energy spectrum are determined by the DM
masses since annihilation occurs near at rest. The model can accommodate DM
candidtes with smaller masses than the values above.

##  VI The leptonic decays $l_{i}\to l_{j}\gamma$ and $l_{i}\to
l_{j}l_{k}\bar{l}_{k}$.

In this section we study the impact of the new particles, the charged scalars
$D_{1,2}^{+}$ and the right-handed neutrinos $N_{s,2,3}$ in the lepton flavor
violating processes $l_{i}\to l_{j}\gamma$. Here we will consider these rare
decays in two cases: one in which we do not care with DM solutions and one in
which we use the parameters for having a DM candidates that also give the
correct lepton masses and the PMNS.

In terms of the leptons mass eigenstates, the interactions with charged
scalars from Eq. (II) are written as

$\displaystyle\mathcal{L}^{l-N}_{Yukawa}$ $\displaystyle=$
$\displaystyle-\bar{l}_{kL}V^{l\dagger}_{ki}\left[G^{\nu}_{id}(N_{2R}D^{-}_{1}+N_{3R}D^{-}_{2})+\frac{v_{\zeta}}{\Lambda}G^{\nu}_{is}N_{s}(D^{-}_{1}-D^{-}_{2})\right],$
(28)

$i,k=e,\mu,\tau$, the values of the entries of the matrices $G_{i,d}^{\nu}$
and $G_{i,s}^{\nu}$ are given in Table 3.

In the model, the allowed lepton flavor violation (LFV) decays $l_{i}\to
l_{j}\gamma$ and $l_{i}\to l_{j}l_{k}\bar{l}_{k}$ arise only at the 1-loop
level. These diagrams are generated by the known SM contribution
$W~{}\&~{}\nu_{l}$ and by the new content $D_{1}^{+}\&N_{2}$,
$D_{2}^{+}\&N_{3}$, $D_{1}^{+}\&N_{s}$ and $D_{2}^{+}\&N_{s}$.

For $l_{i}\to l_{j}\gamma$, it is known that the SM contribution is
extraordinarily suppressed with respect to the experimental capabilities of
detection, see Table 5. As we will show below, the new particle content in the
model predicts signals close to the experimental upper limits for the space of
our considered allowed parameters. Regarding the three body decay $l_{i}\to
l_{j}l_{k}\bar{l}_{k}$, it arises when in $l_{i}\to l_{j}\gamma$ we attach to
the photon the $\gamma l\bar{l}$ coupling. In the following we are interested
in presenting the $\mu\to ee\bar{e}$ channel, because it provides interesting
results near the experimental upper limit, while all the other channels are
out of the experimental interest region because the devices are unable of
reaching such suppressed signals.

In our study we have solved the amplitudes and the loop integrals with the
help of Mathematica, FeynCalc Mertig:1990an ; Shtabovenko:2016sxi , and
Package-X Patel:2015tea .

###  VI.1 Predictions of $\mu\to e\gamma$ and $\mu\to ee\bar{e}$ in the
scotogenic model without dark matter.

We start our numerical analysis of the decays in the scotogenic model without
DM content. Accordingly to the Yukawa values derived in the Sec. IV (see Fig.
2), the obtained values of the Yukawas are
$G_{ad,as}^{\nu}\in[10^{-11},10^{-1}]$, they satisfy the neutrino masses.
Notice that $G_{\tau d}^{\nu}\simeq G_{es}^{\nu}\gg G_{ed,\mu d,\mu s,\tau
s}^{\nu}$. We recall that all the previous analyse were done in the case of
$b_{3}=0$ in which the charged scalars are mass degenerated. As starting point
we consider the mass of the charged scalar in the range $m_{+}\in$ [80,750]
GeV, $N_{2,3,s}$ degenerated as well with values $m_{N}\in$ [250, 4000] GeV.
We have tested the five different parameterizations of the $V_{L}^{l}$
matrices derived in the Sec. IV (see Table 2). They can be separated into two
sets which will have two different behaviours in the processes, the set A is
conformed by the parameterizations P1 and P5, and the B by P2, P3 and P4.

For the channel $\mu\to e\gamma$ such situation occurs when $G_{\tau
d}^{\nu}=G_{es}^{\nu}\sim 10^{-1}$ and $G_{ed,\mu d,\mu s,\tau s}^{\nu}\sim
10^{-5}$. In the Fig. 5 (a)-(b) it is shown the Br$(\mu\to e\gamma)$ as
function of the sterile neutrino mass $m_{N}\in[250,4000]$ GeV with given
values for the charged scalar $m_{D^{+}}=80,250,500,750$ GeV. The current
experimental upper limit Br$(\mu\to e\gamma)^{\text{Exp}}<4.2\times 10^{-13}$,
indicated with the red line in the plots, will constrain the right-handed
neutrino mass $m_{N}$ for given values of $m_{+}$ in order to respect such
limit, those constraints are listed in the Table 6, where it is evident that
the $V_{L}^{l}$ parameterization A in Fig. 5 (a) allows a lighter mass for the
right-handed neutrino than the parameterization B in Fig. 5 (b).

Decay | Current limit | Future limit | SM
---|---|---|---
Br($\mu\to e\gamma$) |  $<4.2\times 10^{-13}$ TheMEG:2016wtm  |  $<6.0\times 10^{-14}$ Baldini:2013ke  | $10^{-48}$
Br($\tau\to e\gamma$) |  $<3.3\times 10^{-8\ }$ Olive:2016xmw  |  $<3.3\times 10^{-9\ }$ Aushev:2010bq  | $10^{-49}$
Br($\tau\to\mu\gamma$) |  $<4.4\times 10^{-8\ }$ Olive:2016xmw  |  $<3.3\times 10^{-9\ }$ Aushev:2010bq  | $10^{-49}$
Table 5: $l_{i}\to l_{j}\gamma$, experimental upper limits and the SM
predictions. $\mu\to e\gamma$
---
$m_{D^{+}}$ [GeV] |  $m_{N}$ [GeV]
$V_{L}^{l}$ parameterization A |  $V_{L}^{l}$ parameterization B
80 |  $>$1140 |  $>$2610
250 |  $>$1000 |  $>$2510
500 |  $>$ 710 |  $>$2300
750 |  $>$ 295 |  $>$2040
Table 6: Mass constraints for the right-handed neutrino with fixed values of
$m_{D^{+}}$ in order to respect Br$(\mu\to e\gamma)^{\text{Exp}}<4.2\times
10^{-13}$, here $G_{\tau d}^{\nu}=G_{es}^{\nu}\sim 10^{-1}$ and $G_{ed,\mu
d,\mu s,\tau s}^{\nu}\sim 10^{-5}$. Br$(\mu\to ee\bar{e})$
---
$V_{L}^{l}$ parameterization A |  $V_{L}^{l}$ parameterization B
$<2.2\times 10^{-15}$ | $<1.4\times 10^{-15}$
Table 7: Br$(\mu\to ee\bar{e})$ predictions from the Br$(\mu\to e\gamma)$
constraints.

Regarding to the subcase $\mu\to ee\bar{e}$, we are able to predict the
branching ratio from the neutrino right-handed mass constraints obtained for
the $\mu\to e\gamma$ channel. These predictions are organized in the Table 7,
and such values are indicated with the green line in the Fig. 5 (c) and (d),
being of the same order of magnitude $\sim 10^{-15}$ for both scenarios.

About the analogous tau decays, for the same space of parameter values than in
the $\mu\to e\gamma$ case, and respecting the obtained mass bounds, we have
found that Br$(\tau\to e\gamma)\leq 10^{-12}$ and Br$(\tau\to e\gamma)\leq
10^{-14}$, which are beyond the current and upcoming experimental capabilities
of detection, see Table 5 for comparison.

###  VI.2 Predictions of $l_{i}\to l_{j}\gamma$ in the scotogenic model with
dark matter

As commented in the Sec. V, the model can be extended to include DM. In order
to estimate the consequences on the transition $\mu\to e\gamma$, we consider
the Yukawa values given in Table 3. The resulting prediction with $G_{\tau
d}^{\nu}=G_{es}^{\nu}=10^{-7}$ is

$\text{Br}(\mu\to e\gamma)=10^{-34},$ (29)

which is beyond the scope of detection.

##  VII Conclusions

Here we have considered an extension of the SM with three scalar doublets of
$SU(2)$ with $S_{3}$ and $\mathbb{Z}_{2}$ symmetries. We had analysed all the
mass spectra in the scalar sectors and used the scotogenic mechanism for
generating neutrino masses. Moreover, we had obtained the PMNS matrix once the
unitary matrices which diagonalize the lepton masses are obtained. Although
the model can have many DM candidates, we have shown two cases in which the DM
candidate is a $C\\!P$ even scalar (scenario-1) and other one in which the DM
is composite of $C\\!P$ odd scalar (scenario-2). But we emphasize that other
possible choices for DM candidates are possible, considering for example,
smaller masses, since besides the SM-like scalar, we have eight additional
neutral scalars in the model. The study of other candidates and other channels
of annihilation will be done soon. We had exemplified in some range of
parameters space, two DM candidates for the model. For the scenarios 1 and 2
presented, DM annihilates mainly in $W^{+}W^{-}$ and $hh$ respectively. We
have also presented some fluxes for this model. Of course, there may be other
possible scenarios which could explain the Galactic gamma ray excess, as well
as the the PAMELA and AMS-02 results. These processes could tightly constrain
the parameter space of this sort of scotogenic models.

The considered scotogenic model without DM provides optimistic predictions for
possible detection of the LFV decay $\mu\to e\gamma$ due to our solution space
of the Yukawa values, which adjusts the squared masses differences for the
neutrinos and the PMNS matrix. Our estimations predict a mass for the right-
handed neutrino starting from $m_{N}>295$ GeV, and from $\mu\to e\gamma$ we
predict Br$(\mu\to ee\bar{e})\lesssim 10^{-15}$. On the other hand,
considering DM content in the model we found that Br$(\mu\to e\gamma)\sim
10^{-34}$, which is out of detection range.

###### Acknowledgements.

ACBM thanks CAPES for financial support, JM thanks to FAPESP for financial
support under the processe number 2013/09173-5, and VP thanks to CNPq for
partial financial support.

## References

  * (1) G. Aad et al. [ATLAS Collaboration], Observation of a new particle in the search for the Standard Model Higgs boson with the ATLAS detector at the LHC, Phys. Lett. B 716, 1 (2012); [arXiv:1207.7214 [hep-ex]].
  * (2) S. Chatrchyan et al. [CMS Collaboration], Observation of a new boson at a mass of 125 GeV with the CMS experiment at the LHC, Phys. Lett. B 716, 30 (2012); [arXiv:1207.7235 [hep-ex]].
  * (3) N. G. Deshpande and E. Ma, Pattern Of Symmetry Breaking With Two Higgs Doublets, Phys. Rev. D 18, 2574 (1978).
  * (4) E. Ma, Verifiable radiative seesaw mechanism of neutrino mass and dark matter, Phys. Rev. D 73, 077301 (2006); [hep-ph/0601225].
  * (5) M. Aoki, J. Kubo and H. Takano, Multicomponent Dark Matter in Radiative Seesaw Model and Monochromatic Neutrino Flux, Phys. Rev. D 90, no. 7, 076011 (2014); [arXiv:1408.1853 [hep-ph]].
  * (6) K. R. Dienes, J. Kumar, B. Thomas and D. Yaylali, Dark-Matter Decay as a Complementary Probe of Multicomponent Dark Sectors, Phys. Rev. Lett. 114, no. 5, 051301 (2015); [arXiv:1406.4868 [hep-ph]].
  * (7) L. Accardo et al. [AMS Collaboration], High Statistics Measurement of the Positron Fraction in Primary Cosmic Rays of 0.5–500 GeV with the Alpha Magnetic Spectrometer on the International Space Station, Phys. Rev. Lett. 113, 121101 (2014).
  * (8) L. Goodenough and D. Hooper, Possible Evidence For Dark Matter Annihilation In The Inner Milky Way From The Fermi Gamma Ray Space Telescope, arXiv:0910.2998 [hep-ph].
  * (9) T. Daylan, D. P. Finkbeiner, D. Hooper, T. Linden, S. K. N. Portillo, N. L. Rodd and T. R. Slatyer, The characterization of the gamma-ray signal from the central Milky Way: A case for annihilating dark matter, Phys. Dark Univ. 12, 1 (2016); [arXiv:1402.6703 [astro-ph.HE]].
  * (10) C. Q. Geng, D. Huang and C. Lai, Revisiting multicomponent dark matter with new AMS-02 data, Phys. Rev. D 91, no. 9, 095006 (2015); [arXiv:1411.4450 [hep-ph]].
  * (11) K. N. Abazajian and R. E. Keeley, Bright gamma-ray Galactic Center excess and dark dwarfs: Strong tension for dark matter annihilation despite Milky Way halo profile and diffuse emission uncertainties, Phys. Rev. D 93, no. 8, 083514 (2016) doi:10.1103/PhysRevD.93.083514 [arXiv:1510.06424 [hep-ph]].
  * (12) A. Albert et al. [Fermi-LAT and DES Collaborations], Searching for Dark Matter Annihilation in Recently Discovered Milky Way Satellites with Fermi-LAT, Astrophys. J. 834, no. 2, 110 (2017) doi:10.3847/1538-4357/834/2/110 [arXiv:1611.03184 [astro-ph.HE]].
  * (13) J. Liu, X. Chen and X. Ji, Current status of direct dark matter detection experiments, Nature Phys. 13, no. 3, 212 (2017).
  * (14) A. C. B. Machado and V. Pleitez, A model with two inert scalar doublets, Annals Phys. 364, 53 (2016); [arXiv:1205.0995 [hep-ph]].
  * (15) M. C. Gonzalez-Garcia, M. Maltoni, J. Salvado and T. Schwetz, Global fit to three neutrino mixing: critical look at present precision, JHEP 1212, 123 (2012); [arXiv:1209.3023 [hep-ph]].
  * (16) E. C. F. S. Fortes, A. C. B. Machado, J. Montaño and V. Pleitez, Scalar dark matter candidates in a two inert Higgs doublet model, J. Phys. G 42, no. 10, 105003 (2015); [arXiv:1407.4749 [hep-ph]].
  * (17) M. Beltran, D. Hooper, E. W. Kolb and Z. C. Krusberg, Deducing the nature of dark matter from direct and indirect detection experiments in the absence of collider signatures of new physics, Phys. Rev. D 80, 043509 (2009); [arXiv:0808.3384 [hep-ph]].
  * (18) G. Bélanger, F. Boudjema, A. Pukhov and A. Semenov, Comput. Phys. Commun. 192, 322 (2015) doi:10.1016/j.cpc.2015.03.003 [arXiv:1407.6129 [hep-ph]].
  * (19) R. Mertig, M. Bohm and A. Denner, FEYN CALC: Computer algebraic calculation of Feynman amplitudes, Comput. Phys. Commun. 64, 345 (1991).
  * (20) V. Shtabovenko, R. Mertig and F. Orellana, New Developments in FeynCalc 9.0, arXiv:1601.01167 [hep-ph].
  * (21) H. H. Patel, Package-X: A Mathematica package for the analytic calculation of one-loop integrals, Comput. Phys. Commun. 197, 276 (2015); [arXiv:1503.01469 [hep-ph]].
  * (22) N. Chakrabarty, D. K. Ghosh, B. Mukhopadhyaya and I. Saha, Dark matter, neutrino masses and high scale validity of an inert Higgs doublet model, Phys. Rev. D 92, no. 1, 015002 (2015); [arXiv:1501.03700 [hep-ph]].
  * (23) L. Ping and F. Y. Yu, Criteria for Copositive Matrices of Order Four, Linear Algebra Appl. 194, 109 (1993).
  * (24) K. Kannike, Vacuum Stability Conditions From Copositivity Criteria, Eur. Phys. J. C 72, 2093 (2012); [arXiv:1205.3781 [hep-ph]].
  * (25) A. M. Baldini et al. [MEG Collaboration], “Search for the lepton flavour violating decay $\mu^{+}\rightarrow\mathrm{e}^{+}\gamma$ with the full dataset of the MEG experiment,” Eur. Phys. J. C 76, no. 8, 434 (2016) doi:10.1140/epjc/s10052-016-4271-x [arXiv:1605.05081 [hep-ex]].
  * (26) A. M. Baldini et al., “MEG Upgrade Proposal,” arXiv:1301.7225 [physics.ins-det].
  * (27) C. Patrignani et al. [Particle Data Group], “Review of Particle Physics,” Chin. Phys. C 40, no. 10, 100001 (2016). doi:10.1088/1674-1137/40/10/100001
  * (28) T. Aushev et al., “Physics at Super B Factory,” arXiv:1002.5012 [hep-ex].

(a)  (b)

Figure 1: One-loop neutrino mass generation. Here A, B, C and D are the
Yukawas given in Eq. (II), with $\textbf{A}=G_{id}^{\nu}$,
$\textbf{B}=G_{jd}^{\nu}$, $\textbf{C}=v_{\zeta}G_{is}/\Lambda$, and
$\textbf{D}=v_{\zeta}G_{js}/\Lambda$.

(a)  (b)

(c)

Figure 2: Dependence of $a_{6}$ with respect to the main Yukawas $G_{\tau
d}^{\nu}$ (a) and $G_{es}^{\nu}$ (b) for fixed $M_{R}$ values. Notice that
$G_{\tau d}^{\nu}\simeq G_{es}^{\nu}$. In (c) the Yukawas $G_{ed,\mu d,\mu
s,\tau s}^{\nu}$ provide the same value for any $M_{R}$ (overlapped curves).

(a)  (b)

Figure 3: Prediction of fluxes for photons, protons, and anti-protons in the
scotogenic model. Scenarios (a) 1 and (b) 2.

(a)  (b)

Figure 4: Decays (a) $l_{i}\to l_{j}\gamma$ and (b) $l_{i}\to
l_{j}l_{k}\bar{l}_{k}$ in the scotogenic model. Generic sample contributions
from a sterile neutrino $N$ and a charged scalar $D^{+}$.

(a)  (b)

(c)  (d)

Figure 5: Decays $\mu\to e\gamma$ in (a)-(b) and $\mu\to ee\bar{e}$ in (c)-(d)
in the scotogenic model without DM, with $G_{\tau d,es}^{\nu}=10^{-1}$ and
$G_{ed,\mu d,\mu s,\tau s}^{\nu}=10^{-5}$.