∎ e1 11institutetext: Peking University, Beijing 100871, People’s Republic of
China 22institutetext: State Key Laboratory of Nuclear Physics and Technology,
Beijing 100871, People’s Republic of China 33institutetext: University of
Science and Technology of China, Hefei 230026, People’s Republic of China
44institutetext: State Key Laboratory of Particle Detection and Electronics,
Hefei 230026, People’s Republic of China

# Sensitivity study of the charged lepton flavor violating process
$\tau\to\gamma\mu$ at STCF

Teng Xiange1,addr1,addr2  Xiao-Dong Shiaddr3,addr4  Da-Yong Wangaddr1,addr2
Xiao-Rong Zhouaddr3,addr4<EMAIL_ADDRESS>
(Received: date / Accepted: date)

###### Abstract

A sensitivity study for the search for the charged lepton flavor violating
process $\tau\to\gamma\mu$ at the Super $\tau$-Charm Facility is performed
with a fast simulation. With the expected performance of the current detector
design and an integrated luminosity of $1\text{\,}\mathrm{a}\mathrm{b}^{-1}$
corresponding to one-year of data taking, the sensitivity on the branching
fraction (BF) of $\tau\to\gamma\mu$ is estimated to be at the level of
${10}^{-8}$. The sensitivity under different detector performances are also
studied. With ideal performance, the BF could be probed to be
$2.8\text{\times}{10}^{-8}$ at $90\text{\,}\mathrm{\char 37\relax}$ confidence
level. The sensitivity is expected to scale with the square root of the
luminosity, therefore with a total luminosity of
$10\text{\,}\mathrm{a}\mathrm{b}^{-1}$ corresponding to ten-year of data
taking, the sensitivity could reach $8.8\text{\times}{10}^{-9}$, which is
about one order of magnitude improvement upon the current best upper limit.

††journal: Eur. Phys. J. C

##  1 Introduction

In the Standard Model (SM), the charged lepton flavor violating (cLFV)
processes can occur through neutrino oscillation, but are highly suppressed
due to the small mass of neutrino SM_prediction (1). For example, the
branching fraction (BF) of $\ell_{1}\to\ell_{2}\gamma$ in the SM is

$\text{BF}(\ell_{1}\to\ell_{2}\gamma)=\frac{3\alpha_{e}}{32\pi}\left|\sum_{i}U_{1i}^{*}U_{2i}\frac{m_{i}^{2}}{M_{W}^{2}}\right|^{2}\approx
10^{-50}\sim 10^{-54},$ (1)

where $U$ is the PMNS matrix PMNS (2), $i$ runs over the three neutrinos,
$m_{i}$ and $m_{W}$ are the masses of neutrinos and $W$ boson. The BF in the
SM is well beyond the sensitivity of current experiments, thus the observation
of cLFV processes would be an unambiguous signature of new physics. On the
other hand, the lepton flavor conservation, differing from other conservation
laws in the SM, is not associated with an underlying conserved current,
therefore many theoretical models beyond the SM naturally introduce cLFV
processes, such as the Minimal Supersymmetric SM MSSM (3), the Grand Unified
Theories GUT1 (4, 5), and seesaw mechanisms seesaw (6). Some of them predict
BFs that are close to the current experimental sensitivity.

As the heaviest lepton, tau has many possible cLFV decay modes, amongst them
$\tau\to\gamma\mu$ is regarded as one of the best probes, and is predicted in
a wide variety of models with rates enhanced to observable level. For example,
the BF is predicted to be up to ${10}^{-9}$ in seesaw models prediction_seesaw
(7), ${10}^{-10}$ in Higgs-mediated SUSY models prediction_MSSM_1 (8, 9),
${10}^{-8}$ in SO(10) SUSY models prediction_SUSY_SO10_1 (10, 11), and
${10}^{-9}$ in non-universal $Z^{\prime}$ models prediction_Zprime (12).
Experimentally, the most stringent upper limit (UL) on the BF of this channel
is given by BABAR to be $4.4\text{\times}{10}^{-8}$ at
$90\text{\,}\mathrm{\char 37\relax}$ confidence level (C.L.) BABAR (13) and
Belle to be $4.2\text{\times}{10}^{-8}$ at $90\text{\,}\mathrm{\char
37\relax}$ C.L. Belle (14). Next generation of experiments are aiming at
pushing the sensitivity down for another one order of magnitude or even
further Snowmass_tau_LFV (15).

The proposed Super $\tau$-Charm Facility (STCF) STCF_CDR (16) in China, which
is an electron-positron collider with a center-of-mass energy in the region of
the $\tau$-charm threshold, is one of such next generation of experiments. In
this paper, the sensitivity of searching for $\tau\to\gamma\mu$ at STCF is
studied to explore the physics potential of STCF and guide the design of the
experiment. STCF has several advantages on searching for $\tau\to\gamma\mu$.
As an electron-positron collider, the total four-momentum is known and the
final state is fully reconstructed, leading to higher efficiency and lower
background. The energy of STCF can be adjusted to be just above the threshold
of tau pair production $e^{+}e^{-}\to\tau^{+}\tau^{-}$, where the cross-
section reaches the maximum tau_pair_xs (17), and the energy of radiative
photons which is one of the main sources of fake signal photons in background
is low thus can be well separated from signal.

##  2 Detector design and Monte Carlo simulation

The proposed STCF is a symmetric electron-positron collider operating at
center-of-mass energy $\sqrt{s}$ from
$2.0\text{\,}\mathrm{G}\mathrm{e}\mathrm{V}$ to
$7.0\text{\,}\mathrm{G}\mathrm{e}\mathrm{V}$ with a designed peaking
luminosity over
$0.5\text{\times}{10}^{35}\text{\,}\mathrm{c}\mathrm{m}^{-2}\,\mathrm{s}^{-1}$
STCF_CDR (16). Such an environment will serve as an important high statistics
and low background platform to test the SM and probe possible new physics
beyond the SM, such as cLFV decays of tau. Assuming that it runs at
$\sqrt{s}=$4.26\text{\,}\mathrm{G}\mathrm{e}\mathrm{V}$$, STCF will accumulate
$3.5\text{\times}{10}^{9}$ tau pairs KKMC1 (18, 19) per year for the
corresponding expected integrated luminosity of
$1\text{\,}\mathrm{a}\mathrm{b}^{-1}$.

As a general purpose detector designed for the electron-positron collider, the
STCF detector consists a tracking system composed of the inner and outer
trackers, a particle identification (PID) system with charged kaon/pion
misidentification rate less than $2\text{\,}\mathrm{\char 37\relax}$ up to
$2\text{\,}\mathrm{G}\mathrm{e}\mathrm{V}\mathrm{/}\mathrm{\text{$c$}}$, an
electromagnetic calorimeter (EMC) with an excellent energy resolution and a
good time resolution, a super-conducting solenoid, and a muon detector (MUD)
that provides good charged pion/muon separation. The design and the expected
performances for each sub-detector are detailed in the conceptual design
report STCF_CDR (16).

At present, the STCF detector and the corresponding offline software system
are under research and development. A fast simulation software package is
therefore developed to access the physics study FastSim (20), which takes the
most common event generators as input to perform a fast and realistic
simulation. The simulation includes resolution and efficiency responses for
tracking of final state charged particles and photons, PID system and
kinematic fit related variables. Instead of performing the detailed simulation
of interaction of the final objects with detector by Geant4 Geant4 (21), the
responses of objects in each sub-system, including efficiency, resolution and
other variables used in physics analyses, are modeled by sampling the shape
and magnitude according to their performances. By default, all the parameters
are based on empirical formulae or extracted from simulation of the BESIII
detector BESIII (22, 23), while the fast simulation also provides flexible
interface for adjusting performance of each sub-system, which can be used to
optimize the detector design according to the physics requirements.

This sensitivity study is performed based on Monte Carlo (MC) samples at
$4.26\text{\,}\mathrm{G}\mathrm{e}\mathrm{V}$, including samples for signal
and expected possible background processes. Using the tag method with selected
modes as detailed in Sec. 3, the major expected background processes are those
that are discussed in the rest of this section. One of them is the dimu
process $e^{+}e^{-}\to(\gamma)\mu^{+}\mu^{-}$ where the two muons and radiated
photons can be misidentified as the tag pion and the signal muon and photon.
Another background is the ditau process $e^{+}e^{-}\to\tau^{+}\tau^{-}$ where
one tau decay is necessarily the same as tag side of signal event and the
other tau decay can be misidentified as $\tau\to\gamma\mu$. The hadronic
process $e^{+}e^{-}\to q\bar{q}(q=u,d,s,c)$ could also contribute to the
background due to its high cross-section at this energy region. Samples for
these processes are simulated with high luminosity, corresponding to
$1\text{\,}\mathrm{a}\mathrm{b}^{-1}$, $6\text{\,}\mathrm{a}\mathrm{b}^{-1}$,
and $1\text{\,}\mathrm{a}\mathrm{b}^{-1}$, respectively. The dimu samples are
generated with Babayaga Babayaga1 (24, 25). For the ditau samples, the
production of tau pair is generated with KKMC KKMC1 (18, 19) and the decay of
tau with Tauola Tauola (26). Hadronic process is generated with LundArlw
LundArlw (27). Other processes at this energy region are expected to be
negligible. The background level of Bhabha process $e^{+}e^{-}\to e^{+}e^{-}$
and digamma process $e^{+}e^{-}\to\gamma\gamma$ are much lower than dimu
process due to the low probability of electron misidentified as muon or the
lack of charged tracks. For the two-photon process $e^{+}e^{-}\to e^{+}e^{-}X$
where $X$ is any system generated from the virtual photons radiated by the
initial electron and positron, the electron and position tend to have small
scattering angles, thus escape from the detector along the beam energy and
cause large energy loss, and the remaining $X$ can hardly pass the event
selection. Dedicated MC samples for these processes are also generated, and it
is demonstrated that they can be removed at early stage of event selection.
The signal MC samples are simulated as process $e^{+}e^{-}\to\tau^{+}\tau^{-}$
with one tau goes to SM decay modes and the other decays to $\gamma\mu$. The
decay $\tau\to\gamma\mu$ is generated with the pure phase space model since
the dynamics is unknown.

##  3 Analysis procedure

At STCF, $\tau^{+}$ and $\tau^{-}$ are produced in pairs, so we can tag
$\tau^{+}$ (denoted as tag side) by its SM decay modes and search for cLFV
decay of its partner $\tau^{-}$ (denoted as signal side). The charge-
conjugated channels are always implied throughout the paper. For the tag side,
amongst the five main 1-prong decay modes of $\tau^{+}$,
$e^{+}\nu_{e}\bar{\nu}_{\tau}$, $\pi^{+}\bar{\nu}_{\tau}$ and
$\pi^{+}\pi^{0}\bar{\nu}_{\tau}$ are selected as tag modes, which account for
$54\text{\,}\mathrm{\char 37\relax}$ of the total BF of tau decays PDG (28).
$\tau^{+}\to\mu^{+}\nu_{\mu}\bar{\nu}_{\tau}$ mode is not used due to high
$e^{+}e^{-}\to(\gamma)\mu^{+}\mu^{-}$ background. As for
$\tau^{+}\to\pi^{+}\pi^{0}\pi^{0}\bar{\nu}_{\tau}$ mode, due to the high
photon multiplicity, the reconstruction efficiency is low and combinatorial
background is high, and the tag photons can be easily misidentified as signal
photon. The signal side consists of a signal photon and a signal muon and is
featured by a peak around the beam energy on the total energy distribution and
a peak around the tau mass on the invariant mass distribution of signal photon
and muon, as shown in Fig. 1.

Figure 1: The box plot is the two-dimensional distribution of $E(\gamma\mu)$
and $M(\gamma\mu)$ of signal samples where larger box indicates higher
density, and the red line shows the signal region.

Charged tracks are selected after passing the criteria in fast simulation. For
each charged track, the polar angle with respect to beam direction is required
to satisfy $\lvert\cos\theta\rvert<0.93$, and the closest approach to the
interaction point must be within $10\text{\,}\mathrm{c}\mathrm{m}$ in the beam
direction and $1\text{\,}\mathrm{c}\mathrm{m}$ in the plane perpendicular to
the beam. Events with two charged tracks and zero net-charge are selected. PID
is also performed for charged tracks by the fast simulation, where the
identification of electron, pion and kaon is based on extracted likelihood of
each particle type hypothesis calculated from the information of PID system
and the one with the highest likelihood is assigned, while muon is identified
by MUD based on preliminary research and development results. One of the
tracks is required to be identified as electron or pion, which is tag charged
particle, and the other is required to be identified as muon, which is signal
muon. $E/p$ information is then used for further identification, where $E$ is
the deposited energy in EMC and $p$ is the momentum of the track. $E/p>0.8$ is
required for electron and $E/p<0.5$ for muon and pion. Photons are selected
with an energy threshold of $25\text{\,}\mathrm{M}\mathrm{e}\mathrm{V}$ in EMC
barrel region ($\lvert\cos\theta\rvert<0.8$) and
$50\text{\,}\mathrm{M}\mathrm{e}\mathrm{V}$ in EMC end cap region
($0.86<\lvert\cos\theta\rvert<0.92$) to reject electronic noise and unrelated
energy depositions. Neutral pions are reconstructed with two-photon
combinations with invariant masses within a $\pi^{0}$ mass window of around
$0.12\text{\,}\mathrm{G}\mathrm{e}\mathrm{V}\mathrm{/}\mathrm{\text{$c$}}^{2}$
to
$0.14\text{\,}\mathrm{G}\mathrm{e}\mathrm{V}\mathrm{/}\mathrm{\text{$c$}}^{2}$
determined by fitting. The exact mass window depends on the detector
performance design described later in Sec. 4. It is required that there is
exactly one remaining photon after neutral pions reconstruction and it is
denoted as the signal photon. To further suppress background, the momentum of
the signal muon and energy of the signal photon are both required to be in
$0.0]\text{\,}\mathrm{0}\,\mathrm{4}$1.7GeV, and the angle between them is
required to satisfy $\cos\theta_{\gamma\mu}<-0.35$, all of which are
constrained by the kinematics (shown in Fig. 2). Finally, a two-dimensional
signal region is chosen on the total energy $E(\gamma\mu)$ and invariant mass
$M(\gamma\mu)$ distributions of signal photon and signal muon. Since the two
distributions are asymmetric and correlated, the signal region is an
asymmetric oblique ellipse, as shown in Fig. 1.

(a)

(b)

(c)

Figure 2: The kinematic distributions of 2(a) momentum of signal muon, 2(b)
energy of signal photon, and 2(c) cosine of angle between signal photon and
muon. The open histogram is signal, and the hatched histograms are background
drawn in stacks. The arrows mark the event selection criteria. The lower end
cutoff on momentum of muon is caused by MUD acceptance.

The tag mode for each event is then assigned based on the event selection
result. The event is classified into $e^{+}\nu_{e}\bar{\nu}_{\tau}$ mode if
one electron is identified, and it is required that there are no neutral pions
reconstructed. If one charged pion is identified, the event is further
classified into $\pi^{+}\bar{\nu}_{\tau}$ or $\pi^{+}\pi^{0}\bar{\nu}_{\tau}$
mode based on whether the number of neutral pions is 0 or 1. Events that do
not fit into the classifications are discarded.

After above initial selections, there are mainly five kinds of background
classified according to the tag modes and background processes. For
$e^{+}\nu_{e}\bar{\nu}_{\tau}$ tag mode, the main background is ditau process
where tag tau radiative decays to electron, signal tau SM decays to muon, and
signal photon is misidentified from radiative photon of tag side. For
$\pi^{+}\bar{\nu}_{\tau}$ tag mode, the main background is dimu and ditau
processes. In the dimu process, one muon is misidentified as tag pion, the
other muon is regarded as signal muon, and signal photon is misidentified from
radiative photon. For backgrounds from the ditau process, signal tau SM decays
to muon, tag tau decays to $\pi\pi^{0}$ with $\pi^{0}$ not successfully
reconstructed and the daughter photons are regarded as signal photon or not
detected. For the $\pi^{+}\pi^{0}\bar{\nu}_{\tau}$ tag mode, one of the main
background is the ditau process with tag tau decaying to $\pi\pi^{0}$ or
$\pi\pi^{0}\pi^{0}$ which is similar to the ditau background process in
$\pi^{+}\bar{\nu}_{\tau}$ tag mode. The other is hadronic process, mainly
$\pi^{+}\pi^{-}+(\geq 1)\pi^{0}$, where the signal muon is misidentified from
charged pion and the signal photon is from $\pi^{0}$(s) that are not
successfully reconstructed.

Further event selection criteria are determined based on the background
characteristics. For the ditau background in the
$e^{+}\nu_{e}\bar{\nu}_{\tau}$ tag mode, the signal photon is from radiative
leptonic decay of tag tau, thus collinear with the tag charged track, as shown
in Fig. 3(a). Moreover, the momentum of tag charged track is lowered due to
the existence of radiative photon (Fig. 3(b)). Since neutrinos in the final
states are not detected, there will be missing four-momentum defined as the
total initial four-momentum subtracted by the four-momenta of all detected
final state particles. There are more neutrinos in background than in signal,
so the missing energy $E_{\text{miss}}$ in background is higher (Fig. 3(c)),
where $E_{\text{miss}}$ is defined as energy component of missing four-
momentum. For the radiative dimu background in $\pi^{+}\bar{\nu}_{\tau}$ tag
mode, $E_{\text{miss}}$ is lower since there are no neutrinos (Fig. 4(a)), and
the direction of missing momentum accumulates along the beam direction since
missing momentum is mainly due to radiative photons which are collinear with
beam and can escape in the beam direction which is beyond detector coverage
(Fig. 4(b)). Furthermore, the energy of the signal photon is lower since it is
from radiation (Fig. 4(c)). For ditau background in $\pi^{+}\bar{\nu}_{\tau}$
tag mode, in contrast to the signal which has only one neutrino, there are
more neutrinos in background, so the missing mass squared
$M^{2}_{\text{miss}}$ is around zero for signal while not for background (Fig.
4(d)). The $M^{2}_{\text{miss}}$ is defined as the square of invariant mass of
the missing four-momentum. For ditau background in
$\pi^{+}\pi^{0}\bar{\nu}_{\tau}$ tag mode, the $M^{2}_{\text{miss}}$
distribution has similar characteristic with $\pi^{+}\bar{\nu}_{\tau}$ tag
mode (Fig. 5(a)). The distribution of helicity angle of signal muon is also
different in signal and background due to different decay dynamics of signal
tau (Fig. 5(b)), where the helicity angle is defined as the angle between the
direction of signal muon in the signal tau rest frame and the direction of
signal tau in center-of-mass frame. For the hadronic background in
$\pi^{+}\pi^{0}\bar{\nu}_{\tau}$ tag mode, which is mainly
$\pi^{+}\pi^{-}+(\geq 1)\pi^{0}$, the missing momentum is due to photons
escaping along the beam direction (Fig. 5(c)).

(a)

(b)

(c)

Figure 3: The comparison of signal (red solid histogram) and background (green
dashed histogram) samples in $e^{+}\nu_{e}\bar{\nu}_{\tau}$ tag mode. 3(a) the
cosine of the angle between signal photon and tag charged track
$\cos\theta_{\text{sig\\_}\gamma,\text{tag\\_charged}}$, 3(b) the momentum of
tag charged track $p_{\text{tag\\_charged}}$, and 3(c) the missing energy
$E_{\text{miss}}$ in ditau background.

(a)

(b)

(c)

(d)

Figure 4: The comparison of signal (red solid histogram) and background (green
dashed histogram) samples in $\pi^{+}\bar{\nu}_{\tau}$ tag mode. 4(a) the
missing energy $E_{\text{miss}}$, 4(b) the absolute value of cosine of angle
of missing momentum with respect to beam direction
$\lvert\cos\theta_{\text{miss}}\rvert$, and 4(c) the energy of signal photon
$E_{\text{sig\\_}\gamma}$ in dimu background, and 4(d) the missing mass
squared $M^{2}_{\text{miss}}$ in ditau background.

(a)

(b)

(c)

Figure 5: The comparison of signal (red solid histogram) and background (green
dashed histogram) samples in $\pi^{+}\pi^{0}\bar{\nu}_{\tau}$ tag mode. 5(a)
the missing mass squared $M^{2}_{\text{miss}}$ and 5(b) the cosine of the
helicity angle of signal muon
$\cos\theta_{\text{sig\\_mu\\_in\\_tau},\text{tau\\_in\\_cm}}$ in ditau
background, and 5(c) direction of missing momentum
$\lvert\cos\theta_{\text{miss}}\rvert$ in hadronic background.

To determine the concrete selection criteria, Punzi significance
$\varepsilon/(1.5+\sqrt{N_{\text{bkg}}})$ Punzi (29) is used as the figure of
merit, where $\varepsilon$ is signal efficiency and $N_{\text{bkg}}$ is the
number of background events. A multidimensional optimization is performed for
all the criteria simultaneously. The optimization result depends on the
detector performance design, and the result with the best performance is shown
here. Table 1 summarizes the further selection criteria and background levels
and signal efficiencies before and after further selection. The final
background level is suppressed to be only a few with signal efficiency of
several percents.

Table 1: The result of further event selection. The first column is the tag
modes, the second column is the selection criteria, the third and fourth
columns are the number of background $N_{\text{bkg}}$ and signal efficiency
$\epsilon$ before and after further selection.

tag mode | selection criteria |  $N_{\text{bkg}}$ ($\epsilon$) before |  $N_{\text{bkg}}$ ($\epsilon$) after
---|---|---|---
$e^{+}\nu_{e}\bar{\nu}_{\tau}$ | $\cos\theta_{\text{sig\\_}\gamma,\text{tag\\_charged}}<-0.2$ | $1.5\text{\times}{10}^{2}$ ($2.6\text{\,}\mathrm{\char 37\relax}$) | $0$ ($1.1\text{\,}\mathrm{\char 37\relax}$)
$p_{\text{tag\\_charged}}>$0.5\text{\,}\mathrm{G}\mathrm{e}\mathrm{V}\mathrm{/}\mathrm{\text{$c$}}$$
$E_{\text{miss}}<$1.7\text{\,}\mathrm{G}\mathrm{e}\mathrm{V}$$
$\pi^{+}\bar{\nu}_{\tau}$ | $E_{\text{miss}}>$0.7\text{\,}\mathrm{G}\mathrm{e}\mathrm{V}$$ | $1.4\text{\times}{10}^{4}$ ($4.0\text{\,}\mathrm{\char 37\relax}$) | $0.3$ ($0.5\text{\,}\mathrm{\char 37\relax}$)
$\lvert\cos\theta_{\text{miss}}\rvert<0.6$
$E_{\text{sig\\_}\gamma}>$0.8\text{\,}\mathrm{G}\mathrm{e}\mathrm{V}$$
$M^{2}_{\text{miss}}<$0.050\text{\,}\mathrm{G}\mathrm{e}\mathrm{V}^{2}\mathrm{/}\mathrm{\text{$c$}}^{4}$$
$\pi^{+}\pi^{0}\bar{\nu}_{\tau}$ | $M^{2}_{\text{miss}}<$0.075\text{\,}\mathrm{G}\mathrm{e}\mathrm{V}^{2}\mathrm{/}\mathrm{\text{$c$}}^{4}$$ | $1.2\text{\times}{10}^{3}$ ($2.6\text{\,}\mathrm{\char 37\relax}$) | $1.2$ ($1.5\text{\,}\mathrm{\char 37\relax}$)
$\cos\theta_{\text{sig\\_mu\\_in\\_tau},\text{tau\\_in\\_cm}}<0.8$
$\lvert\cos\theta_{\text{miss}}\rvert<0.9$
total |  |  $1.6\text{\times}{10}^{4}$ ($9.2\text{\,}\mathrm{\char 37\relax}$) |  $1.5$ ($3.1\text{\,}\mathrm{\char 37\relax}$)

A Bayesian-based maximum likelihood estimator, extended from the profile
likelihood approach UL_method (30), is used to determine the UL on BF of
$\tau\to\gamma\mu$ with statistical fluctuations taking into account. The
likelihood is constructed as

$\begin{split}\mathcal{L}&=\text{Poisson}(N_{\text{obs}},2N_{\tau^{+}\tau^{-}}\times\mathcal{B}\times\epsilon+\sum_{i}N_{\text{bkg},i})\\\
&\times\prod_{i}\text{Poisson}(N_{\text{bkg},i}^{\text{obs}},N_{\text{bkg},i}/f_{i}),\end{split}$
(2)

where $N_{\text{obs}}$ is the observed number of events,
$N_{\tau^{+}\tau^{-}}$ is the number of tau pairs, $\mathcal{B}$ is BF of
$\tau\to\gamma\mu$ and is the parameter of interest, $\epsilon$ is signal
efficiency, $N_{\text{bkg},i}$ with $i$ runs over all the background samples
are the _true_ values of background levels which are nuisance parameters,
$N_{\text{bkg},i}^{\text{obs}}$ and $f_{i}$ are the observed number of events
and scale factors for each background samples. The likelihood is then taken as
probability distribution of parameters, and the posterior distribution of BF
is obtained by integrating over nuisance parameters. Finally, the UL on BF is
determined by integrating the posterior distribution, as shown in Fig. 6. The
sensitivity of BF is defined as the UL that can be achieved assuming there are
no signals. Pseudo-experiments are generated assuming background-only, and the
median, the $\pm 1\sigma$ and $\pm 2\sigma$ quantiles on the distribution of
UL are taken as the expectation and confidence interval of the sensitivity
expected_limit (31). With a total luminosity of
$1\text{\,}\mathrm{a}\mathrm{b}^{-1}$, the sensitivity is estimated to be at
the level of ${10}^{-8}$.

Figure 6: Determination of UL of BF. The line shows the posterior probability
distribution and the arrow marks the UL at $90\text{\,}\mathrm{\char
37\relax}$ C.L.

A full systematic uncertainty evaluation which requires both experimental data
and full MC simulation is not possible at this stage, so it will be
qualitatively discussed. Referring to Eq. 2, the possible sources of
systematic uncertainties include the number of tau pairs, the event selection
efficiency, and the estimation of background. The uncertainty of number of tau
pairs comes from the determination of luminosity and cross-section of
$e^{+}e^{-}\to\tau^{+}\tau^{-}$. For efficiency, the statistical uncertainty
can be negligible with large MC samples; the uncertainty of tracking and PID
of tracks and reconstruction of neutral pions, which is evaluated with
difference between data and MC, can be studied with pure and high statistical
control samples; uncertainties related with other selection criteria can be
evaluated by control samples or varying the criteria and performing the Barlow
test Barlow_test (32); for the modeling of $\tau\to\gamma\mu$ decay, although
the LFV interaction structure in unknown, the uncertainty can be estimated by
assuming extreme cases such as pure $V-A$ and $V+A$ forms. The uncertainty of
background estimation can be evaluated using control samples of data, such as
from sidebands, and verifying that the estimated background is consistent with
that in real data. The total systematic uncertainties at STCF are expected to
be at the level of several percent or less, which only have a minor impact on
the sensitivity.

##  4 Optimization of detector performance

The performance of STCF detector is tunable in the fast simulation, and the
sensitivity of $\tau\to\gamma\mu$ under different performances are studied to
guide the design of the detector.

The detector performance properties that are crucial to this analysis are
determined based on the main background channels where photons and muons are
misidentified as signals. One of the main origins of the signal muon is
misidentification of pion, while the signal photon is misidentified from a
photon with other origins. So, both pion/muon separation capability and photon
detection resolution are relevant to this analysis. Better pion/muon
separation capability can efficiently suppress background caused by pion/muon
misidentification, and better photon detection resolution will improve the
resolution of signal region thus exclude more background. With fast
simulation, three kinds of detector responses can be studied: pion/muon
separation, photon energy resolution and photon position resolution.
Considering the feasibility of particular detector designs, a set of different
values is assumed for each of the performance properties, from conservative to
aggressive. The best detector performance is taken as benchmark, and the
dependence of the sensitivity upon each performance property is checked by
fixing the other properties and only varying the one under study. The
benchmark result is sensitivity of $2.8\text{\times}{10}^{-8}$ with
$1\text{\,}\mathrm{a}\mathrm{b}^{-1}$ luminosity under the detector
performance of $1\text{\,}\mathrm{\char 37\relax}$ in pion/muon
misidentification rate and $3\text{\,}\mathrm{m}\mathrm{m}$ and
$2\text{\,}\mathrm{\char 37\relax}$ in photon position and energy resolutions.
For each performance setting, the analysis is performed individually to reach
the best sensitivity.

### pion/muon separation

On the one hand, better pion/muon separation can suppress the background
caused by pion misidentified as signal muon. On the other hand, better
separation means tighter muon selection, which will cause lower signal
efficiency. Three levels of pion/muon separation capability is assumed with
overall pion to muon misidentification rates of $3\text{\,}\mathrm{\char
37\relax}$, $1.7\text{\,}\mathrm{\char 37\relax}$ and $1\text{\,}\mathrm{\char
37\relax}$, which corresponding to the muon identification efficiencies of
$97\text{\,}\mathrm{\char 37\relax}$, $92\text{\,}\mathrm{\char 37\relax}$ and
$85\text{\,}\mathrm{\char 37\relax}$ at a momentum of
$1\text{\,}\mathrm{G}\mathrm{e}\mathrm{V}\mathrm{/}\mathrm{\text{$c$}}$,
respectively. Table 2 summarizes the efficiency of muon identification in
signal sample and the sensitivity of $\tau\to\gamma\mu$ with respect to
pion/muon misidentification rate. The sensitivity together with its confidence
intervals are also shown in Fig. 7. The result shows that the sensitivity
improves with better pion/muon separation, but the improvement is small. This
indicates that a trade-off between the muon efficiency and the pion/muon
separation capability should be carefully considered.

Table 2: Result of optimization for pion/muon separation. The first column shows the levels of pion/muon separation capability of the detector. The second and third columns show the muon identification efficiency and sensitivity on BF under different detector performance, separately. pion/muon mis-id rate | muon PID eff. | sensitivity/${10}^{-8}$
---|---|---
$3\text{\,}\mathrm{\char 37\relax}$ | $80.6\text{\,}\mathrm{\char 37\relax}$ | $3.3$
$1.7\text{\,}\mathrm{\char 37\relax}$ | $65.4\text{\,}\mathrm{\char 37\relax}$ | $3.0$
$1\text{\,}\mathrm{\char 37\relax}$ | $50.3\text{\,}\mathrm{\char 37\relax}$ | $2.8$
Figure 7: Result for optimization of pion/muon separation capability. The
dashed line is the expected sensitivity, the green and yellow bands are the
$1\sigma$ and $2\sigma$ confidence intervals.

### position resolution for photon

Better photon position resolution will result in better signal region
resolution, thus improve efficiency and suppress background. Besides, better
photon resolution will result in higher tag accuracy and better separation of
noise from real photons. The baseline for photon position resolution is
$6\text{\,}\mathrm{m}\mathrm{m}$ and an improvement of
$30\text{\,}\mathrm{\char 37\relax}$ and $50\text{\,}\mathrm{\char 37\relax}$
is assumed for optimization. The signal resolution under different photon
position resolutions is shown in Fig. 8, and the sensitivity is summarized in
Table 3 and Fig. 9. It is shown that better photon position resolution will
result in better sensitivity, but the influence is rather small, which is
because the baseline resolution is already quite good.

Figure 8: Signal resolution under different photon position resolution performance. The left and right figure shows the invariant mass and total energy of signal photon and muon, separately. The black solid line, the red dashed line and the blue dash-dotted line shows the result under baseline, $30\text{\,}\mathrm{\char 37\relax}$ and $50\text{\,}\mathrm{\char 37\relax}$ improved photon position resolution, separately. Table 3: Result of optimization for photon position resolution. The first column shows the levels of photon position resolution performance of the detector. The second and third columns show the signal efficiency and sensitivity on BF under different detector performance. resolution | $\epsilon$ | sensitivity/${10}^{-8}$
---|---|---
baseline | $3.0\text{\,}\mathrm{\char 37\relax}$ | 3.4
$30\text{\,}\mathrm{\char 37\relax}$ improved | $3.1\text{\,}\mathrm{\char 37\relax}$ | 3.4
$50\text{\,}\mathrm{\char 37\relax}$ improved | $3.1\text{\,}\mathrm{\char 37\relax}$ | 2.8
Figure 9: Result for optimization of photon position resolution.

### energy resolution for photon

Similar to photon position resolution, better photon energy resolution will
also result in better sensitivity. The baseline for photon energy resolution
is $2.5\text{\,}\mathrm{\char 37\relax}$ at
$1\text{\,}\mathrm{G}\mathrm{e}\mathrm{V}$ and an improvement of
$10\text{\,}\mathrm{\char 37\relax}$ and $20\text{\,}\mathrm{\char 37\relax}$
is assumed for optimization. The signal resolution under different photon
energy resolutions is shown in Fig. 10, and the sensitivity is summarized in
Table 4 and Fig. 11. It is shown that better photon energy resolution will
result in better sensitivity.

Figure 10: Signal resolution under different photon energy resolution performance. The left and right figure shows the invariant mass and total energy of signal photon and muon, separately. The black solid line, the red dashed line and the blue dash-dotted line shows the result under baseline, $10\text{\,}\mathrm{\char 37\relax}$ and $20\text{\,}\mathrm{\char 37\relax}$ improved photon energy resolution, separately. Table 4: Result of optimization for photon energy resolution. The first column shows the levels of photon energy resolution performance of the detector. The second and third columns show the signal efficiency and sensitivity on BF under different detector performance. resolution | $\epsilon$ | sensitivity/${10}^{-8}$
---|---|---
baseline | $2.9\text{\,}\mathrm{\char 37\relax}$ | 6.9
$10\text{\,}\mathrm{\char 37\relax}$ improved | $3.0\text{\,}\mathrm{\char 37\relax}$ | 5.1
$20\text{\,}\mathrm{\char 37\relax}$ improved | $3.1\text{\,}\mathrm{\char 37\relax}$ | 2.8
Figure 11: Result for optimization of photon energy resolution.

##  5 Summary and discussion

The sensitivity on the cLFV process $\tau\to\gamma\mu$ at the Super
$\tau$-Charm Facility is studied based on
$1\text{\,}\mathrm{a}\mathrm{b}^{-1}$ MC samples, which corresponds to the
one-year integrated luminosity of STCF. The sensitivity is expected to be at
the level of ${10}^{-8}$. The optimization of detector performance is also
studied in order to get the best sensitivity, and the result shows that the
improvement of each of the three performance properties concerned in this
analysis, namely pion/muon separation capability, position and energy
resolution of photon, can all result in better sensitivity, though the
importance of them is different. With ideal detector performance of
$1\text{\,}\mathrm{\char 37\relax}$ in pion/muon misidentification rate and
$3\text{\,}\mathrm{m}\mathrm{m}$ and $2\text{\,}\mathrm{\char 37\relax}$ in
photon position and energy resolutions, the best sensitivity of
$2.8\text{\times}{10}^{-8}$ at $90\text{\,}\mathrm{\char 37\relax}$ confidence
level is achieved. Since background-free can not be achieved, the sensitivity
is expected to scale with the square root of the luminosity, and could reach
$8.8\text{\times}{10}^{-9}$ with ten-year of data taking, which is about one
order of magnitude improvement upon the current best result.

###### Acknowledgements.

We thank the Hefei Comprehensive National Science Center for their strong
support. This work is supported by the Joint Large-Scale Scientific Facility
Funds of the National Natural Science Foundation of China and Chinese Academy
of Sciences (CAS) under Contract No. U1832207, the National Key R&D Program of
China under Contracts No. 2020YFA0406400, and the international partnership
program of the CAS Grant No. 211134KYSB20200057.

## References

  * (1) Benjamin W. Lee and Robert E. Shrock  “Natural suppression of symmetry violation in gauge theories: Muon- and electron-lepton-number nonconservation”  In _Phys. Rev. D_ 16 American Physical Society, 1977, pp. 1444–1473  DOI: 10.1103/PhysRevD.16.1444
  * (2) Ziro Maki, Masami Nakagawa and Shoichi Sakata  “Remarks on the Unified Model of Elementary Particles”  In _Prog. Theor. Phys._ 28.5, 1962, pp. 870–880  DOI: 10.1143/PTP.28.870
  * (3) Andrea Brignole and Anna Rossi  “Anatomy and phenomenology of mu-tau lepton flavor violation in the MSSM”  In _Nucl. Phys. B_ 701.1, 2004, pp. 3–53  DOI: 10.1016/j.nuclphysb.2004.08.037
  * (4) Savas Dimopoulos and Howard Georgi  “Softly broken supersymmetry and SU(5)”  In _Nucl. Phys. B_ 193.1, 1981, pp. 150–162  DOI: 10.1016/0550-3213(81)90522-8
  * (5) L. Calibbi, A. Faccia, A. Masiero and S.. Vempati  “Lepton flavor violation from supersymmetric grand unified theories: Where do we stand for MEG, PRISM/PRIME, and a super flavor factory”  In _Phys. Rev. D_ 74 American Physical Society, 2006, pp. 116002  DOI: 10.1103/PhysRevD.74.116002
  * (6) John Ellis, Junji Hisano, Martti Raidal and Yasuhiro Shimizu  “New parametrization of the seesaw mechanism and applications in supersymmetric models”  In _Phys. Rev. D_ 66, 2002, pp. 115013  DOI: 10.1103/PhysRevD.66.115013
  * (7) G. Cveti č, C. Dib, C.. Kim and J.. Kim  “Lepton flavor violation in tau decays”  In _Phys. Rev. D_ 66 American Physical Society, 2002, pp. 034008  DOI: 10.1103/PhysRevD.66.034008
  * (8) Athanasios Dedes, John Ellis and Martti Raidal  “Higgs-mediated $B_{s,d}^{0}\to\mu\tau,e\tau$ and $\tau\to 3\mu,e\mu\mu$ decays in supersymmetric seesaw models”  In _Phys. Lett. B_ 549.1, 2002, pp. 159–169  DOI: https://doi.org/10.1016/S0370-2693(02)02900-3
  * (9) Andrea Brignole and Anna Rossi  “Lepton flavour violating decays of supersymmetric Higgs bosons”  In _Phys. Lett. B_ 566.3, 2003, pp. 217–225  DOI: https://doi.org/10.1016/S0370-2693(03)00837-2
  * (10) Antonio Masiero, Sudhir K. Vempati and Oscar Vives  “See-saw and lepton flavour violation in SUSY SO(10)”  In _Nucl. Phys. B_ 649.1, 2003, pp. 189–204  DOI: https://doi.org/10.1016/S0550-3213(02)01031-3
  * (11) Takeshi Fukuyama, Tatsuru Kikuchi and Nobuchika Okada  “Lepton flavor violating processes and muon $g-2$ in the minimal supersymmetric SO(10) model”  In _Phys. Rev. D_ 68 American Physical Society, 2003, pp. 033012  DOI: 10.1103/PhysRevD.68.033012
  * (12) Chongxing Yue, Yanming Zhang and Lanjun Liu  “Non-universal gauge bosons $Z^{\prime}$ and lepton flavor-violating tau decays”  In _Phys. Lett. B_ 547.3, 2002, pp. 252–256  DOI: https://doi.org/10.1016/S0370-2693(02)02781-8
  * (13) B. Aubert  “Searches for Lepton Flavor Violation in the Decays ${\tau}^{\pm{}}\rightarrow{e}^{\pm{}}\gamma$ and ${\tau}^{\pm{}}\rightarrow{\mu}^{\pm{}}\gamma$”  In _Phys. Rev. Lett._ 104 American Physical Society, 2010, pp. 021802  DOI: 10.1103/PhysRevLett.104.021802
  * (14) K. Uno  “Search for lepton-flavor-violating tau-lepton decays to $\ell\gamma$ at Belle”  In _J. High Energ. Phys._ 2021.19 Springer ScienceBusiness Media LLC, 2021  DOI: 10.1007/jhep10(2021)019
  * (15) Swagato Banerjee  “Snowmass 2021 White Paper: Charged lepton flavor violation in the tau sector”, 2022  arXiv:2203.14919 [hep-ph]
  * (16) M. Achasov  “STCF Conceptual Design Report: Volume 1 – Physics & Detector”, 2023  arXiv:2303.15790 [hep-ex]
  * (17) Yung Su Tsai  “Production of polarized $\tau$ pairs and tests of $CP$ violation using polarized $e^{\pm}$ colliders near threshold”  In _Phys. Rev. D_ 51.7 American Physical Society, 1995, pp. 3172–3181  DOI: 10.1103/PhysRevD.51.3172
  * (18) S. Jadach, B… Ward and Z. Was  “The Precision Monte Carlo event generator $\mathcal{KK}$ for two-fermion final states in $e^{+}e^{-}$ collisions”  In _Comput. Phys. Commun._ 130.3, 2000, pp. 260–325  DOI: 10.1016/S0010-4655(00)00048-5
  * (19) B… Ward, S. Jadach and Z. Was  “Precision calculation for $e^{+}e^{-}\to 2f$: The $\mathcal{KK}$ MC project”  In _Nucl. Phys. B Proc. Suppl._ 116, 2003, pp. 73–77  DOI: 10.1016/S0920-5632(03)80147-0
  * (20) Xiao Dong. Shi, Xiao Rong. Zhou, Xiao Shuai. Qin and Hai Ping Peng  “A fast simulation package for STCF detector”  In _J. Instrum._ 16.03, 2021, pp. P03029  DOI: 10.1088/1748-0221/16/03/P03029
  * (21) S. Agostinelli  “Geant4–a simulation toolkit”  In _Nucl. Instrum. Meth. A_ 506.3, 2003, pp. 250–303  DOI: 10.1016/S0168-9002(03)01368-8
  * (22) M. Ablikim  “Design and construction of the BESIII detector”  In _Nucl. Instrum. Meth. A_ 614.3 Elsevier BV, 2010, pp. 345–399  DOI: 10.1016/j.nima.2009.12.050
  * (23) Wei-Dong Li, Ya-Jun Mao and Yi-Fang Wang  “Chapter 2 The BES-III Detector and Offline Software”  In _Int. J. Mod. Phys. A_ 24.supp01, 2009, pp. 9–21  DOI: 10.1142/S0217751X09046424
  * (24) Giovanni Balossini et al.  “Matching perturbative and parton shower corrections to Bhabha process at flavour factories”  In _Nucl. Phys. B_ 758.1, 2006, pp. 227–253  DOI: 10.1016/j.nuclphysb.2006.09.022
  * (25) G. Balossini et al.  “Photon pair production at flavour factories with per mille accuracy”  In _Phys. Lett. B_ 663.3, 2008, pp. 209–213  DOI: 10.1016/j.physletb.2008.04.007
  * (26) S. Jadach, Z. Was, R. Decker and Johann H. Kühn  “The $\tau$ decay library TAUOLA, Version 2.4”  In _Comput. Phys. Commun._ 76.3, 1993, pp. 361–380  DOI: 10.1016/0010-4655(93)90061-G
  * (27) Bo Andersson and Hai Ming Hu  “Few body states in Lund string fragmentation model”  In _arXiv_ , 1999  arXiv:hep-ph/9910285
  * (28) R.. Workman  “Review of Particle Physics”  In _Prog. Theor. Exp. Phys._ 2022, 2022, pp. 083C01  DOI: 10.1093/ptep/ptac097
  * (29) Giovanni Punzi  “Sensitivity of searches for new signals and its optimization”  In _eConf_ C030908, 2003, pp. MODT002  arXiv:physics/0308063
  * (30) Wolfgang A. Rolke, Angel M. López and Jan Conrad  “Limits and confidence intervals in the presence of nuisance parameters”  In _Nucl. Instrum. Meth. A_ 551.2 Elsevier BV, 2005, pp. 493–503  DOI: 10.1016/j.nima.2005.05.068
  * (31) ATLAS and CMS Collaborations  “Procedure for the LHC Higgs boson search combination in Summer 2011”, 2011  URL: http://cds.cern.ch/record/1379837
  * (32) Roger Barlow  “Systematic errors: Facts and fictions”  In _Conference on Advanced Statistical Techniques in Particle Physics_ , 2002, pp. 134–144  arXiv:hep-ex/0207026