Prospects for studies of ${{D}^{*}}^{0}\rightarrow{\mu^{+}\mu^{-}}$ and
${{B}^{*0}_{({s})}}\rightarrow{\mu^{+}\mu^{-}}$ decays

F. Abudinén1, T. Blake1, U. Egede2, T. Gershon1

1 Department of Physics, University of Warwick, Coventry, United Kingdom
2 School of Physics and Astronomy, Monash University, Melbourne, Australia

Weak decays of the vector ${{D}^{*}}^{0}$ and ${{B}^{*0}_{({s})}}$ mesons to
the ${\mu^{+}\mu^{-}}$ final state provide novel potential to test the
Standard Model of particle physics. Such processes have extremely small
branching fractions as the vector mesons are able to decay through
electromagnetic and (for the ${{D}^{*}}^{0}$ meson) strong interactions.
Nonetheless, the production of copious quantities of these particles in LHC
collisions, and the ability to exploit experimental techniques that can
suppress background to low levels, provides good potential to reach
interesting sensitivity. The possibility to reconstruct these processes as
part of the decay chain of ${{B}^{-}}$ or ${B}_{c}^{+}$ mesons appears
particularly attractive due to the clean experimental signature of the
displaced vertex. Indeed, published LHCb data on
${{{B}^{-}}}\rightarrow{{\pi}^{-}}{\mu^{+}\mu^{-}}$ decays already implies a
stringent limit on the branching fraction of
${{D}^{*0}}\rightarrow{\mu^{+}\mu^{-}}$. Estimates are made on the achievable
sensitivity to ${{D}^{*0}}\rightarrow{\mu^{+}\mu^{-}}$ and
${{B}^{*0}_{({s})}}\rightarrow{\mu^{+}\mu^{-}}$ decays with the LHCb
experiment.

## Introduction

Weak decays of heavy-flavoured hadrons provide a range of methods to test the
Standard Model (SM) of particle physics. In particular, many such transitions
are suppressed by particular features of the SM such as the GIM mechanism [1]
and the CKM quark mixing matrix [2, 3]. As a consequence, the SM predicts a
distinctive pattern of decay rates to various different final states, which
may be modified by contributions from physics beyond the SM. Experimental and
theoretical investigations in this area are therefore a priority in the field.

Until now, experimental studies of weak decays have been almost completely
limited to the ground-state hadrons; considering neutral heavy-flavoured
mesons, these are the pseudoscalar ${{D}^{0}}$, ${{B}^{0}}$ and
${{B}^{0}_{s}}$ states. The leptonic decays, to the $\ell^{+}\ell^{-}$ final
state where $\ell=e,\mu$ and (for ${B}^{0}_{({s})}$ decays) $\tau$, have
branching fractions that are suppressed by the chiral structure of the SM weak
interaction, and that can be predicted with small theoretical uncertainties
[4]. These features together make them highly sensitive to potential
contributions from physics beyond the SM. Intense activity on the
${{B}^{0}_{({s})}}\rightarrow{\mu^{+}\mu^{-}}$ channels has resulted in the
observation of the ${{B}^{0}_{s}}\rightarrow{\mu^{+}\mu^{-}}$ decay by the
LHCb, CMS and ATLAS experiments, and a combined limit on the
${{B}^{0}}\rightarrow{\mu^{+}\mu^{-}}$ branching fraction that approaches its
SM value [5, 6, 7, 8, 9, 10]. The experimental limits on ${\cal
B}\left({{D}^{0}}\rightarrow{e^{+}e^{-}}\right)$ [11], ${\cal
B}\left({{D}^{0}}\rightarrow{\mu^{+}\mu^{-}}\right)$ [12], ${\cal
B}\left({{B}^{0}_{({s})}}\rightarrow{e^{+}e^{-}}\right)$ [13] and ${\cal
B}\left({{B}^{0}_{({s})}}\rightarrow{\tau^{+}\tau^{-}}\right)$ [14] are still
several orders of magnitude above their SM predictions.

It is also possible to consider weak decays of the excited counterparts of the
pseudoscalar mesons, the vector ${{D}^{*0}}$, ${{B}^{*0}}$ and
${{B}^{*0}_{s}}$ resonances. In contrast to the situation for pseudoscalar
mesons, the leptonic vector meson decays have no chiral suppression.
Consequently the decay widths for each of the $\ell^{+}\ell^{-}$ final states
are expected to be the same, in the SM, up to effects related to the lepton
mass (e.g., the available phase space), and will be larger compared to those
for the pseudoscalar decays. However, the vector mesons can also decay via
electromagnetic and (for the ${{D}^{*0}}$ meson) strong transitions, which
have widths many orders of magnitude larger than those for the weak decays.
Therefore, the branching fractions of the weak decays are suppressed to what
would usually be considered unobservably small levels. As an illustration, one
can compare the width of the ${{D}^{*+}}$ vector state,
$\Gamma_{{{D}^{*+}}}=83\mathrm{\,ke\kern-1.00006ptV}$ [15], with that of its
${{D}^{+}}$ pseudoscalar counterpart,
$\Gamma_{{{D}^{+}}}=\hbar/\tau_{{{D}^{+}}}\approx\hbar/(1.0{\rm\,ps})=0.7~{}{\rm
meV}$ [16], a difference of over 8 orders of magnitude.111 A similar exercise
for the ${{D}^{(*)0}}$, ${{B}^{(*)0}}$ and ${{B}^{(*)0}_{s}}$ states is not
possible because the widths of the vector resonances have not been measured
and have significant theoretical uncertainty.  This is in the ballpark that
one would naively expect, given that the weak decays are suppressed by the
Fermi constant, but have more phase space available compared to the
electromagnetic and strong transitions in which flavour is conserved. Given
that weak decays lead to a plethora of different final states, the branching
fractions for even the most favoured are unlikely to be above $10^{-9}$,
unless large enhancement factors due to physics beyond the SM are at play.

Hints of physics beyond the SM in $B$ meson decays have, nevertheless,
prompted theoretical activity on weak decays of excited heavy-flavoured
states. Collectively referred to as the flavour anomalies, these hints include
tensions between SM predictions and experimental measurements in branching
fractions and angular observables in decays mediated by $b\rightarrow
s\ell^{+}\ell^{-}$ transitions, including observables that are sensitive to
violations of lepton universality (see Ref. [17] for a recent review). The
leptonic ${{D}^{*0}}$, ${{B}^{*0}}$ and ${{B}^{*0}_{s}}$ decays are of most
interest, since these are theoretically cleanest and, for the ${B}^{*0}_{s}$
case, are sensitive to the same operators which could be responsible for the
flavour anomalies.222 The inclusion of charge conjugate processes is implied
throughout this paper. These decays have been considered as a potential probe
of physics beyond the SM in Refs [18, 19], with further investigations of the
impact of particular extensions of the SM considered in Refs. [20, 21, 22, 23,
24].

There is an extra motivation to search for leptonic ${{D}^{*0}}$ decays. The
relevant operators for the ${{B}^{*0}}$ and ${{B}^{*0}_{s}}$ decays are
already constrained from measurements of pseudoscalar
$B_{(s)}\rightarrow\ell^{+}\ell^{-}$ and $h\ell^{+}\ell^{-}$ transitions
(where $h$ is a hadron), and can be used to set limits on ${\cal
B}\left({{B}^{*0}_{({s})}}\\!\rightarrow\ell^{+}\ell^{-}\right)$ that are
below the current experimentally achievable sensitivity [18]. The
interpretation of results from ${{D}^{0}}\rightarrow\ell^{+}\ell^{-}$ and
$D_{(s)}\rightarrow h\ell^{+}\ell^{-}$ decays, on the other hand, is
challenging due to long-distance effects [25, 26, 27, 28]. Correspondingly,
constraints on the relevant operators are weaker, and the possibility of a
large enhancement to ${\cal
B}\left({{D}^{*}}\\!\rightarrow\ell^{+}\ell^{-}\right)$ from physics beyond
the Standard Model cannot be ruled out. Thus, if sufficient experimental
precision can be obtained, searches for
${{D}^{*0}}\rightarrow\ell^{+}\ell^{-}$ decays can provide an important
complementary approach to probe the operators involved, and may reveal
exciting results.

Since there is no suppression of the coupling of the heavy flavoured vector
resonances to dielectron, compared to dimuon, states, it may be attractive
experimentally to search for these interactions through production in
${e^{+}e^{-}}$ collisions, rather than in decay processes [19]. A search for
the $e^{+}e^{-}\rightarrow D^{*}(2007)^{0}$ process has been carried out by
the CMD-3 collaboration, resulting in an upper limit being set, ${\cal
B}\left({{D}^{*0}}\rightarrow{e^{+}e^{-}}\right)<1.7\times 10^{-6}$ at 90%
confidence level [29]. While it is likely that this result can be
significantly improved through analyses of larger data samples with better
background suppression, the limit is not at such a stringent level to suggest
that other approaches to study these processes would be futile.

In particular, the copious production of heavy flavoured hadrons at the LHC
makes it worthwhile to consider what sensitivity might be achievable with
current and future data samples. As the dimuon signature allows for effective
background suppression in hadron collider experiments, the ${{D}^{*0}}$,
${{B}^{*0}}$ and ${{B}^{*0}_{s}}$ decays to the $\mu^{+}\mu^{-}$ final state
are the most amenable. There are several possible avenues to investigate these
processes using LHC data. In particular, both “prompt” and “displaced”
production can be considered, where in the former the heavy flavoured state is
produced at the primary vertex of the proton-proton collision while in the
latter the signal hadron originates from the decay of another weakly decaying
particle some distance from the primary vertex. Specifically, ${{D}^{*0}}$
mesons are produced at high rates in $b$ hadron decays, and
${{B}^{*0}_{({s})}}$ mesons can be produced in ${{B}_{c}^{+}}$ meson, and
potentially in other doubly-heavy hadron, decays. While prompt production has
the highest rate, the large numbers of tracks originating from the proton-
proton collision vertices lead to large backgrounds that limit the sensitivity
of any rare decay search. In displaced production it is unlikely to find
random tracks appearing to come from the vertex position, so long as the
vertex resolution is sufficient. If in addition the signal is part of an
exclusive decay process, additional constraints can be applied to suppress
further the background. Displaced production therefore appears to provide the
most promising approach.

In the main part of this paper, the potential sensitivity of the LHCb
experiment to ${{D}^{*0}}$, ${{B}^{*0}}$ and ${{B}^{*0}_{s}}$ decays to the
$\mu^{+}\mu^{-}$ final state, using displaced production in exclusive final
states is investigated. Possibilities with prompt production, and with
inclusive and semi-inclusive search approaches with displaced production, are
considered for completeness in Appendices A and B, respectively. In principle
the ATLAS and CMS experiments could also make competitive measurements, but
until now they have fewer relevant measurements making extrapolations
difficult. Moreover, as their vertexing and particle identification capability
is not as good as that of LHCb, it is expected that they will suffer from
larger backgrounds.

##
${{{B}^{-}}}\rightarrow{{D}^{*0}}{{\pi}^{-}}\rightarrow{\mu^{+}\mu^{-}}{{\pi}^{-}}$
decays

The decay chain
${{{B}^{-}}}\rightarrow{{D}^{*0}}{{\pi}^{-}}\rightarrow{\mu^{+}\mu^{-}}{{\pi}^{-}}$
not only provides an excellent illustrative example, it also allows a limit on
the branching fraction for ${{D}^{*0}}\rightarrow{\mu^{+}\mu^{-}}$ to be set
from published data. In particular, LHCb has studied the
${{{B}^{-}}}\rightarrow{\mu^{+}\mu^{-}}{{\pi}^{-}}$ decay using a data sample
corresponding to an integrated luminosity of $3.0\mbox{\,fb}^{-1}$, collected
at centre-of-mass energies of 7 and $8\mathrm{\,Te\kern-1.00006ptV}$ [30]. The
measured differential branching fraction ${\rm d}{\cal B}/{\rm d}{q^{2}}$,
where ${q^{2}}$ is the square of the dimuon invariant mass, is shown in Fig.
1. An upper limit on ${\cal
B}\left({{D}^{*0}}\rightarrow{\mu^{+}\mu^{-}}\right)$ can be set by assuming
conservatively that the
${{{B}^{-}}}\rightarrow{{D}^{*0}}{{\pi}^{-}}\rightarrow{\mu^{+}\mu^{-}}{{\pi}^{-}}$
decay contributes not more than half of the
${{{B}^{-}}}\rightarrow{\mu^{+}\mu^{-}}{{\pi}^{-}}$ signal in the two bins
either side of ${q^{2}}=4{\mathrm{\,Ge\kern-1.00006ptV^{2}\\!/}c^{4}}$, i.e. a
branching fraction of ${\cal
B}\left({{{B}^{-}}}\rightarrow\left[{\mu^{+}\mu^{-}}\right]_{{{D}^{*0}}}{{\pi}^{-}}\right){~{}\raise
1.49994pt\hbox{$<$}\kern-8.50006pt\lower 3.50006pt\hbox{$\sim$}~{}}1.5\times
10^{-9}$. Then, using the world average value of ${\cal
B}\left({{{B}^{-}}}\rightarrow{{D}^{*0}}{{\pi}^{-}}\right)$ [31], one obtains

${\cal B}\left({{D}^{*0}}\rightarrow{\mu^{+}\mu^{-}}\right)=\frac{{\cal
B}\left({{{B}^{-}}}\rightarrow\left[{\mu^{+}\mu^{-}}\right]_{{{D}^{*0}}}{{\pi}^{-}}\right)}{{\cal
B}\left({{{B}^{-}}}\rightarrow{{D}^{*0}}{{\pi}^{-}}\right)}{~{}\raise
1.49994pt\hbox{$<$}\kern-8.50006pt\lower 3.50006pt\hbox{$\sim$}~{}}3\times
10^{-7}\,.$

This limit, while already more stringent than the result on
${{D}^{*0}}\rightarrow{e^{+}e^{-}}$ from the CMD-3 collaboration [29], could
most likely be improved by at least an order of magnitude by a dedicated LHCb
analysis. In particular, the experimental mass resolution, which we expect to
be around $5{\mathrm{\,Me\kern-1.00006ptV\\!/}c^{2}}$, is much better than can
be obtained from the coarse ${q^{2}}$ binning of Fig. 1. Moreover, LHCb has
already collected a significantly larger data sample than was analysed in Ref.
[30], and a somewhat higher selection efficiency could be anticipated in a
dedicated analysis. It is therefore of interest to ask what sensitivity might
ultimately be achieved by LHCb in such a search.

Figure 1: Differential branching fraction of the
${{{B}^{-}}}\rightarrow{\mu^{+}\mu^{-}}{{\pi}^{-}}$ decay as a function of
${q^{2}}=m({\mu^{+}\mu^{-}})^{2}$, taken from Ref. [30]. The hashed regions
correspond to theoretical predictions from Refs. [32, 33, 34].

In LHC experiments, it is advantageous to measure branching fractions relative
to those of appropriate normalisation channels. It is anticipated that the
${{{B}^{-}}}\\!\rightarrow{{J\mskip-3.0mu/\mskip-2.0mu\psi\mskip
2.0mu}}{{K}^{-}}$ decay will be used for this purpose and hence the signal
yield is converted to a measurement of the branching fraction as

$\displaystyle{\cal B}\left({{D}^{*0}}\\!\rightarrow{\mu^{+}\mu^{-}}\right)$
$\displaystyle=$
$\displaystyle\frac{N_{{{{B}^{-}}}\\!\rightarrow{{D}^{*0}}{{\pi}^{-}}}}{N_{{{{B}^{-}}}\\!\rightarrow{{J\mskip-3.0mu/\mskip-2.0mu\psi\mskip
2.0mu}}{{K}^{-}}}}\,\frac{\epsilon_{{{{B}^{-}}}\\!\rightarrow{{J\mskip-3.0mu/\mskip-2.0mu\psi\mskip
2.0mu}}{{K}^{-}}}}{\epsilon_{{{{B}^{-}}}\\!\rightarrow{{D}^{*0}}{{\pi}^{-}}}}\,\frac{{\cal
B}\left({{{B}^{-}}}\\!\rightarrow{{J\mskip-3.0mu/\mskip-2.0mu\psi\mskip
2.0mu}}{{K}^{-}}\right)}{{\cal
B}\left({{{B}^{-}}}\\!\rightarrow{{D}^{*0}}{{\pi}^{-}}\right)}\,{\cal
B}\left({{J\mskip-3.0mu/\mskip-2.0mu\psi\mskip
2.0mu}}\\!\rightarrow{\mu^{+}\mu^{-}}\right)\,,$ (1) $\displaystyle=$
$\displaystyle\alpha_{{{D}^{*0}}\\!\rightarrow{\mu^{+}\mu^{-}}}N_{{{{B}^{-}}}\\!\rightarrow{{D}^{*0}}{{\pi}^{-}}}\,,$
(2)

where $\alpha_{{{D}^{*0}}\\!\rightarrow{\mu^{+}\mu^{-}}}$ is the single-event
sensitivity, which corresponds to the branching fraction at which one signal
event is expected in the data sample. In Eq. (1), $N$ and $\epsilon$ represent
the yield and the efficiency for the decay indicated in the subscript, where
the reconstruction of the ${D}^{*0}$ or ${J\mskip-3.0mu/\mskip-2.0mu\psi\mskip
2.0mu}$ vector meson in the $\mu^{+}\mu^{-}$ final state is implied.

The single-event sensitivity is estimated from the yield of
${{{B}^{-}}}\\!\rightarrow{{J\mskip-3.0mu/\mskip-2.0mu\psi\mskip
2.0mu}}{{K}^{-}}$ decays, scaled from measurements with existing data [30]
appropriately to each integrated luminosity value, and the branching fractions
${\cal B}({{{B}^{-}}}\\!\rightarrow{{J\mskip-3.0mu/\mskip-2.0mu\psi\mskip
2.0mu}}{{K}^{-}})$, ${\cal B}({{J\mskip-3.0mu/\mskip-2.0mu\psi\mskip
2.0mu}}\\!\rightarrow{\mu^{+}\mu^{-}})$ and ${\cal
B}({{{B}^{-}}}\\!\rightarrow{{D}^{*0}}{{\pi}^{-}})$ [31]. It is assumed that
$\epsilon_{{{{B}^{-}}}\\!\rightarrow{{J\mskip-3.0mu/\mskip-2.0mu\psi\mskip
2.0mu}}{{K}^{-}}}\approx\epsilon_{{{{B}^{-}}}\\!\rightarrow{{D}^{*0}}{{\pi}^{-}}}$,
since the final state is the same (apart from $K\leftrightarrow\pi$) and the
efficiency varies slowly with ${q^{2}}$. The single-event sensitivity is shown
in Fig. 2 as a function of the data set size.

The achievable precision depends not only on the single-event sensitivity but
also on the uncertainty on the signal yield, which is often limited by the
background level. To investigate how the achievable limit may scale with
integrated luminosity, pseudoexperiments are generated under a background-only
hypothesis. Three background components are considered: combinatorial
background from random combinations of tracks from two or more decays;
background from nonresonant
${{{B}^{-}}}\\!\rightarrow{\mu^{+}\mu^{-}}{{\pi}^{-}}$ decays; and background
from ${{{B}^{-}}}\\!\rightarrow{{K}^{-}}{\mu^{+}\mu^{-}}$ decays, where the
${K}^{-}$ meson is mistakenly identified as a ${\pi}^{-}$. The combinatorial
background is assumed to be uniformly distributed in the dimuon mass,
$m({\mu^{+}\mu^{-}})$. The backgrounds from
${{{B}^{-}}}\\!\rightarrow{{K}^{-}}{\mu^{+}\mu^{-}}$ and
${{{B}^{-}}}\\!\rightarrow{\mu^{+}\mu^{-}}{{\pi}^{-}}$ decays are assumed to
be uniform in ${q^{2}}=m^{2}({\mu^{+}\mu^{-}})$, consistent with both the
expected [35] and the observed [36] shape of the differential
${{{B}^{-}}}\\!\rightarrow{{K}^{-}}{\mu^{+}\mu^{-}}$ decay rate in the $q^{2}$
range of interest. The backgrounds are generated over the interval
$2<{q^{2}}<6\mathrm{\,Ge\kern-1.00006ptV}^{2}$, which covers the two bins of
the LHCb ${{{B}^{-}}}\\!\rightarrow{\mu^{+}\mu^{-}}{{\pi}^{-}}$ analysis
closest to the ${D}^{*0}$ mass. The level of each of the three backgrounds is
taken from Ref. [30] and is scaled to the considered integrated luminosity.
Proton-proton collisions during LHC Run 2 (2015–18) were at
$13\mathrm{\,Te\kern-1.00006ptV}$ centre-of-mass energy and those in future
run periods are expected to be at similar or slightly higher energies. When
extrapolating to future data sample sizes it is assumed that the
${b}{\overline{b}}$ production cross-section scales linearly with centre-of-
mass energy [37].

Limits are set for each pseudoexperiment using the method described in Ref.
[38] (as implemented in the TRolke class in Root), taking the uncertainty on
the single-event sensitivity into account. Two mass regions are defined, a
signal region $\pm 10{\mathrm{\,Me\kern-1.00006ptV\\!/}c^{2}}$ around the
known ${D}^{*0}$ mass, $m_{{{D}^{*0}}}$, comprising a mixture of signal and
background candidates and two sideband regions
$-35<m({\mu^{+}\mu^{-}})-m_{{{D}^{*0}}}<-15{\mathrm{\,Me\kern-1.00006ptV\\!/}c^{2}}$
and
$15<m({\mu^{+}\mu^{-}})-m_{{{D}^{*0}}}<35{\mathrm{\,Me\kern-1.00006ptV\\!/}c^{2}}$,
comprising only background candidates. The sideband regions are used to
estimate the background in the signal region. The width of the signal region
is taken as $\pm 2$ times the expected $m({\mu^{+}}{\mu^{-}})$ resolution of
$\sim 5{\mathrm{\,Me\kern-1.00006ptV\\!/}c^{2}}$ [39], which is minimised by
applying a kinematic fit to the
${{{B}^{-}}}\\!\rightarrow{\mu^{+}\mu^{-}}{{\pi}^{-}}$ process in which the
${{B}^{-}}$ mass is constrained to its known value.

The results of this pseudoexperiment-based study are shown in Fig. 3. An upper
limit at the level of $10^{-8}$ appears to be possible with the current LHCb
data set. This can be further improved to ${\cal O}(10^{-9})$ with the total
sample of $300\mbox{\,fb}^{-1}$ anticipated with future LHCb upgrades [40,
41]. The rate of reduction of the expected limit as the sample size increases
slows markedly at around $20\mbox{\,fb}^{-1}$ as background start to become
limiting. While combinatorial background and misidentified
${{{B}^{-}}}\\!\rightarrow{{K}^{-}}{\mu^{+}\mu^{-}}$ decays can be further
reduced with tighter selection requirements, the contribution from nonresonant
${{{B}^{-}}}\\!\rightarrow{\mu^{+}\mu^{-}}{{\pi}^{-}}$ decays is irreducible
in this approach.

Figure 2:  Expected single-event sensitivity for ${D}^{*0}$ produced in
${{{B}^{-}}}\\!\rightarrow{{D}^{*0}}{{\pi}^{-}}$ decays as a function of the
integrated luminosity of the LHCb data set.  Figure 3:  Expected upper limit
on ${\cal B}({{D}^{*0}}\\!\rightarrow{\mu^{+}\mu^{-}})$ obtained using
reconstructed ${{{B}^{-}}}\\!\rightarrow{\mu^{+}\mu^{-}}{{\pi}^{-}}$ decays as
a function of the integrated luminosity of the LHCb data set. The curve
represents the median value from an ensemble of pseudoexperiments and the
shaded regions the one and two sigma intervals.

The studies resulting in Fig. 3 neglect interference effects between the
${{{B}^{-}}}\\!\rightarrow\left({{D}^{*0}}\\!\rightarrow{\mu^{+}\mu^{-}}\right){{\pi}^{-}}$
signal and nonresonant ${{{B}^{-}}}\\!\rightarrow{{\pi}^{-}}{\mu^{+}\mu^{-}}$
decays, which is reasonable given the narrow width of the ${D}^{*0}$ meson.
Nonetheless, since the ${D}^{*0}$ quantum numbers are the same as part of the
nonresonant contribution, a net interference effect is expected, with size
depending on the relative phase between the two interfering amplitudes. Even
though the ${D}^{*0}$ width is below the experimental resolution, this effect
could in principle be used, once the sample size is sufficiently large, to
obtain a better limit than indicated in Fig. 3. Indeed, the LHCb collaboration
has already demonstrated the possibility to measure a similar interference
effect between
${{{B}^{-}}}\\!\rightarrow\left({{J\mskip-3.0mu/\mskip-2.0mu\psi\mskip
2.0mu}}\\!\rightarrow{\mu^{+}\mu^{-}}\right){{K}^{-}}$ and nonresonant
${{{B}^{-}}}\\!\rightarrow{{K}^{-}}{\mu^{+}\mu^{-}}$ decays [39]. The results
of this analysis allow a limit on the
${{{B}^{-}}}\\!\rightarrow{{D}^{*0}}{{K}^{-}}$ contribution to
${{{B}^{-}}}\\!\rightarrow{{K}^{-}}{\mu^{+}\mu^{-}}$ decays, and hence a limit
on ${\cal B}\left({{D}^{*0}}\\!\rightarrow{\mu^{+}\mu^{-}}\right)$ can be
derived. Assuming the yield of
${{{B}^{-}}}\\!\rightarrow\left({{D}^{*0}}\\!\rightarrow{\mu^{+}\mu^{-}}\right){{K}^{-}}$
decays is not more than $2\times\sqrt{N}$, where $N\sim 50$ is the yield in
the $m({\mu^{+}\mu^{-}})$ bin around $m_{{{D}^{*0}}}$, taken from Fig. 3 of
Ref. [39], and normalising to the
${{{B}^{-}}}\\!\rightarrow\left({{J\mskip-3.0mu/\mskip-2.0mu\psi\mskip
2.0mu}}\\!\rightarrow{\mu^{+}\mu^{-}}\right){{K}^{-}}$ contribution which has
a yield of $\sim 900\,000$ (around 90% of the total
${{{B}^{-}}}\\!\rightarrow{{K}^{-}}{\mu^{+}\mu^{-}}$ yield), and with an
efficiency ratio of 0.85 (from Fig. 2 of Ref. [39]), then with known branching
fractions [31, 42] one obtains

$\displaystyle{\cal B}\left({{D}^{*0}}\\!\rightarrow{\mu^{+}\mu^{-}}\right)$
$\displaystyle=$
$\displaystyle\frac{N_{{{{B}^{-}}}\\!\rightarrow{{D}^{*0}}{{K}^{-}}}}{N_{{{{B}^{-}}}\\!\rightarrow{{J\mskip-3.0mu/\mskip-2.0mu\psi\mskip
2.0mu}}{{K}^{-}}}}\,\frac{\epsilon_{{{{B}^{-}}}\\!\rightarrow{{J\mskip-3.0mu/\mskip-2.0mu\psi\mskip
2.0mu}}{{K}^{-}}}}{\epsilon_{{{{B}^{-}}}\\!\rightarrow{{D}^{*0}}{{K}^{-}}}}\,\frac{{\cal
B}\left({{{B}^{-}}}\\!\rightarrow{{J\mskip-3.0mu/\mskip-2.0mu\psi\mskip
2.0mu}}{{K}^{-}}\right)}{{\cal
B}\left({{{B}^{-}}}\\!\rightarrow{{D}^{*0}}{{K}^{-}}\right)}\,{\cal
B}\left({{J\mskip-3.0mu/\mskip-2.0mu\psi\mskip
2.0mu}}\\!\rightarrow{\mu^{+}\mu^{-}}\right)\,,$ $<$$\sim$ $\displaystyle
3\times 10^{-6}\,.$

As expected, this mode is not as sensitive as
${{{B}^{-}}}\\!\rightarrow\left({{D}^{*0}}\\!\rightarrow{\mu^{+}\mu^{-}}\right){{\pi}^{-}}$
due to the smaller ${{B}^{-}}$ branching fraction and the larger background
from nonresonant ${{{B}^{-}}}\\!\rightarrow{{K}^{-}}{\mu^{+}\mu^{-}}$ decays.

##
${{B}_{c}^{+}}\rightarrow{{B}^{*0}_{({s})}}{{\pi}^{+}}\rightarrow{\mu^{+}\mu^{-}}{{\pi}^{+}}$
decays

A similar strategy can in principle also be employed to search for
${{B}_{c}^{+}}\rightarrow{{B}^{*0}_{({s})}}{{\pi}^{+}}\rightarrow{\mu^{+}\mu^{-}}{{\pi}^{+}}$
decays, where the
${{B}_{c}^{+}}\rightarrow{{J\mskip-3.0mu/\mskip-2.0mu\psi\mskip
2.0mu}}{{\pi}^{+}}$ decay can be used for normalisation. In this case,
however, some of the necessary ingredients to convert the signal yield to the
${{B}^{0}_{({s})}}\\!\rightarrow{\mu^{+}\mu^{-}}$ branching fraction are not
currently available. Specifically, the equivalent expression to Eq. (1) is

$\displaystyle{\cal
B}\left({{B}^{*0}_{({s})}}\\!\rightarrow{\mu^{+}\mu^{-}}\right)$
$\displaystyle=$
$\displaystyle\frac{N_{{{B}_{c}^{+}}\\!\rightarrow{{B}^{*0}_{({s})}}{{\pi}^{+}}}}{N_{{{B}_{c}^{+}}\\!\rightarrow{{J\mskip-3.0mu/\mskip-2.0mu\psi\mskip
2.0mu}}{{\pi}^{+}}}}\,\frac{\epsilon_{{{B}_{c}^{+}}\\!\rightarrow{{J\mskip-3.0mu/\mskip-2.0mu\psi\mskip
2.0mu}}{{\pi}^{+}}}}{\epsilon_{{{B}_{c}^{+}}\\!\rightarrow{{B}^{*0}_{({s})}}{{\pi}^{+}}}}\,\frac{{\cal
B}\left({{B}_{c}^{+}}\\!\rightarrow{{J\mskip-3.0mu/\mskip-2.0mu\psi\mskip
2.0mu}}{{\pi}^{+}}\right)}{{\cal
B}\left({{B}_{c}^{+}}\\!\rightarrow{{B}^{*0}_{({s})}}{{\pi}^{+}}\right)}\,{\cal
B}\left({{J\mskip-3.0mu/\mskip-2.0mu\psi\mskip
2.0mu}}\\!\rightarrow{\mu^{+}\mu^{-}}\right)\,,$ (3) $\displaystyle=$
$\displaystyle\alpha_{{{B}^{*0}_{({s})}}\\!\rightarrow{\mu^{+}\mu^{-}}}N_{{{B}_{c}^{+}}\\!\rightarrow{{B}^{*0}_{({s})}}{{\pi}^{+}}}\,,$
(4)

and the branching fractions for
${{B}_{c}^{+}}\rightarrow{{B}^{*0}_{s}}{{\pi}^{+}}$ and
${{B}_{c}^{+}}\rightarrow{{B}^{*0}}{{\pi}^{+}}$ decays are at present
unmeasured. It seems possible, however, that at least the former decay could
be observed with existing data. A previous LHCb analysis has observed
${{B}_{c}^{+}}\rightarrow{{B}^{0}_{s}}{{\pi}^{+}}$ with $3\mbox{\,fb}^{-1}$ of
data [43] and this channel has also been used with the full current data
sample of $9\mbox{\,fb}^{-1}$ to measure the ${{B}_{c}^{+}}$ mass [44]. The
signal for ${{B}_{c}^{+}}\rightarrow{{B}^{*0}_{s}}{{\pi}^{+}}$ would be
expected in the same spectrum as a satellite peak shifted below the
${{B}_{c}^{+}}$ mass by approximately $m_{{{B}^{*0}_{s}}}-m_{{{B}^{0}_{s}}}$
due to the soft photon that is not included in the reconstructed candidate.
Similar “partial reconstruction” techniques have been used by LHCb to observe
${{B}_{c}^{+}}\rightarrow{{J\mskip-3.0mu/\mskip-2.0mu\psi\mskip
2.0mu}}{{D}^{*+}_{s}}$ [45] and ${{{B}^{-}}}\rightarrow{{D}^{*0}}{{\pi}^{-}}$
[46] decays, where the soft neutral particles from the ${{D}^{*+}_{s}}$ and
${{D}^{*0}}$ decays are not included in the reconstructed candidate. Assuming
that ${\cal B}({{B}_{c}^{+}}\rightarrow{{B}^{*0}_{s}}{{\pi}^{+}})$ is not much
smaller than ${\cal B}({{B}_{c}^{+}}\rightarrow{{B}^{0}_{s}}{{\pi}^{+}})$, as
is expected theoretically [47, 48, 49, 50, 51], it should be possible to
observe partially reconstructed
${{B}_{c}^{+}}\rightarrow{{B}^{*0}_{s}}{{\pi}^{+}}$ decays, and to measure the
corresponding branching fraction, in the existing LHCb data sample. The
${{B}_{c}^{+}}\rightarrow{{B}^{0}}{{\pi}^{+}}$ and
${{B}_{c}^{+}}\rightarrow{{B}^{*0}}{{\pi}^{+}}$ decays could be searched for
with a similar technique although, since these transitions are Cabibbo-
suppressed relative to ${{B}_{c}^{+}}\rightarrow{{B}^{(*)0}_{s}}{{\pi}^{+}}$,
larger data samples may be required to observe them. A further, albeit, minor
complication is that the $m_{{{B}^{*0}}}-m_{{{B}^{0}}}$ mass difference is as-
yet unmeasured, although it can be predicted rather reliably from the
$m_{{{B}^{*+}}}-m_{{{{B}^{+}}}}$ mass difference invoking isospin symmetry.
(This can also be interpreted as a further motivation for the analysis since a
first measurement of $m_{{{B}^{*0}}}$ may be possible.)

Another apparent problem is that even for ${{B}_{c}^{+}}$ decay modes which
have been observed, there are currently no measurements of absolute branching
fractions. Rather, only the product of the branching fraction with a ratio of
production fractions is known. However, the ${{B}_{c}^{+}}$ meson production
rate cancels out in Eq. (3), allowing some simplification. With ${\cal
B}\left({{B}_{c}^{+}}\\!\rightarrow{{J\mskip-3.0mu/\mskip-2.0mu\psi\mskip
2.0mu}}{{\pi}^{+}}\right)$, which appears in the numerator, measured relative
to ${{{B}^{+}}}\\!\rightarrow{{J\mskip-3.0mu/\mskip-2.0mu\psi\mskip
2.0mu}}{{K}^{+}}$ and ${\cal
B}\left({{B}_{c}^{+}}\\!\rightarrow{{B}^{*0}_{({s})}}{{\pi}^{+}}\right)$,
which appears in the denominator, measured relative to
${{B}^{0}_{s}}\\!\rightarrow{{J\mskip-3.0mu/\mskip-2.0mu\psi\mskip
2.0mu}}\phi$, it is only necessary to know the relative production rate of
${{{B}^{+}}}$ and ${{B}^{0}_{s}}$ mesons, which has been measured precisely
[52]. Taking the yield of $25.2\times 10^{3}$
${{B}_{c}^{+}}\\!\rightarrow{{J\mskip-3.0mu/\mskip-2.0mu\psi\mskip
2.0mu}}{{\pi}^{+}}$ decays in $9\mbox{\,fb}^{-1}$ of LHCb data from Ref. [44]
and other inputs from Refs. [31, 43, 53, 52], and assuming ${\cal
B}\left({{B}_{c}^{+}}\\!\rightarrow{{B}^{*0}_{s}}{{\pi}^{+}}\right)=\frac{1}{2}{\cal
B}\left({{B}_{c}^{+}}\\!\rightarrow{{B}^{0}_{s}}{{\pi}^{+}}\right)$ and that
the ratio of efficiencies in Eq. (3) is unity, a single-event sensitivity with
$9\mbox{\,fb}^{-1}$ of
$\alpha_{{{B}^{*0}_{s}}\\!\rightarrow{\mu^{+}\mu^{-}}}\approx 1.4\times
10^{-8}$ is obtained. Further assuming the
${{B}_{c}^{+}}\\!\rightarrow{{B}^{*0}}{{\pi}^{+}}$ decay has a Cabibbo-
suppression factor of $\left|V_{cd}/V_{cs}\right|^{2}\approx 5\%$ relative to
${{B}_{c}^{+}}\\!\rightarrow{{B}^{*0}_{s}}{{\pi}^{+}}$, the corresponding
single-event sensitivity is found to be
$\alpha_{{{B}^{*0}}\\!\rightarrow{\mu^{+}\mu^{-}}}\approx 2.7\times 10^{-7}$.
Scaling to a data sample of $300\mbox{\,fb}^{-1}$, the achievable single-
event-sensitivities could reach $\approx 3.4\times 10^{-10}$ and $6.8\times
10^{-9}$ for ${{B}^{*0}_{s}}\\!\rightarrow{\mu^{+}\mu^{-}}$ and
${{B}^{*0}}\\!\rightarrow{\mu^{+}\mu^{-}}$ decays, respectively.

Interpretation of these single-event sensitivities must be made with care,
however, since they assume a selection efficiency comparable to that for
${{B}_{c}^{+}}\\!\rightarrow{{J\mskip-3.0mu/\mskip-2.0mu\psi\mskip
2.0mu}}{{\pi}^{+}}$ decays in Ref. [44]. In practice the selection
requirements will be optimised to account for the level of background in the
signal region for the
${{B}_{c}^{+}}\rightarrow{{B}^{*0}_{({s})}}{{\pi}^{+}}\rightarrow{\mu^{+}\mu^{-}}{{\pi}^{+}}$
search. Since there is no published search for
${{B}_{c}^{+}}\rightarrow{\mu^{+}\mu^{-}}{{\pi}^{+}}$ decays outside the mass
regions where the dimuon pair originates from a
${{J\mskip-3.0mu/\mskip-2.0mu\psi\mskip 2.0mu}}$ or $\psi(2S)$ charmonium
state, it is hard to judge what the optimal requirements are likely to be.
Moreover a contribution from nonresonant
${{B}_{c}^{+}}\rightarrow{\mu^{+}\mu^{-}}{{\pi}^{+}}$ decays, which can occur
in the SM through an annihilation diagram, may provide a limiting background
if it is not negligible around $m({\mu^{+}\mu^{-}})\sim
m_{{{B}^{*0}_{({s})}}}$. Nevertheless, it appears possible that interesting
sensitivity to ${{B}^{*0}_{s}}\\!\rightarrow{\mu^{+}\mu^{-}}$ and
${{B}^{*0}}\\!\rightarrow{\mu^{+}\mu^{-}}$ decays may be achievable.

## Summary

In summary, the ${{{B}^{-}}}\\!\rightarrow{\mu^{+}\mu^{-}}{{\pi}^{-}}$ and
${{B}_{c}^{+}}\\!\rightarrow{\mu^{+}\mu^{-}}{{\pi}^{+}}$ decays provide
interesting possibilities to search for the leptonic weak decays of
${D}^{*0}$, ${B}^{*0}$ and ${B}^{*0}_{s}$ vector mesons. Published data allow
a world-leading limit of ${\cal
B}\left({{D}^{*0}}\\!\rightarrow{\mu^{+}\mu^{-}}\right){~{}\raise
1.49994pt\hbox{$<$}\kern-8.50006pt\lower 3.50006pt\hbox{$\sim$}~{}}3\times
10^{-7}$ to be obtained, and this can be significantly improved upon with a
dedicated analysis of existing data. Sensitivity at the level of ${\cal
O}(10^{-9})$ is expected to be possible with the data sample to be collected
by the end of HL-LHC operation with upgrades of the LHCb experiment. Good
sensitivity to ${{B}^{*0}_{({s})}}\\!\rightarrow{\mu^{+}\mu^{-}}$ decays also
appears achievable, although further experimental investigations will be
needed before a firm conclusion can be reached on this point. In particular,
measurements of ${\cal
B}\left({{B}_{c}^{+}}\\!\rightarrow{{B}^{*0}_{({s})}}{{\pi}^{+}}\right)$ and
studies of nonresonant
${{B}_{c}^{+}}\\!\rightarrow{\mu^{+}\mu^{-}}{{\pi}^{+}}$ decays are needed.

## Acknowledgements

The authors wish to thank their colleagues on the LHCb experiment for the
fruitful and enjoyable collaboration that inspired this study. In particular,
they would like to thank Niels Tuning and Mike Williams for helpful comments.
Additionally, the authors acknowledge stimulating discussions with Benjamín
Grinstein and Alexey Petrov. This work is supported by the Royal Society and
the Science and Technology Facilities Council (UK) and the Monash Warwick
Alliance.

##  Appendix A Sensitivity of searches exploiting prompt production

The predominant fraction of heavy-flavoured mesons are produced “promptly” in
LHC collisions, i.e. they originate directly from the $pp$ interaction vertex
(the so-called primary vertex). Thus one might wonder whether the large
production rate could overcome the sizeable background from random
combinations of tracks produced at the primary vertex, and allow competitive
results on ${D}^{*0}$, ${B}^{*0}$ and ${B}^{*0}_{s}$ dimuon decays to be
obtained.

Fortuitously, the LHCb collaboration has published, as part of a search for
dark photons decaying to ${\mu^{+}\mu^{-}}$ [54], the prompt dimuon spectrum
obtained from a data sample corresponding to $1.6\mbox{\,fb}^{-1}$ of proton-
proton collisions recorded at a centre-of-mass energy of
$13\mathrm{\,Te\kern-1.00006ptV}$, as shown in Fig. 4.333 The spectrum used to
obtain limits in Ref. [54] includes additional criteria to select
preferentially candidates from Drell-Yan production processes, which are not
appropriate here. This is also the case in Ref. [55], which is based on a
larger data sample. A spectrum of prompt dimuons without any additional
requirements, appropriate for the search proposed here, is available in Ref.
[56]; however peaks from the ${{J\mskip-3.0mu/\mskip-2.0mu\psi\mskip 2.0mu}}$
and other resonances have been vetoed so that no normalisation sample is
available in the same spectrum.  This can be used to estimate the sensitivity
to the dimuon decays for each of the ${D}^{*0}$, ${B}^{*0}$ and ${B}^{*0}_{s}$
meson. As an example, for the ${D}^{*0}$ case,

$\displaystyle{\cal B}\left({{D}^{*0}}\\!\rightarrow{\mu^{+}\mu^{-}}\right)$
$\displaystyle=$
$\displaystyle\frac{N_{{{D}^{*0}}\\!\rightarrow{\mu^{+}\mu^{-}}}}{N_{{{J\mskip-3.0mu/\mskip-2.0mu\psi\mskip
2.0mu}}\\!\rightarrow{\mu^{+}\mu^{-}}}}\,\frac{\epsilon_{{{J\mskip-3.0mu/\mskip-2.0mu\psi\mskip
2.0mu}}\\!\rightarrow{\mu^{+}\mu^{-}}}}{\epsilon_{{{D}^{*0}}\\!\rightarrow{\mu^{+}\mu^{-}}}}\,\frac{\sigma_{{{J\mskip-3.0mu/\mskip-2.0mu\psi\mskip
2.0mu}}}}{\sigma_{{{D}^{*0}}}}\,{\cal
B}\left({{J\mskip-3.0mu/\mskip-2.0mu\psi\mskip
2.0mu}}\\!\rightarrow{\mu^{+}\mu^{-}}\right)\,,$ (5) $\displaystyle=$
$\displaystyle\alpha^{\rm
prompt}_{{{D}^{*0}}\\!\rightarrow{\mu^{+}\mu^{-}}}N_{{{D}^{*0}}\\!\rightarrow{\mu^{+}\mu^{-}}}\,,$
(6)

where $N$ and $\epsilon$ represent the yields for the decays given in the
subscripts. Since analysis of data from the LHCb experiment is being
considered, $\sigma_{{{J\mskip-3.0mu/\mskip-2.0mu\psi\mskip 2.0mu}}}$ and
$\sigma_{{{D}^{*0}}}$ refer to the inclusive prompt
${p}{p}\rightarrow{{J\mskip-3.0mu/\mskip-2.0mu\psi\mskip 2.0mu}}$ and
${p}{p}\rightarrow{{D}^{*0}}$ production cross-sections in the LHCb
acceptance.

Figure 4: Spectrum of prompt dimuon pairs recorded by LHCb, with background
contributions indicated, from Ref. [54].

The ${J\mskip-3.0mu/\mskip-2.0mu\psi\mskip 2.0mu}$ production cross-section at
$\sqrt{s}=13\mathrm{\,Te\kern-1.00006ptV}$ has been measured to be
$\sigma_{{{J\mskip-3.0mu/\mskip-2.0mu\psi\mskip 2.0mu}}}=15.03\pm 0.03\pm
0.94{\rm\,\upmu b}$ [57]. The cross-section for the ${D}^{*0}$ meson is not
measured, but the approximation $\sigma_{{{D}^{*0}}}\sim\sigma_{{{D}^{*+}}}$
can be made, so that $\sigma_{{{D}^{*+}}}=784\pm 4\pm 87{\rm\,\upmu b}$ at
$13\mathrm{\,Te\kern-1.00006ptV}$ [58] can be used. Hence, estimating
$N_{{{J\mskip-3.0mu/\mskip-2.0mu\psi\mskip
2.0mu}}\\!\rightarrow{\mu^{+}\mu^{-}}}\sim 2\times 10^{7}$ from Fig. 4, and
assuming that the ratio of efficiencies is approximately unity, one obtains a
single-event sensitivity with $1.6\mbox{\,fb}^{-1}$ of
$13\mathrm{\,Te\kern-1.00006ptV}$ data of $\alpha^{\rm
prompt}_{{{D}^{*0}}\\!\rightarrow{\mu^{+}\mu^{-}}}\sim 6\times 10^{-11}$.

The single-event sensitivities for dimuon decays of ${B}^{*0}$ and
${B}^{*0}_{s}$ mesons can be estimated by replacing $\sigma_{{{D}^{*0}}}$ with
the corresponding ${B}^{*0}_{({s})}$ production cross-section. Since these are
not measured, rough estimates need to be made: $\sigma_{{{B}^{*0}}}$ is taken
to be one-tenth of $\sigma_{{{D}^{*0}}}$, with $\sigma_{{{B}^{*0}_{s}}}$
suppressed by a further factor of four, consistent with the relative
production rates of beauty and charm pseudoscalar mesons [58, 59, 52]. This
gives $\alpha^{\rm prompt}_{{{B}^{*0}}\\!\rightarrow{\mu^{+}\mu^{-}}}\sim
6\times 10^{-10}$ and $\alpha^{\rm
prompt}_{{{B}^{*0}_{s}}\\!\rightarrow{\mu^{+}\mu^{-}}}\sim 2\times 10^{-9}$.

As is clear from Fig. 4, searches for rare processes in prompt production will
suffer from large backgrounds. The number of dimuon candidates per bin in the
region around $m_{{{D}^{*0}}}$ is just over $10^{5}$, and since the bin width
is one half of the mass resolution, this corresponds to a background yield of
around $5\times 10^{5}$ in a signal window of $\pm 2$ times the mass
resolution around the signal peak position. Hence the uncertainty on
$N_{{{D}^{*0}}\\!\rightarrow{\mu^{+}\mu^{-}}}$ arising from a fit to this
distribution is expected to be around $700$. Similar uncertainties are
estimated also for $N_{{{B}^{*0}}\\!\rightarrow{\mu^{+}\mu^{-}}}$ and
$N_{{{B}^{*0}_{s}}\\!\rightarrow{\mu^{+}\mu^{-}}}$. Assuming that the upper
limit corresponds to twice this uncertainty then gives

$\displaystyle{\cal B}\left({{D}^{*0}}\\!\rightarrow{\mu^{+}\mu^{-}}\right)$
$<$$\sim$ $\displaystyle 10^{-7}\,,$ $\displaystyle{\cal
B}\left({{B}^{*0}}\\!\rightarrow{\mu^{+}\mu^{-}}\right)$ $<$$\sim$
$\displaystyle 10^{-6}\,,$ $\displaystyle{\cal
B}\left({{B}^{*0}_{s}}\\!\rightarrow{\mu^{+}\mu^{-}}\right)$ $<$$\sim$
$\displaystyle 3\times 10^{-6}\,,$

where the uncertainty on the single-event sensitivity has not been included.

These values are rough estimates of the sensitivity that could be obtained
from the $1.6\mbox{\,fb}^{-1}$ data sample of Ref. [54]. Real measurements
with this approach would require the relevant production cross-sections to be
determined, rather than approximated as done above. Such measurements require
the reconstruction of the soft neutral particles emitted in the
${{D}^{*0}}\rightarrow{{D}^{0}}{{\pi}^{0}}$ or ${{D}^{0}}\gamma$,
${{B}^{*0}}\rightarrow{{B}^{0}}\gamma$ and
${{B}^{*0}_{s}}\rightarrow{{B}^{0}_{s}}\gamma$ decays, for which the LHCb
detector is not optimised. These challenges are not insurmountable, as
demonstrated by LHCb measurements of $\chi_{c}$ and $\chi_{b}$ production with
photons reconstructed either in the calorimeter or through
$\gamma\rightarrow{e^{+}e^{-}}$ conversions in detector material [60, 61, 62].
Nonetheless, given that the limits are background dominated and will therefore
improve only slowly with larger data samples, this approach looks less
attractive compared to production through $B$-meson decays.

##  Appendix B Sensitivity of (semi-)inclusive searches exploiting displaced
production

A possible strategy to avoid the large background in prompt searches could be
to make an inclusive search for dimuon ${D}^{*0}$, ${B}^{*0}$ and
${B}^{*0}_{s}$ decays originating from a displaced vertex (i.e. from
$b$-hadron decays). Considering for example the
${{D}^{*0}}\\!\rightarrow{\mu^{+}\mu^{-}}$ case, the number of signal decays
in the sample will be given by

$N_{{{D}^{*0}}\\!\rightarrow{\mu^{+}\mu^{-}}}={\cal L}\,{\cal
B}\left({{D}^{*0}}\\!\rightarrow{\mu^{+}\mu^{-}}\right)\sum_{i}\sigma_{B_{i}}\,{\cal
B}\left(B_{i}\\!\rightarrow{{D}^{*0}}{\cal
X}\right)\,\epsilon_{B_{i}\\!\rightarrow{{D}^{*0}}{\cal X}}\,,$ (7)

where ${\cal L}$ is the integrated luminosity of the sample, $\sigma_{B_{i}}$
is the production cross-section for $b$-hadron species $B_{i}$ that has an
inclusive branching fraction to decay to a final state containing a ${D}^{*0}$
meson of ${\cal B}\left(B_{i}\\!\rightarrow{{D}^{*0}}{\cal X}\right)$, where
${\cal X}$ indicates any set of additional particles.444 Most generally one
should consider all weakly decaying hadrons with non-zero branching fractions
for decays including ${D}^{*0}$ mesons in the final state. This would include
double charm baryons, for example. However, the additional contributions to
the total inclusive rate are negligible.  The corresponding efficiency is
$\epsilon_{B_{i}\\!\rightarrow{{D}^{*0}}{\cal X}}$, which will depend on the
requirements imposed to select displaced vertices and on the lifetime and
kinematics of the $B_{i}$ hadron. Considering every possible $B_{i}$ species
would be challenging, but for the ${D}^{*0}$ case production through
${{B}^{+}}$ and ${B}^{0}$ decays can be assumed to dominate, and knowledge
exists of their production cross-sections [59] and inclusive branching
fractions [63]. As usual for measurements in hadron collider experiments, it
is experimentally convenient to use a normalisation channel in order to cancel
the luminosity and avoid dependence on the absolute efficiencies. The
${{J\mskip-3.0mu/\mskip-2.0mu\psi\mskip 2.0mu}}\rightarrow{\mu^{+}\mu^{-}}$
channel is again well-suited for this purpose. Contributions from
${{B}^{0}_{s}}$ and ${{\mathchar 28931\relax}^{0}_{b}}$ decays are not
negligible in this case but measurements exist of all needed quantities [31].
This approach however is not currently viable for dimuon ${B}^{*0}$ and
${B}^{*0}_{s}$ decay searches, due to lack of knowledge of the relevant cross-
sections and branching fractions.

Figure 5: Upper limits at 90% confidence level on the production cross-section
times branching fraction $\sigma(X)\,{\cal
B}\left(X\rightarrow{\mu^{+}\mu^{-}}\right)$ obtained from an inclusive search
for displaced $X\rightarrow{\mu^{+}\mu^{-}}$ decays, from Ref. [56]. Limits
are shown as a function of the mass of the hypothesised $X$ particle and its
transverse momentum.

The LHCb collaboration has published the results of a search for inclusive
displaced $X\rightarrow{\mu^{+}\mu^{-}}$ decays, without any requirement that
the $X$ particle is produced at the primary $pp$ collision vertex [56]. This
search was motivated by the possible existence of beyond Standard Model
particles (generically denoted $X$) and includes selection requirements to
suppress background from dimuon pairs produced in $b$-hadron decay chains.
Moreover, only the region below the ${{J\mskip-3.0mu/\mskip-2.0mu\psi\mskip
2.0mu}}$ mass is included in the search so the normalisation channel is not
available. Therefore, it is not useful to attempt to use the results of this
search, shown in Fig. 5, to obtain information about
${{D}^{*0}}\\!\rightarrow{\mu^{+}\mu^{-}}$ decays; nonetheless, Ref. [56]
demonstrates that a dedicated analysis using this approach could be carried
out. However, compared to the exclusive approach emphasised in this paper, the
mass resolution will be worse and higher irreducible backgrounds are expected
as all $b$-hadron decays producing a nonresonant ${\mu^{+}\mu^{-}}$ pair plus
one or more extra particles will contribute. As such, this approach appears
unlikely to be competitive with large data samples.

Background from nonresonant ${\mu^{+}\mu^{-}}$ pairs produced in $b$-hadron
decays is irreducible in a fully inclusive search. However, a semi-inclusive
approach could be used to remove it, in which an additional muon (or electron)
is required to originate from the same vertex. In this scenario, the source of
${D}^{*0}$ mesons is through semileptonic $b$-hadron (dominated by ${{B}^{+}}$
and ${B}^{0}$) decays. Also in this case there is a relevant prior LHCb
publication, in the form of a search for
${{{B}^{+}}}\\!\rightarrow{\mu^{+}\mu^{-}}{\mu^{+}}\nu$ decays [64]. This
analysis, however, includes a requirement that one ${\mu^{+}\mu^{-}}$ pair has
mass below $980{\mathrm{\,Me\kern-1.00006ptV\\!/}c^{2}}$, and therefore cannot
be used directly to obtain a limit on ${{{B}^{+}}}\\!\rightarrow\left({{\kern
1.99997pt\overline{\kern-1.99997ptD}{}}{}^{*0}}\\!\rightarrow{\mu^{+}\mu^{-}}\right){\mu^{+}}\nu$.
Nonetheless, it appears likely that a dedicated analysis, focusing on the
$m({\mu^{+}\mu^{-}})\sim m_{{{D}^{*0}}}$ mass range, could achieve a limit on
${\cal B}\left({{D}^{*0}}\rightarrow{\mu^{+}\mu^{-}}\right)$ of ${\cal
O}\left(10^{-7}\right)$. The possibility to relax selection requirements to
allow additional particles, beyond the neutrino, in the decay could also be
exploited if the corresponding improvement in single-event sensitivity is
sufficient to overcome the increase in background. Thus, if background levels
can be kept very low, this approach may in the long-term be competitive with,
or even better than, the method with exclusive reconstruction of hadronic
decays. Unfortunately, with the currently available information the achievable
sensitivity cannot be reliably quantified. Similarly, the use of
$B_{i}\\!\rightarrow{{B}^{*0}_{({s})}}\mu{\cal X}$ production, with the most
important contribution expected to be from semileptonic ${B}_{c}^{+}$ decays,
to search for ${{B}^{*0}_{({s})}}\\!\rightarrow{\mu^{+}\mu^{-}}$ transitions
appears an interesting possibility, but in the absence of relevant
experimental results the achievable sensitivity cannot be estimated.

## References

  * [1] S. L. Glashow, J. Iliopoulos, and L. Maiani, _Weak interactions with lepton-hadron Symmetry_ , Phys. Rev. D2 (1970) 1285
  * [2] N. Cabibbo, _Unitary symmetry and leptonic decays_ , Phys. Rev. Lett. 10 (1963) 531
  * [3] M. Kobayashi and T. Maskawa, _$C\\!P$ violation in the renormalizable theory of weak interaction_, Prog. Theor. Phys. 49 (1973) 652
  * [4] C. Bobeth et al., _$B_{s,d}\rightarrow l^{+}l^{-}$ in the Standard Model with reduced theoretical uncertainty_, Phys. Rev. Lett. 112 (2014) 101801, arXiv:1311.0903
  * [5] LHCb collaboration, R. Aaij et al., _Measurement of the ${{B}^{0}_{s}}\\!\rightarrow{\mu^{+}\mu^{-}}$ branching fraction and effective lifetime and search for ${{B}^{0}}\\!\rightarrow{\mu^{+}\mu^{-}}$ decays_, Phys. Rev. Lett. 118 (2017) 191801, arXiv:1703.05747
  * [6] CMS collaboration, A. M. Sirunyan et al., _Measurement of properties of B ${}^{0}_{\mathrm{s}}\rightarrow\mu^{+}\mu^{-}$ decays and search for B${}^{0}\rightarrow\mu^{+}\mu^{-}$ with the CMS experiment_, JHEP 04 (2020) 188, arXiv:1910.12127
  * [7] ATLAS collaboration, M. Aaboud et al., _Study of the rare decays of $B^{0}_{s}$ and $B^{0}$ mesons into muon pairs using data collected during 2015 and 2016 with the ATLAS detector_, JHEP 04 (2019) 098, arXiv:1812.03017
  * [8] LHCb collaboration, R. Aaij et al., _Analysis of neutral $B$-meson decays into two muons_, Phys. Rev. Lett. 128 (2022) 041801, arXiv:2108.09284
  * [9] LHCb collaboration, R. Aaij et al., _Measurement of the ${{B}^{0}_{s}}\rightarrow{\mu^{+}}{\mu^{-}}$ decay properties and search for the ${{B}^{0}}\rightarrow{\mu^{+}}{\mu^{-}}$ and ${{B}^{0}_{s}}\rightarrow{\mu^{+}}{\mu^{-}}\gamma$ decays_, Phys. Rev. D 105 (2022) 012010, arXiv:2108.09283
  * [10] ATLAS, CMS and LHCb collaborations, _Combination of the ATLAS, CMS and LHCb results on the $B_{(s)}^{0}\rightarrow\mu^{+}\mu^{-}$ decays_, 2020.  ATLAS-CONF-2020-049, CMS PAS BPH-20-003, LHCb-CONF-2020-002
  * [11] Belle collaboration, M. Petric et al., _Search for leptonic decays of $D^{0}$ mesons_, Phys. Rev. D81 (2010) 091102, arXiv:1003.2345
  * [12] LHCb collaboration, R. Aaij et al., _Search for the rare decay ${{D}^{0}}\\!\rightarrow{\mu^{+}\mu^{-}}$_, Phys. Lett. B725 (2013) 15, arXiv:1305.5059
  * [13] LHCb collaboration, R. Aaij et al., _Search for the rare decays ${{B}^{0}_{s}}\\!\rightarrow{e^{+}e^{-}}$ and ${{B}^{0}}\\!\rightarrow{e^{+}e^{-}}$_, Phys. Rev. Lett. 124 (2020) 211802, arXiv:2003.03999
  * [14] LHCb collaboration, R. Aaij et al., _Search for the decays ${{B}^{0}_{s}}\\!\rightarrow{\tau^{+}}{\tau^{-}}$ and ${{B}^{0}}\\!\rightarrow{\tau^{+}}{\tau^{-}}$_, Phys. Rev. Lett. 118 (2017) 251802, arXiv:1703.02508
  * [15] BaBar collaboration, J. P. Lees et al., _Measurement of the $D^{*}(2010)^{+}$ meson width and the $D^{*}(2010)^{+}-D^{0}$ mass difference_, Phys. Rev. Lett. 111 (2013) 111801, arXiv:1304.5657
  * [16] FOCUS collaboration, J. M. Link et al., _New measurements of the $D^{0}$ and $D^{+}$ lifetimes_, Phys. Lett. B537 (2002) 192, arXiv:hep-ex/0203037
  * [17] J. Albrecht, D. van Dyk, and C. Langenbruch, _Flavour anomalies in heavy quark decays_ , Prog. Part. Nucl. Phys. 120 (2021) 103885, arXiv:2107.04822
  * [18] B. Grinstein and J. Martin Camalich, _Weak decays of excited $B$ mesons_, Phys. Rev. Lett. 116 (2016) 141801, arXiv:1509.05049
  * [19] A. Khodjamirian, T. Mannel, and A. A. Petrov, _Direct probes of flavor-changing neutral currents in ${e^{+}e^{-}}$-collisions_, JHEP 11 (2015) 142, arXiv:1509.07123
  * [20] G.-Z. Xu, Y. Qiu, C.-P. Shen, and Y.-J. Zhang, _$B^{*}_{s,d}\rightarrow\mu^{+}\mu^{-}$ and its impact on $B_{s,d}\rightarrow\mu^{+}\mu^{-}$_, Eur. Phys. J. C76 (2016) 583, arXiv:1601.03386
  * [21] S. Sahoo and R. Mohanta, _Study of the rare decays $B_{s,d}^{*}\rightarrow\mu^{+}\mu^{-}$_, J. Phys. G44 (2017) 035001, arXiv:1612.02543
  * [22] D. Banerjee, P. Maji, and S. Sahoo, _Study of the rare decays $B^{*}_{s,d}\rightarrow l^{+}l^{-}$ in $Z^{\prime}$ model_, Int. J. Mod. Phys. A32 (2017) 1750075, arXiv:1712.05544
  * [23] D. Kumar, J. Saini, S. Gangal, and S. B. Das, _Probing new physics through $B^{*}_{s}\rightarrow\mu^{+}\mu^{-}$ decay_, Phys. Rev. D97 (2018) 035007, arXiv:1711.01989
  * [24] S. Kumbhakar and J. Saini, _New physics effects in purely leptonic $B^{*}_{s}$ decays_, Eur. Phys. J. C79 (2019) 394, arXiv:1807.04055
  * [25] G. Burdman, E. Golowich, J. L. Hewett, and S. Pakvasa, _Rare charm decays in the standard model and beyond_ , Phys. Rev. D66 (2002) 014009, arXiv:hep-ph/0112235
  * [26] S. de Boer and G. Hiller, _Flavor and new physics opportunities with rare charm decays into leptons_ , Phys. Rev. D93 (2016) 074001, arXiv:1510.00311
  * [27] R. Bause, M. Golz, G. Hiller, and A. Tayduganov, _The new physics reach of null tests with $D\rightarrow\pi\ell\ell$ and $D_{s}\rightarrow K\ell\ell$ decays_, Eur. Phys. J. C80 (2020) 65, Erratum ibid. C81 (2021) 219, arXiv:1909.11108
  * [28] A. Bharucha, D. Boito, and C. Méaux, _Disentangling QCD and new physics in $D^{+}\rightarrow\pi^{+}\ell^{+}\ell^{-}$_, JHEP 04 (2021) 158, arXiv:2011.12856
  * [29] CMD-3 collaboration, D. N. Shemyakin, _Search for the process $e^{+}e^{-}\rightarrow D^{*}(2007)^{0}$ with the CMD-3 Detector_, Phys. Atom. Nucl. 83 (2020) 954
  * [30] LHCb collaboration, R. Aaij et al., _First measurement of the differential branching fraction and $C\\!P$ asymmetry of the ${{{B}^{+}}}\\!\rightarrow{{\pi}^{+}}{\mu^{+}\mu^{-}}$ decay_, JHEP 10 (2015) 034, arXiv:1509.00414
  * [31] Particle Data Group, P. A. Zyla et al., _Review of particle physics_ , Prog. Theor. Exp. Phys. 2020 (2020) 083C01
  * [32] A. Ali, A. Y. Parkhomenko, and A. V. Rusov, _Precise calculation of the dilepton invariant-mass spectrum and the decay rate in $B^{\pm}\rightarrow\pi^{\pm}\mu^{+}\mu^{-}$ in the SM_, Phys. Rev. D89 (2014) 094021, arXiv:1312.2523
  * [33] C. Hambrock, A. Khodjamirian, and A. Rusov, _Hadronic effects and observables in $B\rightarrow\pi\ell^{+}\ell^{-}$ decay at large recoil_, Phys. Rev. D92 (2015) 074020, arXiv:1506.07760
  * [34] Fermilab Lattice and MILC collaborations, J. A. Bailey et al., _$B\rightarrow\pi\ell\ell$ form factors for new-physics searches from lattice QCD_, Phys. Rev. Lett. 115 (2015) 152002, arXiv:1507.01618
  * [35] A. Ali, P. Ball, L. T. Handoko, and G. Hiller, _A comparative study of the decays $B\rightarrow$ ($K$, $K^{*)}\ell^{+}\ell^{-}$ in standard model and supersymmetric theories_, Phys. Rev. D61 (2000) 074024, arXiv:hep-ph/9910221
  * [36] LHCb collaboration, R. Aaij et al., _Differential branching fractions and isospin asymmetries of ${B}\\!\rightarrow K^{(*)}{\mu^{+}\mu^{-}}$ decays_, JHEP 06 (2014) 133, arXiv:1403.8044
  * [37] LHCb collaboration, R. Aaij et al., _Measurement of the $b$-quark production cross-section in 7 and 13$~{}T\kern-0.50003pteV$ ${p}{p}$ collisions_, Phys. Rev. Lett. 118 (2017) 052002, Erratum ibid. 119 (2017) 169901, arXiv:1612.05140
  * [38] W. A. Rolke, A. M. Lopez, and J. Conrad, _Limits and confidence intervals in the presence of nuisance parameters_ , Nucl. Instrum. Meth. A551 (2005) 493, arXiv:physics/0403059
  * [39] LHCb collaboration, R. Aaij et al., _Measurement of the phase difference between the short- and long-distance amplitudes in the ${{{B}^{+}}}\\!\rightarrow{{K}^{+}}{\mu^{+}\mu^{-}}$ decay_, Eur. Phys. J. C77 (2017) 161, arXiv:1612.06764
  * [40] LHCb collaboration, _Physics case for an LHCb Upgrade II — Opportunities in flavour physics, and beyond, in the HL-LHC era_ , arXiv:1808.08865
  * [41] LHCb collaboration, _Expression of Interest for a Phase-II LHCb Upgrade: Opportunities in flavour physics, and beyond, in the HL-LHC era_ , CERN-LHCC-2017-003
  * [42] LHCb collaboration, R. Aaij et al., _Measurement of CP observables in $B^{\pm}\rightarrow D^{(*)}K^{\pm}$ and $B^{\pm}\rightarrow D^{(*)}\pi^{\pm}$ decays using two-body $D$ final states_, JHEP 04 (2021) 081, arXiv:2012.09903
  * [43] LHCb collaboration, R. Aaij et al., _Observation of the decay ${{B}_{c}^{+}}\\!\rightarrow{{B}^{0}_{s}}{{\pi}^{+}}$_, Phys. Rev. Lett. 111 (2013) 181801, arXiv:1308.4544
  * [44] LHCb collaboration, R. Aaij et al., _Precision measurement of the ${B}_{c}^{+}$ meson mass_, JHEP 07 (2020) 123, arXiv:2004.08163
  * [45] LHCb collaboration, R. Aaij et al., _Observation of ${{B}_{c}^{+}}\\!\rightarrow{{J\mskip-3.0mu/\mskip-2.0mu\psi\mskip 2.0mu}}{{D}^{+}_{s}}$ and ${{B}_{c}^{+}}\\!\rightarrow{{J\mskip-3.0mu/\mskip-2.0mu\psi\mskip 2.0mu}}{{D}^{*+}_{s}}$ decays_, Phys. Rev. D87 (2013) 112012, arXiv:1304.4530
  * [46] LHCb collaboration, R. Aaij et al., _Measurement of $C\\!P$ observables in ${{B}^{\pm}}\\!\rightarrow D^{(\ast)}{{K}^{\pm}}$ and ${{B}^{\pm}}\\!\rightarrow D^{(\ast)}{{\pi}^{\pm}}$ decays_, Phys. Lett. B777 (2018) 16, arXiv:1708.06370
  * [47] M. A. Ivanov, J. G. Korner, and P. Santorelli, _Exclusive semileptonic and nonleptonic decays of the $B_{c}$ meson_, Phys. Rev. D73 (2006) 054024, arXiv:hep-ph/0602050
  * [48] J.-f. Sun, Y.-l. Yang, W.-j. Du, and H.-l. Ma, _Study of $B_{c}\rightarrow B^{*}P$, $BV$ decays with QCD factorization_, Phys. Rev. D77 (2008) 114004, arXiv:0806.1254
  * [49] A. K. Likhoded and A. V. Luchinsky, _Light hadron production in $B_{c}\rightarrow B_{s}^{(*)}+X$ decays_, Phys. Rev. D82 (2010) 014012, arXiv:1004.0087
  * [50] S. Naimuddin et al., _Nonleptonic two-body ${{B}_{c}^{+}}$-meson decays_, Phys. Rev. D86 (2012) 094028
  * [51] Y.-J. Shi, W. Wang, and Z.-X. Zhao, _$B_{c}\rightarrow B_{sJ}$ form factors and $B_{c}$ decays into $B_{sJ}$ in covariant light-front approach_, Eur. Phys. J. C76 (2016) 555, arXiv:1607.00622
  * [52] LHCb collaboration, R. Aaij et al., _Precise measurement of the $f_{s}/f_{d}$ ratio of fragmentation fractions and of $B^{0}_{s}$ decay branching fractions_, Phys. Rev. D104 (2021) 032005, arXiv:2103.06810
  * [53] LHCb collaboration, R. Aaij et al., _Measurement of ${B}_{c}^{+}$ production in proton-proton collisions at $\sqrt{s}=$8$~{}T\kern-0.50003pteV$_, Phys. Rev. Lett. 114 (2015) 132001, arXiv:1411.2943
  * [54] LHCb collaboration, R. Aaij et al., _Search for dark photons produced in $13~{}T\kern-0.50003pteV$ ${p}{p}$ collisions_, Phys. Rev. Lett. 120 (2018) 061801, arXiv:1710.02867
  * [55] LHCb collaboration, R. Aaij et al., _Search for $A^{\prime}\\!\rightarrow{\mu^{+}\mu^{-}}$ decays_, Phys. Rev. Lett. 124 (2020) 041801, arXiv:1910.06926
  * [56] LHCb collaboration, R. Aaij et al., _Searches for low-mass dimuon resonances_ , JHEP 10 (2020) 156, arXiv:2007.03923
  * [57] LHCb collaboration, R. Aaij et al., _Measurement of forward ${J\mskip-3.0mu/\mskip-2.0mu\psi\mskip 2.0mu}$ production cross-sections in $p$ $p$ collisions at $\sqrt{s}=$13$~{}T\kern-0.50003pteV$_, JHEP 10 (2015) 172, Erratum ibid. 05 (2017) 063, arXiv:1509.00771
  * [58] LHCb collaboration, R. Aaij et al., _Measurements of prompt charm production cross-sections in $p$ $p$ collisions at $\sqrt{s}=$13$~{}T\kern-0.50003pteV$_, JHEP 03 (2016) 159, Erratum ibid. 09 (2016) 013, Erratum ibid. 05 (2017) 074, arXiv:1510.01707
  * [59] LHCb collaboration, R. Aaij et al., _Measurement of the ${B}^{\pm}$ production cross-section in $p$ $p$ collisions at $\sqrt{s}=$7 and 13$~{}T\kern-0.50003pteV$_, JHEP 12 (2017) 026, arXiv:1710.04921
  * [60] LHCb collaboration, R. Aaij et al., _Measurement of the ratio of prompt $\chi_{c}$ to ${J\mskip-3.0mu/\mskip-2.0mu\psi\mskip 2.0mu}$ production in $p$ $p$ collisions at $\sqrt{s}=$7$~{}T\kern-0.50003pteV$_, Phys. Lett. B718 (2012) 431, arXiv:1204.1462
  * [61] LHCb collaboration, R. Aaij et al., _Measurement of the fraction of ${\mathchar 28935\relax{(1S)}}$ originating from $\chi_{b}(1P)$ decays in $p$ $p$ collisions at $\sqrt{s}=$7$~{}T\kern-0.50003pteV$_, JHEP 11 (2012) 031, arXiv:1209.0282
  * [62] LHCb collaboration, R. Aaij et al., _Study of $\chi_{b}$ meson production in $p$ $p$ collisions at $\sqrt{s}=$7 and 8$~{}T\kern-0.50003pteV$ and observation of the decay $\chi_{b}\\!\rightarrow{\mathchar 28935\relax{(3S)}}\gamma$_, Eur. Phys. J. C74 (2014) 3092, arXiv:1407.7734
  * [63] CLEO collaboration, L. Gibbons et al., _The inclusive decays $B\rightarrow DX$ and $B\rightarrow{{D}^{*}}X$_, Phys. Rev. D56 (1997) 3783, arXiv:hep-ex/9703006
  * [64] LHCb collaboration, R. Aaij et al., _Search for the rare decay ${{{B}^{+}}}\\!\rightarrow{\mu^{+}\mu^{-}}{\mu^{+}}{{\nu}_{\mu}}$_, Eur. Phys. J. C79 (2019) 675, arXiv:1812.06004