# Ultrabright and narrowband intra-fiber biphoton source at ultralow pump
power

Alexander Bruns  Institut für Angewandte Physik, Technische Universität
Darmstadt, Hochschulstraße 6, 64289 Darmstadt, Germany  alexander.bruns@tu-
darmstadt.de Chia-Yu Hsu  Institut für Angewandte Physik, Technische
Universität Darmstadt, Hochschulstraße 6, 64289 Darmstadt, Germany  Department
of Physics, National Tsing Hua University, Hsinchu 30013, Taiwan  Sergiy
Stryzhenko  Institut für Angewandte Physik, Technische Universität Darmstadt,
Hochschulstraße 6, 64289 Darmstadt, Germany  Institute of Physics, National
Academy of Science of Ukraine, Nauky Avenue 46, Kyiv 03028, Ukraine  Enno
Giese  Institut für Angewandte Physik, Technische Universität Darmstadt,
Hochschulstraße 6, 64289 Darmstadt, Germany  Leonid P. Yatsenko  Institute of
Physics, National Academy of Science of Ukraine, Nauky Avenue 46, Kyiv 03028,
Ukraine  Ite A. Yu  Department of Physics, National Tsing Hua University,
Hsinchu 30013, Taiwan  Center for Quantum Technology, Hsinchu 30013, Taiwan
Thomas Halfmann  Institut für Angewandte Physik, Technische Universität
Darmstadt, Hochschulstraße 6, 64289 Darmstadt, Germany  Thorsten Peters
Institut für Angewandte Physik, Technische Universität Darmstadt,
Hochschulstraße 6, 64289 Darmstadt, Germany

###### Abstract

Nonclassical photon sources of high brightness are key components of quantum
communication technologies. We here demonstrate the generation of narrowband,
nonclassical photon pairs by employing spontaneous four-wave mixing in an
optically-dense ensemble of cold atoms within a hollow-core fiber. The
brightness of our source approaches the limit of achievable generated spectral
brightness at which successive photon pairs start to overlap in time. For a
generated spectral brightness per pump power of up to
$2\text{\times}{10}^{9}\text{\,}\mathrm{pairs}\text{/}\text{(}\mathrm{s}\text{⋅}\mathrm{MHz}\text{⋅}\mathrm{mW}\text{)}$
we observe nonclassical correlations at pump powers below
$100\text{\,}\mathrm{nW}\text{/}$ and a narrow bandwidth of $2\pi\times$
$6.5\text{\,}\mathrm{MHz}\text{/}$. In this regime we demonstrate that our
source can be used as a heralded single-photon source. By further increasing
the brightness we enter the regime where successive photon pairs start to
overlap in time and the cross-correlation approaches a limit corresponding to
thermal statistics. Our approach of combining the advantages of atomic
ensembles and waveguide environments is an important step towards photonic
quantum networks of ensemble-based elements.

##  I Introduction

Nonclassical photon sources are of paramount importance for optical quantum
communication Sangouard  _et al._ (2011); Slussarenko and Pryde (2019). Key
requirements for these sources are high single-photon purity and high
brightness, which are typically difficult to achieve simultaneously. If these
photon sources are to be interfaced with atomic or solid-state quantum
memories, spectrally narrow single photons in the $\mathrm{MHz}\text{/}$
regime are additionally required to match the relevant atomic transitions
Eisaman  _et al._ (2011). In the context of quantum networks Kimble (2008) and
moving towards real-life applications, the integrability of the technology
into optical waveguides becomes crucial Wang  _et al._ (2021). This, e.g., has
the advantage of better scalability, lower pump powers, and improved
efficiency due to better mode-matching as compared to free-space setups.

While deterministic sources include the timing information by design,
probabilistic sources require an additional heralding mechanism. It is
commonly realized by implementing a probabilistic source of correlated photon
pairs and using one of the photons as a herald to obtain timing information
about the second photon Eisaman  _et al._ (2011). Such correlated photon pairs
are typically generated by nonlinear optical processes such as spontaneous
parametric down-conversion (SPDC) or spontaneous four-wave mixing (SFWM). Both
of these processes can provide a high spectral biphoton brightness at narrow
bandwidths either due to the inherent frequency filtering of SFWM or by
employing an additional cavity. The highest generated spectral brightnesses
(GSB) reported so far in excess of
$1\text{\times}{10}^{5}\text{\,}\mathrm{pairs}\text{/}\text{(}\mathrm{s}\text{⋅}\mathrm{MHz}\text{)}$
were achieved using a variety of different experimental systems. These are
waveguides combined with cavities Luo  _et al._ (2015); Steiner  _et al._
(2021), a bulk crystal inside a cavity Tsai and Chen (2018), and a room-
temperature atomic ensemble Chen  _et al._ (2022). The bandwidths of these
biphoton sources ranged from sub-MHz Chen  _et al._ (2022) to around
$100\text{\,}\mathrm{MHz}\text{/}$ Steiner  _et al._ (2021) and the generated
spectral brightness per pump power (GSBP) ranged from
$3\text{\times}{10}^{4}\text{\,}\mathrm{pairs}\text{/}\text{(}\mathrm{s}\text{⋅}\mathrm{MHz}\text{⋅}\mathrm{mW}\text{)}$
Luo  _et al._ (2015) to
$2\text{\times}{10}^{8}\text{\,}\mathrm{pairs}\text{/}\text{(}\mathrm{s}\text{⋅}\mathrm{MHz}\text{⋅}\mathrm{mW}\text{)}$
Steiner  _et al._ (2021). While each system and technique has its own
advantages and it is therefore difficult to compare the values achieved, they
put our results into context.

In the following, we report on the first implementation of a biphoton source
using SFWM in an ensemble of cold rubidium atoms loaded into a hollow-core
fiber. To avoid collisions of the cold atoms with the fiber wall, thereby
reducing the coherence time, the atoms are guided by an optical dipole trap.
By using cold atoms we achieve an order of magnitude lower biphoton bandwidth
as compared to fibers filled with warm gases Cordier  _et al._ (2020); Lopez-
Huidobro  _et al._ (2021). This was first shown by Corzo et al., who
interfaced cold atoms with a nanofiber to generate narrowband single photons
on-demand Corzo  _et al._ (2019). However, using a hollow-core fiber instead
of a nanofiber has the benefit that all light fields, including the pump, are
guided in the same optical mode. Thus, we obtain intrinsically optimal mode-
matching as well as strong light-atom coupling at orders of magnitude lower
pump powers. This results in a GSBP of up to
$2\text{\times}{10}^{9}\text{\,}\mathrm{pairs}\text{/}\text{(}\mathrm{s}\text{⋅}\mathrm{MHz}\text{⋅}\mathrm{mW}\text{)}$,
which is a 10-fold increase over the previous record Steiner  _et al._ (2021)
at 100-fold reduced pump power and 10-fold lower bandwidth of $2\pi\times$
$6.5\text{\,}\mathrm{MHz}\text{/}$, which is directly compatible with atomic
quantum memories. Moreover, we show that by tuning the brightness of our
source even higher, we reach the ultimate achievable limit of spectral
brightness, at which successive photon pairs start to overlap in time. In this
regime, the cross-correlation approaches a limit expected for thermal
statistics. To characterize our photon source, we present thorough
measurements on the cross- and auto-correlations, bandwidth, and GSB of the
photon pairs. Where possible, we compare our experimental results to
theoretical simulations for a quantitative analysis. Due to the high GSB,
narrow bandwidth, and waveguide-coupled single-mode emission, possible
applications of our biphoton source include integration into photonic quantum
networks Kimble (2008) using atomic ensembles as building blocks Sangouard
_et al._ (2011), quantum key distribution via satellite links Bedington  _et
al._ (2017), and photonic quantum metrology Polino  _et al._ (2020).

Figure 1: (a) Double-$\Lambda$ system for SFWM in ${}^{87}\text{Rb}$. The
excited states decay at rates $\Gamma_{\mathrm{D}1}=2\pi\times$
$5.75\text{\,}\mathrm{MHz}\text{/}$ and $\Gamma_{\mathrm{D}2}=2\pi\times$
$6.06\text{\,}\mathrm{MHz}\text{/}$. We measured the ground-state decoherence
rate (via EIT light storage similar to Peters  _et al._ (2020)) to be
$\gamma_{12}=$0.057\text{\,}\Gamma_{\mathrm{D}1}$$. For all material constants
we use the values given in Steck (2021). (b) Simplified experimental setup.
Solid/dotted arrows depict applied/generated light fields. Colors match those
in the level scheme. Small arrows indicate optical noise before sufficient
filtering. (P/F)BS: (polarizing/fiber) beamsplitters, aHWP: achromatic half-
wave-plate, L: lens, UHV: ultra-high vacuum, DM: dichroic mirror, FE: filter
etalons, BP: optical bandpass filter. Solid lines represent optical single-
mode fibers and dotted lines electronic signals. (c) Simplified temporal
sequence.

##  II Experimental Methods

Our SFWM coupling scheme for photon-pair generation using ${}^{87}\text{Rb}$
atoms is shown in figure 1(a). As we do not intentionally lift the degeneracy
of the Zeeman states, only the hyperfine states are shown. A pump beam
(frequency $\omega_{P}$) with power $P$ on the D2 line, blue-detuned by
$\Delta=$53\text{\,}\Gamma_{\mathrm{D}2}$$ from the transition
$\ket{1}\leftrightarrow\ket{4}$ to minimize optical pumping, generates Stokes
(S) photons (frequency $\omega_{S}$) via spontaneous Raman scattering. The
collinear control beam (frequency $\omega_{C}$) with Rabi frequency
$\Omega_{C}$ on the D1 line is applied resonantly to the transition
$\ket{2}\leftrightarrow\ket{3}$. This creates a narrow transparency window via
electromagnetically induced transparency (EIT) Fleischhauer  _et al._ (2005)
for the correlated anti-Stokes (AS) photons (frequency $\omega_{AS}$) in the
SFWM process Balić  _et al._ (2005).

The nonlinear optical medium in our experiment is an ensemble of up to
$2.5\text{\times}{10}^{5}$ atoms at a temperature of about
$1\text{\,}\mathrm{mK}\text{/}$, a length of about
$L=$6\text{\,}\mathrm{cm}\text{/}$$ and a radial diameter of
$d=$3.4\text{\,}\mathrm{\SIUnitSymbolMicro m}\text{/}$$ that is located inside
a hollow-core photonic bandgap fiber (HCPBGF) Poletti  _et al._ (2013) (NKT
Photonics, HC800-02, $14\text{\,}\mathrm{cm}\text{/}$ long, numerical aperture
$\sim$0.15$$, $5.5\text{\,}\mathrm{\SIUnitSymbolMicro m}\text{/}$ mode field
$1/\mathrm{e}^{2}$ diameter of intensity). The process of preparing the cold
atoms inside a magneto-optical trap (MOT) and guiding them into the HCPBGF
using a far-off resonant trap (FORT) at a wavelength of
$820\text{\,}\mathrm{nm}\text{/}$ was previously explained and characterized
in detail Blatt  _et al._ (2014); Peters  _et al._ (2021). For simplicity, we
therefore omit here all details regarding the preparation of the atomic
ensemble inside the HCPBGF.

The optical setup of the SFWM experiment is schematically shown in figure
1(b). All laser beams are generated using home-built diode laser systems with
linewidths of several hundred kHz (except of the FORT Peters  _et al._
(2021)). The timing and frequency of the laser beams are controlled by
acousto-optic modulators with switching times of about
$100\text{\,}\mathrm{ns}\text{/}$. Because all beams of SFWM are guided in a
single optical mode of the HCPBGF, we achieve Rabi frequencies of the order of
$\Gamma$ with nanowatts of optical power only. The complete experimental setup
requires an optical power of about $300\text{\,}\mathrm{mW}\text{/}$ to
operate the MOT and the FORT. Pump and control fields are orthogonally
linearly polarized and combined in front of the HCPBGF using a polarizing
beamsplitter (PBS). As the involved wavelengths span
$40\text{\,}\mathrm{nm}\text{/}$ we use achromatic half-wave plates to align
the polarization axes of the linearly polarized laser fields to the optical
axis of the HCPBGF at the input. This allows for achieving the highest degree
of polarization at the output and thus optimum polarization filtering as
required for operation at the photon level Peters  _et al._ (2020). As
confirmed by numerical simulations (see appendix A.2) of the SFWM process
including the Zeeman structure and laser field polarizations, the generated S
(AS) field is orthogonally polarized to the pump (control) field. Thus, the
two generated fields can be separated by a PBS which also serves as a first
filter stage. The optical noise originating from residual pump (control) light
in the S (AS) channel separated by $6.8\text{\,}\mathrm{GHz}\text{/}$ is
attenuated by up to $40\text{\,}\mathrm{dB}\text{/}$. After polarization
filtering the photons are coupled into single-mode fibers which serve as
spatial filters to isolate the HCPBGF’s optical mode. As a second filtering
stage we use temperature-tuned monolithic etalons Palittapongarnpim  _et al._
(2012); Ahlrichs  _et al._ (2013) with an attenuation of up to
$45\text{\,}\mathrm{dB}\text{/}$ to further attenuate the optical noise in the
S (AS) channel that originates from the pump (control) beam (separated by
$6.8\text{\,}\mathrm{GHz}\text{/}$) and the control (pump) beam (separated by
$15\text{\,}\mathrm{nm}\text{/}$). Finally, we employ narrowband optical
filters (blocking optical depth (OD) $>4$) to further attenuate the
$15\text{\,}\mathrm{nm}\text{/}$ separated optical noise and broadband noise
originating from stray light. The laser beams not directly involved in the
SFWM process, i.e., preparation beam and FORT, are aligned counter-propagating
to the pump and control beams to protect the photon counting equipment and to
minimize optical noise. The residual optical noise (measured without loading
atoms into the HCPBGF) is about $2800\text{\,}\text{/}\mathrm{s}$
($1200\text{\,}\text{/}\mathrm{s}$) for the S (AS) channel.

To detect and characterize the generated photon pairs, we implement Hanbury
Brown & Twiss setups for both photonic channels by splitting the signals with
50:50 fiber beamsplitters before guiding them to single-photon counting
modules (SPCM, Excelitas, SPCM-AQRH-13). The overall detection efficiency in
both channels is about $8\text{\,}\mathrm{\char 37\relax}\text{/}$, including
the transmission through all elements after the HCPBGF and the intrinsic
quantum efficiencies of our detectors. We acquire the timings of all detected
photons with time-tagging electronics (Swabian Instruments, Timetagger 20) for
further analysis. The combined timing jitter of the detection system is
specified as $<$$400\text{\,}\mathrm{ps}\text{/}$, while the temporal dynamics
of the biphoton waveform are on a timescale of a few
$10\text{\,}\mathrm{ns}\text{/}$ (see Figure 2). Hence, we neglect this jitter
in the analysis of our data.

Figure 1(c) shows the temporal sequence of the experiment. After the atoms are
captured in the MOT and subsequently loaded into the HCPBGF, we start the
experimental SFWM phase. To avoid any influence of the strong FORT on the SFWM
process, we periodically switch off the FORT for periods of
$2.4\text{\,}\mathrm{\SIUnitSymbolMicro s}\text{/}$ before switching it on
again for $1.6\text{\,}\mathrm{\SIUnitSymbolMicro s}\text{/}$ to recapture the
atoms and prevent collisions with the fiber wall. During each period where the
FORT is off, the atoms are first prepared in the ground state $\ket{1}$ by
optical pumping with the SFWM control beam and an additional orthogonally-
polarized preparation beam on the D2 line to address all magnetic sublevels of
ground state $\ket{2}$. Note that we do not prepare the atoms in a single
Zeeman state. Subsequently, while keeping the control beam on, we
simultaneously apply the pump beam to drive the SFWM process for a duration of
$1.2\text{\,}\mathrm{\SIUnitSymbolMicro s}\text{/}$. We acquire data from the
SPCMs only during this SFWM phase. Due to the finite temperature of the atomic
ensemble, the periodic switching of the FORT leads to a non-negligible loss of
atoms over time. To characterize these changing experimental conditions, we
measured time-resolved absorption spectra by probing the AS transition with a
weak laser pulse. We observed a peak OD of $155$ on the
$\ket{1}\leftrightarrow\ket{3}$ transition that decays exponentially with a
time constant of $3.2\text{\,}\mathrm{ms}\text{/}$. Therefore, we choose to
repeat the experimental window $3500$ times for a total duration of
$12\text{\,}\mathrm{ms}\text{/}$, to include the full range of ODs available
to us. After this time, basically all atoms are lost and we start with a new
MOT loading cycle. Because the OD is changing over two orders of magnitude
during the measurements, we also modulate the control power in a similar way
to maintain similar EIT conditions over a larger range of measurement windows.
We can then post-select the range of experimental windows (or ODs) in the
evaluation. Narrow ranges lead to sharply defined experimental conditions,
while wider ranges increase the detection rates at the cost of averaging over
changing conditions. The duty cycle, i.e., the ratio of SFWM data acquisition
phases to the total duration of the experiment depends on the post-selected
parameter ranges and is typically around 1:2000. The generation rates given
throughout this paper are corrected for the respective duty cycle.

##  III Experimental Results

By implementing the experiment as described in the previous section, we
generate correlated S-AS photon pairs for a wide range of the experimentally
controllable parameters OD and pump power. This allows for tuning the spectral
brightness and cross-correlation of our photon-pair source, which we will
discuss later in more detail.

Figure 2: (a) Exemplary temporal biphoton waveform. Cross-correlation
$g^{(2)}_{S,AS}$ as a function of the delay $\tau$ between AS and S detection
events. The error bars depict the propagated Poissonian error of the photon
counts. The red dotted line indicates the upper limit of the classical regime
$g^{(2)}_{S,AS}$$=2$. We apply a pump power of
$P=$14\text{\,}\mathrm{nW}\text{/}$$ and a control power of
$17\text{\,}\mathrm{nW}\text{/}$ corresponding to a Rabi frequencies of
$\Omega_{P}=$3.1\text{\,}\Gamma_{\mathrm{D}1}$$ and
$\Omega_{C}=$2.8\text{\,}\Gamma_{\mathrm{D}1}$$. We include measurements in
the range of $OD=40-80$ to obtain good statistics within an effective (i.e.,
duty-cycle corrected) integration time of $3.5\text{\,}\mathrm{s}\text{/}$.
The bin size is $4\text{\,}\mathrm{ns}\text{/}$. The solid line shows the
results of a numerical simulation of equation (2) using parameters $OD={15}$,
$\Omega_{C}=$2.8\text{\,}\Gamma_{\mathrm{D}1}$$ and
$P=$14\text{\,}\mathrm{nW}\text{/}$$ and is vertically scaled to fit the
datapoints. A fixed factor of $g^{(2)}_{S,AS}$$=1$ is added to the simulation
to account for the uncorrelated background. The inset shows the normalized
biphoton spectrum extracted from this simulation. (b) Measured heralded auto-
correlation $g^{(2)}_{S,S|AS}$ of the S photon conditioned on the detection of
an AS photon. Dataset is the same as in (a). See the main text for details on
the experimental method.

First, however, we discuss the properties of the observed photon pairs at
fixed experimental conditions yielding a good signal-to-noise ratio at a GSBP
of about
$300\text{\,}\mathrm{pairs}\text{/}\text{(}\mathrm{s}\text{⋅}\mathrm{MHz}\text{⋅}\mathrm{nW}\text{)}$
(see discussion of figure 3) which is comparable to the record GSBP reported
in Steiner  _et al._ (2021). Figure 2 (a) shows exemplary data (blue circles)
of the cross-correlation $g^{(2)}_{S,AS}(\tau)$ of the photon pairs as a
function of the time delay $\tau$ between S and subsequent AS detection
events. We obtained the data by normalizing the coincidences to the background
of accidental coincidences. We acquired this background by correlating events
from different measurement windows, hence removing any physical correlation
from the data. Clearly, the cross-correlation exceeds the classical limit of
$g^{(2)}_{S,AS}(\tau)=2$ (red dotted line) for a range of delays. The small
damped oscillations visible in the waveform indicate that we work in the
transition region between the damped Rabi oscillation and the group delay
regimes, as defined in Du  _et al._ (2008); Kolchin (2007). We verified
experimentally that the individual S and AS fields exhibit thermal statistics,
i.e., $g^{(2)}_{S,S}(0)=g^{(2)}_{AS,AS}(0)\approx 2$. To obtain sufficiently
good statistics for these measurements within a feasible integration time, we
had to raise the generation rate by increasing the pump power to
$\gtrsim$100\text{\,}\mathrm{nW}\text{/}$$. To quantify the violation of the
Cauchy-Schwarz inequality Clauser (1974) we calculated
$\mathcal{R}=({g^{(2)}_{S,AS}})^{2}/(g^{(2)}_{S,S}\,g^{(2)}_{AS,AS})$. The
peak value is $\mathcal{R}=$97(24)$$, which clearly violates the classical
limit of $\mathcal{R}\leq 1$ by 4 standard deviations and demonstrates the
nonclassical nature of the photon pairs. To obtain the bandwidth of the
photon-pair source, we compare the experimental data shown in figure 2(a) to a
simulated waveform using the theory presented in Du  _et al._ (2008). The
relevant equations used are summarized in appendix A. First, we evaluated
equation (2) starting with parameters determined from other measurements and
subsequently optimized these parameters for the best agreement between
simulation and experimental data. The features of the experimental data (blue
circles) can be reproduced by the simulation (orange solid line). The
discrepancy between the parameter sets of experiment and simulation might be
explained by deviations of our experiment from the assumptions made in Du  _et
al._ (2008). There, homogeneous atomic density and pump/control intensities
are assumed, which is clearly not the case inside the HCPBGF. Moreover, the
simulation considers only population of a single Zeeman level, whereas in our
case the population is initially distributed among the Zeeman levels of
$\ket{F=1}$. We use the simulated waveform to extract the biphoton spectrum as
shown in the inset of figure 2(a). Here, $\delta$ is the single-photon
detuning of the generated AS photon frequency components from their central
frequency $\omega_{AS}$. The spectral bandwidth (FWHM) is
$2\pi\times$6.5\text{\,}\mathrm{MHz}\text{/}$\approx$1.1\text{\,}\Gamma_{\mathrm{D}1}$$
demonstrating compatibility of our source with other rubidium-based
experiments. The characteristic timescale of the biphotons is thus
$24\text{\,}\mathrm{ns}\text{/}$.

We now turn to interpreting the photon-pair source as a heralded single-photon
source. Instead of using the S for heralding the AS detection events, as it is
typically done, we use the AS to herald the S events for the following reason:
As the Raman gain is not negligible compared to the FWM gain for our current
parameters, uncorrelated photons can be generated which act as additional
noise. This process is mainly relevant for the S channel as the pump is
coupled to the populated state $\ket{1}$, whereas the control is coupled to
the unpopulated state $\ket{2}$. Using the AS detection events as heralding
events for the creation of single photons in the S channel, we evade the
influence of Raman noise on the heralding itself. As the AS photon is delayed
by the slow-light effect of EIT Fleischhauer  _et al._ (2005) and thus exits
the medium after the S photon, choosing the AS as heralding photon is somewhat
counter-intuitive. If necessary, however, the temporal order of S and AS
detection events could be easily changed by sending the S photons through a
fiber optical delay line. In figure 2(b) we plot the auto-correlation of the S
photons conditioned on the detection of an AS photon, $g^{(2)}_{S,S|AS}$. To
obtain data of sufficient quality within a reasonable acquisition time, we use
the following method Fasel  _et al._ (2004); Seri  _et al._ (2018): Any
heralded S1(S2) detection event (see figure 1) is used as a start trigger. For
any subsequent S2(S1) detection event, we record the number of additional AS
events $n$ between the two S events. The number of S1-S2(S2-S1) pairs with $n$
AS events in between gives the amplitude of the $n$-th ($-n$-th) bin in the
resulting histogram. The zero bin thus corresponds to more than one S photon
being heralded by the same AS photon. From this data we can now determine
$g^{(2)}_{S,S|AS}$$(0)$ as the amplitude of the zero bin normalized to the
expected value that we get from a linear fit (red dashed line) to the
remaining bins Fasel  _et al._ (2004). We observe anti-bunching of the
heralded S photons with $g^{(2)}_{S,S|AS}$$(0)=$0.08(1)$$, clearly violating
the classical bound of $g^{(2)}_{S,S|AS}$$(0)\geq 1$ Grangier  _et al._ (1986)
and confirming a non-zero projection onto the single-photon Fock state for
$g^{(2)}_{S,S|AS}$$(0)\leq 0.5$ Grünwald (2019). We calculate the heralding
efficiency as the ratio of the pair generation rate and the heralding (AS)
rate corrected for transmission losses and find a value of
$43\text{\,}\mathrm{\char 37\relax}\text{/}$. The optimal mode matching in the
HCPBGF facilitates such efficient heralding. Similar values were reported
using a nanofiber Corzo  _et al._ (2019).

Figure 3: Photon pair properties versus pump power. The experimental
parameters are $OD=40-80$ and the control Rabi frequency
$\Omega_{C}=$2.8\text{\,}\Gamma_{\mathrm{D}1}$$. The effective integration
times range from
$0.6\text{\,}\mathrm{s}\text{/}3.5\text{\,}\mathrm{s}\text{/}$. (a) Measured
peak cross-correlation $g^{(2)}_{S,AS}$ (blue circles). The data point
indicated by an arrow corresponds to the measurement in figure 2. The dotted
red line visualizes the classical boundary set by the Cauchy-Schwarz
inequality. The dashed blue line is a numerical model including noise
contributions. The parameters are the same as for the model shown in figure
4(c). (b) GSB (left axis, blue circles) and GSBP (right axis, orange
triangles). The dashed blue line is a linear fit to the brightness in the low
pump power regime. The dashed orange line corresponds to the same fit
additionally normalized to the pump power. The error bars include the
Poissonian error of the photon counts and a $10\text{\,}\mathrm{\char
37\relax}\text{/}$ variation in the total detection efficiencies.

Next, we analyze the performance of our biphoton source when varying the pump
power $P$ over three orders of magnitude. Figure 3(a) shows the measured peak
cross-correlation $g^{(2)}_{S,AS}$ (blue circles) as a function of $P$. With
increasing pump power the cross-correlation, i.e., the purity of the photon
pairs, reduces following the expected $1/P$ dependency. Nevertheless, we
observe nonclassical correlations over the full range of pump powers. We use
the model described in appendix B to calculate the peak cross-correlation that
includes the measured unconditional as well as pump power-dependent optical
noise in the S and AS channel (dashed line). The model is described in more
detail in the discussion of figure 4(c). The same parameter set determined in
figure 4(c) is also used for the simulation shown in figure 3(a) without any
additional fitting parameter. We use a linear dependence between the generated
brightness and the pump power as shown in 3(b) (see next paragraph).
Experiment and theory agree well over the whole range of pump powers.

In figure 3(b) we show the corresponding GSB (blue circles, left hand side
axis). We obtain this rate from the detected rate by correcting it for optical
background, transmission losses, detection efficiencies, and the duty cycle of
the experiment. The dashed blue line is a linear fit of type
$\textrm{GSB}=\textrm{GSBP}\cdot P$. The fit confirms the expected linear
dependence for pump powers up to $200\text{\,}\mathrm{nW}\text{/}$. For higher
pump powers, the spectral generation rate increases slower. We suspect that in
this regime population redistribution due to optical pumping is no longer
negligible. When we normalize the spectral generation rate with regard to the
pump power, we obtain the GSBP. The experimental values (orange triangles,
right hand side axis) range from
$200\text{\,}\text{/}\text{(}\mathrm{s}\text{⋅}\mathrm{MHz}\text{⋅}\mathrm{nW}\text{)}410\text{\,}\text{/}\text{(}\mathrm{s}\text{⋅}\mathrm{MHz}\text{⋅}\mathrm{nW}\text{)}$.
The orange dashed line represents the fitted value
$\textrm{GSBP}=$312(24)\text{\,}\mathrm{pairs}\text{/}\text{(}\mathrm{s}\text{⋅}\mathrm{MHz}\text{⋅}\mathrm{nW}\text{)}$\approx$3\text{\times}{10}^{8}\text{\,}\mathrm{pairs}\text{/}\text{(}\mathrm{s}\text{⋅}\mathrm{MHz}\text{⋅}\mathrm{mW}\text{)}$$.
In this intermediate parameter regime the GSBP of our source is comparable to
the highest reported value of
$2\text{\times}{10}^{8}\text{\,}\mathrm{pairs}\text{/}\text{(}\mathrm{s}\text{⋅}\mathrm{MHz}\text{⋅}\mathrm{mW}\text{)}$
using a waveguide coupled to an on-chip microring cavity Steiner  _et al._
(2021), however, at a 10-fold reduced bandwidth and a 100-fold reduced pump
power. This very efficient conversion of pump power into narrowband photon
pairs is enabled by the intrinsically large overlap between the cold atomic
ensemble and the light fields in the HCPBGF as well as the optimal mode
matching between the four involved fields due to all of them being guided in a
single optical mode.

Figure 4: (a) Peak cross-correlation and (b) GSBP vs. pump power and OD. The
control Rabi frequency for $OD\gtrsim 55$ is
$2.8\text{\,}\Gamma_{\mathrm{D}1}$, for smaller ODs it is continuously reduced
to maintain a fixed width of the EIT transmission spectrum. The blue (green)
box marks the data used in figure 2 (figure 3). The open-end color scales
visualize the attainable values due to the limited measurement time. (c) GSB
vs. cross-correlation $g^{(2)}_{S,AS}$. The data is taken from (a) and (b).
The classical regime of $g^{(2)}_{S,AS}$$\leq 2$ is marked by the red shaded
area. The blue-shaded area marks the GBS where subsequent photon pairs overlap
in time (see main text). The gradient illustrates the transition into this
regime. The error bars of the data points correspond to the propagated
statistical error of the photon counts. For the grey data points the
saturation of the SPCMs due to high detection rates can no longer be
neglected. The solid orange line is a single-mode model described by equation
(1). The shaded orange area visualizes the influence of pump-dependent noise
as explained in the main text.

Next, we investigate the dependence of the peak cross-correlation
$g^{(2)}_{S,AS}$ and the GSBP on the pump power as well as the OD. The results
are shown in figure 4. We note that our system allows for an easy tuning of
the OD over an order of magnitude. This, combined with a tuning of the pump
power results in a tunability of the GSBP of our biphoton source by two orders
of magnitude. The regimes of high generation rate but low-quality photon pairs
(upper right corner at high OD and high pump power) and low generation rate
with high-quality photon pairs (lower left corner at low OD and low pump
power) can be clearly identified in figure 4(a) and (b). The parameter range
of the data presented in figures 2 and 3 lies in between these two regimes and
is marked by the blue and green rectangles. The highest values for
$g^{(2)}_{S,AS}$ are observed in the low OD and low pump power regime. When we
restrict ourselves to parameters with $g^{(2)}_{S,AS}$$\geq 3$, the highest
observed GSBP is
$2\text{\times}{10}^{9}\text{\,}\mathrm{pairs}\text{/}\text{(}\mathrm{s}\text{⋅}\mathrm{MHz}\text{⋅}\mathrm{mW}\text{)}$.
For values $g^{(2)}_{S,AS}$$\gtrsim 50$, the relative uncertainties become
larger than $30\text{\,}\mathrm{\char 37\relax}\text{/}$, due to the low
number of counts during the finite measurement duration. This measurement
demonstrates the versatile nature of our photon-pair source. Depending on the
application requirements, the parameter regime can be easily adjusted.

Finally, we investigate the limit of the GSB achieved with our system. To this
end, we plot in figure 4(c) the GSB vs. the peak cross-correlation
$g^{(2)}_{S,AS}$. The data are taken from figures (a) and (b). Note that the
grey datapoints correspond to a count rate where the saturation of the SPCMs
can not be neglected anymore. As we can see, for higher spectral brightnesses
the peak cross-correlation decreases and reaches a lower limit near two. We
fitted the data using the model described in appendix B that is based on a
single-mode description and thermal statistics. We start from equation (18)
and write the pair number as $n=\alpha\cdot\mathrm{GSB}$ with a scaling factor
$\alpha$. The orange line thus corresponds to a fitted function of the form

$g^{(2)}_{\text{S,AS}}=\frac{2+\frac{1}{\alpha\mathrm{GSB}}+\frac{\mathcal{N}_{S}\mathcal{N}_{AS}}{T_{S}T_{AS}(\alpha\mathrm{GSB})^{2}}}{\left[1+\frac{\mathcal{N}_{S}}{T_{S}\alpha\mathrm{GSB}}\right]\left[1+\frac{\mathcal{N}_{AS}}{T_{AS}\alpha\mathrm{GSB}}\right]}$
(1)

with noise contributions $\mathcal{N}_{j}$ and detection efficiencies $T_{j}$,
where we used measured values as described in appendix B. The scaling factor
is the only free parameter for our fit and was determined to be $\alpha=103$.
As the dataset includes points obtained with different pump powers, the pump-
dependent noise contributions also vary. This is visualized by the orange-
shaded area. Nonclassical correlations with $g^{(2)}_{S,AS}$$>3$ are
maintained up to a GSB of around
$2\text{\times}{10}^{5}\text{\,}\mathrm{pairs}\text{/}\text{(}\mathrm{s}\text{⋅}\mathrm{MHz}\text{)}$.
Beyond this brightness we approach the ultimate obtainable limit of
$\mathrm{GSB}=\pi/2\times${10}^{6}\text{\,}\mathrm{pairs}\text{/}\text{(}\mathrm{s}\text{⋅}\mathrm{MHz}\text{)}$$,
where successive photon pairs start to overlap in time Chen  _et al._ (2022).
The transition from temporally separated to overlapping pairs is visualized by
the blue color gradient in figure 4(c). In this range, the peak value of
$g^{(2)}_{S,AS}\to 2$ indicates that thermal statistics apply while the S and
AS fields are still correlated and can be described by a single mode. A
detailed discussion of this regime is beyond the scope of this work and will
be addressed in future work. Nonetheless, our data clearly demonstrates that
our biphoton source can be operated near the ultimate limit of generated
spectral brightness with a high tunability over two orders of magnitude,
narrow bandwidth, and at ultralow pump powers.

##  IV Conclusion and Outlook

We demonstrated the first nonclassical photon-pair source based on SFWM in an
ensemble of cold Rubidium atoms interfaced with a HCPBGF. The strong
confinement of atoms and light fields within the HCPBGF leads to enhanced
optical nonlinearities as compared to free space setups. It results in a GSBP
of up to
$2\text{\times}{10}^{9}\text{\,}\mathrm{pairs}\text{/}\text{(}\mathrm{s}\text{⋅}\mathrm{MHz}\text{⋅}\mathrm{mW}\text{)}$
at pump powers in the regime up to $100\text{\,}\mathrm{nW}\text{/}$ and for a
cross-correlation $\geq 3$. We determined the biphoton linewidth as
$2\pi\times$$6.5\text{\,}\mathrm{MHz}\text{/}$ which is thus compatible with
atomic quantum memories and similar to the linewidth achievable with free-
space setups. This represents a 10-fold increase of the GSBP compared to the
previous record using a microring cavity, at a 10-fold reduced bandwidth and a
100-fold reduced pump power. Furthermore, the photon pairs exhibit orthogonal
polarizations, thereby allowing for a relatively simple separation without the
need for elaborate filtering and separation techniques Caltzidis  _et al._
(2021). We verified the nonclassical nature of the photon pairs by measuring
their cross-correlation with a violation of the Cauchy-Schwarz inequality by a
factor of $97(24)$ as well as a heralded auto-correlation $\ll 0.5$. Moreover,
by increasing the generated spectral brightness even further, we reached the
regime where different photon pairs start to overlap in time. Here, we
demonstrated that the cross-correlation approaches a limit corresponding to
thermal statistics, i.e., S and AS photons exhibit bunching. Our hollow-core
fiber-based biphoton source therefore combines the advantages of free-space
and waveguide photon sources.

Examples of possible applications include easy integration into photonic
networks Kimble (2008) based on atomic ensembles Sangouard  _et al._ (2011)
due to the narrowband emission into a single transverse mode, employment in
quantum key distribution satellite links Bedington  _et al._ (2017) due to
suitable transmission band wavelengths of $780\text{\,}\mathrm{nm}\text{/}$
and $795\text{\,}\mathrm{nm}\text{/}$, and as light source for photonic
quantum metrology Polino  _et al._ (2020). Additionally, the ultralow pump
power required to generate biphotons combined with integrating the HCPBGF with
an atom chip Keil  _et al._ (2016) holds promise for a significant
miniaturization of the setup. Finally, the high GSB could be applied, e.g., to
increase the brightness of temporally-multiplexed photon sources Meyer-Scott
_et al._ (2020).

To further improve and extend our HCPBGF-based biphoton source one could
envision the following options: First, using a ladder-type scheme as in Lee
_et al._ (2016); Davidson  _et al._ (2021) the currently dominating Raman
noise could be avoided. Second, replacing our currently pulsed loading of the
HCPBGF by a continuous loading scheme, the duty cycle could be increased
substantially. Combined with in-fiber magic-wavelength trapping Hilton  _et
al._ (2019); Yoon and Bajcsy (2019) photon pairs could be generated
continuously without the need to periodically modulate the guiding potential.
We estimate that this could enhance the detected pair rates by up to three
orders of magnitude. Third, by implementing intra-fiber laser cooling Wang
_et al._ (2022) the ensemble temperature could be reduced thereby prolonging
the coherence time, i.e., further reducing the bandwidth. Moreover, we note
that as the generation of biphotons only requires atoms loaded into instead of
through the fiber (note that in our case only the first 6 cm of the 14 cm long
fiber are filled), the output side of the HCPBGF could be spliced directly to
a (polarization-maintaining) fiber Thapa  _et al._ (2006); Kristensen  _et
al._ (2008), thereby enabling direct connection to a photonic network,
possibly after wavelength conversion to the telecom band in a waveguide
Albrecht  _et al._ (2014). Finally, a detailed investigation of the photon
statistics for the highest spectral brightness, where successive photon pairs
overlap in time, may permit the generation of multi-photon states.

## Acknowledgments

The authors thank the group of T. Walther for providing us with their home-
built ultra-low noise laser diode driver with high modulation bandwidth. The
project received funding from the European Union’s Horizon 2020 research and
innovation programme under the Marie Skłodowska-Curie grant agreement No.
765075, as well as by the Deutsche Forschungsgemeinschaft (DFG, German
Research Foundation) under project number 410249930.

##  Appendix A Theoretical treatment of SFWM

###  A.1 Biphoton waveform

We here summarize the basic theory to simulate the biphoton waveform and
spectrum based on the work of Du et al. Du  _et al._ (2008). We focus only on
the relevant equations that enable comparison with our experimental results in
figure 2.

The double $\Lambda$-type system is introduced in figure 1(a) of the main
text. We additionally define the single-photon detuning of the AS frequency
components $\omega$ as $\delta=\omega-\omega_{AS}$. Due to the collinear setup
the phase matching condition, written in terms of the wavenumbers $k_{i}$,
reads $\Delta k=k_{AS}+k_{S}-k_{C}-k_{P}=0$. We assume that all atoms are
initially prepared in ground state $\ket{1}$.

We are interested in temporal correlations of the created photon pairs as a
function of the time delay $\tau$ between the paired $S$ and $AS$ photons.
They are determined by the relative biphoton wavefunction

$|\psi(\tau)|^{2}=\left|\frac{L}{2\pi}\int\kappa(\delta)\Phi(\delta)\text{e}^{-\text{i}\delta\tau}\mathrm{d}\delta\right|^{2}$
(2)

with the longitudinal detuning function

$\Phi(\delta)=\text{sinc}\left(\frac{\Delta
kL}{2}\right)\text{e}^{\text{i}(k_{AS}+k_{S})L/2}$ (3)

and the nonlinear parametric coupling coefficient

$\kappa(\delta)=-\text{i}\frac{\sqrt{\omega_{AS}\omega_{S}}}{2c}\chi^{(3)}(\delta)E_{P}E_{C},$
(4)

where $E_{i}$ are the electric fields of the respective transitions. The
third-order nonlinear susceptibility of the AS field is given by

$\displaystyle\chi^{(3)}(\delta)$
$\displaystyle=\frac{\mathcal{N}\mu_{13}\mu_{32}\mu_{24}\mu_{41}/(\epsilon_{0}\hbar^{3})}{(\Delta+\text{i}\frac{\Gamma_{\mathrm{D}2}}{2})\left[|\Omega_{C}|^{2}-4(\delta+\text{i}\frac{\Gamma_{\mathrm{D}1}}{2})(\delta+\text{i}\gamma_{12})\right].}$
(5)

We introduced here the transition dipole matrix elements $\mu_{ij}$, the
atomic density $\mathcal{N}=N/(L\pi(d/2)^{2})$ and the control Rabi frequency
$\Omega_{C}=\mu_{23}E_{C}/\hbar$.

To evaluate the detuning function $\Phi(\delta)$, we calculate the complex
wave numbers of the created fields as
$k_{i}(\delta)=\frac{\omega_{i}}{c}\sqrt{1+\chi_{i}(\delta)}$ with the linear
susceptibilities given by

$\displaystyle\chi_{S}(\delta)$
$\displaystyle=\frac{\mathcal{N}|\mu_{24}|^{2}(\delta-\text{i}\frac{\Gamma_{\mathrm{D}1}}{2})/(\epsilon_{0}\hbar)}{|\Omega_{C}|^{2}-4(\delta-\text{i}\frac{\Gamma_{\mathrm{D}1}}{2})(\delta-\text{i}\gamma_{12})}\times\frac{|\Omega_{P}|^{2}}{\Delta^{2}+\left(\frac{\Gamma_{\mathrm{D}2}}{2}\right)^{2}},$
(6) $\displaystyle\chi_{AS}(\delta)$
$\displaystyle=\frac{4\mathcal{N}|\mu_{13}|^{2}(\delta+\text{i}\gamma_{12})/(\epsilon_{0}\hbar)}{|\Omega_{C}|^{2}-4(\delta+\text{i}\frac{\Gamma_{\mathrm{D}1}}{2})(\delta+\text{i}\gamma_{12})}.$
(7)

Here, $\Omega_{P}=\mu_{14}E_{P}/\hbar$ is the pump Rabi frequency.

Using these equations allows for calculating the expected waveform
$|\psi(\tau)|^{2}$ and the biphoton spectrum
$|\kappa(\delta)\Phi(\delta)|^{2}$.

###  A.2 Polarization-resolved spatio-temporal simulation of SFWM

In this section we describe the procedure for numerical simulation of the SFWM
process taking into account a total of 16 magnetic sublevels of the involved
hyperfine states as well as light field polarizations. This simulation served
to confirm the polarizations of the generated S and AS fields for varying
polarization configurations ($\parallel$ and $\perp$) of the pump and control
fields. This was necessary as we are currently unable to prepare the
population in a single Zeeman state and use arbitrary polarizations
configurations as in free-space setups due to the HCPBGFs birefringence, which
requires the use of linear polarizations.

We assume classical fields, use a plane-wave approximation and describe the
atom-field interactions using a density matrix approach in rotating wave
approximation. We simulate the process of SFWM by solving a system of partial
differential equations which represent light propagation and time evolution of
the density matrix operator $\hat{\rho}$ with appropriate random initial
conditions.

Denoting the set of magnetic sublevels of the ground states $\ket{1}$ and
$\ket{2}$ by the index $g$ and a similar set of magnetic sublevels of the
excited states by the index $e$ we can write the density matrix equations in
the short matrix form. In this notation all quantum operators are spelled as
block matrices:

$\hat{O}=\begin{bmatrix}O_{gg}&O_{ge}\\\ O_{eg}&O_{ee}\\\ \end{bmatrix}.$ (8)

Then the propagation equations read

$\frac{\partial E_{m}(z,t)}{\partial z}=-4\pi
ik_{m}\mathcal{N}\operatorname{tr}[\hat{\mu}^{m}_{ge}\cdot\rho_{eg}(z,t)].$
(9)

Here $E_{m}$ is the complex amplitude of the electric field in mode $m$,
$k_{m}$ is its wavenumber, and $\hat{\mu}^{m}$ is the mode’s dipole moment
operator where the matrix elements are taken from Steck (2021). The mode index
$m$ runs across all fields we take into account (pump, control, Stokes, and
anti-Stokes), as well as their respective polarizations, where we use
$\sigma^{\pm}$ as a polarization basis.

The time evolution matrix equations for the density matrix read (here and
below we assume $\Gamma_{\mathrm{D1}}\simeq\Gamma_{\mathrm{D2}}\simeq\Gamma$)

$\displaystyle\frac{\partial\rho_{gg}}{\partial t}$
$\displaystyle=i[\rho_{gg},\Delta_{g}]+\frac{i}{2}(\rho_{ge}\cdot\Omega_{eg}-\Omega_{ge}\cdot\rho_{eg})+\Gamma\,R_{g}\circ
C_{ge}\cdot\rho_{ee}\cdot C_{eg}-\gamma\circ\rho_{gg},$ (10)
$\displaystyle\frac{\partial\rho_{eg}}{\partial t}$
$\displaystyle=i(\rho_{eg}\cdot\Delta_{g}-\Delta_{e}\cdot\rho_{eg})+\frac{i}{2}(\rho_{ee}\cdot\Omega_{eg}-\Omega_{eg}\cdot\rho_{gg})-\frac{\Gamma}{2}(C_{eg}\cdot
C_{ge})\cdot\rho_{eg},$ $\displaystyle\frac{\partial\rho_{ee}}{\partial t}$
$\displaystyle=i[\rho_{ee},\Delta_{e}]+\frac{i}{2}(\rho_{eg}\cdot\Omega_{ge}-\Omega_{eg}\cdot\rho_{ge})-\frac{\Gamma}{2}\\{C_{eg}\cdot
C_{ge},\rho_{ee}\\}.$

Here, $\Delta_{g}$ and $\Delta_{e}$ are diagonal matrices with the summed
detunings of the considered modes on the diagonal.
$\Omega_{ge}=\Omega_{eg}^{\dagger}$ are submatrices of the Rabi frequency
operator defined by

$\hbar\hat{\Omega}=\sum_{m}E_{m}\hat{\mu}^{m}.$ (11)

The matrices $C_{ge}=C_{eg}^{\dagger}$ contain the Clebsch-Gordan coefficients
as follows:

$\Bra{\begin{matrix}F_{e}\\\ M_{e}\\\
\end{matrix}}C_{eg}\Ket{\begin{matrix}F_{g}\\\ M_{g}\\\
\end{matrix}}=\braket{F_{e},M_{e}}{F_{g},M_{g};1,M_{e}-M_{g}}.$ (12)

$R_{g}$ is the Kronecker delta of the $F$ quantum number:

$\Bra{\begin{matrix}F_{1}\\\ M_{1}\\\
\end{matrix}}R_{g}\Ket{\begin{matrix}F_{2}\\\ M_{2}\\\
\end{matrix}}=\delta_{F_{1}F_{2}}$ (13)

and the operation $\circ$ stands for the element-wise multiplication of
matrices. This way, $R_{g}$ prevents coherence between $\ket{1}$ and $\ket{2}$
being generated when population is transferred from the excited to the ground
levels.

The matrix $\gamma$ describes decoherence effects in the ground states:

$\Bra{\begin{matrix}F_{1}\\\ M_{1}\\\
\end{matrix}}\gamma\Ket{\begin{matrix}F_{2}\\\ M_{2}\\\
\end{matrix}}=\begin{cases}0\text{ if }F_{1}=F_{2}\text{ and }M_{1}=M_{2},\\\
\gamma_{12}\text{ otherwise.}\\\ \end{cases}$ (14)

To imitate the vacuum fluctuations we set the Stokes field at the beginning of
the fiber to a random value with correlation time $1/\Gamma$ in the following
way:

$\displaystyle E_{S\pm}(z=0,t=0)$ $\displaystyle=0,$ (15) $\displaystyle
E_{S\pm}(z=0,t+\delta t)$
$\displaystyle=E_{S\pm}(z=0,t)e^{-\frac{\Gamma_{\mathrm{D2}}}{2}\,\delta t}$
$\displaystyle\quad+\nu_{\pm}E_{\mathrm{vacuum}}\sqrt{1-e^{-\Gamma_{\mathrm{D2}}\,\delta
t}},$

where $\delta t\ll 1/\Gamma$ is the time step, $\nu_{\pm}$ is a random complex
number from the normalized complex Gaussian distribution, and the field
$E_{\mathrm{vacuum}}$ corresponds to the Rabi frequency of $\sim
10^{-5}\Gamma$. The sign $\pm$ in the index points out that we use these
initial conditions both for the left and the right polarization components.

The time evolution equation (10) with initial conditions (15) were solved
numerically using the Kutta-Merson method. On each Kutta-Merson step we solved
the propagation equation (9) by integrating its right-hand side using
Simpson’s rule111L. V. Blake, A Modified Simpson’s Rule and Fortran Subroutine
for Cumulative Numerical Integration of a Function Defined by Data Points,
Tech. Rep. (Defense Technical Information Center, Fort Belvoir, VA, 1971).

The results of these simulations confirmed that the generated S and AS fields
exhibit orthogonal linear polarizations with respect to the injected pump and
control fields of also linear orthogonal polarizations. Therefore it was
experimentally possible to use polarization filtering (see figure 1(b)) in
addition to frequency filtering and obtain a sufficiently high extinction
ratio for the strong collinear pump and control fields.

##  Appendix B  Model for the detected cross-correlation

In this section we present a simple single-mode model Rohde and Ralph (2006)
that incorporates loss and detector inefficiencies for the fit of the cross-
correlation function $g^{(2)}_{S,AS}$. For that, we introduce the bosonic
annihilation operator

$\hat{b}_{j}=t_{j}\hat{a}_{j}+r_{j}\hat{\mu}_{j}$ (16)

of photons detected in channel $j={S,AS}$. Here, $\hat{a}_{j}$ denotes the
(bosonic) annihilation operator of Stokes or anti-Stokes photons generated
during SFWM and $\hat{\mu}_{j}$ the (bosonic) annihilation operator of noise
photons in channel $j$. Imperfect detection is introduced in equation (16)
though a beam splitter transformation with
$T_{j}+R_{j}=|t_{j}|^{2}+|r_{j}|^{2}=1$. Hence, $T_{j}$ is the efficiency of
the detector including all loss in channel $j$.

If we assume no correlation between the noise and the generated photons, i.
e., $\braket{\hat{a}_{j}^{\dagger}\hat{\mu}_{j}}=0$, we can connect the
detected photon numbers
$N_{j}=\braket{\hat{b}_{j}^{\dagger}\hat{b}_{j}}=T_{j}n_{j}+\mathcal{N}_{j}$
to the number of generated Stokes and anti-Stokes photons
$n_{j}=\braket{\hat{a}_{j}^{\dagger}\hat{a}_{j}}$. We furthermore introduced
the noise detected in channel $j$ through
$\mathcal{N}_{j}=R_{j}\braket{\hat{\mu}_{j}^{\dagger}\hat{\mu}_{j}}$.

The detected cross-correlation function
$g^{(2)}_{S,AS}=\braket{\hat{b}_{S}^{\dagger}\hat{b}_{AS}^{\dagger}\hat{b}_{S}\hat{b}_{AS}}/(N_{S}N_{AS})$
takes the form

$g^{(2)}_{S,AS}=\frac{\braket{\hat{a}_{S}^{\dagger}\hat{a}_{AS}^{\dagger}\hat{a}_{S}\hat{a}_{AS}}}{n_{S}n_{AS}}\frac{T_{S}n_{S}T_{AS}n_{AS}}{N_{S}N_{AS}}+\frac{\mathcal{N}_{S}\mathcal{N}_{AS}}{N_{S}N_{AS}}.$
(17)

Here, we have again assumed no correlation between the noise and the generated
photons, i. e.,
$\braket{\hat{a}_{S}^{\dagger}\hat{a}_{AS}^{\dagger}\hat{\mu}_{S}\hat{\mu}_{AS}}=0$
and uncorrelated noise in both channels, i. e.,
$R_{S}R_{AS}\braket{\hat{\mu}_{S}^{\dagger}\hat{\mu}_{AS}^{\dagger}\hat{\mu}_{S}\hat{\mu}_{AS}}=\mathcal{N}_{S}\mathcal{N}_{AS}$.

If there is perfect correlation between Stokes and anti-Stokes fields, the
number of photons generated in each channel corresponds to the number of
generated pairs $n=n_{S}=n_{AS}$. We have verified that both the Stokes and
anti-Stokes fields exhibit close to thermal statistics, which directly implies
$\braket{\hat{a}_{S}^{\dagger}\hat{a}_{AS}^{\dagger}\hat{a}_{S}\hat{a}_{AS}}/(n_{S}n_{AS})=2+1/n$
for perfectly correlated fields. In this case, the cross-correlation function
takes the form

$g^{(2)}_{\text{S,AS}}=\frac{2+\frac{1}{n}+\frac{\mathcal{N}_{S}\mathcal{N}_{AS}}{T_{S}T_{AS}n^{2}}}{\left[1+\frac{\mathcal{N}_{S}}{T_{S}n}\right]\left[1+\frac{\mathcal{N}_{AS}}{T_{AS}n}\right]}$
(18)

and reduces to the ideal case for negligible noise. For dominant noise,
however, the cross-correlation approaches the limit of $1+1/(n+\mathcal{N})$.
Note that in a multi-mode model the number $2$ has to be replaced by $1+1/M$,
where $M$ is the number of detected modes Ivanova  _et al._ (2006).

To determine the noise for the fits in main body of the paper, we make the
ansatz $\mathcal{N}_{j}=[r_{j}^{0}+r_{j}^{0}(P)+r_{j}(P)]\tau_{c}$, where
$r_{i}^{0}$ is the noise rate with no atoms loaded in the HCPBGF and no pump
beam present, i.e., optical noise from stray light, residual control light and
the dark counts of the SPCMs. The additional optical noise caused by the pump
beam itself, but still measured without atoms present in the HCPBGF, is
denoted by $r_{j}^{0}(P)$. The remaining contribution $r_{j}(P)$ accounts for
the noise originating from the cold atoms and it is dominated by Raman noise
and residual optical pumping, but is neglected in the remainder of our
analysis. We measured the contributions without atoms to be
$r_{S}^{0}=$2800\text{\,}\text{/}\mathrm{s}$$,
$r_{AS}^{0}=$1200\text{\,}\text{/}\mathrm{s}$$,
$r_{S}^{0}(P)=$84\text{\,}\text{/}\text{(}\mathrm{s}\text{⋅}\mathrm{nW}\text{)}$\cdot
P$ and
$r_{AS}^{0}(P)=$7.6\text{\,}\text{/}\text{(}\mathrm{s}\text{⋅}\mathrm{nW}\text{)}$\cdot
P$. The characteristic time scale $\tau_{c}=$24\text{\,}\mathrm{n}\mathrm{s}$$
is assumed to be of the order of the temporal duration of a biphoton, obtained
from the data in figure 2(a). Moreover, we use $T_{S}=T_{AS}=$0.08$$ in
accordance with the measured transmission losses and specified detector
efficiencies.

## References

  * Sangouard  _et al._ (2011) N. Sangouard, C. Simon, H. de Riedmatten, and N. Gisin, Reviews of Modern Physics 83, 33 (2011).
  * Slussarenko and Pryde (2019) S. Slussarenko and G. J. Pryde, Applied Physics Reviews 6, 041303 (2019).
  * Eisaman  _et al._ (2011) M. D. Eisaman, J. Fan, A. Migdall, and S. V. Polyakov, Review of Scientific Instruments 82, 071101 (2011).
  * Kimble (2008) H. J. Kimble, Nature 453, 1023 (2008).
  * Wang  _et al._ (2021) Y. Wang, K. D. Jöns, and Z. Sun, Applied Physics Reviews 8, 011314 (2021).
  * Luo  _et al._ (2015) K.-H. Luo, H. Herrmann, S. Krapick, B. Brecht, R. Ricken, V. Quiring, H. Suche, W. Sohler, and C. Silberhorn, New Journal of Physics 17, 073039 (2015).
  * Steiner  _et al._ (2021) T. J. Steiner, J. E. Castro, L. Chang, Q. Dang, W. Xie, J. Norman, J. E. Bowers, and G. Moody, PRX Quantum 2, 010337 (2021).
  * Tsai and Chen (2018) P.-J. Tsai and Y.-C. Chen, Quantum Science and Technology 3, 034005 (2018).
  * Chen  _et al._ (2022) J.-M. Chen, C.-Y. Hsu, W.-K. Huang, S.-S. Hsiao, F.-C. Huang, Y.-H. Chen, C.-S. Chuu, Y.-C. Chen, Y.-F. Chen, and I. A. Yu, Physical Review Research 4, 023132 (2022).
  * Cordier  _et al._ (2020) M. Cordier, P. Delaye, F. Gérôme, F. Benabid, and I. Zaquine, Scientific Reports 10, 1650 (2020).
  * Lopez-Huidobro  _et al._ (2021) S. Lopez-Huidobro, M. Lippl, N. Y. Joly, and M. V. Chekhova, Optics Letters 46, 4033 (2021).
  * Corzo  _et al._ (2019) N. V. Corzo, J. Raskop, A. Chandra, A. S. Sheremet, B. Gouraud, and J. Laurat, Nature 566, 359 (2019).
  * Bedington  _et al._ (2017) R. Bedington, J. M. Arrazola, and A. Ling, npj Quantum Information 3, 1 (2017).
  * Polino  _et al._ (2020) E. Polino, M. Valeri, N. Spagnolo, and F. Sciarrino, AVS Quantum Science 2, 024703 (2020).
  * Peters  _et al._ (2020) T. Peters, T.-P. Wang, A. Neumann, L. S. Simeonov, and T. Halfmann, Optics Express 28, 5340 (2020).
  * Steck (2021) D. A. Steck, “Rubidium 87 D Line Data,” http://steck.us/alkalidata (2021).
  * Fleischhauer  _et al._ (2005) M. Fleischhauer, A. Imamoglu, and J. P. Marangos, Reviews of Modern Physics 77, 633 (2005).
  * Balić  _et al._ (2005) V. Balić, D. A. Braje, P. Kolchin, G. Y. Yin, and S. E. Harris, Physical Review Letters 94, 183601 (2005).
  * Poletti  _et al._ (2013) F. Poletti, M. N. Petrovich, and D. J. Richardson, Nanophotonics 2, 315 (2013).
  * Blatt  _et al._ (2014) F. Blatt, T. Halfmann, and T. Peters, Optics Letters 39, 446 (2014).
  * Peters  _et al._ (2021) T. Peters, L. P. Yatsenko, and T. Halfmann, Physical Review A 103, 063302 (2021).
  * Palittapongarnpim  _et al._ (2012) P. Palittapongarnpim, A. MacRae, and A. I. Lvovsky, Review of Scientific Instruments 83, 066101 (2012).
  * Ahlrichs  _et al._ (2013) A. Ahlrichs, C. Berkemeier, B. Sprenger, and O. Benson, Applied Physics Letters 103, 241110 (2013).
  * Du  _et al._ (2008) S. Du, J. Wen, and M. H. Rubin, JOSA B 25, C98 (2008).
  * Kolchin (2007) P. Kolchin, Physical Review A 75, 033814 (2007).
  * Clauser (1974) J. F. Clauser, Physical Review D 9, 853 (1974).
  * Fasel  _et al._ (2004) S. Fasel, O. Alibart, S. Tanzilli, P. Baldi, A. Beveratos, N. Gisin, and H. Zbinden, New Journal of Physics 6, 163 (2004).
  * Seri  _et al._ (2018) A. Seri, G. Corrielli, D. Lago-Rivera, A. Lenhard, H. de Riedmatten, R. Osellame, and M. Mazzera, Optica 5, 934 (2018).
  * Grangier  _et al._ (1986) P. Grangier, G. Roger, and A. Aspect, Europhysics Letters (EPL) 1, 173 (1986).
  * Grünwald (2019) P. Grünwald, New Journal of Physics 21, 093003 (2019).
  * Caltzidis  _et al._ (2021) I. Caltzidis, H. Kübler, T. Pfau, R. Löw, and M. A. Zentile, Physical Review A 103, 043501 (2021).
  * Keil  _et al._ (2016) M. Keil, O. Amit, S. Zhou, D. Groswasser, Y. Japha, and R. Folman, Journal of Modern Optics 63, 1840 (2016).
  * Meyer-Scott  _et al._ (2020) E. Meyer-Scott, C. Silberhorn, and A. Migdall, Review of Scientific Instruments 91, 041101 (2020).
  * Lee  _et al._ (2016) Y.-S. Lee, S. M. Lee, H. Kim, and H. S. Moon, Optics Express 24, 28083 (2016).
  * Davidson  _et al._ (2021) O. Davidson, R. Finkelstein, E. Poem, and O. Firstenberg, New Journal of Physics 23, 073050 (2021).
  * Hilton  _et al._ (2019) A. Hilton, C. Perrella, A. Luiten, and P. Light, Physical Review Applied 11, 024065 (2019).
  * Yoon and Bajcsy (2019) T. Yoon and M. Bajcsy, Physical Review A 99, 023415 (2019).
  * Wang  _et al._ (2022) Y. Wang, S. Chai, T. Billotte, Z. Chen, M. Xin, W. S. Leong, F. Amrani, B. Debord, F. Benabid, and S.-Y. Lan, Physical Review Research 4, L022058 (2022).
  * Thapa  _et al._ (2006) R. Thapa, K. L. Corwin, and B. R. Washburn, in  _2006 Conference on Lasers and Electro-Optics and 2006 Quantum Electronics and Laser Science Conference_ (2006) pp. 1–2.
  * Kristensen  _et al._ (2008) J. T. Kristensen, A. Houmann, X. Liu, and D. Turchinovich, Optics Express 16, 9986 (2008).
  * Albrecht  _et al._ (2014) B. Albrecht, P. Farrera, X. Fernandez-Gonzalvo, M. Cristiani, and H. de Riedmatten, Nature Communications 5, 3376 (2014).
  * Rohde and Ralph (2006) P. P. Rohde and T. C. Ralph, Journal of Modern Optics 53, 1589 (2006).
  * Ivanova  _et al._ (2006) O. A. Ivanova, T. S. Iskhakov, A. N. Penin, and M. V. Chekhova, Quantum Electronics 36, 951 (2006).


aff1]Fermi National Accelerator Laboratory, Batavia, IL, USA
[cor1]Corresponding author<EMAIL_ADDRESS>
# Integrable RCS as a proposed replacement for Fermilab Booster

Jeffrey Eldred  Alexander Valishev  [

###### Abstract

Integrable optics is an innovation in particle accelerator design that
potentially enables a greater betatron tune spread and damps collective
instabilities. An integrable rapid-cycling synchrotron (RCS) would be an
effective replacement for the Fermilab Booster, as part of a plan to reach
multi-MW beam power at 120 GeV for the Fermilab high-energy neutrino program.
We provide an example integrable lattice with features of a modern RCS -
dispersion-free drifts, low momentum compaction factor, superperiodicity,
chromaticity correction, bounded beta functions, and separate-function
magnets.

##  1 INTRODUCTION

Integrable optics is a development in particle accelerator technology that
enables strong nonlinear focusing without generating parametric resonances
[1]. A promising application of integrable optics is in high-intensity rings,
where it is necessary to avoid resonances associated with a large betatron
tune-spread, while simultaneously suppressing collective instabilities with
Landau damping. The efficacy of an accelerator design incorporating integrable
optics will undergo comprehensive experimental tests at the Fermilab
Integrable Optics Test Accelerator (IOTA) [2] and the University of Maryland
Electron Ring (UMER) [3] over the next several years. In this paper, we
discuss a potential Fermilab integrable rapid-cycling synchotron (iRCS) as a
high-intensity replacement for the Fermilab Booster.

At Fermilab, a core research priority is to improve the proton beam power for
the flagship high-energy neutrino program [4]. In the current running
configuration, a 700 kW, 120 Gev proton beam is delivered to a carbon target
for the NuMI beamline that supports the NOvA, MINERvA, and MINOS neutrino
experiments. Next, the Proton Improvement Plan II (PIP-II) will replace the
400 MeV linac with a new 800 MeV linac that will increase the 120 GeV proton
power of the Fermilab complex to 1.2 MW [5].

The next flagship neutrino experiment at Fermilab will be the LBNF/DUNE [6].
The P5 Report referred to LBNF as “the highest priority project in its
lifetime,” and set a benchmark for a 3$\sigma$ measurement of the CP-violating
phase over 75% the range of its possible values [7]. The P5 benchmark for the
CP-violating phase corresponds to a 900 kt$\cdot$MW$\cdot$year neutrino
exposure requirement [6, 4]. For a proton power of 1.2 MW and a 50 kt LAr
detector, 15 years are required to meet that benchmark. For a proton power of
3.6 MW and a 36 kt LAr detector, 7 years are required.

In order to achieve a 120 GeV proton power significantly beyond the 1.2 MW
delivered by PIP-II, it will be necessary to replace the Fermilab Booster with
a modern RCS [4]. The Fermilab Booster is over 45 years old and faces
limitations from its magnets and its RF alike. There is no beampipe inside the
dipoles, and the magnet laminations generate an impedance instability. The
impedance instability provides $\sim$200 kV deceleration during transition
crossing at current beam intensities [8]. The Booster dipoles are combined
function magnets, which constrain tunability and amplify electron cloud
instabilities [9]. The Booster RF cavities underwent a refurbishment process
and cooling upgrade in order to achieve a 15-Hz Booster ramp rate [10], but
the ramp rate will not be able to exceed 20 Hz without replacing the Booster
RF entirely.

The Fermilab Booster intensity is also limited by betatron resonance losses
incurred by a large Laslett betatron tune-spread. The Laslett tune-spread is
given by

$\Delta\nu(z)\approx\frac{Nr_{0}}{2\pi\epsilon_{N}\beta\gamma^{2}}\left(\frac{\lambda(z)}{\langle\lambda\rangle_{z}}\right)F,$
(1)

where $N$ is the number of particles, $r_{0}$ is the classical electron
radius, $\epsilon_{N}$ is the normalized transverse emittance, $\lambda(z)$ is
the local charge density at position $z$, and $F$ is a transverse form factor
[4].

An integrable RCS could manage a high Laslett tune-spread without a costly
increase in aperture [11]. Detailed simulations of space-charge dominated
beams in integrable lattices is an ongoing work [2, 12], and the ultimate
limitation in Laslett tune-spread is not fully determined.

Table 1 shows how 120-GeV beam power corresponds to the Laslett tune-spread
the iRCS must support. For this exercise, we consider an iRCS with the same
aperture and circumference as the Fermilab Booster, that the iRCS has a 20-Hz
ramp rate, and that there is no slip-stacking in the Fermilab Recycler or Main
Injector.

Laslett Tune-spread | RCS | RCS Extracted | Main Injector
---|---|---|---
Requirement | Intensity | Power at 8 GeV | Power at 120 GeV
-0.27 | 16 e12 | 0.41 MW | 1.2 MW
-0.53 | 32 e12 | 0.82 MW | 2.3 MW
-0.80 | 48 e12 | 1.20 MW | 3.6 MW
Table 1: Beam power requirements for iRCS baseline scenario.

Slip-stacking is an accumulation process that doubles the intensity of the
Fermilab Main Injector, but the feasibility of slip-stacking beyond the PIP-II
era is still under investigation [13, 14]. If 20-Hz slip-stacking can be
sustained in the Main Injector or Recycler, the 120-GeV beam power would
increase by a factor of 1.5 or 2, respectively.

Figure 1 shows one of several siting options for an iRCS and it does not limit
the circumference. Excess circumference can raise the costs of the RF system
by increasing the revolution frequency [11]. The iRCS ramp rate indirectly
affects the 120 GeV beam power by reducing the Main Injector fill time. For a
60-Hz ramp rate, 120 GeV beam power increases by 15%.

Figure 1: Site location for the proposed iRCS, relative to the PIP-II linac,
muon campus, and Main Injector [15].

##  2 iRCS EXAMPLE LATTICE

The iRCS should incorporate the innovations in RCS design that have been
developed after the Fermilab Booster [11]. Periodicity and bounded beta
functions increase the dynamic aperture. Transition crossing can be avoided by
designing the lattice with a low momentum compaction factor. Modern RCS design
also uses separate-function dipole magnets and long dispersion-free drifts. In
this Section, we show an example integrable lattice with these features.

An accelerator can achieve integrable optics with alternating sections of
T-inserts and nonlinear magnets [1]. The T-inserts are arc sections with
$\pi$-integer betatron phase-advance in the horizontal and vertical plane. The
lattice should be dispersion-free in the nonlinear section, and the horizontal
and vertical beta functions should be matched. A special nonlinear elliptical
magnet is matched to the beta functions to provide the nonlinear focusing.

Figure 2 shows an example iRCS lattice and Table 2 shows the key parameters of
this lattice. The lattice is composed of 12 identical achromatic arcs and
dispersion-free drifts. Every other drift hosts a nonlinear insert, so the
lattice forms 6 periodic cells with a T-insert section and a nonlinear insert
section. The drifts in the center of each T-insert arc are used for injection,
extraction, and RF acceleration.

Figure 2: Twiss parameters for one of the six periodic cells. (top) Horizontal
and vertical beta functions shown in black and red, respectively. (middle)
Location and length of magnetic lattice, elements where dipoles are shown as
short blue rectangles, quadrupoles as tall orange rectangles, and sextupoles
as green rectangles. (bottom) Linear dispersion function.

Parameter | Value
---|---
Circumference | 486 m
Periodicity | 6 (12)
Vertical Aperture | 5 cm
Maximum Energy | 8 GeV
Bend Radius | 15.6 m
Peak Dipole Field | 1.25 T
Peak Quadrupole Field | 25 T/m
Peak Sextupole Field | 180 T/m2
Max Beta Function | 35 m
Max Dispersion | 0.8 m

Parameter | Value
---|---
Insertion Length / Cell | 8.1 m
Total Insertion Length | 97 m
Single Dipole Length | 2.8 m
Number of Dipoles | 48
Number of Quadrupoles | 156
Number of Sextupoles | 48
Momentum Compaction | 2$\times 10^{-3}$
Extraction Phase-Slip Factor | -6$\times 10^{-3}$
Betatron Tunes | 19.7
Linear Chromaticities | -10
Second-order Chromaticities | 50

Table 2: Parameters of iRCS Lattice

By extending the design circumference of the iRCS, the design length of the
dispersion-free drifts can readily be extended. Here the iRCS accommodates 40
meters of accelerating RF cavities with a 1.2 MV total voltage and a 20-Hz
ramp rate. The peak dipole field of 1.25 T was chosen to minimize eddy-current
losses. Our design efforts favor perpendicular-biased ferrite RF cavities,
which may deliver twice the accelerating gradient of parallel-biased ferrite
RF cavities. Perpendicular-biased ferrite cavities are subject of an active
R&D effort at Fermilab [16].

The effect of linear chromaticity on integrable motion was examined in [17],
and the effect of nonlinear chromaticity was examined in [18]. Chromaticity is
compatible with integrability if the horizontal and vertical chromaticities
are matched. In the example iRCS lattice, the chromaticities were matched and
reduced in both the first and second order. The chromaticity values shown in
Fig. 2 are the result of correction by the sextupole magnets (green in Fig.
2).

To preserve integrability, sextupole magnets should also be located so that
their effect cancels harmonically (separated by a $\pi$-odd phase-advance)
[17]. To maintain the flexibility of the early design, this constraint was not
imposed on the example lattice shown here. This constraint can be met by
requiring a $\pi$-odd phase-advance for the 12 linear-periodic cells, or by
combining into 6 complex linear-periodic cells.

##  3 LONGITUDINAL CAPTURE

The PIP-II linac is designed for a 2 mA beam current with a pulse duration of
0.6 ms to fill the Fermilab Booster to an intensity of
$6.4~{}\times~{}10^{12}$ protons [5]. To limit the injection time into the
iRCS to 2-3ms, a 5 mA upgrade of the PIP-II linac is necessary.

For a preliminary analysis of longitudinal capture, we considered two
scenarios to fill the RCS while controlling longitudinal emittance. In the
aggressive chopping scenario, the beam is chopped 50% at 44 MHz to match the
RF buckets. The fill time is 3.1 ms for $48~{}\times~{}10^{12}$ protons. In
the adiabatic capture scenario, the beam is injected into 0.12 MV RF buckets,
and the voltage is raised to 1.2 MV over 0.6 ms. Only 5% of the beam is
chopped, and consequently, the fill time is 1.7 ms for $48~{}\times~{}10^{12}$
protons.

Figure 3 shows the longitudinal distribution for the aggressive chopping and
adiabatic capture scenarios. To achieve a more uniform charge density, the
injection energy was painted across a slowly accelerating RF bucket. The
injected beam was painted over 30% of the bucket height, for the aggressive
chopping scenario, and 17% for the adiabatic capture scenario. The
acceleration rate was adjusted until the 97% longitudinal rms emittance of
0.08 eV$\cdot$s was reached, consistent with the PIP-II Booster emittance [5].
If beam from the iRCS is not slip-stacked, the longitudinal emittance can be a
factor of 5 larger before encountering a particle loss limit caused by
transition crossing in the Main Injector. If a second harmonic RF cavity is
used during injection, a more uniform charge density can be achieved while
maintaining this longitudinal emittance [19, 20].

Figure 3: (left) Longitudinal phase-space distribution (top) and longitudinal
profile (bottom) for aggressive chopping scenario. (tight) Longitudinal phase-
space distribution (top) and longitudinal profile (bottom) for adiabatic
capture scenario.

To minimize power losses, the lattice magnets will follow a sinusoidal ramp
curve [11]. A corrector circuit will flatten the sinusoidal ramp curve during
injection, so that the injection energy consistently matches the lattice
energy. For a 20-Hz ramp rate, a 3 ms injection time corresponds to a 6% of
the cycle time and a 2% amplitude correction.

##  4 SUMMARY & FUTURE WORK

To achieve multi-MW beam power for the Fermilab high-energy program, an
integrable RCS replacement for the Fermilab Booster is an option that merits
careful scrutiny. In this paper we explore some of the preliminary design
concerns, including integrable RCS lattice design and longitudinal capture.
Experimental and numerical work on the interaction between integrable optics
and space-charge dominated beams is still ongoing [2, 3, 12].

Upcoming lattice designs will integrate sextupoles with harmonic cancellation.
Thresholds for collective instabilities in the highly nonlinear beam will be
analyzed for their impact on the iRCS parameters. The design of the high-power
H- stripping foil and injection chicane will also be developed. A second
harmonic RF will be added to the longitudinal capture analysis to demonstrate
longitudinal uniformity.

## References

  * Danilov and Nagaitsev [2010] V. Danilov and S. Nagaitsev, Phys. Rev. ST Accel. Beams 13, p. 084002 (2010).
  * Valishev  _et al._ [2012] A. Valishev, S. Nagaitsev, D. Shatilov, and V. Danilov, “Beam physics of Integrable Optics Test Accelerator at Fermilab,” in  _Proceedings of IPAC2012_ (2012).
  * Ruisard  _et al._ [2016] K. Ruisard, H. Baumgartner, B. Beaudoin, I. Haber, D. Matthews, and T. Koeth, “Early tests and simulation of quasi-integrable octupole lattices at the University of Maryland Electron Ring,” in  _Proceedings of HB2016_ (2016).
  * Prebys  _et al._ [2016] E. Prebys _et al._ , “Long term plans to increase Fermilab’s proton intensity to meet the needs of the long baseline neutrino program,” in  _Proceedings of IPAC2016_ (2016).
  * Derwent  _et al._ [2013] P. Derwent _et al._ , “Proton Improvement Plan-II,” Tech. Rep. (Fermi National Accelerator Facility, 2013).
  * DUN [2015] Long-Baseline Neutrino Facility (LBNF) and Deep Underground Neutrino Experiment (DUNE) Conceptual Design Report, 2015 .
  * P5 [2014] Report of the Particle Physics Project Prioritization Panel, 2014 .
  * Lebedev, Ostiguy, and Bhat [2016] V. Lebedev, J.-F. Ostiguy, and C. Bhat, “Beam acceleration and transition crossing in the Fermilab Booster,” in  _Proceedings of HB2016_ (2016).
  * Antipov  _et al._ [2016] S. Antipov, P. Adamson, S. Nagaitsev, and M.-J. Yang, “Study of fast instability in Fermilab Recycler,” in  _Proceedings of IPAC2016_ (2016).
  * Pellico  _et al._ [2014] W. Pellico, K. Domann, F. Garcia, K. Gollwitzer, K. Seiya, and B. Zwaska, “FNAL - The Proton Improvement Plan (PIP),” in  _Proceedings of IPAC2014_ (2014).
  * Tang [2013] J. Tang, Rev. Accel. Sci. Technol. 06, p. 143 (2013).
  * Bruhwiler  _et al._ [2016] D. Bruhwiler _et al._ , “Challenges to demonstrating integrable proton dynamics in IOTA,” Tech. Rep. (Fermi National Accelerator Laboratory, 2016).
  * Brown  _et al._ [2013] B. Brown _et al._ , Phys. Rev. ST Accel. Beams 16, p. 071001 (2013).
  * Eldred and Zwaska [2016] J. Eldred and R. Zwaska, Phys. Rev. Accel. Beams 19, p. 104001 (2016).
  * Dixon [2016] S. Dixon, “Siting/conventional facilities status,” in  _Presented at March 2016 Fermilab PIP- II Machine Advisory Committee Meeting_ (2016).
  * Romanov  _et al._ [2014] G. Romanov _et al._ , “Perpendicular biased ferrite tuned cavities for the Fermilab Booster,” in  _Proceedings of IPAC2014_ (2014).
  * [17] S. Webb, D. Bruhwiler, A. Valishev, R. Kishek, S. Danilov, and S. Nagaitsev, “Chromaticity considerations for nonlinear integrable lattices,” in  _Proceedings of AAC2014, in press_.
  * Cook  _et al._ [2016] N. Cook _et al._ , “Understanding perturbations to nonlinear integrable optics in IOTA,” in  _paper in these Proceedings_ (2016).
  * Hotchi  _et al._ [2012] H. Hotchi _et al._ , Phys. Rev. ST Accel. Beams 15, p. 040402 (2012).
  * Pham, Lee, and Ng [2015] A. Pham, S. Lee, and K. Ng, Phys. Rev. ST Accel. Beams 18, p. 124001 (2015).


# Vietnamese Capitalization and Punctuation Recovery Models

###### Tóm tắt nội dung

Despite the rise of recent performant methods in Automatic Speech Recognition
(ASR), such methods do not ensure proper casing and punctuation for their
outputs. This problem has a significant impact on the comprehension of both
Natural Language Processing (NLP) algorithms and human to process.
Capitalization and punctuation restoration is imperative in pre-processing
pipelines for raw textual inputs. For low resource languages like Vietnamese,
public datasets for this task are scarce. In this paper, we contribute a
public dataset for capitalization and punctuation recovery for Vietnamese; and
propose a joint model for both tasks named JointCapPunc. Experimental results
on the Vietnamese dataset show the effectiveness of our joint model compare to
single model and previous joint learning model. We publicly release our
dataset and the implementation of our model at
https://github.com/anhtunguyen98/JointCapPunc

Index Terms: Punctuation prediction, Capitalization prediction, Text
normalization, Joint learning,

##  1 Introduction

Generic ASR systems only transcribe audio input to raw textual output of
single-lowercase word sequences. For a given output sequence of an ASR system:
’hi uyen how are you’ , the properly capitalized and punctuation recovered
output would be: ’Hi Uyen, how are you?’. Restoring punctuation and
capitalization enormously improves the readability of ASR output. This step
also improves the results for common subsequent tasks in NLP like Named entity
recognition, Intent classification, etc. [1, 2, 3] Data resources for
Vietnamese capitalization and punctuation recovery is limited. There are only
two public datasets [4] for punctuation prediction and no available dataset
for capitalization restoration. These two datasets are aggregated from news
articles and novels, which do not contain more context on spoken language ,
thus do not generalize well enough for applications in ASR systems.

In this work, we aim to apply capitalization and punctuation recovering to ASR
systems in the medical domain. Our method can enhance the readability of ASR-
transcribed records of the conversations between COVID-19 patients and
doctors.

We present a large-scale public dataset for Vietnamese capitalization and
punctuation recovering focus on the medical domain. The dataset contains
context on spoken language that consist thousands of question and answer pairs
that crawled from healthcare websites. We consider common punctuation marks
and capitalization types presented in section 3.

Traditional methods often solve these tasks independently as two sequence
labeling models. These methods exploit probabilistic graphical models like
conditional random fields (CRFs) [5] [6] [7] and combine with common neural
networks such as CNNs and LSTM [8] [4] [9]. For Vietnamese, recently pre-
trained language models [10] have made breakthroughs in various Vietnamese NLP
tasks and applications including analyzing questions [11], punctuation
prediction [12], sentiment analysis [13], sequence tagging [14]. In this
paper, we proposed a new joint learning model that based on pre-trained
transformers language models and extend with a capitalization prediction layer
to produce the soft capitalization features to combine with contextual
features for the punctuation prediction layer. Our contributions are
summarized as follows:

  * •

We introduce a Vietnamese public capitalization and punctuation recovery
dataset—named ViCapPunc.

  * •

We proposed a capitalization and punctuation recovery model named
JointCapPunc. Experimental results on our dataset show the benefits of joint
learning over learning the model separately on each task.

##  2 Related Work

In addition to IQ2111http://www.intelligencesquaredus.org, IWSLT 2011 [15] are
common datasets which used for capitalization and punctuation prediction in
English. All of these text are the transcripts of the conversations that
contain spoken language. To our knowledge, there are two public datasets et al
[4] for punctuation prediction task which on news domain and novels domain in
Vietnamese. However two Vietnamese dataset do not focus on spoken language
domain that make improvement for ASR system. Besides, these dataset do not
consist annotation for capitalization prediction task.

Early approaches to solve the two tasks are using single sequence labeling
model including n-gram language model [16], conditional random fields model
[7] [17], deep neural models [4], [12]. Recently, there are some studies
proposed to solve the two tasks simultaneously following two strategies. The
first strategy is combining annotations of capitalization and punctuation task
and formulate the task as sequence labeling [18] or sequence to sequence
models [19]. The second strategy is to jointly learn a model for both tasks,
which performs better than its single counterpart . [20] employs bidirectional
recurrent neural networks and two independent prediction layers , which does
not leverage capitalization features for punctuation prediction tasks. Monica
et al [21] proposed a method that conditions truecasing prediction on
punctuation output. They concatenate the softmax probabilities of punctuation
output with BERT encoder’s outputs and feed to a linear to predict truecasing.
Our method is similar to this work but we use the output probability of
capitalization head to produce soft capitalization features. We then combine
these features with the contextual word features to predict punctuation marks.
Besides, to improve performance of punctuation task, we extend model with
conditional random field layer. This model is presented in detail in section
4.

##  3 ViCapPunc Dataset

To investigate the two tasks for Vietnamese. We build a large scale dataset
namely ViCapPunc that contains spoken language from the medical domain. We
first crawl 50,000 question and answer pairs from the reputable Vietnamese
healthcare sites online including Vinmec222https://www.vinmec.com,
Alobacsi333https://alobacsi.com and ISOFHCARE444https://isofhcare.com. There
are 425,350 sentences in the ViCapPunc dataset. We define two types of label
for the capitalization prediction task including: word with the first
character capitalized (CAP), word with all capitalized characters (ALL-CAP)
and three common punctuation marks for the punctuation prediction task
including the comma(,), the period(.) and the question mark (?).

We split 50,000 question and answer pairs into training, validation and test
sets randomly with the ratio 50%, 20%, 30% respectively. For pre-processing
steps, we converted all data into lower case, removed the special characters
and punctuation marks, keeping commas, periods and question marks. Finally, we
label the dataset following the format in table 1 that consists of three
columns. Column 1 is the lower case text of input sentences. Column 2 is the
capitalization label, where 0 denotes non-capitalized word, 1 is CAP and 2 is
ALL-CAP. Columns 3 is the punctuation label where O indicates that a word is
not followed by any punctuation. Following the strategy in Thuy et. al. [4]
where the maximum length of a question and answer is 150 words, we do not
segment data into paragraphs. Instead, we decided to split the data into
segments of length equal to 150 and made sure that the last word of a segment
input is the last word of a sentence. Table 2 shows the statistics of our
dataset.

Bảng 1: The format of the dataset. Text |  | Cap label |  | Punctuation label
---|---|---|---|---
chào |  | 1 |  | O
uyên |  | 1 |  | COMMA
bạn |  | 0 |  | O
có |  | 0 |  | O
khoe |  | 0 |  | O
không |  | 0 |  | QMARK
Bảng 2: Statistics of ViCapPunc dataset. Punctuation | Train | Validation | Test
---|---|---|---
COMMA | 228,495 | 92,026 | 137,865
PERIOD | 184,383 | 74,097 | 111,717
QMARK | 27,448 | 10,987 | 16718
Capitalization |  |  |
CAP | 318,654 | 127,222 | 191,767
ALL-CAP | 37,542 | 14,582 | 23,173
Sentences | 211,831 | 85,084 | 128,435

##  4 JointCapPunc

Hình 1: Illustration of our JointCapPunc model.

Figure 1 illustrates the architecture of our joint model—named JointCapPunc,
which consists of three layers including: an encoding layer (i.e. encoder),
and two decoding layers of capitalization prediction and punctuation
prediction
Encoding layer: Given an input sentences consist of n words
$w_{1},w_{2}...w_{n}$. The encoding layer employs a pre-trained language model
based on Transformers encoder [22] like RoBERTa [23], Electra [24]. Because
RoBERTa, Electra segments the input sentence to sub-words which enables the
encoding of rare words with appropriate sub-words. We re-combined sub-word
representation after encoding layer into word representation using sum
operation. The $i^{th}$ word $w_{i}$ is represented by vector $e_{i}$ as
follows:

$e_{i}=PretrainedLM(w_{1:n},i)$ (1)

Capitalization prediction layer: Following the common approached when fine-
tuning a pre-trained language model for token classification task [10], the
punctuation prediction layer is a linear prediction layer which is added on
the top of encoder layer. In particular, we used word vector representation
$e_{i}$ that is fed into feed-forward network layer ($FFN_{cap}$) followed by
softmax predictor for capitalization prediction.

$p_{i}=softmax(FFN_{cap}(e_{i})),$ (2)

where the output size of $FFN_{cap}$ is 3, including non-capitalized words,
words with first character capitalized, words with all capitalized characters.
The cross entropy loss function $\mathcal{L}_{cap}$ is used to optimize
capitalization prediction.

Punctuation prediction layer: The punctuation prediction task is formulated as
sequence labeling problem. First, we create a sequence of vector $x_{1:n}$ in
which $x_{i}$ is the concatenation of the word representation $e_{i}$ and the
soft capitalization embedding $c_{i}$.

$x_{i}=e_{i}\oplus c_{i},$ (3)

where the soft capitalization embedding is computed by multiplying the weight
matrix $W$ with the corresponding output probability vector $p_{i}$

$c_{i}=Wp_{i}.$ (4)

Then each vector $x_{i}$ is fed into the $FFN_{punc}$ layer:

$s_{i}=FFN_{punc}(x_{i}),$ (5)

where the output size of $FFN_{punc}$ layer is the number of punctuation
label. The punctuation prediction layer feeds output vector $s_{i}$ into a
conditional random field layer [5]. Negative log likelihood loss
$\mathcal{L}_{punc}$ is used for punctuation prediction task optimization.

Joint training: The final training objective loss $\mathcal{L}$ of our
proposed JointCapPunc model is the weighted sum of the capitalization
prediction loss $\mathcal{L}_{cap}$ and punctuation prediction loss
$\mathcal{L}_{punc}$

$\mathcal{L}=\lambda\mathcal{L}_{cap}+(1-\lambda)\mathcal{L}_{punc}$ (6)

##  5 Experiments

###  5.1 Evaluation metrics

The performance of our model is evaluated by using precision, recall and
f1-score for both task.

$precision={TP\over{TP+FP}},$ (7) $recall={TP\over{TP+FN}},$ (8)
$F_{1}={2*precision*recall\over{precision+recall}},$ (9)

where $TP$ (true positive) is the number of punctuation mark / capitalization
mark correct predictions. $FP$ (false positive) is the number of punctuation
mark / capitalization mark that are falsely predicted. $FN$ (false negative)
is the number of punctuation marks / capitalization marks that are not
identified.

###  5.2 Experimental Setup

We conduct experiments on our ViCapPunc dataset to study the effectiveness of
joint training model. Here, we employ XLM-R [25] is pre-trained on a 2.5TB
multilingual dataset that contains 137 GB Vietnamese text and vELECTRA [14], a
monolingual variant of ELECTRA for Vietnamese, is pre-trained on 10GB of
Vietnamese text.

Our models were implemented in PyTorch [26] using Huggingface’s Transformers
[27]. Both single-task learning models and jointly learning model were trained
using the AdamW optimizer [28] and set the batch size to 32. We set the
epsilon and weight decay to default value in PyTorch, i.e. 1e-8, the soft
capitalization embedding dim is 256 . The learning rate and mixture weight
$\lambda$ were tuned in $\\{$1e-5, 2e-5, 3e-5, 4e-5, 5e-5$\\}$ and $\\{$ 0.05,
0.1, 0.15 … 0.95$\\}$ respectively. The best learning rate and mixture weight
value was 5e-5 and 0.15 respectively. We train for 10 epochs and calculate the
average score of the $F_{1}$-score for capitalization prediction and
punctuation prediction after each training epoch on validation set and
selected version that obtained the highest average score on the validation set
to apply to the test set.

###  5.3 Experimental Results

Table 3 shows results obtain for our JointCapPunc model compares with baseline
model of single task training and joint model Monica et al [21]. For single
task training model: Following common strategy when fine-tuning a pre-trained
language model for token classification task [10]. We append a linear
prediction layer on the top of pre-trained language models as briefly
described in Section 4 which is applied for the capitalization prediction
task. For punctuation prediction task, we use a punctuation restore model for
Vietnamese [29] that append a CRF prediction layer on top of pre-trained
language models. The single training strategy based on the XLM-R and vELECTRA
In each setting, JointCapPunc model performs better than single training
approach and joint model Monica et al [21] on both task. Here, the highest
improvements are accounted for the punctuation $F_{1}$ (ie., 78.65% $\to$
79.33% $\to$ 80.10%) and the capitalization $F_{1}$ (ie., 83.55% $\to$ 84.16%
$\to$ 86.75%). Thus showing that our joint training model and soft
capitalization features helps to better exploit the relationship between the
two tasks. Besides, vELECTRA outperforms XLM-R (punctuation mirco-$F_{1}$:
80.10% vs 78.74%, capitalization micro-$F_{1}$: 86.75% vs 85.39%).

Bảng 3: Experimental result (in %) on testset  Encoder | Model | Capitalization | Punctuation
---|---|---|---
Precision | Recall | $F_{1}$ | Precision | Recall | $F_{1}$
XLM-R | Single-task | 84.14 | 82.98 | 83.55 | 79.00 | 77.96 | 78.48
Monica et al [21] | 84.84 | 83.36 | 84.10 | 78.97 | 78.25 | 78.58
| JointCapPunc | 85.66 | 85.12 | 85.39 | 79.02 | 78.45 | 78.74
vELECTRA | Single-task | 83.97 | 83.10 | 83.53 | 79.35 | 77.96 | 78.65
Monica et al [21] | 84.67 | 83.65 | 84.16 | 79.36 | 79.31 | 79.33
| JointCapPunc | 86.55 | 86.95 | 86.75 | 80.79 | 79.43 | 80.10
Bảng 4: Ablation study (in %) on testset (the decrease in the F1 score of the modified models is indicated in bold) Component | Modification | Capitalization | Punctuation
---|---|---|---
Precision | Recall | $F_{1}$ | Precision | Recall | $F_{1}$
Full | None | 86.55 | 86.95 | 86.75 | 80.79 | 79.43 | 80.10
Capitalization feature | Removal | 85.40 | 80.99 |  83.13 (3.62$\downarrow$) | 80.52 | 77.80 |  79.13 (0.97$\downarrow$)
CRF layer | Softmax | 86.24 | 81.93 |  83.98 (2.77$\downarrow$) | 79.42 | 78.00 |  78.71 (1.39$\downarrow$)
Contextual embedding | Fasttext | 79.72 | 49.05 |  60.73 (26.02$\downarrow$) | 62.30 | 41.03 |  49.47 (30.63$\downarrow$)

###  5.4 Ablation Study

We evaluate the contribution of individual components of our model compare
with our best model that using vELECTRA as encoding layer and full components
as follows:

  * •

Contextual embedding: We replace contextual word embedding based on pre-
trained transformers language model by context-independent word embedding. In
our experiment, we use Fasttext [30], a variant of Word2Vec [31] which solved
unknown words problem by using sub-word models.

  * •

Capitalization feature: We remove the capitalization feature and only use
contextual embedding to predict punctuation mark.

  * •

Conditional random fields: We replace the CRF layer by softmax function for
inferencing punctuation mark.

Table 4 shows the impact of all component that contributed to the success of
our joint learning model.The performance of the system degrades when we
modified or removed a component. While the full model got capitalization
$F_{1}$ of 86.75% and punctuation $F_{1}$ of 80.10%, the capitalization
$F_{1}$ and punctuation $F_{1}$ reduced to 83.13%, 79.13% respectively when we
remove capitalization feature and only use contextual embedding to prediction
punctuation mark. The $F_{1}$ score of capitalization prediction task and
punctuation prediction task are only 83.98%, 78.71% respectively when we
replaced the CRF layer by softmax function. Finally, we find that replacing
contextual embeddings by independent word embedding (Fasttext) leads to the
biggest decrease 23.43% capitalization $F_{1}$ and 29.86% punctuation $F_{1}$.

##  6 Conclusions

In this paper, we presented the public dataset for Vietnamese capitalization
and punctuation recovery. ViCapPunc dataset can be utilized to develop a text
normalization module for ASR systems focus on medical domain which used to
record the conversation between doctors and COVID-19 patient. In addition, we
also proposed an effective model, namely JointCapPunc, for jointly learning
capitalization and punctuation recovery. In particular, JointCapPunc based on
transformers based pre-trained language models to learn essential words in the
sentences. Soft capitalization embedding combined with word representation can
produce useful features for punctuation prediction task. Therefor our model
are able to capture the relationship between the two tasks. We conduct
experiments and error analysis on our dataset and the results show that
jointly learning model is more effective than baseline models (ie., single
training approach, joint model Monica et al [21]). However our model is based
on transformers pre-trained language models that has a giant parameters scale,
which might face difficulties in ASR pipeline due to the increase of latency.
For future work, we plan to optimize our model to smaller size for faster and
lighter footprint.

##  7 Acknowledgements

We would like to thank AIC Technology Group, Hanoi, Vietnam for financial
support which made this work possible.

## Tài liệu

  * [1] E. Cho, J. Niehues, and A. Waibel, “Segmentation and punctuation prediction in speech language translation using a monolingual translation system,” in _Proceedings of the 9th International Workshop on Spoken Language Translation: Papers_ , 2012.
  * [2] T. B. Nguyen, Q. M. Nguyen, T. T. H. Nguyen, Q. T. Do, and C. M. Luong, “Improving Vietnamese Named Entity Recognition from Speech Using Word Capitalization and Punctuation Recovery Models,” in _Proc. Interspeech 2020_ , 2020, pp. 4263–4267.
  * [3] M. Tkachenko and A. Simanovsky, “Named entity recognition: Exploring features,” in _11th Conference on Natural Language Processing, KONVENS 2012: Empirical Methods in Natural Language Processing-Proceedings of the Conference on Natural Language Processing 2012_ , 2012, pp. 118–127.
  * [4] T. Pham, N. Nguyen, Q. Pham, H. Cao, and B. Nguyen, “Vietnamese punctuation prediction using deep neural networks,” in _International Conference on Current Trends in Theory and Practice of Informatics_. Springer, 2020, pp. 388–400.
  * [5] J. D. Lafferty, A. McCallum, and F. C. N. Pereira, “Conditional random fields: Probabilistic models for segmenting and labeling sequence data,” in _Proceedings of the Eighteenth International Conference on Machine Learning_ , ser. ICML ’01. San Francisco, CA, USA: Morgan Kaufmann Publishers Inc., 2001, p. 282–289.
  * [6] W. Wang, K. Knight, and D. Marcu, “Capitalizing machine translation,” in _Proceedings of the Human Language Technology Conference of the NAACL, Main Conference_ , 2006, pp. 1–8.
  * [7] Q. H. Pham, B. T. Nguyen, and N. V. Cuong, “Punctuation prediction for vietnamese texts using conditional random fields,” in _Proceedings of the Tenth International Symposium on Information and Communication Technology_ , 2019, pp. 322–327.
  * [8] P. Żelasko, P. Szymański, J. Mizgajski, A. Szymczak, Y. Carmiel, and N. Dehak, “Punctuation prediction model for conversational speech,” in _INTERSPEECH_ , 2018.
  * [9] O. T. Tran and V. T. Bui, “Neural text normalization in speech-to-text systems with rich features,” _Applied Artificial Intelligence_ , vol. 35, no. 3, pp. 193–205, 2021.
  * [10] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” in _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , 2019, pp. 4171–4186.
  * [11] N. Anh Tu, H. Thi Thu Uyen, T. Minh Phuong, and N. Xuan Bach, “Analyzing vietnamese legal questions using deep neural networks with biaffine classifiers,” in _International Conference on Neural Information Processing_. Springer, 2021, pp. 513–525.
  * [12] O. T. Tran _et al._ , “Punctuation prediction in vietnamese asrs using transformer-based models,” in _Pacific Rim International Conference on Artificial Intelligence_. Springer, 2021, pp. 191–204.
  * [13] Q. T. Nguyen, T. L. Nguyen, N. H. Luong, and Q. H. Ngo, “Fine-tuning bert for sentiment analysis of vietnamese reviews,” in _2020 7th NAFOSTED Conference on Information and Computer Science (NICS)_. IEEE, 2020, pp. 302–307.
  * [14] T. O. Tran, P. Le Hong _et al._ , “Improving sequence tagging for vietnamese text using transformer-based neural models,” in _Proceedings of the 34th Pacific Asia Conference on Language, Information and Computation_ , 2020, pp. 13–20.
  * [15] M. Federico, S. Stüker, L. Bentivogli, M. Paul, M. Cettolo, T. Herrmann, J. Niehues, and G. Moretti, “The iwslt 2011 evaluation campaign on automatic talk translation,” in _International Conference on Language Resources and Evaluation (LREC)_ , 2012, pp. 3543–3550.
  * [16] A. Gravano, M. Jansche, and M. Bacchiani, “Restoring punctuation and capitalization in transcribed speech,” in _2009 IEEE International Conference on Acoustics, Speech and Signal Processing_. IEEE, 2009, pp. 4741–4744.
  * [17] X. Wang, H. T. Ng, and K. C. Sim, “Dynamic conditional random fields for joint sentence boundary and punctuation prediction,” in _Thirteenth Annual Conference of the International Speech Communication Association_ , 2012.
  * [18] T. T. H. Nguyen, T. B. Nguyen, N. P. Pham, Q. TRUONG, T. L. Le, and C. M. Luong, “Toward human-friendly asr systems: Recovering capitalization and punctuation for vietnamese text,” _IEICE TRANSACTIONS on Information and Systems_ , vol. 104, no. 8, pp. 1195–1203, 2021.
  * [19] B. Nguyen, V. B. H. Nguyen, H. Nguyen, P. N. Phuong, T.-L. Nguyen, Q. T. Do, and L. C. Mai, “Fast and accurate capitalization and punctuation for automatic speech recognition using transformer and chunk merging,” in _2019 22nd Conference of the Oriental COCOSDA International Committee for the Co-ordination and Standardisation of Speech Databases and Assessment Techniques (O-COCOSDA)_. IEEE, 2019, pp. 1–5.
  * [20] V. Pahuja, A. Laha, S. Mirkin, V. Raykar, L. Kotlerman, and G. Lev, “Joint learning of correlated sequence labeling tasks using bidirectional recurrent neural networks, ibm research,” _INTERSPEECH 2017. http://dx. doi. org/10.21437/Interspeech_ , vol. 1247, 2017.
  * [21] M. S. S. R. K. Dixit and S. B. K. Kirchhoff, “Robust prediction of punctuation and truecasing for medical asr,” _ACL 2020_ , p. 53, 2020.
  * [22] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” in _Advances in neural information processing systems_ , 2017, pp. 5998–6008.
  * [23] Y. Liu, M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, and V. Stoyanov, “Roberta: A robustly optimized bert pretraining approach,” _arXiv preprint arXiv:1907.11692_ , 2019.
  * [24] K. Clark, M.-T. Luong, Q. V. Le, and C. D. Manning, “ELECTRA: Pre-training text encoders as discriminators rather than generators,” in _ICLR_ , 2020\.
  * [25] A. Conneau, K. Khandelwal, N. Goyal, V. Chaudhary, G. Wenzek, F. Guzmán, É. Grave, M. Ott, L. Zettlemoyer, and V. Stoyanov, “Unsupervised cross-lingual representation learning at scale,” in _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , 2020, pp. 8440–8451.
  * [26] A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga _et al._ , “Pytorch: An imperative style, high-performance deep learning library,” _Advances in neural information processing systems_ , vol. 32, pp. 8026–8037, 2019.
  * [27] T. Wolf, L. Debut, V. Sanh, J. Chaumond, C. Delangue, A. Moi, P. Cistac, T. Rault, R. Louf, M. Funtowicz, J. Davison, S. Shleifer, P. von Platen, C. Ma, Y. Jernite, J. Plu, C. Xu, T. Le Scao, S. Gugger, M. Drame, Q. Lhoest, and A. Rush, “Transformers: State-of-the-art natural language processing,” in _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations_. Online: Association for Computational Linguistics, Oct. 2020, pp. 38–45. [Online]. Available: https://aclanthology.org/2020.emnlp-demos.6
  * [28] I. Loshchilov and F. Hutter, “Decoupled weight decay regularization,” in _International Conference on Learning Representations_ , 2018.
  * [29] H. Tran, C. V. Dinh, Q. Pham, and B. T. Nguyen, “An efficient transformer-based model for vietnamese punctuation prediction,” in _International Conference on Industrial, Engineering and Other Applications of Applied Intelligent Systems_. Springer, 2021, pp. 47–58.
  * [30] P. Bojanowski, E. Grave, A. Joulin, and T. Mikolov, “Enriching word vectors with subword information,” _Transactions of the Association for Computational Linguistics_ , vol. 5, pp. 135–146, 2017.
  * [31] T. Mikolov, K. Chen, G. S. Corrado, and J. Dean, “Efficient estimation of word representations in vector space,” in _ICLR_ , 2013.