# Nonclassical effects of photon-phonon antibunching in a multifield driven
optomechanical cavity

Joy Ghosh<EMAIL_ADDRESS>School of Nanoscience and Technology,
IIT Kharagpur, West Bengal, 721302, India  Shailendra K. Varshney  School of
Nanoscience and Technology, IIT Kharagpur, West Bengal, 721302, India
Electronics and Electrical Communication Engineering Department, IIT
Kharagpur, West Bengal, 721302, India  Kapil Debnath<EMAIL_ADDRESS>Electronics and Electrical Communication Engineering Department, IIT
Kharagpur, West Bengal, 721302, India  School of Natural and Computing
Sciences, University of Aberdeen, Aberdeen AB24 3UE, UK

###### Abstract

The nonclassical signature of a photon-phonon pair can be tested effectively
by violating Cauchy-Schwarz and Bell’s inequality, which can arise due to
antibunching phenomena in coupled bosonic systems. In this paper, we analyze
the measurement criteria imposed on the second-order coherence functions and
investigate the quantum correlations leading to the suppression of multi-
photon-phonon excitation in a single optomechanical cavity upon driving it
with two pumping fields. It is also shown that the Cauchy-Schwarz violation
can serve as an ideal precursor to demonstrate stronger tests of locality
related to Bell’s inequality. We consider weak driving and optomechanical
coupling coefficient parameters in the system that enables the unconventional
nature of photon (phonon) blockades while operating in the resonance of cavity
detuning and mechanical frequency. These findings are valuable for generating
sub-Poissonian signals in optimal conditions and have potential applications
in hybrid systems for on-demand single photon (phonon) detection.

††preprint: APS/123-QED

##  I Introduction

In what sense does the quantum behavior of a pulse of light become predominant
was the central question that kept the optics community busy from the middle
of the 20th century. Later, the issue was resolved by the pioneering works of
Glauber, Sudarshan, Mandel, and others, where the coherence functions of
quantized electromagnetic fields are introduced to characterize different
nonclassical effects such as antibunching, squeezing, and non-locality [1]. In
this context, the most well-demonstrated examples include twin-beam generation
[2], four-wave mixing [3], parametric down-conversion [4], resonance
fluorescence [5], etc. The central argument for the proposal of nonclassical
demonstration is the phenomena of anticorrelation in antibunching [6], which
can be easily measured by implementing a ubiquitous relation in mathematical
physics and engineering called the Cauchy-Schwarz (CS) inequality [7]. The CS
inequality provides a classical upper bound, which states that products of two
arbitrary fluctuating vectors are bounded by the squared expectation value of
their cross-correlations i.e. $|\langle AB\rangle|\leq\sqrt{\langle
A^{2}\rangle\langle B^{2}\rangle}$, where $A$ and $B$ are two random variable
observables and any two classical signals always obey this fundamental
relation. However, the antibunching property of light can violate this
inequality, which is not accountable in classical optics [8]. Previously, the
violation has been reported in a plethora of optical systems and atomic
ensembles [9, 10, 11, 12, 13], as well as with matter-waves [6] and recently
in magnon pairs [14]. The CS violation is also considered a major prediction
of the spontaneous Hawking radiation in sonic black holes [15]. The
implication of CS violation is two-fold, firstly, it depicts the stronger
quantum correlations between multimode bosonic systems which is absent in the
classical picture, and secondly, it implies the possibility of nonlocal
effects encountered in the Clauser-Horne-Shimony-Holt (CHSH) framework [6, 8].
The CHSH inequality [16] is a particular type of Bell inequality [17] that
falsified the idea of local realism advocated by Einstein, Podolsky, and Rosen
(EPR) in the hidden variable model [18]. Over the past decades CHSH violation
has been reported on different platforms (optical, condensed matter, etc.)[19,
20, 21, 22] which involve correlations between measurements on entangled
particles that are usually represented by Bell states. Violations of the CHSH
inequality have utmost importance in the current discourse of physical
theories, providing evidence for the non-local phenomenon encountered in
quantum systems. On a fundamental level, it is noteworthy to analyze CS and
CHSH violations altogether in micro and nanomechanical systems for future
quantum information and communication technologies.

Over the past decade, considerable efforts have been devoted to exploring the
nonlinear interaction of nanoscale mechanical oscillators with optical
cavities via radiation pressure force of light, which gave birth to the field
of optomechanics (OM) [23]. Ground state cooling [24], normal mode splitting
[25], entanglement between mirror and light [26], and squeezing of mechanical
oscillators [27] showed breakthroughs, which brings OM a considerable
participant while investigating the nonlinear quantum regime on a mesoscopic
scale [28].

Figure 1: (a) Schematic of a single cavity optomechanical system consisting of
a membrane in the middle architecture, under two weak driving fields. The
amplitude of the optical and mechanical driving fields are $\Omega_{c}$ and
$\Omega_{m}$ with corresponding damping rates $\kappa$ and $\gamma$. (b)
Transition pathways of multiple Fock states leading destructive interference,
which is responsible for the two-photon-phonon blockade.

Thus, antibunching properties primarily featured with photons are now
associated with phononic modes also [29]. In the demonstration of
antibunching, a weak Kerr-like nonlinearity (alternatively known as the
unconventional blockades [30]) is essential for the resonant excitation of
single quanta, for this, several improvements of photon and phonon blockades
in OM platforms are proposed via two-level systems [31], parametric
amplification [32], spinning resonator [33], $\mathcal{PT}$ symmetric effects
[34], quadratic coupling [35, 36], etc. However, OM coupling itself resembles
a Kerr-like effect [37], which can cause a violation of classical inequalities
in a single photon-phonon-generated Fock state. On a general ground, it is
possible to exhibit the CHSH violation along with the CS inequality violation
in a single setup while addressing the effects of the antibunching property.

In this paper, we report a complete theoretical analysis of the CS/CHSH
violation based on the strong nonclassical signature of the OM cavity, where
the system is driven by two weak pump fields. For this, the second-order
coherence function designed for the quantum operators is implemented to test
the antibunching phenomena of optical and mechanical modes simultaneously.
From analytical derivations, a single photon-phonon Fock state is established
via the multiple pathway destructive interference [38], which can be generated
upon resonant driving of the system, reflecting the CS/CHSH violation to a
great extent. The sub-Poissonian statistics of the photon-phonon field are
extracted from the joint Fock-space distribution, which holds fundamental
importance in quantum optics studies. Furthermore, the single particle
occupancy probability has achieved maximum in this system while operating in
resonance conditions. This indicates efficient antibunching phenomena
certified for the benefits of sensing and quantum control at a few photon-
phonon levels.

##  II Theoretical framework

Consider a standard Fabry–Pérot system, consisting of a laser-driven cavity,
which causes coherent vibrations in the movable membrane as shown in Fig.1(a).
The cavity frequency is assumed to be linearly coupled to the displacement of
the membrane via the radiation pressure force of light. In addition to the
optical driving force, an extra weak pumping is provided to excite phonons in
the mechanical mode. The weak pumping field can be implemented by using the
piezoelectric effect or a dc voltage signal [39]. The Hamiltonian of this
system in the rotating frame of driving frequency can be expressed as (taking
$\hbar=1$)

$\displaystyle\mathcal{H}$ $\displaystyle=$
$\displaystyle\Delta\hat{a}^{\dagger}\hat{a}+\omega_{m}\hat{b}^{\dagger}\hat{b}-g\hat{a}^{\dagger}\hat{a}(\hat{b}+\hat{b}^{\dagger})+\Omega_{c}\hat{a}^{\dagger}$
(1) $\displaystyle+\Omega_{m}\hat{b}^{\dagger}+\text{h.c}$

Here, $\Delta=\omega_{c}-\omega_{L}$ is the detuning between cavity frequency
($\omega_{c}$) and driving laser frequency ($\omega_{L}$),
$\hat{a}^{\dagger}(\hat{a})$ are the creation (annihilation) operators
associated with the optical field. The mechanical mode frequency is denoted by
$\omega_{m}$ with respective operators $\hat{b}^{\dagger}(\hat{b})$ and $g$
represents the optomechanical coupling strength. The amplitude of the weak
mechanical pump (and laser drive) is expressed as $\Omega_{m}$ ($\Omega_{c}$).
The free part of the Hamiltonian (without driving fields) in Eq.(1) has
eigenvalues of the form $E_{nm}=\Delta n+\omega_{m}m-g^{2}n^{2}/\omega_{m}$
(where $n$ and $m$ are positive integers), which showcases the anharmonicity
of the energy spectrum in the cavity-membrane oscillator system. By taking the
cavity ($\kappa$) and mechanical dissipation ($\gamma$) rates into account,
the dynamical evolution of the system can be described by the following
Lindblad master equation

$\displaystyle\partial_{t}\rho$ $\displaystyle=$
$\displaystyle-i[\mathcal{H},\rho]+\kappa\mathcal{L}(\hat{a})\rho+(n_{th}+1)\gamma\mathcal{L}(\hat{b})\rho$
(2) $\displaystyle+n_{th}\gamma\mathcal{L}(\hat{b}^{\dagger})\rho$

where $\rho$ is the density matrix and
$n_{th}=1/[\exp(\frac{\hbar\omega_{m}}{k_{B}T})-1]$ (where $k_{B}$ is the
Boltzmann constant) is the thermal phonon excitation number at temperature
$T$, and the Lindbladian is expressed as
$\mathcal{L}(\hat{o})=\hat{o}\rho\hat{o}^{\dagger}-\frac{1}{2}\\{\hat{o}^{\dagger}\hat{o},\rho\\}$;
$o=\hat{a},\hat{b}$ with $\\{\cdot\\}$ is the anticommutation operation. From
Eq.(2), the time-delayed second-order coherence function for photons can be
computed as,
$g^{2}(\tau)=\text{Tr}[\hat{a}^{\dagger}\hat{a}^{\dagger}\hat{a}\hat{a}\rho]/\text{Tr}[\hat{a}^{\dagger}\hat{a}\rho]^{2}$,
while in case of phonons, operator $\hat{a}$ is replaced by $\hat{b}$. The
function $g^{2}(\tau)$ has several properties from which the statistical
nature of quanta is distinguished from the classical signal, from which the
following inequalities are always satisfied, $(i)$ $g_{ii}^{2}(0)\geq 1$,
$(ii)$ $g_{ii}^{2}(\tau)\leq g_{ii}^{2}(0)$, and $(iii)$
$g_{ij}^{2}(0)\leq\sqrt{g_{ii}^{2}(0)g_{jj}^{2}(0)}$, where
$i=\hat{a},\hat{b}$ and $i\neq j$ [40]. The first condition describes the
coherent nature of classical fields and its violation indicates the sub-
Poissonian statistics of photons (phonons), while the limit
$g_{ii}^{2}(0)\rightarrow 0$ corresponds to the blockade phenomena in which
only single quanta can be excited corresponding to the optical or mechanical
mode. The violation of the second and third conditions displays the effect of
anticorrelation in antibunching, resulting in CS inequality not satisfying.
These violations can be easily measurable by the output spectrum of an OM
cavity by using a Hanburry, Brown, and Twiss-type experimental setup [41, 42].

Figure 2: Steady-state second-order coherence function $g^{2}(0)$ (in
logarithmic scale) versus cavity detuning $\Delta/\kappa$ for two mechanical
frequencies ($\omega_{m1}=10\kappa$ and $\omega_{m2}=20\kappa$) showing
antibunching effect of photons (red) and phonons (blue) while operating in
resonance on both sides of the central linewidth.

The antibunching statistics characterized by the second-order coherence
function are also linked to the Fano factor, from which mechanical limit
cycles can be identified and the possibility of phonon laser is also
achievable [43]. Additionally, in the case of Gaussian states, the classically
forbidden values of the coherence function are observed as a consequence of
optimized amplitude squeezing [44]. However, the analytical expression of the
second-order coherence function $g^{2}_{ii}(\tau)$ is not directly solvable
from the master equation Eq.(2). To better understand the physical mechanism,
an alternative method is provided to breach the classical inequalities and
demonstrate the antibunching phenomena in the present system. The analytical
description of the temporal evolution (as well as in steady-state) of the
second-order coherence function can be calculated by solving Schrödinger’s
equation in the truncated Fock space by assuming a weak driving condition i.e.
$\\{\Omega_{c},\Omega_{m}\\}\ll\\{\kappa,\gamma\\}$, where the possibility of
multi-particle excitation can be controlled by optimizing the system
parameters. Some important characterization parameters of this model are
considered as, high mechanical quality factor $\omega_{m}/\gamma_{m}\gg 1$,
the resolved-sideband regime $\omega_{m}>\kappa$, and a weak OM coupling
coefficient $g<\omega_{m}$. Assuming the membrane has been cooled to the
ground state initially $(n_{th}\approx 0)$, the weak driving field can be
treated as a perturbation, so the wavefunction (following the rapid
communication [38]) can be expressed in the photon-phonon joint Fock space as

$\displaystyle\ket{\psi}=\sum^{n+m\leq 2}_{n,m}C_{nm}\ket{nm}$ (3)

where $C_{nm}$ is the amplitude of a particular state
$\ket{n}\bigotimes\ket{m}$ (with $n$ photons and $m$ phonon numbers) belonging
to the wavefunction $\ket{\psi}$ with occupation probability $|C_{nm}|^{2}$,
such that $|C_{00}|\gg|C_{10}|$,$|C_{01}|\gg|C_{11}|$,$|C_{20}|$,$|C_{02}|$
satisfies under weak-pumping. Considering dissipation factors of the cavity
and membrane into account, the effective Hamiltonian has the following non-
hermitian form written as,
$\mathcal{H}_{eff}=\mathcal{H}-i\kappa/2\hat{a}^{\dagger}\hat{a}-i\gamma/2\hat{b}^{\dagger}\hat{b}$.
Based on the Schrödinger’s equation i.e.
$i\partial_{t}\ket{\psi}=\mathcal{H}_{eff}\ket{\psi}$, the dynamical evolution
of the transition probabilities $C_{nm}$ are obtained as

$\displaystyle i\partial_{t}C_{00}$ $\displaystyle=$
$\displaystyle\Omega_{c}C_{10}+\Omega_{m}C_{01}$ (4) $\displaystyle
i\partial_{t}C_{10}$ $\displaystyle=$
$\displaystyle\Delta^{\prime}C_{10}+(-g+\Omega_{m})C_{11}+\Omega_{c}C_{00}$
(5) $\displaystyle+\sqrt{2}\Omega_{c}C_{20}$ $\displaystyle
i\partial_{t}C_{01}$ $\displaystyle=$
$\displaystyle\omega^{\prime}_{m}C_{01}+\Omega_{c}C_{11}+\Omega_{m}C_{00}+\sqrt{2}\Omega_{m}C_{02}$
(6) $\displaystyle i\partial_{t}C_{11}$ $\displaystyle=$
$\displaystyle(\Delta^{\prime}+\omega_{m}^{\prime})C_{11}+(-g+\Omega_{m})C_{10}+\Omega_{c}C_{01}$
(7) $\displaystyle i\partial_{t}C_{20}$ $\displaystyle=$ $\displaystyle
2\Delta^{\prime}C_{20}+\sqrt{2}\Omega_{c}C_{10}$ (8) $\displaystyle
i\partial_{t}C_{02}$ $\displaystyle=$ $\displaystyle
2\omega_{m}^{\prime}C_{02}+\sqrt{2}\Omega_{m}C_{01}$ (9)

with $\Delta^{\prime}=\Delta-i\kappa/2$ and
$\omega_{m}^{\prime}=\omega_{m}-i\gamma/2$. The first equation of the
probability amplitudes, i.e. Eq.(4) is always approximately satisfied,
therefore we can consider $|C_{00}|^{2}\approx 1$. The other transitions among
the Fock states of the system can invoke photon-phonon blockade based on the
quantum interference effect of multiple pathways as shown in Fig.1(b). From
the steady-state analysis, the Eq.(5)-(9) is solved iteratively for $C_{nm}$
and $g^{2}_{ii}(0)$ is computed as

$\displaystyle g^{2}_{aa}(0)$ $\displaystyle=$
$\displaystyle\frac{\langle\hat{a}^{\dagger}\hat{a}^{\dagger}\hat{a}\hat{a}\rangle}{\langle\hat{a}^{\dagger}\hat{a}\rangle^{2}}=\frac{2|C_{20}|^{2}}{(|C_{10}|^{2}+|C_{11}|^{2}+2|C_{20}|^{2})^{2}}$
(10) $\displaystyle g^{2}_{bb}(0)$ $\displaystyle=$
$\displaystyle\frac{\langle\hat{b}^{\dagger}\hat{b}^{\dagger}\hat{b}\hat{b}\rangle}{\langle\hat{b}^{\dagger}\hat{b}\rangle^{2}}=\frac{2|C_{02}|^{2}}{(|C_{01}|^{2}+|C_{11}|^{2}+2|C_{02}|^{2})^{2}}$
(11) $\displaystyle g^{2}_{ab}(0)$ $\displaystyle=$
$\displaystyle\frac{\langle\hat{a}^{\dagger}\hat{b}^{\dagger}\hat{b}\hat{a}\rangle}{\langle\hat{a}^{\dagger}\hat{a}\rangle\langle\hat{b}^{\dagger}\hat{b}\rangle}=\frac{|C_{11}|^{2}}{(|C_{10}|^{2}+|C_{11}|^{2}+2|C_{20}|^{2})\times(|C_{01}|^{2}+|C_{11}|^{2}+2|C_{02}|^{2})}$
(12)

##  III Results and discussion

Figure 3: Equal time-delayed second-order coherence function
$\log[g^{2}(\tau)]$ of photons (red) and phonons (blue) versus cavity detuning
$\Delta/\kappa$ for two mechanical frequencies $\omega_{m1}=10\kappa$ and
$\omega_{m2}=20\kappa$, and the optomechanical coupling is set at
$g=0.25\kappa$.

In this section, we determine the solutions of Eq.(10)-(12) by solving
$C_{nm}$ from Eq.(5)-(9) and depict the response of $[g_{ii}^{2}(0)]_{i=a,b}$
(in logarithmic scale) as a function of normalized cavity detuning
($\Delta/\kappa$) in Fig.2. For the numerical simulation, the following
resolved-sideband parameters are taken, the driving field strengths are chosen
by considering weak pumping as $\Omega_{c}/\kappa=5\times 10^{-3}$ and
$\Omega_{m}/\kappa=10^{-4}$, and the mechanical damping rate is
$\gamma/\kappa=0.01$ with optomechanical coupling coefficient $g/\kappa=0.25$.
The detuning of the cavity frequency ($\Delta/\kappa$) is set for a wide range
and the mechanical frequency ($\omega_{m}/\kappa$) is varied accordingly to
record the response of the coherence function $g_{ii}^{2}(0)$ for
$i=\hat{a},\hat{b}$. From Fig.2 violation of the first inequality (proposed
earlier) can be observed, showing a strong antibunching effect i.e.
$\log[g_{ii}^{2}(0)]<0$ for an exact condition of $\Delta=-\omega_{m}$. For
different settings of the mechanical frequency $\omega_{m}/\kappa$, two dips
of $\log[g_{ii}^{2}(0)]$ are obtained respectively, first at the zeroth
position of $\Delta$ and second at $\Delta=-\omega_{m}$. For demonstration
purposes, $\omega_{m}$ is arbitrarily taken as $10\kappa$ and $20\kappa$,
following the good-cavity limit [45] in both blue- and red-sideband
operations. The sharp response of the second-order coherence functions clearly
shows that single photon-phonon blockade simultaneously exhibits in the system
and the sub-Poissonian nature of the excited states is revealed only in
resonance conditions.

Figure 4: Second-order coherence function $g_{ii}^{2}(0)$ for
$i=\hat{a},\hat{b}$ (in logarithmic scale) versus cavity detuting
$\Delta/\kappa$ and OM coupling strength $g/\kappa$ for membrane frequencies
$\omega_{m1}=10\kappa$ (upper panel) and $\omega_{m1}=20\kappa$ (lower panel),
showing the effect of anticorrelation in antibunching leading to the violation
of Cauchy-Schwarz inequality.

Also, by comparing the dips as $\log[g_{aa}^{2}(0)]\approx-2.79$ and
$\log[g_{bb}^{2}(0)]\approx-9.58$ for $\omega_{m1}=10\kappa$, we observe the
phonon antibunching is much stronger than photons when the resonance frequency
matches for both sides of the central linewidth i.e. $\Delta/\kappa>0$ and
$\Delta/\kappa<0$. Although for the central dip, $\Delta/\kappa=0$, the sub-
Poissonian light suppresses phonon excitation with dip values noted as
$\log[g_{aa}^{2}(0)]\approx-4.60$ and $\log[g_{bb}^{2}(0)]\approx-1.83$.
Similar responses are also recorded for an additional resonant frequency,
chosen as $\omega_{m2}=20\kappa$. In this case, the dip values are obtained as
$\log[g_{aa}^{2}(0)]\approx-1.59$ and $\log[g_{bb}^{2}(0)]\approx-8.38$, and
at the central position the dip values are $\log[g_{aa}^{2}(0)]\approx-4.60$
and $\log[g_{bb}^{2}(0)]\approx-0.62$. It can be seen that the central dip of
the cavity response doesn’t change by shifting the mechanical frequency,
however, the additional dips of the system decrease while increasing the
resonant frequency to much higher values. This is because higher resonant
frequencies require stronger driving strengths to maintain the quantum
coherence of photons and phonons.

Figure 5: Clauser-Horne-Shimony-Holt inequality violation with normalized
cavity detuning and optomechanical coupling coefficient for mechanical
frequencies (a) $\omega_{m1}=10\kappa$ and (b) $\omega_{m2}=20\kappa$. (c)
CHSH violation of a continuum of mechanical frequencies with cavity detuning
at fixed optomechanical coupling $g=0.25\kappa$.

Next, the temporal evolution of the second-order coherence function is
measured by the relation,
$[g^{2}(\tau)_{ii}]_{i=a,b}=\langle\hat{i}^{\dagger}(t)\hat{i}^{\dagger}(t+\tau)\hat{i}(t+\tau)\hat{i}(t)\rangle/\langle\hat{i}^{\dagger}(t)\hat{i}(t)\rangle\langle\hat{i}^{\dagger}(t+\tau)\hat{i}(t+\tau)\rangle$,
which depicts the joint probability of detecting one photon (phonon) at time
$t=0$ and the next emitted photon (phonon) at time $t=\tau$ [46]. The dynamics
of $g_{ii}^{2}(\tau)$ for $i=\hat{a},{b}$ is illustrated in Fig.3, clearly
showing the second inequality is also being violated i.e.
$g_{ii}^{2}(\tau)>g_{ii}^{2}(0)$ exhibits for finite time delay $\tau$.
Therefore, it further reflects the sub-Poissonian statistics of the classical
fields while operating on both the resonant frequencies that we have taken.
Also, fluctuations of the second-order coherence function at an initial time
are much lower than later instants of the transient stage, which is nothing
but showcasing the antibunching effect of photons and phonons simultaneously
occurring in this system.

The last and most directly applicable CS inequality proposed in the third
condition reveals that the absolute square of cross-correlation of the photon
and phonon fields is less than equal to the product of auto-correlations of
the individual fields. To demonstrate this effect, the three functional forms
of the coherence functions given in Eq.(10)-(12) are plotted by varying
normalized cavity detuning ($\Delta/\kappa$) and OM coupling strength
($g/\kappa$) in Fig.4. In addition, a dimensionless parameter is introduced to
check the degree of classical violation, defined as
$\rho=g^{2}_{ab}(0)/\sqrt{g^{2}_{aa}(0)g^{2}_{bb}(0)}$ [14]. Note that,
$\rho>1$ indicates the violation of CS inequality, signifying quantumness in
the OM system exhibiting strong photon-phonon correlations. From Fig.4(a) and
4(b), the violation of photon-phonon coherence functions can be observed with
OM coefficient variation up to the strong-coupling limit $g=2\kappa$, while
the membrane frequency is set at $\omega_{m}=10\kappa$. Fig.4(c) depicts the
positive cross-correlation of photon and phonon fields as a consequence of the
strong antibunching effect, suggesting that the two-mode state $\ket{11}$,
represented in Fig.1(b) is occupied. Next, the CS violation factor $\rho$ is
plotted in Fig.4(d), and we obtain $\rho\sim 9.42$, for $g=0.25\kappa$ and
$\Delta=\omega_{m1}=10\kappa$. The lower panel of Fig.4(e)-4(h) represents
similar characteristics of the coherence function obtained for the second
resonance frequency at $\omega_{m2}=20\kappa$. This demonstrates a clear shift
in the cavity detuning from $\Delta/\kappa=-10$ to $\Delta/\kappa=-20$ with
the violation factor found as $\rho\sim 8.79$. It can be noted that the CS
violation parameter is likely to decrease with increasing the resonant
frequency of the vibrating membrane.

As the CS inequality violation is established, we are set to validate the non-
locality test of the antibunched photon-phonon pair generated in the OM
cavity. It is expected that CHSH violation follows similar effects
corresponding to the CS inequality. To confirm this, the CHSH violation factor
$\mathcal{B}$ is calculated (check the appendix for detailed calculation) and
plotted in Fig.5, predicting our claim where $\mathcal{B}>2$ is obtained again
in resonance conditions only. The operating frequency of the mechanical system
is set with the previous values i.e. $\omega_{m}=10\kappa$ and
$\omega_{m}=20\kappa$ in Fig.5(a)-(b), and the CHSH violation can be
visualized in a wide range of OM coefficients ranging to the strong-coupling
$g=2\kappa$. Furthermore, the maximum violation obtained is nearly equal to
$2\sqrt{2}$, which is the upper limit of CHSH violation, also known as
Tsirelson’s bound. Lastly, a continuum of mechanical frequencies along with
the cavity detuning is considered in Fig.5(c), where the CHSH violation can be
realized for the whole range of resonance frequencies guaranteeing the genuine
quantum nature of the photon-phonon pair in this system. Now that the CS/CHSH
inequality violation is established, our claim of anticorrelation in
antibunching is properly tested revealing the strong nonclassical character of
optomechanical systems under optimized conditions.

To explain the earlier results more intuitively, the photon-phonon number
distributions are calculated to illustrate the nonclassical properties of the
second-order coherence functions. In Fig.6(a)-6(b), the steady-state mean
photon (phonon) numbers i.e. $\langle\hat{a}^{\dagger}\hat{a}\rangle$
($\langle\hat{b}^{\dagger}\hat{b}\rangle$) are plotted as a function of
normalized cavity detuning $\Delta/\kappa$ for both the frequencies
$\omega_{m}=10\kappa\quad\text{and}\quad 20\kappa$. The results agree well
with the previous calculations performed on $g^{2}_{ii}(0)$, where peaks of
distributions are obtained in the resonant frequencies only. It correctly
predicts the clustered population of photons and phonons at the position
$\Delta=-10\kappa$ and $-20\kappa$, respectively. Also, by comparing with the
previous analysis of $g^{2}_{ii}(0)$ at zero detuning, the average phonon
numbers are observed negligibly smaller than average photon numbers. Since the
driving strength provided is very low in the system, the mean photon (phonon)
numbers $\langle n_{i}\rangle\ll 1$ as expected. The classical limit breaks,
when the average photon (phonon) number $\langle n_{i}\rangle_{i=a,b}$ becomes
significantly smaller, as $[g^{2}_{ii}(0)]\geq 1-1/\langle n_{i}\rangle$,
indicates $g^{2}_{ii}(0)<1$ [8]. This is called the number-squeezing effect of
the photon-phonon statistical mixture, which is quite visible in
Fig.6(a)-6(b). The number-squeezing parameter can be introduced in this
context, which is defined as, $\eta^{2}=(\langle n^{2}\rangle-\langle
n\rangle^{2})/n_{total}$. This is related to the auto- and cross-correlation
formula by $\eta^{2}=1+(g^{2}_{aa}+g^{2}_{bb}-2g^{2}_{ab}-\langle
n_{-}\rangle^{2})/\langle n_{+}\rangle$, where $\langle n_{-}\rangle=\langle
n_{a}\rangle-\langle n_{b}\rangle$ and $\langle n_{+}\rangle=\langle
n_{a}\rangle+\langle n_{b}\rangle$ [47]. The system is said to be number
squeezed if $\eta^{2}<1$, which implies
$g^{2}_{aa}+g^{2}_{bb}-2g^{2}_{ab}-\langle n_{-}\rangle^{2}<0$. A number-
balanced state i.e. $\langle n_{-}\rangle=0$ always gives symmetrical values
of $g^{2}_{aa}=g^{2}_{bb}$, therefore, the number-squeezing parameter can be
directly linked to the CS violation factor, given by
$\eta^{2}=1+2(1-\rho)g^{2}_{aa}/\langle n_{+}\rangle$. This shows that the
number-squeezing of coherent fields ($\eta^{2}<1$) is responsible for the
violation of CS inequality ($\rho>1$), which can be treated as equivalent to
photon-phonon entanglement. Although in the present scenario, the number state
is not balanced ($g^{2}_{aa}\neq g^{2}_{bb}$), therefore the direct link
between CS inequality and squeezing parameter cannot be established due to the
residual factor $\langle n_{-}\rangle^{2}$, but the physical mechanism of CS
violation and its connection to the number squeezing can be interpreted by the
following way.

Additionally, the deviations of the photon (phonon) distribution from the
standard Poissonian distribution are calculated for the mean sample number,
defined as $\log[\mathcal{P}_{n}/P_{n}]$ [48], where
$\mathcal{P}_{n}=e^{-\langle\hat{a}^{\dagger}\hat{a}\rangle}\langle\hat{a}^{\dagger}\hat{a}\rangle^{n}/n!$
is the standard Poisson statistics. In the case of phonons, the sample mean of
the distribution is replaced by $\langle\hat{b}^{\dagger}\hat{b}\rangle$, and
$P_{n}$ is the photon (phonon) number distribution, which can be found from
the transition probabilities of Eq.(5)-(9). If $P_{n}<\mathcal{P}_{n}$, we say
the number distribution follows sub-Poissonian statistics otherwise it is
called super-Poissonian in nature. For numerical representation, the number
states are truncated to three in Fig.6(c)-6(d) for both the resonant
frequencies of the system and it is observed the relative population located
at $\Delta=-10\kappa$ and $-20\kappa$ are highly suppressed for $n\geq 2$.
This confirms that the photon (phonon) tends to exist singly, revealing the
sub-Poissonian statistics. Therefore it concludes our demonstration of the
antibunching effect deduced from the statistics of the coherent fields
incident on the OM system.

Figure 6: (a) and (b) Photon (phonon) number distribution versus cavity
detuning $\Delta/\kappa$ with red (blue) curves for mechanical frequencies
$\omega_{m1}=10\kappa$ and $\omega_{m2}=20\kappa$ depicting the number
squeezing effect in resonance conditions. (c) and (d) Relative deviation of
the photon (phonon) distribution to the standard Poissonian distribution with
the sample mean particle numbers computed at the respective resonance
frequencies.

##  IV Experimental feasibility

This section presents some remarks on the experimental prospect of the
nonclassical signatures based on the photon-phonon antibunching effect in a
membrane-in-the-middle optomechanical setup. The resolved-sideband parameters
used in the theoretical investigation belong to the experimentally accessible
parameter regimes i.e. $\omega_{m}>\kappa>\gamma$ [49, 50], where we tweak the
optomechanical coupling strength $g<\kappa$, as the single-photon strong
coupling is generally difficult to achieve. The possible realization of the
mechanical membrane is typically done by SiN nanoscale trampolines [51] or
photonic crystal nanobeam resonators [52] having high-quality factor
$\omega_{m}/\gamma>10^{6}$ and low thermal occupation numbers $n_{th}\ll 1$ at
cryogenic temperatures. For instance, the choice of parameters in our scheme
is taken accordingly Ref.[53, 54], where the vibrational mode frequency of
$\omega_{m}\approx 400$ MHz with damping rate $\gamma\approx 10^{-3}$ MHz and
the cavity linewidth $\kappa\approx 0.1$ MHz is achieved. The maximum
optomechanical coupling rate of $g\approx 0.05$ MHz range is observed with the
ultrahigh Q toroid microcavity [55]. The detection protocol deals with the
measurement of the photon-phonon sub-Poissonian field which can be done by a
Hanbury-Brown-Twiss-type experiment [41, 42] for determining the second-order
coherence function. Current experiments based on CS [52] and CHSH [56]
violation are reported in different OM platforms, which can be combined in the
current setup, satisfying all requirements of realistic parameters for the
demonstration of antibunching and related nonclassical effects.

##  V Summary

In summary, we have investigated the antibunching effect in a multifield-
driven optomechanical cavity that leads to violating Cauchy-Schwarz and CHSH
inequality together. The unconventional photon (phonon) blockade effectively
manifests in the current system while operating in the resonance condition
between the optical cavity detuning and the mechanical membranes’ vibrating
frequency. Generation of these highly nonclassical states doesn’t require a
single-photon-strong coupling, and the chosen parameters and techniques
involved in our protocol are fully feasible under the current experimental
conditions. Overall, the physical mechanism of the proposed theory largely
depends on the number-squeezing of coherent fields and its connection to the
classical violation of the general properties of second-order coherence
functions. Moreover, these results can be generalized to a wide range of
hybrid systems leading to a valid test of quantumness with currently available
technology.

##  Appendix A CHSH violation

In the appendix. we perform the CHSH test on the multifield-driven
optomechanical cavity, which put forward the CS inequality violation as a
precursor to check stronger verification of non-locality. Protocols similar to
the other quantum optical systems are considered for this, where the field
intensity measurement of the two modes $\hat{a}$ and $\hat{b}$ upon mixing
with local oscillators (LO) are done in four detectors resulting in two
outcomes per measurement. In the current scenario, the two modes correspond to
one optical and another mechanical mode. However, the measurement of the
mechanical mode is considered difficult to implement, therefore, the phononic
state can be suitably mapped to a photonic state by an optomechanical
parametric process [57] that converts the antibunched photon-phonon pair into
an output mode of photon-photon pair. To implement this, first, the membrane
has to be cool down near the quantum ground state of motion. The OM cavity is
tuned corresponding to the stoke’s sideband, which generates the photon-phonon
pair upon spontaneous parametric down-conversion. In this case, the
interaction Hamiltonian is described by
$\mathcal{H}_{int}=-G\hat{a}^{\dagger}\hat{b}^{\dagger}+\text{h.c}$, where $G$
is the effective OM coupling strength depends upon intracavity photon number
and single-photon coupling strength $g$. Next, the laser detuning is changed
to the red sideband to stimulate the anti-stoke process that initiates the
state transfer mechanism from the phononic to the photonic mode. This mapping
can be realized by the interaction Hamiltonian
$\mathcal{H}_{int}=-G\hat{a}^{\dagger}\hat{b}+\text{h.c}$. Now the resulting
output fields that are generated from the optomechanical parametric
interactions can be analyzed via photodetector intensities given by, $\langle
I_{A_{+}}\rangle$, $\langle I_{A_{-}}\rangle$, $\langle I_{B_{+}}\rangle$, and
$\langle I_{B_{-}}\rangle$ with adjustable polarization angles $\theta$ and
$\phi$ of the LO. The expectations of the intensity correlations is measured
followed by a beam-splitter transformation of the original modes given as

$\displaystyle\begin{pmatrix}A_{+}\\\
A_{-}\end{pmatrix}=\begin{pmatrix}\cos{\theta}&&\sin{\theta}\\\
-\sin{\theta}&&\cos{\theta}\end{pmatrix}\begin{pmatrix}a\\\ b\end{pmatrix}$
(13) $\displaystyle\begin{pmatrix}B_{+}\\\
B_{-}\end{pmatrix}=\begin{pmatrix}\cos{\phi}&&-\sin{\phi}\\\
\sin{\phi}&&\cos{\phi}\end{pmatrix}\begin{pmatrix}a\\\ b\end{pmatrix}$ (14)

Finally, the four correlated pairs of the field intensity measurements i.e.
$\langle I_{A_{+}}I_{B_{+}}\rangle$, $\langle I_{A_{+}}I_{B_{-}}\rangle$,
$\langle I_{A_{-}}I_{B_{+}}\rangle$ and $\langle I_{A_{-}}I_{B_{-}}\rangle$
are done using a Mach-Zehnder interferometer and the following CHSH parameter
is estimated

$\displaystyle E(\theta,\phi)$ $\displaystyle=$ $\displaystyle\frac{\langle
I_{A_{+}}I_{B_{+}}\rangle+\langle I_{A_{-}}I_{B_{-}}\rangle-\langle
I_{A_{+}}I_{B_{-}}\rangle-\langle I_{A_{-}}I_{B_{+}}\rangle}{\langle
I_{A_{+}}I_{B_{+}}\rangle+\langle I_{A_{-}}I_{B_{-}}\rangle+\langle
I_{A_{+}}I_{B_{-}}\rangle+\langle I_{A_{-}}I_{B_{+}}\rangle}$ (15)
$\displaystyle=$
$\displaystyle\frac{\langle(I_{A_{+}}-I_{A_{-}})(I_{B_{+}}-I_{B_{-}})\rangle}{\langle(I_{A_{+}}+I_{A_{-}})(I_{B_{+}}+I_{B_{-}})\rangle}$
$\displaystyle=$
$\displaystyle\frac{\langle:(A^{\dagger}_{+}A_{+}-A^{\dagger}_{-}A_{-})(B^{\dagger}_{+}B_{+}-B^{\dagger}_{-}B_{-}):\rangle}{\langle:(A^{\dagger}_{+}A_{+}+A^{\dagger}_{-}A_{-})(B^{\dagger}_{+}B_{+}+B^{\dagger}_{-}B_{-}):\rangle}\quad\text{($\langle::\rangle\equiv$
normal ordering of operators)}$

Measurement of the first outcome is

$\displaystyle A^{\dagger}_{+}A_{+}+A^{\dagger}_{-}A_{-}$ $\displaystyle=$
$\displaystyle\begin{pmatrix}\hat{a}^{\dagger}&\hat{b}^{\dagger}\end{pmatrix}\begin{pmatrix}\cos{\theta}&\sin{\theta}\\\
-\sin{\theta}&\cos{\theta}\end{pmatrix}$
$\displaystyle\times\begin{pmatrix}\cos{\theta}&\sin{\theta}\\\
-\sin{\theta}&\cos{\theta}\end{pmatrix}^{T}\begin{pmatrix}\hat{a}^{\dagger}\\\
\hat{b}^{\dagger}\end{pmatrix}=\hat{a}^{\dagger}\hat{a}+\hat{b}^{\dagger}\hat{b}$

Similarly, for the second measurement

$\displaystyle B^{\dagger}_{+}B_{+}+B^{\dagger}_{-}B_{-}$ $\displaystyle=$
$\displaystyle\begin{pmatrix}\hat{a}^{\dagger}&\hat{b}^{\dagger}\end{pmatrix}\begin{pmatrix}\cos{\phi}&-\sin{\phi}\\\
\sin{\phi}&\cos{\phi}\end{pmatrix}$
$\displaystyle\times\begin{pmatrix}\cos{\phi}&-\sin{\phi}\\\
\sin{\phi}&\cos{\phi}\end{pmatrix}^{T}\begin{pmatrix}\hat{a}^{\dagger}\\\
\hat{b}^{\dagger}\end{pmatrix}=\hat{a}^{\dagger}\hat{a}+\hat{b}^{\dagger}\hat{b}$

The denominator of $E(\theta,\phi)$ is computed as

$\displaystyle\langle:(A^{\dagger}_{+}A_{+}+A^{\dagger}_{-}A_{-})(B^{\dagger}_{+}B_{+}+B^{\dagger}_{-}B_{-}):\rangle=\langle:(\hat{a}^{\dagger}\hat{a}+\hat{b}^{\dagger}\hat{b})^{2}:\rangle$
$\displaystyle=\langle\hat{a}^{\dagger
2}\hat{a}^{2}\rangle+\langle\hat{b}^{\dagger
2}\hat{b}^{2}\rangle+2\langle\hat{a}^{\dagger}\hat{b}^{\dagger}\hat{b}\hat{a}\rangle$

For the third measurement

$\displaystyle A^{\dagger}_{+}A_{+}-A^{\dagger}_{-}A_{-}$ $\displaystyle=$
$\displaystyle\begin{pmatrix}\hat{a}^{\dagger}&\hat{b}^{\dagger}\end{pmatrix}\begin{pmatrix}\cos{\theta}&\sin{\theta}\\\
\sin{\theta}&-\cos{\theta}\end{pmatrix}$
$\displaystyle\times\begin{pmatrix}\cos{\theta}&\sin{\theta}\\\
-\sin{\theta}&\cos{\theta}\end{pmatrix}\begin{pmatrix}\hat{a}^{\dagger}\\\
\hat{b}^{\dagger}\end{pmatrix}$ $\displaystyle=$
$\displaystyle(\hat{a}^{\dagger}\hat{a}-\hat{b}^{\dagger}\hat{b})\cos{2\theta}+(\hat{a}^{\dagger}\hat{b}+\hat{b}^{\dagger}\hat{a})\sin{2\theta}$

Finally, the last measurement is

$\displaystyle B^{\dagger}_{+}B_{+}-B^{\dagger}_{-}B_{-}$ $\displaystyle=$
$\displaystyle\begin{pmatrix}\hat{a}^{\dagger}&\hat{b}^{\dagger}\end{pmatrix}\begin{pmatrix}\cos{\phi}&-\sin{\phi}\\\
-\sin{\phi}&-\cos{\phi}\end{pmatrix}$
$\displaystyle\times\begin{pmatrix}\cos{\phi}&-\sin{\phi}\\\
\sin{\phi}&\cos{\phi}\end{pmatrix}\begin{pmatrix}\hat{a}^{\dagger}\\\
\hat{b}^{\dagger}\end{pmatrix}$ $\displaystyle=$
$\displaystyle(\hat{a}^{\dagger}\hat{a}-\hat{b}^{\dagger}\hat{b})\cos{2\phi}-(\hat{a}^{\dagger}\hat{b}+\hat{b}^{\dagger}\hat{a})\sin{2\phi}$

The numerator of $E(\theta,\phi)$ is computed as

$\displaystyle\langle:(A^{\dagger}_{+}A_{+}-A^{\dagger}_{-}A_{-})(B^{\dagger}_{+}B_{+}-B^{\dagger}_{-}B_{-}):\rangle$
$\displaystyle=\Bigl{\langle}:[(\hat{a}^{\dagger}\hat{a}-\hat{b}^{\dagger}\hat{b})\cos{2\theta}+(\hat{a}^{\dagger}\hat{b}+\hat{b}^{\dagger}\hat{a})\sin{2\theta}]$
$\displaystyle\times[(\hat{a}^{\dagger}\hat{a}-\hat{b}^{\dagger}\hat{b})\cos{2\phi}-(\hat{a}^{\dagger}\hat{b}+\hat{b}^{\dagger}\hat{a})\sin{2\phi}]:\Bigr{\rangle}$

Therefore, Eq.(15) is simplified as

$\displaystyle E(\theta,\phi)$ $\displaystyle=$
$\displaystyle\frac{1}{\langle\hat{a}^{\dagger
2}\hat{a}^{2}\rangle+\langle\hat{b}^{\dagger
2}\hat{b}^{2}\rangle+2\langle\hat{a}^{\dagger}\hat{b}^{\dagger}\hat{b}\hat{a}\rangle}\times\biggr{[}\langle(\hat{a}^{\dagger}\hat{a}-\hat{b}^{\dagger}\hat{b})^{2}\rangle\cos{2\theta}\cos{2\phi}-\langle(\hat{a}^{\dagger}\hat{b}+\hat{b}^{\dagger}\hat{a})^{2}\rangle\times\sin{2\theta}\sin{2\phi}$
$\displaystyle-\langle\hat{a}^{\dagger}\hat{a}-\hat{b}^{\dagger}\hat{b}\rangle\langle\hat{a}^{\dagger}\hat{b}+\hat{b}^{\dagger}\hat{a}\rangle\cos{2\theta}\sin{2\phi}+\langle\hat{a}^{\dagger}\hat{b}+\hat{b}^{\dagger}\hat{a}\rangle\langle\hat{a}^{\dagger}\hat{a}-\hat{b}^{\dagger}\hat{b}\rangle\sin{2\theta}\cos{2\phi}\biggr{]}$

The CHSH inequality can be tested by violating the following expression which
falsifies the idea of local realism, given as

$\displaystyle\mathcal{B}=\absolutevalue{E(\theta,\phi)+E(\theta^{\prime},\phi^{\prime})+E(\theta^{\prime},\phi)-E(\theta,\phi^{\prime})}\leq
2$ (17)

with the maximum violation can be obtained by the standard choice of
polarization angles chosen as
$\theta=0,\phi=\pi/8,\theta^{\prime}=\pi/4,\phi^{\prime}=3\pi/8$. In terms of
the original OM modes Eq.(17) can be rewritten as

$\displaystyle\mathcal{B}=\sqrt{2}\absolutevalue{\frac{\langle\hat{a}^{\dagger
2}\hat{a}^{2}\rangle+\langle\hat{b}^{\dagger
2}\hat{b}^{2}\rangle-\langle\hat{a}^{\dagger
2}\hat{b}^{2}\rangle-\langle\hat{b}^{\dagger
2}\hat{a}^{2}\rangle-4\langle\hat{a}^{\dagger}\hat{b}^{\dagger}\hat{b}\hat{a}\rangle}{\langle\hat{a}^{\dagger
2}\hat{a}^{2}\rangle+\langle\hat{b}^{\dagger
2}\hat{b}^{2}\rangle+2\langle\hat{a}^{\dagger}\hat{b}^{\dagger}\hat{b}\hat{a}\rangle}}$
(18)

It is now straightforward to calculate the CHSH violation factor $\mathcal{B}$
by replacing the expectation values of the operators $\hat{a}$ and $\hat{b}$
with the transition probabilities obtained from Eq.(5)-(9). The final
expression of $\mathcal{B}$ is given by

$\displaystyle\mathcal{B}=\frac{1}{\sqrt{2}}\absolutevalue{\frac{2\absolutevalue{C_{20}}^{2}+2\absolutevalue{C_{02}}^{2}-4\absolutevalue{C_{20}}\absolutevalue{C_{20}}-4\absolutevalue{C_{11}}^{2}}{\absolutevalue{C_{20}}^{2}+\absolutevalue{C_{02}}^{2}+\absolutevalue{C_{11}}^{2}}}$
(19)

## References

  * Agarwal [2012] G. S. Agarwal,  _Quantum optics_ (Cambridge University Press, 2012).
  * Bücker  _et al._ [2011] R. Bücker, J. Grond, S. Manz, T. Berrada, T. Betz, C. Koller, U. Hohenester, T. Schumm, A. Perrin, and J. Schmiedmayer, Twin-atom beams, Nature Physics 7, 608 (2011).
  * Marino  _et al._ [2008] A. M. Marino, V. Boyer, and P. D. Lett, Violation of the cauchy-schwarz inequality in the macroscopic regime, Physical review letters 100, 233601 (2008).
  * Luo  _et al._ [2017] R. Luo, H. Jiang, S. Rogers, H. Liang, Y. He, and Q. Lin, On-chip second-harmonic generation and broadband parametric down-conversion in a lithium niobate microresonator, Optics express 25, 24531 (2017).
  * Kimble  _et al._ [1977] H. J. Kimble, M. Dagenais, and L. Mandel, Photon antibunching in resonance fluorescence, Phys. Rev. Lett. 39, 691 (1977).
  * Kheruntsyan  _et al._ [2012] K. Kheruntsyan, J.-C. Jaskula, P. Deuar, M. Bonneau, G. B. Partridge, J. Ruaudel, R. Lopes, D. Boiron, and C. I. Westbrook, Violation of the cauchy-schwarz inequality with matter waves, Physical review letters 108, 260401 (2012).
  * Steele [2004] J. Steele,  _The Cauchy-Schwarz Master Class: An Introduction to the Art of Mathematical Inequalities_, MAA problem books series (Cambridge University Press, 2004).
  * Reid and Walls [1986] M. Reid and D. Walls, Violations of classical inequalities in quantum optics, Physical Review A 34, 1260 (1986).
  * Ansari and Zubairy [1988] N. A. Ansari and M. S. Zubairy, Violation of cauchy-schwarz and bell’s inequalities in four-wave mixing, Physical Review A 38, 2380 (1988).
  * Muñoz  _et al._ [2014] C. S. Muñoz, E. del Valle, C. Tejedor, and F. P. Laussy, Violation of classical inequalities by photon frequency filtering, Physical Review A 90, 052111 (2014).
  * Wu and Xiao [2009] H. Wu and M. Xiao, Bright correlated twin beams from an atomic ensemble in the optical cavity, Physical Review A 80, 063415 (2009).
  * Trif and Simon [2015] M. Trif and P. Simon, Photon cross-correlations emitted by a josephson junction in two microwave cavities, Physical Review B 92, 014503 (2015).
  * Araújo  _et al._ [2022] M. O. Araújo, L. S. Marinho, and D. Felinto, Observation of nonclassical correlations in biphotons generated from an ensemble of pure two-level atoms, Physical Review Letters 128, 083601 (2022).
  * Fan  _et al._ [2023] Y. Fan, J. Li, and Y. Wu, Nonclassical magnon pair generation and cauchy-schwarz inequality violation, Physical Review A 108, 053715 (2023).
  * de Nova  _et al._ [2014] J. M. de Nova, F. Sols, and I. Zapata, Violation of cauchy-schwarz inequalities by spontaneous hawking radiation in resonant boson structures, Physical Review A 89, 043808 (2014).
  * Clauser  _et al._ [1969] J. F. Clauser, M. A. Horne, A. Shimony, and R. A. Holt, Proposed experiment to test local hidden-variable theories, Physical review letters 23, 880 (1969).
  * Bell [2004] J. S. Bell,  _Speakable and unspeakable in quantum mechanics: Collected papers on quantum philosophy_ (Cambridge university press, 2004).
  * Einstein  _et al._ [1935] A. Einstein, B. Podolsky, and N. Rosen, Can quantum-mechanical description of physical reality be considered complete?, Physical review 47, 777 (1935).
  * Hensen  _et al._ [2015] B. Hensen, H. Bernien, A. E. Dréau, A. Reiserer, N. Kalb, M. S. Blok, J. Ruitenberg, R. F. Vermeulen, R. N. Schouten, C. Abellán, _et al._ , Loophole-free bell inequality violation using electron spins separated by 1.3 kilometres, Nature 526, 682 (2015).
  * Palacios-Laloy  _et al._ [2010] A. Palacios-Laloy, F. Mallet, F. Nguyen, P. Bertet, D. Vion, D. Esteve, and A. N. Korotkov, Experimental violation of a bell’s inequality in time with weak measurement, Nature Physics 6, 442 (2010).
  * Storz  _et al._ [2023] S. Storz, J. Schär, A. Kulikov, P. Magnard, P. Kurpiers, J. Lütolf, T. Walter, A. Copetudo, K. Reuer, A. Akin, _et al._ , Loophole-free bell inequality violation with superconducting circuits, Nature 617, 265 (2023).
  * Ansmann  _et al._ [2009] M. Ansmann, H. Wang, R. C. Bialczak, M. Hofheinz, E. Lucero, M. Neeley, A. D. O’Connell, D. Sank, M. Weides, J. Wenner, _et al._ , Violation of bell’s inequality in josephson phase qubits, Nature 461, 504 (2009).
  * Aspelmeyer  _et al._ [2014] M. Aspelmeyer, T. J. Kippenberg, and F. Marquardt, Cavity optomechanics, Reviews of Modern Physics 86, 1391 (2014).
  * Guo  _et al._ [2019] J. Guo, R. Norte, and S. Gröblacher, Feedback cooling of a room temperature mechanical oscillator close to its motional ground state, Physical review letters 123, 223602 (2019).
  * Dobrindt  _et al._ [2008] J. M. Dobrindt, I. Wilson-Rae, and T. J. Kippenberg, Parametric normal-mode splitting in cavity optomechanics, Physical Review Letters 101, 263602 (2008).
  * Vitali  _et al._ [2007] D. Vitali, S. Gigan, A. Ferreira, H. Böhm, P. Tombesi, A. Guerreiro, V. Vedral, A. Zeilinger, and M. Aspelmeyer, Optomechanical entanglement between a movable mirror and a cavity field, Physical review letters 98, 030405 (2007).
  * Ghosh  _et al._ [2024] J. Ghosh, S. Mondal, S. K. Varshney, and K. Debnath, Simultaneous control of quantum phase synchronization and entanglement dynamics in a gain-loss optomechanical cavity system, Phys. Rev. A 109, 023512 (2024).
  * Barzanjeh  _et al._ [2022] S. Barzanjeh, A. Xuereb, S. Gröblacher, M. Paternostro, C. A. Regal, and E. M. Weig, Optomechanics for quantum technologies, Nature Physics 18, 15 (2022).
  * Humphries  _et al._ [2023] B. S. Humphries, D. Green, M. O. Borgh, and G. A. Jones, Phonon signatures in photon correlations, Physical Review Letters 131, 143601 (2023).
  * Liew and Savona [2010] T. C. H. Liew and V. Savona, Single photons from coupled quantum modes, Phys. Rev. Lett. 104, 183601 (2010).
  * Wang  _et al._ [2015] H. Wang, X. Gu, Y.-x. Liu, A. Miranowicz, and F. Nori, Tunable photon blockade in a hybrid system consisting of an optomechanical device coupled to a two-level system, Physical Review A 92, 033806 (2015).
  * Wang  _et al._ [2020] D.-Y. Wang, C.-H. Bai, X. Han, S. Liu, S. Zhang, and H.-F. Wang, Enhanced photon blockade in an optomechanical system with parametric amplification, Optics Letters 45, 2604 (2020).
  * Huang  _et al._ [2018] R. Huang, A. Miranowicz, J.-Q. Liao, F. Nori, and H. Jing, Nonreciprocal photon blockade, Physical review letters 121, 153601 (2018).
  * Wang  _et al._ [2019] D.-Y. Wang, C.-H. Bai, S. Liu, S. Zhang, and H.-F. Wang, Distinguishing photon blockade in a pt-symmetric optomechanical system, Physical Review A 99, 043818 (2019).
  * Liao  _et al._ [2013] J.-Q. Liao, F. Nori, _et al._ , Photon blockade in quadratically coupled optomechanical systems, Physical Review A 88, 023853 (2013).
  * Xie  _et al._ [2017] H. Xie, C.-G. Liao, X. Shang, M.-Y. Ye, and X.-M. Lin, Phonon blockade in a quadratically coupled optomechanical system, Physical Review A 96, 013861 (2017).
  * Aldana  _et al._ [2013] S. Aldana, C. Bruder, and A. Nunnenkamp, Equivalence between an optomechanical system and a kerr medium, Physical Review A 88, 043826 (2013).
  * Bamba  _et al._ [2011] M. Bamba, A. Imamoğlu, I. Carusotto, and C. Ciuti, Origin of strong photon antibunching in weakly nonlinear photonic molecules, Physical Review A 83, 021802 (2011).
  * Xu  _et al._ [2015] X.-W. Xu, Y.-x. Liu, C.-P. Sun, and Y. Li, Mechanical pt symmetry in coupled optomechanical systems, Physical Review A 92, 013852 (2015).
  * Miranowicz  _et al._ [2010] A. Miranowicz, M. Bartkowiak, X. Wang, Y.-x. Liu, and F. Nori, Testing nonclassicality in multimode fields: A unified derivation of classical inequalities, Physical Review A 82, 013824 (2010).
  * Brown and Twiss [1956] R. H. Brown and R. Q. Twiss, Correlation between photons in two coherent beams of light, Nature 177, 27 (1956).
  * Cohen  _et al._ [2015] J. D. Cohen, S. M. Meenehan, G. S. MacCabe, S. Gröblacher, A. H. Safavi-Naeini, F. Marsili, M. D. Shaw, and O. Painter, Phonon counting and intensity interferometry of a nanomechanical resonator, Nature 520, 522 (2015).
  * Lörch and Hammerer [2015] N. Lörch and K. Hammerer, Sub-poissonian phonon lasing in three-mode optomechanics, Physical Review A 91, 061803 (2015).
  * Lemonde  _et al._ [2014] M.-A. Lemonde, N. Didier, and A. A. Clerk, Antibunching and unconventional photon blockade with gaussian squeezed states, Physical Review A 90, 063824 (2014).
  * Nunnenkamp  _et al._ [2011] A. Nunnenkamp, K. Børkje, and S. M. Girvin, Single-photon optomechanics, Physical review letters 107, 063602 (2011).
  * Huang and Agarwal [2010] S. Huang and G. Agarwal, Normal-mode splitting and antibunching in stokes and anti-stokes processes in cavity optomechanics: radiation-pressure-induced four-wave-mixing cavity optomechanics, Physical Review A 81, 033830 (2010).
  * Wasak  _et al._ [2014] T. Wasak, P. Szańkowski, P. Ziń, M. Trippenbach, and J. Chwedeńczuk, Cauchy-schwarz inequality and particle entanglement, Physical Review A 90, 033616 (2014).
  * Zhao  _et al._ [2020] C. Zhao, X. Li, S. Chao, R. Peng, C. Li, and L. Zhou, Simultaneous blockade of a photon, phonon, and magnon induced by a two-level atom, Physical Review A 101, 063838 (2020).
  * Thompson  _et al._ [2008] J. Thompson, B. Zwickl, A. Jayich, F. Marquardt, S. Girvin, and J. Harris, Strong dispersive coupling of a high-finesse cavity to a micromechanical membrane, Nature 452, 72 (2008).
  * Sankey  _et al._ [2010] J. C. Sankey, C. Yang, B. M. Zwickl, A. M. Jayich, and J. G. Harris, Strong and tunable nonlinear optomechanical coupling in a low-loss system, Nature Physics 6, 707 (2010).
  * Wei  _et al._ [2019] X. Wei, J. Sheng, C. Yang, Y. Wu, and H. Wu, Controllable two-membrane-in-the-middle cavity optomechanical system, Physical Review A 99, 023851 (2019).
  * Riedinger  _et al._ [2016] R. Riedinger, S. Hong, R. A. Norte, J. A. Slater, J. Shang, A. G. Krause, V. Anant, M. Aspelmeyer, and S. Gröblacher, Non-classical correlations between single photons and phonons from a mechanical oscillator, Nature 530, 313 (2016).
  * Xie  _et al._ [2018] H. Xie, C.-G. Liao, X. Shang, Z.-H. Chen, and X.-M. Lin, Optically induced phonon blockade in an optomechanical system with second-order nonlinearity, Physical Review A 98, 023819 (2018).
  * Yin  _et al._ [2017] T.-S. Yin, X.-Y. Lü, L.-L. Zheng, M. Wang, S. Li, and Y. Wu, Nonlinear effects in modulated quantum optomechanics, Physical Review A 95, 053861 (2017).
  * Armani  _et al._ [2003] D. Armani, T. Kippenberg, S. Spillane, and K. Vahala, Ultra-high-q toroid microcavity on a chip, Nature 421, 925 (2003).
  * Marinković  _et al._ [2018] I. Marinković, A. Wallucks, R. Riedinger, S. Hong, M. Aspelmeyer, and S. Gröblacher, Optomechanical bell test, Physical review letters 121, 220404 (2018).
  * Vivoli  _et al._ [2016] V. C. Vivoli, T. Barnea, C. Galland, and N. Sangouard, Proposal for an optomechanical bell test, Physical review letters 116, 070405 (2016).


#  pose-format: Library for Viewing, Augmenting, and Handling .pose Files

Amit Moryossef
University of Zürich
<EMAIL_ADDRESS>
Mathias Müller
University of Zürich
<EMAIL_ADDRESS>

https://github.com/sign-language-processing/pose

Rebecka Fahrni
University of Zürich
<EMAIL_ADDRESS>
###### Abstract

Managing and analyzing pose data is a complex task, with challenges ranging
from handling diverse file structures and data types to facilitating effective
data manipulations such as normalization and augmentation. This paper presents
pose-format, a comprehensive toolkit designed to address these challenges by
providing a unified, flexible, and easy-to-use interface. The library includes
a specialized file format that encapsulates various types of pose data,
accommodating multiple individuals and an indefinite number of time frames,
thus proving its utility for both image and video data. Furthermore, it offers
seamless integration with popular numerical libraries such as NumPy, PyTorch,
and TensorFlow, thereby enabling robust machine-learning applications. Through
benchmarking, we demonstrate that our .pose file format offers vastly superior
performance against prevalent formats like OpenPose, with added advantages
like self-contained pose specification. Additionally, the library includes
features for data normalization, augmentation, and easy-to-use visualization
capabilities, both in Python and Browser environments. pose-format emerges as
a one-stop solution, streamlining the complexities of pose data management and
analysis.

##  1 Introduction

Working with pose data introduces many complexities, from the diversity in
file structures to the variety of data types that need to be accommodated.
Developers and researchers often find themselves juggling numerous data
manipulation tasks such as normalization, augmentation, and visualization. In
addition to these challenges, pose data itself can be inherently
multidimensional, frequently encompassing multiple individuals and varying
time frames. This creates an intricate ecosystem of variables that can be
challenging to manage and analyze effectively, which is particularly important
in fields like Sign Language Processing.

To overcome these complexities, we designed pose-format, a comprehensive
toolkit to alleviate these challenges by offering a unified, flexible, and
easy-to-use interface for managing and analyzing pose data. Designed with
versatility in mind, the library includes a specialized file format that
accommodates an array of pose types, multiple people, and an indefinite number
of time frames, making it highly adaptable for both video and single-frame
data. Users can effortlessly import .pose files and perform a range of
manipulations such as data normalization and augmentation. The library also
integrates seamlessly with popular numerical libraries like NumPy (Harris et
al., 2020), PyTorch (Paszke et al., 2019), and TensorFlow (Abadi et al.,
2015), allowing for additional computational flexibility for machine learning.
With features for easy visualization and compatibility with other popular pose
data formats like OpenPose (Cao et al., 2019) and MediaPipe Holistic
(Grishchenko and Bazarevsky, 2020), the pose-format library emerges as a one-
stop solution for all pose data management needs.

Figure 1: Examples of human skeletal poses extracted from a sign language
video sequence.

##  2 Background

In the context of our library, a _pose_ consists of _keypoints_ , which are 2D
or 3D coordinates marking points of interest usually on a human body in image
or video frames (Figure 1). Systems like OpenPose and MediaPipe Holistic are
prominent for pose estimation but have differing methodologies and keypoint
configurations. OpenPose, for instance, uses a classification objective and
outputs 135 or 137 keypoints with 2D coordinates. MediaPipe Holistic employs a
regression objective, estimating 543 keypoints with 3D coordinates.

Keypoints are hierarchically organized, often attached to larger body
components like LEFT HAND or FACE. Moreover, models implicitly define which
keypoints are connected, forming an underlying graph structure. Confidence
metrics vary across systems. OpenPose assigns a confidence score to each
classification, while MediaPipe Holistic only predicts the likelihood of each
BODY keypoint’s presence in the original image.

The utility of human pose estimation (Zheng et al., 2023) spans various fields
such as human-computer interaction, motion capture, motion analysis, and mixed
reality, with specialized applications like automatic sign language processing
(Moryossef et al., 2021; Müller et al., 2022).

##  3 Justification

pose-format addresses a void in the ecosystem by delivering a uniform layer of
abstraction over disparate pose estimation system outputs, such as OpenPose
and MediaPipe Holistic. The necessity for this unified interface arises from
three primary factors: inconsistent standards, inadequacy of existing
libraries, and performance bottlenecks.

#### Inconsistent Standards

As delineated in §2, there are competing pose estimation systems, each
adhering to its own data storage and representation scheme. This inconsistency
impedes interoperability between systems and makes the data hard to share or
transition across different platforms. pose-format remedies this by
standardizing how pose data is managed, making it simpler to operate with
multiple systems, switch between them, or even disseminate pose data.

#### Limitations of Existing Libraries

Current libraries focus extensively on low-level operations, lacking the
higher-level abstractions that can expedite routine tasks. For instance, in
the absence of our toolkit, users have to micromanage array values, discerning
between coordinates and confidence scores or handling missing keypoints. Such
intricacies detract from productivity and introduce unnecessary complexity.
Our library fills this gap by offering user-friendly operations, many of which
are indispensable for machine learning research, such as frame rate
interpolation, rotation, scaling, frame dropout, or converting the underlying
data into tensors of a specific machine learning library.

#### Efficiency

As demonstrated in §7, prevailing methods for pose data management suffer from
performance limitations in both speed and storage. These inefficiencies create
bottlenecks for data-intensive tasks, especially those prevalent in machine
learning pipelines. pose-format offers optimized data storage and retrieval,
mitigating these inefficiencies.

##  4 Format Specification

The core of the pose-format library is its specialized file format that
accommodates a wide range of scenarios. This unique format enables the storing
of multidimensional data capturing various pose types, multiple individuals,
and an indefinite number of time frames. Currently, at version $0.1$, the file
format is bifurcated into two components: the Header and the Body.

###  4.1 Header (PoseHeader)

The header contains meta information that defines the overall structure of the
pose data. This information is useful for visualization and code readability.
Specifically, it includes:

#### (float32) Version

The version of the file format.

#### (uint16[3]) Dimensions

Width, height, and depth specifications.

#### (uint16) Number of Components

The number of pose components.

#### Component Details

Each component includes its (string) name, (string) format, and the (uint16)
number of points, (uint16) limbs, and (uint16) colors it contains.

  * •

(string[]) Names of points.

  * •

(uint16[2][]) Start and end indices of limbs.

  * •

(uint16[3][]) Points color RGB values.

###  4.2 Body (PoseBody)

The body of the file comprises the actual pose data and includes the
following:

#### (uint16) FPS

The frame rate of the pose.

#### (uint16) Number of frames

deprecated due to challenges for longer pose sequences.

#### (uint16) Number of People

The number of people included in every frame.

#### (float[][][][]) Data

The coordinate of every point for every person in every frame.

#### (float[][][]) Confidence

The confidence for every point for every person in every frame.

This format’s granularity and modularity make it aptly suited for a wide range
of applications, from simple image-based pose representation to more complex
video analysis tasks. By leveraging this detailed yet flexible format, the
pose-format library ensures ease of use without sacrificing the intricacies
that pose data often necessitates.

###  4.3 v0.1 Limitations

While the pose-format library has been designed to cater to a wide array of
needs, there are some limitations and criticisms in the current file format
that users should be aware of:

  * •

FPS Representation: The FPS is stored as uint16, which does not allow for
floating-point values.

  * •

Number of Frames: The number of frames is also restricted to uint16, which
limits the frame count to 65,535. The current workaround calculates the number
of frames based on the file size, which introduces computational overhead.

  * •

Pose Data Precision: The pose data utilizes 32-bit floating-point values for
storage. However, 16-bit floating-point numbers could be sufficient for many
applications. Support for both types would improve memory efficiency.

  * •

Confidence Precision: Similar to the pose data, the confidence metrics are
stored as 32-bit floating-point numbers. A 16-bit representation would be more
than sufficient for most practical purposes.

##  5 Data Manipulations

One of the key advantages of this toolkit is its robust support for various
data manipulation tasks, which are crucial for the preprocessing and
augmentation of pose data. This section elaborates on how the library
facilitates operations such as normalization and augmentation.

#### Normalization

Normalization is a crucial step to make pose data scale and translation
invariant, thereby improving the effectiveness of downstream tasks like
training machine learning models. Our toolkit offers a simple yet powerful
interface to normalize pose data. For example, when dealing with human body
poses, we can specify the names of the left and right shoulders, and the
skeleton will be scaled such that the mean distance between the shoulders is
equal to $1$, and the center point lies on $(0,0)$. If we deal with 3D poses,
we can also specify a plane by naming three points, to make sure they always
fall on the same plane. These normalizations remove the effect of camera
angles and distance from the subject.

#### Augmentation

Data augmentation is a technique to artificially increase the size and
diversity of your training dataset by applying various transformations. In the
context of pose data, these can include affine transformations such as
translation, scaling, reflection, rotation, and shear, interpolation of frames
at variable speeds, noise, and dropout, to name a few. The pose-format toolkit
provides built-in functions to perform these augmentations effortlessly. You
can either apply these transformations individually or chain them together to
create a complex augmentation pipeline, thereby enhancing the library’s
adaptability to various project needs.

#### Integration with Numerical Libraries

Data manipulations are seamlessly integrated with popular numerical libraries
like NumPy, PyTorch, and TensorFlow. This facilitates easy data flow between
data manipulation and machine learning models, reducing the friction in the
data science pipeline. It allows loading and augmenting the data in a
framework of your choosing, minimizing the number of memory copy operations.

# Frames | OpenPose | pose-format
---|---|---
| Size | Speed | Size | Speed | Speed (Body)
1 | $3.9\,\texttt{KB}$ | $37.4\,\mu\texttt{s}\pm 600\,\texttt{ns}$ | $3.6\,\texttt{KB}$ | $535\,\mu\texttt{s}\pm 66.1\,\mu\texttt{s}$ | $61.7\,\mu\texttt{s}\pm 6.94\,\mu\texttt{s}$
10 | $38\,\texttt{KB}$ | $364\,\mu\texttt{s}\pm 6.9\,\mu\texttt{s}$ | $18\,\texttt{KB}$ | $490\,\mu\texttt{s}\pm 63.8\,\mu\texttt{s}$ | $57.9\,\mu\texttt{s}\pm 2.56\,\mu\texttt{s}$
100 | $388\,\texttt{KB}$ | $3.75\,\texttt{ms}\pm 113\,\mu\texttt{s}$ | $163\,\texttt{KB}$ | $415\,\mu\texttt{s}\pm 49.7\,\mu\texttt{s}$ | $72.4\,\mu\texttt{s}\pm 4.87\,\mu\texttt{s}$
1,000 | $3.9\,\texttt{MB}$ | $43.1\,\texttt{ms}\pm 704\,\mu\texttt{s}$ | $1.6\,\texttt{MB}$ | $658\,\mu\texttt{s}\pm 110\,\mu\texttt{s}$ | $228\,\mu\texttt{s}\pm 9.09\,\mu\texttt{s}$
10,000 | $39\,\texttt{MB}$ | $439\,\texttt{ms}\pm 29.5\,\texttt{ms}$ | $16\,\texttt{MB}$ | $2.72\,\texttt{ms}\pm 110\,\mu\texttt{s}$ | $2.71\,\texttt{ms}\pm 245\,\mu\texttt{s}$
Table 1: Benchmarking pose-format against OpenPose from the Public DGS Corpus.
We compare both the resulting file size, and file read speed. _Speed (Body)_
measures loading the _.pose_ files data only, without metadata.

##  6 Visualization

The ability to visualize pose data is crucial for understanding its
characteristics, debugging algorithms, and even for presentation purposes.

#### Python

In Python, users can make use of the PoseVisualizer class for different
visualization tasks, such as visualizing the pose by itself as a sequence of
still images, a video, a GIF, with the background being either a fixed color
or overlayed on another video. An example of visualizing the pose as a video
would be:

from pose_format import Posefrom pose_format.pose_visualizer
import PoseVisualizerwith open("example.pose", "rb") as f: pose =
Pose.read(f.read())v = PoseVisualizer(pose)v.save_video("example.mp4",
v.draw())

#### Browser

Additionally, for web-based applications or quick interactive viewing, poses
can be visualized in the browser. Unlike the Python visualization, this
visualization is vectorized and is more suitable for client-facing
applications.

<script type="module"
<EMAIL_ADDRESS>
/dist/pose-viewer/pose-viewer.esm.js" /><pose-format src="example.pose" />

##  7 Benchmarking

To evaluate our custom file format, we benchmarked it against OpenPose, a
prevalent standard. Metrics of interest were read speed and file size. We
obtained OpenPose data from a single video in the Public DGS Corpus (Hanke et
al., 2020, DOI: /10.25592/dgs.corpus-3.0-text-1413451-11105600-11163240).
Their format employs a monolithic JSON file to store frames, diverging from
the common one-file-per-frame approach.

To gauge reading performance, we measured OpenPose’s JSON load time in
isolation, sidestepping tensor conversion. For our format, we include both
full-file reads and body-only tensor reads where we skip loading the pose
header, and only load the tensor of coordinates and confidences.

#### Quantitative Edge

Table 1 reveals we achieve up to a $60\%$ file size reduction and outpace
OpenPose in read speed by a staggering $162\times$, thereby obliterating any
machine learning bottlenecks.

#### Qualitative Edge

Our pose-format packs all pose data into a singular, robust file, avoiding the
file fragmentation issues seen in OpenPose. Moreover, our header encodes pose
structure, obviating the need for hard-coded interpretation logic and boosting
both portability and usability.

In summation, pose-format offers superior performance across key metrics,
making it a compelling alternative for pose data management.

##  8 Community Contributions

Our library is fully open-source, and released under an MIT License. We
welcome contributions from the community of any kind, and we encourage
collaboration. Source code and bug reporting are available at
https://github.com/sign-language-processing/pose.

## References

  * Abadi et al. (2015) Martín Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, Greg S. Corrado, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Ian Goodfellow, Andrew Harp, Geoffrey Irving, Michael Isard, Yangqing Jia, Rafal Jozefowicz, Lukasz Kaiser, Manjunath Kudlur, Josh Levenberg, Dandelion Mané, Rajat Monga, Sherry Moore, Derek Murray, Chris Olah, Mike Schuster, Jonathon Shlens, Benoit Steiner, Ilya Sutskever, Kunal Talwar, Paul Tucker, Vincent Vanhoucke, Vijay Vasudevan, Fernanda Viégas, Oriol Vinyals, Pete Warden, Martin Wattenberg, Martin Wicke, Yuan Yu, and Xiaoqiang Zheng. 2015.  TensorFlow: Large-scale machine learning on heterogeneous systems.  Software available from tensorflow.org.
  * Cao et al. (2019) Z. Cao, G. Hidalgo Martinez, T. Simon, S. Wei, and Y. A. Sheikh. 2019.  OpenPose: Realtime multi-person 2D pose estimation using part affinity fields.  _IEEE Transactions on Pattern Analysis and Machine Intelligence_.
  * Grishchenko and Bazarevsky (2020) Ivan Grishchenko and Valentin Bazarevsky. 2020.  Mediapipe holistic.
  * Hanke et al. (2020) Thomas Hanke, Marc Schulder, Reiner Konrad, and Elena Jahn. 2020.  Extending the Public DGS Corpus in size and depth.  In _Proceedings of the LREC2020 9th Workshop on the Representation and Processing of Sign Languages: Sign Language Resources in the Service of the Language Community, Technological Challenges and Application Perspectives_ , pages 75–82, Marseille, France. European Language Resources Association (ELRA).
  * Harris et al. (2020) Charles R. Harris, K. Jarrod Millman, Stéfan van der Walt, Ralf Gommers, Pauli Virtanen, David Cournapeau, Eric Wieser, Julian Taylor, Sebastian Berg, Nathaniel J. Smith, Robert Kern, Matti Picus, Stephan Hoyer, Marten Henric van Kerkwijk, Matthew Brett, Allan Haldane, Jaime Fern’andez del R’io, Marcy Wiebe, Pearu Peterson, Pierre G’erard-Marchant, Kevin Sheppard, Tyler Reddy, Warren Weckesser, Hameer Abbasi, Christoph Gohlke, and Travis E. Oliphant. 2020.  Array programming with numpy.  _Nature_ , 585:357 – 362.
  * Moryossef et al. (2021) Amit Moryossef, Ioannis Tsochantaridis, Joe Dinn, Necati Cihan Camgoz, Richard Bowden, Tao Jiang, Annette Rios, Mathias Muller, and Sarah Ebling. 2021.  Evaluating the immediate applicability of pose estimation for sign language recognition.  In _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , pages 3434–3440.
  * Müller et al. (2022) Mathias Müller, Sarah Ebling, Eleftherios Avramidis, Alessia Battisti, Michèle Berger, Richard Bowden, Annelies Braffort, Necati Cihan Camgöz, Cristina España-bonet, Roman Grundkiewicz, Zifan Jiang, Oscar Koller, Amit Moryossef, Regula Perrollaz, Sabine Reinhard, Annette Rios, Dimitar Shterionov, Sandra Sidler-miserez, and Katja Tissi. 2022.  Findings of the first WMT shared task on sign language translation (WMT-SLT22).  In _Proceedings of the Seventh Conference on Machine Translation (WMT)_ , pages 744–772, Abu Dhabi, United Arab Emirates (Hybrid). Association for Computational Linguistics.
  * Paszke et al. (2019) Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Edward Yang, Zach DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. 2019.  Pytorch: An imperative style, high-performance deep learning library.  In _Neural Information Processing Systems_.
  * Zheng et al. (2023) Ce Zheng, Wenhan Wu, Chen Chen, Taojiannan Yang, Sijie Zhu, Ju Shen, Nasser Kehtarnavaz, and Mubarak Shah. 2023.  Deep learning-based human pose estimation: A survey.  _ACM Comput. Surv._ , 56(1).