# Measurement of $\mathcal{R}(D)$ and $\mathcal{R}(D^{\ast})$ with a
semileptonic tagging method

G. Caria on behalf of the Belle Collaboration  School of Physics, University
of Melbourne, Victoria 3010

###### Abstract

We report a measurement of the ratios of branching fractions
$\mathcal{R}(D^{(\ast)})={\cal B}(\bar{B}\to
D^{(*)}\tau^{-}\bar{\nu}_{\tau})/{\cal B}(\bar{B}\to
D^{(*)}\ell^{-}\bar{\nu}_{\ell})$, where $\ell$ denotes an electron or a muon.
The results are based on a data sample containing $772\times 10^{6}$
$B\bar{B}$ events recorded at the $\Upsilon(4S)$ resonance with the Belle
detector at the KEKB $e^{+}e^{-}$ collider. The tag-side $B$ meson is
reconstructed in a semileptonic decay mode, and the signal-side $\tau$ is
reconstructed in a purely leptonic decay. The results are $\mathcal{R}(D)\
=0.307\pm 0.037\pm 0.016$ and $\mathcal{R}(D^{\ast})\ =0.283\pm 0.018\pm
0.014$, where the first uncertainties are statistical and the second are
systematic. These results are in agreement with the Standard Model predictions
within $0.2$ and $1.1$ standard deviations, respectively.

##  1 Introduction

Semitauonic $B$ meson decays of the type $b\to c\tau\nu_{\tau}$ [1] are
sensitive probes for physics beyond the Standard Model (SM). The ratio of
branching fractions $\mathcal{R}(D^{(\ast)})={\cal B}(\bar{B}\to
D^{(*)}\tau^{-}\bar{\nu}_{\tau})/{\cal B}(\bar{B}\to
D^{(*)}\ell^{-}\bar{\nu}_{\ell})$, where $\ell$ denotes an electron or a muon,
is typically measured instead of the absolute branching fraction of
$\bar{B}\to D^{(*)}\tau^{-}\bar{\nu}_{\tau}$ to reduce common systematic
uncertainties. Hereafter, $\bar{B}\to D^{(*)}\tau^{-}\bar{\nu}_{\tau}$ and
$\bar{B}\to D^{(*)}\ell^{-}\bar{\nu}_{\ell}$ will be referred to as the signal
and normalization modes, respectively. The SM calculations for these ratios,
performed by several groups [2][3, Bigi:2017jbd, Jaiswal:2017rve], are
averaged [4] to obtain $\mathcal{R}(D)=0.299\pm 0.003$ and
$\mathcal{R}(D^{\ast})=0.258\pm 0.005$. The average values of the experimental
results are $\mathcal{R}(D)=0.407\pm 0.039\pm 0.024$ and
$\mathcal{R}(D^{\ast})=0.306\pm 0.013\pm 0.007$ [4], where the first
uncertainty is statistical and the second is systematic. These values exceed
SM predictions by $2.1\sigma$ and $3.0\sigma$, respectively. A combined
analysis of $\mathcal{R}(D)$ and $\mathcal{R}(D^{\ast})$ taking correlations
into account, finds that the deviation from the SM prediction is approximately
$3.8\sigma$ [4].

In this paper, we report the first measurement of $\mathcal{R}(D)$ using the
semileptonic tagging method, and update our earlier measurement of
$\mathcal{R}(D^{\ast})$ [5] by combining results of $B^{0}$ and $B^{+}$ decays
using a more efficient tag reconstruction algorithm. We use the full
$\Upsilon(4S)$ data sample containing $772\times 10^{6}$ $B\bar{B}$ events
recorded with the Belle detector [6] at the KEKB $e^{+}e^{-}$ collider [7].
The Belle detector is described in detail elsewhere [6]. To determine the
reconstruction efficiency and probability density functions (PDFs) for signal,
normalization, and background modes, we use Monte Carlo (MC) simulated events,
generated with the EvtGen event generator [8], and having the detector
response simulated with the GEANT3 package [9].

##  2 Event Reconstruction And Selection

The ${B}_{\rm tag}$ is reconstructed using a hierarchical algorithm based on
“Fast” boosted decision trees (BDT) [10] in the $D\ell\bar{\nu}_{\ell}$ and
$D^{*}\ell\bar{\nu}_{\ell}$ channels, where $\ell=e,\mu$. We select well-
reconstructed ${B}_{\rm tag}$ candidates by requiring their classifier output
to be larger than $10^{-1.5}$. We veto ${B}\to D^{\ast}\tau(\to\ell\nu\nu)\nu$
events on the tag side by applying a selection on
$\cos\theta_{B,D^{(\ast)}\ell}$, defined as
$\cos\theta_{B,D^{(\ast)}\ell}\equiv\frac{2E_{\rm
beam}E_{D^{(*)}\ell}-m_{B}^{2}-m_{D^{(*)}\ell}^{2}}{2|\bm{p}_{B}||\bm{p}_{D^{(*)}\ell}|}$
where $E_{\rm beam}$ is the beam energy, and $E_{D^{*}\ell}$,
$\bm{p}_{D^{*}\ell}$, and $m_{D^{*}\ell}$ are the energy, momentum, and mass,
respectively, of the $D^{*}\ell$ system. The quantity $m_{B}$ is the nominal
$B$ meson mass [11], and $\bm{p}_{B}$ is the $B$ meson momentum. All
quantities are evaluated in the $\Upsilon(4S)$ rest frame. Correctly
reconstructed $B$ candidates in the normalization mode are expected to have a
value of $\cos\theta_{B,D^{(\ast)}\ell}$ between $-1$ and $+1$. Similarly,
correctly reconstructed and misreconstructed $B$ candidates in the signal mode
tend to have $\cos\theta_{B,D^{(\ast)}\ell}$ values more negative than this
range due to additional missing particles. We account for detector resolution
effects and apply the requirement $-2.0<\cos\theta_{B,D^{(\ast)}\ell}<1.0$ for
the ${B}_{\rm tag}$.

In each event with a selected ${B}_{\rm tag}$ candidate, we search for the
signature $D^{(*)}\ell$ among the remaining tracks and calorimeter clusters.
We define four disjoint data samples, denoted $D^{+}\ell^{-}$,
$D^{0}\ell^{-}$, $D^{*+}\ell^{-}$, and $D^{*0}\ell^{-}$. On the signal side,
neutral $D$ mesons are reconstructed in the following decay modes: $D^{0}\to
K^{-}\pi^{+}\pi^{0}$, $K^{-}\pi^{+}\pi^{+}\pi^{-}$, $K^{-}\pi^{+}$,
$K_{S}^{0}\pi^{+}\pi^{-}$, $K_{S}^{0}\pi^{0}$, $K_{S}^{0}K^{+}K^{-}$,
$K^{-}K^{+}$ and $\pi^{-}\pi^{+}$. Similarly, charged $D$ mesons are
reconstructed in the following modes: $D^{+}\to K^{-}\pi^{+}\pi^{+}$,
$K_{S}^{0}\pi^{+}\pi^{0}$, $K_{S}^{0}\pi^{+}\pi^{+}\pi^{-}$,
$K_{S}^{0}\pi^{+}$, $K^{-}K^{+}\pi^{+}$ and $K_{S}^{0}K^{+}$. The combined
branching fractions for reconstructed channels are 30% and 22% for $D^{0}$ and
$D^{+}$, respectively. On the signal side, we require
$\cos\theta_{B,D^{(\ast)}\ell}$ to be less than 1.0 and the $D^{(*)}$ momentum
in the $\Upsilon(4S)$ rest frame to be less than 2.0 GeV/$c$. Finally, we
require that events contain no extra charged tracks, $K_{S}^{0}$ candidates,
or $\pi^{0}$ candidates, which are reconstructed with the same criteria as
those used for the $D$ candidates. When multiple ${B}_{\rm tag}$ or ${B}_{\rm
sig}$ candidates are found in an event, we select the ${B}_{\rm tag}$
candidate with the highest tagging classifier output, and the ${B}_{\rm sig}$
candidate with the highest p-value resulting from the $D$ or $D^{*}$ vertex
fit.

##  3 Signal Extraction

To distinguish signal and normalization events from background processes, we
use the sum of the energies of neutral clusters detected in the ECL that are
not associated with reconstructed particles, denoted as $E_{\rm ECL}$. We
require that $E_{\rm ECL}$ be less than 1.2 GeV. To separate reconstructed
signal and normalization events, we employ a BDT based on the `XGBoost`
package [12]. The input variables to the BDT are
$\cos\theta_{B,D^{(\ast)}\ell}$; the approximate missing mass squared $m_{\rm
miss}^{2}=(E_{\rm
beam}-E_{D^{(*)}}-E_{\ell})^{2}-(\bm{p}_{D^{(*)}}+\bm{p}_{\ell})^{2}$; the
visible energy $E_{\rm vis}=\sum_{i}E_{i}$, where $(E_{i},\bm{p}_{i})$ is the
four-momentum of particle $i$. We do not apply any selection on the BDT
classifier output, denoted as `class`; instead we use it as one of the fitting
variables for the extraction of $\mathcal{R}(D^{(\ast)})$.

We extract the yields of signal and normalization modes from a two-dimensional
(2D) extended maximum-likelihood fit to the variables `class` and $E_{\rm
ECL}$. The fit is performed simultaneously to the four $D^{(*)}\ell$ samples.
The free parameters in the final fit are the yields of signal, normalization,
$B\to D^{**}\ell\nu_{\ell}$ and feed-down from $D^{*}\ell$ to $D\ell$
components. The yields of other backgrounds are fixed to their MC expected
values. The ratios $\mathcal{R}(D^{(\ast)})$ are given by the formula:
$\mathcal{R}(D^{(\ast)})=\frac{1}{2{\cal
B}(\tau^{-}\to\ell^{-}\bar{\nu}_{\ell}\nu_{\tau})}\cdot\frac{\varepsilon_{\rm
norm}}{\varepsilon_{\rm sig}}\cdot\frac{N_{\rm sig}}{N_{\rm norm}}$ where
$\varepsilon_{\rm sig(norm)}$ and $N_{\rm sig(norm)}$ are the detection
efficiency and yields of signal (normalization) modes and ${\cal
B}(\tau^{-}\to\ell^{-}\bar{\nu}_{\ell}\nu_{\tau})$ is the average of the world
averages for $\ell=e$ and $\ell=\mu$. To improve the accuracy of the MC
simulation, we apply a series of correction factors determined from control
sample measurements. Furthermore, we reweight events to account for differing
yields of misreconstructed $D^{(*)}$ and for the different reconstruction
efficiency of the tagging algorithm between data and MC simulations.

##  4 Systematic Uncertainties

The systematic uncertainties are summarized in Table 1, where the label
“$D^{**}$ composition” refers to the uncertainty introduced by the branching
fractions of the $B\to D^{**}\ell\nu_{\ell}$ channels and the decays of the
$D^{**}$ mesons, which are not well known and hence contribute significantly
to the total PDF uncertainty due to $B\to D^{**}\ell\nu_{\ell}$ decays. A
large systematic uncertainty arises from the limited size of MC samples, which
contributes to the uncertainty in the PDF shapes and various efficiency
factors used in the fit. The total systematic uncertainty is estimated by
summing all contributions in quadrature.

##  5 Results and Conclusion

Our results for the measurements of the ratios $\mathcal{R}(D^{(\ast)})={\cal
B}(\bar{B}\to D^{(*)}\tau^{-}\bar{\nu}_{\tau})/{\cal B}(\bar{B}\to
D^{(*)}\ell^{-}\bar{\nu}_{\ell})$, where $\ell$ denotes an electron or a muon,
based on a semileptonic tagging method using a data sample containing
$772\times 10^{6}B\bar{B}$ events collected with the Belle detector, are

$\displaystyle\mathcal{R}(D)$ $\displaystyle=$ $\displaystyle 0.307\pm
0.037\pm 0.016$ (1) $\displaystyle\mathcal{R}(D^{\ast})$ $\displaystyle=$
$\displaystyle 0.283\pm 0.018\pm 0.014,$ (2)

which are in agreement with the SM predictions within $0.2\sigma$ and
$1.1\sigma$, respectively. The combined result agrees with the SM predictions
within $1.2\sigma$. The $E_{\rm ECL}$ projections of the fit are shown in
Figure 1. The 2D combination of the $\mathcal{R}(D)$ and
$\mathcal{R}(D^{\ast})$ results of this analysis are shown in Figure 2. This
work constitutes the most precise measurements of $\mathcal{R}(D)$ and
$\mathcal{R}(D^{\ast})$ performed to date and the first result for
$\mathcal{R}(D)$ based on a semileptonic tagging method.

Figure 1: $E_{\rm ECL}$ fit projections and data points with statistical
uncertainties in the $D^{+}\ell^{-}$ (top left), $D^{0}\ell^{-}$ (top right),
$D^{*+}\ell^{-}$ (bottom left) and $D^{*0}\ell^{-}$ (bottom right) samples,
for the full classifier region. The signal region defined by the selection
class $>0.9$ is shown in the inset.

Figure 2: Fit results are shown on a $\mathcal{R}(D)$ vs.
$\mathcal{R}(D^{\ast})$ plane with the Belle results on $\mathcal{R}(D)$ and
$\mathcal{R}(D^{\ast})$ performed with a hadronic tag [13, Hirose:2016wfn] and
the preliminary Belle average.

Table 1: Systematic uncertainties contributing to the $\mathcal{R}(D^{(\ast)})$ results, with values in percent. Source | $\Delta\mathcal{R}(D)$ | $\Delta\mathcal{R}(D^{\ast})$
---|---|---
$D^{**}$ composition | 0.76 | 1.41
Fake $D^{(*)}$ calibration | 0.19 | 0.11
${B}_{\rm tag}$ calibration | 0.07 | 0.05
Feed-down factors | 1.69 | 0.44
Efficiency factors | 1.93 | 4.12
Lepton efficiency | 0.36 | 0.33
and fake rate |  |
Slow pion efficiency | 0.08 | 0.08
PDF shapes | 4.39 | 2.25
$B$ decay form factors | 0.55 | 0.28
Luminosity and $\mathcal{B}(\Upsilon(4S))$ | 0.10 | 0.04
$\mathcal{B}(B\to D^{(*)}\ell\nu)$ | 0.05 | 0.02
$\mathcal{B}(D)$ | 0.35 | 0.13
$\mathcal{B}(D^{*})$ | 0.04 | 0.02
$\mathcal{B}(\tau^{-}\to\ell^{-}\bar{\nu}_{\ell}\nu_{\tau})$ | 0.15 | 0.14
Total | 5.21 | 4.94

## Acknowledgments

We thank the KEKB group for excellent operation of the accelerator; the KEK
cryogenics group for efficient solenoid operations; and the KEK computer
group, the NII, and PNNL/EMSL for valuable computing and SINET5 network
support. We acknowledge support from MEXT, JSPS and Nagoya’s TLPRC (Japan);
ARC (Australia); FWF (Austria); NSFC and CCEPP (China); MSMT (Czechia); CZF,
DFG, EXC153, and VS (Germany); DST (India); INFN (Italy); MOE, MSIP, NRF,
RSRI, FLRFAS project and GSDC of KISTI and KREONET/GLORIAD (Korea); MNiSW and
NCN (Poland); MSHE (Russia); ARRS (Slovenia); IKERBASQUE (Spain); SNSF
(Switzerland); MOE and MOST (Taiwan); and DOE and NSF (USA). We acknowledge
the support provided by the Albert Shimmins Fund for the writing of this
proceedings.

## References

## References

  * [1] Throughout this proceedings, the inclusion of the charge-conjugate decay mode is implied.
  * [2] D. Bigi and P. Gambino.  Revisiting $B\to D\ell\nu$.  Phys. Rev., D94(9):094008, 2016.
  * [3] F. U. Bernlochner, Z. Ligeti, M. Papucci, and D. J. Robinson.  Combined analysis of semileptonic $B$ decays to $D$ and $D^{*}$: $R(D^{(*)})$, $|V_{cb}|$, and new physics.  Phys. Rev., D95(11):115008, 2017.  [erratum: Phys. Rev.D97, no.5, 059902 (2018)].
  * [4] Y. Amhis et al.  Averages of $b$-hadron, $c$-hadron, and $\tau$-lepton properties as of summer 2016.  Eur. Phys. J., C77:895, 2017.  updated results and plots available at https://hflav.web.cern.ch.
  * [5] Y. Sato et al.  Measurement of the branching ratio of $\bar{B}^{0}\rightarrow D^{*+}\tau^{-}\bar{\nu}_{\tau}$ relative to $\bar{B}^{0}\rightarrow D^{*+}\ell^{-}\bar{\nu}_{\ell}$ decays with a semileptonic tagging method.  Phys. Rev., D94(7):072007, 2016.
  * [6] A. Abashian et al.  The Belle Detector.  Nucl. Instrum. Methods, A479:117–232, 2002.  also see detector section in J.Brodzicka et al., Prog. Theor. Exp. Phys. 2012, 04D001 (2012).
  * [7] S. Kurokawa and E. Kikutani.  Overview of the KEKB accelerators.  Nucl. Instrum. Methods, A499(1):1 – 7, 2003.  and other papers included in this Volume; T.Abe et al., Prog. Theor. Exp. Phys. 2013, 03A001 (2013) and references therein.
  * [8] D. J. Lange.  The EvtGen particle decay simulation package.  Nucl. Instrum. Methods Phys. Res., Sect. A, 462(1):152 – 155, 2001\.
  * [9] R. Brun et al.  GEANT3.  1987\.  CERN Report No. DD/EE/84-1.
  * [10] T. Keck et al.  The Full Event Interpretation.  Computing and Software for Big Science, 3(1):6, 2019.
  * [11] M. Tanabashi et al.  Review of Particle Physics.  Phys. Rev., D98:030001, 2018.
  * [12] T. Chen and C. Guestrin.  XGBoost: A Scalable Tree Boosting System.  2016\.  1603.02754.
  * [13] M. Huschle et al.  Measurement of the branching ratio of $\bar{B}\to D^{(\ast)}\tau^{-}\bar{\nu}_{\tau}$ relative to $\bar{B}\to D^{(\ast)}\ell^{-}\bar{\nu}_{\ell}$ decays with hadronic tagging at Belle.  Phys. Rev., D92(7):072014, 2015.


# Sensing of Side Lobes Interference for Blockage Prediction in Dense mmWave
Networks

Mohamed Sana, Hiba Dakdouk, Benoit Denis  CEA-Leti, Université Grenoble Alpes,
F-38000 Grenoble, France
Email: {mohamed.sana, hiba.dakdouk<EMAIL_ADDRESS>
###### Abstract

The integration of sensing capability in the design of wireless communication
systems is foreseen as a key enabler for efficient radio resource management
in next-generation networks. This paper focuses on millimeter-wave
communications, which are subject to severe attenuation due to blockages,
ultimately detrimental to system performance. In this context, the sensing
functionality can allow measuring or even imaging the wireless environment
allowing anticipation of possible link failures, thus enabling proactive
resource reallocation such as handover. This work proposes a novel mechanism
for opportunistic environment sensing, which leverages existing network
infrastructure with low complexity. More specifically, our approach exploits
the fluctuations of interference, perceived in antenna side lobes, to detect
local activity due to a moving blocker around the reference communication
link. Numerical evaluations show that the proposed method is promising as it
allows effective assessment of the blocker direction, trajectory and possibly,
its location, speed, and size.

###### Index Terms:

Sensing, Blockages Prediction, mmWave Communications, Network densification,
6G Networks.

##  I Introduction

Millimeter-Wave (mmWave) frequencies (ranging, e.g., between $28$ and $300$
GHz) are getting great attention recently for their various advantages over
traditional radio frequencies (sub-6 GHz). With the large spectrum available
at these frequencies, mmWave technology can effectively boost the network
capacity. It also supports advanced beamforming techniques, which allow for
highly directional signal transmissions, reducing interference and enhancing
network performance. However, these advantages also come along with a critical
challenge: mmWave communications suffer from severe path-losses and are very
sensitive to blockages and attenuation by physical obstacles (e.g., buildings,
trees, human body). Penetration losses through the human body can range
between $20-40~{}\mathrm{dB}$, whereas attenuation through buildings can be as
high as $40-80~{}\mathrm{dB}$ [1].

Frequent interruptions and long-duration blockages may cause severe
degradation in the quality of service (QoS) of end-users, requiring frequent
handover procedures that are detrimental to network performance [2].
Therefore, effective blockage prediction mechanisms are needed to enable
efficient radio resource management (RRM). Joint communication and sensing has
been identified as a key feature of future 6G systems [3]. These sensing
capabilities could be used to improve network performance by providing
optimization inputs for network steering, including the ability to detect
objects that (temporarily) obstruct or block the line of sight (LoS) between
two communicating nodes [4]. Sensing the surrounding for detecting moving
blockages in mmWave communications has become a fundamental research topic [5,
6, 7, 8, 9, 10]. A major axis of research that aims to predict and prevent
blockages in mmWave systems considers making use of in-band mmWave signal and
data rate observations. The authors in [11] uses the fluctuation of the
received power level occurring before the shadowing event to predict the
future time instance of a blockage with the aid of deep neural networks.
However, the prediction accuracy decreases as the blocker is far from the
mmWave link, which means that it can only be detected accurately when it is
close to the mmWave communication beam. In [12], the data rate fluctuation
occurring before the shadowing indicates the potential blockage. Using deep
reinforcement learning techniques, the authors could predict handover timings
while obstacle-caused data rate degradation are predicted before the
degradation occurs. On the other hand, authors in [13] propose the use of an
additional passive mmWave beam (guard beam) next to the main communication
beam that is intended to sense the environment by expanding the field of view
of a base station (BS). Thus, a blocker could be detected early by observing
the received signal fluctuation resulting from the non line of sight (NLoS)
component from the user equipment (UE) due to the blocker’s presence within
the field of view. Yet, all these approaches are limited in terms of detection
range, as the blocker should be close to the main communication beam causing
fluctuation in the received signal, and might fail with large-velocity moving
blockers. In addition, the sensing feature may require an additional and
dedicated mechanism (e.g., a dedicated beam).

In contrast, we propose a mechanism that exploits side lobes information for
the passive and opportunistic sensing of a dense mmWave network. Network
densification is a key feature of future networks that will further improve
their capacity [14]. At the same time, it may also lead to increased intra-
and inter-cell interference, which may impact communication performance.
However, in this work, we take advantage of this specific characteristic of
dense networks to detect the presence of moving blockers in the surrounding
environment of communicating nodes. Our method relies on the observation of
the interference fluctuations in antenna side lobes caused by the existence of
moving blockers in angular sectors around the communication link of concern.
Unlike the aforementioned studies, our approach is capable to detect and track
moving objects all around the sensing device, i.e. in range of
$360^{\degree}$. This makes it possible to predict some characteristics of
blockers, including their trajectory and velocity, allowing early detection of
blockage events and avoiding link outages by triggering, e.g., a handover
procedure.

##  II System Model

We consider a dense mmWave network composed of a set
$\mathcal{B}=\\{b_{1},...,b_{M}\\}$ of $M$ BSs deployed in a bi-dimensional
Euclidean space of radius $R$ to provide service coverage to a set
$\mathcal{U}=\\{u_{1},...,u_{K}\\}$ of $K$ UEs. We assume BSs and UEs form two
distinct homogeneous Poisson Point Processes (PPP) with densities
$\lambda_{b}~{}[\mathrm{m}^{-2}]$ and $\lambda_{u}~{}[\mathrm{m}^{-2}]$
respectively such that in average, $\mathbb{E}[M]=\lambda_{b}\pi R^{2}$ and
$\mathbb{E}[K]=\lambda_{u}\pi R^{2}$. In this dense network, a mobile and
passive object (e.g., a robot), modelled as a cylindrical object of radius
$r_{B}$ moves around, causing the blockage of interfering and direct
communication paths. Let $\omega_{B}(t)$ denote its angular velocity and
$d_{B}(t)$ its distance with respect to (w.r.t.) BS $b_{0}$, referred to as
the _typical BS_ and taken as the reference point in the following. Clearly,
$(\omega_{B}(t),d_{B}(t))$ characterizes the trajectory of the blocking
object. In this work, we propose a novel approach for a passive sensing of
such a moving object, partially identifying its trajectory by leveraging the
interference perceived in the side lobes of the antenna radiation patterns.

We focus on an uplink setting with only LoS communications for both direct and
interfering links. In this scenario, an initial access phase allows new UEs to
perform beam training and alignment mechanisms, configuring the appropriate
beams, which exploit the maximum directivity gain w.r.t. serving BSs for the
service phase. For simplicity, we assume each UE gets associated with the
closest BS, as we do not specifically address the user association problem.
However, this problem can be efficiently solved using approaches proposed in
[15] to optimize service coverage. During the service phase, BSs exploit the
fluctuations of interference perceived in their antennas side lobes, resulting
from simultaneous communications with UEs, for sensing their nearby
environment to detect blockages.

Antennas. In our system model, UEs and BSs are equipped with antenna arrays to
perform directional beamforming. For easy analysis, we assume that antenna
arrays operate on the same elevation plane, and accordingly, we set the beam
elevation angle to zero. Therefore, we denote with $G^{\rm Tx}_{\theta}(x)$
and $G^{\rm Rx}_{\vartheta}(x)$ the transmitter and the receiver 2D antenna
radiation pattern respectively, where $\theta$ and $\vartheta$ are the beam
width and, $x$ is the azimuth angle to the main lobe (either $\psi$ or $\phi$
in Fig. 1). For the tractability of analysis, we approximate the actual 2D
array patterns with a sectored Gaussian directional antenna model [16] whose
beamforming gain is given as follows:

$\displaystyle G_{z}^{\ell}(x)=\left\\{\begin{array}[]{l
l}G_{m}^{\ell}e^{-\rho_{z}x^{2}},&\quad\text{if }|x|\leq\frac{z}{2},\\\
G_{s}^{\ell},&\quad\text{otherwise},\end{array},~{}z\in\\{\theta,\vartheta\\},\right.$
(3)

where $\rho_{z}=2.028\dfrac{\ln{(10)}}{z^{2}}$ and $z$ is the beam width. In
addition, $G_{m}^{\ell}$ and $G_{s}^{\ell}$ denote the gain of the main lobe
and the side lobes as per $\ell\in\\{\mathrm{Tx},\mathrm{Rx}\\}$,
respectively. Following these definitions, we define the antenna peak-side-
lobe (PSL) gain as
${\mathrm{PSL}^{\ell}={G_{m}^{\ell}}({G_{s}^{\ell}})^{-1}}$. In particular,
the value of $\mathrm{PSL}^{\ell}$ depends on the number of antenna elements.

Figure 1: Network model with 3 UEs interfering on a communication between a
typical UE $u_{0}$ and serving BS $b_{0}$. In this network, a mobile robot
moves around causing blockages.

Propagation Channel. We adopt the commonly used Friis propagation loss model
[17], where the received power $P^{\mathrm{Rx}}$ is given as a function of the
transmitted power $P^{\mathrm{Tx}}$ and the distance $d$ between the
transmitter and the receiver:

$P^{\mathrm{Rx}}(t)=\chi(t)\zeta(t)P^{\mathrm{Tx}}G_{\theta}^{\mathrm{Tx}}(\varphi)G^{\rm
H}(d)G_{\vartheta}^{\mathrm{Rx}}(\psi),$ (4)

where $\varphi$ and $\psi$ represent the azimuth angles at the transmitter and
receiver respectively. Here, $G^{\rm H}(d)$ is the distance-dependent channel
gain, which captures the effect of path-loss and large-scale shadowing as
follows:

$G^{\rm
H}(d)|_{\text{dB}}=\mathrm{PL}_{0}+10\eta\log_{10}\left(\frac{d}{d_{\rm
ref}}\right)+X_{(\sigma_{s})},$ (5)

where $\mathrm{PL}_{0}$ denotes the pathloss constant, $d_{\rm ref}$ is a
reference distance, $\eta$ denotes the pathloss exponent and
$X_{(\sigma_{s})}$ represents the static shadowing effect, modeled as a
Gaussian variable with zero mean and variance $\sigma_{s}^{2}$. Also,
$\zeta(t)$ represents the small-scale fading coefficient, which follows a
$m$-Nakagami distribution. Eventually, $\chi(t)$ denotes the shadowing effect
due to the passive object moving around the corresponding link. We adopt the
following Gaussian modeling as in [18]:

$\displaystyle\chi(t)|_{\rm
dB}=-A\exp{\left(-\frac{|\psi-\psi_{B}(t)|^{2}}{\sigma_{B}^{2}}\right)}$ (6)

where $\psi_{B}(t)$ is the relative angle between the blocker and the receiver
main lobe. Accordingly, $|\psi-\psi_{B}(t)|$ represents the angle between the
blocker and the interfering link with relative angle $\psi$ w.r.t. the
receiver (see Fig. 1); $A$ denotes the attenuation (in dB) when the link is
fully blocked (i.e., $|\psi-\psi_{B}(t)|=0$) and $\sigma_{B}$ is a parameter
that depends on the characteristics of the blocker (e.g., size) and radio
parameters. Although simplistic, this model allows an effective geometric
analysis of blocking phenomena.

Cell interference. In our system model, as we do not specifically optimize
beamforming, interference results from the overlapping of communication beams
between mmWave BSs and UEs. Indeed, let us consider a typical BS $b_{0}$
placed at a distance $d_{0}$ from its served UE $u_{0}$. We refer to
$u_{0}\rightarrow b_{0}$ as the reference link. An interfering UE $u_{i}$,
located at a distance $d_{i}$ with a relative angle of arrival (AoA)
$\psi_{i}$ w.r.t. $b_{0}$, is served by another BS $b_{j}$ in a relative angle
of departure (AoD) $\varphi_{i,j}$ (see Fig. 1). We denote with $I_{i,j}$, the
resulting interference perceived by BS $b_{0}$:

$\displaystyle I_{i,j}(t)$
$\displaystyle=\chi_{i}(t)\zeta_{i}(t)P_{i}^{\mathrm{Tx}}G^{\rm
Tx}_{\theta}(\varphi_{i,j})G^{\rm H}(d_{i})G^{\rm Rx}_{\vartheta}(\psi_{i})$
(7)

Thus, the total interference perceived by the typical BS as a function of
signal angle of arrival (AoA) $\psi$ reads as:

$\displaystyle
I(\psi,t)=\sum_{\mathclap{\begin{subarray}{c}(u_{i},b_{j})\in\mathcal{U}\times\mathcal{B}\backslash(u_{0},b_{0})\end{subarray}}}I_{i,j}(t)\delta(\psi-\psi_{i}),$
(8)

where, $\delta(\cdot)$ is the Dirac delta function.

##  III Sensing of side lobes interference

###  III-A On the design of the sensing matrix

As we assume spatial reuse of the spectrum across the network, interference
may come from multiple directions, especially in dense networks. We assume
that BSs can sense the sum-interference perceived in their side lobes from a
certain sector. Typically, we define:

$\displaystyle
I_{s(\phi,\alpha)}(t)=\int_{s(\phi,\alpha)}I(\psi,t)\mathrm{d}\psi,$ (9)

where $s(\phi,\alpha)$ denotes sector of angular wide $2\alpha$ with absolute
orientation $\phi$ w.r.t. to horizontal axis as shown in Fig. 1.

We introduce a novel metric, the signal-to-sector-interference-plus-noise
ratio,111In practice, $\gamma_{s(\phi,\alpha)}$ can be estimated using
approaches similar to beam- or angular-domain channel estimation techniques
[19, 20]. Except here, there is no need to estimate channel state information
but rather to capture integrated interference power in each beam. Obviously,
the time spent to acquire the full sensing matrix must be compliant with
blocker mobility dynamics to guarantee observability. These temporal aspects
of the sensing procedure are out of the scope of this paper, but left for
future works. denoted $\gamma_{s(\phi,\alpha)}$, between the typical UE
$u_{0}$ and its serving BS $b_{0}$, which we define as:

$\displaystyle\gamma_{s(\phi,\alpha)}(t)=\frac{P_{0}^{\mathrm{Tx}}\chi_{0}(t)\zeta_{0}(t)G_{m}^{\mathrm{Tx}}G^{\rm
H}(d_{0})G_{m}^{\mathrm{Rx}}}{I_{s(\phi,\alpha)}(t)+N_{0}B}.$ (10)

Here $B$ is the bandwidth, and $N_{0}$ is the noise power spectral density.

Now, let divide the $2\pi$ angular space into $n+1$ non-overlapped sectors of
angle $2\alpha=\frac{2\pi}{n+1}$, here preferably chosen contiguous to cover
all 2D space. We refer to $s_{k}$ as the $k$-th sector with absolute
orientation $\phi_{k}$, where $k=0$, correspond to the sector whose
orientation coincides with the link $(u_{0}\rightarrow b_{0})$. Thus,
$\phi_{k}=\phi_{0}-2k\alpha$. Finally, let us consider a short-term
observation window of size $\tau+1$ and let define:

$\displaystyle\boldsymbol{\Lambda}_{\tau,n}(t)=\begin{pmatrix}\gamma_{s_{0}}(t)&\dots&\gamma_{s_{n}}(t)\\\
\gamma_{s_{0}}(t-1)&\dots&\gamma_{s_{n}}(t-1)\\\ \vdots&\ddots&\vdots\\\
\gamma_{s_{0}}(t-\tau)&\dots&\gamma_{s_{n}}(t-\tau)\end{pmatrix}$ (11)

We refer to $\boldsymbol{\Lambda}_{\tau,n}(t)$ as the sensing matrix. The idea
behind defining such matrix is that, the blockage of an interfering link
around the typical BS in the sectorized regions induces fluctuations of the
values of $\gamma$ whose intensity depends on the shadowing $\chi$, and on the
side lobe gains (more specifically on the PSL of the antenna). By sensing the
side lobes interference, we show that early detection of blockage events is
possible, thus allowing efficient resource reallocation.

Clearly, the accuracy of the detection also depends on sector angular width
$\alpha$ and the density of interferers (here UEs). Indeed, the probability to
have at least one interferer in sector $s_{k}(\alpha)$ is given by:

$\displaystyle p_{1}=1-e^{-\lambda_{u}\alpha R^{2}}.$ (12)

Thus, the smaller the value of $\alpha$, the higher the resolution of matrix
$\boldsymbol{\Lambda}$. At the same time, the smaller is $\alpha$, the scarcer
the matrix $\boldsymbol{\Lambda}$ as $p_{1}\rightarrow 0$.

###  III-B Singular Value decomposition of the sensing matrix for blocker
signature detection

The sensing matrix can be viewed as the sum of three components:

$\displaystyle\boldsymbol{\Lambda}_{\tau,n}(t)=\boldsymbol{\Lambda}_{\tau,n}^{(\circ)}(t)+\boldsymbol{\Lambda}_{\tau,n}^{(\bullet)}(t)+\boldsymbol{\Lambda}_{\tau,n}^{(\rm
noise)}(t),$ (13)

where, $\boldsymbol{\Lambda}_{\tau,n}^{(\circ)}(t)$ denotes the blockage-free
sensing matrix, $\boldsymbol{\Lambda}_{\tau,n}^{(\bullet)}(t)$ denotes the
blockage signature matrix, and $\boldsymbol{\Lambda}_{\tau,n}^{(\rm
noise)}(t)$ is the additive noise due to e.g. , random fading, scattering,
etc. Our objective is thus to capture the moving blockage signature, i.e.,
$\boldsymbol{\Lambda}_{\tau,n}^{(\bullet)}(t)$. To do so, we apply a blind
source separation technique based on singular value decomposition (SVD),
similar to idea used for processing seismic data [21]. Following this idea, we
decompose the sensing matrix $\boldsymbol{\Lambda}_{\tau,n}(t)$ into the
product of three matrices as follows:

$\displaystyle\boldsymbol{\Lambda}_{\tau,n}(t)=\boldsymbol{\mathrm{U}}\boldsymbol{\mathrm{\Sigma}}\boldsymbol{\mathrm{V}}^{T}$
(14)

where $\boldsymbol{\mathrm{U}}\in\mathbb{R}^{n\times r}$,
$\boldsymbol{\mathrm{V}}\in\mathbb{R}^{r\times\tau}$, and
$\boldsymbol{\mathrm{\Sigma}}\in\mathbb{R}^{r\times
r}=\mathrm{diag}(\sigma_{1},\sigma_{2},\dots,\sigma_{r})$, where
$r=\mathrm{rank}(\boldsymbol{\Lambda})\leq\mathrm{min}\left(\tau,n\right)$
(the number of non-zero singular values) and
$\sigma_{1}\geq\sigma_{2}\geq\dots\geq\sigma_{r}>0$ are the singular values.
This decomposition can be further written in the following form:

$\displaystyle\boldsymbol{\Lambda}_{\tau,n}(t)$
$\displaystyle=\sum_{l=1}^{r}\sigma_{l}\boldsymbol{\mathrm{u}}_{l}\boldsymbol{\mathrm{v}}_{l}^{T}$
(15)
$\displaystyle=\underbrace{\sum_{l=1}^{l_{0}}\sigma_{l}\boldsymbol{\mathrm{u}}_{l}\boldsymbol{\mathrm{v}}_{l}^{T}}_{\boldsymbol{\Lambda}_{\tau,n}^{(\circ)}(t)}+\underbrace{\sum_{l=l_{0}+1}^{l_{1}}\sigma_{l}\boldsymbol{\mathrm{u}}_{l}\boldsymbol{\mathrm{v}}_{l}^{T}}_{\boldsymbol{\Lambda}_{\tau,n}^{(\bullet)}(t)}+\underbrace{\sum_{l=l_{1}+1}^{r}\sigma_{l}\boldsymbol{\mathrm{u}}_{l}\boldsymbol{\mathrm{v}}_{l}^{T}}_{\boldsymbol{\Lambda}_{\tau,n}^{(\rm
noise)}(t)},$

where $\boldsymbol{\mathrm{u}}_{l}$ (resp. $\boldsymbol{\mathrm{v}}_{l}$) are
the columns of the semi-unitary matrix $\boldsymbol{\mathrm{U}}$ (resp.
$\boldsymbol{\mathrm{V}}$) and $1\leq l_{0}\leq l_{1}\leq r$. As for seismic
data processing, the mobility of the blocker waves the singular values. Thus,
a proper selection of $l_{0}$ and $l_{1}$ allows a full capture of the
blockage signature matrix. In practice, the value of $l_{0}$ and $l_{1}$ can
be chosen by finding abrupt changes in the slope of the curve of relative
singular values.

##  IV Numerical results

We consider a network of $M$ BSs and $K$ UEs distributed in the space of a
circular industrial environment of radius $R=100~{}\mathrm{m}$ according to
homogeneous PPP with densities $\lambda_{b}=6\times 10^{-4}~{}\mathrm{m}^{-2}$
and $\lambda_{u}=1.5\times 10^{-3}~{}\mathrm{m}^{-2}$ respectively. Each UE is
associated to its closest BS. A BS $b_{0}$ located at the center of the
network is taken as a reference: it performs side lobe sensing and computes
the detection matrix on a link with a randomly chosen associated UE $u_{0}$.
We set the width of the angular sectors to $2\alpha=10\degree$ (i.e. $n+1=36$
sectors overall, covering the 2D space), and the size of the observation
window is set to $\tau=50~{}\mathrm{s}$. The sector width is always set equal
to the antenna beamwidth $\vartheta$ of BS $b_{0}$. When we detect activity in
a sector, the estimated angular position $\hat{\psi}_{B}$ of the moving object
is taken equal to the orientation of the sector resulting in a mean
approximation error of $5\degree$ (if the object is actually there). We
consider the pathloss model in Eq. (5), where the values of its parameters
have been characterized via in-lab experiments based on mmWave channel
sounding in a representative industrial Internet of Things (IoT)-type scenario
[22]. Other simulation parameters are presented in Table I.

###  IV-A Detection of blocker signature

For a random network deployment, we consider a mobile object of radius
$r_{B}=1~{}\mathrm{m}$ moving along a random trajectory with an arbitrary
velocity ($w_{B}\sim[-15\degree,15\degree]s^{-1}$). Figure 2(a) shows an
example of a network deployment, where multiple deployed BSs jointly provide
service to UEs. In this figure, a blocker is moving, following a trajectory,
which crosses the communication link $u_{0}\rightarrow b_{0}$ at a certain
point in the main sector $s_{0}$. Figure 2(b) presents the logarithm of the
raw sensing matrix; $\log(\boldsymbol{\Lambda}_{\tau,n})$ of Eq. (11), before
being processed. The processing applied to the sensing matrix makes it
possible to effectively reveal the signature of the mobile object by
empirically setting $l_{0}=1$ and $l_{1}=17$ in Figure 2(c), where the color
of each pixel quantifies the strength of the signature, i.e. the likelihood of
the blocker being present in the corresponding sector. Thanks to this
processing, it is possible to detect earlier the occurrence of the blockage,
i.e. as the blocker approaches sector $s_{0}$, and therefore to be able to
cope with it before it happens. Besides, it is also possible to get
information on the blocker trajectory as the bottom of Figure 2(a) reveals.
This figure compares the estimated trajectory of the blocker to its actual one
in terms of its angular position w.r.t.the link $u_{0}\rightarrow b_{0}$. We
can observe that the proposed method allows an effective detection of the
blocker and a follow-up of its angular trajectory. Note, however, that due to
the low number of UEs in some regions of the network, which are the
interferers on which this approach depends on to estimate the position of the
blocker, the detection is not effective at all times (i.e., lack of
observations regarding the blocker for some time epochs). Nevertheless, the
detection is still accurate in the main region of interest, i.e. when the
blocker is close to the BS as shown in Figure 2(a), that makes it possible to
anticipate a blockage event. Beyond, both the missed detection events and the
observation quantization error caused by space sectorization (see the step-
wise fluctuations of at most $5\degree$ around the ground-truth angular
trajectory in Figure 2(a)) could be easily overcome by standard Bayesian
filtering tools, such as Extended Kalman Filtering (EKF), even if this extra
processing step does not fall in the scope of this study here. In general, the
closer the blocker gets to the BS and the closer it gets to a dense region of
the network, the better the detection will be. Obviously, from Figure 2(c), it
is also possible to extract additional information on the direction and speed
of the blocker. Besides, although in this paper the trajectory is detected in
terms of angular position, it is possible to be more precise, and even to
locate the blocker, if we consider the sharing of sensed information between
different entities in the network, which will be addressed in future work.

TABLE I: Simulation parameters Parameters | Values
---|---
Carrier frequency | $28~{}~{}\mathrm{GHz}$
Bandwidth $\mathcal{B}$ | $400~{}~{}\mathrm{MHz}$
Pathloss | $60.1+14\log(d~{}[\mathrm{km}])$
Power $P^{\mathrm{Tx}}$ (UE) / $P^{\mathrm{Rx}}$ (BS) | $19.6/33~{}~{}\mathrm{dBm}$
Noise power spectral density $N_{0}$ | $-174~{}\mathrm{dBm}/~{}\mathrm{Hz}$
Small-scale fading $\sim m$-Nakagami | $m=3$
Rx beamwidth $\vartheta$ | 10°
Tx beamwidth $\theta$ | 135°
$G_{0}(z)$ |  $\pi(21.32z+\pi)^{-1}$ [16]
$G_{s}^{\rm Rx}$ | $G_{0}(\vartheta)$
$G_{m}^{\rm Rx}$ | $G_{0}(\vartheta)\times 10^{2.028}$
$G_{s}^{\rm Tx}$ | $0$
$G_{m}^{\rm Tx}$ | $2G_{0}(\theta)\times 10^{2.028}$
Blockage Attenuation $A$ | $100~{}\mathrm{dB}$
$\sigma_{B}$ | $\sqrt{8}~{}r_{B}$

(a) Deployment and blocker trajectory.

(b) Raw sensing matrix $\boldsymbol{\Lambda}_{\tau,n}$ (in log scale).

(c) Blocker signature.

Figure 2: Example of blockage detection using side lobe sensing mechanism.

###  IV-B Detection accuracy vs antenna PSL

In this section, we assess the performance of the proposed approach for
blocker signature detection through $N=500$ Monte-Carlo simulations. To avoid
cumbersome computations, we assume that the circular area around $b_{0}$ is
partitioned into a mesh grid $\mathcal{G}$ consisting of equal-size cells of
angle equals to $10\degree$ and radius equals to $5~{}\mathrm{m}$. We consider
a passive mobile object of radius $r_{B}=1~{}\mathrm{m}$ moving around $b_{0}$
in a circular motion at different distances up to a maximum distance of
$50~{}\mathrm{m}$. The blocker moves from one cell to another in a sequential
manner. For a quantitative evaluation of the detection accuracy, we adopt the
following weighted mean absolute error (wMAE):

$\displaystyle\mathrm{wMAE}=\frac{1}{N}\sum_{k=1}^{N}\mathbb{E}_{(d,\psi)\in\mathcal{G}}\left[w(d_{B})\lvert\psi_{B}-\hat{\psi}_{B}\rvert\right],$
(16)

where $\hat{\psi}_{B}$ is the blocker estimated angular position when its
actual location is in cell $(d_{B},\psi_{B})$ of the grid $\mathcal{G}$. Here,
$w(d_{B})$ is a weighted factor, which depends on the actual distance of the
blocker from the sensing BS $b_{0}$. This allows for a realistic evaluation of
the errors since the farther the blocker is, the more complex it is to predict
its angular position accurately due to path losses. In practice, following the
pathloss model, we define $w(d_{B})=\exp(-\mu d_{B}^{\eta})$, where $\eta$ is
the path loss coefficient and $\mu$ is a scaling factor. In particular,
$\mu=0$, corresponds to the non-weighted MAE.

We start by assessing the impact of antenna PSL (by varying the side lobe
gain) on the MAE. The associated results are presented in Figure 3(a). First,
we can notice that for any value of PSL the accuracy degrades as the blocker
moves away from the center where $b_{0}$ is located. This is due to two main
reasons: $1$) as the blocker moves away from the center, it is less likely to
be obstructing the LoS between $b_{0}$ and the interfering UEs, $2$) the
received power of the interference degrades as the interferer is farther, so
it is highly probable that blocking far interference will not be observed by
$b_{0}$. This result confirms the outcome of the previous experiment. In Fig.
3(b), we show the resulting $\mathrm{wMAE}(\mu)$ for different values of
$\mu$, along with the corresponding $95\%$ confidence intervals with different
values of PSL. We can notice that the detection accuracy degrades as the PSL
value increases. Indeed, with larger values of PSL, the interference perceived
by the side lobes is weaker, and thus the detection of the signature of the
blocker is less effective. For low PSL values, i.e. when the side lobe gain
approaches the main-lobe gain, the detection accuracy decreases as well
because of low signal-to-interference-and-noise ratio (SINR) values in the
sensing matrix. Thus, a trade-off can be found between the accuracy of the
detection i.e. by reducing the PSL, and the quality of the communication i.e.
by increasing the PSL to less suffer from interference. Yet, we can observe
that the proposed mechanism is still highly accurate as
$\mathrm{wMAE}(\mu=0.02)<5\degree$ (half the sector width) and
$\mathrm{wMAE}(\mu=0.01)<10\degree$ for most PSL values. Even the non-weighted
MAE is less than $10\degree$ except for very low side lobe gain (i.e. very
high PSL).

(a) MAE distribution over the network area. Here $\mu=0$.

(b) Angular position error, $\mathrm{wMAE}(\mu)$.

Figure 3: Weighted mean absolute error of blocker angular position vs antenna
peak-side-lobe gain. Figure 4: Mean absolute error of blocker angular position
vs antenna beamwidth and blocker size.

###  IV-C Detection accuracy vs beamwidth and blocker size

Other metrics that impact the performance of the side lobe sensing are the
antenna beamwidth and the blocker size. Similar to the previous experiment,
Figure 4 presents the blocker angular position MAE in each cell of the network
for different combinations of beamwidth and blocker radius $r_{B}$. As the
blocker size increases, it gets more detectable, as it is more capable of
blocking interference coming from different angles, but this indeed weakens
the accuracy especially when the blocker is large and close to the center, as
it could coexist in multiple sectors simultaneously. Also, as the beamwidth
increases and so the sector width, the accuracy degrades since the range of
error in a sector increases, however, the blocker gets more tracked as the
probability to have interferers in the sector increases (see Eq. (12)).
Consequently, the size of the sector should be carefully tuned so that to find
a trade-off between the accuracy and the continuity of detection.

##  V Conclusion

In the context of predicting and avoiding blockages in mmWave systems, this
paper presents a new mechanism that detects moving objects in the surrounding
of a particular communication link. This approach senses the radio environment
using side-lobe information in order to detect moving objects. It relies on
observing the fluctuation in the SINR values caused by the presence of the
blocker in angular sectors around the communication link of interest. We show
that it is capable of detecting moving objects in a range of $360\degree$,
without requiring any additional system unlike other reported methods. This
indeed further provides information on the position, direction, trajectory and
certainly the velocity of a moving object. Using this information, the node
would have enough time to handover to another available base station and avoid
service outage. In order to improve the accuracy of this approach, sharing
information between different entities in the network will be studied in
future work. We will also exploit this method in order to predict blockages
and manage resource allocation. Eventually, this work could be further
extended to cover mobile UEs and to consider multiple moving blockers.

## Acknowledgment

This work was supported by the french government under the Recovery Plan
(CRIIOT project).

## References

  * [1] J. G. Andrews, T. Bai, M. N. Kulkarni, A. Alkhateeb, A. K. Gupta, et al., “Modeling and Analyzing Millimeter Wave Cellular Systems,” IEEE Trans. on Communications, vol. 65, no. 1, pp. 403–430, 2016.
  * [2] M. Sana, A. De Domenico, E. Calvanese Strinati, et al., “Multi-Agent Deep Reinforcement Learning For Distributed Handover Management In Dense MmWave Networks,” in proc. IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 8976–8980, 2020.
  * [3] F. Liu, Y. Cui, C. Masouros, J. Xu, et al., “Integrated Sensing and Communications: Towards Dual-functional Wireless Networks for 6G and Beyond,” IEEE journal on selected areas in communications, 2022.
  * [4] “Joint Communication and Sensing in 6G Networks.” https://www.ericsson.com/en/blog/2021/10/joint-sensing-and-communication-6g.  Accessed: 2023/02/16.
  * [5] Y. Oguma, T. Nishio, K. Yamamoto, and M. Morikura, “Performance Modeling of Camera-assisted Proactive Base Station Selection for Human Blockage Problem in mmWave Communications,” in IEEE Wireless Communications and Networking Conference, pp. 1–7, IEEE, 2016.
  * [6] G. Charan, M. Alrabeiah, and A. Alkhateeb, “Vision-aided 6G Wireless Communications: Blockage Prediction and Proactive Handoff,” IEEE Trans. on Vehicular Technology, vol. 70, no. 10, pp. 10193–10208, 2021.
  * [7] U. Demirhan and A. Alkhateeb, “Radar aided Proactive Blockage Prediction in real-world Millimeter wave Systems,” in proc. IEEE International Conference on Communications, pp. 4547–4552, 2022.
  * [8] D. Marasinghe et al., “LiDAR aided Human Blockage Prediction for 6G,” in IEEE GLOBECOM Workshops, pp. 1–6, IEEE, 2021.
  * [9] Z. Ali, A. Duel-Hallen, and H. Hallen, “Early Warning of mmWave Signal Blockage and AoA Transition using sub-6 GHz Observations,” IEEE Communications Letters, vol. 24, no. 1, pp. 207–211, 2019.
  * [10] M. Alrabeiah and A. Alkhateeb, “Deep learning for mmWave beam and blockage prediction using sub-6 GHz channels,” IEEE Trans. on Communications, vol. 68, no. 9, pp. 5504–5518, 2020.
  * [11] S. Wu et al., “Blockage prediction using wireless signatures: Deep learning enables real-world demonstration,” IEEE Open Journal of the Communications Society, vol. 3, pp. 776–796, 2022.
  * [12] Y. Koda et al., “Handover Management for mmwave Networks with Proactive Performance Prediction using Camera Images and Deep Reinforcement Learning,” IEEE Trans. on Cognitive Communications and Networking, vol. 6, no. 2, pp. 802–816, 2019.
  * [13] R. Hersyandika et al., “Guard Beam: Protecting mmWave Communication through In-Band Early Blockage Prediction,” in proc. IEEE Global Communications Conference, pp. 4093–4098, 2022.
  * [14] M. Kamel et al., “Ultra-dense networks: A survey,” IEEE Communications surveys & tutorials, vol. 18, no. 4, pp. 2522–2545, 2016.
  * [15] M. Sana, N. di Pietro, and E. Calvanase Strinati, “Transferable and Distributed User Association Policies for 5G and Beyond Networks,” in proc. International Symposium on Personal, Indoor and Mobile Radio Communications (PIMRC), pp. 966–971, 2021.
  * [16] G. Yang and M. Xiao, “Performance Analysis of Millimeter-Wave Relaying: Impacts of Beamwidth and Self-Interference,” IEEE Trans. on Communications, vol. 66, no. 2, pp. 589–600, 2018.
  * [17] T. Bai and R. W. Heath, “Coverage and rate analysis for millimeter-wave cellular networks,” IEEE Trans. on Wireless Communications, vol. 14, no. 2, pp. 1100–1114, 2014.
  * [18] W. Qi, J. Huang, J. Sun, Y. Tan, C.-X. Wang, and X. Ge, “Measurements and Modeling of Human Blockage Effects for Multiple Millimeter Wave Bands,” in proc. International Wireless Communications and Mobile Computing Conference (IWCMC), pp. 1604–1609, IEEE, 2017.
  * [19] Z. Gao, L. Dai, et al., “Spatially Common Sparsity Based Adaptive Channel Estimation and Feedback for FDD Massive MIMO,” IEEE Trans. on Signal Processing, vol. 63, no. 23, pp. 6169–6183, 2015.
  * [20] P. Zhao, K. Ma, Z. Wang, and S. Chen, “Virtual Angular-Domain Channel Estimation for FDD Based Massive MIMO Systems With Partial Orthogonal Pilot Design,” IEEE Trans. on Vehicular Technology, vol. 69, no. 5, pp. 5164–5178, 2020.
  * [21] V. D. Vrabie, J. I. Mars, and J.-L. Lacoume, “Modified Singular Value Decomposition by means of Independent Component Analysis,” Signal Processing, vol. 84, no. 3, pp. 645–652, 2004.
  * [22] A. Mudonhi, G. Makhoul, M. Lotti, R. D’Errico, and C. Oestges, “Mm-Wave Massive MIMO Channel Sounding in Industrial IoT Scenarios,” in proc. Joint European Conference on Networks and Communications & 6G Summit (EuCNC/6G Summit), pp. 53–58, IEEE, 2022.


# GD-COMET: A Geo-Diverse Commonsense Inference Model

Mehar Bhatia  Vered Shwartz
University of British Columbia
Vector Institute for AI
{meharb23<EMAIL_ADDRESS>
###### Abstract

With the increasing integration of AI into everyday life, it’s becoming
crucial to design AI systems that serve users from diverse backgrounds by
making them culturally aware. In this paper, we present gd-comet, a geo-
diverse version of the COMET commonsense inference model. gd-comet goes beyond
Western commonsense knowledge and is capable of generating inferences
pertaining to a broad range of cultures. We demonstrate the effectiveness of
gd-comet through a comprehensive human evaluation across 5 diverse cultures,
as well as extrinsic evaluation on a geo-diverse task. The evaluation shows
that gd-comet captures and generates culturally nuanced commonsense knowledge,
demonstrating its potential to benefit NLP applications across the board and
contribute to making NLP more inclusive.

##  1 Introduction

Culture plays a significant role in shaping an individual’s worldviews,
beliefs, behaviours, and communication styles Spradley (1987). A considerable
portion of what is commonly referred to as commonsense knowledge is not
universal but rather culture-specific, including social norms, values,
traditions, and more. An example of cultural differences is greetings, which
may involve a handshake in Western cultures, bowing in some Asian cultures, a
‘namaste’ gesture in India, or ‘wai’ in Thailand.

Figure 1: Inferences from COMET and gd-comet for the sentence “PersonX eats a
dutch baby”, demonstrating lack of culture awareness in COMET.

With AI systems becoming increasingly ubiquitous in society, it is imperative
to go beyond the Western cultural perspective Hershcovich et al. (2022). Lack
of cultural awareness may lead to models perpetuating stereotypes and
reinforcing societal inequalities Hutchinson et al. (2020); Ross et al.
(2021); Søgaard (2022), impeding their effectiveness for users from non-
Western countries.

In this paper, we focus on a popular model for commonsense reasoning, COMET
Bosselut et al. (2019), which is based on an English language model (LM) and
further trained on commonsense inferences collected from North American
crowdsource workers Sap et al. (2019). Consequently, the model exhibits a
certain bias towards the North American cultural perspective. As evidenced by
Fig. 1, COMET displays limited familiarity with the concept of a German
pancake, erroneously interpreting the term “dutch baby” in a literal sense.

We identify a need for more inclusive commonsense reasoning models and propose
gd-comet: Geo-Diverse COMET. As demonstrated in Fig 1, gd-comet gained the
culturally relevant knowledge to interpret “dutch baby” as a legitimate dish.

gd-comet is similarly based on an English LM but is trained on a knowledge
base of cultural knowledge Nguyen et al. (2023) prior to training on COMET’s
original training data. This simple approach is effective, as judged by both
human evaluations as well as extrinsic evaluation on a geo-diverse task Yin et
al. (2021). gd-comet can potentially benefit many downstream NLP applications
where the user population is diverse.111Code available at
github.com/meharbhatia/GD-COMET

##  2 Background

###  2.1 Commonsense Inference Models

Many NLP tasks require reasoning beyond what is explicitly stated in the text.
People fill in those gaps with their commonsense knowledge. NLP models attempt
to do the same by leveraging commonsense knowledge bases (KBs) such as
ConceptNet Speer et al. (2017) and ATOMIC Sap et al. (2019). To achieve better
coverage, knowledge models such as COMET Bosselut et al. (2019) are based on
pre-trained LMs and further fine-tuned on KBs, enabling contextually-relevant
inferences along the KB’s dimensions for new contexts.

COMET’s hybrid approach proved useful for various tasks (e.g., Chakrabarty et
al., 2020; Ammanabrolu et al., 2021; Ravi et al., 2023a). Subsequent versions
of COMET have been developed to draw inferences from paragraphs Gabriel et al.
(2021), images Park et al. (2020), and complex sentences Ravi et al. (2023b).
Further improvements include obtaining additional training data through
crowdsourcing Hwang et al. (2021) or generating synthetic data from LMs West
et al. (2022).

COMET and its successors assume the universality of commonsense knowledge, yet
much of this knowledge may differ among cultures, in traditions (e.g.,
duration of a wedding ceremony; Acharya et al., 2021), foods (e.g., what
counts as breakfast food; Speer et al., 2017), social norms, and more.

###  2.2 Culture-Aware NLP

While multilingual NLP is a popular topic, culture-aware NLP is under-
explored. It is crucial for language technologies to not only serve speakers
of a wide variety of languages but also acknowledge that users come from
diverse cultures Hershcovich et al. (2022). Cultural norms and pragmatic
aspects differ across speakers from different cultures Zhou et al. (2023).
Nevertheless, English LMs primarily reflect a North-American lens due to
training on web data with a US user bias Cao et al. (2023).

Current work in culture-aware NLP addresses various aspects. One line of work
focuses on cultural stereotypes and biases, and ways to measure and mitigate
them (e.g., Hutchinson et al., 2020; Ross et al., 2021; Søgaard, 2022).
Another line of work analyzes the differences in culture-specific commonsense
knowledge, including relational knowledge Yin et al. (2022), grounding of time
expressions Shwartz (2022), food-related customs Palta and Rudinger (2023) and
social values Lin et al. (2021); Arora et al. (2023). At the same time, there
have been efforts to develop benchmarks Yin et al. (2021); Liu et al. (2021),
and adapt models to new cultures Zhou et al. (2023); Yin et al. (2023).
Finally, there are several recent cultural KBs such as StereoKG Deshpande et
al. (2022), Quasimodo Romero et al. (2019), and CANDLE Nguyen et al. (2023).
CANDLE, which we use in this work, is the most comprehensive among them,
containing 1.1M assertions in English about 386 cultures (e.g. “A Dutch baby
is a German pancake that is baked instead of cooked on the stove top”). CANDLE
assertions were extracted from a large web corpus and clustered into _5 facets
of culture_ : food, drinks, clothing, rituals, and traditions.

##  3 gd-comet

We present gd-comet, a geo-diverse version of COMET. The goal of gd-comet is
to generate high-quality commonsense inferences for concepts and events
pertaining to both Western and non-Western cultures. Rather than collecting a
large-scale geo-diverse dataset in the style of ATOMIC, we split the training
into two phases: (1) training the underlying LM on geo-diverse data; (2)
continue training on the large-scale original COMET training data. This is
motivated by Bosselut et al. (2019) that showed that implicit commonsense
knowledge from underlying LM’s pre-training transfers to COMET. We similarly
hypothesize that encoding geo-diverse data into the underlying LM prior to
training on COMET data will transfer this knowledge to gd-comet.

#### Geo-Diverse Training (GD-BART).

We pick 770,000 assertions from CANDLE with a combined score greater than 0.5.
This threshold selects highly distinctive assertions specific and relevant to
the specific region. We fine-tune BART-Large, the underlying LM of the latest
COMET model Hwang et al. (2021), on this data, using BART’s original pre-
training objectives (token masking, token deletion, text infilling and
sentence permutation). We save the model checkpoint with the lowest validation
loss after training for 50 epochs on two NVIDIA A40 GPUs.

#### COMET Training.

We proceed to fine-tuning GD-BART on the large-scale ATOMIC-2020 dataset,
using the same training method and hyper-parameters as Hwang et al. (2021).
Appendix A lists the 34 COMET relations used in this paper.

##  4 Intrinsic Evaluation

|  | 1 | 2 | 3 | Average $\kappa$
---|---|---|---|---|---
COMET | India |  2 .32  |  2 .16  |  2 .65  |  0 .71
S Korea |  1 .93  |  1 .86  |  2 .32  |  0 .67
Nigeria |  1 .97  |  1 .98  |  2 .27  |  0 .61
Iran |  2 .09  |  2 .31  |  2 .42  |  0 .63
Indonesia |  2 .28  |  2 .36  |  2 .55  |  0 .66

|  | 1 | 2 | 3 | Average $\kappa$
---|---|---|---|---|---
gd-comet | India |  2 .62  |  2 .54  |  2 .73  |  0 .74
S Korea |  2 .13  |  1 .92  |  2 .35  |  0 .65
Nigeria |  2 .25  |  1 .92  |  2 .35  |  0 .59
Iran |  2 .27  |  2 .38  |  2 .58  |  0 .76
Indonesia |  2 .43  |  2 .46  |  2 .58  |  0 .77

Table 1: Evaluation of COMET and gd-comet inferences, judged by annotators
from the respective cultures.

To evaluate the quality of gd-comet, we construct a set of input sentences
pertaining to 5 diverse cultures (Table 1). We sample 5 concepts for each
facet and use facet-specific templates (Appendix B) to create 20 sentences for
each culture. For each of COMET and gd-comet, we use beam search to generate 5
inferences for each of the 34 dimensions and convert them to natural language
statements using relation-specific templates based on prior work Bosselut et
al. (2019). The correctness of both inferences were judged by 10 graduate
students, two students from each of the respective cultures. Annotators were
asked to grade inferences along the following criteria on scale of 0 (worst)
to 3 (best):

  1. 1

Cultural Relevance: The inference is factually accurate and reflects the
values, customs, traditions, and societal norms associated with the given
culture.

  2. 2

Stereotype Avoidance: The inference does not perpetuate stereotypes about the
culture.

  3. 3

Linguistic Accuracy: The inference is grammatical, and the vocabulary and
idiomatic expressions are appropriate in that culture.

The annotations yielded a substantial inter-annotator agreement with $\kappa$
= 0.656 for COMET and 0.702 for gd-comet, measured with average Cohen’s Kappa
Cohen (1960) across cultures.

#### Results.

Table 1 reveals that gd-comet consistently outperforms the standard COMET
model. Specifically, gd-comet excels in generating culturally aligned
inferences across chosen diverse cultures, and is more likely than COMET to
avoid biased assumptions. However, there is still room for improvement for
South Korea and Nigeria.

##  5 Extrinsic Evaluation

Traditional benchmarks often fall short in testing models’ knowledge and
comprehension of diverse cultural contexts. To show gd-comet’s utility for
downstream tasks, we evaluate on a multimodal task, GD-VCR (Sec 5.1). We
develop a model inspired by VLC-BERT Ravi et al. (2023a) that generates
inferences and incorporates them into a vision and language (V&L) model (Sec
5.2). We show that gd-comet improves the performance on GD-VCR upon an array
of baselines (Sec 5.3) and demonstrate the inferences contributing to the
performance gains (Sec 5.4).

###  5.1 Dataset

Datasets | Human | VisualBERT* | ViLBERT* | VL-BERT | VLC-BERT w/
---|---|---|---|---|---
GD-BART | COMET | gd-comet
GD-VCR | 88.84 | 53.27 | 58.47 | 58.63 | 52.69 | 59.59 | 63.51
$\circ$ West | 91.23 | 65.82 | 64.37 | 65.27 | 57.69 | 66.78 | 69.93
$\circ$ South Asia | 92.98 | 52.04 | 62.9 | 64.92 | 54.35 | 64.25 | 68.17
$\circ$ Africa | 87.93 | 51.85 | 62.04 | 58.17 | 51.87 | 57.71 | 64.81
$\circ$ East Asia | 83.05 | 45.39 | 46.45 | 47.88 | 41.87 | 49.64 | 53.07
Table 2: Accuracy (%) of the different models on the subset of each region in
GD-VCR. We report the average across 3 runs (see Appendix D for the results of
individual seeds). Results marked with $*$ have been reported in Yin et al.
(2021).

Visual Commonsense Reasoning (VCR; Zellers et al., 2019) is a benchmark for
testing V&L models’ ability to understand and reason beyond a visual scene.
Each example consists of an image extracted from movies or TV series and a
multiple-choice question about the actions or people depicted in the image.
This dataset focuses solely on Western, primarily North American movies.

Geo-Diverse Visual Commonsense Reasoning dataset (GD-VCR; Yin et al., 2021)
follows the same setup of VCR but extends to diverse regions. This evaluation-
only dataset includes 328 images from movies and TV series in East Asian,
South Asian, African and Western countries (See Appendix C). We follow the
original setup and train our model on VCR before testing on GD-VCR.

###  5.2 Model (VLC-BERT with gd-comet)

We take inspiration from VLC-BERT Ravi et al. (2023a), that incorporated COMET
inferences into VL-BERT Su et al. (2019). Instead, we integrate GD-COMET as a
source of contextualized cultural commonsense knowledge for GD-VCR. Figure 2
illustrates the model. We describe below VLC-BERT and where our model deviates
from it.

Figure 2: A model using gd-comet for GD-VCR.

#### Knowledge Generation and Selection.

VLC-BERT uses the question and the object tags as input to COMET. Instead of
object tags, we generate an image caption using BLIP Li et al. (2023) and
extract noun phrases from the caption using SpaCy Honnibal et al. (2020). We
found that the noun phrases provide a more detailed description of the
depicted activities within the image (e.g. “family, burn” in Fig. 2). We
additionally append a country tag to the input. During training on VCR, we use
the tag “North America”, the primary source of movies in the dataset. For the
images in GD-VCR, we extracted country tags from Wikipedia.

We use beam search to generate five inferences for each of the 34 dimensions.
To select the most relevant inferences, we convert the inferences to natural
language sentences using relation-specific templates and select the inferences
that are the most similar to the question using SBERT embeddings Reimers and
Gurevych (2019).

#### Overall Architecture.

The generic input to VL-BERT for VCR is <question, answer tokens, image
regions>. Following Ravi et al. (2023a), we embed each inference with SBERT
and summarize them into a single token with a weighted average based on
learned attention scores. Finally, we feed the output of the [CLS] token into
a classifier to predict a score for each answer choice. We train the model
using binary cross-entropy loss for 20 epochs on 4 NVIDIA RTX6000 GPUs.

Figure 3: Attention analysis of commonsense inferences generated by COMET and
gd-comet for testing samples in GD-VCR.

###  5.3 Results

Table 2 compares our model’s performance on GD-VCR with baselines that: (i) do
not make use of commonsense knowledge (VL-BERT); (ii) generate inferences
using GD-BART; and (iii) use COMET (VLC-BERT w/COMET). Note that the same
signals (i.e., country tag and noun phrases) were used for the GD-BART and
COMET baselines. We also include prior results reported using VisualBERT and
ViLBERT for completeness.

VLC-BERT w/COMET modestly improves upon VL-BERT across most regions, with an
overall improvement of 1.2 points in accuracy. This suggests that COMET
provides some commonsense inferences that are universal. Conversely, gd-comet
shows a substantial improvement of nearly 5 points over VL-BERT and 4 points
over VLC-BERT w/COMET. This highlights the effectiveness of incorporating gd-
comet for downstream tasks that require culture-specific knowledge across
diverse regions. Furthermore, GD-BART performs less effectively than other
methods, underscoring the importance of training on structured knowledge to
generate contextually relevant responses.

###  5.4 Qualitative Analysis

Figures 3 presents several GD-VCR instances along with the models’
predictions, and the inferences generated by COMET and gd-comet for them. In
Figure 3a, gd-comet accurately associates a girl wearing henna in Somalia with
marriage. In Figure 3b, it understands that folding palms during an Indian
festival signifies a greeting or welcome. Finally, in Figure 3c, it recognizes
that bowing in South Korea is a gesture of apology, leading to VLC-BERT w/ gd-
comet to be the only model that provides a correct answer. In contrast,
COMET’s inferences for this example are generic and irrelevant. These examples
highlight gd-comet’s effectiveness in identifying the cultural context and
dynamically generating culturally-relevant commonsense inferences across
ATOMIC’s relations.

##  6 Conclusion

This work challenges the current notion of universally applicable commonsense
knowledge by introducing gd-comet, a geo-diverse variant of COMET. gd-comet
can generate culturally-nuanced commonsense inferences for a broad range of
cultures. Our comprehensive evaluation confirms the effectiveness of gd-comet
in incorporating and leveraging cultural cues. We view our work as a step
towards developing more inclusive and culturally-aware AI systems.

## Limitations

While gd-comet represents a significant advancement in incorporating cultural
commonsense knowledge into AI models, a few limitations need to be
acknowledged.

First, the availability of comprehensive, high-quality data remains a
challenge in training culturally-aware models. While resources like CANDLE
provide a step forward in curating diverse cultural knowledge, it is essential
to note that merely capturing the existence of concepts within a culture is
insufficient. Future efforts should aim to collect data that reflects the
presence of certain concepts and encompasses how people perceive and interpret
those concepts within their specific cultural contexts. This would require
extensive data collection efforts that go beyond surface-level understanding,
and delve into the nuances of cultural perspectives.

A second limitation is the availability of suitable benchmarks for testing
models’ knowledge and understanding of cultural variations. In particular, two
such tasks, GD-VCR and MarVL Liu et al. (2021), focus on vision and language,
while Nguyen et al. (2023) proposes a cultural knowledge quiz. We hope to see
more language-only datasets developed to go beyond testing models on knowledge
about concepts from diverse cultures to understanding cultural nuances.

## Ethics Statement

Despite being designed to be more culturally inclusive, gd-comet runs the risk
of unintentionally perpetuating biases present in CANDLE data. In particular,
CANDLE might misrepresent cultures with stereotypes or underrepresent
cultures. Addressing these concerns requires proactive measures such as
identifying biases using methods such as Mehrabi et al. (2021) and mitigating
them through filtering and additional data collection.

Additionally, the size of evaluation benchmarks means they don’t always
account for cultural variations within the same region. For example, GD-VCR
images in the African region are concentrated in East Africa. Similarly,
addressing this issue would require additional annotation efforts.

## Acknowledgement

This work was funded, in part, by the Vector Institute for AI, Canada CIFAR AI
Chairs program, an NSERC discovery grant, and a research gift from AI2.
Finally, we sincerely thank Sahithya Ravi, Aditya Chinchure, Ward Pennink and
Jan Zimny for valuable feedback and discussions.

## References

  * Acharya et al. (2021) Anurag Acharya, Kartik Talamadupula, and Mark A Finlayson. 2021.  Towards an atlas of cultural commonsense for machine reasoning.  In _Proceedings of the Workshop on Common Sense Knowledge Graphs (CSKGs) at AAAI_.
  * Ammanabrolu et al. (2021) Prithviraj Ammanabrolu, Wesley Cheung, William Broniec, and Mark O Riedl. 2021.  Automated storytelling via causal, commonsense plot ordering.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , volume 35, pages 5859–5867.
  * Arora et al. (2023) Arnav Arora, Lucie-aimée Kaffee, and Isabelle Augenstein. 2023.  Probing pre-trained language models for cross-cultural differences in values.  In _Proceedings of the First Workshop on Cross-Cultural Considerations in NLP (C3NLP)_ , pages 114–130, Dubrovnik, Croatia. Association for Computational Linguistics.
  * Bosselut et al. (2019) Antoine Bosselut, Hannah Rashkin, Maarten Sap, Chaitanya Malaviya, Asli Celikyilmaz, and Yejin Choi. 2019.  COMET: Commonsense transformers for automatic knowledge graph construction.  In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_ , pages 4762–4779, Florence, Italy. Association for Computational Linguistics.
  * Cao et al. (2023) Yong Cao, Li Zhou, Seolhwa Lee, Laura Cabello, Min Chen, and Daniel Hershcovich. 2023.  Assessing cross-cultural alignment between ChatGPT and human societies: An empirical study.  In _Proceedings of the First Workshop on Cross-Cultural Considerations in NLP (C3NLP)_ , pages 53–67, Dubrovnik, Croatia. Association for Computational Linguistics.
  * Chakrabarty et al. (2020) Tuhin Chakrabarty, Debanjan Ghosh, Smaranda Muresan, and Nanyun Peng. 2020.  R^3: Reverse, retrieve, and rank for sarcasm generation with commonsense knowledge.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 7976–7986, Online. Association for Computational Linguistics.
  * Cohen (1960) Jacob Cohen. 1960.  A coefficient of agreement for nominal scales.  _Educational and psychological measurement_ , 20(1):37–46.
  * Deshpande et al. (2022) Awantee Deshpande, Dana Ruiter, Marius Mosbach, and Dietrich Klakow. 2022.  StereoKG: Data-driven knowledge graph construction for cultural knowledge and stereotypes.  In _Proceedings of the Sixth Workshop on Online Abuse and Harms (WOAH)_ , pages 67–78, Seattle, Washington (Hybrid). Association for Computational Linguistics.
  * Gabriel et al. (2021) Saadia Gabriel, Chandra Bhagavatula, Vered Shwartz, Ronan Le Bras, Maxwell Forbes, and Yejin Choi. 2021.  Paragraph-level commonsense transformers with recurrent memory.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , volume 35, pages 12857–12865.
  * Hershcovich et al. (2022) Daniel Hershcovich, Stella Frank, Heather Lent, Miryam de Lhoneux, Mostafa Abdou, Stephanie Brandl, Emanuele Bugliarello, Laura Cabello Piqueras, Ilias Chalkidis, Ruixiang Cui, Constanza Fierro, Katerina Margatina, Phillip Rust, and Anders Søgaard. 2022.  Challenges and strategies in cross-cultural NLP.  In _Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 6997–7013, Dublin, Ireland. Association for Computational Linguistics.
  * Honnibal et al. (2020) Matthew Honnibal, Ines Montani, Sofie Van Landeghem, Adriane Boyd, et al. 2020.  spacy: Industrial-strength natural language processing in python.
  * Hutchinson et al. (2020) Ben Hutchinson, Vinodkumar Prabhakaran, Emily Denton, Kellie Webster, Yu Zhong, and Stephen Denuyl. 2020.  Social biases in NLP models as barriers for persons with disabilities.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 5491–5501, Online. Association for Computational Linguistics.
  * Hwang et al. (2021) Jena D Hwang, Chandra Bhagavatula, Ronan Le Bras, Jeff Da, Keisuke Sakaguchi, Antoine Bosselut, and Yejin Choi. 2021.  (COMET-)ATOMIC 2020: On symbolic and neural commonsense knowledge graphs.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , volume 35, pages 6384–6392.
  * Li et al. (2023) Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. 2023.  Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models.  _arXiv preprint arXiv:2301.12597_.
  * Lin et al. (2021) Bill Yuchen Lin, Seyeon Lee, Xiaoyang Qiao, and Xiang Ren. 2021.  Common sense beyond English: Evaluating and improving multilingual language models for commonsense reasoning.  In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_ , pages 1274–1287, Online. Association for Computational Linguistics.
  * Liu et al. (2021) Fangyu Liu, Emanuele Bugliarello, Edoardo Maria Ponti, Siva Reddy, Nigel Collier, and Desmond Elliott. 2021.  Visually grounded reasoning across languages and cultures.  In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_ , pages 10467–10485, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
  * Mehrabi et al. (2021) Ninareh Mehrabi, Pei Zhou, Fred Morstatter, Jay Pujara, Xiang Ren, and Aram Galstyan. 2021.  Lawyers are dishonest? quantifying representational harms in commonsense knowledge resources.  In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_ , pages 5016–5033, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
  * Nguyen et al. (2023) Tuan-Phong Nguyen, Simon Razniewski, Aparna Varde, and Gerhard Weikum. 2023.  Extracting cultural commonsense knowledge at scale.  In _Proceedings of the ACM Web Conference 2023_ , WWW ’23, page 1907–1917, New York, NY, USA. Association for Computing Machinery.
  * Palta and Rudinger (2023) Shramay Palta and Rachel Rudinger. 2023.  FORK: A bite-sized test set for probing culinary cultural biases in commonsense reasoning models.  In _Findings of the Association for Computational Linguistics: ACL 2023_ , pages 9952–9962, Toronto, Canada. Association for Computational Linguistics.
  * Park et al. (2020) Jae Sung Park, Chandra Bhagavatula, Roozbeh Mottaghi, Ali Farhadi, and Yejin Choi. 2020.  Visualcomet: Reasoning about the dynamic context of a still image.  In _In Proceedings of the European Conference on Computer Vision (ECCV)_.
  * Ravi et al. (2023a) Sahithya Ravi, Aditya Chinchure, Leonid Sigal, Renjie Liao, and Vered Shwartz. 2023a.  Vlc-bert: Visual question answering with contextualized commonsense knowledge.  In _Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision_ , pages 1155–1165.
  * Ravi et al. (2023b) Sahithya Ravi, Raymond Ng, and Vered Shwartz. 2023b.  Comet-m: Reasoning about multiple events in complex sentences.  _arXiv preprint arXiv:2305.14617_.
  * Reimers and Gurevych (2019) Nils Reimers and Iryna Gurevych. 2019.  Sentence-BERT: Sentence embeddings using Siamese BERT-networks.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pages 3982–3992, Hong Kong, China. Association for Computational Linguistics.
  * Romero et al. (2019) Julien Romero, Simon Razniewski, Koninika Pal, Jeff Z. Pan, Archit Sakhadeo, and Gerhard Weikum. 2019.  Commonsense properties from query logs and question answering forums.  In _Proceedings of the 28th ACM International Conference on Information and Knowledge Management_ , pages 1411–1420.
  * Ross et al. (2021) Candace Ross, Boris Katz, and Andrei Barbu. 2021.  Measuring social biases in grounded vision and language embeddings.  In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 998–1008, Online. Association for Computational Linguistics.
  * Sap et al. (2019) Maarten Sap, Ronan Le Bras, Emily Allaway, Chandra Bhagavatula, Nicholas Lourie, Hannah Rashkin, Brendan Roof, Noah A Smith, and Yejin Choi. 2019.  Atomic: An atlas of machine commonsense for if-then reasoning.  In _Proceedings of the AAAI conference on artificial intelligence_ , volume 33, pages 3027–3035.
  * Shwartz (2022) Vered Shwartz. 2022.  Good night at 4 pm?! time expressions in different cultures.  In _Findings of the Association for Computational Linguistics: ACL 2022_ , pages 2842–2853, Dublin, Ireland. Association for Computational Linguistics.
  * Søgaard (2022) Anders Søgaard. 2022.  Should we ban English NLP for a year? In _Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing_ , pages 5254–5260, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
  * Speer et al. (2017) Robyn Speer, Joshua Chin, and Catherine Havasi. 2017.  Conceptnet 5.5: An open multilingual graph of general knowledge.  In _Proceedings of the AAAI conference on artificial intelligence_ , volume 31.
  * Spradley (1987) James P Spradley. 1987.  _Culture and cognition: Rules, maps, and plans_.  Waveland PressInc.
  * Su et al. (2019) Weijie Su, Xizhou Zhu, Yue Cao, Bin Li, Lewei Lu, Furu Wei, and Jifeng Dai. 2019.  VL-BERT: Pre-training of generic visual-linguistic representations.  _arXiv preprint arXiv:1908.08530_.
  * West et al. (2022) Peter West, Chandra Bhagavatula, Jack Hessel, Jena Hwang, Liwei Jiang, Ronan Le Bras, Ximing Lu, Sean Welleck, and Yejin Choi. 2022.  Symbolic knowledge distillation: from general language models to commonsense models.  In _Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 4602–4625, Seattle, United States. Association for Computational Linguistics.
  * Yin et al. (2022) Da Yin, Hritik Bansal, Masoud Monajatipoor, Liunian Harold Li, and Kai-Wei Chang. 2022.  GeoMLAMA: Geo-diverse commonsense probing on multilingual pre-trained language models.  In _Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing_ , pages 2039–2055, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
  * Yin et al. (2023) Da Yin, Feng Gao, Govind Thattai, Michael Johnston, and Kai-Wei Chang. 2023.  Givl: Improving geographical inclusivity of vision-language models with pre-training methods.  _arXiv preprint arXiv:2301.01893_.
  * Yin et al. (2021) Da Yin, Liunian Harold Li, Ziniu Hu, Nanyun Peng, and Kai-Wei Chang. 2021.  Broaden the vision: Geo-diverse visual commonsense reasoning.  In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_ , pages 2115–2129, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
  * Zellers et al. (2019) Rowan Zellers, Yonatan Bisk, Ali Farhadi, and Yejin Choi. 2019.  From recognition to cognition: Visual commonsense reasoning.  In _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , pages 6720–6731.
  * Zhou et al. (2023) Li Zhou, Laura Cabello, Yong Cao, and Daniel Hershcovich. 2023.  Cross-cultural transfer learning for Chinese offensive language detection.  In _Proceedings of the First Workshop on Cross-Cultural Considerations in NLP (C3NLP)_ , pages 8–15, Dubrovnik, Croatia. Association for Computational Linguistics.

##  Appendix A COMET Relations

Table 3 lists COMET relations used in this work.

AtLocation | CapableOf | isBefore
---|---|---
Causes | CausesDesire | isFilledBy
CreatedBy | Desires | oEffect
HasPrerequisite | HasFirstSubevent | oReact
HasA | HasProperty | oWant
InstanceOf | IsA | xAttr
LocatedNear | MadeOf | xEffect
MadeUpOf | MotivatedByGoal | xIntent
ObjectUse | PartOf | xNeed
ReceivesAction | SymbolOf | xReact
UsedFor | isAfter | xReason
xWant |  |
Table 3: COMET relations used in this work.

##  Appendix B Facet Templates

Table 4 presents the templates used for creating input sentences to gd-comet
for each concept associated with a cultural facet.

clothing | ‘‘PersonX wears [concept] in [country]’’
---|---
food | ‘‘PersonX eats [concept] in [country]’’
drink | ‘‘PersonX drinks [concept] in [country]’’
festival | ‘‘PersonX celebrates [concept] in [country]’’
Table 4: Templates used to create input sentences for gd-comet for each CANDLE
facet.

##  Appendix C VCR and GD-VCR Statistics

Table 5 displays the statistics of the VCR and GD-VCR datasets. The bottom
half shows the number of images for each region in GD-VCR.

| # Images | # QA Pairs | avg Q | avg A
---|---|---|---|---
|  |  | length | length
VCR (dev) | 9929 | 26534 | 6.77 | 7.67
GD-VCR | 328 | 886 | 7.38 | 7.68
West | 100 | 275 | 7.36 | 7.19
East Asia | 101 | 282 | 7.59 | 7.59
South Asia | 87 | 221 | 6.85 | 8.00
Africa | 40 | 108 | 7.98 | 8.54
Table 5: Statistics of the VCR and GD-VCR benchmarks.

##  Appendix D Full Performance

Models | Overall | West | South Asia | East Asia | Africa
---|---|---|---|---|---
Human Performance | 88.84 | 91.23 | 92.98 | 83.05 | 87.93
VisualBERT* | 53.27 | 62.91 | 52.04 | 45.39 | 51.85
ViLBERT* | 58.47 | 65.82 | 62.9 | 46.45 | 62.04
VL-BERT (seed 1) | 58.8 | 64.73 | 67.42 | 48.58 | 54.78
VL-BERT (seed 2) | 58.92 | 65.82 | 63.8 | 47.16 | 62.04
VL-BERT (seed 3) | 58.19 | 65.27 | 63.54 | 47.91 | 59.7
VL-BERT (average) | 58.63 | 65.27 | 64.92 | 47.88 | 58.17
VLC-BERT with GD-BART (seed 1) | 53.63 | 58.54 | 54.08 | 41.42 | 52.78
VLC-BERT with GD-BART (seed 2) | 52.92 | 57.27 | 54.08 | 42.09 | 52.13
VLC-BERT with GD-BART (seed 3) | 51.55 | 57.27 | 54.89 | 42.09 | 50.70
VLC-BERT with GD-BART (average) | 52.69 | 57.69 | 54.35 | 41.87 | 51.87
VLC-BERT with COMET (seed 1) | 59.71 | 67.27 | 64.71 | 50.00 | 55.56
VLC-BERT with COMET (seed 2) | 59.82 | 67.27 | 64.71 | 50.36 | 55.56
VLC-BERT with COMET (seed 3) | 59.25 | 65.82 | 63.35 | 48.58 | 62.04
VLC-BERT with COMET (average) | 59.59 | 66.78 | 64.25 | 49.64 | 57.71
VLC-BERT with gd-comet (seed1) | 62.87 | 67.64 | 70.14 | 51.77 | 64.81
VLC-BERT with gd-comet (seed2) | 65.01 | 72.36 | 67.42 | 54.97 | 67.59
VLC-BERT with gd-comet (seed3) | 62.64 | 69.82 | 66.97 | 52.48 | 62.03
VLC-BERT with gd-comet (average) | 63.51 | 69.93 | 68.17 | 53.07 | 64.81
Table 6: Accuracy (%) of the various models on the subset of each region in
GD-VCR. Results marked with $*$ have been reported in Yin et al. (2021).

Extending upon Table 2, we provide a complete summary of the results of
individual seeds on GD-VCR in Table 6.


# Lecture notes on stationary critical and super-critical SPDEs111Notes for
the lecture series “Large-scale behavior of (super)-critical stochastic PDEs”
by F. Toninelli, CIME summer school “Statistical Mechanics and Stochastic
PDEs”, Cetraro (Italy), 2023

###### Abstract

The goal of these lecture notes is to present recent results regarding the
large-scale behaviour of critical and super-critical non-linear stochastic
PDEs, that fall outside the realm of the theory of Regularity Structures.
These include the two-dimensional Anisotropic KPZ equation, the stochastic
Burgers equation in dimension $d\geq 2$ and the stochastic Navier-Stokes
equation with divergence-free noise in dimension $d=2$. Rather than providing
complete proofs, we try to emphasise the main ideas, and some crucial aspects
of our approach: the role of the _generator equation_ and of the _fluctuation-
dissipation theorem_ to identify the limit process; _Wiener chaos
decomposition_ with respect to the stationary measure and its truncation; and
the so-called _Replacement Lemma_ , which controls the _weak coupling limit_
of the equations in the critical dimension and identifies the limiting
diffusivity. For pedagogical reasons, we will focus exclusively on the
stochastic Burgers equation. The notes are based on works in collaboration
with Dirk Erhard and Massimiliano Gubinelli.

Giuseppe Cannizzaro1, Fabio Toninelli2

1University of Warwick, UK, 2Technical University of Vienna, Austria

Key words and phrases. Stochastic Partial Differential Equations, critical and
super-critical dimension, KPZ and Burgers equation, weak coupling scaling,
diffusion coefficient, asymmetric simple exclusion

###### Contents

  1. 1 Introduction
    1. 1.1 The equations

##  1 Introduction

The study of (singular) Stochastic Partial Differential Equations (SPDEs) has
known tremendous advances in recent years. Thanks to the groundbreaking theory
of regularity structures [Hai], paracontrolled calculus [Para] or
renormalization group techniques [Kup], a local solution theory can be shown
to hold for a large family of SPDEs in the so-called sub-critical dimensions
(see [Hai] for the notion of sub-criticality in this context). In contrast,
SPDEs in the critical and super-critical dimensions are, from a mathematical
viewpoint, much less explored and still poorly understood.

In a series of works in collaboration with Dirk Erhard and Massimiliano
Gubinelli (in various combinations), we rigorously determined the large-scale
(Gaussian) fluctuations for a class of SPDEs at and above criticality under
(suitable) diffusive scaling. As detailed below, the class of equations to
which our approach applies include the two-dimensional Anisotropic KPZ
equation [W91, CES], the stochastic Burgers equation in dimension $d\geq 2$
[BKS85] and the stochastic Navier-Stokes equation with divergence-free noise
in dimension $d=2$. Moreover, the methods we established have proven versatile
enough to study the large-scale limit of critical, singular SDEs, such as the
diffusion in the curl of the 2-d GFF [TothValko, CHT], as well as the self-
repelling Brownian polymer [TothValko, CG] (works [CHT, CG] in collaboration
with Levi Haunschmid-Sibitz and Harry Giles, respectively). The aim of these
lecture notes is to give an account of these recent developments focusing on
the main ideas, heuristics and novel technical tools rather than to provide
the complete the proofs, for which we address the interested reader to the
corresponding papers.

Before delving into the details, let us informally introduce the type of
S(P)DEs which can be analysed thanks to the methods presented in these notes
and provide an overview of the main results we obtained. From Section
LABEL:sec:SBE on, we will focus on a specific example, the Stochastic Burgers
Equation (SBE) in dimension $d\geq 2$.

###  1.1 The equations

#### The stochastic Burgers Equation

The stochastic Burgers equation (SBE) is a SPDE which, in any spatial
dimension $d\geq 1$, can (formally) be written as

$\partial_{t}\eta=\tfrac{1}{2}\Delta\eta+\mathfrak{w}\cdot\nabla\eta^{2}+\mathop{\mathrm{div}}\nolimits\boldsymbol{\xi}{\hbox{\rm(1)}\kern
5.69046pt\kern-5.69046pt}:BKS{=}e0{1}$