11institutetext: Department of Astronomy and Atmospheric Sciences, Kyungpook
National University, Daegu 41566, Republic of Korea
11email<EMAIL_ADDRESS>22institutetext: Division of Science Education and
Institute of Science Education, Jeonbuk National University, Jeonju 54896,
Republic of Korea
22email<EMAIL_ADDRESS>33institutetext: Theoretical Astrophysics
Group, Korea Astronomy and Space Science Institute, Daejeon 34055, Republic of
Korea 44institutetext: Center for Theoretical Physics of the Universe,
Institute for Basic Science (IBS), Daejeon 34126, Republic of Korea

# CMASS galaxy sample and the ontological status of
the cosmological principle

Yigon Kim 11 Chan-Gyung Park, Corresponding author22 Hyerim Noh and Jai-chan
Hwang 331144

###### Abstract

Context. The cosmological principle (CP), assuming spatially homogeneous and
isotropic background geometry in the cosmological scale, is a fundamental
assumption in modern cosmology. Recent observations of the galaxy redshift
survey provide relevant data to confront the principle with observation.
Several previous studies claim that the homogeneity scale is reached at a
radius around $70\leavevmode\nobreak\ h^{-1}\textrm{Mpc}$. However, the same
observation shows a dramatic visual structure, SDSS Great Wall, which extends
$300\leavevmode\nobreak\ h^{-1}\textrm{Mpc}$ in linear dimension.

Aims. We present a homogeneity test for the matter distribution using the BOSS
DR12 CMASS galaxy sample and clarify the ontological status of the CP.

Methods. As a homogeneity criterion, we compare the observed data with
similarly constructed random distributions using the number count in the
truncated cones method. Comparisons are also made with three theoretical
results using the same method: (i) the dark matter halo mock catalogs from the
$N$-body simulation, (ii) the log-normal distributions derived from the
theoretical matter power spectrum, and (iii) direct estimation from the
theoretical power spectrum.

Results. We show that the observed distribution is statistically impossible as
a random distribution up to $300\leavevmode\nobreak\ h^{-1}\textrm{Mpc}$ in
radius, which is around the largest statistically available scale. However,
comparisons with the three theoretical results show that the observed
distribution is consistent with these theoretically derived results based on
the CP.

Conclusions. We show that the observed galaxy distribution (light) and the
simulated dark matter distribution (matter) are quite inhomogeneous even on a
large scale. Here, we clarify that there is no inconsistency surrounding the
ontological status of the CP in cosmology. In practice, the CP is applied to
the metric and the metric fluctuation is extremely small in all cosmological
scales. This allows the CP to be valid as the averaged background. The matter
fluctuation, however, is decoupled from the small nature of metric fluctuation
in the subhorizon scale. What is directly related to the matter in Einstein’s
gravity is the curvature, a quadratic derivative of the metric.

###### Key Words.:

large-scale structure of universe – methods: statistical – cosmology: theory

##  1 Introduction

Modern cosmology explains diverse cosmological observations using a relatively
simple model based on Einstein’s gravity modified by the cosmological
constant. The simple aspect of the model is not referring to Einstein’s
gravity (which is a set of highly nonlinear partial differential equations)
nor to the enigmatic cosmological constant. A decisive simplification to make
the problem mathematically tractable was made by Einstein (1917a), and the
assumption was later named as “Einstein’s Cosmological Principle” (Milne,
1934; McCrea & Milne, 1934). The assumption made in the 1917 paper was a
broad-brush wish “If we are concerned with the structure only on a large
scale, we may represent the matter to ourselves as being uniformly distributed
over enormous spaces.” Einstein himself regarded it as too simplistic,
mentioning in a letter to de Sitter in the same year (Einstein, 1917b) “From
the standpoint of astronomy, of course, I have erected but a lofty castle in
the air.” Concerning the validity of the assumption and its potential
testability, he was somewhat pessimistic, mentioning in the same letter,
“Whether the model I formed for myself corresponds to reality is another
question, about which we shall probably never gain information.”

Over the last 100 years, the cosmology Einstein has initiated with his
Cosmological Principle eventually (because the actual history might have been
more turbulent) turns out to be remarkably successful. It allows an easy
theoretical treatment and leads to the eventual success of the theory matching
with observations. The Principle now dominates the modern physical cosmology
without his name attached. The Cosmological Principle (CP) in modern cosmology
can be rephrased in a more precise term as follows: the universe is assumed to
be homogeneous and isotropic on a sufficiently large scale. Our question to be
addressed in this work, despite all the stunning successes in cosmology in
recent years, concerns: what is the sufficient scale? And although Einstein
mentioned the matter in the above quote, what is the CP referring to in
cosmology: in matter or in geometry? This subtle point is vital in our
argument, and even geometry needs later specification: in metric or in
curvature? The matter is tied with geometry (curvature) in Einstein’s gravity,
but not so tightly concerning deviations from the CP (in the metric),
especially on a small scale.

To clarify the case, let us introduce a term referring to the scale beyond
which the CP is valid, the Homogeneity Scale (HS), in measurable quantities
like the matter or the light. The HS depends on the definition of the
homogeneity the cosmology needs and also on the subject the CP is applied. The
homogeneity in theoretical cosmology is, in fact, a mathematical homogeneity
in both matter and geometry, which is obviously an idealization not existing
in nature. The CP represented in the Robertson-Walker metric first adopted by
Friedmann is based on the mathematical homogeneity on spatial geometry
(Friedman, 1922; Friedmann, 1924). It demands the same homogeneity in the
matter as well. Thus, the HS we wish to find out in the universe must be
something different from this idealized mathematical definition (i.e.,
absolute homogeneity). What is it, then? Is the CP confirmed in observation?
Where is the evidence? There is no doubt about the success of modern
cosmology. But, what is the nature of the CP in modern cosmology? We will
investigate these and other related questions in this work.

This paper has two main parts. In Section 2 we explain the status of the
cosmic homogeneity and the nature of the CP in modern cosmology. In Section 3
we search for the HS in matter and light using the BOSS CMASS sample of
galaxies by comparing with the random and a couple of mock catalogs based on
numerical simulation and the power spectrum. We also show a visual
presentation of the cosmic landscape, revealing significant inhomogeneity even
on the largest statistically available scale. Section 4 is a discussion of
future expectations and remaining issues.

##  2 Cosmic homogeneity

Before we probe these questions further, let us examine the observational
situations concerning the CP.

###  2.1 The CP cannot be found in light or matter

The cosmic microwave background radiation (CMB) shows a remarkable level of
isotropy in the temperature distribution displayed on the celestial sphere
centered around us: one-thousandth level in the dipole and $10^{-5}$ level in
higher multiple anisotropies are detected (Smoot et al., 1992; Spergel et al.,
2003; Planck Collaboration et al., 2014). If we disregard the dipole as the
one caused by our motion, the remaining level of isotropy is indeed
extraordinary. The isotropy of the CMB does not imply the homogeneity,
however, of the matter distribution in the three-dimensional space enclosed by
the recombination (last-scattering) surface of the CMB. The near-isotropy of
the CMB implies near-homogeneity of the three-dimensional source distribution
at the recombination epoch. But as the propagation of CMB light is affected by
the spacetime geometry, the near-isotropy only implies near-homogeneity of the
metric (geometry) fluctuation, which is different from the matter fluctuation.
Moreover, the isotropic CMB does not exclude the strong inhomogeneity along
the radial direction.

The set of angular locations of galaxies (or galaxy like objects) and their
redshifts provides more direct data for homogeneity test on the cosmic scale.
Different redshift implies different times in the past. Conversion of redshift
into distance requires a cosmological model that already assumes the
homogeneity in modern cosmology. Still, considering the success of the current
cosmological model, we can proceed with the homogeneity test as a trial case.
In this way, the success will be a consistency check of the CP, not proof.
Three different samples are available for the test: the SDSS Main galaxies
with $63,163$ objects nearby ($z<0.2$), the Luminous Red Galaxies (LRG) sample
with $105,831$ objects in the medium distance, and the CMASS sample with
$931,517$ objects far away (in the past) (Eisenstein et al., 2001; Abazajian
et al., 2009; Kazin et al., 2010; Eisenstein et al., 2011; Anderson et al.,
2012; Alam et al., 2015). The SDSS Main Galaxy Sample shows a well-known
panoramic structure with a colossal $300\leavevmode\nobreak\
h^{-1}\textrm{Mpc}$ in linear scale, the Sloan Great Wall (Gott et al., 2005).
The LRG sample looks more homogeneous, and several studies were made
concerning the homogeneity issue.

Hogg et al. (2005), using half of the final LRG objects, suggested
$70\leavevmode\nobreak\ h^{-1}\textrm{Mpc}$ in radius as the HS; there is even
a consensus on this HS in the literature (Yadav et al., 2005; Bagla et al.,
2008; Sarkar et al., 2009; Yadav et al., 2010; Scrimgeour et al., 2012;
Laurent et al., 2016; Ntelis et al., 2017); see, however, Sylos Labini et al.
(2009); Sylos Labini & Baryshev (2010); Sylos Labini (2011) for a contrary
view. Whereas Park et al. (2017), using the full LRG objects, found no HS up
to $300\leavevmode\nobreak\ h^{-1}\textrm{Mpc}$ in radius, which is around the
largest statistically available scale. Both papers used the behavior of number
counts within spheres (centered at the objects) as the radius of the sphere
increases; while Hogg et al. used the centers of objects in the three-
dimensional space, Park et al. also used the centers of objects in a narrowly
chosen redshift range (thus losing the number of centers while minimizing
mixed objects at different redshifts). Different criteria for the homogeneity
cause the different conclusions. Hogg et al. used a tendency of approaching
the homogeneous distribution of the average number count with (in our opinion)
an ad hoc cut-off criterion. Park et al. compared the galaxy counts with the
random distribution (for both average and dispersion) as the homogeneity
criterion and showed that the LRG distribution is statistically impossible to
be achieved by the random distribution even up to the largest scale available,
which is around $300\leavevmode\nobreak\ h^{-1}\textrm{Mpc}$ in radius. When
one says distribution is homogeneous, what else can we compare besides the
random Poisson one? Park et al. further compared the observation with the
simulation results and showed their consistency; thus, the simulation based on
the CP is similarly not homogeneous.

In this work, we extend the study of Park et al. (2017) now to the CMASS
sample with more objects and far away (in both space and time). Our answer
remains the same: the CP is not in the sky. The observed CMASS distribution
(in both average and dispersion) is statistically impossible (the high
$\sigma$’s appearing are staggering!) to be achieved by the random
distribution even up to the largest scale available, which is still around
$300\leavevmode\nobreak\ h^{-1}\textrm{Mpc}$ in radius, see Section 3.3. This
implies that the observed universe is far more clustered than the random
distribution in all currently available observation scales. Thus, the CP is
not in the light (which may or may not represent the matter).

We also compare the CMASS sample with mock galaxies generated from the
$N$-body simulation and linear matter power spectrum. Both mock data are based
on the theory with the CP in the background. As we will show below, both data
are consistent with the CMASS data and are far from homogeneity compared with
the random data, see Section 3.3. These comparisons confirm why the current
cosmological model is successful, and at the same time, show that the CP
cannot be found in these theoretical data, and nor is it demanded by the
theory. The level of inhomogeneity in the CMASS sample is visualized in Fig.
13. As both the $N$-body simulation and the linear matter power spectrum used
to generate mock galaxies are based on the cold dark matter, our comparisons
reveal that the CP is not in the matters, neither. Therefore, not in Nature.

###  2.2 Resolution

What does this seemingly contradicting situation imply? That is, the
theoretical cosmology based on the CP is successful, while the CP cannot be
found in the observed Universe.

The answer lies in distinguishing the CP theoretically applied to the metric
(geometry) from the attempt to find it in the distribution of matter which
includes luminous objects. When we say geometry, it is crucial to distinguish
between the metric and the curvature. The curvature is the second derivatives
of the metric, and the spatial curvature is the second spatial derivatives of
the metric. The point is that whereas the curvature is directly related to the
matter in Einstein’s equation, the CP is imposed on the metric. In this way,
even with huge matter inhomogeneity, the metric (a double integration of the
curvature inhomogeneity) can be smooth.

The fluctuations in the cosmic metric (related to the gravitational potential
in the Newtonian concept) are indeed quite small in the current cosmological
paradigm. These are around $10^{-4}$ level or even less in nearly all cosmic
scales except near compact objects like neutron stars and black holes, which
are astrophysical subjects rather than cosmology. The strength of metric
fluctuations is characterized by dimensionless combinations $GM/(\ell c^{2})$
and $v^{2}/c^{2}$ where $M$, $\ell$, and $v$ are characteristic scales of the
mass, length, and velocity involved in the system. Thus, in the spacetime
geometry, the deviations from Minkowski or Robertson-Walker metric are
extremely small ($10^{-4}$ level or less) in all cosmological scales. This is
the case independently of high inhomogeneity or nonlinearity in the matter
distribution, which is related to the curvature, the second derivative of the
metric fluctuations.

We can show the difference between geometry and matter as follows. The
Poisson’s equation (some limit of the energy constraint equation in Einstein’s
gravity) in expanding space can be written as $a^{-2}\Delta\delta\Phi=4\pi
G\delta\varrho\sim H^{2}\delta$ with $a$ the cosmic scale factor,
$H\equiv\dot{a}/a$ the Hubble-Lemaître parameter,
$\delta\equiv\delta\varrho/\varrho$ a dimensionless density parameter and
$\delta\Phi$ a perturbed gravitational potential (related to the perturbed
metric as $\delta\Phi/c^{2}$); the intrinsic curvature is of order
$a^{-2}\Delta\Phi/c^{2}$. With $\Delta/a^{2}\sim\ell^{-2}$ ($\ell$ a length
scale considered), the dimensionless potential (metric) fluctuation becomes
$G\delta M/(\ell
c^{2})\sim\delta\Phi/c^{2}\sim(\ell^{2}H^{2}/c^{2})\delta\sim(\ell/\ell_{H})^{2}\delta$,
with $\ell_{H}\equiv c/H$ the horizon scale. Thus, far inside the horizon
($\ell\ll\ell_{H}\sim 3\leavevmode\nobreak\ h^{-1}\textrm{Gpc}$ with the
present value of $H_{0}\equiv 100h$
$\textrm{km}\textrm{s}^{-1}\textrm{Mpc}^{-1}$), the density fluctuation can be
significant while the metric fluctuation remains quite small. This is the case
we encounter in the CMASS observations via light in this work.

Thus, we understand that while the CP is valid (in the metric!), the density
fluctuation is free from the same constraint far inside the horizon. The
remaining task is to check whether the observed fluctuations in the matter (or
light) are consistent with the theory based on the CP.

The gravitational instability causes the matter inhomogeneity to grow in time
even in an expanding background (Lifshitz, 1946). Thus, the early universe is
closer to homogeneity but should still have some inhomogeneity as the seeds
for later growth into the large-scale structure. The success of modern
cosmology is built on the assumption that the metric fluctuations are small
enough. Thus, the CP is imposed in the spacetime metric. In this way, a linear
stability analysis is sufficient to handle the formation and evolution
processes of the large-scale structure in the early universe and even
currently in the large-scale. If the deviations are small (say linear, meaning
we can safely ignore the nonlinear terms in the process), the fluctuations
cancel out by average over space so that the CP applies as the background.
This is the CP adopted in modern cosmology: a spatially homogeneous and
isotropic background plus small enough (preferably linear) perturbations in
geometry such that the background remains meaningful. Even in nonlinear
perturbation theory, the background is assumed to be intact (Hwang & Noh,
2013). In this way, the success of (linear) perturbation theory in the metric,
instead of actually taking the spatial average over the metric, can be
regarded as the validity of the CP in the background.

If the deviations are not small (i.e., the nonlinear terms in the metric
cannot be ignored), however, the fluctuations may not disappear by the
average, and the background universe where the CP is supposed to apply may
lose its meaning. In such a case Einstein’s CP may need modifications.
Historically there are some manageable (mainly only to the background model,
though) examples of inhomogeneous or anisotropic alternative CPs (Krasinski,
1997; Ellis et al., 2012). The extreme isotropy of the CMB should severely
constrain the homogeneous but anisotropic cosmological models. Another example
is the inhomogeneous but spherically symmetric Lemaître(-Tolman) model
(Lemaître, 1933) adopting Lemaître’s CP. As the observations become more
precise, finding situations where metric nonlinearities (i.e., deviations from
the CP) are important in the radial direction in the cosmic scale will be an
interesting discovery. No such a possibility is currently identified within
modern cosmology (Hwang & Noh, 2006; Jeong et al., 2011), meaning the CP is
consistent with observations.

The nonlinear evolution process of the large-scale structure (of baryon and
dark matter) occurs in the matter-dominated era (with the cosmological
constant) far inside the horizon. The numerical simulation based on Newtonian
gravity is assumed to be sufficient to handle this stage (Springel et al.,
2005; Kim et al., 2009; Teyssier et al., 2009; Prada et al., 2012; Angulo et
al., 2012; Heitmann et al., 2015; Kim et al., 2015; Habib et al., 2016; Potter
et al., 2017; Vogelsberger et al., 2020). The Newtonian gravity already
implies that the metric fluctuations deviating from the CP are negligibly
small. This is the case independently of whether the process is nonlinear or
not and how large the inhomogeneity is. The Newtonian theory is, in fact, a
$c$-goes-to-infinity limit of Einstein’s gravity. Thus $GM/(\ell c^{2})$ and
$v^{2}/c^{2}$ are naturally negligibly small. The first-order consideration of
these terms coming from Einstein’s gravity is known as the post-Newtonian
corrections, and currently, these are estimated to be negligibly small in
cosmology (Shibata & Asada, 1995; Hwang et al., 2008). The Poisson equation
shows that, as the scale becomes smaller than the horizon scale, the amplitude
of inhomogeneity in matter distributions $\delta$ is detached from the
potential (metric) fluctuation $\delta\Phi/c^{2}$.

In this work, we will show that the CMASS distribution of galaxies is
consistent with results from the numerical simulation and the theoretical
estimation of matter power spectrum; we use mock galaxy catalogs derived from
the Horizon Run 3 simulation (Kim et al., 2011) and the log-normal
distributions of matter density (Agrawal et al., 2017). Thus, the CP cannot be
found even in objects identified from the cosmological theory based on the CP.
We note that the numerical simulation handling the nonlinear process in the
Newtonian context is directly built on the CP. The simulation is performed in
a comoving coordinate where the background expansion is subtracted using
equations from Einstein’s gravity (or modified ones if needed) combined with
the CP. The fluctuations in the dimensionless gravitational potential from the
Horizon Run 4 simulation (Kim et al., 2015) show dispersions of
$\delta\Phi/c^{2}$ smaller than $10^{-4}$ in all scales we are considering
(Kim, 2021), thus extremely small indeed.

###  2.3 The ontological status of the CP

Therefore, as long as we accept the Robertson-Walker metric as the background,
the HS covers all cosmological scales concerning the metric fluctuations. The
CP applies as long as we can ignore nonlinearity in the metric fluctuations,
and the metric inhomogeneity in current cosmology is estimated to be $10^{-4}$
level or less in all cosmologically relevant scales. As the CP in modern
cosmology concerns the metric only, searching for the HS in the matter or
light is not relevant to the CP. In Section 3, using the CMASS sample, we will
show that the HS cannot be found up to the colossal $300\leavevmode\nobreak\
h^{-1}\textrm{Mpc}$ in radius; we already have an example in the SDSS Sloan
Great Wall showing a dramatic structure! But the distribution is consistent
with theoretical results derived from the CP, and even bigger structures are
expected in the simulation, see Park et al. (2012).

Our study shows that the CP can be found in neither matter nor light; up to
the scale at which the current observation is available. Thus, the CP is not
embedded in Nature. The CP, however, is well valid in the metric. The metric
fluctuations are sufficiently small in the observed cosmic scale, and this is
where the CP is needed in theoretical cosmology; the CP (in the metric) is
perfectly reasonable within modern cosmology (which already assumes the CP as
the background, though). Here is a subtle difference between the theory and
observation. The metric (or potential) is a theoretical concept not directly
observable. This reveals the ontological status of the CP, a prescient
theoretical postulation by Einstein, making all the current success of modern
cosmology possible.

##  3 Test

In this study, as a fiducial model we take a $\Lambda$CDM model with the
current matter and dark energy density parameters $\Omega_{m}=0.27$ and
$\Omega_{\Lambda}=0.73$, respectively. Here the dark energy is based on
Einstein’s cosmological constant $\Lambda$.

Figure 1: Angular distribution of CMASS galaxies of the Northern Galactic Cap
within a slice of $100\leavevmode\nobreak\ h^{-1}\textrm{Mpc}$ thickness
centered at the redshift $z_{\rm{cen}}=0.52$, shown in Mollweide equal-area
projection in the equatorial coordinate system ($\alpha$ and $\delta$ denote
right ascension and declination, respectively). In order to increase
visibility so that the distribution of galaxies is clearly visible, only 14%
of the total data points were randomly selected and displayed. For a
comparison of spatial scales, two circles with radii of $70$ and
$300h^{-1}\textrm{Mpc}$ have been added.

###  3.1 Data

####  3.1.1 BOSS CMASS sample

We used the CMASS sample of massive galaxies from the 12th data release of
SDSS-III Baryonic Oscillation Spectroscopic Survey (BOSS). The target galaxies
in the CMASS sample are selected so that the stellar mass of the systems is
nearly constant at a redshift range of $0.43<z<0.7$ (Eisenstein et al., 2011;
Anderson et al., 2012; Nuza et al., 2013). The CMASS sample is roughly a
volume-limited sample up to the redshift $z=0.5$, but after that it is close
to a magnitude-limited sample. The resulting sample is composed of luminous
galaxies with approximately uniform comoving space density of $3\times
10^{-4}h^{3}\textrm{Mpc}^{-3}$. Although the CMASS sample does not represent
all the galaxies due to various color-magnitude cuts applied to target
galaxies with a certain limit of stellar mass, it is more suitable for testing
homogeneity of matter distribution compared to the former LRG sample
(Eisenstein et al., 2001; Abazajian et al., 2009; Kazin et al., 2010) in the
sense that it provides a less biased sample of massive galaxies by allowing
galaxies with lower luminosity and wider color range (Dawson et al., 2013). It
provides a sufficient number of galaxies with the photometric and
spectroscopic information, a total of $931,517$ galaxies in the Northern
Galactic Cap (NGC) and the Southern Galactic Cap (SGC) that covers
$9,376\leavevmode\nobreak\ \textrm{deg}^{2}$, about one fourth of the whole
sky (Alam et al., 2015).

Figure 2: Number distributions of the LRG and CMASS galaxies over redshift.
The width of redshift in the histograms is $\Delta z=3.25\times 10^{-3}$. Also
shown are the redshift ranges enclosed by spheres with a series of radii from
$50$ to $300\leavevmode\nobreak\ h^{-1}\textrm{Mpc}$, centered at
$z_{\rm{cen}}=0.35$ for LRG and $z_{\rm{cen}}=0.52$ for CMASS (denoted as
vertical dotted lines). For radii of $250$ and $300\leavevmode\nobreak\
h^{-1}\textrm{Mpc}$ in the case of the CMASS sample, the redshift ranges have
been shifted to the higher redshift to include more galaxies within the range
specified.

In this paper, we use only $568,776$ galaxies contained in the NGC (Fig. 1).
This is five times larger than the number of LRG galaxies used in the previous
study (Park et al., 2017), enabling more reliable statistical calculations.
Figure 2 shows the radial distribution of the CMASS galaxies used in this
work, together with that of the LRG galaxies. The redshift ranges enclosed by
spheres that are located at the central redshift of both surveys are also
shown for a series of radii from $50$ to $300\leavevmode\nobreak\
h^{-1}\textrm{Mpc}$. Our fiducial $\Lambda\textrm{CDM}$ model has been used to
transform the redshift to comoving distance. We will use the spheres for the
CMASS sample to define the truncated cones with characteristic radius scale
and count CMASS galaxies within them for testing homogeneity. The CMASS radial
number distribution will be used to make random and mock catalogs so that the
data points can be located similarly to those in the CMASS sample.

Due to the mask of bright stars, centerposts of telescope, observation
failure, and a number of various reasons, the galaxies within the BOSS survey
region were not completely surveyed, and thus the missing and incomplete
regions exist. We reconstruct the completeness of the survey by combining the
BOSS survey boundaries and a collection of mask information provided by the
BOSS website 111The SDSS Science Archive Server,
https://data.sdss.org/sas/dr12/boss/lss/geometry/. The MANGLE software is used
to interpret and combine the mask files with polygon format (Hamilton &
Tegmark, 2004; Swanson et al., 2008). The final output of the MANGLE for the
completeness of the CMASS sample is transformed into the HEALPix format where
the individual pixels cover equal area on the sky (Górski et al., 2005). The
CMASS sector completeness within the BOSS survey region is shown in Fig. 3.
Hereafter, we call this completeness map as the angular selection function
(ASF) of the CMASS sample.

Figure 3: The sector completeness map of the BOSS DR12 CMASS sample. The grey
color indicates the outer region of the survey or no data.

In our analysis, we take into account the fact that different weights are
assigned to galaxies that were placed in various observation environments. For
example, a galaxy is given more weight if the nearest neighbor of this galaxy
is located within the diameter of the spectroscopic fiber and the redshift of
the neighboring galaxy was not obtained due to fiber collision. We correct for
the effect of the fiber collision ($w_{\rm{cp}}$) and the redshift failure
($w_{\rm{zf}}$). Additionally, we also apply weights to account for the
systematic relation between the number density of observed galaxies and
stellar density ($w_{\rm{star}}$) and seeing ($w_{\rm{see}}$). Finally, each
galaxy is assigned the combined weight defined as
$w_{\rm{tot}}=(w_{\rm{cp}}+w_{\rm{zf}}-1)w_{\rm{star}}w_{\rm{see}}$. By
combining the survey completeness ($w_{\rm{sc}}$) at the angular position of a
galaxy (Fig. 3) and the weights assigned to it, we obtain the contribution of
each galaxy in the galaxy count as $w_{\rm{tot}}/w_{\rm{sc}}$.

####  3.1.2 Random catalogs

Before testing the homogeneity of matter distribution using the CMASS
galaxies, first we need to establish a clear criterion for determining
homogeneity. Although homogeneity implies that the physical state is the same
at all locations, in principle it is impossible for a finite number of points
within a space to be homogeneously distributed. This is because fictitious
clustering and density inhomogeneity are always visible when a finite number
of data points are generated even with uniform probability. Therefore, here we
establish the criterion for homogeneity from a statistical point of view. The
simplest way to do this is to use the distribution of random points following
the Poisson distribution as the criterion for homogeneity. The finite number
of points randomly distributed by the Poisson distribution shows fluctuations
in number density from place to place. On a small scale, the actual
distribution of galaxies tends to show a larger fluctuation due to clustering
of galaxies than expected in the Poisson distribution. Since these
fluctuations are expected to decrease with increasing scale, we can determine
the HS from which the distribution of galaxies begins to match the random
distribution within a statistically acceptable range.

In this work, we have made one thousand random catalogs by considering the
observational characteristics of the CMASS sample, each containing data points
equal to the number of CMASS galaxies. The random data points are generated
within the survey region using the radial number distribution and sector
completeness as the probability density functions. However, the random data
points were assigned uniform weight ($w_{\rm{tot}}=1$), unlike the galaxies
with various weights. Thus, the contribution of each point to the count is
$1/w_{\rm{sc}}$ for the random catalog. We analyzed these random catalogs in
the same way as the CMASS sample is analyzed.

####  3.1.3 HR3 mock catalogs

To test the homogeneity of the CMASS galaxy distribution, in addition to the
random distribution, we also analyzed the mock galaxy distribution expected in
the standard cosmological model, and compared the results with that of the
actual galaxy distribution. Here we have produced the CMASS mock catalogs from
the two model-derived data sets.

Figure 4: Radial number distributions of Horizon Run 3 (HR3) mock galaxies
($N=918,135$) from the first of 27 lightcones (only ASF is applied), the CMASS
galaxies ($N=568,776$), and the CMASS galaxies brighter than the absolute
magnitude of $-20.2$ in $r$-band ($N=395,011$).

The first one we use is the 27 all-sky mock BOSS survey data provided by Korea
Institute of Advanced Study222Available at
http://sdss.kias.re.kr/astro/Horizon-Runs/Horizon-Run23.php. The data were
derived from the Horizon Run 3 (HR3) simulation, one of the largest $N$-body
simulation in volume to date with $7,210^{3}$ dark matter particles within the
volume of $(10.815\leavevmode\nobreak\ h^{-1}{\rm{Gpc}})^{3}$ (Kim et al.
2011). The halos were picked up as mock galaxies at redshift range
$0.43<z<0.7$ considering the redshift-dependent mass limit determined from the
expected galaxy number density of the BOSS survey (see $\S 4.2$ of Kim et al.
2011). Figure 4 compares the radial number distributions of HR3 mock galaxies
(extracted from one of the mock BOSS survey data) and the CMASS galaxies. For
HR3 data, the mock galaxies corresponding to the percentage of CMASS sector
completeness have been randomly selected within the survey region. Noting that
the radial number distribution of the selected HR3 mock galaxies is less than
that of the CMASS sample at redshift range of $0.47<z<0.55$, we have excluded
the CMASS galaxies that are fainter than absolute magnitude of $-20.2$ in
$r$-band and made the galaxy number density around $z=0.5$ smaller than the
HR3 data so that mock catalogs for the CMASS sample can be safely obtained.
Within the CMASS survey region, considering the CMASS radial number
distribution and angular selection function, we extracted massive halos with
the same number of the magnitude-limited CMASS galaxies in decreasing order of
halo mass. For each mock SDSS-III survey data that covers the whole sky, it is
possible to extract the four independent mock catalogs for the CMASS NGC
region. Thus, we have made a total of $108$ independent mock catalogs. The
limited number of mock catalogs makes it possible to judge statistical
significance only up to $2\sigma$. Each HR3 mock catalog contains $395,011$
data points. We also have made separate sets of random catalogs that match the
new radial number distribution of the magnitude-limited CMASS sample (see Fig.
4).

Our method of generating mock catalogs has a limitation in that it cannot
specifically mimic the process in which CMASS galaxies are selected by various
color-cuts. However, our method is valid in that it mimicks massive galaxies
of the CMASS sample by preferentially choosing heavier halos, taking into
account the known relation that a more massive halo contains a brighter galaxy
(Kim et al., 2008).

####  3.1.4 Log-normal mock catalogs

The second data is the mock galaxies extracted from the log-normal
simulations, which is an inexpensive way of generating realizations of
nonlinear density fluctuations. We use a publicly available code for
generating log-normal realizations of galaxies in redshift-
space333https://bitbucket.org/komatsu5147/lognormal_galaxies/src/master/
(Agrawal et al., 2017). The code generates log-normal matter density field
from matter power spectrum of the $\Lambda$CDM model, then draws Poisson-
distributed mock galaxies with a number expected by the log-normal density
field at a given position in space. It has been confirmed whether the
generated mock galaxies give a two-point statistics similar to that of a
standard $\Lambda$CDM model (Agrawal et al., 2017).

Using the flat $\Lambda$CDM model favored by the Planck 2018
TT,TE,EE+lowE+lensing data (Planck Collaboration et al. 2020), we obtain the
log-normal realizations of galaxies at redshift $z_{\textrm{cen}}=0.52$. Since
galaxies in the log-normal realizations do not have mass information, there is
a disadvantage that galaxies cannot be selected in order of mass as in the HR3
mock catalog. To obtain the statistics similar to CMASS galaxies, we assume
the bias of galaxies relative to dark matter to be $b=2.2$, which is a value
adopted by BOSS collaboration for generating mock catalogs (Alam et al.,
2017). This value does not statistically deviate from the measurement of other
studies (Guo et al., 2013; Nuza et al., 2013; Zhai et al., 2017). Finally, we
generated one thousand mock catalogs by randomly picking up mock galaxies in
the log-normal realizations considering the radial number distribution and
sector completeness of the CMASS sample so that the generated log-normal
galaxies follow the magnitude-limited CMASS galaxies. We also produced 1,000
log-normal catalogs even assuming a lower bias of $b=1.8$ as suggested in
Rodríguez-Torres et al. (2016). It is shown that the log-normal catalogs with
$b=1.8$ do not provide statistics similar to the CMASS data (see Fig. 6
below).

As in the random catalogs, we assigned the uniform weights ($w_{\rm{tot}}=1$)
for all points in both the HR3 mock catalogs and the log-normal mock catalogs.

###  3.2 Method: Counting galaxies with truncated cones

Figure 5: A truncated cone circumscribed about a sphere of radius $R$. The
apparent angular radius of the truncated cone (or the sphere) as seen from the
observer at O is $\theta_{R}$. Redshifts $z_{1}$ and $z_{2}$, the lower and
upper limits of the truncated cone, respectively, vary as the radius of the
sphere at the central redshift $z_{\rm{cen}}$ varies. The thin slice at
$z_{\rm{cen}}$ has thickness of $\Delta r=7h^{-1}\textrm{Mpc}$. We count
galaxies inside the truncated cone that is centered on every galaxy within the
slice.

To quantitatively test whether the three-dimensional spatial distribution of
galaxies is homogeneous, we have applied the galaxy counting method to the
CMASS sample, random catalogs, and two mock catalogs. Hogg et al. (2005) and
Ntelis et al. (2017) used the count-in-sphere method to judge whether the
distribution of galaxies is homogeneous by comparing the number of galaxies
within a sphere of specific radius with that of the random distributions. On
the other hand, Park et al. (2017) tested the homogeneity of the LRG
distribution by counting galaxies within a truncated cone instead of the
sphere. Unlike the count-in-sphere method that needs the three-dimensional
spatial distribution of galaxies, the count-in-truncated-cone method with
fixed redshift center is less dependent on specific cosmological models in
that galaxies are counted in the given redshift space, not in the real space.
In addition, counting galaxies within a truncated cone enables us to define a
criterion for homogeneity without the need to consider different statistical
weights given to galaxies that are unevenly distributed along the radial
direction. This is because the accumulated statistical weight of the line-of-
sight direction is the same no matter which direction we see within the
truncated cone.

Figure 6: The mean ($\left<\xi\right>$) and standard deviation
($\sigma_{\xi}$) of $\xi$ as a function of radius ($R$). Top panels (a,b)
shows the results of unmodified CMASS sample (blue dots) and the corresponding
$1,000$ random catalogs (cyan dots), together with those of magnitude-limited
CMASS sample (blue pluses) and the corresponding $1,000$ random catalogs
(brown dots). The results of 108 Horizon Run 3 (HR3) mock catalogs (pink dots)
are shown in the middle panels (c,d), together with those of the magnitude-
limited CMASS sample and the corresponding random catalogs. The bottom panels
(e,f) presents the results of the log-normal mock catalogs generated with a
bias factor $b=2.2$ (green dots). For a comparison, the results for $b=1.8$
are shown as cross-marks ($\times$’s). The shaded regions in each panel, in
order from dark to light, represent 1, 2, and 3 sigma confidence regions,
which have been estimated from measured means and standard deviations of
individual random or mock catalogs. The information of the shaded regions
between the specified radius scales are interpolated.

In this work, we use the count-in-truncated-cone method suggested by Park et
al. (2017). We consider a $7\leavevmode\nobreak\ h^{-1}\textrm{Mpc}$-thick
slice for a central redshift $z_{\rm{cen}}=0.52$, where CMASS galaxies are the
most dense. There are $9,770$ CMASS galaxies in this slice. Thickness of the
slice was chosen to control the number of galaxies (to be located at the
center of the truncated cone) to an appropriate level of about $10,000$. This
number is suitable for closely looking at the direction-dependency of galaxy
counts within the survey region and for performing calculations within a
reasonable time. For each galaxy, we consider a sphere of comoving radius $R$
that is centered at the slice but at the galaxy’s angular position, and
construct a truncated cone that is circumscribed about the sphere (Fig. 5).
The apparent angular radius of the truncated cone seen from the observer is
given by $\theta_{R}=\sin^{-1}(R/r_{\rm{cen}})$, where $r_{\rm{cen}}$ is the
comoving distance to the center of the slice. The redshift $z_{1}$ ($z_{2}$)
corresponding to the near (far) side of a truncated cone is transformed from
the comoving distance $r_{\rm{cen}}-R$ ($r_{\rm{cen}}+R$) in the fiducial
$\Lambda$CDM model. For radii from $250$ to $300\leavevmode\nobreak\
h^{-1}\textrm{Mpc}$, however, the $r_{\rm{cen}}-R$ becomes less than the
distance to the minimum redshift ($z=0.43$). Thus, in these cases, $z_{1}$ is
set to $z=0.43$ and $z_{2}$ is set to the higher value so that the interval
$z_{2}-z_{1}$ can span a range of $2R$ and include more galaxies within the
range (see Fig. 2).

Next, we calculate a statistical measure of homogeneity $\xi$, defined as the
number density within the truncated cone divided by the number density within
the slice covering the whole survey area from redshift $z_{1}$ to $z_{2}$,

$\xi=\frac{N_{\rm{trc}}/V_{\rm{trc}}}{N_{\rm{slice}}/V_{\rm{slice}}}$ (1)

where $N_{\rm{trc}}$ ($N_{\rm{slice}}$) denotes the number of galaxies within
the truncated cone (slice) and $V_{\rm{trc}}$ ($V_{\rm{slice}}$) the volume of
the truncated cone (slice). For a homogeneous distribution, the statistic
$\xi$ is expected to converge to 1 as the radius increases. However, when the
radius is large enough to cover the entire survey region, this statistic has a
characteristic of reaching 1 in principle even when the distribution
considered is not homogeneous.

Due to the survey boundaries, masks, and holes, a truncated cone is not
complete. The partial volume of the truncated cone is calculated by summing
the volume elements corresponding to each HEALpix pixel in the pixelized area
occupied by the truncated cone on the sky (see Appendix of Park et al. 2017
for a detailed description). For varying radius from $R=30\leavevmode\nobreak\
h^{-1}\textrm{Mpc}$ to $300\leavevmode\nobreak\ h^{-1}\textrm{Mpc}$ with steps
of $10\leavevmode\nobreak\ h^{-1}\textrm{Mpc}$ (totally 28 steps), we
calculate $\xi$ and the volume of the partial truncated cone for every galaxy
within the slice with $z=z_{\textrm{cen}}$ and $7\leavevmode\nobreak\
h^{-1}\textrm{Mpc}$ width. The maximum radius of $300\leavevmode\nobreak\
h^{-1}\textrm{Mpc}$ is limited by the depth of the CMASS sample, about
$600\leavevmode\nobreak\ h^{-1}\textrm{Mpc}$.

Given individual measurements of $\xi$ for every galaxies within the slice, we
calculate the mean ($\left<\xi\right>$), standard deviation ($\sigma_{\xi}$),
skewness ($skew_{\xi}$), and kurtosis ($kurt_{\xi}$) of the measured $\xi$’s.
During calculating these statistics, we assign each measured $\xi$ a
statistical weight by the volume of the partial truncated cone. Thus, the
$\xi$’s measured near the survey boundary usually have lower statistical
weights than those near the center of the survey region. For each radius
scale, these statistics are compared with those estimated from the random and
mock catalogs. The significant deviation of the measured value from the
expectation of the random or mock distributions may imply the deviation from
the homogeneity or the present cosmological model.

Figure 7: The mean and standard deviation diagrams of the unmodified CMASS
sample (blue markers and solid lines) and the corresponding $1,000$ random
catalogs (cyan dots and histograms) for four different radii at the central
redshift $z_{\rm{cen}}=0.52$. Results of the magnitude-limited CMASS sample
(faint blue markers and dotted lines) and the corresponding $108$ random
catalogs (brown markers) are also shown for comparison. The deviations of the
CMASS sample from the $1,000$ random data are indicated in unit of standard
deviation ($\sigma$). Figure 8: The skewness ($skew_{\xi}$; top) and kurtosis
($kurt_{\xi}$; bottom) of $\xi$ as a function of radius ($R$). The result of
the magnitude-limited CMASS sample (blue dots) is compared with the average
and 1–3$\sigma$ confidence regions of the $108$ random catalogs. Figure 9: The
same as Fig. 7 but for the magnitude-limited CMASS sample, $108$ HR3 mock
catalogs (pink dots and histograms), the corresponding $108$ random catalogs
(brown dots and histograms).

###  3.3 Results

####  3.3.1 Comparison with random data

As the homogeneous distribution, we take the random Poisson distribution.
Comparisons are made in Figs. 6a-b, which show the mean ($\langle\xi\rangle$)
and the standard deviation (${\sigma}_{\xi}$) of $\xi$’s calculated from the
unmodified CMASS sample and the corresponding $1,000$ random catalogs. We can
see that $\langle\xi\rangle$’s of both CMASS sample and random data approach 1
as the radius increases. However, unlike the random distributions, the
$\left<\xi\right>$ of the CMASS sample does not fully converge to 1 but shows
a large deviation, which maintains even at the largest radius. The
$\sigma_{\xi}$ also significantly deviates from the expected range of the
random distributions. For $\langle\xi\rangle$, the deviations are $61\sigma$
at $70\leavevmode\nobreak\ h^{-1}\textrm{Mpc}$ and $5.5\sigma$ at
$300\leavevmode\nobreak\ h^{-1}\textrm{Mpc}$; and for ${\sigma}_{\xi}$, the
deviations are $93\sigma$ at $70\leavevmode\nobreak\ h^{-1}\textrm{Mpc}$ and
$34\sigma$ at $300\leavevmode\nobreak\ h^{-1}\textrm{Mpc}$, see Fig. 7. The
deviations in the mean and standard deviation show that CMASS galaxies are far
from the random distributions even at the largest scale. On the other hand,
the skewness and kurtosis of $\xi$-statistics presented in Fig. 8 show no
significant anomaly of the CMASS sample compared with the random data. Also
shown are the trends in the magnitude-limited CMASS sample and random data,
where deviations from the random data are far more significant than the
original ones.

####  3.3.2 Comparison with two mock catalogs

Figures 6c-d and 6e-f compare $\left<\xi\right>$ and $\sigma_{\xi}$ variations
of the CMASS sample with those of HR3 and log-normal mock catalogs,
respectively. Here, the magnitude-limited CMASS sample and the corresponding
random catalogs have been used. For both $\langle\xi\rangle$ and
$\sigma_{\xi}$, the CMASS sample relatively agrees well with both mock samples
(see Fig. 9 for HR3 mock samples). We note that the log-normal galaxies
generated with a lower value of bias factor ($b=1.8$) have lower amplitudes
for the mean and standard deviation of the $\xi$ statistic. Thus, the CMASS
galaxies are distributed as expected from theoretically estimated
distribution.

For $\left<\xi\right>$, although the values of mock catalogs tend to approach
the result of the random distribution as the scale increases, the
$\left<\xi\right>$ of the CMASS sample tends to stay constant starting from
$230\leavevmode\nobreak\ h^{-1}\textrm{Mpc}$. This anomaly can be explained by
the presence of peculiar overdense regions in the CMASS sample. If there are
noticeable overdense regions with more clustering within the survey volume,
naturally $\langle\xi\rangle$ is expected to be greater than 1. Although not a
major effect, it should also be taken into account that galaxies located near
the CMASS boundary contribute less to $\left<\xi\right>$ because in this case
only the partial volume of the truncated cone is used as weight. The constant
behavior of $\left<\xi\right>$ at large scales with a significant deviation
from unity can be also found in the mock catalogs. Figure 10 shows the
behavior of $\left<\xi\right>$ as a function of radius scale for the cases of
the maximum and minimum $\left<\xi\right>$ at $R=300\leavevmode\nobreak\
h^{-1}\textrm{Mpc}$ among the 1000 log-normal mock catalogs. Figure 11
compares the distributions of $\xi$ on the CMASS survey region for the
magnitude-limited CMASS sample, a (magnitude-limited) random catalog, and two
log-normal mock catalogs that show the maximum or minimum values of
$\left<\xi\right>$ at $300\leavevmode\nobreak\ h^{-1}\textrm{Mpc}$ scale (see
section 3.4 for a detailed description of how to produce these maps). In the
$\xi$ distribution of the CMASS sample, it is seen that the more overdense
area exists in the central part of the survey region. In the case of the mock
catalog showing the maximum $\left<\xi\right>$ value, the overall distribution
is similar. On the other hand, for the mock catalog with the minimum
$\left<\xi\right>$, it appears that there are relatively few overdense areas
in the inner survey region.

Figure 10: Variation of $\left<\xi\right>$ as a function of radius $R$ for
log-normal mock catalogs (green dots with shaded regions) and the magnitude-
limited CMASS sample (blue dots), which is the same as Fig. 6e. Here the cases
of the maximum (black pluses) and the minimum (red cross-marks)
$\left<\xi\right>$ at $300\leavevmode\nobreak\ h^{-1}\textrm{Mpc}$ among the
1000 log-normal mock catalogs have been added. Figure 11: Maps showing
$\xi$-distributions on the CMASS survey region, estimated from the magnitude-
limited CMASS sample (a), a (magnitude-limited) random catalog (b), and two
log-normal catalogs with the maximum (c) and minimum (d) values of
$\left<\xi\right>$ at $R=300\leavevmode\nobreak\ h^{-1}\textrm{Mpc}$ scale.

In $\sigma_{\xi}$, both kinds of mock samples deviate significantly from the
random data, and the deviation becomes even larger for the CMASS galaxies at
the largest scale (see Figs. 6d, 6f, and 7). As shown in Fig. 9, the
deviations of the magnitude-limited CMASS galaxies from the random
expectations are $19\sigma$ for $\left<\xi\right>$ and $40\sigma$ for
$\sigma_{\xi}$ at $R=300\leavevmode\nobreak\ h^{-1}\textrm{Mpc}$. We note that
the CMASS galaxies show similar $\left<\xi\right>$ and $\sigma_{\xi}$ to mock
galaxies on small scales, but the difference between the two becomes more
significant as the scale increases. At the largest scale, the CMASS sample
tends to deviate by $3\sigma$ or more from the mock samples. Further
investigation is needed to explain this discrepancy.

####  3.3.3 Comparison with theoretical power spectrum

In the previous subsection, we have compared the distribution of the CMASS
galaxies with those in two theoretical mock catalogs. The distribution of
galaxies in the mock catalogs contains information of the nonlinear mass
fluctuations. Here we investigate whether the linear matter density
fluctuations, which are the result of the primordial fluctuations grown by the
linear perturbation theory, can explain the CMASS result without taking into
account the effects of nonlinear gravitational evolution. For this, we compare
our estimation of $\sigma_{\xi}$ of the CMASS sample with the theoretically
estimated amplitude of mass fluctuations. The variance of mass fluctuations
expected within a sphere of radius $R$ is given by (Peebles, 1993)

$\sigma^{2}(R)\equiv\langle{\delta({\bf
x},R)^{2}}\rangle={{1}\over{{2\pi}^{2}}}\int P(k)|{W_{R}({\bf
k}})|^{2}k^{2}dk,$ (2)

where $P(k)$ denotes the matter power spectrum at wavenumber $k$, and
$W_{R}({\bf k})$ is the Fourier transformation of a spherical top-hat window
function that assigns $3/(4\pi R^{3})$ inside the sphere and $0$ otherwise and
is written as

$W_{R}(\mathbf{k})=\frac{3}{(kR)^{3}}\left[\sin(kR)-kR\cos(kR)\right].$ (3)

Figure 12: Comparison of $\sigma_{\xi}$ of the CMASS sample, and the HR3 and
log-normal mock catalogs (same as Figs. 6d and 6f) together with the mass
fluctuations $\sigma(R)$ of linear density fields expected in the standard
$\Lambda$CDM model (Planck Collaboration et al., 2020). Figure 13: Left panel:
The CMASS galaxies within a thin slice at the central redshift $z_{\rm{cen}}$.
Circles of different sizes of radius ($R=50$, $70$, $100$, $150$, $200$,
$250$, $300h^{-1}\textrm{Mpc}$) are shown together. The redshift ranges are
presented in Fig. 2. Middle and right panels: Maps of $\xi$ estimated from the
unmodified CMASS sample (left) and a random catalog (right panel).

Figure 12 compares the theoretical prediction of matter density fluctuations
$\sigma(R)$ at various scales with the measured deviation of galaxy number
density $\sigma_{\xi}$ of the CMASS sample and two mock catalogs. In our
calculation, we use the CAMB code (Lewis & Challinor, 2011) to obtain the
linear matter power spectrum at $z=0.52$ in a flat $\Lambda$CDM cosmological
model that best fits the Planck 2018 TT,TE,EE+lowE data (Planck Collaboration
et al., 2020). The $\sigma(R)$ shown in Fig. 12 is obtained by inserting the
linear power spectrum into Eq. (2) and multiplying with the bias factor
$b=2.2$, which is consistent with the values estimated in the previous studies
(Guo et al., 2013; Nuza et al., 2013; Zhai et al., 2017). The $\sigma(R)$
obtained in this way is in good agreement with $\sigma_{\xi}(R)$ of both HR3
and log-normal mock catalogs at all scales above $50\leavevmode\nobreak\
h^{-1}\textrm{Mpc}$.

###  3.4 Cosmic landscape

The count-in-truncated cone method used here to test the homogeneity of the
galaxy distribution is also useful for visualizing the results and making
intuitive comparisons. Figure 13 visualizes the clustered nature of the CMASS
galaxies compared to a random realization. Whereas our statistical studies
were based on galaxy-centered counting, the results shown here are based on
the pixel-centered counting. We divided the survey area into about $132,000$
pixels using the HEALpix program with a resolution parameter as
$\textrm{Nside}=256$ (Górski et al., 2005). Then, we placed a truncated cone
in the center of every pixel and calculated $\xi$. Figure 13 is the
$\xi$-distribution maps and directly shows how the averaged-over-the-radius
density changes with position. To compare with the CMASS sample, just one
random catalog is selected out of the $1,000$ catalogs.

Figure 13 shows that the CMASS galaxies are far more clustered than the random
distribution in all scales, and Table 1 compares the density extremum (maximum
and minimum) between the CMASS sample and the random data. For the often
claimed HS, $70\leavevmode\nobreak\ h^{-1}\textrm{Mpc}$, the density maximum
of the CMASS sample is $79\%$ higher than the mean, while for the random one,
it is only $18\%$; the ratio between the maximum and minimum is $3.30$ for the
CMASS sample, while it is only $1.39$ for the random data. It is not easy to
call a distribution with such large fluctuations homogeneous. Even at
$300\leavevmode\nobreak\ h^{-1}\textrm{Mpc}$, it is difficult to say that the
distribution of galaxies is homogeneous: $\xi$’s at maximum for the CMASS and
random data are $10\%$ and $2\%$ higher than mean, respectively, and the ratio
between the maximum and minimum is $1.20$ versus $1.05$.

| $50\leavevmode\nobreak\ h^{-1}\textrm{Mpc}$ | $70\leavevmode\nobreak\ h^{-1}\textrm{Mpc}$ | $100\leavevmode\nobreak\ h^{-1}\textrm{Mpc}$ | $150\leavevmode\nobreak\ h^{-1}\textrm{Mpc}$ | $200\leavevmode\nobreak\ h^{-1}\textrm{Mpc}$ | $250\leavevmode\nobreak\ h^{-1}\textrm{Mpc}$ | $300\leavevmode\nobreak\ h^{-1}\textrm{Mpc}$
---|---|---|---|---|---|---|---
CMASS galaxies |  33.7 $\sim$ 231% |  54.2 $\sim$ 179% |  68.3 $\sim$ 147% |  78.5 $\sim$ 120% |  85.6 $\sim$ 114% |  90.4 $\sim$ 112% |  91.8 $\sim$ 110%
Random distribution |  71.4 $\sim$ 133% |  84.6 $\sim$ 118% |  89.9 $\sim$ 109% |  95.4 $\sim$ 105% |  95.8 $\sim$ 104% |  97.2 $\sim$ 103% |  97.5 $\sim$ 102%
Table 1: Minimum and maximum $\xi$ values for different radius scales in the
$\xi$-maps shown in Fig. 13. The values represent deviations from the mean
(100%) of the maps.

###  3.5 Conclusion

In this section, we studied the inhomogeneous nature of galaxy distribution
using the BOSS CMASS DR12 sample, which includes $932,517$ galaxies within the
redshift range between $0.43$ and $0.7$ (Alam et al., 2015). We use the number
count within truncated cones centered at every galaxy located at a narrow
region around redshift $0.52$. As the criterion of homogeneity, we
statistically compared with the $1,000$ realization of random Poisson
distributions. Statistical comparisons are also made with two mock catalogs
derived from theory: (i) $108$ mock galaxy catalogs from HR3 $N$-body
numerical simulation (Kim et al., 2011) and (ii) the mock catalogs based on
the log-normal distribution of matter density (Agrawal et al., 2017). We also
compared $\sigma(R)$ estimated from the linear matter power spectrum.

The result shows that the difference between the CMASS sample and the random
data reduces as the scale becomes larger, see Figs. 6a-b, and 8. However, even
in the largest statistically meaningful scale, the radius around
$300\leavevmode\nobreak\ h^{-1}\textrm{Mpc}$, the difference is still
significant. Even at the largest scale, the deviation of the CMASS sample from
the Poisson distribution is around $34\sigma$ to $40\sigma$ in standard
deviation ($\sigma_{\xi}$) and around $5\sigma$ to $19\sigma$ in the mean
($\left<\xi\right>$), see Figs. 7 and 9. Thus, even on this largest available
scale, the CMASS data is impossible to appear in random realizations. The
cosmic landscape shown in Fig. 13 visually confirms our conclusion. The CMASS
galaxies are far more clustered than the randomly distributed points in all
currently available scales.

However, our comparison of the CMASS sample with the two mock catalogs shows
consistency between the observation (CMASS sample) and theory (two mock
catalogs), see Figs. 6c-f, 9, and 12. A statistical anomaly in
$\left<\xi\right>$ identified at a large scale is explained by the presence of
overdense regions located in the CMASS survey area. We confirmed the same
phenomenon occurs in the log-normal mock catalogs with similar overdensities,
see Section 3.3 and Figs. 10 and 11. A statistical anomaly in $\sigma_{\xi}$
compared with the two mock data identified at a large scale in Figs. 6d, 6f,
and 9 needs further study.

Therefore, we conclude that while the observation is consistent with the
current paradigm of modern cosmology, the CP is not in the currently available
galaxy distribution sample and nor is it demanded by theory. A clear
resolution of this seemingly contradictory conclusion is made in Sections 2
and 4.

##  4 Discussion

Einstein’s gravity directly connects the matter with the spacetime curvature.
However, the CP is implemented in the metric. As the curvature is a quadratic
derivative of the metric, the near-homogeneity in the metric does not demand
similar near-homogeneity in the matter distribution. Such a difference is
indeed the case in modern cosmology, especially on the far sub-horizon scale.
Our study in Section 3 confirms that matter distribution is far more
inhomogeneous compared with the random Poisson distribution, which is our
criterion for the homogeneity. At the same time, we show that the matter
distribution is consistent with theoretical predictions based on simulation
and power spectrum. Thus, the CP is not in the observed galaxy distribution,
nor is it expected in theory.

We conclude that it is inadequate to search for the HS or the CP in the
observed light or the matter distribution. As we show in Section 2 the CP is
perfectly fine and well in the metric, a theoretical construct. However, this
conclusion is valid under a subtle condition. Based on the Robertson-Walker
metric as the background, we have shown that the deviations in metric are
extremely small in the observed cosmic scale. Thus, our test implies that the
CP is valid in modern cosmology if we assume the CP in the background. That
is, our’s is only a consistency test, not proof. Whether we can accept such a
background geometry is an entirely different question that may demand another
strategy. Until we resolve this aspect, the CP remains an assumption.

A proper answer to this question may require studying alternative models based
on other than Einstein’s CP. The CMB strongly supports the isotropy around us.
Thus, one interesting possibility is constraining the spherically symmetric
model based on Lemaître’s CP. After all, due to the nature of our observation,
systematically observing past events as we observe further away, even under
Einstein’s CP properly unfolded in the Universe, the observations are destined
to be spherically symmetric with us at the center. The temporal evolution of
cosmic structures and the observational difficulties increasing with distance
will make the observation look spherically symmetric in any case. Constraining
the spherically symmetric world models with us near the center is an important
issue in this regard.

One consequence of our clarification, separating matter from the metric, is
freeing our concept of the universe from dull homogeneity in its matter
content on the large-scale because the CP in modern cosmology mainly concerns
the metric only. Through the light, one may discover more diverse phenomena
that may have distinct characteristics and may await new classification.
Perhaps, if we may, “there are more things in the sky than are dreamt of in
our theory.”

Although we have not found the HS in the currently available galaxy survey,
this does not preclude its future discovery in a more in-depth survey. In
modern cosmology, with $\delta\Phi/c^{2}\sim(\ell/\ell_{H})^{2}\delta$, as the
scale approaches the horizon scale, we should have
$\delta\sim\delta\Phi/c^{2}$, thus the matter fluctuation ($\delta$) becoming
extremely small as the metric fluctuation. In this way, the Friedmann
background world model should be recovered with the CP valid in both metric
and matter, in near-horizon scale structures. No such confirmation has been
made yet. We anticipate practical difficulty as the scale approaches the
horizon scale; the observed space (light-cone) deviates significantly from the
homogeneous space the CP is imposed. Still, although not relevant to the CP’s
validity, the homogeneity test for matter should be continued using more
extensive galaxy survey data undoubtedly available soon in the future.

Now, with the extreme level homogeneity in the cosmic metric known, an
important theoretical issue arises concerning its origin. Inflation provides
an isotropization mechanism for homogeneous but anisotropic cosmologies (Wald,
1983), but how about homogeneity? Why do we live in a universe with such
extreme homogeneity in the metric? This is a mystery yet to be resolved. The
anthropic principle is not an answer we anticipate as it merely claims the
case as it happens to be.

## Acknowledgements

We wish to thank an anonymous referee for valuable comments and suggestions
which help improve the manuscript. We thank Professors Donghui Jeong and Juhan
Kim for useful discussion and suggestions. C.-G.P. was supported by the
National Research Foundation of Korea (NRF) grant funded by the Korea
government (MSIT) (No. 2020R1F1A1069250). H.N. was supported by the National
Research Foundation of Korea funded by the Korean Government (No.
2018R1A2B6002466 and No. 2021R1F1A1045515). J.H. was supported by IBS under
the project code IBSR018-D1 and by Basic Science Research Program through the
National Research Foundation (NRF) of Korea funded by the Ministry of Science,
ICT and Future Planning (No. 2018R1A6A1A06024970 and NRF-2019R1A2C1003031).

## References

  * Abazajian et al. (2009) Abazajian K. N. et al. 2009, ApJS, 182, 543
  * Agrawal et al. (2017) Agrawal A., Makiya R., Chiang C.-T., Jeong D., Saito S., & Komatsu E. 2017, J. Cosmology Astropart. Phys., 2017, 003
  * Alam et al. (2015) Alam S. et al. 2015, ApJS, 219, 12
  * Alam et al. (2017) Alam S. et al. 2017, MNRAS, 470, 2617
  * Anderson et al. (2012) Anderson L. et al. 2012, MNRAS, 427, 3435
  * Angulo et al. (2012) Angulo R. E., Springel V., White S. D. M., Jenkins A., Baugh C. M., & Frenk C. S. 2012, MNRAS, 426, 2046
  * Bagla et al. (2008) Bagla J. S., Yadav J., & Seshadri T. R. 2008, MNRAS, 390, 829
  * Dawson et al. (2013) Dawson K. S. et al. 2013, AJ, 145, 10
  * Einstein (1917a) Einstein A. 1917a, Sitzungsberichte der Königlich Preußischen Akademie der Wissenschaften (Berlin), 142
  * Einstein (1917b) Einstein A. 1917b, in The Collected Papers of Albert Einstein, Volume 8: The Berlin Years: Correspondence, 1914-1918. ed. R. Schulmann, A. J. Kox, M. Janssen, et al. (Princeton University Press, 1998)
  * Eisenstein et al. (2001) Eisenstein D. J. et al. 2001, AJ, 122, 2267
  * Eisenstein et al. (2011) Eisenstein D. J. et al. 2011, AJ, 142, 72
  * Ellis et al. (2012) Ellis G. F. R., Maartens R., & MacCallum M. A. H. 2012, Relativistic Cosmology (Cambridge University Press)
  * Friedman (1922) Friedman A. 1922, Z. Phys., 10, 377
  * Friedmann (1924) Friedmann A. 1924, Z. Phys., 21, 326
  * Górski et al. (2005) Górski K. M., Hivon E., Banday A. J., Wandelt B. D., Hansen F. K., Reinecke M., & Bartelmann M. 2005, ApJ, 622, 759
  * Gott et al. (2005) Gott J. R., Jurić M., Schlegel D., Hoyle F., Vogeley M., Tegmark M., Bahcall N., & Brinkmann J. 2005, ApJ, 624, 463
  * Guo et al. (2013) Guo H. et al. 2013, ApJ, 767, 122
  * Habib et al. (2016) Habib S. et al. 2016, New A, 42, 49
  * Hamilton & Tegmark (2004) Hamilton A. J. S., & Tegmark M. 2004, MNRAS, 349, 115
  * Heitmann et al. (2015) Heitmann K. et al. 2015, ApJS, 219, 34
  * Hogg et al. (2005) Hogg D. W., Eisenstein D. J., Blanton M. R., Bahcall N. A., Brinkmann J., Gunn J. E., & Schneider D. P. 2005, ApJ, 624, 54
  * Hwang & Noh (2006) Hwang J.-c., Noh H. 2006, MNRAS, 367, 1515
  * Hwang & Noh (2013) Hwang J.-c., Noh H. 2013, MNRAS, 433, 3472
  * Hwang et al. (2008) Hwang J.-c., Noh H., & Puetzfeld D. 2008, J. Cosmology Astropart. Phys., 2008, 010
  * Jeong et al. (2011) Jeong D., Gong J.-O., Noh H., & Hwang J.-c. 2011, ApJ, 727, 22
  * Kazin et al. (2010) Kazin E. A., et al. 2010, ApJ, 710, 1444
  * Kim (2021) Kim J. 2021, Private communication
  * Kim et al. (2008) Kim, J., Park, C., & Choi, Y.-Y. 2008, ApJ, 683, 123
  * Kim et al. (2009) Kim J., Park C., Gott J. Richard I., & Dubinski J. 2009, ApJ, 701, 1547
  * Kim et al. (2011) Kim J., Park C., Rossi G., Lee S. M., & Gott J. R. 2011, Journal of Korean Astronomical Society, 44, 217
  * Kim et al. (2015) Kim J., Park C., L’Huillier B., & Hong S. E. 2015, Journal of Korean Astronomical Society, 48, 213
  * Krasinski (1997) Krasinski A. 1997, Inhomogeneous Cosmological Models (Cambridge University Press)
  * Laurent et al. (2016) Laurent P. et al. 2016, J. Cosmology Astropart. Phys., 2016, 060
  * Lemaître (1933) Lemaître G. 1933, Annales de la Société Scientifique de Bruxelles, 53, 51
  * Lewis & Challinor (2011) Lewis A., & Challinor A. 2011, ascl:1102.026
  * Lifshitz (1946) Lifshitz E. M. 1946, J. Phys. (USSR), 10, 116
  * McCrea & Milne (1934) McCrea W. H., & Milne E. A. 1934, The Quarterly Journal of Mathematics, os-5, 73
  * Milne (1934) Milne E. A. 1934, The Quarterly Journal of Mathematics, os-5, 64
  * Ntelis et al. (2017) Ntelis P. et al. 2017, J. Cosmology Astropart. Phys., 2017, 019
  * Nuza et al. (2013) Nuza S. E. et al. 2013, MNRAS, 432, 743
  * Park et al. (2012) Park C., Choi Y.-Y., Kim J., Gott J. Richard I., Kim S. S., & Kim K.-S. 2012, ApJ, 759, L7
  * Park et al. (2017) Park C.-G., Hyun H., Noh H., & Hwang J.-c. 2017, MNRAS, 469, 1924
  * Peebles (1993) Peebles P. J. E. 1993, Principles of Physical Cosmology (Princeton University Press)
  * Planck Collaboration et al. (2014) Planck Collaboration et al. 2014, A&A, 571, A1
  * Planck Collaboration et al. (2020) Planck Collaboration et al. 2020, A&A, 641, A6
  * Potter et al. (2017) Potter D., Stadel J., & Teyssier R. 2017, Computational Astrophysics and Cosmology, 4, 2
  * Prada et al. (2012) Prada F., Klypin A. A., Cuesta A. J., Betancort-Rijo J. E., & Primack J. 2012, MNRAS, 423, 3018
  * Rodríguez-Torres et al. (2016) Rodríguez-Torres S. A. et al. 2016, MNRAS, 460, 1173
  * Sarkar et al. (2009) Sarkar P., Yadav J., Pandey B., & Bharadwaj S. 2009, MNRAS, 399, L128
  * Scrimgeour et al. (2012) Scrimgeour M. I. et al. 2012, MNRAS, 425, 116
  * Shibata & Asada (1995) Shibata M., & Asada H. 1995, Progress of Theoretical Physics, 94, 11
  * Smoot et al. (1992) Smoot G. F. et al. 1992, ApJ, 396, L1
  * Spergel et al. (2003) Spergel D. N. et al. 2003, ApJS, 148, 175
  * Springel et al. (2005) Springel V. et al. 2005, Nature, 435, 629
  * Swanson et al. (2008) Swanson M. E. C., Tegmark M., Hamilton A. J. S., & Hill J. C. 2008, MNRAS, 387, 1391
  * Sylos Labini (2011) Sylos Labini F. 2011, Journal of Cosmology 15, 6110
  * Sylos Labini & Baryshev (2010) Sylos Labini F., & Baryshev Y. V. 2010, J. Cosmology Astropart. Phys., 2010, 021
  * Sylos Labini et al. (2009) Sylos Labini F., Vasilyev N. L., Pietronero L., & Baryshev Y. V. 2009, EPL (Europhysics Letters) 86, 49001
  * Teyssier et al. (2009) Teyssier R. et al. 2009, A&A, 497, 335
  * Vogelsberger et al. (2020) Vogelsberger M., Marinacci F., Torrey P., & Puchwein E. 2020, Nature Reviews Physics, 2, 42
  * Wald (1983) Wald R. M. 1983, Phys. Rev. D, 28, 2118
  * Yadav et al. (2005) Yadav J., Bharadwaj S., Pandey B., & Seshadri T. R. 2005, MNRAS, 364, 601
  * Yadav et al. (2010) Yadav J. K., Bagla J. S., & Khandai N., 2010, MNRAS, 405, 2009
  * Zhai et al. (2017) Zhai Z. et al. 2017, ApJ, 848, 76


# Addressing Issues of Cross-Linguality in Open-Retrieval Question Answering
Systems For Emergent Domains

Alon Albalak, Sharon Levy, William Yang Wang
University of California, Santa Barbara
<EMAIL_ADDRESS>
###### Abstract

Open-retrieval question answering systems are generally trained and tested on
large datasets in well-established domains. However, low-resource settings
such as new and emerging domains would especially benefit from reliable
question answering systems. Furthermore, multilingual and cross-lingual
resources in emergent domains are scarce, leading to few or no such systems.
In this paper, we demonstrate a cross-lingual open-retrieval question
answering system for the emergent domain of COVID-19. Our system adopts a
corpus of scientific articles to ensure that retrieved documents are reliable.
To address the scarcity of cross-lingual training data in emergent domains, we
present a method utilizing automatic translation, alignment, and filtering to
produce English-to-all datasets. We show that a deep semantic retriever
greatly benefits from training on our English-to-all data and significantly
outperforms a BM25 baseline in the cross-lingual setting. We illustrate the
capabilities of our system with examples and release all code necessary to
train and deploy such a system111Code provided at github.com/alon-albalak/XOR-
COVID..

##  1 Introduction

Figure 1: An overview of our cross-lingual COVID-19 open-retrieval question-
answering system.

One challenge of emergent domains is that the originating locality is unknown,
leading to the need for reliable information to cross language barriers.
However, it is unlikely that domain-specific information will be available
across multiple languages for a new domain. Furthermore, information rapidly
changes in emerging domains, compounding the challenge of accessing credible
data.

An example of a prominent emergent domain is COVID-19, which has quickly
spread across the globe. To combat the spread of misinformation about
COVID-19, researchers have developed open-retrieval question answering Chen
and Yih (2020) systems which use large collections of trusted documents. For
example, Lee et al. (2020), Levy et al. (2021), and Esteva et al. (2021) all
develop open-retrieval QA systems using large corpuses of scientific journal
articles. However, because these systems focus on English, they leave a gap
for implementation on emergent domains that do not originate in English-
speaking locations.

To address the limitations of prior systems, we implement a cross-lingual
open-retrieval question answering system that retrieves answers from a large
collection of multilingual documents, where answers may be in a language
different from the question Asai et al. (2021).

In this work we take COVID-19 as an exemplar of an emergent domain and present
our system, which addresses two main areas of importance:

  * •

Cross-linguality: The locality of an emergent domain is unknown ahead of time,
making cross-lingual QA essential. Additionally, because data can rapidly
change in emerging domains, new information may develop in multiple languages,
motivating the need for systems that work across many languages.

  * •

Scarcity of training data: Data scarcity is an expected concern for emergent
domains, but multilingual and cross-lingual data are even more limited. We
demonstrate that by employing automatic translation, alignment, and filtering
methods, this challenge can be overcome in low-resource open-retrieval QA.

This system demonstration provides in-depth technical descriptions of the
individual components of our cross-lingual open-retrieval question answering
system: cross-lingual retrieval and cross-lingual reading comprehension
modules. Then, we describe how to combine the components along with document
re-ranking into the complete system, shown in Figure 1, and present several
examples taken from our system.

##  2 Cross-Lingual Dense Retrieval

Training a dense retriever is challenging in low-resource settings, such as
emergent domains, due to the data-hungry nature of large language models. This
challenge is compounded in the cross-lingual setting, where we aim to train a
model to encode concepts from multiple languages into a similar location in
the embedding space. In this section, we discuss how we overcome these
challenges.

###  2.1 Data

Cross-lingual retrieval requires two datasets; a large-scale multilingual
corpus of scientific articles from which to retrieve documents and a cross-
lingual dataset for training the retriever. However, a very limited number of
COVID-19 datasets have been released, few of which are multilingual and none
of which are cross-lingual.

Figure 2: Multilingual vs. cross-lingual question answering: In the multilingual setting, QA pairs exist for multiple languages in a one-to-one mapping. On the other hand, in cross-lingual QA questions may have answers in any language, creating a one-to-many mapping. COUGH | 9151 (en) | 1077 (es) | 778 (zh) | 697 (fr) | 573 (ja) | 531 (ar)
---|---|---|---|---|---|---
mCORD-19 | 172977 (en) | 1109 (es) | 951 (zh) | 711 (de) | 614 (fr) | 328 (pt)
Table 1: Top 6 languages by count for COUGH and the multilingual CORD-19 datasets. Language codes are the following: en-English, es-Spanish, zh-Chinese, fr-French, de-German, ja-Japanese, ar-Arabic, pt-Portuguese. | Answer
---
Language
Spanish | Mandarin | French | Arabic | German | Russian | Vietnamese | Italian
En2All | 8695 | 8441 | 8372 | 8231 | 8226 | 8156 | 8072 | 8003
| Filtered
---
En2All
6620 | 5869 | 5635 | 5808 | 5867 | 4137 | 531 | 6568
Table 2: QA pairs in our En2All and Filtered En2All variants of the COUGH
dataset, where each question is in English, and the context and answer are in
the language specified above.

CORD-19 Lu Wang et al. (2020) is a large-scale corpus of scientific papers on
COVID-19, however a known limitation is that it contains only English
articles. We draw inspiration from this work to address the lack of a large
scale corpus of multilingual COVID-19 scientific articles. For our system, we
use a manually collected corpus of English abstracts from PubMed, some of
which have parallel abstracts in additional languages. The corpus is collected
using the same query as described by Lu Wang et al. (2020) . We call this
corpus multilingual CORD-19 (mCORD-19), and the language distribution can be
found in Table 1.

To train our retriever we utilize the COUGH Zhang et al. (2021) dataset, which
is a multilingual FAQ retrieval dataset and consists of COVID-19 QA pairs.
Although COUGH is multilingual, containing samples in 9 different languages,
COUGH does not contain any cross-lingual QA pairs. The language distribution
is shown in Table 1.

###  2.2 Cross-lingual Data Generation

To address the lack of cross-lingual data in COUGH we introduce a modification
of the dataset which we call English-to-all (En2All), where we convert the
dataset from the multilingual to cross-lingual setting, as demonstrated in
Figure 2. Because we are interested in a system which will find non-English
answers to English questions, we create En2All through two translation
processes. First, we translate the answer portion of every QA pair from COUGH
into eight languages: Arabic, French, German, Italian, Mandarin, Russian,
Spanish, and Vietnamese. Secondly, we translate the question portion of all QA
pairs from any of the above languages into English222All translations are
generated by the MarianNMT system Junczys-Dowmunt et al. (2018) through the
Huggingface Transformers Wolf et al. (2020) library..

As machine translation models do not perform perfectly, there may be instances
within En2All that contain poor translations. To resolve this problem, we
utilize LaBSE Feng et al. (2020), an existing BERT-based sentence embedding
model that encodes 109 languages into a shared embedding space. The model is
utilized to compare the alignment of translations across different languages.
We take the following steps to filter out any poor translations in the data:

  1. 1.

We step through the current En2All and calculate similarity scores between
translated answers and their original English answers. To do this, we have
eight different comparisons for each translated English QA pair.

  2. 2.

Once the similarity scores have been calculated, we remove translations that
do not meet a threshold and are classified as poor translations.

After going through these steps, roughly one-third of the data samples from
En2All are removed for poor translations.

###  2.3 Methodology: Deep Semantic Retriever

Our retrieval model is based on the dense passage retriever from Karpukhin et
al. (2020). In contrast to their work, we train a unified encoder that encodes
both query and corpus into a shared space. For the encoder, we train the
multilingual BERT (mBERT) Devlin et al. (2019) and XLM-RoBERTa (XLM-R) Conneau
et al. (2020) models. Both models have been pre-trained using a tokenizer
which shares a vocabulary for over 100 languages, allowing the models to
encode all languages into a shared space. We train these models on the FAQ
retrieval task by maximizing the inner product of correct QA pairs and
minimizing the inner product of within-batch incorrect pairs.

###  2.4 Cross-Lingual Retrieval Evaluation

To evaluate our models in the large-scale open-retrieval setting we utilize
the questions from COUGH and En2All as our queries and the mCORD-19 dataset
for our retrieval corpus. Because we have no ground truth labels for correct
documents, and indeed there may be some unanswerable questions given this
corpus, we measure model quality through a fuzzy matching metric, Fuzzy Match
at top k documents (FM@k). FM@k utilizes the multilingual Sentence-BERT model
from Reimers and Gurevych (2019)333We use the ’paraphrase-multilingual-mpnet-
base-v2’ variant. Each of the top k retrieved documents is split into it’s
component sentences and embedded using the sentence-BERT model. Next, each
sentence is compared with the ground truth answer by calculating the cosine
similarity with the reference answer embedding from COUGH. If any of the
cosine similarities for that documents sentences are above a threshold, the
document is evaluated as a positive retrieval.

Model |  | COUGH
---
(FM@5/100)
| COUGH
---
+En2All
(FM@5/100)
BM25 | 18.6/41.4
mBERTbase | 22.8/49.5 | 26.4/50.7
\+ En2All | 28.0/54.9 | 27.7/51.7
XLM-Rbase | 25.0/51.3 | 28.1/51.6
\+ En2All | 30.1/55.4 | 28.4/52.2
| \+ Filtered-
---
En2All
32.9/56.7 | 30.9/53.4
XLM-Rlarge | 30.5/56.6 | 29.8/53.2
\+ En2All | 32.1/56.4 | 29.6/52.9
Table 3: Retrieval evaluation results. All models are trained on COUGH and
additional training data is denoted by "+". The middle column takes queries
from COUGH, the right column from COUGH and En2All. For both columns, the
retrieval corpus is mCORD. FM@5 and FM@100 are the fuzzy matching techniques
proposed to determine open-retrieval accuracy described in section 2.4.
Because BM25 is not cross-lingual, we translate it’s queries into all
languages in order to fairly compare against our cross-lingual models.

The results for our models and a BM25 baseline are found in Table 3. Since a
multilingual BM25 cannot perform cross-lingual retrieval, in order to fairly
compare against cross-lingual models, we translate all queries into every
other language in the mCORD corpus and then perform BM25 retrieval.

BM25 drastically under performs compared to encoder models and demonstrates
the need for a dense retrieval model. Although encoder models outperform BM25
when trained on multilingual data (COUGH), they are further improved by
training on cross-lingual data (En2All). Additionally, after filtering low
quality translations from En2All, we see further improvement in performance.

##  3 Cross-Lingual Reading Comprehension

###  3.1 Data

To train our cross-lingual reading comprehension model, we would ideally use a
cross-lingual covid-specific question answering dataset. However, similarly to
cross-lingual retrieval no such dataset exists so we augment existing
datasets.

Artetxe et al. (2020) introduced XQuAD, a multilingual QA dataset composed of
240 paragraphs and 1190 QA pairs from SQuAD v1.1 which have been
professionally translated into 10 languages. We utilize XQuAD as a pretraining
dataset before performing any training on covid-specific datasets444We open-
source our models pretrained on XQuAD at https://huggingface.co/alon-albalak.
Möller et al. (2020) introduce Covid-QA, a covid-specific QA dataset
consisting of 2019 question-answer pairs, however, it contains english-only
data. We modify Covid-QA with translations from MarianMT Junczys-Dowmunt et
al. (2018) to generate two dataset variants based on the multilingual and
cross-lingual settings shown in Figure 2: Multilingual Covid-QA (MCQA) and
English-to-all (En2All). MCQA is a multilingual version of Covid-QA, created
by translating all QA pairs into 9 languages to match those from XQuAD:
Arabic, German, Greek, Spanish, Hindi, Mandarin, Romanian, Russian, and
Vietnamese. En2All is our cross-lingual variation of Covid-QA, in a similar
spirit to the cross-lingual variant of COUGH. Because Covid-QA is english-
only, to generate En2All we translate all contexts/answers into the same 9
languages as MCQA.

###  3.2 Methodology: Span Extraction

Similar to our dense semantic retriever, we train mBERT and XLM-RoBERTa models
for our reading comprehension task. We formulate reading comprehension as a
span extraction task, where each model learns to find start and end tokens
which represent the answer span in a document.

Model |  | MCQA
---
(EM/F1)
| MCQA+En2All
---
(EM/F1)
mBERTbase | 20.0/57.5 | 19.6/55.4
\+ XQuAD | 21.2/57.7 | 20.5/55.6
\+ En2All | 19.3/56.1 | 19.2/55.8
XLM-Rbase | 25.1/60.0 | 24.4/58.9
\+ XQuAD | 26.7/61.6 | 26.1/61.3
\+ En2All | 24.0/58.8 | 23.9/58.3
XLM-Rlarge | 26.5/62.7 | 26.4/62.2
\+ XQuAD |  29.1/62.1 |  29.0/61.7
\+ En2All | 26.3/61.1 | 26.6/60.8
Table 4: Reading comprehension evaluation results. All models are trained on
MCQA, and additional training data is denoted by "+". The left column shows
evaluation on a multilingual dataset where questions/contexts are always in
the same language. The right column additionally evaluates on a cross-lingual
dataset where questions are in english and context paragraphs may be in any
language.

Figure 3: The main interface of our system. At the top is the search bar,
where the current query is "What are the symptoms of covid in children?" Below
the search bar are the three retrieved articles, ranked by relevance. In this
example, the first retrieved document has been expanded to show the title and
original text in Turkish, on the left. And on the right is the translation of
the answer and the full document into English.

###  3.3 Cross-Lingual Reading Comprehension Evaluation

To evaluate our models in the reading comprehension task, we utilize the QA
datasets described in Section 3.1. We evaluate our models based on exact match
(EM) and F1 metrics by comparing the predicted answer spans with ground-truth
answers.

The results for our models are found in Table 4. We train each of our models
on MCQA and supplement it with data from XQuAD or En2All. Interestingly, we
find that although En2All improved models in the retrieval setting, it only
hurt model performance in QA. We also see that pretraining on XQuAD improves
performance in all metrics for both base models, but leads to a slight
decrease in F1 score for XLM-Rlarge. In our demo, we utilize XLM-Rlarge which
was pretrained on XQuAD because it has only slightly worse F1 score, but
significantly higher exact match compared to the next best model.

##  4 Cross-Lingual Open-Retrieval Question Answering

Figure 4: A retrieved document for the query "What is the death rate of
COVID", which shows multiple correct answers corresponding to different
provinces of South Korea.

Our system is composed of the retrieval and reading comprehension modules
described in sections 2 and 3. The full end-to-end system is shown in Figure
1. After the retriever has been trained, the mCORD-19 corpus is encoded and
stored in the dense multilingual corpus index. When a question is posed to the
system, the query is encoded, and a maximum inner product search is performed
over the index to find documents most similar to the query. Answers are then
extracted from the retrieved documents and the documents are re-ranked based
on answer confidence from the span extraction model. Finally, the answer spans
and full documents are translated into English and presented to the user with
highlighted answers.

##  5 Demo

The demonstration retrieves documents from our mCORD-19 corpus, which has been
encoded by the deep semantic retriever from section 2.3. We provide additional
examples from the demo in Appendix A.

Figure 5: The options sidebar for our demonstration system. The options
include: number of articles to return, article languages to retrieve from, and
publication date range. For visualization purposes we show all language
options.

###  5.1 Sidebar Interface

Our system has an options sidebar, shown in Figure 5, which gives the user
several choices before entering a query. The user can determine how many
documents they would like to see results from, they can select which languages
the retrieved documents should be in, and they can specify a date range for
the publications to search over. If there are no relevant documents in the
desired date range, then the system will retrieve from any date range and
displays a message to inform the user.

###  5.2 Main Interface

To query the system, a user simply selects the desired options from the
sidebar and enters their question into the search bar, as seen in Figure 3.
After the user enters their question, the system will encode the question
using the trained deep semantic retriever and find the most relevant documents
within the given language and date range constraints. Then, the reading
comprehension model will extract the answer (or answers) most relevant to the
query from each retrieved document. Additionally, for any non-English
documents, the system translates both the retrieved article and extracted
answers into English555All translations are generated by MarianNMT Junczys-
Dowmunt et al. (2018) from the Huggingface Transformers library Wolf et al.
(2020).. Finally, the retrieved documents will be re-ranked based on the
confidence scores for the extracted answers.

The desired number of documents will be displayed to the user as a list of
publication dates. Each item can be expanded to show the article title,
original document with highlighted answers, translated answers, and the full
article translation. If an article contains a single answer, it will be
highlighted in red. If there are multiple answers, each answer will be
highlighted with a different color to allow for easy alignment between
original answers and their translations, demonstrated in Figure 4. Additional
examples are provided in Appendix A.

##  6 Conclusion

In this work, we tackled two challenging areas in open-retrieval QA: cross-
linguality and data scarcity. We presented methods for generating cross-
lingual data in an emergent domain, COVID-19. Then, we demonstrated that an
open-retrieval QA system trained on our data significantly outperforms a BM25
baseline. We hope that the methods presented here allow for increased access
to reliable information in future emergent domains.

##  7 Ethics Statement

Crucial to any open-retrieval question-answering system, the credibility and
truthfulness of the documents is paramount, in particular when trying to
prevent and combat misinformation that arises in emergent domains. Any
question-answering system is limited by the corpus used. To this end, we do
our best to ensure that any information included in our corpus is truthful by
including only peer-reviewed scientific articles from
PubMed666https://pubmed.ncbi.nlm.nih.gov/.

Furthermore, there may be emergent domains without peer-reviewed scientific
articles from which to draw answers. In these cases (and in fact in cases
where peer-review does exist) it is imperative to include sources along with
answers. This allows for users to judge the quality of information. In our
system we present the title and date of publication for each returned article
so that users can find the source content if desired.

## References

  * Artetxe et al. (2020) Mikel Artetxe, Sebastian Ruder, and Dani Yogatama. 2020.  On the cross-lingual transferability of monolingual representations.  In _ACL_.
  * Asai et al. (2021) Akari Asai, Jungo Kasai, Jonathan H. Clark, Kenton Lee, Eunsol Choi, and Hannaneh Hajishirzi. 2021.  XOR QA: Cross-lingual open-retrieval question answering.  In _NAACL-HLT_.
  * Chen and Yih (2020) Danqi Chen and Wen-tau Yih. 2020.  Open-domain question answering.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: Tutorial Abstracts_ , pages 34–37, Online. Association for Computational Linguistics.
  * Conneau et al. (2020) Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020.  Unsupervised cross-lingual representation learning at scale.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 8440–8451, Online. Association for Computational Linguistics.
  * Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019.  BERT: Pre-training of deep bidirectional transformers for language understanding.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics.
  * Esteva et al. (2021) Andre Esteva, Anuprit Kale, Romain Paulus, Kazuma Hashimoto, Wenpeng Yin, Dragomir Radev, and Richard Socher. 2021.  Covid-19 information retrieval with deep-learning based semantic search, question answering, and abstractive summarization.  _npj Digital Medicine_ , 4(1):68.
  * Feng et al. (2020) Fangxiaoyu Feng, Yinfei Yang, Daniel Cer, Naveen Arivazhagan, and Wei Wang. 2020\.  Language-agnostic bert sentence embedding.  _arXiv preprint arXiv:2007.01852_.
  * Junczys-Dowmunt et al. (2018) Marcin Junczys-Dowmunt, Roman Grundkiewicz, Tomasz Dwojak, Hieu Hoang, Kenneth Heafield, Tom Neckermann, Frank Seide, Ulrich Germann, Alham Fikri Aji, Nikolay Bogoychev, André F. T. Martins, and Alexandra Birch. 2018.  Marian: Fast neural machine translation in C++.  In _Proceedings of ACL 2018, System Demonstrations_ , pages 116–121, Melbourne, Australia. Association for Computational Linguistics.
  * Karpukhin et al. (2020) Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020.  Dense passage retrieval for open-domain question answering.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 6769–6781, Online. Association for Computational Linguistics.
  * Lee et al. (2020) Jinhyuk Lee, Sean S. Yi, Minbyul Jeong, Mujeen Sung, WonJin Yoon, Yonghwa Choi, Miyoung Ko, and Jaewoo Kang. 2020.  Answering questions on COVID-19 in real-time.  In _Proceedings of the 1st Workshop on NLP for COVID-19 (Part 2) at EMNLP 2020_ , Online. Association for Computational Linguistics.
  * Levy et al. (2021) Sharon Levy, Kevin Mo, Wenhan Xiong, and William Yang Wang. 2021.  Open-Domain question-Answering for COVID-19 and other emergent domains.  In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing: System Demonstrations_ , pages 259–266, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
  * Lu Wang et al. (2020) Lucy Lu Wang, Kyle Lo, Yoganand Chandrasekhar, Russell Reas, Jiangjiang Yang, Darrin Eide, Kathryn Funk, Rodney Kinney, Ziyang Liu, William Merrill, Paul Mooney, Dewey Murdick, Devvret Rishi, Jerry Sheehan, Zhihong Shen, Brandon Stilson, Alex D. Wade, Kuansan Wang, Chris Wilhelm, Boya Xie, Douglas Raymond, Daniel S. Weld, Oren Etzioni, and Sebastian Kohlmeier. 2020.  Cord-19: The covid-19 open research dataset.  _ArXiv_ , page arXiv:2004.10706v2.  32510522[pmid].
  * Möller et al. (2020) Timo Möller, Anthony Reina, Raghavan Jayakumar, and Malte Pietsch. 2020.  COVID-QA: A question answering dataset for COVID-19.  In _Proceedings of the 1st Workshop on NLP for COVID-19 at ACL 2020_ , Online. Association for Computational Linguistics.
  * Reimers and Gurevych (2019) Nils Reimers and Iryna Gurevych. 2019.  Sentence-BERT: Sentence embeddings using Siamese BERT-networks.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pages 3982–3992, Hong Kong, China. Association for Computational Linguistics.
  * Wolf et al. (2020) Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander M. Rush. 2020.  Transformers: State-of-the-art natural language processing.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations_ , pages 38–45, Online. Association for Computational Linguistics.
  * Zhang et al. (2021) Xinliang Frederick Zhang, Heming Sun, Xiang Yue, Simon Lin, and Huan Sun. 2021.  COUGH: A challenge dataset and models for COVID-19 FAQ retrieval.  In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, EMNLP 2021_ , pages 3759–3769.

##  Appendix A Additional Examples

Figure 6: The top 3 non-English results for the query "What are the symptoms
of covid in children?"

Figure 7: The top 3 non-english results for the query "What are the concerns
of having covid and diabetes?"

Figure 8: The top-3 non-english results for the query "Who is most vulnerable
to covid?"