# Stochastic gravitational waves produced by the first-order QCD phase
transition

Xu Han  Guo-yun Shao  School of Physics, Xi’an Jiaotong University, Xi’an,
710049, China

###### Abstract

We investigate the stochastic gravitational waves background arising from the
first-order QCD chiral phase transition, considering three distinct sources:
bubble collisions, sound waves, and fluid turbulence. Within the framework of
the Polyakov-Nambu-Jona-Lasinio (PNJL) model, we calculate the parameters
governing the intensity of the gravitational wave phase transition and
determine their magnitudes along the adiabatic evolutionary path. We introduce
the effective bag constant $B_{\mathrm{eff}}$ related to the dynamical
evolution of quarks to evaluate the intensity of the phase transition. By
calculating expanded potential at the point of false vacuum, we find that all
the bubbles are in the mode of runaway, leading the velocity of the bubble
wall to the speed of light. The resulting gravitational wave energy spectrum
is estimated, revealing a characteristic amplitude of the generated
gravitational waves within the centihertz frequency range. We present the
gravitational wave spectrum and compare it with the sensitivity range of
detectors, and find that the gravitational wave spectra generated by these
sources have the potential to be detected by future detectors such as BBO and
$\mu$ARES.

###### pacs:

12.38.Mh, 25.75.Nq

##  I introduction

Gravitational waves (GWs), as predicted by general relativity, are generated
by alterations in space-time curvature and propagate outwards at the speed of
light in wave-like form. Nowadays, gravitational wave signals from point
sources have been successfully detected multiple times, such as the generated
GWs form the binary compact objects LIGOScientific:2016aoc ;
LIGOScientific:2017vwq . However, there has been no observational data for the
stochastic gravitational wave background (SGWB). Until recently, NANOGrav
released their data, claiming that it might be possible to detect the SWGB
signals within a certain error range NANOGrav:2023hde ; NANOGrav:2023gor ,
European PTA Antoniadis:2023utw ; Antoniadis:2023rey , Parkes Zic:2023gta ;
Reardon:2023gzh  and Chinese Xu:2023wog  PTAs also reported their data, which
has attracted ample attention in the astrophysics, cosmology and particle
physics communities.

The primary source of the stochastic gravitational wave background is phase
transitions that occurred in the early universe during its evolution. The
early universe undergoes multiple phase transitions during its development. If
these changes were of a strong first order, the scalar field shifts from a
symmetric phase to a broken one through quantum tunneling or thermal
fluctuations. This would result in bubble nucleation within the unstable sea.
If this process is sufficiently rapid compared to the expansion rate
determined by the Hubble parameter $H$, these bubbles would expand and
collide, generating gravitational waves witten1984cosmic ;
hogan1986gravitational ; Tamanini:2016zlh ; Athron:2022mmm . Due to the
minimal interaction of the gravitational waves with matter, they overlay and
permeate the entire universe, giving rise to what is known as the Stochastic
Gravitational Wave Background (SGWB).

According to modern cosmology, approximately within $10^{-6}\,s$ after the Big
Bang, the universe was primarily composed of a quark-gluon plasma (QGP) made
up of free quarks and gluons. This state has what is known as chiral symmetry.
With the expansion and cooling of the universe, when the energy reached about
$100\,\mathrm{MeV}$, quarks and gluons began to bind together to form hadrons.
This process is known as color confinement. Simultaneously, chiral symmetry
was broken, implying that left-handed and right-handed quarks are no longer
identical. This marked the occurrence of the QCD chiral phase transition.
Although the standard model of particle physics does not discuss such phase
transitions, they are predicted by non-perturbative QCD Chu:2022mjw ;
Sagunski:2023ynd .

The physical image of the entire process is particularly intriguing. The
scalar field initially lies at the global minimum of its potential function.
As the first order chiral phase transition happens, some of the parameters
change, a new minimum appears, gradually replacing the original minimum as the
parameter continues to change, triggering vacuum decay Coleman:1980 . If the
changing parameter is the background temperature, this decay is known as
thermal decay, leading to bubble nucleation Linde:1983 . After bubble
nucleation, the pressure difference inside and outside the bubble drives the
bubble wall’s expansion. Once bubbles expand beyond inter-bubble separation,
they inevitably collide. These collisions release kinetic energy stored in the
bubble walls, transforming it into gravitational waves carrying energy and
momentum of the phase transition. Considering all other effects, in summary,
the generation of the power spectrum of stochastic gravitational waves from
FOPT is the combined result of three key mechanisms: bubble wall collisions
triggered by scalar field dynamics Kosowsky:1992vn ; laine1994bubble , sound
waves produced by fluid motion Hindmarsh:2013xza ; Hindmarsh:2017gnf , and MHD
turbulence induced by plasma dynamics Caprini:2006jb .

Our work focus on the gravitational wave background generated by the first-
order QCD phase transition in the early universe in the scenario of little
inflation Boeckel:2010 . The calculation is performed within the framework of
one of the effective models describing strong-coupling QCD, the PNJL model
Ratti:2005jh . We adopt a reasonable assumption that the bubbles evolve
adiabatically along the path $s/\rho_{B}=\mathrm{const}$ reiter1998entropy ;
gunther2017qcd ; Shuve:2017jgj  from the local minimum to the true vacuum,
providing relatively exact values for the energy budget parameter $\alpha$
along each evolution line. Furthermore, we use the method of replacing the
local minimum with its second-order Taylor expansion to derive the criterion
for whether the bubble expansion is runaway or not Chen:2017cyc ; Li:2021qer .
And we find that for bubbles evolving along the adiabatic approximation
evolution line, not reaching runaway is an impossible situation. Then the
gravitational wave signals corresponding to the three situations are detailed
and compared with the detection capabilities of current gravitational wave
detectors. The results show that some gravitational signals has strong
potential to be detected in future observations.

This paper is divided into three comprehensive sections. In Sec.II, we offer a
brief introduction of the PNJL model, and use which to explicitly calculate
the thermodynamic parameters of the first-order chiral phase transition for
later computation. In Sec.III.1, we study the chiral phase transition of QCD,
and provide a detailed description of the adiabatic evolution and the criteria
for bubble runaway expansion, from which we obtain the key parameters used for
gravitational wave signal calculations. In Sec.III.2, we calculate the
gravitational wave spectrum, and further determine the corresponding GW signal
and evaluate the prospects for detectability by future observatories. Finally,
a summary is given in Sec.IV.

##  II Polyakov-Nambu-Jona-Lasinio model

The PNJL model is widely used in the description of QCD phase transitions The
Lagrangian density in the PNJL model is formulated as

$\displaystyle\mathcal{L}_{\mathrm{PNJL}}$ $\displaystyle=$
$\displaystyle\,\bar{q}\,(i\gamma^{\mu}D_{\mu}+\gamma_{0}\hat{\mu}-\hat{m_{0}})\,q$
(1)
$\displaystyle+\,G\sum^{8}_{k=0}\left[(\bar{q}\lambda^{k}q)^{2}+(\bar{q}\,i\gamma_{5}\lambda^{k}q)^{2}\right]$
$\displaystyle-\,K\left[\mathrm{det}_{f}(\bar{q}\,(1+\gamma_{5})\,q)+\mathrm{det}_{f}(\bar{q}\,(1-\gamma_{5})\,q)\right]$
$\displaystyle-\,U(\Phi,\bar{\Phi};T).$

The covariant derivative in the Lagrangian is defined as
$D_{\mu}=\partial_{\mu}-iA_{\mu}$. The gluon background field
$A_{\mu}=\delta_{\mu}^{0}A_{0}$ is supposed to be homogeneous and static, with
$A_{0}=g\mathcal{A}_{0}^{\alpha}\frac{\lambda^{\alpha}}{2}$, where
$\frac{\lambda^{\alpha}}{2}$ is $SU(3)$ color generators. The effective
potential $U(\Phi[A],\bar{\Phi}[A],T)$ is expressed with the traced Polyakov
loop $\Phi=(\mathrm{Tr}_{c}L)/N_{C}$ and its conjugate
$\bar{\Phi}=(\mathrm{Tr}_{c}L^{\dagger})/N_{C}$. The Polyakov loop $L$ is a
matrix in color space

$L(\vec{x})=\mathcal{P}\exp\bigg{[}i\int_{0}^{\beta}d\tau
A_{4}(\vec{x},\tau)\bigg{]},$ (2)

where $\beta=1/T$ is the inverse of temperature and $A_{4}=iA_{0}$.

In Eq. (1), $U$ symbolizes the effective potential energy of the Polyakov
loop, which is temperature-dependent. Its role is to encapsulate the kinetic
energy term associated with gluons. However, its selection is not unique. By
juxtaposing this with results from lattice QCD simulations, a logarithmic
effective potential can be ascertained, which is given as follows
Schaefer:2009ui

$\displaystyle\frac{U(\Phi,\bar{\Phi};T)}{T^{4}}=$
$\displaystyle\,b(T)\,\mathrm{ln}\left[1-6\bar{\Phi}\Phi+4(\bar{\Phi}^{3}+\Phi^{3})-3(\bar{\Phi}\Phi)^{2}\right]$
(3) $\displaystyle-\frac{a(T)}{2}\bar{\Phi}\Phi,$

where
$a(T)=a_{0}+a_{1}\left({T_{0}}/{T}\right)+a_{2}\left({T_{0}}/{T}\right)^{2}$,
$b(T)=b_{3}\left({T_{0}}/{T}\right)^{3}$. The parameters in the model are
determined through an analysis of QCD thermodynamics and by fitting to the
pure canonical region of QCD grid points Kaczmarek:2002mc ; Kaczmarek:2007pb ,
as listed in the Table. 1.

Table 1: PNJL model parameters $a_{0}$ | $a_{1}$ | $a_{2}$ | $b_{3}$ |  $T_{0}/MeV$
---|---|---|---|---
3.51 |  -2.47 |  15.2 |  -1.75 |  210

According to the temperature field theory, the giant thermodynamic potential
of the system is as follow

$\displaystyle\Omega_{PNJL}$ $\displaystyle=$
$\displaystyle-\,2\,T\sum_{i=u,d,s}\int\frac{d^{3}p}{({2\pi})^{3}}(F_{1}+F_{2})$
(4)
$\displaystyle-\,2\int_{\Lambda}\frac{d^{3}p}{({2\pi})^{3}}3\,(E_{u}+E_{d}+E_{s})+2\,G\sum_{i=u,d,s}\phi_{i}^{2}$
$\displaystyle-\,4\,K\phi_{u}\phi_{d}\phi_{s}+U(\Phi,\bar{\Phi};T)+C,$

where $E_{i}=\sqrt{\textbf{{p}}^{2}+M_{i}^{2}}$, $F_{1}=\mathrm{ln}(1+3\Phi
e^{{-(E_{i}-\mu_{i})}/{T}}+3\bar{\Phi}e^{{-2(E_{i}-\mu_{i})}/{T}}+e^{{-3(E_{i}-\mu_{i})}/{T}})$
and $F_{2}=\mathrm{ln}(1+3\bar{\Phi}e^{{-(E_{i}+\mu_{i})}/{T}}+3\Phi
e^{{-2(E_{i}+\mu_{i})}/{T}}+e^{{-3(E_{i}+\mu_{i})}/{T}})$.

The thermodynamic potential involves five independent variables: $\phi_{u}$,
$\phi_{d}$, $\phi_{s}$, $\Phi$ and $\bar{\Phi}$. By examining the extremal
values of the thermodynamic potential with respect to these variables, their
corresponding values can be determined. For the clarification of our following
computation, we present the expression of the energy density

$\displaystyle\epsilon$ $\displaystyle=$
$\displaystyle\,T\,s+\sum_{i=u,d,s}\rho_{i}\,\mu_{i}+\Omega_{PNJL}$ (5)
$\displaystyle=$
$\displaystyle-\,2\,N_{c}\sum_{i=u,d,s}\int_{\Lambda}\frac{d^{3}p}{({2\pi})^{3}}E_{i}$
$\displaystyle+\,6\sum_{i=u,d,s}\int\frac{d^{3}p}{({2\pi})^{3}}E_{i}\left[n(E_{i}-\mu_{i})+\bar{n}(E_{i}+\mu_{i})\right]$
$\displaystyle+\,2\,G\sum_{i=u,d,s}\phi_{i}^{2}-4\,K\phi_{u}\phi_{d}\phi_{s}$
$\displaystyle+\,U(\Phi,\bar{\Phi};T)-T~{}\frac{\partial
U(\Phi,\bar{\Phi};T)}{\partial T}+C$

It is important to highlight that the PNJL model requires the determination of
five parameters. These parameters are obtained through fitting: the values of
the masses of the $\pi$, $\mathbf{K}$, $\eta^{\prime}$ mesons: $m_{\pi}=135.0$
MeV, $m_{K}=497.7$ MeV, and $m_{\eta^{\prime}}=957.8$ MeV, as well as the
$\pi$ meson decay constant $f_{\pi}=92.4$ MeV Karsch:2001cy . The specific
values of these parameters for the subsequent calculations are as follows: the
$u$ and $d$ quark masses $m_{u}=m_{d}=5.5$ MeV, the strange quark mass
$m_{s}=140.7$ MeV, the coefficient for the four-fermion interaction
$G\Lambda^{2}=1.835$, the coefficient for the six-fermion interaction
$K\Lambda^{5}=12.56$, and the three-momentum cut-off $\Lambda=602.3$ MeV.
These parameter values remain unchanged throughout the subsequent
calculations.

##  III Numerical results and discussions

###  III.1 Bubble evolution with a first-order phase transition in the PNJL
model

The physical process of gravitational wave production by first-order phase
transitions is described by bubble dynamics. The final shape of the
gravitational wave power spectrum is mainly controlled by two key parameters,
$\alpha$ and $\beta$. Fortunately, both can be determined by making reasonable
assumptions. Regarding the energy budget $\alpha$, its magnitude depends on
the difference between the field values at the global minimum and the local
minimum espinosa2010energy , which is also defined as the ratio of the vacuum
energy density released by the phase transition to the background radiation
energy density

$\alpha=\left.\dfrac{\Delta\epsilon_{\mathrm{vac}}}{\epsilon_{\mathrm{rad}}}\right|_{T=T_{\star}},$
(6)

where $T_{\star}$ is the transition temperature. $\epsilon_{rad}$ describes
the energy contribution of the radiation part in the quark field

$\epsilon_{\mathrm{rad}}=6\sum_{i=u,d,s}\int\frac{d^{3}p}{({2\pi})^{3}}E_{i}\left[n(E_{i}-\mu_{i})+\bar{n}(E_{i}+\mu_{i})\right].$
(7)

The energy budget has been studied in the bag model Giese:2020rtr ;
Wang:2023jto  with

$\Delta\epsilon_{\mathrm{vac}}=\epsilon-\epsilon_{\mathrm{rad}}=B,$ (8)

where $B$ is the bag constant.

However, the simple bag model is not sufficient to accurately describe the
strength of the phase transition. Therefore, inspired by the Bag model, in
this study we construct the effective bag constant with the dynamical quark
mass in the PNJL model, which is defined as

$B_{\mathrm{eff}}=\epsilon-\epsilon_{\mathrm{rad}},$ (9)

and the expression of the effective bag constant Shao:2011ij  is

$\displaystyle B_{\mathrm{eff}}$ $\displaystyle=$
$\displaystyle-\,2\,N_{c}\sum_{i=u,d,s}\int_{\Lambda}\frac{d^{3}p}{({2\pi})^{3}}E_{i}$
(10)
$\displaystyle+\,2\,G\sum_{i=u,d,s}\phi_{i}^{2}-4\,K\phi_{u}\phi_{d}\phi_{s}$
$\displaystyle+\,U(\Phi,\bar{\Phi};T)-T\,\frac{\partial
U(\Phi,\bar{\Phi};T)}{\partial T}+C.$

We find that $B_{\mathrm{eff}}$ behaves similarly as the bag constant in the
MIT-Bag model. However, its value depends not only on the interaction between
different quarks, but also on the density and temperature. And with
$B_{\mathrm{eff}}$, we can now rewrite the energy budget in the following form

$\alpha=\left.\dfrac{\Delta
B_{\mathrm{eff}}}{\epsilon_{\mathrm{rad}}}\right|_{T=T_{\star}}=\left.\dfrac{{B_{\mathrm{eff}}}_{+}-{B_{\mathrm{eff}}}_{-}}{{\epsilon_{+}-B_{\mathrm{eff}}}_{+}}\right|_{T=T_{\star}}.$
(11)

For the bubble, the subscript “+” denotes the state that has not yet decayed
and remains in a metastable state, while the subscript “–” represents the
state that has decayed and transitioned into a vacuum state.

In the early universe, the transition from the quark phase to the hadronic
phase is generally along a path with a small $s/\rho_{B}$, corresponding to
lower baryon chemical potential. However, the assumption of little inflation
suggests a possibility of a first-order phase transition from a high baryon
chemical potential path, dominated by the hot big bang scenario of cosmic
evolution Boeckel:2010 . Therefore, in this scenario it is necessary to
consider the changes in baryon chemical potential when computing the energy
budget of a first-order transition.

Since the local minima are distributed throughout the spatial grid, the choice
of the position of the false vacuum is somewhat arbitrary. Without specifying
the evolution of the bubbles, the value of $\alpha$ cannot be determined but
only approximately constrained. In this paper, we adopt a reasonable
assumption that the bubbles evolve adiabatically along the path from the local
minimum to the true vacuum, providing relatively exact values for $\alpha$
along each evolution line. This path is characterized by a constant value of
$s/\rho_{B}$ as shown in Fig. 1, where $s$ represents the entropy density and
$\rho_{B}$ is the baryon number density.

Figure 1: The phase diagram illustrates the evolution paths of bubbles under
different adiabatic conditions. The black solid line denotes the critical line
of the QCD first-order chiral phase transition. The black dashed line
indicates the crossover line of the chiral phase transition. Each colored
solid line represents the adiabatic evolution curve corresponding to different
values of $s/\rho_{B}$.

The other important quantity is denoted as $\beta$, which characterizes the
duration of the phase transition. It is defined as

$\beta=\left.H(T_{\star})T_{\star}\cdot\frac{\mathrm{d}(S_{3}/T)}{\mathrm{d}T}\right|_{T=T_{\star}}$
(12)

Here, $S_{3}$ denotes the three-dimensional Euclidean bounce action. This
parameter has been accurately given by solving the evolution variance
numerically in many other outstanding works Helmboldt:2019pan ; Chen:2022cgj ;
Bai:2023cqj . Here we refer to the value $\beta=4.75\times 10^{4}$ from the
literature Helmboldt:2019pan  for calculations and discussions.

The speed of the bubble wall is another crutial parameter in the description
of bubble evolution. After the bubbles form, their expansion mechanism depends
on the resistance provided by the background fluid. Specifically, it can be
divided into runaway and non-runaway situations, the former allowing the
bubble wall to reach the speed of light due to insufficient resistance, while
the latter implies the bubble wall which under a stronger resistance
stabilizes at a lower speed, but is still highly likely to be relativistic.

To study whether the bubble is in a runaway state or not, we use the method of
replacing the local minimum with its second-order Taylor expansion to derive
the criterion for whether the bubble expansion is runaway or not, if the value
of the effective potential at the point of the true vacuum is lower than that
at the false vacuum, then the bubble will runaway. We define an intensity
factor $\alpha_{\infty}$ at infinity, which is represented as

$\alpha_{\infty}=\frac{30}{24\pi^{2}}\frac{\sum_{i}\frac{1}{2}N\Delta
m_{i}^{2}}{g_{\star}T_{\star}^{2}},$ (13)

where $c_{i}=n_{i}(c_{i}=n_{i}/2)$ represents the degrees of freedom for boson
(fermion) particles, and $\Delta m_{i}^{2}$ is the square of the mass
difference between the two phases. In the absence of a large-mass gauge boson,
a condition of $\alpha>\alpha_{\infty}$ will result in an uncontrolled bubble
expansion, enabling the bubble wall velocity to reach the speed of light and
thus runaway bodeker2009can . At the runaway juncture, the energy contribution
distribution index of various components can be defined by the following
formula Caprini:2015zlo

$\displaystyle\kappa_{\mathrm{en}}=1-\frac{\alpha_{\infty}}{\alpha},$ (14)
$\displaystyle\kappa_{\mathrm{sw}}=\frac{\alpha_{\infty}}{0.73+0.083\sqrt{\alpha_{\infty}}+\alpha_{\infty}},$
(15) $\displaystyle\kappa_{\mathrm{tu}}=0.05\kappa_{\mathrm{sw}},$ (16)

where $\kappa_{\mathrm{en}}$, $\kappa_{\mathrm{sw}}$ and
$\kappa_{\mathrm{tu}}$ are the efficiency factors of bubble collision, sound
waves and turbulence, respectively. Given that the bubble attains the speed of
light, its wall velocity is $v_{w}=1$.

When $\alpha<\alpha_{\infty}$, the bubble will not runaway. Nevertheless, upon
calculation through the PNJL model, it’s found that for bubbles evolving along
the adiabatic approximation evolution line, this inequality only holds when
the phase transition intensity is extremely severe. In terms of the phase
diagram, this means that only when the bubble evolves along a path with a
lower value of $s/\rho_{B}$ will it not runaway. However, when the value of
$s/\rho_{B}$ is small, the bubbles nucleated in the false vacuum won’t reach
the true vacuum along the evolution line, as both the temperature and the
chemical potential will approach zero after evolution. Thus, from a physical
point of view, a non-runaway scenario cannot be realized. Therefore, we
specifically considers the situation where the bubbles runaway. In the
subsequent calculation of the gravitational wave spectrum, we choose the case
of $s/\rho_{B}=4$ for calculation.

###  III.2 Stochastic gravitational waves

Bubble collisions release kinetic energy stored in the bubble walls,
transforming it into gravitational waves carrying energy and momentum of the
phase transition. However, the sources of gravitational waves are not solely
from bubble wall collisions but also originate from the background fluid
itself. As bubbles expand within the background fluid, the movement of their
walls stimulates fluid to form a persistent sound shell, converting some
vacuum energy into kinetic energy, making sound waves an important source of
gravitational waves. Meanwhile, the turbulent motion of charged particles in
the plasma state of the background fluid causes fluctuations in the magnetic
field, forming magnetohydrodynamic (MHD) turbulence, another significant
source of gravitational waves Hogan:1986 . Therefore, the generation of the
gravitational wave power spectrum is a combined result of three key
mechanisms: bubble wall collisions induced by scalar field dynamics, sound
waves derived from fluid motion, and MHD turbulence induced by plasma
dynamics. Therefore, the total energy spectrum of gravitational waves can be
encapsulated in the following expression

$h^{2}\Omega(f)=h^{2}\Omega_{\mathrm{en}}(f)+h^{2}\Omega_{\mathrm{sw}}(f)+h^{2}\Omega_{\mathrm{tu}}(f).$
(17)

The fitting formula for the gravitational wave energy spectrum, which is
widely acknowledged in current studies due to comprehensive analyses and
numerical simulations, can be represented as huber2008gravitational ;
caprini2009stochastic ; caprini2009general

$\displaystyle
h^{2}\Omega_{\mathrm{en}}(f)=h^{2}{\Omega_{\mathrm{en,0}}}\frac{3.8\left(\frac{f}{f_{\mathrm{en}}}\right)^{2.8}}{1+2.8\left(\frac{f}{f_{\mathrm{en}}}\right)^{3.8}},$
(18) $\displaystyle
h^{2}\Omega_{\mathrm{sw}}(f)=h^{2}{\Omega_{\mathrm{sw,0}}}\left(\frac{f}{f_{\mathrm{sw}}}\right)^{3}\left(\frac{7}{4+3\left(\frac{f}{f_{\mathrm{sw}}}\right)^{2}}\right)^{3.5},$
(19) $\displaystyle
h^{2}\Omega_{\mathrm{tu}}(f)=h^{2}{\Omega_{\mathrm{tu,0}}}\frac{\left(\frac{f}{f_{\mathrm{tu}}}\right)^{2}}{\left(1+\frac{f}{f_{\mathrm{tu}}}\right)^{{11}/{3}}\left(1+\frac{8\pi
f_{\mathrm{tu}}}{h^{\star}}\frac{f}{f_{\mathrm{tu}}}\right)},$ (20)

where the crest factors are

$\displaystyle h^{2}{\Omega_{\mathrm{en,0}}}=3.6\times
10^{-5}\left(\frac{H_{\star}}{\beta}\right)^{2}\left(\frac{\kappa_{\mathrm{en}}\alpha}{1+\alpha}\right)^{2}\left(\frac{10}{g^{\star}}\right)^{\frac{1}{3}}\Gamma,$
(21) $\displaystyle h^{2}{\Omega_{\mathrm{sw,0}}}=5.7\times
10^{-6}\left(\frac{H_{\star}}{\beta}\right)\left(\frac{\kappa_{\mathrm{sw}}\alpha}{1+\alpha}\right)^{2}\left(\frac{10}{g^{\star}}\right)^{\frac{1}{3}}v_{w},$
(22) $\displaystyle h^{2}{\Omega_{\mathrm{tu,0}}}=7.2\times
10^{-4}\left(\frac{H_{\star}}{\beta}\right)\left(\frac{\kappa_{\mathrm{tu}}\alpha}{1+\alpha}\right)^{2}\left(\frac{10}{g^{\star}}\right)^{\frac{1}{3}}v_{w},$
(23)

where $\Gamma={0.11v_{w}^{3}}/{(0.42+v_{w}^{2})}$. And the peak frequencies
are

$\displaystyle\\!f_{\mathrm{en}}\\!=\\!1.12\times\\!10^{-8}\frac{0.62}{1.8-0.1v_{w}+v_{w}^{2}}\left(\frac{\beta}{H_{\star}}\right)\frac{T_{\star}}{100\mathrm{MeV}}\left(\frac{g^{\star}}{10}\right)^{\frac{1}{6}},\,\,\,\,\,\,\,\,$
(24) $\displaystyle f_{\mathrm{sw}}=1.29\times
10^{-8}\frac{1}{v_{w}}\left(\frac{\beta}{H_{\star}}\right)\frac{T_{\star}}{100\mathrm{MeV}}\left(\frac{g^{\star}}{10}\right)^{\frac{1}{6}},$
(25) $\displaystyle f_{\mathrm{tu}}=1.84\times
10^{-8}\frac{1}{v_{w}}\left(\frac{\beta}{H_{\star}}\right)\frac{T_{\star}}{100\mathrm{MeV}}\left(\frac{g^{\star}}{10}\right)^{\frac{1}{6}},$
(26)

where $h$ represents the reduced Hubble parameter, $H_{\star}$ signifies the
Hubble parameter at the transition temperature $T_{\star}$, $g^{\star}$
denotes the number of effective degrees of freedom (taking the value of
$g^{\star}=47.5$ for the PNJL model).

In terms of choosing the characteristic temperature $T_{\star}$ during the
phase transition, we approximately consider the transition temperature as the
CEP temperature equals to the critical temperature of chiral phase transition
and deconfinement phase transition $T_{c}=131\,\mathrm{MeV}$. Upon obtaining
these crucial parameters, the gravitational wave energy spectrum resulting
from the QCD first-order chiral phase transition can be computed.

Figure 2: The gravitational waves spectrum of the first-order QCD chiral phase
transition. The image is plotted in logarithmic coordinates, where the y-axis
represents the gravitational waves spectrum $h^{2}\Omega(f)$, and the x-axis
corresponds to the frequency $f$.

From Figure 2, we gain an insightful perspective into the dynamic spectrum of
gravitational waves generated during the early universe’s phase transitions.
The peak frequency of these waves is notably situated around $f=10^{-2}$ Hz,
which provides a crucial reference point for future observations and
theoretical models. Among the various mechanisms contributing to this
gravitational wave spectrum, it is evident that sound waves play a pivotal
role, particularly in the lower frequency range. Their influence spans a broad
frequency band, from $10^{-4}$ Hz to 1 Hz, showcasing the diverse impacts of
cosmic phenomena on gravitational wave generation. However, this dominance of
sound waves in shaping the gravitational wave spectrum is not uniform across
all frequencies. As we shift towards higher frequencies, particularly beyond
$f=1$ Hz, there is a noticeable decline in the gravitational waves attributed
to sound waves. This decrease is so pronounced that at approximately $f=1$ Hz,
the gravitational waves generated by bubble collisions surpass those produced
by sound waves. Further, at around $f=100$ Hz, the contribution from MHD
turbulence becomes more significant than that of the sound waves, though still
relatively minor in the overall spectrum.

##  IV Conclusions

In this study, we investigate the gravitational waves produced by the QCD
phase transition within the purview of the PNJL model. We made an assumption
about the adiabatic evolution of the bubbles carrying the chiral symmetry
during the phase transition, and concluded that the bubbles gradually runaway
during the evolution. Finally, we plotted the gravitational wave spectrum and
compared it with existing gravitational wave detection methods.

We reasonably select an adiabatic evolutionary trajectory characterized by an
entropy density to energy density ratio of $s/\rho_{B}=4$ to pinpoint the
critical junctures of the phase transition. This approach, which diverges from
the conventional assumption of isothermal transitions, augments the robustness
of our computational analysis. Furthermore, it furnishes a benchmark for
future inquiries focused on delineating the loci pertinent to bubble
nucleation and phase transition dynamics.

Our study also reveals that in first-order QCD chiral phase transitions, if
the phase transition intensity isn’t strong enough (characterized by a small
$\alpha$ value), bubbles originating from the false vacuum fail to gain enough
resistance factors throughout their evolution. Consequently, they gradually
approach the runaway state as their expansion velocity nears the speed of
light. On the other hand, based on the assumption that the entropy to energy
density ratio $s/\rho_{B}$ remains constant, when the phase transition
intensity is overly strong (symbolized by a large $\alpha$ value), bubbles are
unable to reach the true vacuum state through the adiabatic evolution pathway,
for the reason that the overly intense phase transition leads to extremely low
temperatures and densities after the evolution along the adiabatic pathway,
rendering it physically unfeasible.

The results of our investigation highlight a nuanced aspect of cosmological
phase transitions: an exceedingly vigorous phase transition may impede the
system’s progression to the true vacuum state, thereby indicating a diminished
probability of such high-intensity phase transitions occurring within the
early universe. Since the amplitude and detectability of gravitational waves
are intrinsically tied to the intensity of the underlying phase transition,
gravitational waves emanating from these infrequent yet intense phase
transitions represent a challenging detection frontier. Crucially, our
analysis delineates an upper bound on the intensity of chiral phase
transitions, a constraint that inherently limits the peak amplitude of the
resultant gravitational waves to approximately $10^{-14}$. This finding aligns
the peak frequency within a range of $10^{-4}\,\mathrm{Hz}$ to
$1\,\mathrm{Hz}$, a spectrum that falls within the anticipated sensitivity of
emerging gravitational wave observatories.

Notably, our analysis indicates that the overall intensity of the chiral phase
transition, while significant, is not sufficient to produce gravitational
waves strong enough for detection by current observatories like LISA. This
underscores the challenge in observing these subtle cosmic phenomena and the
need for more sensitive detection technology. However, advanced detectors,
such as the proposed Big Bang Observer (BBO), are anticipated to have the
capability to capture these elusive signals. The potential of these future
observatories to detect gravitational waves from phase transitions, especially
those occurring in the centihertz frequency range. This revelation steers the
focus of gravitational wave detection technology towards this specific
frequency domain, thereby enhancing the probability of discerning these subtle
cosmic signals.

## Acknowledgements

This work is supported by the National Natural Science Foundation of China
under Grant No. 11875213.

## References

## References

  * (1) B. P. Abbott et al. [LIGO Scientific and Virgo], Phys. Rev. Lett. 116, no.6, 061102 (2016)
  * (2) B. P. Abbott et al. [LIGO Scientific and Virgo], Phys. Rev. Lett. 119, no.16, 161101 (2017)
  * (3) G. Agazie et al. [NANOGrav], Astrophys. J. Lett. 951, no.1, L9 (2023)
  * (4) G. Agazie et al. [NANOGrav], Astrophys. J. Lett. 951, no.1, L8 (2023)
  * (5) J. Antoniadis et al. [EPTA], Astron. Astrophys. 678, A48 (2023)
  * (6) J. Antoniadis et al. [EPTA and InPTA:], Astron. Astrophys. 678, A50 (2023)
  * (7) A. Zic, D. J. Reardon, A. Kapur, G. Hobbs, R. Mandow, M. Curyło, R. M. Shannon, J. Askew, M. Bailes and N. D. R. Bhat, et al. [arXiv:2306.16230 [astro-ph.HE]].
  * (8) D. J. Reardon et al., Astrophys. J. Lett. 951, L6 (2023).
  * (9) H. Xu et al., Res. Astron. Astrophys. 23, 075024 (2023).
  * (10) E. Witten, Phys. Rev. D 30, 272 (1984).
  * (11) C. J. Hogan, Mon. Not. R. Astron. Soc. 218, 629–636 (1986).
  * (12) N. Tamanini, C. Caprini, E. Barausse, A. Sesana, A. Klein and A. Petiteau, JCAP 04, 002 (2016).
  * (13) P. Athron, C. Balázs and L. Morris, JCAP 03, 006 (2023).
  * (14) C. S. Chu and A. Ito, Phys. Rev. D 105, 123538 (2022).
  * (15) L. Sagunski, P. Schicho and D. Schmitt, Phys. Rev. D 107, 123512 (2023).
  * (16) S. Coleman and F. De Luccia, Phys. Rev. D 21, 3305 (1980).
  * (17) A. D. Linde, Nucl. Phys. B 216, 421 (1983).
  * (18) A. Kosowsky and M. S. Turner, Phys. Rev. D 47, 4372–4391 (1993).
  * (19) M. Laine, Phys. Rev. D 49, 3847–3853 (1994).
  * (20) M. Hindmarsh, S. J. Huber, K. Rummukainen and D. J. Weir, Phys. Rev. Lett. 112, 041301 (2014).
  * (21) M. Hindmarsh, S. J. Huber, K. Rummukainen and D. J. Weir, Phys. Rev. D 96, 103520 (2017).
  * (22) C. Caprini and R. Durrer, Phys. Rev. D 74, 063521 (2006).
  * (23) T. Boeckel, J. Schaffner-Bielich, Phys. Rev. Lett. 105, no.4, 041301 (2010).
  * (24) C. Ratti, M. A. Thaler and W. Weise, Phys. Rev. D 73, 014019 (2006).
  * (25) M. Reiter, A. Dumitru, J. Brachmann, J. A. Maruhn, H. Stöcker and W. Greiner, Nucl. Phys. A 643, 99–112 (1998).
  * (26) J. N. Guenther, R. Bellwied, S. Borsanyi, Z. Fodor, S. D. Katz, A. Pasztor, C. Ratti and K. K. Szabó, Nucl. Phys. A 967, 720–723 (2017).
  * (27) B. Shuve and C. Tamarit, JHEP 10, 122 (2017).
  * (28) Y. Chen, M. Huang and Q.-S. Yan, JHEP 05, 178 (2018).
  * (29) S. L. Li, L. Shao, P. Wu and H. Yu, Phys. Rev. D 104, 043510 (2021).
  * (30) B.-J. Schaefer, M. Wagner and J. Wambach, Phys. Rev. D 81, 074013 (2010).
  * (31) O. Kaczmarek, F. Karsch, P. Petreczky and F. Zantow, Phys. Lett. B 543, 41–47 (2002).
  * (32) O. Kaczmarek, PoS CPOD07, 043 (2007).
  * (33) F. Karsch, Lect. Notes Phys. 583, 209–249 (2002).
  * (34) J. R. Espinosa, T. Konstandin, J. M. No and G. Servant, JCAP 06, 028 (2010).
  * (35) F. Giese, T. Konstandin and J. van de Vis, JCAP 07, no.07, 057 (2020).
  * (36) X. Wang, C. Tian and F. P. Huang, JCAP 07, 006 (2023).
  * (37) J. R. Espinosa, T. Konstandin, J. M. No and G. Servant, JCAP 06, 028 (2010).
  * (38) G. Y. Shao, M. Di Toro, B. Liu, M. Colonna, V. Greco, Y. X. Liu and S. Plumari, Phys. Rev. D 83, 094033 (2011).
  * (39) A. J. Helmboldt, J. Kubo and S. van der Woude, Phys. Rev. D 100, 055025 (2019).
  * (40) Y. Chen, D. Li and M. Huang, JHEP 07, 225 (2023).
  * (41) Y. Bai, T. K. Chen and M. Korwar, (2023). arXiv:2306.17160
  * (42) D. Bodeker and G. D. Moore, JCAP 05, 009 (2009).
  * (43) C. Caprini et al., JCAP 04, 001 (2016).
  * (44) C. J. Hogan, Mon. Not. R. Astron. Soc. 218, 629–636 (1986).
  * (45) S. J. Huber and T. Konstandin, JCAP 09, 022 (2008).
  * (46) C. Caprini, R. Durrer and G. Servant, JCAP 12, 024 (2009).
  * (47) C. Caprini, R. Durrer, T. Konstandin and G. Servant, Phys. Rev. D 79, 083519 (2009).


# Generator-Retriever-Generator Approach for Open-Domain Question Answering

Abdelrahman Abdallah<EMAIL_ADDRESS>University of
InnsbruckInnrain 52InnsbruckAustria and  Adam Jatowt<EMAIL_ADDRESS>University of InnsbruckInnrain 52InnsbruckAustria

###### Abstract.

Open-domain question answering (QA) tasks usually require the retrieval of
relevant information from a large corpus to generate accurate answers. We
propose a novel approach called Generator-Retriever-Generator (GRG) that
combines document retrieval techniques with a large language model (LLM), by
first prompting the model to generate contextual documents based on a given
question. In parallel, a dual-encoder network retrieves documents that are
relevant to the question from an external corpus. The generated and retrieved
documents are then passed to the second LLM, which generates the final answer.
By combining document retrieval and LLM generation, our approach addresses the
challenges of open-domain QA, such as generating informative and contextually
relevant answers. GRG outperforms the state-of-the-art generate-then-read and
retrieve-then-read pipelines (GENREAD and RFiD) improving their performance by
at least by +5.2, +4.2, and +1.6 on TriviaQA, NQ, and WebQ datasets,
respectively. We provide code, datasets, and
checkpoints111https://github.com/abdoelsayed2016/GRG.

Open-domain question answering, Information retrieval, LLM, Dense Retriever,
Document Generation

††copyright: iw3c2w3g††doi: https://arxiv.org/abs/2307.11278††conference:
arxiv; March 2024; arxiv††journalyear: 2024††booktitle: arxiv

##  1\. Introduction

Figure 1. Simplified diagram illustrating the idea behind the Generator-
Retriever-Generator approach.

Open-domain question answering (QA) tasks pose significant challenges since
they require access to large document collections or repositories of domain-
specific knowledge. Existing methods for QA (Karpukhin et al., 2020; Izacard
and Grave, 2020) often rely on a retrieve-then-read pipeline, where relevant
contextual documents are retrieved from external sources like Wikipedia, and
the answer prediction is conditioned on the retrieved documents and the
question. These methods suffer however from several drawbacks. Firstly, the
retrieved documents are often chunked and of fixed size, which can result in
the inclusion of noisy and irrelevant information. The fixed-size document
chunks may not adequately capture the context necessary for finding accurate
answers (Yu et al., 2022). Consequently, the presence of irrelevant
information can lead to noise in the retrieved documents, negatively impacting
the quality and relevance of the generated answers. Secondly, the
representations of questions and documents in current approaches are typically
obtained independently (Oguz et al., 2020). This independent processing fails
to capture the intricate interactions and dependencies between the question
and the documents. As a result, the model’s understanding of the question and
its ability to extract relevant information from the retrieved documents may
be limited. The shallow interaction between questions and documents hinders
the model’s capability to fully exploit the contextual cues present in the
data, thereby limiting its answer generation accuracy. The limitations on
retriever model parameters and embedding sizes, imposed by the need to
efficiently handle large corpora, restrict the model’s capacity to fully
leverage large language models’ parametric knowledge and its deduction
capabilities. Consequently, the retriever models may struggle to capture the
rich semantic and contextual information necessary for accurate answer
generation (Levine et al., 2022).

On the other hand, open-domain QA often involves training a language model to
generate answers for a given question without access to accompanying documents
containing the answer (Zhu et al., 2021). One promising approach in open-
domain QA is to augment the language model with an external knowledge source,
such as Wikipedia, referred to as evidence documents (Izacard and Grave,
2020). This approach comprises two core components: an information retrieval
system (the retriever) to identify relevant text snippets from the knowledge
source and another system (the reader) to generate answers based on the
retrieved documents and the question.

This paper proposes a novel approach called generator-retriever-generator
(GRG) for open-domain question answering. Our method combines document
retrieval techniques with large language models to address the challenges of
generating informative and contextually relevant answers. We leverage the
power of a large language model such as GPT3 and InstructGPT (Brown et al.,
2020; Ouyang et al., 2022) to generate contextual documents based on a given
question while simultaneously employing a dense passage retrieval system
(Singh et al., 2021; Karpukhin et al., 2020) to retrieve relevant documents
from external sources. A second large language model then processes the
generated and retrieved documents to produce the final answer. By integrating
document retrieval and large language model generation, the proposed GRG
approach aims to improve the accuracy of open-domain question answering. Fig.
1 shows the high-level architecture of the GRG approach.

Our contributions can be summarized as follows:

  1. (1)

GRG Approach: We introduced the GRG approach that combines document generation
and retrieval to improve answer generation in open-domain QA.

  2. (2)

Document Generation & Retrieval Methods: We developed a method using
InstructGPT for generating contextually rich documents. We also proposed the
Vector Index Retriever for efficient retrieval of relevant documents.

  3. (3)

Effectiveness of GRG: We validated the effectiveness of our GRG approach
through extensive experiments and analyses on three open-domain QA datasets.

Table 1. Advantages and Disadvantages of Question Answering Approaches

Approach |  Advantages |  Disadvantages
---|---|---
Retriever-Reader (Karpukhin et al., 2020) |  \- Accesses extensive external knowledge.
\- Provides in-depth, context-based decision-making. |  \- Risk of overlooking relevant information.
\- Dependent on the accuracy and coverage of the retrieval process.
Generator-Reader (Yu et al., 2022) |  \- Capable of generating novel, contextually relevant answers.
\- Reduced dependence on pre-existing documents. |  \- High computational requirements.
\- Possible issues with the reliability and accuracy of generated answers.
Retriever-Generator (Izacard and Grave, 2020; Singh et al., 2021) |  \- Merges accurate retrieval with creative generation.
\- Enhances recall by supplementing existing content. |  \- Increased computational complexity.
\- Balancing quality and diversity of answers can be challenging.
Retriever-Only (Lee et al., 2020) |  \- Directly utilizes a broad range of existing documents.
\- Provides well-grounded, context-based responses. |  \- May miss crucial documents.
\- Limited flexibility in handling complex queries.
Generator-Retriever-Generator |  \- Ensures high relevance and accuracy.
\- Adaptable to a wide range of queries. |  \- Significant computational demands.
\- Balancing diverse and high-quality answers can be difficult.

##  2\. Related Work

We describe in this section related works that fall into 4 known open-domain
QA architectures: Retriever-Reader, Generator-Retriever, Generator-Reader, and
Retriever-only.

###  2.1. Retriever Reader

The Retriever-Reader approach is based on the idea of combining information
retrieval (retriever) and machine reading comprehension (reader) techniques.
Previous work in this area includes the use of document retrieval techniques
such as TF-IDF, BM25, or neural ranking models (Rosa et al., 2021; Qaiser and
Ali, 2018) to select relevant documents from a large corpus. mNotable works
include the Stanford Question Answering Dataset (SQuAD) and subsequent
advancements in retriever-reader architectures like DrQA and BiDAF (Seo et
al., 2018). Dense Passage Retrieval (DPR) (Karpukhin et al., 2020) focuses on
dense representations for passage retrieval, utilizing a dual-encoder
architecture to retrieve passages and a reader model to extract the answer.
T5-RC (Raffel et al., 2020), a variant of the T5 model, follows the Retriever-
Reader approach by retrieving relevant passages and applying T5 as a reader
for answer extraction.

###  2.2. Retriever Generator

The Retriever-Generator (Izacard and Grave, 2020; Singh et al., 2021) approach
aims to leverage both generative modeling and retrieval techniques. Previous
work (Zhu et al., 2021) in this direction has explored methods for retrieving
supporting passages using sparse or dense representations. The retrieved
passages are then used as input into a sequence-to-sequence model, such as a
transformer-based architecture, which generates the answer to the question.
This approach has shown improved performance on benchmark datasets like
TriviaQA (Joshi et al., 2017) and NaturalQuestions (Kwiatkowski et al., 2019).

###  2.3. Generator Reader

The Generator-Reader approach (Yu et al., 2022) focuses on generating
contextual documents based on a question and then using a reader model to
extract the answer from the generated context. The approach involves training
large language models, such as Generative Pre-trained Transformer (GPT)
(Radford et al., 2019), to generate coherent and relevant documents given a
prompt. The generated documents are then processed by a reader component,
which can be a reading comprehension model, to extract the answer. On the
other hand, the DocGen approach, introduced in (Askari et al., 2023), focuses
on generating synthetic documents from queries. The DocGen pipeline involves
expanding and highlighting the original query before generating a synthetic
document likely to be relevant to the query. To enhance the relevance between
generated synthetic documents and their corresponding queries, the authors
propose DocGen-RL. This method treats the estimated relevance of the document
as a reward and uses reinforcement learning (RL) to optimize the DocGen
pipeline.

Figure 2. Architecture diagram illustrating the Generator-Retriever-Generator
(GRG) approach, which combines document retrieval techniques and large
language models to generate contextual documents and retrieve relevant
information for answering questions.

###  2.4. Retriever Only

The Retrieval-Only (Lee et al., 2020) approach seeks to reformulate open-
domain question answering as a phrase retrieval problem, eliminating the need
for processing documents during inference. Previous work has explored
retrieval models that heavily rely on sparse representations, such as TF-IDF
or BM25 (Karpukhin et al., 2020) to retrieve relevant phrases or sentences.
However, these models often underperform compared to retriever-reader
approaches. Recent work has then focused on learning dense representations of
phrases alone, leading to stronger performance in open-domain question
answering. This involves training models using reading comprehension tasks and
employing negative sampling techniques. Seo et al. (2019) proposed a phrase
retrieval approach in which they independently encode the representations of
phrases and questions. They then utilize a similarity search over the encoded
phrase representations to identify the correct answer.

Table 1 presents the advantages and disadvantages of each of the 4 approaches
in question answering systems. Retrieve-Reader leverages external knowledge
and document-based context, but there is a possibility of missing relevant
documents and dependency on retrieval performance. Generate-Reader offers
flexibility and adaptability in generating answers, but it requires
substantial computational power, and the generated answers may not always be
accurate. Retrieve-Generate balances retrieval and generation, enhancing
recall but increasing computational complexity. Retrieve-Only leverages
external knowledge and document-based context, but it has limitations in
handling complex queries and lacks flexibility. Generator-Retriever-Generator
provides contextual relevance, improved accuracy, and adaptability, but it
comes with increased computational complexity and the challenge of balancing
quality and diversity. These considerations play a crucial role in designing
effective question-answering systems.

##  3\. Method

Figure 2 presents an architectural diagram depicting the GRG approach and its
sequential process. It comprises three integral components: (i) a large
language model (LLM) for document generation, (ii) a dual-encoder network for
document retrieval, and (iii) a second large language model for answer
generation. In the following sections, we discuss each component in detail and
outline our training methodology.

###  3.1. Document Generation

Few-shot information extraction tasks aim to recognize novel relations and
extract relevant information from unstructured text with limited annotated
instances (Han et al., 2021). Traditional information extraction methods
struggle with data scarcity and often face challenges in identifying emerging
relation types and their associated entity pairs. To overcome this issue, few-
shot learning techniques leverage a small number of labeled samples to
generalize to unseen instances (Liu et al., 2018).

For our case, generating informative and contextually rich background
documents can be used as a few-shot technique when the power of language
models, particularly, InstructGPT (Ouyang et al., 2022), is harnessed. GRG
then uses InstructGPT to generate context by providing an input prompt. For
few-shot information extraction, a suitable prompt structure could be:
”Generate a background document to answer the given question: [question
placeholder]”. By substituting the ”question placeholder” with the actual
question, we instruct the model to generate a document that contains pertinent
information for answering the question. Utilizing InstructGPT, we generate
informative and contextually rich documents that provide relevant information
for answering a given question. These generated documents are then included in
the collection of evidence documents $\mathcal{D}$.

####  3.1.1. Vector Index Retrieval

We propose a vector-based retrieval (Liu, 2022) method to increase relevance
of knowledge in generated documents using the Vector Index Retriever (Xiao et
al., 2022). This approach leverages vector representations and the  Vector
Store Index222https://github.com/jerryjliu/llama_index to efficiently retrieve
documents based on their similarity to the input question. The Vector Index
Retriever is crucial to our information retrieval pipeline. It utilizes the
Vector Store Index, which stores vector representations of documents generated
by a large language model. We capture each document’s semantic and contextual
information by encoding each document with a high-dimensional vector. In the
retrieval process, the Vector Index Retriever employs a similarity-based
approach to identify the most relevant documents. Given a question, it
retrieves a pre-specified number of top k results with the highest similarity
scores. The k parameter can be adjusted to balance the precision and
efficiency. We describe the details of each step below.

Step 1: Generate Documents. We first generate 10 to 50 contextual documents
$D_{G}$ for each question $q\in\mathcal{Q}$ using InstructGPT. Here,
$\mathcal{Q}$ represents the set of questions in the dataset.

Step 2: Encode each Document. Using GTR-T5-large/MiniLM-L6 (Reimers and
Gurevych, 2019; Ni et al., 2021) language model, we encode each document
$d_{i}$, resulting in a 768/384-dimensional vector $\textbf{e}_{i}$ per
document.

Step 3: Vector Index Representation. We store all the embedding vectors
$\\{\textbf{e}_{i}\\}_{i=1}^{|Q|}$ using the Vector Store Index. This allows
for efficient retrieval of documents based on their similarity to the
question.

Step 4: Selection of Generated Documents. After storing the encoded documents,
we utilize the Vector Index Retriever to process the question and select up to
top k (2 or 5 in our experiments) the most relevant documents with a high
cosine similarity score thresholdThe cosine similarity score is calculated
between the encoded question vector and the vectors of the stored documents:

$\text{{Cosine Similarity
Score}}(\mathbf{q},\mathbf{d}_{i})=\frac{{\mathbf{q}\cdot\mathbf{d}_{i}}}{{\|\mathbf{q}\|\cdot\|\mathbf{d}_{i}\|}}$

where $\mathbf{q}$ represents the encoded question vector and $\mathbf{d}_{i}$
represents the vector of the $i$-th stored document.

By comparing the cosine similarity scores of the question vector with the
vectors of the stored documents, we can identify the most relevant documents
that have high similarity to the question. In this case, we retrieve the top 5
documents with similarity above the specified threshold of 0.7. By following
these steps, our approach enables effective retrieval of generated contextual
documents for open-domain question-answering, specifically selecting documents
with high similarity to the question and, thus ones that are likely to contain
the correct answer. This retrieval process leverages vector representations
and similarity-based techniques to prioritize the most relevant and
informative documents.

###  3.2. Document Retriever

The retriever module plays a crucial role in our question-answering model.
Given a collection of evidence documents
$\mathcal{D_{R}}=\\{\boldsymbol{d}_{1},\ldots,\boldsymbol{d}_{M}\\}$ and a
question $\boldsymbol{q}$, its goal is to select a subset of the documents
$\mathcal{Z}\subset\mathcal{D_{R}}$ that are most relevant to the question.
This subset of documents will be used for further processing and answer
generation. For this, our retriever model is based on EMDR (End-to-end
training of Multi-Document Reader and Retriever) (Singh et al., 2021), which
is a dual-encoder network (Vaswani et al., 2017) consisting of two separate
encoders: $f_{q}$ for encoding the question and $f_{d}$ for encoding the
evidence documents. Each encoder takes a sequence (question or document) as
input and produces its fixed-size vector representation. To quantify the
relevance or similarity between a question $q$ and an evidence document
$d_{i}$, we compute their respective encoded vectors using the encoders
$f_{q}$ and $f_{d}$. The retrieval score is then determined by taking the dot
product between these vectors:

(1)
$\displaystyle\text{score}(q,d_{i};\Phi)=\text{enc}(q;\Phi_{q})\cdot\text{enc}(d_{i};\Phi_{d})$

Where $\text{enc}(q;\Phi_{q})$ and $\text{enc}(d_{i};\Phi_{d})$ represent the
encoded vectors of the question and document, respectively, with $\Phi$
denoting the retriever parameters. By calculating the dot product, we capture
the similarity between the question and document, with higher scores
indicating stronger relevance. Based on the retrieval scores, we select the
top-$k$ documents from the collection $\mathcal{D_{R}}$ for a given question
$q$ which are indicated as $\mathcal{Z}={z_{1},\ldots,z_{k}}$.

###  3.3. Generation Model

Our generator is based on a model from the LLaMA family - a collection of
open-source language models pretrained on trillions of tokens using publicly
available datasets, which achieve state-of-the-art performance on many
benchmarks. The generator model takes as input a question $q$ and a set of
retrieved and generated documents to generate an answer.

Each retrieved document $z_{i}$ and generated document $d_{i}$ are
concatenated with the question. We use the newline character (\n) as a
delimiter to ensure separation between the documents. Additionally, we include
the </s> token at the end of each utterance as an end-of-turn token, which
indicates the completion of each input segment.

The input to our generator model is then represented as follows:

$\text{{input}}=[q,z_{im},\text{{\textbackslash
n}},d_{im},\text{{\textbackslash n}},\texttt{</s>}]$

The LLaMA language model uses a novel loss function called cosine loss that
helps the model to better distinguish between similar words and improve its
accuracy. The cosine loss is defined as follows:

$\mathcal{L}_{cos}=-\frac{1}{N}\sum_{i=1}^{N}\log\frac{\exp(\cos(\mathbf{h}_{i},\mathbf{t}_{i})/\tau)}{\sum_{j=1}^{N}\exp(\cos(\mathbf{h}_{i},\mathbf{t}_{j})/\tau)}$

where $\mathbf{h}_{i}$ is the hidden state of the $i$-th token in the sequence
and $\mathbf{t}_{i}$ is the target embedding for that token. $\tau$ is a
temperature parameter that controls the sharpness of the distribution.

By incorporating the question, retrieved documents, and generated documents,
our generator model can generate contextually informed answers tailored to the
specific question and the available input information.

##  4\. Experimental Settings

###  4.1. Datasets

The evaluation is conducted on several datasets, following the same
experimental setup as in (Yu et al., 2022; Izacard and Grave, 2020; Lee et
al., 2019). We consider the following datasets:

  * •

NaturalQuestions (Kwiatkowski et al., 2019): This dataset consists of
questions corresponding to Google search queries. Natural Questions (NQ)333NQ
(Retriever): https://ai.google.com/research/NaturalQuestions/download and NQ
(Generator):
https://drive.google.com/drive/folders/1DNjTTOLKi24wohJKu1Z-v6b4izfymlLu was
generated from real Google search queries, and the answers are spans within
Wikipedia articles. The NQ dataset consists of around 79,168 examples in the
training set, 8,757 examples in the development set, and 3,610 examples in the
test set.

  * •

TriviaQA (Joshi et al., 2017): This dataset contains questions collected from
trivia and quiz-league websites. For open-domain question answering, we use
the unfiltered version of the dataset. TriviaQA444TQA (Retriever):
http://nlp.cs.washington.edu/triviaqa/ and TQA (Generator):
https://drive.google.com/drive/folders/1DNjTTOLKi24wohJKu1Z-v6b4izfymlLu is a
collection of trivia questions sourced from trivia and quiz-league websites.
The dataset includes 78,785 examples in the training set, 8,837 examples in
the development set, and 11,313 examples in the test set.

  * •

WebQ (Berant et al., 2013): WebQuestions (WebQ) 555 WebQ (Retriever):
https://github.com/google-research/language/tree/master/language/orqa and WebQ
(Generator):
https://drive.google.com/drive/folders/1DNjTTOLKi24wohJKu1Z-v6b4izfymlLu
consists of questions obtained using the Google Suggest API, with the answers
being entities from Freebase. The dataset contains approximately 3,417
examples in the training set, 361 examples in the development set, and 2,032
examples in the test set.

To evaluate the performance of our model, we employ the exact match (EM)
score, following Zhu et al. (2021); Yang et al. (2019); Chen et al. (2020).
The EM score measures the correctness of an answer by comparing its normalized
form to the acceptable answer list. Through these evaluations, we aim to
assess the effectiveness of the GRG model in the domain of open-domain
question answering.

We adopt the train/dev/test splits that have been previously used in the open-
domain QA setting, as employed by Izacard and Grave (2020) and Karpukhin et
al. (2020). Table 2 presents the statistics of the dataset sizes, including
the training, development, and test sets. We note that all our models are
trained exclusively on the training data, and we did not include the
development data in our training process. Therefore, the performance numbers
reported in the paper for the dev and test data are independent of the
training data. We split the training data, allocating 90% for model training
and the remaining 10% for testing purposes.

Table 2. Datasets’ statistics. Dataset | Train | Dev | Test
---|---|---|---
WebQ |  03,417 |  00361 |  02,032
NQ | 79,168 | 8,757 |  03,610
TQA | 78,785 | 8,837 | 11,313
Table 3. Training and Hyperparameter Settings for LLaMa-7B Parameter | Value | Parameter | Value
---|---|---|---
Attention heads | 32 | Optimizer | AdamW
n layers | 32 | beta1 | 0.9
dimension | 4096 | beta2 | 0.999
Hardware | A100 and A40 | epsilon | 1e-08
Batch Size | 4 | gradient accumulation steps | 8
CPU | 100 | learning rate | 2e-05
weight decay | 0.0 | max grad norm | 1.0
train batch size | 4 | eval batch size | 4
warmup ratio | 0.03 | Warm-up Steps | 2,000

###  4.2. Choice of Document Number

In our approach, we used only 2 or 5 documents during the generator process
due to computational limitations and the extensive training time required for
the LLaMA model. As Izacard and Grave (2020) reported, training the T5 model
using 100 documents necessitates considerable computational resources, such as
64 Tesla V100 32GB GPUs running for approximately one day. While increasing
the number of documents can enhance model performance (Izacard and Grave,
2020), it incurs significant costs regarding memory consumption and training
time, which should be carefully considered, especially, in the current trend
towards GreenAI (Yu et al., 2022).

###  4.3. Experimental Setup

In this section, we describe the experimental setup for training the LLaMA
model using the DeepSpeed framework (Rajbhandari et al., 2020). DeepSpeed
provides techniques and automated parameter tuning to optimize training
efficiency and memory utilization. We customized the training process using
DeepSpeed’s configuration options. Firstly, we enabled mixed precision
training with bfloat16 (bf16) precision to accelerate training while
maintaining accuracy. The AdamW optimizer was selected, and its
hyperparameters were determined automatically by DeepSpeed. To control the
learning rate, we employed the WarmupDecayLR scheduler. The LLaMA model is
based on the transformer architecture (Vaswani et al., 2017) widely used in
large language models. We utilize the LLaMa-7B model as our backbone for
implementing GRG. The training and hyperparameter settings for LLaMa-7B are
summarized in Table 3.

For memory consumption and speed optimization, we utilized DeepSpeed’s zero
optimization stage 3, offloading the optimizer state and model parameters to
the CPU with pinned memory. Additional hyperparameters were set, including
gradient accumulation steps (8 steps), gradient clipping (determined
automatically), and batch size (value of 4). This experimental setup aimed to
achieve efficient training and optimal performance of our LLaMA model.

In addition to the DeepSpeed experimental setup described above, we conducted
an additional experiment using the LoRA technique (Hu et al., 2021) for fine-
tuning our LLaMA model. LoRA, which stands for ”Low-Overhead Representation
Adaptation,” is a method that allows for the efficient fine-tuning of large
language models. For this experiment, we followed a slightly different
approach. Instead of recreating the entire model from scratch, we generated a
fine-tuning file that would be applied to the base Llama model. This approach
significantly reduces computational overhead and makes the fine-tuning process
more efficient, even on modest hardware.

Our proposed model and relevant baselines are implemented using PyTorch
(Paszke et al., 2019) on a cluster of machines equipped with 100 CPUs, 400GB
of physical memory, and a combination of 4 A40 and 4 A100 GPUs for our
experiments.

Table 4. Performance Comparison of GRG Approach and Baseline Models on
TriviaQA, WebQ, and NQ Datasets.

Models # reader # docu- TriviaQA WebQ NQ parameters ments dev test dev test
dev test *baselines with retrieving from Wikipedia; all numbers reported by
existing papers BM25 + BERT (Lee et al., 2019) 220M 5 47.2 47.1 27.1 21.3 24.8
26.5 REALM (Guu et al., 2020) 330M 5 - - - 40.7 38.2 40.4 DPR (Karpukhin et
al., 2020) 110M 100 - 56.8 - 41.1 - 41.5 RAG (Lewis et al., 2020) 400M 10 -
56.1 - 45.2 - 44.5 FiD-l (Yu et al., 2022) 770M 10 - 61.9 - 48.1 - 46.7 FiD-xl
(Yu et al., 2022) 3B 10 - 66.3 - 50.8 - 50.1 FiD-xl (Yu et al., 2022) 3B 10 -
70.1 - 53.6 - 45.0 FiD (Izacard and Grave, 2020) 770M 100 - 67.6 - 50.5 - 51.4
EMDR (Singh et al., 2021) 440M 50 71.1 71.4 49.9 48.7 50.4 52.5 RFiD-large
(Wang et al., 2023) 990M 100 72.7 72.6 - - 52.5 54.3 *baselines with phrase
retrieval; all numbers reported by existing papers DensePhrases (Lee et al.,
2020) 110M 50 - 34.4 - 17.3 - 14.5 DensePhrases (Lee et al., 2021) 110M 50 -
53.5 - 41.5 - 41.3 *baselines with generated documents; all numbers reported
by existing papers GenRead (FiD-l) (Yu et al., 2022) 770M 10 - 67.8 - 51.5 -
40.3 GenRead (FiD-l) (Yu et al., 2022) 770M 10 - 70.2 - 53.3 - 43.5 GenRead
(FiD-xl) (Yu et al., 2022) 3B 10 - 69.6 - 52.6 - 42.6 GenRead (FiD-xl) (Yu et
al., 2022) 3B 10 - 71.6 - 54.4 - 45.6 *baselines with generated and retrieved
documents COMBO (Zhang et al., 2023) 3B 2 - 74.6 - 54.2 - 53.0 *our proposed
method by combining generated and retrieved documents GRG (LoRA) 1.2B 2 67.6
69.1 48.6 45.2 50.8 49.1 GRG (LoRA) 1.2B 5 69.4 70.8 50.6 42.9 54.8 53.4 GRG
7B 2 76.4 75.7 52.0 53.6 55.4 57.4 GRG 7B 5 77.1 76.8 55.8 56.0 56.2 58.5

Table 5. Recall@K scores for document retrieval using our approach equipped with GTR-T5-large and MiniLM-L6 models on TQA, NQ, and WebQ datasets. Models | TQA | NQ | WebQ
---|---|---|---
dev | test | dev | test | dev | test
MiniLM-L6 | 76.1 | 76.7 | 58.6 | 60.3 | 67.0 | 60.1
GTR-T5 | 78.5 | 79.2 | 62.2 | 63.9 | 72.6 | 68.1

##  5\. Results

We present in this section the experimental results, which are divided into
three subsections: Results of Open-Domain QA, Results of document generation,
and the Ablation study. The document generation analysis aims to evaluate the
effectiveness of our document retrieval method in generating relevant and
informative documents for answering open-domain questions. In the ablation
study, we investigate the impact of different factors (top-k answers,
architecture components, and zero-shot strategy) on the performance.

###  5.1. Results of Open-Domain QA

This section presents the results of the proposed GRG approach, which combines
generated and retrieved documents for question answering. The results of the
experiments are shown in Table 4 using EM score. We compare the performance of
GRG against several baselines and existing state-of-the-art models on three
benchmark datasets: TriviaQA, WebQ, and NQ. We first compare GRG against
baseline models that utilize document retrieval from Wikipedia. These
baselines include BM25 + BERT (Lee et al., 2019), REALM (Guu et al., 2020),
DPR (Karpukhin et al., 2020), RAG (Lewis et al., 2020), FiD-l (Yu et al.,
2022), FiD-xl (Yu et al., 2022), FiD (Izacard and Grave, 2020), EMDR (Singh et
al., 2021), DensePhrases models (Lee et al., 2020, 2021), and RFiD-large (Wang
et al., 2023). The numbers reported for these baselines are taken directly
from their respective papers. GRG consistently outperforms most of the
baseline models across all datasets. Specifically, GRG achieves significant
improvements over BM25 + BERT (29.9% improvement on TriviaQA dev set) and
(29.7% improvement on TriviaQA test set), REALM (15.3% improvement on WebQ
test set), DPR (14.9% improvement on WebQ test set), FiD (7.1% improvement on
NQ test set), and RAG (14.0% improvement on NQ test set), demonstrating the
effectiveness of the combined generated and retrieved documents’ based
approach. Next, we compare GRG against DensePhrases models (Lee et al., 2020,
2021) that employ phrase retrieval. DensePhrases has been shown to perform
well in question-answering tasks. However, GRG approach surpasses the
performance of DensePhrases across all datasets. On TriviaQA dev set, GRG
achieves a 23.3% improvement over DensePhrases (Lee et al., 2020), and on WebQ
test set, it has an 14.5% improvement over DensePhrases (Lee et al., 2021).

Next, we evaluate the performance of GRG against GenRead (Yu et al., 2022)
models that only generate documents. GenRead models have shown promising
results in generating informative documents. Still, our approach consistently
outperforms GenRead regarding question answering accuracy on all the datasets.
On TriviaQA dev set, GRG achieves a 7.3% improvement over GenRead (FiD-l), and
on WebQ test set, it has a 2.1% improvement over GenRead (FiD-l).

Finally, we discuss the performance of GRG with varying configurations. We
evaluate GRG with two numbers of generated documents (2 and 5) using LoRA.
Additionally, we report the performance of GRG without LoRA, utilizing the
same number of generated documents. On the TriviaQA dev set, GRG achieved
76.4% accuracy when using 2 generated documents, which rose to 77.1% for the
case of 5 generated documents. The performance on the WebQ test set of the
model is 52.0% accuracy with 2 generated documents, increasing to 55.8% with 5
generated documents. Lastly, on the NQ test set, the model achieved an
accuracy of 55.4% with 2 generated documents and showed a slight improvement
to 56.2% when 5 generated documents were utilized. GRG outperforms all of the
baselines on all three datasets. When applied on TriviaQA, GRG achieves an
exact match score of 76.8, which is a +5.2 improvement over the previous
state-of-the-art (GenRead). Testing on On WebQ, we see that our model reaches
an exact match score of 56.0, which is a +1.6 improvement over the previous
state-of-the-art (RFiD-large). On the last dataset, NQ, GRG achieves an exact
match score of 58.5, which is a +4.2 improvement over the previous state-of-
the-art (GenRead). we also compare our GRG approach with the COMBO model,
which also utilizes a blend of generated and retrieved documents, and has
shown notable performance, particularly on the TriviaQA and WebQ datasets.
Notably, it achieves an exact match score of 74.6 on the TriviaQA test set and
54.2 on the WebQ test set, representing state-of-the-art performance on these
datasets. However, when compared to GRG, our approach still shows superior
results. Specifically, GRG achieves a higher exact match score of 76.8 on
TriviaQA and 56.0 on WebQ, surpassing COMBO by 2.2 and 1.8 points,
respectively. This suggests that while COMBO’s strategy is effective, the
methodologies employed in GRG allow for even more precise question-answering
capabilities.

Our results demonstrate that GRG performs better than all the baselines and
state-of-the-art models across all the datasets. Including generated and
retrieved documents enables GRG to capture a wider range of relevant
information, improving QA accuracy. Notably, GRG with 5 generated documents
consistently outperforms GRG with 2 generated documents, suggesting the
benefit of incorporating more diverse generated content.

###  5.2. Evaluating Document Generation

In this section, we present the experimental results of our document retrieval
approach for document generation using the GTR-T5-large and MiniLM-L6 models.
We computed the Recall@K of retrieving the documents containing the true
answer for each question same as in (Sachan et al., 2022). To ensure a fair
comparison and consistent evaluation, we utilized the same dataset as in (Yu
et al., 2022). The choice of using the same dataset was motivated by the fact
that the generated context from the InstructGPT model may significantly differ
for every request. We measured the Recall@K of our document retrieval method
by calculating the percentage of questions for which the retrieved document
contained the true answer. These accuracy results highlight the effectiveness
of our vector index retrieval approach in identifying relevant documents for
answering open-domain questions. GTR-T5-large model, with its higher-
dimensional vector encoding, exhibits better performance compared to the
MiniLM-L6 model and the approach proposed by Yu et al. (2022). Table 5
presents Recall@K scores for three question answering datasets: TQA, NQ, and
WebQ. The MiniLM-L6 model achieves scores ranging from 58.6% to 76.7% across
the datasets, while the GTR-T5-large model outperforms it with scores ranging
from 62.2% to 79.2% for the respective datasets.

##  6\. Ablation studies

###  6.1. Zero-Shot Open-Domain QA

Table 6 shows the results of a zero-shot open-domain question answering (QA)
evaluation, where different models are assessed without any external
documents. These models, including FLAN, GLaM, Chinchilla, Gopher,
InstructGPT, GPT-3, and LLaMA (Rae et al., 2021; Wei et al., 2021; Du et al.,
2022; Roberts et al., 2020; Ouyang et al., 2022; Touvron et al., 2023),
possess varying parameter sizes and have been trained on large-scale corpora,
enabling them to capture extensive world knowledge. When examining the
performance of each model on answering questions from the TQA, NQ, and WebQ
datasets, we observe notable variations. LLaMA, with its 7B parameters, stands
out by achieving remarkable results in zero-shot QA. Despite the relatively
smaller parameter size, LLaMA demonstrates the ability to effectively leverage
the knowledge embedded within its parameters, showcasing its potential as a
powerful tool for zero-shot question answering tasks. Models like InstructGPT
and GPT-3, with larger parameter sizes (175B), also demonstrate competitive
performance. InstructGPT achieves a high accuracy of 57.4% on the TQA dataset
and performs consistently well across the other datasets. GPT-3 also achieves
competitive results.

###  6.2. Impact of Architecture Components

We now evaluate the performance of each component used in our approach,
specifically the retriever and the generator, when combined with LLaMA. The
goal is to understand the individual contributions of these components on the
overall performance. We compare the results on the TQA and NQ datasets using
different combinations of models. Figure 3 shows the performance comparison of
DPR+LLaMA and InstructGPT+LLaMA models on TQA and NQ datasets.

On the TQA dataset, the InstructGPT+LLaMA model demonstrated an EM score of
67.1% and 70.1% on the development and test sets, respectively, when trained
with 2 documents. Upon using 5 documents for training, the performance
improved to 68.4% and 71.8% on the development and test sets, respectively.
Shifting the focus to the NQ dataset, the InstructGPT+LLaMA model showed
competitive performance, achieving an EM score of 42.1% on the development set
and 42.0% on the test set with 2 documents. Increasing the number of training
documents to 5 resulted in a modest improvement, with EM scores of 43.6% on
the development set and 44.5% on the test set. These findings indicate that
incorporating more documents during training can positively impact model
performance. There may be however a diminishing return in terms of accuracy
improvement. As a result, striking a careful balance between the number of
training documents and the resulting performance may be crucial to optimize
computational resources and training time.

Figure 3. Performance Comparison (EM) of DPR+LLaMA and InstructGPT+LLaMA models on TQA and NQ.  Table 6.  Comparative Performance of Language Models in Zero-Shot Open-Domain QA. Models | parameters | TQA | NQ | WebQ
---|---|---|---|---
FLAN | 137B | 56.7 | 20.7 | -
GLaM | 64B | - | 21.5 | 19.0
Chinchilla | 70B | 55.4 | 16.6 | -
PaLM | 540B | - | 21.2 | 10.9
Gopher | 280B | 43.5 | 10.1 | 35.6
InstructGPT | 175B | 57.4 | 19.5 | 19.9
GPT-3 | 175B | 49.2 | 14.6 | 14.4
LLaMA | 7B | 50.0 | 16.8 | 28.8

###  6.3. Comparative Analysis GRG (LoRA) Models

We present additional experimental results for document generation using the
GRG (LoRA) and GRG models. The performance of these models is evaluated on the
TQA and NQ datasets, and the results are summarized in Table 7.

Table 7. F1 scores for document generation using GRG and GRG (LoRA) models. Models | # docu- | TQA | NQ
---|---|---|---
ments | dev | test | dev | test
GRG (LoRA) | 2 | 75.6 | 78.8 | 60.4 | 59.5
GRG (LoRA) | 5 | 79.7 | 80.4 | 63.9 | 61.7
GRG | 2 | 84.0 | 83.8 | 64.6 | 65.0
GRG | 5 | 84.6 | 84.7 | 65.4 | 66.1

Table 7 displays the F1 scores obtained by the GRG (LoRA) and GRG models when
generating documents for the TQA and NQ datasets. The models are evaluated on
both the development and test sets.

For GRG (LoRA) model, the results indicate that increasing the number of
documents from 2 to 5 leads to improved performance on both datasets. On the
TQA dataset, the F1 score increases from 75.6 to 79.7 on the development set
and from 78.8 to 80.4 on the test set when moving from 2 to 5 documents.
Similarly, on the NQ dataset, the F1 score improves from 60.4 to 63.9 on the
development set and from 0.595 to 0.6173 on the test set.

The GRG model also demonstrates competitive performance in document
generation. With 2 documents, the model achieves an F1 score of 84.05 on the
TQA development set and 83.8 on the test set. On the NQ dataset, the F1 score
is 64.6 on the development set and 65.0 on the test set. Increasing the number
of documents to 5 further enhances the performance, with F1 scores of 84.6 and
84.7 on the TQA development and test sets, respectively, and 65.4 and 66.1 on
the NQ development and test sets, respectively.

Table 8. Performance comparison of DPR and InstructGPT models on the TQA dataset. Models | Development Set | Test Set
---|---|---
2 Docs | 5 Docs | 2 Docs | 5 Docs
DPR+LLaMA | 66.0% | 69.4% | 66.8% | 69.3%
InstructGPT+LLaMA | 67.1% | 68.4% | 70.1% | 71.8%
Table 9. Performance comparison of DPR and InstructGPT models on the NQ dataset. Models | Development Set | Test Set
---|---|---
2 Docs | 5 Docs | 2 Docs | 5 Docs
DPR+LLaMA | 41.7% | 42.6% | 41.2% | 42.6%
InstructGPT+LLaMA | 42.1% | 43.6% | 42.0% | 44.5%
Table 10. Performance Comparison (EM and F1) Scores of GRG for different top-k
values on NQ and TQA datasets

Top-k NQ TQA NQ TQA Dev EM Test EM Dev EM Test EM Dev F1 Test F1 Dev F1 Test
F1 1 56.2 58.5 77.1 76.8 65.4 66.1 84.6 84.7 2 66.1 67.3 79.9 80.0 72.9 73.3
86.6 86.7 3 68.8 70.3 81.3 81.4 75.3 75.9 87.6 87.8 4 70.6 71.9 82.1 82.1 76.8
77.2 88.3 88.3 5 71.6 72.8 82.6 82.6 77.7 78.4 88.7 88.8

###  6.4. Impact of top-k Answer on Performance

We finally analyze the impact of different top-k values on the performance of
our proposed approach. Table 10 presents the EM and F1 scores for different
top-k values on NQ and TQA datasets. We observe that as the top-k value
increases, the EM scores consistently improve. For example, on the NQ dataset,
the EM score increases from 56.3% at top-1 to 71.6% at top-5. Similarly, on
TQA, the EM score increases from 76.2% at top-1 to 82.6% at top-5.

##  7\. Limitations

This study acknowledges the following potential limitations:

  1. (1)

Generated Document Quality: The performance of our approach depends on the
accuracy and relevance of the documents generated by the language model.
Despite extensive training, there can be instances of inaccurate or irrelevant
information.

  2. (2)

Large language models can be computationally intensive and time-consuming,
especially for complex queries. This can pose scalability challenges when
processing a large number of queries or with limited computing resources. More
details are in Appendix A.1.

##  8\. Conclusions

In this paper, we proposed a Generator-Retriever-Generator approach for
improving open-domain question answering systems. By combining generated and
retrieved documents, we achieved significant performance gains across multiple
benchmark datasets. Our experiments demonstrate that GRG outperforms existing
baselines in terms of accuracy and efficiency. The results indicate also the
effectiveness of incorporating both generated and retrieved documents in the
reading process, leveraging the combined strengths of language models and
retrieval systems.

Future work should focus on improving the accuracy of the document retrieval
approach, potentially through the use of more advanced retrieval models or by
incorporating additional contextual information. Further, more extensive
investigations into hyperparameter configurations, such as the number of
generated and retrieved documents will also be done.

## References

  * (1)
  * Askari et al. (2023) Arian Askari, Mohammad Aliannejadi, Chuan Meng, Evangelos Kanoulas, and Suzan Verberne. 2023.  Expand, Highlight, Generate: RL-driven Document Generation for Passage Reranking. In _Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing_ , Houda Bouamor, Juan Pino, and Kalika Bali (Eds.). Association for Computational Linguistics, Singapore, 10087–10099.  https://doi.org/10.18653/v1/2023.emnlp-main.623
  * Berant et al. (2013) Jonathan Berant, Andrew Chou, Roy Frostig, and Percy Liang. 2013.  Semantic parsing on freebase from question-answer pairs. In _Proceedings of the 2013 conference on empirical methods in natural language processing_. 1533–1544.
  * Brown et al. (2020) Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020\.  Language models are few-shot learners.  _Advances in neural information processing systems_ 33 (2020), 1877–1901.
  * Chen et al. (2020) Wenhu Chen, Ming-Wei Chang, Eva Schlinger, William Wang, and William W Cohen. 2020.  Open question answering over tables and text.  _arXiv preprint arXiv:2010.10439_ (2020).
  * Du et al. (2022) Nan Du, Yanping Huang, Andrew M Dai, Simon Tong, Dmitry Lepikhin, Yuanzhong Xu, Maxim Krikun, Yanqi Zhou, Adams Wei Yu, Orhan Firat, et al. 2022\.  Glam: Efficient scaling of language models with mixture-of-experts. In _International Conference on Machine Learning_. PMLR, 5547–5569.
  * Guu et al. (2020) Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Mingwei Chang. 2020.  Retrieval augmented language model pre-training. In _International conference on machine learning_. PMLR, 3929–3938.
  * Han et al. (2021) Jiale Han, Bo Cheng, and Wei Lu. 2021.  Exploring Task Difficulty for Few-Shot Relation Extraction. In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_. Association for Computational Linguistics, Online and Punta Cana, Dominican Republic, 2605–2616.  https://doi.org/10.18653/v1/2021.emnlp-main.204
  * Hu et al. (2021) Edward Hu, Yelong Shen, Phil Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Lu Wang, and Weizhu Chen. 2021.  LoRA: Low-Rank Adaptation of Large Language Models.  arXiv:2106.09685 [cs.CL]
  * Hunger (2005) Raphael Hunger. 2005.  _Floating point operations in matrix-vector calculus_. Vol. 2019.  Munich University of Technology, Inst. for Circuit Theory and Signal ….
  * Izacard and Grave (2020) Gautier Izacard and Edouard Grave. 2020.  Leveraging passage retrieval with generative models for open domain question answering.  _arXiv preprint arXiv:2007.01282_ (2020).
  * Joshi et al. (2017) Mandar Joshi, Eunsol Choi, Daniel S Weld, and Luke Zettlemoyer. 2017.  Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension.  _arXiv preprint arXiv:1705.03551_ (2017).
  * Kaplan et al. (2020) Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. 2020.  Scaling laws for neural language models.  _arXiv preprint arXiv:2001.08361_ (2020).
  * Karpukhin et al. (2020) Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020.  Dense Passage Retrieval for Open-Domain Question Answering. In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_. 6769–6781.
  * Kwiatkowski et al. (2019) Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, et al. 2019\.  Natural questions: a benchmark for question answering research.  _Transactions of the Association for Computational Linguistics_ 7 (2019), 453–466.
  * Lee et al. (2020) Jinhyuk Lee, Mujeen Sung, Jaewoo Kang, and Danqi Chen. 2020.  Learning dense representations of phrases at scale.  _arXiv preprint arXiv:2012.12624_ (2020).
  * Lee et al. (2021) Jinhyuk Lee, Mujeen Sung, Jaewoo Kang, and Danqi Chen. 2021.  Learning Dense Representations of Phrases at Scale.  arXiv:2012.12624 [cs.CL]
  * Lee et al. (2019) Kenton Lee, Ming-Wei Chang, and Kristina Toutanova. 2019.  Latent retrieval for weakly supervised open domain question answering.  _arXiv preprint arXiv:1906.00300_ (2019).
  * Levine et al. (2022) Yoav Levine, Itay Dalmedigos, Ori Ram, Yoel Zeldes, Daniel Jannai, Dor Muhlgay, Yoni Osin, Opher Lieber, Barak Lenz, Shai Shalev-Shwartz, et al. 2022\.  Standing on the shoulders of giant frozen language models.  _arXiv preprint arXiv:2204.10019_ (2022).
  * Lewis et al. (2020) Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al. 2020\.  Retrieval-augmented generation for knowledge-intensive nlp tasks.  _Advances in Neural Information Processing Systems_ 33 (2020), 9459–9474.
  * Liu (2022) Jerry Liu. 2022.  _LlamaIndex_.  https://doi.org/10.5281/zenodo.1234
  * Liu et al. (2018) Yanbin Liu, Juho Lee, Minseop Park, Saehoon Kim, Eunho Yang, Sung Ju Hwang, and Yi Yang. 2018.  Learning to propagate labels: Transductive propagation network for few-shot learning.  _arXiv preprint arXiv:1805.10002_ (2018).
  * Ni et al. (2021) Jianmo Ni, Chen Qu, Jing Lu, Zhuyun Dai, Gustavo Hernández Ábrego, Ji Ma, Vincent Y Zhao, Yi Luan, Keith B Hall, Ming-Wei Chang, et al. 2021\.  Large dual encoders are generalizable retrievers.  _arXiv preprint arXiv:2112.07899_ (2021).
  * Oguz et al. (2020) Barlas Oguz, Xilun Chen, Vladimir Karpukhin, Stan Peshterliev, Dmytro Okhonko, Michael Schlichtkrull, Sonal Gupta, Yashar Mehdad, and Scott Yih. 2020.  Unik-qa: Unified representations of structured and unstructured knowledge for open-domain question answering.  _arXiv preprint arXiv:2012.14610_ (2020).
  * Ouyang et al. (2022) Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022\.  Training language models to follow instructions with human feedback.  _Advances in Neural Information Processing Systems_ 35 (2022), 27730–27744.
  * Paszke et al. (2019) Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. 2019\.  Pytorch: An imperative style, high-performance deep learning library.  _Advances in neural information processing systems_ 32 (2019).
  * Qaiser and Ali (2018) Shahzad Qaiser and Ramsha Ali. 2018.  Text Mining: Use of TF-IDF to Examine the Relevance of Words to Documents.  _International Journal of Computer Applications_ 181 (07 2018).  https://doi.org/10.5120/ijca2018917395
  * Radford et al. (2019) Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019\.  Language models are unsupervised multitask learners.  _OpenAI blog_ 1, 8 (2019), 9.
  * Rae et al. (2021) Jack W Rae, Sebastian Borgeaud, Trevor Cai, Katie Millican, Jordan Hoffmann, Francis Song, John Aslanides, Sarah Henderson, Roman Ring, Susannah Young, et al. 2021\.  Scaling language models: Methods, analysis & insights from training gopher.  _arXiv preprint arXiv:2112.11446_ (2021).
  * Raffel et al. (2020) Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2020.  Exploring the limits of transfer learning with a unified text-to-text transformer.  _The Journal of Machine Learning Research_ 21, 1 (2020), 5485–5551.
  * Rajbhandari et al. (2020) Samyam Rajbhandari, Jeff Rasley, Olatunji Ruwase, and Yuxiong He. 2020.  ZeRO: Memory Optimizations toward Training Trillion Parameter Models. In _Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis_ (Atlanta, Georgia) _(SC ’20)_. IEEE Press, Article 20, 16 pages.
  * Reimers and Gurevych (2019) Nils Reimers and Iryna Gurevych. 2019.  Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks.  arXiv:1908.10084 [cs.CL]
  * Roberts et al. (2020) Adam Roberts, Colin Raffel, and Noam Shazeer. 2020.  How much knowledge can you pack into the parameters of a language model?  _arXiv preprint arXiv:2002.08910_ (2020).
  * Rosa et al. (2021) Guilherme Moraes Rosa, Ruan Chaves Rodrigues, Roberto Lotufo, and Rodrigo Nogueira. 2021.  Yes, BM25 is a Strong Baseline for Legal Case Retrieval.  arXiv:2105.05686 [cs.IR]
  * Sachan et al. (2022) Devendra Singh Sachan, Mike Lewis, Mandar Joshi, Armen Aghajanyan, Wen-tau Yih, Joelle Pineau, and Luke Zettlemoyer. 2022.  Improving Passage Retrieval with Zero-Shot Question Generation.  (2022).  https://arxiv.org/abs/2204.07496
  * Seo et al. (2018) Minjoon Seo, Aniruddha Kembhavi, Ali Farhadi, and Hannaneh Hajishirzi. 2018.  Bidirectional Attention Flow for Machine Comprehension.  arXiv:1611.01603 [cs.CL]
  * Seo et al. (2019) Minjoon Seo, Jinhyuk Lee, Tom Kwiatkowski, Ankur P Parikh, Ali Farhadi, and Hannaneh Hajishirzi. 2019.  Real-time open-domain question answering with dense-sparse phrase index.  _arXiv preprint arXiv:1906.05807_ (2019).
  * Singh et al. (2021) Devendra Singh, Siva Reddy, Will Hamilton, Chris Dyer, and Dani Yogatama. 2021.  End-to-end training of multi-document reader and retriever for open-domain question answering.  _Advances in Neural Information Processing Systems_ 34 (2021), 25968–25981.
  * Touvron et al. (2023) Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023\.  Llama: Open and efficient foundation language models.  _arXiv preprint arXiv:2302.13971_ (2023).
  * Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017.  Attention Is All You Need. In _Advances in Neural Information Processing Systems_.  https://proceedings.neurips.cc/paper/2017/hash/3f5ee243547dee91fbd053c1c4a845aa-Abstract.html
  * Wang et al. (2023) Cunxiang Wang, Haofei Yu, and Yue Zhang. 2023.  RFiD: Towards Rational Fusion-in-Decoder for Open-Domain Question Answering.  _arXiv preprint arXiv:2305.17041_ (2023).
  * Wei et al. (2021) Jason Wei, Maarten Bosma, Vincent Y Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M Dai, and Quoc V Le. 2021.  Finetuned language models are zero-shot learners.  _arXiv preprint arXiv:2109.01652_ (2021).
  * Xiao et al. (2022) Shitao Xiao, Zheng Liu, Weihao Han, Jianjin Zhang, Defu Lian, Yeyun Gong, Qi Chen, Fan Yang, Hao Sun, Yingxia Shao, et al. 2022\.  Distill-vq: Learning retrieval oriented vector quantization by distilling knowledge from dense embeddings. In _Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval_. 1513–1523.
  * Yang et al. (2019) Wei Yang, Yuqing Xie, Aileen Lin, Xingyu Li, Luchen Tan, Kun Xiong, Ming Li, and Jimmy Lin. 2019.  End-to-end open-domain question answering with bertserini.  _arXiv preprint arXiv:1902.01718_ (2019).
  * Yu et al. (2022) Wenhao Yu, Dan Iter, Shuohang Wang, Yichong Xu, Mingxuan Ju, Soumya Sanyal, Chenguang Zhu, Michael Zeng, and Meng Jiang. 2022.  Generate rather than retrieve: Large language models are strong context generators.  _arXiv preprint arXiv:2209.10063_ (2022).
  * Zhang et al. (2023) Yunxiang Zhang, Muhammad Khalifa, Lajanugen Logeswaran, Moontae Lee, Honglak Lee, and Lu Wang. 2023.  Merging Generated and Retrieved Knowledge for Open-Domain QA.  _arXiv preprint arXiv:2310.14393_ (2023).
  * Zhu et al. (2021) Fengbin Zhu, Wenqiang Lei, Chao Wang, Jianming Zheng, Soujanya Poria, and Tat-Seng Chua. 2021.  Retrieving and reading: A comprehensive survey on open-domain question answering.  _arXiv preprint arXiv:2101.00774_ (2021).

##  Appendix A APPENDIX

Table 11.  Comparison of Generated Answers for Temporal Questions in the TQA Dataset Questions from TQA test |  TQA Label |  GRG |  Google |  GPT
---|---|---|---|---
What star sign is Jamie Lee Curtis? |  Scorpio |  Scorpio |  Sagittarius |  Sagittarius
Which Lloyd Webber musical premiered in the US on 10th December 1993? |  Sunset Blvd |  Sunset Boulevard |  Sunset Boulevard |  Sunset Boulevard
Which actress was voted Miss Greenwich Village in 1942? |  Lauren Becal |  Joanne Woodward |  Lauren Bacall |  No answer

###  A.1. Computational Cost Analysis

In this section, we compare the computational costs of using Dense Passage
Retrieval (DPR) and InstructGPT for document retrieval and generation,
respectively. We use DPR implemented with the T5 model (Roberts et al., 2020),
which has approximately 220 million parameters, and InstructGPT in its largest
configuration with 175 billion parameters.

####  A.1.1. Cost Metrics

We estimate the computational cost in terms of Floating Point Operations
(FLOPs) per token, a metric introduced by Kaplan et al. (2020) and Hunger
(2005). FLOPs provide an estimate of the computational cost required by a
model to process a given input. However, it’s important to note that FLOPs are
not a direct measure of real-world computing costs, as factors such as
latency, power consumption, and hardware efficiency can vary widely.

####  A.1.2. Cost Comparison

The computational costs of DPR and InstructGPT are compared as follow
$DPR=O(|q|\times|D|)$ and $InstructGPT=O(|q|\times|T|)$ . Here, $|q|$
represents the length of the query, $|D|$ is the number of documents in the
corpus, and $|T|$ is the number of tokens in the document.

As shown in Table LABEL:cost_dpr_instruct, the cost of using DPR is
proportional to the number of documents in the corpus, while the cost of using
InstructGPT is proportional to the number of tokens in the document. This
implies that InstructGPT is more efficient for generating documents, while DPR
is more efficient for retrieving documents.

####  A.1.3. Model-Specific Costs

We further break down the costs for each model:

  1. (1)

Document Retriever: T5 Model - The T5 model, with approximately 220 million
parameters and a token limit of 512 tokens per document and question, is used
for document retrieval. The computational costs for encoding all 21 million
Wikipedia documents and retrieving documents for a given question using T5 are
calculated as follows:

FLOPs $\displaystyle=220\times 10^{6}\times 21\times 10^{6}\times 512$
$\displaystyle=2.84\times 10^{18}\text{ FLOPs}$

FLOPs $\displaystyle=220\times 10^{6}\times 20$ $\displaystyle\quad+21\times
10^{6}\times(768+768-1)$ $\displaystyle=3.77\times 10^{11}\text{ FLOPs}$

  2. (2)

Document Generator: InstructGPT Model - The InstructGPT model, with 175
billion parameters and a token limit of 512 tokens per document, is used for
document generation. The computational cost for generating 10 documents for a
given question with 100 words each using InstructGPT is calculated as follows:

FLOPs $\displaystyle=175\times 10^{9}\times 10\times 100=1.75\times
10^{14}\text{ FLOPs}$

  3. (3)

Document Generator Retriever - The cost of encoding all 10 documents with 100
words each using the T5 model is calculated as follows:

FLOPs $\displaystyle=220\times 10^{6}\times 10\times 100=2.2\times
10^{12}\text{ FLOPs}$

  4. (4)

The LLAMA model, with 7 billion parameters and a token limit of 512 tokens per
document, is used for retrieving documents. The computational cost for
retrieving 5 documents using LLAMA is calculated as follows:

FLOPs $\displaystyle=7\times 10^{9}\times 5\times 128=4.48\times 10^{12}\text{
FLOPs}$

Table 12. NQ Dataset Comparison Question |  True Answer |  GRG |  GenRead
---|---|---|---
Who got the first Nobel Prize in Physics? |  Wilhelm Conrad Röntgen |  Wilhelm Conrad Röntgen |  Wilhelm Conrad Röntgen
When was coffee first made into a drink? |  15th century |  15th century |  the 10th century
Who won the MVP for the national league? |  Stanton, Giancarlo |  Giancarlo Stanton |  Christian Yelich
Where do the greasers live in the outsiders? |  Tulsa, Oklahoma |  Tulsa, Oklahoma |  Oklahoma
Who played lionel in ”As time goes by”? |  Geoffrey Dyson Palmer, OBE |  Geoffrey Dyson Palmer |  Geoffrey Palmer

###  A.2. Case study

####  A.2.1. NQ Case Study

Our model’s performance was evaluated using the NQ and TQA datasets, comparing
GRG with GenRead. Table 12 illustrates GRG’s ability to generate accurate
answers that align with the true answers for example questions taken from NQ
dataset, demonstrating an improved understanding of the questions.

####  A.2.2. TQA Case Study

Further analysis with the TQA dataset highlights GRG’s robustness, providing
accurate answers and considering multiple valid responses. Table 11 shows
GRG’s performance in comparison to Google and GPT.