# Ultraviolet Freeze-in Dark Matter
through the Dilaton Portal

Aqeel Ahmed  and Saereh Najjari

###### Abstract

We consider a class of models where the Standard Model (SM) and dark matter
(DM) belong to a conformal/scale-invariant theory at high energies. The scale
invariance is broken spontaneously at scale $f$, and the corresponding
Goldstone boson is the dilaton. In the low-energy theory, we assume DM
interacts with the SM only through the dilaton portal suppressed by the
conformal breaking scale $f$. Assuming $f\\!\gg\\!{\rm TeV}$, the portal
interactions are very feeble, and hence the DM is not in thermal equilibrium
with the SM bath. Therefore, ultraviolet freeze-in production of DM is
realized through the dilaton portal, which is most effective at the maximum
temperature of the SM bath. The temperature evolution critically depends on
the reheating dynamics, which we parameterize by a general equation of state
$w$ and temperature at the end of reheating $T_{\rm rh}$. Implications of the
reheating dynamics are studied for DM production in this framework. We have
identified regions of parameter space that lead to the observed DM relic
abundance for a wide range of DM masses and reheating temperatures for a
scalar, vector, or fermion DM.

##  1 Introduction

The Standard Model (SM) of particle physics presents a remarkably successful
description of visible matter constituents and their interactions. However,
there remain some of the puzzles which are unanswered within the SM framework
including the nature of dark matter (DM), electroweak hierarchy problem, and
fermion mass hierarchy. Some of these puzzles can be addressed within the
framework of strongly coupled gauge theories where the SM Higgs boson emerges
as a composite state or pseudo-Nambu-Goldstone boson (pNGB), see for a review
[1, 2]. Regarding the DM candidate within strongly coupled theories, one
possibility is that it is a pNGB state which ensures it to be naturally
lighter than the composite scale. Hence it forms a good weakly interacting
massive particle (WIMP) DM candidate [3, 4, 5, 6] where DM is produced through
the standard freeze-out mechanism assuming it remains in thermal equilibrium
at temperatures larger than DM mass. There is a vast experimental program of
DM searches including direct detection, indirect detection, and collider
searches for a WIMP DM, see e.g. [7]. However, there is no signal of WIMP DM
to date, which motivates to explore DM candidates beyond the WIMP or in
general DM production through the freeze-out paradigm.

In this work, we consider a framework where the SM and DM belong to a
(strongly coupled) conformal/scale-invariant theory. We assume the conformal
invariance is spontaneously broken at scale $f\gg v_{\rm SM}$, where $v_{\rm
SM}\\!=\\!246\;\hbox{GeV}$ is the electroweak scale. Furthermore, we assume
that the SM and DM do not have direct interactions below the conformal
invariance breaking scale $f$. The spontaneous breaking of scale invariance
results in a pseudo-Goldstone boson called, _dilaton_ [8]. Requiring the scale
invariance of the theory dictates the form of dilaton interactions with the SM
and DM in the low-energy effective theory below scale $\Lambda=4\pi f$ [9, 10,
11, 12, 13]. It turns out that the dilaton interacts with the SM and DM
through dimension-five or higher operators suppressed by the scale $f$. Hence
the effective couplings between the SM and DM through the _dilaton portal_ is
at least dimension-six order. For low scale conformal breaking $f$, the SM and
DM are in thermal equilibrium, and hence the production of DM via the dilaton
portal is realized through the thermal freeze-out mechanism [14, 15, 16, 17,
18, 19, 20]. However, assuming the conformal invariance breaking scale $f\gg
v_{\rm SM}$, it is natural that SM and DM are out of thermal equilibrium.
Hence the possible DM production follows through the freeze-in mechanism,
where DM is produced through the SM annihilation via the dilaton portal or
through direct decays/annihilation of the dilaton field, see also [21, 22, 23,
24].

The freeze-in mechanism relies on the fact that initial DM abundance is
negligible compared to the states in thermal equilibrium [25], see for a
review [26]. Furthermore, if the SM-DM interaction is a higher dimensional
operator, as in our case, the freeze-in mechanism also crucially depends on
the maximum temperature in the early universe. In particular, the DM
production is dominated around the maximum temperature [27]. The maximum
temperature is usually assumed as the temperature at the end of reheating,
$T_{\rm rh}$. This is the only correct description if the reheating is
instantaneous. However, the reheating through perturbative decays of the
inflaton field to the SM would be non-instantaneous, which leads to an
extended period of reheating. Assuming a non-instantaneous reheating scenario,
the maximum temperature $T_{\rm max}$ can be much larger than the temperature
at the end of reheating phase $T_{\rm rh}$ [28]. The maximum temperature
$T_{\rm max}$ depends on inflaton energy density at the end of inflation, the
equation of state $w$ during the reheating period, and duration of reheating
phase which can be parametrized by the temperature at the end of reheating
phase, $T_{\rm rh}$. The end of reheating is defined when the SM energy
density becomes equal to the inflaton energy density. Without specifying the
details of the reheating phase, we consider the general equation of state $w$
in the range $(-1/3,1)$, where $w=0$ corresponds to the matter-dominated phase
during the reheating process. Recently there has been significant studies on
implications of the reheating dynamics on the production of DM, see e.g. [29,
30, 31, 32, 33, 34, 35, 36, 37, 38, 39].

We study the implications of non-standard reheating dynamics on the
ultraviolet (UV) freeze-in production of DM through the dilaton portal. We
consider a DM candidate to be a scalar, fermion, or vector field. For
simplicity, we assume no self-interactions for the DM field, and dark matter
mass is the only parameter of the dark sector. The dilaton portal dynamics are
completely fixed by two parameters; the dilaton mass and the conformal
breaking scale $f$. Furthermore, the reheating dynamics is parametrized
effectively by three parameters, the Hubble scale at the end of inflation, the
equation of state $w$, and the reheating temperature. We study the freeze-in
production of the DM within this framework and identify the parameter space
where the observed DM abundance can be produced.

The paper is organized as follows: In Sec. 2 we present details of the dilaton
portal DM model with the effective low-energy Lagrangian including
interactions of the dilaton field with the SM and DM. In Sec. 3, we describe
early universe cosmology with the non-standard period of reheating defined
with the equation of state $w$. Production of DM via UV freeze-in is given in
Sec. 4, where consider DM production through SM annihilation as well as
dilaton annihilation/decays when kinematically allowed. Finally, in Sec. 5 we
conclude our findings.

##  2 The model

In this section, we lay out a framework where freeze-in dark matter production
via a dilaton portal is realized. We assume a UV completion of the SM and DM
involving a strongly coupled nearly scale-invariant theory. The scale
invariance is broken spontaneously and the corresponding pseudo-Goldstone
boson is the dilaton $\sigma(x)$. Furthermore, we assume the scale symmetry is
realized nonlinearly below the symmetry breaking scale $f$, such that under
the scale transformation $x^{\mu}\to x^{\prime\mu}\\!=\\!e^{-\omega}x^{\mu}$,
the dilaton undergoes a shift symmetry
$\sigma(x)\to\sigma^{\prime}(x^{\prime})\\!=\\!\sigma(x)+\omega f$. It is
instructive to express the dilaton field as a conformal compensator, i.e.

$\chi(x)=fe^{\sigma(x)/f},$ (2.1)

such that it transforms linearly under the scale transformation, i.e.
$\chi(x)\to\chi^{\prime}(x^{\prime})\\!=\\!e^{\omega}\chi(x)$. The vacuum
expectation value (VEV) of $\chi(x)$ sets the scale of spontaneous symmetry
breaking, i.e. $\langle\chi(x)\rangle\\!\equiv\\!f$, which is determined by
the underlying strong sector dynamics at $\Lambda\\!=\\!4\pi f$.

In this work we consider the interactions of DM with the SM only through the
dilaton portal. In particular, we consider the following form of the
Lagrangian

$\displaystyle{\cal L}$ $\displaystyle={\cal L}_{\rm SM}+{\cal L}_{\rm
DM}+{\cal L}_{\rm dilaton}+{\cal L}_{\rm SM}^{\rm int}+{\cal L}_{\rm DM}^{\rm
int},$ (2.2)

where ${\cal L}_{\rm SM}$ is the SM Lagrangians. As mentioned in the
Introduction, we consider three possibilities for DM $X$, i.e. scalar,
fermion, or vector DM, with the following Lagrangian,

$\displaystyle{\cal L}_{\rm DM}$
$\displaystyle=\begin{dcases}\frac{1}{2}\partial_{\mu}X\partial^{\mu}X-\frac{1}{2}m_{\\!X}^{2}X^{2},&\text{Scalar
DM}\\\ i\overline{X}\not{\partial}X-m_{\\!X}\overline{X}X,&\text{Fermion
DM}\\\
\\!\\!-\frac{1}{4}X_{\mu\nu}X^{\mu\nu}+\frac{1}{2}m_{\\!X}^{2}X_{\mu}^{2},\qquad&\text{Vector
DM}\end{dcases}$ (2.3)

where $X_{\mu\nu}\\!=\\!\partial_{\mu}X_{\nu}-\partial_{\nu}X_{\mu}$ is the
field strength tensor to the vector DM $X_{\mu}$. The dilaton Lagrangian is

$\displaystyle\mathcal{L}_{\rm{dilaton}}=\frac{1}{2}\partial_{\mu}\chi\partial^{\mu}\chi-V(\chi),$
(2.4)

where we assume the following form for the dilaton potential,

$\displaystyle V(\chi)$
$\displaystyle=\frac{m_{\sigma}^{2}}{4f^{2}}\,\chi^{4}\bigg{[}\ln\bigg{(}\frac{\chi}{f}\bigg{)}-\frac{1}{4}\bigg{]}.$
(2.5)

It is convenient to rewrite the dilaton field $\chi$ in terms of canonically
normalized physical dilaton fluctuation $\sigma$ by expanding the $\chi$ field
around its VEV, i.e., $\chi=f+\sigma$. The dilaton Lagrangian in the canonical
basis is

$\displaystyle\mathcal{L}_{\rm{dilaton}}=\frac{1}{2}\partial_{\mu}\sigma\partial^{\mu}\sigma-\frac{1}{2}m_{\sigma}^{2}\sigma^{2}-\frac{5}{3!}\frac{m_{\sigma}^{2}}{f}\sigma^{3}-\frac{11}{4!}\frac{m_{\sigma}^{2}}{f^{2}}\sigma^{4}+{\cal
O}(\sigma^{5}).$ (2.6)

The interactions of the dilaton field $\sigma$ with the SM and DM are dictated
by the nonlinearly realized scale invariance below scale $\Lambda\\!=\\!4\pi
f$.

In this work we assume that scale invariance breaking scale $\Lambda$ is much
larger than the electroweak scale, i.e. $\Lambda\gg 1\;\hbox{TeV}$.
Furthermore, as we are interested in UV freeze-in production of dark matter,
therefore usually temperatures involved are much larger than the electroweak
scale. Taking this into account, we define dilaton interactions with the SM
only in the electroweak symmetric phase (EWSP) for temperatures $T$ above
critical temperature $T_{c}\sim 150\;\hbox{GeV}$. Note the SM electroweak
symmetry is restored at $T>T_{c}$ and taking into account the thermal
corrections at one-loop level, Higgs effective potential takes the following
form,

$V(H,T)\simeq\mu^{2}(T)|H|^{2}+\lambda(T)|H|^{4}$ (2.7)

where the Higgs quartic coupling is $\lambda(T)\\!\sim\\!\lambda\simeq 0.13$,
and the Higgs effective mass parameter $\mu^{2}(T)$ can be approximated as,

$\mu^{2}(T)\approx\begin{cases}-\lambda\,v_{\rm EW}^{2}&\hskip
28.45274ptT\lesssim T_{c}\,,\\\ ~{}\beta\,T^{2}&\hskip
28.45274ptT>T_{c}\,.\end{cases}$ (2.8)

with the parameter $\beta\sim 0.4$.

In the SM electroweak symmetric phase, all the SM gauge and fermion fields are
massless. However, the SM Higgs doublet (four real scalar components, $h_{i}$
with $i=0,1,2,3$) mass is $m_{h}^{2}=\beta\,T^{2}$. Note this Higgs mass is
one of the sources of explicit breaking of scale invariance. In this case
dilaton coupling with the Higgs field are,

$\displaystyle{\cal L}_{\rm SM}^{\rm int}$
$\displaystyle\supset\sum_{i=0}^{3}\bigg{(}\frac{\sigma}{f}+\frac{\sigma^{2}}{2f^{2}}\bigg{)}\bigg{[}\partial_{\mu}h_{i}\partial^{\mu}h_{i}-2m_{h}^{2}h_{i}^{2}\bigg{]}+\ldots,$
(2.9)

where ellipses denote higher order interactions. Furthermore, the dialton
interacts with the SM fermions through the Yukawa terms,

$\displaystyle{\cal L}_{\rm SM}^{\rm int}$
$\displaystyle\supset\frac{\sigma}{f}\bigg{[}y_{t}\bar{Q}_{L}\widetilde{H}t_{R}+\ldots\bigg{]}+\hbox{h.c.},$
(2.10)

where $\widetilde{H}=i\sigma_{2}H^{\ast}$ and ellipsis represent the fermions
with smaller Yukawa couplings. Since top Yukawa coupling $y_{t}\sim 1$ being
the largest coupling would be the most relevant for our analysis. The dilaton
interactions with the SM gauge bosons emerge due to RGE running of their gauge
couplings and the following form

$\displaystyle{\cal L}_{\rm SM}^{\rm int}$
$\displaystyle\supset\sum_{i=1}^{3}\frac{\alpha_{i}}{8\pi}b_{i}\frac{\sigma}{f}F_{i\mu\nu}F^{\mu\nu}_{i},$
(2.11)

where $i=1,2,3$ corresponds to SM gauge groups $U(1)_{Y}$, $SU(1)_{L}$, and
$SU(3)_{c}$, respectively. Whereas $\alpha_{i}=g_{i}^{2}/4\pi$ and $b_{i}$ are
the corresponding gauge couplings and beta-function coefficients,
respectively.

#### DM–Dilaton interactions:

At the leading order the interaction Lagrangian for the dilaton field with the
DM is given by,

$\displaystyle{\cal L}_{\rm DM}^{\rm int}$
$\displaystyle=\begin{dcases}\\!\\!\bigg{(}\frac{\sigma}{f}+\frac{\sigma^{2}}{2f^{2}}\bigg{)}\bigg{[}\partial_{\mu}X\partial^{\mu}X-2m_{X}^{2}X^{2}\bigg{]},&\text{Scalar
DM}\\\ \\!\\!-\frac{\sigma}{f}m_{\\!X}\overline{X}X,&\text{Fermion DM}\\\
\bigg{(}\frac{\sigma}{f}+\frac{\sigma^{2}}{2f^{2}}\bigg{)}m_{\\!X}^{2}X_{\mu}^{2},\qquad&\text{Vector
DM}\end{dcases}$ (2.12)

where we consider fermion DM as Dirac particle, however for Majorana fermion
the above interaction term is rescaled by factor 1/2.

##  3 Non-standard cosmology during the reheating phase

We assume a slow-roll inflationary paradigm with quasi-de Sitter expansion of
the Universe with Hubble parameter $H_{I}$. The inflationary epoch of
accelerated expansion ends with a reheating phase where the inflaton field
$\phi$ is assumed to transfer its energy density via perturbative decays to
the SM sector. Without specifying the details of reheating dynamics, we assume
inflaton energy density scales as $\rho_{\phi}\propto a^{-3(1+w)}$, where $w$
is the equation of state during the reheating phase, i.e.
$p_{\phi}=w\rho_{\phi}$. During inflation $w\simeq-1$ and the end of inflation
is marked when $w=-1/3$. During the reheating phase, we assume
$w\in(-1/3,5/3)$, where $w=0$ defines the matter-dominated universe and
$w=1/3$ corresponds to radiation-dominated universe. The Hubble rate is given
by,

$H(a)\equiv\frac{\dot{a}}{a}=\sqrt{\frac{\rho(a)}{3M_{\text{Pl}}^{2}}},$ (3.1)

where $M_{\text{Pl}}=2.435\\!\times\\!10^{18}\;\hbox{GeV}$ is the reduced
Planck mass and over-dot is derivative w.r.t. time $t$. Above the energy
density $\rho(a)$ is sum of the inflaton and the SM radiation energy
densities, i.e. $\rho(a)=\rho_{\phi}(a)+\rho_{\\!R}(a)$. During the reheating
phase inflaton energy density is dominant, i.e.
$\rho_{\phi}(a)\gg\rho_{\\!R}(a)$ for $a_{e}<a<a_{\rm rh}$, where $a_{e}$ and
$a_{\rm rh}$ denote the end of inflation and reheating phase, respectively.
Furthermore, we assume that DM energy density $\rho_{\\!X}$ remains sub-
dominant component of the total energy density. The end of reheating phase is
defined when $\rho_{\phi}(a_{\rm rh})=\rho_{\\!R}(a_{\rm rh})$.

The exact cosmological evolution of the reheating phase is determined by
solving the coupled Boltzmann equations,

$\displaystyle\dot{\rho}_{\phi}+3(1+w)H\rho_{\phi}$
$\displaystyle=-\Gamma_{\phi}\rho_{\phi},$
$\displaystyle\dot{\rho}_{\\!R}+4H\rho_{\\!R}$
$\displaystyle=+\Gamma_{\phi}\rho_{\phi},$ (3.2)

where $\Gamma_{\phi}$ is the perturbative decay width of the inflation field
to SM radiation. Assuming $\Gamma_{\phi}\ll 3(1+w)H$ during the reheating
phase $a_{e}\lesssim a\lesssim a_{\rm rh}$ we can approximately solve the
above Boltzmann equations as,

$\displaystyle\rho_{\phi}$ $\displaystyle\simeq
3M_{\text{Pl}}^{2}H_{\\!I}^{2}\bigg{(}\frac{a_{e}}{a}\bigg{)}^{3(1+w)},$ (3.3)
$\displaystyle\rho_{\\!R}$ $\displaystyle\simeq
3M_{\text{Pl}}^{2}H_{\\!I}^{2}\bigg{(}\frac{a_{e}}{a_{\mathrm{rh}}}\bigg{)}^{3(1+w)/2}\bigg{[}\bigg{(}\frac{a_{e}}{a}\bigg{)}^{3(1+w)/2}-\bigg{(}\frac{a_{e}}{a}\bigg{)}^{4}\bigg{]},$
(3.4)

where we employed $\Gamma_{\phi}\\!\simeq\\!(5-3w)H_{\rm rh}/2$ with $H_{\rm
rh}$ being the Hubble scale at the end of reheating,

$H_{\rm rh}\equiv H(a_{\rm
rh})=H_{\\!I}\bigg{(}\frac{a_{e}}{a_{\mathrm{rh}}}\bigg{)}^{3(1+w)/2}\,.$
(3.5)

After the end of reheating, the inflaton energy density rapidly vanishes and
standard cosmological evolution takes its course where SM radiation is the
dominant energy density until the matter-radiation equality, i.e.
$a\\!=\\!a_{\rm eq}$. During the radiation-dominated epoch, $a_{\rm
rh}\lesssim a\lesssim a_{\rm eq}$, the radiation energy density is given as,

$\rho_{\\!R}\simeq\rho_{\rm
rh}\bigg{(}\frac{a_{\mathrm{rh}}}{a}\bigg{)}^{4},\qquad{\rm
where}\quad\rho_{\rm rh}\equiv
3M_{\text{Pl}}^{2}H_{\mathrm{I}}^{2}\bigg{(}\frac{a_{e}}{a_{\mathrm{rh}}}\bigg{)}^{3(1+w)}.$
(3.6)

It is instructive to write the Hubble parameter as a function of the scale
factor,

$\displaystyle H(a)$ $\displaystyle=\begin{dcases}H_{\rm
I}\bigg{(}\frac{a_{e}}{a}\bigg{)}^{\frac{3(1+w)}{2}},&a_{e}<a\leq a_{\rm
rh}\,,\\\ H_{\rm rh}\bigg{(}\frac{a_{\rm rh}}{a}\bigg{)}^{2},\qquad&a_{\rm
rh}<a\leq a_{\rm eq}\,,\end{dcases}$ (3.7)

where $H_{\rm rh}$ is defined in (3.5). The temperature of the SM bath is
defined in terms of the SM radiation energy density as,

$\displaystyle T$
$\displaystyle=\bigg{(}\frac{30\,\rho_{\\!R}}{\pi^{2}g_{\star}(T)}\bigg{)}^{1/4}\simeq
T_{\rm rh}\bigg{(}\frac{g_{\star}(T_{\rm
rh})}{g_{\star}(T)}\bigg{)}^{1/4}\begin{dcases}\bigg{(}\frac{H(a)}{H_{\rm
rh}}\bigg{)}^{1/4},&a_{\rm max}<a\leq a_{\rm rh}\,,\\\
\bigg{(}\frac{H(a)}{H_{\rm rh}}\bigg{)}^{1/2},\qquad&a>a_{\rm
rh}\,,\end{dcases}$ (3.8)

where $a_{\rm max}$ is the value of scale factor when temperature (or
radiation energy density) has its maximum value, see below Eq. (3.11).
Whereas, $g_{\star}(T)$ is the effective number of relativistic d.o.f.
contributing to the energy density. Temperature at the end of reheating period
$T_{\rm rh}$ is defined as,

$T_{\rm rh}^{2}\equiv\frac{3}{\pi}\sqrt{\frac{10}{g_{\star}(T_{\rm
rh})}}\,M_{\text{Pl}}H_{\rm rh}.$ (3.9)

In the following analysis, we treat temperature $T_{\rm rh}$ as one of the
free parameters of the model.

Note that the initial condition for the radiation energy density at $a=a_{e}$
is $\rho_{\\!R}(a_{e})=0$, therefore the temperature of the SM bath is also
zero at the onset of reheating phase. The maximum of radiation energy density
or the maximum temperature $T_{\rm max}$ is reached during the reheating phase
at $a=a_{\rm max}<a_{\rm rh}$,

$a_{\rm
max}=a_{e}\,\bigg{(}\frac{8}{3(1+w)}\bigg{)}^{\frac{2}{5-3w}},\qquad{\rm
for}~{}-\\!\frac{1}{3}<w<\frac{5}{3}.$ (3.10)

Hence using (3.4) and (3.9) the corresponding maximum temperature $T_{\rm
max}$ can be written as,

$T_{\rm
max}^{4}=\frac{3(5-3w)}{8\pi}\bigg{(}\frac{3(1+w)}{8}\bigg{)}^{\frac{3(1+w)}{5-3w}}\sqrt{\frac{10}{g_{\star}(T_{\rm
rh})}}\,M_{\text{Pl}}H_{\\!I}\,T_{\rm rh}^{2},\qquad{\rm
for}~{}-\\!\frac{1}{3}<w<\frac{5}{3}.$ (3.11)

In the following analysis, we have three free parameters which determine non-
standard cosmological evolution during the reheating phase, namely (i) the
Hubble scale at the end of inflation $H_{\\!I}$, (ii) the temperature at the
end of reheating $T_{\rm rh}$, and (iii) the equation of state $w$ which we
take in the range $(-1/3,5/3)$. The value of the scale factor at the end of
inflation $a_{e}$ is arbitrary. With these three free parameters
$(H_{\\!I},T_{\rm rh},w)$ one can readily calculate all the other quantities
related to cosmological and thermal evolution. Now the inflationary Hubble
scale is constrained by the CMB measurement of the inflationary perturbations.
The current upper-bound from Planck data [40] reads as $H_{\\!I}\lesssim
6\\!\times\\!10^{13}\;\hbox{GeV}$. An upper-bound on the maximum temperature
$T_{\rm max}$ is set by the requirement that radiation energy density is
smaller than the total energy density at the end of inflation, i.e.
$\rho_{\\!R}\lesssim 3M_{\text{Pl}}^{2}H_{\\!I}^{2}$. Employing the upper-
bound on $H_{\\!I}$, we get $T_{\rm max}\lesssim
6.5\\!\times\\!10^{15}\;\hbox{GeV}$. Reheating temperature is by default
smaller than the maximum temperature, i.e. $T_{\rm rh}<T_{\rm max}$.
Furthermore, the BBN sets a lower-bound on the reheating temperature $T_{\rm
rh}\gtrsim 1\;\hbox{MeV}$ [41].

##  4 Dilaton portal dark matter production

In this section we discuss freeze-in production of dark matter via dilaton
portal. The Boltzmann equation for DM $X$ is,

$\dot{n}_{\\!X}+3Hn_{\\!X}={\cal C}_{X}+{\cal D}_{X},$ (4.1)

where ${\cal C}/{\cal D}$ are the collision/decay terms are,

$\displaystyle{\cal C}_{X}$
$\displaystyle\simeq\bar{n}_{\\!X}^{2}\Big{[}\langle\sigma_{XX\to{\rm
SM\,SM}}\,v\rangle+\langle\sigma_{XX\to\sigma\sigma}\,v\rangle\Big{]},$ (4.2)
$\displaystyle{\cal D}_{X}$ $\displaystyle\simeq
2n_{\sigma}\langle\Gamma_{\sigma\to XX}\rangle,$ (4.3)

with $\langle\sigma_{XX\to{\rm SM\,SM}}\,v\rangle$ and $\langle\Gamma_{\phi\to
XX}\rangle$ being the thermally averaged annihilation cross section and
partial width, respectively. Above the equilibrium number density of species
$i$ with spin $J_{i}$ is defined as,

$\bar{n}_{i}=\left(2J_{i}+1\right)\frac{m_{i}^{2}\,T}{2\pi^{2}}K_{2}\left(\frac{m_{i}}{T}\right),$
(4.4)

where $K_{2}(x)$ is the Bessel function of second kind. Note that the decay
term is only relevant when $m_{\sigma}>2m_{\\!X}$. Furthermore, we assume no
interaction between inflaton and dilaton, therefore dilaton is produced
through scatting and inverse decays of the SM fields. Therefore, $n_{\sigma}$
is the number density of the dilaton field. If the interaction rate between
the SM and dilaton is larger than the Hubble scale then dilaton is in thermal
equilibrium, i.e. $n_{\sigma}=\bar{n}_{\sigma}$. In general, we solve the
following Boltzmann equation to get the dilaton number density $n_{\sigma}$,

$\dot{n}_{\sigma}+3Hn_{\sigma}=\bar{n}_{\sigma}\langle\Gamma_{{\rm
SM}\to\sigma}\rangle,$ (4.5)

where the thermally averaged SM annihilation to dilaton $\langle\Gamma_{{\rm
SM}\to\sigma}\rangle$ is dominated by the top-Yukawa interaction term Eq.
(2.10).

It is convenient to recast the Boltzmann equation (4.1) in terms of the
comoving number density $N_{\\!X}=n_{\\!X}a^{3}$ and as a function of
temperature $T$,

$\displaystyle\frac{dN_{\\!X}}{dT}=-\frac{3a_{\rm
rh}^{3}}{\pi}\sqrt{\frac{10}{g_{\star}(T)}}M_{\text{Pl}}\frac{T_{\rm
rh}^{3}}{T^{6}}\Big{[}{\cal C}_{X}+{\cal
D}_{X}\Big{]}\\!\begin{dcases}\frac{8}{3(1+w)}\bigg{(}\frac{T_{\rm
rh}}{T}\bigg{)}^{\frac{7-w}{1+w}}\,,&T\geq T_{\rm rh},\\\ \\\ \frac{g_{\star
s}(T_{\rm rh})}{g_{\star s}(T)}\,,&T<T_{\rm rh},\end{dcases}$ (4.6)

where $g_{\star s}(T)$ denote the effective number of relativistic d.o.f.
contributing to entropy density $s=2\pi^{2}g_{\star s}(T)\,T^{3}/45$. Dark
matter present relic abundance can be calculated as,

$\Omega_{X}h^{2}=\frac{m_{X}n_{X}(a_{0})}{\rho_{c}\,h^{-2}}=\frac{m_{X}}{\rho_{c}\,h^{-2}}\frac{N_{X}(T_{0})}{a_{\rm
rh}^{3}}\frac{s_{0}}{s_{\rm rh}},$ (4.7)

where $n_{\\!X}(a_{0})$ is the DM number at present. Whereas, the second
equality is obtained employing entropy conservation with present entropy
density $s_{0}=2970\,{\rm cm^{-3}}$ and $s_{\rm rh}\equiv s(T_{\rm rh})$.
Above the critical energy density is $\rho_{c}=1.054\times 10^{-5}h^{2}\,{\rm
GeV\,cm^{-3}}$

###  4.1 SM and dilaton scattering

The schematic diagrams contributing to the production of DM due to
annihilation of SM bath particles as well as the annihilation/decay of dilaton
field are shown in Fig. 1. DM production through the annihilation of the
dilaton field is relevant when the latter is in thermal equilibrium with the
SM bath. As mentioned above, since we are interested in the production of DM
through UV freeze-in, therefore we consider SM annihilation to DM only in the
electroweak symmetric phase.

Figure 1: Schematic diagrams for the freeze-in production of DM through the
dilaton portal.

#### Scalar DM $X$:

For the case of vector DM $X_{\mu}$ amplitude squared for the SM and dilaton
scattering processes are,

$\displaystyle|{\cal M}_{XX\to hh}|^{2}$
$\displaystyle=\frac{\left(2m_{h}^{2}+s\right)^{2}\left(2m_{\\!X}^{2}+s\right)^{2}}{2f^{4}\left(m_{\sigma}^{2}\left(\Gamma_{\sigma}^{2}-2s\right)+m_{\sigma}^{4}+s^{2}\right)},$
(4.8) $\displaystyle|{\cal M}_{XX\to V_{i}V_{i}}|^{2}$
$\displaystyle=\frac{b_{i}^{2}\alpha_{i}^{2}}{4\pi^{2}}\frac{s^{2}\left(2m_{\\!X}^{2}+s\right)^{2}}{4f^{4}\left(\Gamma_{\sigma}^{2}m_{\sigma}^{2}+\left(s-m_{\sigma}^{2}\right)^{2}\right)},$
(4.9) $\displaystyle|{\cal M}_{XX\to\sigma\sigma}|^{2}$
$\displaystyle=\frac{1}{2f^{4}}\left[\frac{\left(m_{\sigma}^{2}+2m_{\\!X}^{2}\right){}^{2}}{u-m_{\\!X}^{2}}-\frac{2m_{\sigma}^{2}\left(m_{\sigma}^{2}+8m_{\\!X}^{2}+3s\right)}{m_{\sigma}^{2}-s}+\frac{\left(m_{\sigma}^{2}+2m_{\\!X}^{2}\right){}^{2}}{t-m_{\\!X}^{2}}-6m_{\\!X}^{2}\right]^{2},$
(4.10)

where $b_{i}$ and $\alpha_{i}$ are the beta-function coefficient and the gauge
couplings with $i=1,2,3$ for the SM gauge groups $U(1)_{Y},SU(2)_{L}$, and
$SU(3)_{c}$, respectively.

The scattering rate ${\cal C}_{\\!X}$ can be approximated for the above
processes as,

$\displaystyle{\cal
C}_{X}\approx\begin{dcases}\Big{(}5+\frac{b_{i}^{2}\alpha_{i}^{2}}{8\pi^{2}}\Big{)}\frac{3T^{8}}{4\pi^{5}f^{4}}\,,&T\gg
m_{\sigma},m_{\\!X}\\\
\Big{(}4+\frac{b_{i}^{2}\alpha_{i}^{2}}{8\pi^{2}}\Big{)}\frac{2880T^{12}}{\pi^{5}f^{4}m_{\sigma}^{4}}\,,&m_{\sigma}\gg
T\gg m_{\\!X},\\\
\Big{(}4+\frac{b_{i}^{2}\alpha_{i}^{2}}{8\pi^{2}}\Big{)}\frac{9m_{\\!X}^{9}T^{3}}{8\pi^{4}f^{4}m_{\sigma}^{4}}\exp\\!\Big{(}\frac{-2m_{\\!X}}{T}\Big{)}\,,&m_{\sigma}\gg
m_{\\!X}\gg T\end{dcases}$ (4.11)

where summation over all the SM gauge bosons $i$ is assumed.

#### Fermion DM $X$:

For the case of vector DM $X_{\mu}$ amplitude squared for the SM and dilaton
scattering processes are,

$\displaystyle|{\cal M}_{XX\to hh}|^{2}$
$\displaystyle=\frac{49m_{\\!X}^{2}\left(2m_{h}^{2}+s\right)^{2}\left(s-4m_{\\!X}^{2}\right)}{4f^{4}\left(\Gamma_{\sigma}^{2}m_{\sigma}^{2}+\left(m_{\sigma}^{2}-s\right)^{2}\right)},$
(4.12) $\displaystyle|{\cal M}_{XX\to V_{i}V_{i}}|^{2}$
$\displaystyle=\frac{b_{i}^{2}\alpha_{i}^{2}}{4\pi^{2}}\frac{49s^{2}m_{\\!X}^{2}\left(s-4m_{\\!X}^{2}\right)}{8f^{4}\left(\Gamma_{\sigma}^{2}m_{\sigma}^{2}+\left(m_{\sigma}^{2}-s\right)^{2}\right)},$
(4.13) $\displaystyle|{\cal M}_{XX\to\sigma\sigma}|^{2}$
$\displaystyle=\frac{49m_{\\!X}^{2}\left(s-4m_{\\!X}^{2}\right)\left(15m_{\sigma}^{2}+s\right)^{2}}{16f^{4}\left(\Gamma_{\sigma}^{2}m_{\sigma}^{2}+\left(s-m_{\sigma}^{2}\right)^{2}\right)}.$
(4.14)

The scattering rate ${\cal C}_{\\!X}$ for the fermion DM can be approximated
as,

$\displaystyle{\cal
C}_{X}\approx\begin{dcases}\Big{(}4+\frac{b_{i}^{2}\alpha_{i}^{2}}{8\pi^{2}}\Big{)}\frac{m_{\\!X}^{2}T^{6}}{64\pi^{5}f^{4}}+\frac{25m_{\\!X}^{2}m_{\sigma}^{4}T^{2}}{4096\sqrt{2}\pi^{4}f^{4}}\,,&T\gg
m_{\sigma},m_{\\!X}\\\
\Big{(}4+\frac{b_{i}^{2}\alpha_{i}^{2}}{8\pi^{2}}\Big{)}\frac{18m_{\\!X}^{2}T^{10}}{\pi^{5}f^{4}m_{\sigma}^{4}}\,,&m_{\sigma}\gg
T\gg m_{\\!X},\\\
\Big{(}4+\frac{b_{i}^{2}\alpha_{i}^{2}}{8\pi^{2}}\Big{)}\frac{75m_{\\!X}^{4}T^{4}}{512\pi^{4}f^{4}}\exp\\!\Big{(}\frac{-2m_{\\!X}}{T}\Big{)}\,,&m_{\sigma}\gg
m_{\\!X}\gg T\end{dcases}$ (4.15)

#### Vector DM $X_{\mu}$:

For the case of vector DM $X_{\mu}$ amplitude squared for the SM and dilaton
scattering processes are,

$\displaystyle|{\cal M}_{XX\to hh}|^{2}$
$\displaystyle=\frac{\left(2m_{h}^{2}+s\right)^{2}\left(-4sm_{\\!X}^{2}+12m_{\\!X}^{4}+s^{2}\right)}{2f^{4}\left(\Gamma_{\sigma}^{2}m_{\sigma}^{2}+\left(s-m_{\sigma}^{2}\right)^{2}\right)},$
(4.16) $\displaystyle|{\cal M}_{XX\to V_{i}V_{i}}|^{2}$
$\displaystyle=\frac{b_{i}^{2}\alpha_{i}^{2}}{4\pi^{2}}\frac{s^{2}\left(-4sm_{\\!X}^{2}+12m_{\\!X}^{4}+s^{2}\right)}{4f^{4}\left(\Gamma_{\sigma}^{2}m_{\sigma}^{2}+\left(s-m_{\sigma}^{2}\right)^{2}\right)},$
(4.17) $\displaystyle|{\cal M}_{XX\to\sigma\sigma}|^{2}$
$\displaystyle=\frac{\left(-4sm_{\\!X}^{2}+12m_{\\!X}^{4}+s^{2}\right)\left(7m_{\sigma}^{2}+s\right)^{2}}{2f^{4}\left(\Gamma_{\sigma}^{2}m_{\sigma}^{2}+\left(s-m_{\sigma}^{2}\right)^{2}\right)}.$
(4.18)

In this case, neglecting the leading order phase space factor, the approximate
form of scatting rate ${\cal C}_{\\!X}$ is the same as that of the scalar DM
case, i.e. Eq. (4.11).

###  4.2 Dilaton decays

For the dark matter lighter than the dilaton, the dominant contribution to the
production of DM is through the dilaton decays. In this case, the decay term
${\cal D}_{\\!X}$ in Eq. (4.1) is given by,

$\displaystyle{\cal D}_{\\!X}$
$\displaystyle=2n_{\sigma}\frac{K_{1}\big{(}\\!\frac{m_{\sigma}}{T}\\!\big{)}}{K_{2}\big{(}\\!\frac{m_{\sigma}}{T}\\!\big{)}}\begin{dcases}\\!\\!\frac{m_{\sigma}^{3}}{32\pi
f^{2}}\sqrt{1-\frac{4m_{\\!X}^{2}}{m_{\sigma}^{2}}}\left(1+\frac{2m_{\\!X}^{2}}{m_{\sigma}^{2}}\right)^{2},&\text{Scalar
DM}\\\ \\!\\!\frac{m_{\sigma}m_{\\!X}^{2}}{8\pi
f^{2}}\left(1-\frac{4m_{\\!X}^{2}}{m_{\sigma}^{2}}\right)^{3/2},&\text{Fermion
DM}\\\ \\!\\!\frac{m_{\sigma}^{3}}{32\pi
f^{2}}\sqrt{1-\frac{4m_{\\!X}^{2}}{m_{\sigma}^{2}}}\left(1-\frac{4m_{\\!X}^{2}}{m_{\sigma}^{2}}+\frac{12m_{\\!X}^{4}}{m_{\sigma}^{4}}\right),\qquad&\text{Vector
DM}\end{dcases}$ (4.19)

where $n_{\sigma}$ is the number density of the dilaton field. We note that in
most natural parameter space when $f\gg v_{\rm SM}$ the dilaton field is out
of thermal equilibrium with the SM bath, therefore the number density of the
dilaton $n_{\sigma}$ is calculated through the SM annihilation and through
inverse decays of the SM fields. The dominant contribution to the dilaton
production is due to scattering processes involving top-Yukawa coupling
(2.10), i.e. $y_{t}Q_{L}\widetilde{H}t_{R}$, see also [21, 23]. The condition
for dilaton to be in thermal equilibrium reads as,

$1<\frac{\langle\Gamma_{{\rm SM}\to\sigma}\rangle}{H}\sim
0.04\frac{y_{t}^{2}T^{3}}{f^{2}H}\sim
0.01\frac{M_{\text{Pl}}}{f^{2}}\begin{dcases}\frac{T_{\rm
rh}^{2}}{T},\qquad&T\geq T_{\rm rh}\\\ T,&T<T_{\rm rh}\end{dcases}$ (4.20)

which maximizes for $T=T_{\rm rh}$ and the thermalization condition becomes
$T_{\rm rh}\gtrsim 10^{2}f^{2}/M_{\text{Pl}}$.

###  4.3 Numerical Analysis

Making use of the results from the previous section our goal is to solve the
Boltzmann equation (4.6) to calculate the comoving number density and thus the
relic abundance using Eq. (4.7). Before discussing the numerical results, we
observe that our model contains six free parameters: $m_{\\!X}$, $m_{\sigma}$,
$f$, $H_{I}$, $T_{\rm rh}$, and $w$. Note, however, that dilaton mass cannot
be arbitrarily smaller than the scale invariance breaking scale $f$.
Naturalness suggests the dilaton mass of the same order as breaking scale $f$,
therefore $m_{\sigma}/f$ shows the amount of tuning. In the following, we show
the dependence of dark matter comoving number density w.r.t temperature
evolution for various choices of these parameters.

#### Dark matter mass $m_{\\!X}$ dependance:

In Fig. 2, we present $N_{\\!X}$ as a function of $T_{\rm rh}/T$ for different
choices of DM mass for scalar/vector DM (left-panel) and fermion DM (right-
panel). We fix the remaining parameters as: $w=0$, $f=10^{14}\;\hbox{GeV}$,
$H_{I}=10^{13}\;\hbox{GeV}$ and $T_{\rm rh}=10^{10}\;\hbox{GeV}$, with
$m_{\sigma}/f=1\%$. For these choice of parameters maximum temperature
obtained is $T_{\rm max}\sim 10^{3}T_{\rm rh}$. In this figure, we show
$T=m_{\sigma}=10^{12}\;\hbox{GeV}$ with orange dashed vertical line, to the
left of this line $T>m_{\sigma}$ and hence dilaton can be produced on-shell
via inverse decays of SM fields, and therefore its decays to DM (for
$m_{\sigma}>2m_{\\!X}$) are the dominant source of DM production. However, for
temperatures smaller than dilaton mass $T<m_{\sigma}$, the dominant source of
DM production is SM annihilation to DM through an effective dimension-8
operator, where s-channel dilaton is integrated out. In Fig. 2 (left-panel)
for scalar/vector DM illustrates these features, where comoving number density
acquires maximum value for temperatures $m_{\sigma}>T>T_{\rm rh}$ for DM
masses $m_{\\!X}<T_{\rm max}$. We consider three illustrative values for
$m_{\\!X}=T_{\rm rh},m_{\sigma},2m_{\sigma}$, however, we note that DM mass
$m_{\\!X}<T_{\rm rh}$, the results are same as that of $m_{\\!X}=T_{\rm rh}$
case.

The right-panel of Fig. 2 shows the case when DM is a Dirac fermion. We
consider the three values of DM mass $m_{\\!X}=T_{\rm
rh},m_{\sigma},2m_{\sigma}$ shown as solid, long-dashed, and dashed curves,
respectively. In this case, we also note that maximum comoving number density
is attained when $m_{\sigma}>T>T_{\rm rh}$. As mentioned above, the cross-
section is proportional to fermionic DM mass therefore for dark matter mass
smaller than $T_{\rm rh}$ the DM number density scales as $m_{\\!X}^{2}/T_{\rm
rh}^{2}$ at the end of reheating. Note that once the DM mass becomes larger
than the temperature, i.e. $m_{\\!X}>T$, the number density becomes
exponentially suppressed due to Boltzmann suppression.

Figure 2: Comoving number density $N_{\\!X}$ for scalar/vector DM (left-panel)
and Dirac fermion DM (right-panel) as function of $T_{\rm rh}/T$ for different
choices DM mass $m_{\\!X}$.

#### Dilaton mass $m_{\sigma}$ dependance:

For the case of scalar/vector DM (left-panel) and fermion DM (right-panel) in
Fig. 3, we illustrate the dependance of comoving number density as a function
of temperature for various values of dilaton mass $m_{\sigma}$ with fixed
$m_{\\!X}=T_{\rm rh}=10^{10}\;\hbox{GeV}$, $f=10^{14}\;\hbox{GeV}$,
$H_{I}=10^{13}\;\hbox{GeV}$ and $w=0$. We consider three choices for the
dilaton mass $m_{\sigma}/T_{\rm rh}=100,10,1$, whereas for $m_{\sigma}/T_{\rm
rh}<1$ the comoving number density remains almost same as for the case
$m_{\sigma}/T_{\rm rh}=1$. Again one can notice the scaling of $N_{\\!X}$
w.r.t. $T$ for $T>m_{\sigma}$ where the dilaton can be produced on-shell and
for the case when $T<m_{\sigma}$ where the dilaton can be effectively
integrated out. Furthermore, due to naturalness one expects that the dilaton
mass of the similar order as the conformal breaking scale $f$.

Figure 3: Comoving number density $N_{\\!X}$ for scalar/vector DM (left-panel)
and Dirac fermion DM (right-panel) as function of $T_{\rm rh}/T$ for different
choices dilaton mass $m_{\sigma}$.

#### Dependance on equation of state during reheating $w$:

The non-trivial dependance on the equation of state $w$ during reheating is
given in Eq. (4.6) as $\big{(}T_{\rm rh}/T\big{)}^{(7-w)/(1+w)}$. This shows
suppression in the number density with increasing $w$. In Fig. 4 illustrates
this feature where comoving number density $N_{\\!X}$ is shown as a function
of $T_{\rm rh}/T$ for $w=0,1/3,2/3,1$ with fixed values of
$m_{\\!X}=1\;\hbox{GeV},m_{\sigma}=10^{12}\;\hbox{GeV},f=10^{14}\;\hbox{GeV}$,
and $T_{\rm rh}=10^{10}\;\hbox{GeV}$ for the scalar/vector DM (left-panel) and
fermion DM (right-panel). It is clear that non-standard cosmological effects
are only relevant for temperatures $T<T_{\rm rh}$, i.e. during the reheating
phase.

Figure 4: Comoving number density $N_{\\!X}$ for scalar/vector DM (left-panel)
and Dirac fermion DM (right-panel) as function of $T_{\rm rh}/T$ for different
choices of equation of state $w$.

#### Dark matter relic abundance:

There are six free parameters in our model namely, $m_{\\!X}$, $m_{\sigma}$,
$f$, $H_{I}$, $T_{\rm rh}$, and $w$. In the following, we present regions of
parameter space in the DM mass $m_{\\!X}$ vs the reheating temperature $T_{\rm
rh}$ with fixed choices of all the remaining parameters such that we produced
to observed DM relic abundance $\Omega_{\\!X}h^{2}=0.12$. In particular, in
all the analysis we choose the Hubble scale at the end of inflation
$H_{e}=10^{13}\;\hbox{GeV}$ and fix the ratio of dilaton mass to conformal
breaking scale $m_{\sigma}/f=1\%$. Whereas the value of conformal breaking
scale fixes the observed DM relic abundance for a specific choice of the
equation of state parameter $w$, which we take $w=0,1/3,2/3,1$.

In Fig. 5, we present contours of observed DM relic abundance for the case of
scalar/vector DM in the $T_{\rm rh}$ vs $m_{\chi}$ plane for different values
of equation of state $w$ and $f$ with fixed values of $m_{\sigma}/f=1\%$ and
$H_{I}=10^{13}\;\hbox{GeV}$. The gray shaded region represents parameter space
where the DM would be in thermal equilibrium with the SM bath. The orange
shaded region represents the parameter space where $m_{\\!X}>T_{\rm max}$ and
dashed-orange line denotes $m_{\\!X}=T_{\rm rh}$. The three panels of this
figure show the results for conformal breaking scale
$f=10^{14},10^{12},10^{10}\;\hbox{GeV}$ from left to right, respectively. The
curves show observed DM abundance for DM masses from $10^{-6}\;\hbox{GeV}$ to
$10^{12}\;\hbox{GeV}$ as a function of the reheating temperature. Note that
for a low DM mass $m_{\chi}$ we require larger $T_{\rm rh}$ to produce desired
DM abundance, whereas for the high DM mass $m_{\chi}$ we require smaller
$T_{\rm rh}$. The lower values of $T_{\rm rh}$ imply that the reheating phase
lasts longer and therefore, the impact of non-standard cosmology during this
phase has a significant effect on the DM production. This effect is manifestly
shown in Fig. 5. Note that the dip in these curves for large DM masses is
around $m_{\\!X}\sim m_{\sigma}$ as the temperature dependence of the source
term changes as shown in Eq. (4.11) for the cases of scalar/vector DM.

Figure 5: Contours of observed DM relic abundance for scalar/vector DM as
function of $m_{\\!X}$ and $T_{\rm rh}$ for different choices equation of
state parameter $w$.

In Fig. 6 we show the fermionic DM results for parameters in $m_{\\!X}-T_{\rm
rh}$ plane which produce observed DM relic abundance for different choices of
$w$ and $f$, whereas we fix $m_{\sigma}/f=1\%$ and
$H_{I}=10^{13}\;\hbox{GeV}$. Similar to Fig. 5, in Fig. 6 the gray shaded
region represents the DM in thermal equilibrium with the SM bath and the
orange shaded region shows the DM mass larger than the maximum temperature of
the thermal bath, i.e. $m_{\\!X}>T_{\rm max}$. Note that in the case of
fermionic DM, unlike the bosonic DM case, the scattering cross-section is
proportional to DM mass squared. Therefore the DM thermalization condition is
proportional to DM mass, for larger masses the cross-section is large and
hence it is thermalized, whereas the constraint weakens for lower DM masses.
Furthermore, due to the same DM mass dependence notice that for larger values
of conformal breaking scaling, e.g. $f=10^{14}\;\hbox{GeV}$ (left-panel) the
observed DM abundance can only be achieved for DM mass
$m_{\\!X}>10^{6}\;\hbox{GeV}$, in contrast to the scalar/vector DM case where
sub-GeV masses are also allowed. Note that the effect of non-standard
cosmology during the reheating phase is very similar to that of the
scalar/vector DM case, i.e. for smaller values of reheating temperature the
duration of reheating phase is larger and therefore the effect of the equation
of state $w$ during this phase is significant. However, for larger reheating
temperatures the effect is irrelevant.

Figure 6: Contours of observed DM relic abundance for fermionic DM as function
of $m_{\\!X}$ and $T_{\rm rh}$ for different choices equation of state
parameter $w$.

##  5 Conclusions

In this work, we have studied the implication of non-standard cosmology during
reheating on the ultraviolet freeze-in production of DM via the dilaton
portal. We assume the SM and DM are part of a (strongly coupled)
conformal/scale-invariant theory, where the scale invariance is broken
spontaneously at scale $f$. As a result of spontaneous breaking of scale
invariance low-energy effective theory, below scale $\Lambda=4\pi f$, contains
a dilaton field $\sigma$. Such that the dilaton field naturally couples to the
SM and DM. This coupling is higher dimensional and therefore it is suppressed
by the breaking scale $f$. Furthermore, we assume in the low-energy effective
theory the SM interacts with the DM only through the dilaton portal coupling.
In this scenario, the lowest dimensional interaction between the SM and DM is
a dimension-six operator. For high scale conformal breaking $f$, such
interaction is naturally very small and as a result, the DM is not in thermal
equilibrium with the SM bath. Therefore, the DM can only be produced in the
early universe through the freeze-in mechanism. There are two production
processes through which the DM can be produced: (i) through the annihilation
of SM particles to DM, and (ii) through direct decays of dilaton field to DM
when the dilaton mass is larger than twice the DM mass. We studied the cases
when the DM is a scalar, vector, or a (Dirac) fermion field.

The ultraviolet freeze-in production of DM is highly sensitive to the maximum
temperature of the SM bath particles as well as the reheating dynamics. We
have parameterized the reheating dynamics with three parameters, (a) Hubble
scale at the end of inflation $H_{I}$ which determines inflaton energy density
at the end of inflation, (b) the equation of state $w$ of the inflaton field
during reheating, and (c) the reheating temperature $T_{\rm rh}$ when the
inflaton energy density is equal to that of the SM, therefore it determines
the duration of reheating. The dilaton portal dynamics involve two parameters,
the dilaton mass $m_{\sigma}$, and the conformal breaking scale $f$. Apart
from these five parameters, DM mass $m_{\\!X}$ (for scalar/vector/fermion) is
the only remaining free parameter. In this study, we considered a high scale
inflationary scenario, where the Hubble scale at the end of inflation is fixed
to be, $H_{I}=10^{13}\;\hbox{GeV}$. With this choice of $H_{I}$, assuming
instantaneous thermalization the maximum temperature attained by the SM bath
is $T_{\rm max}\sim 10^{15}\;\hbox{GeV}\times\sqrt{T_{\rm
rh}/(10^{15}\;\hbox{GeV})}$, see Eq. (3.11). For the equation of state
parameter, we choose $w=0,1/3,2/3,1$, where $w=0$ and $w=1/3$ corresponds to
the matter-dominated and radiation-dominated reheating which are achieved for
quadratic and quartic inflaton potentials, respectively. Present the
dependence of the DM number density as a function of temperature on various
parameters of the model in Figs. 2–4.

In Fig. 5 and Fig. 6, we present the results in the $T_{\rm rh}-m_{\\!X}$
plane for the scalar/vector and fermion DM cases, respectively. We consider
high scale conformal breaking scale $f=10^{14},10^{12},10^{10}\;\hbox{GeV}$
and we fix the dilaton mass $m_{\sigma}/f=1\%$. It is shown that DM production
is sensitive to the conformal breaking scale $f$ as the production cross-
section scales as $1/f^{4}$. To realize DM freeze-in production mechanism, one
requires high scale conformal breaking. For the case of scalar/vector DM (Fig.
5), the observed DM abundance can be achieved for DM masses in the range
$10^{-5}-10^{12}\;\hbox{GeV}$ depending on the reheating temperature $T_{\rm
rh}$ and breaking scale $f$. Whereas, for the case of fermion DM (Fig. 6), the
observed DM abundance can be achieved for DM masses in the range
$10^{2}-10^{12}\;\hbox{GeV}$ depending on the reheating temperature $T_{\rm
rh}$ and breaking scale $f$. We conclude that the dilaton portal offers a
natural realization of ultraviolet freeze-in production of DM for a wide DM
mass range and its sensitivity on the reheating dynamics is investigated.

## Acknowledgements

The research of SN is supported by the Cluster of Excellence Precision
Physics, Fundamental Interactions and Structure of Matter (PRISMA+ – EXC
2118/1) within the German Excellence Strategy (project ID 39083149).

## References

  * [1] R. Contino, “The Higgs as a Composite Nambu-Goldstone Boson,” in Theoretical Advanced Study Institute in Elementary Particle Physics: Physics of the Large and the Small, pp. 235–306.  2011\.  arXiv:1005.4269.
  * [2] G. Panico and A. Wulzer, The Composite Nambu-Goldstone Higgs, vol. 913.  Springer, 2016.  arXiv:1506.01961.
  * [3] M. Frigerio, A. Pomarol, F. Riva, and A. Urbano, “Composite Scalar Dark Matter,” JHEP 07 (2012) 015, [arXiv:1204.2808].
  * [4] R. Balkin, M. Ruhdorfer, E. Salvioni, and A. Weiler, “Charged Composite Scalar Dark Matter,” JHEP 11 (2017) 094, [arXiv:1707.07685].
  * [5] L. Da Rold and A. N. Rossia, “The Minimal Simple Composite Higgs Model,” JHEP 12 (2019) 023, [arXiv:1904.02560].
  * [6] A. Ahmed, S. Najjari, and C. B. Verhaaren, “A Minimal Model for Neutral Naturalness and pseudo-Nambu-Goldstone Dark Matter,” JHEP 06 (2020) 007, [arXiv:2003.08947].
  * [7] G. Arcadi, M. Dutra, P. Ghosh, M. Lindner, Y. Mambrini, M. Pierre, S. Profumo, and F. S. Queiroz, “The waning of the WIMP? A review of models, searches, and constraints,” Eur. Phys. J. C 78 no. 3, (2018) 203, [arXiv:1703.07364].
  * [8] A. Salam and J. A. Strathdee, “Nonlinear realizations. 2. Conformal symmetry,” Phys. Rev. 184 (1969) 1760–1768.
  * [9] W. D. Goldberger, B. Grinstein, and W. Skiba, “Distinguishing the Higgs boson from the dilaton at the Large Hadron Collider,” Phys. Rev. Lett. 100 (2008) 111802, [arXiv:0708.1463].
  * [10] C. Csaki, J. Hubisz, and S. J. Lee, “Radion phenomenology in realistic warped space models,” Phys. Rev. D76 (2007) 125015, [arXiv:0705.3844].
  * [11] Z. Chacko and R. K. Mishra, “Effective Theory of a Light Dilaton,” Phys. Rev. D87 no. 11, (2013) 115006, [arXiv:1209.3022].
  * [12] B. Bellazzini, C. Csaki, J. Hubisz, J. Serra, and J. Terning, “A Higgslike Dilaton,” Eur. Phys. J. C73 no. 2, (2013) 2333, [arXiv:1209.3299].
  * [13] A. Ahmed, B. M. Dillon, and S. Najjari, “Dilaton portal in strongly interacting twin Higgs models,” JHEP 02 (2020) 124, [arXiv:1911.05085].
  * [14] Y. Bai, M. Carena, and J. Lykken, “Dilaton-assisted Dark Matter,” Phys. Rev. Lett. 103 (2009) 261803, [arXiv:0909.1319].
  * [15] H. M. Lee, M. Park, and V. Sanz, “Gravity-mediated (or Composite) Dark Matter,” Eur. Phys. J. C74 (2014) 2715, [arXiv:1306.4107].
  * [16] K. Blum, M. Cliche, C. Csaki, and S. J. Lee, “WIMP Dark Matter through the Dilaton Portal,” JHEP 03 (2015) 099, [arXiv:1410.1873].
  * [17] A. Efrati, E. Kuflik, S. Nussinov, Y. Soreq, and T. Volansky, “Constraining the Higgs-Dilaton with LHC and Dark Matter Searches,” Phys. Rev. D91 no. 5, (2015) 055034, [arXiv:1410.2225].
  * [18] M. Kim, S. J. Lee, and A. Parolini, “WIMP Dark Matter in Composite Higgs Models and the Dilaton Portal,” arXiv:1602.05590.
  * [19] A. Ahmed, A. Mariotti, and S. Najjari, “A light dilaton at the LHC,” JHEP 05 (2020) 093, [arXiv:1912.06645].
  * [20] B. Fuks, M. D. Goodsell, D. W. Kang, P. Ko, S. J. Lee, and M. Utsch, “Heavy dark matter through the dilaton portal,” JHEP 10 (2020) 044, [arXiv:2007.08546].
  * [21] P. Brax, K. Kaneta, Y. Mambrini, and M. Pierre, “Metastable Conformal Dark Matter,” Phys. Rev. D 103 no. 11, (2021) 115016, [arXiv:2103.02615].
  * [22] P. Brax, C. van de Bruck, and S. Trojanowski, “Co-interacting dark matter and conformally coupled light scalars,” arXiv:2112.12264.
  * [23] K. Kaneta, P. Ko, and W.-I. Park, “Conformal portal to dark matter,” Phys. Rev. D 104 no. 7, (2021) 075018, [arXiv:2106.01923].
  * [24] B. Barman, N. Bernal, A. Das, and R. Roshan, “Non-minimally Coupled Vector Boson Dark Matter,” arXiv:2108.13447.
  * [25] L. J. Hall, K. Jedamzik, J. March-Russell, and S. M. West, “Freeze-In Production of FIMP Dark Matter,” JHEP 03 (2010) 080, [arXiv:0911.1120].
  * [26] N. Bernal, M. Heikinheimo, T. Tenkanen, K. Tuominen, and V. Vaskonen, “The Dawn of FIMP Dark Matter: A Review of Models and Constraints,” Int. J. Mod. Phys. A 32 no. 27, (2017) 1730023, [arXiv:1706.07442].
  * [27] F. Elahi, C. Kolda, and J. Unwin, “UltraViolet Freeze-in,” JHEP 03 (2015) 048, [arXiv:1410.6157].
  * [28] G. F. Giudice, E. W. Kolb, and A. Riotto, “Largest temperature of the radiation era and its cosmological implications,” Phys. Rev. D 64 (2001) 023508, [hep-ph/0005123].
  * [29] K. Harigaya, M. Kawasaki, K. Mukaida, and M. Yamada, “Dark Matter Production in Late Time Reheating,” Phys. Rev. D 89 no. 8, (2014) 083532, [arXiv:1402.2846].
  * [30] S.-L. Chen and Z. Kang, “On UltraViolet Freeze-in Dark Matter during Reheating,” JCAP 05 (2018) 036, [arXiv:1711.02556].
  * [31] E. W. Kolb and A. J. Long, “Superheavy dark matter through Higgs portal operators,” Phys. Rev. D 96 no. 10, (2017) 103540, [arXiv:1708.04293].
  * [32] A. Ahmed, B. Grzadkowski, and A. Socha, “Production of Purely Gravitational Vector Dark Matter,” Acta Phys. Polon. B 50 (2019) 1809–1821.
  * [33] M. Chianese, B. Fu, and S. F. King, “Impact of Higgs portal on gravity-mediated production of superheavy dark matter,” JCAP 06 (2020) 019, [arXiv:2003.07366].
  * [34] M. A. G. Garcia, K. Kaneta, Y. Mambrini, and K. A. Olive, “Reheating and Post-inflationary Production of Dark Matter,” Phys. Rev. D 101 no. 12, (2020) 123507, [arXiv:2004.08404].
  * [35] A. Ahmed, B. Grzadkowski, and A. Socha, “Gravitational production of vector dark matter,” JHEP 08 (2020) 059, [arXiv:2005.01766].
  * [36] Y. Mambrini and K. A. Olive, “Gravitational Production of Dark Matter during Reheating,” Phys. Rev. D 103 no. 11, (2021) 115009, [arXiv:2102.06214].
  * [37] B. Barman, D. Borah, and R. Roshan, “Nonthermal leptogenesis and UV freeze-in of dark matter: Impact of inflationary reheating,” Phys. Rev. D 104 no. 3, (2021) 035022, [arXiv:2103.01675].
  * [38] M. A. G. Garcia, K. Kaneta, Y. Mambrini, K. A. Olive, and S. Verner, “Freeze-in from Preheating,” arXiv:2109.13280.
  * [39] A. Ahmed, B. Grządkowski, and A. Socha, “Implications of time-dependent inflaton decay on reheating and dark matter production,” arXiv:2111.06065.
  * [40] Planck Collaboration, N. Aghanim et al., “Planck 2018 results. VI. Cosmological parameters,” Astron. Astrophys. 641 (2020) A6, [arXiv:1807.06209]. [Erratum: Astron.Astrophys. 652, C4 (2021)].
  * [41] S. Sarkar, “Big bang nucleosynthesis and physics beyond the standard model,” Rept. Prog. Phys. 59 (1996) 1493–1610, [hep-ph/9602260].


\e \sim&~\text{clip}(\N(0,\sigma^2), -c, c).
\end{align}

(<ref>) Ablation 10: TD3 with clipping. We add value clipping to the loss function of TD3.

(<ref>) Ablation 11: No normalization. We remove the use of AvgL1Norm from TD7. This is identical to no normalization in the design study in <ref>, except this version includes policy checkpoints (the design study does not use checkpoints).

(<ref>) Ablation 12: No fixed encoder. We remove the fixed embeddings from the value function and the policy, this means the networks use embeddings from the current encoder, rather than the fixed encoder from the previous iteration. This makes the input: $Q_{t+1}(z_{t+1}^{sa}, z^s_{t+1}, s, a)$ and $\pi_{t+1}(z^s_{t+1}, s)$. This is identical to no normalization in the design study in <ref>, except this version includes policy checkpoints (the design study does not use checkpoints).

(<ref>) Ablation 13: No implementation. We remove the implementation details differences discussed in <ref>, namely using both value functions when updating the policy, ELU activation functions in the value function network. The target network update is not changed as this may influence the use of fixed encoders.

(<ref>) Ablation 14: Our TD3. TD3 with the implementation detail differences discussed in <ref>, namely using both value functions when updating the policy, ELU activation functions in the value function network, and updating the target network every $250$ time steps, rather than using Polyak averaging.

Average performance on the MuJoCo benchmark at 1M time steps. $\pm$ captures a $95\%$ confidence interval around the average performance. Results are over $10$ seeds.

Algorithm     HalfCheetah     Hopper     Walker2d     Ant     Humanoid

TD7     17434 $\pm$ 155      3512 $\pm$ 315      6097 $\pm$ 570      8509 $\pm$ 422      7429 $\pm$ 153

TD3     10574 $\pm$ 897      3226 $\pm$ 315      3946 $\pm$ 292      3942 $\pm$ 1030      5165 $\pm$ 145

No SALE     12981 $\pm$ 261      3536 $\pm$ 65      5237 $\pm$ 376      5296 $\pm$ 1336      6263 $\pm$ 289

No SALE, with encoder     15639 $\pm$ 548      3544 $\pm$ 371      5350 $\pm$ 427      6003 $\pm$ 920      5480 $\pm$ 83

TD3 with encoder     12495 $\pm$ 813      1750 $\pm$ 302      4226 $\pm$ 491      5255 $\pm$ 579      5082 $\pm$ 317

No checkpoints     17123 $\pm$ 296      3361 $\pm$ 429      5718 $\pm$ 308      8605 $\pm$ 1008      7381 $\pm$ 172

Current policy     17420 $\pm$ 273      2940 $\pm$ 636      5765 $\pm$ 800      8748 $\pm$ 397      7162 $\pm$ 274

TD3 with checkpoints     10255 $\pm$ 656      3414 $\pm$ 77      3266 $\pm$ 474      3843 $\pm$ 749      5349 $\pm$ 72

No LAP     17347 $\pm$ 207      3697 $\pm$ 144      6382 $\pm$ 339      6571 $\pm$ 1504      8082 $\pm$ 260

TD3 with LAP     10324 $\pm$ 1159      3117 $\pm$ 554      4127 $\pm$ 330      4310 $\pm$ 1150      5090 $\pm$ 190

No clipping     17378 $\pm$ 100      3762 $\pm$ 118      6198 $\pm$ 289      7695 $\pm$ 497      7251 $\pm$ 274

TD3 with clipping     10283 $\pm$ 422      2969 $\pm$ 682      3990 $\pm$ 258      3711 $\pm$ 799      5254 $\pm$ 203

No normalization     17391 $\pm$ 275      3640 $\pm$ 95      6256 $\pm$ 317      7807 $\pm$ 266      4829 $\pm$ 1809

No fixed encoder     17145 $\pm$ 138      3710 $\pm$ 120      5869 $\pm$ 531      8287 $\pm$ 379      7412 $\pm$ 337

No implementation     17334 $\pm$ 99      3750 $\pm$ 136      5819 $\pm$ 71      7756 $\pm$ 704      7042 $\pm$ 259

Our TD3     11068 $\pm$ 1399      2791 $\pm$ 632      4179 $\pm$ 297      5489 $\pm$ 448      5186 $\pm$ 108

sb_blue TD7   sb_orange TD3   sb_green No SALE   sb_red No SALE, with encoder   sb_purple TD3 with encoder

SALE. Learning curves on the MuJoCo benchmark, varying the usage of SALE. Results are averaged over $10$ seeds. The shaded area captures a $95\%$ confidence interval around the average performance.

sb_blue TD7   sb_orange TD3   sb_green No checkpoints   sb_red Current policy   sb_purple TD3 with checkpoints

Checkpoints. Learning curves on the MuJoCo benchmark, varying the usage of policy checkpoints. Results are averaged over $10$ seeds. The shaded area captures a $95\%$ confidence interval around the average performance.

sb_blue TD7   sb_orange TD3   sb_green No LAP   sb_red TD3 with LAP

figureLAP. Learning curves on the MuJoCo benchmark, varying the usage of LAP. Results are averaged over $10$ seeds. The shaded area captures a $95\%$ confidence interval around the average performance.

sb_blue TD7   sb_orange TD3   sb_green No clipping   sb_red TD3 with clipping

figureClipping. Learning curves on the MuJoCo benchmark, varying the usage of our proposed value clipping for mitigating extrapolation error. Results are averaged over $10$ seeds. The shaded area captures a $95\%$ confidence interval around the average performance.

sb_blue TD7   sb_orange TD3   sb_green No normalization   sb_red No fixed encoder

figureSALE components. Learning curves on the MuJoCo benchmark, removing components of SALE. Results are averaged over $10$ seeds. The shaded area captures a $95\%$ confidence interval around the average performance.

sb_blue TD7   sb_orange TD3   sb_green No implementation   sb_red Our TD3

figureImplementation differences. Learning curves on the MuJoCo benchmark, varying the minor implementation details between TD3 and TD7. Results are averaged over $10$ seeds. The shaded area captures a $95\%$ confidence interval around the average performance.

§ OFFLINE RL LEARNING CURVES

[trim axis right, trim axis left]
x tick label style=yshift=2pt,
y tick label style=xshift=2pt,
ylabel style=yshift=-3pt,
ylabel=Normalized Score,
xlabel=Time steps (1M),
xtick=0, 0.5, 1.0,
xticklabels=0, 0.5, 1.0,
ytick=0, 20, 40, 60, 80, 100, 120,
yticklabels=0, 20, 40, 60, 80, 100, 120,
graphics [
ymin=-3.6, ymax=123.6,
xmin=-0.06, xmax=1.06,

[trim axis right]
x tick label style=yshift=2pt,
y tick label style=xshift=2pt,
xlabel=Time steps (1M),
xtick=0, 0.5, 1.0,
xticklabels=0, 0.5, 1.0,
ytick=0, 20, 40, 60, 80, 100, 120,
yticklabels=0, 20, 40, 60, 80, 100, 120,
graphics [
ymin=-3.6, ymax=123.6,
xmin=-0.06, xmax=1.06,

[trim axis right]
x tick label style=yshift=2pt,
y tick label style=xshift=2pt,
xlabel=Time steps (1M),
xtick=0, 0.5, 1.0,
xticklabels=0, 0.5, 1.0,
ytick=0, 20, 40, 60, 80, 100, 120,
yticklabels=0, 20, 40, 60, 80, 100, 120,
graphics [
ymin=-3.6, ymax=123.6,
xmin=-0.06, xmax=1.06,

[trim axis right, trim axis left]
x tick label style=yshift=2pt,
y tick label style=xshift=2pt,
ylabel style=yshift=-3pt,
ylabel=Normalized Score,
xlabel=Time steps (1M),
xtick=0, 0.5, 1.0,
xticklabels=0, 0.5, 1.0,
ytick=0, 20, 40, 60, 80, 100, 120,
yticklabels=0, 20, 40, 60, 80, 100, 120,
graphics [
ymin=-3.6, ymax=123.6,
xmin=-0.06, xmax=1.06,

[trim axis right]
x tick label style=yshift=2pt,
y tick label style=xshift=2pt,
xlabel=Time steps (1M),
xtick=0, 0.5, 1.0,
xticklabels=0, 0.5, 1.0,
ytick=0, 20, 40, 60, 80, 100, 120,
yticklabels=0, 20, 40, 60, 80, 100, 120,
graphics [
ymin=-3.6, ymax=123.6,
xmin=-0.06, xmax=1.06,

[trim axis right]
x tick label style=yshift=2pt,
y tick label style=xshift=2pt,
xlabel=Time steps (1M),
xtick=0, 0.5, 1.0,
xticklabels=0, 0.5, 1.0,
ytick=0, 20, 40, 60, 80, 100, 120,
yticklabels=0, 20, 40, 60, 80, 100, 120,
graphics [
ymin=-3.6, ymax=123.6,
xmin=-0.06, xmax=1.06,

[trim axis right, trim axis left]
x tick label style=yshift=2pt,
y tick label style=xshift=2pt,
ylabel style=yshift=-3pt,
ylabel=Normalized Score,
xlabel=Time steps (1M),
xtick=0, 0.5, 1.0,
xticklabels=0, 0.5, 1.0,
ytick=0, 20, 40, 60, 80, 100, 120,
yticklabels=0, 20, 40, 60, 80, 100, 120,
graphics [
ymin=-3.6, ymax=123.6,
xmin=-0.06, xmax=1.06,

[trim axis right]
x tick label style=yshift=2pt,
y tick label style=xshift=2pt,
xlabel=Time steps (1M),
xtick=0, 0.5, 1.0,
xticklabels=0, 0.5, 1.0,
ytick=0, 20, 40, 60, 80, 100, 120,
yticklabels=0, 20, 40, 60, 80, 100, 120,
graphics [
ymin=-3.6, ymax=123.6,
xmin=-0.06, xmax=1.06,

[trim axis right]
x tick label style=yshift=2pt,
y tick label style=xshift=2pt,
xlabel=Time steps (1M),
xtick=0, 0.5, 1.0,
xticklabels=0, 0.5, 1.0,
ytick=0, 20, 40, 60, 80, 100, 120,
yticklabels=0, 20, 40, 60, 80, 100, 120,
graphics [
ymin=-3.6, ymax=123.6,
xmin=-0.06, xmax=1.06,

sb_blue TD7   sb_orange CQL   sb_green TD3+BC   sb_red IQL   sb_purple $\mathcal{X}$-QL

Learning curves on the offline D4RL benchmark. Results are averaged over 10 seeds. The shaded area captures a 95% confidence interval around the average performance.

§ RUN TIME

We benchmark the run time of TD7 and the baseline algorithms. Evaluation episodes were not included in this analysis. Each algorithm is trained using the same deep learning framework, PyTorch [Paszke et al., 2019]. All experiments are run on a single Nvidia Titan X GPU and Intel Core i7-7700k CPU. Software is detailed in <ref>.
We display run time adjusted learning curves in <ref>.

[trim axis right, trim axis left]
x tick label style=yshift=2pt,
y tick label style=xshift=2pt,
ylabel style=yshift=-3pt,
ylabel=Total Reward (1k),
xtick=0, 4, 8, 12, 16, 20,
xticklabels=0, 4, 8, 12, 16, 20,
ytick=0, 4, 8, 12, 16, 20,
yticklabels=0, 4, 8, 12, 16, 20,
graphics [
ymin=-0.6, ymax=20.6,
xmin=-0.57, xmax=19.72,

[trim axis right]
x tick label style=yshift=2pt,
y tick label style=xshift=2pt,
xtick=0, 4, 8, 12, 16, 20,
xticklabels=0, 4, 8, 12, 16, 20,
ytick=0, 1, 2, 3, 4,
yticklabels=0, 1, 2, 3, 4,
graphics [
ymin=-0.135, ymax=4.635,
xmin=-0.53, xmax=18.37,

[trim axis right]
x tick label style=yshift=2pt,
y tick label style=xshift=2pt,
xtick=0, 4, 8, 12, 16, 20,
xticklabels=0, 4, 8, 12, 16, 20,
ytick=0, 2, 4, 6, 8,
yticklabels=0, 2, 4, 6, 8,
graphics [
ymin=-0.24, ymax=8.24,
xmin=-0.54, xmax=18.4,

[trim axis right]
x tick label style=yshift=2pt,
y tick label style=xshift=2pt,
xtick=0, 4, 8, 12, 16, 20,
xticklabels=0, 4, 8, 12, 16, 20,
ytick=0, 3, 6, 9, 12,
yticklabels=0, 3, 6, 9, 12,
graphics [
ymin=-0.36, ymax=12.36,
xmin=-0.56, xmax=19.19,

[trim axis right]
x tick label style=yshift=2pt,
y tick label style=xshift=2pt,
xtick=0, 4, 8, 12, 16, 20,
xticklabels=0, 4, 8, 12, 16, 20,
ytick=0, 2, 4, 6, 8, 10,
yticklabels=0, 2, 4, 6, 8, 10,
graphics [
ymin=-0.33, ymax=11.33,
xmin=-0.59, xmax=20.21,

sb_blue TD7
sb_orange TD3   sb_green SAC   sb_red TQC   sb_purple TD3+OFE

Run time adjusted curves. Learning curves on the MuJoCo benchmark over 5M time steps, where the x-axis is run time. Results are averaged over 10 seeds. The shaded area captures a 95% confidence interval around the average performance.

§ LIMITATIONS

Depth vs. Breadth. Although our empirical study is extensive, the scope of our experimentation emphasizes depth over breadth. There remains many other interesting benchmarks for continuous control, such as examining a setting with image-based observations.

Baselines. We benchmark TD7 against the highest performing agents in both the online and offline setting. However, due to the fact that most representation learning methods do not cover low level states, we only benchmark against a single other method focused on representation learning. While it would be surprising if a method, originally designed for a different setting, outperformed the strongest existing baselines, exploring additional methods could provide new insight into representation learning for low level states.

Computational cost. While TD7 has a lower computational cost than other competing methods, the run time over TD3 (the base algorithm) is more than double.

Theoretical results. We perform extensive empirical analysis to uncover which factors of dynamics-based representation learning are important for performance. However, we do not address the theoretical side of representation learning. There are many important questions regarding why dynamics-based representation learning is effective, or what makes a good representation.