[r_{1},r_{2}]\subset\bigg{(}\tilde{F}_{0}^{-1}(\eta/2),\tilde{F}_{0}^{-1}(1-\eta/2)\bigg{)}\bigg{\\}},$

where $\mathcal{U}_{\eta}$ is the class of non-increasing functions bounded by
$M_{\eta}>0$, i.e.

$\mathcal{U}_{\eta}=\bigg{\\{}h:\mathbb{R}\mapsto[-M_{\eta},M_{\eta}]\
\bigg{|}\ \ h\text{ is non-increasing}\bigg{\\}}.$

Observe that (6.18), (6.19), and (6.28) imply $h_{n}\in\mathcal{H}_{\eta}$
almost surely for all sufficiently large $n$.

From (6.22) and (6.23) we derive that

$\log N_{[\
]}(\epsilon,\mathcal{H}_{\eta},L_{2}(P_{0}))\lesssim\dfrac{1}{\epsilon}.$

Therefore, we conclude that the bracketing integral

$\mathcal{J}_{[\
]}(1,\mathcal{H}_{\eta},L_{2}(P_{0}))=\displaystyle\int_{0}^{1}\sqrt{1+\log
N_{[]}(M_{\eta}\epsilon,\mathcal{H}_{\eta},L_{2}(P_{0}))}d\epsilon\lesssim\displaystyle\int_{0}^{1}\epsilon^{-1/2}d\epsilon,$

which is finite, thereby establishing that $\mathcal{H}_{\eta}$ is
$P_{0}$-Donsker. Also, notice that $|h_{n}|$ is bounded above by $M_{\eta}$,
and Theorem 12(C) indicates that $h_{n}$ converges to $0$ at the continuity
points of $\tilde{\phi}_{0}^{\prime}$. Since $\tilde{\phi}_{0}$ is concave,
$\tilde{\phi}_{0}^{\prime}$ is continuous almost everywhere with respect to
Lebesgue measure on $\mathrm{dom}(\tilde{\phi}_{0})$. Hence, the dominated
convergence theorem yields

$\displaystyle\int_{-\infty}^{\infty}h_{n}(x)^{2}dF_{0}(x)\to_{a.s.}0.$

Since $\mathcal{H}_{\eta}$ is Donsker, Theorem 2.1 of van der Vaart and
Wellner (2007) then implies that

$\displaystyle\int_{-\infty}^{\infty}h_{n}(x)d\mathbb{Z}_{n}(x)=o_{p}(1).$

Finally, an application of Lemma 6 leads to
$\widehat{\mathcal{I}}_{n}(\eta)\to_{a.s.}\mathcal{I}_{f_{0}}(\eta)$, and thus
from (6.1) we conclude that $\sqrt{n}T_{1n}=o_{p}(1)$.

Now we turn our attention to $T_{2n}$. Here we aim to prove (6.25). Observe
that $T_{2n}/\tilde{\delta}_{n}$ can be written as

$\displaystyle\displaystyle\int_{-\xi_{n}}^{\xi_{n}}\dfrac{\tilde{\psi}^{\prime}_{n}(z)}{\widehat{\mathcal{I}}_{n}(\eta)}\dfrac{\bigg{(}g_{0}(z-\tilde{\delta}_{n})-g_{0}(z)\bigg{)}}{\tilde{\delta}_{n}}dz\
=$ $\displaystyle\
\displaystyle\int_{-\xi_{n}}^{\xi_{n}}\dfrac{\tilde{\psi}^{\prime}_{n}(z)}{\widehat{\mathcal{I}}_{n}(\eta)}\dfrac{\displaystyle\int_{z}^{z-\tilde{\delta}_{n}}g_{0}^{\prime}(t)dt}{\tilde{\delta}_{n}}dz$
$\displaystyle=$ $\displaystyle\
\displaystyle\int_{-\xi_{n}-\tilde{\delta}_{n}}^{\xi_{n}}g_{0}^{\prime}(t)\dfrac{\displaystyle\int_{t}^{t+\tilde{\delta}_{n}}\tilde{\psi}^{\prime}_{n}(z)dz}{\tilde{\delta}_{n}\widehat{\mathcal{I}}_{n}(\eta)}dt,$
(6.29)

where the equality follows by Fubini’s theorem since $g_{0}$ is absolutely
continuous. The concavity of $\tilde{\psi}_{n}$ implies that
$\tilde{\psi}^{\prime}_{n}$ is non-increasing. Hence, for any
$t\in\text{int}(\mathrm{dom}(\tilde{\psi}_{0}))$, we have,

$\min\bigg{\\{}\tilde{\psi}^{\prime}_{n}(t),\tilde{\psi}^{\prime}_{n}(t+\tilde{\delta}_{n})\bigg{\\}}\leq\dfrac{\displaystyle\int_{t}^{t+\tilde{\delta}_{n}}\tilde{\psi}^{\prime}_{n}(z)dz}{\tilde{\delta}_{n}}\leq\max\bigg{\\{}\tilde{\psi}^{\prime}_{n}(t),\tilde{\psi}^{\prime}_{n}(t+\tilde{\delta}_{n})\bigg{\\}}.$
(6.30)

Therefore, if $\tilde{\psi}_{0}^{\prime}$ is continuous at
$t\in\text{int}(\mathrm{dom}(\tilde{\psi}_{0}))$, Theorem 12(C) leads to

$\dfrac{\displaystyle\int_{t}^{t+\tilde{\delta}_{n}}\tilde{\psi}^{\prime}_{n}(z)dz}{\tilde{\delta}_{n}}\to_{a.s.}\tilde{\psi}_{0}^{\prime}(t)\quad\text{
as }n\to\infty.$

Since $\tilde{\psi}_{0}$ is concave, $\tilde{\psi}_{0}^{\prime}$ is continuous
almost everywhere with respect to Lebesgue measure on
$\mathrm{dom}(\tilde{\psi}_{0}))$. Since $\tilde{\psi}^{\prime}_{n}$ is non-
increasing, (6.30) further entails that

$\sup_{t\in[-\xi_{n}-\tilde{\delta}_{n},\xi_{n}]}\dfrac{\bigg{|}\displaystyle\int_{t}^{t+\tilde{\delta}_{n}}\tilde{\psi}^{\prime}_{n}(z)dz\bigg{|}}{|\tilde{\delta}_{n}|}\leq\tilde{\psi}^{\prime}_{n}(-\xi_{n}-|\tilde{\delta}_{n}|),$

which can be bounded above by
$\tilde{\psi}_{0}^{\prime}(\tilde{G}_{0}^{-1}(\eta/2))$ using (6.16) and the
fact that $\tilde{\delta}_{n}\to_{a.s.}0$. Hence, we deduce that

$|g_{0}^{\prime}(t)|\dfrac{\bigg{|}\displaystyle\int_{t}^{t+\tilde{\delta}_{n}}\tilde{\psi}^{\prime}_{n}(z)dz\bigg{|}}{|\tilde{\delta}_{n}|}1_{[-\xi_{n}-\tilde{\delta}_{n},\xi_{n}]}(z)\leq|g_{0}^{\prime}(t)|\bigg{(}\tilde{\psi}_{0}^{\prime}(\tilde{G}_{0}^{-1}(\eta/2))\bigg{)}.$

Note that the symmetry of $g_{0}$ about $0$ indicates that

$\displaystyle\int_{-\infty}^{\infty}|g_{0}^{\prime}(t)|dt=2\displaystyle\int_{-\infty}^{0}g_{0}^{\prime}(t)dt=2g(0),$

which is finite because of (3.6). Therefore, (6.1), Lemma 6, and the dominated
convergence theorem yield

$\dfrac{T_{2n}}{\tilde{\delta}_{n}}\to_{a.s.}\displaystyle\int_{-\xi_{0}}^{\xi_{0}}\dfrac{\tilde{\psi}_{0}^{\prime}(z)g_{0}^{\prime}(z)}{\tilde{\mathcal{I}}_{f_{0}}(\eta)}dz,$

which completes the proof of (6.25) since
$g_{0}^{\prime}(z)=\psi_{0}^{\prime}(z)g_{0}(z)$.

Now it only remains to show that the last term $T_{5n}$ converges weakly to a
Gaussian random variable with variance $\tilde{\mathcal{I}}_{f_{0}}^{-1}$.
Observe that a change of variable leads to

$\displaystyle\sqrt{n}T_{5n}=$ $\displaystyle\
\sqrt{n}\int_{\bar{\theta}_{n}-\xi_{n}}^{\bar{\theta}_{n}+\xi_{n}}\dfrac{\tilde{\psi}_{0}^{\prime}(x-\theta_{0})}{\widehat{\mathcal{I}}_{n}(\eta)}d(\mathbb{F}_{n}-F_{0})(x)$
$\displaystyle=$ $\displaystyle\
\int_{\theta_{0}-\xi_{0}}^{\theta_{0}+\xi_{0}}\dfrac{\tilde{\psi}_{0}^{\prime}(x-\theta_{0})}{\widehat{\mathcal{I}}_{n}(\eta)}d\mathbb{Z}_{n}(x)$
$\displaystyle\
+\displaystyle\int_{-\infty}^{\infty}1_{C_{n}}(x)\dfrac{\tilde{\psi}_{0}^{\prime}(x-\theta_{0})}{\widehat{\mathcal{I}}_{n}(\eta)}d\mathbb{Z}_{n}(x),$
(6.31)

where

$C_{n}=[\bar{\theta}_{n}-\xi_{n},\bar{\theta}_{n}+\xi_{n}]\setminus[\theta_{0}-\xi_{0},\theta_{0}+\xi_{0}].$

From Lemma 6 and Slutsky’s theorem it follows that

$\int_{\theta_{0}-\xi_{0}}^{\theta_{0}+\xi_{0}}\dfrac{\tilde{\psi}_{0}^{\prime}(x-\theta_{0})}{\widehat{\mathcal{I}}_{n}(\eta)}d\mathbb{Z}_{n}(x)=\int_{\theta_{0}-\xi_{0}}^{\theta_{0}+\xi_{0}}\dfrac{\tilde{\psi}_{0}^{\prime}(x-\theta_{0})}{\tilde{\mathcal{I}}_{f_{0}}(\eta)}d\mathbb{Z}_{n}(x)(1+o_{p}(1)),$

which equals

$\int_{\theta_{0}-\xi_{0}}^{\theta_{0}+\xi_{0}}\dfrac{\tilde{\psi}_{0}^{\prime}(x-\theta_{0})}{\tilde{\mathcal{I}}_{f_{0}}(\eta)}d\mathbb{Z}_{n}(x)+o_{p}(1),$

because the central limit theorem and (3.3) imply that the first term on the
above display is $O_{p}(1)$.

To deal with the second term, observe that the indicator function $1_{C_{n}}$
belongss to $\mathcal{F}_{1}$, the class of all indicator functions of the
form $1_{[z_{1},z_{2}]\setminus[\theta_{0}-\xi_{0},\theta_{0}+\xi_{0}]}$,
where $z_{1}\leq z_{2}$ with $z_{1},z_{2}\in\mathbb{R}$. Since the latter
class is Donsker by (6.23), Theorem 2.1 of van der Vaart and Wellner (2007)
entails that the second term on the right hand side of (6.1) is of order
$o_{p}(1)$ provided the following holds:

$\displaystyle\int_{-\infty}^{\infty}1_{C_{n}}(x)\dfrac{\tilde{\psi}_{0}^{\prime}(x-\theta_{0})^{2}}{\widehat{\mathcal{I}}_{n}(\eta)^{2}}dF_{0}(x)\to_{a.s.}0.$

To this end note that since $\tilde{\psi}_{0}^{\prime}$ is non-increasing,
(6.18), (6.19), (6.27), and Lemma 6 entail that for all sufficiently large
$n$,

$\displaystyle\int_{-\infty}^{\infty}1_{C_{n}}(x)\dfrac{\tilde{\psi}_{0}^{\prime}(x-\theta_{0})^{2}}{\widehat{\mathcal{I}}_{n}(\eta)^{2}}dF_{0}(x)\leq\dfrac{\tilde{\psi}_{0}^{\prime}(\tilde{G}_{0}^{-1}(\eta/2))^{2}}{\tilde{\mathcal{I}}_{f_{0}}^{2}(\eta)}|\bar{\theta}_{n}-\theta_{0}|\bigg{(}\sup_{x}f_{0}(x)\bigg{)}\quad
a.s.$

Since $f_{0}\in\mathcal{P}_{1}$, from the definition of $\mathcal{P}_{1}$ in
(3.6) and the strong consistency of $\bar{\theta}_{n}$ we obtain that the term
on the right hand side of the last display converges to $0$ almost surely.
Thus we conclude that $\sqrt{n}T_{5n}$ is asymptotically distributed as a
centered Gaussian random variable with variance
$\tilde{\mathcal{I}}_{f_{0}}(\eta)^{-1}$, which completes the proof. $\Box$

###  6.2 Proofs for the MLE

We first state and prove a useful theorem which plays a crucial role in
proving Proposition 1 and Lemma 1. This theorem concerns the continuity of the
function $L(\mathord{\cdot}\ ;F):\mathbb{R}\mapsto\mathbb{R}$ given by

$L(\theta;F)=\sup_{\psi\in\mathcal{SC}_{0}}\Psi(\theta,\psi,F)$ (6.32)

where $F$ is a fixed distribution function, and $\Psi$ is as defined in (1.4).

######  Theorem 13.

Suppose the distribution function $F$ satisfies Condition A. Then the map
$\theta\mapsto L(\theta;F)$ is continuous on $\mathbb{R}$, where the function
$L(\theta;F)$ was defined in (6.32).

###### Proof.

Suppose $\theta_{k}\to\theta$ as $k\to\infty$. Observe that
$\Psi(\theta,\psi,F)$ can also be written as

$\Psi(\theta,\psi,F)=\Psi(0,\psi,F(\mathord{\cdot}+\theta)).$

Hence, to prove Theorem 13, it suffices to show that as $k\to\infty$,

$L(\theta_{k};F)=\sup_{\psi\in\mathcal{SC}_{0}}\Psi(0,\psi,F(\mathord{\cdot}+\theta_{k}))\to\sup_{\psi\in\mathcal{SC}_{0}}\Psi(0,\psi,F(\mathord{\cdot}+\theta))=L(\theta,F).$

Proposition $6$ of Xu and Samworth (2019) implies that under Condition A, the
convergence in the above display holds if the Wasserstein distance

$d_{W}(F(\mathord{\cdot}+\theta_{k}),F(\mathord{\cdot}+\theta))\to
0,\quad\text{ as }k\to\infty.$ (6.33)

Now by Theorem $6.9$ of Villani (2009) (see also Theorem $7.12$ of Villani,
2003), (6.33) follows if the followings hold as $k\to\infty$:

$F(\mathord{\cdot}+\theta_{k})\to_{d}F(\mathord{\cdot}+\theta),$ (6.34)

and

$\displaystyle\int_{-\infty}^{\infty}|x|dF(x+\theta_{k})\to\displaystyle\int_{-\infty}^{\infty}|x|dF(x+\theta).$
(6.35)

To prove (6.34), note that for any bounded continuous function $h$,

$\displaystyle\int_{-\infty}^{\infty}h(x-\theta_{k})dF(x)\to\displaystyle\int_{-\infty}^{\infty}h(x-\theta)dF(x)$

by the dominated convergence theorem since $\theta_{k}\to\theta$ as
$k\to\infty$. For proving (6.35), first notice that $F$ has finite first
moment by Condition A. Therefore, another application of the dominated
convergence yields that as $\theta_{k}\to\theta$,

$\displaystyle\int_{-\infty}^{\infty}|x|dF(x+\theta_{k})=\displaystyle\int_{-\infty}^{\infty}|x-\theta_{k}|dF(x)\to\displaystyle\int_{-\infty}^{\infty}|x|dF(x+\theta),$

which, combined with (6.34), leads to (6.33), and completes the proof.

∎

#### Proof of Proposition 1

Our first step is to show that $L(F)$ is finite. From the definition of $\Psi$
in (1.4), it is not hard to see that

$L(F)\leq\sup_{\psi\in\mathcal{C}}\bigg{(}\displaystyle\int_{-\infty}^{\infty}\psi(x)dF(x)-\displaystyle\int_{-\infty}^{\infty}e^{\psi(x)}dx\bigg{)},$

where $\mathcal{C}$ denotes the set of all real-valued concave functions.
Theorem 2.2 of Dümbgen et al. (2011) entails that under condition A, the term
on the right hand side of the above display is finite. Therefore,
$L(F)<\infty$ follows. To show that $L(F)>-\infty$, we show that the map
$x\mapsto-|x|\in\mathcal{SC}_{0}$. Therefore, (1.4) and (1.6) lead to

$L(F)>-\displaystyle\int_{-\infty}^{\infty}|x|dF(x)-\displaystyle\int_{-\infty}^{\infty}e^{-|x|}dx>-\infty,$

which follows from Condition A. Therefore, we conclude that
$L(F)\in\mathbb{R}$.

Now we have to show that there exist $\theta^{*}(F)\in\mathbb{R}$ and
$\psi^{*}(F)\in\mathcal{SLC}_{0}$ such that

$\Psi(\theta^{*}(F),\psi^{*}(F),F)=\sup_{\theta\in\mathbb{R},\psi\in\mathcal{SLC}_{0}}\Psi(\theta,\psi,F)=\sup_{\theta\in\mathbb{R}}L(\theta;F)=L(F).$

Now there exists a sequence $\\{\theta_{k}\\}_{k\geq 1}$ such that
$L(\theta_{k};F)\uparrow L(F)$ as $k\to\infty$. Suppose the sequence
$\\{\theta_{k}\\}_{k\geq 1}$ is bounded. Then we can find a subsequence
$\\{\theta_{k_{r}}\\}_{r\geq 1}$ converging to some
$\theta^{\prime}\in\mathbb{R}$. Since the map $L(\theta;F)$ is continuous in
$\theta$ by Theorem 13, we also have

$L(\theta^{\prime};F)=\lim_{r\to\infty}L(\theta_{k_{r}};F)=L(F),$

which implies that $\theta^{\prime}$ is a maximizer of $L(\theta;F)$. The rest
of the proof then follows from Proposition 4(iii) of Xu and Samworth (2019),
which states that for each $\theta\in\mathbb{R}$, there exists a unique log-
density ${\psi}_{\theta}$, which maximizes $\Psi(\theta,\psi,F)$ in
$\psi\in\mathcal{SC}_{0}$ when $F$ satisfies condition A. Hence,
$(\theta^{\prime},\psi_{\theta^{\prime}})$ is a candidate for
$(\theta^{*}(F),\psi^{*}(F))$. Thus, to complete the proof, it remains to show
that $\\{\theta_{k}\\}_{k\geq 1}$ is bounded. We will show that
$\theta_{k}\to\pm\infty$ leads to $L(\theta_{k};F)\to-\infty$, which
contradicts the fact that $L(\theta_{k};F)\to L(F)\in\mathbb{R}$, thus
completing the proof.

Consider $\theta_{k}\to\pm\infty$. We have already mentioned that by
Proposition 4(iii) of Xu and Samworth (2019), for each $\theta_{k}$, there
exists a log-density $\psi_{\theta_{k}}\in\mathcal{SC}_{0}$ such that

$L(\theta_{k};F)=\Psi(\theta,\psi_{\theta_{k}},F),$

which equals

$\displaystyle\displaystyle\int_{-\infty}^{\infty}\psi_{\theta_{k}}(x-\theta)dF(x)-1\leq-\displaystyle\int_{-\infty}^{\infty}\log\bigg{(}2|x-\theta_{k}|\bigg{)}dF(x)-1,$

where the last inequality follows from Lemma 12. Now if
$\theta_{k}\to\pm\infty$, using Fatou’s Lemma, we derive that

$\displaystyle\limsup_{k\to\infty}L(\theta_{k};F)\leq-\displaystyle\int_{-\infty}^{\infty}\liminf_{k\to\infty}\bigg{(}\log|x-\theta_{k}|\bigg{)}dF(x)-\log
2=-\infty,$

which leads to the desired contradiction. $\Box$

#### Proof of Lemma 1

The proof follows from (1.5) and Theorem 13 noting that $\mathbb{F}_{n}$
satisfies Condition A. $\Box$

#### Proof of Lemma 9

First we will show that if $F$ satisfies (3.20) and $\psi\in\mathcal{SC}_{0}$,
there exists $\psi_{1}\in\mathcal{SC}_{0}$ such that

$\Psi(\theta^{\prime},\psi,F)\leq\Psi(\theta,\psi_{1},F),$ (6.36)

leading to

$L(\theta^{\prime};F)\leq L(\theta;F),$

which implies $\theta^{*}(F)=\theta$. Note that $G=F(\mathord{\cdot}+\theta)$
satisfies

$G(x)+G(-x)=1.$ (6.37)

Using the symmetry of $\psi$ about $0$ and (6.37) in the fourth step we deduce
that

$\displaystyle\Psi(\theta^{\prime},\psi,F)$ $\displaystyle=$ $\displaystyle\
\displaystyle\int_{-\infty}^{\infty}\psi(x-\theta^{\prime})dG(x-\theta)-\displaystyle\int_{-\infty}^{\infty}e^{\psi(x)}dx$
$\displaystyle=$ $\displaystyle\
\displaystyle\int_{-\infty}^{0}\psi(y+\theta-\theta^{\prime})dG(y)+\displaystyle\int_{0}^{\infty}\psi(y+\theta-\theta^{\prime})dG(y)-\displaystyle\int_{-\infty}^{\infty}e^{\psi(x)}dx$
$\displaystyle=$ $\displaystyle\
\displaystyle\int_{-\infty}^{0}\psi(y+\theta-\theta^{\prime})dG(y)-\displaystyle\int_{-\infty}^{0}\psi(-y+\theta-\theta^{\prime})dG(-y)-\displaystyle\int_{-\infty}^{\infty}e^{\psi(x)}dx$
$\displaystyle=$ $\displaystyle\
\displaystyle\int_{-\infty}^{0}\psi(y+\theta-\theta^{\prime})dG(y)+\displaystyle\int_{-\infty}^{0}\psi(y-\theta+\theta^{\prime})dG(y)-\displaystyle\int_{-\infty}^{\infty}e^{\psi(x)}dx$
$\displaystyle=$ $\displaystyle\
2\displaystyle\int_{-\infty}^{0}\psi_{1}(y)dG(y)-\displaystyle\int_{-\infty}^{\infty}e^{\psi(x)}dx,$

where

$\psi_{1}(x)\ =\
2^{-1}\bigg{(}\psi(x+\theta-\theta^{\prime})+\psi(x-\theta+\theta^{\prime})\bigg{)},\quad
x\in\mathbb{R}.$

Since $\psi_{1}\in\mathcal{SC}_{0}$, we obtain that

$\displaystyle\Psi(\theta^{\prime},\psi,F)\ =$ $\displaystyle\
\displaystyle\int_{-\infty}^{\infty}\psi_{1}(x)dG(x)-\displaystyle\int_{-\infty}^{\infty}e^{\psi(x)}dx$
$\displaystyle=$ $\displaystyle\
\displaystyle\int_{-\infty}^{\infty}\psi_{1}(x-\theta)dF(x)-\displaystyle\int_{-\infty}^{\infty}e^{\psi(x)}dx$
$\displaystyle=$ $\displaystyle\
\Psi(\theta,\psi_{1},F)+\displaystyle\int_{-\infty}^{\infty}e^{\psi_{1}(x)}dx-\displaystyle\int_{-\infty}^{\infty}e^{\psi(x)}dx.$

Also, by the arithmetic mean-geometric mean inequality,

$\displaystyle\int_{-\infty}^{\infty}e^{\psi_{1}(x)}dx\leq\displaystyle\int_{-\infty}^{\infty}2^{-1}\bigg{(}e^{\psi(x+\theta-\theta^{\prime})}+e^{\psi(x-\theta+\theta^{\prime})}\bigg{)}dx=\displaystyle\int_{-\infty}^{\infty}e^{\psi(x)}dx,$

where equality holds if and only if
$\psi(x+\theta-\theta^{\prime})=\psi(x-\theta+\theta^{\prime})$ almost
everywhere with respect to the Lebesgue measure, which only takes place when
$\theta^{\prime}=\theta$. Therefore, we have established the inequality in
(6.36) with equality holding if and only if $\theta^{\prime}=\theta$, which
proves $\theta^{*}(F)=\theta$. Observe that

$\Psi(\theta,\psi,F)=\displaystyle\int_{-\infty}^{\infty}\psi(x)dG(x)-\displaystyle\int_{-\infty}^{\infty}e^{\psi(x)}dx=\Psi(0,\psi,G).$

Finally, Lemma 14 combined with (6.37) and (6.2) entail that

$\operatorname*{arg\,max}_{\psi\in\mathcal{SC}_{0}}\Psi(0,\psi,G)=\operatorname*{arg\,max}_{\phi\in\mathcal{SC}_{0}}\omega(\phi,G)=\operatorname*{arg\,max}_{\phi\in\mathcal{C}}\omega(\phi,G),$

which completes the proof of Lemma 9. $\Box$

#### Proof of Lemma 8

Lemma 8 follows directly from Lemma 4. $\Box$

#### Proof of Lemma 10

Suppose $\overline{J(F)}=\mathbb{R}$. Then $\theta^{*}(F)\in\overline{J(F)}$
by Proposition 1. Otherwise, since $F$ is non-decreasing, $\overline{J(F)}$
takes one of the following three forms: $[a,b]$, $[a,\infty)$, or
$(-\infty,b]$, where $a,b\in\mathbb{R}$. Suppose $\overline{J(F)}=[a,b]$. We
will show that in this case $L(\theta;F)$ defined in (6.32) is non-decreasing
in $\theta$ on $(-\infty,a]$, and non-increasing in $\theta$ on $[b,\infty)$.
Since $\theta^{*}(F)$ equals
$\operatorname*{arg\,max}_{\theta\in\mathbb{R}}L(\theta;F)$, the above implies
$\theta^{*}(F)\in[a,b]$. To show that $L(\theta;F)$ is non-decreasing in
$\theta$ on $(-\infty,a]$, we first note that for $\theta<\theta^{\prime}\leq
a$, and $\psi\in\mathcal{SC}_{0}$,

$\displaystyle\displaystyle\int_{a}^{b}\psi(x-\theta)dF(x)\leq\displaystyle\int_{a}^{b}\psi(x-\theta^{\prime})dF(x),$

since $\psi$ is non-increasing on $[0,\infty)$, and

$0\leq x-\theta^{\prime}<x-\theta$

for $x\geq a$. Therefore, from (6.32), it is not hard to see that
$L(\theta;F)\leq L(\theta^{\prime};F)$. Similarly we can show that for
$\theta>\theta^{\prime}\geq b$,

$\displaystyle\displaystyle\int_{a}^{b}\psi(x-\theta)dF(x)\leq\displaystyle\int_{a}^{b}\psi(x-\theta^{\prime})dF(x),$

since $\psi$ is non-decreasing on $(-\infty,0]$, and
$x-\theta<x-\theta^{\prime}\leq 0$ for $x\leq b$. Therefore, $L(\theta;F)\leq
L(\theta^{\prime};F)$, which completes the proof of $\theta^{*}(F)\in[a,b]$.
When $\overline{J(F)}$ is of the form $[a,\infty)$ or $(-\infty,b]$, we can
prove $\theta^{*}(F)\in\overline{J(F)}$ in a similar way. Hence, the first
part of Lemma 10 is proved. The second part of Lemma 10 follows from Lemma 13.
$\Box$

#### Proof of Theorem 1

First part follows from (1.5) and Proposition 1 noting that $\mathbb{F}_{n}$
satisfies Condition A. The second part follows from Lemma 10. $\Box$

#### Proof of Theorem 10

Since $d_{W}(F_{n},F)\to 0$, from Theorem $6.9$ of Villani (2009) or Theorem
$7.12$ of Villani (2003) it follows that $F_{n}$ converges to $F$ weakly, and

$\displaystyle\int_{-\infty}^{\infty}|x|dF_{n}(x)\to\displaystyle\int_{-\infty}^{\infty}|x|dF(x)<\infty.$
(6.38)

Let us denote the measure corresponding to $F_{n}$ by $P_{n}$.

For the sake of simplicity, we use the abbreviated notations $\psi^{*}_{n}$,
$\phi^{*}_{n}$, and $\theta^{*}_{n}$ in place of $\psi^{*}(F_{n})$,
$\phi^{*}(F_{n})$, and $\theta^{*}(F_{n})$ respectively.

Our proof is motivated by the proof of Proposition $6$ of Xu and Samworth
(2019). Since $F_{n}$ satisfies condition A, $L(F_{n})<\infty$ by Proposition
1. Therefore, starting with any subsequence of $\\{F_{n}\\}_{n\geq 1}$, we can
extract a further subsequence $\\{F_{n_{k}}\\}_{k\geq 1}$, such that
$L(F_{n_{k}})\to\vartheta\in[-\infty,\infty]$, and
$\theta^{*}_{n_{k}}\to\theta^{\prime}\in[-\infty,\infty]$.

The proof of Theorem 10 involves the following steps:

  * (A)

Show that $\vartheta>-\infty$.

  * (B)

Show that $\vartheta<\infty$, and there exists $M>\vartheta+1$ such that

$\limsup_{k\uparrow\infty}\sup_{x\in\mathbb{R}}\psi^{*}_{n_{k}}(x)<M.$

  * (C)

Show that $\theta^{\prime}\in\mathbb{R}$.

  * (D)

Taking $a$ and $b$ to be the left and right endpoints of $J(F)$ respectively,
from Lemma 10, we observe that

$\text{int}(\mathrm{dom}(\psi^{*}(F)))=(-d,d),$ (6.39)

where $d=\max(b-\theta^{*}(F),\theta^{*}(F)-a)$. We now define

$\displaystyle D_{\theta^{\prime}}=$ $\displaystyle\
\begin{cases}\mathbb{R},&\text{ if }\ a=-\infty,b=\infty,\\\
\mathrm{dom}(\psi^{*}(F))+\theta^{*}(F)-\theta^{\prime},&\text{ if
}b-\theta^{*}(F)>\theta^{*}(F)-a,\\\
\mathrm{dom}(\psi^{*}(F))-(\theta^{*}(F)-\theta^{\prime}),&\text{ if
}b-\theta^{*}(F)\leq\theta^{*}(F)-a.\end{cases}$ (6.40)

We Show that $\liminf_{k}\psi^{*}_{n_{k}}(x)>-\infty$ for
$x\in\text{int}(D_{\theta^{\prime}})$, leading to
$\liminf_{k}\psi^{*}_{n_{k}}(0)>-\infty$.

  * (E)

Show that there exist $\alpha^{\prime}$, $\beta^{\prime}>0$ such that for all
sufficiently large $k$,

$\psi^{*}_{n_{k}}(x)\leq-\alpha^{\prime}|x|+\beta^{\prime},\quad\text{ for all
}x\in\mathbb{R}.$

  * (F)

Show that $\vartheta=L(F)$, $\theta^{\prime}=\theta$, and $\psi^{*}_{n_{k}}$
converges pointwise to $\psi^{*}(F)$ on
$\text{int}(\mathrm{dom}(\psi^{*}(F))$.

The proof of $(A)$ is quite straightforward. To see this, observe that the
function $\psi(x)=-|x|$ is a member of $\mathcal{SC}_{0}$. Therefore,

$\vartheta=\lim_{k\to\infty}L(F_{n_{k}})\geq\lim_{k\to\infty}\Psi(0,\psi,F_{n_{k}})\geq-\lim_{k\to\infty}\displaystyle\int_{-\infty}^{\infty}|x|dF_{n_{k}}(x)-\displaystyle\int_{-\infty}^{\infty}e^{-|x|}dx,$

which equals $-\int|x|dF(x)-2$ by (6.38). Since $\int|x|dF(x)<\infty$ by
Condition A, step $(A)$ follows.

The proof of $(B)$ hinges on the proof of Theorem $2.2$ of Dümbgen et al.
(2011). Suppose $\limsup_{k}\sup_{x\in\mathbb{R}}\psi^{*}_{n_{k}}(x)=\infty$.
In that case we can replace $\\{n_{k}\\}$ by a further subsequence such that
$\lim_{k}\sup_{x\in\mathbb{R}}\psi^{*}_{n_{k}}(x)=\infty$. Hence, without loss
of generality, we assume that
$\lim_{k}\sup_{x\in\mathbb{R}}\psi^{*}_{n_{k}}(x)=\infty$. Also, note that
$\sup_{x}\psi^{*}_{n_{k}}(x)=\psi^{*}_{n_{k}}(0)=M_{k}$, say.

Letting $D_{k,t}$ be the level set $\\{\psi^{*}_{n_{k}}\geq t\\}$, for any
$c>0$, we can bound $L(F_{n_{k}})$ in the following way:

$\displaystyle L(F_{n_{k}})=$ $\displaystyle\
\displaystyle\int_{-\infty}^{\infty}\psi^{*}_{n_{k}}(x-\theta^{*}_{n_{k}})dF_{n_{k}}(x)-1$
$\displaystyle=$ $\displaystyle\
\displaystyle\int_{-\infty}^{\infty}\psi^{*}_{n_{k}}(x)dF_{n_{k}}(x+\theta^{*}_{n_{k}})-1$
$\displaystyle\leq$ $\displaystyle-cM_{k}P_{n_{k}}\bigg{(}\mathbb{R}\setminus
D_{k,-cM_{k}}+\theta^{*}_{n_{k}}\bigg{)}+M_{k}P_{n_{k}}(D_{k,-cM_{k}}+\theta^{*}_{n_{k}})-1$
$\displaystyle=$ $\displaystyle\
-cM_{k}\bigg{(}1-P_{n_{k}}(D_{k,-cM_{k}}+\theta^{*}_{n_{k}})\bigg{)}+M_{k}P_{n_{k}}(D_{k,-cM_{k}}+\theta^{*}_{n_{k}})-1$
$\displaystyle=$ $\displaystyle\
-(c+1)M_{k}\bigg{(}\dfrac{c}{c+1}-P_{n_{k}}(D_{k,-cM_{k}}+\theta^{*}_{n_{k}})\bigg{)}-1.$
(6.41)

Let us denote the Lebesue measure on $\mathbb{R}$ by Leb. We intend to show
that $\text{Leb}(D_{k,-cM_{k}})\to 0$ as $k\to\infty$. Applying Lemma $4.1$ of
Dümbgen et al. (2011), we obtain that as $M_{k}\to\infty$,

$\displaystyle\text{Leb}(D_{k,-cM_{k}})\leq$ $\displaystyle\
(1+c)M_{k}e^{-M_{k}}/\displaystyle\int_{0}^{(1+c)M_{k}}te^{-t}dt$
$\displaystyle=$ $\displaystyle\ (1+c)M_{k}e^{-M_{k}}/(1+o(1))\to 0.$ (6.42)

Now we will show that $P_{n_{k}}(D_{k,-cM_{k}}+\theta^{*}_{n_{k}})\to 0$. We
consider two cases, $|\theta^{\prime}|<\infty$ and
$\theta^{\prime}=\pm\infty$. Let us focus on the case
$|\theta^{\prime}|<\infty$ first. Now the concavity of $\psi^{*}_{n_{k}}$
indicates that sets of the form $D_{k,-cM_{k}}$ are intervals including $0$.
Since $\text{Leb}(D_{k,-cM_{k}})\to 0$ and
$\theta^{*}_{n_{k}}\to\theta^{\prime}$ as $k\to\infty$, for sufficiently large
$k$, the set $D_{k,-cM_{k}}+\theta^{*}_{n_{k}}$ is an interval around
$\theta^{\prime}$ whose length shrinks to $0$ as $k$ increases. Now since $F$
is non-degenerate, one can certainly find a $x_{0}\neq\theta^{\prime}$ such
that $x_{0}\in\text{int}(J(F))$. As a result, $x_{0}\notin
D_{k,-cM_{k}}+\theta^{*}_{n_{k}}$ for all sufficiently large $k$. Thus, for
all sufficiently large $k$,

$D_{k,-cM_{k}}+\theta^{*}_{n_{k}}\in\\{C\subset\mathbb{R}\text{ closed and
convex, }\ x_{0}\notin\text{int}(C)\\}=D_{x_{0}},$

say. However, since $F_{n_{k}}$ weakly converges to $F$, Lemma $2.13$ of
Dümbgen et al. (2011) implies that

$\limsup_{k}\sup_{C\in D_{x_{0}}}P_{n_{k}}(C)\leq\sup_{C\in
D_{x_{0}}}P_{0}(C).$ (6.43)

By Lemma $2.13$ of Dümbgen et al. (2011), $\sup_{C\in D_{x_{0}}}P(C)$ is
bounded away from $1$ if

$x_{0}\in\bigcap_{C\in\mathcal{D}}C,$

where

$\mathcal{D}=\bigg{\\{}C\ \bigg{|}\ C\text{ is closed and convex,
}P(C)=1\bigg{\\}}.$

Since $x_{0}\in\text{int}(J(F))$, the above holds. Therefore, we conclude that
the term in the left hand side of (6.43) is less than $c/(c+1)$ for
sufficiently large $c$. Since $D_{k,-cM_{k}}+\theta^{*}_{n_{k}}\in D_{x_{0}}$,
the above implies that $P_{n_{k}}(D_{k,-cM_{k}}+\theta^{*}_{n_{k}})<c/(c+1)$
for sufficiently large $k$. Combined with (6.2), this result leads to

$L(F_{n_{k}})\to-\infty\quad\text{ as }k\to\infty,$

which contradicts the fact that $\vartheta>-\infty$. Therefore, for this case,
we see that $\lim_{k}M_{k}=\infty$ leads to a contradiction.

At this point it is clear that for the other case also, i.e. when
$\theta^{\prime}=\pm\infty$, it is enough to show that
$\limsup_{k}P_{n_{k}}(D_{k,-cM_{k}}+\theta^{*}_{n_{k}})<c/(c+1)$ for
sufficiently large $c$. We will consider the case $\theta^{\prime}=\infty$.
The proof for $\theta^{\prime}=-\infty$ will follow similarly. Consider $y$
such that $1-F(y)<c/(c+1)$. Since $\text{Leb}(D_{k,-cM_{k}})\to 0$, we observe
that as $\theta^{*}_{n_{k}}\to\infty$, for all sufficiently large $k$,

$D_{k,-cM_{k}}+\theta^{*}_{n_{k}}\subset(y,\infty).$

Noting $F_{n_{k}}\to_{w}F$, we obtain that

$\limsup_{k}P_{n_{k}}(D_{k,-cM_{k}}+\theta^{*}_{n_{k}})\leq\limsup_{k}(1-F_{n_{k}}(y))=1-F(y)<c/(c+1),$

which completes the proof of $(B)$. Therefore, we assume that
$\\{M_{k}\\}_{k\geq 1}$ is bounded above by a number, say $M$, which can be
chosen to be greater than $\vartheta+1$.

Our next step is step (C), which follows from Lemma 12. To see this, observe
that Lemma 12 implies

$\phi^{*}_{n_{k}}(x)\leq-\log|2(x-\theta^{*}_{n_{k}})|.$

Suppose $\limsup_{k}|\theta^{*}_{n_{k}}|=\infty$. Then Fatou’s Lemma implies
that

$\displaystyle\liminf_{k}\displaystyle\int_{-\infty}^{\infty}e^{\phi^{*}_{n_{k}}(x)}dx\leq$
$\displaystyle\
\liminf_{k}\displaystyle\int_{-\infty}^{\infty}e^{-\log|2(x-\theta^{*}_{n_{k}})|}dx$
$\displaystyle\leq$ $\displaystyle\
\displaystyle\int_{-\infty}^{\infty}\liminf_{k}e^{-\log|2(x-\theta^{*}_{n_{k}})|}dx=0.$

However, the left hand side of the above display equals $1$ since
$e^{\phi^{*}_{n_{k}}}$ is a density, which leads to a contradiction.
Therefore, step (C) follows.

Next we will prove (D). For $x\in D_{\theta^{\prime}}$, denote
$A_{n_{k}}=(-x+\theta^{*}_{n_{k}},x+\theta^{*}_{n_{k}})$. We calculate

$\displaystyle L(F_{n_{k}})=$ $\displaystyle\
\displaystyle\int_{-\infty}^{\infty}\psi^{*}_{n_{k}}(z)dF_{n_{k}}(z+\theta^{*}_{n_{k}})-1$
$\displaystyle=$
$\displaystyle\displaystyle\int_{-x}^{x}\psi^{*}_{n_{k}}(z)dF_{n_{k}}(z+\theta^{*}_{n_{k}})+\displaystyle\int_{-\infty}^{-x}\psi^{*}_{n_{k}}(z)dF_{n_{k}}(z+\theta^{*}_{n_{k}})$
$\displaystyle\
+\displaystyle\int_{x}^{\infty}\psi^{*}_{n_{k}}(z)dF_{n_{k}}(z+\theta^{*}_{n_{k}})-1$
$\displaystyle\leq$ $\displaystyle\
\psi^{*}_{n_{k}}(0)P_{n_{k}}(A_{n_{k}})+\psi^{*}_{n_{k}}(x)P_{n_{k}}(\mathbb{R}\setminus
A_{n_{k}})-1,$

which follows since $\psi^{*}_{n_{k}}\in\mathcal{SC}_{0}$. Now observe that
the above implies

$\displaystyle\liminf\limits_{k\to\infty}\psi^{*}_{n_{k}}(x)\geq$
$\displaystyle\
\liminf\limits_{k\to\infty}\dfrac{L(F_{n_{k}})+1-\psi^{*}_{n_{k}}(0)P_{n_{k}}(A_{n_{k}})}{P_{n_{k}}(\mathbb{R}\setminus
A_{n_{k}})}$ $\displaystyle\geq$ $\displaystyle\
-\dfrac{M-1-\vartheta}{\liminf\limits_{k\to\infty}P_{n_{k}}(\mathbb{R}\setminus
A_{n_{k}})}.$ (6.44)

Note that for every $\epsilon>0$,

$\theta^{\prime}-\epsilon<\theta^{*}_{n_{k}}<\theta^{\prime}+\epsilon,\quad\text{
for all sufficiently large }k.$

Therefore, we can write

$\displaystyle\liminf\limits_{k\to\infty}P_{n_{k}}(\mathbb{R}\setminus
A_{n_{k}})$ $\displaystyle=$ $\displaystyle\
\liminf\limits_{k\to\infty}F_{n_{k}}(-x+\theta^{*}_{n_{k}})+1-\limsup\limits_{k\to\infty}F_{n_{k}}\bigg{(}(x+\theta^{*}_{n_{k}})-\bigg{)}$
$\displaystyle\geq$ $\displaystyle\
\liminf\limits_{k\to\infty}F_{n_{k}}(-x+\theta^{\prime}-\epsilon)+1-\limsup\limits_{k\to\infty}F_{n_{k}}\bigg{(}(x+\theta^{\prime}+\epsilon)-\bigg{)}$
$\displaystyle\geq$ $\displaystyle\
F\bigg{(}(-x+\theta^{\prime}-\epsilon)-\bigg{)}+1-F\bigg{(}(x+\theta^{\prime}+\epsilon)-\bigg{)}.$

If $J(F)=\mathbb{R}$, it is clear that the right hand side of the last display
is positive. Suppose $J(F)\neq\mathbb{R}$. Then at least one of $a$ and $b$,
the endpoints of $J(F)$, is finite. Now suppose
$b-\theta^{*}(F)>\theta^{*}(F)-a$. Noting
$x\in\text{int}(D_{\theta^{\prime}})$, we argue that for small enough
$\epsilon>0$, $x+\epsilon\in\text{int}(D_{\theta^{\prime}})$. Then from (6.39)
and (6.40) it follows that

$x+\epsilon<b-\theta^{*}(F)+\theta^{*}(F)-\theta^{\prime},$

implying that $x+\epsilon+\theta^{\prime}<b$, leading to
$F((x+\theta^{\prime}+\epsilon)-)<1$ since $b$ is the right endpoint of
$J(F)$. Now consider the case when $b-\theta^{*}(F)\leq\theta^{*}(F)-a$.
Analogous to the previous case, (6.39) and (6.40) yield

$x+\epsilon<\theta^{*}(F)-a-\theta^{*}(F)+\theta^{\prime},$

or equivalently,

$a<-x+\theta^{\prime}-\epsilon,$

implying $F((-x+\theta^{\prime}-\epsilon)-)>0$, which leads to

$\limsup\limits_{k\to\infty}\mathbb{P}_{n}(\mathbb{R}\setminus A_{n_{k}})>0.$

Since in step (B), $M$ was chosen such that $M>\vartheta+1$, (6.2) entails
that

$\liminf\limits_{k\to\infty}\widehat{\psi}_{\theta_{r_{k}}}(x)>-\infty$

which proves (D).

To prove (E), we first observe that since
$\psi^{*}_{n_{k}}\in\mathcal{SC}_{0}$, Lemma $3$ of Pal et al. (2007) implies
that

$\psi^{*}_{n_{k}}(x)\leq\psi^{*}_{n_{k}}(0)+1-e^{\psi^{*}_{n_{k}}(0)}|x|,\quad
x\in\mathbb{R}.$

Observe that (B) and (D), combined with the above, imply that there exist
$\alpha^{\prime}>0$ and $\beta^{\prime}\in\mathbb{R}$ such that for all
sufficiently large $k$,

$\psi^{*}_{n_{k}}(x)\leq\beta^{\prime}-\alpha^{\prime}|x|\quad\text{ for all
}x\in\mathbb{R}.$ (6.45)

Now we move on to step (F). Since we have established (6.45) and showed that
$\liminf\limits_{k\to\infty}\widehat{\psi}_{\theta_{r_{k}}}(x)>-\infty$ for
$x\in\text{int}(D_{\theta^{\prime}})$, by Lemma $4.2$ of Dümbgen et al.
(2011), we can replace $\\{\psi^{*}_{n_{k}}\\}_{k\geq 1}$, if necessary, by a
subsequence such that for a concave function $\bar{\psi}$ the following
conditions are met:

$\displaystyle\begin{matrix}\text{int}(D_{\theta^{\prime}})\subset\mathrm{dom}(\bar{\psi}),&\\\
\bar{\psi}(x)\leq\beta^{\prime}-\alpha^{\prime}|x|&\quad\text{ for all
}x\in\mathbb{R},k\in\mathbb{N},\\\ \lim\limits_{k\to\infty,y\to
x}\psi^{*}_{n_{k}}(y)=\bar{\psi}(x)&\quad\text{ for all
}x\in\text{int}(\mathrm{dom}(\bar{\psi})),\\\ \limsup\limits_{k\to\infty,y\to
x}\psi^{*}_{n_{k}}(y)\leq\bar{\psi}(x)&\quad\text{ for all
}x\in\mathbb{R}.\end{matrix}$ (6.46)

We will now establish that $\bar{\psi}\in\mathcal{SC}_{0}$. To see this, first
notice that since $\psi^{*}_{n_{k}}\in\mathcal{SC}_{0}$, for
$x\notin\mathrm{dom}(\bar{\psi})$,

$\lim_{k\to\infty}\psi^{*}_{n_{k}}(-x)=\lim_{k\to\infty}\psi^{*}_{n_{k}}(x)\leq\bar{\psi}(x)=-\infty.$

Hence, $-x\notin\text{int}(\mathrm{dom}(\bar{\psi}))$, suggesting that either
$-x\notin\mathrm{dom}(\bar{\psi})$, or $-x\in\mathrm{dom}(\bar{\psi})$ is a
boundary point of $\mathrm{dom}(\bar{\psi})$. Therefore,
$\mathrm{dom}(\bar{\psi})$ is of the form $(-c,c)$, $[-c,c]$, $(-c,c]$, or
$[-c,c)$ for some $c>0$. For the last two cases, if we extend $\bar{\psi}$ to
$[-c,c]$ so that $\bar{\psi}$ is continuous on $[-c,c]$ and $-\infty$
everywhere else, $\bar{\psi}$ still satisfies (6.46). Therefore, without loss
of generality, we assume that $\mathrm{dom}(\bar{\psi})$ is of the form
$(-c,c)$ or $[-c,c]$. For $y\in(-c,c)$ we obtain that

$\bar{\psi}(y)=\lim_{k\to\infty}\psi^{*}_{n_{k}}(y)=\lim_{k\to\infty}\psi^{*}_{n_{k}}(-y)=\bar{\psi}(-y).$

It remains to show that when $\mathrm{dom}(\bar{\psi})$ is the closed interval
$[-c,c]$, $\bar{\psi}(-c)$ and $\bar{\psi}(c)$ agree. Since $\psi^{*}$ is
concave, it is right continuous at $-c$, and left continuous at $c$.
Therefore, the symmetry of $\psi^{*}$ on $(-c,c)$ implies symmetry on
$[-c,c]$, which establishes that $\bar{\psi}\in\mathcal{SC}_{0}$. Now we claim
that $e^{\bar{\psi}}\in\mathcal{SLC}_{0}$. To see this observe that since
$\psi^{*}_{n_{k}}(x)\leq\beta^{\prime}-\alpha^{\prime}|x|$, application of the
dominated convergence theorem yields

$\displaystyle\int_{\text{int}(\mathrm{dom}(\bar{\psi}))}e^{\bar{\psi}(x)}dx=\displaystyle\int_{\text{int}(\mathrm{dom}(\bar{\psi}))}e^{\lim_{k}\psi^{*}_{n_{k}}(x)}dx=\lim_{k}\displaystyle\int_{\mathrm{dom}(\bar{\psi})}e^{\psi^{*}_{n_{k}}(x)}dx=1.$

Since Condition B holds, note that if we can show

$\vartheta=\Psi(\theta^{\prime},\bar{\psi},F)=L(F),$

we have $\theta^{\prime}=\theta^{*}(F)$, $\bar{\psi}=\psi^{*}(F)$, and (3.24)
also follows. Therefore, we have $\theta^{*}_{n}\to\theta^{*}(F)$, and (6.46)
leads to the pointwise convergence of $\psi^{*}_{n}$ to $\psi^{*}(F)$. The
pointwise convergence of $\psi^{*}_{n}$’s implies pointwise convergence of
$e^{\psi^{*}_{n}}$, which, in its turn leads to the weak convergence of the
corresponding distribution functions. Then the rest of the proof of Theorem 13
follows from Proposition $2$ of Cule and Samworth (2010). We prove the
equations in the last display by first establishing that

$\vartheta\leq\Psi(\theta^{\prime},\bar{\psi},F)\leq L(F),$ (6.47)

and then showing $\vartheta\geq L(F)$.

Observe that since $F_{n_{k}}$ converges to $F$ weakly, by Skorohod’s theorem,
there exists a probability space $(\Omega,\mathcal{A},P^{*})$ with random
variable $X_{k}\sim F_{n_{k}}$, and $X\sim F$ such that $X_{k}\to_{a.s.}X$ as
$k\to\infty$. Denote by $E^{*}$ the expectation operator with respect to the
probability space $(\Omega,\mathcal{A},P^{*})$. Denoting the positive random
variable $\beta^{\prime}-\alpha^{\prime}|X_{k}|-\phi^{*}_{n_{k}}(X_{k})$ by
$H_{k}$, we calculate

$\displaystyle\vartheta=$ $\displaystyle\
\limsup_{k\to\infty}\displaystyle\int_{-\infty}^{\infty}\phi^{*}_{n_{k}}(x)dF_{n_{k}}(x)-1$
$\displaystyle=$ $\displaystyle\
\lim_{k\to\infty}\displaystyle\int_{-\infty}^{\infty}(\beta^{\prime}-\alpha^{\prime}|x|)dF_{n_{k}}(x)-\liminf_{k\to\infty}E^{*}[H_{k}]-1$
$\displaystyle\leq$ $\displaystyle\
\beta^{\prime}-\alpha^{\prime}\displaystyle\int_{-\infty}^{\infty}|x|dF(x)-E^{*}\bigg{[}\liminf_{k\to\infty}H_{k}\bigg{]}-1,$

which follows using (6.38) and Fatou’s Lemma. As
$\theta^{*}_{n_{k}}\to\theta^{\prime}\in\mathbb{R}$, and $X_{k}\to_{a.s.}X$,
(6.46) indicates that the following holds with probability $1$:

$\displaystyle\liminf_{k\to\infty}H_{k}=\alpha^{\prime}-\beta^{\prime}|X|-\limsup_{k}\psi^{*}_{n_{k}}(X_{k}-\theta^{*}_{n_{k}})\geq\beta^{\prime}-\alpha^{\prime}|X|-\bar{\psi}(X-\theta^{\prime}).$

Since $X\sim F$, the above leads to

$\displaystyle\vartheta\leq$ $\displaystyle\
\beta^{\prime}-\alpha^{\prime}\displaystyle\int_{-\infty}^{\infty}|x|dF(x)-E^{*}\bigg{[}\beta^{\prime}-\alpha^{\prime}|X|-\bar{\psi}(X-\theta^{\prime})\bigg{]}-1$
$\displaystyle=$ $\displaystyle\
\displaystyle\int_{-\infty}^{\infty}\bar{\psi}(x-\theta^{\prime})dF(x)-1$
$\displaystyle=$ $\displaystyle\ \Psi(\theta^{\prime},\bar{\psi},F)\leq L(F),$

which completes the proof of (6.47). Hence, it only remains to prove that
$\vartheta\geq L(F)$.

Let us denote $c_{0}=\psi^{*}(F)(0)$. From Lemma 15 it follows that for each
$\epsilon>0$, there exist $\psi^{(\epsilon)}\in\mathcal{SC}_{0}$ such that

$-|x|/\epsilon+c_{0}\leq\psi^{(\epsilon)}(x)\quad\text{ for all
}x\in\mathbb{R},$ (6.48)

and

$\psi^{*}(F)\leq\psi^{(\epsilon)}\leq c_{0}.$ (6.49)

Further, $\psi^{(\epsilon)}$ decreases to $\psi^{*}(F)$ pointwise as
$\epsilon\downarrow 0$. Using the random variables $X_{k}\sim F_{n_{k}}$ and
$X\sim F$ constructed in the proof of (6.47), we write

$\displaystyle\vartheta=\lim_{k\to\infty}L(F_{n_{k}})\geq$ $\displaystyle\
\lim_{k\to\infty}\Psi(\theta^{*}(F),\psi^{(\epsilon)},F_{n_{k}})$
$\displaystyle=$ $\displaystyle\
\lim_{k\to\infty}E^{*}\bigg{[}\psi^{(\epsilon)}(X_{k}-\theta^{*}(F))\bigg{]}-\displaystyle\int_{-\infty}^{\infty}e^{\psi^{(\epsilon)}(x)}dx.$
(6.50)

Note that (6.48) and (6.49) indicate that $\psi^{(\epsilon)}(x)$ is bounded
above and below by $c_{0}$ and $-|x|/\epsilon+c_{0}$ respectively. From (6.38)
it follows that

$E^{*}|X_{k}|=\displaystyle\int_{-\infty}^{\infty}|x|dF_{n_{k}}(x)\to\displaystyle\int_{-\infty}^{\infty}|x|dF(x),\quad\text{
as }k\to\infty.$

Moreover, as $k\to\infty$, we also have $X_{k}\to_{a.s.}X$. Observe that
$\psi^{(\epsilon)}$ is concave, and by (6.48),
$\mathrm{dom}(\psi^{(\epsilon)})=\mathbb{R}$, which implies that
$\psi^{(\epsilon)}$ is continuous. Therefore,

$\psi^{(\epsilon)}(X_{k}-\theta^{*}(F))\to_{a.s.}\psi^{(\epsilon)}(X-\theta^{*}(F)).$

Therefore, using Lemma 16, from (6.2) we obtain that

$\displaystyle\vartheta\geq$ $\displaystyle\
E\bigg{[}\lim_{k\to\infty}\psi^{(\epsilon)}(X_{k}-\theta^{*}(F))\bigg{]}-\displaystyle\int_{-\infty}^{\infty}e^{\psi^{(\epsilon)}(x)}dx$
$\displaystyle=$ $\displaystyle\
\displaystyle\int_{-\infty}^{\infty}\psi^{(\epsilon)}(x-\theta^{*}(F))dF(x)-\displaystyle\int_{-\infty}^{\infty}e^{\psi^{(\epsilon)}(x)}dx.$

Since $\psi^{(\epsilon)}$ decreases to $\psi^{*}(F)$ pointwise as
$\epsilon\downarrow 0$, the monotone convergence theorem and (6.49) lead to

$\lim_{\epsilon\downarrow
0}\displaystyle\int_{-\infty}^{\infty}\bigg{(}c_{0}-\psi^{(\epsilon)}(x-\theta^{*}(F))\bigg{)}dF(x)=\displaystyle\int_{-\infty}^{\infty}\bigg{(}c_{0}-\psi^{*}(F)(x-\theta^{*}(F))\bigg{)}dF(x),$

which can be rewritten as

$\lim_{\epsilon\downarrow
0}\displaystyle\int_{-\infty}^{\infty}\psi^{(\epsilon)}(x-\theta^{*}(F))dF(x)=\displaystyle\int_{-\infty}^{\infty}\psi^{*}(F)(x-\theta^{*}(F))dF(x).$
(6.51)

On the other hand, note that (6.49) implies that $g^{*}(F)\leq
e^{\psi^{(\epsilon)}}\leq g^{*}(F)(0)$. Since $e^{\psi^{(\epsilon)}}$
decreases to $g^{*}(F)$ pointwise as $\epsilon$ decreases to $0$, the monotone
convergence theorem implies that

$\lim_{\epsilon\downarrow
0}\displaystyle\int_{-\infty}^{\infty}\bigg{(}g^{*}(F)(0)-e^{\psi^{(\epsilon)}(x)}\bigg{)}dx=\displaystyle\int_{-\infty}^{\infty}\bigg{(}g^{*}(F)(0)-g^{*}(F)(x)\bigg{)}dx,$

or equivalently,

$\lim_{\epsilon\downarrow
0}\displaystyle\int_{-\infty}^{\infty}e^{\psi^{(\epsilon)}(x)}dx=\displaystyle\int_{-\infty}^{\infty}g^{*}(F)(x)dx=1.$
(6.52)

Therefore, from (6.51) and (6.52), we conclude that

$\displaystyle\vartheta\geq$ $\displaystyle\ \lim_{\epsilon\downarrow
0}\displaystyle\int_{-\infty}^{\infty}\psi^{(\epsilon)}(x-\theta^{*}(F))\
dF(x)-\lim_{\epsilon\downarrow
0}\displaystyle\int_{-\infty}^{\infty}e^{\psi^{(\epsilon)}(x)}dx$
$\displaystyle=$ $\displaystyle\
\displaystyle\int_{-\infty}^{\infty}\psi^{*}(F)(x-\theta^{*}(F))\
dF(x)-1=L(F),$

which completes the proof. $\Box$

#### Proof of Theorem 9

In their proof of Theorem $3.1$, Pal et al. (2007) show that if a sequence of
log-concave functions $\\{f_{n}\\}_{n\geq 1}$ (which can be stochastic as
well) satisfies

$\sum_{i=1}^{n}\log f_{n}(X_{i})\geq\sum_{i=1}^{n}\log f_{0}(X_{i})\quad
a.s.,$ (6.53)

we have $H(f_{n},f_{0})\to_{a.s.}0$, provided

$P\bigg{(}\sup_{x}\log f_{n}(x)=o\bigg{(}\dfrac{\sqrt{n}}{\log
n}\bigg{)}\bigg{)}=1.$

If we take $f_{n}=\widehat{f}_{n}$, we have

$\sup_{x}\log\widehat{f}_{n}(x)=\widehat{\phi}_{n}(\widehat{\theta}_{n})=\widehat{\psi}_{n}(0).$

Theorem 11(c) and Corollary 4 entail that

$P\bigg{(}\limsup_{n}\widehat{\psi}_{n}(0)<\infty\bigg{)}=1.$

Also, note that being the MLE of $f_{0}$, $\widehat{f}_{n}$ automatically
satisfies (6.53), which implies

$H(\widehat{f}_{n},f_{0})\to_{a.s.}0.$ (6.54)

Since $f_{0}\in\mathcal{P}_{0}$ is log-concave and continuous, Proposition
2(c) of Cule and Samworth (2010) and (6.54) entail that

$\sup\limits_{z\in\mathbb{R}}|\widehat{f}_{n}(z)-f_{0}(z)|\to_{a.s.}0.$ (6.55)

Note that

$\displaystyle 2H^{2}(\widehat{g}_{n},g_{0})=$ $\displaystyle\
\displaystyle\int_{-\infty}^{\infty}\bigg{(}\sqrt{\widehat{g}_{n}(z-\widehat{\theta}_{n})}-\sqrt{g_{0}(z-\widehat{\theta}_{n})}\bigg{)}^{2}dz$
$\displaystyle\leq$ $\displaystyle\
2\displaystyle\int_{-\infty}^{\infty}\bigg{(}\sqrt{\widehat{g}_{n}(z-\widehat{\theta}_{n})}-\sqrt{g_{0}(z-\theta_{0}\vphantom{\widehat{\theta}_{n}})}\bigg{)}^{2}dz$
$\displaystyle\
+2\displaystyle\int_{-\infty}^{\infty}\bigg{(}\sqrt{g_{0}(z-\widehat{\theta}_{n})}-\sqrt{g_{0}(z-\theta_{0}\vphantom{\widehat{\theta}_{n}})}\bigg{)}^{2}dz,$

where the first term on the right hand side of the last display approaches $0$
almost surely by (6.54). The second term is also bounded above by a constant
multiple of $g_{0}(z-\widehat{\theta}_{n})+g_{0}(z-\theta_{0})$, which is
integrable. Therefore, using Lemma 16, Theorem 11(a), and Corollary 4, we
deduce that this term also converge to $0$ almost surely. Hence
$H^{2}(\widehat{g}_{n},g_{0})\to_{a.s.}0$ follows.

Our next step is to find the rate of convergence of
$H(\widehat{f}_{n},f_{0})$. To do so, we first introduce the class of
functions

$\displaystyle\mathcal{P}_{M,0}=\bigg{\\{}f\in\mathcal{LC}\ \bigg{|}\ $
$\displaystyle\ \sup\limits_{x\in\mathbb{R}}f(x)<M,\ f(x)>1/M\text{ for all
}|x|<1,$ $\displaystyle\ \text{Supp}(f)\subset\text{Supp}(f_{0})\bigg{\\}}.$

We will show that without loss of generality, one can assume that
$f_{0}\in\mathcal{P}_{M,0}$ for some $M>0.$ To this end, we translate and
rescale the data letting $\tilde{X}_{i}=\alpha X_{i}+\beta$, where $\alpha>0$
and $\beta\in\mathbb{R}$. Observe that the rescaled data has density
$\tilde{f}_{0}(x)=\alpha^{-1}f_{0}((x-\beta)/\alpha)$. Denote by
$\tilde{f}_{0,n}$ the MLE of $f_{0}$ based on the rescaled data. Note that the
MLE is affine-equivalent, which entails that
$\tilde{f}_{0,n}(x)=\alpha^{-1}\widehat{f}_{n}((x-\beta)/\alpha).$ Noting
Hellinger distance is invariant under affine transformations, we observe that
$H(\widehat{f}_{n},f_{0})=H(\tilde{f}_{0,n},\tilde{f}_{0}).$ Therefore, it
suffices to show that $H(\tilde{f}_{0,n},\tilde{f}_{0})\to_{a.s.}0$. Note that
since $f_{0}$ is log-concave, $\text{int}(\mathrm{dom}(f_{0}))$ contains an
interval. We can choose $\alpha$ and $\beta$ in a way such that
$(x-\beta)/\alpha$ lie inside that interval for $x=\pm 1$. Then it is possible
to find $M>0$ large enough such that

$f_{0}((x-\beta)/\alpha)>\alpha/M,\quad x=\pm 1,$

or

$\min(\tilde{f}_{0}(-1),\tilde{f}_{0}(1))>1/M,$

leading to

$\inf_{x\in[-1,1]}\tilde{f}_{0}(x)>1/M,$

since $f_{0},$ or equivalently $\tilde{f}_{0}$ is unimodal. Hence, without
loss of generality, we can assume that there exists $M>0$ such that
$f_{0}(x)>1/M$ for $x\in[-1,1].$ We can choose $M$ large enough such that
additionally, $\sup\limits_{x\in\mathbb{R}}f_{0}(x)<M$. On the other hand,
(6.55) implies that the following inequalities hold with probability one:

$\limsup_{n}\sup\limits_{x\in\mathbb{R}}\widehat{f}_{n}(x)<M,$

and

$\lim_{n}\widehat{f}_{n}(\pm 1)>1/M.$

Therefore, $f_{0}\in\mathcal{P}_{M,0}$, and with probability one,
$\widehat{f}_{n}\in\mathcal{P}_{M,0}$ as well for all sufficiently large $n$.
Doss and Wellner (2016) obtained the bracketing entropy of the class
$\mathcal{P}_{M,0}$. They showed that for any $\epsilon>0$,

$\log N_{[\ ]}(\epsilon,\mathcal{P}_{M,0},H)\lesssim\epsilon^{-1/2}.$

The rest of the proof now follows from an application of Theorem $3.4.1$ and
$3.4.4$ of Van der Vaart and Wellner (1996). To this end, fixing $\epsilon>0,$
for $f\in\mathcal{P}_{M,0}$ we define the function

$m_{f}(x)=\log\bigg{(}\dfrac{f(x)+f_{0}(x)}{2f_{0}(x)}\bigg{)},$

and we let $\mathcal{M}_{\epsilon}$ denote the class

$\mathcal{M}_{\epsilon}=\\{m_{f}-m_{f_{0}}\ :\ H(f,f_{0})\leq\epsilon\\}.$

Also, we denote

$||\mathbb{Z}_{n}||_{\mathcal{M}_{\epsilon}}=\sup_{h\in\mathcal{M}_{\epsilon}}\int
hd\mathbb{Z}_{n}.$

Then Theorem $3.4.4$ of Van der Vaart and Wellner (1996) yields

$P(m_{f}-m_{f_{0}})\lesssim-H^{2}(f,{f_{0}}),$

and

$E_{P}||\mathbb{Z}_{n}||_{\mathcal{M}_{\epsilon}}\lesssim\tilde{J}_{[\
]}(\epsilon,\mathcal{P}_{M,0},H)\bigg{(}1+\dfrac{\tilde{J}_{[\
]}(\epsilon,\mathcal{P}_{M,0},H)}{\epsilon^{2}\sqrt{n}}\bigg{)},$

where

$\displaystyle\tilde{J}_{[\ ]}(\epsilon,\mathcal{P}_{M,0},H)=$ $\displaystyle\
\displaystyle\int_{0}^{\epsilon}\sqrt{\log N_{[\
]}(t,\mathcal{P}_{M,0},H)}dt\lesssim\epsilon^{3/4}.$

Therefore, it is easy to see that

$E_{P}||\mathbb{Z}_{n}||_{\mathcal{M}_{\epsilon}}\lesssim\epsilon^{3/4}+\epsilon^{-1/2}n^{-1/2}=g(\epsilon),$

where

$g(t)=t^{3/4}+t^{-1/2}n^{-1/2},\quad t>0.$

Notice that $g(t)/t^{3/4}$ is decreasing in $t.$ Also, for any constant $c>0,$
we have

$n^{4/5}g(cn^{-2/5})=(c^{3/4}+c^{-1/2})n^{1/2}.$

Therefore, an application of Theorem $3.4.1$ of Van der Vaart and Wellner
(1996) yields

$H(\widehat{f}_{n},{f_{0}})=O_{p}(n^{-2/5}),$

which completes the proof of part A of Theorem 9.

Now we turn to the proof of part B. If $x-\theta_{0}$ is a continuity point of
$g_{0}^{\prime}$, noting $\widehat{\theta}_{n}\to_{a.s.}\theta_{0}$ by Theorem
11 and Corollary 4, we obtain that

$\dfrac{\sqrt{g_{0}(x-\widehat{\theta}_{n})}-\sqrt{\vphantom{\widehat{\theta}_{n}}g_{0}(x-\theta_{0})}}{(\widehat{\theta}_{n}-\theta_{0})}\to_{a.s.}\dfrac{g_{0}^{\prime}(x-\theta_{0})}{2\sqrt{g_{0}(x-\theta_{0})}}.$

Noting $g_{0}^{\prime}$ is continuous almost everywhere with respect to
Lebesgue measure, and using Fatou’s lemma and part A of the current theorem,
we obtain that

$\displaystyle\liminf_{n}\dfrac{\displaystyle\int_{-\infty}^{\infty}\bigg{(}\sqrt{g_{0}(x-\widehat{\theta}_{n})}-\sqrt{\vphantom{\widehat{\theta}_{n}}g_{0}(x-\theta_{0})}\bigg{)}^{2}dx}{(\widehat{\theta}_{n}-\theta_{0})^{2}}$
$\displaystyle\geq$ $\displaystyle\
\displaystyle\int_{-\infty}^{\infty}\bigg{(}\dfrac{g_{0}^{\prime}(x-\theta_{0})}{2\sqrt{g_{0}(x-\theta_{0})}}\bigg{)}^{2}dx=\dfrac{\mathcal{I}_{f_{0}}}{4}\
a.s.$ (6.56)

Now observe that

$\displaystyle 2H(\widehat{f}_{n},f_{0})^{2}$ $\displaystyle=$ $\displaystyle\
\displaystyle\int_{-\infty}^{\infty}\bigg{(}\sqrt{\widehat{g}_{n}(x-\widehat{\theta}_{n})}-\sqrt{\vphantom{\widehat{g}_{n}}g_{0}(x-\theta_{0})}\bigg{)}^{2}dx$
$\displaystyle=$ $\displaystyle\
\displaystyle\int_{-\infty}^{\infty}\bigg{(}\sqrt{\widehat{g}_{n}(x-\widehat{\theta}_{n})}-\sqrt{\vphantom{\widehat{g}_{n}}g_{0}(x-\widehat{\theta}_{n})}\bigg{)}^{2}dx$
$\displaystyle\
+\displaystyle\int_{-\infty}^{\infty}\bigg{(}\sqrt{g_{0}(x-\widehat{\theta}_{n})}-\sqrt{\vphantom{\widehat{\theta}_{n}}g_{0}(x-\theta_{0})}\bigg{)}^{2}dx+T_{c},$

where

$T_{c}=2\displaystyle\int_{-\infty}^{\infty}\bigg{(}\sqrt{\widehat{g}_{n}(x-\widehat{\theta}_{n})}-\sqrt{g_{0}(x-\widehat{\theta}_{n})}\bigg{)}\bigg{(}\sqrt{g_{0}(x-\widehat{\theta}_{n})}-\sqrt{\vphantom{\widehat{g}_{n}}g_{0}(x-\theta_{0})}\bigg{)}dx.$

The inequality in (6.2) entails that for all sufficiently large $n$,

$2H(\widehat{f}_{n},f_{0})^{2}\geq
2H(\widehat{g}_{n},g_{0})^{2}+\dfrac{(\widehat{\theta}_{n}-\theta_{0})^{2}\mathcal{I}_{f_{0}}}{4}-|T_{c}|\quad
a.s.$

We aim to show that the cross-term $|T_{c}|$ is small. In fact, we show that

$\dfrac{|T_{c}|}{|\widehat{\theta}_{n}-\theta_{0}|^{2}+H(\widehat{g}_{n},g_{0})^{2}}=o_{p}(1).$
(6.57)

Suppose (6.57) holds. Then it follows that

$\displaystyle 2H(\widehat{f}_{n},f_{0})^{2}\geq$ $\displaystyle\
2H(\widehat{g}_{n},g_{0})^{2}$ $\displaystyle\
+\dfrac{(\widehat{\theta}_{n}-\theta_{0})^{2}\mathcal{I}_{f_{0}}}{4}-o_{p}(1)H(\widehat{g}_{n},g_{0})^{2}-o_{p}(1)(\widehat{\theta}_{n}-\theta_{0})^{2},$

which completes the proof because $\mathcal{I}_{f_{0}}>0$.

Hence, it remains to prove (6.57). To this end, notice that $T_{c}$ can be
written as

$\displaystyle
T_{c}=2\displaystyle\int_{-\infty}^{\infty}\bigg{(}\sqrt{\widehat{g}_{n}(x)}-\sqrt{\vphantom{\widehat{g}_{n}}g_{0}(x)}\bigg{)}\bigg{(}\sqrt{\vphantom{\widehat{g}_{n}}g_{0}(x)}-\sqrt{\vphantom{\widehat{g}_{n}}g_{0}(x+\widehat{\theta}_{n}-\theta_{0})}\bigg{)}dx.$

For the sake of simplicity, we will denote $\theta_{0}-\widehat{\theta}_{n}$
by $\delta_{n}$ from now on. Since $g_{0}$ is absolutely continuous, we can
write

$\displaystyle|T_{c}|=$
$\displaystyle\bigg{|}2\displaystyle\int_{-\infty}^{\infty}\bigg{(}\sqrt{\widehat{g}_{n}(x)}-\sqrt{\vphantom{\widehat{g}_{n}}g_{0}(x)}\bigg{)}\bigg{(}\displaystyle\int_{-\delta_{n}}^{0}\dfrac{g_{0}^{\prime}(x+t)}{2\sqrt{g_{0}(x+t)}}dt\bigg{)}dx\bigg{|}.$

Since $g_{0}\in\mathcal{S}_{0}$, we have

$\displaystyle|T_{c}|=$ $\displaystyle\
2\bigg{|}\displaystyle\int_{0}^{\infty}\bigg{(}\sqrt{\widehat{g}_{n}(x)}-\sqrt{\vphantom{\widehat{g}_{n}}g_{0}(x)}\bigg{)}\bigg{(}\displaystyle\int_{-\delta_{n}}^{0}\dfrac{g_{0}^{\prime}(x+t)}{2\sqrt{g_{0}(x+t)}}dt\bigg{)}dx$
$\displaystyle\ -\
\displaystyle\int_{-\infty}^{0}\bigg{(}\sqrt{\widehat{g}_{n}(-x)}-\sqrt{\vphantom{\widehat{g}_{n}}g_{0}(-x)}\bigg{)}\bigg{(}\displaystyle\int_{-\delta_{n}}^{0}\dfrac{g_{0}^{\prime}(-x-t)}{2\sqrt{g_{0}(-x-t)}}dt\bigg{)}dx\bigg{|}$
$\displaystyle=$ $\displaystyle\
2\bigg{|}\displaystyle\int_{0}^{\infty}\bigg{(}\sqrt{\widehat{g}_{n}(x)}-\sqrt{\vphantom{\widehat{g}_{n}}g_{0}(x)}\bigg{)}\bigg{(}\displaystyle\int_{-\delta_{n}}^{0}\dfrac{g_{0}^{\prime}(x+t)}{2\sqrt{g_{0}(x+t)}}dt\bigg{)}dx$
$\displaystyle\
-\displaystyle\int_{0}^{\infty}\bigg{(}\sqrt{\widehat{g}_{n}(x)}-\sqrt{\vphantom{\widehat{g}_{n}}g_{0}(x)}\bigg{)}\bigg{(}\displaystyle\int_{-\delta_{n}}^{0}\dfrac{g_{0}^{\prime}(x-t)}{2\sqrt{g_{0}(x-t)}}dt\bigg{)}dx\bigg{|},$

yielding

$|T_{c}|=2\bigg{|}\displaystyle\int_{0}^{\infty}\bigg{(}\sqrt{\widehat{g}_{n}(x)}-\sqrt{\vphantom{\widehat{g}_{n}}g_{0}(x)}\bigg{)}\bigg{(}\displaystyle\int_{-\delta_{n}}^{0}\bigg{(}\dfrac{g_{0}^{\prime}(x+t)}{2\sqrt{g_{0}(x+t)}}-\dfrac{g_{0}^{\prime}(x-t)}{2\sqrt{g_{0}(x-t)}}\bigg{)}dt\bigg{)}dx\bigg{|}.$
(6.58)

Using the Cauchy-Schwarz inequality, we obtain that

$\displaystyle\dfrac{|T_{c}|}{2|\delta_{n}|}\leq$ $\displaystyle\
\bigg{(}\displaystyle\int_{0}^{\infty}\bigg{(}\sqrt{\widehat{g}_{n}(x)}-\sqrt{\vphantom{\widehat{g}_{n}}g_{0}(x)}\bigg{)}^{2}dx\bigg{)}^{1/2}$
$\displaystyle\
\bigg{(}\displaystyle\int_{0}^{\infty}\bigg{(}\displaystyle\int_{-\delta_{n}}^{0}\dfrac{1}{|\delta_{n}|}\bigg{(}\dfrac{g_{0}^{\prime}(x+t)}{2\sqrt{g_{0}(x+t)}}-\dfrac{g_{0}^{\prime}(x-t)}{2\sqrt{g_{0}(x-t)}}\bigg{)}dt\bigg{)}^{2}dx\bigg{)}^{1/2}.$

Since $\sqrt{\widehat{g}_{n}(x)}-\sqrt{\vphantom{\widehat{g}_{n}}g_{0}(x)}$ is
an even function, the first term on the right hand side of the last inequality
is $\sqrt{2}H(\widehat{g}_{n},g_{0}).$ Hence,

$\displaystyle\dfrac{T_{c}^{2}}{8H(\widehat{g}_{n},g_{0})^{2}\delta_{n}^{2}}\leq$
$\displaystyle\
\displaystyle\int_{0}^{\infty}\bigg{(}\displaystyle\int_{-\delta_{n}}^{0}\dfrac{1}{|\delta_{n}|}\bigg{(}\dfrac{g_{0}^{\prime}(x+t)}{2\sqrt{g_{0}(x+t)}}-\dfrac{g_{0}^{\prime}(x-t)}{2\sqrt{g_{0}(x-t)}}\bigg{)}dt\bigg{)}^{2}dx,$

which, noting that

$t\mapsto\dfrac{g_{0}^{\prime}(x+t)}{2\sqrt{g_{0}(x+t)}}-\dfrac{g_{0}^{\prime}(x-t)}{2\sqrt{g_{0}(x-t)}}$

is an even function for each $x>0$, can be bounded above by

$\displaystyle\int_{0}^{\infty}|\delta_{n}|\bigg{(}\displaystyle\int_{0}^{|\delta_{n}|}\dfrac{1}{(\delta_{n})^{2}}\bigg{(}\dfrac{g_{0}^{\prime}(x+t)}{2\sqrt{g_{0}(x+t)}}-\dfrac{g_{0}^{\prime}(x-t)}{2\sqrt{g_{0}(x-t)}}\bigg{)}^{2}dt\bigg{)}dx$

using the Cauchy-Schwarz inequality. Therefore, we obtain

$\displaystyle\dfrac{T_{c}^{2}}{2H(\widehat{g}_{n},g_{0})^{2}\delta_{n}^{2}}\leq$
$\displaystyle\
\dfrac{1}{|\delta_{n}|}\displaystyle\int_{0}^{|\delta_{n}|}\bigg{[}\displaystyle\int_{0}^{\infty}\bigg{(}\dfrac{g_{0}^{\prime}(x+t)}{\sqrt{g_{0}(x+t)}}\bigg{)}^{2}dx+\displaystyle\int_{0}^{\infty}\bigg{(}\dfrac{g_{0}^{\prime}(x-t)}{\sqrt{g_{0}(x-t)}}\bigg{)}^{2}dx$
$\displaystyle\
-2\displaystyle\int_{0}^{\infty}\dfrac{g_{0}^{\prime}(x-t)}{\sqrt{g_{0}(x-t)}}\dfrac{g_{0}^{\prime}(x+t)}{\sqrt{g_{0}(x+t)}}dx\bigg{]}dt.$
(6.59)

For $t\geq 0,$

$\displaystyle\displaystyle\int_{0}^{\infty}\bigg{(}\dfrac{g_{0}^{\prime}(x+t)}{\sqrt{g_{0}(x+t)}}\bigg{)}^{2}dx=\displaystyle\int_{t}^{\infty}\bigg{(}\dfrac{g_{0}^{\prime}(x)}{\sqrt{g_{0}(x)}}\bigg{)}^{2}dx\leq\dfrac{\mathcal{I}_{f_{0}}}{2}.$

Now observe that for $z\in(-|\delta_{n}|,0),$

$|g_{0}^{\prime}(z)/\sqrt{g_{0}(z)}|=|\psi_{0}^{\prime}(z)|\sqrt{g_{0}(z)}\leq|\psi_{0}^{\prime}(\delta_{n})|\sqrt{g_{0}(0)}=O_{p}(1),$
(6.60)

since $\psi_{0}\in\mathcal{SC}_{0}$, and $\delta_{n}\to_{a.s.}0$. Hence, for
$t\in(0,|\delta_{n}|)$,

$\displaystyle\displaystyle\int_{0}^{\infty}\bigg{(}\dfrac{g_{0}^{\prime}(x-t)}{\sqrt{g_{0}(x-t)}}\bigg{)}^{2}dx\
=$ $\displaystyle\
\displaystyle\int_{-t}^{\infty}\bigg{(}\dfrac{g_{0}^{\prime}(z)}{\sqrt{g_{0}(z)}}\bigg{)}^{2}dz$
$\displaystyle=$ $\displaystyle\
\displaystyle\int_{-t}^{0}\bigg{(}\dfrac{g_{0}^{\prime}(x)}{\sqrt{g_{0}(z)}}\bigg{)}^{2}dz+\displaystyle\int_{0}^{\infty}\bigg{(}\dfrac{g_{0}^{\prime}(z)}{\sqrt{g_{0}(z)}}\bigg{)}^{2}dz$
$\displaystyle\leq$ $\displaystyle\
|\delta_{n}|\psi_{0}^{\prime}(\delta_{n})^{2}\sqrt{g_{0}(0)}+\mathcal{I}_{f_{0}}/2$
$\displaystyle=$ $\displaystyle\ |\delta_{n}|O_{p}(1)+\mathcal{I}_{f_{0}}/2,$

where the last step follows from (6.60). Hence, for any
$t\in(0,|\delta_{n}|),$

$\displaystyle\int_{0}^{\infty}\bigg{(}\dfrac{g_{0}^{\prime}(x+t)}{\sqrt{g_{0}(x+t)}}\bigg{)}^{2}dx+\displaystyle\int_{0}^{\infty}\bigg{(}\dfrac{g_{0}^{\prime}(x-t)}{\sqrt{g_{0}(x-t)}}\bigg{)}^{2}dx=|\delta_{n}|O_{p}(1)+\mathcal{I}_{f_{0}}.$
(6.61)

Our objective is to apply Fatou’s lemma on the third term on the right hand
side of (6.2). Therefore, we want to ensure that the integrand is non-
negative. Note that when $x\geq|\delta_{n}|$ and $t\in(0,|\delta_{n}|)$, we
have $x>t$, which leads to

$g_{0}^{\prime}(x-t)g_{0}^{\prime}(x+t)\geq 0.$ (6.62)

Keeping that in mind, we partition the term

$\displaystyle-\displaystyle\int_{0}^{\infty}\dfrac{g_{0}^{\prime}(x-t)}{\sqrt{g_{0}(x-t)}}\dfrac{g_{0}^{\prime}(x+t)}{\sqrt{g_{0}(x+t)}}dx$
$\displaystyle=$ $\displaystyle\
-\displaystyle\int_{|\delta_{n}|}^{\infty}\dfrac{g_{0}^{\prime}(x-t)}{\sqrt{g_{0}(x-t)}}\dfrac{g_{0}^{\prime}(x+t)}{\sqrt{g_{0}(x+t)}}dx-\displaystyle\int_{0}^{|\delta_{n}|}\dfrac{g_{0}^{\prime}(x-t)}{\sqrt{g_{0}(x-t)}}\dfrac{g_{0}^{\prime}(x+t)}{\sqrt{g_{0}(x+t)}}dx$
$\displaystyle\leq$ $\displaystyle\
-\displaystyle\int_{|\delta_{n}|}^{\infty}\dfrac{g_{0}^{\prime}(x-t)}{\sqrt{g_{0}(x-t)}}\dfrac{g_{0}^{\prime}(x+t)}{\sqrt{g_{0}(x+t)}}dx+|\delta_{n}|O_{p}(1),$

where the last step follows from (6.60). The above combined with (6.2) and
(6.61) leads to

$\displaystyle\limsup_{n}\dfrac{T_{c}^{2}}{2H(\widehat{g}_{n},g_{0})^{2}\delta_{n}^{2}}$
$\displaystyle\leq$ $\displaystyle\
\limsup_{n}\dfrac{1}{|\delta_{n}|}\displaystyle\int_{0}^{|\delta_{n}|}\bigg{[}|\delta_{n}|O_{p}(1)+\mathcal{I}_{f_{0}}-2\displaystyle\int_{|\delta_{n}|}^{\infty}\dfrac{g_{0}^{\prime}(x-t)}{\sqrt{g_{0}(x-t)}}\dfrac{g_{0}^{\prime}(x+t)}{\sqrt{g_{0}(x+t)}}dx\bigg{]}dt$
$\displaystyle=$ $\displaystyle\
O_{p}(1)\limsup_{n}|\delta_{n}|+\mathcal{I}_{f_{0}}$ $\displaystyle\
-2\liminf_{n}\dfrac{1}{|\delta_{n}|}\displaystyle\int_{0}^{|\delta_{n}|}\displaystyle\int_{|\delta_{n}|}^{\infty}\dfrac{g_{0}^{\prime}(x-t)}{\sqrt{g_{0}(x-t)}}\dfrac{g_{0}^{\prime}(x+t)}{\sqrt{g_{0}(x+t)}}dxdt$
$\displaystyle=$ $\displaystyle\
0+\mathcal{I}_{f_{0}}-2\liminf_{n}\displaystyle\int_{|\delta_{n}|}^{\infty}\dfrac{\displaystyle\int_{0}^{|\delta_{n}|}\dfrac{g_{0}^{\prime}(x+t)}{\sqrt{g_{0}(x+t)}}\dfrac{g_{0}^{\prime}(x-t)}{\sqrt{g_{0}(x-t)}}dt}{|\delta_{n}|}dx.$
(6.63)

Therefore, an application of Fatou’s Lemma and (6.62) yield

$\liminf_{n}\displaystyle\int_{|\delta_{n}|}^{\infty}\dfrac{\displaystyle\int_{0}^{|\delta_{n}|}\dfrac{g_{0}^{\prime}(x+t)}{\sqrt{g_{0}(x+t)}}\dfrac{g_{0}^{\prime}(x-t)}{\sqrt{g_{0}(x-t)}}dt}{|\delta_{n}|}dx\geq\displaystyle\int_{0}^{\infty}\dfrac{g_{0}^{\prime}(x)^{2}}{g_{0}(x)}dx=\dfrac{\mathcal{I}_{f_{0}}}{2}.$

Thus (6.2) leads to

$\dfrac{2T_{c}^{2}}{4H(\widehat{g}_{n},g_{0})^{2}\delta_{n}^{2}}=o_{p}(1).$

from which it is obvious that

$\dfrac{\sqrt{2}|T_{c}|}{|\delta_{n}|^{2}+H(\widehat{g}_{n},g_{0})^{2}}\leq\dfrac{\sqrt{2}|T_{c}|}{2H(\widehat{g}_{n},g_{0})|\delta_{n}|}=o_{p}(1),$

which proves (6.57) and thus completes the proof of part B of Theorem 9.
$\Box$

####  6.2.1 Proof of Lemma 7

From part A of Theorem 9, we obtain that $\widehat{f}_{n}$ is strongly
Hellinger consistent for $f_{0}$. As a consequence,
$||\widehat{f}_{n}-f_{0}||_{1}\to_{p}0$. Therefore, from proposition 2(c) of
Cule and Samworth (2010), it follows that,

$\sup\limits_{z\in\mathbb{R}}|\widehat{f}_{n}(x)-f_{0}(x)|\to_{a.s.}0.$ (6.64)

Now consider any compact set $K=[b_{1},b_{2}]\subset\mathrm{dom}(\phi_{0}).$
Observe that

$\displaystyle\sup_{x\in K}|\widehat{\phi}_{n}(x)-\phi_{0}(x)|$
$\displaystyle=$ $\displaystyle\ \sup_{x\in
K}\log\dfrac{\max(\widehat{f}_{n}(x),f_{0}(x))}{\min(\widehat{f}_{n}(x),f_{0}(x))}$
$\displaystyle=$ $\displaystyle\ \sup_{x\in
K}\log\bigg{(}\dfrac{|\widehat{f}_{n}(x)-f_{0}(x)|}{\min(\widehat{f}_{n}(x),f_{0}(x))}+1\bigg{)}$
$\displaystyle\leq\log\bigg{(}\dfrac{\sup_{x\in
K}|\widehat{f}_{n}(x)-f_{0}(x)|}{\min(\widehat{f}_{n}(b_{1}),\widehat{f}_{n}(b_{2}),f_{0}(b_{1}),f_{0}(b_{2}))}+1\bigg{)},$

since $\widehat{f}_{n}$ and $f_{0}$ are unimodal. From (6.64) it follows that
$\sup_{x\in K}|\widehat{f}_{n}(x)-f_{0}(x)|\to_{a.s.}0$. Also, since
$b_{1}\in\mathrm{dom}(\phi_{0})$,
$\widehat{f}_{n}(b_{1})\to_{a.s.}f_{0}(b_{1})>0$. The same holds for
$\widehat{f}_{n}(b_{2})$, which establishes that

$\sup_{x\in K}|\widehat{\phi}_{n}(x)-\phi_{0}(x)|\to_{a.s.}0.$

Now from Lemma $F.10$ of Kuchibhotla et al. (2017), it follows that if
$\phi_{0}$ is differentiable at $x\in K$, leading to

$\widehat{\phi}_{n}^{\prime}(x)\to_{a.s.}\phi_{0}^{\prime}(x),$

which combined with (6.64) yields

$\widehat{f}_{n}^{\prime}(x)\to_{a.s.}f_{0}^{\prime}(x).$

Part A of Theorem 9 also indicates that $H(\widehat{g}_{n},g_{0})\to_{a.s.}0$,
from which, proceeding like above, one can show that similar results hold for
$\widehat{g}_{n}$ and $\widehat{\psi}_{n}$ as well, which completes the proof.
$\Box$

###  6.3 Additional lemmas

###  6.4 Additional lemmas for the one-step estimators

######  Lemma 11.

Consider $\zeta,\zeta^{\prime}\in\text{int}(\mathrm{dom}(\tilde{\psi}_{0}))$
such that $\zeta\leq\zeta^{\prime}$. Suppose $\zeta_{n}\to_{a.s.}\zeta$, and
$\zeta^{\prime}_{n}\to_{a.s.}\zeta^{\prime}$. Letting
$\tilde{\psi}^{\prime}_{n}$ denote $\log\tilde{g}_{n}$, for
$\tilde{g}_{n}=\tilde{g}_{n}^{sym}$, $\widehat{g}_{\bar{\theta}_{n}}$,
$\tilde{g}_{n}^{geo,sym}$, or
$\widehat{h}_{n}(\bar{\theta}_{n}\pm\mathord{\cdot})$, under the conditions of
Theorem 12, we have

$\limsup_{n}\sup_{z\in[\zeta_{n},\zeta_{n}^{\prime}]}|\tilde{\psi}^{\prime}_{n}(z)|\leq
C_{\zeta,\zeta^{\prime}}\quad a.s.,$

where $C_{\zeta,\zeta^{\prime}}$ is a constant depending on $\zeta$ and
$\zeta^{\prime}$. When $\tilde{g}_{n}={(\tilde{g}_{n}^{sym})^{sm}}$ or
${\widehat{h}_{n}^{sm}}(\bar{\theta}_{n}\pm\mathord{\cdot})$, the above
conclusion holds for any $\zeta,\zeta^{\prime}\in\mathbb{R}$.

###### Proof.

When $\tilde{g}_{n}$ equals the densities $\widehat{g}_{\bar{\theta}_{n}}$,
$\widehat{f}_{n}^{geo,sym}$,
$\widehat{h}_{n}(\bar{\theta}_{n}\pm\mathord{\cdot})$, or
${\widehat{h}_{n}^{sm}}(\mathord{\cdot}+\bar{\theta}_{n})$, it is clear that
$\tilde{\psi}_{n}$ is concave. We will consider these choices of
$\tilde{g}_{n}$ first. For the first three cases, the pointwise limit of
$\tilde{g}_{n}$ is $\tilde{g}_{0}$ by Theorem 12. Note that we can choose
$\zeta_{1},\zeta_{1}^{\prime}\in\mathrm{dom}(\tilde{\psi}_{0})$ such that
$\tilde{\psi}_{0}^{\prime}$ is continuous at $\zeta_{1}$ and
$\zeta^{\prime}_{1}$, and
$[\zeta-\epsilon,\zeta^{\prime}+\epsilon]\subset[\zeta_{1},\zeta_{1}^{\prime}]$
for some sufficiently small $\epsilon>0$. Now the conditions on $\zeta_{n}$
and $\zeta_{n}^{\prime}$ underscore that for sufficiently large $n$,

$[\zeta_{n},\zeta_{n}^{\prime}]\subset[\zeta_{1},\zeta_{1}^{\prime}]\quad
a.s.$

Now note that a concave $\tilde{\psi}_{n}$ leads to a non-increasing
$\tilde{\psi}^{\prime}_{n}$. Therefore, on any interval,
$|\tilde{\psi}^{\prime}_{n}|$ attains its maxima on either of the endpoints.
As a consequence, the supremum of $|\tilde{\psi}^{\prime}_{n}|$ on the
interval $[\zeta_{1},\zeta_{1}^{\prime}]$ does not exceed
$|\tilde{\psi}^{\prime}_{n}(\zeta_{1})|+|\tilde{\psi}^{\prime}_{n}(\zeta^{\prime}_{1})|$,
which converges almost surely to
$|\tilde{\psi}_{0}^{\prime}(\zeta_{1})|+|\tilde{\psi}_{0}^{\prime}(\zeta^{\prime}_{1})|$
by Theorem 12. Therefore, we conclude that

$\limsup_{n}\sup_{z\in[\zeta_{n},\zeta_{n}^{\prime}]}|\tilde{\psi}^{\prime}_{n}(z)|<|\tilde{\psi}_{0}^{\prime}(\zeta_{1})|+|\tilde{\psi}_{0}^{\prime}(\zeta^{\prime}_{1})|\quad
a.s.,$ (6.65)

when $\tilde{g}_{n}=\widehat{g}_{\bar{\theta}_{n}}$,
$\widehat{f}_{n}^{geo,sym}$, or
$\widehat{h}_{n}(\mathord{\cdot}+\bar{\theta}_{n})$. Now consider
$\tilde{g}_{n}={\widehat{h}_{n}^{sm}}(\bar{\theta}_{n}\pm\mathord{\cdot})$.
Note that the $L_{1}$ limit of this $\tilde{g}_{n}$ is $\tilde{g}^{sm}_{0}$,
and $\mathrm{dom}(\tilde{\psi}^{sm}_{0})=\mathbb{R}$. Therefore, for any
$\zeta,\zeta^{\prime}\in\mathbb{R}$, the same arguments as the previous case
will lead to

$\limsup_{n}\sup_{z\in[\zeta_{n},\zeta_{n}^{\prime}]}|\tilde{\psi}^{\prime}_{n}(z)|<|(\tilde{\psi}^{sm}_{0})^{\prime}(\zeta_{1})|+|(\tilde{\psi}^{sm}_{0})^{\prime}(\zeta^{\prime}_{1})|\quad
a.s.$ (6.66)

Now let us consider the non-log-concave cases. Assume
$\tilde{g}_{n}=\tilde{g}_{n}^{sym}$. Then from (6.14) we obtain that

$\tilde{\psi}^{\prime}_{n}(x)=\varrho_{n}(x)\bigg{(}(\log\widehat{h}_{n})^{\prime}(\bar{\theta}_{n}+x)\bigg{)}-(1-\varrho_{n}(x))\bigg{(}(\log\widehat{h}_{n})^{\prime}(\bar{\theta}_{n}-x)\bigg{)},$

where $\widehat{h}_{n}$ is the log-concave MLE and $\varrho_{n}(x)<1$, leading
to

$\limsup_{n}\sup_{x\in[\zeta_{n},\zeta_{n}^{\prime}]}|\tilde{\psi}^{\prime}_{n}(x)|\leq\sup_{x\in[\zeta_{n},\zeta_{n}^{\prime}]}|(\log\widehat{h}_{n})^{\prime}(\bar{\theta}_{n}+x)|+\sup_{x\in[\zeta_{n},\zeta_{n}^{\prime}]}|(\log\widehat{h}_{n})^{\prime}(\bar{\theta}_{n}-x)|,$

which is bounded above by (6.65). Finally, when
$\tilde{g}_{n}={(\tilde{g}_{n}^{sym})^{sm}}$, from (6.15) we observe that
$\tilde{\psi}^{\prime}_{n}$ can be expressed in terms of
${\widehat{h}_{n}^{sm}}$ as

$\tilde{\psi}^{\prime}_{n}(x)=\varrho^{sm}_{n}(x)\bigg{(}(\log{\widehat{h}_{n}^{sm}})^{\prime}(\bar{\theta}_{n}+x)\bigg{)}-(1-\varrho^{sm}_{n}(x))\bigg{(}(\log{\widehat{h}_{n}^{sm}})^{\prime}(\bar{\theta}_{n}-x)\bigg{)},$

for some $\varrho^{sm}_{n}(x)<1$. Boundedness of $\tilde{\psi}^{\prime}_{n}$
then follows from (6.66). ∎

###  6.5 Additional lemmas for the MLE

######  Lemma 12.

Suppose $e^{\psi}\in\mathcal{SLC}_{0}$. Then $\psi(x)\leq-\log|2x|$ for all
$x\in\mathbb{R}$.

###### Proof.

Note that since $e^{\psi}\in\mathcal{SLC}_{0}$, for $x>0$, we have,

$1=\displaystyle\int_{-\infty}^{\infty}e^{\psi(z)}dz\geq\displaystyle\int_{-x}^{x}e^{\psi(z)}dz,$

which is bounded below by $2e^{\psi(x)}x$ because $\psi\in\mathcal{SC}_{0}$.
Hence the result follows. ∎

######  Lemma 13.

Suppose $F$ is a distribution function satisfying Condition A. Denote

$\text{int}(J(F))=(a,b),$

for some $a,b\in[-\infty,\infty]$. Suppose $\psi_{\theta}$ is the maximizer of
$\Psi(\theta,\psi,F)$ over $\psi\in\mathcal{SC}_{0}$ for some
$\theta\in\mathbb{R}$. Then

$\text{int}(\mathrm{dom}(\psi_{\theta}))=(-d,d),$

where $d=(b-\theta)\vee(\theta-a).$

Note that $a$ or $b$ can be $\infty$, in which case $d=\infty$, and
$\text{int}(\mathrm{dom}(\psi_{\theta}))=\mathbb{R}$.

###### Proof.

Note that $\text{int}(\mathrm{dom}(\psi_{\theta}))=(-c,c)$ for some $c>0$. We
first show that $c\geq d$. Consider the set
$A=\overline{(-d,d)}\setminus\mathrm{dom}(\psi_{\theta})$. Note that unless
$d=\infty$, $\overline{(-d,d)}=[-d,d]$. Letting $\Delta$ denote the indicator
function $1_{A}$, we observe that $\psi_{\theta}+\Delta=\psi_{\theta}$. Also,
note that

$\psi_{\theta}=\operatorname*{arg\,max}_{\psi\in\mathcal{SC}_{0}}\Psi(0,\psi,F(\mathord{\cdot}+\theta))=\operatorname*{arg\,max}_{\psi\in\mathcal{SC}_{0}}\omega(\psi,F(\mathord{\cdot}+\theta)),$

which follows from (1.4) and (3.7). Hence, Proposition $5$(ii) of Xu and
Samworth (2019) leads to

$\displaystyle\int_{-\infty}^{\infty}\Delta(x)dF(x+\theta)\leq\displaystyle\int_{-\infty}^{\infty}\Delta(x)e^{\psi_{\theta}(x)}dx.$

Since $\Delta=0$ on $\mathrm{dom}(\psi_{\theta})$, it follows that

$\displaystyle\int_{A}dF(x+\theta)=0,$

implying $(a,b)\cap(A+\theta)=\emptyset$.

It is easy to see that if either $a$ or $b$ equals $\infty$, $d=\infty$, and
$A+\theta=\mathbb{R}\setminus\mathrm{dom}(\psi_{\theta})$. Therefore,

$(a,b)\cap(A+\theta)=(a,b)\setminus\bigg{(}\mathrm{dom}(\psi_{\theta})+\theta\bigg{)}=\emptyset,$

which indicates that

$(a,b)\subset(-c,c)+\theta,$ (6.67)

implying $(-c,c)=\mathbb{R}$. Hence $(-c,c)=(-d,d)$, which completes the proof
for this case. Therefore, it only remains to prove Lemma 13 for the case when
both $a$ and $b$ are finite.

When $a,b<\infty$, there can be two sub-cases. In the first case,
$b-\theta\geq\theta-a$, which indicates $d=b-\theta$. In this case,
$[-d,d]+\theta=[2\theta-b,b]$, and

$[a,b]\subset[2\theta-b,b]=[-d,d]+\theta,$

leading to

$(a,b)\setminus\bigg{(}\mathrm{dom}(\psi_{\theta})+\theta\bigg{)}=(a,b)\cap(A+\theta)=\emptyset,$

from which (6.67) follows. Therefore, $b\leq c+\theta$, or $d\leq c$. In the
second case $b-\theta\leq\theta-a$, $d=\theta-a$ and
$[-d,d]+\theta=[a,2\theta-a]$. Proceeding similarly as in the first case, we
can again show that $c\geq d$. Hence, we have established that $c\geq d$ in
general.

Now suppose $c>d$, which implies that $a,b<\infty$. Observe that in this case,
there exists a $\psi\in\mathcal{SC}_{0}$, which equals $\psi_{\theta}$ on
$(-d,d)$ and $-\infty$ everywhere else. It is easy to verify that when $c>d$,

$\displaystyle\Psi(\theta,\psi_{\theta},F)-\Psi(\theta,\psi,F)$
$\displaystyle=$ $\displaystyle\
\displaystyle\int_{-d}^{d}e^{\psi_{\theta}(x)}dx-\displaystyle\int_{-c}^{c}e^{\psi_{\theta}(x)}dx<0.$

However, $\psi_{\theta}$ satisfies
$\Psi(\theta,\psi_{\theta},F)\geq\Psi(\theta,\psi,F)$ for all
$\psi\in\mathcal{SC}_{0}$, which leads to a contradiction. Hence, we conclude
that $c=d$, which completes the proof. ∎

######  Lemma 14.

Suppose $F$ satisfies Condition A and (3.20). Then

$\operatorname*{arg\,max}_{\phi\in\mathcal{C}}\omega(\phi,F)=\operatorname*{arg\,max}_{\phi\in\mathcal{SC}_{\theta}}\omega(\phi,F),$

where the criterion function $\omega$ is defined in (3.7).

###### Proof.

First we consider the special case $\theta=0$. For $\phi\in\mathcal{C}$ and
$F$ symmetric about $0$, using (3.20) with $\theta=0$, the criterion function
$\omega(\phi,F)$ can be written as

$\displaystyle\displaystyle\int_{-\infty}^{0}\phi(x)dF(x)+\displaystyle\int_{0}^{\infty}\phi(x)dF(x)-\displaystyle\int_{-\infty}^{\infty}e^{\phi(x)}dx$
$\displaystyle=$ $\displaystyle\
-\displaystyle\int_{0}^{\infty}\phi(-x)dF(-x)+\displaystyle\int_{0}^{\infty}\phi(x)dF(x)-\displaystyle\int_{-\infty}^{\infty}e^{\phi(x)}dx$
$\displaystyle=$ $\displaystyle\
\displaystyle\int_{0}^{\infty}\phi(-x)dF(x)+\displaystyle\int_{0}^{\infty}\phi(x)dF(x)-\displaystyle\int_{-\infty}^{\infty}e^{\phi(x)}dx$
$\displaystyle=$ $\displaystyle\
\displaystyle\int_{-\infty}^{\infty}\dfrac{\phi(-x)+\phi(x)}{2}dF(x)-\displaystyle\int_{-\infty}^{\infty}e^{\phi(x)}dx,$

which is not larger than

$\displaystyle\int_{-\infty}^{\infty}\dfrac{\phi(x)+\phi(-x)}{2}dF(x)-\displaystyle\int_{-\infty}^{\infty}e^{(\phi(x)+\phi(-x))/2}dx,$

by the convexity of the exponential function. This proves that a
$\phi\in\mathcal{SC}_{0}$ maximizes $\omega(\phi,F)$ over $\mathcal{C}$ when
$F$ is symmetric about $0$. The proof for a general $\theta$ follows in a
similar way. ∎

######  Lemma 15.

For any $\psi\in\mathcal{SC}_{0}$ with nonempty domain, and each $\epsilon>0$,
there exists a function $\psi^{(\epsilon)}\in\mathcal{SC}_{0}$ for each
$\epsilon>0$ such that the following conditions are met:

  1. (A)

$\psi(x)\leq\psi^{(\epsilon)}(x)\leq\psi(0)$ for all $x\in\mathbb{R}$.

  2. (B)

$\psi^{(\epsilon)}(x)\geq-|x|/\epsilon+\psi(0),$

for all $x\in\mathbb{R}$.

  3. (B)

$\psi^{(\epsilon)}(x)\downarrow\psi(x)$ for each $x\in\mathbb{R}$ as
$\epsilon\downarrow 0$.

###### Proof.

Following the construction in Lemma $4.3$ of Dümbgen et al. (2011) we set

$\psi^{(\epsilon)}(x)=\inf_{(t,c)\in A}(tx+c),\quad x\in\mathbb{R},$ (6.68)

where

$A=\bigg{\\{}(t,c)\in\mathbb{R}\times\mathbb{R}\ \bigg{|}\ |t|\leq
1/\epsilon,\ tx+c\geq\psi(x)\text{ for all }x\in\mathbb{R}\bigg{\\}}.$

Note that $\psi^{(\epsilon)}(x)$ is concave by Lemma $4.3$ of Dümbgen et al.
(2011). To show that $\psi^{(\epsilon)}$ is symmetric about $0$, observe that
if $tx+c\geq\psi(x)$ for all $x\in\mathbb{R}$, the symmetry of $\psi$ about
$0$ leads to

$-t(-x)+c\geq\psi(-x)\text{ for all }x\in\mathbb{R},$

which can be rewritten as

$-tx+c\geq\psi(x)\text{ for all }x\in\mathbb{R}.$

Hence, $(t,c)\in A$ implies that $(-t,c)\in A$. As a result,

$\inf_{(t,c)\in A}(tx+c)=\inf_{(-t,c)\in A}(tx+c)=\inf_{(t,c)\in A}(-tx+c),$

which implies that $\psi^{(\epsilon)}\in\mathcal{SC}_{0}$. That
$\psi\leq\psi^{(\epsilon)}$ follows from Lemma $4.3$ of Dümbgen et al. (2011).
Hence, to prove part A, we need to show that $\psi^{(\epsilon)}$ is bounded
above by $\psi(0)$.

Observe that since $\psi^{(\epsilon)}\in\mathcal{SC}_{0}$, it is bounded above
by $\psi^{(\epsilon)}(0)$, which, by (6.68), equals the infrimum of the set

$A^{\prime}=\\{c\ :\ (t,c)\in A\text{ for some }t\in\mathbb{R}\\}.$

However, since $c\in A^{\prime}$ satisfies $tx+c\geq\psi(x)$ for all
$x\in\mathbb{R}$, and some $t$, we conclude that Thus we obtain

$\psi^{(\epsilon)}(0)=\inf A^{\prime}\geq\psi(0).$ (6.69)

In light of (6.69), to prove part A of the current lemma, it suffices to show
that $\psi(0)\in A^{\prime}$. Since $\psi\in\mathcal{SC}_{0}$, for $x>0$, we
have $\psi(x)\leq\psi^{\prime}(0+)x+\psi(0)$. When $x\leq 0$, using the fact
that $\psi^{\prime}(0+)\leq 0$, we calculate

$\psi(x)=\psi(-x)\leq-\psi^{\prime}(0+)x+\psi(0)\leq\psi^{\prime}(0+)x+\psi(0).$

Therefore, for all $x\in\mathbb{R}$, the following holds:

$\psi(x)\leq\psi^{\prime}(0+)x+\psi(0).$

When $-\psi^{\prime}(0+)\leq 1/\epsilon$, (6.68) thus indicates that
$(-\psi^{\prime}(0+),\psi(0))\in A$. In case $-\psi^{\prime}(0+)>1/\epsilon$,
we still have

$\psi(x)\leq\psi^{\prime}(0+)x+\psi(0)<-x/\epsilon+\psi(0),\quad\text{ for all
}x>0.$

For $x\leq 0$,

$\psi(x)=\psi(-x)\leq x/\epsilon+\psi(0)\leq-x/\epsilon+\psi(0).$

Therefore, it turns out that if $-\psi(0+)>1/\epsilon$,

$\psi(x)\leq-x/\epsilon+\psi(0)\text{ for all }x\in\mathbb{R},$

which implies that $(1/\epsilon,\psi(0))\in A$. The above calculations
indicate that there always exists a $t\in\mathbb{R}$ such that $(t,\psi(0))\in
A$, entailing that $\psi(0)\in A^{\prime}$. This fact, combined with (6.69),
yields that $\psi^{(\epsilon)}(0)=\psi(0)$. Part A follows from (6.69).

Note that (6.68) and (6.69) imply that

$\psi^{(\epsilon)}(x)\geq-|x|/\epsilon+\psi(0),\quad x\in\mathbb{R},$

which settles the proof of part B.

Lemma 15(C) follows directly from Lemma $4.3$ of Dümbgen et al. (2011). ∎

The next lemma is Pratt’s lemma (Pratt, 1960, Theorem 1). We state it here for
convenience.

######  Lemma 16.

Suppose $(\Omega,\mathcal{F},\mu)$ is a measure space and $a_{n},b_{n},c_{n}$
are sequences of functions on $\Omega$ converging almost everywhere to
functions $a,b,c$ respectively. Also, all functions are integrable and $\int
a_{n}d\mu\to\int ad\mu$ and $\int c_{n}d\mu\to\int cd\mu.$ Moreover,
$a_{n}\leq b_{n}\leq c_{n}.$ Then

$\displaystyle\int b_{n}d\mu\to\displaystyle\int bd\mu.$

## References

  * Beran (1974) Beran, R. (1974).  Asymptotically efficient adaptive rank estimates in location models.  Ann. Statist., 2, 63–74.
  * Bhattacharyya and Bickel (2013) Bhattacharyya, S. and Bickel, P. J. (2013).  Adaptive estimation in elliptical distributions with extensions to high dimensions by.
  * Bickel et al. (1998) Bickel, P. J., Klaassen, C. A. J., Ritov, Y., and Wellner, J. A. (1998).  Efficient and Adaptive Estimation for Semiparametric Models.  Springer-Verlag, New York.
  * Birgé (1997) Birgé, L. (1997).  Estimation of unimodal densities without smoothness assumptions.  Ann. Statist., 25, 970–981.
  * Chen and Samworth (2013) Chen, Y. and Samworth, R. J. (2013).  Smoothed log-concave maximum likelihood estimation with applications.  Statist. Sinica, 23, 1373–1398.
  * Chen and Samworth (2016) Chen, Y. and Samworth, R. J. (2016).  Generalized additive and index models with shape constraints.  J R Stat Soc Series B Stat Methodol, 78, 729–754.
  * Cule and Samworth (2010) Cule, M. and Samworth, R. (2010).  Theoretical properties of the log-concave maximum likelihood estimator of a multidimensional density.  Electron. J. Statist., 4, 254–270.
  * Devroye (1987) Devroye, L. (1987).  A course in density estimation.  Progress in probability and statistics. Birkhäuser.
  * Doss and Wellner (2016) Doss, C. R. and Wellner, J. A. (2016).  Global rates of convergence of the MLEs of log-concave and $s$-concave densities.  Ann. Statist., 44, 954–981.
  * Doss and Wellner (2019) Doss, C. R. and Wellner, J. A. (2019).  Univariate log-concave density estimation with symmetry or modal constraints.  Electron. J. Stat., 25, 2391–2461.
  * Dümbgen and Rufibach (2009) Dümbgen, L. and Rufibach, K. (2009).  Maximum likelihood estimation of a log-concave density and its distribution function: Basic properties and uniform consistency.  Bernoulli, 15, 40–68.
  * Dümbgen et al. (2011) Dümbgen, L., Samworth, R., and Schuhmacher, D. (2011).  Approximation by log-concave distributions, with applications to regression.  Ann. Statist., 39, 702–730.
  * Groeneboom and Hendrickx (2018) Groeneboom, P. and Hendrickx, K. (2018).  Current status linear regression.  Ann. Statist., 46, 1415–1444.
  * Hodges and Lehmann (1963) Hodges, J. L. and Lehmann, E. L. (1963).  Estimates of location based on rank tests.  Ann. Math. Statist., 34, 598–611.
  * Huber (1964) Huber, P. J. (1964).  Robust estimation of a location parameter.  Ann. Math. Statist., 35, 73–101.
  * Kim and Samworth (2016) Kim, A. K. H. and Samworth, R. J. (2016).  Global rates of convergence in log-concave density estimation.  Ann. Statist., 44, 2756–2779.
  * Kuchibhotla et al. (2017) Kuchibhotla, A. K., Patra, R. K., and Sen, B. (2017).  Efficient estimation in convex single index models.  arXiv:1708.00145v2.
  * Murphy et al. (1999) Murphy, S. A., Van der Vaart, A., and Wellner, J. (1999).  Current status regression.  Math. Methods Statist., 8, 407–425.
  * Pal et al. (2007) Pal, J. K., Woodroofe, M., and Meyer, M. (2007).  Estimating a pólya frequency function2.  Lecture Notes-Monograph Series, 54, 239–249.
  * Pratt (1960) Pratt, J. W. (1960).  On interchanging limits and integrals.  Ann. Math. Statist., 31, 74–77.
  * Rockafellar (1970) Rockafellar, R. T. (1970).  Convex Analysis.  Princeton University Press.
  * Sacks (1975) Sacks, J. (1975).  An asymptotically efficient sequence of estimators of a location parameter.  Ann. Statist., 3, 285–298.
  * Seijo and Sen (2011) Seijo, E. and Sen, B. (2011).  Nonparametric least squares estimation of a multivariate convex regression function.  Ann. Statist., 39, 1633–1657.
  * Shorack (2000) Shorack, G. R. (2000).  Probability for Statisticians.  Springer.
  * Silverman (1982) Silverman, B. W. (1982).  On the estimation of a probability density function by the maximum penalized likelihood method.  Ann. Statist., pages 795–810.
  * Stein (1956) Stein, C. (1956).  Efficient nonparametric testing and estimation.  Proc. Third Berkley Symp. Math. Statist. Prob, 1, 187–196.
  * Stone (1975) Stone, C. J. (1975).  Adaptive maximum likelihood estimators of a location parameter.  Ann. Statist., 3, 267–284.
  * Takeuchi (1975) Takeuchi, K. (1975).  A survey of robust estimation of location: models and procedures, especially in case of measurement of a physical quantity.  Bull. Inst. Internat. Statist., 46, 336–348.  With discussion.
  * Van der Vaart (1998) Van der Vaart, A. (1998).  Asymptotic Statistics.  Asymptotic Statistics. Cambridge University Press.
  * Van der Vaart and Wellner (1996) Van der Vaart, A. W. and Wellner, J. A. (1996).  Weak Convergence and Empirical Processes.  Springer, New York.
  * van der Vaart and Wellner (2007) van der Vaart, A. W. and Wellner, J. A. W. (2007).  Empirical processes indexed by estimated functions.  Asymptotics: Particles, Processes and Inverse Problems, 55, 234–252.
  * Van Eeden (1970) Van Eeden, C. (1970).  Efficiency-robust estimation of location.  Ann. Math. Statist., 41, 172–181.
  * Villani (2003) Villani, C. (2003).  Topics in optimal transportation, volume 58 of Graduate Studies in Mathematics.  American Mathematical Society, Providence, RI.
  * Villani (2009) Villani, C. (2009).  Optimal transport; Old and New, volume 338 of Grundlehren der Mathematischen Wissenschaften.  Springer-Verlag, Berlin.  Old and new.
  * Xu and Samworth (2019) Xu, M. and Samworth, R. J. (2019).  High-dimensional nonparametric density estimation via symmetry and shape constraints.  https://arxiv.org/abs/1903.06092v1.