# Functional estimation in log-concave location families

Vladimir<EMAIL_ADDRESS>[ Martin
<EMAIL_ADDRESS>[ Georgia Institute of Technologym1
School of Mathematics
Georgia Institute of Technology
Atlanta, GA 30332-0160, USA

and
Department of Mathematics
Humboldt Universität zu Berlin
Unter den Linden 6, 10099, Berlin, Germany

###### Abstract

Let $\\{P_{\theta}:\theta\in{\mathbb{R}}^{d}\\}$ be a log-concave location
family with $P_{\theta}(dx)=e^{-V(x-\theta)}dx,$ where
$V:{\mathbb{R}}^{d}\mapsto{\mathbb{R}}$ is a known convex function and let
$X_{1},\dots,X_{n}$ be i.i.d. r.v. sampled from distribution $P_{\theta}$ with
an unknown location parameter $\theta.$ The goal is to estimate the value
$f(\theta)$ of a smooth functional $f:{\mathbb{R}}^{d}\mapsto{\mathbb{R}}$
based on observations $X_{1},\dots,X_{n}.$ In the case when $V$ is
sufficiently smooth and $f$ is a functional from a ball in a Hölder space
$C^{s},$ we develop estimators of $f(\theta)$ with minimax optimal error rates
measured by the $L_{2}({\mathbb{P}}_{\theta})$-distance as well as by more
general Orlicz norm distances. Moreover, we show that if $d\leq n^{\alpha}$
and $s>\frac{1}{1-\alpha},$ then the resulting estimators are asymptotically
efficient in Hájek-LeCam sense with the convergence rate $\sqrt{n}.$ This
generalizes earlier results on estimation of smooth functionals in Gaussian
shift models. The estimators have the form $f_{k}(\hat{\theta}),$ where
$\hat{\theta}$ is the maximum likelihood estimator and
$f_{k}:{\mathbb{R}}^{d}\mapsto{\mathbb{R}}$ (with $k$ depending on $s$) are
functionals defined in terms of $f$ and designed to provide a higher order
bias reduction in functional estimation problem. The method of bias reduction
is based on iterative parametric bootstrap and it has been successfully used
before in the case of Gaussian models.

62H12,

62G20, 62H25, 60B20,

Efficiency,

Smooth functionals,

Log-concave location family,

Bootstrap,

Concentration inequalities,

###### keywords:

[class=AMS]

###### keywords:

and t1Supported in part by NSF grants DMS-1810958 and DMS-2113121 m1Supported
by the Alexander von Humboldt Foundation

August 28, 2024

##  1 Introduction

Let $P$ be a log-concave probability distribution in ${\mathbb{R}}^{d}$ with
density $e^{-V},$ $V:{\mathbb{R}}^{d}\mapsto{\mathbb{R}}$ being a convex
function and let $P_{\theta}$, $\theta\in{\mathbb{R}}^{d}$ be a location
family generated by $P:$ $P_{\theta}(dx)=p_{\theta}(x)dx=e^{-V(x-\theta)}dx$,
$\theta\in{\mathbb{R}}^{d}.$ In other words, a random variable $X\sim
P_{\theta}$ could be represented as $X=\theta+\xi,$ where
$\theta\in{\mathbb{R}}^{d}$ is a location parameter and $\xi\sim P$ is a
random noise with log-concave distribution. Without loss of generality, one
can assume that ${\mathbb{E}}\xi=0$ (otherwise, one can replace function
$V(\cdot)$ by $V(\cdot+{\mathbb{E}}\xi)$). We will also assume that function
$V$ is known and $\theta$ is an unknown parameter of the model to be estimated
based on i.i.d. observations $X_{1},\dots,X_{n}$ of $X.$ We will refer to this
statistical model as a log-concave location family. Our main goal is to study
the estimation of $f(\theta)$ for a given smooth functional
$f:{\mathbb{R}}^{d}\mapsto{\mathbb{R}}.$ A natural estimator of location
parameter is the maximum likelihood estimator (MLE) defined as

$\displaystyle\hat{\theta}:=\operatorname{argmax}_{\theta\in{\mathbb{R}}^{d}}\limits\prod_{j=1}^{n}p_{\theta}(X_{j})=\operatorname{argmin}_{\theta\in{\mathbb{R}}^{d}}\limits\frac{1}{n}\sum_{j=1}^{n}V(X_{j}-\theta).$

Note that, by Lemma 2.2.1 in [7] for a log-concave density
$e^{-V},V:{\mathbb{R}}^{d}\mapsto{\mathbb{R}}$ there exist constants $A,B>0$
such that $e^{-V(x)}\leq Ae^{-B\|x\|}$ for all $x\in\mathbb{R}^{d},$ implying
that $V(x)\to\infty$ as $\|x\|\to\infty.$ It is easy to conclude from this
fact that MLE does exist. Moreover, it is unique if $V$ is strictly convex
(this condition is assumed in what follows). In addition, MLE $\hat{\theta}$
is an equivariant estimator with respect to the translation group in
${\mathbb{R}}^{d}:$

$\displaystyle\hat{\theta}(X_{1}+u,\dots,X_{n}+u)=\hat{\theta}(X_{1},\dots,X_{n})+u,u\in{\mathbb{R}}^{d}.$

Also note that

$\displaystyle{\mathbb{E}}_{\theta}V(X-\theta^{\prime})-{\mathbb{E}}_{\theta}V(X-\theta)=K(P_{\theta}\|P_{\theta^{\prime}}),$

where $K(P_{\theta}\|P_{\theta^{\prime}})$ is the Kullback-Leibler divergence
between $P_{\theta}$ and $P_{\theta^{\prime}},$ implying that $\theta$ is the
unique minimal point of
$\theta^{\prime}\mapsto{\mathbb{E}}_{\theta}V(X-\theta^{\prime}).$ The
uniqueness follows from the identifiability of parameter $\theta:$ if $\theta$
were not identifiable, we would have $V(x)=V(x+h),x\in{\mathbb{R}}^{d}$ for
some $h\neq 0,$ which would contradict the assumption that $V(x)\to\infty$ as
$\|x\|\to\infty.$ Assuming some regularity (including differentiability of
$V$) one can also argue as follows : using strict convexity and a necessary
condition of extremum ${\mathbb{E}}_{\theta}V^{\prime}(X-\theta)=0,$ we can
conclude that
$\mathbb{E}_{\theta}(V(X-\theta^{\prime})-V(X-\theta))>\mathbb{E}_{\theta}\langle
V^{\prime}(X-\theta),\theta-\theta^{\prime}\rangle=0$ for every
$\theta^{\prime}\neq\theta$.

Moreover, for differentiable $V,$ the score function of location family is
$\frac{\partial}{\partial\theta}\log p_{\theta}(X)=-V^{\prime}(X-\theta),$
and, under some regularity, ${\mathbb{E}}_{\theta}V^{\prime}(X-\theta)=0.$ In
addition, the Fisher information matrix of such a log-concave location family
is well defined, does not depend on $\theta$ and is given by

$\displaystyle{\mathcal{I}}$
$\displaystyle={\mathbb{E}}_{\theta}\frac{\partial}{\partial\theta}\log
p_{\theta}(X)\otimes\frac{\partial}{\partial\theta}\log
p_{\theta}(X)={\mathbb{E}}_{\theta}V^{\prime}(X-\theta)\otimes
V^{\prime}(X-\theta)$ $\displaystyle={\mathbb{E}}V^{\prime}(\xi)\otimes
V^{\prime}(\xi)=\int_{{\mathbb{R}}^{d}}V^{\prime}(x)\otimes
V^{\prime}(x)e^{-V(x)}dx$

(provided that the integral in the right hand side exists). Under further
regularity, for twice differentiable $V,$ we also have (via integration by
parts)

$\displaystyle{\mathcal{I}}={\mathbb{E}}V^{\prime\prime}(\xi)=\int_{{\mathbb{R}}^{d}}V^{\prime\prime}(x)e^{-V(x)}dx.$

Finally, if Fisher information ${\mathcal{I}}$ is non-singular, then, for a
fixed $d$ and $n\to\infty,$ MLE $\hat{\theta}$ is an asymptotically normal
estimator of $\theta$ with limit covariance ${\mathcal{I}}^{-1}:$

$\displaystyle\sqrt{n}(\hat{\theta}-\theta)\overset{d}{\to}N(0;{\mathcal{I}}^{-1})\
{\rm as}\ n\to\infty.$

Assumption 1 below suffices for all the above properties to hold.

It seems natural to estimate $f(\theta)$ by the plug-in estimator
$f(\hat{\theta}),$ where $\hat{\theta}$ is the MLE. Such an approach yields
asymptotically efficient estimators for regular statistical models in the case
of fixed dimension $d$ and $n\to\infty.$ In particular, for our location
family, we have by a standard application of the delta method that

$\displaystyle\sqrt{n}(f(\hat{\theta})-f(\theta))\overset{d}{\to}N(0;\sigma^{2}_{f}(\theta)),$

where
$\sigma^{2}_{f}(\theta):=\langle{\mathcal{I}}^{-1}f^{\prime}(\theta),f^{\prime}(\theta)\rangle.$
However, it is well known that plug-in estimators are sub-optimal in high-
dimensional problems mainly due to their large bias and, often, non-trivial
bias reduction methods are needed to achieve an optimal error rate. This has
been one of the difficulties in the problem of estimation of functionals of
parameters of high-dimensional and infinite-dimensional models for a number of
years [21, 22, 11, 4, 12, 25, 5, 26].

One approach to this problem is based on replacing $f$ by another function $g$
for which the bias of estimator $g(\hat{\theta})$ is small. To find such a
function $g,$ one has to solve approximately the “bias equation”
${\mathbb{E}}_{\theta}g(\hat{\theta})=f(\theta),\theta\in{\mathbb{R}}^{d}.$
This equation can be written as ${\mathcal{T}}g=f,$ where

$\displaystyle({\mathcal{T}}g)(\theta):={\mathbb{E}}_{\theta}g(\hat{\theta})=\int_{{\mathbb{R}}^{d}}g(u)P(\theta;du),\theta\in{\mathbb{R}}^{d}$

and $P(\theta;A),\theta\in{\mathbb{R}}^{d},A\subset{\mathbb{R}}^{d}$ is a
Markov kernel on ${\mathbb{R}}^{d}$ (or, more generally, on the parameter
space $\Theta$ of statistical model), providing the distribution of estimator
$\hat{\theta}.$ Denoting ${\mathcal{B}}:={\mathcal{T}}-{\mathcal{I}},$ where
${\mathcal{I}}$ is the identity operator in the space of bounded functions on
${\mathbb{R}^{d}}$ (not to be confused with the Fisher information also
denoted by ${\mathcal{I}}$), and assuming that $\hat{\theta}$ is close to
$\theta$ and, as a consequence, operator ${\mathcal{B}}$ is “small”, one can
view ${\mathcal{T}}={\mathcal{I}}+{\mathcal{B}}$ as a small perturbation of
identity. In such cases, one can try to solve the equation ${\mathcal{T}}g=f$
in terms of Neumann series
$g=({\mathcal{I}}-{\mathcal{B}}+{\mathcal{B}}^{2}-\dots)f.$ In what follows,
we denote by

$\displaystyle
f_{k}(\theta):=\sum_{j=0}^{k}(-1)^{j}({\mathcal{B}}^{j}f)(\theta),\theta\in{\mathbb{R}}^{d}$

the partial sum of this series and we will use $f_{k}(\hat{\theta})$ (for a
suitable choice of $k$ depending on smoothness of functional $f$) as an
estimator of $f(\theta).$ It is easy to see that its bias is

$\displaystyle{\mathbb{E}}_{\theta}f_{k}(\hat{\theta})-f(\theta)$
$\displaystyle=(\mathcal{B}f_{k})(\theta)+f_{k}(\theta)-f(\theta)$
$\displaystyle=(-1)^{k}({\mathcal{B}}^{k+1}f)(\theta),\theta\in{\mathbb{R}}^{d}.$

If ${\mathcal{B}}$ is “small” and $k$ is sufficiently large, one can hope to
achieve a bias reduction through estimator $f_{k}(\theta).$ Another way to
explain this approach is in terms of iterative bias reduction: since the bias
of plug-in estimator $f(\hat{\theta})$ is equal to $({\mathcal{B}}f)(\theta),$
one can estimate the bias by $({\mathcal{B}}f)(\hat{\theta})$ and the first
order bias reduction yields the estimator
$f_{1}(\hat{\theta})=f(\hat{\theta})-({\mathcal{B}}f)(\hat{\theta}).$ Its bias
is equal to $-({\mathcal{B}}^{2}f)(\theta)$ and the second order bias
reduction yields the estimator
$f_{2}(\hat{\theta})=f(\hat{\theta})-({\mathcal{B}}f)(\hat{\theta})+({\mathcal{B}}^{2}f)(\hat{\theta}),$
etc. This is close to the idea of iterative bootstrap bias reduction [10, 9,
13].

Let $\\{\hat{\theta}^{(k)}:k\geq 0\\}$ be the Markov chain with
$\hat{\theta}^{(0)}=\theta$ and with transition probability kernel
$P(\theta;A),\theta\in{\mathbb{R}}^{d},A\subset{\mathbb{R}}^{d}.$ This chain
can be viewed as an output of iterative application of parametric bootstrap to
estimator $\hat{\theta}$ in the model $X_{1},\dots,X_{n}$ i.i.d. $\sim
P_{\theta},\theta\in{\mathbb{R}}^{d}:$ at the first iteration, the data is
sampled from the distribution with parameter $\hat{\theta}^{(0)}=\theta$ and
estimator $\hat{\theta}^{(1)}=\hat{\theta}$ is computed; at the second
iteration, the data is sampled from the distribution $P_{\hat{\theta}}$
(conditionally on the value of $\hat{\theta}$) and bootstrap estimator
$\hat{\theta}^{(2)}$ is computed, and so on. We will call
$\\{\hat{\theta}^{(k)}:k\geq 0\\}$ the bootstrap chain of estimator
$\hat{\theta}.$ Clearly,
$({\mathcal{T}}^{k}f)(\theta)={\mathbb{E}}_{\theta}f(\hat{\theta}^{(k)})$ and,
by Newton’s binomial formula, we also have

$\displaystyle({\mathcal{B}}^{k}f)(\theta)$
$\displaystyle=(({\mathcal{T}}-{\mathcal{I}})^{k}f)(\theta)=\sum_{j=0}^{k}(-1)^{k-j}\binom{k}{j}({\mathcal{T}}^{j}f)(\theta)$
$\displaystyle={\mathbb{E}}_{\theta}\sum_{j=0}^{k}(-1)^{k-j}\binom{k}{j}f(\hat{\theta}^{(j)}).$
(1.1)

This means that $({\mathcal{B}}^{k}f)(\theta)$ is the expectation of the
$k$-th order difference of function $f$ along the bootstrap chain
$\\{\hat{\theta}^{(j)}:j\geq 0\\}.$ In the case when
$\|\hat{\theta}-\theta\|\lesssim\sqrt{\frac{d}{n}}$ with a high probability,
the same bound also holds for the increments
$\hat{\theta}^{(j+1)}-\hat{\theta}^{(j)}$ (conditionally on
$\hat{\theta}^{(j)}$). If functional $f$ is $k$ times differentiable and $d$
is small comparing with $n,$ one could therefore expect that
$({\mathcal{B}}^{k}f)(\theta)\lesssim\bigl{(}\frac{d}{n}\bigr{)}^{k/2}$ (based
on the analogy with the behavior of $k$-th order differences of $k$ times
differentiable functions in the real line). The justification of this
heuristic for general parametric models could be rather involved (see [14, 15,
18, 16]), but it will be shown below that it is much simpler in the case of
equivariant estimators $\hat{\theta}$ (such as the MLE) (see also [17, 19]).

Inserting representation (1.1) into the definition of $f_{k}$ and using a
simple combinatorial identity, we obtain the following useful representation
of function $f_{k}(\theta):$

$\displaystyle
f_{k}(\theta)=\mathbb{E}_{\theta}\sum_{j=0}^{k}(-1)^{j}\binom{k+1}{j+1}f(\hat{\theta}^{(j)}).$
(1.2)

The following notations will be used throughout the paper (and some of them
have been already used). For two variables $A,B\geq 0,$ $A\lesssim B$ means
that there exists an absolute constant $C>0$ such that $A\leq CB.$ The
notation $A\gtrsim B$ means that $B\lesssim A$ and $A\asymp B$ means that
$A\lesssim B$ and $B\lesssim A.$ If the constants in the relationships
$\lesssim,\gtrsim,\asymp$ depend on some parameter(s), say, on $\gamma,$ this
parameter will be used as a subscript of the relationship, say,
$A\lesssim_{\gamma}B.$ Given two square matrices $A$ and $B,$ $A\preceq B$
means that $B-A$ is positively semi-definite and $A\succeq B$ means that
$B\preceq A.$ The norm notation $\|\cdot\|$ (without further subscripts or
superscripts) will be used by default in certain spaces. For instance, it will
always denote the canonical Euclidean norm of ${\mathbb{R}}^{d},$ the operator
norm of matrices (linear transformations) and the operator norm of multilinear
forms. In some other cases, in particular for functional spaces
$L_{\infty},C^{s},$ etc, the corresponding subscripts will be used.

##  2 Main results

Recall that, for a convex non-decreasing function
$\psi:{\mathbb{R}}_{+}\mapsto{\mathbb{R}}_{+}$ with $\psi(0)=0,$ the Orlicz
$\psi$-norm of a r.v. $\eta$ is defined as

$\displaystyle\|\eta\|_{\psi}:=\inf\Bigl{\\{}c\geq
0:{\mathbb{E}}\psi\Bigl{(}\frac{|\eta|}{c}\Bigr{)}\leq 1\Bigr{\\}}.$

The Banach space of all r.v. on a probability space
$(\Omega,\Sigma,{\mathbb{P}})$ with finite $\psi$-norm is denoted by
$L_{\psi}({\mathbb{P}}).$ If $\psi(u)=u^{p},u\geq 0,p\geq 1,$ then the
$\psi$-norm coincides with the $L_{p}$-norm. Another important choice is
$\psi_{\alpha}(u)=e^{u^{\alpha}}-1,u\geq 0,\alpha\geq 1.$ In particular, for
$\alpha=1,$ $L_{\psi_{1}}$ is the space of sub-exponential r.v. and, for
$\alpha=2,$ $L_{\psi_{2}}$ is the space of sub-gaussian r.v. It is also well
known that the $\psi_{\alpha}$-norm is equivalent to the following norm
defined in terms of moments (or the $L_{p}$-norms):

$\displaystyle\|\eta\|_{\psi_{\alpha}}\asymp\sup_{p\geq
1}p^{-1/\alpha}{\mathbb{E}}^{1/p}|\eta|^{p},\alpha\geq 1.$ (2.1)

Note that the right hand side defines a norm for $0<\alpha<1,$ too, whereas
the left hand side is not a norm in this case since function $\psi_{\alpha}$
is not convex for $0<\alpha<1.$ Relationship (2.1) still holds for
$0<\alpha<1,$ but with constants depending on $\alpha$ as $\alpha$ approaches
$0.$ With a slight abuse of notations, we will define
$\|\eta\|_{\psi_{\alpha}}$ by the right hand side of (2.1) for all $\alpha>0.$

We will use the following definition of Hölder $C^{s}$-norms of functions
$f:{\mathbb{R}}^{d}\mapsto{\mathbb{R}}.$ For $j\geq 0,$ $f^{(j)}$ denotes the
$j$-th Fréchet derivative of $f.$ For $x\in{\mathbb{R}}^{d},$ $f^{(j)}(x)$ is
a $j$-linear form on ${\mathbb{R}}^{d}$ and the space of such forms will be
equipped with the operator norm. Clearly, $f^{(0)}=f$ and $f^{(1)}=f^{\prime}$
coincides with the gradient $\nabla f.$ If $f$ is $l$ times differentiable and
$s=l+\rho,$ $\rho\in(0,1],$ define

$\displaystyle\|f\|_{C^{s}}:=\max_{0\leq j\leq
l}\sup_{x\in{\mathbb{R}}^{d}}\|f^{(j)}(x)\|\vee\sup_{x,y\in{\mathbb{R}}^{d},x\neq
y}\frac{\|f^{(l)}(x)-f^{(l)}(y)\|}{\|x-y\|^{\rho}}.$

We will also frequently use $L_{\infty}$ and Lipschitz norms of functions and
their derivatives. For instance,
$\|f^{(j)}\|_{L_{\infty}}=\sup_{x\in{\mathbb{R}}^{d}}\|f^{(j)}(x)\|$ and
$\|f^{(j)}\|_{\rm Lip}=\sup_{x,x^{\prime}\in{\mathbb{R}}^{d},x\neq
x^{\prime}}\frac{\|f^{(j)}(x)-f^{(j)}(x^{\prime})\|}{\|x-x^{\prime}\|}.$

In what follows, we will use some facts related to isoperimetry and
concentration properties of log-concave measures. Given a Borel probability
measure $\mu$ on ${\mathbb{R}}^{d},$ let

$\displaystyle\mu^{+}(A):=\liminf_{\varepsilon\to
0}\frac{\mu(A_{\varepsilon})-\mu(A)}{\varepsilon},\
A\in{\mathcal{B}}({\mathbb{R}}^{d}),$

where $A_{\varepsilon}$ denotes the $\varepsilon$-neighborhood of $A$ and
${\mathcal{B}}({\mathbb{R}}^{d})$ is the Borel $\sigma$-algebra in
${\mathbb{R}}^{d}.$ The so called Cheeger isoperimetric constant of $\mu$ is
defined as

$\displaystyle
I_{C}(\mu):=\inf_{A\in{\mathcal{B}}({\mathbb{R}}^{d})}\frac{\mu^{+}(A)}{\mu(A)\wedge(1-\mu(A))}.$

According to the well known Kannan-Lovàsz-Simonovits (KLS) conjecture, for a
log-concave probability measure $\mu(dx)=e^{-V(x)}dx$ on ${\mathbb{R}}^{d}$
with covariance operator $\Sigma,$ $I_{C}(\mu)\gtrsim\|\Sigma\|^{-1/2}$ with a
dimension-free constant. This conjecture remains open, but the following deep
recent result by Chen [8] provides a lower bound on $I_{C}(\mu)$ that is
almost dimension-free.

######  Theorem 2.1.

There exists a constant $b>0$ such that, for all $d\geq 3$ and for all log-
concave distributions $\mu(dx)=e^{-V(x)}dx$ in ${\mathbb{R}}^{d}$ with
covariance $\Sigma,$

$\displaystyle I_{C}(\mu)\geq\|\Sigma\|^{-1/2}d^{-b(\frac{\log\log d}{\log
d})^{1/2}}.$

Isoperimetric constants $I_{C}(\mu)$ are known to be closely related to
important functional inequalities, in particular, to Poincaré inequality and
its generalizations (see, e.g., [6, 23]). It is said that Poincaré inequality
holds for a r.v. $\xi$ in ${\mathbb{R}}^{d}$ iff, for some constant $C>0$ and
for all locally Lipschitz functions $g:{\mathbb{R}}^{d}\mapsto{\mathbb{R}}$
(which, by Rademacher theorem, are differentiable almost everywhere),

$\displaystyle{\rm Var}(g(\xi))\leq C{\mathbb{E}}\|\nabla g(\xi)\|^{2}.$

The smallest value $c(\xi)$ of constant $C$ in the above inequality is called
the Poincaré constant of $\xi$ (clearly, it depends only on the distribution
of $\xi$). The following property of Poincaré constant will be frequently
used: if r.v. $\xi=(\xi_{1},\dots,\xi_{n})$ has independent components (with
$\xi_{j}$ being a r.v. in ${\mathbb{R}}^{d_{j}}$), then $c(\xi)=\max_{1\leq
j\leq n}c(\xi_{j})$ (see [20], Corollary 5.7).

If now $\xi\sim\mu$ in ${\mathbb{R}}^{d},$ then the following Cheeger’s
inequality holds (see, e.g., [23], Theorem 1.1):

$\displaystyle c(\xi)\leq\frac{4}{I_{C}^{2}(\mu)}.$

Moreover, the following $L_{p}$-version of Poincaré inequality holds for all
$p\geq 1$ and for all locally Lipschitz functions
$g:{\mathbb{R}}^{d}\mapsto{\mathbb{R}}$ (see [6], Theorem 3.1)

$\displaystyle\|g(\xi)-{\mathbb{E}}g(\xi)\|_{L_{p}}\lesssim\frac{p}{I_{C}(\mu)}\|\|\nabla
g(\xi)\|\|_{L_{p}}.$ (2.2)

######  Remark 2.1.

Note that, if $\xi\sim\mu$ and $\mu(dx)=e^{-V(x)}dx$ is log-concave, then (see
[23], Theorem 1.5)

$\displaystyle c(\xi)\asymp\frac{1}{I_{C}^{2}(\mu)}$

and, by Theorem 2.1, we have

$\displaystyle c(\xi)\leq\|\Sigma\|\ d^{2b(\frac{\log\log d}{\log d})^{1/2}}.$
(2.3)

In what follows, we denote the Poincaré constant $c(\xi)$ of r.v. $\xi\sim\mu$
with log-concave distribution $\mu(dx)=e^{-V(x)}dx$ by $c(V).$ Bound (2.3)
implies that $c(V)\lesssim_{\epsilon}\|\Sigma\|d^{\epsilon}$ for all
$\epsilon>0.$ Also, with this notation, we can rewrite the $L_{p}$-version
(2.2) of Poincaré inequality as follows:

$\displaystyle\|g(\xi)-{\mathbb{E}}g(\xi)\|_{L_{p}}\lesssim\sqrt{c(V)}p\
\|\|\nabla g(\xi)\|\|_{L_{p}}.$ (2.4)

This concentration bound will be our main tool in Section 4. It will be
convenient for our purposes to express it in terms of local Lipschitz
constants of $g$ defined as follows:

$\displaystyle(Lg)(x):=\inf_{U\ni x}\sup_{x^{\prime},x^{\prime\prime}\in
U,x^{\prime}\neq
x^{\prime\prime}}\frac{|g(x^{\prime})-g(x^{\prime\prime})|}{\|x^{\prime}-x^{\prime\prime}\|},x\in{\mathbb{R}}^{d}$

where the infimum is taken over all the balls $U$ centered at $x.$ Similar
definition could be also used for vector valued functions $g.$ Clearly,
$\|\nabla g(x)\|\leq(Lg)(x),x\in{\mathbb{R}}^{d}$ and (2.4) implies that

$\displaystyle\|g(\xi)-{\mathbb{E}}g(\xi)\|_{L_{p}}\lesssim\sqrt{c(V)}p\
\|(Lg)(\xi)\|_{L_{p}}.$ (2.5)

The following assumptions on $V$ will be used throughout the paper.

######  Assumption 1.

Suppose that

  1. (i)

$V$ is strictly convex and twice continuously differentiable such that, for
some constants $M,L>0,$ $\|V^{\prime\prime}\|_{L_{\infty}}\leq M$ and
$\|V^{\prime\prime}\|_{\rm Lip}\leq L.$

  2. (ii)

For some constant $m>0,$ ${\mathcal{I}}\succeq mI_{d}.$

Under Assumption 1, we have
${\mathcal{I}}=\mathbb{E}V^{\prime\prime}(\xi)\preceq MI_{d}$ and thus $m\leq
M$.

######  Remark 2.2.

Obviously, Assumption 1 holds in the Gaussian case, when
$V(x)=c_{1}+c_{2}\|x\|^{2},x\in{\mathbb{R}}^{d}.$ In this case,
$V^{\prime\prime}(x)=mI_{d},x\in{\mathbb{R}}^{d}$ for $m=2c_{2}>0,$ which is
much stronger than Assumption 1, (ii). Assumption 1 also holds, for instance,
for $V(x)=\varphi(\|x\|^{2}),x\in{\mathbb{R}}^{d},$ where $\varphi$ is a
$C^{\infty}$ function in ${\mathbb{R}}$ such that $\varphi^{\prime\prime}$ is
supported in $[0,1],$ $\varphi^{\prime\prime}(t)\geq 0,t\in{\mathbb{R}}$ and
$\varphi^{\prime}(0)=0.$ Of course, in this case, the condition
$V^{\prime\prime}(x)\succeq mI_{d}$ does not hold uniformly in $x$ for any
positive $m,$ but Assumption 1, (ii) holds. If
$V(x)=c_{1}+\|x\|^{2p},x\in{\mathbb{R}}^{d}$ for some $p\geq 1/2,$ it is easy
to check that Assumption 1 holds only for $p=1.$

We are now ready to state our main result.

######  Theorem 2.2.

Suppose Assumption 1 holds and $d\leq\gamma n,$ where

$\displaystyle\gamma:=c\Bigl{(}\frac{m}{M}\wedge\frac{m^{2}}{L\sqrt{M}}\Bigr{)}^{2}$

with a small enough constant $c>0.$ Let $f\in C^{s}$ for some $s=k+1+\rho,$
$k\geq 0,$ $\rho\in(0,1].$ Then

$\displaystyle\sup_{\theta\in{\mathbb{R}}^{d}}\Bigl{\|}f_{k}(\hat{\theta})-f(\theta)-n^{-1}\sum_{j=1}^{n}\langle
V^{\prime}(\xi_{j}),{\mathcal{I}}^{-1}f^{\prime}(\theta)\rangle\Bigr{\|}_{L_{\psi_{2/3}}({\mathbb{P}}_{\theta})}$
$\displaystyle\lesssim_{L,M,m,s}\|f\|_{C^{s}}\Bigl{[}\sqrt{\frac{c(V)}{n}}\Bigl{(}\frac{d}{n}\Bigr{)}^{\rho/2}+\Bigl{(}\sqrt{\frac{d}{n}}\Bigr{)}^{s}\Bigr{]}.$

Theorem 2.2 shows that $f_{k}(\hat{\theta})-f(\theta)$ can be approximated by
a normalized sum of i.i.d. mean zero r.v. $n^{-1}\sum_{j=1}^{n}\langle
V^{\prime}(\xi_{j}),{\mathcal{I}}^{-1}f^{\prime}(\theta)\rangle.$ Moreover,
the error of this approximation is of the order $o(n^{-1/2})$ provided that
$d=o(n^{\alpha})$ for some $\alpha\in(0,1)$ satisfying $s>\frac{1}{1-\alpha}$
and that $\|\Sigma\|$ is bounded by a constant. This follows from the fact
that $c(V)\lesssim_{\epsilon}d^{\epsilon}\|\Sigma\|$ for an arbitrarily small
$\epsilon>0$ (see Remark 2.1). In addition, by Lemma 3.1 below, r.v. $\langle
V^{\prime}(\xi),{\mathcal{I}}^{-1}f^{\prime}(\theta)\rangle$ is subgaussian
with

$\displaystyle\|\langle
V^{\prime}(\xi),{\mathcal{I}}^{-1}f^{\prime}(\theta)\rangle\|_{\psi_{2}}\lesssim\sqrt{M}\|{\mathcal{I}}^{-1}f^{\prime}(\theta)\|\lesssim\frac{\sqrt{M}}{m}\|f^{\prime}(\theta)\|.$
(2.6)

As a result, we can obtain the following simple, but important corollaries of
Theorem 2.2. Recall that
$\sigma^{2}_{f}(\theta)=\langle{\mathcal{I}}^{-1}f^{\prime}(\theta),f^{\prime}(\theta)\rangle.$

######  Corollary 2.1.

Under the conditions of Theorem 2.2,

$\displaystyle\sup_{\theta\in{\mathbb{R}}^{d}}\Bigl{|}\|f_{k}(\hat{\theta})-f(\theta)\|_{L_{2}({\mathbb{P}}_{\theta})}-\frac{\sigma_{f}(\theta)}{\sqrt{n}}\Bigr{|}\lesssim_{L,M,m,s}\|f\|_{C^{s}}\Bigl{[}\sqrt{\frac{c(V)}{n}}\Bigl{(}\frac{d}{n}\Bigr{)}^{\rho/2}+\Bigl{(}\sqrt{\frac{d}{n}}\Bigr{)}^{s}\Bigr{]}.$

This corollary immediately follows from the bound of Theorem 2.2 and the fact
that the $L_{2}$-norm is dominated by the $\psi_{2/3}$-norm. It implies the
second claim of the following proposition.

######  Proposition 2.1.

Let $f\in C^{s}$ for some $s>0.$
1\. For $s\in(0,1],$

$\displaystyle\sup_{\|f\|_{C^{s}}\leq
1}\sup_{\theta\in{\mathbb{R}}^{d}}\|f(\hat{\theta})-f(\theta)\|_{L_{2}({\mathbb{P}}_{\theta})}\lesssim_{L,M,m,s}\Bigl{(}\sqrt{\frac{d}{n}}\Bigr{)}^{s}\wedge
1.$

2\. For $s=k+1+\rho$ for some $k\geq 0$ and $\rho\in(0,1],$ suppose Assumption
1 holds and also $\|\Sigma\|\lesssim 1.$ If $d\lesssim n^{\alpha}$ for some
$\alpha\in(0,1),$ then

$\displaystyle\sup_{\|f\|_{C^{s}}\leq
1}\sup_{\theta\in{\mathbb{R}}^{d}}\|f_{k}(\hat{\theta})-f(\theta)\|_{L_{2}({\mathbb{P}}_{\theta})}\lesssim_{L,M,m,s}\Bigl{(}\frac{1}{\sqrt{n}}\vee\Bigl{(}\sqrt{\frac{d}{n}}\Bigr{)}^{s}\Bigr{)}\wedge
1.$ (2.7)

Combining bound (2.7) with the following result shows some form of minimax
optimality of estimator $f_{k}(\hat{\theta}).$

######  Proposition 2.2.

Suppose Assumption 1 holds. Then, for all $s>0,$

$\displaystyle\sup_{\|f\|_{C^{s}}\leq 1}\inf_{\hat{T}_{n}}\sup_{\|\theta\|\leq
1}\|\hat{T}_{n}-f(\theta)\|_{L_{2}({\mathbb{P}}_{\theta})}\gtrsim_{m,M}\Big{(}\frac{1}{\sqrt{n}}$
$\displaystyle\vee\Big{(}\sqrt{\frac{d}{n}}\Big{)}^{s}\Big{)}\wedge 1,$

where the infimum is taken over all estimators
$\hat{T}_{n}=\hat{T}_{n}(X_{1},\dots,X_{n}).$

The proof of this result is similar to the proof of Theorem 2.2 in [18] in the
Gaussian case. Some further comments will be provided in Section 6.

Corollary 2.1 also implies that, for all $\theta\in{\mathbb{R}}^{d},$

$\displaystyle\|f_{k}(\hat{\theta})-f(\theta)\|_{L_{2}({\mathbb{P}}_{\theta})}\leq\frac{\sigma_{f}(\theta)}{\sqrt{n}}+C\|f\|_{C^{s}}\Bigl{[}\sqrt{\frac{c(V)}{n}}\Bigl{(}\frac{d}{n}\Bigr{)}^{\rho/2}+\Bigl{(}\sqrt{\frac{d}{n}}\Bigr{)}^{s}\Bigr{]},$

where $C$ is a constant depending on $M,L,m,s.$ If $d\lesssim n^{\alpha}$ for
some $\alpha\in(0,1)$ and $s>\frac{1}{1-\alpha},$ it easily follows that, for
all $B>0,$

$\displaystyle\limsup_{n\to\infty}\sup_{(f,\theta):\frac{\|f\|_{C^{s}}}{\sigma_{f}(\theta)}\leq
B}\frac{\sqrt{n}\|f_{k}(\hat{\theta})-f(\theta)\|_{L_{2}({\mathbb{P}}_{\theta})}}{\sigma_{f}(\theta)}\leq
1.$ (2.8)

The following minimax lower bound will be proved in Section 6.

######  Proposition 2.3.

Suppose Assumption 1 holds and let $f\in C^{s}$ for some
$s=1+\rho,\rho\in(0,1].$ Then, for all $c>0$ and all
$\theta_{0}\in{\mathbb{R}}^{d},$

$\displaystyle\inf_{\hat{T}_{n}}\sup_{\|\theta-\theta_{0}\|\leq\frac{c}{\sqrt{n}}}\frac{\sqrt{n}\|\hat{T}_{n}-f(\theta)\|_{L_{2}({\mathbb{P}}_{\theta})}}{\sigma_{f}(\theta)}\geq
1-\frac{3\pi}{\sqrt{8m}c}-\frac{2}{\sqrt{m}}\frac{\|f\|_{C^{s}}}{\sigma_{f}(\theta_{0})}\Bigl{(}\frac{c}{\sqrt{n}}\Bigr{)}^{\rho},$

where the infimum is taken over all estimators
$\hat{T}_{n}=\hat{T}_{n}(X_{1},\dots,X_{n}).$

The bound of Proposition 2.3 easily implies that, for all $B>0,$

$\displaystyle\lim_{c\to\infty}\liminf_{n\to\infty}\inf_{(f,\theta_{0}):\frac{\|f\|_{C^{s}}}{\sigma_{f}(\theta_{0})}\leq
B}\inf_{\hat{T}_{n}}\sup_{\|\theta-\theta_{0}\|\leq\frac{c}{\sqrt{n}}}\frac{\sqrt{n}\|\hat{T}_{n}-f(\theta)\|_{L_{2}({\mathbb{P}}_{\theta})}}{\sigma_{f}(\theta)}\geq
1.$

Along with (2.8), it shows local asymptotic minimaxity of estimator
$f_{k}(\hat{\theta}).$

The next corollaries will be based on the results by Rio [27] on convergence
rates in CLT in Wasserstein type distances. For r.v. $\eta_{1},\eta_{2}$ and a
convex non-decreasing function $\psi:{\mathbb{R}}_{+}\mapsto{\mathbb{R}}_{+}$
with $\psi(0)=0,$ define the Wasserstein $\psi$-distance between $\eta_{1}$
and $\eta_{2}$ as

$\displaystyle
W_{\psi}(\eta_{1},\eta_{2}):=\inf\Bigl{\\{}\|\eta_{1}^{\prime}-\eta_{2}^{\prime}\|_{\psi}:\eta_{1}^{\prime}\overset{d}{=}\eta_{1},\eta_{2}^{\prime}\overset{d}{=}\eta_{2}\Bigr{\\}}.$

For $\psi(u)=u^{p},u\geq 0,p\geq 1,$ we will use the notation $W_{p}$ instead
of $W_{\psi}.$ For $\psi=\psi_{\alpha},\alpha>0,$ we will modify the above
definition using a version of $\psi$-norm defined in terms of the moments. Let
$\eta_{1},\dots,\eta_{n}$ be i.i.d. copies of a mean zero r.v. $\eta$ with
${\mathbb{E}}\eta^{2}=1.$ It was proved in [27] (see Theorem 4.1 and equation
(4.3)) that for all $r\in(1,2],$

$\displaystyle
W_{r}\Bigl{(}\frac{\eta_{1}+\dots+\eta_{n}}{\sqrt{n}},Z\Bigr{)}\lesssim\frac{{\mathbb{E}}^{1/r}\eta^{r+2}}{\sqrt{n}},$

where $Z\sim N(0,1).$ Applying this bound to $\eta:=\frac{\langle
V^{\prime}(\xi),{\mathcal{I}}^{-1}f^{\prime}(\theta)\rangle}{\sigma_{f}(\theta)}$
yields

$\displaystyle W_{2}\Bigl{(}\frac{1}{\sqrt{n}}\sum_{j=1}^{n}\langle
V^{\prime}(\xi_{j}),{\mathcal{I}}^{-1}f^{\prime}(\theta)\rangle,\sigma_{f}(\theta)Z\Bigr{)}\lesssim\frac{{\mathbb{E}}^{1/2}\langle
V^{\prime}(\xi),{\mathcal{I}}^{-1}f^{\prime}(\theta)\rangle^{4}}{\sigma_{f}(\theta)}n^{-1/2}.$

Thus, Theorem 2.2 implies the following corollary.

######  Corollary 2.2.

Under the conditions of Theorem 2.2, for all $\theta\in{\mathbb{R}}^{d}$

$\displaystyle
W_{2,{\mathbb{P}}_{\theta}}\Bigl{(}\sqrt{n}(f_{k}(\hat{\theta})-f(\theta)),\sigma_{f}(\theta)Z\Bigr{)}$
$\displaystyle\leq C_{1}\frac{{\mathbb{E}}^{1/2}\langle
V^{\prime}(\xi),{\mathcal{I}}^{-1}f^{\prime}(\theta)\rangle^{4}}{\sigma_{f}(\theta)}n^{-1/2}+C_{2}\|f\|_{C^{s}}\Bigl{[}\sqrt{\frac{c(V)}{n}}\Bigl{(}\frac{d}{n}\Bigr{)}^{\rho/2}+\Bigl{(}\sqrt{\frac{d}{n}}\Bigr{)}^{s}\Bigr{]},$

where $C_{1}>0$ is an absolute constant and $C_{2}>0$ is a constant that could
depend on $M,L,m,s.$

Using (2.6), it is easy to check that, under Assumption 1,

$\displaystyle{\mathbb{E}}^{1/2}\langle
V^{\prime}(\xi),{\mathcal{I}}^{-1}f^{\prime}(\theta)\rangle^{4}\lesssim\frac{M}{m^{2}}\|f^{\prime}(\theta)\|^{2}$

and, in addition, $\sigma_{f}^{2}(\theta)\geq
M^{-1}\|f^{\prime}(\theta)\|^{2}.$ This yields

$\displaystyle\frac{{\mathbb{E}}^{1/2}\langle
V^{\prime}(\xi),{\mathcal{I}}^{-1}f^{\prime}(\theta)\rangle^{4}}{\sigma_{f}(\theta)}\lesssim_{M,m}\|f^{\prime}(\theta)\|\lesssim_{M,m}\|f\|_{C^{s}}.$

Therefore, if $d\leq n^{\alpha}$ for some $\alpha\in(0,1)$ and
$s>\frac{1}{1-\alpha},$ then

$\displaystyle\sup_{\|f\|_{C^{s}}\leq
1}\sup_{\theta\in{\mathbb{R}}^{d}}W_{2,{\mathbb{P}}_{\theta}}\Bigl{(}\sqrt{n}(f_{k}(\hat{\theta})-f(\theta)),\sigma_{f}(\theta)Z\Bigr{)}\to
0\ {\rm as}\ n\to\infty,$

implying asymptotic normality of estimator $f_{k}(\hat{\theta})$ of
$f(\theta)$ with rate $\sqrt{n}$ and limit variance $\sigma_{f}^{2}(\theta).$
It is also easy to show that, under the same conditions on $d$ and $s,$ we
have, for all $B>0,$

$\displaystyle\sup_{(f,\theta):\frac{\|f\|_{C^{s}}}{\sigma_{f}(\theta)}\leq
B}W_{2,{\mathbb{P}}_{\theta}}\Bigl{(}\frac{\sqrt{n}(f_{k}(\hat{\theta})-f(\theta))}{\sigma_{f}(\theta)},Z\Bigr{)}\to
0\ {\rm as}\ n\to\infty,$

which implies

$\displaystyle\sup_{(f,\theta):\frac{\|f\|_{C^{s}}}{\sigma_{f}(\theta)}\leq
B}\sup_{x\in{\mathbb{R}}}\Bigl{|}{\mathbb{P}}_{\theta}\Bigl{\\{}\frac{\sqrt{n}(f_{k}(\hat{\theta})-f(\theta))}{\sigma_{f}(\theta)}\leq
x\Bigr{\\}}-{\mathbb{P}}\\{Z\leq x\\}\Bigr{|}\to 0\ {\rm as}\ n\to\infty.$

It was also proved in [27], Theorem 2.1 that, for i.i.d. copies
$\eta_{1},\dots,\eta_{n}$ of mean zero r.v. $\eta$ with
${\mathbb{E}}\eta^{2}=1$ and $\|\eta\|_{\psi_{1}}<\infty$ and for some
constant $C(\|\eta\|_{\psi_{1}})<\infty,$

$\displaystyle
W_{\psi_{1}}\Bigl{(}\frac{\eta_{1}+\dots+\eta_{n}}{\sqrt{n}},Z\Bigr{)}\lesssim\frac{C(\|\eta\|_{\psi_{1}})}{\sqrt{n}}.$

We will again apply this to $\eta:=\frac{\langle
V^{\prime}(\xi),{\mathcal{I}}^{-1}f^{\prime}(\theta)\rangle}{\sigma_{f}(\theta)}.$
In this case, by Lemma 3.1, we have

$\displaystyle\|\langle
V^{\prime}(\xi),{\mathcal{I}}^{-1}f^{\prime}(\theta)\rangle\|_{\psi_{1}}\lesssim\frac{\sqrt{M}}{m}\|f^{\prime}(\theta)\|.$

Also, $\sigma_{f}(\theta)\geq\frac{\|f^{\prime}(\theta)\|}{\sqrt{M}},$
implying $\|\eta\|_{\psi_{1}}\lesssim\frac{M}{m}.$ As a result, we get

$\displaystyle W_{\psi_{1}}\Bigl{(}\frac{1}{n}\sum_{j=1}^{n}\langle
V^{\prime}(\xi_{j}),{\mathcal{I}}^{-1}f^{\prime}(\theta)\rangle,\frac{\sigma_{f}(\theta)Z}{\sqrt{n}}\Bigr{)}\lesssim_{M,m}\frac{\sigma_{f}(\theta)}{n}.$

Combining this with the bound of Theorem 2.2 yields the following extension of
Corollary 2.1.

######  Corollary 2.3.

Under the conditions of Theorem 2.2, for all convex non-decreasing functions
$\psi:{\mathbb{R}}_{+}\rightarrow{\mathbb{R}}_{+}$ with $\psi(0)=0,$
satisfying the condition $\psi(u)\leq\psi_{2/3}(cu),u\geq 0$ for some constant
$c>0,$

$\displaystyle\sup_{\theta\in{\mathbb{R}}^{d}}\Bigl{|}\|f_{k}(\hat{\theta})-f(\theta)\|_{L_{\psi}({\mathbb{P}}_{\theta})}-\frac{\sigma_{f}(\theta)}{\sqrt{n}}\|Z\|_{\psi}\Bigr{|}$
$\displaystyle\lesssim_{L,M,m,s}\|f\|_{C^{s}}\Bigl{[}\sqrt{\frac{c(V)}{n}}\Bigl{(}\frac{d}{n}\Bigr{)}^{\rho/2}+\Bigl{(}\sqrt{\frac{d}{n}}\Bigr{)}^{s}\Bigr{]}.$

######  Remark 2.3.

Similar results were obtained in [17] in the case of Gaussian shift models, in
[19] in the case of more general Poincaré random shift models and in [14, 15,
18] in the case of Gaussian models with unknown covariance and unknown mean
and covariance (the analysis becomes much more involved in the case when the
functional depends on unknown covariance). In [16], the proposed higher order
bias reduction method was studied in the case of general models with a high-
dimensional parameter $\theta$ for which there exists an estimator
$\hat{\theta}$ admitting high-dimensional normal approximation.

######  Remark 2.4.

If ${\mathbb{E}}\xi=0,$ one can also use $\bar{X}=\frac{X_{1}+\dots+X_{n}}{n}$
as an estimator of $\theta$ and construct the corresponding functions
$\bar{f}_{k}$ based on this estimator. In this case, a bound similar to (2.7)
holds for estimator $\bar{f}_{k}(\bar{X}),$ so, it is also minimax optimal.
This follows from Theorem 2, [19] along with the bound on Poincaré constant
$c(V)$ (see Remark 2.1). Normal approximation of estimator
$\bar{f}_{k}(\bar{X})$ similar to Corollary 2.2 also holds (see [19]).
However, the limit variance of estimator $\bar{f}_{k}(\bar{X})$ is not equal
to $\sigma_{f}^{2}(\theta),$ it is rather equal to
$\langle\Sigma_{\xi}f^{\prime}(\theta),f^{\prime}(\theta)\rangle.$ Since $X$
is an unbiased estimator of $\theta$ (note that ${\mathbb{E}}\xi=0$), it
follows from the Cramér-Rao bound that
$\Sigma_{X}=\Sigma_{\xi}\succeq{\mathcal{I}}^{-1}.$ This fact implies that the
limit variance of estimator $\bar{f}_{k}(\bar{X})$ is suboptimal:

$\displaystyle\langle\Sigma_{\xi}f^{\prime}(\theta),f^{\prime}(\theta)\rangle\geq\langle{\mathcal{I}}^{-1}f^{\prime}(\theta),f^{\prime}(\theta)\rangle$

and this estimator is not asymptotically efficient. This was the main
motivation for the development of estimators $f_{k}(\hat{\theta})$ based on
the MLE in the current paper. We conjecture that asymptotic efficiency also
holds when the MLE is replaced by Pitman’s estimator. Since MLE is defined
implicitly as a solution of an optimization problem, there is an additional
layer of difficulties in the analysis of the problem comparing with the case
of $\bar{X}.$ Similar problems in the case of log-concave location-scale
families seem to be much more challenging.

######  Remark 2.5.

The proof of Theorem 2.2 could be easily modified and, in fact, significantly
simplified to obtain the following result under somewhat different assumptions
than Assumption 1 (they are stronger in the sense that the eigenvalues of the
Hessian $V^{\prime\prime}(x)$ are assumed to be bounded away from zero
uniformly in $x$).

######  Theorem 2.3.

Suppose $V$ is twice continuously differentiable and, for some $M,m>0,$
$\|V^{\prime\prime}\|_{L_{\infty}}\leq M$ and $V^{\prime\prime}(x)\succeq
mI_{d},x\in{\mathbb{R}}^{d}.$ Let $f\in C^{s}$ for some $s=k+1+\rho,$ $k\geq
0,$ $\rho\in(0,1].$ Then, for all $d\lesssim n,$

$\displaystyle\sup_{\theta\in{\mathbb{R}}^{d}}\Bigl{\|}f_{k}(\hat{\theta})-f(\theta)-n^{-1}\sum_{j=1}^{n}\langle
V^{\prime}(\xi_{j}),{\mathcal{I}}^{-1}f^{\prime}(\theta)\rangle\Bigr{\|}_{L_{\psi_{1}}({\mathbb{P}}_{\theta})}$
$\displaystyle\lesssim_{M,m,s}\|f\|_{C^{s}}\Bigl{[}\frac{1}{\sqrt{n}}\Bigl{(}\frac{d}{n}\Bigr{)}^{\rho/2}+\Bigl{(}\sqrt{\frac{d}{n}}\Bigr{)}^{s}\Bigr{]}.$

This result implies that, under the conditions of Theorem 2.3, the second
claim of Proposition 2.1 holds without any assumptions on $d$ and on
$\Sigma_{\xi}$ and that the bound of Corollary 2.3 holds for all convex non-
decreasing functions $\psi:{\mathbb{R}}_{+}\mapsto{\mathbb{R}}_{+}$ with
$\psi(0)=0,$ satisfying the condition $\psi(u)\leq\psi_{1}(cu),u\geq 0$ for
some constant $c>0.$

##  3 Error bounds for the MLE

Our main goal in this section is to obtain upper bounds on the error
$\|\hat{\theta}-\theta\|$ of MLE $\hat{\theta}.$ Namely, the following result
will be proved.

######  Theorem 3.1.

Suppose Assumption 1 holds and let $t\geq 1.$ If $d\vee t\leq\gamma n$ for

$\displaystyle\gamma:=c\Bigl{(}\frac{m}{M}\wedge\frac{m^{2}}{L\sqrt{M}}\Bigr{)}^{2}$
(3.1)

with a small enough constant $c>0$, then, with probability at least
$1-e^{-t}$,

$\displaystyle\|\hat{\theta}-\theta\|\lesssim\frac{\sqrt{M}}{m}\Bigl{(}\sqrt{\frac{d}{n}}\vee\sqrt{\frac{t}{n}}\Bigr{)}.$

Several simple facts will be used in the proof.

For a differentiable function $g:{\mathbb{R}}^{d}\mapsto{\mathbb{R}},$ define
the remainder of its first order Taylor expansion

$\displaystyle S_{g}(x;h):=g(x+h)-g(x)-\langle
g^{\prime}(x),h\rangle,x,h\in{\mathbb{R}}^{d}.$

The next proposition is straightforward.

######  Proposition 3.1.

Let $g$ be twice differentiable. Then, for all
$x,y,h,h^{\prime}\in{\mathbb{R}}^{d},$ the following properties hold:

  1. (i)

$|S_{g}(x;h)|\leq\frac{1}{2}\|g^{\prime\prime}\|_{L_{\infty}}\|h\|^{2}.$

  2. (ii)

$|S_{g}(x;h)-\frac{1}{2}\langle
g^{\prime\prime}(x)h,h\rangle|\leq\frac{1}{6}\|g^{\prime\prime}\|_{\rm
Lip}\|h\|^{3}.$

  3. (iii)

$|S_{g}(x;h)-S_{g}(x;h^{\prime})|\leq\frac{1}{2}\|g^{\prime\prime}\|_{L_{\infty}}\|h-h^{\prime}\|^{2}+\|g^{\prime\prime}\|_{L_{\infty}}\|h\|\|h-h^{\prime}\|.$

  4. (iv)

$|S_{g}(x;h)-S_{g}(y;h)|\leq\frac{1}{4}\|g^{\prime\prime}\|_{\rm
Lip}\|h\|^{2}\|x-y\|.$

If $g\in C^{s}$ for $s=1+\rho,$ $\rho\in(0,1],$ then

  1. (v)

$|S_{g}(x;h)-S_{g}(x;h^{\prime})|\lesssim\|g\|_{C^{s}}(\|h\|^{\rho}\vee\|h^{\prime}\|^{\rho})\|h-h^{\prime}\|.$

Let $\xi_{1},\dots,\xi_{n}$ be i.i.d. copies of $\xi$ (that is,
$\xi_{j}:=X_{j}-\theta$). Define the following convex functions:

$\displaystyle g(h):={\mathbb{E}}V(\xi+h),$ $\displaystyle
g_{n}(h):=n^{-1}\sum_{j=1}^{n}V(\xi_{j}+h),h\in{\mathbb{R}}^{d}.$ (3.2)

Note that ${\mathbb{E}}g_{n}(h)=g(h)$ and $g^{\prime\prime}(0)={\mathcal{I}}.$

We will need simple probabilistic bounds for r.v.
$g_{n}^{\prime}(0)=n^{-1}\sum_{j=1}^{n}V^{\prime}(\xi_{j})$ and
$g_{n}^{\prime\prime}(0)=n^{-1}\sum_{j=1}^{n}V^{\prime\prime}(\xi_{j}).$ We
start with the following lemma.

######  Lemma 3.1.

For all $u\in{\mathbb{R}}^{d},$ $\langle V^{\prime}(\xi),u\rangle$ is a
subgaussian r.v. with

$\displaystyle\|\langle
V^{\prime}(\xi),u\rangle\|_{\psi_{2}}\lesssim\sqrt{M}\|u\|.$

proof. For all $k\geq 1,$ we have

$\displaystyle{\mathbb{E}}\langle
V^{\prime}(\xi),u\rangle^{2k}=\int_{{\mathbb{R}}^{d}}\langle
V^{\prime}(x),u\rangle^{2k}e^{-V(x)}dx.$

By Lemma 2.2.1 in [7], there are constants $A,B>0$ such that $e^{-V(x)}\leq
Ae^{-B\|x\|}$ for all $x\in\mathbb{R}^{d}$. Moreover, by Assumption 1,
$V^{\prime}$ is $M$-Lipschitz which implies that
$\|V^{\prime}(x)\|\leq\|V^{\prime}(0)\|+M\|x\|$ for all $x\in\mathbb{R}^{d}$.
Combining these two facts, the above integral is finite for all $k\geq 1$ and
we obtain that

$\displaystyle\int_{{\mathbb{R}}^{d}}\langle
V^{\prime}(x),u\rangle^{2k}e^{-V(x)}dx$
$\displaystyle=\int_{{\mathbb{R}}^{d}}\langle
V^{\prime}(x),u\rangle^{2k-1}\langle V^{\prime}(x),u\rangle e^{-V(x)}dx$
$\displaystyle=\int_{{\mathbb{R}}^{d}}(2k-1)\langle
V^{\prime}(x),u\rangle^{2k-2}\langle V^{\prime\prime}(x)u,u\rangle
e^{-V(x)}dx,$

where we used integration by parts in the last equality. Therefore,

$\displaystyle{\mathbb{E}}\langle V^{\prime}(\xi),u\rangle^{2k}$
$\displaystyle\leq(2k-1)M\|u\|^{2}\int_{{\mathbb{R}}^{d}}\langle
V^{\prime}(x),u\rangle^{2k-2}e^{-V(x)}dx$
$\displaystyle=(2k-1)M\|u\|^{2}{\mathbb{E}}\langle
V^{\prime}(\xi),u\rangle^{2(k-1)}.$

It follows by induction that

$\displaystyle{\mathbb{E}}\langle
V^{\prime}(\xi),u\rangle^{2k}\leq(2k-1)!!M^{k}\|u\|^{2k}.$

It is easy to conclude that, for all $p\geq 1,$

$\displaystyle\|\langle
V^{\prime}(\xi),u\rangle\|_{L_{p}}\lesssim\sqrt{p}\sqrt{M}\|u\|,$

implying the claim.

∎

An immediate consequence is the following corollary.

######  Corollary 3.1.

For all $t\geq 1,$ with probability at least $1-e^{-t}$

$\displaystyle\|g_{n}^{\prime}(0)\|\lesssim\sqrt{M}\Bigl{(}\sqrt{\frac{d}{n}}\vee\sqrt{\frac{t}{n}}\Bigr{)}.$

proof. Let $S^{d-1}$ be the unit sphere in $\mathbb{R}^{d}$ and let $A\subset
S^{d-1}$ be a $1/2$-net with ${\rm card}(A)\leq 5^{d}.$ Then

$\displaystyle\|g_{n}^{\prime}(0)\|=\sup_{u\in S^{d-1}}\langle
g_{n}^{\prime}(0),u\rangle\leq 2\max_{u\in A}\langle
g_{n}^{\prime}(0),u\rangle.$

By Lemma 3.1, we have for all $u\in A,$

$\displaystyle\|\langle
g_{n}^{\prime}(0),u\rangle\|_{\psi_{2}}=\Bigl{\|}n^{-1}\sum_{j=1}^{n}\langle
V^{\prime}(\xi_{j}),u\rangle\Bigr{\|}_{\psi_{2}}\lesssim\frac{\sqrt{M}}{\sqrt{n}},$

implying that with probability at least $1-e^{-t}$

$\displaystyle|\langle
g_{n}^{\prime}(0),u\rangle|\lesssim\sqrt{M}\sqrt{\frac{t}{n}}.$

It remains to use the union bound and to replace $t$ by $t+d\log(5)$.

∎

######  Proposition 3.2.

For all $t\geq 1,$ with probability at least $1-e^{-t}$

$\displaystyle\|g_{n}^{\prime\prime}(0)-{\mathcal{I}}\|\lesssim
M\Bigl{(}\sqrt{\frac{d}{n}}\vee\sqrt{\frac{t}{n}}\Bigr{)}.$

Moreover,

$\displaystyle\Bigl{\|}\|g_{n}^{\prime\prime}(0)-{\mathcal{I}}\|\Bigr{\|}_{\psi_{2}}\lesssim
M\sqrt{\frac{d}{n}}.$

proof. Similarly to the proof of Corollary 3.1, one can use the fact that
$\|\langle V^{\prime\prime}(\xi)u,v\rangle\|_{\psi_{2}}\lesssim\|\langle
V^{\prime\prime}(\xi)u,v\rangle\|_{L_{\infty}}\lesssim M,u,v\in S^{d-1}$ and
discretization of the unit sphere to prove the first bound.

Moreover, using that $t,d\geq 1$, the first bound implies that

$\displaystyle{\mathbb{P}}\biggl{\\{}\|g_{n}^{\prime\prime}(0)-{\mathcal{I}}\|\geq
C_{1}M\sqrt{\frac{d}{n}}\sqrt{t}\biggr{\\}}\leq e^{-t},t\geq 1,$

which is equivalent to the second bound.

∎

We now turn to the proof of Theorem 3.1.

proof. Note that the minimum of convex function $g$ from (3.2) is attained at
$0$ and also

$\displaystyle\hat{h}:=\operatorname{argmin}_{h\in{\mathbb{R}}^{d}}\limits
g_{n}(h)=\theta-\hat{\theta},$ (3.3)

so, to prove Theorem 3.1, it will be enough to bound $\|\hat{h}\|.$ We will
use the following elementary lemma.

######  Lemma 3.2.

Let $q:{\mathbb{R}}^{d}\mapsto{\mathbb{R}}$ be a convex function attaining its
minimum at $\bar{x}\in{\mathbb{R}}^{d}.$ For all $x_{0}\in{\mathbb{R}}^{d}$
and $\delta>0,$ the condition $\|\bar{x}-x_{0}\|\geq\delta$ implies that
$\inf_{\|x-x_{0}\|=\delta}q(x)-q(x_{0})\leq 0.$

proof. Indeed, assume that $\|\bar{x}-x_{0}\|\geq\delta.$ Clearly,
$q(\bar{x})-q(x_{0})\leq 0.$ Let $x^{\ast}=\lambda\bar{x}+(1-\lambda)x_{0}$
with $\lambda:=\frac{\delta}{\|\bar{x}-x_{0}\|}.$ Then,
$\|x^{\ast}-x_{0}\|=\delta$ and, by convexity of $q,$ $q(x^{\ast})\leq\lambda
q(\bar{x})+(1-\lambda)q(x_{0}),$ implying that
$q(x^{\ast})-q(x_{0})\leq\lambda(q(\bar{x})-q(x_{0}))\leq 0.$

∎

If $\|\hat{h}\|\geq\delta,$ then, by Lemma 3.2,

$\displaystyle\inf_{\|h\|=\delta}g_{n}(h)-g_{n}(0)\leq 0.$ (3.4)

Note that

$\displaystyle g_{n}(h)-g_{n}(0)$ $\displaystyle=\langle
g_{n}^{\prime}(0),h\rangle+S_{g_{n}}(0;h)$ $\displaystyle=\langle
g_{n}^{\prime}(0),h\rangle+S_{g_{n}}(0;h)-\frac{1}{2}\langle
g_{n}^{\prime\prime}(0)h,h\rangle+\frac{1}{2}\langle
g_{n}^{\prime\prime}(0)h,h\rangle.$ (3.5)

For $\|h\|=\delta,$ we have, by Assumption 1, (ii),

$\displaystyle\langle g_{n}^{\prime\prime}(0)h,h\rangle=\langle
g_{n}^{\prime\prime}(0)h,h\rangle-\langle{\mathcal{I}}h,h\rangle+\langle{\mathcal{I}}h,h\rangle\geq
m\delta^{2}-\delta^{2}\|g_{n}^{\prime\prime}(0)-\mathcal{I}\|$

and, by Proposition 3.1, (ii),

$\displaystyle S_{g_{n}}(0;h)-\frac{1}{2}\langle
g_{n}^{\prime\prime}(0)h,h\rangle\geq-\frac{L}{2}\delta^{3}.$

Inserting these inequalities into (3.5) and using (3.4) we can conclude that
if $\|\hat{h}\|\geq\delta$, then

$\displaystyle\|g_{n}^{\prime}(0)\|\delta+\frac{\delta^{2}}{2}\|g_{n}^{\prime\prime}(0)-\mathcal{I}\|\geq\frac{m}{2}\delta^{2}-\frac{L}{2}\delta^{3}.$
(3.6)

To complete the proof, assume that the bound of Corollary 3.1 holds with
constant $C_{1}\geq 1$ and the bound of Proposition 3.2 holds with constant
$C_{2}\geq 1.$ If constant $c$ in the definition of $\gamma$ is small enough,
then the condition $d\vee t\leq\gamma n$ implies that

$\displaystyle
C\Bigl{(}\sqrt{\frac{d}{n}}\vee\sqrt{\frac{t}{n}}\Bigr{)}\leq\frac{m}{M}\wedge\frac{m^{2}}{L\sqrt{M}}$

with $C:=(16C_{1})\vee(4C_{2}).$ Moreover, let

$\displaystyle\delta:=4C_{1}\frac{\sqrt{M}}{m}\Bigl{(}\sqrt{\frac{d}{n}}\vee\sqrt{\frac{t}{n}}\Bigr{)}$

Then, $\delta\leq\frac{m}{4L}$ and, on the event

$\displaystyle E:=\Bigl{\\{}\|g_{n}^{\prime\prime}(0)-\mathcal{I}\|\leq
C_{2}M\Bigl{(}\sqrt{\frac{d}{n}}\vee\sqrt{\frac{t}{n}}\Bigr{)}\Bigr{\\}},$

bound (3.6) implies that $\delta\leq\frac{4}{m}\|g_{n}^{\prime}(0)\|.$ Note
also that, by Proposition 3.2, ${\mathbb{P}}(E^{c})\leq e^{-t}$. By Corollary
3.1, the event $\\{\delta\leq\frac{4}{m}\|g_{n}^{\prime}(0)\|\\}$ occurs with
probability at most $e^{-t}.$ Recall that bound (3.6) follows from
$\|\hat{\theta}-\theta\|=\|\hat{h}\|\geq\delta.$ Thus, with probability at
least $1-2e^{-t},$ $\|\hat{\theta}-\theta\|\leq\delta.$ It remains to adjust
the constants in order to replace the probability bound $1-2e^{-t}$ with
$1-e^{-t}.$

∎

The following fact will be also useful.

######  Corollary 3.2.

Suppose Assumption 1 holds and that $d\leq\gamma n,$ where
$\gamma=c\big{(}\frac{m}{M}\wedge\frac{m^{2}}{L\sqrt{M}}\big{)}^{2}$ with a
small enough constant $c>0.$ Then

$\displaystyle\Bigl{\|}\|\hat{\theta}-\theta\|\wedge\frac{m}{12L}\Bigr{\|}_{\psi_{2}}\lesssim\frac{\sqrt{M}}{m}\sqrt{\frac{d}{n}}+\frac{m}{L\sqrt{\gamma}}\frac{1}{\sqrt{n}}.$

proof. First, for $d\leq\gamma n$, Theorem 3.1 can be formulated as

$\displaystyle{\mathbb{P}}\biggl{\\{}\|\hat{\theta}-\theta\|>C_{1}\Bigl{(}\frac{\sqrt{M}}{m}\sqrt{\frac{d}{n}}\vee\frac{\sqrt{M}}{m}\sqrt{\frac{t}{n}}\Bigr{)}\biggr{\\}}\leq
e^{-t},t\in[1,\gamma n].$

This implies that

$\displaystyle{\mathbb{P}}\biggl{\\{}\|\hat{\theta}-\theta\|>\Bigr{(}C_{1}\frac{\sqrt{M}}{m}\sqrt{\frac{d}{n}}\vee\frac{m}{12L\sqrt{\gamma}}\frac{1}{\sqrt{n}}\Bigl{)}\sqrt{t}\biggr{\\}}\leq
e^{-t},t\in[1,\gamma n],$

using that $t\geq 1$ and
$\sqrt{\gamma}\leq\frac{1}{12C_{1}}\frac{m^{2}}{L\sqrt{M}}$ for $c$
sufficiently small. It follows that

$\displaystyle{\mathbb{P}}\biggl{\\{}\|\hat{\theta}-\theta\|\wedge\frac{m}{12L}>\Bigr{(}C_{1}\frac{\sqrt{M}}{m}\sqrt{\frac{d}{n}}\vee\frac{m}{12L\sqrt{\gamma}}\frac{1}{\sqrt{n}}\Bigl{)}\sqrt{t}\biggr{\\}}\leq
e^{-t},t\geq 1,$

which is equivalent to the claim.

∎

##  4 Concentration bounds

In this section, we prove concentration inequalities for $f(\hat{\theta}),$
where $f$ is a smooth function on ${\mathbb{R}}^{d}.$ Namely, we will prove
the following result.

######  Theorem 4.1.

Let $f\in C^{s}$ for some $s=1+\rho,$ $\rho\in(0,1].$ Suppose that
$d\leq\gamma n,$ where
$\gamma:=c\bigl{(}\frac{m}{M}\wedge\frac{m^{2}}{L\sqrt{M}}\bigr{)}^{2}$ with a
small enough $c>0.$ Then

$\displaystyle\sup_{\theta\in{\mathbb{R}}^{d}}\Bigl{\|}f(\hat{\theta})-{\mathbb{E}}_{\theta}f(\hat{\theta})-n^{-1}\sum_{j=1}^{n}\langle
V^{\prime}(\xi_{j}),{\mathcal{I}}^{-1}f^{\prime}(\theta)\rangle\Bigr{\|}_{L_{\psi_{2/3}}({\mathbb{P}}_{\theta})}$
$\displaystyle\lesssim_{M,L,m}\sqrt{c(V)}\|f\|_{C^{s}}\frac{1}{\sqrt{n}}\Bigl{(}\frac{d}{n}\Bigr{)}^{\rho/2}.$

To derive concentration bounds for $f(\hat{\theta}),$ we need to bound local
Lipschitz constants of estimator $\hat{\theta}(X_{1},\dots,X_{n})$ as a
function of its variables. A good place to start is to show the continuity of
this function. The following fact is, probably, well known. We give its proof
for completeness.

######  Proposition 4.1.

Suppose that $V$ is strictly convex. Then, MLE
$\hat{\theta}(x_{1},\dots,x_{n})$ exists and is unique for all
$(x_{1},\dots,x_{n})\in{\mathbb{R}}^{d}\times\dots\times{\mathbb{R}}^{d}$ and
the function

${\mathbb{R}}^{d}\times\dots\times{\mathbb{R}}^{d}\ni(x_{1},\dots,x_{n})\mapsto\hat{\theta}(x_{1},\dots,x_{n})\in{\mathbb{R}}^{d}$

is continuous.

proof. Let
$(x_{1},\dots,x_{n})\in{\mathbb{R}}^{d}\times\dots\times{\mathbb{R}}^{d},(x_{1}^{(k)},\dots,x_{n}^{(k)})\in{\mathbb{R}}^{d}\times\dots\times{\mathbb{R}}^{d},k\geq
1$ and $(x_{1}^{(k)},\dots,x_{n}^{(k)})\to(x_{1},\dots,x_{n})$ as
$k\to\infty.$ Define

$\displaystyle p(\theta):=n^{-1}\sum_{j=1}^{n}V(x_{j}-\theta),\
p_{k}(\theta):=n^{-1}\sum_{j=1}^{n}V(x_{j}^{(k)}-\theta),\theta\in{\mathbb{R}}^{d},k\geq
1.$

By continuity of $V,$ $p_{k}(\theta)\to p(\theta)$ as $k\to\infty$ for all
$\theta\in{\mathbb{R}}^{d}.$ Since $p_{k}$ and $p$ are convex, this implies
the uniform convergence on all compact subsets of ${\mathbb{R}}^{d}.$

If
$\|\hat{\theta}(x_{1}^{(k)},\dots,x_{n}^{(k)})-\hat{\theta}(x_{1},\dots,x_{n})\|\geq\delta,$
then, by Lemma 3.2,

$\displaystyle\inf_{\|\theta-\hat{\theta}(x_{1},\dots,x_{n})\|=\delta}p_{k}(\theta)-p_{k}(\hat{\theta}(x_{1},\dots,x_{n}))\leq
0$

By the uniform convergence of $p_{k}$ to $p$ on compact sets,

$\displaystyle\inf_{\|\theta-\hat{\theta}(x_{1},\dots,x_{n})\|=\delta}p_{k}(\theta)-p_{k}(\hat{\theta}(x_{1},\dots,x_{n}))\to\inf_{\|\theta-\hat{\theta}(x_{1},\dots,x_{n})\|=\delta}p(\theta)-p(\hat{\theta}(x_{1},\dots,x_{n}))$

as $k\to\infty.$ Due to strict convexity, the minimum
$\hat{\theta}(x_{1},\dots,x_{n})$ of $p(\theta)$ exists and is unique (see the
argument in the introduction), and

$\displaystyle\inf_{\|\theta-\hat{\theta}(x_{1},\dots,x_{n})\|=\delta}p(\theta)-p(\hat{\theta}(x_{1},\dots,x_{n}))>0,$

implying that
$\|\hat{\theta}(x_{1}^{(k)},\dots,x_{n}^{(k)})-\hat{\theta}(x_{1},\dots,x_{n})\|<\delta$
for all large enough $k$ and thus
$\hat{\theta}(x_{1}^{(k)},\dots,x_{n}^{(k)})\to\hat{\theta}(x_{1},\dots,x_{n})$
as $k\to\infty.$

∎

Note that the continuity of $\hat{\theta}$ also follows from the implicit
function theorem in the case when $V$ is twice differentiable with
$V^{\prime\prime}$ being positively definite throughout $\mathbb{R}^{d}.$

We will now study Lipschitz continuity properties of $\hat{\theta}$ as a
function of the data $X_{1},\dots,X_{n}$ needed to prove concentration
inequalities.

######  Proposition 4.2.

Let

$\displaystyle
A_{1}:=\Bigl{\\{}(x_{1},\dots,x_{n})\in{\mathbb{R}}^{d}\times\dots\times{\mathbb{R}}^{d}:\|\hat{\theta}(x_{1},\dots,x_{n})-\theta\|\leq\frac{m}{12L}\Bigr{\\}}$

and

$\displaystyle
A_{2}:=\Bigl{\\{}(x_{1},\dots,x_{n})\in{\mathbb{R}}^{d}\times\dots\times{\mathbb{R}}^{d}:\Bigl{\|}n^{-1}\sum_{j=1}^{n}V^{\prime\prime}(x_{j}-\theta)-\mathcal{I}\Bigr{\|}\leq\frac{m}{4}\Bigr{\\}}$

and let $A:=A_{1}\cap A_{2}.$ Then the function
$A\ni(x_{1},\dots,x_{n})\mapsto\hat{\theta}(x_{1},\dots,x_{n})$ is Lipschitz
with constant $\frac{4M}{m\sqrt{n}}:$ for all
$(x_{1},\dots,x_{n}),(\tilde{x}_{1},\dots,\tilde{x}_{n})\in A,$

$\displaystyle\|\hat{\theta}(x_{1},\dots,x_{n})-\hat{\theta}(\tilde{x}_{1},\dots,\tilde{x}_{n})\|\leq\frac{4M}{m\sqrt{n}}\Bigl{(}\sum_{j=1}^{n}\|x_{j}-\tilde{x}_{j}\|^{2}\Bigr{)}^{1/2}.$

proof. Due to equivariance, we have

$\displaystyle\hat{\theta}(x_{1},\dots,x_{n})-\hat{\theta}(\tilde{x}_{1},\dots,\tilde{x}_{n})=\hat{\theta}(\xi_{1},\dots,\xi_{n})-\hat{\theta}(\tilde{\xi}_{1},\dots,\tilde{\xi}_{n})$

with $\xi_{j}=x_{j}-\theta$ and $\tilde{\xi}_{j}=\tilde{x}_{j}-\theta.$ Hence,
if we abbreviate $\hat{h}=\theta-\hat{\theta}(\xi_{1},\dots,\xi_{n})$ and
$\tilde{h}=\theta-\hat{\theta}(\tilde{\xi}_{1},\dots,\tilde{\xi}_{n})$, then
we have $g_{n}^{\prime}(\hat{h})=0$ with $g_{n}$ from (3.2) and
$\tilde{g}_{n}^{\prime}(\tilde{h})=0$ with

$\displaystyle\tilde{g}_{n}(h):=n^{-1}\sum_{j=1}^{n}V(\tilde{\xi}_{j}+h),h\in{\mathbb{R}}^{d}.$

Recall that $g^{\prime}(0)=0$ and $g^{\prime\prime}(0)={\mathcal{I}}$. By the
first order Taylor expansion for function $g^{\prime},$
$g^{\prime}(h)={\mathcal{I}}h+r(h),$ where

$\displaystyle r(h):=\int_{0}^{1}(g^{\prime\prime}(\lambda
h)-g^{\prime\prime}(0))d\lambda\ h$

is the remainder. Therefore,

$\displaystyle g^{\prime}(\hat{h})={\mathcal{I}}\hat{h}+r(\hat{h}),$

implying that

$\displaystyle\hat{h}$
$\displaystyle={\mathcal{I}}^{-1}(g^{\prime}(\hat{h})-g_{n}^{\prime}(\hat{h}))-{\mathcal{I}}^{-1}r(\hat{h})$
$\displaystyle={\mathcal{I}}^{-1}(g^{\prime}(0)-g_{n}^{\prime}(0))+{\mathcal{I}}^{-1}q_{n}(\hat{h})-{\mathcal{I}}^{-1}r(\hat{h})$
$\displaystyle=-{\mathcal{I}}^{-1}g_{n}^{\prime}(0)+{\mathcal{I}}^{-1}q_{n}(\hat{h})-{\mathcal{I}}^{-1}r(\hat{h}),$
(4.1)

where

$\displaystyle
q_{n}(h):=(g_{n}-g)^{\prime}(h)-(g_{n}-g)^{\prime}(0)=\int_{0}^{1}(g_{n}^{\prime\prime}-g^{\prime\prime})(\lambda
h)d\lambda\ h.$

Similarly, we have
$\tilde{h}=-{\mathcal{I}}^{-1}\tilde{g}_{n}^{\prime}(0)+{\mathcal{I}}^{-1}\tilde{q}_{n}(\tilde{h})-{\mathcal{I}}^{-1}r(\tilde{h})$
with
$\tilde{q}_{n}(h):=(\tilde{g}_{n}-g)^{\prime}(h)-(\tilde{g}_{n}-g)^{\prime}(0).$
Using these representations we now bound the difference between $\hat{h}$ and
$\tilde{h}.$

First note that

$\displaystyle\|g_{n}^{\prime}(0)-\tilde{g}_{n}^{\prime}(0)\|\leq
n^{-1}\sum_{j=1}^{n}\|V^{\prime}(\xi_{j})-V^{\prime}(\tilde{\xi}_{j})\|$
$\displaystyle\leq
Mn^{-1}\sum_{j=1}^{n}\|\xi_{j}-\tilde{\xi}_{j}\|\leq\frac{M}{\sqrt{n}}\Bigl{(}\sum_{j=1}^{n}\|x_{j}-\tilde{x}_{j}\|^{2}\Bigr{)}^{1/2}.$
(4.2)

Also,

$\displaystyle q_{n}(\hat{h})-\tilde{q}_{n}(\tilde{h})$
$\displaystyle=((g_{n}^{\prime\prime}-g^{\prime\prime})(0))(\hat{h}-\tilde{h})$
$\displaystyle+\int_{0}^{1}((g_{n}^{\prime\prime}-g^{\prime\prime})(\lambda\hat{h})-(g_{n}^{\prime\prime}-g^{\prime\prime})(0))d\lambda\
(\hat{h}-\tilde{h})$
$\displaystyle+\int_{0}^{1}[(g_{n}^{\prime\prime}-g^{\prime\prime})(\lambda\hat{h})-(g_{n}^{\prime\prime}-g^{\prime\prime})(\lambda\tilde{h})]d\lambda\
\tilde{h}$
$\displaystyle+\int_{0}^{1}(g_{n}^{\prime\prime}-\tilde{g}_{n}^{\prime\prime})(\lambda\tilde{h})d\lambda\
\tilde{h}.$

Since, by Assumption 1, $V^{\prime\prime}$ is Lipschitz with constant $L,$ the
function $h\mapsto g^{\prime\prime}(h)={\mathbb{E}}V^{\prime\prime}(\xi+h)$
satisfies the Lipschitz condition with the same constant $L$ and the function
$h\mapsto(g_{n}^{\prime\prime}-g^{\prime\prime})(h)$ is Lipschitz with
constant at most $2L.$ In addition,

$\displaystyle\|(g_{n}^{\prime\prime}-\tilde{g}_{n}^{\prime\prime})(\lambda\tilde{h})\|$
$\displaystyle\leq
n^{-1}\sum_{j=1}^{n}\|V^{\prime\prime}(\xi_{j}+\lambda\tilde{h})-V^{\prime\prime}(\tilde{\xi}_{j}+\lambda\tilde{h})\|$
$\displaystyle\leq\frac{L}{n}\sum_{j=1}^{n}\|\xi_{j}-\tilde{\xi}_{j}\|\leq\frac{L}{\sqrt{n}}\biggl{(}\sum_{j=1}^{n}\|x_{j}-\tilde{x}_{j}\|^{2}\biggr{)}^{1/2}.$

Therefore, we easily get

$\displaystyle\|q_{n}(\hat{h})-\tilde{q}_{n}(\tilde{h})\|$
$\displaystyle\leq\|g_{n}^{\prime\prime}(0)-g^{\prime\prime}(0)\|\|\hat{h}-\tilde{h}\|+L(\|\hat{h}\|+\|\tilde{h}\|)\|\hat{h}-\tilde{h}\|$
$\displaystyle+\frac{L}{\sqrt{n}}\|\tilde{h}\|\biggl{(}\sum_{j=1}^{n}\|x_{j}-\tilde{x}_{j}\|^{2}\biggr{)}^{1/2}.$
(4.3)

Similarly, note that

$\displaystyle
r(\hat{h})-r(\tilde{h})=\int_{0}^{1}(g^{\prime\prime}(\lambda\hat{h})-g^{\prime\prime}(\lambda\tilde{h}))d\lambda\
\hat{h}+\int_{0}^{1}(g^{\prime\prime}(\lambda\tilde{h})-g^{\prime\prime}(0))d\lambda\
(\hat{h}-\tilde{h})$

which implies the following bound:

$\displaystyle\|r(\hat{h})-r(\tilde{h})\|\leq\frac{L}{2}(\|\hat{h}\|+\|\tilde{h}\|)\|\hat{h}-\tilde{h}\|.$
(4.4)

It follows from (4), (4) and (4.4) that

$\displaystyle\|\hat{h}-\tilde{h}\|$
$\displaystyle\leq\frac{1}{m}\biggl{(}\Bigl{(}\frac{M}{\sqrt{n}}+\frac{L}{\sqrt{n}}\|\tilde{h}\|\Bigr{)}\Bigl{(}\sum_{j=1}^{n}\|x_{j}-\tilde{x}_{j}\|^{2}\Bigr{)}^{1/2}$
$\displaystyle+\|g_{n}^{\prime\prime}(0)-g^{\prime\prime}(0)\|\|\hat{h}-\tilde{h}\|+\frac{3}{2}L(\|\hat{h}\|+\|\tilde{h}\|)\|\hat{h}-\tilde{h}\|\biggr{)}.$

If $\|g_{n}^{\prime\prime}(0)-g^{\prime\prime}(0)\|\leq\frac{m}{4}$ and
$\|\hat{h}\|\vee\|\tilde{h}\|\leq\frac{m}{12L},$ we easily conclude that

$\displaystyle\|\hat{h}-\tilde{h}\|$
$\displaystyle\leq\frac{4M}{m\sqrt{n}}\Bigl{(}\sum_{j=1}^{n}\|x_{j}-\tilde{x}_{j}\|^{2}\Bigr{)}^{1/2},$

which completes the proof.

∎

Since the Lipschitz condition holds for $\hat{\theta}$ only on set $A,$ it
will be convenient for our purposes to replace $\hat{\theta}$ with its
“smoothed truncated” version $\check{\theta}$ that is Lipschitz in the whole
space. To this end, let $\phi:{\mathbb{R}}\mapsto[0,1]$ be defined as follows:
$\phi(s)=1,s\leq 1,$ $\phi(s)=0,s\geq 2$ and $\phi(s)=2-s,s\in(1,2).$ Clearly,
$\phi$ is Lipschitz with constant $1.$ By Theorem 3.1,
$\|\hat{\theta}-\theta\|\leq\frac{m}{24L}$ with probability at least
$1-e^{-\gamma n},$ where
$\gamma:=c\bigl{(}\frac{m}{M}\wedge\frac{m^{2}}{L\sqrt{M}}\bigr{)}^{2}$ with a
small enough constant $c>0$ and it is assumed that $d\leq\gamma n.$ Similarly,
it follows from Proposition 3.2 that
$\|g_{n}^{\prime\prime}(0)-g^{\prime\prime}(0)\|\leq\frac{m}{8}$ with
probability at least $1-e^{-\beta n},$ where
$\beta=c\bigl{(}\frac{m}{M}\bigr{)}^{2}$ for a small enough constant $c>0$
(and under the assumption that $d\leq\beta n.$ Clearly, we can assume that
$\beta\geq\gamma,$ so, both properties hold with probability at least
$1-2e^{-\gamma n}$ provided that $d\leq\gamma n.$ Define

$\displaystyle\varphi(x_{1},\dots,x_{n}):=\phi\Bigl{(}\frac{24L}{m}\|\hat{\theta}(x_{1},\dots,x_{n})-\theta\|\Bigr{)},$
$\displaystyle\psi(x_{1},\dots,x_{n}):=\phi\Bigl{(}\frac{8}{m}\|g_{n}^{\prime\prime}(0)(x_{1}-\theta,\dots,x_{n}-\theta)-g^{\prime\prime}(0)\|\Bigr{)}.$

and let

$\displaystyle\check{\theta}:=(1-\varphi\psi)\theta+\varphi\psi\hat{\theta}.$

Note that $\check{\theta}-\theta=(\hat{\theta}-\theta)\varphi\psi$ and
$\check{\theta}=\hat{\theta}$ on the event $\\{\varphi=\psi=1\\}$ of
probability at least $1-2e^{-\gamma n}.$

######  Proposition 4.3.

If $d\leq\gamma n,$ then

$\displaystyle\|{\mathbb{E}}_{\theta}\check{\theta}-\theta\|\lesssim_{M,L,m}\frac{d}{n}\qquad\text{and}\qquad{\mathbb{E}}_{\theta}^{1/2}\|\check{\theta}-\theta\|^{2}\lesssim_{M,L,m}\sqrt{\frac{d}{n}}.$

proof. By representation (4) and the fact that $g^{\prime}(0)=0$,

$\displaystyle\check{\theta}-\theta$
$\displaystyle=(\hat{\theta}-\theta)\varphi\psi=-\hat{h}\varphi\psi$
$\displaystyle={\mathcal{I}}^{-1}g_{n}^{\prime}(0)-{\mathcal{I}}^{-1}g_{n}^{\prime}(0)(1-\varphi\psi)-{\mathcal{I}}^{-1}q_{n}(\hat{h})\varphi\psi+{\mathcal{I}}^{-1}r(\hat{h})\varphi\psi.$
(4.5)

Using Corollary 3.1, we get

$\displaystyle\|\|{\mathcal{I}}^{-1}g_{n}^{\prime}(0)(1-\varphi\psi)\|\|_{L_{2}}$
$\displaystyle\leq\|\|{\mathcal{I}}^{-1}g_{n}^{\prime}(0)\|\|_{L_{4}}\|I(\varphi\psi\neq
1)\|_{L_{4}}$
$\displaystyle\lesssim\frac{1}{m}\|\|g_{n}^{\prime}(0)\|\|_{L_{4}}e^{-\gamma
n/4}\lesssim\frac{\sqrt{M}}{m}\sqrt{\frac{d}{n}}e^{-\gamma n/4}.$

Note also that

$\displaystyle\|q_{n}(\hat{h})\|=\Bigl{\|}\int_{0}^{1}(g_{n}^{\prime\prime}-g^{\prime\prime})(\lambda\hat{h})d\lambda\
\hat{h}\Bigr{\|}\leq\|g_{n}^{\prime\prime}(0)-g^{\prime\prime}(0)\|\|\hat{h}\|+L\|\hat{h}\|^{2},$

such that we also have

$\displaystyle{\mathbb{E}}^{1/2}\|{\mathcal{I}}^{-1}q_{n}(\hat{h})\varphi\psi\|^{2}$
$\displaystyle\leq\frac{1}{m}{\mathbb{E}}^{1/2}\|g_{n}^{\prime\prime}(0)-g^{\prime\prime}(0)\|^{2}\Bigl{(}\|\hat{h}\|\wedge\frac{m}{12L}\Bigr{)}^{2}+\frac{L}{m}{\mathbb{E}}^{1/2}\Bigl{(}\|\hat{h}\|\wedge\frac{m}{12L}\Bigr{)}^{4}$
$\displaystyle\leq\frac{1}{m}{\mathbb{E}}^{1/4}\|g_{n}^{\prime\prime}(0)-g^{\prime\prime}(0)\|^{4}{\mathbb{E}}^{1/4}\Bigl{(}\|\hat{h}\|\wedge\frac{m}{12L}\Bigr{)}^{4}+\frac{L}{m}{\mathbb{E}}^{1/2}\Bigl{(}\|\hat{h}\|\wedge\frac{m}{12L}\Bigr{)}^{4}.$

Using the second bound of Proposition 3.2 and the bound of Corollary 3.2, we
get

$\displaystyle{\mathbb{E}}^{1/2}\|{\mathcal{I}}^{-1}q_{n}(\hat{h})\varphi\psi\|^{2}\lesssim_{M,L,m}\frac{d}{n}.$

Similarly, we can show that

$\displaystyle{\mathbb{E}}^{1/2}\|{\mathcal{I}}^{-1}r(\hat{h})\varphi\psi\|^{2}\lesssim_{M,L,m}\frac{d}{n},$

using the fact that

$\displaystyle\|{\mathcal{I}}^{-1}r(\hat{h})\varphi\psi\|\leq\frac{L}{2m}\Bigl{(}\|\hat{h}\|\wedge\frac{m}{12L}\Bigr{)}^{2}$

and the bound of Corollary 3.2.

The above bounds and representation (4) imply that

$\displaystyle\|{\mathbb{E}}_{\theta}\check{\theta}-\theta\|\lesssim_{M,L,m}\frac{d}{n}.$

Using also Corollary 3.1, we get

$\displaystyle{\mathbb{E}}_{\theta}^{1/2}\|\check{\theta}-\theta\|^{2}\lesssim_{M,L,m}\sqrt{\frac{d}{n}}.$

∎

######  Proposition 4.4.

The function $(x_{1},\dots,x_{n})\mapsto\check{\theta}(x_{1},\dots,x_{n})$ is
Lipschitz with constant $\lesssim\frac{M}{m\sqrt{n}}:$ for all
$(x_{1},\dots,x_{n}),(\tilde{x}_{1},\dots,\tilde{x}_{n})\in{\mathbb{R}}^{d}\times\dots\times{\mathbb{R}}^{d},$

$\displaystyle\|\check{\theta}(x_{1},\dots,x_{n})-\check{\theta}(\tilde{x}_{1},\dots,\tilde{x}_{n})\|\lesssim\frac{M}{m\sqrt{n}}\Bigl{(}\sum_{j=1}^{n}\|x_{j}-\tilde{x}_{j}\|^{2}\Bigr{)}^{1/2}.$

proof. By Proposition 4.2, on the set $A,$ $\hat{\theta}$ is Lipschitz with
constant $\frac{4M}{m\sqrt{n}}.$ This implies that function $\varphi$ is also
Lipschitz on the same set with constant $\frac{24L}{m}\frac{4M}{m\sqrt{n}}.$
Note also that

$\displaystyle\|g_{n}^{\prime\prime}(0)-\tilde{g}_{n}^{\prime\prime}(0)\|\leq\frac{L}{\sqrt{n}}\biggl{(}\sum_{j=1}^{n}\|x_{j}-\tilde{x}_{j}\|^{2}\biggr{)}^{1/2},$

implying that $\psi$ is a Lipschitz function (on the whole space) with
constant $\frac{8}{m}\frac{L}{\sqrt{n}}.$ Using also the fact that $\varphi$
and $\psi$ are both bounded by $1$ and
$\|\hat{\theta}-\theta\|\leq\frac{m}{12L}$ on the set $\\{\varphi\neq 0\\},$
it is easy to conclude that $\check{\theta}$ is Lipschitz on $A$ with constant

$\displaystyle\lesssim\frac{4M}{m\sqrt{n}}+\frac{m}{12L}\frac{24L}{m}\frac{4M}{m\sqrt{n}}+\frac{m}{12L}\frac{8}{m}\frac{L}{\sqrt{n}}\lesssim\frac{M}{m\sqrt{n}}.$

It remains to consider the case when $\varphi(x_{1},\dots,x_{n})\in A$ and
$\varphi(\tilde{x}_{1},\dots,\tilde{x}_{n})\in A^{c}$ (the case when both
points are in $A^{c}$ is trivial). In this case, define
$x_{j}^{\lambda}=\lambda
x_{j}+(1-\lambda)\tilde{x}_{j},\lambda\in[0,1],j=1,\dots,n.$ Note that $A$ is
a closed set (by continuity of both $\hat{\theta}(x_{1},\dots,x_{n})$ and
$n^{-1}\sum_{j=1}^{n}V^{\prime\prime}(x_{j}-\theta)$). If $\bar{\lambda}$
denotes the supremum of those $\lambda$ for which
$(x_{1}^{\lambda},\dots,x_{n}^{\lambda})\in A,$ then
$(x_{1}^{\bar{\lambda}},\dots,x_{n}^{\bar{\lambda}})\in\partial A,$
$(\varphi\psi)(x_{1}^{\bar{\lambda}},\dots,x_{n}^{\bar{\lambda}})=0$ and
$\check{\theta}(x_{1}^{\bar{\lambda}},\dots,x_{n}^{\bar{\lambda}})=0=\check{\theta}(\tilde{x}_{1},\dots,\tilde{x}_{1}).$
Therefore,

$\displaystyle\|\check{\theta}(x_{1},\dots,x_{n})-\check{\theta}(\tilde{x}_{1},\dots,\tilde{x}_{n})\|=\|\check{\theta}(x_{1},\dots,x_{n})-\check{\theta}(x_{1}^{\bar{\lambda}},\dots,x_{n}^{\bar{\lambda}})\|$
$\displaystyle\lesssim\frac{M}{m\sqrt{n}}\biggl{(}\sum_{j=1}^{n}\|x_{j}-x_{j}^{\bar{\lambda}}\|^{2}\biggr{)}^{1/2}\lesssim\frac{M}{m\sqrt{n}}\biggl{(}\sum_{j=1}^{n}\|x_{j}-\tilde{x}_{j}\|^{2}\biggr{)}^{1/2},$

where we use the fact that point
$(x_{1}^{\bar{\lambda}},\dots,x_{n}^{\bar{\lambda}})$ is in the line segment
between $(x_{1},\dots,x_{n})$ and $(\tilde{x}_{1},\dots,\tilde{x}_{n}).$

The Lipschitz condition for $\check{\theta}(x_{1},\dots,x_{n})$ now follows.

∎

We will now consider concentration properties of linear forms
$\langle\check{\theta}-\theta,w\rangle,w\in{\mathbb{R}}^{d}.$ The following
result will be proved.

######  Theorem 4.2.

Suppose $d\leq\gamma n,$ where
$\gamma:=c\bigl{(}\frac{m}{M}\wedge\frac{m^{2}}{L\sqrt{M}}\bigr{)}^{2}$ with a
small enough $c>0$. Then

$\displaystyle\sup_{\theta\in{\mathbb{R}}^{d}}\Bigl{\|}\langle\check{\theta}-\theta,w\rangle-{\mathbb{E}}\langle\check{\theta}-\theta,w\rangle-n^{-1}\sum_{j=1}^{n}\langle
V^{\prime}(\xi_{j}),{\mathcal{I}}^{-1}w\rangle\Bigr{\|}_{L_{\psi_{2/3}}({\mathbb{P}}_{\theta})}$
$\displaystyle\lesssim\sqrt{c(V)}\Bigl{(}\frac{M^{2}}{m^{2}}+\frac{M^{3/2}L}{m^{3}}\Bigr{)}\frac{1}{\sqrt{n}}\sqrt{\frac{d}{n}}\|w\|.$

######  Remark 4.1.

Some concentration bounds for linear forms of MLE could be found in [24].

proof. Using representation (4) and the fact that $g^{\prime}(0)=0$, we get

$\displaystyle\langle\check{\theta}-\theta,w\rangle=\langle\hat{\theta}-\theta,w\rangle\varphi\psi=-\langle\hat{h},w\rangle\varphi\psi$
$\displaystyle=\langle g_{n}^{\prime}(0),u\rangle-\langle
g_{n}^{\prime}(0),u\rangle(1-\varphi\psi)-\langle
q_{n}(\hat{h}),u\rangle\varphi\psi+\langle r(\hat{h}),u\rangle\varphi\psi,$
(4.6)

where $u={\mathcal{I}}^{-1}w.$ Since

$\displaystyle\langle g_{n}^{\prime}(0),u\rangle=n^{-1}\sum_{j=1}^{n}\langle
V^{\prime}(\xi_{j}),{\mathcal{I}}^{-1}w\rangle$

has zero mean, it is enough to study the concentration of three other terms in
the right hand side of (4). The first of these terms is $\langle
g_{n}^{\prime}(0),u\rangle(1-\varphi\psi)$ and we have

$\displaystyle\|\langle
g_{n}^{\prime}(0),u\rangle(1-\varphi\psi)\|_{\psi_{1}}\leq\|\langle
g_{n}^{\prime}(0),u\rangle\|_{\psi_{2}}\|1-\varphi\psi\|_{\psi_{2}}$
$\displaystyle\leq\|\langle
g_{n}^{\prime}(0),u\rangle\|_{\psi_{2}}\|I(\varphi\psi\neq
1)\|_{\psi_{2}}\lesssim\frac{\sqrt{M}}{\sqrt{n}}\frac{1}{\sqrt{\gamma
n}}\|u\|\lesssim\sqrt{\frac{M}{\gamma}}\frac{1}{n}\|u\|,$

where we used Lemma 3.1 and the fact that ${\mathbb{P}}\\{\varphi\psi\neq
1\\}\leq 2e^{-\gamma n}$ with $\gamma$ from the statement of Theorem 4.2.
Clearly, we also have

$\displaystyle\|\langle
g_{n}^{\prime}(0),u\rangle(1-\varphi\psi)-{\mathbb{E}}\langle
g_{n}^{\prime}(0),u\rangle(1-\varphi\psi)\|_{\psi_{1}}\lesssim\sqrt{\frac{M}{\gamma}}\frac{1}{n}\|u\|.$
(4.7)

For two other terms in the right hand side of (4), we will provide bounds on
their local Lipschitz constants. It follows from bound (4) and the bound of
Proposition 4.2 that, for all
$(x_{1},\dots,x_{n}),(\tilde{x}_{1},\dots,\tilde{x}_{n})\in A,$

$\displaystyle\|q_{n}(\hat{h})-\tilde{q}_{n}(\tilde{h})\|$
$\displaystyle\lesssim\biggl{(}\frac{M}{m\sqrt{n}}\|g_{n}^{\prime\prime}(0)-g^{\prime\prime}(0)\|+\frac{ML}{m\sqrt{n}}(\|\hat{h}\|+\|\tilde{h}\|)\biggr{)}\biggl{(}\sum_{j=1}^{n}\|x_{j}-\tilde{x}_{j}\|^{2}\biggr{)}^{1/2}.$

Recall that function $\varphi$ is Lipschitz on $A$ with constant
$\frac{24L}{m}\frac{4M}{m\sqrt{n}}$ and function $\psi$ is Lipschitz on the
whole space with constant $\frac{8}{m}\frac{L}{\sqrt{n}}.$ Note also that

$\displaystyle\|q_{n}(\hat{h})\|=\Bigl{\|}\int_{0}^{1}(g_{n}^{\prime\prime}-g^{\prime\prime})(\lambda\hat{h})d\lambda\
\hat{h}\Bigr{\|}\leq\|g_{n}^{\prime\prime}(0)-g^{\prime\prime}(0)\|\|\hat{h}\|+L\|\hat{h}\|^{2}.$

Since, on set $A,$ $\|\hat{h}\|\leq\frac{m}{12L},$ we get

$\displaystyle\|q_{n}(\hat{h})\|\leq\frac{m}{12L}\|g_{n}^{\prime\prime}(0)-g^{\prime\prime}(0)\|+\frac{m}{12}\|\hat{h}\|.$

Denoting
$\varphi:=\varphi(x_{1},\dots,x_{n}),\tilde{\varphi}:=\varphi(\tilde{x}_{1},\dots,\tilde{x}_{n}),\psi:=\psi(x_{1},\dots,x_{n}),\tilde{\psi}:=\psi(\tilde{x}_{1},\dots,\tilde{x}_{n}),$
it easily follows from the facts mentioned above that, for all
$(x_{1},\dots,x_{n}),(\tilde{x}_{1},\dots,\tilde{x}_{n})\in A,$

$\displaystyle\|q_{n}(\hat{h})\varphi\psi-\tilde{q}_{n}(\tilde{h})\tilde{\varphi}\tilde{\psi}\|$
$\displaystyle\lesssim\biggl{(}\frac{M}{m\sqrt{n}}\|g_{n}^{\prime\prime}(0)-g^{\prime\prime}(0)\|+\frac{ML}{m\sqrt{n}}(\|\hat{h}\|+\|\tilde{h}\|)\biggr{)}\biggl{(}\sum_{j=1}^{n}\|x_{j}-\tilde{x}_{j}\|^{2}\biggr{)}^{1/2}.$

This implies the following bound on the local Lipschitz constant of
$q_{n}(\hat{h})\varphi\psi$ on set $A:$

$\displaystyle L(q_{n}(\hat{h})\varphi\psi)(x_{1},\dots,x_{n})$
$\displaystyle\lesssim\biggl{(}\frac{M}{m\sqrt{n}}\Bigl{(}\|g_{n}^{\prime\prime}(0)-g^{\prime\prime}(0)\|\wedge\frac{m}{4}\Bigr{)}+\frac{ML}{m\sqrt{n}}\Bigl{(}\|\hat{h}\|\wedge\frac{m}{12L}\Bigr{)}\biggr{)}.$
(4.8)

The same bound trivially holds on the open set $A^{c}$ (where
$q_{n}(\hat{h})\varphi\psi)(x_{1},\dots,x_{n})=0$) and, by the argument
already used at the end of the proof of Proposition 4.4, it is easy to
conclude that bound (4) holds on the whole space.

Using bound (4.4) and the bound of Proposition 4.2, we get, for all
$(x_{1},\dots,x_{n}),(\tilde{x}_{1},\dots,\tilde{x}_{n})\in A,$

$\displaystyle\|r(\hat{h})-r(\tilde{h})\|\leq\frac{2ML}{m\sqrt{n}}(\|\hat{h}\|+\|\tilde{h}\|)\biggl{(}\sum_{j=1}^{n}\|x_{j}-\tilde{x}_{j}\|^{2}\biggr{)}^{1/2}$

and we also have $\|r(\hat{h})\|\leq\frac{L}{2}\|\hat{h}\|^{2}.$ As a result,
we get the following condition for the function $r(\hat{h})\varphi\psi$ on set
$A:$

$\displaystyle\|r(\hat{h})\varphi\psi-r(\tilde{h})\tilde{\varphi}\tilde{\psi}\|\lesssim\frac{2ML}{m\sqrt{n}}(\|\hat{h}\|+\|\tilde{h}\|)\biggl{(}\sum_{j=1}^{n}\|x_{j}-\tilde{x}_{j}\|^{2}\biggr{)}^{1/2}.$

This implies a bound on the local Lipschitz constant of
$r(\hat{h})\varphi\psi$ on set $A$ that, by the arguments already used, could
be extended to the bound that holds on the whole space:

$\displaystyle
L(r(\hat{h})\varphi\psi)(x_{1},\dots,x_{n})\lesssim\frac{ML}{m\sqrt{n}}\Bigl{(}\|\hat{h}\|\wedge\frac{m}{12L}\Bigr{)}.$

Denoting

$\displaystyle\zeta(x_{1},\dots,x_{n}):=(-\langle
q_{n}(\hat{h}),u\rangle\varphi\psi+\langle
r(\hat{h}),u\rangle\varphi\psi)(x_{1},\dots,x_{n}),$

we can conclude that

$\displaystyle(L\zeta)(x_{1},\dots,x_{n})\lesssim\|u\|\biggl{(}\frac{M}{m\sqrt{n}}\Bigl{(}\|g_{n}^{\prime\prime}(0)-g^{\prime\prime}(0)\|\wedge\frac{m}{4}\Bigr{)}+\frac{ML}{m\sqrt{n}}\Bigl{(}\|\hat{h}\|\wedge\frac{m}{12L}\Bigr{)}\biggr{)}.$
(4.9)

By the second bound of Proposition 3.2,

$\displaystyle\Bigl{\|}\|g_{n}^{\prime\prime}(0)-g^{\prime\prime}(0)\|\Bigr{\|}_{\psi_{2}}\lesssim
M\sqrt{\frac{d}{n}}.$

By the bound of Corollary 3.2,

$\displaystyle\Bigl{\|}\|\hat{\theta}-\theta\|\wedge\frac{m}{12L}\Bigr{\|}_{\psi_{2}}\lesssim\frac{\sqrt{M}}{m}\sqrt{\frac{d}{n}}+\frac{m}{L\sqrt{\gamma}}\frac{1}{\sqrt{n}}.$
(4.10)

Substituting the above bounds in (4.9), we conclude that

$\displaystyle\Bigl{\|}(L\zeta)(X_{1},\dots,X_{n})\Bigr{\|}_{\psi_{2}}$
$\displaystyle\lesssim\Bigl{(}\frac{M^{2}}{m\sqrt{n}}\sqrt{\frac{d}{n}}+\frac{M^{3/2}L}{m^{2}\sqrt{n}}\sqrt{\frac{d}{n}}+\frac{M}{\sqrt{\gamma}}\frac{1}{n}\Bigr{)}\|u\|$
$\displaystyle\lesssim\Bigl{(}\frac{M^{2}}{m\sqrt{n}}\sqrt{\frac{d}{n}}+\frac{M^{3/2}L}{m^{2}\sqrt{n}}\sqrt{\frac{d}{n}}\Bigr{)}\|u\|,$

where we also used the fact that the term $\frac{M}{\sqrt{\gamma}}\frac{1}{n}$
is dominated by other terms.

We are now ready to use concentration inequalities for functions of log-
concave r.v. to control
$\zeta(X_{1},\dots,X_{n})-{\mathbb{E}}\zeta(X_{1},\dots,X_{n}).$ For all
$p\geq 1,$ we have

$\displaystyle\Bigl{\|}\zeta(X_{1},\dots,X_{n})-{\mathbb{E}}\zeta(X_{1},\dots,X_{n})\Bigr{\|}_{L_{p}}$
$\displaystyle\lesssim\sqrt{c(V)}p\Bigl{\|}(L\zeta)(X_{1},\dots,X_{n})\Bigr{\|}_{L_{p}}\lesssim\sqrt{c(V)}p^{3/2}\Bigl{\|}(L\zeta)(X_{1},\dots,X_{n})\Bigr{\|}_{\psi_{2}}.$

It follows that

$\displaystyle\Bigl{\|}\zeta(X_{1},\dots,X_{n})-{\mathbb{E}}\zeta(X_{1},\dots,X_{n})\Bigr{\|}_{\psi_{2/3}}$
$\displaystyle\lesssim\sqrt{c(V)}\Bigl{\|}(L\zeta)(X_{1},\dots,X_{n})\Bigr{\|}_{\psi_{2}}$
$\displaystyle\lesssim\sqrt{c(V)}\Bigl{(}\frac{M^{2}}{m\sqrt{n}}\sqrt{\frac{d}{n}}+\frac{M^{3/2}L}{m^{2}\sqrt{n}}\sqrt{\frac{d}{n}}\Bigr{)}\|u\|.$

Recalling representation (4) and bound (4.7), we get

$\displaystyle\Bigl{\|}\langle\check{\theta}-\theta,w\rangle-{\mathbb{E}}\langle\check{\theta}-\theta,w\rangle-\langle{\mathcal{I}}^{-1}g_{n}^{\prime}(0),w\rangle\Bigr{\|}_{\psi_{2/3}}$
$\displaystyle\lesssim\sqrt{c(V)}\Bigl{(}\frac{M^{2}}{m\sqrt{n}}\sqrt{\frac{d}{n}}+\frac{M^{3/2}L}{m^{2}\sqrt{n}}\sqrt{\frac{d}{n}}\Bigr{)}\|u\|+\sqrt{\frac{M}{\gamma}}\frac{1}{n}\|u\|.$

Since $c(V)\geq\|\Sigma\|\geq\|\mathcal{I}^{-1}\|$ and $M\geq\|\mathcal{I}\|$,
we have $c(V)M\geq 1.$ Recalling the definition of $\gamma$ and also that
$\|u\|=\|{\mathcal{I}}^{-1}w\|\leq\frac{1}{m}\|w\|,$ it is easy to complete
the proof.

∎

We are ready to prove Theorem 4.1.

proof. Note that

$\displaystyle f(\check{\theta})-f(\theta)=\langle
f^{\prime}(\theta),\check{\theta}-\theta\rangle+S_{f}(\theta;\check{\theta}-\theta).$

Therefore,

$\displaystyle f(\check{\theta})-{\mathbb{E}}_{\theta}f(\check{\theta})$
$\displaystyle=\langle
f^{\prime}(\theta),\check{\theta}-\theta\rangle-{\mathbb{E}}_{\theta}\langle
f^{\prime}(\theta),\check{\theta}-\theta\rangle+S_{f}(\theta;\check{\theta}-\theta)-{\mathbb{E}}_{\theta}S_{f}(\theta;\check{\theta}-\theta).$
(4.11)

By the bound of Theorem 4.2,

$\displaystyle\Bigl{\|}\langle
f^{\prime}(\theta),\check{\theta}-\theta\rangle-{\mathbb{E}}\langle
f^{\prime}(\theta),\check{\theta}-\theta\rangle-n^{-1}\sum_{j=1}^{n}\langle
V^{\prime}(\xi_{j}),{\mathcal{I}}^{-1}f^{\prime}(\theta)\rangle\Bigr{\|}_{\psi_{2/3}}$
$\displaystyle\leq\sqrt{c(V)}\Bigl{(}\frac{M^{2}}{m^{2}}+\frac{M^{3/2}L}{m^{3}}\Bigr{)}\frac{1}{\sqrt{n}}\sqrt{\frac{d}{n}}\|f^{\prime}(\theta)\|.$
(4.12)

Thus, it remains to control
$S_{f}(\theta;\check{\theta}-\theta)-{\mathbb{E}}_{\theta}S_{f}(\theta;\check{\theta}-\theta).$
By Proposition 3.1 (v), for function $f\in C^{s},$ $s=1+\rho,$ $\rho\in(0,1],$
we have

$\displaystyle|S_{f}(\theta;h)-S_{f}(\theta;h^{\prime})|\lesssim\|f\|_{C^{s}}(\|h\|^{\rho}\vee\|h^{\prime}\|^{\rho})\|h-h^{\prime}\|,\theta,h,h^{\prime}\in{\mathbb{R}}^{d}.$

Combining this with the bound of Proposition 4.4, we easily get

$\displaystyle\Bigl{|}S_{f}(\theta;\check{\theta}(x_{1},\dots,x_{n})-\theta)-S_{f}(\theta;\check{\theta}(\tilde{x}_{1},\dots,\tilde{x}_{n})-\theta)\Bigr{|}$
$\displaystyle\lesssim\|f\|_{C^{s}}\frac{M}{m\sqrt{n}}(\|\check{\theta}(x_{1},\dots,x_{n})-\theta\|^{\rho}\vee\|\check{\theta}(\tilde{x}_{1},\dots,\tilde{x}_{n})-\theta\|^{\rho})\Bigl{(}\sum_{j=1}^{n}\|x_{j}-\tilde{x}_{j}\|^{2}\Bigr{)}^{1/2},$

which implies the following bound on the local Lipschitz function of function
$S_{f}(\theta;\check{\theta}-\theta):$

$\displaystyle(LS_{f}(\theta;\check{\theta}-\theta))(x_{1},\dots,x_{n})\lesssim\|f\|_{C^{s}}\frac{M}{m\sqrt{n}}\|\check{\theta}(x_{1},\dots,x_{n})-\theta\|^{\rho}.$

Using concentration bounds for log-concave r.v., we get

$\displaystyle\Bigl{\|}S_{f}(\theta;\check{\theta}-\theta)-{\mathbb{E}}_{\theta}S_{f}(\theta;\check{\theta}-\theta)\Bigr{\|}_{L_{p}}\lesssim\sqrt{c(V)}p\Bigl{\|}(LS_{f}(\theta;\check{\theta}-\theta))(X_{1},\dots,X_{n})\Bigr{\|}_{L_{p}}$
$\displaystyle\lesssim\sqrt{c(V)}p\|f\|_{C^{s}}\frac{M}{m\sqrt{n}}\Bigl{\|}\|\check{\theta}-\theta\|^{\rho}\Bigr{\|}_{L_{p}}\lesssim\sqrt{c(V)}p\|f\|_{C^{s}}\frac{M}{m\sqrt{n}}\Bigl{\|}\|\check{\theta}-\theta\|\Bigr{\|}_{L_{p}}^{\rho}$
$\displaystyle\lesssim\sqrt{c(V)}p\|f\|_{C^{s}}\frac{M}{m\sqrt{n}}\Bigl{\|}\|\hat{\theta}-\theta\|\wedge\frac{m}{12L}\Bigr{\|}_{L_{p}}^{\rho}\lesssim\sqrt{c(V)}p^{1+\rho/2}\|f\|_{C^{s}}\frac{M}{m\sqrt{n}}\Bigl{\|}\|\hat{\theta}-\theta\|\wedge\frac{m}{12L}\Bigr{\|}_{\psi_{2}}^{\rho},$

which, using bound (4.10), implies that

$\displaystyle\Bigl{\|}S_{f}(\theta;\check{\theta}-\theta)-{\mathbb{E}}_{\theta}S_{f}(\theta;\check{\theta}-\theta)\Bigr{\|}_{\psi_{2/(2+\rho)}}\lesssim\sqrt{c(V)}\|f\|_{C^{s}}\frac{M}{m\sqrt{n}}\Bigl{\|}\|\hat{\theta}-\theta\|\wedge\frac{m}{12L}\Bigr{\|}_{\psi_{2}}^{\rho}$
$\displaystyle\lesssim\sqrt{c(V)}\|f\|_{C^{s}}\biggl{(}\frac{M^{1+\rho/2}}{m^{1+\rho}}\frac{1}{\sqrt{n}}\Bigl{(}\frac{d}{n}\Bigr{)}^{\rho/2}+\frac{M}{L^{\rho}m^{1-\rho}\gamma^{\rho/2}}\frac{1}{n^{(1+\rho)/2}}\biggr{)}.$

Combining this with (4) and (4), we get

$\displaystyle\Bigl{\|}f(\check{\theta})-{\mathbb{E}}_{\theta}f(\check{\theta})-n^{-1}\sum_{j=1}^{n}\langle
V^{\prime}(\xi_{j}),{\mathcal{I}}^{-1}f^{\prime}(\theta)\rangle\Bigr{\|}_{\psi_{2/3}}$
$\displaystyle\lesssim\sqrt{c(V)}\|f^{\prime}(\theta)\|\Bigl{(}\frac{M^{2}}{m^{2}}+\frac{M^{3/2}L}{m^{3}}\Bigr{)}\frac{1}{\sqrt{n}}\sqrt{\frac{d}{n}}$
$\displaystyle+\sqrt{c(V)}\|f\|_{C^{s}}\biggl{(}\frac{M^{1+\rho/2}}{m^{1+\rho}}\frac{1}{\sqrt{n}}\Bigl{(}\frac{d}{n}\Bigr{)}^{\rho/2}+\frac{M}{L^{\rho}m^{1-\rho}\gamma^{\rho/2}}\frac{1}{n^{(1+\rho)/2}}\biggr{)}$
$\displaystyle\lesssim_{M,L,m}\sqrt{c(V)}\|f\|_{C^{s}}\frac{1}{\sqrt{n}}\Bigl{(}\frac{d}{n}\Bigr{)}^{\rho/2}.$

It remains to replace in the above bound $\check{\theta}$ by $\hat{\theta}.$
To this end, observe that $|f(\hat{\theta})-f(\check{\theta})|\leq
2\|f\|_{L_{\infty}}I(\check{\theta}\neq\hat{\theta}).$ This implies

$\displaystyle\|f(\hat{\theta})-f(\check{\theta})\|_{\psi_{1}}\leq
2\|f\|_{L_{\infty}}\|I(\check{\theta}\neq\hat{\theta})\|_{\psi_{1}}\lesssim\frac{\|f\|_{L_{\infty}}}{\gamma
n}\lesssim\frac{\|f\|_{C^{s}}}{\gamma n},$

which allows us to complete the proof.

∎

##  5 Bias reduction

We turn now to the bias reduction method outlined in Section 1. The
justification of this method is much simpler in the case of equivariant
estimators $\hat{\theta}$ of location parameter. Indeed, in this case

$\displaystyle\hat{\theta}(X_{1},\dots,X_{n})=\theta+\hat{\theta}(\xi_{1},\dots,\xi_{n}),$

where $\xi_{j}=X_{j}-\theta,j=1,\dots,n$ are i.i.d. $\sim P,$
$P(dx)=e^{-V(x)}dx.$ Denote $\vartheta:=\hat{\theta}(\xi_{1},\dots,\xi_{n})$
and let $\\{\vartheta_{k}\\}$ be a sequence of i.i.d. copies of $\vartheta$
defined as follows:
$\vartheta_{k}:=\hat{\theta}(\xi_{1}^{(k)},\dots,\xi_{n}^{(k)}),$
$\xi_{j}^{(k)},j=1,\dots,n,k\geq 1$ being i.i.d. copies of $\xi.$ Then, the
bootstrap chain $\\{\hat{\theta}^{(k)}:k\geq 0\\}$ has the same distribution
as the sequence of r.v. $\\{\theta+\sum_{j=1}^{k}\vartheta_{j}:k\geq 0\\}.$
Moreover, let

$\displaystyle\vartheta(t_{1},\dots,t_{k}):=\sum_{j=1}^{k}t_{j}\vartheta_{j},\
(t_{1},\dots,t_{k})\in[0,1]^{k}.$

Then, for $(t_{1},\dots,t_{k})\in\\{0,1\\}$ with $\sum_{i=1}^{n}t_{i}=j,$ we
have $\theta+\vartheta(t_{1},\dots,t_{k})\overset{d}{=}\hat{\theta}^{(j)}.$
Therefore, we can write

$\displaystyle({\mathcal{B}}^{k}f)(\theta)$
$\displaystyle={\mathbb{E}}_{\theta}\sum_{j=0}^{k}(-1)^{k-j}\binom{k}{j}f(\hat{\theta}^{(j)})$
$\displaystyle={\mathbb{E}}\sum_{(t_{1},\dots,t_{k})\in\\{0,1\\}^{k}}(-1)^{k-\sum_{j=1}^{k}t_{j}}f(\theta+\vartheta(t_{1},\dots,t_{k}))$
$\displaystyle={\mathbb{E}}\Delta_{1}\dots\Delta_{k}f(\theta+\vartheta(t_{1},\dots,t_{k})),$

where

$\displaystyle\Delta_{j}\varphi(t_{1},\dots,t_{k}):=\varphi(t_{1},\dots,t_{k})_{|t_{j}=1}-\varphi(t_{1},\dots,t_{k})_{|t_{j}=0}.$

If function $\varphi$ is $k$ times continuously differentiable, then by
Newton-Leibniz formula

$\displaystyle\Delta_{1}\dots\Delta_{k}\varphi(t_{1},\dots,t_{k})=\int_{0}^{1}\dots\int_{0}^{1}\frac{\partial^{k}\varphi(t_{1},\dots,t_{k})}{\partial
t_{1}\dots\partial t_{k}}dt_{1}\dots dt_{k}.$

If $f:{\mathbb{R}}^{d}\mapsto{\mathbb{R}}$ is $k$ times continuously
differentiable, then

$\displaystyle\frac{\partial^{k}}{\partial t_{1}\dots\partial
t_{k}}f(\theta+\vartheta(t_{1},\dots,t_{k}))=f^{(k)}(\theta+\vartheta(t_{1},\dots,t_{k}))[\vartheta_{1},\dots,\vartheta_{k}]$

and we end up with the following integral representation formula

$\displaystyle({\mathcal{B}}^{k}f)(\theta)={\mathbb{E}}\int_{0}^{1}\dots\int_{0}^{1}f^{(k)}(\theta+\vartheta(t_{1},\dots,t_{k}))[\vartheta_{1},\dots,\vartheta_{k}]dt_{1}\dots
dt_{k}$ (5.1)

that plays an important role in the analysis of functions ${\mathcal{B}}^{k}f$
and $f_{k}.$

It will be convenient to apply this formula not directly to MLE
$\hat{\theta},$ but to its smoothed and truncated approximation
$\check{\theta},$ defined in the previous section. Note that
$\check{\theta}(X_{1},\dots,X_{n})=\theta+\check{\vartheta},$ where

$\displaystyle\check{\vartheta}$
$\displaystyle=\check{\vartheta}(\xi_{1},\dots,\xi_{n})$
$\displaystyle:=\hat{\theta}(\xi_{1},\dots,\xi_{n})\phi\Bigl{(}\frac{24L}{m}\|\hat{\theta}(\xi_{1},\dots,\xi_{n})\|\Bigr{)}\phi\Bigl{(}\frac{8}{m}\|g_{n}^{\prime\prime}(0)(\xi_{1},\dots,\xi_{n})-g^{\prime\prime}(0)\|\Bigr{)}.$

Let $\check{\vartheta}_{k},k\geq 1$ be i.i.d. copies of $\check{\vartheta}$
defined as follows:

$\displaystyle\check{\vartheta}_{k}:=\vartheta_{k}\phi\Bigl{(}\frac{24L}{m}\|\vartheta_{k}\|\Bigr{)}\phi\Bigl{(}\frac{8}{m}\|g_{n}^{\prime\prime}(0)(\xi_{1}^{(k)},\dots,\xi_{n}^{(k)})-g^{\prime\prime}(0)\|\Bigr{)}.$

Note that, for all $k\geq 1,$ $\vartheta_{k}=\check{\vartheta}_{k}$ with
probability at least $1-2e^{-\gamma n}.$

Let $\check{\theta}^{(k)}:=\theta+\sum_{j=1}^{k}\check{\vartheta}_{j},k\geq
0.$ We will also introduce the operators
$(\check{\mathcal{T}}g)(\theta):={\mathbb{E}}_{\theta}g(\check{\theta}),\theta\in{\mathbb{R}}^{d}$
and $\check{\mathcal{B}}:=\check{\mathcal{T}}-{\mathcal{I}}.$ Let
$\check{f}_{k}:=\sum_{j=0}^{k}(-1)^{j}\check{\mathcal{B}}^{j}f.$ Since
$\vartheta_{j}=\check{\vartheta}_{j},j=1,\dots,k$ with probability at least
$1-2ke^{-\gamma n},$ we easily conclude that, if we identify
$\hat{\theta}^{(k)}$ with $\theta+\sum_{j=1}^{k}\vartheta_{j}$, $k\geq 0$,
then the event $E:=\\{\check{\theta}^{(j)}=\hat{\theta}^{(j)},j=1,\dots,k\\}$
occurs with the same probability. This immediately implies the following
proposition.

######  Proposition 5.1.

For all $k\geq 1,$

$\displaystyle\|f_{k}-\check{f}_{k}\|_{L_{\infty}}\leq
k2^{k+3}\|f\|_{L_{\infty}}e^{-\gamma n}$ (5.2)

and

$\displaystyle\|f_{k}\|_{L_{\infty}}\leq
2^{k+1}\|f\|_{L_{\infty}},\qquad\|\check{f}_{k}\|_{L_{\infty}}\leq
2^{k+1}\|f\|_{L_{\infty}}.$ (5.3)

proof. For all $\theta\in{\mathbb{R}}^{d}$ and all $j=1,\dots,k,$

$\displaystyle|{\mathbb{E}}_{\theta}f(\hat{\theta}^{(j)})-{\mathbb{E}}_{\theta}f(\check{\theta}^{(j)})|$
$\displaystyle=|{\mathbb{E}}_{\theta}f(\hat{\theta}^{(j)})I_{E^{c}}-{\mathbb{E}}_{\theta}f(\check{\theta}^{(j)})I_{E^{c}}|$
$\displaystyle\leq 2\|f\|_{L_{\infty}}{\mathbb{P}}(E^{c})\leq
4\|f\|_{L_{\infty}}ke^{-\gamma n}.$

Therefore, applying (1.2) to $f_{k}$ and $\check{f}_{k}$, we arrive at

$\displaystyle|f_{k}(\theta)-\check{f}_{k}(\theta)|$
$\displaystyle\leq\sum_{j=0}^{k}\binom{k+1}{j+1}|{\mathbb{E}}_{\theta}f(\hat{\theta}^{(j)})-{\mathbb{E}}_{\theta}f(\check{\theta}^{(j)})|$
$\displaystyle\leq k2^{k+3}\|f\|_{L_{\infty}}e^{-\gamma n},$

which proofs Proposition (5.1). Bounds (5.3) follow by a similar argument.

∎

Similarly to (5.1), we get

$\displaystyle(\check{\mathcal{B}}^{k}f)(\theta)$
$\displaystyle={\mathbb{E}}\int_{0}^{1}\dots\int_{0}^{1}f^{(k)}(\theta+\check{\vartheta}(t_{1},\dots,t_{k}))[\check{\vartheta}_{1},\dots,\check{\vartheta}_{k}]dt_{1}\dots
dt_{k}$
$\displaystyle={\mathbb{E}}f^{(k)}(\theta+\check{\vartheta}(\tau_{1},\dots,\tau_{k}))[\check{\vartheta}_{1},\dots,\check{\vartheta}_{k}],$
(5.4)

where
$\check{\vartheta}(t_{1},\dots,t_{k}):=\sum_{j=1}^{k}t_{j}\check{\vartheta}_{j},\
(t_{1},\dots,t_{k})\in[0,1]^{k}$ and $\tau_{1},\dots,\tau_{k}$ are i.i.d. r.v.
with uniform distribution in $[0,1]$ (independent of
$\\{\check{\vartheta}_{j}\\}$).

The next proposition follows from representation (5) and differentiation under
the expectation sign.

######  Proposition 5.2.

Let $f\in C^{s}$ for $s=k+1+\rho,$ where $k\geq 1$ and $\rho\in(0,1].$ Then,
for all $j=1,\dots,k,$

$\displaystyle\|\check{\mathcal{B}}^{j}f\|_{C^{1+\rho}}\lesssim\|f\|_{C^{s}}({\mathbb{E}}\|\check{\vartheta}\|)^{j}.$

If ${\mathbb{E}}\|\check{\vartheta}\|\leq 1/2,$ then

$\displaystyle\|\check{f}_{k}\|_{C^{1+\rho}}\lesssim\|f\|_{C^{s}}.$

We can also use representation (5) and smoothness of function
$\check{\mathcal{B}}^{k}f$ to obtain a bound on the bias of “estimator”
$\check{f}_{k}(\check{\theta}).$

######  Proposition 5.3.

Let $f\in C^{s}$ for $s=k+1+\rho,$ where $k\geq 1$ and $\rho\in(0,1].$ Then,
for all $\theta\in{\mathbb{R}}^{d},$

$\displaystyle|{\mathbb{E}}_{\theta}\check{f}_{k}(\check{\theta})-f(\theta)|\lesssim\|f\|_{C^{s}}({\mathbb{E}}\|\check{\vartheta}\|)^{k}(\|{\mathbb{E}}\check{\vartheta}\|+{\mathbb{E}}\|\check{\vartheta}\|^{1+\rho}).$

Moreover,

$\displaystyle|{\mathbb{E}}_{\theta}\check{f}_{k}(\check{\theta})-f(\theta)|\lesssim_{M,L,m}\|f\|_{C^{s}}\Bigl{(}\sqrt{\frac{d}{n}}\Bigr{)}^{s}.$

proof. Note that

$\displaystyle{\mathbb{E}}_{\theta}\check{f}_{k}(\check{\theta})-f(\theta)=(-1)^{k}(\check{\mathcal{B}}^{k+1}f)(\theta).$

We also have

$\displaystyle(\check{\mathcal{B}}^{k+1}f)(\theta)$
$\displaystyle={\mathbb{E}}_{\theta}(\check{\mathcal{B}}^{k}f)(\check{\theta})-(\check{\mathcal{B}}^{k}f)(\theta)$
$\displaystyle=\langle(\check{\mathcal{B}}^{k}f)^{\prime}(\theta),{\mathbb{E}}\check{\vartheta}\rangle+{\mathbb{E}}S_{\check{\mathcal{B}}^{k}f}(\theta;\check{\vartheta}).$

Using bounds of Proposition 5.2 and of Proposition 3.1, we get

$\displaystyle|(\check{\mathcal{B}}^{k+1}f)(\theta)|$
$\displaystyle\lesssim\|(\check{\mathcal{B}}^{k}f)^{\prime}\|\|{\mathbb{E}}\check{\vartheta}\|+\|\check{\mathcal{B}}^{k}f\|_{C^{1+\rho}}{\mathbb{E}}\|\check{\vartheta}\|^{1+\rho}$
$\displaystyle\lesssim\|f\|_{C^{s}}({\mathbb{E}}\|\check{\vartheta}\|)^{k}(\|{\mathbb{E}}\check{\vartheta}\|+{\mathbb{E}}\|\check{\vartheta}\|^{1+\rho}).$

Using also Proposition 4.3, we get

$\displaystyle|{\mathbb{E}}_{\theta}\check{f}_{k}(\check{\theta})-f(\theta)|\lesssim_{M,L,m}\|f\|_{C^{s}}\Bigl{(}\frac{d}{n}\Bigr{)}^{k/2}\Bigl{(}\frac{d}{n}+\Bigl{(}\frac{d}{n}\Bigr{)}^{(1+\rho)/2}\Bigr{)}$

which allows to complete the proof.

∎

In view of bound (5.3), the bound of Proposition 5.1 and the fact that
$\check{\theta}=\hat{\theta}$ with probability at least $1-2e^{-\gamma n},$ we
easily conclude that the following proposition holds:

######  Proposition 5.4.

Let $f\in C^{s}$ for $s=k+1+\rho,$ where $k\geq 1$ and $\rho\in(0,1].$ Then,
for all $\theta\in{\mathbb{R}}^{d},$

$\displaystyle|{\mathbb{E}}_{\theta}\check{f}_{k}(\hat{\theta})-f(\theta)|\lesssim_{M,L,m,s}\|f\|_{C^{s}}\Bigl{(}\sqrt{\frac{d}{n}}\Bigr{)}^{s}$

and

$\displaystyle|{\mathbb{E}}_{\theta}f_{k}(\hat{\theta})-f(\theta)|\lesssim_{M,L,m,s}\|f\|_{C^{s}}\Bigl{(}\sqrt{\frac{d}{n}}\Bigr{)}^{s}.$
(5.5)

It is now easy to prove Theorem 2.2.

proof. For all $\theta\in\mathbb{R}^{d}$,

$\displaystyle\Bigl{\|}f_{k}(\hat{\theta})-f(\theta)-n^{-1}\sum_{j=1}^{n}\langle
V^{\prime}(\xi_{j}),{\mathcal{I}}^{-1}f^{\prime}(\theta)\rangle\Bigr{\|}_{\psi_{2/3}}$
$\displaystyle\leq\Bigl{\|}\check{f}_{k}(\hat{\theta})-{\mathbb{E}}_{\theta}\check{f}_{k}(\hat{\theta})-n^{-1}\sum_{j=1}^{n}\langle
V^{\prime}(\xi_{j}),{\mathcal{I}}^{-1}\check{f}_{k}^{\prime}(\theta)\rangle\Bigr{\|}_{\psi_{2/3}}$
$\displaystyle+\Bigl{\|}n^{-1}\sum_{j=1}^{n}\langle
V^{\prime}(\xi_{j}),{\mathcal{I}}^{-1}\check{f}^{\prime}_{k}(\theta)-{\mathcal{I}}^{-1}f^{\prime}(\theta)\rangle\Bigr{\|}_{\psi_{2/3}}$
$\displaystyle+\|f_{k}-\check{f}_{k}\|_{L_{\infty}}+|\mathbb{E}_{\theta}\check{f}_{k}(\hat{\theta})-f(\theta)|.$
(5.6)

Applying Theorem 4.1 to function $\check{f}_{k}$ and using the second bound of
Proposition 5.2, we get

$\displaystyle\Bigl{\|}\check{f}_{k}(\hat{\theta})-{\mathbb{E}}_{\theta}\check{f}_{k}(\hat{\theta})-n^{-1}\sum_{j=1}^{n}\langle
V^{\prime}(\xi_{j}),{\mathcal{I}}^{-1}\check{f}_{k}^{\prime}(\theta)\rangle\Bigr{\|}_{\psi_{2/3}}$
$\displaystyle\lesssim_{M,L,m}\sqrt{c(V)}\|\check{f}_{k}\|_{C^{1+\rho}}\frac{1}{\sqrt{n}}\Bigl{(}\frac{d}{n}\Bigr{)}^{\rho/2}\lesssim_{M,L,m}\sqrt{c(V)}\|f\|_{C^{s}}\frac{1}{\sqrt{n}}\Bigl{(}\frac{d}{n}\Bigr{)}^{\rho/2}.$

Moreover, by Lemma 3.1 and the first bound in Proposition 5.2, we have

$\displaystyle\Bigl{\|}n^{-1}\sum_{j=1}^{n}\langle
V^{\prime}(\xi_{j}),{\mathcal{I}}^{-1}\check{f}^{\prime}_{k}(\theta)-{\mathcal{I}}^{-1}f^{\prime}(\theta)\rangle\Bigr{\|}_{\psi_{2/3}}$
$\displaystyle\lesssim\frac{1}{m}\frac{\sqrt{M}}{\sqrt{n}}\|\check{f}^{\prime}_{k}(\theta)-f^{\prime}(\theta)\|\leq\frac{1}{m}\frac{\sqrt{M}}{\sqrt{n}}\sum_{j=1}^{k}\|(\mathcal{B}^{j}f)^{\prime}(\theta)\|\lesssim\|f\|_{C^{s}}\frac{1}{m}\frac{\sqrt{M}}{\sqrt{n}}.$

Inserting these inequalities into (5.6) and applying Propositions 5.1 and 5.3
to the last two term in (5.6), allows to complete the proof.

∎

Next we provide the proof of Proposition 2.1.

proof. The minimum with $1$ in both bounds is due to the fact that
$\|f_{k}\|_{L_{\infty}}\lesssim\|f\|_{L_{\infty}}\leq\|f\|_{C^{s}};$ so, the
left hand side is trivially bounded up to a constant by $\|f\|_{C^{s}}.$

To prove the first claim, note that, for $f$ with $\|f\|_{C^{s}}\leq 1,$

$\displaystyle\|f(\hat{\theta})-f(\theta)\|_{L_{2}({\mathbb{P}}_{\theta})}$
$\displaystyle\leq\Bigl{\|}\Bigl{(}\|\hat{\theta}-\theta\|\wedge\frac{m}{12L}\Bigr{)}^{s}\Bigr{\|}_{L_{2}({\mathbb{P}}_{\theta})}+2\Bigl{\|}I(\|\hat{\theta}-\theta\|\geq
m/(12L))\Bigr{\|}_{L_{2}({\mathbb{P}}_{\theta})}$
$\displaystyle\leq\Bigl{(}\Bigl{\|}\|\hat{\theta}-\theta\|\wedge\frac{m}{12L}\Bigr{\|}_{L_{2}({\mathbb{P}}_{\theta})}\Bigr{)}^{s}+2{\mathbb{P}}_{\theta}^{1/2}\Bigl{\\{}\|\hat{\theta}-\theta\|\geq
m/(12L)\Bigr{\\}}.$

Using the bound of Corollary 3.2, we get

$\displaystyle\Bigl{(}\Bigl{\|}\|\hat{\theta}-\theta\|\wedge\frac{m}{12L}\Bigr{\|}_{L_{2}({\mathbb{P}}_{\theta})}\Bigr{)}^{s}\lesssim_{M,L,m}\Bigl{(}\sqrt{\frac{d}{n}}\Big{)}^{s},$

and, by the bound of Theorem 3.1, we easily get

$\displaystyle{\mathbb{P}}_{\theta}\Bigl{\\{}\|\hat{\theta}-\theta\|\geq
m/(12L)\Bigr{\\}}\leq e^{-\gamma n}.$

The first claim now easily follows.

The proof of the second claim easily follows from Corollary 2.1. We can assume
that $d\leq\gamma n$ (otherwise, the bound is obvious) and we can drop the
term $\sqrt{\frac{c(V)}{n}}\Bigl{(}\frac{d}{n}\Bigr{)}^{\rho/2}$ in the bound
of Corollary 2.1: it is smaller than $\frac{1}{\sqrt{n}}$ since
$c(V)\lesssim_{\epsilon}d^{\epsilon}\|\Sigma\|$ for all $\epsilon>0$ and
$\|\Sigma\|\lesssim 1.$

∎

We will sketch the proof of Theorem 2.3.

proof. Under the stronger condition $V^{\prime\prime}(x)\succeq mI_{d},$ the
proof of Theorem 2.2 could be significantly simplified. Recall that, for
$\hat{h}=\theta-\hat{\theta},$ $g_{n}^{\prime}(\hat{h})=0.$ This implies that

$\displaystyle
g_{n}^{\prime}(0)=g_{n}^{\prime}(0)-g_{n}^{\prime}(\hat{h})=-\int_{0}^{1}g_{n}^{\prime\prime}(\lambda\hat{h})d\lambda\
\hat{h}.$ (5.7)

The condition $V^{\prime\prime}(x)\succeq mI_{d}$ easily implies that

$\displaystyle\int_{0}^{1}g_{n}^{\prime\prime}(\lambda\hat{h})d\lambda=\int_{0}^{1}n^{-1}\sum_{j=1}^{n}V^{\prime\prime}(\xi_{j}+\lambda\hat{h})d\lambda\succeq
mI_{d}.$

Therefore, $\Bigl{\|}\int_{0}^{1}g_{n}^{\prime\prime}(\lambda\hat{h})d\lambda\
u\Bigr{\|}\geq m\|u\|,u\in{\mathbb{R}}^{d}.$ Combining this with (5.7) yields
$\|\hat{h}\|\leq\frac{\|g_{n}^{\prime}(0)\|}{m}.$ By Corollary 3.1, we get
that for all $t\geq 1,$ with probability at least $1-e^{-t}$

$\displaystyle\|\hat{\theta}-\theta\|=\|\hat{h}\|\lesssim\frac{\sqrt{M}}{m}\Bigl{(}\sqrt{\frac{d}{n}}\vee\sqrt{\frac{t}{n}}\Bigr{)}.$

Unlike the case of Theorem 3.1, the above bound holds in the whole range of
$t\geq 1$ and it immediately implies that
${\mathbb{E}}_{\theta}^{1/2}\|\hat{\theta}-\theta\|^{2}\lesssim\frac{\sqrt{M}}{m}\sqrt{\frac{d}{n}},$
and, moreover,
$\|\|\hat{\theta}-\theta\|\|_{L_{\psi_{2}}({\mathbb{P}}_{\theta})}\lesssim\frac{\sqrt{M}}{m}\sqrt{\frac{d}{n}}.$

Quite similarly, one can show that, unlike the case of Proposition 4.2, the
Lipschitz condition for the function
${\mathbb{R}}^{d}\times\dots\times{\mathbb{R}}^{d}\ni(x_{1},\dots,x_{n})\mapsto\hat{\theta}(x_{1},\dots,x_{n})\in{\mathbb{R}}^{d}$
holds not just on set $A,$ but on the whole space. Indeed, recall that
$g_{n}^{\prime}(\hat{h})=0$ and $\tilde{g}_{n}^{\prime}(\tilde{h})=0.$ This
implies that

$\displaystyle\tilde{g}_{n}^{\prime}(\tilde{h})-g_{n}^{\prime}(\tilde{h})=g_{n}^{\prime}(\hat{h})-g_{n}^{\prime}(\tilde{h})=\int_{0}^{1}g_{n}^{\prime\prime}(\tilde{h}+\lambda(\hat{h}-\tilde{h}))d\lambda\
(\hat{h}-\tilde{h}).$

Since
$\int_{0}^{1}g_{n}^{\prime\prime}(\tilde{h}+\lambda(\hat{h}-\tilde{h}))d\lambda\succeq
mI_{d},$ we get

$\displaystyle\|\tilde{g}_{n}^{\prime}(\tilde{h})-g_{n}^{\prime}(\tilde{h})\|\geq
m\|\hat{h}-\tilde{h}\|,$

which implies

$\displaystyle\|\tilde{\theta}-\hat{\theta}\|$
$\displaystyle=\|\hat{h}-\tilde{h}\|\leq
m^{-1}n^{-1}\sum_{j=1}^{n}\|V^{\prime}(\tilde{h}+\tilde{\xi}_{j})-V^{\prime}(\tilde{h}+\xi_{j})\|$
$\displaystyle\leq
m^{-1}n^{-1}M\sum_{j=1}^{n}\|\tilde{\xi}_{j}-\xi_{j}\|\leq\frac{M}{m\sqrt{n}}\Bigl{(}\sum_{j=1}^{n}\|\tilde{\xi}_{j}-\xi_{j}\|^{2}\Bigr{)}^{1/2}$
$\displaystyle=\frac{M}{m\sqrt{n}}\Bigl{(}\sum_{j=1}^{n}\|\tilde{x}_{j}-x_{j}\|^{2}\Bigr{)}^{1/2},$

and the Lipschitz condition holds for the function
${\mathbb{R}}^{d}\times\dots\times{\mathbb{R}}^{d}\ni(x_{1},\dots,x_{n})\mapsto\hat{\theta}(x_{1},\dots,x_{n})\in{\mathbb{R}}^{d}$
with constant $\frac{M}{m\sqrt{n}}.$ Due to this fact, there is no need to
consider a “smoothed version” $\check{\theta}$ of function $\hat{\theta}$ in
the remainder of the proof (as it was done in the proof of Theorem 2.2). All
the arguments could be applied directly to $\hat{\theta}.$

Finally, recall that, if $\xi\sim P,$ $P(dx)=e^{-V(x)}dx$ with
$V^{\prime\prime}(x)\succeq mI_{d},x\in{\mathbb{R}}^{d},$ then, for all
locally Lipschitz functions $g:{\mathbb{R}}^{d}\mapsto{\mathbb{R}},$ the
following logarirthmic Sobolev inequality holds

$\displaystyle{\mathbb{E}}g^{2}(\xi)\log
g^{2}(\xi)-{\mathbb{E}}g^{2}(\xi)\log{\mathbb{E}}g^{2}(\xi)\lesssim\frac{1}{m}{\mathbb{E}}\|\nabla
g(\xi)\|^{2}$

(see, e.g., [20], Theorem 5.2). It was proved in [2] (see also [1]) that this
implies the following moment bound:

$\displaystyle\|g(\xi)-{\mathbb{E}}g(\xi)\|_{L_{p}}\lesssim_{m}\sqrt{p}\|\|\nabla
g(\xi)\|\|_{L_{p}},p\geq 2.$

This bound is used to modify the concentration inequalities of Section 4,
which yields the claim of Theorem 2.3.

∎

##  6 Minimax lower bounds

In this section, we provide lower bounds for the estimation of the location
parameter and functionals thereof that match the upper bounds obtained in the
previous sections up to constants.

We start with a comment on the proof of Proposition 2.2.

proof. It follows the same line of arguments as in the proof of Theorem 2.2 in
[18]. It is based on a construction of a set ${\mathcal{F}}$ of smooth
functionals such that the existence of estimators of $f(\theta)$ for all
$f\in{\mathcal{F}}$ with some error rate would allow one to design an
estimator of parameter $\theta$ itself with a certain error rate. This rate is
then compared with a minimax lower bound
$\inf_{\hat{\theta}}\max_{\theta\in\Theta}\mathbb{E}_{\theta}\|\hat{\theta}-\theta\|^{2}$
in the parameter estimation problem, where $\Theta$ is a maximal
$\varepsilon$-net of the unit sphere (for a suitable $\varepsilon$), yielding
as a result a minimax lower bound in the functional estimation. Minimax lower
bound in the parameter estimation problem can be deduced in a standard way
from Theorem 2.5 of [28] using KL-divergence (Fano’s type argument). In fact,
while in the Gaussian location model KL-divergence coincides with $1/2$ times
the squared Euclidean distance, a similar property also holds for our log-
concave location models:

$\displaystyle K(P_{\theta}\|P_{\theta^{\prime}})$
$\displaystyle=\mathbb{E}(V(\xi+\theta-\theta^{\prime})-V(\xi))$
$\displaystyle=\mathbb{E}(V(\xi+\theta-\theta^{\prime})-V(\xi)-\langle
V^{\prime}(\xi),\theta-\theta^{\prime}\rangle)\leq
M\|\theta-\theta^{\prime}\|^{2}/2,$

where we used Proposition 3.1 and Assumption 1 in the inequality.

∎

Our next goal is to provide the proof of the local minimax lower bound of
Proposition 2.3. It will be based on Bayes risk lower bounds for the
estimation of location parameter as well as functionals thereof that might be
of independent interest.

Let $(\mathcal{X},\mathcal{F},(\mathbb{P}_{\theta})_{\theta\in\Theta})$ be a
statistical model and let $G$ be a topological group acting on the measurable
space $(\mathcal{X},\mathcal{F})$ and $\Theta.$ Let $\Theta$ be such that
$(\mathbb{P}_{\theta})_{\theta\in\Theta}$ is $G$-equivariant, i.e.,
$\mathbb{P}_{g\theta}(gA)=\mathbb{P}_{\theta}(A)$ for all $g\in
G,\theta\in\Theta$ and $A\in\mathcal{F}.$ Suppose that
$g\mapsto\mathbb{P}_{g\theta}(A)$ is measurable for every $A\in\mathcal{F}$,
$\theta\in\Theta$.

Recall that, for two probability measures $\mu,\nu$ on an arbitrary measurable
space with $\mu$ being absolutely continuous w.r.t. $\nu,$ the
$\chi^{2}$-divergence $\chi^{2}(\mu,\nu)$ is defined as

$\displaystyle\chi^{2}(\mu,\nu):=\int\Bigl{(}\frac{d\mu}{d\nu}-1\Bigr{)}^{2}d\nu=\int\Bigl{(}\frac{d\mu}{d\nu}\Bigr{)}^{2}d\nu-1.$

The key ingredient in our proofs is the following version of an equivariant
van Trees type inequality established in [30], Proposition 1.

######  Lemma 6.1.

Let $\Pi$ be a Borel probability measure on $G,$ let
$\psi:\Theta\rightarrow\mathbb{R}^{m}$ be a derived parameter such that
$\int_{G}\|\psi(g\theta)\|^{2}\Pi(dg)<\infty$ for all $\theta\in\Theta$ and
let $\hat{\psi}:\mathcal{X}\rightarrow\mathbb{R}^{m}$ be an estimator of
$\psi(\theta),$ based on an observation
$X\sim{\mathbb{P}}_{\theta},\theta\in\Theta.$ Then, for all $\theta\in\Theta$
and all $h_{1},\dots,h_{m}\in G$, we have

$\displaystyle\int_{G}\mathbb{E}_{g\theta}\|\hat{\psi}(X)-\psi(g\theta)\|^{2}\,\Pi(dg)$
$\displaystyle\geq\frac{\big{(}\sum_{j=1}^{m}\int_{G}(\psi_{j}(gh_{j}^{-1}\theta)-\psi_{j}(g\theta))\,\Pi(dg)\big{)}^{2}}{\sum_{j=1}^{m}\big{(}\chi^{2}(\mathbb{P}_{h_{j}\theta},\mathbb{P}_{\theta})+\chi^{2}(\Pi\circ
R_{h_{j}},\Pi)+\chi^{2}(\mathbb{P}_{h_{j}\theta},\mathbb{P}_{\theta})\chi^{2}(\Pi\circ
R_{h_{j}},\Pi)\big{)}},$

with $\Pi\circ R_{h_{j}}$ defined by $(\Pi\circ R_{h_{j}})(B)=\Pi(Bh_{j})$ for
all Borel sets $B\subset G.$

This lemma will be applied to our log-concave location model with $G$ being
the group of all translations of ${\mathbb{R}}^{d}.$ The Bayes risk lower
bound will be formulated for the class of all prior density functions
$\pi:\mathbb{R}^{d}\mapsto{\mathbb{R}}_{+}$ with respect to the Lebesgue
measure on $\mathbb{R}^{d}$ satisfying one of the following two conditions:

  1. (P1)

$\pi=e^{-W}$ with $W:\mathbb{R}^{d}\mapsto\mathbb{R}$ being twice
differentiable such that $\|W^{\prime\prime}\|_{L_{\infty}}$ and
$\|W^{\prime\prime}\|_{\operatorname{Lip}}$ are finite,

  2. (P2)

$\pi$ has bounded support and is twice differentiable such that
$\|\pi^{(j)}\|_{L_{\infty}}$ and $\|\pi^{(j)}\|_{\operatorname{Lip}}$ are
finite for $j=0,1,2$ (actually, it suffices to assume it only for $j=2$ since
the support is bounded).

Our first result deals with the estimation of the location parameter itself.
We assume that i.i.d. observations $X_{1},\dots,X_{n}$ are sampled from a
distribution belonging to a log-concave location family
$e^{-V(x-\theta)}dx,\theta\in{\mathbb{R}}^{d}$ with convex function $V$
satisfying Assumption 1.

######  Theorem 6.1.

Let $\Pi$ be a probability measure on $\mathbb{R}^{d}$ with density
$\pi:\mathbb{R}^{d}\mapsto{\mathbb{R}}_{+}$ with respect to the Lebesgue
measure satisfying either (P1) or (P2). Suppose that $\pi$ has finite Fisher
information matrix

$\displaystyle\mathcal{J}_{\pi}=\int_{\mathbb{R}^{d}}\frac{\pi^{\prime}(\theta)\otimes\pi^{\prime}(\theta)}{\pi(\theta)}\,d\theta.$

Then, for all $\delta>0$, we have

$\displaystyle\inf_{\hat{\theta}_{n}}\int_{\mathbb{R}^{d}}\mathbb{E}_{\theta}\|\hat{\theta}_{n}-\theta\|^{2}\,\Pi_{\delta}(d\theta)\geq\frac{1}{n}\operatorname{tr}\Big{(}\Big{(}\mathcal{I}+\frac{1}{\delta^{2}n}\mathcal{J}_{\pi}\Big{)}^{-1}\Big{)},$
(6.1)

where $\Pi_{\delta}$ is the prior measure with density
$\pi_{\delta}(\theta)=\delta^{-d}\pi(\delta^{-1}\theta)$,
$\theta\in\mathbb{R}^{d}$ and
$\hat{\theta}_{n}=\hat{\theta}_{n}(X_{1},\dots,X_{n})$ is an arbitrary
estimator based on $(X_{1},\dots,X_{n}).$

Let us discuss two simple implications. First, if we choose $\pi=e^{-V}$ that
satisfies (P1) (in view of Assumption 1) and let $\delta\to\infty,$ then
Theorem 6.1 implies

$\displaystyle\inf_{\hat{\theta}_{n}}\sup_{\theta\in\mathbb{R}^{d}}\mathbb{E}_{\theta}\|\hat{\theta}_{n}-\theta\|^{2}\geq\frac{1}{n}\operatorname{tr}(\mathcal{I}^{-1}).$

Secondly, if we choose

$\displaystyle\pi(\theta)=\prod_{j=1}^{d}\frac{3}{4}\cos^{3}(\theta_{j})I_{[-\frac{\pi}{2},\frac{\pi}{2}]}(\theta_{j}),$

that satisfies (P2), then we have, by an easy computation,
$\mathcal{J}_{\pi}=\frac{9}{2}I_{d}.$ Moreover, for
$\delta=\frac{2c}{\pi\sqrt{n}}$, $c>0$, the prior $\pi_{\delta}$ has support
in
$\\{\theta\in\mathbb{R}^{d}:\max_{j}|\theta_{j}|\leq\frac{c}{\sqrt{n}}\\}\subseteq\\{\theta\in\mathbb{R}^{d}:\|\theta\|\leq
c\sqrt{\frac{d}{n}}\\}$, and Theorem 6.1 implies

$\displaystyle\inf_{\hat{\theta}_{n}}\sup_{\|\theta\|\leq
c\sqrt{\frac{d}{n}}}\mathbb{E}_{\theta}\|\hat{\theta}_{n}-\theta\|^{2}\geq\frac{1}{n}\operatorname{tr}\Big{(}\Big{(}\mathcal{I}+\frac{9\pi^{2}}{8c^{2}}I_{d}\Big{)}^{-1}\Big{)}.$

The proof of Theorem 6.1 will be based on the following lemma.

######  Lemma 6.2.

Let $\pi:\mathbb{R}^{d}\mapsto{\mathbb{R}}_{+}$ be a probability density
function with respect to the Lebesgue measure $\lambda$ satisfying (P2) and
$\int\frac{\|\pi^{\prime}\|^{2}}{\pi}\,d\lambda<\infty$. Moreover, let
$p=e^{-W}$, $W:\mathbb{R}^{d}\mapsto\mathbb{R}$, be a probability density
function with respect $\lambda$ satisfying (P1). Suppose that $p$ is constant
on the support of $\pi$ and that
$\int_{\mathbb{R}^{d}}\|\theta\|^{2}\,p(\theta)d\theta<\infty$. For
$\epsilon>0$, set

$\displaystyle\pi_{\epsilon}:=\frac{q+\epsilon}{1+\epsilon}\ p,\qquad
q:=\frac{\pi}{\int\pi p\,d\lambda}.$ (6.2)

Then, for every $\epsilon>0$, $\pi_{\epsilon}$ is a probability density
function with respect to $\lambda$ satisfying (P1) and
$\int\frac{\|\pi_{\epsilon}^{\prime}\|^{2}}{\pi_{\epsilon}}\,d\lambda<\infty$.
Moreover, as $\epsilon\rightarrow 0$,

$\displaystyle\mathcal{J}_{\pi_{\epsilon}}=\int\frac{\pi_{\epsilon}^{\prime}\otimes\pi_{\epsilon}^{\prime}}{\pi_{\epsilon}}\,d\lambda\to\int\frac{\pi^{\prime}\otimes\pi^{\prime}}{\pi}\,d\lambda=\mathcal{J}_{\pi}.$

proof. To see that $\pi_{\epsilon}$ satisfies (P1), we have to show that
$W_{\epsilon}:\mathbb{R}^{d}\mapsto\mathbb{R}$ defined by
$\pi_{\epsilon}=e^{-W_{\epsilon}}$ is twice differentiable with
$\|W_{\epsilon}^{\prime\prime}\|_{L_{\infty}},\|W_{\epsilon}^{\prime\prime}\|_{\operatorname{Lip}}<\infty$.
Write $q_{\epsilon}=\frac{q+\epsilon}{1+\epsilon}$ such that
$W_{\epsilon}=-\log q_{\epsilon}+W$. Hence,

$\displaystyle
W^{\prime\prime}_{\epsilon}=W^{\prime\prime}-\frac{q_{\epsilon}^{\prime\prime}}{q_{\epsilon}}+\frac{q^{\prime}_{\epsilon}\otimes
q^{\prime}_{\epsilon}}{q_{\epsilon}^{2}}=W^{\prime\prime}-\frac{q^{\prime\prime}}{q+\epsilon}+\frac{q^{\prime}\otimes
q^{\prime}}{(q+\epsilon)^{2}}.$

Using the fact that $q+\epsilon$ is lower bounded by $\epsilon$ and that all
involved functions $W^{\prime\prime},q^{\prime},q^{\prime\prime}$ and
$q+\epsilon$ are bounded and have bounded Lipschitz constant, it follows that
$\pi_{\epsilon}$ satisfies (P1). Moreover, by the assumptions on $\pi$ and
$W$, we get

$\displaystyle\pi^{\prime}_{\epsilon}=\frac{q^{\prime}e^{-W}+(q+\epsilon)W^{\prime}e^{-W}}{1+\epsilon}=\frac{q^{\prime}e^{-W}+\epsilon
W^{\prime}e^{-W}}{1+\epsilon},$

and

$\displaystyle\int\frac{\|\pi_{\epsilon}^{\prime}\|^{2}}{\pi_{\epsilon}}\,d\lambda\leq\frac{2}{1+\epsilon}\int\Big{(}\frac{\|q^{\prime}\|^{2}e^{-W}}{q+\epsilon}+\|W^{\prime}\|^{2}e^{-W}\epsilon\Big{)}\,d\lambda<\infty.$

Moreover, using again that $W$ is constant on the support of $\pi$, we get

$\displaystyle\mathcal{J}_{\pi_{\epsilon}}$
$\displaystyle=\frac{1}{1+\epsilon}\int_{\mathbb{R}^{d}}\Big{(}\frac{(q^{\prime}\otimes
q^{\prime})e^{-W}}{q+\epsilon}+\epsilon(W^{\prime}\otimes
W^{\prime})e^{-W}\Big{)}\,d\lambda$
$\displaystyle\to\int_{\mathbb{R}^{d}}\frac{(q^{\prime}\otimes
q^{\prime})e^{-W}}{q}\,d\lambda=\int\frac{\pi^{\prime}\otimes\pi^{\prime}}{\pi}\,d\lambda=\mathcal{J}_{\pi}\qquad\text{as
}\epsilon\rightarrow 0,$

where we applied the dominated convergence theorem in the last step.∎

We are now ready to prove Theorem 6.1.

proof. We first consider the case where $\pi$ satisfies (P1). We assume that
the Bayes risk on the left-hand side in (6.1) is finite because otherwise the
result is trivial. Since the location model is an example of an equivariant
statistical model (with translation group acting on parameter space and sample
space), we can apply Lemma 6.1 to $\psi(\theta)=\theta$, which yields that for
any $\theta_{1},\dots,\theta_{d}\in\mathbb{R}^{d}$,

$\displaystyle\inf_{\hat{\theta}_{n}}\int_{\mathbb{R}^{d}}\mathbb{E}_{\theta}\|\hat{\theta}_{n}-\theta\|^{2}\,\Pi_{\delta}(d\theta)$
(6.3) $\displaystyle\geq\frac{\big{(}\sum_{j=1}^{d}\langle
e_{j},\theta_{j}\rangle\big{)}^{2}}{\sum_{j=1}^{d}\big{(}\chi^{2}(P^{\otimes
n}_{\theta_{j}},P^{\otimes
n}_{0})+\chi^{2}(\Pi_{\delta,\theta_{j}},\Pi_{\delta})+\chi^{2}(P^{\otimes
n}_{\theta_{j}},P^{\otimes
n}_{0})\chi^{2}(\Pi_{\delta,\theta_{j}},\Pi_{\delta})\big{)}},$

where $e_{1},\dots,e_{d}$ is the standard basis in $\mathbb{R}^{d}$ and
$\Pi_{\delta,\theta_{j}}$ is the probability measure with density
$\delta^{-d}\pi(\delta^{-1}(\theta+\theta_{j}))$, $\theta\in\mathbb{R}^{d}$.
Let us now apply a limiting argument. First, we have

$\displaystyle\chi^{2}(P_{\theta_{j}},P_{0})$
$\displaystyle=\mathbb{E}e^{2V(\xi)-2V(\xi-\theta_{j})}-1$ $\displaystyle\leq
e^{L\|\theta_{j}\|^{3}}\mathbb{E}e^{2\langle
V^{\prime}(\xi),\theta_{j}\rangle-\langle
V^{\prime\prime}(\xi)\theta_{j},\theta_{j}\rangle}-1,$ (6.4)

where we used Proposition 3.1 in the inequality. If we set $\theta_{j}=th_{j}$
with $t>0$ and $h_{j}\in\mathbb{R}^{d}$, and then combine (6.4) with Lemma 3.1
and Assumption 1, we get

$\displaystyle\limsup_{t\rightarrow
0}\frac{1}{t^{2}}\chi^{2}(P_{th_{j}},P_{0})\leq-\mathbb{E}\langle
V^{\prime\prime}(\xi)h_{j},h_{j}\rangle+2\mathbb{E}\langle
V^{\prime}(\xi),h_{j}\rangle^{2}=\langle h_{j},\mathcal{I}h_{j}\rangle$

and thus also

$\displaystyle\limsup_{t\rightarrow 0}\frac{1}{t^{2}}\chi^{2}(P^{\otimes
n}_{th_{j}},P^{\otimes n}_{0})\leq n\langle h_{j},\mathcal{I}h_{j}\rangle.$

Moreover, since the prior density $\pi$ satisfies (P1), Proposition 3.1 and
Lemma 3.1 are still applicable, and we also have

$\displaystyle\limsup_{t\rightarrow
0}\frac{1}{t^{2}}\chi^{2}(\Pi_{\delta,th_{j}},\Pi_{\delta})\leq\langle
h_{j},\delta^{-2}\mathcal{J}_{\pi}h_{j}\rangle.$

Substituting these formulas into (6.3), we get for every
$h_{1},\dots,h_{d}\in\mathbb{R}^{d}$,

$\displaystyle\inf_{\hat{\theta}_{n}}\int_{\mathbb{R}^{d}}\mathbb{E}_{\theta}\|\hat{\theta}_{n}-\theta\|^{2}\,\Pi_{\delta}(d\theta)\geq\frac{\big{(}\sum_{j=1}^{d}\langle
e_{j},h_{j}\rangle\big{)}^{2}}{\sum_{j=1}^{d}\langle
h_{j},(n\mathcal{I}+\delta^{-2}\mathcal{J}_{\pi})h_{j}\rangle}.$

Setting $h_{j}=(n\mathcal{I}+\delta^{-2}\mathcal{J}_{\pi})^{-1}e_{j}$, $j\leq
d$, we arrive at

$\displaystyle\inf_{\hat{\theta}_{n}}\int_{\mathbb{R}^{d}}\mathbb{E}_{\delta\theta}\|\hat{\theta}_{n}-\delta\theta\|^{2}\,\pi(\theta)d\theta=\inf_{\hat{\theta}_{n}}\int_{\mathbb{R}^{d}}\mathbb{E}_{\theta}\|\hat{\theta}_{n}-\theta\|^{2}\,\Pi_{\delta}(d\theta)$
$\displaystyle\geq\sum_{j=1}^{d}\langle
e_{j},(n\mathcal{I}+\delta^{-2}\mathcal{J}_{\pi})^{-1}e_{j}\rangle=\operatorname{tr}((n\mathcal{I}+\delta^{-2}\mathcal{J}_{\pi})^{-1}).$
(6.5)

It remains to extend (6.5) to all densities $\pi$ satisfying (P2). To this
end, we apply Lemma 6.2 to get $\pi_{\epsilon}$, $\epsilon>0$, from (6.2)
satisfying (P1) and $\lim_{\epsilon\rightarrow
0}\mathcal{J}_{\pi_{\epsilon}}=\mathcal{J}_{\pi}$. Note also that $qp=\pi.$
Therefore,

$\displaystyle\inf_{\hat{\theta}_{n}}\int_{\mathbb{R}^{d}}\mathbb{E}_{\delta\theta}\|\hat{\theta}_{n}-\delta\theta\|^{2}\,\pi(\theta)d\theta=\lim_{\epsilon\to
0}\inf_{\hat{\theta}_{n}}\int_{\mathbb{R}^{d}}\mathbb{E}_{\delta\theta}\|\hat{\theta}_{n}-\delta\theta\|^{2}\,(\pi(\theta)+\epsilon
p(\theta))d\theta$ $\displaystyle=\lim_{\epsilon\to
0}\frac{1}{1+\epsilon}\inf_{\hat{\theta}_{n}}\int_{\mathbb{R}^{d}}\mathbb{E}_{\delta\theta}\|\hat{\theta}_{n}-\delta\theta\|^{2}\,(q(\theta)p(\theta)+\epsilon
p(\theta))d\theta$ $\displaystyle=\lim_{\epsilon\to
0}\inf_{\hat{\theta}_{n}}\int_{\mathbb{R}^{d}}\mathbb{E}_{\delta\theta}\|\hat{\theta}_{n}-\delta\theta\|^{2}\,\pi_{\epsilon}(\theta)d\theta\geq\lim_{\epsilon\to
0}\operatorname{tr}((n\mathcal{I}+\delta^{-2}\mathcal{J}_{\pi_{\epsilon}})^{-1})$
$\displaystyle=\operatorname{tr}((n\mathcal{I}+\delta^{-2}\mathcal{J}_{\pi})^{-1}),$

where we applied (6.5) to $\pi_{\epsilon}.$

∎

We now turn to the estimation of functionals of the location parameter. For a
continuous function $g:\mathbb{R}^{d}\rightarrow\mathbb{R}^{d}$ and
$x_{0}\in\mathbb{R}^{d}$, the local continuity modulus of $g$ at point $x_{0}$
is defined by

$\displaystyle\omega_{g}(x_{0},\delta)=\sup_{\|x-x_{0}\|\leq\delta}\|g(x)-g(x_{0})\|,\
\delta\geq 0.$

######  Theorem 6.2.

Let $f:\mathbb{R}^{d}\mapsto\mathbb{R}$ be a continuously differentiable
function and let $\theta_{0}\in\mathbb{R}^{d}$. Let
$\pi:\mathbb{R}\mapsto{\mathbb{R}}_{+}$ be a probability density function
satisfying (P2) for $d=1$. Suppose that

$\displaystyle\mathcal{J}_{\pi}=\int_{\mathbb{R}}\frac{(\pi^{\prime}(s))^{2}}{\pi(s)}\,ds<\infty.$

Then there exists $v\in\mathbb{R}^{d}$ with $\|v\|=1$, such that, for every
$\delta>0$,

$\displaystyle\inf_{\hat{T}_{n}}$
$\displaystyle\Big{(}\int_{\mathbb{R}}n\mathbb{E}_{\theta_{0}+sv}(\hat{T}_{n}-f(\theta_{0}+sv))^{2}\,\Pi_{\delta}(ds)\Big{)}^{1/2}$
$\displaystyle\geq\|\mathcal{I}^{-1/2}f^{\prime}(\theta_{0})\|-\sqrt{\frac{\mathcal{J}_{\pi}}{\delta^{2}n}}\|\mathcal{I}^{-1}f^{\prime}(\theta_{0})\|-\int_{\mathbb{R}}\omega_{\mathcal{I}^{-1/2}f^{\prime}}(\theta_{0},|s|)\,\Pi_{\delta}(ds),$

where $\Pi_{\delta}$ is the prior distribution with density
$\delta^{-1}\pi(\delta^{-1}s)$, $s\in\mathbb{R}$ and
$\hat{T}_{n}=\hat{T}_{n}(X_{1},\dots,X_{n})$ is an arbitrary estimator based
on $(X_{1},\dots,X_{n}).$

proof. Without loss of generality we may assume that $\theta_{0}=0.$ Our goal
is to apply Lemma 6.1 to $\psi=f$ and to the one-dimensional subgroup
$G=\mathbb{R}v=\\{sv:s\in\mathbb{R}\\}$ with direction $v\in\mathbb{R}^{d}$,
$\|v\|=1$, to be determined later. As in the proof of Theorem 6.1, we first
establish a lower bound for densities $\pi=e^{-W}$ satisfying (P1) with $d=1$
and for the special case that $f$ and $f^{\prime}$ are bounded on
$\mathbb{R}v.$ In this case, applying Lemma 6.1, we get for every
$t\in\mathbb{R}$,

$\displaystyle\inf_{\hat{T}_{n}}\int_{\mathbb{R}}\mathbb{E}_{sv}(\hat{T}_{n}-f(sv))^{2}\,\Pi_{\delta}(ds)$
$\displaystyle\geq\frac{\big{(}\int_{\mathbb{R}}(f((s-t)v)-f(sv))\,\Pi_{\delta}(ds)\big{)}^{2}}{\chi^{2}(P^{\otimes
n}_{tv},P^{\otimes
n}_{0})+\chi^{2}(\Pi_{\delta,t},\Pi_{\delta})+\chi^{2}(P^{\otimes
n}_{tv},P^{\otimes n}_{0})\chi^{2}(\Pi_{\delta,t},\Pi_{\delta})},$ (6.6)

where $\Pi_{\delta,t}$ is the probability measure with density
$\pi_{\delta,t}(s)=\delta^{-1}\pi(\delta^{-1}(s+t))$, $s\in\mathbb{R}$. Now,
using that $\pi$ satisfies (P1), we have

$\displaystyle\limsup_{t\rightarrow 0}\frac{1}{t^{2}}\chi^{2}(P^{\otimes
n}_{tv},P^{\otimes n}_{0})\leq n\langle v,\mathcal{I}v\rangle,$
$\displaystyle\limsup_{t\rightarrow
0}\frac{1}{t^{2}}\chi^{2}(\Pi_{\delta,t},\Pi_{\delta})\leq\delta^{-2}\mathcal{J}_{\pi}=\delta^{-2}\int_{\mathbb{R}}W^{\prime\prime}(s)e^{-W(s)}\,ds,$

as shown in the proof of Theorem 6.1. Moreover, using that $f$ and
$f^{\prime}$ are bounded on $\mathbb{R}v,$ standard results on the
differentiation of integrals where the integrand depends on a real parameter
(e.g. [3, Corollary 5.9]) yield

$\displaystyle\lim_{t\rightarrow
0}\frac{1}{t}\int_{\mathbb{R}}(f((s-t)v)-f(sv))\,\Pi_{\delta}(ds)=-\int_{\mathbb{R}}\langle
f^{\prime}(sv),v\rangle\,\Pi_{\delta}(ds).$

Substituting these formulas into (6.3) and letting $t$ go to zero, we get

$\displaystyle\inf_{\hat{T}_{n}}\int_{\mathbb{R}}\mathbb{E}_{sv}(\hat{T}_{n}-f(sv))^{2}\,\Pi_{\delta}(ds)$
$\displaystyle\geq\frac{\big{(}\int_{\mathbb{R}}\langle
f^{\prime}(sv),v\rangle\,\Pi_{\delta}(ds)\big{)}^{2}}{n\langle
v,\mathcal{I}v\rangle+\delta^{-2}\mathcal{J}_{\pi}}.$ (6.7)

While this inequality holds for all densities $\pi$ satisfying (P1), we can
apply Lemma 6.2 to extend this inequality to all probability densities
satisfying (P2) (see the proof of Theorem 6.1 for the detailed argument).
Moreover, for densities $\pi$ with bounded support, we can also drop the
boundedness conditions on $f$ and $f^{\prime}$. In fact, the latter can be
achieved by applying (6.7) to a functional $g$ with $g,g^{\prime}$ bounded on
$\mathbb{R}v$ and $g=f$ on the support of $\Pi_{\delta}$ (times $v$). As a
consequence, under the assumptions of Theorem 6.2, we have for every
$v\in\mathbb{R}$, $\|v\|=1$,

$\displaystyle\inf_{\hat{T}_{n}}\int_{\mathbb{R}}\mathbb{E}_{sv}(\hat{T}_{n}-f(sv))^{2}\,\Pi_{\delta}(ds)$
$\displaystyle\geq\frac{\big{(}\int_{\mathbb{R}}\langle
f^{\prime}(sv),v\rangle\,\Pi_{\delta}(ds)\big{)}^{2}}{n\langle
v,\mathcal{I}v\rangle+\delta^{-2}\mathcal{J}_{\pi}}=\frac{1}{n}\frac{\big{(}\int_{\mathbb{R}}\langle
f^{\prime}(sv),v\rangle\,\Pi_{\delta}(ds)\big{)}^{2}}{\langle
v,(\mathcal{I}+\frac{\mathcal{J}_{\pi}}{\delta^{2}n}I_{d})v\rangle}.$

Choosing

$\displaystyle v=\frac{A^{-1}f^{\prime}(0)}{\|A^{-1}f^{\prime}(0)\|},\qquad
A=\mathcal{I}+\frac{\mathcal{J}_{\pi}}{\delta^{2}n}I_{d},$

we obtain

$\displaystyle\inf_{\hat{T}_{n}}\Big{(}\int_{\mathbb{R}}n\mathbb{E}_{sv}(\hat{T}_{n}-f(sv))^{2}\,\Pi_{\delta}(ds)\Big{)}^{1/2}$
$\displaystyle\geq\frac{|\int_{\mathbb{R}}\langle
A^{-1/2}f^{\prime}(sv),A^{-1/2}f^{\prime}(0)\rangle\,\Pi_{\delta}(ds)|}{\|A^{-1/2}f^{\prime}(0)\|}.$

Using the inequality

$\displaystyle|\langle A^{-1/2}f^{\prime}(sv),A^{-1/2}f^{\prime}(0)\rangle|$
$\displaystyle\geq\|A^{-1/2}f^{\prime}(0)\|^{2}-\|A^{-1/2}f^{\prime}(0)\|\|A^{-1/2}f^{\prime}(0)-A^{-1/2}f^{\prime}(sv)\|$
$\displaystyle\geq\|A^{-1/2}f^{\prime}(0)\|(\|A^{-1/2}f^{\prime}(0)\|-\omega_{A^{-1/2}f^{\prime}}(0,|s|)),$

we arrive at

$\displaystyle\inf_{\hat{T}_{n}}\Big{(}\int_{\mathbb{R}}n\mathbb{E}_{sv}(\hat{T}_{n}-f(sv))^{2}\,\Pi_{\delta}(ds)\Big{)}^{1/2}$
$\displaystyle\geq\|A^{-1/2}f^{\prime}(0)\|-\int_{\mathbb{R}}\omega_{A^{-1/2}f^{\prime}}(0,|s|)\,d\Pi_{\delta}(s).$
(6.8)

Since
$A^{-1}=\mathcal{I}^{-1}-\frac{\mathcal{J}_{\pi}}{\delta^{2}n}A^{-1}\mathcal{I}^{-1}\succeq
0$ and ${\mathcal{I}}^{-2}\succeq A^{-1}\mathcal{I}^{-1},$ we have

$\displaystyle\|A^{-1/2}f^{\prime}(0)\|$ $\displaystyle=\langle
A^{-1}f^{\prime}(0),f^{\prime}(0)\rangle^{1/2}$
$\displaystyle=(\langle{\mathcal{I}}^{-1}f^{\prime}(0),f^{\prime}(0)\rangle-\langle\frac{\mathcal{J}_{\pi}}{\delta^{2}n}A^{-1}\mathcal{I}^{-1}f^{\prime}(0),f^{\prime}(0)\rangle)^{1/2}$
$\displaystyle\geq\langle{\mathcal{I}}^{-1}f^{\prime}(0),f^{\prime}(0)\rangle^{1/2}-\langle\frac{\mathcal{J}_{\pi}}{\delta^{2}n}A^{-1}\mathcal{I}^{-1}f^{\prime}(0),f^{\prime}(0)\rangle^{1/2}$
$\displaystyle\geq\|\mathcal{I}^{-1/2}f^{\prime}(0)\|-\sqrt{\frac{\mathcal{J}_{\pi}}{\delta^{2}n}}\|\mathcal{I}^{-1}f^{\prime}(0)\|.$

Note also that $A^{-1}\preceq{\mathcal{I}}^{-1},$ implying
$\|A^{-1}u\|\leq\|{\mathcal{I}}^{-1}u\|,u\in{\mathbb{R}}^{d}$ and, as a
consequence,
$\omega_{A^{-1/2}f^{\prime}}(0,|s|)\leq\omega_{\mathcal{I}^{-1/2}f^{\prime}}(0,|s|).$
These bounds along with (6.8) imply the claim of the theorem. ∎

Finally, we prove Proposition 2.3.

proof. Let us choose
$\pi(s)=\frac{3}{4}\cos^{3}(\theta)I_{[-\frac{\pi}{2},\frac{\pi}{2}]}(\theta)$
in which case (P2) holds and $\mathcal{J}_{\pi}=\frac{9}{2}$. Choosing
additionally $\delta=\frac{2c}{\pi\sqrt{n}}$, $c>0$, Theorem 6.2 yields

$\displaystyle\inf_{\hat{T}_{n}}\sup_{\|\theta-\theta_{0}\|\leq\frac{c}{\sqrt{n}}}(n\mathbb{E}_{\theta}(\hat{T}_{n}-f(\theta))^{2})^{1/2}$
$\displaystyle\geq\|\mathcal{I}^{-1/2}f^{\prime}(\theta_{0})\|-\frac{3\pi}{\sqrt{8}c}\|\mathcal{I}^{-1}f^{\prime}(\theta_{0})\|-\omega_{\mathcal{I}^{-1/2}f^{\prime}}\Big{(}\theta_{0},\frac{c}{\sqrt{n}}\Big{)}.$
(6.9)

Under Assumption 1,
$\|\mathcal{I}^{-1}f^{\prime}(\theta_{0})\|\leq\frac{1}{\sqrt{m}}\|\mathcal{I}^{-1/2}f^{\prime}(\theta_{0})\|.$
In addition, for $f\in C^{s},$ where $s=1+\rho,$ $\rho\in(0,1],$

$\displaystyle\omega_{\mathcal{I}^{-1/2}f^{\prime}}\Big{(}\theta_{0},\frac{c}{\sqrt{n}}\Big{)}\leq\frac{1}{\sqrt{m}}\omega_{f^{\prime}}\Big{(}\theta_{0},\frac{c}{\sqrt{n}}\Big{)}\leq\frac{1}{\sqrt{m}}\|f\|_{C^{s}}\Bigl{(}\frac{c}{\sqrt{n}}\Bigr{)}^{\rho}.$

Recalling that
$\|\mathcal{I}^{-1/2}f^{\prime}(\theta_{0})\|=\sigma_{f}(\theta_{0}),$ bound
(6) implies

$\displaystyle\inf_{\hat{T}_{n}}\sup_{\|\theta-\theta_{0}\|\leq\frac{c}{\sqrt{n}}}\frac{\sqrt{n}\|\hat{T}_{n}-f(\theta)\|_{L_{2}({\mathbb{P}}_{\theta})}}{\sigma_{f}(\theta_{0})}$
$\displaystyle\geq
1-\frac{3\pi}{\sqrt{8m}c}-\frac{1}{\sqrt{m}}\frac{\|f\|_{C^{s}}}{\sigma_{f}(\theta_{0})}\Bigl{(}\frac{c}{\sqrt{n}}\Bigr{)}^{\rho}.$
(6.10)

Note that, for all $\theta$ satisfying
$\|\theta-\theta_{0}\|\leq\frac{c}{\sqrt{n}},$

$\displaystyle|\sigma_{f}(\theta)-\sigma_{f}(\theta_{0})|=|\|\mathcal{I}^{-1/2}f^{\prime}(\theta)\|-\|\mathcal{I}^{-1/2}f^{\prime}(\theta_{0})\||$
$\displaystyle\leq\omega_{\mathcal{I}^{-1/2}f^{\prime}}\Big{(}\theta_{0},\frac{c}{\sqrt{n}}\Big{)}\leq\frac{1}{\sqrt{m}}\|f\|_{C^{s}}\Bigl{(}\frac{c}{\sqrt{n}}\Bigr{)}^{\rho}.$

Therefore,

$\displaystyle\sup_{\|\theta-\theta_{0}\|\leq\frac{c}{\sqrt{n}}}\frac{\sqrt{n}\|\hat{T}_{n}-f(\theta)\|_{L_{2}({\mathbb{P}}_{\theta})}}{\sigma_{f}(\theta_{0})}$
$\displaystyle\leq\sup_{\|\theta-\theta_{0}\|\leq\frac{c}{\sqrt{n}}}\frac{\sqrt{n}\|\hat{T}_{n}-f(\theta)\|_{L_{2}({\mathbb{P}}_{\theta})}}{\sigma_{f}(\theta)}\sup_{\|\theta-\theta_{0}\|\leq\frac{c}{\sqrt{n}}}\frac{\sigma_{f}(\theta)}{\sigma_{f}(\theta_{0})}$
$\displaystyle\leq\sup_{\|\theta-\theta_{0}\|\leq\frac{c}{\sqrt{n}}}\frac{\sqrt{n}\|\hat{T}_{n}-f(\theta)\|_{L_{2}({\mathbb{P}}_{\theta})}}{\sigma_{f}(\theta)}\Bigl{(}1+\sup_{\|\theta-\theta_{0}\|\leq\frac{c}{\sqrt{n}}}\frac{|\sigma_{f}(\theta)-\sigma_{f}(\theta_{0})|}{\sigma_{f}(\theta_{0})}\Bigr{)}$
$\displaystyle\leq\sup_{\|\theta-\theta_{0}\|\leq\frac{c}{\sqrt{n}}}\frac{\sqrt{n}\|\hat{T}_{n}-f(\theta)\|_{L_{2}({\mathbb{P}}_{\theta})}}{\sigma_{f}(\theta)}\Bigl{(}1+\frac{1}{\sqrt{m}}\frac{\|f\|_{C^{s}}}{\sigma_{f}(\theta_{0})}\Bigl{(}\frac{c}{\sqrt{n}}\Bigr{)}^{\rho}\Bigr{)}.$

Using this bound together with (6) easily yields

$\displaystyle\sup_{\|\theta-\theta_{0}\|\leq\frac{c}{\sqrt{n}}}\frac{\sqrt{n}\|\hat{T}_{n}-f(\theta)\|_{L_{2}({\mathbb{P}}_{\theta})}}{\sigma_{f}(\theta)}\geq
1-\frac{3\pi}{\sqrt{8m}c}-\frac{2}{\sqrt{m}}\frac{\|f\|_{C^{s}}}{\sigma_{f}(\theta_{0})}\Bigl{(}\frac{c}{\sqrt{n}}\Bigr{)}^{\rho}.$

∎

## References

  * [1] R. Adamczak, W. Bednorz and P. Wolff.  Moment estimates implied by modified log-Sobolev inequalities.  ESAIM Probab. Stat., 2017, 21, 467–494.
  * [2] S. Aida and D. Stroock.  Moment estimates derived from Poincaré and logarithmic Sobolev inequalities.  Math. Res. Lett., 1994, 1, 75–86.
  * [3] R.G. Bartle.  The elements of integration and Lebesgue measure. John Wiley & Sons, Inc., New York, 1995.
  * [4] P. Bickel and Y. Ritov.  Estimating integrated square density derivatives: sharp best order of convergence estimates.  Sankhya, 1988, 50, 381–393.
  * [5] L. Birgé and P. Massart.  Estimation of integral functionals of a density.  Annals of Statistics, 1995, 23, 11-29.
  * [6] S. Bobkov and C. Houdré. Isoperimetric constants for product probability measures. Annals of Probability, 1997, 25, 1, 184-205.
  * [7] S. Brazitikos, A. Giannopoulos, P. Valettas and B.-H. Vritsiou.  Geometry of isotropic convex bodies. American Mathematical Society, Providence, RI, 2014.
  * [8] Y. Chen.  An Almost Constant Lower Bound of the Isoperimetric Coefficient in the KLS Conjecture.  Geom. Funct. Anal., 2021, 31, 34–61.
  * [9] P. Hall.  The Bootstrap and Edgeworth Expansion. Springer-Verlag, New York, 1992.
  * [10] P. Hall and M.A. Martin.  On Bootstrap Resampling and Iteration.  Biometrika, 1988, 75, 4, 661–671.
  * [11] I. A. Ibragimov and R.Z. Khasminskii.  Statistical Estimation: Asymptotic Theory.  Springer-Verlag, New York, 1981.
  * [12] I.A. Ibragimov, A.S. Nemirovski and R.Z. Khasminskii.  Some problems of nonparametric estimation in Gaussian white noise.  Theory of Probab. and Appl., 1987, 31, 391–406.
  * [13] J. Jiao, Y. Han and T. Weissman.  Bias correction with Jackknife, Bootstrap and Taylor Series.  IEEE Transactions on Information Theory, 2020, 66, 7, 4392–4418.
  * [14] V. Koltchinskii.  Asymptotically Efficient Estimation of Smooth Functionals of Covariance Operators.  J. European Mathematical Society, 2021, 23, 3, 765–843.
  * [15] V. Koltchinskii.  Asymptotic Efficiency in High-Dimensional Covariance Estimation.  Proc. ICM 2018, Rio de Janeiro, 2018, vol. 3, 2891–2912.
  * [16] V. Koltchinskii.  Estimation of smooth functionals in high-dimensional models: bootstrap chains and Gaussian approximation.  2020, arXiv:2011.03789.
  * [17] V. Koltchinskii and M. Zhilova.  Efficient estimation of smooth functionals in Gaussian shift models.  Ann. Inst. Henri Poincaré Probab. Stat., 2021, 57, 1, 351–386.
  * [18] V. Koltchinskii and M. Zhilova.  Estimation of Smooth Functionals in Normal Models: Bias Reduction and Asymptotic Efficiency. Annals of Statistics, 2021, to appear. arXiv:1912.08877.
  * [19] V. Koltchinskii and M. Zhilova.  Estimation of smooth functionals of location parameter in Gaussian and Poincaré random shift models. Sankhya, 2021, to appear.
  * [20] M. Ledoux.  The Concentration of Measure Phenomenon.  American Mathematical Society. 2001.
  * [21] B. Levit.  On the efficiency of a class of non-parametric estimates.  Theory of Prob. and applications, 1975, 20(4), 723–740.
  * [22] B. Levit.  Asymptotically efficient estimation of nonlinear functionals.  Probl. Peredachi Inf. (Problems of Information Transmission), 1978, 14(3), 65–72.
  * [23] E. Milman.  On the role of convexity in isoperimetry, spectral gap and concentration.  Invent. Math., 2009, 177(1), 1-43.
  * [24] Y. Miao.  Concentration inequality of maximum likelihood estimator.  Applied Mathematics Letters, 2010, 23,10, 1305–1309.
  * [25] A. Nemirovski.  On necessary conditions for the efficient estimation of functionals of a nonparametric signal which is observed in white noise.  Theory of Probab. and Appl., 1990, 35, 94–103.
  * [26] A. Nemirovski.  Topics in Non-parametric Statistics.  Ecole d’Ete de Probabilités de Saint-Flour.  Lecture Notes in Mathematics, v. 1738, Springer, New York, 2000.
  * [27] E. Rio.  Upper bounds for minimal distances in the central limit theorem.  Ann. Inst. Henri Poincaré Probab. Stat., 2009, 45, 3, 802–817.
  * [28] A.B. Tsybakov.  Introduction to nonparametric estimation.  Springer, New York, 2009.
  * [29] R. Vershynin.  High-Dimensional Probability: An Introduction with Applications in Data Science.  Cambridge University Press, 2018.
  * [30] M. Wahl.  Lower bounds for invariant statistical models with applications to principal component analysis.  Annales de l’Institut Henri Poincaré Probabilités et Statistiques, 2021, to appear. arXiv:2005.06869.