# From robust tests to Bayes-like posterior distributions

Yannick BARAUD  Department of Mathematics, University of Luxembourg Maison du
nombre 6 avenue de la Fonte L-4364 Esch-sur-Alzette Grand Duchy of Luxembourg
<EMAIL_ADDRESS>
(Date: August 27, 2024)

###### Abstract.

In the Bayes paradigm and for a given loss function, we propose the
construction of a new type of posterior distributions, that extends the
classical Bayes one, for estimating the law of an $n$-sample. The loss
functions we have in mind are based on the total variation and Hellinger
distances as well as some ${\mathbb{L}}_{j}$-ones. We prove that, with a
probability close to one, this new posterior distribution concentrates its
mass in a neighbourhood of the law of the data, for the chosen loss function,
provided that this law belongs to the support of the prior or, at least, lies
close enough to it. We therefore establish that the new posterior distribution
enjoys some robustness properties with respect to a possible misspecification
of the prior, or more precisely, its support. For the total variation and
squared Hellinger losses, we also show that the posterior distribution keeps
its concentration properties when the data are only independent, hence not
necessarily i.i.d., provided that most of their marginals or the average of
these are close enough to some probability distribution around which the prior
puts enough mass. The posterior distribution is therefore also stable with
respect to the equidistribution assumption. We illustrate these results by
several applications. We consider the problems of estimating a location
parameter or both the location and the scale of a density in a nonparametric
framework. Finally, we also tackle the problem of estimating a density, with
the squared Hellinger loss, in a high-dimensional parametric model under some
sparsity conditions. The results established in this paper are non-asymptotic
and provide, as much as possible, explicit constants.

###### Key words and phrases:

Bayes procedure – Gibbs estimator – Posterior distribution – Robustness –
Hellinger distance – Total variation distance.

###### 1991 Mathematics Subject Classification:

Primary 62G05, 62G35, 62F35, 62F15

This project has received funding from the European Union’s Horizon 2020
research and innovation programme under grant agreement No 811017

##  1\. Introduction

Observe $n$ i.i.d. random variables $X_{1},\ldots,X_{n}$ with values in a
measurable space $(E,{\mathcal{E}})$ and assume that their common distribution
$P^{\star}$ belongs to a family ${\mathscr{M}}$ of candidate probabilities, or
at least lies close enough to it in a suitable sense. We consider the problem
of estimating $P^{\star}$ from the observation of
${\boldsymbol{X}}=(X_{1},\ldots,X_{n})$ and we evaluate the performance of an
estimator with values in ${\mathscr{M}}$ by means of a given loss function
$\ell:{\mathscr{P}}\times{\mathscr{M}}\to{\mathbb{R}}_{+}$ where
${\mathscr{P}}$ denotes a convex set of probabilities containing $P^{\star}$.

The loss functions we have in mind are based on the Hellinger or the total
variation distances between distributions or the ${\mathbb{L}}_{j}$-ones
between their densities (with respect to some reference measure). We recall
that the total variation distance $\left\|{P-Q}\right\|$ and the squared
Hellinger one $h^{2}(P,Q)$ between two probabilities $P$ and $Q$ on
$(E,{\mathcal{E}})$ are respectively given by the formulas

(1)
$\left\|{P-Q}\right\|=\frac{1}{2}\int_{E}\left|{\frac{dP}{d\mu}-\frac{dQ}{d\mu}}\right|d\mu$

and

(2)
$h^{2}(P,Q)=\frac{1}{2}\int_{E}\left({\sqrt{\frac{dP}{d\mu}}-\sqrt{\frac{dQ}{d\mu}}}\right)^{2}d\mu$

where $\mu$ dominates both $P$ and $Q$, the result being independent of the
choice of $\mu$.

Our approach to solve this estimation problem has a Bayesian flavour since we
endow ${\mathscr{M}}$ with a $\sigma$-algebra ${\mathcal{A}}$ and a
probability measure $\pi$ on $({\mathscr{M}},{\mathcal{A}})$ that plays the
same role as the prior in the Bayes paradigm. Our aim is to design a posterior
distribution $\widehat{\pi}_{{\boldsymbol{X}}}$, solely based on
${\boldsymbol{X}}$ and the choice of $\ell$, that concentrates its mass, with
a probability close to one, on an $\ell$-ball, i.e. a set of the form

${\mathscr{B}}(P^{\star},r)=\left\\{{P\in{\mathscr{M}},\;\ell(P^{\star},P)\leqslant
r}\right\\}\quad\text{with}\quad r>0.$

This means that with a probability close to 1, a point $\widehat{P}$ which is
randomly drawn according to our (random) distribution
$\widehat{\pi}_{{\boldsymbol{X}}}$ is likely to estimate $P^{\star}$ with an
accuracy (with respect to the chosen loss $\ell$) not larger than $r$. Our
objective is to design $\widehat{\pi}_{{\boldsymbol{X}}}$ in such a way that
this concentration property holds for small values of $r$ and under mild
assumptions on $P^{\star}$ and ${\mathscr{M}}$. In particular, we do not want
to assume that $P^{\star}$ belongs to ${\mathscr{M}}$. Instead, we want that
our posterior distribution $\widehat{\pi}_{{\boldsymbol{X}}}$ enjoys some
stability property: if the concentration property we look for is satisfied
when the data $X_{1},\ldots,X_{n}$ are drawn according to a distribution
$P^{\star}$, that typically belongs to our model ${\mathscr{M}}$, we want that
this property be also satisfied when the distribution of $X_{1},\ldots,X_{n}$
is close enough to $P^{\star}$ (for the chosen loss $\ell$). Of special
interest is the choice of $\ell$ given by the total variation or the Hellinger
distances since, as we shall see, the stability of our posterior distribution
for such losses automatically leads to estimators
$\widehat{P}\sim\widehat{\pi}_{{\boldsymbol{X}}}$ that are naturally robust to
model specification, outliers and contamination.

In the literature, many authors have studied the concentration properties of
the classical Bayes posterior distribution on Hellinger balls. We refer to the
pioneer papers by van der Vaart and his co-authors — see for example Ghosal,
Ghosh and van der Vaart (2000). They show that the concentration property
holds as $n$ tends to infinity provided that the prior $\pi$ puts enough mass
on sets of the form
${\mathcal{K}}(P^{\star},{\varepsilon})=\\{P\in{\mathscr{M}},\;K(P^{\star},P)<{\varepsilon}\\}$
where ${\varepsilon}$ is a positive number and $K(P^{\star},P)$ the Kullback-
Leibler divergence between $P^{\star}$ and $P$. This assumption may, however,
be quite restrictive even in the favorable situation where $P^{\star}$ belongs
to the model ${\mathscr{M}}$, since such sets can be empty and the condition
therefore unsatisfied when the probabilities in ${\mathscr{M}}$ are not
equivalent. This is for example the case when ${\mathscr{M}}$ is the set of
all uniform distributions $P_{\theta}$ on $[\theta-1/2,\theta+1/2]$, with
$\theta\in{\mathbb{R}}$, although the problem of estimating
$P^{\star}\in{\mathscr{M}}$ in this setting is quite easy, even in the
Bayesian paradigm. The assumption appears even more restrictive when the
probability $P^{\star}$ does not belong to ${\mathscr{M}}$, hence when the
model is mispecified. When the probabilities in ${\mathscr{M}}$ are equivalent
and the condition satisfied for an element $\overline{P}$ in the model
${\mathscr{M}}$, it no longer holds true for
$P^{\star}=(1-10^{-10})\overline{P}+10^{-10}R$ when $\overline{P}$ does not
dominate the distribution $R$. This means that the condition may be satisfied
for one distribution and not for another even though both distributions are so
close that they are statistically indistinguishable from any $n$-sample of
realistic size.

Unfortunately, strong conditions are necessary to establish the concentration
properties of the classical Bayes posterior distribution. It is well-known
that the instability we have described is inherent of this approach and that
Bayes estimators are not robust in general. Examples illustrating the poor
performance of the Bayes posterior distribution under a misspecification of
the model are given in Jiang and Tanner (2008) and Baraud and Birgé (2020) for
example. This weakness is due to the fact that the posterior distribution is
based on the log-likelihood function which can be unstable when all the data
are not exactly drawn as the model expects. This instability is also
encountered when one uses the maximum likelihood estimator for the same
reason.

In order to obtain the concentration and stability properties we look for, we
replace the classical log-likelihood function in the expression of the
posterior distribution by a more suitable one based on robust test statistics.
Substituting another function to the log-likelihood one is not new in the
literature and leads to what is called quasi-posterior distributions. The
resulting estimators, called quasi-Bayesian estimators or Laplace type
estimators, have been studied by many statisticians among which Chernozhukov
and Hong (2003) and Bissiri et al. (2016) (we also refer to the references
therein). These papers, however, do not address the problem of
misspecification. In contrast, it is addressed in Jiang and Tanner (2008) for
solving the problem of variable selection in the logistic model. The authors
show that the classical Bayesian approach is no longer reliable when the model
is slightly misspecified while their Gibbs posterior distribution performs
well and offers thus a much safer alternative. The problem of estimating a
high-dimensional parameter $\theta\in{\mathbb{R}}^{d}$ under a sparsity
condition was considered in Atchadé (2017). His quasi-posterior distribution
is obtained by replacing the joint density of the data by a more suitable one
and by using some specific prior that forces sparsity. He proves that the so-
defined posterior distribution contracts around the true parameter
$\theta^{\star}$ at rate $\sqrt{(s^{\star}\log d)/n}$ (where $s^{\star}$ is
the number of nonzero coordinates of $\theta^{\star}$) when both $d$ and $n$
tends to infinity, hence not only $n$. A common feature of the papers we have
cited above lies in their asymptotic nature. This is not the case for
Bhattacharya, Pati and Yang (2019) who replaced the likelihood function in the
expression of the posterior distribution by the fractional likelihood (i.e. a
suitable power of the likelihood function). The authors also consider the
situation where the model is possibly misspecified but their result involves
the $\alpha$-divergence which, as the Kullback one, can be infinite even when
the true distribution of the data is close to the model for the total
variation distance or the Hellinger one.

Baraud and Birgé (2020) propose a surrogate to the Bayes posterior
distribution that is called the $\rho-$posterior distribution in reference to
the theory of $\rho-$estimation that was developped in the series of papers
Baraud et al. (2017) and Baraud and Birgé (2018). The $\rho-$posterior
distribution preserves some of the nice features of the classical Bayes one
but also possesses the robustness property we are interested in. The authors
show that their posterior distribution concentrates on a Hellinger ball around
$P^{\star}$ as soon as the prior puts enough mass around a point which is
close enough to $P^{\star}$. However, the $\rho-$posterior distribution is
mainly of theoretical interest because it is difficult to compute. It provides
a benchmark to compare against. These difficulties are linked to the
calculations of some suprema of empirical processes that are involved in the
definition of the $\rho-$posterior distribution. More deceiving is the fact
that the $\rho-$posterior distribution does not seem to lead to any
improvement as compared to the frequentist approach based on
$\rho-$estimation. For a suitable choice of the prior, an estimator based on
the $\rho-$posterior distribution would satisfy similar risks bound as those
established for $\rho-$estimators. As a consequence, $\rho-$Bayes estimators
do not seem to benefit from any gain that would result from a good guess of a
prior as compared to the frequentist approach that presumes nothing.

Closer to our approach are the aggregation methods and PAC-Bayesian techniques
that have been popularized by Olivier Catoni in statistical learning (see
Catoni (2004)). This approach has mainly been applied for the purpose of
empirical risk minimization and statistical learning (see for example Alquier
(2008)). Our aim is to extend these techniques toward a versatile tool that
can solve our Bayes-like estimation problem for various loss functions
simultaneously.

The problem of designing a good estimator of $P^{\star}$ for a given loss
function $\ell$ was solved in the frequentist paradigm in Baraud (2021).
There, the author provides a general framework that enables one to deal with
various loss functions of interest among which the total variation,
1-Wasserstein, Hellinger, and ${\mathbb{L}}_{j}$-losses among others. His
approach relies on the construction of a suitable family of robust tests and
lies in the line of the former work of Le Cam (1973), Birgé (1983) and Birgé
(2006). The aim of the present paper is to transpose this theory from the
frequentist to the Bayesian paradigm. More precisely, we show how to build a
posterior distribution that possesses the property to concentrate it mass
around the true distribution for the chosen loss function $\ell$. If $\ell$ is
the Kullback-Leibler divergence, our construction recovers the classical Bayes
posterior distribution even though this is not the choice we would recommend
for the reasons we have explained before.

For very general models ${\mathscr{M}}$ and priors $\pi$, we prove that our
posterior distribution $\widehat{\pi}_{{\boldsymbol{X}}}$ concentrates on an
$\ell$-ball centered at $P^{\star}$ with radius $r=r(n)$ when $\pi$ puts
enough mass around $P^{\star}$. When $n$ grows to infinity, the rate $r=r(n)$
is usually of minimax order which shows that our approach is optimal when the
model is well-specified. From a non-asymptotic point of view, we show that
$\widehat{\pi}_{{\boldsymbol{X}}}$ keeps its nice concentration properties as
long as $P^{\star}$ remains close enough to an element $P$ in ${\mathscr{M}}$
around which the prior put enough mass, that is, even in the situation where
the model is slightly misspecified. Actually, we establish the stronger result
that even when the data are only independent, and not i.i.d., the above
conclusion remains true for the average $\overline{P}^{\star}$ of their
marginal distributions in place of $P^{\star}$. We therefore show that the
posterior distribution $\widehat{\pi}_{{\boldsymbol{X}}}$ enjoys some
robustness properties with respect to the equidistribution assumption we
started from. These results contrast sharply with the instability of the
classical Bayes posterior distribution we underlined earlier.

Quite surprisingly, the concentration properties that we establish here
require almost no assumption on ${\mathscr{M}}$ and the distribution of the
data (apart from independence). They mostly depend on the choices of the prior
$\pi$ and the loss function $\ell$. For a suitable element $P$ which belongs
to the model ${\mathscr{M}}$ and lies close enough to $P^{\star}$ (or
$\overline{P}^{\star}$ when the data are only independent), these
concentration properties depend on the minimal value of the radius $r$ over
which the log-ratio
$V(P,r)=\log\left[{\pi({\mathscr{B}}(P,2r))/\pi({\mathscr{B}}(P,r))}\right]$
increases at most linearly with $r$. This log-ratio was introduced in Birgé
(2015) for the purpose of analyzing the behaviour of the classical Bayes
posterior distribution. In our Bayes-like paradigm, we show that the behaviour
of the quantities $V(P,r)$ for $P\in{\mathscr{M}}$ and $r>0$ completely
encapsulates the complexity of the model ${\mathscr{M}}$. In particular, no
assumption on the VC nor on the metric dimension of the model ${\mathscr{M}}$
is required. From this point of view, the nature of the results we establish
here is different from those obtained in the frequentist and Bayesian
paradigms in Baraud (2021) and Baraud and Birgé (2020) which do require such
assumptions.

Another difference with Baraud and Birgé (2020) lies in the construction of
the posterior distribution. In the present paper, it does not involve any
suprema of empirical processes but only integrals that can be approximated by
Monte Carlo. Even if these approximations may not be necessarily easy to get,
especially in high dimension, the computation of our posterior distribution
appears more tractable than that proposed in Baraud and Birgé (2020).

We have specified the numerical constants that appear in our main theorems for
the losses given by the total variation and the squared Hellinger distances.
These constants are unfortunately a bit large. However, we do not believe that
these constants are sharp. It appears that it is difficult to obtain sharp
constants in this non-asymptotic setting under no assumption on the model
${\mathscr{M}}$ and the prior $\pi$.

The main theorems are illustrated by examples which are deliberately chosen to
be as general and simple as possible. Our aim is to give a flavour of results
that can be established with our Bayes-like posterior, avoiding as much as
possible the technicalities that would result from the choice of ad-hoc priors
introduced to solve specific problems. Instead, we wish to discuss the
optimality and robustness properties of our construction for solving general
parametric and nonparametric estimation problems in the density framework
under assumptions that we wish to be as weak as possible. These posterior
distributions will therefore provide a benchmark against which to compare.
Their practical implementation will be the subject of future work.

The present paper is organized as follows. We present our statistical setting
in Section 2. We consider there independent but not necessarily i.i.d. data in
order to analyse the behaviour of the posterior distribution with respect to a
possible departure from equidistribution. Our main assumptions on the loss
function are given and commented on in Section 3. In the remaining part of the
paper, we mainly focus on the total variation and squared Hellinger losses.
The construction of the posterior and its properties are presented in Section
4. As already explained in this introduction, the construction relies on
suitable test statistics. In particular, when applied with the log-likelihood
ratios, our construction recovers the classical Bayes-posterior distribution.
We mainly illustrate the properties of our posterior distribution in Section
5. We discuss there how the concentration properties of the posterior is
affected by the choice of the prior and by a possible misspecification of the
model. We also consider the problems of estimating a density in a location-
scale family and a high-dimensional parameter in a parametric model under a
sparsity constraint. We also show how our estimation strategy leads to unusual
rates of convergence for estimating a translation parameter in a non-regular
statistical model. Finally, Section 6 is devoted to the proofs of the main
theorems and Section 7 to the other proofs.

##  2\. The statistical setting

Let ${\boldsymbol{X}}=(X_{1},\ldots,X_{n})$ be an $n$-tuple of independent
random variables with values in a measurable space $(E,{\mathcal{E}})$ and
joint distribution ${\mathbf{P}}^{\star}=\bigotimes_{i=1}^{n}P_{i}^{\star}$.
The probabilities $P_{i}^{\star}$ are assumed to belong to a given convex set
${\mathscr{P}}$ of probability measures on $(E,{\mathcal{E}})$. Even though
this might not be true, we pretend that the $X_{i}$ are i.i.d. and our aim is
to estimate their (presumed) common distribution $P^{\star}$ from the
observation of ${\boldsymbol{X}}$. To do so, we introduce a family
${\mathscr{M}}$ that consists of candidate probabilities or merely finite
signed measures. The reason for considering finite signed measures lies in the
fact that statisticians sometimes estimate probability densities by integrable
functions that are not necessarily densities (but elements of a suitable
linear space for instance). We endow ${\mathscr{M}}$ with a $\sigma$-algebra
${\mathcal{A}}$ and a probability measure $\pi$, that we call a prior, and we
refer to the resulting pair $({\mathscr{M}},\pi)$ as our model. The model
$({\mathscr{M}},\pi)$ plays here a similar role as in the classical Bayes
paradigm. It encapsulates the a priori information that the statistician has
on $P^{\star}$. Nevertheless, we do not assume that $P^{\star}$, if it ever
exists, belongs to ${\mathscr{M}}$ nor that the true marginals $P_{i}^{\star}$
do. We rather assume that the model $({\mathscr{M}},\pi)$ is approximately
correct in the sense that the average distribution

$\overline{P}^{\star}=\frac{1}{n}\sum_{i=1}^{n}P_{i}^{\star}$

is close enough to some point $P$ in ${\mathscr{M}}$ around which the prior
$\pi$ puts enough mass. Note that $\overline{P}^{\star}$ belongs to
${\mathscr{P}}$ since ${\mathscr{P}}$ is convex.

We measure the estimation accuracy by means of a loss function
$\ell:({\mathscr{M}}\cup{\mathscr{P}})\times{\mathscr{M}}\to{\mathbb{R}}_{+}$
that is not constantly equal to 0 in order to avoid trivialities. Even though
$\ell$ may not be a genuine distance in general, we assume that it shares some
similar features and we interpret it as if it were. For this reason, we call
$\ell$-ball (or ball for short) centered at
$P\in{\mathscr{P}}\cup{\mathscr{M}}$ with radius $r>0$ the subset of
${\mathscr{M}}$

${\mathscr{B}}(P,r)=\left\\{{Q\in{\mathscr{M}},\;\ell(P,Q)\leqslant
r}\right\\}.$

Our aim is to built a posterior distribution
$\widehat{\pi}_{{\boldsymbol{X}}}$ on $({\mathscr{M}},{\mathcal{A}})$, hence
depending on our observation ${\boldsymbol{X}}$, which concentrates with a
probability close to 1 on an $\ell$-ball of the form
${\mathscr{B}}(\overline{P}^{\star},r_{n})$ where we wish the value of
$r_{n}>0$ to be small.

Throughout this paper, we use the following notations. For $x\in{\mathbb{R}}$,
$(x)_{+}=\max(x,0)$ and $(x)_{-}=\max\\{-x,0\\}$. The subsets of
${\mathbb{R}}$ or ${\mathbb{R}}^{k}$ will be equipped with their Borel
$\sigma$-algebras. The cardinality of a set $A$ is denoted $|A|$ and the
elements of ${\mathbb{R}}^{k}$ with $k>1$ are denoted with bold letters, e.g.
${\mathbf{x}}=(x_{1},\ldots,x_{k})$ and $\boldsymbol{0}=(0,\ldots,0)$. For
${\mathbf{x}}\in{\mathbb{R}}^{k}$,
$|{\mathbf{x}}|_{\infty}=\max_{i\in\\{1,\ldots,k\\}}|x_{i}|$ while
$\left|{{\mathbf{x}}}\right|$ denotes the Euclidean norm of ${\mathbf{x}}$. By
convention $\inf_{{\varnothing}}=+\infty$. For all suitable functions $f$ on
$(E^{n},{\mathcal{E}}^{\otimes n})$,
${\mathbb{E}}\left[{f({\boldsymbol{X}})}\right]$ means
$\int_{E^{n}}fd{\mathbf{P}}^{\star}$ while for $f$ on $(E,{\mathcal{E}})$,
${\mathbb{E}}_{S}\left[{f(X)}\right]$ denotes the integral $\int_{E}fdS$ with
respect to some measure $S$ on $(E,{\mathcal{E}})$. For $j\in[1,+\infty)$, we
denote by ${\mathscr{L}}_{j}(E,{\mathcal{E}},\mu)$, the set of measurable
functions $f$ on $(E,{\mathcal{E}})$ such that
$\left\|{f}\right\|_{j,\mu}=[\int_{E}|f|^{j}d\mu]^{1/j}<+\infty$. Finally,
$\left\|{f}\right\|_{\infty}=\sup_{x\in E}|f(x)|$ is the supremum norm of a
function $f$ on $E$.

##  3\. Our main assumptions on the loss function

######  Assumption 1.

For all $S\in{\mathscr{P}}\cup{\mathscr{M}}$, the mapping

$\begin{array}[]{l|rcl}\ell(S,\cdot):&({\mathscr{M}},{\mathcal{A}})&\longrightarrow&{\mathbb{R}}_{+}\\\
&P&\longmapsto&\ell(S,P)\end{array}$

is measurable.

Under such an assumption, $\ell$-balls are measurable, i.e. belong to
${\mathcal{A}}$ and the quantities $\pi({\mathscr{B}}(P,r))$ for
$P\in{\mathscr{P}}\cup{\mathscr{M}}$ and $r>0$ are well-defined.

######  Assumption 2.

There exists a positive number $\tau$ such that, for all $S\in{\mathscr{P}}$
and $P,Q\in{\mathscr{M}}$,

(3) $\displaystyle\ell(S,Q)$
$\displaystyle\leqslant\tau\left[{\ell(S,P)+\ell(P,Q)}\right]$ (4)
$\displaystyle\ell(S,Q)$ $\displaystyle\geqslant\tau^{-1}\ell(P,Q)-\ell(S,P).$

When $\ell$ is a genuine distance, inequalities (3) and (4) are satisfied with
$\tau=1$ since they correspond to the triangle inequality. When $\ell$ is the
square of a distance, these inequalities are satisfied with $\tau=2$.

The construction of the posterior distribution not only depends on the prior
$\pi$ but also on the choice of the loss function $\ell$. We assume that it
can be associated to a family
${\mathscr{T}}(\ell,{\mathscr{M}})=\left\\{{t_{(P,Q)},\;(P,Q)\in{\mathscr{M}}^{2}}\right\\}$
of functions on $(E,{\mathcal{E}})$ with properties that are described below.
We shall see that many classical loss functions (among which the total
variation distance, the 1-Wasserstein distance, the squared Hellinger loss,
etc.) can be associated to families ${\mathscr{T}}(\ell,{\mathscr{M}})$
satisfying our Assumption 3.

######  Assumption 3.

The elements $t_{(P,Q)}$ of ${\mathscr{T}}(\ell,{\mathscr{M}})$ satisfy:

  1. (i)

The mapping

$\begin{array}[]{l|rcl}t:&(E\times{\mathscr{M}}\times{\mathscr{M}},{\mathcal{E}}\otimes{\mathcal{A}}\otimes{\mathcal{A}})&\longrightarrow&{\mathbb{R}}\\\
&(x,P,Q)&\longmapsto&t_{(P,Q)}(x)\end{array}$

is measurable.

  2. (ii)

For all $P,Q\in{\mathscr{M}}$, $t_{(P,Q)}=-t_{(Q,P)}$.

  3. (iii)

there exist positive numbers $a_{0},a_{1}$ such that, for all
$S\in{\mathscr{P}}$ and $P,Q\in{\mathscr{M}}$,

(5) ${\mathbb{E}}_{S}\left[{t_{(P,Q)}(X)}\right]\leqslant
a_{0}\ell(S,P)-a_{1}\ell(S,Q).$

  4. (iv)

For all $P,Q\in{\mathscr{M}}$,

$\sup_{x\in E}t_{(P,Q)}(x)-\inf_{x\in E}t_{(P,Q)}(x)\leqslant 1.$

Under assumption (ii), $t_{(P,P)}=0$ and we deduce from (5) that
$(a_{0}-a_{1})\ell(S,P)\geqslant 0$, hence that $a_{0}\geqslant a_{1}$ since
$\ell$ is not constantly equal to 0.

Some families ${\mathscr{T}}(\ell,{\mathscr{M}})$ possess the additional
property given below.

######  Assumption 4.

Additionally to Assumption 3, there exists $a_{2}>0$ such that

  1. (iv)

for all $S\in{\mathscr{P}}$ and $P,Q\in{\mathscr{M}}$,

$\mathop{\rm Var}\nolimits_{S}\left[{t_{(P,Q)}(X)}\right]\leqslant
a_{2}\left[{\ell(S,P)+\ell(S,Q)}\right].$

This assumption is typically satisfied when $\ell$ behaves as the square of a
distance.

In the proposition below we provide families
${\mathscr{T}}(\ell,{\mathscr{M}})$ that do satisfy our requirements for loss
functions $\ell$ of interest. These results have been established in Baraud
(2021) except for the squared Hellinger loss for which we refer to Baraud and
Birgé (2018)[Proposition 3]. The list below is not exhaustive and other losses
can also be considered, especially those which are defined by a variational
formula of the form

$\ell(P,Q)=\sup_{f\in{\mathscr{F}}}\left[{\int_{E}fdP-\int_{E}fdQ}\right]$

where ${\mathscr{F}}$ is a suitable class of bounded functions. We refer to
Baraud (2021) for more details on the way the families
${\mathscr{T}}(\ell,{\mathscr{M}})$ can be obtained from such loss functions
$\ell$.

######  Proposition 1.

The following holds:

  1. (1)

Total variation. Let ${\mathscr{P}}$ be the set of all probability measures on
$(E,{\mathcal{E}})$ and for $P,Q\in{\mathscr{P}}$,
$\ell(P,Q)=\left\|{P-Q}\right\|$ be the total variation loss (TV-loss for
short) between $P$ and $Q$. For any subset ${\mathscr{M}}$ of ${\mathscr{P}}$
dominated by some reference measure $\mu$, the family
${\mathscr{T}}(\ell,{\mathscr{M}})$ of functions $t_{(P,Q)}$ defined for
$P=p\cdot\mu,Q=q\cdot\mu\in{\mathscr{M}}$ by

(6) $t_{(P,Q)}=\frac{1}{2}\left[{1\hskip-2.6pt{\rm
l}_{q>p}-Q(q>p)}\right]-\frac{1}{2}\left[{1\hskip-2.6pt{\rm
l}_{p>q}-P(p>q)}\right]$

satisfies Assumption 2 with $\tau=1$ and Assumption 3 with $a_{0}=3/2$ and
$a_{1}=1/2$.

  2. (2)

Hellinger distance. Let ${\mathscr{P}}$ be the set of all probability measures
on $(E,{\mathcal{E}})$ and $\ell(P,Q)=h^{2}(P,Q)$ the squared Hellinger
distance between the probabilities $P$ and $Q$. Besides, let $\psi$ be the
function defined by

$\begin{array}[]{l|rcl}\psi:&[0,+\infty]&\longrightarrow&[-1,1]\\\
&x&\longmapsto&{\begin{cases}\displaystyle{\frac{x-1}{x+1}}&\text{ if
$x\in[0,+\infty)$}\\\ 1&\text{ if $x=+\infty$.}\end{cases}}\end{array}$

For any subset ${\mathscr{M}}$ of ${\mathscr{P}}$ dominated by some reference
measure $\mu$, the family ${\mathscr{T}}(\ell,{\mathscr{M}})$ of functions
$t_{(P,Q)}$ defined for $P=p\cdot\mu,Q=q\cdot\mu\in{\mathscr{M}}$ by

(7) $t_{(P,Q)}=\frac{1}{2}\psi\left({\sqrt{\frac{q}{p}}}\right)$

(with the conventions $0/0=1$ and $x/0=+\infty$ for all $x>0$) satisfies
Assumption 2 with $\tau=2$ and Assumption 4 with $a_{0}=2$, $a_{1}=3/16$,
$a_{2}=3\sqrt{2}/4$.

  3. (3)

${\mathbb{L}}_{j}$-loss with $1<j<+\infty$. For $j\in(1,+\infty)$, let
${\mathscr{P}}_{j}$ be the set of all finite and signed measures on
$(E,{\mathcal{E}},\mu)$ of the form $P=p\cdot\mu$ with
$p\in{\mathscr{L}}_{j}(E,\mu)\cap{\mathscr{L}}_{1}(E,\mu)$. Let
${\mathscr{P}}$ be the subset that gathers the probability measures that are
in ${\mathscr{P}}_{j}$. For any subset
${\mathscr{M}}=\left\\{{P=p\cdot\mu,\;p\in{\mathcal{M}}}\right\\}$ of
${\mathscr{P}}_{j}$ for which ${\mathcal{M}}$ satisfies for some $R>0$

(8) $\hskip 22.76219pt\left\|{p-q}\right\|_{\infty}\leqslant
R\left\|{p-q}\right\|_{\mu,j}\quad\text{for all $p,q\in{\mathcal{M}}$,}$

we define for $P=p\cdot\mu$ and $Q=q\cdot\mu$ in ${\mathscr{M}}$,

$\hskip
34.1433ptf_{(P,Q)}=\frac{\left({p-q}\right)_{+}^{j-1}-\left({p-q}\right)_{-}^{j-1}}{\left\|{p-q}\right\|_{\mu,j}^{j-1}}\quad\text{when
}P\neq Q\quad\text{and}\quad f_{(P,P)}=0.$

Then, the family ${\mathscr{T}}(\ell,{\mathscr{M}})$ of functions $t_{(P,Q)}$
defined for $P,Q\in{\mathscr{M}}$ by

(9)
$t_{(P,Q)}=\frac{1}{2R^{j-1}}\left[{\int_{E}f_{(P,Q)}\frac{dP+dQ}{2}-f_{(P,Q)}}\right]$

satisfies Assumption 2 with $\tau=1$ and Assumption 3 with
$a_{0}=3/(4R^{j-1})$ and $a_{1}=1/(4R^{j-1})$ for the loss $\ell=\ell_{j}$
with $\ell_{j}(P,Q)=\left\|{p-q}\right\|_{\mu,j}$ for all $P=p\cdot\mu$ and
$Q=q\cdot\mu$ in ${\mathscr{P}}_{j}$.

For the TV-loss, the family of functions ${\mathscr{T}}(\ell,{\mathscr{M}})$
defined by (6) may occasionally satisfy the stronger Assumption 4 for some
specific models ${\mathscr{M}}$, as we shall see in Section 5.2.

When $j=2$, (8) is typically satisfied when ${\mathcal{M}}$ is a subset of a
linear space enjoying good connections between the ${\mathbb{L}}_{2}$ and the
supremum norms. Many finite dimensional linear spaces with good approximation
properties do satisfy such connections (e.g. piecewise polynomials of a fixed
degree on a regular partition of $[0,1]$, trigonometric polynomials on $[0,1)$
etc.). We refer the reader to Birgé and Massart (1998)[Section 3] for
additional examples. The property may also hold for infinite dimensional
linear spaces as proven in Baraud (2021).

##  4\. Construction of the posterior distribution and main results

###  4.1. Construction of the posterior distribution

It relies on two positive numbers $\beta$ and $\lambda$ such that

(10) $\lambda=(1+{c})\beta\quad\text{with}\quad{c}>0\>\text{ satisfying
}\>{c}_{0}=(1+{c})-{c}(a_{0}/a_{1})>0.$

Given $\ell$, ${\mathscr{M}}$ and the family
${\mathscr{T}}(\ell,{\mathscr{M}})$, we set

${\mathbf{T}}({\boldsymbol{X}},P,Q)=\sum_{i=1}^{n}t_{(P,Q)}(X_{i})\quad\text{for
all $P,Q\in{\mathscr{M}}$}$

and define $\widetilde{\pi}_{{\boldsymbol{X}}}(\cdot|P)$ as the probability on
$({\mathscr{M}},{\mathcal{A}},\pi)$ with density

$\frac{d\widetilde{\pi}_{{\boldsymbol{X}}}(\cdot|P)}{d\pi}:Q\mapsto\frac{\exp\left[{\lambda{\mathbf{T}}({\boldsymbol{X}},P,Q)}\right]}{\int_{{\mathscr{M}}}\exp\left[{\lambda{\mathbf{T}}({\boldsymbol{X}},P,Q)}\right]d\pi(Q)}.$

Then, for $P\in{\mathscr{M}}$ we set

$\displaystyle{\mathbf{T}}({\boldsymbol{X}},P)$
$\displaystyle=\int_{{\mathscr{M}}}{\mathbf{T}}({\boldsymbol{X}},P,Q)d\widetilde{\pi}_{{\boldsymbol{X}}}(Q|P)$
$\displaystyle=\int_{{\mathscr{M}}}{\mathbf{T}}({\boldsymbol{X}},P,Q)\frac{\exp\left[{\lambda{\mathbf{T}}({\boldsymbol{X}},P,Q)}\right]d\pi(Q)}{\int_{{\mathscr{M}}}\exp\left[{\lambda{\mathbf{T}}({\boldsymbol{X}},P,Q)}\right]d\pi(Q)}$

and finally define the posterior distribution
$\widehat{\pi}_{{\boldsymbol{X}}}$ on $({\mathscr{M}},{\mathcal{A}},\pi)$ with
density

(11)
$\frac{d\widehat{\pi}_{{\boldsymbol{X}}}}{d\pi}:P\mapsto\frac{\exp\left[{-\beta{\mathbf{T}}({\boldsymbol{X}},P)}\right]}{\int_{{\mathscr{M}}}\exp\left[{-\beta{\mathbf{T}}({\boldsymbol{X}},P)}\right]d\pi(P)}.$

Our Assumption 3-(i) ensures that
$d\widetilde{\pi}_{{\boldsymbol{X}}}(\cdot|P)/d\pi$ is a measurable function
of $({\boldsymbol{X}},P,Q)$ and $d\widehat{\pi}_{{\boldsymbol{X}}}/d\pi$ a
measurable function of $({\boldsymbol{X}},P)$.

Even though we focus on the concentration properties of the posterior
distribution $\widehat{\pi}_{{\boldsymbol{X}}}$, one may alternatively be
interested in some estimators that derives from
$\widehat{\pi}_{{\boldsymbol{X}}}$ and that are obtained by calculating
integrals of the form

$I=\int_{{\mathscr{M}}}F(P)d\widehat{\pi}_{{\boldsymbol{X}}}(P)$

where $F$ is a real-valued $\pi$-integrable function on
$({\mathscr{M}},{\mathcal{A}})$. For typical choices of $F$, $I$ gives the
expected mean, mode or median of the posterior distribution
$\widehat{\pi}_{{\boldsymbol{X}}}$ whenever these quantities make sense. One
may also choose $F:P\mapsto 1\hskip-2.6pt{\rm
l}_{P\in{\mathscr{B}}(P_{0},{\varepsilon})}$ with $P_{0}\in{\mathscr{M}}$ and
${\varepsilon}>0$ in order to compute the (posterior) probability that
$\ell(\widehat{P},P_{0})$ is not larger than ${\varepsilon}$ when
$\widehat{P}\sim\widehat{\pi}_{{\boldsymbol{X}}}$.

Interestingly, the integral $I$ can be approximated by Monte Carlo as follows.
Assume that the prior $\pi$ admits a density of the form $C^{-1}\Pi$ with
respect to a given probability measure $\nu$ where $\Pi$ is a nonnegative
$\nu$-integrable function on $({\mathscr{M}},{\mathcal{A}})$ and
$C=\int_{{\mathscr{M}}}\Pi(P)d\nu(P)>0$ a positive normalizing constant (that
will not be involved in our calculation). Let $P_{1},\ldots,P_{N}$ be an
$N$-sample with distribution $\nu$ and for each $i\in\\{1,\ldots,N\\}$,
$Q_{i}^{(1)},\ldots,Q_{i}^{(N^{\prime})}$ an independent $N^{\prime}$-sample
with the same distribution. We may approximate $I$ by

$\widehat{I}_{N,N^{\prime}}=\sum_{i=1}^{N}F(P_{i})\frac{\exp\left[{-\beta
W_{i,N^{\prime}}(P_{i})}\right]\Pi(P_{i})}{\sum_{i^{\prime}=1}^{N}\exp\left[{-\beta
W_{i^{\prime},N^{\prime}}(P_{i^{\prime}})}\right]\Pi(P_{i^{\prime}})}$

where for all $i\in\\{1,\ldots,N\\}$,

$W_{i,N^{\prime}}(P_{i})=\sum_{j=1}^{N^{\prime}}T({\boldsymbol{X}},P_{i},Q_{i}^{(j)})\frac{\exp\left[{\lambda
T({\boldsymbol{X}},P_{i},Q_{i}^{(j)})}\right]\Pi(Q_{i}^{(j)})}{\sum_{j^{\prime}=1}^{N^{\prime}}\exp\left[{\lambda
T({\boldsymbol{X}},P_{i},Q_{i}^{(j^{\prime})})}\right]\Pi(Q_{i}^{(j^{\prime})})}.$

It is then easy to check that by the law of large numbers,

$\lim_{N\to+\infty}\left[{\lim_{N^{\prime}\to+\infty}\widehat{I}_{N,N^{\prime}}}\right]=I.$

###  4.2. Connection with the classical Bayes posterior distribution

As we shall see now, the classical Bayes posterior distribution is a
particular case of the posterior-type ones we have introduced in the previous
section. We recall that the Kullback-Leibler divergence loss $K(P,Q)$ between
two probabilities $P,Q$ on $(E,{\mathcal{E}})$ is defined by

$K(P,Q)=\begin{cases}\displaystyle{\int_{E}\log\left({\frac{dP}{dQ}}\right)dP}&\text{when
}P\ll Q;\\\ +\infty&\text{otherwise }.\end{cases}$

Let us now consider a family of equivalent probabilities ${\mathscr{M}}$ that
satisfy for some $a>0$ and suitable versions of their densities $dQ/dP$ the
following inequalities

$e^{-a}\leqslant\frac{dP}{dQ}(x)\leqslant e^{a}\quad\text{for all $x\in E$ and
$P,Q\in{\mathscr{M}}$}.$

The loss $\ell(\cdot,\cdot)=K(\cdot,\cdot)$ is then well-defined on
${\mathscr{M}}$ and it follows from Baraud (2021)[Proposition 12] that the
families of functions

${\mathscr{T}}(\ell,{\mathscr{M}})=\left\\{{t_{(P,Q)}=\frac{1}{2a}\log\left({\frac{dQ}{dP}}\right),\;P,Q\in{\mathscr{M}}}\right\\}$

satisfies our Assumptions 3 and 4 with $a_{0}=a_{1}=1/(2a)$ and
$a_{2}=2a/[\tanh(a/2)]$. Note that given $P,Q\in{\mathscr{M}}$, $P\neq Q$, the
test based on the sign of $t_{(P,Q)}$ is the classical likelihood ratio test
between $P$ and $Q$.

If we apply the construction described in Section 4.1 to the family
${\mathscr{T}}(\ell,{\mathscr{M}})$ with an arbitrary choice of $\lambda>0$,
we obtain that for all $P,Q,P_{0}\in{\mathscr{M}}$,
$t_{(P,Q)}=t_{(P_{0},Q)}-t_{(P,P_{0})}$ hence

${\mathbf{T}}({\boldsymbol{X}},P,Q)={\mathbf{T}}({\boldsymbol{X}},P_{0},Q)-{\mathbf{T}}({\boldsymbol{X}},P_{0},P)$

and the function

$\displaystyle
Q\mapsto\frac{\exp\left[{\lambda{\mathbf{T}}({\boldsymbol{X}},P,Q)}\right]}{\int_{{\mathscr{M}}}\exp\left[{\lambda{\mathbf{T}}({\boldsymbol{X}},P,Q)}\right]d\pi(Q)}=\frac{\exp\left[{\lambda{\mathbf{T}}({\boldsymbol{X}},P_{0},Q)}\right]}{\int_{{\mathscr{M}}}\exp\left[{\lambda{\mathbf{T}}({\boldsymbol{X}},P_{0},Q)}\right]d\pi(Q)}$

is the density (with respect to $\pi$) of a probability measure
$\widetilde{\pi}_{{\boldsymbol{X}}}$ which is independent of $P$.
Consequently,

$\displaystyle{\mathbf{T}}({\boldsymbol{X}},P)$
$\displaystyle=\int_{{\mathscr{M}}}{\mathbf{T}}({\boldsymbol{X}},P,Q)d\widetilde{\pi}_{{\boldsymbol{X}}}(Q)$
$\displaystyle=\int_{{\mathscr{M}}}{\mathbf{T}}({\boldsymbol{X}},P_{0},Q)\widetilde{\pi}_{{\boldsymbol{X}}}(Q)-{\mathbf{T}}({\boldsymbol{X}},P_{0},P)$
$\displaystyle=\int_{{\mathscr{M}}}{\mathbf{T}}({\boldsymbol{X}},P_{0},Q)\widetilde{\pi}_{{\boldsymbol{X}}}(Q)-\frac{1}{2a}\sum_{i=1}^{n}\log\left({\frac{dP}{dP_{0}}}\right)(X_{i}).$

Our posterior distribution is then given by

$\displaystyle\frac{d\widehat{\pi}_{{\boldsymbol{X}}}}{d\pi}(P)$
$\displaystyle=\frac{\exp\left[{-\beta{\mathbf{T}}({\boldsymbol{X}},P)}\right]}{\int_{{\mathscr{M}}}\exp\left[{-\beta{\mathbf{T}}({\boldsymbol{X}},P)}\right]d\pi(P)}=\frac{\left[{\prod_{i=1}^{n}(dP/dP_{0})(X_{i})}\right]^{\beta/(2a)}}{\int_{{\mathscr{M}}}\left[{\prod_{i=1}^{n}(dP/dP_{0})(X_{i})}\right]^{\beta/(2a)}d\pi(P)}.$

It recovers the classical Bayes posterior one for the choice $\beta=2a$ while
for other values of $\beta$ it leads to those posterior distributions based on
fractional likelihoods.

Nevertheless, in the present paper we restrict our study to loss functions
that satisfy some triangle-type inequality – see Assumption 2– which excludes
the Kullback-Leibler divergence.

###  4.3. The influence of the prior

The Bayesian paradigm offers the possibility to favour some elements of
${\mathscr{M}}$ as compared to others. In order to evaluate how much the prior
$\pi$ advantages or disadvantages an element $P\in{\mathscr{M}}$, we fix some
number $\gamma>0$ and introduce the set

${\mathcal{R}}(\beta,P)=\left\\{{{r}\geqslant\frac{1}{n\beta
a_{1}},\;\frac{\pi\left({{\mathscr{B}}(P,2{r}^{\prime})}\right)}{\pi\left({{\mathscr{B}}(P,{r}^{\prime})}\right)}\leqslant\exp\left({\gamma
n\beta a_{1}{r}^{\prime}}\right),\text{ for all
${r}^{\prime}\geqslant{r}$}}\right\\}$

with the convention $a/0=+\infty$ for all $a\geqslant 0$. We then define

(12) ${r}_{n}(\beta,P)=\inf{\mathcal{R}}(\beta,P).$

It follows from the definition of ${r}_{n}(\beta,P)$ that

(13) $\displaystyle
0<\pi\left({{\mathscr{B}}(P,2{r})}\right)\leqslant\exp\left({\gamma n\beta
a_{1}{r}}\right)\pi\left({{\mathscr{B}}(P,{r})}\right)\quad\text{for all
}{r}>{r}_{n}(\beta,P).$

Letting ${r}$ decrease to ${r}_{n}(\beta,P)$, we derive that (13) also holds
for ${r}={r}_{n}(\beta,P)$. In particular,
$\pi\left({{\mathscr{B}}(P,r)}\right)>0$ for ${r}={r}_{n}(\beta,P)$.

The connection between the behaviour of the prior $\pi$ in the vicinity of an
element $P\in{\mathscr{M}}$ and the quantity ${r}_{n}(\beta,P)$ can be made as
follows. Clearly, if the prior puts no mass on the $\ell$-ball
${\mathscr{B}}(P,{r})$ then ${r}_{n}(\beta,P)>{r}$ and ${r}_{n}(\beta,P)$ is
therefore large if ${r}$ is large. In the opposite case, if the prior puts
enough mass on ${\mathscr{B}}(P,{r})$ in the sense that

(14) $\pi\left({{\mathscr{B}}(P,{r})}\right)\geqslant\exp\left({-\gamma n\beta
a_{1}{r}}\right),$

then for all ${r}^{\prime}\geqslant{r}$,

$\displaystyle\pi\left({{\mathscr{B}}(P,{r}^{\prime})}\right)$
$\displaystyle\geqslant\exp\left({-\gamma n\beta
a_{1}{r}}\right)\geqslant\exp\left({-\gamma n\beta a_{1}{r}^{\prime}}\right)$
$\displaystyle\geqslant\exp\left({-\gamma n\beta
a_{1}{r}^{\prime}}\right)\pi\left({{\mathscr{B}}(P,2{r}^{\prime})}\right)$

hence,

$\frac{\pi\left({{\mathscr{B}}(P,2{r}^{\prime})}\right)}{\pi\left({{\mathscr{B}}(P,{r}^{\prime})}\right)}\leqslant\exp\left({\gamma
n\beta a_{1}{r}^{\prime}}\right)\quad\text{and
${r}_{n}(\beta,P)\leqslant{r}$.}$

The quantity ${r}_{n}(\beta,P)$ is therefore small if ${r}$ is small. Although
(14) is not equivalent to (13) (it is actually stronger), the previous
arguments provide a partial view on the relationship between $\pi$ and
${r}_{n}$ and conditions to decide whether $P$ is favoured by $\pi$ or not,
according to the size of ${r}_{n}(\beta,P)$.

###  4.4. A first result on the concentration property of the posterior
distribution

Following the previous discussion, when the set

(15)
${\mathscr{M}}(\beta)=\left\\{{P\in{\mathscr{M}},\;{r}_{n}(\beta,P)\leqslant
a_{1}^{-1}\beta}\right\\}$

is nonempty, it gathers the most favoured elements of the model
$({\mathscr{M}},\pi)$ at level $a_{1}^{-1}\beta$. If
$a_{1}^{-1}\beta\geqslant(n\beta a_{1})^{-1}$ or equivalently $\beta\geqslant
1/\sqrt{n}$, the set ${\mathscr{M}}(\beta)$ can alternatively be defined as

(16) $\displaystyle{\mathscr{M}}(\beta)$
$\displaystyle=\left\\{{P\in{\mathscr{M}},\;\sup_{r\geqslant
a_{1}^{-1}\beta}\left[{\frac{1}{\gamma
na_{1}{r}}\log\left({\frac{\pi\left({{\mathscr{B}}(P,2{r})}\right)}{\pi\left({{\mathscr{B}}(P,{r})}\right)}}\right)}\right]\leqslant\beta}\right\\}.$

It is sometimes easier to use this latter form for calculations. The set
${\mathscr{M}}(\beta)$ will play a crucial role in our first result below.

######  Theorem 1.

Assume that the model $({\mathscr{M}},\pi)$ and the loss $\ell$ satisfy
Assumption 1 and 2 and the family ${\mathscr{T}}(\ell,{\mathscr{M}})$
Assumption 3. Let $\gamma<(c_{0}\wedge{c})/(2\tau)$ and let $\beta\geqslant
1/\sqrt{n}$ be chosen in such a way that the set ${\mathscr{M}}(\beta)$
defined by (15) is not empty. Then, the posterior distribution
$\widehat{\pi}_{{\boldsymbol{X}}}$ defined by (11) possesses the following
property. There exists $\kappa_{0}>0$ only depending on ${c},\tau,\gamma$ and
the ratio $a_{0}/a_{1}$ such that, for all $\xi>0$ and distribution
${\mathbf{P}}^{\star}$ with marginals in ${\mathscr{P}}$, with

(17)
${r}=\inf_{P\in{\mathscr{M}}(\beta)}\ell(\overline{P}^{\star},P)+\frac{1}{a_{1}}\left({\beta+\frac{2\xi}{n\beta}}\right).$

In particular,

$\displaystyle{\mathbb{P}}\left[{\widehat{\pi}_{{\boldsymbol{X}}}\left({\leftidx{{}^{\mathsf{c}}}{\\!{{\mathscr{B}}}}{}(\overline{P}^{\star},\kappa_{0}{r})}\right)\geqslant
e^{-\xi/2}}\right]\leqslant 2e^{-\xi/2}.$

The value of $\kappa_{0}$ is given in (88) in the proof. It only depends on
the choice of the loss function $\ell$ (and not on $\pi$). Hence, for a given
loss function $\ell$, $\kappa_{0}$ is a numerical constant.

For the total variation distance we may take $a_{0}=3/2$, $a_{1}=1/2$,
${c}={c}_{0}=1/3$, $\tau=1$ and $\gamma=1/100$. Then (LABEL:eq-thm01) is
satisfied with $\kappa_{0}=220$ and Theorem 1 can be rephrased as follows.

######  Corollary 1.

Assume that $({\mathscr{M}},\pi)$ is a model that satisfies Assumption 1 for
the TV-loss $\ell=\left\|{\cdot}\right\|$. Let $\beta\geqslant 1/\sqrt{n}$ and
$\widehat{\pi}_{{\boldsymbol{X}}}$ the posterior distribution defined by (11)
with ${c}=1/3$ and the family ${\mathscr{T}}(\ell,{\mathscr{M}})$ given in
Proposition 1-(1). Whatever the number $\xi>0$ and the distribution
${\mathbf{P}}^{\star}$, with a probability at least $1-2e^{-\xi/2}$ a
randomized estimator $\widehat{P}$ with distribution
$\widehat{\pi}_{{\boldsymbol{X}}}$ satisfies with a probability at least
$1-e^{-\xi/2}$

$\left\|{\widehat{P}-\overline{P}^{\star}}\right\|\leqslant
220\left\\{{\inf_{P\in{\mathscr{M}}(\beta)}\left\|{P-\overline{P}^{\star}}\right\|+2\left({\beta+\frac{2\xi}{n\beta}}\right)}\right\\}$

where

${\mathscr{M}}(\beta)=\left\\{{P\in{\mathscr{M}},\;\sup_{r\geqslant
2\beta}\left[{\frac{200}{n{r}}\log\left({\frac{\pi\left({{\mathscr{B}}(P,2{r})}\right)}{\pi\left({{\mathscr{B}}(P,{r})}\right)}}\right)}\right]\leqslant\beta}\right\\}.$

Let us now comment on Theorem 1. When the data are truly i.i.d. and the prior
puts enough mass around their common distribution $P^{\star}$, in the sense
that $P^{\star}\in{\mathscr{M}}(\beta)$, then
$r=a_{1}^{-1}[\beta+2\xi/(n\beta)]$. When this ideal situation is not met,
either because the data are not identically distributed or because $P^{\star}$
does not belong to ${\mathscr{M}}(\beta)$, $r$ increases by at most an
additive term of order
$\inf_{P\in{\mathscr{M}}(\beta)}\ell(\overline{P}^{\star},P)$. When this
quantity remains small as compared to $a_{1}^{-1}\beta$, the value of $r$ does
not deteriorate too much as compared to the previous situation.

In the case of the TV-loss, we may write by convexity

$\inf_{P\in{\mathscr{M}}(\beta)}\left\|{P-\overline{P}^{\star}}\right\|\leqslant\inf_{P\in{\mathscr{M}}(\beta)}\left[{\frac{1}{n}\sum_{i=1}^{n}\left\|{P-P_{i}^{\star}}\right\|}\right]$

and the left-hand side is therefore small when there exists
$P\in{\mathscr{M}}(\beta)$ that approximates well most of the marginals of
${\mathbf{P}}^{\star}$. For such a loss, the concentration properties of the
posterior distribution is therefore stable with respect to a possible
misspecification of the model and a departure from the equidistribution
assumption.

In fact, as we shall see in our Example 1, the average distribution
$\overline{P}^{\star}$ may belong to ${\mathscr{M}}(\beta)$ even when none of
the marginals $P_{i}^{\star}$ does. This means that in good situations, the
posterior distribution may concentrates around $\overline{P}^{\star}$, as it
would do in the i.i.d. case when the distribution of the data does belong to
${\mathscr{M}}(\beta)$, even when the data are non-i.i.d. and their marginals
do not belong to ${\mathscr{M}}(\beta)$.

The value of ${r}$ given by (LABEL:eq-thm01) depends on the choice of the
parameter $\beta$. Since the set ${\mathscr{M}}(\beta)$ is increasing with
$\beta$ (for the inclusion), the two terms
$\inf_{P\in{\mathscr{M}}(\beta)}\ell(\overline{P}^{\star},P)$ and
$a_{1}^{-1}\beta$ vary in opposite directions when $\beta$ increases. The set
${\mathscr{M}}(\beta)$ must be large enough to provide a suitable
approximation of $\overline{P}^{\star}$ and therefore include as many elements
of ${\mathscr{M}}$ as possible since $\overline{P}^{\star}$ is unknown, but
$\beta$ must not be too large in order to keep $a_{1}^{-1}\beta$ to a
reasonable size. Practically, we recommend to choose

(18)
$\beta=\beta_{\alpha}=\inf\left\\{{\beta^{\prime}\geqslant\frac{1}{\sqrt{n}},\;\pi\left({{\mathscr{M}}(\beta^{\prime})}\right)\geqslant
1-\alpha}\right\\}$

for some small $\alpha\in(0,1/10)$. In order to specify what would be the
order of magnitude of such a value of $\beta$, and consequently the value of
the radius ${r}$ defined by (17), we first consider the following simple
example below. Another example will be presented in the translation model in
Section 5.1.

######  Example 1.

Assume that ${\mathscr{M}}$ is a parametric model indexed by a suitable
bounded subset $\Theta$ of ${\mathbb{R}}^{D}$, $D\geqslant 1$, and that $\pi$
behaves like the uniform distribution on $\Theta$. More precisely, assume that
for all $P\in{\mathscr{M}}$ and ${r}>0$

$\left({A{r}}\right)^{D}\wedge
1\leqslant\pi\left({{\mathscr{B}}(P,{r})}\right)\leqslant\left({B{r}}\right)^{D}\wedge
1$

for some positive numbers $A\leqslant B$. Note that this assumption implies
that $\pi\left({{\mathscr{B}}(P,A^{-1})}\right)=1$ for all $P\in{\mathscr{M}}$
so that the diameter of the support of $\pi$ is bounded by $2\tau A^{-1}$.
Then,

(19)
$\frac{\pi\left({{\mathscr{B}}(P,2{r})}\right)}{\pi\left({{\mathscr{B}}(P,{r})}\right)}\leqslant\left({\frac{2B}{A}}\right)^{D}\quad\text{for
all }P\in{\mathscr{M}}\quad\text{and}\quad r>0$

which implies that for all $P\in{\mathscr{M}}$

$\sup_{r\geqslant a_{1}^{-1}\beta}\left[{\frac{1}{\gamma
na_{1}{r}}\log\left({\frac{\pi\left({{\mathscr{B}}(P,2{r})}\right)}{\pi\left({{\mathscr{B}}(P,{r})}\right)}}\right)}\right]\leqslant\frac{D}{\gamma
n\beta}\log\left({\frac{2B}{A}}\right)$

and we note that the right-hand side is not larger than $\beta\geqslant
1/\sqrt{n}$ for

(20) $\beta=\sqrt{\frac{D\log(2B/A)}{\gamma n}}\vee\frac{1}{\sqrt{n}}.$

This means that for such a value of $\beta$, which does not depend on the
distribution the data, the set ${\mathscr{M}}(\beta)$ contains
$P\in{\mathscr{M}}$ and since $P$ is arbitrary we obtain that
${\mathscr{M}}(\beta)={\mathscr{M}}$, hence $\pi({\mathscr{M}}(\beta))=1$. We
derive from Theorem 1 that the distribution $\widehat{\pi}_{{\boldsymbol{X}}}$
concentrates on an $\ell$-ball centered at $\overline{P}^{\star}$ with a
radius of order

${r}=\inf_{P\in{\mathscr{M}}}\ell(\overline{P}^{\star},P)+\frac{1}{a_{1}}\left({\sqrt{\frac{D}{n}}+\frac{2\xi}{\sqrt{nD}}}\right).$

In particular we derive, using Proposition 1, that for the TV-loss

${r}=\inf_{P\in{\mathscr{M}}}\left\|{\overline{P}^{\star}-P}\right\|+2\left({\sqrt{\frac{D}{n}}+\frac{2\xi}{\sqrt{nD}}}\right).$

If for each $i\in\\{1,\ldots,n\\}$, $P_{i}^{\star}$ is the uniform
distributions on $[(i-1)/n,i/n]$ and ${\mathscr{M}}$ contains the uniform
distribution on $[0,1]$, ${\mathscr{M}}$ contains $\overline{P}^{\star}$, even
when none of the marginals $P_{i}^{\star}$ belongs to ${\mathscr{M}}$, and we
derive that

${r}=2\left({\sqrt{\frac{D}{n}}+\frac{2\xi}{\sqrt{nD}}}\right).$

The posterior distribution then concentrates around $\overline{P}^{\star}$ at
parametric rate.

If ${\mathscr{M}}$ is now of the form
${\mathscr{M}}=\left\\{{P=p\cdot\mu,\;p\in{\mathcal{M}}}\right\\}$ with
${\mathcal{M}}$ satisfying (8) and $\ell$ is the $\ell_{j}$-loss with
$j\in(1,+\infty)$, we obtain that for any distribution
${\mathbf{P}}^{\star}=\bigotimes_{1=1}^{n}(p_{i}^{\star}\cdot\mu)$ with
$p_{1}^{\star},\ldots,p_{n}^{\star}\in{\mathscr{L}}_{j}(E,{\mathcal{E}},\mu)$,

$r=\inf_{p\in{\mathcal{M}}}\left\|{\frac{1}{n}\sum_{i=1}^{n}p_{i}^{\star}-p}\right\|_{\mu,j}+4R^{j-1}\left({\sqrt{\frac{D}{n}}+\frac{2\xi}{\sqrt{nD}}}\right).$

In this preliminary example, we see that the choice $\beta$ depends on the
dimension of the statistical model ${\mathscr{M}}$ and on the number of
observations $n$. Quite surprisingly, the status of $\beta$ changes
drastically when Assumption 4 is met as we shall see in the next section.

###  4.5. The concentration property of the posterior distribution under
Assumption 4

Let us define the mapping

(21)
$\begin{array}[]{l|rcl}\phi:&(0,+\infty)&\longrightarrow&{\mathbb{R}}_{+}\\\
&z&\longmapsto&\displaystyle{\phi(z)=\frac{2\left({e^{z}-1-z}\right)}{z^{2}}}.\end{array}$

The function $\phi$ is increasing on $(0,+\infty)$ and tends to $1$ when $z$
tends to 0.

######  Theorem 2.

Assume that the model $({\mathscr{M}},\pi)$ and the loss $\ell$ satisfy
Assumption 1 and 2 and the family ${\mathscr{T}}(\ell,{\mathscr{M}})$
Assumption 4. Define

(22) $\displaystyle\overline{c}_{1}$ $\displaystyle={c}_{0}-\beta
a_{2}a_{1}^{-1}\tau^{2}\phi\left[{\beta(1+2{c})}\right](1+2{c}(1+{c}));$ (23)
$\displaystyle\overline{c}_{2}$ $\displaystyle={c}-\beta
a_{2}a_{1}^{-1}\tau^{2}\phi\left[{\beta(1+2{c})}\right]{c}^{2};$ (24)
$\displaystyle\overline{c}_{3}$ $\displaystyle=(2+{c})-\beta
a_{2}a_{1}^{-1}\tau^{2}\phi\left[{\beta(3+2{c})}\right](2+{c})^{2}.$

Let
$\gamma<(\overline{c}_{1}\wedge\overline{c}_{2}\wedge\overline{c}_{3})/(2\tau)$
and $\beta_{0}$ be the value of $\beta$ for which
$\overline{c}_{1}\wedge\overline{c}_{2}\wedge\overline{c}_{3}=0$. Then, for
$\beta\in(0,\beta_{0})$, the posterior distribution
$\widehat{\pi}_{{\boldsymbol{X}}}$ defined by (11) satisfies the following
property. There exists $\kappa_{0}>0$ only depending on
$a_{0},a_{1},a_{2},{c},\tau,\beta$ and $\gamma$ such that, for all $\xi>0$ and
distribution ${\mathbf{P}}^{\star}$ with marginals in ${\mathscr{P}}$,

(25)
$\displaystyle{\mathbb{E}}\left[{\widehat{\pi}_{{\boldsymbol{X}}}\left({\leftidx{{}^{\mathsf{c}}}{\\!{{\mathscr{B}}}}{}(\overline{P}^{\star},\kappa_{0}{r})}\right)}\right]\leqslant
2e^{-\xi}$

with

(26)
${r}=\inf_{P\in{\mathscr{M}}}\left[{\ell(\overline{P}^{\star},P)+{r}_{n}(\beta,P)}\right]+\frac{2\xi}{n\beta
a_{1}}.$

In particular,

$\displaystyle{\mathbb{P}}\left[{\widehat{\pi}_{{\boldsymbol{X}}}\left({\leftidx{{}^{\mathsf{c}}}{\\!{{\mathscr{B}}}}{}(\overline{P}^{\star},\kappa_{0}{r})}\right)\geqslant
e^{-\xi/2}}\right]\leqslant 2e^{-\xi/2}.$

The value of $\kappa_{0}$ is available from the proof (more precisely (101))
as functions of $a_{0},a_{1},a_{2},{c},\tau$ and $\gamma$. Note that the
constraints on ${c},\beta$ and $\gamma$ that are required in our Theorem 2
only depend on $a_{0},a_{1}$ and $a_{2}$, hence on the choice of the loss
function $\ell$, but not on the model $({\mathscr{M}},\pi)$. In particular,
unlike Theorem 1, the value of $\beta$ can be chosen as a universal constant
once the loss function is fixed.

For example, when $\ell=h^{2}$, we know from Proposition 1 that $a_{0}=2$,
$a_{1}=3/16$ and $a_{2}=3\sqrt{2}/4$, and we may take ${c}=1/125$,
$\beta=2\gamma=1/500$, $\tau=2$ and $\kappa_{0}=1694$. Then, Theorem 2 can
then be rephrased as follows.

######  Corollary 2.

Assume that $({\mathscr{M}},\pi)$ is a model that satisfies Assumption 1 for
the square Hellinger loss $\ell=h^{2}$. The posterior distribution
$\widehat{\pi}_{{\boldsymbol{X}}}$ defined by (11) with $\beta=1/500$,
${c}=1/125$ and the family ${\mathscr{T}}(\ell,{\mathscr{M}})$ given in
Proposition 1-(2) satisfies the following property. Whatever the number
$\xi>0$ and the distribution ${\mathbf{P}}^{\star}$, with a probability at
least $1-2e^{-\xi/2}$ a randomized estimator $\widehat{P}$ with distribution
$\widehat{\pi}_{{\boldsymbol{X}}}$ satisfies with a probability at least
$1-e^{-\xi/2}$

$h^{2}\left({\overline{P}^{\star},\widehat{P}}\right)\leqslant
1694\left\\{{\inf_{P\in{\mathscr{M}}}\left[{h^{2}\left({\overline{P}^{\star},P}\right)+{r}_{n}(\beta,P)}\right]+\frac{5334\xi}{n}}\right\\}$

where, for all $P\in{\mathscr{M}}$,

${r}_{n}(\beta,P)=\inf\left\\{{{r}\geqslant\frac{8000}{3n},\;\frac{\pi\left({{\mathscr{B}}(P,2{r}^{\prime})}\right)}{\pi\left({{\mathscr{B}}(P,{r}^{\prime})}\right)}\leqslant\exp\left({\frac{3n{r}^{\prime}}{8.10^{6}}}\right)\quad\text{for
all ${r}^{\prime}\geqslant{r}$}}\right\\}.$

As for the total variation distance, we may write by convexity that

$\inf_{P\in{\mathscr{M}}}h^{2}\left({\overline{P}^{\star},P}\right)\leqslant\inf_{P\in{\mathscr{M}}}\left[{\frac{1}{n}\sum_{i=1}^{n}h^{2}\left({P_{i}^{\star},P}\right)}\right]$

and the left-hand side is small when there exists an element
$P\in{\mathscr{M}}$ that approximate well most of the marginal distribution
$P_{i}^{\star}$ with respect to the Hellinger distance. If for such a $P$, the
quantity ${r}_{n}(\beta,P)$ is small enough, the posterior distribution
concentrates around $\overline{P}^{\star}$ just as it would do if the data
were truly i.i.d. with distribution $P\in{\mathscr{M}}$.

In order to illustrate Corollary 2 and compare it with Corollary 1, let us go
back to the framework of Example 1.

######  Example 2 (Example 1 continued).

Assume that $\ell=h^{2}$ is the squared Hellinger loss and that the constants
${c}$ and $\beta=2\gamma=1/500$ have been chosen as in Corollary 2. We see
that that the right-hand side of (19) is not larger than $\exp(\gamma
na_{1}\beta{r})$ provided that

${r}\geqslant\frac{D\log(2B/A)}{\gamma na_{1}\beta}.$

Since $\gamma<1/2<D\log(2B/A)$, the right-hand side of this latter inequality
is not smaller than $1/(n\beta a_{1})$ and we derive from the definition (12)
of ${r}_{n}(\beta,P)$ that

${r}_{n}(\beta,P)\leqslant\frac{D\log(2B/A)}{\gamma na_{1}\beta}\quad\text{for
all $P\in{\mathscr{M}}$.}$

By applying Corollary 2 we obtain that the posterior distribution
$\widehat{\pi}_{{\boldsymbol{X}}}$ concentrates on an $\ell$-ball with radius
of order $\inf_{P\in{\mathscr{M}}}\ell(\overline{P}^{\star},P)+D/n$, hence at
rate $1/n$ when the data are i.i.d. with distribution
$P^{\star}\in{\mathscr{M}}$.

Applying our Theorem 1 under the only Assumption 3 and ignoring the fact that
for the loss $\ell=h^{2}$ the family ${\mathscr{T}}(\ell,{\mathscr{M}})$
additionally satisfies Assumption 4, would lead to the weaker result that the
posterior distribution concentrates on an $\ell$-ball with radius of order
$\inf_{P\in{\mathscr{M}}}\ell(\overline{P}^{\star},P)+\sqrt{D/n}$, as for the
TV-loss. We conclude that Theorem 2 leads to a stronger result on the
concentration properties of $\widehat{\pi}_{{\boldsymbol{X}}}$ as compared to
Theorem 1 when the loss function $\ell$ satisfies Assumption 4 on the model
${\mathscr{M}}$.

##  5\. Applications

###  5.1. How to choose $\beta$ in Theorem 1 for a translation model?

In this section, we consider the translation model
${\mathscr{M}}=\\{P_{\theta}=p(\cdot-\theta)\cdot\mu,\theta\in{\mathbb{R}}\\}$
associated to a density $p$ on ${\mathbb{R}}$ with respect to the Lebesgue
measure $\mu$. Given a density $q$ on the real line and a scale parameter
$\sigma$, we estimate the location parameter $\theta$ by choosing the prior
$\nu_{\sigma}$ with density
$q_{\sigma}:\theta\mapsto\sigma^{-1}q(\theta/\sigma)$ with respect to $\mu$.
The prior $\pi$ on ${\mathscr{M}}$ is the image of $\nu_{\sigma}$ by the
mapping $\theta\mapsto P_{\theta}$ and we use the total variation distance to
measure the quality of an estimator of $P_{\theta}$.

If we want to apply our Bayesian-like approach to estimate the distribution
$P_{\theta}$, or equivalently $\theta$, we need to choose a suitable value of
$\beta$. Theorem 1 tells us that our posterior distribution concentrates
around $\overline{P}^{\star}$ with a radius of order

$r=\inf_{P\in{\mathscr{M}}(\beta)}\ell(\overline{P}^{\star},P)+\beta$

where ${\mathscr{M}}(\beta)$ is defined by (16). The size of
${\mathscr{M}}(\beta)$ increases with $\beta$ and in Section 4.4 we suggested
to choose $\beta=\beta_{\alpha}$ satisfying (18) for some $\alpha\in(0,1)$. In
this section, we want to be more specific and evaluate the size of a value of
$\beta$ that satisfies (18). Given an interval of the form $I=[-\sigma
t,\sigma t]$ with $t>0$, our aim is to find a value
$\overline{\beta}=\overline{\beta}(\sigma t)$ of $\beta$ for which the set
${\mathscr{M}}(\overline{\beta})$ is large enough to contain the subset
$\\{P_{\theta},\;\theta\in I\\}$, so that

$\pi\left({{\mathscr{M}}(\overline{\beta})}\right)\geqslant\pi\left({\\{P_{\theta},\;\theta\in
I\\}}\right)=\nu_{\sigma}(I)=\nu([-t,t]).$

In particular, if $q$ is symmetric and $t$ is the $(1-\alpha/2)$-quantile of
$q$, we deduce that
$\pi\left({{\mathscr{M}}(\overline{\beta})}\right)\geqslant 1-\alpha$, hence
that $\beta_{\alpha}\leqslant\overline{\beta}$.

We assume the following.

######  Assumption 5.

The density $q$ is positive, symmetric and decreasing on ${\mathbb{R}}_{+}$.
Besides, there exists $L\in(0,+\infty]$ such that the mapping $H$ defined by

(27) $\begin{array}[]{l|rcl}H:&[0,L)&\longrightarrow&[0,1)\\\
&t&\longmapsto&\left\|{P_{t}-P_{0}}\right\|\end{array}$

is bijective.

Under Assumption 5, $H$ is necessarily increasing on $[0,L)$ and we may define
its inverse $G:[0,1)\to[0,L)$. We set

(28) $\overline{\Gamma}=\max\left\\{{\left[{\sup_{0<r\leqslant
1/4}\frac{G(2r)}{G(r)}}\right]q(0),\frac{1}{2G(1/4)}}\right\\}$

and assume that this quantity is finite. Note that $\overline{\Gamma}$ only
depends on the choices of $p$ and $q$. For example, when $p(x)=(1/2)e^{-|x|}$,
$L=+\infty$, $H:t\mapsto 1-\exp\left[{-t/2}\right]$, $G:r\mapsto-2\log(1-r)$
and since the mapping $r\mapsto[G(2r)/G(r)]$ is increasing,

$\overline{\Gamma}=\frac{1}{\log(4/3)}\max\left\\{{q(0)\log
2,\frac{1}{4}}\right\\}.$

If $p:x\mapsto(\alpha/2)(1-|x|)^{-1+\alpha}1\hskip-2.6pt{\rm l}_{|x|<1}$ with
$\alpha>0$, $L=2$, $H:t\mapsto 1-(1-t/2)^{\alpha}$, $G:r\mapsto
2[1-(1-r)^{1/\alpha}]$. Since $G(r)\sim 2r/\alpha$ in a neighbourhood of 0,
the mapping $r\mapsto G(2r)/G(r)$ is continuous on $[0,1/4]$ and therefore
bounded. Given $q(0)$, $\overline{\Gamma}$ is therefore a finite number.

The following result is proven in Section (7.1).

######  Proposition 2.

Let $\gamma\leqslant\log 4$ and $t$ be a $(1-\alpha)$-quantile of $q$ with
$\alpha\leqslant 1/4$. Under Assumption 5, the set ${\mathscr{M}}(\beta)$
contains the subset $\left\\{{P_{\theta},\theta\in[-\sigma t,\sigma
t]}\right\\}$ if

(29)
$\beta\geqslant\overline{\beta}=\sqrt{\frac{1}{n\gamma}\max\left\\{{\log\left({\frac{\overline{\Gamma}\left({\sigma\vee
1}\right)}{q(2t)}}\right),\log 4}\right\\}}$

where $\overline{\Gamma}$ is defined by (28).

Let us now discuss this result. For a given prior $\nu_{\sigma}$, we see that
$\overline{\beta}$ is of order $1/\sqrt{n}$ and by choosing
$\beta=\overline{\beta}$, our posterior distribution
$\widehat{\pi}_{{\boldsymbol{X}}}$ concentrates around $\overline{P}^{\star}$
with a radius of order

$r=\inf_{P\in{\mathscr{M}}(\overline{\beta})}\ell(\overline{P}^{\star},P)+\frac{C}{\sqrt{n}}\leqslant\inf_{\theta\in[-\sigma
t,\sigma t]}\ell(\overline{P}^{\star},P_{\theta})+\frac{C}{\sqrt{n}}$

with

$C=\sqrt{\frac{1}{\gamma}\max\left\\{{\log\left({\frac{\overline{\Gamma}\left({\sigma\vee
1}\right)}{q(2t)}}\right),\log 4}\right\\}}.$

We note that the larger $I=[-\sigma t,\sigma t]$, the smaller the
approximation term $\inf_{\theta\in[-\sigma t,\sigma
t]}\ell(\overline{P}^{\star},P_{\theta})$. Increasing the value of $\sigma$ or
$t$ to enlarge the interval $I=[-\sigma t,\sigma t]$ has an influence on the
value of $C=C(\sigma,t)$. In the first case, increasing $\sigma$ makes the
prior $\nu_{\sigma}$ flatter and for a fixed value of $t>0$, $C=C(\sigma)$
increases as $\sqrt{\log\sigma}$ when $\sigma$ is larger than 1. In the other
case, for a fixed value of $\sigma$ that determines the prior $\nu_{\sigma}$,
$C=C(t)$ increases as $\sqrt{\log(1/q(2t))}$. When $q$ is the density of a
standard Gaussian random variable, $\sqrt{\log(1/q(2t))}$ is of order $t$,
while for the Laplace and the Cauchy distributions it is of order $\sqrt{t}$
and $\sqrt{\log t}$ respectively.

###  5.2. Fast rates

We go back to the statistical framework described in Section 5.1 in the
special case where $p$ is the density $x\mapsto\alpha
x^{\alpha-1}1\hskip-2.6pt{\rm l}_{(0,1]}$ with $\alpha\in(0,1]$. As before, we
choose the TV-loss. Since $\ell$ is a distance we may take $\tau=1$ in
Assumption 1. Besides, we have seen in Proposition 1 that the family
${\mathscr{T}}({\mathscr{M}},\ell)$ given by (6) satisfies our Assumption 3
with $a_{0}=3/2$ and $a_{1}=1/2$. In fact, it turns out that in this specific
situation the TV-loss also satisfies Assumption 4 with $a_{2}=1$. This means
that from a more statistical point of view, the TV-loss rather behaves here as
the square of a distance. In addition, some simple calculations show that

(30)
$\left\|{P_{\theta}-P_{\theta^{\prime}}}\right\|=\left|{\theta-\theta^{\prime}}\right|^{\alpha}\wedge
1\quad\text{for all $\theta,\theta^{\prime}\in{\mathbb{R}}$.}$

These facts are proven in Baraud (2021) [Examples 5 and 6].

Since Assumption 4 holds true, we may apply Theorem 2 and the reader can check
that the constants ${c}=\beta=0.1$ and $\gamma=0.01$ satisfy the requirements
of this theorem. Besides, one may take in (25) $\kappa_{0}=144$.

To estimate the location parameter $\theta$, we choose a prior
$\nu_{\sigma}=\sigma^{-1}q(\cdot/\sigma)\cdot\mu$ associated to a density $q$
that satisfies the requirements of the first part of Assumption 5 and it
follows from (30) that the second part of Assumption 5 is satisfied with
$L=1$, $G:r\to r^{1/\alpha}$ and

$\overline{\Gamma}=2^{1/\alpha}\max\left\\{{q(0),2^{(1/\alpha)-1}}\right\\}.$

We prove in Section 7.2 the following result.

######  Proposition 3.

Let $t_{0}$ be the third quartile of $\nu_{1}$. If the density $q$ is
positive, symmetric and decreasing on $[0,+\infty)$, for all
$\theta\in{\mathbb{R}}$,
$r_{n}(\beta,P_{\theta})\leqslant\overline{r}_{n}(\beta,P_{\theta})$ with

(31)
$\overline{r}_{n}(\beta,P_{\theta})=\frac{2000}{n}\max\left\\{{\log\left({\frac{\overline{\Gamma}\left({\sigma\vee
1}\right)}{{q\left[{2\left({\frac{|\theta|}{\sigma}\vee
t_{0}}\right)}\right]}}}\right),\log 4}\right\\}.$

By applying Theorem 2, we conclude that for all $\xi>0$, with a probability at
least $1-2e^{-\xi/2}$, the posterior distribution satisfies

$\widehat{\pi}_{{\boldsymbol{X}}}\left({{\mathscr{B}}(\overline{P}^{\star},144r)}\right)\geqslant
1-e^{-\xi/2}$

with

(32)
${r}=\inf_{\theta\in{\mathbb{R}}}\left[{\left\|{\overline{P}^{\star}-P_{\theta}}\right\|+r_{n}(\beta,P_{\theta})}\right]+\frac{40\xi}{n}.$

In particular, when the data are i.i.d. with distribution
$P_{\theta^{\star}}$, with probability close to 1, an element
$P_{\widehat{\theta}}$ drawn according to the posterior distribution
$\widehat{\pi}_{{\boldsymbol{X}}}$ satisfies with a probability close to 1,

$\displaystyle\left|{\theta^{\star}-\widehat{\theta}}\right|^{\alpha}\wedge 1$
$\displaystyle=\left\|{P_{\theta^{\star}}-P_{\widehat{\theta}}}\right\|\leqslant\frac{C(\xi,\alpha,q,\theta^{\star},\sigma)}{n}.$

This inequality implies, at least for $n$ large enough, that

$\displaystyle\left|{\theta^{\star}-\widehat{\theta}}\right|\leqslant\frac{C^{1/\alpha}(\xi,\alpha,q,\theta^{\star},\sigma)}{n^{1/\alpha}},$

which means that the parameter $\theta^{\star}$ is estimated at rate
$n^{-1/\alpha}$. This rate is much faster than the usual
$1/\sqrt{n}$-parametric one that is reached by an estimator based on a moment
method for instance. Since $p$ is unbounded, note that the maximum likelihood
estimator for $\theta^{\star}$ does not exist and is therefore useless.

It is well-known, mainly from the work of Le Cam, that it is impossible to
estimate a distribution in a translation model at a rate faster than $1/n$ for
the TV-loss. Because of (30), the rate we get is not only optimal for
estimating $P_{\theta^{\star}}$ but also for estimating $\theta^{\star}$ with
respect to the Euclidean distance.

An alternative rate-optimal estimator for estimating $\theta^{\star}$ is that
given by the minimum of the observations. This estimator is unfortunately
obviously non-robust to the presence of an outlier among the sample. Our
construction provides an estimator which possesses the property of being both
rate-optimal and robust.

When the data are i.i.d. with distribution $P_{\theta^{\star}}$, we deduce
from (32) that the concentration radius ${r}$ satisfies

${r}\leqslant\overline{r}_{n}(\beta,P_{\theta^{\star}})+\frac{40\xi}{n}$

with $\overline{r}_{n}(\beta,P_{\theta^{\star}})$ given by (31) and it also
interesting to see how this bound deteriorates under a misspecification of the
prior $\nu_{\sigma}$, i.e. when the size of the parameter $\theta^{\star}$ is
large compared to $\sigma$. When $q$ is Gaussian,
$\overline{r}_{n}(\beta,P_{\theta^{\star}})$ increases by a factor of order
$(\theta^{\star}/\sigma)^{2}$ while for the Laplace and Cauchy distributions
it is of order $|\theta^{\star}|/\sigma$ and $\log(|\theta^{\star}|/\sigma)$
respectively. From these results, we conclude that the Cauchy distribution
possesses some advantages over the other two distributions when little
information is available on the location of the parameter $\theta^{\star}$.

###  5.3. A general result under entropy

In this section, we equip
$(E,{\mathcal{E}})=({\mathbb{R}}^{k},{\mathscr{B}}({\mathbb{R}}^{k}))$ with
the Lebesgue measure $\mu$ and the norm $\left|{\cdot}\right|_{\infty}$. We
consider the TV-loss $\ell$ and the location-scale family

(33)
${\mathscr{M}}=\left\\{{P_{(p,{\mathbf{m}},\sigma)}=\frac{1}{\sigma^{k}}p\left({\frac{\cdot-{\mathbf{m}}}{\sigma}}\right)\cdot\mu,\
p\in{\mathcal{M}}_{0},\;{\mathbf{m}}\in{\mathbb{R}}^{k},\sigma>0}\right\\},$

where ${\mathcal{M}}_{0}$ is a set of densities on
$({\mathbb{R}}^{k},{\mathscr{B}}({\mathbb{R}}^{k}),\mu)$. Our aim is to
estimate the density $p\in{\mathcal{M}}_{0}$, the location parameter
${\mathbf{m}}$ and the scale parameter $\sigma$. We assume that the set of
densities ${\mathcal{M}}_{0}$ satisfies the following conditions.

######  Assumption 6.

Let $\widetilde{D}$ be a continuous nonincreasing mapping from $(0,+\infty)$
to $[1,+\infty)$ such that
$\lim_{\eta\to+\infty}\eta^{-2}\widetilde{D}(\eta)=0$. For all $\eta>0$, there
exists a finite subset ${\mathcal{M}}_{0}[\eta]\subset{\mathcal{M}}_{0}$
satisfying

(34)
$\left|{{\mathcal{M}}_{0}[\eta]}\right|\leqslant\exp\left[{\widetilde{D}(\eta)}\right]$

and for all $p\in{\mathcal{M}}_{0}$, there exists
$\overline{p}\in{\mathcal{M}}_{0}[\eta]$ such that

(35)
$\ell\left({P_{(p,\boldsymbol{0},1)},P_{(\overline{p},\boldsymbol{0},1)}}\right)=\frac{1}{2}\int_{{\mathbb{R}}^{k}}\left|{p-\overline{p}}\right|d\mu\leqslant\eta.$

Besides, we assume that there exist $A,\alpha>0$ such that for all
$p\in{\mathcal{M}}_{0}$, ${\mathbf{m}}\in{\mathbb{R}}^{k}$ and
$\sigma\geqslant 1$,

(36)
$\ell\left({P_{(p,\boldsymbol{0},1)},P_{(p,{\mathbf{m}},\sigma)}}\right)\leqslant\left[{A\left({\left(\left|{\frac{{\mathbf{m}}}{\sigma}}\right|_{\infty}\right)^{\alpha}+\left({1-\frac{1}{\sigma}}\right)^{\alpha}}\right)}\right]\bigwedge
1.$

The first part of Assumption 6, corresponding to inequalities (34) and (35),
aims at measuring the size of the set ${\mathcal{M}}_{0}$ by means of its
entropy. The entropy of a set controls its metric dimension and usually
determines the minimax rate of convergence over it as shown in Birgé (1983).
With the second part of Assumption 6, namely inequality (36), we require some
regularity properties of the TV-loss with respect to the location and scale
parameters. It will be commented on later and we shall see that there is
actually no need for the densities in ${\mathcal{M}}_{0}$ to be smooth to
fulfil such a requirement.

Let us now turn to the choice of our prior. For $\eta,\delta>0$, we define

$\Theta[\eta,\delta]=\left\\{{\left({\overline{p},(1+\delta)^{j_{0}}\delta{\mathbf{j}},(1+\delta)^{j_{0}}}\right),\;(\overline{p},j_{0},{\mathbf{j}})\in{\mathcal{M}}_{0}[\eta]\times{\mathbb{Z}}\times{\mathbb{Z}}^{k}}\right\\}$

and for
$\theta=\theta(\overline{p},j_{0},{\mathbf{j}})\in\Theta[\eta,\delta]$, set

(37)
$L_{\theta}=(k+1)L+\log\left|{{\mathcal{M}}_{0}[\eta]}\right|+2\sum_{i=0}^{k}\log(1+|j_{i}|)$

with $L=\log\left[{(\pi^{2}/3)-1}\right]$. It is not difficult to check that
$\sum_{\theta\in\Theta[\eta,\delta]}e^{-L_{\theta}}=1$, and we may therefore
endow ${\mathscr{M}}$ with the prior $\pi$ defined as

(38) $\pi(\left\\{{P_{\theta}}\right\\})=e^{-L_{\theta}}\quad\text{for all
$\theta\in\Theta[\eta,\delta]$.}$

With such a prior, our posterior distribution
$\widehat{\pi}_{{\boldsymbol{X}}}$ possesses the following properties.

######  Corollary 3.

Let $\xi>0$, $K>1$, $\ell$ be the TV-loss and ${\mathscr{M}}$ the family of
probabilities given by (33) where ${\mathcal{M}}_{0}$ satisfies Assumption 6.
Consider the parameters

(39) $\displaystyle\eta$
$\displaystyle=\eta_{n}=\inf{\mathscr{D}}_{n}\quad\text{with}\quad{\mathscr{D}}_{n}=\left\\{{\eta>0,\;\widetilde{D}(\eta)\leqslant\frac{n\eta^{2}}{24}}\right\\}$
(40) $\displaystyle\delta$
$\displaystyle=\delta_{n}=\left({\frac{\eta_{n}}{2A}}\right)^{1/\alpha},$ (41)
$\displaystyle\beta$
$\displaystyle=\beta_{n}=\frac{1}{2}\left[{K\eta_{n}+2\sqrt{\frac{18.6(k+1)}{n}}}\right]$

and the subset ${\mathscr{M}}_{n}(K)$ of ${\mathscr{M}}$ that gathers the
elements $P_{(p,{\mathbf{m}},\sigma)}$ such that

(42)
$|\log\sigma|\vee\left|{\frac{{\mathbf{m}}}{\sigma}}\right|_{\infty}\leqslant\Lambda_{n}=\exp\left[{\frac{(K^{2}-1)n\eta_{n}^{2}}{48(k+1)}+\log\log(1+\delta_{n})}\right].$

Then, the posterior distribution $\widehat{\pi}_{{\boldsymbol{X}}}$ associated
to the value $\lambda=4\beta/3$, the prior $\pi$ given by (38) and the family
${\mathscr{T}}(\ell,{\mathscr{M}})$ given by (6) possesses the following
property: there exists a numerical constant $\kappa_{0}^{\prime}>0$ such that
for all $\xi>0$,

(43)
${\mathbb{E}}\left[{\widehat{\pi}_{{\boldsymbol{X}}}\left({\leftidx{{}^{\mathsf{c}}}{\\!{{\mathscr{B}}}}{}(\overline{P}^{\star},\kappa_{0}^{\prime}{r}_{n})}\right)}\right]\leqslant
2e^{-\xi}$

with

(44)
${r}_{n}=\inf_{P\in{\mathscr{M}}_{n}(K)}\ell(\overline{P}^{\star},P)+K\eta_{n}+\sqrt{\frac{k+1}{n}}+\frac{\xi}{\sqrt{n}}.$

The radius ${r}_{n}$ is the sum of three main terms, omitting the dependency
with respect to $\xi$. The first one,
$\inf_{P\in{\mathscr{M}}_{n}(K)}\ell(\overline{P}^{\star},P)$, corresponds to
the approximation of $\overline{P}^{\star}$ by an element of ${\mathscr{M}}$
whose location and scale parameters satisfy the constraint (42). The quantity
$\eta_{n}$, involved in the second term, usually corresponds to the minimax
rate for solely estimating a density $p\in{\mathcal{M}}_{0}$. Finally, the
third term $\sqrt{(k+1)/n}$ corresponds to the rate we would get for solely
estimating the location and translation parameters
$({\mathbf{m}},\sigma)\in{\mathbb{R}}^{k+1}$ when the density $p$ is known.

Let us now comment on condition (36). The following result shows that it is
satisfied when the densities in ${\mathcal{M}}_{0}$ are regular enough.

######  Lemma 1.

Assume that the set ${\mathcal{M}}_{0}$ consists of densities $p$ that are
supported on $[0,1]^{k}$, satisfy
$\sup_{p\in{\mathcal{M}}_{0}}\left\|{p}\right\|_{\infty}\leqslant L_{0}$ and

(45)
$\sup_{p\in{\mathcal{M}}_{0}}\left|{p({\mathbf{x}})-p({\mathbf{x}}^{\prime})}\right|\leqslant
L_{1}\left|{{\mathbf{x}}-{\mathbf{x}}^{\prime}}\right|^{\alpha}\quad\text{for
all ${\mathbf{x}},{\mathbf{x}}^{\prime}\in{\mathbb{R}}^{k}$,}$

with constants $L_{0},L_{1}>0$ and $\alpha\in(0,1]$. Then (36) is satisfied
with $A=L_{1}\vee[(1+L_{1}k^{\alpha/2}+L_{0})/2]$.

Nevertheless, inequality (36) may also hold true for families
${\mathcal{M}}_{0}$ of densities which are not smooth, as shown in Lemma 2
below. It makes it possible to consider the following example.

######  Example 3.

We consider here the situation where $k=1$ and ${\mathcal{M}}_{0}$ is the set
of all nonincreasing densities on $[0,1]$ that are bounded by $B>1$. Then,
${\mathscr{M}}$ consists of all the probabilities whose densities are
nonincreasing and supported on intervals $I$ with positive lengths and which
are bounded by $B/\mu(I)$. Birman and Solomjak (1967) proved that
${\mathcal{M}}_{0}$ satisfies Assumption 6 with $\widetilde{D}(\eta)$ of order
$(1/\eta)\vee 1$ (up to some constant that depends on $B$). We deduce from
(39) that $\eta_{n}$ is therefore of order $n^{-1/3}$. Besides, it follows
from Lemma 2 below that (36) is satisfied with $A=B$ and $\alpha=1$. We may
therefore apply Corollary 3. For a value of $K$ large enough compared to 1,
$\Lambda_{n}$ defined by (42) is larger than
$\exp\left[{CK^{2}n^{1/3}}\right]$ for some constant $C>0$ (depending on $A$).
In particular, if $X_{1},\ldots,X_{n}$ are i.i.d. with a density of the form

$x\mapsto
p^{\star}(x)=\frac{1}{\sigma^{\star}}p\left({\frac{x-m^{\star}}{\sigma^{\star}}}\right)$

where $p\in{\mathcal{M}}_{0}$,
$|m^{\star}/\sigma^{\star}|\leqslant\exp\left[{CK^{2}n^{1/3}}\right]$ and

$\exp\left[{-\exp\left[{CK^{2}n^{1/3}}\right]}\right]\leqslant\sigma^{\star}\leqslant\exp\left[{\exp\left[{CK^{2}n^{1/3}}\right]}\right],$

the posterior distribution $\widehat{\pi}_{{\boldsymbol{X}}}$ satisfies for
all $\xi>0$, with a probability at least $1-2e^{-\xi/2}$,
$\widehat{\pi}_{{\boldsymbol{X}}}\left[{{\mathscr{B}}(P^{\star},C^{\prime}n^{-1/3})}\right]\geqslant
1-e^{-\xi/2}$ where the constant $C^{\prime}>0$ only depends on $\xi,K,B$ but
not on $m^{\star}$ and $\sigma^{\star}$. This means that the concentration
properties of $\widehat{\pi}_{{\boldsymbol{X}}}$ hold true over a huge range
of translation and scale parameters ${\mathbf{m}}$ and $\sigma$ when $n$ is
large enough.

######  Lemma 2.

Let $p$ be a nonincreasing density on $(0,+\infty)$. For all $\sigma\geqslant
1$

(46)
$\frac{1}{2}\int_{{\mathbb{R}}}\left|{\frac{1}{\sigma}p\left({\frac{x}{\sigma}}\right)-p(x)}\right|dx\leqslant\left({1-\frac{1}{\sigma}}\right).$

If, furthermore, $p$ is bounded by $B\geqslant 1$, for all $m\in{\mathbb{R}}$,

(47)
$\frac{1}{2}\int_{{\mathbb{R}}}\left|{p(x)-p(x-m)}\right|dx\leqslant(|m|B)\wedge
1.$

In particular, for all $m\in{\mathbb{R}}$ and $\sigma\geqslant 1$,

(48)
$\frac{1}{2}\int_{{\mathbb{R}}}\left|{\frac{1}{\sigma}p\left({\frac{x-m}{\sigma}}\right)-p(x)}\right|dx\leqslant\left[{B\left|{\frac{m}{\sigma}}\right|+\left({1-\frac{1}{\sigma}}\right)}\right]\wedge
1.$

###  5.4. Estimating a parameter under sparsity

Let us consider a family of distributions
${\mathscr{M}}=\left\\{{P_{\boldsymbol{\theta}}=p_{\boldsymbol{\theta}}\cdot\mu,\;\boldsymbol{\theta}\in{\mathbb{R}}^{k}}\right\\}$
that are parametrized by ${\mathbb{R}}^{k}$ where $\mu$ is the Lebesgue
measure and the dimension $k$ is large. We presume, even though this might not
be true, that the data are i.i.d. with a distribution
$P_{\boldsymbol{\theta}^{\star}}\in{\mathscr{M}}$ associated to a parameter
$\boldsymbol{\theta}^{\star}=(\theta_{1}^{\star},\ldots,\theta_{k}^{\star})$
the coordinates of which are all zero except a small number of these. For
$m\subset\\{1,\ldots,k\\}$, $m\neq{\varnothing}$, we introduce the sub-family
${\mathscr{M}}_{m}$ that gathers the distributions
$P_{\boldsymbol{\theta}}\in{\mathscr{M}}$ for which the coordinates of
$\boldsymbol{\theta}=(\theta_{1},\ldots,\theta_{k})$ are all zero except those
with an index $i\in m$. We denote by $\Theta_{m}$ the set of such parameters
so that
${\mathscr{M}}_{m}=\\{P_{\boldsymbol{\theta}},\;\boldsymbol{\theta}\in\Theta_{m}\\}$
for all $m\subset\\{1,\ldots,k\\}$, $m\neq{\varnothing}$.

Throughout this section we consider the squared Hellinger loss and, given some
$R>0$, we assume that there exist $\alpha\in(0,1]$ and a positive number
$B_{k}=B_{k}(R)$ possibly depending on $k$ and $R$ (although we drop the
dependency with respect to $R$), such that for all
$\boldsymbol{\theta},\boldsymbol{\theta}^{\prime}\in{\mathbb{R}}^{k}$ with
$\left|{\boldsymbol{\theta}}\right|_{\infty}\vee\left|{\boldsymbol{\theta}^{\prime}}\right|_{\infty}\leqslant
R$

(49)
$h\left({P_{\boldsymbol{\theta}},P_{\boldsymbol{\theta}^{\prime}}}\right)\leqslant\sqrt{B_{k}}\left|{\boldsymbol{\theta}-\boldsymbol{\theta}^{\prime}}\right|_{\infty}^{\alpha}.$

As a consequence, the mapping $\boldsymbol{\theta}\mapsto
P_{\boldsymbol{\theta}}$ is continuous. We endow ${\mathscr{M}}$ with the
Borel $\sigma$-algebra ${\mathcal{A}}$ associated to the Hellinger distance.
The mapping $Q\mapsto\ell(P,Q)=h^{2}(P,Q)$ is continuous, hence measurable on
$({\mathscr{M}},{\mathcal{A}})$ for all probabilities $P$ on
$(E,{\mathcal{E}})$, and Assumption 1 is satisfied as well as Assumption 2
with $\tau=2$.

Given a nonempty subset $m$ of $\\{1,\ldots,p\\}$, we endow $\Theta_{m}$ with
the uniform distribution $\nu_{m}$ on the cube
$\Theta_{m}(R)=\\{\boldsymbol{\theta}\in\Theta_{m},\;\left|{\boldsymbol{\theta}}\right|_{\infty}\leqslant
R\\}$. This leads to a prior $\pi_{m}=\pi_{m}(R)$ on ${\mathscr{M}}_{m}$
defined as the image of $\nu_{m}$ by the mapping $\boldsymbol{\theta}\mapsto
P_{\boldsymbol{\theta}}$. For $m={\varnothing}$, we set
$\Theta_{{\varnothing}}=\Theta_{{\varnothing}}(R)=\\{\boldsymbol{0}\\}$ and we
endow it with the Dirac mass at $\boldsymbol{0}$ so that
${\mathscr{M}}_{{\varnothing}}=\\{P_{\boldsymbol{0}}\\}$ and
$\pi_{{\varnothing}}$ is the Dirac mass at $P_{\boldsymbol{0}}$. We finally
define our prior $\pi=\pi(R)$ on $({\mathscr{M}},{\mathcal{A}})$ as

(50)
$\pi=\sum_{m\subset\\{1,\ldots,k\\}}e^{-L_{m}}\pi_{m}\quad\text{with}\quad
L_{m}=|m|\log k+k\log\left({1+\frac{1}{k}}\right).$

It is not difficult to check that
$\sum_{m\subset\\{1,\ldots,k\\}}e^{-L_{m}}=1$, hence that $\pi$ is a genuine
probability on $({\mathscr{M}},{\mathcal{A}})$.

The following result holds.

######  Corollary 4.

Let $\ell=h^{2}$ be the squared Hellinger distance, $R$ a positive number and
${\mathscr{M}}=\\{P_{\boldsymbol{\theta}}=p_{\boldsymbol{\theta}}\cdot\mu,\;\boldsymbol{\theta}\in{\mathbb{R}}^{k}\\}$
a statistical model that satisfies (49) with $RB_{k}^{1/(2\alpha)}\geqslant
1$. Assume furthermore that the mapping

$\begin{array}[]{l|rcl}p:&E\times{\mathbb{R}}^{k}&\longrightarrow&{\mathbb{R}}_{+}\\\
&(x,\boldsymbol{\theta})&\longmapsto&p_{\boldsymbol{\theta}}(x)\end{array}$

is measurable. We endow ${\mathscr{M}}$ with the prior $\pi=\pi(R)$ given by
(50) and define the posterior distribution $\widehat{\pi}_{{\boldsymbol{X}}}$
by (11) with $\beta=1/500$, $c=1/125$ and the family
${\mathscr{T}}(\ell,{\mathscr{M}})$ given by (7). Then, there exists a
numerical constant $\kappa_{0}^{\prime}>0$ such that for all distribution
${\mathbf{P}}^{\star}$ and $\xi>0$

${\mathbb{E}}\left[{\widehat{\pi}_{{\boldsymbol{X}}}\left({\leftidx{{}^{\mathsf{c}}}{\\!{{\mathscr{B}}}}{}(\overline{P}^{\star},\kappa_{0}^{\prime}{r})}\right)}\right]\leqslant
2e^{-\xi}$

where

(51)
$r=\inf_{m\subset\\{1,\ldots,k\\}}\left[{\inf_{\boldsymbol{\theta}\in\Theta_{m}(R)}\ell(\overline{P}^{\star},P_{\boldsymbol{\theta}})+\frac{|m|\log\left({2kR(nB_{k})^{1/(2\alpha)}}\right)+\xi}{n}}\right].$

Let us now comment on and illustrate this result.

When $B_{k}$ does not increase faster than a power of $k$, the radius $r$ only
depends logarithmically on the dimension $k$ of the parameter space, as
expected.

Since the mapping

$R\mapsto\sup\left\\{{\frac{h\left({P_{\boldsymbol{\theta}},P_{\boldsymbol{\theta}^{\prime}}}\right)}{\left|{\boldsymbol{\theta}-\boldsymbol{\theta}^{\prime}}\right|_{\infty}^{\alpha}}\left|\;\boldsymbol{\theta},\boldsymbol{\theta}^{\prime}\in{\mathbb{R}}^{k},\boldsymbol{\theta}\neq\boldsymbol{\theta},^{\prime}\;|\boldsymbol{\theta}|_{\infty}\vee\left|{\boldsymbol{\theta}^{\prime}}\right|_{\infty}\leqslant
R\right.}\right\\}$

is nondecreasing at tends to infinity as $R$ becomes large, our condition
$RB_{k}^{1/(2\alpha)}=R[B_{k}(R)]^{1/(2\alpha)}\geqslant 1$ holds for $R$
large enough.

Let us now illustrate the result of Corollary 4 by choosing some specific
models
${\mathscr{M}}=\\{P_{\boldsymbol{\theta}},\;\boldsymbol{\theta}\in{\mathbb{R}}^{k}\\}$.
If $P_{\boldsymbol{\theta}}$ is the Gaussian distribution with mean
$\boldsymbol{\theta}\in{\mathbb{R}}^{k}$ and covariance matrix
$\sigma^{2}I_{k}$ where $I_{k}$ denotes the $k\times k$ identity matrix. Then,

$\displaystyle
h^{2}(P_{\boldsymbol{\theta}},P_{\boldsymbol{\theta}^{\prime}})=1-\exp\left[{-\frac{\left|{\boldsymbol{\theta}-\boldsymbol{\theta}^{\prime}}\right|^{2}}{8\sigma^{2}}}\right]\leqslant\frac{\left|{\boldsymbol{\theta}-\boldsymbol{\theta}^{\prime}}\right|^{2}}{8\sigma^{2}}\leqslant\frac{k\left|{\boldsymbol{\theta}-\boldsymbol{\theta}^{\prime}}\right|_{\infty}^{2}}{8\sigma^{2}}$

and (49) is satisfied with $B_{k}=k/(8\sigma^{2})$ and $\alpha=1$. In
particular, our condition $RB_{k}^{1/(2\alpha)}\geqslant 1$ is equivalent to
$R\geqslant 2\sigma\sqrt{(2/k)}$. In this case, the value of $r$ given by (51)
is of order

$\inf_{m\subset\\{1,\ldots,k\\}}\left[{\inf_{\boldsymbol{\theta}\in\Theta_{m}(R)}\ell(\overline{P}^{\star},P_{\boldsymbol{\theta}})+\frac{|m|\log\left({knR/\sigma}\right)+\xi}{n}}\right].$

More generally, when
${\mathscr{M}}=\\{P_{\boldsymbol{\theta}},\boldsymbol{\theta}\in{\mathbb{R}}^{k}\\}$
is a regular statistical model with Fisher information
${\mathbf{I}}(\boldsymbol{\theta})$ at $\boldsymbol{\theta}$, we know from the
book of Ibragimov and Has’minskiĭ (1981)[Theorem 7.1 p.81] that for all
$\boldsymbol{\theta},\boldsymbol{\theta}^{\prime}\in{\mathbb{R}}^{k}$ such
that
$\left|{\boldsymbol{\theta}}\right|_{\infty}\vee\left|{\boldsymbol{\theta}^{\prime}}\right|_{\infty}\leqslant
R$

$h^{2}(P_{\boldsymbol{\theta}},P_{\boldsymbol{\theta}^{\prime}})\leqslant\frac{\left|{\boldsymbol{\theta}-\boldsymbol{\theta}^{\prime}}\right|^{2}}{8}\sup_{\boldsymbol{\theta}^{\prime\prime}\in{\mathbb{R}}^{k},\left|{\boldsymbol{\theta}^{\prime\prime}}\right|_{\infty}\leqslant
R}{\rm tr}\left({{\mathbf{I}}(\boldsymbol{\theta}^{\prime\prime})}\right).$

Then, Assumption (49) holds with $\alpha=1$ and

$B_{k}=\frac{k^{2}}{8}\sup_{\boldsymbol{\theta}^{\prime\prime}\in{\mathbb{R}}^{k},\left|{\boldsymbol{\theta}^{\prime\prime}}\right|\leqslant
R}\varrho\left({{\mathbf{I}}(\boldsymbol{\theta}^{\prime\prime})}\right)$

where $\varrho\left({{\mathbf{I}}(\boldsymbol{\theta}^{\prime\prime})}\right)$
denotes the largest eigenvalue of the matrix
${\mathbf{I}}(\boldsymbol{\theta}^{\prime\prime})$. It is well known that this
value is independent of $\boldsymbol{\theta}^{\prime\prime}$ when
${\mathscr{M}}$ is a translation model.

##  6\. Proofs of Theorems 1 and 2

Throughout this section we fix some ${\overline{Q}}\in{\mathscr{M}}$,
${r},\beta>0$ and use the following notations: ${c}_{1}=1+{c}$,
${c}_{2}=2+{c}$,

${\mathcal{V}}(\pi,{\overline{Q}})=\left\\{{{r}>0,\pi\left({{\mathscr{B}}({\overline{Q}},{r})}\right)>0}\right\\}$

and for ${r}\in{\mathcal{V}}(\pi,{\overline{Q}})$ ,
${\mathscr{B}}={\mathscr{B}}({\overline{Q}},{r})$ and
$\pi_{{\mathscr{B}}}=\left[{\pi({\mathscr{B}})}\right]^{-1}1\hskip-2.6pt{\rm
l}_{{\mathscr{B}}}\cdot\pi$.

In the proofs of Theorems 1 and 2, we use the following consequence of our
Assumption 3. Since ${\mathscr{P}}$ is convex and the marginal distributions
$P_{1}^{\star},\ldots,P_{n}^{\star}$ belong to ${\mathscr{P}}$ we may write

$\frac{1}{n}\sum_{i=1}^{n}{\mathbb{E}}\left[{t_{(P,Q)}(X_{i})}\right]={\mathbb{E}}_{S}\left[{t_{(P,Q)}(X)}\right]\quad\text{with
$S=\overline{P}^{\star}\in{\mathscr{P}}$}$

and we deduce from (5) that for all $P,Q\in{\mathscr{M}}$,

(52)
$\frac{1}{n}\sum_{i=1}^{n}{\mathbb{E}}\left[{t_{(P,Q)}(X_{i})}\right]\leqslant
a_{0}\ell(\overline{P}^{\star},P)-a_{1}\ell(\overline{P}^{\star},Q).$

Besides, using the anti-symmetry property (ii) we also obtain that

(53)
$\frac{1}{n}\sum_{i=1}^{n}{\mathbb{E}}\left[{t_{(P,Q)}(X_{i})}\right]\geqslant
a_{1}\ell(\overline{P}^{\star},P)-a_{0}\ell(\overline{P}^{\star},Q).$

In the proof of Theorems 2, we also use the following consequence of our
Assumption 4. By taking $S=\overline{P}^{\star}$ and using the convexity of
the mapping $u\mapsto u^{2}$, we deduce that for all $P,Q\in{\mathscr{M}}$

$\displaystyle\frac{1}{n}\sum_{i=1}^{n}\mathop{\rm
Var}\nolimits\left[{t_{(P,Q)}(X_{i})}\right]$
$\displaystyle={\mathbb{E}}_{S}\left[{t_{(P,Q)}^{2}(X)}\right]-\frac{1}{n}\sum_{i=1}^{n}\left({{\mathbb{E}}\left[{t_{(P,Q)}(X_{i})}\right]}\right)^{2}$
$\displaystyle\leqslant{\mathbb{E}}_{S}\left[{t_{(P,Q)}^{2}(X)}\right]-\left({{\mathbb{E}}_{S}\left[{t_{(P,Q)}(X)}\right]}\right)^{2}$
$\displaystyle=\mathop{\rm Var}\nolimits_{S}\left[{t_{(P,Q)}(X)}\right]$

and it derives thus from Assumption 4 (iv) that for all $P,Q\in{\mathscr{M}}$

(54) $\frac{1}{n}\sum_{i=1}^{n}\mathop{\rm
Var}\nolimits\left[{t_{(P,Q)}(X_{i})}\right]\leqslant
a_{2}\left[{\ell(\overline{P}^{\star},P)+\ell(\overline{P}^{\star},Q)}\right].$

###  6.1. Preliminary results

The proofs of our main results rely on the following lemmas.

######  Lemma 3.

Let $(U,V)$ be a pair of random variables with values in a product space
$(E\times F,{\mathcal{E}}\otimes{\mathcal{F}})$ and marginal distributions
$P_{U}$ and $P_{V}$ respectively. For all measurable function $h$ on $(E\times
F,{\mathcal{E}}\otimes{\mathcal{F}})$,

${\mathbb{E}}_{U}\left[{\frac{1}{{\mathbb{E}}_{V}\left[{\exp\left[{-h(U,V)}\right]}\right]}}\right]\leqslant\left[{{\mathbb{E}}_{V}\left[{\frac{1}{{\mathbb{E}}_{U}\left[{\exp\left[{h(U,V)}\right]}\right]}}\right]}\right]^{-1}.$

This lemma is proven in Audibert and Catoni (2011) [Lemma 4.2, P. 28].

######  Lemma 4.

For $P,Q\in{\mathscr{M}}$, we set

${\mathbf{M}}(P,Q)=\log\left[{\int_{{\mathscr{M}}}{\mathbb{E}}\left[{\exp\left[{\beta\left({{c}{\mathbf{T}}({\boldsymbol{X}},P,Q^{\prime})-{c}_{1}{\mathbf{T}}({\boldsymbol{X}},P,Q)}\right)}\right]d\pi(Q^{\prime})}\right]}\right].$

For all ${r}\in{\mathcal{V}}(\pi,{\overline{Q}})$ and $P\in{\mathscr{M}}$,

(55)
$\displaystyle{\mathbb{E}}\left[{\exp\left[{-\beta{\mathbf{T}}({\boldsymbol{X}},P)}\right]}\right]\leqslant\frac{1}{\pi({\mathscr{B}})}\left[{\int_{{\mathscr{B}}}\exp\left[{-{\mathbf{M}}(P,Q)}\right]d\pi_{{\mathscr{B}}}(Q)}\right]^{-1}.$

###### Proof.

Let ${r}\in{\mathcal{V}}(\pi,{\overline{Q}})$. For $P,Q\in{\mathscr{M}}$, we
set

$I({\boldsymbol{X}},P,Q)={c}_{1}\beta{\mathbf{T}}({\boldsymbol{X}},P,Q)-\log\int_{{\mathscr{M}}}\exp\left[{{c}\beta{\mathbf{T}}({\boldsymbol{X}},P,Q^{\prime})}\right]d\pi(Q^{\prime}).$

Then,

$\displaystyle{\mathbb{E}}\left[{\exp\left[{-I({\boldsymbol{X}},P,Q)}\right]}\right]$
$\displaystyle={\mathbb{E}}\left[{\exp\left[{-{c}_{1}\beta{\mathbf{T}}({\boldsymbol{X}},P,Q)+\log\int_{{\mathscr{M}}}\exp\left[{{c}\beta{\mathbf{T}}({\boldsymbol{X}},P,Q^{\prime})}\right]d\pi(Q^{\prime})}\right]}\right]$
$\displaystyle={\mathbb{E}}\left[{\int_{{\mathscr{M}}}\exp\left[{{c}\beta{\mathbf{T}}({\boldsymbol{X}},P,Q^{\prime})-{c}_{1}\beta{\mathbf{T}}({\boldsymbol{X}},P,Q)}\right]d\pi(Q^{\prime})}\right]$
(56) $\displaystyle=\exp\left[{{\mathbf{M}}(P,Q)}\right].$

Since $\lambda={c}_{1}\beta=(1+{c})\beta$, it follows from the convexity of
the exponential that

$\displaystyle{\mathbb{E}}\left[{\exp\left[{-\beta{\mathbf{T}}({\boldsymbol{X}},P)}\right]}\right]$
$\displaystyle={\mathbb{E}}\left[{\exp\left[{\int_{{\mathscr{M}}}[-\beta{\mathbf{T}}({\boldsymbol{X}},P,Q)]d\widetilde{\pi}_{{\boldsymbol{X}}}(Q)}\right]}\right]$
$\displaystyle\leqslant{\mathbb{E}}\left[{{\int_{{\mathscr{M}}}{\exp\left[{-\beta{\mathbf{T}}({\boldsymbol{X}},P,Q)}\right]d\widetilde{\pi}_{{\boldsymbol{X}}}(Q)}}}\right]$
$\displaystyle={\mathbb{E}}\left[{\frac{\int_{{\mathscr{M}}}\exp\left[{{c}\beta{\mathbf{T}}({\boldsymbol{X}},P,Q)}\right]d\pi(Q)}{\int_{{\mathscr{M}}}\exp\left[{{c}_{1}\beta{\mathbf{T}}({\boldsymbol{X}},P,Q)}\right]d\pi(Q)}}\right]$
$\displaystyle\leqslant{\mathbb{E}}\left[{\frac{\int_{{\mathscr{M}}}\exp\left[{{c}\beta{\mathbf{T}}({\boldsymbol{X}},P,Q)}\right]d\pi(Q)}{\int_{{\mathscr{B}}}\exp\left[{{c}_{1}\beta{\mathbf{T}}({\boldsymbol{X}},P,Q)}\right]d\pi(Q)}}\right].$

Hence,

$\displaystyle{\mathbb{E}}\left[{\exp\left[{-\beta{\mathbf{T}}({\boldsymbol{X}},P)}\right]}\right]$
$\displaystyle\leqslant{\mathbb{E}}\left[{\frac{1}{\int_{{\mathscr{B}}}\exp\left[{I({\boldsymbol{X}},P,Q)}\right]d\pi(Q)}}\right]$
$\displaystyle=\frac{1}{\pi({\mathscr{B}})}{\mathbb{E}}\left[{\frac{1}{\int_{{\mathscr{B}}}\exp\left[{I({\boldsymbol{X}},P,Q)}\right]d\pi_{{\mathscr{B}}}(Q)}}\right].$

Applying Lemma 3 with $U={\boldsymbol{X}}$, $V=Q$ with distribution
$\pi_{{\mathscr{B}}}$, and $h(U,V)=-I({\boldsymbol{X}},P,Q)$, we obtain that

$\displaystyle{\mathbb{E}}\left[{\exp\left[{-\beta{\mathbf{T}}({\boldsymbol{X}},P)}\right]}\right]$
$\displaystyle\leqslant\frac{1}{\pi({\mathscr{B}})}\left[{\int_{{\mathscr{B}}}\frac{1}{{\mathbb{E}}\left[{\exp\left[{-I({\boldsymbol{X}},P,Q)}\right]}\right]}d\pi_{{\mathscr{B}}}(Q)}\right]^{-1}$

and (55) follows from (56). ∎

######  Lemma 5.

For $P,Q\in{\mathscr{M}}$, we set

${\mathbf{L}}(P,Q)=\log\int_{{\mathscr{M}}}{\mathbb{E}}\left[{\exp\left[{\beta\left({{c}_{2}{\mathbf{T}}({\boldsymbol{X}},P,Q^{\prime})-{c}_{1}{\mathbf{T}}({\boldsymbol{X}},P,Q)}\right)}\right]}\right]d\pi(Q^{\prime}).$

For all ${r}\in{\mathcal{V}}(\pi,{\overline{Q}})$,

$\displaystyle{\mathbb{E}}\left[{\frac{1}{\int_{{\mathscr{M}}}\exp\left[{-\beta{\mathbf{T}}({\boldsymbol{X}},P)}\right]d\pi(P)}}\right]$
$\displaystyle\leqslant\frac{1}{\pi^{2}({\mathscr{B}})}\left[{\int_{{\mathscr{B}}^{2}}\exp\left[{-{\mathbf{L}}(P,Q)}\right]d\pi_{{\mathscr{B}}}(P)d\pi_{{\mathscr{B}}}(Q)}\right]^{-1}.$

###### Proof.

For $P,Q\in{\mathscr{M}}$, we set

$H({\boldsymbol{X}},P,Q)=\beta{c}_{1}{\mathbf{T}}({\boldsymbol{X}},P,Q)-\log\left[{\int_{{\mathscr{M}}}\exp\left[{{c}_{2}\beta{\mathbf{T}}({\boldsymbol{X}},P,Q^{\prime})}\right]d\pi(Q^{\prime})}\right].$

Then,

$\displaystyle{\mathbb{E}}\left[{\exp\left[{-H({\boldsymbol{X}},P,Q)}\right]}\right]$
$\displaystyle={\mathbb{E}}\left[{\exp\left[{-\beta{c}_{1}{\mathbf{T}}({\boldsymbol{X}},P,Q)}\right]\int_{{\mathscr{M}}}\exp\left[{{c}_{2}\beta{\mathbf{T}}({\boldsymbol{X}},P,Q^{\prime})}\right]d\pi(Q^{\prime})}\right]$
$\displaystyle={\mathbb{E}}\left[{\int_{{\mathscr{M}}}\exp\left[{\beta\left({{c}_{2}{\mathbf{T}}({\boldsymbol{X}},P,Q^{\prime})-{c}_{1}{\mathbf{T}}({\boldsymbol{X}},P,Q)}\right)}\right]d\pi(Q^{\prime})}\right]$
(57) $\displaystyle=\exp\left[{{\mathbf{L}}(P,Q)}\right].$

It follows from the convexity of the exponential and the fact that
$\lambda={c}_{1}\beta$ that for all $P\in{\mathscr{M}}$,

$\displaystyle{\mathbb{E}}\left[{\exp\left[{\beta{\mathbf{T}}({\boldsymbol{X}},P)}\right]}\right]$
$\displaystyle={\mathbb{E}}\left[{\exp\left[{\int_{{\mathscr{M}}}[\beta{\mathbf{T}}({\boldsymbol{X}},P,Q)]d\widetilde{\pi}_{{\boldsymbol{X}}}(Q)}\right]}\right]$
$\displaystyle\leqslant{\mathbb{E}}\left[{{\int_{{\mathscr{M}}}{\exp\left[{\beta{\mathbf{T}}({\boldsymbol{X}},P,Q)}\right]d\widetilde{\pi}_{{\boldsymbol{X}}}(Q)}}}\right]$
$\displaystyle={\mathbb{E}}\left[{\frac{\int_{{\mathscr{M}}}\exp\left[{{c}_{2}\beta{\mathbf{T}}({\boldsymbol{X}},P,Q)}\right]d\pi(Q)}{\int_{{\mathscr{M}}}\exp\left[{{c}_{1}\beta{\mathbf{T}}({\boldsymbol{X}},P,Q)}\right]d\pi(Q)}}\right]$
$\displaystyle={\mathbb{E}}\left[{\frac{1}{\int_{{\mathscr{M}}}\exp\left[{H({\boldsymbol{X}},P,Q)}\right]d\pi(Q)}}\right].$

Applying Lemma 3 with $U={\boldsymbol{X}}$ and $V=Q$ with distribution $\pi$
we obtain that

$\displaystyle{\mathbb{E}}\left[{\exp\left[{\beta{\mathbf{T}}({\boldsymbol{X}},P)}\right]}\right]\leqslant\left[{\int_{{\mathscr{M}}}\frac{1}{{\mathbb{E}}\left[{\exp\left[{-H({\boldsymbol{X}},P,Q)}\right]}\right]}d\pi(Q)}\right]^{-1}.$

We deduce from (57) that for all $P\in{\mathscr{M}}$

$\displaystyle{\mathbb{E}}\left[{\exp\left[{\beta{\mathbf{T}}({\boldsymbol{X}},P)}\right]}\right]$
$\displaystyle\leqslant\left[{\int_{{\mathscr{M}}}\exp\left[{-{\mathbf{L}}(P,Q)}\right]d\pi(Q)}\right]^{-1}$
(58)
$\displaystyle\leqslant\frac{1}{\pi({\mathscr{B}})}\left[{\int_{{\mathscr{B}}}\exp\left[{-{\mathbf{L}}(P,Q)}\right]d\pi_{{\mathscr{B}}}(Q)}\right]^{-1}.$

Applying Lemma 3 with $U={\boldsymbol{X}}$, $V=P$ with distribution $\pi$ and
$h(U,V)=\beta{\mathbf{T}}({\boldsymbol{X}},P)$, gives

$\displaystyle{\mathbb{E}}\left[{\frac{1}{\int_{{\mathscr{M}}}\exp\left[{-\beta{\mathbf{T}}({\boldsymbol{X}},P)}\right]d\pi(P)}}\right]$
$\displaystyle\leqslant\left[{\int_{{\mathscr{M}}}\frac{1}{{\mathbb{E}}\left[{\exp\left[{\beta{\mathbf{T}}({\boldsymbol{X}},P)}\right]}\right]}d\pi(P)}\right]^{-1}$
$\displaystyle\leqslant\frac{1}{\pi({\mathscr{B}})}\left[{\int_{{\mathscr{B}}}\frac{1}{{\mathbb{E}}\left[{\exp\left[{\beta{\mathbf{T}}({\boldsymbol{X}},P)}\right]}\right]}d\pi_{{\mathscr{B}}}(P)}\right]^{-1}$

which together with (58) leads to the result.

∎

The proofs of Theorems 1 and 2 rely on suitable bounds on the Laplace
transforms of sums of independent random variables and on a summation lemma.
These results are presented below.

######  Lemma 6.

For all $\beta\in{\mathbb{R}}$ and random variable $U$ with values in an
interval of length $l\in(0,+\infty)$,

(59) $\log{\mathbb{E}}\left[{\exp\left[{\beta
U}\right]}\right]\leqslant\beta{\mathbb{E}}\left[{U}\right]+\frac{\beta^{2}l^{2}}{8}.$

######  Lemma 7.

Let $U$ be a squared integrable random variable not larger than $b>0$. For all
$\beta>0$,

(60) $\log{\mathbb{E}}\left[{\exp\left[{\beta
U}\right]}\right]\leqslant\beta{\mathbb{E}}\left[{U}\right]+\beta^{2}{\mathbb{E}}\left[{U^{2}}\right]\frac{\phi(\beta
b)}{2},$

where $\phi$ is defined by (21).

The proofs of Lemmas 6 and 7 can be found on pages 21 and 23 in Massart (2007)
(where our function $\phi$ is defined as twice his).

######  Lemma 8.

Let $J\in{\mathbb{N}}$, $\gamma>0$ and ${\overline{Q}}\in{\mathscr{M}}$. If
${r}$ satisfies $n\beta a_{1}{r}\geqslant 1$ and (13), for all
$\gamma_{0}>2\gamma$

$\displaystyle\int_{\leftidx{{}^{\mathsf{c}}}{\\!{{\mathscr{B}}}}{}({\overline{Q}},2^{J}{r})}\exp\left[{-\gamma_{0}n\beta
a_{1}\ell({\overline{Q}},P)}\right]d\pi(P)$ (61)
$\displaystyle\leqslant\pi\left({{\mathscr{B}}}\right)\exp\left[{\Xi-\left({\gamma_{0}-2\gamma}\right)n\beta
a_{1}2^{J}{r}}\right]$

with

$\displaystyle\Xi$
$\displaystyle=-\gamma+\log\left[{\frac{1}{1-\exp\left[{-\left({\gamma_{0}-2\gamma}\right)}\right]}}\right]$

Besides,

(62) $\displaystyle\int_{{\mathscr{M}}}\exp\left[{-\gamma_{0}n\beta
a_{1}\ell({\overline{Q}},P)}\right]d\pi(P)\leqslant\pi({\mathscr{B}})\exp\left[{\Xi^{\prime}}\right]$

with

$\Xi^{\prime}=\log\left[{1+\frac{\exp\left[{-\left({\gamma_{0}-\gamma}\right)}\right]}{1-\exp\left[{-\left({\gamma_{0}-2\gamma}\right)}\right]}}\right].$

###### Proof.

From (13), we deduce by induction that for all $j\geqslant 0$

$\displaystyle\pi\left({{\mathscr{B}}({\overline{Q}},2^{j+1}{r})}\right)$
$\displaystyle\leqslant\exp\left[{\gamma n\beta
a_{1}{r}\sum_{k=0}^{j}2^{k}}\right]\pi\left({{\mathscr{B}}}\right)$
$\displaystyle=\exp\left[{(2^{j+1}-1)\gamma n\beta
a_{1}{r}}\right]\pi\left({{\mathscr{B}}}\right)$

Consequently,

$\displaystyle\int_{\leftidx{{}^{\mathsf{c}}}{\\!{{\mathscr{B}}}}{}({\overline{Q}},2^{J}{r})}\exp\left[{-\gamma_{0}n\beta
a_{1}\ell({\overline{Q}},P)}\right]d\pi(P)$ $\displaystyle=\sum_{j\geqslant
J}\int_{{\mathscr{B}}({\overline{Q}},2^{j+1}{r})\setminus{\mathscr{B}}({\overline{Q}},2^{j}{r})}\exp\left[{-\gamma_{0}\beta
na_{1}\ell({\overline{Q}},P)}\right]d\pi(P)$
$\displaystyle\leqslant\pi\left({{\mathscr{B}}}\right)\sum_{j\geqslant
J}\frac{\pi\left({{\mathscr{B}}({\overline{Q}},2^{j+1}{r})}\right)}{\pi\left({{\mathscr{B}}}\right)}\exp\left[{-\gamma_{0}n\beta
a_{1}2^{j}{r}}\right]$
$\displaystyle\leqslant\pi\left({{\mathscr{B}}}\right)\sum_{j\geqslant
J}\exp\left[{\gamma n\beta a_{1}(2^{j+1}-1){r}-\gamma_{0}n\beta
a_{1}2^{j}{r}}\right]$
$\displaystyle=\pi\left({{\mathscr{B}}}\right)\exp\left[{-\gamma n\beta
a_{1}{r}}\right]\sum_{j\geqslant
J}\exp\left[{-\left({\gamma_{0}-2\gamma}\right)n\beta a_{1}2^{j}{r}}\right]$
$\displaystyle=\pi\left({{\mathscr{B}}}\right)\exp\left[{-\gamma n\beta
a_{1}{r}}\right]\sum_{j\geqslant
0}\exp\left[{-\left({\gamma_{0}-2\gamma}\right)n\beta
a_{1}2^{j}2^{J}{r}}\right].$

Since $2^{j}\geqslant j+1$ for all $j\geqslant 0$ we obtain that

$\displaystyle\int_{\leftidx{{}^{\mathsf{c}}}{\\!{{\mathscr{B}}}}{}({\overline{Q}},2^{J}r)}\exp\left[{-\gamma_{0}n\beta
a_{1}\ell({\overline{Q}},P)}\right]d\pi(P)$
$\displaystyle\leqslant\pi\left({{\mathscr{B}}}\right)\exp\left[{-\gamma
n\beta a_{1}{r}}\right]\sum_{j\geqslant
0}\exp\left[{-\left({\gamma_{0}-2\gamma}\right)n\beta
a_{1}(j+1)2^{J}{r}}\right]$
$\displaystyle\leqslant\pi\left({{\mathscr{B}}}\right)\exp\left[{-\gamma
n\beta a_{1}{r}-\left({\gamma_{0}-2\gamma}\right)n\beta
a_{1}2^{J}{r}}\right]\sum_{j\geqslant
0}\exp\left[{-j\left({\gamma_{0}-2\gamma}\right)n\beta a_{1}2^{J}{r}}\right]$
$\displaystyle=\pi\left({{\mathscr{B}}}\right)\frac{\exp\left[{-\gamma n\beta
a_{1}{r}}\right]}{1-\exp\left[{-\left({\gamma_{0}-2\gamma}\right)n\beta
a_{1}2^{J}{r}}\right]}\exp\left[{-\left({\gamma_{0}-2\gamma}\right)n\beta
a_{1}2^{J}{r}}\right].$

which leads to (61) since $n\beta a_{1}2^{J}{r}\geqslant n\beta
a_{1}{r}\geqslant 1$. Finally, by applying this inequality with $J=0$ we
obtain that

$\displaystyle\int_{{\mathscr{M}}}\exp\left[{-\beta
na_{1}\gamma_{0}\ell({\overline{Q}},P)}\right]d\pi(P)$
$\displaystyle=\int_{{\mathscr{B}}}\exp\left[{-\beta
na_{1}\gamma_{0}\ell({\overline{Q}},P)}\right]d\pi(P)+\int_{\leftidx{{}^{\mathsf{c}}}{\\!{{\mathscr{B}}}}{}}\exp\left[{-\beta
na_{1}\gamma_{0}\ell({\overline{Q}},P)}\right]d\pi(P)$
$\displaystyle\leqslant\pi({\mathscr{B}})\left[{1+\frac{\exp\left[{-\gamma-\left({\gamma_{0}-2\gamma}\right)n\beta
a_{1}{r}}\right]}{1-\exp\left[{-\left({\gamma_{0}-2\gamma}\right)}\right]}}\right]$
$\displaystyle\leqslant\pi({\mathscr{B}})\left[{1+\frac{\exp\left[{-\left({\gamma_{0}-\gamma}\right)}\right]}{1-\exp\left[{-\left({\gamma_{0}-2\gamma}\right)}\right]}}\right],$

which is (62). ∎

###  6.2. Main parts of the proofs of Theorems 1 and 2

Throughout the proofs of these two theorems, we fix some arbitrary element
${\overline{Q}}\in{\mathscr{M}}$ and
${r}\geqslant{r}_{n}(\beta,{\overline{Q}})$. It follows from the definition of
${r}_{n}(\beta,{\overline{Q}})$ that ${r}$ satisfies both $n\beta
a_{1}{r}\geqslant 1$ and inequality (13). For a positive number $z$, that will
be chosen later as well, we set

$A=\left\\{{\int_{{\mathscr{M}}}\exp\left[{-\beta{\mathbf{T}}({\boldsymbol{X}},P)}\right]d\pi(P)>z}\right\\}.$

It follows from the definition (11) of $\widehat{\pi}_{{\boldsymbol{X}}}$
that, given $J\in{\mathbb{N}}$,

$\displaystyle{\mathbb{E}}\left[{\widehat{\pi}_{{\boldsymbol{X}}}\left({\leftidx{{}^{\mathsf{c}}}{\\!{{\mathscr{B}}}}{}({\overline{Q}},2^{J}{r})}\right)}\right]$
$\displaystyle={\mathbb{E}}\left[{\widehat{\pi}_{{\boldsymbol{X}}}\left({\leftidx{{}^{\mathsf{c}}}{\\!{{\mathscr{B}}}}{}({\overline{Q}},2^{J}{r})}\right)1\hskip-2.6pt{\rm
l}_{\leftidx{{}^{\mathsf{c}}}{\\!{A}}{}}}\right]+{\mathbb{E}}\left[{\widehat{\pi}_{{\boldsymbol{X}}}\left({\leftidx{{}^{\mathsf{c}}}{\\!{{\mathscr{B}}}}{}({\overline{Q}},2^{J}{r})}\right)1\hskip-2.6pt{\rm
l}_{A}}\right]$
$\displaystyle\leqslant{\mathbb{P}}(\leftidx{{}^{\mathsf{c}}}{\\!{A}}{})+\frac{1}{z}{\mathbb{E}}\left[{\int_{\leftidx{{}^{\mathsf{c}}}{\\!{{\mathscr{B}}}}{}({\overline{Q}},2^{J}{r})}\exp\left[{-\beta{\mathbf{T}}({\boldsymbol{X}},P)}\right]d\pi(P)}\right]$
(63)
$\displaystyle={\mathbb{P}}(\leftidx{{}^{\mathsf{c}}}{\\!{A}}{})+\frac{1}{z}\int_{\leftidx{{}^{\mathsf{c}}}{\\!{{\mathscr{B}}}}{}({\overline{Q}},2^{J}{r})}{\mathbb{E}}\left[{\exp\left[{-\beta{\mathbf{T}}({\boldsymbol{X}},P)}\right]}\right]d\pi(P).$

In a first step, we prove that for some well chosen values of $\beta,z,{r}$
and for $J$ large enough, each of the two terms in the right-hand side of (63)
is not larger than $e^{-\xi}$. To achieve this goal, we bound the first term
of the right-hand side of (63) by first applying Markov’s inequality

$\displaystyle{\mathbb{P}}(\leftidx{{}^{\mathsf{c}}}{\\!{A}}{})$
$\displaystyle={\mathbb{P}}\left[{\int_{{\mathscr{M}}}\exp\left[{-\beta{\mathbf{T}}({\boldsymbol{X}},P)}\right]d\pi(P)\leqslant
z}\right]$
$\displaystyle={\mathbb{P}}\left[{\left[{\int_{{\mathscr{M}}}\exp\left[{-\beta{\mathbf{T}}({\boldsymbol{X}},P)}\right]d\pi(P)}\right]^{-1}\geqslant
z^{-1}}\right]$ $\displaystyle\leqslant
z{\mathbb{E}}\left[{\frac{1}{\int_{{\mathscr{M}}}\exp\left[{-\beta{\mathbf{T}}({\boldsymbol{X}},P)}\right]d\pi(P)}}\right]$

and then by using Lemma 5, we obtain that

(64)
$\displaystyle{\mathbb{P}}(\leftidx{{}^{\mathsf{c}}}{\\!{A}}{})\leqslant\frac{z}{\pi^{2}({\mathscr{B}})}\left[{\int_{{\mathscr{B}}^{2}}\exp\left[{-{\mathbf{L}}(P,Q)}\right]d\pi_{{\mathscr{B}}}(P)d\pi_{{\mathscr{B}}}(Q)}\right]^{-1}.$

To show that the first term in the right hand-side of (63) is not larger than
$e^{-\xi}$ we therefore prove that this is the case of the right-hand side of
(64) for $z$ small enough. We bound the second term of (63) by using Lemma 4.

We then finish the proofs of Theorems 1 and 2 as follows. In the context of
Theorem 1, we finally establish that for a suitable value of $J$ and all
${\overline{Q}}\in{\mathscr{M}}(\beta)$,

$\displaystyle{\mathbb{E}}\left[{\widehat{\pi}_{{\boldsymbol{X}}}\left({\leftidx{{}^{\mathsf{c}}}{\\!{{\mathscr{B}}}}{}({\overline{Q}},2^{J}{r})}\right)}\right]\leqslant
2e^{-\xi}\quad\text{with}\quad
r=r({\overline{Q}})=\ell(\overline{P}^{\star},{\overline{Q}})+a_{1}^{-1}\left({\beta+\frac{2\xi}{n\beta}}\right).$

By (3),
${\mathscr{B}}({\overline{Q}},2^{J}r)\subset{\mathscr{B}}(\overline{P}^{\star},\tau\ell(\overline{P}^{\star},{\overline{Q}})+\tau
2^{J}r)$ for all ${\overline{Q}}\in{\mathscr{M}}(\beta)$, and consequently
${\mathbb{E}}\left[{\widehat{\pi}_{{\boldsymbol{X}}}\left({\leftidx{{}^{\mathsf{c}}}{\\!{{\mathscr{B}}}}{}(\overline{P}^{\star},\overline{r}}\right)}\right]\leqslant
2e^{-\xi}$ with

$\displaystyle\overline{r}=\overline{r}({\overline{Q}})=\tau\left[{\ell(\overline{P}^{\star},{\overline{Q}})+2^{J}{r}}\right]=\tau\left[{(1+2^{J})\ell(\overline{P}^{\star},{\overline{Q}})+2^{J}a_{1}^{-1}\left({\beta+\frac{2\xi}{n\beta}}\right)}\right].$

We obtain (LABEL:eq-thm01) by monotone convergence, taking a sequence
$({\overline{Q}}_{N})_{N\geqslant 0}\subset{\mathscr{M}}(\beta)$ such that
$\ell(\overline{P}^{\star},{\overline{Q}}_{N})$ is nonincreasing to
$\inf_{P\in{\mathscr{M}}(\beta)}\ell(\overline{P}^{\star},P)$, so that

$\displaystyle\lim_{N\to+\infty}\overline{r}({\overline{Q}}_{N})$
$\displaystyle=\tau\left[{(1+2^{J})\inf_{{\overline{Q}}\in{\mathscr{M}}(\beta)}\ell(\overline{P}^{\star},{\overline{Q}})+2^{J}a_{1}^{-1}\left({\beta+\frac{2\xi}{n\beta}}\right)}\right]$
$\displaystyle\leqslant\tau(1+2^{J})\left[{\inf_{{\overline{Q}}\in{\mathscr{M}}(\beta)}\ell(\overline{P}^{\star},{\overline{Q}})+a_{1}^{-1}\left({\beta+\frac{2\xi}{n\beta}}\right)}\right]$

and (LABEL:eq-thm01) holds provided that $\kappa_{0}\geqslant\tau(2^{J}+1)$.

In the context of Theorem 2, we show that for some suitable value of $J$ and
all ${\overline{Q}}\in{\mathscr{M}}$,

$\displaystyle{\mathbb{E}}\left[{\widehat{\pi}_{{\boldsymbol{X}}}\left({\leftidx{{}^{\mathsf{c}}}{\\!{{\mathscr{B}}}}{}({\overline{Q}},2^{J}{r})}\right)}\right]\leqslant
2e^{-\xi}\quad\text{with}\quad
r=\ell(\overline{P}^{\star},{\overline{Q}})+{r}_{n}({\overline{Q}},\beta)+\frac{2\xi}{n\beta
a_{1}},$

and we get (25) by arguing similarly.

###  6.3. Proof of Theorem 1

For all $i\in\\{1,\ldots,n\\}$ and $P,Q,Q^{\prime}\in{\mathscr{M}}$, let us
set

(65) $\displaystyle U_{i}$
$\displaystyle={c}\left({t_{(P,Q^{\prime})}(X_{i})-{\mathbb{E}}\left[{t_{(P,Q^{\prime})}(X_{i})}\right]}\right)$
$\displaystyle\quad-{c}_{1}\left({t_{(P,Q)}(X_{i})-{\mathbb{E}}\left[{t_{(P,Q)}(X_{i})}\right]}\right)$
(66) $\displaystyle V_{i}$
$\displaystyle={c}_{2}\left({t_{(P,Q^{\prime})}(X_{i})-{\mathbb{E}}\left[{t_{(P,Q^{\prime})}(X_{i})}\right]}\right)$
$\displaystyle\quad-{c}_{1}\left({t_{(P,Q)}(X_{i})-{\mathbb{E}}\left[{t_{(P,Q)}(X_{i})}\right]}\right).$

The random variables $U_{i}$ are independent and under Assumption 3-(iv), they
takes their values in an interval of length $l_{1}={c}+{c}_{1}=1+2{c}$. The
$V_{i}$ are also independent and they takes their values in an interval of
length $l_{2}={c}_{1}+{c}_{2}=3+2{c}$. Applying Lemma 6, we obtain that

(67) $\displaystyle\prod_{i=1}^{n}{\mathbb{E}}\left[{\exp\left[{\beta
U_{i}}\right]}\right]$
$\displaystyle\leqslant\exp\left[{\frac{l_{1}^{2}n\beta^{2}}{8}}\right]$

and

(68) $\displaystyle\prod_{i=1}^{n}{\mathbb{E}}\left[{\exp\left[{\beta
V_{i}}\right]}\right]$
$\displaystyle\leqslant\exp\left[{\frac{l_{2}^{2}n\beta^{2}}{8}}\right].$

By using Assumption 2 and the fact that ${c}_{0}={c}_{1}-{c}a_{0}/a_{1}>0$,

$\displaystyle{c}\left({a_{0}\ell(\overline{P}^{\star},P)-a_{1}\ell(\overline{P}^{\star},Q^{\prime})}\right)-{c}_{1}\left({a_{1}\ell(\overline{P}^{\star},P)-a_{0}\ell(\overline{P}^{\star},Q)}\right)$
$\displaystyle=-\left({{c}_{1}a_{1}-{c}a_{0}}\right)\ell(\overline{P}^{\star},P)-{c}a_{1}\ell(\overline{P}^{\star},Q^{\prime})+{c}_{1}a_{0}\ell(\overline{P}^{\star},Q)$
$\displaystyle\leqslant-{c}_{0}a_{1}\left[{\tau^{-1}\ell({\overline{Q}},P)-\ell(\overline{P}^{\star},{\overline{Q}})}\right]-{c}a_{1}\left[{\tau^{-1}\ell({\overline{Q}},Q^{\prime})-\ell(\overline{P}^{\star},{\overline{Q}})}\right]$
$\displaystyle\quad+\tau{c}_{1}a_{0}\left[{\ell(\overline{P}^{\star},{\overline{Q}})+\ell({\overline{Q}},Q)}\right]$
(69)
$\displaystyle=e_{0}a_{1}\ell(\overline{P}^{\star},{\overline{Q}})-\tau^{-1}{c}_{0}a_{1}\ell({\overline{Q}},P)-\tau^{-1}{c}a_{1}\ell({\overline{Q}},Q^{\prime})+\tau{c}_{1}a_{0}\ell({\overline{Q}},Q)$

with

(70) $e_{0}={c}_{0}+{c}+\frac{\tau{c}_{1}a_{0}}{a_{1}}.$

It follows from (69) and Assumptions 3-(iii), more precisely its consequences
(52) and (53), that

$\displaystyle
n^{-1}\left\\{{{c}{\mathbb{E}}\left[{{\mathbf{T}}({\boldsymbol{X}},P,Q^{\prime})}\right]-{c}_{1}{\mathbb{E}}\left[{{\mathbf{T}}({\boldsymbol{X}},P,Q)}\right]}\right\\}$
$\displaystyle\leqslant{c}\left[{a_{0}\ell(\overline{P}^{\star},P)-a_{1}\ell(\overline{P}^{\star},Q^{\prime})}\right]-{c}_{1}\left[{a_{1}\ell(\overline{P}^{\star},P)-a_{0}\ell(\overline{P}^{\star},Q)}\right]$
(71) $\displaystyle\leqslant
e_{0}a_{1}\ell(\overline{P}^{\star},{\overline{Q}})-\tau^{-1}{c}_{0}a_{1}\ell({\overline{Q}},P)-\tau^{-1}{c}a_{1}\ell({\overline{Q}},Q^{\prime})+\tau{c}_{1}a_{0}\ell({\overline{Q}},Q).$

Since $a_{0}\geqslant a_{1}$ and ${c}_{2}>{c}_{1}$,
${c}_{0}^{\prime}={c}_{2}(a_{0}/a_{1})-{c}_{1}>0$ and by arguing as above, we
obtain similarly that

$\displaystyle
n^{-1}\left\\{{{c}_{2}{\mathbb{E}}\left[{{\mathbf{T}}({\boldsymbol{X}},P,Q^{\prime})}\right]-{c}_{1}{\mathbb{E}}\left[{{\mathbf{T}}({\boldsymbol{X}},P,Q)}\right]}\right\\}$
$\displaystyle\leqslant{c}_{2}\left({a_{0}\ell(\overline{P}^{\star},P)-a_{1}\ell(\overline{P}^{\star},Q^{\prime})}\right)-{c}_{1}\left({a_{1}\ell(\overline{P}^{\star},P)-a_{0}\ell(\overline{P}^{\star},Q)}\right)$
$\displaystyle={c}_{0}^{\prime}a_{1}\ell(\overline{P}^{\star},P)-{c}_{2}a_{1}\ell(\overline{P}^{\star},Q^{\prime})+{c}_{1}a_{0}\ell(\overline{P}^{\star},Q)$
$\displaystyle\leqslant\tau{c}_{0}^{\prime}a_{1}\left[{\ell(\overline{P}^{\star},{\overline{Q}})+\ell({\overline{Q}},P)}\right]-{c}_{2}a_{1}\left[{\tau^{-1}\ell({\overline{Q}},Q^{\prime})-\ell(\overline{P}^{\star},{\overline{Q}})}\right]$
$\displaystyle\quad+\tau{c}_{1}a_{0}\left[{\ell(\overline{P}^{\star},{\overline{Q}})+\ell({\overline{Q}},Q)}\right]$
$\displaystyle\leqslant\left({e_{1}+{c}_{2}}\right)a_{1}\ell(\overline{P}^{\star},{\overline{Q}})+\tau{c}_{0}^{\prime}a_{1}\ell({\overline{Q}},P)$
(72)
$\displaystyle\quad-\tau^{-1}{c}_{2}a_{1}\ell({\overline{Q}},Q^{\prime})+\tau{c}_{1}a_{0}\ell({\overline{Q}},Q),$

with

(73)
$e_{1}=\tau\left[{{c}_{0}^{\prime}+{c}_{1}a_{0}/a_{1}}\right]=\tau\left[{{c}_{2}(a_{0}/a_{1})+\
{c}_{1}\left({a_{0}/a_{1}-1}\right)}\right].$

Using (67) and (71), we deduce that for all $P,Q,Q^{\prime}\in{\mathscr{M}}$

$\displaystyle{\mathbb{E}}\left[{\exp\left[{\beta\left({{c}{\mathbf{T}}({\boldsymbol{X}},P,Q^{\prime})-{c}_{1}{\mathbf{T}}({\boldsymbol{X}},P,Q)}\right)}\right]}\right]$
$\displaystyle=\prod_{i=1}^{n}{\mathbb{E}}\left[{\exp\left[{\beta\left({{c}t_{(P,Q^{\prime})}(X_{i})-{c}_{1}t_{(P,Q)}(X_{i})}\right)}\right]}\right]$
$\displaystyle=\exp\left[{\beta\left({{c}{\mathbb{E}}\left[{{\mathbf{T}}({\boldsymbol{X}},P,Q^{\prime})}\right]-{c}_{1}{\mathbb{E}}\left[{{\mathbf{T}}({\boldsymbol{X}},P,Q)}\right]}\right)}\right]\prod_{i=1}^{n}{\mathbb{E}}\left[{\exp\left[{\beta
U_{i}}\right]}\right]$ (74)
$\displaystyle\leqslant\exp\left[{n\beta\left[{\Delta_{1}(P,Q)-\tau^{-1}{c}a_{1}\ell({\overline{Q}},Q^{\prime})}\right]}\right]$

with

(75)
$\displaystyle\Delta_{1}(P,Q)=e_{0}a_{1}\ell(\overline{P}^{\star},{\overline{Q}})+\tau{c}_{1}a_{0}\ell({\overline{Q}},Q)+\frac{l_{1}^{2}\beta}{8}-\tau^{-1}{c}_{0}a_{1}\ell({\overline{Q}},P).$

Using (68) and (72), we obtain similarly that for all
$P,Q,Q^{\prime}\in{\mathscr{M}}$

$\displaystyle{\mathbb{E}}\left[{\exp\left[{\beta\left({{c}_{2}{\mathbf{T}}({\boldsymbol{X}},P,Q^{\prime})-{c}_{1}{\mathbf{T}}({\boldsymbol{X}},P,Q)}\right)}\right]}\right]$
(76)
$\displaystyle\leqslant\exp\left[{n\beta\left[{\Delta_{2}(P,Q)-\tau^{-1}{c}_{2}a_{1}\ell({\overline{Q}},Q^{\prime})}\right]}\right]$

with

$\displaystyle\Delta_{2}(P,Q)$
$\displaystyle=\left({e_{1}+{c}_{2}}\right)a_{1}\ell(\overline{P}^{\star},{\overline{Q}})+\tau{c}_{0}^{\prime}a_{1}\ell({\overline{Q}},P)+\tau{c}_{1}a_{0}\ell({\overline{Q}},Q)$
(77) $\displaystyle\quad+\frac{l_{2}^{2}\beta}{8}.$

Since $2\gamma<\tau^{-1}{c}<\tau^{-1}{c}_{2}$, we may apply Lemma 8 with
$\gamma_{0}=\tau^{-1}{c}$ and $\gamma_{0}=\tau^{-1}{c}_{2}$ successively which
leads to

(78) $\displaystyle\int_{{\mathscr{M}}}\exp\left[{-\tau^{-1}{c}n\beta
a_{1}\ell({\overline{Q}},Q^{\prime})}\right]d\pi(Q^{\prime})\leqslant\pi\left({{\mathscr{B}}}\right)\exp\left[{\Xi_{1}}\right]$

and

(79) $\displaystyle\int_{{\mathscr{M}}}\exp\left[{-\tau^{-1}{c}_{2}n\beta
a_{1}\ell({\overline{Q}},Q^{\prime})}\right]d\pi(Q^{\prime})\leqslant\pi\left({{\mathscr{B}}}\right)\exp\left[{\Xi_{1}}\right]$

with

(80) $\displaystyle\Xi_{1}$
$\displaystyle=\log\left[{1+\frac{\exp\left[{-\left({\tau^{-1}{c}-\gamma}\right)}\right]}{1-\exp\left[{-\left({\tau^{-1}{c}-2\gamma}\right)}\right]}}\right]$
$\displaystyle\geqslant\log\left[{1+\frac{\exp\left[{-\left({\tau^{-1}{c}_{2}-\gamma}\right)}\right]}{1-\exp\left[{-\left({\tau^{-1}{c}_{2}-2\gamma}\right)}\right]}}\right].$

Putting (76) and (79) together leads to

$\displaystyle\exp\left[{{\mathbf{L}}(P,Q)}\right]$
$\displaystyle=\int_{{\mathscr{M}}}{\mathbb{E}}\left[{\exp\left[{\beta\left({{c}_{2}{\mathbf{T}}({\boldsymbol{X}},P,Q^{\prime})-{c}_{1}{\mathbf{T}}({\boldsymbol{X}},P,Q)}\right)}\right]}\right]d\pi(Q^{\prime})$
$\displaystyle\leqslant\exp\left[{n\beta\Delta_{2}(P,Q)}\right]\int_{{\mathscr{M}}}\exp\left[{-\tau^{-1}{c}_{2}n\beta
a_{1}\ell({\overline{Q}},Q^{\prime})}\right]d\pi(Q^{\prime})$
$\displaystyle\leqslant\pi\left({{\mathscr{B}}}\right)\exp\left[{\Xi_{1}+n\beta\Delta_{2}(P,Q)}\right],$

and since, for all $(P,Q)\in{\mathscr{B}}^{2}$, by definition (77) of
$\Delta_{2}(P,Q)$,

$\displaystyle\Delta_{2}(P,Q)$
$\displaystyle\leqslant\left({e_{1}+{c}_{2}}\right)a_{1}\ell(\overline{P}^{\star},{\overline{Q}})+\left[{\tau{c}_{0}^{\prime}a_{1}+\tau{c}_{1}a_{0}}\right]{r}+\frac{l_{2}^{2}\beta}{8}$
(81)
$\displaystyle=\left({e_{1}+{c}_{2}}\right)a_{1}\ell(\overline{P}^{\star},{\overline{Q}})+e_{1}a_{1}{r}+\frac{l_{2}^{2}\beta}{8}=\Delta_{2}$

we derive that

$\displaystyle\left[{\int_{{\mathscr{B}}^{2}}\exp\left[{-{\mathbf{L}}(P,Q)}\right]d\pi_{{\mathscr{B}}}(P)d\pi_{{\mathscr{B}}}(Q)}\right]^{-1}\leqslant\pi\left({{\mathscr{B}}}\right)\exp\left[{\Xi_{1}+n\beta\Delta_{2}}\right].$

We deduce from (64) that

$\displaystyle{\mathbb{P}}(\leftidx{{}^{\mathsf{c}}}{\\!{A}}{})$
$\displaystyle\leqslant\frac{z}{\pi\left({{\mathscr{B}}}\right)}\exp\left[{\Xi_{1}+n\beta\Delta_{2}}\right].$

In particular, ${\mathbb{P}}(\leftidx{{}^{\mathsf{c}}}{\\!{A}}{})\leqslant
e^{-\xi}$ for $z$ satisfying

(82)
$\displaystyle\log\left({\frac{1}{z}}\right)=\xi+\log\frac{1}{\pi({\mathscr{B}})}+\Xi_{1}+n\beta\Delta_{2}.$

Putting (74) and (78) together, we obtain that

$\displaystyle\exp\left[{{\mathbf{M}}(P,Q)}\right]$
$\displaystyle=\int_{{\mathscr{M}}}{\mathbb{E}}\left[{\exp\left[{\beta\left({{c}{\mathbf{T}}({\boldsymbol{X}},P,Q^{\prime})-{c}_{1}{\mathbf{T}}({\boldsymbol{X}},P,Q)}\right)}\right]}\right]d\pi(Q^{\prime})$
$\displaystyle\leqslant\exp\left[{n\beta\Delta_{1}(P,Q)}\right]\int_{{\mathscr{M}}}\exp\left[{-\tau^{-1}{c}n\beta
a_{1}\ell({\overline{Q}},Q^{\prime})}\right]d\pi(Q^{\prime})$
$\displaystyle\leqslant\pi\left({{\mathscr{B}}}\right)\exp\left[{\Xi_{1}+n\beta\Delta_{1}(P,Q)}\right].$

It follows from the definition (75) of $\Delta_{1}(P,Q)$ that for all
$P\in{\mathscr{M}}$ and for all $Q\in{\mathscr{B}}$,

$\displaystyle\Delta_{1}(P,Q)$ $\displaystyle\leqslant
e_{0}a_{1}\ell(\overline{P}^{\star},{\overline{Q}})+\tau{c}_{1}a_{0}{r}+\frac{l_{1}^{2}\beta}{8}-\tau^{-1}{c}_{0}a_{1}\ell({\overline{Q}},P),$

and consequently, for all $P\in{\mathscr{M}}$ and $Q\in{\mathscr{B}}$

$\displaystyle\exp\left[{{\mathbf{M}}(P,Q)}\right]$
$\displaystyle\leqslant\pi\left({{\mathscr{B}}}\right)\exp\left[{\Xi_{1}+n\beta\left({e_{0}a_{1}\ell(\overline{P}^{\star},{\overline{Q}})+\tau{c}_{1}a_{0}{r}+\frac{l_{1}^{2}\beta}{8}-\tau^{-1}{c}_{0}a_{1}\ell({\overline{Q}},P)}\right)}\right].$

We derive from Lemma 4 that

$\displaystyle{\mathbb{E}}\left[{\exp\left[{-\beta{\mathbf{T}}({\boldsymbol{X}},P)}\right]}\right]$
$\displaystyle\leqslant\frac{1}{\pi({\mathscr{B}})}\left[{\int_{{\mathscr{B}}}\exp\left[{-{\mathbf{M}}(P,Q)}\right]d\pi_{{\mathscr{B}}}(Q)}\right]^{-1}$
$\displaystyle\leqslant\exp\left[{\Xi_{1}+n\beta\left({e_{0}a_{1}\ell(\overline{P}^{\star},{\overline{Q}})+\tau{c}_{1}a_{0}{r}+\frac{l_{1}^{2}\beta}{8}-\tau^{-1}{c}_{0}a_{1}\ell({\overline{Q}},P)}\right)}\right],$

hence,

(83)
$\displaystyle\int_{\leftidx{{}^{\mathsf{c}}}{\\!{{\mathscr{B}}}}{}({\overline{Q}},2^{J}{r})}$
$\displaystyle{\mathbb{E}}\left[{\exp\left[{-\beta{\mathbf{T}}({\boldsymbol{X}},P)}\right]}\right]d\pi(P)$
$\displaystyle\leqslant\exp\left[{\Xi_{1}+n\beta\left({e_{0}a_{1}\ell(\overline{P}^{\star},{\overline{Q}})+\tau{c}_{1}a_{0}{r}+\frac{l_{1}^{2}\beta}{8}}\right)}\right]$
$\displaystyle\quad\times\int_{\leftidx{{}^{\mathsf{c}}}{\\!{{\mathscr{B}}}}{}({\overline{Q}},2^{J}{r})}\exp\left[{-\tau^{-1}{c}_{0}n\beta
a_{1}\ell({\overline{Q}},P)}\right]d\pi(P).$

Applying Lemma 8 with $\gamma_{0}=\tau^{-1}{c}_{0}>2\gamma$ and setting
$e_{2}=\tau^{-1}{c}_{0}-2\gamma$, we get

$\displaystyle\int_{\leftidx{{}^{\mathsf{c}}}{\\!{{\mathscr{B}}}}{}({\overline{Q}},2^{J}{r})}\exp\left[{-\tau^{-1}{c}_{0}n\beta
a_{1}\ell({\overline{Q}},P)}\right]d\pi(P)\leqslant\pi\left({{\mathscr{B}}}\right)\exp\left[{\Xi_{2}-e_{2}n\beta
a_{1}2^{J}{r}}\right]$

with

(84)
$\Xi_{2}=-\gamma+\log\left[{\frac{1}{1-\exp\left[{-e_{2}}\right]}}\right],$

which together with (83) leads to

$\displaystyle\log\int_{\leftidx{{}^{\mathsf{c}}}{\\!{{\mathscr{B}}}}{}({\overline{Q}},2^{J}{r})}{\mathbb{E}}\left[{\exp\left[{-\beta{\mathbf{T}}({\boldsymbol{X}},P)}\right]}\right]d\pi(P)$
$\displaystyle\leqslant\log\left[{\pi\left({{\mathscr{B}}}\right)}\right]+\Xi_{1}+\Xi_{2}$
(85)
$\displaystyle\quad+n\beta\left[{e_{0}a_{1}\ell(\overline{P}^{\star},{\overline{Q}})+\tau{c}_{1}a_{0}{r}+\frac{l_{1}^{2}\beta}{8}-e_{2}a_{1}2^{J}{r}}\right].$

Using the definitions (82) of $z$ and (81) of $\Delta_{2}$ we deduce from (85)
that

$\displaystyle\log\left[{\frac{1}{z}\int_{\leftidx{{}^{\mathsf{c}}}{\\!{{\mathscr{B}}}}{}({\overline{Q}},2^{J}{r})}{\mathbb{E}}\left[{\exp\left[{-\beta{\mathbf{T}}({\boldsymbol{X}},P)}\right]}\right]d\pi(P)}\right]$
$\displaystyle\leqslant\log\left({\frac{1}{z}}\right)+\log\left[{\pi\left({{\mathscr{B}}}\right)}\right]+\Xi_{1}+\Xi_{2}$
$\displaystyle\quad+n\beta\left[{e_{0}a_{1}\ell(\overline{P}^{\star},{\overline{Q}})+\tau{c}_{1}a_{0}{r}+\frac{l_{1}^{2}\beta}{8}-e_{2}a_{1}2^{J}{r}}\right]$
$\displaystyle=\xi+\log\frac{1}{\pi({\mathscr{B}})}+\Xi_{1}+n\beta\Delta_{2}+\log\left[{\pi\left({{\mathscr{B}}}\right)}\right]+\Xi_{1}+\Xi_{2}$
$\displaystyle\quad+n\beta\left[{e_{0}a_{1}\ell(\overline{P}^{\star},{\overline{Q}})+\tau{c}_{1}a_{0}{r}+\frac{l_{1}^{2}\beta}{8}-e_{2}a_{1}2^{J}{r}}\right]$
$\displaystyle=n\beta\left[{\left({e_{1}+{c}_{2}+e_{0}}\right)a_{1}\ell(\overline{P}^{\star},{\overline{Q}})+e_{1}a_{1}{r}+\frac{l_{2}^{2}\beta}{8}++\tau{c}_{1}a_{0}{r}+\frac{l_{1}^{2}\beta}{8}}\right]$
$\displaystyle\quad+\xi+2\Xi_{1}+\Xi_{2}-e_{2}n\beta a_{1}2^{J}{r}$
$\displaystyle=n\beta\left[{\left({e_{0}+e_{1}+{c}_{2}}\right)a_{1}\ell(\overline{P}^{\star},{\overline{Q}})+\left[{e_{1}+\frac{\tau{c}_{1}a_{0}}{a_{1}}}\right]a_{1}{r}+\frac{(l_{1}^{2}+l_{2}^{2})\beta}{8}}\right]$
(86) $\displaystyle\quad+\xi+2\Xi_{1}+\Xi_{2}-e_{2}n\beta a_{1}2^{J}{r}.$

Setting,

$C_{1}=e_{0}+e_{1}+{c}_{2}\quad\text{and}\quad
C_{2}=e_{1}+\frac{\tau{c}_{1}a_{0}}{a_{1}},$

we see that the right-hand side of (86) is not larger than $-\xi$, provided
that

$\displaystyle e_{2}n\beta a_{1}2^{J}{r}$ $\displaystyle\geqslant
2\xi+2\Xi_{1}+\Xi_{2}+n\beta\left[{C_{1}a_{1}\ell(\overline{P}^{\star},{\overline{Q}})+C_{2}a_{1}{r}+\frac{(l_{1}^{2}+l_{2}^{2})\beta}{8}}\right]$

or equivalently if

(87) $\displaystyle 2^{J}$
$\displaystyle\geqslant\frac{1}{e_{2}}\left[{\frac{2\xi+2\Xi_{1}+\Xi_{2}}{\beta
na_{1}{r}}+\frac{C_{1}\ell(\overline{P}^{\star},{\overline{Q}})}{{r}}+C_{2}+\frac{\left[{l_{1}^{2}+l_{2}^{2}}\right]\beta}{8a_{1}{r}}}\right].$

Choosing ${\overline{Q}}$ in ${\mathscr{M}}(\beta)$ and using the inequalities
$a_{1}^{-1}\beta\geqslant{r}_{n}(\beta,{\overline{Q}})\geqslant 1/(\beta
na_{1})$, for

${r}=\ell(\overline{P}^{\star},{\overline{Q}})+\frac{1}{a_{1}}\left({\beta+\frac{2\xi}{n\beta}}\right)\geqslant\frac{1}{\beta
na_{1}}$

we obtain that the right-hand side of (87) satisfies

$\displaystyle\frac{1}{e_{2}}\left[{\frac{2\xi+2\Xi_{1}+\Xi_{2}}{\beta
na_{1}{r}}+\frac{C_{1}\ell(\overline{P}^{\star},{\overline{Q}})+C_{2}{r}}{{r}}+\frac{\left[{l_{1}^{2}+l_{2}^{2}}\right]\beta}{8a_{1}{r}}}\right]$
$\displaystyle\leqslant\frac{1}{e_{2}}\left[{C_{2}+2\Xi_{1}+\Xi_{2}+\frac{C_{3}}{{r}}\left({\ell(\overline{P}^{\star},{\overline{Q}})+\frac{1}{a_{1}}\left({\beta+\frac{2\xi}{n\beta}}\right)}\right)}\right]$
$\displaystyle=\frac{1}{e_{2}}\left[{C_{2}+2\Xi_{1}+\Xi_{2}+C_{3}}\right]$

with $C_{3}=\max\\{1,C_{1},\left[{l_{1}^{2}+l_{2}^{2}}\right]/8\\}$.
Inequality (87) is therefore satisfied for $J\in{\mathbb{N}}$ such that

$2^{J}\geqslant\frac{C_{2}+2\Xi_{1}+\Xi_{2}+C_{3}}{e_{2}}\vee 1>2^{J-1},$

and we may take

(88)
$\displaystyle\kappa_{0}=\tau\left[{\frac{2\left({C_{2}+2\Xi_{1}+\Xi_{2}+C_{3}}\right)}{e_{2}}\vee
1+1}\right]\geqslant\tau\left({2^{J}+1}\right).$

We recall below, the list of constants depending on $a_{0},a_{1},c,\tau$ and
$\gamma$ and we have used along the proof.

$\displaystyle c_{0}$ $\displaystyle=1+c-\frac{ca_{0}}{a_{1}},$ $\displaystyle
c_{1}$ $\displaystyle=1+c,$ $\displaystyle c_{2}$ $\displaystyle=2+c,$
$\displaystyle c^{\prime}_{0}$ $\displaystyle=\frac{c_{2}a_{0}}{a_{1}}-c_{1},$
$\displaystyle l_{1}$ $\displaystyle=1+2c,$ $\displaystyle l_{2}$
$\displaystyle=3+2c,$ $\displaystyle e_{0}$ $\displaystyle=c_{0}+c+\frac{\tau
c_{1}a_{0}}{a_{1}},$ $\displaystyle e_{1}$
$\displaystyle=\tau\left[{c_{0}^{\prime}+c_{1}\frac{a_{0}}{a_{1}}}\right],$
$\displaystyle e_{2}$ $\displaystyle=\tau^{-1}c_{0}-2\gamma,$ $\displaystyle
C_{1}$ $\displaystyle=e_{0}+e_{1}+c_{2},$ $\displaystyle C_{2}$
$\displaystyle=e_{1}+\frac{\tau c_{1}a_{0}}{a_{1}},$ $\displaystyle C_{3}$
$\displaystyle=\max\left\\{{1,C_{1},\frac{l_{1}^{2}+l_{2}^{2}}{8}}\right\\},$

and

$\displaystyle\Xi_{1}$
$\displaystyle=\log\left[{1+\frac{\exp\left[{-\left({\tau^{-1}{c}-\gamma}\right)}\right]}{1-\exp\left[{-\left({\tau^{-1}{c}-2\gamma}\right)}\right]}}\right],\quad\Xi_{2}=-\gamma+\log\left[{\frac{1}{1-\exp\left[{-e_{2}}\right]}}\right].$

###  6.4. Proof of Theorem 2

The proof follows the same lines as that of Theorem 1. Under Assumption
3-(iv), the random variables $U_{i}$ and $V_{i}$ defined by (65) and (66) are
not larger than with $b={c}+{c}_{1}=l_{1}$ and $b={c}_{2}+{c}_{1}=l_{2}$
respectively. Since under Assumption 4, more precisely its consequence (54),
that

$\displaystyle\frac{1}{n}\sum_{i=1}^{n}{\mathbb{E}}\left[{U_{i}^{2}}\right]$
$\displaystyle\leqslant 2\left[{\frac{{c}^{2}}{n}\sum_{i=1}^{n}\mathop{\rm
Var}\nolimits\left[{t_{(P,Q^{\prime})}(X_{i})}\right]+\frac{{c}_{1}^{2}}{n}\sum_{i=1}^{n}\mathop{\rm
Var}\nolimits\left[{t_{(P,Q)}(X_{i})}\right]}\right]$ $\displaystyle\leqslant
2a_{2}\left[{({c}^{2}+{c}_{1}^{2})\ell(\overline{P}^{\star},P)+{c}^{2}\ell(\overline{P}^{\star},Q^{\prime})+{c}_{1}^{2}\ell(\overline{P}^{\star},Q)}\right]$

and

$\displaystyle\frac{1}{n}\sum_{i=1}^{n}{\mathbb{E}}\left[{V_{i}^{2}}\right]$
$\displaystyle\leqslant
2a_{2}\left[{({c}_{2}^{2}+{c}_{1}^{2})\ell(\overline{P}^{\star},P)+{c}_{2}^{2}\ell(\overline{P}^{\star},Q^{\prime})+{c}_{1}^{2}\ell(\overline{P}^{\star},Q)}\right]$

we may apply Lemma 7 and using the notations $\Lambda_{1}=\tau\phi(\beta
l_{1})$, $\Lambda_{2}=\tau\phi(\beta l_{2})$ and Assumption 1, we get

$\displaystyle\frac{1}{n\beta}\log\left[{\prod_{i=1}^{n}{\mathbb{E}}\left[{\exp\left[{\beta
U_{i}}\right]}\right]}\right]$ $\displaystyle\leqslant{\phi(\beta l_{1})}\beta
a_{2}\left[{({c}^{2}+{c}_{1}^{2})\ell(\overline{P}^{\star},P)+{c}^{2}\ell(\overline{P}^{\star},Q^{\prime})+{c}_{1}^{2}\ell(\overline{P}^{\star},Q)}\right]$
$\displaystyle\leqslant 2\Lambda_{1}\beta
a_{2}\left[{{c}^{2}+{c}_{1}^{2}}\right]\ell(\overline{P}^{\star},{\overline{Q}})$
(89) $\displaystyle\quad+\Lambda_{1}\beta
a_{2}\left[{({c}^{2}+{c}_{1}^{2})\ell({\overline{Q}},P)+{c}^{2}\ell({\overline{Q}},Q^{\prime})+{c}_{1}^{2}\ell({\overline{Q}},Q)}\right]$

and similarly

$\displaystyle\frac{1}{n\beta}\log\left[{\prod_{i=1}^{n}{\mathbb{E}}\left[{\exp\left[{\beta
V_{i}}\right]}\right]}\right]$ $\displaystyle\leqslant 2\Lambda_{2}\beta
a_{2}\left[{{c}_{2}^{2}+{c}_{1}^{2}}\right]\ell(\overline{P}^{\star},{\overline{Q}})$
(90) $\displaystyle\quad+\Lambda_{2}\beta
a_{2}\left[{({c}_{2}^{2}+{c}_{1}^{2})\ell({\overline{Q}},P)+{c}_{2}^{2}\ell({\overline{Q}},Q^{\prime})+{c}_{1}^{2}\ell({\overline{Q}},Q)}\right].$

It follows from (71) that

$\displaystyle E_{1}$
$\displaystyle=n^{-1}\left\\{{{c}{\mathbb{E}}\left[{{\mathbf{T}}({\boldsymbol{X}},P,Q^{\prime})}\right]-{c}_{1}{\mathbb{E}}\left[{{\mathbf{T}}({\boldsymbol{X}},P,Q)}\right]}\right\\}$
$\displaystyle\quad+2\Lambda_{1}\beta
a_{2}\left[{{c}^{2}+{c}_{1}^{2}}\right]\ell(\overline{P}^{\star},{\overline{Q}})$
$\displaystyle\quad+\Lambda_{1}\beta
a_{2}\left[{({c}^{2}+{c}_{1}^{2})\ell({\overline{Q}},P)+{c}^{2}\ell({\overline{Q}},Q^{\prime})+{c}_{1}^{2}\ell({\overline{Q}},Q)}\right]$
$\displaystyle\leqslant\left[{e_{0}a_{1}+2\Lambda_{1}\beta
a_{2}\left({{c}^{2}+{c}_{1}^{2}}\right)}\right]\ell(\overline{P}^{\star},{\overline{Q}})$
$\displaystyle\quad-\left[{\tau^{-1}{c}_{0}a_{1}-\Lambda_{1}\beta
a_{2}({c}^{2}+{c}_{1}^{2})}\right]\ell({\overline{Q}},P)$
$\displaystyle\quad-\left[{\tau^{-1}{c}a_{1}-\Lambda_{1}\beta
a_{2}{c}^{2}}\right]\ell({\overline{Q}},Q^{\prime})$
$\displaystyle\quad+\left[{\tau{c}_{1}a_{0}+\Lambda_{1}\beta
a_{2}{c}_{1}^{2}}\right]\ell({\overline{Q}},Q).$

Using the definitions (22) of $\overline{c}_{1}$ and (23) of
$\overline{c}_{2}$, i.e.

$\displaystyle\overline{c}_{1}={c}_{0}-\tau\Lambda_{1}\beta
a_{2}a_{1}^{-1}({c}^{2}+{c}_{1}^{2})\quad\text{and}\quad\overline{c}_{2}={c}-\tau\Lambda_{1}\beta
a_{2}a_{1}^{-1}{c}^{2}$

and setting

$\displaystyle e_{3}$
$\displaystyle=e_{0}+2\Lambda_{1}\beta\frac{a_{2}\left({{c}^{2}+{c}_{1}^{2}}\right)}{a_{1}}$
$\displaystyle e_{4}$
$\displaystyle=\frac{1}{a_{1}}\left[{\tau{c}_{1}a_{0}+\Lambda_{1}\beta
a_{2}{c}_{1}^{2}}\right]$

and arguing as in the proof of inequality (74), we deduce from (89) that

$\displaystyle\log{\mathbb{E}}\left[{\exp\left[{\beta\left({{c}{\mathbf{T}}({\boldsymbol{X}},P,Q^{\prime})-{c}_{1}{\mathbf{T}}({\boldsymbol{X}},P,Q)}\right)}\right]}\right]$
$\displaystyle\leqslant n\beta E_{1}$ (91) $\displaystyle\leqslant n\beta
a_{1}\left[{e_{3}\ell(\overline{P}^{\star},{\overline{Q}})-\tau^{-1}\left[{\overline{c}_{1}\ell({\overline{Q}},P)+\overline{c}_{2}\ell({\overline{Q}},Q^{\prime})}\right]+e_{4}\ell({\overline{Q}},Q)}\right].$

It follows from (72) that

$\displaystyle E_{2}$
$\displaystyle=n^{-1}\left\\{{{c}_{2}{\mathbb{E}}\left[{{\mathbf{T}}({\boldsymbol{X}},P,Q^{\prime})}\right]-{c}_{1}{\mathbb{E}}\left[{{\mathbf{T}}({\boldsymbol{X}},P,Q)}\right]}\right\\}$
$\displaystyle\quad+2\Lambda_{2}\beta
a_{2}\left[{{c}_{2}^{2}+{c}_{1}^{2}}\right]\ell(\overline{P}^{\star},{\overline{Q}})$
$\displaystyle\quad+\Lambda_{2}\beta
a_{2}\left[{({c}_{2}^{2}+{c}_{1}^{2})\ell({\overline{Q}},P)+{c}_{2}^{2}\ell({\overline{Q}},Q^{\prime})+{c}_{1}^{2}\ell({\overline{Q}},Q)}\right]$
$\displaystyle\leqslant\left({e_{1}+{c}_{2}}\right)a_{1}\ell(\overline{P}^{\star},{\overline{Q}})+\tau{c}_{0}^{\prime}a_{1}\ell({\overline{Q}},P)-\tau^{-1}{c}_{2}a_{1}\ell({\overline{Q}},Q^{\prime})$
$\displaystyle\quad+\tau{c}_{1}a_{0}\ell({\overline{Q}},Q)+2\Lambda_{2}\beta
a_{2}\left[{{c}_{2}^{2}+{c}_{1}^{2}}\right]\ell(\overline{P}^{\star},{\overline{Q}})$
$\displaystyle\quad+\Lambda_{2}\beta