Adaptive regression with Brownian path covariate
Karine Bertin Nicolas Klutchnikoff
math.ST stat.TH






Adaptive regression with Brownian path covariate
    
Karine Bertin

CIMFAV-INGEMAT, Universidad de Valparaíso, General Cruz 222,
Valparaíso, Chile,
<karine.bertin@uv.cl>
Nicolas Klutchnikoff

Univ Rennes, CNRS, IRMAR – UMR 6625, F-35000 Rennes, France, 
<nicolas.klutchnikoff@univ-rennes2.fr>

    
=====================================================================================================================================================================================================================================================




This paper deals with estimation with functional covariates. More precisely, we aim at estimating the regression function m of a continuous outcome Y against a standard Wiener coprocess W. Following <cit.> and <cit.> the Wiener-Itô decomposition of m(W) is used to construct a family of estimators. The minimax rate of convergence over specific smoothness classes is obtained. A data-driven selection procedure is defined following the ideas developed by <cit.>. An oracle-type inequality is obtained which leads to adaptive results.

Keywords: Functional regression, Wiener-Itô chaos expansion, Oracle inequalities, Adaptive minimax rates of convergence.

AMS Subject Classification:  62G08, 62H12




§ INTRODUCTION



The problem of regression estimation  is one of the most studied in statistics and different models have been considered depending on the nature of the data.
In an increasing number of applications, it seems natural to assume that the covariate takes values in a functional space. The book of <cit.> provides an overview on the subject of functional data analysis. In this context, several authors studied linear functional regression models <cit.>. Nonparametric functional regression models have also been investigated <cit.>.
In this paper, we are interested in such a model where the covariate is a Wiener Process.
More precisely, let ε be a real-valued random variable and W=(W(t) : 0≤ t ≤ 1) be a standard Brownian motion independent of ε. We define

    Y = m(W) + ε

where m:→ is a mapping defined on the set  of all continuous functions w:[0,1]→ and we assume that both m(W) and ε are square integrable random variables. Our goal is to estimate the function m using a dataset (Y_1,W_1),…,(Y_n,W_n) of independent realizations of (Y,W).


Since this framework is a specific case of the more general functional regression framework, usual approaches (which mainly consist in extending classical local methods such as k-nearest neighbors, kernel smoothing or local polynomial smoothing) could be used. However, in our context, these methods are known to lead to slow rates of convergence over classical models (see below for detailed references).
Taking advantage of the probabilistic properties of the Wiener coprocess, we aim at defining a new family of models as well as dedicated estimation procedures with faster rates of convergence (in both minimax and adaptive minimax senses). Despite the fact that considering Brownian paths covariates seems restrictive for pratical purposes, several Brownian diffusion paths could also be considered. Albeilt the systematic theoretical study of such models is beyond the scope of this paper (and is left to further developments), we propose some extensions of our framework as well as some examples of usual processes that can be considered, such as geometric Brownian motions or Ornstein-Uhlenbeck processes.

In usual functional approaches the set 𝒞 is endowed with a metric d <cit.> which allows to extend several nonparametric estimators. For example a simple version of the Nadaraya-Watson estimator is given, for any function w∈ and any bandwidth h>0, by:

    m̃_h(w) = ∑_i=1^n Y_i _{d(W_i,w)≤ h}/∑_j=1^n _{d(W_j,w)≤ h},

where  stands for the indicator function.
The properties of these estimators are related to the behavior of a quantity known as the small ball probability defined for w∈ and h>0 by φ_w(h)=(d(W,w)≤ h). Pointwise risks of such methods can be generally bounded, up to a positive factor by

    h^β + (1/nφ_w(h))^1/2

where β denotes the smoothness of the mapping m measured in a Hölder sense. For example, if 0<β≤1 it is assumed that there exists L>0 such that |m(w)-m(w')|≤ Ld(w,w')^β for any w,w'∈𝒞. Under additional assumptions similar results can be obtained for integrated risks.

The classical assumption φ_w(h) ≍ h^k corresponds roughly to the situation where the covariate W lies in some space of finite dimension k  <cit.>. This framework corresponds  to the usual nonparametric case. The minimax rates of convergence are then given by n^-β/(2β+k) <cit.>. However if W lies in a functional space, the behavior of φ_w(h) is quite different. In our context, where W is a standard Wiener process, it is well-known <cit.> that

    log(sup_t∈[0,1]|W(t)|≤ h) = logφ_0(h) h→ 0≍-h^-2

which leads to slower rates of convergence of the form (log n)^-β/2 assuming a β-Hölder condition on m. We refer the reader to <cit.> for recent results with different behavior of φ_w(h).

In practical situations, since β is unknown, finding adaptive procedures to select the smoothing parameter h is of prime interest. To our best knowledge few papers deal with this problem.
Adaptive procedures based on cross validation have been used in <cit.>.  also propose an adaptation of the method developed by Goldenshluger and Lepski <cit.> using an empirical version of the quantity φ_w(h). Lower bounds have been investigated by <cit.>. In all these papers, the pointwise risk is studied in terms of φ_w(h) and theoretical properties are obtained assuming a β-Hölder condition on m with respect to the metric d with smoothness β∈(0,1].

In this paper we follow a different strategy. Taking advantage of probabilistic properties of the Wiener process, similarly to the methodology developed by <cit.> and <cit.>,  we consider the Wiener-Itô chaotic decomposition of m(W). Indeed, every random variable that belongs to ^2_W = {(W) | : → and ((W))^2 < +∞} can be decomposed as a sum of multiple stochastic integrals <cit.>. There exists a unique sequence of functions (f_ℓ)_ℓ≥ 1 such that

    m(W) ^2=(Y) + ∑_ℓ=1^∞1/ℓ! I_ℓ(f_ℓ)(W),

where f_ℓ belongs to (Δ_ℓ), the set of symmetric and square integrable real-valued functions defined on Δ_ℓ=[0,1]^ℓ and

    I_ℓ(f_ℓ)(W) = ∫_Δ_ℓ f_ℓ  W^⊗ℓ
    =  ∫_Δ_ℓ f_ℓ(u_1,…,u_ℓ) W( u_1)⋯ W( u_ℓ).

We recall that f is symmetric on Δ_ℓ if for any (t_1,…,t_ℓ)∈Δ_ℓ and any permutation σ of {1,2,…,ℓ}, f(t_1,…,t_ℓ)=f(t_σ(1),…,t_σ(ℓ)).  Note that the symmetry implies that the functions f_ℓ are isotropic.  
The iterated integral I_ℓ(f_ℓ)(W) is called a chaos of order ℓ.

Our approach consists in defining kernel-type estimators f_ℓ of f_ℓ using the Itô's isometry, see (<ref>). Then, based on (<ref>), we propose the following estimator of m

    m̂_ℒ(W) = 1/n∑_i=1^n Y_i  + ∑_ℓ=1^ℒ1/ℓ!I_ℓ(f_ℓ)(W),

with ℒ∈. To study these estimators, we  assume that m belongs to a specific class of mappings that satisfy

    ∑_ℓ=1^∞e^2γℓ/ℓ!f_ℓ^2_Δ_ℓ≤ M^2,

for some γ>0 and M>0, ·_Δ_ℓ is the classical L_2 norm on Δ_ℓ and that the f_ℓ defined by (<ref>) are Hölderian. Such classes are quite natural in our context and are connected with the usual Meyer-Watanabe test function space (see section <ref> for more details).  

In this case, we find rates of convergence for the prediction error in 𝕃^p norm. Contrary to the classical functional framework, where logarithmic rates are derived, the rates we obtain are intermediate between logarithmic and polynomial rates.

If we assume moreover that the summation in (<ref>) stops at a known index L, we prove that the estimators  m̂_L achieve optimal rates of convergence. We derive minimax rates of convergence which are polynomial in n with an exponent that depends on the smoothness of the functions f_ℓ.  A data-driven procedure, based on the method developed by <cit.>, is then defined to tune the bandwidths used in the estimation of the functions f_ℓ. The resulting estimator of m satisfies an oracle-type inequality that allows us to derive adaptive results.

The paper is organized as follows. Section <ref> presents the model and the studied problem. Section <ref> describes the construction of the estimators. Section <ref> gives the main results and Section <ref> is dedicated to the proofs.





§ STATISTICAL FRAMEWORK




 §.§ Model


Let  W=(W(t) : 0≤ t ≤ 1) be a standard Brownian motion and let ε be a centered real-valued random variable independent of W. We define:

    Y = m(W) + ε

where m:→ is a given mapping. We assume that m(W) as well as ε belong to ^2, the set of square integrable random variables, then

    m(W) = (Y) + ∑_ℓ=1^∞1/ℓ! I_ℓ(f_ℓ)(W),

where for ℓ∈, f_ℓ belongs to (Δ_ℓ).
As mentioned in the introduction we also assume that f_ℓ is a regular function. Below we define precisely the functional classes used to measure the smoothness of each function f_ℓ.


    Set ℓ∈ s_ℓ>0 and  Λ_ℓ>0. The Hölder ball _ℓ(s_ℓ, Λ_ℓ)  is the set of all functions f:Δ_ℓ→ that satisfy the following properties:
    
        
  * For any α=(α_1,…,α_ℓ)∈^ℓ such that |α|=∑_i α_i ≤s_ℓ=max{k∈| k<s_ℓ}, the partial derivative
        D^α f exists where
        
    D^α f = ∂^|α| f/∂ x_1^α_1…∂ x_ℓ^α_ℓ.

        
  * For any x and y in Δ_ℓ we have:
        
    ∑_|α|=s_ℓ| D^αf(x)-D^αf(y) |
                ≤Λ_ℓ |x-y|^s_ℓ-s_ℓ,

        where |·| stands for the Euclidean norm of ^ℓ.
      
  * We have f∈(Δ_ℓ).
    


Equipped with these notations we can define  a scale of classes (s,Λ,γ,M) for the mapping m. Roughly, we impose some restrictions on the functions f_ℓ that appear in (<ref>) of two kinds: a minimal smoothness, for each f_ℓ, is imposed and the growth of the 𝐋^2-norm  of the f_ℓ is controlled.


Set s=(s_1,s_2,…)∈(0,+∞)^, Λ=(Λ_1,Λ_2,…)∈(0,+∞)^, γ>0 and M>0.  We say that m:→ belongs to the mapping class (s,Λ,γ,M)  if there exist a∈ and a sequence of functions (f_ℓ)_ℓ∈ satisfying 

    m(W) ^2= a + ∑_ℓ=1^∞1/ℓ! I_ℓ(f_ℓ)(W),

with f_ℓ∈_ℓ(s_ℓ, Λ_ℓ) and

    ∑_ℓ=1^∞e^2γℓ/ℓ!f_ℓ^2_Δ_ℓ≤ M^2

where f_ℓ_Δ_ℓ = (∫_Δ_ℓ f^2_ℓ(u)   u)^1/2.



  Equation (<ref>) implies that 
  
    m(W)∈⋂_k≥0
    1< p<e^γ+1𝔻_k,p

  where 𝔻_k,p denotes the usual Sobolev space over the Wiener space defined in <cit.>. Note also that, if, for any ℓ≥1 we have f_ℓ_Δ_ℓ≤ C^ℓ for some positive constant C, then (<ref>) is fulfilled for any γ≥0.


We also define subclasses of the classes (s,Λ,γ,M) assuming that the summation in (<ref>) stops at a finite index L∈. 

 Set L∈.
  Set s=(s_1,…,s_L)∈(0,+∞)^L and Λ=(Λ_1,…, Λ_L)∈(0,+∞)^L.  We say that m:→ belongs to the mapping class (s,Λ,L,M)  if there exist a∈ and a sequence of functions (f_ℓ)_1≤ℓ≤ L satisfying 
  
    m(W) = a + ∑_ℓ=1^L1/ℓ! I_ℓ(f_ℓ)(W),

  with f_ℓ∈_ℓ(s_ℓ, Λ_ℓ) and f_ℓ^2_Δ_ℓ≤ M^2 ℓ! for any ℓ=1,…,L.
   

More precisely, for any s, Λ and M, the subclasses (s,Λ,L,M) satisfy (s,Λ,L,M)⊂(s,Λ,γ,M') with M'^2=M^2∑_ℓ=1^L  e^-2γℓ. Let us comment on the above definitions since the framework we consider in this paper is quite different to the usual functional framework recalled in the introduction.  In our framework the “regularity” of a map m is  seen through the prism of the chaotic decomposition of m(W) and, thus, the functions f_ℓ. This is not directly linked with the regularity of the mapping m between the space 𝒞 endowed with the topology induced by the 𝐋^2 norm and . For example, it can be easily seen that the mapping m defined, for any w∈𝒞 by m(w) = w^2(1) is not continuous (which implies that this function is not hölderian and, thus, cannot be considered in the usual framework). However it is well known that m(W) = 1 + I_2(1)(W). As a consequence, the mapping m falls within our scope since m belongs to ℳ(s,Λ, 2, 1) for any s and Λ. 



 §.§ Minimax and adaptive framework


The observations consist in a n-sample (Y_1,W_1), …, (Y_n,W_n) distributed as and independent of (Y,W).  Our first goal is to investigate the estimation of m, based on these observations, over the classes (s,Λ,L,M) and  (s,Λ,γ,M) for 0<s≤ s^* where  is fixed. To measure the accuracy of an arbitrary estimator m̃_n = m̃( · ; (Y_1,W_1), …, (Y_n,W_n)) of m, we consider the prediction risk:

    R_p(m̃_n, m) = (|m̃_n(W) - m(W)|^p)^1/p

where p≥ 2. The maximal risk of an arbitrary estimator m̃_n over a given class of mappings  is defined by:

    R_p(m̃_n, ) = sup_∈ R_p(m̃_n, ),

whereas the minimax risk is defined, taking the infimum over all possible estimators, by:

    Φ_n(,p) = inf_m̃_n R_p(m̃_n, ).

An estimator m̃_n whose maximal risk is asymptotically bounded, up to a multiplicative factor, by Φ_n(,p) is called minimax over . Such an estimator is well-adapted to the estimation over  but it can perform poorly over another class of mappings. The problem of adaptive estimation consists in finding a single estimation procedure that is simultaneously minimax over a scale of mapping classes. 

Our second goal is to investigate the adaptive estimation of m over the scale of classes 
ℳ(s^*,L,M) = {_s,Λ,L,M| s∈(0,)^L, Λ∈(0,+∞)^L} where >0, L∈ and M>0 are fixed and known by the statistician. More precisely our goal is to construct a single estimation procedure m_n^* such that, for any ∈ℳ(s^*,L,M), the risk  R_p(m_n^*, ) is asymptotically bounded, up to a multiplicative constant, by Φ_n(,p).
One of the main tools to prove such a result is to find an oracle-type inequality that guarantees that this procedure performs almost as well as the best estimator in a rich family of estimators. Ideally, we would like to have, for any ∈⋃_s,Λ,L, M, an inequality of the following form:

    R_p(m_n^*, ) ≤inf_η∈ H R_p(m̃_n,η, ),

where  {m̃_n,η|η∈ H} is a family of estimators well-adapted to our problem in the following sense: for any ∈ℳ(s^*,L,M), there exists η∈ H such that m̃_n,η is minimax over .
However, in many situations, (<ref>) is relaxed and we prove a weaker inequality of the type:

    R_p(m_n^*, ) ≤Υ_1,pinf_η∈ H R_p^*(, η) + Υ_2,p(log n/n)^1/2,

where Υ_1,p and Υ_2,p are two positive constants and  R_p^*(, η) is an appropriate quantity to be determined that can be viewed as a tight upper bound on R_p(m̃_n,η, ). Inequalities of the form (<ref>) are called oracle-type inequalities.

Theorems <ref> and <ref> below correspond respectively to an oracle-type inequality and an adaptive result of these types.



 §.§ Extensions to our model


In this paper, we focus on pure Brownian coprocesses. However our framework allows us to consider a larger class of covariates. Assume that we aim at estimating the regression function g : 𝒞→ in the model:

    Y = g(X) + ε,

where X is a process driven by the SDE:

    dX_t = b(t, X_t)dt + σ(t, X_t)dW_t,
         0≤ t≤ 1.

Here σ and b are assumed to be known functions and we also assume that  assumptions guaranteeing the existence and uniqueness of the solution of (<ref>) are fulfilled. If  for any 0≤ t ≤ 1,  σ(t, X_t)>0, then, under mild integrability conditions, we have:

    W_t =  ∫_0^t d X_s/σ(s, X_s)  -  ∫_0^t μ(s, X_s)/σ(s, X_s)d s.

This implies that there exists a known invertible function ϕ:𝒞→𝒞 such that  W = ϕ(X). In general, this function can be computed by numerical integration. However, in some situations, an exact expression can be obtained using Itô's formula. This is the case for two parametric families of processes widely used to model several practical situations.
First, Ornstein–Uhlenbeck processes X_t are driven by the following SDE:

    X_t = -θ(X_t - μ)  t + σ W_t 
    
            X_0 = x_0,

where x_0∈ is fixed and θ>0, μ∈ℝ and σ>0 are known parameters. By Itô's formula we have:

    W_t = 1/σ[ X_t - x_0 + θ∫_0^t ( X_s - μ)  s].

Next, Geometric Brownian motions are used to model stock prices in the Black–Scholes model. Let μ∈ and σ>0 be given parameters. We assume that the process X=(X_t : t∈ I) is driven by the following SDE:

    X_t = X_t( μ t + σ W_t )
    
            X_0 = x_0.

By Itô's formula we have:

    W_t = 1/σ[ log(X_t/x_0) + 
        (σ^2/2 - μ) t].
 


  In practical situation the parameters μ, σ and θ in the above examples are not known. However, estimators of these parameters could be used to estimate the coprocess W. This leads to new models where the covariate in observed with errors. The study of such models is beyond the scope of this paper and left to further developments.   



In view of (<ref>), equation (<ref>) can be written as:

    Y = m(W) + ε  where  
      m = g∘ϕ^-1.

Thus, the regression problem (<ref>) falls into our framework. The estimation strategy consists of estimating the function m based on the reconstruction of Brownian path W=ϕ(X). This can be summarized by the formula:

    ĝ = g∘ϕ^-1∘ϕ = m̂∘ϕ.
 
Remark also that, in this context, it is relevant to assume that the chaotic decomposition of m(W) is finite. Indeed, under mild assumptions on b and σ <cit.>, if g(X) is a polynomial of the terminal value X_1 of the process X, then the mapping  m(W) = g(X ) can be written as a finite chaotic decomposition with smooth functions f_ℓ.





§ ESTIMATOR CONSTRUCTION


In this section we present our estimation procedure. To do so, we first recall classical properties satisfied by Wiener chaos which allow us to construct a family of “simple” estimators that depends on a multivariate tuning parameter. Next we construct a procedure which selects, in a data-driven way, this tuning parameter using the methodology developed by <cit.>.



 §.§ Classical properties of the chaos


Throughout this paper and in the construction of our statistical procedure, we use the following two fundamental properties satisfied by the iterated integrals.

For ℓ,ℓ'∈ℕ, Itô's isometry <cit.> ensures that, if g∈(Δ_ℓ) and g'∈(Δ_ℓ'), then

    (I_ℓ(g)(W)I_ℓ'(g')(W)) = δ_ℓ,ℓ'ℓ! ∫_Δ_ℓ g(u)g'(u)   u

where δ_ℓ,ℓ' denotes the Kronecker delta.

The hypercontractivity property <cit.> will be used to control the concentration of our estimators. Set q≥ 2 and ℓ∈. For any g∈(Δ_ℓ) we have:

    ( |I_ℓ(g)(W)|^q)^1/q≤_ℓ(q)
    ( I_ℓ^2(g)(W))^1/2  where  _ℓ(q)=(q-1)^ℓ/2.



 §.§ A simple family of estimators


Let :→ be a function that satisfies the following properties:  is continuous inside [0,1], (x)=0 for any x∉[0,1],

    ∫_0^1 (x)   x = 1
            and    ∫_0^1 x^s(x)   x = 0,   s=1,…,.

Let ℓ∈. A natural estimator of the function f_ℓ is given, for h∈(0,1), by:

    f̂^(ℓ)_h(t) = 1/n∑_i=1^n Y_i I_ℓ(ℓh(t,·))(W_i),
           t∈Δ_ℓ

where 
ℓh is a multivariate kernel defined by:

    ℓh(t, u ) = 1/h^ℓ∏_k=1^ℓ(ς(t_k)t_k-u_k/h)
          with    ς(·) = 2I_(1/2,1) (·) - 1.

This specific construction allows one to obtain an estimator free of boundary bias <cit.>.

Indeed, note that for any t∈Δ_ℓ and under regularity assumptions on f_ℓ we have:

    f_ℓ(t)
         (h→0)≈∫_Δ_ℓ f_ℓ(u) ℓh(t,u)  u
       =
      (1/ℓ!I_ℓ(f_ℓ)(W) I_ℓ(ℓh(t,·))(W))
       = (m(W) I_ℓ(ℓh(t,·))(W))

where the last two lines are obtained using (<ref>) and (<ref>). Since ε is centered and independent of W we have:

    f_ℓ(t)
         (h→0)≈(Y I_ℓ(ℓh(t,·))(W))
       (n→+∞)≈f̂^(ℓ)_h(t).

Equipped with these notations we define a family of plugin estimators of the mapping m. For ℒ∈ and all h=(h_1,…,h_ℒ)∈(0,1)^ℒ we set:

    m̂_h,ℒ(W) = Y_n + ∑_ℓ=1^ℒ1/ℓ! I_ℓ(f̂^(ℓ)_h_ℓ)(W)

where Y_n=∑_i=1^n Y_i/n.
In the following, we study the rate of convergence of the estimator (<ref>) when ℒ=L_n where (L_n)_n∈ is a sequence of integers that tends to ∞ as n tends to ∞ (see Theorem <ref>) and ℒ=L where L is a known fixed integer (see Theorem <ref>).




 §.§ Selection procedure


Set >0, L∈ and M>0. Assume that μ_4=(|ε|^4)^1/4 exists. Let ℓ∈{1,…,L} be fixed and define

    _ℓ={h∈(0,1): n^-1/(2+ℓ)≤ h≤ (log n)^-1}∩{e^-k: k∈}.

Now, define

    M(ℓ,h) = ν(ℓ)(1+4√(log (1/h^ℓ)))/√(nh^ℓ)

where

    ν(ℓ)=(μ_4+∑_k=1^L _4(k)M ) √(b_ℓ,2)/2    and       b_ℓ,2=_ℓ^2(4)2^ℓℓ!_[0,1]^2ℓ,

where the constants _ℓ(k) are defined in (<ref>).
Define for h∈_ℓ

    B(ℓ,h)=max_h'∈_ℓ{f̂_h'^(ℓ)-f̂_h∨ h'^(ℓ)_Δ_ℓ -M(ℓ,h') -M(ℓ, h∨ h')  }_+

and set

    ĥ_ℓ= min_h∈_ℓ{ B(ℓ,h)+M(ℓ,h)}.

The estimation procedure is the defined by m̂_L=m̂_ĥ,L where ĥ=(ĥ_1,…,ĥ_L).


This selection rule follows the principles and the ideas developed by Goldenshluger and Lepski in a series of papers <cit.>. The quantity M(ℓ, h), which is called a majorant in the papers cited above, is a penalized version of the standard deviation of the estimator f̂^(ℓ)_h while the quantity B(ℓ,h) is, in some sense, closed to its bias term, see (<ref>). Finding tight majorants is the key point of the method since ĥ_ℓ is chosen in (<ref>) in order to realize an empirical trade-off between these two quantities.

It is worth noting that the procedure depends on a hyperparameter >0 which can be chosen arbitrary small. The introduction of this parameter is due to technical reasons, see (<ref>) in the proof of Lemma <ref>. This additional assumption (we would like to take =0) implies some restrictions on Theorem <ref> below.





§ MAIN RESULTS





 §.§ Result for the infinite chaos model




Our first result studies the risk of our family of estimators over the class (s,Λ,γ,M). In this class, the function m is decomposed into an infinite sum of chaos:
 
    m(W) ^2=(Y) + ∑_ℓ=1^∞1/ℓ! I_ℓ(f_ℓ)(W).







Set p≥ 2,  Λ_*>0 and >0. Set  s∈(0,)^,  Λ∈(Λ_*,+∞)^ and  M>0 and let γ be such   that 2γ>max(2s^* + log(p-1), log(3)). Assume that μ_p=|ε|^p<+∞. 
Define


    L_n = [( log n)^1/2],     C_2=(6(p-1))^1/2_[0,1]

where [·] denotes the integer part and  h_n=(h_n^(ℓ)(s,Λ))_ℓ=1,…,L_n∈(0,1)^L_n where for any ℓ=1,…,L_n:

    h_n^(ℓ)(s,Λ) = ( C_2^2L_n/Λ_ℓ^2n)^1/2s_ℓ+ℓ.
  
There exists a positive constant κ depending on ,  Λ_*, μ_2, μ_p, γ and M such that

    R_p(m̂_h_n,L_n,    (s,Λ,γ,M)    )
      ≤κ∑_ℓ=1^L_nΛ_ℓ^2ℓ/(2s_ℓ+ℓ)( C_2^2L_n/n)^s_ℓ/2s_ℓ+ℓ.



Let us briefly comment on this result. Assume first that the parameters s_ℓ are constant and denote by s_0 their common value. In this case we obtain

    R_p(m̂_h_n,L_n,    (s,Λ,γ,M)  )
      ≤κ(max_ℓΛ_ℓ^2)L_n ( C_2^2L_n/n)^s_0/2s_0+L_n.

This implies that, for n large enough, R_p(m̂_h_n,L_n,    (s,Λ,γ,M)  ), is upperbounded, up to a multiplicative constant by

    u_n=
      (log n)^1/2exp(-s_0/2 (log n)^1/2).

Remark that such a rate of convergence lies in-between polylogarithmic rates of convergence and polynomial ones. This result can be compared with those obtained by <cit.>. Recall that, in this paper, the authors study a similar model with a Poisson point process covariate. The rates v_n obtained in this paper are slightly better than ours since they obtain, for some α∈(0,1)

    log(v_n) ∼ -α/2( log n)^1/2(loglog n )^1/2

whereas, in our case,

    log(u_n) ∼ -s_0/2( log n)^1/2.

However remark that their study is limited to p=2 and s_0=1 and that, moreover, they assume that the response Y is a bounded variable. In our situation neither m(W) nor ε are assumed to be bounded.




 §.§ Results for finite chaos model


In the three following results, we assume that it exists a known integer L∈ such that 

    m(W) ^2=(Y) + ∑_ℓ=1^L 1/ℓ! I_ℓ(f_ℓ)(W).

Our second result proves that the minimax rate of convergence over the class _s,Λ,L,M is of the same order as:

    ϕ_n(s,Λ) = max{Λ_ℓ^2ℓ/(2s_ℓ+ℓ)n^-s_ℓ/(2s_ℓ+ℓ)|ℓ=1,…,L}.


  Set p≥ 2, s∈(0,)^L, Λ∈(Λ_*,+∞)^L, M>0 and assume that μ_p = (|ε|^p)^1/p<+∞. Define  h̃_n=(h̃_n^(ℓ)(s,Λ))_ℓ=1,…,L∈(0,1)^L where:
  
    h̃_n^(ℓ)(s,Λ) = (1/Λ_ℓ^2 n)^1/2s_ℓ+ℓ,
            ℓ=1,…,L.

There exist two positive constants κ_* and κ^* that depend only on L, M, Λ_*, μ_2, μ_p and  such that
  
    lim sup_n→+∞ϕ_n^-1(s,Λ) R_p(m̂_h̃_n,L, _s,Λ,L,M) ≤κ^*

  and
  
    lim inf_n→+∞ϕ_n^-1(s,Λ) Φ_n(_s,Λ,L,M,p) ≥κ_*.



Note that this result also ensures that the family of estimators constructed in Section <ref> is well-adapted to our problem. The next result states an oracle-type inequality satisfied by our data-driven estimator m̂_L.

 Set p≥2 and assume that 
for any ℓ=1,…,L, f_ℓ_Δ_ℓ^2≤ M^2ℓ! and that for any q≥ 1 the moment μ_q=(|ε|^q)^1/q exists.
Then:

    R_p(m̂_L, m)≤Υ_1∑_ℓ=1^L inf_h∈_ℓ[ max_h'∈_ℓ
     h'≤ hf̂_h'^ℓ-f_ℓ_Δ_ℓ +M(ℓ,h) ] +Υ_2(log n/n)^1/2,

where Υ_1 and Υ_2 are two positive constants that depend on L, M,  and .


Using Theorems <ref> and <ref> we can derive our last result: the data-driven estimation procedure is adaptive, up to a logarithmic factor, over the scale {_s,Λ, L, M:s∈ (,s^*)^L, Λ∈(0,+∞)^L, M>0}.

 Set p≥ 2 and assume that for any q≥ 1 the moment μ_q=(|ε|^q)^1/q exists. For any s∈ (,s^*)^L, any Λ∈(0,+∞)^L, any M>0, we have

    lim sup_n→+∞ϕ̃^-1_n(s) R_p(m̂_L, _s,Λ, L, M)≤κ^**

where κ^** is a positive constant that depends on Λ, L, M,  and  and

    ϕ̃_n(s)=max{(log n/n)^s_ℓ/(2s_ℓ+ℓ)|ℓ=1,…,L}.




While the selection procedure is defined using the ^2-norms, the procedure is adaptive for any p≥ 2. This phenomenon is due to the hypercontractivity property, see (<ref>). 
Note that in Theorem <ref>, the quantity 
    max_h'∈_ℓ
     h'≤ hf̂_h'^ℓ-f_ℓ_Δ_ℓ
 is a tight upper bound of the bias term of the estimator f̂_h^ℓ.

This result ensures that our data-driven procedure is adaptive, up to a logarithmic factor, over a large scale of mapping classes.

The presence of the extra logarithmic factor in the adaptive rate of convergence is not usual for prediction risks. This term is introduced in the definition of M(ℓ, h) to control the deviation of the estimator (<ref>) based on the variables I_ℓ(K_h^(ℓ)(t,·))(W_i). See (<ref>) for more details.




§ PROOFS


We first consider some notations and lemmas. Define for i∈{1,…,n}, ℓ∈{1,…,L} and h∈ (0,1)

    ξ_i,ℓ(t,h) = I_ℓ(K_h^ℓ(t,·))(W_i),
          and  ξ_ℓ(t,h) = I_ℓ(K_h^ℓ(t,·))(W)

and

    Θ_i,ℓ = I_ℓ(f_ℓ)(W_i)
          and  Θ_ℓ = I_ℓ(f_ℓ)(W).



We have, for any ℓ∈{1,…,L}, h∈_ℓ and  r≥ 1

    (ξ_ℓ^2r(t,h))^1/r≤ b_ℓ,r h^-ℓ  with   b_ℓ,r=_ℓ^2(2r)2^ℓℓ!_[0,1]^2ℓ.

Moreover for φ>0 and q≥ 1

    (|ξ_ℓ(t,h)|^r_|ξ_ℓ(t,h)| > φ)
    ≤  (b_ℓ,r)^r/2 (b_ℓ,q)^q/2φ^-qh^-ℓ(r+q)/2.



Let ℓ∈{1,…,L} and h∈_ℓ. Let χ,χ_1,…,χ_n be i.i.d random variables such that, for any r≥ 1

    ( |χ|^2r)^1/(2r)≤ a_r <+∞.

Define

    (t)=1/n∑_i=1^n{χ_i ξ_i,ℓ(t,h)
        - (
        χ_i ξ_i,ℓ(t,h)
        )
        }

and

    T= (1+δ) a_2√(b_ℓ,2)/2√(nh^ℓ)    with    δ = 4√(log (1/h^ℓ)).

Then there exists a positive constant C>0 such that

    {_Δ_ℓ-T}_+^2≤ Cn^-1.



The following Lemma recalls the Bousquet's version of Talagrand's concentration  inequality <cit.>.

Let X_1,…, X_n be independent identically distributed random variables. Let 𝒮 be a countable set of functions and define Z=sup_s∈𝒮∑_i=1^n s(X_i). Assume that, for all i=1,…, n and s∈𝒮, we have s(X_i)=0 and s(X_i)≤ 1 almost surely. Assume also that v=2 Z+sup_s∈𝒮∑_i=1^n  (s(X_i))^2<∞. Then we have for all t>0

    (Z- Z≥ t) ≤exp{ -t^2/2(v+t/3)}.





 §.§ Proof of Theorem <ref>


Set p≥ 2, s∈(0,)^, Λ∈(Λ_*,+∞)^, γ>0, and  M>0.
For the sake of readability we denote h=h_n, h_ℓ=h^(ℓ)_n(s,Λ), K_ℓ = ℓh_ℓ, ξ_ℓ(t)=ξ_ℓ(t, h_ℓ) and f̂_ℓ = f̂^(ℓ)_h_ℓ. 



  
Decomposition of the risk.
Using the triangle inequality we have:

    R_p(m̂_h,L_n,m)
           ≤(|Y̅_n-(Y)|^p)^1/p
        + ∑_ℓ=1^L_n1/ℓ!(|I_ℓ(f̂_ℓ-f_ℓ)(W)|^p)^1/p
           + ∑_ℓ=L_n+1^+∞1/ℓ!(|I_ℓ(f_ℓ)(W)|^p)^1/p
       ≤(|Y̅_n-(Y)|^p)^1/p
        + ∑_ℓ=1^L_n_ℓ(p)/ℓ!(|I_ℓ(f̂_ℓ-f_ℓ)(W)|^2)^1/2
           + ∑_ℓ=L_n+1^+∞_ℓ(p)/ℓ!(|I_ℓ(f_ℓ)(W)|^2)^1/2

Last line comes from the hypercontractivity property. Now, using Itô's isometry, we obtain:

    R_p(m̂_h,L_n,m)
         ≤(|Y̅_n-(Y)|^p)^1/p
        + ∑_ℓ=1^L_n_ℓ(p)/√(ℓ!)(f̂_ℓ-f_ℓ^2_Δ_ℓ)^1/2
            +∑_ℓ=L_n+1^+∞_ℓ(p)/√(ℓ!)f_ℓ_Δ_ℓ
       ≤(|Y̅_n-(Y)|^p)^1/p+ ∑_ℓ=1^L_n_ℓ(p)/√(ℓ!)(B(ℓ) + V(ℓ))
            + ∑_ℓ=L_n+1^+∞_ℓ(p)/√(ℓ!)f_ℓ_Δ_ℓ

where the bias term B(ℓ) and the stochastic term V(ℓ) are defined by:

    B(ℓ) = f̂_ℓ-f_ℓ_Δ_ℓ  and  
        V(ℓ) =( f̂_ℓ-f̂_ℓ^2_Δ_ℓ)^1/2.




  
Study of the constant term.
Remark that

    (|Y̅_n- Y|^p)^1/p   = (|1/n∑_i=1^n (Y_i- Y_i)|^p)^1/p
       ≤C_1,p/n^1-1/pσ_Y(p) + C_2,p/n^1/2σ_Y(2)

where the last line is obtained using Rosenthal's inequality <cit.>. Here C_1,p and C_2,p denote two positive constants while σ_Y(p) = (|Y -  Y|^p)^1/p.
Moreover since

    Y -  Y
        = ∑_ℓ = 1^+∞Θ_ℓ/ℓ! + ε

the hypercontractivity property, implies that, for any p≥ 2

    σ_Y(p)
        ≤∑_ℓ = 1^+∞1/ℓ!(| Θ_ℓ|^p)^1/p
     + (|ε|^p)^1/p
       ≤∑_ℓ = 1^+∞_ℓ(p)/ℓ!(Θ_ℓ^2)^1/2
     + (|ε|^p)^1/p
       = ∑_ℓ = 1^+∞_ℓ(p)/√(ℓ!)f_ℓ_Δ_ℓ
     + (|ε|^p)^1/p

Last line comes from Itô's isometry. Now, using that m∈(s,Λ,γ,M) and applying Cauchy-Schwarz inequality we obtain: 

    σ_Y(p)
      ≤ M ( ∑_ℓ=1^+∞exp(-2γℓ) _ℓ^2(p) )^1/2+(|ε|^p)^1/p = s_Y(p)

where, using the definition of _ℓ(p) and the fact that 2γ>log(p-1)

    s_Y(p)
    =M ( ∑_ℓ=1^+∞exp(-2γℓ) (p-1)^ℓ)^1/2 +(|ε|^p)^1/p
    <+∞.

We finally obtain

    (|Y̅_n- Y|^p)^1/p   ≤C_1,ps_Y(p)/n^1-1/p + C_2,ps_Y(2)/n^1/2
       ≤κ_0/n^1/2,

with κ_0 = C_1,ps_Y(p)+C_2,ps_Y(2) depends only on M,  γ, μ_2 and μ_p.



  
Study of the bias term.

Set ℓ∈{1,…,L_n} and note that:

    f̂_ℓ(t)
        = ((Yξ_ℓ(t))(W))
        = (m(W)ξ_ℓ(t))
        = 1/ℓ!(Θ_ℓξ_ℓ(t))

Using Itô's isometry we thus obtain:

    f̂_ℓ(t) =  ∫_Δ_ℓ f_ℓ(u) K_ℓ(t,u)   u.

To apply multivariate Taylor formula we introduce, for any α=(α_1,…,α_ ℓ)∈(∪{0})^ℓ, the notation |α|=α_1+…+α_ℓ. Moreover we define s_ℓ = m_ℓ+γ_ℓ with m_ℓ∈∪{0} and 0 < γ_ℓ≤ 1.
Since f_ℓ∈_ℓ(s_ℓ,Λ_ℓ), we obtain, using classical arguments <cit.>, that:

    B(ℓ)
         ≤[ 2^ℓ m_ℓ∑_|α|=m_ℓ∏_i=1^ℓ∫_0^1 |(y)| y^α_i+γ_ℓ  y ] Λ_ℓ h_ℓ^s_ℓ
       ≤_ℓ(,s)Λ_ℓ h_ℓ^s_ℓ

where

    _ℓ(,s) = (⌊ s^* ⌋℘(⌊ s^* ⌋))(2_[0,1])^ℓℓ^⌊ s^* ⌋

and ℘(·) denotes the partition function of an integer.
We then obtain:

    ∑_ℓ=1^L_n_ℓ(p)/√(ℓ!)  B(ℓ)
      ≤(⌊ s^* ⌋℘(⌊ s^* ⌋))∑_ℓ=1^L_n(2(p-1)^1/2_[0,1])^ℓ/√(ℓ!)Λ_ℓ h_ℓ^s_ℓ

Since the sequence

    (2(p-1)^1/2_[0,1])^ℓ/√(ℓ!)

tends to 0 as ℓ goes to infinity, there exists an absolute constant C_0>0 that depends only on p, s^* and _[0,1] such that:

    ∑_ℓ=1^L_n_ℓ(p)/√(ℓ!)  B(ℓ) 
      ≤ C_0 
      ∑_ℓ=1^L_nΛ_ℓ h_ℓ^s_ℓ.
 



  
Study of the stochastic term V(ℓ)

Set ℓ∈{1,…,L_n}. We have:

    V(ℓ)
           = (_1(ℓ)+_2(ℓ)/n)^1/2
       ≤(_1(ℓ))^1/2+(_2(ℓ))^1/2/n^1/2

where

    _1(ℓ) =  ∫_Δ_ℓ(ε^2 ξ_ℓ^2(t))   t

and

    _2(ℓ) = ∫_Δ_ℓ(m(W) ξ_ℓ(t) )   t.

Then we have 

    ∑_ℓ=1^L_n_ℓ(p)/√(ℓ!)  V(ℓ) 
      ≤∑_ℓ=1^L_n_ℓ(p)(V_1(ℓ)/nℓ!)^1/2+∑_ℓ=1^L_n_ℓ(p)(V_2(ℓ)/nℓ!)^1/2

Since W and ε are independent Lemma <ref> implies:

    _1(ℓ)
    = ∫_Δ_ℓ(ε^2) (ξ_ℓ^2(t)) dt≤μ_2^2 b_ℓ,1/h_ℓ^ℓ.

Then we have,

    ∑_ℓ=1^L_n_ℓ(p)(V_1(ℓ)/nℓ!)^1/2≤μ_2∑_ℓ=1^L_n a_1^ℓ(1/nh_ℓ^ℓ)^1/2≤μ_2a_1^L_n∑_ℓ=1^L_n(1/nh_ℓ^ℓ)^1/2

where

    a_1=(2(p-1))^1/2_[0,1]≥ 1.

Now we have:

    _2(ℓ)
           = ∫_Δ_ℓ(∑_k=1^L_n1/k!Θ_kξ_ℓ(t))^2   t
       =∑_k,k'=1^L_n1/k!k'!∫_Δ_ℓ(Θ_kΘ_k'ξ_ℓ^2(t))   t.

Using Cauchy-Schwarz inequality we obtain:

    _2(ℓ)
        ≤∑_k,k'=1^L_n(Θ_k^4)^1/4(Θ_k'^4)^1/4/k!k'!∫_Δ_ℓ(ξ_ℓ^4(t))^1/2  t.

Now, using Lemma <ref> and that m∈(s,Λ,γ,M), 
we obtain:

    _2(ℓ)
       ≤(∑_k^L_n1/√(k!)_k(4) f_k_Δ_k)^2b_ℓ,2h_ ℓ^-ℓ
       ≤ M^2 (∑_k=1^∞e^-2γ k_k^2(4))b_ℓ,2h_ ℓ^-ℓ.

This implies that 

    ∑_ℓ=1^L_n_ℓ(p)(V_2(ℓ)/nℓ!)^1/2≤ a_2∑_ℓ=1^L_n C_2^ℓ(1/nh_ℓ^ℓ)^1/2≤ a_2C_2^L_n∑_ℓ=1^L_n(1/nh_ℓ^ℓ)^1/2

where

    C_2=√(3)a_1
        and    
    a_2=M  (∑_k=1^∞e^-2γ k_k^2(4))^1/2<∞.

Note that a_2 is finite since 2γ>log(3)
Combining (<ref>) and (<ref>), we obtain, denoting C_1= a_2+μ_2, that 

    ∑_ℓ=1^L_n_ℓ(p)/√(ℓ!)  V(ℓ) 
        ≤ C_1 C_2^L_n∑_ℓ=1^L_n(1/nh_ℓ^ℓ)^1/2.




  
General bound on the risk

Combining (<ref>), (<ref>) and (<ref>), the following bound can be easily obtained:

    R_p(m̂_𝐡,L_n, m)
      ≤   κ_0 n^-1/2 + C_0 
      ∑_ℓ=1^L_nΛ_ℓ h_ℓ^s_ℓ+ C_1 C_2^L_n∑_ℓ=1^L_n(1/nh_ℓ^ℓ)^1/2
           +∑_ℓ=L_n+1^+∞_ℓ(p)/√(ℓ!)f_ℓ_Δ_ℓ




  
Study of the residual term

Finally we have using that m∈(s,Λ,γ,M)

    ∑_ℓ=L_n+1^+∞_ℓ(p)/√(ℓ!)f_ℓ_Δ_ℓ   ≤ψ_γ(L_n)
      (∑_ℓ=L_n+1^+∞ (p-1)^ℓe^-2γℓ)^1/2
       ≤ψ_γ(L_n)/√(2γ_p) e^-γ_p L_n

where

    γ_p = γ - log(p-1)/2  and  ψ_γ(L) = ( ∑_ℓ=L+1^+∞f_ℓ_Δ_ℓ^2 e^2γℓ/ℓ!)^1/2.

Note that ψ_γ(L) tends to 0 as L tends to infinity. 



  
Upper bound

Using the definitions of L_n and h_ℓ we have 

    R_p(m̂_𝐡,L_n, m)
      ≤κ_0 n^-1/2 + (C_0+C_1) ∑_ℓ=1^L_nΛ_ℓ^2ℓ/(2s_ℓ+ℓ)( C_2^2L_n/n)^s_ℓ/2s_ℓ+ℓ
      + ψ_γ(L_n)/√(2γ_p) e^-γ_p L_n.

Now, remark that, since γ_p>s^* and C_2>1,
we have 

    e^-γ_p L_n   ≤   e^-s^* L_n≤ e^s^*e^-s^*(log n)^1/2=e^s^*e^-s^*log n/L_n[L_n+1/(log n)^1/2-1/(log n)^1/2]
       ≤ e^3s^*e^-s^*log n/L_n≤ e^3s^* e^-s^*log n/2s^*+L_n=e^3s^*n^-s^*/2s^*+L_n≤ e^3s^*( C_2^2L_n/n)^s^*/2s^*+L_n.

Then there exists a positive constant a_4 that depends on Λ_* and s^* such that

    e^-γ_p L_n≤ a_4 ∑_ℓ=1^L_nΛ_ℓ^2ℓ/(2s_ℓ+ℓ)( C_2^2L_n/n)^s_ℓ/2s_ℓ+ℓ.

This implies that

    R_p(m̂_𝐡,L_n, m) 
      ≤ (C_0+C_1) ∑_ℓ=1^L_nΛ_ℓ^2ℓ/(2s_ℓ+ℓ)( C_2^2L_n/n)^s_ℓ/2s_ℓ+ℓ + ρ_n

where ρ_n is a negligeable reminder term.



 §.§ Proof of Theorem <ref>


This proof is decomposed into two parts. We first prove the upper bound (<ref>) and then the lower bound (<ref>).



  §.§.§ Proof of the upper bound


For the sake of readability we denote h̃=h̃_n and h_ℓ=h̃^(ℓ)_n(s,Λ). Following the same notations as in the proof of Theorem <ref>, we have

    R_p(m̂_h̃,L,m)
        ≤(|Y̅_n-(Y)|^p)^1/p+ ∑_ℓ=1^L_ℓ(p)/√(ℓ!)(B(ℓ) + V(ℓ)).

Note that in this case there is no residual term.
Similarly to the proof of Theorem <ref>, and using the same notations, we have

    (|Y̅_n-(Y)|^p)^1/p≤κ_0n^-1/2,

with κ_0 depending on M, μ_2 and μ_p.
The bias term  satisfies

    ∑_ℓ=1^L_ℓ(p)/√(ℓ!) B(ℓ)
      ≤ 
      C_0 ∑_ℓ=1^L Λ_ℓ h_ℓ^s_ℓ
 and the stochastic term satisfies

    ∑_ℓ=1^L_ℓ(p)/√(ℓ!)  V(ℓ)≤ C_4 ∑_ℓ=1^L (1/nh_ℓ^ℓ)^1/2,

where C_4 depends on L, M, and μ_2.
Now  by  substituting h_ℓ by its value, we obtain

    R_p(m̂_h̃,L,m)
           ≤κ^* max{Λ_ℓ^2ℓ/(2s_ℓ+ℓ) n^-s_ℓ/(2s_ℓ+ℓ)|ℓ=1,…,L}

where κ^* is a positive constant that depends only on L, M μ_2, μ_p, Λ_* and .
This ends the proof of the upper bound. Now, let us prove the lower bound.



  §.§.§ Proof of the lower bound


Note that for any p≥2 and any estimator m̃_n of m we have R_p(m̃_n, m) ≥ R_2(m̃_n, m). This implies that, to prove the lower bound, it is sufficient to consider the case p=2.




  
Method.

We fix s∈(0,)^L, Λ∈(0,+∞)^L and M>0. To prove the lower bound over the space (s,Λ,L,M), we define

    ℓ = _k=1,…,LΛ_k^2k/(2s_k+k)n^-s_k/(2s_k+k)

and we follow the strategy developed by <cit.>. In particular Lemma 6.1 of this paper implies (using Itô's isometry combined with Theorem 2.5 in <cit.>) that the problem boils down to find a finite family of functions {g_ω}_ω∈𝒲 with cardinal |𝒲|≥ 2 that satisfies the following assumptions:


  (i) the null function 0∈{g_ω}_ω∈𝒲.
    
  (ii) for any ω∈𝒲, the function g_ω∈_ℓ(s_ℓ, Λ_ℓ, M) and g_ω_Δ_ℓ^2≤ℓ! M^2
    
  (iii) there exists κ_*>0 such that for ω≠ω', g_ω-g_ω'_Δ_ℓ≥ 2κ_*ϕ_n(s,Λ)
    
  (iv) there exists 0<α<1/8 such that
    
    n/|𝒲|∑_ω∈𝒲 g_ω_Δ_ℓ^2≤ 2αlog(|𝒲|).


Under these assumptions, the lower-bound (<ref>) holds for p=2.





  
Notation. Here, we construct a finite set of functions used in the rest of the proof. We consider the function ψ:→ defined, for any u∈ by

    ψ(u)=exp(-1/(1-u^2))_(-1,1)(u).

This function is in L_2 and we denote ψ^2=∫_ψ^2(x) dx < 2. 
Note that, since the function ψ is infinitely differentiable with compact support,  we have:

    λ^* = max_0 < s <sup_x≠ y|ψ^(⌊ s⌋)(x) - ψ^(⌊ s⌋)(y)|/|x-y|^s < +∞.


Now we consider 0<α<1/8,

    κ_*^2=1/32(ψ^2/2)^ L
    c_1^2
 and

    c_1=min{ 1, M, (2Lλ^*)^-1, (αlog(2)/8· 2^L)^1/2}.





We consider the bandwidth

    h = (1/Λ_ℓ^2  n)^1/2 s_ℓ+ℓ

and we set R=1/(2h). We assume, without loss of generality, that R is an integer and nh^ℓ≥ 1. Let ℛ={0,…,R-1}^ℓ and define, for any r=(r_1,…,r_ℓ)∈ℛ, the function ϕ_r:Δ_ℓ→ by:

    ϕ_r(y) = ∏_i=1^ℓψ(y_i-x_i^(r)/h).

where x_i^(r)=(2r_i+1)h. Finally, for any w:ℛ→{0,1} we define:

    g_w = ρ_n∑_r∈ℛ w(r) ϕ_r

where

    ρ_n = c_1 Λ_ℓ h^s_ℓ = c_1 1/√(nh^ℓ).




  
Proof of (ii).
Set w:ℛ→{0,1}. The following property can be readily verified:

    g_w_Δ_ℓ^2 = |w| ψ^2ℓρ_n^2 h^ℓ    where    
        |w| = (∑_r∈ w(r)) ≤ R^ℓ = 1/2^ℓ h^ℓ.

This implies that

    g_w_Δ_ℓ^2 ≤(ψ^2/2)^ℓρ_n^2≤ c_1^21/nh^ℓ≤ℓ! M^2.

Moreover note that, for any y∈Δ_ℓ and α=(α_1,…,α_ℓ) such that |α|=⌊ s_ℓ⌋, we have:

    D^αϕ_r(y)
           =
        1/h^|α|∏_i=1^ℓψ^(α_i)(y_i - x_i^(r)/h)

which implies that, for any z∈Δ_ℓ we have

    |D^αϕ_r(y)-D^αϕ_r(z)|
       ≤ψ_∞^ℓ-1/h^|α|∑_i=1^ℓ|
        ψ^(α_i)(y_i-x_i^(r)/h) - ψ^(α_i)(z_i-x_i^(r)/h)
    | 
       ≤λ^*/h^s_ℓ∑_i=1^ℓ |y_i-z_i|^s_ℓ - ⌊ s_ℓ⌋
       ≤ℓλ^*/h^s_ℓ |y-z|^s_ℓ - ⌊ s_ℓ⌋

This also implies, since the function ψ vanishes outside (-1,1), that

    |D^α g_w(y)-D^α g_w(z)|
       ≤(2ℓλ^*)
    ρ_n/h^s_ℓ|y-z|^s_ℓ - ⌊ s_ℓ⌋
       ≤ c_1(2ℓλ^*) Λ_ℓ|y-z|^s_ℓ - ⌊ s_ℓ⌋
       ≤Λ_ℓ|y-z|^s_ℓ - ⌊ s_ℓ⌋.

Using (<ref>), we deduce that  g_w belongs to _ℓ(s_ℓ, Λ_ℓ). Combining with (<ref>), (ii) is fulfilled.



  
Proof of (i) and (iii). Using Lemma 2.9 of <cit.>, there exists a set ⊂{w:→{0,1}} such that the null function belongs to , log_2 ||≥ R^ℓ/8 and

    ∀ w≠ w'∈ ,  ∑_r∈ℛ| w(r) -w'(r)|≥ R^ℓ/8.

Let w, w'∈ such that w≠ w'. We have

    g_w-g_w'^2_Δ_ℓ=   ρ_n^2∑_r∈(w(r)-w'(r))^2 ϕ_r_Δ_ℓ^2
    
        =   ρ_n^2∑_r∈|w(r)-w'(r)| h^ℓψ^2ℓ
    ≥   ρ_n^2h^ℓψ^2ℓ R^ℓ/8
    ≥   1/8(ψ^2/2)^ L
    c_1^2Λ_ℓ^2h^2s_ℓ
    ≥    4 κ_*^2 ϕ_n^2(s,Λ).

Then Assumptions (i) and (iii) are fulfilled.



  
Proof of (iv).
Using (<ref>), we deduce that using the definition of c_1

    n/|𝒲|∑_ω∈𝒲 g_ω_Δ_ℓ^2≤    c_1^2h^-ℓ
    ≤   α/8(2h)^-ℓlog(2)
    ≤   αlog ||.

Then Assumption (iv) is fulfilled.



 §.§ Proof of Theorem <ref> 


We have using (<ref>) and (<ref>) that

    (  |m̂_L(W)-m(W)|^p )^1/p   ≤κ_0n^-1/2+∑_ℓ=1^L _ℓ(p)/√(ℓ!)(f̂_ĥ_ℓ^(ℓ)-f_ℓ^2_Δ_ℓ)^1/2.

 Let ℓ∈{1,…,L}. Let h∈_ℓ.
 We have
 
    f̂_ĥ_ℓ^(ℓ)-f_ℓ_Δ_ℓ   ≤f̂_h^(ℓ)-f_ℓ_Δ_ℓ+ f̂_ĥ_ℓ∨ h^(ℓ)-f̂_ h^(ℓ)_Δ_ℓ+f̂_ĥ_ℓ^(ℓ)-f̂_ĥ_ℓ∨ h^(ℓ)_Δ_ℓ
       ≤f̂_h^(ℓ)-f_ℓ_Δ_ℓ+ B(ℓ,h)+M(ℓ,ĥ_ℓ)
            + B(ℓ,ĥ_ℓ)+ M(ℓ,h)+ 2M(ℓ,ĥ_ℓ∨ h)
       ≤f̂_h^(ℓ)-f_ℓ_Δ_ℓ +4(B(ℓ,h)+M(ℓ,h)).

Then we have

    (f̂_ĥ_ℓ^(ℓ)-f_ℓ^2_Δ_ℓ)^1/2   ≤(f̂_h^(ℓ)-f_ℓ^2_Δ_ℓ)^1/2+ 4 M(ℓ,h) + 4(  B^2(ℓ,h) )^1/2.

Note that we have

    (f̂_h^(ℓ)-f_ℓ^2_Δ_ℓ)^1/2   ≤f̂_h^(ℓ)-f_ℓ_Δ_ℓ + V(ℓ)
       ≤f̂_h^(ℓ)-f_ℓ_Δ_ℓ + √(V_1(ℓ))+√(V_2(ℓ))/√(n)
       ≤ C( f̂_h^(ℓ)-f_ℓ_Δ_ℓ+M(ℓ,h)),

where we use the properties of V_1(ℓ) and V_2(ℓ) stated in page eq:V2. In the following, we will demonstrate that

    (  B^2(ℓ,h) )^1/2≤ C(max_h'≤ hf̂_h'^(ℓ)-f_ℓ_Δ_ℓ+ M(ℓ,h))+ O(√(log n/n)).


Combining (<ref>) with (<ref>) and (<ref>), we obtain that

    (f̂_ĥ_ℓ^(ℓ)-f_ℓ^2_Δ_ℓ)^1/2≤ C(max_h'≤ hf̂_h'^(ℓ)-f_ℓ_Δ_ℓ+ M(ℓ,h))+ O(√(log n/n)).

Theorem <ref> is then a direct consequence of the above inequality and (<ref>).



  
Proof of (<ref>)

Now let us control B(ℓ,h) for h∈_ℓ. We have

    B(ℓ,h)   =max_h'≤ h{f̂_h'^(ℓ)-f̂_h∨ h'^(ℓ)_Δ_ℓ -M(ℓ,h') -M(ℓ, h∨ h')  }_+
       ≤max_h'≤ h{{f̂_h'^(ℓ)-f̂_ h'^(ℓ)_Δ_ℓ -M(ℓ,h')}_+} + {f̂_h^(ℓ)-f̂_ h^(ℓ)_Δ_ℓ -M(ℓ,h)}_+ 
            +2max_h'≤ hf̂_ h'^(ℓ)-f_ℓ_Δ_ℓ.

Then

    (  B^2(ℓ,h) )^1/2≤ 2max_h'≤ hf̂_ h'^(ℓ)-f_ℓ_Δ_ℓ+ 2(#(_ℓ) max_h'≤ hA_ℓ(h')   )^1/2

where

    A_ℓ (h') = [{f̂_h'^(ℓ)-f̂_h'^(ℓ)_Δ_ℓ-M(ℓ,h')}_+^2].

We have

    f̂_h'^(ℓ)-f̂_h'^(ℓ)
          = ∑_k=1^L _k,ℓ + _ℓ

where

    k!_k,ℓ(t) =
         1/n∑_i=1^n{Θ_i,kξ_i,ℓ(t,h')
        - (
        Θ_i,kξ_i,ℓ(t,h')
        )
        }

and

    _ℓ(t) =
        1/n∑_i=1^n
        ε_i ξ_i,ℓ(t,h').

Using these notations we have:

    A_ℓ(h') ≤ 2^L+1(
        ∑_k=0^L {_k,ℓ_Δ_ℓ-T(k,ℓ)}_+^2
        + {_ℓ_Δ_ℓ-T(ℓ)}_+^2
        ),

where

    T(ℓ)=μ_4(1+4√(log (1/(h')^ℓ)))/√(n(h')^ℓ)

and

    T(k, ℓ)=_4(k)Mk!(1+4√(log (1/(h')^ℓ)))/√(n(h')^ℓ).

Now note that

    (ε_i^4)^1/4=μ_4<∞

and  for k∈{0,…,L}

    (Θ_i,k^4)^1/4≤_4(k)Mk!.

Using Lemma <ref> with =_ℓ (respectively =k! _k,ℓ), T=T(ℓ)  (respectively T=T(k,ℓ)) and χ_i=ε_i (respectively χ=Θ_i,k), we deduce that for all for h'≤ h

    A_ℓ(h')≤ C n^-1.

This implies that

    #(_ℓ) max_h'≤ hA_ℓ(h') ≤ C (log n)n^-1.

Now (<ref>) and (<ref>) entail (<ref>).



 §.§ Proof of Theorem <ref>.


Let s=(s_1,…,s_L)∈ [,)^L, Λ∈(0,+∞)^L, M>0 and f∈_s,Λ, L, M. Define for ℓ∈{1,…,L}

    k_ℓ=⌊1/2s_ℓ+ℓlog(n/log n)⌋.

For n large enough, we have h_ℓ=e^-k_ℓ∈_ℓ. Using  (<ref>) and (<ref>), Theorem <ref> implies that

    R_2(m̂_L, m)
             ≤ C∑_ℓ=1^L {max_h'≤ h_ℓ (h')^s_ℓ + M(ℓ, h_ℓ)  } + Υ_2(log n/n )^1/2
       ≤  C∑_ℓ=1^L {h_ℓ^s_ℓ + (log n/nh_ℓ^ℓ)^1/2} + Υ_2(log n/n)^1/2
       ≤ Cϕ̃_n(s) + Υ_2(log n/n)^1/2
       ≤ Cϕ̃_n(s).

where C is a constant that changes from line to line and depends on s, Λ and M. Since C does not depend on m∈(s,Λ,L,M), this ends the proof.



 §.§ Proof of Lemma <ref>.


We have

    ( |ξ_ℓ(t,h)|^2r)^1/r   ≤_ℓ^2(2r) |ξ_ℓ(t,h)|^2
       ≤_ℓ^2(2r) ℓ !∫_Δ_ℓ(K_h^(ℓ)(t,u))^2  u
       ≤_ℓ^2(2r) ℓ ! 2^ℓ_[0,1]^2ℓh^-ℓ
       ≤ b_ℓ,rh^-ℓ.


Moreover we have

    (|ξ_ℓ(t,h)|^r_|ξ_ℓ(t,h)| > φ)
           ≤( |ξ_ℓ(t,h)|^2r)^1/2((|ξ_ℓ(t,h)| > φ))^1/2
       ≤ (b_ℓ,rh^-ℓ)^r/2( |ξ_ℓ(t,h)|^2q/φ^2q)^1/2
       ≤ (b_ℓ,r)^r/2 (b_ℓ,q)^q/2 h^-ℓ(r+q)/2φ^-q.





 §.§ Proof of Lemma <ref>

In this proof, C is a positive constant that changes of value from line to line. 
Since h is fixed, we simplify the notation and use in the proof ξ_i,ℓ(t,h)=ξ_i,ℓ(t) and ξ_ℓ(t,h)=ξ_ℓ(t). Now, we have for k≥ 1:

    (t)       = η̅_0(t) + η_1(t) + η_2(t) + η_3(t)

where

    η̅_0(t)    = η_0(t) - η_0(t)
            η_0(t) =  1/n∑_i=1^n χ_i_|χ_i| ≤φ(n)ξ_i,ℓ(t) _|ξ_i,ℓ| ≤ψ(n)
    η_1(t)    = η_0(t) - (χξ_ℓ(t))
    η_2(t)    = 1/n∑_i=1^n χ_i_|χ_i| > φ(n)ξ_i,ℓ(t)
    η_3(t)    = 1/n∑_i=1^n χ_i_|χ_i| ≤φ(n)ξ_i,ℓ(t) _|ξ_i,ℓ(t)| > ψ(n).

where for any ℓ=1,…, L:

    ψ(n)=n^α  with  α = α(,ℓ) =+ℓ/4+2ℓ

and

    φ(n) = n^β  with  β = 1/2 - α > 0.

Note that both α and β are positive numbers.


  §.§.§ Control of η_1


We have

    |η_1(t)|    = |-( χξ_ℓ(t)_|χ|>φ(n))- ( χξ_ℓ(t)_|χ|≤φ(n)_|ξ_ℓ(t)|> ψ(n))|
       ≤ ( |χξ_ℓ(t)|_|χ|>φ(n))+ ( |χξ_ℓ(t)|_|χ|≤φ(n)_|ξ_ℓ(t)|> ψ(n))

Note that we have using Cauchy-Schwarz and Markov inequality

    ( |χξ_ℓ(t)|_|χ|>φ(n))    ≤((|χ|^2_|χ|>φ(n))  |ξ_ℓ(t)|^2)^1/2
       ≤(b_ℓ,1)^1/2 h^-ℓ/2 a_2 ((|χ|>φ(n))  )^1/4
       ≤(b_ℓ,1)^1/2 h^-ℓ/2  a_2  ((|χ|^8/β))^1/4
       ≤(b_ℓ,1)^1/2 h^-ℓ/2  a_2  (a_4/β)^2/β  (φ(n))^-2/β
       ≤Cn^-1.




Moreover since h∈_ℓ, using Lemma <ref> with q=8+6ℓ/, we have

    (( |χξ_ℓ(t)|_|χ|≤φ(n)_|ξ_ℓ(t)|> ψ(n)))^2≤    a_1^2 (|ξ_ℓ(t)|^2_|ξ_ℓ(t)|> ψ(n))
       ≤a_1^2 (b_ℓ,2) (b_ℓ,q)(ψ(n))^-qh^-ℓ(2+q)/2
       ≤ Cn^-2.

Now using (<ref>), (<ref>) and (<ref>), we finally obtain

    (η_1_Δ_ℓ^2) ≤ Cn^-1.




  §.§.§ Control of η_2


Define

    μ_n(t) = (χ_|χ| > φ(n)ξ_ℓ(t)).

We have

    (η_2_Δ_ℓ^2)
           ≤ 2(A + B)

where using Lemma <ref> with q=4/β

    A     = 1/n∫_Δ_ℓ(χ_|χ| > φ(n)ξ_ℓ(t)-μ_n(t))^2 dt
       ≤1/n∫_Δ_ℓ(
            (χ^4 _|χ| > φ(n))
            ξ_ℓ^4(t)
        )^1/2 dt 
       ≤ n^-1 a_4^2 ( |χ|>φ(n) )^1/4 b_ℓ,2h^-ℓ
       ≤ a_4^2 b_ℓ,2( |χ|>φ(n) )^1/4
       ≤ a_4^2 b_ℓ,2 (a_q/2)^q/4(φ(n))^-q/4
       ≤ Cn^-1

and following (<ref>)

    B
        = 1/n∫_Δ_ℓμ_n^2(t) dt≤ Cn^-1.






  §.§.§ Control of η_3


Define

    ν_n(t) = (χ_|χ| ≤φ(n)ξ_ℓ(t)_|ξ_ℓ(t)| > ψ(n)).

We have

    (η_3_Δ_ℓ^2)
           ≤ 2(A + B)

where

    A = 1/n∫_Δ_ℓ(χ_|χ| ≤φ(n)ξ_ℓ(t)_|ξ_ℓ(t)| > ψ(n)-ν_n(t))^2 dt

and

    B
           = 1/n∫_Δ_ℓν_n^2(t) dt

Note that using similar arguments as above with q=(8+4ℓ)/,

    A
           ≤1/n∫_Δ_ℓ(
            (χ^4 )
            (ξ_ℓ^4(t) _|ξ_ℓ(t)| > ψ(n))
        )^1/2 dt 
       ≤ a_2^2b_ℓ,4 (b_ℓ,q)^q/4 (ψ(n))^-q/2 h^-ℓ q/4
       ≤ C n^-1

and following (<ref>) B≤ Cn^-1.






  §.§.§ Control of η̅_0


We have to bound

    {η̅_0_Δ_ℓ - T }_+^2
           ≤∫_0^+∞(η̅_0_Δ_ℓ - T > √(u)) du
       ≤ 2∫_0^+∞ u(η̅_0_Δ_ℓ > u + T ) du.

Note that, using duality arguments, there exists  a countable set  of functions s∈^2(Δ_ℓ) such that s_Δ_ℓ≤ 1 and

    η̅_0_Δ_ℓ   = sup_s ∈∫_Δ_ℓ s(t)η̅_0(t) dt
       = 2φ(n)ψ(n)/n Z=2Z/√(n)

where

    Z = sup_s∈∑_i=1^n X_i,s

and, for s∈, we have:

    X_i,s = ∫_Δ_ℓ s(t)X_i(t) dt
            with    
        X_i(t) = g_i(t)- g_i(t)

and

    g_i(t) = 1/2√(n)χ_i _{|χ_i|≤φ(n)}ξ_i,ℓ(t)_{|ξ_i,ℓ(t)|≤ψ(n)}.


Note that we have both (X_i,s) = 0 and X_i,s_∞≤ 1. Now, let us control:

    v = 2(sup_s∈∑_i=1^n X_i,s) +  nsup_s∈ X_1,s^2.

Using Cauchy-schwarz's inequality and Fubini's theorem we obtain:

    X_1,s^2
           ≤(∫_Δ_ℓ s^2(t) dt ) (∫_Δ_ℓ X_i^2(t) dt )
       ≤∫_Δ_ℓ g_i^2(t) dt

We have

    ∫_Δ_ℓ g_i^2(t) dt
           ≤a_2^2b_ℓ,2/4nh^ℓ

and

    (sup_s∈∑_i=1^n X_i,s)
           = ∑_i=1^n X_i(·)_Δ_ℓ
       ≤(∑_i=1^n X_i(·)_Δ_ℓ^2)^1/2
       ≤1/4(∫_Δ_ℓχ^2ξ^2_ℓ(t) t)^1/2
       ≤a_2√(b_ℓ,2)/4h^ℓ/2


Combining the previous results we have:

    v ≤θ_n+√(θ_n)  with  θ_n = a_2^2b_ℓ,2/4h^ℓ.

Define

    T = (1+δ) a_2√(b_ℓ,2)/2√(nh^ℓ)=(1+δ)√(θ_n/n).

We have:

    (η̅_0_Δ_ℓ > u + T)
           ≤(Z -  Z > √(n)u/2 + δ/2√(θ_n)).

Define:

    = δ√(nθ_n)/2,
           = δ^2 θ_n/4,
           = √(n)/3,
         = 2θ_n + 2√(θ_n) (1 +δ/6).

Using Bousquet's inequality we have:

    (Z -  Z > √(n)u/2 + δ/2√(θ_n))
           ≤ C_n(u) D_n(u)

where

    C_n(u) = exp(-nu^2/4( u+ ))

and

    D_n(u) = exp(- u + / u + ).


Since ->0, we have, D_n(u)≤ D_n(0), that is:

    D_n(u) ≤exp(-/)≤exp(-δ^2θ_n/4(2θ_n + 2√(θ_n) (1 +δ/6))).

Since h∈_ℓ, for n large enough (1+δ/6) ≤√(θ_n) we have D_n(u) ≤ h^ℓ. Moreover we have doing the change of variables v=√(nh^ℓ)u

    {η̅_0_Δ_ℓ - T}_+^2
           ≤ 2∫_0^+∞ u(η̅_0_Δ_ℓ > u + T) du
       ≤ 2h^ℓ∫_0^+∞ u C_n(u) du
       ≤C/n∫_0^+∞ v exp(- Cv^2/1+v) v.

This implies that:

    {η̅_0_Δ_ℓ - T}_+^2
           ≤ C n^-1.


Combining results of Sections <ref>, <ref>, <ref> and <ref>, we obtain (<ref>)




§ ACKNOWLEDGEMENTS

The authors have been supported by Fondecyt projects 1171335 and 1190801, and Mathamsud projects 19-MATH-06 and 20-MATH-05.

plainnat



