Finally, Section \[sec:PW\] contrasts our results with previous work in the literature and discusses future directions of research.

Preliminaries {#sec:BM}
=============

In this section, we give the definitions, assumptions and basic facts that will be used throughout the paper. For $\beta>0$, let $\ell$ be the greatest integer strictly less than $\beta$. We denote by ${\cal F_\beta}(L)$ the set of all functions $f:\mathbb{R}^d\to \mathbb{R}$ that are $\ell$ times differentiable and satisfy, for all $x,z \in {\Theta}$ the Hölder-type condition $$\bigg|f(z)-\sum_{0\le |m|\leq \ell} \frac{1}{m!}D^{m}f(x)(z-x)^{m} \bigg|\leq L \|z-x\|^{\beta},
\label{eq:Hclass}$$ where $L >0$, the sum is over the multi-index $m=(m_{1},...,m_{d}) \in \mathbb{N}^d$, we used the notation $m!=m_{1}! \cdots m_{d}!$, $|m|=m_{1}+ \cdots+m_{d}$, and we defined $$D^{m}f(x)\nu^{m} = \frac{\partial ^{|m|}f(x)}{\partial ^{m_{1}}x_{1} \cdots\partial ^{m_{d}}x_{d}}\nu_{1}^{m_{1}} \cdots \nu_{d}^{m_{d}}, \quad \forall \nu=(\nu_1,\dots, \nu_d) \in \mathbb{R}^{d}.$$ In this paper, we assume that the gradient estimator defined by Algorithm \[algo\] uses a kernel function $K:[-1,1]\to \mathbb{R}$ satisfying $$\int K(u) du =0,  \int u K(u) du =1,  \int u^j K(u) du =0, \ j=2,\dots, \ell,~
\int |u|^{\beta} |K(u)| du  <\infty.$$ Examples of such kernels obtained as weighted sums of Legendre polynomials are given in [@PT90] and further discussed in [@BP2016].

\[ass1\] It holds, for all $t \in \{1,\dots,T\}$, that: the random variables $\xi_t$ 