of the averaged recursive algorithm relies on a fine control of the asymptotic behavior of the Robbins-Monro algorithms, as stated in the following proposition.

\[theol2l4\] If assumptions 1-3(b) hold, there is a positive constant $C'$, and for all $\beta \in \left( \alpha , 2 \alpha\right)$, there is a positive constant $C_{\beta}$ such that for all $n \geq 1$, $$\begin{aligned}
 & \mathbb{E}\left[ \left\| V_{n} - \Gamma_{m} \right\|_{F}^{2}\right] \leq \frac{C'}{n^{\alpha}}, \\
 & \mathbb{E}\left[ \left\| V_{n+1} -  \Gamma_{m} \right\|_{F}^{4}\right] \leq \frac{C''}{n^{\beta}}.\end{aligned}$$

The obtention of an upper bound for the rate of convergence at the order four of the Robbins-Monro algorithm is crucial in the proofs. Furthermore, the following proposition ensures that the exhibited rate in quadratic mean is the optimal one.

Under assumptions 1-3(b), there is a positive constant $c'$ such that for all $n \geq 1$, $$\begin{aligned}
\mathbb{E}\left[ \left\| V_{n} - \Gamma_{m} \right\|_{F}^{2}\right] \geq \frac{c'}{n^{\alpha}}. \end{aligned}$$ \[prop:RMVn\]

Finally, the following theorem is the most important theoretical result of this work. It shows that, in spite of the fact that it only considers the observed data one by one, the averaged recursive estimation procedure gives an estimator which has a classical parametric $\sqrt{n}$ rate of convergence in the Hilbert-Schmidt norm.

\[th:cvgeqm\] Under Assumptions 