of probability density functions (p.d.f.) on ${\mathsf{Y}}$ with respect to some $\sigma$-finite measure $\eta({\mathrm{d}}y)$, and we assume that $\theta_\star:=\operatorname*{argmax}_{\theta\in\Theta}{\mathbb{E}}[\log f_\theta(Y_1)]$ is well-defined, with ${\mathbb{E}}$ the expectation operator associated to $(\Omega,{\mathcal{F}},{\mathbb{P}})$. We adopt a Bayesian perspective and let $\tilde{\pi}_0$ be a prior distribution for $\theta$ and $
\pi_t({\mathrm{d}}\theta)\propto \prod_{s=1}^t f_\theta(Y_s) \tilde{\pi}_0({\mathrm{d}}\theta)
$ be the posterior distribution associated to observations $\{Y_s\}_{s=1}^t$, for all $t\geq 1$. In this work we consider the problem of approximating $(\pi_t)_{t\geq 1}$ in an online fashion, where by online we mean that the memory and computational requirement to process each observation $Y_t$ is finite and bounded uniformly in $t$.

Current approaches to compute an online approximation of $(\pi_t)_{t\geq 1}$ essentially reduces to online variational methods [see e.g. @broderick2013streaming; @nguyen2017online; @zeno2018bayesian]. However, theoretical justifications for these algorithms are rather limited [see however the recent results obtained by @cherief2019generalization] and there is no guarantee that, as $t$ tends to infinity, their approximation error vanishes. Although not online, the iterated batch importance sampling (IBIS) algorithm of @MR1929161, which approximates $(\pi_t)_{t\geq 1}$ sequentially by Monte Carlo, can be an efficient alternative to online methods in situations where the number of observations to process is not too large and computing $f_\theta(y)$ is cheap 