current parameter values. The full gradient estimate hence consists of individual gradients estimated for different parameter values; the most recently computed gradients are accurate but the ones computed long time ago may correspond to vastly different parameter values. This introduces bias [@saga], but especially towards the convergence the variance of the estimated full batch gradient is considerably smaller than that of the latest mini-batch, speeding up convergence.

METHOD {#sec:method}
======

All gradient-based optimization algorithms follow the same basic pattern of computing a gradient for a mini-batch of samples and updating the parameters. The computational effort required goes almost solely into evaluating the gradient of the loss. To speed up the optimization, we next present a technique that allows computationally lighter evaluation of the gradient in scenarios where the gradient is computed using a Monte Carlo approximation. The presentation here is based on the reparameterization estimate that benefits more off this treatment – as will become evident later – but for completeness we discuss also the score function estimate in Section \[sec:score\].

The Monte Carlo approximation for estimating the gradient depends on the data $x$ and a set of $M$ parameters $z_m$ drawn from the current approximation. As highlighted in , 