the importance sampling estimate would not necessarily provide improvement over careful adaptation of element-wise learning rates.

SCORE FUNCTION ESTIMATE {#sec:score}
-----------------------

Above we discussed the importance sampling estimate from the perspective of the reparameterization estimate. For completeness we also show how it can be applied for the score function estimate, and discuss why it is less useful there.

The Monte Carlo approximation for the score function is obtained directly by drawing samples from the approximation $q(\lambda)$. To approximate the gradient for $q(\lambda')$ we merely use the standard importance sampling equation to obtain the approximation $$\frac{1}{M} \sum_{m=1}^M w_m \left[\log p(x, z_m) - \log
  q_\lambda(z_m)\right] \nabla_{\lambda} \log q_\lambda(z_m),$$ where $w_m = \frac{q(z|\lambda')}{q(z|\lambda)}$. This is still an unbiased estimate, but the computational saving is typically smaller than in the reparameterization case. We do not need to evaluate $\log
p(x, z_m)$ since the samples $z_m$ are kept constant, but all other terms need to be computed again and evaluating the gradient of the approximation is not cheap. This estimate is only useful when the evaluation of the log probability utterly dominates the total computation.

ALGORITHMS {#sec:algorithms}
==========

In the following we describe example optimization algorithms based on the importance sampling idea. The details are 