for high-dimensional densities the weights $w_m$ easily tend to zero. Variational approximation is, however, often conducted for approximations that factorize over the parameters of the approximation as $q(\lambda) = \prod_{s} q(\lambda_{S_s})$, where $\{S_s\}$ is a partitioning of the parameter vector. The importance sampling estimate can – and should – be done for each factor separately, since the gradient $\nabla_z \log p(x, z_m)$ can be computed for each approximation factor independently even if $\log p(x, z_m)$ itself does not factorize. We show in Section \[sec:dimensionality\] that the technique helps at least until factors consisting of roughly ten parameters.

$\epsilon_m' \leftarrow
  f^{-1}(z_m, \lambda')$ $w_m \leftarrow
  \frac{\phi(\epsilon'_m)}{\phi(\epsilon_m)}$ Calculate $\nabla^i_{\lambda} \mathcal{L}(\lambda)$ using

REPARAMETERIZATION GRADIENT EXAMPLE
-----------------------------------

To further clarify the derivation above, we next illustrate the procedure for the common scenario of Gaussian reparameterization combining $\phi(\epsilon) = \mathcal{N}(0, I)$ with $z = \mu + L
\epsilon$, denoting $\lambda = \{\mu, L\}$. Given a set of $z_m$ drawn from $q(\lambda)$ we want to estimate the gradient evaluated at $\lambda'$ using .

First we compute $\epsilon_m' = f^{-1}(z_m, \lambda') = L^{-1}(z_m -
\mu)$ and evaluate $\phi(\epsilon_m')$ under the standard normal distribution. The weight $w_m$ can then readily be evaluated using , computing $\phi(\epsilon_m)$ as well if 