is readily guaranteed. Finally, one only needs to create an outer loop to iteratively update the $\{\theta_{k}\}$ sequence, until some convergence condition is met.

Debiased VAE {#subsec:vae_bias_correction}
------------

The automated EM algorithm can be further used to improve the popular VAE model [@kingma2014stochastic]. VAE has the same goal of seeking the maximum likelihood estimator for $\theta$, but it uses the variational Bayes technique to maximize a lower bound of $\ell(\theta;x)$. Let $q(u|x)$ be any conditional density function, and then VAE maximizes the function $\tilde{\ell}(\theta;x)$, defined by $$\tilde{\ell}(\theta;x)=\mathbb{E}_{u\sim q}[\log p(x|u;\theta)]-\mathrm{KL}[q(u|x)\Vert p(u)],\label{eq:elbo}$$ where $p(u)$ is the marginal density of $u$, and $p(x|u;\theta)$ is the conditional distribution of $X$ given $U=u$. In most VAE settings, $U\sim N(0,I)$, and $q(u|x)$ is taken to be a normal distribution whose mean and variance parameters are represented by a deep neural network. VAE has been successfully applied to many problems, but its most critical weakness is that VAE does not maximize the exact log-likelihood, which induces a bias in the final $\theta$.

Here we show that using the SAGD framework, the bias of VAE can be removed via an additional refining step. First, it is easy to show that $\tilde{\ell}(\theta;x)$ has another representation, $\tilde{\ell}(\theta;x)=\ell(\theta;x)-\mathrm{KL}[q(u|x)\Vert p(u|x;\theta)]$. That is, 