size $\alpha_{t}=0.2$. For SAGD, Langevin parameters are specified as $\delta_{t}=0.1/\sqrt{t}$ and $K_{t}=t+20$, with the first 100 Langevin iterations discarded as burn-in, similar to that in MCMC. Figure \[fig:path\_loglik\](a) demonstrates the path of $(a,b)$ values on the surface of the true log-likelihood function after three M-steps, and Figure \[fig:path\_loglik\](b) gives the log-likelihood values at each gradient update. Clearly, Figure \[fig:path\_loglik\] shows that the path of SAGD nicely approximates that of exact GD, which further verifies the validity of the SAGD algorithm.

[0.48]{} ![image](true_ll){width="\textwidth"}

[0.36]{} ![image](ll_iter){width="\textwidth"}

Debiased VAE {#debiased-vae}
------------

In the second experiment, we use synthetic data to show that even in the simplest setting, VAE can lead to biased distribution estimation, but its bias can be effectively corrected by SAGD. The observed data are generated as follows: given independent latent variables $Z_{i}\sim\pi(z)$, we set $X_{i}=Z_{i}+e_{i}$, where $e_{i}\sim N(0,1)$ is independent of $Z_{i}$, $i=1,2,\ldots,n$. The target is to recover the unknown latent distribution $\pi(z)$ from $X_{1},\ldots,X_{n}$. In the VAE framework, we first represent $Z$ by a deep neural network transformation $Z=h_{\theta}(U)$, where $U\sim N(0,1)$, and then we have $p(x|u;\theta)\equiv N(h_{\theta}(u),1)$. Once the neural network function $h_{\theta}$ has been learned, we can simulate random variates of $Z=h_{\theta}(U)$ by generating random $U\sim N(0,1)$, 