this article we have developed the SAGD framework for optimizing objective functions that can be expressed as a mathematical expectation with intractable gradients. SAGD uses the Langevin algorithm to construct an approximate gradient in each iteration, whose accuracy is carefully controlled. Theoretical analysis shows that SAGD has the same convergence property as SGD, and more importantly, all the hyperparameters of SAGD are transparent so that the algorithm can be practically implemented. We have successfully applied SAGD to both the automated EM algorithm and the debiased VAE. To summarize, SAGD is an alternative to the ordinary SGD in a broader realm, and it is hoped that SAGD can be used to solve more statistical and machine learning problems both efficiently and reliably.

We mention two future directions for the research on SAGD. First, one might be interested in improving the Langevin algorithm, as the assumptions we have made are mild yet not the weakest. A second direction is to study the convergence of SAGD combined with various acceleration techniques, such as the momentum methods.

Appendix
========

Techincal Lemmas
----------------

In this section we provide a number of inequalities and lemmas that are useful for other theorems. First define the function $$\Gamma_{c}(x,y)=\frac{\gamma^{2}}{4}\Vert 