results developed in this article have much broader practical use.

The benefit of our new results is that we can easily verify the assumptions for popular machine learning models. For example, the following corollary justifies the use of Langevin algorithm to sample from deep generative models (e.g. VAE). Consider a single-layer neural network $h(z)=a(Wz+b)$, where $z\in\mathbb{R}^{r}$, $b\in\mathbb{R}^{m}$, $W\in\mathbb{R}^{m\times r}$, and the activation function is $a(x)=\log(1+e^{x})$. Then we have the following result.

\[cor:nn\]Assume that $Z\sim N(0,I_{r})$ and $X|\{Z=z\}\sim N(h(z),\sigma^{2}I)$, where $\sigma^{2}$ is a constant. Let $p(z|x)$ denote the conditional density of $Z$ given $X=x$, and then $V(z)=-\log p(z|x)$ satisfies Assumptions \[assu:v\_boundedness\] and \[assu:v\_dissipative\].

For brevity we omit the multi-layer case, but it can be analyzed similarly. In later part of this article we will discuss the application of SAGD to VAE model in more details.

Stochastic Approximate Gradient Descent
=======================================

With the statistical properties of the underdamped Langevin algorithm studied in Theorem \[thm:control\_bias\_mse\], the SAGD framework can then be readily developed. Recall that our target is to minimize the function $F(\theta)=\mathbb{E}[f(\theta;\xi)]$, whose true gradient $g(\theta)=\mathbb{E}[\nabla f(\theta;\xi)]$ is hard to compute exactly. Using the technique developed in the previous section, we can construct a stochastic gradient, $\tilde{g}(\theta)=K^{-1}\sum_{k=0}^{K-1}\nabla f(\theta;\xi_{k})$, to approximate $g(\theta)$. Unlike 