============

The two main ingredients of the proposed SAGD framework are SGD and the Langevin algorithm. SGD has been extensively studied in the literature, and recent research mainly focused on its acceleration, for example variance reduction methods [@johnson2013accelerating; @reddi2016stochastic], adaptive step sizes [@duchi2011adaptive; @zeiler2012adadelta], momentum methods [@kingma2014adam; @luo2019adabound], etc. In this article, the proposed SAGD framework is based on the original version of SGD, but it can be easily adapted to those acceleration methods.

The Langevin algorithm has two variants, the *overdamped* and the *underdamped* versions. Most of the analysis in literature was based on the overdamped version [@dalalyan2017theoretical; @durmus2016high; @durmus2017nonasymptotic; @cheng2017convergence], whereas some recent research suggests that the underdamped version has faster convergence for some special classes of distributions [@cheng2018sharp; @ma2019there]. Due to this reason, we use the underdamped Langevin algorithm to develop the SAGD framework. As a byproduct, we have derived new results for the underdamped Langevin algorithm that complement prior art.

The Langevin algorithm can be compared to MCMC, as they are both useful sampling techniques. In fact, there are MCMC algorithms derived from the Langevin algorithm such as the Metropolis-based overdamped Langevin algorithm [@roberts1996exponential] and the underdamped Langevin MCMC [@cheng2018underdamped]. Nevertheless, the Langevin algorithm has several 