is unclear what is the value of $$\var \pth{\grad_w L_{\calB_t}(w_t)} \triangleq \mathbb{E} \norm{\grad_w L_{\calB_t}(w_t)}^2 - \norm{\mathbb{E} \grad_w L_{\calB_t}(w_t)}^2.$$

Intuitively, we should have $$\label{eq:sgVariancePropto}
        \var \pth{\grad_w L_{\calB_t}(w_t)} \; \propto \; \frac{n^2}{b} \var \pth{\grad_w L(w_t)}$$ where $n$ is the number of training samples and stochasticity on the right-hand side comes from mini-batch samples behind $w_t$. The works in [@smith2017bayesian; @gower2019sgd] also point out this relationship, but a rigorous proof is missing. In addition, even the quantities $\grad_w L(w_t)$ and $\var \pth{\grad_w L(w_t)}$ are still challenging to compute as we do not have direct formulas of their precise values. Besides, as we choose different $b$’s, their values are not comparable as we end up with different $w_t$’s.

A plausible idea to address these issues is to represent $\mathbb{E} \grad_w L_{\calB_t}(w_t)$ and $\var \pth{\grad_w L_{\calB_t}(w_t)}$ using the fixed and known quantities $w_0, b, t$, and $\alpha_t$. In this way, we can further discover the properties, like decreasing with respect to $b$, of $\mathbb{E} \grad_w L_{\calB_t}(w_t)$ and $\var \pth{\grad_w L_{\calB_t}(w_t)}$. The biggest challenge is how to connect the quantities in iteration $t$ with those of iteration $0$. This is similar to discovering the properties of a stochastic differential equation at time $t$ given only the 