but with no constant term. This backs the important intuition that the variance is approximately inversely proportional to the mini-batch size $b$. Besides, note that if we consider $b \goto \infty$, intuitively we should have $\condvar{}{g_{t,i}^b}{\calF_0} \goto 0, i=1,2$. This observation aligns with the statement of Theorem \[thm:varianceRepresentation\].

\[thm:varianceRepresentation\] Given $t \ge 0$, value $\condvar{}{g_{t,i}^b}{\calF_0}, i=1,2$ can be written as a polynomial of $\frac{1}{b}$ with degree at most $2\cdot 3^t$ with no constant term. Formally, we have $$\condvar{}{g_{t,i}^b}{\calF_0} = \beta_1 \frac{1}{b} + \cdots + \beta_r \frac{1}{b^r}, \label{eq:2.7}$$ where $r \le 2\cdot 3^{t+1}$ and each $\beta_i$ is a constant independent of $b$.

Finally, to show the that the variance is a decreasing function of $b$ for large enough $b$, we only need to show that the leading coefficient $\beta_1$ is non-negative. This is guaranteed by the fact that variance is always non-negative. We therefore have Theorem \[thm:varianceDecrease\].

\[thm:varianceDecrease\] Given $t \in \mathbb{N}$, there exists a constant $b_0$ such that for all $b \ge b_0 $ function $\condvar{}{g_{t,i}^b}{\calF_0}, i=1,2$ is a decreasing function of $b$.

In conclusion, we present the relationship between any multiplicative terms of parameter matrices $\sth{g_{t,i}^b, W_{t,i}^b, i=1,2}$ and constant matrices $\sth{W_1^*, W_2^*}$ and the initial weights $W_{0,1}, W_{0,2}$ 