= N_0 + N_1 \frac{1}{b} + \cdots + N_q \frac{1}{b^q},
            \end{aligned}$$ where $N_k = \sum_{i=1}^{m_k} \prod_{j=1}^{m_{ki}} \trace{M_{ij}^k} M_{i0}^k, k\in \fromto{0}{q}  $. Here $m_k, m_{ki}$ and $q \le \frac{1}{2}(3^{t+1}-1)d + \frac{1}{2}(3^{t}-1)d'$ are constants independent of $b$, and $\sum_{j=0}^{m_{ki}} \deg\pth{M_{ij}^k} \le 3^{t} (3d + d')$.

By changing the role of parameter and constant matrices we obtain the following corollary.

\[thm:representationoCorollary\] Given $t \ge 0$, for any multiplicative terms $M_i, i \in \fromto{0}{m} $ of parameter matrices $\sth{W_{t,1}^b, W_{t,2}^b, \calW_t^b}$ and constant matrices $\sth{W_1^*, W_2^*}$ such that $\sum_{i=1}^2 \deg \pth{W_{t,i}^b; M} = d$ and $\deg \pth{\calW_t^b; M} = d'$, we denote $M = \prod_{i=1}^m \trace{M_i} M_0$. There exists a set of multiplicative terms $\sth{M_{ij}^k, i\in [m_k], j \in \fromto{0}{m_{ki}}, k \in \fromto{0}{q}}$ of parameter matrices $\sth{W_{0,1}^b, W_{0,2}^b}$ and constant matrices $\sth{W_1^*, W_2^*}$ such that $$\begin{aligned}
                \condexp{}{M}{\calF_0} = N_0 + N_1 \frac{1}{b} + \cdots + N_q \frac{1}{b^q},
            \end{aligned}$$ where $N_k = \sum_{i=1}^{m_k} \prod_{j=1}^{m_{ki}} \trace{M_{ij}^k} M_{i0}^k, k\in \fromto{0}{q}  $. Here $m_k, m_{ki}$ and $q \le 3^{t}\pth{d + 2d'}$ are constants independent of $b$, and $\sum_{j=0}^{m_{ki}} \deg\pth{M_{ij}^k} \le 3^{t} \pth{d + 2d'}$.

As a special case of Theorem \[thm:representation\], Theorem \[thm:varianceRepresentation\] shows that the variance of the stochastic gradient estimators is also a polynomial of $\frac{1}{b}$ 