be replaced by some suitably chosen auxiliary distribution $Q_W$ whenever one wants to numerically evaluate the generalization bounds that we derive later in this section.[^4] In the remainder of this paper, all bounds will be given in terms of $P_W$. Thanks to this choice, many of the terms that appear in our results will be expressible in terms of familiar information-theoretic quantities. However, through repeated references to Remark \[rem:uncond\_conversion\_to\_Q\_bounds\], we will emphasize that the bounds can easily be generalized to the case in which $P_W$ is replaced by an auxiliary distribution $Q_W$.

Average Generalization Error Bounds {#sec:uncond_exp}
-----------------------------------

We now use Theorem \[thm:unconditional\_subgaussian\] to obtain an average generalization error bound of the form given in .

\[cor:exp\_uncond\] Under the setting of Theorem \[thm:unconditional\_subgaussian\], $$\label{eq:cor_exp_uncond}
    \abs{ \Ex{P_{W\! {\boldsymbol{Z}}}}{{{\textnormal{gen}}(W,{\boldsymbol{Z}})}}} \leq \sqrt{\frac{2\sigma^2}{n}I(W;{\boldsymbol{Z}})}.$$

We apply Jensen’s inequality to , resulting in $$\exp\lefto(\lambda\Exop_{P_{W\! {\boldsymbol{Z}}}}\lefto[{{\textnormal{gen}}(W,{\boldsymbol{Z}})}\right] - \frac{\lambda^2\sigma^2}{2n} 
 -\Ex{P_{W\! {\boldsymbol{Z}}}}{{{\imath}(W,{\boldsymbol{Z}})}} \right)  \leq   1.$$ Note that $\Exop_{P_{W\! {\boldsymbol{Z}}}}\lefto[\imath(W, {\boldsymbol{Z}})\right]=\relent{P_{W\!{\boldsymbol{Z}}}}{P_W\!P_{{\boldsymbol{Z}}} }=I(W;{\boldsymbol{Z}})$. By reorganizing terms, we get $$\lambda \Exop_{P_{W\! {\boldsymbol{Z}}}}\lefto[{{\textnormal{gen}}(W,{\boldsymbol{Z}})}\right]-\lambda^2\frac{\sigma^2}{2n}  - \relent{P_{W\!{\boldsymbol{Z}}}}{P_W\!P_{{\boldsymbol{Z}}} } \leq 0.$$ We now set $\lambda=n\Exop_{P_{W\! {\boldsymbol{Z}}}}\lefto[{{\textnormal{gen}}(W,{\boldsymbol{Z}})}\right]/\sigma^2$ to optimize the bound, and thereby get $$\label{eq:exp_uncond_final_line_of_proof}
\Exop^2_{P_{W\! {\boldsymbol{Z}}}}\lefto[{{\textnormal{gen}}(W,{\boldsymbol{Z}})}\right] - \frac{2\sigma^2}{n}\relent{P_{W\!{\boldsymbol{Z}}}}{P_W\!P_{{\boldsymbol{Z}}} }\leq 0$$ from which  directly follows.

The bound in  coincides with the result reported in [@xu17-05a Thm. 1]. 