$\sigma$-subgaussian for all $w\in\mathcal{W}$ and the $Z_i$ are ,  $(1/n)\sum_{i=1}^n  \ell(w,Z_i)$ is $\sigma/\sqrt{n}$-subgaussian for all $w\in\mathcal{W}$, as remarked after Lemma \[lem:hoeffdings\_inequality\]. Thus, for all $w\in\mathcal{W}$, $$\label{eq:proof_of_uncond_thm_subgaussian_assuption}
    \Exop_{P_{{\boldsymbol{Z}}} } \lefto[\exp\lefto(\lambda\left(\Exop_{P_Z}\lefto[\ell(w,Z)\right]-\frac{1}{n}\sum_{i=1}^n \ell(w,Z_i) \right)\right)\right]
    \leq \exp\lefto(\frac{\lambda^2\sigma^2}{2n}\right).$$ Reorganizing terms and taking the expectation with respect to $P_W$, we get$$\label{eq:thm_1_proof_before_E}
    \Exop_{P_W\! P_{{\boldsymbol{Z}}} } \lefto[\exp\lefto(\lambda {{\textnormal{gen}}(W,{\boldsymbol{Z}})}- \frac{\lambda^2\sigma^2}{2n}\right)\right]\leq 1.$$ Now, let $\setE=\textnormal{supp}(P_{W\!{\boldsymbol{Z}}})$ be the support of $P_{W\!{\boldsymbol{Z}}}$. From , it follows that $$\Exop_{P_W\! P_{{\boldsymbol{Z}}} } \lefto[1_\setE\cdot\exp\lefto(\lambda {{\textnormal{gen}}(W,{\boldsymbol{Z}})}- \frac{\lambda^2\sigma^2}{2n}\right)\right]\leq 1$$ where $1_\setE$ is the indicator function of the set $\setE$. To obtain , we perform a change measure from $P_W\! P_{{\boldsymbol{Z}}}$ to $P_{W\! {\boldsymbol{Z}}}$ [@polyanskiy19-a Prop. 17.1].

Note that Theorem \[thm:unconditional\_subgaussian\] holds *verbatim* if $P_W$ is replaced with an auxiliary distribution $Q_W$, under a suitable absolute continuity assumption. This is detailed in the next remark.

\[rem:uncond\_conversion\_to\_Q\_bounds\] Consider the setting of Theorem \[thm:unconditional\_subgaussian\], but with the altered absolute continuity assumption that $P_{W\!{\boldsymbol{Z}}}\ll Q_W\!P_{{\boldsymbol{Z}}}$ for some distribution $Q_W$ on $\mathcal{W}$. Then, $$\label{eq:unconditional_subgaussian_with_Q}
    \Exop_{P_{W\! {\boldsymbol{Z}}}}\lefto[{\exp\lefto(\lambda {{\textnormal{gen}}(W,{\boldsymbol{Z}})}-\frac{\lambda^2\sigma^2}{2n}-\log\frac{\dv P_{W\!{\boldsymbol{Z}}} }{\dv Q_W\!P_{{\boldsymbol{Z}}}}  \right)}\right]\leq 1.$$

For the bounds that we will later derive, the choice $Q_W=P_W$ is optimal. Unfortunately, since the data distribution $P_Z$ is considered to be unknown in the statistical learning framework, the marginal distribution $P_W$ is also unavailable. Hence, $P_W$ needs to 