}{\dv P_W\!P_{{\boldsymbol{Z}}}}$$ where $\dv P_{W\!{\boldsymbol{Z}}}/\dv P_W\!P_{{\boldsymbol{Z}}}$ is the Radon-Nikodym derivative of $P_{W\!{\boldsymbol{Z}}}$ with respect to $P_W\!P_{{\boldsymbol{Z}}}$. Here, $P_W$ is the distribution induced on the hypothesis space $\mathcal W$ by $P_Z$ through $P_{W\vert {\boldsymbol{Z}}}$. The information density is well defined whenever $P_{W\!{\boldsymbol{Z}}}$ is absolutely continuous with respect to $P_W\!P_{{\boldsymbol{Z}}}$, which we denote by $P_{W\!{\boldsymbol{Z}}}\ll P_W\!P_{{\boldsymbol{Z}}}$. The name information density is motivated by the fact that its expectation under $P_{W\!{\boldsymbol{Z}}}$ is the mutual information $I(W;{\boldsymbol{Z}})$. In the random-subset setting, several of our bounds will be in terms of the conditional information density $$\label{eq:conditional_info_density}
\imath(W,{\boldsymbol{S}}\vert {\widetilde{\boldsymbol{Z}}}) = \log \frac{\dv P_{W\! {\widetilde{\boldsymbol{Z}}}\! {\boldsymbol{S}}} }{\dv P_{W\vert {\widetilde{\boldsymbol{Z}}}}P_{{\widetilde{\boldsymbol{Z}}}\!{\boldsymbol{S}}} }$$ where $P_{W\vert {\widetilde{\boldsymbol{Z}}}}$ is a conditional distribution on $\mathcal{W}$ given ${\widetilde{\boldsymbol{Z}}}$, obtained by marginalizing out ${\boldsymbol{S}}$. Here, the absolute continuity requirement is that $P_{W\!{\widetilde{\boldsymbol{Z}}}\!{\boldsymbol{S}}}\ll P_{W\vert{\widetilde{\boldsymbol{Z}}}}P_{{\widetilde{\boldsymbol{Z}}}\!{\boldsymbol{S}}}$. In the random-subset setting, this is satisfied since $P_{W\vert {\widetilde{\boldsymbol{Z}}}}$ is obtained by marginalizing out the discrete random variable ${\boldsymbol{S}}$ from $P_{W\vert {\widetilde{\boldsymbol{Z}}}\!{\boldsymbol{S}}}$. If we take the expectation of $\imath(W,{\boldsymbol{S}}\vert {\widetilde{\boldsymbol{Z}}})$ under the joint distribution $P_{W\!{\widetilde{\boldsymbol{Z}}}\!{\boldsymbol{S}}}$, we obtain the conditional mutual information $I(W;{\boldsymbol{S}}\vert {\widetilde{\boldsymbol{Z}}})$, a key quantity in the bounds developed in [@steinke20-a].

Let $\alpha\in (0,1)\cup(1,\infty)$. The Rényi divergence of order $\alpha$ is defined as [@VanErven2014] $$\label{eq:def_renyi_divergence}
\alpharelent{\alpha}{P_{W\!{\boldsymbol{Z}}}}{P_W\!P_{{\boldsymbol{Z}}}}=\frac{1}{\alpha-1} \log \Exop_{P_W\!P_{{\boldsymbol{Z}}}}\lefto[\exp\lefto(\alpha{{\imath}(W,{\boldsymbol{Z}})}\right)\right].$$ In the 