$\X$ is continuous and $\Y$ is discrete and when $\X$ and $\Y$ may have any mixture of continuous and discrete components (respectively, cases 2 and 3 in Section \[sec:intro\]). These rates can then be used to derive ensemble estimators that achieve the parametric MSE rate. Let $f_{X}(x)$, $f_{Y}(y)$, and $f_{XY}(x,y)$ be $d_{X}$, $d_{Y}$, and $d_{X}+d_{Y}=d$-dimensional densities. Let $g(t_{1},t_{2})=g\left(\frac{t_{1}}{t_{2}}\right)$ (e.g. $g(t_{1},t_{2})=\log(t_{1}/t_{2})$ for Shannon information). We define a family of MIs as $$G_{1}(\mathbf{X};\mathbf{Y})=\int g\left(\frac{f_{X}(x)f_{Y}(y)}{f_{XY}(x,y)}\right)f_{XY}(x,y)dxdy.\label{eq:MI}$$

The KDE Plug-in Estimator
-------------------------

When both $\mathbf{X}$ and $\mathbf{Y}$ are continuous with marginal densities $f_{X}$ and $f_{Y}$, the MI functional $G_{1}(\mathbf{X};\mathbf{Y})$ can be estimated using KDEs. Assume that $N$ i.i.d. samples $\left\{ \mathbf{Z}_{1},\dots,\mathbf{Z}_{N}\right\} $ are available from the joint density $f_{XY}$ with $\mathbf{Z}_{i}=\left(\mathbf{X}_{i},\mathbf{Y}_{i}\right)^{T}$. Let $M=N-1$ and let $h_{X}$, $h_{Y}$ be kernel bandwidths. Let $K_{X}(\cdot)$ and $K_{Y}(\cdot)$ be kernel functions with $||K_{X}||_{\infty},\,||K_{Y}||_{\infty}<\infty$ where $||K||_{\infty}=\sup_{x}|K(x)|$. The KDE for $f_{X}$ is $$\begin{aligned}
\ft X(\mathbf{X}_{j}) & = & \frac{1}{Mh_{X}^{d_{X}}}\sum_{\substack{i=1\\
i\neq j
}
}^{N}K_{X}\left(\frac{\mathbf{X}_{j}-\mathbf{X}_{i}}{h_{X}}\right).\label{eq:fx}\end{aligned}$$ The KDEs $\ft Y(\Y_{j})$ and $\ft Z(\mathbf{X}_{j},\mathbf{Y}_{j})$ (where $h_{Z}=(h_{X},h_{Y})$) for estimating $f_{Y}$ and $f_{XY}$, respectively, are defined similarly using $K_{Y}$ and the product kernel $K_{X}\cdot K_{Y}$. Then $G_{1}(\mathbf{X};\mathbf{Y})$ is estimated as $$\gt=\frac{1}{N}\sum_{i=1}^{N}g\left(\frac{\ft X(\mathbf{X}_{i})\ft Y(\mathbf{Y}_{i})}{\ft Z(\mathbf{X}_{i},\mathbf{Y}_{i})}\right).\label{eq:Gest}$$

Convergence Rates
-----------------

To derive the convergence rates of $\gt$ we assume that 