are analogous to those in Theorems \[thm:bias\], \[thm:variance\], and \[thm:bias\_mixed\] as the variance will be $O(1/N)$ and the bias will depend on expansions of the bandwidths for the various KDEs. Ensemble methods can then be applied to improve the MSE convergence rates as described in the next section.

Ensemble Estimation of MI\[sec:mixed\_ensemble\]
================================================

Mixed Random Variables {#subsec:mixed_ensemble}
----------------------

We again focus on the case where $\mathbf{X}$ is continuous and $\mathbf{Y}$ is discrete (case 2 in Section \[sec:intro\]). If no bias correction is performed, then Theorem \[thm:bias\_mixed\] shows that the optimal bias rate of the plug-in estimator $\g{h_{X},h_{X|Y}}$ is $O\left(1/N^{1/(d_{X}+1)}\right)$, which converges very slowly to zero when $d_{X}$ is not small. We use the theory of optimally weighted ensemble estimation to improve this rate. An ensemble of estimators is formed by choosing different bandwidth values. Consider first the case where (\[eq:bias\_mixed1\]) applies. Let $\mathcal{L}$ be a set of real positive numbers with $|\mathcal{L}|=L$. This set will parameterize the bandwidths for $\ft X$ and $\ft{X|y}$ resulting in $L$ estimators in the ensemble. While different parameter sets for $\ft X$ and $\ft{X|y}$ can be chosen, we only use one set here for simplicity of exposition. To ensure that the final terms in (\[eq:bias\_mixed1\]) 