ensemble. Assume that the functional $g$ is Lipschitz in both arguments with Lipschitz constant $C_{g}$ and that $h_{X}(l_{X}),\,h_{Y}(l_{Y})=o(1)$, $N\rightarrow\infty$, and $Nh_{X}^{d_{X}}(l_{X}),\,Nh_{Y}^{d_{Y}}(l_{Y})\rightarrow\infty$ for each $l_{X}\in\mathcal{L}_{X}$ and $l_{Y}\in\mathcal{L}_{Y}$. Then for fixed $\mathcal{L}_{X}$ and $\mathcal{L}_{Y}$, and if $\mathbf{S}$ is a standard normal random variable, $$\Pr\left(\left(\g w^{cont}-\bE\left[\g w^{cont}\right]\right)/\sqrt{\var\left[\g w^{cont}\right]}\leq t\right)\rightarrow\Pr\left(\mathbf{S}\leq t\right).$$

The proof is based on an application of Slutsky’s Theorem preceded by an application of the Efron-Stein inequality (see Appendix \[sec:cltProof\]).

If the space $\mathcal{S}_{Y}$ is finite, then the ensemble estimators for the mixed component case also obey a central limit theorem. The proof follows by an application of Slutsky’s Theorem combined with Theorem \[thm:clt\].

Let $\g w$ be a weighted ensemble estimator when $\X$ is continuous and $\Y$ is discrete with bandwidths $h_{X}(l)$ and $h_{X|y}(l)$ for each estimator in the ensemble. Assume that the functional $g$ is Lipschitz in both arguments and that $h_{X},\,h_{X|y}=o(1)$, $N\rightarrow\infty$, and $Nh_{X}^{d_{X}},\,Nh_{X|y}^{d_{X}}\rightarrow\infty$ for each $l\in\mathcal{L}$ and $\forall y\in\mathcal{S}_{Y}$ with $\mathcal{S}_{y}$ finite. Then for fixed $\mathcal{L}$, $$\Pr\left(\left(\g w-\bE\left[\g w\right]\right)/\sqrt{\var\left[\g w\right]}\leq t\right)\rightarrow\Pr\left(\mathbf{S}\leq t\right).$$

Experimental Validation {#sec:experiments}
=======================

In this section, we validate our theory by estimating the Rényi-$\alpha$ MI integral (i.e. $g(x)=x^{\alpha}$ in (\[eq:MI\_cond\]); see [@principe2010information]) where $\mathbf{X}$ is a mixture of truncated Gaussian random variables restricted to the 