are $O(1/\sqrt{N})$ when $s\geq d$, for each estimator in the ensemble we choose $h_{X}(l)=lN^{-1/(2d_{X})}$ and $\mathbf{h}_{X|y}(l)=l\mathbf{N}_{y}^{-1/(2d_{X})}$ where $l\in\mathcal{L}$. Define $w$ to be a weight vector parameterized by $l\in\mathcal{L}$ with $\sum_{l\in\mathcal{L}}w(l)=1$ and define $$\g{w,1}=\sum_{l\in\mathcal{L}}w(l)\sum_{y\in\mathcal{S}_{Y}}\frac{\mathbf{N}_{y}}{N}\g{h_{X}(l),h_{X|y}(l)}.\label{eq:ensemble}$$ From Theorem \[thm:bias\_mixed\], the bias of $\g{w,1}$ is $$\begin{aligned}
\bias\left[\g{w,1}\right] & =\sum_{l\in\mathcal{L}}\sum_{i=1}^{r}\theta\left(w(l)l^{i}N^{\frac{-i}{2d_{X}}}\right)\nonumber \\
 & +O\left(\sqrt{L}||w||_{2}\left(N^{\frac{-s}{2d_{X}}}+N^{\frac{-1}{2}}\right)\right),\label{eq:weight_bias}\end{aligned}$$ where we use $\theta$ notation to omit the constants.

We use the general theory of optimally weighted ensemble estimation in [@moon2016isit] to improve the MSE convergence rate of the plug-in estimator by using the weights to cancel the lower order terms in (\[eq:weight\_bias\]). The theory is as follows. Let $\left\{ \hat{\mathbf{E}}_{l}\right\} _{l\in\mathcal{L}}$ be an indexed ensemble of estimators with the weighted ensemble estimator $\hat{\mathbf{E}}_{w}=\sum_{l\in\mathcal{L}}w(l)\hat{\mathbf{E}}_{l}$ satisfying:

-   $\mathcal{C}.1$. Let $c_{i}$ be constants depending on the underlying density, $J=\{i_{1},\dots i_{I}\}$ a finite index set with $I<L$, $\psi_{i}(l)$ basis functions depending only on the parameter $l$ and not on $N$, $\phi_{i}(N)$ functions of the sample size $N$ that are independent of $l$. Assume the bias is $$\bias\left[\hat{\mathbf{E}}_{l}\right]=\sum_{i\in J}c_{i}\psi_{i}(l)\phi_{i}(N)+O\left(\frac{1}{\sqrt{N}}\right).$$

-   $\mathcal{C}.2$. Assume the variance is $$\var\left[\hat{\mathbf{E}}_{l}\right]=c_{v}\left(\frac{1}{N}\right)+o\left(\frac{1}{N}\right).$$

[@moon2016isit] \[thm:opt\_weight\]If conditions $\mathcal{C}.1$ and $\mathcal{C}.2$ hold for an ensemble of estimators $\left\{ \hat{\mathbf{E}}_{l}\right\} _{l\in\mathcal{L}}$, then there exists a weight vector $w_{0}$ such that the MSE of $\hat{\mathbf{E}}_{w_{0}}$ attains the 