this setting of mixed random variables.

If the mixed derivatives of the functional $g$ satisfy the extra condition required for (\[eq:bias\_mixed2\]), we can define an ensemble estimator $\g{w_{0},2}$ that achieves the parametric MSE rate if $s>d_{X}/2$. For simplicity, we focus primarily on $\g{w_{0},1}$. See Appendix \[subsec:Odin2\] for details on $\g{w_{0},2}$.

In practice, the optimization problem in (\[eq:optimize\]) typically results in a very large increase in variance. Thus we follow the lead of [@moon2016arxiv; @moon2014isit; @moon2014nips; @sricharan2013ensemble] and use a relaxed version of (\[eq:optimize\]): $$\begin{array}{rl}
\min_{w} & \epsilon\\
subject\,to & \sum_{l\in\mathcal{L}}w(l)=1,\\
 & \left|\gamma_{w}(i)N^{\frac{1}{2}}\phi_{i}(N)\right|\leq\epsilon,\,\,i\in J,\\
 & \left\Vert w\right\Vert _{2}^{2}\leq\eta.
\end{array}\label{eq:relaxed}$$ As shown in [@moon2016arxiv; @moon2014isit; @moon2014nips; @sricharan2013ensemble], the ensemble estimator $\g{w_{0},1}$ using the resulting weight vector from the optimization problem in (\[eq:relaxed\]) still achieves the parametric MSE convergence rate under the same assumptions as described previously. It was also shown in [@moon2016arxiv] that the heuristic of setting $\eta=\epsilon$ works well in practice. Algorithm \[alg:estimator\] summarizes the estimator $\g{w_{0},1}$.

$L$ positive real numbers $\mathcal{L}$, samples $\left\{ \mathbf{Z}_{1},\dots,\mathbf{Z}_{N}\right\} $ from $f_{XY}$, dimension $d_{X}$, function $g$, kernel $K_{X}$

The optimally weighted MI estimator $\g{w_{0},1}$

Solve for $w_{0}$ using (\[eq:relaxed\]) with basis functions $\psi_{i}(l)=l^{i}$, $\phi_{i}(N)=N^{-i/(2d_{X})},$ $l\in\mathcal{L}$, and $0\leq i\leq d_{X}$.

$\mathbf{N}_{y}\leftarrow\sum_{i=1}^{N}1_{\{\mathbf{Y}_{i}=y\}}$

$h_{X}(l)\leftarrow lN^{-1/(2d_{X})},$ $\mathbf{h}_{X|y}(l)\leftarrow l\mathbf{N}_{y}^{-1/(2d_{Y})}$

Calculate 