$\ftl X(\mathbf{X}_{i})$, $\ftl{X|y}(\mathbf{X}_{i})$ as described in the text

$\g{h_{X}(l),\mathbf{h}_{X|y}(l)}\leftarrow\frac{1}{\mathbf{N}_{y}}\sum_{\mathbf{X}\in\mathcal{X}_{y}}g\left(\frac{\ftl X(\mathbf{X})}{\ftl{X|y}(\mathbf{X})}\right)$

$\g{w_{0},1}\leftarrow\sum_{l\in\mathcal{L}}w_{0}(l)\sum_{y\in\mathcal{S}_{y}}\frac{\mathbf{N}_{y}}{N}\g{h_{X}(l),\mathbf{h}_{X|y}(l)}$

A similar approach can be used to derive an ensemble estimator $\g{w_{0},1}^{cont}$ for the case when $\X$ and $\Y$ are continuous (case 1 in Section \[sec:intro\]). See Appendix \[subsec:cont\_ensemble\] for details. The case where $\X$ and $\Y$ both contain a mixture of discrete and continuous components follows similarly.

Parameter Selection
-------------------

In theory, the theoretical results of the previous sections hold for any choice of the bandwidth vectors as determined by$\mathcal{L}$. In practice, we find that the following rules-of-thumb for tuning the parameters lead to high-quality estimates in the finite sample regime.

1.  Select the minimum and maximum bandwidth parameter to produce density estimates that satisfy the following: first the minimum bandwidth should not lead to a zero-valued density estimate at any sample point; second the maximum bandwidth should be smaller than the diameter of the support.

2.  Ensure the bandwidths are sufficiently distinct. Similar bandwidth values lead to negligible decrease in bias and many bandwidth values may increase $||w_{0}||_{2}$ resulting in an increase in variance [@sricharan2013ensemble].

3.  Select $L=|\mathcal{L}|>|J|=I$ to obtain a feasible solution for the optimization problems in (\[eq:optimize\]) and (\[eq:relaxed\]). We find that choosing a value 