@moon2016isit] uses an ensemble approach to estimate entropy or divergence functionals. These works construct an ensemble of simple plug-in estimators by varying the neighborhood size of the density estimators. They then take a weighted average of the estimators where the weights are chosen to decrease the bias with only a small increase in the variance. The parametric rate of convergence is achieved when the densities are either $d$ [@sricharan2013ensemble; @moon2014isit; @moon2014nips] or $(d+1)/2$ [@moon2016arxiv; @moon2016isit] times differentiable. These approaches are simple to implement as they only require simple plug-in estimates and the solution of an offline convex optimization problem. These estimators have also performed well in various applications [@szabo2012distributed; @gliske2015intrinsic; @moon2015Bayes; @moon2015partI; @moon2015partII]

Finally, the authors of [@gao2015efficient] showed that $k$-nn or KDE based approaches underestimate the MI when the MI is large. As MI increases, the dependencies between random variables increase which results in less smooth densities. Thus a common approach to overcome this issue is to require the densities to be smooth [@krishnamurthy2014divergence; @kandasamy2015nonparametric; @singh2014exponential; @singh2014renyi; @sricharan2013ensemble; @moon2014isit; @moon2014nips; @moon2016arxiv; @moon2016isit].

Contributions
-------------

In the context of this related work, we make the following novel contributions in this paper: (1) For continuous random variables (case 1), we extend 