unit cube and $\mathbf{Y}$ is a categorical random variable. We choose Rényi MI as it has received recent interest (e.g. [@pal2010estimation]) and the estimation problem does not reduce to entropy estimation in contrast with Shannon MI. Thus this is a clear case where there are no other nonparametric estimators that are known to achieve the parametric MSE rate.

We consider two cases. In the first case, $\mathbf{Y}$ has three possible outcomes (i.e. $|\mathcal{S}_{Y}|=3$) and respective probabilities $\Pr(\mathbf{Y}=0)=\Pr(\mathbf{Y}=1)=2/5$ and $\Pr(\mathbf{Y}=2)=1/5$. The conditional covariance matrices are all $0.1\times I_{d}$ and the conditional means are, respectively, $\bar{\mu}_{0}=0.25\times\bar{1}_{d}$, $\bar{\mu}_{1}=0.75\times\bar{1}_{d}$, and $\bar{\mu}_{2}=0.5\times\bar{1}_{d}$, where $I_{d}$ is the $d\times d$ identity matrix and $\bar{1}_{d}$ is a $d$-dimensional vector of ones. This experiment can be viewed as the problem of estimating MI (e.g. for feature selection or Bayes error bounds) of a classification problem where each discrete value corresponds to a distinct class, the distribution of each class overlaps slightly with others, and the class probabilities are unequal. We use $\alpha=0.5$. We set $\mathcal{L}$ to be 40 linearly spaced values between 1.2 and 3. The bandwidth in the KDE plug-in estimator is also set to $2.1N^{-1/(2d)}$.

The top three plots in Figure \[fig:mseplot\] shows the MSE (200 trials) 