effectiveness of the TQWT for diagnosing PD patients from vowel vocalization data. This method, as presented by Peng et al. [@feature], attempts to maximize the dependency of the response (having the disease) on the predictors. The mutual information definition of dependency has many attractive properties, but estimability is not one of them. Consequently, the authors solve a simpler, but related, problem: minimum redundancy-maximum relevance. Peng et al. [@feature] minimize the mutual information between predictors while simultaneously maximizing the mutual information between the predictors as a whole and the response.

The mutual information of random variables $X$ and $Y$, $I(X,Y)$ is defined in terms of their joint density function $f(X,Y)$ and marginal densities $f(X)$ and $f(Y)$: $$I(X,Y) = \iint f(X,Y) \log\frac{f(X,Y)}{f(X)f(Y)} dX dY
    \label{eq:i}$$

From this, definition of mutual information is derived from the measure of feature relevance $D$ for feature $X_{\cdot i}$ within feature set $S$, and class $c$: $$D(S,c) = \frac{1}{|S|} \sum_{X_{\cdot i} \in S} I(X_{\cdot i}, c)
    \label{eq:d}$$

Similarly, the redundancy of features totals the mutual information between features in the model: $$R(S) = \frac{1}{|S|^2} \sum_{X_{\cdot i},X_{\cdot j} \in S} I(X_{\cdot i}, X_{\cdot j})
    \label{eq:r}$$

Thus, the mRMR statistic of a given feature set $\Phi$ can be defined 