$X$ is known [@kullback1997information]. The MI can also be rewritten as $I(X;Y) = H(X) + H(Y) - H(X,Y)$, where $H(X,Y)$ is the joint entropy of $X$ and $Y$. When the variables $X$ and $Y$ are independent, the MI of the two variables is zero. Compared to the Pearson correlation coefficient which only measures linear relationships, MI can measure any relationship between variables [@kullback1997information].

### Computation of MI in Classification Problems

Consider the class label $Y$ for an $m$-class classification problem and let the number of observations in the $y$-th class be $n_y$ such that $n_1+n_2+...+n_m = n$, as mentioned in . In terms of classification problems, we are interested in finding the relevance of the $j$-th feature $X_{.j}$ with the class label $Y$, which is measured by their Mutual Information (MI) $$\label{24}
    I(X_{.j};Y) =  H(Y) - H(Y|X_{.j}),$$ In this equation, the entropy of class variable $Y$ can be computed using as $$\label{25}
    \Hat{H}(Y) = -\sum_{y=1}^{m} \Hat{p}(y)\;log\; \Hat{p}(y).$$ We can easily estimate $p(y)$ by $\Hat{p}(y)=n_y/n$. The estimation of $H(Y|X_{.j})$ is given by when $X_{.j}$ is discrete and by when $X_{.j}$ is continuous. For discrete feature variables, estimating the probability densities is straightforward by means of a histogram approximation [@kwak2002input].

For continuous 