and the probability density function (pdf) is $p(x) = Pr\{X = x\},\;\; x \in \mathscr{X}$, then the entropy is defined by $$\label{16}
    H(X) = -\sum_{x \in \mathscr{X}} p(x)\;log\; p(x).$$ Here the unit of entropy is the bit and the base of the logarithm is 2. The joint entropy of $X$ and $Y$ is defined as $$\label{17}
    H(X,Y) = -\sum_{x \in \mathscr{X}} \sum_{y \in \mathscr{Y}} p(x,y)\;log\; p(x,y),$$ where $p(x,y)$ is the joint pdf of $X$ and $Y$, both of them being discrete.

The conditional entropy quantifies the resulting uncertainty of $X$ given $Y$, given by $$\label{18}
    H(Y|X) = -\sum_{x \in \mathscr{X}} \sum_{y \in \mathscr{Y}} p(x,y)\;log\; p(y|x),$$ where $p(y|x)$ is the conditional pdf of $Y$ given $X$. The mathematical definition of MI quantifying the dependency of two random variables is given by $$\label{19}
    I(X;Y) = \sum_{x \in \mathscr{X}} \sum_{y \in \mathscr{Y}} p(x,y)\;log\; \frac{p(x,y)}{p(x)p(y)}\;.$$ For continuous random variables, the definitions of entropy and of MI are $$\begin{aligned}
    H(X) &=& -\int_{x} p(x)\;log\; p(x)\; dx \label{20},\\
    H(Y|X) &=& -\int_x \int_y p(x,y)\;log\; p(y|x)\; dx \label{21},\\
    I(X;Y) &=& \int_{x} \int_{y} p(x,y)\;log\; \frac{p(x,y)}{p(x)p(y)}\;dx\;dy \label{22}.\end{aligned}$$ The entropy and MI satisfy the following relationship $$\label{23}
    I(X;Y) = H(Y) - H(Y|X),$$ which is the reduction of the uncertainty of $Y$ when 