between the random variables possessed by any two subsets $\mathcal{W}$ and $\mathcal{U}$ of agents is given by [@ref21] $$I(X_{\mathcal{W}};X_{\mathcal{U}})=H(X_{\mathcal{W}})+H(X_{\mathcal{U}})-H(X_{\mathcal{W}},X_{\mathcal{U}}).$$

The total amount of information in the network is given by the joint entropy of the random variables of individual agents $H(\mathcal{X})=H(X_{1},X_{2},X_{3},...,X_{N}),$ where $H(\mathcal{X})\in\overrightarrow{{\bf H}}$.

The mutual information between any two agents $i$ and $j$ is given by $I(X_{i};X_{j})=H(X_{i})-H(X_{i}|X_{j})$, where $H(X_{i}|X_{j})$ is the *conditional entropy* which represents the additional information attained by agent $j$ from connecting to $i$, i.e. the amount of extra information that $j$ gets when getting the information of $i$. If this benefit is low, it means that $I(X_{i};X_{j})$ is high, i.e. $X_{i}$ and $X_{j}$ are highly correlated, and vice versa. Note that mutual information is symmetric, i.e. $I(X_{i};X_{j})=H(X_{i})-H(X_{i}|X_{j})=H(X_{j})-H(X_{j}|X_{i})$. Finally we quantify the total amount of redundant information in the network. Let ${\bf p}\left(\mathcal{X}\right) =p(X_{1},X_{2},...,X_{N})$ and ${\bf q}\left(\mathcal{X}\right) =\Pi_{i=1}^{N}p(X_{i})$, where $p(X_{i})$ is the pmf of $X_{i}$. The *Kullback Leibler* (KL) divergence for these distributions can be computed as follows [@ref21] $$\begin{aligned}
D\left({\bf p}||{\bf q}\right) &=\sum_{\mathcal{X}} {\bf p}\left(\mathcal{X}\right) \log\left(\frac{{\bf p}\left(\mathcal{X}\right)}{{\bf q}\left(\mathcal{X}\right)}\right) \nonumber \\
&=\sum_{i=1}^{N}H(X_{i})-H(X_{1},X_{2},...,X_{N}).
\label{KL22}\end{aligned}$$ The KL divergence is a natural metric for quantifying the distance between probability measures, and it can be obtained in terms of the entropy 