The mutual information,
is the relative entropy of the joint probability distribution  from the product  of the two marginal probability distributions — i.e. the expected number of extra bits that must be transmitted to identify  and  if they are coded using only their marginal distributions instead of the joint distribution. Equivalently, if the joint probability  is known, it is the expected number of extra bits that must on average be sent to identify  if the value of  is not already known to the receiver.