= D_{KL}(p(x,y)||p(x)p(y)).$$

To see that $I(\Theta,X) = H(\Theta) - H(\Theta|X)$, we use the definition of joint probability and rewrite the mutual information as: $$\begin{aligned}
I(\Theta,X) &= \int_\Theta \int_X p(\theta|x)p(x) \log_2\left(\frac{p(\theta|x)p(x)}{p(x)p(\theta)}\right) dxd\theta \nonumber \\
            &= \int_\Theta \int_X p(\theta|x)p(x) \log_2\left(\frac{p(\theta|x)}{p(\theta)}\right) dxd\theta. \nonumber\end{aligned}$$ Then, simplifying, we find that: $$\begin{aligned}
I(\Theta,X) &= \int_\Theta \int_X p(\theta|x)p(x) \log_2 p(\theta|x) dxd\theta \nonumber \\
            &\ \ \ \ \ - \int_\Theta \int_X p(\theta|x)p(x) \log_2 p(\theta) dxd\theta \nonumber \\
            &= -H(\Theta|X) - \int_\Theta \log_2 p(\theta)\left(\int_X p(\theta|x)p(x)dx\right)d\theta \nonumber \\
%            &= -H(\Theta|X) - \int_\Theta p(\theta)\log_2 p(\theta) \left(\int_X p(x)dx\right)d\theta \nonumber \\
            &= -H(\Theta|X) - \int_\Theta p(\theta)\log_2 p(\theta)d\theta \nonumber \\
            &= -H(\Theta|X) + H(\Theta),\end{aligned}$$ which completes the proof. Thus, it is the [*difference*]{} in entropy, which is dimensionless, that we care about rather than the entropy of the prior or posterior by itself, which, for a continuous probability density, is dependent on the units of the parameters.

Mutual information, much like the Shannon entropy, is derived from coding theory, and it finds significant use in the field of systems biology (e.g. @journals/ploscb/LiepeFKS13), where it is similarly used to inform experimental design, and we propose that it is also appropriate for this optimization problem. However, we add the modification that it is the 