In the context of machine learning,  is often called the information gain achieved if  would be used instead of  which is currently used. By analogy with information theory, it is called the relative entropy of  with respect to .
Expressed in the language of Bayesian inference,  is a measure of the information gained by revising one's beliefs from the prior probability distribution  to the posterior probability distribution . In other words, it is the amount of information lost when  is used to approximate .