While relative entropy is a statistical distance, it is not a metric on the space of probability distributions, but instead it is a divergence. While metrics are symmetric and generalize linear distance, satisfying the triangle inequality, divergences are asymmetric in general and generalize squared distance, in some cases satisfying a generalized Pythagorean theorem. In general  does not equal , and while this can be symmetrized (see ), the asymmetry is an important part of the geometry.
It generates a topology on the space of probability distributions. More concretely, if  is a sequence of distributions such that
then it is said that
Pinsker's inequality entails that
where the latter stands for the usual convergence in total variation.