Bayesian Data Analysis (p. 64) says, regarding a normal model:

a sensible vague prior density for $\mu$ and $\sigma$, assuming prior independence of location and scale parameters, is uniform on $(\mu, \log \sigma)$, or, equivalently, $$ p(\mu, \sigma^2) \propto (\sigma^2)^{-1}. $$

Why is uniform over $p(\mu, \log \sigma)$ equivalent to $p(\mu, \sigma^2) \propto (\sigma^2)^{-1}$?

Why is that a "sensible" prior?


Let $\phi = \log (\sigma) = \tfrac{1}{2} \log (\sigma^2)$ so that you have the inverse transformation $\sigma^2 = \exp (2\phi)$. Now we apply the standard rule for transformations of random variables to get:

$$p(\sigma^2) = p(\phi) \cdot \Bigg| \frac{d \phi}{d\sigma^2} \Bigg| \propto 1 \cdot \frac{1}{2\sigma^2} \propto (\sigma^2)^{-1}.$$

Since the parameters are independent in this prior, we then have:

$$p(\mu, \sigma^2) = p(\mu) p(\sigma^2) \propto (\sigma^2)^{-1}.$$

This gives the stated form for the improper prior density. As to the justification for why this prior is sensible, there are several avenues of appeal. The simplest justification is that we would like to take $\mu$ and $\log \sigma$ to be uniform to represent "ignorance" about these parameters. Taking the logarithm of the variance is a transformation that ensures that our beliefs about that parameter are scale invariant. (Our beliefs about the mean parameter are also location and scale invariant.) In other words, we would like our representation of ignorance for the two parameters to be invariant to arbitrary changes in the measurement scale of the variables.

For the derivation above, we have used an improper uniform prior on the log-variance parameter. It is possible to get the same result in a limiting sense, by using a proper prior for the log-scale that tends towards uniformity, and finding the proper prior for the variance that corresponds to this, and then taking the limit to obtain the present improper variance prior. This is really just a reflection of the fact that improper priors can generally be interpreted as limits of proper priors.

There are many other possible justifications for this improper prior, and these appeal to the theory of representing prior "ignorance". There is a large literature on this subject, but a shorter discussion can be found in Irony and Singpurwalla (1997) (discussion with José Bernardo) which talks about the various methods by which we try to represent "ignorance". The improper prior you are dealing with here is the limiting version of the conjugate prior for the normal model, with the prior variance for each parameter taken to infinity.


Your Answer

