Wishart distribution

| cdf =
 | mean =formula_1|
 | median =
 | mode = for 
 | variance =formula_2
 | skewness =
 | kurtosis =
 | entropy =see below
 | mgf =
 | char =formula_3
In statistics, the Wishart distribution is a generalization to multiple dimensions of the gamma distribution. It is named in honor of John Wishart, who first formulated the distribution in 1928.
It is a family of probability distributions defined over symmetric, nonnegative-definite random matrices (i.e. matrix-valued random variables). In random matrix theory, the space of Wishart matrices is called the "Wishart ensemble".
These distributions are of great importance in the estimation of covariance matrices in multivariate statistics. In Bayesian statistics, the Wishart distribution is the conjugate prior of the inverse covariance-matrix of a multivariate-normal random-vector.
Definition.
Suppose is a matrix, each column of which is independently drawn from a -variate normal distribution with zero mean:
Then the Wishart distribution is the probability distribution of the random matrix 
known as the scatter matrix. One indicates that has that probability distribution by writing
The positive integer is the number of "degrees of freedom". Sometimes this is written . For the matrix is invertible with probability if is invertible.
If then this distribution is a chi-squared distribution with degrees of freedom.
Occurrence.
The Wishart distribution arises as the distribution of the sample covariance matrix for a sample from a multivariate normal distribution. It occurs frequently in likelihood-ratio tests in multivariate statistical analysis. It also arises in the spectral theory of random matrices and in multidimensional Bayesian analysis. It is also encountered in wireless communications, while analyzing the performance of Rayleigh fading MIMO wireless channels .
Probability density function.
The Wishart distribution can be characterized by its probability density function as follows:
Let be a symmetric matrix of random variables that is positive semi-definite. Let be a (fixed) symmetric positive definite matrix of size .
Then, if , has a Wishart distribution with degrees of freedom if it has the probability density function
where formula_8 is the determinant of formula_9 and is the multivariate gamma function defined as
The density above is not the joint density of all the formula_11 elements of the random matrix (such density does not exist because of the symmetry constrains formula_12), it is rather the joint density of formula_13 elements formula_14 for formula_15 (, page 38). Also, the density formula above applies only to positive definite matrices formula_16 for other matrices the density is equal to zero.
The joint-eigenvalue density for the eigenvalues formula_17 of a random matrix formula_18 is,
where formula_20is a constant.
In fact the above definition can be extended to any real . If , then the Wishart no longer has a density—instead it represents a singular distribution that takes values in a lower-dimension subspace of the space of matrices.
Use in Bayesian statistics.
In Bayesian statistics, in the context of the multivariate normal distribution, the Wishart distribution is the conjugate prior to the precision matrix , where is the covariance matrix.
Choice of parameters.
The least informative, proper Wishart prior is obtained by setting .
The prior mean of is , suggesting that a reasonable choice for would be , where is some prior guess for the covariance matrix.
Properties.
Log-expectation.
The following formula plays a role in variational Bayes derivations for Bayes networks
involving the Wishart distribution:
where formula_22 is the multivariate digamma function (the derivative of the log of the multivariate gamma function).
Log-variance.
The following variance computation could be of help in Bayesian statistics:
where formula_24 is the trigamma function. This comes up when computing the Fisher information of the Wishart random variable.
Entropy.
The information entropy of the distribution has the following formula:
where is the normalizing constant of the distribution:
This can be expanded as follows:
Cross-entropy.
The cross entropy of two Wishart distributions formula_28 with parameters formula_29 and formula_30 with parameters formula_31 is
Note that when formula_33 and formula_34we recover the entropy.
KL-divergence.
The Kullback–Leibler divergence of formula_30 from formula_28 is
Characteristic function.
The characteristic function of the Wishart distribution is
where denotes expectation. (Here is any matrix with the same dimensions as , indicates the identity matrix, and is a square root of ). Properly interpreting this formula requires a little care, because noninteger complex powers are multivalued; when is noninteger, the correct branch must be determined via analytic continuation.
Theorem.
If a random matrix has a Wishart distribution with degrees of freedom and variance matrix — write formula_39 — and is a matrix of rank , then 
Corollary 1.
If is a nonzero constant vector, then:
In this case, formula_42 is the chi-squared distribution and formula_43 (note that formula_44 is a constant; it is positive because is positive definite).
Corollary 2.
Consider the case where (that is, the -th element is one and all others zero). Then corollary 1 above shows that
gives the marginal distribution of each of the elements on the matrix's diagonal.
George Seber points out that the Wishart distribution is not called the “multivariate chi-squared distribution” because the marginal distribution of the off-diagonal elements is not chi-squared. Seber prefers to reserve the term multivariate for the case when all univariate marginals belong to the same family.
Estimator of the multivariate normal distribution.
The Wishart distribution is the sampling distribution of the maximum-likelihood estimator (MLE) of the covariance matrix of a multivariate normal distribution. A derivation of the MLE uses the spectral theorem.
Bartlett decomposition.
The Bartlett decomposition of a matrix from a -variate Wishart distribution with scale matrix and degrees of freedom is the factorization:
where is the Cholesky factor of , and:
where formula_48 and independently. This provides a useful method for obtaining random samples from a Wishart distribution.
Marginal distribution of matrix elements.
Let be a variance matrix characterized by correlation coefficient and its lower Cholesky factor:
Multiplying through the Bartlett decomposition above, we find that a random sample from the Wishart distribution is
The diagonal elements, most evidently in the first element, follow the distribution with degrees of freedom (scaled by ) as expected. The off-diagonal element is less familiar but can be identified as a normal variance-mean mixture where the mixing density is a distribution. The corresponding marginal probability density for the off-diagonal element is therefore the variance-gamma distribution
where is the modified Bessel function of the second kind. Similar results may be found for higher dimensions, but the interdependence of the off-diagonal correlations becomes increasingly complicated. It is also possible to write down the moment-generating function even in the "noncentral" case (essentially the "n"th power of Craig (1936) equation 10) although the probability density becomes an infinite sum of Bessel functions.
The range of the shape parameter.
It can be shown that the Wishart distribution can be defined if and only if the shape parameter belongs to the set
This set is named after Gindikin, who introduced it in the 1970s in the context of gamma distributions on homogeneous cones. However, for the new parameters in the discrete spectrum of the Gindikin ensemble, namely,
the corresponding Wishart distribution has no Lebesgue density.