G-test

In statistics, "G"-tests are likelihood-ratio or maximum likelihood statistical significance tests that are increasingly being used in situations where chi-squared tests were previously recommended.
The general formula for "G" is
where formula_2 is the observed count in a cell, formula_3 is the expected count under the null hypothesis, formula_4 denotes the natural logarithm, and the sum is taken over all non-empty cells. Furthermore, the total observed count should be equal to the total expected count:formula_5where formula_6 is the total number of observations.
"G"-tests have been recommended at least since the 1981 edition of "Biometry", a statistics textbook by Robert R. Sokal and F. James Rohlf.
Derivation.
We can derive the value of the "G"-test from the log-likelihood ratio test where the underlying model is a multinomial model.
Suppose we had a sample formula_7 where each formula_8 is the number of times that an object of type formula_9 was observed. Furthermore, let formula_10 be the total number of objects observed. If we assume that the underlying model is multinomial, then the test statistic is defined byformula_11where formula_12 is the null hypothesis and formula_13 is the maximum likelihood estimate (MLE) of the parameters given the data. Recall that for the multinomial model, the MLE of formula_14 given some data is defined byformula_15Furthermore, we may represent each null hypothesis parameter formula_16 asformula_17Thus, by substituting the representations of formula_12 and formula_19 in the log-likelihood ratio, the equation simplifies toformula_20Relabel the variables formula_21 with formula_22 and formula_8 with formula_24. Finally, multiply by a factor of formula_25 (used to make the G test formula asymptotically equivalent to the Pearson's chi-squared test formula) to achieve the form
formula_26
Heuristically, one can imagine formula_27 as continuous and approaching zero, in which case formula_28 and terms with zero observations can simply be dropped. However the "expected" count in each cell must be strictly greater than zero for each cell (formula_29) to apply the method.
Distribution and use.
Given the null hypothesis that the observed frequencies result from random sampling from a distribution with the given expected frequencies, the distribution of "G" is approximately a chi-squared distribution, with the same number of degrees of freedom as in the corresponding chi-squared test.
For very small samples the multinomial test for goodness of fit, and Fisher's exact test for contingency tables, or even Bayesian hypothesis selection are preferable to the "G"-test. McDonald recommends to always use an exact test (exact test of goodness-of-fit, Fisher's exact test) if the total sample size is less than 1 000 .
Relation to the chi-squared test.
The commonly used chi-squared tests for goodness of fit to a distribution and for independence in contingency tables are in fact approximations of the log-likelihood ratio on which the "G"-tests are based.
The general formula for Pearson's chi-squared test statistic is
The approximation of "G" by chi squared is obtained by a second order Taylor expansion of the natural logarithm around 1. To see this consider 
and let formula_32 with formula_33 so that the total number of counts remains the same. Upon substitution we find,
A Taylor expansion around formula_35 can be performed using formula_36. The result is
Now, using the fact that formula_39 and formula_40 we can write the result,
This shows that formula_42 when the observed counts formula_27 are close to the expected counts formula_44 When this difference is large, however, the formula_45 approximation begins to break down. Here, the effects of outliers in data will be more pronounced, and this explains the why formula_45 tests fail in situations with little data. 
For samples of a reasonable size, the "G"-test and the chi-squared test will lead to the same conclusions. However, the approximation to the theoretical chi-squared distribution for the "G"-test is better than for the Pearson's chi-squared test. In cases where formula_47 for some cell case the "G"-test is always better than the chi-squared test.
For testing goodness-of-fit the "G"-test is infinitely more efficient than the chi squared test in the sense of Bahadur, but the two tests are equally efficient in the sense of Pitman or in the sense of Hodges and Lehmann.
Relation to Kullback–Leibler divergence.
The "G"-test statistic is proportional to the Kullback–Leibler divergence of the theoretical distribution from the empirical distribution:
where "N" is the total number of observations and formula_49 and formula_50 are the empirical and theoretical frequencies, respectively.
Relation to mutual information.
For analysis of contingency tables the value of "G" can also be expressed in terms of mutual information.
Let
Then "G" can be expressed in several alternative forms:
where the entropy of a discrete random variable formula_58 is defined as
and where
is the mutual information between the row vector "r" and the column vector "c" of the contingency table.
It can also be shown that the inverse document frequency weighting commonly used for text retrieval is an approximation of "G" applicable when the row sum for the query is much smaller than the row sum for the remainder of the corpus. Similarly, the result of Bayesian inference applied to a choice of single multinomial distribution for all rows of the contingency table taken together versus the more general alternative of a separate multinomial per row produces results very similar to the "G" statistic.