I'm writing a paper on making probability estimates, and it's been asserted to me that I should take the median of the estimates given by my participants, rather than the mean. I've been told I should do this because the mean is more affected by sampling error than the median.

Why is this? Is this something that is always true, or which only holds under certain circumstances?

  • 2
    $\begingroup$ Median is highly sensitive to flipping a considerable proportion of points over it (the old median), even if the distance of that transfer wasn't great. $\endgroup$ – ttnphns Oct 30 '13 at 12:23
  • $\begingroup$ You might want to read Wiki's article on robust statistics en.m.wikipedia.org/wiki/Robust_statistics . The mean is very sensitive to gross errors, the median not. $\endgroup$ – Michael M Oct 30 '13 at 21:32
  • $\begingroup$ Just a minor comment: one thing the median is not robust against are small sample sizes. See here. $\endgroup$ – COOLSerdash Oct 31 '13 at 14:33
  • 2
    $\begingroup$ @COOLSerdash OK, but when you get down to sample sizes of 1 or 2, then the mean is identical to the median, and as such definitely no better. More generally, that's the limit as sample sizes diminish. $\endgroup$ – Nick Cox Oct 31 '13 at 14:36

Imagine that a variable takes values 0 and 1 with probability both 0.5. Sample from that distribution and most of the medians will be 0 or 1 and a very few exactly 0.5. The means will vary far less. The mean is much more stable in this circumstance.

Here is a sample graph of results. The plots are quantile plots, i.e. ordered values versus plotting position, a modified cumulative probability. The results are for 10,000 bootstrap samples from 1000 values, 500 each 0 and 1. The means range fortuitously but nicely from 0.436 to 0.564 with standard error 0.016. The medians are as said, with standard error 0.493. (Closed-form results are no doubt possible here too, but a graph makes the point vivid for all.)

enter image description here

But that is exceptional. It illustrates the least favourable case for medians, a symmetric bimodal distribution such that the median is likely to flip between different halves of the data. However, symmetric bimodal distributions are not especially common, but watch out for so-called U-shaped distributions in which the extremes are most common and intermediate values uncommon. Distributions that are unimodal, or in which the number of modes has only a small effect on median or mean, are more common.

As advised by every treatment of robust statistics, a very common situation is that your data come with tails heavier than Gaussian and/or with outliers, and in those circumstances median will almost always be more robust. The point is that that is not a universal general result.

All that said, what relevance is a general result? You can at a minimum establish by bootstrapping the relative variability of mean and median for your own data. That's what you care about.

  • 2
    $\begingroup$ The spread of the sampling distribution of the median is inversely proportional to the density in the middle of the distribution, which has nothing at all to do with the tails. One thing that makes the robust stats advice work is the implicit assumption that the distribution is unimodal (or reasonably close to it). $\endgroup$ – whuber Oct 31 '13 at 14:50
  • $\begingroup$ ... so long as that density is positive. $\endgroup$ – Nick Cox Oct 31 '13 at 14:56
  • 2
    $\begingroup$ Properly interpreted, the result is true even for zero density. :-) $\endgroup$ – whuber Oct 31 '13 at 17:51

Where did you hear this? The usual reason for preferring the median is that it is less affected by extreme values than the mean. However, it is in general less sensitive to changes in the data.

I ran a tiny example in R

true <- rnorm(1000)
smallerror <- true + rnorm(1000,0,.1)
largeerror <- true + rnorm(1000, 0, 1)
bias <- true + rnorm(1000,1, .5)

mean(true) - mean(smallerror)
quantile(true, .5) - quantile(smallerror, .5)

mean(true) - mean(largeerror)
quantile(true, .5) - quantile(largeerror, .5)

In this particular case, the mean was more affected than the median.

  • 1
    $\begingroup$ This was advice emailed to me by an academic (not an academic in mathematics or statistics). I think his reasoning related to the fact that my sample size is rather small and/or the fact that I'm sampling probabilities, and I was interested to see if any respondents cited those as reasons for using the median instead of the mean. Are they in fact good reasons? $\endgroup$ – user1205901 Oct 30 '13 at 11:41

Your Answer

