I'm testing for mutations in DNA and I'm using the Benjamini-Hochberg method to modify the threshold I'm testing my p-values against. The method is basically to rank the p-values and compare them to a new threshold defined by

$q(k) = k/n*\alpha$,

where $\alpha$ is your original threshold, commonly 0.05, $k$ is the rank of the p-value you compare with, and $n$ is the total amount of p-values (== total amount of tests).

Due to the nature of my data, many p-values are identical. The following ordered set of p-values could be an example:

$p_1$ = 0.01 $p_2$ = 0.03 $p_3$ = 0.03 $p_4$ = 0.03 $p_5$ = 0.09

Assuming a threshold of $\alpha$ = 0.05, what I have to compare my values to are:

$q_1$ = 0.01 $q_2$ = 0.02 $q_3$ = 0.03 $q_4$ = 0.04 $q_5$ = 0.05

And thus I accept the second test and reject the third and fourth even if they have the same original p-value. I tried to search the literature for a solution to this, but nothing came up. Is there an established method for this? And, if not, what is your preferred ad hoc solution?


Verifying also in the multtest package from Bioconductor, I would suggest to give them the same rank - and very importantly - increment the rank by one for the following p-value(s) rather than using their index+1 in an array! This would have the following result:

considering your examplemulttest's BH would rank $r_1$: 1, $r_2$: 2, $r_3$: 2, $r_4$: 2, $r_5$: 3 rather than of $r_2$: 2, $r_3$: 2, $r_4$: 2, $r_5$: 5

| cite | improve this answer | |
  • $\begingroup$ Thank you. I accept this answer although it is essentially the same as the one by Maciej because of the extra detail. $\endgroup$ – Viktiglemma Oct 5 '12 at 12:48
  • $\begingroup$ Thank you - I effectively picked up on the comment of Maciej. I often miss the possibility to accept more than one answer! $\endgroup$ – dmeu Oct 16 '12 at 13:09
  • $\begingroup$ I doubt that it is correct. In practice very rarely one can get two equal p-values. $\endgroup$ – Viktor Nov 8 '17 at 15:17

Possible ad hoc solution is to give repeated p-values the same rank.

| cite | improve this answer | |
  • $\begingroup$ I've just exploring my multtest package results and I see that I have a subset of the same FDR corrected values [1] 0.08199 0.08199 0.08199 0.08199 0.08199 0.08199 0.08199 0.08199 0.08199 [10] 0.08199 So maybe it is as I wrote. $\endgroup$ – boczniak767 Nov 24 '11 at 19:36

Your Answer

