Simultaneous Hypothesis Testing Using Ranks Among Negative Controls

Consider a simultaneous hypothesis testing problem where each hypothesis is associated with a test statistic. Suppose it is difficult to obtain the null distribution of the test statistics, but some null hypotheses--referred to as the internal negative controls--are known to be true. When it is reasonable to assume that the test statistics associated with the negative controls are exchangeable with those associated with the unknown true null hypotheses, we propose to use a statistic's Rank Among Negative Controls (RANC) as a p-value for the corresponding hypothesis. We provide two theoretical prospectives on this proposal. First, we view the empirical distribution of the negative control statistics as an estimate of the null distribution. We use this to show that, when the test statistics are exchangeable, the RANC p-values are individually valid and have a positive regression dependence on the subset of true nulls. Second, we study the empirical processes of the test statistics indexed by the rejection threshold. We use this to show that the Benjamini-Hochberg procedure applied to the RANC p-values may still control the false discovery rate when the test statistics are not exchangeable. The practical performance of our method is illustrated using numerical simulations and a real proteomic dataset.