32
18
v1v2v3v4v5 (latest)

Higher Criticism to Compare Two Large Frequency Tables, with sensitivity to Possible Rare and Weak Differences

Abstract

We adapt Higher Criticism (HC) to the comparison of two frequency tables which may -- or may not -- exhibit moderate differences between the tables in some unknown, relatively small subset out of a large number of categories. Our analysis of the power of the proposed HC test quantifies the rarity and size of assumed differences and applies moderate deviations-analysis to determine the asymptotic powerfulness/powerlessness of our proposed HC procedure. Our analysis considers the null hypothesis of no difference in underlying generative model against a rare/weak perturbation alternative, in which the frequencies of N1βN^{1-\beta} out of the NN categories are perturbed by r(logN)/2nr(\log N)/2n in the Hellinger distance; here nn is the size of each sample. Our proposed Higher Criticism (HC) test for this setting uses P-values obtained from NN exact binomial tests. We characterize the asymptotic performance of the HC-based test in terms of the rarity parameter β\beta and the perturbation intensity parameter rr. Specifically, we derive a region in the (β,r)(\beta,r)-plane where the test asymptotically has maximal power, while having asymptotically no power outside this region. Our analysis distinguishes between cases in which the counts in both tables are low, versus cases in which counts are high, corresponding to the cases of sparse and dense frequency tables. The phase transition curve of HC in the high-counts regime matches formally the curve delivered by HC in a two-sample normal means model.

View on arXiv
Comments on this paper