Exact testing with random permutations

Permutation tests and other methods based on the permutation principle are highly popular, for instance in omics data analysis. Often random permutations are used, to limit the computation time. However, the existing theory on permutation tests usually assumes that all permutations are enumerated; little theory exists on testing with random permutations. It is known that naively using random permutations instead of the full permutation group, leads to anti-conservativeness. Also it has been claimed that adding the original observation to the random permutations solves this problem. A proof however is lacking. In this paper existing claims on validity of tests with random permutations are proven. Further, novel exact tests are presented.