Safe Testing

We develop the theory of hypothesis testing based on the e-value, a notion of evidence that, unlike the p-value, allows for effortlessly combining results from several tests. Even in the common scenario of optional continuation, where the decision to perform a new test depends on previous test outcomes, 'safe' tests based on e-values generally preserve Type-I error guarantees. Our main result shows that e-values exist for completely general testing problems with composite null and alternatives. Their prime interpretation is in terms of gambling or investing, each e-value corresponding to a particular investment. Surprisingly, optimal 'GROW' e-values, which lead to fastest capital growth, are fully characterized by the joint information projection (JIPr) between the set of all Bayes marginal distributions on H0 and H1. Thus, optimal e-values also have an interpretation as Bayes factors, with priors given by the JIPr. We illustrate the theory using several 'classic' examples including a one-sample safe t-test and the 2 x 2 contingency table. Sharing Fisherian, Neymanian and Jeffreys-Bayesian interpretations, e-values and safe tests may provide a methodology acceptable to adherents of all three schools.