110
v1v2 (latest)

Logical contradictions in the One-way ANOVA and Tukey-Kramer multiple comparisons tests with more than two groups of observations

Abstract

We show that the One-way ANOVA and Tukey-Kramer (TK) tests agree on any sample with two groups. This result is based on a simple identity connecting the Fisher-Snedecor and studentized probabilistic distributions and is proven without any additional assumptions; in particular, the standard ANOVA assumptions (independence, normality, and homoscedasticity (INAH)) are not needed. In contrast, it is known that for a sample with k > 2 groups of observations, even under the INAH assumptions, with the same significance level α\alpha, the above two tests may give opposite results: (i) ANOVA rejects its null hypothesis H0A:μ1==μkH_0^{A}: \mu_1 = \ldots = \mu_k, while the TK one, H0TK(i,j):μi=μjH_0^{TK}(i,j): \mu_i = \mu_j, is not rejected for any pair i,j{1,,k}i, j \in \{1, \ldots, k\}; (ii) the TK test rejects H0TK(i,j)H_0^{TK}(i,j) for a pair (i,j)(i, j) (with iji \neq j) while ANOVA does not reject H0AH_0^{A}. We construct two large infinite pseudo-random families of samples of both types satisfying INAH: in case (i) for any k3k \geq 3 and in case (ii) for some larger kk. Furthermore, in case (ii) ANOVA, being restricted to the pair of groups (i,j)(i,j), may reject equality μi=μj\mu_i = \mu_j with the same α\alpha. This is an obvious contradiction, since μ1==μk\mu_1 = \ldots = \mu_k implies μi=μj\mu_i = \mu_j for all i,j{1,,k}.i, j \in \{1, \ldots, k\}. Similar contradictory examples are constructed for the Multivariable Linear Regression (MLR). However, for these constructions it seems difficult to verify the Gauss-Markov assumptions, which are standardly required for MLR.

View on arXiv
Comments on this paper