Mixture Decompositions using a Decomposition of the Sample Space

We study the problem of finding the smallest $m$ for which every element $p$ of an exponential family $\E$ with finite sample space can be written as a mixture of $m$ elements of another exponential family $\E'$ as $p=\sum_{i=1}^m \alpha_i f_i$, where $f_i\in\mathcal{E}'$, $\alpha_i \geq 0$ $\forall i$ and $\sum_{i=1}^m \alpha_i =1$. Our approach is based on coverings and packings of the face lattice of the corresponding convex support polytopes. We use the notion of $S$-sets, subsets of the sample space such that every probability distribution that they support is contained in the closure of $\E$. We find, in particular, that $m=q^{N-1}$ yields the smallest mixtures of product distributions containing all distributions of $N$ $q$-ary variables, and that any distribution of $N$ binary variables is a mixture of $m = 2^{N-(k+1)}(1+ 1/(2^k-1))$ elements of the $k$-interaction exponential family ($k=1$ describes product distributions).