The Four Point Permutation Test for Latent Block Structure in Incidence Matrices

Transactional data may be represented as a bipartite graph , where denotes agents, denotes objects visible to many agents, and an edge in denotes an interaction between an agent and an object. Unsupervised learning seeks to detect block structures in the adjacency matrix between and , thus grouping together sets of agents with similar object interactions. New results on quasirandom permutations suggest a non-parametric \textbf{four point test} to measure the amount of block structure in , with respect to vertex orderings on and . Take disjoint 4-edge random samples, order these four edges by left endpoint, and count the relative frequencies of the possible orderings of the right endpoint. When these orderings are equiprobable, the edge set corresponds to a quasirandom permutation of symbols. Total variation distance of the relative frequency vector away from the uniform distribution on 24 permutations measures the amount of block structure. Such a test statistic, based on samples, is computable in time on processors. Possibly block structure may be enhanced by precomputing \textbf{natural orders} on and , related to the second eigenvector of graph Laplacians. In practice this takes time, where is the graph diameter. Five open problems are described.
View on arXiv