Sparse Subspace Clustering by Orthogonal Matching Pursuit

Subspace clustering methods based on $\ell_1$, $\ell_2$ or nuclear norm regularization have become very popular due to their simplicity, theoretical guarantees and empirical success. However, the choice of the regularizer can greatly impact both theory and practice. For instance, $\ell_1$ regularization is guaranteed to give the correct clustering under broad conditions (e.g., arbitrary subspaces and corrupted data), but requires solving a large scale convex optimization problem. On the other hand, $\ell_2$ and nuclear norm regularization provide efficient closed form solutions, but require very strong assumptions to guarantee the correct clustering (e.g., independent subspaces and uncorrupted data). This paper proposes a new subspace clustering method based on orthogonal matching pursuit that is computationally efficient and guaranteed to provide the correct clustering for arbitrary subspaces. Experiments on synthetic data verify our theoretical analysis, and applications in handwritten digit and face clustering show that our approach achieves the best trade off between accuracy and efficiency. Moreover, our approach is the only one that can handle 100,000 points.