Progressive Stochastic Greedy Sparse Reconstruction and Support Selection

Sparse reconstruction and sparse support selection, i.e., the tasks of inferring an arbitrary $m$-dimensional sparse vector $\mathbf{x}$ having $k$ nonzero entries from $n$ measurements of linear combinations of its components, are often encountered in machine learning, computer vision, and signal processing. Existing greedy-based algorithms achieve optimal $n = \mathcal{O}(k\log\frac{m}{k})$ sampling complexity with computational complexity that is linear in the size of the data $m$ and cardinality constraint $k$. However, the ${\mathcal{O}}(mk)$ computational complexity is still prohibitive for large-scale datasets. In this paper, we present the first sparse support selection algorithm for arbitrary sparse vectors that achieves exact identification of the optimal subset from $n = \mathcal{O}(k\log\frac{m}{k})$ measurements with $\tilde{\mathcal{O}}(m)$ computational complexity. The proposed scheme utilizes the idea of randomly restricting search space of the greedy method in a progressive manner to reduce the computational cost while maintaining the same order of sampling complexity as the existing greedy schemes. Simulation results including an application of the algorithm to the task of column subset selection demonstrate efficacy of the proposed algorithm.