On Clustering and Embedding Manifolds using a Low Rank Neighborhood Approach

Simultaneous clustering and embedding of multiple manifolds is an important task in the field of manifold learning. Manifold clustering and embedding algorithms perform particularly poorly on the embedding task for highly nonlinear manifolds. In this paper we propose a novel algorithm for improved manifold clustering and embedding. The new algorithm is similar to a variety of existing algorithms for manifold clustering in that it is graph based and uses a specific strategy to ensure that only data-points belonging to the same manifold are chosen as neighbors for each point. The new algorithm attempts to acheive this objective by adding a low-rank criterion on the neighborhood of each data-point to ensure that only data-points belonging to the same manifold are "prioritized" for neighbor selection. Following this a reconstruction matrix is calculated to express each data-point as an affine combination of its neighbors on the same manifold. If the low rank neighborhood criterion succeeds in choosing (prioritizing) data-points belonging to the same manifold as neighbors, then the reconstruction matrix is (near) block diagonal. This reconstruction matrix can then be used for clustering and embedding. Over a variety of simulated and real data-sets we will show that the low rank neighborhood based algorithm shows improvements on the state-of-the-art manifold clustering and embedding algorithms in terms of both clustering and embedding performance.