Patch Tracking-based Online Tensor Ring Completion for Streaming Visual Data

Tensor completion is the problem of estimating the missing entries of a partially observed tensor by exploiting its low-rank structure. In streaming applications where the frames arrive sequentially such as video completion, the missing entries of the tensor need to be dynamically recovered in an online fashion. Traditional online completion algorithms treat the entire visual data as a tensor, which may not work satisfactorily when there is a big change in the tensor subspace along the temporal dimension, such as due to strong motion across the video frames. In this paper, we develop a novel patch tracking-based online tensor completion framework for streaming data. Each incoming tensor is extracted into small patches, and similar patches are tracked along the temporal domain. We propose a new patch tracking strategy that can accurately and efficiently track the patches with missing data. Further, a new online tensor ring completion method is proposed which can efficiently and accurately update the latent core tensors and complete the missing entries of the tracked patches. Extensive experimental results demonstrate the superior performance of the proposed algorithms compared with both online and offline state-of-the-art tensor completion methods.