CSVideoNet: A Real-time End-to-end Learning Framework for High-frame-rate Video Compressive Sensing

This paper addresses the real-time encoding-decoding problem for high-frame-rate video compressive sensing (CS). Unlike prior works that perform reconstruction using iterative optimization-based approaches, we propose a non-iterative model, named "CSVideoNet". CSVideoNet directly learns the inverse mapping of CS and reconstructs the original input in a single forward propagation. To overcome the limitations of the existing CS cameras, we propose a multi-rate CNN and a synthesizing RNN to improve compression ratio and spatial-temporal resolution of the reconstructed videos. The experimental results demonstrate that CSVideoNet significantly outperforms the state-of-the-art approaches. With no pre/post-processing, we achieve 25dB PSNR recovery qulity at 100x compression ratio(CR), with a frame rate of 125 fps. Due to the feedforward and high-data-concurrency natures of CSVideoNet, it can take advantage of GPU acceleration to achieve three orders of magnitude speed-up over conventional iterative-based approaches. We share source code at https://github.com/PSCLab-ASU/CSVideoNet.