Stochastic Dynamics for Video Infilling

This paper introduces a stochastic generation framework (SDVI) to infill long intervals in video sequences. Video interpolation aims to produce transitional content for a short interval between every two frames to increase the temporal resolution. Video Infilling, however, aims to infill long intervals in a video with a possible sequence. Our framework models the infilling as a constrained stochastic generation process and sequentially samples dynamics from the inferred distribution. SDVI consists of two parts: (1) a bi-directional constraint propagation to guarantee the spatial-temporal coherency among frames, (2) a stochastic sampling process to generate dynamics from the inferred distributions. Experimental results show that SDVI can generate clear and varied sequences. Moreover, motions in the generated sequences are realistic and able to transfer smoothly from the referenced start frame to the terminal frame. Full paper see arXiv:1809.00263