Dense Optical Flow Prediction from a Static Image

Given a scene, what is going to move, and in what direction will it move? Such a question could be considered a non-semantic form of action prediction. In this work, we present predictive convolutional neural networks (P-CNN). Given a static image, P-CNN predicts the future motion of each and every pixel in the image in terms of optical flow. Our P-CNN model leverages the data in tens of thousands of realistic videos to train our model. Our method relies on absolutely no human labeling and is able to predict motion based on the context of the scene. Since P-CNNs make no assumptions about the underlying scene they can predict future optical flow on a diverse set of scenarios. In terms of quantitative performance, P-CNN outperforms all previous approaches by large margins.