Coarse-to-fine Depth Estimation from a Single Image via Coupled Regression and Dictionary Learning

In this work we present a novel coarse-to-fine approach to depth prediction from single images. The coarse estima- tion part of the framework predicts a low-resolution depth map by means of a holistic method that considers the whole image at once in order to exploit global information and context in the scene. This initial estimate is then used as a prior to guide subsequent local refinements at higher res- olutions that add missing details and improve the overall accuracy of the system. Both the coarse estimation as well as the successive refinements formulate the task as a regres- sion problem from the image domain to the depth space. However, rather than directly regressing on depth, we pro- pose to learn a compact depth dictionary and a mapping from image features to reconstructive depth weights. The dictionary and the mapping are coupled and jointly opti- mized by our learning scheme. We demonstrate that this results in a significant improvement in accuracy compared to direct depth regression or approaches using depth dictio- naries learned disjointly from the mapping. Experiments on the NYUv2 [17] and KITTIDepth [7] datasets show that our method outperforms the existing state-of-the-art by a large margin.