Multi-Path Feedback Recurrent Neural Network for Scene Parsing

In this paper, we consider the scene parsing problem. We propose a novel \textbf{M}ulti-\textbf{P}ath \textbf{F}eedback recurrent neural network (MPF-RNN) to enhance the capability of RNNs on modeling long-range context information at multiple levels and better distinguish pixels that are easy to confuse in pixel-wise classification. In contrast to CNNs without feedback and RNNs with only a single feedback path, MPF-RNN propagates the contextual features learned at top layers through weighted recurrent connections to \emph{multiple} bottom layers to help them learn better features with such "hindsight". Besides, we propose a new training strategy which considers the loss accumulated at multiple recurrent steps to improve performance of the MPF-RNN on parsing small objects as well as stabilize the training procedure. We empirically demonstrate that such an architecture with multiple feedback paths can effectively enhance the capability of deep neural networks in classifying pixels which are hard to distinguish without higher-level context information. With these two novel components, MPF-RNN provides new state-of-the-art results on four challenging scene parsing benchmarks, including SiftFlow, Barcelona, CamVid and Stanford Background.