PatchBatch: a Batch Augmented Loss for Optical Flow

We propose new loss functions for learning patch based descriptors via deep Convolutional Neural Networks. The learned descriptors are compared using the L2 norm and do not require network processing of pairs of patches. The success of the method is based on a few technical novelties, including an innovative loss function that, for each training batch, computes higher moments of the score distributions. Combined with an Approximate Nearest Neighbor patch matching method and a flow interpolating method, state of the art performance is obtained on the most challenging and competitive optical flow benchmarks.