Unsupervised Learning of Spatiotemporally Coherent Metrics

In this work we study feature learning in the context of temporally coherent video data. We focus on feature learning from unlabeled video data, using the assumption that adjacent video frames contain semantically similar information. This assumption is exploited to train a convolutional pooling auto-encoder regularized by slowness and sparsity. We establish a connection between slow feature learn- ing and metric learning, and show that the trained encoder can be used to define a more temporally and semantically coherent metric.