Unsupervised Learning on Neural Network Outputs

The outputs of a trained neural network contain much richer information than just an one-hot classifier. For example, a neural network may give an image of a dog the probability of one in a million of being a cat but it is still much larger than the probability of being a car. To reveal the hidden structure in them, we apply two unsupervised learning algorithms, PCA and ICA, to the outputs of a deep convolutional neural network trained on the ImageNet of 1000 classes. The PCA/ICA embedding of the object classes reveals their visual similarity and the PCA/ICA components can be interpreted as common features shared by visually similar object classes. For an application, we show that the learned PCA/ICA can be useful for zero-shot learning. Our new zero-shot learning method outperforms previous state-of-the-art methods on the ImageNet of over 20000 classes.