Neural Networks Regularization Through Invariant Features Learning

Training deep neural networks is known to require a large number of training samples. However, in many applications only few training samples are available. In this work, we tackle the issue of training neural networks for classification task when few training samples are available. We attempt to solve this issue by proposing a regularization term that constrains the hidden layers of a network to learn class-wise invariant features. In our regularization framework, learning invariant features is generalized to the class membership where samples with the same class should have the same feature representation. Numerical experiments over MNIST and its variants showed that our proposal is more efficient for the case of few training samples. Moreover, we show an intriguing property of representation learning within neural networks. The source code of our framework is freely available https://github.com/sbelharbi/learning-class-invariant-features.