Distributional Smoothing by Virtual Adversarial Examples

We propose a novel regularization technique for supervised and semi-supervised training of large models like deep neural network. By including into objective function the local smoothness of predictive distribution around each training data point, not only were we able to extend the Adversarial training to the setting of semi-supervised training, we were also able to eclipse current state of the art supervised and semi-supervised methods on the permutation invariant MNIST classification task.