Art of singular vectors and universal adversarial perturbations

Vulnerability of state-of-the-art deep neural networks to adversarial attacks has been attracting a lot of attention recently. In this work we propose a new algorithm for constructing universal adversarial perturbations. Our approach is based on computing the so called $(p, q)$-singular vectors of the Jacobian matrices of hidden layers of a network. Resulting perturbations present interesting visual patterns and by using a batch of just $64$ images we can construct adversarial perturbations with relatively high fooling rate. We also investigate a correlation between the singular values of the Jacobian matrices and the fooling rate of a corresponding singular vector.