Convergence of gradient descent for deep neural networks

This article presents a new criterion for convergence of gradient descent to a global minimum. The criterion is used to show that gradient descent with proper initialization converges to a global minimum when training any feedforward neural network with smooth and strictly increasing activation functions, provided that the input dimension is greater than or equal to the number of data points. The main difference with prior work is that the width of the network can be a fixed number instead of unrealistically growing as some multiple or power of the number of data points.