Revisiting Initialization of Neural Networks

Good initialization of weights is crucial for effective training of deep neural networks. In this paper we discuss an intialization scheme based on rigorous estimation of the local curvature. The proposed approach is more systematic and general than the state-of-art intialization from Glorot at al. and follow-up works.