A trans-disciplinary review of deep learning research for water resources scientists

Deep learning (DL), a new-generation artificial neural network research, has made profound strides in recent years. This review paper is intended to provide hydrologists with a simple technical overview, trans-disciplinary progress update, and potentially inspirations about DL. Novel architectures, large and more accessible data, and new computing power enabled the success of DL. The review reveals that DL is rapidly transforming myriad scientific disciplines including high-energy physics, astronomy, chemistry, genomics and remote sensing, where systematic DL toolkits, innovative customizations, and sub-disciplines have emerged. However, with a few exceptions, its adoption in hydrology has so far been gradual. The literature suggests that novel regularization techniques can effectively prevent high-capacity deep networks from overfitting. As a result, in most scientific disciplines, DL models demonstrated superior predictive and generalization performance to conventional methods. Meanwhile, less noticed is that DL may also serve as a scientific exploratory tool. A new area termed "AI neuroscience", has been born. This budding sub-discipline is accumulating a significant body of work, e.g., distilling knowledge obtained in DL networks to interpretable models, attributing decisions to inputs via back-propagation of relevance, or visualization of activations. These methods are designed to interpret the decision process of deep networks and derive insights. While scientists so far have mostly been using customized, ad-hoc methods for interpretation, vast opportunities await for DL to propel science advancement, along with greatly enhanced predictive capability.