Trustworthy Optimization: A Novel Approach to Counter Numerical Instability in 16-bit Neural Network Training

In this research, we address critical trustworthiness concerns related to the numerical instability observed in 16-bit computations of machine learning models. Such instability, particularly when employing popular optimization algorithms like RMSProp and Adam, often leads to unreliable training of deep neural networks. This not only disrupts the learning process but also poses significant challenges in deploying dependable models in real-world applications. Our investigation identifies the epsilon hyperparameter as the primary source of this instability. A nuanced exploration reveals that subtle adjustments to epsilon within 16-bit computations can enhance the reliability of RMSProp and Adam, enabling more trustworthy training of 16-bit neural networks. We propose a novel, dependable approach that leverages updates from the Adam optimizer to bolster the stability of the learning process. Our contributions provide deeper insights into optimization challenges in low-precision computations and offer solutions to ensure the trustworthiness and stability of deep neural network training, paving the way for their dependable use in various applications.