Orthogonalising gradients to speed up neural network optimisation

14 February 2022

Papers citing "Orthogonalising gradients to speed up neural network optimisation"

3 / 3 papers shown

Title
Generalized Gradient Norm Clipping & Non-Euclidean $(L_0,L_1)$ -Smoothness Thomas Pethick Wanyun Xie Mete Erdogan Kimon Antonakopoulos Tony Silveti-Falls Volkan Cevher 41 0 0 02 Jun 2025
SUMO: Subspace-Aware Moment-Orthogonalization for Accelerating Memory-Efficient LLM Training Yehonathan Refael Guy Smorodinsky Tom Tirer Ofir Lindenbaum 25 0 0 30 May 2025
Understanding Gradient Orthogonalization for Deep Learning via Non-Euclidean Trust-Region Optimization Dmitry Kovalev 132 5 0 16 Mar 2025