Fractional Underdamped Langevin Dynamics: Retargeting SGD with Momentum
under Heavy-Tailed Gradient Noise

v1v2 (latest)

Fractional Underdamped Langevin Dynamics: Retargeting SGD with Momentum under Heavy-Tailed Gradient Noise

13 February 2020

Mert Gurbuzbalaban

ArXiv (abs)PDF HTML

Papers citing "Fractional Underdamped Langevin Dynamics: Retargeting SGD with Momentum under Heavy-Tailed Gradient Noise"

15 / 15 papers shown

Title
On the Convergence of Continual Learning with Adaptive Methods Seung Han Yeon-Jae Kim Taehyun Cho Jungwoo Lee CLL 65 4 0 08 Apr 2024
Fast Replica Exchange Stochastic Gradient Langevin Dynamics Guanxun Li Guang Lin Zecheng Zhang Quan Zhou 437 4 0 05 Jan 2023
Taming Fat-Tailed ("Heavier-Tailed'' with Potentially Infinite Variance) Noise in Federated Learning Haibo Yang Pei-Yuan Qiu Jia Liu FedML 74 12 0 03 Oct 2022
Unbiased Estimation using Underdamped Langevin Dynamics Hamza Ruzayqat Neil K. Chada Ajay Jasra 70 6 0 14 Jun 2022
Heavy-tailed Sampling via Transformed Unadjusted Langevin Algorithm Ye He Krishnakumar Balasubramanian Murat A. Erdogdu 83 5 0 20 Jan 2022
Intrinsic Dimension, Persistent Homology and Generalization in Neural Networks Tolga Birdal Aaron Lou Leonidas Guibas Umut cSimcsekli 73 65 0 25 Nov 2021
Revisiting the Characteristics of Stochastic Gradient Noise and Dynamics Yixin Wu Rui Luo Chen Zhang Jun Wang Yaodong Yang 98 7 0 20 Sep 2021
On the Sample Complexity and Metastability of Heavy-tailed Policy Search in Continuous Control Amrit Singh Bedi Anjaly Parayil Junyu Zhang Mengdi Wang Alec Koppel 88 15 0 15 Jun 2021
Fractal Structure and Generalization Properties of Stochastic Optimization Algorithms A. Camuto George Deligiannidis Murat A. Erdogdu Mert Gurbuzbalaban Umut cSimcsekli Lingjiong Zhu 80 29 0 09 Jun 2021
Heavy Tails in SGD and Compressibility of Overparametrized Neural Networks Melih Barsbey Romain Chor Murat A. Erdogdu Gaël Richard Umut Simsekli 66 41 0 07 Jun 2021
Characterization of Generalizability of Spike Timing Dependent Plasticity trained Spiking Neural Networks Biswadeep Chakraborty Saibal Mukhopadhyay 125 15 0 31 May 2021
On Proximal Policy Optimization's Heavy-tailed Gradients Saurabh Garg Joshua Zhanson Emilio Parisotto Adarsh Prasad J. Zico Kolter Zachary Chase Lipton Sivaraman Balakrishnan Ruslan Salakhutdinov Pradeep Ravikumar 98 13 0 20 Feb 2021
The shifted ODE method for underdamped Langevin MCMC James Foster Terry Lyons Harald Oberhauser 93 16 0 10 Jan 2021
Hausdorff Dimension, Heavy Tails, and Generalization in Neural Networks Umut Simsekli Ozan Sener George Deligiannidis Murat A. Erdogdu 86 56 0 16 Jun 2020
Generalized Energy Based Models Michael Arbel Liang Zhou Arthur Gretton DRL 169 81 0 10 Mar 2020