First Exit Time Analysis of Stochastic Gradient Descent Under
Heavy-Tailed Gradient Noise

First Exit Time Analysis of Stochastic Gradient Descent Under Heavy-Tailed Gradient Noise

21 June 2019

Mert Gurbuzbalaban

Papers citing "First Exit Time Analysis of Stochastic Gradient Descent Under Heavy-Tailed Gradient Noise"

15 / 15 papers shown

Title
Privacy of SGD under Gaussian or Heavy-Tailed Noise: Guarantees without Gradient Clipping Umut Simsekli Mert Gurbuzbalaban S. Yıldırım Lingjiong Zhu 43 2 0 04 Mar 2024
From Mutual Information to Expected Dynamics: New Generalization Bounds for Heavy-Tailed SGD Benjamin Dupuis Paul Viallard 23 3 0 01 Dec 2023
Efficient Sampling of Stochastic Differential Equations with Positive Semi-Definite Models Anant Raj Umut Simsekli Alessandro Rudi DiffM 33 1 0 30 Mar 2023
Algorithmic Stability of Heavy-Tailed SGD with General Loss Functions Anant Raj Lingjiong Zhu Mert Gurbuzbalaban Umut Simsekli 39 15 0 27 Jan 2023
Two Facets of SDE Under an Information-Theoretic Lens: Generalization of SGD via Training Trajectories and via Terminal States Ziqiao Wang Yongyi Mao 35 10 0 19 Nov 2022
Taming Fat-Tailed ("Heavier-Tailed'' with Potentially Infinite Variance) Noise in Federated Learning Haibo Yang Pei-Yuan Qiu Jia Liu FedML 45 12 0 03 Oct 2022
Trajectory-dependent Generalization Bounds for Deep Neural Networks via Fractional Brownian Motion Chengli Tan Jiang Zhang Junmin Liu 48 1 0 09 Jun 2022
Anticorrelated Noise Injection for Improved Generalization Antonio Orvieto Hans Kersting F. Proske Francis R. Bach Aurelien Lucchi 78 44 0 06 Feb 2022
Exponential escape efficiency of SGD from sharp minima in non-stationary regime Hikaru Ibayashi Masaaki Imaizumi 34 4 0 07 Nov 2021
On the Sample Complexity and Metastability of Heavy-tailed Policy Search in Continuous Control Amrit Singh Bedi Anjaly Parayil Junyu Zhang Mengdi Wang Alec Koppel 38 15 0 15 Jun 2021
Fractal Structure and Generalization Properties of Stochastic Optimization Algorithms A. Camuto George Deligiannidis Murat A. Erdogdu Mert Gurbuzbalaban Umut cSimcsekli Lingjiong Zhu 38 29 0 09 Jun 2021
On the Validity of Modeling SGD with Stochastic Differential Equations (SDEs) Zhiyuan Li Sadhika Malladi Sanjeev Arora 49 78 0 24 Feb 2021
Hausdorff Dimension, Heavy Tails, and Generalization in Neural Networks Umut Simsekli Ozan Sener George Deligiannidis Murat A. Erdogdu 49 55 0 16 Jun 2020
A Diffusion Theory For Deep Learning Dynamics: Stochastic Gradient Descent Exponentially Favors Flat Minima Zeke Xie Issei Sato Masashi Sugiyama ODL 28 17 0 10 Feb 2020
On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima N. Keskar Dheevatsa Mudigere J. Nocedal M. Smelyanskiy P. T. P. Tang ODL 312 2,896 0 15 Sep 2016