Train faster, generalize better: Stability of stochastic gradient descent

3 September 2015

Benjamin Recht

Papers citing "Train faster, generalize better: Stability of stochastic gradient descent"

50 / 199 papers shown

Title
Resampling Sensitivity of High-Dimensional PCA Haoyu Wang 21 0 0 30 Dec 2022
On the Overlooked Structure of Stochastic Gradients Zeke Xie Qian-Yuan Tang Mingming Sun P. Li 25 6 0 05 Dec 2022
Two Facets of SDE Under an Information-Theoretic Lens: Generalization of SGD via Training Trajectories and via Terminal States Ziqiao Wang Yongyi Mao 27 10 0 19 Nov 2022
On the Algorithmic Stability and Generalization of Adaptive Optimization Methods Han Nguyen Hai Pham Sashank J. Reddi Barnabás Póczos ODL AI4CE 15 2 0 08 Nov 2022
Do highly over-parameterized neural networks generalize since bad solutions are rare? Julius Martinetz T. Martinetz 22 1 0 07 Nov 2022
Distributed DP-Helmet: Scalable Differentially Private Non-interactive Averaging of Single Layers Moritz Kirschte Sebastian Meiser Saman Ardalan Esfandiar Mohammadi FedML 29 0 0 03 Nov 2022
Optimal Algorithms for Stochastic Complementary Composite Minimization Alexandre d’Aspremont Cristóbal Guzmán Clément Lezane 25 3 0 03 Nov 2022
On Stability and Generalization of Bilevel Optimization Problem Meng Ding Ming Lei Yunwen Lei Di Wang Jinhui Xu 29 0 0 03 Oct 2022
Stability Analysis and Generalization Bounds of Adversarial Training Jiancong Xiao Yanbo Fan Ruoyu Sun Jue Wang Zhimin Luo AAML 24 30 0 03 Oct 2022
Adaptive Smoothness-weighted Adversarial Training for Multiple Perturbations with Its Stability Analysis Jiancong Xiao Zeyu Qin Yanbo Fan Baoyuan Wu Jue Wang Zhimin Luo AAML 31 7 0 02 Oct 2022
Exploring the Algorithm-Dependent Generalization of AUPRC Optimization with List Stability Peisong Wen Qianqian Xu Zhiyong Yang Yuan He Qingming Huang 53 10 0 27 Sep 2022
On the Stability Analysis of Open Federated Learning Systems Youbang Sun H. Fernando Tianyi Chen Shahin Shahrampour FedML 29 1 0 25 Sep 2022
Stability and Generalization for Markov Chain Stochastic Gradient Methods Puyu Wang Yunwen Lei Yiming Ying Ding-Xuan Zhou 16 18 0 16 Sep 2022
On Generalization of Decentralized Learning with Separable Data Hossein Taheri Christos Thrampoulidis FedML 27 10 0 15 Sep 2022
Differentially Private Stochastic Gradient Descent with Low-Noise Puyu Wang Yunwen Lei Yiming Ying Ding-Xuan Zhou FedML 43 5 0 09 Sep 2022
Generalisation under gradient descent via deterministic PAC-Bayes Eugenio Clerico Tyler Farghly George Deligiannidis Benjamin Guedj Arnaud Doucet 26 4 0 06 Sep 2022
SYNTHESIS: A Semi-Asynchronous Path-Integrated Stochastic Gradient Method for Distributed Learning in Computing Clusters Zhuqing Liu Xin Zhang Jia-Wei Liu 25 1 0 17 Aug 2022
Uniform Stability for First-Order Empirical Risk Minimization Amit Attia Tomer Koren 18 5 0 17 Jul 2022
Bootstrap State Representation using Style Transfer for Better Generalization in Deep Reinforcement Learning Md Masudur Rahman Yexiang Xue OffRL 23 4 0 15 Jul 2022
Sparse Double Descent: Where Network Pruning Aggravates Overfitting Zhengqi He Zeke Xie Quanzhi Zhu Zengchang Qin 69 27 0 17 Jun 2022
Trajectory-dependent Generalization Bounds for Deep Neural Networks via Fractional Brownian Motion Chengli Tan Jiang Zhang Junmin Liu 35 1 0 09 Jun 2022
Multi-class Classification with Fuzzy-feature Observations: Theory and Algorithms Guangzhi Ma Jie Lu Feng Liu Zhen Fang Guangquan Zhang 8 6 0 09 Jun 2022
Subject Membership Inference Attacks in Federated Learning Anshuman Suri Pallika H. Kanani Virendra J. Marathe Daniel W. Peterson 30 25 0 07 Jun 2022
Dimension Independent Generalization of DP-SGD for Overparameterized Smooth Convex Optimization Yi-An Ma T. V. Marinov Tong Zhang 17 8 0 03 Jun 2022
AANG: Automating Auxiliary Learning Lucio Dery Paul Michel M. Khodak Graham Neubig Ameet Talwalkar 36 9 0 27 May 2022
Selective Classification Via Neural Network Training Dynamics Stephan Rabanser Anvith Thudi Kimia Hamidieh Adam Dziedzic Nicolas Papernot 24 21 0 26 May 2022
Learning from time-dependent streaming data with online stochastic algorithms Antoine Godichon-Baggioni Nicklas Werge Olivier Wintenberger 22 3 0 25 May 2022
Uniform Generalization Bound on Time and Inverse Temperature for Gradient Descent Algorithm and its Application to Analysis of Simulated Annealing Keisuke Suzuki AI4CE 30 0 0 25 May 2022
Beyond Lipschitz: Sharp Generalization and Excess Risk Bounds for Full-Batch GD Konstantinos E. Nikolakakis Farzin Haddadpour Amin Karbasi Dionysios S. Kalogerias 40 17 0 26 Apr 2022
Sharper Utility Bounds for Differentially Private Models Yilin Kang Yong Liu Jian Li Weiping Wang FedML 26 3 0 22 Apr 2022
Stability and Risk Bounds of Iterative Hard Thresholding Xiao-Tong Yuan P. Li 37 12 0 17 Mar 2022
Stability vs Implicit Bias of Gradient Methods on Separable Data and Beyond Matan Schliserman Tomer Koren 22 23 0 27 Feb 2022
Benign Underfitting of Stochastic Gradient Descent Tomer Koren Roi Livni Yishay Mansour Uri Sherman MLT 15 13 0 27 Feb 2022
Thinking Outside the Ball: Optimal Learning with Gradient Descent for Generalized Linear Stochastic Convex Optimization I Zaghloul Amir Roi Livni Nathan Srebro 22 6 0 27 Feb 2022
Tackling benign nonconvexity with smoothing and stochastic gradients Harsh Vardhan Sebastian U. Stich 20 8 0 18 Feb 2022
Differentially Private SGDA for Minimax Problems Zhenhuan Yang Shu Hu Yunwen Lei Kush R. Varshney Siwei Lyu Yiming Ying 36 19 0 22 Jan 2022
Low-Pass Filtering SGD for Recovering Flat Optima in the Deep Learning Optimization Landscape Devansh Bisla Jing Wang A. Choromańska 25 34 0 20 Jan 2022
Stability Based Generalization Bounds for Exponential Family Langevin Dynamics A. Banerjee Tiancong Chen Xinyan Li Yingxue Zhou 23 8 0 09 Jan 2022
Stable Conformal Prediction Sets Eugène Ndiaye 35 20 0 19 Dec 2021
Communication-Efficient Distributed Learning via Sparse and Adaptive Stochastic Gradient Xiaoge Deng Dongsheng Li Tao Sun Xicheng Lu FedML 16 0 0 08 Dec 2021
Black-box tests for algorithmic stability Byol Kim Rina Foygel Barber AAML 17 13 0 30 Nov 2021
Multi-fidelity Stability for Graph Representation Learning Yihan He Joan Bruna 17 0 0 25 Nov 2021
Subspace Adversarial Training Tao Li Yingwen Wu Sizhe Chen Kun Fang Xiaolin Huang AAML OOD 38 56 0 24 Nov 2021
Gaussian Process Inference Using Mini-batch Stochastic Gradient Descent: Convergence Guarantees and Empirical Benefits Hao Chen Lili Zheng Raed Al Kontar Garvesh Raskutti 15 3 0 19 Nov 2021
Training Neural Networks with Fixed Sparse Masks Yi-Lin Sung Varun Nair Colin Raffel FedML 18 196 0 18 Nov 2021
Convergence and Stability of the Stochastic Proximal Point Algorithm with Momentum J. Kim Panos Toulis Anastasios Kyrillidis 24 8 0 11 Nov 2021
Differentially Private Coordinate Descent for Composite Empirical Risk Minimization Paul Mangold A. Bellet Joseph Salmon Marc Tommasi 27 14 0 22 Oct 2021
Towards Open-World Feature Extrapolation: An Inductive Graph Learning Approach Qitian Wu Chenxiao Yang Junchi Yan 19 32 0 09 Oct 2021
On the Generalization of Models Trained with SGD: Information-Theoretic Bounds and Implications Ziqiao Wang Yongyi Mao FedML MLT 37 22 0 07 Oct 2021
Spectral Bias in Practice: The Role of Function Frequency in Generalization Sara Fridovich-Keil Raphael Gontijo-Lopes Rebecca Roelofs 35 28 0 06 Oct 2021