Train faster, generalize better: Stability of stochastic gradient descent

3 September 2015

Benjamin Recht

Papers citing "Train faster, generalize better: Stability of stochastic gradient descent"

50 / 199 papers shown

Title
Adversarial Representation Learning With Closed-Form Solvers Bashir Sadeghi Lan Wang Vishnu Naresh Boddeti 34 5 0 12 Sep 2021
NASI: Label- and Data-agnostic Neural Architecture Search at Initialization Yao Shu Shaofeng Cai Zhongxiang Dai Beng Chin Ooi K. H. Low 16 43 0 02 Sep 2021
Stability and Generalization for Randomized Coordinate Descent Puyu Wang Liang Wu Yunwen Lei 16 7 0 17 Aug 2021
Pointer Value Retrieval: A new benchmark for understanding the limits of neural network generalization Chiyuan Zhang M. Raghu Jon M. Kleinberg Samy Bengio OOD 24 30 0 27 Jul 2021
Improved Learning Rates for Stochastic Optimization: Two Theoretical Viewpoints Shaojie Li Yong Liu 20 13 0 19 Jul 2021
Shuffle Private Stochastic Convex Optimization Albert Cheu Matthew Joseph Jieming Mao Binghui Peng FedML 20 25 0 17 Jun 2021
Learning subtree pattern importance for Weisfeiler-Lehmanbased graph kernels Dai Hai Nguyen Canh Hao Nguyen Hiroshi Mamitsuka 14 6 0 08 Jun 2021
RATT: Leveraging Unlabeled Data to Guarantee Generalization Saurabh Garg Sivaraman Balakrishnan J. Zico Kolter Zachary Chase Lipton 28 30 0 01 May 2021
Positive-Negative Momentum: Manipulating Stochastic Gradient Noise to Improve Generalization Zeke Xie Li-xin Yuan Zhanxing Zhu Masashi Sugiyama 19 29 0 31 Mar 2021
Distributed Deep Learning Using Volunteer Computing-Like Paradigm Medha Atre B. Jha Ashwini Rao 18 11 0 16 Mar 2021
Membership Inference Attacks on Machine Learning: A Survey Hongsheng Hu Z. Salcic Lichao Sun Gillian Dobbie Philip S. Yu Xuyun Zhang MIACV 30 412 0 14 Mar 2021
Reframing Neural Networks: Deep Structure in Overcomplete Representations Calvin Murdock George Cazenavette Simon Lucey BDL 35 4 0 10 Mar 2021
Nonlinear Projection Based Gradient Estimation for Query Efficient Blackbox Attacks Huichen Li Linyi Li Xiaojun Xu Xiaolu Zhang Shuang Yang Bo-wen Li AAML 20 17 0 25 Feb 2021
Generalization Bounds for Meta-Learning via PAC-Bayes and Uniform Stability Alec Farid Anirudha Majumdar 24 34 0 12 Feb 2021
Stability of SGD: Tightness Analysis and Improved Bounds Yikai Zhang Wenjia Zhang Sammy Bald Vamsi Pingali Chao Chen Mayank Goswami MLT 19 36 0 10 Feb 2021
Generalization of Model-Agnostic Meta-Learning Algorithms: Recurring and Unseen Tasks Alireza Fallah Aryan Mokhtari Asuman Ozdaglar 23 49 0 07 Feb 2021
Estimating informativeness of samples with Smooth Unique Information Hrayr Harutyunyan Alessandro Achille Giovanni Paolini Orchid Majumder Avinash Ravichandran Rahul Bhotika Stefano Soatto 16 24 0 17 Jan 2021
Mixed-Privacy Forgetting in Deep Networks Aditya Golatkar Alessandro Achille Avinash Ravichandran M. Polito Stefano Soatto CLL MU 127 159 0 24 Dec 2020
A case for new neural network smoothness constraints Mihaela Rosca T. Weber A. Gretton S. Mohamed AAML 25 48 0 14 Dec 2020
Federated Composite Optimization Honglin Yuan Manzil Zaheer Sashank J. Reddi FedML 29 57 0 17 Nov 2020
A Bayesian Perspective on Training Speed and Model Selection Clare Lyle Lisa Schut Binxin Ru Y. Gal Mark van der Wilk 36 23 0 27 Oct 2020
Deep generative demixing: Recovering Lipschitz signals from noisy subgaussian mixtures Aaron Berk 14 0 0 13 Oct 2020
GraphNorm: A Principled Approach to Accelerating Graph Neural Network Training Tianle Cai Shengjie Luo Keyulu Xu Di He Tie-Yan Liu Liwei Wang GNN 21 158 0 07 Sep 2020
Hybrid Differentially Private Federated Learning on Vertically Partitioned Data Chang Wang Jian Liang Mingkai Huang Bing Bai Kun Bai Hao Li FedML 15 39 0 06 Sep 2020
Stochastic Hamiltonian Gradient Methods for Smooth Games Nicolas Loizou Hugo Berard Alexia Jolicoeur-Martineau Pascal Vincent Simon Lacoste-Julien Ioannis Mitliagkas 31 50 0 08 Jul 2020
SGD for Structured Nonconvex Functions: Learning Rates, Minibatching and Interpolation Robert Mansel Gower Othmane Sebbouh Nicolas Loizou 25 74 0 18 Jun 2020
Shape Matters: Understanding the Implicit Bias of the Noise Covariance Jeff Z. HaoChen Colin Wei J. Lee Tengyu Ma 29 93 0 15 Jun 2020
Stability of Stochastic Gradient Descent on Nonsmooth Convex Losses Raef Bassily Vitaly Feldman Cristóbal Guzmán Kunal Talwar MLT 6 192 0 12 Jun 2020
Speedy Performance Estimation for Neural Architecture Search Binxin Ru Clare Lyle Lisa Schut M. Fil Mark van der Wilk Y. Gal 18 35 0 08 Jun 2020
Private Stochastic Convex Optimization: Optimal Rates in Linear Time Vitaly Feldman Tomer Koren Kunal Talwar 6 203 0 10 May 2020
Detached Error Feedback for Distributed SGD with Random Sparsification An Xu Heng-Chiao Huang 36 9 0 11 Apr 2020
Understanding Self-Training for Gradual Domain Adaptation Ananya Kumar Tengyu Ma Percy Liang CLL TTA 28 226 0 26 Feb 2020
Coherent Gradients: An Approach to Understanding Generalization in Gradient Descent-based Optimization S. Chatterjee ODL OOD 11 48 0 25 Feb 2020
Stochastic Polyak Step-size for SGD: An Adaptive Learning Rate for Fast Convergence Nicolas Loizou Sharan Vaswani I. Laradji Simon Lacoste-Julien 27 181 0 24 Feb 2020
Performative Prediction Juan C. Perdomo Tijana Zrnic Celestine Mendler-Dünner Moritz Hardt 20 307 0 16 Feb 2020
Statistical Learning with Conditional Value at Risk Tasuku Soma Yuichi Yoshida 8 38 0 14 Feb 2020
A Diffusion Theory For Deep Learning Dynamics: Stochastic Gradient Descent Exponentially Favors Flat Minima Zeke Xie Issei Sato Masashi Sugiyama ODL 20 17 0 10 Feb 2020
On the distance between two neural networks and the stability of learning Jeremy Bernstein Arash Vahdat Yisong Yue Ming-Yu Liu ODL 197 57 0 09 Feb 2020
Reasoning About Generalization via Conditional Mutual Information Thomas Steinke Lydia Zakynthinou 15 159 0 24 Jan 2020
Layer-Dependent Importance Sampling for Training Deep and Large Graph Convolutional Networks Difan Zou Ziniu Hu Yewen Wang Song Jiang Yizhou Sun Quanquan Gu GNN 23 277 0 17 Nov 2019
Information-Theoretic Generalization Bounds for SGLD via Data-Dependent Estimates Jeffrey Negrea Mahdi Haghifam Gintare Karolina Dziugaite Ashish Khisti Daniel M. Roy FedML 110 146 0 06 Nov 2019
Sharper bounds for uniformly stable algorithms Olivier Bousquet Yegor Klochkov Nikita Zhivotovskiy 16 120 0 17 Oct 2019
Improved Sample Complexities for Deep Networks and Robust Classification via an All-Layer Margin Colin Wei Tengyu Ma AAML OOD 36 85 0 09 Oct 2019
Partial differential equation regularization for supervised machine learning Jillian R. Fisher 24 2 0 03 Oct 2019
Private Stochastic Convex Optimization with Optimal Rates Raef Bassily Vitaly Feldman Kunal Talwar Abhradeep Thakurta 8 236 0 27 Aug 2019
Mix and Match: An Optimistic Tree-Search Approach for Learning Models from Mixture Distributions Matthew Faw Rajat Sen Karthikeyan Shanmugam C. Caramanis Sanjay Shakkottai 28 3 0 23 Jul 2019
Does Learning Require Memorization? A Short Tale about a Long Tail Vitaly Feldman TDI 21 481 0 12 Jun 2019
Importance Resampling for Off-policy Prediction M. Schlegel Wesley Chung Daniel Graves Jian Qian Martha White OffRL 6 41 0 11 Jun 2019
Gram-Gauss-Newton Method: Learning Overparameterized Neural Networks for Regression Problems Tianle Cai Ruiqi Gao Jikai Hou Siyu Chen Dong Wang Di He Zhihua Zhang Liwei Wang ODL 21 57 0 28 May 2019
Orthogonal Deep Neural Networks K. Jia Shuai Li Yuxin Wen Tongliang Liu Dacheng Tao 28 131 0 15 May 2019