Anticorrelated Noise Injection for Improved Generalization

6 February 2022

Antonio Orvieto

Papers citing "Anticorrelated Noise Injection for Improved Generalization"

33 / 33 papers shown

Title
SASSHA: Sharpness-aware Adaptive Second-order Optimization with Stable Hessian Approximation Dahun Shin Dongyeop Lee Jinseok Chung Namhoon Lee ODL AAML 87 0 0 25 Feb 2025
Stochastic Gradient Descent Jittering for Inverse Problems: Alleviating the Accuracy-Robustness Tradeoff Peimeng Guan Mark A. Davenport 9 0 0 18 Oct 2024
An Adaptive Stochastic Gradient Method with Non-negative Gauss-Newton Stepsizes Antonio Orvieto Lin Xiao 30 2 0 05 Jul 2024
Why is SAM Robust to Label Noise? Christina Baek Zico Kolter Aditi Raghunathan NoLa AAML 18 9 0 06 May 2024
Singular-limit analysis of gradient descent with noise injection Anna Shalova André Schlichting M. Peletier 25 1 0 18 Apr 2024
Communication-Efficient Distributed Learning with Local Immediate Error Compensation Yifei Cheng Li Shen Linli Xu Xun Qian Shiwei Wu Yiming Zhou Tie Zhang Dacheng Tao Enhong Chen 16 0 0 19 Feb 2024
Critical Influence of Overparameterization on Sharpness-aware Minimization Sungbin Shin Dongyeop Lee Maksym Andriushchenko Namhoon Lee AAML 28 1 0 29 Nov 2023
Evolutionary algorithms as an alternative to backpropagation for supervised training of Biophysical Neural Networks and Neural ODEs James Hazelden Yuhan Helena Liu Eli Shlizerman E. Shea-Brown 23 2 0 17 Nov 2023
PAC-tuning:Fine-tuning Pretrained Language Models with PAC-driven Perturbed Gradient Descent Guang-Da Liu Zhiyu Xue Xitong Zhang K. Johnson Rongrong Wang 10 5 0 26 Oct 2023
Data Optimization in Deep Learning: A Survey Ou Wu Rujing Yao 15 1 0 25 Oct 2023
Enhancing Deep Neural Network Training Efficiency and Performance through Linear Prediction Hejie Ying Mengmeng Song Yaohong Tang S. Xiao Zimin Xiao 11 8 0 17 Oct 2023
Correlated Noise Provably Beats Independent Noise for Differentially Private Learning Christopher A. Choquette-Choo Krishnamurthy Dvijotham Krishna Pillutla Arun Ganesh Thomas Steinke Abhradeep Thakurta 10 13 0 10 Oct 2023
On the curvature of the loss landscape Alison Pouplin Hrittik Roy Sidak Pal Singh Georgios Arvanitidis 16 0 0 10 Jul 2023
Edge Intelligence Over the Air: Two Faces of Interference in Federated Learning Zihan Chen Howard H. Yang Tony Q. S. Quek 27 9 0 17 Jun 2023
The Split Matters: Flat Minima Methods for Improving the Performance of GNNs N. Lell A. Scherp 20 1 0 15 Jun 2023
Decentralized SGD and Average-direction SAM are Asymptotically Equivalent Tongtian Zhu Fengxiang He Kaixuan Chen Mingli Song Dacheng Tao 24 15 0 05 Jun 2023
Evolutionary Algorithms in the Light of SGD: Limit Equivalence, Minima Flatness, and Transfer Learning Andrei Kucharavy R. Guerraoui Ljiljana Dolamic 20 1 0 20 May 2023
An Adaptive Policy to Employ Sharpness-Aware Minimization Weisen Jiang Hansi Yang Yu Zhang James T. Kwok AAML 69 31 0 28 Apr 2023
ELF: Federated Langevin Algorithms with Primal, Dual and Bidirectional Compression Avetik G. Karagulyan Peter Richtárik FedML 11 4 0 08 Mar 2023
The Lie-Group Bayesian Learning Rule E. M. Kıral Thomas Möllenhoff Mohammad Emtiyaz Khan BDL 6 2 0 08 Mar 2023
Utility-based Perturbed Gradient Descent: An Optimizer for Continual Learning Mohamed Elsayed A. R. Mahmood CLL 11 5 0 07 Feb 2023
Gradient Descent with Linearly Correlated Noise: Theory and Applications to Differential Privacy Anastasia Koloskova Ryan McKenna Zachary B. Charles Keith Rush Brendan McMahan 17 8 0 02 Feb 2023
Implicit regularization in Heavy-ball momentum accelerated stochastic gradient descent Avrajit Ghosh He Lyu Xitong Zhang Rongrong Wang 21 20 0 02 Feb 2023
A Study on FGSM Adversarial Training for Neural Retrieval Simon Lupart S. Clinchant AAML 10 7 0 25 Jan 2023
Improving Generalization of Pre-trained Language Models via Stochastic Weight Averaging Peng Lu I. Kobyzev Mehdi Rezagholizadeh Ahmad Rashid A. Ghodsi Philippe Langlais MoMe 17 8 0 12 Dec 2022
Disentangling the Mechanisms Behind Implicit Regularization in SGD Zachary Novack Simran Kaur Tanya Marwah Saurabh Garg Zachary Chase Lipton FedML 12 2 0 29 Nov 2022
EF21-P and Friends: Improved Theoretical Communication Complexity for Distributed Optimization with Bidirectional Compression Kaja Gruntkowska A. Tyurin Peter Richtárik 23 18 0 30 Sep 2022
Explicit Regularization in Overparametrized Models via Noise Injection Antonio Orvieto Anant Raj Hans Kersting Francis R. Bach 8 25 0 09 Jun 2022
Beyond accuracy: generalization properties of bio-plausible temporal credit assignment rules Yuhan Helena Liu Arna Ghosh Blake A. Richards E. Shea-Brown Guillaume Lajoie 6 9 0 02 Jun 2022
Special Properties of Gradient Descent with Large Learning Rates Amirkeivan Mohtashami Martin Jaggi Sebastian U. Stich MLT 11 7 0 30 May 2022
What Happens after SGD Reaches Zero Loss? --A Mathematical Framework Zhiyuan Li Tianhao Wang Sanjeev Arora MLT 69 98 0 13 Oct 2021
Shift-Curvature, SGD, and Generalization Arwen V. Bradley C. Gomez-Uribe Manish Reddy Vuyyuru 12 2 0 21 Aug 2021
On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima N. Keskar Dheevatsa Mudigere J. Nocedal M. Smelyanskiy P. T. P. Tang ODL 273 2,696 0 15 Sep 2016