Averaging Weights Leads to Wider Optima and Better Generalization

14 March 2018

Dmitry Vetrov

Papers citing "Averaging Weights Leads to Wider Optima and Better Generalization"

50 / 305 papers shown

Title
Rethinking Rotated Object Detection with Gaussian Wasserstein Distance Loss Xue Yang Junchi Yan Qi Ming Wentao Wang Xiaopeng Zhang Qi Tian 108 399 0 28 Jan 2021
Exponential Moving Average Normalization for Self-supervised and Semi-supervised Learning Zhaowei Cai Avinash Ravichandran Subhransu Maji Charless C. Fowlkes Z. Tu Stefano Soatto 34 118 0 21 Jan 2021
LightXML: Transformer with Dynamic Negative Sampling for High-Performance Extreme Multi-label Text Classification Ting Jiang Deqing Wang Leilei Sun Huayi Yang Zhengyang Zhao Fuzhen Zhuang VLM 115 137 0 09 Jan 2021
FedADC: Accelerated Federated Learning with Drift Control Emre Ozfatura Kerem Ozfatura Deniz Gunduz FedML 27 37 0 16 Dec 2020
DeepLesionBrain: Towards a broader deep-learning generalization for multiple sclerosis lesion segmentation R. A. Kamraoui Vinh-Thong Ta T. Tourdias Boris Mansencal J. V. Manjón Pierrick Coupé OOD 12 50 0 14 Dec 2020
Fine-tuning BERT for Low-Resource Natural Language Understanding via Active Learning Daniel Grießhaber J. Maucher Ngoc Thang Vu 17 46 0 04 Dec 2020
A Random Matrix Theory Approach to Damping in Deep Learning Diego Granziol Nicholas P. Baskerville AI4CE ODL 24 2 0 15 Nov 2020
Underspecification Presents Challenges for Credibility in Modern Machine Learning Alexander DÁmour Katherine A. Heller D. Moldovan Ben Adlam B. Alipanahi ... Kellie Webster Steve Yadlowsky T. Yun Xiaohua Zhai D. Sculley OffRL 48 669 0 06 Nov 2020
Measuring and Harnessing Transference in Multi-Task Learning Christopher Fifty Ehsan Amid Zhe Zhao Tianhe Yu Rohan Anil Chelsea Finn 22 15 0 29 Oct 2020
Combining Ensembles and Data Augmentation can Harm your Calibration Yeming Wen Ghassen Jerfel Rafael Muller Michael W. Dusenberry Jasper Snoek Balaji Lakshminarayanan Dustin Tran UQCV 24 63 0 19 Oct 2020
Regularizing Neural Networks via Adversarial Model Perturbation Yaowei Zheng Richong Zhang Yongyi Mao AAML 22 95 0 10 Oct 2020
Uncovering the Limits of Adversarial Training against Norm-Bounded Adversarial Examples Sven Gowal Chongli Qin J. Uesato Timothy A. Mann Pushmeet Kohli AAML 17 323 0 07 Oct 2020
Sharpness-Aware Minimization for Efficiently Improving Generalization Pierre Foret Ariel Kleiner H. Mobahi Behnam Neyshabur AAML 16 1,276 0 03 Oct 2020
SESQA: semi-supervised learning for speech quality assessment Joan Serra Jordi Pons Santiago Pascual 6 42 0 01 Oct 2020
Conversational Semantic Parsing Armen Aghajanyan Jean Maillard Akshat Shrivastava K. Diedrick Mike Haeger ... Yashar Mehdad Ves Stoyanov Anuj Kumar M. Lewis S. Gupta 11 48 0 28 Sep 2020
Adversarial Training with Stochastic Weight Average Joong-won Hwang Youngwan Lee Sungchan Oh Yuseok Bae OOD AAML 11 11 0 21 Sep 2020
Kaggle forecasting competitions: An overlooked learning opportunity Casper Solheim Bojer Jens Peder Meldgaard AI4TS 11 207 0 16 Sep 2020
Ramifications of Approximate Posterior Inference for Bayesian Deep Learning in Adversarial and Out-of-Distribution Settings John Mitros A. Pakrashi Brian Mac Namee UQCV 11 2 0 03 Sep 2020
Obtaining Adjustable Regularization for Free via Iterate Averaging Jingfeng Wu Vladimir Braverman Lin F. Yang 25 2 0 15 Aug 2020
Directional Pruning of Deep Neural Networks Shih-Kang Chao Zhanyu Wang Yue Xing Guang Cheng ODL 6 33 0 16 Jun 2020
Learning Rates as a Function of Batch Size: A Random Matrix Theory Approach to Neural Network Training Diego Granziol S. Zohren Stephen J. Roberts ODL 27 48 0 16 Jun 2020
Calibrating Deep Neural Network Classifiers on Out-of-Distribution Datasets Zhihui Shao Jianyi Yang Shaolei Ren OODD 27 11 0 16 Jun 2020
Hindsight Logging for Model Training Rolando Garcia Eric Liu Vikram Sreekanti Bobby Yan Anusha Dandamudi Joseph E. Gonzalez J. M. Hellerstein Koushik Sen VLM 14 10 0 12 Jun 2020
A benchmark study on reliable molecular supervised learning via Bayesian learning Doyeong Hwang Grace Lee Hanseok Jo Seyoul Yoon Seongok Ryu 17 9 0 12 Jun 2020
An Overview of Deep Semi-Supervised Learning Yassine Ouali C´eline Hudelot Myriam Tami SSL HAI 19 294 0 09 Jun 2020
Detached Error Feedback for Distributed SGD with Random Sparsification An Xu Heng-Chiao Huang 31 9 0 11 Apr 2020
Bayesian Deep Learning and a Probabilistic Perspective of Generalization A. Wilson Pavel Izmailov UQCV BDL OOD 17 639 0 20 Feb 2020
Pitfalls of In-Domain Uncertainty Estimation and Ensembling in Deep Learning Arsenii Ashukha Alexander Lyzhov Dmitry Molchanov Dmitry Vetrov UQCV FedML 17 314 0 15 Feb 2020
No Routing Needed Between Capsules Adam Byerly T. Kalganova I. Dear 28 67 0 24 Jan 2020
Curriculum Labeling: Revisiting Pseudo-Labeling for Semi-Supervised Learning Paola Cascante-Bonilla Fuwen Tan Yanjun Qi Vicente Ordonez ODL 35 23 0 16 Jan 2020
Stochastic Weight Averaging in Parallel: Large-Batch Training that Generalizes Well Vipul Gupta S. Serrano D. DeCoste MoMe 30 55 0 07 Jan 2020
Big Transfer (BiT): General Visual Representation Learning Alexander Kolesnikov Lucas Beyer Xiaohua Zhai J. Puigcerver Jessica Yung Sylvain Gelly N. Houlsby MQ 38 1,183 0 24 Dec 2019
TRADI: Tracking deep neural network weight distributions for uncertainty estimation Gianni Franchi Andrei Bursuc Emanuel Aldea Séverine Dubuisson Isabelle Bloch UQCV 20 51 0 24 Dec 2019
Deep Ensembles: A Loss Landscape Perspective Stanislav Fort Huiyi Hu Balaji Lakshminarayanan OOD UQCV 13 617 0 05 Dec 2019
Disentangle, align and fuse for multimodal and semi-supervised image segmentation A. Chartsias G. Papanastasiou Chengjia Wang S. Semple D. Newby R. Dharmakumar Sotirios A. Tsaftaris 14 13 0 11 Nov 2019
Segmenting Ships in Satellite Imagery With Squeeze and Excitation U-Net R. Venkatesh Anand Metha SSeg 11 4 0 27 Oct 2019
Self-Correction for Human Parsing Peike Li Yunqiu Xu Yunchao Wei Yezhou Yang 11 324 0 22 Oct 2019
BoTorch: A Framework for Efficient Monte-Carlo Bayesian Optimization Maximilian Balandat Brian Karrer Daniel R. Jiang Sam Daulton Benjamin Letham A. Wilson E. Bakshy 11 93 0 14 Oct 2019
Towards Understanding the Transferability of Deep Representations Hong Liu Mingsheng Long Jianmin Wang Michael I. Jordan 14 25 0 26 Sep 2019
On Model Stability as a Function of Random Seed Pranava Madhyastha Dhruv Batra 26 61 0 23 Sep 2019
Visualizing and Understanding the Effectiveness of BERT Y. Hao Li Dong Furu Wei Ke Xu 22 181 0 15 Aug 2019
Lookahead Optimizer: k steps forward, 1 step back Michael Ruogu Zhang James Lucas Geoffrey E. Hinton Jimmy Ba ODL 11 717 0 19 Jul 2019
Subspace Inference for Bayesian Deep Learning Pavel Izmailov Wesley J. Maddox Polina Kirichenko T. Garipov Dmitry Vetrov A. Wilson UQCV BDL 17 141 0 17 Jul 2019
AssemblyNet: A Novel Deep Decision-Making Process for Whole Brain MRI Segmentation Pierrick Coupé Boris Mansencal Michael Clement Rémi Giraud B. D. D. Senneville T. Thong Vincent Lepetit J. V. Manjón 9 17 0 05 Jun 2019
Modeling Uncertainty by Learning a Hierarchy of Deep Neural Connections R. Y. Rohekar Yaniv Gurwicz Shami Nisimov Gal Novik BDL UQCV 16 13 0 30 May 2019
Ensemble Model Patching: A Parameter-Efficient Variational Bayesian Neural Network Oscar Chang Yuling Yao David Williams-King Hod Lipson BDL UQCV 26 8 0 23 May 2019
A neural network-based framework for financial model calibration Shuaiqiang Liu Anastasia Borovykh L. Grzelak C. Oosterlee 22 103 0 23 Apr 2019
$UG$^{2+}$ Track 2: A Collective Benchmark Effort for Evaluating and Advancing Image Understanding in Poor Visibility Environments$ UG $^{2+}$ Track 2: A Collective Benchmark Effort for Evaluating and Advancing Image Understanding in Poor Visibility Environments Ye Yuan Wenhan Yang Wenqi Ren Xin Liu Cheng Chi Haiquan Wang 3DV 37 227 0 09 Apr 2019
DeepCMB: Lensing Reconstruction of the Cosmic Microwave Background with Deep Neural Networks J. Caldeira W. L. K. Wu Brian D. Nord Camille Avestruz Shubhendu Trivedi K. Story 6 63 0 02 Oct 2018
Non-local NetVLAD Encoding for Video Classification Yongyi Tang Xing Zhang Jingwen Wang Shaoxiang Chen Lin Ma Yu-Gang Jiang 11 41 0 29 Sep 2018