Exponential Moving Average of Weights in Deep Learning: Dynamics and Benefits

27 November 2024

Papers citing "Exponential Moving Average of Weights in Deep Learning: Dynamics and Benefits"

7 / 7 papers shown

Title
FOCUS: First Order Concentrated Updating Scheme Yizhou Liu Ziming Liu Jeff Gore ODL 99 0 0 21 Jan 2025
Align and Distill: Unifying and Improving Domain Adaptive Object Detection Justin Kay T. Haucke Suzanne Stathatos Siqi Deng Erik Young Pietro Perona Sara Beery Grant Van Horn 34 3 0 18 Mar 2024
Emerging Properties in Self-Supervised Vision Transformers Mathilde Caron Hugo Touvron Ishan Misra Hervé Jégou Julien Mairal Piotr Bojanowski Armand Joulin 283 4,299 0 29 Apr 2021
SWAD: Domain Generalization by Seeking Flat Minima Junbum Cha Sanghyuk Chun Kyungjae Lee Han-Cheol Cho Seunghyun Park Yunsung Lee Sungrae Park MoMe 213 338 0 17 Feb 2021
There Are Many Consistent Explanations of Unlabeled Data: Why You Should Average Ben Athiwaratkun Marc Finzi Pavel Izmailov A. Wilson 178 232 0 14 Jun 2018
Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles Balaji Lakshminarayanan Alexander Pritzel Charles Blundell UQCV BDL 268 4,940 0 05 Dec 2016
On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima N. Keskar Dheevatsa Mudigere J. Nocedal M. Smelyanskiy P. T. P. Tang ODL 273 2,696 0 15 Sep 2016