v1v2 (latest)

On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima

15 September 2016

Papers citing "On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima"

50 / 1,653 papers shown

Title
DIVEBATCH: Accelerating Model Training Through Gradient-Diversity Aware Batch Size Adaptation Yuen Chen Yian Wang Hari Sundaram 84 0 0 19 Sep 2025
Pre-training under infinite compute Konwoo Kim Suhas Kotha Abigail Z. Jacobs Tatsunori Hashimoto 204 1 0 18 Sep 2025
OASIS: A Deep Learning Framework for Universal Spectroscopic Analysis Driven by Novel Loss Functions Chris Young Juejing Liu Marie L. Mortensen Yifu Feng Elizabeth Li Zheming Wang Xiaofeng Guo K. Rosso Xin Zhang 21 0 0 15 Sep 2025
On the Escaping Efficiency of Distributed Adversarial Training Algorithms Ying Cao Kun Yuan Ali H. Sayed AAML 93 0 0 14 Sep 2025
How Far Are We from True Unlearnability?International Conference on Learning Representations (ICLR), 2025 Kai Ye Liangcai Su Chenxiong Qian 134 4 0 09 Sep 2025
Explaining How Quantization Disparately Skews a Model Abhimanyu Bellam Jung-Eun Kim MQ 128 0 0 08 Sep 2025
On Using Large-Batches in Federated Learning Sahil Tyagi FedML 90 0 0 05 Sep 2025
CoT-Space: A Theoretical Framework for Internal Slow-Thinking via Reinforcement Learning Zeyu Gan Hao Yi Yong Liu OffRL LRM 164 1 0 04 Sep 2025
Enhancing Gradient Variance and Differential Privacy in Quantum Federated Learning Duc-Thien Phan Minh-Duong Nguyen Quoc-Viet Pham Huilong Pi FedML 76 1 0 04 Sep 2025
LSAM: Asynchronous Distributed Training with Landscape-Smoothed Sharpness-Aware Minimization Yunfei Teng Sixin Zhang 117 0 0 03 Sep 2025
VASSO: Variance Suppression for Sharpness-Aware Minimization Bingcong Li Yilang Zhang G. Giannakis 232 1 0 02 Sep 2025
Adaptive Heavy-Tailed Stochastic Gradient Descent Bodu Gong Gustavo Enrique Batista Pierre Lafaye de Micheaux 112 0 0 29 Aug 2025
MERIT: Maximum-normalized Element-wise Ratio for Language Model Large-batch Training Yang Luo Zangwei Zheng Ziheng Qin Zirui Zhu Yong Liu Yang You ALM 80 0 0 28 Aug 2025
Flatness-aware Curriculum Learning via Adversarial Difficulty Hiroaki Aizawa Yoshikazu Hayashi ODL 212 0 0 26 Aug 2025
C-Flat++: Towards a More Efficient and Powerful Framework for Continual Learning Wei Li Hangjie Yuan Zixiang Zhao Yifan Zhu Aojun Lu Tao Feng Yanan Sun 156 1 0 26 Aug 2025
Optimal Sparsity of Mixture-of-Experts Language Models for Reasoning Tasks Taishi Nakamura Satoki Ishikawa Masaki Kawamura Takumi Okamoto Daisuke Nohara Jun Suzuki Rio Yokota MoE LRM 131 0 0 26 Aug 2025
Algebraic Approach to Ridge-Regularized Mean Squared Error Minimization in Minimal ReLU Neural Network Ryoya Fukasaku Y. Kabata Akifumi Okuno 88 0 0 25 Aug 2025
Convergence and Generalization of Anti-Regularization for Parametric Models Dongseok Kim Wonjun Jeong Gisung Oh 205 0 0 24 Aug 2025
The Lifecycle Principle: Stabilizing Dynamic Neural Networks with State Memory Zichuan Yang 85 0 0 24 Aug 2025
Balanced Sharpness-Aware Minimization for Imbalanced Regression Yahao Liu Qin Wang Lixin Duan Wen Li 113 1 0 23 Aug 2025
WISCA: A Lightweight Model Transition Method to Improve LLM Training via Weight Scaling Jiacheng Li Jianchao Tan Zhidong Yang Pingwei Sun Feiye Huo ... Xiangyu Zhang Maoxin He Guangming Tan Weile Jia Tong Zhao 88 3 0 21 Aug 2025
Twin-Boot: Uncertainty-Aware Optimization via Online Two-Sample Bootstrapping Carlos Stein Brito UQCV 100 0 0 20 Aug 2025
Inter-Class Relational Loss for Small Object Detection: A Case Study on License Plates Dian Ning Dong Seog Han 92 0 0 20 Aug 2025
Wormhole Dynamics in Deep Neural NetworksIEEE Transactions on Neural Networks and Learning Systems (IEEE TNNLS), 2025 Yen-Lung Lai Zhe Jin AI4CE 128 1 0 20 Aug 2025
FLAIR: Frequency- and Locality-Aware Implicit Neural Representations Sukhun Ko Dahyeon Kye Kyle Min Chanho Eom Jihyong Oh Jihyong Oh 159 0 0 19 Aug 2025
Fisher-Orthogonal Projection Methods for Natural Gradient Descent with Large Batches Yishun Lu Wesley Armour ODL 329 1 0 19 Aug 2025
Optimal Condition for Initialization Variance in Deep Neural Networks: An SGD Dynamics Perspective Hiroshi Horii Sothea Has 80 0 0 18 Aug 2025
Training Machine Learning Models on Human Spatio-temporal Mobility Data: An Experimental Study [Experiment Paper] Yueyang Liu Lance Kennedy Ruochen Kong Joon-Seok Kim Andreas Züfle 80 0 0 18 Aug 2025
Multi-level Collaborative Distillation Meets Global Workspace Model: A Unified Framework for OCIL Shibin Su Guoqiang Liang De Cheng Shizhou Zhang Lingyan Ran Yanning Zhang CLL 124 0 0 12 Aug 2025
Statistical Theory of Multi-stage Newton Iteration Algorithm for Online Continual Learning Xinjia Lu Chuhan Wang Qian Zhao Lixing Zhu Xuehu Zhu 100 0 0 10 Aug 2025
Tractable Sharpness-Aware Learning of Probabilistic Circuits Hrithik Suresh Sahil Sidheekh Vishnu Shreeram M.P S. Natarajan N. C. Krishnan TPM 156 0 0 07 Aug 2025
Sensitivity of Stability: Theoretical & Empirical Analysis of Replicability for Adaptive Data Selection in Transfer Learning Prabhav Singh Jessica Sorrell 108 0 0 06 Aug 2025
Superior resilience to poisoning and amenability to unlearning in quantum machine learning Yu-Qin Chen Shi-Xin Zhang AAML 152 3 0 04 Aug 2025
Efficiently Seeking Flat Minima for Better Generalization in Fine-Tuning Large Language Models and Beyond Jiaxin Deng Qingcheng Zhu Junbiao Pang Linlin Yang Zhongqian Fu Baochang Zhang 125 0 0 01 Aug 2025
Communication-Efficient Distributed Training for Collaborative Flat Optima Recovery in Deep Learning Tolga Dimlioglu A. Choromańska FedML 238 1 0 27 Jul 2025
Irredundant $k$ -Fold Cross-Validation Jesus S. Aguilar-Ruiz 112 0 0 26 Jul 2025
The Price equation reveals a universal force-metric-bias law of algorithmic learning and natural selection Steven A. Frank FedML 320 0 0 24 Jul 2025
Large Learning Rates Simultaneously Achieve Robustness to Spurious Correlations and Compressibility Melih Barsbey Lucas Prieto Stefanos Zafeiriou Tolga Birdal 256 0 0 23 Jul 2025
Small Batch Size Training for Language Models: When Vanilla SGD Works, and Why Gradient Accumulation Is Wasteful Martin Marek Sanae Lotfi Aditya Somasundaram A. Wilson Micah Goldblum LRM 352 11 0 09 Jul 2025
DGSAM: Domain Generalization via Individual Sharpness-Aware Minimization Youngjun Song Youngsik Hwang Jonghun Lee Heechang Lee Dong-Young Lim AAML 267 0 0 01 Jul 2025
Both Asymptotic and Non-Asymptotic Convergence of Quasi-Hyperbolic Momentum using Increasing Batch Size Kento Imaizumi Hideaki Iiduka 180 0 0 30 Jun 2025
Optimization-Induced Dynamics of Lipschitz Continuity in Neural Networks Róisín Luo James McDermott Christian Gagné Qiang Sun C. O'Riordan 138 0 0 23 Jun 2025
Thermometry of simulated Bose--Einstein condensates using machine learning Jack Griffiths Steven A. Wrathmall Simon A. Gardiner 113 0 0 20 Jun 2025
The Butterfly Effect: Neural Network Training Trajectories Are Highly Sensitive to Initial Conditions Devin Kwok Gül Sena Altıntaş Colin Raffel David Rolnick 335 2 0 16 Jun 2025
From Sharpness to Better Generalization for Speech Deepfake Detection Wen-Chin Huang Xuechen Liu Xin Eric Wang Junichi Yamagishi Yanmin Qian 145 3 0 13 Jun 2025
Generalization Bound of Gradient Flow through Training Trajectory and Data-dependent Kernel Yilan Chen Zhichao Wang Wei Huang Andi Han Taiji Suzuki Arya Mazumdar MLT 198 0 0 12 Jun 2025
FEDTAIL: Federated Long-Tailed Domain Generalization with Sharpness-Guided Gradient Matching Sunny Gupta Nikita Jangid Shounak Das Amit Sethi FedML 219 0 0 10 Jun 2025
Promoting Ensemble Diversity with Interactive Bayesian Distributional Robustness for Fine-tuning Foundation Models Ngoc-Quan Pham Tuan Truong Quyen Tran T. H. Nguyen Dinh Q. Phung T. Le 211 3 0 08 Jun 2025
SAFE: Finding Sparse and Flat Minima to Improve Pruning Dongyeop Lee Kwanhee Lee Jinseok Chung Namhoon Lee 281 4 0 07 Jun 2025
Towards Better Generalization via Distributional Input Projection Network Yifan Hao Yanxin Lu Xinwei Shen Tong Zhang Tong Zhang 246 0 0 05 Jun 2025

All Papers

On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima

Papers citing "On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima"