ResearchTrend.AI
In Search of Adam's Secret Sauce

In Search of Adam's Secret Sauce

Antonio Orvieto
Robert Gower
ArXiv (abs)PDFHTML

Papers citing "In Search of Adam's Secret Sauce"

2 / 2 papers shown
Title
Is your batch size the problem? Revisiting the Adam-SGD gap in language modeling
Is your batch size the problem? Revisiting the Adam-SGD gap in language modeling
Teodora Srećković
Jonas Geiping
Antonio Orvieto
MoE
32
0
0
14 Jun 2025
How Does Critical Batch Size Scale in Pre-training?
How Does Critical Batch Size Scale in Pre-training?
Hanlin Zhang
Depen Morwani
Nikhil Vyas
Jingfeng Wu
Difan Zou
Udaya Ghai
Dean Phillips Foster
Sham Kakade
191
18
0
29 Oct 2024