Dichotomy of Early and Late Phase Implicit Biases Can Provably Induce Grokking

30 November 2023

Papers citing "Dichotomy of Early and Late Phase Implicit Biases Can Provably Induce Grokking"

5 / 5 papers shown

Title
Out-of-distribution generalization via composition: a lens through induction heads in Transformers Jiajun Song Zhuoyan Xu Yiqiao Zhong 78 4 0 31 Dec 2024
Information-Theoretic Progress Measures reveal Grokking is an Emergent Phase Transition Kenzo Clauw S. Stramaglia Daniele Marinazzo 45 3 0 16 Aug 2024
Grokking phase transitions in learning local rules with gradient descent Bojan Žunkovič E. Ilievski 50 16 0 26 Oct 2022
Omnigrok: Grokking Beyond Algorithmic Data Ziming Liu Eric J. Michaud Max Tegmark 54 76 0 03 Oct 2022
What Happens after SGD Reaches Zero Loss? --A Mathematical Framework Zhiyuan Li Tianhao Wang Sanjeev Arora MLT 83 98 0 13 Oct 2021