Explaining grokking through circuit efficiency

5 September 2023

Papers citing "Explaining grokking through circuit efficiency"

30 / 30 papers shown

Title
When Data Falls Short: Grokking Below the Critical Threshold Vaibhav Singh Eugene Belilovsky Rahaf Aljundi 48 0 0 06 Nov 2025
Egalitarian Gradient Descent: A Simple Approach to Accelerated Grokking Ali Saheb Pasand Elvis Dohmatob 56 0 0 06 Oct 2025
Explaining Grokking and Information Bottleneck through Neural Collapse Emergence Keitaro Sakamoto Issei Sato 108 0 0 25 Sep 2025
Predator-Prey Model: Driven Hunt for Accelerated Grokking I. A. Lopatin S. V. Kozyrev A. N. Pechen 32 1 0 10 Sep 2025
Learning words in groups: fusion algebras, tensor ranks and grokking Maor Shutman Oren Louidor Ran Tessler 84 1 0 08 Sep 2025
What Can Grokking Teach Us About Learning Under Nonstationarity? Clare Lyle Gharda Sokar Razvan Pascanu András Gyorgy 76 2 0 26 Jul 2025
Mechanistic Indicators of Understanding in Large Language Models Pierre Beckmann Matthieu Queloz 155 1 0 07 Jul 2025
ExPLAIND: Unifying Model, Data, and Training Attribution to Study Model Behavior Florian Eichin Yupei Du Philipp Mondorf Maria Matveev Barbara Plank Michael A. Hedderich FAtt 346 0 0 26 May 2025
On the creation of narrow AI: hierarchy and nonlocality of neural network skills Eric J. Michaud Asher Parker-Sartori Max Tegmark 342 2 0 21 May 2025
Questioning Representational Optimism in Deep Learning: The Fractured Entangled Representation Hypothesis Akarsh Kumar Jeff Clune Joel Lehman Kenneth O. Stanley OOD 231 8 0 16 May 2025
Quiet Feature Learning in Algorithmic Tasks Prudhviraj Naidu Zixian Wang Leon Bergen R. Paturi VLM 270 0 0 06 May 2025
A Mathematical Philosophy of Explanations in Mechanistic Interpretability -- The Strange Science Part I.i Kola Ayonrinde Louis Jaburi MILM 430 3 0 01 May 2025
Let Me Grok for You: Accelerating Grokking via Embedding Transfer from a Weaker ModelInternational Conference on Learning Representations (ICLR), 2025 Zhiwei Xu Zhiyu Ni Yixin Wang Wei Hu CLL 278 3 0 17 Apr 2025
Low Rank and Sparse Fourier Structure in Recurrent Networks Trained on Modular AdditionIEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2025 Akshay Rangamani 185 0 0 28 Mar 2025
All Roads Lead to Likelihood: The Value of Reinforcement Learning in Fine-Tuning Gokul Swamy Sanjiban Choudhury Wen Sun Zhiwei Steven Wu J. Andrew Bagnell OffRL 331 41 0 03 Mar 2025
Emergent Misalignment: Narrow finetuning can produce broadly misaligned LLMs Jan Betley Daniel Tan Niels Warncke Anna Sztyber-Betley Xuchan Bao Martín Soto Nathan Labenz Owain Evans AAML 577 97 0 24 Feb 2025
Grokking at the Edge of Numerical StabilityInternational Conference on Learning Representations (ICLR), 2025 Lucas Prieto Melih Barsbey Pedro A.M. Mediano Tolga Birdal 321 16 0 08 Jan 2025
Arithmetic Without Algorithms: Language Models Solve Math With a Bag of HeuristicsInternational Conference on Learning Representations (ICLR), 2024 Yaniv Nikankin Anja Reusch Aaron Mueller Yonatan Belinkov AIFin LRM 291 57 0 28 Oct 2024
Composing Global Solutions to Reasoning Tasks via Algebraic Objects in Neural Nets Yuandong Tian 326 4 0 02 Oct 2024
Emergence in non-neural models: grokking modular arithmetic via average gradient outer product Neil Rohit Mallinar Daniel Beaglehole Libin Zhu Adityanarayanan Radhakrishnan Parthe Pandit Misha Belkin 287 14 0 29 Jul 2024
A Practical Review of Mechanistic Interpretability for Transformer-Based Language Models Daking Rai Yilun Zhou Shi Feng Abulhair Saparov Ziyu Yao 509 77 0 02 Jul 2024
A rationale from frequency perspective for grokking in training neural network Zhangchen Zhou Yaoyu Zhang Z. Xu 219 2 0 24 May 2024
σ-GPTs: A New Approach to Autoregressive Models Arnaud Pannatier Evann Courdier Franccois Fleuret AI4TS 256 18 0 15 Apr 2024
Unified View of Grokking, Double Descent and Emergent Abilities: A Perspective from Circuits Competition Yufei Huang Shengding Hu Xu Han Zhiyuan Liu Maosong Sun 154 21 0 23 Feb 2024
Critical Data Size of Language Models from a Grokking Perspective Xuekai Zhu Yao Fu Bowen Zhou Zhouhan Lin 201 23 0 19 Jan 2024
Grokking as the Transition from Lazy to Rich Training DynamicsInternational Conference on Learning Representations (ICLR), 2023 Tanishq Kumar Blake Bordelon Samuel Gershman Cengiz Pehlevan 294 59 0 09 Oct 2023
Predicting Emergent Abilities with Infinite Resolution EvaluationInternational Conference on Learning Representations (ICLR), 2023 Shengding Hu Xin Liu Xu Han Xinrong Zhang Chaoqun He ... Ning Ding Zebin Ou Guoyang Zeng Zhiyuan Liu Maosong Sun ELM LRM 219 25 0 05 Oct 2023
Benign Overfitting and Grokking in ReLU Networks for XOR Cluster Data Zhiwei Xu Yutong Wang Spencer Frei Gal Vardi Wei Hu MLT 198 34 0 04 Oct 2023
Towards Best Practices of Activation Patching in Language Models: Metrics and MethodsInternational Conference on Learning Representations (ICLR), 2023 Fred Zhang Neel Nanda LLMSV 436 163 0 27 Sep 2023
Faith and Fate: Limits of Transformers on CompositionalityNeural Information Processing Systems (NeurIPS), 2023 Nouha Dziri Ximing Lu Melanie Sclar Xiang Lorraine Li Liwei Jian ... Sean Welleck Xiang Ren Allyson Ettinger Zaïd Harchaoui Yejin Choi ReLM LRM 398 477 0 29 May 2023