Progress measures for grokking via mechanistic interpretability

Progress measures for grokking via mechanistic interpretability

12 January 2023

Jacob Steinhardt

Papers citing "Progress measures for grokking via mechanistic interpretability"

13 / 63 papers shown

Title
Scaling TabPFN: Sketching and Feature Selection for Tabular Prior-Data Fitted Networks Ben Feuer Chinmay Hegde Niv Cohen 25 10 0 17 Nov 2023
Uncovering Intermediate Variables in Transformers using Circuit Probing Michael A. Lepori Thomas Serre Ellie Pavlick 70 7 0 07 Nov 2023
Towards a Mechanistic Interpretation of Multi-Step Reasoning Capabilities of Language Models Yifan Hou Jiaoda Li Yu Fei Alessandro Stolfo Wangchunshu Zhou Guangtao Zeng Antoine Bosselut Mrinmaya Sachan LRM 30 39 0 23 Oct 2023
Deep Neural Networks Can Learn Generalizable Same-Different Visual Relations Alexa R. Tartaglini Sheridan Feucht Michael A. Lepori Wai Keen Vong Charles Lovering Brenden Lake Ellie Pavlick ViT OOD 17 3 0 14 Oct 2023
Interpretable Diffusion via Information Decomposition Xianghao Kong Ollie Liu Han Li Dani Yogatama Greg Ver Steeg 16 19 0 12 Oct 2023
Grokking as Compression: A Nonlinear Complexity Perspective Ziming Liu Ziqian Zhong Max Tegmark 30 9 0 09 Oct 2023
Towards Best Practices of Activation Patching in Language Models: Metrics and Methods Fred Zhang Neel Nanda LLMSV 26 96 0 27 Sep 2023
Towards Vision-Language Mechanistic Interpretability: A Causal Tracing Tool for BLIP Vedant Palit Rohan Pandey Aryaman Arora Paul Pu Liang 24 20 0 27 Aug 2023
Arithmetic with Language Models: from Memorization to Computation Davide Maltoni Matteo Ferrara KELM LRM 22 4 0 02 Aug 2023
Schema-learning and rebinding as mechanisms of in-context learning and emergence Siva K. Swaminathan Antoine Dedieu Rajkumar Vasudeva Raju Murray Shanahan Miguel Lazaro-Gredilla Dileep George 24 8 0 16 Jun 2023
Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small Kevin Wang Alexandre Variengien Arthur Conmy Buck Shlegeris Jacob Steinhardt 210 491 0 01 Nov 2022
In-context Learning and Induction Heads Catherine Olsson Nelson Elhage Neel Nanda Nicholas Joseph Nova Dassarma ... Tom B. Brown Jack Clark Jared Kaplan Sam McCandlish C. Olah 240 456 0 24 Sep 2022
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Jason W. Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Brian Ichter F. Xia Ed H. Chi Quoc Le Denny Zhou LM&Ro LRM AI4CE ReLM 315 8,402 0 28 Jan 2022