What needs to go right for an induction head? A mechanistic study of in-context learning circuits and their formation

10 April 2024

Papers citing "What needs to go right for an induction head? A mechanistic study of in-context learning circuits and their formation"

8 / 8 papers shown

Title
Strategy Coopetition Explains the Emergence and Transience of In-Context Learning Aaditya K. Singh Ted Moskovitz Sara Dragutinovic Felix Hill Stephanie C. Y. Chan Andrew Saxe 53 0 0 07 Mar 2025
Out-of-distribution generalization via composition: a lens through induction heads in Transformers Jiajun Song Zhuoyan Xu Yiqiao Zhong 60 4 0 31 Dec 2024
Toward Understanding In-context vs. In-weight Learning Bryan Chan Xinyi Chen András Gyorgy Dale Schuurmans 56 3 0 30 Oct 2024
ELICIT: LLM Augmentation via External In-Context Capability Futing Wang Jianhao Yan Yue Zhang Tao Lin 33 0 0 12 Oct 2024
Wrong-of-Thought: An Integrated Reasoning Framework with Multi-Perspective Verification and Wrong Information Yongheng Zhang Qiguang Chen Jingxuan Zhou Peng Wang Jiasheng Si Jin Wang Wenpeng Lu Libo Qin LRM 36 3 0 06 Oct 2024
Representing Rule-based Chatbots with Transformers Dan Friedman Abhishek Panigrahi Danqi Chen 47 1 0 15 Jul 2024
Talking Heads: Understanding Inter-layer Communication in Transformer Language Models Jack Merullo Carsten Eickhoff Ellie Pavlick 38 2 0 13 Jun 2024
Survival of the Fittest Representation: A Case Study with Modular Addition Xiaoman Delores Ding Zifan Carl Guo Eric J. Michaud Ziming Liu Max Tegmark 27 3 0 27 May 2024