Iteration Head: A Mechanistic Study of Chain-of-Thought

Iteration Head: A Mechanistic Study of Chain-of-Thought

4 June 2024

Vivien A. Cabannes

Francois Charton

Papers citing "Iteration Head: A Mechanistic Study of Chain-of-Thought"

10 / 10 papers shown

Title
Towards Understanding Fine-Tuning Mechanisms of LLMs via Circuit Analysis X. Wang Yan Hu Wenyu Du Reynold Cheng Benyou Wang Difan Zou 51 0 0 17 Feb 2025
Lower Bounds for Chain-of-Thought Reasoning in Hard-Attention Transformers Alireza Amiri Xinting Huang Mark Rofin Michael Hahn LRM 90 0 0 04 Feb 2025
Understanding and Mitigating Gender Bias in LLMs via Interpretable Neuron Editing Zeping Yu Sophia Ananiadou KELM 43 1 0 24 Jan 2025
Attention Heads of Large Language Models: A Survey Zifan Zheng Yezhaohui Wang Yuxin Huang Shichao Song Mingchuan Yang Bo Tang Feiyu Xiong Zhiyu Li LRM 48 21 0 05 Sep 2024
Revisiting Feature Prediction for Learning Visual Representations from Video Adrien Bardes Q. Garrido Jean Ponce Xinlei Chen Michael G. Rabbat Yann LeCun Mahmoud Assran Nicolas Ballas MDE VLM 79 70 0 15 Feb 2024
How Many Pretraining Tasks Are Needed for In-Context Learning of Linear Regression? Jingfeng Wu Difan Zou Zixiang Chen Vladimir Braverman Quanquan Gu Peter L. Bartlett 114 48 0 12 Oct 2023
How does GPT-2 compute greater-than?: Interpreting mathematical abilities in a pre-trained language model Michael Hanna Ollie Liu Alexandre Variengien LRM 184 116 0 30 Apr 2023
Dissecting Recall of Factual Associations in Auto-Regressive Language Models Mor Geva Jasmijn Bastings Katja Filippova Amir Globerson KELM 189 260 0 28 Apr 2023
Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small Kevin Wang Alexandre Variengien Arthur Conmy Buck Shlegeris Jacob Steinhardt 210 486 0 01 Nov 2022
In-context Learning and Induction Heads Catherine Olsson Nelson Elhage Neel Nanda Nicholas Joseph Nova Dassarma ... Tom B. Brown Jack Clark Jared Kaplan Sam McCandlish C. Olah 240 453 0 24 Sep 2022