A Mechanistic Analysis of a Transformer Trained on a Symbolic Multi-Step
Reasoning Task

A Mechanistic Analysis of a Transformer Trained on a Symbolic Multi-Step Reasoning Task

19 February 2024

Jannik Brinkmann

Abhay Sheshadri

Christian Bartelt

Papers citing "A Mechanistic Analysis of a Transformer Trained on a Symbolic Multi-Step Reasoning Task"

10 / 10 papers shown

Title
Transformers Use Causal World Models in Maze-Solving Tasks Alex F Spies William Edwards Michael I. Ivanitskiy Adrians Skapars Tilman Rauker Katsumi Inoue A. Russo Murray Shanahan 87 1 0 16 Dec 2024
Interacting Large Language Model Agents. Interpretable Models and Social Learning Adit Jain Vikram Krishnamurthy LLMAG 25 0 0 02 Nov 2024
A Practical Review of Mechanistic Interpretability for Transformer-Based Language Models Daking Rai Yilun Zhou Shi Feng Abulhair Saparov Ziyu Yao 54 18 0 02 Jul 2024
Look Before You Leap: A Universal Emergent Decomposition of Retrieval Tasks in Language Models Alexandre Variengien Eric Winsor LRM ReLM 72 5 0 13 Dec 2023
Structured World Representations in Maze-Solving Transformers Michael I. Ivanitskiy Alex F Spies Tilman Rauker Guillaume Corlouer Chris Mathwin ... Rusheb Shah Dan Valentine Cecilia G. Diniz Behn Katsumi Inoue Samy Wu Fung 47 2 0 05 Dec 2023
The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets Samuel Marks Max Tegmark HILM 91 164 0 10 Oct 2023
Dissecting Recall of Factual Associations in Auto-Regressive Language Models Mor Geva Jasmijn Bastings Katja Filippova Amir Globerson KELM 186 260 0 28 Apr 2023
Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small Kevin Wang Alexandre Variengien Arthur Conmy Buck Shlegeris Jacob Steinhardt 207 486 0 01 Nov 2022
Language Models Are Greedy Reasoners: A Systematic Formal Analysis of Chain-of-Thought Abulhair Saparov He He ELM LRM ReLM 116 270 0 03 Oct 2022
Large Language Models are Zero-Shot Reasoners Takeshi Kojima S. Gu Machel Reid Yutaka Matsuo Yusuke Iwasawa ReLM LRM 291 2,712 0 24 May 2022