A distributional simplicity bias in the learning dynamics of transformers

17 February 2025

Papers citing "A distributional simplicity bias in the learning dynamics of transformers"

5 / 5 papers shown

Title
A Mathematical Philosophy of Explanations in Mechanistic Interpretability -- The Strange Science Part I.i Kola Ayonrinde Louis Jaburi MILM 68 1 0 01 May 2025
Shaping Shared Languages: Human and Large Language Models' Inductive Biases in Emergent Communication Tom Kouwenhoven Max Peeperkorn R. D. Kleijn Tessa Verhoef 55 0 0 06 Mar 2025
Training Dynamics of In-Context Learning in Linear Attention Yedi Zhang Aaditya K. Singh Peter E. Latham Andrew Saxe MLT 51 1 0 28 Jan 2025
How transformers learn structured data: insights from hierarchical filtering Jerome Garnier-Brun Marc Mézard Emanuele Moscato Luca Saglietti 18 2 0 27 Aug 2024
Towards a theory of how the structure of language is acquired by deep neural networks Francesco Cagnetta M. Wyart 23 8 0 28 May 2024