Transformers can optimally learn regression mixture models

Transformers can optimally learn regression mixture models

International Conference on Learning Representations (ICLR), 2023

14 November 2023

ArXiv (abs)PDF HTML

Papers citing "Transformers can optimally learn regression mixture models"

7 / 7 papers shown

Title
Theory of Scaling Laws for In-Context Regression: Depth, Width, Context and Time Blake Bordelon Mary I. Letey Cengiz Pehlevan 88 0 0 01 Oct 2025
Limitations of refinement methods for weak to strong generalization Seamus Somerstep Yaácov Ritov Mikhail Yurochkin Subha Maity Yuekai Sun 76 1 0 23 Aug 2025
On the Robustness of Transformers against Context Hijacking for Linear Classification Tianle Li Chenyang Zhang Xingwu Chen Yuan Cao Difan Zou 269 3 0 24 Feb 2025
On the Training Convergence of Transformers for In-Context Classification of Gaussian Mixtures Wei Shen Ruida Zhou Jing Yang Cong Shen 238 6 0 15 Oct 2024
A Theoretical Understanding of Self-Correction through In-context Alignment Yifei Wang Yuyang Wu Zeming Wei Stefanie Jegelka Yisen Wang LRM 206 50 0 28 May 2024
In-Context Learning of a Linear Transformer Block: Benefits of the MLP Component and One-Step GD Initialization Ruiqi Zhang Jingfeng Wu Peter L. Bartlett 237 25 0 22 Feb 2024
Linear Transformers are Versatile In-Context Learners Max Vladymyrov J. Oswald Mark Sandler Rong Ge 142 27 0 21 Feb 2024