How Many Pretraining Tasks Are Needed for In-Context Learning of Linear
Regression?

How Many Pretraining Tasks Are Needed for In-Context Learning of Linear Regression?

12 October 2023

Vladimir Braverman

Quanquan Gu

Peter L. Bartlett

Papers citing "How Many Pretraining Tasks Are Needed for In-Context Learning of Linear Regression?"

9 / 9 papers shown

Title
Vector-ICL: In-context Learning with Continuous Vector Representations Yufan Zhuang Chandan Singh Liyuan Liu Jingbo Shang Jianfeng Gao 47 3 0 21 Feb 2025
Toward Understanding In-context vs. In-weight Learning Bryan Chan Xinyi Chen András Gyorgy Dale Schuurmans 56 3 0 30 Oct 2024
Spin glass model of in-context learning Yuhao Li Ruoran Bai Haiping Huang LRM 32 0 0 05 Aug 2024
Towards Better Understanding of In-Context Learning Ability from In-Context Uncertainty Quantification Shang Liu Zhongze Cai Guanting Chen Xiaocheng Li UQCV 20 1 0 24 May 2024
Asymptotic theory of in-context learning by linear attention Yue M. Lu Mary I. Letey Jacob A. Zavatone-Veth Anindita Maiti C. Pehlevan 19 10 0 20 May 2024
Transformers are Provably Optimal In-context Estimators for Wireless Communications Vishnu Teja Kunde Vicram Rajagopalan Chandra Shekhara Kaushik Valmeekam Krishna R. Narayanan S. Shakkottai D. Kalathil J. Chamberland 24 4 0 01 Nov 2023
How Do Transformers Learn Topic Structure: Towards a Mechanistic Understanding Yuchen Li Yuan-Fang Li Andrej Risteski 89 61 0 07 Mar 2023
Finite-Sample Analysis of Learning High-Dimensional Single ReLU Neuron Jingfeng Wu Difan Zou Zixiang Chen Vladimir Braverman Quanquan Gu Sham Kakade 76 6 0 03 Mar 2023
Last Iterate Risk Bounds of SGD with Decaying Stepsize for Overparameterized Linear Regression Jingfeng Wu Difan Zou Vladimir Braverman Quanquan Gu Sham Kakade 94 20 0 12 Oct 2021