Explaining Emergent In-Context Learning as Kernel Regression

Explaining Emergent In-Context Learning as Kernel Regression

22 May 2023

Heng Ji

Papers citing "Explaining Emergent In-Context Learning as Kernel Regression"

14 / 14 papers shown

Title
Exact Learning Dynamics of In-Context Learning in Linear Transformers and Its Application to Non-Linear Transformers Nischal Mainali Lucas Teixeira 24 0 0 17 Apr 2025
On the Learn-to-Optimize Capabilities of Transformers in In-Context Sparse Recovery Renpu Liu Ruida Zhou Cong Shen Jing Yang 23 0 0 17 Oct 2024
Wrong-of-Thought: An Integrated Reasoning Framework with Multi-Perspective Verification and Wrong Information Yongheng Zhang Qiguang Chen Jingxuan Zhou Peng Wang Jiasheng Si Jin Wang Wenpeng Lu Libo Qin LRM 46 3 0 06 Oct 2024
Does learning the right latent variables necessarily improve in-context learning? Sarthak Mittal Eric Elmoznino Léo Gagnon Sangnie Bhardwaj Dhanya Sridhar Guillaume Lajoie 27 4 0 29 May 2024
Can large language models explore in-context? Akshay Krishnamurthy Keegan Harris Dylan J. Foster Cyril Zhang Aleksandrs Slivkins LM&Ro LLMAG LRM 118 20 0 22 Mar 2024
Visual In-Context Learning for Large Vision-Language Models Yucheng Zhou Xiang Li Qianning Wang Jianbing Shen MLLM 27 57 0 18 Feb 2024
Transformers Learn Nonlinear Features In Context: Nonconvex Mean-field Dynamics on the Attention Landscape Juno Kim Taiji Suzuki 16 18 0 02 Feb 2024
A Survey on In-context Learning Qingxiu Dong Lei Li Damai Dai Ce Zheng Jingyuan Ma ... Zhiyong Wu Baobao Chang Xu Sun Lei Li Zhifang Sui ReLM AIMat 20 460 0 31 Dec 2022
In-context Learning and Induction Heads Catherine Olsson Nelson Elhage Neel Nanda Nicholas Joseph Nova Dassarma ... Tom B. Brown Jack Clark Jared Kaplan Sam McCandlish C. Olah 240 456 0 24 Sep 2022
Large Language Models are Zero-Shot Reasoners Takeshi Kojima S. Gu Machel Reid Yutaka Matsuo Yusuke Iwasawa ReLM LRM 291 4,048 0 24 May 2022
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Jason W. Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Brian Ichter F. Xia Ed H. Chi Quoc Le Denny Zhou LM&Ro LRM AI4CE ReLM 315 8,402 0 28 Jan 2022
Open Vocabulary Electroencephalography-To-Text Decoding and Zero-shot Sentiment Classification Zhenhailong Wang Heng Ji 84 71 0 05 Dec 2021
Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity Yao Lu Max Bartolo Alastair Moore Sebastian Riedel Pontus Stenetorp AILaw LRM 277 1,114 0 18 Apr 2021
What Makes Good In-Context Examples for GPT- $3$ ? Jiachang Liu Dinghan Shen Yizhe Zhang Bill Dolan Lawrence Carin Weizhu Chen AAML RALM 275 1,311 0 17 Jan 2021