Do pretrained Transformers Learn In-Context by Gradient Descent?

Do pretrained Transformers Learn In-Context by Gradient Descent?

12 October 2023

Lingfeng Shen

Daniel Khashabi

Papers citing "Do pretrained Transformers Learn In-Context by Gradient Descent?"

5 / 5 papers shown

Title
ICL CIPHERS: Quantifying "Learning'' in In-Context Learning via Substitution Ciphers Zhouxiang Fang Aayush Mishra Muhan Gao Anqi Liu Daniel Khashabi 40 0 0 28 Apr 2025
The Learnability of In-Context Learning Noam Wies Yoav Levine Amnon Shashua 108 89 0 14 Mar 2023
In-context Learning and Induction Heads Catherine Olsson Nelson Elhage Neel Nanda Nicholas Joseph Nova Dassarma ... Tom B. Brown Jack Clark Jared Kaplan Sam McCandlish C. Olah 240 453 0 24 Sep 2022
Large Language Models are Zero-Shot Reasoners Takeshi Kojima S. Gu Machel Reid Yutaka Matsuo Yusuke Iwasawa ReLM LRM 291 2,712 0 24 May 2022
Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity Yao Lu Max Bartolo Alastair Moore Sebastian Riedel Pontus Stenetorp AILaw LRM 274 882 0 18 Apr 2021