What and How does In-Context Learning Learn? Bayesian Model Averaging,
Parameterization, and Generalization

What and How does In-Context Learning Learn? Bayesian Model Averaging, Parameterization, and Generalization

30 May 2023

Papers citing "What and How does In-Context Learning Learn? Bayesian Model Averaging, Parameterization, and Generalization"

16 / 16 papers shown

Title
Toward Efficient Exploration by Large Language Model Agents Dilip Arumugam Thomas L. Griffiths LLMAG 87 0 0 29 Apr 2025
Zero-shot Model-based Reinforcement Learning using Large Language Models Abdelhakim Benechehab Youssef Attia El Hili Ambroise Odonnat Oussama Zekri Albert Thomas Giuseppe Paolo Maurizio Filippone I. Redko Balázs Kégl OffRL 53 1 0 17 Feb 2025
Are Transformers Able to Reason by Connecting Separated Knowledge in Training Data? Yutong Yin Zhaoran Wang LRM ReLM 43 0 0 27 Jan 2025
Dissecting the Interplay of Attention Paths in a Statistical Mechanics Theory of Transformers Lorenzo Tiberi Francesca Mignacco Kazuki Irie H. Sompolinsky 36 5 0 24 May 2024
Towards Better Understanding of In-Context Learning Ability from In-Context Uncertainty Quantification Shang Liu Zhongze Cai Guanting Chen Xiaocheng Li UQCV 38 1 0 24 May 2024
An Information-Theoretic Analysis of In-Context Learning Hong Jun Jeon Jason D. Lee Qi Lei Benjamin Van Roy 13 18 0 28 Jan 2024
In-Context Learning Learns Label Relationships but Is Not Conventional Learning Jannik Kossen Y. Gal Tom Rainforth 22 27 0 23 Jul 2023
The Learnability of In-Context Learning Noam Wies Yoav Levine Amnon Shashua 108 89 0 14 Mar 2023
A Kernel-Based View of Language Model Fine-Tuning Sadhika Malladi Alexander Wettig Dingli Yu Danqi Chen Sanjeev Arora VLM 66 60 0 11 Oct 2022
Relational Reasoning via Set Transformers: Provable Efficiency and Applications to MARL Fengzhuo Zhang Boyi Liu Kaixin Wang Vincent Y. F. Tan Zhuoran Yang Zhaoran Wang OffRL LRM 42 10 0 20 Sep 2022
Large Language Models are Zero-Shot Reasoners Takeshi Kojima S. Gu Machel Reid Yutaka Matsuo Yusuke Iwasawa ReLM LRM 291 2,712 0 24 May 2022
Instruction Induction: From Few Examples to Natural Language Task Descriptions Or Honovich Uri Shaham Samuel R. Bowman Omer Levy ELM LRM 107 133 0 22 May 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 301 11,730 0 04 Mar 2022
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Jason W. Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Brian Ichter F. Xia Ed H. Chi Quoc Le Denny Zhou LM&Ro LRM AI4CE ReLM 315 8,261 0 28 Jan 2022
Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity Yao Lu Max Bartolo Alastair Moore Sebastian Riedel Pontus Stenetorp AILaw LRM 274 1,114 0 18 Apr 2021
What Makes Good In-Context Examples for GPT- $3$ ? Jiachang Liu Dinghan Shen Yizhe Zhang Bill Dolan Lawrence Carin Weizhu Chen AAML RALM 275 1,296 0 17 Jan 2021