A Mechanism for Sample-Efficient In-Context Learning for Sparse Retrieval Tasks

26 May 2023

Papers citing "A Mechanism for Sample-Efficient In-Context Learning for Sparse Retrieval Tasks"

21 / 21 papers shown

Title
Transformers versus the EM Algorithm in Multi-class Clustering Yihan He Hong-Yu Chen Yuan Cao Jianqing Fan Han Liu 55 0 0 09 Feb 2025
SymDPO: Boosting In-Context Learning of Large Multimodal Models with Symbol Demonstration Direct Preference Optimization Hongrui Jia Chaoya Jiang Haiyang Xu Wei Ye Mengfan Dong Ming Yan Ji Zhang Fei Huang Shikun Zhang MLLM 89 2 0 17 Nov 2024
One-Layer Transformer Provably Learns One-Nearest Neighbor In Context Zihao Li Yuan Cao Cheng Gao Yihan He Han Liu Jason M. Klusowski Jianqing Fan Mengdi Wang MLT 47 6 0 16 Nov 2024
Toward Understanding In-context vs. In-weight Learning Bryan Chan Xinyi Chen András Gyorgy Dale Schuurmans 65 3 0 30 Oct 2024
How Transformers Utilize Multi-Head Attention in In-Context Learning? A Case Study on Sparse Linear Regression Xingwu Chen Lei Zhao Difan Zou 36 6 0 08 Aug 2024
How Far Can In-Context Alignment Go? Exploring the State of In-Context Alignment Heyan Huang Yinghao Li Huashan Sun Yu Bai Yang Gao 33 3 0 17 Jun 2024
Towards Efficient LLM Grounding for Embodied Multi-Agent Collaboration Yang Zhang Shixin Yang Chenjia Bai Fei Wu Xiu Li Zhen Wang Xuelong Li LLMAG 31 25 0 23 May 2024
Can large language models explore in-context? Akshay Krishnamurthy Keegan Harris Dylan J. Foster Cyril Zhang Aleksandrs Slivkins LM&Ro LLMAG LRM 118 22 0 22 Mar 2024
In-context Exploration-Exploitation for Reinforcement Learning Zhenwen Dai Federico Tomasi Sina Ghiassian OffRL OnRL 33 3 0 11 Mar 2024
The Evolution of Statistical Induction Heads: In-Context Learning Markov Chains Benjamin L. Edelman Ezra Edelman Surbhi Goel Eran Malach Nikolaos Tsilivis BDL 21 39 0 16 Feb 2024
Linear attention is (maybe) all you need (to understand transformer optimization) Kwangjun Ahn Xiang Cheng Minhak Song Chulhee Yun Ali Jadbabaie S. Sra 18 43 1 02 Oct 2023
Reason for Future, Act for Now: A Principled Framework for Autonomous LLM Agents with Provable Sample Efficiency Zhihan Liu Hao Hu Shenao Zhang Hongyi Guo Shuqi Ke Boyi Liu Zhaoran Wang LLMAG LRM 23 33 0 29 Sep 2023
Supervised Pretraining Can Learn In-Context Reinforcement Learning Jonathan Lee Annie Xie Aldo Pacchiano Yash Chandak Chelsea Finn Ofir Nachum Emma Brunskill OffRL 22 73 0 26 Jun 2023
Trained Transformers Learn Linear Models In-Context Ruiqi Zhang Spencer Frei Peter L. Bartlett 14 173 0 16 Jun 2023
The Crucial Role of Normalization in Sharpness-Aware Minimization Yan Dai Kwangjun Ahn S. Sra 21 17 0 24 May 2023
The Learnability of In-Context Learning Noam Wies Yoav Levine Amnon Shashua 117 91 0 14 Mar 2023
Learning threshold neurons via the "edge of stability" Kwangjun Ahn Sébastien Bubeck Sinho Chewi Y. Lee Felipe Suarez Yi Zhang MLT 31 36 0 14 Dec 2022
In-context Learning and Induction Heads Catherine Olsson Nelson Elhage Neel Nanda Nicholas Joseph Nova Dassarma ... Tom B. Brown Jack Clark Jared Kaplan Sam McCandlish C. Olah 240 456 0 24 Sep 2022
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Jason W. Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Brian Ichter F. Xia Ed H. Chi Quoc Le Denny Zhou LM&Ro LRM AI4CE ReLM 315 8,402 0 28 Jan 2022
Making Pre-trained Language Models Better Few-shot Learners Tianyu Gao Adam Fisch Danqi Chen 241 1,913 0 31 Dec 2020
Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks Chelsea Finn Pieter Abbeel Sergey Levine OOD 243 11,659 0 09 Mar 2017