An Explanation of In-context Learning as Implicit Bayesian Inference

3 November 2021

Papers citing "An Explanation of In-context Learning as Implicit Bayesian Inference"

50 / 531 papers shown

Title
Towards Graph Foundation Models: Learning Generalities Across Graphs via Task-Trees Zehong Wang Zheyuan Zhang Tianyi Ma Nitesh V. Chawla Chuxu Zhang Yanfang Ye AI4CE 71 0 0 21 Dec 2024
Enriching Social Science Research via Survey Item Linking Tornike Tsereteli Daniel Ruffinelli Simone Paolo Ponzetto LRM 70 0 0 20 Dec 2024
Understanding Knowledge Hijack Mechanism in In-context Learning through Associative Memory Shuo Wang Issei Sato 69 0 0 16 Dec 2024
Rank It, Then Ask It: Input Reranking for Maximizing the Performance of LLMs on Symmetric Tasks Mohsen Dehghankar Abolfazl Asudeh 61 1 0 30 Nov 2024
Selective Attention: Enhancing Transformer through Principled Context Control Xuechen Zhang Xiangyu Chang Mingchen Li A. Roy-Chowdhury J. Chen Samet Oymak 62 2 0 19 Nov 2024
Re-examining learning linear functions in context Omar Naim Guilhem Fouilhé Nicholas Asher 63 0 0 18 Nov 2024
One-Layer Transformer Provably Learns One-Nearest Neighbor In Context Zihao Li Yuan Cao Cheng Gao Yihan He Han Liu Jason M. Klusowski Jianqing Fan Mengdi Wang MLT 47 6 0 16 Nov 2024
IntentGPT: Few-shot Intent Discovery with Large Language Models Juan A. Rodriguez Nicholas Botzer David Vazquez Christopher Pal M. Pedersoli I. Laradji VLM 68 2 0 16 Nov 2024
A Practical Guide to Fine-tuning Language Models with Limited Data Márton Szép Daniel Rueckert Rüdiger von Eisenhart-Rothe Florian Hinterwimmer SyDa ALM 42 2 0 14 Nov 2024
Can Custom Models Learn In-Context? An Exploration of Hybrid Architecture Performance on In-Context Learning Tasks Ryan Campbell Nelson Lojo Kesava Viswanadha Christoffer Grondal Tryggestad Derrick Han Sun Sriteja Vijapurapu August Rolfsen Anant Sahai 16 0 0 06 Nov 2024
EXPLORA: Efficient Exemplar Subset Selection for Complex Reasoning Kiran Purohit Venktesh V Raghuram Devalla Krishna Mohan Yerragorla Sourangshu Bhattacharya Avishek Anand LRM 27 1 0 06 Nov 2024
Fair In-Context Learning via Latent Concept Variables Karuna Bhaila Minh-Hao Van Kennedy Edemacu Chen Zhao Feng Chen Xintao Wu 24 0 0 04 Nov 2024
Comparative Analysis of Demonstration Selection Algorithms for LLM In-Context Learning Dong Shu Mengnan Du 35 0 0 30 Oct 2024
Toward Understanding In-context vs. In-weight Learning Bryan Chan Xinyi Chen András Gyorgy Dale Schuurmans 65 3 0 30 Oct 2024
On the Role of Depth and Looping for In-Context Learning with Task Diversity Khashayar Gatmiry Nikunj Saunshi Sashank J. Reddi Stefanie Jegelka Sanjiv Kumar 22 2 0 29 Oct 2024
CFSafety: Comprehensive Fine-grained Safety Assessment for LLMs Zhihao Liu Chenhui Hu ALM ELM 18 1 0 29 Oct 2024
Random Policy Enables In-Context Reinforcement Learning within Trust Horizons Weiqin Chen Santiago Paternain OffRL 35 0 0 25 Oct 2024
LanFL: Differentially Private Federated Learning with Large Language Models using Synthetic Samples Huiyu Wu Diego Klabjan FedML 33 0 0 24 Oct 2024
Bayesian scaling laws for in-context learning Aryaman Arora Dan Jurafsky Christopher Potts Noah D. Goodman 24 2 0 21 Oct 2024
Mesa-Extrapolation: A Weave Position Encoding Method for Enhanced Extrapolation in LLMs Xin Ma Yang Liu J. Liu Xiaoxu Ma 11 1 0 21 Oct 2024
Bayesian Concept Bottleneck Models with LLM Priors Jean Feng Avni Kothari Luke Zier Chandan Singh Yan Shuo Tan 13 2 0 21 Oct 2024
Make LLMs better zero-shot reasoners: Structure-orientated autonomous reasoning Pengfei He Zitao Li Yue Xing Yaling Li Jiliang Tang Bolin Ding LLMAG LRM 27 1 0 18 Oct 2024
In-context learning and Occam's razor Eric Elmoznino Tom Marty Tejas Kasetty Léo Gagnon Sarthak Mittal Mahan Fathi Dhanya Sridhar Guillaume Lajoie 32 1 0 17 Oct 2024
Aggregation Artifacts in Subjective Tasks Collapse Large Language Models' Posteriors Georgios Chochlakis Alexandros Potamianos Kristina Lerman Shrikanth Narayanan 25 0 0 17 Oct 2024
On the Training Convergence of Transformers for In-Context Classification Wei Shen Ruida Zhou Jing Yang Cong Shen 18 3 0 15 Oct 2024
Cognitive Overload Attack:Prompt Injection for Long Context Bibek Upadhayay Vahid Behzadan Amin Karbasi AAML 28 2 0 15 Oct 2024
Bypassing the Exponential Dependency: Looped Transformers Efficiently Learn In-context by Multi-step Gradient Descent Bo Chen Xiaoyu Li Yingyu Liang Zhenmei Shi Zhao-quan Song 83 19 0 15 Oct 2024
Inference and Verbalization Functions During In-Context Learning Junyi Tao Xiaoyin Chen Nelson F. Liu ReLM LRM 16 0 0 12 Oct 2024
VoxelPrompt: A Vision-Language Agent for Grounded Medical Image Analysis Andrew Hoopes V. Butoi John Guttag Adrian V. Dalca MedIm LM&MA 35 1 0 10 Oct 2024
Can Looped Transformers Learn to Implement Multi-step Gradient Descent for In-context Learning? Khashayar Gatmiry Nikunj Saunshi Sashank J. Reddi Stefanie Jegelka Sanjiv Kumar 67 17 0 10 Oct 2024
Let's Ask GNN: Empowering Large Language Model for Graph In-Context Learning Zhengyu Hu Yichuan Li Zhengyu Chen J. Wang Han Liu Kyumin Lee Kaize Ding GNN 107 1 0 09 Oct 2024
Copiloting Diagnosis of Autism in Real Clinical Scenarios via LLMs Yi Jiang Qingyang Shen Shuzhong Lai Shunyu Qi Qian Zheng Lin Yao Yueming Wang Gang Pan 18 1 0 08 Oct 2024
Everything Everywhere All at Once: LLMs can In-Context Learn Multiple Tasks in Superposition Zheyang Xiong Ziyang Cai John Cooper Albert Ge Vasilis Papageorgiou ... Saurabh Agarwal Grigorios G Chrysos Samet Oymak Kangwook Lee Dimitris Papailiopoulos LRM 22 1 0 08 Oct 2024
Transformers learn variable-order Markov chains in-context Ruida Zhou C. Tian Suhas Diggavi 23 0 0 07 Oct 2024
Density estimation with LLMs: a geometric investigation of in-context learning trajectories Toni J. B. Liu Nicolas Boullé Raphael Sarfati Christopher Earls 20 0 0 07 Oct 2024
Task Diversity Shortens the ICL Plateau Jaeyeon Kim Sehyun Kwon Joo Young Choi Jongho Park Jaewoong Cho Jason D. Lee Ernest K. Ryu MoMe 29 2 0 07 Oct 2024
Wrong-of-Thought: An Integrated Reasoning Framework with Multi-Perspective Verification and Wrong Information Yongheng Zhang Qiguang Chen Jingxuan Zhou Peng Wang Jiasheng Si Jin Wang Wenpeng Lu Libo Qin LRM 44 3 0 06 Oct 2024
Simplicity bias and optimization threshold in two-layer ReLU networks Etienne Boursier Nicolas Flammarion 24 2 0 03 Oct 2024
Large Language Models as Markov Chains Oussama Zekri Ambroise Odonnat Abdelhakim Benechehab Linus Bleistein Nicolas Boullé I. Redko 34 9 0 03 Oct 2024
Bayes' Power for Explaining In-Context Learning Generalizations Samuel G. Müller Noah Hollmann Frank Hutter BDL 29 1 0 02 Oct 2024
Mitigating Copy Bias in In-Context Learning through Neuron Pruning Ameen Ali Lior Wolf Ivan Titov 27 2 0 02 Oct 2024
Racing Thoughts: Explaining Contextualization Errors in Large Language Models Michael A. Lepori Michael Mozer Asma Ghandeharioun LRM 80 1 0 02 Oct 2024
Are Large Language Models In-Context Personalized Summarizers? Get an iCOPERNICUS Test Done! Divya Patel Pathik Patel Ankush Chander Sourish Dasgupta Tanmoy Chakraborty 16 1 0 30 Sep 2024
"Oh LLM, I'm Asking Thee, Please Give Me a Decision Tree": Zero-Shot Decision Tree Induction and Embedding with Large Language Models Ricardo Knauer Mario Koddenbrock Raphael Wallsberger Nicholas M. Brisson Georg N. Duda Deborah Falla David W. Evans Erik Rodner 33 0 0 27 Sep 2024
Can Vision Language Models Learn from Visual Demonstrations of Ambiguous Spatial Reasoning? Bowen Zhao Leo Parker Dirac Paulina Varshavskaya VLM LRM 11 0 0 25 Sep 2024
Zero-shot forecasting of chaotic systems Yuanzhao Zhang William Gilpin AI4TS 37 4 0 24 Sep 2024
In-Context Learning May Not Elicit Trustworthy Reasoning: A-Not-B Errors in Pretrained Language Models Pengrui Han Peiyang Song Haofei Yu Jiaxuan You ReLM LRM 21 1 0 23 Sep 2024
Provable In-Context Learning of Linear Systems and Linear Elliptic PDEs with Transformers Frank Cole Yulong Lu Riley OÑeill Tianhao Zhang 34 2 0 18 Sep 2024
Multidimensional Human Activity Recognition With Large Language Model: A Conceptual Framework Syed Mhamudul Hasan 24 0 0 16 Sep 2024
Meta-Whisper: Speech-Based Meta-ICL for ASR on Low-Resource Languages Ming-Hao Hsu Kuan Po Huang Hung-yi Lee 25 0 0 16 Sep 2024