An Explanation of In-context Learning as Implicit Bayesian Inference

3 November 2021

Papers citing "An Explanation of In-context Learning as Implicit Bayesian Inference"

50 / 531 papers shown

Title
Interpreting Pretrained Language Models via Concept Bottlenecks Zhen Tan Lu Cheng Song Wang Yuan Bo Jundong Li Huan Liu LRM 22 20 0 08 Nov 2023
In-Context Exemplars as Clues to Retrieving from Large Associative Memory Jiachen Zhao 8 7 0 06 Nov 2023
Active Instruction Tuning: Improving Cross-Task Generalization by Training on Prompt Sensitive Tasks Po-Nien Kung Fan Yin Di Wu Kai-Wei Chang Nanyun Peng 67 39 0 01 Nov 2023
The Mystery of In-Context Learning: A Comprehensive Survey on Interpretation and Analysis Yuxiang Zhou Jiazheng Li Yanzheng Xiang Hanqi Yan Lin Gui Yulan He 22 14 0 01 Nov 2023
Transformers are Provably Optimal In-context Estimators for Wireless Communications Vishnu Teja Kunde Vicram Rajagopalan Chandra Shekhara Kaushik Valmeekam Krishna R. Narayanan S. Shakkottai D. Kalathil J. Chamberland 29 4 0 01 Nov 2023
Defining a New NLP Playground Sha Li Chi Han Pengfei Yu Carl N. Edwards Manling Li ... Yi Ren Fung Charles Yu Joel R. Tetreault Eduard H. Hovy Heng Ji 31 5 0 31 Oct 2023
Which Examples to Annotate for In-Context Learning? Towards Effective and Efficient Selection Costas Mavromatis Balasubramaniam Srinivasan Zhengyuan Shen Jiani Zhang Huzefa Rangwala Christos Faloutsos George Karypis 19 21 0 30 Oct 2023
Social Contract AI: Aligning AI Assistants with Implicit Group Norms Jan-Philipp Fränken Sam Kwok Peixuan Ye Kanishk Gandhi Dilip Arumugam Jared Moore Alex Tamkin Tobias Gerstenberg Noah D. Goodman 24 7 0 26 Oct 2023
In-Context Learning Dynamics with Random Binary Sequences Eric J. Bigelow Ekdeep Singh Lubana Robert P. Dick Hidenori Tanaka T. Ullman 21 4 0 26 Oct 2023
Unpacking the Ethical Value Alignment in Big Models Xiaoyuan Yi Jing Yao Xiting Wang Xing Xie 11 11 0 26 Oct 2023
The Expressive Power of Low-Rank Adaptation Yuchen Zeng Kangwook Lee 28 49 0 26 Oct 2023
Deja Vu: Contextual Sparsity for Efficient LLMs at Inference Time Zichang Liu Jue Wang Tri Dao Tianyi Zhou Binhang Yuan ... Anshumali Shrivastava Ce Zhang Yuandong Tian Christopher Ré Beidi Chen BDL 17 191 0 26 Oct 2023
Exploring Question Decomposition for Zero-Shot VQA Zaid Khan B. Vijaykumar S. Schulter Manmohan Chandraker Yun Fu ReLM 17 10 0 25 Oct 2023
CycleAlign: Iterative Distillation from Black-box LLM to White-box Models for Better Human Alignment Jixiang Hong Quan Tu C. Chen Xing Gao Ji Zhang Rui Yan ALM 6 10 0 25 Oct 2023
Dissecting In-Context Learning of Translations in GPTs Vikas Raunak Hany Awadalla Arul Menezes 42 1 0 24 Oct 2023
In-Context Learning Creates Task Vectors Roee Hendel Mor Geva Amir Globerson 7 131 0 24 Oct 2023
MindLLM: Pre-training Lightweight Large Language Model from Scratch, Evaluations and Domain Applications Yizhe Yang Huashan Sun Jiawei Li Runheng Liu Yinghao Li Yuhang Liu Heyan Huang Yang Gao ALM LRM 8 8 0 24 Oct 2023
Large Language Models are Visual Reasoning Coordinators Liangyu Chen Bo Li Sheng Shen Jingkang Yang Chunyuan Li Kurt Keutzer Trevor Darrell Ziwei Liu VLM LRM 34 47 0 23 Oct 2023
Function Vectors in Large Language Models Eric Todd Millicent Li Arnab Sen Sharma Aaron Mueller Byron C. Wallace David Bau 8 99 0 23 Oct 2023
Implicit meta-learning may lead language models to trust more reliable sources Dmitrii Krasheninnikov Egor Krasheninnikov Bruno Mlodozeniec Tegan Maharaj David M. Krueger 11 3 0 23 Oct 2023
Confronting LLMs with Traditional ML: Rethinking the Fairness of Large Language Models in Tabular Classifications Yanchen Liu Srishti Gautam Jiaqi Ma Himabindu Lakkaraju LMTD 24 11 0 23 Oct 2023
Evoke: Evoking Critical Thinking Abilities in LLMs via Reviewer-Author Prompt Editing Xinyu Hu Pengfei Tang Simiao Zuo Zihan Wang Bowen Song Qiang Lou Jian Jiao Denis Xavier Charles LRM 33 7 0 20 Oct 2023
MusicAgent: An AI Agent for Music Understanding and Generation with Large Language Models Dingyao Yu Kaitao Song Peiling Lu Tianyu He Xu Tan Wei Ye Shikun Zhang Jiang Bian LLMAG 16 16 0 18 Oct 2023
Investigating Uncertainty Calibration of Aligned Language Models under the Multiple-Choice Setting Guande He Peng Cui Jianfei Chen Wenbo Hu Jun Zhu 45 11 0 18 Oct 2023
IDEAL: Influence-Driven Selective Annotations Empower In-Context Learners in Large Language Models Shaokun Zhang Xiaobo Xia Zhaoqing Wang Ling-Hao Chen Jiale Liu Qingyun Wu Tongliang Liu 29 20 0 16 Oct 2023
How Do Transformers Learn In-Context Beyond Simple Functions? A Case Study on Learning with Representations Tianyu Guo Wei Hu Song Mei Huan Wang Caiming Xiong Silvio Savarese Yu Bai 19 46 0 16 Oct 2023
Demonstrations Are All You Need: Advancing Offensive Content Paraphrasing using In-Context Learning Anirudh Som Karan Sikka Helen Gent Ajay Divakaran A. Kathol D. Vergyri 16 4 0 16 Oct 2023
Generative Calibration for In-context Learning Zhongtao Jiang Yuanzhe Zhang Cao Liu Jun Zhao Kang Liu 157 17 0 16 Oct 2023
DemoNSF: A Multi-task Demonstration-based Generative Framework for Noisy Slot Filling Task Guanting Dong Tingfeng Hui Zhuoma Gongque Jinxu Zhao Daichi Guo Gang Zhao Keqing He Weiran Xu DiffM 11 7 0 16 Oct 2023
A Search for Prompts: Generating Structured Answers from Contracts Adam Roegiest Radha Chitta Jonathan Donnelly Maya Lash A. Vtyurina Franccois Longtin ELM AILaw 28 2 0 16 Oct 2023
Assessing the Reliability of Large Language Model Knowledge Weixuan Wang Barry Haddow Alexandra Birch Wei Peng KELM HILM 63 15 0 15 Oct 2023
Transformers as Decision Makers: Provable In-Context Reinforcement Learning via Supervised Pretraining Licong Lin Yu Bai Song Mei OffRL 30 42 0 12 Oct 2023
Do pretrained Transformers Learn In-Context by Gradient Descent? Lingfeng Shen Aayush Mishra Daniel Khashabi 17 7 0 12 Oct 2023
How Many Pretraining Tasks Are Needed for In-Context Learning of Linear Regression? Jingfeng Wu Difan Zou Zixiang Chen Vladimir Braverman Quanquan Gu Peter L. Bartlett 118 48 0 12 Oct 2023
Not All Demonstration Examples are Equally Beneficial: Reweighting Demonstration Examples for In-Context Learning Zhe Yang Damai Dai Peiyi Wang Zhifang Sui 34 9 0 12 Oct 2023
Is attention required for ICL? Exploring the Relationship Between Model Architecture and In-Context Learning Ability Ivan Lee Nan Jiang Taylor Berg-Kirkpatrick 32 12 0 12 Oct 2023
In-Context Unlearning: Language Models as Few Shot Unlearners Martin Pawelczyk Seth Neel Himabindu Lakkaraju MU 21 99 0 11 Oct 2023
KwaiYiiMath: Technical Report Jia-Yi Fu Lei Lin Xiaoyang Gao Pengli Liu Zhengzong Chen ... Zijia Lin Fuzheng Zhang Zhongyuan Wang Di Zhang Kun Gai LRM ReLM RALM 43 2 0 11 Oct 2023
From Supervised to Generative: A Novel Paradigm for Tabular Deep Learning with Large Language Models Xumeng Wen Han Zhang Shun Zheng Wei Xu Jiang Bian LMTD ALM 62 20 0 11 Oct 2023
Humans and language models diverge when predicting repeating text Aditya R. Vaidya Javier S. Turek Alexander G. Huth 17 5 0 10 Oct 2023
BC4LLM: Trusted Artificial Intelligence When Blockchain Meets Large Language Models Haoxiang Luo Jian Luo Athanasios V. Vasilakos 19 9 0 10 Oct 2023
Parrot Mind: Towards Explaining the Complex Task Reasoning of Pretrained Large Language Models with Template-Content Structure Haotong Yang Fanxu Meng Zhouchen Lin Muhan Zhang LRM 23 2 0 09 Oct 2023
In-Context Convergence of Transformers Yu Huang Yuan-Chia Cheng Yingbin Liang MLT 32 59 0 08 Oct 2023
Towards Better Chain-of-Thought Prompting Strategies: A Survey Zihan Yu Liang He Zhen Wu Xinyu Dai Jiajun Chen LRM 118 44 0 08 Oct 2023
Understanding In-Context Learning in Transformers and LLMs by Learning to Learn Discrete Functions S. Bhattamishra Arkil Patel Phil Blunsom Varun Kanade 19 41 0 04 Oct 2023
Large language models in textual analysis for gesture selection Laura Birka Hensel Nutchanon Yongsatianchot P. Torshizi E. Minucci Stacy Marsella SLR 30 7 0 04 Oct 2023
From Language Modeling to Instruction Following: Understanding the Behavior Shift in LLMs after Instruction Tuning Xuansheng Wu Wenlin Yao Jianshu Chen Xiaoman Pan Xiaoyang Wang Ninghao Liu Dong Yu LRM 20 26 0 30 Sep 2023
Dynamic Demonstrations Controller for In-Context Learning Fei Zhao Taotian Pang Zhen Wu Zheng Ma Shujian Huang Xinyu Dai 27 5 0 30 Sep 2023
Decoding In-Context Learning: Neuroscience-inspired Analysis of Representations in Large Language Models Safoora Yousefi Leo Betthauser Hosein Hasanbeig Raphael Milliere Ida Momennejad 13 4 0 30 Sep 2023
Understanding In-Context Learning from Repetitions Jianhao Yan Jin Xu Chiyu Song Chenming Wu Yafu Li Yue Zhang 17 20 0 30 Sep 2023