An Explanation of In-context Learning as Implicit Bayesian Inference

3 November 2021

Papers citing "An Explanation of In-context Learning as Implicit Bayesian Inference"

50 / 531 papers shown

Title
Reason for Future, Act for Now: A Principled Framework for Autonomous LLM Agents with Provable Sample Efficiency Zhihan Liu Hao Hu Shenao Zhang Hongyi Guo Shuqi Ke Boyi Liu Zhaoran Wang LLMAG LRM 23 33 0 29 Sep 2023
Batch Calibration: Rethinking Calibration for In-Context Learning and Prompt Engineering Han Zhou Xingchen Wan Lev Proleev Diana Mincu Jilin Chen Katherine A. Heller Subhrajit Roy UQLM 18 53 0 29 Sep 2023
Identifying and Mitigating Privacy Risks Stemming from Language Models: A Survey Victoria Smith Ali Shahin Shamsabadi Carolyn Ashurst Adrian Weller PILM 27 24 0 27 Sep 2023
HyPoradise: An Open Baseline for Generative Speech Recognition with Large Language Models Cheng Chen Yuchen Hu Chao-Han Huck Yang Sabato Marco Siniscalchi Pin-Yu Chen E. Chng 21 42 0 27 Sep 2023
Generative Speech Recognition Error Correction with Large Language Models and Task-Activating Prompting Chao-Han Huck Yang Yile Gu Yi-Chieh Liu Shalini Ghosh I. Bulyko A. Stolcke KELM LRM 25 40 0 27 Sep 2023
Privacy-Preserving In-Context Learning with Differentially Private Few-Shot Generation Xinyu Tang Richard Shin Huseyin A. Inan Andre Manoel Fatemehsadat Mireshghallah Zinan Lin Sivakanth Gopi Janardhan Kulkarni Robert Sim 33 52 0 21 Sep 2023
In-Context Learning for Text Classification with Many Labels Aristides Milios Siva Reddy Dzmitry Bahdanau 13 34 0 19 Sep 2023
Human-AI Interactions and Societal Pitfalls Francisco Castro Jian Gao Sébastien Martin 20 2 0 19 Sep 2023
Understanding Catastrophic Forgetting in Language Models via Implicit Inference Suhas Kotha Jacob Mitchell Springer Aditi Raghunathan CLL 23 58 0 18 Sep 2023
Ambiguity-Aware In-Context Learning with Large Language Models Lingyu Gao Aditi Chaudhary Krishna Srinivasan Kazuma Hashimoto K. Raman Michael Bendersky 19 7 0 14 Sep 2023
Can Whisper perform speech-based in-context learning? Siyin Wang Chao-Han Huck Yang Ji Wu Chao Zhang 19 24 0 13 Sep 2023
Uncovering mesa-optimization algorithms in Transformers J. Oswald Eyvind Niklasson Maximilian Schlegel Seijin Kobayashi Nicolas Zucchet ... Mark Sandler Blaise Agüera y Arcas Max Vladymyrov Razvan Pascanu João Sacramento 17 53 0 11 Sep 2023
MAmmoTH: Building Math Generalist Models through Hybrid Instruction Tuning Xiang Yue Xingwei Qu Ge Zhang Yao Fu Wenhao Huang Huan Sun Yu-Chuan Su Wenhu Chen AIMat LRM 54 361 0 11 Sep 2023
Can you text what is happening? Integrating pre-trained language encoders into trajectory prediction models for autonomous driving Ali Keysan Andreas Look Eitan Kosman Gonca Gürsun Jörg Wagner Yu Yao Barbara Rakitsch 22 29 0 11 Sep 2023
Leveraging Large Language Models for Exploiting ASR Uncertainty Pranay Dighe Yi Su Shangshang Zheng Yunshu Liu Vineet Garg Xiaochuan Niu Ahmed H. Tewfik 13 12 0 09 Sep 2023
Are Emergent Abilities in Large Language Models just In-Context Learning? Sheng Lu Irina Bigoulaeva Rachneet Sachdeva Harish Tayyar Madabushi Iryna Gurevych LRM ELM ReLM 49 91 0 04 Sep 2023
Inductive-bias Learning: Generating Code Models with Large Language Model Toma Tanaka Naofumi Emoto Tsukasa Yumibayashi AI4CE 10 0 0 19 Aug 2023
DiagGPT: An LLM-based and Multi-agent Dialogue System with Automatic Topic Management for Flexible Task-Oriented Dialogue Lang Cao LM&MA LLMAG 30 3 0 15 Aug 2023
CausalLM is not optimal for in-context learning Nan Ding Tomer Levinboim Jialin Wu Sebastian Goodman Radu Soricut 24 23 0 14 Aug 2023
Text2KGBench: A Benchmark for Ontology-Driven Knowledge Graph Generation from Text Nandana Mihindukulasooriya Sanju Tiwari Carlos F. Enguix K. Lata 22 52 0 04 Aug 2023
When Large Language Models Meet Personalization: Perspectives of Challenges and Opportunities Jin Chen Zheng Liu Xunpeng Huang Chenwang Wu Qi Liu ... Yuxuan Lei Xiaolong Chen Xingmei Wang Defu Lian Enhong Chen ALM 22 110 0 31 Jul 2023
Uncertainty in Natural Language Generation: From Theory to Applications Joris Baan Nico Daheim Evgenia Ilia Dennis Ulmer Haau-Sing Li Raquel Fernández Barbara Plank Rico Sennrich Chrysoula Zerva Wilker Aziz UQLM 17 40 0 28 Jul 2023
Investigating the Learning Behaviour of In-context Learning: A Comparison with Supervised Learning Xindi Wang Yufei Wang Can Xu Xiubo Geng Bowen Zhang Chongyang Tao Frank Rudzicz Robert E. Mercer Daxin Jiang 14 11 0 28 Jul 2023
In-Context Learning Learns Label Relationships but Is Not Conventional Learning Jannik Kossen Y. Gal Tom Rainforth 32 27 0 23 Jul 2023
What can a Single Attention Layer Learn? A Study Through the Random Features Lens Hengyu Fu Tianyu Guo Yu Bai Song Mei MLT 13 22 0 21 Jul 2023
Overthinking the Truth: Understanding how Language Models Process False Demonstrations Danny Halawi Jean-Stanislas Denain Jacob Steinhardt 20 52 0 18 Jul 2023
On the (In)Effectiveness of Large Language Models for Chinese Text Correction Yinghui Li Haojing Huang Shirong Ma Yong-jia Jiang Y. Li F. Zhou Haitao Zheng Qingyu Zhou 29 43 0 18 Jul 2023
Learning to Retrieve In-Context Examples for Large Language Models Liang Wang Nan Yang Furu Wei RALM 30 38 0 14 Jul 2023
Large Language Models Michael R Douglas LLMAG LM&MA 24 555 0 11 Jul 2023
Towards Understanding In-Context Learning with Contrastive Demonstrations and Saliency Maps Fuxiao Liu Paiheng Xu Zongxi Li Yue Feng Hyemi Song 11 31 0 11 Jul 2023
Large Language Models as General Pattern Machines Suvir Mirchandani F. Xia Peter R. Florence Brian Ichter Danny Driess Montse Gonzalez Arenas Kanishka Rao Dorsa Sadigh Andy Zeng LLMAG 39 183 0 10 Jul 2023
Bidirectional Attention as a Mixture of Continuous Word Experts Kevin Christian Wibisono Yixin Wang MoE 8 0 0 08 Jul 2023
One Step of Gradient Descent is Provably the Optimal In-Context Learner with One Layer of Linear Self-Attention Arvind V. Mahankali Tatsunori B. Hashimoto Tengyu Ma MLT 14 81 0 07 Jul 2023
Amplifying Limitations, Harms and Risks of Large Language Models Michael OÑeill M. Connor 10 7 0 06 Jul 2023
Scaling In-Context Demonstrations with Structured Attention Tianle Cai Kaixuan Huang Jason D. Lee Mengdi Wang LRM 23 8 0 05 Jul 2023
External Reasoning: Towards Multi-Large-Language-Models Interchangeable Assistance with Human Feedback Akide Liu KELM LRM 13 0 0 05 Jul 2023
Trainable Transformer in Transformer A. Panigrahi Sadhika Malladi Mengzhou Xia Sanjeev Arora VLM 22 12 0 03 Jul 2023
Still No Lie Detector for Language Models: Probing Empirical and Conceptual Roadblocks B. Levinstein Daniel A. Herrmann 6 53 0 30 Jun 2023
Meta-training with Demonstration Retrieval for Efficient Few-shot Learning Aaron Mueller Kanika Narang Lambert Mathias Qifan Wang Hamed Firooz RALM 11 3 0 30 Jun 2023
DisasterResponseGPT: Large Language Models for Accelerated Plan of Action Development in Disaster Response Scenarios Vinicius G. Goecks Nicholas R. Waytowich 10 26 0 29 Jun 2023
Could Small Language Models Serve as Recommenders? Towards Data-centric Cold-start Recommendations Xuansheng Wu Huachi Zhou Yucheng Shi Wenlin Yao Xiao Shi Huang Ninghao Liu LRM 21 8 0 29 Jun 2023
Understanding In-Context Learning via Supportive Pretraining Data Xiaochuang Han Daniel Simig Todor Mihaylov Yulia Tsvetkov Asli Celikyilmaz Tianlu Wang AIMat 26 33 0 26 Jun 2023
Pretraining task diversity and the emergence of non-Bayesian in-context learning for regression Allan Raventós Mansheej Paul F. Chen Surya Ganguli 19 70 0 26 Jun 2023
Supervised Pretraining Can Learn In-Context Reinforcement Learning Jonathan Lee Annie Xie Aldo Pacchiano Yash Chandak Chelsea Finn Ofir Nachum Emma Brunskill OffRL 22 73 0 26 Jun 2023
Beyond Scale: the Diversity Coefficient as a Data Quality Metric Demonstrates LLMs are Pre-trained on Formally Diverse Data Alycia Lee Brando Miranda Sudharsan Sundar Sanmi Koyejo 26 17 0 24 Jun 2023
Harnessing the Power of Adversarial Prompting and Large Language Models for Robust Hypothesis Generation in Astronomy I. Ciucă Y. Ting 丁 Sandor Kruk K. Iyer 8 7 0 20 Jun 2023
Democratizing LLMs for Low-Resource Languages by Leveraging their English Dominant Abilities with Linguistically-Diverse Prompts Xuan-Phi Nguyen Sharifah Mahani Aljunied Shafiq R. Joty Lidong Bing 18 32 0 20 Jun 2023
Trained Transformers Learn Linear Models In-Context Ruiqi Zhang Spencer Frei Peter L. Bartlett 14 173 0 16 Jun 2023
Pushing the Limits of ChatGPT on NLP Tasks Xiaofei Sun Linfeng Dong Xiaoya Li Zhen Wan Shuhe Wang ... Jiwei Li Fei Cheng Lingjuan Lyu Fei Wu Guoyin Wang AI4MH LRM 29 29 0 16 Jun 2023
Schema-learning and rebinding as mechanisms of in-context learning and emergence Siva K. Swaminathan Antoine Dedieu Rajkumar Vasudeva Raju Murray Shanahan Miguel Lazaro-Gredilla Dileep George 24 8 0 16 Jun 2023