In-Context Learning Creates Task Vectors

24 October 2023

Papers citing "In-Context Learning Creates Task Vectors"

50 / 112 papers shown

Title
Understanding In-context Learning of Addition via Activation Subspaces Xinyan Hu Kayo Yin Michael I. Jordan Jacob Steinhardt Lijie Chen 51 0 0 08 May 2025
Embracing Collaboration Over Competition: Condensing Multiple Prompts for Visual In-Context Learning J. Wang Tianci Luo Yaohua Zha Yan Feng Ruisheng Luo B. Chen Tao Dai Long Chen Yaowei Wang Shu-Tao Xia VLM 50 0 0 30 Apr 2025
Improving Reasoning Performance in Large Language Models via Representation Engineering Bertram Højer Oliver Jarvis Stefan Heinrich LRM 83 1 0 28 Apr 2025
Functional Abstraction of Knowledge Recall in Large Language Models Zijian Wang Chang Xu KELM 32 0 0 20 Apr 2025
On Linear Representations and Pretraining Data Frequency in Language Models Jack Merullo Noah A. Smith Sarah Wiegreffe Yanai Elazar 35 0 0 16 Apr 2025
When is Task Vector Provably Effective for Model Editing? A Generalization Analysis of Nonlinear Transformers Hongkang Li Yihua Zhang Shuai Zhang M. Wang Sijia Liu Pin-Yu Chen MoMe 60 2 0 15 Apr 2025
Mimic In-Context Learning for Multimodal Tasks Yuchu Jiang Jiale Fu Chenduo Hao Xinting Hu Yingzhe Peng Xin Geng Xu Yang 27 0 0 11 Apr 2025
Steering off Course: Reliability Challenges in Steering Language Models Patrick Queiroz Da Silva Hari Sethuraman Dheeraj Rajagopal Hannaneh Hajishirzi Sachin Kumar LLMSV 29 1 0 06 Apr 2025
Contextualize-then-Aggregate: Circuits for In-Context Learning in Gemma-2 2B Aleksandra Bakalova Yana Veitsman Xinting Huang Michael Hahn 31 0 0 31 Mar 2025
Unlocking General Long Chain-of-Thought Reasoning Capabilities of Large Language Models via Representation Engineering Xinyu Tang Xiaolei Wang Zhihao Lv Yingqian Min Wayne Xin Zhao Binbin Hu Ziqi Liu Zhiqiang Zhang LRM 76 2 0 14 Mar 2025
Take Off the Training Wheels Progressive In-Context Learning for Effective Alignment Zhenyu Liu Dongfang Li Xinshuo Hu X. Zhao Yibin Chen Baotian Hu Min-Ling Zhang 46 1 0 13 Mar 2025
Representation Engineering for Large-Language Models: Survey and Research Challenges Lukasz Bartoszcze Sarthak Munshi Bryan Sukidi Jennifer Yen Zejia Yang David Williams-King Linh Le Kosi Asuzu Carsten Maple 100 0 0 24 Feb 2025
Activation Steering in Neural Theorem Provers Shashank Kirtania LLMSV 117 0 0 21 Feb 2025
Repetition Neurons: How Do Language Models Produce Repetitions? Tatsuya Hiraoka Kentaro Inui MILM 73 6 0 21 Feb 2025
Do we Really Need Visual Instructions? Towards Visual Instruction-Free Fine-tuning for Large Vision-Language Models Zikang Liu K. Zhou Wayne Xin Zhao Dawei Gao Yaliang Li Ji-Rong Wen MLLM VLM LRM 79 0 0 17 Feb 2025
Unlocking the Power of Function Vectors for Characterizing and Mitigating Catastrophic Forgetting in Continual Instruction Tuning Gangwei Jiang Caigao Jiang Zhaoyi Li Siqiao Xue Jun-ping Zhou Linqi Song Defu Lian Yin Wei CLL MU 56 0 0 16 Feb 2025
The Representation and Recall of Interwoven Structured Knowledge in LLMs: A Geometric and Layered Analysis Ge Lei Samuel J. Cooper KELM 47 0 0 15 Feb 2025
Superpose Singular Features for Model Merging Haiquan Qiu You Wu Quanming Yao MoMe 43 0 0 15 Feb 2025
Task-driven Layerwise Additive Activation Intervention Hieu Trung Nguyen Bao Nguyen Binh Nguyen V. Nguyen KELM 45 0 0 10 Feb 2025
FairDropout: Using Example-Tied Dropout to Enhance Generalization of Minority Groups Géraldin Nanfack Eugene Belilovsky 54 0 0 10 Feb 2025
Learning Task Representations from In-Context Learning Baturay Saglam Zhuoran Yang Dionysis Kalogerias Amin Karbasi 55 0 0 08 Feb 2025
Efficient Knowledge Feeding to Language Models: A Novel Integrated Encoder-Decoder Architecture Sachin Kumar Rishi Gottimukkala Supriya Devidutta K. Spindler RALM KELM 3DV 47 0 0 07 Feb 2025
Physics of Skill Learning Ziming Liu Yizhou Liu Eric J. Michaud Jeff Gore Max Tegmark 44 1 0 21 Jan 2025
Agent-Centric Projection of Prompting Techniques and Implications for Synthetic Training Data for Large Language Models Dhruv Dhamani Mary Lou Maher LLMAG 52 0 0 14 Jan 2025
ICLR: In-Context Learning of Representations Core Francisco Park Andrew Lee Ekdeep Singh Lubana Yongyi Yang Maya Okawa Kento Nishi Martin Wattenberg Hidenori Tanaka AIFin 114 3 0 29 Dec 2024
Sparse Attention Vectors: Generative Multimodal Model Features Are Discriminative Vision-Language Classifiers Chancharik Mitra Brandon Huang Tianning Chai Zhiqiu Lin Assaf Arbelle Rogerio Feris Leonid Karlinsky Trevor Darrell Deva Ramanan Roei Herzig VLM 121 4 0 28 Nov 2024
The Semantic Hub Hypothesis: Language Models Share Semantic Representations Across Languages and Modalities Zhaofeng Wu Xinyan Velocity Yu Dani Yogatama Jiasen Lu Yoon Kim AIFin 46 10 0 07 Nov 2024
Teaching Models to Improve on Tape L. Bezalel Eyal Orgad Amir Globerson 37 0 0 03 Nov 2024
Toward Understanding In-context vs. In-weight Learning Bryan Chan Xinyi Chen András Gyorgy Dale Schuurmans 65 3 0 30 Oct 2024
Chatting with Bots: AI, Speech Acts, and the Edge of Assertion Iwan Williams Tim Bayne 34 1 0 22 Oct 2024
Steering Knowledge Selection Behaviours in LLMs via SAE-Based Representation Engineering Yu Zhao Alessio Devoto Giwon Hong Xiaotang Du Aryo Pradipta Gema Hongru Wang Xuanli He Kam-Fai Wong Pasquale Minervini KELM LLMSV 32 16 0 21 Oct 2024
State-space models can learn in-context by gradient descent Neeraj Mohan Sushma Yudou Tian Harshvardhan Mestha Nicolo Colombo David Kappel Anand Subramoney 35 3 0 15 Oct 2024
Improving Instruction-Following in Language Models through Activation Steering Alessandro Stolfo Vidhisha Balachandran Safoora Yousefi Eric Horvitz Besmira Nushi LLMSV 52 14 0 15 Oct 2024
Divide, Reweight, and Conquer: A Logit Arithmetic Approach for In-Context Learning Chengsong Huang Langlin Huang Jiaxin Huang MoMe 27 1 0 14 Oct 2024
Inference and Verbalization Functions During In-Context Learning Junyi Tao Xiaoyin Chen Nelson F. Liu ReLM LRM 21 0 0 12 Oct 2024
ELICIT: LLM Augmentation via External In-Context Capability Futing Wang Jianhao Yan Yue Zhang Tao Lin 39 0 0 12 Oct 2024
The Geometry of Concepts: Sparse Autoencoder Feature Structure Yuxiao Li Eric J. Michaud David D. Baek Joshua Engels Xiaoqing Sun Max Tegmark 50 7 0 10 Oct 2024
Everything Everywhere All at Once: LLMs can In-Context Learn Multiple Tasks in Superposition Zheyang Xiong Ziyang Cai John Cooper Albert Ge Vasilis Papageorgiou ... Saurabh Agarwal Grigorios G Chrysos Samet Oymak Kangwook Lee Dimitris Papailiopoulos LRM 29 1 0 08 Oct 2024
How Language Models Prioritize Contextual Grammatical Cues? Hamidreza Amirzadeh A. Alishahi Hosein Mohebbi 21 0 0 04 Oct 2024
Disentangling Latent Shifts of In-Context Learning Through Self-Training Josip Jukić Jan Snajder 21 0 0 02 Oct 2024
Mitigating Copy Bias in In-Context Learning through Neuron Pruning Ameen Ali Lior Wolf Ivan Titov 27 2 0 02 Oct 2024
Optimal ablation for interpretability Maximilian Li Lucas Janson FAtt 44 2 0 16 Sep 2024
Attend First, Consolidate Later: On the Importance of Attention in Different LLM Layers Amit Ben Artzy Roy Schwartz 27 5 0 05 Sep 2024
Large Language Models Might Not Care What You Are Saying: Prompt Format Beats Descriptions Chenming Tang Zhixiang Wang Yunfang Wu LRM 21 0 0 16 Aug 2024
Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2 Tom Lieberum Senthooran Rajamanoharan Arthur Conmy Lewis Smith Nicolas Sonnerat Vikrant Varma János Kramár Anca Dragan Rohin Shah Neel Nanda 21 81 0 09 Aug 2024
Active Testing of Large Language Model via Multi-Stage Sampling Yuheng Huang Jiayang Song Qiang Hu Felix Juefei-Xu Lei Ma 21 2 0 07 Aug 2024
Multilingual Blending: LLM Safety Alignment Evaluation with Language Mixture Jiayang Song Yuheng Huang Zhehua Zhou Lei Ma 37 6 0 10 Jul 2024
A Practical Review of Mechanistic Interpretability for Transformer-Based Language Models Daking Rai Yilun Zhou Shi Feng Abulhair Saparov Ziyu Yao 75 19 0 02 Jul 2024
Deciphering the Factors Influencing the Efficacy of Chain-of-Thought: Probability, Memorization, and Noisy Reasoning Akshara Prabhakar Thomas L. Griffiths R. Thomas McCoy LRM 42 16 0 01 Jul 2024
The Remarkable Robustness of LLMs: Stages of Inference? Vedang Lad Wes Gurnee Max Tegmark 33 33 0 27 Jun 2024