Finding Skill Neurons in Pre-trained Transformer-based Language Models

14 November 2022

Xiaozhi Wang

Kaiyue Wen

Zhengyan Zhang

Lei Hou

Zhiyuan Liu

Juanzi Li

MILM

MoE

ArXiv PDF HTML

Papers citing "Finding Skill Neurons in Pre-trained Transformer-based Language Models"

50 / 50 papers shown

Title
Model Utility Law: Evaluating LLMs beyond Performance through Mechanism Interpretable Metric Yixin Cao Jiahao Ying Y. Wang Xipeng Qiu Xuanjing Huang Yugang Jiang ELM 30 2 0 10 Apr 2025
Neuron-level Balance between Stability and Plasticity in Deep Reinforcement Learning Jiahua Lan Sen Zhang Haixia Pan Ruijun Liu Li Shen Dacheng Tao CLL 28 0 0 09 Apr 2025
M2IV: Towards Efficient and Fine-grained Multimodal In-Context Learning in Large Vision-Language Models Yanshu Li Hongyang He Yi Cao Qisen Cheng Xiang Fu Ruixiang Tang VLM 40 0 0 06 Apr 2025
Triple Phase Transitions: Understanding the Learning Dynamics of Large Language Models from a Neuroscience Perspective Yuko Nakagi Keigo Tada Sota Yoshino Shinji Nishimoto Yu Takagi LRM 37 0 0 28 Feb 2025
From Text to Space: Mapping Abstract Spatial Models in LLMs during a Grid-World Navigation Task Nicolas Martorell LLMAG 50 1 0 23 Feb 2025
Concept Corrector: Erase concepts on the fly for text-to-image diffusion models Zheling Meng Bo Peng Xiaochuan Jin Yueming Lyu Wei Wang Jing Dong DiffM 38 2 0 22 Feb 2025
Brain-Inspired Exploration of Functional Networks and Key Neurons in Large Language Models Yiheng Liu Xiaohui Gao Haiyang Sun Bao Ge Tianming Liu Junwei Han X. Hu 36 0 0 13 Feb 2025
Learning Task Representations from In-Context Learning Baturay Saglam Zhuoran Yang Dionysis Kalogerias Amin Karbasi 55 0 0 08 Feb 2025
Neuron-Level Sequential Editing for Large Language Models Houcheng Jiang Junfeng Fang Tianyu Zhang An Zhang Ruipeng Wang Tao Liang Xiang Wang KELM 43 4 0 05 Oct 2024
Exploring the Benefit of Activation Sparsity in Pre-training Zhengyan Zhang Chaojun Xiao Qiujieli Qin Yankai Lin Zhiyuan Zeng Xu Han Zhiyuan Liu Ruobing Xie Maosong Sun Jie Zhou MoE 58 3 0 04 Oct 2024
Property Neurons in Self-Supervised Speech Transformers T. Lin Guan-Ting Lin Hung-yi Lee Hao Tang MILM 25 0 0 07 Sep 2024
A Practical Review of Mechanistic Interpretability for Transformer-Based Language Models Daking Rai Yilun Zhou Shi Feng Abulhair Saparov Ziyu Yao 75 19 0 02 Jul 2024
Memorizing Documents with Guidance in Large Language Models Bumjin Park Jaesik Choi KELM RALM 34 1 0 23 Jun 2024
Pre-trained Large Language Models Use Fourier Features to Compute Addition Tianyi Zhou Deqing Fu Vatsal Sharan Robin Jia LRM 34 9 0 05 Jun 2024
ConceptPrune: Concept Editing in Diffusion Models via Skilled Neuron Pruning Ruchika Chavhan Da Li Timothy M. Hospedales 36 15 0 29 May 2024
Large Language Model Bias Mitigation from the Perspective of Knowledge Editing Ruizhe Chen Yichen Li Zikai Xiao Zuo-Qiang Liu KELM 36 13 0 15 May 2024
Binary Hypothesis Testing for Softmax Models and Leverage Score Models Yeqi Gao Yuzhou Gu Zhao-quan Song 33 0 0 09 May 2024
Assessing the Brittleness of Safety Alignment via Pruning and Low-Rank Modifications Boyi Wei Kaixuan Huang Yangsibo Huang Tinghao Xie Xiangyu Qi Mengzhou Xia Prateek Mittal Mengdi Wang Peter Henderson AAML 55 79 0 07 Feb 2024
ReLU $^2$ Wins: Discovering Efficient Activation Functions for Sparse LLMs Zhengyan Zhang Yixin Song Guanghui Yu Xu Han Yankai Lin Chaojun Xiao Chenyang Song Zhiyuan Liu Zeyu Mi Maosong Sun 20 31 0 06 Feb 2024
APT: Adaptive Pruning and Tuning Pretrained Language Models for Efficient Training and Inference Bowen Zhao Hannaneh Hajishirzi Qingqing Cao 21 17 0 22 Jan 2024
Universal Neurons in GPT2 Language Models Wes Gurnee Theo Horsley Zifan Carl Guo Tara Rezaei Kheirkhah Qinyi Sun Will Hathaway Neel Nanda Dimitris Bertsimas MILM 92 37 0 22 Jan 2024
A Comprehensive Study of Knowledge Editing for Large Language Models Ningyu Zhang Yunzhi Yao Bo Tian Peng Wang Shumin Deng ... Lei Liang Zhiqiang Zhang Xiao-Jun Zhu Jun Zhou Huajun Chen KELM 26 76 0 02 Jan 2024
Towards Vision Enhancing LLMs: Empowering Multimodal Knowledge Storage and Sharing in LLMs Yunxin Li Baotian Hu Wei Wang Xiaochun Cao Min Zhang 14 4 0 27 Nov 2023
The Expressibility of Polynomial based Attention Scheme Zhao-quan Song Guangyi Xu Junze Yin 27 5 0 30 Oct 2023
Towards a fuller understanding of neurons with Clustered Compositional Explanations Biagio La Rosa Leilani H. Gilpin Roberto Capobianco 22 2 0 27 Oct 2023
CRaSh: Clustering, Removing, and Sharing Enhance Fine-tuning without Full Large Language Model Kaiyan Zhang Ning Ding Biqing Qi Xuekai Zhu Xinwei Long Bowen Zhou 38 4 0 24 Oct 2023
Function Vectors in Large Language Models Eric Todd Millicent Li Arnab Sen Sharma Aaron Mueller Byron C. Wallace David Bau 8 99 0 23 Oct 2023
The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets Samuel Marks Max Tegmark HILM 91 168 0 10 Oct 2023
Establishing Trustworthiness: Rethinking Tasks and Model Evaluation Robert Litschko Max Müller-Eberstein Rob van der Goot Leon Weber Barbara Plank LRM 16 2 0 09 Oct 2023
On the Relationship between Skill Neurons and Robustness in Prompt Tuning Leon Ackermann Xenia Ohmer AAML 16 0 0 21 Sep 2023
A Fast Optimization View: Reformulating Single Layer Attention in LLM Based on Tensor and SVM Trick, and Solving It in Matrix Multiplication Time Yeqi Gao Zhao-quan Song Weixin Wang Junze Yin 18 25 0 14 Sep 2023
GradientCoin: A Peer-to-Peer Decentralized Large Language Models Yeqi Gao Zhao-quan Song Junze Yin 21 18 0 21 Aug 2023
Soft Merging of Experts with Adaptive Routing Mohammed Muqeeth Haokun Liu Colin Raffel MoMe MoE 24 45 0 06 Jun 2023
Examining the Causal Effect of First Names on Language Models: The Case of Social Commonsense Reasoning Sullam Jeoung Jana Diesner H. Kilicoglu LRM 11 5 0 01 Jun 2023
Emergent Modularity in Pre-trained Transformers Zhengyan Zhang Zhiyuan Zeng Yankai Lin Chaojun Xiao Xiaozhi Wang Xu Han Zhiyuan Liu Ruobing Xie Maosong Sun Jie Zhou MoE 37 23 0 28 May 2023
Editing Large Language Models: Problems, Methods, and Opportunities Yunzhi Yao Peng Wang Bo Tian Shuyang Cheng Zhoubo Li Shumin Deng Huajun Chen Ningyu Zhang KELM 30 278 0 22 May 2023
Knowledge Rumination for Pre-trained Language Models Yunzhi Yao Peng Wang Shengyu Mao Chuanqi Tan Fei Huang Huajun Chen Ningyu Zhang KELM 22 3 0 15 May 2023
Finding Neurons in a Haystack: Case Studies with Sparse Probing Wes Gurnee Neel Nanda Matthew Pauly Katherine Harvey Dmitrii Troitskii Dimitris Bertsimas MILM 153 186 0 02 May 2023
Language-Specific Representation of Emotion-Concept Knowledge Causally Supports Emotion Inference Ming Li Yusheng Su Hsiu-Yuan Huang Jiali Cheng Xin Hu ... Yujia Qin Xiaozhi Wang Kristen A. Lindquist Zhi-Yun Liu Dan Zhang 24 5 0 19 Feb 2023
Task-Specific Skill Localization in Fine-tuned Language Models A. Panigrahi Nikunj Saunshi Haoyu Zhao Sanjeev Arora MoMe 21 66 0 13 Feb 2023
Evaluating Neuron Interpretation Methods of NLP Models Yimin Fan Fahim Dalvi Nadir Durrani Hassan Sajjad 35 8 0 30 Jan 2023
Does Localization Inform Editing? Surprising Differences in Causality-Based Localization vs. Knowledge Editing in Language Models Peter Hase Mohit Bansal Been Kim Asma Ghandeharioun MILM 29 167 0 10 Jan 2023
Reasoning with Language Model Prompting: A Survey Shuofei Qiao Yixin Ou Ningyu Zhang Xiang Chen Yunzhi Yao Shumin Deng Chuanqi Tan Fei Huang Huajun Chen ReLM ELM LRM 51 310 0 19 Dec 2022
Fast Model Editing at Scale E. Mitchell Charles Lin Antoine Bosselut Chelsea Finn Christopher D. Manning KELM 219 341 0 21 Oct 2021
SPoT: Better Frozen Model Adaptation through Soft Prompt Transfer Tu Vu Brian Lester Noah Constant Rami Al-Rfou Daniel Matthew Cer VLM LRM 137 277 0 15 Oct 2021
P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Tasks Xiao Liu Kaixuan Ji Yicheng Fu Weng Lam Tam Zhengxiao Du Zhilin Yang Jie Tang VLM 236 805 0 14 Oct 2021
The Power of Scale for Parameter-Efficient Prompt Tuning Brian Lester Rami Al-Rfou Noah Constant VPVLM 280 3,843 0 18 Apr 2021
Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inference Timo Schick Hinrich Schütze 258 1,586 0 21 Jan 2020
Language Models as Knowledge Bases? Fabio Petroni Tim Rocktaschel Patrick Lewis A. Bakhtin Yuxiang Wu Alexander H. Miller Sebastian Riedel KELM AI4MH 406 2,584 0 03 Sep 2019
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 294 6,943 0 20 Apr 2018