Knowledge Circuits in Pretrained Transformers

Knowledge Circuits in Pretrained Transformers

28 May 2024

Ningyu Zhang

Shumin Deng

Huajun Chen

Papers citing "Knowledge Circuits in Pretrained Transformers"

17 / 17 papers shown

Title
SetKE: Knowledge Editing for Knowledge Elements Overlap Yifan Wei Xiaoyan Yu Ran Song Hao Peng Angsheng Li KELM 51 0 0 29 Apr 2025
Landscape of Thoughts: Visualizing the Reasoning Process of Large Language Models Zhanke Zhou Zhaocheng Zhu Xuan Li Mikhail Galkin Xiao Feng Sanmi Koyejo Jian Tang Bo Han LRM 36 0 0 28 Mar 2025
Capability Localization: Capabilities Can be Localized rather than Individual Knowledge Xiusheng Huang Jiaxiang Liu Yequan Wang Jun Zhao Kang Liu 49 0 0 28 Feb 2025
Rethinking LLM Unlearning Objectives: A Gradient Perspective and Go Beyond Qizhou Wang Jin Peng Zhou Zhanke Zhou Saebyeol Shin Bo Han Kilian Q. Weinberger AILaw ELM MU 57 3 0 26 Feb 2025
Quantifying Logical Consistency in Transformers via Query-Key Alignment Eduard Tulchinskii Anastasia Voznyuk Laida Kushnareva Andrei Andriiainen Irina Piontkovskaya Evgeny Burnaev Serguei Barannikov LRM 59 0 0 24 Feb 2025
Revealing and Mitigating Over-Attention in Knowledge Editing Pinzheng Wang Zecheng Tang Keyan Zhou J. Li Qiaoming Zhu M. Zhang KELM 99 2 0 21 Feb 2025
How Do LLMs Acquire New Knowledge? A Knowledge Circuits Perspective on Continual Pre-Training Yixin Ou Yunzhi Yao N. Zhang Hui Jin Jiacheng Sun Shumin Deng Z. Li H. Chen KELM CLL 46 0 0 16 Feb 2025
Knowledge Entropy Decay during Language Model Pretraining Hinders New Knowledge Acquisition Jiyeon Kim Hyunji Lee Hyowon Cho Joel Jang Hyeonbin Hwang Seungpil Won Youbin Ahn Dohaeng Lee Minjoon Seo KELM 31 2 0 02 Oct 2024
Attention Heads of Large Language Models: A Survey Zifan Zheng Yezhaohui Wang Yuxin Huang Shichao Song Mingchuan Yang Bo Tang Feiyu Xiong Zhiyu Li LRM 41 1 0 05 Sep 2024
Knowledge Mechanisms in Large Language Models: A Survey and Perspective Meng Wang Yunzhi Yao Ziwen Xu Shuofei Qiao Shumin Deng ... Yong-jia Jiang Pengjun Xie Fei Huang Huajun Chen Ningyu Zhang 34 1 0 22 Jul 2024
Cutting Off the Head Ends the Conflict: A Mechanism for Interpreting and Mitigating Knowledge Conflicts in Language Models Zhuoran Jin Pengfei Cao Hongbang Yuan Yubo Chen Jiexin Xu Huaijun Li Xiaojian Jiang Kang Liu Jun Zhao 178 7 0 28 Feb 2024
Information Flow Routes: Automatically Interpreting Language Models at Scale Javier Ferrando Elena Voita 34 10 0 27 Feb 2024
Dictionary Learning Improves Patch-Free Circuit Discovery in Mechanistic Interpretability: A Case Study on Othello-GPT Zhengfu He Xuyang Ge Qiong Tang Tianxiang Sun Qinyuan Cheng Xipeng Qiu 26 8 0 19 Feb 2024
Attribution Patching Outperforms Automated Circuit Discovery Aaquib Syed Can Rager Arthur Conmy 50 18 0 16 Oct 2023
Dissecting Recall of Factual Associations in Auto-Regressive Language Models Mor Geva Jasmijn Bastings Katja Filippova Amir Globerson KELM 180 152 0 28 Apr 2023
Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small Kevin Wang Alexandre Variengien Arthur Conmy Buck Shlegeris Jacob Steinhardt 205 486 0 01 Nov 2022
Toy Models of Superposition Nelson Elhage Tristan Hume Catherine Olsson Nicholas Schiefer T. Henighan ... Sam McCandlish Jared Kaplan Dario Amodei Martin Wattenberg C. Olah AAML MILM 117 183 0 21 Sep 2022