Dissecting Recall of Factual Associations in Auto-Regressive Language Models

28 April 2023

Papers citing "Dissecting Recall of Factual Associations in Auto-Regressive Language Models"

39 / 39 papers shown

Title
Bigram Subnetworks: Mapping to Next Tokens in Transformer Language Models Tyler A. Chang Benjamin Bergen 38 0 0 21 Apr 2025
HyperDAS: Towards Automating Mechanistic Interpretability with Hypernetworks Jiuding Sun Jing Huang Sidharth Baskaran Karel DÓosterlinck Christopher Potts Michael Sklar Atticus Geiger AI4CE 55 0 0 13 Mar 2025
Model Lakes Koyena Pal David Bau Renée J. Miller 60 0 0 24 Feb 2025
Building Bridges, Not Walls -- Advancing Interpretability by Unifying Feature, Data, and Model Component Attribution Shichang Zhang Tessa Han Usha Bhalla Hima Lakkaraju FAtt 143 0 0 17 Feb 2025
ReLearn: Unlearning via Learning for Large Language Models Haoming Xu Ningyuan Zhao Liming Yang Sendong Zhao Shumin Deng Mengru Wang Bryan Hooi Nay Oo H. Chen N. Zhang KELM CLL MU 51 0 0 16 Feb 2025
ConTrans: Weak-to-Strong Alignment Engineering via Concept Transplantation Weilong Dong Xinwei Wu Renren Jin Shaoyang Xu Deyi Xiong 45 6 0 31 Dec 2024
Devils in Middle Layers of Large Vision-Language Models: Interpreting, Detecting and Mitigating Object Hallucinations via Attention Lens Zhangqi Jiang Junkai Chen Beier Zhu Tingjin Luo Yankun Shen Xu Yang 77 4 0 23 Nov 2024
Understanding Multimodal LLMs: the Mechanistic Interpretability of Llava in Visual Question Answering Zeping Yu Sophia Ananiadou 38 0 0 17 Nov 2024
Fact Recall, Heuristics or Pure Guesswork? Precise Interpretations of Language Models for Fact Completion Denitsa Saynova Lovisa Hagström Moa Johansson Richard Johansson Marco Kuhlmann HILM 25 0 0 18 Oct 2024
The Geometry of Numerical Reasoning: Language Models Compare Numeric Properties in Linear Subspaces Ahmed Oumar El-Shangiti Tatsuya Hiraoka Hilal AlQuabeh Benjamin Heinzerling Kentaro Inui 26 1 0 17 Oct 2024
ChroKnowledge: Unveiling Chronological Knowledge of Language Models in Multiple Domains Yein Park Chanwoong Yoon Jungwoo Park Donghyeon Lee Minbyul Jeong Jaewoo Kang KELM 38 1 0 13 Oct 2024
Towards Interpreting Visual Information Processing in Vision-Language Models Clement Neo Luke Ong Philip H. S. Torr Mor Geva David M. Krueger Fazl Barez 78 6 0 09 Oct 2024
From Tokens to Words: On the Inner Lexicon of LLMs Guy Kaplan Matanel Oren Yuval Reif Roy Schwartz 32 12 0 08 Oct 2024
Erasing Conceptual Knowledge from Language Models Rohit Gandikota Sheridan Feucht Samuel Marks David Bau KELM ELM MU 37 5 0 03 Oct 2024
Geometric Signatures of Compositionality Across a Language Model's Lifetime Jin Hwa Lee Thomas Jiralerspong Lei Yu Yoshua Bengio Emily Cheng CoGe 75 0 0 02 Oct 2024
Knowledge Entropy Decay during Language Model Pretraining Hinders New Knowledge Acquisition Jiyeon Kim Hyunji Lee Hyowon Cho Joel Jang Hyeonbin Hwang Seungpil Won Youbin Ahn Dohaeng Lee Minjoon Seo KELM 36 2 0 02 Oct 2024
Personality Alignment of Large Language Models Minjun Zhu Linyi Yang Yue Zhang Yue Zhang ALM 39 5 0 21 Aug 2024
Knowledge in Superposition: Unveiling the Failures of Lifelong Knowledge Editing for Large Language Models Chenhui Hu Pengfei Cao Yubo Chen Kang Liu Jun Zhao KELM 44 2 0 14 Aug 2024
A Practical Review of Mechanistic Interpretability for Transformer-Based Language Models Daking Rai Yilun Zhou Shi Feng Abulhair Saparov Ziyu Yao 49 18 0 02 Jul 2024
Beyond Individual Facts: Investigating Categorical Knowledge Locality of Taxonomy and Meronomy Concepts in GPT Models Christopher Burger Yifan Hu Thai Le KELM 18 0 0 22 Jun 2024
How Do Large Language Models Acquire Factual Knowledge During Pretraining? Hoyeon Chang Jinho Park Seonghyeon Ye Sohee Yang Youngkyung Seo Du-Seong Chang Minjoon Seo KELM 18 30 0 17 Jun 2024
Knowledge Circuits in Pretrained Transformers Yunzhi Yao Ningyu Zhang Zekun Xi Meng Wang Ziwen Xu Shumin Deng Huajun Chen KELM 45 19 0 28 May 2024
Emergence of a High-Dimensional Abstraction Phase in Language Transformers Emily Cheng Diego Doimo Corentin Kervadec Iuri Macocco Jade Yu A. Laio Marco Baroni 101 11 0 24 May 2024
HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Models Bernal Jiménez Gutiérrez Yiheng Shu Yu Gu Michihiro Yasunaga Yu-Chuan Su RALM CLL 38 27 0 23 May 2024
Investigating Continual Pretraining in Large Language Models: Insights and Implications cCaugatay Yildiz Nishaanth Kanna Ravichandran Prishruit Punia Matthias Bethge B. Ermiş CLL KELM LRM 28 23 0 27 Feb 2024
Cracking Factual Knowledge: A Comprehensive Analysis of Degenerate Knowledge Neurons in Large Language Models Yuheng Chen Pengfei Cao Yubo Chen Yining Wang Shengping Liu Kang Liu Jun Zhao KELM 14 1 0 21 Feb 2024
Black-Box Access is Insufficient for Rigorous AI Audits Stephen Casper Carson Ezell Charlotte Siegmann Noam Kolt Taylor Lynn Curtis ... Michael Gerovitch David Bau Max Tegmark David M. Krueger Dylan Hadfield-Menell AAML 6 75 0 25 Jan 2024
Forbidden Facts: An Investigation of Competing Objectives in Llama-2 Tony T. Wang Miles Wang Kaivu Hariharan Nir Shavit 8 2 0 14 Dec 2023
A Glitch in the Matrix? Locating and Detecting Language Model Grounding with Fakepedia Giovanni Monea Maxime Peyrard Martin Josifoski Vishrav Chaudhary Jason Eisner Emre Kiciman Hamid Palangi Barun Patra Robert West KELM 44 12 0 04 Dec 2023
Identifying Linear Relational Concepts in Large Language Models David Chanin Anthony Hunter Oana-Maria Camburu LLMSV KELM 8 4 0 15 Nov 2023
Towards a Mechanistic Interpretation of Multi-Step Reasoning Capabilities of Language Models Yifan Hou Jiaoda Li Yu Fei Alessandro Stolfo Wangchunshu Zhou Guangtao Zeng Antoine Bosselut Mrinmaya Sachan LRM 17 39 0 23 Oct 2023
Towards Best Practices of Activation Patching in Language Models: Metrics and Methods Fred Zhang Neel Nanda LLMSV 8 95 0 27 Sep 2023
Evaluating the Ripple Effects of Knowledge Editing in Language Models Roi Cohen Eden Biran Ori Yoran Amir Globerson Mor Geva KELM 11 155 0 24 Jul 2023
Quantifying Context Mixing in Transformers Hosein Mohebbi Willem H. Zuidema Grzegorz Chrupała A. Alishahi 161 24 0 30 Jan 2023
Crawling the Internal Knowledge-Base of Language Models Roi Cohen Mor Geva Jonathan Berant Amir Globerson 170 74 0 30 Jan 2023
Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small Kevin Wang Alexandre Variengien Arthur Conmy Buck Shlegeris Jacob Steinhardt 205 486 0 01 Nov 2022
Fast Model Editing at Scale E. Mitchell Charles Lin Antoine Bosselut Chelsea Finn Christopher D. Manning KELM 217 254 0 21 Oct 2021
The Bottom-up Evolution of Representations in the Transformer: A Study with Machine Translation and Language Modeling Objectives Elena Voita Rico Sennrich Ivan Titov 179 181 0 03 Sep 2019
Language Models as Knowledge Bases? Fabio Petroni Tim Rocktaschel Patrick Lewis A. Bakhtin Yuxiang Wu Alexander H. Miller Sebastian Riedel KELM AI4MH 391 2,216 0 03 Sep 2019