Linearity of Relation Decoding in Transformer Language Models

17 August 2023

Papers citing "Linearity of Relation Decoding in Transformer Language Models"

50 / 76 papers shown

Title
Llama See, Llama Do: A Mechanistic Perspective on Contextual Entrainment and Distraction in LLMs Jingcheng Niu Xingdi Yuan Tong Wang Hamidreza Saghir Amir H. Abdi 11 0 0 14 May 2025
Exploring How LLMs Capture and Represent Domain-Specific Knowledge Mirian Hipolito Garcia Camille Couturier Daniel Madrigal Diaz Ankur Mallick Anastasios Kyrillidis Robert Sim Victor Rühle Saravan Rajmohan 25 0 0 23 Apr 2025
Do Large Language Models know who did what to whom? Joseph M. Denning Xiaohan Bryor Snefjella Idan A. Blank 50 1 0 23 Apr 2025
Functional Abstraction of Knowledge Recall in Large Language Models Zijian Wang Chang Xu KELM 32 0 0 20 Apr 2025
On Linear Representations and Pretraining Data Frequency in Language Models Jack Merullo Noah A. Smith Sarah Wiegreffe Yanai Elazar 35 0 0 16 Apr 2025
Lost in Multilinguality: Dissecting Cross-lingual Factual Inconsistency in Transformer Language Models Mingyang Wang Heike Adel Lukas Lange Yihong Liu Ercong Nie Jannik Strötgen Hinrich Schütze HILM 56 0 0 05 Apr 2025
Page Classification for Print Imaging Pipeline Shaoyuan Xu Cheng Lu Mark Shaw Peter Bauer J. Allebach VLM 35 1 0 03 Apr 2025
The Representation and Recall of Interwoven Structured Knowledge in LLMs: A Geometric and Layered Analysis Ge Lei Samuel J. Cooper KELM 47 0 0 15 Feb 2025
Learning Task Representations from In-Context Learning Baturay Saglam Zhuoran Yang Dionysis Kalogerias Amin Karbasi 55 0 0 08 Feb 2025
ConTrans: Weak-to-Strong Alignment Engineering via Concept Transplantation Weilong Dong Xinwei Wu Renren Jin Shaoyang Xu Deyi Xiong 54 6 0 31 Dec 2024
Towards Unifying Interpretability and Control: Evaluation via Intervention Usha Bhalla Suraj Srinivas Asma Ghandeharioun Himabindu Lakkaraju 38 5 0 07 Nov 2024
All or None: Identifiable Linear Properties of Next-token Predictors in Language Modeling Emanuele Marconato Sébastien Lachapelle Sebastian Weichwald Luigi Gresele 64 3 0 30 Oct 2024
Identifying Sub-networks in Neural Networks via Functionally Similar Representations Tian Gao Amit Dhurandhar K. Ramamurthy Dennis L. Wei 43 0 0 21 Oct 2024
The Geometry of Numerical Reasoning: Language Models Compare Numeric Properties in Linear Subspaces Ahmed Oumar El-Shangiti Tatsuya Hiraoka Hilal AlQuabeh Benjamin Heinzerling Kentaro Inui 34 1 0 17 Oct 2024
Safety-Aware Fine-Tuning of Large Language Models Hyeong Kyu Choi Xuefeng Du Yixuan Li 35 11 0 13 Oct 2024
Generalization from Starvation: Hints of Universality in LLM Knowledge Graph Learning David D. Baek Yuxiao Li Max Tegmark 44 2 0 10 Oct 2024
Adaptive Large Language Models By Layerwise Attention Shortcuts Prateek Verma Mert Pilanci KELM OffRL 42 0 0 17 Sep 2024
Optimal ablation for interpretability Maximilian Li Lucas Janson FAtt 44 2 0 16 Sep 2024
Questioning Internal Knowledge Structure of Large Language Models Through the Lens of the Olympic Games Juhwan Choi Youngbin Kim 38 0 0 10 Sep 2024
Representational Analysis of Binding in Language Models Qin Dai Benjamin Heinzerling Kentaro Inui 29 0 0 09 Sep 2024
Residual Stream Analysis with Multi-Layer SAEs Tim Lawson Lucy Farnik Conor Houghton Laurence Aitchison 24 3 0 06 Sep 2024
Attention Heads of Large Language Models: A Survey Zifan Zheng Yezhaohui Wang Yuxin Huang Shichao Song Mingchuan Yang Bo Tang Feiyu Xiong Zhiyu Li LRM 52 21 0 05 Sep 2024
Relation Also Knows: Rethinking the Recall and Editing of Factual Associations in Auto-Regressive Transformer Language Models Xiyu Liu Zhengxiao Liu Naibin Gu Zheng-Shen Lin Wanli Ma Ji Xiang Weiping Wang KELM 44 0 0 27 Aug 2024
Relational Composition in Neural Networks: A Survey and Call to Action Martin Wattenberg Fernanda Viégas CoGe 36 9 0 19 Jul 2024
NNsight and NDIF: Democratizing Access to Open-Weight Foundation Model Internals Jaden Fiotto-Kaufman Alexander R. Loftus Eric Todd Jannik Brinkmann Caden Juang ... Carla Brodley Arjun Guha Jonathan Bell Byron C. Wallace David Bau 24 2 0 18 Jul 2024
States Hidden in Hidden States: LLMs Emerge Discrete State Representations Implicitly Junhao Chen Shengding Hu Zhiyuan Liu Maosong Sun LRM 32 5 0 16 Jul 2024
Functional Faithfulness in the Wild: Circuit Discovery with Differentiable Computation Graph Pruning Lei Yu Jingcheng Niu Zining Zhu Gerald Penn 31 5 0 04 Jul 2024
A Practical Review of Mechanistic Interpretability for Transformer-Based Language Models Daking Rai Yilun Zhou Shi Feng Abulhair Saparov Ziyu Yao 75 18 0 02 Jul 2024
Monitoring Latent World States in Language Models with Propositional Probes Jiahai Feng Stuart Russell Jacob Steinhardt HILM 32 6 0 27 Jun 2024
Transformer Normalisation Layers and the Independence of Semantic Subspaces S. Menary Samuel Kaski Andre Freitas 41 2 0 25 Jun 2024
Locating and Extracting Relational Concepts in Large Language Models Zijian Wang Britney White Chang Xu KELM 38 0 0 19 Jun 2024
Estimating Knowledge in Large Language Models Without Generating a Single Token Daniela Gottesman Mor Geva 37 10 0 18 Jun 2024
Who's asking? User personas and the mechanics of latent misalignment Asma Ghandeharioun Ann Yuan Marius Guerard Emily Reif Michael A. Lepori Lucas Dixon LLMSV 41 7 0 17 Jun 2024
Breaking the Attention Bottleneck Kalle Hilsenbek 81 0 0 16 Jun 2024
Self-Control of LLM Behaviors by Compressing Suffix Gradient into Prefix Controller Min Cai Yuchen Zhang Shichang Zhang Fan Yin Difan Zou Yisong Yue Ziniu Hu 21 0 0 04 Jun 2024
Editing the Mind of Giants: An In-Depth Exploration of Pitfalls of Knowledge Editing in Large Language Models Cheng-Hsun Hsueh Paul Kuo-Ming Huang Tzu-Han Lin Che-Wei Liao Hung-Chieh Fang Chao-Wei Huang Yun-Nung Chen KELM 31 5 0 03 Jun 2024
Knowledge Circuits in Pretrained Transformers Yunzhi Yao Ningyu Zhang Zekun Xi Meng Wang Ziwen Xu Shumin Deng Huajun Chen KELM 64 19 0 28 May 2024
Exploring the LLM Journey from Cognition to Expression with Linear Representations Yuzi Yan J. Li Yipin Zhang Dong Yan 41 1 0 27 May 2024
Implicit In-context Learning Zhuowei Li Zihao Xu Ligong Han Yunhe Gao Song Wen Di Liu Hao Wang Dimitris N. Metaxas 38 1 0 23 May 2024
Poser: Unmasking Alignment Faking LLMs by Manipulating Their Internals Joshua Clymer Caden Juang Severin Field CVBM 32 1 0 08 May 2024
Position: Understanding LLMs Requires More Than Statistical Generalization Patrik Reizinger Szilvia Ujváry Anna Mészáros A. Kerekes Wieland Brendel Ferenc Huszár 36 12 0 03 May 2024
Mechanistic Interpretability for AI Safety -- A Review Leonard Bereska E. Gavves AI4CE 38 111 0 22 Apr 2024
In-Context Learning State Vector with Inner and Momentum Optimization Dongfang Li Zhenyu Liu Xinshuo Hu Zetian Sun Baotian Hu Min Zhang 24 5 0 17 Apr 2024
Evidence from counterfactual tasks supports emergent analogical reasoning in large language models Taylor W. Webb K. Holyoak Hongjing Lu LRM ELM 33 4 0 14 Apr 2024
Locating and Editing Factual Associations in Mamba Arnab Sen Sharma David Atkinson David Bau KELM 68 28 0 04 Apr 2024
Unveiling LLMs: The Evolution of Latent Representations in a Temporal Knowledge Graph Marco Bronzini Carlo Nicolini Bruno Lepri Jacopo Staiano Andrea Passerini KELM 28 0 0 04 Apr 2024
On Large Language Models' Hallucination with Regard to Known Facts Che Jiang Biqing Qi Xiangyu Hong Dayuan Fu Yang Cheng Fandong Meng Mo Yu Bowen Zhou Jie Zhou HILM LRM 31 16 0 29 Mar 2024
Mechanistic Understanding and Mitigation of Language Model Non-Factual Hallucinations Lei Yu Meng Cao Jackie Chi Kit Cheung Yue Dong HILM 33 7 0 27 Mar 2024
Embedded Named Entity Recognition using Probing Classifiers Nicholas Popovic Michael Färber 40 1 0 18 Mar 2024
SelfIE: Self-Interpretation of Large Language Model Embeddings Haozhe Chen Carl Vondrick Chengzhi Mao 19 17 0 16 Mar 2024