Jump to Conclusions: Short-Cutting Transformers With Linear Transformations

16 March 2023

Papers citing "Jump to Conclusions: Short-Cutting Transformers With Linear Transformations"

50 / 55 papers shown

Title
One Jump Is All You Need: Short-Cutting Transformers for Early Exit Prediction with One Jump to Fit All Exit Levels Amrit Diggavi Seshadri BDL 26 0 0 18 Apr 2025
Language Models, Graph Searching, and Supervision Adulteration: When More Supervision is Less and How to Make More More Arvid Frydenlund LRM 48 0 0 13 Mar 2025
LLM-Microscope: Uncovering the Hidden Role of Punctuation in Context Memory of Transformers Anton Razzhigaev Matvey Mikhalchuk Temurbek Rahmatullaev Elizaveta Goncharova Polina Druzhinina Ivan V. Oseledets Andrey Kuznetsov 57 1 0 20 Feb 2025
Discovering Chunks in Neural Embeddings for Interpretability Shuchen Wu Stephan Alaniz Eric Schulz Zeynep Akata 42 0 0 03 Feb 2025
GREEN-CODE: Learning to Optimize Energy Efficiency in LLM-based Code Generation Shashikant Ilager Lukas Florian Briem Ivona Brandić 29 0 0 19 Jan 2025
Look Ahead Text Understanding and LLM Stitching Junlin Julian Jiang Xin Li 39 2 0 16 Dec 2024
Towards Unifying Interpretability and Control: Evaluation via Intervention Usha Bhalla Suraj Srinivas Asma Ghandeharioun Himabindu Lakkaraju 38 5 0 07 Nov 2024
From Tokens to Words: On the Inner Lexicon of LLMs Guy Kaplan Matanel Oren Yuval Reif Roy Schwartz 41 12 0 08 Oct 2024
Mitigating Copy Bias in In-Context Learning through Neuron Pruning Ameen Ali Lior Wolf Ivan Titov 27 2 0 02 Oct 2024
Normalized Narrow Jump To Conclusions: Normalized Narrow Shortcuts for Parameter Efficient Early Exit Transformer Prediction Amrit Diggavi Seshadri 14 1 0 21 Sep 2024
Optimal ablation for interpretability Maximilian Li Lucas Janson FAtt 44 2 0 16 Sep 2024
Multimodal Contrastive In-Context Learning Yosuke Miyanishi Minh Le Nguyen 32 2 0 23 Aug 2024
Attention Is All You Need But You Don't Need All Of It For Inference of Large Language Models Georgy Tyukin G. Dovonon Jean Kaddour Pasquale Minervini LRM 28 0 0 22 Jul 2024
Knowledge Mechanisms in Large Language Models: A Survey and Perspective Meng Wang Yunzhi Yao Ziwen Xu Shuofei Qiao Shumin Deng ... Yong-jia Jiang Pengjun Xie Fei Huang Huajun Chen Ningyu Zhang 47 27 0 22 Jul 2024
A Practical Review of Mechanistic Interpretability for Transformer-Based Language Models Daking Rai Yilun Zhou Shi Feng Abulhair Saparov Ziyu Yao 75 18 0 02 Jul 2024
Who's asking? User personas and the mechanics of latent misalignment Asma Ghandeharioun Ann Yuan Marius Guerard Emily Reif Michael A. Lepori Lucas Dixon LLMSV 41 7 0 17 Jun 2024
Self-Control of LLM Behaviors by Compressing Suffix Gradient into Prefix Controller Min Cai Yuchen Zhang Shichang Zhang Fan Yin Difan Zou Yisong Yue Ziniu Hu 21 0 0 04 Jun 2024
Evidence of Learned Look-Ahead in a Chess-Playing Neural Network Erik Jenner Shreyas Kapur Vasil Georgiev Cameron Allen Scott Emmons Stuart J. Russell 26 10 0 02 Jun 2024
Your Transformer is Secretly Linear Anton Razzhigaev Matvey Mikhalchuk Elizaveta Goncharova Nikolai Gerasimenko Ivan V. Oseledets Denis Dimitrov Andrey Kuznetsov 27 4 0 19 May 2024
LayerSkip: Enabling Early Exit Inference and Self-Speculative Decoding Mostafa Elhoushi Akshat Shrivastava Diana Liskovich Basil Hosmer Bram Wasti ... Saurabh Agarwal Ahmed Roman Ahmed Aly Beidi Chen Carole-Jean Wu LRM 33 82 0 25 Apr 2024
Mechanistic Interpretability for AI Safety -- A Review Leonard Bereska E. Gavves AI4CE 38 111 0 22 Apr 2024
LM Transparency Tool: Interactive Tool for Analyzing Transformer Language Models Igor Tufanov Karen Hambardzumyan Javier Ferrando Elena Voita KELM 28 6 0 10 Apr 2024
CQIL: Inference Latency Optimization with Concurrent Computation of Quasi-Independent Layers Longwei Zou Qingyang Wang Han Zhao Jiangang Kong Yi Yang Yangdong Deng 19 0 0 10 Apr 2024
The Unreasonable Ineffectiveness of the Deeper Layers Andrey Gromov Kushal Tirumala Hassan Shapourian Paolo Glorioso Daniel A. Roberts 41 79 0 26 Mar 2024
Diffusion Lens: Interpreting Text Encoders in Text-to-Image Pipelines Michael Toker Hadas Orgad Mor Ventura Dana Arad Yonatan Belinkov DiffM 58 12 0 09 Mar 2024
Do Large Language Models Latently Perform Multi-Hop Reasoning? Sohee Yang E. Gribovskaya Nora Kassner Mor Geva Sebastian Riedel ReLM LRM 40 75 0 26 Feb 2024
LLM Inference Unveiled: Survey and Roofline Model Insights Zhihang Yuan Yuzhang Shang Yang Zhou Zhen Dong Zhe Zhou ... Yong Jae Lee Yan Yan Beidi Chen Guangyu Sun Kurt Keutzer 37 79 0 26 Feb 2024
The Hidden Space of Transformer Language Adapters Jesujoba Oluwadara Alabi Marius Mosbach Matan Eyal Dietrich Klakow Mor Geva 48 7 1 20 Feb 2024
When Only Time Will Tell: Interpreting How Transformers Process Local Ambiguities Through the Lens of Restart-Incrementality Brielen Madureira Patrick Kahardipraja David Schlangen 31 2 0 20 Feb 2024
Backward Lens: Projecting Language Model Gradients into the Vocabulary Space Shahar Katz Yonatan Belinkov Mor Geva Lior Wolf 47 10 1 20 Feb 2024
LaCo: Large Language Model Pruning via Layer Collapse Yifei Yang Zouying Cao Hai Zhao 10 52 0 17 Feb 2024
Do LLMs Know about Hallucination? An Empirical Investigation of LLM's Hidden States Hanyu Duan Yi Yang K. Tam HILM 19 27 0 15 Feb 2024
Spectral Filters, Dark Signals, and Attention Sinks Nicola Cancedda 56 16 0 14 Feb 2024
SLEB: Streamlining LLMs through Redundancy Verification and Elimination of Transformer Blocks Jiwon Song Kyungseok Oh Taesu Kim Hyungjun Kim Yulhwa Kim Jae-Joon Kim 64 20 0 14 Feb 2024
From Understanding to Utilization: A Survey on Explainability for Large Language Models Haoyan Luo Lucia Specia 29 21 0 23 Jan 2024
Patchscopes: A Unifying Framework for Inspecting Hidden Representations of Language Models Asma Ghandeharioun Avi Caciularu Adam Pearce Lucas Dixon Mor Geva 25 87 0 11 Jan 2024
Weight subcloning: direct initialization of transformers using larger pretrained ones Mohammad Samragh Mehrdad Farajtabar Sachin Mehta Raviteja Vemulapalli Fartash Faghri Devang Naik Oncel Tuzel Mohammad Rastegari 16 25 0 14 Dec 2023
EE-LLM: Large-Scale Training and Inference of Early-Exit Large Language Models with 3D Parallelism Yanxi Chen Xuchen Pan Yaliang Li Bolin Ding Jingren Zhou LRM 15 31 0 08 Dec 2023
In-context Learning and Gradient Descent Revisited Gilad Deutch Nadav Magar Tomer Bar Natan Guy Dar 22 7 0 13 Nov 2023
Future Lens: Anticipating Subsequent Tokens from a Single Hidden State Koyena Pal Jiuding Sun Andrew Yuan Byron C. Wallace David Bau 22 50 0 08 Nov 2023
Analyzing Vision Transformers for Image Classification in Class Embedding Space Martina G. Vilas Timothy Schaumlöffel Gemma Roig ViT 14 23 0 29 Oct 2023
The Curious Case of Hallucinatory (Un)answerability: Finding Truths in the Hidden States of Over-Confident Large Language Models Aviv Slobodkin Omer Goldman Avi Caciularu Ido Dagan Shauli Ravfogel HILM LRM 45 24 0 18 Oct 2023
DecoderLens: Layerwise Interpretation of Encoder-Decoder Transformers Anna Langedijk Hosein Mohebbi Gabriele Sarti Willem H. Zuidema Jaap Jumelet 19 10 0 05 Oct 2023
Parameter and Computation Efficient Transfer Learning for Vision-Language Pre-trained Models Qiong Wu Wei Yu Yiyi Zhou Shubin Huang Xiaoshuai Sun R. Ji VLM 16 7 0 04 Sep 2023
Linearity of Relation Decoding in Transformer Language Models Evan Hernandez Arnab Sen Sharma Tal Haklay Kevin Meng Martin Wattenberg Jacob Andreas Yonatan Belinkov David Bau KELM 11 82 0 17 Aug 2023
Does Circuit Analysis Interpretability Scale? Evidence from Multiple Choice Capabilities in Chinchilla Tom Lieberum Matthew Rahtz János Kramár Neel Nanda G. Irving Rohin Shah Vladimir Mikulik 15 99 0 18 Jul 2023
A Mechanistic Interpretation of Arithmetic Reasoning in Language Models using Causal Mediation Analysis Alessandro Stolfo Yonatan Belinkov Mrinmaya Sachan MILM KELM LRM 27 47 0 24 May 2023
VISIT: Visualizing and Interpreting the Semantic Information Flow of Transformers Shahar Katz Yonatan Belinkov 34 26 0 22 May 2023
Explaining How Transformers Use Context to Build Predictions Javier Ferrando Gerard I. Gállego Ioannis Tsiamas Marta R. Costa-jussá 18 31 0 21 May 2023
The Benefits of Bad Advice: Autocontrastive Decoding across Model Layers Ariel Gera Roni Friedman Ofir Arviv Chulaka Gunasekara Benjamin Sznajder Noam Slonim Eyal Shnarch 38 19 0 02 May 2023