Your Transformer is Secretly Linear

19 May 2024

Papers citing "Your Transformer is Secretly Linear"

7 / 7 papers shown

Title
How Redundant Is the Transformer Stack in Speech Representation Models? Teresa Dorszewski Albert Kjøller Jacobsen Lenka Tětková Lars Kai Hansen 104 0 0 20 Jan 2025
Graph-Aware Isomorphic Attention for Adaptive Dynamics in Transformers Markus J. Buehler AI4CE 35 1 0 04 Jan 2025
Everybody Prune Now: Structured Pruning of LLMs with only Forward Passes Lucio Dery Steven Kolawole Jean-Francois Kagey Virginia Smith Graham Neubig Ameet Talwalkar 39 27 0 08 Feb 2024
MetaOptimize: A Framework for Optimizing Step Sizes and Other Meta-parameters Arsalan Sharifnassab Saber Salehkaleybar Richard Sutton 14 3 0 04 Feb 2024
SliceGPT: Compress Large Language Models by Deleting Rows and Columns Saleh Ashkboos Maximilian L. Croci Marcelo Gennari do Nascimento Torsten Hoefler James Hensman VLM 125 143 0 26 Jan 2024
Toy Models of Superposition Nelson Elhage Tristan Hume Catherine Olsson Nicholas Schiefer T. Henighan ... Sam McCandlish Jared Kaplan Dario Amodei Martin Wattenberg C. Olah AAML MILM 120 314 0 21 Sep 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 303 11,730 0 04 Mar 2022