Length-Adaptive Transformer: Train Once with Length Drop, Use Anytime with Search

14 October 2020

Papers citing "Length-Adaptive Transformer: Train Once with Length Drop, Use Anytime with Search"

14 / 14 papers shown

Title
Saliency-driven Dynamic Token Pruning for Large Language Models Yao Tao Yehui Tang Yun Wang Mingjian Zhu Hailin Hu Yunhe Wang 34 0 0 06 Apr 2025
Beyond Matryoshka: Revisiting Sparse Coding for Adaptive Representation Tiansheng Wen Yifei Wang Zequn Zeng Zhong Peng Yudi Su Xinyang Liu Bo Chen Hongwei Liu Stefanie Jegelka Chenyu You CLL 60 2 0 03 Mar 2025
CipherPrune: Efficient and Scalable Private Transformer Inference Yancheng Zhang J. Xue Mengxin Zheng Mimi Xie Mingzhe Zhang Lei Jiang Qian Lou 48 2 0 24 Feb 2025
Training Noise Token Pruning Mingxing Rao Bohan Jiang Daniel Moyer ViT 72 0 0 27 Nov 2024
LLMLingua: Compressing Prompts for Accelerated Inference of Large Language Models Huiqiang Jiang Qianhui Wu Chin-Yew Lin Yuqing Yang Lili Qiu 18 100 0 09 Oct 2023
Zero-TPrune: Zero-Shot Token Pruning through Leveraging of the Attention Graph in Pre-Trained Transformers Hongjie Wang Bhishma Dedhia N. Jha ViT VLM 28 26 0 27 May 2023
Do We Really Need a Large Number of Visual Prompts? Youngeun Kim Yuhang Li Abhishek Moitra Ruokai Yin Priyadarshini Panda VLM VPVLM 34 5 0 26 May 2023
SpanDrop: Simple and Effective Counterfactual Learning for Long Sequences Peng Qi Guangtao Wang Jing Huang 6 0 0 03 Aug 2022
Pyramid-BERT: Reducing Complexity via Successive Core-set based Token Selection Xin Huang A. Khetan Rene Bidart Zohar S. Karnin 17 14 0 27 Mar 2022
Shortformer: Better Language Modeling using Shorter Inputs Ofir Press Noah A. Smith M. Lewis 219 88 0 31 Dec 2020
Big Bird: Transformers for Longer Sequences Manzil Zaheer Guru Guruganesh Kumar Avinava Dubey Joshua Ainslie Chris Alberti ... Philip Pham Anirudh Ravula Qifan Wang Li Yang Amr Ahmed VLM 249 2,009 0 28 Jul 2020
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 226 4,424 0 23 Jan 2020
Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism M. Shoeybi M. Patwary Raul Puri P. LeGresley Jared Casper Bryan Catanzaro MoE 243 1,815 0 17 Sep 2019
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 294 6,943 0 20 Apr 2018