Accelerating Transformer Inference for Translation via Parallel Decoding

17 May 2023

Valentino Maiorca

Papers citing "Accelerating Transformer Inference for Translation via Parallel Decoding"

50 / 73 papers shown

Title
Jupiter: Fast and Resource-Efficient Collaborative Inference of Generative LLMs on Edge Devices Shengyuan Ye Bei Ouyang Liekang Zeng Tianyi Qian Xiaowen Chu Jian Tang Xu Chen 29 0 0 11 Apr 2025
FastVAR: Linear Visual Autoregressive Modeling via Cached Token Pruning Hang Guo Yawei Li Taolin Zhang J. Wang Tao Dai Shu-Tao Xia Luca Benini 67 1 0 30 Mar 2025
Accelerate Parallelizable Reasoning via Parallel Decoding within One Sequence Yijiong Yu LRM AIMat 90 1 0 26 Mar 2025
Collaborative Speculative Inference for Efficient LLM Inference Serving Luyao Gao Jianchun Liu Hongli Xu Xichong Zhang Yunming Liao Liusheng Huang 41 0 0 13 Mar 2025
Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Models Marianne Arriola Aaron Gokaslan Justin T Chiu Zhihan Yang Zhixuan Qi Jiaqi Han S. Sahoo Volodymyr Kuleshov DiffM 67 4 0 12 Mar 2025
DiffPO: Diffusion-styled Preference Optimization for Efficient Inference-Time Alignment of Large Language Models Ruizhe Chen Wenhao Chai Zhifei Yang Xiaotian Zhang Joey Tianyi Zhou Tony Q. S. Quek Soujanya Poria Zuozhu Liu 48 0 0 06 Mar 2025
Accelerating Vision-Language-Action Model Integrated with Action Chunking via Parallel Decoding Wenxuan Song Jiayi Chen Pengxiang Ding H. Zhao Wei Zhao Zhide Zhong Zongyuan Ge Jun Ma Haoang Li 43 3 0 04 Mar 2025
DuoDecoding: Hardware-aware Heterogeneous Speculative Decoding with Dynamic Multi-Sequence Drafting Kai Lv Honglin Guo Qipeng Guo Xipeng Qiu 41 0 0 02 Mar 2025
Tutorial Proposal: Speculative Decoding for Efficient LLM Inference Heming Xia Cunxiao Du Y. Li Qian Liu Wenjie Li 34 0 0 01 Mar 2025
Fuzzy Speculative Decoding for a Tunable Accuracy-Runtime Tradeoff Maximilian Holsman Yukun Huang Bhuwan Dhingra 33 0 0 28 Feb 2025
Speculative Decoding and Beyond: An In-Depth Survey of Techniques Y. Hu Zining Liu Zhenyuan Dong Tianfan Peng Bradley McDanel S. Zhang 88 0 0 27 Feb 2025
Learning to Keep a Promise: Scaling Language Model Decoding Parallelism with Learned Asynchronous Decoding Tian Jin Ellie Y. Cheng Zack Ankner Nikunj Saunshi Blake M. Elias Amir Yazdanbakhsh Jonathan Ragan-Kelley Suvinay Subramanian Michael Carbin 52 2 0 24 Feb 2025
KVLink: Accelerating Large Language Models via Efficient KV Cache Reuse Jingbo Yang Bairu Hou Wei Wei Yujia Bao Shiyu Chang VLM 41 2 0 21 Feb 2025
Acceleration Multiple Heads Decoding for LLM via Dynamic Tree Attention Zhendong Zhang 56 0 0 09 Feb 2025
Lossless Acceleration of Large Language Models with Hierarchical Drafting based on Temporal Locality in Speculative Decoding Sukmin Cho S. Choi T. Hwang Jeongyeon Seo Soyeong Jeong Huije Lee Hoyun Song Jong C. Park Youngjin Kwon 51 0 0 08 Feb 2025
From Cool Demos to Production-Ready FMware: Core Challenges and a Technology Roadmap Gopi Krishnan Rajbahadur G. Oliva Dayi Lin Ahmed E. Hassan 41 1 0 28 Jan 2025
Towards Scalable and Stable Parallelization of Nonlinear RNNs Xavier Gonzalez Andrew Warrington Jimmy T.H. Smith Scott W. Linderman 83 8 0 17 Jan 2025
PLD+: Accelerating LLM inference by leveraging Language Model Artifacts Shwetha Somasundaram Anirudh Phukan Apoorv Saxena 77 1 0 02 Dec 2024
The N-Grammys: Accelerating Autoregressive Inference with Learning-Free Batched Speculation Lawrence Stewart Matthew Trager Sujan Kumar Gonugondla Stefano Soatto 45 5 0 06 Nov 2024
A Theoretical Perspective for Speculative Decoding Algorithm Ming Yin Minshuo Chen Kaixuan Huang Mengdi Wang 32 0 0 30 Oct 2024
Faster Language Models with Better Multi-Token Prediction Using Tensor Decomposition Artem Basharin Andrei Chertkov Ivan V. Oseledets 36 1 0 23 Oct 2024
Remote Timing Attacks on Efficient Language Model Inference Nicholas Carlini Milad Nasr 14 2 0 22 Oct 2024
COrAL: Order-Agnostic Language Modeling for Efficient Iterative Refinement Yuxi Xie Anirudh Goyal Xiaobao Wu Xunjian Yin Xiao Xu Min-Yen Kan Liangming Pan William Yang Wang LRM 66 1 0 12 Oct 2024
SWIFT: On-the-Fly Self-Speculative Decoding for LLM Inference Acceleration Heming Xia Yongqi Li Jun Zhang Cunxiao Du Wenjie Li LRM 46 5 0 09 Oct 2024
ParallelSpec: Parallel Drafter for Efficient Speculative Decoding Zilin Xiao Hongming Zhang Tao Ge Siru Ouyang Vicente Ordonez Dong Yu 39 5 0 08 Oct 2024
RevMUX: Data Multiplexing with Reversible Adapters for Efficient LLM Batch Inference Yige Xu Xu Guo Zhiwei Zeng Chunyan Miao 34 0 0 06 Oct 2024
Accelerating Auto-regressive Text-to-Image Generation with Training-free Speculative Jacobi Decoding Yao Teng Han Shi Xian Liu Xuefei Ning Guohao Dai Yu Wang Zhenguo Li Xihui Liu 48 10 0 02 Oct 2024
Closed-Loop Long-Horizon Robotic Planning via Equilibrium Sequence Modeling Jinghan Li Zhicheng Sun Fei Li 88 1 0 02 Oct 2024
KOALA: Enhancing Speculative Decoding for LLM via Multi-Layer Draft Heads with Adversarial Learning Kaiqi Zhang Jing Zhao Rui Chen 34 1 0 15 Aug 2024
Efficiency Unleashed: Inference Acceleration for LLM-based Recommender Systems with Speculative Decoding Yunjia Xi Hangyu Wang Bo Chen Jianghao Lin Menghui Zhu W. Liu Ruiming Tang Zhewei Wei W. Zhang Yong Yu OffRL 90 4 0 11 Aug 2024
Inference Optimization of Foundation Models on AI Accelerators Youngsuk Park Kailash Budhathoki Liangfu Chen Jonas M. Kübler Jiaji Huang Matthäus Kleindessner Jun Huan V. Cevher Yida Wang George Karypis 37 3 0 12 Jul 2024
Make Some Noise: Unlocking Language Model Parallel Inference Capability through Noisy Training Yixuan Wang Xianzhen Luo Fuxuan Wei Yijun Liu Qingfu Zhu Xuanyu Zhang Qing Yang Dongliang Xu Wanxiang Che 35 3 0 25 Jun 2024
EAGLE-2: Faster Inference of Language Models with Dynamic Draft Trees Yuhui Li Fangyun Wei Chao Zhang Hongyang R. Zhang 88 51 0 24 Jun 2024
New Solutions on LLM Acceleration, Optimization, and Application Yingbing Huang Lily Jiaxin Wan Hanchen Ye Manvi Jha Jinghua Wang Yuhong Li Xiaofan Zhang Deming Chen 37 12 0 16 Jun 2024
GRAM: Generative Retrieval Augmented Matching of Data Schemas in the Context of Data Security Xuanqing Liu Luyang Kong Runhui Wang Patrick Song Austin Nevins Henrik Johnson Nimish Amlathe Davor Golac 34 2 0 04 Jun 2024
S3D: A Simple and Cost-Effective Self-Speculative Decoding Scheme for Low-Memory GPUs Wei Zhong Manasa Bharadwaj 33 5 0 30 May 2024
SpecDec++: Boosting Speculative Decoding via Adaptive Candidate Lengths Kaixuan Huang Xudong Guo Mengdi Wang 32 18 0 30 May 2024
Hardware-Aware Parallel Prompt Decoding for Memory-Efficient Acceleration of LLM Inference Hao Chen Wayne Luk Ka-Fai Cedric Yiu Rui Li Konstantin Mishchenko Stylianos I. Venieris Hongxiang Fan 34 7 0 28 May 2024
A Comprehensive Survey of Accelerated Generation Techniques in Large Language Models Mahsa Khoshnoodi Vinija Jain Mingye Gao Malavika Srikanth Aman Chadha OffRL 28 1 0 15 May 2024
TriForce: Lossless Acceleration of Long Sequence Generation with Hierarchical Speculative Decoding Hanshi Sun Zhuoming Chen Xinyu Yang Yuandong Tian Beidi Chen 38 46 0 18 Apr 2024
On Speculative Decoding for Multimodal Large Language Models Mukul Gagrani Raghavv Goel Wonseok Jeon Junyoung Park Mingu Lee Christopher Lott LRM 32 7 0 13 Apr 2024
SDSAT: Accelerating LLM Inference through Speculative Decoding with Semantic Adaptive Tokens Chengbo Liu Yong Zhu 23 0 0 27 Mar 2024
Efficient Encoder-Decoder Transformer Decoding for Decomposable Tasks Bo-Ru Lu Nikita Haduong Chien-Yu Lin Hao Cheng Noah A. Smith Mari Ostendorf AI4CE 32 0 0 19 Mar 2024
Recurrent Drafter for Fast Speculative Decoding in Large Language Models Aonan Zhang Chong-Jun Wang Yi Wang Xuanyu Zhang Yunfei Cheng 26 15 0 14 Mar 2024
CLLMs: Consistency Large Language Models Siqi Kou Lanxiang Hu Zhe He Zhijie Deng Hao Zhang 39 27 0 28 Feb 2024
Think Big, Generate Quick: LLM-to-SLM for Fast Autoregressive Decoding Benjamin Bergner Andrii Skliar Amelie Royer Tijmen Blankevoort Yuki Markus Asano B. Bejnordi 58 5 0 26 Feb 2024
LLM Inference Unveiled: Survey and Roofline Model Insights Zhihang Yuan Yuzhang Shang Yang Zhou Zhen Dong Zhe Zhou ... Yong Jae Lee Yan Yan Beidi Chen Guangyu Sun Kurt Keutzer 37 79 0 26 Feb 2024
Chimera: A Lossless Decoding Method for Accelerating Large Language Models Inference by Fusing all Tokens Ziqian Zeng Jiahong Yu Qianshi Pang Zihao W. Wang Huiping Zhuang Cen Chen Xiaofeng Zou 26 4 0 24 Feb 2024
Ouroboros: Generating Longer Drafts Phrase by Phrase for Faster Speculative Decoding Weilin Zhao Yuxiang Huang Xu Han Wang Xu Chaojun Xiao Xinrong Zhang Yewei Fang Kaihuo Zhang Zhiyuan Liu Maosong Sun 35 11 0 21 Feb 2024
ProPD: Dynamic Token Tree Pruning and Generation for LLM Parallel Decoding Shuzhang Zhong Zebin Yang Meng Li Ruihao Gong Runsheng Wang Ru Huang 32 6 0 21 Feb 2024