A Short Study on Compressing Decoder-Based Language Models

A Short Study on Compressing Decoder-Based Language Models

16 October 2021

Yassir El Mesbahi

Habib Hajimolahoseini

Yang Liu

Mehdi Rezagholizadeh

Papers citing "A Short Study on Compressing Decoder-Based Language Models"

9 / 9 papers shown

Title
Merino: Entropy-driven Design for Generative Language Models on IoT Devices Youpeng Zhao Ming Lin Huadong Tang Qiang Wu Jun Wang 73 0 0 28 Jan 2025
IDTraffickers: An Authorship Attribution Dataset to link and connect Potential Human-Trafficking Operations on Text Escort Advertisements V. Saxena Benjamin Bashpole Gijs Van Dijck Gerasimos Spanakis 28 2 0 09 Oct 2023
Training Acceleration of Low-Rank Decomposed Networks using Sequential Freezing and Rank Quantization Habib Hajimolahoseini Walid Ahmed Yang Liu OffRL MQ 11 6 0 07 Sep 2023
Learn What Is Possible, Then Choose What Is Best: Disentangling One-To-Many Relations in Language Through Text-based Games Benjamin Towle Ke Zhou SyDa 12 4 0 14 Apr 2023
What Matters In The Structured Pruning of Generative Language Models? Michael Santacroce Zixin Wen Yelong Shen Yuan-Fang Li 10 32 0 07 Feb 2023
Kronecker Decomposition for GPT Compression Ali Edalati Marzieh S. Tahaei Ahmad Rashid V. Nia J. Clark Mehdi Rezagholizadeh 28 33 0 15 Oct 2021
ZeRO-Offload: Democratizing Billion-Scale Model Training Jie Ren Samyam Rajbhandari Reza Yazdani Aminabadi Olatunji Ruwase Shuangyang Yang Minjia Zhang Dong Li Yuxiong He MoE 155 399 0 18 Jan 2021
BERT-of-Theseus: Compressing BERT by Progressive Module Replacing Canwen Xu Wangchunshu Zhou Tao Ge Furu Wei Ming Zhou 210 196 0 07 Feb 2020
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 294 6,927 0 20 Apr 2018