TernaryBERT: Distillation-aware Ultra-low Bit BERT

27 September 2020

Lifeng Shang

Xin Jiang

Qun Liu

ArXiv PDF HTML

Papers citing "TernaryBERT: Distillation-aware Ultra-low Bit BERT"

32 / 32 papers shown

Title
COBRA: Algorithm-Architecture Co-optimized Binary Transformer Accelerator for Edge Inference Ye Qiao Zhiheng Cheng Yian Wang Yifan Zhang Yunzhe Deng Sitao Huang 77 0 0 22 Apr 2025
GSQ-Tuning: Group-Shared Exponents Integer in Fully Quantized Training for LLMs On-Device Fine-tuning Sifan Zhou Shuo Wang Zhihang Yuan Mingjia Shi Yuzhang Shang Dawei Yang ALM MQ 85 0 0 18 Feb 2025
BEEM: Boosting Performance of Early Exit DNNs using Multi-Exit Classifiers as Experts Divya J. Bajpai M. Hanawal 65 0 0 02 Feb 2025
HadamRNN: Binary and Sparse Ternary Orthogonal RNNs Armand Foucault Franck Mamalet François Malgouyres MQ 74 0 0 28 Jan 2025
Accurate Block Quantization in LLMs with Outliers Nikita Trukhanov I. Soloveychik MQ 26 3 0 29 Mar 2024
Self-Distilled Quantization: Achieving High Compression Rates in Transformer-Based Language Models James OÑeill Sourav Dutta VLM MQ 32 1 0 12 Jul 2023
LLM-QAT: Data-Free Quantization Aware Training for Large Language Models Zechun Liu Barlas Oğuz Changsheng Zhao Ernie Chang Pierre Stock Yashar Mehdad Yangyang Shi Raghuraman Krishnamoorthi Vikas Chandra MQ 46 187 0 29 May 2023
Efficient and Effective Methods for Mixed Precision Neural Network Quantization for Faster, Energy-efficient Inference Deepika Bablani J. McKinstry S. K. Esser R. Appuswamy D. Modha MQ 17 4 0 30 Jan 2023
NoisyQuant: Noisy Bias-Enhanced Post-Training Activation Quantization for Vision Transformers Yijiang Liu Huanrui Yang Zhen Dong Kurt Keutzer Li Du Shanghang Zhang MQ 29 45 0 29 Nov 2022
Can Open-Domain QA Reader Utilize External Knowledge Efficiently like Humans? Neeraj Varshney Man Luo Chitta Baral RALM 19 11 0 23 Nov 2022
SQuAT: Sharpness- and Quantization-Aware Training for BERT Zheng Wang Juncheng Billy Li Shuhui Qu Florian Metze Emma Strubell MQ 11 7 0 13 Oct 2022
Block Format Error Bounds and Optimal Block Size Selection I. Soloveychik I. Lyubomirsky Xin Eric Wang S. Bhoja MQ 27 4 0 11 Oct 2022
Outlier Suppression: Pushing the Limit of Low-bit Transformer Language Models Xiuying Wei Yunchen Zhang Xiangguo Zhang Ruihao Gong Shanghang Zhang Qi Zhang F. Yu Xianglong Liu MQ 22 145 0 27 Sep 2022
Efficient Methods for Natural Language Processing: A Survey Marcos Vinícius Treviso Ji-Ung Lee Tianchu Ji Betty van Aken Qingqing Cao ... Emma Strubell Niranjan Balasubramanian Leon Derczynski Iryna Gurevych Roy Schwartz 28 109 0 31 Aug 2022
A Length Adaptive Algorithm-Hardware Co-design of Transformer on FPGA Through Sparse Attention and Dynamic Pipelining Hongwu Peng Shaoyi Huang Shiyang Chen Bingbing Li Tong Geng ... Weiwen Jiang Wujie Wen J. Bi Hang Liu Caiwen Ding 45 54 0 07 Aug 2022
ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformers Z. Yao Reza Yazdani Aminabadi Minjia Zhang Xiaoxia Wu Conglong Li Yuxiong He VLM MQ 45 440 0 04 Jun 2022
Compression of Generative Pre-trained Language Models via Quantization Chaofan Tao Lu Hou Wei Zhang Lifeng Shang Xin Jiang Qun Liu Ping Luo Ngai Wong MQ 27 103 0 21 Mar 2022
VAQF: Fully Automatic Software-Hardware Co-Design Framework for Low-Bit Vision Transformer Mengshu Sun Haoyu Ma Guoliang Kang Yifan Jiang Tianlong Chen Xiaolong Ma Zhangyang Wang Yanzhi Wang ViT 25 45 0 17 Jan 2022
NN-LUT: Neural Approximation of Non-Linear Operations for Efficient Transformer Inference Joonsang Yu Junki Park Seongmin Park Minsoo Kim Sihwa Lee Dong Hyun Lee Jungwook Choi 27 48 0 03 Dec 2021
Prune Once for All: Sparse Pre-Trained Language Models Ofir Zafrir Ariel Larey Guy Boudoukh Haihao Shen Moshe Wasserblat VLM 25 82 0 10 Nov 2021
Kronecker Decomposition for GPT Compression Ali Edalati Marzieh S. Tahaei Ahmad Rashid V. Nia J. Clark Mehdi Rezagholizadeh 34 33 0 15 Oct 2021
MoEfication: Transformer Feed-forward Layers are Mixtures of Experts Zhengyan Zhang Yankai Lin Zhiyuan Liu Peng Li Maosong Sun Jie Zhou MoE 19 117 0 05 Oct 2021
Towards Efficient Post-training Quantization of Pre-trained Language Models Haoli Bai Lu Hou Lifeng Shang Xin Jiang Irwin King M. Lyu MQ 71 47 0 30 Sep 2021
Greenformers: Improving Computation and Memory Efficiency in Transformer Models via Low-Rank Approximation Samuel Cahyawijaya 26 12 0 24 Aug 2021
FLAT: An Optimized Dataflow for Mitigating Attention Bottlenecks Sheng-Chun Kao Suvinay Subramanian Gaurav Agrawal Amir Yazdanbakhsh T. Krishna 30 57 0 13 Jul 2021
Post-Training Quantization for Vision Transformer Zhenhua Liu Yunhe Wang Kai Han Siwei Ma Wen Gao ViT MQ 39 324 0 27 Jun 2021
Pre-Trained Models: Past, Present and Future Xu Han Zhengyan Zhang Ning Ding Yuxian Gu Xiao Liu ... Jie Tang Ji-Rong Wen Jinhui Yuan Wayne Xin Zhao Jun Zhu AIFin MQ AI4MH 37 813 0 14 Jun 2021
Scalable Vision Transformers with Hierarchical Pooling Zizheng Pan Bohan Zhuang Jing Liu Haoyu He Jianfei Cai ViT 25 126 0 19 Mar 2021
BinaryBERT: Pushing the Limit of BERT Quantization Haoli Bai Wei Zhang Lu Hou Lifeng Shang Jing Jin Xin Jiang Qun Liu Michael Lyu Irwin King MQ 140 221 0 31 Dec 2020
CascadeBERT: Accelerating Inference of Pre-trained Language Models via Calibrated Complete Models Cascade Lei Li Yankai Lin Deli Chen Shuhuai Ren Peng Li Jie Zhou Xu Sun 26 51 0 29 Dec 2020
Q-BERT: Hessian Based Ultra Low Precision Quantization of BERT Sheng Shen Zhen Dong Jiayu Ye Linjian Ma Z. Yao A. Gholami Michael W. Mahoney Kurt Keutzer MQ 225 575 0 12 Sep 2019
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 297 6,950 0 20 Apr 2018