ROSITA: Refined BERT cOmpreSsion with InTegrAted techniques

ROSITA: Refined BERT cOmpreSsion with InTegrAted techniques

21 March 2021

Zheng Lin

ArXiv (abs)PDF HTML Github (14★)

Papers citing "ROSITA: Refined BERT cOmpreSsion with InTegrAted techniques"

13 / 13 papers shown

Title
Light-PEFT: Lightening Parameter-Efficient Fine-Tuning via Early Pruning Naibin Gu Peng Fu Xiyu Liu Bowen Shen Zheng Lin Weiping Wang 69 10 0 06 Jun 2024
STAT: Shrinking Transformers After Training Megan Flynn Alexander Wang Dean Edward Alvarez Christopher De Sa Anil Damle 79 2 0 29 May 2024
$$\rm SP^3$: Enhancing Structured Pruning via PCA Projection$ $\rm SP^3$ : Enhancing Structured Pruning via PCA Projection Yuxuan Hu Jing Zhang Zhe Zhao Chengliang Zhao Xiaodong Chen Cuiping Li Hong Chen 63 3 0 31 Aug 2023
Deep Model Compression Also Helps Models Capture Ambiguity Hancheol Park Jong C. Park 65 2 0 12 Jun 2023
X-Pruner: eXplainable Pruning for Vision Transformers Lu Yu Wei Xiang ViT 60 50 0 08 Mar 2023
Gradient-Free Structured Pruning with Unlabeled Data Azade Nova H. Dai Dale Schuurmans SyDa 86 22 0 07 Mar 2023
HEAT: Hardware-Efficient Automatic Tensor Decomposition for Transformer Compression Jiaqi Gu Ben Keller Jean Kossaifi Anima Anandkumar Brucek Khailany David Z. Pan ViT 66 9 0 30 Nov 2022
Compressing And Debiasing Vision-Language Pre-Trained Models for Visual Question Answering Q. Si Yuanxin Liu Zheng Lin Peng Fu Weiping Wang VLM 120 1 0 26 Oct 2022
A Win-win Deal: Towards Sparse and Robust Pre-trained Language Models Yuanxin Liu Fandong Meng Zheng Lin JiangNan Li Peng Fu Yanan Cao Weiping Wang Jie Zhou 87 6 0 11 Oct 2022
Task-specific Compression for Multi-task Language Models using Attribution-based Pruning Nakyeong Yang Yunah Jang Hwanhee Lee Seohyeong Jung Kyomin Jung 28 9 0 09 May 2022
A Fast Post-Training Pruning Framework for Transformers Woosuk Kwon Sehoon Kim Michael W. Mahoney Joseph Hassoun Kurt Keutzer A. Gholami 113 157 0 29 Mar 2022
BERMo: What can BERT learn from ELMo? Sangamesh Kodge Kaushik Roy 65 3 0 18 Oct 2021
Marginal Utility Diminishes: Exploring the Minimum Knowledge for BERT Knowledge Distillation Yuanxin Liu Fandong Meng Zheng Lin Weiping Wang Jie Zhou 33 6 0 10 Jun 2021