Communities
Connect sessions
AI calendar
Organizations
Join Slack
Contact Sales
Search
Open menu
Home
Papers
2010.01791
Cited By
Pruning Redundant Mappings in Transformer Models via Spectral-Normalized Identity Prior
Findings (Findings), 2020
5 October 2020
Zi Lin
Jeremiah Zhe Liu
Ziao Yang
Nan Hua
Dan Roth
Re-assign community
ArXiv (abs)
PDF
HTML
Papers citing
"Pruning Redundant Mappings in Transformer Models via Spectral-Normalized Identity Prior"
22 / 22 papers shown
SparsyFed: Sparse Adaptive Federated Training
Adriano Guastella
Lorenzo Sani
Alex Iacob
Alessio Mora
Paolo Bellavista
Nicholas D. Lane
FedML
403
0
0
07 Apr 2025
STAT: Shrinking Transformers After Training
Megan Flynn
Alexander Wang
Dean Edward Alvarez
Christopher De Sa
Anil Damle
296
3
0
29 May 2024
The Need for Speed: Pruning Transformers with One Recipe
Samir Khaki
Konstantinos N. Plataniotis
351
14
0
26 Mar 2024
Model Compression and Efficient Inference for Large Language Models: A Survey
Wenxiao Wang
Wei Chen
Yicong Luo
Yongliu Long
Zhengkai Lin
Liye Zhang
Binbin Lin
Deng Cai
Xiaofei He
MQ
284
88
0
15 Feb 2024
A Comprehensive Survey of Compression Algorithms for Language Models
Seungcheol Park
Jaehyeon Choi
Sojin Lee
U. Kang
MQ
329
20
0
27 Jan 2024
Breaking through Deterministic Barriers: Randomized Pruning Mask Generation and Selection
Jianwei Li
Weizhi Gao
Qi Lei
Dongkuan Xu
344
3
0
19 Oct 2023
Sub-network Discovery and Soft-masking for Continual Learning of Mixed Tasks
Zixuan Ke
Bing Liu
Wenhan Xiong
Asli Celikyilmaz
Haoran Li
CLL
240
10
0
13 Oct 2023
Pit One Against Many: Leveraging Attention-head Embeddings for Parameter-efficient Multi-head Attention
Conference on Empirical Methods in Natural Language Processing (EMNLP), 2023
Huiyin Xue
Nikolaos Aletras
317
1
0
11 Oct 2023
S
P
3
\rm SP^3
S
P
3
: Enhancing Structured Pruning via PCA Projection
Annual Meeting of the Association for Computational Linguistics (ACL), 2023
Yuxuan Hu
Jing Zhang
Zhe Zhao
Chengliang Zhao
Xiaodong Chen
Cuiping Li
Hong Chen
245
3
0
31 Aug 2023
Accurate Retraining-free Pruning for Pretrained Encoder-based Language Models
International Conference on Learning Representations (ICLR), 2023
Seungcheol Park
Ho-Jin Choi
U. Kang
VLM
230
12
0
07 Aug 2023
Towards Learning Discrete Representations via Self-Supervision for Wearables-Based Human Activity Recognition
Italian National Conference on Sensors (INS), 2023
H. Haresamudram
Irfan Essa
Thomas Ploetz
233
11
0
01 Jun 2023
HighLight: Efficient and Flexible DNN Acceleration with Hierarchical Structured Sparsity
Yannan Nellie Wu
Po-An Tsai
Saurav Muralidharan
A. Parashar
Vivienne Sze
J. Emer
185
42
0
22 May 2023
Gradient-Free Structured Pruning with Unlabeled Data
International Conference on Machine Learning (ICML), 2023
Azade Nova
H. Dai
Dale Schuurmans
SyDa
262
34
0
07 Mar 2023
Learning a Consensus Sub-Network with Polarization Regularization and One Pass Training
Data mining and knowledge discovery (DMKD), 2023
Xiaoying Zhi
Varun Babbar
P. Sun
Fran Silavong
Ruibo Shi
Sean J. Moran
Sean Moran
443
1
0
17 Feb 2023
An Empirical Study on the Transferability of Transformer Modules in Parameter-Efficient Fine-Tuning
Conference on Empirical Methods in Natural Language Processing (EMNLP), 2023
Mohammad AkbarTajari
S. Rajaee
Mohammad Taher Pilehvar
228
2
0
01 Feb 2023
Adapting a Language Model While Preserving its General Knowledge
Conference on Empirical Methods in Natural Language Processing (EMNLP), 2023
Zixuan Ke
Yijia Shao
Haowei Lin
Hu Xu
Lei Shu
Yinan Han
KELM
CLL
VLM
177
26
0
21 Jan 2023
A Fast Post-Training Pruning Framework for Transformers
Neural Information Processing Systems (NeurIPS), 2022
Woosuk Kwon
Sehoon Kim
Michael W. Mahoney
Joseph Hassoun
Kurt Keutzer
A. Gholami
214
201
0
29 Mar 2022
A Survey on Model Compression and Acceleration for Pretrained Language Models
AAAI Conference on Artificial Intelligence (AAAI), 2022
Canwen Xu
Julian McAuley
354
84
0
15 Feb 2022
Vision Transformer Slimming: Multi-Dimension Searching in Continuous Optimization Space
Computer Vision and Pattern Recognition (CVPR), 2022
Arnav Chavan
Zhiqiang Shen
Zhuang Liu
Zechun Liu
Kwang-Ting Cheng
Eric P. Xing
ViT
269
79
0
03 Jan 2022
From Dense to Sparse: Contrastive Pruning for Better Pre-trained Language Model Compression
Runxin Xu
Fuli Luo
Chengyu Wang
Baobao Chang
Yanjie Liang
Songfang Huang
Fei Huang
VLM
90
31
0
14 Dec 2021
Learned Token Pruning for Transformers
Sehoon Kim
Sheng Shen
D. Thorsley
A. Gholami
Woosuk Kwon
Joseph Hassoun
Kurt Keutzer
339
192
0
02 Jul 2021
Compressing Large-Scale Transformer-Based Models: A Case Study on BERT
Transactions of the Association for Computational Linguistics (TACL), 2020
Prakhar Ganesh
Yao Chen
Xin Lou
Mohammad Ali Khan
Yifan Yang
Hassan Sajjad
Preslav Nakov
Deming Chen
Marianne Winslett
AI4CE
425
213
0
27 Feb 2020
1
