Communities
Connect sessions
AI calendar
Organizations
Join Slack
Contact Sales
Search
Open menu
Home
Papers
2001.04246
Cited By
v1
v2 (latest)
AdaBERT: Task-Adaptive BERT Compression with Differentiable Neural Architecture Search
International Joint Conference on Artificial Intelligence (IJCAI), 2020
13 January 2020
Daoyuan Chen
Yaliang Li
Minghui Qiu
Zhen Wang
Bofang Li
Bolin Ding
Hongbo Deng
Yanjie Liang
Jialin Li
Jingren Zhou
MQ
Re-assign community
ArXiv (abs)
PDF
HTML
Papers citing
"AdaBERT: Task-Adaptive BERT Compression with Differentiable Neural Architecture Search"
50 / 62 papers shown
Elastic Architecture Search for Efficient Language Models
IEEE International Conference on Multimedia and Expo (ICME), 2025
Shang Wang
KELM
172
0
0
30 Oct 2025
EvoPress: Accurate Dynamic Model Compression via Evolutionary Search
Oliver Sieberling
Denis Kuznedelev
Eldar Kurtic
Dan Alistarh
MQ
561
5
0
18 Oct 2024
Automatic Pruning of Fine-tuning Datasets for Transformer-based Language Models
Mohammadreza Tayaranian
S. H. Mozafari
Brett H. Meyer
J. Clark
Warren J. Gross
178
1
0
11 Jul 2024
DIR-BHRNet: A Lightweight Network for Real-time Vision-based Multi-person Pose Estimation on Smartphones
Gongjin Lan
Yu Wu
Qi Hao
3DH
283
12
0
01 Jul 2024
Structural Pruning of Pre-trained Language Models via Neural Architecture Search
Aaron Klein
Jacek Golebiowski
Xingchen Ma
Valerio Perrone
Cédric Archambeau
233
6
0
03 May 2024
Model Compression and Efficient Inference for Large Language Models: A Survey
Wenxiao Wang
Wei Chen
Yicong Luo
Yongliu Long
Zhengkai Lin
Liye Zhang
Binbin Lin
Deng Cai
Xiaofei He
MQ
378
95
0
15 Feb 2024
A Comprehensive Survey of Compression Algorithms for Language Models
Seungcheol Park
Jaehyeon Choi
Sojin Lee
U. Kang
MQ
390
22
0
27 Jan 2024
Vesper: A Compact and Effective Pretrained Model for Speech Emotion Recognition
IEEE Transactions on Affective Computing (IEEE Trans. Affective Comput.), 2023
Weidong Chen
Xiaofen Xing
Peihao Chen
Xiangmin Xu
VLM
345
75
0
20 Jul 2023
A Survey of Techniques for Optimizing Transformer Inference
Journal of systems architecture (JSA), 2023
Krishna Teja Chitty-Venkata
Sparsh Mittal
M. Emani
V. Vishwanath
Arun Somani
399
137
0
16 Jul 2023
DDNAS: Discretized Differentiable Neural Architecture Search for Text Classification
ACM Transactions on Intelligent Systems and Technology (ACM TIST), 2023
Kuan-Yu Chen
Cheng Li
Kuo-Jung Lee
257
4
0
12 Jul 2023
AutoML in the Age of Large Language Models: Current Challenges, Future Opportunities and Risks
Alexander Tornede
Difan Deng
Theresa Eimer
Joseph Giovanelli
Aditya Mohan
...
Sarah Segel
Daphne Theodorakopoulos
Tanja Tornede
Henning Wachsmuth
Marius Lindauer
383
40
0
13 Jun 2023
SqueezeLLM: Dense-and-Sparse Quantization
International Conference on Machine Learning (ICML), 2023
Sehoon Kim
Coleman Hooper
A. Gholami
Zhen Dong
Xiuyu Li
Sheng Shen
Michael W. Mahoney
Kurt Keutzer
MQ
564
297
0
13 Jun 2023
ALT: An Automatic System for Long Tail Scenario Modeling
IEEE International Conference on Data Engineering (ICDE), 2023
Ya-Lin Zhang
Jun Zhou
Yankun Ren
Yue Zhang
Xinxing Yang
Meng Li
Qitao Shi
Longfei Li
178
0
0
19 May 2023
Auto-CARD: Efficient and Robust Codec Avatar Driving for Real-time Mobile Telepresence
Computer Vision and Pattern Recognition (CVPR), 2023
Y. Fu
Yuecheng Li
Chenghui Li
Jason M. Saragih
Peizhao Zhang
Xiaoliang Dai
Yingyan Lin
3DH
415
8
0
24 Apr 2023
Efficient Automation of Neural Network Design: A Survey on Differentiable Neural Architecture Search
ACM Computing Surveys (ACM Comput. Surv.), 2023
Alexandre Heuillet
A. Nasser
Hichem Arioui
Hedi Tabia
AI4CE
371
31
0
11 Apr 2023
EdgeTran: Co-designing Transformers for Efficient Inference on Mobile Edge Platforms
Shikhar Tuli
N. Jha
364
4
0
24 Mar 2023
Neural Architecture Search for Effective Teacher-Student Knowledge Transfer in Language Models
Aashka Trivedi
Takuma Udagawa
Michele Merler
Yikang Shen
Yousef El-Kurdi
Bishwaranjan Bhattacharjee
343
9
0
16 Mar 2023
Gradient-Free Structured Pruning with Unlabeled Data
International Conference on Machine Learning (ICML), 2023
Azade Nova
H. Dai
Dale Schuurmans
SyDa
367
38
0
07 Mar 2023
Speculative Decoding with Big Little Decoder
Neural Information Processing Systems (NeurIPS), 2023
Sehoon Kim
K. Mangalam
Suhong Moon
Jitendra Malik
Michael W. Mahoney
A. Gholami
Kurt Keutzer
MoE
594
176
0
15 Feb 2023
Efficient Non-Parametric Optimizer Search for Diverse Tasks
Neural Information Processing Systems (NeurIPS), 2022
Ruochen Wang
Yuanhao Xiong
Minhao Cheng
Cho-Jui Hsieh
398
6
0
27 Sep 2022
Compressing Pre-trained Transformers via Low-Bit NxM Sparsity for Natural Language Understanding
Connor Holmes
Minjia Zhang
Yuxiong He
Bo Wu
189
3
0
30 Jun 2022
FlexiBERT: Are Current Transformer Architectures too Homogeneous and Rigid?
Journal of Artificial Intelligence Research (JAIR), 2022
Shikhar Tuli
Bhishma Dedhia
Shreshth Tuli
N. Jha
343
18
0
23 May 2022
Prompting to Distill: Boosting Data-Free Knowledge Distillation via Reinforced Prompt
International Joint Conference on Artificial Intelligence (IJCAI), 2022
Xinyin Ma
Xinchao Wang
Gongfan Fang
Yongliang Shen
Weiming Lu
174
13
0
16 May 2022
Meta Learning for Natural Language Processing: A Survey
North American Chapter of the Association for Computational Linguistics (NAACL), 2022
Hung-yi Lee
Shang-Wen Li
Ngoc Thang Vu
443
55
0
03 May 2022
Enable Deep Learning on Mobile Devices: Methods, Systems, and Applications
Han Cai
Ji Lin
Chengyue Wu
Zhijian Liu
Haotian Tang
Hanrui Wang
Ligeng Zhu
Song Han
288
136
0
25 Apr 2022
A Fast Post-Training Pruning Framework for Transformers
Neural Information Processing Systems (NeurIPS), 2022
Woosuk Kwon
Sehoon Kim
Michael W. Mahoney
Joseph Hassoun
Kurt Keutzer
A. Gholami
282
212
0
29 Mar 2022
Fast Monte-Carlo Approximation of the Attention Mechanism
AAAI Conference on Artificial Intelligence (AAAI), 2022
Hyunjun Kim
Jeonggil Ko
335
6
0
30 Jan 2022
AutoMC: Automated Model Compression based on Domain Knowledge and Progressive search strategy
IEEE International Conference on Data Engineering (ICDE), 2022
Chunnan Wang
Hongzhi Wang
Xiangyu Shi
174
1
0
24 Jan 2022
AutoDistill: an End-to-End Framework to Explore and Distill Hardware-Efficient Language Models
Xiaofan Zhang
Zongwei Zhou
Deming Chen
Yu Emma Wang
207
12
0
21 Jan 2022
Which Student is Best? A Comprehensive Knowledge Distillation Exam for Task-Specific BERT Models
Made Nindyatama Nityasya
Haryo Akbarianto Wibowo
Rendi Chevi
Radityo Eko Prasojo
Alham Fikri Aji
260
7
0
03 Jan 2022
RT-RCG: Neural Network and Accelerator Search Towards Effective and Real-time ECG Reconstruction from Intracardiac Electrograms
ACM Journal on Emerging Technologies in Computing Systems (JETC), 2021
Yongan Zhang
Anton Banta
Yonggan Fu
M. John
A. Post
M. Razavi
Joseph R. Cavallaro
B. Aazhang
Yingyan Lin
209
5
0
04 Nov 2021
Differentiable NAS Framework and Application to Ads CTR Prediction
Ravi Krishna
Aravind Kalaiah
Bichen Wu
Maxim Naumov
Dheevatsa Mudigere
M. Smelyanskiy
Kurt Keutzer
207
8
0
25 Oct 2021
SuperShaper: Task-Agnostic Super Pre-training of BERT Models with Variable Hidden Dimensions
Vinod Ganesan
Gowtham Ramesh
Pratyush Kumar
165
10
0
10 Oct 2021
Towards Efficient Post-training Quantization of Pre-trained Language Models
Haoli Bai
Lu Hou
Lifeng Shang
Xin Jiang
Irwin King
Michael R. Lyu
MQ
250
53
0
30 Sep 2021
Distiller: A Systematic Study of Model Distillation Methods in Natural Language Processing
Haoyu He
Xingjian Shi
Jonas W. Mueller
Zha Sheng
Mu Li
George Karypis
177
10
0
23 Sep 2021
Low-Latency Incremental Text-to-Speech Synthesis with Distilled Context Prediction Network
Automatic Speech Recognition & Understanding (ASRU), 2021
Takaaki Saeki
Shinnosuke Takamichi
Hiroshi Saruwatari
269
6
0
22 Sep 2021
RankNAS: Efficient Neural Architecture Search by Pairwise Ranking
Chi Hu
Chenglong Wang
Xiangnan Ma
Xia Meng
Yinqiao Li
Tong Xiao
Jingbo Zhu
Changliang Li
262
14
0
15 Sep 2021
EfficientBERT: Progressively Searching Multilayer Perceptron via Warm-up Knowledge Distillation
Chenhe Dong
Guangrun Wang
Hang Xu
Jiefeng Peng
Xiaozhe Ren
Xiaodan Liang
214
28
0
15 Sep 2021
AutoTinyBERT: Automatic Hyper-parameter Optimization for Efficient Pre-trained Language Models
Annual Meeting of the Association for Computational Linguistics (ACL), 2021
Yichun Yin
Cheng Chen
Lifeng Shang
Xin Jiang
Xiao Chen
Qun Liu
VLM
206
52
0
29 Jul 2021
AutoBERT-Zero: Evolving BERT Backbone from Scratch
AAAI Conference on Artificial Intelligence (AAAI), 2021
Jiahui Gao
Hang Xu
Han Shi
Xiaozhe Ren
Philip L. H. Yu
Xiaodan Liang
Xin Jiang
Zhenguo Li
232
39
0
15 Jul 2021
Scene-adaptive Knowledge Distillation for Sequential Recommendation via Differentiable Architecture Search
Lei-tai Chen
Fajie Yuan
Jiaxi Yang
Min Yang
Chengming Li
233
4
0
15 Jul 2021
LV-BERT: Exploiting Layer Variety for BERT
Findings (Findings), 2021
Weihao Yu
Zihang Jiang
Fei Chen
Qibin Hou
Jiashi Feng
MQ
181
0
0
22 Jun 2021
RoSearch: Search for Robust Student Architectures When Distilling Pre-trained Language Models
Xin Guo
Jianlei Yang
Haoyi Zhou
Xucheng Ye
Jianxin Li
174
2
0
07 Jun 2021
You Only Compress Once: Towards Effective and Elastic BERT Compression via Exploit-Explore Stochastic Nature Gradient
Shaokun Zhang
Xiawu Zheng
Chenyi Yang
Yuchao Li
Yan Wang
Jiayi Ji
Mengdi Wang
Shen Li
Jun Yang
Rongrong Ji
MQ
256
23
0
04 Jun 2021
NAS-BERT: Task-Agnostic and Adaptive-Size BERT Compression with Neural Architecture Search
Knowledge Discovery and Data Mining (KDD), 2021
Jin Xu
Xu Tan
Renqian Luo
Kaitao Song
Jian Li
Tao Qin
Tie-Yan Liu
MQ
188
92
0
30 May 2021
Improved Customer Transaction Classification using Semi-Supervised Knowledge Distillation
Rohan Sukumaran
186
2
0
15 Feb 2021
LightSpeech: Lightweight and Fast Text to Speech with Neural Architecture Search
IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2021
Renqian Luo
Xu Tan
Rui Wang
Tao Qin
Jinzhu Li
Sheng Zhao
Enhong Chen
Tie-Yan Liu
161
72
0
08 Feb 2021
Model Compression for Domain Adaptation through Causal Effect Estimation
Transactions of the Association for Computational Linguistics (TACL), 2021
Guy Rotman
Amir Feder
Roi Reichart
CML
283
8
0
18 Jan 2021
Auto-Agent-Distiller: Towards Efficient Deep Reinforcement Learning Agents via Neural Architecture Search
Y. Fu
Zhongzhi Yu
Yongan Zhang
Yingyan Lin
341
5
0
24 Dec 2020
Meta-KD: A Meta Knowledge Distillation Framework for Language Model Compression across Domains
Annual Meeting of the Association for Computational Linguistics (ACL), 2020
Haojie Pan
Chengyu Wang
Minghui Qiu
Yichang Zhang
Yaliang Li
Yanjie Liang
261
67
0
02 Dec 2020
1
2
Next
Page 1 of 2
