DynaBERT: Dynamic BERT with Adaptive Width and Depth

DynaBERT: Dynamic BERT with Adaptive Width and Depth

8 April 2020

Lifeng Shang

Xin Jiang

Qun Liu

Papers citing "DynaBERT: Dynamic BERT with Adaptive Width and Depth"

14 / 64 papers shown

Title
Towards Joint Intent Detection and Slot Filling via Higher-order Attention Dongsheng Chen Zhiqi Huang Xian Wu Shen Ge Yuexian Zou 29 20 0 18 Sep 2021
HAN: Higher-order Attention Network for Spoken Language Understanding Dongsheng Chen Zhiqi Huang Yuexian Zou 18 1 0 26 Aug 2021
Joint Multiple Intent Detection and Slot Filling via Self-distillation Lisong Chen Peilin Zhou Yuexian Zou VLM 16 31 0 18 Aug 2021
AutoBERT-Zero: Evolving BERT Backbone from Scratch Jiahui Gao Hang Xu Han Shi Xiaozhe Ren Philip L. H. Yu Xiaodan Liang Xin Jiang Zhenguo Li 21 37 0 15 Jul 2021
Learned Token Pruning for Transformers Sehoon Kim Sheng Shen D. Thorsley A. Gholami Woosuk Kwon Joseph Hassoun Kurt Keutzer 9 145 0 02 Jul 2021
IA-RED $^2$ : Interpretability-Aware Redundancy Reduction for Vision Transformers Bowen Pan Rameswar Panda Yifan Jiang Zhangyang Wang Rogerio Feris A. Oliva VLM ViT 39 153 0 23 Jun 2021
SGCN:Sparse Graph Convolution Network for Pedestrian Trajectory Prediction Liushuai Shi Le Wang Chengjiang Long Sanping Zhou Mo Zhou Zhenxing Niu G. Hua 30 217 0 04 Apr 2021
MiniLMv2: Multi-Head Self-Attention Relation Distillation for Compressing Pretrained Transformers Wenhui Wang Hangbo Bao Shaohan Huang Li Dong Furu Wei MQ 19 257 0 31 Dec 2020
BinaryBERT: Pushing the Limit of BERT Quantization Haoli Bai Wei Zhang Lu Hou Lifeng Shang Jing Jin Xin Jiang Qun Liu Michael Lyu Irwin King MQ 142 221 0 31 Dec 2020
AdapterDrop: On the Efficiency of Adapters in Transformers Andreas Rucklé Gregor Geigle Max Glockner Tilman Beck Jonas Pfeiffer Nils Reimers Iryna Gurevych 46 254 0 22 Oct 2020
Utility is in the Eye of the User: A Critique of NLP Leaderboards Kawin Ethayarajh Dan Jurafsky ELM 24 51 0 29 Sep 2020
TernaryBERT: Distillation-aware Ultra-low Bit BERT Wei Zhang Lu Hou Yichun Yin Lifeng Shang Xiao Chen Xin Jiang Qun Liu MQ 25 208 0 27 Sep 2020
Q-BERT: Hessian Based Ultra Low Precision Quantization of BERT Sheng Shen Zhen Dong Jiayu Ye Linjian Ma Z. Yao A. Gholami Michael W. Mahoney Kurt Keutzer MQ 227 575 0 12 Sep 2019
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 297 6,956 0 20 Apr 2018