FastBERT: a Self-distilling BERT with Adaptive Inference Time

5 April 2020

Papers citing "FastBERT: a Self-distilling BERT with Adaptive Inference Time"

30 / 30 papers shown

Title
DYNAMAX: Dynamic computing for Transformers and Mamba based architectures Miguel Nogales Matteo Gambella Manuel Roveri 56 0 0 29 Apr 2025
BEEM: Boosting Performance of Early Exit DNNs using Multi-Exit Classifiers as Experts Divya J. Bajpai M. Hanawal 63 0 0 02 Feb 2025
On the Impact of White-box Deployment Strategies for Edge AI on Latency and Model Performance Jaskirat Singh Bram Adams Ahmed E. Hassan VLM 29 0 0 01 Nov 2024
Accelerating Large Language Model Inference with Self-Supervised Early Exits Florian Valade LRM 36 1 0 30 Jul 2024
Tiny Models are the Computational Saver for Large Models Qingyuan Wang B. Cardiff Antoine Frappé Benoît Larras Deepu John 22 2 0 26 Mar 2024
On the Impact of Black-box Deployment Strategies for Edge AI on Latency and Model Performance Jaskirat Singh Emad Fallahzadeh Bram Adams Ahmed E. Hassan MQ 32 3 0 25 Mar 2024
DE $^3$ -BERT: Distance-Enhanced Early Exiting for BERT based on Prototypical Networks Jianing He Qi Zhang Weiping Ding Duoqian Miao Jun Zhao Liang Hu LongBing Cao 23 3 0 03 Feb 2024
AutoMix: Automatically Mixing Language Models Pranjal Aggarwal Aman Madaan Ankit Anand Srividya Pranavi Potharaju Swaroop Mishra ... Karthik Kappaganthu Yiming Yang Shyam Upadhyay Manaal Faruqui Mausam 40 17 0 19 Oct 2023
An Ensemble Approach to Question Classification: Integrating Electra Transformer, GloVe, and LSTM Sanad Aburass O. Dorgham Maha Abu Rumman 14 3 0 13 Aug 2023
Vesper: A Compact and Effective Pretrained Model for Speech Emotion Recognition Weidong Chen Xiaofen Xing Peihao Chen Xiangmin Xu VLM 18 34 0 20 Jul 2023
Zero-TPrune: Zero-Shot Token Pruning through Leveraging of the Attention Graph in Pre-Trained Transformers Hongjie Wang Bhishma Dedhia N. Jha ViT VLM 23 25 0 27 May 2023
F-PABEE: Flexible-patience-based Early Exiting for Single-label and Multi-label text Classification Tasks Xiangxiang Gao Wei-wei Zhu Jiasheng Gao Congrui Yin VLM 19 12 0 21 May 2023
Let's Sample Step by Step: Adaptive-Consistency for Efficient Reasoning and Coding with LLMs Pranjal Aggarwal Aman Madaan Yiming Yang Mausam LRM 13 33 0 19 May 2023
A Comprehensive Survey of AI-Generated Content (AIGC): A History of Generative AI from GAN to ChatGPT Yihan Cao Siyu Li Yixin Liu Zhiling Yan Yutong Dai Philip S. Yu Lichao Sun 19 493 0 07 Mar 2023
Momentum Decoding: Open-ended Text Generation As Graph Exploration Tian Lan Yixuan Su Shuhang Liu Heyan Huang Xian-Ling Mao 26 5 0 05 Dec 2022
Efficiently Controlling Multiple Risks with Pareto Testing Bracha Laufer-Goldshtein Adam Fisch Regina Barzilay Tommi Jaakkola 25 16 0 14 Oct 2022
Fluid Batching: Exit-Aware Preemptive Serving of Early-Exit Neural Networks on Edge NPUs Alexandros Kouris Stylianos I. Venieris Stefanos Laskaridis Nicholas D. Lane 30 8 0 27 Sep 2022
HiVLP: Hierarchical Vision-Language Pre-Training for Fast Image-Text Retrieval Feilong Chen Xiuyi Chen Jiaxin Shi Duzhen Zhang Jianlong Chang Qi Tian VLM CLIP 29 6 0 24 May 2022
PALBERT: Teaching ALBERT to Ponder Nikita Balagansky Daniil Gavrilov MoE 13 6 0 07 Apr 2022
A Simple Hash-Based Early Exiting Approach For Language Understanding and Generation Tianxiang Sun Xiangyang Liu Wei-wei Zhu Zhichao Geng Lingling Wu Yilong He Yuan Ni Guotong Xie Xuanjing Huang Xipeng Qiu 17 40 0 03 Mar 2022
Towards Efficient NLP: A Standard Evaluation and A Strong Baseline Xiangyang Liu Tianxiang Sun Junliang He Jiawen Wu Lingling Wu Xinyu Zhang Hao Jiang Zhao Cao Xuanjing Huang Xipeng Qiu ELM 11 46 0 13 Oct 2021
CascadeBERT: Accelerating Inference of Pre-trained Language Models via Calibrated Complete Models Cascade Lei Li Yankai Lin Deli Chen Shuhuai Ren Peng Li Jie Zhou Xu Sun 16 51 0 29 Dec 2020
Pretrained Transformers for Text Ranking: BERT and Beyond Jimmy J. Lin Rodrigo Nogueira Andrew Yates VLM 214 607 0 13 Oct 2020
BERT-QE: Contextualized Query Expansion for Document Re-ranking Zhi Zheng Kai Hui Ben He Xianpei Han Le Sun Andrew Yates 6 93 0 15 Sep 2020
It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learners Timo Schick Hinrich Schütze 13 950 0 15 Sep 2020
Knowledge Distillation: A Survey Jianping Gou B. Yu Stephen J. Maybank Dacheng Tao VLM 17 2,822 0 09 Jun 2020
Pre-trained Models for Natural Language Processing: A Survey Xipeng Qiu Tianxiang Sun Yige Xu Yunfan Shao Ning Dai Xuanjing Huang LM&MA VLM 232 1,444 0 18 Mar 2020
K-BERT: Enabling Language Representation with Knowledge Graph Weijie Liu Peng Zhou Zhe Zhao Zhiruo Wang Qi Ju Haotang Deng Ping Wang 220 773 0 17 Sep 2019
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 294 6,927 0 20 Apr 2018
Convolutional Neural Networks for Sentence Classification Yoon Kim AILaw VLM 241 13,283 0 25 Aug 2014