BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models

18 June 2021

Papers citing "BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models"

50 / 815 papers shown

Title
Universal Deep Image Compression via Content-Adaptive Optimization with Adapters Koki Tsubota Hiroaki Akutsu Kiyoharu Aizawa 19 17 0 02 Nov 2022
Adapter-Based Extension of Multi-Speaker Text-to-Speech Model for New Speakers Cheng-Ping Hsieh Subhankar Ghosh Boris Ginsburg 41 18 0 01 Nov 2022
A Close Look into the Calibration of Pre-trained Language Models Yangyi Chen Lifan Yuan Ganqu Cui Zhiyuan Liu Heng Ji 25 43 0 31 Oct 2022
AdaMix: Mixture-of-Adaptations for Parameter-efficient Model Tuning Yaqing Wang Sahaj Agarwal Subhabrata Mukherjee Xiaodong Liu Jing Gao Ahmed Hassan Awadallah Jianfeng Gao MoE 19 118 0 31 Oct 2022
GPS: Genetic Prompt Search for Efficient Few-shot Learning Hanwei Xu Yujun Chen Yulun Du Nan Shao Yanggang Wang Haiyu Li Zhilin Yang VLM 14 28 0 31 Oct 2022
Parameter-Efficient Tuning Makes a Good Classification Head Zhuoyi Yang Ming Ding Yanhui Guo Qingsong Lv Jie Tang VLM 35 14 0 30 Oct 2022
What Language Model to Train if You Have One Million GPU Hours? Teven Le Scao Thomas Wang Daniel Hesslow Lucile Saulnier Stas Bekman ... Lintang Sutawika Jaesung Tae Zheng-Xin Yong Julien Launay Iz Beltagy MoE AI4CE 225 103 0 27 Oct 2022
Evaluating Parameter Efficient Learning for Generation Peng-Tao Xu M. Patwary Shrimai Prabhumoye Virginia Adams R. Prenger Wei Ping Nayeon Lee M. Shoeybi Bryan Catanzaro MoE 20 3 0 25 Oct 2022
Different Tunes Played with Equal Skill: Exploring a Unified Optimization Subspace for Delta Tuning Jing Yi Weize Chen Yujia Qin Yankai Lin Ning Ding Xu Han Zhiyuan Liu Maosong Sun Jie Zhou 12 2 0 24 Oct 2022
Efficiently Tuned Parameters are Task Embeddings Wangchunshu Zhou Canwen Xu Julian McAuley 11 8 0 21 Oct 2022
Late Prompt Tuning: A Late Prompt Could Be Better Than Many Prompts Xiangyang Liu Tianxiang Sun Xuanjing Huang Xipeng Qiu VLM 34 27 0 20 Oct 2022
Prompting through Prototype: A Prototype-based Prompt Learning on Pretrained Vision-Language Models Yue Zhang Hongliang Fei Dingcheng Li Tan Yu Ping Li VPVLM VLM 20 9 0 19 Oct 2022
Incorporating Relevance Feedback for Information-Seeking Retrieval using Few-Shot Document Re-Ranking Tim Baumgärtner Leonardo F. R. Ribeiro Nils Reimers Iryna Gurevych 27 6 0 19 Oct 2022
Revision Transformers: Instructing Language Models to Change their Values Felix Friedrich Wolfgang Stammer P. Schramowski Kristian Kersting KELM 30 6 0 19 Oct 2022
Hidden State Variability of Pretrained Language Models Can Guide Computation Reduction for Transfer Learning Shuo Xie Jiahao Qiu Ankita Pasad Li Du Qing Qu Hongyuan Mei 32 16 0 18 Oct 2022
Tiny-Attention Adapter: Contexts Are More Important Than the Number of Parameters Hongyu Zhao Hao Tan Hongyuan Mei MoE 31 16 0 18 Oct 2022
Scaling & Shifting Your Features: A New Baseline for Efficient Model Tuning Dongze Lian Daquan Zhou Jiashi Feng Xinchao Wang 34 247 0 17 Oct 2022
Multitask Pre-training of Modular Prompt for Chinese Few-Shot Learning Tianxiang Sun Zhengfu He Qinen Zhu Xipeng Qiu Xuanjing Huang VLM VPVLM 12 20 0 14 Oct 2022
A Kernel-Based View of Language Model Fine-Tuning Sadhika Malladi Alexander Wettig Dingli Yu Danqi Chen Sanjeev Arora VLM 68 60 0 11 Oct 2022
Model Cascading: Towards Jointly Improving Efficiency and Accuracy of NLP Systems Neeraj Varshney Chitta Baral 17 27 0 11 Oct 2022
Exploring Efficient-tuning Methods in Self-supervised Speech Models Zih-Ching Chen Chin-Lun Fu Chih-Ying Liu Shang-Wen Li Hung-yi Lee 24 38 0 10 Oct 2022
Unified Detoxifying and Debiasing in Language Generation via Inference-time Adaptive Optimization Zonghan Yang Xiaoyuan Yi Peng Li Yang Liu Xing Xie 25 33 0 10 Oct 2022
Parameter-Efficient Tuning with Special Token Adaptation Xiaoocong Yang James Y. Huang Wenxuan Zhou Muhao Chen 26 12 0 10 Oct 2022
Knowledge Injected Prompt Based Fine-tuning for Multi-label Few-shot ICD Coding Zhichao Yang Shufan Wang Bhanu Pratap Singh Rawat Avijit Mitra Hong-ye Yu 119 49 0 07 Oct 2022
Polyhistor: Parameter-Efficient Multi-Task Adaptation for Dense Vision Tasks Yen-Cheng Liu Chih-Yao Ma Junjiao Tian Zijian He Z. Kira 120 47 0 07 Oct 2022
Improving the Sample Efficiency of Prompt Tuning with Domain Adaptation Xu Guo Boyang Albert Li Han Yu VLM 39 22 0 06 Oct 2022
Visual Prompt Tuning for Generative Transfer Learning Kihyuk Sohn Yuan Hao José Lezama Luisa F. Polanía Huiwen Chang Han Zhang Irfan Essa Lu Jiang VPVLM VLM 53 81 0 03 Oct 2022
Towards a Unified View on Visual Parameter-Efficient Transfer Learning Bruce X. B. Yu Jianlong Chang Lin Liu Qi Tian Changan Chen VPVLM VLM 68 34 0 03 Oct 2022
Differentially Private Bias-Term Fine-tuning of Foundation Models Zhiqi Bu Yu-Xiang Wang Sheng Zha George Karypis 23 46 0 30 Sep 2022
Depth-Wise Attention (DWAtt): A Layer Fusion Method for Data-Efficient Classification Muhammad N. ElNokrashy Badr AlKhamissi Mona T. Diab MoMe 17 4 0 30 Sep 2022
Towards Parameter-Efficient Integration of Pre-Trained Language Models In Temporal Video Grounding Erica K. Shimomoto Edison Marrese-Taylor Hiroya Takamura Ichiro Kobayashi Hideki Nakayama Yusuke Miyao 27 7 0 26 Sep 2022
WinoDict: Probing language models for in-context word acquisition Julian Martin Eisenschlos Jeremy R. Cole Fangyu Liu William W. Cohen KELM 19 11 0 25 Sep 2022
Automatic Label Sequence Generation for Prompting Sequence-to-sequence Models Zichun Yu Tianyu Gao Zhengyan Zhang Yankai Lin Zhiyuan Liu Maosong Sun Jie Zhou VLM LRM 28 1 0 20 Sep 2022
Parameter-Efficient Conformers via Sharing Sparsely-Gated Experts for End-to-End Speech Recognition Ye Bai Jie Li W. Han Hao Ni Kaituo Xu Zhuo Zhang Cheng Yi Xiaorui Wang MoE 21 1 0 17 Sep 2022
Test-Time Prompt Tuning for Zero-Shot Generalization in Vision-Language Models Manli Shu Weili Nie De-An Huang Zhiding Yu Tom Goldstein Anima Anandkumar Chaowei Xiao VLM VPVLM 186 280 0 15 Sep 2022
StoryDALL-E: Adapting Pretrained Text-to-Image Transformers for Story Continuation A. Maharana Darryl Hannan Mohit Bansal DiffM 19 77 0 13 Sep 2022
Efficient Methods for Natural Language Processing: A Survey Marcos Vinícius Treviso Ji-Ung Lee Tianchu Ji Betty van Aken Qingqing Cao ... Emma Strubell Niranjan Balasubramanian Leon Derczynski Iryna Gurevych Roy Schwartz 28 109 0 31 Aug 2022
Transformers with Learnable Activation Functions Haishuo Fang Ji-Ung Lee N. Moosavi Iryna Gurevych AI4CE 17 7 0 30 Aug 2022
Prompt-Matched Semantic Segmentation Lingbo Liu Jianlong Chang Bruce X. B. Yu Liang Lin Qi Tian Changrui Chen VPVLM VLM 22 27 0 22 Aug 2022
Conv-Adapter: Exploring Parameter Efficient Transfer Learning for ConvNets Hao Chen R. Tao Han Zhang Yidong Wang Xiang Li Weirong Ye Jindong Wang Guosheng Hu Marios Savvides VPVLM 21 52 0 15 Aug 2022
Branch-Train-Merge: Embarrassingly Parallel Training of Expert Language Models Margaret Li Suchin Gururangan Tim Dettmers M. Lewis Tim Althoff Noah A. Smith Luke Zettlemoyer MoMe 26 142 0 05 Aug 2022
Prompt Tuning for Generative Multimodal Pretrained Models Han Yang Junyang Lin An Yang Peng Wang Chang Zhou Hongxia Yang VLM LRM VPVLM 37 30 0 04 Aug 2022
Efficient Fine-Tuning of Compressed Language Models with Learners Danilo Vucetic Mohammadreza Tayaranian M. Ziaeefard J. Clark B. Meyer W. Gross 11 2 0 03 Aug 2022
Efficient NLP Model Finetuning via Multistage Data Filtering Ouyang Xu S. Ansari F. Lin Yangfeng Ji 22 2 0 28 Jul 2022
Two-Stage Fine-Tuning: A Novel Strategy for Learning Class-Imbalanced Data Taha ValizadehAslani Yiwen Shi Jing Wang Ping Ren Yi Zhang Meng Hu Lianggong Zhao Hualou Liang 28 7 0 22 Jul 2022
Parameter-Efficient Prompt Tuning Makes Generalized and Calibrated Neural Text Retrievers Weng Lam Tam Xiao Liu Kaixuan Ji Lilong Xue Xing Zhang Yuxiao Dong Jiahua Liu Maodi Hu Jie Tang VPVLM 19 27 0 14 Jul 2022
Convolutional Bypasses Are Better Vision Transformer Adapters Shibo Jie Zhi-Hong Deng VPVLM 10 131 0 14 Jul 2022
Meta-Learning the Difference: Preparing Large Language Models for Efficient Adaptation Zejiang Hou Julian Salazar George Polovets 17 14 0 07 Jul 2022
On-Device Training Under 256KB Memory Ji Lin Ligeng Zhu Wei-Ming Chen Wei-Chen Wang Chuang Gan Song Han MQ 25 194 0 30 Jun 2022
ST-Adapter: Parameter-Efficient Image-to-Video Transfer Learning Junting Pan Ziyi Lin Xiatian Zhu Jing Shao Hongsheng Li 19 190 0 27 Jun 2022