BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models

18 June 2021

Papers citing "BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models"

50 / 816 papers shown

Title
IAPT: Instruction-Aware Prompt Tuning for Large Language Models Wei-wei Zhu Aaron Xuxiang Tian Congrui Yin Yuan Ni Xiaoling Wang Guotong Xie 45 0 0 28 May 2024
Sparsity- and Hybridity-Inspired Visual Parameter-Efficient Fine-Tuning for Medical Diagnosis Mingyuan Liu Lu Xu Shengnan Liu Jicong Zhang 44 1 0 28 May 2024
Diffusion Model Patching via Mixture-of-Prompts Seokil Ham Sangmin Woo Jin-Young Kim Hyojun Go Byeongjun Park Changick Kim VLM 26 2 0 28 May 2024
LoRA-XS: Low-Rank Adaptation with Extremely Small Number of Parameters Klaudia Bałazy Mohammadreza Banaei Karl Aberer Jacek Tabor 43 17 0 27 May 2024
DoRA: Enhancing Parameter-Efficient Fine-Tuning with Dynamic Rank Distribution Yulong Mao Kaiyu Huang Changhao Guan Ganglin Bao Fengran Mo Jinan Xu 29 10 0 27 May 2024
$$\textit{Trans-LoRA}$: towards data-free Transferable Parameter Efficient Finetuning$ $\textit{Trans-LoRA}$ : towards data-free Transferable Parameter Efficient Finetuning Runqian Wang Soumya Ghosh David D. Cox Diego Antognini Aude Oliva Rogerio Feris Leonid Karlinsky 32 1 0 27 May 2024
Laboratory-Scale AI: Open-Weight Models are Competitive with ChatGPT Even in Low-Resource Settings Robert Wolfe Isaac Slaughter Bin Han Bingbing Wen Yiwei Yang ... Bernease Herman E. Brown Zening Qu Nicholas Weber Bill Howe 35 4 0 27 May 2024
Optimizing Large Language Models for OpenAPI Code Completion Bohdan Petryshyn M. Lukoševičius LLMAG ALM 32 0 0 24 May 2024
Bridging The Gap between Low-rank and Orthogonal Adaptation via Householder Reflection Adaptation Shen Yuan Haotian Liu Hongteng Xu 44 2 0 24 May 2024
Thinking Forward: Memory-Efficient Federated Finetuning of Language Models Kunjal Panchal Nisarg Parikh Sunav Choudhary Lijun Zhang Yuriy Brun Hui Guan 53 3 0 24 May 2024
VB-LoRA: Extreme Parameter Efficient Fine-Tuning with Vector Banks Yang Li Shaobo Han Shihao Ji MoE 27 10 0 24 May 2024
Sparse-Tuning: Adapting Vision Transformers with Efficient Fine-tuning and Inference Ting Liu Xuyang Liu Liangtao Shi Zunnan Xu Siteng Huang Yi Xin Quanjun Yin 43 5 0 23 May 2024
Maintaining Structural Integrity in Parameter Spaces for Parameter Efficient Fine-tuning Chongjie Si Xuehui Wang Xue Yang Zhengqin Xu Qingyun Li Jifeng Dai Yu Qiao Xiaokang Yang Wei Shen 31 8 0 23 May 2024
Erasing the Bias: Fine-Tuning Foundation Models for Semi-Supervised Learning Kai Gan Tong Wei 41 10 0 20 May 2024
Efficient Prompt Tuning by Multi-Space Projection and Prompt Fusion Pengxiang Lan Enneng Yang Yuting Liu Guibing Guo Linying Jiang Jianzhe Zhao Xingwei Wang VLM AAML 33 1 0 19 May 2024
Low-Rank Adaptation of Time Series Foundational Models for Out-of-Domain Modality Forecasting Divij Gupta Anubhav Bhatti Surajsinh Parmar Chen Dan Yuwei Liu Bingjie Shen San Lee AI4TS 37 2 0 16 May 2024
SA-FedLora: Adaptive Parameter Allocation for Efficient Federated Learning with LoRA Tuning Yuning Yang Xiaohong Liu Tianrun Gao Xiaodong Xu Guangyu Wang 37 5 0 15 May 2024
DP-DyLoRA: Fine-Tuning Transformer-Based Models On-Device under Differentially Private Federated Learning using Dynamic Low-Rank Adaptation Jie Xu Karthikeyan P. Saravanan Rogier van Dalen Haaris Mehmood David Tuckey Mete Ozay 56 5 0 10 May 2024
Memory-Space Visual Prompting for Efficient Vision-Language Fine-Tuning Shibo Jie Yehui Tang Ning Ding Zhi-Hong Deng Kai Han Yunhe Wang VLM 33 6 0 09 May 2024
FlexEControl: Flexible and Efficient Multimodal Control for Text-to-Image Generation Xuehai He Jian Zheng Jacob Zhiyuan Fang Robinson Piramuthu Mohit Bansal Vicente Ordonez Gunnar A. Sigurdsson Nanyun Peng Xin Eric Wang DiffM 45 1 0 08 May 2024
Parameter-Efficient Fine-Tuning with Discrete Fourier Transform Ziqi Gao Qichao Wang Aochuan Chen Zijing Liu Bingzhe Wu Liang Chen Jia Li 38 25 0 05 May 2024
Random Masking Finds Winning Tickets for Parameter Efficient Fine-tuning Jing Xu Jingzhao Zhang 39 7 0 04 May 2024
SPAFIT: Stratified Progressive Adaptation Fine-tuning for Pre-trained Large Language Models Samir Arora Liangliang Wang 22 0 0 30 Apr 2024
FeDeRA:Efficient Fine-tuning of Language Models in Federated Learning Leveraging Weight Decomposition Yuxuan Yan Qianqian Yang Shunpu Tang Zhiguo Shi 38 13 0 29 Apr 2024
HFT: Half Fine-Tuning for Large Language Models Tingfeng Hui Zhenyu Zhang Shuohuan Wang Weiran Xu Yu Sun Hua-Hong Wu CLL 37 4 0 29 Apr 2024
Chameleon: A Data-Efficient Generalist for Dense Visual Prediction in the Wild Donggyun Kim Seongwoong Cho Semin Kim Chong Luo Seunghoon Hong VLM 42 2 0 29 Apr 2024
BlockLLM: Multi-tenant Finer-grained Serving for Large Language Models Jiamin Li Le Xu Hong-Yu Xu Aditya Akella 24 1 0 28 Apr 2024
Towards Incremental Learning in Large Language Models: A Critical Review M. Jovanovic Peter Voss ELM CLL KELM 37 5 0 28 Apr 2024
AutoGluon-Multimodal (AutoMM): Supercharging Multimodal AutoML with Foundation Models Zhiqiang Tang Haoyang Fang Su Zhou Taojiannan Yang Zihan Zhong Tony Hu Katrin Kirchhoff George Karypis 40 11 0 24 Apr 2024
Generalization Measures for Zero-Shot Cross-Lingual Transfer Saksham Bassi Duygu Ataman Kyunghyun Cho 29 0 0 24 Apr 2024
FL-TAC: Enhanced Fine-Tuning in Federated Learning via Low-Rank, Task-Specific Adapter Clustering Siqi Ping Yuzhu Mao Yang Liu Xiao-Ping Zhang Wenbo Ding FedML 26 3 0 23 Apr 2024
Advances and Open Challenges in Federated Learning with Foundation Models Chao Ren Han Yu Hongyi Peng Xiaoli Tang Anran Li ... A. Tan Bo Zhao Xiaoxiao Li Zengxiang Li Qiang Yang FedML AIFin AI4CE 75 7 0 23 Apr 2024
ColA: Collaborative Adaptation with Gradient Learning Enmao Diao Qi Le Suya Wu Xinran Wang Ali Anwar Jie Ding Vahid Tarokh 30 1 0 22 Apr 2024
MixLoRA: Enhancing Large Language Models Fine-Tuning with LoRA-based Mixture of Experts Dengchun Li Yingzi Ma Naizheng Wang Zhengmao Ye Zhiyuan Cheng ... Yan Zhang Lei Duan Jie Zuo Cal Yang Mingjie Tang MoE 32 42 0 22 Apr 2024
Parameter Efficient Fine Tuning: A Comprehensive Analysis Across Applications Charith Chandra Sai Balne S. Bhaduri Tamoghna Roy Vinija Jain Aman Chadha 40 12 0 21 Apr 2024
A Large-scale Medical Visual Task Adaptation Benchmark Shentong Mo Xufang Luo Yansen Wang Dongsheng Li MedIm 27 1 0 19 Apr 2024
AdaIR: Exploiting Underlying Similarities of Image Restoration Tasks with Adapters Hao-Wei Chen Yu-Syuan Xu Kelvin C. K. Chan Hsien-Kai Kuo Chun-Yi Lee Ming-Hsuan Yang 29 1 0 17 Apr 2024
Diffscaler: Enhancing the Generative Prowess of Diffusion Transformers Nithin Gopalakrishnan Nair Jeya Maria Jose Valanarasu Vishal M. Patel 40 1 0 15 Apr 2024
LoRA Dropout as a Sparsity Regularizer for Overfitting Control Yang Lin Xinyu Ma Xu Chu Yujie Jin Zhibang Yang Yasha Wang Hong-yan Mei 49 19 0 15 Apr 2024
How to build the best medical image segmentation algorithm using foundation models: a comprehensive empirical study with Segment Anything Model Han Gu Haoyu Dong Jichen Yang Maciej Mazurowski MedIm VLM 80 14 0 15 Apr 2024
Navigating the Landscape of Large Language Models: A Comprehensive Review and Analysis of Paradigms and Fine-Tuning Strategies Benjue Weng LM&MA 44 7 0 13 Apr 2024
Intuition-aware Mixture-of-Rank-1-Experts for Parameter Efficient Finetuning Yijiang Liu Rongyu Zhang Huanrui Yang Kurt Keutzer Yuan Du Li Du Shanghang Zhang MoE 38 6 0 13 Apr 2024
Diffusion Models Meet Remote Sensing: Principles, Methods, and Perspectives Yidan Liu Jun Yue Shaobo Xia Pedram Ghamisi Weiying Xie Leyuan Fang DiffM 42 12 0 13 Apr 2024
Emerging Property of Masked Token for Effective Pre-training Hyesong Choi Hunsang Lee Seyoung Joung Hyejin Park Jiyeong Kim Dongbo Min 36 9 0 12 Apr 2024
Salience-Based Adaptive Masking: Revisiting Token Dynamics for Enhanced Pre-training Hyesong Choi Hyejin Park Kwang Moo Yi Sungmin Cha Dongbo Min 39 9 0 12 Apr 2024
iVPT: Improving Task-relevant Information Sharing in Visual Prompt Tuning by Cross-layer Dynamic Connection Nan Zhou Jiaxin Chen Di Huang 35 1 0 08 Apr 2024
DLoRA: Distributed Parameter-Efficient Fine-Tuning Solution for Large Language Model Chao Gao Sai Qian Zhang ALM 118 7 0 08 Apr 2024
Facial Affective Behavior Analysis with Instruction Tuning Yifan Li Anh Dao Wentao Bao Zhen Tan Tianlong Chen Huan Liu Yu Kong CVBM 57 15 0 07 Apr 2024
Parameter Efficient Quasi-Orthogonal Fine-Tuning via Givens Rotation Xinyu Ma Xu Chu Zhibang Yang Yang Lin Xin Gao Junfeng Zhao 40 7 0 05 Apr 2024
ReFT: Representation Finetuning for Language Models Zhengxuan Wu Aryaman Arora Zheng Wang Atticus Geiger Daniel Jurafsky Christopher D. Manning Christopher Potts OffRL 35 58 0 04 Apr 2024