BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models

18 June 2021

Papers citing "BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models"

50 / 816 papers shown

Title
PERP: Rethinking the Prune-Retrain Paradigm in the Era of LLMs Max Zimmer Megi Andoni Christoph Spiegel Sebastian Pokutta VLM 52 10 0 23 Dec 2023
Hazards from Increasingly Accessible Fine-Tuning of Downloadable Foundation Models Alan Chan Ben Bucknall Herbie Bradley David M. Krueger 14 6 0 22 Dec 2023
ElasticTrainer: Speeding Up On-Device Training with Runtime Elastic Tensor Selection Kai Huang Boyuan Yang Wei Gao 29 18 0 21 Dec 2023
PPEA-Depth: Progressive Parameter-Efficient Adaptation for Self-Supervised Monocular Depth Estimation Yue-Jiang Dong Yuanchen Guo Ying-Tian Liu Fang-Lue Zhang Song-Hai Zhang MDE 37 4 0 20 Dec 2023
Parameter-Efficient Fine-Tuning Methods for Pretrained Language Models: A Critical Review and Assessment Lingling Xu Haoran Xie S. J. Qin Xiaohui Tao F. Wang 46 135 0 19 Dec 2023
Sparse is Enough in Fine-tuning Pre-trained Large Language Models Weixi Song Z. Li Lefei Zhang Hai Zhao Bo Du VLM 23 7 0 19 Dec 2023
Efficiency-oriented approaches for self-supervised speech representation learning Luis Lugo Valentin Vielzeuf SSL 26 1 0 18 Dec 2023
When Parameter-efficient Tuning Meets General-purpose Vision-language Models Yihang Zhai Haixin Wang Jianlong Chang Xinlong Yang Jinan Sun Shikun Zhang Qi Tian VLM MLLM 17 1 0 16 Dec 2023
SPT: Fine-Tuning Transformer-based Language Models Efficiently with Sparsification Yuntao Gui Xiao Yan Peiqi Yin Han Yang James Cheng 37 2 0 16 Dec 2023
Gradient-based Parameter Selection for Efficient Fine-Tuning Zhi Zhang Qizhe Zhang Zijun Gao Renrui Zhang Ekaterina Shutova Shiji Zhou Shanghang Zhang 28 15 0 15 Dec 2023
Context-PEFT: Efficient Multi-Modal, Multi-Task Fine-Tuning Avelina Asada Hadji-Kyriacou Ognjen Arandjelović 29 0 0 14 Dec 2023
VMT-Adapter: Parameter-Efficient Transfer Learning for Multi-Task Dense Scene Understanding Yi Xin Junlong Du Qiang Wang Zhiwen Lin Ke Yan VPVLM 89 49 0 14 Dec 2023
MmAP : Multi-modal Alignment Prompt for Cross-domain Multi-task Learning Yi Xin Junlong Du Qiang Wang Ke Yan Shouhong Ding VLM 44 45 0 14 Dec 2023
EZ-CLIP: Efficient Zeroshot Video Action Recognition Shahzad Ahmad S. Chanda Y. S. Rawat VLM 28 7 0 13 Dec 2023
READ-PVLA: Recurrent Adapter with Partial Video-Language Alignment for Parameter-Efficient Transfer Learning in Low-Resource Video-Language Modeling Thong Nguyen Xiaobao Wu Xinshuai Dong Khoi M. Le Zhiyuan Hu Cong-Duy Nguyen See-Kiong Ng Anh Tuan Luu 33 2 0 12 Dec 2023
Mutual Enhancement of Large and Small Language Models with Cross-Silo Knowledge Transfer Yongheng Deng Ziqing Qiao Ju Ren Yang Liu Yaoxue Zhang 23 11 0 10 Dec 2023
Initialization Matters for Adversarial Transfer Learning Andong Hua Jindong Gu Zhiyu Xue Nicholas Carlini Eric Wong Yao Qin AAML 27 6 0 10 Dec 2023
Batched Low-Rank Adaptation of Foundation Models Yeming Wen Swarat Chaudhuri OffRL 21 19 0 09 Dec 2023
Stronger, Fewer, & Superior: Harnessing Vision Foundation Models for Domain Generalized Semantic Segmentation Zhixiang Wei Lin Chen Yi Jin Xiaoxiao Ma Tianle Liu Pengyang Lin Ben Wang H. Chen Jinjin Zheng 39 41 0 07 Dec 2023
MoSA: Mixture of Sparse Adapters for Visual Efficient Tuning Qizhe Zhang Bocheng Zou Ruichuan An Jiaming Liu Shanghang Zhang MoE 27 2 0 05 Dec 2023
Clustering Pseudo Language Family in Multilingual Translation Models with Fisher Information Matrix Xinyu Ma Xuebo Liu Min Zhang 37 1 0 05 Dec 2023
Towards General Purpose Vision Foundation Models for Medical Image Analysis: An Experimental Study of DINOv2 on Radiology Benchmarks Mohammed Baharoon Waseem Qureshi J. Ouyang Yanwu Xu Abdulrhman Aljouie Wei Peng MedIm AI4CE 43 8 0 04 Dec 2023
The Efficiency Spectrum of Large Language Models: An Algorithmic Survey Tianyu Ding Tianyi Chen Haidong Zhu Jiachen Jiang Yiqi Zhong Jinxin Zhou Guangzhi Wang Zhihui Zhu Ilya Zharkov Luming Liang 27 22 0 01 Dec 2023
Grounding Foundation Models through Federated Transfer Learning: A General Framework Yan Kang Tao Fan Hanlin Gu Xiaojin Zhang Lixin Fan Qiang Yang AI4CE 68 19 0 29 Nov 2023
Efficient Stitchable Task Adaptation Haoyu He Zizheng Pan Jing Liu Jianfei Cai Bohan Zhuang 31 3 0 29 Nov 2023
End-to-End Temporal Action Detection with 1B Parameters Across 1000 Frames Shuming Liu Chen-Da Liu-Zhang Chen Zhao Guohao Li 33 25 0 28 Nov 2023
BIM: Block-Wise Self-Supervised Learning with Masked Image Modeling Yixuan Luo Mengye Ren Sai Qian Zhang 28 0 0 28 Nov 2023
Parameter Efficient Fine-tuning via Cross Block Orchestration for Segment Anything Model Zelin Peng Zhengqin Xu Zhilin Zeng Lingxi Xie Qi Tian Wei Shen 12 10 0 28 Nov 2023
A Rank Stabilization Scaling Factor for Fine-Tuning with LoRA Damjan Kalajdzievski ALM 24 77 0 28 Nov 2023
Side4Video: Spatial-Temporal Side Network for Memory-Efficient Image-to-Video Transfer Learning Huanjin Yao Wenhao Wu Zhiheng Li VLM 95 9 0 27 Nov 2023
Adapter is All You Need for Tuning Visual Tasks Dongshuo Yin Leiyi Hu Bin Li Youqun Zhang 16 15 0 25 Nov 2023
PrivateLoRA For Efficient Privacy Preserving LLM Yiming Wang Yu Lin Xiaodong Zeng Guannan Zhang 47 11 0 23 Nov 2023
ComPEFT: Compression for Communicating Parameter Efficient Updates via Sparsification and Quantization Prateek Yadav Leshem Choshen Colin Raffel Mohit Bansal 32 13 0 22 Nov 2023
Towards Better Parameter-Efficient Fine-Tuning for Large Language Models: A Position Paper Chengyu Wang Junbing Yan Wei Zhang Jun Huang ALM 42 3 0 22 Nov 2023
Descriptor and Word Soups: Overcoming the Parameter Efficiency Accuracy Tradeoff for Out-of-Distribution Few-shot Learning Christopher Liao Theodoros Tsiligkaridis Brian Kulis OODD 46 5 0 21 Nov 2023
Applications of Large Scale Foundation Models for Autonomous Driving Yu Huang Yue Chen Zhu Li ELM AI4CE LRM ALM LM&Ro 61 15 0 20 Nov 2023
LQ-LoRA: Low-rank Plus Quantized Matrix Decomposition for Efficient Language Model Finetuning Han Guo P. Greengard Eric P. Xing Yoon Kim MQ 36 43 0 20 Nov 2023
Zero redundancy distributed learning with differential privacy Zhiqi Bu Justin Chiu Ruixuan Liu Sheng Zha George Karypis 45 8 0 20 Nov 2023
Sparse Low-rank Adaptation of Pre-trained Language Models Ning Ding Xingtai Lv Qiaosen Wang Yulin Chen Bowen Zhou Zhiyuan Liu Maosong Sun 22 55 0 20 Nov 2023
MultiLoRA: Democratizing LoRA for Better Multi-Task Learning Yiming Wang Yu Lin Xiaodong Zeng Guannan Zhang MoMe 41 20 0 20 Nov 2023
Adapters: A Unified Library for Parameter-Efficient and Modular Transfer Learning Clifton A. Poth Hannah Sterz Indraneil Paul Sukannya Purkayastha Leon Arne Engländer Timo Imhof Ivan Vulić Sebastian Ruder Iryna Gurevych Jonas Pfeiffer 32 45 0 18 Nov 2023
A Framework for Monitoring and Retraining Language Models in Real-World Applications Jaykumar Kasundra Claudia Schulz Melicaalsadat Mirsafian Stavroula Skylaki OffRL LRM 28 1 0 16 Nov 2023
Tied-Lora: Enhancing parameter efficiency of LoRA with weight tying Adithya Renduchintala Tugrul Konuk Oleksii Kuchaiev MoMe 21 41 0 16 Nov 2023
SiRA: Sparse Mixture of Low Rank Adaptation Yun Zhu Nevan Wichers Chu-Cheng Lin Xinyi Wang Tianlong Chen ... Han Lu Canoee Liu Liangchen Luo Jindong Chen Lei Meng MoE 23 27 0 15 Nov 2023
Tunable Soft Prompts are Messengers in Federated Learning Chenhe Dong Yuexiang Xie Bolin Ding Ying Shen Yaliang Li FedML 40 7 0 12 Nov 2023
Aggregate, Decompose, and Fine-Tune: A Simple Yet Effective Factor-Tuning Method for Vision Transformer Dongping Chen 33 3 0 12 Nov 2023
Parameter-Efficient Orthogonal Finetuning via Butterfly Factorization Weiyang Liu Zeju Qiu Yao Feng Yuliang Xiu Yuxuan Xue ... Songyou Peng Yandong Wen Michael J. Black Adrian Weller Bernhard Schölkopf 50 57 0 10 Nov 2023
Mini but Mighty: Finetuning ViTs with Mini Adapters Imad Eddine Marouf Enzo Tartaglione Stéphane Lathuilière 30 5 0 07 Nov 2023
Task-Agnostic Low-Rank Adapters for Unseen English Dialects Zedian Xiao William B. Held Yanchen Liu Diyi Yang 33 7 0 02 Nov 2023
Improving Prompt Tuning with Learned Prompting Layers Wei Zhu Ming Tan VLM 31 1 0 31 Oct 2023