v1v2 (latest)

Revisiting Parameter-Efficient Tuning: Are We Really There Yet?

Conference on Empirical Methods in Natural Language Processing (EMNLP), 2022

16 February 2022

Papers citing "Revisiting Parameter-Efficient Tuning: Are We Really There Yet?"

50 / 66 papers shown

Title
Adaptation of Foundation Models for Medical Image Analysis: Strategies, Challenges, and Future Directions Karma Phuntsho Abdullah Kyungmi Lee Ickjai Lee Euijoon Ahn MedIm AI4CE 136 0 0 03 Nov 2025
All You Need is One: Capsule Prompt Tuning with a Single Vector Yiyang Liu James Chenhao Liang Heng Fan Wenhao Yang Yiming Cui Xiaotian Han Lifu Huang Dongfang Liu Qifan Wang Cheng Han VLM 113 1 0 19 Oct 2025
VIPAMIN: Visual Prompt Initialization via Embedding Selection and Subspace Expansion Jaekyun Park Hye Won Chung VPVLM VLM 193 0 0 18 Oct 2025
Prompt-based Adaptation in Large-scale Vision Models: A Survey Xi Xiao Yunbei Zhang Lin Zhao Y. Liu Xiaoying Liao ... Xue Lin Min Xu Qifan Wang Tianyang Wang Cheng Han VPVLM VLM 217 5 0 15 Oct 2025
MoA: Heterogeneous Mixture of Adapters for Parameter-Efficient Fine-Tuning of Large Language Models Jie Cao Tianwei Lin Hongyang He Rolan Yan Wenqiao Zhang Juncheng Billy Li D. Zhang Siliang Tang Yueting Zhuang MoE 175 0 0 06 Jun 2025
Visual Variational Autoencoder Prompt Tuning Xi Xiao Yunbei Zhang Yongqian Li Xingjian Li Tianyang Wang Jihun Hamm Xiang Wang Min Xu VPVLM VLM 310 6 0 22 Mar 2025
LoRASculpt: Sculpting LoRA for Harmonizing General and Specialized Knowledge in Multimodal Large Language ModelsComputer Vision and Pattern Recognition (CVPR), 2025 Jian Liang Wenke Huang Guancheng Wan Qu Yang Mang Ye MoMe CLL AI4CE 296 11 0 21 Mar 2025
1LoRA: Summation Compression for Very Low-Rank Adaptation Alessio Quercia Zhuo Cao Arya Bangun Richard D. Paul Abigail Morrison Ira Assent Hanno Scharr 166 1 0 11 Mar 2025
Understanding Why Adam Outperforms SGD: Gradient Heterogeneity in Transformers Akiyoshi Tomihari Issei Sato ODL 658 5 0 31 Jan 2025
Quantized Delta Weight Is Safety Keeper Yule Liu Zhen Sun Xinlei He Xinyi Huang 345 9 0 29 Nov 2024
CleaR: Towards Robust and Generalized Parameter-Efficient Fine-Tuning for Noisy Label LearningAnnual Meeting of the Association for Computational Linguistics (ACL), 2024 Yeachan Kim Junho Kim SangKeun Lee NoLa AAML 316 4 0 31 Oct 2024
LoRA Done RITE: Robust Invariant Transformation Equilibration for LoRA OptimizationInternational Conference on Learning Representations (ICLR), 2024 Jui-Nan Yen Si Si Zhao Meng Felix X. Yu Sai Surya Duvvuri Inderjit Dhillon Cho-Jui Hsieh Sanjiv Kumar 222 14 0 27 Oct 2024
Understanding Layer Significance in LLM Alignment Guangyuan Shi Zexin Lu Xiaoyu Dong Wenlong Zhang Xuanyu Zhang Yujie Feng Xiao-Ming Wu 432 11 0 23 Oct 2024
DARE the Extreme: Revisiting Delta-Parameter Pruning For Fine-Tuned ModelsInternational Conference on Learning Representations (ICLR), 2024 Wenlong Deng Yize Zhao V. Vakilian Minghui Chen Xiaoxiao Li Christos Thrampoulidis 431 8 0 12 Oct 2024
DeltaDQ: Ultra-High Delta Compression for Fine-Tuned LLMs via Group-wise Dropout and Separate Quantization Yanfeng Jiang Zelan Yang B. Chen Shen Li Shen Li Tao Li MQ 128 4 0 11 Oct 2024
Lessons and Insights from a Unifying Study of Parameter-Efficient Fine-Tuning (PEFT) in Visual RecognitionComputer Vision and Pattern Recognition (CVPR), 2024 Zheda Mai Ping Zhang Cheng-Hao Tu Hong-You Chen Li Zhang Wei-Lun Chao 259 0 0 24 Sep 2024
Efficient and Versatile Robust Fine-Tuning of Zero-shot ModelsEuropean Conference on Computer Vision (ECCV), 2024 Sungyeon Kim Boseung Jeong Donghyun Kim Suha Kwak VLM 183 8 0 11 Aug 2024
A Federated Learning-Friendly Approach for Parameter-Efficient Fine-Tuning of SAM in 3D Segmentation Mothilal Asokan Difei Gao Joya Chen Mike Zheng Shou FedML MedIm 203 4 0 31 Jul 2024
The Impact of LoRA Adapters on LLMs for Clinical Text Classification Under Computational and Data Constraints Thanh-Dung Le T. Nguyen Vu Nguyen Ha Symeon Chatzinotas P. Jouvet R. Noumeir 285 0 0 27 Jul 2024
Exploring the Effectiveness and Consistency of Task Selection in Intermediate-Task Transfer Learning Pin-Jie Lin Miaoran Zhang Marius Mosbach Dietrich Klakow 177 0 0 23 Jul 2024
Test-Time Low Rank Adaptation via Confidence Maximization for Zero-Shot Generalization of Vision-Language Models Raza Imam Hanan Gani Muhammad Huzaifa Karthik Nandakumar VLM 132 13 0 22 Jul 2024
GPT vs RETRO: Exploring the Intersection of Retrieval and Parameter-Efficient Fine-Tuning Aleksander Ficek Jiaqi Zeng Oleksii Kuchaiev 150 3 0 05 Jul 2024
Boosting Consistency in Dual Training for Long-Tailed Semi-Supervised Learning Kai Gan Tong Wei Min-Ling Zhang 226 1 0 19 Jun 2024
Fighting Randomness with Randomness: Mitigating Optimisation Instability of Fine-Tuning using Delayed Ensemble and Noisy Interpolation Branislav Pecher Ján Cegin Róbert Belanec Jakub Simko Ivan Srba Maria Bielikova 177 1 0 18 Jun 2024
An Empirical Study on Parameter-Efficient Fine-Tuning for MultiModal Large Language ModelsAnnual Meeting of the Association for Computational Linguistics (ACL), 2024 Xiongtao Zhou Jie He Yuhua Ke Guangyao Zhu Víctor Gutiérrez-Basulto Jeff Z. Pan 190 27 0 07 Jun 2024
Understanding Linear Probing then Fine-tuning Language Models from NTK Perspective Akiyoshi Tomihari Issei Sato 216 9 0 27 May 2024
Erasing the Bias: Fine-Tuning Foundation Models for Semi-Supervised Learning Kai Gan Tong Wei 214 20 0 20 May 2024
Feature-Adaptive and Data-Scalable In-Context LearningAnnual Meeting of the Association for Computational Linguistics (ACL), 2024 Jiahao Li Quan Wang Li Zhang Guoqing Jin Zhendong Mao 247 4 0 17 May 2024
RST-LoRA: A Discourse-Aware Low-Rank Adaptation for Long Document Abstractive Summarization Dongqi Pu Vera Demberg 340 8 0 01 May 2024
XFT: Unlocking the Power of Code Instruction Tuning by Simply Merging Upcycled Mixture-of-Experts Yifeng Ding Jiawei Liu Yuxiang Wei Terry Yue Zhuo Lingming Zhang ALM MoE 155 6 0 23 Apr 2024
Personalized LLM Response Generation with Parameterized Memory Injection Kai Zhang Lizhi Qing Yangyang Kang 305 20 0 04 Apr 2024
An Empirical Study of Parameter Efficient Fine-tuning on Vision-Language Pre-train ModelIEEE International Conference on Multimedia and Expo (ICME), 2024 Yuxin Tian Mouxing Yang Yunfan Li Dayiheng Liu Xingzhang Ren Xiaocui Peng Jiancheng Lv VLM 111 1 0 13 Mar 2024
BitDelta: Your Fine-Tune May Only Be Worth One Bit James Liu Guangxuan Xiao Kai Li Jason D. Lee Song Han Tri Dao Tianle Cai 203 36 0 15 Feb 2024
Facing the Elephant in the Room: Visual Prompt Tuning or Full Finetuning?International Conference on Learning Representations (ICLR), 2024 Cheng Han Qifan Wang Yiming Cui Wenguan Wang Lifu Huang Siyuan Qi Dongfang Liu VLM 348 37 0 23 Jan 2024
Astraios: Parameter-Efficient Instruction Tuning Code Large Language Models Terry Yue Zhuo A. Zebaze Nitchakarn Suppattarachai Leandro von Werra H. D. Vries Qian Liu Niklas Muennighoff ALM 256 29 0 01 Jan 2024
A Comprehensive Evaluation of Parameter-Efficient Fine-Tuning on Software Engineering Tasks Wentao Zou Qi Li Jidong Ge Chuanyi Li Xiaoyu Shen LiGuo Huang Bin Luo 210 9 0 25 Dec 2023
Federated Full-Parameter Tuning of Billion-Sized Language Models with Communication Cost under 18 Kilobytes Zhen Qin Daoyuan Chen Bingchen Qian Bolin Ding Yaliang Li Shuiguang Deng FedML 257 55 0 11 Dec 2023
Sim-to-Real Causal Transfer: A Metric Learning Approach to Causally-Aware Interaction Representations Yuejiang Liu Ahmad Rahimi Po-Chien Luan Frano Rajic Alexandre Alahi 374 6 0 07 Dec 2023
Towards Better Parameter-Efficient Fine-Tuning for Large Language Models: A Position Paper Chengyu Wang Junbing Yan Wei Zhang Jun Huang ALM 173 4 0 22 Nov 2023
SiRA: Sparse Mixture of Low Rank Adaptation Yun Zhu Nevan Wichers Chu-Cheng Lin Xinyi Wang Tianlong Chen ... Han Lu Canoee Liu Liangchen Luo Jindong Chen Lei Meng MoE 186 35 0 15 Nov 2023
Low-Rank Adaptation for Multilingual Summarization: An Empirical Study Chenxi Whitehouse Fantine Huot Jasmijn Bastings Mostafa Dehghani Chu-Cheng Lin Mirella Lapata 233 13 0 14 Nov 2023
On the Analysis of Cross-Lingual Prompt Tuning for Decoder-based Multilingual Model Nohil Park Joonsuk Park Kang Min Yoo Sungroh Yoon 242 4 0 14 Nov 2023
Mixture-of-Linguistic-Experts Adapters for Improving and Interpreting Pre-trained Language ModelsConference on Empirical Methods in Natural Language Processing (EMNLP), 2023 Raymond Li Gabriel Murray Giuseppe Carenini MoE 145 3 0 24 Oct 2023
ScaLearn: Simple and Highly Parameter-Efficient Task Transfer by Learning to ScaleAnnual Meeting of the Association for Computational Linguistics (ACL), 2023 Markus Frohmann Carolin Holtermann Shahed Masoudian Anne Lauscher Navid Rekabsaz 309 2 0 02 Oct 2023
Deep Model Fusion: A Survey Weishi Li Yong Peng Miao Zhang Liang Ding Han Hu Li Shen FedML MoMe 277 86 0 27 Sep 2023
A Comprehensive Overview of Large Language ModelsACM Transactions on Intelligent Systems and Technology (ACM TIST), 2023 Humza Naveed Asad Ullah Khan Shi Qiu Muhammad Saqib Saeed Anwar Muhammad Usman Naveed Akhtar Nick Barnes Lin Wang OffRL 784 1,119 0 12 Jul 2023
Stitched ViTs are Flexible Vision BackbonesEuropean Conference on Computer Vision (ECCV), 2023 Zizheng Pan Jing Liu Haoyu He Jianfei Cai Bohan Zhuang 167 4 0 30 Jun 2023
Efficient Storage of Fine-Tuned Models via Low-Rank Approximation of Weight Residuals Simo Ryu S. Seo Jaejun Yoo 191 11 0 28 May 2023
Neural Architecture Search for Parameter-Efficient Fine-tuning of Large Pre-trained Language ModelsAnnual Meeting of the Association for Computational Linguistics (ACL), 2023 Neal Lawton Anoop Kumar Govind Thattai Aram Galstyan Greg Ver Steeg 136 23 0 26 May 2023
A Language Model of Java Methods with Train/Test Deduplication Chia-Yi Su Aakash Bansal Vijayanta Jain S. Ghanavati Collin McMillan SyDa VLM 178 14 0 15 May 2023