Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Survey

21 March 2024

Papers citing "Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Survey"

35 / 35 papers shown

Title
Pushing the Limits of Low-Bit Optimizers: A Focus on EMA Dynamics Cong Xu Wenbin Liang Mo Yu Anan Liu K. Zhang Lizhuang Ma J. Wang J. Wang W. Zhang MQ 9 0 0 01 May 2025
Efficient LLMs with AMP: Attention Heads and MLP Pruning Leandro Giusti Mugnaini Bruno Yamamoto Lucas Lauton de Alcantara Victor Zacarias Edson Bollis Lucas Pellicer A. H. R. Costa Artur Jordao 23 0 0 29 Apr 2025
ReCIT: Reconstructing Full Private Data from Gradient in Parameter-Efficient Fine-Tuning of Large Language Models Jin Xie Ruishi He Songze Li Xiaojun Jia Shouling Ji SILM AAML 43 0 0 29 Apr 2025
DYNAMAX: Dynamic computing for Transformers and Mamba based architectures Miguel Nogales Matteo Gambella Manuel Roveri 32 0 0 29 Apr 2025
Neuroplasticity in Artificial Intelligence -- An Overview and Inspirations on Drop In & Out Learning Yupei Li M. Milling Björn Schuller AI4CE 64 0 0 27 Mar 2025
Compositional Subspace Representation Fine-tuning for Adaptive Large Language Models Andy Zhou MoMe 60 0 0 13 Mar 2025
WorldSimBench: Towards Video Generation Models as World Simulators Yiran Qin Zhelun Shi Jiwen Yu Xijun Wang Enshen Zhou ... Lu Sheng Jing Shao Lei Bai Wanli Ouyang Ruimao Zhang EGVM VGen 70 35 0 23 Oct 2024
DLoRA: Distributed Parameter-Efficient Fine-Tuning Solution for Large Language Model Chao Gao Sai Qian Zhang ALM 100 3 0 08 Apr 2024
PeriodicLoRA: Breaking the Low-Rank Bottleneck in LoRA Optimization Xiangdi Meng Damai Dai Weiyao Luo Zhe Yang Shaoxiang Wu Xiaochen Wang Peiyi Wang Qingxiu Dong Liang Chen Zhifang Sui 93 6 0 25 Feb 2024
Infusing Hierarchical Guidance into Prompt Tuning: A Parameter-Efficient Framework for Multi-level Implicit Discourse Relation Recognition Haodong Zhao Ruifang He Mengnan Xiao Jing Xu 85 7 0 23 Feb 2024
LCM-LoRA: A Universal Stable-Diffusion Acceleration Module Simian Luo Yiqin Tan Suraj Patil Daniel Gu Patrick von Platen Apolinário Passos Longbo Huang Jian Li Hang Zhao MoMe 86 97 0 09 Nov 2023
Unified Low-Resource Sequence Labeling by Sample-Aware Dynamic Sparse Finetuning Sarkar Snigdha Sarathi Das Ranran Haoran Zhang Peng Shi Wenpeng Yin Rui Zhang 88 4 0 07 Nov 2023
Orthogonal Subspace Learning for Language Model Continual Learning Xiao Wang Tianze Chen Qiming Ge Han Xia Rong Bao Rui Zheng Qi Zhang Tao Gui Xuanjing Huang CLL 99 37 0 22 Oct 2023
Prototype-based HyperAdapter for Sample-Efficient Multi-task Tuning Hao Zhao Jie Fu Zhaofeng He 83 4 0 18 Oct 2023
PTP: Boosting Stability and Performance of Prompt Tuning with Perturbation-Based Regularizer Lichang Chen Heng-Chiao Huang Varun Madhavan AAML 92 8 0 03 May 2023
FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPU Ying Sheng Lianmin Zheng Binhang Yuan Zhuohan Li Max Ryabinin ... Joseph E. Gonzalez Percy Liang Christopher Ré Ion Stoica Ce Zhang 107 208 0 13 Mar 2023
XPrompt: Exploring the Extreme of Prompt Tuning Fang Ma Chen Zhang Lei Ren Jingang Wang Qifan Wang Wei Yu Wu Xiaojun Quan Dawei Song VLM 90 22 0 10 Oct 2022
MaPLe: Multi-modal Prompt Learning Muhammad Uzair Khattak H. Rasheed Muhammad Maaz Salman Khan F. Khan VPVLM VLM 151 298 0 06 Oct 2022
Test-Time Prompt Tuning for Zero-Shot Generalization in Vision-Language Models Manli Shu Weili Nie De-An Huang Zhiding Yu Tom Goldstein Anima Anandkumar Chaowei Xiao VLM VPVLM 141 182 0 15 Sep 2022
Diffusion Models in Vision: A Survey Florinel-Alin Croitoru Vlad Hondru Radu Tudor Ionescu M. Shah DiffM VLM MedIm 159 697 0 10 Sep 2022
Diffusion Models: A Comprehensive Survey of Methods and Applications Ling Yang Zhilong Zhang Yingxia Shao Shenda Hong Runsheng Xu Yue Zhao Wentao Zhang Bin Cui Ming-Hsuan Yang DiffM MedIm 179 800 0 02 Sep 2022
Prompt-aligned Gradient for Prompt Tuning Beier Zhu Yulei Niu Yucheng Han Yuehua Wu Hanwang Zhang VLM 155 162 0 30 May 2022
AdaptFormer: Adapting Vision Transformers for Scalable Visual Recognition Shoufa Chen Chongjian Ge Zhan Tong Jiangliu Wang Yibing Song Jue Wang Ping Luo 111 373 0 26 May 2022
PointCLIP: Point Cloud Understanding by CLIP Renrui Zhang Ziyu Guo Wei Zhang Kunchang Li Xupeng Miao Bin Cui Yu Qiao Peng Gao Hongsheng Li VLM 3DPC 127 291 0 04 Dec 2021
Masked Autoencoders Are Scalable Vision Learners Kaiming He Xinlei Chen Saining Xie Yanghao Li Piotr Dollár Ross B. Girshick ViT TPM 233 5,353 0 11 Nov 2021
Tip-Adapter: Training-free CLIP-Adapter for Better Vision-Language Modeling Renrui Zhang Rongyao Fang Wei Zhang Peng Gao Kunchang Li Jifeng Dai Yu Qiao Hongsheng Li VLM 146 281 0 06 Nov 2021
SPoT: Better Frozen Model Adaptation through Soft Prompt Transfer Tu Vu Brian Lester Noah Constant Rami Al-Rfou Daniel Matthew Cer VLM LRM 114 235 0 15 Oct 2021
Exploring Universal Intrinsic Task Subspace via Prompt Tuning Yujia Qin Xiaozhi Wang Yusheng Su Yankai Lin Ning Ding ... Juanzi Li Lei Hou Peng Li Maosong Sun Jie Zhou VLM VPVLM 94 18 0 15 Oct 2021
P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Tasks Xiao Liu Kaixuan Ji Yicheng Fu Weng Lam Tam Zhengxiao Du Zhilin Yang Jie Tang VLM 201 604 0 14 Oct 2021
Raise a Child in Large Language Model: Towards Effective and Generalizable Fine-tuning Runxin Xu Fuli Luo Zhiyuan Zhang Chuanqi Tan Baobao Chang Songfang Huang Fei Huang LRM 114 140 0 13 Sep 2021
Learning to Prompt for Vision-Language Models Kaiyang Zhou Jingkang Yang Chen Change Loy Ziwei Liu VPVLM CLIP VLM 298 2,108 0 02 Sep 2021
The Power of Scale for Parameter-Efficient Prompt Tuning Brian Lester Rami Al-Rfou Noah Constant VPVLM 252 2,999 0 18 Apr 2021
Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision Chao Jia Yinfei Yang Ye Xia Yi-Ting Chen Zarana Parekh Hieu H. Pham Quoc V. Le Yun-hsuan Sung Zhen Li Tom Duerig VLM CLIP 265 2,875 0 11 Feb 2021
Unifying Vision-and-Language Tasks via Text Generation Jaemin Cho Jie Lei Hao Tan Mohit Bansal MLLM 227 461 0 04 Feb 2021
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 267 6,003 0 20 Apr 2018