Parameter-Efficient Fine-Tuning Methods for Pretrained Language Models: A Critical Review and Assessment

19 December 2023

Haoran Xie

Papers citing "Parameter-Efficient Fine-Tuning Methods for Pretrained Language Models: A Critical Review and Assessment"

26 / 26 papers shown

Title
Fine-Tuning Large Language Models and Evaluating Retrieval Methods for Improved Question Answering on Building Codes Mohammad Aqib Mohd Hamza Qipei Mei Ying Hei Chui RALM ELM 47 0 0 07 May 2025
Block Circulant Adapter for Large Language Models Xinyu Ding Meiqi Wang Siyu Liao Zhongfeng Wang 29 0 0 01 May 2025
Can Differentially Private Fine-tuning LLMs Protect Against Privacy Attacks? Hao Du Shang Liu Yang Cao AAML 45 0 0 28 Apr 2025
Misaligned Roles, Misplaced Images: Structural Input Perturbations Expose Multimodal Alignment Blind Spots Erfan Shayegani G M Shahariar Sara Abdali Lei Yu Nael B. Abu-Ghazaleh Yue Dong AAML 53 0 0 01 Apr 2025
KaSA: Knowledge-Aware Singular-Value Adaptation of Large Language Models Fan Wang Juyong Jiang Chansung Park Sunghun Kim Jing Tang 91 1 0 08 Dec 2024
Adaptive Blind All-in-One Image Restoration David Serrano-Lozano Luis Herranz Shaolin Su Javier Vázquez-Corral VLM 92 0 0 27 Nov 2024
Parameter-Efficient Fine-Tuning in Large Models: A Survey of Methodologies L. Wang Sheng Chen Linnan Jiang Shu Pan Runze Cai Sen Yang Fei Yang 44 3 0 24 Oct 2024
DLP-LoRA: Efficient Task-Specific LoRA Fusion with a Dynamic, Lightweight Plugin for Large Language Models Yuxuan Zhang Ruizhe Li MoMe 48 0 0 02 Oct 2024
Towards Generalizable Scene Change Detection Jaewoo Kim Uehwan Kim 38 0 0 10 Sep 2024
CLIP with Generative Latent Replay: a Strong Baseline for Incremental Learning Emanuele Frascaroli Aniello Panariello Pietro Buzzega Lorenzo Bonicelli Angelo Porrello Simone Calderara VLM CLL 35 3 0 22 Jul 2024
CorDA: Context-Oriented Decomposition Adaptation of Large Language Models for Task-Aware Parameter-Efficient Fine-tuning Yibo Yang Xiaojie Li Zhongzhu Zhou S. Song Jianlong Wu Liqiang Nie Bernard Ghanem 45 6 0 07 Jun 2024
Compressible Dynamics in Deep Overparameterized Low-Rank Learning & Adaptation Can Yaras Peng Wang Laura Balzano Qing Qu AI4CE 27 12 0 06 Jun 2024
FeTT: Continual Class Incremental Learning via Feature Transformation Tuning Sunyuan Qiang Xuxin Lin Yanyan Liang Jun Wan Du Zhang CLL 37 1 0 20 May 2024
LoRA Dropout as a Sparsity Regularizer for Overfitting Control Yang Lin Xinyu Ma Xu Chu Yujie Jin Zhibang Yang Yasha Wang Hong-yan Mei 42 18 0 15 Apr 2024
How to build the best medical image segmentation algorithm using foundation models: a comprehensive empirical study with Segment Anything Model Han Gu Haoyu Dong Jichen Yang Maciej Mazurowski MedIm VLM 75 11 0 15 Apr 2024
PiSSA: Principal Singular Values and Singular Vectors Adaptation of Large Language Models Fanxu Meng Zhaohui Wang Muhan Zhang VLM 59 67 0 03 Apr 2024
Let Your Graph Do the Talking: Encoding Structured Data for LLMs Bryan Perozzi Bahare Fatemi Dustin Zelle Anton Tsitsulin Mehran Kazemi Rami Al-Rfou Jonathan J. Halcrow GNN 30 55 0 08 Feb 2024
Parameter-Efficient Tuning with Special Token Adaptation Xiaoocong Yang James Y. Huang Wenxuan Zhou Muhao Chen 18 12 0 10 Oct 2022
ATTEMPT: Parameter-Efficient Multi-task Tuning via Attentional Mixtures of Soft Prompts Akari Asai Mohammadreza Salehi Matthew E. Peters Hannaneh Hajishirzi 120 100 0 24 May 2022
SPoT: Better Frozen Model Adaptation through Soft Prompt Transfer Tu Vu Brian Lester Noah Constant Rami Al-Rfou Daniel Matthew Cer VLM LRM 137 276 0 15 Oct 2021
Raise a Child in Large Language Model: Towards Effective and Generalizable Fine-tuning Runxin Xu Fuli Luo Zhiyuan Zhang Chuanqi Tan Baobao Chang Songfang Huang Fei Huang LRM 136 177 0 13 Sep 2021
The Power of Scale for Parameter-Efficient Prompt Tuning Brian Lester Rami Al-Rfou Noah Constant VPVLM 278 3,835 0 18 Apr 2021
Memorisation versus Generalisation in Pre-trained Language Models Michael Tänzer Sebastian Ruder Marek Rei 84 50 0 16 Apr 2021
Beyond Fully-Connected Layers with Quaternions: Parameterization of Hypercomplex Multiplications with $1/n$ Parameters Aston Zhang Yi Tay Shuai Zhang Alvin Chan A. Luu S. Hui Jie Fu MQ 166 83 0 17 Feb 2021
WARP: Word-level Adversarial ReProgramming Karen Hambardzumyan Hrant Khachatrian Jonathan May AAML 254 340 0 01 Jan 2021
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 294 6,927 0 20 Apr 2018