Scaling Down to Scale Up: A Guide to Parameter-Efficient Fine-Tuning

28 March 2023

Papers citing "Scaling Down to Scale Up: A Guide to Parameter-Efficient Fine-Tuning"

50 / 107 papers shown

Title
Fine-Tuning Large Language Models and Evaluating Retrieval Methods for Improved Question Answering on Building Codes Mohammad Aqib Mohd Hamza Qipei Mei Ying Hei Chui RALM ELM 50 0 0 07 May 2025
Federated Client-tailored Adapter for Medical Image Segmentation Guyue Hu Siyuan Song Yukun Kang Z. Yin Gangming Zhao Chenglong Li Jin Tang FedML MedIm 136 0 0 25 Apr 2025
FISH-Tuning: Enhancing PEFT Methods with Fisher Information Kang Xue Ming Dong Xinhui Tu Tingting He 19 0 0 05 Apr 2025
MoST: Efficient Monarch Sparse Tuning for 3D Representation Learning Xu Han Yuan Tang Jinfeng Xu Xianzhi Li 48 0 0 24 Mar 2025
On the Robustness Tradeoff in Fine-Tuning Kunyang Li Jean-Charles Noirot Ferrand Ryan Sheatsley Blaine Hoak Yohan Beugin Eric Pauley Patrick D. McDaniel 57 0 0 19 Mar 2025
Rethinking Few-Shot Adaptation of Vision-Language Models in Two Stages Matteo Farina Massimiliano Mancini Giovanni Iacca Elisa Ricci VLM 53 0 0 14 Mar 2025
Compositional Subspace Representation Fine-tuning for Adaptive Large Language Models Andy Zhou MoMe 92 0 0 13 Mar 2025
Parameter-Efficient Fine-Tuning of Large Language Models via Deconvolution in Subspace Jia-Chen Zhang Yu-Jie Xiong Chun-Ming Xia Dong-Hai Zhu Xi-He Qiu 64 1 0 03 Mar 2025
Sparsity May Be All You Need: Sparse Random Parameter Adaptation Jesus Rios Pierre L. Dognin Ronny Luss K. Ramamurthy 32 1 0 21 Feb 2025
An Efficient Row-Based Sparse Fine-Tuning Cen-Jhih Li Aditya Bhaskara 52 0 0 17 Feb 2025
TabPFN Unleashed: A Scalable and Effective Solution to Tabular Classification Problems Si-Yang Liu Han-Jia Ye 66 5 0 04 Feb 2025
EDoRA: Efficient Weight-Decomposed Low-Rank Adaptation via Singular Value Decomposition Hamid Nasiri Peter Garraghan 36 1 0 21 Jan 2025
A Resource-Efficient Training Framework for Remote Sensing Text--Image Retrieval Weihang Zhang Jihao Li Shuoke Li Ziqing Niu Jialiang Chen Wenkai Zhang VLM 44 0 0 18 Jan 2025
Tuning Vision-Language Models with Candidate Labels by Prompt Alignment Zhifang Zhang Yuwei Niu Xin Liu Beibei Li VPVLM VLM 54 0 0 31 Dec 2024
A Contrastive Pretrain Model with Prompt Tuning for Multi-center Medication Recommendation Q. Liu Zhaopeng Qiu Xiangyu Zhao X. Wu Zijian Zhang Tong Bill Xu Feng Tian 28 0 0 31 Dec 2024
GaLore $+$ : Boosting Low-Rank Adaptation for LLMs with Cross-Head Projection Xutao Liao Shaohui Li Yuhui Xu Zhi Li Y. Liu You He VLM 54 2 0 31 Dec 2024
KaSA: Knowledge-Aware Singular-Value Adaptation of Large Language Models Fan Wang Juyong Jiang Chansung Park Sunghun Kim Jing Tang 91 1 0 08 Dec 2024
CULL-MT: Compression Using Language and Layer pruning for Machine Translation Pedram Rostami M. Dousti 27 0 0 10 Nov 2024
MambaPEFT: Exploring Parameter-Efficient Fine-Tuning for Mamba Masakazu Yoshimura Teruaki Hayashi Yota Maeda Mamba 72 2 0 06 Nov 2024
Parameter-Efficient Fine-Tuning of Large Language Models for Unit Test Generation: An Empirical Study André Storhaug Jingyue Li ALM 43 1 0 04 Nov 2024
Pin-Tuning: Parameter-Efficient In-Context Tuning for Few-Shot Molecular Property Prediction Liang Wang Qiang Liu Shaozhen Liu Xin Sun Shu Wu Liang Wang 37 2 0 02 Nov 2024
Human-inspired Perspectives: A Survey on AI Long-term Memory Zihong He Weizhe Lin Hao Zheng Fan Zhang Matt Jones Laurence Aitchison X. Xu Miao Liu Per Ola Kristensson Junxiao Shen 77 2 0 01 Nov 2024
Demystifying Large Language Models for Medicine: A Primer Qiao Jin Nicholas Wan Robert Leaman Shubo Tian Zhizheng Wang ... Chunhua Weng Ronald M. Summers Qingyu Chen Yifan Peng Zhiyong Lu LM&MA 32 3 0 24 Oct 2024
Parameter-Efficient Fine-Tuning in Large Models: A Survey of Methodologies L. Wang Sheng Chen Linnan Jiang Shu Pan Runze Cai Sen Yang Fei Yang 44 3 0 24 Oct 2024
Evolutionary Retrofitting Mathurin Videau M. Zameshina Alessandro Leite Laurent Najman Marc Schoenauer O. Teytaud 33 0 0 15 Oct 2024
Recent Advances of Multimodal Continual Learning: A Comprehensive Survey Dianzhi Yu Xinni Zhang Yankai Chen Aiwei Liu Yifei Zhang Philip S. Yu Irwin King VLM CLL 39 9 0 07 Oct 2024
Solution for OOD-CV UNICORN Challenge 2024 Object Detection Assistance LLM Counting Ability Improvement Zhouyang Chi Qingyuan Jiang Yang Yang 19 0 0 05 Oct 2024
Revisiting Prefix-tuning: Statistical Benefits of Reparameterization among Prompts Minh Le Chau Nguyen Huy Nguyen Quyen Tran Trung Le Nhat Ho 30 3 0 03 Oct 2024
Theoretical Insights into Fine-Tuning Attention Mechanism: Generalization and Optimization Xinhao Yao Hongjin Qian Xiaolin Hu Gengze Xu Wei Liu Jian Luan B. Wang Y. Liu 48 0 0 03 Oct 2024
Contextual Compression in Retrieval-Augmented Generation for Large Language Models: A Survey Sourav Verma RALM 3DV 24 2 0 20 Sep 2024
Multi-Modal Parameter-Efficient Fine-tuning via Graph Neural Network Bin Cheng Jiaxuan Lu 22 0 0 01 Aug 2024
Forecast-PEFT: Parameter-Efficient Fine-Tuning for Pre-trained Motion Forecasting Models Jifeng Wang Kaouther Messaoud Yuejiang Liu Juergen Gall Alexandre Alahi 34 0 0 28 Jul 2024
Straightforward Layer-wise Pruning for More Efficient Visual Adaptation Ruizi Han Jinglei Tang 41 1 0 19 Jul 2024
Soft Begging: Modular and Efficient Shielding of LLMs against Prompt Injection and Jailbreaking based on Prompt Tuning Simon Ostermann Kevin Baum Christoph Endres Julia Masloh P. Schramowski AAML 41 1 0 03 Jul 2024
LoRA-Guard: Parameter-Efficient Guardrail Adaptation for Content Moderation of Large Language Models Hayder Elesedy Pedro M. Esperança Silviu Vlad Oprea Mete Ozay KELM 18 2 0 03 Jul 2024
Exploring Advanced Large Language Models with LLMsuite Giorgio Roffo LLMAG 17 0 0 01 Jul 2024
Structured Unrestricted-Rank Matrices for Parameter Efficient Fine-tuning Arijit Sehanobish Avinava Dubey Krzysztof Choromanski Somnath Basu Roy Chowdhury Deepali Jain Vikas Sindhwani Snigdha Chaturvedi ALM 27 1 0 25 Jun 2024
Synergizing Foundation Models and Federated Learning: A Survey Shenghui Li Fanghua Ye Meng Fang Jiaxu Zhao Yun-Hin Chan Edith C. -H. Ngai Thiemo Voigt AI4CE 43 5 0 18 Jun 2024
Compress then Serve: Serving Thousands of LoRA Adapters with Little Overhead Rickard Brüel-Gabrielsson Jiacheng Zhu Onkar Bhardwaj Leshem Choshen Kristjan Greenewald Mikhail Yurochkin Justin Solomon 30 5 0 17 Jun 2024
A Survey on Large Language Models from General Purpose to Medical Applications: Datasets, Methodologies, and Evaluations Jinqiang Wang Huansheng Ning Yi Peng Qikai Wei Daniel Tesfai Wenwei Mao Tao Zhu Runhe Huang LM&MA AI4MH ELM 38 4 0 14 Jun 2024
MiLoRA: Harnessing Minor Singular Components for Parameter-Efficient LLM Finetuning Hanqing Wang Zeguan Xiao Shuo Wang Guanhua Chen Guanhua Chen 30 19 0 13 Jun 2024
SuperPos-Prompt: Enhancing Soft Prompt Tuning of Language Models with Superposition of Multi Token Embeddings MohammadAli SadraeiJavaeri Ehsaneddin Asgari A. Mchardy Hamid R. Rabiee VLM AAML 19 0 0 07 Jun 2024
Low-Rank Few-Shot Adaptation of Vision-Language Models Maxime Zanella Ismail Ben Ayed OffRL VLM 46 26 0 28 May 2024
Semantic are Beacons: A Semantic Perspective for Unveiling Parameter-Efficient Fine-Tuning in Knowledge Learning Renzhi Wang Piji Li 35 4 0 28 May 2024
SA-FedLora: Adaptive Parameter Allocation for Efficient Federated Learning with LoRA Tuning Yuning Yang Xiaohong Liu Tianrun Gao Xiaodong Xu Guangyu Wang 35 5 0 15 May 2024
Refining Joint Text and Source Code Embeddings for Retrieval Task with Parameter-Efficient Fine-Tuning Karim Galliamov Leila Khaertdinova Karina Denisova 24 1 0 07 May 2024
Random Masking Finds Winning Tickets for Parameter Efficient Fine-tuning Jing Xu Jingzhao Zhang 34 6 0 04 May 2024
A Survey of Time Series Foundation Models: Generalizing Time Series Representation with Large Language Model Jiexia Ye Weiqi Zhang Ke Yi Yongzi Yu Ziyue Li Jia Li Fugee Tsung AI4TS AI4CE 43 23 0 03 May 2024
Exploring the landscape of large language models: Foundations, techniques, and challenges M. Moradi Ke Yan David Colwell Matthias Samwald Rhona Asgari OffRL 42 1 0 18 Apr 2024
LoRA Dropout as a Sparsity Regularizer for Overfitting Control Yang Lin Xinyu Ma Xu Chu Yujie Jin Zhibang Yang Yasha Wang Hong-yan Mei 42 19 0 15 Apr 2024