v1v2v3 (latest)

UniPELT: A Unified Framework for Parameter-Efficient Language Model Tuning

14 October 2021

Hao Ma

Madian Khabsa

Papers citing "UniPELT: A Unified Framework for Parameter-Efficient Language Model Tuning"

50 / 134 papers shown

Title
PrunePEFT: Iterative Hybrid Pruning for Parameter-Efficient Fine-tuning of LLMs Tongzhou Yu Zhuhao Zhang Guanghui Zhu Shen Jiang Meikang Qiu Yihua Huang 36 0 0 09 Jun 2025
MoA: Heterogeneous Mixture of Adapters for Parameter-Efficient Fine-Tuning of Large Language Models Jie Cao Tianwei Lin Hongyang He Rolan Yan Wenqiao Zhang Juncheng Billy Li D. Zhang Siliang Tang Yueting Zhuang MoE 48 0 0 06 Jun 2025
Continuous Subspace Optimization for Continual Learning Quan Cheng Yuanyu Wan Lingyu Wu Chenping Hou Lijun Zhang CLL 75 0 0 17 May 2025
Investigating Task Arithmetic for Zero-Shot Information Retrieval Marco Braga Pranav Kasela Alessandro Raganato G. Pasi RALM 131 0 0 01 May 2025
Histogram-based Parameter-efficient Tuning for Passive Sonar Classification Amirmohammad Mohammadi Davelle Carreiro A. V. Dine Joshua Peeples 100 0 0 21 Apr 2025
FISH-Tuning: Enhancing PEFT Methods with Fisher Information Kang Xue Ming Dong Xinhui Tu Tingting He 191 0 0 05 Apr 2025
Named Entity Recognition in Context Colin Brisson Ayoub Kahfy Marc Bui Frédéric Constant 138 0 0 26 Mar 2025
CLLoRA: An Approach to Measure the Effects of the Context Length for LLM Fine-Tuning Ping Zhang Zhaorui Zhang Sheng Di Yao Xin Benben Liu 99 2 0 26 Feb 2025
NEAT: Nonlinear Parameter-efficient Adaptation of Pre-trained Models Yibo Zhong Haoxiang Jiang Lincan Li Ryumei Nakada Tianci Liu Linjun Zhang Huaxiu Yao Haoyu Wang 253 3 0 24 Feb 2025
LowRA: Accurate and Efficient LoRA Fine-Tuning of LLMs under 2 Bits Zikai Zhou Qizheng Zhang Hermann Kumbong Kunle Olukotun MQ 616 0 0 12 Feb 2025
TabPFN Unleashed: A Scalable and Effective Solution to Tabular Classification Problems Si-Yang Liu Han-Jia Ye 179 12 0 04 Feb 2025
Memory-Efficient Fine-Tuning of Transformers via Token Selection Antoine Simoulin Namyong Park Xiaoyi Liu Grey Yang 195 1 0 31 Jan 2025
Safe Gradient Flow for Bilevel Optimization Sina Sharifi Nazanin Abolfazli Erfan Yazdandoost Hamedani Mahyar Fazlyab 84 0 0 27 Jan 2025
Parameter-Efficient Fine-Tuning for Foundation Models Dan Zhang Tao Feng Lilong Xue Yuandong Wang Yuxiao Dong J. Tang 234 12 0 23 Jan 2025
CM3T: Framework for Efficient Multimodal Learning for Inhomogeneous Interaction Datasets Tanay Agrawal Mohammed Guermal Michal Balazia François Brémond 68 0 0 08 Jan 2025
KaSA: Knowledge-Aware Singular-Value Adaptation of Large Language Models Fan Wang Juyong Jiang Chansung Park Sunghun Kim Jing Tang 203 2 0 08 Dec 2024
A Practical Guide to Fine-tuning Language Models with Limited Data Márton Szép Daniel Rueckert Rüdiger von Eisenhart-Rothe Florian Hinterwimmer SyDa ALM 130 2 0 14 Nov 2024
MambaPEFT: Exploring Parameter-Efficient Fine-Tuning for Mamba Masakazu Yoshimura Teruaki Hayashi Yota Maeda Mamba 300 2 0 06 Nov 2024
Parameter-Efficient Fine-Tuning in Large Models: A Survey of Methodologies Liwen Wang Sheng Chen Linnan Jiang Shu Pan Runze Cai Sen Yang Fei Yang 176 7 0 24 Oct 2024
LoRA-C: Parameter-Efficient Fine-Tuning of Robust CNN for IoT Devices Chuntao Ding Xu Cao Jianhang Xie Linlin Fan Shangguang Wang Zhichao Lu 89 1 0 22 Oct 2024
Towards Optimal Adapter Placement for Efficient Transfer Learning Aleksandra I. Nowak Otniel-Bogdan Mercea Anurag Arnab Jonas Pfeiffer Yann N. Dauphin Utku Evci 76 1 0 21 Oct 2024
Layer-wise Importance Matters: Less Memory for Better Performance in Parameter-efficient Fine-tuning of Large Language Models Kai Yao P. Gao Lichun Li Yuan Zhao Xiaofeng Wang Wei Wang Jianke Zhu 56 2 0 15 Oct 2024
Lessons and Insights from a Unifying Study of Parameter-Efficient Fine-Tuning (PEFT) in Visual Recognition Zheda Mai Ping Zhang Cheng-Hao Tu Hong-You Chen Li Zhang Wei-Lun Chao 41 1 0 24 Sep 2024
Multiple-Exit Tuning: Towards Inference-Efficient Adaptation for Vision Transformer Zheng Liu Jinchao Zhu Nannan Li Gao Huang 84 0 0 21 Sep 2024
AI Act for the Working Programmer Holger Hermanns Anne Lauber-Rönsberg Philip Meinel Sarah Sterz Hanwei Zhang 69 1 0 23 Jul 2024
Enhancing Parameter Efficiency and Generalization in Large-Scale Models: A Regularized and Masked Low-Rank Adaptation Approach Yuzhu Mao Siqi Ping Zihao Zhao Yang Liu Wenbo Ding 93 1 0 16 Jul 2024
Foundation Model Engineering: Engineering Foundation Models Just as Engineering Software Dezhi Ran Mengzhou Wu Wei Yang Tao Xie AI4CE 79 2 0 11 Jul 2024
Hadamard Adapter: An Extreme Parameter-Efficient Adapter Tuning Method for Pre-trained Language Models Yuyan Chen Qiang Fu Ge Fan Lun Du Jian-Guang Lou Shi Han Dongmei Zhang Zhixu Li Yanghua Xiao MoE 79 17 0 04 Jul 2024
HyperLoader: Integrating Hypernetwork-Based LoRA and Adapter Layers into Multi-Task Transformers for Sequence Labelling Jesús-Germán Ortiz-Barajas Helena Gómez-Adorno Thamar Solorio 58 2 0 01 Jul 2024
A Survey on Mixture of Experts in Large Language Models Weilin Cai Juyong Jiang Fan Wang Jing Tang Sunghun Kim Jiayi Huang MoE 66 70 0 26 Jun 2024
Structured Unrestricted-Rank Matrices for Parameter Efficient Fine-tuning Arijit Sehanobish Avinava Dubey Krzysztof Choromanski Somnath Basu Roy Chowdhury Deepali Jain Vikas Sindhwani Snigdha Chaturvedi ALM 81 3 0 25 Jun 2024
Fighting Randomness with Randomness: Mitigating Optimisation Instability of Fine-Tuning using Delayed Ensemble and Noisy Interpolation Branislav Pecher Ján Cegin Róbert Belanec Jakub Simko Ivan Srba Maria Bielikova 83 1 0 18 Jun 2024
Adapter-X: A Novel General Parameter-Efficient Fine-Tuning Framework for Vision Minglei Li Peng Ye Yongqi Huang Lin Zhang Tao Chen Tong He Jiayuan Fan Wanli Ouyang MoE 117 5 0 05 Jun 2024
Low-Rank Adaption on Transformer-based Oriented Object Detector for Satellite Onboard Processing of Remote Sensing Images Xinyang Pu Feng Xu 70 3 0 04 Jun 2024
Random Masking Finds Winning Tickets for Parameter Efficient Fine-tuning Jing Xu Jingzhao Zhang 102 7 0 04 May 2024
Towards Neural Synthesis for SMT-Assisted Proof-Oriented Programming Saikat Chakraborty Gabriel Ebner Siddharth Bhat Sarah Fakhoury Sakina Fatima Shuvendu K. Lahiri Nikhil Swamy 80 16 0 03 May 2024
RST-LoRA: A Discourse-Aware Low-Rank Adaptation for Long Document Abstractive Summarization Dongqi Pu Vera Demberg 138 6 0 01 May 2024
Towards Incremental Learning in Large Language Models: A Critical Review M. Jovanovic Peter Voss ELM CLL KELM 116 5 0 28 Apr 2024
Efficient Remote Sensing with Harmonized Transfer Learning and Modality Alignment Tengjun Huang 117 0 0 28 Apr 2024
Navigating the Landscape of Large Language Models: A Comprehensive Review and Analysis of Paradigms and Fine-Tuning Strategies Benjue Weng LM&MA 117 10 0 13 Apr 2024
DLoRA: Distributed Parameter-Efficient Fine-Tuning Solution for Large Language Model Chao Gao Sai Qian Zhang ALM 155 7 0 08 Apr 2024
Facial Affective Behavior Analysis with Instruction Tuning Yifan Li Anh Dao Wentao Bao Zhen Tan Tianlong Chen Huan Liu Yu Kong CVBM 116 15 0 07 Apr 2024
Unlocking Parameter-Efficient Fine-Tuning for Low-Resource Language Translation Tong Su Xin Peng Sarubi Thillainathan David Guzmán Surangika Ranathunga En-Shiun Annie Lee 65 3 0 05 Apr 2024
Parameter Efficient Quasi-Orthogonal Fine-Tuning via Givens Rotation Xinyu Ma Xu Chu Zhibang Yang Yang Lin Xin Gao Junfeng Zhao 96 10 0 05 Apr 2024
Query-driven Relevant Paragraph Extraction from Legal Judgments Santosh T.Y.S.S Elvin Quero Hernandez Matthias Grabmair AILaw 95 2 0 31 Mar 2024
Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Survey Zeyu Han Chao Gao Jinyang Liu Jeff Zhang Sai Qian Zhang 297 403 0 21 Mar 2024
BiLoRA: A Bi-level Optimization Framework for Overfitting-Resilient Low-Rank Adaptation of Large Pre-trained Models Rushi Qiang Ruiyi Zhang Pengtao Xie AI4CE 72 10 0 19 Mar 2024
Introducing Routing Functions to Vision-Language Parameter-Efficient Fine-Tuning with Low-Rank Bottlenecks Tingyu Qu Tinne Tuytelaars Marie-Francine Moens MoE 84 2 0 14 Mar 2024
Dynamic Adapter Meets Prompt Tuning: Parameter-Efficient Transfer Learning for Point Cloud Analysis Xin Zhou Dingkang Liang Wei Xu Xingkui Zhu Yihan Xu Zhikang Zou Xiang Bai 94 28 0 03 Mar 2024
FlexLLM: A System for Co-Serving Large Language Model Inference and Parameter-Efficient Finetuning Xupeng Miao Gabriele Oliaro Xinhao Cheng Vineeth Kada Ruohan Gao ... April Yang Yingcheng Wang Mengdi Wu Colin Unger Zhihao Jia MoE 173 10 0 29 Feb 2024