SparseAdapter: An Easy Approach for Improving the Parameter-Efficiency of Adapters

9 October 2022

Liang Ding

Papers citing "SparseAdapter: An Easy Approach for Improving the Parameter-Efficiency of Adapters"

50 / 63 papers shown

Title
Concept-Aware LoRA for Domain-Aligned Segmentation Dataset Generation Minho Park S. Park Jungsoo Lee Hyojin Park Kyuwoong Hwang Fatih Porikli Jaegul Choo Sungha Choi 44 0 0 28 Mar 2025
Faster Parameter-Efficient Tuning with Token Redundancy Reduction Kwonyoung Kim Jungin Park Jin-Hwa Kim Hyeongjun Kwon Kwanghoon Sohn 75 0 0 26 Mar 2025
Sparse High Rank Adapters K. Bhardwaj N. Pandey Sweta Priyadarshi Viswanath Ganapathy Rafael Esteves ... P. Whatmough Risheek Garrepalli M. V. Baalen Harris Teague Markus Nagel MQ 43 4 0 28 Jan 2025
Parameter-Efficient Fine-Tuning for Foundation Models Dan Zhang Tao Feng Lilong Xue Yuandong Wang Yuxiao Dong J. Tang 56 9 0 23 Jan 2025
An Efficient Adaptive Compression Method for Human Perception and Machine Vision Tasks Lei Liu Zhenghao Chen Zhihao Hu Dong Xu 44 4 0 08 Jan 2025
AdaRankGrad: Adaptive Gradient-Rank and Moments for Memory-Efficient LLMs Training and Fine-Tuning Yehonathan Refael Jonathan Svirsky Boris Shustin Wasim Huleihel Ofir Lindenbaum 52 3 0 31 Dec 2024
FineGates: LLMs Finetuning with Compression using Stochastic Gates Jonathan Svirsky Yehonathan Refael Ofir Lindenbaum 84 0 0 17 Dec 2024
Rethinking Weight Decay for Robust Fine-Tuning of Foundation Models Junjiao Tian Chengyue Huang Z. Kira 44 1 0 03 Nov 2024
LoRA Done RITE: Robust Invariant Transformation Equilibration for LoRA Optimization Jui-Nan Yen Si Si Zhao Meng Felix X. Yu Sai Surya Duvvuri Inderjit Dhillon Cho-Jui Hsieh Sanjiv Kumar 35 3 0 27 Oct 2024
Parameter-Efficient Fine-Tuning in Large Models: A Survey of Methodologies Liwen Wang Sheng Chen Linnan Jiang Shu Pan Runze Cai Sen Yang Fei Yang 52 3 0 24 Oct 2024
CogSteer: Cognition-Inspired Selective Layer Intervention for Efficient Semantic Steering in Large Language Models Xintong Wang Jingheng Pan Longqin Jiang Liang Ding Xingshan Li Chris Biemann LLMSV 34 0 0 23 Oct 2024
Communication-Efficient and Tensorized Federated Fine-Tuning of Large Language Models Sajjad Ghiasvand Yifan Yang Zhiyu Xue Mahnoosh Alizadeh Zheng Zhang Ramtin Pedarsani FedML 43 3 0 16 Oct 2024
Layer-wise Importance Matters: Less Memory for Better Performance in Parameter-efficient Fine-tuning of Large Language Models Kai Yao P. Gao Lichun Li Yuan Zhao Xiaofeng Wang Wei Wang Jianke Zhu 26 1 0 15 Oct 2024
FLoRA: Federated Fine-Tuning Large Language Models with Heterogeneous Low-Rank Adaptations Ziyao Wang Zheyu Shen Yexiao He Guoheng Sun Hongyi Wang Lingjuan Lyu Ang Li 37 30 0 09 Sep 2024
Step-by-Step Unmasking for Parameter-Efficient Fine-tuning of Large Language Models Aradhye Agarwal Suhas Kamasetty Ramesh Ayan Sengupta Tanmoy Chakraborty 30 1 0 26 Aug 2024
Understanding the Performance and Estimating the Cost of LLM Fine-Tuning Yuchen Xia Jiho Kim Yuhan Chen Haojie Ye Souvik Kundu Cong Hao Nishil Talati MoE 40 22 0 08 Aug 2024
CELLM: An Efficient Communication in Large Language Models Training for Federated Learning Raja Vavekanand Kira Sam 58 0 0 30 Jul 2024
Rapid Switching and Multi-Adapter Fusion via Sparse High Rank Adapters Kartikeya Bhardwaj N. Pandey Sweta Priyadarshi Viswanath Ganapathy Rafael Esteves ... Paul N. Whatmough Risheek Garrepalli M. V. Baalen Harris Teague Markus Nagel MoMe 35 0 0 22 Jul 2024
Foundation Model Engineering: Engineering Foundation Models Just as Engineering Software Dezhi Ran Mengzhou Wu Wei Yang Tao Xie AI4CE 44 1 0 11 Jul 2024
Increasing Model Capacity for Free: A Simple Strategy for Parameter Efficient Fine-tuning Haobo Song Hao Zhao Soumajit Majumder Tao Lin 30 3 0 01 Jul 2024
Federated LoRA with Sparse Communication Kevin Kuo Arian Raje Kousik Rajesh Virginia Smith 51 8 0 07 Jun 2024
Adapter-X: A Novel General Parameter-Efficient Fine-Tuning Framework for Vision Minglei Li Peng Ye Yongqi Huang Lin Zhang Tao Chen Tong He Jiayuan Fan Wanli Ouyang MoE 47 4 0 05 Jun 2024
QuanTA: Efficient High-Rank Fine-Tuning of LLMs with Quantum-Informed Tensor Adaptation Zhuo Chen Rumen Dangovski Charlotte Loh Owen Dugan Di Luo Marin Soljacic MQ 39 8 0 31 May 2024
Group-aware Parameter-efficient Updating for Content-Adaptive Neural Video Compression Zhenghao Chen Luping Zhou Zhihao Hu Dong Xu 28 10 0 07 May 2024
A Survey of Time Series Foundation Models: Generalizing Time Series Representation with Large Language Model Weiqi Zhang Jiexia Ye Ke Yi Yongzi Yu Ziyue Li Jia Li Fugee Tsung AI4TS AI4CE 51 22 0 03 May 2024
DLoRA: Distributed Parameter-Efficient Fine-Tuning Solution for Large Language Model Chao Gao Sai Qian Zhang ALM 123 7 0 08 Apr 2024
Facial Affective Behavior Analysis with Instruction Tuning Yifan Li Anh Dao Wentao Bao Zhen Tan Tianlong Chen Huan Liu Yu Kong CVBM 67 15 0 07 Apr 2024
Parameter Efficient Quasi-Orthogonal Fine-Tuning via Givens Rotation Xinyu Ma Xu Chu Zhibang Yang Yang Lin Xin Gao Junfeng Zhao 51 7 0 05 Apr 2024
ReFT: Representation Finetuning for Language Models Zhengxuan Wu Aryaman Arora Zheng Wang Atticus Geiger Daniel Jurafsky Christopher D. Manning Christopher Potts OffRL 43 58 0 04 Apr 2024
Rethinking Pruning for Vision-Language Models: Strategies for Effective Sparsity and Performance Restoration Shwai He Ang Li Tianlong Chen VLM 67 1 0 03 Apr 2024
Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Survey Zeyu Han Chao Gao Jinyang Liu Jeff Zhang Sai Qian Zhang 150 319 0 21 Mar 2024
FineDiffusion: Scaling up Diffusion Models for Fine-grained Image Generation with 10,000 Classes Ziying Pan Kun Wang Gang Li Feihong He Yongxuan Lai 42 0 0 28 Feb 2024
Healthcare Copilot: Eliciting the Power of General LLMs for Medical Consultation Zhiyao Ren Yibing Zhan Baosheng Yu Liang Ding Dacheng Tao LM&MA 34 13 0 20 Feb 2024
Generalizability of Mixture of Domain-Specific Adapters from the Lens of Signed Weight Directions and its Application to Effective Model Pruning Tuc Nguyen Thai Le MoMe 28 3 0 16 Feb 2024
Model Compression and Efficient Inference for Large Language Models: A Survey Wenxiao Wang Wei Chen Yicong Luo Yongliu Long Zhengkai Lin Liye Zhang Binbin Lin Deng Cai Xiaofei He MQ 46 48 0 15 Feb 2024
APT: Adaptive Pruning and Tuning Pretrained Language Models for Efficient Training and Inference Bowen Zhao Hannaneh Hajishirzi Qingqing Cao 34 17 0 22 Jan 2024
PersianMind: A Cross-Lingual Persian-English Large Language Model Pedram Rostami Ali Salemi M. Dousti CLL LRM 37 5 0 12 Jan 2024
PERP: Rethinking the Prune-Retrain Paradigm in the Era of LLMs Max Zimmer Megi Andoni Christoph Spiegel Sebastian Pokutta VLM 55 10 0 23 Dec 2023
Parameter-Efficient Fine-Tuning Methods for Pretrained Language Models: A Critical Review and Assessment Lingling Xu Haoran Xie S. J. Qin Xiaohui Tao F. Wang 65 137 0 19 Dec 2023
MoSA: Mixture of Sparse Adapters for Visual Efficient Tuning Qizhe Zhang Bocheng Zou Ruichuan An Jiaming Liu Shanghang Zhang MoE 29 2 0 05 Dec 2023
Mini but Mighty: Finetuning ViTs with Mini Adapters Imad Eddine Marouf Enzo Tartaglione Stéphane Lathuilière 41 5 0 07 Nov 2023
On Surgical Fine-tuning for Language Encoders Abhilasha Lodha Gayatri Belapurkar Saloni Chalkapurkar Yuanming Tao Reshmi Ghosh Samyadeep Basu Dmitrii Petrov Soundararajan Srinivasan 33 3 0 25 Oct 2023
Zero-Shot Sharpness-Aware Quantization for Pre-trained Language Models Miaoxi Zhu Qihuang Zhong Li Shen Liang Ding Juhua Liu Bo Du Dacheng Tao MQ VLM 34 1 0 20 Oct 2023
Decomposed Prompt Tuning via Low-Rank Reparameterization Yao Xiao Lu Xu Jiaxi Li Wei Lu Xiaoli Li VLM 25 6 0 16 Oct 2023
Unlikelihood Tuning on Negative Samples Amazingly Improves Zero-Shot Translation Junjie Yang Liang Ding Li Shen Matthieu Labeau Yibing Zhan Weifeng Liu Dacheng Tao VLM 45 4 0 28 Sep 2023
MerA: Merging Pretrained Adapters For Few-Shot Learning Shwai He Run-Ze Fan Liang Ding Li Shen Dinesh Manocha Dacheng Tao MoMe 42 11 0 30 Aug 2023
Fine-Tuning Llama 2 Large Language Models for Detecting Online Sexual Predatory Chats and Abusive Texts Thanh Thi Nguyen Campbell Wilson Janis Dalins 32 23 0 28 Aug 2023
IncreLoRA: Incremental Parameter Allocation Method for Parameter-Efficient Fine-tuning Feiyu F. Zhang Liangzhi Li Jun-Cheng Chen Zhouqian Jiang Bowen Wang Yiming Qian 51 33 0 23 Aug 2023
Revisiting the Parameter Efficiency of Adapters from the Perspective of Precision Redundancy Shibo Jie Haoqing Wang Zhiwei Deng 32 31 0 31 Jul 2023
Exploring the Impact of Model Scaling on Parameter-Efficient Tuning Yusheng Su Chi-Min Chan Jiali Cheng Yujia Qin Yankai Lin ... Ning Ding Xingzhi Sun Guotong Xie Zhiyuan Liu Maosong Sun 37 6 0 04 Jun 2023