Parameter-Efficient Transfer Learning for NLP

2 February 2019

N. Houlsby

A. Giurgiu

Stanislaw Jastrzebski

Bruna Morrone

Quentin de Laroussilhe

Papers citing "Parameter-Efficient Transfer Learning for NLP"

50 / 729 papers shown

Title
Save It All: Enabling Full Parameter Tuning for Federated Large Language Models via Cycle Block Gradient Descent Lin Wang Zhichao Wang Xiaoying Tang 37 1 0 17 Jun 2024
On Giant's Shoulders: Effortless Weak to Strong by Dynamic Logits Fusion Chenghao Fan Zhenyi Lu Wei Wei Jie Tian Xiaoye Qu Dangyang Chen Yu Cheng MoMe 48 5 0 17 Jun 2024
Compress then Serve: Serving Thousands of LoRA Adapters with Little Overhead Rickard Brüel-Gabrielsson Jiacheng Zhu Onkar Bhardwaj Leshem Choshen Kristjan Greenewald Mikhail Yurochkin Justin Solomon 38 5 0 17 Jun 2024
Promoting Data and Model Privacy in Federated Learning through Quantized LoRA Jianhao Zhu Changze Lv Xiaohua Wang Muling Wu Wenhao Liu Tianlong Li Zixuan Ling Cenyuan Zhang Xiaoqing Zheng Xuanjing Huang 36 2 0 16 Jun 2024
ShareLoRA: Parameter Efficient and Robust Large Language Model Fine-tuning via Shared Low-Rank Adaptation Yurun Song Junchen Zhao Ian G. Harris S. Jyothi 32 3 0 16 Jun 2024
A Survey on Large Language Models from General Purpose to Medical Applications: Datasets, Methodologies, and Evaluations Jinqiang Wang Huansheng Ning Yi Peng Qikai Wei Daniel Tesfai Wenwei Mao Tao Zhu Runhe Huang LM&MA AI4MH ELM 42 4 0 14 Jun 2024
MiLoRA: Harnessing Minor Singular Components for Parameter-Efficient LLM Finetuning Hanqing Wang Zeguan Xiao Shuo Wang Guanhua Chen Guanhua Chen 44 19 0 13 Jun 2024
The Impact of Initialization on LoRA Finetuning Dynamics Soufiane Hayou Nikhil Ghosh Bin Yu AI4CE 36 10 0 12 Jun 2024
Crayon: Customized On-Device LLM via Instant Adapter Blending and Edge-Server Hybrid Inference Jihwan Bang Juntae Lee Kyuhong Shim Seunghan Yang Simyung Chang 29 5 0 11 Jun 2024
Learning Fine-Grained Controllability on Speech Generation via Efficient Fine-Tuning Chung-Ming Chien Andros Tjandra Apoorv Vyas Matt Le Bowen Shi Wei-Ning Hsu 32 0 0 10 Jun 2024
CorDA: Context-Oriented Decomposition Adaptation of Large Language Models for Task-Aware Parameter-Efficient Fine-tuning Yibo Yang Xiaojie Li Zhongzhu Zhou S. Song Jianlong Wu Liqiang Nie Bernard Ghanem 45 6 0 07 Jun 2024
Light-PEFT: Lightening Parameter-Efficient Fine-Tuning via Early Pruning Naibin Gu Peng Fu Xiyu Liu Bowen Shen Zheng-Shen Lin Weiping Wang 30 6 0 06 Jun 2024
HYDRA: Model Factorization Framework for Black-Box LLM Personalization Yuchen Zhuang Haotian Sun Yue Yu Rushi Qiang Qifan Wang Chao Zhang Bo Dai AAML 43 14 0 05 Jun 2024
Inv-Adapter: ID Customization Generation via Image Inversion and Lightweight Adapter Peng-Fei Xing Ning Wang Jianbo Ouyang Zechao Li DiffM 38 1 0 05 Jun 2024
Low-Rank Adaption on Transformer-based Oriented Object Detector for Satellite Onboard Processing of Remote Sensing Images Xinyang Pu Feng Xu 32 3 0 04 Jun 2024
TS-Align: A Teacher-Student Collaborative Framework for Scalable Iterative Finetuning of Large Language Models Chen Zhang Chengguang Tang Dading Chong Ke Shi Guohua Tang Feng Jiang Haizhou Li 31 4 0 30 May 2024
One Token Can Help! Learning Scalable and Pluggable Virtual Tokens for Retrieval-Augmented Large Language Models Yutao Zhu Zhaoheng Huang Zhicheng Dou Ji-Rong Wen RALM 56 5 0 30 May 2024
LLaMA-Reg: Using LLaMA 2 for Unsupervised Medical Image Registration Mingrui Ma Yu Yang LM&MA 32 2 0 29 May 2024
$$\textit{Trans-LoRA}$: towards data-free Transferable Parameter Efficient Finetuning$ $\textit{Trans-LoRA}$ : towards data-free Transferable Parameter Efficient Finetuning Runqian Wang Soumya Ghosh David D. Cox Diego Antognini Aude Oliva Rogerio Feris Leonid Karlinsky 32 1 0 27 May 2024
Generating clickbait spoilers with an ensemble of large language models M. Woźny Mateusz Lango 21 1 0 25 May 2024
Large Language Models Meet NLP: A Survey Libo Qin Qiguang Chen Xiachong Feng Yang Wu Yongheng Zhang Yinghui Li Min Li Wanxiang Che Philip S. Yu ALM LM&MA ELM LRM 40 47 0 21 May 2024
Towards Modular LLMs by Building and Reusing a Library of LoRAs O. Ostapenko Zhan Su E. Ponti Laurent Charlin Nicolas Le Roux Matheus Pereira Lucas Page-Caccia Alessandro Sordoni MoMe 34 30 0 18 May 2024
Low-Rank Adaptation of Time Series Foundational Models for Out-of-Domain Modality Forecasting Divij Gupta Anubhav Bhatti Surajsinh Parmar Chen Dan Yuwei Liu Bingjie Shen San Lee AI4TS 35 2 0 16 May 2024
DARA: Domain- and Relation-aware Adapters Make Parameter-efficient Tuning for Visual Grounding Ting Liu Xuyang Liu Siteng Huang Honggang Chen Quanjun Yin Long Qin Donglin Wang Yue Hu 35 5 0 10 May 2024
DP-DyLoRA: Fine-Tuning Transformer-Based Models On-Device under Differentially Private Federated Learning using Dynamic Low-Rank Adaptation Jie Xu Karthikeyan P. Saravanan Rogier van Dalen Haaris Mehmood David Tuckey Mete Ozay 56 5 0 10 May 2024
Large Language Models for Cyber Security: A Systematic Literature Review HanXiang Xu Shenao Wang Ningke Li K. Wang Yanjie Zhao Kai Chen Ting Yu Yang Janet Liu H. Wang 31 23 0 08 May 2024
PhilHumans: Benchmarking Machine Learning for Personal Health Vadim Liventsev Vivek Kumar Allmin Pradhap Singh Susaiyah Zixiu "Alex" Wu Ivan Rodin ... Milan Petkovic Diego Reforgiato Recupero Ehud Reiter Daniele Riboni Raymond Sterling AI4MH LM&MA 34 0 0 04 May 2024
The Trade-off between Performance, Efficiency, and Fairness in Adapter Modules for Text Classification Minh Duc Bui K. Wense 31 0 0 03 May 2024
FeDeRA:Efficient Fine-tuning of Language Models in Federated Learning Leveraging Weight Decomposition Yuxuan Yan Qianqian Yang Shunpu Tang Zhiguo Shi 38 13 0 29 Apr 2024
Simplifying Multimodality: Unimodal Approach to Multimodal Challenges in Radiology with General-Domain Large Language Model Seonhee Cho Choonghan Kim Jiho Lee Chetan Chilkunda Sujin Choi Joo Heung Yoon 48 0 0 29 Apr 2024
HFT: Half Fine-Tuning for Large Language Models Tingfeng Hui Zhenyu Zhang Shuohuan Wang Weiran Xu Yu Sun Hua-Hong Wu CLL 37 4 0 29 Apr 2024
Do Vision & Language Decoders use Images and Text equally? How Self-consistent are their Explanations? Letitia Parcalabescu Anette Frank MLLM CoGe VLM 82 3 0 29 Apr 2024
MRScore: Evaluating Radiology Report Generation with LLM-based Reward System Yunyi Liu Zhanyu Wang Yingshu Li Xinyu Liang Lingqiao Liu Lei Wang Luping Zhou LM&MA 11 3 0 27 Apr 2024
HateTinyLLM : Hate Speech Detection Using Tiny Large Language Models Tanmay Sen Ansuman Das Mrinmay Sen 39 4 0 26 Apr 2024
No Train but Gain: Language Arithmetic for training-free Language Adapters enhancement Mateusz Klimaszewski Piotr Andruszkiewicz Alexandra Birch MoMe 42 4 0 24 Apr 2024
Ultrasound SAM Adapter: Adapting SAM for Breast Lesion Segmentation in Ultrasound Images Zhengzheng Tu Le Gu Xixi Wang Bo Jiang MedIm 41 3 0 23 Apr 2024
Unified Unsupervised Salient Object Detection via Knowledge Transfer Yao Yuan Wutao Liu Pan Gao Qun Dai Jie Qin 32 4 0 23 Apr 2024
ColA: Collaborative Adaptation with Gradient Learning Enmao Diao Qi Le Suya Wu Xinran Wang Ali Anwar Jie Ding Vahid Tarokh 24 1 0 22 Apr 2024
Adaptive Memory Replay for Continual Learning James Seale Smith Lazar Valkov Shaunak Halbe V. Gutta Rogerio Feris Z. Kira Leonid Karlinsky 36 6 0 18 Apr 2024
Personalized Collaborative Fine-Tuning for On-Device Large Language Models Nicolas Wagner Dongyang Fan Martin Jaggi 37 3 0 15 Apr 2024
LoRA Dropout as a Sparsity Regularizer for Overfitting Control Yang Lin Xinyu Ma Xu Chu Yujie Jin Zhibang Yang Yasha Wang Hong-yan Mei 46 19 0 15 Apr 2024
How to build the best medical image segmentation algorithm using foundation models: a comprehensive empirical study with Segment Anything Model Han Gu Haoyu Dong Jichen Yang Maciej Mazurowski MedIm VLM 75 12 0 15 Apr 2024
Navigating the Landscape of Large Language Models: A Comprehensive Review and Analysis of Paradigms and Fine-Tuning Strategies Benjue Weng LM&MA 35 7 0 13 Apr 2024
Improving Continuous Sign Language Recognition with Adapted Image Models Lianyu Hu Tongkai Shi Liqing Gao Zekang Liu Wei Feng VLM 18 5 0 12 Apr 2024
AdapterSwap: Continuous Training of LLMs with Data Removal and Access-Control Guarantees William Fleshman Aleem Khan Marc Marone Benjamin Van Durme CLL KELM 50 3 0 12 Apr 2024
Data-Augmentation-Based Dialectal Adaptation for LLMs Fahim Faisal Antonios Anastasopoulos 31 2 0 11 Apr 2024
Automating Research Synthesis with Domain-Specific Large Language Model Fine-Tuning Teo Susnjak Peter Hwang N. Reyes A. Barczak Timothy R. McIntosh Surangika Ranathunga 68 22 0 08 Apr 2024
A Note on LoRA Vlad Fomenko Han Yu Jongho Lee Stanley Hsieh Weizhu Chen 30 5 0 07 Apr 2024
Unlocking Parameter-Efficient Fine-Tuning for Low-Resource Language Translation Tong Su Xin Peng Sarubi Thillainathan David Guzmán Surangika Ranathunga En-Shiun Annie Lee 35 2 0 05 Apr 2024
Parameter Efficient Quasi-Orthogonal Fine-Tuning via Givens Rotation Xinyu Ma Xu Chu Zhibang Yang Yang Lin Xin Gao Junfeng Zhao 38 6 0 05 Apr 2024