Scaling Down to Scale Up: A Guide to Parameter-Efficient Fine-Tuning

28 March 2023

Papers citing "Scaling Down to Scale Up: A Guide to Parameter-Efficient Fine-Tuning"

50 / 107 papers shown

Title
Navigating the Landscape of Large Language Models: A Comprehensive Review and Analysis of Paradigms and Fine-Tuning Strategies Benjue Weng LM&MA 35 7 0 13 Apr 2024
Parameter Efficient Quasi-Orthogonal Fine-Tuning via Givens Rotation Xinyu Ma Xu Chu Zhibang Yang Yang Lin Xin Gao Junfeng Zhao 38 6 0 05 Apr 2024
Foundation Model for Advancing Healthcare: Challenges, Opportunities, and Future Directions Yuting He Fuxiang Huang Xinrui Jiang Yuxiang Nie Minghao Wang Jiguang Wang Hao Chen LM&MA AI4CE 71 27 0 04 Apr 2024
FreeSeg-Diff: Training-Free Open-Vocabulary Segmentation with Diffusion Models Barbara Toniella Corradini Mustafa Shukor Paul Couairon Guillaume Couairon Franco Scarselli Matthieu Cord DiffM VLM 38 4 0 29 Mar 2024
Dynamic Tuning Towards Parameter and Inference Efficiency for ViT Adaptation Wangbo Zhao Jiasheng Tang Yizeng Han Yibing Song Kai Wang Gao Huang F. Wang Yang You 35 11 0 18 Mar 2024
PreCurious: How Innocent Pre-Trained Language Models Turn into Privacy Traps Ruixuan Liu Tianhao Wang Yang Cao Li Xiong AAML SILM 35 15 0 14 Mar 2024
Tracking Meets LoRA: Faster Training, Larger Model, Stronger Performance Liting Lin Heng Fan Zhipeng Zhang Yaowei Wang Yong-mei Xu Haibin Ling 42 24 0 08 Mar 2024
Levels of AI Agents: from Rules to Large Language Models Yu Huang AI4CE ELM LM&Ro 38 2 0 06 Mar 2024
Controllable Prompt Tuning For Balancing Group Distributional Robustness Hoang Phan Andrew Gordon Wilson Qi Lei 43 5 0 05 Mar 2024
Asymmetry in Low-Rank Adapters of Foundation Models Jiacheng Zhu Kristjan Greenewald Kimia Nadjahi Haitz Sáez de Ocáriz Borde Rickard Brüel-Gabrielsson Leshem Choshen Marzyeh Ghassemi Mikhail Yurochkin Justin Solomon 42 27 0 26 Feb 2024
Prompting a Pretrained Transformer Can Be a Universal Approximator Aleksandar Petrov Philip H. S. Torr Adel Bibi 26 11 0 22 Feb 2024
On the Byzantine-Resilience of Distillation-Based Federated Learning Christophe Roux Max Zimmer S. Pokutta AAML 49 1 0 19 Feb 2024
T-RAG: Lessons from the LLM Trenches M. Fatehkia J. Lucas Sanjay Chawla LLMAG 32 21 0 12 Feb 2024
Learning to Route Among Specialized Experts for Zero-Shot Generalization Mohammed Muqeeth Haokun Liu Yufan Liu Colin Raffel MoMe 37 33 0 08 Feb 2024
Time-, Memory- and Parameter-Efficient Visual Adaptation Otniel-Bogdan Mercea Alexey Gritsenko Cordelia Schmid Anurag Arnab VLM 35 13 0 05 Feb 2024
HiFT: A Hierarchical Full Parameter Fine-Tuning Strategy Yongkang Liu Yiqun Zhang Qian Li Tong Liu Shi Feng Daling Wang Yifei Zhang Hinrich Schütze 33 6 0 26 Jan 2024
APT: Adaptive Pruning and Tuning Pretrained Language Models for Efficient Training and Inference Bowen Zhao Hannaneh Hajishirzi Qingqing Cao 21 17 0 22 Jan 2024
PersianMind: A Cross-Lingual Persian-English Large Language Model Pedram Rostami Ali Salemi M. Dousti CLL LRM 16 5 0 12 Jan 2024
A Comprehensive Study of Knowledge Editing for Large Language Models Ningyu Zhang Yunzhi Yao Bo Tian Peng Wang Shumin Deng ... Lei Liang Zhiqiang Zhang Xiao-Jun Zhu Jun Zhou Huajun Chen KELM 26 76 0 02 Jan 2024
A Reliable Knowledge Processing Framework for Combustion Science using Foundation Models Vansh Sharma Venkat Raman 21 7 0 31 Dec 2023
A Comprehensive Evaluation of Parameter-Efficient Fine-Tuning on Software Engineering Tasks Wentao Zou Qi Li Jidong Ge Chuanyi Li Xiaoyu Shen LiGuo Huang Bin Luo 24 5 0 25 Dec 2023
PERP: Rethinking the Prune-Retrain Paradigm in the Era of LLMs Max Zimmer Megi Andoni Christoph Spiegel S. Pokutta VLM 50 10 0 23 Dec 2023
Parameter-Efficient Fine-Tuning Methods for Pretrained Language Models: A Critical Review and Assessment Lingling Xu Haoran Xie S. J. Qin Xiaohui Tao F. Wang 30 130 0 19 Dec 2023
Efficiency-oriented approaches for self-supervised speech representation learning Luis Lugo Valentin Vielzeuf SSL 19 1 0 18 Dec 2023
Adapting Vision Transformer for Efficient Change Detection Yang Zhao Yuxiang Zhang Yanni Dong Bo Du VLM 29 2 0 08 Dec 2023
Applications of Large Scale Foundation Models for Autonomous Driving Yu Huang Yue Chen Zhu Li ELM AI4CE LRM ALM LM&Ro 46 15 0 20 Nov 2023
Adapters: A Unified Library for Parameter-Efficient and Modular Transfer Learning Clifton A. Poth Hannah Sterz Indraneil Paul Sukannya Purkayastha Leon Arne Engländer Timo Imhof Ivan Vulić Sebastian Ruder Iryna Gurevych Jonas Pfeiffer 17 45 0 18 Nov 2023
Aggregate, Decompose, and Fine-Tune: A Simple Yet Effective Factor-Tuning Method for Vision Transformer Dongping Chen 28 3 0 12 Nov 2023
Parameter-Efficient Orthogonal Finetuning via Butterfly Factorization Weiyang Liu Zeju Qiu Yao Feng Yuliang Xiu Yuxuan Xue ... Songyou Peng Yandong Wen Michael J. Black Adrian Weller Bernhard Schölkopf 48 56 0 10 Nov 2023
When Do Prompting and Prefix-Tuning Work? A Theory of Capabilities and Limitations Aleksandar Petrov Philip H. S. Torr Adel Bibi VPVLM 25 21 0 30 Oct 2023
A Communication Theory Perspective on Prompting Engineering Methods for Large Language Models Yuanfeng Song Yuanqin He Xuefang Zhao Hanlin Gu Di Jiang Haijun Yang Lixin Fan Qiang Yang 30 3 0 24 Oct 2023
Rethinking Memory and Communication Cost for Efficient Large Language Model Training Chan Wu Hanxiao Zhang Lin Ju Jinjing Huang Youshao Xiao ... Siyuan Li Fanzhuang Meng Lei Liang Xiaolu Zhang Jun Zhou 13 4 0 09 Oct 2023
Profit: Benchmarking Personalization and Robustness Trade-off in Federated Prompt Tuning Liam Collins Shanshan Wu Sewoong Oh K. Sim FedML 34 9 0 06 Oct 2023
Model Compression in Practice: Lessons Learned from Practitioners Creating On-device Machine Learning Experiences Fred Hohman Mary Beth Kery Donghao Ren Dominik Moritz 19 16 0 06 Oct 2023
Empirical Study of PEFT techniques for Winter Wheat Segmentation Mohamad Hasan Zahweh Hasan Nasrallah Mustafa Shukor G. Faour A. Ghandour 12 3 0 03 Oct 2023
MA-SAM: Modality-agnostic SAM Adaptation for 3D Medical Image Segmentation Cheng Chen Juzheng Miao Dufan Wu Zhiling Yan Sekeun Kim ... Lichao Sun Xiang Li Tianming Liu Pheng-Ann Heng Quanzheng Li MedIm 49 58 0 16 Sep 2023
DePT: Decomposed Prompt Tuning for Parameter-Efficient Fine-tuning Zhengxiang Shi Aldo Lipani VLM 26 30 0 11 Sep 2023
Fine-Tuning Llama 2 Large Language Models for Detecting Online Sexual Predatory Chats and Abusive Texts Thanh Thi Nguyen Campbell Wilson Janis Dalins 17 15 0 28 Aug 2023
Efficient Benchmarking of Language Models Yotam Perlitz Elron Bandel Ariel Gera Ofir Arviv L. Ein-Dor Eyal Shnarch Noam Slonim Michal Shmueli-Scheuer Leshem Choshen ALM 11 24 0 22 Aug 2023
Comparison between parameter-efficient techniques and full fine-tuning: A case study on multilingual news article classification Olesya Razuvayevskaya Ben Wu João A. Leite Freddy Heppell Ivan Srba Carolina Scarton Kalina Bontcheva Xingyi Song 14 8 0 14 Aug 2023
SuryaKiran at MEDIQA-Sum 2023: Leveraging LoRA for Clinical Dialogue Summarization Kunal Suri Prakhar Mishra Saumajit Saha Ashutosh Kumar Singh LM&MA 29 2 0 11 Jul 2023
Ladder Fine-tuning approach for SAM integrating complementary network Shurong Chai R. Jain Shiyu Teng Jiaqing Liu Yinhao Li T. Tateyama Yen-wei Chen MedIm 14 31 0 22 Jun 2023
BiomedGPT: A Unified and Generalist Biomedical Generative Pre-trained Transformer for Vision, Language, and Multimodal Tasks Kai Zhang Jun Yu Eashan Adhikarla Rong-Er Zhou Zhilin Yan ... Xun Chen Yong Chen Quanzheng Li Hongfang Liu Lichao Sun LM&MA MedIm 24 154 0 26 May 2023
READ: Recurrent Adaptation of Large Transformers Sida I. Wang John Nguyen Ke Li Carole-Jean Wu 20 11 0 24 May 2023
Adapting Language Models to Compress Contexts Alexis Chevalier Alexander Wettig Anirudh Ajith Danqi Chen LLMAG 11 173 0 24 May 2023
AMR Parsing with Instruction Fine-tuned Pre-trained Language Models Young-Suk Lee Ramón Fernández Astudillo Radu Florian Tahira Naseem Salim Roukos 20 4 0 24 Apr 2023
Large Language Model Instruction Following: A Survey of Progresses and Challenges Renze Lou Kai Zhang Wenpeng Yin ALM LRM 27 20 0 18 Mar 2023
A Kernel-Based View of Language Model Fine-Tuning Sadhika Malladi Alexander Wettig Dingli Yu Danqi Chen Sanjeev Arora VLM 68 60 0 11 Oct 2022
GLM-130B: An Open Bilingual Pre-trained Model Aohan Zeng Xiao Liu Zhengxiao Du Zihan Wang Hanyu Lai ... Jidong Zhai Wenguang Chen Peng-Zhen Zhang Yuxiao Dong Jie Tang BDL LRM 245 1,071 0 05 Oct 2022
Parameter-Efficient and Student-Friendly Knowledge Distillation Jun Rao Xv Meng Liang Ding Shuhan Qi Dacheng Tao 16 46 0 28 May 2022