Random Masking Finds Winning Tickets for Parameter Efficient Fine-tuning

Random Masking Finds Winning Tickets for Parameter Efficient Fine-tuning

4 May 2024

Jing Xu

Papers citing "Random Masking Finds Winning Tickets for Parameter Efficient Fine-tuning"

6 / 6 papers shown

Title
Understanding Layer Significance in LLM Alignment Guangyuan Shi Zexin Lu Xiaoyu Dong Wenlong Zhang Xuanyu Zhang Yujie Feng Xiao-Ming Wu 45 2 0 23 Oct 2024
MoFO: Momentum-Filtered Optimizer for Mitigating Forgetting in LLM Fine-Tuning Yupeng Chen Senmiao Wang Zhihang Lin Zhihang Lin Yushun Zhang Tian Ding Ruoyu Sun Ruoyu Sun CLL 72 1 0 30 Jul 2024
Just One Byte (per gradient): A Note on Low-Bandwidth Decentralized Language Model Finetuning Using Shared Randomness E. Zelikman Qian Huang Percy Liang Nick Haber Noah D. Goodman 62 14 0 16 Jun 2023
A Kernel-Based View of Language Model Fine-Tuning Sadhika Malladi Alexander Wettig Dingli Yu Danqi Chen Sanjeev Arora VLM 66 60 0 11 Oct 2022
Raise a Child in Large Language Model: Towards Effective and Generalizable Fine-tuning Runxin Xu Fuli Luo Zhiyuan Zhang Chuanqi Tan Baobao Chang Songfang Huang Fei Huang LRM 136 177 0 13 Sep 2021
The Power of Scale for Parameter-Efficient Prompt Tuning Brian Lester Rami Al-Rfou Noah Constant VPVLM 278 3,784 0 18 Apr 2021