LoRA Training in the NTK Regime has No Spurious Local Minima

19 February 2024

Papers citing "LoRA Training in the NTK Regime has No Spurious Local Minima"

15 / 15 papers shown

Title
When is Task Vector Provably Effective for Model Editing? A Generalization Analysis of Nonlinear Transformers Hongkang Li Yihua Zhang Shuai Zhang M. Wang Sijia Liu Pin-Yu Chen MoMe 60 2 0 15 Apr 2025
Understanding the Learning Dynamics of LoRA: A Gradient Flow Perspective on Low-Rank Adaptation in Matrix Factorization Ziqing Xu Hancheng Min Lachlan Ewen MacDonald Jinqi Luo Salma Tarmoun Enrique Mallada René Vidal AI4CE 49 0 0 10 Mar 2025
Training Dynamics of In-Context Learning in Linear Attention Yedi Zhang Aaditya K. Singh Peter E. Latham Andrew Saxe MLT 62 1 0 28 Jan 2025
Gradient dynamics for low-rank fine-tuning beyond kernels Arif Kerem Dayi Sitan Chen 72 1 0 23 Nov 2024
On the Crucial Role of Initialization for Matrix Factorization Bingcong Li Liang Zhang Aryan Mokhtari Niao He 26 1 0 24 Oct 2024
ALLoRA: Adaptive Learning Rate Mitigates LoRA Fatal Flaws Hai Huang Randall Balestriero 30 0 0 13 Oct 2024
Parameter-Efficient Fine-Tuning of State Space Models Kevin Galim Wonjun Kang Yuchen Zeng H. Koo Kangwook Lee 29 4 0 11 Oct 2024
AutoLoRA: AutoGuidance Meets Low-Rank Adaptation for Diffusion Models Artur Kasymov Marcin Sendera Michał Stypułkowski Maciej Ziȩba P. Spurek 28 1 0 04 Oct 2024
Propulsion: Steering LLM with Tiny Fine-Tuning Md. Kowsher Nusrat Jahan Prottasha Prakash Bhat 38 4 0 17 Sep 2024
A Survey on LoRA of Large Language Models Yuren Mao Yuhang Ge Yijiang Fan Wenyi Xu Yu Mi Zhonghao Hu Yunjun Gao ALM 52 23 0 08 Jul 2024
Continual Diffusion: Continual Customization of Text-to-Image Diffusion with C-LoRA James Smith Yen-Chang Hsu Lingyu Zhang Ting Hua Z. Kira Yilin Shen Hongxia Jin DiffM 123 95 0 12 Apr 2023
A Kernel-Based View of Language Model Fine-Tuning Sadhika Malladi Alexander Wettig Dingli Yu Danqi Chen Sanjeev Arora VLM 66 60 0 11 Oct 2022
Making Pre-trained Language Models Better Few-shot Learners Tianyu Gao Adam Fisch Danqi Chen 241 1,913 0 31 Dec 2020
Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inference Timo Schick Hinrich Schütze 258 1,584 0 21 Jan 2020
Convex Sparse Matrix Factorizations Francis R. Bach Julien Mairal Jean Ponce 127 143 0 10 Dec 2008