LoRA-Pro: Are Low-Rank Adapters Properly Optimized?

LoRA-Pro: Are Low-Rank Adapters Properly Optimized?

25 July 2024

Jian Liang

Ran He

Zilei Wang

Tieniu Tan

Papers citing "LoRA-Pro: Are Low-Rank Adapters Properly Optimized?"

14 / 14 papers shown

Title
Improving the Serving Performance of Multi-LoRA Large Language Models via Efficient LoRA and KV Cache Management Hang Zhang Jiuchen Shi Yixiao Wang Quan Chen Yizhou Shan Minyi Guo 15 0 0 19 Apr 2025
Hiding Images in Diffusion Models by Editing Learned Score Functions Haoyu Chen Yunqiao Yang Nan Zhong Kede Ma DiffM 58 0 0 24 Mar 2025
Make LoRA Great Again: Boosting LoRA with Adaptive Singular Values and Mixture-of-Experts Optimization Alignment Chenghao Fan Zhenyi Lu Sichen Liu Xiaoye Qu Wei Wei Chengfeng Gu Yu-Xi Cheng MoE 40 0 0 24 Feb 2025
A Stronger Mixture of Low-Rank Experts for Fine-Tuning Foundation Models Mengyang Sun Yihao Wang Tao Feng Dan Zhang Yifan Zhu J. Tang MoE 24 0 0 20 Feb 2025
How to Alleviate Catastrophic Forgetting in LLMs Finetuning? Hierarchical Layer-Wise and Element-Wise Regularization Shezheng Song Hao Xu Jun Ma Shasha Li Long Peng Qian Wan Xiaodong Liu Jie Yu KELM CLL ALM 57 0 0 23 Jan 2025
A Hessian-informed hyperparameter optimization for differential learning rate Shiyun Xu Zhiqi Bu Yiliang Zhang Ian J. Barnett 29 0 0 12 Jan 2025
Initialization using Update Approximation is a Silver Bullet for Extremely Efficient Low-Rank Fine-Tuning Kaustubh Ponkshe Raghav Singhal Eduard A. Gorbunov Alexey Tumanov Samuel Horváth Praneeth Vepakomma 59 1 0 29 Nov 2024
Chameleon: Adaptive Caching and Scheduling for Many-Adapter LLM Inference Environments Nikoleta Iliakopoulou Jovan Stojkovic Chloe Alverti Tianyin Xu Hubertus Franke Josep Torrellas 62 2 0 24 Nov 2024
Flat-LoRA: Low-Rank Adaption over a Flat Loss Landscape Tao Li Zhengbao He Yujun Li Yasheng Wang Lifeng Shang X. Huang 35 0 0 22 Sep 2024
LoRA-GA: Low-Rank Adaptation with Gradient Approximation Shaowen Wang Linxi Yu Jian Li ALM AI4CE 24 4 0 06 Jul 2024
Riemannian Preconditioned LoRA for Fine-Tuning Foundation Models Fangzhao Zhang Mert Pilanci AI4CE 35 13 0 04 Feb 2024
Learning to Prompt for Vision-Language Models Kaiyang Zhou Jingkang Yang Chen Change Loy Ziwei Liu VPVLM CLIP VLM 319 2,108 0 02 Sep 2021
The Power of Scale for Parameter-Efficient Prompt Tuning Brian Lester Rami Al-Rfou Noah Constant VPVLM 275 3,784 0 18 Apr 2021
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 220 3,054 0 23 Jan 2020