PeriodicLoRA: Breaking the Low-Rank Bottleneck in LoRA Optimization

PeriodicLoRA: Breaking the Low-Rank Bottleneck in LoRA Optimization

25 February 2024

Zhe Yang

Peiyi Wang

Qingxiu Dong

Zhifang Sui

ArXiv (abs)PDF HTML

Papers citing "PeriodicLoRA: Breaking the Low-Rank Bottleneck in LoRA Optimization"

7 / 7 papers shown

Title
LoSiA: Efficient High-Rank Fine-Tuning via Subnet Localization and Optimization Xujia Wang Yunjia Qi Bin Xu 124 0 0 06 Jul 2025
DL-QAT: Weight-Decomposed Low-Rank Quantization-Aware Training for Large Language ModelsConference on Empirical Methods in Natural Language Processing (EMNLP), 2025 Wenjin Ke Zhe Li D. Li Lu Tian E. Barsoum MQ 188 6 0 12 Apr 2025
GoRA: Gradient-driven Adaptive Low Rank Adaptation Haonan He Peng Ye Yuchen Ren Yuan Yuan Luyang Zhou Shucun Ju Lei Chen AI4TS AI4CE 914 3 0 13 Feb 2025
AdaRankGrad: Adaptive Gradient-Rank and Moments for Memory-Efficient LLMs Training and Fine-TuningInternational Conference on Learning Representations (ICLR), 2024 Yehonathan Refael Jonathan Svirsky Boris Shustin Wasim Huleihel Ofir Lindenbaum 204 10 0 31 Dec 2024
Parameter-Efficient Fine-Tuning in Large Models: A Survey of Methodologies Liwen Wang Sheng Chen Linnan Jiang Shu Pan Runze Cai Sen Yang Fei Yang 409 14 0 24 Oct 2024
A Survey on LoRA of Large Language Models Yuren Mao Yuhang Ge Yijiang Fan Wenyi Xu Yu Mi Zhonghao Hu Yunjun Gao ALM 476 86 0 08 Jul 2024
Sparse Spectral Training and Inference on Euclidean and Hyperbolic Neural Networks Jialin Zhao Yingtao Zhang Xinghang Li Huaping Liu C. Cannistraci 310 2 0 24 May 2024