SORSA: Singular Values and Orthonormal Regularized Singular Vectors Adaptation of Large Language Models

21 August 2024

Yang Cao

Papers citing "SORSA: Singular Values and Orthonormal Regularized Singular Vectors Adaptation of Large Language Models"

3 / 3 papers shown

Title
Grams: Gradient Descent with Adaptive Momentum Scaling Yang Cao Xiaoyu Li Zhao-quan Song ODL 83 2 0 22 Dec 2024
Bypassing the Exponential Dependency: Looped Transformers Efficiently Learn In-context by Multi-step Gradient Descent Bo Chen Xiaoyu Li Yingyu Liang Zhenmei Shi Zhao-quan Song 59 18 0 15 Oct 2024
The Power of Scale for Parameter-Efficient Prompt Tuning Brian Lester Rami Al-Rfou Noah Constant VPVLM 275 3,784 0 18 Apr 2021