GaLore $+$ : Boosting Low-Rank Adaptation for LLMs with Cross-Head Projection

31 December 2024

Papers citing "GaLore$+$: Boosting Low-Rank Adaptation for LLMs with Cross-Head Projection"

2 / 2 papers shown

Title
LORENZA: Enhancing Generalization in Low-Rank Gradient LLM Training via Efficient Zeroth-Order Adaptive SAM Yehonathan Refael Iftach Arbel Ofir Lindenbaum Tom Tirer 59 0 0 26 Feb 2025
CoLA: Compute-Efficient Pre-Training of LLMs via Low-Rank Activation Z. Liu Ruijie Zhang Z. Wang Zi Yang Paul Hovland Bogdan Nicolae Franck Cappello Z. Zhang 37 0 0 16 Feb 2025