ResearchTrend.AI
  • Communities
  • Connect sessions
  • AI calendar
  • Organizations
  • Join Slack
  • Contact Sales
Papers
Communities
Social Events
Terms and Conditions
Pricing
Contact Sales
Parameter LabParameter LabTwitterGitHubLinkedInBlueskyYoutube

© 2026 ResearchTrend.AI, All rights reserved.

  1. Home
  2. Papers
  3. 2403.03507
  4. Cited By
GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projection
v1v2 (latest)

GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projection

6 March 2024
Jiawei Zhao
Zhenyu Zhang
Beidi Chen
Zinan Lin
A. Anandkumar
Yuandong Tian
ArXiv (abs)PDFHTMLHuggingFace (189 upvotes)

Papers citing "GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projection"

19 / 219 papers shown
LoQT: Low Rank Adapters for Quantized Training
LoQT: Low Rank Adapters for Quantized Training
Sebastian Loeschcke
M. Toftrup
M. Kastoryano
Serge Belongie
Vésteinn Snæbjarnarson
MQ
233
0
0
26 May 2024
MicroAdam: Accurate Adaptive Optimization with Low Space Overhead and
  Provable Convergence
MicroAdam: Accurate Adaptive Optimization with Low Space Overhead and Provable Convergence
Ionut-Vlad Modoranu
M. Safaryan
Grigory Malinovsky
Eldar Kurtic
Thomas Robert
Peter Richtárik
Dan Alistarh
MQ
194
21
0
24 May 2024
Sparse Matrix in Large Language Model Fine-tuning
Sparse Matrix in Large Language Model Fine-tuning
Haoze He
Juncheng Billy Li
Xuan Jiang
Heather Miller
MoE
312
7
0
24 May 2024
Sparse Spectral Training and Inference on Euclidean and Hyperbolic Neural Networks
Sparse Spectral Training and Inference on Euclidean and Hyperbolic Neural Networks
Jialin Zhao
Yingtao Zhang
Xinghang Li
Huaping Liu
C. Cannistraci
374
2
0
24 May 2024
CoMERA: Computing- and Memory-Efficient Training via Rank-Adaptive
  Tensor Optimization
CoMERA: Computing- and Memory-Efficient Training via Rank-Adaptive Tensor OptimizationNeural Information Processing Systems (NeurIPS), 2024
Zi Yang
Samridhi Choudhary
Xinfeng Xie
Cao Gao
Siegfried Kunzmann
Zheng Zhang
VLM
356
16
0
23 May 2024
LoRA Learns Less and Forgets Less
LoRA Learns Less and Forgets Less
D. Biderman
Jose Javier Gonzalez Ortiz
Jacob P. Portes
Mansheej Paul
Philip Greengard
...
Sam Havens
Vitaliy Chiley
Jonathan Frankle
Cody Blakeney
John P. Cunningham
CLL
344
230
0
15 May 2024
Assisted Debate Builder with Large Language Models
Assisted Debate Builder with Large Language ModelsEuropean Conference on Artificial Intelligence (ECAI), 2024
Elliot Faugier
Frédéric Armetta
Angela Bonifati
Bruno Yun
71
1
0
14 May 2024
Q-Newton: Hybrid Quantum-Classical Scheduling for Accelerating Neural Network Training with Newton's Gradient Descent
Q-Newton: Hybrid Quantum-Classical Scheduling for Accelerating Neural Network Training with Newton's Gradient Descent
Pingzhi Li
Junyu Liu
Hanrui Wang
Tianlong Chen
584
2
0
30 Apr 2024
Med-MoE: Mixture of Domain-Specific Experts for Lightweight Medical
  Vision-Language Models
Med-MoE: Mixture of Domain-Specific Experts for Lightweight Medical Vision-Language Models
Songtao Jiang
Tuo Zheng
Yan Zhang
Yeying Jin
Li Yuan
Zuozhu Liu
MoE
401
46
0
16 Apr 2024
Proof-of-Learning with Incentive Security
Proof-of-Learning with Incentive Security
Zishuo Zhao
Zhixuan Fang
Xuechao Wang
Xi Chen
Yuan Zhou
AAML
590
8
0
13 Apr 2024
SambaLingo: Teaching Large Language Models New Languages
SambaLingo: Teaching Large Language Models New Languages
Zoltan Csaki
Bo Li
Jonathan Li
Qiantong Xu
Pian Pawakapan
Leon Zhang
Yun Du
Hengyu Zhao
Changran Hu
Urmish Thakker
242
13
0
08 Apr 2024
Lossless and Near-Lossless Compression for Foundation Models
Lossless and Near-Lossless Compression for Foundation Models
Moshik Hershcovitch
Leshem Choshen
Andrew Wood
Ilias Enmouri
Peter Chin
S. Sundararaman
Danny Harnik
273
12
0
05 Apr 2024
BAdam: A Memory Efficient Full Parameter Optimization Method for Large
  Language Models
BAdam: A Memory Efficient Full Parameter Optimization Method for Large Language ModelsNeural Information Processing Systems (NeurIPS), 2024
Qi Luo
Hengxu Yu
Xiao Li
264
16
0
03 Apr 2024
LISA: Layerwise Importance Sampling for Memory-Efficient Large Language
  Model Fine-Tuning
LISA: Layerwise Importance Sampling for Memory-Efficient Large Language Model Fine-Tuning
Boyao Wang
Xiang Liu
Shizhe Diao
Renjie Pi
Jipeng Zhang
Chi Han
Tong Zhang
383
93
0
26 Mar 2024
Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Survey
Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Survey
Zeyu Han
Chao Gao
Jinyang Liu
Jeff Zhang
Sai Qian Zhang
800
707
0
21 Mar 2024
LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Models
LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Models
Yaowei Zheng
Richong Zhang
Junhao Zhang
Yanhan Ye
Zheyan Luo
Zhangchi Feng
Yongqiang Ma
662
1,179
0
20 Mar 2024
Second-Order Fine-Tuning without Pain for LLMs:A Hessian Informed Zeroth-Order Optimizer
Second-Order Fine-Tuning without Pain for LLMs:A Hessian Informed Zeroth-Order Optimizer
Yanjun Zhao
Sizhe Dang
Haishan Ye
Guang Dai
Yi Qian
Ivor W.Tsang
672
29
0
23 Feb 2024
Flora: Low-Rank Adapters Are Secretly Gradient Compressors
Flora: Low-Rank Adapters Are Secretly Gradient CompressorsInternational Conference on Machine Learning (ICML), 2024
Yongchang Hao
Yanshuai Cao
Lili Mou
291
86
0
05 Feb 2024
QFT: Quantized Full-parameter Tuning of LLMs with Affordable Resources
QFT: Quantized Full-parameter Tuning of LLMs with Affordable Resources
Zhikai Li
Xiaoxuan Liu
Banghua Zhu
Zhen Dong
Qingyi Gu
Kurt Keutzer
MQ
275
11
0
11 Oct 2023
Previous
12345