Communities
Connect sessions
AI calendar
Organizations
Join Slack
Contact Sales

Terms and Conditions

Twitter GitHub LinkedIn Bluesky Youtube

© 2026 ResearchTrend.AI, All rights reserved.

Home
Papers
2403.03507
Cited By

GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projection

v1v2 (latest)

GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projection

6 March 2024

Yuandong Tian

ArXiv (abs)PDF HTML HuggingFace (189 upvotes)

Papers citing "GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projection"

50 / 219 papers shown

PEFT-Factory: Unified Parameter-Efficient Fine-Tuning of Autoregressive Large Language Models

PEFT-Factory: Unified Parameter-Efficient Fine-Tuning of Autoregressive Large Language Models

Róbert Belanec

Maria Bielikova

440

0

0

02 Dec 2025

ZO-ASR: Zeroth-Order Fine-Tuning of Speech Foundation Models without Back-Propagation

111

0

0

01 Dec 2025

The Path Not Taken: RLVR Provably Learns Off the Principals

The Path Not Taken: RLVR Provably Learns Off the Principals

...

198

4

0

11 Nov 2025

EcoSpa: Efficient Transformer Training with Coupled Sparsity

EcoSpa: Efficient Transformer Training with Coupled Sparsity

...

81

1

0

09 Nov 2025

Subsampled Randomized Fourier GaLore for Adapting Foundation Models in Depth-Driven Liver Landmark Segmentation

Subsampled Randomized Fourier GaLore for Adapting Foundation Models in Depth-Driven Liver Landmark Segmentation

Sergi Kavtaradze

Matthew J. Clarkson

Mobarak I. Hoque

102

0

0

05 Nov 2025

An All-Reduce Compatible Top-K Compressor for Communication-Efficient Distributed Learning

An All-Reduce Compatible Top-K Compressor for Communication-Efficient Distributed Learning

281

0

0

30 Oct 2025

IBNorm: Information-Bottleneck Inspired Normalization for Representation Learning

IBNorm: Information-Bottleneck Inspired Normalization for Representation Learning

103

0

0

29 Oct 2025

MISA: Memory-Efficient LLMs Optimization with Module-wise Importance Sampling

MISA: Memory-Efficient LLMs Optimization with Module-wise Importance Sampling

148

0

0

28 Oct 2025

From Memorization to Reasoning in the Spectrum of Loss Curvature

From Memorization to Reasoning in the Spectrum of Loss Curvature

Srihita Vatsavaya

211

1

0

28 Oct 2025

Kernelized Sparse Fine-Tuning with Bi-level Parameter Competition for Vision Models

Kernelized Sparse Fine-Tuning with Bi-level Parameter Competition for Vision Models

136

0

0

28 Oct 2025

How do simple rotations affect the implicit bias of Adam?

How do simple rotations affect the implicit bias of Adam?

Vasileios Charisopoulos

Rebecca Willett

364

0

0

27 Oct 2025

Improving the Straight-Through Estimator with Zeroth-Order Information

Improving the Straight-Through Estimator with Zeroth-Order Information

288

0

0

27 Oct 2025

Towards Fast LLM Fine-tuning through Zeroth-Order Optimization with Projected Gradient-Aligned Perturbations

Towards Fast LLM Fine-tuning through Zeroth-Order Optimization with Projected Gradient-Aligned Perturbations

145

0

0

21 Oct 2025

Unbiased Gradient Low-Rank Projection

Unbiased Gradient Low-Rank Projection

149

0

0

20 Oct 2025

Improving Model Representation and Reducing KV Cache via Skip Connections with First Value Heads

Improving Model Representation and Reducing KV Cache via Skip Connections with First Value Heads

157

0

0

19 Oct 2025

A Guardrail for Safety Preservation: When Safety-Sensitive Subspace Meets Harmful-Resistant Null-Space

A Guardrail for Safety Preservation: When Safety-Sensitive Subspace Meets Harmful-Resistant Null-Space

289

1

0

16 Oct 2025

Noise-Adaptive Layerwise Learning Rates: Accelerating Geometry-Aware Optimization for Deep Neural Network Training

Noise-Adaptive Layerwise Learning Rates: Accelerating Geometry-Aware Optimization for Deep Neural Network Training

152

0

0

15 Oct 2025

AdaPM: a Partial Momentum Algorithm for LLM Training

AdaPM: a Partial Momentum Algorithm for LLM Training

146

0

0

10 Oct 2025

ConceptSplit: Decoupled Multi-Concept Personalization of Diffusion Models via Token-wise Adaptation and Attention Disentanglement

ConceptSplit: Decoupled Multi-Concept Personalization of Diffusion Models via Token-wise Adaptation and Attention Disentanglement

Gyeong-Moon Park

165

0

0

06 Oct 2025

QDeepGR4J: Quantile-based ensemble of deep learning and GR4J hybrid rainfall-runoff models for extreme flow prediction with uncertainty quantification

QDeepGR4J: Quantile-based ensemble of deep learning and GR4J hybrid rainfall-runoff models for extreme flow prediction with uncertainty quantification

Rohitash Chandra

123

3

0

06 Oct 2025

REG: A Regularization Optimizer for Robust Training Dynamics

REG: A Regularization Optimizer for Robust Training Dynamics

111

0

0

04 Oct 2025

Optimizing Fine-Tuning through Advanced Initialization Strategies for Low-Rank Adaptation

Optimizing Fine-Tuning through Advanced Initialization Strategies for Low-Rank Adaptation

143

0

0

04 Oct 2025

Memory-Efficient Backpropagation for Fine-Tuning LLMs on Resource-Constrained Mobile Devices

Memory-Efficient Backpropagation for Fine-Tuning LLMs on Resource-Constrained Mobile Devices

123

0

0

03 Oct 2025

Randomized Gradient Subspaces for Efficient Large Language Model Training

Randomized Gradient Subspaces for Efficient Large Language Model Training

Samanvay Vajpayee

Sirisha Rambhatla

111

0

0

02 Oct 2025

Sample-Efficient Differentially Private Fine-Tuning via Gradient Matrix Denoising

Sample-Efficient Differentially Private Fine-Tuning via Gradient Matrix Denoising

95

0

0

01 Oct 2025

Finetune Once: Decoupling General & Domain Learning with Dynamic Boosted Annealing

Finetune Once: Decoupling General & Domain Learning with Dynamic Boosted Annealing

114

0

0

30 Sep 2025

PrunedLoRA: Robust Gradient-Based structured pruning for Low-rank Adaptation in Fine-tuning

PrunedLoRA: Robust Gradient-Based structured pruning for Low-rank Adaptation in Fine-tuning

246

0

0

30 Sep 2025

Conda: Column-Normalized Adam for Training Large Language Models Faster

Conda: Column-Normalized Adam for Training Large Language Models Faster

246

0

0

29 Sep 2025

Effective Quantization of Muon Optimizer States

Effective Quantization of Muon Optimizer States

Abhishek Shivanna

D. T. Braithwaite

142

0

0

27 Sep 2025

Memory-Efficient Fine-Tuning via Low-Rank Activation Compression

Memory-Efficient Fine-Tuning via Low-Rank Activation Compression

131

0

0

27 Sep 2025

Partial Parameter Updates for Efficient Distributed Training

Partial Parameter Updates for Efficient Distributed Training

Anastasiia Filippova

Angelos Katharopoulos

Ronan Collobert

136

0

0

26 Sep 2025

Provable Scaling Laws of Feature Emergence from Learning Dynamics of Grokking

Provable Scaling Laws of Feature Emergence from Learning Dynamics of Grokking

204

0

0

25 Sep 2025

No Prior, No Leakage: Revisiting Reconstruction Attacks in Trained Neural Networks

No Prior, No Leakage: Revisiting Reconstruction Attacks in Trained Neural Networks

Yehonatan Refael

Guy Smorodinsky

Ofir Lindenbaum

305

0

0

25 Sep 2025

Faster Than SVD, Smarter Than SGD: The OPLoRA Alternating Update

Faster Than SVD, Smarter Than SGD: The OPLoRA Alternating Update

Abdulla Jasem Almansoori

Andrey Veprikov

Aleksandr Beznosikov

109

0

0

24 Sep 2025

CR-Net: Scaling Parameter-Efficient Training with Cross-Layer Low-Rank Structure

CR-Net: Scaling Parameter-Efficient Training with Cross-Layer Low-Rank Structure

160

1

0

23 Sep 2025

Development of Deep Learning Optimizers: Approaches, Concepts, and Update Rules

Development of Deep Learning Optimizers: Approaches, Concepts, and Update Rules

Doğay Altınel

135

0

0

22 Sep 2025

BEFT: Bias-Efficient Fine-Tuning of Language Models

BEFT: Bias-Efficient Fine-Tuning of Language Models

Ananth Balashankar

127

0

0

19 Sep 2025

Distribution-Aligned Decoding for Efficient LLM Task Adaptation

Distribution-Aligned Decoding for Efficient LLM Task Adaptation

239

2

0

19 Sep 2025

Low-rank surrogate modeling and stochastic zero-order optimization for training of neural networks with black-box layers

Low-rank surrogate modeling and stochastic zero-order optimization for training of neural networks with black-box layers

Andrei Chertkov

Stanislav Straupe

142

0

0

18 Sep 2025

Low-rank Orthogonalization for Large-scale Matrix Optimization with Applications to Foundation Model Training

Low-rank Orthogonalization for Large-scale Matrix Optimization with Applications to Foundation Model Training

168

2

0

15 Sep 2025

From PowerSGD to PowerSGD+: Low-Rank Gradient Compression for Distributed Optimization with Convergence Guarantees

From PowerSGD to PowerSGD+: Low-Rank Gradient Compression for Distributed Optimization with Convergence Guarantees

117

0

0

14 Sep 2025

An Efficient Subspace Algorithm for Federated Learning on Heterogeneous Data

An Efficient Subspace Algorithm for Federated Learning on Heterogeneous Data

120

0

0

05 Sep 2025

ReSURE: Regularizing Supervision Unreliability for Multi-turn Dialogue Fine-tuning

ReSURE: Regularizing Supervision Unreliability for Multi-turn Dialogue Fine-tuning

178

1

0

27 Aug 2025

DropLoRA: Sparse Low-Rank Adaptation for Parameter-Efficient Fine-Tuning

DropLoRA: Sparse Low-Rank Adaptation for Parameter-Efficient Fine-Tuning

92

2

0

24 Aug 2025

Empowering Multimodal LLMs with External Tools: A Comprehensive Survey

Empowering Multimodal LLMs with External Tools: A Comprehensive Survey

182

1

0

14 Aug 2025

LLMC+: Benchmarking Vision-Language Model Compression with a Plug-and-play Toolkit

LLMC+: Benchmarking Vision-Language Model Compression with a Plug-and-play Toolkit

162

0

0

13 Aug 2025

LOST: Low-rank and Sparse Pre-training for Large Language Models

LOST: Low-rank and Sparse Pre-training for Large Language Models

152

2

0

04 Aug 2025

Efficiently Seeking Flat Minima for Better Generalization in Fine-Tuning Large Language Models and Beyond

Efficiently Seeking Flat Minima for Better Generalization in Fine-Tuning Large Language Models and Beyond

150

0

0

01 Aug 2025

From LLMs to Edge: Parameter-Efficient Fine-Tuning on Edge Devices

From LLMs to Edge: Parameter-Efficient Fine-Tuning on Edge Devices

Francesco Corti

113

0

0

31 Jul 2025

TorchAO: PyTorch-Native Training-to-Serving Model Optimization

TorchAO: PyTorch-Native Training-to-Serving Model Optimization

Daniel Vega-Myhre

Charles David Hernandez

...

Christian Puhrsch

Aleksandar Samardžić

171

4

0

21 Jul 2025