v1v2v3 (latest)

Fine-Tuning Language Models with Just Forward Passes

Neural Information Processing Systems (NeurIPS), 2023

27 May 2023

ArXiv (abs)PDF HTML HuggingFace (3 upvotes)

Papers citing "Fine-Tuning Language Models with Just Forward Passes"

50 / 188 papers shown

ZO-ASR: Zeroth-Order Fine-Tuning of Speech Foundation Models without Back-Propagation

111

01 Dec 2025

Dialect Identification Using Resource-Efficient Fine-Tuning ApproachesAsia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA), 2025

Zirui Lin

Haris Gulzar

Monnika Roslianna Busto

Akiko Masaki

Takeharu Eda

K. Nakadai

30 Nov 2025

Ghosting Your LLM: Without The Knowledge of Your Gradient and Data

Abeer Matar A. Almalky

207

27 Nov 2025

Low-Rank Curvature for Zeroth-Order Optimization in LLM Fine-Tuning

Hyunseok Seung

Jaewoo Lee

Hyunsuk Ko

11 Nov 2025

Towards Straggler-Resilient Split Federated Learning: An Unbalanced Update Approach

186

24 Oct 2025

More Than Memory Savings: Zeroth-Order Optimization Mitigates Forgetting in Continual Learning

243

23 Oct 2025

Language Ranker: A Lightweight Ranking framework for LLM Decoding

203

23 Oct 2025

Revisiting Zeroth-Order Optimization: Minimum-Variance Two-Point Estimators and Directionally Aligned PerturbationsInternational Conference on Learning Representations (ICLR), 2025

Shaocong Ma

Heng Huang

153

22 Oct 2025

On the Optimal Construction of Unbiased Gradient Estimators for Zeroth-Order Optimization

Shaocong Ma

Heng Huang

138

22 Oct 2025

Towards Fast LLM Fine-tuning through Zeroth-Order Optimization with Projected Gradient-Aligned Perturbations

145

21 Oct 2025

Zeroth-Order Sharpness-Aware Learning with Exponential Tilting

Xuchen Gong

Tian Li

148

17 Oct 2025

Noise-Adaptive Layerwise Learning Rates: Accelerating Geometry-Aware Optimization for Deep Neural Network Training

152

15 Oct 2025

Memory-Efficient Backpropagation for Fine-Tuning LLMs on Resource-Constrained Mobile Devices

Congzheng Song

Xinyu Tang

123

03 Oct 2025

Learning a Zeroth-Order Optimizer for Fine-Tuning LLMs

179

01 Oct 2025

Downgrade to Upgrade: Optimizer Simplification Enhances Robustness in LLM Unlearning

356

01 Oct 2025

Evolution Strategies at Scale: LLM Fine-Tuning Beyond Reinforcement Learning

196

29 Sep 2025

CR-Net: Scaling Parameter-Efficient Training with Cross-Layer Low-Rank Structure

160

23 Sep 2025

The Multi-Query Paradox in Zeroth-Order Optimization

Wei Lin

Qingyu Song

Hong Xu

170

19 Sep 2025

Low-rank surrogate modeling and stochastic zero-order optimization for training of neural networks with black-box layers

142

18 Sep 2025

Low-rank Orthogonalization for Large-scale Matrix Optimization with Applications to Foundation Model Training

168

15 Sep 2025

L1RA: Dynamic Rank Assignment in LoRA Fine-Tuning

110

05 Sep 2025

Warming Up for Zeroth-Order Federated Pre-Training with Low Resource Clients

112

03 Sep 2025

Forward-Only Continual Learning

166

01 Sep 2025

GradES: Significantly Faster Training in Transformers with Gradient-Based Early Stopping

264

01 Sep 2025

On the Evolution of Federated Post-Training Large Language Models: A Model Accessibility View

109

22 Aug 2025

End-to-End On-Device Quantization-Aware Training for LLMs at Inference Cost

...

226

21 Aug 2025

Efficient Knowledge Graph Unlearning with Zeroth-order Information

160

19 Aug 2025

Unpacking the Implicit Norm Dynamics of Sharpness-Aware Minimization in Tensorized Models

Tianxiao Cao

Kyohei Atarashi

H. Kashima

227

14 Aug 2025

Communication-Efficient Zero-Order and First-Order Federated Learning Methods over Wireless Networks

Mohamad Assaad

Zeinab Nehme

Mérouane Debbah

11 Aug 2025

RCR-Router: Efficient Role-Aware Context Routing for Multi-Agent LLM Systems with Structured Memory

...

208

06 Aug 2025

Test-Time Model Adaptation for Quantized Neural Networks

155

04 Aug 2025

DREAM: Scalable Red Teaming for Text-to-Image Generative Systems via Distribution Modeling

229

22 Jul 2025

Memory-Efficient Personalization of Text-to-Image Diffusion Models via Selective Optimization Strategies

151

14 Jul 2025

Greedy Low-Rank Gradient Compression for Distributed Learning with Convergence Guarantees

625

11 Jul 2025

SharpZO: Hybrid Sharpness-Aware Vision Language Model Prompt Tuning via Forward-Only Passes

Yifan Yang

Zhen-ying Zhang

Rupak Vignesh Swaminathan

191

26 Jun 2025

Private Training & Data Generation by Clustering Embeddings

191

20 Jun 2025

Memory-Efficient Differentially Private Training with Gradient Random Projection

243

18 Jun 2025

Private Aggregation for Byzantine-Resilient Heterogeneous Federated Learning

Maximilian Egger

Rawad Bitar

279

11 Jun 2025

MobiEdit: Resource-efficient Knowledge Editing for Personalized On-device LLMs

211

05 Jun 2025

Learning long range dependencies through time reversal symmetry breaking

Guillaume Pourcel

Maxence Ernoult

351

05 Jun 2025

Leveraging Coordinate Momentum in SignSGD and Muon: Memory-Optimized Zero-Order

410

04 Jun 2025

Provable Reinforcement Learning from Human Feedback with an Unknown Link Function

Qining Zhang

Lei Ying

252

03 Jun 2025

Reconciling Hessian-Informed Acceleration and Scalar-Only Communication for Efficient Federated Zeroth-Order Fine-Tuning

242

03 Jun 2025

MLorc: Momentum Low-rank Compression for Memory Efficient Large Language Model Adaptation

326

02 Jun 2025

Structured Gradient Guidance for Few-Shot Adaptation in Large Language Models

141

31 May 2025

A Structured Tour of Optimization with Finite Differences

364

26 May 2025

KerZOO: Kernel Function Informed Zeroth-Order Optimization for Accurate and Accelerated LLM Fine-Tuning

356

24 May 2025

Subquadratic Algorithms and Hardness for Attention with Any Temperature

265

20 May 2025

Fine-tuning Quantized Neural Networks with Zeroth-order Optimization

353

19 May 2025

Memory-Efficient LLM Training by Various-Grained Low-Rank Projection of Gradients

236

03 May 2025