Communities
Connect sessions
AI calendar
Organizations
Join Slack
Contact Sales
Search
Open menu
Home
Papers
2305.17333
Cited By
v1
v2
v3 (latest)
Fine-Tuning Language Models with Just Forward Passes
Neural Information Processing Systems (NeurIPS), 2023
27 May 2023
Sadhika Malladi
Tianyu Gao
Eshaan Nichani
Alexandru Damian
Jason D. Lee
Danqi Chen
Sanjeev Arora
Re-assign community
ArXiv (abs)
PDF
HTML
HuggingFace (3 upvotes)
Papers citing
"Fine-Tuning Language Models with Just Forward Passes"
50 / 188 papers shown
Stochastic Subspace Descent Accelerated via Bi-fidelity Line Search
Nuojin Cheng
Alireza Doostan
Stephen Becker
359
0
0
30 Apr 2025
POPri: Private Federated Learning using Preference-Optimized Synthetic Data
Charlie Hou
Mei-Yu Wang
Yige Zhu
Daniel Lazar
Giulia Fanti
FedML
529
7
0
23 Apr 2025
Efficient Model Editing with Task-Localized Sparse Fine-tuning
International Conference on Learning Representations (ICLR), 2025
Leonardo Iurada
Marco Ciccone
Tatiana Tommasi
KELM
MoMe
350
10
0
03 Apr 2025
A stochastic gradient descent algorithm with random search directions
Eméric Gbaguidi
ODL
246
0
0
25 Mar 2025
Efficient Personalization of Quantized Diffusion Model without Backpropagation
Computer Vision and Pattern Recognition (CVPR), 2025
H. Seo
Wongi Jeong
Kyungryeol Lee
Se Young Chun
DiffM
MQ
366
1
0
19 Mar 2025
FW-Merging: Scaling Model Merging with Frank-Wolfe Optimization
Hao Mark Chen
S. Hu
Wayne Luk
Timothy M. Hospedales
Hongxiang Fan
MoMe
457
3
0
16 Mar 2025
ZO2: Scalable Zeroth-Order Fine-Tuning for Extremely Large Language Models with Limited GPU Memory
Liangyu Wang
Jie Ren
Hang Xu
Junxiao Wang
Huanyi Xie
David E. Keyes
Di Wang
360
2
0
16 Mar 2025
A Closer Look at Adversarial Suffix Learning for Jailbreaking LLMs: Augmented Adversarial Trigger Learning
North American Chapter of the Association for Computational Linguistics (NAACL), 2025
Zhe Wang
Yanjun Qi
411
0
0
16 Mar 2025
A Survey on Federated Fine-tuning of Large Language Models
Yebo Wu
Chunlin Tian
Jingguang Li
He Sun
Kahou Tam
Zhanting Zhou
Haicheng Liao
Zhijiang Guo
Li Li
Chengzhong Xu
FedML
511
5
0
15 Mar 2025
Visualising Policy-Reward Interplay to Inform Zeroth-Order Preference Optimisation of Large Language Models
Annual Meeting of the Association for Computational Linguistics (ACL), 2025
Alessio Galatolo
Zhenbang Dai
Katie Winkle
Meriem Beloucif
267
0
0
05 Mar 2025
Towards hyperparameter-free optimization with differential privacy
International Conference on Learning Representations (ICLR), 2025
Zhiqi Bu
Ruixuan Liu
254
7
0
02 Mar 2025
SubZero: Composing Subject, Style, and Action via Zero-Shot Personalization
Shubhankar Borse
K. Bhardwaj
Mohammad Reza Karimi Dastjerdi
Hyojin Park
Shreya Kadambi
...
Prathamesh Mandke
Ankita Nayak
Harris Teague
Munawar Hayat
Fatih Porikli
DiffM
440
2
0
27 Feb 2025
LORENZA: Enhancing Generalization in Low-Rank Gradient LLM Training via Efficient Zeroth-Order Adaptive SAM
Yehonathan Refael
Iftach Arbel
Ofir Lindenbaum
Tom Tirer
426
2
0
26 Feb 2025
Scalable Back-Propagation-Free Training of Optical Physics-Informed Neural Networks
Yequan Zhao
Xinling Yu
Xian Xiao
Zhe Chen
Ziyue Liu
G. Kurczveil
R. Beausoleil
Shixuan Liu
Zheng Zhang
360
1
0
17 Feb 2025
A Survey of Personalized Large Language Models: Progress and Future Directions
Jiahong Liu
Zexuan Qiu
Zhongyang Li
Quanyu Dai
Jieming Zhu
Minda Hu
Menglin Yang
Irwin King
Tat-Seng Chua
Irwin King
LM&MA
337
30
0
17 Feb 2025
An Efficient Sparse Fine-Tuning with Low Quantization Error via Neural Network Pruning
Cen-Jhih Li
Aditya Bhaskara
413
0
0
17 Feb 2025
QuZO: Quantized Zeroth-Order Fine-Tuning for Large Language Models
Jiajun Zhou
Yifan Yang
Kai Zhen
Ziyue Liu
Yequan Zhao
Ershad Banijamali
Athanasios Mouchtaris
Ngai Wong
Zheng Zhang
MQ
308
4
0
17 Feb 2025
Efficient Zero-Order Federated Finetuning of Language Models for Resource-Constrained Devices
Mohamed Aboelenien Ahmed
Kilian Pfeiffer
R. Khalili
Heba Khdr
J. Henkel
FedML
527
0
0
14 Feb 2025
Model Diffusion for Certifiable Few-shot Transfer Learning
Fady Rezk
Royson Lee
Henry Gouk
Timothy M. Hospedales
Minyoung Kim
433
0
0
10 Feb 2025
Bilevel ZOFO: Efficient LLM Fine-Tuning and Meta-Training
Reza Shirkavand
Qi He
Qi He
Heng-Chiao Huang
ALM
575
1
0
05 Feb 2025
Memory-Efficient Fine-Tuning of Transformers via Token Selection
Conference on Empirical Methods in Natural Language Processing (EMNLP), 2025
Antoine Simoulin
Namyong Park
Xiaoyi Liu
Grey Yang
427
6
0
31 Jan 2025
Decentralized Low-Rank Fine-Tuning of Large Language Models
Sajjad Ghiasvand
Mahnoosh Alizadeh
Ramtin Pedarsani
ALM
638
9
0
26 Jan 2025
An Enhanced Zeroth-Order Stochastic Frank-Wolfe Framework for Constrained Finite-Sum Optimization
Haishan Ye
Yinghui Huang
Hao Di
Xiangyu Chang
435
1
0
13 Jan 2025
Stochastic Taylor Derivative Estimator: Efficient amortization for arbitrary differential operators
Neural Information Processing Systems (NeurIPS), 2024
Zekun Shi
Zheyuan Hu
Min Lin
Kenji Kawaguchi
1.0K
16
0
27 Nov 2024
Poor Man's Training on MCUs: A Memory-Efficient Quantized Back-Propagation-Free Approach
Yequan Zhao
Hai Li
Ian Young
Zheng Zhang
MQ
384
3
0
07 Nov 2024
Stepping Forward on the Last Mile
Neural Information Processing Systems (NeurIPS), 2024
Chen Feng
Shaojie Zhuo
Xiaopeng Zhang
R. Ramakrishnan
Zhaocong Yuan
Andrew Zou Li
426
0
0
06 Nov 2024
Normalization Layer Per-Example Gradients are Sufficient to Predict Gradient Noise Scale in Transformers
Neural Information Processing Systems (NeurIPS), 2024
Gavia Gray
Aman Tiwari
Shane Bergsma
Joel Hestness
357
2
0
01 Nov 2024
On the Crucial Role of Initialization for Matrix Factorization
International Conference on Learning Representations (ICLR), 2024
Bingcong Li
Liang Zhang
Aryan Mokhtari
Niao He
414
10
0
24 Oct 2024
CKSP: Cross-species Knowledge Sharing and Preserving for Universal Animal Activity Recognition
Biosystems Engineering (Biosyst. Eng.), 2024
Axiu Mao
Meilu Zhu
Zhaojin Guo
Zheng He
Tomas Norton
Kai Liu
144
1
0
22 Oct 2024
Understanding Forgetting in LLM Supervised Fine-Tuning and Preference Learning - A Convex Optimization Perspective
H. Fernando
Han Shen
Parikshit Ram
Yi Zhou
Horst Samulowitz
Nathalie Baracaldo
Tianyi Chen
CLL
460
10
0
20 Oct 2024
Implicit Regularization of Sharpness-Aware Minimization for Scale-Invariant Problems
Neural Information Processing Systems (NeurIPS), 2024
Bingcong Li
Liang Zhang
Niao He
284
9
0
18 Oct 2024
A Theoretical Survey on Foundation Models
Shi Fu
Yuzhu Chen
Yingjie Wang
Dacheng Tao
304
0
0
15 Oct 2024
Divide, Reweight, and Conquer: A Logit Arithmetic Approach for In-Context Learning
Chengsong Huang
Langlin Huang
Jiaxin Huang
MoMe
412
8
0
14 Oct 2024
Federated Data-Efficient Instruction Tuning for Large Language Models
Annual Meeting of the Association for Computational Linguistics (ACL), 2024
Zhen Qin
Zhaomin Wu
Bingsheng He
Shuiguang Deng
FedML
371
3
0
14 Oct 2024
Simultaneous Computation and Memory Efficient Zeroth-Order Optimizer for Fine-Tuning Large Language Models
Fei Wang
Li Shen
Liang Ding
Chao Xue
Ye Liu
Changxing Ding
264
4
0
13 Oct 2024
Zeroth-Order Fine-Tuning of LLMs in Random Subspaces
Ziming Yu
Pan Zhou
Sike Wang
Jia Li
Hua Huang
Hua Huang
361
0
0
11 Oct 2024
Addax: Utilizing Zeroth-Order Gradients to Improve Memory Efficiency and Performance of SGD for Fine-Tuning Language Models
International Conference on Learning Representations (ICLR), 2024
Zeman Li
Xinwei Zhang
Peilin Zhong
Yuan Deng
Meisam Razaviyayn
Vahab Mirrokni
286
11
0
09 Oct 2024
Chemistry-Inspired Diffusion with Non-Differentiable Guidance
International Conference on Learning Representations (ICLR), 2024
Yuchen Shen
Chenhao Zhang
Sijie Fu
Chenghui Zhou
Newell Washburn
Barnabás Póczos
352
4
0
09 Oct 2024
FLOPS: Forward Learning with OPtimal Sampling
International Conference on Learning Representations (ICLR), 2024
Tao Ren
Zishi Zhang
Jinyang Jiang
Guanghao Li
Zeliang Zhang
Mingqian Feng
Yijie Peng
442
2
0
08 Oct 2024
LoRTA: Low Rank Tensor Adaptation of Large Language Models
Ignacio Hounie
Charilaos I. Kanatsoulis
Arnuv Tandon
Alejandro Ribeiro
487
2
0
05 Oct 2024
Variance-Reduced Gradient Estimator for Nonconvex Zeroth-Order Distributed Optimization
American Control Conference (ACC), 2024
Huaiyi Mu
Yujie Tang
Zhongkui Li
104
2
0
29 Sep 2024
Zeroth-Order Policy Gradient for Reinforcement Learning from Human Feedback without Reward Inference
International Conference on Learning Representations (ICLR), 2024
Qining Zhang
Lei Ying
OffRL
475
9
0
25 Sep 2024
Communication and Energy Efficient Federated Learning using Zero-Order Optimization Technique
IEEE Transactions on Signal Processing (IEEE TSP), 2024
Elissa Mhanna
Mohamad Assaad
202
0
0
24 Sep 2024
MobiZO: Enabling Efficient LLM Fine-Tuning at the Edge via Inference Engines
Lei Gao
Amir Ziashahabi
Yue Niu
Salman Avestimehr
M. Annavaram
278
0
0
23 Sep 2024
A Unified Causal Framework for Auditing Recommender Systems for Ethical Concerns
Vibhhu Sharma
Shantanu Gupta
Nil-Jana Akpinar
Zachary C. Lipton
Liu Leqi
CML
MLAU
167
0
0
20 Sep 2024
Self-Contrastive Forward-Forward Algorithm
Nature Communications (Nat. Commun.), 2024
Xing Chen
Dongshu Liu
Jérémie Laydevant
Julie Grollier
587
6
0
17 Sep 2024
Ferret: Federated Full-Parameter Tuning at Scale for Large Language Models
Yao Shu
Wenyang Hu
Szu Hui Ng
Bryan Kian Hsiang Low
Fei Richard Yu
FedML
455
2
0
10 Sep 2024
Scalable Multitask Learning Using Gradient-based Estimation of Task Affinity
Knowledge Discovery and Data Mining (KDD), 2024
Dongyue Li
Aneesh Sharma
Hongyang R. Zhang
334
11
0
09 Sep 2024
Towards training digitally-tied analog blocks via hybrid gradient computation
Neural Information Processing Systems (NeurIPS), 2024
Timothy Nest
M. Ernoult
287
6
0
05 Sep 2024
Towards General Industrial Intelligence: A Survey on IIoT-Enhanced Continual Large Models
Jiao Chen
Jiayi He
Fangfang Chen
Zuohong Lv
Jianhua Tang
Weihua Li
Zuozhu Liu
Howard H. Yang
Guangjie Han
AI4CE
284
1
0
02 Sep 2024
Previous
1
2
3
4
Next
