A Zeroth-Order Block Coordinate Descent Algorithm for Huge-Scale Black-Box Optimization

21 February 2021

Papers citing "A Zeroth-Order Block Coordinate Descent Algorithm for Huge-Scale Black-Box Optimization"

29 / 29 papers shown

Title
ZO2: Scalable Zeroth-Order Fine-Tuning for Extremely Large Language Models with Limited GPU Memory Liangyu Wang Jie Ren Hang Xu Junxiao Wang Huanyi Xie David E. Keyes Di Wang 60 0 0 16 Mar 2025
LORENZA: Enhancing Generalization in Low-Rank Gradient LLM Training via Efficient Zeroth-Order Adaptive SAM Yehonathan Refael Iftach Arbel Ofir Lindenbaum Tom Tirer 71 0 0 26 Feb 2025
Efficient Zero-Order Federated Finetuning of Language Models for Resource-Constrained Devices Mohamed Aboelenien Ahmed Kilian Pfeiffer R. Khalili Heba Khdr J. Henkel FedML 91 0 0 17 Feb 2025
Scalable Back-Propagation-Free Training of Optical Physics-Informed Neural Networks Yequan Zhao Xinling Yu Xian Xiao Zhengzhang Chen Z. Liu G. Kurczveil R. Beausoleil S. Liu Z. Zhang 56 0 0 17 Feb 2025
Poor Man's Training on MCUs: A Memory-Efficient Quantized Back-Propagation-Free Approach Yequan Zhao Hai Li Ian Young Zheng-Wei Zhang MQ 39 2 0 07 Nov 2024
Zeroth-Order Policy Gradient for Reinforcement Learning from Human Feedback without Reward Inference Qining Zhang Lei Ying OffRL 37 2 0 25 Sep 2024
How to Boost Any Loss Function Richard Nock Yishay Mansour 34 0 0 02 Jul 2024
Gradient Compressed Sensing: A Query-Efficient Gradient Estimator for High-Dimensional Zeroth-Order Optimization Ruizhong Qiu Hanghang Tong 40 3 0 27 May 2024
Achieving Dimension-Free Communication in Federated Learning via Zeroth-Order Optimization Zhe Li Bicheng Ying Zidong Liu Haibo Yang Haibo Yang FedML 59 3 0 24 May 2024
A New Formulation for Zeroth-Order Optimization of Adversarial EXEmples in Malware Detection Marco Rando Luca Demetrio Lorenzo Rosasco Fabio Roli AAML 32 1 0 23 May 2024
Binary Hypothesis Testing for Softmax Models and Leverage Score Models Yeqi Gao Yuzhou Gu Zhao-quan Song 33 0 0 09 May 2024
Sparse MeZO: Less Parameters for Better Performance in Zeroth-Order LLM Fine-Tuning Yong Liu Zirui Zhu Chaoyu Gong Minhao Cheng Cho-Jui Hsieh Yang You MoE 37 16 0 24 Feb 2024
Second-Order Fine-Tuning without Pain for LLMs:A Hessian Informed Zeroth-Order Optimizer Yanjun Zhao Sizhe Dang Haishan Ye Guang Dai Yi Qian Ivor W.Tsang 66 8 0 23 Feb 2024
Revisiting Zeroth-Order Optimization for Memory-Efficient LLM Fine-Tuning: A Benchmark Yihua Zhang Pingzhi Li Junyuan Hong Jiaxiang Li Yimeng Zhang ... Wotao Yin Mingyi Hong Zhangyang Wang Sijia Liu Tianlong Chen 25 45 0 18 Feb 2024
An Optimal Transport Approach for Computing Adversarial Training Lower Bounds in Multiclass Classification Nicolas García Trillos Matt Jacobs Jakwang Kim Matthew Werenski AAML 45 2 0 17 Jan 2024
The Expressibility of Polynomial based Attention Scheme Zhao-quan Song Guangyi Xu Junze Yin 32 5 0 30 Oct 2023
DeepZero: Scaling up Zeroth-Order Optimization for Deep Model Training Aochuan Chen Yimeng Zhang Jinghan Jia James Diffenderfer Jiancheng Liu Konstantinos Parasyris Yihua Zhang Zheng-Wei Zhang B. Kailkhura Sijia Liu 30 43 0 03 Oct 2023
A Fast Optimization View: Reformulating Single Layer Attention in LLM Based on Tensor and SVM Trick, and Solving It in Matrix Multiplication Time Yeqi Gao Zhao-quan Song Weixin Wang Junze Yin 20 25 0 14 Sep 2023
Tensor-Compressed Back-Propagation-Free Training for (Physics-Informed) Neural Networks Yequan Zhao Xinling Yu Zhixiong Chen Z. Liu Sijia Liu Zheng-Wei Zhang PINN 27 11 0 18 Aug 2023
Convergence of Two-Layer Regression with Nonlinear Units Yichuan Deng Zhao-quan Song Shenghao Xie 26 7 0 16 Aug 2023
Fine-Tuning Language Models with Just Forward Passes Sadhika Malladi Tianyu Gao Eshaan Nichani Alexandru Damian Jason D. Lee Danqi Chen Sanjeev Arora 27 177 0 27 May 2023
Certified Zeroth-order Black-Box Defense with Robust UNet Denoiser Astha Verma A. Subramanyam Siddhesh Bangar Naman Lal R. Shah Shiníchi Satoh 37 4 0 13 Apr 2023
Zeroth-Order Hard-Thresholding: Gradient Error vs. Expansivity William de Vazelhes Hualin Zhang Huisi Wu Xiao-Tong Yuan Bin Gu 35 2 0 11 Oct 2022
Convergence of Batch Updating Methods with Approximate Gradients and/or Noisy Measurements: Theory and Computational Results Tadipatri Uday M. Vidyasagar 23 0 0 12 Sep 2022
Stochastic Zeroth order Descent with Structured Directions Marco Rando C. Molinari S. Villa Lorenzo Rosasco 31 6 0 10 Jun 2022
How to Robustify Black-Box ML Models? A Zeroth-Order Optimization Perspective Yimeng Zhang Yuguang Yao Jinghan Jia Jinfeng Yi Min-Fong Hong Shiyu Chang Sijia Liu AAML 23 33 0 27 Mar 2022
Curvature-Aware Derivative-Free Optimization Bumsu Kim HanQin Cai Daniel McKenzie W. Yin ODL 22 10 0 27 Sep 2021
Asynchronous Distributed Reinforcement Learning for LQR Control via Zeroth-Order Block Coordinate Descent Gangshan Jing H. Bai Jemin George A. Chakrabortty P. Sharma 14 8 0 26 Jul 2021
Zeroth-Order Regularized Optimization (ZORO): Approximately Sparse Gradients and Adaptive Sampling HanQin Cai Daniel McKenzie W. Yin Zhenliang Zhang 38 48 0 29 Mar 2020