Gradients without Backpropagation

17 February 2022

Papers citing "Gradients without Backpropagation"

46 / 46 papers shown

Title
Memory-Efficient LLM Training by Various-Grained Low-Rank Projection of Gradients Yezhen Wang Zhouhao Yang Brian K Chen Fanyi Pu Bo-wen Li Tianyu Gao Kenji Kawaguchi 41 0 0 03 May 2025
Binary Neural Networks for Large Language Model: A Survey Liangdong Liu Zhitong Zheng Cong Wang Tianhuang Su Z. Yang MQ 65 0 0 26 Feb 2025
Efficient Zero-Order Federated Finetuning of Language Models for Resource-Constrained Devices Mohamed Aboelenien Ahmed Kilian Pfeiffer R. Khalili Heba Khdr J. Henkel FedML 89 0 0 17 Feb 2025
Training neural networks without backpropagation using particles Deepak Kumar 70 0 0 07 Dec 2024
Randomized Forward Mode Gradient for Spiking Neural Networks in Scientific Machine Learning Ruyin Wan Qian Zhang George Karniadakis 23 0 0 11 Nov 2024
Poor Man's Training on MCUs: A Memory-Efficient Quantized Back-Propagation-Free Approach Yequan Zhao Hai Li Ian Young Zheng-Wei Zhang MQ 37 2 0 07 Nov 2024
Stepping Forward on the Last Mile Chen Feng Shaojie Zhuo Xiaopeng Zhang R. Ramakrishnan Zhaocong Yuan Andrew Zou Li 33 0 0 06 Nov 2024
FLOPS: Forward Learning with OPtimal Sampling Tao Ren Zishi Zhang Jinyang Jiang Guanghao Li Zeliang Zhang Mingqian Feng Yijie Peng 35 1 0 08 Oct 2024
Second-Order Forward-Mode Automatic Differentiation for Optimization Adam D. Cobb Atılım Güneş Baydin Barak A. Pearlmutter Susmit Jha ODL 31 1 0 19 Aug 2024
Online Pseudo-Zeroth-Order Training of Neuromorphic Spiking Neural Networks Mingqing Xiao Qingyan Meng Zongpeng Zhang D.K. He Zhouchen Lin 32 0 0 17 Jul 2024
Solving Token Gradient Conflict in Mixture-of-Experts for Large Vision-Language Model Longrong Yang Dong Shen Chaoxiang Cai Fan Yang Size Li Di Zhang Xi Li MoE 50 2 0 28 Jun 2024
Towards Biologically Plausible Computing: A Comprehensive Comparison Changze Lv Yufei Gu Zhengkang Guo Zhibo Xu Yixin Wu ... Tianlong Li Jianhao Zhu Cenyuan Zhang Zixuan Ling Xiaoqing Zheng 30 1 0 23 Jun 2024
Memory-Efficient Gradient Unrolling for Large-Scale Bi-level Optimization Qianli Shen Yezhen Wang Zhouhao Yang Xiang Li Haonan Wang Yang Zhang Jonathan Scarlett Zhanxing Zhu Kenji Kawaguchi AI4CE 64 4 0 20 Jun 2024
Towards Interpretable Deep Local Learning with Successive Gradient Reconciliation Yibo Yang Xiaojie Li Motasem Alfarra Hasan Hammoud Adel Bibi Philip H. S. Torr Bernard Ghanem 37 2 0 07 Jun 2024
Thinking Forward: Memory-Efficient Federated Finetuning of Language Models Kunjal Panchal Nisarg Parikh Sunav Choudhary Lijun Zhang Yuriy Brun Hui Guan 53 3 0 24 May 2024
FFCL: Forward-Forward Net with Cortical Loops, Training and Inference on Edge Without Backpropagation Ali Karkehabadi Houman Homayoun Avesta Sasan 24 9 0 21 May 2024
Gradient-Free Training of Recurrent Neural Networks using Random Perturbations Jesus Garcia Fernandez Sander Keemink Marcel van Gerven AAML 40 4 0 14 May 2024
Accelerating Legacy Numerical Solvers by Non-intrusive Gradient-based Meta-solving S. Arisaka Qianxiao Li 19 0 0 05 May 2024
Improving deep learning with prior knowledge and cognitive models: A survey on enhancing explainability, adversarial robustness and zero-shot learning F. Mumuni A. Mumuni AAML 37 5 0 11 Mar 2024
Scaling Supervised Local Learning with Augmented Auxiliary Networks Chenxiang Ma Jibin Wu Chenyang Si Kay Chen Tan 42 2 0 27 Feb 2024
Referee Can Play: An Alternative Approach to Conditional Generation via Model Inversion Xuantong Liu Tianyang Hu Wenjia Wang Kenji Kawaguchi Yuan Yao DiffM 73 3 0 26 Feb 2024
Revisiting Zeroth-Order Optimization for Memory-Efficient LLM Fine-Tuning: A Benchmark Yihua Zhang Pingzhi Li Junyuan Hong Jiaxiang Li Yimeng Zhang ... Wotao Yin Mingyi Hong Zhangyang Wang Sijia Liu Tianlong Chen 20 45 0 18 Feb 2024
End-to-End Training Induces Information Bottleneck through Layer-Role Differentiation: A Comparative Analysis with Layer-wise Training Keitaro Sakamoto Issei Sato 19 4 0 14 Feb 2024
Training Convolutional Neural Networks with the Forward-Forward algorithm Riccardo Scodellaro A. Kulkarni Frauke Alves Matthias Schröter 24 7 0 22 Dec 2023
How to guess a gradient Utkarsh Singhal Brian Cheung Kartik Chandra Jonathan Ragan-Kelley Joshua B. Tenenbaum Tomaso Poggio Stella X. Yu ODL 26 3 0 07 Dec 2023
Forward Gradients for Data-Driven CFD Wall Modeling Jan Huckelheim Tadbhagya Kumar Krishnan Raghavan Pinaki Pal AI4CE 15 0 0 20 Nov 2023
Randomized Forward Mode of Automatic Differentiation For Optimization Algorithms Khemraj Shukla Yeonjong Shin ODL 23 4 0 22 Oct 2023
DPZero: Private Fine-Tuning of Language Models without Backpropagation Liang Zhang Bingcong Li K. K. Thekumparampil Sewoong Oh Niao He 28 11 0 14 Oct 2023
Leader-Follower Neural Networks with Local Error Signals Inspired by Complex Collectives Chenzhong Yin Mingxi Cheng Xiongye Xiao Xinghe Chen Shahin Nazarian Andrei Irimia Paul Bogdan 19 2 0 11 Oct 2023
Smoothing Methods for Automatic Differentiation Across Conditional Branches J. N. Kreikemeyer Philipp Andelfinger 36 6 0 05 Oct 2023
DeepZero: Scaling up Zeroth-Order Optimization for Deep Model Training Aochuan Chen Yimeng Zhang Jinghan Jia James Diffenderfer Jiancheng Liu Konstantinos Parasyris Yihua Zhang Zheng-Wei Zhang B. Kailkhura Sijia Liu 30 43 0 03 Oct 2023
Effective Learning with Node Perturbation in Multi-Layer Neural Networks Sander Dalm Marcel van Gerven Nasir Ahmad AAML 25 0 0 02 Oct 2023
Hebbian learning inspired estimation of the linear regression parameters from queries Johannes Schmidt-Hieber Wouter M. Koolen 18 3 0 26 Sep 2023
Convergence guarantees for forward gradient descent in the linear regression model Thijs Bos Johannes Schmidt-Hieber 17 3 0 26 Sep 2023
FwdLLM: Efficient FedLLM using Forward Gradient Mengwei Xu Dongqi Cai Yaozong Wu Xiang Li Shangguang Wang FedML 49 24 0 26 Aug 2023
Tensor-Compressed Back-Propagation-Free Training for (Physics-Informed) Neural Networks Yequan Zhao Xinling Yu Zhixiong Chen Z. Liu Sijia Liu Zheng-Wei Zhang PINN 22 11 0 18 Aug 2023
Extending the Forward Forward Algorithm Saumya Gandhi Ritu Gala Jonah Kornberg Advaith Sridhar 25 6 0 09 Jul 2023
Can Forward Gradient Match Backpropagation? Louis Fournier Stephane Rivaud Eugene Belilovsky Michael Eickenberg Edouard Oyallon 19 16 0 12 Jun 2023
Online learning of long-range dependencies Nicolas Zucchet Robert Meier Simon Schug Asier Mujika João Sacramento CLL 39 18 0 25 May 2023
The Integrated Forward-Forward Algorithm: Integrating Forward-Forward and Shallow Backpropagation With Local Losses De Tang 21 3 0 22 May 2023
Feed-Forward Optimization With Delayed Feedback for Neural Networks Katharina Flügel D. Coquelin Marie Weiel Charlotte Debus Achim Streit Markus Goetz AI4CE 32 7 0 26 Apr 2023
Low-Variance Forward Gradients using Direct Feedback Alignment and Momentum Florian Bacho Dominique F. Chu 13 7 0 14 Dec 2022
Hybrid Decentralized Optimization: Leveraging Both First- and Zeroth-Order Optimizers for Faster Convergence Shayan Talaei Giorgi Nadiradze Giorgi Nadiradze Dan Alistarh 11 0 0 14 Oct 2022
Compute-Efficient Deep Learning: Algorithmic Trends and Opportunities Brian Bartoldson B. Kailkhura Davis W. Blalock 29 47 0 13 Oct 2022
Scaling Forward Gradient With Local Losses Mengye Ren Simon Kornblith Renjie Liao Geoffrey E. Hinton 78 49 0 07 Oct 2022
Optimization without Backpropagation Gabriel Belouze 19 7 0 13 Sep 2022