Decoupled Neural Interfaces using Synthetic Gradients

18 August 2016

Max Jaderberg

Wojciech M. Czarnecki

David Silver

Papers citing "Decoupled Neural Interfaces using Synthetic Gradients"

50 / 209 papers shown

Title
Meta-Representational Predictive Coding: Biomimetic Self-Supervised Learning Alexander Ororbia Karl J. Friston Rajesh P. N. Rao SSL 57 0 0 22 Mar 2025
Training neural networks without backpropagation using particles Deepak Kumar 78 0 0 07 Dec 2024
Beyond Backpropagation: Optimization with Multi-Tangent Forward Gradients Katharina Flügel D. Coquelin Marie Weiel Achim Streit Markus Gotz 27 0 0 23 Oct 2024
Replacement Learning: Training Vision Tasks with Fewer Learnable Parameters Yuming Zhang Peizhe Wang Shouxin Zhang Dongzhi Guan Jiabin Liu Junhao Su 38 0 0 02 Oct 2024
Counter-Current Learning: A Biologically Plausible Dual Network Approach for Deep Learning Chia-Hsiang Kao Bharath Hariharan 48 1 0 30 Sep 2024
Unifying back-propagation and forward-forward algorithms through model predictive control Lianhai Ren Qianxiao Li 36 1 0 29 Sep 2024
Contrastive Learning in Memristor-based Neuromorphic Systems Cory Merkel Alexander Ororbia 27 1 0 17 Sep 2024
Training Spiking Neural Networks via Augmented Direct Feedback Alignment Yongbo Zhang Katsuma Inoue M. Nakajima Toshikazu Hashimoto Yasuo Kuniyoshi Kohei Nakajima 38 0 0 12 Sep 2024
Emulating Brain-like Rapid Learning in Neuromorphic Edge Computing Kenneth Stewart Michael Neumeier Sumit Bam Shrestha Garrick Orchard Emre Neftci 43 0 0 28 Aug 2024
HPFF: Hierarchical Locally Supervised Learning with Patch Feature Fusion Junhao Su Chenghao He Feiyu Zhu Xiaojie Xu Dongzhi Guan Chenyang Si 56 2 0 08 Jul 2024
Momentum Auxiliary Network for Supervised Local Learning Junhao Su Changpeng Cai Feiyu Zhu Chenghao He Xiaojie Xu Dongzhi Guan Chenyang Si 35 4 0 08 Jul 2024
Prospective Messaging: Learning in Networks with Communication Delays Ryan Fayyazi Christian D. Weilbach Frank Wood 27 0 0 07 Jul 2024
MLAAN: Scaling Supervised Local Learning with Multilaminar Leap Augmented Auxiliary Network Yuming Zhang Shouxin Zhang Peizhe Wang Feiyu Zhu Dongzhi Guan Junhao Su Jiabin Liu Changpeng Cai 33 2 0 24 Jun 2024
Towards Interpretable Deep Local Learning with Successive Gradient Reconciliation Yibo Yang Xiaojie Li Motasem Alfarra Hasan Hammoud Adel Bibi Philip Torr Guohao Li 37 2 0 07 Jun 2024
PETRA: Parallel End-to-end Training with Reversible Architectures Stéphane Rivaud Louis Fournier Thomas Pumir Eugene Belilovsky Michael Eickenberg Edouard Oyallon 25 0 0 04 Jun 2024
Self-Selected Attention Span for Accelerating Large Language Model Inference Tian Jin W. Yazar Zifei Xu Sayeh Sharify Xin Wang LRM 29 1 0 14 Apr 2024
Lightweight Inference for Forward-Forward Algorithm Amin Aminifar Baichuan Huang Azra Abtahi Amir Aminifar 35 3 0 08 Apr 2024
Approximated Likelihood Ratio: A Forward-Only and Parallel Framework for Boosting Neural Network Training Zeliang Zhang Jinyang Jiang Zhuo Liu Susan Liang Yijie Peng Chenliang Xu 29 0 0 18 Mar 2024
Forward Learning of Graph Neural Networks Namyong Park Xing Wang Antoine Simoulin Shuai Yang Grey Yang Ryan Rossi Puja Trivedi Nesreen K. Ahmed GNN 47 1 0 16 Mar 2024
Decoupled Vertical Federated Learning for Practical Training on Vertically Partitioned Data Avi Amalanshu Yash Sirvi David I. Inouye FedML 37 1 0 06 Mar 2024
Respiratory motion forecasting with online learning of recurrent neural networks for safety enhancement in externally guided radiotherapy Michel Pohl Mitsuru Uesaka Hiroyuki Takahashi K. Demachi R. B. Chhatkuli 32 2 0 03 Mar 2024
Scaling Supervised Local Learning with Augmented Auxiliary Networks Chenxiang Ma Jibin Wu Chenyang Si Kay Chen Tan 49 2 0 27 Feb 2024
Moonwalk: Inverse-Forward Differentiation Dmitrii Krylov Armin Karamzade Roy Fox 19 1 0 22 Feb 2024
Revisiting Zeroth-Order Optimization for Memory-Efficient LLM Fine-Tuning: A Benchmark Yihua Zhang Pingzhi Li Junyuan Hong Jiaxiang Li Yimeng Zhang ... Wotao Yin Mingyi Hong Zhangyang Wang Sijia Liu Tianlong Chen 28 45 0 18 Feb 2024
A Review of Neuroscience-Inspired Machine Learning Alexander Ororbia A. Mali Adam Kohan Beren Millidge Tommaso Salvatori 29 7 0 16 Feb 2024
Two Tales of Single-Phase Contrastive Hebbian Learning R. Høier Christopher Zach 33 1 0 13 Feb 2024
FedImpro: Measuring and Improving Client Update in Federated Learning Zhenheng Tang Yonggang Zhang S. Shi Xinmei Tian Tongliang Liu Bo Han Xiaowen Chu FedML 26 13 0 10 Feb 2024
Multi-Sender Persuasion: A Computational Perspective Safwan Hossain Tonghan Wang Tao Lin Yiling Chen David C. Parkes Haifeng Xu 35 10 0 07 Feb 2024
Unlocking Deep Learning: A BP-Free Approach for Parallel Block-Wise Training of Neural Networks Anzhe Cheng Zhenkun Wang Chenzhong Yin Mingxi Cheng Heng Ping Xiongye Xiao Shahin Nazarian Paul Bogdan 23 2 0 20 Dec 2023
Go beyond End-to-End Training: Boosting Greedy Local Learning with Context Supply Chengting Yu Fengzhao Zhang Hanzhi Ma Aili Wang Er-ping Li 29 1 0 12 Dec 2023
How to guess a gradient Utkarsh Singhal Brian Cheung Kartik Chandra Jonathan Ragan-Kelley Joshua B. Tenenbaum Tomaso Poggio Stella X. Yu ODL 28 3 0 07 Dec 2023
The Trifecta: Three simple techniques for training deeper Forward-Forward networks Thomas Dooms Ing Jyh Tsang José Oramas 38 4 0 29 Nov 2023
Leader-Follower Neural Networks with Local Error Signals Inspired by Complex Collectives Chenzhong Yin Mingxi Cheng Xiongye Xiao Xinghe Chen Shahin Nazarian Andrei Irimia Paul Bogdan 24 2 0 11 Oct 2023
DeepZero: Scaling up Zeroth-Order Optimization for Deep Model Training Aochuan Chen Yimeng Zhang Jinghan Jia James Diffenderfer Jiancheng Liu Konstantinos Parasyris Yihua Zhang Zheng-Wei Zhang B. Kailkhura Sijia Liu 30 43 0 03 Oct 2023
Module-wise Training of Neural Networks via the Minimizing Movement Scheme Skander Karkar Bhaskar Sen Emmanuel de Bezenac Patrick Gallinari 33 2 0 29 Sep 2023
Can Forward Gradient Match Backpropagation? Louis Fournier Stéphane Rivaud Eugene Belilovsky Michael Eickenberg Edouard Oyallon 19 16 0 12 Jun 2023
Online learning of long-range dependencies Nicolas Zucchet Robert Meier Simon Schug Asier Mujika João Sacramento CLL 49 18 0 25 May 2023
Block-local learning with probabilistic latent representations David Kappel Khaleelulla Khan Nazeer Cabrel Teguemne Fokam Christian Mayr Anand Subramoney 24 4 0 24 May 2023
ADA-GP: Accelerating DNN Training By Adaptive Gradient Prediction Vahid Janfaza Shantanu Mandal Farabi Mahmud A. Muzahid 19 2 0 22 May 2023
Feed-Forward Optimization With Delayed Feedback for Neural Networks Katharina Flügel D. Coquelin Marie Weiel Charlotte Debus Achim Streit Markus Goetz AI4CE 40 7 0 26 Apr 2023
Decouple Graph Neural Networks: Train Multiple Simple GNNs Simultaneously Instead of One Hongyuan Zhang Yanan Zhu Xuelong Li 27 9 0 20 Apr 2023
Contrastive-Signal-Dependent Plasticity: Forward-Forward Learning of Spiking Neural Systems Alexander Ororbia 25 12 0 30 Mar 2023
Dual Propagation: Accelerating Contrastive Hebbian Learning with Dyadic Neurons R. Høier D. Staudt Christopher Zach 31 11 0 02 Feb 2023
Local Learning with Neuron Groups Adeetya Patel Michael Eickenberg Eugene Belilovsky 29 5 0 18 Jan 2023
The Predictive Forward-Forward Algorithm Alexander Ororbia A. Mali 32 37 0 04 Jan 2023
Local Learning on Transformers via Feature Reconstruction P. Pathak Jingwei Zhang Dimitris Samaras ViT 24 5 0 29 Dec 2022
Deep Incubation: Training Large Models by Divide-and-Conquering Zanlin Ni Yulin Wang Jiangwei Yu Haojun Jiang Yu Cao Gao Huang VLM 18 11 0 08 Dec 2022
Convolutional Neural Generative Coding: Scaling Predictive Coding to Natural Images Alexander Ororbia A. Mali BDL 35 10 0 22 Nov 2022
Scaling Laws Beyond Backpropagation Matthew J. Filipovich Alessandro Cappelli Daniel Hesslow Julien Launay 19 3 0 26 Oct 2022
Block-wise Training of Residual Networks via the Minimizing Movement Scheme Skander Karkar Ibrahim Ayed Emmanuel de Bézenac Patrick Gallinari 33 1 0 03 Oct 2022