Decoupled Neural Interfaces using Synthetic Gradients

18 August 2016

Max Jaderberg

Wojciech M. Czarnecki

David Silver

Papers citing "Decoupled Neural Interfaces using Synthetic Gradients"

50 / 209 papers shown

Title
Hebbian Deep Learning Without Feedback Adrien Journé Hector Garcia Rodriguez Qinghai Guo Timoleon Moraitis AAML 31 49 0 23 Sep 2022
Active Predicting Coding: Brain-Inspired Reinforcement Learning for Sparse Reward Robotic Control Problems Alexander Ororbia A. Mali 40 7 0 19 Sep 2022
Biologically Plausible Training of Deep Neural Networks Using a Top-down Credit Assignment Network Jian-Hui Chen Cheng-Lin Liu Zuoren Wang 26 0 0 01 Aug 2022
Layer-Wise Partitioning and Merging for Efficient and Scalable Deep Learning S. Akintoye Liangxiu Han H. Lloyd Xin Zhang Darren Dancey Haoming Chen Daoqiang Zhang FedML 34 5 0 22 Jul 2022
GLEAM: Greedy Learning for Large-Scale Accelerated MRI Reconstruction Batu Mehmet Ozturkler Arda Sahiner Tolga Ergen Arjun D Desai Christopher M. Sandino S. Vasanawala John M. Pauly Morteza Mardani Mert Pilanci 24 4 0 18 Jul 2022
Emergent Abilities of Large Language Models Jason W. Wei Yi Tay Rishi Bommasani Colin Raffel Barret Zoph ... Tatsunori Hashimoto Oriol Vinyals Percy Liang J. Dean W. Fedus ELM ReLM LRM 84 2,354 0 15 Jun 2022
A Robust Backpropagation-Free Framework for Images Timothy Zee Alexander Ororbia A. Mali Ifeoma Nwogu 20 1 0 03 Jun 2022
BackLink: Supervised Local Training with Backward Links Wenzhe Guo M. Fouda A. Eltawil K. Salama 21 2 0 14 May 2022
Signal Propagation: A Framework for Learning and Inference In a Forward Pass Adam A. Kohan E. Rietman H. Siegelmann 21 26 0 04 Apr 2022
The Frost Hollow Experiments: Pavlovian Signalling as a Path to Coordination and Communication Between Agents P. Pilarski Andrew Butcher Elnaz Davoodi Michael Bradley Johanson Dylan J. A. Brenneis Adam S. R. Parker Leslie Acker M. Botvinick Joseph Modayil Adam White AI4CE 14 3 0 17 Mar 2022
Gradient Correction beyond Gradient Descent Zefan Li Bingbing Ni Teng Li WenJun Zhang Wen Gao 9 0 0 16 Mar 2022
MetAug: Contrastive Learning via Meta Feature Augmentation Jiangmeng Li Wenwen Qiang Changwen Zheng Bing-Huang Su Hui Xiong 32 23 0 10 Mar 2022
Efficient Attribute Unlearning: Towards Selective Removal of Input Attributes from Feature Representations Tao Guo Song Guo Jiewei Zhang Wenchao Xu Junxiao Wang MU 27 17 0 27 Feb 2022
Gradients without Backpropagation A. G. Baydin Barak A. Pearlmutter Don Syme Frank Wood Philip Torr 38 66 0 17 Feb 2022
Error-driven Input Modulation: Solving the Credit Assignment Problem without a Backward Pass Giorgia Dellaferrera Gabriel Kreiman 29 53 0 27 Jan 2022
Learning to Predict Gradients for Semi-Supervised Continual Learning Yan Luo Yongkang Wong Mohan S. Kankanhalli Qi Zhao SSL CLL 44 6 0 23 Jan 2022
Neural Capacitance: A New Perspective of Neural Network Selection via Edge Dynamics Chunheng Jiang Tejaswini Pedapati Pin-Yu Chen Yizhou Sun Jianxi Gao 24 2 0 11 Jan 2022
Layer-Parallel Training of Residual Networks with Auxiliary-Variable Networks Qi Sun Hexin Dong Zewei Chen Jiacheng Sun Zhenguo Li Bin Dong 29 1 0 10 Dec 2021
Target Propagation via Regularized Inversion Vincent Roulet Zaïd Harchaoui BDL AAML 27 4 0 02 Dec 2021
On Training Implicit Models Zhengyang Geng Xinyu Zhang Shaojie Bai Yisen Wang Zhouchen Lin 66 69 0 09 Nov 2021
Cortico-cerebellar networks as decoupling neural interfaces J. Pemberton E. Boven Richard Apps Rui Ponte Costa 33 6 0 21 Oct 2021
Biologically Plausible Training Mechanisms for Self-Supervised Learning in Deep Networks Mufeng Tang Yibo Yang Y. Amit SSL 16 7 0 30 Sep 2021
LayerPipe: Accelerating Deep Neural Network Training by Intra-Layer and Inter-Layer Gradient Pipelining and Multiprocessor Scheduling Nanda K. Unnikrishnan Keshab K. Parhi AI4CE 15 5 0 14 Aug 2021
Knowledge accumulating: The general pattern of learning Zhuoran Xu Hao Liu CLL 24 0 0 09 Aug 2021
Few-Shot and Continual Learning with Attentive Independent Mechanisms Eugene Lee Cheng-Han Huang Chen-Yi Lee CLL 23 25 0 29 Jul 2021
Backprop-Free Reinforcement Learning with Active Neural Generative Coding Alexander Ororbia A. Mali 41 15 0 10 Jul 2021
Decoupled Greedy Learning of CNNs for Synchronous and Asynchronous Distributed Learning Eugene Belilovsky Louis Leconte Lucas Caccia Michael Eickenberg Edouard Oyallon 14 7 0 11 Jun 2021
LocoProp: Enhancing BackProp via Local Loss Optimization Ehsan Amid Rohan Anil Manfred K. Warmuth ODL 13 19 0 11 Jun 2021
Front Contribution instead of Back Propagation Swaroop Mishra Anjana Arunkumar 12 0 0 10 Jun 2021
Bottom-up and top-down approaches for the design of neuromorphic processing systems: Tradeoffs and synergies between natural and artificial intelligence Charlotte Frenkel D. Bol Giacomo Indiveri 34 33 0 02 Jun 2021
Prediction of the Position of External Markers Using a Recurrent Neural Network Trained With Unbiased Online Recurrent Optimization for Safe Lung Cancer Radiotherapy Michel Pohl Mitsuru Uesaka Hiroyuki Takahashi K. Demachi R. B. Chhatkuli 25 5 0 02 Jun 2021
Current State and Future Directions for Learning in Biological Recurrent Neural Networks: A Perspective Piece Luke Y. Prince Roy Henha Eyono E. Boven Arna Ghosh Joe Pemberton ... Rui Ponte Costa Wolfgang Maass Blake A. Richards Cristina Savin K. Wilmes CLL 13 3 0 12 May 2021
Perceptual Gradient Networks Dmitry Nikulin Roman Suvorov Aleksei Ivakhnenko Victor Lempitsky 13 0 0 05 May 2021
Local Critic Training for Model-Parallel Learning of Deep Neural Networks Hojung Lee Cho-Jui Hsieh Jong-Seok Lee 28 15 0 03 Feb 2021
Revisiting Locally Supervised Learning: an Alternative to End-to-end Training Yulin Wang Zanlin Ni Shiji Song Le Yang Gao Huang 9 82 0 26 Jan 2021
Training Deep Architectures Without End-to-End Backpropagation: A Survey on the Provably Optimal Methods Shiyu Duan José C. Príncipe MQ 38 3 0 09 Jan 2021
Advances in Electron Microscopy with Deep Learning Jeffrey M. Ede 35 2 0 04 Jan 2021
Differentiable Programming à la Moreau Vincent Roulet Zaïd Harchaoui 20 5 0 31 Dec 2020
Reservoir Transformers Sheng Shen Alexei Baevski Ari S. Morcos Kurt Keutzer Michael Auli Douwe Kiela 35 17 0 30 Dec 2020
Hardware Beyond Backpropagation: a Photonic Co-Processor for Direct Feedback Alignment Julien Launay Iacopo Poli Kilian Muller Gustave Pariente I. Carron L. Daudet Florent Krzakala S. Gigan MoE 15 18 0 11 Dec 2020
Parallel Training of Deep Networks with Local Updates Michael Laskin Luke Metz Seth Nabarrao Mark Saroufim Badreddine Noune Carlo Luschi Jascha Narain Sohl-Dickstein Pieter Abbeel FedML 16 26 0 07 Dec 2020
The Neural Coding Framework for Learning Generative Models Alexander Ororbia Daniel Kifer GAN 26 65 0 07 Dec 2020
Accumulated Decoupled Learning: Mitigating Gradient Staleness in Inter-Layer Model Parallelization Huiping Zhuang Zhiping Lin Kar-Ann Toh 26 4 0 03 Dec 2020
On-Chip Error-triggered Learning of Multi-layer Memristive Spiking Neural Networks Melika Payvand M. Fouda Fadi J. Kurdahi A. Eltawil Emre Neftci 32 29 0 21 Nov 2020
Self Normalizing Flows Thomas Anderson Keller Jorn W. T. Peters P. Jaini Emiel Hoogeboom Patrick Forré Max Welling 30 14 0 14 Nov 2020
Fast & Slow Learning: Incorporating Synthetic Gradients in Neural Memory Controllers Tharindu Fernando Simon Denman Sridha Sridharan Clinton Fookes 14 0 0 10 Nov 2020
From Eye-blinks to State Construction: Diagnostic Benchmarks for Online Representation Learning Banafsheh Rafiee Zaheer Abbas Sina Ghiassian Raksha Kumaraswamy R. Sutton Elliot A. Ludvig Adam White OffRL 19 17 0 09 Nov 2020
SplitEasy: A Practical Approach for Training ML models on Mobile Devices Kamalesh Palanisamy Vivek Khimani Moin Hussain Moti Dimitris Chatzopoulos 22 20 0 09 Nov 2020
Modular-Relatedness for Continual Learning Ammar Shaker Shujian Yu Francesco Alesiani KELM CLL 4 2 0 02 Nov 2020
Why Layer-Wise Learning is Hard to Scale-up and a Possible Solution via Accelerated Downsampling Wenchi Ma Miao Yu Kaidong Li Guanghui Wang 14 5 0 15 Oct 2020