Decoupled Neural Interfaces using Synthetic Gradients

18 August 2016

Max Jaderberg

Wojciech M. Czarnecki

David Silver

Papers citing "Decoupled Neural Interfaces using Synthetic Gradients"

50 / 209 papers shown

Title
Just Pick a Sign: Optimizing Deep Multitask Models with Gradient Sign Dropout Zhao Chen Jiquan Ngiam Yanping Huang Thang Luong Henrik Kretzschmar Yuning Chai Dragomir Anguelov 41 206 0 14 Oct 2020
Interlocking Backpropagation: Improving depthwise model-parallelism Aidan Gomez Oscar Key Kuba Perlin Stephen Gou Nick Frosst J. Dean Y. Gal 8 19 0 08 Oct 2020
Feed-Forward On-Edge Fine-tuning Using Static Synthetic Gradient Modules R. Neven Marian Verhelst Tinne Tuytelaars Toon Goedemé 21 1 0 21 Sep 2020
Review: Deep Learning in Electron Microscopy Jeffrey M. Ede 34 79 0 17 Sep 2020
Learning Functors using Gradient Descent Bruno Gavranovic 19 5 0 15 Sep 2020
A Practical Layer-Parallel Training Algorithm for Residual Networks Qi Sun Hexin Dong Zewei Chen Weizhen Dian Jiacheng Sun Yitong Sun Zhenguo Li Bin Dong ODL 22 2 0 03 Sep 2020
LoCo: Local Contrastive Representation Learning Yuwen Xiong Mengye Ren R. Urtasun SSL DRL 35 69 0 04 Aug 2020
Universality of Gradient Descent Neural Network Training G. Welper 21 7 0 27 Jul 2020
Meta-rPPG: Remote Heart Rate Estimation Using a Transductive Meta-Learner Eugene Lee E. Chen Chen-Yi Lee 24 159 0 14 Jul 2020
A Theoretical Framework for Target Propagation Alexander Meulemans Francesco S. Carzaniga Johan A. K. Suykens João Sacramento Benjamin Grewe AAML 27 77 0 25 Jun 2020
Direct Feedback Alignment Scales to Modern Deep Learning Tasks and Architectures Julien Launay Iacopo Poli Franccois Boniface Florent Krzakala 36 62 0 23 Jun 2020
Extension of Direct Feedback Alignment to Convolutional and Recurrent Neural Network for Bio-plausible Deep Learning Donghyeon Han Gwangtae Park Junha Ryu H. Yoo 3DV 10 5 0 23 Jun 2020
Parameter-Based Value Functions Francesco Faccio Louis Kirsch Jürgen Schmidhuber OffRL 20 24 0 16 Jun 2020
Interaction Networks: Using a Reinforcement Learner to train other Machine Learning algorithms Florian Dietz 17 1 0 15 Jun 2020
On the Impossibility of Global Convergence in Multi-Loss Optimization Alistair Letcher 24 32 0 26 May 2020
Deep Learning: Our Miraculous Year 1990-1991 J. Schmidhuber 3DGS MedIm 22 6 0 12 May 2020
Modularizing Deep Learning via Pairwise Learning With Kernels Shiyu Duan Shujian Yu José C. Príncipe MoMe 27 20 0 12 May 2020
Empirical Bayes Transductive Meta-Learning with Synthetic Gradients S. Hu Pablo G. Moreno Yanghua Xiao Xin Shen G. Obozinski Neil D. Lawrence Andreas C. Damianou BDL 27 125 0 27 Apr 2020
Self-Supervised 3D Human Pose Estimation via Part Guided Novel Image Synthesis Jogendra Nath Kundu Siddharth Seth Varun Jampani M. Rakesh R. Venkatesh Babu Anirban Chakraborty SSL 3DH 16 79 0 09 Apr 2020
Policy Evaluation Networks J. Harb Tom Schaul Doina Precup Pierre-Luc Bacon OffRL 4 36 0 26 Feb 2020
Bounding the expected run-time of nonconvex optimization with early stopping Thomas Flynn K. Yu A. Malik Nicolas DÍmperio Shinjae Yoo 18 2 0 20 Feb 2020
Towards Crowdsourced Training of Large Neural Networks using Decentralized Mixture-of-Experts Max Ryabinin Anton I. Gusev FedML 27 48 0 10 Feb 2020
Large-Scale Gradient-Free Deep Learning with Recursive Local Representation Alignment Alexander Ororbia A. Mali Daniel Kifer C. Lee Giles 13 2 0 10 Feb 2020
Sideways: Depth-Parallel Training of Video Models Mateusz Malinowski G. Swirszcz João Carreira Viorica Patraucean MDE 33 13 0 17 Jan 2020
Questions to Guide the Future of Artificial Intelligence Research J. Ott 14 3 0 21 Dec 2019
Advances and Open Problems in Federated Learning Peter Kairouz H. B. McMahan Brendan Avent A. Bellet M. Bennis ... Zheng Xu Qiang Yang Felix X. Yu Han Yu Sen Zhao FedML AI4CE 74 6,079 0 10 Dec 2019
Ghost Units Yield Biologically Plausible Backprop in Deep Neural Networks Thomas Mesnard Gaetan Vignoud João Sacramento Walter Senn Yoshua Bengio 27 7 0 15 Nov 2019
Label-Conditioned Next-Frame Video Generation with Neural Flows Sergey Tarasenko VGen 21 1 0 16 Oct 2019
Decoupling Hierarchical Recurrent Neural Networks With Locally Computable Losses Asier Mujika Felix Weissenberger Angelika Steger 10 0 0 11 Oct 2019
Learning to Remember from a Multi-Task Teacher Yuwen Xiong Mengye Ren R. Urtasun CLL KELM OOD 24 4 0 10 Oct 2019
Meta-Learning Deep Energy-Based Memory Models Sergey Bartunov Jack W. Rae Simon Osindero Timothy Lillicrap 45 34 0 07 Oct 2019
Gated Linear Networks William H. Guss Tor Lattimore David Budden Avishkar Bhoopchand Christopher Mattern ... Ruslan Salakhutdinov Jianan Wang Peter Toth Simon Schmitt Marcus Hutter AI4CE 18 40 0 30 Sep 2019
Ouroboros: On Accelerating Training of Transformer-Based Language Models Qian Yang Zhouyuan Huo Wenlin Wang Heng-Chiao Huang Lawrence Carin 17 9 0 14 Sep 2019
On the Acceleration of Deep Learning Model Parallelism with Staleness An Xu Zhouyuan Huo Heng-Chiao Huang 8 37 0 05 Sep 2019
Learning without feedback: Fixed random learning signals allow for feedforward training of deep neural networks Charlotte Frenkel M. Lefebvre D. Bol 11 23 0 03 Sep 2019
Spiking Neural Predictive Coding for Continual Learning from Data Streams Alexander Ororbia 21 25 0 23 Aug 2019
Metalearned Neural Memory Tsendsuren Munkhdalai Alessandro Sordoni Tong Wang Adam Trischler KELM 17 60 0 23 Jul 2019
Compositional Deep Learning Bruno Gavranovic BDL AI4CE 17 7 0 16 Jul 2019
A Unified Framework of Online Learning Algorithms for Training Recurrent Neural Networks O. Marschall Kyunghyun Cho Cristina Savin FedML 33 72 0 05 Jul 2019
Fully Decoupled Neural Network Learning Using Delayed Gradients Huiping Zhuang Yi Wang Qinglai Liu Shuai Zhang Zhiping Lin FedML 14 29 0 21 Jun 2019
Divide and Conquer: Leveraging Intermediate Feature Representations for Quantized Training of Neural Networks Ahmed T. Elthakeb Prannoy Pilligundla Alex Cloninger H. Esmaeilzadeh MQ 26 8 0 14 Jun 2019
Associated Learning: Decomposing End-to-end Backpropagation based on Auto-encoders and Target Propagation Yu-Wei Kao Hung-Hsuan Chen BDL 20 5 0 13 Jun 2019
Principled Training of Neural Networks with Direct Feedback Alignment Julien Launay Iacopo Poli Florent Krzakala 19 35 0 11 Jun 2019
Learning to solve the credit assignment problem B. Lansdell P. Prakash Konrad Paul Kording 11 50 0 03 Jun 2019
Using local plasticity rules to train recurrent neural networks O. Marschall Kyunghyun Cho Cristina Savin 9 2 0 28 May 2019
Putting An End to End-to-End: Gradient-Isolated Learning of Representations Sindy Löwe Peter O'Connor Bastiaan S. Veeling SSL 6 143 0 28 May 2019
Lifelong Neural Predictive Coding: Learning Cumulatively Online without Forgetting Alexander Ororbia A. Mali Daniel Kifer C. Lee Giles CLL KELM 32 15 0 25 May 2019
Efficient Optimization of Loops and Limits with Randomized Telescoping Sums Alex Beatson Ryan P. Adams 17 21 0 16 May 2019
Formal derivation of Mesh Neural Networks with their Forward-Only gradient Propagation Federico A. Galatolo M. G. Cimino G. Vaglini AI4CE 6 2 0 16 May 2019
Differentiable Game Mechanics Alistair Letcher David Balduzzi S. Racanière James Martens Jakob N. Foerster K. Tuyls T. Graepel 37 79 0 13 May 2019