An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networks

21 December 2013

Aaron Courville

Papers citing "An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networks"

50 / 292 papers shown

Title
Regularizing Towards Permutation Invariance in Recurrent Models Edo Cohen-Karlik Avichai Ben David Amir Globerson OOD 24 16 0 25 Oct 2020
Continual Learning in Low-rank Orthogonal Subspaces Arslan Chaudhry Naeemullah Khan P. Dokania Philip Torr CLL 33 115 0 22 Oct 2020
CLAR: Contrastive Learning of Auditory Representations Haider Al-Tahan Y. Mohsenzadeh SSL 118 56 0 19 Oct 2020
FedAT: A High-Performance and Communication-Efficient Federated Learning System with Asynchronous Tiers Zheng Chai Yujing Chen Ali Anwar Liang Zhao Yue Cheng Huzefa Rangwala FedML 26 122 0 12 Oct 2020
Linear Mode Connectivity in Multitask and Continual Learning Seyed Iman Mirzadeh Mehrdad Farajtabar Dilan Görür Razvan Pascanu H. Ghasemzadeh CLL 37 139 0 09 Oct 2020
A Theoretical Analysis of Catastrophic Forgetting through the NTK Overlap Matrix T. Doan Mehdi Abbana Bennani Bogdan Mazoure Guillaume Rabusseau Pierre Alquier CLL 20 80 0 07 Oct 2020
Robust High-dimensional Memory-augmented Neural Networks G. Karunaratne Manuel Schmuck Manuel Le Gallo G. Cherubini Luca Benini Abu Sebastian Abbas Rahimi 23 99 0 05 Oct 2020
Reusing a Pretrained Language Model on Languages with Limited Corpora for Unsupervised NMT Alexandra Chronopoulou Dario Stojanovski Alexander Fraser 18 33 0 16 Sep 2020
An Empirical Analysis of Backward Compatibility in Machine Learning Systems Megha Srivastava Besmira Nushi Ece Kamar S. Shah Eric Horvitz AAML 31 44 0 11 Aug 2020
Anatomy of Catastrophic Forgetting: Hidden Representations and Task Semantics V. Ramasesh Ethan Dyer M. Raghu CLL 24 174 0 14 Jul 2020
RATT: Recurrent Attention to Transient Tasks for Continual Image Captioning Riccardo Del Chiaro Bartlomiej Twardowski Andrew D. Bagdanov Joost van de Weijer CLL VLM 30 40 0 13 Jul 2020
Graph-Based Continual Learning Binh Tang David S. Matteson BDL CLL 29 37 0 09 Jul 2020
Gradient-based Editing of Memory Examples for Online Task-free Continual Learning Xisen Jin Arka Sadhu Junyi Du Xiang Ren CLL KELM BDL 28 96 0 27 Jun 2020
Supermasks in Superposition Mitchell Wortsman Vivek Ramanujan Rosanne Liu Aniruddha Kembhavi Mohammad Rastegari J. Yosinski Ali Farhadi SSL CLL 33 279 0 26 Jun 2020
Generalisation Guarantees for Continual Learning with Orthogonal Gradient Descent Mehdi Abbana Bennani Thang Doan Masashi Sugiyama CLL 50 61 0 21 Jun 2020
Understanding the Role of Training Regimes in Continual Learning Seyed Iman Mirzadeh Mehrdad Farajtabar Razvan Pascanu H. Ghasemzadeh CLL 21 219 0 12 Jun 2020
Coresets via Bilevel Optimization for Continual Learning and Streaming Zalan Borsos Mojmír Mutný Andreas Krause CLL 38 226 0 06 Jun 2020
Learning to Recognize Code-switched Speech Without Forgetting Monolingual Speech Recognition Sanket Shah Basil Abraham M. GurunathReddy Sunayana Sitaram Vikas Joshi 21 18 0 01 Jun 2020
Common Sense or World Knowledge? Investigating Adapter-Based Knowledge Injection into Pretrained Transformers Anne Lauscher Olga Majewska Leonardo F. R. Ribeiro Iryna Gurevych Nikolai Rozanov Goran Glavavs KELM 39 79 0 24 May 2020
Spiking Neural Networks Hardware Implementations and Challenges: a Survey Maxence Bouvier A. Valentian T. Mesquida F. Rummens M. Reyboz Elisa Vianello E. Beigné 56 113 0 04 May 2020
Recall and Learn: Fine-tuning Deep Pretrained Language Models with Less Forgetting Sanyuan Chen Yutai Hou Yiming Cui Wanxiang Che Ting Liu Xiangzhan Yu KELM CLL 21 213 0 27 Apr 2020
Generative Data Augmentation for Commonsense Reasoning Yiben Yang Chaitanya Malaviya Jared Fernandez Swabha Swayamdipta Ronan Le Bras Ji-ping Wang Chandra Bhagavatula Yejin Choi Doug Downey LRM 30 91 0 24 Apr 2020
Continual Learning with Extended Kronecker-factored Approximate Curvature Janghyeon Lee H. Hong Donggyu Joo Junmo Kim CLL 20 52 0 16 Apr 2020
Unpacking Information Bottlenecks: Unifying Information-Theoretic Objectives in Deep Learning Andreas Kirsch Clare Lyle Y. Gal 27 16 0 27 Mar 2020
An Energy-Aware Online Learning Framework for Resource Management in Heterogeneous Platforms Sumit K. Mandal Ganapati Bhat J. Doppa P. Pande Ümit Y. Ogras 9 39 0 20 Mar 2020
Synaptic Metaplasticity in Binarized Neural Networks Axel Laborieux M. Ernoult T. Hirtzlin D. Querlioz CLL 26 62 0 07 Mar 2020
Training Binary Neural Networks using the Bayesian Learning Rule Xiangming Meng Roman Bachmann Mohammad Emtiyaz Khan BDL MQ 30 40 0 25 Feb 2020
Three Approaches for Personalization with Applications to Federated Learning Yishay Mansour M. Mohri Jae Hun Ro A. Suresh FedML 45 565 0 25 Feb 2020
Using Hindsight to Anchor Past Knowledge in Continual Learning Arslan Chaudhry Albert Gordo P. Dokania Philip Torr David Lopez-Paz KELM CLL 27 234 0 19 Feb 2020
Effective Data Augmentation with Multi-Domain Learning GANs Shin'ya Yamaguchi Sekitoshi Kanai Takeharu Eda 27 27 0 25 Dec 2019
Direction Concentration Learning: Enhancing Congruency in Machine Learning Yan Luo Yongkang Wong Mohan Kankanhalli Qi Zhao 23 12 0 17 Dec 2019
Neural Machine Translation: A Review and Survey Felix Stahlberg 3DV AI4TS MedIm 25 312 0 04 Dec 2019
Adaptive Online Planning for Continual Lifelong Learning Kevin Lu Igor Mordatch Pieter Abbeel OffRL OnRL CLL 11 15 0 03 Dec 2019
Latent Replay for Real-Time Continual Learning Lorenzo Pellegrini G. Graffieti Vincenzo Lomonaco Davide Maltoni CLL 28 163 0 02 Dec 2019
REFIT: A Unified Watermark Removal Framework For Deep Learning Systems With Limited Data Xinyun Chen Wenxiao Wang Chris Bender Yiming Ding R. Jia Bo-wen Li D. Song AAML 27 107 0 17 Nov 2019
OpenLORIS-Object: A Robotic Vision Dataset and Benchmark for Lifelong Deep Learning Qi She Fan Feng Xinyue Hao Qihan Yang Chuanlin Lan ... Zhengwei Wang Yao Guo Yimin Zhang Fei Qiao Rosa H. M. Chan VLM 38 75 0 15 Nov 2019
Knowledge Distillation for Incremental Learning in Semantic Segmentation Umberto Michieli Pietro Zanuttigh CLL VLM 28 98 0 08 Nov 2019
Adversarial Feature Alignment: Avoid Catastrophic Forgetting in Incremental Task Lifelong Learning Hang Zhou Ziwei Liu Chenglei Wu Ping Luo Lifeng Sun CLL 15 22 0 24 Oct 2019
Continual Learning Using Bayesian Neural Networks Honglin Li Payam Barnaghi Shirin Enshaeifar F. Ganz BDL 19 38 0 09 Oct 2019
Gated Linear Networks William H. Guss Tor Lattimore David Budden Avishkar Bhoopchand Christopher Mattern ... Ruslan Salakhutdinov Jianan Wang Peter Toth Simon Schmitt Marcus Hutter AI4CE 18 40 0 30 Sep 2019
Meta-learnt priors slow down catastrophic forgetting in neural networks G. Spigler CLL 25 10 0 09 Sep 2019
Domain Adaptive Text Style Transfer Dianqi Li Yizhe Zhang Zhe Gan Yu Cheng Chris Brockett Ming-Ting Sun W. Dolan OOD 31 55 0 25 Aug 2019
Federated Learning with Additional Mechanisms on Clients to Reduce Communication Costs Xin Yao Tianchi Huang Chenglei Wu Ruixiao Zhang Lifeng Sun FedML 18 38 0 16 Aug 2019
Towards Making the Most of BERT in Neural Machine Translation Jiacheng Yang Mingxuan Wang Hao Zhou Chengqi Zhao Yong Yu Weinan Zhang Lei Li CLL 21 156 0 15 Aug 2019
Continual Learning by Asymmetric Loss Approximation with Single-Side Overestimation Dongmin Park Seokil Hong Bohyung Han Kyoung Mu Lee AAML CLL 24 39 0 08 Aug 2019
Continual Learning for Robotics: Definition, Framework, Learning Strategies, Opportunities and Challenges Timothée Lesort Vincenzo Lomonaco Andrei Stoian Davide Maltoni David Filliat Natalia Díaz Rodríguez CLL 19 248 0 29 Jun 2019
End-to-End 3D-PointCloud Semantic Segmentation for Autonomous Driving Mohammed Abdou Mahmoud Elkhateeb Ibrahim Sobh Ahmad El-Sallab 3DPC 19 5 0 26 Jun 2019
Conditional Computation for Continual Learning Min Lin Jie Fu Yoshua Bengio CLL 31 10 0 16 Jun 2019
Sequential mastery of multiple visual tasks: Networks naturally learn to learn and forget to forget Guy Davidson Michael C. Mozer CLL 23 23 0 26 May 2019
Meta-learning of Sequential Strategies Pedro A. Ortega Jane X. Wang Mark Rowland Tim Genewein Z. Kurth-Nelson ... Yee Whye Teh H. V. Hasselt Nando de Freitas M. Botvinick Shane Legg OffRL 27 96 0 08 May 2019