Generalisation Guarantees for Continual Learning with Orthogonal Gradient Descent

21 June 2020

Papers citing "Generalisation Guarantees for Continual Learning with Orthogonal Gradient Descent"

45 / 45 papers shown

Title
Memory-Statistics Tradeoff in Continual Learning with Structural Regularization Haoran Li Jingfeng Wu Vladimir Braverman CLL 32 0 0 05 Apr 2025
Global Convergence of Continual Learning on Non-IID Data Fei Zhu Yujing Liu Wenzhuo Liu Zhaoxiang Zhang 40 0 0 24 Mar 2025
Theory on Mixture-of-Experts in Continual Learning Hongbo Li Sen-Fon Lin Lingjie Duan Yingbin Liang Ness B. Shroff MoE MoMe CLL 151 14 0 20 Feb 2025
Continual Memorization of Factoids in Language Models Howard Chen Jiayi Geng Adithya Bhaskar Dan Friedman Danqi Chen KELM 45 1 0 11 Nov 2024
Buffer-based Gradient Projection for Continual Federated Learning Shenghong Dai Jy-yong Sohn Yicong Chen S. Alam Ravikumar Balakrishnan Suman Banerjee N. Himayat Kangwook Lee FedML 66 2 0 03 Sep 2024
Parameter-Efficient Fine-Tuning for Continual Learning: A Neural Tangent Kernel Perspective Jingren Liu Zhong Ji Yunlong Yu Jiale Cao Yanwei Pang Jungong Han Xuelong Li CLL 29 3 0 24 Jul 2024
Local vs Global continual learning Giulia Lanzillotta Sidak Pal Singh Benjamin Grewe Thomas Hofmann CLL 31 0 0 23 Jul 2024
Data-dependent and Oracle Bounds on Forgetting in Continual Learning Lior Friedman Ron Meir 70 0 0 13 Jun 2024
A Statistical Theory of Regularization-Based Continual Learning Xuyang Zhao Huiyuan Wang Weiran Huang Wei Lin 27 12 0 10 Jun 2024
Disentangling and Mitigating the Impact of Task Similarity for Continual Learning Naoki Hiratani CLL 35 2 0 30 May 2024
Understanding Forgetting in Continual Learning with Linear Regression Meng Ding Kaiyi Ji Di Wang Jinhui Xu CLL 24 8 0 27 May 2024
NTK-Guided Few-Shot Class Incremental Learning Jingren Liu Zhong Ji Yanwei Pang Yunlong Yu CLL 32 3 0 19 Mar 2024
Last Iterate Convergence of Incremental Methods and Applications in Continual Learning Xu Cai Jelena Diakonikolas 25 5 0 11 Mar 2024
On the Diminishing Returns of Width for Continual Learning E. Guha V. Lakshman CLL 24 4 0 11 Mar 2024
A streamlined Approach to Multimodal Few-Shot Class Incremental Learning for Fine-Grained Datasets T. Doan Sima Behpour Xin Li Wenbin He Liangke Gou Liu Ren VLM 38 1 0 10 Mar 2024
Distributed Continual Learning with CoCoA in High-dimensional Linear Regression Martin Hellkvist Ayça Özçelikkale Anders Ahlén OOD 28 1 0 04 Dec 2023
Assurance for Deployed Continual Learning Systems Ari Goodman Ryan P. O'Shea Noam Hirschorn Hubert Chrostowski 11 0 0 16 Nov 2023
Towards guarantees for parameter isolation in continual learning Giulia Lanzillotta Sidak Pal Singh Benjamin Grewe Thomas Hofmann 19 0 0 02 Oct 2023
On the Disconnect Between Theory and Practice of Neural Networks: Limits of the NTK Perspective Jonathan Wenger Felix Dangel Agustinus Kristiadi 25 0 0 29 Sep 2023
Learnability and Algorithm for Continual Learning Gyuhak Kim Changnan Xiao Tatsuya Konishi Bin Liu CLL 29 23 0 22 Jun 2023
Continual Learning in Linear Classification on Separable Data Itay Evron E. Moroshko G. Buzaglo M. Khriesh B. Marjieh Nathan Srebro Daniel Soudry CLL 26 14 0 06 Jun 2023
The Ideal Continual Learner: An Agent That Never Forgets Liangzu Peng Paris V. Giampouras René Vidal CLL 106 26 0 29 Apr 2023
Fixed Design Analysis of Regularization-Based Continual Learning Haoran Li Jingfeng Wu Vladimir Braverman CLL 21 9 0 17 Mar 2023
Theory on Forgetting and Generalization of Continual Learning Sen Lin Peizhong Ju Yitao Liang Ness B. Shroff CLL 29 40 0 12 Feb 2023
A Comprehensive Survey of Continual Learning: Theory, Method and Application Liyuan Wang Xingxing Zhang Hang Su Jun Zhu KELM CLL 30 592 0 31 Jan 2023
CyclicFL: A Cyclic Model Pre-Training Approach to Efficient Federated Learning Peng Zhang Yingbo Zhou Ming Hu Xin Fu Xian Wei Mingsong Chen FedML 13 1 0 28 Jan 2023
Restricted Orthogonal Gradient Projection for Continual Learning Zeyuan Yang Zonghan Yang Peng Li Yang Liu CLL 19 4 0 28 Jan 2023
Selective Amnesia: On Efficient, High-Fidelity and Blind Suppression of Backdoor Effects in Trojaned Machine Learning Models Rui Zhu Di Tang Siyuan Tang XiaoFeng Wang Haixu Tang AAML FedML 21 13 0 09 Dec 2022
A Theoretical Study on Solving Continual Learning Gyuhak Kim Changnan Xiao Tatsuya Konishi Zixuan Ke Bin Liu CLL 19 70 0 04 Nov 2022
ATLAS: Universal Function Approximator for Memory Retention H. V. Deventer Anna Sergeevna Bosman 14 0 0 10 Aug 2022
Analysis of Catastrophic Forgetting for Random Orthogonal Transformation Tasks in the Overparameterized Regime Daniel Goldfarb Paul Hand CLL 15 10 0 01 Jun 2022
How catastrophic can catastrophic forgetting be in linear regression? Itay Evron E. Moroshko Rachel A. Ward Nati Srebro Daniel Soudry CLL 22 48 0 19 May 2022
Provable and Efficient Continual Representation Learning Yingcong Li Mingchen Li M. Salman Asif Samet Oymak CLL 19 11 0 03 Mar 2022
Continual Learning Beyond a Single Model T. Doan Seyed Iman Mirzadeh Mehrdad Farajtabar CLL 20 16 0 20 Feb 2022
Provable Continual Learning via Sketched Jacobian Approximations Reinhard Heckel CLL 11 9 0 09 Dec 2021
Learning Curves for Continual Learning in Neural Networks: Self-Knowledge Transfer and Forgetting Ryo Karakida S. Akaho CLL 18 11 0 03 Dec 2021
Defeating Catastrophic Forgetting via Enhanced Orthogonal Weights Modification Yanni Li Bing Liu Kaicheng Yao Xiaoli Kou Pengfan Lv Yueshen Xu J. Cui CLL 13 0 0 19 Nov 2021
Provable Lifelong Learning of Representations Xinyuan Cao Weiyang Liu Santosh Vempala CLL 17 13 0 27 Oct 2021
Formalizing the Generalization-Forgetting Trade-off in Continual Learning Krishnan Raghavan Prasanna Balaprakash CLL 54 29 0 28 Sep 2021
Continual Learning in the Teacher-Student Setup: Impact of Task Similarity Sebastian Lee Sebastian Goldt Andrew M. Saxe CLL 19 73 0 09 Jul 2021
TAG: Task-based Accumulated Gradients for Lifelong learning Pranshu Malviya B. Ravindran Sarath Chandar CLL 35 5 0 11 May 2021
Gradient Projection Memory for Continual Learning Gobinda Saha Isha Garg Kaushik Roy VLM CLL 34 266 0 17 Mar 2021
How do Quadratic Regularizers Prevent Catastrophic Forgetting: The Role of Interpolation Ekdeep Singh Lubana Puja Trivedi Danai Koutra Robert P. Dick 23 11 0 04 Feb 2021
A Theoretical Analysis of Catastrophic Forgetting through the NTK Overlap Matrix T. Doan Mehdi Abbana Bennani Bogdan Mazoure Guillaume Rabusseau Pierre Alquier CLL 15 80 0 07 Oct 2020
Understanding the Role of Training Regimes in Continual Learning Seyed Iman Mirzadeh Mehrdad Farajtabar Razvan Pascanu H. Ghasemzadeh CLL 13 219 0 12 Jun 2020