Orthogonal Gradient Descent for Continual Learning

15 October 2019

Papers citing "Orthogonal Gradient Descent for Continual Learning"

50 / 74 papers shown

Title
Streaming Krylov-Accelerated Stochastic Gradient Descent Stephen Thomas 21 0 0 11 May 2025
Improvements of Dark Experience Replay and Reservoir Sampling towards Better Balance between Consolidation and Plasticity Taisuke Kobayashi CLL 41 0 0 29 Apr 2025
Mathematics of Continual Learning Liangzu Peng René Vidal CLL 70 0 0 24 Apr 2025
Task-conditioned Ensemble of Expert Models for Continuous Learning Renu Sharma Debasmita Pal Arun Ross OOD KELM 151 0 0 11 Apr 2025
TiC-LM: A Web-Scale Benchmark for Time-Continual LLM Pretraining Jeffrey Li Mohammadreza Armandpour Iman Mirzadeh Sachin Mehta Vaishaal Shankar ... Samy Bengio Oncel Tuzel Mehrdad Farajtabar Hadi Pouransari Fartash Faghri CLL KELM 61 0 0 02 Apr 2025
Continual Learning With Quasi-Newton Methods Steven Vander Eeckt Hugo Van hamme CLL BDL 69 0 0 25 Mar 2025
Orthogonal Calibration for Asynchronous Federated Learning Jiayun Zhang Shuheng Li Haiyu Huang Xiaofan Yu Rajesh K. Gupta Jingbo Shang FedML 65 0 0 21 Feb 2025
Theory on Mixture-of-Experts in Continual Learning Hongbo Li Sen-Fon Lin Lingjie Duan Yingbin Liang Ness B. Shroff MoE MoMe CLL 153 14 0 20 Feb 2025
Achieving Upper Bound Accuracy of Joint Training in Continual Learning Saleh Momeni Bing Liu CLL 84 1 0 17 Feb 2025
Analysis of Overparameterization in Continual Learning under a Linear Model Daniel Goldfarb Paul Hand CLL 39 0 0 11 Feb 2025
Sequence Transferability and Task Order Selection in Continual Learning T. Nguyen Cuong N. Nguyen Quang-Cuong Pham B. T. Nguyen Savitha Ramasamy Xiaoli Li Cuong V Nguyen 80 0 0 10 Feb 2025
CENSOR: Defense Against Gradient Inversion via Orthogonal Subspace Bayesian Sampling Kaiyuan Zhang Siyuan Cheng Guangyu Shen Bruno Ribeiro Shengwei An Pin-Yu Chen Xinming Zhang Ninghui Li 100 1 0 28 Jan 2025
Multi-Task Model Merging via Adaptive Weight Disentanglement Feng Xiong Runxi Cheng Wang Chen Zhanqiu Zhang Yiwen Guo Chun Yuan Ruifeng Xu MoMe 99 4 0 10 Jan 2025
Rehearsal-Free Continual Federated Learning with Synergistic Synaptic Intelligence Yichen Li Yuying Wang Haozhao Wang Yining Qi Tianzhe Xiao Ruixuan Li 121 0 0 18 Dec 2024
Gradient Localization Improves Lifelong Pretraining of Language Models Jared Fernandez Yonatan Bisk Emma Strubell KELM 36 1 0 07 Nov 2024
Dual Low-Rank Adaptation for Continual Learning with Pre-Trained Models Huancheng Chen Jingtao Li Nidham Gazagnadou Weiming Zhuang Chen Chen Lingjuan Lyu VLM CLL 47 1 0 01 Nov 2024
Controlled Low-Rank Adaptation with Subspace Regularization for Continued Training on Large Language Models Yuheng Lu Bingshuo Qian Caixia Yuan Huixing Jiang Xiaojie Wang CLL 34 0 0 22 Oct 2024
AlphaEdit: Null-Space Constrained Knowledge Editing for Language Models Fan Zhang Houcheng Jiang Kun Wang Yunshan Ma Shi Jie Xiangnan He Tat-Seng Chua Tat-seng Chua KELM 39 33 0 03 Oct 2024
LoRanPAC: Low-rank Random Features and Pre-trained Models for Bridging Theory and Practice in Continual Learning Liangzu Peng Juan Elenter Joshua Agterberg Alejandro Ribeiro René Vidal VLM CLL 46 1 0 01 Oct 2024
Buffer-based Gradient Projection for Continual Federated Learning Shenghong Dai Jy-yong Sohn Yicong Chen S. Alam Ravikumar Balakrishnan Suman Banerjee N. Himayat Kangwook Lee FedML 75 2 0 03 Sep 2024
An Effective Dynamic Gradient Calibration Method for Continual Learning Weichen Lin Jiaxiang Chen Ru Huang Huihua Ding CLL 44 0 0 30 Jul 2024
MoFO: Momentum-Filtered Optimizer for Mitigating Forgetting in LLM Fine-Tuning Yupeng Chen Senmiao Wang Zhihang Lin Zhihang Lin Yushun Zhang Tian Ding Ruoyu Sun Ruoyu Sun CLL 77 1 0 30 Jul 2024
Machine Unlearning via Null Space Calibration Huiqiang Chen Tianqing Zhu Xin Yu Wanlei Zhou 41 6 0 21 Apr 2024
Continual Learning on Graphs: A Survey Zonggui Tian Duanhao Zhang Hong-Ning Dai 47 5 0 09 Feb 2024
Benchmarking Continual Learning from Cognitive Perspectives Xiaoqian Liu Junge Zhang Mingyi Zhang Peipei Yang 21 0 0 06 Dec 2023
Two Complementary Perspectives to Continual Learning: Ask Not Only What to Optimize, But Also How Timm Hess Tinne Tuytelaars Gido M. van de Ven 38 7 0 08 Nov 2023
Elephant Neural Networks: Born to Be a Continual Learner Qingfeng Lan A. R. Mahmood CLL 42 9 0 02 Oct 2023
HPCR: Holistic Proxy-based Contrastive Replay for Online Continual Learning Huiwei Lin Shanshan Feng Baoquan Zhang Xutao Li Y. Ong CLL 37 0 0 26 Sep 2023
Clustering-based Domain-Incremental Learning Christiaan Lamers René Vidal A. Belbachir N. V. Stein Thomas Baeck Paris Giampouras CLL OOD 29 10 0 21 Sep 2023
UER: A Heuristic Bias Addressing Approach for Online Continual Learning Huiwei Lin Shanshan Feng Baoquan Zhang Hongliang Qiao Xutao Li Yunming Ye CLL 32 4 0 08 Sep 2023
Disentanglement Learning via Topology Nikita Balabin Daria Voronkova I. Trofimov Evgeny Burnaev S. Barannikov DRL 58 2 0 24 Aug 2023
Continual Pre-Training of Large Language Models: How to (re)warm your model? Kshitij Gupta Benjamin Thérien Adam Ibrahim Mats L. Richter Quentin G. Anthony Eugene Belilovsky Irina Rish Timothée Lesort KELM 24 99 0 08 Aug 2023
Can Continual Learning Improve Long-Tailed Recognition? Toward a Unified Framework Mahdiyar Molahasani Michael A. Greenspan Ali Etemad 24 2 0 23 Jun 2023
SketchOGD: Memory-Efficient Continual Learning Benjamin Wright Youngjae Min Jeremy Bernstein Navid Azizan CLL 23 0 0 25 May 2023
CoDeC: Communication-Efficient Decentralized Continual Learning Sakshi Choudhary Sai Aparna Aketi Gobinda Saha Kaushik Roy CLL 45 3 0 27 Mar 2023
Fixed Design Analysis of Regularization-Based Continual Learning Haoran Li Jingfeng Wu Vladimir Braverman CLL 26 9 0 17 Mar 2023
Subspace based Federated Unlearning Guang-Ming Li Li Shen Yan Sun Yuejun Hu Han Hu Dacheng Tao MU FedML 28 20 0 24 Feb 2023
Online Reinforcement Learning in Non-Stationary Context-Driven Environments Pouya Hamadanian Arash Nasr-Esfahany Malte Schwarzkopf Siddartha Sen MohammadIman Alizadeh CLL OffRL 50 0 0 04 Feb 2023
Continual Graph Learning: A Survey Qiao Yuan S. Guan Pin Ni Tianlun Luo Ka Lok Man Prudence W. H. Wong Victor I. Chang CLL 24 14 0 28 Jan 2023
Restricted Orthogonal Gradient Projection for Continual Learning Zeyuan Yang Zonghan Yang Peng Li Yang Liu CLL 24 4 0 28 Jan 2023
Progressive Learning without Forgetting Tao Feng Hangjie Yuan Mang Wang Ziyuan Huang Ang Bian Jianzhou Zhang CLL KELM 44 4 0 28 Nov 2022
Domain-incremental Cardiac Image Segmentation with Style-oriented Replay and Domain-sensitive Feature Whitening Kang Li Lequan Yu Pheng-Ann Heng CLL 24 22 0 09 Nov 2022
Is Multi-Task Learning an Upper Bound for Continual Learning? Zihao Wu Huy Tran Hamed Pirsiavash Soheil Kolouri SSL CLL 26 4 0 26 Oct 2022
Continual Learning by Modeling Intra-Class Variation L. Yu Tianyang Hu Lanqing Hong Zhen Liu Adrian Weller Weiyang Liu CLL 48 12 0 11 Oct 2022
Uncertainty-Aware Meta-Learning for Multimodal Task Distributions Cesar Almecija Apoorva Sharma Navid Azizan OOD UQCV 24 3 0 04 Oct 2022
ATLAS: Universal Function Approximator for Memory Retention H. V. Deventer Anna Sergeevna Bosman 22 0 0 10 Aug 2022
S-Prompts Learning with Pre-trained Transformers: An Occam's Razor for Domain Incremental Learning Yabin Wang Zhiwu Huang Xiaopeng Hong CLL VLM 27 208 0 26 Jul 2022
Incremental Task Learning with Incremental Rank Updates Rakib Hyder Ken Shao Boyu Hou P. Markopoulos Ashley Prater-Bennette M. Salman Asif CLL 16 13 0 19 Jul 2022
Debugging using Orthogonal Gradient Descent Narsimha Chilkuri C. Eliasmith 21 1 0 17 Jun 2022
Switchable Representation Learning Framework with Self-compatibility Shengsen Wu Yan Bai Yihang Lou Xiongkun Linghu Jianzhong He Ling-yu Duan 22 1 0 16 Jun 2022