Mitigating Catastrophic Forgetting in Large Language Models with Self-Synthesized Rehearsal

2 March 2024

Leyang Cui

Linfeng Song

Papers citing "Mitigating Catastrophic Forgetting in Large Language Models with Self-Synthesized Rehearsal"

27 / 27 papers shown

Title
Reinforced Interactive Continual Learning via Real-time Noisy Human Feedback Yutao Yang Jie Zhou Junsong Li Qianjun Pan Bihao Zhan Qin Chen Xipeng Qiu Liang He CLL 17 0 0 15 May 2025
Memorization and Knowledge Injection in Gated LLMs Xu Pan Ely Hahami Zechen Zhang H. Sompolinsky KELM CLL RALM 104 0 0 30 Apr 2025
Replay to Remember: Retaining Domain Knowledge in Streaming Language Models Sneh Pillai KELM CLL 61 0 0 24 Apr 2025
Mitigating Memorization in LLMs using Activation Steering Manan Suri Nishit Anand Amisha Bhaskar LLMSV 50 2 0 08 Mar 2025
R1-T1: Fully Incentivizing Translation Capability in LLMs via Reasoning Learning Minggui He Yilun Liu Shimin Tao Yuanchang Luo Hongyong Zeng ... Daimeng Wei Weibin Meng Hao Yang Boxing Chen Osamu Yoshie LRM 61 2 0 27 Feb 2025
Recurrent Knowledge Identification and Fusion for Language Model Continual Learning Yujie Feng Xujia Wang Zexin Lu Shenghong Fu Guangyuan Shi Yongxin Xu Yasha Wang Philip S. Yu Xu Chu Xiao-Ming Wu CLL KELM 41 1 0 22 Feb 2025
GSCE: A Prompt Framework with Enhanced Reasoning for Reliable LLM-driven Drone Control Wenhao Wang Yanyan Li Long Jiao Jiawei Yuan 78 1 0 18 Feb 2025
Unlocking the Power of Function Vectors for Characterizing and Mitigating Catastrophic Forgetting in Continual Instruction Tuning Gangwei Jiang Caigao Jiang Zhaoyi Li Siqiao Xue Jun-ping Zhou Linqi Song Defu Lian Yin Wei CLL MU 56 0 0 16 Feb 2025
Minimum Tuning to Unlock Long Output from LLMs with High Quality Data as the Key Yingda Chen Xingjun Wang Jintao Huang Yunlin Mao Daoze Zhang Yuze Zhao ALM 21 0 0 14 Oct 2024
A Learning Rate Path Switching Training Paradigm for Version Updates of Large Language Models Zhihao Wang Shiyu Liu Jianheng Huang Zheng Wang Yixuan Liao Xiaoxin Chen Junfeng Yao Jinsong Su 24 1 0 05 Oct 2024
How Much Can We Forget about Data Contamination? Sebastian Bordt Suraj Srinivas Valentyn Boreiko U. V. Luxburg 43 1 0 04 Oct 2024
MoFO: Momentum-Filtered Optimizer for Mitigating Forgetting in LLM Fine-Tuning Yupeng Chen Senmiao Wang Zhihang Lin Zhihang Lin Yushun Zhang Tian Ding Ruoyu Sun Ruoyu Sun CLL 72 1 0 30 Jul 2024
The Emerged Security and Privacy of LLM Agent: A Survey with Case Studies Feng He Tianqing Zhu Dayong Ye Bo Liu Wanlei Zhou Philip S. Yu PILM LLMAG ELM 68 23 0 28 Jul 2024
Mitigating Catastrophic Forgetting in Language Transfer via Model Merging Anton Alexandrov Veselin Raychev Mark Niklas Muller Ce Zhang Martin Vechev Kristina Toutanova MoMe CLL KELM 40 13 0 11 Jul 2024
Self-MoE: Towards Compositional Large Language Models with Self-Specialized Experts Junmo Kang Leonid Karlinsky Hongyin Luo Zhen Wang Jacob A. Hansen James Glass David D. Cox Rameswar Panda Rogerio Feris Alan Ritter MoMe MoE 34 8 0 17 Jun 2024
Towards Lifelong Learning of Large Language Models: A Survey Junhao Zheng Shengjie Qiu Chengming Shi Qianli Ma KELM CLL 28 14 0 10 Jun 2024
HPE-CogVLM: New Head Pose Grounding Task Exploration on Vision Language Model Yu Tian Tianqi Shao Tsukasa Demizu Xuyang Wu Hsin-Tai Wu 24 0 0 04 Jun 2024
Towards Incremental Learning in Large Language Models: A Critical Review M. Jovanovic Peter Voss ELM CLL KELM 26 5 0 28 Apr 2024
Continual Learning of Large Language Models: A Comprehensive Survey Haizhou Shi Zihao Xu Hengyi Wang Weiyi Qin Wenyuan Wang Yibin Wang Zifeng Wang Sayna Ebrahimi Hao Wang CLL KELM LRM 39 63 0 25 Apr 2024
A Survey on Self-Evolution of Large Language Models Zhengwei Tao Ting-En Lin Xiancai Chen Hangyu Li Yuchuan Wu Yongbin Li Zhi Jin Fei Huang Dacheng Tao Jingren Zhou LRM LM&Ro 51 22 0 22 Apr 2024
MM-Soc: Benchmarking Multimodal Large Language Models in Social Media Platforms Yiqiao Jin Minje Choi Gaurav Verma Jindong Wang Srijan Kumar 49 19 0 21 Feb 2024
Can LLMs Learn New Concepts Incrementally without Forgetting? Junhao Zheng Shengjie Qiu Qianli Ma CLL 27 0 0 13 Feb 2024
Security and Privacy Challenges of Large Language Models: A Survey B. Das M. H. Amini Yanzhao Wu PILM ELM 19 102 0 30 Jan 2024
Response Enhanced Semi-supervised Dialogue Query Generation Jianheng Huang Ante Wang Linfeng Gao Linfeng Song Jinsong Su RALM 22 5 0 20 Dec 2023
CITB: A Benchmark for Continual Instruction Tuning Zihan Zhang Meng Fang Ling-Hao Chen Mohammad-Reza Namazi-Rad ALM CLL 39 23 0 23 Oct 2023
Two-stage LLM Fine-tuning with Less Specialization and More Generalization Yihan Wang Si Si Daliang Li Michal Lukasik Felix X. Yu Cho-Jui Hsieh Inderjit S Dhillon Sanjiv Kumar 34 29 0 01 Nov 2022
Fine-tuned Language Models are Continual Learners Thomas Scialom Tuhin Chakrabarty Smaranda Muresan CLL LRM 139 116 0 24 May 2022