Overcoming Catastrophic Forgetting in Massively Multilingual Continual
Learning

Overcoming Catastrophic Forgetting in Massively Multilingual Continual Learning

25 May 2023

Genta Indra Winata

Karthik Radhakrishnan

Pengxiang Cheng

Mayank Kulkarni

Daniel Preotiuc-Pietro

Papers citing "Overcoming Catastrophic Forgetting in Massively Multilingual Continual Learning"

16 / 16 papers shown

Title
Elastic Weight Consolidation for Full-Parameter Continual Pre-Training of Gemma2 Vytenis Šliogeris Povilas Daniušis Arturas Nakvosas CLL 35 0 0 09 May 2025
What Causes Knowledge Loss in Multilingual Language Models? Maria Khelli Samuel Cahyawijaya Ayu Purwarianti Genta Indra Winata CLL 56 0 0 29 Apr 2025
High-Dimensional Interlingual Representations of Large Language Models Bryan Wilie Samuel Cahyawijaya Junxian He Pascale Fung 57 0 0 14 Mar 2025
Beyond Cosine Decay: On the effectiveness of Infinite Learning Rate Schedule for Continual Pre-training Paul Janson Vaibhav Singh Paria Mehrbod Adam Ibrahim Irina Rish Eugene Belilovsky Benjamin Thérien CLL 78 0 0 04 Mar 2025
Towards Rehearsal-Free Multilingual ASR: A LoRA-based Case Study on Whisper Tianyi Xu Kaixun Huang Pengcheng Guo Yu Zhou Longtao Huang Hui Xue Lei Xie CLL 35 0 0 20 Aug 2024
SLCA++: Unleash the Power of Sequential Fine-tuning for Continual Learning with Pre-training Gengwei Zhang Liyuan Wang Guoliang Kang Ling Chen Yunchao Wei VLM CLL 37 2 0 15 Aug 2024
The Emerged Security and Privacy of LLM Agent: A Survey with Case Studies Feng He Tianqing Zhu Dayong Ye Bo Liu Wanlei Zhou Philip S. Yu PILM LLMAG ELM 68 23 0 28 Jul 2024
Reuse, Don't Retrain: A Recipe for Continued Pretraining of Language Models Jupinder Parmar Sanjev Satheesh M. Patwary M. Shoeybi Bryan Catanzaro 48 27 0 09 Jul 2024
Breaking Language Barriers: Cross-Lingual Continual Pre-Training at Scale Wenzhen Zheng Wenbo Pan Xu Xu Libo Qin Li Yue Ming Zhou CLL 34 6 0 02 Jul 2024
Towards Lifelong Learning of Large Language Models: A Survey Junhao Zheng Shengjie Qiu Chengming Shi Qianli Ma KELM CLL 28 14 0 10 Jun 2024
Continual Learning of Large Language Models: A Comprehensive Survey Haizhou Shi Zihao Xu Hengyi Wang Weiyi Qin Wenyuan Wang Yibin Wang Zifeng Wang Sayna Ebrahimi Hao Wang CLL KELM LRM 41 63 0 25 Apr 2024
Security and Privacy Challenges of Large Language Models: A Survey B. Das M. H. Amini Yanzhao Wu PILM ELM 19 103 0 30 Jan 2024
Breaking the Curse of Multilinguality with Cross-lingual Expert Language Models Terra Blevins Tomasz Limisiewicz Suchin Gururangan Margaret Li Hila Gonen Noah A. Smith Luke Zettlemoyer 44 22 0 19 Jan 2024
Continual Learning Under Language Shift Evangelia Gogoulou Timothée Lesort Magnus Boman Joakim Nivre KELM CLL 27 3 0 02 Nov 2023
Continual Pre-Training of Large Language Models: How to (re)warm your model? Kshitij Gupta Benjamin Thérien Adam Ibrahim Mats L. Richter Quentin G. Anthony Eugene Belilovsky Irina Rish Timothée Lesort KELM 24 99 0 08 Aug 2023
Cross-lingual Lifelong Learning Meryem M'hamdi Xiang Ren Jonathan May CLL 35 8 0 23 May 2022