Understanding Catastrophic Forgetting in Language Models via Implicit Inference

18 September 2023

Suhas Kotha

Jacob Mitchell Springer

Aditi Raghunathan

CLL

ArXiv PDF HTML

Papers citing "Understanding Catastrophic Forgetting in Language Models via Implicit Inference"

20 / 20 papers shown

Title
Crosslingual Reasoning through Test-Time Scaling Zheng-Xin Yong Muhammad Farid Adilazuarda Jonibek Mansurov Ruochen Zhang Niklas Muennighoff Carsten Eickhoff Genta Indra Winata Julia Kreutzer Stephen H. Bach Alham Fikri Aji LRM ELM 60 0 0 08 May 2025
HSplitLoRA: A Heterogeneous Split Parameter-Efficient Fine-Tuning Framework for Large Language Models Zheng Lin Yuxin Zhang Zhe Chen Zihan Fang Xianhao Chen Praneeth Vepakomma Wei Ni Jun-Jie Luo Yue Gao MoE 32 1 0 05 May 2025
Memorization and Knowledge Injection in Gated LLMs Xu Pan Ely Hahami Zechen Zhang H. Sompolinsky KELM CLL RALM 104 0 0 30 Apr 2025
Unlocking the Power of Function Vectors for Characterizing and Mitigating Catastrophic Forgetting in Continual Instruction Tuning Gangwei Jiang Caigao Jiang Zhaoyi Li Siqiao Xue Jun-ping Zhou Linqi Song Defu Lian Yin Wei CLL MU 53 0 0 16 Feb 2025
Evaluating Concurrent Robustness of Language Models Across Diverse Challenge Sets Vatsal Gupta Pranshu Pandya Tushar Kataria Vivek Gupta Dan Roth AAML 51 1 0 03 Jan 2025
Self-Data Distillation for Recovering Quality in Pruned Large Language Models Vithursan Thangarasa Ganesh Venkatesh Mike Lasby Nish Sinnadurai Sean Lie SyDa 33 0 0 13 Oct 2024
Endless Jailbreaks with Bijection Learning Brian R. Y. Huang Maximilian Li Leonard Tang AAML 70 5 0 02 Oct 2024
MoFO: Momentum-Filtered Optimizer for Mitigating Forgetting in LLM Fine-Tuning Yupeng Chen Senmiao Wang Zhihang Lin Zhihang Lin Yushun Zhang Tian Ding Ruoyu Sun Ruoyu Sun CLL 72 1 0 30 Jul 2024
Evaluating the Factuality of Large Language Models using Large-Scale Knowledge Graphs Xiaoze Liu Feijie Wu Tianyang Xu Zhuo Chen Yichi Zhang Xiaoqian Wang Jing Gao HILM 33 8 0 01 Apr 2024
Language Models are Multilingual Chain-of-Thought Reasoners Freda Shi Mirac Suzgun Markus Freitag Xuezhi Wang Suraj Srivats ... Yi Tay Sebastian Ruder Denny Zhou Dipanjan Das Jason W. Wei ReLM LRM 165 320 0 06 Oct 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 301 11,730 0 04 Mar 2022
Multitask Prompted Training Enables Zero-Shot Task Generalization Victor Sanh Albert Webson Colin Raffel Stephen H. Bach Lintang Sutawika ... T. Bers Stella Biderman Leo Gao Thomas Wolf Alexander M. Rush LRM 203 1,651 0 15 Oct 2021
Gradient-based Adversarial Attacks against Text Transformers Chuan Guo Alexandre Sablayrolles Hervé Jégou Douwe Kiela SILM 98 225 0 15 Apr 2021
The Pile: An 800GB Dataset of Diverse Text for Language Modeling Leo Gao Stella Biderman Sid Black Laurence Golding Travis Hoppe ... Horace He Anish Thite Noa Nabeshima Shawn Presser Connor Leahy AIMat 245 1,977 0 31 Dec 2020
Making Pre-trained Language Models Better Few-shot Learners Tianyu Gao Adam Fisch Danqi Chen 241 1,898 0 31 Dec 2020
Meta Learning Backpropagation And Improving It Louis Kirsch Jürgen Schmidhuber 40 56 0 29 Dec 2020
Fine-Tuning Language Models from Human Preferences Daniel M. Ziegler Nisan Stiennon Jeff Wu Tom B. Brown Alec Radford Dario Amodei Paul Christiano G. Irving ALM 275 1,561 0 18 Sep 2019
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 294 6,927 0 20 Apr 2018
Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks Chelsea Finn Pieter Abbeel Sergey Levine OOD 243 11,568 0 09 Mar 2017
Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation Yonghui Wu M. Schuster Z. Chen Quoc V. Le Mohammad Norouzi ... Alex Rudnick Oriol Vinyals G. Corrado Macduff Hughes J. Dean AIMat 716 6,724 0 26 Sep 2016