Progressive Prompts: Continual Learning for Language Models

29 January 2023

Anastasia Razdaibiedina

Madian Khabsa

Papers citing "Progressive Prompts: Continual Learning for Language Models"

28 / 28 papers shown

Title
GDI-Bench: A Benchmark for General Document Intelligence with Vision and Reasoning Decoupling Siqi Li Yufan Shen Xiangnan Chen Jiayi Chen Hengwei Ju ... Licheng Wen Botian Shi Y. Liu Xinyu Cai Yu Qiao VLM ELM 89 0 0 30 Apr 2025
POET: Prompt Offset Tuning for Continual Human Action Adaptation Prachi Garg Joseph K J V. Balasubramanian Necati Cihan Camgöz Chengde Wan Kenrick Kin Weiguang Si Shugao Ma Fernando De la Torre 66 0 0 25 Apr 2025
Distribution-aware Forgetting Compensation for Exemplar-Free Lifelong Person Re-identification Shiben Liu Huijie Fan Qiang Wang Baojie Fan Yandong Tang Liangqiong Qu OOD CLL 57 0 0 21 Apr 2025
TiC-LM: A Web-Scale Benchmark for Time-Continual LLM Pretraining Jeffrey Li Mohammadreza Armandpour Iman Mirzadeh Sachin Mehta Vaishaal Shankar ... Samy Bengio Oncel Tuzel Mehrdad Farajtabar Hadi Pouransari Fartash Faghri CLL KELM 59 0 0 02 Apr 2025
Privacy Ripple Effects from Adding or Removing Personal Information in Language Model Training Jaydeep Borkar Matthew Jagielski Katherine Lee Niloofar Mireshghallah David A. Smith Christopher A. Choquette-Choo PILM 80 1 0 24 Feb 2025
Achieving Upper Bound Accuracy of Joint Training in Continual Learning Saleh Momeni Bing Liu CLL 84 1 0 17 Feb 2025
Unlocking the Power of Function Vectors for Characterizing and Mitigating Catastrophic Forgetting in Continual Instruction Tuning Gangwei Jiang Caigao Jiang Zhaoyi Li Siqiao Xue Jun-ping Zhou Linqi Song Defu Lian Yin Wei CLL MU 60 0 0 16 Feb 2025
Empowering Large Language Model for Continual Video Question Answering with Collaborative Prompting Chen Cai Zheng Wang J. Gao Wenyang Liu Ye Lu Runzhong Zhang Kim-Hui Yap CLL 52 0 0 20 Jan 2025
LSEBMCL: A Latent Space Energy-Based Model for Continual Learning Xiaodi Li Dingcheng Li Rujun Gao Mahmoud Zamani Latifur Khan CLL KELM 44 0 0 09 Jan 2025
Controlled Low-Rank Adaptation with Subspace Regularization for Continued Training on Large Language Models Yuheng Lu Bingshuo Qian Caixia Yuan Huixing Jiang Xiaojie Wang CLL 34 0 0 22 Oct 2024
Self-Data Distillation for Recovering Quality in Pruned Large Language Models Vithursan Thangarasa Ganesh Venkatesh Mike Lasby Nish Sinnadurai Sean Lie SyDa 38 1 0 13 Oct 2024
MoFO: Momentum-Filtered Optimizer for Mitigating Forgetting in LLM Fine-Tuning Yupeng Chen Senmiao Wang Zhihang Lin Zhihang Lin Yushun Zhang Tian Ding Ruoyu Sun Ruoyu Sun CLL 72 1 0 30 Jul 2024
Towards Understanding Multi-Task Learning (Generalization) of LLMs via Detecting and Exploring Task-Specific Neurons Yongqi Leng Deyi Xiong 39 5 0 09 Jul 2024
Few-shot Personalization of LLMs with Mis-aligned Responses Jaehyung Kim Yiming Yang 47 7 0 26 Jun 2024
HFT: Half Fine-Tuning for Large Language Models Tingfeng Hui Zhenyu Zhang Shuohuan Wang Weiran Xu Yu Sun Hua-Hong Wu CLL 37 4 0 29 Apr 2024
From Matching to Generation: A Survey on Generative Information Retrieval Xiaoxi Li Jiajie Jin Yujia Zhou Yuyao Zhang Peitian Zhang Yutao Zhu Zhicheng Dou 3DV 75 46 0 23 Apr 2024
Continual Segmentation with Disentangled Objectness Learning and Class Recognition Yizheng Gong Siyue Yu Xiaoyang Wang Jimin Xiao CLL 30 5 0 06 Mar 2024
Mitigating Catastrophic Forgetting in Large Language Models with Self-Synthesized Rehearsal Jianheng Huang Leyang Cui Ante Wang Chengyi Yang Xinting Liao Linfeng Song Junfeng Yao Jinsong Su KELM CLL 32 32 0 02 Mar 2024
Investigating Continual Pretraining in Large Language Models: Insights and Implications cCaugatay Yildiz Nishaanth Kanna Ravichandran Prishruit Punia Matthias Bethge B. Ermiş CLL KELM LRM 50 25 0 27 Feb 2024
Can LLMs Learn New Concepts Incrementally without Forgetting? Junhao Zheng Shengjie Qiu Qianli Ma CLL 35 0 0 13 Feb 2024
Learn or Recall? Revisiting Incremental Learning with Pre-trained Language Models Junhao Zheng Shengjie Qiu Qianli Ma 25 9 0 13 Dec 2023
Residual Prompt Tuning: Improving Prompt Tuning with Residual Reparameterization Anastasia Razdaibiedina Yuning Mao Rui Hou Madian Khabsa M. Lewis Jimmy Ba Amjad Almahairi VLM 19 42 0 06 May 2023
How to Reuse and Compose Knowledge for a Lifetime of Tasks: A Survey on Continual Learning and Functional Composition Jorge Armando Mendez Mendez Eric Eaton KELM CLL 29 27 0 15 Jul 2022
SPoT: Better Frozen Model Adaptation through Soft Prompt Transfer Tu Vu Brian Lester Noah Constant Rami Al-Rfou Daniel Matthew Cer VLM LRM 137 277 0 15 Oct 2021
P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Tasks Xiao Liu Kaixuan Ji Yicheng Fu Weng Lam Tam Zhengxiao Du Zhilin Yang Jie Tang VLM 238 805 0 14 Oct 2021
LFPT5: A Unified Framework for Lifelong Few-shot Language Learning Based on Prompt Tuning of T5 Chengwei Qin Shafiq R. Joty CLL 176 98 0 14 Oct 2021
The Power of Scale for Parameter-Efficient Prompt Tuning Brian Lester Rami Al-Rfou Noah Constant VPVLM 280 3,848 0 18 Apr 2021
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 297 6,956 0 20 Apr 2018