InsCL: A Data-efficient Continual Learning Paradigm for Fine-tuning
Large Language Models with Instructions

InsCL: A Data-efficient Continual Learning Paradigm for Fine-tuning Large Language Models with Instructions

18 March 2024

Yujiu Yang

Papers citing "InsCL: A Data-efficient Continual Learning Paradigm for Fine-tuning Large Language Models with Instructions"

15 / 15 papers shown

Title
Unlocking the Power of Function Vectors for Characterizing and Mitigating Catastrophic Forgetting in Continual Instruction Tuning Gangwei Jiang Caigao Jiang Zhaoyi Li Siqiao Xue Jun-ping Zhou Linqi Song Defu Lian Yin Wei CLL MU 56 0 0 16 Feb 2025
Boosting LLM Translation Skills without General Ability Loss via Rationale Distillation Junhong Wu Yang Zhao Yangyifan Xu Bing Liu Chengqing Zong CLL 33 1 0 17 Oct 2024
Recent Advances of Multimodal Continual Learning: A Comprehensive Survey Dianzhi Yu Xinni Zhang Yankai Chen Aiwei Liu Yifei Zhang Philip S. Yu Irwin King VLM CLL 39 9 0 07 Oct 2024
LoRTA: Low Rank Tensor Adaptation of Large Language Models Ignacio Hounie Charilaos I. Kanatsoulis Arnuv Tandon Alejandro Ribeiro 29 0 0 05 Oct 2024
Towards LifeSpan Cognitive Systems Yu Wang Chi Han Tongtong Wu Xiaoxin He Wangchunshu Zhou ... Zexue He Wei Wang Gholamreza Haffari Heng Ji Julian McAuley KELM CLL 100 1 0 20 Sep 2024
MoFO: Momentum-Filtered Optimizer for Mitigating Forgetting in LLM Fine-Tuning Yupeng Chen Senmiao Wang Zhihang Lin Zhihang Lin Yushun Zhang Tian Ding Ruoyu Sun Ruoyu Sun CLL 72 1 0 30 Jul 2024
Orthogonal Subspace Learning for Language Model Continual Learning Xiao Wang Tianze Chen Qiming Ge Han Xia Rong Bao Rui Zheng Qi Zhang Tao Gui Xuanjing Huang CLL 112 88 0 22 Oct 2023
Fine-tuned Language Models are Continual Learners Thomas Scialom Tuhin Chakrabarty Smaranda Muresan CLL LRM 134 116 0 24 May 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 303 11,881 0 04 Mar 2022
Achieving Forgetting Prevention and Knowledge Transfer in Continual Learning Zixuan Ke Bing-Quan Liu Nianzu Ma Hu Xu Lei Shu CLL 181 122 0 05 Dec 2021
Multitask Prompted Training Enables Zero-Shot Task Generalization Victor Sanh Albert Webson Colin Raffel Stephen H. Bach Lintang Sutawika ... T. Bers Stella Biderman Leo Gao Thomas Wolf Alexander M. Rush LRM 211 1,654 0 15 Oct 2021
LFPT5: A Unified Framework for Lifelong Few-shot Language Learning Based on Prompt Tuning of T5 Chengwei Qin Shafiq R. Joty CLL 170 97 0 14 Oct 2021
Phrase-BERT: Improved Phrase Embeddings from BERT with an Application to Corpus Exploration Shufan Wang Laure Thompson Mohit Iyyer 169 66 0 13 Sep 2021
On the Transformer Growth for Progressive BERT Training Xiaotao Gu Liyuan Liu Hongkun Yu Jing Li C. L. P. Chen Jiawei Han VLM 61 51 0 23 Oct 2020
Geometric Dataset Distances via Optimal Transport David Alvarez-Melis Nicolò Fusi OT 70 192 0 07 Feb 2020