SAPT: A Shared Attention Framework for Parameter-Efficient Continual
Learning of Large Language Models

SAPT: A Shared Attention Framework for Parameter-Efficient Continual Learning of Large Language Models

16 January 2024

Wanxiang Che

Papers citing "SAPT: A Shared Attention Framework for Parameter-Efficient Continual Learning of Large Language Models"

10 / 10 papers shown

Title
SEFE: Superficial and Essential Forgetting Eliminator for Multimodal Continual Instruction Tuning Jinpeng Chen Runmin Cong Yuzhi Zhao Hongzheng Yang Guangneng Hu H. Ip Sam Kwong CLL KELM 61 0 0 05 May 2025
Unlocking the Power of Function Vectors for Characterizing and Mitigating Catastrophic Forgetting in Continual Instruction Tuning Gangwei Jiang Caigao Jiang Zhaoyi Li Siqiao Xue Jun-ping Zhou Linqi Song Defu Lian Yin Wei CLL MU 56 0 0 16 Feb 2025
Orthogonal Subspace Learning for Language Model Continual Learning Xiao Wang Tianze Chen Qiming Ge Han Xia Rong Bao Rui Zheng Qi Zhang Tao Gui Xuanjing Huang CLL 112 85 0 22 Oct 2023
Lifelong Sequence Generation with Dynamic Module Expansion and Adaptation Chengwei Qin Junwon Lee Shafiq R. Joty CLL 43 12 0 15 Oct 2023
ATTEMPT: Parameter-Efficient Multi-task Tuning via Attentional Mixtures of Soft Prompts Akari Asai Mohammadreza Salehi Matthew E. Peters Hannaneh Hajishirzi 120 98 0 24 May 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 303 11,730 0 04 Mar 2022
LFPT5: A Unified Framework for Lifelong Few-shot Language Learning Based on Prompt Tuning of T5 Chengwei Qin Shafiq R. Joty CLL 155 96 0 14 Oct 2021
Building and Evaluating Open-Domain Dialogue Corpora with Clarifying Questions Mohammad Aliannejadi Julia Kiseleva A. Chuklin Jeffrey Stephen Dalton Mikhail Burtsev 71 95 0 13 Sep 2021
The Power of Scale for Parameter-Efficient Prompt Tuning Brian Lester Rami Al-Rfou Noah Constant VPVLM 278 3,784 0 18 Apr 2021
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 294 6,927 0 20 Apr 2018