Recyclable Tuning for Continual Pre-training

Recyclable Tuning for Continual Pre-training

15 May 2023

Xu Han

Yankai Lin

Zhiyuan Liu

Maosong Sun

Jie Zhou

Papers citing "Recyclable Tuning for Continual Pre-training"

11 / 11 papers shown

Title
Efficient Model Development through Fine-tuning Transfer Pin-Jie Lin Rishab Balasubramanian Fengyuan Liu Nikhil Kandpal Tu Vu 59 0 0 25 Mar 2025
MUSCLE: A Model Update Strategy for Compatible LLM Evolution Jessica Echterhoff Fartash Faghri Raviteja Vemulapalli Ting-Yao Hu Chun-Liang Li Oncel Tuzel Hadi Pouransari KELM 56 2 0 12 Jul 2024
Towards Lifelong Learning of Large Language Models: A Survey Junhao Zheng Shengjie Qiu Chengming Shi Qianli Ma KELM CLL 28 14 0 10 Jun 2024
Continual Learning of Large Language Models: A Comprehensive Survey Haizhou Shi Zihao Xu Hengyi Wang Weiyi Qin Wenyuan Wang Yibin Wang Zifeng Wang Sayna Ebrahimi Hao Wang CLL KELM LRM 39 62 0 25 Apr 2024
Simple and Scalable Strategies to Continually Pre-train Large Language Models Adam Ibrahim Benjamin Thérien Kshitij Gupta Mats L. Richter Quentin Anthony Timothée Lesort Eugene Belilovsky Irina Rish KELM CLL 44 51 0 13 Mar 2024
Continual Learning for Large Language Models: A Survey Tongtong Wu Linhao Luo Yuan-Fang Li Shirui Pan Thuy-Trang Vu Gholamreza Haffari CLL LRM KELM 21 102 0 02 Feb 2024
Towards Robust and Efficient Continual Language Learning Adam Fisch Amal Rannen-Triki Razvan Pascanu J. Bornschein Angeliki Lazaridou E. Gribovskaya MarcÁurelio Ranzato CLL 24 1 0 11 Jul 2023
Knowledge Card: Filling LLMs' Knowledge Gaps with Plug-in Specialized Language Models Shangbin Feng Weijia Shi Yuyang Bai Vidhisha Balachandran Tianxing He Yulia Tsvetkov KELM 45 28 0 17 May 2023
Multitask Prompted Training Enables Zero-Shot Task Generalization Victor Sanh Albert Webson Colin Raffel Stephen H. Bach Lintang Sutawika ... T. Bers Stella Biderman Leo Gao Thomas Wolf Alexander M. Rush LRM 211 1,656 0 15 Oct 2021
Towards Continual Knowledge Learning of Language Models Joel Jang Seonghyeon Ye Sohee Yang Joongbo Shin Janghoon Han Gyeonghun Kim Stanley Jungkyu Choi Minjoon Seo CLL KELM 222 150 0 07 Oct 2021
Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inference Timo Schick Hinrich Schütze 258 1,586 0 21 Jan 2020