Efficient Continual Pre-training for Building Domain Specific Large
Language Models

Efficient Continual Pre-training for Building Domain Specific Large Language Models

Annual Meeting of the Association for Computational Linguistics (ACL), 2023

14 November 2023

ArXiv (abs)PDF HTML

Papers citing "Efficient Continual Pre-training for Building Domain Specific Large Language Models"

19 / 19 papers shown

Title
Evontree: Ontology Rule-Guided Self-Evolution of Large Language Models Mingchen Tu Zhiqiang Liu Juan Li Liangyurui Liu Junjie Wang Lei Liang W. Zhang 89 0 0 30 Oct 2025
AI-Driven Generation of Old English: A Framework for Low-Resource Languages Rodrigo Gabriel Salazar Alva Matías Nuñez Cristian López Javier Martín Arista 91 0 0 27 Jul 2025
Decoupling Knowledge and Reasoning in LLMs: An Exploration Using Cognitive Dual-System Theory Mutian Yang Jiandong Gao Ji Wu 178 0 0 24 Jul 2025
Beyond Benchmarks: A Novel Framework for Domain-Specific LLM Evaluation and Knowledge Mapping Nitin Sharma Thomas Wolfers Çağatay Yıldız ALM 149 0 0 09 Jun 2025
The Future of Continual Learning in the Era of Foundation Models: Three Key Directions Jack Bell Luigi Quarantiello Eric Nuertey Coleman Lanpei Li Malio Li Mauro Madeddu Elia Piccoli Vincenzo Lomonaco KELM 252 5 0 03 Jun 2025
Nemotron-CLIMB: CLustering-based Iterative Data Mixture Bootstrapping for Language Model Pre-training Shizhe Diao Yu Yang Y. Fu Xin Dong Jane Polak Scowcroft ... Hongxu Yin M. Patwary Yingyan Jan Kautz Pavlo Molchanov 271 17 0 17 Apr 2025
Continually Evolved Multimodal Foundation Models for Cancer Prognosis Jie Peng Shuang Zhou Longwei Yang Yiran Song Mohan Zhang Kaixiong Zhou Feng Xie Mingquan Lin Rui Zhang Tianlong Chen 385 0 0 30 Jan 2025
Unveiling the Secret Recipe: A Guide For Supervised Fine-Tuning Small LLMsInternational Conference on Learning Representations (ICLR), 2024 Aldo Pareja Nikhil Shivakumar Nayak Hao Wang Krishnateja Killamsetty Shivchander Sudalairaj ... Guangxuan Xu Kai Xu Ligong Han Luke Inglis Akash Srivastava 410 28 0 17 Dec 2024
Delving into the Reversal Curse: How Far Can Large Language Models Generalize?Neural Information Processing Systems (NeurIPS), 2024 Zhengkai Lin Z. Fu Kai Liu Liang Xie Binbin Lin Wenxiao Wang Xiaofei He Yue Wu Jieping Ye LRM 350 7 0 24 Oct 2024
ZIP-FIT: Embedding-Free Data Selection via Compression-Based Alignment Elyas Obbad Iddah Mlauzi Alycia Lee Rylan Schaeffer Kamal Obbad Suhana Bedi Sanmi Koyejo CVBM 277 0 0 23 Oct 2024
Exploring Continual Fine-Tuning for Enhancing Language Ability in Large Language Model Divyanshu Aggarwal Sankarshan Damle Navin Goyal Satya Lokam Sunayana Sitaram CLL 211 3 0 21 Oct 2024
Mixing It Up: The Cocktail Effect of Multi-Task Fine-Tuning on LLM Performance -- A Case Study in Finance Meni Brief Oded Ovadia Gil Shenderovitz Noga Ben Yoash Rachel Lemberg Eitam Sheetrit 220 10 0 01 Oct 2024
Towards LifeSpan Cognitive Systems Yu Wang Chi Han Tongtong Wu Xiaoxin He Wangchunshu Zhou ... Zexue He Wei Wang Gholamreza Haffari Heng Ji Julian McAuley KELM CLL 949 8 0 20 Sep 2024
Extend Model Merging from Fine-Tuned to Pre-Trained Large Language Models via Weight Disentanglement Le Yu Bowen Yu Haiyang Yu Fei Huang Yongbin Li MoMe 225 9 0 06 Aug 2024
Task Oriented In-Domain Data Augmentation Xiao Liang Xinyu Hu Simiao Zuo Yeyun Gong Qiang Lou Yi Liu Shao-Lun Huang Jian Jiao 161 8 0 24 Jun 2024
Continual Learning of Large Language Models: A Comprehensive Survey Haizhou Shi Zihao Xu Hengyi Wang Weiyi Qin Wenyuan Wang Yibin Wang Zifeng Wang Sayna Ebrahimi Hao Wang CLL KELM LRM 361 146 0 25 Apr 2024
From Matching to Generation: A Survey on Generative Information Retrieval Xiaoxi Li Jiajie Jin Yujia Zhou Yuyao Zhang Peitian Zhang Yutao Zhu Zhicheng Dou 3DV 522 125 0 23 Apr 2024
Simple and Scalable Strategies to Continually Pre-train Large Language Models Adam Ibrahim Benjamin Thérien Kshitij Gupta Mats L. Richter Quentin Anthony Timothée Lesort Eugene Belilovsky Irina Rish KELM CLL 338 89 0 13 Mar 2024
Continual Learning for Large Language Models: A Survey Tongtong Wu Linhao Luo Yuan-Fang Li Shirui Pan Thuy-Trang Vu Gholamreza Haffari CLL LRM KELM 348 148 0 02 Feb 2024