Instruction Pre-Training: Language Models are Supervised Multitask
Learners

Instruction Pre-Training: Language Models are Supervised Multitask Learners

20 June 2024

Papers citing "Instruction Pre-Training: Language Models are Supervised Multitask Learners"

16 / 16 papers shown

Title
QuaDMix: Quality-Diversity Balanced Data Selection for Efficient LLM Pretraining Fengze Liu Weidong Zhou Binbin Liu Zhimiao Yu Yifan Zhang ... Yifeng Yu Bingni Zhang Xiaohuan Zhou Taifeng Wang Yong Cao 52 0 0 23 Apr 2025
Programming Every Example: Lifting Pre-training Data Quality Like Experts at Scale Fan Zhou Zengzhi Wang Qian Liu Junlong Li Pengfei Liu ALM 83 14 0 17 Feb 2025
SimRAG: Self-Improving Retrieval-Augmented Generation for Adapting Large Language Models to Specialized Domains Ran Xu Hui Liu Sreyashi Nag Zhenwei Dai Yaochen Xie ... Chen Luo Yang Li Joyce C. Ho Carl Yang Qi He RALM 63 8 0 28 Jan 2025
Domaino1s: Guiding LLM Reasoning for Explainable Answers in High-Stakes Domains Xu Chu Zhijie Tan Hanlin Xue Guanyu Wang Tong Mo Weiping Li ELM LRM 46 1 0 24 Jan 2025
Unleashing the Power of Data Tsunami: A Comprehensive Survey on Data Assessment and Selection for Instruction Tuning of Language Models Yulei Qin Yuncheng Yang Pengcheng Guo Gang Li Hang Shao Yuchen Shi Zihan Xu Yun Gu Ke Li Xing Sun ALM 64 11 0 31 Dec 2024
On Domain-Specific Post-Training for Multimodal Large Language Models Daixuan Cheng Shaohan Huang Ziyu Zhu Xintong Zhang Wayne Xin Zhao Zhongzhi Luan Bo Dai Zhenliang Zhang VLM 82 2 0 29 Nov 2024
MiniPLM: Knowledge Distillation for Pre-Training Language Models Yuxian Gu Hao Zhou Fandong Meng Jie Zhou Minlie Huang 46 5 0 22 Oct 2024
Mastering the Craft of Data Synthesis for CodeLLMs Meng Chen Philip Arthur Qianyu Feng Cong Duy Vu Hoang Yu-Heng Hong ... Mark Johnson K. K. Don Dharmasiri Long Duong Yuan-Fang Li SyDa 40 1 0 16 Oct 2024
An Investigation of Warning Erroneous Chat Translations in Cross-lingual Communication Yunmeng Li Jun Suzuki Makoto Morishita Kaori Abe Kentaro Inui 35 1 0 28 Aug 2024
LLM2LLM: Boosting LLMs with Novel Iterative Data Enhancement Nicholas Lee Thanakul Wattanawong Sehoon Kim K. Mangalam Sheng Shen Gopala Anumanchipalli Michael W. Mahoney Kurt Keutzer A. Gholami 56 12 0 22 Mar 2024
The Falcon Series of Open Language Models Ebtesam Almazrouei Hamza Alobeidli Abdulaziz Alshamsi Alessandro Cappelli Ruxandra-Aimée Cojocaru ... Quentin Malartic Daniele Mazzotta Badreddine Noune B. Pannier Guilherme Penedo AI4TS ALM 104 389 0 28 Nov 2023
Pre-Training to Learn in Context Yuxian Gu Li Dong Furu Wei Minlie Huang CLIP LRM ReLM 104 37 0 16 May 2023
WikiWhy: Answering and Explaining Cause-and-Effect Questions Matthew Ho Aditya Sharma Justin Chang Michael Stephen Saxon Sharon Levy Yujie Lu William Yang Wang ReLM KELM LRM 47 16 0 21 Oct 2022
Multitask Prompted Training Enables Zero-Shot Task Generalization Victor Sanh Albert Webson Colin Raffel Stephen H. Bach Lintang Sutawika ... T. Bers Stella Biderman Leo Gao Thomas Wolf Alexander M. Rush LRM 203 1,651 0 15 Oct 2021
The Pile: An 800GB Dataset of Diverse Text for Language Modeling Leo Gao Stella Biderman Sid Black Laurence Golding Travis Hoppe ... Horace He Anish Thite Noa Nabeshima Shawn Presser Connor Leahy AIMat 236 1,508 0 31 Dec 2020
PubMedQA: A Dataset for Biomedical Research Question Answering Qiao Jin Bhuwan Dhingra Zhengping Liu William W. Cohen Xinghua Lu 196 791 0 13 Sep 2019