Text Quality-Based Pruning for Efficient Training of Language Models

26 April 2024

Luke Zettlemoyer

Papers citing "Text Quality-Based Pruning for Efficient Training of Language Models"

3 / 3 papers shown

Title
Reverse Modeling in Large Language Models S. Yu Yuanchen Xu Cunxiao Du Yanying Zhou Minghui Qiu Q. Sun Hao Zhang Jiawei Wu 21 1 0 13 Oct 2024
RegMix: Data Mixture as Regression for Language Model Pre-training Qian Liu Xiaosen Zheng Niklas Muennighoff Guangtao Zeng Longxu Dou Tianyu Pang Jing Jiang Min-Bin Lin MoE 45 34 1 01 Jul 2024
The Pile: An 800GB Dataset of Diverse Text for Language Modeling Leo Gao Stella Biderman Sid Black Laurence Golding Travis Hoppe ... Horace He Anish Thite Noa Nabeshima Shawn Presser Connor Leahy AIMat 236 1,508 0 31 Dec 2020