gzip Predicts Data-dependent Scaling Laws

26 May 2024

Papers citing "gzip Predicts Data-dependent Scaling Laws"

5 / 5 papers shown

Title
RegMix: Data Mixture as Regression for Language Model Pre-training Qian Liu Xiaosen Zheng Niklas Muennighoff Guangtao Zeng Longxu Dou Tianyu Pang Jing Jiang Min-Bin Lin MoE 64 36 1 01 Jul 2024
Compression Represents Intelligence Linearly Yuzhen Huang Jinghan Zhang Zifei Shan Junxian He 39 24 0 15 Apr 2024
Mission: Impossible Language Models Julie Kallini Isabel Papadimitriou Richard Futrell Kyle Mahowald Christopher Potts ELM LRM 42 19 0 12 Jan 2024
DeepSeek LLM: Scaling Open-Source Language Models with Longtermism DeepSeek-AI Xiao Bi : Xiao Bi Deli Chen Guanting Chen ... Yao Zhao Shangyan Zhou Shunfeng Zhou Qihao Zhu Yuheng Zou LRM ALM 139 298 0 05 Jan 2024
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 226 4,424 0 23 Jan 2020