gzip Predicts Data-dependent Scaling Laws

26 May 2024

Papers citing "gzip Predicts Data-dependent Scaling Laws"

10 / 10 papers shown

Title
Scaling Behaviors of LLM Reinforcement Learning Post-Training: An Empirical Study in Mathematical Reasoning Zelin Tan Hejia Geng M. Zhang Xiaohang Yu Guancheng Wan ... Zaibin Zhang G. Zhang Chen Zhang Z. Yin Wenlong Zhang OffRL LRM 122 1 1 29 Sep 2025
Transformers Can Learn Connectivity in Some Graphs but Not Others Amit Roy Abulhair Saparov LRM 48 0 0 26 Sep 2025
Mechanistic evaluation of Transformers and state space models Aryaman Arora Neil Rathi Nikil Roashan Selvam Róbert Csordás Dan Jurafsky Christopher Potts 351 3 0 21 May 2025
BioVFM-21M: Benchmarking and Scaling Self-Supervised Vision Foundation Models for Biomedical Image Analysis Jiarun Liu Hong-Yu Zhou Weijian Huang Hao Yang Dongning Song Tao Tan Yong Liang Shanshan Wang MedIm 214 0 0 14 May 2025
Bayesian scaling laws for in-context learning Aryaman Arora Dan Jurafsky Christopher Potts Noah D. Goodman 369 11 0 21 Oct 2024
A Hitchhiker's Guide to Scaling Law Estimation Leshem Choshen Yang Zhang Jacob Andreas 237 14 0 15 Oct 2024
CMR Scaling Law: Predicting Critical Mixture Ratios for Continual Pre-training of Language Models Jiawei Gu Zacc Yang Chuanghao Ding Rui Zhao Fei Tan CLL 241 15 0 24 Jul 2024
Entropy Law: The Story Behind Data Compression and LLM Performance Mingjia Yin Chuhan Wu Yufei Wang Hao Wang Wei Guo Yasheng Wang Yong Liu Ruiming Tang Defu Lian Enhong Chen 238 41 0 09 Jul 2024
RegMix: Data Mixture as Regression for Language Model Pre-training Qian Liu Xiaosen Zheng Niklas Muennighoff Guangtao Zeng Longxu Dou Tianyu Pang Jing Jiang Min Lin MoE 273 90 1 01 Jul 2024
Scaling Laws and Compute-Optimal Training Beyond Fixed Training Durations Alexander Hägele Elie Bakouch Atli Kosson Loubna Ben Allal Leandro von Werra Martin Jaggi 326 89 0 28 May 2024