Scaling Laws in Linear Regression: Compute, Parameters, and Data

Scaling Laws in Linear Regression: Compute, Parameters, and Data

12 June 2024

Peter L. Bartlett

Jason D. Lee

Papers citing "Scaling Laws in Linear Regression: Compute, Parameters, and Data"

18 / 18 papers shown

Title
Learning curves theory for hierarchically compositional data with power-law distributed features Francesco Cagnetta Hyunmo Kang M. Wyart 24 0 0 11 May 2025
A Multi-Power Law for Loss Curve Prediction Across Learning Rate Schedules Kairong Luo Haodong Wen Shengding Hu Zhenbo Sun Zhiyuan Liu Maosong Sun Kaifeng Lyu Wenguang Chen CLL 54 0 0 17 Mar 2025
Weak-to-Strong Generalization Even in Random Feature Networks, Provably Marko Medvedev Kaifeng Lyu Dingli Yu Sanjeev Arora Zhiyuan Li Nathan Srebro 102 0 0 04 Mar 2025
Scaling Law Phenomena Across Regression Paradigms: Multiple and Kernel Approaches Yifang Chen Xuyang Guo Xiaoyu Li Yingyu Liang Zhenmei Shi Zhao-quan Song 56 3 0 03 Mar 2025
Deep Learning is Not So Mysterious or Different Andrew Gordon Wilson 36 1 0 03 Mar 2025
How to Upscale Neural Networks with Scaling Law? A Survey and Practical Guidelines Ayan Sengupta Yash Goel Tanmoy Chakraborty 41 0 0 17 Feb 2025
Scaling Laws for Floating Point Quantization Training X. Sun Shuaipeng Li Ruobing Xie Weidong Han Kan Wu ... Yangyu Tao Zhanhui Kang C. Xu Di Wang Jie Jiang MQ AIFin 53 0 0 05 Jan 2025
Scaling Laws for Precision Tanishq Kumar Zachary Ankner Benjamin Spector Blake Bordelon Niklas Muennighoff Mansheej Paul C. Pehlevan Christopher Ré Aditi Raghunathan AIFin MoMe 46 12 0 07 Nov 2024
How Does Critical Batch Size Scale in Pre-training? Hanlin Zhang Depen Morwani Nikhil Vyas Jingfeng Wu Difan Zou Udaya Ghai Dean Phillips Foster Sham Kakade 64 8 0 29 Oct 2024
Analyzing Neural Scaling Laws in Two-Layer Networks with Power-Law Data Spectra Roman Worschech B. Rosenow 36 0 0 11 Oct 2024
How Feature Learning Can Improve Neural Scaling Laws Blake Bordelon Alexander B. Atanasov C. Pehlevan 46 11 0 26 Sep 2024
Resolving Discrepancies in Compute-Optimal Scaling of Language Models Tomer Porian Mitchell Wortsman J. Jitsev Ludwig Schmidt Y. Carmon 48 19 0 27 Jun 2024
Scaling laws for learning with real and surrogate data Ayush Jain Andrea Montanari Eren Sasoglu 35 11 0 06 Feb 2024
A Dynamical Model of Neural Scaling Laws Blake Bordelon Alexander B. Atanasov C. Pehlevan 44 36 0 02 Feb 2024
Revisiting Neural Scaling Laws in Language and Vision Ibrahim M. Alabdulmohsin Behnam Neyshabur Xiaohua Zhai 148 101 0 13 Sep 2022
Last Iterate Risk Bounds of SGD with Decaying Stepsize for Overparameterized Linear Regression Jingfeng Wu Difan Zou Vladimir Braverman Quanquan Gu Sham Kakade 96 20 0 12 Oct 2021
Learning Curve Theory Marcus Hutter 128 56 0 08 Feb 2021
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 223 4,424 0 23 Jan 2020