A Constructive Prediction of the Generalization Error Across Scales

27 September 2019

Jonathan S. Rosenfeld

Papers citing "A Constructive Prediction of the Generalization Error Across Scales"

50 / 159 papers shown

Title
A Model Zoo on Phase Transitions in Neural Networks Konstantin Schurholt Léo Meynent Yefan Zhou Haiquan Lu Yaoqing Yang Damian Borth 68 0 0 25 Apr 2025
Scaling Laws for Data-Efficient Visual Transfer Learning Wenxuan Yang Qingqu Wei Chenxi Ma Weimin Tan Bo Yan 28 0 0 17 Apr 2025
On Model and Data Scaling for Skeleton-based Self-Supervised Gait Recognition Adrian Cosma Andy Catruna Emilian Radoi 31 0 0 10 Apr 2025
Towards Combinatorial Interpretability of Neural Computation Micah Adler Dan Alistarh Nir Shavit FAtt 113 1 0 10 Apr 2025
Data Scaling Laws for End-to-End Autonomous Driving Alexander Naumann Xunjiang Gu Tolga Dimlioglu Mariusz Bojarski Alperen Degirmenci A. Popov Devansh Bisla Marco Pavone Urs Muller B. Ivanovic 48 0 0 06 Apr 2025
Compression Laws for Large Language Models Ayan Sengupta Siddhant Chaudhary Tanmoy Chakraborty 26 0 0 06 Apr 2025
Hyperflows: Pruning Reveals the Importance of Weights Eugen Barbulescu Antonio Alexoaie 21 0 0 06 Apr 2025
Scaling Laws of Synthetic Data for Language Models Zeyu Qin Qingxiu Dong Xingxing Zhang Li Dong Xiaolong Huang ... Hany Awadalla Yi R. Fung Weizhu Chen Minhao Cheng Furu Wei SyDa 75 2 0 25 Mar 2025
Improving Quantization with Post-Training Model Expansion Giuseppe Franco Pablo Monteagudo-Lago Ian Colbert Nicholas J. Fraser Michaela Blott MQ 57 1 0 21 Mar 2025
Compute Optimal Scaling of Skills: Knowledge vs Reasoning Nicholas Roberts Niladri S. Chatterji Sharan Narang Mike Lewis Dieuwke Hupkes 46 2 0 13 Mar 2025
Non-Gaussianities in Collider Metric Binning Andrew J. Larkoski 52 1 0 05 Mar 2025
Scaling Law Phenomena Across Regression Paradigms: Multiple and Kernel Approaches Yifang Chen Xuyang Guo Xiaoyu Li Yingyu Liang Zhenmei Shi Zhao-quan Song 65 3 0 03 Mar 2025
Position: Solve Layerwise Linear Models First to Understand Neural Dynamical Phenomena (Neural Collapse, Emergence, Lazy/Rich Regime, and Grokking) Yoonsoo Nam Seok Hyeong Lee Clementine Domine Yea Chan Park Charles London Wonyl Choi Niclas Goring Seungjai Lee AI4CE 38 0 0 28 Feb 2025
(Mis)Fitting: A Survey of Scaling Laws Margaret Li Sneha Kudugunta Luke Zettlemoyer 69 2 0 26 Feb 2025
Distributional Scaling Laws for Emergent Capabilities Rosie Zhao Tian Qin David Alvarez-Melis Sham Kakade Naomi Saphra LRM 39 0 0 24 Feb 2025
Model-agnostic Coreset Selection via LLM-based Concept Bottlenecks Akshay Mehra Trisha Mittal Subhadra Gopalakrishnan Joshua Kimball 45 0 0 23 Feb 2025
Scaling Trends in Language Model Robustness Nikolhaus Howe Michal Zajac I. R. McKenzie Oskar Hollinsworth Tom Tseng Aaron David Tucker Pierre-Luc Bacon Adam Gleave 109 2 0 21 Feb 2025
Unveiling the Secret Recipe: A Guide For Supervised Fine-Tuning Small LLMs Aldo Pareja Nikhil Shivakumar Nayak Hao Wang Krishnateja Killamsetty Shivchander Sudalairaj ... Guangxuan Xu Kai Xu Ligong Han Luke Inglis Akash Srivastava 88 6 0 17 Dec 2024
Scaling Sequential Recommendation Models with Transformers Pablo Zivic Hernán Ceferino Vázquez Jorge Sanchez OffRL LRM 79 1 0 10 Dec 2024
Sloth: scaling laws for LLM skills to predict multi-benchmark performance across families Felipe Maia Polo S. Kamath S Leshem Choshen Yuekai Sun Mikhail Yurochkin 89 6 0 09 Dec 2024
Understanding Scaling Laws with Statistical and Approximation Theory for Transformer Neural Networks on Intrinsically Low-dimensional Data Alex Havrilla Wenjing Liao 34 8 0 11 Nov 2024
Scaling Laws for Pre-training Agents and World Models Tim Pearce Tabish Rashid Dave Bignell Raluca Georgescu Sam Devlin Katja Hofmann LM&Ro 40 6 0 07 Nov 2024
Does equivariance matter at scale? Johann Brehmer S. Behrends P. D. Haan Taco S. Cohen 44 10 0 30 Oct 2024
Relaxed Recursive Transformers: Effective Parameter Sharing with Layer-wise LoRA Sangmin Bae Adam Fisch Hrayr Harutyunyan Ziwei Ji Seungyeon Kim Tal Schuster KELM 78 5 0 28 Oct 2024
A Simple Model of Inference Scaling Laws Noam Levi LRM 32 6 0 21 Oct 2024
A Hitchhiker's Guide to Scaling Law Estimation Leshem Choshen Yang Zhang Jacob Andreas 41 6 0 15 Oct 2024
Anisotropic Diffusion Probabilistic Model for Imbalanced Image Classification Jingyu Kong Yuan Guo Yu Wang Yuping Duan DiffM MedIm 31 0 0 22 Sep 2024
EchoDFKD: Data-Free Knowledge Distillation for Cardiac Ultrasound Segmentation using Synthetic Data Grégoire Petit Nathan Palluau Axel Bauer Clemens Dlaska 55 0 0 11 Sep 2024
Sam2Rad: A Segmentation Model for Medical Images with Learnable Prompts Assefa Seyoum Wahd B. Felfeliyan Yuyue Zhou Shrimanti Ghosh Adam McArthur Jiechen Zhang Jacob L. Jaremko A. Hareendranathan VLM MedIm 45 1 0 10 Sep 2024
Unified Neural Network Scaling Laws and Scale-time Equivalence Akhilan Boopathy Ila Fiete 40 0 0 09 Sep 2024
Breaking Neural Network Scaling Laws with Modularity Akhilan Boopathy Sunshine Jiang William Yue Jaedong Hwang Abhiram Iyer Ila Fiete OOD 39 2 0 09 Sep 2024
An Empirical Study of Scaling Laws for Transfer Matthew Barnett 27 1 0 30 Aug 2024
Evaluating Time-Series Training Dataset through Lens of Spectrum in Deep State Space Models Sekitoshi Kanai Yasutoshi Ida Kazuki Adachi Mihiro Uchida Tsukasa Yoshida Shinýa Yamaguchi 20 1 0 29 Aug 2024
Scaling Training Data with Lossy Image Compression Katherine L. Mentzer Andrea Montanari 36 0 0 25 Jul 2024
Resolving Discrepancies in Compute-Optimal Scaling of Language Models Tomer Porian Mitchell Wortsman J. Jitsev Ludwig Schmidt Y. Carmon 55 20 0 27 Jun 2024
Towards an Improved Understanding and Utilization of Maximum Manifold Capacity Representations Rylan Schaeffer Victor Lecomte Dhruv Pai Andres Carranza Berivan Isik ... Yann LeCun SueYeon Chung Andrey Gromov Ravid Shwartz-Ziv Sanmi Koyejo 49 5 0 13 Jun 2024
Scaling Laws in Linear Regression: Compute, Parameters, and Data Licong Lin Jingfeng Wu Sham Kakade Peter L. Bartlett Jason D. Lee LRM 41 15 0 12 Jun 2024
Reconciling Kaplan and Chinchilla Scaling Laws Tim Pearce Jinyeop Song 34 8 0 12 Jun 2024
SAVA: Scalable Learning-Agnostic Data Valuation Samuel Kessler Tam Le Vu Nguyen TDI 53 0 0 03 Jun 2024
Scaling Laws for the Value of Individual Data Points in Machine Learning Ian Covert Wenlong Ji Tatsunori Hashimoto James Y. Zou TDI 37 8 0 30 May 2024
Phase Transitions in the Output Distribution of Large Language Models Julian Arnold Flemming Holtorf Frank Schafer Niels Lörch 41 1 0 27 May 2024
gzip Predicts Data-dependent Scaling Laws Rohan Pandey 25 10 0 26 May 2024
Scaling Laws for Galaxy Images Mike Walmsley Micah Bowles Anna M. M. Scaife Jason Shingirai Makechemu Alexander J. Gordon ... Chris J. Lintott K. Mantha Devina Mohan David O’Ryan Inigo V. Slijepevic 21 4 0 03 Apr 2024
Language models scale reliably with over-training and on downstream tasks S. Gadre Georgios Smyrnis Vaishaal Shankar Suchin Gururangan Mitchell Wortsman ... Y. Carmon Achal Dave Reinhard Heckel Niklas Muennighoff Ludwig Schmidt ALM ELM LRM 108 40 0 13 Mar 2024
A Case for Validation Buffer in Pessimistic Actor-Critic Michal Nauman M. Ostaszewski Marek Cygan 34 0 0 01 Mar 2024
Improving Neural-based Classification with Logical Background Knowledge Arthur Ledaguenel Céline Hudelot M. Khouadjia NAI 44 1 0 20 Feb 2024
A Tale of Tails: Model Collapse as a Change of Scaling Laws Elvis Dohmatob Yunzhen Feng Pu Yang Francois Charton Julia Kempe 21 62 0 10 Feb 2024
Scaling laws for learning with real and surrogate data Ayush Jain Andrea Montanari Eren Sasoglu 35 11 0 06 Feb 2024
Selecting Large Language Model to Fine-tune via Rectified Scaling Law Haowei Lin Baizhou Huang Haotian Ye Qinyu Chen Zihao Wang Sujian Li Jianzhu Ma Xiaojun Wan James Y. Zou Yitao Liang 82 20 0 04 Feb 2024
Enhancing In-context Learning via Linear Probe Calibration Momin Abbas Yi Zhou Parikshit Ram Nathalie Baracaldo Horst Samulowitz Theodoros Salonidis Tianyi Chen 76 9 0 22 Jan 2024