Papers
Communities
Events
Blog
Pricing
Search
Open menu
Home
Papers
2210.16859
Cited By
A Solvable Model of Neural Scaling Laws
30 October 2022
A. Maloney
Daniel A. Roberts
J. Sully
Re-assign community
ArXiv
PDF
HTML
Papers citing
"A Solvable Model of Neural Scaling Laws"
50 / 51 papers shown
Title
Superposition Yields Robust Neural Scaling
Yizhou Liu
Ziming Liu
Jeff Gore
MILM
10
0
0
15 May 2025
Learning curves theory for hierarchically compositional data with power-law distributed features
Francesco Cagnetta
Hyunmo Kang
M. Wyart
24
0
0
11 May 2025
Scaling Laws and Representation Learning in Simple Hierarchical Languages: Transformers vs. Convolutional Architectures
Francesco Cagnetta
Alessandro Favero
Antonio Sclocchi
M. Wyart
21
0
0
11 May 2025
A Multi-Power Law for Loss Curve Prediction Across Learning Rate Schedules
Kairong Luo
Haodong Wen
Shengding Hu
Zhenbo Sun
Zhiyuan Liu
Maosong Sun
Kaifeng Lyu
Wenguang Chen
CLL
59
1
0
17 Mar 2025
Uncertainty Quantification From Scaling Laws in Deep Neural Networks
Ibrahim Elsharkawy
Yonatan Kahn
Benjamin Hooberman
UQCV
45
0
0
07 Mar 2025
Position: Solve Layerwise Linear Models First to Understand Neural Dynamical Phenomena (Neural Collapse, Emergence, Lazy/Rich Regime, and Grokking)
Yoonsoo Nam
Seok Hyeong Lee
Clementine Domine
Yea Chan Park
Charles London
Wonyl Choi
Niclas Goring
Seungjai Lee
AI4CE
33
0
0
28 Feb 2025
Physics of Skill Learning
Ziming Liu
Yizhou Liu
Eric J. Michaud
Jeff Gore
Max Tegmark
41
0
0
21 Jan 2025
Loss-to-Loss Prediction: Scaling Laws for All Datasets
David Brandfonbrener
Nikhil Anand
Nikhil Vyas
Eran Malach
Sham Kakade
77
3
0
19 Nov 2024
Scaling Laws for Pre-training Agents and World Models
Tim Pearce
Tabish Rashid
Dave Bignell
Raluca Georgescu
Sam Devlin
Katja Hofmann
LM&Ro
37
7
0
07 Nov 2024
High-dimensional Analysis of Knowledge Distillation: Weak-to-Strong Generalization and Scaling Laws
M. E. Ildiz
Halil Alperen Gozeten
Ege Onur Taga
Marco Mondelli
Samet Oymak
54
2
0
24 Oct 2024
A Simple Model of Inference Scaling Laws
Noam Levi
LRM
32
6
0
21 Oct 2024
Analyzing Neural Scaling Laws in Two-Layer Networks with Power-Law Data Spectra
Roman Worschech
B. Rosenow
36
0
0
11 Oct 2024
Strong Model Collapse
Elvis Dohmatob
Yunzhen Feng
Arjun Subramonian
Julia Kempe
26
9
0
07 Oct 2024
How Feature Learning Can Improve Neural Scaling Laws
Blake Bordelon
Alexander B. Atanasov
C. Pehlevan
49
12
0
26 Sep 2024
Unified Neural Network Scaling Laws and Scale-time Equivalence
Akhilan Boopathy
Ila Fiete
30
0
0
09 Sep 2024
Risk and cross validation in ridge regression with correlated samples
Alexander B. Atanasov
Jacob A. Zavatone-Veth
C. Pehlevan
27
4
0
08 Aug 2024
Resolving Discrepancies in Compute-Optimal Scaling of Language Models
Tomer Porian
Mitchell Wortsman
J. Jitsev
Ludwig Schmidt
Y. Carmon
50
19
0
27 Jun 2024
Towards an Improved Understanding and Utilization of Maximum Manifold Capacity Representations
Rylan Schaeffer
Victor Lecomte
Dhruv Pai
Andres Carranza
Berivan Isik
...
Yann LeCun
SueYeon Chung
Andrey Gromov
Ravid Shwartz-Ziv
Sanmi Koyejo
41
5
0
13 Jun 2024
Scaling Laws in Linear Regression: Compute, Parameters, and Data
Licong Lin
Jingfeng Wu
Sham Kakade
Peter L. Bartlett
Jason D. Lee
LRM
33
15
0
12 Jun 2024
Reconciling Kaplan and Chinchilla Scaling Laws
Tim Pearce
Jinyeop Song
32
7
0
12 Jun 2024
Hardness of Learning Neural Networks under the Manifold Hypothesis
B. Kiani
Jason Wang
Melanie Weber
34
2
0
03 Jun 2024
Classifying Overlapping Gaussian Mixtures in High Dimensions: From Optimal Classifiers to Neural Nets
Khen Cohen
Noam Levi
Yaron Oz
BDL
23
1
0
28 May 2024
Bayesian RG Flow in Neural Network Field Theories
Jessica N. Howard
Marc S. Klinger
Anindita Maiti
A. G. Stapleton
60
1
0
27 May 2024
Model Collapse Demystified: The Case of Regression
Elvis Dohmatob
Yunzhen Feng
Julia Kempe
32
32
0
12 Feb 2024
A Dynamical Model of Neural Scaling Laws
Blake Bordelon
Alexander B. Atanasov
C. Pehlevan
44
36
0
02 Feb 2024
Scaling Laws in Jet Classification
Joshua D. Batson
Yonatan Kahn
16
2
0
04 Dec 2023
More is Better in Modern Machine Learning: when Infinite Overparameterization is Optimal and Overfitting is Obligatory
James B. Simon
Dhruva Karkada
Nikhil Ghosh
Mikhail Belkin
AI4CE
BDL
26
13
0
24 Nov 2023
The Universal Statistical Structure and Scaling Laws of Chaos and Turbulence
Noam Levi
Yaron Oz
AI4CE
21
1
0
02 Nov 2023
Scaling Laws for Associative Memories
Vivien A. Cabannes
Elvis Dohmatob
A. Bietti
11
19
0
04 Oct 2023
The semantic landscape paradigm for neural networks
Shreyas Gokhale
21
2
0
18 Jul 2023
Spectral-Bias and Kernel-Task Alignment in Physically Informed Neural Networks
Inbar Seroussi
Asaf Miron
Z. Ringel
PINN
32
0
0
12 Jul 2023
Large Language Models
Michael R Douglas
LLMAG
LM&MA
33
555
0
11 Jul 2023
The Underlying Scaling Laws and Universal Statistical Structure of Complex Datasets
Noam Levi
Yaron Oz
27
4
0
26 Jun 2023
Scaling MLPs: A Tale of Inductive Bias
Gregor Bachmann
Sotiris Anagnostidis
Thomas Hofmann
29
38
0
23 Jun 2023
A Rainbow in Deep Network Black Boxes
Florentin Guth
Brice Ménard
G. Rochette
S. Mallat
14
10
0
29 May 2023
Structures of Neural Network Effective Theories
cCaugin Ararat
Tianji Cai
Cem Tekin
Zhengkang Zhang
47
7
0
03 May 2023
Variational operator learning: A unified paradigm marrying training neural operators and solving partial differential equations
Tengfei Xu
Dachuan Liu
Peng Hao
Bo Wang
28
4
0
09 Apr 2023
Inductive biases in deep learning models for weather prediction
Jannik Thümmel
Matthias Karlbauer
S. Otte
C. Zarfl
Georg Martius
...
Thomas Scholten
Ulrich Friedrich
V. Wulfmeyer
B. Goswami
Martin Volker Butz
AI4CE
31
4
0
06 Apr 2023
The Quantization Model of Neural Scaling
Eric J. Michaud
Ziming Liu
Uzay Girit
Max Tegmark
MILM
22
77
0
23 Mar 2023
Unifying Grokking and Double Descent
Peter W. Battaglia
David Raposo
Kelsey
30
31
0
10 Mar 2023
Learning curves for deep structured Gaussian feature models
Jacob A. Zavatone-Veth
C. Pehlevan
MLT
10
11
0
01 Mar 2023
A Simplistic Model of Neural Scaling Laws: Multiperiodic Santa Fe Processes
L. Debowski
MILM
8
11
0
17 Feb 2023
Over-parametrization via Lifting for Low-rank Matrix Sensing: Conversion of Spurious Solutions to Strict Saddle Points
Ziye Ma
Igor Molybog
Javad Lavaei
Somayeh Sojoudi
11
3
0
15 Feb 2023
Bayesian Interpolation with Deep Linear Networks
Boris Hanin
Alexander Zlokapa
34
25
0
29 Dec 2022
The Onset of Variance-Limited Behavior for Networks in the Lazy and Rich Regimes
Alexander B. Atanasov
Blake Bordelon
Sabarish Sainathan
C. Pehlevan
14
26
0
23 Dec 2022
Error Scaling Laws for Kernel Classification under Source and Capacity Conditions
Hugo Cui
Bruno Loureiro
Florent Krzakala
Lenka Zdeborová
38
10
0
29 Jan 2022
Explaining Neural Scaling Laws
Yasaman Bahri
Ethan Dyer
Jared Kaplan
Jaehoon Lee
Utkarsh Sharma
17
245
0
12 Feb 2021
Learning Curve Theory
Marcus Hutter
128
58
0
08 Feb 2021
How Can We Accelerate Progress Towards Human-like Linguistic Generalization?
Tal Linzen
216
188
0
03 May 2020
Spectrum Dependent Learning Curves in Kernel Regression and Wide Neural Networks
Blake Bordelon
Abdulkadir Canatar
C. Pehlevan
131
200
0
07 Feb 2020
1
2
Next