Towards Understanding the Spectral Bias of Deep Learning

3 December 2019

Quanquan Gu

Papers citing "Towards Understanding the Spectral Bias of Deep Learning"

50 / 129 papers shown

Title
Towards a Machine-Learned Poisson Solver for Low-Temperature Plasma Simulations in Complex Geometries Ihda Chaerony Siffa M. Becker K. Weltmann J. Trieschmann 30 2 0 13 Jun 2023
The Law of Parsimony in Gradient Descent for Learning Deep Linear Networks Can Yaras P. Wang Wei Hu Zhihui Zhu Laura Balzano Qing Qu 33 17 0 01 Jun 2023
Tight conditions for when the NTK approximation is valid Enric Boix-Adserà Etai Littwin 30 0 0 22 May 2023
Deep ReLU Networks Have Surprisingly Simple Polytopes Fenglei Fan Wei Huang Xiang-yu Zhong Lecheng Ruan T. Zeng Huan Xiong Fei-Yue Wang 57 5 0 16 May 2023
Simulation and Prediction of Countercurrent Spontaneous Imbibition at Early and Late Times Using Physics-Informed Neural Networks J. Abbasi P. Andersen PINN 14 4 0 06 May 2023
On the Stepwise Nature of Self-Supervised Learning James B. Simon Maksis Knutins Liu Ziyin Daniel Geisz Abraham J. Fetterman Joshua Albrecht SSL 32 29 0 27 Mar 2023
Regularize implicit neural representation by itself Zhemin Li Hongxia Wang Deyu Meng 20 9 0 27 Mar 2023
Linear CNNs Discover the Statistical Structure of the Dataset Using Only the Most Dominant Frequencies Hannah Pinson Joeri Lenaerts V. Ginis 13 3 0 03 Mar 2023
PIFON-EPT: MR-Based Electrical Property Tomography Using Physics-Informed Fourier Networks Xinling Yu José E. C. Serrallés Ilias I. Giannakopoulos Z. Liu Luca Daniel R. Lattanzi Zheng-Wei Zhang 14 10 0 23 Feb 2023
$Generalization Ability of Wide Neural Networks on $\mathbb{R}$$ Generalization Ability of Wide Neural Networks on $\mathbb{R}$ Jianfa Lai Manyun Xu Rui Chen Qi-Rong Lin 16 21 0 12 Feb 2023
AttNS: Attention-Inspired Numerical Solving For Limited Data Scenarios Zhongzhan Huang Mingfu Liang Liang Lin Liang Lin 26 5 0 05 Feb 2023
Understanding the Spectral Bias of Coordinate Based MLPs Via Training Dynamics J. Lazzari Xiuwen Liu 24 3 0 14 Jan 2023
Incremental Spatial and Spectral Learning of Neural Operators for Solving Large-Scale PDEs Robert Joseph George Jiawei Zhao Jean Kossaifi Zong-Yi Li Anima Anandkumar AI4CE 27 9 0 28 Nov 2022
An Empirical Analysis of the Advantages of Finite- v.s. Infinite-Width Bayesian Neural Networks Jiayu Yao Yaniv Yacoby Beau Coker Weiwei Pan Finale Doshi-Velez 19 1 0 16 Nov 2022
Deep Reinforcement Learning for IRS Phase Shift Design in Spatiotemporally Correlated Environments Spilios Evmorfos Athina P. Petropulu H. Vincent Poor OOD 13 3 0 02 Nov 2022
Bayesian deep learning framework for uncertainty quantification in high dimensions Jeahan Jung Minseok Choi BDL UQCV 13 1 0 21 Oct 2022
Random Weight Factorization Improves the Training of Continuous Neural Representations Sifan Wang Hanwen Wang Jacob H. Seidman P. Perdikaris 21 9 0 03 Oct 2022
FINDE: Neural Differential Equations for Finding and Preserving Invariant Quantities Takashi Matsubara Takaharu Yaguchi PINN 14 7 0 01 Oct 2022
Extrapolation and Spectral Bias of Neural Nets with Hadamard Product: a Polynomial Net Study Yongtao Wu Zhenyu Zhu Fanghui Liu Grigorios G. Chrysos V. Cevher 28 10 0 16 Sep 2022
Semi-analytic PINN methods for singularly perturbed boundary value problems G. Gie Youngjoon Hong Chang-Yeol Jung PINN 8 5 0 19 Aug 2022
Efficient Climate Simulation via Machine Learning Method Xin Wang Wei Xue Yilun Han Guangwen Yang AILaw 26 2 0 15 Aug 2022
On the Activation Function Dependence of the Spectral Bias of Neural Networks Q. Hong Jonathan W. Siegel Qinyan Tan Jinchao Xu 32 22 0 09 Aug 2022
Hidden Progress in Deep Learning: SGD Learns Parities Near the Computational Limit Boaz Barak Benjamin L. Edelman Surbhi Goel Sham Kakade Eran Malach Cyril Zhang 27 123 0 18 Jul 2022
Implicit regularization of dropout Zhongwang Zhang Zhi-Qin John Xu 19 26 0 13 Jul 2022
Momentum Diminishes the Effect of Spectral Bias in Physics-Informed Neural Networks G. Farhani Alexander Kazachek Boyu Wang 19 6 0 29 Jun 2022
Strong Lensing Source Reconstruction Using Continuous Neural Fields S. Mishra-Sharma Ge Yang 69 13 0 29 Jun 2022
Finite Expression Method for Solving High-Dimensional Partial Differential Equations Senwei Liang Haizhao Yang 21 18 0 21 Jun 2022
Overcoming the Spectral Bias of Neural Value Approximation Ge Yang Anurag Ajay Pulkit Agrawal 32 25 0 09 Jun 2022
Spectral Bias Outside the Training Set for Deep Networks in the Kernel Regime Benjamin Bowman Guido Montúfar 14 14 0 06 Jun 2022
The Directional Bias Helps Stochastic Gradient Descent to Generalize in Kernel Regression Models Yiling Luo X. Huo Y. Mei 11 0 0 29 Apr 2022
The Spectral Bias of Polynomial Neural Networks Moulik Choraria L. Dadi Grigorios G. Chrysos Julien Mairal V. Cevher 22 18 0 27 Feb 2022
Overview frequency principle/spectral bias in deep learning Z. Xu Yaoyu Zhang Tao Luo FaML 25 65 0 19 Jan 2022
Implicit Bias of MSE Gradient Optimization in Underparameterized Neural Networks Benjamin Bowman Guido Montúfar 18 11 0 12 Jan 2022
Subspace Decomposition based DNN algorithm for elliptic type multi-scale PDEs Xi-An Li Z. Xu Lei Zhang 13 27 0 10 Dec 2021
ConDA: Unsupervised Domain Adaptation for LiDAR Segmentation via Regularized Domain Concatenation Lingdong Kong N. Quader Venice Erin Liong 13 48 0 30 Nov 2021
The Three Stages of Learning Dynamics in High-Dimensional Kernel Methods Nikhil Ghosh Song Mei Bin Yu 17 20 0 13 Nov 2021
Understanding Layer-wise Contributions in Deep Neural Networks through Spectral Analysis Yatin Dandi Arthur Jacot FAtt 18 4 0 06 Nov 2021
Mean-field Analysis of Piecewise Linear Solutions for Wide ReLU Networks A. Shevchenko Vyacheslav Kungurtsev Marco Mondelli MLT 36 13 0 03 Nov 2021
The Eigenlearning Framework: A Conservation Law Perspective on Kernel Regression and Wide Neural Networks James B. Simon Madeline Dickens Dhruva Karkada M. DeWeese 42 27 0 08 Oct 2021
Improved architectures and training algorithms for deep operator networks Sifan Wang Hanwen Wang P. Perdikaris AI4CE 47 105 0 04 Oct 2021
Finite Basis Physics-Informed Neural Networks (FBPINNs): a scalable domain decomposition approach for solving differential equations Benjamin Moseley Andrew Markham T. Nissen‐Meyer PINN 45 209 0 16 Jul 2021
Neural Contextual Bandits without Regret Parnian Kassraie Andreas Krause OffRL 15 38 0 07 Jul 2021
Neural Active Learning with Performance Guarantees Pranjal Awasthi Christoph Dann Claudio Gentile Ayush Sekhari Zhilei Wang 24 22 0 06 Jun 2021
Reverse Engineering the Neural Tangent Kernel James B. Simon Sajant Anand M. DeWeese 22 9 0 06 Jun 2021
An Upper Limit of Decaying Rate with Respect to Frequency in Deep Neural Network Tao Luo Zheng Ma Zhiwei Wang Z. Xu Yaoyu Zhang 4 4 0 25 May 2021
Deep Kronecker neural networks: A general framework for neural networks with adaptive activation functions Ameya Dilip Jagtap Yeonjong Shin Kenji Kawaguchi George Karniadakis ODL 37 131 0 20 May 2021
Principal Components Bias in Over-parameterized Linear Models, and its Manifestation in Deep Neural Networks Guy Hacohen D. Weinshall 11 10 0 12 May 2021
Universal scaling laws in the gradient descent training of neural networks Maksim Velikanov Dmitry Yarotsky 46 9 0 02 May 2021
Sensitivity as a Complexity Measure for Sequence Classification Tasks Michael Hahn Dan Jurafsky Richard Futrell 150 22 0 21 Apr 2021
SAPE: Spatially-Adaptive Progressive Encoding for Neural Optimization Amir Hertz Or Perel Raja Giryes O. Sorkine-Hornung Daniel Cohen-Or 26 67 0 19 Apr 2021