Towards Understanding the Spectral Bias of Deep Learning

3 December 2019

Quanquan Gu

Papers citing "Towards Understanding the Spectral Bias of Deep Learning"

50 / 129 papers shown

Title
Physics-informed neural network estimation of active material properties in time-dependent cardiac biomechanical models Matthias Höfler Francesco Regazzoni S. Pagani Elias Karabelas Christoph M. Augustin Gundolf Haase Gernot Plank Federica Caforio 29 0 0 06 May 2025
Deep Learning Optimization Using Self-Adaptive Weighted Auxiliary Variables Yaru Liu Yiqi Gu Michael K. Ng ODL 52 0 0 30 Apr 2025
Hadamard product in deep learning: Introduction, Advances and Challenges Grigorios G. Chrysos Yongtao Wu Razvan Pascanu Philip Torr V. Cevher AAML 98 0 0 17 Apr 2025
Representation Learning for Tabular Data: A Comprehensive Survey Jun-Peng Jiang Si-Yang Liu Hao-Run Cai Qile Zhou Han-Jia Ye LMTD 46 0 0 17 Apr 2025
AH-GS: Augmented 3D Gaussian Splatting for High-Frequency Detail Representation Chenyang Xu XingGuo Deng Rui Zhong 34 0 0 28 Mar 2025
Neural Tangent Kernel of Neural Networks with Loss Informed by Differential Operators Weiye Gan Yicheng Li Q. Lin Zuoqiang Shi 39 0 0 14 Mar 2025
Do ImageNet-trained models learn shortcuts? The impact of frequency shortcuts on generalization Shunxin Wang Raymond N. J. Veldhuis N. Strisciuglio VLM 71 0 0 05 Mar 2025
On the study of frequency control and spectral bias in Wavelet-Based Kolmogorov Arnold networks: A path to physics-informed KANs Juan Daniel Meshir Abel Palafox Edgar Alejandro Guerrero 62 3 0 01 Feb 2025
SNeRV: Spectra-preserving Neural Representation for Video Jina Kim Jihoo Lee Je-Won Kang 35 3 0 03 Jan 2025
Addressing Spectral Bias of Deep Neural Networks by Multi-Grade Deep Learning Ronglong Fang Yuesheng Xu 24 3 0 21 Oct 2024
Fast Training of Sinusoidal Neural Fields via Scaling Initialization Taesun Yeom Sangyoon Lee Jaeho Lee 53 2 0 07 Oct 2024
Tuning Frequency Bias of State Space Models Annan Yu Dongwei Lyu S. H. Lim Michael W. Mahoney N. Benjamin Erichson 38 2 0 02 Oct 2024
Neural Video Representation for Redundancy Reduction and Consistency Preservation Taiga Hayami Takahiro Shindo Shunsuke Akamatsu Hiroshi Watanabe 34 1 0 27 Sep 2024
Overfitting Behaviour of Gaussian Kernel Ridgeless Regression: Varying Bandwidth or Dimensionality Marko Medvedev Gal Vardi Nathan Srebro 56 3 0 05 Sep 2024
Many Perception Tasks are Highly Redundant Functions of their Input Data Rahul Ramesh Anthony Bisulco Ronald W. DiTullio Linran Wei Vijay Balasubramanian Kostas Daniilidis Pratik Chaudhari 38 2 0 18 Jul 2024
Deep Learning without Global Optimization by Random Fourier Neural Networks Owen Davis Gianluca Geraci Mohammad Motamed BDL 52 0 0 16 Jul 2024
Fine-grained Analysis of In-context Linear Estimation: Data, Architecture, and Beyond Yingcong Li A. S. Rawat Samet Oymak 23 6 0 13 Jul 2024
Model-based learning for multi-antenna multi-frequency location-to-channel mapping Baptiste Chatelier Vincent Corlay M. Crussiére Luc Le Magoarou 31 1 0 17 Jun 2024
Forgetting Order of Continual Learning: Examples That are Learned First are Forgotten Last Guy Hacohen Tinne Tuytelaars 19 2 0 14 Jun 2024
VS-PINN: A fast and efficient training of physics-informed neural networks using variable-scaling methods for solving PDEs with stiff behavior Seungchan Ko Sang Hyeon Park 35 1 0 10 Jun 2024
Physics-enhanced Neural Operator for Simulating Turbulent Transport Shengyu Chen P. Givi Can Zheng Xiaowei Jia AI4CE 36 2 0 31 May 2024
Can the accuracy bias by facial hairstyle be reduced through balancing the training data? Kagan Öztürk Haiyu Wu Kevin W. Bowyer CVBM 25 3 0 30 May 2024
A rationale from frequency perspective for grokking in training neural network Zhangchen Zhou Yaoyu Zhang Z. Xu 38 2 0 24 May 2024
Understanding the dynamics of the frequency bias in neural networks Juan Molina Mircea Petrache F. Sahli Costabal Matías Courdurier 27 1 0 23 May 2024
Bounds for the smallest eigenvalue of the NTK for arbitrary spherical data of arbitrary dimension Kedar Karhadkar Michael Murray Guido Montúfar 32 2 0 23 May 2024
Loss Jump During Loss Switch in Solving PDEs with Neural Networks Zhiwei Wang Lulu Zhang Zhongwang Zhang Z. Xu 29 0 0 06 May 2024
FastVPINNs: Tensor-Driven Acceleration of VPINNs for Complex Geometries T. Anandh Divij Ghose Himanshu Jain Sashikumaar Ganesan 22 4 0 18 Apr 2024
Robust NAS under adversarial training: benchmark, theory, and beyond Yongtao Wu Fanghui Liu Carl-Johann Simon-Gabriel Grigorios G. Chrysos V. Cevher AAML OOD 27 3 0 19 Mar 2024
Physics-informed MeshGraphNets (PI-MGNs): Neural finite element solvers for non-stationary and nonlinear simulations on arbitrary meshes Tobias Würth Niklas Freymuth C. Zimmerling Gerhard Neumann Luise Kärger AI4CE 24 1 0 16 Feb 2024
Efficient Stagewise Pretraining via Progressive Subnetworks Abhishek Panigrahi Nikunj Saunshi Kaifeng Lyu Sobhan Miryoosefi Sashank J. Reddi Satyen Kale Sanjiv Kumar 32 5 0 08 Feb 2024
Towards Understanding Inductive Bias in Transformers: A View From Infinity Itay Lavie Guy Gur-Ari Z. Ringel 32 1 0 07 Feb 2024
A Novel Paradigm in Solving Multiscale Problems Jing Wang Zheng Li Pengyu Lai Rui Wang Di Yang Dewu Yang Hui Xu Wenquan Tao AI4CE 14 0 0 07 Feb 2024
Comparing Spectral Bias and Robustness For Two-Layer Neural Networks: SGD vs Adaptive Random Fourier Features Aku Kammonen Lisi Liang Anamika Pandey Raúl Tempone 26 2 0 01 Feb 2024
Anchor function: a type of benchmark functions for studying language models Zhongwang Zhang Zhiwei Wang Junjie Yao Zhangchen Zhou Xiaolong Li E. Weinan Z. Xu 32 5 0 16 Jan 2024
A Survey on Statistical Theory of Deep Learning: Approximation, Training Dynamics, and Generative Models Namjoon Suh Guang Cheng MedIm 22 12 0 14 Jan 2024
Physics-Informed Neural Networks for High-Frequency and Multi-Scale Problems using Transfer Learning Abdul Hannan Mustajab Hao Lyu Z. Rizvi Frank Wuttke AI4CE PINN 18 9 0 05 Jan 2024
Generalization in Kernel Regression Under Realistic Assumptions Daniel Barzilai Ohad Shamir 29 14 0 26 Dec 2023
Model-based Deep Learning for Beam Prediction based on a Channel Chart Taha Yassine Baptiste Chatelier Vincent Corlay M. Crussiére S. Paquelet Olav Tirkkonen Luc Le Magoarou 22 5 0 04 Dec 2023
Spectral-wise Implicit Neural Representation for Hyperspectral Image Reconstruction Huan Chen Wangcai Zhao Tingfa Xu Shiyun Zhou Peifu Liu Jianan Li 44 20 0 02 Dec 2023
Frequency Domain-based Dataset Distillation DongHyeok Shin Seungjae Shin Il-Chul Moon DD 35 19 0 15 Nov 2023
Harnessing Synthetic Datasets: The Role of Shape Bias in Deep Neural Network Generalization Elior Benarous Sotiris Anagnostidis Luca Biggio Thomas Hofmann 25 3 0 10 Nov 2023
Neural Tangent Kernels Motivate Graph Neural Networks with Cross-Covariance Graphs Shervin Khalafi Saurabh Sihag Alejandro Ribeiro 11 0 0 16 Oct 2023
How Graph Neural Networks Learn: Lessons from Training Dynamics Chenxiao Yang Qitian Wu David Wipf Ruoyu Sun Junchi Yan AI4CE GNN 19 1 0 08 Oct 2023
Model-based learning for location-to-channel mapping Baptiste Chatelier Luc Le Magoarou Vincent Corlay M. Crussiére 21 4 0 28 Aug 2023
Six Lectures on Linearized Neural Networks Theodor Misiakiewicz Andrea Montanari 34 12 0 25 Aug 2023
An Expert's Guide to Training Physics-informed Neural Networks Sifan Wang Shyam Sankaran Hanwen Wang P. Perdikaris PINN 28 96 0 16 Aug 2023
Controlling the Inductive Bias of Wide Neural Networks by Modifying the Kernel's Spectrum Amnon Geifman Daniel Barzilai Ronen Basri Meirav Galun 24 4 0 26 Jul 2023
SPDER: Semiperiodic Damping-Enabled Object Representation Kathan Shah Chawin Sitawarin 19 2 0 27 Jun 2023
Neural Volumetric Reconstruction for Coherent Synthetic Aperture Sonar Albert W. Reed Juhyeon Kim Thomas E. Blanford Adithya Pediredla Daniel C. Brown Suren Jayasuriya 19 16 0 16 Jun 2023
Understanding and Mitigating Extrapolation Failures in Physics-Informed Neural Networks Lukas Fesser Luca DÁmico-Wong Richard Qiu 28 4 0 15 Jun 2023