Training (Overparametrized) Neural Networks in Near-Linear Time

20 June 2020

Papers citing "Training (Overparametrized) Neural Networks in Near-Linear Time"

21 / 21 papers shown

Title
Fast Gradient Computation for RoPE Attention in Almost Linear Time Yifang Chen Jiayan Huo Xiaoyu Li Yingyu Liang Zhenmei Shi Zhao-quan Song 57 11 0 03 Jan 2025
How to Protect Copyright Data in Optimization of Large Language Models? T. Chu Zhao-quan Song Chiwun Yang 28 29 0 23 Aug 2023
Efficient SGD Neural Network Training via Sublinear Activated Neuron Identification Lianke Qin Zhao-quan Song Yuanyuan Yang 20 9 0 13 Jul 2023
Learning the Positions in CountSketch Yi Li Honghao Lin Simin Liu A. Vakilian David P. Woodruff 29 18 0 11 Jun 2023
Fast and Efficient Matching Algorithm with Deadline Instances Zhao-quan Song Weixin Wang Chenbo Yin Junze Yin 8 7 0 15 May 2023
$A Nearly-Optimal Bound for Fast Regression with $\ell_\infty$ Guarantee$ A Nearly-Optimal Bound for Fast Regression with $\ell_\infty$ Guarantee Zhao-quan Song Mingquan Ye Junze Yin Licheng Zhang 14 10 0 01 Feb 2023
Bypass Exponential Time Preprocessing: Fast Neural Network Training via Weight-Data Correlation Preprocessing Josh Alman Jiehao Liang Zhao-quan Song Ruizhe Zhang Danyang Zhuo 69 31 0 25 Nov 2022
Sketching for First Order Method: Efficient Algorithm for Low-Bandwidth Channel and Vulnerability Zhao-quan Song Yitan Wang Zheng Yu Licheng Zhang FedML 23 28 0 15 Oct 2022
Bounding the Width of Neural Networks via Coupled Initialization -- A Worst Case Analysis Alexander Munteanu Simon Omlor Zhao-quan Song David P. Woodruff 19 15 0 26 Jun 2022
Training Multi-Layer Over-Parametrized Neural Network in Subquadratic Time Zhao-quan Song Licheng Zhang Ruizhe Zhang 23 63 0 14 Dec 2021
Fast Graph Neural Tangent Kernel via Kronecker Sketching Shunhua Jiang Yunze Man Zhao-quan Song Zheng Yu Danyang Zhuo 19 6 0 04 Dec 2021
Pixelated Butterfly: Simple and Efficient Sparse training for Neural Network Models Tri Dao Beidi Chen Kaizhao Liang Jiaming Yang Zhao-quan Song Atri Rudra Christopher Ré 25 75 0 30 Nov 2021
Breaking the Linear Iteration Cost Barrier for Some Well-known Conditional Gradient Methods Using MaxIP Data-structures Anshumali Shrivastava Zhao-quan Song Zhaozhuo Xu 25 28 0 30 Nov 2021
Dynamic Network-Assisted D2D-Aided Coded Distributed Learning Nikita Zeulin O. Galinina N. Himayat Sergey D. Andreev R. Heath 23 5 0 26 Nov 2021
Does Preprocessing Help Training Over-parameterized Neural Networks? Zhao-quan Song Shuo Yang Ruizhe Zhang 22 49 0 09 Oct 2021
Improved architectures and training algorithms for deep operator networks Sifan Wang Hanwen Wang P. Perdikaris AI4CE 42 104 0 04 Oct 2021
Oblivious sketching for logistic regression Alexander Munteanu Simon Omlor David P. Woodruff UQCV 46 17 0 14 Jul 2021
FL-NTK: A Neural Tangent Kernel-based Framework for Federated Learning Convergence Analysis Baihe Huang Xiaoxiao Li Zhao-quan Song Xin Yang FedML 23 16 0 11 May 2021
Lifelong Learning with Sketched Structural Regularization Haoran Li A. Krishnan Jingfeng Wu Soheil Kolouri Praveen K. Pilly Vladimir Braverman CLL 14 17 0 17 Apr 2021
Quantum-Inspired Algorithms from Randomized Numerical Linear Algebra Nadiia Chepurko K. Clarkson L. Horesh Honghao Lin David P. Woodruff 6 24 0 09 Nov 2020
Generalized Leverage Score Sampling for Neural Networks J. Lee Ruoqi Shen Zhao-quan Song Mengdi Wang Zheng Yu 13 42 0 21 Sep 2020