Does Preprocessing Help Training Over-parameterized Neural Networks?

9 October 2021

Papers citing "Does Preprocessing Help Training Over-parameterized Neural Networks?"

32 / 32 papers shown

Title
Video Latent Flow Matching: Optimal Polynomial Projections for Video Interpolation and Extrapolation Yang Cao Zhao-quan Song Chiwun Yang VGen 46 2 0 01 Feb 2025
Fast Gradient Computation for RoPE Attention in Almost Linear Time Yifang Chen Jiayan Huo Xiaoyu Li Yingyu Liang Zhenmei Shi Zhao-quan Song 61 11 0 03 Jan 2025
HSR-Enhanced Sparse Attention Acceleration Bo Chen Yingyu Liang Zhizhou Sha Zhenmei Shi Zhao-quan Song 89 18 0 14 Oct 2024
The Fine-Grained Complexity of Gradient Computation for Training Large Language Models Josh Alman Zhao-quan Song 19 12 0 07 Feb 2024
Fast Heavy Inner Product Identification Between Weights and Inputs in Neural Network Training Lianke Qin Saayan Mitra Zhao-quan Song Yuanyuan Yang Tianyi Zhou 27 0 0 19 Nov 2023
An Automatic Learning Rate Schedule Algorithm for Achieving Faster Convergence and Steeper Descent Zhao-quan Song Chiwun Yang 27 9 0 17 Oct 2023
Generalizing Medical Image Representations via Quaternion Wavelet Networks Luigi Sigillo Eleonora Grassucci A. Uncini Danilo Comminiello MedIm 25 5 0 16 Oct 2023
A Fast Optimization View: Reformulating Single Layer Attention in LLM Based on Tensor and SVM Trick, and Solving It in Matrix Multiplication Time Yeqi Gao Zhao-quan Song Weixin Wang Junze Yin 18 25 0 14 Sep 2023
Online Adaptive Mahalanobis Distance Estimation Lianke Qin Aravind Reddy Zhao-quan Song 41 1 0 02 Sep 2023
Solving Attention Kernel Regression Problem via Pre-conditioner Zhao-quan Song Junze Yin Licheng Zhang 28 9 0 28 Aug 2023
GradientCoin: A Peer-to-Peer Decentralized Large Language Models Yeqi Gao Zhao-quan Song Junze Yin 21 18 0 21 Aug 2023
Fast Quantum Algorithm for Attention Computation Yeqi Gao Zhao-quan Song Xin Yang Ruizhe Zhang LRM 31 19 0 16 Jul 2023
Efficient SGD Neural Network Training via Sublinear Activated Neuron Identification Lianke Qin Zhao-quan Song Yuanyuan Yang 20 9 0 13 Jul 2023
InfoPrompt: Information-Theoretic Soft Prompt Tuning for Natural Language Understanding Junda Wu Tong Yu Rui Wang Zhao-quan Song Ruiyi Zhang Handong Zhao Chaochao Lu Shuai Li Ricardo Henao VLM 31 22 0 08 Jun 2023
Efficient Alternating Minimization with Applications to Weighted Low Rank Approximation Zhao-quan Song Mingquan Ye Junze Yin Licheng Zhang 24 7 0 07 Jun 2023
Federated Empirical Risk Minimization via Second-Order Method S. Bian Zhao-quan Song Junze Yin FedML 33 8 0 27 May 2023
Efficient Asynchronize Stochastic Gradient Algorithm with Structured Data Zhao-quan Song Mingquan Ye 22 4 0 13 May 2023
An Over-parameterized Exponential Regression Yeqi Gao Sridhar Mahadevan Zhao-quan Song 16 35 0 29 Mar 2023
A General Algorithm for Solving Rank-one Matrix Sensing Lianke Qin Zhao-quan Song Ruizhe Zhang 25 15 0 22 Mar 2023
Pruning Before Training May Improve Generalization, Provably Hongru Yang Yingbin Liang Xiaojie Guo Lingfei Wu Zhangyang Wang MLT 19 1 0 01 Jan 2023
Adaptive and Dynamic Multi-Resolution Hashing for Pairwise Summations Lianke Qin Aravind Reddy Zhao-quan Song Zhaozhuo Xu Danyang Zhuo 20 13 0 21 Dec 2022
Bypass Exponential Time Preprocessing: Fast Neural Network Training via Weight-Data Correlation Preprocessing Josh Alman Jiehao Liang Zhao-quan Song Ruizhe Zhang Danyang Zhuo 71 31 0 25 Nov 2022
Sketching for First Order Method: Efficient Algorithm for Low-Bandwidth Channel and Vulnerability Zhao-quan Song Yitan Wang Zheng Yu Licheng Zhang FedML 23 28 0 15 Oct 2022
Dynamic Tensor Product Regression Aravind Reddy Zhao-quan Song Licheng Zhang 37 20 0 08 Oct 2022
A Sublinear Adversarial Training Algorithm Yeqi Gao Lianke Qin Zhao-quan Song Yitan Wang GAN 24 25 0 10 Aug 2022
Training Overparametrized Neural Networks in Sublinear Time Yichuan Deng Han Hu Zhao-quan Song Omri Weinstein Danyang Zhuo BDL 16 28 0 09 Aug 2022
Federated Adversarial Learning: A Framework with Convergence Analysis Xiaoxiao Li Zhao-quan Song Jiaming Yang FedML 27 19 0 07 Aug 2022
Bounding the Width of Neural Networks via Coupled Initialization -- A Worst Case Analysis Alexander Munteanu Simon Omlor Zhao-quan Song David P. Woodruff 27 15 0 26 Jun 2022
Training Multi-Layer Over-Parametrized Neural Network in Subquadratic Time Zhao-quan Song Licheng Zhang Ruizhe Zhang 23 63 0 14 Dec 2021
Fast Graph Neural Tangent Kernel via Kronecker Sketching Shunhua Jiang Yunze Man Zhao-quan Song Zheng Yu Danyang Zhuo 23 6 0 04 Dec 2021
Breaking the Linear Iteration Cost Barrier for Some Well-known Conditional Gradient Methods Using MaxIP Data-structures Anshumali Shrivastava Zhao-quan Song Zhaozhuo Xu 25 28 0 30 Nov 2021
Linear Bandit Algorithms with Sublinear Time Complexity Shuo Yang Tongzheng Ren Sanjay Shakkottai Eric Price Inderjit S. Dhillon Sujay Sanghavi 21 13 0 03 Mar 2021