v1v2 (latest)

Fast Attention Requires Bounded Entries

26 February 2023

Papers citing "Fast Attention Requires Bounded Entries"

21 / 71 papers shown

Title
GradientCoin: A Peer-to-Peer Decentralized Large Language Models Yeqi Gao Zhao Song Junze Yin 89 18 0 21 Aug 2023
Convergence of Two-Layer Regression with Nonlinear Units Yichuan Deng Zhao Song Shenghao Xie 80 7 0 16 Aug 2023
Zero-th Order Algorithm for Softmax Attention Optimization Yichuan Deng Zhihang Li Sridhar Mahadevan Zhao Song 62 14 0 17 Jul 2023
Fast Quantum Algorithm for Attention Computation Yeqi Gao Zhao Song Xin Yang Ruizhe Zhang LRM 81 23 0 16 Jul 2023
Efficient SGD Neural Network Training via Sublinear Activated Neuron Identification Lianke Qin Zhao Song Yuanyuan Yang 59 9 0 13 Jul 2023
H $_2$ O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models Zhenyu Zhang Ying Sheng Dinesh Manocha Tianlong Chen Lianmin Zheng ... Yuandong Tian Christopher Ré Clark W. Barrett Zhangyang Wang Beidi Chen VLM 173 314 0 24 Jun 2023
InfoPrompt: Information-Theoretic Soft Prompt Tuning for Natural Language Understanding Junda Wu Tong Yu Rui Wang Zhao Song Ruiyi Zhang Handong Zhao Chaochao Lu Shuai Li Ricardo Henao VLM 92 25 0 08 Jun 2023
A Mathematical Abstraction for Balancing the Trade-off Between Creativity and Reality in Large Language Models Ritwik Sinha Zhao Song Dinesh Manocha 102 25 0 04 Jun 2023
Federated Empirical Risk Minimization via Second-Order Method S. Bian Zhao Song Junze Yin FedML 85 8 0 27 May 2023
Efficient Asynchronize Stochastic Gradient Algorithm with Structured Data Zhao Song Mingquan Ye 67 4 0 13 May 2023
Differentially Private Attention Computation Yeqi Gao Zhao Song Xin Yang 90 21 0 08 May 2023
An Iterative Algorithm for Rescaled Hyperbolic Functions Regression Yeqi Gao Zhao Song Junze Yin 89 33 0 01 May 2023
The Closeness of In-Context Learning and Weight Shifting for Softmax Regression Shuai Li Zhao Song Yu Xia Tong Yu Dinesh Manocha 84 43 0 26 Apr 2023
Attention Scheme Inspired Softmax Regression Yichuan Deng Zhihang Li Zhao Song 93 43 0 20 Apr 2023
Solving Tensor Low Cycle Rank Approximation Yichuan Deng Yeqi Gao Zhao Song 80 6 0 13 Apr 2023
Randomized and Deterministic Attention Sparsification Algorithms for Over-parameterized Feature Dimension Yichuan Deng Sridhar Mahadevan Zhao Song 54 35 0 10 Apr 2023
An Over-parameterized Exponential Regression Yeqi Gao Sridhar Mahadevan Zhao Song 81 39 0 29 Mar 2023
Solving Regularized Exp, Cosh and Sinh Regression Problems Zhihang Li Zhao Song Dinesh Manocha 88 39 0 28 Mar 2023
Streaming Kernel PCA Algorithm With Small Space Yichuan Deng Zhao Song Zifan Wang Hangke Zhang 114 4 0 08 Mar 2023
Bypass Exponential Time Preprocessing: Fast Neural Network Training via Weight-Data Correlation Preprocessing Josh Alman Jiehao Liang Zhao Song Ruizhe Zhang Danyang Zhuo 136 31 0 25 Nov 2022
Dynamic Maintenance of Kernel Density Estimation Data Structure: From Practice to Theory Jiehao Liang Zhao Song Zhaozhuo Xu Junze Yin Danyang Zhuo OOD 65 4 0 08 Aug 2022