How to Capture Higher-order Correlations? Generalizing Matrix Softmax Attention to Kronecker Computation

6 October 2023

Papers citing "How to Capture Higher-order Correlations? Generalizing Matrix Softmax Attention to Kronecker Computation"

11 / 11 papers shown

Title
Looped ReLU MLPs May Be All You Need as Practical Programmable Computers Yingyu Liang Zhizhou Sha Zhenmei Shi Zhao-quan Song Yufa Zhou 89 17 0 21 Feb 2025
Fast Gradient Computation for RoPE Attention in Almost Linear Time Yifang Chen Jiayan Huo Xiaoyu Li Yingyu Liang Zhenmei Shi Zhao-quan Song 57 11 0 03 Jan 2025
HSR-Enhanced Sparse Attention Acceleration Bo Chen Yingyu Liang Zhizhou Sha Zhenmei Shi Zhao-quan Song 79 17 0 14 Oct 2024
Differentially Private Kernel Density Estimation Erzhi Liu Jerry Yao-Chieh Hu Alex Reneau Zhao Song Han Liu 56 3 0 03 Sep 2024
When big data actually are low-rank, or entrywise approximation of certain function-generated matrices Stanislav Budzinskiy 51 2 0 03 Jul 2024
Outlier-Efficient Hopfield Layers for Large Transformer-Based Models Jerry Yao-Chieh Hu Pei-Hsuan Chang Haozheng Luo Hong-Yu Chen Weijian Li Wei-Po Wang Han Liu 31 25 0 04 Apr 2024
Uniform Memory Retrieval with Larger Capacity for Modern Hopfield Models Dennis Wu Jerry Yao-Chieh Hu Teng-Yun Hsiao Han Liu 38 28 0 04 Apr 2024
Fast Heavy Inner Product Identification Between Weights and Inputs in Neural Network Training Lianke Qin Saayan Mitra Zhao-quan Song Yuanyuan Yang Tianyi Zhou 24 0 0 19 Nov 2023
Differentially Private Attention Computation Yeqi Gao Zhao-quan Song Xin Yang 42 19 0 08 May 2023
Bypass Exponential Time Preprocessing: Fast Neural Network Training via Weight-Data Correlation Preprocessing Josh Alman Jiehao Liang Zhao-quan Song Ruizhe Zhang Danyang Zhuo 64 32 0 25 Nov 2022
On The Computational Complexity of Self-Attention Feyza Duman Keles Pruthuvi Maheshakya Wijewardena C. Hegde 63 107 0 11 Sep 2022