v1v2v3 (latest)

Nyströmformer: A Nyström-Based Algorithm for Approximating Self-Attention

7 February 2021

Mingxing Tan

Yin Li

ArXiv (abs)PDF HTML Github (376★)

Papers citing "Nyströmformer: A Nyström-Based Algorithm for Approximating Self-Attention"

50 / 146 papers shown

Title
GeistBERT: Breathing Life into German NLP Raphael Scheible-Schmitt Johann Frei VLM 35 0 0 13 Jun 2025
Revisiting Transformers with Insights from Image Filtering Laziz U. Abdullaev Maksim Tkachenko Tan M. Nguyen ViT 129 0 0 12 Jun 2025
Plug-and-Play Linear Attention for Pre-trained Image and Video Restoration Models Srinivasan Kidambi Pravin Nair 33 0 0 10 Jun 2025
Log-Linear Attention Han Guo Songlin Yang Tarushii Goel Eric P. Xing Tri Dao Yoon Kim Mamba 160 1 0 05 Jun 2025
Leaner Transformers: More Heads, Less Depth Hemanth Saratchandran Damien Teney Simon Lucey 34 0 0 27 May 2025
TuneComp: Joint Fine-tuning and Compression for Large Foundation Models Xiangyu Chen Jing Liu Ye Wang Matthew Brand Wang T. Koike-Akino 64 0 0 27 May 2025
MonarchAttention: Zero-Shot Conversion to Fast, Hardware-Aware Structured Attention Can Yaras Alec S. Xu Pierre Abillama Changwoo Lee Laura Balzano 34 0 0 24 May 2025
Advancing Multiple Instance Learning with Continual Learning for Whole Slide Imaging Xianrui Li Yufei Cui Jun Li Antoni B. Chan CLL VLM 92 0 0 15 May 2025
OLinear: A Linear Model for Time Series Forecasting in Orthogonally Transformed Domain Wenzhen Yue Yang Liu Haoxuan Li Hao Wang Xianghua Ying Ruohao Guo Bowei Xing Ji Shi AI4TS OOD 90 0 0 12 May 2025
Robust Multimodal Survival Prediction with the Latent Differentiation Conditional Variational AutoEncoder Junjie Zhou Jiao Tang Yingli Zuo Peng Wan Daoqiang Zhang Wei Shao 222 1 0 12 Mar 2025
MIRAM: Masked Image Reconstruction Across Multiple Scales for Breast Lesion Risk Prediction H. Q. Vo Pengyu Yuan Zheng Yin Kelvin K. Wong Chika F. Ezeana S. Ly Stephen T. C. Wong H. Nguyen 57 0 0 10 Mar 2025
Union of Experts: Adapting Hierarchical Routing to Equivalently Decomposed Transformer Yujiao Yang Jing Lian Linhui Li MoE 139 0 0 04 Mar 2025
Attention Condensation via Sparsity Induced Regularized Training Eli Sason Darya Frolova Boris Nazarov Felix Goldberd 514 0 0 03 Mar 2025
Foundation-Model-Boosted Multimodal Learning for fMRI-based Neuropathic Pain Drug Response Prediction Wenrui Fan L. M. Riza Rizky Jiayang Zhang Chen Chen Haiping Lu Kevin Teh Dinesh Selvarajah Shuo Zhou 92 0 0 28 Feb 2025
Enhancing Video Understanding: Deep Neural Networks for Spatiotemporal Analysis Amir Hosein Fadaei M. Dehaqani 91 0 0 11 Feb 2025
SparseFormer: Detecting Objects in HRW Shots via Sparse Vision Transformer Wenxi Li Yuchen Guo Jilai Zheng Haozhe Lin Chao Ma Lu Fang Xiaokang Yang ViT 158 5 0 11 Feb 2025
ZETA: Leveraging Z-order Curves for Efficient Top-k Attention Qiuhao Zeng Jerry Huang Peng Lu Gezheng Xu Boxing Chen Charles Ling Boyu Wang 195 3 0 24 Jan 2025
EfficientViM: Efficient Vision Mamba with Hidden State Mixer based State Space Duality Sanghyeok Lee Joonmyung Choi Hyunwoo J. Kim 233 3 0 22 Nov 2024
Breaking the Low-Rank Dilemma of Linear Attention Qihang Fan Huaibo Huang Ran He 111 2 0 12 Nov 2024
Rethinking Transformer for Long Contextual Histopathology Whole Slide Image Analysis Honglin Li Yunlong Zhang Pingyi Chen Zhongyi Shui Chenglu Zhu Lin Yang MedIm 99 5 0 18 Oct 2024
Training Ultra Long Context Language Model with Fully Pipelined Distributed Transformer Jinghan Yao Sam Ade Jacobs Masahiro Tanaka Olatunji Ruwase Hari Subramoni D. Panda 102 2 0 30 Aug 2024
Snuffy: Efficient Whole Slide Image Classifier Hossein Jafarinia Alireza Alipanah Danial Hamdi Saeed Razavi Nahal Mirzaie M. Rohban 3DH 96 2 0 15 Aug 2024
DGR-MIL: Exploring Diverse Global Representation in Multiple Instance Learning for Whole Slide Image Classification Wenhui Zhu Xiwen Chen Peijie Qiu Aristeidis Sotiras Abolfazl Razi Yalin Wang 79 7 0 04 Jul 2024
Fredformer: Frequency Debiased Transformer for Time Series Forecasting Xihao Piao Zheng Chen Taichi Murayama Yasuko Matsubara Yasushi Sakurai AI4TS 85 25 0 13 Jun 2024
Learning in Feature Spaces via Coupled Covariances: Asymmetric Kernel SVD and Nyström method Qinghua Tao F. Tonin Alex Lambert Yingyi Chen Panagiotis Patrinos Johan A. K. Suykens 105 2 0 13 Jun 2024
Spectraformer: A Unified Random Feature Framework for Transformer Duke Nguyen Du Yin Aditya Joshi Flora D. Salim 69 1 0 24 May 2024
Focus on Low-Resolution Information: Multi-Granular Information-Lossless Model for Low-Resolution Human Pose Estimation Zejun Gu Zhongqiu Zhao Hao Shen Zhao Zhang 3DH 90 1 0 19 May 2024
Enhancing Maritime Trajectory Forecasting via H3 Index and Causal Language Modelling (CLM) Nicolas Drapier Aladine Chetouani A. Chateigner 62 3 0 15 May 2024
Mamba-360: Survey of State Space Models as Transformer Alternative for Long Sequence Modelling: Methods, Applications, and Challenges Badri N. Patro Vijay Srinivas Agneeswaran Mamba 116 45 0 24 Apr 2024
iMD4GC: Incomplete Multimodal Data Integration to Advance Precise Treatment Response Prediction and Survival Analysis for Gastric Cancer Fengtao Zhou Ying Xu Yanfen Cui Shenyang Zhang Yun Zhu ... Louis Ho Shing Lau Chu Han Dafu Zhang Zhenhui Li Hao Chen 59 2 0 01 Apr 2024
NoMAD-Attention: Efficient LLM Inference on CPUs Through Multiply-add-free Attention Tianyi Zhang Jonah Yi Bowen Yao Zhaozhuo Xu Anshumali Shrivastava MQ 104 7 0 02 Mar 2024
Feature Re-Embedding: Towards Foundation Model-Level Performance in Computational Pathology Wenhao Tang Fengtao Zhou Shengyue Huang Xiang Zhu Yi Zhang Bo Liu 137 25 0 27 Feb 2024
Hybrid Quantum Vision Transformers for Event Classification in High Energy Physics Eyup B. Unlu Marçal Comajoan Cara Gopal Ramesh Dahale Zhongtian Dong Roy T. Forestano ... Daniel Justice Kyoungchul Kong Tom Magorsch Konstantin T. Matchev Katia Matcheva 94 12 0 01 Feb 2024
CascadedGaze: Efficiency in Global Context Extraction for Image Restoration Amirhosein Ghasemabadi Muhammad Kamran Janjua Mohammad Salameh Chunhua Zhou Fengyu Sun Di Niu 95 12 0 26 Jan 2024
OnDev-LCT: On-Device Lightweight Convolutional Transformers towards federated learning Chu Myaet Thwal Minh N. H. Nguyen Ye Lin Tun Seongjin Kim My T. Thai Choong Seon Hong 125 7 0 22 Jan 2024
Efficient generative adversarial networks using linear additive-attention Transformers Emilio Morales-Juarez Gibran Fuentes Pineda 88 3 0 17 Jan 2024
MIMONets: Multiple-Input-Multiple-Output Neural Networks Exploiting Computation in Superposition Nicolas Menet Michael Hersche G. Karunaratne Luca Benini Abu Sebastian Abbas Rahimi 80 14 0 05 Dec 2023
Fast Multipole Attention: A Divide-and-Conquer Attention Mechanism for Long Sequences Yanming Kang Giang Tran H. Sterck 100 5 0 18 Oct 2023
Which Transformer to Favor: A Comparative Analysis of Efficiency in Vision Transformers Tobias Christian Nauen Sebastián M. Palacio Federico Raue Andreas Dengel 145 4 0 18 Aug 2023
LaFiCMIL: Rethinking Large File Classification from the Perspective of Correlated Multiple Instance Learning Tiezhu Sun Weiguo Pian N. Daoudi Kevin Allix Tegawende F. Bissyande Jacques Klein 126 1 0 30 Jul 2023
Adaptive Frequency Filters As Efficient Global Token Mixers Zhipeng Huang Zhizheng Zhang Cuiling Lan Zhengjun Zha Yan Lu B. Guo 78 45 0 26 Jul 2023
Dual-Query Multiple Instance Learning for Dynamic Meta-Embedding based Tumor Classification Simon Holdenried-Krafft Peter Somers Ivonne A. Montes-Majarro Diana Silimon Cristina Tarín F. Fend Hendrik P. A. Lensch MedIm 102 3 0 14 Jul 2023
When to Use Efficient Self Attention? Profiling Text, Speech and Image Transformer Variants Anuj Diwan Eunsol Choi David Harwath 81 0 0 14 Jun 2023
Primal-Attention: Self-attention through Asymmetric Kernel SVD in Primal Representation Yingyi Chen Qinghua Tao F. Tonin Johan A. K. Suykens 99 22 0 31 May 2023
Recasting Self-Attention with Holographic Reduced Representations Mohammad Mahmudul Alam Edward Raff Stella Biderman Tim Oates James Holt 66 9 0 31 May 2023
A Quantitative Review on Language Model Efficiency Research Meng Jiang Hy Dang Lingbo Tong 76 0 0 28 May 2023
Scalable Transformer for PDE Surrogate Modeling Zijie Li Dule Shu A. Farimani 118 84 0 27 May 2023
Fourier Transformer: Fast Long Range Modeling by Removing Sequence Redundancy with FFT Operator Ziwei He Meng Yang Minwei Feng Jingcheng Yin Xiang Wang Jingwen Leng Zhouhan Lin ViT 97 14 0 24 May 2023
In-Context Learning with Many Demonstration Examples Mukai Li Shansan Gong Jiangtao Feng Yiheng Xu Jinchao Zhang Zhiyong Wu Lingpeng Kong 111 38 0 09 Feb 2023
Continuous Spatiotemporal Transformers Antonio H. O. Fonseca E. Zappala J. O. Caro David van Dijk 78 8 0 31 Jan 2023