Sparse Sinkhorn Attention

26 February 2020

Papers citing "Sparse Sinkhorn Attention"

50 / 79 papers shown

Title
Spectral Informed Mamba for Robust Point Cloud Processing Ali Bahri Moslem Yazdanpanah Mehrdad Noori Sahar Dastani Milad Cheraghalikhani David Osowiechi G. A. V. Hakim Farzad Beizaee Ismail ben Ayed Christian Desrosiers Mamba 3DPC 74 0 0 06 Mar 2025
Attention Condensation via Sparsity Induced Regularized Training Eli Sason Darya Frolova Boris Nazarov Felix Goldberd 183 0 0 03 Mar 2025
Enhancing Video Understanding: Deep Neural Networks for Spatiotemporal Analysis Amir Hosein Fadaei M. Dehaqani 45 0 0 11 Feb 2025
ZETA: Leveraging Z-order Curves for Efficient Top-k Attention Qiuhao Zeng Jerry Huang Peng Lu Gezheng Xu Boxing Chen Charles Ling Boyu Wang 49 1 0 24 Jan 2025
Token Pruning using a Lightweight Background Aware Vision Transformer Sudhakar Sah Ravish Kumar Honnesh Rohmetra Ehsan Saboori ViT 26 1 0 12 Oct 2024
Large Language Model Inference Acceleration: A Comprehensive Hardware Perspective Jinhao Li Jiaming Xu Shan Huang Yonghua Chen Wen Li ... Jiayi Pan Li Ding Hao Zhou Yu Wang Guohao Dai 62 16 0 06 Oct 2024
Let the Code LLM Edit Itself When You Edit the Code Zhenyu He Jun Zhang Shengjie Luo Jingjing Xu Z. Zhang Di He KELM 36 0 0 03 Jul 2024
Short-Long Convolutions Help Hardware-Efficient Linear Attention to Focus on Long Sequences Zicheng Liu Siyuan Li Li Wang Zedong Wang Yunfan Liu Stan Z. Li 35 7 0 12 Jun 2024
Segmentation Guided Sparse Transformer for Under-Display Camera Image Restoration Jingyun Xue Tao Wang Jun Wang Kaihao Zhang ViT 51 2 0 09 Mar 2024
Orchid: Flexible and Data-Dependent Convolution for Sequence Modeling Mahdi Karami Ali Ghodsi VLM 48 6 0 28 Feb 2024
Zebra: Extending Context Window with Layerwise Grouped Local-Global Attention Kaiqiang Song Xiaoyang Wang Sangwoo Cho Xiaoman Pan Dong Yu 34 7 0 14 Dec 2023
MCANet: Medical Image Segmentation with Multi-Scale Cross-Axis Attention Hao Shao Quansheng Zeng Qibin Hou Jufeng Yang 51 13 0 14 Dec 2023
Long-MIL: Scaling Long Contextual Multiple Instance Learning for Histopathology Whole Slide Image Analysis Honglin Li Yunlong Zhang Chenglu Zhu Jiatong Cai Sunyi Zheng Lin Yang VLM 35 4 0 21 Nov 2023
Transformer-VQ: Linear-Time Transformers via Vector Quantization Albert Mohwald 31 15 0 28 Sep 2023
Pruning Self-Attention for Zero-Shot Multi-Speaker Text-to-Speech Hyungchan Yoon Changhwan Kim Eunwoo Song Hyun-Wook Yoon Hong-Goo Kang 37 1 0 28 Aug 2023
Which Transformer to Favor: A Comparative Analysis of Efficiency in Vision Transformers Tobias Christian Nauen Sebastián M. Palacio Federico Raue Andreas Dengel 42 3 0 18 Aug 2023
Learning Feature Matching via Matchable Keypoint-Assisted Graph Neural Network Zizhuo Li Jiayi Ma 29 2 0 04 Jul 2023
Training-free Diffusion Model Adaptation for Variable-Sized Text-to-Image Synthesis Zhiyu Jin Xuli Shen Bin Li Xiangyang Xue 24 36 0 14 Jun 2023
CageViT: Convolutional Activation Guided Efficient Vision Transformer Hao Zheng Jinbao Wang Xiantong Zhen H. Chen Jingkuan Song Feng Zheng ViT 20 0 0 17 May 2023
Reconstruct Before Summarize: An Efficient Two-Step Framework for Condensing and Summarizing Meeting Transcripts Haochen Tan Han Wu Wei Shao Xinyun Zhang Mingjie Zhan Zhaohui Hou Ding Liang Linqi Song 41 0 0 13 May 2023
The Role of Global and Local Context in Named Entity Recognition Arthur Amalvy Vincent Labatut Richard Dufour 38 4 0 04 May 2023
Efficiency 360: Efficient Vision Transformers Badri N. Patro Vijay Srinivas Agneeswaran 26 6 0 16 Feb 2023
Efficient Attention via Control Variates Lin Zheng Jianbo Yuan Chong-Jun Wang Lingpeng Kong 34 18 0 09 Feb 2023
Dynamic Grained Encoder for Vision Transformers Lin Song Songyang Zhang Songtao Liu Zeming Li Xuming He Hongbin Sun Jian Sun Nanning Zheng ViT 26 34 0 10 Jan 2023
Convolution-enhanced Evolving Attention Networks Yujing Wang Yaming Yang Zhuowan Li Jiangang Bai Mingliang Zhang Xiangtai Li Jiahao Yu Ce Zhang Gao Huang Yu Tong ViT 27 6 0 16 Dec 2022
Efficient Long Sequence Modeling via State Space Augmented Transformer Simiao Zuo Xiaodong Liu Jian Jiao Denis Xavier Charles Eren Manavoglu Tuo Zhao Jianfeng Gao 125 36 0 15 Dec 2022
Regularized Optimal Transport Layers for Generalized Global Pooling Operations Hongteng Xu Minjie Cheng 36 4 0 13 Dec 2022
Perceiver-VL: Efficient Vision-and-Language Modeling with Iterative Latent Attention Zineng Tang Jaemin Cho Jie Lei Joey Tianyi Zhou VLM 24 9 0 21 Nov 2022
Evade the Trap of Mediocrity: Promoting Diversity and Novelty in Text Generation via Concentrating Attention Wenhao Li Xiaoyuan Yi Jinyi Hu Maosong Sun Xing Xie 29 0 0 14 Nov 2022
Museformer: Transformer with Fine- and Coarse-Grained Attention for Music Generation Botao Yu Peiling Lu Rui Wang Wei Hu Xu Tan Wei Ye Shikun Zhang Tao Qin Tie-Yan Liu MGen 25 55 0 19 Oct 2022
The Devil in Linear Transformer Zhen Qin Xiaodong Han Weixuan Sun Dongxu Li Lingpeng Kong Nick Barnes Yiran Zhong 36 70 0 19 Oct 2022
CAB: Comprehensive Attention Benchmarking on Long Sequence Modeling Jinchao Zhang Shuyang Jiang Jiangtao Feng Lin Zheng Lingpeng Kong 3DV 43 9 0 14 Oct 2022
Hierarchical3D Adapters for Long Video-to-text Summarization Pinelopi Papalampidi Mirella Lapata VGen 29 12 0 10 Oct 2022
Bird-Eye Transformers for Text Generation Models Lei Sha Yuhang Song Yordan Yordanov Tommaso Salvatori Thomas Lukasiewicz 30 0 0 08 Oct 2022
Liquid Structural State-Space Models Ramin Hasani Mathias Lechner Tsun-Hsuan Wang Makram Chahine Alexander Amini Daniela Rus AI4TS 104 95 0 26 Sep 2022
Adaptable Butterfly Accelerator for Attention-based NNs via Hardware and Algorithm Co-design Hongxiang Fan Thomas C. P. Chau Stylianos I. Venieris Royson Lee Alexandros Kouris Wayne Luk Nicholas D. Lane Mohamed S. Abdelfattah 37 57 0 20 Sep 2022
Vision-Centric BEV Perception: A Survey Yuexin Ma Tai Wang Xuyang Bai Huitong Yang Yuenan Hou Yaming Wang Yu Qiao Ruigang Yang Tianyi Zhou Xinge Zhu 56 129 0 04 Aug 2022
Momentum Transformer: Closing the Performance Gap Between Self-attention and Its Linearization T. Nguyen Richard G. Baraniuk Robert M. Kirby Stanley J. Osher Bao Wang 32 9 0 01 Aug 2022
Neural Architecture Search on Efficient Transformers and Beyond Zexiang Liu Dong Li Kaiyue Lu Zhen Qin Weixuan Sun Jiacheng Xu Yiran Zhong 35 19 0 28 Jul 2022
Chefs' Random Tables: Non-Trigonometric Random Features Valerii Likhosherstov K. Choromanski Kumar Avinava Dubey Frederick Liu Tamás Sarlós Adrian Weller 33 17 0 30 May 2022
Temporal Latent Bottleneck: Synthesis of Fast and Slow Processing Mechanisms in Sequence Learning Aniket Didolkar Kshitij Gupta Anirudh Goyal Nitesh B. Gundavarapu Alex Lamb Nan Rosemary Ke Yoshua Bengio AI4CE 118 17 0 30 May 2022
VTP: Volumetric Transformer for Multi-view Multi-person 3D Pose Estimation Yuxing Chen Renshu Gu Ouhan Huang Gangyong Jia 3DH 41 11 0 25 May 2022
Leveraging Locality in Abstractive Text Summarization Yixin Liu Ansong Ni Linyong Nan Budhaditya Deb Chenguang Zhu Ahmed Hassan Awadallah Dragomir R. Radev 30 18 0 25 May 2022
Attention Mechanism in Neural Networks: Where it Comes and Where it Goes Derya Soydaner 3DV 44 149 0 27 Apr 2022
A Call for Clarity in Beam Search: How It Works and When It Stops Jungo Kasai Keisuke Sakaguchi Ronan Le Bras Dragomir R. Radev Yejin Choi Noah A. Smith 26 6 0 11 Apr 2022
Linearizing Transformer with Key-Value Memory Yizhe Zhang Deng Cai 20 5 0 23 Mar 2022
ERNIE-SPARSE: Learning Hierarchical Efficient Transformer Through Regularized Self-Attention Yang Liu Jiaxiang Liu L. Chen Yuxiang Lu Shi Feng Zhida Feng Yu Sun Hao Tian Huancheng Wu Hai-feng Wang 28 9 0 23 Mar 2022
Open-Vocabulary DETR with Conditional Matching Yuhang Zang Wei Li Kaiyang Zhou Chen Huang Chen Change Loy ObjD VLM 27 197 0 22 Mar 2022
Dynamic N:M Fine-grained Structured Sparse Attention Mechanism Zhaodong Chen Yuying Quan Zheng Qu L. Liu Yufei Ding Yuan Xie 33 22 0 28 Feb 2022
cosFormer: Rethinking Softmax in Attention Zhen Qin Weixuan Sun Huicai Deng Dongxu Li Yunshen Wei Baohong Lv Junjie Yan Lingpeng Kong Yiran Zhong 24 212 0 17 Feb 2022