KVT: k-NN Attention for Boosting Vision Transformers

28 May 2021

Hao Li

Papers citing "KVT: k-NN Attention for Boosting Vision Transformers"

50 / 69 papers shown

Title
Progressive Focused Transformer for Single Image Super-Resolution Wei Long Xingyu Zhou Leheng Zhang Shuhang Gu ViT 68 0 0 26 Mar 2025
Iterative Optimal Attention and Local Model for Single Image Rain Streak Removal Xiangyu Li Wanshu Fan Yue Shen C. Wang Wei-Qun Wang X. Yang Qiang Zhang D. Zhou 45 0 0 20 Mar 2025
Fraesormer: Learning Adaptive Sparse Transformer for Efficient Food Recognition Shun Zou Yi Zou Mingya Zhang Shipeng Luo Zhihao Chen Guangwei Gao ViT 38 0 0 15 Mar 2025
$k$ NN Attention Demystified: A Theoretical Exploration for Scalable Transformers Themistoklis Haris 26 0 0 06 Nov 2024
LevAttention: Time, Space, and Streaming Efficient Algorithm for Heavy Attentions R. Kannan Chiranjib Bhattacharyya Praneeth Kacham David P. Woodruff 18 0 0 07 Oct 2024
Genetic Information Analysis of Age-Related Macular Degeneration Fellow Eye Using Multi-Modal Selective ViT Yoichi Furukawa Satoshi Kamiya Yoichi Sakurada Kenji Kashiwagi Kazuhiro Hotta 16 0 0 17 Sep 2024
Hierarchical Separable Video Transformer for Snapshot Compressive Imaging Ping Wang Yulun Zhang Lishun Wang Xin Yuan ViT 24 1 0 16 Jul 2024
PADRe: A Unifying Polynomial Attention Drop-in Replacement for Efficient Vision Transformer Pierre-David Létourneau Manish Kumar Singh Hsin-Pai Cheng Shizhong Han Yunxiao Shi Dalton Jones M. H. Langston Hong Cai Fatih Porikli 26 0 0 16 Jul 2024
Looking 3D: Anomaly Detection with 2D-3D Alignment A. Bhunia Changjian Li Hakan Bilen 24 3 0 27 Jun 2024
MultiMax: Sparse and Multi-Modal Attention Learning Yuxuan Zhou Mario Fritz M. Keuper 33 1 0 03 Jun 2024
LeMeViT: Efficient Vision Transformer with Learnable Meta Tokens for Remote Sensing Image Interpretation Wentao Jiang Jing Zhang Di Wang Qiming Zhang Zengmao Wang Bo Du 22 5 0 16 May 2024
Frequency Decoupling for Motion Magnification via Multi-Level Isomorphic Architecture Fei Wang Dan Guo Kun Li Zhun Zhong Mengqing Wang 26 16 0 12 Mar 2024
Segmentation Guided Sparse Transformer for Under-Display Camera Image Restoration Jingyun Xue Tao Wang Jun Wang Kaihao Zhang ViT 39 2 0 09 Mar 2024
PromptCharm: Text-to-Image Generation through Multi-modal Prompting and Refinement Zhijie Wang Yuheng Huang Da Song Lei Ma Tianyi Zhang DiffM 32 3 0 06 Mar 2024
A Survey on Transformer Compression Yehui Tang Yunhe Wang Jianyuan Guo Zhijun Tu Kai Han Hailin Hu Dacheng Tao 21 26 0 05 Feb 2024
A Manifold Representation of the Key in Vision Transformers Li Meng Morten Goodwin Anis Yazidi P. Engelstad 13 0 0 01 Feb 2024
Prompt-In-Prompt Learning for Universal Image Restoration Zilong Li Yiming Lei Chenglong Ma Junping Zhang Hongming Shan VLM 30 25 0 08 Dec 2023
EulerMormer: Robust Eulerian Motion Magnification via Dynamic Filtering within Transformer Fei Wang Dan Guo Kun Li Mengqing Wang 20 24 0 07 Dec 2023
One Fits All: Universal Time Series Analysis by Pretrained LM and Specially Designed Adaptors Tian Zhou Peisong Niu Xue Wang Liang Sun Rong Jin AI4TS 65 2 0 24 Nov 2023
Linear Log-Normal Attention with Unbiased Concentration Yury Nahshan Dor-Joseph Kampeas E. Haleva 20 7 0 22 Nov 2023
Hourglass Tokenizer for Efficient Transformer-Based 3D Human Pose Estimation Wenhao Li Mengyuan Liu Hong Liu Pichao Wang Jia Cai N. Sebe ViT 3DH 17 10 0 20 Nov 2023
CAIT: Triple-Win Compression towards High Accuracy, Fast Inference, and Favorable Transferability For ViTs Ao Wang Hui Chen Zijia Lin Sicheng Zhao J. Han Guiguang Ding ViT 16 6 0 27 Sep 2023
SCT: A Simple Baseline for Parameter-Efficient Fine-Tuning via Salient Channels Henry Hengyuan Zhao Pichao Wang Yuyang Zhao Hao Luo F. Wang Mike Zheng Shou ViT 19 14 0 15 Sep 2023
Multi-stage Factorized Spatio-Temporal Representation for RGB-D Action and Gesture Recognition Yujun Ma Benjia Zhou Ruili Wang Pichao Wang SLR 11 9 0 23 Aug 2023
Patch Is Not All You Need Chang-bo Li Jie M. Zhang Yang Wei Zhilong Ji Jinfeng Bai Shiguang Shan ViT 36 1 0 21 Aug 2023
Memory-and-Anticipation Transformer for Online Action Understanding Jiahao Wang Guo Chen Yifei Huang Liming Wang Tong Lu OffRL 46 20 0 15 Aug 2023
Graph-Segmenter: Graph Transformer with Boundary-aware Attention for Semantic Segmentation Zizhang Wu Yuanzhu Gan Tianhao Xu Fan Wang ViT 23 8 0 15 Aug 2023
Reciprocal Attention Mixing Transformer for Lightweight Image Restoration Haram Choi Cheolwoong Na Jihyeon Oh Seungjae Lee Jinseop S. Kim Subeen Choe Jeongmin Lee Taehoon Kim Jihoon Yang 41 5 0 19 May 2023
Swap Attention in Spatiotemporal Diffusions for Text-to-Video Generation Wenjing Wang Huan Yang Zixi Tuo Huiguo He Junchen Zhu Jianlong Fu Jiaying Liu DiffM VGen 35 113 0 18 May 2023
FM-ViT: Flexible Modal Vision Transformers for Face Anti-Spoofing Ajian Liu Zichang Tan Zitong Yu Chenxu Zhao Jun Wan Yanyan Liang Zhen Lei Du Zhang Stan Z. Li G. Guo CVBM 15 23 0 05 May 2023
MA-ViT: Modality-Agnostic Vision Transformers for Face Anti-Spoofing Ajian Liu Yanyan Liang 30 48 0 15 Apr 2023
Vision Transformer with Quadrangle Attention Qiming Zhang Jing Zhang Yufei Xu Dacheng Tao ViT 12 38 0 27 Mar 2023
Learning A Sparse Transformer Network for Effective Image Deraining Xiang Chen Hao‐Ran Li Mingqiang Li Jin-shan Pan ViT 20 209 0 21 Mar 2023
All4One: Symbiotic Neighbour Contrastive Learning via Self-Attention and Redundancy Reduction Imanol G. Estepa Ignacio Sarasúa Bhalaji Nagarajan P. Radeva SSL 32 8 0 16 Mar 2023
Revisit Parameter-Efficient Transfer Learning: A Two-Stage Paradigm Hengyuan Zhao Hao Luo Yuyang Zhao Pichao Wang F. Wang Mike Zheng Shou 10 5 0 14 Mar 2023
One Fits All:Power General Time Series Analysis by Pretrained LM Tian Zhou Peisong Niu Xue Wang Liang Sun Rong Jin AI4TS 14 370 0 23 Feb 2023
A Unified Multimodal De- and Re-coupling Framework for RGB-D Motion Recognition Benjia Zhou Pichao Wang Jun Wan Yan-Ni Liang Fan Wang 29 17 0 16 Nov 2022
Data Level Lottery Ticket Hypothesis for Vision Transformers Xuan Shen Zhenglun Kong Minghai Qin Peiyan Dong Geng Yuan Xin Meng Hao Tang Xiaolong Ma Yanzhi Wang 22 4 0 02 Nov 2022
Effective Vision Transformer Training: A Data-Centric Perspective Benjia Zhou Pichao Wang Jun Wan Yan-Ni Liang Fan Wang 16 5 0 29 Sep 2022
ClusTR: Exploring Efficient Self-attention via Clustering for Vision Transformers Yutong Xie Jianpeng Zhang Yong-quan Xia A. Hengel Qi Wu 18 4 0 28 Aug 2022
DynaST: Dynamic Sparse Transformer for Exemplar-Guided Image Generation Songhua Liu Jingwen Ye Sucheng Ren Xinchao Wang 8 44 0 13 Jul 2022
VSA: Learning Varied-Size Window Attention in Vision Transformers Qiming Zhang Yufei Xu Jing Zhang Dacheng Tao 17 50 0 18 Apr 2022
Dynamic Focus-aware Positional Queries for Semantic Segmentation Haoyu He Jianfei Cai Zizheng Pan Jing Liu Jing Zhang Dacheng Tao Bohan Zhuang 20 16 0 04 Apr 2022
Entroformer: A Transformer-based Entropy Model for Learned Image Compression Yichen Qian Ming Lin Xiuyu Sun Zhiyu Tan Rong Jin ViT 6 137 0 11 Feb 2022
VAQF: Fully Automatic Software-Hardware Co-Design Framework for Low-Bit Vision Transformer Mengshu Sun Haoyu Ma Guoliang Kang Yifan Jiang Tianlong Chen Xiaolong Ma Zhangyang Wang Yanzhi Wang ViT 12 45 0 17 Jan 2022
QuadTree Attention for Vision Transformers Shitao Tang Jiahui Zhang Siyu Zhu Ping Tan ViT 148 154 0 08 Jan 2022
SPViT: Enabling Faster Vision Transformers via Soft Token Pruning Zhenglun Kong Peiyan Dong Xiaolong Ma Xin Meng Mengshu Sun ... Geng Yuan Bin Ren Minghai Qin H. Tang Yanzhi Wang ViT 13 141 0 27 Dec 2021
ELSA: Enhanced Local Self-Attention for Vision Transformer Jingkai Zhou Pichao Wang Fan Wang Qiong Liu Hao Li Rong Jin ViT 13 37 0 23 Dec 2021
Decoupling and Recoupling Spatiotemporal Representation for RGB-D-based Motion Recognition Benjia Zhou Pichao Wang Jun Wan Yanyan Liang Fan Wang Du Zhang Zhen Lei Hao Li Rong Jin 25 27 0 16 Dec 2021
Self-slimmed Vision Transformer Zhuofan Zong Kunchang Li Guanglu Song Yali Wang Yu Qiao B. Leng Yu Liu ViT 6 22 0 24 Nov 2021