AutoFormer: Searching Transformers for Visual Recognition

1 July 2021

Papers citing "AutoFormer: Searching Transformers for Visual Recognition"

50 / 155 papers shown

Title
Changes by Butterflies: Farsighted Forecasting with Group Reservoir Transformer Md Kowsher Abdul Rafae Khan Jia Xu 24 0 0 14 Feb 2024
A Survey on Transformer Compression Yehui Tang Yunhe Wang Jianyuan Guo Zhijun Tu Kai Han Hailin Hu Dacheng Tao 29 27 0 05 Feb 2024
ParZC: Parametric Zero-Cost Proxies for Efficient NAS Peijie Dong Lujun Li Xinglin Pan Zimian Wei Xiang Liu Qiang-qiang Wang Xiaowen Chu 45 3 0 03 Feb 2024
Progressive Multi-task Anti-Noise Learning and Distilling Frameworks for Fine-grained Vehicle Recognition Dichao Liu 11 0 0 25 Jan 2024
SuperServe: Fine-Grained Inference Serving for Unpredictable Workloads Alind Khare Dhruv Garg Sukrit Kalra Snigdha Grandhi Ion Stoica Alexey Tumanov 23 3 0 27 Dec 2023
Weight-Entanglement Meets Gradient-Based Neural Architecture Search R. Sukthanker Arjun Krishnakumar Mahmoud Safari Frank Hutter 24 3 0 16 Dec 2023
Foundation Models for Weather and Climate Data Understanding: A Comprehensive Survey Shengchao Chen Guodong Long Jing Jiang Dikai Liu Chengqi Zhang SyDa AI4CE 26 23 0 05 Dec 2023
TVT: Training-Free Vision Transformer Search on Tiny Datasets Zimian Wei H. Pan Lujun Li Peijie Dong Zhiliang Tian Xin-Yi Niu Dongsheng Li ViT 39 7 0 24 Nov 2023
Large Pre-trained time series models for cross-domain Time series analysis tasks Harshavardhan Kamarthi B. A. Prakash VLM AI4TS 15 10 0 19 Nov 2023
PEMS: Pre-trained Epidemic Time-series Models Harshavardhan Kamarthi B. A. Prakash AI4TS 19 2 0 14 Nov 2023
FLORA: Fine-grained Low-Rank Architecture Search for Vision Transformer Chi-Chih Chang Yuan-Yao Sung Shixing Yu N. Huang Diana Marculescu Kai-Chiang Wu ViT 13 1 0 07 Nov 2023
SBCFormer: Lightweight Network Capable of Full-size ImageNet Classification at 1 FPS on Single Board Computers Xiangyong Lu Masanori Suganuma Takayuki Okatani 25 10 0 07 Nov 2023
MCUFormer: Deploying Vision Transformers on Microcontrollers with Limited Memory Yinan Liang Ziwei Wang Xiuwei Xu Yansong Tang Jie Zhou Jiwen Lu 21 9 0 25 Oct 2023
No Token Left Behind: Efficient Vision Transformer via Dynamic Token Idling Xuwei Xu Changlin Li Yudong Chen Xiaojun Chang Jiajun Liu Sen Wang ViT 10 5 0 09 Oct 2023
Plug n' Play: Channel Shuffle Module for Enhancing Tiny Vision Transformers Xuwei Xu Sen Wang Yudong Chen Jiajun Liu ViT 15 1 0 09 Oct 2023
The Inhibitor: ReLU and Addition-Based Attention for Efficient Transformers Rickard Brannvall 16 0 0 03 Oct 2023
Evolutionary Neural Architecture Search for Transformer in Knowledge Tracing Shangshang Yang Xiaoshan Yu Ye Tian Xueming Yan Haiping Ma Xingyi Zhang ViT KELM AI4Ed 11 2 0 02 Oct 2023
On Sparse Modern Hopfield Model Jerry Yao-Chieh Hu Donglin Yang Dennis Wu Chenwei Xu Bo-Yu Chen Han Liu VLM 28 28 0 22 Sep 2023
DeViT: Decomposing Vision Transformers for Collaborative Inference in Edge Devices Guanyu Xu Zhiwei Hao Yong Luo Han Hu J. An Shiwen Mao ViT 30 13 0 10 Sep 2023
GrowCLIP: Data-aware Automatic Model Growing for Large-scale Contrastive Language-Image Pre-training Xi Deng Han Shi Runhu Huang Changlin Li Hang Xu Jianhua Han James T. Kwok Shen Zhao Wei Zhang Xiaodan Liang CLIP VLM 29 3 0 22 Aug 2023
ShiftNAS: Improving One-shot NAS via Probability Shift Mingyang Zhang Xinyi Yu Haodong Zhao Linlin Ou 17 5 0 17 Jul 2023
A Survey of Techniques for Optimizing Transformer Inference Krishna Teja Chitty-Venkata Sparsh Mittal M. Emani V. Vishwanath Arun Somani 29 62 0 16 Jul 2023
Neural Fine-Tuning Search for Few-Shot Learning Panagiotis Eustratiadis L. Dudziak Da Li Timothy M. Hospedales 23 3 0 15 Jun 2023
AutoML in the Age of Large Language Models: Current Challenges, Future Opportunities and Risks Alexander Tornede Difan Deng Theresa Eimer Joseph Giovanelli Aditya Mohan ... Sarah Segel Daphne Theodorakopoulos Tanja Tornede Henning Wachsmuth Marius Lindauer 28 22 0 13 Jun 2023
One-for-All: Generalized LoRA for Parameter-Efficient Fine-tuning Arnav Chavan Zhuang Liu D. K. Gupta Eric P. Xing Zhiqiang Shen 25 87 0 13 Jun 2023
ShiftAddViT: Mixture of Multiplication Primitives Towards Efficient Vision Transformer Haoran You Huihong Shi Yipin Guo Yingyan Lin Lin 26 16 0 10 Jun 2023
FasterViT: Fast Vision Transformers with Hierarchical Attention Ali Hatamizadeh Greg Heinrich Hongxu Yin Andrew Tao J. Álvarez Jan Kautz Pavlo Molchanov ViT 12 66 0 09 Jun 2023
Auto-Spikformer: Spikformer Architecture Search Kaiwei Che Zhaokun Zhou Zhengyu Ma Wei Fang Yanqing Chen Shuaijie Shen Liuliang Yuan Yonghong Tian 21 4 0 01 Jun 2023
MixFormerV2: Efficient Fully Transformer Tracking Yutao Cui Tian-Shu Song Gangshan Wu Liming Wang 21 53 0 25 May 2023
EfficientViT: Memory Efficient Vision Transformer with Cascaded Group Attention Xinyu Liu Houwen Peng Ningxin Zheng Yuqing Yang Han Hu Yixuan Yuan ViT 20 273 0 11 May 2023
Consolidator: Mergeable Adapter with Grouped Connections for Visual Adaptation Tianxiang Hao Hui Chen Yuchen Guo Guiguang Ding 42 16 0 30 Apr 2023
PVP: Pre-trained Visual Parameter-Efficient Tuning Zhao-quan Song Ke Yang Naiyang Guan Junjie Zhu Peng Qiao Qingyong Hu VPVLM VLM 27 3 0 26 Apr 2023
AutoTaskFormer: Searching Vision Transformers for Multi-task Learning Yang Liu Shen Yan Yuge Zhang Kan Ren Quan Zhang Zebin Ren Deng Cai Mi Zhang ViT 6 0 0 18 Apr 2023
Efficient Multimodal Fusion via Interactive Prompting Yaowei Li Ruijie Quan Linchao Zhu Yezhou Yang 28 43 0 13 Apr 2023
ElegansNet: a brief scientific report and initial experiments Francesco Bardozzo Andrea Terlizzi Pietro Lio' R. Tagliaferri 19 1 0 06 Apr 2023
Core-Periphery Principle Guided Redesign of Self-Attention in Transformers Xiao-Xing Yu Lu Zhang Haixing Dai Yanjun Lyu Lin Zhao Zihao Wu David Liu Tianming Liu Dajiang Zhu GNN 23 9 0 27 Mar 2023
Weight-sharing Supernet for Searching Specialized Acoustic Event Classification Networks Across Device Constraints Guan-Ting Lin Qingming Tang Chieh-Chi Kao Viktor Rozgic Chao Wang 20 0 0 18 Mar 2023
ElasticViT: Conflict-aware Supernet Training for Deploying Fast Vision Transformer on Diverse Mobile Devices Chen Tang Li Lyna Zhang Huiqiang Jiang Jiahang Xu Ting Cao Quanlu Zhang Yuqing Yang Zhi Wang Mao Yang 20 11 0 17 Mar 2023
CrossFormer++: A Versatile Vision Transformer Hinging on Cross-scale Attention Wenxiao Wang Wei Chen Qibo Qiu Long Chen Boxi Wu Binbin Lin Xiaofei He Wei Liu 22 38 0 13 Mar 2023
TFormer: A Transmission-Friendly ViT Model for IoT Devices Zhichao Lu Chuntao Ding Felix Juefei Xu Vishnu Naresh Boddeti Shangguang Wang Yun Yang 19 13 0 15 Feb 2023
A Survey on Efficient Training of Transformers Bohan Zhuang Jing Liu Zizheng Pan Haoyu He Yuetian Weng Chunhua Shen 18 47 0 02 Feb 2023
Fairness-aware Vision Transformer via Debiased Self-Attention Yao Qiang Chengyin Li Prashant Khanduri D. Zhu ViT 64 8 0 31 Jan 2023
Time Series Forecasting via Semi-Asymmetric Convolutional Architecture with Global Atrous Sliding Window Yuanpeng He AI4TS 22 0 0 31 Jan 2023
PIT: Optimization of Dynamic Sparse Deep Learning Models via Permutation Invariant Transformation Ningxin Zheng Huiqiang Jiang Quan Zhang Zhenhua Han Yuqing Yang ... Fan Yang Chengruidong Zhang Lili Qiu Mao Yang Lidong Zhou 32 26 0 26 Jan 2023
Neural Architecture Search: Insights from 1000 Papers Colin White Mahmoud Safari R. Sukthanker Binxin Ru T. Elsken Arber Zela Debadeepta Dey Frank Hutter 3DV AI4CE 30 128 0 20 Jan 2023
Dynamic Grained Encoder for Vision Transformers Lin Song Songyang Zhang Songtao Liu Zeming Li Xuming He Hongbin Sun Jian-jun Sun Nanning Zheng ViT 13 34 0 10 Jan 2023
OVO: One-shot Vision Transformer Search with Online distillation Zimian Wei H. Pan Xin-Yi Niu Dongsheng Li ViT 14 1 0 28 Dec 2022
Rethinking Vision Transformers for MobileNet Size and Speed Yanyu Li Ju Hu Yang Wen Georgios Evangelidis Kamyar Salahi Yanzhi Wang Sergey Tulyakov Jian Ren ViT 21 157 0 15 Dec 2022
NoisyQuant: Noisy Bias-Enhanced Post-Training Activation Quantization for Vision Transformers Yijiang Liu Huanrui Yang Zhen Dong Kurt Keutzer Li Du Shanghang Zhang MQ 29 44 0 29 Nov 2022
Castling-ViT: Compressing Self-Attention via Switching Towards Linear-Angular Attention at Vision Transformer Inference Haoran You Yunyang Xiong Xiaoliang Dai Bichen Wu Peizhao Zhang Haoqi Fan Peter Vajda Yingyan Lin 27 31 0 18 Nov 2022