AutoFormer: Searching Transformers for Visual Recognition

1 July 2021

Papers citing "AutoFormer: Searching Transformers for Visual Recognition"

50 / 155 papers shown

Title
HeatViT: Hardware-Efficient Adaptive Token Pruning for Vision Transformers Peiyan Dong Mengshu Sun Alec Lu Yanyue Xie Li-Yu Daisy Liu ... Xin Meng Z. Li Xue Lin Zhenman Fang Yanzhi Wang ViT 18 56 0 15 Nov 2022
Data Level Lottery Ticket Hypothesis for Vision Transformers Xuan Shen Zhenglun Kong Minghai Qin Peiyan Dong Geng Yuan Xin Meng Hao Tang Xiaolong Ma Yanzhi Wang 30 6 0 02 Nov 2022
Clinically-Inspired Multi-Agent Transformers for Disease Trajectory Forecasting from Multimodal Data Huy Hoang Nguyen Matthew B. Blaschko S. Saarakkala A. Tiulpin MedIm AI4CE 48 15 0 25 Oct 2022
Pareto-aware Neural Architecture Generation for Diverse Computational Budgets Yong Guo Yaofo Chen Yin Zheng Qi Chen P. Zhao Jian Chen Junzhou Huang Mingkui Tan 21 5 0 14 Oct 2022
Expediting Large-Scale Vision Transformer for Dense Prediction without Fine-tuning Weicong Liang Yuhui Yuan Henghui Ding Xiao Luo Weihong Lin Ding Jia Zheng-Wei Zhang Chao Zhang Hanhua Hu 17 25 0 03 Oct 2022
EAPruning: Evolutionary Pruning for Vision Transformers and CNNs Qingyuan Li Bo-Wen Zhang Xiangxiang Chu ViT VLM 11 3 0 01 Oct 2022
NasHD: Efficient ViT Architecture Performance Ranking using Hyperdimensional Computing Dongning Ma Pengfei Zhao Xun Jiao 17 0 0 23 Sep 2022
Neural Architecture Search as Multiobjective Optimization Benchmarks: Problem Formulation and Performance Assessment Zhichao Lu Ran Cheng Yaochu Jin Kay Chen Tan Kalyanmoy Deb 13 52 0 08 Aug 2022
Neural Architecture Search on Efficient Transformers and Beyond Zexiang Liu Dong Li Kaiyue Lu Zhen Qin Weixuan Sun Jiacheng Xu Yiran Zhong 27 19 0 28 Jul 2022
TinyViT: Fast Pretraining Distillation for Small Vision Transformers Kan Wu Jinnian Zhang Houwen Peng Mengchen Liu Bin Xiao Jianlong Fu Lu Yuan ViT 13 242 0 21 Jul 2022
UFO: Unified Feature Optimization Teng Xi Yifan Sun Deli Yu Bi Li Nan Peng ... Haocheng Feng Junyu Han Jingtuo Liu Errui Ding Jingdong Wang 32 10 0 21 Jul 2022
Dynamic Spatial Sparsification for Efficient Vision Transformers and Convolutional Neural Networks Yongming Rao Zuyan Liu Wenliang Zhao Jie Zhou Jiwen Lu ViT 42 36 0 04 Jul 2022
EATFormer: Improving Vision Transformer Inspired by Evolutionary Algorithm Jiangning Zhang Xiangtai Li Yabiao Wang Chengjie Wang Yibo Yang Yong Liu Dacheng Tao ViT 28 32 0 19 Jun 2022
Neural Prompt Search Yuanhan Zhang Kaiyang Zhou Ziwei Liu VPVLM VLM 31 143 0 09 Jun 2022
EfficientFormer: Vision Transformers at MobileNet Speed Yanyu Li Geng Yuan Yang Wen Eric Hu Georgios Evangelidis Sergey Tulyakov Yanzhi Wang Jian Ren ViT 16 343 0 02 Jun 2022
Fast Vision Transformers with HiLo Attention Zizheng Pan Jianfei Cai Bohan Zhuang 28 151 0 26 May 2022
ShiftAddNAS: Hardware-Inspired Search for More Accurate and Efficient Neural Networks Haoran You Baopu Li Huihong Shi Y. Fu Yingyan Lin 37 17 0 17 May 2022
MiniViT: Compressing Vision Transformers with Weight Multiplexing Jinnian Zhang Houwen Peng Kan Wu Mengchen Liu Bin Xiao Jianlong Fu Lu Yuan ViT 16 123 0 14 Apr 2022
A Survey of Supernet Optimization and its Applications: Spatial and Temporal Optimization for Neural Architecture Search Stephen Cha Taehyeon Kim Hayeon Lee Se-Young Yun AI4CE 20 2 0 08 Apr 2022
LightHuBERT: Lightweight and Configurable Speech Representation Learning with Once-for-All Hidden-Unit BERT Rui Wang Qibing Bai Junyi Ao Long Zhou Zhixiang Xiong Zhihua Wei Yu Zhang Tom Ko Haizhou Li 28 61 0 29 Mar 2022
SepViT: Separable Vision Transformer Wei Li Xing Wang Xin Xia Jie Wu Jiashi Li Xuefeng Xiao Min Zheng Shiping Wen ViT 24 39 0 29 Mar 2022
Automated Progressive Learning for Efficient Training of Vision Transformers Changlin Li Bohan Zhuang Guangrun Wang Xiaodan Liang Xiaojun Chang Yi Yang 16 46 0 28 Mar 2022
Training-free Transformer Architecture Search Qinqin Zhou Kekai Sheng Xiawu Zheng Ke Li Xing Sun Yonghong Tian Jie Chen Rongrong Ji ViT 27 46 0 23 Mar 2022
DATA: Domain-Aware and Task-Aware Self-supervised Learning Qing Chang Junran Peng Lingxi Xie Jiajun Sun Hao Yin Qi Tian Zhaoxiang Zhang 26 8 0 17 Mar 2022
STDAN: Deformable Attention Network for Space-Time Video Super-Resolution Hai Wang Xiaoyu Xiang Yapeng Tian Wenming Yang Q. Liao AI4TS 32 13 0 14 Mar 2022
Multi-trial Neural Architecture Search with Lottery Tickets Zimian Wei H. Pan Lujun Li Menglong Lu Xin-Yi Niu Peijie Dong Dongsheng Li ViT 29 0 0 08 Mar 2022
LiteTransformerSearch: Training-free Neural Architecture Search for Efficient Language Models Mojan Javaheripi Gustavo de Rosa Subhabrata Mukherjee S. Shah Tomasz Religa C. C. T. Mendes Sébastien Bubeck F. Koushanfar Debadeepta Dey 18 18 0 04 Mar 2022
Multi-Tailed Vision Transformer for Efficient Inference Yunke Wang Bo Du Wenyuan Wang Chang Xu ViT 206 5 0 03 Mar 2022
Auto-scaling Vision Transformers without Training Wuyang Chen Wei Huang Xianzhi Du Xiaodan Song Zhangyang Wang Denny Zhou ViT 25 23 0 24 Feb 2022
BViT: Broad Attention based Vision Transformer Nannan Li Yaran Chen Weifan Li Zixiang Ding Dong Zhao ViT 17 23 0 13 Feb 2022
Generalized Global Ranking-Aware Neural Architecture Ranker for Efficient Image Classifier Search Bicheng Guo Tao Chen Shibo He Haoyu Liu Lili Xu Peng Ye Jiming Chen 39 13 0 30 Jan 2022
AutoDistil: Few-shot Task-agnostic Neural Architecture Search for Distilling Large Language Models Dongkuan Xu Subhabrata Mukherjee Xiaodong Liu Debadeepta Dey Wenhui Wang Xiang Zhang Ahmed Hassan Awadallah Jianfeng Gao 17 4 0 29 Jan 2022
VAQF: Fully Automatic Software-Hardware Co-Design Framework for Low-Bit Vision Transformer Mengshu Sun Haoyu Ma Guoliang Kang Yifan Jiang Tianlong Chen Xiaolong Ma Zhangyang Wang Yanzhi Wang ViT 20 45 0 17 Jan 2022
Vision Transformer Slimming: Multi-Dimension Searching in Continuous Optimization Space Arnav Chavan Zhiqiang Shen Zhuang Liu Zechun Liu Kwang-Ting Cheng Eric P. Xing ViT 12 70 0 03 Jan 2022
SPViT: Enabling Faster Vision Transformers via Soft Token Pruning Zhenglun Kong Peiyan Dong Xiaolong Ma Xin Meng Mengshu Sun ... Geng Yuan Bin Ren Minghai Qin H. Tang Yanzhi Wang ViT 21 141 0 27 Dec 2021
Searching the Search Space of Vision Transformer Minghao Chen Kan Wu Bolin Ni Houwen Peng Bei Liu Jianlong Fu Hongyang Chao Haibin Ling ViT 14 52 0 29 Nov 2021
MAE-DET: Revisiting Maximum Entropy Principle in Zero-Shot NAS for Efficient Object Detection Zhenhong Sun Ming Lin Xiuyu Sun Zhiyu Tan Hao Li Rong Jin 15 32 0 26 Nov 2021
Pruning Self-attentions into Convolutional Layers in Single Path Haoyu He Jianfei Cai Jing Liu Zizheng Pan Jing Zhang Dacheng Tao Bohan Zhuang ViT 29 40 0 23 Nov 2021
Mesa: A Memory-saving Training Framework for Transformers Zizheng Pan Peng Chen Haoyu He Jing Liu Jianfei Cai Bohan Zhuang 13 20 0 22 Nov 2021
Sliced Recursive Transformer Zhiqiang Shen Zechun Liu Eric P. Xing ViT 17 27 0 09 Nov 2021
Global Vision Transformer Pruning with Hessian-Aware Saliency Huanrui Yang Hongxu Yin Maying Shen Pavlo Molchanov Hai Helen Li Jan Kautz ViT 28 38 0 10 Oct 2021
DS-Net++: Dynamic Weight Slicing for Efficient Inference in CNNs and Transformers Changlin Li Guangrun Wang Bing Wang Xiaodan Liang Zhihui Li Xiaojun Chang 25 9 0 21 Sep 2021
Joint Graph Learning and Matching for Semantic Feature Correspondence He Liu Tao Wang Yidong Li Congyan Lang Yi Jin Haibin Ling 16 20 0 01 Sep 2021
Rethinking and Improving Relative Position Encoding for Vision Transformer Kan Wu Houwen Peng Minghao Chen Jianlong Fu Hongyang Chao ViT 28 324 0 29 Jul 2021
GLiT: Neural Architecture Search for Global and Local Image Transformer Boyu Chen Peixia Li Chuming Li Baopu Li Lei Bai Chen Lin Ming-hui Sun Junjie Yan Wanli Ouyang ViT 17 85 0 07 Jul 2021
ViTAS: Vision Transformer Architecture Search Xiu Su Shan You Jiyang Xie Mingkai Zheng Fei-Yue Wang Chao Qian Changshui Zhang Xiaogang Wang Chang Xu ViT 16 54 0 25 Jun 2021
Less is More: Pay Less Attention in Vision Transformers Zizheng Pan Bohan Zhuang Haoyu He Jing Liu Jianfei Cai ViT 11 81 0 29 May 2021
Visformer: The Vision-friendly Transformer Zhengsu Chen Lingxi Xie Jianwei Niu Xuefeng Liu Longhui Wei Qi Tian ViT 109 208 0 26 Apr 2021
SoT: Delving Deeper into Classification Head for Transformer Jiangtao Xie Rui Zeng Qilong Wang Ziqi Zhou P. Li ViT 26 12 0 22 Apr 2021
BossNAS: Exploring Hybrid CNN-transformers with Block-wisely Self-supervised Neural Architecture Search Changlin Li Tao Tang Guangrun Wang Jiefeng Peng Bing Wang Xiaodan Liang Xiaojun Chang ViT 19 105 0 23 Mar 2021