BossNAS: Exploring Hybrid CNN-transformers with Block-wisely Self-supervised Neural Architecture Search

23 March 2021

Xiaodan Liang

Xiaojun Chang

ViT

ArXiv PDF HTML

Papers citing "BossNAS: Exploring Hybrid CNN-transformers with Block-wisely Self-supervised Neural Architecture Search"

50 / 69 papers shown

Title
Transferrable Surrogates in Expressive Neural Architecture Search Spaces Shiwen Qin Gabriela Kadlecová Martin Pilát Shay B. Cohen Roman Neruda Elliot J. Crowley Jovita Lukasik Linus Ericsson AI4CE 74 0 0 17 Apr 2025
Evolutionary algorithms meet self-supervised learning: a comprehensive survey Adriano Vinhas João Correia Penousal Machado SSL SyDa 59 0 0 09 Apr 2025
Flexiffusion: Segment-wise Neural Architecture Search for Flexible Denoising Schedule Hongtao Huang Xiaojun Chang Lina Yao 30 0 0 26 Sep 2024
Efficient Training of Large Vision Models via Advanced Automated Progressive Learning Changlin Li Jiawei Zhang Sihao Lin Zongxin Yang Junwei Liang Xiaodan Liang Xiaojun Chang VLM 21 0 0 06 Sep 2024
Quasar-ViT: Hardware-Oriented Quantization-Aware Architecture Search for Vision Transformers Zhengang Li Alec Lu Yanyue Xie Zhenglun Kong Mengshu Sun ... Peiyan Dong Caiwen Ding Yanzhi Wang Xue Lin Zhenman Fang 26 5 0 25 Jul 2024
einspace: Searching for Neural Architectures from Fundamental Operations Linus Ericsson Miguel Espinosa Chenhongyi Yang Antreas Antoniou Amos Storkey Shay B. Cohen Steven G. McDonagh Elliot J. Crowley 38 1 0 31 May 2024
From Algorithm to Hardware: A Survey on Efficient and Safe Deployment of Deep Neural Networks Xue Geng Zhe Wang Chunyun Chen Qing Xu Kaixin Xu ... Zhenghua Chen M. Aly Jie Lin Min-man Wu Xiaoli Li 33 1 0 09 May 2024
Once for Both: Single Stage of Importance and Sparsity Search for Vision Transformer Compression Hancheng Ye Chong Yu Peng Ye Renqiu Xia Yansong Tang Jiwen Lu Tao Chen Bo-Wen Zhang 48 3 0 23 Mar 2024
Unsupervised Graph Neural Architecture Search with Disentangled Self-supervision Zeyang Zhang Xin Eric Wang Ziwei Zhang Guangyao Shen Shiqi Shen Wenwu Zhu 40 12 0 08 Mar 2024
DNA Family: Boosting Weight-Sharing NAS with Block-Wise Supervisions Guangrun Wang Changlin Li Liuchun Yuan Jiefeng Peng Xiaoyu Xian Xiaodan Liang Xiaojun Chang Liang Lin 38 1 0 02 Mar 2024
Lightweight Diffusion Models with Distillation-Based Block Neural Architecture Search Siao Tang Xin Wang Hong Chen Chaoyu Guan Yansong Tang Wenwu Zhu 18 3 0 08 Nov 2023
MGAS: Multi-Granularity Architecture Search for Trade-Off Between Model Effectiveness and Efficiency Xiaoyun Liu Divya Saxena Jiannong Cao Yuqing Zhao Penghui Ruan 36 0 0 23 Oct 2023
AutoDiffusion: Training-Free Optimization of Time Steps and Architectures for Automated Diffusion Model Acceleration Lijiang Li Huixia Li Xiawu Zheng Jie Wu Xuefeng Xiao Rui Wang Min Zheng Xin Pan Fei Chao R. Ji 77 35 0 19 Sep 2023
Efficient Representation Learning for Healthcare with Cross-Architectural Self-Supervision P. Singh Jacopo Cirrone OOD SSL 19 2 0 19 Aug 2023
ShiftNAS: Improving One-shot NAS via Probability Shift Mingyang Zhang Xinyi Yu Haodong Zhao Linlin Ou 25 5 0 17 Jul 2023
CB-HVTNet: A channel-boosted hybrid vision transformer network for lymphocyte assessment in histopathological images Momina Liaqat Ali Zunaira Rauf Asifullah Khan A. Sohail Rafi Ullah Jeonghwan Gwak MedIm ViT 34 2 0 16 May 2023
Can GPT-4 Perform Neural Architecture Search? Mingkai Zheng Xiu Su Shan You Fei Wang Chao Qian Chang Xu Samuel Albanie ELM 34 57 0 21 Apr 2023
PSLT: A Light-weight Vision Transformer with Ladder Self-Attention and Progressive Shift Gaojie Wu Weishi Zheng Yutong Lu Q. Tian ViT 40 15 0 07 Apr 2023
Visual Dependency Transformers: Dependency Tree Emerges from Reversed Attention Mingyu Ding Yikang Shen Lijie Fan Zhenfang Chen Z. Chen Ping Luo J. Tenenbaum Chuang Gan ViT 77 14 0 06 Apr 2023
Neural Architecture Search: Insights from 1000 Papers Colin White Mahmoud Safari R. Sukthanker Binxin Ru T. Elsken Arber Zela Debadeepta Dey Frank Hutter 3DV AI4CE 32 128 0 20 Jan 2023
A Survey on Self-supervised Learning: Algorithms, Applications, and Future Trends Jie Gui Tuo Chen Jing Zhang Qiong Cao Zhe Sun Haoran Luo Dacheng Tao 29 122 0 13 Jan 2023
HeatViT: Hardware-Efficient Adaptive Token Pruning for Vision Transformers Peiyan Dong Mengshu Sun Alec Lu Yanyue Xie Li-Yu Daisy Liu ... Xin Meng Z. Li Xue Lin Zhenman Fang Yanzhi Wang ViT 26 57 0 15 Nov 2022
SaiT: Sparse Vision Transformers through Adaptive Token Pruning Ling Li D. Thorsley Joseph Hassoun ViT 25 17 0 11 Oct 2022
Searching a High-Performance Feature Extractor for Text Recognition Network Hui Zhang Quanming Yao James T. Kwok X. Bai 26 7 0 27 Sep 2022
MaiT: Leverage Attention Masks for More Efficient Image Transformers Ling Li Ali Shafiee Ardestani Joseph Hassoun 14 1 0 06 Jul 2022
EATFormer: Improving Vision Transformer Inspired by Evolutionary Algorithm Jiangning Zhang Xiangtai Li Yabiao Wang Chengjie Wang Yibo Yang Yong Liu Dacheng Tao ViT 30 32 0 19 Jun 2022
MLP-3D: A MLP-like 3D Architecture with Grouped Time Mixing Zhaofan Qiu Ting Yao Chong-Wah Ngo Tao Mei ViT 22 15 0 13 Jun 2022
ShiftAddNAS: Hardware-Inspired Search for More Accurate and Efficient Neural Networks Haoran You Baopu Li Huihong Shi Y. Fu Yingyan Lin 39 17 0 17 May 2022
Arch-Graph: Acyclic Architecture Relation Predictor for Task-Transferable Neural Architecture Search Minbin Huang Zhijian Huang Changlin Li Xin Chen Hangyang Xu Zhenguo Li Xiaodan Liang 28 18 0 12 Apr 2022
When NAS Meets Trees: An Efficient Algorithm for Neural Architecture Search Guocheng Qian Xuanyang Zhang G. Li Chen Zhao Yukang Chen X. Zhang Bernard Ghanem Jian-jun Sun 37 4 0 11 Apr 2022
A Survey of Supernet Optimization and its Applications: Spatial and Temporal Optimization for Neural Architecture Search Stephen Cha Taehyeon Kim Hayeon Lee Se-Young Yun AI4CE 27 2 0 08 Apr 2022
DaViT: Dual Attention Vision Transformers Mingyu Ding Bin Xiao Noel Codella Ping Luo Jingdong Wang Lu Yuan ViT 30 240 0 07 Apr 2022
SepViT: Separable Vision Transformer Wei Li Xing Wang Xin Xia Jie Wu Jiashi Li Xuefeng Xiao Min Zheng Shiping Wen ViT 26 39 0 29 Mar 2022
Automated Progressive Learning for Efficient Training of Vision Transformers Changlin Li Bohan Zhuang Guangrun Wang Xiaodan Liang Xiaojun Chang Yi Yang 16 46 0 28 Mar 2022
Practical Blind Image Denoising via Swin-Conv-UNet and Data Synthesis K. Zhang Yawei Li Jingyun Liang Jiezhang Cao Yulun Zhang H. Tang Deng-Ping Fan Radu Timofte Luc Van Gool OOD 28 122 0 24 Mar 2022
Beyond Fixation: Dynamic Window Visual Transformer Pengzhen Ren Changlin Li Guangrun Wang Yun Xiao Qing Du Xiaodan Liang Qing Du Xiaodan Liang Xiaojun Chang ViT 20 32 0 24 Mar 2022
One Network Doesn't Rule Them All: Moving Beyond Handcrafted Architectures in Self-Supervised Learning Sharath Girish Debadeepta Dey Neel Joshi Vibhav Vineet S. Shah C. C. T. Mendes Abhinav Shrivastava Yale Song SSL 30 2 0 15 Mar 2022
Multi-Tailed Vision Transformer for Efficient Inference Yunke Wang Bo Du Wenyuan Wang Chang Xu ViT 208 6 0 03 Mar 2022
Auto-scaling Vision Transformers without Training Wuyang Chen Wei Huang Xianzhi Du Xiaodan Song Zhangyang Wang Denny Zhou ViT 25 23 0 24 Feb 2022
BViT: Broad Attention based Vision Transformer Nannan Li Yaran Chen Weifan Li Zixiang Ding Dong Zhao ViT 27 23 0 13 Feb 2022
Searching the Search Space of Vision Transformer Minghao Chen Kan Wu Bolin Ni Houwen Peng Bei Liu Jianlong Fu Hongyang Chao Haibin Ling ViT 16 52 0 29 Nov 2021
BaLeNAS: Differentiable Architecture Search via the Bayesian Learning Rule Miao Zhang Jilin Hu Steven W. Su Shirui Pan Xiaojun Chang B. Yang Gholamreza Haffari OOD 31 15 0 25 Nov 2021
Pruning Self-attentions into Convolutional Layers in Single Path Haoyu He Jianfei Cai Jing Liu Zizheng Pan Jing Zhang Dacheng Tao Bohan Zhuang ViT 29 40 0 23 Nov 2021
Searching for TrioNet: Combining Convolution with Local and Global Self-Attention Huaijin Pi Huiyu Wang Yingwei Li Zizhang Li Alan Yuille ViT 19 3 0 15 Nov 2021
DS-Net++: Dynamic Weight Slicing for Efficient Inference in CNNs and Transformers Changlin Li Guangrun Wang Bing Wang Xiaodan Liang Zhihui Li Xiaojun Chang 25 9 0 21 Sep 2021
Searching for Efficient Multi-Stage Vision Transformers Yi-Lun Liao S. Karaman Vivienne Sze ViT 11 19 0 01 Sep 2021
Pi-NAS: Improving Neural Architecture Search by Reducing Supernet Training Consistency Shift Jiefeng Peng Jiqi Zhang Changlin Li Guangrun Wang Xiaodan Liang Liang Lin AAML OOD 16 16 0 22 Aug 2021
Generic Neural Architecture Search via Regression Yuhong Li Cong Hao Pan Li Jinjun Xiong Deming Chen 24 27 0 04 Aug 2021
CycleMLP: A MLP-like Architecture for Dense Prediction Shoufa Chen Enze Xie Chongjian Ge Runjian Chen Ding Liang Ping Luo 13 231 0 21 Jul 2021
AutoFormer: Searching Transformers for Visual Recognition Minghao Chen Houwen Peng Jianlong Fu Haibin Ling ViT 36 259 0 01 Jul 2021