TinyMIM: An Empirical Study of Distilling MIM Pre-trained Models

TinyMIM: An Empirical Study of Distilling MIM Pre-trained Models

3 January 2023

Zheng-Wei Zhang

Papers citing "TinyMIM: An Empirical Study of Distilling MIM Pre-trained Models"

17 / 17 papers shown

Title
UNIP: Rethinking Pre-trained Attention Patterns for Infrared Semantic Segmentation Tao Zhang Jinyong Wen Zhen Chen Kun Ding S. Xiang Chunhong Pan 72 1 0 04 Feb 2025
How Well Do Supervised 3D Models Transfer to Medical Imaging Tasks? Wenxuan Li Alan L. Yuille Zongwei Zhou MedIm 41 8 0 20 Jan 2025
Dataset Distillation via Factorization Songhua Liu Kai Wang Xingyi Yang Jingwen Ye Xinchao Wang DD 124 141 0 30 Oct 2022
MobileViTv3: Mobile-Friendly Vision Transformer with Simple and Effective Fusion of Local, Global and Input Features S. Wadekar Abhishek Chaurasia ViT 93 85 0 30 Sep 2022
Contrastive Learning Rivals Masked Image Modeling in Fine-tuning via Feature Distillation Yixuan Wei Han Hu Zhenda Xie Zheng-Wei Zhang Yue Cao Jianmin Bao Dong Chen B. Guo CLIP 80 123 0 27 May 2022
Revealing the Dark Secrets of Masked Image Modeling Zhenda Xie Zigang Geng Jingcheng Hu Zheng-Wei Zhang Han Hu Yue Cao VLM 186 105 0 26 May 2022
Masked Autoencoders Are Scalable Vision Learners Kaiming He Xinlei Chen Saining Xie Yanghao Li Piotr Dollár Ross B. Girshick ViT TPM 258 7,337 0 11 Nov 2021
MobileViT: Light-weight, General-purpose, and Mobile-friendly Vision Transformer Sachin Mehta Mohammad Rastegari ViT 189 1,200 0 05 Oct 2021
Mobile-Former: Bridging MobileNet and Transformer Yinpeng Chen Xiyang Dai Dongdong Chen Mengchen Liu Xiaoyi Dong Lu Yuan Zicheng Liu ViT 172 474 0 12 Aug 2021
Unifying Global-Local Representations in Salient Object Detection with Transformer Sucheng Ren Qiang Wen Nanxuan Zhao Guoqiang Han Shengfeng He ViT 21 25 0 05 Aug 2021
Emerging Properties in Self-Supervised Vision Transformers Mathilde Caron Hugo Touvron Ishan Misra Hervé Jégou Julien Mairal Piotr Bojanowski Armand Joulin 286 5,723 0 29 Apr 2021
Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions Wenhai Wang Enze Xie Xiang Li Deng-Ping Fan Kaitao Song Ding Liang Tong Lu Ping Luo Ling Shao ViT 263 3,604 0 24 Feb 2021
Zero-Shot Text-to-Image Generation Aditya A. Ramesh Mikhail Pavlov Gabriel Goh Scott Gray Chelsea Voss Alec Radford Mark Chen Ilya Sutskever VLM 253 4,735 0 24 Feb 2021
SEED: Self-supervised Distillation For Visual Representation Zhiyuan Fang Jianfeng Wang Lijuan Wang Lei Zhang Yezhou Yang Zicheng Liu SSL 231 186 0 12 Jan 2021
MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications Andrew G. Howard Menglong Zhu Bo Chen Dmitry Kalenichenko Weijun Wang Tobias Weyand M. Andreetto Hartwig Adam 3DH 948 20,471 0 17 Apr 2017
Semantic Understanding of Scenes through the ADE20K Dataset Bolei Zhou Hang Zhao Xavier Puig Tete Xiao Sanja Fidler Adela Barriuso Antonio Torralba SSeg 249 1,821 0 18 Aug 2016
ImageNet Large Scale Visual Recognition Challenge Olga Russakovsky Jia Deng Hao Su J. Krause S. Satheesh ... A. Karpathy A. Khosla Michael S. Bernstein Alexander C. Berg Li Fei-Fei VLM ObjD 282 39,170 0 01 Sep 2014