Bootstrapped Masked Autoencoders for Vision BERT Pretraining

European Conference on Computer Vision (ECCV), 2022

14 July 2022

Jianmin Bao

Lu Yuan

ArXiv (abs)PDF HTML Github (97★)

Papers citing "Bootstrapped Masked Autoencoders for Vision BERT Pretraining"

50 / 57 papers shown

Title
MINR: Implicit Neural Representations with Masked Image Modelling Sua Lee Joonhun Lee Myungjoo Kang 115 1 0 30 Jul 2025
Uni4D: A Unified Self-Supervised Learning Framework for Point Cloud Videos Zhi Zuo Chenyi Zhuang Zhiqiang Shen Pan Gao Jie Qin Nicu Sebe 3DPC 301 1 0 07 Apr 2025
Linguistics-aware Masked Image Modeling for Self-supervised Scene Text RecognitionComputer Vision and Pattern Recognition (CVPR), 2025 Yifei Zhang Yu Xie Jin Wei Xiaomeng Yang Yu Zhou Can Ma Xiangyang Ji 260 7 0 24 Mar 2025
PiLaMIM: Toward Richer Visual Representations by Integrating Pixel and Latent Masked Image Modeling Junmyeong Lee Eui Jun Hwang Sukmin Cho Jong C. Park 156 0 0 06 Jan 2025
PR-MIM: Delving Deeper into Partial Reconstruction in Masked Image Modeling Zhong-Yu Li Yunheng Li Deng-Ping Fan Ming-Ming Cheng 325 0 0 24 Nov 2024
A Survey of the Self Supervised Learning Mechanisms for Vision Transformers Asifullah Khan A. Sohail Mustansar Fiaz Mehdi Hassan Tariq Habib Afridi ... Muhammad Zaigham Zaheer Kamran Ali Tangina Sultana Ziaurrehman Tanoli Naeem Akhter 887 11 0 30 Aug 2024
CLIP-CID: Efficient CLIP Distillation via Cluster-Instance DiscriminationAAAI Conference on Artificial Intelligence (AAAI), 2024 Kaicheng Yang Tiancheng Gu Xiang An Haiqiang Jiang Xiangzi Dai Ziyong Feng Weidong Cai Jiankang Deng VLM 240 20 0 18 Aug 2024
Masked Image Modeling: A SurveyInternational Journal of Computer Vision (IJCV), 2024 Vlad Hondru Florinel-Alin Croitoru Shervin Minaee Radu Tudor Ionescu Andrii Zadaianchuk 405 17 0 13 Aug 2024
ColorMAE: Exploring data-independent masking strategies in Masked AutoEncoders Carlos Hinojosa Shuming Liu Guohao Li 193 8 0 17 Jul 2024
Self-Supervised Alignment Learning for Medical Image Segmentation Haofeng Li Yiming Ouyang Xiang Wan 170 9 0 22 Jun 2024
SemanticMIM: Marring Masked Image Modeling with Semantics Compression for General Visual Representation Yike Yuan Huanzhang Dou Fengjun Guo Xi Li 206 2 0 15 Jun 2024
Visualizing the loss landscape of Self-supervised Vision Transformer Youngwan Lee Jeffrey Willette Jonghee Kim Sung Ju Hwang ViT 144 1 0 28 May 2024
Efficient Pretraining Model based on Multi-Scale Local Visual Field Feature Reconstruction for PCB CT Image Element SegmentationApplied Optics (Appl. Opt.), 2024 Chen Chen Kai Qiao Jie Yang Jian Chen Bin Yan 121 3 0 09 May 2024
An Experimental Study on Exploring Strong Lightweight Vision Transformers via Masked Image Modeling Pre-Training Jin Gao Shubo Lin Shaoru Wang Yutong Kou Zeming Li Liang Li Congxuan Zhang Xiaoqin Zhang Yizheng Wang Weiming Hu 244 5 0 18 Apr 2024
Emerging Property of Masked Token for Effective Pre-training Hyesong Choi Hunsang Lee Seyoung Joung Hyejin Park Jiyeong Kim Dongbo Min 145 10 0 12 Apr 2024
Salience-Based Adaptive Masking: Revisiting Token Dynamics for Enhanced Pre-training Hyesong Choi Hyejin Park Kwang Moo Yi Sungmin Cha Dongbo Min 226 10 0 12 Apr 2024
Transformer based Pluralistic Image Completion with Reduced Information Loss Qiankun Liu Yuqi Jiang Zhentao Tan DongDong Chen Ying Fu Qi Chu Gang Hua Nenghai Yu ViT 206 22 0 31 Mar 2024
Dynamic Tuning Towards Parameter and Inference Efficiency for ViT AdaptationNeural Information Processing Systems (NeurIPS), 2024 Wangbo Zhao Jiasheng Tang Yizeng Han Yibing Song Kai Wang Gao Huang F. Wang Yang You 288 22 0 18 Mar 2024
Masked AutoDecoder is Effective Multi-Task Vision GeneralistComputer Vision and Pattern Recognition (CVPR), 2024 Han Qiu Jiaxing Huang Shiyang Feng Lewei Lu Xiaoqin Zhang Shijian Lu 179 5 0 12 Mar 2024
Self-Supervised Facial Representation Learning with Facial Region Awareness Zheng Gao Ioannis Patras SSL 185 27 0 04 Mar 2024
Masked Modeling for Self-supervised Representation Learning on Vision and Beyond Siyuan Li Luyuan Zhang Zedong Wang Di Wu Lirong Wu ... Jun Xia Cheng Tan Yang Liu Baigui Sun Stan Z. Li SSL 235 27 0 31 Dec 2023
Morphing Tokens Draw Strong Masked Image ModelsInternational Conference on Learning Representations (ICLR), 2023 Taekyung Kim Byeongho Heo Dongyoon Han 579 3 0 30 Dec 2023
Bootstrap Masked Visual Modeling via Hard Patches Mining Haochen Wang Junsong Fan Yuxi Wang Kaiyou Song Tiancai Wang Xiangyu Zhang Zhaoxiang Zhang 183 6 0 21 Dec 2023
Semantic-Aware Autoregressive Image Modeling for Visual Representation LearningAAAI Conference on Artificial Intelligence (AAAI), 2023 Kaiyou Song Shan Zhang Tong Wang VLM 134 2 0 16 Dec 2023
EfficientSAM: Leveraged Masked Image Pretraining for Efficient Segment AnythingComputer Vision and Pattern Recognition (CVPR), 2023 Yunyang Xiong Bala Varadarajan Lemeng Wu Xiaoyu Xiang Fanyi Xiao ... Dilin Wang Fei Sun Forrest N. Iandola Raghuraman Krishnamoorthi Vikas Chandra VLM 280 230 0 01 Dec 2023
PersonMAE: Person Re-Identification Pre-Training with Masked AutoEncoders Hezhen Hu Xiaoyi Dong Jianmin Bao Dongdong Chen Lu Yuan Dong Chen Houqiang Li 235 11 0 08 Nov 2023
Learning with Unmasked Tokens Drives Stronger Vision Learners Taekyung Kim Sanghyuk Chun Byeongho Heo Dongyoon Han SSL 254 3 0 20 Oct 2023
Enhancing Representations through Heterogeneous Self-Supervised LearningIEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2023 Zhongyu Li Bo-Wen Yin Yongxiang Liu Tianpeng Liu Ming-Ming Cheng SSL 290 3 0 08 Oct 2023
Self-distilled Masked Attention guided masked image modeling with noise Regularized Teacher (SMART) for medical image analysis Jue Jiang Aneesh Rangnekar Chloe Choi Harini Veeraraghavan MedIm 164 1 0 02 Oct 2023
Masked Image Residual Learning for Scaling Deeper Vision TransformersNeural Information Processing Systems (NeurIPS), 2023 Guoxi Huang Hongtao Fu A. Bors 212 8 0 25 Sep 2023
AMLP: Adjustable Masking Lesion Patches for Self-Supervised Medical Image SegmentationIEEE Transactions on Medical Imaging (TMI), 2023 Xiang-Fei Wang Ruizhi Wang Jie Zhou Thomas Lukasiewicz 162 0 0 08 Sep 2023
DropPos: Pre-Training Vision Transformers by Reconstructing Dropped PositionsNeural Information Processing Systems (NeurIPS), 2023 Haochen Wang Junsong Fan Yuxi Wang Kaiyou Song Tong Wang Zhaoxiang Zhang 202 23 0 07 Sep 2023
PUMA: Secure Inference of LLaMA-7B in Five Minutes Ye Dong Wen-jie Lu Yancheng Zheng Haoqi Wu Derun Zhao Jin Tan Zhicong Huang Cheng Hong Tao Wei Wen-Chang Cheng 145 90 0 24 Jul 2023
Image as First-Order Norm+Linear Autoregression: Unveiling Mathematical Invariance Yinpeng Chen Xiyang Dai Dongdong Chen Xiyang Dai Lu Yuan Zicheng Liu Youzuo Lin 181 2 0 25 May 2023
Img2Vec: A Teacher of High Token-Diversity Helps Masked AutoEncoders Heng Pan Chenyang Liu Wenxiao Wang Liejie Yuan Hongfa Wang Zhifeng Li Wen Liu VLM 141 3 0 25 Apr 2023
Contrastive Tuning: A Little Help to Make Masked Autoencoders ForgetAAAI Conference on Artificial Intelligence (AAAI), 2023 Johannes Lehner Benedikt Alkin Andreas Fürst Elisabeth Rumetshofer Lukas Miklautz Sepp Hochreiter 275 21 0 20 Apr 2023
Hard Patches Mining for Masked Image ModelingComputer Vision and Pattern Recognition (CVPR), 2023 Haochen Wang Kaiyou Song Junsong Fan Yuxi Wang Jin Xie Zhaoxiang Zhang 156 81 0 12 Apr 2023
DIME-FM: DIstilling Multimodal and Efficient Foundation ModelsIEEE International Conference on Computer Vision (ICCV), 2023 Ximeng Sun Pengchuan Zhang Peizhao Zhang Hardik Shah Kate Saenko Xide Xia VLM 298 34 0 31 Mar 2023
Towards General Purpose Medical AI: Continual Learning Medical Foundation Model Huahui Yi Ziyuan Qin Qicheng Lao Wei Xu Zekun Jiang Yi Xu Shaoting Zhang Kang Li OOD MedIm CLL 184 16 0 12 Mar 2023
Efficient Masked Autoencoders with Self-ConsistencyIEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2023 Zhaowen Li Yousong Zhu Zhiyang Chen Wei Li Honghui Dong Rui Zhao Ming Tang Jinqiao Wang 242 3 0 28 Feb 2023
Remote Sensing Scene Classification with Masked Image Modeling (MIM)Remote Sensing (RS), 2023 Liya Wang A. Tien 191 4 0 28 Feb 2023
Layer Grafted Pre-training: Bridging Contrastive Learning And Masked Image Modeling For Label-Efficient RepresentationsInternational Conference on Learning Representations (ICLR), 2023 Ziyu Jiang Yinpeng Chen Xiyang Dai Dongdong Chen Xiyang Dai Lu Yuan Zicheng Liu Zinan Lin SSL VLM CLIP 210 19 0 27 Feb 2023
Anatomical Invariance Modeling and Semantic Alignment for Self-supervised Learning in 3D Medical Image AnalysisIEEE International Conference on Computer Vision (ICCV), 2023 Yankai Jiang Ming Sun Heng Guo Xiaoyu Bai K. Yan Le Lu Minfeng Xu MedIm 217 32 0 11 Feb 2023
Aerial Image Object Detection With Vision Transformer Detector (ViTDet)IEEE International Geoscience and Remote Sensing Symposium (IGARSS), 2023 Liya Wang A. Tien 350 16 0 28 Jan 2023
CLIP Itself is a Strong Fine-tuner: Achieving 85.7% and 88.0% Top-1 Accuracy with ViT-B and ViT-L on ImageNet Xiaoyi Dong Jianmin Bao Ting Zhang Dongdong Chen Shuyang Gu Weiming Zhang Lu Yuan Dong Chen Fang Wen Nenghai Yu CLIP 137 46 0 12 Dec 2022
Masked autoencoders are effective solution to transformer data-hungry Jia-ju Mao Honggu Zhou Xuesong Yin Binling Nie MedIm 319 9 0 12 Dec 2022
Masked Video Distillation: Rethinking Masked Feature Modeling for Self-supervised Video Representation LearningComputer Vision and Pattern Recognition (CVPR), 2022 Rui Wang Dongdong Chen Zuxuan Wu Yinpeng Chen Xiyang Dai Xiyang Dai Lu Yuan Yu-Gang Jiang VGen 257 117 0 08 Dec 2022
Spatio-Temporal Crop Aggregation for Video Representation LearningIEEE International Conference on Computer Vision (ICCV), 2022 Sepehr Sameni Simon Jenni Paolo Favaro 251 4 0 30 Nov 2022
Self-Supervised Learning based on Heat Equation Yinpeng Chen Xiyang Dai Dongdong Chen Xiyang Dai Lu Yuan Zicheng Liu Youzuo Lin 130 5 0 23 Nov 2022
CAE v2: Context Autoencoder with CLIP Target Xinyu Zhang Jiahui Chen Junkun Yuan Qiang Chen Jian Wang ... Jimin Pi Kun Yao Junyu Han Errui Ding Jingdong Wang VLM CLIP 248 25 0 17 Nov 2022