Designing BERT for Convolutional Networks: Sparse and Hierarchical Masked Modeling

9 January 2023

Papers citing "Designing BERT for Convolutional Networks: Sparse and Hierarchical Masked Modeling"

50 / 70 papers shown

Title
A BERT-Style Self-Supervised Learning CNN for Disease Identification from Retinal Images Xin Li Wenhui Zhu Peijie Qiu Oana Dumitrascu Amal Youssef Y. Wang SSL MedIm 87 0 0 25 Apr 2025
Multi-Scale Neighborhood Occupancy Masked Autoencoder for Self-Supervised Learning in LiDAR Point Clouds Mohamed Abdelsamad Michael Ulrich Claudius Gläser Abhinav Valada 3DPC 42 0 0 27 Feb 2025
Escaping The Big Data Paradigm in Self-Supervised Representation Learning Carlos Vélez García Miguel Cazorla Jorge Pomares 46 0 0 25 Feb 2025
Scaling Spike-driven Transformer with Efficient Spike Firing Approximation Training Man Yao Xuerui Qiu Tianxiang Hu J. Hu Yuhong Chou Keyu Tian Jianxing Liao Luziwei Leng Bo Xu Guoqi Li 74 4 0 25 Nov 2024
PR-MIM: Delving Deeper into Partial Reconstruction in Masked Image Modeling Zhong-Yu Li Yunheng Li Deng-Ping Fan Ming-Ming Cheng 64 0 0 24 Nov 2024
GaussianPretrain: A Simple Unified 3D Gaussian Representation for Visual Pre-training in Autonomous Driving Shaoqing Xu Fang Li Shengyin Jiang Ziying Song Li Liu Zhi-xin Yang 3DGS SSL 77 0 0 19 Nov 2024
Revisiting MAE pre-training for 3D medical image segmentation Tassilo Wald Constantin Ulrich Stanislav Lukyanenko Andrei Goncharov Alberto Paderno Leander Maerkisch Paul F. Jäger Paul F. Jäger Klaus Maier-Hein 37 2 0 30 Oct 2024
PK-YOLO: Pretrained Knowledge Guided YOLO for Brain Tumor Detection in Multiplanar MRI Slices Ming Kang F. F. Ting Raphaël C.-W. Phan C. Ting ViT MedIm 52 1 0 29 Oct 2024
Self-Supervised Learning for Real-World Object Detection: a Survey Alina Ciocarlan Sidonie Lefebvre S. L. Hégarat-Mascle Arnaud Woiselle ObjD 19 0 0 09 Oct 2024
Task-Oriented Pre-Training for Drivable Area Detection Fulong Ma Guoyang Zhao Weiqing Qi Ming Liu Jun Ma VLM 25 0 0 30 Sep 2024
Leveraging Text Localization for Scene Text Removal via Text-aware Masked Image Modeling Zixiao Wang Hongtao Xie Yuxin Wang Yadong Qu Fengjun Guo Pengwei Liu DiffM 26 0 0 20 Sep 2024
Whitening Consistently Improves Self-Supervised Learning András Kalapos Bálint Gyires-Tóth SSL 29 0 0 14 Aug 2024
CNN-JEPA: Self-Supervised Pretraining Convolutional Neural Networks Using Joint Embedding Predictive Architecture András Kalapos Bálint Gyires-Tóth 28 2 0 14 Aug 2024
Masked Image Modeling: A Survey Vlad Hondru Florinel-Alin Croitoru Shervin Minaee Radu Tudor Ionescu N. Sebe 61 6 0 13 Aug 2024
HySparK: Hybrid Sparse Masking for Large Scale Medical Image Pre-Training Fenghe Tang Ronghao Xu Qingsong Yao Xueming Fu Quan Quan Heqin Zhu Zaiyi Liu S. Kevin Zhou SSL MedIm 27 4 0 11 Aug 2024
POA: Pre-training Once for Models of All Sizes Yingying Zhang Xin Guo Jiangwei Lao Lei Yu Lixiang Ru Jian Wang Guo Ye Huimei He Jingdong Chen Ming Yang 53 1 0 02 Aug 2024
Sparse Refinement for Efficient High-Resolution Semantic Segmentation Zhijian Liu Zhuoyang Zhang Samir Khaki Shang Yang Haotian Tang Chenfeng Xu Kurt Keutzer Song Han SSeg 31 1 0 26 Jul 2024
Improving Representation of High-frequency Components for Medical Visual Foundation Models Yuetan Chu Yilan Zhang Zhongyi Han Changchun Yang Longxi Zhou Gongning Luo Chao Huang Xin Gao MedIm 32 1 0 19 Jul 2024
Let Occ Flow: Self-Supervised 3D Occupancy Flow Prediction Yili Liu Linzhan Mou Xuan Yu Chenrui Han Sitong Mao R. Xiong Yue Wang 3DPC 30 7 0 10 Jul 2024
AnatoMask: Enhancing Medical Image Segmentation with Reconstruction-guided Self-masking Yuheng Li Tianyu Luan Yizhou Wu Shaoyan Pan Yenho Chen Xiaofeng Yang 28 4 0 09 Jul 2024
VIPriors 4: Visual Inductive Priors for Data-Efficient Deep Learning Challenges Robert-Jan Bruintjes A. Lengyel Marcos Baptista-Rios O. Kayhan Davide Zambrano Nergis Tomen J. C. V. Gemert VLM 29 0 0 26 Jun 2024
LetsMap: Unsupervised Representation Learning for Semantic BEV Mapping Nikhil Gosala Kürsat Petek B. R. Kiran S. Yogamani Paulo L. J. Drews-Jr Wolfram Burgard Abhinav Valada SSL 26 0 0 29 May 2024
Self-supervised learning for classifying paranasal anomalies in the maxillary sinus Debayan Bhattacharya F. Behrendt B. Becker Lennart Maack Dirk Beyersdorff ... B. Cheng D. Eggert C. Betz A. Hoffmann Alexander Schlaefer SSL 11 0 0 29 Apr 2024
AMU-Tuning: Effective Logit Bias for CLIP-based Few-shot Learning Yuwei Tang Zhenyi Lin Qilong Wang Pengfei Zhu Qinghua Hu 26 11 0 13 Apr 2024
D2SL: Decouple Defogging and Semantic Learning for Foggy Domain-Adaptive Segmentation Xuan Sun Zhanfu An Yuyu Liu 26 0 0 07 Apr 2024
Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Prediction Keyu Tian Yi-Xin Jiang Zehuan Yuan Bingyue Peng Liwei Wang VGen 25 248 0 03 Apr 2024
Unifying Local and Global Multimodal Features for Place Recognition in Aliased and Low-Texture Environments Alberto García-Hernández Riccardo Giubilato Klaus H. Strobl Javier Civera Rudolph Triebel 32 2 0 20 Mar 2024
NTK-Guided Few-Shot Class Incremental Learning Jingren Liu Zhong Ji Yanwei Pang Yunlong Yu CLL 27 3 0 19 Mar 2024
MIM4D: Masked Modeling with Multi-View Video for Autonomous Driving Representation Learning Jialv Zou Bencheng Liao Qian Zhang Wenyu Liu Xinggang Wang 36 0 0 13 Mar 2024
VideoMAC: Video Masked Autoencoders Meet ConvNets Gensheng Pei Tao Chen XiRuo Jiang Huafeng Liu Zeren Sun Yazhou Yao VGen 32 9 0 29 Feb 2024
Distillation Enhanced Time Series Forecasting Network with Momentum Contrastive Learning Haozhi Gao Qianqian Ren Jinbao Li AI4TS 19 2 0 31 Jan 2024
Forging Vision Foundation Models for Autonomous Driving: Challenges, Methodologies, and Opportunities Xu Yan Haiming Zhang Yingjie Cai Jingming Guo Weichao Qiu ... Lihui Jiang Wei Zhang Hongbo Zhang Dengxin Dai Bingbing Liu 51 16 0 16 Jan 2024
Spikformer V2: Join the High Accuracy Club on ImageNet with an SNN Ticket Zhaokun Zhou Kaiwei Che Wei Fang Keyu Tian Yuesheng Zhu Shuicheng Yan Yonghong Tian Liuliang Yuan ViT 34 27 0 04 Jan 2024
Masked Modeling for Self-supervised Representation Learning on Vision and Beyond Siyuan Li Luyuan Zhang Zedong Wang Di Wu Lirong Wu ... Jun-Xiong Xia Cheng Tan Yang Liu Baigui Sun Stan Z. Li SSL 29 13 0 31 Dec 2023
MIMIR: Masked Image Modeling for Mutual Information-based Adversarial Robustness Xiaoyun Xu Shujian Yu Jingzheng Wu S. Picek AAML 33 0 0 08 Dec 2023
Guided Reconstruction with Conditioned Diffusion Models for Unsupervised Anomaly Detection in Brain MRIs F. Behrendt Debayan Bhattacharya R. Mieling Lennart Maack Julia Kruger R. Opfer Alexander Schlaefer DiffM MedIm 14 10 0 07 Dec 2023
Masked Autoencoders Are Robust Neural Architecture Search Learners Yiming Hu Xiangxiang Chu Bo-Wen Zhang OOD 19 0 0 20 Nov 2023
UniPAD: A Universal Pre-training Paradigm for Autonomous Driving Honghui Yang Sha Zhang Di Huang Xiaoyang Wu Haoyi Zhu ... Hengshuang Zhao Qibo Qiu Binbin Lin Xiaofei He Wanli Ouyang SSL 21 44 0 12 Oct 2023
PonderV2: Pave the Way for 3D Foundation Model with A Universal Pre-training Paradigm Haoyi Zhu Honghui Yang Xiaoyang Wu Di Huang Sha Zhang ... Hengshuang Zhao Chunhua Shen Yu Qiao Tong He Wanli Ouyang SSL 69 42 0 12 Oct 2023
I $^2$ KD-SLU: An Intra-Inter Knowledge Distillation Framework for Zero-Shot Cross-Lingual Spoken Language Understanding Tianjun Mao Chenghong Zhang VLM 25 0 0 04 Oct 2023
CAIT: Triple-Win Compression towards High Accuracy, Fast Inference, and Favorable Transferability For ViTs Ao Wang Hui Chen Zijia Lin Sicheng Zhao J. Han Guiguang Ding ViT 24 6 0 27 Sep 2023
Species196: A One-Million Semi-supervised Dataset for Fine-grained Species Recognition W. He Kai Han Ying Nie Chengcheng Wang Yunhe Wang VLM 27 6 0 25 Sep 2023
Gold-YOLO: Efficient Object Detector via Gather-and-Distribute Mechanism Chengcheng Wang Wei He Ying Nie Jianyuan Guo Chuanjian Liu Kai Han Yunhe Wang ObjD 16 198 0 20 Sep 2023
COSE: A Consistency-Sensitivity Metric for Saliency on Image Classification Rangel Daroya Aaron Sun Subhransu Maji 11 0 0 20 Sep 2023
Heterogeneous Generative Knowledge Distillation with Masked Image Modeling Ziming Wang Shumin Han Xiaodi Wang Jing Hao Xianbin Cao Baochang Zhang VLM 11 0 0 18 Sep 2023
An Effective Two-stage Training Paradigm Detector for Small Dataset Zheng Wang Dong Xie Hanzhi Wang Jiang Tian ObjD 8 0 0 11 Sep 2023
RevColV2: Exploring Disentangled Representations in Masked Image Modeling Qi Han Yuxuan Cai Xiangyu Zhang 25 7 0 02 Sep 2023
Joint Semantic-Native Communication and Inference via Minimal Simplicial Structures Qiyang Zhao Han Zou M. Bennis Merouane Debbah Ebtesam Almazrouei Faouzi Bader 33 1 0 31 Aug 2023
A General-Purpose Self-Supervised Model for Computational Pathology Richard J. Chen Tong Ding Ming Y. Lu Drew F. K. Williamson Guillaume Jaume ... Judy J. Wang Walt Williams L. Le Georg Gerber Faisal Mahmood MedIm 20 42 0 29 Aug 2023
Adversarial Attacks on Foundational Vision Models Nathan Inkawhich Gwendolyn McDonald R. Luley VLM 33 11 0 28 Aug 2023