BEiT: BERT Pre-Training of Image Transformers

15 June 2021

Papers citing "BEiT: BERT Pre-Training of Image Transformers"

50 / 1,790 papers shown

Title
Open Challenges and Opportunities in Federated Foundation Models Towards Biomedical Healthcare Xingyu Li Lu Peng Yuping Wang Weihua Zhang AI4CE MedIm LM&MA 71 5 0 10 May 2024
MaskMatch: Boosting Semi-Supervised Learning Through Mask Autoencoder-Driven Feature Learning Wenjin Zhang Keyi Li Sen Yang Chenyang Gao Wanzhao Yang Sifan Yuan I. Marsic 33 1 0 10 May 2024
Self-Supervised Pre-training with Symmetric Superimposition Modeling for Scene Text Recognition Zuan Gao Yuxin Wang Yadong Qu Boqiang Zhang Zixiao Wang Jianjun Xu Hongtao Xie ViT 42 9 0 09 May 2024
Similarity Guided Multimodal Fusion Transformer for Semantic Location Prediction in Social Media Zhizhen Zhang Ning Wang Haojie Li Zhihui Wang 29 0 0 09 May 2024
Efficient Pretraining Model based on Multi-Scale Local Visual Field Feature Reconstruction for PCB CT Image Element Segmentation Chen Chen Kai Qiao Jie Yang Jian Chen Bin Yan 27 1 0 09 May 2024
OpenESS: Event-based Semantic Scene Understanding with Open Vocabularies Lingdong Kong You-Chen Liu Lai Xing Ng Benoit R. Cottereau Wei Tsang Ooi VLM 34 13 0 08 May 2024
EVA-X: A Foundation Model for General Chest X-ray Analysis with Self-supervised Learning Jingfeng Yao Xinggang Wang Yuehao Song Huangxuan Zhao Jun Ma Yajie Chen Wenyu Liu Bo Wang ViT 36 5 0 08 May 2024
Is Sora a World Simulator? A Comprehensive Survey on General World Models and Beyond Zheng Zhu Xiaofeng Wang Wangbo Zhao Chen Min Nianchen Deng ... Dawei Zhao Liang Xiao Jian-jun Zhao Jiwen Lu Guan Huang VGen LM&Ro 84 37 0 06 May 2024
Class-relevant Patch Embedding Selection for Few-Shot Image Classification Weihao Jiang Haoyang Cui Kun He VLM 44 0 0 06 May 2024
Intra-task Mutual Attention based Vision Transformer for Few-Shot Learning Weihao Jiang Chang-Shu Liu Kun He ViT 58 0 0 06 May 2024
Self-Supervised Learning for Interventional Image Analytics: Towards Robust Device Trackers Saahil Islam Venkatesh N. Murthy Dominik Neumann B. K. Das Puneet Sharma Andreas K. Maier D. Comaniciu Florin-Cristian Ghesu 29 1 0 02 May 2024
Spider: A Unified Framework for Context-dependent Concept Segmentation Xiaoqi Zhao Youwei Pang Wei Ji Baicheng Sheng Jiaming Zuo Lihe Zhang Huchuan Lu 39 6 0 02 May 2024
Self-supervised Pre-training of Text Recognizers M. Kišš Michal Hradiš SSL 32 1 0 01 May 2024
Masked Multi-Query Slot Attention for Unsupervised Object Discovery Rishav Pramanik José-Fabian Villa-Vásquez M. Pedersoli OCL 40 0 0 30 Apr 2024
Chameleon: A Data-Efficient Generalist for Dense Visual Prediction in the Wild Donggyun Kim Seongwoong Cho Semin Kim Chong Luo Seunghoon Hong VLM 42 2 0 29 Apr 2024
Self-supervised visual learning in the low-data regime: a comparative evaluation Sotirios Konstantakos Despina Ioanna Chalkiadaki Ioannis Mademlis Yuki M. Asano E. Gavves Georgios Th. Papadopoulos 36 6 0 26 Apr 2024
Exploring Learngene via Stage-wise Weight Sharing for Initializing Variable-sized Models Shiyu Xia Wenxuan Zhu Xu Yang Xin Geng 29 1 0 25 Apr 2024
OccFeat: Self-supervised Occupancy Feature Prediction for Pretraining BEV Segmentation Networks Sophia Sirko-Galouchenko Alexandre Boulch Spyros Gidaris Andrei Bursuc Antonín Vobecký Patrick Pérez Renaud Marlet 3DPC 35 7 0 22 Apr 2024
HybridFlow: Infusing Continuity into Masked Codebook for Extreme Low-Bitrate Image Compression Lei Lu Yanyue Xie Wei Jiang Wei Wang Xue Lin Yanzhi Wang 39 4 0 20 Apr 2024
Wills Aligner: A Robust Multi-Subject Brain Representation Learner Guangyin Bao Zixuan Gong Qi Zhang Jialei Zhou Wei Fan Kun Yi Usman Naseem Liang Hu Duoqian Miao 49 0 0 20 Apr 2024
An Experimental Study on Exploring Strong Lightweight Vision Transformers via Masked Image Modeling Pre-Training Jin Gao Shubo Lin Shaoru Wang Yutong Kou Zeming Li Liang Li Congxuan Zhang Xiaoqin Zhang Yizheng Wang Weiming Hu 44 1 0 18 Apr 2024
Cross-Problem Learning for Solving Vehicle Routing Problems Zhuoyi Lin Yaoxin Wu Bangjian Zhou Zhiguang Cao Wen Song Yingqian Zhang Senthilnath Jayavelu 46 10 0 17 Apr 2024
Advancing Long-Term Multi-Energy Load Forecasting with Patchformer: A Patch and Transformer-Based Approach Qiuyi Hong Fanlin Meng Felipe Maldonado AI4TS 26 1 0 16 Apr 2024
Cross-Modal Self-Training: Aligning Images and Pointclouds to Learn Classification without Labels Amaya Dharmasiri Muzammal Naseer Salman Khan Fahad Shahbaz Khan VLM 3DPC 46 1 0 15 Apr 2024
XoFTR: Cross-modal Feature Matching Transformer Önder Tuzcuoglu Aybora Köksal Bugra Sofu Sinan Kalkan Aydin Alatan ViT 50 10 0 15 Apr 2024
Weight Copy and Low-Rank Adaptation for Few-Shot Distillation of Vision Transformers Diana-Nicoleta Grigore Mariana-Iuliana Georgescu J. A. Justo T. Johansen Andreea-Iuliana Ionescu Radu Tudor Ionescu 36 0 0 14 Apr 2024
Navigating the Landscape of Large Language Models: A Comprehensive Review and Analysis of Paradigms and Fine-Tuning Strategies Benjue Weng LM&MA 44 7 0 13 Apr 2024
AMU-Tuning: Effective Logit Bias for CLIP-based Few-shot Learning Yuwei Tang Zhenyi Lin Qilong Wang Pengfei Zhu Qinghua Hu 30 11 0 13 Apr 2024
Multimodal Cross-Document Event Coreference Resolution Using Linear Semantic Transfer and Mixed-Modality Ensembles Abhijnan Nath Huma Jamil Shafiuddin Rehan Ahmed George Baker Rahul Ghosh James H. Martin Nathaniel Blanchard Nikhil Krishnaswamy 32 2 0 13 Apr 2024
OmniSat: Self-Supervised Modality Fusion for Earth Observation Guillaume Astruc Nicolas Gonthier Clement Mallet Loic Landrieu 38 25 0 12 Apr 2024
Emerging Property of Masked Token for Effective Pre-training Hyesong Choi Hunsang Lee Seyoung Joung Hyejin Park Jiyeong Kim Dongbo Min 36 9 0 12 Apr 2024
Salience-Based Adaptive Masking: Revisiting Token Dynamics for Enhanced Pre-training Hyesong Choi Hyejin Park Kwang Moo Yi Sungmin Cha Dongbo Min 39 9 0 12 Apr 2024
GLID: Pre-training a Generalist Encoder-Decoder Vision Model Jihao Liu Jinliang Zheng Yu Liu Hongsheng Li VLM 29 3 0 11 Apr 2024
NeuroNet: A Novel Hybrid Self-Supervised Learning Framework for Sleep Stage Classification Using Single-Channel EEG Cheol-Hui Lee Hakseung Kim Hyun-jee Han Min-Kyung Jung Byung C. Yoon Dong-Joo Kim 37 5 0 10 Apr 2024
Adapting LLaMA Decoder to Vision Transformer Jiahao Wang Wenqi Shao Mengzhao Chen Chengyue Wu Yong Liu Taiqiang Wu Kaipeng Zhang Songyang Zhang Kai-xiang Chen Ping Luo MLLM 38 4 0 10 Apr 2024
Multi-modal Document Presentation Attack Detection With Forensics Trace Disentanglement Changsheng Chen Yongyi Deng Liangwei Lin Zitong Yu Zhimao Lai AAML 19 0 0 10 Apr 2024
Social-MAE: Social Masked Autoencoder for Multi-person Motion Representation Learning Mahsa Ehsanpour Ian Reid Hamid Rezatofighi ViT 34 0 0 08 Apr 2024
iVPT: Improving Task-relevant Information Sharing in Visual Prompt Tuning by Cross-layer Dynamic Connection Nan Zhou Jiaxin Chen Di Huang 35 1 0 08 Apr 2024
HAPNet: Toward Superior RGB-Thermal Scene Parsing via Hybrid, Asymmetric, and Progressive Heterogeneous Feature Fusion Jiahang Li Peng Yun Qijun Chen Rui Fan 36 8 0 04 Apr 2024
Multi Positive Contrastive Learning with Pose-Consistent Generated Images Sho Inayoshi Aji Resindra Widya Satoshi Ozaki Junji Otsuka Takeshi Ohashi 3DH 52 1 0 04 Apr 2024
Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Prediction Keyu Tian Yi-Xin Jiang Zehuan Yuan Bingyue Peng Liwei Wang VGen 31 251 0 03 Apr 2024
Foundation Models for Structural Health Monitoring Luca Benfenati Daniele Jahier Pagliari Luca Zanatta Yhorman Alexander Bedoya Velez Andrea Acquaviva M. Poncino Enrico Macii Luca Benini Alessio Burrello AI4CE 31 2 0 03 Apr 2024
A Unified Membership Inference Method for Visual Self-supervised Encoder via Part-aware Capability Jie Zhu Jirong Zha Ding Li Leye Wang 37 6 0 03 Apr 2024
SyncMask: Synchronized Attentional Masking for Fashion-centric Vision-Language Pretraining Chull Hwan Song Taebaek Hwang Jooyoung Yoon Shunghyun Choi Yeong Hyeon Gu 23 4 0 01 Apr 2024
Lipsum-FT: Robust Fine-Tuning of Zero-Shot Models Using Random Text Guidance G. Nam Byeongho Heo Juho Lee VLM 34 5 0 01 Apr 2024
SpiralMLP: A Lightweight Vision MLP Architecture Haojie Mu Burhan Ul Tayyab Nicholas Chua 43 0 0 31 Mar 2024
Transformer based Pluralistic Image Completion with Reduced Information Loss Qiankun Liu Yuqi Jiang Zhentao Tan Dongdong Chen Ying Fu Qi Chu Gang Hua Nenghai Yu ViT 68 11 0 31 Mar 2024
ST-LLM: Large Language Models Are Effective Temporal Learners Ruyang Liu Chen Li Haoran Tang Yixiao Ge Ying Shan Ge Li 43 68 0 30 Mar 2024
DenseNets Reloaded: Paradigm Shift Beyond ResNets and ViTs Donghyun Kim Byeongho Heo Dongyoon Han 47 14 0 28 Mar 2024
A Two-Phase Recall-and-Select Framework for Fast Model Selection Jianwei Cui Wenhang Shi Honglin Tao Wei Lu Xiaoyong Du 38 0 0 28 Mar 2024