MultiMAE: Multi-modal Multi-task Masked Autoencoders

4 April 2022

Papers citing "MultiMAE: Multi-modal Multi-task Masked Autoencoders"

43 / 43 papers shown

Title
The Moon's Many Faces: A Single Unified Transformer for Multimodal Lunar Reconstruction Tom Sander Moritz Tenthoff Kay Wohlfarth Christian Wöhler 19 0 0 08 May 2025
Multimodal Masked Autoencoder Pre-training for 3D MRI-Based Brain Tumor Analysis with Missing Modalities Lucas Robinet Ahmad Berjaoui Elizabeth Cohen-Jonathan Moyal 21 0 0 01 May 2025
Synergy-CLIP: Extending CLIP with Multi-modal Integration for Robust Representation Learning Sangyeon Cho Jangyeong Jeon Mingi Kim Junyeong Kim CLIP VLM 74 0 0 30 Apr 2025
Adept: Annotation-Denoising Auxiliary Tasks with Discrete Cosine Transform Map and Keypoint for Human-Centric Pretraining Weizhen He Yunfeng Yan Shixiang Tang Yiheng Deng Yangyang Zhong Pengxin Luo Donglian Qi VLM 86 1 0 29 Apr 2025
TAPNext: Tracking Any Point (TAP) as Next Token Prediction Artem Zholus Carl Doersch Yi Yang Skanda Koppula Viorica Patraucean Xu He Ignacio Rocco Mehdi S. M. Sajjadi Sarath Chandar Ross Goroshin 28 0 0 08 Apr 2025
Matrix3D: Large Photogrammetry Model All-in-One Yuanxun Lu Jingyang Zhang Tian Fang Jean-Daniel Nahmias Yanghai Tsin Long Quan Xun Cao Yao Yao Shiwei Li 103 4 0 11 Feb 2025
Efficient Masked AutoEncoder for Video Object Counting and A Large-Scale Benchmark Bing Cao Quanhao Lu Jiekang Feng Pengfei Zhu Q. Hu Qilong Wang 64 0 0 20 Nov 2024
SPA: 3D Spatial-Awareness Enables Effective Embodied Representation Haoyi Zhu Honghui Yang Yating Wang Jiange Yang Limin Wang Tong He 3DH 43 5 0 10 Oct 2024
Analysis of Spatial augmentation in Self-supervised models in the purview of training and test distributions Abhishek Jha Tinne Tuytelaars 18 0 0 26 Sep 2024
What to align in multimodal contrastive learning? Benoit Dufumier J. Castillo-Navarro D. Tuia Jean-Philippe Thiran 22 3 0 11 Sep 2024
IVGF: The Fusion-Guided Infrared and Visible General Framework Fangcen Liu Chenqiang Gao Fang Chen Pengcheng Li Junjie Guo Deyu Meng 29 0 0 02 Sep 2024
Membership Inference Attack Against Masked Image Modeling Z. Li Xinlei He Ning Yu Yang Zhang 35 1 0 13 Aug 2024
Look Ahead or Look Around? A Theoretical Comparison Between Autoregressive and Masked Pretraining Qi Zhang Tianqi Du Haotian Huang Yifei Wang Yisen Wang 21 3 0 01 Jul 2024
Siamese Vision Transformers are Scalable Audio-visual Learners Yan-Bo Lin Gedas Bertasius 27 5 0 28 Mar 2024
MedFLIP: Medical Vision-and-Language Self-supervised Fast Pre-Training with Masked Autoencoder Lei Li Tianfang Zhang Xinglin Zhang Jiaqi Liu Bingqi Ma Yan-chun Luo Tao Chen MedIm 16 0 0 07 Mar 2024
Rethinking Patch Dependence for Masked Autoencoders Letian Fu Long Lian Renhao Wang Baifeng Shi Xudong Wang Adam Yala Trevor Darrell Alexei A. Efros Ken Goldberg 18 14 0 25 Jan 2024
Fus-MAE: A cross-attention-based data fusion approach for Masked Autoencoders in remote sensing Hugo Chan-To-Hing B. Veeravalli 19 7 0 05 Jan 2024
4M: Massively Multimodal Masked Modeling David Mizrahi Roman Bachmann Ouguzhan Fatih Kar Teresa Yeo Mingfei Gao Afshin Dehghan Amir Zamir MLLM 25 62 0 11 Dec 2023
Leveraging Multimodal Fusion for Enhanced Diagnosis of Multiple Retinal Diseases in Ultra-wide OCTA Hao Wei Peilun Shi Guitao Bai Minqing Zhang Shuangle Li Wu Yuan 11 0 0 17 Nov 2023
PolyMaX: General Dense Prediction with Mask Transformer Xuan S. Yang Liangzhe Yuan Kimberly Wilber Astuti Sharma Xiuye Gu ... Stephanie Debats Huisheng Wang Hartwig Adam Mikhail Sirotenko Liang-Chieh Chen 21 14 0 09 Nov 2023
PonderV2: Pave the Way for 3D Foundation Model with A Universal Pre-training Paradigm Haoyi Zhu Honghui Yang Xiaoyang Wu Di Huang Sha Zhang ... Hengshuang Zhao Chunhua Shen Yu Qiao Tong He Wanli Ouyang SSL 57 42 0 12 Oct 2023
EVE: Efficient Vision-Language Pre-training with Masked Prediction and Modality-Aware MoE Junyi Chen Longteng Guo Jianxiang Sun Shuai Shao Zehuan Yuan Liang Lin Dongyu Zhang MLLM VLM MoE 35 9 0 23 Aug 2023
TaskExpert: Dynamically Assembling Multi-Task Representations with Memorial Mixture-of-Experts Hanrong Ye Dan Xu MoE 11 26 0 28 Jul 2023
Visual Prompt Flexible-Modal Face Anti-Spoofing Zitong Yu Rizhao Cai Yawen Cui Ajian Liu Changsheng Chen 28 6 0 26 Jul 2023
InvPT++: Inverted Pyramid Multi-Task Transformer for Visual Scene Understanding Hanrong Ye Dan Xu ViT 13 10 0 08 Jun 2023
A vector quantized masked autoencoder for audiovisual speech emotion recognition Samir Sadok Simon Leglaive Renaud Séguier SSL 63 6 0 05 May 2023
Mask and Restore: Blind Backdoor Defense at Test Time with Masked Autoencoder Tao Sun Lu Pang Chao Chen Haibin Ling AAML 28 9 0 27 Mar 2023
Images Speak in Images: A Generalist Painter for In-Context Visual Learning Xinlong Wang Wen Wang Yue Cao Chunhua Shen Tiejun Huang VLM MLLM 30 244 0 05 Dec 2022
Towards Good Practices for Missing Modality Robust Action Recognition Sangmin Woo Sumin Lee Yeonju Park Muhammad Adi Nugroho Changick Kim 22 42 0 25 Nov 2022
CroCo v2: Improved Cross-view Completion Pre-training for Stereo Matching and Optical Flow Philippe Weinzaepfel Thomas Lucas Vincent Leroy Yohann Cabon Vaibhav Arora Romain Brégier G. Csurka L. Antsfeld Boris Chidlovskii Jérôme Revaud ViT 13 79 0 18 Nov 2022
ViT-DD: Multi-Task Vision Transformer for Semi-Supervised Driver Distraction Detection Yunsheng Ma Ziran Wang ViT 27 12 0 19 Sep 2022
Masked Vision and Language Modeling for Multi-modal Representation Learning Gukyeong Kwon Zhaowei Cai Avinash Ravichandran Erhan Bas Rahul Bhotika Stefano Soatto 16 66 0 03 Aug 2022
A Survey on Masked Autoencoder for Self-supervised Learning in Vision and Beyond Chaoning Zhang Chenshuang Zhang Junha Song John Seon Keun Yi Kang Zhang In So Kweon SSL 36 70 0 30 Jul 2022
Efficient Self-supervised Vision Pretraining with Local Masked Reconstruction Jun Chen Ming Hu Boyang Albert Li Mohamed Elhoseiny 25 36 0 01 Jun 2022
GMML is All you Need Sara Atito Muhammad Awais J. Kittler ViT VLM 24 18 0 30 May 2022
Unleashing Vanilla Vision Transformer with Masked Image Modeling for Object Detection Yuxin Fang Shusheng Yang Shijie Wang Yixiao Ge Ying Shan Xinggang Wang 6 54 0 06 Apr 2022
CMX: Cross-Modal Fusion for RGB-X Semantic Segmentation with Transformers Jiaming Zhang Huayao Liu Kailun Yang Xinxin Hu Ruiping Liu Rainer Stiefelhagen ViT 21 292 0 09 Mar 2022
Omnivore: A Single Model for Many Visual Modalities Rohit Girdhar Mannat Singh Nikhil Ravi L. V. D. van der Maaten Armand Joulin Ishan Misra 209 222 0 20 Jan 2022
Masked Autoencoders Are Scalable Vision Learners Kaiming He Xinlei Chen Saining Xie Yanghao Li Piotr Dollár Ross B. Girshick ViT TPM 258 7,337 0 11 Nov 2021
Emerging Properties in Self-Supervised Vision Transformers Mathilde Caron Hugo Touvron Ishan Misra Hervé Jégou Julien Mairal Piotr Bojanowski Armand Joulin 283 5,723 0 29 Apr 2021
VATT: Transformers for Multimodal Self-Supervised Learning from Raw Video, Audio and Text Hassan Akbari Liangzhe Yuan Rui Qian Wei-Hong Chuang Shih-Fu Chang Yin Cui Boqing Gong ViT 231 573 0 22 Apr 2021
Simple Copy-Paste is a Strong Data Augmentation Method for Instance Segmentation Golnaz Ghiasi Yin Cui A. Srinivas Rui Qian Tsung-Yi Lin E. D. Cubuk Quoc V. Le Barret Zoph ISeg 223 962 0 13 Dec 2020
Meta Pseudo Labels Hieu H. Pham Zihang Dai Qizhe Xie Minh-Thang Luong Quoc V. Le VLM 245 648 0 23 Mar 2020