4M: Massively Multimodal Masked Modeling

11 December 2023

Papers citing "4M: Massively Multimodal Masked Modeling"

50 / 58 papers shown

Title
The Moon's Many Faces: A Single Unified Transformer for Multimodal Lunar Reconstruction Tom Sander Moritz Tenthoff Kay Wohlfarth Christian Wöhler 19 0 0 08 May 2025
Multimodal Masked Autoencoder Pre-training for 3D MRI-Based Brain Tumor Analysis with Missing Modalities Lucas Robinet Ahmad Berjaoui Elizabeth Cohen-Jonathan Moyal 21 0 0 01 May 2025
Boosting Generative Image Modeling via Joint Image-Feature Synthesis Theodoros Kouzelis Efstathios Karypidis Ioannis Kakogeorgiou Spyros Gidaris N. Komodakis DiffM 26 0 0 22 Apr 2025
TerraMesh: A Planetary Mosaic of Multimodal Earth Observation Data Benedikt Blumenstiel P. Fraccaro V. Marsocci Johannes Jakubik Stefano Maurogiovanni ... Rocco Sedona Gabriele Cavallaro Thomas Brunschwiler Juan Bernabé-Moreno Nicolas Longépé 25 2 0 15 Apr 2025
Neural Encoding and Decoding at Scale Yizi Zhang Yanchen Wang Mehdi Azabou Alexandre Andre Zixuan Wang Hanrui Lyu International Brain Laboratory Eva L. Dyer Liam Paninski Cole Hurwitz AI4CE 21 0 0 11 Apr 2025
MAPLE: Encoding Dexterous Robotic Manipulation Priors Learned From Egocentric Videos Alexey Gavryushin Xi Wang Robert J. S. Malate Chenyu Yang X. Jia Shubh Goel Davide Liconti René Zurbrugg Robert K. Katzschmann Marc Pollefeys 34 0 0 08 Apr 2025
We Need Improved Data Curation and Attribution in AI for Scientific Discovery Mara Graziani Antonio Foncubierta Dimitrios Christofidellis Irina Espejo Morales Malina Molnar Marvin Alberts Matteo Manica Jannis Born 39 0 0 03 Apr 2025
MMGen: Unified Multi-modal Image Generation and Understanding in One Go Jiepeng Wang Zhaoqing Wang H. Pan Yuan Liu Dongdong Yu Changhu Wang Wenping Wang DiffM 76 0 0 26 Mar 2025
Halton Scheduler For Masked Generative Image Transformer Victor Besnier Mickael Chen David Hurych Eduardo Valle Matthieu Cord 44 1 0 21 Mar 2025
The Power of Context: How Multimodality Improves Image Super-Resolution Kangfu Mei Hossein Talebi Mojtaba Ardakani Vishal M. Patel P. Milanfar M. Delbracio DiffM 77 1 0 18 Mar 2025
MaskSDM with Shapley values to improve flexibility, robustness, and explainability in species distribution modeling Robin Zbinden Nina Van Tiel Gencer Sumbul Chiara Vanalli B. Kellenberger D. Tuia 39 0 0 17 Mar 2025
RealGeneral: Unifying Visual Generation via Temporal In-Context Learning with Video Models Yijing Lin Mengqi Huang Shuhan Zhuang Zhendong Mao VGen 41 0 0 13 Mar 2025
Robust Latent Matters: Boosting Image Generation with Sampling Error Synthesis Kai Qiu X. Li Jason Kuen H. Chen Xiaohao Xu Jiuxiang Gu Yinyi Luo Bhiksha Raj Zhe-nan Lin Marios Savvides 55 0 0 11 Mar 2025
Matrix3D: Large Photogrammetry Model All-in-One Yuanxun Lu Jingyang Zhang Tian Fang Jean-Daniel Nahmias Yanghai Tsin Long Quan Xun Cao Yao Yao Shiwei Li 103 4 0 11 Feb 2025
Wearable Accelerometer Foundation Models for Health via Knowledge Distillation Salar Abbaspourazad Anshuman Mishra Joseph D. Futoma Andrew C. Miller Ian Shapiro 83 0 0 15 Dec 2024
XQ-GAN: An Open-source Image Tokenization Framework for Autoregressive Generation X. Li Kai Qiu H. Chen Jason Kuen Jiuxiang Gu J. Wang Zhe-nan Lin Bhiksha Raj VLM 112 3 0 02 Dec 2024
ENAT: Rethinking Spatial-temporal Interactions in Token-based Image Synthesis Zanlin Ni Yulin Wang Renping Zhou Yizeng Han Jiayi Guo Zhiyuan Liu Yuan Yao Gao Huang 48 4 0 11 Nov 2024
Multi-Transmotion: Pre-trained Model for Human Motion Prediction Yang Gao Po-Chien Luan Alexandre Alahi 21 5 0 04 Nov 2024
Adapting Diffusion Models for Improved Prompt Compliance and Controllable Image Synthesis Deepak Sridhar Abhishek Peri Rohith Rachala Nuno Vasconcelos DiffM 25 0 0 29 Oct 2024
Croc: Pretraining Large Multimodal Models with Cross-Modal Comprehension Yin Xie Kaicheng Yang Ninghua Yang Weimo Deng Xiangzi Dai ... Yumeng Wang Xiang An Yongle Zhao Ziyong Feng Jiankang Deng MLLM VLM 35 1 0 18 Oct 2024
Skipping Computations in Multimodal LLMs Mustafa Shukor Matthieu Cord 24 2 0 12 Oct 2024
Masked Generative Priors Improve World Models Sequence Modelling Capabilities Cristian Meo Mircea Lica Zarif Ikram Akihiro Nakano Vedant Shah Aniket Didolkar Dianbo Liu Anirudh Goyal Justin Dauwels OffRL 76 0 0 10 Oct 2024
ImageFolder: Autoregressive Image Generation with Folded Tokens Xiang Li Kai Qiu Hao Chen Jason Kuen Jiuxiang Gu Bhiksha Raj Zhe-nan Lin VLM 34 17 0 02 Oct 2024
PixWizard: Versatile Image-to-Image Visual Assistant with Open-Language Instructions Weifeng Lin Xinyu Wei Renrui Zhang Le Zhuo Shitian Zhao ... Junlin Xie Junlin Xie Yu Qiao Peng Gao Hongsheng Li MLLM DiffM 50 10 0 23 Sep 2024
Detect Fake with Fake: Leveraging Synthetic Data-driven Representation for Synthetic Image Detection Hina Otake Yoshihiro Fukuhara Yoshiki Kubotani Shigeo Morishima ViT 46 0 0 13 Sep 2024
SimMAT: Exploring Transferability from Vision Foundation Models to Any Image Modality Chenyang Lei Liyi Chen Jun Cen Xiao Chen Zhen Lei Felix Heide Ziwei Liu Qifeng Chen Zhaoxiang Zhang 26 0 0 12 Sep 2024
What to align in multimodal contrastive learning? Benoit Dufumier J. Castillo-Navarro D. Tuia Jean-Philippe Thiran 22 3 0 11 Sep 2024
PoseEmbroider: Towards a 3D, Visual, Semantic-aware Human Pose Representation Ginger Delmas Philippe Weinzaepfel Francesc Moreno-Noguer Grégory Rogez 27 2 0 10 Sep 2024
AdaNAT: Exploring Adaptive Policy for Token-Based Image Generation Zanlin Ni Yulin Wang Renping Zhou Rui Lu Jiayi Guo Jinyi Hu Zhiyuan Liu Yuan Yao Gao Huang 25 7 0 31 Aug 2024
Masked Image Modeling: A Survey Vlad Hondru Florinel-Alin Croitoru Shervin Minaee Radu Tudor Ionescu N. Sebe 59 6 0 13 Aug 2024
SlowFast-LLaVA: A Strong Training-Free Baseline for Video Large Language Models Mingze Xu Mingfei Gao Zhe Gan Hong-You Chen Zhengfeng Lai Haiming Gang Kai Kang Afshin Dehghan 48 48 0 22 Jul 2024
DiffX: Guide Your Layout to Cross-Modal Generative Modeling Zeyu Wang Jingyu Lin Yifei Qian Yi Huang Shicen Tian ... Qu Yang Lan Du Cunjian Chen Yufei Guo Kejie Huang DiffM VLM 23 0 0 22 Jul 2024
Global atmospheric data assimilation with multi-modal masked autoencoders T. Vandal Kate Duffy Daniel J. McDuff Yoni Nachmany Chris Hartshorn AI4Cl AI4CE 17 2 0 16 Jul 2024
Unraveling Molecular Structure: A Multimodal Spectroscopic Dataset for Chemistry Marvin Alberts Oliver Schilter F. Zipoli Nina Hartrampf Teodoro Laino 20 5 0 04 Jul 2024
ControlVAR: Exploring Controllable Visual Autoregressive Modeling Xiang Li Kai Qiu Hao Chen Jason Kuen Zhe-nan Lin Rita Singh Bhiksha Raj DiffM 35 21 0 14 Jun 2024
4M-21: An Any-to-Any Vision Model for Tens of Tasks and Modalities Roman Bachmann Oğuzhan Fatih Kar David Mizrahi Ali Garjani Mingfei Gao David Griffiths Jiaming Hu Afshin Dehghan Amir Zamir MoE VLM MLLM 28 14 0 13 Jun 2024
An Image is Worth 32 Tokens for Reconstruction and Generation Qihang Yu Mark Weber XueQing Deng Xiaohui Shen Daniel Cremers Liang-Chieh Chen VLM ViT 39 77 0 11 Jun 2024
The Evolution of Multimodal Model Architectures S. Wadekar Abhishek Chaurasia Aman Chadha Eugenio Culurciello 41 13 0 28 May 2024
Implicit Multimodal Alignment: On the Generalization of Frozen LLMs to Multimodal Inputs Mustafa Shukor Matthieu Cord 59 5 0 26 May 2024
Visual Echoes: A Simple Unified Transformer for Audio-Visual Generation Shiqi Yang Zhi-Wei Zhong Mengjie Zhao Shusuke Takahashi Masato Ishii Takashi Shibuya Yuki Mitsufuji 43 2 0 23 May 2024
MMEarth: Exploring Multi-Modal Pretext Tasks For Geospatial Representation Learning Vishal Nedungadi A. Kariryaa Stefan Oehmcke Serge J. Belongie Christian Igel Nico Lang 27 24 0 04 May 2024
What Makes Multimodal In-Context Learning Work? Folco Bertini Baldassini Mustafa Shukor Matthieu Cord Laure Soulier Benjamin Piwowarski 32 18 0 24 Apr 2024
BRAVE: Broadening the visual encoding of vision-language models Ouguzhan Fatih Kar A. Tonioni Petra Poklukar Achin Kulshrestha Amir Zamir Federico Tombari MLLM VLM 42 25 0 10 Apr 2024
UniTable: Towards a Unified Framework for Table Recognition via Self-Supervised Pretraining Sheng-Hsuan Peng Aishwarya Chakravarthy Seongmin Lee Xiaojing Wang Rajarajeswari Balasubramaniyan Duen Horng Chau LMTD 34 0 0 07 Mar 2024
Multiple Random Masking Autoencoder Ensembles for Robust Multimodal Semi-supervised Learning Alexandru-Raul Todoran Marius Leordeanu 20 0 0 12 Feb 2024
A Simple Latent Diffusion Approach for Panoptic Segmentation and Mask Inpainting Wouter Van Gansbeke Bert De Brabandere DiffM 19 11 0 18 Jan 2024
Muse: Text-To-Image Generation via Masked Generative Transformers Huiwen Chang Han Zhang Jarred Barber AJ Maschinot José Lezama ... Kevin Patrick Murphy William T. Freeman Michael Rubinstein Yuanzhen Li Dilip Krishnan DiffM 197 515 0 02 Jan 2023
Exploring Target Representations for Masked Autoencoders Xingbin Liu Jinghao Zhou Tao Kong Xianming Lin Rongrong Ji 76 49 0 08 Sep 2022
UViM: A Unified Modeling Approach for Vision with Learned Guiding Codes Alexander Kolesnikov André Susano Pinto Lucas Beyer Xiaohua Zhai Jeremiah Harmsen N. Houlsby 103 67 0 20 May 2022
MulT: An End-to-End Multitask Learning Transformer Deblina Bhattacharjee Tong Zhang Sabine Süsstrunk Mathieu Salzmann ViT 29 62 0 17 May 2022