Chameleon: Mixed-Modal Early-Fusion Foundation Models

16 May 2024

Papers citing "Chameleon: Mixed-Modal Early-Fusion Foundation Models"

50 / 192 papers shown

Title
TokLIP: Marry Visual Tokens to CLIP for Multimodal Comprehension and Generation Haokun Lin Teng Wang Yixiao Ge Yuying Ge Zhichao Lu Ying Wei Qingfu Zhang Zhenan Sun Ying Shan MLLM VLM 51 0 0 08 May 2025
Unified Multimodal Understanding and Generation Models: Advances, Challenges, and Opportunities X. Zhang Jintao Guo Shanshan Zhao Minghao Fu Lunhao Duan Guo-Hua Wang Qing-Guo Chen Zhao Xu Weihua Luo Kaifu Zhang DiffM 57 0 0 05 May 2025
Ming-Lite-Uni: Advancements in Unified Architecture for Natural Multimodal Interaction Biao Gong Cheng Zou Dandan Zheng Hu Yu Jingdong Chen ... Qingpei Guo Rui Liu Weilong Chai Xinyu Xiao Ziyuan Huang MLLM 66 1 0 05 May 2025
R1-Reward: Training Multimodal Reward Model Through Stable Reinforcement Learning Yi-Fan Zhang Xingyu Lu X. Hu Chaoyou Fu Bin Wen ... J. Chen Fan Yang Z. Zhang Tingting Gao Liang Wang OffRL LRM 30 0 0 05 May 2025
Interleave-VLA: Enhancing Robot Manipulation with Interleaved Image-Text Instructions Cunxin Fan Xiaosong Jia Yihang Sun Yixiao Wang Jianglan Wei ... Xiangyu Zhao M. Tomizuka Xue Yang Junchi Yan Mingyu Ding LM&Ro VLM 54 2 0 04 May 2025
A Survey of Interactive Generative Video Jiwen Yu Yiran Qin Haoxuan Che Quande Liu X. Wang Pengfei Wan Di Zhang Kun Gai Hao Chen Xihui Liu VGen 53 0 0 30 Apr 2025
X-Fusion: Introducing New Modality to Frozen Large Language Models Sicheng Mo Thao Nguyen Xun Huang Siddharth Srinivasan Iyer Yijun Li ... Eli Shechtman Krishna Kumar Singh Yong Jae Lee Bolei Zhou Yuheng Li 71 0 0 29 Apr 2025
YoChameleon: Personalized Vision and Language Generation Thao Nguyen Krishna Kumar Singh Jing Shi Trung H. Bui Yong Jae Lee Yuheng Li MLLM 79 0 0 29 Apr 2025
Learning Streaming Video Representation via Multitask Training Yibin Yan Jilan Xu Shangzhe Di Yikun Liu Yudi Shi Qirui Chen Zeqian Li Yifei Huang Weidi Xie CLL 76 0 0 28 Apr 2025
Token-Shuffle: Towards High-Resolution Image Generation with Autoregressive Models Xu Ma Peize Sun Haoyu Ma Hao Tang Chih-Yao Ma ... Matt Feiszli Peizhao Zhang Peter Vajda Sam S. Tsai Y. Fu 65 1 0 24 Apr 2025
Generative Multimodal Pretraining with Discrete Diffusion Timestep Tokens Kaihang Pan Wang Lin Zhongqi Yue Tenglong Ao Liyu Jia Wei Zhao Juncheng Billy Li Siliang Tang Hanwang Zhang 32 1 0 20 Apr 2025
Personalized Text-to-Image Generation with Auto-Regressive Models Kaiyue Sun Xian Liu Yao Teng Xihui Liu 33 0 0 17 Apr 2025
NoisyRollout: Reinforcing Visual Reasoning with Data Augmentation Xiangyan Liu Jinjie Ni Zijian Wu Chao Du Longxu Dou H. Wang Tianyu Pang Michael Shieh OffRL LRM 43 0 0 17 Apr 2025
SimpleAR: Pushing the Frontier of Autoregressive Visual Generation through Pretraining, SFT, and RL Junke Wang Zhi Tian X. Wang Xinyu Zhang Weilin Huang Zuxuan Wu Yu Jiang VGen 40 2 0 15 Apr 2025
The Scalability of Simplicity: Empirical Analysis of Vision-Language Learning with a Single Transformer Weixian Lei Jiacong Wang Haochen Wang X. Li Jun Hao Liew Jiashi Feng Zilong Huang 23 1 0 14 Apr 2025
GeoUni: A Unified Model for Generating Geometry Diagrams, Problems and Problem Solutions Jo-Ku Cheng Zeren Zhang Ran Chen Jingyang Deng Ziran Qin Jinwen Ma 28 0 0 14 Apr 2025
FocalLens: Instruction Tuning Enables Zero-Shot Conditional Image Representations Cheng-Yu Hsieh Pavan Kumar Anasosalu Vasu Fartash Faghri Raviteja Vemulapalli Chun-Liang Li Ranjay Krishna Oncel Tuzel Hadi Pouransari VLM 57 0 0 11 Apr 2025
Have we unified image generation and understanding yet? An empirical study of GPT-4o's image generation ability Ning Li Jingran Zhang Justin Cui MLLM 64 1 0 09 Apr 2025
Decoupling Contrastive Decoding: Robust Hallucination Mitigation in Multimodal Large Language Models Wei Chen Xin Yan Bin Wen Fan Yang Tingting Gao Di Zhang Long Chen MLLM 79 0 0 09 Apr 2025
OmniCaptioner: One Captioner to Rule Them All Yiting Lu Jiakang Yuan Zhen Li Shitian Zhao Qi Qin ... Lei Bai Zhibo Chen Peng Gao Bo Zhang Peng Gao MLLM 79 0 0 09 Apr 2025
Transfer between Modalities with MetaQueries Xichen Pan Satya Narayan Shukla Aashu Singh Zhuokai Zhao Shlok Kumar Mishra ... Jiuhai Chen Kunpeng Li F. Xu Ji Hou Saining Xie DiffM 35 6 0 08 Apr 2025
MME-Unify: A Comprehensive Benchmark for Unified Multimodal Understanding and Generation Models Wulin Xie Y. Zhang Chaoyou Fu Yang Shi Bingyan Nie Hongkai Chen Z. Zhang Liang Wang T. Tan 31 1 0 04 Apr 2025
Fine-Tuning Visual Autoregressive Models for Subject-Driven Generation Jiwoo Chung Sangeek Hyun Hyunjun Kim Eunseo Koh MinKyu Lee Jae-Pil Heo 33 0 0 03 Apr 2025
VARGPT-v1.1: Improve Visual Autoregressive Large Unified Model via Iterative Instruction Tuning and Reinforcement Learning Xianwei Zhuang Yuxin Xie Yufan Deng Dongchao Yang Liming Liang Jinghan Ru Yuguo Yin Yuexian Zou 61 1 0 03 Apr 2025
Envisioning Beyond the Pixels: Benchmarking Reasoning-Informed Visual Editing Xiangyu Zhao Peiyuan Zhang Kexian Tang Hao Li Zicheng Zhang Guangtao Zhai Junchi Yan Hua Yang Xue Yang Haodong Duan VLM LRM 41 0 0 03 Apr 2025
GPT-ImgEval: A Comprehensive Benchmark for Diagnosing GPT4o in Image Generation Zhiyuan Yan Junyan Ye Weijia Li Zilong Huang Shenghai Yuan Xiangyang He Kaiqing Lin Jun-Jian He Conghui He Li Yuan MLLM EGVM 88 8 0 03 Apr 2025
UniViTAR: Unified Vision Transformer with Native Resolution Limeng Qiao Yiyang Gan Bairui Wang Jie Qin Shuang Xu Siqi Yang Lin Ma 42 0 0 02 Apr 2025
ILLUME+: Illuminating Unified MLLM with Dual Visual Tokenization and Diffusion Refinement Runhui Huang Chunwei Wang Junwei Yang Guansong Lu Yunlong Yuan ... Lu Hou Wei Zhang Lanqing Hong Hengshuang Zhao Hang Xu MLLM 76 1 0 02 Apr 2025
Orchestrate Multimodal Data with Batch Post-Balancing to Accelerate Multimodal Large Language Model Training Yijie Zheng Bangjun Xiao Lei Shi Xiaoyang Li Faming Wu Tianyu Li Xuefeng Xiao Y. Zhang Y. Wang Shouda Liu MLLM MoE 48 1 0 31 Mar 2025
Make Some Noise: Towards LLM audio reasoning and generation using sound tokens Shivam Mehta Nebojsa Jojic Hannes Gamper 26 0 0 28 Mar 2025
Exploiting Mixture-of-Experts Redundancy Unlocks Multimodal Generative Abilities Raman Dutt Harleen Hanspal Guoxuan Xia Petru-Daniel Tudosiu Alexander Black Yongxin Yang Steven G. McDonagh Sarah Parisot MoE 33 0 0 28 Mar 2025
CoT-VLA: Visual Chain-of-Thought Reasoning for Vision-Language-Action Models Qingqing Zhao Yao Lu Moo Jin Kim Zipeng Fu Zhuoyang Zhang ... Ankur Handa Ming-Yu Liu Donglai Xiang Gordon Wetzstein Tsung-Yi Lin LM&Ro LRM 37 9 0 27 Mar 2025
UGen: Unified Autoregressive Multimodal Model with Progressive Vocabulary Learning Hongxuan Tang Hao Liu Xinyan Xiao 37 1 0 27 Mar 2025
Harmonizing Visual Representations for Unified Multimodal Understanding and Generation Size Wu W. Zhang Lumin Xu Sheng Jin Zhonghua Wu Qingyi Tao Wentao Liu Wei Li Chen Change Loy VGen 53 2 0 27 Mar 2025
Beyond Words: Advancing Long-Text Image Generation via Multimodal Autoregressive Models Alex Jinpeng Wang Linjie Li Z. Yang Lijuan Wang Min Li DiffM 68 0 0 26 Mar 2025
Unified Multimodal Discrete Diffusion Alexander Swerdlow Mihir Prabhudesai Siddharth Gandhi Deepak Pathak Katerina Fragkiadaki DiffM 63 0 0 26 Mar 2025
Gemma 3 Technical Report Gemma Team Aishwarya B Kamath Johan Ferret Shreya Pathak Nino Vieillard ... Harshal Tushar Lehri Hussein Hazimeh Ian Ballantyne Idan Szpektor Ivan Nardini VLM 82 24 0 25 Mar 2025
ImageGen-CoT: Enhancing Text-to-Image In-context Learning with Chain-of-Thought Reasoning Jiaqi Liao Z. Yang Linjie Li Dianqi Li Kevin Qinghong Lin Yu-Xi Cheng Lijuan Wang MLLM LRM 57 0 0 25 Mar 2025
FullDiT: Multi-Task Video Generative Foundation Model with Full Attention Xuan Ju Weicai Ye Quande Liu Qiulin Wang Xintao Wang Pengfei Wan Di Zhang Kun Gai Qiang Xu VGen 39 1 0 25 Mar 2025
Adaptive Unimodal Regulation for Balanced Multimodal Information Acquisition Chengxiang Huang Yake Wei Zequn Yang D. Hu 37 0 0 24 Mar 2025
Breaking the Encoder Barrier for Seamless Video-Language Understanding Handong Li Yiyuan Zhang Longteng Guo Xiangyu Yue Jing Liu VLM 69 0 0 24 Mar 2025
CO-SPY: Combining Semantic and Pixel Features to Detect Synthetic Images by AI Siyuan Cheng Lingjuan Lyu Zhenting Wang X. Zhang Vikash Sehwag 30 0 0 24 Mar 2025
Expanding the Boundaries of Vision Prior Knowledge in Multi-modal Large Language Models Qiao Liang Yanjiang Liu Ben He Y. Lu Hongyu Lin Jia Zheng Xianpei Han Le Sun Yingfei Sun 39 0 0 23 Mar 2025
Uni-3DAR: Unified 3D Generation and Understanding via Autoregression on Compressed Spatial Tokens Shuqi Lu Haowei Lin Lin Yao Zhifeng Gao Xiaohong Ji W. Elwasif Linfeng Zhang Guolin Ke 41 0 0 20 Mar 2025
Bridging Continuous and Discrete Tokens for Autoregressive Visual Generation Y. Wang Zhijie Lin Yao Teng Yuanzhi Zhu Shuhuai Ren Jiashi Feng Xihui Liu 46 0 0 20 Mar 2025
Cube: A Roblox View of 3D Intelligence Foundation AI Team Roblox Kiran Bhat Nishchaie Khanna Karun Channa Tinghui Zhou ... Kyle Price Steve Han Yiqing Wang A. Singh David Baszucki 51 0 0 19 Mar 2025
DualToken: Towards Unifying Visual Understanding and Generation with Dual Visual Vocabularies Wei Song Y. Wang Zijia Song Yadong Li Haoze Sun Weipeng Chen Zenan Zhou Jianhua Xu Jiaqi Wang Kaicheng Yu 56 2 0 18 Mar 2025
Unified Autoregressive Visual Generation and Understanding with Continuous Tokens Lijie Fan Luming Tang Siyang Qin Tianhong Li Xuan S. Yang ... Tao Zhu Michael Rubinstein Michalis Raptis Deqing Sun Radu Soricut 50 4 0 17 Mar 2025
Next-Scale Autoregressive Models are Zero-Shot Single-Image Object View Synthesizers Shiran Yuan Hao Zhao DiffM 47 0 0 17 Mar 2025
UniVG: A Generalist Diffusion Model for Unified Image Generation and Editing Tsu-jui Fu Yusu Qian Chen Chen Wenze Hu Zhe Gan Y. Yang 85 1 0 16 Mar 2025