Transfusion: Predict the Next Token and Diffuse Images with One Multi-Modal Model

20 August 2024

Lili Yu

Papers citing "Transfusion: Predict the Next Token and Diffuse Images with One Multi-Modal Model"

50 / 117 papers shown

Title
X-Driver: Explainable Autonomous Driving with Vision-Language Models Wei Liu J. A. Zhang Binxiong Zheng Yufeng Hu Yingzhan Lin Zengfeng Zeng VLM LRM 60 0 0 08 May 2025
TokLIP: Marry Visual Tokens to CLIP for Multimodal Comprehension and Generation Haokun Lin Teng Wang Yixiao Ge Yuying Ge Zhichao Lu Ying Wei Qingfu Zhang Zhenan Sun Ying Shan MLLM VLM 70 0 0 08 May 2025
Unified Multimodal Understanding and Generation Models: Advances, Challenges, and Opportunities Xuzhi Zhang Jintao Guo Shanshan Zhao Minghao Fu Lunhao Duan Guo-Hua Wang Qing-Guo Chen Zhao Xu Weihua Luo Kaifu Zhang DiffM 74 0 0 05 May 2025
T2I-R1: Reinforcing Image Generation with Collaborative Semantic-level and Token-level CoT D. Jiang Ziyu Guo Renrui Zhang Zhuofan Zong Hao Li Le Zhuo Shilin Yan Pheng-Ann Heng Hao Li LRM 69 1 0 01 May 2025
A Survey of Interactive Generative Video Jiwen Yu Yiran Qin Haoxuan Che Quande Liu Xinyu Wang Pengfei Wan Di Zhang Kun Gai Hao Chen Xihui Liu VGen 65 0 0 30 Apr 2025
Nexus-Gen: A Unified Model for Image Understanding, Generation, and Editing Hong Zhang Zhongjie Duan Xingjun Wang Yuze Zhao Weiyi Lu Zhipeng Di Yongjun Xu Yingda Chen Yu Zhang MLLM 94 1 0 30 Apr 2025
X-Fusion: Introducing New Modality to Frozen Large Language Models Sicheng Mo Thao Nguyen Xun Huang Siddharth Srinivasan Iyer Yijun Li ... Eli Shechtman Krishna Kumar Singh Yong Jae Lee Bolei Zhou Yuheng Li 77 0 0 29 Apr 2025
YoChameleon: Personalized Vision and Language Generation Thao Nguyen Krishna Kumar Singh Jing Shi Trung H. Bui Yong Jae Lee Yuheng Li MLLM 82 0 0 29 Apr 2025
Token-Shuffle: Towards High-Resolution Image Generation with Autoregressive Models Xu Ma Peize Sun Haoyu Ma Hao Tang Chih-Yao Ma ... Matt Feiszli Peizhao Zhang Peter Vajda Sam S. Tsai Y. Fu 68 1 0 24 Apr 2025
DiMeR: Disentangled Mesh Reconstruction Model Lutao Jiang Jiantao Lin Kanghao Chen Wenhang Ge Xin Yang Yifan Jiang Y. Lyu Xu Zheng Yingcong Chen 3DV 70 1 0 24 Apr 2025
A Call for New Recipes to Enhance Spatial Reasoning in MLLMs Huanyu Zhang Chengzu Li Wenshan Wu Shaoguang Mao Yan Xia Ivan Vulić Z. Zhang Liang Wang Tieniu Tan Furu Wei LRM 34 2 0 21 Apr 2025
Seeing from Another Perspective: Evaluating Multi-View Understanding in MLLMs Chun-Hsiao Yeh Chenyu Wang Shengbang Tong Ta-Ying Cheng Rouyu Wang Tianzhe Chu Yuexiang Zhai Yubei Chen Shenghua Gao Yi Ma LRM 66 0 0 21 Apr 2025
Generative Multimodal Pretraining with Discrete Diffusion Timestep Tokens Kaihang Pan Wang Lin Zhongqi Yue Tenglong Ao Liyu Jia Wei Zhao Juncheng Billy Li Siliang Tang Hanwang Zhang 49 2 0 20 Apr 2025
Generative Recommendation with Continuous-Token Diffusion Haohao Qu Wenqi Fan Shanru Lin DiffM 92 0 0 16 Apr 2025
SimpleAR: Pushing the Frontier of Autoregressive Visual Generation through Pretraining, SFT, and RL Junke Wang Zhi Tian Xinyu Wang Xinyu Zhang Weilin Huang Zuxuan Wu Yu Jiang VGen 49 4 0 15 Apr 2025
PixelFlow: Pixel-Space Generative Models with Flow Shoufa Chen Chongjian Ge Shilong Zhang Peize Sun Ping Luo VLM DRL 37 0 0 10 Apr 2025
Decoupling Contrastive Decoding: Robust Hallucination Mitigation in Multimodal Large Language Models Wei Chen Xin Yan Bin Wen Fan Yang Tingting Gao Di Zhang Long Chen MLLM 97 0 0 09 Apr 2025
Have we unified image generation and understanding yet? An empirical study of GPT-4o's image generation ability Ning Li Jingran Zhang Justin Cui MLLM 70 1 0 09 Apr 2025
Transfer between Modalities with MetaQueries Xichen Pan Satya Narayan Shukla Aashu Singh Zhuokai Zhao Shlok Kumar Mishra ... Jiuhai Chen Kunpeng Li F. Xu Ji Hou Saining Xie DiffM 46 6 0 08 Apr 2025
UniToken: Harmonizing Multimodal Understanding and Generation through Unified Visual Encoding Yang Jiao Haibo Qiu Zequn Jie S. Chen Jingjing Chen Lin Ma Yu Jiang 34 2 0 06 Apr 2025
MME-Unify: A Comprehensive Benchmark for Unified Multimodal Understanding and Generation Models Wulin Xie Yuyao Zhang Chaoyou Fu Yang Shi Bingyan Nie Hongkai Chen Z. Zhang Liang Wang Tieniu Tan 33 1 0 04 Apr 2025
GPT-ImgEval: A Comprehensive Benchmark for Diagnosing GPT4o in Image Generation Zhiyuan Yan Junyan Ye Weijia Li Zilong Huang Shenghai Yuan Xiangyang He Kaiqing Lin Jun-Jian He Conghui He Li Yuan MLLM EGVM 88 8 0 03 Apr 2025
Unified World Models: Coupling Video and Action Diffusion for Pretraining on Large Robotic Datasets Chuning Zhu Raymond Yu S. Feng Benjamin Burchfiel Paarth Shah Abhishek Gupta VGen 57 0 0 03 Apr 2025
Evolutionary Prompt Optimization Discovers Emergent Multimodal Reasoning Strategies in Vision-Language Models Sid Bharthulwar John Rho Katrina Brown ReLM VLM LRM 58 0 0 30 Mar 2025
UGen: Unified Autoregressive Multimodal Model with Progressive Vocabulary Learning Hongxuan Tang Hao Liu Xinyan Xiao 45 1 0 27 Mar 2025
CoT-VLA: Visual Chain-of-Thought Reasoning for Vision-Language-Action Models Qingqing Zhao Yao Lu Moo Jin Kim Zipeng Fu Zhuoyang Zhang ... Ankur Handa Xuan Li Donglai Xiang Gordon Wetzstein Nayeon Lee LM&Ro LRM 43 11 0 27 Mar 2025
Harmonizing Visual Representations for Unified Multimodal Understanding and Generation Size Wu W. Zhang Lumin Xu Sheng Jin Zhonghua Wu Qingyi Tao Wentao Liu Wei Li Chen Change Loy VGen 153 2 0 27 Mar 2025
TransDiffSBDD: Causality-Aware Multi-Modal Structure-Based Drug Design Xiuyuan Hu Guoqing Liu Can Chen Yang Zhao Hao Zhang Xue Liu 58 2 0 26 Mar 2025
Vision as LoRA Han Wang Yongjie Ye Bingru Li Yuxiang Nie Jinghui Lu Jingqun Tang Yanjie Wang Can Huang 88 0 0 26 Mar 2025
Unified Multimodal Discrete Diffusion Alexander Swerdlow Mihir Prabhudesai Siddharth Gandhi Deepak Pathak Katerina Fragkiadaki DiffM 77 0 0 26 Mar 2025
FullDiT: Multi-Task Video Generative Foundation Model with Full Attention Xuan Ju Weicai Ye Quande Liu Qiulin Wang Xintao Wang Pengfei Wan Di Zhang Kun Gai Qiang Xu VGen 46 1 0 25 Mar 2025
Long-Context Autoregressive Video Modeling with Next-Frame Prediction Yuchao Gu Weijia Mao Mike Zheng Shou VGen 84 2 0 25 Mar 2025
Teller: Real-Time Streaming Audio-Driven Portrait Animation with Autoregressive Motion Generation Dingcheng Zhen Shunshun Yin Shiyang Qin Hou Yi Ziwei Zhang Siyuan Liu Gan Qi Ming Tao VGen 71 0 0 24 Mar 2025
Breaking the Encoder Barrier for Seamless Video-Language Understanding Handong Li Yiyuan Zhang Longteng Guo Xiangyu Yue Jing Liu VLM 72 0 0 24 Mar 2025
Position: Interactive Generative Video as Next-Generation Game Engine Jiwen Yu Yiran Qin Haoxuan Che Quande Liu Xintao Wang Pengfei Wan Di Zhang Xihui Liu VGen 45 1 0 21 Mar 2025
D2C: Unlocking the Potential of Continuous Autoregressive Image Generation with Discrete Tokens Panpan Wang Liqiang Niu Fandong Meng Jinan Xu Yufeng Chen Jie Zhou DiffM 50 0 0 21 Mar 2025
BlockDance: Reuse Structurally Similar Spatio-Temporal Features to Accelerate Diffusion Transformers Hui Zhang Tingwei Gao Jie Shao Zuxuan Wu 69 0 0 20 Mar 2025
Unified Autoregressive Visual Generation and Understanding with Continuous Tokens Lijie Fan Luming Tang Siyang Qin Tianhong Li Xuan S. Yang ... Tao Zhu Michael Rubinstein Michalis Raptis Deqing Sun Radu Soricut 54 4 0 17 Mar 2025
Efficient Multimodal 3D Object Detector via Instance-Level Contrastive Distillation Zhuoqun Su Huimin Lu Shuaifeng Jiao Junhao Xiao Yunhong Wang Xieyuanli Chen 3DPC 56 0 0 17 Mar 2025
UniVG: A Generalist Diffusion Model for Unified Image Generation and Editing Tsu-jui Fu Yusu Qian Chen Chen Wenze Hu Zhe Gan Yuqing Yang 97 1 0 16 Mar 2025
Perceive, Understand and Restore: Real-World Image Super-Resolution with Autoregressive Multimodal Generative Models Hongyang Wei S. Liu C. Yuan L. Zhang 42 0 0 14 Mar 2025
HybridVLA: Collaborative Diffusion and Autoregression in a Unified Vision-Language-Action Model Jiaming Liu Hao Chen Pengju An Zhuoyang Liu Renrui Zhang ... Chengkai Hou Mengdi Zhao KC alex Zhou Pheng-Ann Heng S. Zhang 72 8 0 13 Mar 2025
PlanGen: Towards Unified Layout Planning and Image Generation in Auto-Regressive Vision Language Models Runze He Bo Cheng Yuhang Ma Qingxiang Jia Shanyuan Liu Ao Ma Xiaoyu Wu Liebucha Wu Dawei Leng Yuhui Yin DiffM VLM 54 0 0 13 Mar 2025
Autoregressive Image Generation with Randomized Parallel Decoding Haopeng Li Jinyue Yang Guoqi Li Huan Wang 55 0 0 13 Mar 2025
Piece it Together: Part-Based Concepting with IP-Priors Elad Richardson Kfir Goldberg Yuval Alaluf Daniel Cohen-Or DiffM 66 0 0 13 Mar 2025
A Survey on Knowledge-Oriented Retrieval-Augmented Generation Mingyue Cheng Yucong Luo Jie Ouyang Qiang Liu Huijie Liu ... Bohou Zhang Jiawei Cao Jie Ma Daoyu Wang Enhong Chen 3DV 70 3 0 11 Mar 2025
OmniMamba: Efficient and Unified Multimodal Understanding and Generation via State Space Models Jialv Zou Bencheng Liao Qian Zhang Wenyu Liu Xinggang Wang Mamba MLLM 82 1 0 11 Mar 2025
3D Point Cloud Generation via Autoregressive Up-sampling Ziqiao Meng Qichao Wang Zhipeng Zhou Irwin King Peilin Zhao 3DPC 61 0 0 11 Mar 2025
WISE: A World Knowledge-Informed Semantic Evaluation for Text-to-Image Generation Yuwei Niu Munan Ning Mengren Zheng Bin Lin Peng Jin Jiaqi Liao Kunpeng Ning Bin Zhu Li Yuan EGVM 64 11 0 10 Mar 2025
Unleashing the Potential of Large Language Models for Text-to-Image Generation through Autoregressive Representation Alignment Xing Xie Jiawei Liu Ziyue Lin Huijie Fan Zhi Han Yandong Tang Liangqiong Qu 44 0 0 10 Mar 2025