Transfusion: Predict the Next Token and Diffuse Images with One Multi-Modal Model

20 August 2024

Lili Yu

Papers citing "Transfusion: Predict the Next Token and Diffuse Images with One Multi-Modal Model"

50 / 117 papers shown

Title
WISE: A World Knowledge-Informed Semantic Evaluation for Text-to-Image Generation Yuwei Niu Munan Ning Mengren Zheng Bin Lin Peng Jin Jiaqi Liao Kunpeng Ning Bin Zhu Li Yuan EGVM 64 11 0 10 Mar 2025
Unleashing the Potential of Large Language Models for Text-to-Image Generation through Autoregressive Representation Alignment Xing Xie Jiawei Liu Ziyue Lin Huijie Fan Zhi Han Yandong Tang Liangqiong Qu 44 0 0 10 Mar 2025
LMM-R1: Empowering 3B LMMs with Strong Reasoning Abilities Through Two-Stage Rule-Based RL Yingzhe Peng Gongrui Zhang Miaosen Zhang Zhiyuan You Jie Liu Qipeng Zhu Kai Yang Xingzhong Xu Xin Geng Xu Yang LRM ReLM 90 33 0 10 Mar 2025
SemHiTok: A Unified Image Tokenizer via Semantic-Guided Hierarchical Codebook for Multimodal Understanding and Generation Z. Chen Chunwei Wang Xiuwei Chen Hang Xu J. Han Xiandan Liang VLM 71 1 0 09 Mar 2025
Unlocking Pretrained LLMs for Motion-Related Multimodal Generation: A Fine-Tuning Approach to Unify Diffusion and Next-Token Prediction Shinichi Tanaka Zhao Wang Yoichi Kato Jun Ohya DiffM 51 0 0 08 Mar 2025
Frequency Autoregressive Image Generation with Continuous Tokens Hu Yu Hao Luo Hangjie Yuan Yu Rong Feng Zhao VGen 44 2 0 07 Mar 2025
DoraCycle: Domain-Oriented Adaptation of Unified Generative Model in Multimodal Cycles Rui Zhao Weijia Mao Mike Zheng Shou 66 0 0 05 Mar 2025
MINT: Multi-modal Chain of Thought in Unified Generative Models for Enhanced Image Generation Yi Wang Mushui Liu Wanggui He Longxiang Zhang Z. Huang ... Hao Li Weilong Dai Mingli Song Jie Song Hao Jiang MLLM MoE LRM 80 1 0 03 Mar 2025
Direct Discriminative Optimization: Your Likelihood-Based Visual Generative Model is Secretly a GAN Discriminator Kaiwen Zheng Yongxin Chen Huayu Chen Guande He Xuan Li Jun Zhu Qinsheng Zhang DiffM 49 0 0 03 Mar 2025
FlexVAR: Flexible Visual Autoregressive Modeling without Residual Prediction Siyu Jiao Gengwei Zhang Yinlong Qian Jiancheng Huang Yao Zhao Humphrey Shi Lin Ma Y. X. Wei Zequn Jie VLM 49 1 0 27 Feb 2025
HermesFlow: Seamlessly Closing the Gap in Multimodal Understanding and Generation L. Yang Xinchen Zhang Ye Tian Chenming Shang Minghao Xu Wentao Zhang Bin Cui 102 1 0 17 Feb 2025
UniMoD: Efficient Unified Multimodal Transformers with Mixture-of-Depths Weijia Mao Z. Yang Mike Zheng Shou MoE 78 0 0 10 Feb 2025
Generative AI for Cel-Animation: A Survey Yunlong Tang Junjia Guo Pinxin Liu Zhiyuan Wang Hang Hua ... Jing Bi Mingqian Feng Xuzhao Li Zeliang Zhang Chenliang Xu VGen 88 7 0 08 Jan 2025
EditAR: Unified Conditional Generation with Autoregressive Models Jiteng Mu Nuno Vasconcelos Xinyu Wang DiffM 43 4 0 08 Jan 2025
CAT: Content-Adaptive Image Tokenization Junhong Shen Kushal Tirumala Michihiro Yasunaga Ishan Misra Luke Zettlemoyer Lili Yu Chunting Zhou 35 0 0 06 Jan 2025
HoVLE: Unleashing the Power of Monolithic Vision-Language Models with Holistic Vision-Language Embedding Chenxin Tao Shiqian Su X. Zhu Chenyu Zhang Zhe Chen ... Wenhai Wang Lewei Lu Gao Huang Yu Qiao Jifeng Dai MLLM VLM 104 2 0 20 Dec 2024
Next Patch Prediction for Autoregressive Visual Generation Yatian Pang Peng Jin Shuo Yang Bin Lin Bin Zhu ... Liuhan Chen Francis E. H. Tay Ser-Nam Lim Harry Yang Li Yuan 120 9 0 19 Dec 2024
E-CAR: Efficient Continuous Autoregressive Image Generation via Multistage Modeling Zhihang Yuan Yuzhang Shang H. Zhang Tongcheng Fang Rui Xie Bingxin Xu Yan Yan Shengen Yan Guohao Dai Yu Wang DiffM 100 1 0 18 Dec 2024
SoftVQ-VAE: Efficient 1-Dimensional Continuous Tokenizer H. Chen Z. Wang Xianrui Li Xingchen Sun Fangyi Chen Jiang Liu J. Wang Bhiksha Raj Zicheng Liu Emad Barsoum VLM 114 6 0 14 Dec 2024
Olympus: A Universal Task Router for Computer Vision Tasks Yuanze Lin Yunsheng Li Dongdong Chen Weijian Xu Ronald Clark Philip H. S. Torr VLM ObjD 197 0 0 12 Dec 2024
UniReal: Universal Image Generation and Editing via Learning Real-world Dynamics Xi Chen Zhifei Zhang He Zhang Yuqian Zhou S. Kim ... Nanxuan Zhao Yilin Wang Hui Ding Zhe Lin Hengshuang Zhao VGen DiffM 123 21 0 10 Dec 2024
From Slow Bidirectional to Fast Autoregressive Video Diffusion Models Tianwei Yin Qiang Zhang Richard Zhang William T. Freeman F. Durand Eli Shechtman Xun Huang VGen DiffM 81 5 0 10 Dec 2024
[MASK] is All You Need Vincent Tao Hu Bjorn Ommer DiffM 137 2 0 09 Dec 2024
MuMu-LLaMA: Multi-modal Music Understanding and Generation via Large Language Models Shansong Liu Atin Sakkeer Hussain Qilong Wu Chenshuo Sun Ying Shan AuLLM 69 3 0 09 Dec 2024
Diffusion-VLA: Scaling Robot Foundation Models via Unified Diffusion and Autoregression Junjie Wen Minjie Zhu Bo Li Zhibin Tang Jinming Li ... Chengmeng Li Xiaoyu Liu Yaxin Peng Chaomin Shen Feifei Feng 88 15 0 04 Dec 2024
RandAR: Decoder-only Autoregressive Visual Generation in Random Orders Ziqi Pang Tianyuan Zhang Fujun Luan Yunze Man Hao Tan Kai Zhang William T. Freeman Yu-Xiong Wang VGen 76 14 0 02 Dec 2024
XQ-GAN: An Open-source Image Tokenization Framework for Autoregressive Generation Xianrui Li Kai Qiu H. Chen Jason Kuen Jiuxiang Gu J. Wang Zhe-nan Lin Bhiksha Raj VLM 125 3 0 02 Dec 2024
Unleashing In-context Learning of Autoregressive Models for Few-shot Image Manipulation Bolin Lai F. Xu Miao Liu Xiaoliang Dai Nikhil Mehta ... Zeyi Huang James M. Rehg Sangmin Lee Ning Zhang Tong Xiao 73 2 0 02 Dec 2024
OmniFlow: Any-to-Any Generation with Multi-Modal Rectified Flows Shufan Li Konstantinos Kallidromitis Akash Gokul Zichun Liao Yusuke Kato Kazuki Kozuka Aditya Grover VGen 95 5 0 02 Dec 2024
Puzzle: Distillation-Based NAS for Inference-Optimized LLMs Akhiad Bercovich Tomer Ronen Talor Abramovich Nir Ailon Nave Assaf ... Ido Shahaf Oren Tropp Omer Ullman Argov Ran Zilberstein Ran El-Yaniv 77 1 0 28 Nov 2024
Orthus: Autoregressive Interleaved Image-Text Generation with Modality-Specific Heads Siqi Kou Jiachun Jin Chang Liu Ye Ma Jian Jia Quan Chen Peng Jiang Zhijie Deng Zhijie Deng DiffM VGen VLM 135 6 0 28 Nov 2024
Collaborative Decoding Makes Visual Auto-Regressive Modeling Efficient Zigeng Chen Xinyin Ma Gongfan Fang Xinchao Wang VLM 89 4 0 26 Nov 2024
One Diffusion to Generate Them All Duong H. Le Tuan Pham Sangho Lee Christopher Clark Aniruddha Kembhavi Stephan Mandt Ranjay Krishna Jiasen Lu VLM 74 5 0 25 Nov 2024
Artificial Intelligence for Biomedical Video Generation Linyuan Li Jianing Qiu Anujit Saha Lin Li Poyuan Li Mengxian He Ziyu Guo Wu Yuan VGen 63 1 0 12 Nov 2024
Autoregressive Models in Vision: A Survey Jing Xiong Gongye Liu Lun Huang Chengyue Wu Taiqiang Wu ... M. Zhang Guillermo Sapiro Jiebo Luo Ping Luo Ngai Wong VGen 48 9 0 08 Nov 2024
Analyzing The Language of Visual Tokens David M. Chan Rodolfo Corona J. S. Park Cheol Jun Cho Yutong Bai Trevor Darrell 23 2 0 07 Nov 2024
GameGen-X: Interactive Open-world Game Video Generation Haoxuan Che Xuanhua He Quande Liu C. Jin Hao Chen VGen 62 17 0 01 Nov 2024
Towards Unifying Understanding and Generation in the Era of Vision Foundation Models: A Survey from the Autoregression Perspective Shenghao Xie Wenqiang Zu Mingyang Zhao Duo Su Shilong Liu Ruohua Shi Guoqi Li Shanghang Zhang Lei Ma LRM 44 3 0 29 Oct 2024
Unbounded: A Generative Infinite Game of Character Life Simulation Jialu Li Yuanzhen Li Neal Wadhwa Yael Pritch David E. Jacobs Michael Rubinstein Joey Tianyi Zhou Nataniel Ruiz VGen AI4CE 36 4 0 24 Oct 2024
Infinity-MM: Scaling Multimodal Performance with Large-Scale and High-Quality Instruction Data Shuhao Gu Jialing Zhang Siyuan Zhou Kevin Yu Zhaohu Xing ... Yufeng Cui Xinlong Wang Yaoqi Liu Fangxiang Feng Guang Liu SyDa VLM MLLM 32 17 0 24 Oct 2024
Fluid: Scaling Autoregressive Text-to-image Generative Models with Continuous Tokens Lijie Fan Tianhong Li Siyang Qin Yuanzhen Li Chen Sun Michael Rubinstein Deqing Sun Kaiming He Yonglong Tian VLM DiffM 48 41 0 17 Oct 2024
PUMA: Empowering Unified MLLM with Multi-granular Visual Generation Rongyao Fang Chengqi Duan Kun Wang Hao Li H. Tian Xingyu Zeng Rui Zhao Jifeng Dai Hongsheng Li Xihui Liu MLLM 36 11 0 17 Oct 2024
$γ-$ MoD: Exploring Mixture-of-Depth Adaptation for Multimodal Large Language Models Yaxin Luo Gen Luo Jiayi Ji Yiyi Zhou Xiaoshuai Sun Zhiqiang Shen Rongrong Ji VLM MoE 42 1 0 17 Oct 2024
Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generation Chengyue Wu Xiaokang Chen Z. F. Wu Yiyang Ma Xingchao Liu ... Wen Liu Zhenda Xie Xingkai Yu Chong Ruan Ping Luo AI4TS 57 74 0 17 Oct 2024
MEV Capture Through Time-Advantaged Arbitrage Robin Fritsch Maria Ines Silva A. Mamageishvili Benjamin Livshits E. Felten 36 1 0 14 Oct 2024
MMIE: Massive Multimodal Interleaved Comprehension Benchmark for Large Vision-Language Models Peng Xia Siwei Han Shi Qiu Yiyang Zhou Zhaoyang Wang ... Chenhang Cui Mingyu Ding Linjie Li Lijuan Wang Huaxiu Yao 54 10 0 14 Oct 2024
Toward Guidance-Free AR Visual Generation via Condition Contrastive Alignment Huayu Chen Hang Su Peize Sun Jun Zhu VLM 48 3 0 12 Oct 2024
CryoFM: A Flow-based Foundation Model for Cryo-EM Densities Yi Zhou Yilai Li Jing Yuan Quanquan Gu 33 1 0 11 Oct 2024
DART: Denoising Autoregressive Transformer for Scalable Text-to-Image Generation Jiatao Gu Yuyang Wang Yizhe Zhang Qihang Zhang Dinghuai Zhang Navdeep Jaitly Josh Susskind Shuangfei Zhai DiffM 33 12 0 10 Oct 2024
Mono-InternVL: Pushing the Boundaries of Monolithic Multimodal Large Language Models with Endogenous Visual Pre-training Gen Luo Xue Yang Wenhan Dou Zhaokai Wang Jifeng Dai Jifeng Dai Yu Qiao Xizhou Zhu VLM MLLM 65 25 0 10 Oct 2024