Chameleon: Mixed-Modal Early-Fusion Foundation Models

16 May 2024

Papers citing "Chameleon: Mixed-Modal Early-Fusion Foundation Models"

50 / 192 papers shown

Title
BREEN: Bridge Data-Efficient Encoder-Free Multimodal Learning with Learnable Queries Tianle Li Yongming Rao Winston Hu Yu Cheng MLLM 61 0 0 16 Mar 2025
Perceive, Understand and Restore: Real-World Image Super-Resolution with Autoregressive Multimodal Generative Models Hongyang Wei S. Liu C. Yuan L. Zhang 42 0 0 14 Mar 2025
Direction-Aware Diagonal Autoregressive Image Generation Yijia Xu Jianzhong Ju Jian Luan J. Cui 45 0 0 14 Mar 2025
GoT: Unleashing Reasoning Capability of Multimodal Large Language Model for Visual Generation and Editing Rongyao Fang Chengqi Duan Kun Wang Linjiang Huang Hao Li ... Xingyu Zeng R. Zhao Jifeng Dai Xihui Liu Hongsheng Li MLLM ReLM LRM 104 5 0 13 Mar 2025
Autoregressive Image Generation with Randomized Parallel Decoding Haopeng Li Jinyue Yang Guoqi Li Huan Wang 53 0 0 13 Mar 2025
Proxy-Tuning: Tailoring Multimodal Autoregressive Models for Subject-Driven Image Generation Yi Wu Lingting Zhu Lei Liu Wandi Qiao Ziqiang Li Lequan Yu Bin Li DiffM 47 0 0 13 Mar 2025
PlanGen: Towards Unified Layout Planning and Image Generation in Auto-Regressive Vision Language Models Runze He Bo Cheng Yuhang Ma Qingxiang Jia Shanyuan Liu Ao Ma Xiaoyu Wu Liebucha Wu Dawei Leng Yuhui Yin DiffM VLM 43 0 0 13 Mar 2025
Neighboring Autoregressive Modeling for Efficient Visual Generation Yefei He Yuanyu He Shaoxuan He Feng Chen Hong Zhou K. Zhang Bohan Zhuang 51 1 0 12 Mar 2025
HaploVL: A Single-Transformer Baseline for Multi-Modal Understanding Rui Yang Lin Song Yicheng Xiao Runhui Huang Yixiao Ge Ying Shan Hengshuang Zhao MLLM 62 0 0 12 Mar 2025
Aligning Text to Image in Diffusion Models is Easier Than You Think J. Lee Byunghee Cha Jeongsol Kim Jong Chul Ye 50 0 0 11 Mar 2025
ProtTeX: Structure-In-Context Reasoning and Editing of Proteins with Large Language Models Zicheng Ma Chuanliu Fan Zhicong Wang Zhenyu Chen Xiaohan Lin Y. Li Shihao Feng Jun Zhang Ziqiang Cao Y. Gao 37 0 0 11 Mar 2025
OmniMamba: Efficient and Unified Multimodal Understanding and Generation via State Space Models Jialv Zou Bencheng Liao Qian Zhang Wenyu Liu Xinggang Wang Mamba MLLM 72 1 0 11 Mar 2025
LMM-R1: Empowering 3B LMMs with Strong Reasoning Abilities Through Two-Stage Rule-Based RL Yingzhe Peng Gongrui Zhang Miaosen Zhang Zhiyuan You Jie Liu Qipeng Zhu Kai Yang Xingzhong Xu Xin Geng Xu Yang LRM ReLM 86 29 0 10 Mar 2025
V2Flow: Unifying Visual Tokenization and Large Language Model Vocabularies for Autoregressive Image Generation Guiwei Zhang Tianyu Zhang Mohan Zhou Yalong Bai Biye Li 59 0 0 10 Mar 2025
Unleashing the Potential of Large Language Models for Text-to-Image Generation through Autoregressive Representation Alignment Xing Xie Jiawei Liu Ziyue Lin Huijie Fan Zhi-Long Han Yandong Tang Liangqiong Qu 34 0 0 10 Mar 2025
Should VLMs be Pre-trained with Image Data? Sedrick Scott Keh Jean-Pierre Mercat S. Gadre Kushal Arora Igor Vasiljevic ... Shuran Song Russ Tedrake Thomas Kollar Ludwig Schmidt Achal Dave VLM 49 0 0 10 Mar 2025
WISE: A World Knowledge-Informed Semantic Evaluation for Text-to-Image Generation Yuwei Niu Munan Ning Mengren Zheng Bin Lin Peng Jin Jiaqi Liao Kunpeng Ning Bin Zhu Li Yuan EGVM 53 10 0 10 Mar 2025
SemHiTok: A Unified Image Tokenizer via Semantic-Guided Hierarchical Codebook for Multimodal Understanding and Generation Z. Chen Chunwei Wang Xiuwei Chen Hang Xu J. Han Xiandan Liang VLM 67 1 0 09 Mar 2025
USP: Unified Self-Supervised Pretraining for Image Generation and Understanding Xiangxiang Chu Renda Li Yong Wang 58 0 0 08 Mar 2025
DoraCycle: Domain-Oriented Adaptation of Unified Generative Model in Multimodal Cycles Rui Zhao Weijia Mao Mike Zheng Shou 64 0 0 05 Mar 2025
Enhancing Spoken Discourse Modeling in Language Models Using Gestural Cues Varsha Suresh Muhammad Hamza Mughal Christian Theobalt Vera Demberg 51 0 0 05 Mar 2025
Teaching Metric Distance to Autoregressive Multimodal Foundational Models Jiwan Chung Saejin Kim Yongrae Jo J. Park Dongjun Min Youngjae Yu 69 0 0 04 Mar 2025
WeGen: A Unified Model for Interactive Multimodal Generation as We Chat Zhipeng Huang Shaobin Zhuang Canmiao Fu Binxin Yang Ying Zhang Chong Sun Zhizheng Zhang Yali Wang Chen Li Zheng-Jun Zha DiffM 65 1 0 03 Mar 2025
MINT: Multi-modal Chain of Thought in Unified Generative Models for Enhanced Image Generation Yi Wang Mushui Liu Wanggui He Longxiang Zhang Z. Huang ... H. Li Weilong Dai Mingli Song Jie Song Hao Jiang MLLM MoE LRM 69 1 0 03 Mar 2025
Multimodal Representation Alignment for Image Generation: Text-Image Interleaved Control Is Easier Than You Think L. Chen S. Bai Wenhao Chai Weichu Xie Haozhe Zhao Leon Vinci Junyang Lin Baobao Chang DiffM 82 4 0 27 Feb 2025
Chitranuvad: Adapting Multi-Lingual LLMs for Multimodal Translation Shaharukh Khan Ayush Tarun Ali Faraz Palash Kamble Vivek Dahiya Praveen Kumar Pokala Ashish Kulkarni Chandra Khatri Abhinav Ravi Shubham Agarwal 44 0 0 27 Feb 2025
FlexVAR: Flexible Visual Autoregressive Modeling without Residual Prediction Siyu Jiao Gengwei Zhang Yinlong Qian Jiancheng Huang Yao Zhao Humphrey Shi Lin Ma Y. X. Wei Zequn Jie VLM 39 1 0 27 Feb 2025
Knowledge Bridger: Towards Training-free Missing Multi-modality Completion Guanzhou Ke Shengfeng He X. Wang Bo Wang Guoqing Chao Y. Zhang Yi Xie HeXing Su 48 0 0 27 Feb 2025
Audio-FLAN: A Preliminary Release Liumeng Xue Ziya Zhou J. Pan Z. Li Shuai Fan ... Haohe Liu Emmanouil Benetos Ge Zhang Yike Guo Wei Xue MLLM AuLLM CLIP VLM 57 1 0 23 Feb 2025
SAE-V: Interpreting Multimodal Models for Enhanced Alignment Hantao Lou Changye Li Jiaming Ji Yaodong Yang 38 0 0 22 Feb 2025
HermesFlow: Seamlessly Closing the Gap in Multimodal Understanding and Generation L. Yang Xinchen Zhang Ye Tian Chenming Shang Minghao Xu Wentao Zhang Bin Cui 88 1 0 17 Feb 2025
GRAPHGPT-O: Synergistic Multimodal Comprehension and Generation on Graphs Yi Fang Bowen Jin Jiacheng Shen Sirui Ding Qiaoyu Tan J. Han 45 1 0 17 Feb 2025
UniMoD: Efficient Unified Multimodal Transformers with Mixture-of-Depths Weijia Mao Z. Yang Mike Zheng Shou MoE 63 0 0 10 Feb 2025
LANTERN++: Enhancing Relaxed Speculative Decoding with Static Tree Drafting for Visual Auto-regressive Models Sihwan Park Doohyuk Jang Sungyub Kim Souvik Kundu Eunho Yang 62 0 0 10 Feb 2025
Rethinking Homogeneity of Vision and Text Tokens in Large Vision-and-Language Models Chia-Wen Kuo Sijie Zhu Fan Chen Xiaohui Shen Longyin Wen VLM 60 1 0 04 Feb 2025
MJ-VIDEO: Fine-Grained Benchmarking and Rewarding Video Preferences in Video Generation Haibo Tong Zhaoyang Wang Z. Chen Haonian Ji Shi Qiu ... Peng Xia Mingyu Ding Rafael Rafailov Chelsea Finn Huaxiu Yao EGVM VGen 76 2 0 03 Feb 2025
Visual Generation Without Guidance Huayu Chen Kai Jiang Kaiwen Zheng Jianfei Chen Hang Su J. Zhu 55 0 0 28 Jan 2025
VARGPT: Unified Understanding and Generation in a Visual Autoregressive Multimodal Large Language Model Xianwei Zhuang Yuxin Xie Yufan Deng Liming Liang Jinghan Ru Yuguo Yin Yuexian Zou MLLM VLM LRM 98 4 0 21 Jan 2025
Democratizing Text-to-Image Masked Generative Models with Compact Text-Aware One-Dimensional Tokens Dongwon Kim Ju He Qihang Yu Chenglin Yang Xiaohui Shen Suha Kwak Liang-Chieh Chen VLM 40 6 0 13 Jan 2025
Generative AI for Cel-Animation: A Survey Yunlong Tang Junjia Guo Pinxin Liu Zhiyuan Wang Hang Hua ... Jing Bi Mingqian Feng X. Li Zeliang Zhang Chenliang Xu VGen 86 7 0 08 Jan 2025
EditAR: Unified Conditional Generation with Autoregressive Models Jiteng Mu Nuno Vasconcelos X. Wang DiffM 38 3 0 08 Jan 2025
Clinical Insights: A Comprehensive Review of Language Models in Medicine Nikita Neveditsin Pawan Lingras V. Mago LM&MA 46 3 0 08 Jan 2025
Automated Generation of Challenging Multiple-Choice Questions for Vision Language Model Evaluation Yuhui Zhang Yuchang Su Yiming Liu Xiaohan Wang James Burgess ... Josiah Aklilu Alejandro Lozano Anjiang Wei Ludwig Schmidt Serena Yeung-Levy 48 3 0 06 Jan 2025
Prepending or Cross-Attention for Speech-to-Text? An Empirical Comparison Tsz Kin Lam Marco Gaido Sara Papi L. Bentivogli Barry Haddow 29 0 0 04 Jan 2025
VisionLLM v2: An End-to-End Generalist Multimodal Large Language Model for Hundreds of Vision-Language Tasks Jiannan Wu Muyan Zhong Sen Xing Zeqiang Lai Zhaoyang Liu ... Lewei Lu Tong Lu Ping Luo Yu Qiao Jifeng Dai MLLM VLM LRM 88 45 0 03 Jan 2025
DrivingGPT: Unifying Driving World Modeling and Planning with Multi-modal Autoregressive Transformers Yuntao Chen Yuqi Wang Zhaoxiang Zhang 56 6 0 24 Dec 2024
Parallelized Autoregressive Visual Generation Y. Wang Shuhuai Ren Zhijie Lin Yujin Han Haoyuan Guo Zhenheng Yang Difan Zou Jiashi Feng Xihui Liu VGen 84 11 0 19 Dec 2024
Next Patch Prediction for Autoregressive Visual Generation Yatian Pang Peng Jin Shuo Yang Bin Lin Bin Zhu ... Liuhan Chen Francis E. H. Tay Ser-Nam Lim Harry Yang Li Yuan 117 8 0 19 Dec 2024
E-CAR: Efficient Continuous Autoregressive Image Generation via Multistage Modeling Zhihang Yuan Yuzhang Shang H. Zhang Tongcheng Fang Rui Xie Bingxin Xu Yan Yan Shengen Yan Guohao Dai Yu Wang DiffM 91 1 0 18 Dec 2024
Performance Gap in Entity Knowledge Extraction Across Modalities in Vision Language Models Ido Cohen Daniela Gottesman Mor Geva Raja Giryes VLM 79 0 1 18 Dec 2024