Chameleon: Mixed-Modal Early-Fusion Foundation Models

16 May 2024

Papers citing "Chameleon: Mixed-Modal Early-Fusion Foundation Models"

50 / 192 papers shown

Title
IDEA-Bench: How Far are Generative Models from Professional Designing? C. Liang Lianghua Huang Jingwu Fang Huanzhang Dou Wei Wang Zhi-Fan Wu Yupeng Shi Junge Zhang Xin Zhao Yu Liu 3DV 77 1 0 16 Dec 2024
Towards Unified Benchmark and Models for Multi-Modal Perceptual Metrics Sara Ghazanfari Siddharth Garg Nicolas Flammarion P. Krishnamurthy Farshad Khorrami Francesco Croce VLM 84 0 0 13 Dec 2024
Owl-1: Omni World Model for Consistent Long Video Generation Yuanhui Huang Wenzhao Zheng Yuan Gao Xin Tao Pengfei Wan Di Zhang Jie Zhou Jiwen Lu VGen VLM 82 0 0 12 Dec 2024
Olympus: A Universal Task Router for Computer Vision Tasks Yuanze Lin Yunsheng Li Dongdong Chen Weijian Xu Ronald Clark Philip H. S. Torr VLM ObjD 107 0 0 12 Dec 2024
[MASK] is All You Need Vincent Tao Hu Bjorn Ommer DiffM 135 2 0 09 Dec 2024
MuMu-LLaMA: Multi-modal Music Understanding and Generation via Large Language Models Shansong Liu Atin Sakkeer Hussain Qilong Wu Chenshuo Sun Ying Shan AuLLM 61 3 0 09 Dec 2024
Diffusion-VLA: Scaling Robot Foundation Models via Unified Diffusion and Autoregression Junjie Wen Minjie Zhu Y. X. Zhu Zhibin Tang Jinming Li ... Chengmeng Li Xiaoyu Liu Yaxin Peng Chaomin Shen Feifei Feng 85 13 0 04 Dec 2024
RandAR: Decoder-only Autoregressive Visual Generation in Random Orders Ziqi Pang Tianyuan Zhang Fujun Luan Yunze Man Hao Tan Kai Zhang William T. Freeman Yu-Xiong Wang VGen 61 8 0 02 Dec 2024
Unleashing In-context Learning of Autoregressive Models for Few-shot Image Manipulation Bolin Lai F. Xu Miao Liu Xiaoliang Dai Nikhil Mehta ... Zeyi Huang James M. Rehg Sangmin Lee Ning Zhang Tong Xiao 71 2 0 02 Dec 2024
OmniFlow: Any-to-Any Generation with Multi-Modal Rectified Flows Shufan Li Konstantinos Kallidromitis Akash Gokul Zichun Liao Yusuke Kato Kazuki Kozuka Aditya Grover VGen 87 5 0 02 Dec 2024
On Domain-Specific Post-Training for Multimodal Large Language Models Daixuan Cheng Shaohan Huang Ziyu Zhu Xintong Zhang Wayne Xin Zhao Zhongzhi Luan Bo Dai Zhenliang Zhang VLM 87 2 0 29 Nov 2024
Orthus: Autoregressive Interleaved Image-Text Generation with Modality-Specific Heads Siqi Kou Jiachun Jin Chang Liu Ye Ma Jian Jia Quan Chen Peng Jiang Zhijie Deng Zhijie Deng DiffM VGen VLM 105 5 0 28 Nov 2024
One Diffusion to Generate Them All Duong H. Le Tuan Pham Sangho Lee Christopher Clark Aniruddha Kembhavi Stephan Mandt Ranjay Krishna Jiasen Lu VLM 59 5 0 25 Nov 2024
Efficient Online Inference of Vision Transformers by Training-Free Tokenization Leonidas Gee Wing Yan Li V. Sharmanska Novi Quadrianto ViT 79 0 0 23 Nov 2024
Panther: Illuminate the Sight of Multimodal LLMs with Instruction-Guided Visual Prompts Honglin Li Yuting Gao Chenglu Zhu Jingdong Chen M. Yang Lin Yang MLLM 79 0 0 21 Nov 2024
Enhancing the Reasoning Ability of Multimodal Large Language Models via Mixed Preference Optimization Weiyun Wang Zhe Chen Wenhai Wang Yue Cao Yangzhou Liu ... Jinguo Zhu X. Zhu Lewei Lu Yu Qiao Jifeng Dai LRM 52 45 1 15 Nov 2024
Jailbreak Attacks and Defenses against Multimodal Generative Models: A Survey Xuannan Liu Xing Cui Peipei Li Zekun Li Huaibo Huang Shuhan Xia Miaoxuan Zhang Yueying Zou Ran He AAML 53 4 0 14 Nov 2024
Autoregressive Models in Vision: A Survey Jing Xiong Gongye Liu Lun Huang Chengyue Wu Taiqiang Wu ... M. Zhang Guillermo Sapiro Jiebo Luo Ping Luo Ngai Wong VGen 46 9 0 08 Nov 2024
Analyzing The Language of Visual Tokens David M. Chan Rodolfo Corona J. S. Park Cheol Jun Cho Yutong Bai Trevor Darrell 21 2 0 07 Nov 2024
Clustering in Causal Attention Masking Nikita Karagodin Yury Polyanskiy Philippe Rigollet 52 5 0 07 Nov 2024
The Semantic Hub Hypothesis: Language Models Share Semantic Representations Across Languages and Modalities Zhaofeng Wu Xinyan Velocity Yu Dani Yogatama Jiasen Lu Yoon Kim AIFin 41 10 0 07 Nov 2024
Addressing Representation Collapse in Vector Quantized Models with One Linear Layer Yongxin Zhu B. Li Yifei Xin Linli Xu 30 10 0 04 Nov 2024
Randomized Autoregressive Visual Generation Qihang Yu Ju He XueQing Deng Xiaohui Shen Liang-Chieh Chen VGen DiffM 50 28 1 01 Nov 2024
Replace-then-Perturb: Targeted Adversarial Attacks With Visual Reasoning for Vision-Language Models Jonggyu Jang Hyeonsu Lyu Jungyeon Koh H. Yang VLM AAML 26 0 0 01 Nov 2024
Towards Unifying Understanding and Generation in the Era of Vision Foundation Models: A Survey from the Autoregression Perspective Shenghao Xie Wenqiang Zu Mingyang Zhao Duo Su Shilong Liu Ruohua Shi Guoqi Li Shanghang Zhang Lei Ma LRM 40 3 0 29 Oct 2024
Unbounded: A Generative Infinite Game of Character Life Simulation Jialu Li Yuanzhen Li Neal Wadhwa Yael Pritch David E. Jacobs Michael Rubinstein Mohit Bansal Nataniel Ruiz VGen AI4CE 26 4 0 24 Oct 2024
Where Am I and What Will I See: An Auto-Regressive Model for Spatial Localization and View Prediction Junyi Chen Di Huang Weicai Ye Wanli Ouyang Tong He LRM 30 1 0 24 Oct 2024
Mini-InternVL: A Flexible-Transfer Pocket Multimodal Model with 5% Parameters and 90% Performance Zhangwei Gao Zhe Chen Erfei Cui Yiming Ren Weiyun Wang ... Lewei Lu Tong Lu Yu Qiao Jifeng Dai Wenhai Wang VLM 62 22 0 21 Oct 2024
Elucidating the design space of language models for image generation Xuantong Liu Shaozhe Hao Xianbiao Qi Tianyang Hu Jun Wang Rong Xiao Yuan Yao VLM 27 3 0 21 Oct 2024
Ichigo: Mixed-Modal Early-Fusion Realtime Voice Assistant Alan Dao Dinh Bach Vu Huy Hoang Ha AuLLM VLM 57 3 0 20 Oct 2024
Efficient Vision-Language Models by Summarizing Visual Tokens into Compact Registers Yuxin Wen Qingqing Cao Qichen Fu Sachin Mehta Mahyar Najibi VLM 22 4 0 17 Oct 2024
PUMA: Empowering Unified MLLM with Multi-granular Visual Generation Rongyao Fang Chengqi Duan Kun Wang Hao Li H. Tian Xingyu Zeng Rui Zhao Jifeng Dai Hongsheng Li Xihui Liu MLLM 34 11 0 17 Oct 2024
Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generation Chengyue Wu Xiaokang Chen Z. F. Wu Yiyang Ma Xingchao Liu ... Wen Liu Zhenda Xie Xingkai Yu Chong Ruan Ping Luo AI4TS 46 70 0 17 Oct 2024
Latent Action Pretraining from Videos Seonghyeon Ye Joel Jang Byeongguk Jeon Sejune Joo Jianwei Yang ... Kimin Lee Jianfeng Gao Luke Zettlemoyer Dieter Fox Minjoon Seo 30 19 0 15 Oct 2024
MLLM can see? Dynamic Correction Decoding for Hallucination Mitigation Chenxi Wang Xiang Chen N. Zhang Bozhong Tian Haoming Xu Shumin Deng H. Chen MLLM LRM 23 4 0 15 Oct 2024
MEV Capture Through Time-Advantaged Arbitrage Robin Fritsch Maria Ines Silva A. Mamageishvili Benjamin Livshits E. Felten 23 5 0 14 Oct 2024
Spatial-Aware Efficient Projector for MLLMs via Multi-Layer Feature Aggregation Shun Qian Bingquan Liu Chengjie Sun Zhen Xu Baoxun Wang 26 0 0 14 Oct 2024
MMIE: Massive Multimodal Interleaved Comprehension Benchmark for Large Vision-Language Models Peng Xia Siwei Han Shi Qiu Yiyang Zhou Zhaoyang Wang ... Chenhang Cui Mingyu Ding Linjie Li Lijuan Wang Huaxiu Yao 45 10 0 14 Oct 2024
Locality Alignment Improves Vision-Language Models Ian Covert Tony Sun James Y. Zou Tatsunori Hashimoto VLM 58 3 0 14 Oct 2024
Toward Guidance-Free AR Visual Generation via Condition Contrastive Alignment Huayu Chen Hang Su Peize Sun J. Zhu VLM 27 3 0 12 Oct 2024
ZipVL: Efficient Large Vision-Language Models with Dynamic Token Sparsification and KV Cache Compression Yefei He Feng Chen Jing Liu Wenqi Shao Hong Zhou K. Zhang Bohan Zhuang VLM 41 11 0 11 Oct 2024
Scaling Laws For Diffusion Transformers Zhengyang Liang Hao He Ceyuan Yang Bo Dai 21 8 0 10 Oct 2024
DART: Denoising Autoregressive Transformer for Scalable Text-to-Image Generation Jiatao Gu Yuyang Wang Yizhe Zhang Qihang Zhang Dinghuai Zhang Navdeep Jaitly Josh Susskind Shuangfei Zhai DiffM 31 12 0 10 Oct 2024
Unsupervised Data Validation Methods for Efficient Model Training Yurii Paniv 20 1 0 10 Oct 2024
Mono-InternVL: Pushing the Boundaries of Monolithic Multimodal Large Language Models with Endogenous Visual Pre-training Gen Luo Xue Yang Wenhan Dou Zhaokai Wang Jifeng Dai Jifeng Dai Yu Qiao Xizhou Zhu VLM MLLM 49 25 0 10 Oct 2024
To Preserve or To Compress: An In-Depth Study of Connector Selection in Multimodal Large Language Models Junyan Lin Haoran Chen Dawei Zhu Xiaoyu Shen 20 1 0 09 Oct 2024
CAR: Controllable Autoregressive Modeling for Visual Generation Ziyu Yao Jialin Li Yifeng Zhou Yong Liu Xi Jiang Chengjie Wang Feng Zheng Yuexian Zou Lei Li DiffM 35 13 0 07 Oct 2024
TLDR: Token-Level Detective Reward Model for Large Vision Language Models Deqing Fu Tong Xiao Rui Wang Wang Zhu Pengchuan Zhang Guan Pang Robin Jia Lawrence Chen 55 5 0 07 Oct 2024
Gradient-based Jailbreak Images for Multimodal Fusion Models Javier Rando Hannah Korevaar Erik Brinkman Ivan Evtimov Florian Tramèr AAML 24 3 0 04 Oct 2024
DocKD: Knowledge Distillation from LLMs for Open-World Document Understanding Models Sungnyun Kim Haofu Liao Srikar Appalaraju Peng Tang Zhuowen Tu R. Satzoda R. Manmatha Vijay Mahadevan Stefano Soatto 34 0 0 04 Oct 2024