Chameleon: Mixed-Modal Early-Fusion Foundation Models

16 May 2024

Papers citing "Chameleon: Mixed-Modal Early-Fusion Foundation Models"

42 / 192 papers shown

Title
AuroraCap: Efficient, Performant Video Detailed Captioning and a New Benchmark Wenhao Chai Enxin Song Y. Du Chenlin Meng Vashisht Madhavan Omer Bar-Tal Jeng-Neng Hwang Saining Xie Christopher D. Manning 3DV 61 25 0 04 Oct 2024
LANTERN: Accelerating Visual Autoregressive Models with Relaxed Speculative Decoding Doohyuk Jang Sihwan Park J. Yang Yeonsung Jung Jihun Yun Souvik Kundu Sung-Yub Kim Eunho Yang 33 7 0 04 Oct 2024
From Pixels to Tokens: Byte-Pair Encoding on Quantized Visual Modalities Wanpeng Zhang Zilong Xie Yicheng Feng Yijiang Li Xingrun Xing Sipeng Zheng Zongqing Lu MLLM 18 0 0 03 Oct 2024
A Spark of Vision-Language Intelligence: 2-Dimensional Autoregressive Transformer for Efficient Finegrained Image Generation Liang Chen Sinan Tan Zefan Cai Weichu Xie Haozhe Zhao Yichi Zhang Junyang Lin Jinze Bai Tianyu Liu Baobao Chang ViT 47 3 0 02 Oct 2024
Visual Perception in Text Strings Qi Jia Xiang Yue Shanshan Huang Ziheng Qin Yizhu Liu Bill Yuchen Lin Yang You VLM 34 1 0 02 Oct 2024
Accelerating Auto-regressive Text-to-Image Generation with Training-free Speculative Jacobi Decoding Yao Teng Han Shi Xian Liu Xuefei Ning Guohao Dai Yu Wang Zhenguo Li Xihui Liu 48 10 0 02 Oct 2024
Characterizing and Efficiently Accelerating Multimodal Generation Model Inference Yejin Lee Anna Y. Sun Basil Hosmer Bilge Acun Can Balioglu ... Ram Pasunuru Scott Yih Sravya Popuri Xing Liu Carole-Jean Wu 47 2 0 30 Sep 2024
Multimodal Markup Document Models for Graphic Design Completion Kotaro Kikuchi Naoto Inoue Mayu Otani E. Simo-Serra Kota Yamaguchi VLM 23 4 0 27 Sep 2024
Emu3: Next-Token Prediction is All You Need Xinlong Wang Xiaosong Zhang Zhengxiong Luo Quan-Sen Sun Yufeng Cui ... Xi Yang Jingjing Liu Yonghua Lin Tiejun Huang Zhongyuan Wang MLLM 31 147 0 27 Sep 2024
MIO: A Foundation Model on Multimodal Tokens Zekun Wang King Zhu Chunpu Xu Wangchunshu Zhou Jiaheng Liu ... Yuanxing Zhang Ge Zhang Ke Xu Jie Fu Wenhao Huang MLLM AuLLM 42 11 0 26 Sep 2024
MonoFormer: One Transformer for Both Diffusion and Autoregression Chuyang Zhao Yuxing Song Wenhao Wang Haocheng Feng Errui Ding Yifan Sun Xinyan Xiao Jingdong Wang DiffM 26 17 0 24 Sep 2024
Multi-Modal Generative AI: Multi-modal LLM, Diffusion and Beyond Hong Chen Xin Wang Yuwei Zhou Bin Huang Yipeng Zhang Wei Feng Houlun Chen Zeyang Zhang Siao Tang Wenwu Zhu DiffM 44 7 0 23 Sep 2024
OmniGen: Unified Image Generation Shitao Xiao Yueze Wang Junjie Zhou Huaying Yuan Xingrun Xing Ruiran Yan Shuting Wang Tiejun Huang Zheng Liu DiffM VLM SyDa 44 61 0 17 Sep 2024
Adaptive Large Language Models By Layerwise Attention Shortcuts Prateek Verma Mert Pilanci KELM OffRL 34 0 0 17 Sep 2024
GP-GPT: Large Language Model for Gene-Phenotype Mapping Yanjun Lyu Zihao Wu Lu Zhang Jing Zhang Yiwei Li ... Rongjie Liu Chao Huang Wentao Li Tianming Liu Dajiang Zhu LM&MA 22 3 0 15 Sep 2024
Recall: Empowering Multimodal Embedding for Edge Devices Dongqi Cai Shangguang Wang Chen Peng Zeling Zhang Mengwei Xu 22 3 0 09 Sep 2024
Open-MAGVIT2: An Open-Source Project Toward Democratizing Auto-regressive Visual Generation Zhuoyan Luo Fengyuan Shi Yixiao Ge Yujiu Yang Limin Wang Ying Shan VLM 34 50 0 06 Sep 2024
Wavelet GPT: Wavelet Inspired Large Language Models Prateek Verma AI4TS 13 0 0 04 Sep 2024
From Latent to Engine Manifolds: Analyzing ImageBind's Multimodal Embedding Space Andrew Hamara Pablo Rivas 16 1 0 30 Aug 2024
GlaLSTM: A Concurrent LSTM Stream Framework for Glaucoma Detection via Biomarker Mining Cheng Huang Weizheng Xie Jian Zhou Karanjit S Kooner Karanjit Kooner Yishen Liu 33 1 0 28 Aug 2024
Show-o: One Single Transformer to Unify Multimodal Understanding and Generation Jinheng Xie Weijia Mao Zechen Bai David Junhao Zhang Weihao Wang Kevin Qinghong Lin Yuchao Gu Zhijie Chen Zhenheng Yang Mike Zheng Shou 38 159 0 22 Aug 2024
Transfusion: Predict the Next Token and Diffuse Images with One Multi-Modal Model Chunting Zhou Lili Yu Arun Babu Kushal Tirumala Michihiro Yasunaga Leonid Shamis Jacob Kahn Xuezhe Ma Luke Zettlemoyer Omer Levy DiffM 23 145 0 20 Aug 2024
An End-to-End Model for Photo-Sharing Multi-modal Dialogue Generation Peiming Guo Sinuo Liu Yanzhao Zhang Dingkun Long Pengjun Xie Meishan Zhang M. Zhang DiffM 45 1 0 16 Aug 2024
LLMServingSim: A HW/SW Co-Simulation Infrastructure for LLM Inference Serving at Scale Jaehong Cho Minsu Kim Hyunmin Choi Guseul Heo Jongse Park 30 8 0 10 Aug 2024
Lumina-mGPT: Illuminate Flexible Photorealistic Text-to-Image Generation with Multimodal Generative Pretraining Dongyang Liu Shitian Zhao Le Zhuo Weifeng Lin Yu Qiao Xinyue Li Qi Qin Yu Qiao Hongsheng Li Peng Gao MLLM 58 48 0 05 Aug 2024
On Pre-training of Multimodal Language Models Customized for Chart Understanding Wan-Cyuan Fan Yen-Chun Chen Mengchen Liu Lu Yuan Leonid Sigal 36 4 0 19 Jul 2024
Physics-Inspired Generative Models in Medical Imaging: A Review Dennis Hein Afshin Bozorgpour Dorit Merhof Ge Wang DiffM MedIm AI4CE 21 0 0 15 Jul 2024
Representation Learning and Identity Adversarial Training for Facial Behavior Understanding Mang Ning A. A. Salah Itir Onal Ertugrul CVBM 64 4 0 15 Jul 2024
SEED-Story: Multimodal Long Story Generation with Large Language Model Shuai Yang Yuying Ge Yang Li Yukang Chen Yixiao Ge Ying Shan Yingcong Chen VGen DiffM 73 25 0 11 Jul 2024
A Single Transformer for Scalable Vision-Language Modeling Yangyi Chen Xingyao Wang Hao Peng Heng Ji LRM 40 10 0 08 Jul 2024
ANOLE: An Open, Autoregressive, Native Large Multimodal Models for Interleaved Image-Text Generation Ethan Chern Jiadi Su Yan Ma Pengfei Liu MLLM 21 26 0 08 Jul 2024
Lateralization LoRA: Interleaved Instruction Tuning with Modality-Specialized Adaptations Zhiyang Xu Minqian Liu Ying Shen Joy Rimchala Jiaxin Zhang Qifan Wang Yu Cheng Lifu Huang VLM 31 2 0 04 Jul 2024
MUMU: Bootstrapping Multimodal Image Generation from Text-to-Image Data William Berman A. Peysakhovich 23 4 0 26 Jun 2024
Generative Visual Instruction Tuning Jefferson Hernandez Ruben Villegas Vicente Ordonez VLM 30 3 0 17 Jun 2024
MuirBench: A Comprehensive Benchmark for Robust Multi-image Understanding Fei Wang Xingyu Fu James Y. Huang Zekun Li Qin Liu ... Kai-Wei Chang Dan Roth Sheng Zhang Hoifung Poon Muhao Chen VLM 23 47 0 13 Jun 2024
4M-21: An Any-to-Any Vision Model for Tens of Tasks and Modalities Roman Bachmann Oğuzhan Fatih Kar David Mizrahi Ali Garjani Mingfei Gao David Griffiths Jiaming Hu Afshin Dehghan Amir Zamir MoE VLM MLLM 23 14 0 13 Jun 2024
mOSCAR: A Large-scale Multilingual and Multimodal Document-level Corpus Matthieu Futeral A. Zebaze Pedro Ortiz Suarez Julien Abadji Rémi Lacroix Cordelia Schmid Rachel Bawden Benoît Sagot 29 3 0 13 Jun 2024
Autoregressive Model Beats Diffusion: Llama for Scalable Image Generation Peize Sun Yi Jiang Shoufa Chen Shilong Zhang Bingyue Peng Ping Luo Zehuan Yuan VLM 53 216 0 10 Jun 2024
Towards Semantic Equivalence of Tokenization in Multimodal LLM Shengqiong Wu Hao Fei Xiangtai Li Jiayi Ji Hanwang Zhang Tat-Seng Chua Shuicheng Yan MLLM 55 25 0 07 Jun 2024
SEED-X: Multimodal Models with Unified Multi-granularity Comprehension and Generation Yuying Ge Sijie Zhao Jinguo Zhu Yixiao Ge Kun Yi Lin Song Chen Li Xiaohan Ding Ying Shan VLM 54 103 0 22 Apr 2024
IsoBench: Benchmarking Multimodal Foundation Models on Isomorphic Representations Deqing Fu Ghazal Khalighinejad Ollie Liu Bhuwan Dhingra Dani Yogatama Robin Jia W. Neiswanger 23 14 0 01 Apr 2024
Zero-Shot Text-to-Image Generation Aditya A. Ramesh Mikhail Pavlov Gabriel Goh Scott Gray Chelsea Voss Alec Radford Mark Chen Ilya Sutskever VLM 253 4,735 0 24 Feb 2021