Generating Images with Multimodal Language Models

26 May 2023

Daniel Fried

Papers citing "Generating Images with Multimodal Language Models"

49 / 49 papers shown

Title
RAGAR: Retrieval Augment Personalized Image Generation Guided by Recommendation Run Ling W. Wang Yuting Liu G. Guo Linying Jiang Xingwei Wang DiffM 52 0 0 03 May 2025
X-Fusion: Introducing New Modality to Frozen Large Language Models Sicheng Mo Thao Nguyen Xun Huang Siddharth Srinivasan Iyer Yijun Li ... Eli Shechtman Krishna Kumar Singh Yong Jae Lee Bolei Zhou Yuheng Li 71 0 0 29 Apr 2025
Generative Multimodal Pretraining with Discrete Diffusion Timestep Tokens Kaihang Pan Wang Lin Zhongqi Yue Tenglong Ao Liyu Jia Wei Zhao Juncheng Billy Li Siliang Tang Hanwang Zhang 42 1 0 20 Apr 2025
Envisioning Beyond the Pixels: Benchmarking Reasoning-Informed Visual Editing Xiangyu Zhao Peiyuan Zhang Kexian Tang Hao Li Zicheng Zhang Guangtao Zhai Junchi Yan Hua Yang Xue Yang Haodong Duan VLM LRM 41 0 0 03 Apr 2025
FireEdit: Fine-grained Instruction-based Image Editing via Region-aware Vision Language Model Jun Zhou J. Li Zunnan Xu Hanhui Li Yiji Cheng Fa-Ting Hong Qin Lin Qinglin Lu Xiaodan Liang DiffM 65 1 0 25 Mar 2025
Multimodal Chain-of-Thought Reasoning: A Comprehensive Survey Y. Wang Shengqiong Wu Y. Zhang William Yang Wang Ziwei Liu Jiebo Luo Hao Fei LRM 87 7 0 16 Mar 2025
Long-horizon Visual Instruction Generation with Logic and Attribute Self-reflection Yucheng Suo Fan Ma Kaixin Shen Linchao Zhu Yi Yang VLM 47 0 0 12 Mar 2025
WeGen: A Unified Model for Interactive Multimodal Generation as We Chat Zhipeng Huang Shaobin Zhuang Canmiao Fu Binxin Yang Ying Zhang Chong Sun Zhizheng Zhang Yali Wang Chen Li Zheng-Jun Zha DiffM 69 1 0 03 Mar 2025
Traffic Scene Generation from Natural Language Description for Autonomous Vehicles with Large Language Model Bo-Kai Ruan Hao-Tang Tsui Yung-Hui Li Hong-Han Shuai LM&Ro 76 4 0 20 Feb 2025
I Think, Therefore I Diffuse: Enabling Multimodal In-Context Reasoning in Diffusion Models Zhenxing Mi Kuan-Chieh Jackson Wang Guocheng Qian Hanrong Ye Runtao Liu Sergey Tulyakov Kfir Aberman Dan Xu LRM 42 0 0 12 Feb 2025
MedCoDi-M: A Multi-Prompt Foundation Model for Multimodal Medical Data Generation Daniele Molino Francesco Di Feola E. Faiella Deborah Fazzini D. Santucci Linlin Shen V. Guarrasi Paolo Soda SyDa MedIm 39 0 0 10 Jan 2025
VisionLLM v2: An End-to-End Generalist Multimodal Large Language Model for Hundreds of Vision-Language Tasks Jiannan Wu Muyan Zhong Sen Xing Zeqiang Lai Zhaoyang Liu ... Lewei Lu Tong Lu Ping Luo Yu Qiao Jifeng Dai MLLM VLM LRM 91 46 0 03 Jan 2025
DynamicControl: Adaptive Condition Selection for Improved Text-to-Image Generation Q. He Jinlong Peng P. Xu Boyuan Jiang Xiaobin Hu ... Y. Liu Y. Wang Chengjie Wang X. Li J. Zhang DiffM 120 1 0 04 Dec 2024
Spider: Any-to-Many Multimodal LLM Jinxiang Lai Jie Zhang Jun Liu Jian Li Xiaocheng Lu Song Guo MLLM 56 2 0 14 Nov 2024
MMIE: Massive Multimodal Interleaved Comprehension Benchmark for Large Vision-Language Models Peng Xia Siwei Han Shi Qiu Yiyang Zhou Zhaoyang Wang ... Chenhang Cui Mingyu Ding Linjie Li Lijuan Wang Huaxiu Yao 52 10 0 14 Oct 2024
Look Twice Before You Answer: Memory-Space Visual Retracing for Hallucination Mitigation in Multimodal Large Language Models Xin Zou Yizhou Wang Yibo Yan Yuanhuiyi Lyu Kening Zheng ... Junkai Chen Peijie Jiang J. Liu Chang Tang Xuming Hu 78 7 0 04 Oct 2024
MIO: A Foundation Model on Multimodal Tokens Zekun Wang King Zhu Chunpu Xu Wangchunshu Zhou Jiaheng Liu ... Yuanxing Zhang Ge Zhang Ke Xu Jie Fu Wenhao Huang MLLM AuLLM 51 11 0 26 Sep 2024
PixWizard: Versatile Image-to-Image Visual Assistant with Open-Language Instructions Weifeng Lin Xinyu Wei Renrui Zhang Le Zhuo Shitian Zhao ... Junlin Xie Junlin Xie Yu Qiao Peng Gao Hongsheng Li MLLM DiffM 52 10 0 23 Sep 2024
GenCAD: Image-Conditioned Computer-Aided Design Generation with Transformer-Based Contrastive Representation and Diffusion Priors Md Ferdous Alam Faez Ahmed DiffM 31 5 0 08 Sep 2024
Harnessing Multimodal Large Language Models for Multimodal Sequential Recommendation Yuyang Ye Zhi Zheng Yishan Shen Tianshu Wang Hengruo Zhang Peijun Zhu Runlong Yu Kai Zhang Hui Xiong 35 7 0 19 Aug 2024
Lumina-mGPT: Illuminate Flexible Photorealistic Text-to-Image Generation with Multimodal Generative Pretraining Dongyang Liu Shitian Zhao Le Zhuo Weifeng Lin Yu Qiao Xinyue Li Qi Qin Yu Qiao Hongsheng Li Peng Gao MLLM 62 48 0 05 Aug 2024
MIA-Bench: Towards Better Instruction Following Evaluation of Multimodal LLMs Yusu Qian Hanrong Ye J. Fauconnier Peter Grasch Yinfei Yang Zhe Gan 108 13 0 01 Jul 2024
BigCodeBench: Benchmarking Code Generation with Diverse Function Calls and Complex Instructions Terry Yue Zhuo Minh Chien Vu Jenny Chim Han Hu Wenhao Yu ... David Lo Daniel Fried Xiaoning Du H. D. Vries Leandro von Werra 65 128 0 22 Jun 2024
Holistic Evaluation for Interleaved Text-and-Image Generation Minqian Liu Zhiyang Xu Zihao Lin Trevor Ashby Joy Rimchala Jiaxin Zhang Lifu Huang EGVM 36 7 0 20 Jun 2024
CoMM: A Coherent Interleaved Image-Text Dataset for Multimodal Understanding and Generation Wei Chen Lin Li Yongqi Yang Bin Wen Fan Yang Tingting Gao Yu Wu Long Chen VLM VGen 43 6 0 15 Jun 2024
Can I understand what I create? Self-Knowledge Evaluation of Large Language Models Zhiquan Tan Lai Wei Jindong Wang Xing Xie Weiran Huang ELM LRM 35 5 0 10 Jun 2024
Towards Semantic Equivalence of Tokenization in Multimodal LLM Shengqiong Wu Hao Fei Xiangtai Li Jiayi Ji Hanwang Zhang Tat-Seng Chua Shuicheng Yan MLLM 59 31 0 07 Jun 2024
User-Friendly Customized Generation with Multi-Modal Prompts Linhao Zhong Yan Hong Wentao Chen Binglin Zhou Yiyi Zhang Jianfu Zhang Liqing Zhang DiffM 35 0 0 26 May 2024
DEEM: Diffusion Models Serve as the Eyes of Large Language Models for Image Perception Run Luo Yunshui Li Longze Chen Wanwei He Ting-En Lin ... Zikai Song Xiaobo Xia Tongliang Liu Min Yang Binyuan Hui VLM DiffM 70 14 0 24 May 2024
TheaterGen: Character Management with LLM for Consistent Multi-turn Image Generation Junhao Cheng Baiqiao Yin Kaixin Cai Minbin Huang Hanhui Li ... Yue Li Yifei Li Yuhao Cheng Yiqiang Yan Xiaodan Liang DiffM MLLM 32 12 0 29 Apr 2024
ECOR: Explainable CLIP for Object Recognition Ali Rasekh Sepehr Kazemi Ranjbar Milad Heidari Wolfgang Nejdl VLM 33 4 0 19 Apr 2024
DialogGen: Multi-modal Interactive Dialogue System for Multi-turn Text-to-Image Generation Minbin Huang Yanxin Long Xinchi Deng Ruihang Chu Jiangfeng Xiong Xiaodan Liang Hong Cheng Qinglin Lu Wei Liu MLLM EGVM 59 8 0 13 Mar 2024
Exploring the Potential of Large Language Models for Improving Digital Forensic Investigation Efficiency Akila Wickramasekara F. Breitinger Mark Scanlon 42 7 0 29 Feb 2024
StoryGPT-V: Large Language Models as Consistent Story Visualizers Xiaoqian Shen Mohamed Elhoseiny VLM 90 10 0 04 Dec 2023
$M$^{2}$Chat: Empowering VLM for Multimodal LLM Interleaved Text-Image Generation$ M $^{2}$ Chat: Empowering VLM for Multimodal LLM Interleaved Text-Image Generation Xiaowei Chi Rongyu Zhang Zhengkai Jiang Yijiang Liu Ziyi Lin ... Chaoyou Fu Peng Gao Shanghang Zhang Qi-fei Liu Yi-Ting Guo MLLM 33 1 0 29 Nov 2023
LLaVA-Plus: Learning to Use Tools for Creating Multimodal Agents Shilong Liu Hao Cheng Haotian Liu Hao Zhang Feng Li ... Hang Su Jun Zhu Lei Zhang Jianfeng Gao Chun-yue Li MLLM VLM 52 103 0 09 Nov 2023
Beyond Segmentation: Road Network Generation with Multi-Modal LLMs Sumedh Rasal Sanjay K. Boddhu 22 5 0 15 Oct 2023
Towards Semantic Communication Protocols for 6G: From Protocol Learning to Language-Oriented Approaches Jihong Park Seung-Woo Ko Jinho D. Choi Seong-Lyun Kim M. Bennis 26 7 0 14 Oct 2023
Jointly Training Large Autoregressive Multimodal Models Emanuele Aiello L. Yu Yixin Nie Armen Aghajanyan Barlas Oğuz 11 29 0 27 Sep 2023
SEED-Bench: Benchmarking Multimodal LLMs with Generative Comprehension Bohao Li Rui Wang Guangzhi Wang Yuying Ge Yixiao Ge Ying Shan MLLM ELM 16 496 0 30 Jul 2023
A Survey of Safety and Trustworthiness of Large Language Models through the Lens of Verification and Validation Xiaowei Huang Wenjie Ruan Wei Huang Gao Jin Yizhen Dong ... Sihao Wu Peipei Xu Dengyu Wu André Freitas Mustafa A. Mustafa ALM 27 81 0 19 May 2023
Unlimiformer: Long-Range Transformers with Unlimited Length Input Amanda Bertsch Uri Alon Graham Neubig Matthew R. Gormley RALM 94 122 0 02 May 2023
CoBIT: A Contrastive Bi-directional Image-Text Generation Model Haoxuan You Mandy Guo Zhecan Wang Kai-Wei Chang Jason Baldridge Jiahui Yu DiffM 37 12 0 23 Mar 2023
Transformers generalize differently from information stored in context vs in weights Stephanie C. Y. Chan Ishita Dasgupta Junkyung Kim D. Kumaran Andrew Kyle Lampinen Felix Hill 100 45 0 11 Oct 2022
Re-Imagen: Retrieval-Augmented Text-to-Image Generator Wenhu Chen Hexiang Hu Chitwan Saharia William W. Cohen VLM 114 161 0 29 Sep 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 303 11,881 0 04 Mar 2022
Can Wikipedia Help Offline Reinforcement Learning? Machel Reid Yutaro Yamada S. Gu 3DV RALM OffRL 127 95 0 28 Jan 2022
Learning to Prompt for Vision-Language Models Kaiyang Zhou Jingkang Yang Chen Change Loy Ziwei Liu VPVLM CLIP VLM 322 2,249 0 02 Sep 2021
Zero-Shot Text-to-Image Generation Aditya A. Ramesh Mikhail Pavlov Gabriel Goh Scott Gray Chelsea Voss Alec Radford Mark Chen Ilya Sutskever VLM 253 4,764 0 24 Feb 2021