ShareGPT4V: Improving Large Multi-Modal Models with Better Captions

21 November 2023

Conghui He

Dahua Lin

Papers citing "ShareGPT4V: Improving Large Multi-Modal Models with Better Captions"

50 / 467 papers shown

Title
GalleryGPT: Analyzing Paintings with Large Multimodal Models Yi Bin Wenhao Shi Yujuan Ding Zhiqiang Hu Zheng Wang Yang Yang See-Kiong Ng H. Shen MLLM 30 11 0 01 Aug 2024
SynthVLM: High-Efficiency and High-Quality Synthetic Data for Vision Language Models Zheng Liu Hao Liang Xijie Huang Wentao Xiong Qinhan Yu Linzhuang Sun Chong Chen Conghui He Bin Cui Wentao Zhang SyDa 47 0 0 30 Jul 2024
LLAVADI: What Matters For Multimodal Large Language Models Distillation Shilin Xu Xiangtai Li Haobo Yuan Lu Qi Yunhai Tong Ming-Hsuan Yang 36 3 0 28 Jul 2024
LLaVA-Read: Enhancing Reading Ability of Multimodal Language Models Ruiyi Zhang Yufan Zhou Jian Chen Jiuxiang Gu Changyou Chen Tongfei Sun VLM 41 6 0 27 Jul 2024
Data Processing Techniques for Modern Multimodal Models Yinheng Li Han Ding Hang Chen VLM 29 0 0 27 Jul 2024
Power-LLaVA: Large Language and Vision Assistant for Power Transmission Line Inspection Jiahao Wang Mingxuan Li Haichen Luo Jinguo Zhu A. Yang M. Rong Xiaohua Wang 25 3 0 27 Jul 2024
Dallah: A Dialect-Aware Multimodal Large Language Model for Arabic Fakhraddin Alwajih Gagan Bhatia Muhammad Abdul-Mageed 29 5 0 25 Jul 2024
$VILA^2$ : VILA Augmented VILA Yunhao Fang Ligeng Zhu Yao Lu Yan Wang Pavlo Molchanov Jang Hyun Cho Marco Pavone Song Han Hongxu Yin VLM 39 7 0 24 Jul 2024
INF-LLaVA: Dual-perspective Perception for High-Resolution Multimodal Large Language Model Yiwei Ma Zhibin Wang Xiaoshuai Sun Weihuang Lin Qiang-feng Zhou Jiayi Ji Rongrong Ji MLLM VLM 51 1 0 23 Jul 2024
MMInstruct: A High-Quality Multi-Modal Instruction Tuning Dataset with Extensive Diversity Yangzhou Liu Yue Cao Zhangwei Gao Weiyun Wang Zhe Chen ... Lewei Lu Xizhou Zhu Tong Lu Yu Qiao Jifeng Dai VLM MLLM 52 23 0 22 Jul 2024
LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understanding Haoning Wu Dongxu Li Bei Chen Junnan Li 33 105 0 22 Jul 2024
Evaluating Linguistic Capabilities of Multimodal LLMs in the Lens of Few-Shot Learning Mustafa Dogan .Ilker Kesen Iacer Calixto Aykut Erdem Erkut Erdem LRM 29 1 0 17 Jul 2024
F-HOI: Toward Fine-grained Semantic-Aligned 3D Human-Object Interactions Jie-jin Yang Xuesong Niu Nan Jiang Ruimao Zhang Siyuan Huang 30 9 0 17 Jul 2024
FIRE: A Dataset for Feedback Integration and Refinement Evaluation of Multimodal Models Pengxiang Li Zhi Gao Bofei Zhang Tao Yuan Yuwei Wu Mehrtash Harandi Yunde Jia Song-Chun Zhu Qing Li VLM MLLM 40 3 0 16 Jul 2024
VLMEvalKit: An Open-Source Toolkit for Evaluating Large Multi-Modality Models Haodong Duan Junming Yang Junming Yang Xinyu Fang Lin Chen ... Yuhang Zang Pan Zhang Jiaqi Wang Dahua Lin Kai Chen LM&MA VLM 36 115 0 16 Jul 2024
MAVIS: Mathematical Visual Instruction Tuning Renrui Zhang Xinyu Wei Dongzhi Jiang Yichi Zhang Ziyu Guo ... Aojun Zhou Bin Wei Shanghang Zhang Peng Gao Hongsheng Li MLLM 36 25 0 11 Jul 2024
HiRes-LLaVA: Restoring Fragmentation Input in High-Resolution Large Vision-Language Models Runhui Huang Xinpeng Ding Chunwei Wang J. N. Han Yulong Liu Hengshuang Zhao Hang Xu Lu Hou Wei Zhang Xiaodan Liang VLM 28 8 0 11 Jul 2024
The Synergy between Data and Multi-Modal Large Language Models: A Survey from Co-Development Perspective Zhen Qin Daoyuan Chen Wenhao Zhang Liuyi Yao Yilun Huang Bolin Ding Yaliang Li Shuiguang Deng 57 5 0 11 Jul 2024
DenseFusion-1M: Merging Vision Experts for Comprehensive Multimodal Perception Xiaotong Li Fan Zhang Haiwen Diao Yueze Wang Xinlong Wang Ling-yu Duan VLM 31 25 0 11 Jul 2024
IDA-VLM: Towards Movie Understanding via ID-Aware Large Vision-Language Model Yatai Ji Shilong Zhang Jie Wu Peize Sun Weifeng Chen Xuefeng Xiao Sidi Yang Y. Yang Ping Luo VLM 40 3 0 10 Jul 2024
Multimodal Self-Instruct: Synthetic Abstract Image and Visual Reasoning Instruction Using Language Model Wenqi Zhang Zhenglin Cheng Yuanyu He Mengna Wang Yongliang Shen ... Guiyang Hou Mingqian He Yanna Ma Weiming Lu Yueting Zhuang SyDa 66 9 0 09 Jul 2024
Graph-Based Captioning: Enhancing Visual Descriptions by Interconnecting Region Captions Yu-Guan Hsieh Cheng-Yu Hsieh Shih-Ying Yeh Louis Béthune Hadi Pour Ansari Pavan Kumar Anasosalu Vasu Chun-Liang Li Ranjay Krishna Oncel Tuzel Marco Cuturi 63 4 0 09 Jul 2024
A Single Transformer for Scalable Vision-Language Modeling Yangyi Chen Xingyao Wang Hao Peng Heng Ji LRM 40 14 0 08 Jul 2024
UltraEdit: Instruction-based Fine-Grained Image Editing at Scale Haozhe Zhao Xiaojian Ma Liang Chen Shuzheng Si Rujie Wu Kaikai An Peiyu Yu Minjia Zhang Qing Li Baobao Chang 34 42 0 07 Jul 2024
OmChat: A Recipe to Train Multimodal Language Models with Strong Long Context and Video Understanding Tiancheng Zhao Qianqian Zhang Kyusong Lee Peng Liu Lu Zhang Chunxin Fang Jiajia Liao Kelei Jiang Yibo Ma Ruochen Xu MLLM VLM 51 5 0 06 Jul 2024
A Systematic Survey and Critical Review on Evaluating Large Language Models: Challenges, Limitations, and Recommendations Md Tahmid Rahman Laskar Sawsan Alqahtani M Saiful Bari Mizanur Rahman Mohammad Abdullah Matin Khan ... Chee Wei Tan Md. Rizwan Parvez Enamul Hoque Shafiq R. Joty Jimmy Huang ELM ALM 27 27 0 04 Jul 2024
Lateralization LoRA: Interleaved Instruction Tuning with Modality-Specialized Adaptations Zhiyang Xu Minqian Liu Ying Shen Joy Rimchala Jiaxin Zhang Qifan Wang Yu Cheng Lifu Huang VLM 39 2 0 04 Jul 2024
InternLM-XComposer-2.5: A Versatile Large Vision Language Model Supporting Long-Contextual Input and Output Pan Zhang Xiaoyi Dong Yuhang Zang Yuhang Cao Rui Qian ... Kai Chen Jifeng Dai Yu Qiao Dahua Lin Jiaqi Wang 45 99 0 03 Jul 2024
MIA-Bench: Towards Better Instruction Following Evaluation of Multimodal LLMs Yusu Qian Hanrong Ye J. Fauconnier Peter Grasch Yinfei Yang Zhe Gan 108 13 0 01 Jul 2024
Tarsier: Recipes for Training and Evaluating Large Video Description Models Jiawei Wang Liping Yuan Yuchen Zhang 44 52 0 30 Jun 2024
Investigating and Mitigating the Multimodal Hallucination Snowballing in Large Vision-Language Models Weihong Zhong Xiaocheng Feng Liang Zhao Qiming Li Lei Huang Yuxuan Gu Weitao Ma Yuan Xu Bing Qin MLLM 41 9 0 30 Jun 2024
Web2Code: A Large-scale Webpage-to-Code Dataset and Evaluation Framework for Multimodal LLMs Sukmin Yun Haokun Lin Rusiru Thushara Mohammad Qazim Bhat Yongxin Wang ... Timothy Baldwin Zhengzhong Liu Eric P. Xing Xiaodan Liang Zhiqiang Shen 43 10 0 28 Jun 2024
STLLaVA-Med: Self-Training Large Language and Vision Assistant for Medical Guohao Sun Can Qin Huazhu Fu Linwei Wang Zhiqiang Tao LM&MA 32 3 0 28 Jun 2024
MM-Instruct: Generated Visual Instructions for Large Multimodal Model Alignment Jihao Liu Xin Huang Jinliang Zheng Boxiao Liu Jia Wang Osamu Yoshie Yu Liu Hongsheng Li MLLM SyDa 35 3 0 28 Jun 2024
Solving Token Gradient Conflict in Mixture-of-Experts for Large Vision-Language Model Longrong Yang Dong Shen Chaoxiang Cai Fan Yang Size Li Di Zhang Xi Li MoE 56 2 0 28 Jun 2024
OMG-LLaVA: Bridging Image-level, Object-level, Pixel-level Reasoning and Understanding Tao Zhang Xiangtai Li Hao Fei Haobo Yuan Shengqiong Wu Shunping Ji Chen Change Loy Shuicheng Yan LRM MLLM VLM 49 48 0 27 Jun 2024
HuatuoGPT-Vision, Towards Injecting Medical Visual Knowledge into Multimodal LLMs at Scale Junying Chen Ruyi Ouyang Anningzhe Gao Shunian Chen Guiming Hardy Chen ... Zhenyang Cai Ke Ji Guangjun Yu Xiang Wan Benyou Wang MedIm LM&MA 43 30 0 27 Jun 2024
OmniJARVIS: Unified Vision-Language-Action Tokenization Enables Open-World Instruction Following Agents Zihao Wang Shaofei Cai Zhancun Mu Haowei Lin Ceyao Zhang Xuejie Liu Qing Li Anji Liu Xiaojian Ma Yitao Liang LM&Ro 46 12 0 27 Jun 2024
MATE: Meet At The Embedding -- Connecting Images with Long Texts Young Kyun Jang Junmo Kang Yong Jae Lee Donghyun Kim VLM 38 5 0 26 Jun 2024
Evaluating Fairness in Large Vision-Language Models Across Diverse Demographic Attributes and Prompts Xuyang Wu Yuan Wang Hsin-Tai Wu Zhiqiang Tao Yi Fang VLM 35 8 0 25 Jun 2024
MG-LLaVA: Towards Multi-Granularity Visual Instruction Tuning Xiangyu Zhao Xiangtai Li Haodong Duan Haian Huang Yining Li Kai Chen Hua Yang VLM MLLM 39 10 0 25 Jun 2024
LIPE: Learning Personalized Identity Prior for Non-rigid Image Editing Aoyang Liu Qingnan Fan Shuai Qin Hong Gu Yansong Tang DiffM 58 1 0 25 Jun 2024
Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMs Shengbang Tong Ellis L Brown Penghao Wu Sanghyun Woo Manoj Middepogu ... Xichen Pan Austin Wang Rob Fergus Yann LeCun Saining Xie 3DV MLLM 48 279 0 24 Jun 2024
ResMaster: Mastering High-Resolution Image Generation via Structural and Fine-Grained Guidance Shuwei Shi Wenbo Li Yuechen Zhang Jingwen He Biao Gong Yinqiang Zheng 49 10 0 24 Jun 2024
Prism: A Framework for Decoupling and Assessing the Capabilities of VLMs Yuxuan Qiao Haodong Duan Xinyu Fang Junming Yang Lin Chen Songyang Zhang Jiaqi Wang Dahua Lin Kai Chen LRM 39 18 0 20 Jun 2024
MMBench-Video: A Long-Form Multi-Shot Benchmark for Holistic Video Understanding Xinyu Fang Kangrui Mao Haodong Duan Xiangyu Zhao Yining Li Dahua Lin Kai Chen VLM 55 61 0 20 Jun 2024
IWISDM: Assessing instruction following in multimodal models at scale Xiaoxuan Lei Lucas Gomez Hao Yuan Bai P. Bashivan VLM 33 1 0 20 Jun 2024
VGA: Vision GUI Assistant -- Minimizing Hallucinations through Image-Centric Fine-Tuning Ziyang Meng Yu Dai Zezheng Gong Shaoxiong Guo Minglong Tang Tongquan Wei VLM 32 3 0 20 Jun 2024
From Descriptive Richness to Bias: Unveiling the Dark Side of Generative Image Caption Enrichment Yusuke Hirota Ryo Hachiuma Chao-Han Huck Yang Yuta Nakashima VLM 33 3 0 20 Jun 2024
TroL: Traversal of Layers for Large Language and Vision Models Byung-Kwan Lee Sangyun Chung Chae Won Kim Beomchan Park Yong Man Ro 29 6 0 18 Jun 2024