ShareGPT4V: Improving Large Multi-Modal Models with Better Captions

21 November 2023

Conghui He

Dahua Lin

Papers citing "ShareGPT4V: Improving Large Multi-Modal Models with Better Captions"

50 / 467 papers shown

Title
Unveiling Reasoning Thresholds in Language Models: Scaling, Fine-Tuning, and Interpretability through Attention Maps Yen-Che Hsiao Abhishek Dutta LRM ReLM ELM 61 0 0 24 Feb 2025
Chitrarth: Bridging Vision and Language for a Billion People Shaharukh Khan Ayush Tarun Abhinav Ravi Ali Faraz Akshat Patidar Praveen Kumar Pokala Anagha Bhangare Raja Kolla Chandra Khatri Shubham Agarwal VLM 118 1 0 21 Feb 2025
Megrez-Omni Technical Report Boxun Li Yadong Li Z. Li Congyi Liu Weilin Liu ... Dong Zhou Yueqing Zhuang Shengen Yan Guohao Dai Y. Wang 44 0 0 19 Feb 2025
Magma: A Foundation Model for Multimodal AI Agents Jianwei Yang Reuben Tan Qianhui Wu Ruijie Zheng Baolin Peng ... Seonghyeon Ye Joel Jang Yuquan Deng Lars Liden Jianfeng Gao VLM AI4TS 116 9 0 18 Feb 2025
Understanding and Rectifying Safety Perception Distortion in VLMs Xiaohan Zou Jian Kang George Kesidis Lu Lin 163 1 0 18 Feb 2025
Do we Really Need Visual Instructions? Towards Visual Instruction-Free Fine-tuning for Large Vision-Language Models Zikang Liu K. Zhou Wayne Xin Zhao Dawei Gao Yaliang Li Ji-Rong Wen MLLM VLM LRM 85 0 0 17 Feb 2025
Unhackable Temporal Rewarding for Scalable Video MLLMs En Yu Kangheng Lin Liang Zhao Yana Wei Zining Zhu ... Jianjian Sun Zheng Ge X. Zhang Jingyu Wang Wenbing Tao 52 4 0 17 Feb 2025
GAIA: A Global, Multi-modal, Multi-scale Vision-Language Dataset for Remote Sensing Image Analysis Angelos Zavras Dimitrios Michail Xiao Xiang Zhu Begum Demir Ioannis Papoutsis VLM 83 0 0 13 Feb 2025
Effective Black-Box Multi-Faceted Attacks Breach Vision Large Language Model Guardrails Yijun Yang L. Wang Xiao Yang Lanqing Hong Jun Zhu AAML 61 0 0 09 Feb 2025
Rethinking Homogeneity of Vision and Text Tokens in Large Vision-and-Language Models Chia-Wen Kuo Sijie Zhu Fan Chen Xiaohui Shen Longyin Wen VLM 62 1 0 04 Feb 2025
SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-training Tianzhe Chu Yuexiang Zhai Jihan Yang Shengbang Tong Saining Xie Dale Schuurmans Quoc V. Le Sergey Levine Yi-An Ma OffRL 70 56 0 28 Jan 2025
Baichuan-Omni-1.5 Technical Report Yadong Li J. Liu Tao Zhang Tao Zhang S. Chen ... Jianhua Xu Haoze Sun Mingan Lin Zenan Zhou Weipeng Chen AuLLM 72 10 0 28 Jan 2025
StreamingRAG: Real-time Contextual Retrieval and Generation Framework Murugan Sankaradas Ravi K.Rajendran Srimat T.Chakradhar 39 1 0 23 Jan 2025
Omni-RGPT: Unifying Image and Video Region-level Understanding via Token Marks Miran Heo Min-Hung Chen De-An Huang Sifei Liu Subhashree Radhakrishnan Seon Joo Kim Yu-Chun Wang Ryo Hachiuma ObjD VLM 156 2 0 14 Jan 2025
ChartCoder: Advancing Multimodal Large Language Model for Chart-to-Code Generation Xuanle Zhao Xianzhen Luo Qi Shi C. L. P. Chen Shuo Wang Wanxiang Che Zhiyuan Liu Maosong Sun MLLM 54 2 0 11 Jan 2025
MM-GEN: Enhancing Task Performance Through Targeted Multimodal Data Curation S. Joshi Besmira Nushi Vidhisha Balachandran Varun Chandrasekaran Vibhav Vineet Neel Joshi Baharan Mirzasoleiman MLLM VLM 44 0 0 07 Jan 2025
Visual Large Language Models for Generalized and Specialized Applications Yifan Li Zhixin Lai Wentao Bao Zhen Tan Anh Dao Kewei Sui Jiayi Shen Dong Liu Huan Liu Yu Kong VLM 88 11 0 06 Jan 2025
Efficient Architectures for High Resolution Vision-Language Models Miguel Carvalho Bruno Martins MLLM VLM 40 0 0 05 Jan 2025
VisionLLM v2: An End-to-End Generalist Multimodal Large Language Model for Hundreds of Vision-Language Tasks Jiannan Wu Muyan Zhong Sen Xing Zeqiang Lai Zhaoyang Liu ... Lewei Lu Tong Lu Ping Luo Yu Qiao Jifeng Dai MLLM VLM LRM 96 48 0 03 Jan 2025
Altogether: Image Captioning via Re-aligning Alt-text Hu Xu Po-Yao (Bernie) Huang Xiaoqing Ellen Tan Ching-Feng Yeh Jacob Kahn ... Luke Zettlemoyer Wen-tau Yih Shang-Wen Li Saining Xie Christoph Feichtenhofer DiffM 41 6 0 31 Dec 2024
Multimodal Preference Data Synthetic Alignment with Reward Model Robert Wijaya Ngoc-Bao Nguyen Ngai-man Cheung MLLM SyDa 62 2 0 23 Dec 2024
Where am I? Cross-View Geo-localization with Natural Language Descriptions Junyan Ye Honglin Lin Leyan Ou Dairong Chen Zihao Wang Conghui He Weijia Li Weijia Li 76 0 0 22 Dec 2024
A High-Quality Text-Rich Image Instruction Tuning Dataset via Hybrid Instruction Generation Shijie Zhou R. Zhang Yufan Zhou Changyou Chen VLM 77 1 0 20 Dec 2024
I0T: Embedding Standardization Method Towards Zero Modality Gap Na Min An Eunki Kim James Thorne Hyunjung Shim VLM 72 1 0 18 Dec 2024
CA-Edit: Causality-Aware Condition Adapter for High-Fidelity Local Facial Attribute Editing Xiaole Xian Xilin He Zenghao Niu Junliang Zhang Weicheng Xie Siyang Song Zitong Yu L. Shen DiffM 74 0 0 18 Dec 2024
LLaVA-UHD v2: an MLLM Integrating High-Resolution Semantic Pyramid via Hierarchical Window Transformer Yipeng Zhang Y. Liu Zonghao Guo Yidan Zhang Xuesong Yang ... Yuan Yao Zhiyuan Liu Tat-Seng Chua Maosong Sun Maosong Sun MLLM VLM 84 0 0 18 Dec 2024
InternLM-XComposer2.5-OmniLive: A Comprehensive Multimodal System for Long-term Streaming Video and Audio Interactions Pan Zhang Xiaoyi Dong Yuhang Cao Yuhang Zang Rui Qian ... X. Zhang K. Chen Yu Qiao D. Lin Jiaqi Wang KELM 84 12 0 12 Dec 2024
Dynamic-VLM: Simple Dynamic Visual Token Compression for VideoLLM H. Wang Yuxiang Nie Yongjie Ye Deng GuanYu Yanjie Wang Shuai Li Haiyang Yu Jinghui Lu Can Huang VLM MLLM 79 1 0 12 Dec 2024
jina-clip-v2: Multilingual Multimodal Embeddings for Text and Images Andreas Koukounas Georgios Mastrapas Bo Wang Mohammad Kalim Akram Sedigheh Eslami Michael Gunther Isabelle Mohr Saba Sturua Scott Martens Nan Wang VLM 103 7 0 11 Dec 2024
RoboMM: All-in-One Multimodal Large Model for Robotic Manipulation Feng Yan Fanfan Liu Liming Zheng Yufeng Zhong Yiyang Huang Zechao Guan Chengjian Feng Lin Ma 82 2 0 10 Dec 2024
LLaVA-SpaceSGG: Visual Instruct Tuning for Open-vocabulary Scene Graph Generation with Enhanced Spatial Relations Mingjie Xu Mengyang Wu Yuzhi Zhao Jason Chun Lok Li Weifeng Ou LRM SyDa VLM 67 2 0 09 Dec 2024
Chimera: Improving Generalist Model with Domain-Specific Experts Tianshuo Peng M. Li Hongbin Zhou Renqiu Xia Renrui Zhang ... Aojun Zhou Botian Shi Tao Chen Bo Zhang Xiangyu Yue 88 4 0 08 Dec 2024
Florence-VL: Enhancing Vision-Language Models with Generative Vision Encoder and Depth-Breadth Fusion Jiuhai Chen Jianwei Yang Haiping Wu Dianqi Li Jianfeng Gao Tianyi Zhou Bin Xiao VLM 60 4 0 05 Dec 2024
FLAIR: VLM with Fine-grained Language-informed Image Representations Rui Xiao Sanghwan Kim Mariana-Iuliana Georgescu Zeynep Akata Stephan Alaniz VLM CLIP 71 2 0 04 Dec 2024
VLsI: Verbalized Layers-to-Interactions from Large to Small Vision Language Models Byung-Kwan Lee Ryo Hachiuma Yu-Chiang Frank Wang Y. Ro Yueh-Hua Wu VLM 81 0 0 02 Dec 2024
Align-KD: Distilling Cross-Modal Alignment Knowledge for Mobile Vision-Language Model Qianhan Feng Wenshuo Li Tong Lin Xinghao Chen VLM 67 0 0 02 Dec 2024
PainterNet: Adaptive Image Inpainting with Actual-Token Attention and Diverse Mask Control Ruichen Wang Junliang Zhang Qingsong Xie Chen Chen H. Lu DiffM 90 1 0 02 Dec 2024
COSMOS: Cross-Modality Self-Distillation for Vision Language Pre-training Sanghwan Kim Rui Xiao Mariana-Iuliana Georgescu Stephan Alaniz Zeynep Akata VLM 74 1 0 02 Dec 2024
Switti: Designing Scale-Wise Transformers for Text-to-Image Synthesis Anton Voronov Denis Kuznedelev Mikhail Khoroshikh Valentin Khrulkov Dmitry Baranchuk 108 2 0 02 Dec 2024
Beyond Pixels: Text Enhances Generalization in Real-World Image Restoration Haoze Sun W. J. Li J. Liu Kaiwen Zhou Yongqiang Chen Yong Guo Y. Li Renjing Pei Long Peng Y. Yang DiffM 71 1 0 01 Dec 2024
Advancing Myopia To Holism: Fully Contrastive Language-Image Pre-training Haicheng Wang Chen Ju Weixiong Lin Shuai Xiao Mengting Chen ... Mingshuai Yao Jinsong Lan Ying Chen Qingwen Liu Yanfeng Wang VLM CLIP 70 4 0 30 Nov 2024
ForgerySleuth: Empowering Multimodal Large Language Models for Image Manipulation Detection Zhihao Sun Haoran Jiang Haoran Chen Yixin Cao Xipeng Qiu Zuxuan Wu Yu Jiang 70 1 0 29 Nov 2024
On Domain-Specific Post-Training for Multimodal Large Language Models Daixuan Cheng Shaohan Huang Ziyu Zhu Xintong Zhang Wayne Xin Zhao Zhongzhi Luan Bo Dai Zhenliang Zhang VLM 89 2 0 29 Nov 2024
Open-Sora Plan: Open-Source Large Video Generation Model Bin Lin Yunyang Ge Xinhua Cheng Zongjian Li Bin Zhu ... Zhang Pan Xing Zhou Shaoling Dong Yonghong Tian Li-xin Yuan VLM VGen 116 58 0 28 Nov 2024
Detailed Object Description with Controllable Dimensions Xinran Wang H. Zhang Baoteng Li Kongming Liang Hao Sun Zhongjiang He Z. Ma Jun Guo 81 0 0 28 Nov 2024
Orthus: Autoregressive Interleaved Image-Text Generation with Modality-Specific Heads Siqi Kou Jiachun Jin Chang Liu Ye Ma Jian Jia Quan Chen Peng Jiang Zhijie Deng Zhijie Deng DiffM VGen VLM 126 5 0 28 Nov 2024
TimeMarker: A Versatile Video-LLM for Long and Short Video Understanding with Superior Temporal Localization Ability Shimin Chen Xiaohan Lan Yitian Yuan Zequn Jie Lin Ma VLM MLLM 73 13 0 27 Nov 2024
Critic-V: VLM Critics Help Catch VLM Errors in Multimodal Reasoning Di Zhang Jingdi Lei Junxian Li Xunzhi Wang Y. Liu ... S. M. I. Simon X. Yang Jianbo Wu Peng Ye Wanli Ouyang Dongzhan Zhou OffRL LRM 107 6 0 27 Nov 2024
A Topic-level Self-Correctional Approach to Mitigate Hallucinations in MLLMs Lehan He Zeren Chen Zhelun Shi Tianyu Yu Jing Shao Lu Sheng MLLM 111 1 0 26 Nov 2024
Noise Diffusion for Enhancing Semantic Faithfulness in Text-to-Image Synthesis Boming Miao C. Li X. U. Wang Andi Zhang Rui Sun Zizhe Wang Yao Zhu DiffM 74 0 0 25 Nov 2024