Visual Instruction Tuning with Polite Flamingo

3 July 2023

Papers citing "Visual Instruction Tuning with Polite Flamingo"

37 / 37 papers shown

Title
MLLM-Selector: Necessity and Diversity-driven High-Value Data Selection for Enhanced Visual Instruction Tuning Yiwei Ma Guohai Xu Xiaoshuai Sun Jiayi Ji Jie Lou Debing Zhang Rongrong Ji 84 0 0 26 Mar 2025
A Survey on fMRI-based Brain Decoding for Reconstructing Multimodal Stimuli Pengyu Liu Guohua Dong D. Guo Kun Li Fengling Li Xun Yang Meng Wang Xiaomin Ying AI4CE 38 0 0 20 Mar 2025
From Head to Tail: Towards Balanced Representation in Large Vision-Language Models through Adaptive Data Calibration Mingyang Song Xiaoye Qu Jiawei Zhou Yu-Xi Cheng VLM 41 1 0 17 Mar 2025
Adversarial Training for Multimodal Large Language Models against Jailbreak Attacks Liming Lu Shuchao Pang Siyuan Liang Haotian Zhu Xiyu Zeng Aishan Liu Yunhuai Liu Yongbin Zhou AAML 49 1 0 05 Mar 2025
Playing Devil's Advocate: Unmasking Toxicity and Vulnerabilities in Large Vision-Language Models Abdulkadir Erol Trilok Padhi Agnik Saha Ugur Kursuncu Mehmet Emin Aktas 37 0 0 17 Jan 2025
Learning to Correction: Explainable Feedback Generation for Visual Commonsense Reasoning Distractor Jiali Chen Xusen Hei Yuqi Xue Yuancheng Wei Jiayuan Xie Yi Cai Qing Li MLLM LRM 58 4 0 08 Dec 2024
MLAN: Language-Based Instruction Tuning Improves Zero-Shot Generalization of Multimodal Large Language Models Jianhong Tu Zhuohao Ni Nicholas Crispino Zihao Yu Michael Bendersky ... Ruoxi Jia Xin Liu Lingjuan Lyu Dawn Song Chenguang Wang VLM MLLM 45 0 0 15 Nov 2024
SemiHVision: Enhancing Medical Multimodal Models with a Semi-Human Annotated Dataset and Fine-Tuned Instruction Generation Junda Wang Yujan Ting Eric Z. Chen Hieu Tran Hong-ye Yu Weijing Huang Terrence Chen VLM LM&MA 25 1 0 19 Oct 2024
Can visual language models resolve textual ambiguity with visual cues? Let visual puns tell you! Jiwan Chung Seungwon Lim Jaehyun Jeon Seungbeen Lee Youngjae Yu 18 0 0 01 Oct 2024
Learning to Localize Actions in Instructional Videos with LLM-Based Multi-Pathway Text-Video Alignment Yuxiao Chen K. Li Wentao Bao Deep Patel Yu Kong Martin Renqiang Min Dimitris N. Metaxas DiffM 13 1 0 22 Sep 2024
Enhancing Perception of Key Changes in Remote Sensing Image Change Captioning Cong Yang Zuchao Li Hongzan Jiao Zhi Gao Lefei Zhang 29 1 0 19 Sep 2024
From Linguistic Giants to Sensory Maestros: A Survey on Cross-Modal Reasoning with Large Language Models Shengsheng Qian Zuyi Zhou Dizhan Xue Bing Wang Changsheng Xu LRM 26 1 0 19 Sep 2024
Visual Agents as Fast and Slow Thinkers Guangyan Sun Mingyu Jin Zhenting Wang Cheng-Long Wang Siqi Ma Qifan Wang Ying Nian Wu Ying Nian Wu Dongfang Liu Dongfang Liu LLMAG LRM 63 11 0 16 Aug 2024
Reflective Instruction Tuning: Mitigating Hallucinations in Large Vision-Language Models Jinrui Zhang Teng Wang Haigang Zhang Ping Lu Feng Zheng MLLM LRM VLM 19 3 0 16 Jul 2024
The Synergy between Data and Multi-Modal Large Language Models: A Survey from Co-Development Perspective Zhen Qin Daoyuan Chen Wenhao Zhang Liuyi Yao Yilun Huang Bolin Ding Yaliang Li Shuiguang Deng 43 5 0 11 Jul 2024
X-LLaVA: Optimizing Bilingual Large Vision-Language Alignment Dongjae Shin Hyunseok Lim Inho Won Changsu Choi Minjun Kim Seungwoo Song Hangyeol Yoo Sangmin Kim Kyungtae Lim 16 5 0 18 Mar 2024
On the Essence and Prospect: An Investigation of Alignment Approaches for Big Models Xinpeng Wang Shitong Duan Xiaoyuan Yi Jing Yao Shanlin Zhou Zhihua Wei Peng Zhang Dongkuan Xu Maosong Sun Xing Xie OffRL 27 16 0 07 Mar 2024
The Revolution of Multimodal Large Language Models: A Survey Davide Caffagni Federico Cocchi Luca Barsellotti Nicholas Moratelli Sara Sarto Lorenzo Baraldi Lorenzo Baraldi Marcella Cornia Rita Cucchiara LRM VLM 32 41 0 19 Feb 2024
Advancing Large Multi-modal Models with Explicit Chain-of-Reasoning and Visual Question Generation Kohei Uehara Nabarun Goswami Hanqin Wang Toshiaki Baba Kohtaro Tanaka ... Takagi Naoya Ryo Umagami Yingyi Wen Tanachai Anakewat Tatsuya Harada LRM 18 2 0 18 Jan 2024
Self-Imagine: Effective Unimodal Reasoning with Multimodal Models using Self-Imagination Syeda Nahida Akter Aman Madaan Sangwu Lee Yiming Yang Eric Nyberg ReLM VLM LRM 22 2 0 16 Jan 2024
Visual Instruction Tuning towards General-Purpose Multimodal Model: A Survey Jiaxing Huang Jingyi Zhang Kai Jiang Han Qiu Shijian Lu 25 22 0 27 Dec 2023
MM-SafetyBench: A Benchmark for Safety Evaluation of Multimodal Large Language Models Xin Liu Yichen Zhu Jindong Gu Yunshi Lan Chao Yang Yu Qiao 19 80 0 29 Nov 2023
Vision-Language Instruction Tuning: A Review and Analysis Chen Li Yixiao Ge Dian Li Ying Shan VLM 22 12 0 14 Nov 2023
Unveiling Safety Vulnerabilities of Large Language Models George Kour Marcel Zalmanovici Naama Zwerdling Esther Goldbraich Ora Nova Fandina Ateret Anaby-Tavor Orna Raz E. Farchi AAML 8 13 0 07 Nov 2023
CapsFusion: Rethinking Image-Text Data at Scale Qiying Yu Quan-Sen Sun Xiaosong Zhang Yufeng Cui Fan Zhang Yue Cao Xinlong Wang Jingjing Liu VLM 18 53 0 31 Oct 2023
On the Evaluation and Refinement of Vision-Language Instruction Tuning Datasets Ning Liao Shaofeng Zhang Renqiu Xia Min Cao Yu Qiao Junchi Yan MLLM 13 0 0 10 Oct 2023
Improved Baselines with Visual Instruction Tuning Haotian Liu Chunyuan Li Yuheng Li Yong Jae Lee VLM MLLM 22 2,400 0 05 Oct 2023
Sparkles: Unlocking Chats Across Multiple Images for Multimodal Instruction-Following Models Yupan Huang Zaiqiao Meng Fangyu Liu Yixuan Su Nigel Collier Yutong Lu MLLM 15 21 0 31 Aug 2023
InstructionGPT-4: A 200-Instruction Paradigm for Fine-Tuning MiniGPT-4 Lai Wei Zihao Jiang Weiran Huang Lichao Sun VLM MLLM 10 55 0 23 Aug 2023
From Instructions to Intrinsic Human Values -- A Survey of Alignment Goals for Big Models Jing Yao Xiaoyuan Yi Xiting Wang Jindong Wang Xing Xie ALM 6 41 0 23 Aug 2023
Emu: Generative Pretraining in Multimodality Quan-Sen Sun Qiying Yu Yufeng Cui Fan Zhang Xiaosong Zhang Yueze Wang Hongcheng Gao Jingjing Liu Tiejun Huang Xinlong Wang MLLM 19 126 0 11 Jul 2023
RemoteCLIP: A Vision Language Foundation Model for Remote Sensing F. Liu Delong Chen Zhan-Rong Guan Xiaocong Zhou Jiale Zhu Qiaolin Ye Liyong Fu Jun Zhou VLM 66 65 0 19 Jun 2023
mPLUG-Owl: Modularization Empowers Large Language Models with Multimodality Qinghao Ye Haiyang Xu Guohai Xu Jiabo Ye Ming Yan ... Junfeng Tian Qiang Qi Ji Zhang Feiyan Huang Jingren Zhou VLM MLLM 198 883 0 27 Apr 2023
Instruction Tuning with GPT-4 Baolin Peng Chunyuan Li Pengcheng He Michel Galley Jianfeng Gao SyDa ALM LM&MA 154 576 0 06 Apr 2023
Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering Pan Lu Swaroop Mishra Tony Xia Liang Qiu Kai-Wei Chang Song-Chun Zhu Oyvind Tafjord Peter Clark A. Kalyan ELM ReLM LRM 198 1,089 0 20 Sep 2022
ProtoCLIP: Prototypical Contrastive Language Image Pretraining Delong Chen Zhao Wu Fan Liu Zaiquan Yang Huaxi Huang Ying Tan Erjin Zhou VLM CLIP 11 28 0 22 Jun 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 301 11,730 0 04 Mar 2022