Visual Instruction Tuning

17 April 2023

Papers citing "Visual Instruction Tuning"

50 / 2,159 papers shown

Title
OpenHelix: A Short Survey, Empirical Analysis, and Open-Source Dual-System VLA Model for Robotic Manipulation Can Cui Pengxiang Ding Wenxuan Song Shuanghao Bai Xinyang Tong ... Yang Liu Bofang Jia H. Zhao Siteng Huang Donglin Wang 14 1 0 06 May 2025
Reinforced Correlation Between Vision and Language for Precise Medical AI Assistant Haonan Wang Jiaji Mao Lehan Wang Qixiang Zhang Marawan Elbatel ... Weifeng Qin H. Li Jialin Liang Jun Shen Xiaomeng Li MedIm 28 0 0 06 May 2025
SynSHRP2: A Synthetic Multimodal Benchmark for Driving Safety-critical Events Derived from Real-world Driving Data Liang Shi Boyu Jiang Zhenyuan Yuan Miguel A. Perez Feng Guo 24 0 0 06 May 2025
SEFE: Superficial and Essential Forgetting Eliminator for Multimodal Continual Instruction Tuning Jinpeng Chen Runmin Cong Yuzhi Zhao Hongzheng Yang Guangneng Hu H. Ip Sam Kwong CLL KELM 70 0 0 05 May 2025
SuperEdit: Rectifying and Facilitating Supervision for Instruction-Based Image Editing Ming Li Xin Gu Fan Chen X. Xing Longyin Wen C. L. P. Chen Sijie Zhu DiffM 81 1 0 05 May 2025
Task-Oriented Semantic Communication in Large Multimodal Models-based Vehicle Networks Baoxia Du H. Du Dusit Niyato Ruidong Li 53 0 0 05 May 2025
LISAT: Language-Instructed Segmentation Assistant for Satellite Imagery Jerome Quenum Wen-Han Hsieh Tsung-Han Wu Ritwik Gupta Trevor Darrell David M. Chan MLLM VLM 54 0 0 05 May 2025
Using Knowledge Graphs to harvest datasets for efficient CLIP model training Simon Ging Sebastian Walter Jelena Bratulić Johannes Dienert Hannah Bast Thomas Brox CLIP 22 0 0 05 May 2025
Unified Multimodal Understanding and Generation Models: Advances, Challenges, and Opportunities X. Zhang Jintao Guo Shanshan Zhao Minghao Fu Lunhao Duan Guo-Hua Wang Qing-Guo Chen Zhao Xu Weihua Luo Kaifu Zhang DiffM 62 0 0 05 May 2025
Structure Causal Models and LLMs Integration in Medical Visual Question Answering Zibo Xu Qiang Li Weizhi Nie Weijie Wang Anan Liu CML MedIm 42 0 0 05 May 2025
DriveAgent: Multi-Agent Structured Reasoning with LLM and Multimodal Sensor Fusion for Autonomous Driving Xinmeng Hou Wuqi Wang Long Yang Hao Lin Jinglun Feng Haigen Min Xiangmo Zhao 37 0 0 04 May 2025
R-Bench: Graduate-level Multi-disciplinary Benchmarks for LLM & MLLM Complex Reasoning Evaluation Meng-Hao Guo Jiajun Xu Yi Zhang Jiaxi Song Haoyang Peng ... Yongming Rao Houwen Peng Han Hu Gordon Wetzstein Shi-Min Hu ELM LRM 57 0 0 04 May 2025
Compositional Image-Text Matching and Retrieval by Grounding Entities Madhukar Reddy Vongala Saurabh Srivastava Jana Kosecka CLIP CoGe VLM 36 0 0 04 May 2025
Interleave-VLA: Enhancing Robot Manipulation with Interleaved Image-Text Instructions Cunxin Fan Xiaosong Jia Yihang Sun Yixiao Wang Jianglan Wei ... Xiangyu Zhao M. Tomizuka Xue Yang Junchi Yan Mingyu Ding LM&Ro VLM 64 2 0 04 May 2025
Enhancing the Learning Experience: Using Vision-Language Models to Generate Questions for Educational Videos Markos Stamatakis Joshua Berger Christian Wartena Ralph Ewerth Anett Hoppe AI4Ed 41 0 0 03 May 2025
RAGAR: Retrieval Augment Personalized Image Generation Guided by Recommendation Run Ling W. Wang Yuting Liu G. Guo Linying Jiang Xingwei Wang DiffM 52 0 0 03 May 2025
RESAnything: Attribute Prompting for Arbitrary Referring Segmentation Ruiqi Wang Hao Zhang VLM 54 0 0 03 May 2025
Dynamic Robot Tool Use with Vision Language Models Noah Trupin Zixing Wang A. H. Qureshi 37 0 0 02 May 2025
Improving Editability in Image Generation with Layer-wise Memory Daneul Kim Jaeah Lee Jaesik Park DiffM KELM 55 0 0 02 May 2025
Transferable Adversarial Attacks on Black-Box Vision-Language Models Kai Hu Weichen Yu L. Zhang Alexander Robey Andy Zou Chengming Xu Haoqi Hu Matt Fredrikson AAML VLM 54 0 0 02 May 2025
TEMPURA: Temporal Event Masked Prediction and Understanding for Reasoning in Action Jen-Hao Cheng Vivian Wang Huayu Wang Huapeng Zhou Yi-Hao Peng ... Wenhao Chai Yi-Ling Chen Vibhav Vineet Qin Cai Jenq-Neng Hwang AI4TS 78 0 0 02 May 2025
VideoHallu: Evaluating and Mitigating Multi-modal Hallucinations for Synthetic Videos Zongxia Li Xiyang Wu Yubin Qin Guangyao Shi Hongyang Du Dinesh Manocha Tianyi Zhou Jordan Boyd-Graber MLLM 46 0 0 02 May 2025
Towards Efficient Online Tuning of VLM Agents via Counterfactual Soft Reinforcement Learning Lang Feng Weihao Tan Zhiyi Lyu Longtao Zheng Haiyang Xu M. Yan Fei Huang Bo An 24 0 0 01 May 2025
SpatialLLM: A Compound 3D-Informed Design towards Spatially-Intelligent Large Multimodal Models Wufei Ma Luoxin Ye Nessa McWeeney Celso M de Melo A. Yuille Jieneng Chen LRM 57 1 0 01 May 2025
Robotic Visual Instruction Y. Li Ziyang Gong H. Li Xiaoqi Huang Haolan Kang Guangping Bai Xianzheng Ma LM&Ro 69 0 0 01 May 2025
T2I-R1: Reinforcing Image Generation with Collaborative Semantic-level and Token-level CoT D. Jiang Ziyu Guo Renrui Zhang Zhuofan Zong Hao Li Le Zhuo Shilin Yan Pheng-Ann Heng H. Li LRM 57 0 0 01 May 2025
MINERVA: Evaluating Complex Video Reasoning Arsha Nagrani Sachit Menon Ahmet Iscen Shyamal Buch Ramin Mehran ... Yukun Zhu Carl Vondrick Mikhail Sirotenko Cordelia Schmid Tobias Weyand 56 0 0 01 May 2025
ScaleTrack: Scaling and back-tracking Automated GUI Agents Jing Huang Zhixiong Zeng WenKang Han Yufeng Zhong Liming Zheng Shuai Fu Jingyuan Chen Lin Ma 78 0 0 01 May 2025
Unlearning Sensitive Information in Multimodal LLMs: Benchmark and Attack-Defense Evaluation Vaidehi Patil Yi-Lin Sung Peter Hase Jie Peng Tianlong Chen Mohit Bansal AAML MU 81 3 0 01 May 2025
JointDiT: Enhancing RGB-Depth Joint Modeling with Diffusion Transformers Kwon Byung-Ki Qi Dai Lee Hyoseok Chong Luo Tae-Hyun Oh 59 0 0 01 May 2025
Pushing the Limits of Low-Bit Optimizers: A Focus on EMA Dynamics Cong Xu Wenbin Liang Mo Yu Anan Liu K. Zhang Lizhuang Ma J. Wang J. Wang W. Zhang MQ 51 0 0 01 May 2025
Iterative Tool Usage Exploration for Multimodal Agents via Step-wise Preference Tuning Pengxiang Li Zhi Gao Bofei Zhang Yapeng Mi Xiaojian Ma ... Tao Yuan Yuwei Wu Yunde Jia Song-Chun Zhu Qing Li LLMAG 70 0 0 30 Apr 2025
Investigating Zero-Shot Diagnostic Pathology in Vision-Language Models with Efficient Prompt Design Vasudev Sharma Ahmed Alagha Abdelhakim Khellaf Vincent Quoc-Huy Trinh Mahdi S. Hosseini 33 0 0 30 Apr 2025
COMPACT: COMPositional Atomic-to-Complex Visual Capability Tuning Xindi Wu Hee Seung Hwang Polina Kirichenko Olga Russakovsky VLM CoGe 68 0 0 30 Apr 2025
Towards Film-Making Production Dialogue, Narration, Monologue Adaptive Moving Dubbing Benchmarks Chaoyi Wang Junjie Zheng Zihao Chen Shiyu Xia Chaofan Ding Xiaohao Zhang Xi Tao Xiaoming He Xinhan Di AuLLM 76 0 0 30 Apr 2025
VividListener: Expressive and Controllable Listener Dynamics Modeling for Multi-Modal Responsive Interaction Shiying Li Xingqun Qi Bingkun Yang Chen Weile Zezhao Tian Muyi Sun Qifeng Liu Man Zhang Zhenan Sun 59 0 0 30 Apr 2025
Rethinking Visual Layer Selection in Multimodal LLMs H. Chen Junyan Lin Xinhao Chen Yue Fan Xin Jin Hui Su Jianfeng Dong Jinlan Fu Xiaoyu Shen VLM 95 0 0 30 Apr 2025
UniBiomed: A Universal Foundation Model for Grounded Biomedical Image Interpretation Linshan Wu Yuxiang Nie Sunan He Jiaxin Zhuang Hao Chen LM&MA MedIm 68 0 0 30 Apr 2025
Mcity Data Engine: Iterative Model Improvement Through Open-Vocabulary Data Selection Daniel Bogdoll Rajanikant Ananta Abeyankar Giridharan Isabel Moore Gregory Stevens Henry X. Liu VLM 51 0 0 30 Apr 2025
Detecting and Mitigating Hateful Content in Multimodal Memes with Vision-Language Models Minh-Hao Van Xintao Wu VLM 81 0 0 30 Apr 2025
Antidote: A Unified Framework for Mitigating LVLM Hallucinations in Counterfactual Presupposition and Object Perception Yuanchen Wu Lu Zhang Hang Yao Junlong Du Ke Yan Shouhong Ding Yunsheng Wu X. Li MLLM 71 0 0 29 Apr 2025
X-Fusion: Introducing New Modality to Frozen Large Language Models Sicheng Mo Thao Nguyen Xun Huang Siddharth Srinivasan Iyer Yijun Li ... Eli Shechtman Krishna Kumar Singh Yong Jae Lee Bolei Zhou Yuheng Li 71 0 0 29 Apr 2025
YoChameleon: Personalized Vision and Language Generation Thao Nguyen Krishna Kumar Singh Jing Shi Trung H. Bui Yong Jae Lee Yuheng Li MLLM 82 0 0 29 Apr 2025
Multimodal Large Language Models for Medicine: A Comprehensive Survey Jiarui Ye Hao Tang LM&MA 84 0 0 29 Apr 2025
Classifier-to-Bias: Toward Unsupervised Automatic Bias Detection for Visual Classifiers Quentin Guimard Moreno DÍncà Massimiliano Mancini Elisa Ricci SSL 72 0 0 29 Apr 2025
CMT: A Cascade MAR with Topology Predictor for Multimodal Conditional CAD Generation Jianyu Wu Yizhou Wang Xiangyu Yue Xinzhu Ma J. Guo Dongzhan Zhou Wanli Ouyang Shixiang Tang 66 0 0 29 Apr 2025
CompleteMe: Reference-based Human Image Completion Yu-Ju Tsai Brian L. Price Qing Liu Luis Figueroa D. Pakhomov Zhihong Ding Scott D. Cohen Ming Yang 3DH 47 0 0 28 Apr 2025
PhenoAssistant: A Conversational Multi-Agent AI System for Automated Plant Phenotyping Feng Chen Ilias Stogiannidis Andrew Wood Danilo Bueno Dominic Williams ... Stephen A. Rolfe Tracy Lawson Tony Pridmore M. Giuffrida Sotirios A. Tsaftaris 62 0 0 28 Apr 2025
VCM: Vision Concept Modeling Based on Implicit Contrastive Learning with Vision-Language Instruction Fine-Tuning Run Luo Renke Shan Longze Chen Z. Liu Lu Wang Min Yang Xiaobo Xia MLLM VLM 92 0 0 28 Apr 2025
Learning Streaming Video Representation via Multitask Training Yibin Yan Jilan Xu Shangzhe Di Yikun Liu Yudi Shi Qirui Chen Zeqian Li Yifei Huang Weidi Xie CLL 82 0 0 28 Apr 2025