Visual Instruction Tuning

17 April 2023

Papers citing "Visual Instruction Tuning"

50 / 3,212 papers shown

Title
DataPlatter: Boosting Robotic Manipulation Generalization with Minimal Costly Data Liming Zheng Feng Yan Fanfan Liu C. Feng Yufeng Zhong Yiyang Huang Lin Ma 42 0 0 25 Mar 2025
Scaling Vision Pre-Training to 4K Resolution Baifeng Shi Boyi Li Han Cai Y. Lu Sifei Liu ... Jan Kautz Song Han Trevor Darrell Pavlo Molchanov Hongxu Yin CLIP 124 0 0 25 Mar 2025
Gemma 3 Technical Report Gemma Team Aishwarya B Kamath Johan Ferret Shreya Pathak Nino Vieillard ... Harshal Tushar Lehri Hussein Hazimeh Ian Ballantyne Idan Szpektor Ivan Nardini VLM 85 30 0 25 Mar 2025
Towards Online Multi-Modal Social Interaction Understanding X. Li Shijian Deng Bolin Lai Weiguo Pian James M. Rehg Yapeng Tian 44 0 0 25 Mar 2025
LangBridge: Interpreting Image as a Combination of Language Embeddings Jiaqi Liao Yuwei Niu Fanqing Meng Hao Li Changyao Tian ... Dianqi Li X. Zhu Li Yuan Jifeng Dai Yu Cheng MLLM 72 0 0 25 Mar 2025
Reverse Prompt: Cracking the Recipe Inside Text-to-Image Generation Zhiyao Ren Yibing Zhan B. Yu Dacheng Tao DiffM 69 0 0 25 Mar 2025
LEGO-Puzzles: How Good Are MLLMs at Multi-Step Spatial Reasoning? Kexian Tang Junyao Gao Yanhong Zeng Haodong Duan Yanan Sun Zhening Xing Wenran Liu Kaifeng Lyu Kai-xiang Chen ELM LRM 56 1 0 25 Mar 2025
LayerCraft: Enhancing Text-to-Image Generation with CoT Reasoning and Layered Object Integration Yuyao Zhang Jinghao Li Yu-Wing Tai DiffM 64 0 0 25 Mar 2025
Improved Alignment of Modalities in Large Vision Language Models Kartik Jangra Aman Kumar Singh Yashwani Mann Geetanjali Rathee VLM 50 0 0 25 Mar 2025
DWIM: Towards Tool-aware Visual Reasoning via Discrepancy-aware Workflow Generation & Instruct-Masking Tuning Fucai Ke Vijay Kumar B G Xingjian Leng Zhixi Cai Zaid Khan Weiqing Wang P. D. Haghighi H. Rezatofighi Manmohan Chandraker 42 0 0 25 Mar 2025
FireEdit: Fine-grained Instruction-based Image Editing via Region-aware Vision Language Model Jun Zhou J. Li Zunnan Xu Hanhui Li Yiji Cheng Fa-Ting Hong Qin Lin Qinglin Lu Xiaodan Liang DiffM 67 1 0 25 Mar 2025
TopV: Compatible Token Pruning with Inference Time Optimization for Fast and Low-Memory Multimodal Vision Language Model Cheng Yang Yang Sui Jinqi Xiao Lingyi Huang Yu Gong ... Jinghua Yan Y. Bai P. Sadayappan Xia Hu Bo Yuan VLM 53 0 0 24 Mar 2025
Video-XL-Pro: Reconstructive Token Compression for Extremely Long Video Understanding Xiangrui Liu Yan Shu Zheng Liu Ao Li Yang Tian Bo Zhao VGen VLM 97 0 0 24 Mar 2025
HOIGPT: Learning Long Sequence Hand-Object Interaction with Language Models Mingzhen Huang Fu-Jen Chu Bugra Tekin Kevin J Liang Haoyu Ma ... Hongfei Xue Siwei Lyu Kris M. Kitani Matt Feiszli Hao Tang VLM 65 0 0 24 Mar 2025
Efficient Continual Adaptation of Pretrained Robotic Policy with Online Meta-Learned Adapters Ruiqi Zhu Endong Sun Guanhe Huang Oya Celiktutan CLL OnRL 59 0 0 24 Mar 2025
SlowFast-LLaVA-1.5: A Family of Token-Efficient Video Large Language Models for Long-Form Video Understanding Mingze Xu Mingfei Gao Shiyu Li Jiasen Lu Zhe Gan Zhengfeng Lai Meng Cao Kai Kang Y. Yang Afshin Dehghan 51 1 0 24 Mar 2025
Open-Vocabulary Functional 3D Scene Graphs for Real-World Indoor Spaces Chenyangguang Zhang Alexandros Delitzas Fangjinhua Wang Ruida Zhang Xiangyang Ji Marc Pollefeys Francis Engelmann 3DV 3DPC 47 4 0 24 Mar 2025
Bridging Writing Manner Gap in Visual Instruction Tuning by Creating LLM-aligned Instructions Dong Jing Nanyi Fei Zhiwu Lu 42 0 0 24 Mar 2025
Training-Free Personalization via Retrieval and Reasoning on Fingerprints Deepayan Das Davide Talon Yiming Wang Massimiliano Mancini Elisa Ricci VLM LRM 48 0 0 24 Mar 2025
From Fragment to One Piece: A Survey on AI-Driven Graphic Design Xingxing Zou Wen Zhang Nanxuan Zhao 54 0 0 24 Mar 2025
On the Perception Bottleneck of VLMs for Chart Understanding Junteng Liu Weihao Zeng Xiwen Zhang Yijun Wang Zifei Shan Junxian He 60 0 0 24 Mar 2025
Image-to-Text for Medical Reports Using Adaptive Co-Attention and Triple-LSTM Module Yishen Liu Shengda Liu Hudan Pan MedIm 50 0 0 24 Mar 2025
Maximum Redundancy Pruning: A Principle-Driven Layerwise Sparsity Allocation for LLMs Chang Gao Kang Zhao J. Chen Liping Jing 42 0 0 24 Mar 2025
OCRT: Boosting Foundation Models in the Open World with Object-Concept-Relation Triad Luyao Tang Yuxuan Yuan C. L. P. Chen Zeyu Zhang Yue Huang Kun Zhang 48 0 0 24 Mar 2025
MuMA: 3D PBR Texturing via Multi-Channel Multi-View Generation and Agentic Post-Processing Lingting Zhu Jingrui Ye Runze Zhang Zeyu Hu Yingda Yin ... Jinnan Chen Shengju Qian Xin Wang Qingmin Liao L. Yu 52 2 0 24 Mar 2025
A Simple yet Effective Layout Token in Large Language Models for Document Understanding Zhaoqing Zhu Chuwei Luo Zirui Shao Feiyu Gao Hangdi Xing Qi Zheng Ji Zhang 50 0 0 24 Mar 2025
LLaVAction: evaluating and training multi-modal large language models for action recognition Shaokai Ye Haozhe Qi Alexander Mathis Mackenzie W. Mathis 68 1 0 24 Mar 2025
VTD-CLIP: Video-to-Text Discretization via Prompting CLIP Wencheng Zhu Yuexin Wang Hongxuan Li Pengfei Zhu Q. Hu CLIP 48 0 0 24 Mar 2025
Towards Training-free Anomaly Detection with Vision and Language Foundation Models Jinjin Zhang Guodong Wang Yizhou Jin Di Huang 42 1 0 24 Mar 2025
Compositional Caching for Training-free Open-vocabulary Attribute Detection Marco Garosi Alessandro Conti Gaowen Liu Elisa Ricci Massimiliano Mancini ObjD VLM 50 0 0 24 Mar 2025
Instruction-Aligned Visual Attention for Mitigating Hallucinations in Large Vision-Language Models Bin Li Dehong Gao Yeyuan Wang Linbo Jin Shanqing Yu Xiaoyan Cai Libin Yang VLM 46 0 0 24 Mar 2025
MMCR: Advancing Visual Language Model in Multimodal Multi-Turn Contextual Reasoning Dawei Yan Y. Li Qing-Guo Chen Weihua Luo Peng Wang H. Zhang Chunhua Shen VGen VLM LRM 67 0 0 24 Mar 2025
Global-Local Tree Search in VLMs for 3D Indoor Scene Generation Wei Deng Mengshi Qi Huadong Ma 3DV 37 0 0 24 Mar 2025
Coeff-Tuning: A Graph Filter Subspace View for Tuning Attention-Based Large Models Zichen Miao Wei Chen Qiang Qiu 90 1 0 24 Mar 2025
SG-Tailor: Inter-Object Commonsense Relationship Reasoning for Scene Graph Manipulation Haoliang Shang Hanyu Wu Guangyao Zhai Boyang Sun Fangjinhua Wang F. Tombari Marc Pollefeys 57 0 0 23 Mar 2025
PHT-CAD: Efficient CAD Parametric Primitive Analysis with Progressive Hierarchical Tuning Ke Niu Yuwen Chen Haiyang Yu Z. Chen Xianghui Que Bin Li Xiangyang Xue 55 0 0 23 Mar 2025
Vision-R1: Evolving Human-Free Alignment in Large Vision-Language Models via Vision-Guided Reinforcement Learning Yufei Zhan Yousong Zhu Shurong Zheng Hongyin Zhao Fan Yang Ming Tang J. T. Wang VLM 67 3 0 23 Mar 2025
Unseen from Seen: Rewriting Observation-Instruction Using Foundation Models for Augmenting Vision-Language Navigation Ziming Wei Bingqian Lin Yunshuang Nie Jiaqi Chen Shikui Ma Hang Xu Xiaodan Liang 56 0 0 23 Mar 2025
Expanding the Boundaries of Vision Prior Knowledge in Multi-modal Large Language Models Qiao Liang Yanjiang Liu Ben He Y. Lu Hongyu Lin Jia Zheng Xianpei Han Le Sun Yingfei Sun 39 0 0 23 Mar 2025
Debiasing Multimodal Large Language Models via Noise-Aware Preference Optimization Zefeng Zhang Hengzhu Tang Jiawei Sheng Zhenyu Zhang Yiming Ren Zhenyang Li Dawei Yin Duohe Ma Tingwen Liu 47 0 0 23 Mar 2025
Safe RLHF-V: Safe Reinforcement Learning from Human Feedback in Multimodal Large Language Models Jiaming Ji X. Chen Rui Pan Han Zhu C. Zhang ... Juntao Dai Chi-Min Chan Sirui Han Yike Guo Y. Yang OffRL 74 2 0 22 Mar 2025
Towards Transformer-Based Aligned Generation with Self-Coherence Guidance Shulei Wang Wang Lin Hai Huang Hanting Wang Sihang Cai ... Tao Jin Jingyuan Chen Jiacheng Sun Jieming Zhu Zhou Zhao DiffM 55 2 0 22 Mar 2025
good4cir: Generating Detailed Synthetic Captions for Composed Image Retrieval Pranavi Kolouju Eric Xing Robert Pless Nathan Jacobs Abby Stylianou 3DV 55 0 0 22 Mar 2025
CODA: Repurposing Continuous VAEs for Discrete Tokenization Zeyu Liu Zanlin Ni Yeguo Hua Xin Deng Xiao Ma Cheng Zhong Gao Huang 44 0 0 22 Mar 2025
4D-Bench: Benchmarking Multi-modal Large Language Models for 4D Object Understanding Wenxuan Zhu Bing Li Cheng Zheng Jinjie Mai Jun-Cheng Chen ... Abdullah Hamdi Sara Rojas Martinez Chia-Wen Lin Mohamed Elhoseiny Bernard Ghanem VLM 48 0 0 22 Mar 2025
GOAL: Global-local Object Alignment Learning Hyungyu Choi Young Kyun Jang Chanho Eom VLM 121 0 0 22 Mar 2025
Position: Interactive Generative Video as Next-Generation Game Engine Jiwen Yu Yiran Qin Haoxuan Che Quande Liu Xintao Wang Pengfei Wan Di Zhang Xihui Liu VGen 45 1 0 21 Mar 2025
MedAgent-Pro: Towards Multi-modal Evidence-based Medical Diagnosis via Reasoning Agentic Workflow Ziyue Wang Junde Wu Chang Han Low Yueming Jin LRM 57 1 0 21 Mar 2025
LoRASculpt: Sculpting LoRA for Harmonizing General and Specialized Knowledge in Multimodal Large Language Models Jian Liang Wenke Huang Guancheng Wan Qu Yang Mang Ye MoMe CLL AI4CE 60 1 0 21 Mar 2025
ModalTune: Fine-Tuning Slide-Level Foundation Models with Multi-Modal Information for Multi-task Learning in Digital Pathology Vishwesh Ramanathan Tony Xu Pushpak Pati Faruk Ahmed Maged Goubran Anne L. Martel 45 0 0 21 Mar 2025