Visual Instruction Tuning

17 April 2023

Papers citing "Visual Instruction Tuning"

50 / 3,278 papers shown

Title
CAT: Enhancing Multimodal Large Language Model to Answer Questions in Dynamic Audio-Visual Scenarios Qilang Ye Zitong Yu Rui Shao Xinyu Xie Philip Torr Xiaochun Cao MLLM 56 24 0 07 Mar 2024
UniTable: Towards a Unified Framework for Table Recognition via Self-Supervised Pretraining Sheng-Hsuan Peng Aishwarya Chakravarthy Seongmin Lee Xiaojing Wang Rajarajeswari Balasubramaniyan Duen Horng Chau LMTD 46 0 0 07 Mar 2024
TextMonkey: An OCR-Free Large Multimodal Model for Understanding Document Yuliang Liu Biao Yang Qiang Liu Zhang Li Zhiyin Ma Shuo Zhang Xiang Bai MLLM VLM 54 92 0 07 Mar 2024
Effectiveness Assessment of Recent Large Vision-Language Models Yao Jiang Xinyu Yan Ge-Peng Ji Keren Fu Meijun Sun Huan Xiong Deng-Ping Fan Fahad Shahbaz Khan 44 14 0 07 Mar 2024
On the Essence and Prospect: An Investigation of Alignment Approaches for Big Models Xinpeng Wang Shitong Duan Xiaoyuan Yi Jing Yao Shanlin Zhou Zhihua Wei Peng Zhang Dongkuan Xu Maosong Sun Xing Xie OffRL 50 16 0 07 Mar 2024
CoTBal: Comprehensive Task Balancing for Multi-Task Visual Instruction Tuning Yanqi Dai Dong Jing Nanyi Fei Zhiwu Lu Nanyi Fei Guoxing Yang Zhiwu Lu 61 3 0 07 Mar 2024
Popeye: A Unified Visual-Language Model for Multi-Source Ship Detection from Remote Sensing Imagery Wei Zhang Miaoxin Cai Tong Zhang Guoqiang Lei Zhuang Yin Xuerui Mao 35 7 0 06 Mar 2024
SNIFFER: Multimodal Large Language Model for Explainable Out-of-Context Misinformation Detection Peng Qi Zehong Yan Wynne Hsu Mong Li Lee MLLM 58 33 0 05 Mar 2024
Feast Your Eyes: Mixture-of-Resolution Adaptation for Multimodal Large Language Models Gen Luo Yiyi Zhou Yuxin Zhang Xiawu Zheng Xiaoshuai Sun Rongrong Ji VLM 36 57 0 05 Mar 2024
Multi-modal Instruction Tuned LLMs with Fine-grained Visual Perception Jun-Yan He Yifan Wang Lijun Wang Huchuan Lu Jun-Yan He Jinpeng Lan Bin Luo Xuansong Xie MLLM VLM 39 19 0 05 Mar 2024
Neural Image Compression with Text-guided Encoding for both Pixel-level and Perceptual Fidelity Hagyeong Lee Minkyu Kim Jun-Hyuk Kim Seungeon Kim Dokwan Oh Jaeho Lee DiffM 43 6 0 05 Mar 2024
Here Comes The AI Worm: Unleashing Zero-click Worms that Target GenAI-Powered Applications Stav Cohen Ron Bitton Ben Nassi 40 19 0 05 Mar 2024
Finetuned Multimodal Language Models Are High-Quality Image-Text Data Filters Weizhi Wang Khalil Mrini Linjie Yang Sateesh Kumar Yu Tian Xifeng Yan Heng Wang 46 16 0 05 Mar 2024
Vision-Language Models for Medical Report Generation and Visual Question Answering: A Review Iryna Hartsock Ghulam Rasool 54 64 0 04 Mar 2024
RegionGPT: Towards Region Understanding Vision Language Model Qiushan Guo Shalini De Mello Hongxu Yin Wonmin Byeon Ka Chun Cheung Yizhou Yu Ping Luo Sifei Liu VLM 49 35 0 04 Mar 2024
Contrastive Region Guidance: Improving Grounding in Vision-Language Models without Training David Wan Jaemin Cho Elias Stengel-Eskin Mohit Bansal VLM ObjD 53 30 0 04 Mar 2024
Beyond Specialization: Assessing the Capabilities of MLLMs in Age and Gender Estimation Maksim Kuprashevich Grigorii Alekseenko Irina Tolstykh ELM 66 4 0 04 Mar 2024
Position: Towards Implicit Prompt For Text-To-Image Models Yue Yang Yuqi Lin Hong Liu Wenqi Shao Runjian Chen Hailong Shang Yu Wang Yu Qiao Kaipeng Zhang Ping Luo EGVM VLM 52 2 0 04 Mar 2024
VTG-GPT: Tuning-Free Zero-Shot Video Temporal Grounding with GPT Yifang Xu Yunzhuo Sun Zien Xie Benxiang Zhai Sidan Du 56 6 0 04 Mar 2024
NPHardEval4V: A Dynamic Reasoning Benchmark of Multimodal Large Language Models Lizhou Fan Wenyue Hua Xiang Li Kaijie Zhu Mingyu Jin ... Haoyang Ling Jinkui Chi Jindong Wang Xin Ma Yongfeng Zhang LRM 50 14 0 04 Mar 2024
Leveraging Biomolecule and Natural Language through Multi-Modal Learning: A Survey Qizhi Pei Lijun Wu Kaiyuan Gao Jinhua Zhu Yue Wang Zun Wang Tao Qin Rui Yan AI4CE 62 19 0 03 Mar 2024
Diff-Plugin: Revitalizing Details for Diffusion-based Low-level Tasks Yuhao Liu Zhanghan Ke Fang Liu Nanxuan Zhao Rynson W. H. Lau DiffM 48 19 0 01 Mar 2024
VisionLLaMA: A Unified LLaMA Backbone for Vision Tasks Xiangxiang Chu Jianlin Su Bo Zhang Chunhua Shen MLLM 49 10 0 01 Mar 2024
HALC: Object Hallucination Reduction via Adaptive Focal-Contrast Decoding Zhaorun Chen Zhuokai Zhao Hongyin Luo Huaxiu Yao Bo Li Jiawei Zhou MLLM 46 60 0 01 Mar 2024
UNITS: A Unified Multi-Task Time Series Model Shanghua Gao Teddy Koker Owen Queen Thomas Hartvigsen Theodoros Tsiligkaridis Marinka Zitnik AI4TS 54 17 0 29 Feb 2024
Panda-70M: Captioning 70M Videos with Multiple Cross-Modality Teachers Tsai-Shien Chen Aliaksandr Siarohin Willi Menapace Ekaterina Deyneka Hsiang-wei Chao ... Yuwei Fang Hsin-Ying Lee Jian Ren Ming-Hsuan Yang Sergey Tulyakov VGen 89 180 0 29 Feb 2024
The All-Seeing Project V2: Towards General Relation Comprehension of the Open World Weiyun Wang Yiming Ren Hao Luo Tiantong Li Chenxiang Yan ... Qingyun Li Lewei Lu Xizhou Zhu Yu Qiao Jifeng Dai MLLM 55 48 0 29 Feb 2024
Navigating Hallucinations for Reasoning of Unintentional Activities Shresth Grover Vibhav Vineet Yogesh S Rawat LRM 57 1 0 29 Feb 2024
Exploring the Potential of Large Language Models for Improving Digital Forensic Investigation Efficiency Akila Wickramasekara Frank Breitinger Mark Scanlon 52 8 0 29 Feb 2024
Beyond Language Models: Byte Models are Digital World Simulators Shangda Wu Xu Tan Zili Wang Rui Wang Xiaobing Li Maosong Sun 35 12 0 29 Feb 2024
Unveiling Typographic Deceptions: Insights of the Typographic Vulnerability in Large Vision-Language Model Hao-Ran Cheng Erjia Xiao Jindong Gu Le Yang Jinhao Duan Jize Zhang Jiahang Cao Kaidi Xu Renjing Xu 39 6 0 29 Feb 2024
Enhancing Visual Document Understanding with Contrastive Learning in Large Visual-Language Models Xin Li Yunfei Wu Xinghua Jiang Zhihao Guo Ming Gong Haoyu Cao Yinsong Liu Deqiang Jiang Xing Sun VLM 39 12 0 29 Feb 2024
Percept, Chat, and then Adapt: Multimodal Knowledge Transfer of Foundation Models for Open-World Video Recognition Boyu Chen Siran Chen Kunchang Li Qinglin Xu Yu Qiao Yali Wang 34 3 0 29 Feb 2024
Grounding Language Models for Visual Entity Recognition Zilin Xiao Ming Gong Paola Cascante-Bonilla Xingyao Zhang Jie Wu Vicente Ordonez VLM 51 9 0 28 Feb 2024
Large Language Models and Games: A Survey and Roadmap Roberto Gallotta Graham Todd Marvin Zammit Sam Earle Antonios Liapis Julian Togelius Georgios N. Yannakakis LLMAG LM&MA AI4CE LRM 55 73 0 28 Feb 2024
IBD: Alleviating Hallucinations in Large Vision-Language Models via Image-Biased Decoding Lanyun Zhu Deyi Ji Tianrun Chen Peng Xu Jieping Ye Jun Liu MLLM 56 45 0 28 Feb 2024
From Summary to Action: Enhancing Large Language Models for Complex Tasks with Open World APIs Yulong Liu Yunlong Yuan Chunwei Wang Jianhua Han Yongqiang Ma Li Zhang Nanning Zheng Hang Xu LLMAG 45 5 0 28 Feb 2024
Polos: Multimodal Metric Learning from Human Feedback for Image Captioning Yuiga Wada Kanta Kaneda Daichi Saito Komei Sugiura 39 24 0 28 Feb 2024
All in an Aggregated Image for In-Image Learning Lei Wang Wanyu Xu Zhiqiang Hu Yihuai Lan Shan Dong Hao Wang Roy Ka-wei Lee Ee-Peng Lim VLM 51 1 0 28 Feb 2024
A Cognitive Evaluation Benchmark of Image Reasoning and Description for Large Vision-Language Models Xiujie Song Mengyue Wu Ke Zhu Chunhao Zhang Yanyi Chen LRM ELM 41 3 0 28 Feb 2024
ShapeLLM: Universal 3D Object Understanding for Embodied Interaction Zekun Qi Runpei Dong Shaochen Zhang Haoran Geng Chunrui Han Zheng Ge Li Yi Kaisheng Ma 47 52 0 27 Feb 2024
OSCaR: Object State Captioning and State Change Representation Nguyen Nguyen Jing Bi Ali Vosoughi Yapeng Tian Pooyan Fazli Chenliang Xu 48 8 0 27 Feb 2024
Transparent Image Layer Diffusion using Latent Transparency Lvmin Zhang Maneesh Agrawala 37 43 0 27 Feb 2024
Diffusion Model-Based Image Editing: A Survey Yi Huang Jiancheng Huang Yifan Liu Mingfu Yan Jiaxi Lv Jianzhuang Liu Wei Xiong He Zhang Liangliang Cao Liangliang Cao EGVM 71 88 0 27 Feb 2024
GROUNDHOG: Grounding Large Language Models to Holistic Segmentation Yichi Zhang Ziqiao Ma Xiaofeng Gao Suhaila Shakiah Qiaozi Gao Joyce Chai MLLM VLM 55 39 0 26 Feb 2024
PhyGrasp: Generalizing Robotic Grasping with Physics-informed Large Multimodal Models Dingkun Guo Yuqi Xiang Shuqi Zhao Xinghao Zhu Masayoshi Tomizuka Mingyu Ding Wei Zhan 40 10 0 26 Feb 2024
Towards Open-ended Visual Quality Comparison Haoning Wu Hanwei Zhu Zicheng Zhang Erli Zhang Chaofeng Chen ... Qiong Yan Xiaohong Liu Guangtao Zhai Shiqi Wang Weisi Lin AAML 67 49 0 26 Feb 2024
RoboCodeX: Multimodal Code Generation for Robotic Behavior Synthesis Yao Mu Junting Chen Qinglong Zhang Shoufa Chen Qiaojun Yu ... Wenhai Wang Jifeng Dai Yu Qiao Mingyu Ding Ping Luo 51 22 0 25 Feb 2024
GraphWiz: An Instruction-Following Language Model for Graph Problems Nuo Chen Yuhan Li Jianheng Tang Jia Li 50 28 0 25 Feb 2024
NaVid: Video-based VLM Plans the Next Step for Vision-and-Language Navigation JIazhao Zhang Kunyu Wang Rongtao Xu Gengze Zhou Yicong Hong Xiaomeng Fang Qi Wu Zhizheng Zhang Wang He LM&Ro 44 45 0 24 Feb 2024