Visual Instruction Tuning

17 April 2023

Papers citing "Visual Instruction Tuning"

50 / 2,159 papers shown

Title
Chain of Images for Intuitively Reasoning Fanxu Meng Haotong Yang Yiding Wang Muhan Zhang LRM 28 6 0 09 Nov 2023
What Makes for Good Visual Instructions? Synthesizing Complex Visual Reasoning Instructions for Visual Instruction Tuning Yifan Du Hangyu Guo Kun Zhou Wayne Xin Zhao Jinpeng Wang Chuyuan Wang Mingchen Cai Ruihua Song Ji-Rong Wen VLM MLLM LRM 57 22 0 02 Nov 2023
Emotion Detection for Misinformation: A Review Zhiwei Liu Tianlin Zhang Kailai Yang Paul Thompson Zeping Yu Sophia Ananiadou 8 27 0 01 Nov 2023
Relax: Composable Abstractions for End-to-End Dynamic Machine Learning Ruihang Lai Junru Shao Siyuan Feng Steven Lyubomirsky Bohan Hou ... Sunghyun Park Prakalp Srivastava Jared Roesch T. Mowry Tianqi Chen 45 9 0 01 Nov 2023
GG-LLM: Geometrically Grounding Large Language Models for Zero-shot Human Activity Forecasting in Human-Aware Task Planning Moritz Graule Volkan Isler LM&Ro 19 14 0 30 Oct 2023
Image Clustering Conditioned on Text Criteria Sehyun Kwon Jaeseung Park Minkyu Kim Jaewoong Cho Ernest K. Ryu Kangwook Lee VLM 34 11 0 27 Oct 2023
JudgeLM: Fine-tuned Large Language Models are Scalable Judges Lianghui Zhu Xinggang Wang Xinlong Wang ELM ALM 54 106 0 26 Oct 2023
Semantic and Expressive Variation in Image Captions Across Languages Andre Ye Sebastin Santy Jena D. Hwang Amy X. Zhang Ranjay Krishna VLM 46 3 0 22 Oct 2023
Audio Editing with Non-Rigid Text Prompts Francesco Paissan Luca Della Libera Zhepei Wang Mirco Ravanelli Paris Smaragdis Cem Subakan DiffM 27 5 0 19 Oct 2023
VLIS: Unimodal Language Models Guide Multimodal Language Generation Jiwan Chung Youngjae Yu VLM 22 1 0 15 Oct 2023
Improving Compositional Text-to-image Generation with Large Vision-Language Models Song Wen Guian Fang Renrui Zhang Peng Gao Hao Dong Dimitris N. Metaxas 21 17 0 10 Oct 2023
Improving Discriminative Multi-Modal Learning with Large-Scale Pre-Trained Models Chenzhuang Du Yue Zhao Chonghua Liao Jiacheng You Jie Fu Hang Zhao 12 2 0 08 Oct 2023
Toolink: Linking Toolkit Creation and Using through Chain-of-Solving on Open-Source Model Cheng Qian Chenyan Xiong Zhenghao Liu Zhiyuan Liu LRM 24 12 0 08 Oct 2023
ConceptGraphs: Open-Vocabulary 3D Scene Graphs for Perception and Planning Yuanyi Zhong Alihusein Kuwajerwala Sacha Morin Krishna Murthy Jatavallabhula Bipasha Sen ... Celso Miguel de Melo Joshua B. Tenenbaum Antonio Torralba Florian Shkurti Liam Paull LM&Ro 27 166 0 28 Sep 2023
AnyMAL: An Efficient and Scalable Any-Modality Augmented Language Model Avamarie Brueggeman Andrea Madotto Zhaojiang Lin Tushar Nagarajan Matt Smith ... Peyman Heidari Yue Liu Kavya Srinet Babak Damavandi Anuj Kumar MLLM 24 92 0 27 Sep 2023
Jointly Training Large Autoregressive Multimodal Models Emanuele Aiello L. Yu Yixin Nie Armen Aghajanyan Barlas Oğuz 11 29 0 27 Sep 2023
Rethinking Channel Dimensions to Isolate Outliers for Low-bit Weight Quantization of Large Language Models Jung Hwan Heo Jeonghoon Kim Beomseok Kwon Byeongwook Kim Se Jung Kwon Dongsoo Lee MQ 38 9 0 27 Sep 2023
InternLM-XComposer: A Vision-Language Large Model for Advanced Text-image Comprehension and Composition Pan Zhang Xiaoyi Wang Bin Wang Yuhang Cao Chao Xu ... Conghui He Xingcheng Zhang Yu Qiao Da Lin Jiaqi Wang MLLM 61 222 0 26 Sep 2023
Multimodal Foundation Models: From Specialists to General-Purpose Assistants Chunyuan Li Zhe Gan Zhengyuan Yang Jianwei Yang Linjie Li Lijuan Wang Jianfeng Gao MLLM 110 226 0 18 Sep 2023
ImageBind-LLM: Multi-modality Instruction Tuning Jiaming Han Renrui Zhang Wenqi Shao Peng Gao Peng-Tao Xu ... Yafei Wen Xiaoxin Chen Xiangyu Yue Hongsheng Li Yu Qiao MLLM 30 115 0 07 Sep 2023
Cognitive Architectures for Language Agents T. Sumers Shunyu Yao Karthik Narasimhan Thomas L. Griffiths LLMAG LM&Ro 34 151 0 05 Sep 2023
CIEM: Contrastive Instruction Evaluation Method for Better Instruction Tuning Hongyu Hu Jiyuan Zhang Minyi Zhao Zhenbang Sun MLLM 25 41 0 05 Sep 2023
Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Models Yue Zhang Yafu Li Leyang Cui Deng Cai Lemao Liu ... Longyue Wang A. Luu Wei Bi Freda Shi Shuming Shi RALM LRM HILM 41 518 0 03 Sep 2023
Sparkles: Unlocking Chats Across Multiple Images for Multimodal Instruction-Following Models Yupan Huang Zaiqiao Meng Fangyu Liu Yixuan Su Nigel Collier Yutong Lu MLLM 28 22 0 31 Aug 2023
Diffusion Language Models Can Perform Many Tasks with Scaling and Instruction-Finetuning Jiasheng Ye Zaixiang Zheng Yu Bao Lihua Qian Quanquan Gu DiffM 52 14 0 23 Aug 2023
An Examination of the Compositionality of Large Generative Vision-Language Models Teli Ma Rong Li Junwei Liang CoGe 19 2 0 21 Aug 2023
LLM4TS: Aligning Pre-Trained LLMs as Data-Efficient Time-Series Forecasters Ching Chang Wei-Yao Wang Wenjie Peng Tien-Fu Chen AI4TS 30 45 0 16 Aug 2023
Link-Context Learning for Multimodal LLMs Yan Tai Weichen Fan Zhao Zhang Feng Zhu Rui Zhao Ziwei Liu ReLM LRM 21 17 0 15 Aug 2023
Revisiting DETR Pre-training for Object Detection Yan Ma Weicong Liang Bo-Ying Chen Yiduo Hao Bojian Hou Xiangyu Yue Chao Zhang Yuhui Yuan VLM ViT 25 4 0 02 Aug 2023
FinVis-GPT: A Multimodal Large Language Model for Financial Chart Analysis Ziao Wang Yuhang Li Junda Wu Jaehyeon Soon Xiaofeng Zhang MLLM 17 14 0 31 Jul 2023
Bridging the Gap: Exploring the Capabilities of Bridge-Architectures for Complex Visual Reasoning Tasks Kousik Rajesh Mrigank Raman M. A. Karim Pranit Chawla VLM 23 2 0 31 Jul 2023
SEED-Bench: Benchmarking Multimodal LLMs with Generative Comprehension Bohao Li Rui Wang Guangzhi Wang Yuying Ge Yixiao Ge Ying Shan MLLM ELM 16 496 0 30 Jul 2023
Jailbreak in pieces: Compositional Adversarial Attacks on Multi-Modal Language Models Erfan Shayegani Yue Dong Nael B. Abu-Ghazaleh 20 126 0 26 Jul 2023
Foundational Models Defining a New Era in Vision: A Survey and Outlook Muhammad Awais Muzammal Naseer Salman Khan Rao Muhammad Anwer Hisham Cholakkal M. Shah Ming Yang F. Khan VLM 18 117 0 25 Jul 2023
BuboGPT: Enabling Visual Grounding in Multi-Modal LLMs Yang Zhao Zhijie Lin Daquan Zhou Zilong Huang Jiashi Feng Bingyi Kang MLLM 33 106 0 17 Jul 2023
Linear Alignment of Vision-language Models for Image Captioning Fabian Paischer M. Hofmarcher Sepp Hochreiter Thomas Adler CLIP VLM 40 0 0 10 Jul 2023
GPT4RoI: Instruction Tuning Large Language Model on Region-of-Interest Shilong Zhang Pei Sun Shoufa Chen Min Xiao Wenqi Shao Wenwei Zhang Yu Liu Kai-xiang Chen Ping Luo VLM MLLM 83 223 0 07 Jul 2023
Visual Instruction Tuning with Polite Flamingo Delong Chen Jianfeng Liu Wenliang Dai Baoyuan Wang MLLM 20 42 0 03 Jul 2023
JourneyDB: A Benchmark for Generative Image Understanding Keqiang Sun Junting Pan Yuying Ge Hao Li Haodong Duan ... Yi Wang Jifeng Dai Yu Qiao Limin Wang Hongsheng Li 31 101 0 03 Jul 2023
Investigating Prompting Techniques for Zero- and Few-Shot Visual Question Answering Rabiul Awal Le Zhang Aishwarya Agrawal LRM 38 12 0 16 Jun 2023
Towards AGI in Computer Vision: Lessons Learned from GPT and Large Language Models Lingxi Xie Longhui Wei Xiaopeng Zhang Kaifeng Bi Xiaotao Gu Jianlong Chang Qi Tian 29 7 0 14 Jun 2023
Sticker820K: Empowering Interactive Retrieval with Stickers Sijie Zhao Yixiao Ge Zhongang Qi Lin Song Xiaohan Ding Zehua Xie Ying Shan 20 6 0 12 Jun 2023
Valley: Video Assistant with Large Language model Enhanced abilitY Ruipu Luo Ziwang Zhao Min Yang Junwei Dong Da Li Pengcheng Lu Tao Wang Linmei Hu Ming-Hui Qiu MLLM 40 188 0 12 Jun 2023
Biologically-Motivated Learning Model for Instructed Visual Processing R. Abel S. Ullman 13 0 0 04 Jun 2023
The False Promise of Imitating Proprietary LLMs Arnav Gudibande Eric Wallace Charles Burton Snell Xinyang Geng Hao Liu Pieter Abbeel Sergey Levine Dawn Song ALM 27 196 0 25 May 2023
IdealGPT: Iteratively Decomposing Vision and Language Reasoning via Large Language Models Haoxuan You Rui Sun Zhecan Wang Long Chen Gengyu Wang Hammad A. Ayyubi Kai-Wei Chang Shih-Fu Chang VLM MLLM LRM 39 43 0 24 May 2023
ONE-PEACE: Exploring One General Representation Model Toward Unlimited Modalities Peng Wang Shijie Wang Junyang Lin Shuai Bai Xiaohuan Zhou Jingren Zhou Xinggang Wang Chang Zhou VLM MLLM ObjD 16 114 0 18 May 2023
SpeechGPT: Empowering Large Language Models with Intrinsic Cross-Modal Conversational Abilities Dong Zhang Shimin Li Xin Zhang Jun Zhan Pengyu Wang Yaqian Zhou Xipeng Qiu AuLLM MLLM 43 287 0 18 May 2023
Evaluating Object Hallucination in Large Vision-Language Models Yifan Li Yifan Du Kun Zhou Jinpeng Wang Wayne Xin Zhao Ji-Rong Wen MLLM LRM 52 691 0 17 May 2023
Instruction Tuning with GPT-4 Baolin Peng Chunyuan Li Pengcheng He Michel Galley Jianfeng Gao SyDa ALM LM&MA 157 579 0 06 Apr 2023