Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering

2 December 2016

Devi Parikh

Papers citing "Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering"

50 / 969 papers shown

Title
Prismatic VLMs: Investigating the Design Space of Visually-Conditioned Language Models Siddharth Karamcheti Suraj Nair Ashwin Balakrishna Percy Liang Thomas Kollar Dorsa Sadigh MLLM VLM 57 95 0 12 Feb 2024
Open-ended VQA benchmarking of Vision-Language models by exploiting Classification datasets and their semantic hierarchy Simon Ging M. A. Bravo Thomas Brox VLM 38 11 0 11 Feb 2024
ViGoR: Improving Visual Grounding of Large Vision Language Models with Fine-Grained Reward Modeling Siming Yan Min Bai Weifeng Chen Xiong Zhou Qixing Huang Erran L. Li VLM 19 18 0 09 Feb 2024
WebLINX: Real-World Website Navigation with Multi-Turn Dialogue Xing Han Lù Zdeněk Kasner Siva Reddy 22 59 0 08 Feb 2024
Question Aware Vision Transformer for Multimodal Reasoning Roy Ganz Yair Kittenplon Aviad Aberdam Elad Ben Avraham Oren Nuriel Shai Mazor Ron Litman 24 20 0 08 Feb 2024
Convincing Rationales for Visual Question Answering Reasoning Kun Li G. Vosselman Michael Ying Yang 34 1 0 06 Feb 2024
Video-LaVIT: Unified Video-Language Pre-training with Decoupled Visual-Motional Tokenization Yang Jin Zhicheng Sun Kun Xu Kun Xu Liwei Chen ... Yuliang Liu Di Zhang Yang Song Kun Gai Yadong Mu VGen 47 42 0 05 Feb 2024
Knowledge Generation for Zero-shot Knowledge-based VQA Rui Cao Jing Jiang 19 2 0 04 Feb 2024
GeReA: Question-Aware Prompt Captions for Knowledge-based Visual Question Answering Ziyu Ma Shutao Li Bin Sun Jianfei Cai Zuxiang Long Fuyan Ma 21 1 0 04 Feb 2024
Can MLLMs Perform Text-to-Image In-Context Learning? Yuchen Zeng Wonjun Kang Yicong Chen Hyung Il Koo Kangwook Lee MLLM 23 9 0 02 Feb 2024
Common Sense Reasoning for Deepfake Detection Yue Zhang Ben Colman Xiao Guo Ali Shahriyari Gaurav Bharaj 17 30 0 31 Jan 2024
MouSi: Poly-Visual-Expert Vision-Language Models Xiaoran Fan Tao Ji Changhao Jiang Shuo Li Senjie Jin ... Qi Zhang Xipeng Qiu Xuanjing Huang Zuxuan Wu Yunchun Jiang VLM 24 16 0 30 Jan 2024
MoE-LLaVA: Mixture of Experts for Large Vision-Language Models Bin Lin Zhenyu Tang Yang Ye Jiaxi Cui Bin Zhu ... Jinfa Huang Junwu Zhang Yatian Pang Munan Ning Li-ming Yuan VLM MLLM MoE 33 151 0 29 Jan 2024
Muffin or Chihuahua? Challenging Multimodal Large Language Models with Multipanel VQA Yue Fan Jing Gu KAI-QING Zhou Qianqi Yan Shan Jiang Ching-Chen Kuo Xinze Guan Xin Eric Wang 24 6 0 29 Jan 2024
Improving Data Augmentation for Robust Visual Question Answering with Effective Curriculum Learning Yuhang Zheng Zhen Wang Long Chen 10 2 0 28 Jan 2024
CMMU: A Benchmark for Chinese Multi-modal Multi-type Question Understanding and Reasoning Zheqi He Xinya Wu Pengfei Zhou Richeng Xuan Guang Liu Xi Yang Qiannan Zhu Hua Huang ELM LRM 25 14 0 25 Jan 2024
WebVoyager: Building an End-to-End Web Agent with Large Multimodal Models Hongliang He Wenlin Yao Kaixin Ma Wenhao Yu Yong Dai Hongming Zhang Zhenzhong Lan Dong Yu LLMAG 30 121 0 25 Jan 2024
Demystifying Chains, Trees, and Graphs of Thoughts Maciej Besta Florim Memedi Zhenyu Zhang Robert Gerstenberger Guangyuan Piao ... Aleš Kubíček H. Niewiadomski Aidan O'Mahony Onur Mutlu Torsten Hoefler AI4CE LRM 63 26 0 25 Jan 2024
MM-LLMs: Recent Advances in MultiModal Large Language Models Duzhen Zhang Yahan Yu Jiahua Dong Chenxing Li Dan Su Chenhui Chu Dong Yu OffRL LRM 37 173 0 24 Jan 2024
SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilities Boyuan Chen Zhuo Xu Sean Kirmani Brian Ichter Danny Driess Pete Florence Dorsa Sadigh Leonidas J. Guibas Fei Xia LRM ReLM 36 194 0 22 Jan 2024
Inducing High Energy-Latency of Large Vision-Language Models with Verbose Images Kuofeng Gao Yang Bai Jindong Gu Shu-Tao Xia Philip H. S. Torr Zhifeng Li Wei Liu VLM 14 38 0 20 Jan 2024
Q&A Prompts: Discovering Rich Visual Clues through Mining Question-Answer Prompts for VQA requiring Diverse World Knowledge Haibi Wang Weifeng Ge LRM 19 3 0 19 Jan 2024
Supervised Fine-tuning in turn Improves Visual Foundation Models Xiaohu Jiang Yixiao Ge Yuying Ge Dachuan Shi Chun Yuan Ying Shan VLM CLIP 38 8 0 18 Jan 2024
MM-Interleaved: Interleaved Image-Text Generative Modeling via Multi-modal Feature Synchronizer Changyao Tian Xizhou Zhu Yuwen Xiong Weiyun Wang Zhe Chen ... Tong Lu Jie Zhou Hongsheng Li Yu Qiao Jifeng Dai AuLLM 83 40 0 18 Jan 2024
Advancing Large Multi-modal Models with Explicit Chain-of-Reasoning and Visual Question Generation Kohei Uehara Nabarun Goswami Hanqin Wang Toshiaki Baba Kohtaro Tanaka ... Takagi Naoya Ryo Umagami Yingyi Wen Tanachai Anakewat Tatsuya Harada LRM 21 2 0 18 Jan 2024
Beyond Anti-Forgetting: Multimodal Continual Instruction Tuning with Positive Forward Transfer Junhao Zheng Qianli Ma Zhen Liu Binquan Wu Hu Feng CLL 20 14 0 17 Jan 2024
COCO is "ALL'' You Need for Visual Instruction Fine-tuning Xiaotian Han Yiqi Wang Bohan Zhai Quanzeng You Hongxia Yang VLM MLLM 23 2 0 17 Jan 2024
Uncovering the Full Potential of Visual Grounding Methods in VQA Daniel Reich Tanja Schultz 25 4 0 15 Jan 2024
MM-SAP: A Comprehensive Benchmark for Assessing Self-Awareness of Multimodal Large Language Models in Perception Yuhao Wang Yusheng Liao Heyang Liu Hongcheng Liu Yu Wang Yanfeng Wang LRM VLM 22 13 0 15 Jan 2024
BOK-VQA: Bilingual outside Knowledge-Based Visual Question Answering via Graph Representation Pretraining Minjun Kim Seungwoo Song Youhan Lee Haneol Jang Kyungtae Lim 16 6 0 12 Jan 2024
Distilling Vision-Language Models on Millions of Videos Yue Zhao Long Zhao Xingyi Zhou Jialin Wu Chun-Te Chu ... Hartwig Adam Ting Liu Boqing Gong Philipp Krahenbuhl Liangzhe Yuan VLM 21 13 0 11 Jan 2024
Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMs Shengbang Tong Zhuang Liu Yuexiang Zhai Yi-An Ma Yann LeCun Saining Xie VLM MLLM 18 281 0 11 Jan 2024
GroundingGPT:Language Enhanced Multi-modal Grounding Model Zhaowei Li Qi Xu Dong Zhang Hang Song Yiqing Cai ... Junting Pan Zefeng Li Van Tu Vu Zhida Huang Tao Wang 18 36 0 11 Jan 2024
Efficient Vision-and-Language Pre-training with Text-Relevant Image Patch Selection Wei Ye Chaoya Jiang Haiyang Xu Chenhao Ye Chenliang Li Mingshi Yan Shikun Zhang Songhang Huang Fei Huang VLM 29 0 0 11 Jan 2024
InfiAgent-DABench: Evaluating Agents on Data Analysis Tasks Xueyu Hu Ziyu Zhao Shuang Wei Ziwei Chai Qianli Ma ... Jiwei Li Kun Kuang Yang Yang Hongxia Yang Fei Wu LMTD ELM 11 41 0 10 Jan 2024
Exploring the Reasoning Abilities of Multimodal Large Language Models (MLLMs): A Comprehensive Survey on Emerging Trends in Multimodal Reasoning Yiqi Wang Wentao Chen Xiaotian Han Xudong Lin Haiteng Zhao Yongfei Liu Bohan Zhai Jianbo Yuan Quanzeng You Hongxia Yang LRM 33 66 0 10 Jan 2024
Incorporating Visual Experts to Resolve the Information Loss in Multimodal Large Language Models Xin He Longhui Wei Lingxi Xie Qi Tian 43 8 0 06 Jan 2024
LLaVA-Phi: Efficient Multi-Modal Assistant with Small Language Model Yichen Zhu Minjie Zhu Ning Liu Zhicai Ou Xiaofeng Mou Jian Tang 63 89 0 04 Jan 2024
SyCoCa: Symmetrizing Contrastive Captioners with Attentive Masking for Multimodal Alignment Ziping Ma Furong Xu Jian Liu Ming Yang Qingpei Guo VLM 34 3 0 04 Jan 2024
Instruct-Imagen: Image Generation with Multi-modal Instruction Hexiang Hu Kelvin C. K. Chan Yu-Chuan Su Wenhu Chen Yandong Li ... Xue Ben Boqing Gong William W. Cohen Ming-Wei Chang Xuhui Jia MLLM 33 42 0 03 Jan 2024
GPT-4V(ision) is a Generalist Web Agent, if Grounded Boyuan Zheng Boyu Gou Jihyung Kil Huan Sun Yu-Chuan Su MLLM VLM LLMAG 41 205 0 03 Jan 2024
GOAT-Bench: Safety Insights to Large Multimodal Models through Meme-Based Social Abuse Hongzhan Lin Ziyang Luo Bo Wang Ruichao Yang Jing Ma 25 24 0 03 Jan 2024
COSMO: COntrastive Streamlined MultimOdal Model with Interleaved Pre-Training Alex Jinpeng Wang Linjie Li K. Lin Jianfeng Wang Kevin Lin Zhengyuan Yang Lijuan Wang Mike Zheng Shou VLM VGen 11 12 0 01 Jan 2024
Unified-IO 2: Scaling Autoregressive Multimodal Models with Vision, Language, Audio, and Action Jiasen Lu Christopher Clark Sangho Lee Zichen Zhang Savya Khosla Ryan Marten Derek Hoiem Aniruddha Kembhavi VLM MLLM 27 143 0 28 Dec 2023
MIVC: Multiple Instance Visual Component for Visual-Language Models Wenyi Wu Qi Li Leon Wenliang Zhong Junzhou Huang 29 3 0 28 Dec 2023
Visual Instruction Tuning towards General-Purpose Multimodal Model: A Survey Jiaxing Huang Jingyi Zhang Kai Jiang Han Qiu Shijian Lu 28 22 0 27 Dec 2023
ChartBench: A Benchmark for Complex Visual Reasoning in Charts Zhengzhuo Xu Sinan Du Yiyan Qi Chengjin Xu Chun Yuan Jian Guo 33 33 0 26 Dec 2023
InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks Zhe Chen Jiannan Wu Wenhai Wang Weijie Su Guo Chen ... Bin Li Ping Luo Tong Lu Yu Qiao Jifeng Dai VLM MLLM 156 895 0 21 Dec 2023
Parrot Captions Teach CLIP to Spot Text Yiqi Lin Conghui He Alex Jinpeng Wang Bin Wang Weijia Li Mike Zheng Shou 20 7 0 21 Dec 2023
Multimodal Federated Learning with Missing Modality via Prototype Mask and Contrast Guangyin Bao Qi Zhang Duoqian Miao Zixuan Gong Liang Hu Ke Liu Yang Liu Chongyang Shi 29 7 0 21 Dec 2023