IdealGPT: Iteratively Decomposing Vision and Language Reasoning via Large Language Models

24 May 2023

Papers citing "IdealGPT: Iteratively Decomposing Vision and Language Reasoning via Large Language Models"

35 / 35 papers shown

Title
DWIM: Towards Tool-aware Visual Reasoning via Discrepancy-aware Workflow Generation & Instruct-Masking Tuning Fucai Ke Vijay Kumar B G Xingjian Leng Zhixi Cai Zaid Khan Weiqing Wang P. D. Haghighi H. Rezatofighi Manmohan Chandraker 37 0 0 25 Mar 2025
Towards Top-Down Reasoning: An Explainable Multi-Agent Approach for Visual Question Answering Zeqing Wang Wentao Wan Qiqing Lao Runmeng Chen Minjie Lang Keze Wang Liang Lin Liang Lin LRM 79 3 0 17 Feb 2025
CAP: Evaluation of Persuasive and Creative Image Generation Aysan Aghazadeh Adriana Kovashka EGVM 83 1 0 10 Dec 2024
Natural Language Understanding and Inference with MLLM in Visual Question Answering: A Survey Jiayi Kuang Jingyou Xie Haohao Luo Ronghao Li Zhe Xu Xianfeng Cheng Yinghui Li Xika Lin Ying Shen LRM 85 2 0 26 Nov 2024
MOSABench: Multi-Object Sentiment Analysis Benchmark for Evaluating Multimodal Large Language Models Understanding of Complex Image Shezheng Song Chengxiang He Shasha Li Shan Zhao Chengyu Wang ... Xiaopeng Li Qian Wan Jun Ma Jie Yu Xiaoguang Mao VLM 75 1 0 25 Nov 2024
AutoBench-V: Can Large Vision-Language Models Benchmark Themselves? Han Bao Yue Huang Yanbo Wang Jiayi Ye Xiangqi Wang Xiuying Chen Mohamed Elhoseiny X. Zhang Mohamed Elhoseiny Xiangliang Zhang 39 7 0 28 Oct 2024
An X-Ray Is Worth 15 Features: Sparse Autoencoders for Interpretable Radiology Report Generation Ahmed Abdulaal Hugo Fry Nina Montaña-Brown Ayodeji Ijishakin Jack Gao Stephanie L. Hyland Daniel C. Alexander Daniel Coelho De Castro MedIm 23 7 0 04 Oct 2024
JourneyBench: A Challenging One-Stop Vision-Language Understanding Benchmark of Generated Images Zhecan Wang Junzhang Liu Chia-Wei Tang Hani Alomari Anushka Sivakumar ... Haoxuan You A. Ishmam Kai-Wei Chang Shih-Fu Chang Chris Thomas CoGe VLM 31 2 0 19 Sep 2024
From Linguistic Giants to Sensory Maestros: A Survey on Cross-Modal Reasoning with Large Language Models Shengsheng Qian Zuyi Zhou Dizhan Xue Bing Wang Changsheng Xu LRM 24 1 0 19 Sep 2024
Benchmarking VLMs' Reasoning About Persuasive Atypical Images Sina Malakouti Aysan Aghazadeh Ashmit Khandelwal Adriana Kovashka VLM 18 2 0 16 Sep 2024
NEUSIS: A Compositional Neuro-Symbolic Framework for Autonomous Perception, Reasoning, and Planning in Complex UAV Search Missions Zhixi Cai Cristian Rojas Cardenas Kevin Leo Chenyuan Zhang Kal Backman ... Yuan-Fang Li Mor Vered Peter James Stuckey M. G. D. L. Banda Hamid Rezatofighi 16 5 0 16 Sep 2024
Visual Agents as Fast and Slow Thinkers Guangyan Sun Mingyu Jin Zhenting Wang Cheng-Long Wang Siqi Ma Qifan Wang Ying Nian Wu Ying Nian Wu Dongfang Liu Dongfang Liu LLMAG LRM 63 11 0 16 Aug 2024
ExoViP: Step-by-step Verification and Exploration with Exoskeleton Modules for Compositional Visual Reasoning Y. Wang Alan Yuille Zhuowan Li Zilong Zheng LRM 24 1 0 05 Aug 2024
HaloQuest: A Visual Hallucination Dataset for Advancing Multimodal Reasoning Zhecan Wang Garrett Bingham Adams Wei Yu Quoc V. Le Thang Luong Golnaz Ghiasi MLLM LRM 23 9 0 22 Jul 2024
Certainly Uncertain: A Benchmark and Metric for Multimodal Epistemic and Aleatoric Awareness Khyathi Raghavi Chandu Linjie Li Anas Awadalla Ximing Lu Jae Sung Park Jack Hessel Lijuan Wang Yejin Choi 28 2 0 02 Jul 2024
LogiCode: an LLM-Driven Framework for Logical Anomaly Detection Yiheng Zhang Yunkang Cao Xiaohao Xu Weiming Shen 21 13 0 07 Jun 2024
MMCTAgent: Multi-modal Critical Thinking Agent Framework for Complex Visual Reasoning Somnath Kumar Yash Gadhia T. Ganu A. Nambi LRM 37 1 0 28 May 2024
Autonomous Evaluation and Refinement of Digital Agents Jiayi Pan Yichi Zhang Nicholas Tomlin Yifei Zhou Sergey Levine Alane Suhr ELM 28 44 0 09 Apr 2024
HYDRA: A Hyper Agent for Dynamic Compositional Visual Reasoning Fucai Ke Zhixi Cai Simindokht Jahangard Weiqing Wang P. D. Haghighi Hamid Rezatofighi LRM 30 8 0 19 Mar 2024
Modeling Collaborator: Enabling Subjective Vision Classification With Minimal Human Effort via LLM Tool-Use Imad Eddine Toubal Aditya Avinash N. Alldrin Jan Dlabal Wenlei Zhou ... Chun-Ta Lu Howard Zhou Ranjay Krishna Ariel Fuxman Tom Duerig VLM 61 7 0 05 Mar 2024
Generalizing Visual Question Answering from Synthetic to Human-Written Questions via a Chain of QA with a Large Language Model Taehee Kim Yeongjae Cho Heejun Shin Yohan Jo Dongmyung Shin 24 3 0 12 Jan 2024
V*: Guided Visual Search as a Core Mechanism in Multimodal LLMs Penghao Wu Saining Xie LRM 42 120 0 21 Dec 2023
Good Questions Help Zero-Shot Image Reasoning Kaiwen Yang Tao Shen Xinmei Tian Xiubo Geng Chongyang Tao Dacheng Tao Tianyi Zhou LRM 22 6 0 04 Dec 2023
Woodpecker: Hallucination Correction for Multimodal Large Language Models Shukang Yin Chaoyou Fu Sirui Zhao Tong Bill Xu Hao Wang Dianbo Sui Yunhang Shen Ke Li Xingguo Sun Enhong Chen VLM MLLM 17 112 0 24 Oct 2023
ViCor: Bridging Visual Understanding and Commonsense Reasoning with Large Language Models KAI-QING Zhou Kwonjoon Lee Teruhisa Misu Xin Eric Wang LRM 11 3 0 09 Oct 2023
MLLM-DataEngine: An Iterative Refinement Approach for MLLM Zhiyuan Zhao Linke Ouyang Bin Wang Siyuan Huang Pan Zhang Xiao-wen Dong Jiaqi Wang Conghui He MLLM 13 5 0 25 Aug 2023
VIGC: Visual Instruction Generation and Correction Bin Wang Fan Wu Xiao Han Jiahui Peng Huaping Zhong ... Xiao-wen Dong Weijia Li Wei Li Jiaqi Wang Conghui He MLLM 13 58 0 24 Aug 2023
Tackling Vision Language Tasks Through Learning Inner Monologues Diji Yang Kezhen Chen Jinmeng Rao Xiaoyuan Guo Yawen Zhang Jie Yang Y. Zhang MLLM 10 7 0 19 Aug 2023
A Comprehensive Overview of Large Language Models Humza Naveed Asad Ullah Khan Shi Qiu Muhammad Saqib Saeed Anwar Muhammad Usman Naveed Akhtar Nick Barnes Ajmal Saeed Mian OffRL 25 499 0 12 Jul 2023
A Survey on Multimodal Large Language Models Shukang Yin Chaoyou Fu Sirui Zhao Ke Li Xing Sun Tong Bill Xu Enhong Chen MLLM LRM 14 515 0 23 Jun 2023
LVLM-eHub: A Comprehensive Evaluation Benchmark for Large Vision-Language Models Peng-Tao Xu Wenqi Shao Kaipeng Zhang Peng Gao Shuo Liu Meng Lei Fanqing Meng Siyuan Huang Yu Qiao Ping Luo ELM MLLM 21 158 0 15 Jun 2023
CoBIT: A Contrastive Bi-directional Image-Text Generation Model Haoxuan You Mandy Guo Zhecan Wang Kai-Wei Chang Jason Baldridge Jiahui Yu DiffM 37 12 0 23 Mar 2023
Self-Consistency Improves Chain of Thought Reasoning in Language Models Xuezhi Wang Jason W. Wei Dale Schuurmans Quoc Le Ed H. Chi Sharan Narang Aakanksha Chowdhery Denny Zhou ReLM BDL LRM AI4CE 297 3,163 0 21 Mar 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 301 11,730 0 04 Mar 2022
BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation Junnan Li Dongxu Li Caiming Xiong S. Hoi MLLM BDL VLM CLIP 380 4,010 0 28 Jan 2022