Visual Programming: Compositional visual reasoning without training

Computer Vision and Pattern Recognition (CVPR), 2022

18 November 2022

ArXiv (abs)PDF HTML HuggingFace (1 upvotes)

Papers citing "Visual Programming: Compositional visual reasoning without training"

50 / 379 papers shown

Title
Think-Program-reCtify: 3D Situated Reasoning with Large Language Models Qingrong He Kejun Lin Shizhe Chen Anwen Hu Qin Jin LRM 180 4 0 23 Apr 2024
A Multimodal Automated Interpretability Agent Tamar Rott Shaham Sarah Schwettmann Franklin Wang Achyuta Rajaram Evan Hernandez Jacob Andreas Antonio Torralba 473 43 0 22 Apr 2024
Fact :Teaching MLLMs with Faithful, Concise and Transferable Rationales Minghe Gao Shuang Chen Liang Pang Xingtai Lv Jisheng Dang Wenqiao Zhang Juncheng Li Siliang Tang Yueting Zhuang Tat-Seng Chua LRM 135 10 0 17 Apr 2024
Enhancing Visual Question Answering through Question-Driven Image Captions as Prompts Övgü Özdemir Erdem Akagündüz 252 18 0 12 Apr 2024
OpenBias: Open-set Bias Detection in Text-to-Image Generative Models Moreno DÍncà E. Peruzzo Massimiliano Mancini Dejia Xu Vidit Goel Xingqian Xu Zinan Lin Humphrey Shi Andrii Zadaianchuk 222 65 0 11 Apr 2024
Learning to Localize Objects Improves Spatial Reasoning in Visual-LLMs Kanchana Ranasinghe Satya Narayan Shukla Omid Poursaeed Michael S. Ryoo Tsung-Yu Lin LRM 169 58 0 11 Apr 2024
MoReVQA: Exploring Modular Reasoning Models for Video Question Answering Juhong Min Shyamal Buch Arsha Nagrani Minsu Cho Cordelia Schmid LRM 344 60 0 09 Apr 2024
Visually Descriptive Language Model for Vector Graphics Reasoning Zhenhailong Wang Joy Hsu Xingyao Wang Kuan-Hao Huang Pengfei Yu Jiajun Wu Mengyue Yang MLLM VLM LRM 258 7 0 09 Apr 2024
Self-Training Large Language Models for Improved Visual Program Synthesis With Visual Reinforcement Zaid Khan B. Vijaykumar S. Schulter Yun Fu Manmohan Chandraker LRM ReLM 202 15 0 06 Apr 2024
Idea-2-3D: Collaborative LMM Agents Enable 3D Model Generation from Interleaved Multimodal Inputs Junhao Chen Xiang Li Xiaojun Ye Chao Li Zhaoxin Fan Hao Zhao VGen 3DV 360 6 0 05 Apr 2024
Visual Knowledge in the Big Model Era: Retrospect and Prospect Wenguan Wang Yi Yang Yunhe Pan VLM 256 29 0 05 Apr 2024
PREGO: online mistake detection in PRocedural EGOcentric videosComputer Vision and Pattern Recognition (CVPR), 2024 Alessandro Flaborea Guido Maria DÁmely di Melendugno Leonardo Plini Luca Scofano Edoardo De Matteis Antonino Furnari G. Farinella Yuta Kyuragi EgoV 230 29 0 02 Apr 2024
Evaluating Text-to-Visual Generation with Image-to-Text Generation Zhiqiu Lin Deepak Pathak Baiqi Li Jiayao Li Xide Xia Graham Neubig Pengchuan Zhang Deva Ramanan EGVM 397 329 0 01 Apr 2024
Chat Modeling: Natural Language-based Procedural Modeling of Biological Structures without Training Donggang Jia Yunhai Wang Ivan Viola 165 1 0 01 Apr 2024
LLMs are Good Sign Language Translators Jia Gong Lin Geng Foo Yixuan He Hossein Rahmani Jun Liu SLR 222 65 0 01 Apr 2024
Planning and Editing What You Retrieve for Enhanced Tool Learning Tenghao Huang Dongwon Jung Muhao Chen KELM 242 17 0 30 Mar 2024
Change-Agent: Towards Interactive Comprehensive Remote Sensing Change Interpretation and Analysis Chenyang Liu Keyan Chen Haotian Zhang Zipeng Qi Zhengxia Zou Z. Shi 240 68 0 28 Mar 2024
Residual-based Language Models are Free Boosters for Biomedical Imaging Zhixin Lai Jing Wu Suiyao Chen Yucheng Zhou N. Hovakimyan MedIm 338 35 0 26 Mar 2024
PropTest: Automatic Property Testing for Improved Visual Programming Jaywon Koo Ziyan Yang Paola Cascante-Bonilla Baishakhi Ray Vicente Ordonez LRM 159 7 0 25 Mar 2024
SYNAPSE: SYmbolic Neural-Aided Preference Synthesis Engine Sadanand Modak Noah T Patton Işıl Dillig Joydeep Biswas 255 0 0 25 Mar 2024
Synthesize Step-by-Step: Tools, Templates and LLMs as Data Generators for Reasoning-Based Chart VQA Zhuowan Li Bhavan A. Jasani Peng Tang Shabnam Ghadar LRM 274 23 0 25 Mar 2024
VURF: A General-purpose Reasoning and Self-refinement Framework for Video Understanding Ahmad A Mahmood Ashmal Vayani Muzammal Naseer Salman Khan Fahad Shahbaz Khan LRM 370 11 0 21 Mar 2024
HYDRA: A Hyper Agent for Dynamic Compositional Visual Reasoning Fucai Ke Zhixi Cai Simindokht Jahangard Weiqing Wang P. D. Haghighi Hamid Rezatofighi LRM 216 21 0 19 Mar 2024
What Are Tools Anyway? A Survey from the Language Model Perspective Zhiruo Wang Zhoujun Cheng Hao Zhu Daniel Fried Graham Neubig 270 48 0 18 Mar 2024
VideoAgent: A Memory-augmented Multimodal Agent for Video Understanding Yue Fan Xiaojian Ma Rujie Wu Yuntao Du Jiaqi Li Zhi Gao Qing Li VLM LLMAG 279 144 0 18 Mar 2024
m&m's: A Benchmark to Evaluate Tool-Use for multi-step multi-modal Tasks Zixian Ma Weikai Huang Jieyu Zhang Tanmay Gupta Ranjay Krishna 297 34 0 17 Mar 2024
Borrowing Treasures from Neighbors: In-Context Learning for Multimodal Learning with Missing Modalities and Data Scarcity Zhuo Zhi Ziquan Liu M. Elbadawi Adam Daneshmend Mine Orlu Abdul Basit Andreas Demosthenous Miguel R. D. Rodrigues 224 4 0 14 Mar 2024
StableToolBench: Towards Stable Large-Scale Benchmarking on Tool Learning of Large Language ModelsAnnual Meeting of the Association for Computational Linguistics (ACL), 2024 Zhicheng Guo Sijie Cheng Hao Wang Shihao Liang Yujia Qin Peng Li Zhiyuan Liu Maosong Sun Yang Liu ELM 357 64 0 12 Mar 2024
A Modular Approach for Multimodal Summarization of TV Shows Louis Mahon Mirella Lapata 421 12 0 06 Mar 2024
What Is Missing in Multilingual Visual Reasoning and How to Fix It Yueqi Song Simran Khanuja Graham Neubig VLM LRM 553 8 0 03 Mar 2024
Data-free Multi-label Image Recognition via LLM-powered Prompt Tuning Shuo Yang Zirui Shang Yongqi Wang Derong Deng Hongwei Chen Qiyuan Cheng Xinxiao Wu VLM 142 10 0 02 Mar 2024
From Summary to Action: Enhancing Large Language Models for Complex Tasks with Open World APIs Yulong Liu Yunlong Yuan Chunwei Wang Jianhua Han Yongqiang Ma Li Zhang Nanning Zheng Hang Xu LLMAG 112 11 0 28 Feb 2024
ShapeLLM: Universal 3D Object Understanding for Embodied Interaction Zekun Qi Runpei Dong Shaochen Zhang Haoran Geng Chunrui Han Zheng Ge Li Yi Kaisheng Ma 422 107 0 27 Feb 2024
OmniACT: A Dataset and Benchmark for Enabling Multimodal Generalist Autonomous Agents for Desktop and Web Raghav Kapoor Y. Butala M. Russak Jing Yu Koh Kiran Kamble Waseem Alshikh Ruslan Salakhutdinov LLMAG 435 102 0 27 Feb 2024
Selective "Selective Prediction": Reducing Unnecessary Abstention in Vision-Language Reasoning Tejas Srinivasan Jack Hessel Tanmay Gupta Bill Yuchen Lin Yejin Choi Jesse Thomason Khyathi Chandu 243 14 0 23 Feb 2024
AutoMMLab: Automatically Generating Deployable Models from Language Instructions for Computer Vision Tasks Zekang Yang Wang Zeng Sheng Jin Chao Qian Ping Luo Wentao Liu MLLM VLM 236 14 0 23 Feb 2024
Large Multimodal Agents: A Survey Junlin Xie Zhihong Chen Ruifei Zhang Xiang Wan Guanbin Li LM&Ro LLMAG 204 78 0 23 Feb 2024
RoboScript: Code Generation for Free-Form Manipulation Tasks across Real and Simulation Junting Chen Yao Mu Qiaojun Yu Tianming Wei Silang Wu ... Wenqi Shao Yu Qiao Huazhe Xu Mingyu Ding Ping Luo LM&Ro 189 24 0 22 Feb 2024
CODIS: Benchmarking Context-Dependent Visual Comprehension for Multimodal Large Language Models Ziyue Wang Chi Chen Zihao Wan Zhaolu Kang Qidong Yan ... Xiaoyue Mi Peng Li Ning Ma Maosong Sun Yang Liu 268 11 0 21 Feb 2024
Tell Me More! Towards Implicit User Intention Understanding of Language Model Driven Agents Cheng Qian Bingxiang He Zhuang Zhong Jia Deng Yujia Qin ... Zhong Zhang Jie Zhou Yankai Lin Zhiyuan Liu Maosong Sun 196 58 0 14 Feb 2024
L3GO: Language Agents with Chain-of-3D-Thoughts for Generating Unconventional Objects Yutaro Yamada Khyathi Chandu Yuchen Lin Jack Hessel Ilker Yildirim Yejin Choi AI4CE 136 23 0 14 Feb 2024
Diffusion-ES: Gradient-free Planning with Diffusion for Autonomous Driving and Zero-Shot Instruction Following Brian Yang Huangyuan Su N. Gkanatsios Tsung-Wei Ke Ayush Jain Jeff Schneider Katerina Fragkiadaki DiffM 237 36 0 09 Feb 2024
LLMs for Coding and Robotics Education Peng Shu Huaqin Zhao Hanqi Jiang Yiwei Li Shaochen Xu ... Zheng Liu Guoyu Lu Le Guan Gong Chen Xianqiao Wang Tianming Liu 202 7 0 09 Feb 2024
Editable Scene Simulation for Autonomous Driving via Collaborative LLM-Agents Yuxi Wei Zi Wang Yifan Lu Chenxin Xu Chang-rui Liu Hao Zhao Siheng Chen Yanfeng Wang VGen 253 105 0 08 Feb 2024
MIGC: Multi-Instance Generation Controller for Text-to-Image Synthesis Dewei Zhou You Li Fan Ma Zongxin Yang Yi Yang DiffM 250 102 0 08 Feb 2024
Open-Universe Indoor Scene Generation using LLM Program Synthesis and Uncurated Object Databases Rio Aguina-Kang Maxim Gumin Do Heon Han Stewart Morris Seung Jean Yoo Aditya Ganeshan R. K. Jones Qiuhong Anna Wei Kailiang Fu Daniel E. Ritchie 3DV 221 39 0 05 Feb 2024
Solution-oriented Agent-based Models Generation with Verifier-assisted Iterative In-context Learning Tong Niu Weihao Zhang Rong Zhao LLMAG 170 3 0 04 Feb 2024
Common Sense Reasoning for Deepfake Detection Yue Zhang Ben Colman Xiao Guo Ali Shahriyari Gaurav Bharaj 438 56 0 31 Jan 2024
ReGAL: Refactoring Programs to Discover Generalizable Abstractions Elias Stengel-Eskin Archiki Prasad Mohit Bansal 201 19 0 29 Jan 2024
GraphiMind: LLM-centric Interface for Information Graphics Design Qiruin Huang Min Lu J. Lanir Dani Lischinski Daniel Cohen-Or Hui Huang MLLM 164 12 0 24 Jan 2024

All Papers

Visual Programming: Compositional visual reasoning without training

Papers citing "Visual Programming: Compositional visual reasoning without training"