ViperGPT: Visual Inference via Python Execution for Reasoning

14 March 2023

Carl Vondrick

Papers citing "ViperGPT: Visual Inference via Python Execution for Reasoning"

50 / 339 papers shown

Title
Neurosymbolic Grounding for Compositional World Models Atharva Sehgal Arya Grayeli Jennifer J. Sun Swarat Chaudhuri 14 5 0 19 Oct 2023
Survey of Vulnerabilities in Large Language Models Revealed by Adversarial Attacks Erfan Shayegani Md Abdullah Al Mamun Yu Fu Pedram Zaree Yue Dong Nael B. Abu-Ghazaleh AAML 147 139 0 16 Oct 2023
Octopus: Embodied Vision-Language Programmer from Environmental Feedback Jingkang Yang Yuhao Dong Shuai Liu Bo-wen Li Ziyue Wang ... Haoran Tan Jiamu Kang Yuanhan Zhang Kaiyang Zhou Ziwei Liu LM&Ro 31 45 0 12 Oct 2023
Idea2Img: Iterative Self-Refinement with GPT-4V(ision) for Automatic Image Design and Generation Zhengyuan Yang Jianfeng Wang Linjie Li Kevin Qinghong Lin Chung-Ching Lin Zicheng Liu Lijuan Wang LRM MLLM DiffM 13 22 0 12 Oct 2023
Towards Robust Multi-Modal Reasoning via Model Selection Xiangyan Liu Rongxue Li Wei Ji Tao Lin LLMAG LRM 22 3 0 12 Oct 2023
OpenLEAF: Open-Domain Interleaved Image-Text Generation and Evaluation Jie An Zhengyuan Yang Linjie Li Jianfeng Wang K. Lin Zicheng Liu Lijuan Wang Jiebo Luo 14 11 0 11 Oct 2023
Lemur: Harmonizing Natural Language and Code for Language Agents Yiheng Xu Hongjin Su Chen Xing Boyu Mi Qian Liu ... Siheng Zhao Lingpeng Kong Bailin Wang Caiming Xiong Tao Yu 27 67 0 10 Oct 2023
What If the TV Was Off? Examining Counterfactual Reasoning Abilities of Multi-modal Language Models Letian Zhang Xiaotong Zhai Zhongkai Zhao Yongshuo Zong Xin Wen Bingchen Zhao LRM 11 0 0 10 Oct 2023
ViCor: Bridging Visual Understanding and Commonsense Reasoning with Large Language Models KAI-QING Zhou Kwonjoon Lee Teruhisa Misu Xin Eric Wang LRM 19 3 0 09 Oct 2023
InstructDET: Diversifying Referring Object Detection with Generalized Instructions Ronghao Dang Jiangyan Feng Haodong Zhang Chongjian Ge Lin Song ... Chengju Liu Qi Chen Feng Zhu Rui Zhao Yibing Song ObjD 8 11 0 08 Oct 2023
Language Agent Tree Search Unifies Reasoning Acting and Planning in Language Models Xiaoxiao Sun Yang Yang Michal Shlapentokh-Rothman Haohan Wang Yu-xiong Wang LRM AI4CE LM&Ro LLMAG 29 183 0 06 Oct 2023
GRID: A Platform for General Robot Intelligence Development Sai H. Vemprala Shuhang Chen Abhinav Shukla Dinesh Narayanan Ashish Kapoor 17 10 0 02 Oct 2023
CRAFT: Customizing LLMs by Creating and Retrieving from Specialized Toolsets Lifan Yuan Yangyi Chen Xingyao Wang Yi Ren Fung Hao Peng Heng Ji LLMAG KELM 21 56 0 29 Sep 2023
Guiding Instruction-based Image Editing via Multimodal Large Language Models Johannes Frey Wenze Hu Xianzhi Du William Yang Wang Yinfei Yang Zhe Gan 40 86 0 29 Sep 2023
Compositional Sculpting of Iterative Generative Processes Yixuan Wang Sebastiaan De Peuter Mingtong Zhang Vikas K. Garg Samuel Kaski Tommi Jaakkola DiffM 17 15 0 28 Sep 2023
VideoDirectorGPT: Consistent Multi-scene Video Generation via LLM-Guided Planning Han Lin Abhaysinh Zala Jaemin Cho Mohit Bansal LM&Ro VGen DiffM 32 74 0 26 Sep 2023
DreamLLM: Synergistic Multimodal Comprehension and Creation Runpei Dong Chunrui Han Yuang Peng Zekun Qi Zheng Ge ... Hao-Ran Wei Xiangwen Kong Xiangyu Zhang Kaisheng Ma Li Yi MLLM 28 168 0 20 Sep 2023
Kosmos-2.5: A Multimodal Literate Model Tengchao Lv Yupan Huang Jingye Chen Lei Cui Shuming Ma ... Weiyao Luo Shaoxiang Wu Guoxin Wang Cha Zhang Furu Wei VLM MLLM 21 63 0 20 Sep 2023
Multimodal Multi-Hop Question Answering Through a Conversation Between Tools and Efficiently Finetuned Large Language Models Hossein Rajabzadeh Suyuchen Wang Hyock Ju Kwon Bang Liu KELM 16 3 0 16 Sep 2023
D3: Data Diversity Design for Systematic Generalization in Visual Question Answering Amir Rahimi Vanessa D’Amario Moyuru Yamada Kentaro Takemoto Tomotake Sasaki Xavier Boix 17 1 0 15 Sep 2023
TextBind: Multi-turn Interleaved Multimodal Instruction-following in the Wild Huayang Li Siheng Li Deng Cai Longyue Wang Lemao Liu Taro Watanabe Yujiu Yang Shuming Shi MLLM 44 17 0 14 Sep 2023
Hypothesis Search: Inductive Reasoning with Language Models Ruocheng Wang E. Zelikman Gabriel Poesia Yewen Pu Nick Haber Noah D. Goodman ReLM LRM 11 94 0 11 Sep 2023
Gesture-Informed Robot Assistance via Foundation Models Li-Heng Lin Yuchen Cui Yilun Hao Fei Xia Dorsa Sadigh LM&Ro SLR 13 19 0 06 Sep 2023
PointLLM: Empowering Large Language Models to Understand Point Clouds Runsen Xu Xiaolong Wang Tai Wang Yilun Chen Jiangmiao Pang Dahua Lin MLLM 51 146 0 31 Aug 2023
Examining User-Friendly and Open-Sourced Large GPT Models: A Survey on Language, Multimodal, and Scientific GPT Models Kaiyuan Gao Su He Zhenyu He Jiacheng Lin Qizhi Pei Jie Shao Wei Zhang LM&MA SyDa 30 4 0 27 Aug 2023
A Survey on Large Language Model based Autonomous Agents Lei Wang Chengbang Ma Xueyang Feng Zeyu Zhang Hao-ran Yang ... Xu Chen Yankai Lin Wayne Xin Zhao Zhewei Wei Ji-Rong Wen LLMAG AI4CE LM&Ro 39 1,088 0 22 Aug 2023
Through the Lens of Core Competency: Survey on Evaluation of Large Language Models Ziyu Zhuang Qiguang Chen Longxuan Ma Mingda Li Yi Han Yushan Qian Haopeng Bai Zixian Feng Weinan Zhang Ting Liu ELM 19 9 0 15 Aug 2023
VisIT-Bench: A Benchmark for Vision-Language Instruction Following Inspired by Real-World Use Yonatan Bitton Hritik Bansal Jack Hessel Rulin Shao Wanrong Zhu Anas Awadalla Josh Gardner Rohan Taori L. Schimdt VLM 29 76 0 12 Aug 2023
Dynamic Planning with a LLM Gautier Dagan Frank Keller A. Lascarides LLMAG 92 34 0 11 Aug 2023
TPTU: Large Language Model-based AI Agents for Task Planning and Tool Usage Jingqing Ruan Yihong Chen Bin Zhang Zhiwei Xu Tianpeng Bao ... Shiwei Shi Hangyu Mao Ziyue Li Xingyu Zeng Rui Zhao LLMAG LM&Ro 39 31 0 07 Aug 2023
Tool Documentation Enables Zero-Shot Tool-Usage with Large Language Models Cheng-Yu Hsieh Sibei Chen Chun-Liang Li Yasuhisa Fujii Alexander Ratner Chen-Yu Lee Ranjay Krishna Tomas Pfister LLMAG SyDa 29 40 0 01 Aug 2023
AntGPT: Can Large Language Models Help Long-term Action Anticipation from Videos? Qi Zhao Shijie Wang Ce Zhang Changcheng Fu Minh Quan Do Nakul Agarwal Kwonjoon Lee Chen Sun LM&Ro 44 48 0 31 Jul 2023
WavJourney: Compositional Audio Creation with Large Language Models Xubo Liu Zhongkai Zhu Haohe Liu Yiitan Yuan Meng Cui ... Jinhua Liang Yin Cao Qiuqiang Kong Mark D. Plumbley Wenwu Wang AuLLM 21 25 0 26 Jul 2023
Foundational Models Defining a New Era in Vision: A Survey and Outlook Muhammad Awais Muzammal Naseer Salman Khan Rao Muhammad Anwer Hisham Cholakkal M. Shah Ming Yang F. Khan VLM 18 116 0 25 Jul 2023
A Comprehensive Overview of Large Language Models Humza Naveed Asad Ullah Khan Shi Qiu Muhammad Saqib Saeed Anwar Muhammad Usman Naveed Akhtar Nick Barnes Ajmal Saeed Mian OffRL 46 499 0 12 Jul 2023
VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Models Wenlong Huang Chen Wang Ruohan Zhang Yunzhu Li Jiajun Wu Li Fei-Fei LM&Ro 28 474 0 12 Jul 2023
AmadeusGPT: a natural language interface for interactive animal behavioral analysis Shaokai Ye Jessy Lauer Mu Zhou Alexander Mathis Mackenzie W. Mathis MLLM LLMAG 27 17 0 10 Jul 2023
GPT4RoI: Instruction Tuning Large Language Model on Region-of-Interest Shilong Zhang Pei Sun Shoufa Chen Min Xiao Wenqi Shao Wenwei Zhang Yu Liu Kai-xiang Chen Ping Luo VLM MLLM 80 223 0 07 Jul 2023
Statler: State-Maintaining Language Models for Embodied Reasoning Takuma Yoneda Jiading Fang Peng Li Huanyu Zhang Tianchong Jiang Shengjie Lin Ben Picker David Yunis Hongyuan Mei Matthew R. Walter LM&Ro 15 32 0 30 Jun 2023
Look, Remember and Reason: Grounded reasoning in videos with language models Apratim Bhattacharyya Sunny Panchal Mingu Lee Reza Pourreza Pulkit Madan Roland Memisevic LRM 30 7 0 30 Jun 2023
Towards Language Models That Can See: Computer Vision Through the LENS of Natural Language William Berrios Gautam Mittal Tristan Thrush Douwe Kiela Amanpreet Singh MLLM VLM 13 59 0 28 Jun 2023
Investigating Prompting Techniques for Zero- and Few-Shot Visual Question Answering Rabiul Awal Le Zhang Aishwarya Agrawal LRM 38 12 0 16 Jun 2023
Tell Me Where to Go: A Composable Framework for Context-Aware Embodied Robot Navigation Harel Biggie Ajay Narasimha Mopidevi Dusty Woods Christoffer Heckman LM&Ro 11 11 0 15 Jun 2023
Toward Grounded Commonsense Reasoning Minae Kwon Hengyuan Hu Vivek Myers Siddharth Karamcheti Anca Dragan Dorsa Sadigh LM&Ro ReLM LRM 20 8 0 14 Jun 2023
Towards AGI in Computer Vision: Lessons Learned from GPT and Large Language Models Lingxi Xie Longhui Wei Xiaopeng Zhang Kaifeng Bi Xiaotao Gu Jianlong Chang Qi Tian 29 6 0 14 Jun 2023
AssistGPT: A General Multi-modal Assistant that can Plan, Execute, Inspect, and Learn Difei Gao Lei Ji Luowei Zhou Kevin Lin Joya Chen Zihan Fan Mike Zheng Shou MLLM 19 71 0 14 Jun 2023
AVIS: Autonomous Visual Information Seeking with Large Language Model Agent Ziniu Hu Ahmet Iscen Chen Sun Kai-Wei Chang Yizhou Sun David A. Ross Cordelia Schmid Alireza Fathi 23 11 0 13 Jun 2023
Data-Copilot: Bridging Billions of Data and Humans with Autonomous Workflow Wenqi Zhang Yongliang Shen Weiming Lu Y. Zhuang LLMAG 23 50 0 12 Jun 2023
Looking Around Corners: Generative Methods in Terrain Extension Alec Reed Christoffer Heckman 13 1 0 12 Jun 2023
LAMM: Language-Assisted Multi-Modal Instruction-Tuning Dataset, Framework, and Benchmark Zhen-fei Yin Jiong Wang Jianjian Cao Zhelun Shi Dingning Liu ... Lei Bai Xiaoshui Huang Zhiyong Wang Jing Shao Wanli Ouyang MLLM 22 151 0 11 Jun 2023