Tool Documentation Enables Zero-Shot Tool-Usage with Large Language Models

1 August 2023

Cheng-Yu Hsieh

Sibei Chen

Chun-Liang Li

Yasuhisa Fujii

Alexander Ratner

Chen-Yu Lee

Tomas Pfister

Papers citing "Tool Documentation Enables Zero-Shot Tool-Usage with Large Language Models"

46 / 46 papers shown

Title
VLM Q-Learning: Aligning Vision-Language Models for Interactive Decision-Making Jake Grigsby Yuke Zhu Michael S Ryoo Juan Carlos Niebles OffRL VLM 26 0 0 06 May 2025
Bridging Language Models and Financial Analysis Alejandro Lopez-Lira Jihoon Kwon Sangwoon Yoon Jy-yong Sohn Chanyeol Choi AIFin 36 0 0 14 Mar 2025
AgentStudio: A Toolkit for Building General Virtual Agents Longtao Zheng Zhiyuan Huang Zhenghai Xue Xinrun Wang Bo An Shuicheng Yan 69 14 0 17 Feb 2025
Exposing Limitations of Language Model Agents in Sequential-Task Compositions on the Web Hiroki Furuta Yutaka Matsuo Aleksandra Faust Izzeddin Gur CLL 71 13 0 03 Jan 2025
Mathematical Language Models: A Survey W. Liu Hanglei Hu Jie Zhou Yuyang Ding Junsong Li ... Mengliang He Qin Chen Bo Jiang Aimin Zhou Liang He LRM 65 12 0 03 Jan 2025
CAD-Assistant: Tool-Augmented VLLMs as Generic CAD Task Solvers Dimitrios Mallis Ahmet Serdar Karadeniz Sebastian Cavada Danila Rukhovich Niki Maria Foteinopoulou K. Cherenkova Anis Kacem Djamila Aouada 68 1 0 18 Dec 2024
From Exploration to Mastery: Enabling LLMs to Master Tools via Self-Driven Interactions Changle Qu Sunhao Dai Xiaochi Wei Hengyi Cai Shuaiqiang Wang Dawei Yin Jun Xu Ji-Rong Wen 51 9 0 10 Oct 2024
LLM With Tools: A Survey Zhuocheng Shen 23 8 0 24 Sep 2024
A Taxonomy of Architecture Options for Foundation Model-based Agents: Analysis and Decision Model Jingwen Zhou Qinghua Lu Jieshan Chen Liming Zhu Xiwei Xu Zhenchang Xing Stefan Harrer 31 0 0 06 Aug 2024
Segment Anything for Videos: A Systematic Survey Chunhui Zhang Yawen Cui Weilin Lin Guanjie Huang Yan Rong Li Liu Shiguang Shan VLM 29 6 0 31 Jul 2024
CodeUpdateArena: Benchmarking Knowledge Editing on API Updates Zeyu Leo Liu Shrey Pandit Xi Ye Eunsol Choi Greg Durrett KELM ALM 46 4 0 08 Jul 2024
Simulating Financial Market via Large Language Model based Agents Shen Gao Yuntao Wen Minghang Zhu Jianing Wei Yuhan Cheng Qunzi Zhang Shuo Shang AIFin 16 11 0 28 Jun 2024
Towards a Science Exocortex Kevin G. Yager 66 0 0 24 Jun 2024
CodeNav: Beyond tool-use to using real-world codebases with LLM agents Tanmay Gupta Luca Weihs Aniruddha Kembhavi LLMAG ELM 51 0 0 18 Jun 2024
Visual Sketchpad: Sketching as a Visual Chain of Thought for Multimodal Language Models Yushi Hu Weijia Shi Xingyu Fu Dan Roth Mari Ostendorf Luke Zettlemoyer Noah A. Smith Ranjay Krishna LRM 32 34 0 13 Jun 2024
Robust Planning with LLM-Modulo Framework: Case Study in Travel Planning Atharva Gundawar Mudit Verma L. Guan Karthik Valmeekam Siddhant Bhambri Subbarao Kambhampati 45 17 0 31 May 2024
Tool Learning with Large Language Models: A Survey Changle Qu Sunhao Dai Xiaochi Wei Hengyi Cai Shuaiqiang Wang Dawei Yin Jun Xu Jirong Wen LLMAG 31 77 0 28 May 2024
Tool Learning in the Wild: Empowering Language Models as Automatic Tool Agents Zhengliang Shi Shen Gao Xiuyi Chen Yue Feng Lingyong Yan Haibo Shi Dawei Yin Zhumin Chen Suzan Verberne LLMAG 47 14 0 26 May 2024
WorkBench: a Benchmark Dataset for Agents in a Realistic Workplace Setting Olly Styles Sam Miller Patricio Cerda-Mardini T. Guha Victor Sanchez Bertie Vidgen LLMAG 20 3 0 01 May 2024
From Persona to Personalization: A Survey on Role-Playing Language Agents Jiangjie Chen Xintao Wang Rui Xu Siyu Yuan Yikai Zhang ... Caiyu Hu Siye Wu Scott Ren Ziquan Fu Yanghua Xiao 50 72 0 28 Apr 2024
What Are Tools Anyway? A Survey from the Language Model Perspective Zhiruo Wang Zhoujun Cheng Hao Zhu Daniel Fried Graham Neubig 48 26 0 18 Mar 2024
Modeling Collaborator: Enabling Subjective Vision Classification With Minimal Human Effort via LLM Tool-Use Imad Eddine Toubal Aditya Avinash N. Alldrin Jan Dlabal Wenlei Zhou ... Chun-Ta Lu Howard Zhou Ranjay Krishna Ariel Fuxman Tom Duerig VLM 64 7 0 05 Mar 2024
AnyTool: Self-Reflective, Hierarchical Agents for Large-Scale API Calls Yu Du Fangyun Wei Hongyang R. Zhang LLMAG 24 32 0 06 Feb 2024
GraphiMind: LLM-centric Interface for Information Graphics Design Qiruin Huang Min Lu J. Lanir Dani Lischinski Daniel Cohen-Or Hui Huang MLLM 8 7 0 24 Jan 2024
Exploring the Reasoning Abilities of Multimodal Large Language Models (MLLMs): A Comprehensive Survey on Emerging Trends in Multimodal Reasoning Yiqi Wang Wentao Chen Xiaotian Han Xudong Lin Haiteng Zhao Yongfei Liu Bohan Zhai Jianbo Yuan Quanzeng You Hongxia Yang LRM 17 66 0 10 Jan 2024
The Tyranny of Possibilities in the Design of Task-Oriented LLM Systems: A Scoping Survey Dhruv Dhamani Mary Lou Maher 14 1 0 29 Dec 2023
When does In-context Learning Fall Short and Why? A Study on Specification-Heavy Tasks Hao Peng Xiaozhi Wang Jianhui Chen Weikai Li Y. Qi ... Zhili Wu Kaisheng Zeng Bin Xu Lei Hou Juanzi Li 11 27 0 15 Nov 2023
MetaTool Benchmark for Large Language Models: Deciding Whether to Use Tools and Which to Use Yue Huang Jiawen Shi Yuan Li Chenrui Fan Siyuan Wu ... Yixin Liu Pan Zhou Yao Wan Neil Zhenqiang Gong Lichao Sun LLMAG 29 82 0 04 Oct 2023
GRID: A Platform for General Robot Intelligence Development Sai H. Vemprala Shuhang Chen Abhinav Shukla Dinesh Narayanan Ashish Kapoor 17 10 0 02 Oct 2023
TPE: Towards Better Compositional Reasoning over Conceptual Tools with Multi-persona Collaboration Hongru Wang Huimin Wang Lingzhi Wang Minda Hu Rui Wang Boyang Xue Hongyuan Lu Fei Mi Kam-Fai Wong LRM KELM LLMAG 14 12 0 28 Sep 2023
ZYN: Zero-Shot Reward Models with Yes-No Questions for RLAIF Víctor Gallego SyDa 16 4 0 11 Aug 2023
BOLAA: Benchmarking and Orchestrating LLM-augmented Autonomous Agents Zhiwei Liu Weiran Yao Jianguo Zhang Le Xue Shelby Heinecke ... Ran Xu P. Mùi Haiquan Wang Caiming Xiong Silvio Savarese LLMAG 13 81 0 11 Aug 2023
A Comprehensive Overview of Large Language Models Humza Naveed Asad Ullah Khan Shi Qiu Muhammad Saqib Saeed Anwar Muhammad Usman Naveed Akhtar Nick Barnes Ajmal Saeed Mian OffRL 27 499 0 12 Jul 2023
Domain-specific ChatBots for Science using Embeddings Kevin G. Yager 12 8 0 15 Jun 2023
A Survey on Segment Anything Model (SAM): Vision Foundation Model Meets Prompt Engineering Chaoning Zhang Fachrina Dewi Puspitasari Sheng Zheng Chenghao Li Yu Qiao ... Caiyan Qin François Rameau Lik-Hang Lee Sung-Ho Bae Choong Seon Hong VLM 76 61 0 12 May 2023
Multimodal Procedural Planning via Dual Text-Image Prompting Yujie Lu Pan Lu Zhiyu Zoey Chen Wanrong Zhu X. Wang William Yang Wang LM&Ro 48 43 0 02 May 2023
Unlimiformer: Long-Range Transformers with Unlimited Length Input Amanda Bertsch Uri Alon Graham Neubig Matthew R. Gormley RALM 91 122 0 02 May 2023
Sparks of Artificial General Intelligence: Early experiments with GPT-4 Sébastien Bubeck Varun Chandrasekaran Ronen Eldan J. Gehrke Eric Horvitz ... Scott M. Lundberg Harsha Nori Hamid Palangi Marco Tulio Ribeiro Yi Zhang ELM AI4MH AI4CE ALM 197 2,953 0 22 Mar 2023
Foundation Models for Decision Making: Problems, Methods, and Opportunities Sherry Yang Ofir Nachum Yilun Du Jason W. Wei Pieter Abbeel Dale Schuurmans LM&Ro OffRL LRM AI4CE 87 148 0 07 Mar 2023
Mind's Eye: Grounded Language Model Reasoning through Simulation Ruibo Liu Jason W. Wei S. Gu Te-Yen Wu Soroush Vosoughi Claire Cui Denny Zhou Andrew M. Dai ReLM LRM 106 78 0 11 Oct 2022
ReAct: Synergizing Reasoning and Acting in Language Models Shunyu Yao Jeffrey Zhao Dian Yu Nan Du Izhak Shafran Karthik Narasimhan Yuan Cao LLMAG ReLM LRM 208 2,413 0 06 Oct 2022
Generate rather than Retrieve: Large Language Models are Strong Context Generators W. Yu Dan Iter Shuohang Wang Yichong Xu Mingxuan Ju Soumya Sanyal Chenguang Zhu Michael Zeng Meng-Long Jiang RALM AIMat 205 318 0 21 Sep 2022
Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering Pan Lu Swaroop Mishra Tony Xia Liang Qiu Kai-Wei Chang Song-Chun Zhu Oyvind Tafjord Peter Clark A. Kalyan ELM ReLM LRM 198 1,089 0 20 Sep 2022
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Jason W. Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Brian Ichter F. Xia Ed H. Chi Quoc Le Denny Zhou LM&Ro LRM AI4CE ReLM 315 8,261 0 28 Jan 2022
Internet-Augmented Dialogue Generation M. Komeili Kurt Shuster Jason Weston RALM 228 278 0 15 Jul 2021
What Makes Good In-Context Examples for GPT- $3$ ? Jiachang Liu Dinghan Shen Yizhe Zhang Bill Dolan Lawrence Carin Weizhu Chen AAML RALM 275 1,296 0 17 Jan 2021