GPT4Tools: Teaching Large Language Model to Use Tools via Self-instruction

30 May 2023

Sijie Zhao

Ying Shan

Papers citing "GPT4Tools: Teaching Large Language Model to Use Tools via Self-instruction"

34 / 34 papers shown

Title
When2Call: When (not) to Call Tools Hayley Ross Ameya Sunil Mahabaleshwarkar Yoshi Suhara 92 0 0 26 Apr 2025
UI-Vision: A Desktop-centric GUI Benchmark for Visual Perception and Interaction Shravan Nayak Xiangru Jian Kevin Qinghong Lin Juan A. Rodriguez Montek Kalsi ... David Vazquez Christopher Pal Perouz Taslakian Spandana Gella Sai Rajeswar 79 0 0 19 Mar 2025
Exposing Limitations of Language Model Agents in Sequential-Task Compositions on the Web Hiroki Furuta Yutaka Matsuo Aleksandra Faust Izzeddin Gur CLL 78 13 0 03 Jan 2025
CATP-LLM: Empowering Large Language Models for Cost-Aware Tool Planning Duo Wu J. Wang Yuan Meng Yanning Zhang Le Sun Zhi Wang 96 0 0 25 Nov 2024
From Exploration to Mastery: Enabling LLMs to Master Tools via Self-Driven Interactions Changle Qu Sunhao Dai Xiaochi Wei Hengyi Cai Shuaiqiang Wang Dawei Yin Jun Xu Ji-Rong Wen 51 9 0 10 Oct 2024
Learning Evolving Tools for Large Language Models Guoxin Chen Zhong Zhang Xin Cong Fangda Guo Yesai Wu Yankai Lin Wenzheng Feng Yasheng Wang KELM 52 1 0 09 Oct 2024
LLM With Tools: A Survey Zhuocheng Shen 25 8 0 24 Sep 2024
SONICS: Synthetic Or Not -- Identifying Counterfeit Songs Md Awsafur Rahman Zaber Ibn Abdul Hakim Najibul Haque Sarker Bishmoy Paul S. Fattah 33 6 0 26 Aug 2024
ReasonPix2Pix: Instruction Reasoning Dataset for Advanced Image Editing Ying Jin Pengyang Ling Xiao-wen Dong Pan Zhang Jiaqi Wang Dahua Lin 24 2 0 18 May 2024
A Survey on Self-Evolution of Large Language Models Zhengwei Tao Ting-En Lin Xiancai Chen Hangyu Li Yuchuan Wu Yongbin Li Zhi Jin Fei Huang Dacheng Tao Jingren Zhou LRM LM&Ro 43 21 0 22 Apr 2024
StableToolBench: Towards Stable Large-Scale Benchmarking on Tool Learning of Large Language Models Zhicheng Guo Sijie Cheng Hao Wang Shihao Liang Yujia Qin Peng Li Zhiyuan Liu Maosong Sun Yang Janet Liu ELM 41 22 0 12 Mar 2024
API Pack: A Massive Multi-Programming Language Dataset for API Call Generation Zhen Guo Adriana Meza Soria Wei Sun Yikang Shen Rameswar Panda ELM ALM 34 1 0 14 Feb 2024
Mobile-Agent: Autonomous Multi-Modal Mobile Device Agent with Visual Perception Junyang Wang Haiyang Xu Jiabo Ye Mingshi Yan Weizhou Shen Ji Zhang Fei Huang Jitao Sang 10 103 0 29 Jan 2024
MLLM-Tool: A Multimodal Large Language Model For Tool Agent Learning Chenyu Wang Weixin Luo Qianyu Chen Haonan Mai Jindi Guo Sixun Dong Xiaohua Xuan MLLM LLMAG 36 17 0 19 Jan 2024
LISA++: An Improved Baseline for Reasoning Segmentation with Large Language Model Senqiao Yang Tianyuan Qu Xin Lai Zhuotao Tian Bohao Peng Shu-Lin Liu Jiaya Jia VLM 21 28 0 28 Dec 2023
LLaMA-VID: An Image is Worth 2 Tokens in Large Language Models Yanwei Li Chengyao Wang Jiaya Jia VLM MLLM 26 259 0 28 Nov 2023
MMC: Advancing Multimodal Chart Understanding with Large-scale Instruction Tuning Fuxiao Liu Xiaoyang Wang Wenlin Yao Jianshu Chen Kaiqiang Song Sangwoo Cho Yaser Yacoob Dong Yu 10 98 0 15 Nov 2023
How Well Do Large Language Models Truly Ground? Hyunji Lee Se June Joo Chaeeun Kim Joel Jang Doyoung Kim Kyoung-Woon On Minjoon Seo HILM 8 6 0 15 Nov 2023
LLaVA-Plus: Learning to Use Tools for Creating Multimodal Agents Shilong Liu Hao Cheng Haotian Liu Hao Zhang Feng Li ... Hang Su Jun Zhu Lei Zhang Jianfeng Gao Chun-yue Li MLLM VLM 41 102 0 09 Nov 2023
ToolChain: Efficient Action Space Navigation in Large Language Models with A Search Yuchen Zhuang Xiang Chen Tong Yu Saayan Mitra Victor S. Bursztyn Ryan A. Rossi Somdeb Sarkhel Chao Zhang LLMAG 24 52 0 20 Oct 2023
ImageBind-LLM: Multi-modality Instruction Tuning Jiaming Han Renrui Zhang Wenqi Shao Peng Gao Peng-Tao Xu ... Yafei Wen Xiaoxin Chen Xiangyu Yue Hongsheng Li Yu Qiao MLLM 16 115 0 07 Sep 2023
Tool Documentation Enables Zero-Shot Tool-Usage with Large Language Models Cheng-Yu Hsieh Sibei Chen Chun-Liang Li Yasuhisa Fujii Alexander Ratner Chen-Yu Lee Ranjay Krishna Tomas Pfister LLMAG SyDa 27 40 0 01 Aug 2023
Insert-expansions for Tool-enabled Conversational Agents Andreas Göldi Roman Rietsche KELM 21 1 0 04 Jul 2023
Making Language Models Better Tool Learners with Execution Feedback Shuofei Qiao Honghao Gui Chengfei Lv Qianghuai Jia Huajun Chen Ningyu Zhang LLMAG 22 45 0 22 May 2023
Shap-E: Generating Conditional 3D Implicit Functions Heewoo Jun Alex Nichol DiffM 184 300 0 03 May 2023
Instruction Tuning with GPT-4 Baolin Peng Chunyuan Li Pengcheng He Michel Galley Jianfeng Gao SyDa ALM LM&MA 154 576 0 06 Apr 2023
UViM: A Unified Modeling Approach for Vision with Learned Guiding Codes Alexander Kolesnikov André Susano Pinto Lucas Beyer Xiaohua Zhai Jeremiah Harmsen N. Houlsby 92 67 0 20 May 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 301 11,730 0 04 Mar 2022
PromptSource: An Integrated Development Environment and Repository for Natural Language Prompts Stephen H. Bach Victor Sanh Zheng-Xin Yong Albert Webson Colin Raffel ... Khalid Almubarak Xiangru Tang Dragomir R. Radev Mike Tian-Jian Jiang Alexander M. Rush VLM 212 335 0 02 Feb 2022
BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation Junnan Li Dongxu Li Caiming Xiong S. Hoi MLLM BDL VLM CLIP 382 4,010 0 28 Jan 2022
An Empirical Study of GPT-3 for Few-Shot Knowledge-Based VQA Zhengyuan Yang Zhe Gan Jianfeng Wang Xiaowei Hu Yumao Lu Zicheng Liu Lijuan Wang 164 401 0 10 Sep 2021
Workshop on Autonomous Driving at CVPR 2021: Technical Report for Streaming Perception Challenge Songyang Zhang Lin Song Songtao Liu Zheng Ge Zeming Li Xuming He Jian-jun Sun 25 10 0 27 Jul 2021
Internet-Augmented Dialogue Generation M. Komeili Kurt Shuster Jason Weston RALM 231 278 0 15 Jul 2021
Learning Dynamic Routing for Semantic Segmentation Yanwei Li Lin Song Yukang Chen Zeming Li X. Zhang Xingang Wang Jian-jun Sun SSeg 77 161 0 23 Mar 2020