VLM Agents Generate Their Own Memories: Distilling Experience into Embodied Programs of Thought

VLM Agents Generate Their Own Memories: Distilling Experience into Embodied Programs of Thought

20 June 2024

Gabriel H. Sarch

Michael J. Tarr

William W. Cohen

Katerina Fragkiadaki

Papers citing "VLM Agents Generate Their Own Memories: Distilling Experience into Embodied Programs of Thought"

15 / 15 papers shown

Title
VideoWebArena: Evaluating Long Context Multimodal Agents with Video Understanding Web Tasks Lawrence Jang Yinheng Li Charles Ding Justin Lin Paul Pu Liang Dan Zhao Rogerio Bonatti K. Koishida 23 5 0 24 Oct 2024
Skill Set Optimization: Reinforcing Language Model Behavior via Transferable Skills Kolby Nottingham Bodhisattwa Prasad Majumder Bhavana Dalvi Sameer Singh Peter Clark Roy Fox 24 7 0 05 Feb 2024
V*: Guided Visual Search as a Core Mechanism in Multimodal LLMs Penghao Wu Saining Xie LRM 44 120 0 21 Dec 2023
DANLI: Deliberative Agent for Following Natural Language Instructions Yichi Zhang Jianing Yang Jiayi Pan Shane Storks N. Devraj Ziqiao Ma Keunwoo Peter Yu Yuwei Bao J. Chai LM&Ro 42 16 0 22 Oct 2022
Don't Copy the Teacher: Data and Model Challenges in Embodied Dialogue So Yeon Min Hao Zhu Ruslan Salakhutdinov Yonatan Bisk LM&Ro 42 12 0 10 Oct 2022
ProgPrompt: Generating Situated Robot Task Plans using Large Language Models Ishika Singh Valts Blukis Arsalan Mousavian Ankit Goyal Danfei Xu Jonathan Tremblay D. Fox Jesse Thomason Animesh Garg LM&Ro LLMAG 107 616 0 22 Sep 2022
Large Language Models are Zero-Shot Reasoners Takeshi Kojima S. Gu Machel Reid Yutaka Matsuo Yusuke Iwasawa ReLM LRM 291 2,712 0 24 May 2022
Self-Consistency Improves Chain of Thought Reasoning in Language Models Xuezhi Wang Jason W. Wei Dale Schuurmans Quoc Le Ed H. Chi Sharan Narang Aakanksha Chowdhery Denny Zhou ReLM BDL LRM AI4CE 297 3,163 0 21 Mar 2022
DialFRED: Dialogue-Enabled Agents for Embodied Instruction Following Xiaofeng Gao Qiaozi Gao Ran Gong Kaixiang Lin Govind Thattai Gaurav Sukhatme LM&Ro 75 69 0 27 Feb 2022
Improving Intrinsic Exploration with Language Abstractions Jesse Mu Victor Zhong Roberta Raileanu Minqi Jiang Noah D. Goodman Tim Rocktaschel Edward Grefenstette 89 63 0 17 Feb 2022
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Jason W. Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Brian Ichter F. Xia Ed H. Chi Quoc Le Denny Zhou LM&Ro LRM AI4CE ReLM 315 8,261 0 28 Jan 2022
Ego4D: Around the World in 3,000 Hours of Egocentric Video Kristen Grauman Andrew Westbury Eugene Byrne Zachary Chavis Antonino Furnari ... Mike Zheng Shou Antonio Torralba Lorenzo Torresani Mingfei Yan Jitendra Malik EgoV 212 682 0 13 Oct 2021
FILM: Following Instructions in Language with Modular Methods So Yeon Min Devendra Singh Chaplot Pradeep Ravikumar Yonatan Bisk Ruslan Salakhutdinov LM&Ro 190 159 0 12 Oct 2021
TEACh: Task-driven Embodied Agents that Chat Aishwarya Padmakumar Jesse Thomason Ayush Shrivastava P. Lange Anjali Narayan-Chen Spandana Gella Robinson Piramithu Gökhan Tür Dilek Z. Hakkani-Tür LM&Ro 142 179 0 01 Oct 2021
Guiding Reinforcement Learning Exploration Using Natural Language Brent Harrison Upol Ehsan Mark O. Riedl OffRL 63 27 0 26 Jul 2017