Interactive Grounded Language Acquisition and Generalization in a 2D World

31 January 2018

Papers citing "Interactive Grounded Language Acquisition and Generalization in a 2D World"

50 / 54 papers shown

Title
CoPESD: A Multi-Level Surgical Motion Dataset for Training Large Vision-Language Models to Co-Pilot Endoscopic Submucosal Dissection Guankun Wang Han Xiao Huxin Gao Renrui Zhang Long Bai Xiaoxiao Yang Zhen Li Hongsheng Li Hongliang Ren 36 4 0 10 Oct 2024
Improve the efficiency of deep reinforcement learning through semantic exploration guided by natural language Zhourui Guo Meng Yao Yang Yu Qiyue Yin OnRL 21 1 0 21 Sep 2023
Type-to-Track: Retrieve Any Object via Prompt-based Tracking Pha Nguyen Kha Gia Quach Kris M. Kitani Khoa Luu 30 18 0 22 May 2023
Distilling Internet-Scale Vision-Language Models into Embodied Agents T. Sumers Kenneth Marino Arun Ahuja Rob Fergus Ishita Dasgupta LM&Ro 21 24 0 29 Jan 2023
Do Embodied Agents Dream of Pixelated Sheep: Embodied Decision Making using Language Guided World Modelling Kolby Nottingham Prithviraj Ammanabrolu Alane Suhr Yejin Choi Hannaneh Hajishirzi Sameer Singh Roy Fox LLMAG LM&Ro 28 76 0 28 Jan 2023
A Data-Efficient Visual-Audio Representation with Intuitive Fine-tuning for Voice-Controlled Robots Peixin Chang Shuijing Liu Tianchen Ji Neeloy Chakraborty Kaiwen Hong Katherine Driggs-Campbell 32 3 0 23 Jan 2023
A Short Survey of Systematic Generalization Yuanpeng Li AI4CE 22 1 0 22 Nov 2022
Contrastive language and vision learning of general fashion concepts P. Chia Giuseppe Attanasio Federico Bianchi Silvia Terragni A. Magalhães Diogo Gonçalves C. Greco Jacopo Tagliabue CLIP 13 42 0 08 Apr 2022
CALVIN: A Benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation Tasks Oier Mees Lukás Hermann Erick Rosete-Beas Wolfram Burgard LM&Ro 14 239 0 06 Dec 2021
Explainable Semantic Space by Grounding Language to Vision with Cross-Modal Contrastive Learning Yizhen Zhang Minkyu Choi Kuan Han Zhongming Liu VLM 13 15 0 13 Nov 2021
In a Nutshell, the Human Asked for This: Latent Goals for Following Temporal Specifications Borja G. Leon Murray Shanahan Francesco Belardinelli AI4CE 18 15 0 18 Oct 2021
Systematic Generalization on gSCAN: What is Nearly Solved and What is Next? Linlu Qiu Hexiang Hu Bowen Zhang Peter Shaw Fei Sha 20 21 0 25 Sep 2021
Evolving Decomposed Plasticity Rules for Information-Bottlenecked Meta-Learning Fan Wang Hao Tian Haoyi Xiong Hua-Hong Wu Jie Fu Yang Cao Yu Kang Haifeng Wang AI4CE 15 3 0 08 Sep 2021
Learning Visual-Audio Representations for Voice-Controlled Robots Peixin Chang Shuijing Liu D. L. McPherson Katherine Driggs-Campbell SSL 11 4 0 07 Sep 2021
Vision-Language Navigation: A Survey and Taxonomy Wansen Wu Tao Chang Xinmeng Li LM&Ro 6 19 0 26 Aug 2021
Communicative Learning with Natural Gestures for Embodied Navigation Agents with Human-in-the-Scene Qi Wu Cheng-Ju Wu Yixin Zhu Jungseock Joo 38 14 0 05 Aug 2021
Interactive Explanations: Diagnosis and Repair of Reinforcement Learning Based Agent Behaviors Christian Arzate Cruz Takeo Igarashi 11 7 0 27 May 2021
Fast and Slow Learning of Recurrent Independent Mechanisms Kanika Madan Rosemary Nan Ke Anirudh Goyal Bernhard Schölkopf Yoshua Bengio OCL 11 40 0 18 May 2021
Language in a (Search) Box: Grounding Language Learning in Real-World Human-Machine Interaction Federico Bianchi C. Greco Jacopo Tagliabue 27 9 0 18 Apr 2021
Reinforcement Learning of Implicit and Explicit Control Flow in Instructions Ethan A. Brooks Janarthanan Rajendran Richard L. Lewis Satinder Singh 8 10 0 25 Feb 2021
LTL2Action: Generalizing LTL Instructions for Multi-Task RL Pashootan Vaezipoor Andrew C. Li Rodrigo Toro Icarte Sheila A. McIlraith OffRL AI4CE 31 73 0 13 Feb 2021
A Survey on Deep Reinforcement Learning for Audio-Based Applications S. Latif Heriberto Cuayáhuitl Farrukh Pervez Fahad Shamshad Hafiz Shehbaz Ali Erik Cambria OffRL 37 73 0 01 Jan 2021
Visually Grounding Language Instruction for History-Dependent Manipulation Hyemin Ahn Obin Kwon Kyungdo Kim Jaeyeon Jeong Howoong Jun Hongjung Lee Dongheui Lee Songhwai Oh LM&Ro 13 6 0 16 Dec 2020
Ask Your Humans: Using Human Instructions to Improve Generalization in Reinforcement Learning Valerie Chen Abhinav Gupta Kenneth Marino OffRL 10 40 0 01 Nov 2020
Multimodal Aggregation Approach for Memory Vision-Voice Indoor Navigation with Meta-Learning Liqi Yan Dongfang Liu Yaoxian Song Changbin (Brad) Yu 6 14 0 01 Sep 2020
Compositional Networks Enable Systematic Generalization for Grounded Language Understanding Yen-Ling Kuo Boris Katz Andrei Barbu 26 22 0 06 Aug 2020
Systematic Generalisation through Task Temporal Logic and Deep Reinforcement Learning Borja G. Leon Murray Shanahan Francesco Belardinelli NAI AI4CE 31 28 0 12 Jun 2020
Probing Emergent Semantics in Predictive Agents via Question Answering Abhishek Das Federico Carnevale Hamza Merzic Laura Rimell R. Schneider ... Alden Hung Arun Ahuja S. Clark Greg Wayne Felix Hill 22 18 0 01 Jun 2020
Human Instruction-Following with Deep Reinforcement Learning via Transfer-Learning from Text Felix Hill Soňa Mokrá Nathaniel Wong Tim Harley LM&Ro 11 81 0 19 May 2020
Language Conditioned Imitation Learning over Unstructured Data Corey Lynch P. Sermanet LM&Ro 22 241 0 15 May 2020
Robust and Interpretable Grounding of Spatial References with Relation Networks Tsung-Yen Yang Andrew S. Lan Karthik Narasimham 19 12 0 02 May 2020
Zero-Shot Compositional Policy Learning via Language Grounding Tianshi Cao Jingkang Wang Yining Zhang S. Manivasagam LM&Ro 28 1 0 15 Apr 2020
Exploiting Language Instructions for Interpretable and Compositional Reinforcement Learning Michiel van der Meer Matteo Pirotta Elia Bruni 14 1 0 13 Jan 2020
Look, Listen, and Act: Towards Audio-Visual Embodied Navigation Chuang Gan Yiwei Zhang Jiajun Wu Boqing Gong J. Tenenbaum 9 137 0 25 Dec 2019
Learning to Request Guidance in Emergent Communication Benjamin Kolb Leon Lang H. Bartsch Arwin Gansekoele Raymond Koopmanschap Leonardo Romor David Speck Mathijs Mul Elia Bruni 15 0 0 11 Dec 2019
Automated curriculum generation for Policy Gradients from Demonstrations A. Srinivasan Dzmitry Bahdanau Maxime Chevalier-Boisvert Yoshua Bengio 15 1 0 01 Dec 2019
Emergence of Numeric Concepts in Multi-Agent Autonomous Communication Shangmin Guo LLMAG 8 3 0 04 Nov 2019
Task-Oriented Language Grounding for Language Input with Multiple Sub-Goals of Non-Linear Order Vladislav Kurenkov Bulat Maksudov Adil Khan 6 1 0 27 Oct 2019
Environmental drivers of systematicity and generalization in a situated agent Felix Hill Andrew Kyle Lampinen R. Schneider S. Clark M. Botvinick James L. McClelland Adam Santoro OOD 4 103 0 01 Oct 2019
Robot Sound Interpretation: Combining Sight and Sound in Learning-Based Control Peixin Chang Shuijing Liu Haonan Chen Katherine Driggs-Campbell 12 8 0 19 Sep 2019
Mastering emergent language: learning to guide in simulated navigation Mathijs Mul Diane Bouchacourt Elia Bruni LLMAG 16 9 0 14 Aug 2019
Why Build an Assistant in Minecraft? Arthur Szlam Jonathan Gray Kavya Srinet Yacine Jernite Armand Joulin ... Siddharth Goyal Demi Guo Dan Rothermel C. L. Zitnick Jason Weston LLMAG 12 28 0 22 Jul 2019
A Survey of Reinforcement Learning Informed by Natural Language Jelena Luketina Nantas Nardelli Gregory Farquhar Jakob N. Foerster Jacob Andreas Edward Grefenstette Shimon Whiteson Tim Rocktaschel LM&Ro KELM OffRL LRM 11 278 0 10 Jun 2019
REVERIE: Remote Embodied Visual Referring Expression in Real Indoor Environments Yuankai Qi Qi Wu Peter Anderson X. Wang W. Wang Chunhua Shen A. Hengel LM&Ro 12 316 0 23 Apr 2019
ACTRCE: Augmenting Experience via Teacher's Advice For Multi-Goal Reinforcement Learning Harris Chan Yuhuai Wu J. Kiros Sanja Fidler Jimmy Ba 23 34 0 12 Feb 2019
Embodied Multimodal Multitask Learning Devendra Singh Chaplot Lisa Lee Ruslan Salakhutdinov Devi Parikh Dhruv Batra LM&Ro 16 24 0 04 Feb 2019
Learning to Learn How to Learn: Self-Adaptive Visual Navigation Using Meta-Learning Mitchell Wortsman Kiana Ehsani Mohammad Rastegari Ali Farhadi Roozbeh Mottaghi SSL 18 222 0 03 Dec 2018
Object-oriented Targets for Visual Navigation using Rich Semantic Representations Jean-Benoit Delbrouck Stéphane Dupont 18 3 0 22 Nov 2018
Neural Modular Control for Embodied Question Answering Abhishek Das Georgia Gkioxari Stefan Lee Devi Parikh Dhruv Batra LM&Ro 123 127 0 26 Oct 2018
Visual Semantic Navigation using Scene Priors Wei Yang X. Wang Ali Farhadi Abhinav Gupta Roozbeh Mottaghi LM&Ro 11 320 0 15 Oct 2018