Supervised Pretraining Can Learn In-Context Reinforcement Learning

26 June 2023

Papers citing "Supervised Pretraining Can Learn In-Context Reinforcement Learning"

26 / 26 papers shown

Title
Toward Efficient Exploration by Large Language Model Agents Dilip Arumugam Thomas L. Griffiths LLMAG 87 0 0 29 Apr 2025
Text-to-Decision Agent: Learning Generalist Policies from Natural Language Supervision Shilin Zhang Zican Hu Wenhao Wu Xinyi Xie Jianxiang Tang Chunlin Chen Daoyi Dong Yu Cheng Zhenhong Sun Zhi Wang OffRL 45 0 0 21 Apr 2025
Yes, Q-learning Helps Offline In-Context RL Denis Tarasov Alexander Nikulin Ilya Zisman Albina Klepach Andrei Polubarov Nikita Lyubaykin Alexander Derevyagin Igor Kiselev Vladislav Kurenkov OffRL OnRL 82 0 0 24 Feb 2025
A Large Recurrent Action Model: xLSTM enables Fast Inference for Robotics Tasks Thomas Schmied Thomas Adler Vihang Patil M. Beck Korbinian Poppel Johannes Brandstetter G. Klambauer Razvan Pascanu Sepp Hochreiter 70 4 0 21 Feb 2025
Beyond Numeric Awards: In-Context Dueling Bandits with LLM Agents Fanzeng Xia Hao Liu Yisong Yue Tongxin Li 57 1 0 03 Jan 2025
ICLR: In-Context Learning of Representations Core Francisco Park Andrew Lee Ekdeep Singh Lubana Yongyi Yang Maya Okawa Kento Nishi Martin Wattenberg Hidenori Tanaka AIFin 111 3 0 29 Dec 2024
BALROG: Benchmarking Agentic LLM and VLM Reasoning On Games Davide Paglieri Bartłomiej Cupiał Samuel Coward Ulyana Piterbarg Maciej Wolczyk ... Lerrel Pinto Rob Fergus Jakob Foerster Jack Parker-Holder Tim Rocktaschel LLMAG LRM 101 10 0 20 Nov 2024
N-Gram Induction Heads for In-Context RL: Improving Stability and Reducing Data Needs Ilya Zisman Alexander Nikulin Andrei Polubarov Nikita Lyubaykin Vladislav Kurenkov Andrei Polubarov Igor Kiselev Vladislav Kurenkov OffRL 44 1 0 04 Nov 2024
Problem Solving Through Human-AI Preference-Based Cooperation Subhabrata Dutta Timo Kaufmann Goran Glavas Ivan Habernal Kristian Kersting Frauke Kreuter Mira Mezini Iryna Gurevych Eyke Hüllermeier Hinrich Schuetze 82 1 0 14 Aug 2024
BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack Yuri Kuratov Aydar Bulatov Petr Anokhin Ivan Rodkin Dmitry Sorokin Artyom Sorokin Mikhail Burtsev RALM ALM LRM ReLM ELM 42 57 0 14 Jun 2024
Pretraining Decision Transformers with Reward Prediction for In-Context Multi-task Structured Bandit Learning Subhojyoti Mukherjee Josiah P. Hanna Qiaomin Xie Robert Nowak 58 2 0 07 Jun 2024
From Words to Actions: Unveiling the Theoretical Underpinnings of LLM-Driven Autonomous Systems Jianliang He Siyu Chen Fengzhuo Zhang Zhuoran Yang LM&Ro LLMAG 40 2 0 30 May 2024
Preparing for Black Swans: The Antifragility Imperative for Machine Learning Ming Jin 27 2 0 18 May 2024
Towards an Information Theoretic Framework of Context-Based Offline Meta-Reinforcement Learning Lanqing Li Hai Zhang Xinyu Zhang Shatong Zhu Junqiao Zhao Junqiao Zhao Pheng-Ann Heng OffRL 26 7 0 04 Feb 2024
XLand-MiniGrid: Scalable Meta-Reinforcement Learning Environments in JAX Alexander Nikulin Vladislav Kurenkov Ilya Zisman Artem Agarkov Viacheslav Sinii Sergey Kolesnikov 21 23 0 19 Dec 2023
Transformers are Provably Optimal In-context Estimators for Wireless Communications Vishnu Teja Kunde Vicram Rajagopalan Chandra Shekhara Kaushik Valmeekam Krishna R. Narayanan S. Shakkottai D. Kalathil J. Chamberland 29 4 0 01 Nov 2023
Transformers as Decision Makers: Provable In-Context Reinforcement Learning via Supervised Pretraining Licong Lin Yu Bai Song Mei OffRL 27 42 0 12 Oct 2023
PASTA: Pretrained Action-State Transformer Agents Raphael Boige Yannis Flet-Berliac Arthur Flajolet Guillaume Richard Thomas Pierrot LM&Ro OffRL 22 5 0 20 Jul 2023
Large Language Models as General Pattern Machines Suvir Mirchandani F. Xia Peter R. Florence Brian Ichter Danny Driess Montse Gonzalez Arenas Kanishka Rao Dorsa Sadigh Andy Zeng LLMAG 37 183 0 10 Jul 2023
The Learnability of In-Context Learning Noam Wies Yoav Levine Amnon Shashua 114 89 0 14 Mar 2023
Foundation Models for Decision Making: Problems, Methods, and Opportunities Sherry Yang Ofir Nachum Yilun Du Jason W. Wei Pieter Abbeel Dale Schuurmans LM&Ro OffRL LRM AI4CE 90 148 0 07 Mar 2023
Structured State Space Models for In-Context Reinforcement Learning Chris Xiaoxuan Lu Yannick Schroecker Albert Gu Emilio Parisotto Jakob N. Foerster Satinder Singh Feryal M. P. Behbahani AI4TS 84 80 0 07 Mar 2023
In-context Learning and Induction Heads Catherine Olsson Nelson Elhage Neel Nanda Nicholas Joseph Nova Dassarma ... Tom B. Brown Jack Clark Jared Kaplan Sam McCandlish C. Olah 240 453 0 24 Sep 2022
COMBO: Conservative Offline Model-Based Policy Optimization Tianhe Yu Aviral Kumar Rafael Rafailov Aravind Rajeswaran Sergey Levine Chelsea Finn OffRL 197 412 0 16 Feb 2021
Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems Sergey Levine Aviral Kumar George Tucker Justin Fu OffRL GP 321 1,944 0 04 May 2020
Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks Chelsea Finn Pieter Abbeel Sergey Levine OOD 243 11,568 0 09 Mar 2017