Transformers as Decision Makers: Provable In-Context Reinforcement Learning via Supervised Pretraining

12 October 2023

Papers citing "Transformers as Decision Makers: Provable In-Context Reinforcement Learning via Supervised Pretraining"

39 / 39 papers shown

Title
Free Random Projection for In-Context Reinforcement Learning Tomohiro Hayase B. Collins Nakamasa Inoue 14 0 0 09 Apr 2025
On the Robustness of Transformers against Context Hijacking for Linear Classification Tianle Li Chenyang Zhang Xingwu Chen Yuan Cao Difan Zou 67 0 0 24 Feb 2025
Training a Generally Curious Agent Fahim Tajwar Yiding Jiang Abitha Thankaraj Sumaita Sadia Rahman J. Zico Kolter Jeff Schneider Ruslan Salakhutdinov 115 1 0 24 Feb 2025
OT-Transformer: A Continuous-time Transformer Architecture with Optimal Transport Regularization Kelvin Kan Xingjian Li Stanley Osher 89 2 0 30 Jan 2025
Hierarchical Prompt Decision Transformer: Improving Few-Shot Policy Generalization with Global and Adaptive Guidance Zhe Wang Haozhu Wang Yanjun Qi OffRL 76 0 0 01 Dec 2024
HVAC-DPT: A Decision Pretrained Transformer for HVAC Control Anaïs Berkes AI4CE 67 0 0 29 Nov 2024
BALROG: Benchmarking Agentic LLM and VLM Reasoning On Games Davide Paglieri Bartłomiej Cupiał Samuel Coward Ulyana Piterbarg Maciej Wolczyk ... Lerrel Pinto Rob Fergus Jakob Foerster Jack Parker-Holder Tim Rocktaschel LLMAG LRM 101 10 0 20 Nov 2024
Reinforcement Learning Gradients as Vitamin for Online Finetuning Decision Transformers Kai Yan A. Schwing Yu-xiong Wang OffRL OnRL 34 0 0 31 Oct 2024
Provable optimal transport with transformers: The essence of depth and prompt engineering Hadi Daneshmand OT 20 0 0 25 Oct 2024
Random Policy Enables In-Context Reinforcement Learning within Trust Horizons Weiqin Chen Santiago Paternain OffRL 30 0 0 25 Oct 2024
CoPS: Empowering LLM Agents with Provable Cross-Task Experience Sharing Chen Yang Chenyang Zhao Q. Gu Dongruo Zhou LRM 38 0 0 22 Oct 2024
Active-Dormant Attention Heads: Mechanistically Demystifying Extreme-Token Phenomena in LLMs Tianyu Guo Druv Pai Yu Bai Jiantao Jiao Michael I. Jordan Song Mei 29 9 0 17 Oct 2024
Context-Scaling versus Task-Scaling in In-Context Learning Amirhesam Abedsoltan Adityanarayanan Radhakrishnan Jingfeng Wu M. Belkin ReLM LRM 32 3 0 16 Oct 2024
On the Training Convergence of Transformers for In-Context Classification Wei Shen Ruida Zhou Jing Yang Cong Shen 16 3 0 15 Oct 2024
A Theoretical Survey on Foundation Models Shi Fu Yuzhu Chen Yingjie Wang Dacheng Tao 21 0 0 15 Oct 2024
Retrieval-Augmented Decision Transformer: External Memory for In-context RL Thomas Schmied Fabian Paischer Vihang Patil M. Hofmarcher Razvan Pascanu Sepp Hochreiter OffRL 28 6 0 09 Oct 2024
EVOLvE: Evaluating and Optimizing LLMs For Exploration Allen Nie Yi Su Bo Chang Jonathan N. Lee Ed H. Chi Quoc V. Le Minmin Chen 16 5 0 08 Oct 2024
Task Diversity Shortens the ICL Plateau Jaeyeon Kim Sehyun Kwon Joo Young Choi Jongho Park Jaewoong Cho Jason D. Lee Ernest K. Ryu MoMe 29 2 0 07 Oct 2024
Non-asymptotic Convergence of Training Transformers for Next-token Prediction Ruiquan Huang Yingbin Liang Jing Yang 16 5 0 25 Sep 2024
Provable In-Context Learning of Linear Systems and Linear Elliptic PDEs with Transformers Frank Cole Yulong Lu Riley OÑeill Tianhao Zhang 32 2 0 18 Sep 2024
Mental Modeling of Reinforcement Learning Agents by Language Models Wenhao Lu Xufeng Zhao Josua Spisak Jae Hee Lee Stefan Wermter LLMAG LRM LM&Ro 22 2 0 26 Jun 2024
XLand-100B: A Large-Scale Multi-Task Dataset for In-Context Reinforcement Learning Alexander Nikulin Ilya Zisman Alexey Zemtsov Viacheslav Sinii 99 4 0 13 Jun 2024
Pretraining Decision Transformers with Reward Prediction for In-Context Multi-task Structured Bandit Learning Subhojyoti Mukherjee Josiah P. Hanna Qiaomin Xie Robert Nowak 56 2 0 07 Jun 2024
From Words to Actions: Unveiling the Theoretical Underpinnings of LLM-Driven Autonomous Systems Jianliang He Siyu Chen Fengzhuo Zhang Zhuoran Yang LM&Ro LLMAG 38 2 0 30 May 2024
Benchmarking General-Purpose In-Context Learning Fan Wang Chuan Lin Yang Cao Yu Kang 25 1 0 27 May 2024
Automatic Domain Adaptation by Transformers in In-Context Learning Ryuichiro Hataya Kota Matsui Masaaki Imaizumi 26 1 0 27 May 2024
Towards Better Understanding of In-Context Learning Ability from In-Context Uncertainty Quantification Shang Liu Zhongze Cai Guanting Chen Xiaocheng Li UQCV 38 1 0 24 May 2024
Understanding the Training and Generalization of Pretrained Transformer for Sequential Decision Making Hanzhao Wang Yu Pan Fupeng Sun Shang Liu K. Talluri Guanting Chen Xiaocheng Li OffRL 45 1 0 23 May 2024
Towards a Theoretical Understanding of the 'Reversal Curse' via Training Dynamics Hanlin Zhu Baihe Huang Shaolun Zhang Michael I. Jordan Jiantao Jiao Yuandong Tian Stuart Russell LRM AI4CE 41 13 0 07 May 2024
U-Nets as Belief Propagation: Efficient Classification, Denoising, and Diffusion in Generative Hierarchical Models Song Mei 3DV AI4CE DiffM 31 11 0 29 Apr 2024
Can large language models explore in-context? Akshay Krishnamurthy Keegan Harris Dylan J. Foster Cyril Zhang Aleksandrs Slivkins LM&Ro LLMAG LRM 118 20 0 22 Mar 2024
In-Context Learning of a Linear Transformer Block: Benefits of the MLP Component and One-Step GD Initialization Ruiqi Zhang Jingfeng Wu Peter L. Bartlett 28 12 0 22 Feb 2024
Towards an Information Theoretic Framework of Context-Based Offline Meta-Reinforcement Learning Lanqing Li Hai Zhang Xinyu Zhang Shatong Zhu Junqiao Zhao Junqiao Zhao Pheng-Ann Heng OffRL 26 7 0 04 Feb 2024
Transformers Learn Nonlinear Features In Context: Nonconvex Mean-field Dynamics on the Attention Landscape Juno Kim Taiji Suzuki 13 18 0 02 Feb 2024
In-Context Reinforcement Learning for Variable Action Spaces Viacheslav Sinii Alexander Nikulin Vladislav Kurenkov Ilya Zisman Sergey Kolesnikov 13 14 0 20 Dec 2023
Can a Transformer Represent a Kalman Filter? Gautam Goel Peter L. Bartlett 16 11 0 12 Dec 2023
Transformers Implement Functional Gradient Descent to Learn Non-Linear Functions In Context Xiang Cheng Yuxin Chen S. Sra 16 35 0 11 Dec 2023
Foundation Models for Decision Making: Problems, Methods, and Opportunities Sherry Yang Ofir Nachum Yilun Du Jason W. Wei Pieter Abbeel Dale Schuurmans LM&Ro OffRL LRM AI4CE 90 148 0 07 Mar 2023
Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks Chelsea Finn Pieter Abbeel Sergey Levine OOD 243 11,568 0 09 Mar 2017