Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Task

24 October 2022

Papers citing "Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Task"

50 / 200 papers shown

Title
Transformer Based Planning in the Observation Space with Applications to Trick Taking Card Games Douglas Rebstock Christopher Solinas Nathan R Sturtevant M. Buro 24 0 0 19 Apr 2024
mOthello: When Do Cross-Lingual Representation Alignment and Cross-Lingual Transfer Emerge in Multilingual Models? Tianze Hua Tian Yun Ellie Pavlick LRM 29 9 0 18 Apr 2024
Test-Time Model Adaptation with Only Forward Passes Shuaicheng Niu Chunyan Miao Guohao Chen Pengcheng Wu Peilin Zhao TTA 53 19 0 02 Apr 2024
A Survey on Large Language Model-Based Game Agents Sihao Hu Tiansheng Huang Gaowen Liu Ramana Rao Kompella Gaowen Liu Selim Furkan Tekin Yichang Xu Zachary Yahn Ling Liu LLMAG LM&Ro AI4CE LM&MA 71 52 0 02 Apr 2024
MANGO: A Benchmark for Evaluating Mapping and Navigation Abilities of Large Language Models Peng Ding Jiading Fang Peng Li Kangrui Wang Xiaochen Zhou Mo Yu Jing Li Matthew R. Walter Hongyuan Mei RALM ELM 53 6 0 29 Mar 2024
Emergent World Models and Latent Variable Estimation in Chess-Playing Language Models Adam Karvonen 42 19 0 21 Mar 2024
Towards a theory of model distillation Enric Boix-Adserà FedML VLM 49 6 0 14 Mar 2024
Large Language Models and Games: A Survey and Roadmap Roberto Gallotta Graham Todd Marvin Zammit Sam Earle Antonios Liapis Julian Togelius Georgios N. Yannakakis LLMAG LM&MA AI4CE LRM 55 73 0 28 Feb 2024
Language Models Represent Beliefs of Self and Others Wentao Zhu Zhining Zhang Yizhou Wang MILM LRM 57 8 0 28 Feb 2024
TruthX: Alleviating Hallucinations by Editing Large Language Models in Truthful Space Shaolei Zhang Tian Yu Yang Feng HILM KELM 42 40 0 27 Feb 2024
What Do Language Models Hear? Probing for Auditory Representations in Language Models Jerry Ngo Yoon Kim AuLLM MILM 32 8 0 26 Feb 2024
ChatMusician: Understanding and Generating Music Intrinsically with LLM Ti-Fen Pan Hanfeng Lin Yi Wang Zeyue Tian Shangda Wu ... Gus Xia Roger Dannenberg Wei Xue Shiyin Kang Yike Guo 101 36 0 25 Feb 2024
Foot In The Door: Understanding Large Language Model Jailbreaking via Cognitive Psychology Zhenhua Wang Wei Xie Baosheng Wang Enze Wang Zhiwen Gui Shuoyoucheng Ma Kai Chen 36 14 0 24 Feb 2024
Fine-Tuning Enhances Existing Mechanisms: A Case Study on Entity Tracking Nikhil Prakash Tamar Rott Shaham Tal Haklay Yonatan Belinkov David Bau 51 57 0 22 Feb 2024
Q-Probe: A Lightweight Approach to Reward Maximization for Language Models Kenneth Li Samy Jelassi Hugh Zhang Sham Kakade Martin Wattenberg David Brandfonbrener 35 9 0 22 Feb 2024
On the Tip of the Tongue: Analyzing Conceptual Representation in Large Language Models with Reverse-Dictionary Probe Ningyu Xu Qi Zhang Menghan Zhang Peng Qian Xuanjing Huang LRM 77 3 0 22 Feb 2024
Understanding and Patching Compositional Reasoning in LLMs Zhaoyi Li Gangwei Jiang Hong Xie Linqi Song Defu Lian Ying Wei LRM 63 22 0 22 Feb 2024
WorldCoder, a Model-Based LLM Agent: Building World Models by Writing Code and Interacting with the Environment Hao Tang Darren Key Kevin Ellis LLMAG 25 28 0 19 Feb 2024
Dictionary Learning Improves Patch-Free Circuit Discovery in Mechanistic Interpretability: A Case Study on Othello-GPT Zhengfu He Xuyang Ge Qiong Tang Tianxiang Sun Qinyuan Cheng Xipeng Qiu 44 21 0 19 Feb 2024
Language Models Don't Learn the Physical Manifestation of Language Bruce W. Lee Jaehyuk Lim LRM 42 3 0 17 Feb 2024
Robust agents learn causal world models Jonathan G. Richens Tom Everitt OOD 124 37 0 16 Feb 2024
Towards Uncovering How Large Language Model Works: An Explainability Perspective Haiyan Zhao Fan Yang Bo Shen Himabindu Lakkaraju Mengnan Du 40 10 0 16 Feb 2024
Do Llamas Work in English? On the Latent Language of Multilingual Transformers Chris Wendler V. Veselovsky Giovanni Monea Robert West 58 103 0 16 Feb 2024
Learning Interpretable Concepts: Unifying Causal Representation Learning and Foundation Models Goutham Rajendran Simon Buchholz Bryon Aragam Bernhard Schölkopf Pradeep Ravikumar AI4CE 96 21 0 14 Feb 2024
Opening the AI black box: program synthesis via mechanistic interpretability Eric J. Michaud Isaac Liao Vedang Lad Ziming Liu Anish Mudide Chloe Loughridge Zifan Carl Guo Tara Rezaei Kheirkhah Mateja Vukelić Max Tegmark 28 12 0 07 Feb 2024
Position: Stop Making Unscientific AGI Performance Claims Patrick Altmeyer Andrew M. Demetriou Antony Bartlett Cynthia C. S. Liem 37 3 0 06 Feb 2024
Vision-Language Models Provide Promptable Representations for Reinforcement Learning William Chen Oier Mees Aviral Kumar Sergey Levine VLM LM&Ro 57 24 0 05 Feb 2024
Real Sparks of Artificial Intelligence and the Importance of Inner Interpretability Alex Grzankowski 32 4 0 31 Jan 2024
Learning Universal Predictors Jordi Grau-Moya Tim Genewein Marcus Hutter Laurent Orseau Grégoire Delétang ... Anian Ruoss Wenliang Kevin Li Christopher Mattern Matthew Aitchison J. Veness 41 12 0 26 Jan 2024
From Understanding to Utilization: A Survey on Explainability for Large Language Models Haoyan Luo Lucia Specia 56 22 0 23 Jan 2024
Learning Cognitive Maps from Transformer Representations for Efficient Planning in Partially Observed Environments Antoine Dedieu Wolfgang Lehrach Guangyao Zhou Dileep George Miguel Lazaro-Gredilla 45 2 0 11 Jan 2024
A Mechanistic Understanding of Alignment Algorithms: A Case Study on DPO and Toxicity Andrew Lee Xiaoyan Bai Itamar Pres Martin Wattenberg Jonathan K. Kummerfeld Rada Mihalcea 77 104 0 03 Jan 2024
A Comprehensive Study of Knowledge Editing for Large Language Models Ningyu Zhang Yunzhi Yao Bo Tian Peng Wang Shumin Deng ... Lei Liang Qing Cui Xiao-Jun Zhu Jun Zhou Huajun Chen KELM 55 77 0 02 Jan 2024
Emergence and Function of Abstract Representations in Self-Supervised Transformers Quentin RV. Ferry Joshua Ching Takashi Kawai 32 2 0 08 Dec 2023
Structured World Representations in Maze-Solving Transformers Michael Ivanitskiy Alex F Spies Tilman Rauker Guillaume Corlouer Chris Mathwin ... Rusheb Shah Dan Valentine Cecilia G. Diniz Behn Katsumi Inoue Samy Wu Fung 68 5 0 05 Dec 2023
Labeling Neural Representations with Inverse Recognition Kirill Bykov Laura Kopf Shinichi Nakajima Marius Kloft Marina M.-C. Höhne BDL 46 15 0 22 Nov 2023
Compositional Capabilities of Autoregressive Transformers: A Study on Synthetic, Interpretable Tasks Rahul Ramesh Ekdeep Singh Lubana Mikail Khona Robert P. Dick Hidenori Tanaka CoGe 39 8 0 21 Nov 2023
Predictive Minds: LLMs As Atypical Active Inference Agents Jan Kulveit Clem von Stengel Roman Leventov LLMAG KELM LRM 56 1 0 16 Nov 2023
Divergences between Language Models and Human Brains Yuchen Zhou Emmy Liu Graham Neubig Michael J. Tarr Leila Wehbe 40 1 0 15 Nov 2023
In-context Vectors: Making In Context Learning More Effective and Controllable Through Latent Space Steering Sheng Liu Haotian Ye Lei Xing James Y. Zou 26 87 0 11 Nov 2023
The Linear Representation Hypothesis and the Geometry of Large Language Models Kiho Park Yo Joong Choe Victor Veitch LLMSV MILM 45 146 0 07 Nov 2023
Uncovering Intermediate Variables in Transformers using Circuit Probing Michael A. Lepori Thomas Serre Ellie Pavlick 78 7 0 07 Nov 2023
Emergence of Abstract State Representations in Embodied Sequence Modeling Tian Yun Zilai Zeng Kunal Handa Ashish V. Thapliyal Bo Pang Ellie Pavlick Chen Sun LM&Ro 35 7 0 03 Nov 2023
Comparing Optimization Targets for Contrast-Consistent Search Hugo Fry S. Fallows Ian Fan Jamie Wright Nandi Schoots 19 2 0 01 Nov 2023
Large Language Models: The Need for Nuance in Current Debates and a Pragmatic Perspective on Understanding Bram van Dijk Tom Kouwenhoven M. Spruit Max J. van Duijn 37 19 0 30 Oct 2023
In-Context Learning Dynamics with Random Binary Sequences Eric J. Bigelow Ekdeep Singh Lubana Robert P. Dick Hidenori Tanaka T. Ullman 34 4 0 26 Oct 2023
How do Language Models Bind Entities in Context? Jiahai Feng Jacob Steinhardt 27 35 0 26 Oct 2023
Implicit meta-learning may lead language models to trust more reliable sources Dmitrii Krasheninnikov Egor Krasheninnikov Bruno Mlodozeniec Tegan Maharaj David M. Krueger 34 4 0 23 Oct 2023
When Language Models Fall in Love: Animacy Processing in Transformer Language Models Michael Hanna Yonatan Belinkov Sandro Pezzelle 30 11 0 23 Oct 2023
AI for Mathematics: A Cognitive Science Perspective Cedegao E. Zhang Katherine M. Collins Adrian Weller Joshua B. Tenenbaum 38 10 0 19 Oct 2023