Systematic Generalization and Emergent Structures in Transformers Trained on Structured Tasks

2 October 2022

Papers citing "Systematic Generalization and Emergent Structures in Transformers Trained on Structured Tasks"

19 / 19 papers shown

Title
Synthetic Tabular Data Detection In the Wild G. C. N. Kindji Elisa Fromont L. Rojas-Barahona Tanguy Urvoy LMTD 58 1 0 03 Mar 2025
Cross-table Synthetic Tabular Data Detection G. C. N. Kindji L. Rojas-Barahona Elisa Fromont Tanguy Urvoy LMTD 75 0 0 17 Dec 2024
Reasoning in Large Language Models: A Geometric Perspective Romain Cosentino Sarath Shekkizhar LRM 42 2 0 02 Jul 2024
Explicitly Encoding Structural Symmetry is Key to Length Generalization in Arithmetic Tasks Mahdi Sabbaghi George Pappas Hamed Hassani Surbhi Goel 29 4 0 04 Jun 2024
Understanding Transformer Reasoning Capabilities via Graph Algorithms Clayton Sanford Bahare Fatemi Ethan Hall Anton Tsitsulin Seyed Mehran Kazemi Jonathan J. Halcrow Bryan Perozzi Vahab Mirrokni 27 29 0 28 May 2024
Enhancing Length Extrapolation in Sequential Models with Pointer-Augmented Neural Memory Hung Le D. Nguyen Kien Do Svetha Venkatesh T. Tran 19 0 0 18 Apr 2024
A Neural Rewriting System to Solve Algorithmic Problems Flavio Petruzzellis Alberto Testolin A. Sperduti NAI 34 0 0 27 Feb 2024
Benchmarking GPT-4 on Algorithmic Problems: A Systematic Evaluation of Prompting Strategies Flavio Petruzzellis Alberto Testolin A. Sperduti ELM 26 7 0 27 Feb 2024
Positional Description Matters for Transformers Arithmetic Ruoqi Shen Sébastien Bubeck Ronen Eldan Yin Tat Lee Yuanzhi Li Yi Zhang 16 37 0 22 Nov 2023
A Hybrid System for Systematic Generalization in Simple Arithmetic Problems Flavio Petruzzellis Alberto Testolin A. Sperduti AIMat LRM 21 1 0 29 Jun 2023
Schema-learning and rebinding as mechanisms of in-context learning and emergence Siva K. Swaminathan Antoine Dedieu Rajkumar Vasudeva Raju Murray Shanahan Miguel Lazaro-Gredilla Dileep George 24 8 0 16 Jun 2023
Randomized Positional Encodings Boost Length Generalization of Transformers Anian Ruoss Grégoire Delétang Tim Genewein Jordi Grau-Moya Róbert Csordás Mehdi Abbana Bennani Shane Legg J. Veness LLMAG 18 99 0 26 May 2023
Mind meets machine: Unravelling GPT-4's cognitive psychology Sifatkaur Dhingra Manmeet Singh Vaisakh S.B. Neetiraj Malviya S. Gill AI4MH 11 41 0 20 Mar 2023
Achieving and Understanding Out-of-Distribution Generalization in Systematic Reasoning in Small-Scale Transformers A. Nam Mustafa Abdool Trevor C. Maxfield James L. McClelland NAI LRM AI4CE 23 1 0 07 Oct 2022
Neural Networks and the Chomsky Hierarchy Grégoire Delétang Anian Ruoss Jordi Grau-Moya Tim Genewein L. Wenliang ... Chris Cundy Marcus Hutter Shane Legg Joel Veness Pedro A. Ortega UQCV 94 129 0 05 Jul 2022
Using cognitive psychology to understand GPT-3 Marcel Binz Eric Schulz ELM LLMAG 242 434 0 21 Jun 2022
The CLRS Algorithmic Reasoning Benchmark Petar Velivcković Adria Puigdomenech Badia David Budden Razvan Pascanu Andrea Banino Mikhail Dashevskiy R. Hadsell Charles Blundell 157 87 0 31 May 2022
Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation Ofir Press Noah A. Smith M. Lewis 242 695 0 27 Aug 2021
Zero-Shot Text-to-Image Generation Aditya A. Ramesh Mikhail Pavlov Gabriel Goh Scott Gray Chelsea Voss Alec Radford Mark Chen Ilya Sutskever VLM 253 4,764 0 24 Feb 2021