The Neural Data Router: Adaptive Control Flow in Transformers Improves
Systematic Generalization

The Neural Data Router: Adaptive Control Flow in Transformers Improves Systematic Generalization

14 October 2021

Róbert Csordás

Jürgen Schmidhuber

Papers citing "The Neural Data Router: Adaptive Control Flow in Transformers Improves Systematic Generalization"

15 / 15 papers shown

Title
TRA: Better Length Generalisation with Threshold Relative Attention Mattia Opper Roland Fernandez P. Smolensky Jianfeng Gao 41 0 0 29 Mar 2025
Int2Int: a framework for mathematics with transformers François Charton ViT 41 0 0 22 Feb 2025
Investigating Recurrent Transformers with Dynamic Halt Jishnu Ray Chowdhury Cornelia Caragea 34 1 0 01 Feb 2024
Carrying over algorithm in transformers J. Kruthoff 24 0 0 15 Jan 2024
Positional Description Matters for Transformers Arithmetic Ruoqi Shen Sébastien Bubeck Ronen Eldan Yin Tat Lee Yuanzhi Li Yi Zhang 21 37 0 22 Nov 2023
Compositional Capabilities of Autoregressive Transformers: A Study on Synthetic, Interpretable Tasks Rahul Ramesh Ekdeep Singh Lubana Mikail Khona Robert P. Dick Hidenori Tanaka CoGe 31 6 0 21 Nov 2023
SALSA VERDE: a machine learning attack on Learning With Errors with sparse small secrets Cathy Li Emily Wenger Zeyuan Allen-Zhu François Charton Kristin E. Lauter AAML 25 10 0 20 Jun 2023
The Construction of Reality in an AI: A Review J. W. Johnston 3DV 11 1 0 03 Feb 2023
CTL++: Evaluating Generalization on Never-Seen Compositional Patterns of Known Functions, and Compatibility of Neural Representations Róbert Csordás Kazuki Irie Jürgen Schmidhuber NAI 19 11 0 12 Oct 2022
Systematic Generalization and Emergent Structures in Transformers Trained on Structured Tasks Yuxuan Li James L. McClelland 29 17 0 02 Oct 2022
Block-Recurrent Transformers DeLesley S. Hutchins Imanol Schlag Yuhuai Wu Ethan Dyer Behnam Neyshabur 16 94 0 11 Mar 2022
Linear algebra with transformers Franccois Charton AIMat 27 56 0 03 Dec 2021
PonderNet: Learning to Ponder Andrea Banino Jan Balaguer Charles Blundell PINN AIMat 94 80 0 12 Jul 2021
Going Beyond Linear Transformers with Recurrent Fast Weight Programmers Kazuki Irie Imanol Schlag Róbert Csordás Jürgen Schmidhuber 26 58 0 11 Jun 2021
On the Binding Problem in Artificial Neural Networks Klaus Greff Sjoerd van Steenkiste Jürgen Schmidhuber OCL 224 254 0 09 Dec 2020