Going Beyond Linear Transformers with Recurrent Fast Weight Programmers

Going Beyond Linear Transformers with Recurrent Fast Weight Programmers

11 June 2021

Róbert Csordás

Jürgen Schmidhuber

Papers citing "Going Beyond Linear Transformers with Recurrent Fast Weight Programmers"

14 / 14 papers shown

Title
Unlocking State-Tracking in Linear RNNs Through Negative Eigenvalues Riccardo Grazzi Julien N. Siems Jörg K.H. Franke Arber Zela Frank Hutter Massimiliano Pontil 84 11 0 19 Nov 2024
Learning to (Learn at Test Time): RNNs with Expressive Hidden States Yu Sun Xinhao Li Karan Dalal Jiarui Xu Arjun Vikram ... Xinlei Chen Xiaolong Wang Sanmi Koyejo Tatsunori Hashimoto Carlos Guestrin 56 92 0 05 Jul 2024
Efficient Attention via Control Variates Lin Zheng Jianbo Yuan Chong-Jun Wang Lingpeng Kong 24 18 0 09 Feb 2023
Learning One Abstract Bit at a Time Through Self-Invented Experiments Encoded as Neural Networks Vincent Herrmann Louis Kirsch Jürgen Schmidhuber AI4CE 38 4 0 29 Dec 2022
General-Purpose In-Context Learning by Meta-Learning Transformers Louis Kirsch James Harrison Jascha Narain Sohl-Dickstein Luke Metz 29 72 0 08 Dec 2022
Meta-Learning Fast Weight Language Models Kevin Clark Kelvin Guu Ming-Wei Chang Panupong Pasupat Geoffrey E. Hinton Mohammad Norouzi KELM 19 13 0 05 Dec 2022
Images as Weight Matrices: Sequential Image Generation Through Synaptic Learning Rules Kazuki Irie Jürgen Schmidhuber 24 5 0 07 Oct 2022
Learning to Generalize with Object-centric Agents in the Open World Survival Game Crafter Aleksandar Stanić Yujin Tang David R Ha Jürgen Schmidhuber ELM 19 11 0 05 Aug 2022
Object Representations as Fixed Points: Training Iterative Refinement Algorithms with Implicit Differentiation Michael Chang Thomas L. Griffiths Sergey Levine OCL 47 59 0 02 Jul 2022
Neural Differential Equations for Learning to Program Neural Nets Through Continuous Learning Rules Kazuki Irie Francesco Faccio Jürgen Schmidhuber AI4TS 25 11 0 03 Jun 2022
Unsupervised Learning of Temporal Abstractions with Slot-based Transformers Anand Gopalakrishnan Kazuki Irie Jürgen Schmidhuber Sjoerd van Steenkiste OffRL 19 16 0 25 Mar 2022
Block-Recurrent Transformers DeLesley S. Hutchins Imanol Schlag Yuhuai Wu Ethan Dyer Behnam Neyshabur 16 94 0 11 Mar 2022
TransDreamer: Reinforcement Learning with Transformer World Models Changgu Chen Yi-Fu Wu Jaesik Yoon Sungjin Ahn OffRL 27 90 0 19 Feb 2022
LambdaNetworks: Modeling Long-Range Interactions Without Attention Irwan Bello 260 179 0 17 Feb 2021