LSTM Networks Can Perform Dynamic Counting

LSTM Networks Can Perform Dynamic Counting

9 June 2019

Sebastian Gehrmann

Yonatan Belinkov

Stuart M. Shieber

Papers citing "LSTM Networks Can Perform Dynamic Counting"

19 / 19 papers shown

Title
Training Neural Networks as Recognizers of Formal Languages Alexandra Butoi Ghazal Khalighinejad Anej Svete Josef Valvoda Ryan Cotterell Brian DuSell NAI 46 2 0 11 Nov 2024
Separations in the Representational Capabilities of Transformers and Recurrent Architectures S. Bhattamishra Michael Hahn Phil Blunsom Varun Kanade GNN 44 9 0 13 Jun 2024
What Languages are Easy to Language-Model? A Perspective from Learning Probabilistic Regular Languages Nadav Borenstein Anej Svete R. Chan Josef Valvoda Franz Nowak Isabelle Augenstein Eleanor Chodroff Ryan Cotterell 42 12 0 06 Jun 2024
Foundation Metrics for Evaluating Effectiveness of Healthcare Conversations Powered by Generative AI Mahyar Abbasian Elahe Khatibi Iman Azimi David Oniani Zahra Shakeri Hossein Abad ... Bryant Lin Olivier Gevaert Li-Jia Li Ramesh C. Jain Amir M. Rahmani LM&MA ELM AI4MH 47 66 0 21 Sep 2023
Theoretical Conditions and Empirical Failure of Bracket Counting on Long Sequences with Linear Recurrent Networks Nadine El-Naggar Pranava Madhyastha Tillman Weyde 22 1 0 07 Apr 2023
Exploring the Long-Term Generalization of Counting Behavior in RNNs Nadine El-Naggar Pranava Madhyastha Tillman Weyde 21 5 0 29 Nov 2022
Simplicity Bias in Transformers and their Ability to Learn Sparse Boolean Functions S. Bhattamishra Arkil Patel Varun Kanade Phil Blunsom 27 46 0 22 Nov 2022
Benchmarking Compositionality with Formal Languages Josef Valvoda Naomi Saphra Jonathan Rawski Adina Williams Ryan Cotterell NAI CoGe 38 9 0 17 Aug 2022
Exploring Length Generalization in Large Language Models Cem Anil Yuhuai Wu Anders Andreassen Aitor Lewkowycz Vedant Misra V. Ramasesh Ambrose Slone Guy Gur-Ari Ethan Dyer Behnam Neyshabur ReLM LRM 38 160 0 11 Jul 2022
Neural Networks and the Chomsky Hierarchy Grégoire Delétang Anian Ruoss Jordi Grau-Moya Tim Genewein L. Wenliang ... Chris Cundy Marcus Hutter Shane Legg Joel Veness Pedro A. Ortega UQCV 109 133 0 05 Jul 2022
The Limitations of Limited Context for Constituency Parsing Yuchen Li Andrej Risteski 26 5 0 03 Jun 2021
Representing Numbers in NLP: a Survey and a Vision Avijit Thawani Jay Pujara Pedro A. Szekely Filip Ilievski 34 114 0 24 Mar 2021
Formal Language Theory Meets Modern NLP William Merrill AI4CE NAI 26 12 0 19 Feb 2021
Can RNNs learn Recursive Nested Subject-Verb Agreements? Yair Lakretz T. Desbordes J. King Benoît Crabbé Maxime Oquab S. Dehaene 160 19 0 06 Jan 2021
ChemBERTa: Large-Scale Self-Supervised Pretraining for Molecular Property Prediction Seyone Chithrananda Gabriel Grand Bharath Ramsundar AI4CE 37 389 0 19 Oct 2020
How LSTM Encodes Syntax: Exploring Context Vectors and Semi-Quantization on Natural Text Chihiro Shibata Kei Uchiumi D. Mochihashi 22 7 0 01 Oct 2020
Multi-timescale Representation Learning in LSTM Language Models Shivangi Mahto Vy A. Vo Javier S. Turek Alexander G. Huth 15 29 0 27 Sep 2020
On the Computational Power of Transformers and its Implications in Sequence Modeling S. Bhattamishra Arkil Patel Navin Goyal 33 66 0 16 Jun 2020
Memory-Augmented Recurrent Neural Networks Can Learn Generalized Dyck Languages Mirac Suzgun Sebastian Gehrmann Yonatan Belinkov Stuart M. Shieber 32 50 0 08 Nov 2019