On the Computational Power of Transformers and its Implications in Sequence Modeling

16 June 2020

Papers citing "On the Computational Power of Transformers and its Implications in Sequence Modeling"

15 / 15 papers shown

Title
Can Large Language Models Learn Formal Logic? A Data-Driven Training and Evaluation Framework Yuan Xia Akanksha Atrey Fadoua Khmaissia Kedar S. Namjoshi LRM ELM 45 0 0 28 Apr 2025
Lower Bounds for Chain-of-Thought Reasoning in Hard-Attention Transformers Alireza Amiri Xinting Huang Mark Rofin Michael Hahn LRM 143 0 0 04 Feb 2025
Training Neural Networks as Recognizers of Formal Languages Alexandra Butoi Ghazal Khalighinejad Anej Svete Josef Valvoda Ryan Cotterell Brian DuSell NAI 36 2 0 11 Nov 2024
Representing Rule-based Chatbots with Transformers Dan Friedman Abhishek Panigrahi Danqi Chen 59 1 0 15 Jul 2024
On the Representational Capacity of Neural Language Models with Chain-of-Thought Reasoning Franz Nowak Anej Svete Alexandra Butoi Ryan Cotterell ReLM LRM 44 12 0 20 Jun 2024
Separations in the Representational Capabilities of Transformers and Recurrent Architectures S. Bhattamishra Michael Hahn Phil Blunsom Varun Kanade GNN 33 9 0 13 Jun 2024
Limits of Deep Learning: Sequence Modeling through the Lens of Complexity Theory Nikola Zubić Federico Soldá Aurelio Sulser Davide Scaramuzza LRM BDL 50 5 0 26 May 2024
Transformers as Decision Makers: Provable In-Context Reinforcement Learning via Supervised Pretraining Licong Lin Yu Bai Song Mei OffRL 30 42 0 12 Oct 2023
A Theoretical Understanding of Shallow Vision Transformers: Learning, Generalization, and Sample Complexity Hongkang Li M. Wang Sijia Liu Pin-Yu Chen ViT MLT 35 56 0 12 Feb 2023
What Can Transformers Learn In-Context? A Case Study of Simple Function Classes Shivam Garg Dimitris Tsipras Percy Liang Gregory Valiant 21 447 0 01 Aug 2022
Pure Transformers are Powerful Graph Learners Jinwoo Kim Tien Dat Nguyen Seonwoo Min Sungjun Cho Moontae Lee Honglak Lee Seunghoon Hong 32 187 0 06 Jul 2022
Overcoming a Theoretical Limitation of Self-Attention David Chiang Peter A. Cholak 25 76 0 24 Feb 2022
Inductive Biases and Variable Creation in Self-Attention Mechanisms Benjamin L. Edelman Surbhi Goel Sham Kakade Cyril Zhang 22 115 0 19 Oct 2021
Representing Numbers in NLP: a Survey and a Vision Avijit Thawani Jay Pujara Pedro A. Szekely Filip Ilievski 19 114 0 24 Mar 2021
OpenNMT: Open-Source Toolkit for Neural Machine Translation Guillaume Klein Yoon Kim Yuntian Deng Jean Senellart Alexander M. Rush 254 1,896 0 10 Jan 2017