ResearchTrend.AI
  • Communities
  • Connect sessions
  • AI calendar
  • Organizations
  • Join Slack
  • Contact Sales
Papers
Communities
Social Events
Terms and Conditions
Pricing
Contact Sales
Parameter LabParameter LabTwitterGitHubLinkedInBlueskyYoutube

© 2026 ResearchTrend.AI, All rights reserved.

  1. Home
  2. Papers
  3. 2305.18654
  4. Cited By
Faith and Fate: Limits of Transformers on Compositionality
v1v2v3 (latest)

Faith and Fate: Limits of Transformers on Compositionality

Neural Information Processing Systems (NeurIPS), 2023
29 May 2023
Nouha Dziri
Ximing Lu
Melanie Sclar
Xiang Lorraine Li
Liwei Jian
Bill Yuchen Lin
Peter West
Chandra Bhagavatula
Ronan Le Bras
Jena D. Hwang
Soumya Sanyal
Sean Welleck
Xiang Ren
Allyson Ettinger
Zaïd Harchaoui
Yejin Choi
    ReLMLRM
ArXiv (abs)PDFHTMLHuggingFace (7 upvotes)

Papers citing "Faith and Fate: Limits of Transformers on Compositionality"

50 / 327 papers shown
Title
On Limitation of Transformer for Learning HMMs
On Limitation of Transformer for Learning HMMs
Jiachen Hu
Qinghua Liu
Chi Jin
206
7
0
06 Jun 2024
Exact Conversion of In-Context Learning to Model Weights in
  Linearized-Attention Transformers
Exact Conversion of In-Context Learning to Model Weights in Linearized-Attention Transformers
Brian K Chen
Tianyang Hu
Hui Jin
Hwee Kuan Lee
Kenji Kawaguchi
183
5
0
05 Jun 2024
Language Models Do Hard Arithmetic Tasks Easily and Hardly Do Easy
  Arithmetic Tasks
Language Models Do Hard Arithmetic Tasks Easily and Hardly Do Easy Arithmetic Tasks
Andrew Gambardella
Yusuke Iwasawa
Yutaka Matsuo
LRM
129
16
0
04 Jun 2024
Explicitly Encoding Structural Symmetry is Key to Length Generalization
  in Arithmetic Tasks
Explicitly Encoding Structural Symmetry is Key to Length Generalization in Arithmetic Tasks
Mahdi Sabbaghi
George Pappas
Hamed Hassani
Surbhi Goel
237
8
0
04 Jun 2024
ACCORD: Closing the Commonsense Measurability Gap
ACCORD: Closing the Commonsense Measurability Gap
François Roewer-Després
Jinyue Feng
Zining Zhu
Frank Rudzicz
LRM
328
0
0
04 Jun 2024
Alice in Wonderland: Simple Tasks Showing Complete Reasoning Breakdown in State-Of-the-Art Large Language Models
Alice in Wonderland: Simple Tasks Showing Complete Reasoning Breakdown in State-Of-the-Art Large Language Models
Marianna Nezhurina
Lucia Cipolina-Kun
Mehdi Cherti
J. Jitsev
LLMAGLRMELMReLM
723
58
0
04 Jun 2024
Contextual Counting: A Mechanistic Study of Transformers on a
  Quantitative Task
Contextual Counting: A Mechanistic Study of Transformers on a Quantitative Task
Siavash Golkar
Alberto Bietti
Mariel Pettee
Michael Eickenberg
M. Cranmer
...
Ruben Ohana
Liam Parker
Bruno Régaldo-Saint Blancard
Kyunghyun Cho
Shirley Ho
161
5
0
30 May 2024
Language Models Need Inductive Biases to Count Inductively
Language Models Need Inductive Biases to Count Inductively
Yingshan Chang
Yonatan Bisk
LRM
248
14
0
30 May 2024
Transformers Can Do Arithmetic with the Right Embeddings
Transformers Can Do Arithmetic with the Right Embeddings
Sean McLeish
Arpit Bansal
Alex Stein
Neel Jain
John Kirchenbauer
...
B. Kailkhura
A. Bhatele
Jonas Geiping
Avi Schwarzschild
Tom Goldstein
164
63
0
27 May 2024
THREAD: Thinking Deeper with Recursive Spawning
THREAD: Thinking Deeper with Recursive Spawning
Philip Schroeder
Nathaniel Morgan
Hongyin Luo
James R. Glass
LRMLLMAGReLM
284
7
0
27 May 2024
Limits of Deep Learning: Sequence Modeling through the Lens of Complexity Theory
Limits of Deep Learning: Sequence Modeling through the Lens of Complexity Theory
Nikola Zubić
Federico Soldá
Aurelio Sulser
Davide Scaramuzza
LRMBDL
347
15
0
26 May 2024
From Frege to chatGPT: Compositionality in language, cognition, and deep neural networks
From Frege to chatGPT: Compositionality in language, cognition, and deep neural networks
Jacob Russin
Sam Whitman McGrath
Danielle J. Williams
AI4CE
489
6
0
24 May 2024
Grokked Transformers are Implicit Reasoners: A Mechanistic Journey to
  the Edge of Generalization
Grokked Transformers are Implicit Reasoners: A Mechanistic Journey to the Edge of Generalization
Boshi Wang
Xiang Yue
Yu-Chuan Su
Huan Sun
LRM
336
72
0
23 May 2024
From Explicit CoT to Implicit CoT: Learning to Internalize CoT Step by
  Step
From Explicit CoT to Implicit CoT: Learning to Internalize CoT Step by Step
Yuntian Deng
Yejin Choi
Stuart M. Shieber
ReLMLRM
231
119
0
23 May 2024
Investigating Symbolic Capabilities of Large Language Models
Investigating Symbolic Capabilities of Large Language Models
Neisarg Dave
Daniel Kifer
C. Lee Giles
A. Mali
ELMLRM
132
4
0
21 May 2024
A General Theory for Compositional Generalization
A General Theory for Compositional Generalization
Jingwen Fu
Zhizheng Zhang
Yan Lu
Nanning Zheng
AI4CECoGe
206
2
0
20 May 2024
LLM-Generated Black-box Explanations Can Be Adversarially Helpful
LLM-Generated Black-box Explanations Can Be Adversarially Helpful
R. Ajwani
Shashidhar Reddy Javaji
Frank Rudzicz
Zining Zhu
AAML
267
23
0
10 May 2024
Initialization is Critical to Whether Transformers Fit Composite Functions by Reasoning or Memorizing
Initialization is Critical to Whether Transformers Fit Composite Functions by Reasoning or MemorizingNeural Information Processing Systems (NeurIPS), 2024
Zhongwang Zhang
Pengxiao Lin
Zhiwei Wang
Yaoyu Zhang
Z. Xu
552
3
0
08 May 2024
Chain of Thoughtlessness? An Analysis of CoT in Planning
Chain of Thoughtlessness? An Analysis of CoT in Planning
Kaya Stechly
Kaya Stechly
Subbarao Kambhampati
LRMLM&Ro
542
94
0
08 May 2024
Exploring the Compositional Deficiency of Large Language Models in
  Mathematical Reasoning
Exploring the Compositional Deficiency of Large Language Models in Mathematical ReasoningConference on Empirical Methods in Natural Language Processing (EMNLP), 2024
Jun Zhao
Jingqi Tong
Yurong Mou
Ming-bo Wen
Tao Gui
Xuanjing Huang
LRM
231
12
0
05 May 2024
What makes Models Compositional? A Theoretical View: With Supplement
What makes Models Compositional? A Theoretical View: With SupplementInternational Joint Conference on Artificial Intelligence (IJCAI), 2024
Parikshit Ram
Tim Klinger
Alexander G. Gray
CoGe
237
8
0
02 May 2024
Transcrib3D: 3D Referring Expression Resolution through Large Language
  Models
Transcrib3D: 3D Referring Expression Resolution through Large Language Models
Jiading Fang
Xiangshan Tan
Shengjie Lin
Igor Vasiljevic
Vitor Campagnolo Guizilini
Hongyuan Mei
Rares Andrei Ambrus
Gregory Shakhnarovich
Matthew R. Walter
LM&Ro
171
7
0
30 Apr 2024
OCK: Unsupervised Dynamic Video Prediction with Object-Centric Kinematics
OCK: Unsupervised Dynamic Video Prediction with Object-Centric Kinematics
Yeon-Ji Song
Jaein Kim
Suhyung Choi
Jin-Hwa Kim
Byoung-Tak Zhang
259
0
0
29 Apr 2024
Language Models Still Struggle to Zero-shot Reason about Time Series
Language Models Still Struggle to Zero-shot Reason about Time Series
Mike A. Merrill
Mingtian Tan
Vinayak Gupta
Tom Hartvigsen
Tim Althoff
AI4TSLRM
240
66
0
17 Apr 2024
On the Empirical Complexity of Reasoning and Planning in LLMs
On the Empirical Complexity of Reasoning and Planning in LLMs
Liwei Kang
Zirui Zhao
David Hsu
Wee Sun Lee
LRM
190
8
0
17 Apr 2024
Elephants Never Forget: Memorization and Learning of Tabular Data in
  Large Language Models
Elephants Never Forget: Memorization and Learning of Tabular Data in Large Language Models
Sebastian Bordt
Harsha Nori
Vanessa Rodrigues
Besmira Nushi
Rich Caruana
284
25
0
09 Apr 2024
Characterizing Multimodal Long-form Summarization: A Case Study on
  Financial Reports
Characterizing Multimodal Long-form Summarization: A Case Study on Financial Reports
Tianyu Cao
Natraj Raman
Danial Dervovic
Chenhao Tan
139
7
0
09 Apr 2024
Counting Like Transformers: Compiling Temporal Counting Logic Into
  Softmax Transformers
Counting Like Transformers: Compiling Temporal Counting Logic Into Softmax Transformers
Andy Yang
David Chiang
208
26
0
05 Apr 2024
Iterated Learning Improves Compositionality in Large Vision-Language
  Models
Iterated Learning Improves Compositionality in Large Vision-Language ModelsComputer Vision and Pattern Recognition (CVPR), 2024
Chenhao Zheng
Jieyu Zhang
Aniruddha Kembhavi
Ranjay Krishna
VLMCoGe
247
15
0
02 Apr 2024
Beyond Accuracy: Evaluating the Reasoning Behavior of Large Language
  Models -- A Survey
Beyond Accuracy: Evaluating the Reasoning Behavior of Large Language Models -- A Survey
Philipp Mondorf
Barbara Plank
ELMLRMLM&MA
340
85
0
02 Apr 2024
A Theory for Length Generalization in Learning to Reason
A Theory for Length Generalization in Learning to Reason
Changnan Xiao
Bing Liu
LRM
304
12
0
31 Mar 2024
Reasoning Abilities of Large Language Models: In-Depth Analysis on the
  Abstraction and Reasoning Corpus
Reasoning Abilities of Large Language Models: In-Depth Analysis on the Abstraction and Reasoning CorpusACM Transactions on Intelligent Systems and Technology (ACM TIST), 2024
Seungpil Lee
Woochang Sim
Donghyeon Shin
Sanha Hwang
Wongyu Seo
Jiwon Park
Seokki Lee
Sejin Kim
Sundong Kim
LRM
169
44
0
18 Mar 2024
Elephants Never Forget: Testing Language Models for Memorization of
  Tabular Data
Elephants Never Forget: Testing Language Models for Memorization of Tabular Data
Sebastian Bordt
Harsha Nori
Rich Caruana
LMTD
204
21
0
11 Mar 2024
The pitfalls of next-token prediction
The pitfalls of next-token predictionInternational Conference on Machine Learning (ICML), 2024
Gregor Bachmann
Vaishnavh Nagarajan
417
131
0
11 Mar 2024
Will GPT-4 Run DOOM?
Will GPT-4 Run DOOM?IEEE Transactions on Games (IEEE Trans. Games), 2024
Adrian de Wynter
LM&RoMLLM
172
6
0
08 Mar 2024
Exploring Continual Learning of Compositional Generalization in NLI
Exploring Continual Learning of Compositional Generalization in NLI
Xiyan Fu
Anette Frank
CLLLRM
226
5
0
07 Mar 2024
Learning to Use Tools via Cooperative and Interactive Agents
Learning to Use Tools via Cooperative and Interactive Agents
Zhengliang Shi
Shen Gao
Xiuyi Chen
Zhumin Chen
Lingyong Yan
Haibo Shi
D. Yin
Sudipta Singha Roy
Suzan Verberne
Zhaochun Ren
LLMAG
319
50
0
05 Mar 2024
Masked Thought: Simply Masking Partial Reasoning Steps Can Improve
  Mathematical Reasoning Learning of Language Models
Masked Thought: Simply Masking Partial Reasoning Steps Can Improve Mathematical Reasoning Learning of Language Models
Changyu Chen
Xiting Wang
Ting-En Lin
Ang Lv
Yuchuan Wu
Xin Gao
Ji-Rong Wen
Rui Yan
Yongbin Li
ReLMLRM
208
20
0
04 Mar 2024
Formulation Comparison for Timeline Construction using LLMs
Formulation Comparison for Timeline Construction using LLMs
Kimihiro Hasegawa
Nikhil Kandukuri
Susan Holm
Yukari Yamakawa
Teruko Mitamura
272
1
0
01 Mar 2024
A Neural Rewriting System to Solve Algorithmic Problems
A Neural Rewriting System to Solve Algorithmic Problems
Flavio Petruzzellis
Alberto Testolin
A. Sperduti
NAI
221
2
0
27 Feb 2024
Successfully Guiding Humans with Imperfect Instructions by Highlighting
  Potential Errors and Suggesting Corrections
Successfully Guiding Humans with Imperfect Instructions by Highlighting Potential Errors and Suggesting Corrections
Lingjun Zhao
Khanh Nguyen
Hal Daumé
203
4
0
26 Feb 2024
Do Large Language Models Latently Perform Multi-Hop Reasoning?
Do Large Language Models Latently Perform Multi-Hop Reasoning?
Sohee Yang
E. Gribovskaya
Nora Kassner
Mor Geva
Sebastian Riedel
ReLMLRM
347
154
0
26 Feb 2024
Data Science with LLMs and Interpretable Models
Data Science with LLMs and Interpretable Models
Sebastian Bordt
Benjamin J. Lengerich
Harsha Nori
Rich Caruana
SyDaAI4CE
188
9
0
22 Feb 2024
Understanding and Patching Compositional Reasoning in LLMs
Understanding and Patching Compositional Reasoning in LLMs
Zhaoyi Li
Gangwei Jiang
Hong Xie
Linqi Song
Defu Lian
Ying Wei
LRM
224
41
0
22 Feb 2024
Can LLMs Reason with Rules? Logic Scaffolding for Stress-Testing and
  Improving LLMs
Can LLMs Reason with Rules? Logic Scaffolding for Stress-Testing and Improving LLMs
Siyuan Wang
Zhongyu Wei
Yejin Choi
Xiang Ren
ReLMELMLRM
166
34
0
18 Feb 2024
Transformers Can Achieve Length Generalization But Not Robustly
Transformers Can Achieve Length Generalization But Not Robustly
Yongchao Zhou
Uri Alon
Xinyun Chen
Xuezhi Wang
Rishabh Agarwal
Denny Zhou
260
65
0
14 Feb 2024
Using Counterfactual Tasks to Evaluate the Generality of Analogical
  Reasoning in Large Language Models
Using Counterfactual Tasks to Evaluate the Generality of Analogical Reasoning in Large Language Models
Martha Lewis
Melanie Mitchell
ELMLRM
208
34
0
14 Feb 2024
On Limitations of the Transformer Architecture
On Limitations of the Transformer Architecture
Binghui Peng
Srini Narayanan
Christos H. Papadimitriou
226
63
0
13 Feb 2024
On the Self-Verification Limitations of Large Language Models on
  Reasoning and Planning Tasks
On the Self-Verification Limitations of Large Language Models on Reasoning and Planning Tasks
Kaya Stechly
Kaya Stechly
Subbarao Kambhampati
ReLMLRM
163
97
0
12 Feb 2024
Towards an Understanding of Stepwise Inference in Transformers: A
  Synthetic Graph Navigation Model
Towards an Understanding of Stepwise Inference in Transformers: A Synthetic Graph Navigation Model
Mikail Khona
Maya Okawa
Jan Hula
Rahul Ramesh
Kento Nishi
Robert P. Dick
Ekdeep Singh Lubana
Hidenori Tanaka
187
9
0
12 Feb 2024
Previous
1234567
Next