Understanding Transformers via N-gram Statistics

Understanding Transformers via N-gram Statistics

30 June 2024

Timothy Nguyen

Papers citing "Understanding Transformers via N-gram Statistics"

9 / 9 papers shown

Title
Learning curves theory for hierarchically compositional data with power-law distributed features Francesco Cagnetta Hyunmo Kang M. Wyart 19 0 0 11 May 2025
Scaling Laws and Representation Learning in Simple Hierarchical Languages: Transformers vs. Convolutional Architectures Francesco Cagnetta Alessandro Favero Antonio Sclocchi M. Wyart 16 0 0 11 May 2025
Bigram Subnetworks: Mapping to Next Tokens in Transformer Language Models Tyler A. Chang Benjamin Bergen 38 0 0 21 Apr 2025
Benchmarking Systematic Relational Reasoning with Large Language and Reasoning Models Irtaza Khalid Amir Masoud Nourollah Steven Schockaert LRM 34 0 0 30 Mar 2025
How do language models learn facts? Dynamics, curricula and hallucinations Nicolas Zucchet J. Bornschein Stephanie C. Y. Chan Andrew Kyle Lampinen Razvan Pascanu Soham De KELM HILM LRM 69 1 1 27 Mar 2025
Cognitive Activation and Chaotic Dynamics in Large Language Models: A Quasi-Lyapunov Analysis of Reasoning Mechanisms Xiaojian Li Yongkang Leng Ruiqing Ding Hangjie Mo Shanlin Yang LRM 44 0 0 15 Mar 2025
Jet Expansions of Residual Computation Yihong Chen Xiangxiang Xu Yao Lu Pontus Stenetorp Luca Franceschi 19 2 0 08 Oct 2024
AI as Humanity's Salieri: Quantifying Linguistic Creativity of Language Models via Systematic Attribution of Machine Text against Web Text Ximing Lu Melanie Sclar Skyler Hallinan Niloofar Mireshghallah Jiacheng Liu ... Allyson Ettinger Liwei Jiang Khyathi Raghavi Chandu Nouha Dziri Yejin Choi DeLMO 42 11 0 05 Oct 2024
Zoology: Measuring and Improving Recall in Efficient Language Models Simran Arora Sabri Eyuboglu Aman Timalsina Isys Johnson Michael Poli James Zou Atri Rudra Christopher Ré 56 65 0 08 Dec 2023