A Measure-Theoretic Characterization of Tight Language Models

20 December 2022

Li Du

Lucas Torroba Hennigen

Papers citing "A Measure-Theoretic Characterization of Tight Language Models"

27 / 27 papers shown

Title
What do Language Model Probabilities Represent? From Distribution Estimation to Response Prediction Eitan Wagner Omri Abend 39 0 0 04 May 2025
Training Neural Networks as Recognizers of Formal Languages Alexandra Butoi Ghazal Khalighinejad Anej Svete Josef Valvoda Ryan Cotterell Brian DuSell NAI 41 2 0 11 Nov 2024
Gumbel Counterfactual Generation From Language Models Shauli Ravfogel Anej Svete Vésteinn Snæbjarnarson Ryan Cotterell LRM CML 33 0 0 11 Nov 2024
CONTESTS: a Framework for Consistency Testing of Span Probabilities in Language Models Eitan Wagner Yuli Slavutsky Omri Abend 18 1 0 30 Sep 2024
On the Role of Context in Reading Time Prediction Andreas Opedal Eleanor Chodroff Ryan Cotterell Ethan Gotlieb Wilcox 33 7 0 12 Sep 2024
How to Compute the Probability of a Word Tiago Pimentel Clara Meister 24 14 0 20 Jun 2024
On the Representational Capacity of Neural Language Models with Chain-of-Thought Reasoning Franz Nowak Anej Svete Alexandra Butoi Ryan Cotterell ReLM LRM 51 12 0 20 Jun 2024
Analyzing constrained LLM through PDFA-learning Matías Carrasco Franz Mayr S. Yovine Johny Kidd Martín Iturbide Juan da Silva Alejo Garat 26 0 0 12 Jun 2024
On Affine Homotopy between Language Encoders Robin SM Chan Reda Boumasmoud Anej Svete Yuxin Ren Qipeng Guo ... Shauli Ravfogel Mrinmaya Sachan Bernhard Schölkopf Mennatallah El-Assady Ryan Cotterell 43 3 0 04 Jun 2024
Lower Bounds on the Expressivity of Recurrent Neural Language Models Anej Svete Franz Nowak Anisha Mohamed Sahabdeen Ryan Cotterell 34 0 0 29 May 2024
Transformers Can Represent $n$ -gram Language Models Anej Svete Ryan Cotterell 32 17 0 23 Apr 2024
The pitfalls of next-token prediction Gregor Bachmann Vaishnavh Nagarajan 37 62 0 11 Mar 2024
On Efficiently Representing Regular Languages as RNNs Anej Svete R. Chan Ryan Cotterell 36 1 0 24 Feb 2024
Principled Gradient-based Markov Chain Monte Carlo for Text Generation Li Du Afra Amini Lucas Torroba Hennigen Xinyan Velocity Yu Jason Eisner Holden Lee Ryan Cotterell BDL 23 1 0 29 Dec 2023
Structure-Aware Path Inference for Neural Finite State Transducers Weiting Tan Chu-cheng Lin Jason Eisner 18 0 0 21 Dec 2023
On the Representational Capacity of Recurrent Neural Language Models Franz Nowak Anej Svete Li Du Ryan Cotterell 13 1 0 19 Oct 2023
Recurrent Neural Language Models as Probabilistic Finite-state Automata Anej Svete Ryan Cotterell 32 2 0 08 Oct 2023
Language Models as a Service: Overview of a New Paradigm and its Challenges Emanuele La Malfa Aleksandar Petrov Simon Frieder Christoph Weinhuber Ryan Burnell Raza Nazar Anthony Cohn Nigel Shadbolt Michael Wooldridge ALM ELM 35 3 0 28 Sep 2023
A Geometric Notion of Causal Probing Clément Guerner Anej Svete Tianyu Liu Alex Warstadt Ryan Cotterell LLMSV 38 12 0 27 Jul 2023
On the Efficacy of Sampling Adapters Clara Meister Tiago Pimentel Luca Malagutti Ethan Gotlieb Wilcox Ryan Cotterell 35 12 0 07 Jul 2023
A Cross-Linguistic Pressure for Uniform Information Density in Word Order T. H. Clark Clara Meister Tiago Pimentel Michael Hahn Ryan Cotterell Richard Futrell Saarland University 27 13 0 06 Jun 2023
Structured Voronoi Sampling Afra Amini Li Du Ryan Cotterell DiffM 27 1 0 05 Jun 2023
MixCE: Training Autoregressive Language Models by Mixing Forward and Reverse Cross-Entropies Shiyue Zhang Shijie Wu Ozan Irsoy Steven Lu Joey Tianyi Zhou Mark Dredze David S. Rosenberg 23 9 0 26 May 2023
Deriving Language Models from Masked Language Models Lucas Torroba Hennigen Yoon Kim 27 11 0 24 May 2023
Locally Typical Sampling Clara Meister Tiago Pimentel Gian Wiher Ryan Cotterell 143 86 0 01 Feb 2022
A Recipe For Arbitrary Text Style Transfer with Large Language Models Emily Reif Daphne Ippolito Ann Yuan Andy Coenen Chris Callison-Burch Jason W. Wei 224 117 0 08 Sep 2021
Language Models as Knowledge Bases? Fabio Petroni Tim Rocktaschel Patrick Lewis A. Bakhtin Yuxiang Wu Alexander H. Miller Sebastian Riedel KELM AI4MH 415 2,588 0 03 Sep 2019