Tokenization counts: the impact of tokenization on arithmetic in frontier LLMs

22 February 2024

Papers citing "Tokenization counts: the impact of tokenization on arithmetic in frontier LLMs"

35 / 35 papers shown

Title
Geospatial Mechanistic Interpretability of Large Language Models Stef De Sabbata Stefano Mizzaro Kevin Roitero AI4CE 24 0 0 06 May 2025
SuperBPE: Space Travel for Language Models Alisa Liu J. Hayase Valentin Hofmann Sewoong Oh Noah A. Smith Yejin Choi 43 1 0 17 Mar 2025
Bringing Comparative Cognition To Computers Konstantinos Voudouris Lucy G. Cheke Eric Schulz ELM 73 0 0 04 Mar 2025
Adversarial Tokenization Renato Lui Geh Zilei Shao Guy Van den Broeck SILM AAML 85 0 0 04 Mar 2025
The Lookahead Limitation: Why Multi-Operand Addition is Hard for LLMs Tanja Baeumel Josef van Genabith Simon Ostermann LRM 56 1 0 27 Feb 2025
Scaling LLM Pre-training with Vocabulary Curriculum Fangyuan Yu 62 1 0 25 Feb 2025
AtmosSci-Bench: Evaluating the Recent Advance of Large Language Model for Atmospheric Science Chenyue Li Wen Deng Mengqian Lu Binhang Yuan ELM AI4Cl LRM 87 0 0 03 Feb 2025
DateLogicQA: Benchmarking Temporal Biases in Large Language Models Gagan Bhatia MingZe Tang Cristina Mahanta Madiha Kazi 71 0 0 17 Dec 2024
Number Cookbook: Number Understanding of Language Models and How to Improve It Haotong Yang Yi Hu Shijia Kang Zhouchen Lin Muhan Zhang LRM 41 2 0 06 Nov 2024
MrT5: Dynamic Token Merging for Efficient Byte-level Language Models Julie Kallini Shikhar Murty Christopher D. Manning Christopher Potts Róbert Csordás 27 2 0 28 Oct 2024
Supervised Chain of Thought Xiang Zhang Dujian Ding LRM AI4CE 20 1 0 18 Oct 2024
Language Models Encode Numbers Using Digit Representations in Base 10 Amit Arnold Levy Mor Geva 19 4 0 15 Oct 2024
Grounding Partially-Defined Events in Multimodal Data Kate Sanders Reno Kriz David Etter Hannah Recknor Alexander Martin Cameron Carpenter Jingyang Lin Benjamin Van Durme 22 1 0 07 Oct 2024
Gradient Routing: Masking Gradients to Localize Computation in Neural Networks Alex Cloud Jacob Goldman-Wetzler Evžen Wybitul Joseph Miller Alexander Matt Turner 14 3 0 06 Oct 2024
Large Language Models as Markov Chains Oussama Zekri Ambroise Odonnat Abdelhakim Benechehab Linus Bleistein Nicolas Boullé I. Redko 34 9 0 03 Oct 2024
BPE Gets Picky: Efficient Vocabulary Refinement During Tokenizer Training Pavel Chizhov Catherine Arnett Elizaveta Korotkova Ivan P. Yamshchikov 37 2 0 06 Sep 2024
Where is the signal in tokenization space? Renato Lui Geh Honghua Zhang Kareem Ahmed Benjie Wang Guy Van den Broeck 20 4 0 16 Aug 2024
Can LLMs predict the convergence of Stochastic Gradient Descent? Hiroki Sakaji Abdelhakim Benechehab Wataru Kuramoto LRM 41 2 0 03 Aug 2024
Perceptions of Linguistic Uncertainty by Language Models and Humans Catarina G Belém Markelle Kelly M. Steyvers Sameer Singh Padhraic Smyth 33 3 0 22 Jul 2024
Improving Self Consistency in LLMs through Probabilistic Tokenization Ashutosh Sathe Divyanshu Aggarwal Sunayana Sitaram 30 4 0 04 Jul 2024
VarBench: Robust Language Model Benchmarking Through Dynamic Variable Perturbation Kun Qian Shunji Wan Claudia Tang Youzhi Wang Xuanming Zhang Maximillian Chen Zhou Yu AAML 35 8 0 25 Jun 2024
MatText: Do Language Models Need More than Text & Scale for Materials Modeling? Nawaf Alampara Santiago Miret K. Jablonka 43 8 0 25 Jun 2024
Understanding and Mitigating Tokenization Bias in Language Models Buu Phan Marton Havasi Matthew Muckley Karen Ullrich 37 2 0 24 Jun 2024
Evaluating Numerical Reasoning in Text-to-Image Models Ivana Kajić Olivia Wiles Isabela Albuquerque Matthias Bauer Su Wang Jordi Pont-Tuset Aida Nematzadeh EGVM ReLM 75 0 0 20 Jun 2024
Integrating Large Language Models with Graph-based Reasoning for Conversational Question Answering Parag Jain Mirella Lapata 31 0 0 14 Jun 2024
VersiCode: Towards Version-controllable Code Generation Tongtong Wu Weigang Wu Xingyu Wang Kang Xu Suyu Ma Bo Jiang Ping Yang Zhenchang Xing Yuan-Fang Li Gholamreza Haffari 34 4 0 11 Jun 2024
Through the Thicket: A Study of Number-Oriented LLMs derived from Random Forest Models M. Romaszewski Przemysław Sekuła P. Głomb M. Cholewa Katarzyna Kołodziej 22 0 0 07 Jun 2024
Large Language Models as In-context AI Generators for Quality-Diversity Bryan Lim Manon Flageat Antoine Cully 24 4 0 24 Apr 2024
Evaluating Subword Tokenization: Alien Subword Composition and OOV Generalization Challenge Khuyagbaatar Batsuren Ekaterina Vylomova Verna Dankers Tsetsuukhei Delgerbaatar Omri Uzan Yuval Pinter Gábor Bella 27 9 0 20 Apr 2024
Advancing Social Intelligence in AI Agents: Technical Challenges and Open Questions Leena Mathur Paul Pu Liang Louis-Philippe Morency LLMAG 25 6 0 17 Apr 2024
NumeroLogic: Number Encoding for Enhanced LLMs' Numerical Reasoning Eli Schwartz Leshem Choshen J. Shtok Sivan Doveh Leonid Karlinsky Assaf Arbelle 26 13 0 30 Mar 2024
Adversarial Math Word Problem Generation Roy Xie Chengxuan Huang Junlin Wang Bhuwan Dhingra AAML 21 1 0 27 Feb 2024
Large Language Models are Zero-Shot Reasoners Takeshi Kojima S. Gu Machel Reid Yutaka Matsuo Yusuke Iwasawa ReLM LRM 291 2,712 0 24 May 2022
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Jason W. Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Brian Ichter F. Xia Ed H. Chi Quoc Le Denny Zhou LM&Ro LRM AI4CE ReLM 315 8,261 0 28 Jan 2022
The Pile: An 800GB Dataset of Diverse Text for Language Modeling Leo Gao Stella Biderman Sid Black Laurence Golding Travis Hoppe ... Horace He Anish Thite Noa Nabeshima Shawn Presser Connor Leahy AIMat 245 1,977 0 31 Dec 2020