Data Mixture Inference: What do BPE Tokenizers Reveal about their
Training Data?

Data Mixture Inference: What do BPE Tokenizers Reveal about their Training Data?

23 July 2024

Yejin Choi

Papers citing "Data Mixture Inference: What do BPE Tokenizers Reveal about their Training Data?"

8 / 8 papers shown

Title
Learning Dynamics in Continual Pre-Training for Large Language Models Xingjin Wang Howe Tissue Lu Wang Linjing Li D. Zeng CLL 21 0 0 12 May 2025
SuperBPE: Space Travel for Language Models Alisa Liu J. Hayase Valentin Hofmann Sewoong Oh Noah A. Smith Yejin Choi 43 1 0 17 Mar 2025
Cross-Lingual Transfer of Debiasing and Detoxification in Multilingual LLMs: An Extensive Investigation Vera Neplenbroek Arianna Bisazza Raquel Fernández 97 0 0 17 Feb 2025
Rethinking Tokenization: Crafting Better Tokenizers for Large Language Models Jinbiao Yang LLMAG 44 9 0 01 Mar 2024
OLMo: Accelerating the Science of Language Models Dirk Groeneveld Iz Beltagy Pete Walsh Akshita Bhagia Rodney Michael Kinney ... Jesse Dodge Kyle Lo Luca Soldaini Noah A. Smith Hanna Hajishirzi OSLM 130 349 0 01 Feb 2024
Data Augmentation for Code Translation with Comparable Corpora and Multiple References Yiqing Xie Atharva Naik Daniel Fried Carolyn Rose 34 6 0 01 Nov 2023
Speak, Memory: An Archaeology of Books Known to ChatGPT/GPT-4 Kent K. Chang Mackenzie Cramer Sandeep Soni David Bamman RALM 138 109 0 28 Apr 2023
Extracting Training Data from Large Language Models Nicholas Carlini Florian Tramèr Eric Wallace Matthew Jagielski Ariel Herbert-Voss ... Tom B. Brown D. Song Ulfar Erlingsson Alina Oprea Colin Raffel MLAU SILM 267 1,798 0 14 Dec 2020