v1v2 (latest)

Data Portraits: Recording Foundation Model Training Data

Neural Information Processing Systems (NeurIPS), 2023

6 March 2023

Marc Marone

Benjamin Van Durme

ArXiv (abs)PDF HTML

Papers citing "Data Portraits: Recording Foundation Model Training Data"

25 / 25 papers shown

Title
Computational Budget Should Be Considered in Data Selection Weilin Wan Weizhong Zhang Cheng Jin 32 0 0 19 Oct 2025
TechOps: Technical Documentation Templates for the AI Act Laura Lucaj Alex Loosley Hakan Jonsson Urs Gasser Patrick van der Smagt 36 0 0 12 Aug 2025
Certified Mitigation of Worst-Case LLM Copyright Infringement Jingyu Zhang Jiacan Yu Marc Marone Benjamin Van Durme Daniel Khashabi MoMe 736 1 0 22 Apr 2025
Unleashing the Power of Data Tsunami: A Comprehensive Survey on Data Assessment and Selection for Instruction Tuning of Language Models Yulei Qin Yuncheng Yang Pengcheng Guo Gang Li Hang Shao Yuchen Shi Zihan Xu Yun Gu Ke Li Xing Sun ALM 418 19 0 31 Dec 2024
CODECLEANER: Elevating Standards with A Robust Data Contamination Mitigation Toolkit Jialun Cao Songqiang Chen Wuqi Zhang Hau Ching Lo Shing-Chi Cheung 152 1 0 16 Nov 2024
Does Data Contamination Detection Work (Well) for LLMs? A Survey and Evaluation on Detection AssumptionsNorth American Chapter of the Association for Computational Linguistics (NAACL), 2024 Yujuan Fu Özlem Uzuner Meliha Yetisgen Fei Xia 229 15 0 24 Oct 2024
What's New in My Data? Novelty Exploration via Contrastive GenerationInternational Conference on Learning Representations (ICLR), 2024 Masaru Isonuma Ivan Titov 114 0 0 18 Oct 2024
Improving governance outcomes through AI documentation: Bridging theory and practiceInternational Conference on Human Factors in Computing Systems (CHI), 2024 Amy A. Winecoff Miranda Bogen 133 5 0 13 Sep 2024
AdaCAD: Adaptively Decoding to Balance Conflicts between Contextual and Parametric KnowledgeNorth American Chapter of the Association for Computational Linguistics (NAACL), 2024 Han Wang Archiki Prasad Elias Stengel-Eskin Joey Tianyi Zhou 240 15 0 11 Sep 2024
Demystifying Verbatim Memorization in Large Language Models Jing Huang Diyi Yang Christopher Potts ELM PILM MU 203 41 0 25 Jul 2024
Evaluating Copyright Takedown Methods for Language Models Boyi Wei Weijia Shi Yangsibo Huang Noah A. Smith Chiyuan Zhang Luke Zettlemoyer Kai Li Peter Henderson 258 32 0 26 Jun 2024
Navigating the Future of Federated Recommendation Systems with Foundation Models Zhiwei Li Guodong Long Chunxu Zhang Honglei Zhang Jing Jiang Chengqi Zhang 513 0 0 12 May 2024
Verifiable by Design: Aligning Language Models to Quote from Pre-Training Data Jingyu Zhang Marc Marone Tianjian Li Benjamin Van Durme Daniel Khashabi 317 11 0 05 Apr 2024
Dated Data: Tracing Knowledge Cutoffs in Large Language Models Jeffrey Cheng Marc Marone Orion Weller Dawn J Lawrie Daniel Khashabi Benjamin Van Durme 180 42 0 19 Mar 2024
Proving membership in LLM pretraining data via data watermarks Johnny Tian-Zheng Wei Ryan Yixiang Wang Robin Jia WaLM 184 43 0 16 Feb 2024
On Catastrophic Inheritance of Large Foundation Models Hao Chen Bhiksha Raj Xing Xie Yongfeng Zhang AI4CE 193 13 0 02 Feb 2024
Infini-gram: Scaling Unbounded n-gram Language Models to a Trillion Tokens Hamish Ivison Sewon Min Luke Zettlemoyer Yejin Choi Hannaneh Hajishirzi 218 89 0 30 Jan 2024
Oasis: Data Curation and Assessment System for Pretraining of Large Language ModelsInternational Joint Conference on Artificial Intelligence (IJCAI), 2023 Tong Zhou Yubo Chen Pengfei Cao Kang Liu Jun Zhao Shengping Liu 121 3 0 21 Nov 2023
NLP Evaluation in trouble: On the Need to Measure LLM Data Contamination for each BenchmarkConference on Empirical Methods in Natural Language Processing (EMNLP), 2023 Oscar Sainz Jon Ander Campos Iker García-Ferrero Julen Etxaniz Oier López de Lacalle Eneko Agirre 145 242 0 27 Oct 2023
Cognitive Mirage: A Review of Hallucinations in Large Language Models Hongbin Ye Tong Liu Aijia Zhang Wei Hua Weiqiang Jia HILM 224 104 0 13 Sep 2023
Position: Key Claims in LLM Research Have a Long Tail of FootnotesInternational Conference on Machine Learning (ICML), 2023 Anna Rogers A. Luccioni 251 22 0 14 Aug 2023
"According to ...": Prompting Language Models Improves Quoting from Pre-Training DataConference of the European Chapter of the Association for Computational Linguistics (EACL), 2023 Orion Weller Marc Marone Nathaniel Weir Dawn J Lawrie Daniel Khashabi Benjamin Van Durme HILM 191 54 0 22 May 2023
Stop Uploading Test Data in Plain Text: Practical Strategies for Mitigating Data Contamination by Evaluation BenchmarksConference on Empirical Methods in Natural Language Processing (EMNLP), 2023 Alon Jacovi Avi Caciularu Omer Goldman Yoav Goldberg 163 119 0 17 May 2023
StarCoder: may the source be with you! Raymond Li Loubna Ben Allal Yangtian Zi Niklas Muennighoff Denis Kocetkov ... Sean M. Hughes Thomas Wolf Arjun Guha Leandro von Werra H. D. Vries 273 969 0 09 May 2023
Speak, Memory: An Archaeology of Books Known to ChatGPT/GPT-4Conference on Empirical Methods in Natural Language Processing (EMNLP), 2023 Kent K. Chang Mackenzie Cramer Sandeep Soni David Bamman RALM 385 150 0 28 Apr 2023