Koala: An Index for Quantifying Overlaps with Pre-training Corpora

Koala: An Index for Quantifying Overlaps with Pre-training Corpora

26 March 2023

Gholamreza Haffari

Ehsan Shareghi

Papers citing "Koala: An Index for Quantifying Overlaps with Pre-training Corpora"

16 / 16 papers shown

Title
ICon: In-Context Contribution for Automatic Data Selection Yixin Yang Qingxiu Dong Linli Yao Fangwei Zhu Zhifang Sui 41 0 0 08 May 2025
D3: Diversity, Difficulty, and Dependability-Aware Data Selection for Sample-Efficient LLM Instruction Tuning Jia Zhang Chen-Xi Zhang Yao Liu Yi-Xuan Jin Xiao-Wen Yang Bo Zheng Y. Liu Lan-Zhe Guo 47 2 0 14 Mar 2025
Language Model Preference Evaluation with Multiple Weak Evaluators Zhengyu Hu Jieyu Zhang Zhihan Xiong Alexander Ratner Hui Xiong Ranjay Krishna 39 3 0 14 Oct 2024
Language model developers should report train-test overlap Andy K. Zhang Kevin Klyman Yifan Mai Yoav Levine Yian Zhang Rishi Bommasani Percy Liang VLM ELM 21 8 0 10 Oct 2024
ConStat: Performance-Based Contamination Detection in Large Language Models Jasper Dekoninck Mark Niklas Muller Martin Vechev 32 5 0 25 May 2024
Selective Reflection-Tuning: Student-Selected Data Recycling for LLM Instruction-Tuning Ming Li Lichang Chen Jiuhai Chen Shwai He Jiuxiang Gu Tianyi Zhou 8 50 0 15 Feb 2024
Evading Data Contamination Detection for Language Models is (too) Easy Jasper Dekoninck Mark Niklas Muller Maximilian Baader Marc Fischer Martin Vechev 82 18 0 05 Feb 2024
Infini-gram: Scaling Unbounded n-gram Language Models to a Trillion Tokens Jiacheng Liu Sewon Min Luke Zettlemoyer Yejin Choi Hannaneh Hajishirzi 32 50 0 30 Jan 2024
MoDS: Model-oriented Data Selection for Instruction Tuning Qianlong Du Chengqing Zong Jiajun Zhang ALM 13 75 0 27 Nov 2023
Which Prompts Make The Difference? Data Prioritization For Efficient Human LLM Evaluation M. Boubdir Edward Kim B. Ermiş Marzieh Fadaee Sara Hooker ALM 23 18 0 22 Oct 2023
From Quantity to Quality: Boosting LLM Performance with Self-Guided Data Selection for Instruction Tuning Ming Li Yong Zhang Zhitao Li Jiuhai Chen Lichang Chen Ning Cheng Jianzong Wang Tianyi Zhou Jing Xiao 25 168 0 23 Aug 2023
Large Language Models are Zero-Shot Reasoners Takeshi Kojima S. Gu Machel Reid Yutaka Matsuo Yusuke Iwasawa ReLM LRM 291 2,712 0 24 May 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 301 11,730 0 04 Mar 2022
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Jason W. Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Brian Ichter F. Xia Ed H. Chi Quoc Le Denny Zhou LM&Ro LRM AI4CE ReLM 315 8,261 0 28 Jan 2022
Memorisation versus Generalisation in Pre-trained Language Models Michael Tänzer Sebastian Ruder Marek Rei 84 50 0 16 Apr 2021
The Pile: An 800GB Dataset of Diverse Text for Language Modeling Leo Gao Stella Biderman Sid Black Laurence Golding Travis Hoppe ... Horace He Anish Thite Noa Nabeshima Shawn Presser Connor Leahy AIMat 242 1,977 0 31 Dec 2020