v1v2 (latest)

Enhancing LLMs via High-Knowledge Data Selection

20 May 2025

Papers citing "Enhancing LLMs via High-Knowledge Data Selection"

27 / 27 papers shown

Title
Physics of Language Models: Part 3.3, Knowledge Capacity Scaling Laws Zeyuan Allen-Zhu Yuanzhi Li KELM 53 70 0 08 Apr 2024
DsDm: Model-Aware Dataset Selection with Datamodels Logan Engstrom Axel Feldmann Aleksander Madry OODD 104 61 0 23 Jan 2024
AboutMe: Using Self-Descriptions in Webpages to Document the Effects of English Pretraining Data Filters L. Lucy Suchin Gururangan Luca Soldaini Emma Strubell David Bamman Lauren Klein Jesse Dodge 117 17 0 12 Jan 2024
A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions Lei Huang Weijiang Yu Weitao Ma Weihong Zhong Zhangyin Feng ... Qianglong Chen Weihua Peng Xiaocheng Feng Bing Qin Ting Liu LRM HILM 142 935 0 09 Nov 2023
Self-Influence Guided Data Reweighting for Language Model Pre-training Megh Thakkar Tolga Bolukbasi Sriram Ganapathy Shikhar Vashishth Sarath Chandar Partha P. Talukdar MILM 109 26 0 02 Nov 2023
Textbooks Are All You Need II: phi-1.5 technical report Yuan-Fang Li Sébastien Bubeck Ronen Eldan Allison Del Giorno Suriya Gunasekar Yin Tat Lee ALM LRM 171 482 0 11 Sep 2023
Llama 2: Open Foundation and Fine-Tuned Chat Models Hugo Touvron Louis Martin Kevin R. Stone Peter Albert Amjad Almahairi ... Sharan Narang Aurelien Rodriguez Robert Stojnic Sergey Edunov Thomas Scialom AI4MH ALM 498 12,124 0 18 Jul 2023
CMMLU: Measuring massive multitask language understanding in Chinese Haonan Li Yixuan Zhang Fajri Koto Yifei Yang Hai Zhao Yeyun Gong Nan Duan Tim Baldwin ALM ELM 116 273 0 15 Jun 2023
Unifying Large Language Models and Knowledge Graphs: A Roadmap Shirui Pan Linhao Luo Yufei Wang Chen Chen Jiapu Wang Xindong Wu KELM 156 789 0 14 Jun 2023
DoReMi: Optimizing Data Mixtures Speeds Up Language Model Pretraining Sang Michael Xie Hieu H. Pham Xuanyi Dong Nan Du Hanxiao Liu Yifeng Lu Percy Liang Quoc V. Le Tengyu Ma Adams Wei Yu MoMe MoE 152 204 0 17 May 2023
C-Eval: A Multi-Level Multi-Discipline Chinese Evaluation Suite for Foundation Models Yuzhen Huang Yuzhuo Bai Zhihao Zhu Junlei Zhang Jinghan Zhang ... Yikai Zhang Jiayi Lei Yao Fu Maosong Sun Junxian He ELM LRM 121 552 0 15 May 2023
Data Selection for Language Models via Importance Resampling Sang Michael Xie Shibani Santurkar Tengyu Ma Percy Liang 131 196 0 06 Feb 2023
A Survey on In-context Learning Qingxiu Dong Lei Li Damai Dai Ce Zheng Jingyuan Ma ... Zhiyong Wu Baobao Chang Xu Sun Lei Li Zhifang Sui ReLM AIMat 152 546 0 31 Dec 2022
Large Language Models Struggle to Learn Long-Tail Knowledge Nikhil Kandpal H. Deng Adam Roberts Eric Wallace Colin Raffel RALM KELM 143 419 0 15 Nov 2022
Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Them Mirac Suzgun Nathan Scales Nathanael Scharli Sebastian Gehrmann Yi Tay ... Aakanksha Chowdhery Quoc V. Le Ed H. Chi Denny Zhou Jason W. Wei ALM ELM LRM ReLM 280 1,143 0 17 Oct 2022
Whose Language Counts as High Quality? Measuring Language Ideologies in Text Data Selection Suchin Gururangan Dallas Card Sarah K. Drier E. K. Gade Leroy Z. Wang Zeyu Wang Luke Zettlemoyer Noah A. Smith 268 81 0 25 Jan 2022
Scaling Language Models: Methods, Analysis & Insights from Training Gopher Jack W. Rae Sebastian Borgeaud Trevor Cai Katie Millican Jordan Hoffmann ... Jeff Stanway L. Bennett Demis Hassabis Koray Kavukcuoglu G. Irving 179 1,327 0 08 Dec 2021
FewCLUE: A Chinese Few-shot Learning Evaluation Benchmark Liang Xu Xiaojing Lu Chenyang Yuan Xuanwei Zhang Huilin Xu ... Guoao Wei X. Pan Xin Tian Libo Qin Hai Hu ELM 90 57 0 15 Jul 2021
Quality at a Glance: An Audit of Web-Crawled Multilingual Datasets Julia Kreutzer Isaac Caswell Lisa Wang Ahsan Wahab D. Esch ... Duygu Ataman Orevaoghene Ahia Oghenefego Ahia Sweta Agrawal Mofetoluwa Adeyemi 64 280 0 22 Mar 2021
Think you have Solved Direct-Answer Question Answering? Try ARC-DA, the Direct-Answer AI2 Reasoning Challenge Sumithra Bhakthavatsalam Daniel Khashabi Tushar Khot Bhavana Dalvi Kyle Richardson Ashish Sabharwal Carissa Schoenick Oyvind Tafjord Peter Clark RALM AI4CE 70 66 0 05 Feb 2021
The Pile: An 800GB Dataset of Diverse Text for Language Modeling Leo Gao Stella Biderman Sid Black Laurence Golding Travis Hoppe ... Horace He Anish Thite Noa Nabeshima Shawn Presser Connor Leahy AIMat 484 2,126 0 31 Dec 2020
Measuring Massive Multitask Language Understanding Dan Hendrycks Collin Burns Steven Basart Andy Zou Mantas Mazeika Basel Alomair Jacob Steinhardt ELM RALM 207 4,580 0 07 Sep 2020
Language Models are Few-Shot Learners Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan ... Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever Dario Amodei BDL 1.0K 42,651 0 28 May 2020
CLUE: A Chinese Language Understanding Evaluation Benchmark Liang Xu Hai Hu Xuanwei Zhang Lu Li Chenjie Cao ... Cong Yue Xinrui Zhang Zhen-Yi Yang Kyle Richardson Zhenzhong Lan ELM 110 388 0 13 Apr 2020
Stochastic Beams and Where to Find Them: The Gumbel-Top-k Trick for Sampling Sequences Without Replacement W. Kool H. V. Hoof Max Welling 135 220 0 14 Mar 2019
Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answering Todor Mihaylov Peter Clark Tushar Khot Ashish Sabharwal 130 1,571 0 08 Sep 2018
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 1.2K 7,210 0 20 Apr 2018