Data Selection for Language Models via Importance Resampling

6 February 2023

Papers citing "Data Selection for Language Models via Importance Resampling"

50 / 147 papers shown

Title
R&B: Domain Regrouping and Data Mixture Balancing for Efficient Foundation Model Training Albert Ge Tzu-Heng Huang John Cooper Avi Trost Ziyi Chu Satya Sai Srinath Namburi GNVV Ziyang Cai Kendall Park Nicholas Roberts Frederic Sala 53 0 0 01 May 2025
Combatting Dimensional Collapse in LLM Pre-Training Data via Diversified File Selection Ziqing Fan Siyuan Du Shengchao Hu Pingjie Wang Li Shen Y. Zhang Dacheng Tao Y. Wang 41 1 0 29 Apr 2025
Studying Small Language Models with Susceptibilities Garrett Baker George Wang Jesse Hoogland Daniel Murfet AAML 73 1 0 25 Apr 2025
QuaDMix: Quality-Diversity Balanced Data Selection for Efficient LLM Pretraining Fengze Liu Weidong Zhou Binbin Liu Zhimiao Yu Yifan Zhang ... Yifeng Yu Bingni Zhang Xiaohuan Zhou Taifeng Wang Yong Cao 55 0 0 23 Apr 2025
A Survey of Foundation Model-Powered Recommender Systems: From Feature-Based, Generative to Agentic Paradigms Chengkai Huang Hongtao Huang Tong Yu Kaige Xie Junda Wu Shuai Zhang Julian McAuley Dietmar Jannach Lina Yao LRM AI4CE 24 0 0 23 Apr 2025
The Rise of Small Language Models in Healthcare: A Comprehensive Survey Muskan Garg Shaina Raza Shebuti Rayana Xingyi Liu Sunghwan Sohn LM&MA AILaw 87 0 0 23 Apr 2025
ReSpec: Relevance and Specificity Grounded Online Filtering for Learning on Video-Text Data Streams C. Kim Jihwan Moon Sangwoo Moon Heeseung Yun Sihaeng Lee Aniruddha Kembhavi Soonyoung Lee Gunhee Kim Sangho Lee Christopher Clark 23 0 0 21 Apr 2025
Meta-rater: A Multi-dimensional Data Selection Method for Pre-training Language Models Xinlin Zhuang Jiahui Peng Ren Ma Y. Wang Tianyi Bai Xingjian Wei Jiantao Qiu Chi Zhang Ying Qian Conghui He 39 0 0 19 Apr 2025
CLIMB: CLustering-based Iterative Data Mixture Bootstrapping for Language Model Pre-training Shizhe Diao Yu Yang Y. Fu Xin Dong Dan Su ... Hongxu Yin M. Patwary Yingyan Jan Kautz Pavlo Molchanov 33 0 0 17 Apr 2025
Transferable text data distillation by trajectory matching Rong Yao Hailin Hu Yifei Fu Hanting Chen Wenyi Fang Fanyi Du Kai Han Yunhe Wang 26 0 0 14 Apr 2025
SoTA with Less: MCTS-Guided Sample Selection for Data-Efficient Visual Reasoning Self-Improvement X. Wang Z. Yang Chao Feng Hongjin Lu Linjie Li Chung-Ching Lin Kevin Qinghong Lin Furong Huang Lijuan Wang OODD ReLM VLM LRM 69 1 0 10 Apr 2025
Efficient Evaluation of Large Language Models via Collaborative Filtering Xu-Xiang Zhong Chao Yi Han-Jia Ye 24 0 0 05 Apr 2025
ToReMi: Topic-Aware Data Reweighting for Dynamic Pre-Training Data Selection Xiaoxuan Zhu Zhouhong Gu Baiqian Wu Suhang Zheng Tao Wang Tianyu Li Hongwei Feng Yanghua Xiao 40 0 0 01 Apr 2025
Towards Automatic Continual Learning: A Self-Adaptive Framework for Continual Instruction Tuning Peiyi Lin Fukai Zhang Kai Niu Hao Fu CLL 64 0 0 20 Mar 2025
MASS: Mathematical Data Selection via Skill Graphs for Pretraining Large Language Models J. Li Lu Yu Qing Cui Zhiqiang Zhang Jun Zhou Yanfang Ye Chuxu Zhang 59 0 0 19 Mar 2025
Neuronal Activation States as Sample Embeddings for Data Selection in Task-Specific Instruction Tuning Da Ma Gonghu Shang Zhi Chen L. Qin Yijie Luo Lei Pan Shuai Fan L. Chen Kai Yu 36 0 0 19 Mar 2025
Add-One-In: Incremental Sample Selection for Large Language Models via a Choice-Based Greedy Paradigm Z. Li Yuhao Du Xiaoqi Jiao Yiwen Guo Yuege Feng Xiang Wan Anningzhe Gao Jinpeng Hu 63 0 0 04 Mar 2025
SampleMix: A Sample-wise Pre-training Data Mixing Strategey by Coordinating Data Quality and Diversity Xiangyu Xi Deyang Kong Jian Yang Jiawei Yang Z. Chen Wei Wang J. T. Wang Xunliang Cai Shikun Zhang Wei Ye 60 0 0 03 Mar 2025
Predictive Data Selection: The Data That Predicts Is the Data That Teaches Kashun Shum Y. Huang Hongjian Zou Qi Ding Yixuan Liao X. Chen Qian Liu Junxian He 60 2 0 02 Mar 2025
CritiQ: Mining Data Quality Criteria from Human Preferences Honglin Guo Kai Lv Qipeng Guo Tianyi Liang Zhiheng Xi ... Qiuyinzhe Zhang Y. Sun K. Chen Xipeng Qiu Tao Gui 33 0 0 26 Feb 2025
Constraining Sequential Model Editing with Editing Anchor Compression Hao-Xiang Xu Jun-Yu Ma Zhen-Hua Ling Ningyu Zhang Jia-Chen Gu KELM 47 1 0 25 Feb 2025
Is Free Self-Alignment Possible? Dyah Adila Changho Shin Yijing Zhang Frederic Sala MoMe 108 2 0 24 Feb 2025
Unsupervised Topic Models are Data Mixers for Pre-training Language Models Jiahui Peng Xinlin Zhuang Qiu Jiantao Ren Ma Jing Yu Tianyi Bai Conghui He 36 0 0 24 Feb 2025
Programming Every Example: Lifting Pre-training Data Quality Like Experts at Scale Fan Zhou Zengzhi Wang Qian Liu Junlong Li Pengfei Liu ALM 100 15 0 17 Feb 2025
Do we really have to filter out random noise in pre-training data for language models? Jinghan Ru Yuxin Xie Xianwei Zhuang Yuguo Yin Yuexian Zou 83 2 0 10 Feb 2025
FRAMES: Boosting LLMs with A Four-Quadrant Multi-Stage Pretraining Strategy Xuemiao Zhang Feiyu Duan Liangyu Xu Yongwei Zhou Sirui Wang Rongxiang Weng J. Wang Xunliang Cai 60 0 0 08 Feb 2025
Ensembles of Low-Rank Expert Adapters Yinghao Li Vianne Gao Chao Zhang MohamadAli Torkamani 60 0 0 31 Jan 2025
Swift Cross-Dataset Pruning: Enhancing Fine-Tuning Efficiency in Natural Language Understanding Binh-Nguyen Nguyen Yang He 33 1 0 05 Jan 2025
Unleashing the Power of Data Tsunami: A Comprehensive Survey on Data Assessment and Selection for Instruction Tuning of Language Models Yulei Qin Yuncheng Yang Pengcheng Guo Gang Li Hang Shao Yuchen Shi Zihan Xu Yun Gu Ke Li Xing Sun ALM 88 12 0 31 Dec 2024
Maximize Your Data's Potential: Enhancing LLM Accuracy with Two-Phase Pretraining Steven Feng Shrimai Prabhumoye Kezhi Kong Dan Su M. Patwary M. Shoeybi Bryan Catanzaro 67 2 0 18 Dec 2024
Weak-to-Strong Generalization Through the Data-Centric Lens Changho Shin John Cooper Frederic Sala 83 5 0 05 Dec 2024
ROSE: A Reward-Oriented Data Selection Framework for LLM Task-Specific Instruction Tuning Yang Wu Huayi Zhang Yizheng Jiao Lin Ma Xiaozhong Liu Jinhong Yu Dongyu Zhang Dezhi Yu Wei Xu 78 1 0 01 Dec 2024
Paint Outside the Box: Synthesizing and Selecting Training Data for Visual Grounding Zilin Du Haoxin Li Jianfei Yu Boyang Li 120 0 0 01 Dec 2024
From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judge Dawei Li Bohan Jiang Liangjie Huang Alimohammad Beigi Chengshuai Zhao ... Canyu Chen Tianhao Wu Kai Shu Lu Cheng Huan Liu ELM AILaw 113 65 0 25 Nov 2024
LIMBA: An Open-Source Framework for the Preservation and Valorization of Low-Resource Languages using Generative Models S. Carta Stefano Chessa Giulia Contu Andrea Corriga Andrea Deidda ... Alessia Pisu Alessandro Sebastian Podda Livio Pompianu S. Sandro Gabriele Tiddia 78 0 0 20 Nov 2024
Training Bilingual LMs with Data Constraints in the Targeted Language Skyler Seto Maartje ter Hoeve He Bai Natalie Schluter David Grangier 77 0 0 20 Nov 2024
Clustering Algorithms and RAG Enhancing Semi-Supervised Text Classification with Large LLMs Shan Zhong Jiahao Zeng Yongxin Yu Bohong Lin 34 1 0 09 Nov 2024
ZIP-FIT: Embedding-Free Data Selection via Compression-Based Alignment Elyas Obbad Iddah Mlauzi Brando Miranda Rylan Schaeffer Kamal Obbad Suhana Bedi Sanmi Koyejo CVBM 48 0 0 23 Oct 2024
Influential Language Data Selection via Gradient Trajectory Pursuit Zhiwei Deng Tao Li Yang Li 24 1 0 22 Oct 2024
Compute-Constrained Data Selection Junjie Oscar Yin Alexander M. Rush 39 0 0 21 Oct 2024
TSDS: Data Selection for Task-Specific Model Finetuning Zifan Liu Amin Karbasi Theodoros Rekatsinas 29 3 0 15 Oct 2024
Reverse Modeling in Large Language Models S. Yu Yuanchen Xu Cunxiao Du Yanying Zhou Minghui Qiu Q. Sun Hao Zhang Jiawei Wu 29 2 0 13 Oct 2024
Multi-Agent Collaborative Data Selection for Efficient LLM Pretraining Tianyi Bai Ling Yang Zhen Hao Wong Jiahui Peng Xinlin Zhuang ... Lijun Wu Jiantao Qiu Wentao Zhang Binhang Yuan Conghui He LLMAG 23 4 0 10 Oct 2024
Koala-36M: A Large-scale Video Dataset Improving Consistency between Fine-grained Conditions and Video Content Qiuheng Wang Yukai Shi Jiarong Ou R. J. Chen Ke Lin ... Mingwu Zheng Xin Tao Fei Yang Pengfei Wan Di Zhang VGen 86 18 0 10 Oct 2024
SEAL: Safety-enhanced Aligned LLM Fine-tuning via Bilevel Data Selection Han Shen Pin-Yu Chen Payel Das Tianyi Chen ALM 26 11 0 09 Oct 2024
Data Selection via Optimal Control for Language Models Yuxian Gu Li Dong Hongning Wang Y. Hao Qingxiu Dong Furu Wei Minlie Huang AI4CE 48 4 0 09 Oct 2024
DecorateLM: Data Engineering through Corpus Rating, Tagging, and Editing with Language Models Ranchi Zhao Zhen Leng Thai Yifan Zhang Shengding Hu Yunqi Ba Jie Zhou Jie Cai Zhiyuan Liu Maosong Sun 28 1 0 08 Oct 2024
Rule-based Data Selection for Large Language Models Xiaomin Li Mingye Gao Zhiwei Zhang Chang Yue Hong Hu 30 4 0 07 Oct 2024
Language Model-Driven Data Pruning Enables Efficient Active Learning Abdul Hameed Azeemi I. Qazi Agha Ali Raza VLM 25 1 0 05 Oct 2024
Scaling Parameter-Constrained Language Models with Quality Data Ernie Chang Matteo Paltenghi Yang Li Pin-Jie Lin Changsheng Zhao Patrick Huber Zechun Liu Rastislav Rabatin Yangyang Shi Vikas Chandra 54 1 0 04 Oct 2024