Adaptive Data Optimization: Dynamic Sample Selection with Scaling Laws

Adaptive Data Optimization: Dynamic Sample Selection with Scaling Laws

15 October 2024

Yiding Jiang

Sadhika Malladi

Papers citing "Adaptive Data Optimization: Dynamic Sample Selection with Scaling Laws"

9 / 9 papers shown

Title
R&B: Domain Regrouping and Data Mixture Balancing for Efficient Foundation Model Training Albert Ge Tzu-Heng Huang John Cooper Avi Trost Ziyi Chu Satya Sai Srinath Namburi GNVV Ziyang Cai Kendall Park Nicholas Roberts Frederic Sala 47 0 0 01 May 2025
Data Mixture Optimization: A Multi-fidelity Multi-scale Bayesian Framework Thomson Yen Andrew Siah Haozhe Chen Tianyi Peng Daniel Guetta Hongseok Namkoong 43 0 0 26 Mar 2025
Not-Just-Scaling Laws: Towards a Better Understanding of the Downstream Impact of Language Model Design Decisions E. Liu Amanda Bertsch Lintang Sutawika Lindia Tjuatja Patrick Fernandes ... S. Carolin (Haas) Lawrence Aditi Raghunathan Kiril Gashteovski Graham Neubig 57 0 0 05 Mar 2025
Mixtera: A Data Plane for Foundation Model Training Maximilian Böther Xiaozhe Yao Tolga Kerimoglu Ana Klimovic Viktor Gsteiger Ana Klimovic MoE 72 0 0 27 Feb 2025
Between Circuits and Chomsky: Pre-pretraining on Formal Languages Imparts Linguistic Biases Michael Y. Hu Jackson Petty Chuan Shi William Merrill Tal Linzen AI4CE 59 1 0 26 Feb 2025
PiKE: Adaptive Data Mixing for Multi-Task Learning Under Low Gradient Conflicts Zeman Li Yuan Deng Peilin Zhong Meisam Razaviyayn Vahab Mirrokni MoMe 75 1 0 10 Feb 2025
Paint Outside the Box: Synthesizing and Selecting Training Data for Visual Grounding Zilin Du Haoxin Li Jianfei Yu Boyang Li 87 0 0 01 Dec 2024
Loss-to-Loss Prediction: Scaling Laws for All Datasets David Brandfonbrener Nikhil Anand Nikhil Vyas Eran Malach Sham Kakade 77 2 0 19 Nov 2024
Upsample or Upweight? Balanced Training on Heavily Imbalanced Datasets Tianjian Li Haoran Xu Weiting Tan Kenton Murray Daniel Khashabi 35 1 0 06 Oct 2024