LESS: Selecting Influential Data for Targeted Instruction Tuning

LESS: Selecting Influential Data for Targeted Instruction Tuning

6 February 2024

Sadhika Malladi

Suchin Gururangan

Papers citing "LESS: Selecting Influential Data for Targeted Instruction Tuning"

13 / 13 papers shown

Title
Adversarial Cooperative Rationalization: The Risk of Spurious Correlations in Even Clean Datasets W. Liu Zhongyu Niu Lang Gao Zhiying Deng Jun Wang H. Wang Ruixuan Li 28 47 0 04 May 2025
R&B: Domain Regrouping and Data Mixture Balancing for Efficient Foundation Model Training Albert Ge Tzu-Heng Huang John Cooper Avi Trost Ziyi Chu Satya Sai Srinath Namburi GNVV Ziyang Cai Kendall Park Nicholas Roberts Frederic Sala 38 32 0 01 May 2025
Reinforcement Learning for Reasoning in Large Language Models with One Training Example Yiping Wang Qing Yang Zhiyuan Zeng Liliang Ren L. Liu ... Jianfeng Gao Weizhu Chen S. Wang Simon S. Du Yelong Shen OffRL ReLM LRM 97 54 0 29 Apr 2025
Extracting and Transferring Abilities For Building Multi-lingual Ability-enhanced Large Language Models Zhipeng Chen Liang Song K. Zhou Wayne Xin Zhao B. Wang Weipeng Chen Ji-Rong Wen 43 67 0 10 Oct 2024
Data Diversity Matters for Robust Instruction Tuning Alexander Bukharin Tuo Zhao 43 15 0 21 Nov 2023
Simfluence: Modeling the Influence of Individual Training Examples by Simulating Training Runs Kelvin Guu Albert Webson Ellie Pavlick Lucas Dixon Ian Tenney Tolga Bolukbasi TDI 43 25 0 14 Mar 2023
A Kernel-Based View of Language Model Fine-Tuning Sadhika Malladi Alexander Wettig Dingli Yu Danqi Chen Sanjeev Arora VLM 46 40 0 11 Oct 2022
Understanding Influence Functions and Datamodels via Harmonic Analysis Nikunj Saunshi Arushi Gupta M. Braverman Sanjeev Arora TDI 29 13 0 03 Oct 2022
On the SDEs and Scaling Rules for Adaptive Gradient Algorithms Sadhika Malladi Kaifeng Lyu A. Panigrahi Sanjeev Arora 77 26 0 20 May 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 298 7,763 0 04 Mar 2022
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Jason W. Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Brian Ichter F. Xia Ed H. Chi Quoc Le Denny Zhou LM&Ro LRM AI4CE ReLM 295 4,807 0 28 Jan 2022
Multitask Prompted Training Enables Zero-Shot Task Generalization Victor Sanh Albert Webson Colin Raffel Stephen H. Bach Lintang Sutawika ... T. Bers Stella Biderman Leo Gao Thomas Wolf Alexander M. Rush LRM 203 1,436 0 15 Oct 2021
GRAD-MATCH: Gradient Matching based Data Subset Selection for Efficient Deep Model Training Krishnateja Killamsetty D. Sivasubramanian Ganesh Ramakrishnan A. De Rishabh K. Iyer OOD 66 134 0 27 Feb 2021