BERT on a Data Diet: Finding Important Examples by Gradient-Based
Pruning

BERT on a Data Diet: Finding Important Examples by Gradient-Based Pruning

10 November 2022

Ehsan Aghazadeh

Mohammad Taher Pilehvar

Yadollah Yaghoobzadeh

Samira Ebrahimi Kahou

Papers citing "BERT on a Data Diet: Finding Important Examples by Gradient-Based Pruning"

14 / 14 papers shown

Title
As easy as PIE: understanding when pruning causes language models to disagree Pietro Tropeano Maria Maistro Tuukka Ruotsalo Christina Lioma 55 0 0 27 Mar 2025
Swift Cross-Dataset Pruning: Enhancing Fine-Tuning Efficiency in Natural Language Understanding Binh-Nguyen Nguyen Yang He 23 1 0 05 Jan 2025
A Bayesian Approach to Data Point Selection Xinnuo Xu Minyoung Kim Royson Lee Brais Martínez Timothy M. Hospedales 33 0 0 06 Nov 2024
MelissaDL x Breed: Towards Data-Efficient On-line Supervised Training of Multi-parametric Surrogates with Active Learning Sofya Dymchenko Abhishek Purandare Bruno Raffin AI4CE 26 0 0 08 Oct 2024
Automatic Pruning of Fine-tuning Datasets for Transformer-based Language Models Mohammadreza Tayaranian S. H. Mozafari Brett H. Meyer J. Clark Warren J. Gross 27 1 0 11 Jul 2024
Sexism Detection on a Data Diet Rabiraj Bandyopadhyay Dennis Assenmacher J. Alonso-Moral Claudia Wagner 31 0 0 07 Jun 2024
SMART: Submodular Data Mixture Strategy for Instruction Tuning Kowndinya Renduchintala S. Bhatia Ganesh Ramakrishnan 36 3 0 13 Mar 2024
Efficient Backpropagation with Variance-Controlled Adaptive Sampling Ziteng Wang Jianfei Chen Jun Zhu BDL 27 2 0 27 Feb 2024
Efficient Architecture Search via Bi-level Data Pruning Chongjun Tu Peng Ye Weihao Lin Hancheng Ye Chong Yu Tao Chen Baopu Li Wanli Ouyang 37 2 0 21 Dec 2023
Influence Scores at Scale for Efficient Language Data Sampling Nikhil Anand Joshua Tan Maria Minakova TDI 21 3 0 27 Nov 2023
D2 Pruning: Message Passing for Balancing Diversity and Difficulty in Data Pruning A. Maharana Prateek Yadav Mohit Bansal 14 28 0 11 Oct 2023
NLU on Data Diets: Dynamic Data Subset Selection for NLP Classification Tasks Jean-Michel Attendu Jean-Philippe Corbeil 15 15 0 05 Jun 2023
PaD: Program-aided Distillation Can Teach Small Models Reasoning Better than Chain-of-thought Fine-tuning Xuekai Zhu Biqing Qi Kaiyan Zhang Xingwei Long Zhouhan Lin Bowen Zhou ALM LRM 28 19 0 23 May 2023
Clean or Annotate: How to Spend a Limited Data Collection Budget Derek Chen Zhou Yu Samuel R. Bowman 27 13 0 15 Oct 2021