An Empirical Study of Example Forgetting during Deep Neural Network Learning

12 December 2018

Mariya Toneva

Alessandro Sordoni

Rémi Tachet des Combes

Papers citing "An Empirical Study of Example Forgetting during Deep Neural Network Learning"

50 / 482 papers shown

Title
Diversity Measurement and Subset Selection for Instruction Tuning Datasets Peiqi Wang Yikang Shen Zhen Guo Matt Stallone Yoon Kim Polina Golland Rameswar Panda 23 8 0 04 Feb 2024
What Will My Model Forget? Forecasting Forgotten Examples in Language Model Refinement Xisen Jin Xiang Ren KELM CLL 20 6 0 02 Feb 2024
LTAU-FF: Loss Trajectory Analysis for Uncertainty in Atomistic Force Fields Joshua A. Vita Amit Samanta Fei Zhou Vincenzo Lordi 25 2 0 01 Feb 2024
FedCore: Straggler-Free Federated Learning with Distributed Coresets Hongpeng Guo Haotian Gu Xiaoyang Wang Bo Chen Eun Kyung Lee Tamar Eilam Deming Chen K. Nahrstedt FedML 18 1 0 31 Jan 2024
Contributing Dimension Structure of Deep Feature for Coreset Selection Zhijing Wan Zhixiang Wang Yuran Wang Zheng Wang Hongyuan Zhu Shiníchi Satoh 25 6 0 29 Jan 2024
Importance-Aware Adaptive Dataset Distillation Guang Li Ren Togo Takahiro Ogawa Miki Haseyama DD 28 6 0 29 Jan 2024
BackdoorBench: A Comprehensive Benchmark and Analysis of Backdoor Learning Baoyuan Wu Hongrui Chen Mingda Zhang Zihao Zhu Shaokui Wei Danni Yuan Mingli Zhu Ruotong Wang Li Liu Chaoxiao Shen AAML ELM 75 9 0 26 Jan 2024
One Step Learning, One Step Review Xiaolong Huang Qiankun Li Xueran Li Xuesong Gao 26 1 0 19 Jan 2024
lpNTK: Better Generalisation with Less Data via Sample Interaction During Learning Shangmin Guo Yi Ren Stefano V.Albrecht Kenny Smith 28 3 0 16 Jan 2024
Effective pruning of web-scale datasets based on complexity of concept clusters Amro Abbas E. Rusak Kushal Tirumala Wieland Brendel Kamalika Chaudhuri Ari S. Morcos VLM CLIP 34 22 0 09 Jan 2024
Towards Explainable Artificial Intelligence (XAI): A Data Mining Perspective Haoyi Xiong Xuhong Li Xiaofei Zhang Jiamin Chen Xinhao Sun Yuchen Li Zeyi Sun Mengnan Du XAI 40 8 0 09 Jan 2024
Dataset Difficulty and the Role of Inductive Bias Devin Kwok Nikhil Anand Jonathan Frankle Gintare Karolina Dziugaite David Rolnick 40 5 0 03 Jan 2024
ReliCD: A Reliable Cognitive Diagnosis Framework with Confidence Awareness Yunfei Zhang Chuan Qin Dazhong Shen Haiping Ma Le Zhang Xingyi Zhang Hengshu Zhu 17 5 0 29 Dec 2023
M3D: Dataset Condensation by Minimizing Maximum Mean Discrepancy Hansong Zhang Shikun Li Pengju Wang Dan Zeng Shiming Ge DD 19 21 0 26 Dec 2023
Mixture Data for Training Cannot Ensure Out-of-distribution Generalization Songming Zhang Yuxiao Luo Qizhou Wang Haoang Chi Xiaofeng Chen Bo Han Jinyan Li OODD 30 0 0 25 Dec 2023
On the Convergence of Loss and Uncertainty-based Active Learning Algorithms Daniel Haimovich Dima Karamshuk Fridolin Linder Niek Tax Milan Vojnovic 21 0 0 21 Dec 2023
Efficient Architecture Search via Bi-level Data Pruning Chongjun Tu Peng Ye Weihao Lin Hancheng Ye Chong Yu Tao Chen Baopu Li Wanli Ouyang 40 2 0 21 Dec 2023
Learning and Forgetting Unsafe Examples in Large Language Models Jiachen Zhao Zhun Deng David Madras James Zou Mengye Ren MU KELM CLL 83 16 0 20 Dec 2023
ConsistentEE: A Consistent and Hardness-Guided Early Exiting Method for Accelerating Language Models Inference Ziqian Zeng Yihuai Hong Hongliang Dai Huiping Zhuang Cen Chen 19 10 0 19 Dec 2023
Dataset Distillation via Adversarial Prediction Matching Mingyang Chen Bo Huang Junda Lu Bing Li Yi Wang Minhao Cheng Wei Wang DD 18 5 0 14 Dec 2023
Reconciling Shared versus Context-Specific Information in a Neural Network Model of Latent Causes Qihong Lu Tan Nguyen Qiong Zhang Uri Hasson Thomas L. Griffiths Jeffrey M. Zacks Samuel Gershman K. A. Norman 30 4 0 13 Dec 2023
Not All Data Matters: An End-to-End Adaptive Dataset Pruning Framework for Enhancing Model Performance and Efficiency Suorong Yang Hongchao Yang Suhan Guo Shen Furao Jian Zhao 17 2 0 09 Dec 2023
Bad Students Make Great Teachers: Active Learning Accelerates Large-Scale Visual Understanding Talfan Evans Shreya Pathak Hamza Merzic Jonathan Schwarz Ryutaro Tanno Olivier J. Hénaff 18 16 0 08 Dec 2023
Transferable Candidate Proposal with Bounded Uncertainty Kyeongryeol Go Kye-Hyeon Kim 24 0 0 07 Dec 2023
On the Diversity and Realism of Distilled Dataset: An Efficient Dataset Distillation Paradigm Peng Sun Bei Shi Daiwei Yu Tao Lin DD 17 37 0 06 Dec 2023
Influence Scores at Scale for Efficient Language Data Sampling Nikhil Anand Joshua Tan Maria Minakova TDI 32 3 0 27 Nov 2023
RISAM: Referring Image Segmentation via Mutual-Aware Attention Features Mengxi Zhang Yiming Liu Xiangjun Yin Huanjing Yue Jingyu Yang 31 0 0 27 Nov 2023
Dataset Distillation in Latent Space Yuxuan Duan Jianfu Zhang Liqing Zhang DD 45 6 0 27 Nov 2023
Class-Adaptive Sampling Policy for Efficient Continual Learning Hossein Rezaei Mohammad Sabokrou 19 0 0 27 Nov 2023
Spanning Training Progress: Temporal Dual-Depth Scoring (TDDS) for Enhanced Dataset Pruning Xin Zhang Jiawei Du Yunsong Li Weiying Xie Joey Tianyi Zhou 37 7 0 22 Nov 2023
Bayesian Neural Networks: A Min-Max Game Framework Junping Hong E. Kuruoglu 25 0 0 18 Nov 2023
Refined Coreset Selection: Towards Minimal Coreset Size under Model Performance Constraints Xiaobo Xia Jiale Liu Shaokun Zhang Qingyun Wu Hongxin Wei Tongliang Liu 42 9 0 15 Nov 2023
Embarassingly Simple Dataset Distillation Yunzhen Feng Ramakrishna Vedantam Julia Kempe DD 36 5 0 13 Nov 2023
Memorisation Cartography: Mapping out the Memorisation-Generalisation Continuum in Neural Machine Translation Verna Dankers Ivan Titov Dieuwke Hupkes 35 5 0 09 Nov 2023
SoK: Memorisation in machine learning Dmitrii Usynin Moritz Knolle Georgios Kaissis 17 1 0 06 Nov 2023
A Simple and Efficient Baseline for Data Attribution on Images Vasu Singla Pedro Sandoval-Segura Micah Goldblum Jonas Geiping Tom Goldstein FAtt 30 3 0 03 Nov 2023
Data-Centric Long-Tailed Image Recognition Yanbiao Ma Licheng Jiao Fang Liu Shuyuan Yang Xu Liu Puhua Chen 40 1 0 03 Nov 2023
Robust Data Pruning under Label Noise via Maximizing Re-labeling Accuracy Dongmin Park Seola Choi Doyoung Kim Hwanjun Song Jae-Gil Lee NoLa 60 20 0 02 Nov 2023
The Memory Perturbation Equation: Understanding Model's Sensitivity to Data Peter Nickl Lu Xu Dharmesh Tailor Thomas Möllenhoff Mohammad Emtiyaz Khan 24 10 0 30 Oct 2023
TRIAGE: Characterizing and auditing training data for improved regression Nabeel Seedat Jonathan Crabbé Zhaozhi Qian M. Schaar 13 5 0 29 Oct 2023
Data Optimization in Deep Learning: A Survey Ou Wu Rujing Yao 32 1 0 25 Oct 2023
Data Pruning via Moving-one-Sample-out Haoru Tan Sitong Wu Fei Du Yukang Chen Zhibin Wang Fan Wang Xiaojuan Qi 47 31 0 23 Oct 2023
You Only Condense Once: Two Rules for Pruning Condensed Datasets Yang He Lingao Xiao Joey Tianyi Zhou 37 14 0 21 Oct 2023
Harnessing Dataset Cartography for Improved Compositional Generalization in Transformers Osman Batur .Ince Tanin Zeraati Semih Yagcioglu Yadollah Yaghoobzadeh Erkut Erdem Aykut Erdem 19 1 0 18 Oct 2023
ASP: Automatic Selection of Proxy dataset for efficient AutoML Peng Yao Chao Liao Jiyuan Jia Jianchao Tan Bin Chen Chengru Song Di Zhang 13 2 0 17 Oct 2023
Relearning Forgotten Knowledge: on Forgetting, Overfit and Training-Free Ensembles of DNNs Uri Stern D. Weinshall CLL 21 0 0 17 Oct 2023
IDEAL: Influence-Driven Selective Annotations Empower In-Context Learners in Large Language Models Shaokun Zhang Xiaobo Xia Zhaoqing Wang Ling-Hao Chen Jiale Liu Qingyun Wu Tongliang Liu 36 20 0 16 Oct 2023
AST: Effective Dataset Distillation through Alignment with Smooth and High-Quality Expert Trajectories Jiyuan Shen Wenzhuo Yang Kwok-Yan Lam DD 29 1 0 16 Oct 2023
KAKURENBO: Adaptively Hiding Samples in Deep Neural Network Training Truong Thao Nguyen Balazs Gerofi Edgar Josafat Martinez-Noriega Franccois Trahay M. Wahib 24 1 0 16 Oct 2023
Farzi Data: Autoregressive Data Distillation Noveen Sachdeva Zexue He Wang-Cheng Kang Jianmo Ni D. Cheng Julian McAuley DD 19 3 0 15 Oct 2023