An Empirical Study of Example Forgetting during Deep Neural Network Learning

12 December 2018

Mariya Toneva

Alessandro Sordoni

Rémi Tachet des Combes

Papers citing "An Empirical Study of Example Forgetting during Deep Neural Network Learning"

50 / 482 papers shown

Title
LOPS: Learning Order Inspired Pseudo-Label Selection for Weakly Supervised Text Classification Dheeraj Mekala Chengyu Dong Jingbo Shang 6 20 0 25 May 2022
Memorization Without Overfitting: Analyzing the Training Dynamics of Large Language Models Kushal Tirumala Aram H. Markosyan Luke Zettlemoyer Armen Aghajanyan TDI 26 185 0 22 May 2022
Unintended memorisation of unique features in neural networks J. Hartley Sotirios A. Tsaftaris 30 1 0 20 May 2022
Dataset Pruning: Reducing Training Data by Examining Generalization Influence Shuo Yang Zeke Xie Hanyu Peng Minjing Xu Mingming Sun P. Li DD 149 106 0 19 May 2022
Exploring the Learning Difficulty of Data Theory and Measure Weiyao Zhu Ou Wu Fengguang Su Yingjun Deng 27 5 0 16 May 2022
ELODI: Ensemble Logit Difference Inhibition for Positive-Congruent Training Yue Zhao Yantao Shen Yuanjun Xiong Shuo Yang Wei Xia Z. Tu Bernt Shiele Stefano Soatto BDL 38 6 0 12 May 2022
Task-specific Compression for Multi-task Language Models using Attribution-based Pruning Nakyeong Yang Yunah Jang Hwanhee Lee Seohyeong Jung Kyomin Jung 13 8 0 09 May 2022
Learning from Pixel-Level Noisy Label : A New Perspective for Light Field Saliency Detection Mingtao Feng Li-Yu Daisy Liu Liangkai Zhang Hongshan Yu Yaonan Wang Ajmal Saeed Mian 19 18 0 28 Apr 2022
Data-Efficient Backdoor Attacks Pengfei Xia Ziqiang Li Wei Zhang Bin Li AAML FedML 11 31 0 22 Apr 2022
DeepCore: A Comprehensive Library for Coreset Selection in Deep Learning Chengcheng Guo B. Zhao Yanbing Bai OOD 31 131 0 18 Apr 2022
A Comprehensive Survey on Data-Efficient GANs in Image Generation Ziqiang Li Beihao Xia Jing Zhang Chaoyue Wang Bin Li 19 33 0 18 Apr 2022
Q-TART: Quickly Training for Adversarial Robustness and in-Transferability Madan Ravi Ganesh S. Y. Sekeh Jason J. Corso AAML 21 0 0 14 Apr 2022
Few-shot Learning with Noisy Labels Kevin J Liang Samrudhdhi B. Rangrej Vladan Petrovic Tal Hassner NoLa 19 47 0 12 Apr 2022
Understanding out-of-distribution accuracies through quantifying difficulty of test samples Berfin Simsek Melissa Hall Levent Sagun 23 5 0 28 Mar 2022
An Empirical Study of Memorization in NLP Xiaosen Zheng Jing Jiang TDI 17 24 1 23 Mar 2022
Dataset Distillation by Matching Training Trajectories George Cazenavette Tongzhou Wang Antonio Torralba Alexei A. Efros Jun-Yan Zhu FedML DD 58 363 0 22 Mar 2022
Representative Subset Selection for Efficient Fine-Tuning in Self-Supervised Speech Recognition Abdul Hameed Azeemi I. Qazi Agha Ali Raza 21 0 0 18 Mar 2022
Layer Ensembles: A Single-Pass Uncertainty Estimation in Deep Learning for Segmentation Kaisar Kushibar Víctor M. Campello Lidia Garrucho Moras Akis Linardos P. Radeva Karim Lekadir UQCV 11 18 0 16 Mar 2022
Reducing Flipping Errors in Deep Neural Networks Xiang Deng Yun Xiao Bo Long Zhongfei Zhang AAML 30 3 0 16 Mar 2022
Feeding What You Need by Understanding What You Learned Xiaoqiang Wang Bang Liu Fangli Xu Bowei Long Siliang Tang Lingfei Wu 57 6 0 05 Mar 2022
CAFE: Learning to Condense Dataset by Aligning Features Kai Wang Bo-Lu Zhao Xiangyu Peng Zheng Zhu Shuo Yang Shuo Wang Guan Huang Hakan Bilen Xinchao Wang Yang You DD 29 0 0 03 Mar 2022
Adaptive Discriminative Regularization for Visual Classification Qingsong Zhao Yi Wang Shuguang Dou Chen Gong Yin Wang Cairong Zhao 18 0 0 02 Mar 2022
Deconstructing Distributions: A Pointwise Framework of Learning Gal Kaplun Nikhil Ghosh Saurabh Garg Boaz Barak Preetum Nakkiran OOD 25 21 0 20 Feb 2022
Measuring Unintended Memorisation of Unique Private Features in Neural Networks J. Hartley Sotirios A. Tsaftaris 19 7 0 16 Feb 2022
Predicting on the Edge: Identifying Where a Larger Model Does Better Taman Narayan Heinrich Jiang Sen Zhao Surinder Kumar 23 7 0 15 Feb 2022
A Lagrangian Duality Approach to Active Learning Juan Elenter Navid Naderializadeh Alejandro Ribeiro 22 21 0 08 Feb 2022
Measuring and Reducing Model Update Regression in Structured Prediction for NLP Deng Cai Elman Mansimov Yi-An Lai Yixuan Su Lei Shu Yi Zhang KELM 67 8 0 07 Feb 2022
FORML: Learning to Reweight Data for Fairness Bobby Yan Skyler Seto N. Apostoloff FaML 23 11 0 03 Feb 2022
Datamodels: Predicting Predictions from Training Data Andrew Ilyas Sung Min Park Logan Engstrom Guillaume Leclerc A. Madry TDI 38 130 0 01 Feb 2022
TrustAL: Trustworthy Active Learning using Knowledge Distillation Beong-woo Kwak Youngwook Kim Yu Jin Kim Seung-won Hwang Jinyoung Yeo 6 7 0 26 Jan 2022
On Sampling Collaborative Filtering Datasets Noveen Sachdeva Carole-Jean Wu Julian McAuley 15 16 0 13 Jan 2022
Memory-Guided Semantic Learning Network for Temporal Sentence Grounding Daizong Liu Xiaoye Qu Xing Di Yu Cheng Zichuan Xu Pan Zhou 25 58 0 03 Jan 2022
Continual Learning for Unsupervised Anomaly Detection in Continuous Auditing of Financial Accounting Data Hamed Hemati Marco Schreyer Damian Borth 18 9 0 25 Dec 2021
On the Impact of Hard Adversarial Instances on Overfitting in Adversarial Training Chen Liu Zhichao Huang Mathieu Salzmann Tong Zhang Sabine Süsstrunk AAML 15 13 0 14 Dec 2021
On the Convergence of Shallow Neural Network Training with Randomly Masked Neurons Fangshuo Liao Anastasios Kyrillidis 36 16 0 05 Dec 2021
Hard Sample Aware Noise Robust Learning for Histopathology Image Classification Chuang Zhu Wenkai Chen T. Peng Ying Wang M. Jin NoLa 26 71 0 05 Dec 2021
Sample Prior Guided Robust Model Learning to Suppress Noisy Labels Wenkai Chen Chuang Zhu Yi Chen Mengting Li Tiejun Huang NoLa 11 11 0 02 Dec 2021
Accelerating Deep Learning with Dynamic Data Pruning Ravi Raju Kyle Daruwalla Mikko H. Lipasti 4 31 0 24 Nov 2021
DataCLUE: A Benchmark Suite for Data-centric NLP Liang Xu Jiacheng Liu X. Pan Xiaojing Lu Xiaofeng Hou ELM 6 16 0 16 Nov 2021
On the Importance of Difficulty Calibration in Membership Inference Attacks Lauren Watson Chuan Guo Graham Cormode Alex Sablayrolles 23 118 0 15 Nov 2021
MERCURY: Accelerating DNN Training By Exploiting Input Similarity Vahid Janfaza Kevin Weston Moein Razavi Shantanu Mandal Farabi Mahmud Alex Hilty A. Muzahid 39 5 0 28 Oct 2021
How Important is Importance Sampling for Deep Budgeted Training? Eric Arazo Diego Ortego Paul Albert Noel E. O'Connor Kevin McGuinness 13 7 0 27 Oct 2021
MEST: Accurate and Fast Memory-Economic Sparse Training Framework on the Edge Geng Yuan Xiaolong Ma Wei Niu Zhengang Li Zhenglun Kong ... Minghai Qin Bin Ren Yanzhi Wang Sijia Liu Xue Lin 15 89 0 26 Oct 2021
Addressing out-of-distribution label noise in webly-labelled data Paul Albert Diego Ortego Eric Arazo Noel E. O'Connor Kevin McGuinness NoLa 16 16 0 26 Oct 2021
Wide Neural Networks Forget Less Catastrophically Seyed Iman Mirzadeh Arslan Chaudhry Dong Yin Huiyi Hu Razvan Pascanu Dilan Görür Mehrdad Farajtabar CLL 25 60 0 21 Oct 2021
On Hard Episodes in Meta-Learning S. Basu Amr Sharaf Nicolò Fusi S. Feizi 8 1 0 21 Oct 2021
Trivial or impossible -- dichotomous data difficulty masks model differences (on ImageNet and beyond) Kristof Meding Luca M. Schulze Buschoff Robert Geirhos Felix Wichmann 28 40 0 12 Oct 2021
Dataset Condensation with Distribution Matching Bo-Lu Zhao Hakan Bilen DD 18 282 0 08 Oct 2021
Data Summarization via Bilevel Optimization Zalan Borsos Mojmír Mutný Marco Tagliasacchi Andreas Krause 30 8 0 26 Sep 2021
Dynamic Knowledge Distillation for Pre-trained Language Models Lei Li Yankai Lin Shuhuai Ren Peng Li Jie Zhou Xu Sun 20 49 0 23 Sep 2021