What Neural Networks Memorize and Why: Discovering the Long Tail via Influence Estimation

9 August 2020

Papers citing "What Neural Networks Memorize and Why: Discovering the Long Tail via Influence Estimation"

36 / 86 papers shown

Title
ModelDiff: A Framework for Comparing Learning Algorithms Harshay Shah Sung Min Park Andrew Ilyas A. Madry SyDa 51 26 0 22 Nov 2022
The Curious Case of Benign Memorization Sotiris Anagnostidis Gregor Bachmann Lorenzo Noci Thomas Hofmann AAML 43 8 0 25 Oct 2022
Finding Memo: Extractive Memorization in Constrained Sequence Generation Tasks Vikas Raunak Arul Menezes 30 13 0 24 Oct 2022
Bird-Eye Transformers for Text Generation Models Lei Sha Yuhang Song Yordan Yordanov Tommaso Salvatori Thomas Lukasiewicz 17 0 0 08 Oct 2022
Understanding Transformer Memorization Recall Through Idioms Adi Haviv Ido Cohen Jacob Gidron R. Schuster Yoav Goldberg Mor Geva 26 48 0 07 Oct 2022
Metadata Archaeology: Unearthing Data Subsets by Leveraging Training Dynamics Shoaib Ahmed Siddiqui Nitarshan Rajkumar Tegan Maharaj David M. Krueger Sara Hooker 37 27 0 20 Sep 2022
Lazy vs hasty: linearization in deep networks impacts learning schedule based on example difficulty Thomas George Guillaume Lajoie A. Baratin 23 5 0 19 Sep 2022
Data Isotopes for Data Provenance in DNNs Emily Wenger Xiuyu Li Ben Y. Zhao Vitaly Shmatikov 18 12 0 29 Aug 2022
The Privacy Onion Effect: Memorization is Relative Nicholas Carlini Matthew Jagielski Chiyuan Zhang Nicolas Papernot Andreas Terzis Florian Tramèr PILM MIACV 30 99 0 21 Jun 2022
Selective Classification Via Neural Network Training Dynamics Stephan Rabanser Anvith Thudi Kimia Hamidieh Adam Dziedzic Nicolas Papernot 18 21 0 26 May 2022
Memorization Without Overfitting: Analyzing the Training Dynamics of Large Language Models Kushal Tirumala Aram H. Markosyan Luke Zettlemoyer Armen Aghajanyan TDI 23 185 0 22 May 2022
Adapting and Evaluating Influence-Estimation Methods for Gradient-Boosted Decision Trees Jonathan Brophy Zayd Hammoudeh Daniel Lowd TDI 19 22 0 30 Apr 2022
Knowledge Distillation as Efficient Pre-training: Faster Convergence, Higher Data-efficiency, and Better Transferability Ruifei He Shuyang Sun Jihan Yang Song Bai Xiaojuan Qi 24 36 0 10 Mar 2022
Deduplicating Training Data Mitigates Privacy Risks in Language Models Nikhil Kandpal Eric Wallace Colin Raffel PILM MU 28 274 0 14 Feb 2022
Understanding Rare Spurious Correlations in Neural Networks Yao-Yuan Yang Chi-Ning Chou Kamalika Chaudhuri AAML 16 25 0 10 Feb 2022
Datamodels: Predicting Predictions from Training Data Andrew Ilyas Sung Min Park Logan Engstrom Guillaume Leclerc A. Madry TDI 35 130 0 01 Feb 2022
Fortuitous Forgetting in Connectionist Networks Hattie Zhou Ankit Vani Hugo Larochelle Aaron Courville CLL 6 42 0 01 Feb 2022
Identifying a Training-Set Attack's Target Using Renormalized Influence Estimation Zayd Hammoudeh Daniel Lowd TDI 18 28 0 25 Jan 2022
Towards Adversarial Evaluations for Inexact Machine Unlearning Shashwat Goel Ameya Prabhu Amartya Sanyal Ser-Nam Lim Philip H. S. Torr Ponnurangam Kumaraguru AAML ELM MU 26 45 0 17 Jan 2022
Reconstructing Training Data with Informed Adversaries Borja Balle Giovanni Cherubin Jamie Hayes MIACV AAML 32 158 0 13 Jan 2022
Membership Inference Attacks From First Principles Nicholas Carlini Steve Chien Milad Nasr Shuang Song Andreas Terzis Florian Tramèr MIACV MIALM 24 639 0 07 Dec 2021
Scaling Up Influence Functions Andrea Schioppa Polina Zablotskaia David Vilar Artem Sokolov TDI 25 90 0 06 Dec 2021
SHAPr: An Efficient and Versatile Membership Privacy Risk Metric for Machine Learning Vasisht Duddu S. Szyller Nadarajah Asokan 24 12 0 04 Dec 2021
ModelPred: A Framework for Predicting Trained Model from Training Data Yingyan Zeng Jiachen T. Wang Si-An Chen H. Just Ran Jin R. Jia TDI MU 25 2 0 24 Nov 2021
SoK: Machine Learning Governance Varun Chandrasekaran Hengrui Jia Anvith Thudi Adelin Travers Mohammad Yaghini Nicolas Papernot 32 16 0 20 Sep 2021
Associative Memories via Predictive Coding Tommaso Salvatori Yuhang Song Yujian Hong Simon Frieder Lei Sha Zhenghua Xu Rafal Bogacz Thomas Lukasiewicz 21 61 0 16 Sep 2021
Pointer Value Retrieval: A new benchmark for understanding the limits of neural network generalization Chiyuan Zhang M. Raghu Jon M. Kleinberg Samy Bengio OOD 24 30 0 27 Jul 2021
Memorization in Deep Neural Networks: Does the Loss Function matter? Deep Patel P. Sastry TDI 19 8 0 21 Jul 2021
Rethinking Graph Auto-Encoder Models for Attributed Graph Clustering Nairouz Mrabah Mohamed Bouguessa M. Touati Riadh Ksantini 28 62 0 19 Jul 2021
Deduplicating Training Data Makes Language Models Better Katherine Lee Daphne Ippolito A. Nystrom Chiyuan Zhang Douglas Eck Chris Callison-Burch Nicholas Carlini SyDa 242 591 0 14 Jul 2021
Deep Learning Through the Lens of Example Difficulty R. Baldock Hartmut Maennel Behnam Neyshabur 28 155 0 17 Jun 2021
The Curious Case of Hallucinations in Neural Machine Translation Vikas Raunak Arul Menezes Marcin Junczys-Dowmunt 13 189 0 14 Apr 2021
FastIF: Scalable Influence Functions for Efficient Model Interpretation and Debugging Han Guo Nazneen Rajani Peter Hase Mohit Bansal Caiming Xiong TDI 33 102 0 31 Dec 2020
Efficient Estimation of Influence of a Training Instance Sosuke Kobayashi Sho Yokoi Jun Suzuki Kentaro Inui TDI 27 15 0 08 Dec 2020
Modifying Memories in Transformer Models Chen Zhu A. S. Rawat Manzil Zaheer Srinadh Bhojanapalli Daliang Li Felix X. Yu Sanjiv Kumar KELM 13 190 0 01 Dec 2020
How benign is benign overfitting? Amartya Sanyal P. Dokania Varun Kanade Philip H. S. Torr NoLa AAML 23 57 0 08 Jul 2020