Does Learning Require Memorization? A Short Tale about a Long Tail

12 June 2019

Papers citing "Does Learning Require Memorization? A Short Tale about a Long Tail"

50 / 104 papers shown

Title
Skellam Mixture Mechanism: a Novel Approach to Federated Learning with Differential Privacy Ergute Bao Yizheng Zhu X. Xiao Y. Yang Beng Chin Ooi B. Tan Khin Mi Mi Aung FedML 28 18 0 08 Dec 2022
CrossSplit: Mitigating Label Noise Memorization through Data Splitting Jihye Kim A. Baratin Yan Zhang Simon Lacoste-Julien NoLa 18 7 0 03 Dec 2022
Prototypical Fine-tuning: Towards Robust Performance Under Varying Data Sizes Yiqiao Jin Xiting Wang Y. Hao Yizhou Sun Xing Xie 35 11 0 24 Nov 2022
SA-DPSGD: Differentially Private Stochastic Gradient Descent based on Simulated Annealing Jie Fu Zhili Chen Xinpeng Ling 19 0 0 14 Nov 2022
Finding Memo: Extractive Memorization in Constrained Sequence Generation Tasks Vikas Raunak Arul Menezes 30 13 0 24 Oct 2022
Improving Data Quality with Training Dynamics of Gradient Boosting Decision Trees M. Ponti L. Oliveira Mathias Esteban Valentina Garcia J. Román Luis Argerich TDI 22 4 0 20 Oct 2022
Verifiable and Provably Secure Machine Unlearning Thorsten Eisenhofer Doreen Riepel Varun Chandrasekaran Esha Ghosh O. Ohrimenko Nicolas Papernot AAML MU 33 26 0 17 Oct 2022
Mitigating Unintended Memorization in Language Models via Alternating Teaching Zhe Liu Xuedong Zhang Fuchun Peng 17 3 0 13 Oct 2022
Transformers generalize differently from information stored in context vs in weights Stephanie C. Y. Chan Ishita Dasgupta Junkyung Kim D. Kumaran Andrew Kyle Lampinen Felix Hill 104 45 0 11 Oct 2022
Understanding Transformer Memorization Recall Through Idioms Adi Haviv Ido Cohen Jacob Gidron R. Schuster Yoav Goldberg Mor Geva 26 48 0 07 Oct 2022
On the Impossible Safety of Large AI Models El-Mahdi El-Mhamdi Sadegh Farhadkhani R. Guerraoui Nirupam Gupta L. Hoang Rafael Pinot Sébastien Rouault John Stephan 30 31 0 30 Sep 2022
Metadata Archaeology: Unearthing Data Subsets by Leveraging Training Dynamics Shoaib Ahmed Siddiqui Nitarshan Rajkumar Tegan Maharaj David M. Krueger Sara Hooker 37 27 0 20 Sep 2022
Data Provenance via Differential Auditing Xin Mu Ming Pang Feida Zhu 6 1 0 04 Sep 2022
Data Isotopes for Data Provenance in DNNs Emily Wenger Xiuyu Li Ben Y. Zhao Vitaly Shmatikov 18 12 0 29 Aug 2022
SNAP: Efficient Extraction of Private Properties with Poisoning Harsh Chaudhari John Abascal Alina Oprea Matthew Jagielski Florian Tramèr Jonathan R. Ullman MIACV 34 30 0 25 Aug 2022
Efficient Augmentation for Imbalanced Deep Learning Damien Dablain C. Bellinger Bartosz Krawczyk Nitesh V. Chawla 27 7 0 13 Jul 2022
Generalization-Memorization Machines Zhen Wang Y. Shao 14 6 0 08 Jul 2022
Distilling Model Failures as Directions in Latent Space Saachi Jain Hannah Lawrence Ankur Moitra A. Madry 18 89 0 29 Jun 2022
The Privacy Onion Effect: Memorization is Relative Nicholas Carlini Matthew Jagielski Chiyuan Zhang Nicolas Papernot Andreas Terzis Florian Tramèr PILM MIACV 30 99 0 21 Jun 2022
Reconstructing Training Data from Trained Neural Networks Niv Haim Gal Vardi Gilad Yehudai Ohad Shamir Michal Irani 40 132 0 15 Jun 2022
Neural Collapse: A Review on Modelling Principles and Generalization Vignesh Kothapalli 21 71 0 08 Jun 2022
Memorization Without Overfitting: Analyzing the Training Dynamics of Large Language Models Kushal Tirumala Aram H. Markosyan Luke Zettlemoyer Armen Aghajanyan TDI 23 185 0 22 May 2022
Provably Confidential Language Modelling Xuandong Zhao Lei Li Yu-Xiang Wang MU 14 15 0 04 May 2022
Memory Bounds for Continual Learning Xi Chen Christos H. Papadimitriou Binghui Peng CLL LRM 27 22 0 22 Apr 2022
Truth Serum: Poisoning Machine Learning Models to Reveal Their Secrets Florian Tramèr Reza Shokri Ayrton San Joaquin Hoang Minh Le Matthew Jagielski Sanghyun Hong Nicholas Carlini MIACV 30 106 0 31 Mar 2022
Does Label Differential Privacy Prevent Label Inference Attacks? Ruihan Wu Jinfu Zhou Kilian Q. Weinberger Chuan Guo 15 15 0 25 Feb 2022
Deletion Inference, Reconstruction, and Compliance in Machine (Un)Learning Ji Gao Sanjam Garg Mohammad Mahmoody Prashant Nalini Vasudevan MIACV AAML 19 22 0 07 Feb 2022
Evaluating natural language processing models with generalization metrics that do not need access to any training or testing data Yaoqing Yang Ryan Theisen Liam Hodgkinson Joseph E. Gonzalez Kannan Ramchandran Charles H. Martin Michael W. Mahoney 86 17 0 06 Feb 2022
Datamodels: Predicting Predictions from Training Data Andrew Ilyas Sung Min Park Logan Engstrom Guillaume Leclerc A. Madry TDI 35 130 0 01 Feb 2022
Reconstructing Training Data with Informed Adversaries Borja Balle Giovanni Cherubin Jamie Hayes MIACV AAML 30 158 0 13 Jan 2022
AutoBalance: Optimized Loss Functions for Imbalanced Data Mingchen Li Xuechen Zhang Christos Thrampoulidis Jiasi Chen Samet Oymak 14 67 0 04 Jan 2022
Learning Theory Can (Sometimes) Explain Generalisation in Graph Neural Networks P. Esser L. C. Vankadara D. Ghoshdastidar 28 53 0 07 Dec 2021
Membership Inference Attacks From First Principles Nicholas Carlini Steve Chien Milad Nasr Shuang Song Andreas Terzis Florian Tramèr MIACV MIALM 22 639 0 07 Dec 2021
Scaling Up Influence Functions Andrea Schioppa Polina Zablotskaia David Vilar Artem Sokolov TDI 25 90 0 06 Dec 2021
On the Convergence of Shallow Neural Network Training with Randomly Masked Neurons Fangshuo Liao Anastasios Kyrillidis 36 16 0 05 Dec 2021
SHAPr: An Efficient and Versatile Membership Privacy Risk Metric for Machine Learning Vasisht Duddu S. Szyller Nadarajah Asokan 24 12 0 04 Dec 2021
Differentially Private Fine-tuning of Language Models Da Yu Saurabh Naik A. Backurs Sivakanth Gopi Huseyin A. Inan ... Y. Lee Andre Manoel Lukas Wutschitz Sergey Yekhanin Huishuai Zhang 134 346 0 13 Oct 2021
Not all noise is accounted equally: How differentially private learning benefits from large sampling rates Friedrich Dörmann Osvald Frisk L. Andersen Christian Fischer Pedersen FedML 54 25 0 12 Oct 2021
Robin Hood and Matthew Effects: Differential Privacy Has Disparate Impact on Synthetic Data Georgi Ganev Bristena Oprisanu Emiliano De Cristofaro 37 57 0 23 Sep 2021
SoK: Machine Learning Governance Varun Chandrasekaran Hengrui Jia Anvith Thudi Adelin Travers Mohammad Yaghini Nicolas Papernot 32 16 0 20 Sep 2021
ComSum: Commit Messages Summarization and Meaning Preservation Leshem Choshen Idan Amit 17 4 0 23 Aug 2021
Large-Scale Differentially Private BERT Rohan Anil Badih Ghazi Vineet Gupta Ravi Kumar Pasin Manurangsi 27 131 0 03 Aug 2021
Memorization in Deep Neural Networks: Does the Loss Function matter? Deep Patel P. Sastry TDI 16 8 0 21 Jul 2021
Covariance-Aware Private Mean Estimation Without Private Covariance Estimation Gavin Brown Marco Gaboardi Adam D. Smith Jonathan R. Ullman Lydia Zakynthinou FedML 28 48 0 24 Jun 2021
Membership Inference on Word Embedding and Beyond Saeed Mahloujifar Huseyin A. Inan Melissa Chase Esha Ghosh Marcello Hasegawa MIACV SILM 19 46 0 21 Jun 2021
The Curious Case of Hallucinations in Neural Machine Translation Vikas Raunak Arul Menezes Marcin Junczys-Dowmunt 13 189 0 14 Apr 2021
Clusterability as an Alternative to Anchor Points When Learning with Noisy Labels Zhaowei Zhu Yiwen Song Yang Liu NoLa 13 91 0 10 Feb 2021
Fast and Memory Efficient Differentially Private-SGD via JL Projections Zhiqi Bu Sivakanth Gopi Janardhan Kulkarni Y. Lee J. Shen U. Tantipongpipat FedML 24 41 0 05 Feb 2021
Modifying Memories in Transformer Models Chen Zhu A. S. Rawat Manzil Zaheer Srinadh Bhojanapalli Daliang Li Felix X. Yu Sanjiv Kumar KELM 13 190 0 01 Dec 2020
On the Privacy Risks of Algorithmic Fairness Hong Chang Reza Shokri FaML 16 109 0 07 Nov 2020