Training Production Language Models without Memorizing User Data

21 September 2020

Swaroop Indra Ramaswamy

Papers citing "Training Production Language Models without Memorizing User Data"

30 / 30 papers shown

Title
Efficient Full-Stack Private Federated Deep Learning with Post-Quantum Security Yiwei Zhang R. Behnia A. Yavuz Reza Ebrahimi E. Bertino FedML 45 0 0 09 May 2025
Controlled privacy leakage propagation throughout overlapping grouped learning Shahrzad Kiani Franziska Boenisch S. Draper FedML 72 0 0 06 Mar 2025
PAPILLON: Privacy Preservation from Internet-based and Local Language Model Ensembles Li Siyan Vethavikashini Chithrra Raghuram Omar Khattab Julia Hirschberg Zhou Yu 29 9 0 22 Oct 2024
Undesirable Memorization in Large Language Models: A Survey Ali Satvaty Suzan Verberne Fatih Turkmen ELM PILM 86 7 0 03 Oct 2024
REVS: Unlearning Sensitive Information in Language Models via Rank Editing in the Vocabulary Space Tomer Ashuach Martin Tutek Yonatan Belinkov KELM MU 71 4 0 13 Jun 2024
Delving into Differentially Private Transformer Youlong Ding Xueyang Wu Yining Meng Yonggang Luo Hao Wang Weike Pan 39 5 0 28 May 2024
RAIFLE: Reconstruction Attacks on Interaction-based Federated Learning with Adversarial Data Manipulation Dzung Pham Shreyas Kulkarni Amir Houmansadr 33 0 0 29 Oct 2023
Forgetting Private Textual Sequences in Language Models via Leave-One-Out Ensemble Zhe Liu Ozlem Kalinli MU KELM 28 2 0 28 Sep 2023
A Note On Interpreting Canary Exposure Matthew Jagielski 20 4 0 31 May 2023
Training Data Extraction From Pre-trained Language Models: A Survey Shotaro Ishihara 32 46 0 25 May 2023
Can Public Large Language Models Help Private Cross-device Federated Learning? Wei Ping Yibo Jacky Zhang Yuan Cao Bo-wen Li H. B. McMahan Sewoong Oh Zheng Xu Manzil Zaheer FedML 29 37 0 20 May 2023
Personalized Privacy-Preserving Framework for Cross-Silo Federated Learning Van Tuan Tran Huy Hieu Pham Kok-Seng Wong FedML 39 7 0 22 Feb 2023
Context-Aware Differential Privacy for Language Modeling M. H. Dinh Ferdinando Fioretto 30 2 0 28 Jan 2023
Differentially Private Natural Language Models: Recent Advances and Future Directions Lijie Hu Ivan Habernal Lei Shen Di Wang AAML 35 18 0 22 Jan 2023
Learning to Generate Image Embeddings with User-level Differential Privacy Zheng Xu Maxwell D. Collins Yuxiao Wang Liviu Panait Sewoong Oh S. Augenstein Ting Liu Florian Schroff H. B. McMahan FedML 30 29 0 20 Nov 2022
Mitigating Unintended Memorization in Language Models via Alternating Teaching Zhe Liu Xuedong Zhang Fuchun Peng 38 3 0 13 Oct 2022
CANIFE: Crafting Canaries for Empirical Privacy Measurement in Federated Learning Samuel Maddock Alexandre Sablayrolles Pierre Stock FedML 20 22 0 06 Oct 2022
Dordis: Efficient Federated Learning with Dropout-Resilient Differential Privacy Zhifeng Jiang Wei Wang Ruichuan Chen 43 7 0 26 Sep 2022
Training Large-Vocabulary Neural Language Models by Private Federated Learning for Resource-Constrained Devices Mingbin Xu Congzheng Song Ye Tian Neha Agrawal Filip Granqvist ... Shiyi Han Yaqiao Deng Leo Liu Anmol Walia Alex Jin FedML 15 22 0 18 Jul 2022
Using Autoencoders on Differentially Private Federated Learning GANs Gregor Schram Rui Wang K. Liang FedML AI4CE 19 1 0 24 Jun 2022
On Privacy and Personalization in Cross-Silo Federated Learning Ziyu Liu Shengyuan Hu Zhiwei Steven Wu Virginia Smith FedML 22 53 0 16 Jun 2022
Truth Serum: Poisoning Machine Learning Models to Reveal Their Secrets Florian Tramèr Reza Shokri Ayrton San Joaquin Hoang Minh Le Matthew Jagielski Sanghyun Hong Nicholas Carlini MIACV 38 107 0 31 Mar 2022
Improved Differential Privacy for SGD via Optimal Private Linear Operators on Adaptive Streams S. Denisov H. B. McMahan J. Rush Adam D. Smith Abhradeep Thakurta FedML 33 60 0 16 Feb 2022
What Does it Mean for a Language Model to Preserve Privacy? Hannah Brown Katherine Lee Fatemehsadat Mireshghallah Reza Shokri Florian Tramèr PILM 52 232 0 11 Feb 2022
Resource-Efficient Federated Learning A. Abdelmoniem Atal Narayan Sahu Marco Canini Suhaib A. Fahmy FedML 32 55 0 01 Nov 2021
Differentially Private Fine-tuning of Language Models Da Yu Saurabh Naik A. Backurs Sivakanth Gopi Huseyin A. Inan ... Y. Lee Andre Manoel Lukas Wutschitz Sergey Yekhanin Huishuai Zhang 134 350 0 13 Oct 2021
The Skellam Mechanism for Differentially Private Federated Learning Naman Agarwal Peter Kairouz Ziyu Liu FedML 22 122 0 11 Oct 2021
Efficient and Private Federated Learning with Partially Trainable Networks Hakim Sidahmed Zheng Xu Ankush Garg Yuan Cao Mingqing Chen FedML 49 13 0 06 Oct 2021
Selective Differential Privacy for Language Modeling Weiyan Shi Aiqi Cui Evan Li R. Jia Zhou Yu 20 68 0 30 Aug 2021
Privacy Regularization: Joint Privacy-Utility Optimization in Language Models Fatemehsadat Mireshghallah Huseyin A. Inan Marcello Hasegawa Victor Rühle Taylor Berg-Kirkpatrick Robert Sim 19 40 0 12 Mar 2021