Large Language Models Can Be Strong Differentially Private Learners

12 October 2021

Tatsunori Hashimoto

Papers citing "Large Language Models Can Be Strong Differentially Private Learners"

50 / 59 papers shown

Title
NoEsis: Differentially Private Knowledge Transfer in Modular LLM Adaptation Rob Romijnders Stefanos Laskaridis Ali Shahin Shamsabadi Hamed Haddadi 57 0 0 25 Apr 2025
How Private is Your Attention? Bridging Privacy with In-Context Learning Soham Bonnerjee Zhen Wei Yeon Anna Asch Sagnik Nandy Promit Ghosal 40 0 0 22 Apr 2025
Protecting Users From Themselves: Safeguarding Contextual Privacy in Interactions with Conversational Agents Ivoline Ngong Swanand Kadhe Hao Wang K. Murugesan Justin D. Weisz Amit Dhurandhar K. Ramamurthy 44 2 0 22 Feb 2025
Memory-Efficient Fine-Tuning of Transformers via Token Selection Antoine Simoulin Namyong Park Xiaoyi Liu Grey Yang 110 0 0 31 Jan 2025
LegalGuardian: A Privacy-Preserving Framework for Secure Integration of Large Language Models in Legal Practice M. Mikail Demir Hakan T. Otal M. A. Canbaz AILaw 31 0 0 19 Jan 2025
DP-2Stage: Adapting Language Models as Differentially Private Tabular Data Generators Tejumade Afonja Hui-Po Wang Raouf Kerkouche Mario Fritz SyDa 108 2 0 03 Dec 2024
Parameter-Efficient Fine-Tuning in Large Models: A Survey of Methodologies L. Wang Sheng Chen Linnan Jiang Shu Pan Runze Cai Sen Yang Fei Yang 44 3 0 24 Oct 2024
Data-adaptive Differentially Private Prompt Synthesis for In-Context Learning Fengyu Gao Ruida Zhou T. Wang Cong Shen Jing Yang 29 2 0 15 Oct 2024
DiSK: Differentially Private Optimizer with Simplified Kalman Filter for Noise Reduction Xinwei Zhang Zhiqi Bu Borja Balle Mingyi Hong Meisam Razaviyayn Vahab Mirrokni 74 2 0 04 Oct 2024
Undesirable Memorization in Large Language Models: A Survey Ali Satvaty Suzan Verberne Fatih Turkmen ELM PILM 69 7 0 03 Oct 2024
Generated Data with Fake Privacy: Hidden Dangers of Fine-tuning Large Language Models on Generated Data Atilla Akkus Mingjie Li Junjie Chu Junjie Chu Michael Backes Sinem Sav Sinem Sav SILM SyDa 35 1 0 12 Sep 2024
A Different Level Text Protection Mechanism With Differential Privacy Qingwen Fu 33 0 0 05 Sep 2024
Differentially Private Kernel Density Estimation Erzhi Liu Jerry Yao-Chieh Hu Alex Reneau Zhao Song Han Liu 61 3 0 03 Sep 2024
Forget to Flourish: Leveraging Machine-Unlearning on Pretrained Language Models for Privacy Leakage Md. Rafi Ur Rashid Jing Liu T. Koike-Akino Shagufta Mehnaz Ye Wang MU SILM 36 3 0 30 Aug 2024
Data Shapley in One Training Run Jiachen T. Wang Prateek Mittal Dawn Song Ruoxi Jia TDI 29 7 0 16 Jun 2024
REVS: Unlearning Sensitive Information in Language Models via Rank Editing in the Vocabulary Space Tomer Ashuach Martin Tutek Yonatan Belinkov KELM MU 63 4 0 13 Jun 2024
Noise-Aware Differentially Private Regression via Meta-Learning Ossi Raisa Stratis Markou Matthew Ashman W. Bruinsma Marlon Tobaben Antti Honkela Richard E. Turner 57 1 0 12 Jun 2024
Bayesian Power Steering: An Effective Approach for Domain Adaptation of Diffusion Models Ding Huang Ting Li Jian Huang DiffM 31 1 0 06 Jun 2024
DP-DyLoRA: Fine-Tuning Transformer-Based Models On-Device under Differentially Private Federated Learning using Dynamic Low-Rank Adaptation Jie Xu Karthikeyan P. Saravanan Rogier van Dalen Haaris Mehmood David Tuckey Mete Ozay 56 5 0 10 May 2024
To Each (Textual Sequence) Its Own: Improving Memorized-Data Unlearning in Large Language Models George-Octavian Barbulescu Peter Triantafillou MU 29 16 0 06 May 2024
Public-data Assisted Private Stochastic Optimization: Power and Limitations Enayat Ullah Michael Menart Raef Bassily Cristóbal Guzmán Raman Arora 30 1 0 06 Mar 2024
On the Challenges and Opportunities in Generative AI Laura Manduchi Kushagra Pandey Robert Bamler Ryan Cotterell Sina Daubener ... F. Wenzel Frank Wood Stephan Mandt Vincent Fortuin Vincent Fortuin 56 17 0 28 Feb 2024
Private Fine-tuning of Large Language Models with Zeroth-order Optimization Xinyu Tang Ashwinee Panda Milad Nasr Saeed Mahloujifar Prateek Mittal 44 18 0 09 Jan 2024
PCDP-SGD: Improving the Convergence of Differentially Private SGD via Projection in Advance Haichao Sha Ruixuan Liu Yi-xiao Liu Hong Chen 47 1 0 06 Dec 2023
DP-NMT: Scalable Differentially-Private Machine Translation Timour Igamberdiev Doan Nam Long Vu Felix Künnecke Zhuo Yu Jannik Holmer Ivan Habernal 27 7 0 24 Nov 2023
FLTrojan: Privacy Leakage Attacks against Federated Language Models Through Selective Weight Tampering Md. Rafi Ur Rashid Vishnu Asutosh Dasu Kang Gu Najrin Sultana Shagufta Mehnaz AAML FedML 44 10 0 24 Oct 2023
Assessing Privacy Risks in Language Models: A Case Study on Summarization Tasks Ruixiang Tang Gord Lueck Rodolfo Quispe Huseyin A. Inan Janardhan Kulkarni Xia Hu 21 6 0 20 Oct 2023
PrivImage: Differentially Private Synthetic Image Generation using Diffusion Models with Semantic-Aware Pretraining Kecen Li Chen Gong Zhixiang Li Yuzhong Zhao Xinwen Hou Tianhao Wang 23 10 0 19 Oct 2023
Privacy Preserving Large Language Models: ChatGPT Case Study Based Vision and Framework Imdad Ullah Najm Hassan S. Gill Basem Suleiman T. Ahanger Zawar Shah Junaid Qadir S. Kanhere 35 16 0 19 Oct 2023
Private Distribution Learning with Public Data: The View from Sample Compression Shai Ben-David Alex Bie C. Canonne Gautam Kamath Vikrant Singhal 27 11 0 11 Aug 2023
PILLAR: How to make semi-private learning more effective Francesco Pinto Yaxian Hu Fanny Yang Amartya Sanyal 30 11 0 06 Jun 2023
Private Meeting Summarization Without Performance Loss Seolhwa Lee Anders Søgaard 22 2 0 25 May 2023
Differentially Private Synthetic Data via Foundation Model APIs 1: Images Zi-Han Lin Sivakanth Gopi Janardhan Kulkarni Harsha Nori Sergey Yekhanin 33 36 0 24 May 2023
Privacy-Preserving Prompt Tuning for Large Language Model Services Yansong Li Zhixing Tan Yang Liu SILM VLM 43 63 0 10 May 2023
Private GANs, Revisited Alex Bie Gautam Kamath Guojun Zhang 4 14 0 06 Feb 2023
Dissociating language and thought in large language models Kyle Mahowald Anna A. Ivanova I. Blank Nancy Kanwisher J. Tenenbaum Evelina Fedorenko ELM ReLM 23 209 0 16 Jan 2023
Training Differentially Private Graph Neural Networks with Random Walk Sampling Morgane Ayle Jan Schuchardt Lukas Gosch Daniel Zügner Stephan Günnemann FedML 21 6 0 02 Jan 2023
Synthetic Text Generation with Differential Privacy: A Simple and Practical Recipe Xiang Yue Huseyin A. Inan Xuechen Li Girish Kumar Julia McAnallen Hoda Shajari Huan Sun David Levitan Robert Sim 32 79 0 25 Oct 2022
Fine-Tuning with Differential Privacy Necessitates an Additional Hyperparameter Search Yannis Cattan Christopher A. Choquette-Choo Nicolas Papernot Abhradeep Thakurta 13 20 0 05 Oct 2022
Differentially Private Optimization on Large Model at Small Cost Zhiqi Bu Yu-Xiang Wang Sheng Zha George Karypis 27 52 0 30 Sep 2022
Training Large-Vocabulary Neural Language Models by Private Federated Learning for Resource-Constrained Devices Mingbin Xu Congzheng Song Ye Tian Neha Agrawal Filip Granqvist ... Shiyi Han Yaqiao Deng Leo Liu Anmol Walia Alex Jin FedML 13 22 0 18 Jul 2022
When Does Differentially Private Learning Not Suffer in High Dimensions? Xuechen Li Daogao Liu Tatsunori Hashimoto Huseyin A. Inan Janardhan Kulkarni Y. Lee Abhradeep Thakurta 19 58 0 01 Jul 2022
Pretrained Models for Multilingual Federated Learning Orion Weller Marc Marone Vladimir Braverman Dawn J Lawrie Benjamin Van Durme VLM FedML AI4CE 33 41 0 06 Jun 2022
A Blessing of Dimensionality in Membership Inference through Regularization Jasper Tan Daniel LeJeune Blake Mason Hamid Javadi Richard G. Baraniuk 16 18 0 27 May 2022
Recovering Private Text in Federated Learning of Language Models Samyak Gupta Yangsibo Huang Zexuan Zhong Tianyu Gao Kai Li Danqi Chen FedML 25 74 0 17 May 2022
Provably Confidential Language Modelling Xuandong Zhao Lei Li Yu-Xiang Wang MU 14 15 0 04 May 2022
Differentially Private Fine-tuning of Language Models Da Yu Saurabh Naik A. Backurs Sivakanth Gopi Huseyin A. Inan ... Y. Lee Andre Manoel Lukas Wutschitz Sergey Yekhanin Huishuai Zhang 134 346 0 13 Oct 2021
Hyperparameter Tuning with Renyi Differential Privacy Nicolas Papernot Thomas Steinke 123 119 0 07 Oct 2021
Opacus: User-Friendly Differential Privacy Library in PyTorch Ashkan Yousefpour I. Shilov Alexandre Sablayrolles Davide Testuggine Karthik Prasad ... Sayan Gosh Akash Bharadwaj Jessica Zhao Graham Cormode Ilya Mironov VLM 144 348 0 25 Sep 2021
On a Utilitarian Approach to Privacy Preserving Text Generation Zekun Xu Abhinav Aggarwal Oluwaseyi Feyisetan Nathanael Teissier 29 24 0 23 Apr 2021