Papers
Communities
Events
Blog
Pricing
Search
Open menu
Home
Papers
2110.06500
Cited By
v1
v2 (latest)
Differentially Private Fine-tuning of Language Models
13 October 2021
Da Yu
Saurabh Naik
A. Backurs
Sivakanth Gopi
Huseyin A. Inan
Gautam Kamath
Janardhan Kulkarni
Y. Lee
Andre Manoel
Lukas Wutschitz
Sergey Yekhanin
Huishuai Zhang
Re-assign community
ArXiv (abs)
PDF
HTML
Papers citing
"Differentially Private Fine-tuning of Language Models"
50 / 81 papers shown
Title
Understanding Users' Security and Privacy Concerns and Attitudes Towards Conversational AI Platforms
Mutahar Ali
Arjun Arunasalam
Habiba Farrukh
SILM
126
1
0
09 Apr 2025
Empirical Privacy Variance
Yuzheng Hu
Fan Wu
Ruicheng Xian
Yuhang Liu
Lydia Zakynthinou
Pritish Kamath
Chiyuan Zhang
David A. Forsyth
145
0
0
16 Mar 2025
Differentially Private Synthetic Data via APIs 3: Using Simulators Instead of Foundation Model
Zinan Lin
Tadas Baltrusaitis
Wenyu Wang
Sergey Yekhanin
SyDa
178
4
0
08 Feb 2025
FedTLU: Federated Learning with Targeted Layer Updates
Jong-Ik Park
Carlee Joe-Wong
FedML
166
0
0
28 Jan 2025
Privacy in Fine-tuning Large Language Models: Attacks, Defenses, and Future Directions
Hao Du
Shang Liu
Lele Zheng
Yang Cao
Atsuyoshi Nakamura
Lei Chen
AAML
180
5
0
21 Dec 2024
DP-2Stage: Adapting Language Models as Differentially Private Tabular Data Generators
Tejumade Afonja
Hui-Po Wang
Raouf Kerkouche
Mario Fritz
SyDa
194
2
0
03 Dec 2024
DiSK: Differentially Private Optimizer with Simplified Kalman Filter for Noise Reduction
Xinwei Zhang
Zhiqi Bu
Borja Balle
Mingyi Hong
Meisam Razaviyayn
Vahab Mirrokni
132
2
0
04 Oct 2024
Differentially Private Kernel Density Estimation
Erzhi Liu
Jerry Yao-Chieh Hu
Alex Reneau
Zhao Song
Han Liu
135
3
0
03 Sep 2024
ObfuscaTune: Obfuscated Offsite Fine-tuning and Inference of Proprietary LLMs on Private Datasets
Ahmed Frikha
Nassim Walha
Ricardo Mendes
Krishna Kanth Nakka
Xue Jiang
Xuebing Zhou
155
3
0
03 Jul 2024
Noise-Aware Differentially Private Regression via Meta-Learning
Ossi Raisa
Stratis Markou
Matthew Ashman
W. Bruinsma
Marlon Tobaben
Antti Honkela
Richard Turner
182
1
0
12 Jun 2024
PrivacyRestore: Privacy-Preserving Inference in Large Language Models via Privacy Removal and Restoration
Huiping Zhuang
Jianwei Wang
Zhengdong Lu
Huiping Zhuang
Haoran Li
Huiping Zhuang
Cen Chen
RALM
KELM
119
8
0
03 Jun 2024
DP-DyLoRA: Fine-Tuning Transformer-Based Models On-Device under Differentially Private Federated Learning using Dynamic Low-Rank Adaptation
Jie Xu
Karthikeyan P. Saravanan
Rogier van Dalen
Haaris Mehmood
David Tuckey
Mete Ozay
163
8
0
10 May 2024
Private Fine-tuning of Large Language Models with Zeroth-order Optimization
Xinyu Tang
Ashwinee Panda
Milad Nasr
Saeed Mahloujifar
Prateek Mittal
205
26
0
09 Jan 2024
Hot PATE: Private Aggregation of Distributions for Diverse Task
Edith Cohen
Benjamin Cohen-Wang
Xin Lyu
Jelani Nelson
Tamas Sarlos
Uri Stemmer
115
4
0
04 Dec 2023
Critical Influence of Overparameterization on Sharpness-aware Minimization
Sungbin Shin
Dongyeop Lee
Maksym Andriushchenko
Namhoon Lee
AAML
154
2
0
29 Nov 2023
Privately Aligning Language Models with Reinforcement Learning
Fan Wu
Huseyin A. Inan
A. Backurs
Varun Chandrasekaran
Janardhan Kulkarni
Robert Sim
92
7
0
25 Oct 2023
DP-Forward: Fine-tuning and Inference on Language Models with Differential Privacy in Forward Pass
Minxin Du
Xiang Yue
Sherman S. M. Chow
Tianhao Wang
Chenyu Huang
Huan Sun
SILM
102
65
0
13 Sep 2023
Unlocking Accuracy and Fairness in Differentially Private Image Classification
Leonard Berrada
Soham De
J. Shen
Jamie Hayes
Robert Stanforth
David Stutz
Pushmeet Kohli
Samuel L. Smith
Borja Balle
67
15
0
21 Aug 2023
Large-Scale Public Data Improves Differentially Private Image Generation Quality
Ruihan Wu
Chuan Guo
Kamalika Chaudhuri
96
2
0
04 Aug 2023
Harnessing large-language models to generate private synthetic text
Alexey Kurakin
Natalia Ponomareva
Umar Syed
Liam MacDermed
Andreas Terzis
SILM
SyDa
76
42
0
02 Jun 2023
Differentially Private Synthetic Data via Foundation Model APIs 1: Images
Zinan Lin
Sivakanth Gopi
Janardhan Kulkarni
Harsha Nori
Sergey Yekhanin
158
44
0
24 May 2023
Privacy-Preserving Prompt Tuning for Large Language Model Services
Yansong Li
Zhixing Tan
Yang Liu
SILM
VLM
96
69
0
10 May 2023
Advancing Differential Privacy: Where We Are Now and Future Directions for Real-World Deployment
Rachel Cummings
Damien Desfontaines
David Evans
Roxana Geambasu
Yangsibo Huang
...
Li Xiong
Sergey Yekhanin
Da Yu
Huanyu Zhang
Wanrong Zhang
54
44
0
14 Apr 2023
Differentially Private Diffusion Models Generate Useful Synthetic Images
Sahra Ghalebikesabi
Leonard Berrada
Sven Gowal
Ira Ktena
Robert Stanforth
Jamie Hayes
Soham De
Samuel L. Smith
Olivia Wiles
Borja Balle
DiffM
76
75
0
27 Feb 2023
Why Is Public Pretraining Necessary for Private Model Training?
Arun Ganesh
Mahdi Haghifam
Milad Nasr
Sewoong Oh
Thomas Steinke
Om Thakkar
Abhradeep Thakurta
Lun Wang
68
39
0
19 Feb 2023
Position: Considerations for Differentially Private Learning with Large-Scale Public Pretraining
Florian Tramèr
Gautam Kamath
Nicholas Carlini
SILM
124
72
0
13 Dec 2022
Exploring the Limits of Differentially Private Deep Learning with Group-wise Clipping
Jiyan He
Xuechen Li
Da Yu
Huishuai Zhang
Janardhan Kulkarni
Y. Lee
A. Backurs
Nenghai Yu
Jiang Bian
118
49
0
03 Dec 2022
Synthetic Text Generation with Differential Privacy: A Simple and Practical Recipe
Xiang Yue
Huseyin A. Inan
Xuechen Li
Girish Kumar
Julia McAnallen
Hoda Shajari
Huan Sun
David Levitan
Robert Sim
147
86
0
25 Oct 2022
Differentially Private Optimization on Large Model at Small Cost
Zhiqi Bu
Yu Wang
Sheng Zha
George Karypis
100
55
0
30 Sep 2022
Pre-trained Perceptual Features Improve Differentially Private Image Generation
Fredrik Harder
Milad Jalali Asadabadi
Danica J. Sutherland
Mijung Park
121
29
0
25 May 2022
Unlocking High-Accuracy Differentially Private Image Classification through Scale
Soham De
Leonard Berrada
Jamie Hayes
Samuel L. Smith
Borja Balle
95
233
0
28 Apr 2022
Mixed Differential Privacy in Computer Vision
Aditya Golatkar
Alessandro Achille
Yu Wang
Aaron Roth
Michael Kearns
Stefano Soatto
PICV
VLM
93
50
0
22 Mar 2022
Submix: Practical Private Prediction for Large-Scale Language Models
Antonio A. Ginart
Laurens van der Maaten
James Zou
Chuan Guo
79
23
0
04 Jan 2022
Public Data-Assisted Mirror Descent for Private Model Training
Ehsan Amid
Arun Ganesh
Rajiv Mathews
Swaroop Indra Ramaswamy
Shuang Song
Thomas Steinke
Vinith Suriyakumar
Om Thakkar
Abhradeep Thakurta
89
51
0
01 Dec 2021
Large-Scale Differentially Private BERT
Rohan Anil
Badih Ghazi
Vineet Gupta
Ravi Kumar
Pasin Manurangsi
90
138
0
03 Aug 2021
Benchmarking Differential Privacy and Federated Learning for BERT Models
Priya Basu
Tiasa Singha Roy
Rakshit Naidu
Zumrut Muftuoglu
Sahib Singh
Fatemehsadat Mireshghallah
FedML
AI4MH
78
51
0
26 Jun 2021
BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models
Elad Ben-Zaken
Shauli Ravfogel
Yoav Goldberg
207
1,244
0
18 Jun 2021
LoRA: Low-Rank Adaptation of Large Language Models
J. E. Hu
Yelong Shen
Phillip Wallis
Zeyuan Allen-Zhu
Yuanzhi Li
Shean Wang
Lu Wang
Weizhu Chen
OffRL
AI4TS
AI4CE
ALM
AIMat
540
10,565
0
17 Jun 2021
Large Scale Private Learning via Low-rank Reparametrization
Da Yu
Huishuai Zhang
Wei Chen
Jian Yin
Tie-Yan Liu
67
106
0
17 Jun 2021
Compacter: Efficient Low-Rank Hypercomplex Adapter Layers
Rabeeh Karimi Mahabadi
James Henderson
Sebastian Ruder
MoE
121
492
0
08 Jun 2021
Numerical Composition of Differential Privacy
Sivakanth Gopi
Y. Lee
Lukas Wutschitz
86
183
0
05 Jun 2021
The Power of Scale for Parameter-Efficient Prompt Tuning
Brian Lester
Rami Al-Rfou
Noah Constant
VPVLM
633
4,107
0
18 Apr 2021
Do Not Let Privacy Overbill Utility: Gradient Embedding Perturbation for Private Learning
Da Yu
Huishuai Zhang
Wei Chen
Tie-Yan Liu
FedML
SILM
167
115
0
25 Feb 2021
Leveraging Public Data for Practical Private Query Release
Terrance Liu
G. Vietri
Thomas Steinke
Jonathan R. Ullman
Zhiwei Steven Wu
198
60
0
17 Feb 2021
Prefix-Tuning: Optimizing Continuous Prompts for Generation
Xiang Lisa Li
Percy Liang
252
4,314
0
01 Jan 2021
Intrinsic Dimensionality Explains the Effectiveness of Language Model Fine-Tuning
Armen Aghajanyan
Luke Zettlemoyer
Sonal Gupta
110
573
1
22 Dec 2020
Extracting Training Data from Large Language Models
Nicholas Carlini
Florian Tramèr
Eric Wallace
Matthew Jagielski
Ariel Herbert-Voss
...
Tom B. Brown
Basel Alomair
Ulfar Erlingsson
Alina Oprea
Colin Raffel
MLAU
SILM
531
1,958
0
14 Dec 2020
When is Memorization of Irrelevant Training Data Necessary for High-Accuracy Learning?
Gavin Brown
Mark Bun
Vitaly Feldman
Adam D. Smith
Kunal Talwar
314
102
0
11 Dec 2020
AdapterDrop: On the Efficiency of Adapters in Transformers
Andreas Rucklé
Gregor Geigle
Max Glockner
Tilman Beck
Jonas Pfeiffer
Nils Reimers
Iryna Gurevych
125
267
0
22 Oct 2020
Enabling Fast Differentially Private SGD via Just-in-Time Compilation and Vectorization
P. Subramani
Nicholas Vadivelu
Gautam Kamath
99
83
0
18 Oct 2020
1
2
Next