Large Scale Private Learning via Low-rank Reparametrization

17 June 2021

Papers citing "Large Scale Private Learning via Low-rank Reparametrization"

21 / 21 papers shown

Title
NoEsis: Differentially Private Knowledge Transfer in Modular LLM Adaptation Rob Romijnders Stefanos Laskaridis Ali Shahin Shamsabadi Hamed Haddadi 64 0 0 25 Apr 2025
DC-SGD: Differentially Private SGD with Dynamic Clipping through Gradient Norm Distribution Estimation Chengkun Wei Weixian Li Chen Gong Wenzhi Chen 58 0 0 29 Mar 2025
Differentially Private Parameter-Efficient Fine-tuning for Large ASR Models Hongbin Liu Lun Wang Om Thakkar Abhradeep Thakurta Arun Narayanan 26 0 0 02 Oct 2024
Learning Privacy-Preserving Student Networks via Discriminative-Generative Distillation Shiming Ge Bochao Liu Pengju Wang Yong Li Dan Zeng FedML 42 9 0 04 Sep 2024
DP-DyLoRA: Fine-Tuning Transformer-Based Models On-Device under Differentially Private Federated Learning using Dynamic Low-Rank Adaptation Jie Xu Karthikeyan P. Saravanan Rogier van Dalen Haaris Mehmood David Tuckey Mete Ozay 56 5 0 10 May 2024
Private Fine-tuning of Large Language Models with Zeroth-order Optimization Xinyu Tang Ashwinee Panda Milad Nasr Saeed Mahloujifar Prateek Mittal 44 18 0 09 Jan 2024
Differentially Private Image Classification by Learning Priors from Random Processes Xinyu Tang Ashwinee Panda Vikash Sehwag Prateek Mittal 23 20 0 08 Jun 2023
PILLAR: How to make semi-private learning more effective Francesco Pinto Yaxian Hu Fanny Yang Amartya Sanyal 46 11 0 06 Jun 2023
Privacy-Preserving Prompt Tuning for Large Language Model Services Yansong Li Zhixing Tan Yang Liu SILM VLM 47 63 0 10 May 2023
Differentially Private Natural Language Models: Recent Advances and Future Directions Lijie Hu Ivan Habernal Lei Shen Di Wang AAML 24 18 0 22 Jan 2023
A New Linear Scaling Rule for Private Adaptive Hyperparameter Optimization Ashwinee Panda Xinyu Tang Saeed Mahloujifar Vikash Sehwag Prateek Mittal 43 11 0 08 Dec 2022
Privately Fine-Tuning Large Language Models with Differential Privacy R. Behnia Mohammadreza Ebrahimi Jason L. Pacheco B. Padmanabhan 24 44 0 26 Oct 2022
Synthetic Text Generation with Differential Privacy: A Simple and Practical Recipe Xiang Yue Huseyin A. Inan Xuechen Li Girish Kumar Julia McAnallen Hoda Shajari Huan Sun David Levitan Robert Sim 47 79 0 25 Oct 2022
Scaling Private Deep Learning with Low-Rank and Sparse Gradients Ryuichi Ito Seng Pei Liew Tsubasa Takahashi Yuya Sasaki Makoto Onizuka 28 1 0 06 Jul 2022
Improving Differentially Private SGD via Randomly Sparsified Gradients Junyi Zhu Matthew B. Blaschko 21 5 0 01 Dec 2021
Privately Publishable Per-instance Privacy Rachel Redberg Yu-Xiang Wang 29 17 0 03 Nov 2021
Differentially Private Fine-tuning of Language Models Da Yu Saurabh Naik A. Backurs Sivakanth Gopi Huseyin A. Inan ... Y. Lee Andre Manoel Lukas Wutschitz Sergey Yekhanin Huishuai Zhang 134 346 0 13 Oct 2021
Do Not Let Privacy Overbill Utility: Gradient Embedding Perturbation for Private Learning Da Yu Huishuai Zhang Wei Chen Tie-Yan Liu FedML SILM 94 110 0 25 Feb 2021
Extracting Training Data from Large Language Models Nicholas Carlini Florian Tramèr Eric Wallace Matthew Jagielski Ariel Herbert-Voss ... Tom B. Brown D. Song Ulfar Erlingsson Alina Oprea Colin Raffel MLAU SILM 290 1,814 0 14 Dec 2020
Tempered Sigmoid Activations for Deep Learning with Differential Privacy Nicolas Papernot Abhradeep Thakurta Shuang Song Steve Chien Ulfar Erlingsson AAML 141 178 0 28 Jul 2020
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 297 6,956 0 20 Apr 2018