Federated Learning of Gboard Language Models with Differential Privacy

29 May 2023

Zheng Xu

Yanxiang Zhang

Galen Andrew

Christopher A. Choquette-Choo

Papers citing "Federated Learning of Gboard Language Models with Differential Privacy"

50 / 61 papers shown

Title
UnifyFL: Enabling Decentralized Cross-Silo Federated Learning Sarang S Druva Dhakshinamoorthy Aditya Shiva Sharma Yuvraj Singh Bhadauria Siddharth Chaitra Vivek Arihant Bansal Arnab K. Paul FedML 31 0 0 26 Apr 2025
Private Federated Learning using Preference-Optimized Synthetic Data Charlie Hou Mei-Yu Wang Yige Zhu Daniel Lazar Giulia Fanti FedML Presented at ResearchTrend Connect \| FedML on 07 May 2025 57 0 0 23 Apr 2025
Accelerating Differentially Private Federated Learning via Adaptive Extrapolation Shokichi Takakura Seng Pei Liew Satoshi Hasegawa FedML 30 0 0 14 Apr 2025
$Infinitely Divisible Noise for Differential Privacy: Nearly Optimal Error in the High $\varepsilon$ Regime$ Infinitely Divisible Noise for Differential Privacy: Nearly Optimal Error in the High $\varepsilon$ Regime Charlie Harrison Pasin Manurangsi 24 0 0 07 Apr 2025
Empirical Privacy Variance Yuzheng Hu Fan Wu Ruicheng Xian Yuhang Liu Lydia Zakynthinou Pritish Kamath Chiyuan Zhang David A. Forsyth 62 0 0 16 Mar 2025
Synthesizing Privacy-Preserving Text Data via Finetuning without Finetuning Billion-Scale LLMs Bowen Tan Zheng Xu Eric P. Xing Zhiting Hu Shanshan Wu SyDa 85 0 0 16 Mar 2025
Multi-Agent Collaboration Mechanisms: A Survey of LLMs Khanh-Tung Tran Dung Dao Minh-Duong Nguyen Quoc-Viet Pham Barry O’Sullivan Hoang D. Nguyen LLMAG 93 23 0 10 Jan 2025
Secure Stateful Aggregation: A Practical Protocol with Applications in Differentially-Private Federated Learning Marshall Ball James Bell-Clark Adria Gascon Peter Kairouz Sewoong Oh Zhiye Xie FedML 26 0 0 15 Oct 2024
Federated Learning in Practice: Reflections and Projections Katharine Daly Hubert Eichner Peter Kairouz H. B. McMahan Daniel Ramage Zheng Xu FedML 53 5 0 11 Oct 2024
Debiasing Federated Learning with Correlated Client Participation Zhenyu Sun Ziyang Zhang Zheng Xu Gauri Joshi Pranay Sharma Ermin Wei FedML 24 0 0 02 Oct 2024
Byzantine-Robust Aggregation for Securing Decentralized Federated Learning Diego Cajaraville-Aboy Ana Fernández-Vilas R. Redondo Manuel Fernández-Veiga 18 2 0 26 Sep 2024
Scalable Differential Privacy Mechanisms for Real-Time Machine Learning Applications Jessica Smith David Williams Emily Brown 21 0 0 16 Sep 2024
A Statistical Viewpoint on Differential Privacy: Hypothesis Testing, Representation and Blackwell's Theorem Weijie J. Su 23 1 0 14 Sep 2024
A Hassle-free Algorithm for Private Learning in Practice: Don't Use Tree Aggregation, Use BLTs H. B. McMahan Zheng Xu Yanxiang Zhang FedML 32 5 0 16 Aug 2024
Federated Learning as a Service for Hierarchical Edge Networks with Heterogeneous Models Wentao Gao Omid Tavallaie Shuaijun Chen Albert Zomaya FedML 25 3 0 30 Jul 2024
Universally Harmonizing Differential Privacy Mechanisms for Federated Learning: Boosting Accuracy and Convergence Shuya Feng Meisam Mohammady Hanbin Hong Shenao Yan Ashish Kundu Binghui Wang Yuan Hong FedML 34 3 0 20 Jul 2024
Fine-Tuning Large Language Models with User-Level Differential Privacy Zachary Charles Arun Ganesh Ryan McKenna H. B. McMahan Nicole Mitchell Krishna Pillutla Keith Rush 21 11 0 10 Jul 2024
Synergizing Foundation Models and Federated Learning: A Survey Shenghui Li Fanghua Ye Meng Fang Jiaxu Zhao Yun-Hin Chan Edith C. -H. Ngai Thiemo Voigt AI4CE 43 5 0 18 Jun 2024
Empirical Guidelines for Deploying LLMs onto Resource-constrained Edge Devices Ruiyang Qin Dancheng Liu Zheyu Yan Zhaoxuan Tan Zixuan Pan Zhenge Jia Meng-Long Jiang Ahmed Abbasi Jinjun Xiong Yiyu Shi 51 10 0 06 Jun 2024
PrE-Text: Training Language Models on Private Federated Data in the Age of LLMs Charlie Hou Akshat Shrivastava Hongyuan Zhan Rylan Conway Trang Le Adithya Sagar Giulia Fanti Daniel Lazar 24 8 0 05 Jun 2024
DP-DyLoRA: Fine-Tuning Transformer-Based Models On-Device under Differentially Private Federated Learning using Dynamic Low-Rank Adaptation Jie Xu Karthikeyan P. Saravanan Rogier van Dalen Haaris Mehmood David Tuckey Mete Ozay 56 5 0 10 May 2024
Improved Communication-Privacy Trade-offs in $L_2$ Mean Estimation under Streaming Differential Privacy Wei-Ning Chen Berivan Isik Peter Kairouz Albert No Sewoong Oh Zheng Xu 41 3 0 02 May 2024
Efficient and Near-Optimal Noise Generation for Streaming Differential Privacy Krishnamurthy Dvijotham H. B. McMahan Krishna Pillutla Thomas Steinke Abhradeep Thakurta 35 10 0 25 Apr 2024
Confidential Federated Computations Hubert Eichner Daniel Ramage Kallista A. Bonawitz Dzmitry Huba Tiziano Santoro ... Albert Cheu Katharine Daly Adria Gascon Marco Gruteser Brendan McMahan 35 2 0 16 Apr 2024
pfl-research: simulation framework for accelerating research in Private Federated Learning Filip Granqvist Congzheng Song Áine Cahill Rogier van Dalen Martin Pelikan Yi Sheng Chan Xiaojun Feng Natarajan Krishnaswami Vojta Jina Mona Chitnis FedML 26 5 0 09 Apr 2024
Prompt Public Large Language Models to Synthesize Data for Private On-device Applications Shanshan Wu Zheng Xu Yanxiang Zhang Yuanbo Zhang Daniel Ramage SyDa 21 9 0 05 Apr 2024
Efficient Language Model Architectures for Differentially Private Federated Learning Jae Hun Ro Srinadh Bhojanapalli Zheng Xu Yanxiang Zhang A. Suresh FedML 39 2 0 12 Mar 2024
Differentially Private Knowledge Distillation via Synthetic Text Generation James Flemings Murali Annavaram SyDa 32 11 0 01 Mar 2024
Teach LLMs to Phish: Stealing Private Information from Language Models Ashwinee Panda Christopher A. Choquette-Choo Zhengming Zhang Yaoqing Yang Prateek Mittal PILM 21 20 0 01 Mar 2024
Private Gradient Descent for Linear Regression: Tighter Error Bounds and Instance-Specific Uncertainty Estimation Gavin Brown Krishnamurthy Dvijotham Georgina Evans Daogao Liu Adam D. Smith Abhradeep Thakurta 34 3 0 21 Feb 2024
Auditing Private Prediction Karan Chadha Matthew Jagielski Nicolas Papernot Christopher A. Choquette-Choo Milad Nasr 30 4 0 14 Feb 2024
Momentum Approximation in Asynchronous Private Federated Learning Tao Yu Congzheng Song Jianyu Wang Mona Chitnis FedML 30 1 0 14 Feb 2024
A Survey on Efficient Federated Learning Methods for Foundation Model Training Herbert Woisetschläger Alexander Isenko Shiqiang Wang R. Mayer Hans-Arno Jacobsen FedML 55 23 0 09 Jan 2024
Lotto: Secure Participant Selection against Adversarial Servers in Federated Learning Zhifeng Jiang Peng Ye Shiqi He Wei Wang Ruichuan Chen Bo Li 11 2 0 05 Jan 2024
Venn: Resource Management for Collaborative Learning Jobs Jiachen Liu Fan Lai Ding Ding Yiwen Zhang Mosharaf Chowdhury FedML 42 1 0 13 Dec 2023
Grounding Foundation Models through Federated Transfer Learning: A General Framework Yan Kang Tao Fan Hanlin Gu Xiaojin Zhang Lixin Fan Qiang Yang AI4CE 68 19 0 29 Nov 2023
Enabling On-Device Large Language Model Personalization with Self-Supervised Data Selection and Synthesis Ruiyang Qin Jun Xia Zhenge Jia Meng-Long Jiang Ahmed Abbasi Peipei Zhou Jingtong Hu Yiyu Shi 24 18 0 21 Nov 2023
Robust and Actively Secure Serverless Collaborative Learning Olive Franzese Adam Dziedzic Christopher A. Choquette-Choo Mark R. Thomas Muhammad Ahmad Kaleem Stephan Rabanser Cong Fang Somesh Jha Nicolas Papernot Xiao Wang OOD 17 2 0 25 Oct 2023
FedTherapist: Mental Health Monitoring with User-Generated Linguistic Expressions on Smartphones via Federated Learning Jaemin Shin Hyungjun Yoon Seungjoo Lee Sungjoon Park Yunxin Liu Jinho D. Choi Sung-Ju Lee 17 5 0 25 Oct 2023
User Inference Attacks on Large Language Models Nikhil Kandpal Krishna Pillutla Alina Oprea Peter Kairouz Christopher A. Choquette-Choo Zheng Xu SILM AAML 28 15 0 13 Oct 2023
Optimization of Federated Learning's Client Selection for Non-IID Data Based on Grey Relational Analysis Shuaijun Chen Omid Tavallaie Michael Henri Hambali S. M. Zandavi Hamed Haddadi Nicholas D. Lane Song Guo Albert Y. Zomaya FedML 21 1 0 12 Oct 2023
Correlated Noise Provably Beats Independent Noise for Differentially Private Learning Christopher A. Choquette-Choo Krishnamurthy Dvijotham Krishna Pillutla Arun Ganesh Thomas Steinke Abhradeep Thakurta 17 13 0 10 Oct 2023
Federated Learning with Differential Privacy for End-to-End Speech Recognition Martin Pelikan Sheikh Shams Azam Vitaly Feldman Jan Honza Silovsky Kunal Talwar Tatiana Likhomanenko 30 7 0 29 Sep 2023
How to Protect Copyright Data in Optimization of Large Language Models? T. Chu Zhao-quan Song Chiwun Yang 28 29 0 23 Aug 2023
Private Distribution Learning with Public Data: The View from Sample Compression Shai Ben-David Alex Bie C. Canonne Gautam Kamath Vikrant Singhal 22 11 0 11 Aug 2023
When Federated Learning meets Watermarking: A Comprehensive Overview of Techniques for Intellectual Property Protection Mohammed Lansari R. Bellafqira K. Kapusta V. Thouvenot Olivier Bettan G. Coatrieux FedML 23 15 0 07 Aug 2023
Samplable Anonymous Aggregation for Private Federated Data Analysis Kunal Talwar Shan Wang Audra McMillan Vojta Jina Vitaly Feldman ... Congzheng Song Karl Tarbe Sebastian Vogt L. Winstrom Shundong Zhou FedML 28 13 0 27 Jul 2023
Private Federated Learning with Autotuned Compression Enayat Ullah Christopher A. Choquette-Choo Peter Kairouz Sewoong Oh FedML 15 6 0 20 Jul 2023
Private Federated Learning in Gboard Yuanbo Zhang Daniel Ramage Zheng Xu Yanxiang Zhang Shumin Zhai Peter Kairouz FedML 17 7 0 26 Jun 2023
(Amplified) Banded Matrix Factorization: A unified approach to private training Christopher A. Choquette-Choo Arun Ganesh Ryan McKenna H. B. McMahan Keith Rush Abhradeep Thakurta Zheng Xu FedML 15 35 0 13 Jun 2023