Improving LoRA in Privacy-preserving Federated Learning

18 March 2024

Papers citing "Improving LoRA in Privacy-preserving Federated Learning"

38 / 38 papers shown

Title
Towards Artificial General or Personalized Intelligence? A Survey on Foundation Models for Personalized Federated Intelligence Yu Qiao Huy Q. Le Avi Deb Raha Phuong-Nam Tran Apurba Adhikary Mengchun Zhang Loc X. Nguyen Eui-nam Huh Dusit Niyato C. Hong AI4CE 31 0 0 11 May 2025
Federated Adapter on Foundation Models: An Out-Of-Distribution Approach Yiyuan Yang Guodong Long Tianyi Zhou Qinghua Lu Shanshan Ye Jing Jiang OODD 168 1 0 02 May 2025
Communication-Efficient Wireless Federated Fine-Tuning for Large-Scale AI Models Bumjun Kim Wan Choi 26 0 0 01 May 2025
A Survey on Parameter-Efficient Fine-Tuning for Foundation Models in Federated Learning Jieming Bian Yuanzhe Peng Lei Wang Yin Huang Jie Xu FedML 65 0 0 29 Apr 2025
ReCIT: Reconstructing Full Private Data from Gradient in Parameter-Efficient Fine-Tuning of Large Language Models Jin Xie Ruishi He Songze Li Xiaojun Jia Shouling Ji SILM AAML 66 0 0 29 Apr 2025
FedMerge: Federated Personalization via Model Merging Shutong Chen Tianyi Zhou Guodong Long Jing Jiang Chengqi Zhang FedML MoMe 49 0 0 09 Apr 2025
Communication-Efficient and Personalized Federated Foundation Model Fine-Tuning via Tri-Matrix Adaptation Y. Li Bo Liu Sheng Huang Z. Zhang Xiaotong Yuan Richang Hong 46 0 0 31 Mar 2025
FedALT: Federated Fine-Tuning through Adaptive Local Training with Rest-of-the-World LoRA Jieming Bian Lei Wang Letian Zhang Jie Xu 52 1 0 14 Mar 2025
Federated Multimodal Learning with Dual Adapters and Selective Pruning for Communication and Computational Efficiency Duy Phuong Nguyen J. P. Muñoz Tanya Roosta Ali Jannesari FedML 67 0 0 10 Mar 2025
Fairness-Aware Low-Rank Adaptation Under Demographic Privacy Constraints Parameswaran Kamalaruban Mark Anderson Stuart Burrell Maeve Madigan Piotr Skalski David Sutton 54 0 0 07 Mar 2025
Can Textual Gradient Work in Federated Learning? Minghui Chen Ruinan Jin Wenlong Deng Yuanyuan Chen Zhi Huang Han Yu Xiaoxiao Li FedML 81 2 0 27 Feb 2025
Robust Federated Finetuning of LLMs via Alternating Optimization of LoRA Shuangyi Chen Yuanxin Guo Yue Ju Harik Dalal Ashish Khisti 48 1 0 03 Feb 2025
Decentralized Low-Rank Fine-Tuning of Large Language Models Sajjad Ghiasvand Mahnoosh Alizadeh Ramtin Pedarsani ALM 66 0 0 26 Jan 2025
Aggregating Low Rank Adapters in Federated Fine-tuning Evelyn Trautmann Ian Hales Martin F. Volk AI4CE FedML 39 0 0 10 Jan 2025
Initialization using Update Approximation is a Silver Bullet for Extremely Efficient Low-Rank Fine-Tuning Kaustubh Ponkshe Raghav Singhal Eduard A. Gorbunov Alexey Tumanov Samuel Horváth Praneeth Vepakomma 68 1 0 29 Nov 2024
Personalized Federated Fine-Tuning for LLMs via Data-Driven Heterogeneous Model Architectures Yicheng Zhang Zhen Qin Zhaomin Wu Shuiguang Deng 80 2 0 28 Nov 2024
Communication-Efficient and Tensorized Federated Fine-Tuning of Large Language Models Sajjad Ghiasvand Yifan Yang Zhiyu Xue Mahnoosh Alizadeh Zheng Zhang Ramtin Pedarsani FedML 33 3 0 16 Oct 2024
DEeR: Deviation Eliminating and Noise Regulating for Privacy-preserving Federated Low-rank Adaptation Meilu Zhu Axiu Mao Jun Liu Yixuan Yuan 34 1 0 16 Oct 2024
Federated Data-Efficient Instruction Tuning for Large Language Models Zhen Qin Zhaomin Wu Bingsheng He Shuiguang Deng FedML 35 2 0 14 Oct 2024
Randomized Asymmetric Chain of LoRA: The First Meaningful Theoretical Framework for Low-Rank Adaptation Grigory Malinovsky Umberto Michieli Hasan Hammoud Taha Ceritli Hayder Elesedy Mete Ozay Peter Richtárik AI4CE 27 1 0 10 Oct 2024
Selective Aggregation for Low-Rank Adaptation in Federated Learning Pengxin Guo Shuang Zeng Y. Wang Huijie Fan Feifei Wang Liangqiong Qu FedML 44 8 0 02 Oct 2024
Federated Large Language Models: Current Progress and Future Directions Yuhang Yao Jianyi Zhang Junda Wu Chengkai Huang Yu Xia ... Ang Li Lina Yao Julian McAuley Yiran Chen Carlee Joe-Wong FedML AIFin 63 8 0 24 Sep 2024
On-Device Collaborative Language Modeling via a Mixture of Generalists and Specialists Dongyang Fan Bettina Messmer N. Doikov Martin Jaggi MoMe MoE 44 1 0 20 Sep 2024
Communication-Efficient Federated Low-Rank Update Algorithm and its Connection to Implicit Regularization Haemin Park Diego Klabjan FedML 32 0 0 19 Sep 2024
Leveraging Unstructured Text Data for Federated Instruction Tuning of Large Language Models Rui Ye Rui Ge Yuchi Fengting Jingyi Chai Yanfeng Wang Siheng Chen FedML 32 1 0 11 Sep 2024
CELLM: An Efficient Communication in Large Language Models Training for Federated Learning Raja Vavekanand Kira Sam 45 0 0 30 Jul 2024
A Survey on LoRA of Large Language Models Yuren Mao Yuhang Ge Yijiang Fan Wenyi Xu Yu Mi Zhonghao Hu Yunjun Gao ALM 54 24 0 08 Jul 2024
Synergizing Foundation Models and Federated Learning: A Survey Shenghui Li Fanghua Ye Meng Fang Jiaxu Zhao Yun-Hin Chan Edith C. -H. Ngai Thiemo Voigt AI4CE 51 5 0 18 Jun 2024
Federated LoRA with Sparse Communication Kevin Kuo Arian Raje Kousik Rajesh Virginia Smith 38 7 0 07 Jun 2024
FedLLM-Bench: Realistic Benchmarks for Federated Learning of Large Language Models Rui Ye Rui Ge Xinyu Zhu Jingyi Chai Yaxin Du Yang Liu Yanfeng Wang Siheng Chen FedML 39 14 0 07 Jun 2024
RE-Adapt: Reverse Engineered Adaptation of Large Language Models William Fleshman Benjamin Van Durme VLM 29 3 0 23 May 2024
Towards Efficient Communication and Secure Federated Recommendation System via Low-rank Training Ngoc-Hieu Nguyen Tuan Nguyen Tuan Nguyen Vu Tien Hoang Dung D. Le Kok-Seng Wong FedML 15 6 0 08 Jan 2024
GLM-130B: An Open Bilingual Pre-trained Model Aohan Zeng Xiao Liu Zhengxiao Du Zihan Wang Hanyu Lai ... Jidong Zhai Wenguang Chen Peng-Zhen Zhang Yuxiao Dong Jie Tang BDL LRM 250 1,073 0 05 Oct 2022
Differentially Private Fine-tuning of Language Models Da Yu Saurabh Naik A. Backurs Sivakanth Gopi Huseyin A. Inan ... Y. Lee Andre Manoel Lukas Wutschitz Sergey Yekhanin Huishuai Zhang 134 346 0 13 Oct 2021
Opacus: User-Friendly Differential Privacy Library in PyTorch Ashkan Yousefpour I. Shilov Alexandre Sablayrolles Davide Testuggine Karthik Prasad ... Sayan Gosh Akash Bharadwaj Jessica Zhao Graham Cormode Ilya Mironov VLM 152 349 0 25 Sep 2021
The Power of Scale for Parameter-Efficient Prompt Tuning Brian Lester Rami Al-Rfou Noah Constant VPVLM 280 3,844 0 18 Apr 2021
Extracting Training Data from Large Language Models Nicholas Carlini Florian Tramèr Eric Wallace Matthew Jagielski Ariel Herbert-Voss ... Tom B. Brown D. Song Ulfar Erlingsson Alina Oprea Colin Raffel MLAU SILM 290 1,814 0 14 Dec 2020
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 297 6,956 0 20 Apr 2018