When Federated Learning Meets Pre-trained Language Models' Parameter-Efficient Tuning Methods

20 December 2022

Papers citing "When Federated Learning Meets Pre-trained Language Models' Parameter-Efficient Tuning Methods"

44 / 44 papers shown

Title
Federated Adapter on Foundation Models: An Out-Of-Distribution Approach Yiyuan Yang Guodong Long Tianyi Zhou Qinghua Lu Shanshan Ye Jing Jiang OODD 102 1 0 02 May 2025
A Survey on Parameter-Efficient Fine-Tuning for Foundation Models in Federated Learning Jieming Bian Yuanzhe Peng Lei Wang Yin Huang Jie Xu FedML 55 0 0 29 Apr 2025
Towards Harnessing the Collaborative Power of Large and Small Models for Domain Tasks Yang Janet Liu Bingjie Yan Tianyuan Zou Jianqing Zhang Zixuan Gu ... J. Li Xiaozhou Ye Ye Ouyang Qiang Yang Y. Zhang ALM 86 1 0 24 Apr 2025
FHBench: Towards Efficient and Personalized Federated Learning for Multimodal Healthcare Penghao Wang Qian Chen Teng Zhang Y. Zhang Wang Lu Yiqiang Chen 21 0 0 15 Apr 2025
Communication-Efficient and Personalized Federated Foundation Model Fine-Tuning via Tri-Matrix Adaptation Y. Li Bo Liu Sheng Huang Z. Zhang Xiaotong Yuan Richang Hong 41 0 0 31 Mar 2025
Federated Multimodal Learning with Dual Adapters and Selective Pruning for Communication and Computational Efficiency Duy Phuong Nguyen J. P. Muñoz Tanya Roosta Ali Jannesari FedML 62 0 0 10 Mar 2025
PriFFT: Privacy-preserving Federated Fine-tuning of Large Language Models via Hybrid Secret Sharing Zhichao You Xuewen Dong Ke Cheng Xutong Mu Jiaxuan Fu Shiyang Ma Qiang Qu Yulong Shen FedML 84 0 0 05 Mar 2025
Decentralized Low-Rank Fine-Tuning of Large Language Models Sajjad Ghiasvand Mahnoosh Alizadeh Ramtin Pedarsani ALM 64 0 0 26 Jan 2025
Aggregating Low Rank Adapters in Federated Fine-tuning Evelyn Trautmann Ian Hales Martin F. Volk AI4CE FedML 36 0 0 10 Jan 2025
Rate-My-LoRA: Efficient and Adaptive Federated Model Tuning for Cardiac MRI Segmentation Xiaoxiao He H. Shi Ligong Han Chaowei Tan Bo Liu Zihao Xu Meng Ye L. Axel Kang Li Dimitris N. Metaxas FedML 33 0 0 06 Jan 2025
Personalized Federated Fine-Tuning for LLMs via Data-Driven Heterogeneous Model Architectures Yicheng Zhang Zhen Qin Zhaomin Wu Shuiguang Deng 70 2 0 28 Nov 2024
Photon: Federated LLM Pre-Training Lorenzo Sani Alex Iacob Zeyu Cao Royson Lee Bill Marino ... Dongqi Cai Zexi Li Wanru Zhao Xinchi Qiu Nicholas D. Lane AI4CE 24 7 0 05 Nov 2024
On Sampling Strategies for Spectral Model Sharding Denis Korzhenkov Christos Louizos FedML 27 0 0 31 Oct 2024
Local Superior Soups: A Catalyst for Model Merging in Cross-Silo Federated Learning Minghui Chen Meirui Jiang Xin Zhang Qi Dou Zehua Wang Xiaoxiao Li MoMe FedML 35 2 0 31 Oct 2024
Communication-Efficient and Tensorized Federated Fine-Tuning of Large Language Models Sajjad Ghiasvand Yifan Yang Zhiyu Xue Mahnoosh Alizadeh Zheng Zhang Ramtin Pedarsani FedML 33 3 0 16 Oct 2024
DEeR: Deviation Eliminating and Noise Regulating for Privacy-preserving Federated Low-rank Adaptation Meilu Zhu Axiu Mao Jun Liu Yixuan Yuan 32 1 0 16 Oct 2024
Federated Large Language Models: Current Progress and Future Directions Yuhang Yao Jianyi Zhang Junda Wu Chengkai Huang Yu Xia ... Ang Li Lina Yao Julian McAuley Yiran Chen Carlee Joe-Wong FedML AIFin 61 8 0 24 Sep 2024
Ferret: Federated Full-Parameter Tuning at Scale for Large Language Models Yao Shu Wenyang Hu S. Ng Bryan Kian Hsiang Low Fei Richard Yu FedML 32 0 0 10 Sep 2024
FedMCP: Parameter-Efficient Federated Learning with Model-Contrastive Personalization Qianyi Zhao Chen Qu Cen Chen Mingyuan Fan Yanhao Wang 16 1 0 28 Aug 2024
Fine-Tuning and Deploying Large Language Models Over Edges: Issues and Approaches Yanjie Dong Xiaoyi Fan Fangxin Wang Chengming Li Victor C. M. Leung Xiping Hu 26 5 0 20 Aug 2024
CELLM: An Efficient Communication in Large Language Models Training for Federated Learning Raja Vavekanand Kira Sam 20 0 0 30 Jul 2024
FedBiOT: LLM Local Fine-tuning in Federated Learning without Full Model Feijie Wu Zitao Li Yaliang Li Bolin Ding Jing Gao 24 41 0 25 Jun 2024
Recent Advances in Federated Learning Driven Large Language Models: A Survey on Architecture, Performance, and Security Youyang Qu Ming Liu Tianqing Zhu Longxiang Gao Shui Yu Wanlei Zhou MU FedML 54 2 0 14 Jun 2024
Deconstructing The Ethics of Large Language Models from Long-standing Issues to New-emerging Dilemmas Chengyuan Deng Yiqun Duan Xin Jin Heng Chang Yijun Tian ... Kuofeng Gao Sihong He Jun Zhuang Lu Cheng Haohan Wang AILaw 38 16 0 08 Jun 2024
Federated LoRA with Sparse Communication Kevin Kuo Arian Raje Kousik Rajesh Virginia Smith 38 7 0 07 Jun 2024
PrivacyRestore: Privacy-Preserving Inference in Large Language Models via Privacy Removal and Restoration Ziqian Zeng Jianwei Wang Zhengdong Lu Huiping Zhuang Cen Chen RALM KELM 30 7 0 03 Jun 2024
Personalized Adapter for Large Meteorology Model on Devices: Towards Weather Foundation Models Shengchao Chen Guodong Long Jing Jiang Chengqi Zhang 46 4 0 24 May 2024
Thinking Forward: Memory-Efficient Federated Finetuning of Language Models Kunjal Panchal Nisarg Parikh Sunav Choudhary Lijun Zhang Yuriy Brun Hui Guan 37 3 0 24 May 2024
Time-FFM: Towards LM-Empowered Federated Foundation Model for Time Series Forecasting Qingxiang Liu Xu Liu Chenghao Liu Qingsong Wen Yuxuan Liang AI4TS AI4CE 32 6 0 23 May 2024
FeDeRA:Efficient Fine-tuning of Language Models in Federated Learning Leveraging Weight Decomposition Yuxuan Yan Qianqian Yang Shunpu Tang Zhiguo Shi 22 13 0 29 Apr 2024
Advances and Open Challenges in Federated Learning with Foundation Models Chao Ren Han Yu Hongyi Peng Xiaoli Tang Anran Li ... A. Tan Bo Zhao Xiaoxiao Li Zengxiang Li Qiang Yang FedML AIFin AI4CE 68 6 0 23 Apr 2024
FedPFT: Federated Proxy Fine-Tuning of Foundation Models Zhaopeng Peng Xiaoliang Fan Yufan Chen Zheng Wang Shirui Pan Chenglu Wen Ruisheng Zhang Cheng-i Wang 62 8 0 17 Apr 2024
Dual-Personalizing Adapter for Federated Foundation Models Yiyuan Yang Guodong Long Taoshu Shen Jing Jiang Michael Blumenstein 86 19 0 28 Mar 2024
The Risk of Federated Learning to Skew Fine-Tuning Features and Underperform Out-of-Distribution Robustness Mengyao Du Miao Zhang Yuwen Pu Kai Xu Shouling Ji Quanjun Yin 16 1 0 25 Jan 2024
Heterogeneous LoRA for Federated Fine-tuning of On-Device Foundation Models Yae Jee Cho Luyang Liu Zheng Xu Aldi Fahrezi Gauri Joshi 14 45 0 12 Jan 2024
Orchestration of Emulator Assisted Mobile Edge Tuning for AI Foundation Models: A Multi-Agent Deep Reinforcement Learning Approach Wen-li Yu Terence Jie Chua Junfeng Zhao 16 2 0 26 Oct 2023
FedPEAT: Convergence of Federated Learning, Parameter-Efficient Fine Tuning, and Emulator Assisted Tuning for Artificial Intelligence Foundation Models with Mobile Edge Computing Terence Jie Chua Wen-li Yu Junfeng Zhao Kwok-Yan Lam FedML 14 5 0 26 Oct 2023
Privacy in Large Language Models: Attacks, Defenses and Future Directions Haoran Li Yulin Chen Jinglong Luo Yan Kang Xiaojin Zhang Qi Hu Chunkit Chan Yangqiu Song PILM 38 40 0 16 Oct 2023
Reusing Pretrained Models by Multi-linear Operators for Efficient Training Yu Pan Ye Yuan Yichun Yin Zenglin Xu Lifeng Shang Xin Jiang Qun Liu 37 16 0 16 Oct 2023
Federated Fine-Tuning of LLMs on the Very Edge: The Good, the Bad, the Ugly Herbert Woisetschläger Alexander Erben Shiqiang Wang R. Mayer Hans-Arno Jacobsen FedML 24 17 0 04 Oct 2023
FedJudge: Federated Legal Large Language Model Linan Yue Qi Liu Yichao Du Weibo Gao Ye Liu Fangzhou Yao ELM AILaw FedML 14 8 0 15 Sep 2023
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 303 11,881 0 04 Mar 2022
P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Tasks Xiao Liu Kaixuan Ji Yicheng Fu Weng Lam Tam Zhengxiao Du Zhilin Yang Jie Tang VLM 236 804 0 14 Oct 2021
The Power of Scale for Parameter-Efficient Prompt Tuning Brian Lester Rami Al-Rfou Noah Constant VPVLM 280 3,835 0 18 Apr 2021