Federated Learning of Large Language Models with Parameter-Efficient
Prompt Tuning and Adaptive Optimization

Federated Learning of Large Language Models with Parameter-Efficient Prompt Tuning and Adaptive Optimization

23 October 2023

Victor S. Sheng

Papers citing "Federated Learning of Large Language Models with Parameter-Efficient Prompt Tuning and Adaptive Optimization"

14 / 14 papers shown

Title
HSplitLoRA: A Heterogeneous Split Parameter-Efficient Fine-Tuning Framework for Large Language Models Zheng Lin Yuxin Zhang Zhe Chen Zihan Fang Xianhao Chen Praneeth Vepakomma Wei Ni Jun-Jie Luo Yue Gao MoE 34 1 0 05 May 2025
Mitigating Group-Level Fairness Disparities in Federated Visual Language Models Chaomeng Chen Zitong Yu J. Dong Sen Su L. Shen Shutao Xia Xiaochun Cao FedML VLM 102 0 0 03 May 2025
Communication-Efficient and Personalized Federated Foundation Model Fine-Tuning via Tri-Matrix Adaptation Y. Li Bo Liu Sheng Huang Z. Zhang Xiaotong Yuan Richang Hong 41 0 0 31 Mar 2025
Selective Aggregation for Low-Rank Adaptation in Federated Learning Pengxin Guo Shuang Zeng Y. Wang Huijie Fan Feifei Wang Liangqiong Qu FedML 36 8 0 02 Oct 2024
Mobile Edge Intelligence for Large Language Models: A Contemporary Survey Guanqiao Qu Qiyuan Chen Wei Wei Zheng Lin Xianhao Chen Kaibin Huang 35 41 0 09 Jul 2024
Save It All: Enabling Full Parameter Tuning for Federated Large Language Models via Cycle Block Gradient Descent Lin Wang Zhichao Wang Xiaoying Tang 29 1 0 17 Jun 2024
Recent Advances in Federated Learning Driven Large Language Models: A Survey on Architecture, Performance, and Security Youyang Qu Ming Liu Tianqing Zhu Longxiang Gao Shui Yu Wanlei Zhou MU FedML 54 2 0 14 Jun 2024
Personalized Collaborative Fine-Tuning for On-Device Large Language Models Nicolas Wagner Dongyang Fan Martin Jaggi 37 3 0 15 Apr 2024
On Mask-based Image Set Desensitization with Recognition Support Qilong Li Ji Liu Yifan Sun Chongsheng Zhang Dejing Dou CVBM 26 3 0 14 Dec 2023
Federated Full-Parameter Tuning of Billion-Sized Language Models with Communication Cost under 18 Kilobytes Zhen Qin Daoyuan Chen Bingchen Qian Bolin Ding Yaliang Li Shuiguang Deng FedML 32 32 0 11 Dec 2023
ATTEMPT: Parameter-Efficient Multi-task Tuning via Attentional Mixtures of Soft Prompts Akari Asai Mohammadreza Salehi Matthew E. Peters Hannaneh Hajishirzi 120 100 0 24 May 2022
Multitask Prompted Training Enables Zero-Shot Task Generalization Victor Sanh Albert Webson Colin Raffel Stephen H. Bach Lintang Sutawika ... T. Bers Stella Biderman Leo Gao Thomas Wolf Alexander M. Rush LRM 211 1,654 0 15 Oct 2021
The Power of Scale for Parameter-Efficient Prompt Tuning Brian Lester Rami Al-Rfou Noah Constant VPVLM 280 3,835 0 18 Apr 2021
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 294 6,943 0 20 Apr 2018