Federated Full-Parameter Tuning of Billion-Sized Language Models with Communication Cost under 18 Kilobytes

11 December 2023

Zhen Qin

Papers citing "Federated Full-Parameter Tuning of Billion-Sized Language Models with Communication Cost under 18 Kilobytes"

30 / 30 papers shown

Title
Communication-Efficient and Personalized Federated Foundation Model Fine-Tuning via Tri-Matrix Adaptation Y. Li Bo Liu Sheng Huang Z. Zhang Xiaotong Yuan Richang Hong 32 0 0 31 Mar 2025
A Survey on Federated Fine-tuning of Large Language Models Yebo Wu Chunlin Tian Jingguang Li He Sun Kahou Tam Li Li Chengzhong Xu FedML 78 0 0 15 Mar 2025
PriFFT: Privacy-preserving Federated Fine-tuning of Large Language Models via Hybrid Secret Sharing Zhichao You Xuewen Dong Ke Cheng Xutong Mu Jiaxuan Fu Shiyang Ma Qiang Qu Yulong Shen FedML 76 0 0 05 Mar 2025
Can Textual Gradient Work in Federated Learning? Minghui Chen Ruinan Jin Wenlong Deng Yuanyuan Chen Zhi Huang Han Yu Xiaoxiao Li FedML 68 2 0 27 Feb 2025
Ten Challenging Problems in Federated Foundation Models Tao Fan Hanlin Gu Xuemei Cao Chee Seng Chan Qian Chen ... Y. Zhang Xiaojin Zhang Zhenzhe Zheng Lixin Fan Qiang Yang FedML 73 3 0 14 Feb 2025
Vertical Federated Learning in Practice: The Good, the Bad, and the Ugly Zhaomin Wu Zhen Qin Junyi Hou Haodong Zhao Qinbin Li Bingsheng He Lixin Fan FedML 59 0 0 12 Feb 2025
ElasticZO: A Memory-Efficient On-Device Learning with Combined Zeroth- and First-Order Optimization Keisuke Sugiura Hiroki Matsutani MQ 36 1 0 08 Jan 2025
Personalized Federated Fine-Tuning for LLMs via Data-Driven Heterogeneous Model Architectures Yicheng Zhang Zhen Qin Zhaomin Wu Shuiguang Deng 65 2 0 28 Nov 2024
Federated Data-Efficient Instruction Tuning for Large Language Models Zhen Qin Zhaomin Wu Bingsheng He Shuiguang Deng FedML 32 2 0 14 Oct 2024
Federated Large Language Models: Current Progress and Future Directions Yuhang Yao Jianyi Zhang Junda Wu Chengkai Huang Yu Xia ... Ang Li Lina Yao Julian McAuley Yiran Chen Carlee Joe-Wong FedML AIFin 59 8 0 24 Sep 2024
Ferret: Federated Full-Parameter Tuning at Scale for Large Language Models Yao Shu Wenyang Hu S. Ng Bryan Kian Hsiang Low Fei Richard Yu FedML 27 0 0 10 Sep 2024
Fine-Tuning and Deploying Large Language Models Over Edges: Issues and Approaches Yanjie Dong Xiaoyi Fan Fangxin Wang Chengming Li Victor C. M. Leung Xiping Hu 21 4 0 20 Aug 2024
The Synergy between Data and Multi-Modal Large Language Models: A Survey from Co-Development Perspective Zhen Qin Daoyuan Chen Wenhao Zhang Liuyi Yao Yilun Huang Bolin Ding Yaliang Li Shuiguang Deng 45 5 0 11 Jul 2024
On the Limitations of Compute Thresholds as a Governance Strategy Sara Hooker 37 14 0 08 Jul 2024
Synergizing Foundation Models and Federated Learning: A Survey Shenghui Li Fanghua Ye Meng Fang Jiaxu Zhao Yun-Hin Chan Edith C. -H. Ngai Thiemo Voigt AI4CE 40 5 0 18 Jun 2024
FedRDMA: Communication-Efficient Cross-Silo Federated LLM via Chunked RDMA Transmission Zeling Zhang Dongqi Cai Yiran Zhang Mengwei Xu Shangguang Wang Ao Zhou 25 4 0 01 Mar 2024
On the Convergence of Zeroth-Order Federated Tuning for Large Language Models Zhenqing Ling Daoyuan Chen Liuyi Yao Yaliang Li Ying Shen FedML 43 12 0 08 Feb 2024
A Survey on Efficient Federated Learning Methods for Foundation Model Training Herbert Woisetschläger Alexander Isenko Shiqiang Wang R. Mayer Hans-Arno Jacobsen FedML 49 21 0 09 Jan 2024
EvoFed: Leveraging Evolutionary Strategies for Communication-Efficient Federated Learning Mohammad Mahdi Rahimi Hasnain Irshad Bhatti Younghyun Park Humaira Kousar Jaekyun Moon FedML 24 2 0 13 Nov 2023
Federated Fine-Tuning of LLMs on the Very Edge: The Good, the Bad, the Ugly Herbert Woisetschläger Alexander Erben Shiqiang Wang R. Mayer Hans-Arno Jacobsen FedML 17 17 0 04 Oct 2023
FwdLLM: Efficient FedLLM using Forward Gradient Mengwei Xu Dongqi Cai Yaozong Wu Xiang Li Shangguang Wang FedML 47 24 0 26 Aug 2023
Towards Open Federated Learning Platforms: Survey and Vision from Technical and Legal Perspectives Moming Duan Qinbin Li Linshan Jiang Bingsheng He FedML 21 4 0 05 Jul 2023
Just One Byte (per gradient): A Note on Low-Bandwidth Decentralized Language Model Finetuning Using Shared Randomness E. Zelikman Qian Huang Percy Liang Nick Haber Noah D. Goodman 57 14 0 16 Jun 2023
Decentralized Federated Learning: A Survey and Perspective Liangqi Yuan Ziran Wang Lichao Sun Philip S. Yu Christopher G. Brinton FedML 30 74 0 02 Jun 2023
Efficient Personalized Federated Learning via Sparse Model-Adaptation Daoyuan Chen Fandong Meng Dawei Gao Bolin Ding Yaliang Li FedML 96 45 0 04 May 2023
FS-Real: Towards Real-World Cross-Device Federated Learning Daoyuan Chen Dawei Gao Yuexiang Xie Xuchen Pan Zitao Li Yaliang Li Bolin Ding Jingren Zhou 100 26 0 23 Mar 2023
Communication-Efficient Stochastic Zeroth-Order Optimization for Federated Learning Wenzhi Fang Ziyi Yu Yuning Jiang Yuanming Shi Colin N. Jones Yong Zhou FedML 68 53 0 24 Jan 2022
The Power of Scale for Parameter-Efficient Prompt Tuning Brian Lester Rami Al-Rfou Noah Constant VPVLM 275 3,784 0 18 Apr 2021
Towards Personalized Federated Learning A. Tan Han Yu Li-zhen Cui Qiang Yang FedML AI4CE 177 832 0 01 Mar 2021
Federated Learning on Non-IID Data Silos: An Experimental Study Q. Li Yiqun Diao Quan Chen Bingsheng He FedML OOD 82 936 0 03 Feb 2021