FedBiOT: LLM Local Fine-tuning in Federated Learning without Full Model

25 June 2024

Feijie Wu

Zitao Li

Yaliang Li

Bolin Ding

Jing Gao

ArXiv PDF HTML

Papers citing "FedBiOT: LLM Local Fine-tuning in Federated Learning without Full Model"

31 / 31 papers shown

Title
Communication-Efficient Federated Fine-Tuning of Language Models via Dynamic Update Schedules Michail Theologitis V. Samoladas Antonios Deligiannakis 20 0 0 07 May 2025
HSplitLoRA: A Heterogeneous Split Parameter-Efficient Fine-Tuning Framework for Large Language Models Zheng Lin Yuxin Zhang Zhe Chen Zihan Fang Xianhao Chen Praneeth Vepakomma Wei Ni Jun-Jie Luo Yue Gao MoE 32 0 0 05 May 2025
A Survey on Parameter-Efficient Fine-Tuning for Foundation Models in Federated Learning Jieming Bian Yuanzhe Peng Lei Wang Yin Huang Jie Xu FedML 55 0 0 29 Apr 2025
Towards Harnessing the Collaborative Power of Large and Small Models for Domain Tasks Yang Janet Liu Bingjie Yan Tianyuan Zou Jianqing Zhang Zixuan Gu ... J. Li Xiaozhou Ye Ye Ouyang Qiang Yang Y. Zhang ALM 59 0 0 24 Apr 2025
Towards Resilient Federated Learning in CyberEdge Networks: Recent Advances and Future Trends Kai Li Zhengyang Zhang Azadeh Pourkabirian Wei Ni Falko Dressler Ozgur B. Akan 43 0 0 01 Apr 2025
Communication-Efficient and Personalized Federated Foundation Model Fine-Tuning via Tri-Matrix Adaptation Y. Li Bo Liu Sheng Huang Z. Zhang Xiaotong Yuan Richang Hong 41 0 0 31 Mar 2025
A Survey on Federated Fine-tuning of Large Language Models Yebo Wu Chunlin Tian Jingguang Li He Sun Kahou Tam Li Li Chengzhong Xu FedML 81 0 0 15 Mar 2025
FedALT: Federated Fine-Tuning through Adaptive Local Training with Rest-of-the-World LoRA Jieming Bian Lei Wang Letian Zhang Jie Xu 50 1 0 14 Mar 2025
Efficient Federated Fine-Tuning of Large Language Models with Layer Dropout Shilong Wang Jianchun Liu Hongli Xu Jiaming Yan Xianjun Gao 45 0 0 13 Mar 2025
PluralLLM: Pluralistic Alignment in LLMs via Federated Learning Mahmoud Srewa Tianyu Zhao Salma Elmalaki FedML 48 0 0 13 Mar 2025
FedMABench: Benchmarking Mobile Agents on Decentralized Heterogeneous User Data Wenhao Wang Zijie Yu Rui Ye J. Zhang S. Chen Yanfeng Wang FedML 45 0 0 07 Mar 2025
Can Textual Gradient Work in Federated Learning? Minghui Chen Ruinan Jin Wenlong Deng Yuanyuan Chen Zhi Huang Han Yu Xiaoxiao Li FedML 79 2 0 27 Feb 2025
Ten Challenging Problems in Federated Foundation Models Tao Fan Hanlin Gu Xuemei Cao Chee Seng Chan Qian Chen ... Y. Zhang Xiaojin Zhang Zhenzhe Zheng Lixin Fan Qiang Yang FedML 73 3 0 14 Feb 2025
A Hybrid Swarm Intelligence Approach for Optimizing Multimodal Large Language Models Deployment in Edge-Cloud-based Federated Learning Environments Gaith Rjouba Hanae Elmekki Saidul Islam Jamal Bentahar Rachida Dssouli 36 0 0 04 Feb 2025
FedTLU: Federated Learning with Targeted Layer Updates Jong-Ik Park Carlee Joe-Wong FedML 82 0 0 28 Jan 2025
Navigating the Designs of Privacy-Preserving Fine-tuning for Large Language Models Haonan Shi Tu Ouyang An Wang 31 0 0 08 Jan 2025
Large Language Model Federated Learning with Blockchain and Unlearning for Cross-Organizational Collaboration Xuhan Zuo Minghao Wang Tianqing Zhu Shui Yu Wanlei Zhou MU 64 2 0 18 Dec 2024
Review of Mathematical Optimization in Federated Learning Shusen Yang Fangyuan Zhao Zihao Zhou Liang Shi Xuebin Ren Zongben Xu FedML AI4CE 67 0 0 02 Dec 2024
Personalized Federated Fine-Tuning for LLMs via Data-Driven Heterogeneous Model Architectures Yicheng Zhang Zhen Qin Zhaomin Wu Shuiguang Deng 70 2 0 28 Nov 2024
When IoT Meet LLMs: Applications and Challenges İbrahim Kök Orhan Demirci S. Özdemir 65 4 0 20 Nov 2024
Federated Data-Efficient Instruction Tuning for Large Language Models Zhen Qin Zhaomin Wu Bingsheng He Shuiguang Deng FedML 35 2 0 14 Oct 2024
FedEP: Tailoring Attention to Heterogeneous Data Distribution with Entropy Pooling for Decentralized Federated Learning Chao Feng Hongjie Guan Alberto Huertas Celdrán Jan von der Assen Gérome Bovet Burkhard Stiller FedML 23 1 0 10 Oct 2024
Selective Aggregation for Low-Rank Adaptation in Federated Learning Pengxin Guo Shuang Zeng Y. Wang Huijie Fan Feifei Wang Liangqiong Qu FedML 34 8 0 02 Oct 2024
Hierarchical Federated Learning with Multi-Timescale Gradient Correction Wenzhi Fang Dong-Jun Han Evan Chen Shiqiang Wang Christopher G. Brinton 19 4 0 27 Sep 2024
Federated Large Language Models: Current Progress and Future Directions Yuhang Yao Jianyi Zhang Junda Wu Chengkai Huang Yu Xia ... Ang Li Lina Yao Julian McAuley Yiran Chen Carlee Joe-Wong FedML AIFin 61 8 0 24 Sep 2024
Self-Introspective Decoding: Alleviating Hallucinations for Large Vision-Language Models Fushuo Huo Wenchao Xu Zhong Zhang Haozhao Wang Zhicheng Chen Peilin Zhao VLM MLLM 58 18 0 04 Aug 2024
FIARSE: Model-Heterogeneous Federated Learning via Importance-Aware Submodel Extraction Feijie Wu Xingchen Wang Yaqing Wang Tianci Liu Lu Su Jing Gao FedML 43 3 0 28 Jul 2024
When Foundation Model Meets Federated Learning: Motivations, Challenges, and Future Directions Weiming Zhuang Chen Chen Lingjuan Lyu C. L. P. Chen Yaochu Jin Lingjuan Lyu AIFin AI4CE 83 84 0 27 Jun 2023
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 301 11,730 0 04 Mar 2022
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Jason W. Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Brian Ichter F. Xia Ed H. Chi Quoc Le Denny Zhou LM&Ro LRM AI4CE ReLM 315 8,261 0 28 Jan 2022
The Power of Scale for Parameter-Efficient Prompt Tuning Brian Lester Rami Al-Rfou Noah Constant VPVLM 278 3,784 0 18 Apr 2021