Heterogeneous LoRA for Federated Fine-tuning of On-Device Foundation Models

12 January 2024

Papers citing "Heterogeneous LoRA for Federated Fine-tuning of On-Device Foundation Models"

35 / 35 papers shown

Title
HSplitLoRA: A Heterogeneous Split Parameter-Efficient Fine-Tuning Framework for Large Language Models Zheng Lin Yuxin Zhang Zhe Chen Zihan Fang Xianhao Chen Praneeth Vepakomma Wei Ni Jun-Jie Luo Yue Gao MoE 32 1 0 05 May 2025
Federated Adapter on Foundation Models: An Out-Of-Distribution Approach Yiyuan Yang Guodong Long Tianyi Zhou Qinghua Lu Shanshan Ye Jing Jiang OODD 84 1 0 02 May 2025
Communication-Efficient Wireless Federated Fine-Tuning for Large-Scale AI Models Bumjun Kim Wan Choi 26 0 0 01 May 2025
A Survey on Parameter-Efficient Fine-Tuning for Foundation Models in Federated Learning Jieming Bian Yuanzhe Peng Lei Wang Yin Huang Jie Xu FedML 55 0 0 29 Apr 2025
Collaborative Learning of On-Device Small Model and Cloud-Based Large Model: Advances and Future Directions Chaoyue Niu Yucheng Ding Junhui Lu Zhengxiang Huang Hang Zeng Yutong Dai Xuezhen Tu Chengfei Lv Fan Wu Guihai Chen 27 1 0 17 Apr 2025
FedMerge: Federated Personalization via Model Merging Shutong Chen Tianyi Zhou Guodong Long Jing Jiang Chengqi Zhang FedML MoMe 47 0 0 09 Apr 2025
A Survey on Federated Fine-tuning of Large Language Models Yebo Wu Chunlin Tian Jingguang Li He Sun Kahou Tam Li Li Chengzhong Xu FedML 81 0 0 15 Mar 2025
FedALT: Federated Fine-Tuning through Adaptive Local Training with Rest-of-the-World LoRA Jieming Bian Lei Wang Letian Zhang Jie Xu 50 1 0 14 Mar 2025
Efficient Federated Fine-Tuning of Large Language Models with Layer Dropout Shilong Wang Jianchun Liu Hongli Xu Jiaming Yan Xianjun Gao 45 1 0 13 Mar 2025
Federated Multimodal Learning with Dual Adapters and Selective Pruning for Communication and Computational Efficiency Duy Phuong Nguyen J. P. Muñoz Tanya Roosta Ali Jannesari FedML 59 0 0 10 Mar 2025
CLLoRA: An Approach to Measure the Effects of the Context Length for LLM Fine-Tuning Ping Zhang Zhaorui Zhang Sheng Di Yao Xin Benben Liu 43 1 0 26 Feb 2025
Fed-SB: A Silver Bullet for Extreme Communication Efficiency and Performance in (Private) Federated LoRA Fine-Tuning Raghav Singhal Kaustubh Ponkshe Rohit Vartak Lav R. Varshney Praneeth Vepakomma FedML 74 0 0 24 Feb 2025
Decentralized Low-Rank Fine-Tuning of Large Language Models Sajjad Ghiasvand Mahnoosh Alizadeh Ramtin Pedarsani ALM 64 0 0 26 Jan 2025
Aggregating Low Rank Adapters in Federated Fine-tuning Evelyn Trautmann Ian Hales Martin F. Volk AI4CE FedML 34 0 0 10 Jan 2025
Efficient Federated Finetuning of Tiny Transformers with Resource-Constrained Devices Kilian Pfeiffer Mohamed Aboelenien Ahmed R. Khalili J. Henkel 30 0 0 12 Nov 2024
Federated LLMs Fine-tuned with Adaptive Importance-Aware LoRA Yang Su Na Yan Yansha Deng 21 0 0 10 Nov 2024
Communication-Efficient and Tensorized Federated Fine-Tuning of Large Language Models Sajjad Ghiasvand Yifan Yang Zhiyu Xue Mahnoosh Alizadeh Zheng Zhang Ramtin Pedarsani FedML 27 3 0 16 Oct 2024
DEeR: Deviation Eliminating and Noise Regulating for Privacy-preserving Federated Low-rank Adaptation Meilu Zhu Axiu Mao Jun Liu Yixuan Yuan 32 1 0 16 Oct 2024
Federated Learning in Practice: Reflections and Projections Katharine Daly Hubert Eichner Peter Kairouz H. B. McMahan Daniel Ramage Zheng Xu FedML 53 5 0 11 Oct 2024
Selective Aggregation for Low-Rank Adaptation in Federated Learning Pengxin Guo Shuang Zeng Y. Wang Huijie Fan Feifei Wang Liangqiong Qu FedML 34 8 0 02 Oct 2024
On-Device Collaborative Language Modeling via a Mixture of Generalists and Specialists Dongyang Fan Bettina Messmer N. Doikov Martin Jaggi MoMe MoE 42 1 0 20 Sep 2024
Communication-Efficient Federated Low-Rank Update Algorithm and its Connection to Implicit Regularization Haemin Park Diego Klabjan FedML 27 0 0 19 Sep 2024
Leveraging Unstructured Text Data for Federated Instruction Tuning of Large Language Models Rui Ye Rui Ge Yuchi Fengting Jingyi Chai Yanfeng Wang Siheng Chen FedML 27 1 0 11 Sep 2024
FLoRA: Federated Fine-Tuning Large Language Models with Heterogeneous Low-Rank Adaptations Ziyao Wang Zheyu Shen Yexiao He Guoheng Sun Hongyi Wang Lingjuan Lyu Ang Li 26 25 0 09 Sep 2024
RBLA: Rank-Based-LoRA-Aggregation for Fine-tuning Heterogeneous Models in FLaaS Shuaijun Chen Omid Tavallaie Niousha Nazemi Albert Y. Zomaya 34 3 0 16 Aug 2024
A Survey on LoRA of Large Language Models Yuren Mao Yuhang Ge Yijiang Fan Wenyi Xu Yu Mi Zhonghao Hu Yunjun Gao ALM 52 22 0 08 Jul 2024
FLoCoRA: Federated learning compression with low-rank adaptation Lucas Grativol Ribeiro Mathieu Léonardon Guillaume Muller Virginie Fresse Matthieu Arzel AI4CE 19 1 0 20 Jun 2024
Synergizing Foundation Models and Federated Learning: A Survey Shenghui Li Fanghua Ye Meng Fang Jiaxu Zhao Yun-Hin Chan Edith C. -H. Ngai Thiemo Voigt AI4CE 43 5 0 18 Jun 2024
Federated LoRA with Sparse Communication Kevin Kuo Arian Raje Kousik Rajesh Virginia Smith 38 7 0 07 Jun 2024
SA-FedLora: Adaptive Parameter Allocation for Efficient Federated Learning with LoRA Tuning Yuning Yang Xiaohong Liu Tianrun Gao Xiaodong Xu Guangyu Wang 33 5 0 15 May 2024
Grounding Foundation Models through Federated Transfer Learning: A General Framework Yan Kang Tao Fan Hanlin Gu Xiaojin Zhang Lixin Fan Qiang Yang AI4CE 68 19 0 29 Nov 2023
Large Language Models are Zero-Shot Reasoners Takeshi Kojima S. Gu Machel Reid Yutaka Matsuo Yusuke Iwasawa ReLM LRM 291 4,048 0 24 May 2022
A Field Guide to Federated Optimization Jianyu Wang Zachary B. Charles Zheng Xu Gauri Joshi H. B. McMahan ... Mi Zhang Tong Zhang Chunxiang Zheng Chen Zhu Wennan Zhu FedML 167 410 0 14 Jul 2021
The Power of Scale for Parameter-Efficient Prompt Tuning Brian Lester Rami Al-Rfou Noah Constant VPVLM 278 3,784 0 18 Apr 2021
Adaptive Federated Learning in Resource Constrained Edge Computing Systems Shiqiang Wang Tiffany Tuor Theodoros Salonidis K. Leung C. Makaya T. He Kevin S. Chan 141 1,663 0 14 Apr 2018