Towards Building the Federated GPT: Federated Instruction Tuning

9 May 2023

Jianyi Zhang

Saeed Vahidian

Yiran Chen

Papers citing "Towards Building the Federated GPT: Federated Instruction Tuning"

35 / 35 papers shown

Title
FedTDP: A Privacy-Preserving and Unified Framework for Trajectory Data Preparation via Federated Learning Zhihao Zeng Ziquan Fang Wei Shao Lu Chen Yunjun Gao FedML 49 0 0 08 May 2025
HSplitLoRA: A Heterogeneous Split Parameter-Efficient Fine-Tuning Framework for Large Language Models Zheng Lin Yuxin Zhang Zhe Chen Zihan Fang Xianhao Chen Praneeth Vepakomma Wei Ni Jun-Jie Luo Yue Gao MoE 34 1 0 05 May 2025
Federated Adapter on Foundation Models: An Out-Of-Distribution Approach Yiyuan Yang Guodong Long Tianyi Zhou Qinghua Lu Shanshan Ye Jing Jiang OODD 116 1 0 02 May 2025
Communication-Efficient Wireless Federated Fine-Tuning for Large-Scale AI Models Bumjun Kim Wan Choi 26 0 0 01 May 2025
FedMerge: Federated Personalization via Model Merging Shutong Chen Tianyi Zhou Guodong Long Jing Jiang Chengqi Zhang FedML MoMe 47 0 0 09 Apr 2025
Fed-SB: A Silver Bullet for Extreme Communication Efficiency and Performance in (Private) Federated LoRA Fine-Tuning Raghav Singhal Kaustubh Ponkshe Rohit Vartak Lav R. Varshney Praneeth Vepakomma FedML 74 0 0 24 Feb 2025
Proactive Privacy Amnesia for Large Language Models: Safeguarding PII with Negligible Impact on Model Utility Martin Kuo Jingyang Zhang Jianyi Zhang Minxue Tang Louis DiValentin ... William Chen Amin Hass Tianlong Chen Y. Chen H. Li MU KELM 37 2 0 24 Feb 2025
Ten Challenging Problems in Federated Foundation Models Tao Fan Hanlin Gu Xuemei Cao Chee Seng Chan Qian Chen ... Y. Zhang Xiaojin Zhang Zhenzhe Zheng Lixin Fan Qiang Yang FedML 75 4 0 14 Feb 2025
Privacy in Fine-tuning Large Language Models: Attacks, Defenses, and Future Directions Hao Du Shang Liu Lele Zheng Yang Cao Atsuyoshi Nakamura Lei Chen AAML 111 3 0 21 Dec 2024
Federated Instruction Tuning of LLMs with Domain Coverage Augmentation Zezhou Wang Yaxin Du Zhuzhong Qian Yugang Jiang Zhuzhong Qian Siheng Chen FedML 86 0 0 30 Sep 2024
On-Device Collaborative Language Modeling via a Mixture of Generalists and Specialists Dongyang Fan Bettina Messmer N. Doikov Martin Jaggi MoMe MoE 42 1 0 20 Sep 2024
On ADMM in Heterogeneous Federated Learning: Personalization, Robustness, and Fairness Shengkun Zhu Jinshan Zeng Sheng Wang Yuan Sun Xiaodong Li Yuan Yao Zhiyong Peng 36 0 0 23 Jul 2024
Mobile Edge Intelligence for Large Language Models: A Contemporary Survey Guanqiao Qu Qiyuan Chen Wei Wei Zheng Lin Xianhao Chen Kaibin Huang 35 41 0 09 Jul 2024
FDLoRA: Personalized Federated Learning of Large Language Model via Dual LoRA Tuning Jiaxing Qi Zhongzhi Luan Shaohan Huang Carol J. Fung Hailong Yang Depei Qian 32 12 0 12 Jun 2024
DAGER: Exact Gradient Inversion for Large Language Models Ivo Petrov Dimitar I. Dimitrov Maximilian Baader Mark Niklas Muller Martin Vechev FedML 46 3 0 24 May 2024
Personalized Collaborative Fine-Tuning for On-Device Large Language Models Nicolas Wagner Dongyang Fan Martin Jaggi 37 3 0 15 Apr 2024
Federated Full-Parameter Tuning of Billion-Sized Language Models with Communication Cost under 18 Kilobytes Zhen Qin Daoyuan Chen Bingchen Qian Bolin Ding Yaliang Li Shuiguang Deng FedML 32 32 0 11 Dec 2023
A Survey on Privacy in Graph Neural Networks: Attacks, Preservation, and Applications Yi Zhang Yuying Zhao Zhaoqing Li Xueqi Cheng Yu-Chiang Frank Wang Olivera Kotevska Philip S. Yu Tyler Derr 18 10 0 31 Aug 2023
Federated Few-shot Learning Song Wang Xingbo Fu Kaize Ding Chen Chen Huiyuan Chen Jundong Li FedML 42 22 0 17 Jun 2023
Instruction Tuning with GPT-4 Baolin Peng Chunyuan Li Pengcheng He Michel Galley Jianfeng Gao SyDa ALM LM&MA 157 579 0 06 Apr 2023
When Do Curricula Work in Federated Learning? Saeed Vahidian Sreevatsank Kadaveru Woo-Ram Baek Weijia Wang Vyacheslav Kungurtsev C. L. P. Chen M. Shah Bill Lin FedML 30 11 0 24 Dec 2022
Rethinking Normalization Methods in Federated Learning Zhixu Du Jingwei Sun Ang Li Pin-Yu Chen Jianyi Zhang H. Li Yiran Chen FedML 29 28 0 07 Oct 2022
Efficient Distribution Similarity Identification in Clustered Federated Learning via Principal Angles Between Client Data Subspaces Saeed Vahidian Mahdi Morafah Weijia Wang Vyacheslav Kungurtsev C. L. P. Chen M. Shah Bill Lin FedML 37 45 0 21 Sep 2022
Decentralized digital twins of complex dynamical systems Omer San Suraj Pawar Adil Rasheed AI4CE 28 11 0 07 Jul 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 303 11,881 0 04 Mar 2022
A Comprehensive Survey on Federated Learning: Concept and Applications Dhurgham Hassan Mahlool Mohammed Hamzah Abed FedML 23 22 0 23 Jan 2022
Multitask Prompted Training Enables Zero-Shot Task Generalization Victor Sanh Albert Webson Colin Raffel Stephen H. Bach Lintang Sutawika ... T. Bers Stella Biderman Leo Gao Thomas Wolf Alexander M. Rush LRM 211 1,654 0 15 Oct 2021
P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Tasks Xiao Liu Kaixuan Ji Yicheng Fu Weng Lam Tam Zhengxiao Du Zhilin Yang Jie Tang VLM 236 804 0 14 Oct 2021
Federated Natural Language Generation for Personalized Dialogue System Yujie Lu Chao-Wei Huang Huanli Zhan Yong Zhuang FedML 77 9 0 13 Oct 2021
Personalized Federated Learning by Structured and Unstructured Pruning under Data Heterogeneity Saeed Vahidian Mahdi Morafah Bill Lin 54 57 0 02 May 2021
The Power of Scale for Parameter-Efficient Prompt Tuning Brian Lester Rami Al-Rfou Noah Constant VPVLM 280 3,835 0 18 Apr 2021
Straggler-Resilient Federated Learning: Leveraging the Interplay Between Statistical Accuracy and System Heterogeneity Amirhossein Reisizadeh Isidoros Tziotis Hamed Hassani Aryan Mokhtari Ramtin Pedarsani FedML 162 98 0 28 Dec 2020
FedML: A Research Library and Benchmark for Federated Machine Learning Chaoyang He Songze Li Jinhyun So Xiao Zeng Mi Zhang ... Yang Liu Ramesh Raskar Qiang Yang M. Annavaram Salman Avestimehr FedML 162 563 0 27 Jul 2020
FedNER: Privacy-preserving Medical Named Entity Recognition with Federated Learning Suyu Ge Fangzhao Wu Chuhan Wu Tao Qi Yongfeng Huang Xing Xie 156 56 0 20 Mar 2020
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 294 6,943 0 20 Apr 2018