FederatedScope-LLM: A Comprehensive Package for Fine-tuning Large Language Models in Federated Learning

1 September 2023

Jingren Zhou

Papers citing "FederatedScope-LLM: A Comprehensive Package for Fine-tuning Large Language Models in Federated Learning"

36 / 86 papers shown

Title
The Future of Large Language Model Pre-training is Federated Lorenzo Sani Alexandru Iacob Zeyu Cao Bill Marino Yan Gao ... Wanru Zhao William F. Shen Preslav Aleksandrov Xinchi Qiu Nicholas D. Lane AI4CE 35 13 0 17 May 2024
Snake Learning: A Communication- and Computation-Efficient Distributed Learning Framework for 6G Xiaoxue Yu Xingfu Yi Rongpeng Li Fei Wang Chenghui Peng Zhifeng Zhao Honggang Zhang 72 1 0 06 May 2024
Advances and Open Challenges in Federated Learning with Foundation Models Chao Ren Han Yu Hongyi Peng Xiaoli Tang Anran Li ... A. Tan Bo Zhao Xiaoxiao Li Zengxiang Li Qiang Yang FedML AIFin AI4CE 78 7 0 23 Apr 2024
FedPFT: Federated Proxy Fine-Tuning of Foundation Models Zhaopeng Peng Xiaoliang Fan Yufan Chen Zheng Wang Shirui Pan Chenglu Wen Ruisheng Zhang Cheng-i Wang 69 8 0 17 Apr 2024
pfl-research: simulation framework for accelerating research in Private Federated Learning Filip Granqvist Congzheng Song Áine Cahill Rogier van Dalen Martin Pelikan Yi Sheng Chan Xiaojun Feng Natarajan Krishnaswami Vojta Jina Mona Chitnis FedML 39 6 0 09 Apr 2024
Dual-Personalizing Adapter for Federated Foundation Models Yiyuan Yang Guodong Long Taoshu Shen Jing Jiang Michael Blumenstein 94 19 0 28 Mar 2024
Improving LoRA in Privacy-preserving Federated Learning Youbang Sun Zitao Li Yaliang Li Bolin Ding 32 57 0 18 Mar 2024
Large language models in 6G security: challenges and opportunities Tri Nguyen Huong Nguyen Ahmad Ijaz Saeid Sheikhi Athanasios V. Vasilakos Panos Kostakos ELM 28 8 0 18 Mar 2024
OpenFedLLM: Training Large Language Models on Decentralized Private Data via Federated Learning Rui Ye Wenhao Wang Jingyi Chai Dihan Li Zexi Li Yinda Xu Yaxin Du Yanfeng Wang Siheng Chen ALM FedML AIFin 11 76 0 10 Feb 2024
On the Convergence of Zeroth-Order Federated Tuning for Large Language Models Zhenqing Ling Daoyuan Chen Liuyi Yao Yaliang Li Ying Shen FedML 47 12 0 08 Feb 2024
Leveraging Chat-Based Large Vision Language Models for Multimodal Out-Of-Context Detection Fatma Shalabi Hichem Felouat H. Nguyen Isao Echizen MLLM 33 3 0 22 Jan 2024
A Fast, Performant, Secure Distributed Training Framework For Large Language Model Wei Huang Yinggui Wang Anda Cheng Aihui Zhou Chaofan Yu Lei Wang ALM 25 14 0 18 Jan 2024
A Survey on Efficient Federated Learning Methods for Foundation Model Training Herbert Woisetschläger Alexander Isenko Shiqiang Wang R. Mayer Hans-Arno Jacobsen FedML 65 23 0 09 Jan 2024
From Google Gemini to OpenAI Q* (Q-Star): A Survey of Reshaping the Generative Artificial Intelligence (AI) Research Landscape Timothy R. McIntosh Teo Susnjak Tong Liu Paul Watters Malka N. Halgamuge 91 46 0 18 Dec 2023
Federated Full-Parameter Tuning of Billion-Sized Language Models with Communication Cost under 18 Kilobytes Zhen Qin Daoyuan Chen Bingchen Qian Bolin Ding Yaliang Li Shuiguang Deng FedML 34 32 0 11 Dec 2023
Large Scale Foundation Models for Intelligent Manufacturing Applications: A Survey Haotian Zhang S. D. Semujju Zhicheng Wang Xianwei Lv Kang Xu ... Jing Wu Zhuo Long Wensheng Liang Xiaoguang Ma Ruiyan Zhuang UQCV AI4TS AI4CE 29 4 0 11 Dec 2023
A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Ugly Yifan Yao Jinhao Duan Kaidi Xu Yuanfang Cai Eric Sun Yue Zhang PILM ELM 39 475 0 04 Dec 2023
Grounding Foundation Models through Federated Transfer Learning: A General Framework Yan Kang Tao Fan Hanlin Gu Xiaojin Zhang Lixin Fan Qiang Yang AI4CE 68 19 0 29 Nov 2023
Tunable Soft Prompts are Messengers in Federated Learning Chenhe Dong Yuexiang Xie Bolin Ding Ying Shen Yaliang Li FedML 43 7 0 12 Nov 2023
Personalized Federated Learning via ADMM with Moreau Envelope Shengkun Zhu Jinshan Zeng Sheng Wang Yuan Sun Zhiyong Peng 28 0 0 12 Nov 2023
FedPEAT: Convergence of Federated Learning, Parameter-Efficient Fine Tuning, and Emulator Assisted Tuning for Artificial Intelligence Foundation Models with Mobile Edge Computing Terence Jie Chua Wen-li Yu Junfeng Zhao Kwok-Yan Lam FedML 26 5 0 26 Oct 2023
BC4LLM: Trusted Artificial Intelligence When Blockchain Meets Large Language Models Haoxiang Luo Jian Luo Athanasios V. Vasilakos 34 9 0 10 Oct 2023
ZooPFL: Exploring Black-box Foundation Models for Personalized Federated Learning Wang Lu Hao Yu Jindong Wang Damien Teney Haohan Wang Yiqiang Chen Qiang Yang Xing Xie Xiangyang Ji 70 8 0 08 Oct 2023
Towards Open Federated Learning Platforms: Survey and Vision from Technical and Legal Perspectives Moming Duan Qinbin Li Linshan Jiang Bingsheng He FedML 34 4 0 05 Jul 2023
When Foundation Model Meets Federated Learning: Motivations, Challenges, and Future Directions Weiming Zhuang Chen Chen Lingjuan Lyu Cheng Chen Yaochu Jin Lingjuan Lyu AIFin AI4CE 99 85 0 27 Jun 2023
Artificial General Intelligence for Medical Imaging Xiang Li Lu Zhang Zihao Wu Zheng Liu Lin Zhao ... Pingkuan Yan Quanzheng Li Wei Liu Tianming Liu Dinggang Shen LM&MA AI4CE 19 40 0 08 Jun 2023
FS-Real: Towards Real-World Cross-Device Federated Learning Daoyuan Chen Dawei Gao Yuexiang Xie Xuchen Pan Zitao Li Yaliang Li Bolin Ding Jingren Zhou 117 26 0 23 Mar 2023
GLM-130B: An Open Bilingual Pre-trained Model Aohan Zeng Xiao Liu Zhengxiao Du Zihan Wang Hanyu Lai ... Jidong Zhai Wenguang Chen Peng-Zhen Zhang Yuxiao Dong Jie Tang BDL LRM 250 1,073 0 05 Oct 2022
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Jason W. Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Brian Ichter F. Xia Ed H. Chi Quoc Le Denny Zhou LM&Ro LRM AI4CE ReLM 379 8,495 0 28 Jan 2022
P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Tasks Xiao Liu Kaixuan Ji Yicheng Fu Weng Lam Tam Zhengxiao Du Zhilin Yang Jie Tang VLM 238 806 0 14 Oct 2021
RAFT: A Real-World Few-Shot Text Classification Benchmark Neel Alex Eli Lifland Lewis Tunstall A. Thakur Pegah Maham ... Carolyn Ashurst Paul Sedille A. Carlier M. Noetel Andreas Stuhlmuller RALM 184 56 0 28 Sep 2021
The Power of Scale for Parameter-Efficient Prompt Tuning Brian Lester Rami Al-Rfou Noah Constant VPVLM 280 3,848 0 18 Apr 2021
Towards Personalized Federated Learning A. Tan Han Yu Li-zhen Cui Qiang Yang FedML AI4CE 209 840 0 01 Mar 2021
ZeRO-Offload: Democratizing Billion-Scale Model Training Jie Ren Samyam Rajbhandari Reza Yazdani Aminabadi Olatunji Ruwase Shuangyang Yang Minjia Zhang Dong Li Yuxiong He MoE 177 414 0 18 Jan 2021
Federated Bayesian Optimization via Thompson Sampling Zhongxiang Dai K. H. Low Patrick Jaillet FedML 89 109 0 20 Oct 2020
Teaching Machines to Read and Comprehend Karl Moritz Hermann Tomás Kociský Edward Grefenstette L. Espeholt W. Kay Mustafa Suleyman Phil Blunsom 175 3,510 0 10 Jun 2015