On Large-Cohort Training for Federated Learning

15 June 2021

Papers citing "On Large-Cohort Training for Federated Learning"

50 / 70 papers shown

Title
SparsyFed: Sparse Adaptive Federated Training Adriano Guastella Lorenzo Sani Alex Iacob Alessio Mora Paolo Bellavista Nicholas D. Lane FedML 31 0 0 07 Apr 2025
AIGC-assisted Federated Learning for Edge Intelligence: Architecture Design, Research Challenges and Future Directions Xianke Qiang Zheng Chang Ying-Chang Liang FedML 75 0 0 26 Mar 2025
Communication-Efficient Language Model Training Scales Reliably and Robustly: Scaling Laws for DiLoCo Zachary B. Charles Gabriel Teston Lucio Dery Keith Rush Nova Fallen Zachary Garrett Arthur Szlam Arthur Douillard 122 0 0 12 Mar 2025
Sketched Adaptive Federated Deep Learning: A Sharp Convergence Analysis Zhijie Chen Qiaobo Li A. Banerjee FedML 30 0 0 11 Nov 2024
Photon: Federated LLM Pre-Training Lorenzo Sani Alex Iacob Zeyu Cao Royson Lee Bill Marino ... Dongqi Cai Zexi Li Wanru Zhao Xinchi Qiu Nicholas D. Lane AI4CE 26 7 0 05 Nov 2024
Personalized Federated Learning with Mixture of Models for Adaptive Prediction and Model Fine-Tuning P. M. Ghari Yanning Shen FedML 23 1 0 28 Oct 2024
Tackling Data Heterogeneity in Federated Learning via Loss Decomposition Shuang Zeng Pengxin Guo Shuai Wang Jianbo Wang Yuyin Zhou Liangqiong Qu FedML 29 2 0 22 Aug 2024
Recent Advances in Federated Learning Driven Large Language Models: A Survey on Architecture, Performance, and Security Youyang Qu Ming Liu Tianqing Zhu Longxiang Gao Shui Yu Wanlei Zhou MU FedML 54 2 0 14 Jun 2024
FLea: Addressing Data Scarcity and Label Skew in Federated Learning via Privacy-preserving Feature Augmentation Tong Xia Abhirup Ghosh Xinchi Qiu Cecilia Mascolo 19 3 0 13 Jun 2024
Federated LoRA with Sparse Communication Kevin Kuo Arian Raje Kousik Rajesh Virginia Smith 38 7 0 07 Jun 2024
FedStaleWeight: Buffered Asynchronous Federated Learning with Fair Aggregation via Staleness Reweighting Jeffrey Ma Alan Tu Yiling Chen Vijay Janapa Reddi FedML 32 0 0 05 Jun 2024
Federated Unsupervised Domain Generalization using Global and Local Alignment of Gradients Farhad Pourpanah Mahdiyar Molahasani Milad Soltany Michael A. Greenspan Ali Etemad FedML OOD 81 2 0 25 May 2024
Harnessing Increased Client Participation with Cohort-Parallel Federated Learning Akash Dhasade Anne-Marie Kermarrec Tuan-Anh Nguyen Rafael Pires M. Vos FedML 33 0 0 24 May 2024
Worldwide Federated Training of Language Models Alexandru Iacob Lorenzo Sani Bill Marino Preslav Aleksandrov William F. Shen Nicholas D. Lane FedML 33 2 0 23 May 2024
The Limits and Potentials of Local SGD for Distributed Heterogeneous Learning with Intermittent Communication Kumar Kshitij Patel Margalit Glasgow Ali Zindari Lingxiao Wang Sebastian U. Stich Ziheng Cheng Nirmit Joshi Nathan Srebro 44 6 0 19 May 2024
The Future of Large Language Model Pre-training is Federated Lorenzo Sani Alexandru Iacob Zeyu Cao Bill Marino Yan Gao ... Wanru Zhao William F. Shen Preslav Aleksandrov Xinchi Qiu Nicholas D. Lane AI4CE 33 12 0 17 May 2024
Towards Fairness in Provably Communication-Efficient Federated Recommender Systems Kirandeep Kaur Sujit Gujar Shweta Jain FedML 44 0 0 03 May 2024
Not All Federated Learning Algorithms Are Created Equal: A Performance Evaluation Study Gustav A. Baumgart Jaemin Shin Ali Payani Myungjin Lee Ramana Rao Kompella FedML 26 6 0 26 Mar 2024
Improving Group Connectivity for Generalization of Federated Deep Learning Zexi Li Jie Lin Zhiqi Li Didi Zhu Chao Wu AI4CE FedML 38 0 0 29 Feb 2024
FLASH: Federated Learning Across Simultaneous Heterogeneities Xiangyu Chang Sk. Miraj Ahmed S. Krishnamurthy Başak Güler A. Swami Samet Oymak A. Roy-Chowdhury FedML 24 2 0 13 Feb 2024
FedImpro: Measuring and Improving Client Update in Federated Learning Zhenheng Tang Yonggang Zhang S. Shi Xinmei Tian Tongliang Liu Bo Han Xiaowen Chu FedML 13 13 0 10 Feb 2024
Version age-based client scheduling policy for federated learning Xinyi Hu Nikolaos Pappas Howard H. Yang 28 3 0 08 Feb 2024
Asynchronous Wireless Federated Learning with Probabilistic Client Selection Jiarong Yang Yuan Liu Fangjiong Chen Wen Chen Changle Li FedML 15 5 0 28 Nov 2023
Federated Learning with Manifold Regularization and Normalized Update Reaggregation Xuming An Li Shen Han Hu Yong Luo FedML 36 4 0 10 Nov 2023
DYNAMITE: Dynamic Interplay of Mini-Batch Size and Aggregation Frequency for Federated Learning with Static and Streaming Dataset Weijie Liu Xiaoxi Zhang Jingpu Duan Carlee Joe-Wong Zhi Zhou Xu Chen 13 7 0 20 Oct 2023
Federated Learning with Differential Privacy for End-to-End Speech Recognition Martin Pelikan Sheikh Shams Azam Vitaly Feldman Jan Honza Silovsky Kunal Talwar Tatiana Likhomanenko 30 8 0 29 Sep 2023
Stochastic Controlled Averaging for Federated Learning with Communication Compression Xinmeng Huang Ping Li Xiaoyun Li 32 195 0 16 Aug 2023
Towards Federated Foundation Models: Scalable Dataset Pipelines for Group-Structured Learning Zachary B. Charles Nicole Mitchell Krishna Pillutla Michael Reneer Zachary Garrett FedML AI4CE 28 28 0 18 Jul 2023
Pollen: High-throughput Federated Learning Simulation via Resource-Aware Client Placement Lorenzo Sani Pedro Gusmão Alexandru Iacob Wanru Zhao Xinchi Qiu Yan Gao Javier Fernandez-Marques Nicholas D. Lane 29 0 0 30 Jun 2023
Synthetic data shuffling accelerates the convergence of federated learning under data heterogeneity Bo-wen Li Yasin Esfandiari Mikkel N. Schmidt T. S. Alstrøm Sebastian U. Stich FedML 25 3 0 23 Jun 2023
Improving Accelerated Federated Learning with Compression and Importance Sampling Michal Grudzieñ Grigory Malinovsky Peter Richtárik FedML 27 8 0 05 Jun 2023
Perspectives on AI Architectures and Co-design for Earth System Predictability M. Mudunuru James A. Ang M. Halappanavar Simon D. Hammond Maya Gokhale ... Tushar Krishna S. Sreepathi Matthew R. Norman Ivy Bo Peng Philip W. Jones 15 0 0 07 Apr 2023
A Comparative Study of Federated Learning Models for COVID-19 Detection Erfan Darzidehkalani N. Sijtsema P. V. Ooijen FedML OOD 11 4 0 28 Mar 2023
Green Federated Learning Ashkan Yousefpour Sheng Guo Ashish Shenoy Sayan Ghosh Pierre Stock Kiwan Maeng Schalk-Willem Kruger Michael G. Rabbat Carole-Jean Wu Ilya Mironov FedML AI4CE 36 10 0 26 Mar 2023
An Empirical Evaluation of Federated Contextual Bandit Algorithms Alekh Agarwal H. B. McMahan Zheng Xu FedML 19 2 0 17 Mar 2023
Making Batch Normalization Great in Federated Deep Learning Jike Zhong Hong-You Chen Wei-Lun Chao FedML 21 9 0 12 Mar 2023
FLINT: A Platform for Federated Learning Integration Ewen N. Wang Ajaykumar Kannan Yuefeng Liang Boyi Chen Mosharaf Chowdhury 33 24 0 24 Feb 2023
Federated Gradient Matching Pursuit Halyun Jeong Deanna Needell Jing Qin FedML 35 1 0 20 Feb 2023
Revisiting Weighted Aggregation in Federated Learning with Neural Networks Zexi Li Tao R. Lin Xinyi Shang Chao-Xiang Wu FedML 35 59 0 14 Feb 2023
Topology-aware Federated Learning in Edge Computing: A Comprehensive Survey Jiajun Wu Steve Drew Fan Dong Zhuangdi Zhu Jiayu Zhou FedML 45 45 0 06 Feb 2023
Federated Automatic Differentiation Keith Rush Zachary B. Charles Zachary Garrett FedML 34 1 0 18 Jan 2023
On Noisy Evaluation in Federated Hyperparameter Tuning Kevin Kuo Pratiksha Thaker M. Khodak John Nguyen Daniel Jiang Ameet Talwalkar Virginia Smith FedML 35 8 0 17 Dec 2022
Analysis of Error Feedback in Federated Non-Convex Optimization with Biased Compression Xiaoyun Li Ping Li FedML 24 4 0 25 Nov 2022
FLamby: Datasets and Benchmarks for Cross-Silo Federated Learning in Realistic Healthcare Settings Jean Ogier du Terrail Samy Ayed Edwige Cyffers Felix Grimberg Chaoyang He ... Sai Praneeth Karimireddy Marco Lorenzi Giovanni Neglia Marc Tommasi M. Andreux FedML 36 142 0 10 Oct 2022
Taming Fat-Tailed ("Heavier-Tailed'' with Potentially Infinite Variance) Noise in Federated Learning Haibo Yang Pei-Yuan Qiu Jia Liu FedML 27 12 0 03 Oct 2022
Federated Select: A Primitive for Communication- and Memory-Efficient Federated Learning Zachary B. Charles Kallista A. Bonawitz Stanislav Chiknavaryan H. B. McMahan Blaise Agüera y Arcas FedML 15 13 0 19 Aug 2022
Reconciling Security and Communication Efficiency in Federated Learning Karthik Prasad Sayan Ghosh Graham Cormode Ilya Mironov Ashkan Yousefpour Pierre Stock FedML 22 8 0 26 Jul 2022
TCT: Convexifying Federated Learning using Bootstrapped Neural Tangent Kernels Yaodong Yu Alexander Wei Sai Praneeth Karimireddy Yi-An Ma Michael I. Jordan FedML 15 30 0 13 Jul 2022
Variance Reduced ProxSkip: Algorithm, Theory and Application to Federated Learning Grigory Malinovsky Kai Yi Peter Richtárik FedML 29 38 0 09 Jul 2022
Motley: Benchmarking Heterogeneity and Personalization in Federated Learning Shan-shan Wu Tian Li Zachary B. Charles Yu Xiao Ziyu Liu Zheng Xu Virginia Smith FedML 25 44 0 18 Jun 2022