Towards Federated Foundation Models: Scalable Dataset Pipelines for Group-Structured Learning

18 July 2023

Papers citing "Towards Federated Foundation Models: Scalable Dataset Pipelines for Group-Structured Learning"

20 / 20 papers shown

Title
Federated Adapter on Foundation Models: An Out-Of-Distribution Approach Yiyuan Yang Guodong Long Tianyi Zhou Qinghua Lu Shanshan Ye Jing Jiang OODD 78 1 0 02 May 2025
Ten Challenging Problems in Federated Foundation Models Tao Fan Hanlin Gu Xuemei Cao Chee Seng Chan Qian Chen ... Y. Zhang Xiaojin Zhang Zhenzhe Zheng Lixin Fan Qiang Yang FedML 73 3 0 14 Feb 2025
DEPT: Decoupled Embeddings for Pre-training Language Models Alex Iacob Lorenzo Sani Meghdad Kurmanji William F. Shen Xinchi Qiu Dongqi Cai Yan Gao Nicholas D. Lane VLM 47 0 0 07 Oct 2024
Harnessing Increased Client Participation with Cohort-Parallel Federated Learning Akash Dhasade Anne-Marie Kermarrec Tuan-Anh Nguyen Rafael Pires M. Vos FedML 26 0 0 24 May 2024
Federated Prompting and Chain-of-Thought Reasoning for Improving LLMs Answering Xiangyang Liu Tianqi Pang Chenyou Fan FedML LRM 40 23 0 27 Apr 2023
How to DP-fy ML: A Practical Guide to Machine Learning with Differential Privacy Natalia Ponomareva Hussein Hazimeh Alexey Kurakin Zheng Xu Carson E. Denison H. B. McMahan Sergei Vassilvitskii Steve Chien Abhradeep Thakurta 94 165 0 01 Mar 2023
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Jason W. Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Brian Ichter F. Xia Ed H. Chi Quoc Le Denny Zhou LM&Ro LRM AI4CE ReLM 315 8,261 0 28 Jan 2022
P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Tasks Xiao Liu Kaixuan Ji Yicheng Fu Weng Lam Tam Zhengxiao Du Zhilin Yang Jie Tang VLM 236 780 0 14 Oct 2021
Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation Ofir Press Noah A. Smith M. Lewis 237 690 0 27 Aug 2021
The Power of Scale for Parameter-Efficient Prompt Tuning Brian Lester Rami Al-Rfou Noah Constant VPVLM 278 3,784 0 18 Apr 2021
Federated Evaluation and Tuning for On-Device Personalization: System Design & Applications Matthias Paulik M. Seigel Henry Mason Dominic Telaar Joris Kluivers ... Dominic Hughes O. Javidbakht Fei Dong Rehan Rishi Stanley Hung FedML 167 124 0 16 Feb 2021
tf.data: A Machine Learning Data Processing Framework D. Murray Jiří Šimša Ana Klimovic Ihor Indyk PINN AI4CE LMTD 39 86 0 28 Jan 2021
The Pile: An 800GB Dataset of Diverse Text for Language Modeling Leo Gao Stella Biderman Sid Black Laurence Golding Travis Hoppe ... Horace He Anish Thite Noa Nabeshima Shawn Presser Connor Leahy AIMat 245 1,977 0 31 Dec 2020
Extracting Training Data from Large Language Models Nicholas Carlini Florian Tramèr Eric Wallace Matthew Jagielski Ariel Herbert-Voss ... Tom B. Brown D. Song Ulfar Erlingsson Alina Oprea Colin Raffel MLAU SILM 267 1,798 0 14 Dec 2020
Big Bird: Transformers for Longer Sequences Manzil Zaheer Guru Guruganesh Kumar Avinava Dubey Joshua Ainslie Chris Alberti ... Philip Pham Anirudh Ravula Qifan Wang Li Yang Amr Ahmed VLM 249 1,982 0 28 Jul 2020
FedML: A Research Library and Benchmark for Federated Machine Learning Chaoyang He Songze Li Jinhyun So Xiao Zeng Mi Zhang ... Yang Liu Ramesh Raskar Qiang Yang M. Annavaram Salman Avestimehr FedML 156 553 0 27 Jul 2020
Device Heterogeneity in Federated Learning: A Superquantile Approach Yassine Laguel Krishna Pillutla J. Malick Zaïd Harchaoui FedML 19 22 0 25 Feb 2020
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 223 4,424 0 23 Jan 2020
Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks Chelsea Finn Pieter Abbeel Sergey Levine OOD 243 11,568 0 09 Mar 2017
Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation Yonghui Wu M. Schuster Z. Chen Quoc V. Le Mohammad Norouzi ... Alex Rudnick Oriol Vinyals G. Corrado Macduff Hughes J. Dean AIMat 716 6,724 0 26 Sep 2016