Federated Learning with Server Learning: Enhancing Performance for Non-IID Data

Federated Learning (FL) has emerged as a means of performing distributed learning using local data stored at clients and a coordinating server. But, recent studies showed that FL can suffer from poor performance when client training data are not independently and identically distributed (non-IID). This paper proposes a new complementary approach to mitigating this performance degradation when the server has access to a small dataset, on which it can perform auxiliary learning. Our analysis and experiments show that incorporating server learning with FL in an incremental fashion can provide significant benefits when the distribution of server data is similar to that of the aggregate samples of all clients, even when the server dataset is small, and improve the convergence rate considerably at the beginning of the learning process.