73
0

Trading-off Accuracy and Communication Cost in Federated Learning

Abstract

Leveraging the training-by-pruning paradigm introduced by Zhou et al. and Isik et al. introduced a federated learning protocol that achieves a 34-fold reduction in communication cost. We achieve a compression improvements of orders of orders of magnitude over the state-of-the-art. The central idea of our framework is to encode the network weights w\vec w by a the vector of trainable parameters p\vec p, such that w=Qp\vec w = Q\cdot \vec p where QQ is a carefully-generate sparse random matrix (that remains fixed throughout training). In such framework, the previous work of Zhou et al. [NeurIPS'19] is retrieved when QQ is diagonal and p\vec p has the same dimension of w\vec w. We instead show that p\vec p can effectively be chosen much smaller than w\vec w, while retaining the same accuracy at the price of a decrease of the sparsity of QQ. Since server and clients only need to share p\vec p, such a trade-off leads to a substantial improvement in communication cost. Moreover, we provide theoretical insight into our framework and establish a novel link between training-by-sampling and random convex geometry.

View on arXiv
@article{villani2025_2503.14246,
  title={ Trading-off Accuracy and Communication Cost in Federated Learning },
  author={ Mattia Jacopo Villani and Emanuele Natale and Frederik Mallmann-Trenn },
  journal={arXiv preprint arXiv:2503.14246},
  year={ 2025 }
}
Comments on this paper