Quality Inference in Federated Learning with Secure Aggregation

Collaborative machine learning algorithms are developed both for efficiency reasons, and to ensure the privacy protection of sensitive data used for processing. Federated learning is the most popular of these methods, where 1) learning is done locally, and 2) only a subset of the participants contribute in each training round. Despite individual data is not shared explicitly, recent studies showed that federated learning models could still leak information. In this paper we focus on the quality of individual training datasets, and show that such information could be inferred and connected to specific participants even when secure aggregation is applied. Specifically, we use three simple scoring rules for evaluating per round aggregated updates in the federated learning process, and mount a novel differential quality inference attack (i.e., relative quality ordering reconstruction). Through a series of image recognition experiments we show that the attack is able to infer the relative quality ordering of participants. Whilst an attack in the traditional sense, quality inference could also improve the federated learning process: we demonstrate how it can be used to (i) boost training efficiency and (ii) detect misbehavior. Finally, as a system designer might want to alleviate quality inference in certain use-cases, we discuss mitigation approaches.