In federated AI, participant selection matters under a strict training limit

Federated learning lets separate devices or organizations train a shared AI model without pooling their raw data. Participants train locally, then send model updates for aggregation. But those repeated exchanges can be expensive, especially where network capacity is limited.

One response is to involve fewer participants in each training round. That creates another problem: whose information gets left out?

A preprint by Max Halleryd and colleagues investigates this question using image-classification experiments. Its central finding is that, under a strict participation limit, selection methods associated with more balanced contributions achieved better test accuracy than a less balanced alternative. However, the bandwidth savings still came with an accuracy penalty compared with involving everyone. The findings are provisional, not established evidence of performance in deployed systems.

Three ways to choose participants

The researchers simulated 20 clients, each holding a separate collection of training images. They compared a baseline involving all 20 with strategies selecting just three clients per round, after two initial rounds involving everyone. Model updates were combined using federated averaging.

The experiments used MNIST handwritten digits and ImageNet50, a collection of 50 classes drawn from ImageNet. Images were distributed unevenly across clients, producing differences in both dataset size and class representation. This modeled a central difficulty of federated learning: participants do not necessarily hold interchangeable samples of the same population.

The team tested three selection strategies:

• Power-of-Choice first sampled 10 candidates, giving clients with larger datasets a greater chance of entering that pool. It then chose the three with the highest local loss—a measure of how poorly the model handled their data.
• Fisher information ranked clients using a gradient-based measure of how sensitive model parameters were to their local examples.
• Centered Kernel Alignment, or CKA, compared representations produced by local and global models, favoring clients with lower similarity.

To assess participation, the researchers used the Gini coefficient, which measures inequality in how often clients are selected. They also measured how far the selected clients’ class distribution departed from a uniform distribution. Lower values on that second measure indicated more balanced representation across image classes.

Large upload savings, lower accuracy

Selecting three clients instead of 20 reduced reported upstream communication—the data clients uploaded—by a factor of 6.46, including the initial full-participation rounds.

For MNIST, reported uploads fell from 288.79 megabytes to 44.72 megabytes. Test accuracy, averaged over the final 15 rounds, was 94.95% with full participation. Power-of-Choice reached 91.76%, Fisher information 91.04%, and CKA 76.41%.

The harder ImageNet50 benchmark showed a similar ranking, with smaller differences among the restricted-participation strategies. Full participation achieved 59.77% test accuracy, compared with 53.22% for Power-of-Choice, 52.92% for Fisher information and 51.84% for CKA. Reported uploads fell from 780.05 gigabytes to 120.79 gigabytes.

Across the experiments, more balanced participation and class coverage accompanied stronger results. Power-of-Choice delivered the highest test accuracy among the restricted-participation methods; CKA showed persistently uneven selection and the lowest accuracy.

Yet Power-of-Choice did not include everyone consistently. In MNIST, some clients were almost entirely excluded after the initial rounds. Their limited class variety was associated with lower local loss, making them less likely to be chosen. Better overall balance therefore did not mean every participant remained represented.

What the study does not settle

These experiments support a relationship between participation diversity and accuracy, but do not isolate diversity as the sole cause. The selection methods also differ in the information they use to rank clients.

An important missing comparison is random selection. The authors excluded it on reproducibility grounds, leaving unanswered whether these strategies outperform a simple random-participation baseline. The paper also does not report uncertainty across repeated runs, limiting interpretation of small accuracy differences.

The communication figures describe uploads, rather than a demonstrated end-to-end accounting of downloads, selection overhead or elapsed training time. And two image benchmarks with simulated data partitions cannot establish how the methods would perform with real-world availability or connectivity constraints.

The practical lesson is consequently narrower than eliminating the trade-off between efficiency and quality: choosing participants carefully can improve accuracy within a restricted communication budget. In these tests, however, involving everyone still produced the strongest model.