FedLECC: Cluster- and Loss-Guided Client Selection for Federated Learning under Non-IID Data

Federated Learning (FL) enables distributed Artificial Intelligence (AI) across cloud-edge environments by allowing collaborative model training without centralizing data. In cross-device deployments, FL systems face strict communication and participation constraints, as well as strong non-independent and identically distributed (non-IID) data that degrades convergence and model quality. Since only a subset of devices (a.k.a clients) can participate per training round, intelligent client selection becomes a key systems challenge. This paper proposes FedLECC (Federated Learning with Enhanced Cluster Choice), a lightweight, cluster-aware, and loss-guided client selection strategy for cross-device FL. FedLECC groups clients by label-distribution similarity and prioritizes clusters and clients with higher local loss, enabling the selection of a small yet informative and diverse set of clients. Experimental results under severe label skew show that FedLECC improves test accuracy by up to 12%, while reducing communication rounds by approximately 22% and overall communication overhead by up to 50% compared to strong baselines. These results demonstrate that informed client selection improves the efficiency and scalability of FL workloads in cloud-edge systems.

💡 Research Summary

Federated Learning (FL) enables collaborative model training across cloud‑edge infrastructures without moving raw data, but cross‑device deployments face two intertwined challenges: (1) strict communication and participation budgets that limit the number of clients that can be active in each round, and (2) severe statistical heterogeneity, especially label‑skew, which causes client updates to diverge and slows convergence. Existing works either treat client selection as uniform random sampling—known to be suboptimal under non‑IID data—or rely on a single heuristic such as loss‑based priority or clustering alone. This paper introduces FedLECC (Federated Learning with Enhanced Cluster Choice), a lightweight, two‑stage client selection mechanism that simultaneously enforces diversity and informativeness.

The first stage quantifies each client’s label distribution by having the client send a normalized histogram of its local labels. This information is low‑dimensional, privacy‑preserving, and can be exchanged once or when the distribution changes significantly. The server computes pairwise Hellinger distances between histograms and applies the OPTICS clustering algorithm, which automatically discovers clusters of clients with similar label distributions without requiring a preset number of clusters. Clustering prevents repeatedly selecting clients that share almost identical data, thereby preserving diversity in the global update.

In the second stage, during every communication round, each participating client reports its current local empirical loss after a local training epoch. The server aggregates these losses at the cluster level, ranks clusters by their mean loss, and selects the top‑J clusters (J ≤ Jmax). Within each selected cluster, the server picks the z = ⌈m/J⌉ clients with the highest individual loss, where m is the total client budget for the round. If a cluster does not contain enough high‑loss clients, remaining slots are filled from the next highest‑loss clusters. This loss‑guided prioritization ensures that the selected set contains the most “informative” updates—i.e., from clients where the global model currently underperforms—while the clustering step guarantees that these informative updates are spread across diverse data modes.

FedLECC does not modify the local training algorithm or the server‑side aggregation rule (e.g., FedAvg’s weighted averaging), so it inherits the convergence guarantees of biased client selection frameworks such as Power‑of‑Choice. Empirically, the authors evaluate FedLECC on MNIST and Fashion‑MNIST with artificially induced label skew ranging from mild (α = 0.5) to extreme (α = 0.1). They compare against uniform random sampling, Power‑of‑Choice (loss‑only), HACCS (cluster‑only with latency awareness), and FedCLS (label‑based clustering with Hamming distance). Across all skew levels, FedLECC achieves up to a 12 % absolute improvement in test accuracy, reduces the number of communication rounds needed to reach a target accuracy by roughly 22 %, and cuts total transmitted bytes by up to 50 % relative to the strongest baselines. Convergence curves show faster early‑stage progress and lower variance in later stages, indicating more stable training dynamics.

The paper also discusses practical considerations: the label‑histogram exchange incurs negligible overhead (proportional to the number of classes), clustering is performed once or upon significant distribution change, and the selection algorithm runs in O(K log K) time per round, making it suitable for large‑scale cross‑device FL. Limitations and future directions include extending the similarity metric beyond label histograms (e.g., feature‑conditioned distances), incorporating dynamic reclustering as data evolves, and integrating differential privacy or secure multi‑party computation to further protect the histogram information.

In summary, FedLECC offers a pragmatic, system‑aware solution that leverages both cluster‑level diversity and loss‑driven informativeness to address the twin challenges of communication constraints and severe non‑IID data in federated learning. Its compatibility with existing FL algorithms and substantial empirical gains make it a compelling addition to the toolbox of researchers and practitioners deploying FL at scale.