Federated Latent Space Alignment for Multi-user Semantic Communications

Semantic communication aims to convey meaning for effective task execution, but differing latent representations in AI-native devices can cause semantic mismatches that hinder mutual understanding. This paper introduces a novel approach to mitigating latent space misalignment in multi-agent AI- native semantic communications. In a downlink scenario, we consider an access point (AP) communicating with multiple users to accomplish a specific AI-driven task. Our method implements a protocol that shares a semantic pre-equalizer at the AP and local semantic equalizers at user devices, fostering mutual understanding and task-oriented communication while considering power and complexity constraints. To achieve this, we employ a federated optimization for the decentralized training of the semantic equalizers at the AP and user sides. Numerical results validate the proposed approach in goal-oriented semantic communication, revealing key trade-offs among accuracy, com- munication overhead, complexity, and the semantic proximity of AI-native communication devices.

💡 Research Summary

The paper addresses a fundamental challenge in emerging semantic communication systems: the misalignment of latent feature spaces between a transmitter (access point, AP) and multiple heterogeneous AI‑native receivers. In conventional wireless networks, the goal is to reliably transmit bits; in semantic communication, the goal is to convey meaning efficiently for a downstream task such as image classification. When the AP and each user employ different pre‑trained deep neural networks (DNNs), the same raw data are mapped to distinct latent representations, creating “semantic noise” that degrades task performance. Direct joint training of encoder and decoder across devices is often infeasible due to privacy, intellectual‑property, and hardware constraints, especially in multi‑vendor environments. Therefore, a mechanism that aligns the latent spaces without sharing full models or private data is required.

System Model
The authors consider a downlink scenario with one AP equipped with (N_T) antennas and (L) users each having (N_R) antennas. A raw data point (z\in\mathbb{R}^q) is processed by the AP’s pre‑trained encoder into a real vector (s_{\text{AP}}\in\mathbb{R}^d). The first half of the entries are paired with the second half to form a complex vector (x\in\mathbb{C}^{d/2}). A linear “semantic pre‑equalizer” (f(\cdot)) represented by matrix (\mathbf{F}\in\mathbb{C}^{K N_T\times d/2}) compresses (x) into (\mathbf{x}’=\mathbf{F}x) that will be transmitted over (K) channel uses. The compression factor is (\zeta = K d/2). Each user (l) experiences a MIMO Rayleigh fading matrix (\mathbf{H}_l\in\mathbb{C}^{N_T\times N_R}) (assumed constant over the (K) uses) and adds complex Gaussian noise (\mathbf{n}_l). At the receiver side, a linear “semantic equalizer” (g_l(\cdot)) with matrix (\mathbf{G}_l\in\mathbb{C}^{m_l\times K N_R}) maps the received signal to a complex vector (\hat{\mathbf{y}}_l), which is then demapped to a real latent vector (\hat{s}_l). The overall relation is

\