A Multi-Modal Foundational Model for Wireless Communication and Sensing

Artificial intelligence is a key enabler for next-generation wireless communication and sensing. Yet, today’s learning-based wireless techniques do not generalize well: most models are task-specific, environment-dependent, and limited to narrow sensing modalities, requiring costly retraining when deployed in new scenarios. This work introduces a task-agnostic, multi-modal foundational model for physical-layer wireless systems that learns transferable, physics-aware representations across heterogeneous modalities, enabling robust generalization across tasks and environments. Our framework employs a physics-guided self-supervised pretraining strategy incorporating a dedicated physical token to capture cross-modal physical correspondences governed by electromagnetic propagation. The learned representations enable efficient adaptation to diverse downstream tasks, including massive multi-antenna optimization, wireless channel estimation, and device localization, using limited labeled data. Our extensive evaluations demonstrate superior generalization, robustness to deployment shifts, and reduced data requirements compared to task-specific baselines.

💡 Research Summary

The paper addresses three fundamental challenges that have limited the deployment of AI in next‑generation wireless communication and sensing: (1) task‑specific models that cannot be reused across different functions, (2) environment‑dependent performance that degrades when the system is moved to a new site, and (3) reliance on a single sensing modality, which restricts robustness. To overcome these issues, the authors propose a multi‑modal foundational model that learns physics‑aware representations spanning heterogeneous data sources—channel state information (CSI), a 3‑D static scene map, and user location.
Key innovations include:

1. **Physical Token (