A Multi-Modal Fusion Platform for Joint Environment Sensing and Channel Sounding in Highly Dynamic Scenarios

6G system is evolving toward full-spectrum coverage,ultra-wide bandwidth, and high mobility, resulting in increasingly complex propagation environments. The deep integration of communication and sensing is widely recognized as a core 6G vision, underscoring the importance of comprehensive environment awareness. Accurate channel modeling forms the foundation of 6G system design and optimization, and channel sounders provide the essential empirical basis. However, existing channel sounders, although supporting wide bandwidth and large antenna arrays in selected bands, generally lack cross-band capability, struggle in dynamic scenarios, and provide limited environmental awareness. The absence of detailed environmental information restricts the development of environment-aware channel models. To address this gap, we propose a multi-modal sensing and channel sounding fusion platform that enables temporally and spatially synchronized acquisition of images, point clouds, geolocation information, and multi-band multi-antenna channel data. The modular architecture facilitates rapid deployment in diverse dynamic environments. The platform supports Sub-6 GHz and mmWave bands with up to 1 GHz bandwidth and 1 ns delay resolution, enabling multi-antenna measurements with a channel switching rate of 8 ms. Moreover, it achieves centimeter-level and 360° environmental sensing accuracy and meter-level positioning accuracy. Key performance metrics of the platform, including dynamic range, phase stability, delay resolution, and multimodal data synchronization, are validated through vehicle-to-infrastructure measurement campaign. The established platform supports environment-channel joint modeling, enabling analysis and optimization of channel models in dynamic 6G scenarios.

💡 Research Summary

The paper addresses a critical gap in 6G research: the lack of measurement platforms that can simultaneously capture wide‑band, multi‑antenna channel data across multiple frequency bands and high‑resolution environmental information in highly dynamic scenarios. Existing channel sounders either operate in a single band, provide limited mobility support, or completely omit environmental sensing, which hampers the development of environment‑aware channel models essential for integrated sensing‑communication (ISA‑C) systems.

To fill this void, the authors design and implement a modular multi‑modal fusion platform that integrates two main subsystems: (1) a dual‑band channel sounding subsystem and (2) a visual‑LiDAR sensing subsystem. The channel sounding subsystem is built on National Instruments (NI) hardware. For Sub‑6 GHz operation, a vector signal generator/analyzer (VSG/VSA) provides up to 1 GHz instantaneous bandwidth and 1 ns delay resolution. For mmWave operation, an intermediate‑frequency direct‑sampling FlexRIO architecture is employed, preserving the same bandwidth and resolution while avoiding the need for separate RF front‑ends. Both subsystems support SIMO measurements with a channel‑switching time of 8 ms, enabling up to 50 snapshots per second. Antenna arrays are interchangeable, allowing future expansion to massive MIMO or beam‑forming configurations.

The visual‑LiDAR subsystem combines a 4K 360° panoramic camera and a high‑precision LiDAR (10 Hz scan rate). Both sensors are rigidly mounted on a custom circular magnetic frame that also holds the RF antennas, ensuring unobstructed LiDAR scanning and full‑sphere imaging. The camera communicates via Bluetooth to a smartphone that supplies time, velocity, and GNSS data, while the LiDAR connects through Ethernet for direct timestamp alignment. All devices are disciplined by a rubidium clock and GNSS antenna, providing a common time and position reference.

Four layers of synchronization are implemented: (i) hardware integration sharing power and clock, (ii) geospatial location matching via GNSS, (iii) millisecond‑level timestamp alignment to the rubidium reference, and (iv) multi‑frame image registration that fuses LiDAR point clouds with camera images into a unified 3‑D coordinate system. Because the subsystems sample at different rates (120 Hz GNSS, 100 Hz camera, 10 Hz LiDAR, 50 snapshots/s channel), interpolation/extrapolation is applied where necessary, but the overall temporal drift remains below 1 ms.

The platform’s performance is validated in a vehicle‑to‑infrastructure (V2I) measurement campaign on an urban road. The vehicle moves at approximately 30 km/h while simultaneously recording: (a) channel impulse responses in Sub‑6 GHz (3.5 GHz) and mmWave (28 GHz) bands, (b) 360° RGB images at 4K resolution, (c) LiDAR point clouds up to 200 m range, and (d) GNSS positions at 1 m accuracy. Key metrics achieved are: dynamic range ≈ 70 dB, phase stability < 0.5°, delay resolution = 1 ns, environmental sensing accuracy ≈ 5 cm, and positioning error ≤ 1 m. Compared with prior single‑band or static sounders, the proposed system improves spatial resolution by a factor of 2–3 and dynamic measurement capability by roughly fivefold.

The authors demonstrate how the synchronized multi‑modal dataset can be processed to extract multipath parameters, reconstruct a centimeter‑accurate 3‑D environment, and establish explicit mappings between physical objects (buildings, vehicles, foliage) and channel components (delay, angle, gain). This enables the creation of environment‑aware stochastic or deterministic channel models that adapt to real‑time scene changes, a prerequisite for advanced 6G features such as predictive beamforming, joint communication‑radar, and AI‑driven network optimization.

Contributions are summarized as: (1) a dual‑band, multi‑antenna channel sounding system with 1 ns delay resolution and 8 ms switching, (2) a 360°, centimeter‑level visual‑LiDAR sensing suite, (3) a comprehensive synchronization framework that aligns heterogeneous data streams in space and time, and (4) a modular architecture that permits easy addition of new sensors (e.g., radar, THz) or expansion to massive MIMO.

Limitations include the current focus on SIMO measurements; extending to full massive MIMO or real‑time beam‑forming would require additional RF switching hardware and more sophisticated calibration. The platform also demands high‑performance computing for real‑time processing and large storage capacity for the multi‑modal data, which may be challenging for field deployments with strict power or weight constraints.

Future work suggested by the authors involves (i) scaling the RF front‑end to massive MIMO, (ii) integrating AI algorithms for on‑the‑fly channel‑environment inference, and (iii) developing low‑power, embedded versions of the platform for long‑duration field trials.

In conclusion, the presented multi‑modal fusion platform provides a practical, high‑fidelity testbed that bridges the gap between channel sounding and environmental perception, thereby laying a solid experimental foundation for the next generation of 6G communication systems that are truly aware of, and responsive to, their surrounding physical world.