SCA-LLM: Spectral-Attentive LLM-Based Wireless World Modeling for Agentic Communications

Future AI-native wireless networks are moving from reactive optimization to agentic decision-making that can sense, predict, and plan under fast-varying channels. This calls for wireless world models that can predict and roll out channel dynamics, for which multi-step channel state information (CSI) prediction offers a practical short-horizon look-ahead. Recent advances in foundation sequence models further motivate large language models (LLMs) as general-purpose dynamics learners when suitably adapted to non-text time-series signals. However, bridging CSI to LLMs is non-trivial because an effective adapter must expose informative spectral and temporal evolution patterns, while prior designs provide limited inductive bias to capture such channel structures. To this end, we propose SCA-LLM, a spectral-attentive LLM-based wireless world modeling framework that bridges CSI to LLMs via a spectral-channel attention (SCA) adapter. Specifically, the SCA adapter performs multi-spectral representation learning to extract informative channel features and align CSI with the LLM’s sequence modeling capability, enabling parameter-efficient adaptation while keeping the LLM backbone largely frozen. Extensive simulations show that SCA-LLM achieves state-of-the-art prediction performance and strong zero-shot generalization, yielding up to -2.4 dB normalized mean squared error (NMSE) advantage over the previous LLM based method. Our ablation studies further confirm the effectiveness of the proposed SCA adapter in mitigating domain mismatch.

💡 Research Summary

The paper addresses the emerging need for agentic decision‑making in AI‑native wireless networks, where fast‑varying channels demand short‑horizon, multi‑step channel state information (CSI) prediction. While deep learning models such as RNNs, LSTMs, and Transformers have been applied to CSI forecasting, they are typically scenario‑specific and lack robust generalization. Recent interest in large language models (LLMs) stems from their powerful sequence‑modeling capabilities, yet direct use of LLMs on raw CSI suffers from severe domain mismatch because LLMs are pre‑trained on textual data and have no built‑in understanding of the spectral‑temporal structure of wireless channels.

To bridge this gap, the authors propose SCA‑LLM, a framework that couples a pre‑trained LLM with a novel Spectral‑Channel Attention (SCA) adapter. The system model assumes a TDD MIMO‑OFDM link following the 3GPP TR 38.901 geometry‑based stochastic channel model. The prediction task is to forecast future uplink CSI (and thus downlink CSI via reciprocity) given a history of observed CSI matrices.

The SCA adapter first applies a 2‑D discrete cosine transform (DCT) to each CSI matrix, converting spatial‑frequency data into a set of spectral components. Unlike the earlier LLM4CP approach, which uses global average pooling and retains only the lowest‑frequency component, SCA retains multiple DCT bases, preserving low, mid, and high‑frequency information. These components pass through multi‑spectral channel‑attention layers that learn joint antenna‑time correlations, producing a rich feature tensor. The tensor is tokenized and fed into a frozen LLM (e.g., a GPT‑style model). The LLM leverages its long‑range dependency modeling to generate future CSI tokens, which are then decoded by a lightweight output head back into complex‑valued channel matrices. Only the adapter and output head are fine‑tuned, keeping the LLM parameters essentially unchanged, which yields parameter‑efficient adaptation and fast convergence.

Extensive simulations cover urban macro and micro scenarios, 4×4 MIMO, 64–128 subcarriers, user speeds from 3 km/h to 120 km/h, and SNRs from 0 dB to 30 dB. Results show that SCA‑LLM achieves up to –2.4 dB improvement in normalized mean‑squared error (NMSE) over the prior LLM‑based method, with particularly large gains in high‑mobility cases where high‑frequency spectral cues are critical. Zero‑shot tests on unseen deployment configurations demonstrate strong generalization, confirming that the spectral adapter effectively mitigates domain mismatch. Ablation studies reveal that (i) removing the DCT‑based spectral attention degrades NMSE by ~1.5 dB, (ii) using only GAP (low‑frequency) leads to severe performance loss at high speeds, and (iii) fine‑tuning the entire LLM inflates parameter count by an order of magnitude with marginal accuracy gains.

The authors argue that SCA‑LLM constitutes a reusable “wireless world model” capable of supporting downstream AI‑RAN tasks such as proactive beamforming, risk‑aware antenna selection, and resource scheduling. Future work is suggested on adapter lightweighting, multi‑agent collaborative planning, online continual learning, and hardware‑accelerated deployment. Overall, the paper demonstrates that a carefully designed spectral‑aware adapter can unlock the latent sequence‑modeling power of LLMs for high‑performance, generalizable wireless channel prediction.