Nonparametric Variational Bayesian Learning for Channel Estimation with OTFS Modulation

Orthogonal time frequency space (OTFS) modulation has demonstrated significant advantages in high-mobility scenarios in future 6G networks. However, existing channel estimation methods often overlook the structured sparsity and clustering characteristics inherent in realistic clustered delay line (CDL) channels, leading to degraded performance in practical systems. To address this issue, we propose a novel nonparametric Bayesian learning (NPBL) framework for OTFS channel estimation. Specifically, a stick-breaking process is introduced to automatically infer the number of multipath components and assign each path to its corresponding cluster. The channel coefficients within each cluster are modeled by a Gaussian mixture distribution to capture complex fading statistics. Furthermore, an effective pruning criterion is designed to eliminate spurious multipath components, thereby enhancing estimation accuracy and reducing computational complexity. Simulation results demonstrate that the proposed method achieves superior performance in terms of normalized mean squared error compared to existing methods.

💡 Research Summary

This paper addresses the challenging problem of channel estimation for Orthogonal Time Frequency Space (OTFS) modulation, which is poised to play a pivotal role in high‑mobility scenarios of future 6G networks. While many existing OTFS channel estimators rely on simplified sparse models or assume integer‑grid delays and Dopplers, they fail to capture the structured sparsity and clustering inherent in realistic propagation environments, especially those described by the Clustered Delay Line (CDL) model. To bridge this gap, the authors propose a Non‑Parametric Bayesian Learning (NPBL) framework that jointly models the number of multipath components, their assignment to clusters, and the statistical distribution of channel coefficients within each cluster.

The core of the methodology is a truncated stick‑breaking process, which serves as a finite‑dimensional approximation of a Dirichlet‑process mixture. This construction allows the algorithm to infer both the number of clusters and the number of paths per cluster directly from the received pilot observations, without any a‑priori specification. Each cluster is modeled by a Gaussian mixture: the complex channel gains of paths belonging to the same cluster share a common precision (inverse variance) parameter, while the delay and Doppler offsets are drawn from Gaussian distributions centered at cluster‑specific means. Hyper‑priors for these precisions are chosen as Gamma distributions, enabling conjugate updates within a variational inference scheme.

Because OTFS operates in the delay‑Doppler (DD) domain, true path delays and Dopplers are generally off‑grid relative to the discretized DD lattice. The authors mitigate this mismatch by applying a first‑order linear (Taylor) approximation to the sensing matrix Φ around the current estimates ( (\hat{k}\nu, \hat{l}\tau) ). The resulting corrected matrix (\bar{\Phi}) incorporates gradient terms with respect to delay and Doppler, and is iteratively refined as the variational parameters converge, thereby reducing off‑grid bias.

Variational Bayesian inference is employed to approximate the posterior distribution of all latent variables (\Omega = {h, k_\nu, l_\tau, \alpha_h, \alpha_k, \alpha_l, \alpha_w, \mu_h, \mu_k, \mu_l, \lambda_1, \lambda_2, r, C}). Assuming a mean‑field factorization, the evidence lower bound (ELBO) is maximized with respect to each factor. Closed‑form update equations are derived for the stick‑breaking Beta parameters, the cluster assignment probabilities (r_{i,t}), the channel coefficient mean vector (\mu_h) and covariance (\Sigma_h), as well as for the precision parameters of the gains, delays, and Dopplers. The assignment probabilities (r_{i,t}) are particularly rich, involving digamma functions of the updated Gamma shape parameters and expectations of squared magnitudes of the channel coefficients, thus integrating information from all model layers.

A novel pruning criterion is introduced to curb computational load and avoid over‑fitting to spurious paths. The expected precision (\gamma_{h,i} = \sum_t r_{i,t} \langle \alpha_{h,t} \rangle) serves as a measure of the relevance of each path; paths whose (\gamma_{h,i}) falls below a predefined threshold are deemed virtual and removed from the model. This sparsity‑promoting step dramatically reduces the number of active components during later iterations, leading to lower runtime and memory usage while preserving estimation accuracy.

Simulation results are presented for several 3GPP CDL configurations (e.g., CDL‑A, CDL‑C) across a wide SNR range (0–30 dB). The proposed NPBL estimator consistently outperforms state‑of‑the‑art compressed‑sensing based methods such as Two‑Choice Hard Thresholding Pursuit (TCHTP) and Turbo‑IFSLA‑VBI. In low‑SNR regimes, the normalized mean‑squared error (NMSE) improvement reaches 3–5 dB relative to the best competing technique. Moreover, the average number of variational iterations required for convergence is reduced by roughly 30 %, and the pruning mechanism eliminates up to 20 % of the computational burden compared with an unpruned baseline.

In summary, the paper makes three principal contributions: (1) a non‑parametric stick‑breaking prior that automatically discovers the latent cluster structure of OTFS channels; (2) a Gaussian‑mixture representation of intra‑cluster fading that captures non‑Gaussian statistics; and (3) an effective variational‑posterior‑based pruning rule that balances accuracy and complexity. The work demonstrates that integrating sophisticated Bayesian non‑parametrics with practical off‑grid compensation can substantially enhance OTFS channel estimation, paving the way for robust high‑mobility communications in future wireless standards. Future directions suggested include extension to massive MIMO OTFS, real‑time tracking of time‑varying clusters, and hardware‑level validation.