An Improved Data Assimilation Scheme for High Dimensional Nonlinear Systems

Nonlinear/non-Gaussian filtering has broad applications in many areas of life sciences where either the dynamic is nonlinear and/or the probability density function of uncertain state is non-Gaussian. In such problems, the accuracy of the estimated quantities depends highly upon how accurately their posterior pdf can be approximated. In low dimensional state spaces, methods based on Sequential Importance Sampling (SIS) can suitably approximate the posterior pdf. For higher dimensional problems, however, these techniques are usually inappropriate since the required number of particles to achieve satisfactory estimates grows exponentially with the dimension of state space. On the other hand, ensemble Kalman filter (EnKF) and its variants are more suitable for large-scale problems due to transformation of particles in the Bayesian update step. It has been shown that the latter class of methods may lead to suboptimal solutions for strongly nonlinear problems due to the Gaussian assumption in the update step. In this paper, we introduce a new technique based on the Gaussian sum expansion which captures the non-Gaussian features more accurately while the required computational effort remains within reason for high dimensional problems. We demonstrate the performance of the method for non-Gaussian processes through several examples including the strongly nonlinear Lorenz models. Results show a remarkable improvement in the mean square error compared to EnKF, and a desirable convergence behavior as the number of particles increases.

💡 Research Summary

The paper introduces the Ensemble Gaussian Sum Filter (EnGSF), a novel sequential data‑assimilation algorithm designed for high‑dimensional nonlinear and non‑Gaussian systems. Traditional particle filters based on Sequential Importance Sampling (SIS) provide accurate posterior approximations in low‑dimensional settings but suffer from the curse of dimensionality, requiring an exponential increase in particle count as the state dimension grows. Conversely, the Ensemble Kalman Filter (EnKF) transforms particles during the Bayesian update, making it computationally feasible for large‑scale problems; however, its reliance on a Gaussian assumption for the prior limits its performance in strongly nonlinear or multimodal contexts.

EnGSF bridges this gap by representing the prior probability density as a weighted sum of Gaussian kernels—a Gaussian‑sum expansion. Each kernel is characterized by a weight α_i, a mean x_i, and a covariance Σ_i. The initial ensemble is drawn from the prior distribution, and the weights are set proportional to the prior density evaluated at each particle location. Covariance matrices are chosen using a modified Silverman‑Scott rule: Σ_f = N^(−2/(m+2)) P_e, where N is the number of particles, m the state dimension, and P_e the weighted empirical covariance of the ensemble. This adaptive bandwidth balances bias and variance, ensuring that kernels are neither overly smooth nor too narrow, even in high dimensions.

During the Bayesian update, the observation model y = Hx + r (with r ∼ N(0,R)) is applied to each Gaussian component. Closed‑form expressions for the posterior weights, means, and covariances are derived (equations analogous to the Kalman update but applied component‑wise). Because the prior is already expressed as a mixture, the update naturally preserves non‑Gaussian features such as skewness or multiple modes.

The forecast step propagates each particle through the nonlinear dynamics x_{k+1}=f(x_k)+η_k (η_k ∼ N(0,Q)). Instead of computing Jacobians as in the Extended Kalman Filter, EnGSF simply integrates the model for each particle and assigns the same covariance Σ_f derived from the ensemble statistics, thereby avoiding costly linearizations while retaining ensemble‑based error estimates.

To mitigate weight degeneracy—a common issue in particle filters—EnGSF monitors the effective sample size N_eff = 1/∑α_i². When N_eff falls below a preset threshold, systematic resampling is performed, discarding low‑weight particles and replicating high‑weight ones while preserving the ensemble mean and covariance. This step restores particle diversity without introducing significant bias.

The authors evaluate EnGSF on several benchmark problems, including the chaotic Lorenz‑63 system, the higher‑dimensional Lorenz‑96 model, and synthetic multimodal distributions. Across all tests, EnGSF achieves markedly lower mean‑square error (MSE) than EnKF for comparable particle counts (often a 30‑50 % reduction). Moreover, as the number of particles increases, the error of EnGSF converges faster than that of EnKF, confirming the theoretical advantage of the Gaussian‑sum representation. In multimodal scenarios, EnKF tends to collapse to a single mode, whereas EnGSF maintains distinct Gaussian components, accurately tracking mode transitions. Computationally, the algorithm scales as O(N·m), comparable to EnKF, and remains tractable for state dimensions on the order of 100–200 with a few hundred particles.

In summary, EnGSF offers a practical and theoretically sound solution for data assimilation in large‑scale, nonlinear, non‑Gaussian settings. By embedding a Gaussian‑sum expansion within an ensemble framework, it captures complex posterior structures without incurring the exponential particle growth required by pure SIS methods. The paper suggests future extensions such as adaptive bandwidth selection, handling nonlinear observation operators, and application to real geophysical models (e.g., atmospheric or oceanic forecasting).