Nonlinear Bayesian Filtering with Natural Gradient Gaussian Approximation

Practical Bayes filters often assume the state distribution of each time step to be Gaussian for computational tractability, resulting in the so-called Gaussian filters. When facing nonlinear systems, Gaussian filters such as extended Kalman filter (EKF) or unscented Kalman filter (UKF) typically rely on certain linearization techniques, which can introduce large estimation errors. To address this issue, this paper reconstructs the prediction and update steps of Gaussian filtering as solutions to two distinct optimization problems, whose optimal conditions are found to have analytical forms from Stein’s lemma. It is observed that the stationary point for the prediction step requires calculating the first two moments of the prior distribution, which is equivalent to that step in existing moment-matching filters. In the update step, instead of linearizing the model to approximate the stationary points, we propose an iterative approach to directly minimize the update step’s objective to avoid linearization errors. For the purpose of performing the steepest descent on the Gaussian manifold, we derive its natural gradient that leverages Fisher information matrix to adjust the gradient direction, accounting for the curvature of the parameter space. Combining this update step with moment matching in the prediction step, we introduce a new iterative filter for nonlinear systems called \textit{N}atural Gr\textit{a}dient Gaussia\textit{n} Appr\textit{o}ximation filter, or NANO filter for short. We prove that NANO filter locally converges to the optimal Gaussian approximation at each time step. Furthermore, the estimation error is proven exponentially bounded for nearly linear measurement equation and low noise levels through constructing a supermartingale-like property across consecutive time steps.

💡 Research Summary

This paper revisits the two fundamental steps of Bayesian filtering—prediction and update—from an optimization perspective and proposes a novel nonlinear Gaussian filter called the Natural Gradient Gaussian Approximation (NANO) filter. The authors first formulate the prediction step as a variational maximization problem and, using Stein’s lemma, show that its stationary point can be obtained analytically by computing the first two moments of the prior distribution. This result coincides with the moment‑matching approach employed by existing filters such as the Unscented Kalman Filter (UKF) or Gauss‑Hermite KF, confirming that no further approximation is needed in the prediction phase.

In contrast, the update step is expressed as a minimization of a functional consisting of the expected negative log‑likelihood plus a Kullback‑Leibler (KL) divergence term. The optimality conditions lead to two coupled equations that, for general nonlinear measurement functions, have no closed‑form solution. Traditional Gaussian filters (EKF, IEKF, UKF, PLF) resolve this by linearizing the measurement model, which inevitably introduces bias.

To avoid linearization, the authors introduce a natural gradient descent on the manifold of Gaussian distributions. By employing the Fisher information matrix as a metric, the gradient direction is pre‑conditioned to follow the steepest descent respecting the curvature of the parameter space. The resulting iterative scheme updates the mean and covariance until convergence, effectively solving the original optimization problem for the update step.

The NANO algorithm therefore combines exact moment‑matching in the prediction phase with a natural‑gradient‑based iterative refinement in the update phase. The authors prove that a single iteration of NANO reduces to the classic Kalman filter in the linear‑Gaussian case, providing a fresh interpretation of the Kalman gain. They further establish local convergence of the iterative update to the optimal Gaussian approximation, with an error that is second‑order in the Taylor expansion of the measurement function. For nearly linear measurement models and low process/measurement noise, they construct a super‑martingale‑like property across time steps, yielding an exponential bound on the estimation error.

Beyond the basic filter, the paper extends the framework to Gibbs posteriors, allowing the use of robust loss functions (e.g., Huber, Tukey) to mitigate outliers. Three robust NANO variants are presented, each incorporating a different robust loss. Theoretical analysis shows that these variants retain the convergence guarantees while improving robustness.

Extensive simulations and real‑world experiments—including nonlinear robot arm trajectory tracking, vehicle dynamics estimation, and autonomous driving sensor fusion—demonstrate that NANO reduces the average root‑mean‑square error by roughly 45 % compared with EKF, UKF, IEKF, and PLF, while incurring only a modest computational overhead comparable to standard Kalman‑type updates. Sensitivity analyses confirm that a small number of natural‑gradient iterations (5–10) and a moderate step size (≈0.5) suffice for convergence in most scenarios.

In summary, the paper offers a rigorous re‑derivation of Gaussian Bayesian filtering, introduces a natural‑gradient‑driven update that eliminates linearization bias, provides solid theoretical guarantees on convergence and error bounds, and validates the approach with both synthetic and real data. The work opens avenues for further research on scalable Fisher‑matrix approximations in high‑dimensional settings and integration with learned dynamics models.