Model Predictive Control via Probabilistic Inference: A Tutorial and Survey

This paper presents a tutorial and survey on probabilistic inference-based model predictive control (PI-MPC) for robotics. PI-MPC reformulates finite-horizon optimal control as inference over an optimal control distribution expressed as a Boltzmann distribution weighted by a control prior, and generates actions through variational inference. In the tutorial part, we derive this formulation and explain action generation via variational inference, highlighting Model Predictive Path Integral (MPPI) control as a representative algorithm with a closed-form sampling update. In the survey part, we organize existing PI-MPC research around key design dimensions, including prior design, multi-modality, constraint handling, scalability, hardware acceleration, and theoretical analysis. This paper provides a unified conceptual perspective on PI-MPC and a practical entry point for robotics researchers and practitioners.

💡 Research Summary

This paper provides a comprehensive tutorial and survey on Probabilistic Inference‑based Model Predictive Control (PI‑MPC), a class of sampling‑based MPC methods that formulate finite‑horizon optimal control as a probabilistic inference problem. The authors first derive the optimal control distribution from the classic optimal control problem. By introducing a binary optimality variable for each time step and representing the dynamics as a graphical model, they obtain a posterior trajectory distribution proportional to the product of a likelihood term (encoding the cost) and a prior over trajectories. Assuming a Boltzmann likelihood p(O=1|τ)=η⁻¹exp(−J(τ)/λ), the optimal control distribution over action sequences becomes

π*(u₀:T‑1) ∝ exp(−Jτ/λ) p(u₀:T‑1),

where p(u) is a user‑defined prior (often Gaussian) and λ is a temperature parameter that balances cost minimisation against adherence to the prior. The derivation is cast as a KL‑divergence minimisation problem: the controllable distribution π is chosen to minimise D_KL(π‖p(·|optimality)). Using a Lagrange multiplier to enforce normalisation yields the closed‑form expression above.

The tutorial then focuses on Model Predictive Path Integral (MPPI) control as a concrete instantiation of this framework. MPPI samples N control sequences from the prior, rolls each out through the dynamics, evaluates the cumulative cost J, and re‑weights the samples with wⁱ∝exp(−Jⁱ/λ). The weighted average of the sampled controls updates the mean of the prior, producing a new control distribution for the next receding‑horizon iteration. Because the sampling, rollout, and weighting steps are embarrassingly parallel, MPPI maps naturally onto GPUs, TPUs, or even FPGA accelerators, enabling real‑time operation for high‑dimensional robots.

The second half of the paper surveys the PI‑MPC literature, organising it around six design dimensions:

1. Prior Design – Choices range from simple isotropic Gaussians to mixture models and learned priors from data or neural policies. The prior encodes actuator limits, nominal behaviours, and exploration noise. Its covariance directly influences sample diversity and the “symmetry‑breaking” phenomenon where multimodal distributions collapse to a single mode as the robot approaches obstacles.

2. Multi‑Modality Handling – When several qualitatively different trajectories are optimal (e.g., left/right avoidance), methods such as importance‑weighted resampling, clustering of samples, or adding entropy bonuses are used to preserve multiple modes during inference.

3. Constraint Handling – Input constraints are incorporated by shaping the prior (e.g., truncating a Gaussian) while state constraints are typically enforced softly via penalty terms or indicator functions in the cost. Recent work explores projection‑based sampling and Lagrangian multiplier updates to treat hard constraints more rigorously.

4. Scalability and Dimensionality Reduction – For long horizons or high‑DOF systems, researchers employ low‑dimensional action primitives, hierarchical sampling trees, or model‑based pruning to keep the required sample count tractable. Adaptive temperature schedules and sample‑size selection strategies further improve efficiency.

5. Hardware Acceleration – The authors catalogue implementations that exploit CUDA kernels, TensorRT optimisations, and custom FPGA pipelines. Benchmarks show MPPI achieving millisecond‑scale planning for quadrotors, legged robots, and autonomous vehicles when run on modern GPUs.

6. Theoretical Analysis – Recent advances provide convergence guarantees under certain regularity conditions, bound the required number of samples as a function of λ and the KL‑regularisation term, and relate the temperature to exploration‑exploitation trade‑offs in reinforcement‑learning analogues. Connections to Energy‑Based Models (EBMs) are highlighted, noting that PI‑MPC augments EBMs with an explicit action prior derived from system dynamics.

The paper also releases a PyTorch‑based MPPI playground (https://github.com/kohonda/mppi_playground) to lower the entry barrier for practitioners. Throughout, the authors stress that PI‑MPC offers three core advantages: (i) parallel‑friendly computation, (ii) natural stochasticity for exploration and data augmentation, and (iii) differentiable pipelines that can be integrated with learned dynamics or cost models.

In conclusion, the tutorial demystifies the probabilistic foundations of PI‑MPC, while the survey maps the current research landscape, identifies open challenges (automatic λ and prior tuning, rigorous hard‑constraint handling, and tighter theoretical bounds), and points toward future directions such as hybrid deterministic‑probabilistic controllers and end‑to‑end learning of the entire PI‑MPC pipeline. This work serves as both an educational resource and a practical guide for robotics researchers aiming to adopt or extend PI‑MPC in real‑world systems.