Continuous Normalizing Flows for Uncertainty-Aware Human Pose Estimation

Human Pose Estimation (HPE) is increasingly important for applications like virtual reality and motion analysis, yet current methods struggle with balancing accuracy, computational efficiency, and reliable uncertainty quantification (UQ). Traditional regression-based methods assume fixed distributions, which might lead to poor UQ. Heatmap-based methods effectively model the output distribution using likelihood heatmaps, however, they demand significant resources. To address this, we propose Continuous Flow Residual Estimation (CFRE), an integration of Continuous Normalizing Flows (CNFs) into regression-based models, which allows for dynamic distribution adaptation. Through extensive experiments, we show that CFRE leads to better accuracy and uncertainty quantification with retained computational efficiency on both 2D and 3D human pose estimation tasks.

💡 Research Summary

The paper addresses a central challenge in human pose estimation (HPE): achieving high accuracy and computational efficiency while providing reliable uncertainty quantification (UQ). Existing approaches fall into two categories. Regression‑based methods are fast but rely on fixed distributional assumptions (typically Gaussian or Laplace), which limits their ability to model the true, often asymmetric, joint‑position distributions and leads to poorly calibrated uncertainties. Heatmap‑based methods treat keypoint detection as a dense classification problem, producing likelihood heatmaps that naturally encode uncertainty, yet they demand substantial memory and compute, making them unsuitable for real‑time or edge deployments.

To bridge this gap, the authors propose Continuous Flow Residual Estimation (CFRE), a novel framework that augments a standard heteroscedastic regression network with a Continuous Normalizing Flow (CNF). The regression head predicts, for each joint, a mean µ̂ and a scale σ̂ (interpreted as the parameters of a Laplace distribution). These parameters define a simple reference distribution that captures per‑sample variance but remains limited in expressive power. The CNF then transforms a base isotropic Gaussian z(0)∼N(0,I) into a complex target distribution through a neural ordinary differential equation (neural ODE) defined by a time‑dependent vector field f(z(t),t;θ).

Mathematically, the CNF obeys the continuous change‑of‑variables formula:

∂ₜ log Pθ(z(t)) = − tr(∂f/∂z(t)).

Integrating this differential equation from t = 0 to t = 1 yields the log‑density of the transformed variable. The final conditional density of joint locations given an image I is obtained by applying the standard change‑of‑variables with the regression‑derived scaling and shifting:

Pβ,θ(x|I) = (1/σ̂) · Pθ((x − µ̂)/σ̂).

Training minimizes the negative log‑likelihood (NLL) of this composite distribution. Direct computation of the Jacobian trace is expensive; the authors therefore employ the Hutchinson stochastic estimator, which approximates the trace via random vector projections.

A key contribution is the decoupled training strategy. The total loss combines the regression loss Lreg (a Laplace NLL) and a flow loss Lflow, weighted by a self‑adaptive factor λ(σ̂) = c·(1 − σ̂). This factor reduces the influence of the flow component when the regression network is uncertain, allowing the regression head to learn a robust baseline before the CNF refines it. To further simplify optimization, the authors upper‑bound the flow loss (LUB) and design an explicit optimal transport path between the base and target distributions, turning the Jacobian regularization into a simple L2 matching term between the learned vector field and a closed‑form transport velocity.

Experiments are conducted on two standard benchmarks: COCO‑Pose for 2D keypoint detection and Human3.6M for 3D pose estimation. Evaluation metrics include mean Average Precision (mAP) for accuracy and two UQ‑specific scores—Area under Sparsification Error (AUSE) and Area under Reliability Curve (AURG).

Results show that CFRE consistently outperforms pure regression baselines by 1–2 % mAP while using the same backbone (ResNet‑152) and input resolution (384 × 233). Compared to state‑of‑the‑art heatmap methods such as HRNet, CFRE achieves comparable mAP but with dramatically lower FLOPs, confirming its efficiency. In terms of uncertainty, CFRE yields substantially lower AUSE and higher AURG, indicating that its predicted confidence scores are well calibrated with the actual localization error. Qualitative visualizations of sampled joint distributions reveal that the CNF learns anisotropic, multi‑modal shapes that better reflect occlusions and limb ambiguities than the symmetric Laplace or Gaussian assumptions.

Importantly, during inference the CNF module is omitted; only the regression network is executed. Consequently, the runtime cost matches that of a standard regression model, preserving real‑time capability while benefiting from the richer distribution learned during training.

In summary, the paper introduces a principled way to enrich regression‑based human pose estimation with continuous normalizing flows, achieving a rare combination of high accuracy, computational thrift, and trustworthy uncertainty estimates. The approach opens avenues for further research, such as lightweight CNF architectures, multi‑person extensions, and temporal CNFs for video‑based pose tracking.