DiffVL: Diffusion-Based Visual Localization on 2D Maps via BEV-Conditioned GPS Denoising

Accurate visual localization is crucial for autonomous driving, yet existing methods face a fundamental dilemma: While high-definition (HD) maps provide high-precision localization references, their costly construction and maintenance hinder scalability, which drives research toward standard-definition (SD) maps like OpenStreetMap. Current SD-map-based approaches primarily focus on Bird’s-Eye View (BEV) matching between images and maps, overlooking a ubiquitous signal-noisy GPS. Although GPS is readily available, it suffers from multipath errors in urban environments. We propose DiffVL, the first framework to reformulate visual localization as a GPS denoising task using diffusion models. Our key insight is that noisy GPS trajectory, when conditioned on visual BEV features and SD maps, implicitly encode the true pose distribution, which can be recovered through iterative diffusion refinement. DiffVL, unlike prior BEV-matching methods (e.g., OrienterNet) or transformer-based registration approaches, learns to reverse GPS noise perturbations by jointly modeling GPS, SD map, and visual signals, achieving sub-meter accuracy without relying on HD maps. Experiments on multiple datasets demonstrate that our method achieves state-of-the-art accuracy compared to BEV-matching baselines. Crucially, our work proves that diffusion models can enable scalable localization by treating noisy GPS as a generative prior-making a paradigm shift from traditional matching-based methods.

💡 Research Summary

DiffVL introduces a fundamentally new paradigm for visual localization that moves away from traditional image‑to‑map matching and instead treats the problem as a conditional denoising task for noisy GPS trajectories. The authors first identify a key limitation of existing standard‑definition (SD) map‑based methods: they rely heavily on Bird’s‑Eye‑View (BEV) feature matching while ignoring the ubiquitous GPS signal, which, despite being noisy due to multipath effects in urban environments, carries valuable coarse positional information. To exploit this, DiffVL leverages denoising diffusion probabilistic models (DDPMs) to learn a reverse diffusion process that progressively refines a GPS trajectory conditioned on multimodal cues from a front‑view camera image and an SD map.

The architecture consists of four modules. The Image Encoder uses a ResNet‑101 backbone to extract multi‑scale features, predicts per‑pixel depth distributions, and applies a differentiable polar‑to‑Cartesian view transformation to generate geometrically consistent BEV feature maps. The Map Encoder rasterizes OpenStreetMap vector data into a three‑channel image (roads, buildings, natural features) and extracts semantic priors with a VGG‑16 network. The Diffusion Guidance Generator fuses BEV and map features via cross‑attention, producing a conditional latent vector z that encodes both visual perception and map semantics. Finally, the Diffusion Head receives a noisy GPS sequence (with Gaussian noise injected at each diffusion timestep) and iteratively removes the noise, outputting a refined 3‑DoF pose (x, y, θ).

Training is driven by a dual‑objective loss. The primary trajectory refinement loss L_diff enforces kinematic and temporal coherence, penalizing deviations from smooth motion and encouraging realistic speed/acceleration profiles. The auxiliary localization prior loss L_loc aligns the denoised pose with BEV‑map features, acting as a geometric regularizer that keeps predictions anchored to the road network. This combination forces shared encoders to learn representations that are both discriminative for visual matching and consistent with the map’s coordinate system.

Experiments on three large‑scale autonomous‑driving datasets—KITTI, nuScenes, and MGL—show that DiffVL outperforms state‑of‑the‑art BEV‑matching baselines such as OrienterNet. Average position error drops from ~0.42 m (baseline) to ~0.28 m (DiffVL), a reduction of over 30 %, while heading error falls below 2°. Notably, in regions where raw GPS error exceeds 5 m, the diffusion model’s ability to capture multimodal distributions enables it to select the most plausible pose, demonstrating robustness to severe GPS noise and perceptual aliasing. Ablation studies confirm that both the cross‑modal guidance and the localization prior loss are essential: removing the guidance reduces performance dramatically, and omitting L_loc leads to systematic drift away from the road network.

The method runs at approximately 30 FPS on a modern GPU, indicating feasibility for real‑time deployment. The authors discuss future directions, including lightweight variants for edge devices, integration of additional sensors (LiDAR, IMU), online adaptation to map updates, and reducing diffusion steps for faster inference.

In summary, DiffVL redefines noisy GPS as a generative prior rather than a discarded signal, and by conditioning a diffusion model on BEV visual features and SD maps, it achieves sub‑meter localization without the prohibitive cost of HD maps. This work opens a new research avenue where diffusion‑based conditional generation can replace deterministic matching in visual localization, offering both higher accuracy and greater robustness in real‑world, large‑scale deployments.