MVGSR: Multi-View Consistent 3D Gaussian Super-Resolution via Epipolar Guidance

Scenes reconstructed by 3D Gaussian Splatting (3DGS) trained on low-resolution (LR) images are unsuitable for high-resolution (HR) rendering. Consequently, a 3DGS super-resolution (SR) method is needed to bridge LR inputs and HR rendering. Early 3DGS SR methods rely on single-image SR networks, which lack cross-view consistency and fail to fuse complementary information across views. More recent video-based SR approaches attempt to address this limitation but require strictly sequential frames, limiting their applicability to unstructured multi-view datasets. In this work, we introduce Multi-View Consistent 3D Gaussian Splatting Super-Resolution (MVGSR), a framework that focuses on integrating multi-view information for 3DGS rendering with high-frequency details and enhanced consistency. We first propose an Auxiliary View Selection Method based on camera poses, making our method adaptable for arbitrarily organized multi-view datasets without the need of temporal continuity or data reordering. Furthermore, we introduce, for the first time, an epipolar-constrained multi-view attention mechanism into 3DGS SR, which serves as the core of our proposed multi-view SR network. This design enables the model to selectively aggregate consistent information from auxiliary views, enhancing the geometric consistency and detail fidelity of 3DGS representations. Extensive experiments demonstrate that our method achieves state-of-the-art performance on both object-centric and scene-level 3DGS SR benchmarks.

💡 Research Summary

**
The paper addresses a critical gap in 3‑D Gaussian Splatting (3DGS): models trained on low‑resolution (LR) images cannot produce high‑resolution (HR) renderings because the Gaussian primitives lack sufficient texture detail. Existing super‑resolution (SR) solutions fall into two categories. Single‑image SR (SISR) methods treat each view independently, which leads to severe cross‑view inconsistency and fails to exploit complementary information across viewpoints. Video‑based SR approaches, on the other hand, rely on temporal continuity and frame ordering, making them unsuitable for unstructured multi‑view datasets that are common in photogrammetry pipelines.

MVGSR (Multi‑View Consistent 3D Gaussian Super‑Resolution) proposes a unified framework that works on arbitrarily organized multi‑view collections without requiring any temporal ordering. The system consists of three major components: (1) an auxiliary view selection algorithm that uses camera pose information, (2) a multi‑view SR network equipped with an epipolar‑constrained attention mechanism, and (3) a joint optimization of the 3DGS representation using both the original LR images and the newly generated HR images.

Auxiliary View Selection
Given COLMAP‑estimated intrinsic and extrinsic parameters for all input cameras, the method first filters candidate auxiliary views for each target view based on three geometric criteria: (i) the auxiliary camera must be positioned closer to the scene centre than the target (ensuring finer detail capture), (ii) there must be sufficient overlap of viewing cones (guaranteeing shared content), and (iii) the auxiliary pose should not be too close to the target to avoid redundancy. After filtering, a mixed distance metric combines Euclidean position distance and directional cosine distance, weighted by a parameter λ (empirically set to 0). Candidates are then ranked by this metric, and a fixed number N of auxiliary views are sampled at regular intervals along the sorted list, ensuring both informativeness and diversity.

Multi‑View SR Network Architecture
The network is organized into three modules:

1. Multi‑View Feature Extraction (MVFE) – three Residual Epipolar Transformer (RET) blocks operate at progressively coarser scales. Each RET contains an Epipolar‑Guided Spatial Transformer (EST) that projects target and auxiliary feature maps onto epipolar lines using the known camera geometry. The EST computes attention weights only along geometrically consistent correspondences, effectively suppressing mismatched regions and reducing computational load.

2. Single‑Image Prior (SIP) – a SwinIR‑based feature extractor pre‑trained on large SISR datasets supplies a strong single‑image prior for the target view. This module supplies high‑frequency detail that may be missing from the auxiliary set, especially when the number of auxiliary views is limited.

3. Multi‑Scale Feature Fusion (MSFF) – at each scale, MVFE’s multi‑view features are concatenated with SIP’s single‑image features, fused via 1×1 and 3×3 convolutions, and then up‑sampled to the next scale. This progressive fusion yields a final HR image that blends consistent multi‑view information with powerful single‑image priors.

Epipolar‑Constrained Multi‑View Attention
The EST is the core novelty. By leveraging the epipolar geometry inherent to any pair of calibrated cameras, the attention mechanism restricts the search space to epipolar lines, dramatically improving both efficiency and geometric consistency. The process involves (a) projecting each pixel from the target view onto the corresponding epipolar line in an auxiliary view using the relative pose and depth estimate, (b) computing similarity scores (e.g., cosine similarity) only along these lines, and (c) normalizing to obtain attention weights. This ensures that only geometrically plausible correspondences contribute to the fused representation, mitigating the “ghosting” artifacts common in naïve multi‑view attention.

3DGS Optimization with Anti‑Aliasing Sub‑Pixel Loss
After generating HR images, the framework jointly optimizes the 3DGS parameters (Gaussian positions, covariances, colors, and opacities). In addition to the standard photometric loss between rendered and HR images, an anti‑aliasing sub‑pixel loss is introduced to penalize high‑frequency discrepancies at sub‑pixel precision, encouraging the Gaussian primitives to align more accurately with the HR texture. This loss is crucial for preventing blur and aliasing when the splatted Gaussians are rasterized at high resolution.

Experimental Validation
The authors evaluate MVGSR on both object‑centric datasets (Blender) and large‑scale scene datasets (LLFF, Tanks & Temples). Baselines include SRGS, GaussianSR, SuperGS (SISR‑based) and SM (video‑SR‑based). Metrics comprise PSNR, SSIM, LPIPS, and a newly proposed cross‑view consistency score. MVGSR consistently outperforms all baselines, achieving the highest PSNR/SSIM gains (often >2 dB) and markedly lower LPIPS, indicating sharper textures. The consistency metric improves by more than 10 % relative to the best baseline, confirming the effectiveness of the epipolar‑guided attention. Ablation studies demonstrate that (i) random auxiliary view selection degrades performance, (ii) removing EST and using global attention reduces both quality and consistency, and (iii) omitting the SIP module lowers high‑frequency detail recovery.

Limitations and Future Work
The current implementation fixes the number of auxiliary views N, which may be sub‑optimal for extremely dense capture setups where more views could provide additional detail. Moreover, EST assumes reasonably accurate depth estimates; large depth errors can misplace epipolar correspondences and hurt performance. Future directions include adaptive selection of N based on scene complexity, probabilistic epipolar attention that accounts for depth uncertainty, and lightweight variants of the network for real‑time applications.

Impact
MVGSR delivers a practical solution for upgrading low‑resolution multi‑view captures to high‑quality 3DGS representations. By marrying pose‑driven auxiliary view selection with geometry‑aware attention, it resolves the long‑standing trade‑off between texture fidelity and cross‑view consistency. This advancement opens the door to high‑resolution neural rendering in domains such as virtual tourism, digital twins, and immersive VR/AR, where high‑quality, real‑time view synthesis from unstructured photo collections is essential.