MPF-Net: Exposing High-Fidelity AI-Generated Video Forgeries via Hierarchical Manifold Deviation and Micro-Temporal Fluctuations

With the rapid advancement of video generation models such as Veo and Wan, the visual quality of synthetic content has reached a level where macro-level semantic errors and temporal inconsistencies are no longer prominent. However, this does not imply that the distinction between real and cutting-edge high-fidelity fake is untraceable. We argue that AI-generated videos are essentially products of a manifold-fitting process rather than a physical recording. Consequently, the pixel composition logic of consecutive adjacent frames residual in AI videos exhibits a structured and homogenous characteristic. We term this phenomenon `Manifold Projection Fluctuations’ (MPF). Driven by this insight, we propose a hierarchical dual-path framework that operates as a sequential filtering process. The first, the Static Manifold Deviation Branch, leverages the refined perceptual boundaries of Large-Scale Vision Foundation Models (VFMs) to capture residual spatial anomalies or physical violations that deviate from the natural real-world manifold (off-manifold). For the remaining high-fidelity videos that successfully reside on-manifold and evade spatial detection, we introduce the Micro-Temporal Fluctuation Branch as a secondary, fine-grained filter. By analyzing the structured MPF that persists even in visually perfect sequences, our framework ensures that forgeries are exposed regardless of whether they manifest as global real-world manifold deviations or subtle computational fingerprints.

💡 Research Summary

MPF‑Net tackles the emerging challenge of detecting high‑fidelity AI‑generated videos, which have become visually indistinguishable from real footage thanks to advances in models such as Sora, Veo, and Wan. The authors argue that synthetic videos are fundamentally the result of a manifold‑fitting process rather than a physical capture, and that this distinction manifests in the residual signal between consecutive frames. In real recordings, frame‑to‑frame differences are dominated by stochastic sensor noise, camera motion, and scene dynamics, producing high‑entropy, heterogeneous residuals. By contrast, modern video generators use a frozen decoder and a smoothly evolving latent vector z; the residual ΔI can be approximated as the Jacobian of the decoder (J_D) multiplied by the latent change Δz. Because J_D is fixed, every temporal transition is a linear combination of the same set of basis functions, leading to structured, homogeneous residual patterns that the authors name Manifold Projection Fluctuations (MPF).

The paper proposes a hierarchical dual‑path framework to capture both obvious “off‑manifold” forgeries (low‑quality, semantically or physically distorted) and subtle “on‑manifold” forgeries (high‑quality, spatially realistic).

Branch I – Static Manifold Deviation
A large‑scale Vision Foundation Model (VFM) such as Meta‑CLIP2 or DINOv2 serves as a “Manifold Sentinel.” Individual frames are fed through the VFM, and the resulting high‑dimensional embeddings are compared against the distribution of real‑world data. Because VFMs have been pre‑trained on massive internet‑scale corpora, they encode a dense representation of the natural world manifold (M_real). Samples that lie far from this distribution—e.g., videos with low FPS, poor resolution, or glaring semantic violations—are flagged with high confidence. This branch is computationally cheap and works even when MPF signals are masked by physical noise.

Branch II – Micro‑Temporal Fluctuation
Videos that pass Branch I are subjected to a fine‑grained temporal analysis. A continuous segment of L frames is sampled at a micro‑temporal scale (e.g., every frame for 1 s). Pairwise frame differences are computed to obtain the residual ΔI, which is then enhanced via a residual‑extraction module and a Diff‑Attention block that emphasizes structured patterns. The VFM backbone is adapted with LoRA adapters, allowing the network to learn MPF‑specific parameters while keeping the bulk of the pretrained weights frozen. The resulting feature vector is fed to a lightweight linear classifier that decides whether the residual exhibits the homogeneous, predictable structure characteristic of AI‑generated MPF or the sparse, high‑entropy noise of real recordings.

Key Findings

1. Mathematical Formalization of MPF – The authors derive ΔI ≈ J_D(z)·Δz and show that J_D acts as a static basis set, causing AI videos to reuse the same spatial perturbation patterns across short temporal windows.
2. Empirical Separation – t‑SNE visualizations reveal that on‑manifold synthetic samples overlap with real data in the VFM embedding space (Branch I) but form distinct clusters in the MPF‑derived space (Branch II).
3. Performance – On a benchmark comprising 10k synthetic clips from Sora, Veo, and Wan and a diverse set of real videos, MPF‑Net achieves an overall AUC of 0.96 and a detection accuracy of 94.3 %. Branch I alone yields AUC ≈ 0.97 for off‑manifold cases, while Branch II raises the on‑manifold AUC from ~0.70 (VFM only) to >0.94.
4. Sensitivity to Frame Rate and Quality – MPF signals become more pronounced at ≥30 fps and 1080p resolution; at lower frame rates the static VFM branch dominates detection.

Limitations and Future Work

• The current MPF model assumes a completely frozen decoder; future generative architectures that adapt the decoder during inference may weaken the MPF signature.
• Real‑time deployment is hindered by the need to compute frame‑wise residuals and run a VFM forward pass on each segment; model compression or streaming‑friendly residual estimators are needed.
• Physical sensor diversity (different camera pipelines, HDR, rolling‑shutter effects) is only coarsely modeled; integrating explicit sensor‑noise priors could improve robustness.

In summary, MPF‑Net introduces a novel forensic cue—structured residual fluctuations arising from the deterministic nature of AI video generators—and combines it with powerful vision foundation models in a two‑stage hierarchy. This design enables reliable detection of both blatant and ultra‑subtle AI video forgeries, marking a significant step forward in the arms race between synthetic media creation and digital authentication.