FlashVideo: Flowing Fidelity to Detail for Efficient High-Resolution Video Generation

DiT models have achieved great success in text-to-video generation, leveraging their scalability in model capacity and data scale. High content and motion fidelity aligned with text prompts, however, often require large model parameters and a substantial number of function evaluations (NFEs). Realistic and visually appealing details are typically reflected in high-resolution outputs, further amplifying computational demands-especially for single-stage DiT models. To address these challenges, we propose a novel two-stage framework, FlashVideo, which strategically allocates model capacity and NFEs across stages to balance generation fidelity and quality. In the first stage, prompt fidelity is prioritized through a low-resolution generation process utilizing large parameters and sufficient NFEs to enhance computational efficiency. The second stage achieves a nearly straight ODE trajectory between low and high resolutions via flow matching, effectively generating fine details and fixing artifacts with minimal NFEs. To ensure a seamless connection between the two independently trained stages during inference, we carefully design degradation strategies during the second-stage training. Quantitative and visual results demonstrate that FlashVideo achieves state-of-the-art high-resolution video generation with superior computational efficiency. Additionally, the two-stage design enables users to preview the initial output and accordingly adjust the prompt before committing to full-resolution generation, thereby significantly reducing computational costs and wait times as well as enhancing commercial viability.

💡 Research Summary

FlashVideo introduces a two‑stage framework designed to tackle the prohibitive computational cost of high‑resolution text‑to‑video generation while preserving both prompt fidelity and visual detail. In the first stage, a large‑capacity 5‑billion‑parameter DiT model (CogVideoX‑5B) is fine‑tuned using parameter‑efficient LoRA adapters to operate at a low resolution of roughly 270p. This stage retains the full 50 function evaluations (NFE) typical of state‑of‑the‑art diffusion models, allowing it to generate videos that align closely with the input text and motion cues within about 30 seconds. The output of this stage serves as a preview that users can inspect and edit before committing to full‑resolution synthesis.

The second stage focuses on upscaling to 1080p and enriching fine‑grained details. A smaller 2‑billion‑parameter DiT (CogVideoX‑2B) equipped with 3‑D Rotary Positional Embedding (RoPE) processes the low‑resolution latent representation. Rather than starting from Gaussian noise, FlashVideo employs flow‑matching: it defines a time‑independent target as the difference between high‑resolution and low‑resolution latents (T_target = Z_HR − Z_LR) and linearly interpolates between them. This creates an almost straight ordinary differential equation (ODE) trajectory, enabling the use of a simple Euler solver with only four steps (four NFEs) to reach the high‑resolution latent.

Training the high‑resolution model requires paired low‑ and high‑resolution latents. To simulate low‑resolution inputs, the authors apply a combination of pixel‑space degradations (random blur and resize) and latent‑space noise injection, a process they term “DEG pixel & DEG latent.” This aggressive degradation removes fine structures from the high‑quality videos, forcing the second‑stage model to truly reconstruct missing details rather than merely copying them. Full 3‑D attention is retained to ensure temporal coherence across frames, preventing the model from cheating by replicating static details.

Empirical results on the VBench‑Long benchmark show FlashVideo achieving a top score of 83.29, indicating superior semantic and motion consistency. In terms of speed, generating a 1080p, 6‑second video (8 fps) takes on average 102.3 seconds with only four NFEs, compared to 2150 seconds for a single‑stage 5 B DiT and 571.5 seconds for conventional cascade diffusion that still starts from noise. This represents roughly a 20‑fold speedup over the single‑stage baseline and a 5‑fold improvement over existing cascades.

Beyond raw performance, the two‑stage design offers practical benefits: users can preview low‑resolution results, adjust prompts, and avoid unnecessary high‑resolution computation, reducing both cloud costs and end‑user latency. The authors also discuss limitations, noting that extending the approach to longer sequences or 4K resolutions will require additional memory and algorithmic optimizations, and that more complex motion may benefit from non‑linear interpolation or adaptive solvers.

In summary, FlashVideo’s contributions are threefold: (1) decoupling prompt fidelity and visual quality into separate, capacity‑matched stages; (2) introducing flow‑matching with near‑linear ODE trajectories to drastically cut high‑resolution NFEs; and (3) designing a robust degradation pipeline that forces genuine detail reconstruction. This combination delivers state‑of‑the‑art high‑resolution video generation with unprecedented efficiency, opening the door for scalable commercial deployment of text‑to‑video services.