FastFlow: Accelerating The Generative Flow Matching Models with Bandit Inference

Flow-matching models deliver state-of-the-art fidelity in image and video generation, but the inherent sequential denoising process renders them slower. Existing acceleration methods like distillation, trajectory truncation, and consistency approaches are static, require retraining, and often fail to generalize across tasks. We propose FastFlow, a plug-and-play adaptive inference framework that accelerates generation in flow matching models. FastFlow identifies denoising steps that produce only minor adjustments to the denoising path and approximates them without using the full neural network models used for velocity predictions. The approximation utilizes finite-difference velocity estimates from prior predictions to efficiently extrapolate future states, enabling faster advancements along the denoising path at zero compute cost. This enables skipping computation at intermediary steps. We model the decision of how many steps to safely skip before requiring a full model computation as a multi-armed bandit problem. The bandit learns the optimal skips to balance speed with performance. FastFlow integrates seamlessly with existing pipelines and generalizes across image generation, video generation, and editing tasks. Experiments demonstrate a speedup of over 2.6x while maintaining high-quality outputs. The source code for this work can be found at https://github.com/Div290/FastFlow.

💡 Research Summary

FastFlow addresses the inference bottleneck of flow‑matching (FM) generative models, which, despite requiring fewer sampling steps than diffusion models, still suffer from a sequential denoising process that demands a neural‑network evaluation at every timestep. The authors observe that FM trajectories are often close to linear because the models are trained to follow straight‑line paths in latent space. Consequently, many intermediate steps contribute only marginal adjustments to the generated sample.

To exploit this redundancy, FastFlow replaces a subset of full model evaluations with a cheap finite‑difference extrapolation. Using the most recent velocity predictions vₖ and an earlier reference vₚ, the method computes an approximate velocity at a future time via a first‑order Taylor expansion:
 v̂ₜ₊Δt ≈ vₜ + Δt·(vₜ − vₚ)/(t − tₚ).
When this approximated velocity is used in an Euler update, the state can be advanced by m steps without invoking the heavy neural network, achieving “zero compute” for those steps.

The central challenge is deciding how many steps can be safely skipped before the accumulated error becomes noticeable. FastFlow formulates this decision as an online multi‑armed bandit (MAB) problem. At each timestep t, a bandit selects an action α from a discrete set (e.g., skip 1–5 steps). The reward balances speed and fidelity:
 r(α) = μ·α − ℓ(v̂, v),
where μ controls the trade‑off and ℓ is a discrepancy measure such as mean‑squared error between the approximated and true velocities. The bandit employs an Upper‑Confidence‑Bound (UCB) strategy, initially exploring all arms and later exploiting those that yield high rewards. This adaptive policy automatically reduces the skip length in regions of high curvature or rapid motion, while taking larger jumps in smooth regions.

Theoretical analysis (Theorem 3.1) shows that if |S| steps are approximated out of T total steps with uniform step size Δt, the final state error satisfies
 e_T = ‖x̂_T − x_T‖ = O(|S|·Δt³).
Thus, the error grows linearly with the number of skipped steps but is heavily dampened by the cubic dependence on the step size, guaranteeing stability for typical FM discretizations.

FastFlow requires no retraining, auxiliary networks, or task‑specific hyper‑parameters beyond the simple scalar μ. It can be inserted as a plug‑and‑play module into any existing FM pipeline, whether the model is a text‑to‑image generator, an image editor, or a text‑to‑video synthesizer.

Experiments span three domains: (1) text‑to‑image generation on the GenEval benchmark, (2) image editing on COCO‑Edit, and (3) text‑to‑video synthesis on a custom video benchmark. Baselines include the vanilla FM model, the cache‑based TeaCache method, knowledge‑distillation FM variants, and consistency‑trained FM models. Across all settings, FastFlow achieves an average speed‑up of 2.6× (up to 3.1× in favorable cases) while preserving generation quality. Quantitative metrics such as FID, IS, LPIPS for images, and FVD, VMAF for videos show only marginal degradation (typically <2 % increase in FID or <0.01 LPIPS). Qualitative analysis reveals that the bandit dynamically reduces skip lengths for complex prompts or fast‑moving video frames, preventing noticeable artifacts.

Ablation studies dissect the contribution of each component: (i) using only the finite‑difference extrapolation without a bandit yields modest speed‑ups but suffers from uncontrolled error; (ii) a static skip schedule (e.g., always skip 2 steps) underperforms the adaptive bandit; (iii) varying μ demonstrates the expected trade‑off curve between speed and fidelity. Computational overhead of the bandit is negligible, consisting of simple reward updates and confidence bound calculations.

Limitations include reliance on a first‑order Taylor approximation, which may be insufficient for highly nonlinear dynamics; extending the method to second‑ or higher‑order extrapolations could further improve speed without sacrificing quality. Additionally, the scalar μ must be tuned per application; future work could explore meta‑learning or reinforcement‑learning approaches to automate this tuning.

In summary, FastFlow introduces a principled, training‑free acceleration technique for flow‑matching generative models. By combining finite‑difference velocity extrapolation with an online multi‑armed bandit that learns per‑sample skip policies, it delivers substantial inference speed gains across image and video generation tasks while maintaining high visual fidelity. The method’s simplicity, theoretical grounding, and broad applicability make it a compelling addition to the toolbox of practitioners seeking real‑time or resource‑constrained deployment of FM‑based generative systems.