Balanced conic rectified flow

Rectified flow is a generative model that learns smooth transport mappings between two distributions through an ordinary differential equation (ODE). Unlike diffusion-based generative models, which require costly numerical integration of a generative ODE to sample images with state-of-the-art quality, rectified flow uses an iterative process called reflow to learn smooth and straight ODE paths. This allows for relatively simple and efficient generation of high-quality images. However, rectified flow still faces several challenges. 1) The reflow process requires a large number of generative pairs to preserve the target distribution, leading to significant computational costs. 2) Since the model is typically trained using only generated image pairs, its performance heavily depends on the 1-rectified flow model, causing it to become biased towards the generated data. In this work, we experimentally expose the limitations of the original rectified flow and propose a novel approach that incorporates real images into the training process. By preserving the ODE paths for real images, our method effectively reduces reliance on large amounts of generated data. Instead, we demonstrate that the reflow process can be conducted efficiently using a much smaller set of generated and real images. In CIFAR-10, we achieved significantly better FID scores, not only in one-step generation but also in full-step simulations, while using only of the generative pairs compared to the original method. Furthermore, our approach induces straighter paths and avoids saturation on generated images during reflow, leading to more robust ODE learning while preserving the distribution of real images.

💡 Research Summary

The paper “Balanced Conic Rectified Flow” addresses critical bottlenecks in the Rectified Flow framework, a generative modeling paradigm designed to learn smooth, straight ODE paths between distributions for efficient sampling. While Rectified Flow offers a significant advantage over traditional diffusion models by reducing the need for complex numerical integration through the “reflow” process, the authors identify two fundamental flaws in the current implementation: computational inefficiency and distribution bias.

The first issue, computational cost, arises because the reflow process requires a massive number of generated image pairs to effectively straighten the transport paths, leading to prohibitive training expenses. The second, more insidious problem, is distribution bias. Since the reflow-based training relies exclusively on pairs generated by a previous model (the 1-rectified flow), the learning process becomes trapped in a feedback loop. This causes the model to drift away from the true data distribution, essentially learning to replicate the artifacts and errors present in the previously generated data.

To overcome these challenges, the authors propose “Balanced Conic Rectified Flow.” The core innovation lies in the strategic integration of real images into the reflow training process. By incorporating real images, the model can preserve the integrity of the ODE paths relative to the true data distribution, acting as a regularizer that prevents the model from becoming overly biased toward generated samples. This “balanced” approach allows the reflow process to be conducted much more efficiently, requiring a significantly smaller set of generated pairs to achieve superior results.

Experimental results on the CIFAR-10 dataset demonstrate the superiority of this method. The proposed approach achieves significantly better FID scores in both one-step generation and full-step simulations. Furthermore, the method induces straighter ODE trajectories and effectively prevents the “saturation” effect on generated images, which often plagues purely generative-based reflow. By balancing real and generated data, the proposed framework ensures more robust ODE learning and maintains the fidelity of the target distribution. This research provides a vital blueprint for developing next-generation generative models that are both computationally efficient and distributionally accurate.