Smaller is Better: Generative Models Can Power Short Video Preloading

Preloading is widely used in short video platforms to minimize playback stalls by downloading future content in advance. However, existing strategies face a tradeoff. Aggressive preloading reduces stalls but wastes bandwidth, while conservative strategies save data but increase the risk of playback stalls. This paper presents PromptPream, a computation powered preloading paradigm that breaks this tradeoff by using local computation to reduce bandwidth demand. Instead of transmitting pixel level video chunks, PromptPream sends compact semantic prompts that are decoded into high quality frames using generative models such as Stable Diffusion. We propose three core techniques to enable this paradigm: (1) a gradient based prompt inversion method that compresses frames into small sets of compact token embeddings; (2) a computation aware scheduling strategy that jointly optimizes network and compute resource usage; and (3) a scalable searching algorithm that addresses the enlarged scheduling space introduced by scheduler. Evaluations show that PromptStream reduces both stalls and bandwidth waste by over 31%, and improves Quality of Experience (QoE) by 45%, compared to traditional strategies.

💡 Research Summary

The paper “Smaller is Better: Generative Models Can Power Short Video Preloading” introduces PromptPream, a novel preloading paradigm that replaces traditional pixel‑level video chunk transmission with compact semantic prompts decoded by a generative diffusion model (e.g., Stable Diffusion). The authors argue that the long‑standing stall‑vs‑bandwidth‑waste trade‑off in short‑form video platforms can be broken by leveraging idle GPU/NPUs on mobile devices to perform local computation instead of sending large video bitstreams.

Three core technical contributions enable this vision:

1. Gradient‑Based Prompt Inversion – For each video frame, an image‑to‑text model first generates a textual description. A small set of learnable tokens <T> is appended to this description, and only the embeddings of these tokens are optimized via back‑propagation through the full diffusion pipeline (CLIP encoder + denoising UNet). The loss combines pixel‑wise MSE and perceptual LPIPS. After a modest number of iterations, the learned token embeddings, together with the short textual prompt, can reconstruct the original frame with high fidelity. Experiments show that as few as four token embeddings achieve reconstruction quality comparable to full sentence embeddings, while occupying far fewer bits.

2. Computation‑Aware Scheduling – Prompt‑based decoding is orders of magnitude slower than H.265 hardware decoding (≈1 ms vs. >1 s on mobile GPUs). Therefore the scheduler must jointly consider network bandwidth, compute latency, and playback deadlines. Each chunk i is assigned four metrics: visual quality qᵢ, quality variation vᵢ = |qᵢ‑qᵢ₋₁|, stall duration σᵢ (derived from download, decode, and buffer timelines), and bandwidth cost bᵢ. A weighted score fᵢ = w₁·qᵢ – w₂·vᵢ – w₃·σᵢ – w₄·bᵢ guides the selection and ordering of chunks. The system adopts a hybrid encoding: prompt‑inverted tokens are used only for keyframes (I‑frames), while the remaining B/P frames are encoded with low‑bitrate H.265. This dramatically reduces the overall bitrate while keeping decoding latency manageable.

3. Scalable Decision Search via Monte‑Carlo Tree Search (MCTS) – Introducing multiple codecs, bitrates, and out‑of‑order download/decoding creates an exponentially large decision space. The authors design an MCTS algorithm augmented with aggressive pruning to discard branches that cannot meet deadline constraints early. The tree expands only promising schedules, allowing near‑optimal planning within the tight time budget required for real‑time video playback.

The system architecture consists of three roles: an encoder that produces both H.265 streams and prompt‑based representations, a server that stores all versions with metadata (decode latency, bitrate, quality), and a client that runs the computation‑aware scheduler and a decoder dispatcher. The dispatcher routes H.265 chunks to the CPU‑Video Decoder (VD) and prompt chunks to GPU/NPUs running the diffusion model, exploiting parallelism across these hardware units.

Evaluation is performed using the PDAS short‑video simulator, real mobile network traces, and user swipe logs. Compared with state‑of‑the‑art preloading strategies, PromptPream reduces average stall time and bandwidth waste by more than 31 % and improves overall Quality‑of‑Experience (QoE) by 45 %. The gains are especially pronounced under volatile bandwidth or aggressive user scrolling, where prompt‑based keyframes can be decoded ahead of playback while traditional chunks continue downloading.

The paper acknowledges several limitations: the approach relies on devices equipped with capable GPUs or NPUs, which may not hold for low‑end phones; diffusion decoding is energy‑intensive, raising battery‑life concerns; server‑side prompt inversion incurs extra compute cost; and the use of generative models introduces potential copyright and security considerations. Future work includes developing lightweight diffusion variants, energy‑aware scheduling policies, and addressing legal aspects of generated content.

In summary, PromptPream demonstrates that “smaller is better” when the “smaller” refers to semantic prompt representations and the “better” comes from leveraging on‑device generative computation. By tightly integrating gradient‑based prompt inversion, computation‑aware scheduling, and scalable tree‑search planning, the system simultaneously cuts bandwidth consumption, eliminates playback stalls, and delivers higher perceived video quality, pointing toward a new direction for mobile video streaming research.