Foveated Video Streaming for Cloud Gaming

Good user experience with interactive cloud-based multimedia applications, such as cloud gaming and cloud-based VR, requires low end-to-end latency and large amounts of downstream network bandwidth at the same time. In this paper, we present a foveated video streaming system for cloud gaming. The system adapts video stream quality by adjusting the encoding parameters on the fly to match the player’s gaze position. We conduct measurements with a prototype that we developed for a cloud gaming system in conjunction with eye tracker hardware. Evaluation results suggest that such foveated streaming can reduce bandwidth requirements by even more than 50% depending on parametrization of the foveated video coding and that it is feasible from the latency perspective.

💡 Research Summary

The paper addresses the bandwidth‑latency dilemma inherent in cloud gaming by introducing a real‑time foveated video streaming system that leverages the player’s gaze to adapt encoding quality on a per‑macroblock basis. Human visual acuity is highest within the fovea (approximately 2 % of the visual field), so the authors propose to encode the region around the current gaze with low quantization (high quality) while increasing the quantization parameter (QP) for peripheral macroblocks, thereby reducing the overall bitrate.

Implementation details: the authors built a prototype using the open‑source GamingAnywhere platform as the cloud gaming server and a Tobii 4C consumer eye‑tracker on the client. Gaze coordinates are streamed from client to server with minimal latency. On the server, the x264 encoder runs in CRF (Constant Rate Factor) mode with zerolatency tuning. A QP offset array QO(i,j) is computed for each macroblock using a two‑dimensional Gaussian: QO(i,j)=QOmax·