Game-RL: Synthesizing Multimodal Verifiable Game Data to Boost VLMs' General Reasoning

Vision-language reinforcement learning (RL) has primarily focused on narrow domains (e.g. geometry or chart reasoning). This leaves broader training scenarios and resources underexplored, limiting the exploration and learning of Vision Language Models (VLMs) through RL. We find video games inherently provide rich visual elements and mechanics that are easy to verify. To fully use the multimodal and verifiable reward in video games, we propose Game-RL, constructing diverse game tasks for RL training to boost VLMs general reasoning ability. To obtain training data, we propose Code2Logic, a novel approach that adapts game code to synthesize game reasoning task data, thus obtaining the GameQA dataset of 30 games and 158 tasks with controllable difficulty gradation. Unexpectedly, RL training solely on GameQA enables multiple VLMs to achieve performance improvements across 7 diverse vision-language benchmarks, demonstrating the value of Game-RL for enhancing VLMs’ general reasoning. Furthermore, this suggests that video games may serve as valuable scenarios and resources to boost general reasoning abilities. Our code, dataset and models are available at the GitHub repository.

💡 Research Summary

The paper “Game-RL: Synthesating Multimodal Verifiable Game Data to Boost VLMs’ General Reasoning” introduces a groundbreaking framework designed to overcome the domain-specific limitations of current Vision-Language Model (VLM) reinforcement learning. While existing RL approaches for VLMs have been confined to narrow, specialized domains such as geometry or chart reasoning, this research proposes utilizing the rich, interactive, and rule-based environment of video games to foster broader, general reasoning capabilities.

The core motivation stems from the observation that video games provide a unique combination of high-fidelity visual elements and “verifiable rewards.” Unlike static datasets, video games operate on underlying programmatic logic, allowing for an unambiguous feedback loop where the model’s actions can be instantly validated by the game engine. This inherent verifiability is crucial for effective reinforcement learning.

To operationalize this idea, the authors developed “Code2Logic,” a novel methodology that extracts logic from game code to synthesize structured reasoning tasks. This process transforms raw game mechanics into a high-quality, multimodal dataset known as “GameQA.” The GameQA dataset comprises 158 tasks spanning 30 different games, featuring a controllable difficulty gradation. This hierarchical structure enables a curriculum learning approach, where models can progress from fundamental visual recognition to complex, multi-step strategic reasoning.

The experimental results are highly significant. The researchers demonstrated that training VLMs solely on the GameQA dataset leads to measurable performance improvements across seven diverse, non-gaming vision-language benchmarks. This unexpected result proves that the reasoning skills acquired within the game environment are not merely task-specific but are transferable to broader visual reasoning contexts. This suggests that the “reasoning primitives” learned through game-based RL can enhance the model’s ability to interpret complex, real-world visual scenarios.

In conclusion, the paper establishes video games as a potent and scalable resource for training the next generation of intelligent agents. By leveraging the programmable nature of game engines, Game-RL provides a blueprint for creating large-scale, verifiable, and diverse training environments that are essential for the development of General Artificial Intelligence (AGI). The study opens new avenues for using interactive digital simulations as a primary engine for boosting the cognitive breadth of multimodal models.