A research team involving miHoYo's Honkai AI R&D team and university collaborators — Tsinghua University, the University of Chinese Academy of Sciences, the University of Hong Kong, and others — has open-sourced GameScaling, a family of models that play video games by generating keyboard and mouse actions from visual observations. The release includes eight checkpoints ranging from 800 million to 27 billion parameters, built through continued pre-training of Qwen3.5 vision-language models.
How it works
GameScaling models are Qwen3.5 vision-language models with continued pre-training on keyboard-and-mouse gameplay. Each call takes the current game frame (plus a short history of past frames and actions) and returns the next 200 milliseconds of keyboard and mouse actions: 6 steps of 33 milliseconds each, plus one mouse movement and scroll for the whole chunk. The model may think before it acts.
The released checkpoints
| Model | Training steps | Weights | GPUs (bf16) |
|---|---|---|---|
| GameScaling-0.8B-S1 | 6,736 | 2.2 GB | 1 |
| GameScaling-0.8B-S3 | 6,760 | 2.2 GB | 1 |
| GameScaling-2B-S1 | 6,736 | 5.4 GB | 1 |
| GameScaling-2B-S3 | 6,760 | 5.4 GB | 1 |
| GameScaling-9B-S1 | 6,736 | 18.8 GB | 1 |
| GameScaling-9B-S3 | 6,760 | 18.8 GB | 1 |
| GameScaling-27B-S1 | 6,995 | 54.7 GB | 2 (tensor parallel) |
| GameScaling-27B-S3 | 7,020 | 54.7 GB | 2 (tensor parallel) |
S1 checkpoints were trained on Minecraft gameplay only. S3 is a balanced mixture: 27 percent Minecraft, the rest from other games. All checkpoints use 1,600 hours of gameplay.
Demonstrated capabilities
The project demonstrates that the models can:
- Minecraft: mining and crafting
- Doom: combat and navigation
- Escape-room environments: puzzle-solving
The models use current and recent game frames together with previous actions to generate short sequences of controls.
Technical notes
The GitHub repository holds the inference client, the standard system-prompt template, example programs and tests; it does not include training code. The client is served with vLLM:
git clone https://github.com/THUSI-Lab/GameScaling.git
cd GameScaling
pip install "vllm>=0.17"
bash scripts/serve_vllm.sh THUSI-Lab/GameScaling-9B-S1 8000
Each developer must tailor the system prompt for their own game. The standard template contains three placeholders: GAME_NAME, GAME_RULES and KEY_BINDINGS.
Licensing and scope
- Models and code: Apache 2.0
- Gameplay data: not released (the training corpus stays private)
- Game rights: remain with the respective publishers
The project is a research initiative; no commercial activity is being conducted with the models.
Why it matters
GameScaling is one of the most complete open releases of game-playing agents to date: eight checkpoints across four backbone sizes, a documented action format, and working inference code under a permissive license. For game developers and AI researchers, it is a concrete benchmark for how training across different virtual worlds affects interactive AI performance — and a ready-made testbed for agent research without any proprietary stack.
Sources: