TRANSMISSION // 2611.23219v1

Can language models learn to improve at puzzle games?

Source published

In a controlled study, some newer frozen language models improved at puzzle games by devising practice and revising text instructions. The results do not establish general self-improvement.

AI-generated conceptual illustration for Can language models learn to improve at puzzle games?
AI-generated conceptual illustration; not documentary evidence.

Could a language model work out how to practise without being given a learning plan? Researchers tested this in a game sandbox, where some models generated games, observed attempts and edited text notes. The study examines improvement in this narrow setting—not autonomous learning in general.

Who designs the learning process?

People typically decide what examples a model sees, how its performance is assessed and how its instructions are updated. This study asks whether a language-model agent can organize those steps itself when given a broad goal and basic tools, but no prescribed improvement loop.

The researchers tested the idea with puzzle games whose mechanics were not disclosed to the player. Two instances of the same frozen model took on different roles: a “teacher” could create games, observe a player’s attempts and edit the text instructions available to that player. The teacher could change prompts and skill notes, but not the model’s underlying weights.

The teacher had a limited budget and did not receive real-world tasks or held-out benchmark scores as feedback during learning. Researchers used held-out games to assess the resulting text afterward. This distinction is important: the experiment concerns context—the written material supplied to a model—not retraining the model itself.

How the study worked

The team ran three independent teacher sessions for each of six models from three model families. In the main setup, a teacher could observe up to 150 gameplays and make up to 20 updates. The paper defines an update as a single edit tool-call to one context file. The practice environment used PuzzleScript, a format for grid-based games. Researchers compared player performance with the edited text against performance with the starting instructions, using games held back from practice.

They also assessed transfer to longer PuzzleScript games, Level 1 games from ARC-AGI-3, and MazeBench, a 3D navigation environment presented through ASCII-style views. Further experiments changed the teacher’s tools and the information it received about evaluation settings. The canonical PuzzleScript practice setup used a 20-turn game horizon; transfer tests included longer horizons, including a 100-turn limit in MazeBench.

What the models did

The newer models in each family improved on the main held-out PuzzleScript test, while older models showed little or no improvement. The paper reports gains of 20% for Claude Opus 5, 10% for GPT 6-Astra and 6% for GLM 5.3 on that test. These findings come from three teacher sessions per model and describe this experimental setup, not a general ranking of models.

Effective teachers sometimes generated games with calibrated difficulty, compared the player with and without updated notes, and checked whether advice helped or introduced setbacks. Some designed games to expose weaknesses. In some cases, teachers spent more effort on games with low-to-medium observed win rates. These behaviors resemble ideas from research on self-regulated learning and self-generated curricula. The study finds associations between such strategies and successful improvement; it does not establish that the strategies caused the gains.

The authors report benchmark-specific transfer: optimized text improved results on PuzzleScript-t20 and the longer-horizon PuzzleScript-t30, as well as on ARC-AGI-3 Level 1 and MazeBench. These results concern the tested benchmarks and configurations; they do not establish general transfer to all game mechanics or 3D navigation tasks. Extra information about evaluation settings had mixed effects: it could help teachers select relevant practice, but could also encourage specialization that weakened transfer. In one experiment, a teacher with file-editing and command-line tools created and ran game simulations without a dedicated gameplay tool. That suggests this tool may not be essential for some models in this setup; it does not show that models need no scaffolding more broadly.

What remains uncertain

The work studies particular frozen language models, text-context changes and controlled game tasks. It does not show that agents can improve at open-ended real-world work or change their own model weights. Although the teacher received no external data during learning, models may have acquired useful game-related knowledge during their original training.

The authors also caution that their measures do not explain why improvement occurred. The link between recognizable learning strategies and gains is not shown to be causal, and other behaviors may matter. Additional editing produced diminishing returns in some experiments. The authors further note that games may be a favorable domain because models could already have useful prior knowledge about them.

This article is based on the paper text for an arXiv preprint. A preprint is not, by itself, evidence of peer-reviewed consensus. The results are best read as a demonstration that some models can organize limited, text-based learning in a game sandbox—not as proof of reliable, general-purpose AI self-improvement.

Automatically matched by shared topic keywords. These links do not imply that the sources agree or establish a consensus.