Evolving Prompts Alongside Model Weights Improves AI Benchmark Results

Language models can improve their task performance through two distinct routes: updating their internal numerical parameters, or changing the instructions supplied before a task. A study by Lunjun Zhang, Ryan Chen and Bradly C. Stadie combines these routes in Evolutionary System Prompt Learning, or E-SPL. Their arXiv preprint, examined here in its fourth version dated November 4, 2026, reports improvements on mathematical reasoning and search-based question answering.

The central proposal is to evolve a collection of system prompts while reinforcement learning updates the model. Rather than finding one instruction set before training and keeping it fixed, E-SPL continually revises instructions using evidence from the model’s latest attempts.

Two learning processes sharing experience

A system prompt supplies overarching instructions, such as checking a calculation with an alternative method or choosing a particular mathematical technique. Reinforcement learning changes model weights using rewards assigned to generated answers. E-SPL uses the same task attempts to inform both processes.

In the reported setup, each training batch contains 10 problems. The model generates five attempts per problem under each of three selected prompts. Rewards support weight updates and comparisons among prompts evaluated on the same problems with the same model weights.

The method tracks prompt performance with TrueSkill, a rating system that maintains both an estimated skill and uncertainty. This matters because raw scores from different training stages are not directly comparable: the problems and the model’s weights change. Selection balances promising prompts with less-tested candidates.

A separate, fixed reference model examines successful and unsuccessful attempts and proposes localized instruction edits. This mutation process develops new prompts from the best-performing selected prompt. An occasional crossover step combines complementary guidance from multiple prompts. Mutation reflections use problems with both successful and failed attempts, alongside ground-truth answers; this is supervised training feedback, not learning without external answers.

What the experiments found

The mathematical experiments used DeepSeek-v3.1, with weight changes implemented through low-rank adaptation, a method that trains a comparatively small set of additional parameters. Tests compared E-SPL with fixed-prompt reinforcement learning, iterative self-reflection without weight updates, and prompt evolution without weight updates.

Training on 100 selected DAPO problems and testing on AIME 2025 produced a reported success rate of 60.6% for E-SPL, versus 56.3% for the GRPO reinforcement-learning baseline. On the November 2025 HMMT test, after training on earlier competition problems, the corresponding scores were 52.7% and 50.0%.

The largest reported mathematical gain came from training on AIME problems from 2022–2024 and testing on BeyondAIME, whose problems are at or above AIME difficulty. E-SPL scored 45.1%, compared with 38.8% for GRPO: a difference of 6.3 percentage points.

For search-based question answering, the researchers used gpt-oss-120b and a subset of Natural Questions and HotpotQA, with 196 training problems and 64 test problems. E-SPL reached 48.6%, versus 44.2% for reinforcement learning alone. It also exceeded evolution-only results across the reported task settings.

Accounting for extra computation

The main comparison has an important qualification: E-SPL generated 15 attempts per problem, while the standard reinforcement-learning baseline generated five. The appendix therefore includes a GRPO baseline with 15 attempts, matching the reinforcement-learning component’s rollout budget.

That larger baseline scored 50.5% on HMMT and 39.2% on BeyondAIME, still below E-SPL’s 52.7% and 45.1%. However, this comparison does not eliminate E-SPL’s separate reflection cost. The current implementation adds about 20% wall-clock time per training iteration, largely because reflection runs synchronously.

Most experiments were single runs. Three E-SPL runs on BeyondAIME yielded a mean success rate of 44.8%, with a standard deviation of 0.4 percentage points. This provides some evidence of repeatability for that setting, not a comprehensive uncertainty assessment across all comparisons.

Readable instructions, fallible lessons

Learned prompts included advice on mathematical verification, avoiding unjustified symmetry assumptions and organizing search workflows. The authors suggest that prompts capture verbalizable strategies while weight updates improve execution. They explicitly describe this division as an interpretation of qualitative observations, not a directly measured mechanism.

Readable instructions also expose mistakes. The paper documents learned mathematical rules that omitted conditions or overstated what a calculation established. Being inspectable makes such rules available for correction; it does not make them reliable automatically.

E-SPL also depends on a sufficiently capable fixed reference model, and its best performance still relies on evolved prompts. The safety of autonomous instruction rewriting remains an open question. The reported results support joint prompt-and-weight optimization in these benchmark settings, rather than establishing broadly reliable or unrestricted autonomous self-improvement.