Coordinated AI agents improve vulnerability exploitation in controlled benchmarks

Turning a software flaw into a working exploit can require many experiments, failed hypotheses and intermediate discoveries. A new preprint presents DoGNAVY-Exploit, a system that coordinates multiple AI agents to preserve that progress rather than repeatedly starting over.

A preprint posted on November 3, 2026, by Yuxuan Wang and colleagues describes a way to improve open-weight AI models’ ability to turn software vulnerabilities into working exploits. Their system, DoGNAVY-Exploit, organizes agents around stages, parallel experiments and shared records of verified progress. The authors report improvements across three controlled cybersecurity benchmarks, although the strongest claims require attention to task selection and comparison conditions.

The central problem is not simply recognizing a bug. Exploit development requires finding useful capabilities, testing whether they work and combining them into a successful attack. According to the paper, agents can become stuck repeating unproductive experiments, while accumulated interaction histories make it harder to retain the overall objective. Relevant security knowledge can also be ineffective when supplied at the wrong stage.

Dividing the work without losing progress

DoGNAVY-Exploit divides the process into three broad stages: reproducing and understanding the vulnerability, discovering useful intermediate capabilities, and assembling those capabilities into a complete exploit. Progress depends on concrete milestones supported by runtime evidence, rather than an agent’s assertion that it has succeeded.

Within each milestone, a route planner proposes different approaches. Task planners translate those approaches into verifiable experiments, and executors investigate them in parallel. A controller evaluates the evidence and determines whether routes should continue, merge or stop. The main experiments allowed up to three candidate routes and three parallel executors.

The system also supplies expert knowledge according to both stage and role. Planners receive strategic guidance, executors receive information relevant to experiments, and controllers receive guidance for evaluating progress. Shared memory records verified facts, rejected assumptions, reusable artifacts and steps for replaying intermediate achievements. Compact summaries carry this information forward without retaining every low-level interaction.

Strong results on two smaller benchmarks

Using GLM-5.2, DoGNAVY-Exploit solved 39 of 40 Cybench tasks, or 97.5%. Cybench covers broader cybersecurity problem solving, not exclusively exploitation of known vulnerabilities. Under the same model and eight-hour limit, the ReAct baseline solved 33 tasks and OpenCode solved 35. Each individual DoGNAVY-Exploit agent was itself built on OpenCode, making that comparison useful for assessing the added orchestration.

On CVE-Bench, which tests exploitation of vulnerable web applications, the system solved 39 of 40 tasks in the zero-day setting and all 40 in the one-day setting. Here, zero-day means the agent must first locate the vulnerability; one-day provides vulnerability information. These results therefore do not demonstrate discovery of previously unknown vulnerabilities outside the benchmark. OpenCode scored 65% and 87.5%, respectively, with the same backbone.

The paper also compares these scores with public frontier-model results. Those comparisons involve different task coverage and original evaluation settings, so they are not controlled head-to-head tests.

Harder tasks expose important qualifications

ExploitGym tests exploit development against userspace programs, the V8 engine and the Linux kernel. Because a full evaluation was resource-intensive, the researchers used subsets selected by their security experts. DoGNAVY-Exploit achieved 79 on-target successes among 225 evaluated tasks with GLM-5.2, and 157 among 485 with GLM-5.3.

Public Claude Code baselines using those models solved 39 and 130 tasks, respectively, across all 869 benchmark tasks. DoGNAVY-Exploit’s larger absolute counts are notable, but the different coverage prevents a like-for-like success-rate comparison. Its eight-hour limit also exceeded the public baselines’ six hours; the authors argue that differing API speeds complicate wall-clock comparisons.

Reaching the benchmark’s flag was not always equivalent to exploiting the intended vulnerability. GLM-5.2 captured flags in 110 cases, but 31 captures were off-target. GLM-5.3 recorded 197 captures, including 40 off-target. Nearly all such cases involved userspace targets, highlighting a distinction between achieving an outcome and demonstrating the specified exploit.

Cost and component evidence

Reported average costs were $4.80 per case for GLM-5.2 and $20.91 for GLM-5.3, calculated using official API prices. The authors estimated 18.94 and 8.64 successful exploits per $1,000. Their metric divides on-target successes by all 869 tasks, treating unevaluated tasks as failures. It is a cost-normalized estimate, not an observed deployment yield, and does not make the selected subsets directly comparable to full-benchmark runs.

A separate ablation study used the same 40 ExploitGym cases with GLM-5.3. The complete system solved 30; removing stages or expert knowledge reduced that to 24, removing parallel execution to 18, and removing memory to 28. Every removal also increased reported resource consumption.

These preprint results support the value of exploitation-specific coordination under the tested conditions. They do not establish equivalent performance against arbitrary deployed software: the evaluations used controlled environments, selected subsets on the hardest benchmark and single-run outcomes rather than repeated-run reliability estimates.