DualWorldBench tests whether AI agents can turn delivery goals into workable plans

An AI agent can identify destinations and still produce a delivery plan that breaks the rules or travels unnecessarily far. In the preprint DualWorldBench: Can Agents Plan Deliveries across Symbolic and Grounded Worlds?, posted on November 3, 2026, Yang Chen and colleagues introduce a benchmark designed to distinguish these failures.

The benchmark uses a shared pickup–delivery interface across three components: TextWorld for symbolic scheduling, SimWorld for map-based planning, and DualWorld-VQA for diagnostic questions. Its central finding is qualified: some models perform reliably in several settings, but local spatial understanding does not consistently translate into efficient, constraint-respecting plans.

Two kinds of planning difficulty

TextWorld supplies structured information rather than images. Agents must choose action sequences while respecting carrying capacity, collecting items before delivering them, and accounting for task values and timing. In mixed-task settings, carrying a priority Type-C item prevents delivery of lower-priority Type-A or Type-B items until the Type-C items are delivered.

The benchmark varies both task composition and release timing. Some instances contain only one task type; others mix three types with different rewards and lateness penalties. In static instances, every task is available immediately and deadlines are non-binding. In online instances, new tasks arrive over time. An arrival can interrupt movement, discard the current plan and trigger replanning from the agent’s updated position.

SimWorld instead focuses on spatial structure. Its current release makes all tasks available at the start and gives them uniform rewards. Thirteen simulated scenes span open layouts, two groups of procedural environments, indoor space and campus traversal. Walls, corridors and detours mean that nearby coordinates need not correspond to short walking routes.

Agents receive coordinates, an annotated bird’s-eye map, or both. They output a sequence of nodes; the evaluator translates that sequence into movement, pickup and delivery actions. Travel costs come from preconstructed navigation graphs and local access offsets, rather than requiring the model to control every movement.

Strong rewards do not guarantee efficient scheduling

The researchers evaluate 14 language models and three solver baselines in TextWorld, with task counts ranging from five to 15. Gemini-2.5-Flash has the highest average model reward, 265.73, compared with 264.36 for Online DP and 278.17 for Offline DP.

Those comparisons require context. Online DP plans using currently visible tasks, whereas Offline DP receives future task releases as well. The latter is a hindsight reference, not an online agent operating with the same information. Completion time also means simulated delivery time, not model inference speed. Gemini-2.5-Flash’s average completion time is longer than either DP baseline’s.

The error analysis separates missing deliveries from late deliveries and priority violations. Gemini-2.5-Flash earns 93.4% of the total base reward available across the evaluated instances, with its remaining loss attributed to lateness. Weaker models lose much more through tasks they never deliver. This distinguishes basic coverage problems from the harder scheduling problem of delivering on time.

Coverage, legality and route quality diverge

SimWorld evaluates six vision-language models. Its metrics deliberately separate feasibility, the fraction of tasks completed, and travel efficiency. Valid deliveries can receive credit even when other actions violate constraints, so high completion does not establish that an entire plan is legal.

Qwen3-VL-8B illustrates the opposite problem: its text-only plans contain no violations in the aggregate diagnostics, but complete only 54.4% of tasks on average. Only 32.3% of those runs finish every task without violations. With multimodal input, its complete-and-legal success rate rises to 94.2%.

Other models show different patterns. Qwen3.7-Plus completes every task without violations in all evaluated multimodal runs. Claude-Opus-5, however, has a lower complete-and-legal success rate with multimodal input than with text alone. Adding an image therefore does not reliably improve every model.

Across all six models, multimodal input produces the highest aggregate completion and complete-and-legal success rates, while text-only input produces the fewest violations. Route costs generally rise relative to the optimal route for the delivered task subset as task counts increase.

Diagnostic evidence and limits

DualWorld-VQA contains 1,665 questions testing visual geometry, navigable topology and constrained route planning. GPT-5.4 Mini scores 55.9% on topology questions but 6.3% on planning. Planning success requires a feasible sequence within a relative distance tolerance of the reference, so failure can reflect excessive route length, not just illegality. The appendix distinguishes wrong task-node selection from feasible but overly long routes.

The benchmark’s scope remains important. Temporal complexity and grounded topology are tested separately, not combined within shared instances. TextWorld’s reported breakdowns use one seed, and the paper reports no error bars, confidence intervals or significance tests. The results characterize performance under these evaluation interfaces; they do not establish real-world delivery reliability.