Training a web agent’s planner and actor together improves benchmark success

Completing a web task involves more than recognizing a button. An agent asked to cancel an order must navigate to the right page, identify the order and carry out the cancellation without getting sidetracked. A single model can handle both planning and clicking, but must learn these different responsibilities together.

In the JHiMA-WebRL preprint, Shanka Subhra Mondal and Jiayun Wang of Accenture’s Center for Advanced AI investigate a different arrangement: two separately parameterized models that learn complementary roles. Their results support this approach within one web benchmark, while leaving broader generalization unresolved.

Two roles, one outcome

The system contains a task decomposer and an actor, each initialized from Qwen2.5-VL-7B. Together, they have approximately 14 billion parameters. The decomposer receives the instruction and starting screenshot and produces an ordered list of subtasks. The actor uses that list, the current screenshot and its action history to choose interface actions.

The planner does not continuously supervise execution. It generates its plan once, at the beginning, and cannot revise it when the website behaves unexpectedly. The actor must work with that initial planning context throughout the attempt.

To teach the interface between these roles, the researchers used GPT-5.2-Codex to annotate existing ScaleCUA demonstrations. The teacher examined instructions, screenshots and action sequences and generated high-level subtasks. This produced 76,396 decomposition examples. Both models then received supervised fine-tuning, with the actor learning to predict demonstrated actions while conditioning on the annotated subtasks.

Learning which plans are executable

The next stage jointly adapts both models through reinforcement learning. For each training task, the decomposer proposes three plans. The actor makes seven complete attempts under each plan, producing 21 trajectories per task. Each attempt receives a binary reward for eventual task success rather than detailed rewards for individual steps.

The method extends Group Relative Policy Optimization, or GRPO, by making different comparisons for the two roles. Actor updates compare attempts made under the same plan. Decomposer updates compare plans according to their average downstream success. Both models are updated from the same collection of attempts.

This gives the planner feedback about which instructions its particular actor can execute. However, the feedback remains coarse: it does not identify precisely which subtask or action caused failure.

What the benchmark shows

The researchers evaluated on WebArena-Lite-v2, a benchmark containing 154 tasks across shopping, shopping administration, Reddit, GitLab and maps environments. Agents interacted visually, without HTML or document-object-model information, and had at most 15 interaction steps. Reported success rates average four evaluation runs.

JHiMA-WebRL achieved 37.2% success, compared with 24.0% for a ScaleCUA-32B model that the authors trained with reinforcement learning on the same training tasks and evaluated under the same protocol. That is a 13.2-percentage-point difference, although most tasks still failed.

The smaller parameter count should not be confused with a demonstrated training-cost advantage. JHiMA-WebRL used 21 trajectories per training task, whereas the 32-billion-parameter baseline used seven. The hierarchical reinforcement-learning stage took approximately 720 GPU-hours on eight H100 GPUs.

Ablations provide a narrower test of the proposed coordination mechanism. Keeping the decomposer frozen while training the actor reduced success to 33.4%. Actor-only systems reached 34.9% with 21 training trajectories and 35.2% with seven. The supervised-only two-model system scored 35.1%. Joint training therefore improved on these alternatives, but the authors caution that differences of 2.0–3.8 percentage points are small enough for run-to-run variation to matter.

Where the evidence stops

The benchmark does not establish universal superiority over larger agents. Some comparison scores came from earlier studies with different answer-matching judges; ScaleCUA-32B, for example, had previously reported 44.2% under its original setup. JHiMA-WebRL also performed poorly on maps, scoring 12.5%, with one illustrated failure repeatedly toggling zoom controls until its step budget ran out.

Training and evaluation tasks were disjoint, but shared website environments and could share instruction templates. The results therefore concern generalization within those environments, not transfer to unseen websites. Teacher-generated planning annotations were not human-verified, and the study evaluated only one benchmark.

The preprint’s contribution is consequently specific: jointly learning planning and execution improved this modular system’s measured success. Whether that benefit survives unfamiliar websites, changing interfaces and live-web conditions remains untested.