Why a plausible repair can still fail: teaching AI to respect assembly dependencies

Repair is more than finding the faulty part

Imagine a piece of an interlocking structure is missing or turned the wrong way. If another piece blocks access, the repair may require temporarily moving it, fixing the problem, and restoring the displaced part. A system that identifies the fault but skips one of those steps has not completed the repair. Even individually reasonable actions can be impossible in sequence—or leave the assembly in the wrong final state.

Fuqiang Wang and colleagues call this challenge fault-conditioned executable rework generation. Their paper introduces FORGE, a method for generating repair programs, and FORGE-Bench, a benchmark for evaluating them. The question is whether a model can diagnose a faulty assembly and produce a sequence of actions that is legal at each step and ends in the specified target configuration.

From recovery dependencies to executable actions

FORGE-Bench contains 4,000 tasks involving 26 interlocking assemblies. The authors divide the data into 3,000 training tasks and 1,000 test tasks, with the split made by assembly: 21 assemblies are used for training and five are held out to test generalization to unseen structures. The benchmark includes no-fault, single-fault, and multi-fault cases. Faults include missing or misplaced parts, incorrect orientations, and problems involving blocking dependencies.

Each task provides structured information about the observed and target states, assembly relationships, constraints, available actions, and parts. The model must decide whether rework is needed, diagnose any fault, and generate a plan. If the assembly is already correct, the appropriate plan is empty, avoiding unnecessary intervention.

FORGE separates the task into two linked functions. Rework Structuring identifies the parts and dependencies relevant to recovery—for example, what must be moved to reach a fault and later restored. Action Grounding turns that recovery structure into actions allowed in the current symbolic state. The model outputs the diagnosis and plan, and a symbolic transition system replays the actions to check their legality and whether they restore the target.

That replay makes the evaluation stricter than judging whether a plan sounds plausible. A successful result must follow the benchmark’s action rules and reach the specified final state.

What the authors report

The authors compare FORGE with several language models and code-generation methods. FORGE receives an Overall-Core score of 72.75, the highest among the listed methods. This aggregate is the unweighted average of six automatic metrics covering aspects such as action-sequence agreement, parameter fidelity, verified recovery, and performance on hard cases. A separate score from an LLM judge is reported but is not included in the aggregate.

A component-removal analysis helps clarify the proposed design. Vanilla supervised fine-tuning scores 67.53 on Overall-Core, compared with 72.75 for full FORGE. Removing Action Grounding lowers the score to 61.35; removing Rework Structuring yields 67.89. The effects vary across metrics: the version without Rework Structuring has the highest score on one parameter-fidelity measure. So the results support using the two components together under this benchmark, not a claim that each component improves every measured ability.

What the results do—and do not—show

The evaluation uses symbolic assembly states, a finite set of Luban lock assemblies, and discrete actions. The authors say that extending the approach to richer assembly types, uncertain perception, continuous physical execution, or interactive recovery would require further work. Passing a symbolic replay shows that a program follows the benchmark’s rules; it does not show that a machine could perform the actions reliably in a workshop.

The paper is labeled a preprint, so its findings should be treated as preliminary rather than settled evidence. The reported scores describe performance on the authors’ benchmark and metrics; they do not establish effectiveness on real-world repairs or structures outside that test set. The narrower takeaway is that, in these simulated tasks, connecting recovery-dependency reasoning with state-aware action generation was associated with better verified restoration.