Forethought gives AI tool use a checklist—and checks the steps

Turning reasoning into a program

Many AI agents do more than answer questions: they select tools, fill in arguments and use one result to decide what to do next. Forethought’s authors argue that this work can be difficult to inspect when buried in a language-model reasoning trace. Their alternative is to represent tool use as a program whose steps can be checked and recorded.

The system combines symbolic components with neural ones. Symbolic components follow explicit rules, such as checking a data format or ordering tasks according to dependencies. Fine-tuned small language models handle operations the authors describe as harder to formalize, such as breaking a request into tasks or matching a task to a tool. A typed programming language connects the components, and each operation has an output contract: conditions its result must satisfy.

For example, the workflow checks that required parameters are present before issuing a tool call. It also keeps a scratchpad for information carried across turns, checks prerequisites and records a trace of operations and contract checks. When a check fails, the engine can retry, use a fallback or report an error. In the tool-calling kernel, an unmet prerequisite or missing function can also lead it to skip that task and continue to the next one.

What the checks can—and cannot—guarantee

The distinction between symbolic and neural steps matters. Symbolic operations can guarantee specified structural properties: for example, the system will not issue a function call with missing required parameters. Neural operations are different. A model can produce an answer that satisfies its contract and still be wrong in meaning. The authors say neural steps were empirically validated, not formally proved correct.

That boundary is central to the paper’s use of “verification.” Contracts check conditions designers have specified; they do not establish that every tool choice, interpretation or final answer is correct. They apply to the reasoning trace, not necessarily to the base model’s final response, and the authors did not measure how often that response departs from the trace.

The system’s development and validation also relied on authored examples. The team manually worked through 300 multi-turn scenarios to refine the workflow and document its steps. Training examples for its five small-model primitives were derived from those solutions and expanded with synthetic variations, making 16,502 examples in total. The paper reports that fine-tuning raised average exact-match accuracy on held-out synthetic tests from 76.4% to 92.5%. These results measure the primitives on structured outputs; they do not prove that all contract-satisfying judgments are semantically correct.

Benchmark results

The researchers evaluated Forethought on four tool-calling benchmark suites: τ²-Bench, BFCL v3 and v4 multi-turn tests, and LiveMCPBench. Across five matched comparisons—each comparing Forethought with direct prompting for the same base model—the authors report higher aggregate averages with Forethought. The median relative improvement across those pairs was 32%.

The reported gains tended to be larger for weaker base models. Systems using Forethought with small responder models reached aggregate scores near a frontier-model baseline. The best aggregate result exceeded that baseline only by a margin the authors say is within single-run noise. Because most configurations were tested once, repeated runs with uncertainty estimates are needed to tell whether small differences are meaningful.

An ablation test helps clarify the role of the checks. When the researchers removed the contract-and-control layer but kept the same primitives and program structure, performance fell in both tested configurations. For GPT-OSS-120B, the reported average dropped from 77.7 with the full system to 39.1 without the layer, below the 59.3 score for direct prompting. This suggests that, in these tests, splitting work into smaller operations was not enough on its own. The ablation removed the layer as a bundle, however, so it does not identify which individual checks or recovery mechanisms mattered most.

The paper also reports that a DeepSeek-V3 pipeline with Forethought outperformed its reasoning-model counterpart, DeepSeek-R1, by 34 percentage points on the reported aggregate. This is not a controlled test of where reasoning should happen: the models differ in more than reasoning style, and Forethought adds five fine-tuned 12-billion-parameter models. The result describes these systems on these tasks, not a general comparison of reasoning approaches.

How certain are the results?

The results are promising, but several limitations matter. The authors report a human review of BFCL v4 cases that changed scores. They say the corrected labels have not been independently audited and do not claim that the annotators were independent of system development. The paper treats the original automated scores as primary.

The main comparison is also not compute-matched: Forethought includes fine-tuned models that direct prompting does not. The paper does not report per-task token use, latency or inference cost, so it cannot establish whether the approach is cheaper in practice. The authors also have not shown how readily it transfers to unfamiliar tool registries or other reasoning tasks.

This work is an arXiv preprint. Its evidence supports a narrower conclusion: on the tested tool-calling benchmarks, a contract-checked workflow improved aggregate accuracy over direct prompting of the same base models in five matched comparisons. Whether those gains generalize—and how often the checks prevent consequential errors outside benchmarks—remains open.