CPU-ARC turns language-model solutions into reusable programs

A language model asked to count a number’s divisors can explain prime factorization and calculate the answer. Ask about another number, and it may generate much the same explanation again. CPU-ARC asks whether that reusable procedure can instead become a program, leaving a smaller model to interpret the next request.

In a preprint, Peter Belcak, Greg Heinrich, Saurav Muralidharan, Shizhe Diao and Pavlo Molchanov introduce “agentic reasoning caching.” Rather than saving an answer for later retrieval, their system stores executable components intended to solve a family of problems using the same underlying reasoning.

Turning a solution into code

The approach separates three roles. A source model solves the original problem. A builder model analyzes its reasoning transcript and constructs reusable code. A smaller target model handles language-processing tasks during subsequent use, such as extracting numbers or identifying units.

Construction has five phases: isolate the input and expected output, decompose the reasoning, identify variable parameters and their valid ranges, plan the program, and implement it. The builder tests and revises candidate functions using execution feedback and language-model judgments. End-to-end checks compare results with both the original transcript and the benchmark’s reference answer.

The resulting Python components, called “semantic functions,” describe their inputs and operations in natural language. Instead of needing an exact function name, one component can request another by describing what it should do. An embedding model and a CPU-based similarity index support these lookups.

For an incoming question, the system searches for a suitable cached procedure, extracts its parameters and executes it. Unmatched requests can fall back to the source model. This is therefore a proposed complement to large-model inference, not a universal replacement.

What remains for the small model

A worked example calculates gas pressure using the ideal gas law. Code performs the arithmetic and pressure-unit conversions, while model calls extract numerical values and classify the temperature unit. The program also records a readable execution narration.

A stable cache can be compiled into a standalone Python module, replacing internal similarity searches with direct function calls and dictionary lookups. Compilation does not eliminate every model dependency: extraction and other language-processing calls remain where required. Changes to the compiled function network require recompilation.

For the main setup, Claude Opus 4.5 served as both source and builder. The researchers initially used Claude Haiku 4.5 as the target, then fine-tuned Llama 3.2 1B. That training used roughly 20% of collected benchmark invocation logs, augmented with synthetic examples generated by the source model.

Accuracy and reuse

The results table covers eight benchmark settings across six families, including competition mathematics, graduate-level science questions and textbook science problems. The authors report cache accuracy matching or exceeding source accuracy in seven settings.

On MATH-500, reported accuracy was 98.8% for the cache versus 98.6% for the source. On AIME 2025, it was 93.3% versus 83.3%. GSM8K was the exception: the cache scored 98.0%, compared with 98.5% for the source. The authors attribute some improvements to code avoiding computational slips in otherwise useful source reasoning.

To test reuse, they generated questions with changed wording, changed parameters, or both. Reported pass rates ranged from 90.8% to 99.3%. Crucially, these tests included only entries that already answered the original question correctly. The variants were generated and screened by language models; changed-parameter answers were evaluated against model-generated solutions rather than independently supplied benchmark labels.

Lower invocation costs, upfront investment

Using March 2026 API prices and assigned token rates for the fine-tuned target model, the authors estimate invocation costs 16–291 times lower than the source model’s original solve. These are cost ratios, not measured speedups.

Building a cache entry cost approximately 12–84 times one source-model solve. The paper estimates that repeated use recovers this investment after about 13 invocations for AIME procedures and about 90 for GSM8K. Those savings depend on enough applicable requests arriving; the accounting is based on model-call costs, not a full measured deployment budget.

What the evidence does not establish

The benchmark questions themselves supplied material for cache construction, and reference answers participated in validation. The results therefore demonstrate reconstruction and reuse around known problems, not unrestricted performance on unseen tasks. The description also does not establish a benchmark-disjoint evaluation of the fine-tuned target.

Generation and judging relied on the same model family, leaving potential shared errors. Inspectable code and readable narrations aid review, but do not automatically prove correct retrieval, extraction or execution. Production reliability, hardware-level efficiency and broad edge deployment remain questions for further testing. The supported promise is narrower: repeatedly used solution procedures may be cheaper to execute than to regenerate.