Can an AI Agent Build Useful Equations From Real-World Data?

From measurements to models

System identification means inferring a mathematical description of a system from observations of its inputs and outputs. Measurements of a robot’s commands and movement, for example, can help determine an equation for how it moves. But an analyst must choose more than parameter values: they must also decide what form the equation should take, how to fit it, and how to judge whether it suits its intended use.

Erik Derner, Jiří Kubalík and Robert Babuška present ASPIRE, a framework in which a large language model agent chooses and carries out those steps. The agent can inspect data, write and run code, and propose and compare different model structures. It may also use information such as a system description, intended application or physical requirements. The requested output is a closed-form model: an explicit mathematical expression, rather than only a black-box predictor.

A separate software “harness” runs the agent’s commands, checks submitted models, records the work and calculates errors. Two kinds of checking are important to distinguish. The harness independently checks formal physical requirements, such as specified relationships or behavior over a stated range. Separately, it evaluates prediction error on sealed test data that the agent cannot access. Some requirements were supplied in ordinary language rather than formally and were assessed separately by the authors.

Three measured systems, repeated runs

The study tested a mobile robot, a rotating apparatus affected by dry friction, and pressure changes in a laboratory fermenter. Each system was presented in two versions: with a description and relevant prior knowledge, or with the same data but without those details. The researchers ran each of the six tasks five times, for 30 runs, using one language-model configuration. They also compared the results with two equation-finding tools, PySR and PySINDy, under specified settings.

Every run produced a pool of models that passed the framework’s validity checks, and all 126 candidate models were eligible for evaluation. That is encouraging evidence that the workflow could complete these tasks under the tested conditions. It does not mean every model was physically proven correct: for some formal checks, the harness found no violation but could not establish a proof, so the outcome remained unresolved.

Predictive results varied by system and comparison. The agent’s recommended models had lower test errors than the tested PySINDy and PySR results for the mobile robot and fermenter. For the friction system, PySR in the matched setting had a lower mean test error than the agent’s recommendations. Yet all 12 comparison-tool models for that described task violated one specified physical requirement, while none of the agent’s candidates had a demonstrated violation. For some agent candidates, however, checks were unresolved rather than proven. Prediction error and physical requirements are distinct measures; neither alone tells the whole story.

Providing a description did not improve average test error in these experiments. It did influence model form: described tasks more often yielded physically named structures, and the described fermenter models addressed the stated physical requirements. The authors caution that each condition had only five runs and that the test data were limited. These observations do not show that descriptions generally worsen performance or that blind modeling is preferable.

A record is not the same as an audit

ASPIRE is designed to document equations, parameter values, fitting procedures and interpretations. The authors assessed five aspects of the recommended model records across all 30 runs. Of 150 assessment items, 132 fully met the criteria and 18 partly met them. The most frequent issue was that nine records made inaccurate or insufficiently supported claims about how the model structure had been selected. Execution logs let the researchers compare those accounts with what the agent actually did.

The study thus shows both the promise and the remaining work in making agent-built models inspectable. A detailed report is useful, but its claims still need checking. In one interactive example, user feedback changed the candidate pool and led to simpler expressions, while the recommended model’s test error stayed the same. This single case study does not establish how different users would fare.

Several limitations narrow the conclusions. The experiments used one language model, whose provider had not published a dated snapshot, and only three measured systems. The framework assumes a single scored output, measured or directly constructible states, and a supplied measurement equation; it does not cover every system-identification problem. The authors also note that test data were sealed from the agent, but not from the researchers during framework and task development. ASPIRE is described in an arXiv preprint, which may not have undergone peer review. The results support a cautious claim: the framework completed a small set of real-data modeling tasks with useful checks and records, but broader testing is needed to establish how reliably it generalizes.