AI agents organize thermoelectric research records—and reveal why traceability matters

A team of agents, not a single chatbot

In their preprint, Yusuke Hashimoto and Takaaki Tomai describe Multi-Agentic AI, or MAA, a system designed to analyze experimental thermoelectric materials data repeatedly while preserving records that later analyses can consult. The study examines what accumulates in those records, rather than demonstrating the discovery of a new material.

The data came from Starrydata2, which compiles experimental measurements extracted from published papers. The snapshot used for the largest experiment contained more than four million measurement points across all its covered fields, not exclusively thermoelectrics. Analyzing such data involves selecting measurements, constructing numerical descriptors, training models and interpreting results.

MAA divides that work among nine agents. A President selects themes, a Boss coordinates tasks, and six Workers handle data preparation, physical checks, machine learning, implementation, review and documentation. A separate Monitor watches for stalled execution and initiates recovery. Here, “autonomous” means operating under preset instructions and monitoring—not conducting research without human-designed constraints.

Numerical processing is delegated to code and machine-learning models. The agents save reports, metrics and structured knowledge records, creating an inspectable history rather than relying only on a chatbot’s final response.

What repeated analysis accumulated

Following a preliminary trial, the authors conducted four experiments varying the number of themes and repeated cycles. The largest comprised 100 analyses: ten themes repeated over ten cycles. Themes included particular material families and broader topics such as nanostructuring and practical evaluation.

To examine the records, the authors constructed a graph of normalized concepts. Connections represented either explicitly recorded relationships or concepts appearing together in multiple analyses. Knowledge bases from 98 analyses could be loaded for this graph, which contained 133 nodes and 528 connections.

Among 457 connections supported by at least two analyses, 236 had supporting analyses confined to one theme. That was 8.4 times the expectation under a model that selected supporting analyses independently of theme. The result indicates theme-specific clustering in the outputs; it does not establish that the connections are scientifically correct.

Most connections represented shared vocabulary without an explicit relationship description. Only 101 of those 457 connections carried such descriptions. Among tracked concept pairs initially lacking a description, roughly one in ten acquired one in a later cycle. The authors emphasize that later appearance alone proves neither that earlier records caused it nor that the relationship is valid.

More records did not mean better predictions

The prediction results did not show steady improvement with repetition. During the first seven cycles, when analyses targeted the absolute Seebeck coefficient, the ten-theme mean of reported R² remained around 0.65–0.67 initially, then fell to 0.56–0.60. R² is a measure of predictive fit, but the changing data and features complicate comparisons.

From cycle eight onward, some analyses switched to zT or transformed targets. The saved documents recorded these changes without establishing the reasoning behind choosing zT. Scores for different targets cannot be compared directly.

Retrospective checks also found inconsistent evaluation procedures, including cases where training information entered reported evaluations. Usable saved predictions allowed R² recalculation for only 19 of the 100 analyses. The reported scores therefore do not provide a uniform, independently verified measure of predictive progress.

Searchable answers, traceable mistakes

The authors also built a specialized assistant using 6,393 entries from all 100 analyses. It searches an index, then is instructed to read relevant reports, metrics and feature-importance files before answering with analysis identifiers.

In a demonstration concerning Mg3Sb2, the assistant identified compositions incorrectly grouped into that material family and excluded them based on their formulas. However, subsequent tracing found mistakes in prediction labels and source attribution. The candidate compositions were already measured materials, not demonstrated discoveries of unknown compositions. The five-question demonstration was not a benchmark, and its design guidelines remain physically unvalidated.

A particularly consequential check found unconverted Z values, measured in inverse kelvin, mixed into targets treated as dimensionless zT in six of twelve relevant analyses. These affected approximately 0.28–2.29% of model rows. Removing them and retraining changed reported out-of-fold R² by −0.0009 to +0.0254; some material-specific comparisons changed substantially.

An inspectable workflow, not guaranteed knowledge

The authors’ central lesson is that traceability makes errors findable, not absent. AI assistants also performed much of the retrospective checking, and some judgments awaited author adjudication. The exact experimental-data snapshot is not publicly archived, derived row-level tables are withheld pending permission, and analysis records and code are available on request. These constraints limit reproducibility. The study supports preserving evidence trails while leaving the scientific validity of accumulated findings—and whether iteration reliably deepens knowledge—as open questions.