Beyond Chat History: A Preprint Connects AI Assistants to Persistent Data Engines

An assistant may remember that an analyst requested a forecast without retaining the trained model needed to revise it. In the preprint “AI Data Analysis: A New Approach — Multi-engine Joint State Data Analysis,” Jean Ye proposes connecting conversational reasoning to persistent computational environments. The division of labor is explicit: the language model interprets requests, coordinates tools and explains results; specialized engines execute queries and calculations.

The proposal combines Python and R kernels, SQL access to relational databases, and Cypher access to graph databases within one language-model conversation. Ye’s central distinction is between merely offering several tools and actually connecting their working contexts. Outputs must move between engines, and accumulated guidance must be reusable across them.

Keeping the working state, not just the conversation

In the proposed architecture, Python and R retain data tables, variables and models between requests. An analyst can ask for another breakdown or a different modeling algorithm without rebuilding the entire working environment. The paper describes cross-kernel transfers through shared CSV files and recovery through on-disk intermediate results.

Reusable guidance provides a second connection. A schema file describes database structure, while a Markdown notes file records lessons such as business-specific date conventions or misleading field names. Ye calls the combination of advance grounding and multi-round execution feedback M+S: the agent starts with reference material but can still execute, inspect feedback and revise its work.

This is not simply a promise to place every database inside an enormous prompt. Although the design emphasizes million-token context windows, the larger production example uses a table index and selective extraction of relevant schema fragments. The paper also distinguishes structural metadata from actual data values: inspecting records remains useful when the dictionary does not resolve an ambiguity.

The strongest evidence comes from SQL

The main controlled comparison uses 126 questions on a modified AdventureWorks database: 96 official questions and 30 self-authored ones. Answers are scored by result-set equivalence rather than matching query text, using a relaxed criterion that permits additional output columns when they accurately contain the reference result.

Across three runs, M+S achieved an average accuracy of 91.3%, with a reported bootstrap 95% confidence interval of 86.8–95.2%. Pure-exploration agents supplied with a data dictionary averaged 91.0%, while the dictionary-equipped single-shot configuration averaged 85.2%. The paper reports no significant accuracy difference between M+S and pure exploration under its union-of-three-runs comparison; not every comparison with single-shot configurations reached statistical significance.

The practical advantage over exploration was reduced work. Reported averages fell from 10.6 interaction rounds to 4.8, and from roughly 450,000 tokens to 275,000 per question. That is a saving relative to exploration, not single-shot efficiency: the paper lists roughly 10,000–16,000 tokens for single-shot queries.

Repeatability also matters. M+S answered 85.7% of questions correctly in all three runs, compared with 95.2% answered correctly at least once. Those measures describe different experiences: a question that succeeds occasionally is not necessarily dependable on a user’s only attempt.

Four engines, different kinds of evidence

For graph queries, Ye reports 96.5% average execution accuracy across three runs on 401 questions from CypherBench’s movie graph, with 94.0% correct in every run. This tests one of the benchmark’s 11 graphs. Comparisons with published competing systems use different models and environments, rather than a matched local experiment.

The machine-learning demonstrations answer a different question: can conversational orchestration run substantial computational workflows? On TabReD weather data, the paper reports processing 16.95 million rows, with Python and R gradient-boosting implementations producing closely aligned test errors. It explicitly presents the default-parameter reproduction as distinct from a tuned leaderboard result.

Cross-engine reuse is demonstrated through workflows such as R cleaning, Python modeling and R visualization. Shared notes also support SQL, R and Python sessions on a production database described by a 370-table dictionary. That larger-schema demonstration is qualitative, without a benchmark accuracy score.

Reading the claims at their proper scale

Evaluation changes require care. Ye reports correcting reference-answer errors, and the improvement trajectories combine evolving notes, added evidence and reference revisions. They should not be read as controlled measurements of notes alone. The headline comparison with a published 74.45% SQL result also spans different systems and database coverage, not a strict like-for-like contest.

The paper acknowledges cold starts on unfamiliar databases, model-dependent notes and remaining output variability. Its “deterministic” label describes the engineering method, not identical answers. Governance features are described as implemented but lie outside the empirical evaluation; a relational–graph demonstration establishes connectivity, not effective downstream action.

The proposal’s useful focus is therefore the working context around the model: persistent computation, transferable intermediate results and explicit accumulated knowledge. The reported evidence supports investigating that combination without treating every architectural claim as equally validated.