AI Can Write SQL. Making It Trustworthy Takes More Than a Bigger Model

A question such as “How many orders were placed in 2023?” can become a short database query. More complicated requests may require combining several tables, applying nested calculations and interpreting terms whose meaning depends on the business. Producing valid SQL—the language used to query relational databases—is only part of the job. The query must also reflect the question’s intent.

In Reimagining Text-to-SQL, Shikhar Dave, Bhaskarjit Sarmah and Stefano Pasquali review more than 100 papers and technical resources, focusing on work from 2022 to 2026. Their preprint is a synthesis of existing research and deployment practices, rather than a report of a newly tested system. Its central argument is that progress depends on how language models connect to databases, tools and verified knowledge, not simply on increasing model size.

Five routes from questions to queries

The authors organize recent methods into five overlapping categories. In-context learning supplies examples and instructions without changing a model’s weights. Fine-tuning adapts a model using task-specific training data; parameter-efficient versions update only a small subset of its weights.

Agentic systems divide the work among components that retrieve database information, plan queries, execute candidates and repair mistakes. Retrieval-augmented generation brings relevant examples, documentation or schema details into the model’s context. Reinforcement learning trains models using rewards, often based on whether their generated queries return the expected results.

These are not mutually exclusive approaches. A system can retrieve examples, use a model trained with execution-based rewards, then test and refine several candidate queries. The survey describes a shift from one-shot translation toward such hybrid workflows.

A recurring difficulty is schema linking: connecting words in a question to the correct tables and columns. Another is value grounding, which means resolving references such as names, dates or product codes to actual database values. Mistakes at either stage can produce SQL that runs successfully but answers the wrong question.

Strong scores do not settle reliability

The survey reports that Agentar-Scale-SQL achieved 81.67% execution accuracy on the BIRD test set. That system combines reinforcement-learning-enhanced generation, retrieval, iterative repairs and candidate selection. Its results illustrate the potential of spending additional computation at query time, while also making cost an explicit trade-off.

More realistic workflows remain difficult. The survey reports approximately 35.8% execution accuracy for ReFoRCE on Spider 2.0’s Snow split. Spider 2.0 includes enterprise-scale schemas and multi-stage analytical tasks. These figures come from different benchmarks and configurations; they are not a controlled comparison between the two systems.

Even the definition of success needs care. Exact matching can penalize differently written queries that mean the same thing. Execution accuracy instead checks whether a generated query returns the same result as a reference query, but incorrect queries can sometimes match by coincidence. Testing across multiple database instances helps expose those cases, at additional computational cost.

Benchmark quality is another concern. The survey cites audits finding substantial annotation errors in examined datasets. It also describes ReViSQL experiments in which replacing original training examples with expert-verified data improved single-generation accuracy by 8.2–13.9 percentage points while holding the reinforcement-learning algorithm fixed. That finding, as reported in the survey, suggests that cleaner data can matter as much as architectural changes.

Teaching systems what business terms mean

A major deployment theme is the semantic layer: an explicit mapping between business concepts and physical database structures. Instead of asking a model to infer what “revenue” or “active users” means, an organization can define those metrics centrally, alongside approved relationships between tables and recognized synonyms.

The survey examines knowledge-graph-based approaches and Wren AI’s declarative modeling language as examples. The authors argue that these layers can reduce ambiguity, make calculations more consistent and isolate business definitions from changes to underlying tables. They are architectural supports, not evidence that every generated answer will be correct.

Knowing when not to answer

The authors also highlight systems that ask clarifying questions or abstain when uncertain. That matters when a plausible but wrong answer could influence financial or healthcare decisions.

Their deployment recommendations include restricted database permissions, read-only execution, sandboxing, query validation and audit logs. They also discuss prompt injection through database content and the exposure of sensitive information.

The survey’s conclusion is practical: natural-language database access is a full-stack engineering problem. Accuracy must be assessed alongside robustness, latency, cost and safety. A fluent interface is useful, but trustworthy answers still require grounded definitions, careful testing and opportunities for human intervention.