VCoT-SQL puts AI-generated SQL reasoning through a second check

Turning an everyday question into a database query requires more than producing valid SQL syntax. A system must identify the relevant tables, select appropriate columns and translate the question into the right conditions and calculations. In a preprint, Rui Zhang and colleagues at Huazhong University of Science and Technology present VCoT-SQL, a framework built around checking both training-time reasoning and inference-time SQL.

Their central argument is that a correct final answer is not enough to establish the quality of a synthetic training example. A model can produce the expected SQL while supplying unreliable intermediate explanations. Training another model on those explanations risks preserving the errors the explanations contain.

Checking the examples before teaching the model

VCoT-SQL trains two complementary models: a generator that writes SQL and a validator that diagnoses and corrects it. Both learn from structured, synthetic chain-of-thought examples, but their reasoning templates serve different purposes.

For generation, the template breaks the task into SQL clauses and smaller decisions about columns, conditions, aggregations and joins. The authors place reasoning about the FROM clause after the other clauses, so table selection follows identification of what the query needs.

The synthesis process supplies reference SQL and three handcrafted examples. It then filters generated material for structure and length, checks whether the resulting query returns the same results as the reference, and removes explanations that explicitly refer to the supplied “Gold SQL.” Rule-based checks also assess whether the reasoning references the relevant database elements.

Validator training starts with deliberately corrupted SQL. The authors introduce errors involving columns, tables, conditions, aggregations and data types, then generate diagnostic explanations and corrections. Beyond checking execution results, they extract the analysis and correction steps and ask a model to reconstruct the corrected SQL from those steps. The purpose is to test whether the explanation supports the correction rather than merely accompanying it.

After filtering, the generator dataset contains 5,477 Spider examples and 3,669 BIRD examples. Validator training uses another 947 examples derived from BIRD.

Narrowing the database context

Before generating the final query, VCoT-SQL also retrieves a smaller, relevant database schema—the description of its tables and columns.

One retrieval path drafts preliminary SQL and extracts database elements from it. Another matches the question and extracted keywords against column representations using semantic embeddings. A third searches database values to find relevant columns that the question may not name directly. The system combines the retrieved columns and presents them with supporting schema information.

The generator then writes an initial query. That query is executed to collect error feedback, and the validator checks syntax, meaning and data-type handling before returning a refined version. Execution feedback is therefore evidence for correction, not the sole basis for deciding whether a query answers the question.

What the results establish

The authors report 66.82% execution accuracy on the BIRD development set using a XiYanSQL-QwenCoder-7B backbone, compared with 59.78% for that backbone in their main comparison: a gain of 7.04 percentage points. Their system reaches 86.8% on Spider test.

The comparisons need their qualifiers. VCoT-SQL exceeds the reported BIRD results for GPT-4o and GPT-4-turbo under the paper’s evaluation setup. But OmniSQL scores higher on Spider test, at 87.9%, while using a much larger reported fine-tuning corpus. These results support a claim of competitive, data-efficient specialization, not universal superiority.

The ablations help separate the contributions. With Qwen2.5-Coder-7B, validated logic-order reasoning data produces 64.86% BIRD accuracy, versus 61.60% with unvalidated logic-order data. Adding the trained validator raises accuracy to 66.49%. The appendix specifies that corrections improve 1.83% of all examples while harmful changes affect 0.20%, yielding a net 1.63-point gain.

Simply adding an untrained validator is not reliably beneficial: a base DeepSeek-Coder validator reduces BIRD accuracy by 4.57 points. A separate comparison also finds that GPT-5.4 generally outperforms the trained 7B validator when given the same generator outputs.

Quality control, not a finished solution

The preprint’s strongest message is about supervision quality: smaller models can benefit substantially when synthetic explanations are structured and checked, rather than accepted because their final query looks correct.

The authors identify a remaining limitation in supervised fine-tuning itself. It trains models to imitate examples rather than directly optimize executable, accurate SQL. They propose reinforcement learning using execution results and reward signals as future work. For now, VCoT-SQL presents benchmark evidence for better-curated training and targeted correction—not a guarantee of correct database answers.