AI can spot a simulator’s flaw and still choose the wrong design

Simulators let systems researchers compare designs without deploying every candidate on real hardware. They simplify how computers handle work, communication and shared resources, making experiments more tractable. But those simplifications create a practical question: does an inaccurate prediction actually change which design is best?

In the SIRIS preprint, Jinwoo Hwang and colleagues at KAIST and Microsoft Research Asia test whether large language models can answer that question. Their focus is AI-driven systems research, where models help propose designs, interpret simulation results and decide what to explore next. The work was submitted to arXiv; the supplied record does not report a peer-reviewed publication.

Testing decisions, not just fault-finding

SIRIS uses paired problems to separate recognizing a simulator limitation from judging its importance. Each pair shares the same candidate designs, deliberately introduced limitation and simulator recommendation. Only the workload—the requests the system must process—changes.

For one workload, an unmodified reference simulator agrees with the altered simulator’s recommendation. The researchers call this an “upheld” problem. For the other, the reference favors a different design: an “overturned” problem. The introduced limitation changes predicted performance in both cases.

This arrangement prevents a simple strategy from succeeding on both problems. Always trusting the recommendation fails when it is overturned. Always rejecting it fails when it is upheld. Correct choices require judging how the limitation affects competing designs under the particular workload.

The researchers constructed 58 problems in 29 pairs using a simulator based on LLMServingSim. Seven controlled changes altered computations involving such features as reuse of previously processed prompt content, network sharing and processing costs. These were experimental modifications, not reported defects in LLMServingSim.

The problems covered routing requests, choosing where to continue generating an answer after processing its prompt, and scheduling prompt processing alongside ongoing generation. Each offered four candidate designs. Performance measures included the 95th-percentile delay, which captures relatively slow requests rather than the average.

What the models could inspect

Thirteen models from the GPT, Grok and Gemini families tackled each problem three times under three evidence conditions: no simulator evidence, evidence from the altered simulator, or evidence from the reference simulator.

The agents could read files but could not execute or modify code. Simulator evidence included source code, cost parameters, configuration, predictions and execution records. Each conversation contained one problem, without revealing its paired counterpart or whether the recommendation should stand.

After an initial choice, the agent received a review request. In the simulator-evidence conditions, this asked it to reconsider its recommendation and identify any consequential modeling limitation. The no-simulator review omitted the follow-or-reconsider and limitation fields. No new evidence or correctness feedback was supplied between turns, although agents could read previously unopened files.

Recognizing a flaw was not enough

Without simulator evidence, models scored near the one-in-four chance level. With reference-simulator evidence, most performed nearly perfectly. Altered-simulator evidence exposed much larger differences: final average accuracy across upheld and overturned problems ranged from about 49% to 95%.

GPT-6 Astra and GPT-6.1 Sol performed best on this evaluation. Yet even these models sometimes identified the limitation and discarded a recommendation that remained correct. Across the evaluated responses, about a quarter of those identifying the limitation still chose incorrectly.

One example involved network transfers. The altered simulator gave transfers between GPU pairs independent links rather than making them share a GPU’s network interface. Correcting that assumption increased transfer delays. For one workload, the increase changed the best design; for another, the recommended design retained enough of an advantage to remain best. Merely noticing missing contention could not resolve both decisions.

Review also had mixed effects. Under altered-simulator evidence, it corrected 31 choices but introduced 10 errors for GPT-5.6 Sol. For GPT-5.6 Terra, 17 corrections were largely offset by 15 new errors. Reconsideration could change the tendency to trust recommendations without substantially improving accuracy.

What the results establish

The answers are defined by a reference simulator, not a perfect representation of real hardware. The study does not demonstrate that overriding invalid recommendations improves real-system performance. It also covers one serving domain, one simulator family and one introduced limitation per problem, so it does not establish a general model ranking.

The review experiment cannot separate reconsideration from extra computation, newly inspected files or prompt-induced doubt. Its findings concern this particular review procedure, not self-correction in general.

The central lesson is narrower but important: evaluating an AI research agent requires checking the design decisions that follow its critique, not simply whether it can name a flaw.