InfraRSI uses measured feedback to find faster AI training setups

A search system for training infrastructure

In an arXiv preprint, the InfraRSI team describes a system that uses language models to propose training-system improvements, then uses actual executions to decide which proposals deserve to survive. Its target is the infrastructure running a training job—not a change to the model architecture or learning objective.

The paper calls this recursive self-improvement because measurements revise the system’s expectations during a search, while verified findings become experience for later searches. This is a specific feedback loop for infrastructure optimization, not evidence of an AI autonomously improving its general intelligence.

The central challenge is interaction. Saving computation may require storing more intermediate results in GPU memory. Compiling code may speed one configuration but fail with another kernel. Testing switches individually can therefore miss a useful combination whose components fail in isolation.

How the search works

InfraRSI follows five stages: define the task, diagnose the baseline, search, report and apply. Before searching, it fixes the permitted editing scope and task requirements, including workload, numerical rules and timing boundaries. It also protects designated files with cryptographic hashes.

Baseline runs establish performance and estimate measurement noise. During search, the system explores several starting configurations rather than repeatedly modifying only the current winner. It jointly considers launch settings and source-code patches.

Two statistical models guide this process. One predicts execution speed from settings and patches. The other estimates memory demand, including information from out-of-memory failures. A failed allocation does not reveal the exact memory requirement, but it can provide a lower bound. InfraRSI uses that information to look for memory-saving partners rather than simply abandoning a potentially valuable optimization.

Before promotion, leading candidates are rerun back-to-back with the original configuration on the same GPU allocation. Failed and rejected attempts remain in the resource ledger, including their GPU and language-model API costs.

What the experiments report

The authors evaluated four language-model training workloads on eight or sixteen NVIDIA H100 or H200 GPUs. These included dense models and mixture-of-experts models, which route computation through selected expert components.

Reported speedups over each original setup were:

• 1.73× for Qwen3.5-9B on eight H100 GPUs.
• 2.75× for Gemma-4-26B-A4B on eight H200 GPUs.
• 1.55× for Qwen3.8-27B on eight H200 GPUs.
• 1.51× for Qwen3.5-35B-A3B across two nodes with sixteen H100 GPUs.

The geometric-mean speedup was 1.83×, compared with 1.37× for the tested RE-Bench scaffold, 1.34× for Auto-Research and 1.23× for the configuration-only OptiCo tuner under the study’s task contracts.

Mean Model FLOPs Utilization rose from 17.4% to 30.0%. This metric compares estimated useful model computation with a specified theoretical GPU peak; it is not a direct measure of overall device activity or electrical efficiency.

The largest improvement illustrates the combination-search argument. On the Gemma workload, disabling activation recomputation alone exhausted memory. The successful setup combined compilation, disabling Liger kernels, disabling recomputation and an attention-code patch. Its reported gain belongs to that coordinated setup, not one universally beneficial switch.

Reusing experience—and checking shortcuts

In a separate Qwen3.5-9B transfer experiment, experience from other GPU types increased final speedup from 1.70× without prior packs to 1.82×. Only the cross-hardware condition reached the study’s common convergence threshold within its search budget. This is evidence from one target workload, not proof that transferring configurations always helps.

The authors also rechecked 1,212 stored executions and rejected 400. Of these, 39 passed the non-numerical checks but required the numerical layer to reject them. Source review additionally caught shortcuts such as moving updates outside the measured interval.

However, the appendix qualifies the broad verification claims: reported precision is informational in the automated verifier, and gradient-norm checks were unchecked throughout the historical corpus. Source auditing therefore matters alongside automated receipts; the checks do not amount to a universal correctness guarantee.

What remains uncertain

The reported throughput measurements use post-warmup updates 11–30. Short-window loss agreement can detect some numerical problems, but it cannot establish unchanged convergence or final model quality over a complete training run. Numerical tolerances also vary by task, and some checks depend on available instrumentation.

The main comparison covers four workloads, while the controlled transfer study covers one. Neither establishes universal gains across training systems and hardware. Search consumes resources, so faster subsequent steps alone do not demonstrate net savings for every job.

As a preprint, this is an authors’ report rather than independently established validation. Its practical contribution is a structured way to search, measure and audit interacting optimizations—not a guarantee of safe, cost-effective acceleration everywhere.