LEAP predicts which training hardware offers better value before jobs launch

Choosing before running

A training job can finish successfully while running on hardware that offers poor value. In a fleet containing different accelerator vendors and generations, the useful question is not just which device is fastest, but which delivers the most training work for its computational cost.

In the arXiv preprint Look Before You Launch: Learned Throughput Prediction for Hardware Selection in a Production Training Fleet, Hossein Askari and colleagues describe LEAP, a system developed at Meta to make that judgment before a job starts. The manuscript reports production experience; its findings should be read as the authors’ reported results, not independently established performance guarantees.

The fleet averages about 25,000 training attempts daily across roughly 660 model types and more than 20 hardware types. LEAP handles prediction and ranking, while other systems determine hardware eligibility and make the final placement.

Learning from previous attempts

LEAP estimates throughput: the number of training samples processed per second. Its inputs include trainer count, batch size, model-architecture summaries, hardware specifications and performance recorded during earlier runs of the same job family. It does not require a profiling run of the current attempt before recommending hardware.

The predictor uses gradient-boosted decision trees, which combine many decision trees into a prediction model. Because throughput varies enormously between jobs, LEAP predicts its logarithm and divides the problem among 13 specialist regressors. A classifier chooses the specialist for each prediction. The training objective targets median throughput rather than average throughput.

Each night, the pipeline trains on approximately 750,000 attempts from the previous 30 days. A snapshot of job-history statistics travels with the model, avoiding a separate history-service lookup during launch. Previously unseen job families receive an explicit missing-history value.

For each candidate hardware type, LEAP divides predicted throughput by the job’s computational cost on that type. It also filters premium hardware unless predicted utilization justifies it. The resulting list expresses preferences, not instructions the scheduler must obey. If prediction fails, an empty list lets scheduling proceed without LEAP’s advice.

What the accuracy figures mean

Across 30 consecutive nightly retraining runs, the authors report a median accuracy of 77.0%, ranging from 73.5% to 80.6%. Here, “accuracy” means the share of held-out job attempts whose predicted throughput falls within ±20% of the measured value. Each evaluation day follows its training window.

That figure is not the percentage of jobs assigned to the best hardware. It measures throughput estimates on observed job attempts, not whether every alternative hardware ranking was correct. It also weights attempts equally, rather than by GPU-hours, so frequent small jobs contribute more than rare, resource-intensive jobs.

A separate comparison examined 7,131 attempts on one held-out day. The previous lookup table supplied estimates for only 946, or 13.3%. On those same 946 attempts, LEAP achieved 85.1% within the ±20% tolerance, versus 53.2% for the table. LEAP supplied estimates for the entire comparison set. The 85.1% result therefore describes a narrower subset than the headline 77.0% figure.

Coverage is not placement or savings

During the first half of 2026, the reported share of eligible training GPU-hours receiving a LEAP ranking increased from 58% to 95%; the paper associates that coverage with 77 million GPU-hours. Receiving a ranking does not mean running on its first choice.

In a separate six-day August window, jobs placed on LEAP’s top-ranked hardware represented 19.9% of attempts and 30.0% of GPU-hours. Available capacity can prevent the scheduler from following the recommendation.

The authors estimate that LEAP reduced the served fleet’s total computational training infrastructure cost by about 3% during the first half of 2026. This is a fleet-specific estimate, not a demonstrated saving for every job or another organization’s hardware mix.

Where the approach struggles

The tested language models, text encoders and tabular alternatives did not outperform the deployed tree-based cascade on matched evaluations. A graph-based neural predictor matched its accuracy but required GPU serving and a computation-graph cache of about 400 GB. These comparisons support the authors’ deployment choice for this workload, not a universal ranking of learning methods.

History dependence creates important boundaries. Hardware specifications alone did not enable reliable predictions for unseen hardware types. New model families also caused large accuracy losses; one single-family experiment reduced the cascade’s accuracy by 49.7 percentage points.

The paper additionally identifies serving-time feature consistency as unfinished work: some parallelism and scaling inputs should either reach the serving path or be removed from the model. Proposed improvements include representative trial jobs for new hardware and selectively using graph-based prediction for unfamiliar models. These remain future directions, not demonstrated capabilities.