Estimating what delivery reliability is worth over time

How much should a delivery platform spend to make its service more reliable? Extra courier capacity costs money immediately, while the benefits of fewer delays may emerge slowly through repeat orders and customer retention. Measuring only the first few weeks could miss much of that response.

In “What is the Long-Term Value of Reliability?”, Chenyu Qiu, Xu Kuang, Inessa Liskovich, Ali Rauh and Stefan Wager describe Chronos LTV, a system built by Uber to estimate these longer-term effects from observational records. The November 6, 2026 arXiv revision is listed as forthcoming in the Harvard Data Science Review. Its public validation uses a simulated marketplace, not disclosed Uber customer results: the authors say confidentiality prevents them from publishing their real-data findings.

Following the consequences of a delay

Chronos represents customers as having changing, partly hidden states, such as satisfaction, preferences and willingness to order. A delivery experience can change that state, which then influences later purchases. The mathematical framework is a Markov decision process: the current state carries the information needed to describe subsequent evolution.

The target is a marginal policy effect. Rather than pricing one particular late order, Chronos asks how a very small change in delay probabilities would change average customer outcomes over time. Its specified intervention shifts delay probabilities equally across observed order characteristics. Applying this local response to a larger operational change is an approximation, not a guarantee for arbitrary policies.

Crucially, delays must be effectively random after conditioning on observed information. Two otherwise comparable orders may encounter different delivery outcomes, but unmeasured factors must not jointly determine delays and future customer behavior. The method also requires sufficient chances of both delayed and on-time delivery among comparable orders, and assumes past delays’ effects eventually fade.

Making the comparison fairer

Simply comparing customers after late and on-time orders can mislead. Delivery distance, tips, preparation time and ordering patterns can affect both delay exposure and the outcomes being compared.

Chronos reweights orders using their estimated delay probabilities. Its preferred implementation, covariate-balancing propensity scores, explicitly balances measured features between the weighted delayed and on-time groups. It then compares accumulated outcomes over a chosen forward-looking window. Uncertainty is estimated by resampling whole customers, preserving their repeated observations together.

The window matters: too short, and the estimate misses lingering effects. The authors suggest examining several horizons to see whether estimates stabilize. This still requires records extending far enough into customers’ subsequent behavior; it does not create long-term information from an immediately observed outcome.

What the simulated experiments found

The simulator includes customer differences, memories of delays, changing engagement states, courier supply fluctuations and queues. A delay means delivery arrives roughly 20 minutes or more beyond its quoted time. The authors explicitly warn that the simulator does not represent any real marketplace’s production systems or operating parameters.

In one test, a market initialized with 100,000 customers ran for 250 days before a 120-day randomized experiment. The intervention allowed dispatch up to five minutes before food preparation finished. Predictions used a precomputed short-term delay reduction of 4.15 percentage points.

Chronos with covariate balancing predicted long-term customer-value increases of 5.34% and 5.60%, using 56- and 70-day outcome horizons. The simulated experiment measured 5.82%, with a 95% confidence interval of 4.99% to 6.65%. An unadjusted comparison predicted 61.24% at the 56-day horizon; weighting with neural-network-estimated delay probabilities still predicted 17.04%.

The experimental response accumulated slowly, stabilizing around weeks 12–14. However, this customer-randomized comparison assessed direct effects, not the full consequences of increased demand feeding back into marketplace congestion.

A second test combined Chronos with a short market-wide switchback experiment, alternating baseline courier supply and a 25% increase. Across 200 regenerated evaluation simulations, the combined prediction averaged a 10.71% long-term value increase, against a simulated benchmark of 10.93%. The switchback’s direct customer-value reading averaged only 2.44%.

Where the evidence stops

Those 200 runs reused the observational history and fitted Chronos coefficient. Their 92% coverage for nominal 95% confidence intervals therefore does not establish performance under complete regeneration of the entire study.

The supply calculation also assumes delay reductions are uniformly spread across customer states. Different effects across states, or additional marketplace spillovers, may require other adjustments.

An additional simulation exposes the central causal risk: hidden kitchen problems can both delay food and independently discourage future orders through poor quality. When that independent demand effect persisted for 14 days, Chronos overpredicted the experimental benefit by about 29%.

The findings support the method’s promise under the tested conditions, not a universal price for reliability or proof of profitability. This report uses the supplied PDF text extraction, which may omit figures and damage mathematical or table layout.