AI models know time’s rules but struggle to predict their own runtimes

How long will this take? For an AI assistant choosing whether to search the web, solve a problem directly or coordinate with another agent, that question matters alongside accuracy. A preprint by Ryan Liu and colleagues tests temporal competence through four experiments: understanding timing principles, estimating durations, judging written events and choosing whether a tool is worth the wait.

The results reveal an uneven profile rather than a universal inability to reason about time. Models generally recognized qualitative principles, but their estimates of agent runtimes were often far too short. Giving them measured completion times changed their decisions. The work is under review and has not completed peer review.

Knowing the rules

The researchers first built a simplified theory of task duration around skill, fatigue and the cost of switching between unrelated tasks. Its assumptions capture ideas such as practice improving performance and rest reducing fatigue. From these, they derived conditional predictions—for example, that batching unrelated work saves switching costs when fatigue and skill decay are negligible.

They translated the theory into short scenarios across eight work domains. Models answered questions about whether a change would make work longer, shorter or potentially either. Each model produced 6,720 responses across repeated trials.

Reported accuracy was 92.0% for Claude Opus 5, 97.5% for GPT-5.6 Sol and 96.3% for Gemini 3.7 Flash. Claude’s score excludes refusals affecting 14.2% of its responses. These are strong results within the researchers’ framework, not evidence that its simplifying assumptions describe every real workplace.

Predicting minutes is harder

For duration estimates, the team used 16 software and research tasks from HCAST with publicly available human and agent completion-time data. Models received task instructions and actor descriptions, then supplied estimates and uncertainty intervals, conditional on successful completion.

Human-duration estimates were moderately informative. Claude’s median estimate-to-reference ratio corresponded to 9% overestimation; GPT and Gemini underestimated by 13% and 67%. Their estimates also broadly distinguished shorter from longer human tasks.

Agent-runtime estimates were much worse. This experiment used different model versions because those had published runtime measurements: Claude Opus 4.6 and GPT-5.2 estimated themselves, while Gemini 3.1 Pro Preview estimated the unavailable Gemini 3 Pro—a proxy, not literal self-estimation.

Median underestimation was 37%, 99.5% and 97.6%, respectively. All three conditions also had negative rank correlations: tasks predicted to take longer tended not to be the ones that actually did. The problem was therefore not simply a consistent conversion error that could be fixed by multiplying every estimate by the same number.

The extreme errors did not extend unchanged to ordinary chat. On 500 sampled user requests, repeated five times per model, newer models underestimated response latency much less severely. GPT-5.6 Sol placed 73.4% of estimates within a factor of two of measured durations.

Wording changes duration judgments

A separate experiment compared model judgments with human labels for 5,000 events described in web text. The three main models correlated more closely with average human judgments than a single held-out annotator did. That means agreement with a collective judgment, not superiority at measuring actual elapsed time.

Their sensitivity to sentence features nevertheless differed. Across ten features, the models’ responses were about 1.27–1.35 times as strong as people’s. An exploratory breakdown suggested greater sensitivity to the grammatical form of the event expression and less to surrounding context. Tests using paired descriptions of the same event also found wording-dependent differences, including reversals of a human pattern for some event types.

Would you wait twenty minutes?

Finally, models chose between answering directly and using a calculator or search tool. The stated tool runtime ranged from one second to twenty minutes—a 1,200-fold increase.

GPT-5.6 Sol’s tool-use rates barely changed with that increase when only tool runtime was supplied. Other models showed varying sensitivity. But when prompts supplied measured total completion times for both options, delay mattered more in several conditions. For GPT’s search decisions with reasoning, tool use at twenty minutes fell from about 96% to 49%.

This suggests that explicit runtime information can help models incorporate time costs. It does not establish an optimal balance between speed and accuracy: users’ preferences were not specified, and the second condition measured choices rather than final-answer accuracy.

The study also has limited task coverage, incomplete access to original agent environments and a text-only comparison with human duration judgments. Its experiments do not establish that lacking direct experience of time causes the failures. The narrower, practical finding is that explaining timing principles and predicting operational runtimes are distinct abilities—and measured feedback can improve how models use the latter.