Deadlines as a First-Class Input
Last Updated on September 22, 2026 by Editorial Team Author(s): Shrashti Singhal Originally published on Towards AI. Propagating a latency budget into the reasoning, so the agent trades depth for time on purpose instead of timing out mid-thought. Here is a race condition currently running in production at more companies than would enjoy hearing it, described in a post-mortem pattern that’s been written up publicly more than once. An agent calls a tool with an eight-second budget. The tool, independently, has an eight-second timeout of its own. Sounds compatible. It isn’t — because the two clocks start at different moments. The agent’s budget starts ticking when the model begins emitting the tool call; the tool’s timer starts when the request actually reaches it. Between those two instants sit a second of prefill, a few hundred milliseconds of routing, some queueing. So the agent has quietly spent 1.4 seconds of its eight before the tool believes the race has begun. At 7.9 seconds on the agent’s clock the agent declares the call dead and replans; at 8.0 on the tool’s clock, the tool finishes successfully and delivers a correct answer to a corpse. The agent, meanwhile, has retried —…
