AI inference is getting cheaper, but your agents are getting more expensive

The good news is that large language model (LLM) token costs are coming down. The conundrum: The overall cost of AI workloads is going up. Gartner research predicts that, while token costs will fall by 95% by 2030, inference costs for agentic workflows will increase more than fivefold over the next two years. This is because AI app builders are using more, and often more expensive, tokens as LLMs get ever more complex. It is what Gartner calls the “inference paradox.” In other words, “the rate of innovation is outpacing the cost curve,” Gartner analysts Will Sommer and Sabine Zimmerhansl noted in their report. “The market is captured by a token-deflation illusion.” Buyers dangerously assume that as AI providers improve token economics, these savings will be reflected in their roadmaps. But, simply put, “they will not.” ‘Swarms’ of agents eating up tokens There is no doubt that AI delivers massive value, and is often better, and faster, than people at many routine tasks, the analysts pointed out. Agents can also more quickly identify patterns across siloed systems. For instance, Gartner has seen customer success agents reduce response times by 99%. But as AI evolves, token usage in…