Product leaders in the tech industry are confronting a major financial hurdle as artificial intelligence transitions from simple assistant tools into complex autonomous systems. While the cost of individual AI tokens continues to drop, the overall expense of running sophisticated automated tasks is skyrocketing. According to new research released by Gartner, AI inference costs per agentic workflow will increase more than fivefold through 2028.

This phenomenon is creating what industry experts call the Inference Paradox. As foundational models become cheaper and more efficient, companies are using those savings to deploy vastly more powerful and demanding AI applications. Rather than reducing operational budgets, these technical advances are actually driving up total spending because modern agentic workflows require significantly more computational power.

The fundamental shift in how AI operates lies at the heart of this spending spike. Traditional chatbots only need to read a prompt and generate a basic response. In contrast, modern AI agents must continuously reason, adapt, and process multiple steps to complete a given task. “The harsh economics of the Inference Paradox are exemplified by the differences between a simple chatbot and an AI agent,” said Sommer. “Where a simple chatbot must read and interpret a query and quickly respond with a probabilistically reasonable answer, an AI agent must constantly reason, negotiate, and question itself.”

These increased responsibilities mean that routing tasks to agentic reasoning models raises provider inference costs by at least five times compared to basic chatbot interactions. As task complexity grows, these expenses can multiply even further. The rapid pace of innovation is simply moving faster than the falling cost of underlying technology, which leaves product teams struggling to maintain healthy profit margins.

To prevent budget overruns, tech companies must rethink how they build and deploy their software ecosystems. Relying on a single, super-smart AI model for every task is no longer financially sustainable. “Product leaders cannot rely on more efficient token economics to rationalize AI costs,” said Will Sommer, Sr. Director Analyst. “Each successive generation of AI capability will necessitate more, and often more expensive, tokens. There is no reliable, economical one-size-fits-all model on the horizon. Producing competitive AI products will require developing and maintaining complex multimodel ecosystems.”

Achieving a strong return on investment will require companies to implement clever routing techniques and model tiering. By carefully directing simpler tasks to cheaper models and reserving heavy reasoning engines for complex problems, businesses can control their total spending. “Defaulting to generic autonomous intelligence will result in unbounded costs orders of magnitude higher than those of optimized product ecosystems,” said Sommer. Without careful orchestration, the financial burden of advanced AI could easily overwhelm its business

Gartner clients can read more in The Inference Paradox: Inference Tiering Is Critical to Protect Margins.