Agent-Optimized Inference Cuts AI Token Costs
15m
AI token costs keep climbing even as the price of a single token falls. Mike Vizard talks with Jack O'Brien, co-founder and CEO of Subconscious, an MIT-born startup that raised $5.1 million. Together, they explore why long-running AI agents drive so much spending and how smarter inference can bring it down.
Why agents drive AI token costs
Token consumption is doubling roughly every 11 weeks, Jack says. Furthermore, most of those tokens now go to agentic systems rather than chatbots. A chatbot session might run 50 or 100 messages. An agent, however, often runs hundreds of messages and hundreds of thousands of tokens on one task.
Routing is a red herring
Routing requests to cheaper models works for short tasks. For long-running agents, though, switching models mid-task breaks the reasoning chain and invalidates cached tokens. As a result, the system slows down and AI token costs go up. Instead, Subconscious built an inference engine around runtime memory compression and efficient caching.
More users per GPU
GPU scarcity is only part of the problem; utilization is the other part. According to Jack, the compression approach supports two to three times as many concurrent users on the same hardware. In addition, the engine runs on GPUs, CPUs and NPUs, and customers can use an API or deploy on their own GPUs.
Lowering AI token costs with good enough models
Many companies still run models that are a generation or two old. Consequently, moving to newer open models and serving them efficiently can cut AI token costs while improving accuracy. Looking ahead, Jack expects agents to run much of the work behind every company, and he sees standards such as MCP sticking around.
Smarter inference is becoming as important as smarter models. Explore more AI coverage and the latest Techstrong TV interviews.
For more information please visit subconscious.dev