Agent-Optimized Inference Cuts AI Token Costs
Techstrong TV Interviews
•
15m
AI token costs keep climbing even as the price of a single token falls. Mike Vizard talks with Jack O'Brien, co-founder and CEO of Subconscious, an MIT-born startup that raised $5.1 million. Together, they explore why long-running AI agents drive so much spending and how smarter inference can bring it down.
Why agents drive AI token costs
Token consumption is doubling roughly every 11 weeks, Jack says. Furthermore, most of those tokens now go to agentic systems rather than chatbots. A chatbot session might run 50 or 100 messages. An agent, however, often runs hundreds of messages and hundreds of thousands of tokens on one task.
Routing is a red herring
Routing requests to cheaper models works for short tasks. For long-running agents, though, switching models mid-task breaks the reasoning chain and invalidates cached tokens. As a result, the system slows down and AI token costs go up. Instead, Subconscious built an inference engine around runtime memory compression and efficient caching.
More users per GPU
GPU scarcity is only part of the problem; utilization is the other part. According to Jack, the compression approach supports two to three times as many concurrent users on the same hardware. In addition, the engine runs on GPUs, CPUs and NPUs, and customers can use an API or deploy on their own GPUs.
Lowering AI token costs with good enough models
Many companies still run models that are a generation or two old. Consequently, moving to newer open models and serving them efficiently can cut AI token costs while improving accuracy. Looking ahead, Jack expects agents to run much of the work behind every company, and he sees standards such as MCP sticking around.
Smarter inference is becoming as important as smarter models. Explore more AI coverage and the latest Techstrong TV interviews.
For more information please visit subconscious.dev
Up Next in Techstrong TV Interviews
-
AI Scanners Demand Smarter Vulnerabil...
Vulnerability prioritization is fast becoming the most important skill in cybersecurity. Mike Vizard talks with David Lindner, chief information security and data privacy officer at Contrast Security. Together, they explore why AI scanners are flooding security teams with findings and what CISOs ...
-
It's Not the Technology That Pwns You...
Online scam protection starts with understanding the trick, not the technology. Alan Shimel welcomes Frank Riccardi, a cybersecurity and privacy expert. His new book is CTRL+ALT+PWN: The Hacker's Playbook (And How to Beat It). Furthermore, Frank wrote the book for everyday people who do not know ...
-
Liquibase Secure 6.0 Puts Guardrails ...
Database change governance is becoming critical as AI agents start touching production. Alan Shimel welcomes back Ryan McCurdy, VP of Marketing at Liquibase. Furthermore, Ryan walks through the new Liquibase Secure 6.0 release and what it means for teams adopting AI.
What is new in Liquibase S...