AI Coding Hits the Context Window Wall
Techstrong TV Interviews
•
16m
More Code, Less Understanding
The context window has become a hidden limit on AI coding. Teams are adopting coding tools quickly. Yet many skip the step of helping AI understand the system it is changing.
Blitzy Field CTO Neeraj Deshmukh examines that gap with Mike Vizard. He says expectations need to change first. AI can be very smart, but it can also be very literal. Without precise instructions, it resolves ambiguity on its own. Small detours build on each other until the output is useless.
Why Bigger Prompts Fall Short
AI can write strong code. The harder question is whether it sees downstream effects. A change in one file can affect modules thousands of lines away.
Loading the whole code base into a model sounds like the answer. In practice, the context window gets in the way. Frontier models advertise about a million tokens. Much of that goes to instructions, history and bookkeeping. Deshmukh says the effective context is closer to 200,000 to 300,000 tokens. That covers roughly 20,000 to 30,000 lines of code.
Beyond that, the model compresses information and results degrade.
A Coordinated Swarm of Agents
Blitzy takes a different approach. It reverse engineers the code base into a knowledge layer, even for hundreds of millions of lines. A swarm of agents then works from it. Each agent gets only the context it needs, so no single context window fills up.
Before writing code, the agents build an action plan for human review. Planning agents decide which tasks run in parallel and which must run in order. As they work, the agents validate their code against the success criteria the human defines. They keep fixing issues until every test passes.
Deshmukh warns that shared context across agents hits limits even faster. He also notes that token-based pricing gives many AI vendors little reason to change. He adds that the SDLC itself must change. AI speeds up coding, and that exposes bottlenecks in code review and QA. That, he says, is a process problem, not a technology one.
Up Next in Techstrong TV Interviews
-
AWS Is Building Its Databases for AI ...
AI agents now use more enterprise data than ever. Alan Shimel talks with G2 Krishnamoorthy, VP of AWS Database Services, about how AWS databases are changing for this agentic world. They cover the new Duck Labs integration in Aurora PostgreSQL, MCP servers and serverless scale.
The breadth of ...
-
The DevOps Standard Finally Defines D...
DevOps has gone more than a decade without a formal definition. Alan Shimel talks with Marc Hornbeek, CEO and founder of Engineering DevOps Consulting, and Markus Bause, VP of product at PeopleCert, about The DevOps Standard, the new DevOps Institute book that finally defines DevOps and lays out ...
-
Open Mainframe Project Opens Mainfram...
A Neutral Home for Mainframe Open Source
John Mertic is director of program management at the Linux Foundation. He has spent 11 years with the Open Mainframe Project. He and Alan Shimel once co-hosted the Open Mainframe podcast together.The project began around Linux on IBM Z after IBM's LinuxO...