NEW RESEARCH REPORT: Overcoming the limitations of coding agents in complex software systems
Research report
Overcoming the limitations of coding agents in complex software systems
Strategies for engineering leaders in financial services, silicon design, networking, and other mission-critical industries
Introduction
THE MODERN ERA
OF SOFTWARE ENGINEERING
The volume of code created is growing at an unprecedented rate. But we all know there’s a catch. The dramatic reduction of the cost of creating code has simply moved the bottlenecks downstream, where engineers struggle with code comprehension, debugging, and incident investigations.
If the promised productivity gains are to be realized, AI must be applied to the software delivery cycle, not just code generation. But AI has so far proven itself unequal to effective comprehension and debugging of complex, large-scale codebases.
Part 1
Where time is spent now
Writing code has become cheaper, but in mission-critical codebases, that code must still be understood. The more demanding task is understanding what that code does, how it affects existing codebases, and debugging it when an application doesn’t behave the way it’s expected to.
42
%
of engineering teams’ time is spent on debugging.
Based on the assumption of an average 40-hour working week.
How engineering teams’ time is divided between tasks
9.8
hrs/week
Producing code
9.7
hrs/week
Debugging issues during development
7.2
hrs/week
Debugging issues found by customers in production
13.3
hrs/week
All other tasks
Nine in ten (91%) engineering leaders say code comprehension and debugging take longer in complex C/C++ codebases. As AI-generated code increases, the challenge will grow.
Part 2
The impact of coding agents on release cycles
Coding agents promise faster software delivery, but unless AI is applied across the entire software development lifecycle then the bottleneck simply shifts downstream.
Now that AI is generating most of the code being produced, engineers no longer have the inherent understanding they used to. That makes it easier for defects to escape, and when something inevitably goes wrong, nobody has the knowledge to trace the failure back to its root cause.
35%
of AI-generated code has not been fully comprehended before engineering teams push it to production.
Challenges engineering teams face in accelerating software delivery with AI agents
Agents introduce incorrect code too frequently, creating rework that delays delivery cycles
55%
Engineers are required to spend excessive time reviewing agent outputs compared to how long it takes to generate the code
50%
Agents cannot reliably infer what code will do at runtime, and often hallucinate when debugging more complex codebases
48%
Engineers find it difficult to review the large amounts of code generated by agents, and merge too many AI-generated errors into the codebase
46%
It takes too many prompts and re-runs to identify the root cause of test failures with AI agents
38%
Agents lack the ability to diagnose issues that are hard to reproduce such as intermittent failures
28%
Engineering leaders are in an impossible bind: they’re expected to deliver significant productivity gains from AI without compromising quality or accountability.
But as AI-generated code proliferates, review fatigue grows. The risk is that this will lead to more code pushed to production before engineers have comprehended it, more escaped defects, more unresolved tickets, and further gaps in accountability.
Issues organizations have encountered as a result of their use of AI coding tools
Loss of productivity due to engineers needing to ‘unpick’ AI-generated code
29%
94%
Budget overruns due to agents needing to re-run tasks multiple times to reach the correct output
19%
93%
Incorrect diagnosis of the root cause of an issue due to a hallucination
18%
93%
Test escapes, serious defects, or poorly optimized code entering production
8%
91%
Production incident / service outage affecting internal users or customers
14%
81%
Multiple times per month
At least once in the past six months
79
%
of engineering leaders say despite being able to produce code much faster, release cycles are no faster than before.
The problems created by coding agents are most acute in the complex environments that power mission-critical industries. Trading platforms, network operating systems, and applications of a similar nature run on legacy codebases with poorly documented interdependencies, while critical institutional knowledge lives only in engineers’ heads.
AI inevitably makes subtle mistakes. But so much code is being generated that it is difficult for engineers to review and understand it properly.
It’s no wonder that so few enterprises have seen a meaningful improvement in release cycles.
Ultimately, the productivity gains promised by AI will remain limited unless intelligent engineers can also apply it effectively to comprehension and debugging.
If they fail to do so, the time engineers save in code generation is simply absorbed by slower investigation and debugging, or lost later through production incidents and security failures.
Part 3
Calculating the true value of AI
Measuring lines of code generated only tells the beginning of the story about the impact of AI on engineering productivity. To get the real story of productivity gains from coding agents, it’s important to look at metrics like defect rate and mean time to resolution.
AI model families engineering teams use to understand and debug complex codebases
Different models have their individual strengths and weaknesses, and there are trade-offs between capability and economy for any given task. Engineering teams use a whole host of models for debugging and comprehension, but GPT and Gemini ride high on their list.
But few engineering teams are using the most capable models for code comprehension and debugging, even in more complex environments.
Nearly two-thirds (63%) of engineering leaders say the cost is prohibitive to authorize the use of higher tier AI models for this work.
Rising costs are a sizable barrier to increasing the use of more powerful AI
63%
Of leaders say the cost is prohibitive to authorize the use of higher tier AI models for debugging complex codebases.
22%
Only one in five would use Mythos for these tasks if it was available to them.
57%
More than half of engineering leaders opt for lower cost AI models for code comprehension and debugging.
There’s also a degree of cynicism about the impact the planned IPOs of Anthropic and OpenAI will have on the trajectory of token costs in future. Four in five (82%) engineering leaders are concerned that the costs of coding agents will go through the roof as the labs prioritize making Wall Street happy.
To scale AI without breaking the budget, engineering leaders need to find a way to close the gap between lower cost models and those at the frontier.
82
%
of respondents think most enterprises will be priced out of Mythos or Fable-level AI models.
Part 4
Problem-solving in complex codebases
More than a third (37%) of teams use AI agents for comprehension and debugging only for straightforward codebases.
When they do use AI for code comprehension and debugging in more complex codebases, engineering teams rely on a range of workarounds to give a greater degree of confidence.
80%
of engineering leaders say coding agents struggle to solve difficult problems in large-scale, complex codebases.
Techniques organizations use to mitigate the challenges and risks of using AI agents for code comprehension and debugging
66
%
Using a human in the loop to approve any AI agent action
55
%
Enriching source code analysis and logs with specific context about what happened at runtime
48
%
Developing hypotheses and then testing them on agents rather than asking the agent to identify the cause / solution
40
%
Using agents to check the work of other agents in autonomous workflows
82
%
say AI agents would be far more useful for code comprehension and debugging if they were grounded in the context of what happened at runtime.
It’s critical to give agents all the evidence and data they need to reach accurate conclusions independently.
The most effective way to provide that context is to give the agent a recording of program runtime behavior, so it can see everything code did on execution and identify the precise moment something went wrong.
Runtime context improves both accuracy and cost. With a complete execution recording, lower-cost models can outperform frontier models working from source code alone – at a fraction of the cost. [Ref. conference poster: comparison data on how runtime context improves the economics of AI]
How Undo can help
Undo enables coding agents to solve the most complex problems on the most complex codebases.
Fully automated root-cause analysis
Undo creates deterministic, self-contained, portable recordings of complete program executions, giving agents the runtime context to reason about dynamic behavior like static code.
With Undo, developers have been able to solve problems up to 100x faster in some of the world’s most demanding software environments including at AMD, AWS, Cisco, and Palo Alto Networks.
Find out how Undo could help your engineering team unlock the potential of AI agents.
METHODOLOGY
This report is based on a survey of 300 senior engineering leaders in large organizations ($250m+ revenue) that deliver mission-critical software built on large, complex codebases, including 93% working with C/C++. The fieldwork and analysis was conducted on behalf of Undo by independent research firm Coleman Parkes during July and August 2026.
Request
Industry-Specific Edition
Access the industry-specific edition for peer benchmarking