
Your AI Coding Agents Are Faster. So Why Isn’t Engineering Getting Cheaper?
What factory electrification, the productivity paradox, and the theory of constraints reveal about measuring AI ROI in engineering.
Your engineers are using AI coding assistants. Your teams are generating code faster, producing tests in less time, and experimenting with autonomous agents that can implement entire features.
Your developers are impressed. Your engineering managers are reporting productivity improvements. Your leadership team is asking what this means for delivery.
And eventually, someone in the executive meeting asks the question that matters most:
“If AI makes our engineers so much more productive, why isn't engineering getting proportionally cheaper?”
It is a fair question.
If a task that took eight hours now takes two, shouldn't engineering costs fall by 75%? If an agent can implement a feature overnight, shouldn't the delivery timeline shrink dramatically? If developers can produce twice as much code, shouldn't the organization deliver twice as much value?
Not necessarily.
The mistake is assuming that the time saved writing code translates directly into business value realized by the organization.
It doesn't. Not automatically.
To understand why, we need to look beyond AI—and back more than a century, to a technology that transformed almost every industrial economy: electricity.
1. The factory that bought electricity but didn't become more productive
Imagine walking into a manufacturing plant in the late nineteenth century.
The factory runs on a large central steam engine. Power travels through shafts, belts, and pulleys to drive machines throughout the building. The entire facility is designed around the physical constraints of that central power source.
Now electricity arrives.
The factory owner replaces the steam engine with an electric motor.
The electricity bill changes. The machinery is modernized. The factory has adopted the new technology.
But the machines remain arranged around the same central shaft. The workflow hasn't fundamentally changed. The building still follows the logic of the old system.
The factory has electrified its equipment without redesigning its production process.
Its potential has increased. Its realized productivity may not have changed much.
The real gains came when factories began redesigning their layouts around distributed electric motors. Individual machines could operate independently. Production lines could be reorganized. Workflows could become more flexible. Buildings no longer had to revolve around one central source of mechanical power.
Electricity didn't merely replace steam. It enabled an entirely different way to organize production.
In his 1990 paper, The Dynamo and the Computer: An Historical Perspective on the Modern Productivity Paradox, economist Paul David explored this historical parallel to explain why the productivity benefits of major technologies can take considerable time to appear in economic statistics.
The lesson: Adopting AI is not the same as redesigning engineering around AI.
Adopting AI is not the same as redesigning engineering around AI.
If your developers use AI to generate code but your organization still reviews, tests, approves, and deploys that code through processes designed for a pre-AI world, you may be modernizing the machinery while preserving the constraints of the old factory.
The result is more output at one stage of the process without a corresponding increase in end-to-end throughput.
And that is where many AI productivity initiatives begin to disappoint.
2. The productivity paradox is showing up in software engineering
The productivity paradox is not a claim that technology doesn't work. It is a reminder that technology adoption and measurable economic gains are different things.
In a 1987 interview, economist Robert Solow famously observed that the computer age was visible everywhere except in the productivity statistics.
The observation became shorthand for a recurring problem: organizations invest in transformative technologies, but the aggregate gains take time to materialize—or are difficult to measure with the metrics being used.
There are several reasons for this.
First, organizations need time to learn how to use the technology effectively.
Second, the technology often requires complementary investments in processes, skills, infrastructure, and organizational design.
Third, the benefits may initially appear as better quality, faster experimentation, or improved responsiveness rather than immediate cost reductions.
And finally, the metrics used to measure productivity may fail to capture the value being created.
AI-assisted software engineering has all four characteristics.
Consider a developer implementing a new data pipeline.
Before AI, the developer might spend eight hours understanding the requirements, writing transformations, creating tests, debugging failures, and preparing the pull request.
With AI assistance, the developer may generate the initial implementation in two hours.
That is a meaningful improvement.
But what happens next?
The generated code must still be checked against the source schema. The transformations must preserve business logic. Data quality expectations must be verified. Security and access controls must remain intact. Tests must demonstrate the required behavior. Reviewers must understand the implementation. The deployment must succeed. The pipeline must continue operating correctly in production.
If those activities take another six hours, the overall task still consumes eight hours.
The time hasn't necessarily disappeared. It has moved.
And if AI introduces unfamiliar patterns, incorrect assumptions, or subtle edge cases, verification and rework can consume even more time.
This is the critical distinction:
- Local productivity: How quickly an individual activity is completed.
- System productivity: How efficiently the complete delivery process produces a verified, usable outcome. AI can dramatically improve the first without immediately improving the second.
That doesn't make AI less valuable. It means we need to measure the right thing.
3. Your bottleneck has probably moved
The Theory of Constraints, popularized by Eliyahu Goldratt, offers a useful way to understand what happens when one part of a system becomes significantly faster.
Every delivery system has constraints. Improving an activity that is not currently limiting overall throughput may create local efficiency without materially improving the system.
Software engineering is no exception.
A simplified delivery workflow looks like this:
- Requirements and design — Understand what needs to be built.
- Implementation — Write, generate, and modify code.
- Verification and review — Test correctness, quality, security, and business rules. Potential bottleneck.
- Release and deployment — Approve, promote, and deploy safely.
- Verified production outcome — Working software that delivers business value.
Before AI, implementation might have been the slowest step. Developers had limited capacity to write, debug, and test code.
AI changes that balance.
Implementation accelerates. More changes arrive at review. More code must be validated. Test suites may need to cover more generated behavior. Release managers may face a larger queue of changes waiting for approval.
The bottleneck moves downstream.
Imagine an engineering team that can previously complete ten changes per week because developers can only implement ten. AI doubles implementation capacity to twenty changes, but the review and verification process can validate only twelve.
The team now has the potential to implement twenty changes but can deliver only twelve.
Implementation has become faster. System throughput has increased by only 20%.
The remaining capacity sits in a queue.
This is a simplified example, not a universal prediction. The actual bottleneck depends on the organization, the work, the maturity of its testing practices, and the quality of the generated code.
But the principle holds: the value of accelerating a stage is constrained by the capacity of the rest of the system.
The right question is no longer simply, “How much faster can our developers write code?”
It becomes:
“How quickly can we turn an idea into a verified, production-ready outcome without increasing risk or rework?”
That is a fundamentally different engineering management problem.
4. Faster code generation can create a verification debt
Technical debt is familiar to most engineering leaders. Teams make shortcuts today and pay for them later through maintenance, complexity, and slower change.
AI introduces another potential liability: verification debt.
Verification debt accumulates when the amount of generated or modified code grows faster than the organization's ability to establish that the code is correct.
It can appear in several forms:
- Pull requests that are larger than reviewers can reasonably understand.
- Tests that confirm implementation details without validating business behavior.
- Generated code that looks plausible but misinterprets requirements.
- Dependencies, permissions, or configuration changes that receive insufficient scrutiny.
- Data transformations that pass basic tests but violate downstream assumptions.
- Documentation that confidently describes behavior the system does not actually implement. This is especially consequential in data engineering.
A pipeline can complete successfully while producing incorrect data. A SQL transformation can run without errors and still double-count records. A schema change can be technically valid while breaking a downstream consumer. A generated mapping can look reasonable while misrepresenting a healthcare business definition.
In such systems, execution success is not the same as correctness.
And when the data supports healthcare analytics, risk adjustment, quality measurement, payment integrity, or regulatory reporting, the consequences can extend well beyond engineering rework.
Verification is therefore not administrative overhead that AI should eliminate. It is a core part of producing a trustworthy outcome.
The goal is to make verification faster, more systematic, and more reliable—not to remove it.
That distinction becomes even more important as organizations move from AI-assisted coding toward agents capable of planning and executing multi-step development tasks.
An agent that produces more code does not automatically produce more verified value. It may simply increase the amount of work waiting for human or automated assurance.
5. The J-curve: why AI investments can initially look disappointing
A second economic concept helps explain why organizations can invest heavily in AI and still struggle to demonstrate immediate returns.
In their 2021 paper, The Turing Trap: The Promise & Peril of Human-Like Artificial Intelligence, Erik Brynjolfsson and co-authors discuss the broader implications of AI for productivity and the importance of how its benefits are realized. Related research on the productivity J-curve, including work by Brynjolfsson, Rock, and Syverson, describes how general-purpose technologies can create a temporary gap between investment and measured productivity.
The basic idea is straightforward.
When an organization adopts a transformative technology, it often needs to invest in complementary assets before the gains become visible.
Those assets can include:
- New workflows and operating models.
- Better data and internal documentation.
- Automated testing and validation.
- Developer training and changed engineering practices.
- Security controls and governance.
- Tool integration and infrastructure.
- New ways of coordinating teams and measuring outcomes. These investments consume time and money.
They may initially make the organization appear less productive because people are learning, processes are changing, and old and new approaches coexist.
Only after the complementary capabilities mature do the benefits become fully visible.
For AI-enabled engineering, this means the first phase may look like this:
-
The company purchases AI tools.
-
Developers experiment and report time savings.
-
Engineering leaders introduce new workflows and guardrails.
-
Teams spend time improving tests, documentation, and automated checks.
-
The organization begins delivering more reliable outcomes with less manual effort.
The early costs are visible immediately. The full benefits may take longer.
This does not mean every AI investment will eventually pay off. A J-curve is not a promise of future success. Poor use cases, weak adoption, excessive licensing costs, and ineffective implementation can prevent returns from materializing.
It means leaders should distinguish between an investment that is still building necessary capabilities and an investment that has failed to produce measurable value.
The way to make that distinction is to define the expected outcome, baseline the existing process, and track progress through the transition.
Without those disciplines, "the benefits will come later" becomes an excuse that can justify almost any level of spending.
6. AI productivity is uneven—and that matters
There is another reason not to assume that AI produces uniform productivity gains across engineering.
Research on the "jagged technological frontier," including the 2023 working paper Navigating the Jagged Technological Frontier by Fabrizio Dell'Acqua and colleagues, examined how generative AI affected knowledge workers across different types of tasks.
The central insight is that AI capability is uneven. A tool can perform impressively on one task and struggle with another that appears very similar to a human observer.
Software engineering exhibits the same pattern.
AI may perform well when asked to:
- Generate boilerplate code.
- Explain an unfamiliar function.
- Produce an initial unit-test suite.
- Refactor a well-defined module.
- Translate a clear requirement into a straightforward implementation. It may be less reliable when asked to:
- Infer undocumented business rules.
- Diagnose an intermittent production failure spanning several systems.
- Make a schema change with many hidden downstream dependencies.
- Preserve subtle healthcare data semantics.
- Design a secure architecture under complex constraints.
- Validate a result when no reliable test oracle exists. These are illustrative examples, not a universal ranking of AI capabilities. The frontier changes as models, tools, context, and workflows improve.
The management implication is still important: AI adoption should be evaluated at the task and workflow level, not by a single company-wide productivity multiplier.
A team implementing well-specified API endpoints may experience different gains from a team reverse-engineering a poorly documented legacy data pipeline.
A team with strong automated tests may safely delegate more implementation work than a team that relies on manual review and production monitoring to discover errors.
The same AI tool can create very different economic outcomes in these environments.
This is why "AI makes developers 30% more productive" is an incomplete business case unless the organization can explain which work improved, how the gain was measured, what happened to quality, and whether the gain translated into additional delivered value.
7. The measurement trap: activity is not economic value
One of the easiest mistakes in AI adoption is measuring the outputs that are easiest to count.
Lines of code generated. Number of prompts. Number of AI-assisted pull requests. Percentage of developers using AI. Number of tests generated.
These metrics can help answer adoption or tooling questions. They cannot, on their own, establish that the organization is receiving a financial return.
More code is not inherently better. More pull requests are not inherently valuable. A high adoption rate does not prove that a workflow is more efficient.
Consider two teams.
Team A uses AI to produce twice as much code. Its pull requests become larger, review time increases, and more defects are discovered late in the delivery cycle.
Team B uses AI to generate smaller, better-tested changes. Its implementation time falls, review becomes more predictable, and verified releases reach production sooner.
Team A may look more productive on an activity dashboard. Team B may be creating substantially more business value.
The difference is what happens between code generation and a successful outcome.
A useful engineering measurement model must therefore connect three layers:
| Layer | What to measure | What it tells you |
|---|---|---|
| Adoption | Active usage, eligible tasks assisted | Whether AI is being used |
| Workflow | Implementation time, review time, rework, cycle time | How the delivery process is changing |
| Business outcome | Cost per verified delivery, time to value, realized capacity, quality-adjusted delivery | Whether the change creates economic value |
The third layer is the one executive leaders ultimately need.
And it requires more than a dashboard.
It requires a clear definition of the outcome, a credible baseline, quality safeguards, and an explanation of how operational gains translate into business results.
8. What should engineering leaders do differently?
The answer is not to slow down AI adoption or to demand immediate cost reductions from every team.
It is to stop treating AI as an isolated coding tool and start treating it as a change to the entire delivery system.
That begins with five decisions.
First, baseline the current workflow. Measure how long representative work takes from an agreed starting point to verified production delivery. Include implementation, review, testing, rework, and release.
Second, identify the actual constraint. Determine where work waits, where errors are discovered, and which activity limits throughput. Do not assume implementation is still the bottleneck.
Third, invest in verification. Strengthen automated tests, contracts, static analysis, security checks, data quality assertions, and release gates. Match the assurance process to the risk of the change.
Fourth, evaluate AI by task category. Start with work that is well understood, measurable, and amenable to automated validation. Expand as evidence supports it.
Fifth, connect capacity gains to a business outcome. If implementation becomes faster, determine whether the organization can deliver more, shorten lead times, reduce external spending, improve quality, or avoid future hiring. Be explicit about which benefit is actually being realized.
These decisions will not make every AI initiative successful. They will make success measurable—and failure easier to recognize before the investment grows.
The real question isn't how fast AI writes code
The arrival of AI coding agents is a meaningful shift in software engineering. It can change the cost of implementation, expand what small teams can accomplish, and allow engineers to spend less time on repetitive work.
But the economic result depends on the system in which those capabilities operate.
History offers a useful warning. Technologies that transform individual activities do not automatically transform the organizations that use them. The largest gains often come when workflows, incentives, infrastructure, and operating models change alongside the technology.
The same principle applies to AI engineering.
If code generation accelerates while review, testing, deployment, and operational assurance remain unchanged, the organization may see more activity without a proportional increase in value.
If verification becomes a bottleneck, generating even more code may make the problem worse.
And if leaders measure adoption instead of outcomes, they may declare victory long before the business case has been proven.
The objective should not be to generate the maximum amount of code at the minimum cost.
It should be to produce the greatest amount of verified, valuable software with the least sustainable effort and acceptable risk.
That is the difference between adopting AI and building an AI-enabled engineering organization.
In Part 2, we'll move from the economic argument to the operating model: how to measure cost per verified production delivery, reduce verification bottlenecks, redesign release controls for AI-generated code, and connect engineering capacity gains to actual business value.
Because faster code is a capability.
Verified delivery is the outcome.
References and further reading
-
David, P. A. (1990). The Dynamo and the Computer: An Historical Perspective on the Modern Productivity Paradox. The American Economic Review.
-
Solow, R. M. (1987). We'd Better Watch Out. The New York Times Book Review.
-
Brynjolfsson, E., Rock, D., & Syverson, C. (2021). The Productivity J-Curve: How Intangibles Complement General Purpose Technologies. American Economic Journal: Macroeconomics.
-
Dell'Acqua, F., et al. (2023). Navigating the Jagged Technological Frontier. Harvard Business School working paper.
-
Goldratt, E. M. (1984). The Goal: A Process of Ongoing Improvement.
Reactions
Discussion
No comments yet
Sign in as a member to join the discussion.
Keep reading
All stories
The Real AI ROI Playbook — How to Measure Engineering Productivity in the Age of AI
What factory electrification, the productivity paradox, and the theory of constraints reveal about measuring AI ROI in engineering—and how to build an operating model that delivers real economic value.

We Didn’t Eliminate Data Engineering Work. We Just Turned It Into Paranoid Auditing.
On paper, our data engineering velocity is up 400%. We’ve compressed three-week ETL pipeline builds into a couple of days using autonomous agents. In reality, the time we spend untangling silent data bugs has quietly doubled. We spent the last three years building the most sophisticated data agents

How Databricks Turns a Messy Bucket of Parquet Files Into a Reliable Database
Data lakes used to be chaotic swamps where crashed pipelines left corrupted files and concurrent jobs silently overwrote each other. Discover how Delta Lake changes the game by using a single, hidden folder to bring ACID transactions, time travel, and absolute reliability to your cloud storage.